#latent-space

Latent operations are edits, traversals, or measurements performed on vectors inside a model latent space. Common operations include shifting a vector along a learned direction, interpolating between two vectors, subtracting one concept vector from another, masking dimensions, or sampling nearby points. These operations are used to explore how a model organizes concepts and to guide generation without rewriting the whole model.

“A designer might interpolate between two image embeddings to create a smooth transition from one visual concept to another.”

Latent reasoning is the idea that a language model can carry part of its reasoning process inside continuous hidden-state vectors instead of only through explicit words. In an LLM, the prompt is projected into high-dimensional representations, transformed through model layers, and decoded into text. Research on latent reasoning treats a model final hidden state as a reusable representation of an intermediate thought, allowing reasoning to continue directly in latent space.

“Instead of forcing every reasoning step into written text, an experiment may feed a hidden-state vector back into the model and let the next step happen in latent space.”

Latent space surgery is a model-editing technique that identifies directions in a latent space corresponding to concepts, styles, or behaviors, then adds, subtracts, or dampens those directions to change outputs or internal representations. Instead of retraining an entire model, practitioners can use targeted vector edits to nudge behavior, such as moving a music embedding from a classical direction toward a jazz direction.

“If a vector direction reliably represents sentiment, latent space surgery can increase or reduce that direction to alter the tone of generated text.”