#latent-space

Latent operations are edits, traversals, or measurements performed on vectors inside a model latent space. Common operations include shifting a vector along a learned direction, interpolating between two vectors, subtracting one concept vector from another, masking dimensions, or sampling nearby points. These operations are used to explore how a model organizes concepts and to guide generation without rewriting the whole model.

Latent reasoning is the idea that a language model can carry part of its reasoning process inside continuous hidden-state vectors instead of only through explicit words. In an LLM, the prompt is projected into high-dimensional representations, transformed through model layers, and decoded into text. Research on latent reasoning treats a model final hidden state as a reusable representation of an intermediate thought, allowing reasoning to continue directly in latent space.

Latent space surgery is a model-editing technique that identifies directions in a latent space corresponding to concepts, styles, or behaviors, then adds, subtracts, or dampens those directions to change outputs or internal representations. Instead of retraining an entire model, practitioners can use targeted vector edits to nudge behavior, such as moving a music embedding from a classical direction toward a jazz direction.