Latent space

A latent space, also called an embedding space, is an abstract high-dimensional representation where a model maps similar items to nearby vectors. It is usually learned automatically from data: latent variables capture hidden features and arrange items across a manifold that is often smaller and easier to compute over than the raw feature space. In LLMs, tokens and hidden states live in latent space, where semantic relationships between words, phrases, prompts, and responses are encoded before being decoded back into language.

“A model can place 'cat' and 'dog' near each other in latent space because both share semantic features, while placing 'invoice' farther away.”