#tokens

The maximum amount of text (measured in tokens) that a language model can process and "remember" in a single interaction. Information outside the context window is inaccessible to the model, making context management critical for long-form tasks.

“The model kept losing track of earlier instructions because the codebase exceeded its context window.”

The process of converting raw text into discrete units called tokens that a language model can process. Tokens are typically subword units — common words become single tokens while rare words split into multiple tokens. All LLM pricing and context limits are measured in tokens, not characters or words.

“The word "unbelievable" tokenized into three pieces: "un", "believ", "able".”