Transformer
[/trænsˈfɔːrmər/]
Definitions
A neural network architecture introduced in 2017 ("Attention Is All You Need") that underlies virtually all modern language models. Transformers use self-attention mechanisms to process entire sequences in parallel, capturing long-range dependencies that earlier recurrent architectures struggled with.
“Every major LLM from GPT to Claude is built on the transformer architecture.”
Related Terms
- Neural NetworkAI & Technology
A computational model loosely inspired by biological neurons, consisting of interconnected layers of mathematical functions (nodes) that transform input data into output prediction...
- Diffusion ModelAI & Technology
A class of generative AI model that learns to create images, audio, or video by reversing a noise-adding process. During training the model learns to denoise progressively; during ...
- RAGAI & Technology
Retrieval-Augmented Generation - An AI architecture pattern that combines a language model with external knowledge retrieval to provide more accurate and up-to-date responses.
- AgenticAI & Technology
Describing AI systems capable of autonomous action, planning, and decision-making. An agentic AI can break down tasks, use tools, and work toward goals with minimal human intervent...
- AI AlignmentAI & Technology
The research field focused on ensuring that AI systems pursue goals that match human values and intentions. A misaligned AI might optimize for a metric that appears correct but pro...
- Chain of ThoughtAI & Technology
A prompting technique where a language model is encouraged or required to show its step-by-step reasoning before providing a final answer. Chain-of-thought prompting significantly ...