AI Alignment
[/eɪ aɪ əˈlaɪnmənt/]
Definitions
The research field focused on ensuring that AI systems pursue goals that match human values and intentions. A misaligned AI might optimize for a metric that appears correct but produces harmful or unintended outcomes at scale.
“AI alignment researchers worry that optimizing for user engagement could misalign with genuine user wellbeing.”
Related Terms
- Constitutional AIAI & Technology
A training methodology developed by Anthropic where a set of guiding principles (a "constitution") is used to self-supervise and refine AI outputs. The model critiques and rewrites...
- GuardrailsAI & Technology
Safety constraints and filters applied to AI systems to prevent harmful, offensive, or out-of-scope outputs. Guardrails can be implemented at the model level (via training), prompt...
- JailbreakAI & Technology
A technique used to bypass the safety filters and content policies of an AI model, typically by framing harmful requests in ways the model's defenses don't recognize. Jailbreaks of...
- AgenticAI & Technology
Describing AI systems capable of autonomous action, planning, and decision-making. An agentic AI can break down tasks, use tools, and work toward goals with minimal human intervent...
- Chain of ThoughtAI & Technology
A prompting technique where a language model is encouraged or required to show its step-by-step reasoning before providing a final answer. Chain-of-thought prompting significantly ...
- Context WindowAI & Technology
The maximum amount of text (measured in tokens) that a language model can process and "remember" in a single interaction. Information outside the context window is inaccessible to ...