Inference
[/ˈɪnfərəns/]
Definitions
The act of running a trained machine learning model on new input data to generate predictions or outputs. Inference is distinct from training — it is the "serving" phase where the model is used in production, and its speed and cost are critical for real-world applications.
“Inference latency dropped from 2 seconds to 200ms after switching to a quantized model.”
Related Terms
- EmbeddingsAI & Technology
Dense numerical vector representations of words, sentences, or other data that capture semantic meaning. Similar concepts have similar embeddings (nearby in vector space), allowing...
- Fine-TuningAI & Technology
The process of further training a pre-trained model on a smaller, task-specific dataset to adapt its behavior for a particular domain or style. Fine-tuning updates the model's weig...
- Neural NetworkAI & Technology
A computational model loosely inspired by biological neurons, consisting of interconnected layers of mathematical functions (nodes) that transform input data into output prediction...
- Synthetic DataAI & Technology
Artificially generated data that mimics the statistical properties of real-world data, used for training or testing AI models. Synthetic data can be created by generative models, r...
- AgenticAI & Technology
Describing AI systems capable of autonomous action, planning, and decision-making. An agentic AI can break down tasks, use tools, and work toward goals with minimal human intervent...
- AI AlignmentAI & Technology
The research field focused on ensuring that AI systems pursue goals that match human values and intentions. A misaligned AI might optimize for a metric that appears correct but pro...