#compute

The act of running a trained machine learning model on new input data to generate predictions or outputs. Inference is distinct from training — it is the "serving" phase where the model is used in production, and its speed and cost are critical for real-world applications.

“Inference latency dropped from 2 seconds to 200ms after switching to a quantized model.”