Inference
[/ˈɪnfərəns/]
nounAI & Technology#ai#ml#deployment#compute0 views1 definitions
Definitions
1
+869
The act of running a trained machine learning model on new input data to generate predictions or outputs. Inference is distinct from training — it is the "serving" phase where the model is used in production, and its speed and cost are critical for real-world applications.
“Inference latency dropped from 2 seconds to 200ms after switching to a quantized model.”
by @mlresearcher1/1/1970