Latency
[/ˈleɪtənsi/]
Definitions
The time delay between initiating an action and receiving the first response. In networking, latency is the round-trip time for a data packet; in AI, it often refers to time-to-first-token or end-to-end inference time. Lower latency means faster, more responsive user experiences.
“The new model has lower latency but slightly less accuracy — a classic speed/quality trade-off.”
Related Terms
- BaseTechnology
The foundational infrastructure layer of the PlatPhorm News Network (base.platphormnews.com). Base provides core shared services — authentication, storage, and routing — that all o...
- Edge ComputingTechnology
A computing paradigm that processes data at or near its source — at the "edge" of the network — rather than sending it all to a central cloud datacenter. Edge computing reduces lat...
- Rate LimitingTechnology
A technique for controlling the frequency of requests a client can make to an API or service within a given time window. Rate limiting protects systems from abuse, prevents overloa...
- ServerlessTechnology
A cloud execution model where the provider manages server infrastructure automatically. Developers deploy individual functions that scale from zero to millions of invocations witho...
- ThroughputTechnology
The amount of work a system can process in a given time period. In APIs it's usually measured in requests per second; in AI inference it's tokens per second. Throughput and latency...
- API-FirstTechnology
A design philosophy where the API contract is defined and agreed upon before any implementation begins. API-first teams treat the API as the product — writing the specification fir...