Latency is the elapsed time between an operation starting and its result becoming available. For streamed AI output, time to first token and total response time describe different user-visible delays.
Latency
Latency is the elapsed time between an operation starting and its result becoming available. For streamed AI output, time to first token and total response time describe different user-visible delays.