Usage & Performance Metrics

When completions finish, GoingMerry includes statistics in the final JSON chunk. These parameters allow you to monitor local system load and calculate inference performance.

---

1. Metrics Fields Reference

The final JSON response includes:

Parameter	Unit	Description
total_duration	nanoseconds	Total wall-clock duration of the request.
load_duration	nanoseconds	Time spent loading model weights into memory.
prompt_eval_count	count	Number of tokens in the prompt.
prompt_eval_duration	nanoseconds	Time spent evaluating the prompt.
eval_count	count	Number of tokens in the generated response.
eval_duration	nanoseconds	Time spent generating output tokens.

---

2. Calculating Tokens Per Second (tok/s)

Tokens per second = (eval_count / eval_duration) * 10^9

Python Example

import json

# Final chunk data
data = {
    "eval_count": 250,
    "eval_duration": 5000000000 # 5 seconds
}

tokens_per_second = data["eval_count"] / (data["eval_duration"] * 1e-9)
print(f"Velocity: {tokens_per_second:.2f} tok/s") # Output: 50.00 tok/s