Skip to main content

Streaming Completions

Streaming allows GoingMerry to output tokens chunk-by-chunk as they are generated by the model. This significantly lowers perceived latency and improves the user experience.

1. Enabling Streaming​

Streaming is enabled by default for both the /api/generate and /api/chat endpoints. If you want to configure streaming, ensure the "stream" flag is set to true (or omitted).

curl Example​

curl http://localhost:11434/api/generate -d '{
  "model": "gemma4",
  "prompt": "Count to 5"
}'

2. Consuming the Stream​

The server will stream responses using standard line-delimited JSON format. Each line is a valid JSON object:

{"model":"gemma4","response":"1","done":false}
{"model":"gemma4","response":",","done":false}
{"model":"gemma4","response":" 2","done":false}
{"model":"gemma4","response":",","done":false}
{"model":"gemma4","response":" 3","done":false}
{"model":"gemma4","response":"...","done":true}

Disabling Streaming​

If you prefer a single complete response instead of a stream, set "stream": false in your request payload. The server will buffer all tokens and return a single JSON object.