Streaming

Receive chat answers as server-sent events as they are written, and detect the final frame.

Chat streams unless you send "stream": false. A streamed response is text/event-stream: the answer arrives in pieces as the model writes it, then a final frame carries the complete result.

Wire format

Each frame is one data: line followed by a blank line. There are no event: names and no [DONE] marker.

data: {"chunk": "The vendor is "}

data: {"chunk": "**Northwind Office Supply Co.**"}

data: {"query": "Who is the vendor?", "response": "The vendor is **Northwind Office Supply Co.**", "session_id": "98898bfd-…", "chat_id": "87aa7baa-…", "data": {"knowledge": {"relevant_sources": {…}}}, "is_answered": "yes", …}

Frames with a chunk field carry the next piece of answer text.
The final frame is the complete answer object — the same fields as a non-streaming response. Recognise it by its response and session_id fields.
If generation fails after the stream started, a frame with a non-empty error arrives instead, and the HTTP status is still 200. (The final frame of a successful answer also has an error field, set to null — check its value, not whether the key exists.)

Because the frames are standard data: lines, any SSE parser reads them. The samples below parse them directly.

Python

import json
import os

import requests

API_ROOT = "https://futuresmart.ai/document-intelligence/api"
COLLECTION_ID = os.environ["DOCUMENT_INTELLIGENCE_COLLECTION_ID"]

with requests.post(
    f"{API_ROOT}/v1/{COLLECTION_ID}/chat",
    headers={"x-api-key": os.environ["DOCUMENT_INTELLIGENCE_API_KEY"]},
    json={"query": "Who is the vendor on the invoice?", "stream": True},
    stream=True,
    timeout=120,
) as r:
    r.raise_for_status()
    final = None
    for line in r.iter_lines(decode_unicode=True):
        if not line.startswith("data: "):
            continue
        frame = json.loads(line[6:])
        if frame.get("error"):
            raise SystemExit(f"Stream failed: {frame['error']}")
        if "chunk" in frame:
            print(frame["chunk"], end="", flush=True)
        elif "response" in frame:
            final = frame

print()
print("session:", final["session_id"])
print("sources:", [s["title"] for s in final["data"]["knowledge"]["relevant_sources"]["data"]])

JavaScript

const API_ROOT = "https://futuresmart.ai/document-intelligence/api";
const COLLECTION_ID = process.env.DOCUMENT_INTELLIGENCE_COLLECTION_ID;

const res = await fetch(`${API_ROOT}/v1/${COLLECTION_ID}/chat`, {
  method: "POST",
  headers: {
    "x-api-key": process.env.DOCUMENT_INTELLIGENCE_API_KEY,
    "Content-Type": "application/json",
  },
  body: JSON.stringify({ query: "Who is the vendor on the invoice?", stream: true }),
});
if (!res.ok) throw new Error(`${res.status} ${await res.text()}`);

const decoder = new TextDecoder();
let buffer = "";
let final = null;
for await (const bytes of res.body) {
  buffer += decoder.decode(bytes, { stream: true });
  let newline;
  while ((newline = buffer.indexOf("\n")) >= 0) {
    const line = buffer.slice(0, newline).trim();
    buffer = buffer.slice(newline + 1);
    if (!line.startsWith("data: ")) continue;
    const frame = JSON.parse(line.slice(6));
    if (frame.error) throw new Error(`Stream failed: ${frame.error}`);
    if (frame.chunk !== undefined) process.stdout.write(frame.chunk);
    else if (frame.response !== undefined) final = frame;
  }
}
console.log();
console.log("session:", final.session_id);

Shell

curl -sS -N -X POST "https://futuresmart.ai/document-intelligence/api/v1/$DOCUMENT_INTELLIGENCE_COLLECTION_ID/chat" \
  -H "x-api-key: $DOCUMENT_INTELLIGENCE_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"query": "Who is the vendor on the invoice?"}'

Errors before the stream starts

Authentication, permission, validation and quota errors (401, 403, 404, 422, 429) are ordinary JSON responses with that status, sent before any frame. Check the status before reading the stream, as the samples do.

Streaming

#Wire format

#Errors before the stream starts

Wire format

Errors before the stream starts