Streaming structured outputs without breaking the UI

Partial JSON is a UX problem first, a parser problem second. How we render incomplete tool calls cleanly.

Amy Team1 min read

A streaming JSON parser is the obvious thing to reach for. The less obvious problem is what to show while you have half a tool call.

We built a tolerant parser, looked at the UX, and threw the parser away. What users wanted wasn't half a JSON object — they wanted to see the model's intent (the tool name, the rough shape of the args) and then a "preparing…" state until the call was complete enough to act on. Two states beat seventeen partial ones every time.

More in Amy Engineering

View all →

How is Amy's credit system

How we ship a credit-based ledger that survives partial failures, refunds, and webhook re-deliveries — without losing a single cent.

Henry Ng3 min read