JSON to TOON

JSON to TOON automatically detects all embedded JSON objects and arrays within request messages and converts them to TOON (Token-Oriented Object Notation) format. TOON achieves 30–60% token reduction compared to JSON by using indentation-based nesting and CSV-style tabular arrays — making it significantly more efficient for LLM context windows.

This module has no configuration — it scans all text fields in every message and replaces each detected JSON blob with its TOON equivalent automatically.

Stage

Stage	Behavior
PRE only	Scans all message text fields for embedded JSON and converts each detected blob to TOON format before forwarding to the LLM

What is TOON?

TOON (Token-Oriented Object Notation) is a compact, LLM-friendly alternative to JSON. It preserves structural semantics while using significantly fewer tokens by:

Replacing JSON punctuation ({, }, [, ], ,, ") with indentation and minimal syntax.
Encoding arrays as CSV-style tables when entries share a common schema.

Example

JSON (original, ~120 tokens):

{
  "users": [
    {"id": 1, "name": "Alice", "role": "admin"},
    {"id": 2, "name": "Bob", "role": "viewer"},
    {"id": 3, "name": "Carol", "role": "editor"}
  ],
  "total": 3
}

TOON (converted, ~60 tokens):

users:
  id,name,role
  1,Alice,admin
  2,Bob,viewer
  3,Carol,editor
total: 3

The LLM receives the same information in roughly half the tokens, reducing cost and fitting more data into the context window.

Configuration

This module has no configuration parameters. It operates automatically on all valid JSON found within request messages.

Conversion Behavior

All message fields are scanned — content, system, and any multi-part message arrays.
All valid JSON blobs embedded in text are replaced, even when surrounded by prose.
Invalid JSON is left unchanged — no errors are raised for malformed blobs.
Arrays with uniform objects are collapsed into CSV-style tables.
Nested structures are rendered using indentation-based hierarchy.

Example Use Cases

RAG pipelines: When injecting large JSON-encoded document metadata or retrieved records into a prompt, TOON reduces token overhead significantly.
Structured data Q&A: Sending database query results or API response payloads as context to the LLM.
Cost optimization: Reducing token usage on high-volume deployments where prompts contain structured data.
Context window extension: Fitting more data into the same context window by compressing JSON payloads.

LLM compatibility

TOON is designed to be natively understood by modern LLMs. The format uses familiar indentation and tabular patterns that LLMs consistently parse correctly. If your downstream system needs standard JSON output, pair this module with JSON Normalizer in the POST stage to ensure the LLM response is valid JSON.