The Structured Output Benchmark: A Multi-Source Benchmark for Evaluating Structured Output Quality in Large Language Models
Large Language Models are increasingly being deployed to extract structured data from unstructured and semi-structured sources: parsing invoices, medical records, and converting PDF documents to database entries. Yet existing benchmarks for structured output generation either focus on schema compliance alone, or evaluate value correctness within a single source domain. We introduce SOB (The Structured Output Benchmark), a multi-source benchmark spanning three source modalities: native text, images, and audio conversations. All models receive a text-normalized representation of their context regardless of source modality; this deliberate design isolates structured-output capability from ra...
| Authors | Abhinav Kumar Singh, Harsha Vardhan Khurdula, Yoeven D Khemlani et al. |
|---|---|
| Primary category | cs.CL |
| Published | 28 April 2026 |
| Last updated at source | 28 April 2026 |
Popularity, last 11 days
0 now. Counts as reported by the source, snapshotted daily by Silytics.
Connected in the knowledge graph
- abderahmenn/qwen3.5-9b-sob-json-loraModel · cited by
- abderahmenn/qwen3.5-9b-sob-jsonModel · cited by
Models and datasets referencing this paper
Silytics shows public metadata from arXiv and links to the original. Information may be incomplete or out of date. Last refreshed 27 October 2026. Licenses and terms of the original apply to the work itself. Not financial, legal or professional advice.