ThoughtTrace: Understanding User Thoughts in Real-World LLM Interactions
Conversational AI has now reached billions of users, yet existing datasets capture only what people say, not what they think. We introduce ThoughtTrace, the first large-scale dataset that pairs real-world multi-turn human--AI conversations with users' self-reported thoughts: their reasons for sending prompts and reactions to assistant responses. ThoughtTrace comprises 1,058 users, 2,155 conversations, 17,058 turns, and 10,174 thought annotations collected across 20 language models. Our analysis shows that ThoughtTrace captures long-horizon, topically diverse interactions, and that thoughts are semantically distinct from messages, difficult for frontier LLMs to infer from context, diverse ...
| Authors | Chuanyang Jin, Binze Li, Haopeng Xie et al. |
|---|---|
| Primary category | cs.CL |
| Published | 19 May 2026 |
| Last updated at source | 21 May 2026 |
Models and datasets referencing this paper
- SCAI-JHU/ThoughtTraceDataset
Silytics shows public metadata from arXiv and links to the original. Information may be incomplete or out of date. Last refreshed 17 October 2026. Licenses and terms of the original apply to the work itself. Not financial, legal or professional advice.