The Cold-Start Safety Gap in LLM Agents
Are tool-calling LLM agents equally safe throughout a conversation? We discover they are not: agents are most vulnerable at the very start of a session and become substantially safer after a few regular agentic tasks -- a phenomenon we term the cold-start safety gap. To study this systematically, we introduce Safety Over Depth for Agents (SODA), a benchmark that controls how many regular agentic tasks the agent completes before encountering a safety threat, supporting up to 20 preceding tasks. Evaluating 7 models from 4 families, safety improves by 9--52% as the number of preceding regular agentic tasks increases from zero to twenty. Representation analysis confirms that model hidden stat...
| Authors | Chung-En Sun, Linbo Liu, Tsui-Wei Weng |
|---|---|
| Primary category | cs.CL |
| Published | 5 June 2026 |
| Last updated at source | 27 August 2026 |
Connected in the knowledge graph
- cesun/SODADataset · cited by
Models and datasets referencing this paper
- cesun/SODADataset
Silytics shows public metadata from arXiv and links to the original. Information may be incomplete or out of date. Last refreshed 17 October 2026. Licenses and terms of the original apply to the work itself. Not financial, legal or professional advice.