Causal Latent Workspaces for Looped Transformers
Reusing a Transformer block across depth increases computation without replicating its parameters, but successive passes have no dedicated state through which positions can exchange compact summaries of their causal prefix. We introduce the causal latent workspace Transformer: an anchored, four-round shared-depth architecture in which each position writes gated vectors into prefix-specific latent slots and retrieves a content-addressed summary before the next shared block application. This path complements causal self-attention while the original input embedding remains accessible at every round. Across three byte-level corpora, three seeds, and two matched parameter budgets, the proposed...
| Authors | Ilia Zaznov |
|---|---|
| Primary category | cs.CL |
| Published | 16 October 2026 |
| Last updated at source | 16 October 2026 |
Popularity, last 6 days
0 now. Counts as reported by the source, snapshotted daily by Silytics.
Silytics shows public metadata from arXiv and links to the original. Information may be incomplete or out of date. Last refreshed 24 October 2026. Licenses and terms of the original apply to the work itself. Not financial, legal or professional advice.