Viral Proteins Reveal Geometry of Protein Language Models
Protein language models are trained on highly imbalanced datasets, raising the question of how they represent underrepresented biological sequences. Using viral proteins as a case study across ESM model families, we identify a dominant nativeness axis in embedding space, aligned with masked reconstruction perplexity, that orders sequences from well-modeled cellular proteins through viral proteins to shuffled and random sequences. Scaling contracts this axis unevenly across viral families. Despite this, protein language model embeddings retain viral-specific signal: viral proteins remain linearly separable beyond zero-shot perplexity and shallow sequence features. Together, these results s...
| Authors | Arthur Bigot, Harmon Bhasin, Core Francisco Park et al. |
|---|---|
| Primary category | cs.LG |
| Published | 10 June 2026 |
| Last updated at source | 10 June 2026 |
Connected in the knowledge graph
- jang1563/narrow-model-safety-evalDataset · cited by
Models and datasets referencing this paper
Silytics shows public metadata from arXiv and links to the original. Information may be incomplete or out of date. Last refreshed 17 October 2026. Licenses and terms of the original apply to the work itself. Not financial, legal or professional advice.