Computational Biology

Repeated Tokens, Attention Sinks, and LLM Context Loss

A look at research linking repeated-token divergence to attention-sink circuitry, what the authors establish mathematically, and why implications for biological applications still need direct testing.

1. The Illusion of Comprehension and the "Attention Sink"

Large Language Models (LLMs) can fail at seemingly simple repetition tasks: when prompted to repeat a token many times, some models eventually generate unrelated text. Yona et al. (2025) investigate this repeated-token divergence and connect it to the attention-sink mechanism. The paper also discusses how this behavior can be exploited in some settings, but the result should be understood as a specific failure mode rather than evidence that LLMs generally lack comprehension.

The study links this behavior to attention sinks, in which the first token often receives disproportionately high attention. In the models analyzed, the authors describe a circuit involving the first attention layer and specific neurons in a later multilayer perceptron (MLP): the circuit helps mark the first token and amplify its hidden-state norm. Under long repeated-token inputs, the same mechanism can also be triggered by later repeated tokens, which then attract unusually high attention and disrupt the model's normal processing. These mechanistic findings were demonstrated in the models and input patterns examined in the paper.

2. Repeated Tokens and Softmax Leakage

A central result is conditional rather than universal. Yona et al. provide a formal theorem (Theorem 4.1) for a decoder-only Transformer sequence with a fixed prefix followed by repeated copies of one token. In that construction, the final-token representation converges toward the representation of the repeated token in a singleton sequence as the repetitions increase.

Consider a sequence $S_n$ consisting of $k$ fixed prefix tokens (the context) followed by $n$ repetitions of a token. The researchers proved that as $n \to \infty$, the representation of the final token converges strongly to the representation of a singleton sequence $S^*$ containing only the repeated token:

$\lim_{n\to\infty} ||T^{(L)}(S_n)_{n+k} - T^{(L)}(S^*)_1|| = 0$

The paper also analyzes a theoretical limit known as softmax leakage. When a fixed prefix is followed by an increasingly long run of the same token, the prefix's relative influence on the final token representation can approach zero. This result applies to the sequence construction studied in the paper; it is not a general rule that every long prompt loses its context.

3. What This Could Mean for Biological Applications

It would be premature to treat this result as proof of a general "context amnesia" problem in biological AI. The study does not evaluate gene-expression interpretation, protein folding, clinical time series, or biological sequence models. Those tasks involve different inputs and tokenization choices, and their reliability must be assessed empirically. The paper instead provides a specific mechanistic result that can motivate careful testing of repeated or highly structured inputs in scientific workflows.

This mechanism raises questions about repetitive inputs in scientific workflows, including genomic or proteomic text. However, the cited study does not test biological sequences or establish that LLMs necessarily forget biological context when processing them. Applying the result to such workflows remains a hypothesis that requires task-specific evaluation using realistic tokenization, sequence lengths, and appropriate controls. The evidence supports a narrower conclusion: repeated-token patterns can disrupt attention-sink circuitry in the models examined.

💡 My Practical Perspective: The Limits of Generative AI in the Lab

The practical lesson is to test model behavior rather than assume that a general benchmark guarantees reliability on every input pattern.

For biological workflows, researchers could evaluate repeated or structured inputs using representative sequences, realistic tokenization, appropriate controls, and task-relevant outcome measures. This paper does not demonstrate that LLMs necessarily erase biological context in DNA sequences or clinical time series; it identifies a failure mode in particular models and input constructions that may justify targeted evaluation.

Biological interpretation depends on experimental context, so conclusions from language-model behavior should be translated into domain-specific claims only when supported by direct evidence. The study offers a useful testable question, not proof that LLMs are inherently unable to preserve biological context.