Trace
Left: attention; center: total; right: units.
Features
Layer strength bar; IDs list top words.
Words
Three strongest Lens guesses per layer.
An introspection harness for language models
Coming tonight: replay data is not available yet.
Each word moves through Gemma’s layers one at a time. Play one of the recorded questions.
The words Gemma leans toward at each layer come from a lens in the logit-lens and tuned-lens line, after Anthropic’s Jacobian lens (Gurnee et al., 2026). The features are Google DeepMind’s Gemma Scope 2, and the trace of attention and units is an activation trace like TransformerLens-style tools.
Left: attention; center: total; right: units.
Layer strength bar; IDs list top words.
Three strongest Lens guesses per layer.
Each state is a mix of Gemma Scope 2 features. Pick a word and a layer to see their shares and the part left unexplained.
Coming tonight: feature-share data is not available yet.
Feature IDs, their share of this state, and the words linked to each ID.
That the days of the week form a ring inside language models is a known finding (Engels et al., 2024). We test it here on Gemma 3 27B: on this model the ring shows only in the early layers. Later, the Lens favors the computed day before the answer, while the input token keeps its original day.
Handing a model one of its own hidden states and asking what it holds builds on earlier work: Patchscopes (Ghandeharioun et al., 2024), SelfIE (Chen et al., 2024) and studies of introspection (Lindsey, 2025; Macar et al., 2026; Singh, Linzen and Ravfogel, 2026). Here it is tried on Gemma with simple tasks whose true value is known at every token.
Coming tonight: self-reading examples are not available yet.
Some early 4B states name the input token, but that is weak evidence because they stay close to the token itself. At 27B, the description follows the day topic but does not name the computed day, even though the Lens reads the answer there.
Coming tonight: answer-zone layers, matched pairs and causal swaps.
Coming tonight: refusal comparisons and forced-choice day scores.
Self-reading detail will appear here when its data is available.
Letting a model choose edits to its own features has been explored before (Black and Bloom, 2026), and steering with sparse-autoencoder features is established work (for example CorrSteer, Cho et al., 2026). Here it is tried in one recorded 4B session: a feature menu lets Gemma set a feature, undo it, or pass, followed by a read-back check. The menu was unreadable, Gemma replied PASS, and no token change was confirmed.
Coming tonight: 27B self-steering trials.
Coming tonight: held-out 27B choice checks.