Part 4 of 5 in the LibriBrain series. The project page summarises the whole project. Previous: Eighth in public, seventeenth in private.
A decoder that reads words from magnetoencephalography ought to fail when the brain is taken away. The usual check replaces the recording with noise, runs the same pipeline and confirms that the score falls to chance. Two recent papers have done most of the work of turning that check into an audit, and what follows builds directly on them.
Jayalath and Parker Jones (“Removing Timing Shortcuts Improves Non-Invasive Brain-to-Text”, arXiv 2609.40359), from the group that organises LibriBrain, work on this very dataset. Their synthetic control is a signal of smoothed random pulses generated independently of the words and their onsets, then cut into three-second windows at the true word onsets exactly as the MEG is. Because neighbouring windows overlap at offsets set by the real timing, the overlap alone carries duration, and joint decoding of the synthetic signal comes close to joint decoding of MEG. They also include a Timing condition in which each window is replaced by the log interval to the next word and passed through the same sentence-level model. It reaches 22.9% balanced top-1, against 22.3% for joint MEG decoding. In their Section 3.1 and Appendix C.1 they note that a three-second window can contain neural responses to subsequent words, and they test whether their decoder benefits from those words, finding that it does not appear to do so significantly. Zhang et al. (arXiv 2605.24524) audit MEG-to-audio retrieval on the Gwilliams and MOUS datasets. They show that signal-blind noise scores far above chance under variable-length decoding and collapses to near chance under fixed-duration windows and stimulus-identity splits, and they propose an inference-time method, Group Context Bias, for measuring how much retrieval gains from pooling evidence across a sentence.
My addition is narrow. Both papers establish what timing can do through window overlap, variable length or explicit intervals. I wanted a control and a floor for the case they leave closest to clean, a fixed window decoded in isolation, without sentence context, in which the only route for timing is the brain’s own response to the words that follow.
What a noise control removes
Noise has no response locked to word onsets. Real MEG has one at every word the listener hears, small but at a fixed lag from the onset. A fixed three-second window placed at one word therefore also contains the onsets of the words that follow it, and the number and position of those onsets depend on how long the current word lasted. In LibriBrain the fifty target words are almost all short function words, and duration separates many of them. A model that can locate the next onsets inside its window has a cue to identity that owes nothing to how the brain encodes the word itself.
A noise control answers whether the pipeline leaks through window geometry, which is an important question, and leaves untested whether the brain’s response to the following word leaks the current word’s identity. To test the second question I built what I have been calling a timing twin. It is the same noise with one fixed onset kernel added at every true word onset. The kernel is identical for every word, so the stream contains when words occurred and nothing about which words they were.
The measurements
The measurements in this section and the next come from one subject, the Deep listener, decoded on a held-out day (Sherlock1, sessions 11 and 12, run 2) with a single training seed. Scores are balanced over the fifty words and reported as top-1 and top-10, against chance levels of 0.02 and 0.20. Each session contributes about 1,300 rows. The text heard on the held-out day shares no 5-gram with any training run, so the split is held out by day and by text, which is the stimulus-identity condition that Zhang et al. recommend.
The reference model is a screening decoder with a fixed three-second window and no sentence context, which is a different and much weaker model than my competition entry. This no-context screening decoder scores 0.0725 top-1 and 0.458 top-10. Pure shared noise through the same pipeline scores 0.023 and 0.221, which is chance, consistent with Zhang et al.’s finding that fixed windows remove the structural leak; here the windows are also not clipped at sentence edges. A twin whose onset kernel has an amplitude of two noise standard deviations scores 0.159 and 0.659.
That kernel is about fifteen times the real onset response. A generic onset response fitted to real MEG explains 0.1% of the variance on a held-out run, and its peak sits at about 0.135 residual standard deviations. A twin built at that amplitude on every channel still scores 0.119 top-1 and 0.611 top-10, above the no-context screening decoder. That figure turned out to be an artefact of the background. The twin’s noise is white and independent across channels, so a 306-channel network can pool the fixed pattern across sensors far more efficiently than it could through real background, which is strongly correlated in space and drifts in time. A realistic twin measures the contribution directly. For each recording I removed the fitted onset response from the real MEG, shifted what remained by a random offset between a quarter and three quarters of the run so that no word-locked activity stays aligned with its word, and added the same recording’s onset response back at the true onsets. The background is then real MEG with its own spatial covariance and dynamics, the onset response has its real amplitude and shape, and no response specific to any word survives. The same network, trained and tested exactly as on real data, scores 0.031 top-1 and 0.251 top-10 on the held-out day, against chance at 0.02 and 0.20 and a label-permutation null of 0.019 top-1. Real MEG gives the same network 0.071 and 0.453 over three seeds. Timing read through the brain’s generic onset response therefore accounts for about a fifth of the decoder’s margin over chance, roughly 0.05 of the 0.25 on top-10 and 0.011 of 0.051 on top-1. The earlier one-channel readout, 0.029 top-1 at three seconds, had landed close to this figure.
A floor made of event times
If each word’s onset is known exactly, a boosted classifier can be given only the onsets that fall within the interval from the word’s onset to W seconds later, and asked which word began the interval. This uses the event files and nothing else. On the same held-out day it scores 0.109 top-1 and 0.554 top-10 with W = 0.5 s, 0.113 and 0.568 at 1 s, and 0.136 and 0.588 at 3 s. Expressed as Packard’s (1990) conditional information, the onset pattern in the window, meaning the offset of the next onset and the number of onsets, carries between 0.53 and 0.66 bits about the word on held-out data. This floor differs from the timing-only score of 0.632 in my earlier post, which used sentence context and relative timing inputs on a different split; the 0.588 here comes from exact onsets inside one fixed window.
I regard this floor as a legitimate baseline for the benchmark. The onsets are given to every entrant as metadata, because a decoder must know where each word lies in order to cut a window around it, and nothing about their use breaks the competition’s rules. Once a window outlasts the word, the metadata alone predicts the label well above chance, and the scientifically interesting quantity becomes the margin that brain data adds over what the event times already supply. The floor can be computed by anyone from the event files alone, with no network and no GPU.
At three seconds the event-only floor exceeds the no-context screening decoder, at 0.136 against 0.0725 top-1. A classifier that sees onsets exactly is not a ceiling on a MEG decoder, and a decoder that falls below it may still be reading something other than timing; real MEG renders onsets faint. The comparison does show that chance is the wrong reference for a fixed window of this length. A stronger model makes the point from the other side. A sentence-context decoder of the competition kind, without timing inputs and trained with the same day held out, scores 0.693 top-10 on these rows, against 0.588 for the event-only floor.
On the leaderboard
The louder twin also went to Kaggle as a late submission, with an isolated one-second window. On the Deep track it scored 0.5915 on the public split and 0.5805 on the private split. A Broad version, built with the Broad recipe, scored 0.3066 and 0.3741. My selected Deep entry scored 0.697 privately. These comparisons are not like for like, since the entry used sentence context and the twin’s onset response is about fifteen times stronger than the brain’s, but a model that never received a response specific to any word covered most of the distance from chance to the entry on Deep.
The floor can be rerun from the event files alone in a public Kaggle notebook, LibriBrain: How Much of the Label Is Timing?, and the twin, the floor and the deconvolution are in the libribrain-floor toolkit.
What remains open
The realistic twin rests on one seed, and it carries only the linear onset response common to every word. Effects of timing that a linear kernel misses, such as a response whose size depends on the interval since the previous word, are absent from it, so it measures the generic onset channel and not every route by which timing could reach a decoder. A related check points the same way. Subtracting the common onset response from real MEG before training leaves the screening decoder at 0.063 top-1 and 0.469 top-10, about 0.01 lower on top-1 and slightly higher on top-10. All of the numbers above come from one subject and one seed, and the controls in the two papers cited at the start cover more conditions and datasets than mine do.
The change I would propose to standard practice is small. A control that removes the brain should keep the stimulus timing, since the timing is given to every decoder and the brain’s response to it is present in every real recording. In the meantime the event-only floor costs nothing to compute and could be reported beside chance in any benchmark that supplies word onsets.
Next in the series: Noise as an equation, which models the MEG background itself. A philosophical note on what these comparisons imply for decoding claims is in The baseline is part of the claim.