52.4 nDCG@10—that's the result ModernBERT achieves after Vocabulary Transfer. Just yesterday it stumbled over letter case, and today it outranks the best sparse retrievers.

A neural network flunks search because it can't agree on words. 'Token' and 'token' are two distinct universes to it. That's the vocabulary gap: a powerful transformer learns a multitude of useless forms instead of grasping meaning.

Sparse retrieval is when queries and documents become vectors akin to an inverted index: most components are zero, and only a few keywords carry weight. This approach is fast and interpretable but demands the right vocabulary. You'd think a stronger net means better search, but the graphs tell a different story. The fresh ModernBERT, which beats BERT in dense search, systematically lags behind the old BERT-base-uncased in sparse retrieval. The reason? The tokenizer. Modern models, designed for lossless text reconstruction, inflate the vocabulary: case, typos, rare forms create hundreds of variants for a single concept. It's a Babel of confusion, with no 'common language' in sight.

Vocabulary Transfer (VT) acts like a seasoned interpreter. First, pick the target language — the normalized vocabulary from BERT-base-uncased, where 'Token' and 'token' merge into one. Then build a semantic bridge. For shared tokens, copy the embeddings. For new ones, synthesize them as a weighted combination of the nearest 'anchors' from the overlap. The weights come from sparsemax, an operator that zeroes out weak links, keeping only the most similar neighbors. This preserves the topology of the meaning space. A short masked-language-model fine-tuning, just 500 steps, and the model learns the new language. The final touch is activation potential calibration (APC): tweaking the bias so about 40% of neurons fire, avoiding 'dead zones' and representation collapse.

On the BEIR benchmark, ModernBERT-VT hits 52.4 nDCG — a gain of +4.7 points over standard SPLADE. The method even resurrects hopeless models. RoBERTa-large with its case-sensitive BPE vocabulary scored a pitiful 1.4 nDCG, but after transfer it soars to 51.3. BERT-cased — from 10.6 to 50.2. Curiously, without APC calibration, the trained model slips back to 48.8. VT also sets a record for inference-free architectures: 51.5 nDCG when the query encoder is frozen.

Vocabulary design turns out to be no minor architectural tweak but a fundamental bottleneck in neural search. The theoretical framework using Rademacher complexity confirms: vocabulary normalization that preserves semantics improves generalization. Vocabulary Transfer becomes a universal bridge, giving powerful models a common language for sparse tasks. In the future — automatic vocabulary selection, merging dense and sparse approaches, scaling to giant systems. So, by overcoming the chaos of word forms, the neural net finally speaks the language of precise and fast search.