ModernVBERT: a small model, on-prem ready
ColModernVBERT reaches near-ColPali quality with 250M parameters and fits on modest hardware.
This translation was produced automatically using AI. The German version is the editorially reviewed original.
The most exciting number in visual retrieval in 2025 is not the biggest, it is the smallest. ColModernVBERT reaches almost the level of ColPali, a model a year older and around ten times larger, with 250 million parameters. That sounds like a detail for model collectors, but it is exactly the difference that makes visual document search affordable on-prem.
01. WHY SIZE BECOMES AN OBSTACLE HERE
Visual retrieval models such as ColPali or ColQwen are accurate, but they are also large. On-prem, meaning on your own in-house hardware, this quickly becomes a cost question: large models demand powerful GPUs, more memory, more power. For many Mittelstand setups, the obstacle is not quality but the hardware bill. This is exactly where a smaller model creates leverage.
02. WHAT MODERNVBERT DOES DIFFERENTLY
ModernVBERT is a compact vision-language encoder with 250M parameters (paper). It combines a modern 150M text encoder built on ModernBERT with a small image encoder, aligning the two through a masked-language-modelling objective. ColModernVBERT is the retrieval fine-tuned variant. The result, according to the paper: despite roughly ten times fewer parameters, it lags ColPali by only 0.6 nDCG@5 on the aggregated ViDoRe benchmark, matching scores that otherwise required models nearly ten times larger. The model and code are openly available.
03. WHY THIS MATTERS FOR SOVEREIGNTY
A small, open, capable model is exactly the combination that makes local operation possible. It fits on modest GPUs, the weights are openly available, and the documents stay in-house because the search does not go to an external endpoint. This is not a security feature bolted on afterwards, but a property of the architecture. We cover the broader context, why local and open belong together, in On-Premise versus Cloud.
04. WHEN THE SMALL MODEL IS THE RIGHT CHOICE
You do not always need the largest model. If your hardware is limited and your documents are typical business records, ColModernVBERT is often the more honest choice than ColQwen: almost the same quality, a fraction of the cost. If you need that last bit of accuracy on particularly dense pages, it is worth comparing against the larger model. How the two work together in the overall stack with Qdrant is covered in the overview.
