Late interaction explained: why Qdrant and ColQwen build the better knowledge base
Late interaction compares every search term against every page excerpt. Qdrant stores that natively.
This translation was produced automatically using AI. The German version is the editorially reviewed original.
The difference between a mediocre and a good vector search often sits in one detail: whether a page is represented by a single vector or by many. Late interaction picks many, and that is exactly what makes models like ColQwen so accurate on visually dense documents. Qdrant is one of the vector databases that supports this natively. This piece explains the principle without the math baggage.
01. THE PROBLEM WITH THE SINGLE VECTOR
Classic vector search compresses an entire passage or an entire page into a single vector. That is economical, but it averages away what matters locally. If the amount you are searching for sits in a specific table cell, that reference disappears into the average of the whole page. For clean running text, this is often enough. For the documents that visual search exists for in the first place, it is not enough.
02. LATE INTERACTION, IN ONE SENTENCE
Late interaction keeps many vectors per page and compares them late, only at search time. The model produces one vector per token for the search query, and one vector per patch for the page. The scoring is called MaxSim: for each query token, the best-matching page patch is found, and these best matches are summed. This lets the word "remaining balance" dock precisely onto the table cell it appears in. The technique originates from ColBERT and was carried over to images with ColPali. If you want to see the whole toolkit, the OCR-free stack places the models in context.
03. WHERE QDRANT COMES INTO PLAY
Late interaction needs a database that understands multiple vectors per object. Qdrant supports such multi-vectors directly, without pre- or post-processing, and can carry any late interaction model such as ColBERT or ColPali (docs). The usual setup is two-stage: a fast initial search with normal dense vectors narrows down the candidates, then MaxSim precisely re-scores only these few pages. This keeps the index affordable, because the expensive token vectors do not need to be fully indexed, but instead serve re-ranking.
04. WHAT THIS MEANS IN PRACTICE
You get results that respect layout, without making search unaffordably expensive. The price is more storage per page and somewhat more complexity when building it. Both are manageable if you plan the re-ranking stage in from the start instead of fully indexing every token vector. For very long contexts, it is worth looking at subquadratic models; for the basic idea of retrieval, What is RAG remains the starting point.
