Methodik
·
4 MIN
Query decomposition and the advanced-RAG toolkit: match the method to the failure
Query decomposition, HyDE, RAG-Fusion, GraphRAG each fix one failure. The sophisticated 2026 RAG knows when to use none.
LOCATION
Worldwide
SERIES
RAG Architectures
AUTHOR
Sayan Sinha
PUBLISHED
The advanced-RAG techniques are not a maturity ladder you climb. They are a diagnostic toolkit, indexed by the failure you actually observe. Query decomposition fixes multi-hop questions. HyDE fixes vocabulary mismatch. GraphRAG answers corpus-wide questions that are not really retrieval at all. Stacking all of them onto every query multiplies your cost and latency for queries that never needed the help. The sophisticated 2026 system is not the one with the most techniques; it is the one that knows when to use none of them.
01.
Start at the floor, not the ceiling
Before any clever method, get the baseline right, because it fixes most complaints. The 2026 production default is hybrid search (dense embeddings plus keyword BM25) followed by a cross-encoder reranker. Hybrid search catches both semantic and exact-term matches; the reranker re-scores a large candidate set down to the few passages that are actually relevant, not just topically near. Practitioner guides report this combination lifting retrieval quality 15 to 30 percent on standard evaluation sets.
This matters because most real-world failures are the simple kind: the answer was in the documents, but the system did not surface it. That is a recall and ranking problem, and the floor above fixes it. Prove you need more before you build more. Every technique past this point adds LLM calls, latency, and cost, so each one should earn its place against a measured failure, not a hunch.
02.
The toolkit, indexed by the failure it fixes
The useful way to hold the whole zoo of methods is to map each one to the single failure mode it addresses. Reach for a method when you see its failure, not before.
The failure you see | The method that fixes it |
|---|---|
Multi-part or multi-hop question, facts spread across documents | Query decomposition into sub-questions |
Terse or ambiguous query that embeds poorly | HyDE, query rewriting |
One phrasing misses relevant passages | RAG-Fusion (several query variants, fused) |
Question needs a general principle first | Step-back prompting |
Heterogeneous corpora and query types | Routing to the right index or pipeline |
Hard structured constraints (dates, types) | Self-querying (metadata filters) |
Retrieval silently returns wrong documents | Corrective RAG (a grader plus fallback) |
Corpus-wide global question across everything | GraphRAG (entity graph plus summaries) |
Each row has real provenance: HyDE comes from a 2022 CMU paper on zero-shot dense retrieval; step-back prompting from Google DeepMind, 2023; GraphRAG from Microsoft, 2024. This is the same decision-by-failure discipline behind our look at RAG-Fusion and how it differs from plain RAG.
03.
Query decomposition, specifically
Query decomposition splits a complex query into independent sub-questions, retrieves for each, then synthesises one answer. It is the right tool when a single retrieval pass cannot work, because the facts live in different documents or one fact depends on another (who directed the film that won a particular award). The lineage runs from least-to-most prompting in 2022 to the sub-question query engines in today's frameworks.
The honest part is that it is not free, and it can hurt. A July 2025 study from HU Berlin measured decomposition plus reranking lifting multi-hop recall (Hits@10) from 74.7 to 87.2 percent, a real gain. The same study measured the cost: about 16.7 seconds per query against 0.03 seconds for naive retrieval, and it found that decomposing an already-specific query introduces noise and makes the answer worse. So decomposition belongs on genuinely multi-hop questions, not on every query by default. The skill is telling the two apart.
04.
The lever that pays for itself is routing
If you take one operational idea from this, make it routing. Classify the query first, then spend complexity only where it is earned. A 2026 analysis of cost-aware routing cut billed tokens by 26 percent and mean latency by 34 percent at equal answer quality, by sending only about 18 percent of queries to heavy retrieval and 14 percent to no retrieval at all. The expensive methods were reserved for the queries that needed them.
For a German Mittelstand team this is a governance and cost story as much as a quality one. Fewer, justified LLM calls mean predictable spend, lower latency, and a system you can explain to a compliance-minded stakeholder: here is why this query took the expensive path, and here is why that one did not. Over-engineered retrieval is not just slow, it is unexplainable spend. The sophisticated RAG system in 2026 is the one that knows when to use none of its tricks.
