I keep seeing posts pitching semantic search as the answer to retrieval. It isn't — it's half of it.
Here's the failure mode nobody in those posts shows you: a bi-encoder embeds meaning, not exact tokens. Ask it to find "error code E-4471" and it will happily return documents about "error codes" and "hardware faults" — semantically close, but useless to the engineer who needs the one document containing that exact string. Product SKUs, acronyms, part numbers, error codes — anything where exact match matters — this is where pure vector search silently degrades. Not loudly. Silently. It returns something plausible-looking, and that's what makes it dangerous in production.
BM25 doesn't have this problem. It's decades-old, unglamorous, keyword-based — and it finds "E-4471" every time. But BM25 alone is blind to paraphrase: ask about "shutting down the system safely" when the document says "graceful termination procedure," and it misses entirely.
Neither one is wrong. They're wrong about different things. That's the actual argument for hybrid search — not "more is better," but "these two failure modes don't overlap."
The part that's harder to get right than people assume: BM25 and bi-encoder scores live on different scales. You can't just add them. Reciprocal Rank Fusion sidesteps the scaling problem entirely — it fuses based on rank position in each list, not raw score, which is what makes combining two structurally different retrieval methods actually sound rather than a hack.
I built this pipeline end to end against a fixed corpus and a fixed query set specifically so I could measure the failure mode, not just claim it exists: naive bi-encoder retrieval failing on exact-match queries, BM25 recovering them, RRF fusion combining both without one drowning out the other, then a cross-encoder reranking the fused candidates for the queries where ranking order actually mattered.
None of this is exotic. It's also not something you can shortcut by picking a vector DB with a nice UI. If your RAG system only has a bi-encoder in it, you already have a blind spot — you just haven't hit the query that exposes it yet.