Module 5.2

Two rankings walk into a formula

Real BM25 keyword scoring, real cosine similarity, blended by an actual formula — reciprocal rank fusion — then rescored by a cross-encoder that reads the query and document together instead of separately.

QUERY: "why did my export fail" — RANKING METHOD

The math

Three formulas, one final ranking

BM25 (keyword)

score = Σ IDF(qᵢ)·f(qᵢ,d)(k₁+1) / (f(qᵢ,d)+k₁(1−b+b·|d|/avgdl))

rewards exact term matches, penalizes long documents

doc #1 scores highest — has both "export" and "fail"

Reciprocal rank fusion

RRF(d) = Σᵢ 1 / (k + rankᵢ(d))

k = 60 (standard constant); combines RANKS, not raw scores

works even when the two score scales are wildly different

Cross-encoder

score = f(query ⊕ doc)

query and doc encoded together, not separately — no precomputing

more accurate, too slow to run on millions of docs

RRF fuses rankings, not scores, specifically because BM25 scores and cosine similarities live on completely different numeric scales — averaging them directly would be meaningless.

The code

Fuse, then rerank the top few

hybrid_rerank.py

def reciprocal_rank_fusion(rankings, k=60):
    scores = {}
    for ranking in rankings:                    # e.g. [bm25_ranking, cosine_ranking]
        for rank, doc_id in enumerate(ranking):
            scores[doc_id] = scores.get(doc_id, 0) + 1 / (k + rank + 1)
    return sorted(scores, key=scores.get, reverse=True)

# fusion is cheap and runs on the whole corpus; reranking is expensive
# and only affordable on a short list — so rerank AFTER fusion narrows it down
top_20 = reciprocal_rank_fusion([bm25_rank, cosine_rank])[:20]
reranked = cross_encoder.rank(query, [corpus[i] for i in top_20])

This two-stage shape — cheap fusion over everything, expensive reranking over a short list — is the standard production pattern for exactly this cost reason.

The Nimbus thread

Why "export" alone isn't enough for Nimbus support

BM25 alone would rank any document containing "export" reasonably high — including doc #3 (the free-tier row cap) and doc #6 (the maintenance queue notice), neither of which actually explains a failure. Cosine similarity alone already ranks doc #1 first because it captures meaning, not just the literal word. Fusing both means Nimbus-bot's retrieval survives both failure modes: pure paraphrases (cosine catches those) and rare exact terms an embedding might blur past (BM25 catches those).

So what

Hybrid search isn't "better vibes" — it's a formula that fuses two disagreeing rankings, so either one's blind spot gets covered by the other.