Chapter 8: Hybrid Search

Time: 40 minutes. Cost: $0. Parts 1 to 3 of the lab need only the shared dataset. Part 4 uses the PostgreSQL container from Chapter 7.

The short version

• Vector search matches meaning. It is weak on exact names, codes, and rare words, like an error code or a product number.
• Keyword search, with a ranking method called BM25, is strong exactly there. The two miss different questions, so many production search systems run both and merge the results.
• Merge carefully. A plain, equal merge did worse than keywords alone on this chapter's data. Measure each method on its own first.

Chapter 4 left a number hanging. Exact vector search, with no approximation at all, put the paragraph each question was written from in its top 10 only 59% of the time. Every index since then has been measured against that exact top 10. Tuning an index moves you toward that 59%. It does not fix what the embedding model misses.

Here is a keyword method from the 1990s that, on the same 1,000 questions, does 88%:

  method                             found@1  found@10    MRR
  vector (exact, nomic-embed-text)       0.36       0.59   0.43
  BM25 keywords                          0.68       0.88   0.75

Each question has one right paragraph, the one it was written about, so this is the customer check from Chapter 3. found@10 is how often that paragraph is anywhere in the top 10, and found@1 is how often it comes first. MRR, mean reciprocal rank, gives a question 1 point when the right paragraph is first, half a point when it is second, a third when it is third, and so on, then averages. Higher is better on all three.

That is not a typo, and it is not the end of vector search. It is the reason most production search systems run both. This chapter explains why the old method wins here, builds it by hand, and then combines the two.

Why the old method wins here

Start with the honest part. This dataset favors keywords. The people who wrote SQuAD's questions were looking at the paragraph while they wrote them, so the questions reuse its words. The authors of Dense Passage Retrieval, a 2020 paper that trained vector search for exactly this kind of question, saw the same thing. Their model beat BM25 on four of five question datasets and lost on SQuAD. They offered two likely reasons, and the first is this one: the annotators wrote questions after seeing the passage, so high word overlap gives BM25 a clear advantage.

That does not make the result a fluke. It makes it a warning. Your users do the same thing whenever they type an exact name: an error code, a product SKU, a person, a drug, a clause number. Look at the example from the lab:

  Example question: "What does CATOBAR allow for?"
  word        in paragraphs    idf
  catobar                 4   8.34
  allow                 255   4.30
  Source paragraph is from 'Aircraft carrier'.
  BM25 ranks it #3. Vector search does not have it in its top 100.

CATOBAR is a way of launching and recovering aircraft on a carrier. Only 4 of 18,896 paragraphs contain the word, so to BM25 it is gold. To the embedding model it is a rare string the model probably saw a handful of times in training, split into fragments, and squeezed into the same 768 numbers as everything else in the question. The vector captures "a question about what something allows." It loses the one word that mattered.

Here is the help-desk version of the same failure. A user pastes 0x80070057 into your support search. To an embedding model, that is noise that looks like other hexadecimal noise. To a keyword index, it is an exact match on the one article that mentions it.

BM25: three ideas in one formula

BM25, short for "best matching," is the keyword ranking that Elasticsearch, OpenSearch, and Solr use out of the box. Think of a recruiter skimming résumés for a job. Three habits make a good skimmer, and BM25 has all three. It scores a paragraph by adding up, for every word in the question:

1. Rare words count more. Every applicant mentions "Excel." The one who mentions "COBOL" tells you something. That is IDF from Chapter 1. CATOBAR, in 4 paragraphs, is worth almost twice as much as "allow," in 255.
2. Repeats help, but less each time. A paragraph that says "carrier" three times is probably more about carriers than one that says it once. Thirty times is not ten times better, any more than a résumé that says "team player" thirty times is. The setting k1 controls how quickly the score levels off. This is called saturation.
3. Long paragraphs get scaled down. A ten-page résumé mentions almost everything once. The setting b discounts matches in longer-than-average paragraphs.

Lucene's defaults are k1 = 1.2 and b = 0.75, and so are the lab's. The README has you change both and watch what happens: with saturation effectively turned off, found@10 drops from 0.88 to 0.72.

The formula, and where BM25 came from

For each question word, BM25 adds:

idf(word) × count × (k1 + 1) / (count + k1 × (1 − b + b × length / average length))

count is how many times the word appears in the paragraph, and length is the paragraph's length in words.

BM25 comes from a line of work you have already met. Luhn counted words in 1957. Karen Spärck Jones introduced inverse document frequency in 1972. In 1976, Stephen Robertson and Spärck Jones turned relevance weighting into a probabilistic model. Robertson's team at City University London ran an experimental search system called Okapi, and in 1994 they presented the version known as BM25 at the TREC-3 conference. It became the default ranking in Lucene 6.0 in 2016, which is why Elasticsearch, OpenSearch, and Solr use it.

BM25 is also fast, for an old reason. The lab builds an inverted index, a dictionary from every word to the paragraphs that contain it. Scoring a question only touches paragraphs that share at least one word with it. That is the same structure Chapter 4 borrowed for IVF.

They miss different questions

The most useful numbers in the lab are not the averages. They are this tally, question by question:

  Source paragraph in the top 10 of:
    both                 547
    BM25 only           335
    vector only           44
    neither               74

BM25 wins far more often on this data, and vector search still finds 44 that BM25 cannot. Look at what they are:

  Found by vector search only, for example:
    "Where do samurais' teachings live on?"
    "What has helped geneologists researching slaves?"
    "Who led the group which created the Soviet state?"

"Samurais'" is not the word "samurai" to a keyword index. "Geneologists" is misspelled. "The Soviet state" is a paraphrase of a name the paragraph spells differently. A keyword index matches strings. An embedding matches meaning. When users misspell, paraphrase, or describe instead of name, vector search wins. When they name the exact thing, keywords win.

You want both. The question is how to combine two ranked lists.

Sparse vectors: the same idea, learned

You can describe BM25 in this track's own terms. Give every paragraph a vector with one position for each of the 76,336 words in the collection. Almost every position is zero. Only the words in the paragraph get a weight. That is a sparse vector, and BM25 is one way to fill in the weights. The embeddings from Chapter 1 onward are dense vectors: 768 positions, all of them used.

Learned sparse models keep the sparse format and let a neural network choose the weights. One of the best known, SPLADE, also adds related words the paragraph never used, so a paragraph about "cars" can carry some weight for "automobile." The result still works with an inverted index, which is why sparse search stays fast. pgvector added a sparsevec type in version 0.7.0 for vectors like these.

I left SPLADE out of the lab for a practical reason. On my laptop's CPU, encoding just 500 of the paragraphs took three and a half minutes, which puts the whole collection at over two hours. A GPU should make it far faster, and if you have one, it is worth trying.

Fusing two lists

Here is the problem with simply adding the scores. Cosine similarity in this lab stays between about 0.2 and 0.9. A question's best BM25 score ranges from about 12 to over 40, depending on the question. Add them, and BM25 decides everything. You need either to ignore the scores or to put them on the same scale.

Reciprocal rank fusion (RRF) ignores the scores. Think of two friends who each give you their 10 favorite restaurants in town. A place near the top of both lists is a safer bet than one that tops only one list. RRF turns that into arithmetic. Each list gives every paragraph a vote of 1 ÷ (60 + its rank), and the paragraph with the most votes wins. A paragraph ranked first in one list gets 1/61, tenth gets 1/70, and a paragraph near the top of both lists beats one at the very top of only one. The 60 is the constant its authors recommended when they published it in 2009. RRF needs no tuning and no knowledge of either score.

Score blending keeps the scores. It is like averaging two exams, one marked out of 10 and one out of 100: convert both to percentages first. Rescale each list to run from 0 to 1, then add them with a weight, such as 50% vector and 50% BM25.

  method                             found@1  found@10    MRR
  vector alone                           0.36       0.59   0.43
  BM25 alone                             0.68       0.88   0.75
  RRF, k=60, equal votes                 0.52       0.85   0.61
  RRF, k=60, BM25 votes count 2x         0.56       0.90   0.66
  score blend, 50% vector                0.70       0.92   0.77
  score blend, 30% vector                0.71       0.91   0.77

Look at the third row before celebrating the fifth. Plain RRF did worse than BM25 alone. RRF treats both lists as equally trustworthy, and on this data they are not. Vector search's confident wrong answers got the same votes as BM25's right ones, and they pulled the right paragraph down. Counting BM25's votes twice helped. Blending the scores helped most, because it keeps how confident each method was, and a high BM25 score for a rare word like CATOBAR says something that rank 1 alone does not.

There is a catch in that last row, and I want to be candid about it. I picked the blend weights by looking at the answers. On your data, you would choose weights on one set of questions and check them on another, or you are grading your own homework.

My recommendation is still to start with RRF, as a baseline rather than a final answer. It cannot be thrown off by score scales, it needs no tuning, and many search engines and databases now offer it as a built-in option. Then measure each retriever on its own, the way Part 2 does. That is how this lab found the problem: vector search was much weaker than BM25 on these questions, so equal votes hurt. If one retriever is clearly stronger on your data, weight it, or try a blend.

Hybrid search in PostgreSQL

PostgreSQL has had full-text search built in for years. Part 4 of the lab adds a column of normalized words to the Chapter 7 table, indexes it, and runs the same 1,000 questions:

  method                                   rows back  found@1  found@10
  full-text, every word must match               0.6     0.26      0.28
  full-text, any word may match                 10.0     0.63      0.84
  hybrid: full-text + pgvector with RRF         10.0     0.50      0.85

The first row is a trap worth knowing. PostgreSQL's query functions, such as websearch_to_tsquery, require every word by default. A natural question rarely has every one of its words in one paragraph, so most questions matched nothing at all: 0.6 rows back on average. Putting "or" between the words fixed it.

The second point is quieter. PostgreSQL's ts_rank is not BM25. It does not weigh rare words the way IDF does. Here it came reasonably close, 0.84 against 0.88. If you need real BM25 inside PostgreSQL, extensions provide it, such as ParadeDB's pg_search.

Here is the hybrid query from the lab. Each half is an ordinary query, and the last SELECT is RRF in SQL:

WITH keyword AS (
    SELECT id, rank() OVER (ORDER BY ts_rank(search, query) DESC) AS rank
    FROM paragraphs, websearch_to_tsquery('english', 'what or does or catobar or allow or for') query
    WHERE search @@ query
    ORDER BY ts_rank(search, query) DESC
    LIMIT 50
),
semantic AS (
    SELECT id, rank() OVER (ORDER BY embedding <=> '[0.01, -0.03, ...]') AS rank
    FROM paragraphs
    ORDER BY embedding <=> '[0.01, -0.03, ...]'
    LIMIT 50
)
SELECT coalesce(keyword.id, semantic.id) AS id,
       coalesce(1.0 / (60 + keyword.rank), 0) + coalesce(1.0 / (60 + semantic.rank), 0) AS score
FROM keyword FULL OUTER JOIN semantic ON keyword.id = semantic.id
ORDER BY score DESC
LIMIT 10;

One detail from Chapter 7 bites here. The vector half asks for 50 rows, but an HNSW index returns at most hnsw.ef_search rows, 40 by default. The lab raises ef_search to 100 first. Without that, the vector half quietly returns 40.

Hands-on lab: keywords, vectors, and fusion

You will build BM25 by hand, compare it with vector search question by question, try two ways of fusing the lists, and then run hybrid search as a single SQL query.

Full instructions: download the Vector Databases labs ZIP. If you have not built the shared dataset yet, follow labs/vector-databases/dataset/README.md first. Then open labs/vector-databases/08-hybrid-search and follow its README. For Part 4, start the Chapter 7 container first with docker start vdb-postgres.

The output is the tables in this chapter. BM25 has no randomness, so your keyword numbers should match exactly.

Checkpoint

BM25 found the source paragraph 88% of the time and vector search 59%. Why should you not conclude that BM25 is better in general?

SQuAD's questions were written by people looking at the paragraph, so they reuse its words, which favors keyword matching. The Dense Passage Retrieval authors saw vector search lose on SQuAD while winning on four other datasets. On questions that paraphrase or describe instead of naming, the balance shifts toward vector search. Measure on your own users' questions.

Name two kinds of question that vector search found and BM25 missed in the lab, and why BM25 missed them.

A misspelling ("geneologists") and a different word form ("samurais'" instead of "samurai"). Both are different strings, so a keyword index has nothing to match, while the embedding still lands near the right meaning. Paraphrases, like "the Soviet state" for a name the paragraph spells differently, fail the same way.

Why did PostgreSQL's full-text search return only 0.6 rows per question until the query said "or"?

Its query functions require every word by default. A natural-language question rarely has all of its words in a single paragraph, so most questions matched no rows. Joining the words with "or" lets any word match, and ranking then puts the best matches first.

Check Your Knowledge

Click to start the quiz

1. A product page stuffs the word "refund" into its text 30 times. Your real refund policy, about the same length, says it twice. With BM25 at its default settings, what happens when someone searches "refund"?

The stuffed page scores about 15 times higher, because BM25 counts every occurrenceBoth pages score exactly the same, because BM25 only checks whether a word appearsThe stuffed page scores lower, because BM25 penalizes words that repeatThe stuffed page scores higher, but by much less than 15 times, because k1 makes each extra occurrence add less than the one before

2. A teammate builds hybrid search by adding the raw cosine similarity (between 0.2 and 0.9) to the raw BM25 score (often 12 to 40). What will happen?

BM25 will decide almost every ranking, because its scores are much larger; rescale both first, or fuse by rank with RRFThe two will balance out, because both scores measure relevanceVector search will dominate, because cosine similarity is more preciseNothing goes wrong as long as both lists have the same length

3. On your evaluation set, hybrid search with equal-vote RRF finds the right document less often than keyword search alone. What is the most likely explanation, and the next step?

RRF only works with exactly three lists, so add a third retrieverThe vector retriever is much weaker on this data, and equal votes let its wrong answers pull the right ones down; measure each retriever alone and weight the stronger oneHybrid search never helps, so drop vector searchThe constant 60 is wrong for every dataset and should always be 1

4. Your hybrid SQL query asks the pgvector half for LIMIT 100, but it never returns more than 40 rows. Why?

PostgreSQL limits subqueries to 40 rowsThe full outer join removes the other 60 rowsThe HNSW index returns at most hnsw.ef_search rows, which defaults to 40; raise it to at least the LIMITpgvector can only return 40 rows for cosine distance

What's next

Look at the best row in Part 3 again: found@10 of 0.92, but found@1 of 0.70. Most of the time the right paragraph is in the top 10, just not first. If your application only sends the top three to a language model, rank matters. Chapter 9 adds a second pass that reads each candidate together with the question and reorders them: a reranker. You will measure what it fixes, what it cannot fix, and what it costs in milliseconds.