Hybrid search & retrieval: BM25 plus vectors

Hybrid search & retrieval runs BM25 and dense-vector search over the same index, then combines the two ranked lists with reciprocal rank fusion or a weighted linear combination. RRF uses result positions and needs no score normalization. Linear combination normalizes both score ranges and exposes weights you tune against judgment sets. Filtering and reranking sit on top of either method.

How Hybrid Search & Retrieval Combines BM25 and Vectors

Hybrid search & retrieval runs a lexical BM25 query and a dense-vector semantic query against the same data, then merges the two ranked lists into one result set. Neither branch is a fallback for the other. Each retrieves documents the other misses, which is why the two are run in parallel rather than in sequence.

The workshop that this article is built from was delivered by Jeff Vestal, an AI architect at Elastic, and James Williams, a solutions architect on Elastic's search specialist team, at an AI Engineer event published on 11 October 2026. Elastic is the company behind Elasticsearch, the search and analytics engine that hosts the retrieval layer they describe. Both presenters frame hybrid retrieval as Elastic's core thesis rather than a feature choice: lexical search alone breaks down outside exact keywords and phrases, and semantic search alone breaks down on identifiers that carry no linguistic meaning.

The failure modes are symmetric and easy to reproduce. A query for a specific product ID or an exit code such as 137 has no useful embedding neighborhood, so a semantic retriever returns documents that are topically similar but wrong. A query that rephrases a known concept, such as asking how to configure TLS without using the word TLS, may miss a document that explains exactly that because BM25 scores token overlap rather than meaning.

Elasticsearch exposes the semantic side through a semantic_text field. When you index text into that field, the engine chunks it, sends the chunks to an inference endpoint, and stores the resulting vectors. At query time the same field sends the query text through the same model. The model identity is stored in the field mapping, which removes the most common operational mistake in vector search: indexing with one embedding model and querying with another.

The presenters describe a shift over the past roughly 18 months in which vector data platforms started hosting inference themselves, so embedding generation happens inside the database rather than in an external service the application has to coordinate. Elastic's version is the Elastic Inference Service, which serves the ELSER-family models Elastic gained through an acquisition and now develops in-house, focused on dense-vector embedding and reranking.

Why Vector Search Alone Breaks on Exact Keywords

Dense-vector search fails when the query's meaning depends on a token that has no semantic neighborhood, such as a product ID, a SKU, or a numeric error code. Embeddings map text into a space where similar meanings sit near each other. An arbitrary identifier sits nowhere useful, because nothing about its spelling relates to its purpose.

The presenters use this asymmetry to justify the second lab. After participants see semantic search return plausible but wrong documents for an exact-identifier query, the workshop introduces BM25 lexical retrieval through what is now the lexical query type, which allows BM25-scored fields and vector fields to be queried in a single request.

Recall and ranking are separate problems here. A retriever can return the correct document somewhere in the top 50 and still fail the application if the ranking pushes it past the cutoff that feeds a language model. This distinction matters because the fusion methods later in the workshop operate on ranking, not on whether a document was found at all.

Elastic's documentation notes a related working assumption from the session: chunking defaults matter, and the default for the semantic_text field is 250-token chunks with a sentence of overlap. Chunk size changes both what the embedding represents and how many candidates reach the fusion step.

Reciprocal Rank Fusion vs Linear Combination

Reciprocal rank fusion discards the raw scores from each retrieval branch and combines documents by their rank position, which is why it works without any score normalization. Linear combination keeps the scores, rescales both branches into a common range such as zero to one, and applies weights you can tune.

The scale mismatch is the reason fusion is not trivial. BM25 produces unbounded positive scores, while cosine similarity from a dense-vector search normally falls between zero and one. A weak BM25 match can therefore outscore a strong semantic match if the two score sets are merged naively.

Method	What it combines	Tuning surface	Best first use
Reciprocal rank fusion	Rank positions from each branch	A rank constant	Unknown data and unknown query mix
Linear combination	Normalized scores from each branch	Per-branch weights	You have judgment sets to test against
Reranking on top of either	Query and document jointly	Model choice and candidate count	Order matters within the top few results

The presenters are explicit that RRF is the starting point, not the finish line. RRF works reasonably well without the operator knowing much about the data or the query distribution, which is why the workshop recommends it first. It is also described as an approach Elastic did not invent; the company implements it alongside other retrieval features.

Linear combination is where relevance tuning becomes concrete. You need a judgment set: a list of queries and the documents a human considers correct for each. You then run the queries with different weights and measure how closely the ranking matches the judgments. The workshop treats this as introductory tuning and notes that an authoritative formula for deriving weights from the shape of the data or the queries does not exist yet.

Elastic's reference documentation covers both retrievers under retrievers, including the RRF and linear retrievers and the rank constant that adjusts how quickly rank position decays in RRF scoring.

Filtering Before You Fuse

Filtering narrows the candidate pool before fusion, and a smaller pool tends to produce better results than casting the net across the entire index. Keyword, date, and geolocation filters are the common cases. Elasticsearch applies them quickly because filtering is a first-class part of the query model rather than a post-processing step.

The presenters give a deliberately plain analogy: if the query is food but the intent is pizza, filtering to a restaurant category gets the useful results faster than tuning relevance across every kind of food. The same logic applies to date-bounded or region-bounded queries, where a filter removes whole classes of irrelevant documents that no ranking function would have been able to separate cleanly.

Search templates extend this idea across whole strategies. A template stores a parameterized query body, so an application can hold three strategies (RRF, linear, and a reranked variant) and route each incoming query to one of them. The routing layer can be an LLM with a prompt that classifies the query, which the presenters describe as a practical substitute for automatic relevance tuning. Search templates are not covered in the labs but are available in Elasticsearch.

Reranking: Pointwise vs Listwise Models

Reranking adds a second phase after retrieval in which a model scores each candidate against the query jointly, which changes the order but costs another inference step. The first phase casts a wide net; the second phase reorders a smaller candidate set with a model that sees the query and the document together. Because that scoring happens at query time, the candidate text cannot be pre-embedded in advance.

Pointwise rerankers score each candidate independently, looking only at the query and that single document. Listwise rerankers score the candidate set together, which lets them detect nuance among documents that all appear to be about the same thing. Elastic's ELSER V3 reranker is described in the session as listwise, handling candidate groups that the presenters put at up to 64 documents per evaluation.

The tradeoff is memory and latency. A listwise model has to hold embeddings for the whole candidate group at once. A pointwise model processes candidates one at a time and handles diverse candidate sets well. When several candidates are nearly equivalent in meaning, the listwise view is where the difference shows up.

The decision rule the presenters give is about the latency budget, not about model quality in the abstract. If the first phase returns documents that are good enough and an agent receives a broad candidate set, a reranker may not pay for itself. If the application depends on which chunk lands first, second, or third, because only a few chunks are passed to the model, the extra inference step is where ranking stops being a guess.

This evaluation pattern shows up in agentic search: teams test their retrieval by checking whether the first few chunks contain the answer, which leaves a narrow window for the ranking to be wrong. Passing more context to the model is not a neutral fix, because a larger candidate set can present several answers that look equally plausible.

Where Quantization and Memory Fit

Quantization reduces the memory footprint of the vector index by storing compressed vectors while keeping the original float32 vectors on disk for rescoring. Elastic introduced int8 quantization of the HNSW graph vectors more than two years before the session, and the presenters describe it as the default behavior for storing float32 vectors in memory.

HNSW stands for hierarchical navigable small worlds, the graph structure Elasticsearch uses by default for approximate nearest-neighbor search. The index keeps a graph of vector connections and traverses it to find neighbors quickly, which is what makes sub-second retrieval possible across large collections. Elasticsearch also supports disk-based variants, and the session points to a newer variant the presenters say is proprietary to Elastic.

The memory arithmetic is what makes quantization matter. Storing raw float32 vectors costs four bytes per dimension per vector, so a million 768-dimension vectors occupy roughly 3.07 GB before any graph overhead, and the quantized and disk variants cut that footprint substantially. The presenters tie these reductions to Elastic's target of multi-billion-vector datasets, where per-vector memory overhead decides whether the index fits on the available hardware at all.

The session also returns to a recurring question about graph databases. Vestal and Williams say Elastic is not a graph database and that they have not found graph traversal necessary for the retrieval workloads they target. Their stated alternative for relationship-style reasoning is classification models and named-entity recognition, which run as small models during query processing and extract people, locations, organizations, or product names that an agent can use to build a more targeted query.

Agent Builder and Citing Retrieved Sources

Agent Builder, Elastic's agentic platform, connects the retrieval layer to tools, skills, and workflows so that an agent can search repeatedly and answer with citations back to the documents it used. Skills describe how a task should be performed; tools perform actions such as running a search, listing workflows, or calling an external MCP server.

The demonstration question is concrete: a cluster keeps dying with exit code 137, why does it happen, and how do you change settings to prevent it. That is two questions in one, so the agent runs the skill, searches, discovers that exit code 137 means the Linux out-of-memory killer terminated the process, and then retrieves guidance on adjusting JVM heap settings to prevent the failure. The sources it cites link back to the underlying documents rather than being generated from model memory.

The source-citation behavior is the part worth taking away. A language model does not look up facts in a table; it generates tokens from training. When it does not know a specific answer but knows adjacent material, it can produce a confident and wrong response. Supplying the actual retrieved text and requiring the model to attribute claims to it changes the failure mode, though the presenters are careful to describe the result as more trustworthy rather than guaranteed correct.

Everything configured through the Agent Builder interface is also reachable through APIs, which means a prototype built in the UI can be reused by an application rather than rebuilt. The presenters say the tools in Agent Builder, both built-in and custom, are exposed through an MCP server, so an external agent can call them without Elastic-specific Claude Code.

ESQL and Parameterized Retrieval for Agents

ES|QL is Elastic's piped query language for Elasticsearch, combining SQL-like syntax with a pipe operator that passes results from one stage to the next. It is designed for querying documents, and the pipe structure makes multi-stage retrieval readable in a way that deeply nested JSON query bodies are not.

Two tools matter for agent workflows. An ES|QL tool runs a query an agent writes against a specific index and returns named fields. An index-search tool describes what indices exist, what each one is for, and how fields map, which stops an agent from discovering the schema one exploratory round trip at a time. Those round trips consume tokens, and the presenters describe the index-search tool as a way to cut that cost directly.

The presenters see ESQL as the primary innovation surface for Elastic's search work going forward, and the session mentions federated queries against external data, including Parquet files, as something in development. Nothing announced in the session should be read as a shipped capability, so treat the federated direction as a roadmap statement from the presenters rather than a current feature.

For developers, the practical value in the near term is debugging. Running ESQL by hand while prototyping shows which searches return the documents a use case needs, before that logic is buried in application code.

FAQ

• What is hybrid search and retrieval? Hybrid search and retrieval runs two retrieval branches over the same index, a lexical BM25 branch and a dense-vector semantic branch, then merges their ranked results with a fusion method such as reciprocal rank fusion or a weighted linear combination. The two branches catch different failure modes, so the pairing outperforms either one alone on mixed query workloads.

• When is vector search alone enough without BM25? Dense-vector search is often sufficient when every query is a paraphrase and no query depends on an exact identifier. The presenters still recommend keeping the lexical branch wired in from the start, because adding synonyms and advanced lexical tuning techniques later is more work than running both branches now.

• What is the difference between RRF and linear combination in Elasticsearch? Reciprocal rank fusion merges result lists by rank position and needs no score normalization, which makes it a good default when the data and query mix are unknown. Linear combination normalizes both score ranges and exposes per-branch weights, which you tune against judgment sets once you have them.

• Do I need a reranker on top of hybrid search? Add a reranker only when the order within the top few results changes the outcome and the latency budget allows another inference step. If the first retrieval phase returns good enough documents and the model receives a broad candidate set, the reranker may add latency without changing the answer.

• What is the difference between pointwise and listwise reranking? A pointwise reranker scores each query-document pair independently. A listwise reranker scores the candidate group together, so it can separate documents that are nearly equivalent in meaning. Listwise scoring uses more memory because it holds embeddings for the whole group at once.

• How does Elasticsearch handle chunking for semantic fields? The semantic_text field chunks incoming text automatically, with a default of 250 tokens per chunk and a sentence of overlap. Both values are configurable, and the documentation covers the available settings for teams that need smaller or larger chunks.

• Why does vector search miss product IDs and error codes? Embedding models map text into a space organized by meaning. An arbitrary identifier has no meaningful neighborhood in that space, so the nearest-neighbor graph returns topically similar documents instead of the exact match. BM25 handles those queries because it scores token overlap directly.

• Can an agent cite its sources when it answers from retrieved documents? Yes, when the retrieved documents are supplied to the model and the prompt requires attribution. The model still generates text rather than performing a lookup, so citations make an answer more checkable, not automatically correct. The presenters describe this as the main reason retrieval stays in the architecture even as models improve.

• What does the exit code 137 example demonstrate about retrieval? The agent answers a two-part question about out-of-memory termination and JVM settings by running searches, reading the results, and linking back to the documents it used. The demonstration is about the evidence the agent retrieves, not about the agent's reasoning alone.

Make Retrieval Measurable Before You Make It Clever

The workshop's through line is that retrieval quality is an engineering surface, not a model choice. BM25 and dense vectors fail in different, predictable ways. Fusion decides which failures survive into the candidate set. Filtering, judgment sets, and reranking decide whether the final order matches what the application needs. None of those decisions require a new model. They require knowing what your queries look like and what correct results are.

The same discipline applies to turning any recorded walkthrough into something people can read and search. If you have a workshop, an interview, or a long technical explanation sitting in a YouTube video, Skala Blog takes the video URL, transcribes it, and generates a structured article you can review and publish. The knowledge is already there. The retrieval problem is making it findable.

For developers working through the same material in Portuguese, Gustavo Dev Doido covers this ground in the Crazystack Typescript track, and the Bootcamp do Dev Doido runs the same hybrid retrieval concepts as structured exercises. The Crazystack platform hosts those tracks alongside other stack-specific training.

Source video