reranker

A reranker is a model that reorders an initial set of retrieved documents by how relevant each one is to a query, sharpening the results before they reach the application that will use them. It forms the second stage of a two-stage retrieval pipeline, most often within retrieval-augmented generation (RAG). In a RAG system, that application is a large language model, so the passages that survive the reordering are the ones that shape its answer.

The first stage uses a fast retriever, typically an embedding model over a vector database, to pull a broad candidate set with approximate nearest-neighbor search tuned for recall, favoring catching every relevant document over being selective. The reranker then scores each query-document pair for precision and sorts the candidates so the most relevant passages rise to the top, as this pipeline shows:

Most rerankers are cross-encoders: they pass the query and a document through a transformer together, so the two texts attend to each other and produce a single relevance score. This is more accurate than bi-encoder retrieval, which embeds each text separately, but far more expensive, because nothing can be precomputed.

That’s why a reranker runs only over the small candidate set from the first stage, not the whole document collection being searched. Hosted services such as Cohere Rerank and open models like the BGE reranker provide ready-made options.

Related Resources

Tutorial

Embeddings and Vector Databases With ChromaDB

Vector databases are a crucial component of many NLP applications. This tutorial will give you hands-on experience with ChromaDB, an open-source vector database that's quickly gaining traction. Along the way, you'll learn what's needed to understand vector databases with practical examples.

advanced ai databases data-science machine-learning

For additional information on related topics, take a look at the following resources:

• Vector Databases and Embeddings With ChromaDB (Course)
• Hugging Face Transformers: Leverage Open-Source AI in Python (Tutorial)
• How to Use Ollama to Run Large Language Models Locally (Tutorial)
• Natural Language Processing With spaCy in Python (Tutorial)
• Natural Language Processing With Python's NLTK Package (Tutorial)
• Embeddings and Vector Databases With ChromaDB (Quiz)
• Vector Databases and Embeddings With ChromaDB (Quiz)
• Hugging Face Transformers (Quiz)
• How to Use Ollama to Run Large Language Models Locally (Quiz)
• Natural Language Processing With Python's NLTK Package (Quiz)