📚 AI 工程与智能体教程站

English

简体中文

English

简体中文

Chapter 8 · Semantic Search ​

Chapter goals: Master the complete semantic search pipeline — text chunking, embeddings, vector indexing, BM25 keyword retrieval, reranking, and finally assembling a RAG (Retrieval-Augmented Generation) pipeline.

8.1 Preparation ​

The official notebook corresponding to this chapter is chapter08/Chapter 8 - Semantic Search.ipynb.

If you are viewing the notebook on Google Colab (or any other cloud environment), you need to uncomment and run the code block below to install the dependencies for this chapter:

python
# %%capture
# !pip install langchain==0.2.5 faiss-cpu==1.8.0 cohere==5.5.8 langchain-community==0.2.5 rank_bm25==0.2.2 sentence-transformers==3.0.1
# !pip install llama-cpp-python==0.2.78  --extra-index-url https://abetlen.github.io/llama-cpp-python/whl/cu124

## IMPORTANT: Make sure to restart the session after installing the packages above.

💡 NOTE: Running the examples in this chapter requires a GPU. In Google Colab, go to Runtime > Change runtime type > Hardware accelerator > GPU > GPU type > T4.

8.2 Dense Retrieval Example ​

8.2.1 Fetching and Chunking the Text Corpus ​

First prepare the Cohere client (you need to apply for an API key yourself):

python
import cohere

# Paste your API key here. Remember to not share publicly
api_key = ''

# Create and retrieve a Cohere API key from os.cohere.ai
co = cohere.Client(api_key)

Prepare a text corpus about the movie Interstellar and split it into sentence chunks:

python
text = """
Interstellar is a 2014 epic science fiction film co-written, directed, and produced by Christopher Nolan.
It stars Matthew McConaughey, Anne Hathaway, Jessica Chastain, Bill Irwin, Ellen Burstyn, Matt Damon, and Michael Caine.
Set in a dystopian future where humanity is struggling to survive, the film follows a group of astronauts who travel through a wormhole near Saturn in search of a new home for mankind.

Brothers Christopher and Jonathan Nolan wrote the screenplay, which had its origins in a script Jonathan developed in 2007.
Caltech theoretical physicist and 2017 Nobel laureate in Physics[4] Kip Thorne was an executive producer, acted as a scientific consultant, and wrote a tie-in book, The Science of Interstellar.
Cinematographer Hoyte van Hoytema shot it on 35 mm movie film in the Panavision anamorphic format and IMAX 70 mm.
Principal photography began in late 2013 and took place in Alberta, Iceland, and Los Angeles.
Interstellar uses extensive practical and miniature effects and the company Double Negative created additional digital effects.

Interstellar premiered on October 26, 2014, in Los Angeles.
In the United States, it was first released on film stock, expanding to venues using digital projectors.
The film had a worldwide gross over $677 million (and $773 million with subsequent re-releases), making it the tenth-highest grossing film of 2014.
It received acclaim for its performances, direction, screenplay, musical score, visual effects, ambition, themes, and emotional weight.
It has also received praise from many astronomers for its scientific accuracy and portrayal of theoretical astrophysics. Since its premiere, Interstellar gained a cult following,[5] and now is regarded by many sci-fi experts as one of the best science-fiction films of all time.
Interstellar was nominated for five awards at the 87th Academy Awards, winning Best Visual Effects, and received numerous other accolades"""

# Split into a list of sentences
texts = text.split('.')

# Clean up to remove empty spaces and new lines
texts = [t.strip(' \n') for t in texts]

8.2.2 Vectorizing Text Chunks ​

Call co.embed to convert each sentence chunk into an embedding vector (note: documents use the search_document type):

python
import numpy as np

# Get the embeddings
response = co.embed(
  texts=texts,
  input_type="search_document",
).embeddings

embeds = np.array(response)
print(embeds.shape)

8.2.3 Building the Retrieval Index ​

Build a flat L2-distance vector index with FAISS:

python
import faiss

dim = embeds.shape[1]
index = faiss.IndexFlatL2(dim)
index.add(np.float32(embeds))

8.2.4 Querying the Index ​

Wrap a complete retrieval function: query vectorization → nearest-neighbor search → formatted output:

python
import pandas as pd

def search(query, number_of_results=3):

  # 1. Get the query's embedding
  query_embed = co.embed(texts=[query],
                input_type="search_query",).embeddings[0]

  # 2. Retrieve the nearest neighbors
  distances , similar_item_ids = index.search(np.float32([query_embed]), number_of_results)

  # 3. Format the results
  texts_np = np.array(texts) # Convert texts list to numpy for easier indexing
  results = pd.DataFrame(data={'texts': texts_np[similar_item_ids[0]],
                              'distance': distances[0]})

  # 4. Print and return the results
  print(f"Query:'{query}'\nNearest neighbors:")
  return results

Test the query "how precise was the science":

python
query = "how precise was the science"
results = search(query)
results

8.2.5 BM25 Keyword Retrieval ​

Beyond dense retrieval, traditional frequency-based BM25 lexical retrieval remains an important baseline. First define the tokenizer:

python
from rank_bm25 import BM25Okapi
from sklearn.feature_extraction import _stop_words
import string

def bm25_tokenizer(text):
    tokenized_doc = []
    for token in text.lower().split():
        token = token.strip(string.punctuation)

        if len(token) > 0 and token not in _stop_words.ENGLISH_STOP_WORDS:
            tokenized_doc.append(token)
    return tokenized_doc

Tokenize the entire corpus and build the BM25 index:

python
from tqdm import tqdm

tokenized_corpus = []
for passage in tqdm(texts):
    tokenized_corpus.append(bm25_tokenizer(passage))

bm25 = BM25Okapi(tokenized_corpus)

Implement the keyword retrieval function:

python
def keyword_search(query, top_k=3, num_candidates=15):
    print("Input question:", query)

    ##### BM25 search (lexical search) #####
    bm25_scores = bm25.get_scores(bm25_tokenizer(query))
    top_n = np.argpartition(bm25_scores, -num_candidates)[-num_candidates:]
    bm25_hits = [{'corpus_id': idx, 'score': bm25_scores[idx]} for idx in top_n]
    bm25_hits = sorted(bm25_hits, key=lambda x: x['score'], reverse=True)

    print(f"Top-3 lexical search (BM25) hits")
    for hit in bm25_hits[0:top_k]:
        print("\t{:.3f}\t{}".format(hit['score'], texts[hit['corpus_id']].replace("\n", " ")))
python
keyword_search(query = "how precise was the science")

8.2.6 Caveats of Dense Retrieval ​

Dense retrieval is not a panacea — try a query that shares no wording with the corpus at all (the mass of the moon) and see what happens:

python
query = "What is the mass of the moon?"
results = search(query)
results

8.3 Reranking Example ​

Reranking is a key technique for improving retrieval quality: first cast a wide net to collect candidates, then rerank with a stronger model. Call Cohere's Rerank API directly:

python
query = "how precise was the science"
results = co.rerank(query=query, documents=texts, top_n=3, return_documents=True)
results.results

Inspect the relevance score for each result:

python
for idx, result in enumerate(results.results):
    print(idx, result.relevance_score , result.document.text)

Combine BM25 keyword retrieval with reranking into a hybrid retrieval scheme:

python
def keyword_and_reranking_search(query, top_k=3, num_candidates=10):
    print("Input question:", query)

    ##### BM25 search (lexical search) #####
    bm25_scores = bm25.get_scores(bm25_tokenizer(query))
    top_n = np.argpartition(bm25_scores, -num_candidates)[-num_candidates:]
    bm25_hits = [{'corpus_id': idx, 'score': bm25_scores[idx]} for idx in top_n]
    bm25_hits = sorted(bm25_hits, key=lambda x: x['score'], reverse=True)

    print(f"Top-3 lexical search (BM25) hits")
    for hit in bm25_hits[0:top_k]:
        print("\t{:.3f}\t{}".format(hit['score'], texts[hit['corpus_id']].replace("\n", " ")))

    #Add re-ranking
    docs = [texts[hit['corpus_id']] for hit in bm25_hits]

    print(f"\nTop-3 hits by rank-API ({len(bm25_hits)} BM25 hits re-ranked)")
    results = co.rerank(query=query, documents=docs, top_n=top_k, return_documents=True)
    for hit in results.results:
        print("\t{:.3f}\t{}".format(hit.relevance_score, hit.document.text.replace("\n", " ")))
python
keyword_and_reranking_search(query = "how precise was the science")

8.4 Retrieval-Augmented Generation ​

8.4.1 Example: Grounded Generation Using the LLM API ​

The complete RAG flow: retrieve relevant documents → hand them to the LLM to generate an evidence-based answer.

python
query = "income generated"

# 1- Retrieval
# We'll use embedding search. But ideally we'd do hybrid
results = search(query)

# 2- Grounded Generation
docs_dict = [{'text': text} for text in results['texts']]
response = co.chat(
    message = query,
    documents=docs_dict
)

print(response.text)

The response object also contains citation information:

python
response
python
response.citations

8.4.2 Example: Local RAG Using a Local Model ​

You can also build a complete RAG pipeline without relying on a cloud API.

Loading the generative model — download the quantized GGUF weights for Phi-3:

python
!wget https://huggingface.co/microsoft/Phi-3-mini-4k-instruct-gguf/resolve/main/Phi-3-mini-4k-instruct-q4.gguf
python
from langchain import LlamaCpp

# Make sure the model path is correct for your system!
llm = LlamaCpp(
    model_path="Phi-3-mini-4k-instruct-q4.gguf",
    n_gpu_layers=-1,
    max_tokens=500,
    n_ctx=2048,
    seed=42,
    verbose=False
)

Loading the embedding model:

python
from langchain.embeddings.huggingface import HuggingFaceEmbeddings

# Embedding Model for converting text to numerical representations
embedding_model = HuggingFaceEmbeddings(
    model_name='BAAI/bge-small-en-v1.5'
)

Preparing the vector database:

python
from langchain.vectorstores import FAISS

# Create a local vector database
db = FAISS.from_texts(texts, embedding_model)

Writing the RAG prompt and wiring all components into a RetrievalQA pipeline:

python
from langchain import PromptTemplate
from langchain.chains import RetrievalQA


# Create a prompt template
template = """<|user|>
Relevant information:
{context}

Provide a concise answer the following question using the relevant information provided above:
{question}<|end|>
<|assistant|>"""
prompt = PromptTemplate(
    template=template,
    input_variables=["context", "question"]
)

# RAG Pipeline
rag = RetrievalQA.from_chain_type(
    llm=llm,
    chain_type='stuff',
    retriever=db.as_retriever(),
    chain_type_kwargs={
        "prompt": prompt
    },
    verbose=True
)
python
rag.invoke('Income generated')

8.5 Chapter Summary ​

  • The semantic search four-step pipeline: text chunking → embedding vectorization → FAISS vector index → nearest-neighbor retrieval; documents and queries use two different embedding types: search_document and search_query;
  • BM25 is the classic lexical-retrieval baseline based on term frequency, complementary to dense retrieval — when the query shares no wording with the corpus, dense retrieval may fail;
  • Reranking adopts a "wide recall + fine rank" strategy: BM25 first collects candidates, then the Rerank API reranks them by relevance;
  • RAG = retrieval + grounded generation: the cloud API lets you pass documents directly via co.chat to get cited answers; locally you combine LlamaCpp with the bge-small embedding model through LangChain's FAISS vector store + RetrievalQA pipeline.

🧪 随堂测验

点击你认为正确的选项。答错时会展示正确答案与原因解析。

1. When calling co.embed, which input_type should documents and queries use respectively?

2. The poor performance of the query "What is the mass of the moon?" in dense retrieval illustrates what?

3. What is the flow of the BM25 + Reranking hybrid scheme?

4. What is the correct component combination for the local RAG pipeline in Section 8.4.2?

🛠️ Hands-on Practice ​

  1. Change the chunking strategy in Section 8.2.1 from "split by period" to "split by paragraph" (separated by \n\n); rebuild the embeddings and FAISS index; compare the differing return results for the query "how precise was the science".
  2. Run the failing query "What is the mass of the moon?" through dense retrieval, BM25 keyword retrieval, and the hybrid reranking retrieval from Section 8.3; compile a comparison table of the Top-3 results from all three methods and explain the differences.
  3. Following the local RAG pipeline in Section 8.4.2, swap the embedding model to 'sentence-transformers/all-MiniLM-L6-v2', rebuild the vector store, then call rag.invoke('Income generated') and rag.invoke('Who directed Interstellar?') respectively; observe whether the answer quality changes.