Chapter 20 · Capstone 1: Building a Semantic Search Knowledge-Base Q&A (RAG)

Chapter goals: -综合运用 embed / embedMany / cosineSimilarity to build a complete RAG semantic search Q&A system

• Master the code组织 of the three-stage pipeline: "vectorize → similarity retrieval → citation generation"
• Learn to控制 retrieval quality with threshold filtering to avoid低相关片段 polluting the answer
• Throughout, use either AI Gateway or a custom OpenAI-compatible provider;切换只需一处改动

20.1 Project Goal and Technical Approach

We will build a "knowledge-base Q&A bot": given a set of内部文档 fragments, when the user asks a question, the system first 检索s the most relevant fragments, then has the LLM 只基于检索到的内容 answer.

Three-stage architecture:

User question ──▶ [1] embed to vectorize the question
                │
                ▼
          [2] cosineSimilarity compares against each KB entry, sorts, filters
                │
                ▼
          [3] generateText injects Top-K fragments as context into the prompt to generate the answer

This pattern is the classic RAG (Retrieval-Augmented Generation). All data in this chapter lives in an in-memory array; no database needed to run; for production, simply swap the array for a vector database (e.g., pgvector, Pinecone).

20.2 Environment and Provider Setup

Create a new project and install dependencies:

mkdir rag-demo && cd rag-demo && pnpm init
pnpm add ai zod dotenv @ai-sdk/openai-compatible

Create a .env file. Choose one of two access methods:

# Method 1: Vercel AI Gateway (recommended)
AI_GATEWAY_API_KEY=your_gateway_api_key

# Method 2: Custom OpenAI-compatible service
# OPENAI_COMPATIBLE_BASE_URL=http://localhost:11434/v1
# OPENAI_COMPATIBLE_API_KEY=optional_if_not_needed

统一在一个模块里构造模型; subsequent code只依赖 the exported here:

import { createGateway } from 'ai';
import { createOpenAICompatible } from '@ai-sdk/openai-compatible';
import type { LanguageModel, EmbeddingModel } from 'ai';

export function getLanguageModel(): LanguageModel {
  if (process.env.AI_GATEWAY_API_KEY) {
    // Method 1: Vercel AI Gateway
    const gateway = createGateway({
      apiKey: process.env.AI_GATEWAY_API_KEY,
    });
    return gateway('openai/gpt-5');
  }

  // Method 2: Custom OpenAI-compatible
  const myProvider = createOpenAICompatible({
    name: 'my-provider',
    baseURL: process.env.OPENAI_COMPATIBLE_BASE_URL ?? '',
    apiKey: process.env.OPENAI_COMPATIBLE_API_KEY ?? '',
  });
  return myProvider('gpt-4o-mini');
}

export function getEmbeddingModel(): EmbeddingModel {
  if (process.env.AI_GATEWAY_API_KEY) {
    const gateway = createGateway({
      apiKey: process.env.AI_GATEWAY_API_KEY,
    });
    return gateway.embeddingModel('openai/text-embedding-3-small');
  }

  const myProvider = createOpenAICompatible({
    name: 'my-provider',
    baseURL: process.env.OPENAI_COMPATIBLE_BASE_URL ?? '',
    apiKey: process.env.OPENAI_COMPATIBLE_API_KEY ?? '',
  });
  return myProvider.embeddingModel('text-embedding-3-small');
}

20.3 Step One: Vectorize the Knowledge Base

Read documents, chunk them, generate embeddings, and store in an in-memory array:

import { embedMany } from 'ai';
import { getEmbeddingModel } from './provider';

export interface DocumentChunk {
  id: string;
  text: string;
  embedding: number[];
  source: string;
}

export async function ingestDocuments(documents: Array<{ id: string; text: string; source: string }>): Promise<DocumentChunk[]> {
  const CHUNK_SIZE = 500;
  const OVERLAP = 50;

  const chunks: Array<{ id: string; text: string; source: string }> = [];

  for (const doc of documents) {
    const words = doc.text.split(/\s+/);
    for (let i = 0; i < words.length; i += CHUNK_SIZE - OVERLAP) {
      const chunkWords = words.slice(i, i + CHUNK_SIZE);
      if (chunkWords.length >= 50) {
        chunks.push({
          id: `${doc.id}-chunk-${Math.floor(i / (CHUNK_SIZE - OVERLAP))}`,
          text: chunkWords.join(' '),
          source: doc.source,
        });
      }
    }
  }

  const { embeddings } = await embedMany({
    model: getEmbeddingModel(),
    values: chunks.map(c => c.text),
  });

  return chunks.map((chunk, index) => ({
    ...chunk,
    embedding: embeddings[index],
  }));
}

20.4 Step Two: Semantic Retrieval with Threshold Filtering

import { cosineSimilarity, embed } from 'ai';
import { getEmbeddingModel } from './provider';
import type { DocumentChunk } from './knowledge-base';

const SIMILARITY_THRESHOLD = 0.75;
const TOP_K = 5;

export async function retrieveRelevantChunks(
  query: string,
  chunks: DocumentChunk[],
): Promise<DocumentChunk[]> {
  const { embedding } = await embed({
    model: getEmbeddingModel(),
    value: query,
  });

  const scored = chunks
    .map(chunk => ({
      chunk,
      similarity: cosineSimilarity(embedding, chunk.embedding),
    }))
    .filter(({ similarity }) => similarity >= SIMILARITY_THRESHOLD)
    .sort((a, b) => b.similarity - a.similarity)
    .slice(0, TOP_K);

  return scored.map(({ chunk }) => chunk);
}

The SIMILARITY_THRESHOLD is critical:太低的阈值会引入不相关片段污染回答;太高的阈值可能导致无结果. Tune based on your data.

20.5 Step Three: Generate the Answer

import { generateText } from 'ai';
import { getLanguageModel } from './provider';
import { retrieveRelevantChunks } from './retriever';
import type { DocumentChunk } from './knowledge-base';

export async function answerQuestion(
  question: string,
  knowledgeBase: DocumentChunk[],
): Promise<{ answer: string; sources: string[] }> {
  const relevantChunks = await retrieveRelevantChunks(question, knowledgeBase);

  const context = relevantChunks
    .map((chunk, index) => `[Source ${index + 1}]: ${chunk.text}`)
    .join('\n\n');

  const { text } = await generateText({
    model: getLanguageModel(),
    prompt: `Answer the following question based ONLY on the provided context.
If the context does not contain enough information to answer, say "I cannot find relevant information in the knowledge base."

Context:
${context}

Question: ${question}

Please cite your sources by number [1], [2], etc.`,
  });

  const sources = relevantChunks.map(c => c.source);
  return { answer: text, sources };
}

20.6 Full Demo Script

import 'dotenv/config';
import { ingestDocuments } from './knowledge-base';
import { answerQuestion } from './rag-agent';

async function main() {
  // Ingest documents
  const documents = [
    {
      id: 'doc1',
      source: 'product-manual.md',
      text: 'The Widget Pro supports up to 100 simultaneous connections. Setup requires configuring the network settings in the admin panel...',
    },
    {
      id: 'doc2',
      source: 'faq.md',
      text: 'Common issues: 1) Connection timeout - check your firewall settings. 2) Authentication failed - verify your API key...',
    },
  ];

  const knowledgeBase = await ingestDocuments(documents);
  console.log(`Ingested ${knowledgeBase.length} chunks`);

  // Ask a question
  const { answer, sources } = await answerQuestion(
    'How many simultaneous connections does Widget Pro support?',
    knowledgeBase,
  );

  console.log('Answer:', answer);
  console.log('Sources:', sources);
}

main().catch(console.error);

20.7 Production Considerations

• Vector database: Swap the in-memory array for pgvector, Pinecone, Weaviate, etc.
• Embedding model choice: Use a model with sufficient dimensionality for your data; text-embedding-3-small is a good default.
• Chunking strategy: Adjust CHUNK_SIZE and OVERLAP based on your document structure; semantic boundaries > fixedtoken counts.
• Caching: Cache embedding results for unchanged documents to save成本.
• Reranking: After initial retrieval, use areranker model to re-sort top candidates for higher precision.

20.8 Chapter Summary

• RAG three-stage pipeline: embed question → cosine similarity检索 → generate with context;
• SIMILARITY_THRESHOLD filterslow-relevance chunks; tune for your data;
• Unified provider模式: getLanguageModel() and getEmbeddingModel() switch between AI Gateway and自定义 provider with one change;
• Citation via source numbers in the prompt; the LLM references [1], [2] in the answer.

🧪 随堂测验

点击你认为正确的选项。答错时会展示正确答案与原因解析。

1. What is the role of SIMILARITY_THRESHOLD in RAG retrieval?

2. Why does the unified provider pattern only require one change to switch models?

3. What is the purpose of citing sources by number in the RAG prompt?

4. Which is the recommended production替换 for the in-memory document array?

🛠️ Hands-On Exercises

1. Extend the demo with a third document; verify the threshold filtering correctly excludes低相关 chunks.
2. Implement a simple reranker: after the initial cosine similarity retrieval, re-score the top-10 candidates with a second embedding model and return the top-3.
3. Add embedding caching: store已 embed chunks in a Map keyed by text hash; skip embedding for unchanged documents on re-run.

Capstone 1 complete. Next: Capstone 2: Multi-Agent Customer Ticket System.