Chapter 9 · transformContext: Context Transformation

Chapter goals: Master the complete order of the message processing pipeline, and use transformContext to implement history truncation and context compaction strategies.

9.1 The Complete Pipeline

Before every LLM call, messages go through two stages of processing:

AgentMessage[]
      │
      ▼
transformContext(messages, signal)   ← Async; can truncate / compact / inject
      │  Returns AgentMessage[] (application-layer form)
      ▼
convertToLlm(messages)               ← Filter custom types, convert to standard format
      │  Returns Message[] (LLM-layer form)
      ▼
    LLM request

Their responsibilities differ: transformContext handles "how much to show the model" (the quantity problem); convertToLlm handles "in what form to show it" (the shape problem).

9.2 Basic Usage: Keep the Last N Rounds

const agent = new Agent({
  streamFn: models.streamSimple.bind(models),
  initialState: { systemPrompt: "...", model },
  // Async signature; accepts an AbortSignal
  transformContext: async (messages, signal) => {
    const MAX = 40; // Keep the last 40 messages
    if (messages.length <= MAX) return messages;

    // Simple sliding window: discard the oldest messages
    // Note: toolResult cannot exist independently without its paired assistant tool call;
    // the truncation point must avoid the middle of "tool pairs"
    return messages.slice(-MAX);
  },
});

Truncation pitfall: tool pair integrity

An assistant message may contain a toolCall, and it must always be followed by a corresponding toolResult. If you cut from the middle, the LLM will receive an "orphaned tool result" and error. The safe approach is to cut at turn boundaries (at user messages).

9.3 Safe Truncation by Turn Boundaries

// Group by user messages as boundaries, then keep the last N complete turns
function truncateByTurns(messages, maxTurns = 10) {
  // Find the index of each user message as a turn start point
  const starts = messages
    .map((m, i) => (m.role === "user" ? i : -1))
    .filter((i) => i >= 0);

  if (starts.length <= maxTurns) return messages;
  const cutFrom = starts[starts.length - maxTurns];
  return messages.slice(cutFrom);
}

const agent2 = new Agent({
  transformContext: async (msgs) => truncateByTurns(msgs, 8),
  initialState: { systemPrompt: "...", model },
  streamFn: models.streamSimple.bind(models),
});

9.4 Advanced: Compaction Strategy

What is smarter than discarding is summary replacement — compress old conversations into a summary:

// Concept sketch: when exceeding a threshold, hand the old turns to a cheaper model to compress
transformContext: async (messages) => {
  if (estimateTokens(messages) < 50_000) return messages;

  const [oldPart, recentPart] =
    splitAtTurnBoundary(messages, keepRecentTurns = 6);

  // Use a small model to generate a summary (reusing the same models collection)
  const summaryMsg = await models.completeSimple(cheapModel, {
    systemPrompt: "Compress the following conversation into key points, preserving key facts and unfinished tasks.",
    messages: toLlmShape(oldPart),
  });

  // Inject the summary as a user message at the beginning
  return [
    {
      role: "user",
      content: `[History Summary] ${summaryMsg.content[0].text}`,
      timestamp: Date.now(),
    },
    ...recentPart,
  ];
},

A production-grade implementation should also cache summary results (compress the same batch of old messages only once) and pair with the shouldStopAfterTurn hook to make compaction decisions between turns.

9.5 Other Uses: Injecting External Context

// Besides trimming, you can also inject: e.g. real-time retrieved materials
transformContext: async (messages, signal) => {
  const lastUser = [...messages].reverse().find((m) => m.role === "user");
  const docs = await retrieve(lastUser?.content ?? "", signal); // RAG retrieval

  return [
    ...messages,
    {
      role: "user",
      content: `[Reference Materials]\n${docs}`,
      timestamp: Date.now(),
    },
  ];
};

9.6 Chapter Summary

• Pipeline order is fixed: transformContext → convertToLlm → LLM;
• transformContext solves "quantity", convertToLlm solves "shape";
• Truncation must happen at turn boundaries to avoid breaking the assistant(toolCall) + toolResult pair;
• Compaction beats discarding; pair with shouldStopAfterTurn for graceful execution between turns;
• Injecting RAG materials and dynamic system context are also legitimate uses.

🧪 随堂测验

点击你认为正确的选项。答错时会展示正确答案与原因解析。

1. How do transformContext and convertToLlm divide responsibilities?

2. Why can a simple slice-by-count truncation cause problems?

3. Compared to directly discarding old messages, what is the core advantage of the "compaction" strategy?

4. What is the function signature of transformContext?

🛠️ Hands-on Practice

1. Implement a combined strategy of "keep the last 8 turns + compress older content into a summary" and measure token changes with a long conversation.
2. Add caching to your compaction logic: when the same chunk of old messages triggers a second time, reuse the existing summary directly.
3. Use transformContext to inject the current date and user timezone, and verify the model's correctness when answering "what date is today?"

Next Chapter 10: Where do API keys come from? Deep dive into auth resolution and credential management.