Chapter 16 · Cross-Provider Handoffs

Chapter goals: Master the mechanism for switching different LLM providers mid-session, and understand the automatic conversion rules for thinking blocks and tool calls during context migration.

16.1 Why Handoffs Are Needed

A common requirement in real products: use a cheap model (Gemini Flash) for chitchat, and seamlessly switch to a strong model (Claude Sonnet) when the user asks a tough question; or fall back to a backup when the primary model is rate-limited or down. The difficulty lies in — each provider has different format requirements for historical messages; feeding Claude's messages directly to GPT will error.

pi-ai has a built-in cross-provider message conversion layer, making handoffs as simple as swapping a model object.

16.2 Registering Multiple Providers

First register all providers you might need — setProvider() can be called consecutively without overwriting each other:

import { createModels, type Context } from '@earendil-works/pi-ai';
import { anthropicProvider } from '@earendil-works/pi-ai/providers/anthropic';
import { openaiProvider } from '@earendil-works/pi-ai/providers/openai';
import { googleProvider } from '@earendil-works/pi-ai/providers/google';

const models = createModels();
models.setProvider(anthropicProvider());
models.setProvider(openaiProvider());
models.setProvider(googleProvider());

const context: Context = { messages: [] };

16.3 Complete Example of Mid-Round Switching

The same context runs throughout; the only variable is the model passed to each complete:

// Round 1: answer with Claude
const claude = models.getModel('anthropic', 'claude-sonnet-4-5')!;
context.messages.push({ role: 'user', content: 'What is 25 * 18?', timestamp: Date.now() });
context.messages.push(
  await models.completeSimple(claude, context, { reasoning: 'medium' })
);

// Round 2: switch to GPT — it will see Claude's thinking converted to <thinking> tags
const gpt5 = models.getModel('openai', 'gpt-5-mini')!;
context.messages.push({ role: 'user', content: 'Is that calculation correct?', timestamp: Date.now() });
context.messages.push(await models.complete(gpt5, context));

// Round 3: switch to Gemini
const gemini = models.getModel('google', 'gemini-2.5-flash')!;
context.messages.push({ role: 'user', content: 'What was the original question?', timestamp: Date.now() });
const geminiResponse = await models.completeSimple(gemini, context);

All three calls share the complete context — GPT knows what Claude calculated; Gemini knows everything from both previous rounds.

16.4 Message Conversion Rules

When messages migrate from provider A to provider B, the library自动 executes the following conversions:

Message Type	Cross-Provider Handling
User messages / Tool results	Passed through unchanged
Assistant messages from the same provider	Kept as-is
Assistant messages from different providers	Thinking blocks converted to <thinking> tag-wrapped text
Tool calls and plain text	Kept as-is

// Conversion illustration: Claude's thinking block becomes this in GPT's eyes:
// <thinking>Need to inspect package metadata first.</thinking>

Thinking blocks degrade to text

Cross-provider, thinking content is not lost, but degrades from structured blocks to tag-wrapped plain text. The new model sees the "previous reasoning" as text, not its own native reasoning format.

16.5 In Practice: Smart Fallback Routing

Put handoffs to work in production — dynamically select models by task complexity; auto-fallback on failure:

// Smart routing: strong model for complex questions, cheaper for others; auto-fallback on failure
async function smartComplete(models: Models, context: Context, complex: boolean) {
  const chain = complex
    ? ['anthropic:claude-sonnet-4-5', 'openai:gpt-5-mini']   // Strong model first
    : ['google:gemini-2.5-flash', 'anthropic:claude-haiku']; // Cheap first

  for (const spec of chain) {
    const [provider, id] = spec.split(':');
    const model = models.getModel(provider, id);
    if (!model) continue;

    const res = await models.completeSimple(model, context);
    if (res.stopReason !== 'error') {
      context.messages.push(res); // Success: write to context for subsequent handoffs
      return res;
    }
    console.warn(`${spec} failed, falling back to the next model`);
  }
  throw new Error('All models are unavailable');
}

Chapter Summary

• Handoffs let you switch providers mid-session with the full context preserved;
• After registering multiple providers, switching is as simple as changing the model parameter;
• Conversion rules: user/toolResult pass through; same-provider assistants kept; cross-provider thinking blocks converted to <thinking> text; tool calls kept;
• Thinking blocks cross-vendor degrade to text form — not lost but format changes;
• Typical application: difficulty-tiered routing + failure auto-fallback chains.

🧪 随堂测验

点击你认为正确的选项。答错时会展示正确答案与原因解析。

1. What does a thinking block generated by Claude become after the session switches to GPT?

2. What happens to tool calls and tool results when passed across providers?

3. What is the correct way to register multiple providers?

4. When building a "failure fallback chain", what is the reliable basis for judging a generation failure?

🛠️ Hands-on Practice

1. Implement a three-way conversation: Claude poses a question → GPT answers → Gemini comments; share the same context throughout.
2. Write a test to verify that after a handoff, the new model indeed sees <thinking> tag text (combined with Faux).
3. Add a fallback chain to DeepSeek Provider from Chapter 14: when DeepSeek fails, automatically switch to OpenAI.

Next we solve the long-session memory problem — Chapter 17 · Session Persistence.