Chapter 6 · Generation Settings and Lifecycle Callbacks

Chapter goals:

• Master the meaning and tradeoffs of language model call options like maxOutputTokens, temperature, topP
• Distinguish Call Options (affecting generation behavior) from Request Options (affecting transport behavior)
• Learn to设置 timeouts via the timeout object for total duration, per-step, first-chunk, and tool execution
• Master the execution order and application scenarios of lifecycle callbacks like onStart / onStepStart / onLanguageModelCallEnd / onToolExecutionEnd

6.1 Common Settings Overview

💡 The model variable in this chapter's examples is constructed per Chapter 3: either createGateway() (Vercel AI Gateway) or createOpenAICompatible() (custom OpenAI-compatible Provider); the settings are provider-agnostic.

Large language models (LLMs) typically expose一些 settings to adjust output. Beyond model, prompt, and provider-specific settings, all AI SDK functions support the following common settings:

const result = await generateText({
  model,
  maxOutputTokens: 512,
  temperature: 0.3,
  maxRetries: 5,
  timeout: 10000,
  prompt: 'Invent a new holiday and describe its traditions.',
});

⚠️ Some providers do not support all通用 settings. Using an unsupported setting produces a warning, viewable via the result object's warnings property.

These settings are divided into two categories:

• Language Model Call Options: affect how the model generates the response (token上限, sampling behavior, penalties, stop sequences, seed, reasoning)
• Request Options: affect the transport layer — retries, cancellation, timeouts; they do not affect generation behavior itself

6.2 Language Model Call Options

maxOutputTokens

The maximum number of tokens to generate.

temperature

The temperature setting. The value is passed through to the provider; the range depends on the provider and model. For most providers, 0 means the result is nearly deterministic; higher values increase randomness. Recommend setting either temperature or topP, not both.

💡 As of AI SDK 5.0, temperature no longer defaults to 0.

topP

Nucleus sampling. For most providers it is a value between 0 and 1; for example, 0.1 means only considering the top 10% of tokens by probability mass. Also recommends choosing one of temperature or topP.

topK

Only sample from the K highest-probability options for each subsequent token, used to trim "long-tail" low-probability responses. Only recommended for advanced scenarios; usually temperature alone suffices.

presencePenalty and frequencyPenalty

• presencePenalty: affects the model's tendency to repeat information already in the prompt
• frequencyPenalty: affects the model's tendency to重复 use the same words or phrases

For most providers, 0 means no penalty; the range depends on the specific provider and model.

stopSequences

Stop sequences. When set, the model stops generating text upon generating any of the sequences. The provider may limit the number of stop sequences.

seed

A seed (integer) for random sampling. If set and the model supports it, the call produces deterministic results.

reasoning

Controls how much reasoning the model performs before answering:

Value	Behavior
'provider-default'	Use the provider's default reasoning behavior (the default when omitted)
'none'	Disable reasoning
'minimal'	Minimal reasoning
'low'	Quick, concise reasoning
'medium'	Balanced reasoning
'high'	Thorough reasoning
'xhigh'	Maximum reasoning

6.3 Request Options

maxRetries

Maximum number of自动 retries on failure. Defaults to 2.

timeout

Timeout configuration. Supports granular timeout settings:

const result = await generateText({
  model,
  prompt: '...',
  timeout: {
    // Total timeout for the entire generation
    total: 30000,
    // Timeout per step
    perStep: 10000,
    // Timeout until the first chunk arrives
    firstChunk: 5000,
    // Timeout for each tool execution
    toolExecution: 5000,
  },
});

signal

Pass an AbortSignal to cancel the generation:

const controller = new AbortController();

setTimeout(() => controller.abort(), 5000);

const result = await generateText({
  model,
  prompt: '...',
  signal: controller.signal,
});

6.4 Lifecycle Callbacks

onStart

Called when generation starts:

const result = await streamText({
  model,
  prompt: '...',
  onStart: ({ snapshot }) => {
    console.log('Started with', snapshot.steps.length, 'previous steps');
  },
});

onStepStart / onStepEnd

Called before and after each step (a step = one model call or one tool execution):

const result = await streamText({
  model,
  prompt: '...',
  onStepStart: ({ step }) => {
    console.log('Step started:', step.stepType);
  },
  onStepEnd: ({ step, finishReason, usage }) => {
    console.log('Step ended:', finishReason, 'Tokens:', usage.totalTokens);
  },
});

onLanguageModelCallEnd

Called after each model call:

const result = await streamText({
  model,
  prompt: '...',
  onLanguageModelCallEnd: ({ model, input, response }) => {
    console.log('Model call finished:', model.modelId);
    console.log('Input tokens:', input.options?.maxTokens);
  },
});

onToolExecutionEnd

Called after each tool execution:

const result = await streamText({
  model,
  prompt: '...',
  tools: {
    getWeather: tool({ /* ... */ }),
  },
  onToolExecutionEnd: ({ toolName, result: toolResult }) => {
    console.log('Tool execution finished:', toolName);
  },
});

6.5 Chapter Summary

• Call options (maxOutputTokens, temperature, topP, etc.) affect generation behavior; request options (maxRetries, timeout, signal) affect transport;
• timeout supports granular settings: total, perStep, firstChunk, toolExecution;
• Lifecycle callbacks cover the full generation lifecycle: onStart → onStepStart → onLanguageModelCallEnd / onToolExecutionEnd → onStepEnd → onEnd;
• Use callbacks for logging, telemetry, and real-time monitoring.

🧪 随堂测验

点击你认为正确的选项。答错时会展示正确答案与原因解析。

1. What is the difference between maxOutputTokens and maxRetries?

2. When should you use the perStep timeout vs. the total timeout?

3. Which callback is called after each tool execution completes?

4. What happens if you set both temperature and topP?

🛠️ Hands-On Exercises

1. Experiment with different temperature values (0, 0.3, 0.7, 1.0) on the same prompt; observe the output diversity.
2. Set a perStep timeout of 3 seconds and a total timeout of 10 seconds; trigger a slow tool and verify the timeout behavior.
3. Add onStepStart and onStepEnd callbacks to log each step's duration; calculate the total generation time vs. the sum of per-step times.

Next: Chapter 7 · Structured Output