Chapter 16 · Observability

Chapter goals: Master Mastra's tracing, logging, and monitoring system; locate Agent behavior anomalies in production.

16.1 Why Agents Need Observability

An Agent's behavior depends on the combination of prompts, context, and tool return values — "looking at code" is far from enough when things go wrong. Mastra's observability system lets you see, for every run:

• Each agent run, workflow step, tool call, model interaction;
• Input/output, latency, token usage, and errors for each环节.

This data is persisted by the Storage domain and可回放 in Studio.

16.2 Enabling Tracing

// src/mastra/index.ts
import { Mastra } from '@mastra/core'

export const mastra = new Mastra({
  agents: { support: supportAgent },
  observability: {
    // Default exporter writes spans to configured storage;
    // swap for OTLP exporter to Langfuse/Jaeger etc.
    exporters: [
      { type: 'otlp', endpoint: process.env.OTLP_ENDPOINT },
    ],
  },
})

After enabling, each call auto-generates a trace tree:

// Typical span hierarchy from one generate call
// trace: agent.run
//   ├── span: agent.generate          total latency/token stats
//   │     ├── span: llm.request       model request and response
//   │     ├── span: tool.execute:search   tool execution details
//   │     └── span: llm.request       follow-up request after tool result

16.3 Structured Logging

// src/mastra/logger.ts
import { Logger } from '@mastra/core'

export const logger = new Logger({
  level: 'info',
  // Structured JSON output for collection into ELK/Datadog
})

logger.info('Order query complete', { orderId: 'A-1001', latencyMs: 320 })

Instrument inside tools to give each execution business context:

import { createTool } from '@mastra/core/tools'
import { z } from 'zod'
import { logger } from '../logger'

export const queryOrder = createTool({
  id: 'query-order',
  description: 'Query order status by order ID',
  inputSchema: z.object({ orderId: z.string() }),
  execute: async ({ context }) => {
    logger.info('Starting order query', { orderId: context.orderId })
    // ...query logic
    return { status: 'shipped' }
  },
})

16.4 Production Debugging Patterns

Symptom	First Check
Answering off-topic	Check the trace for correct system prompt and检索 content injection
Tool not triggered	Verify the tool description is clear and schema matches
Latency spike	Sort spans by latency; usually one LLM request or slow SQL
Cost doubling	Track the token field; check for repeated large context injection

16.5 Recommended Monitoring Metrics

• Quality: eval scores (Chapter 15), user negative feedback rate;
• Performance: P95 first-token latency, average tokens per round;
• Stability: tool error rate, workflow suspend stall duration.

16.6 Chapter Summary

• Mastra auto-generates trace spans for run/step/tool/model layers and persists to storage;
• OTLP exporter可对接第三方平台 (Langfuse/Jaeger etc.);
• Structured logs should carry业务 fields for trace correlation;
• Debug顺序: prompt injection → tool descriptions → slowest span.

🧪 随堂测验

点击你认为正确的选项。答错时会展示正确答案与原因解析。

1. What does one agent call produce in Mastra's observability system?

2. To send trace data to Langfuse or Jaeger, what should you configure?

3. The agent never calls a certain tool. What should you check first?

4. Production cost suddenly doubles. What is the most effective first step?

🛠️ Hands-On Exercises

1. After enabling tracing,故意 write a vague tool description; observe the uncalled tool in the Studio trace.
2. Point the OTLP exporter at a local Jaeger instance;回放 a full agent run in the UI.
3. Add structured logs to three core tools; simulate a故障 and定位 the root cause via logs + traces.

Next: Chapter 17 · Integrating React and Next.js