Chapter 13 · Observability

Chapter goals: Distinguish Flue's two event surfaces, learn to subscribe to runtime events with observe(), read token usage, and export telemetry to OpenTelemetry, Sentry, and Braintrust.

13.1 Two Event Surfaces

Flue exposes "what the agent did" as typed runtime events: model turns, tool calls, structured logs, compaction, settlement. Note that this is a different surface from the chat UI's message stream:

① Runtime event surface (this chapter): subscribe via observe(); for ops and debugging
② Session message surface: Routing + Flue Agent SDK; for chat interface逐条 messages

// Ops-oriented event subscription: observe()
import { observe } from '@flue/runtime';

const stop = observe((event) => {
  // Each event is a discriminated union type; handle with switch
  switch (event.type) {
    case 'model_turn':
      console.log('[model]', event.model, 'tokens:', event.usage);
      break;
    case 'tool_call':
      console.log('[tool]', event.name, 'latency:', event.durationMs, 'ms');
      break;
    case 'log':
      console.log('[log ]', event.level, event.message);
      break;
  }
});

// Stop subscribing when no longer needed
// stop();

13.2 What's in the Event Stream

Model turn events自带 token usage and provider diagnostic info — this is the raw data source for cost monitoring:

// Cost stats: aggregate model_turn event usage
import { observe } from '@flue/runtime';

const usageByDay = new Map<string, { input: number; output: number }>();

observe((event) => {
  if (event.type !== 'model_turn') return;
  const day = new Date().toISOString().slice(0, 10); // Aggregate by day
  const cur = usageByDay.get(day) ?? { input: 0, output: 0 };
  usageByDay.set(day, {
    input: cur.input + (event.usage?.inputTokens ?? 0),
    output: cur.output + (event.usage?.outputTokens ?? 0),
  });
});

// Print usage report hourly
setInterval(() => {
  for (const [day, u] of usageByDay) {
    console.log(day, 'input:', u.input, 'output:', u.output);
  }
}, 60 * 60 * 1000);

13.3 Exporting to OpenTelemetry

@flue/opentelemetry converts runtime events to OTel trace/spans, connecting to any compatible backend (Jaeger, Grafana Tempo etc.):

npm install @flue/opentelemetry @opentelemetry/sdk-node @opentelemetry/exporter-trace-otlp-http

// src/otel.ts
// OpenTelemetry export adapter
import { flueOtel } from '@flue/opentelemetry';
import { NodeSDK } from '@opentelemetry/sdk-node';
import { OTLPTraceExporter } from '@opentelemetry/exporter-trace-otlp-http';

const sdk = new NodeSDK({
  // Flue-provided adapter maps agent activity to spans
  traceExporter: new OTLPTraceExporter({
    url: process.env.OTEL_EXPORTER_OTLP_ENDPOINT, // e.g. http://localhost:4318/v1/traces
  }),
});

// Register the Flue events → OTel bridge
flueOtel({ sdk });
sdk.start();

13.4 Sentry and Braintrust

Two ready-made integrations cover "error tracking" and "LLM evaluation" needs:

// Sentry: send agent exceptions and context into error tracking
import * as Sentry from '@sentry/node';
import { observe } from '@flue/runtime';

Sentry.init({ dsn: process.env.SENTRY_DSN });

observe((event) => {
  if (event.type === 'error') {
    Sentry.captureException(event.error, {
      tags: { agent: event.agentId },
      contexts: { flue: { event: event.type } },
    });
  }
});

// Braintrust: export eval data for LLM evaluation tracking
import { braintrustExporter } from '@flue/braintrust';

braintrustExporter({
  apiKey: process.env.BRAINTRUST_API_KEY,
  projectName: 'my-flue-agent',
});

13.5 Chapter Summary

• Flue exposes two event surfaces: runtime events (observe()) and session messages (SDK routing);
• observe() subscribes to typed events: model_turn, tool_call, log, error, compaction, settlement;
• Model turn events自带 token usage — the raw source for cost monitoring;
• @flue/opentelemetry bridges to Jaeger/Grafana Tempo; @sentry/node for error tracking; @flue/braintrust for LLM eval tracking.

🧪 随堂测验

点击你认为正确的选项。答错时会展示正确答案与原因解析。

1. What are Flue's two event surfaces?

2. What token information is available on model_turn events?

3. What does @flue/opentelemetry do?

4. When should you use the Sentry integration?

🛠️ Hands-On Exercises

1. Set up observe() to log all model_turn and tool_call events; verify the token stats are correct.
2. Connect the OpenTelemetry exporter to a local Jaeger instance; replay a session and visualize the trace in the Jaeger UI.
3. Add Sentry error tracking;故意 trigger an agent error and verify it appears in the Sentry dashboard with context tags.

Observability mastered. Next, CLI and local run.