Chapter 19 · Capstone 3: Code Review Agent

Chapter goals: Build a PR-triggered auto code review Agent — run tests in a sandbox to验证 changes, produce structured review意见 across four dimensions (安全/性能/命名/测试覆盖), submit graded GitHub Reviews, and master anti-noise strategies so developers actually want to read its comments.

19.1 Goal and Trigger Chain

The Chapter 17 Triage Agent handles bug reports; this chapter's Reviewer Agent handles every Pull Request:

PR opened/updated → webhook trigger → fetch diff (chunked filter)
                 → sandbox checkout + run tests
                 → four-dimension checklist review (security/perf/naming/test coverage)
                 → graded review comments (blocker / suggestion / praise)
                 → submit GitHub Review (only blockers block merge)

The key difference from the triage capstone: review is a high-frequency, low-noise-tolerance scenario — one useless bot comment and the team disables the entire integration. So half this chapter is "how to review" and the other half is "how not to spam."

The trigger入口 uses Flue's Channels to receive the GitHub webhook and route into the Agent session:

// channels/github.ts — PR event entry: verify signature then dispatch to Reviewer session
import { dispatch } from '@flue/runtime';

export async function onPullRequest(payload: any, signature: string) {
  // Verify signature first, then process —防 forged requests (key from env var, never hardcoded)
  verifyWebhookSignature(payload, signature, process.env.GH_WEBHOOK_SECRET);

  // Only care about opened and synchronize actions; close/merge do not trigger review
  if (!['opened', 'synchronize'].includes(payload.action)) return;

  const pr = payload.pull_request.number;
  const repo = payload.repository.full_name;
  // Dispatch into the Agent session: pass the PR number as task context
  await dispatch('reviewer', `Review ${repo} PR #${pr}`, {
    env: { REPO: repo },   // Injected for the tool layer
  });
}

19.2 Tool Layer: tools/review.ts (I): Fetching the Diff

Three typed tools cover all external actions needed for review. Start with the read-only diff fetch:

// tools/review.ts — Tool 1: fetch PR change content
'use flue-tool';
import { defineTool } from '@flue/runtime';
import { exec } from '@flue/runtime/node';

export const getDiff = defineTool({
  name: 'get_diff',
  description: 'Get the PR changed file list; when path is specified, return that file\'s diff',
  inputSchema: {
    type: 'object',
    properties: {
      pr: { type: 'number', description: 'PR number' },
      path: { type: 'string', description: 'Optional; only diff a specific file' },
    },
    required: ['pr'],
  },
  async execute({ pr, path }) {
    // gh api read-only; no write operations
    if (path) {
      const { stdout } = await exec(`gh api repos/${process.env.REPO}/pulls/${pr}/files --jq '.[] | select(.filename=="${path}") | .patch'`);
      return { file: path, patch: stdout };
    }
    const { stdout } = await exec(
      `gh api repos/${process.env.REPO}/pulls/${pr}/files --jq '.[] | {file: .filename, adds: .additions, dels: .deletions}'`
    );
    return { files: stdout.split('\n').filter(Boolean).map(JSON.parse) };
  },
});

Then implement the two side-effect actions: run tests to验证 changes, and submit graded reviews.

// tools/review.ts (continued) — Tools 2 & 3: run tests, post review
'use flue-tool';
import { defineTool } from '@flue/runtime';
import { exec } from '@flue/runtime/node';

export const runTests = defineTool({
  name: 'run_tests',
  description: 'Run npm test in the local sandbox; return pass/fail summary',
  inputSchema: { type: 'object', properties: {}, required: [] },
  async execute() {
    const { code, stdout } = await exec('npm test -- --reporter=dot', { timeout: 300_000 });
    // Compress raw logs into a structured conclusion; save tokens vs the model reading全量 logs
    const failed = code !== 0;
    const failedFiles = stdout.split('\n').filter(l => l.startsWith('FAIL')).slice(0, 10);
    return { passed: !failed, failedFiles };
  },
});

export const postReview = defineTool({
  name: 'post_review',
  description: 'Submit a PR Review: body is the overall comment; comments are line-level; event decides APPROVE or REQUEST_CHANGES',
  inputSchema: {
    type: 'object',
    properties: {
      pr: { type: 'number' },
      body: { type: 'string', description: 'Overall comment (Markdown)' },
      event: { type: 'string', enum: ['APPROVE', 'REQUEST_CHANGES'] },
      comments: {
        type: 'array',
        description: 'Line-level comment list',
        items: {
          type: 'object',
          properties: { path: { type: 'string' }, line: { type: 'number' }, body: { type: 'string' } },
          required: ['path', 'line', 'body'],
        },
      },
    },
    required: ['pr', 'body', 'event'],
  },
  async execute({ pr, body, event, comments = [] }) {
    // The only write entry: all comments must go through here for audit and rate-limiting
    await exec(`gh api repos/${process.env.REPO}/pulls/${pr}/reviews -f body=${JSON.stringify(body)} -f event=${event} -f comments=${JSON.stringify(comments)}`);
    return { posted: true, inlineCount: comments.length };
  },
});

19.3 Review Skill: skills/review-checklist/SKILL.md

固化的团队审查标准 into a reusable skill pack, rather than散落在 prompts:

# Code Review Checklist

Check the diff against these four dimensions; each must output "pass" or a specific issue.

## 1. Security (highest priority)
- Is user input直接拼接进 SQL / shell command / HTML? (injection vulnerabilities)
- Do new API endpoints validate auth and permissions?
- Are secrets/tokens hardcoded or written to logs?
- Did dependency upgrades引入 known CVEs? (check with npm audit)

## 2. Performance
- N+1 queries or redundant computation inside loops?
- Large collection ops: pagination / streaming considered?
- Synchronous blocking calls that could stall the event loop?

## 3. Naming and Readability
- Do names express intent而非 implementation? (`flag2` is a code smell)
- Functions exceeding 50 lines /明显高 cyclomatic complexity?
- Consistent with the project's既有 style?

## 4. Test Coverage
- Does新增逻辑 have对应 tests?
- Are edge cases (empty input, oversized input, concurrency) covered?
- If existing tests break,必须指出失败用例名.

## Output Discipline
- Each comment标注级别: [blocker] must fix / [suggestion] improve / [praise] worth noting
- Uncertain issues: use questions, not assertions
- Overall comment ≤ 200 characters; conclusion first, details after

Note that the SKILL.md writes both how to check and how to speak — the latter is exactly where auto-review最容易翻车.

19.4 Assembling the Review Agent

// agents/reviewer.ts — PR auto-review Agent
'use agent';
import { useModel, useSandbox, useSkill, useTool } from '@flue/runtime';
import { local } from '@flue/runtime/node';
import checklist from '../skills/review-checklist/SKILL.md';
import { getDiff, runTests, postReview } from '../tools/review.ts';

export function Reviewer() {
  // Review needs较强的推理能力, but not the最强 model — the checklist已经收敛了行为
  useModel('anthropic/claude-sonnet-4-6');
  // Sandbox required: runTests must execute未知代码 for real
  useSandbox(local());
  useSkill(checklist);
  useTool(getDiff);
  useTool(runTests);
  useTool(postReview);

  return `
You are a strict code reviewer. Execute this flow for the current PR:

1. Call get_diff to fetch the changed file list;
2. Skip lock files and auto-generated files; review the full diff of core files one by one;
3. Call run_tests to验证 the test suite状态 after changes;
4. Check against the four dimensions in review-checklist; output a conclusion per dimension;
5. Summarize into a Review and call post_review:
   - Any blocker → event: REQUEST_CHANGES
   - Otherwise → APPROVE; list suggestions and praise in the overall comment.

Hard rules:
- Only comment on code实际 appearing in the diff; do not臆测 unmodified files;
- Each comment must cite the specific line number and the问题 code snippet;
- When tests fail,优先 report the failing test name, then hypothesize causes.
`;
}

19.5 Graded Review: Letting Blockers Block, Suggestions Float

The keyanti-noise strategy: only blockers trigger REQUEST_CHANGES; suggestions and praise go into the overall comment body without blocking merge. This ensures the PR author sees useful feedback without feelingblocked by every minor suggestion.

Additionally, the skill's output discipline ("conclusion first, details after; ≤200 chars overall") ensures the review comment is scannable — developers will actually read it instead of ignoring a wall of text.

19.6 Chapter Summary

• Trigger: GitHub webhook → channel verify → dispatch to Reviewer session;
• Tools: getDiff (read-only), runTests (sandboxed execution), postReview (single write entry for audit);
• Skill: four-dimension checklist + output discipline (how to check + how to speak);
• Anti-noise: only blockers block; suggestions float; ≤200 char overall comment;
• Sandbox: local() required because runTests executes未知代码.

🧪 随堂测验

点击你认为正确的选项。答错时会展示正确答案与原因解析。

1. Why is a sandbox required for the Reviewer agent?

2. What is the anti-noise strategy in this capstone?

3. Why is postReview the "only write entry"?

4. What does the output discipline in the review checklist prevent?

🛠️ Hands-On Exercises

1. Build the full review agent: implement getDiff/runTests/postReview tools, write the review-checklist skill, assemble in one agent function, and test with a real PR.
2. Verify the anti-noise strategy: confirm that suggestions do not block merge while blockers do.
3. Add a rate-limit guard to postReview (e.g., max 10 inline comments per review) and verify it enforces the limit.

Capstone 3 complete. Course complete!