Chapter 26 · Best Practices and Production Checklist

Chapter goals: Gather all engineering discipline scattered across the tutorial into an executable checklist — prompt polishing, tool governance, cost control, security and observability — and finish with a pre-go-live checklist and ten anti-patterns.

26.1 role / goal / backstory Polishing Checklist

Role prompts are CrewAI's highest-leverage productivity factor. Self-check every item:

import os
from crewai import Agent, LLM

llm = LLM(
    model="openai/deepseek-chat",
    base_url="https://api.deepseek.com/v1",
    api_key=os.getenv("DEEPSEEK_API_KEY"),
)

# ✅ Good role definition: single responsibility + verifiable behavioral guidelines + clear boundaries
good_analyst = Agent(
    role="E-commerce Repurchase Analyst",
    goal="Locate categories with declining repurchase rates from order data and produce 2 actionable hypotheses",
    backstory=(
        "You have 5 years of growth analysis experience. Work准则:\n"
        "1) Every conclusion must cite specific numbers;\n"
        "2) Distinguish 'correlation' from 'causation'; if uncertain, say so;\n"
        "3) Recommendations must include expected effect magnitude and validation method.\n"
        "You do NOT负责: writing code implementations, creating marketing copy."
    ),
    llm=llm,
    max_iter=5,
)

Self-check checklist:

• [ ] role is "function + domain" not a title ("E-commerce Repurchase Analyst" > "Senior Expert");
• [ ] goal contains a verifiable output form ("2 hypotheses" can be counted, "deep analysis" cannot);
• [ ] backstory writes behavioral guidelines and negative lists (what you do NOT负责), each statement can be scored;
• [ ] One Agent is responsible for one Task; when you find two tasks sharing a role with conflicting requirements, split the role.

26.2 Tool Governance: Granularity, Count, and Return Values

Discipline	Threshold / Practice	Violation Consequence
Tools per Agent	Recommended < 10	Selection error rate rises with count
Granularity	One tool = one verb (search/fetch/read)	"All-in-one" tools have complex parameters and fail easily
Parameter validation	Must have Pydantic args_schema	Model fills wrong params with silent failure
Return value	Trim to the minimum information needed for decisions	Token runaway, context pollution
Destructive operations	Whitelist filtering + secondary confirmation (human_input)	Accidental data deletion

from crewai.tools import tool


@tool("search_orders")
def search_orders(query: str, limit: int = 5) -> str:
    """Search orders by keyword, return at most limit summaries (id/amount/date).
    Do NOT use this tool for statistics; use count_orders for that."""
    # Docstring is the model's sole basis for tool selection:
    # First line says what it does; second line划清 what it does not do
    return f"[{query}] matched 128 total, showing top {limit} summaries..."

26.3 Cost Control: Model Tiering and Iteration Caps

Three cost-saving measures in priority order:

1. Model-tiered routing: classification/extraction/formatting with cheap models, creation/reasoning with strong models. Mixing tiers within the same crew is fully legal:

cheap_llm = LLM(
    model="openai/deepseek-chat", temperature=0,
    base_url="https://api.deepseek.com/v1", api_key=os.getenv("DEEPSEEK_API_KEY"),
)
strong_llm = LLM(
    model="openai/deepseek-reasoner", temperature=0.7,
    base_url="https://api.deepseek.com/v1", api_key=os.getenv("DEEPSEEK_API_KEY"),
)

router_agent = Agent(role="Intent Classifier", goal="Route user requests into 6 predefined categories",
                     backstory="Output only category numbers, never explain", llm=cheap_llm, max_iter=2)
analyst_agent = Agent(role="Senior Analyst", goal="Deeply attribute classification results",
                      backstory="Industry expert who speaks with numbers", llm=strong_llm, max_iter=4)

2. max_iter per role: the default is lenient; production recommends 2–6; runaway loops are the #1 cause of cost explosions;
3. max_rpm guardrails: limit requests per minute, protecting against upstream throttling and retry storms; pair with Chapter 21's token_usage guardrails for a closed loop.

26.4 Credential Management and Security Baseline

• Credentials exist only in environment variables / secret management systems; .env goes in .gitignore; images are credential-free (Chapter 22);
• MCP servers and third-party tools reviewed before integration: whitelist-filter dangerous tools (Chapter 19);
• Exposed kickoff APIs must have authentication and rate-limiting — crew execution is real compute spend;
• human_input=True for confirmation gates before irreversible operations;
• Log sanitization: if prompts carry user data, assess compliance boundaries before piping to observability platforms.

26.5 The Observability Trinity

Minimum production crew configuration (corresponds to Chapter 21):

1. metrics: record token_usage four-field + wall-clock time per kickoff into a database;
2. callbacks: task_callback pushes each task's output summary and usage to the logging pipeline for step-level attribution;
3. tracing: one-line Langtrace integration or OpenLit→Langfuse; retain traces for at least two weeks for retrospective analysis.

Minimal metrics + callback implementation:

import json
import time
from pathlib import Path


def audit_task(output):
    """Attached to each Task's callback: step-level usage and output summary to disk"""
    record = {
        "ts": time.time(),
        "description": output.description[:50],
        "chars": len(output.raw or ""),
        "token_usage": str(getattr(output, "token_usage", "")),
    }
    with Path("task_audit.jsonl").open("a") as f:
        f.write(json.dumps(record, ensure_ascii=False) + "\n")

Add callback=audit_task to key tasks; combined with writing result.token_usage at the end of each kickoff, you can answer "which task burned the most money this month" with a single jq command.

26.6 Top 10 Common Anti-Patterns

1. All-in-one Agent: one Agent that researches, writes, and reviews → split roles, single responsibility;
2. Uncapped iteration: no max_iter set → loops burn money until timeout;
3. context flood: all tasks' context includes all history → only pass the upstream artifacts each step needs;
4. tool returns full table: DataFrame full to_string → trim to top-N and key columns;
5. hierarchical as hammer: fixed-order flow but using manager process → sequential suffices;
6. LLM doing deterministic judgment: asking the model to compare two dates → extract as code logic or a tool;
7. hardcoded credentials: keys written into yaml/source → environment variable injection;
8. no evaluation before go-live: judging quality by feel → Chapter 20 regression set + LLM judge dual insurance;
9. in-memory state deployment: job tables / file artifacts stored in process-local memory → externalize to stateless storage;
10. swallowing exceptions silently: kickoff fails and only prints one line → callback reporting + checkpoint recoverable.

26.7 Pre-Go-Live Checklist

Functional Correctness
  [ ] Regression evaluation set all passing (fixed input + feature assertions)
  [ ] crewai test -n 3 each task score >= threshold (e.g. 8.0)
Stability
  [ ] Every Agent has max_iter set (<=6)
  [ ] External dependencies (search/API tools) have timeouts and failure fallback
  [ ] Checkpoint enabled; interruption recovery rehearsed
Cost
  [ ] Token guardrail assertions active in CI
  [ ] Classification/small tasks routed to low-tier models
Security
  [ ] All credentials via environment variables; no leaks in git history
  [ ] kickoff接口 has authentication and rate-limiting
  [ ] Dangerous tools filtered or gated with human_input
Observability
  [ ] usage_metrics ingested with alert thresholds
  [ ] tracing platform可见 complete call chain
Deployment
  [ ] Service statelessness four-checks passed (job table / artifacts / memory / checkpoints)
  [ ] Docker image不含 credentials, runs as non-root
  [ ] Rollback plan: previous working version image + config snapshot

Chapter Summary

• Role prompt three要素: verifiable goal, behavioral-guideline backstory, clear negative list;
• Tool governance five disciplines: <10 tools, single-verb granularity, args_schema validation, minimal return values, destructive operations gated;
• Cost three strikes: model-tiered routing > max_iter gating > max_rpm throttling;
• Security and observability are baseline configurations: env-var credentials, metrics/callbacks/tracing trinity;
• Top-3 costliest anti-patterns: uncapped iteration, context flooding, in-memory state deployment.

🧪 随堂测验

点击你认为正确的选项。答错时会展示正确答案与原因解析。

1. Which goal phrasing best符合 "verifiable output" principle?

2. Regarding tools per Agent, what upper limit does this tutorial recommend and why?

3. A team hands an Agent the task "determine whether user input contains a phone number" and expects natural-language reasoning — which anti-pattern is this?

4. Before go-live you发现 the crew occasionally enters infinite loops causing billing spikes. Which two defense layers should be prioritized?

🛠️ Hands-On Exercises

1. Use this chapter's 26.1 checklist to review the agents.yaml / Agent definitions you wrote in Chapters 23 and 24; find at least 2 descriptions that violate the "verifiable output" principle and rewrite them; compare the crewai test -n 2 score changes before and after modification.
2. Implement a "cost dashboard" script for your project: read the last N kickoff token_usage records (self-persisted to a database), output daily cost trends and Top 3 costliest tasks, exit with non-zero code when thresholds are exceeded.
3. Conduct a complete "go-live rehearsal"对照 26.7 checklist: from regression set, max_iter audit, checkpoint recovery drill to statelessness four-checks; record actual evidence (command output / screenshot notes) for each item; after filling gaps, rehearse again.

🎉 Tutorial complete. Learning path review: core concepts (01–05) → models & tools (06–08) → processes & memory (09–13) → engineering (14–19) → evaluation, observability, deployment (20–22) → capstones (23–25) → production checklist (this chapter). The best next exercise is to take a real workflow from your own business, remodel it as a CrewAI application, and let this checklist guard your go-live.