The short version
Useful agents are narrow. Let them research an account from public sources, classify it against your ICP, draft a first-pass rationale, enrich missing fields and route the record, then show their work to a person. Don’t let them send unreviewed messages to strategic accounts or silently change CRM stages.
A good agent shortens the path to a decision and leaves a trail you can audit. If you can’t explain why it did what it did, it isn’t ready for production. Scope is the whole game: one job, a defined input, a defined output, a log.
Writing the job description first
Before touching a prompt, write a one-page spec: the trigger, the input fields, the allowed sources, the output schema, and the stop conditions. For example: “When an account is created with status New, read the company website and the latest annual report, and return five fields: summary, ICP label, three evidence bullets with links, two suggested roles, and confidence.”
Include a section titled “must not”. It should list actions the agent cannot take: no emailing, no CRM stage changes, no deleting data, no inventing sources. Specifying prohibitions in writing sounds obvious, but it is the quickest way to find out whether your tool setup actually enforces them or merely hopes.
Designing the output for a two-minute review
The reviewer should see the verdict first and the proof second. A good layout is a label (fit, maybe, not fit), one sentence of rationale, then evidence bullets each carrying a link. Anything that cannot be checked in two minutes will be skimmed, and skimmed outputs eventually pass through errors nobody saw.
Force a confidence field with fixed values: high, medium, low. Route low-confidence records into a separate queue rather than a general inbox. Agents are often fluent when wrong, so a structural prompt such as “if fewer than two sources agree, return low” catches cases that a tone check never would.
Using rejection tags as a training signal
When a rep rejects or edits an output, require one tag from a short list: wrong company, outdated source, wrong persona, weak rationale, missing data. A monthly count by tag tells you where to intervene. If “outdated source” dominates, change the retrieval step; if “weak rationale” dominates, change the prompt.
Keep a golden set of twenty accounts whose correct outputs a human has agreed. Whenever you alter the prompt or model, re-run the set and compare. This takes ten minutes and catches regressions that would otherwise show up as a gradual loss of trust three weeks later.
Widening scope without losing control
Expand in stages: read-only research, then field enrichment with approval, then drafting, then limited autonomous actions on low-risk records, such as tagging. Each stage needs a month of reviewed outputs above your acceptance threshold. Skipping stages is how teams end up with an agent that edits stages nobody asked it to touch.
Set a hard budget and a kill switch. Cap daily runs and cost, and log every call. If acceptance drops below your floor for two weeks running, the agent is paused automatically and a person looks at why.
Worked example (illustrative)
A hypothetical team pilots a single-purpose research agent on 200 new accounts over a month, with every output reviewed by a rep.
What changed: Most of the saving came from review replacing research. Rejection tags showed stale sources were the main fault, so changing the retrieval step lifted acceptance further the next month, without touching the humans.
Do it: the steps
Tick them off as you go. Your progress stays on this device.
Traps to avoid
- ×
Giving an agent write access to your CRM stages or your outbound inbox before it has a month of reviewed, accepted outputs behind it.
- ×
Combining research, scoring and outreach drafting in one agent. When it fails, nobody can tell which job went wrong, and fixes break the parts that were working.
- ×
Evaluating only on easy accounts. Test against messy ones, such as subsidiaries, recent mergers and companies with thin websites, where fabricated details are most likely.
What to track
- Acceptance rate per field
- outputs accepted without edit divided by outputs reviewed; below 80% means fix the prompt or data.
- Review time per account
- median minutes a human spends; target under five.
- Unsourced claim rate
- claims without a working link divided by total claims; the target is zero.
Benchmarks are typical planning ranges, not guarantees. Calibrate to your own data.
Questions people ask
Should an agent ever send messages on its own?
Not for strategic accounts. For low-value, high-volume cases you might allow templated, pre-approved messages with strict rules, but even then I would start with a human approving batches. The cost of one wrong message to an important account is higher than the saving from removing a review step.
How do I measure whether the agent is worth it?
Compare reviewed hours saved against build, run and review costs, and track acceptance rate by field. A simple test: does the rep make a decision faster, or more accurately, than without the output? If the answer is no for two months, simplify the scope or retire it.