What to automate and what to keep human
Automation fits work that is high in volume, repetitive and cheap to correct. It does not fit work where one mistake loses a meeting or breaks a law. That rule sorts most of the tasks in a reply workflow.
| Task | Automate? | Reason |
|---|---|---|
| Classify the reply | Yes, with a human review sample | High volume, one wrong label is fixable |
| Pull out a referred name or a return date | Yes, with review | Short text, easy to check |
| Score the account against the ICP | Yes | The score exists before the reply arrives |
| Create or update the CRM record | Yes | Pure data entry |
| Route to an owner | Yes, by written rules | The rules decide, the model does not |
| Suppress after an unsubscribe | Yes, plus a keyword rule and spot checks | A miss has legal and deliverability cost |
| First answer to a positive reply | No. A person sends it | This is the moment the meeting is won or lost |
| Answer objections, pricing, complaints | No | Context and judgment |
| Offer times and send the booking link | Yes, after the first human answer | Routine once the prospect is engaged |
The scripts and response targets for the human steps are in handle replies and book the call.
Two of our engagements used AI qualification in different places. A B2B consulting firm saw 400% more qualified leads and a 70% lower cost, using scraping, enrichment and AI qualification (our work). An AI feedback SaaS booked 40+ meetings a month from 1,000+ emails a day on multi-inbox infrastructure with AI qualification (our work). The setup behind this kind of work is described on our automation service page.
Score, route and schedule
Keep two scores apart. Fit score says how well the account matches your ICP, and it exists before you send anything. Reply class says what the person did. Routing uses both.
In our engagements, fit scores came from AI scoring of the list. For Fema Tape, 12,000 firms were found and 8,000 remained after we cut everything below 50 out of 100 (Fema Tape case study). Rapid Lanyard segmented targets scored 75 or higher out of 100 (Rapid Lanyard case study). Igrek had 9,000+ firms found and 1,800+ ideal after scoring (Igrek case study). Those scores ranked a list before sending. Carrying the same number into the CRM lets routing use it later.
| Condition | Owner | Deadline |
|---|---|---|
| Positive reply, fit score at or above your cutoff | Account executive or founder | 1 business hour |
| Positive reply, fit score below the cutoff | SDR reviews before any booking | 1 business hour |
| Objection | SDR | Same business day |
| Not now with a date | Task on the existing owner, dated | On the date |
| Wrong person with a name | SDR adds the contact and writes to them | Same business day |
| Unsubscribe | Automatic suppression, weekly audit | Immediately |
| Other, out-of-office, unclear | Human queue | Next business day |
Give each account one owner and write the rule down. If two people can both claim a company, someone will email the same prospect twice. The CRM cleanup playbook covers ownership fields.
Scheduling comes next. n8n's Calendly Trigger node listens for two events, Event created and Event canceled, and the docs add that Calendly webhooks fire only for bookings and cancellations managed by Calendly (n8n docs, checked 18 October 2026). Use the first to move the contact to a meeting-booked stage and stop all follow-up, and the second to create a reschedule task for the owner.
Build outline in n8n
This is an example build, so adapt it. Node names and options are from the n8n and Smartlead documentation (checked 18 October 2026), so confirm them against your own versions.
- Trigger. Smartlead's webhook documentation lists six event types, including EMAIL_REPLIED, which fires when a reply is received. The payload includes the lead's email, first and last name and company name, the campaign ID, and a reply object with subject, body and received time (Smartlead API docs). If you read replies from a mailbox instead, n8n's Gmail Trigger node has a Message Received event, caps emails per poll at 50 (default 10), and by default picks up unread mail only (n8n docs).
- Keyword pre-filter. An IF or Switch node sends opt-out phrases to the suppression branch.
- Text Classifier with the categories above and the Other branch switched on.
- CRM write. n8n's HubSpot node lists operations to create or update a contact, search contacts, and search companies by domain. It lists no separate upsert operation, so search by email first and branch on the result (n8n docs).
- Notify the owner by Slack or email with the reply text, the class and the fit score. For a positive reply, include a draft answer for the person to edit and send.
- Suppression. Add opt-outs to the shared suppression list used by every sending tool. The EMAIL_UNSUBSCRIBED webhook event fires when the unsubscribe link is clicked, which means a reply that says "remove me" never triggers it. Your classifier and keyword rule cover that gap.
- A second workflow with the Calendly Trigger node, as above.
- An error workflow. In n8n, an error workflow starts with the Error Trigger node and receives the failed execution's ID, URL, error message and last node executed, plus the workflow name. The Stop and Error node lets you fail a run on purpose, for example when the classifier returns a label that is not on your list (n8n docs). Send those alerts to a channel someone reads.
CRM sync fields
Store the model's output in its own fields so a person's correction never overwrites it. Our suggested set is below.
| Field | Content | Why it exists |
|---|---|---|
| Reply class (AI) | The label from the classifier | Accuracy measurement |
| Reply class (confirmed) | The label after human review | Source of truth for reporting |
| Reply received at | Timestamp from the webhook | Response-time reporting |
| Fit score | The ICP score carried over from the list | Routing |
| Campaign and sequence step | IDs from the payload | Which message earned the reply |
| Owner and next-step date | Set by the routing rule | Nothing stays unowned |
Duplicates are the usual way this breaks. HubSpot matches contacts on email, but its documentation says companies created through the API are not deduplicated by domain (HubSpot Knowledge Base, checked 18 October 2026). A workflow that creates companies on every positive reply will build duplicates, so search by domain before it creates anything.
Accuracy checks and failure modes
n8n separates two kinds of evaluation. A light evaluation runs before deployment on a small, often hand-made set, and you compare results by eye. A metric-based evaluation runs after deployment on a larger dataset, often built from production executions, scores each run with a numeric metric, and doubles as a regression test after you change a prompt (n8n docs, checked 18 October 2026).
Here is the version we would run. Export 200 past replies from your sequencer, label them by hand, and run the classifier on the same set. Count the matches in a table of true label against predicted label. Weight the two expensive errors above the rest: a positive labeled as anything else loses a meeting, and an unsubscribe labeled as anything else invites a complaint. As an illustration of the arithmetic, 190 correct out of 200 is 95 percent, and a 95 percent Wilson interval around that result runs from about 91 to 97 percent (our calculation). A set of 200 can tell 95 percent from 85 percent, but it cannot tell 95 from 93. After launch, read 25 randomly chosen classifications every week, plus every reply a person relabeled, and rerun the full set whenever you edit a prompt or change the model.
| Failure | Example | Guard |
|---|---|---|
| Polite phrasing hides a refusal | "Great, take me off your list" | Keyword rule first, human check on positives |
| Two intents in one reply | "Not now, ask Priya" | Allow multiple classes or send to a person |
| Auto-replies read as real replies | Out-of-office labeled "not now" | A separate category and a return-date rule |
| Unmatched reply disappears | Node left on Discard Item | Use the Other branch |
| Reply never picked up | Gmail Trigger set to unread only, and someone opened it first | Use the sequencer webhook, or change the read-status filter |
| Booking not synced | Meeting booked outside Calendly | Add a manual stage change |
| Quality drifts | A prompt edit shifts labels | Rerun the labeled set after every change |
Non-English replies need their own test. If a market answers in another language, add 30 or so labeled examples in that language before trusting the results there.
Questions
Can AI classify cold email replies reliably?
It can sort most replies, but you have to measure it on your own data. Label about 200 past replies by hand, run the classifier on them, and check the two costly errors: positives missed and opt-outs missed. Keep people on the first answer to every positive reply.
Which n8n node classifies replies?
The Text Classifier node. Each item goes to a connected chat model, which picks one of the categories you define. Set When No Clear Match to the Other branch, because the default discards unmatched items (n8n docs, checked 18 October 2026).
Should AI send the first reply to an interested prospect?
We recommend a person sends it, with a draft prepared for them. The first answer decides whether the meeting happens, and a wrong or tone-deaf automated reply costs more than the minutes it saves.
How do I keep AI qualification from creating duplicate CRM records?
Search before you create. HubSpot matches contacts on email, but companies created through its API are not deduplicated by domain, so look up the company by domain first and write only when no match comes back.
Sources
- n8n Docs: Text Classifier node
- n8n Docs: Error handling
- n8n Docs: HubSpot node
- n8n Docs: Gmail Trigger node
- n8n Docs: Calendly Trigger node
- n8n Docs: Evaluations overview
- Smartlead API docs: Webhooks
- HubSpot Knowledge Base: Deduplicate records in HubSpot
Worked examples are illustrations unless a case study is linked. How we write and check these pages.