Case study
Support triage that stops at the right line
Built an agent workflow that classifies, enriches and drafts answers for inbound support tickets, resolves the routine ones with one-click approval and escalates the rest with full context. First-response time fell while a person stayed on every risky reply.
Measured outcomes
- -71% first-response time on routine categories
- 58% of tickets drafted by the agent and approved without edits
- 0 invented policies, because every answer cites a source or does not ship
Details
- Client
- B2B SaaS company, 25-person support team
- Sector
- Software
- Services used
Challenge
Where it started
Two thousand tickets a month, forty percent of them variations of the same thirty questions. Senior agents spent their day on the routine cases while the hard ones waited. A previous chatbot had been switched off after it invented a refund policy.
Approach
What I did, in order
-
Measured the baseline. Volume by category, first-response time, resolution time, and where the previous bot had failed.
-
Built retrieval over the product documentation and resolved tickets, with answers restricted to sourced content and a confidence threshold below which the agent does not draft.
-
Designed the workflow with three lanes. Auto-draft for routine categories with one-click approval, enrichment only for anything touching billing or personal data, and direct escalation for the rest.
-
Ran it in shadow mode for two weeks against real tickets, compared drafts to human answers, then switched on approvals lane by lane.
Deliverables
What the team kept
- Workflow implementation as code on the company's agent runtime, with an evaluation set and regression tests
- Retrieval pipeline over documentation and ticket history, with a source citation in every draft
- Dashboards for volume, draft acceptance rate, latency and cost per ticket
- Runbook for adding categories and refreshing the evaluation set
The bot that got switched off failed because it had no line it could not cross. This one has three, and it stops at each of them. That is why the team trusts it.
Next step
Facing something similar?
Tell me what is slowing your team down. I answer within one business day, and the first 30 minutes are on me.