AI Support Agents Work Better Paired With a Human Partner

12 min read
  • Public Sector
  • AI Agents
  • Deep dive
Contents9 sections

A Resident Who Could Not Get Through

Pam Stepansky, a resident of Orland Park, Illinois, told the Village Board on October 5 that she "never could be connected to a person at all," and drew the obvious conclusion: "So maybe we should be able to talk to people at the village if there's a question or a comment." She was describing Humphrey, the AI phone agent the village launched in August, named for its first mayor. Another resident, Karen Drake, described finding herself in the same position when she called with a question about the Great Pumpkin Party at Centennial Park on October 3.

Orland Park paused the AI agent after those complaints, while reporting that 71 percent of 2,687 September calls were transferred to a department and that staff testing had not reproduced the failures. Those are village figures, and we have not seen an independent audit of them.

What the Village's Own Numbers Show

Village staff presented September data before the pause. Of 2,687 calls, 71 percent were transferred to a village department. Ninety-one percent of calls were scored as neutral in sentiment and 6 percent as negative.

Seventy-one percent of calls moving to a department measures how many people called the village needing someone with the authority to act. In our reading, that figure looks less like a sign of failure than evidence the assistant was doing triage at volume and handing off, which is close to what we would expect from a well-scoped municipal phone agent, though we have not seen a published benchmark for an ideal transfer rate. What residents raised at the board meeting concerned what happened on the other side of the transfer.

For public agencies and enterprises serving constituents and customers, the case shows that an assistant's value is capped when service ends at the AI's responses. We argued in Agent Pilots Fail Because of Delivery Discipline, Not Technology that enterprise agent pilots stall on delivery practice rather than model capability, with unclear success criteria, insufficient data access, and evaluation drift as the addressable causes. A related readiness gap shows up on the data side of the public sector: a 2024 survey of 46 states by the National Association of State Chief Information Officers (NASCIO) found 95 percent of respondents acknowledging AI's growing importance for data management while only 22 percent had implemented a dedicated data quality program. In our view, a comparable discipline gap now appears on the resident-facing side, where the consequences arrive in a board meeting.

Where Assistants Reach Their Limit

A beige desk phone handset resting off the hook on scattered paperwork under a fluorescent lamp.

Our experience implementing AI virtual assistants is that they are genuinely good at a specific shape of work: high-volume, well-bounded, repeatable questions. Trash pickup day. Permit office hours. Status of a request already in the system. These consume disproportionate staff time and carry almost no judgment, and an assistant can typically handle them at any hour without a hold queue.

The limit is predictable. Exceptions, complaints, and multi-step requests need a person: a resident disputing a water bill, a business owner working through a three-agency permit sequence, a parent whose situation matches no category in the service catalog. In these cases, judgment and authority are the service itself. A benchmarking study of municipal chatbots in Dutch public administration, by researchers at Vrije Universiteit Amsterdam, put it directly: "Municipal chatbots exhibited notable shortcomings in competence and integrity, with some failing to meet basic standards of lawful and equal service provision."

Even mature human contact centers leave a meaningful share of contacts open on the first touch. The Service Quality Measurement Group's industry standard for a good first contact resolution rate sits between 70 and 79 percent, meaning roughly 30 percent of tickets take more than one interaction. Zendesk puts the share of call centers reaching 80 percent or higher at 5 percent, without naming a source for that second figure. Handoffs are a permanent feature of any service operation at this scale. What matters is what the resident and the staff member each see when one happens.

The Cost of a Broken Handoff

When the handoff is treated as an afterthought, the cost is measurable. NPS Prism, which sells customer experience benchmarking data, surveyed customers about chatbot experiences across 40 major banking, retail, telecom, and airline brands and found that the timing of the bot-to-human handoff determines whether the interaction helps or hurts the brand. In telecom, NPS Prism reports an episode Net Promoter Score of minus 48 when customers were transferred to a human only after the bot failed, against plus 10 when human support was offered proactively, a 58-point gap. Those are NPS Prism's own benchmarking figures and have not been independently corroborated. Their account of why is blunt: "With a reactive handoff, the customer may have already repeated their issue, encountered a loop, and watched the bot fail to resolve their problem."

The gap between what service leaders believe and what their systems do is the part worth sitting with. Simplesat, a customer feedback software vendor, reports figures from Five9's 2026 Business Leaders CX Report, based on a Hanover Research survey of 3,000 consumers and 600 customer service decision-makers across the United States, United Kingdom, and Germany: 96 percent of decision-makers say their organization preserves context effectively during AI-to-human handoffs, while 83 percent of consumers say they at least sometimes have to repeat themselves after a bot transfers them, and 35 percent say it happens often or always. Thirty-seven percent of those same decision-makers said their agents cannot see the earlier conversation with the bot. That is vendor-commissioned research, but the shape of it is hard to explain away: the belief and the experience are measuring different things.

The context loss itself has a price. ETS Labs, which sells contact center AI services, describes one contact center where customer satisfaction for escalations dropped 34 percent until the handoff architecture was rebuilt, a vendor account we could not independently corroborate. All of these datasets come from firms selling into this problem and differ in scope, population, and method. They agree on direction: context that does not travel with the caller costs time, satisfaction, and repeat contacts.

What the village reported about its own system is narrow. The AI could not transfer callers to the operator or to the Parks and Recreation Department because of the phone numbers associated with them, and staff testing ahead of time had not surfaced the issues residents later described. In our implementation experience, that pattern usually points to routing that was validated in a staging environment rather than tested against live department lines. Orland Park's transfer problem is what happens when the human layer is a phone number rather than part of the design.

One Conversation Record, Two Service Layers

A support specialist wearing a headset works at a single monitor showing an open conversation thread.

The model we build toward is simple in concept and demanding in execution. Staff and the AI work from the same conversation record. When a request moves to a person, that staff member opens a view that already contains what the resident asked, what the assistant answered, what it could not resolve, and a summary of the exchange with suggested next steps. The resident never repeats themselves. The assistant does not stop working at that moment; it keeps supporting the staff member with retrieval, drafting, and policy lookup while the conversation continues.

The productivity case for that second layer is established outside vendor material. Studying roughly 5,000 customer support agents at a Fortune 500 software company, National Bureau of Economic Research working paper 31161 found that access to an AI assistant raised the number of issues resolved per hour by 13.8 percent, with agents spending about 9 percent less time per chat. The gains were concentrated among less experienced and lower-skilled staff, who saw improvements of 35 percent. For a municipal department where the person picking up an escalated call may be new to the role, that distribution matters more than the average.

Design practice on the escalation trigger is maturing too. Cognizant, a consulting firm that sells customer experience implementation services, describes one design approach it recommends to clients: assigning an empathy threshold score that fires a warm handoff with the context the agent needs, driven by signals in the caller's own language. In Cognizant's account, "the use of escalation language such as 'This is ridiculous' and 'I've already explained this' would drive up the system's empathy score."

Our AI and application development practices, together with our user experience research and digital experience work, build this shared-context model as part of the initial deployment: one conversation record, AI assistance for staff, tested routing, and monitoring across both layers. Routing is exercised against live department numbers and live queues before the assistant takes its first resident call.

A Standard Public Agencies Can Act On

Agencies already operate inside policy that points this way. Memorandum M-24-10 from the Office of Management and Budget (OMB) requires covered federal agencies to "provide and maintain a mechanism for individuals to conveniently opt-out from the AI functionality in favor of a human alternative, where practicable and consistent with applicable law," and specifies that the mechanism "must be prominent, readily available, and accessible." The Generative AI Profile from the National Institute of Standards and Technology (NIST AI 600-1) directs organizations to use human moderation and review in line with human-AI configuration policies set in the Govern function. Municipal policy is converging on the same requirement: the Kansas Health Institute's review of city-level AI policies found that they "must include human intervention options, particularly when AI fails to meet expectations," with city employees expected "to have the authority to override AI systems when necessary."

Accessibility obligations run alongside. The U.S. Department of Justice's April 2024 Title II rule sets technical requirements for state and local government web content and mobile applications. An interim final rule published in April 2026 pushed the deadlines back a year, to April 26, 2027 for entities with a population of 50,000 or more and April 26, 2028 for smaller entities and special districts. The underlying requirement did not change, and neither did the standard. A caller who cannot reach a person has a service access problem before anyone gets to debug the technology.

None of this is a reason to slow down. We have set out our position in Governance-First AI Adoption Is the Faster Path to Scale: designing the oversight path up front is what lets a deployment survive contact with real users and keep expanding.

Mapping the First Ninety Days

There is a concrete version of this that an agency or enterprise can start on now. Map your top 20 request types. For each one, mark where it should move from AI to staff, and be specific about the trigger. Then confirm, by testing against live numbers and live queues rather than a staging environment, that staff see the full conversation and the AI summary at that point. Then track resolution and repeat contacts across both layers for the first 90 days.

Measurement is where most of these programs go thin. An escalation rate tells you how often the assistant hands off, not why, and as Dimension Labs, a conversation analytics vendor, puts it, "the fix for a bot that never learned a topic is different from the fix for one that keeps misreading the question." Record every conversation's topic and, where it was not resolved, its cause, then count by both. Link repeat contacts back to the originating assistant conversation, and ask the one survey question that isolates handoff quality: did the person have to give their information twice. Benchmark first contact resolution against the 70 to 79 percent industry range so the 90-day numbers mean something.

Design the Handoff, Not Just the Bot

Orland Park's staff did the testing they thought they needed, and the sentiment data looked reasonable: 91 percent neutral, 6 percent negative. What the data did not capture was the experience of a resident who reached the end of the AI's capability and found nothing behind it. The village is working to fix the system, and it should, because the underlying approach is sound.

We treat the shared-context human layer as part of the initial deployment because the assistant's value compounds when a person can take over inside the same conversation, and stalls when they cannot; that discipline, not any single statistic in this piece, is what the Orland Park case and the broader evidence point toward. The evidence for how to design that moment exists now, the platform capability exists now, and the policy expectation is already written down. Agencies and enterprises standing up virtual assistants this year can build the handoff on day one rather than discovering it at a board meeting.

Sources

  1. Olivia Stevens. "Orland Park, Ill., Works to Fix AI-Powered Phone System." Daily Southtown, via GovTech, November 6, 2026
  2. GovTech News Staff. "NASCIO Survey Exposes the Hard Data Truths of AI Adoption." GovTech, September 10, 2024
  3. Ralfs Rudzitis and Kristina Sabrina Weissmuller. "Benchmarking Municipal AI Chatbot Performance: Mixed Methods Insights into Competence, Integrity, and Algorithmic Discrimination in Dutch Public Administration." SocArXiv preprint, Center for Open Science, 2025
  4. Zendesk. "What Is First Contact Resolution (FCR)? Benefits + Best Practices." Updated June 18, 2026
  5. Carlos Rabago. "For AI Bots, Proactive Escalation to Human Support Is Key to Protecting Customer Loyalty." NPS Prism, October 1, 2026
  6. ETS Labs. "Why AI-to-Human Handoffs Fail: 5 Causes and How to Fix Them."
  7. Simplesat. "Chatbot to Human Handoff: Do Customers Repeat Themselves?" October 22, 2026, reporting figures from Five9's 2026 Business Leaders CX Report
  8. National Bureau of Economic Research. "Measuring the Productivity Impact of Generative AI." NBER Digest, June 2023, summarizing Working Paper 31161
  9. Sumit Bachani and Eliza FitzGibbons. "How to Time the Chatbot-to-Human Handoff." Cognizant, February 9, 2026
  10. Office of Management and Budget. "M-24-10: Advancing Governance, Innovation, and Risk Management for Agency Use of Artificial Intelligence." March 28, 2024
  11. National Institute of Standards and Technology. "Artificial Intelligence Risk Management Framework: Generative AI Profile." NIST AI 600-1, July 26, 2024
  12. Tatiana Y. Lin. "Policy Analysis: City-Level Artificial Intelligence Policies and Guidelines." Kansas Health Institute, January 2025
  13. U.S. Department of Justice. "State and Local Governments: First Steps Toward Complying with the Americans with Disabilities Act Title II Web and Mobile Application Accessibility Rule." ADA.gov
  14. Maddox Hale. "How to Measure Why Customers Escalate From a Chatbot." Dimension Labs, October 27, 2026

Want our take on your AI roadmap?

We help leaders turn strategy into production AI systems. Let's talk about what you're building.