OTTAWA — Artificial intelligence experts are cautioning the Canada Revenue Agency (CRA) against leaning too heavily on AI to solve its service woes, saying the agency must first fix its human accuracy and training problems before expanding its chatbot programs.
The warning follows a scathing report by Auditor General Karen Hogan, who revealed that the CRA’s call centres provided accurate information to fewer than one in five Canadians seeking help with personal income tax questions. The report also found that the CRA’s existing AI tool, a scripted chatbot named Charlie, only gave correct answers about one-third of the time.
Despite these findings, the CRA is piloting a new generative AI system designed to answer a wider range of questions and operate for longer hours — a move experts say could be premature without fixing underlying human and data issues.
Anatoliy Gruzd, the Canada Research Chair in Privacy Preserving Digital Technologies, said the CRA needs to ensure that its human staff are getting accurate information right before scaling up AI. “If you’re a government agency launching a chatbot, you must have your human process nailed down first,” he said. “AI systems depend on good data — and that data comes from people.”
Adegboyega Ojo, a Carleton University professor specializing in AI governance, said the CRA should adopt a hybrid human-AI model, allowing machines to handle repetitive, simple inquiries while escalating complex or sensitive questions to trained agents. “AI can reduce workload and improve response times,” he said, “but humans remain essential — especially when context, empathy, or judgment are required.”
Ojo warned that even sophisticated chatbots can “get tripped up” without human oversight. “Accountability depends on maintaining that human check,” he added.
Jasmin Manseau, of the University of Ottawa’s Telfer School of Management, agreed that AI has potential but said trust and accuracy are critical. “Routine tasks can be automated, but not everything should be,” he explained. “The key is reaching a threshold of at least 70 to 90 per cent accuracy before the public will rely on it.”
While experts acknowledge that AI could play a key role in improving federal service delivery, they agree that the CRA’s credibility problem begins with people, not machines. As Gruzd put it bluntly: “If the humans can’t get the answers right, the algorithms won’t either.”