Community pharmacies are increasingly turning to AI voice agents to handle an overwhelming volume of inbound patient calls. While automating appointment bookings and routine inquiries reduces staff burnout and expands access to care, deploying these AI agents in a clinical context introduces significant risk.
If a voice agent misunderstands a patient’s accent, offers unauthorized medical advice, or mishandles protected health information, the consequences can be severe. Yet, current industry evaluation approaches often fall short. Standard metrics like call resolution rates tell you if a call ended, but they provide little insight into why a failure occurs, or which patient demographics are most vulnerable to system errors.
MedMe Health recognized this gap. The Toronto-based digital health company powers over 4,500 pharmacies across North America, facilitating over 30 million patient interactions to date. As they looked to scale their AI Patient Concierge, they needed to ensure the technology was safe, equitable, and trustworthy.
MedMe came to Vector’s HealthSpark program to move beyond basic performance metrics and build a rigorous, scenario-driven AI evaluation methodology.
Stress-testing AI voice agents for the real world
Over four months, MedMe Health worked alongside Vector’s AI experts to design and implement a persona-based testing and evaluation framework that integrates directly into MedMe’s existing technology stack.
The goal was simple: proactively surface failure modes before the voice agent interacts with real patients. The teams developed a structured suite of patient personas to systematically probe the system’s safety and fairness across the Canadian population, accounting for variability in accents, speech patterns, language proficiency, and accessibility needs.
This was operationalized through a scalable, code-based testing pipeline. By introducing red teaming and tiered evaluation strategies, the framework goes beyond traditional quality assurance. It identifies precise vulnerabilities, such as medical advice violations or escalation failures, allowing MedMe to iteratively improve system logic, prompt design, and safety guardrails ahead of pilot deployment.
“The HealthSpark program enabled us to gain AI expertise and assurance via expert testing that the voice agents we are building are accessible and equitable to all across Canada.”
Scaling a responsible AI Patient Concierge
The HealthSpark collaboration accelerated MedMe’s AI Patient Concierge product development and delivered concrete economic impact. The project resulted in two commercialized products and services, with three prototypes ready for demonstration in an operational environment.
The partnership also drove significant talent and employment outcomes. Vector’s team trained and mentored MedMe staff members, transferring critical AI evaluation knowledge directly into the company. Furthermore, the project resulted in the creation of new jobs and the preservation of several existing roles within MedMe’s professional, science, and technology teams.
The robust evaluation plan developed through HealthSpark ensures that as MedMe’s AI Patient Concierge continues its commercial rollout, it will do so safely. For the thousands of pharmacies relying on MedMe’s infrastructure, this means access to an AI tool that doesn’t just answer the phone, but understands the patient on the other end of the line.
Vector’s HealthSpark program is supported by a Government of Canada investment through the Federal Economic Development Agency for Southern Ontario (FedDev Ontario).
