Trust gap limits AI adoption in primary care

AI-enabled clinical decision support (AI-CDS) tools promise to improve clinical workflows, reduce administrative burden and help clinicians apply evidence-based care, but a market analysis by the Centre for Effective Practice concluded that concerns about privacy, bias and transparency may have to be addressed before widespread adoption occurs.In a March 25 webinar presented by Digital Health Canada, Valentina Gnanapragasam, clinical design specialist at the Centre for Effective Practice, revealed positive trends in the number and diversity of solutions that have the potential to support primary care clinicians. However, the analysis also discovered a lack of transparency, standardization and accountability as well as limited visibility into how the tools work.“These are not minor issues,” she complained. “They directly impact clinical trust, patient safety and equity. AI-CDS has the potential to transform primary care, but the market will not naturally align with our system needs on its own.”The market analysis covered 53 solutions through an eScan and responses to a request for information from 35 vendors. Gnanapragasam revealed that 91% of solutions claimed to be AI-powered, but only 58% explicitly reveal any AI-related information and only 15% provided details on their underlying methodology.“When we assess methodology,” she explained, “we want to understand the type of model used and how it functions, the data it was trained on, including whether it reflects Canadian populations, the recency and source of the evidence base and whether recommendations are clearly linked to supporting evidence.”According to Gnanapragasam, more than half of vendors disclose some information about their evidence base, but the descriptions are often vague, mentioning peer-reviewed journals or clinical guidelines without specifying precise sources. Also, only 6% reference Canadian clinical sources.Among other findings, only 21% cite PIPEDA compliance, only 11% indicate completion of a privacy assessment and only 34% explicitly state that they do not use patient data for secondary purposes.“This creates an unacceptable risk and uncertainty regarding the potential resale of patient data or its use in commercial research without clear consent,” said Gnanapragasam. “Without knowing about the training data, we cannot assess whether these tools introduce or amplify bias (and) without understanding how models work, we cannot anticipate issues like hallucinations or performance drift.”Emily Ha, research associate at Women’s College Hospital Institute for Health Systems Solutions and Virtual Care, conducted interviews with a group of 10 primary care clinicians to understand how AI-CDS tools align with physician needs and expectations. Burdened as they are by administrative tasks, clinicians warned that if AI-CDS tools are made available to them but don’t save them time in their practice, they are unlikely to be used.Ha also found that familiarity with AI-CDS tools was limited and that a lot of clinicians questioned whether they were even necessary. “Adoption really wasn’t so much a technical issue as it was about perception and trust,” she said.Doctors expressed interest in AI-CDS tools that reduce administrative burden, support screening, proactive care and chronic disease management, but favoured on-demand rather than intrusive diagnosis and treatment support. They also want to know how recommendations and information pulled from AI-CDS tools are generated.Ariane Siegel, general counsel and chief privacy officer at OntarioMD, outlined some of the risks that came to light following an environmental scan of AI-CDS tools through a legal and regulatory lens. They include privacy and security risks, secondary use of data, data residency, bias and the intelligence of the tools. “These are some of the critical issues that we’re going to have to tackle in the risk landscape,” she noted.Dr. Ali Damji, clinical lead, digital implementation of quality initiatives at Ontario Health, provided an update on the Evidence2Practice (E2P) program. E2P tools offer up-to-date evidence, quality standards and best practices for heart failure, anxiety and depression, diabetes, COPD and sickle cell disease.They were developed, said Dr. Damji, because of a 17-year gap between high quality evidence coming out and changes in front line clinical practice, leading to “40% of patients not getting the high-quality care that they should be getting and even 25% of care being unnecessary or even harmful.”The tools are integrated with hospital information systems and the most widely used EMRs to better align with clinical workflows and have improved adherence to best practices for heart failure and reduced time to first opioid use for sickle cell patients presenting with acute pain.Close to 13,000 acute care clinicians have access to E2P tools based on 35 implementations across 22 hospitals. In primary care, 4,754 primary care clinicians have access to E2P tools through their EMRs and more than 650 of them are active users.Dr. Damji outlined plans for the further development of the E2P program, including increased usage, the addition of new conditions, and the introduction of AI.“We are also looking at things like dashboards, which would allow us to see the most important, relevant data that affects clinical care so we can track things over time, revisit scoring tools and use that to guide our clinical decision-making in keeping with the evidence,” he said.Alongside E2P, he added, “AI can enhance a physician’s efficiency by tackling routine challenges and cognitive load” by summarizing information about a patient prior to an appointment, sourcing guidelines, checking for drug interactions and allergies in real time during an appointment and automatically completing post-visit tasks.“At the end of the day,” he concluded, “there’s going to be a lot of AI out there, but we need to be able to cut through the noise and find the ones that are actually going to help our patients. We want to make sure that it’s safe, that it’s secure, that the legal elements have been fully explored and that there’s clinical value. Ultimately, we want to test these things out, do a proof of concept through a living lab and use the results to scale up a broader implementation and adoption across the province.”

Trust gap limits AI adoption in primary care

Register for free access or login.

Related Topics