Customer Facing Evaluations:
AI Evaluation at Abridge
Rigorous evaluation is a baseline requirement for developing AI products that clinicians can trust. At Abridge, where our products are trusted by tens of thousands of clinicians, we honor and maintain that trust through rigorous evaluation of those products. As we build, we constantly ask: “Is a new update ready to launch? Are the systems we have deployed behaving as expected? Are our products having a positive impact?” In other words: are our products trustworthy?
Answering these questions is a team effort, involving clinicians, engineers, machine learning researchers, product managers, and statisticians. By collaborating across areas of expertise, we develop evaluations that are informed not only by frontier AI research, but also by domain expertise, quantitative and qualitative research methods, user experience, and engineering considerations.
This blog post is the first in a series where we provide an overview of our multi-faceted approach to evaluation. Along the way, we will discuss the role that different evaluations play in assessing the trustworthiness of our AI products and their readiness for enterprise deployment.
In this first post, we highlight the breadth of our evaluations and present an organizing framework; subsequent posts present specific evaluations within the context of the framework. This framework is based on three questions that characterize any evaluation we run: when do we evaluate, what do we evaluate, and how do we evaluate?
Evaluation plays a role in every part of the product life cycle, and our evaluations look different depending on when in the life cycle they are conducted.
Every system we build first undergoes “offline” pre-deployment evaluations. These evaluations verify that a system is sufficiently accurate, fair/equitable, safe, and usable to be previewed to clinicians.
Our offline evaluations rely on datasets curated from de-identified Abridge encounters. These datasets are chosen to reflect particular settings and properties that are relevant to the system in question. For example, to evaluate how systems for generating cardiology notes handle complex patient histories, we select a set of cardiology encounters that have multiple prior procedures and diagnoses. The outputs of the system on these curated encounters are used to compute automated metrics, and are also reviewed by physicians in blinded head-to-head comparisons.
Systems that clear the quality bar in offline evaluation proceed to a staged rollout process, which for net-new features is a limited release with a curated set of partners who act as early adopters. During the staged rollout we carefully review feedback from these early adopters and monitor the system for unexpected behaviors or unanticipated edge cases, iterating as necessary until we are confident that it is ready for broader release.
Once a system shows sufficient quality or improvement in both pre-deployment evaluations and staged rollout to fully deploy, we begin “online” evaluations. These evaluations run continually on our deployed systems, to monitor quality at scale on live production traffic. They use a variety of signals (from automated evaluations to user feedback) to help us validate that output quality is maintained across the diverse range of partners that we serve, and guard against any possible drift in output quality. These evaluations also help us identify challenging scenarios for our systems that (in turn) inform the improvement of our offline pre-deployment evaluations.
Beyond the evaluations we run internally, we also provide technical support to research teams at partner health systems to run independent studies evaluating our products. A recent Nature paper reports that of 4,609 peer-reviewed studies evaluating LLMs in clinical medicine, only 19 were prospective randomized trials. Through our technical support, we enable external research teams to conduct prospective randomized trials evaluating the impact of Abridge products, as well as other studies that help deepen our understanding of ambient AI.
At any given point in the product life cycle, we aim to cover many different dimensions of a product with our evaluations.
The table stakes for any product are safety (e.g. generating notes that are free of clinically relevant errors) and accuracy (e.g. generating notes that use appropriate medical terminology). Beyond these basic dimensions, we measure fairness/equity (e.g. generating notes of equal quality across diverse clinician and patient populations) and usability (e.g. integrating easily into clinician workflows). For each of these four dimensions, we rely on established research and domain expertise in conceptualizing and operationalizing the quality dimension to inform a range of more specific metrics. We describe this process in detail in subsequent posts.
When operationalizing these dimensions, we consider both intrinsic attributes of a system alone, and outcome-based attributes that reflect how users interact with and rely on the system. For our clinical decision support product, intrinsic measures of accuracy include completeness of responses, while outcome-based measures of accuracy include diagnostic quality, which depends on how the clinician follows through on responses. For our nursing product, intrinsic measures of fairness/equity include whether the system is equally accurate regardless of patient demographics, while outcome-based measures of fairness/equity include whether improvements to patient experience are shared across sensitive groups.
Intrinsic attributes tell us how good our systems are, while outcome-based attributes tell us whether our systems are having a positive impact on the health systems we serve, and we need to measure both to understand the end-to-end quality of our products.
Once we know when we are evaluating and what we are evaluating, we choose an evaluation methodology.
In the offline (pre-deployment) setting, we often evaluate using structured feedback provided by expert clinical reviewers. Beyond collecting structured feedback on known dimensions of quality, we also collect feedback on the holistic preferences of clinical reviewers, who compare the outputs of different systems (while blinded to the system that generated the output) and indicate the outputs they prefer. For structured tasks with correct answers, we collect expert annotations of the correct answer for various instances, and then construct internal benchmarks against which we can track progress. These include a benchmark for evaluating medical calculator use by agents as well as a benchmark for measuring the accuracy of medical coding models.
For many of our internal datasets and benchmarks, we collect annotations on de-identified encounters where Abridge was used, to reflect the complexity and challenges of real-world healthcare settings.
By back-testing on de-identified encounters we are able to evaluate our systems in realistic settings that publicly available and external data rarely reflect.
In the online setting, we receive feedback from our clinicians, which can be explicit or implicit. Explicit feedback comes to us in a variety of ways, from clinicians leaving star ratings to partners surfacing specific errors they identified in the process of reviewing and revising the generated note. Implicit feedback, in contrast, comes to us through the edits clinicians make to generated notes, or the kinds of follow-up questions they ask our clinical decision support agent. Covering these two kinds of feedback in our evaluations allows us to resolve specific pain-points for customers and also track the overall customer experience.
Finally, we use additional AI systems (often referred to as “LLM Judges”) to perform automated evaluations of our core systems in both the offline and online settings. Offline, LLM judges provide a quick signal on model improvement that is useful for early iteration; online, LLM judges are well-suited to handling the large volume of production traffic we receive from our clinicians. These LLM judges are themselves first validated offline on clinician annotations to ensure that the results they give are consistent with clinician adjudications.
The remaining posts in this series will explore different evaluations we run at Abridge and contextualize them within the when-what-how framework introduced above, starting with a deep-dive into our note generation evaluations. We will begin with a look at how we define what we are measuring when we talk about note quality, then describe one of the methods we use for our offline evals, and conclude with two posts covering several aspects of our online note quality evaluations that we are most often asked about in greater depth. Subsequent posts will show how these principles extend to other products, like clinical decision support, nursing flowsheet generation, and clinical documentation integrity.
Throughout this series, we emphasize that evaluation is a process and not a destination: no static set of evaluations can cover the growing set of quality dimensions we and our clinicians continually identify. Our product and evaluation teams are always improving upon existing evaluations and building new evaluations to obtain a more complete picture of product quality.
By continually expanding the breadth of our evaluation suite, we steadily increase the trustworthiness of our products, and continue to earn the trust of the clinicians who rely on the products we build.