C-Path’s Electronic Clinical Outcome Assessment Consortium 15 Years of Progress

Introduction

Making progress is not easy. Making progress within a consensus-driven, pre-competitive collective adds a layer of complexity that attracts inertia as a consequence. Couple this with the persistent reality that everyone contributing to these efforts has an often-challenging job to do to pay the bills, and it can appear to be painfully close to impossible to achieve anything. And yet, since the very earliest discussions that Stephen J. Coons and his contemporaries had around the need to create a space to have both scientific and operational discussions around electronic patient-reported outcome (ePRO) use in clinical research, the ePRO Consortium — the predecessor to what is now the Electronic Clinical Outcome Assessment (eCOA) Consortium — has created and tended that space, and sought to drive understanding, alignment and evolution within the eCOA domain.

In my own eCOA career journey, which now exceeds 23 years since someone said the word “eDiary” in a meeting at what was Eli Lilly and permanently piqued my interest, I often found myself scouring industry commentary and various provider-published resources for any reference to an eDiary or ePRO in general, to what sort of consensus there was, and craving some level of regulatory endorsement of a position that would help assuage fears clinical teams would have when taking first steps in ePRO usage. I had no idea about the ePRO Consortium’s genesis at the time, but from what I know now about how driven the Consortium has been to fill those knowledge gaps, I wish I’d been involved sooner.

The eCOA Consortium has now been in existence for 15 years, and we who lead the collective today believe that this is well worth celebrating. The scope and membership profile of the Consortium has evolved significantly, and the ambition has expanded, but what it has always been since the very beginning is an entity that works tirelessly and diligently to bring about the changes that we need to see for the wider benefit of human health. We’ve had an impact for 15 years, and I know whoever is part of moving the Consortium forward, will remain driven to continue its legacy of maximum impact.

How it all Began

Stephen J. Coons had an idea. His academic research in the field of patient-reported outcomes (PRO) led to the realization that electronic methods for PRO data collection were generally best in clinical trials. Developing his knowledge and network within the nascent electronic PRO (ePRO) space, Stephen committed to advancing the practical and scientific underpinnings of ePRO applications; the 2005 dissertation of Sulabha Ramachandran, a graduate student of Stephen’s, compared paper and touch-screen device implementations of the EQ-5D, and became a key reference work in this emerging space.

Stephen J. Coons, former Vice President COA program, C-Path

When the draft FDA guidance Patient-Reported Outcome Measures: Use in Medical Product Development to Support Labeling Claims was released in February 2006, it recognized ePRO data collection as an acceptable approach. However, the guidance classified migration of a paper-based PRO measure to an ePRO device as a modification, and that it “generally plans to evaluate the modified instrument as a new one.” This position, if retained in the final version, was likely to discourage the development and use of ePRO systems — the nascent ePRO industry needed FDA to be receptive to electronic data collection without setting the measurement comparability bar inappropriately and unnecessarily high. FDA sought input from the scientific community, and in response to that need Stephen’s team at the University of Arizona, including Jason Lundy, convened the ePRO Consensus Development Working Group.

The working group met for the first time on March 23, 2006 to review the state of the science surrounding ePRO applications and discuss the recently released draft “PRO Guidance.” The working group included academic researchers, ePRO industry scientists, and pharma scientists, and was supported by various pharmaceutical companies as well as Assist Technologies (now AssisTek), ClinPhone and invivodata. The working group evolved, with FDA participation, into an ISPOR task force that generated “Recommendations on Evidence Needed to Support Measurement Equivalence between Electronic and Paper-Based Patient-Reported Outcome Measures: ISPOR ePRO Good Research Practices Task Force Report” in 2009.  Jason was a member of both the ePRO Consensus Development Working Group and the ISPOR ePRO Good Research Practices Task Force Report.

Stephen joined C-Path as Director of the PRO Consortium in June 2009, with Jason joining in January 2010.  The new measures being developed within the PRO Consortium were to be operationalized electronically, and those connections with the ePRO provider community would be crucial to that. In parallel, interest was burgeoning among the ePRO providers in creating a non-competitive space to collectively tackle open scientific questions and real or perceived operational issues impeding the adoption of ePRO. In addition, the ePRO providers wanted a more formal and collaborative relationship with the pharma industry members of the PRO Consortium to help address mutually impactful issues, particularly those related to regulatory acceptance of ePRO-generated data.

After a considerable amount of discussion within C-Path and with external stakeholders, it was decided that the ePRO Consortium could be established if 10 firms formally committed to become members. That proved to be an unrealistic goal at the time and was later reduced to five firms. Once the fifth firm signed the membership agreement in April 2011, C-Path established the ePRO Consortium and Jason was named its first director. The first ePRO Consortium Coordinating Committee meeting was held on May 22, 2011, and featured representation from the founding members CRF Health, ERT, ICON, invivodata and PHT.

Willie Muehlhausen, Safira Clinical Research

Achievements

Consortium founders co-authored the 2009 ISPOR Guidance Recommendations on Evidence Needed to Support Measurement Equivalence between Electronic and Paper-Based Patient-Reported Outcome Measures: ISPOR ePRO Good Research Practices Task Force Report in Value in Health.

Membership

• 2011: ePRO Consortium launched in April with five members: CRF Health, ERT, ICON, invivodata and PHT
• 2014: Reached 10 members, adding Bracket, Exco InTouch, Medidata, Parexel and YPrime
• 2020: MAPI Research Trust joined as the first non-eCOA provider member
• 2022: RWS joined as the first language service provider member
• 2024: Evinova join as first wholly sponsor-owned member
• 2024: Reached 25 members

Selected Publications & Initiatives

2014: Publication of first ePRO Consortium-driven and still highly-referenced article Capturing Patient-Reported Outcome (PRO) Data Electronically: The Past, Present, and Promise of ePRO Measurement in Clinical Trials in The Patient – Patient-Centered Outcomes Research in The Patient – Patient-Centered Outcomes Research

2018:

• Publication of first ePRO Consortium article exploring sensor-based wearable use in Selection of and Evidentiary Considerations for Wearable Devices and Their Measurements for Use in Regulatory Decision Making: Recommendations from the ePRO Consortium in Value in Health

• The self-published white paper Best Practices for Electronic Implementation of Response Scales for Patient-Reported Outcome Measures.

2019: 

• Perceived Burden of Completion of Patient-Reported Outcome Measures in Clinical Trials: Results of a Preliminary Study published in Spinger Nature, Therapeutic Innovation and Regulatory Science 
• Launched the eCOA: Getting Better Together Initiative (GBTI): A collaboration with the PRO Consortium and FDA

2020: 

• Published first outputs from GBTI created in the Process Workflow, Roles & Responsibilities project.
• Self-published white paper COVID-19: Risk Assessment and Mitigation Strategies for the Collection of Patient-Reported Outcome Data through Clinical Sites
• Demystifying Submissions of eCOA Documentation for Ethics Review: Are We Making Submissions More Difficult than Necessary? published in Applied Clinical Trials

2022:

• Measurement Comparability of Electronic and Paper Administration of Visual Analogue Scales: A Review of Published Studies published in Therapeutic Innovation and Regulatory Science
• Recommendations for the Electronic Migration and Implementation of Clinician-reported Outcome Assessments in Clinical Trials published in Value in Health*
• Comparability of a Provisioned Device Versus Bring Your Own Device for Completion of Patient-Reported Outcome Measures by Participants with Chronic Obstructive Pulmonary Disease: Qualitative Interview Findings published in Journal of Patient-Reported Outcomes*
• Launched the Translations and Licensing Management (TLM) project in collaboration with PRO Consortium and ISOQOL’s Translations and Cultural Adaptation Special Interest Group, the largest eCOA-related collaboration to date
• Launched the eCOA Consortium article series in Applied Clinical Trials 

2023:

• Launched the eCOA Educational Series featuring three different webinar types: eCOA Exchange, eCOA Clinic & eCOA Masterclass
• Launched the eCOA Forum, an in-person multistakeholder discussion held between members of the eCOA Consortium, PRO Consortium and FDA, with 2 further Virtual eCOA Forum events held across the year
• The first output from GBTI’s Support Flexible Approaches to PRO Data Collection project Flexible Approaches to eCOA Administration in Clinical Trials: The Site Perspective published in Contemporary Clinical Trials Communications*
• Best Practice Recommendations for Electronic Clinical Outcome Assessment Data Changes published in Journal of the Society for Clinical Data Management*
• Self-published white paper eCOA Systems and CE Certification
• Best Practice Recommendations for Electronic Patient-Reported Outcome (ePRO) Dataset Structure and Standardization to Support Drug Development published in Value in Health*

2024:

• Our seminal article Best Practices for the Electronic Implementation and Migration of Patient-Reported Outcome Measures is published in Value in Health
• Implementing Sensor-based Digital Health Technologies in Clinical Trials: Key Considerations from the eCOA Consortium is published in Clinical and Translational Science 
• GBTI project BYOD delivers final outputs*
• Completed the eCOA: Getting Better Together Initiative*

2025: 

• Best practice recommendations and considerations for designing and electronically implementing event-driven diaries in clinical trials published in Nature Digital Medicine
• Launched eCOA Conversations, the eCOA Consortium Podcast Series

2026: 

• Electronic Clinical Outcome Assessments and the European Union’s Clinical Trial Regulation: Smoothing the Path to Friction? published in Contemporary Clinical Trials Communications
• Launched the DHT Accelerator

* In collaboration with C-Path’s PRO Consortium, now the Patient-Centered Evidence Consortium

Face-to-Face Meetings

In-person interactions are crucial to our work.

Testimonials

Adam Butler, Ametris:

“When ePRO and eCOA in general were emerging as viable evidence generation tools, suitable for regulatory decision making, the burden of delivering compliant and usable tools was on the tech developers, not biopharma. Critical Path provided an essential forum for the community of technology developers to agree on the standards methods that would best serve patients, drug developers, and regulators. The increase in adoption of these data would not have been possible without a suitable pre-competitive forum. I feel confident saying that the way we collect and analyze patient data today is better than it was before the eCOA Consortium.”

Kelly Dumais, Clario:

“Getting to participate in C-Path’s eCOA Consortium activities is one of the highlights of my job. It is refreshing and so rewarding to see how people of all organizations come together with a combined purpose and pure desire to better the field of eCOA. The collaboration is a joy to watch, and the results we produce remind us of the important work we do, and significant responsibility we have as a Consortium.”

Shawn McKown, IQVIA:

“The eCOA Consortium is the most engaging and dynamic professional organization I’ve ever been a part of. The discussions are lively, frank, and productive, and the topics never fail to be timely and actionable. Scottie’s leadership style ensures that all perspectives are considered, and Pam and Alisa bring great skill and expertise to their roles.”

Estelle Haenel, Kayentis:

“eCOA Consortium is many things, but amongst all what it entails, it is a group of knowledgeable, interesting, inspiring and even friendly people, with whom thinking, proposing, exchanging, daring, and even criticizing (yes, truly!) and laughing (also) is always allowed… and without whom the science of eCOA would not advance with the same rigor and energy.”

Bill Byrom, Signant Health

Valdo Arnera, Clario

Flo Mowlem, uMotif

Kai Langel, DEEP Measures

Karl McEvoy, Suvoda

Paul O’Donohoe, Medidata Solutions

Closing

While C-Path and both the current and past membership of the eCOA Consortium can and should take pride in the work done and the impact realized, there is still much to do. We must carve the path away from embedded poor practices, do more to remove the points of friction in the process, and fanfare the unique scientific value that electronic methods of collecting the clinical trial participant experience can deliver.

The list of people I can and should thank and acknowledge would likely double the length of this webpage, but we would be remiss if we did not recognize selected individuals for their presence and influence on the Consortium history and its today. So to all those who were there at the start (Stephen J. Coons, Jason Lundy, Jean Paty, Willie Muelhausen), those who carried it forward and built it to be what it has become (Cindy Howry, Bill Byrom) those who have been a constant guiding presence for much of the Consortium’s existence (Paul O’Donohoe, Valdo Arnera, Jill Platko, Karl McEvoy), those who have been so pivotal to the success of recent years (Flo Mowlem, Elena Izmailova, Lisa Charlton, Shawn McKown, Shelly Steele, Ian Pallett, Estelle Haenel, Caroline Anfray, Mark Wade and many others), those in the C-Path team that have given so much to make it work (Theresa “T” Griffey, Sonya Eremenco, Christian Noll, Janelle Russell), the appreciation for all your contributions is deep. Finally, and most sincerely, to those that make it a pleasure to be part of (Pamela Dasher Lopez and Alisa Heinzman) my gratitude for you both is limitless.

Scottie Kern
Executive Director, eCOA