The FDA has released important draft guidance on the use of real-world data (RWD) to support clinical trials, but many unanswered questions remain. How can the life sciences industry continue to make clinical trials faster, safer, and more effective?
In September 2021, the FDA published new draft guidance as part of its ongoing efforts to explore the use of real-world data (RWD) and real-world evidence (RWE) within its regulatory decision-making process.
The guidance breaks down many of the toughest challenges of using EHR data and medical claims data to support the safety and effectiveness of therapeutics, including the integrity and accuracy of complex datasets.
From data missingness to inconsistencies in endpoint validation, the guidance clearly states that we have much work to do before RWD and RWE can reach their full potential.
You might not think that a real-world data company like COTA would be happy to hear that. But we are, in fact, delighted that the FDA is recognizing the same issues we are seeing and solving for every day.
We applaud the Agency for challenging the industry to reflect on how they are collecting, curating, and leveraging RWD within their clinical research. And as we dig into the guidance, we have plenty of questions, comments, and ideas to continue pushing our community forward.
Data missingness: where do we draw the line?
The draft guidance begins with a discussion of “data missingness,” or gaps in key variables that may impact the outcomes of the analysis. Unfortunately, electronic healthcare data is prone to inconsistencies due to the industry’s complex infrastructure and varying methodologies for inputting data at the point of care.
The FDA makes a point of highlighting the issue and acknowledging that standard statistical workarounds may be applicable in some situations. But it stops short of providing a specific framework for when, where, and how those transformations may or may not be applied. For many clinical trial sponsors, this is a key concern. Without a detailed breakdown, how will they know which datasets will be accepted?
Future guidance will address the issue more thoroughly, the Agency assures stakeholders. Until then, clinical trial sponsors should focus on developing protocols and statistical analysis plans that are “based on an understanding of reasons for the presence and absence of information,” the FDA states.
Improving data generation methodologies to support strong science
The challenges of data missingness can be solved by working closely with clinicians at the keyboard to improve data integrity and quality at the point of care. Preventing data missingness from the beginning will ensure that curation and analytics tools function appropriately and that the conclusions drawn from RWD are trustworthy and repeatable.
To do so, we must engage with provider and professional organizations to change documentation behavior and patterns. Additionally, we need to invest in deep scientific thought about how to generate and collect the best possible data while simultaneously exploring innovative techniques to address missing data.
FDA will likely have a role to play in both activities. Additional guidance for healthcare providers would be helpful, as would sustained conversations with clinical trial sponsors about best practices for data gathering, curation, and analysis.
Creating consensus around meaningful data validation
Data validation is intended to foster general consensus around the legitimacy of a study. However, validation itself is a highly variable process. One can “validate” data through a peer reviewed article, a presentation, or a conversation, but that doesn’t necessarily mean that the findings are truly sound.
The FDA notes that the industry needs to improve its validation strategies, including the validation of study design elements, the linkage of disparate data sources, and the workings of artificial intelligence (AI) and machine learning algorithms. Strong, uniform validation criteria reduce the variability of interpretation and potentially make it faster and easier to work through the regulatory review process.
Just like with the issue of data missingness, however, the FDA has not provided stringent guardrails for how validation should be conducted. The guidance should be interpreted as a call to action – to gather forces in the RWD industry to develop a consensus. To that end, it is very possible we could develop a consensus on the primary areas of focus in this document: outcomes, exposure and covariates. The first priority should be exposure so we start a conversation on how to do this in the real world.
Supporting a RWD ecosystem that balances guidance with innovation
The FDA draft guidance is a helpful, necessary addition to the conversation about real-world data in therapeutic development. While the Agency clarified their position in some areas, there are still many questions about RWD that currently have no cut-and-dried answer.
We are glad to see that the FDA is being appropriately cautious, but not overly prescriptive, about the innovation currently taking place within the RWD environment. While we eagerly await their future contributions to the discussion, we are excited that the Agency is supportive of RWD and its game-changing role in clinical research.
The life sciences community is only at the beginning of what we can do with real-world data and real-world evidence. As we incorporate the FDA’s insights into our life-changing work, clinical trials will continue to evolve and advance to bring new therapies to patients in a safe, speedy, and impactful manner.
