How to Handle Potential Biases When Dealing With Real-World Data Featuring Laura Fernandes
Subscribe on your favorite platform:
Summary
Real-world data holds great potential for clinical trials, and it can support clinical development and regulatory decision-making.
But when working with data, we need to be conscious of potential biases. In this episode of the Real World Talk podcast, our host Christie Zettler welcomes Laura Fernandes, Senior Statistical Director at COTA. They talk about the JSM conference, AML response validation work, and how to account for potential biases when dealing with RWD.
Highlights
- [00:59] The JSM Conference — Laura talks about her attendance at the JSM conference, which is short for Joint Statistical Meetings. It’s one of the largest conferences in the field of statistics.
- [03:07] Discussing the FDA draft guidance on real-world data — The FDA issued several guidance documents in the fall of 2021, and one of them was on electronic health records and how you could use them. Laura explains, “It covered different aspects, but the three main aspects that it covered were — what is the relevance of the data source? What were the different study elements that the data captures? How do you ensure that you have appropriate quality control checks in your database?”
- [04:29] AML response validation work overview — Laura provides an overview of the AML response validation work that COTA has done. She says, “I walked through the results of this validation study. And I put it into the context of using real-world data and an endpoint that is routinely collected in clinical trials.”
- [05:58] What criteria were used to make the comparison between physician-assessed response versus derived response? — Laura explains, “In a nutshell, if we can show that the response, as assessed by the physician in the real-world setting, is equal to or comparable to that derived by using these ELN criteria, then we can show that both the responses are comparable and equivalent, and we have, in some sense, validated this endpoint for this particular disease setting.”
- [07:28] Going from 3000 patients to 400 patients in the sample — COTA started the research with 3000 patients in the database. Laura talks about the criteria they applied for excluding or including patients in the analysis.
- [11:35] Explaining the Sankey diagram — Laura explains, “What the Sankey diagram shows over here is the patients who had a discordance in their response, so it’s 34.9%. And what it shows is the patients who had a CR, as calculated using the derived ELN criteria, where they fall as reported by the physician.”
- [14:33] Bias in real-world data — We typically hear about bias in real-world data in the context of an external control arm. Laura says that one of the most common biases is selection bias.
- [15:53] Disparities in clinical trials — There’s a lack of access to the general patient population. In the US, we don’t often have adequate representation in clinical trials, which is why disparities occur.
- [16:45] What is temporal bias? — Laura explains one of the most talked about biases in real-world data. She says, “We say that patients who were in your real-world setting, who were enrolled say 20 years ago — they might not be applicable to the patients who are being studied right now in 2022, which is a valid point.”
- [18:04] Discussing the observer bias — The observer bias can also occur in clinical trials when two independent radiologists look at the same tumor and report different outcomes.
Key Points
- We can improve the efficiency of clinical development by utilizing real-world data. Real-world data can make a huge difference in clinical trials and regulatory decision-making. But there are some potential biases we should be aware of, such as disparities in clinical trials, temporal bias, observer bias, etc.
- ELN criteria (or European LeukemiaNet criteria) is used to evaluate an endpoint in clinical trials. Laura talks about the ALM validation study and the criteria that they used. She says, “In the real-world setting, we have what is known as the physician-assessed response. This is what has been collected for a patient who receives AML therapy in the real-world setting. At the same time, we also have data that can be used to derive this response. And so we have data that is collected in the form of lab tests, bone marrow, biopsies, and various other tests that are done on a patient. So, these can be used to validate the outcome in clinical trials; it’s called the ELN criteria, which stands for European LeukemiaNet criteria. And the ELN criteria is what is actually used to evaluate an endpoint in a clinical trial.”
- What are the main types of bias in RWD? There are different types of bias that can occur when working with RWD in clinical trials. Laura mentions a couple of them — temporal bias and observer bias. She explains, “We say that patients who were in your real-world setting, who were enrolled say 20 years ago — they might not be applicable to the patients who are being studied right now in 2022, which is a valid point, but I feel that it is context-dependent. If the standard of care has not changed, like in the example of brain cancer or glioblastoma, where we’ve not had any new therapies, and the standard of care has been the same in this particular cancer type. So, in that particular instance, temporal bias might not be such a big issue.”
