Suppose you have an international clinical trial showing that a new drug (SuperDrug) is better than the previous standard of care (OldDrug). It was also hypothesized that individuals with specific comorbidities (which we refer to as EF) would be less responsive to SuperDrug treatment. If you live in a country where EF comorbidities are common, how effective do you think SuperDrug would be in your population?
This is the question asked Turner et al. (2023) in their latest pharmacoeconomics Paper. Common issues facing national policymakers include the following:
External validity is more uncertain when the study population is not randomly selected from the target population, and the distribution of effect modifiers (characteristics that predict changes in treatment effect) may differ between the trial sample and the target population.
Many of you may have guessed that my comorbidity EF actually stands for effect modifier. The four categories of effect moderators considered by the authors include:
- Patient/disease characteristics (e.g. biomarker prevalence),
- environment (e.g. place of care and access to care),
- Treatment (e.g., timing, dose, comparator therapy, concomitant medications)
- results (such as follow-up actions or
- measure time)
look Beer et al. (2022) Gets a potential list of effect modifiers.
In their paper, the authors examine the transportability issue.what is Transportability?
Generalizability relates to whether the study’s inferences can be extended to the target population sampled by the study dataset, whereas transferability relates to whether the study’s inferences can be extended to the target population sampled by the study dataset.
Inferences can be extended to separate (external) groups from which the study sample did not originate.
Key cross-country differences that may contribute to transportability issues include effect moderators
Examples include disease characteristics, comparative therapies, and treatment settings.
What are the questions of interest:
Typically, decision makers are interested in the target population average treatment effect (PATE): the average effect of a treatment if all individuals in the target population were assigned the treatment. However, researchers often only have access to the sample and must estimate the study sample mean treatment effect (SATE).
The key assumptions for estimating PATE are as follows:
First, there are two key questions that need to be addressed (at least for randomized controlled trials): (i) whether there are differences in the distribution of characteristics between studies and populations in the target countries, and (ii) whether these characteristics have effect-modifying factors [or for single arm trials with external controls, prognostic factors].
One can test for differences in covariate distributions using mean differences in propensity scores, examining propensity score distributions, and formal diagnostic tests to determine whether overlap exists. Univariate standardized mean differences (and related tests) can then be used to examine the drivers of overall differences. If only summary information is available, you may be limited to comparing differences in average values.
To test whether a variable is an effect modifier, the author recommends the following approach:
Parametric models with treatment-covariate interactions can be used to detect effect modification.Small research sample leads to power issues or situations where functionality is unknown
form increases the risk of model misspecification, consider machine learning techniques such as Bayesian additive regression trees, and the use of directed acyclic
In this case, the chart may be particularly important for selecting effect modifiers.
Methods for adjusting effect modifiers will vary depending on whether the study has access to individual patient data.
- With IPD: Using outcome regression-based methods, matching, stratification, inverse odds with participation weights, and a doubly robust approach that combines matching/weighting with regression adjustments.
- Does not contain IPD. Use population-adjusted indirect treatment comparisons (e.g., pairwise-adjusted indirect comparisons).
To determine which domestic data (usually real-world data) should be used for the target population, a number of tools can be considered, e.g. EUnetHTA requirements or data suitability assessment
tool(data satellite) tool from NICE.
You can read more advice on how to best verify transportability issues in the full article here.
