Imagine you are a policy maker and an academic researcher presents you with evidence for a new health intervention that will significantly improve health outcomes. He shows you the study results, estimated effects, and p-values less than 0.05. How much confidence should you give this result? What quantitative approach should you take to determine whether the government should recommend this new health intervention?
One way to make this decision is to Estimated BAyeSian Interpretation (BASICE) method. BASIE was originally Mathematica Report 2019 (See other related papers at the end of this article). BASIE aims to estimate the likelihood that an intervention will have a meaningful impact, taking into account impact estimates and prior evidence on the effects of broadly similar interventions. The specific steps required to implement BASIE are as follows.
These steps should come as no surprise to those familiar with Bayesian methods. A key challenge in implementing Bayesian methods is choosing a good prior. For educational interventions, this paper recommends the use of What Works Clearinghouse (world women’s conference); in health, systematic literature reviews, Cochrane reviews or clinical guidelines may be useful starting points. When creating the prior, the authors caution to ensure that the population is homogeneous, that estimates are adjusted for sample size, and that the prior distribution is centered at 0.
When estimating intervention effects, the authors recommend using both traditional estimates (ie, based on study data only, with p-values) and shrink estimates, which shrink this estimate to the prior distribution.
When using shrinkage estimates, confidence intervals can also be generated from the posterior distribution. Confidence intervals are often considered the Bayesian approach to confidence intervals. However, confidence intervals should (i) be interpreted relative to selected prior distributions only, and (2) not be predictive statements about future effects, but retrospective statements about the effects of interventions in the assessed setting. For example, it can be said that Intervention X has a 90% chance of increasing survival by 10%, considering therapeutic trials and previous clinical trial evidence of drugs from the same therapeutic class treating the same disease. The probability that the intervention effect exceeds the minimum meaningful effect size should also be reported.
The report also includes R code to explain how to calculate the posterior distribution, the code below shows how to do this with a simple toy example. Although the BASICE method is applicable to educational intervention methods, the same statistical methods can also be used in health economics or any other field of science.
appendix
BASIE is mainly derived from the following academic studies:
- Gelman, A. (2011). Induction and deduction in Bayesian data analysis. Special Issue, Statistical Science and Philosophy of Science: Where did they (should) meet in 2011 and beyond? Reason, Markets, and Morality, 2, 67-78.
- Gelman, A. (July 15, 2015). Prior information, not prior beliefs. http://andrewgelman.com/2015/07/15/prior-information-not-prior-belief/
- Gelman, A. (April 23, 2016). What is a “true prior distribution”? A tough answer. http://andrewgelman.com/2016/04/23/what-is-the-true-prior-distribution-a-hard-nosedanswer/
- Gelman, A. and Hennig, C. (2017). Beyond Subjectivity and Objectivity in Statistics. Journal of the Royal Statistical Society, Series A (Social Statistics), 180(4), 967–1033.
- Gelman, A. and Shalizi, C. (2013). Philosophy and Practice of Bayesian Statistics (with discussion). British Journal of Mathematical and Statistical Psychology, 66, 8-80.
