In October 2021, Philippe Tadger was invited to deliver a one-day R software session at the Cochrane Anaesthesia, Critical and Emergency Care Group editorial meeting in Vienna. The session walked Cochrane reviewers through the practical R code needed to run — and to critique — modern random-effects meta-analyses, including the specific diagnostic-test-accuracy models that were the subject of his Hasselt University Master’s thesis. This article summarises what was taught, why it matters for clinical evidence, and where to read the full methodological background.
Why meta-analysis is a clinician’s problem
Every time a physiotherapist chooses a treatment or orders a test, they lean on evidence that was usually synthesised — somewhere up the chain — by a meta-analysis. Those synthesis methods are not interchangeable. The choice of random-effects model, the way heterogeneity is handled, the assumptions behind a prediction interval, and the statistical machinery for combining diagnostic studies all shape the pooled estimates that practitioners eventually read in a guideline. When the machinery is wrong, the clinical answer is wrong too — which is exactly why Cochrane invests in internal statistical training for its editors and authors.
What the Vienna session covered
1. Seven random-effects models, benchmarked
The session opened with the Jackson et al. (2018) comparison of seven random-effects models for meta-analysis of binary outcomes. In R, the attendees walked through running several of them side by side on the classic Olkin (1995) thrombolytic-therapy dataset:
- Model 1 — DerSimonian–Laird and its REML counterpart (
metafor::rma.uni,meta::metabinwithmethod.tau = "REML"). Simple, widely implemented, the pragmatic default when data are not rare. - Model 4 — UM.FS GLMM (
metafor::rma.glmm(..., model="UM.FS")). A modified Simmonds & Higgins model that Jackson et al. recommend as a serious alternative to the conventional approach. - Model 5 — UM.RS GLMM with random study-specific effects (
model="UM.RS"). - Model 7 — CM.EL conditional hypergeometric-normal (
model="CM.EL") and its approximate version CM.AL, both exact-likelihood approaches preferred for rare events and when statistical expertise is available.
The take-home for Cochrane editors: the conventional DerSimonian–Laird model remains usable because it is simple and robust, but once events become rare or heterogeneity becomes extreme, Model 4 (UM.FS) or Model 7 (CM.EL) deliver materially better estimates. Every example was shown with the full R code and a forest plot, so editors could reproduce each analysis back in their own reviews.
2. Prediction intervals — and why normality is a problem
A prediction interval answers the question most clinicians actually care about: “if I apply this treatment to my next patient, what range of effects can I expect?” The conventional prediction interval assumes the distribution of true effects is normal — an assumption that breaks down in small or heterogeneous reviews. The session introduced the pimeta package and its bootstrap prediction interval, which sidesteps the normality assumption entirely. A worked example on the ThirdWave psychotherapy dataset showed how to graft the bootstrap PI onto a meta::metagen model so the final forest plot displays a defensible “Bootstrap PI” line.
3. Bayesian meta-analysis of diagnostic test accuracy
The session’s most technically demanding section was the Bayesian analysis of diagnostic-test-accuracy (DTA) studies. Three models were compared on the classic Glas telomerase dataset:
- Reitsma’s normal–normal model (
mada::reitsma) — presented as a historical reference, not recommended in practice because it imposes a normality assumption that is rarely satisfied. - Binomial-normal bivariate random-effects model (
metafor::rma.mvon logit-transformed sensitivities and specificities) — a frequentist step up that respects the binomial nature of the data. - Bayesian binomial-normal with
bamdit— the preferred approach. It uses the exact binomial distribution, converges reliably, and accepts a binomial-mixture-of-normals alternative for severe heterogeneity. Example code ran a JAGS-backedmetadiag()fit with informative priors on the correlation, then plotted posterior densities for the between-study variances and the sensitivity–specificity correlation.
The visual output was a bivariate forest plot via DTAplots, posterior sensitivity/specificity densities, and diagnostic traceplots via coda — a complete Bayesian DTA workflow in a single RStudio session.
Where this connects to the Hasselt Master’s thesis
The Bayesian DTA section of the Vienna session was not a one-off lecture — it was a compressed version of the methodological work Philippe had completed at Hasselt University for his MSc Statistics master’s thesis. That thesis, titled “Meta-Research on Statistical Methods of Combining Diagnostic Studies”, systematically compares the statistical machinery used to pool diagnostic accuracy studies — the assumptions, failure modes, and practical guidance for picking one model over another. For anyone who wants to go deeper than a one-day course can allow, the thesis is the place to read the full derivation, simulation results and recommendations.
→ Full thesis on the Hasselt University repository: hdl.handle.net/1942/32304
Download the Vienna slide deck
The original slide deck from the Vienna session is available below. It contains the R code blocks discussed above along with the worked forest plots and the Bayesian DTA output. The file is access-controlled; please get in touch if you need the opening key for classroom use.
Takeaways
- Meta-analysis is not a single method — picking the right random-effects model matters, especially for rare events and high heterogeneity.
- Prediction intervals should use bootstrap methods when normality cannot be defended.
- Diagnostic-test-accuracy meta-analysis belongs in the Bayesian binomial-normal family, not the Reitsma normal-normal one.
- Cochrane editorial training is real work, and statistical fluency is part of what keeps a systematic review trustworthy.