MetaAnalyst Methods......................................................................................................... 1
Table of Contents............................................................................................................ 1
Binary Data..................................................................................................................... 2
Continuity Corrections for 0 cells............................................................................... 2
Binary, 2 group........................................................................................................ 2
Binary, 1 group........................................................................................................ 3
Fixed effects analyses: Inverse Variance.................................................................... 3
Fixed effects analyses: Mantel-Haenszel method....................................................... 3
Fixed effects analyses: Peto (Odds Ratio only).......................................................... 4
Random effects analyses: DerSimonian-Laird model................................................. 4
Expectation Maximization........................................................................................... 4
Bayes........................................................................................................................... 4
Continuous Data.............................................................................................................. 6
Fixed effects analyses: Inverse Variance.................................................................... 6
Random effects analyses: DerSimonian-Laird model................................................. 6
Expectation Maximization........................................................................................... 6
Bayes........................................................................................................................... 6
Diagnostic Data............................................................................................................... 8
Fixed effects analyses: Inverse Variance.................................................................... 8
Random effects analyses: DerSimonian-Laird model................................................. 8
Expectation Maximization........................................................................................... 8
Fixed SROC................................................................................................................ 8
Bayes........................................................................................................................... 8
Random SROC........................................................................................................... 9
Bivariate...................................................................................................................... 9
References..................................................................................................................... 10
Denote the cells of binary data in the presentation of formulae using the following variable names:
|
Study i |
Event |
No Event |
|
Treatment |
ai |
bi |
|
Control |
ci |
di |
Currently, if any of the four cells (a through d) is zero, MetaAnalyst adds 0.5 to all cells the contingency table if any of the cell expectations would cause a division by zero error. This is otherwise called the Woolf-Haldane correction (for the odds ratio).1
The continuity correction is used in the following calculations:
|
|
|
Presentation of effect size per study |
Calculation of summary effect |
Comment |
|
|
Method |
Metric |
Point Estimate |
Standard Error |
|
|
|
Inverse variance |
Odds ratio |
Yes |
Yes |
Same as for the presentation per study |
Not defined when there are 0 events in both arms |
|
Inverse variance |
Risk ratio |
Yes |
Yes |
Same as for the presentation per study |
Not defined when there are 0 events in both arms |
|
Inverse variance |
Risk difference |
No |
Yes |
Same |
Defined when there are 0 events in both arms |
|
Peto |
Odds ratio |
Yes |
Yes |
No |
No need for corrections |
|
M-H |
Odds ratio |
Yes |
Yes |
Only when all studies have 0s in the same cell of the 2by2 table |
Otherwise, no corrections are used |
|
M-H |
Risk ratio |
Yes |
Yes |
Only when all studies have 0s in the same cell of the 2by2 table |
Otherwise, no corrections are used |
|
M-H |
Risk difference |
No |
Yes |
Only when all studies have 0s in the same cell of the 2by2 table |
Otherwise, no corrections are used |
|
Bayes |
Any metric |
No |
No |
No |
No need for corrections |
|
|
Event |
No Event |
|
Study i |
ai |
bi |
We add 0.5 when one of the two cells is 0 (proportion is 0% or 100%), so that the logit transformation results in quantities that can be defined.
Currently, the output of MetaAnalysts lists proportions per study using the continuity correction. So for a study that has 0/100 events, the proportion listed in the output is 0.005 rather than 0.000.
Inverse variance synthesis follows the usual formulae. The summary effect is a linear combination of the effects of the individual studies, where the weights are inversely proportional to the study variance and sum to unity. A review of the formulae is found in several textbooks.2,3
Here we implement the Mantel-Haenszel method for the RR, the OR and the RD using standard formulae.4
Here we implement the Peto assumption-free method for the OR using standard formulae.5 Although the Peto method is an “assumption-free”, it results in estimates similar to those from fixed effects analyses.
Here we implement a random effects model using the moment-based estimator of between-study variance proposed by DerSimonian and Laird.6 See also Fleiss and Gross (first extension to the OR metric).7
In the formula for the between-study variance we used the Mantel-Haenszel fixed effects summary estimate.
Based on the EM algorithm described in McIntosh (1996a, 1996b) generalized to allow for covariates.
The Bayesian model for binary data offers the option of either an exact method based on the binomial distribution for events in the treated and control groups or an approximate normal distribution for functions of the treatment effect (risk difference, log odds ratio, log risk ratio) and the control rate (logit). With the exact method (and the normal method for the risk difference), no correction for zero cells is necessary. In both formulations, the random effects distributions of the study treatment effects and control rates take a bivariate normal distribution centered about a mean (or regression).
The Bayesian analysis is computed via Markov chain Monte Carlo (MCMC) using the Open BUGS calculation engine.8 The actual models are shown in Appendix A. User inputs include the number of Monte Carlo iterations for each of a specifed number of chains as well as the number of runs to save. Convergence of the Markov chain is assessed using the Gelman Rubin convergence criteria of the relative size of between-chain to within-chain variation. Convergence is achieved when this ratio is less than 1.1 for all model parameters. Upon convergence, the last half of the accumulated runs are saved. If this number is not sufficient to achieve the number of saved runs desired, then the number of additional necessary runs are simulated and saved. If convergence is not achieved by the maxium number of runs allowed, then the algorithm will terminate with a message that convergence was not achieved. The user may wish to increase the number of iterations allowed as this may permit convergence to occur.
Starting values for the MCMC algorithm are chosen automatically by the program based on a least squares approximation to the estimated parameters based on means.
Noninformative priors (normal distributions with zero mean and large variance) are used for mean and regression parameters. Priors on variance parameters take an inverse gamma form with user input as to the scale and shape of the gamma distributions. One variance is needed for the control rate (if a comparison of two groups is being made) and one for the treatment effect.
Each model can include predictor variables in a meta-regression and will then produce posterior distributions of the regression parameters. One special type of meta-regression involves use of the rate of events in the control group as the predictor. In this case, the predictor is correlated with the outcome and is measured with error because both the control rate and the treatment effect are random variables (McIntosh, 1996). Meta-Analyst fits two types of control rate regressions. The first uses a linear model for the control rate and the second uses a quadratic model.
The standard output is a summary of the posterior distributions of each model parameter including its mean, standard error, median, 2.5th and 97.5th percentiles as well as the posterior probability that the parameter is greater than zero. The covariance and correlation matrices of the parameters are also available for display. The user may also request kernel density plots of each posterior by specifying the number of points at which to calculate the kernel densities.
Inverse variance synthesis follows the usual formulae. The summary effect is a linear combination of the effects of the individual studies, where the weights are inversely proportional to the study variance and sum to unity. A review of the formulae is found in several textbooks.2,3
Here we implement a random effects model using the moment-based estimator of between-study variance proposed by DerSimonian and Laird.6 See also Fleiss and Gross (first extension to the OR metric).7
In the formula for the between-study variance we used the Mantel-Haenszel fixed effects summary estimate.
Based on the EM algorithm described in McIntosh (1996a, 1996b) generalized to allow for covariates.
The Bayesian model for continuous data uses a normal distribution for the treatment effect and control mean. Tthe random effects distributions of the study treatment effects and control rates take a bivariate normal distribution centered about a mean (or regression).
The Bayesian analysis is computed via Markov chain Monte Carlo (MCMC) using the Open BUGS calculation engine.8 The actual models are shown in Appendix A. User inputs include the number of Monte Carlo iterations for each of a specifed number of chains as well as the number of runs to save. Convergence of the Markov chain is assessed using the Gelman Rubin convergence criteria of the relative size of between-chain to within-chain variation. Convergence is achieved when this ratio is less than 1.1 for all model parameters. Upon convergence, the last half of the accumulated runs are saved. If this number is not sufficient to achieve the number of saved runs desired, then the number of additional necessary runs are simulated and saved. If convergence is not achieved by the maxium number of runs allowed, then the algorithm will terminate with a message that convergence was not achieved. The user may wish to increase the number of iterations allowed as this may permit convergence to occur.
Starting values for the MCMC algorithm are chosen automatically by the program based on a least squares approximation to the estimated parameters based on means.
Noninformative priors (normal distributions with zero mean and large variance) are used for mean and regression parameters. Priors on variance parameters take an inverse gamma form with user input as to the scale and shape of the gamma distributions. One variance is needed for the control group (if a comparison of two groups is being made) and one for the treatment effect.
Meta-Analyst offers three choices (common, equal and unequal) for computing the treatment effect variance. The common and equal variances assumptions use a pooled variance (across treatment and control groups) to compute the treatment effect variance in each study. The difference is that the equal variance assumption uses a different pooled variance in each study, whereas the common variance assumption computes a single pooled variance across studies and then uses this in each study.With the unequal variance assumption, the treatment effect variance in each study is computed based on separate control and treatment group variances.
Each model can include predictor variables in a meta-regression and will then produce posterior distributions of the regression parameters. One special type of meta-regression involves use of the rate of events in the control group as the predictor. In this case, the predictor is correlated with the outcome and is measured with error because both the control rate and the treatment effect are random variables (McIntosh, 1996). Meta-Analyst fits two types of control rate regressions. The first uses a linear model for the control rate and the second uses a quadratic model.
The standard output is a summary of the posterior distributions of each model parameter including its mean, standard error, median, 2.5th and 97.5th percentiles as well as the posterior probability that the parameter is greater than zero. The covariance and correlation matrices of the parameters are also available for display. The user may also request kernel density plots of each posterior by specifying the number of points at which to calculate the kernel densities.
The diagnostic data module runs analyses for 8 different outcomes: sensitivity, specificity, accuracy, positive and negative predictive value, positive and negative likelihood ratio, and the diagnostic odds ratio. The first five outcomes are based on analyses of proportions, the likelihood ratios involved analyses of relative risks and the diagnostic odds ratio is of course based on methods for odds ratios.
Inverse variance synthesis follows the usual formulae. The summary effect is a linear combination of the effects of the individual studies, where the weights are inversely proportional to the study variance and sum to unity. A review of the formulae is found in several textbooks.2,3
Here we implement a random effects model using the moment-based estimator of between-study variance proposed by DerSimonian and Laird.6 See also Fleiss and Gross (first extension to the OR metric).7
In the formula for the between-study variance we used the Mantel-Haenszel fixed effects summary estimate.
Based on the EM algorithm described in McIntosh9-10 generalized to allow for covariates.
This implements the unweighted and weighted Summary ROC method.12
The Bayesian analysis is computed via Markov chain Monte Carlo (MCMC) using the Open BUGS calculation engine.8 The actual models are shown in Appendix A. User inputs include the number of Monte Carlo iterations for each of a specifed number of chains as well as the number of runs to save. Convergence of the Markov chain is assessed using the Gelman Rubin convergence criteria of the relative size of between-chain to within-chain variation. Convergence is achieved when this ratio is less than 1.1 for all model parameters. Upon convergence, the last half of the accumulated runs are saved. If this number is not sufficient to achieve the number of saved runs desired, then the number of additional necessary runs are simulated and saved. If convergence is not achieved by the maxium number of runs allowed, then the algorithm will terminate with a message that convergence was not achieved. The user may wish to increase the number of iterations allowed as this may permit convergence to occur.
Starting values for the MCMC algorithm are chosen automatically by the program based on a least squares approximation to the estimated parameters based on means.
Noninformative priors (normal distributions with zero mean and large variance) are used for mean and regression parameters. Priors on variance parameters take an inverse gamma form with user input as to the scale and shape of the gamma distributions.
Each model can include predictor variables in a meta-regression and will then produce posterior distributions of the regression parameters.
The standard output is a summary of the posterior distributions of each model parameter including its mean, standard error, median, 2.5th and 97.5th percentiles as well as the posterior probability that the parameter is greater than zero. The covariance and correlation matrices of the parameters are also available for display. The user may also request kernel density plots of each posterior by specifying the number of points at which to calculate the kernel densities.
Currently unavailable
The bivariate module allows simultaneous estimation of the sensitivity and specificity using a bivariate normal random effects distribution with a separate exact binomial distributions on the number of positive results among diseased and non-diseased individuals. It is a fit using a Bayesian model. See the Bayes sections for details.
(1) Schlesselman J, Stolley P. Case-control studies. Design, conduct, analysis. New York: Oxford University Press; 1982.
(2) Hedges LV, Olkin I. Statistical methods for meta-analysis. San Diego: Academic Press Inc.; 1985.
(3) Hedges LV. Fixed effects models. In: Cooper H, Hedges LV, editors. The handbook of research synthesis. 2nd ed. New York: Russel Sage Foundation; 1994. p. 285-301.
(4) Mantel N and Haenzel W. Statistical aspects of the analysis of data from retrospective studies of disease. J Natl Cancer Inst 1959; 22: 719-48.
(5) Yusuf S, Peto R, Lewis J, Collins R, Sleight P. Beta blockade during and after myocardial infarction: an overview of the randomized trials. Prog Cardiovasc Dis 1985; 27: 335-71.
(6) DerSimonian R, Laird N. Meta-analysis in clinical trials. Control Clin Trials 1986; 7: 177-88.
(7) Fleiss JL, Gross AJ. Meta-analysis in epidemiology, with special reference to studies of the association between exposure to environmental tobacco smoke and lung cancer: a critique. J Clin Epidemiol 1991; 44: 127-39.
(8) Thomas A, O'Hara B, Ligges U, Sturtz S. Making BUGS open. R News 2006; 6: 12-7.
(9) McIntosh M. Controlling for an ecological parameter in meta-analysis and hierarchical models. PhD Dissertation. Harvard University, Department of Statistics. Ann Arbor: University Microfilms, 1996.
(10) McIntosh M. The population risk as an explanatory variable in research synthesis of clinical trials. Statistics in Medicine 15: 1713-1728, 1996.
(11) Moses LE, Shapiro D and Littenberg B (1993). Combining independent studies of a diagnostic test into a summary ROC curve: data-analytic approaches and some additional considerations. Statistics in Medicine 12: 1293-1316.