Managing overlapping review studies in a second-order meta-analysis

Describing and exploring existing strategies

Abstract

This study explores different techniques for managing overlaps in second-order systematic reviews to highlight promising practices for open, distance, and digital education (ODDE) research. Second-order reviews, also known as overviews, provide unique evidence to better describe mean effects and assess their variability, but they also introduce new methodological challenges for quality control due to overlapping primary studies. Recent efforts to categorize and evaluate best practices for managing overlaps in other disciplines highlight significant inconsistencies in practice. As the frequency of ODDE systematic reviews and meta-analyses grows, there is a need to produce high-quality second-order studies to ensure policy makers can properly translate evidence into practice. Several leading overlap management methods are examined, including visualization tools such as citation matrices, bipartite citation networks, stacked bar charts, and intersection heatmaps; quantification tools such as corrected covered area (CCA), adjusted CCA and weighted CCA; decision-based and researcher-based approaches for resolving overlap. These approaches, largely developed in healthcare contexts, are illustrated using an existing dataset from a recent overview (Sun et al., 2025). CCA quantifies the degree of overlap among primary studies, while adjusted CCA further accounts for structural missingness arising when particular primary studies could not have been included in certain reviews because of differences in eligibility or timing. Although wCCA incorporates primary-study sample size into the assessment of overlap, it may be difficult to apply when information is unavailable. Existing thresholds for managing overlaps are largely based on practical guidance, lacking empirical evidence supporting specific cut-offs as decision thresholds.