Imai, Kosuke, Gary King, and Clayton Nall. (2009). ``The Essential Role of Pair Matching in Cluster-Randomized Experiments, with Application to the Mexican Universal Health Insurance Evaluation.'' (with discussions and rejoinder) Statistical Science, Vol. 24, No. 1 (February), pp. 29-53.

Abstract

A basic feature of many field experiments is that investigators are only able to randomize clusters of individuals - such as households, communities, firms, medical practices, schools, or classrooms - even when the individual is the unit of interest. To recoup the resulting efficiency loss, some studies pair similar clusters and randomize treatment within pairs. However, many other studies avoid pairing, in part because of claims in the literature, echoed by clinical trials standards organizations, that this matched-pair, cluster-randomization design has serious problems. We argue that all such claims are unfounded. We also prove that the estimator recommended for this design in the literature is unbiased only in situations when matching is unnecessary; and its standard error is also invalid. To overcome this problem without modeling assumptions, we develop a simple design-based estimator with much improved statistical properties. We also propose a model-based approach that includes some of the benefits of our design-based estimator as well as the estimator in the literature. Our methods also address individual-level noncompliance, which is common in applications but not allowed for in most existing methods. We show that from the perspective of bias, efficiency, power, robustness, or research costs, and in large or small samples, pairing should be used in cluster-randomized experiments whenever feasible; failing to do so is equivalent to discarding a considerable fraction of one’s data. We develop these techniques in the context of a randomized evaluation we are conducting of the Mexican Universal Health Insurance Program.

Replication Archive

Software

experiment: R Package for Designing and Analyzing Randomized Experiments — CRAN

Related Papers

Sengupta, Souhardya, Kosuke Imai, and Georgia Papadogeorgou (2025). ``Low-rank Covariate Balancing Estimators under Interference.''
Mukaigawara, Mitsuru, and Kosuke Imai, Jason Lyall, and Georgia Papadogeorgou (2025). ``Spatiotemporal causal inference with arbitrary spillover and carryover effects.''
Tarr, Alexander and Kosuke Imai (2025). ``Estimating Average Treatment Effects with Support Vector Machines.'' Statistics in Medicine, Vol. 44, No. 5, e70006.
Imai, Kosuke, In Song Kim, and Erik Wang (2023). ``Matching Methods for Causal Inference with Time-Series Cross-Sectional Data.'' American Journal of Political Science, Vol. 67, No. 3 (July), pp. 587-605.
Fan, Jianqing, Kosuke Imai, Inbeom Lee, Han Liu, Yang Ning, and Xiaolin Yang (2023). ``Optimal Covariate Balancing Conditions in Propensity Score Estimation.'' Journal of Business & Economic Statistics, Vol. 41, No. 1, pp. 97-110.
Bertsimas, Dimitris, Kosuke Imai, and Michael Lingzhi Li (2023). ``Distributionally Robust Causal Inference with Observational Data.''
Papadogeorgou, Georgia, Kosuke Imai, Jason Lyall, and Fan Li (2022). ``Causal Inference with Spatio-temporal Data: Estimating the Effects of Airstrikes on Insurgent Violence in Iraq.'' Journal of the Royal Statistical Society, Series B (Statistical Methodology), Vol. 84, No. 5 (November), pp. 1969-1999.
Imai, Kosuke and In Song Kim (2021). ``On the Use of Two-way Fixed Effects Regression Models for Causal Inference with Panel Data.'' Political Analysis, Vol. 29, No. 3 (July), pp. 405-415.
Imai, Kosuke and James Lo (2021). ``Robustness of Empirical Evidence for the Democratic Peace: A Nonparametric Sensitivity Analysis.'' International Organization, Vol. 75, No. 3 (Summer), pp. 901-919.
Ning, Yang, Sida Peng, and Kosuke Imai (2020). ``Robust Estimation of Causal Effects via High-Dimensional Covariate Balancing Propensity Score.'' Biometrika, Vol. 107, No. 3 (September), pp. 533-554.
Zhao, Shandong, David A. van Dyk, and Kosuke Imai (2020). ``Propensity-Score Based Methods for Causal Inference in Observational Studies with Non-Binary Treatments.'' Statistical Methods in Medical Research, Vol. 29, No. 3 (March), pp. 709-727.
Imai, Kosuke and In Song Kim (2019). ``When Should We Use Unit Fixed Effects Regression Models for Causal Inference with Longitudinal Data?.'' American Journal of Political Science, Vol. 63, No. 2 (April), pp. 467-490.
Imai, Kosuke, and Zhichao Jiang (2018). ``A Sensitivity Analysis for Missing Outcomes Due to Truncation-by-Death under the Matched-Pairs Design.'' Statistics in Medicine, Vol. 37, No. 20 (September), pp. 2907-2922.
Fong, Christian, Chad Hazlett, and Kosuke Imai (2018). ``Covariate Balancing Propensity Score for a Continuous Treatment: Application to the Efficacy of Political Advertisements.'' Annals of Applied Statistics, Vol. 12, No. 1, pp. 156-177.
Imai, Kosuke and Marc Ratkovic (2015). ``Robust Estimation of Inverse Probability Weights for Marginal Structural Models.'' Journal of the American Statistical Association, Vol. 110, No. 511 (September), pp. 1013-1023.
Imai, Kosuke and Marc Ratkovic (2014). ``Covariate Balancing Propensity Score.'' Journal of the Royal Statistical Society, Series B (Statistical Methodology), Vol. 76, No. 1 (January), pp. 243-263.
Ho, Daniel E., Kosuke Imai, Gary King, and Elizabeth Stuart (2011). ``MatchIt: Nonparametric Preprocessing for Parametric Causal Inference.'' Journal of Statistical Software, Vol. 42, No. 8 (Special Volume on Political Methodology), pp. 1-28.
King, Gary, Emmanuela Gakidou, Kosuke Imai, Jason Lakin, Ryan T. Moore, Clayton Nall, Nirmala Ravishankar, Manett Vargas, Martha María Téllez-Rojo, Juan Eugenio Hernández Ávila, Mauricio Hernández Ávila, and Héctor Hernández Llamas (2009). ``Public Policy for the Poor? A Randomised Assessment of the Mexican Universal Health Insurance Programme.'' (with a comment) The Lancet, Vol. 373, No. 9673 (April), pp. 1447-1454.
Imai, Kosuke, Gary King, and Clayton Nall (2009). ``Rejoinder: Matched Pairs and the Future of Cluster-Randomized Experiments.'' Statistical Science, Vol. 24, No. 1 (February), pp. 65-72.
Imai, Kosuke (2008). ``Variance Identification and Efficiency Analysis in Randomized Experiments under the Matched-Pair Design.'' Statistics in Medicine, Vol. 27, No. 24 (October), pp. 4857-4873.
Imai, Kosuke, Gary King, and Elizabeth A. Stuart (2008). ``Misunderstandings among Experimentalists and Observationalists about Causal Inference.'' Journal of the Royal Statistical Society, Series A (Statistics in Society), Vol. 171, No. 2 (April), pp. 481-502. Reprinted in Field Experiments and their Critics, D. Teele ed. (2014), New Haven: Yale University Press.
Ho, Daniel E., Kosuke Imai, Gary King, and Elizabeth A. Stuart (2007). ``Matching as Nonparametric Preprocessing for Reducing Model Dependence in Parametric Causal Inference.'' Political Analysis, Vol. 15, No.3 (Summer), pp. 199-236. Winner of Miller Prize.
Imai, Kosuke (2005). ``Do Get-Out-The-Vote Calls Reduce Turnout? The Importance of Statistical Methods for Field Experiments.'' American Political Science Review, Vol. 99, No. 2 (May), pp. 283-300.
Imai, Kosuke, and David A. van Dyk (2004). ``Causal Inference With General Treatment Regimes: Generalizing the Propensity Score.'' Journal of the American Statistical Association, Vol. 99, No. 467 (September), pp. 854-866.
© Kosuke Imai