Abstract

Missing data are frequently encountered in the statistical analysis of randomized experiments. In this article, I propose statistical methods that can be used to analyze randomized experiments with a nonignorable missing binary outcome where the missing-data mechanism may depend on the unobserved values of the outcome variable itself even after taking into account the information in the fully observed variables. The motivating empirical example is the German election experiment where researchers are worried that the act of voting may increase the probability of participation in the post-election survey through which the outcome variable, turnout, was measured. To address this problem, I first introduce an identification strategy for the average treatment effect under the nonignorability assumption and compare it with the existing alternative approaches in the literature. I then derive the maximum likelihood estimator and its asymptotic properties, and discuss possible estimation methods. Furthermore, since the proposed identification assumption is not directly verifiable from the data, I show how to conduct a sensitivity analysis based on the parameterization that links the key identification assumption with the causal quantities of interest. Finally, the proposed methodology is extended to the analysis of randomized experiments with noncompliance. Although the method introduced in this article may not directly apply to randomized experiments with non-binary outcomes, I briefly discuss possible identification strategies in more general situations.

Related Papers

Blackwell, Matthew, Jacob R. Brown, Sophie Hill, Kosuke Imai, and Teppei Yamamoto (2025). ``Priming bias versus post-treatment bias in experimental designs.'' Political Analysis, Vol. 33, No. 4 (October), pp. 361-377. Winner of Political Analysis Editors' Choice Award.

Goplerud, Max, Kosuke Imai, Nicole E. Pashley (2025). ``Estimating Heterogeneous Causal Effects of High-Dimensional Treatments: Application to Conjoint Analysis.'' Annals of Applied Statistics, Vol. 19, No. 2 (June), pp. 866-888.

Ham, Dae Woong, Kosuke Imai, and Lucas Janson (2024). ``Using Machine Learning to Test Causal Hypotheses in Conjoint Analysis.'' Political Analysis, Vol. 32, No. 3 (July), pp. 329-344.

Johnson, Rebecca A., Tyler Simko, and Kosuke Imai (2024). ``A Summer Bridge Program for First-Generation Low-Income Students Stretches Academic Ambitions with No Adverse Impacts on GPA.'' Proceedings of the National Academy of Sciences, Vol. 121, No. 50, e2404924121.

de la Cuesta, Brandon, Naoki Egami, and Kosuke Imai (2022). ``Improving the External Validity of Conjoint Analysis: The Essential Role of Profile Distribution.'' Political Analysis, Vol. 30, No. 1 (January), pp. 19-45.

Egami, Naoki, and Kosuke Imai (2019). ``Causal Interaction in Factorial Experiments: Application to Conjoint Analysis.'' Journal of the American Statistical Association, Vol. 114, No. 526 (June), pp. 529-540.

Imai, Kosuke, and Zhichao Jiang (2018). ``A Sensitivity Analysis for Missing Outcomes Due to Truncation-by-Death under the Matched-Pairs Design.'' Statistics in Medicine, Vol. 37, No. 20 (September), pp. 2907-2922.

Imai, Kosuke, Dustin Tingley, and Teppei Yamamoto (2013). ``Experimental Designs for Identifying Causal Mechanisms.'' (with discussions) Journal of the Royal Statistical Society, Series A (Statistics in Society), Vol. 176, No. 1 (January), pp. 5-51. Read before the Royal Statistical Society in March, 2012.

Imai, Kosuke, and Teppei Yamamoto (2010). ``Causal Inference with Differential Measurement Error: Nonparametric Identification and Sensitivity Analysis.'' American Journal of Political Science, Vol. 54, No. 2 (April), pp. 543-560.

Ho, Daniel E., and Kosuke Imai (2008). ``Estimating Causal Effects of Ballot Order from a Randomized Natural Experiment: California Alphabet Lottery, 1978-2002.'' Public Opinion Quarterly, Vol. 72, No. 2 (Summer), pp. 216-240.

Imai, Kosuke (2008). ``Sharp Bounds on the Causal Effects in Randomized Experiments with ``Truncation-by-Death.'' Statistics & Probability Letters, Vol. 78, No. 2 (February), pp. 144-149.

Horiuchi, Yusaku, Kosuke Imai, and Naoko Taniguchi (2007). ``Designing and Analyzing Randomized Experiments: Application to a Japanese Election Survey Experiment.'' American Journal of Political Science, Vol. 51, No. 3 (July), pp. 669-687.

Ho, Daniel E., and Kosuke Imai (2006). ``Randomization Inference with Natural Experiments: An Analysis of Ballot Effects in the 2003 California Recall Election.'' Journal of the American Statistical Association, Vol. 101, No. 475 (September), pp. 888-900.