Blair, Graeme, Winston Chou, and Kosuke Imai. ``List Experiments with Measurement Error.'' Political Analysis, Forthcoming.



Measurement error threatens the validity of survey research especially when studying sensitive questions. In the context of list experiments, Ahlquist (2017) introduces the notion of ``top-biased'' response error, in which a random fraction of respondents provide the maximal response regardless of their truthful answer to the sensitive question. Ahlquist conducts simulation studies based on this scenario and finds that the maximum likelihood (ML) regression estimator, proposed in Imai (2011) and further extended in Blair and Imai (2012), exhibits severe model misspecification bias when the sensitive trait is rare. Unfortunately, Ahlquist stops short of offering any solution to the general problem of measurement error in list experiments. We take up this challenge and provide new tools for diagnosing and mitigating measurement error in list experiments. First, we point out that top-biased error is unlikely for truly sensitive questions, as it implies that respondents are willing to admit having a sensitive trait even when they do not. Second, we show that the nonlinear least squares (NLS) regression estimator is robust to top-biased error. Third, we consider an alternative form of response error, mentioned but not studied in Ahlquist (2017), in which a small fraction of respondents offer a random response to the list experiment. We show that both ML and NLS regression estimators are robust to such error. Fourth, we propose a statistical test for detecting general model misspecification caused by misreporting. Fifth, we demonstrate how to directly model nonstrategic respondent error and how to build a more robust regression model. Finally, we reanalyze the empirical examples studied in Ahlquist (2017) and demonstrate that simple diagnostic tools can be used to avoid the problems identified in the original article. We conclude this article with a set of practical recommendations for applied researchers. The software package is made available to implement the proposed methodology.

  Other Information

Please see this page for the information about the project on the elicitation of truthful answers to sensitive survey questions.
The software package that implements the proposed method is available here for download.

© Kosuke Imai
 Last modified: Sat Sep 15 11:26:32 EDT 2018