Conjoint analysis is a popular
experimental design used to measure multidimensional
preferences. Many researchers focus on estimating the average
marginal effects of each factor while averaging over the other
factors. Although this allows for straightforward design-based
estimation, the results critically depend on the ways in which
factors interact with one another. An alternative model-based
approach can compute various quantities of interest, but requires
correct model specifications, a challenging task for conjoint
analysis with many factors. We propose a new hypothesis testing
approach based on the conditional randomization test (CRT) to answer
the most fundamental question of conjoint analysis: Does a factor of
interest matter in any way given the other factors? Although it only
provides a formal test of these binary questions, the CRT is solely
based on the randomization of factors, and hence requires no
modeling assumption. This means that the CRT can provide a powerful
and assumption-free statistical test by enabling the use of any test
statistic, including those based on complex machine learning
algorithms. We also show how to test commonly used regularity
assumptions. Finally, we apply the proposed methodology to conjoint
analysis of immigration preferences. An open-source software package
is available for implementing the proposed methodology. he proposed
methodology is implemented via an open-source software R package
CRTConjoint, available through the Comprehensive R Archive Network.