Cognitive laboratory experiences: On pre-testing computerised questionnaires and data quality

Cognitive (pre-)testing research aims to improve the quality of research data by improving the questionnaire. The questionnaire is validated by testing it out on a small scale, that is, errors in the questionnaire—that cause systematic errors in the respondent's question-and-answer process in an interview—are detected, explained and improved (in an iterative process). In this way, the questionnaire adapts to the question-and-answer process, becomes easier to answer, takes less time, and becomes more respondent-friendly, resulting in reduction of measurement errors, i.e., increase in data quality (internal validity), and decrease in respondent burden. This is the CASM paradigm (Cognititve Aspects of Survey Methodology). The CASM paradigm defines the central hypothesis of this dissertation: pre-testing in a cognitive laboratory validates questionnaires.

In this dissertation, Snijkers describes his experiences with pre-testing research at Statistics Netherlands’ Questionnaire Lab, a cognitive laboratory that started in 1992. The purpose of this dissertation is documentation of these practical experiences. This thesis does not consist of a theoretical treatise on cognitive laboratory methods, but describes the application of these methods: designing and conducting pre-testing research, analyzing the data and presenting the results. This dissertation provides a schematic and systematic description of the practice of pre-testing research at CBS’s Questionnaire Lab in the period 1992-2000.

Snijkers, G. J. M. E. (2002). Cognitive laboratory experiences: On pre-testing computerised questionnaires and data quality. Dissertation, Utrecht University, handle:1874/13401.