Evaluating bias of sequential mixed-mode designs against benchmark surveys

This study evaluated three types of bias – total, measurement, and selection bias – in three sequential mixed-mode designs of the Dutch Crime Victimization Survey: telephone, mail, and web, where nonrespondents were followed up face-to-face. In the absence of true scores, all biases were estimated as mode effects against two different types of benchmarks. In the single-mode benchmark (SMB), effects were evaluated against a face-to-face reference survey. In an alternative analysis, a ‘hybrid-mode benchmark’ (HMB) was used, where effects were evaluated against a mix of the measurements of a web survey and the selection bias of a face-to-face survey. A special re-interview design made available additional auxiliary data exploited in estimation for a range of survey variables. Depending on the SMB and HMB perspectives, a telephone, mail, or web design with a face-to-face follow-up (SMB) or a design involving only mail and/or web but not a face-to-face follow-up (HMB) is recommended based on the empirical findings.