Online panel research : a data quality perspective
- Responsibility
- editors, Mario Callegaro (Survey Research Scientist, Quantitative Marketing Team, Google UK), Reg Baker (Senior Consultant, Market Strategies International, USA), Jelke Bethlehem (Senior Advisor, Statistics Netherlands, The Netherlands), Anja S. Göritz (Professor of Occupational and Consumer Psychology, University of Freiburg, Germany), Jon A. Krosnick (Professor of Political Science, Communication and Psychology, Stanford University, USA), Paul J. Lavrakas (Independent Research Psychologist/Research Methodologist, USA).
- Publication
- Chichester, West Sussex : Wiley, 2014.
- Copyright notice
- ©2014
- Physical description
- xxix, 477 pages : illustrations ; 25 cm.
- Series
- Wiley series in survey methodology.
Description
Creators/Contributors
- Contributor
- Callegaro, Mario, editor.
Contents/Summary
- Bibliography
- Includes bibliographical references and index.
- Contents
-
- Preface xv
- Acknowledgments xvii
- About the Editors xix
- About the Contributors xxiii
- 1 Online panel research: History, concepts, applications and a look at the future 1 Mario Callegaro, Reg Baker, Jelke Bethlehem, Anja S. Goeritz, Jon A. Krosnick, and Paul J. Lavrakas
- 1.1 Introduction 1
- 1.2 Internet penetration and online panels 2
- 1.3 Definitions and terminology 2
- 1.4 A brief history of online panels 4
- 1.5 Development and maintenance of online panels 6
- 1.6 Types of studies for which online panels are used 15
- 1.7 Industry standards, professional associations guidelines, and advisory groups 15
- 1.8 Data quality issues 17
- 1.9 Looking ahead to the future of online panels 17
- 2 A critical review of studies investigating the quality of data obtained with online panels based on probability and nonprobability samples 23 Mario Callegaro, Ana Villar, David Yeager, and Jon A. Krosnick
- 2.1 Introduction 23
- 2.2 Taxonomy of comparison studies 24
- 2.3 Accuracy metrics 27
- 2.4 Large-scale experiments on point estimates 28
- 2.5 Weighting adjustments 35
- 2.6 Predictive relationship studies 36
- 2.7 Experiment replicability studies 38
- 2.8 The special case of pre-election polls 42
- 2.9 Completion rates and accuracy 43
- 2.10 Multiple panel membership 43
- 2.11 Online panel studies when the offline population is less of a concern 46
- 2.12 Life of an online panel member 47
- 2.13 Summary and conclusion 48
- Part I COVERAGE 55
- Introduction to Part I 56 Mario Callegaro and Jon A. Krosnick
- 3 Assessing representativeness of a probability-based online panel in Germany 61 Bella Struminskaya, Lars Kaczmirek, Ines Schaurer, and Wolfgang Bandilla
- 3.1 Probability-based online panels 61
- 3.2 Description of the GESIS Online Panel Pilot 62
- 3.3 Assessing recruitment of the Online Panel Pilot 66
- 3.4 Assessing data quality: Comparison with external data 68
- 3.5 Results 74
- 3.6 Discussion and conclusion 80
- 4 Online panels and validity: Representativeness and attrition in the Finnish eOpinion panel 86 Kimmo Groenlund and Kim Strandberg
- 4.1 Introduction 86
- 4.2 Online panels: Overview of methodological considerations 87
- 4.3 Design and research questions 88
- 4.4 Data and methods 90
- 4.5 Findings 92
- 4.6 Conclusion 100
- 5 The untold story of multi-mode (online and mail) consumer panels: From optimal recruitment to retention and attrition 104 Allan L. McCutcheon, Kumar Rao, and Olena Kaminska
- 5.1 Introduction 104
- 5.2 Literature review 107
- 5.3 Methods 108
- 5.4 Results 115
- 5.5 Discussion and conclusion 124
- Part II NONRESPONSE 127
- Introduction to Part II 128 Jelke Bethlehem and Paul J. Lavrakas
- 6 Nonresponse and attrition in a probability-based online panel for the general population 135 Peter Lugtig, Marcel Das, and Annette Scherpenzeel
- 6.1 Introduction 135
- 6.2 Attrition in online panels versus offline panels 137
- 6.3 The LISS panel 139
- 6.4 Attrition modeling and results 142
- 6.5 Comparison of attrition and nonresponse bias 148
- 6.6 Discussion and conclusion 150
- 7 Determinants of the starting rate and the completion rate in online panel studies 154 Anja S. Goeritz
- 7.1 Introduction 154
- 7.2 Dependent variables 155
- 7.3 Independent variables 156
- 7.4 Hypotheses 156
- 7.5 Method 163
- 7.6 Results 164
- 7.7 Discussion and conclusion 166
- 8 Motives for joining nonprobability online panels and their association with survey participation behavior 171 Florian Keusch, Bernad Batinic, and Wolfgang Mayerhofer
- 8.1 Introduction 171
- 8.2 Motives for survey participation and panel enrollment 173
- 8.3 Present study 176
- 8.4 Results 179
- 8.5 Conclusion 185
- 9 Informing panel members about study results: Effects of traditional and innovative forms of feedback on participation 192 Annette Scherpenzeel and Vera Toepoel
- 9.1 Introduction 192
- 9.2 Background 193
- 9.3 Method 196
- 9.4 Results 199
- 9.5 Discussion and conclusion 207
- Part III MEASUREMENT ERROR 215
- Introduction to Part III 216 Reg Baker and Mario Callegaro
- 10 Professional respondents in nonprobability online panels 219 D. Sunshine Hillygus, Natalie Jackson, and McKenzie Young
- 10.1 Introduction 219
- 10.2 Background 220
- 10.3 Professional respondents and data quality 221
- 10.4 Approaches to handling professional respondents 223
- 10.5 Research hypotheses 224
- 10.6 Data and methods 225
- 10.7 Results 226
- 10.8 Satisficing behavior 229
- 10.9 Discussion 232
- 11 The impact of speeding on data quality in nonprobability and freshly recruited probability-based online panels 238 Robert Greszki, Marco Meyer, and Harald Schoen
- 11.1 Introduction 238
- 11.2 Theoretical framework 239
- 11.3 Data and methodology 242
- 11.4 Response time as indicator of data quality 243
- 11.5 How to measure "speeding"? 246
- 11.6 Does speeding matter? 251
- 11.7 Conclusion 257
- Part IV WEIGHTING ADJUSTMENTS 263
- Introduction to Part IV 264 Jelke Bethlehem and Mario Callegaro
- 12 Improving web survey quality: Potentials and constraints of propensity score adjustments 273 Stephanie Steinmetz, Annamaria Bianchi, Kea Tijdens, and Silvia Biffignandi
- 12.1 Introduction 273
- 12.2 Survey quality and sources of error in nonprobability web surveys 274
- 12.3 Data, bias description, and PSA 277
- 12.4 Results 284
- 12.5 Potentials and constraints of PSA to improve nonprobability web survey quality: Conclusion 286
- 13 Estimating the effects of nonresponses in online panels through imputation 299 Weiyu Zhang
- 13.1 Introduction 299
- 13.2 Method 302
- 13.3 Measurements 303
- 13.4 Findings 303
- 13.5 Discussion and conclusion 308
- Part V NONRESPONSE AND MEASUREMENT ERROR 311
- Introduction to Part V 312 Anja S. Goeritz and Jon A. Krosnick
- 14 The relationship between nonresponse strategies and measurement error: Comparing online panel surveys to traditional surveys 313 Neil Malhotra, Joanne M. Miller, and Justin Wedeking
- 14.1 Introduction 313
- 14.2 Previous research and theoretical overview 314
- 14.3 Does interview mode moderate the relationship between nonresponse strategies and data quality? 317
- 14.4 Data 318
- 14.5 Measures 320
- 14.6 Results 324
- 14.7 Discussion and conclusion 332
- 15 Nonresponse and measurement error in an online panel: Does additional effort to recruit reluctant respondents result in poorer quality data? 337 Caroline Roberts, Nick Allum, and Patrick Sturgis
- 15.1 Introduction 337
- 15.2 Understanding the relation between nonresponse and measurement error 338
- 15.3 Response propensity and measurement error in panel surveys 341
- 15.4 The present study 342
- 15.5 Data 343
- 15.6 Analytical strategy 344
- 15.7 Results 350
- 15.8 Discussion and conclusion 357
- Part VI SPECIAL DOMAINS 363
- Introduction to Part VI 364 Reg Baker and Anja S. Goeritz
- 16 An empirical test of the impact of smartphones on panel-based online data collection 367 Frank Drewes
- 16.1 Introduction 367
- 16.2 Method 369
- 16.3 Results 371
- 16.4 Discussion and conclusion 385
- 17 Internet and mobile ratings panels 387 Philip M. Napoli, Paul J. Lavrakas, and Mario Callegaro
- 17.1 Introduction 387
- 17.2 History and development of Internet ratings panels 388
- 17.3 Recruitment and panel cooperation 390
- 17.4 Compliance and panel attrition 394
- 17.5 Measurement issues 396
- 17.6 Long tail and panel size 398
- 17.7 Accuracy and validation studies 400
- 17.8 Statistical adjustment and modeling 401
- 17.9 Representative research 402
- 17.10 The future of Internet audience measurement 403
- Part VII OPERATIONAL ISSUES IN ONLINE PANELS 409
- Introduction to Part VII 410 Paul J. Lavrakas and Anja S. Goeritz
- 18 Online panel software 413 Tim Macer
- 18.1 Introduction 413
- 18.2 What does online panel software do? 414
- 18.3 Survey of software providers 415
- 18.4 A typology of panel research software 416
- 18.5 Support for the different panel software typologies 417
- 18.6 The panel database 418
- 18.7 Panel recruitment and profile data 421
- 18.8 Panel administration 423
- 18.9 Member portal 425
- 18.10 Sample administration 428
- 18.11 Data capture, data linkage and interoperability 430
- 18.12 Diagnostics and active panel management 433
- 18.13 Conclusion and further work 436
- 19 Validating respondents identity in online samples: The impact of efforts to eliminate fraudulent respondents 441 Reg Baker, Chuck Miller, Dinaz Kachhi, Keith Lange, Lisa Wilding-Brown, and Jacob Tucker
- 19.1 Introduction 441
- 19.2 The 2011 study 443
- 19.3 The 2012 study 444
- 19.4 Results 446
- 19.5 Discussion 449
- 19.6 Conclusion 450
- References 451
- Appendix 19.A 452
- Index 457.
- (source: Nielsen Book Data)
- Publisher's summary
-
Provides new insights into the accuracy and value of online panels for completing surveys Over the last decade, there has been a major global shift in survey and market research towards data collection, using samples selected from online panels. Yet despite their widespread use, remarkably little is known about the quality of the resulting data. This edited volume is one of the first attempts to carefully examine the quality of the survey data being generated by online samples. It describes some of the best empirically-based research on what has become a very important yet controversial method of collecting data. Online Panel Research presents 19 chapters of previously unpublished work addressing a wide range of topics, including coverage bias, nonresponse, measurement error, adjustment techniques, the relationship between nonresponse and measurement error, impact of smartphone adoption on data collection, Internet rating panels, and operational issues. The datasets used to prepare the analyses reported in the chapters are available on the accompanying website: www.wiley.com/go/online-panel Covers controversial topics such as professional respondents, speeders, and respondent validation. Addresses cutting-edge topics such as the challenge of smartphone survey completion, software to manage online panels, and Internet and mobile ratings panels. Discusses and provides examples of comparison studies between online panels and other surveys or benchmarks. Describes adjustment techniques to improve sample representativeness. Addresses coverage, nonresponse, attrition, and the relationship between nonresponse and measurement error with examples using data from the United States and Europe. Addresses practical questions such as motivations for joining an online panel and best practices for managing communications with panelists. Presents a meta-analysis of determinants of response quantity. Features contributions from 50 international authors with a wide variety of backgrounds and expertise. This book will be an invaluable resource for opinion and market researchers, academic researchers relying on web-based data collection, governmental researchers, statisticians, psychologists, sociologists, and other research practitioners.
(source: Nielsen Book Data)
Subjects
Bibliographic information
- Publication date
- 2014
- Copyright date
- 2014
- Series
- Wiley series in survey methodology
- ISBN
- 9781119941774 (pbk.)
- 1119941776 (pbk.)
Browse related items
Start at call number:
