*New* CSDE Computational Demography Working Group (CDWG) Hosts İhsan Kahveci on Evaluating Online Recruitment in COVID-19 Prosocial Behavior Surveys: Comparison of Social Media Sampling with Probability Sampling (4/3/2024)
Posted: 3/28/2024 ()
On 4/3 from 9:00 AM – 10:00 AM, CDWG will host İhsan Kahveci to present his research. Ihsan Kahveci is a Ph.D. candidate in the Department of Sociology and an affiliated student at the International Max Planck Research School for Population, Health and Data Science. His research areas include 1) online and network sampling methods for survey data collection and 2) (mis)information diffusion through social networks. CDWG Will be Hybrid in the Spring Quarter of 2024. Register for Zoom here or attend in-person in Raitt 223.
Learn more about the talk in the full story!
Title: “Evaluating Online Recruitment in COVID-19 Prosocial Behavior Surveys: Comparison of Social Media Sampling with Probability Sampling”
Abstract: There has been a significant increase in the need for rapid, high-quality online surveys. The recent pandemic typified this. Because of this, there is a large need for methods to improve the generalizability of these online samples. While the literature shows that careful recruitment of respondents is critical for a quality survey, post-hoc adjustments can help achieve representativeness. One of the known limitations of online data collection is its ability to produce very large but highly biased samples. One solution to this problem is to stratify (e.g., gender) the recruitment process made available on the advertising platform (e.g., Facebook). Because the researcher does not have control of the randomization process on the advertising platform, it remains unclear if such an approach is, in general, beneficial or likely to cause even more bias. In this article, we test the viability of such an approach by analyzing a pro-social survey collected via Facebook’s advertising platform and reweighted using the US Census American Community survey data and a representative online panel-based survey known to be demographically representative of the United States. Our results show that the survey fielded on Facebook is biased towards college graduates and younger ages, even after post-adjustments. We implemented propensity score weighting based on a reference survey to minimize the observed bias and how it affected the public health measures across surveys. We found that propensity adjustment worked well for time-invariant measures such as the number of people with chronic health conditions. However, behavioral time-varied measures such as mask-wearing are still vastly underestimated. This work suggests that running small, high-quality probability sampling reference surveys to supplement the results of large-scale online survey projects can be a viable solution to selection bias in online surveys.