Contact: Jon Meerdink ([email protected])

ANN ARBOR — A new paper from a team at the Institute for Social Research, led by Brady West and Bill Axinn, explores new methods for collecting data for demographic research from national samples.

In “Toward a New Approach to Creating Population-Representative Data for Demographic Research,” the team examines the use of web-based data collection, leveraging a study conducted from 2020 to 2022 to explore the viability of online data collection. Analyses presented in the paper demonstrate that web-based data collection appears to be a viable alternative to other more expensive methods when collecting survey data from a national sample.

A national probability sample using web and mail data collection methods could still allow researchers to generate the same estimates as with face to face methods when they analyze the survey data.

“We sought to show how all the methodological advancements in web and mail data collection have led us to a point right now where we can more or less replicate expensive face-to-face data collection using these more inexpensive modes,” said West.

The paper finds that web and mail modes can produce a reasonable sample, but post-survey weighting procedures are still necessary to offset unbalanced representation among selected sociodemographic subgroups. With the necessary weighting adjustments, the team found high consistency in the estimates generated by a face-to-face survey measuring the same content and those based on the web and mail modes, potentially saving a great deal of time and money for survey researchers. 

“We viewed this as very, very promising given the advancements in web survey data collection technology for people working on limited budgets who want to collect data from national samples,” said West.

Cost, however, is not a primary driver of the research. While early work on web-based survey modes tended to focus almost exclusively on cost, West and Axinn emphasized that quality data is much more important than the cost factor. 

“It’s not about how cheaply we can do this, it’s about how well we can do this,” said Axinn. “That was a huge driver of this entire paper.”

Having shown that their methodology works on national samples, West says the next step is harnessing this approach for a wider range of studies. In addition to national studies,  the methods could potentially be used in areas as small as a particular state or even a specific city, town, or zip-code. A web-based approach targeted at local levels could give researchers important insights on the difference between states or towns on things like demand for specific services. 

But above all, Axinn emphasizes that web-based methods are intended to be a part of the survey researcher’s toolbox, not their exclusive means of data collection.

“We’re continuously ratcheting up the quality with what we’re doing with these web surveys so that they’re meeting the standards of our highest quality studies,” Axinn said. “We’re trying to find the best possible approach.”
“Toward a New Approach to Creating Population-Representative Data for Demographic Research” is available now in Demography.