August 2003
Volume 41 Number 4

Web-Based Surveys

Thomas M. Archer
Leader, Program Development and Evaluation
Ohio State University Extension
Columbus, Ohio
Internet Address: archer.3@osu.edu

Introduction

The telephone. Random sampling. Electronic surveys. Dillman states that these are the three most significant advances in survey technology in the twentieth century (Dillman, 2000, p. 352). If one thinks about the impact of the telephone and random sampling, one can soon realize the potential of Web-based surveys.

A Web-based survey is the collection of data through a self-administered electronic set of questions on the Web. With Web-based surveys, the manager has control over the physical appearance and can create attractive and inviting forms. Web-based surveys can include radio buttons and drop-down lists that permit only one choice for the response. Check boxes allow multiple answers. Text boxes can be one line with a limited number of characters, or they may permit unlimited text entry.

Using a commercial survey product, this author has developed, administered, and managed over 80 Web-based surveys since April 2002. This article discusses advantages, limitations, design guidelines, and tips for conducting Web-based surveys.

Advantages of Web-Based Surveys over Paper Surveys

1. Paper, postage, mail out, and data entry costs are almost completely eliminated (Dillman, 2000, p. 352).
2. Time required for implementation can be reduced (Dillman, 2000, p. 352).
3. Once electronic data collection system is developed, cost of surveying additional respondents is much lower (Dillman, 2000, p. 353).
4. Display of response data can be simultaneous with completion of surveys. Often, data from Web-based surveys are available in real time in graphic and numerical format.
5. Reminders and follow-up on non-respondents are relatively easy.
6. Data from Web-based surveys can be easily imported into data analysis programs.

Limitations of Web-Based Surveys

1. Not everyone is connected, so this survey method will not work with all populations (Dillman, 2000, p. 355).
2. Even if connected, not all potential respondents are equally computer literate (Dillman, 2000, p. 357).
3. Screen configurations may appear significantly different from one respondent to another, depending on settings of individual computers (Dillman, 2000, p. 357).
4. Sampling of e-mail addresses is difficult. There are no directories. Sometimes there is more than one e-mail address per respondent. Addresses are not standardized (Dillman, 2000, p. 356).
5. The decision not to respond is likely to be made more quickly.

Design Guidelines for Web-Based Surveys

1. Utilize a multiple contact strategy much like that used for regular mail surveys (Dillman, 2000, p. 367).
2. Personalize contacts through e-mail if possible (Dillman, 2000, p. 366).
3. Keep the invitation brief (Dillman, 2000, p. 366).
4. Begin with an interesting, but simple to answer, question (Dillman, 2000, p. 370).
5. Introduce a Web survey with a welcome screen that is motivational, emphasizes the ease of response, and instructs about how to proceed to the survey (Dillman, 2000, p. 377).
6. Present each question in a conventional format similar to that normally used on paper, self-administered surveys (Dillman, 2000, p. 379).
7. Do not require respondents to provide an answer to each question before being allowed to answer subsequent questions (Dillman, 2000, p. 394).
8. Make it possible for each question, and corresponding potential responses to that question to be visible on the screen at one time.

Tips for Conducting Web-Based Surveys

1. Shorten the timing between notice and reminders, and the total duration of the response period. Most of the time, 8-10 working days or less is sufficient.
2. Shorten the length of invitation and reminder messages.
3. Keep the questionnaires short.
4. Simplify the questions even more so than in paper surveys.
5. Think of the survey as an outline version of a conversation. There should be a natural flow, with transitions between one thought and the next.
6. Pilot test each survey with a variety of people using different browsers.
7. Avoid undeliverable e-mail invitations by developing accurate potential respondent e-mail lists.
8. Extract narrative text responses from data before importing into the numerical data analysis program SPSS. Extraneous commas and other characters will require "cleaning" the data.

Comparison of Cash Expenses: Traditional vs. Web-Based Survey

For the purposes of this discussion, assume a survey of a sample size of 300 with a postcard pre-notification, a first mailing with complete survey and return envelop and postage, a reminder with complete survey and return envelop with postage, and a total response of 200. Also assume clerical assistance and data entry at $10/hour, and professional time to analyze data and generate report at $40/ hour (Table 1).

As can be seen from Table 1, for the same survey, Web-based administration has no cash expenses compared to $681 for traditionally mailed. Total hours expended on similar surveys is reduced from 33 to 8 hours using the Web-based approach. This DOES NOT include the cost of the Web-based survey programming or subscription.

Web-Based Surveys: Some Examples

One can use any Internet search engine and locate a plethora of Web-based survey tools, their features, and costs by using the search words "Internet Surveys." This author began using Zoomerang (Zoomerang.com) in April 2002. He launched more than 80 Web-based surveys using Zoomerang with close to 3,000 total respondents through May 2003. The cost for a 1-year subscription to Zoomerang is $599 per year for up to 10,000 responses, regardless of the number of surveys, with $200 for each additional 5,000 response increments. There is a $350 rate per year for educational institutions.

A survey titled "Technology Access, Support and Skills of Extension Personnel" was launched via e-mail invitation through the Zoomerang survey program in August 2002 to 1,207 potential Extension employee respondents. There were 16 questions on this survey:

1. Five questions with multiple responses possible from checklists;
2. Two YES/ NO questions;
3. Three questions with multiple (7, 9, and 19) sub-question five point rating scales;
4. Four demographic drop down lists; and
5. Two open ended questions.

Six hundred thirty-one (52% response rate) Ohio Extension employees responded to this survey with two reminders in 11 working days (August 12-26, 2002). Based on the 1-year subscription rate, these 631 respondents cost $.55 per returned survey. If the entire 10,000 responses included in the subscription were expended through other surveys, the cost would be $.035 per response. Data in graphic and table form were available on the Web immediately after the survey was launched.

In another example, a survey titled "County Electronic Format Delivery " was launched through the same Zoomerang subscription on November 6, 2002 to 125 people who had been receiving electronic communication from a county Extension office for almost 2 years on agriculturally related topics. Because the county agent wanted the e-mail to come directly from him and because he wanted the responses to be completely anonymous, only a URL was generated by the Zoomerang survey program. The agent included this URL in his individual e-mail invitations to direct the potential respondents to this Web-based survey.

There were six questions on this survey:

1. Two questions with five sub-question rating scales, each with a five point agreement scale;
2. Two questions that were drop-down lists that generated impact data on money generated and ideas implemented; and
3. Two open-end questions, one requesting examples of how electronically shared information from Extension had been used, and the other inviting "Other Comments."

The survey was left open until December 6, and there were 61 respondents (49% response rate).

Another survey, titled "Cost Recovery 2003" was launched and the URL shared with 927 Extension program personnel on April 29 and closed on May 15, 2003. Two hundred sixty-two people responded (28% response rate) to 18 questions:

1. Four questions with multiple responses possible from checklists;
2. Three YES/ NO questions;
3. Two demographic drop-down lists; and
4. Nine open ended questions. One of the open-ended questions requested an e-mail address of the respondent, if that respondent wanted to be contacted for further information. Eighty-one of the respondents shared an e-mail address.

So What?

As budgets become tighter, personnel become fewer in number, connectivity becomes more widespread, and technology skills improve, Extension would be just plain stupid not to judiciously utilize Web-based surveys. The time is NOW for administering Web-based surveys that are "in-house" and SOON for the general public!

References

Dillman, D. A. (2000). Mail and Internet surveys--The tailored design method. New York : John Wiley & Sons, Inc.

Zoomerang.com. Web-based survey tool [On-line]. Available at: http://www.zoomerang.com/

This article is online at http://www.joe.org/joe/2003august/tt6.shtml.

Copyright © by Extension Journal, Inc. ISSN 1077-5315. Articles appearing in the Journal become the property of the Journal. Single copies of articles may be reproduced in electronic or print form for use in educational or training activities. Inclusion of articles in other publications, electronic sources, or systematic large-scale distribution may be done only with prior electronic or written permission of the Journal Editorial Office, joe-ed@joe.org.

If you have difficulties viewing or printing this page, please contact JOE Technical Support.