August 2003
Volume 41 Number 4

Tools of the Trade

Participatory Workshops: A Book Review

Larry Lev
Associate Professor/Extension Economist
Department of Agricultural and Resource Economics
Oregon State University
Corvallis, Oregon
Internet Address: larry.lev@oregonstate.edu

Stuck in a Rut

Robert Chambers has provided us with a truly useful book in Participatory Workshops: A Sourcebook of 21 Sets of Ideas & Activities. To understand why, consider this scenario:

It is 5:00 p.m., and you thoughts turn to the workshop you will be offering the next morning. You did some planning for the event a few weeks back but really not very much because you have put on a hundred similar events over the years. Suddenly, in a lightening bolt of inspiration, you think "Wouldn't it be great to offer a hundred DIFFERENT events instead of largely the same event one hundred times?"

In a flush of enthusiasm, you turn to your bookshelf knowing that the good ideas you need must be lurking there. But after a few minutes of leafing through books, you give up. Something may be there, but you just don't have the time to find it.

Another idea pops into your head--"Google". You launch a search for "meeting design getting started," confident of finding a clever opening for tomorrow's workshop. The response is immediate, it's overwhelming, and it's TOTALLY USELESS. You never imagined that so many documents and Web sites would contain these key words. A few might have something useful to say, but good luck finding them. So the reality sinks in. Your 101st workshop will be much like those that preceded it.

A Source of Ideas

An investment of less than $20 provides the possibility of achieving a different, better outcome. This investment will allow you to practice "just-in-time" meeting preparation and still incorporate something new, exciting, and maybe even dangerous.

Robert Chambers organized Participatory Workshops (PW) similar to a cookbook so that it can be used in a comparable fashion. Both the table of contents and the index are designed so that the reader can quickly find specific topic areas or techniques. The PW's subtitle is, "A Sourcebook of 21 Sets of Ideas & Activities," and each of the 21 lists has 21 items. As with cookbooks (which many people read cover to cover), PW provides rich rewards to those who read it from start to finish. But you don't have to do that to benefit from the book.

Chambers is well known for his work and publications focused on developing countries, and this book does address the unique constraints of those societies. Some of the suggestions may seem exotic, strange, or silly to American readers. But just like discovering a new cuisine, even the odder suggestions may provide a new way of looking at your work.

Chambers establishes his credibility by demonstrating that he has faced the same problems as you and survived to tell the tale:

• In "21 Horrors in Participatory Workshops," Chambers discusses how dreadful it can be when a "PAIN (Pompous and Insensitive Notable) opens the workshop."

• In "21 Mistakes I Made," he discusses the danger of "Spinning out the Start: Taking too long with initial hiccups-preamble, introductions, jokes . . . so that the start on substance is late."

• In "21 Ways Not to Answer a Question," he suggests a two-stage approach: "Stage 1: Flatter," followed by "Stage 2: Evade."

As should be expected, many of the 441 items didn't resonate with me. No one, however, buys a cookbook with the intention of preparing all of the recipes. I had no problem leafing through PW and identifying interesting suggestions and ideas. Here are four of my favorites.

• Johari's window: The group constructs a two-by-two matrix that examines what outside professionals and local citizens know and don't know about a situation (from both perspectives).

• Participatory PowerPoint: Just as it sounds--a group jointly works on a PowerPoint document.

• Knotty Problem (only for the adventurous): Participants hold hands and works themselves into a knotty mess. One or more outsiders try to instruct them on how to untangle by following verbal instructions only. Next, the group members are allowed to untangle on their own. The second approach is much quicker and less complicated. This provides a real-life example of the advantage of allowing groups to solve their own problems.

• The Candy Game: The discussion leader provides very large hard candies to each person who makes a comment. Participants are only allowed to suck the candies and are thus forced to free up the "air space" for others to contribute to the discussion.

A Change for the Better

Do not expect PW to transform your workshops and conferences. The task just isn't that simple. The changes you make as a result of adding this reference to your library are likely to be small and subtle. A step in the right direction should still be viewed as progress. If each of us only stopped one more participant per workshop from nodding off, imagine how much more productive we would be.

Reference

Chambers, R. (2002) Participatory workshops: A sourcebook of 21 sets of ideas & activities. Sterling, Va.: Earthscan Publications Ltd.

Celebrating Diversity 365 Days a Year: An Educational Calendar for Planning Programs

Lisa Guion
Assistant Professor
Internet Address: laguion@ifas.ufl.edu

Samantha Chattaraj
Project Coordinator, Diversity Project
Internet Address: sraj@ufl.edu

Stephanie Sullivan Lytle
Project Coordinator, Diversity Project
Internet Address: lytle@ufl.edu

Department of Family, Youth and Community Sciences
IFAS, Florida Cooperative Extension
University of Florida
Gainesville, Florida

Introduction

The Celebrating Diversity calendar is an educational resource developed to support the efforts of the Designing and Implementing Programs to Effectively Reach Diverse Audiences Project (Diversity Project) in the Department of Family, Youth and Community Sciences at the University of Florida. The Diversity Project is funded by the CYFERNet Program of the Children, Youth and Families at Risk Initiative (CYFAR).

The primary objective of the Diversity Project is to develop a curriculum that focuses on knowledge and skills needed to work more effectively with ethnically diverse audiences. The diversity calendar complements the newly developed Strengthening Programs to Reach Diverse Audiences curriculum that will be posted on the CYFERNet Web site <http://www.cyfernet.org/> beginning September 2003.

The Celebrating Diversity calendar covers an 18-month period from July 2003 through December 2004. It serves as a source of information on ethnic celebrations, holidays, and other significant events. Knowledge of those ethnic observances aid Extension professionals in planning their educational programs in several key ways. This article provides a description of the utility of the calendar as an Extension program-planning tool.

Using the Calendar as Program Planning Tool

As an educational tool, the calendar provides a plethora of information that can assist Extension educators in celebrating diversity within their programs, organization, and communities.

First, significant ethnic and cultural dates are noted in each month of the calendar. Educators can refer to these dates when scheduling programs and events to avoid conflicting with cultural, religious, and social holidays and observances that are important to a given ethnic or cultural group. Educators can show their appreciation and respect for diversity by including information on the observances in their newsletters, on bulletin boards in the office, in local newspaper columns, and in other forms of communication. The dates can also serve as a reference point for educators whose staff may be requesting time off to observe an occasion that is significant to their culture.

Second, there is information on the federally recognized history and heritage months, as well. Educators can celebrate these observances with special programs in the community. Educators could arrange presentations by members of the ethnic group to afford them opportunities to share important information about their culture. Exhibits and/or cultural fairs could be organized to celebrate the observance. Also, the educators could seek out and participate in events that are already scheduled in the communities.

Third, there are descriptions of some of the key holidays and other observances. This can serve to increase educators' knowledge and understanding of culturally relevant traditions, holidays, and events.

Fourth, there are ethnic quotes and proverbs from different cultural groups included for each month. These quotes and proverbs can increase educators' awareness of the values and beliefs that are held by those ethnic groups.

Equally important, the calendar can be used to stimulate conversations between educators and the ethnically diverse audiences they serve. This might occur when the educator learns how or if the specific individual client and his/her family celebrate an observance. Ethnic holidays and special occasions are recognized and celebrated in many different ways, and not everyone in an ethnic group will celebrate the occasion in the same way. Families, and in some cases communities, may have added unique rituals and traditions to the celebrations. Additionally, not everyone in an ethnic group will celebrate a given observance.

Celebrating Diversity is based on the lunar calendar, so some of the dates are subject to change in subsequent years (2005 and beyond). However, some information will not change such as history/heritage months, ethnic quotes/proverbs, birthdates of significant figures in a particular ethnic group, etc.

Availability of the Calendar

The calendar can be ordered from the CYFERNet Web site (http://www.cyfernet.org/).

Summary

While there is a lot of information included in this calendar, there are many other relevant ethnic observances that occur in diverse communities throughout the nation. Therefore, this calendar should be used as a starting point to initiate dialogue about the important events that help shape family and community life with the specific diverse audience that an educator is targeting for his or her programs.

Building Community: A Tool Kit for Youth and Adults in Charting Assets and Creating Change

Lucinda J. Garthwaite
Innovation Center for Community and Youth Development
Chevy Chase, Maryland
Internet Address: ljgarth@gwi.net

Beth K. Tucker
Director/FCS Educator
Coconino County Cooperative Extension
Flagstaff, Arizona
University of Arizona Cooperative Extension
Internet Address: tucker@cals.arizona.edu

People need each other. They need each other for comfort, love, safety, and companionship. They need each other to link together ideas and skills to create positive change. They need each other to grow.

Extension workers know this; they are always looking for ways to link communities with Extension resources and discovering new ways to help communities become more of what they want to be.

 Developed Collaboratively, Demonstrated Effectiveness

The Innovation Center for Community and Youth Development and National 4-H Council recognized these needs and produced Building Community: A Tool Kit for Youth and Adults in Charting Assets and Creating Change.

The Tool Kit was developed in partnership with 14 communities throughout the United States, with the help of Extension workers in many of those communities.

The toolkit's effectiveness has been demonstrated by consistent outcomes wherever it is implemented. Communities create shared visions, develop new awareness of community resources, and translate their new collective knowledge into strategic action.

When communities implement the tool kit, youth participation and leadership increase, as does collaboration among community-based organizations and groups.

A Focus on Community Gifts, Especially Youth Adult Partnerships

The Building Community Tool Kit emphasizes the principles of youth-adult partnerships, mapping community assets, and engaging in intentional reflection and learning. It takes users through a series of activities designed and successfully tested to guide and instruct community change agents as they work to create their visions.

Its focus is not on problems, but on the gifts that a community brings to their desire for change--gifts of the past, place, people, and relationships.

Sustainable Change

The tool kit has proven itself to be a powerful tool for sustainable community change because of its focus on gifts, reflection, and the engagement of participants from diverse sectors of a community.

Too often in communities people work hard only to meet the needs of their isolated "pocket," often defined by age, race, ethnicity, class, neighborhood, or religious affiliation.

This sometimes leaves broader community change efforts unrecognized and ultimately unsustainable. The Innovation Center's research and experience suggests that this often happens because the planning and implementation processes are missing the input and participation of the various sectors of the community.

Creating an inclusive community process requires skills and, in many cases, a paradigm shift. The Tool Kit creates concrete opportunities and approaches to develop and apply these skills.

Communities Using the Tool Kit

Having applied the strategies in the Building Community Tool Kit, communities are implementing priority projects, finding new funding, and creating sustainable, effective change.

Broadus, Montana

In the town of Broadus in eastern Montana, community members used the Tool Kit to create a vision for after-school opportunities for youth and children. They identified their resources, developed a plan, and successfully created a collaborative after-school program in which high school students staff programs for younger children.

Oxford County, Maine

In Oxford County, Maine, young people and adults had begun talking to each other about respect. Working with the Tool Kit helped them expand the scope of their work, making it more inclusive of the entire community.

Now, the Respect Team hosts round table discussions, promotes inclusion and diversity in local media, and successfully promotes networks of community groups working for change.

These and other communities are changing because younger and older people are working together, as partners, to discover their communities' strengths and reflect on the lessons learned.

"An Excellent Resource"

The Tool Kit has caught the attention of Extension workers and other community and youth development organizations throughout the world. Over 500 copies of the Tool Kit have been distributed in less than a year, mainly to Extension audiences.

Currently, it is being used in almost every U.S. state and five other countries. Everywhere the Innovation Center presents the Building Community approach, it has garnered a high level of interest (including at conferences such as the Search Institute, National Service-Learning Conference, National Conference on Community Service, and Volunteerism and CYFAR).

A team of reviewers from Iowa State University reviewed the Tool Kit along with dozens of community leadership training models. They gave it an extremely favorable review and deemed it an excellent resource for teaching ways to build community capacity, empower excluded people, incorporate class and cultural differences, and develop vision.

The Building Community Tool Kit is unique, basing its guidance on principles of youth-adult partnership, a focus on gifts, and on learning and reflection. It offers a shift from a traditional leader/organizer approach to one that creates the conditions needed for many community members to engage in the work. The result is an effective tool for sustainable change, both in the communities and the changers themselves.

For more information, contact the Innovation Center for Community and Youth Development at 301-961-2837 or info@innovationcenter.org.

Using a Retrospective Pre-Post Questionnaire to Determine Program Impact

Gregory A. Davis
Extension District Specialist, Community Development
Ohio State University Extension
Findlay, Ohio
Internet Address: davis.1081@osu.edu

Introduction

Evaluating program impact is important for all Extension educators in today's political economy. If passage of the Government Performance and Results Act of 1993 has not yet placed a renewed emphasis upon Extension's program effectiveness component (Richardson, Gamble, & Mustian, 1998; O'Neill, 1998), most certainly shrinking budgets for Extension will. Diem (2003) indicated that documenting such impact is not only a requirement of the agencies and political bodies that provide Extension funding, it also serves as a way to build and maintain credibility as well as justify use of limited resources. For these reasons and others, evaluation and documentation of Extension programming impact are beginning to receive increased emphasis in Extension work (Arnold, 2002).

A brief analysis of recent JOE articles on the topic revealed that a variety of methods and techniques can be used for program evaluation, including: Logic Modeling, children's drawings, formal qualitative and quantitative methods, and the retrospective pretest program evaluation. This article describes how the retrospective pretest methodology was used to determine change in knowledge, skills, and attitudes toward organizational strategic planning of 35 economic development professionals involved in a traditional Extension educational program.

The Retrospective Pretest

One Administration

Documenting changes in knowledge and behavior can be done simply and efficiently using the retrospective pretest evaluation (Rockwell & Kohn, 1989; Stevens & Lodl, 1999). According to Rockwell and Kohn, this tool "is specifically useful for evaluating the impact of Extension programs by asking participants to report actual changes in behavior" (in Stevens & Lodl). The retrospective pretest design, unlike the typical pretest-posttest, is administered only once. Because of time limitations, this characteristic made using the method more appealing to my audience and to me as the administrator of the instrument. Only a few minutes were required to complete the 13-item questionnaire.

Improved Accuracy

With the retrospective pretest, participants are asked to share the knowledge or attitude they had toward a particular subject before some experience, program, or treatment and after. When participants are asked to respond to a question about how much they know about a particular subject after they have some basic knowledge of the subject itself, they are more able to accurately reflect on the degree of change in knowledge or attitude (Rockwell & Kohn, 1989). Furthermore, respondents oftentimes overestimate their level of knowledge on a particular subject when using the traditional pretest-posttest (Pratt, McGuigan, & Katzev, 2000). With the retrospective pretest methodology, respondents are given an opportunity to learn how much they know about a subject prior to responding to a questionnaire.

My audience indicated that they had some experience with the topic prior to the program. Enabling them to more accurately assess their baseline level of understanding after the program provided them an opportunity to better illustrate the degree of change as a result of the program and provided me (and ultimately my stakeholders) with more meaningful data.

Using the Retrospective Pretest to Measure Change

For this evaluation effort, a one-page questionnaire was used that contained four background questions designed to collect basic data such as: role played in economic development; number of years of experience in these roles; population of the community on which these efforts are focused; and frequency of formal organizational strategic planning processes undertaken. The reverse side contained the retrospective pretest.

The retrospective pretest was designed with instructions at the top, an example, and nine statements. The statements were developed using the learning objectives for the strategic planning workshop. Participants were asked to indicate their level of agreement with each statement before and after the workshop using a six-point, Likert-type scale; (1-strongly disagree and 6-strongly agree).

Administration

Participants were asked to complete the one-page questionnaire at the conclusion of the workshop. A conscious attempt was made by the instructor to downplay the instrument, and there was no verbal instruction provided for completing the two-part questionnaire. Participants were simply asked to place their completed questionnaire on a table at the back of the room as they exited. Of 35 workshop participants, 32 completed questionnaires.

Data input/Analysis

Questionnaire data were analyzed using SPSS 10.1 to determine if participation in the workshop affected participant knowledge, awareness, confidence, and attitude. While the SPSS software is quite capable of examining the degree of change (among numerous other data analysis procedures), the degree of change was not examined. Group means (before and after) were also examined.

Results and Discussion

The retrospective pretest's nine workshop indicators revealed that participants experienced a positive change in knowledge, awareness, confidence, and attitudes. Eight of the nine indicators registered positive change for at least one third of the respondents. The overall mean for the nine items increased from 3.9 (before) to 4.9 (after) (Table 1).

Conclusions and Recommendations

I found this program evaluation tool to provide rich data with a modest investment of time, relative to more traditional pretest-post test evaluative measures. Program participants had little difficulty understanding and completing the questionnaire. Furthermore, participants were able to complete the instrument in a timely fashion, yielding very useful data compared to other evaluation tools requiring a similar investment of time. The data gathered were relatively easy to analyze and communicate a change in knowledge, awareness, confidence, and attitudes as ably as other more complex and involved evaluative measures.

In short, I found the retrospective pretest a useful tool for evaluating this traditional Extension program. While my use of the instrument focused primarily on immediate impact, the tool could also be used for demonstrating intermediate and long-term outcomes of Extension programs.

References

Arnold, M. E. (2002). Be "Logical" about program evaluation: Begin with learning assessment. Journal of Extension [On-line], 40(3). Available at: http://www.joe.org/joe/2002june/a4.html

Diem, K. G. (2003). Program development in a political world--It's all about impact! Journal of Extension [On-line], 41(1). Available at: http://www.joe.org/joe/2003february/a6.shtml

Diem, K. G. (2002). Using research methods to evaluate your Extension program. Journal of Extension [On-line], 41(1). Available at: http://www.joe.org/joe/2002december/a1.shtml

O'Neill, B. (1998). Money Talks: Documenting the economic impact of Extension personal finance programs. Journal of Extension [On-line], 36(5). Available at: http://www.joe.org/joe/1998october/a2.html

Pratt, C. C., McGuigan, W. M., Katzev, A. R. (2000). Measuring program outcomes: Using retrospective pretest methodology. American Journal of Evaluation. 21(3).

Richardson, J. G., Gamble, K. J., Mustian R. (1998). Creation of a web based accomplishment reporting system. Journal of Extension  [On-line]. 36(2). Available at: http://www.joe.org/joe/1998april/a1.html

Rockwell, S. K., & Kohn, H. (1989). Post-then-pre evaluation. Journal of Extension [On-line]. 27(2). Available at: http://www.joe.org/joe/1989summer/a5.html

Stevens, G. L., & Lodl, K. A. (1999). Community coalitions: Identifying changes in coalition members as a result of training. Journal of Extension [On-line]. 37(2). Available at: http://www.joe.org/joe/1999april/rb2.html

Single Commodity Stakeholder Groups as Valuable Advisors to Comprehensive Extension Programs for Crop Production in New Jersey

Michelle L. Infante-Casella
Assistant Professor and Agricultural Agent
Gloucester County, New Jersey
Internet Address: minfante@aesop.rutgers.edu
Internet: http://gloucester.rce.rutgers.edu/

Wesley L. Kline
Associate Professor and Agricultural Agent
Cumberland County, New Jersey
Internet Address: wkline@aesop.rutgers.edu

Rutgers Cooperative Extension

Introduction

Extension programs and research related to crop production are often directed to solving a specific problem in the field or looking at one aspect of a crop. Often the cause of the problem is a complex mix of factors that needs to be investigated comprehensively and systematically and not for just one specific reason. In some circles this concept is labeled "holistic management." In plain terms for farmers it means, "what can I do to continue farming and have a profitable business". The most successful Extension programs are identified and planned using stakeholder input. In the case of crop production, this often means input from farmers.

Forming stakeholder groups to identify clientele needs and programmatic direction is key to a successful Extension program. In the field, farmers encounter many challenges and successes. Sharing this information with each other and with Extension personnel is one of the most valuable tools for developing methods to improve pest management, variety selection, cultural practices, and overall productivity of a crop. Because production techniques differ for each commodity, it is important to look at all factors when developing a comprehensive management plan for an individual crop. Additionally, investigating alternative production methods and adaptability of these methods to a region is key in solving questions that are raised by stakeholders.

Methods

The Rutgers Cooperative Extension Vegetable Working Group (RCE VWG) team began a pepper advisory committee, lead by Dr. Wesley Kline, in 1998. Growers invited to participate in this single commodity advisory committee were chosen annually by agricultural agents from counties represented in the RCE VWG. The committee was comprised of 16 commercial pepper producers representing all regions of the state and members of the RCE VWG.

The meetings start with introduction of attendees and then move into a comment period for grower representatives. During the comment period, each issue presented stimulates a round-table discussion that addresses causes and effects of the problem and ideas to find a solution. Members discuss pepper production methods related to pest control, cultural practices, environmental conditions, and other issues that were either beneficial or harmful the outcome of the crop. After the grower comment period, Extension specialists and agricultural agents give condensed presentations on recent and future pepper related research projects. Growers make suggestions for research and comment on project proposals that would be submitted for grant funding. Grower involvement leads to modifications in the direction of pepper research and educational outreach programs.

Through this advisory committee, growers have a venue to express their needs and are willing to share ideas and offer resources to assist with projects. The committee meets each winter for a formal advisory meeting to discuss issues from the season and informally throughout the year at related events. Growers participate directly during the season by sponsoring research trials in their fields and assisting with data collection and evaluations.

In 2001, the RCE VWG formed a tomato advisory committee, again led by Dr. Wesley Kline, as a result of securing funds for a comprehensive tomato research and Extension program. The committee was modeled on the pepper advisory committee, except that industry representatives were included, such as seed companies, supermarkets, and buyers. Discussions from this group have lead to multiple research and Extension projects that include variety trials, disease control studies, insect control studies, season-extending research, taste preference studies, and a new Web site http://www.rce.rutgers.edu/tomato/. The Web site lists projects related to the RCE VWG comprehensive tomato program and targets the grower, food industry, and consumer audiences.

Results and Discussion

The Rutgers Cooperative Extension Pepper Advisory and Tomato Advisory Committees are on-going groups and are viewed as important tools by growers and Extension personnel. A grower member stated after the initial pepper advisory meeting, "This is the first time I have been able to express my opinion in this format". Another member replied after 4 years of participating, "I look forward to these meetings and learn a great deal from the in-depth discussions."

The increased communication between stakeholder groups and Extension personnel has fostered more trust and teamwork among these groups. Additional resources have been found in the form of grants, research sites, Extension education sites, field equipment, and labor through cooperative efforts of these groups.

Conditions on commercial farms can be quite different from those found at the research farms. Therefore, conducting research where the problem actually lies is extremely important to the success of finding the solution. Other single commodity stakeholder advisory committees for growers and Extension personnel will continue to be developed in New Jersey as interested participants are identified and as research needs are assessed for individual crops.

Conclusions

Through stakeholder participation in all aspects of the research and Extension education program planning and implementation, adoption of improved practices by farmers has increased and has occurred at a more rapid rate. Additionally, ideas for research and Extension education have been tailored to meet key grower needs. Utilizing the single commodity stakeholder group format to direct research and Extension programming has been and will continue to be an important tool in the advancement of crop production in New Jersey.

Web-Based Surveys

Thomas M. Archer
Leader, Program Development and Evaluation
Ohio State University Extension
Columbus, Ohio
Internet Address: archer.3@osu.edu

Introduction

The telephone. Random sampling. Electronic surveys. Dillman states that these are the three most significant advances in survey technology in the twentieth century (Dillman, 2000, p. 352). If one thinks about the impact of the telephone and random sampling, one can soon realize the potential of Web-based surveys.

A Web-based survey is the collection of data through a self-administered electronic set of questions on the Web. With Web-based surveys, the manager has control over the physical appearance and can create attractive and inviting forms. Web-based surveys can include radio buttons and drop-down lists that permit only one choice for the response. Check boxes allow multiple answers. Text boxes can be one line with a limited number of characters, or they may permit unlimited text entry.

Using a commercial survey product, this author has developed, administered, and managed over 80 Web-based surveys since April 2002. This article discusses advantages, limitations, design guidelines, and tips for conducting Web-based surveys.

Advantages of Web-Based Surveys over Paper Surveys

1. Paper, postage, mail out, and data entry costs are almost completely eliminated (Dillman, 2000, p. 352).
2. Time required for implementation can be reduced (Dillman, 2000, p. 352).
3. Once electronic data collection system is developed, cost of surveying additional respondents is much lower (Dillman, 2000, p. 353).
4. Display of response data can be simultaneous with completion of surveys. Often, data from Web-based surveys are available in real time in graphic and numerical format.
5. Reminders and follow-up on non-respondents are relatively easy.
6. Data from Web-based surveys can be easily imported into data analysis programs.

Limitations of Web-Based Surveys

1. Not everyone is connected, so this survey method will not work with all populations (Dillman, 2000, p. 355).
2. Even if connected, not all potential respondents are equally computer literate (Dillman, 2000, p. 357).
3. Screen configurations may appear significantly different from one respondent to another, depending on settings of individual computers (Dillman, 2000, p. 357).
4. Sampling of e-mail addresses is difficult. There are no directories. Sometimes there is more than one e-mail address per respondent. Addresses are not standardized (Dillman, 2000, p. 356).
5. The decision not to respond is likely to be made more quickly.

Design Guidelines for Web-Based Surveys

1. Utilize a multiple contact strategy much like that used for regular mail surveys (Dillman, 2000, p. 367).
2. Personalize contacts through e-mail if possible (Dillman, 2000, p. 366).
3. Keep the invitation brief (Dillman, 2000, p. 366).
4. Begin with an interesting, but simple to answer, question (Dillman, 2000, p. 370).
5. Introduce a Web survey with a welcome screen that is motivational, emphasizes the ease of response, and instructs about how to proceed to the survey (Dillman, 2000, p. 377).
6. Present each question in a conventional format similar to that normally used on paper, self-administered surveys (Dillman, 2000, p. 379).
7. Do not require respondents to provide an answer to each question before being allowed to answer subsequent questions (Dillman, 2000, p. 394).
8. Make it possible for each question, and corresponding potential responses to that question to be visible on the screen at one time.

Tips for Conducting Web-Based Surveys

1. Shorten the timing between notice and reminders, and the total duration of the response period. Most of the time, 8-10 working days or less is sufficient.
2. Shorten the length of invitation and reminder messages.
3. Keep the questionnaires short.
4. Simplify the questions even more so than in paper surveys.
5. Think of the survey as an outline version of a conversation. There should be a natural flow, with transitions between one thought and the next.
6. Pilot test each survey with a variety of people using different browsers.
7. Avoid undeliverable e-mail invitations by developing accurate potential respondent e-mail lists.
8. Extract narrative text responses from data before importing into the numerical data analysis program SPSS. Extraneous commas and other characters will require "cleaning" the data.

Comparison of Cash Expenses: Traditional vs. Web-Based Survey

For the purposes of this discussion, assume a survey of a sample size of 300 with a postcard pre-notification, a first mailing with complete survey and return envelop and postage, a reminder with complete survey and return envelop with postage, and a total response of 200. Also assume clerical assistance and data entry at $10/hour, and professional time to analyze data and generate report at $40/ hour (Table 1).

As can be seen from Table 1, for the same survey, Web-based administration has no cash expenses compared to $681 for traditionally mailed. Total hours expended on similar surveys is reduced from 33 to 8 hours using the Web-based approach. This DOES NOT include the cost of the Web-based survey programming or subscription.

Web-Based Surveys: Some Examples

One can use any Internet search engine and locate a plethora of Web-based survey tools, their features, and costs by using the search words "Internet Surveys." This author began using Zoomerang (Zoomerang.com) in April 2002. He launched more than 80 Web-based surveys using Zoomerang with close to 3,000 total respondents through May 2003. The cost for a 1-year subscription to Zoomerang is $599 per year for up to 10,000 responses, regardless of the number of surveys, with $200 for each additional 5,000 response increments. There is a $350 rate per year for educational institutions.

A survey titled "Technology Access, Support and Skills of Extension Personnel" was launched via e-mail invitation through the Zoomerang survey program in August 2002 to 1,207 potential Extension employee respondents. There were 16 questions on this survey:

1. Five questions with multiple responses possible from checklists;
2. Two YES/ NO questions;
3. Three questions with multiple (7, 9, and 19) sub-question five point rating scales;
4. Four demographic drop down lists; and
5. Two open ended questions.

Six hundred thirty-one (52% response rate) Ohio Extension employees responded to this survey with two reminders in 11 working days (August 12-26, 2002). Based on the 1-year subscription rate, these 631 respondents cost $.55 per returned survey. If the entire 10,000 responses included in the subscription were expended through other surveys, the cost would be $.035 per response. Data in graphic and table form were available on the Web immediately after the survey was launched.

In another example, a survey titled "County Electronic Format Delivery " was launched through the same Zoomerang subscription on November 6, 2002 to 125 people who had been receiving electronic communication from a county Extension office for almost 2 years on agriculturally related topics. Because the county agent wanted the e-mail to come directly from him and because he wanted the responses to be completely anonymous, only a URL was generated by the Zoomerang survey program. The agent included this URL in his individual e-mail invitations to direct the potential respondents to this Web-based survey.

There were six questions on this survey:

1. Two questions with five sub-question rating scales, each with a five point agreement scale;
2. Two questions that were drop-down lists that generated impact data on money generated and ideas implemented; and
3. Two open-end questions, one requesting examples of how electronically shared information from Extension had been used, and the other inviting "Other Comments."

The survey was left open until December 6, and there were 61 respondents (49% response rate).

Another survey, titled "Cost Recovery 2003" was launched and the URL shared with 927 Extension program personnel on April 29 and closed on May 15, 2003. Two hundred sixty-two people responded (28% response rate) to 18 questions:

1. Four questions with multiple responses possible from checklists;
2. Three YES/ NO questions;
3. Two demographic drop-down lists; and
4. Nine open ended questions. One of the open-ended questions requested an e-mail address of the respondent, if that respondent wanted to be contacted for further information. Eighty-one of the respondents shared an e-mail address.

So What?

As budgets become tighter, personnel become fewer in number, connectivity becomes more widespread, and technology skills improve, Extension would be just plain stupid not to judiciously utilize Web-based surveys. The time is NOW for administering Web-based surveys that are "in-house" and SOON for the general public!

References

Dillman, D. A. (2000). Mail and Internet surveys--The tailored design method. New York : John Wiley & Sons, Inc.

Zoomerang.com. Web-based survey tool [On-line]. Available at: http://www.zoomerang.com/

Copyright © by Extension Journal, Inc. ISSN 1077-5315. Articles appearing in the Journal become the property of the Journal. Single copies of articles may be reproduced in electronic or print form for use in educational or training activities. Inclusion of articles in other publications, electronic sources, or systematic large-scale distribution may be done only with prior electronic or written permission of the Journal Editorial Office, joe-ed@joe.org.

If you have difficulties viewing or printing this page, please contact JOE Technical Support.