December 2004
Volume 42 Number 6

Features

Smith Lever 3(d) Extension Evaluation and Outcome Reporting--A Scorecard to Assist Federal Program Leaders

Bill Hoffman
Doctoral Candidate
George Washington University
Washington, DC
hoffy@gwu.edu

Barbara Grabowski
Associate Professor of Education
Penn State University
University Park, Pennsylvania
bgrabowski@psu.edu

The Government Performance Results Act (GPRA) requires that all federal agencies and programs set goals and measure outcomes (USGAO, 1996). Goals that are the product of national leadership and stakeholder input help to clearly articulate program priorities and prevent mission creep. Measuring program outcomes can quantify productivity, determine efficiency and effectiveness of processes used, and highlight the usefulness of programs in terms of accomplishment of program goals.

For many program managers, the most difficult aspect of GPRA implementation is the transition from measuring program outputs to developing outcome-related program measures (USGAO, 1997). The United States Department of Agriculture's Cooperative State Research, Education and Extension Service (CSREES) is one of many agencies whose program managers have found this to be a challenging mandate.

CSREES administers funding for Extension programs that intend to help the citizenry put university research to practical use through various forms of educational programming (ECOP, 1997). Extension programming is one area where outcome measurement challenges have been documented (Nelson, 1999).

The Hoffman EEOR Scorecard of LOGIC model-based questions was developed to illuminate the utilization of Extension evaluation and outcome reporting (EEOR) ideal practices by Smith Lever 3(d) programs, one sub-set of CSREES Extension funded programming efforts. This scorecard was developed from an extensive review of the Extension program literature within the context of GPRA (Hoffman, 2003). This article provides a brief overview of this research, including an example of its findings for one Smith Lever 3(d) program: Extension IPM Implementation. The lead author of this publication is professionally responsible for the state reporting function of that program.

Review of Current Literature

Current literature from evaluation, GPRA implementation guidance, and Extension evaluation contributed to the development of the scorecard.

Evaluation Background

A central concept in Extension program evaluation and the GPRA is the differentiation between outcomes and outputs. Outcomes refer to results of program objectives that are defined by the underlying purpose of the federal investment (Nelson, 1999). They include variables such as improvement in agricultural profitability, increases in agricultural systems efficiency, enhanced environmental quality, and decreases in farm worker injuries. Outputs refer to the activities or efforts of a program used to produce outcomes (Nelson, 1999). They include variables such as number of training sessions held, the number of participants trained, the number of publications developed, or the number of farms visited.

Change agents such as Extension educators achieve outcomes directly through programming outputs and indirectly through secondary interpersonal educational networks that exist within social systems (Rogers, 1998). This includes program participants sharing information with peers and clients, which has the potential to multiply the effects of Extension educational activity. For this reason, Extension programming can be expected to achieve outcomes that exceed those that directly result from programming outputs.

Output information can help to contextualize outcome data by helping to explain the program's role in achieving these outcomes. However, output information in the absence of outcome data does not illuminate program effectiveness, efficiency, or productivity toward reaching an educational program's objectives (USGAO, 1996).

GPRA Implementation Guidance

The United States General Accounting Office distinguishes between different types of outcomes. "Ultimate outcomes" are those that represent the achievement of the underlying purpose of the federal investment (USGAO, 1998). An example of an ultimate outcome is decreased surface water pollution caused by dairy farming operations. Outcomes that contribute or lead to this ultimate purpose are known as "intermediate outcomes." An example of an intermediate outcome that could lead to the aforementioned ultimate outcome is the adoption of environmentally friendly manure management practices by dairy farmers.

If research supports a strong connection between intermediate and ultimate outcomes, the measurement of intermediate outcomes alone can be used to satisfy GPRA requirements (USGAO, 1998). These are commonly referred to a "proxy measures."

Currently used evaluation models in the instructional systems and Extension education evaluation fields make similar distinctions between outcomes and outputs as well as different types of outcomes. Examination of the LOGIC model can help to clarify these distinctions and provide guidance for federal Extension evaluation and outcome reporting.

Extension Evaluation

The University of Wisconsin's LOGIC model is pictured in Figure 1 (UWEC, 2002). The model has at its roots Kirkpatrick's four-level and Bennett's seven-level evaluation models (Kirkpatrick, 1959; Bennett, 1975).

Figure 1.
University of Wisconsin's LOGIC model. (Retrieved from http://www.uwex.edu/ces/pdande/copyright.html and reprinted according to guidelines from the publisher)

The model defines three outcome types: Learning, Action, and Conditions. Though measurements of learning through pre-tests and post-tests of participants can be considered an intermediate outcome, data that describes how this learning is transferred to action is much more valuable (Houlton, 1996). Action outcomes include changes in behavior and adoption of practices that have resulted, in part, from the aforementioned learning. Action outcomes generally represent intermediate outcomes that may reveal progress toward ultimate outcome progress. Condition outcomes are advancements in social, economic, civic, and environmental conditions that are generally analogous to the "ultimate outcomes" described earlier.

Non-outcome categories of the LOGIC model include Inputs, Activity Outputs, Participation Outputs, External Factors, and Assumptions. Inputs of resources are invested to support learning activities (Bennett, 1975). The LOGIC model overcomes Houlton's criticism (1996) of Kirkpatrick's earlier work by acknowledging the role of external factors, which include new technologies and social pressures that can slow or accelerate practice adoption.

Finally, the LOGIC model acknowledges the importance of assumptions made by educators regarding how educational programming may influence outcomes. These assumptions include the mix of educational tactics and the proper audiences to target, which the educator perceives will provide the greatest impact within resource constraints. Though these non-outcome categories do not address outcomes themselves, they describe the process and strategy used by educators to achieve outcomes through input investment.

Methods

Based on the reviewed literature, three Extension evaluation and outcome reporting ideal practices were designated. From these, a series of LOGIC model-based questions, that is, a scorecard, was developed to examine their utilization. This section discusses these activities and outlines limitations of the research.

Extension Evaluation and Outcome Reporting (EEOR) Ideal Practices

Guidance provided by the GAO regarding GPRA implementation and the nature of Extension work suggests three Extension evaluation and outcome reporting (EEOR) ideal practices to be followed by federal program managers:

EEOR Ideal Practice #1--National Outcome Definition and Measurement: Define and measure national ultimate program (condition) outcomes, using research-supported proxies (learning and action outcomes) where appropriate.

Ideal EEOR Practice #2--Sub-National (State) Outcome Reporting: Have a user-friendly system for individual awardees (henceforth referred to as "state programs") or groups of state programs to report on nationally defined outcomes or proxies directly. Locally defined outcomes could be used and reported if they are consistent with and complementary to nationally defined and measured goals.

EEOR Ideal Practice #3--Sub-National (State) Non-Outcome Reporting: Report non-outcome data (outputs, inputs, external factors, assumptions) to contextualize outcomes, not as program results.

Articulating desired national outcomes and measuring progress toward them helps to clarify programmatic purposes. Measurement of intermediate (action) outcomes can be substituted for ultimate (condition) outcomes if there is a strong, research-supported link between the two phenomena. An example is measuring the action phenomenon of the number of servings of fruits and vegetables consumed per day as a proxy for the health benefits associated with this activity.

National ultimate and intermediate outcomes can often be measured through third party data, such as surveys conducted by other agencies of the federal government. A user-friendly state outcome reporting system can provide evidence of a local program's role in attaining national outcomes. Finally, non-outcome data such as number of participants and external factors can be useful to contextualize reported outcomes. While non-outcome data from all of these categories are of some potential use, this data should be used to contextualize rather than replace outcome measurement.

The aforementioned three EEOR ideal practices would not necessarily ensure complete GPRA compliance themselves. However, their utilization would go a long way toward overcoming an impediment to GPRA implementation: Defining and measuring outcome goals instead of outputs.

Development of an Evaluation Scorecard

Simply asking "does the program utilize practice x?" would not yield the depth of answer desired. The LOGIC model was used to develop the Hoffman EEOR Scorecard to assess how and in what ways these programs utilize these three EEOR ideal practices. This scorecard is shown in Table 1. This table also references the components of the LOGIC model that the questions intend to illuminate.

Limitations of the Research

It is important to note that these questions were designed to illuminate the utilization of selected EEOR ideal practices that are consistent with GPRA compliance. Utilization of these practices alone will not guarantee complete GPRA compliance.

Answers were obtained primarily through publicly available extant data including requests for applications, plans of work, annual reports, and other components of CSREES reporting systems. To supplement this, some CSREES National Program Leaders were consulted to provide further clarification. This focus on extant data had the potential to produce less than exhaustive information regarding the program's evaluation and results reporting efforts, particularly if a majority of these efforts take place "behind the curtain" and are not publicly documented.

Abridged Example Report of Findings

The original research examined the following programs: Extended Food and Nutrition, Children, Youth and Families at Risk, Extension Integrated Pest Management, Farm Safety combined with Youth Farm Safety Certification, Extension Indian Reservation Program, Sustainable Agricultural Research and Extension, and Regional Rural Development. Due to the space limitations of this forum, this article provides an abridged example of findings for the Extension Integrated Pest Management (IPM) Program. This includes a brief explanation of the IPM program and examination of compliance with each of the three EEOR practices. To aid the reader, LOGIC model components are italicized when mentioned in the regular text and included in parentheses when referred to indirectly.

Explanation of IPM Program

The Integrated Pest Management Program teaches common pest management principles to a wide variety of audiences. CSREES provides formula funding to states and territories to further these efforts. One of the co-authors works directly with the state outcome-reporting element of this program.

Program Utilization of EEOR Ideal Practice #1: National Program Outcome Definition and Measurement

The IPM Program's utilization of practice #1 is summarized in Table 2.

The Smith Lever IPM Program articulates four broad national goals:

1. To safeguard human health and the environment through improved utilization of integrated pest management strategies and systems (conditions outcomes through action outcomes).

2. To increase the range of benefits obtained through improved utilization of integrated pest management strategies and systems (condition outcomes through action outcome).

3. To increase the implementation of effective integrated pest management strategies and systems (action outcome).

4. To enhance collaborations among stakeholders interested in the development and implementation of improved integrated pest management strategies and systems (activity output to improve action outcomes). (Reprinted by permission of CSREES from the Performance Planning and Reporting Web site, 2002.)

From 1995 to 2000, the national program leadership defined and measured progress toward the intermediate outcome of IPM adoption (action outcome) through third party data. A goal was set of 75% nationwide IPM adoption by the year 2000, which is a research-supported proxy for reduced pesticide use.

The program is currently concluding the stakeholder input phase of a process to define new national measures with a stronger emphasis on condition outcomes (Hoffman, 2002). These new national measures are being developed in response to a 2001 General Accounting Office report that urged a stronger tie between program objectives and reductions in pesticide use (GAO, 2001). Results of this process will influence future measurement of conditions and action outcome proxies produced and measured nationally by the program.

Program Utilization of EEOR Ideal Practice #2: IPM State Outcome Reporting

The IPM Program's utilization of practice #2 is summarized in Table 3.

Statewide program coordinators choose commodities or pest management situations important to their state as areas of program emphasis and then decide which outcome and non-outcome indicators best match the efforts on that commodity. Maine may choose to report on pest management efforts in potatoes, sweet corn, and apples. Michigan could choose to report on broccoli, blueberries, and potatoes (all five commodities are grown in both states). The two states also choose to report progress using one or all of the following 16 Smith Lever IPM Program indicators of outcomes, outputs, inputs, and processes:

1. Number of production units or entities using IPM (action outcome),
2. Transition from high risk to lower risk pesticides (action outcome),
3. Total amount of high risk pesticides applied (action outcome),
4. Diversity of IPM practices adopted (action outcome),
5. Economic benefit obtained (condition outcome),
6. IPM Personnel employed (input),
7. Satisfied IPM clientele (participation output),
8. IPM strategies and systems validated (activity output),
9. IPM educational materials delivered (activity output),
10. People participating (participation output),
11. Producers trained (participation output),
12. Private sector personnel trained (participation output),
13. Public sector personnel trained (participation output),
14. Other individuals trained (participation output),
15. Public events involving collaborations (activity output), and
16. Non-federal dollars leveraged (input). (Reprinted by permission of CSREES from the Performance Planning and Reporting Web site, 2002)

Though numbers 1-5 can provide evidence of the individual state program's role in achieving national outcomes, this commodity and indicator selection latitude often prevents meaningful outcome data aggregation. This lack of data aggregation is important for two reasons.

First, if the national leadership of the program would like to assess its outcomes related to blueberries, this data would be incomplete unless all major blueberry-producing states choose to report on that commodity. Second, even if all major blueberry-producing states choose to report on the commodity, this data would be difficult to compile unless each state self-selected the same outcome indicators. If the program were trying to "roll up" the state outputs to come up with national outcome data, this would present a serious problem. The fact that the national program leadership measures national outcomes using third party data makes this lack of aggregation somewhat less important.

Furthermore, it is possible under current guidelines for a state to select only from indicators 6-16, thus not reporting on outcomes. Efforts are currently underway to require at least one outcome indicator for each program and encouraging one outcome indicator from each area of emphasis.

Program Utilization of EEOR Ideal Practice #3: IPM State Non-Outcome Reporting

The IPM Program's utilization of practice #3 is summarized in Table 4.

For the crops identified, programs can choose to report on non-outcome indicators numbers 6-16 from the 16-item list above.

In addition to this crop-specific data, the state programs are asked to provide program wide narratives and resource information. Five-year plans of work and annual reports are used to report assumptions and external factors in narrative form, and alternate funding (input) data in numerical form.

As mentioned earlier, it is possible under current guidelines for non-outcome data to completely replace outcome measurement on the state level through local indicator selection. The national program leadership is currently attempting to close this loophole.

Results from Using the Scorecard for the Extension IPM Program

When the Extension IPM Program's Extension outcome and reporting practices were compared to the Ideal EEOR practices using the Hoffman EEOR Scorecard, three major areas for further improvement were identified:

• Demarcation of outcome measures verses non-outcome supporting data for state level reporting to ensure the collection of both;

• Use of third party data for use as an efficient measurement tool; and

• Multi-state cooperation for goal setting and outcome measurement to foster more meaningful data collection, reporting, and aggregation.

Current proposed guidelines designed to separate outcome measures from non-outcome supporting data should be implemented as soon as practical, and/or this tactic should be a part of any future proposed changes in the state evaluation and reporting system. As new national outcome measures are formed, every effort should be made to seek out third party data as guided by the scorecard at the federal and state levels to improve the overall quality of evaluation and outcome reporting and ease the reporting burden on individual awardees. As these measures are more closely linked to condition outcomes, data availability on condition outcomes and closely linked action outcome proxies should be thoroughly investigated. Finally, cooperation among states to coordinate outcome measurement could provide greater opportunities for data aggregation and more meaningful results interpretation.

Conclusion

When a judge examines a group of dogs, chickens, or cows at an animal show, he or she typically compares each member of the class to a theoretical ideal animal. Regardless of their ranking within the class, the owners and breeders of those animals are given valuable information on ways to improve their kennel, flock, or herd so successive generations of their stock may approach that ideal. The three EEOR ideal practices described in this article, along with the scorecard to evaluate their utilization, are not unlike that theoretical ideal animal that is used for comparisons.

Using the Hoffman EEOR Scorecard and making such comparisons can help Smith Lever 3(d) program leaders identify how closely their practices come to the three EEOR ideal practices. Such a comparison is potentially useful in diagnosing where current evaluation efforts could be improved and the general direction that this improvement could take. This information can help program leaders to:

• Alter the program's overall evaluation and outcome reporting framework to further GPRA compliance,

• Identify third party data that can serve as an outcome measurement indicators for ultimate programmatic outcomes through national goal clarification,

• Draw clear distinctions between outcome versus non-outcome measurements that could foster clear communications to individual awardees, and

• Ensure that reporting efforts undertaken by individual awardees and/or groups of awardees complement national measurement efforts.

For the example program documented in this article, Extension IPM, this comparison yielded three major areas for further improvement:

• Demarcation of outcome measures verses non-outcome supporting data for state level reporting to ensure the collection of both;

• Use of third party data for use as an efficient measurement tool; and

• Multi-state cooperation for goal setting and outcome measurement to foster more meaningful data collection, reporting, and aggregation.

Based on examination by this scorecard, the Extension IPM program is pursuing these three areas of potential improvement at this time.

References

Bennett, C. F. (1975). Up the hierarchy. Journal of Extension [On-line], 13(2). Available at: http://www.joe.org/joe/1975march/index.html

Extension Committee on Organizational Policy (ECOP) (1997). Strategic directions of the cooperative extension system. Retrieved May, 2002 from: http://www.reeusda.gov/part/gpra/direct.htm

GPRA Page (CSREES Web site). (n.d.). Retrieved May, 2002 from: http://www.reeusda.gov/part/gpra/gprahome.htm

Hoffman, W. (2003). Smith lever 3(d) program evaluation and outcome reporting -- a federal perspective. Unpublished master's thesis. Penn State University.

Houlton, E. F., III (1996). The flawed four-level evaluation model. Human Resource Development Quarterly, 7.

Kirkpatrick, D. L. (1998). Evaluating training programs (2nd ed.). San Francisco: Berrett-Koehler.

Minimum standards for Extension IPM implementation program annual reporting. (2002). (CSREES Working Paper) Washington, DC.

Nelson, D. E. (1999). Generic observations with regard to the 2000-2004 plans of work. Retrieved May, 2002 from: http://www.reeusda.gov/part/areera/generic.htm

Richardson, J. G. (2001). Proactively addressing accountability in Extension. The Forum [On-line], 6, 2. Retrieved July, 2002 from: http://www.ces.ncsu.edu/depts/fcs/pub/2001sp/richardson.html

North Carolina State University (NCSU). (n.d). The performance planning and reporting system. Retrieved June, 2002: http://www.pprs.info

Rogers, E. M. (1995). Diffusion of innovations (4th ed.). New York: The Free Press.

Stake, R. E. (1975). Evaluating the arts in education, A responsive approach. Columbus: Merrill.

Taylor-Powell, E., Steele, S., & Douglah, M. (1996). Planning a program evaluation. Retrieved July 2002, from University of Wisconsin-Extension-Cooperative Extension, Program Development and Evaluation Unit Web site: http://www1.uwex.edu/ces/pubs/pdf/G3658_1.PDF

United States General Accounting Office. (1996). Executive guide: Effectively implementing the government performance results act. GAO/GGD-96-118. Washington, DC: USGAO.

United States General Accounting Office. (1997). Managing for results: Analytic challenges in measuring performance. GAO/HEHS/GGD-97-138. Washington, DC: USGAO.

United States General Accounting Office. (1998). Managing for results: Measuring program results that are under limited federal control. GAO/GDD-99-16. Washington, DC: USGAO.

University of Wisconsin Cooperative Extension (UWCE). Evaluation logic model (2002). Retrieved November, 2003 from http://www.uwex.edu/ces/pdande/evaluation/evallogicmodel.html

United States General Accounting Office. (2001). Agricultural pesticides: Management improvement needed to further promote integrated pest management. GAO-01-815. Washington DC: USGAO.

Worthen, B. R., Sanders, J. R., & Fitzpatrick, J. L. (1997). Program evaluation: Alternative approaches and practical guidelines, 2nd ed. Boston: Addison, Wesley, Longman.

Moving Towards Ecologically Based Pest Management: A Case Study Using Perimeter Trap Cropping

T. Jude Boucher
Associate Extension Educator/Sustainable Agriculture
University of Connecticut Cooperative Extension System
Vernon, Connecticut
jude.boucher@uconn.edu

Robert Durgy
Program Assistant
Department of Plant Science
University of Connecticut
Storrs, Connecticut
robert.durgy@uconn.edu

Introduction

In recent years, Integrated Pest Management (IPM) Programs have been criticized for relying too much on chemical solutions and for having a low adoption rate of low-risk, biologically based tactics (Anonymous, 2001; Ehler & Bottrell, 2000; Lewis, van Lenteren, Phatak, & Tumlinson, 1997). Some have argued that the original meaning of the term "integrated" in IPM has been lost. It is claimed that the term originally referred to "compatible" management techniques that minimized disruption of natural enemies (Ehler & Bottrell, 2000).

Lewis et al. (1997) argue that we must move away from quests for short-term, therapeutic interventions ("silver bullets"), such as pesticides, that merely treat the symptoms of an unbalanced ecosystem. They advocate that researchers concentrate on developing long-lasting solutions that build in an array of preventative, natural regulators.

This means that merely switching from chemicals to selective microbial pesticides, biocontrol agents, or biotech products does not address the underlying weaknesses of conventional pest management systems. The basic tenet described by Lewis et al. (1997) and others is that nature will always counter therapeutic approaches and render them ineffective in the long term. Therapeutic tactics should serve as a backup to built-in preventative measures while balance is restored to the system, not as the primary means of pest control.

To implement an ecologically based approach to pest management, we need to modify crop production designs, using principles capable of containing population levels of a variety of pests and pest complexes on multiple commodities. In ecological terms, this involves moderating or dampening pest populations' boom and bust cycles so that populations of individual pests remain at low carrying capacities and, ideally, below economic thresholds. To accomplish these goals, Lewis et al. (1997) suggest that new designs concentrate on managing the farm environment through ecosystem enhancements (i.e., landscape ecology), crop attributes, or other means that help stabilize the population of species throughout the food web.

To scientists and Extension educators involved exclusively in conventional agriculture or pesticide-oriented IPM programs, Lewis' suggestions might seem little more than a pipe dream. It is often hard to imagine how a redesigned system might successfully incorporate good ecological principles, eliminate pesticides from crops, and meet all the short-term demands of modern agriculture and the marketplace. Perimeter trap cropping represents one possible redesign of the crop production and pest management system that incorporates natural population regulators, plant attributes, and a conservative trap crop spatial orientation to improve pest control.

Definition and Function of Perimeter Trap Cropping (PTC)

Webster's Dictionary (Guralnik, 1980) defines "perimeter" as "the outer boundary of a figure or area" and as "a boundary strip where defenses are set up." Perimeter trap cropping involves planting a more attractive host plant to completely encircle and protect the main cash crop like fortress walls. Other perimeter defenses such as border sprays or biological, mechanical, and/or cultural controls can be added to help increase efficacy.

Perimeter trap cropping functions by intercepting pest migration, regardless of the direction of attack. It then concentrates pest population(s) in the border area, where they can be retained or controlled. Natural enemies are conserved by eliminating insecticide use on the main cash crop. Crop losses can be further reduced if the target pest(s) are also disease vectors (Boucher & Durgy, 2004).

Perimeter trap cropping has provided excellent pest control and dramatically reduced pesticide use and costs on a variety of crops. Researchers in Florida were able to keep diamondback moth infestations from reaching action thresholds in commercial cabbage fields by surrounding them with two rows of collards (Mitchell, Hu, & Johanowicz, 2000). A naturally occurring parasitic wasp helped control the moth population on the collards and reduced the number of individuals that spread into the cabbage. Cabbage fields protected by the PTC system in the study used 56% fewer insecticide sprays than conventional fields and saved $47 to $63 per acre in chemical costs.

Brewer and Schmidt (1995) used early-maturing sunflowers to surround and protect oilseed sunflowers from the red sunflower seed weevil. Yield and damage levels were similar in PTC sunflower fields and those treated with full-field sprays, but the trap crop system was more economical. Aluja et al. (1997) almost eliminated papaya fruit fly damage in an unsprayed papaya planting in Mexico by using a PTC system.

Boucher and Durgy (2003) used a sprayed trap crop of 'Blue Hubbard' squash around summer squash to reduce cucumber beetle populations by up to 93% compared with check plots. Commercial growers using PTC for squash stated that the system improved and simplified pest control, reduced pesticide use (93%) and crop loss (18%), and saved time and money compared to their conventional programs (Boucher & Durgy, 2004). Boucher, Ashley, Durgy, Sciabarrasi, and Calderwood (2003) used hot cherry peppers to protect bell pepper plots from attack by the pepper maggot. In research plots, pepper maggot damage was reduced by over 90% using PTC. Economic analysis of PTC use in a commercial pepper field showed that net profits were increased by $153 per acre.

Field Demonstrations

In 2003, four Connecticut growers surrounded 18.25 acres of peppers with a hot cherry pepper trap crop. The PTC plantings ranged from 1 to 12 acres in size. These growers have all used PTC to protect their bell peppers from pepper maggot for 2 to 4 years. One additional grower tried to protect eggplant from pepper maggots with hot cherry peppers for the first time in 2003. He converted to PTC after having 100% of the eggplant crop damaged by maggots in the past few years, despite multiple full-field sprays. There are currently no effective insecticides registered to control pepper maggots on eggplant.

Growers used 1 to 4 rows of trap crop along the length of their plantings. Two to six cherry pepper plants were used at both ends of each pepper or eggplant row to complete the perimeter barrier. The trap crop was transplanted at the same in- and between-row spacing as the main crop. Two growers used bare-ground culture to produce their peppers or eggplants, while the others used a plastic-mulched system of production, with either trickle or overhead irrigation.

Prior to their first season using pepper maggot PTC, Extension personnel met with growers and provided fact sheets and advice to help them implement and maintain the system on their farms. Certain important concepts were emphasized with growers prior to the beginning of the program:

1. Plant the trap crop on good ground, so that it remains healthy and completely encircles the main crop, without large gaps in the perimeter;

2. Apply a foliar insecticide application to the perimeter within a week of finding pepper maggot fruit stings (oviposition scars) on the trap crop or within a week of capturing adult flies on baited traps (Boucher & Ashley, 2001); and

3. Monitor the field continuously until mid-August and be prepared to make 1-3 additional perimeter applications, if necessary.

Repeat perimeter applications were considered justified if additional pepper maggot flies were found while checking traps at weekly intervals, or if stings continued to accumulate on the trap crop fruit (Boucher & Ashley, 2001). Full-field sprays for pepper maggot have never been needed on any of the farms while using PTC with sprayed perimeters. Extension personnel helped three growers monitor pest populations and time perimeter pesticide applications in 2003. The other growers did their own monitoring or scouting after the year of their initial training, or relied on a consultant to provide information about insecticide timing.

Growers used backpacks, boom sprayers, or mist blowers to apply dimethoate or acephate to the trap crop row(s). Three growers who used boom sprayers or mist blowers applied sprays to the outer 12 to 25 feet of the block (trap and main crop) by circling the field once.

At the end of the 2003 growing season, participants were surveyed and asked to compare the results of using the pepper maggot PTC system to prior years using a conventional program that relied on full-field insecticide sprays. Growers were asked to comment on PTC and to rate a list of possible benefits on a scale of zero (no benefit) to three (high benefit). They were also asked to rate the PTC program for simplicity/complexity, describe their overall satisfaction level with the system, and rate the training program overall.

Results of PTC User Surveys

All (100%) of the growers stated that their pest control was much better using PTC than in previous years without a trap crop system. Mean damage estimates due to pepper maggots were 12% using multiple full-field sprays and < 1% using PTC.

All respondents reported pesticide savings using the trap crop system. Growers applied an average of 1.4 insecticide sprays targeting pepper maggots to the trap crop, compared with 2.2 full-field sprays using their conventional programs. The use of insecticide active ingredient for pepper maggot control was reduced by 0.7 pounds per acre (90%) using PTC. All growers had a history of applying additional sprays for aphids or other secondary pests prior to using PTC. None of the growers required sprays for secondary pests in 2003.

Growers estimated that the total cost of installing and maintaining the PTC system ranged between $30 and $93 per acre, yet all said that they saved money using the trap crop system. They estimated their overall savings from using PTC at between $1 to $1,000 per acre and attributed most of the savings to improved pest control, crop quality, and yields.

Eighty percent of the growers rated the PTC system as simpler (20%) or much simpler to use (60%) than their traditional pest control program on peppers, while one grower said that using a trap crop was a little more complex. Sixty percent of the growers stated that they saved time using the PTC system compared to using their conventional program. One producer said that PTC took approximately the same amount of time as multiple full-field sprays, and one grower said that it took more time.

All of the growers gave the PTC system high marks when rating a list of possible PTC benefits (Table 1). A majority of respondents rated 10 of the categories a 3, the highest possible rating. They also mentioned the following additional benefits of using PTC: the security of "knowing that you're controlling the pest" with PTC and doing "less mechanical damage by not going in [to the field] with a sprayer when the [eggplant] crop is big and has lodged."

All program participants said that they were either very satisfied (60%) or thrilled (40%) with the overall performance of the trap crop system. All final comments about PTC were positive:

• "PTC works well and does its job...the key is you still have to do that scouting."

• "The more I use PTC, the more comfortable I am using it."

• "Perimeter trap cropping works."

• "I highly recommend PTC, especially for big commercial growers...you're crazy not to do it!"

All growers rated the training program as excellent and stated that they would continue using the pepper maggot PTC system in the future.

Implications for Extension and IPM

Despite almost a half-century of IPM research and Extension efforts, pesticide usage continues to rise (Anonymous, 2001; Lewis et al., 1997). Scientists and policy-makers are beginning to express frustration with the lack of progress and are beginning to point the finger at IPM as part of the problem (Anonymous, 2001; Ehler & Bottrell, 2000; Lewis et al., 1997). Yet many Extension IPM programs remain heavily dependent on multiple, full-field pesticide applications as the primary or solitary pest control tactic (Hoffman, 2000). The monitor and spray approach has even picked up a derogatory nickname: "integrated pesticide management."

Policy-makers are now calling for future government funding to be tied to true pesticide reductions (Ehler & Bottrell, 2000). These same critics are no longer willing to accept merely substituting one pesticide for another low-use-rate chemical as progress. They want the amount of crop acreage treated with chemicals to be reduced. If Extension and IPM are not viewed as capable of reducing future pesticide use, organizations with more sustainable ideas on pest management may soon become the leaders in agricultural education, with government funding following them.

Critics also fear the consequences of substituting new microbial or biotech products, insect growth regulators, or other selective materials without understanding the potential impact on the biota of the agricultural ecosystem. For instance, Extension specialists often tout the substitution of a selective microbial pesticide (e.g., spinosad) for an older product as an advancement (Burkness, Hutchison, & Pahl, 2000). However, sometimes evidence surfaces at a later time that shows just how detrimental such a material can be to important parasitoids in the cropping systems (Lyon, Van Driesche, Smith, & Lopes, 2002). Critics are calling for a rethinking of Extension's methodology, along with new cropping designs that take advantage of natural preventative measures that can suppress pest populations.

Extension continues to blame low adoption rates for alternative management practices on "conservative grower attitudes" (Hoffman, 2000). Meanwhile, farmers complain that many IPM programs are "too difficult to implement" and that they have limited time for things such as scouting (Hoffman et al. 1997). The fact that it takes an entire book or manual (Boucher & Ashley, 2001) to provide IPM recommendations for a single commodity may help explain the slow adoption of alternative-based programs. Farmers have too much to do and often reach for the quickest or simplest solution to save time and to improve their quality of life. For Extension to increase the adoption rate of IPM programs, we need simpler solutions to problems that are often complex.

Finally, Stephenson (2003) has criticized Extension for relying on outdated versions of the Innovative Diffusion Theory and for targeting innovative new programs at well-educated, wealthier farmers, who tend to be successful early innovators. He cautions that some of our more elaborate solutions may only be applicable to a select group of elite growers and may even harm the disenfranchised portion of our clientele. Extension must design solutions and use delivery methods that are inclusive of the greater farming community, or risk being held accountable for who succeeds and who doesn't.

Summary

Perimeter trap cropping is an example of a redesigned crop production system that helps bring pest populations down to acceptable levels with a minimum of ecological disruption, as advocated by Lewis et al. (1997). This system attempts to minimize disruption from therapeutic approaches, but does not seek to eliminate pesticides entirely. Perimeter trap cropping produces true pesticide use reductions. It is a simpler, cheaper solution that works on multiple crops and pests.

The technology used in PTC is applicable to all growers, not just the better educated or wealthier growers who tend to be early implementors. Perimeter trap cropping involves relatively simple changes in the crop production system that produce substantial advantages. An array of ecologically based solutions must be developed for Extension to maintain its leadership role in the area of pest control in the 21st century.

Acknowledgment

Funding for this project was provided by USDA CSREES NE IPM and NE SARE competitive grant programs.

References

Aluja, M., Jimenez, A., Camino, M., Pinero, J., Aldana, L., Caserjon, V., & Valdes, M. E. (1997). Habitat manipulation to reduce papaya fruit fly (diptera: tephritidae) damage: Orchard design, use of trap crops and border trapping. Journal of Economic Entomology, 90: 1567-1576.

Anonymous. (2001). Agricultural pesticides: Management improvements needed to further promote Integrated Pest Management. US General Accounting Office Publication Number. GAO-01-815.

Boucher, T. J. & Ashley, R. A. (Eds). (2001). Northeast pepper Integrated Pest Management (IPM) manual. University of Connecticut Cooperative Extension System Publication. pp. 136.

Boucher, T. J., Ashley, R., Durgy, R., Sciabarrasi, M. & Calderwood, W. (2003). Managing the pepper maggot (diptera: tephritidae) using perimeter trap cropping. Journal of Economic Entomology, 96(2): 420-432.

Boucher, T. J. & Durgy, R. (2003). Perimeter trap cropping for summer squash & cucumbers. Proceedings: New England vegetable & berry conference and trade show. pp. 217-219.

Boucher, T. J. & Durgy, R. (2004). Demonstrating a perimeter trap crop approach to pest management on summer squash in New England. Journal of Extension [On-line], 42(5). Available at: http://www.joe.org/joe/2004october/rb2.shtml

Brewer, G. J., & Schmidt, G. (1995). Trap cropping to manage the red sunflower seed weevil in oilseed sunflower. American Journal of Alternative Agriculture, 10: 184-187.

Burkness, E., Hutchison, B. & Pahl, G. (2000). Implementing cabbage IPM: The value of on-farm research. Applied vegetable IPM relative to crop management: ESA proceedings for the formal conference in vegetable entomology. University of Georgia Agricultural Experiment Station Research Report Number 669. 

Ehler, L. E. & Bottrell, D. G. (2000). The illusion of Integrated Pest Management. Issues in Science and Technology, Spring 2000: 1-6. Available at: http://www.nap.edu/issues/16.3/ehler.htm

Guralnik, D. B. (Ed). (1980). Webster's New World Dictionary. William Collins Publishing, Inc. Cleveland, OH.

Hoffmann, M., Petzoldt, C., Prostak, D., Fleisher, S., Spangler, S., Zitter, T., Reiners, S., Bellinder, R., Shelton, A., Eckhardt, L. & Hetherington, M. (1997). Integrated Pest Management for diversified fresh market vegetable producers in New Jersey, New York & Pennsylvania. Progress Report Phase I IPM Management Initiative Project.

Hoffmann, M. P. (2000). IPM for onions: Present and future challenges. Applied vegetable IPM relative to crop management: ESA proceedings for the formal conference in vegetable entomology. University of Georgia Agricultural Experiment Station Research Report Number 669. 

Lewis, W. J., van Lenteren, J. C., Phatak, S. C., & Tumlinson, J. H. (1997). A total system approach to sustainable pest management. Proceedings National Academy of Science. USA 94: 12243-12248.

Lyon, S.,Van Driesche, R. G., Smith, T. & Lopes, P. (2002). Spinosad (Conserve®): Can we use it in combination with natural enemies in our greenhouses? UMass Floral Notes, May-June 2002: 10-13.

Mitchell, E. R., Hu, G., & Johanowicz, D. (2000). Management of diamondback moth (lepidoptera: plutellidae) in cabbage using collard as a trap crop. HortScience, 35: 875-879.

Stephenson, G. (2003). The somewhat flawed theoretical foundation of the Extension Service. Journal of Extension [On-line], 41(4). Available at: http://www.joe.org/joe/2003august/a1.shtml

A Profile of Female County Agricultural Agents in Today's CES

Brenda S. Seevers
Professor
Department of Agricultural and Extension Education
New Mexico State University
Las Cruces, New Mexico
bseevers@nmsu.edu

Billye B. Foster
Associate Professor
Department of Agricultural Education
University of Arizona
Tucson, Arizona
billye@ag.arizona.edu

Introduction

Who could anticipate the plethora of choices and careers open to the educated woman in 2004? Fields and professions once monopolized by men now open their arms to the throngs of educated and enthusiastic young women. Even the most traditional and conservative fields have found value in the acceptance of gender diversity. In 1991, The Council on Diversity in Extension submitted their strategic plan " Valuing Differences and Celebrating Diversity," emphasizing the need to strengthen diversity and pluralism in the Cooperative Extension Service (CES).

However, in 1996, an analysis of CES professional staffing data showed that only a slow and minimal process of change in recruitment, selection, and retention practices had occurred. Further findings indicate that women and minorities were significantly under represented in all levels of management and all areas of CES. While CES has long used women in the areas of family and consumer education, agriculture remained the domain of men. In determining the population for the 2003 study discussed here, the researchers found that little progress had been made. Less than 12% of all CES agents with agricultural responsibilities nationally were women.

Statement of the Problem

Like any other institution, the American labor force continues to experience multiple changes. In 1997, women accounted for 46% of the labor force as compared to only 29% in 1950. Other changes are more interesting. For example, 90% of male executives under 40 are fathers, yet only 35% of female executives under 40 are mothers (National Multicultural Institute, 1997). The U.S. Department of Labor also reports that the ratio of women's earning to men's earning is still unbalanced. In 1970, women averaged 59.7% of men's salaries in similar positions. In 2000, women averaged 76% of the male counterparts' salaries (Women's Bureau, 2001).

In 1986, the mythical "glass ceiling" was first labeled by two Wall Street reporters in reference to the invisible barriers that block women from top jobs (Catalyst Report, 1993). The U.S. Department of Labor (1991) defines the "glass ceiling" as artificial barriers based on attitudinal or organizational bias that prevent qualified women from advancing into mid- and senior level administrative positions.

Similar patterns in attitudes, barriers, and bias are seen when women desire to take on nontraditional occupations. The US Department of Labor lists over 110 nontraditional occupations for women. Nontraditional is defined as any occupation where one gender comprises 25% or less of the total employees (USDOL, 2001). Women in agricultural and Extension education are considered a minority population, or nontraditional.

In a study by Foster (2001), the number of female secondary agricultural education teachers nationally was 16%. This number had not changed in more than 10 years. At the university level, female agricultural and Extension educators in academic departments comprised only 14.7% of the total population (Seevers & Foster, 2003). Maddy's qualitative study (1991) using a census of female CES directors or associate directors had only a population of eight.

In a traditionally male dominated field, like agricultural and Extension education, the concept of the "glass ceiling" is a real and dominant force. Barriers inhibiting women in nontraditional fields are complex and inter-related. According to a 1999 survey by Catalyst, the barriers to women's advancement as seen by successful women included:

1. Male stereotyping and preconceptions about women,

2. Exclusion from informal networks of communications, and

3. Lack of significant experience (Catalyst, 2001).

In more recent studies, the top barriers facing women in agricultural education at the secondary level (Foster, 2001) and university women in agricultural and Extension education (Seevers & Foster, 2003) were:

1. Acceptance by peers and other males in the agricultural industry,

2. Balancing family and career, and

3. Acceptance by administrators.

According to USDA (ECOP-PODC, 1997), barriers facing women and other minorities in CES include lack of commitment from senior managers and university administration, resistance of some clientele groups to work with staff from diverse backgrounds, and lack of specific goals and targets for attaining a diverse workforce.

Purpose of the Study

The study discussed here was designed to create a profile of women involved in Cooperative Extension as county agents with primary responsibility for agricultural programs for adult audiences. Knowledge about women who have pioneered positions in Extension education provides valuable role model information for upcoming generations of female educators. Additionally, the study sought to describe challenges and barriers perceived to be unique to women in agricultural Extension.

Methods and Procedures

The population of the study was a census of female county Extension agricultural agents in 49 states. One state declined to participate in the study. Subjects were identified using the 2002-2003 County Agents Directory. Names were verified when necessary by contacting states to obtain or verify information and/or using state developed websites. The final accessible population was N = 488.

The instrument created by the researchers contained five sections designed to address the objectives of the study. Face and content validity were assessed using a panel of experts in research/statistics and agricultural education and 21 Cooperative Extension county 4-H agricultural agents or state specialists with CES in agriculture. The slightly modified instrument has been used in two previous studies (Foster, 2001; Seevers & Foster, 2003). Reliability assessment for those two studies was high. Reliability was not reassessed for this study.

Data were collected April through June 2003 following a modified Dillman's (1978) procedure for a mail questionnaire. Responses from early and late subjects were compared, and no differences were found to exist. The results were generalized to the target population (Miller & Smith, 1983). The final usable response rate was 79% (N = 386). Descriptive statistics were used to summarize the data.

Findings

The majority of female county agricultural agents are married (68.1%) or have never been married (21%). Less than 10% of the respondents indicated they were divorced or widowed. Only slightly more than 50% of the subjects reported having children. Ages of children ranged greatly from infants to grown adults. The majority of women fell in three age categories: 26- 30 (18.5%), 41-45 (18.8%), and 46-50 (18.5%). Ethnicity reported was overwhelmingly Caucasian, with 93.2% (N= 359). Minority populations of Hispanic, African-American, and Native American all reported between 1.3 and 1.8%s of the total population.

Women working with agricultural programs within CES have been employed from 1 to 32 years, with a mean of 8.6 years. Areas of specialty ranged widely to include the common areas of livestock and crop production to more uncommon areas of marine science and maple syrup production. Over 42% of the subjects reported their professional duties involved serving clientele in more than one county. Salary ranges varied greatly, with the largest number of respondents reporting an annual salary between $40,000 - $44,000. Salaries over $65,000 were rarely reported.

Several questions related to job responsibilities and division of time. Slightly less than 50% of the respondents reported holding administrative responsibilities in relation to their position. Subjects reported spending the greatest percent of their time working with adults in agricultural programs (70%), followed by 4-H and youth programs (18%) and other duties (9%). Only 30% of the respondents reported supervision of interns as part of their job responsibilities. The number of interns supervised varied, but the majority had supervised only one student intern.

Additionally, subjects were asked the degree of comfort they felt with current job responsibilities and the degree of preparedness they felt to address the content areas in their current position (Table 1). Responding to a 5-point Likert-type scale, the majority of respondents were very comfortable or comfortable (90.5%), while over 85% felt prepared or very prepared to address content areas within their position.

A summary of major job responsibilities addresses almost every process and subject area possible. Agents were involved in needs assessment, planning, teaching, evaluation, serving on boards and committees, making farm/ranch visits, conducting research, and working with clients, volunteers, media, and the larger community. They addressed every topic between agronomy and zoology. Common areas of expertise were livestock, crop production, horticulture, and agri-business. Despite the content area, the primary responsibility reported was meeting the needs of clientele served.

Most female agricultural agents reported their highest educational degree as a masters (68.1%). Some reported only a bachelor's degree (21.6%), while 8.3% had a doctorate or equivalent. Degree areas were almost exclusively in specialized agricultural disciplines, including animal science, agronomy and horticulture, natural resources, and agri-business. Thirty-six percent of the women expressed an interest in achieving a higher degree. Previous experiences in agricultural and Extension education while in high school were high, although more subjects participated in 4-H (34.8%) than FFA (20.5%). Those that did not participate in either program identified the primary reasons as agricultural education classes were not available or not having an interest.

Almost 80% of the respondents reported an average of 5.5 years of agriculture-related work experience prior to entering work with the Cooperative Extension Service. Type of work experience varied greatly. In addition, many subjects (60.2%) completed professional internship experiences prior to being hired by CES.

When asked if they felt they had experienced any barriers or challenges in their profession due to gender, 57% of the women responded positively. Common themes emerged in the identification of these challenges. The most common were: lack of acceptance from male colleagues and clients; the need to "prove yourself"; no mentoring or inclusion by male peers; and the "good 'ole boy system." One woman said, "Peers don't seem to give the same level of respect for subject matter knowledge. Clients are at times hesitant to talk to a woman." Others recognized the challenges and coped the best they could. One woman said, "Folks still call and say, 'is the man in?' I laugh and say that the man is now a woman!"

Women were also asked to identify what they perceived to be the greatest barrier faced by female agents with agricultural responsibilities even if they had not experienced them. Responses were similar to the ones previously mentioned, but additional comments included topics such as personal attitude, balancing work and family and lack of mentors or role models. One women said, "The greatest barrier is their [the woman's] own attitude of not being good enough." Another respondent said, "there are not many women as mentors/examples; it's still a bit of a man's profession." And finally, one individual summed it up by saying, "the typical sexist barriers--lower pay; glass-ceilings; harder to get promoted; having to do it better than our male co-workers; not having 'someone like me' to work with; It's very difficult during pregnancy or with young children."

When asked to identify any sacrifices made to reach a current level of achievement in their career, the number of responses was significant. Common sacrifices noted included time away from family; lack of personal or social time; the decision to not have more children or to delay having a family; and a firm commitment that the pay offered is too low for the educational requirements of the job. When asked if they would make the same sacrifices again, 42.0% said "yes," 13.7% said "no," and 25.1% were unsure. Many subjects agreed that sacrifices were made but also felt that other careers also demanded decisions and sacrifices.

Women Extension agents in agricultural programs are satisfied with their jobs. Almost 85% of all respondents reported being satisfied or very satisfied. Additionally, more than 81% would encourage other women to pursue the same career in CES.

However, when asked to identify why they would support or encourage others, the responses varied greatly and were not as overwhelmingly positive. One woman said that, "until we are paid our worth and are treated with respect, there's no need to encourage anyone to go into this discipline." Others responded positively without hesitation, "It is valuable, productive work. We make a difference in people's lives and environment. I could be their mentor and help them." Still others had a positive experience themselves but were reluctant to encourage others due to unique circumstances. One woman wrote, "Not in the current budget and political climate. I have a feeling that Extension's days are numbered."

Conclusions/Implications/Recommendations

A profile of female county Extension agents with agricultural program responsibilities reveals that most are between the ages of 26 -30 and 41-50. There was a noticeable drop in the age category between 31-40. This age range is typically when women are having and/or raising their children. Many women struggling to balance work and family will choose to leave the profession or seek other employment.

How do we recruit or retain quality employees in positions that demand extra time and irregular hours? Can jobs be modified? Are there some tasks that could be completed with properly trained community volunteers? Can jobs be shared? In the traditional patriarchal society, the male is the breadwinner, and the female is the nurturer and caregiver. Should any employee be forced to make a decision between a career and a family?

Extension agents are predominately Caucasian. This is consistent with previous studies of women in agriculture. It should be noted and of concern that not only are women in general under represented in these professions, but women of an ethnic minority background are severely under represented. A pressing issue with the Cooperative Extension Service is diversity and pluralism. What can be done to attract women and minorities in these roles? 

The majority of female agents were married, and more had children than did not. However, more Extension agents fell into the never married category than did secondary agricultural education teachers (Foster, 2001) or university women faculty (Seevers & Foster, 2003). Extension agents were well educated, with the majority possessing a master's degree. This is not a surprising finding because most states require a master's degree for hire or obtaining one within a specified period of time.

However, average salary levels for the position were only between $30,000 - $40,000. Consistent comments from the subjects reported that the salary was too low and the hours too long for the education required for the job. Given low salaries and demanding hours, is CES able to attract and retain the best and brightest? Previous studies have indicated these types of motivational factors are common causes for high turnover.

Subjects found to have more than 5 years of agricultural work experience prior to beginning a career with Extension were likely to have been involved in either 4-H or FFA in high school. Coming from an agricultural background better prepares individuals for the roles and responsibilities they will be undertaking as well as assists them in understanding the clientele they will be serving.

Extension agents, overall, were very satisfied with their current position in CES. They felt comfortable with expected job responsibilities and were prepared to address the content areas of the position they were hired for.

Most women had completed at least one internship prior to working with Extension. While practical internships provide only a snapshot of an organization or job, they are key experiences in helping to understand the organization, the career, and the job. Many career decisions made are based on the quality of an internship experience. They can be a valuable tool in matching they right person to a profession and reducing turnover. Internships as pre-service experiences should be required and continued. Women were not only satisfied with their job, they also felt they were supportive and encouraging to other women in the profession or desiring to be in the profession.

Regardless of gender, CES jobs are demanding of time and energy. Despite a high level of job satisfaction, almost 60% of the women felt they had experienced barriers and challenges in their profession as a result of gender. These barriers are consistent with those previously identified in other disciplines as well as in agricultural education (Catalyst Report, 1993; GenderWatch, 2001; Williams, 2001; Foster, 2001; Seevers & Foster, 2003). Barriers (perceived and real) can be addressed only through awareness and communication, but first there must be agreement that they do exist. Lack of acknowledgment or failure to address ultimately leads to conflict, job dissatisfaction, and high turnover rates.

References

Catalyst. (1993, December). Successful initiatives for breaking the glass ceiling to upward mobility for minorities and women. (Report funded for USDA, Glass Ceiling Commission) New York: NY

The changing face of the United States. (1998, April). Value added: Celebrate diversity in the college of agriculture. 5(4) 1. As cited in the National Multi-Cultural Institute, 1997.

Commission on the Status of Women. (1993). Employment. Genderwatch (5-31-1993). p. 1. Retrieved May 29, 2001 from the World Wide Web at: http://www.newsfirstsearch.oclc.org

Dillman, D. A. (1978). Mail and telephone surveys: The total design method. New York, New York: John Wile & Sons.

Doane Agricultural Services Company. (2002). County agents directory 2002-2003. St. Louis, MO.

Foster, B. (2001). Women in agricultural education: Who are you? Proceedings of the national agricultural education research meeting. December 12, 2001. New Orleans, LA. Available at: http://www.aaaeonline.org/

Maddy, D. J. (1992). Women who shattered the glass ceiling: Postpositivist inquiry into the aspirations, values, motives and actions of women serving as CEO's of Cooperative Extension Systems (Doctoral dissertation, The Ohio State University, 1992). Dissertation Abstracts International, M32.

Managing a diverse workforce in the Cooperative Extension Service: ECOP-PODC, recruitment, selection and retention. (1997, November). United States Department of Agriculture. Retrieved on July 31, 2001 from the World Wide Web: http://www.creesusda.gov/ecs/workforce.htm

Miller, L. E., & Smith, K. L. (1983). Handling non-response issues. Journal of Extension [On-line], 21(5). Available at: http://www.joe.org/joe/1983september/index.html

Seevers, B. S., & Foster, B. B. (2003). Women in agricultural and Extension education... A minority report. North American Association of Colleges and Teachers of Agriculture (NACTA) Journal. 47, (1) 32-37.

Women's Bureau, U.S. Department of Labor. (2001). Facts on working women. Retrieved September 30, 2001 from World Wide Web: http://www.dol.gov/dol/wb/public/wb_pubs/20fact01.htm

Agent Performance and Customer Satisfaction

Bryan D. Terry
Coordinator, Statistical Research
bterry@mail.ifas.ufl.edu

Glenn D. Israel
Professor
gdi@mail.ifas.ufl.edu

Department of Agricultural Education and Communication
University of Florida
Gainesville, Florida

Introduction

Given the importance of ensuring program relevance, quality, and impacts, as well as the use of customer satisfaction surveys in accountability, understanding the relationship that exists between employee performance and customer satisfaction is critical to identifying how well an organization is fulfilling its mission. Thus, Cooperative Extension must deliver relevant, high-quality programs that, in turn, help improve the lives of clients (Ladewig, 1999).

In Florida, these attributes (relevance, quality, and impact) are measured, in part, using a statewide customer satisfaction survey. The survey includes questions about clients' experience with quality of service, short-term outcomes, and overall satisfaction with Extension. The survey was initiated in 1988 in response to the Florida Board of Regents' recommendation that Florida Cooperative Extension survey their clients to assess the quality of services delivered to the citizens of Florida (Florida Board of Regents, 1988).

With the passage of the Government Performance and Accountability Act in 1994, Florida joined Oregon, Texas, and the federal government in requiring agencies to establish measurable performance objectives as part of the budget processes. Since 1997, the annual customer satisfaction survey has been used annually as part of the overall organizational evaluation system for the University of Florida. For Florida Cooperative Extension, the survey serves as the primary indicator of organizational performance. Specifically, the performance standard for Florida Cooperative Extension is that 98% of clientele will indicate that they are satisfied or very satisfied with the quality of service received.

In the study discussed here, we combined Florida customer satisfaction survey data with Extension personnel data to explore the relationship between customer satisfactions and agent performance.

Background

The causes and consequences of customer satisfaction have become the focus of recent research. Of special interest is the link between employee performance and customer satisfaction. Heskett, Jones, Loveman, Sasser, and Schlesinger (1994) establish a framework in which internal service quality drives employee satisfaction, which, in turn, drives employee performance that generates service quality. Finally, service quality drives customer satisfaction that leads to customer retention and profits. This framework was used successfully to improve organizational objectives at Sears Roebuck Co. (Rucci, Kirn, & Quinn, 1998). Similarly, Frederick Reichheld (2000) concluded that employee performance is essential to customer satisfaction, which, in turn, creates customer loyalty (Figure 1).

Figure 1.
Service-Profit Model

Among the factors that affect the quality of services delivered to clients are employee performance, experience, and the level of staffing. Employee performance is key to the success of most organizations and must therefore be evaluated. Measuring job performance is the process of determining how closely a record of behaviors and/or outcomes that occurred during a specified period matches the most nearly perfect record that could have been achieved during the period and then assigning it a corresponding number (Kane & Freeman, 1997).

In addition to employee behavior, other factors affect employee performance. Functional experience accords employees the opportunity to develop the skills and competencies specific to a discipline or program area (e.g., youth development or crop production), as well as the expertise in the methods of working in an area (Gelekanycz & Black, 2001). Number of employees has often been associated with employee performance and organizational outcomes (Anderson, Hsieh, & Su, 1998).

Purpose and Objectives

In the study, we explored the relationship between customer satisfaction and employee performance. Specifically, a logistic regression model was created to examine the effects that the determinants of service quality and employee performance indicators have on overall customer satisfaction. In keeping with the current research on customer loyalty, the study compared satisfied customers with very satisfied customers. It is the authors' belief that in the public sector it is very satisfied customers who will continually use services of Extension a manner similar to repeat private sector.

Methods

Data

The analysis is based upon data collected from Extension clientele from 1997 to 2000 using a customer satisfaction survey. A sample of Extension clientele from 47 of 67 Florida counties yielded 2,028 useable responses. Information from administrative records, including employee performance scores (ratings range from 1 to 7), employee experience (years of service), and the number of agents in a particular county, was linked with the client surveys based on the content of the information provided to clients. The data collected for the research represent 147 agents with an average of 14 survey responses per agent.

Survey Instrument

A questionnaire was originally developed using Bennett's (1982) Rapid Appraisal of Programs model and later revised to obtain service quality feedback from Extension clientele, type of clientele contact, and demographic information, including age, race, gender, educational attainment, and previous experience with Extension. This is consistent with Parasuraman, Zeithaml, and Berry's (1985) and Cronin and Taylor's (1992) work, which stressed the importance of collecting customer perceptions of service quality relating to reliability, responsiveness, competence, communication, and knowing the customer.

To ascertain clientele perceptions of service quality, the survey included five questions related to their experiences with Florida Cooperative Extension. These included:

1. Was the information accurate and up-to-date? (coded as "yes" or "no/don't know");

2. Was the information you received relevant to your situation? ("yes" or "no/don't know");

3. Did you have an opportunity to use the information? ("yes" or "no");

4. Did the information solve your problem or meet your need? ("yes" or "no/don't know"); and

5. Did you share the information with anyone else? ("yes" or "no").

A sixth question asked clients, "How do you rate the quality of the service you received?" to obtain an overall assessment of customer satisfaction. For the study, only the "satisfied" and "very satisfied" responses were used. The survey also included questions about respondents' age (in years), gender, race-ethnicity (coded as "white, non-Hispanic" or "non-white"), education ("high school or less," "some college," "college degree," or "graduate or professional degree"), and employment status ("employed," "unemployed," or "not in the labor force").

Survey Procedures

To generate a representative sample of Extension clientele, a procedure was to collect the names, addresses, phone numbers, and nature of the information provided (the procedures are detailed in Israel, 2000). For a 30-day period, sign-in sheets for visitors to the Extension office were established. Telephone logs collected client contacts by phone. Finally, agents presenting planned programs (e.g., demonstrations, field days, and workshops) collected client information prior to each program. At the end of the contact collection process, a sample of 60 clients was selected using a systematic random sample methodology for each county.

Approximately 1 month after the initial clientele contact, county faculty, support staff, and volunteers interviewed customers over the telephone. Responses were recorded, and completed surveys were mailed to Program Development and Evaluation Center (PDEC) for coding and analysis. The telephone survey produced an unadjusted response rate of 72%.

Analysis of the data included descriptive statistics, distribution of client responses by agent attribute, bivariate analysis, and logistic regression (a multi-variate technique to compare "satisfied" with "very satisfied" responses).

Findings

Tabulations for the five service quality determinants showed that respondents indicated that the information was up-to-date and accurate (94%), and relevant (93%), that they had the opportunity to use it (76%), that the information solved the problem for those using it (81%), and that they shared information with others (66%). Twenty percent of respondents indicted they were satisfied, and 80% were very satisfied with the service received. These results are similar to those reported for Extension clients in South Carolina and Texas (Radhakrishna, 2002).

In addition, the "typical" agent in the study had 13.5 years of experience and an evaluation score of 5.3 out of a possible score of 7. Also, a "typical" county office included a staff of approximately six agents.

Our initial analysis examined each agent attribute and overall customer satisfaction. The results indicate that a statistical relationship exists between customer satisfaction and both agent experience and evaluation score (Table 1). The distribution of clientele responses for agent experience indicated that the percentage that was very satisfied dropped substantially among agents having 20 or more years of experience. Agents with 5 to 19 years of experience had the highest percentage of very satisfied clients. With the exception of agents with an evaluation score of three, clientele were less likely to indicate that they were very satisfied compared to satisfied as an agent's evaluation score increased.

But agent evaluation score was not significant after other predictors were included in the logistic regression model. Regarding the number of agents within a county, there was no pattern in the data, meaning there is not much difference between a large professional staff and a small one.

Our findings for agent evaluation score were similar to those found by Rucci, Kirn, and Quinn (1998) and Davis and Verma (1993). These studies could not establish a direct relationship between employee performance and overall customer satisfaction. However, the findings for employee experience differ somewhat from those of other studies. Geletkanycz and Black (2000) found that agent experience accords employees the opportunity to develop skills and competencies specific to their discipline. While this is probably true during the initial period of employment, long-tenured agents showed markedly lower client satisfaction.

The relationship between the service quality determinants and overall customer satisfaction were examined similarly. The results in Table 2 indicate that all of the service quality determinants except whether clientele had the opportunity to use the information have a statistically significant relationship with overall satisfaction. When information is up-to-date and accurate, clientele are 21 percentage points more likely to indicate that they are very satisfied compared to satisfied. When information is relevant to a respondent's situation, results conclude that respondents are 34 percentage points more likely to indicate that they are very satisfied compared to only satisfied. When information solves a client's problem, they are 18 percentage points more likely to indicate that they are very satisfied versus satisfied. Finally, respondents who share information with others indicated that they were 16 percentage points more likely to be very satisfied.

In addition to service quality determinants and agent attributes, client attributes were included in the bivariate analysis and logistic regression to determine if there were any sub-groups of clientele who were less well served. Findings show that respondent education and age are statistically significant with overall satisfaction (Table 3). The distribution of responses shows that clientele who have obtained more formal education also are more likely to indicate that they are very satisfied compared to satisfied. Similarly, more of the older respondents were very satisfied than were younger ones. Distributions for gender and race did not indicate that any particular group was more or less satisfied. This is consistent with similar studies conducted by South Carolina Cooperative Extension (Nielson, 1999).

Discussion

The study discussed here focused on the relationships between employee performance and customer satisfaction in Florida Cooperative Extension. Agent attributes, service quality determinants, and clientele attributes were examined in order to understand their relationship with overall customer satisfaction. We found that customer satisfaction was not significantly influenced by agent performance (as measured by the annual evaluation score).

This finding contradicts conventional wisdom that Extension's top performers have the highest quality programs and, in turn, generate the greatest benefits for clients. This raises questions about whether the current employee evaluation system adequately measures aspects of agents' performance that are important to the mission of the organization. Given that the organization has established the importance of customer satisfaction as the performance measure for the Florida Legislature, we suggest that the annual performance assessment process use customer satisfaction as a major factor in assigning employee performance scores.

We also found that Florida Cooperative Extension benefits from the experience of its workforce (at least up to a point) and therefore should examine policies that increase employee satisfaction. This might include compensation, benefits, and work environment. In addition, hiring practices should be reviewed to emphasize relevant experience as criteria for employment in the organization.

We found that service quality determinants have a substantial effect on overall satisfaction. Though only one of the agent attributes was statistically significant in the logistic regression model, it is likely that these have an indirect influence on customer satisfaction via the service quality determinants (Figure 2). While we found that increasing experience had a positive effect for agents who were relatively new to Extension, long-time agents showed lower levels of customer satisfaction. Further study also is needed to identify reasons why this is the case so that professional development opportunities can be developed to address this area of concern.

Figure 2.
Service-Satisfaction Model for Extension

In addition to employee performance, service quality, as defined by the five determinants, was the most important determinant for overall customer satisfaction. This means that county agents must develop and maintain skills in assessing and responding to the needs of clientele, which can ensure that clientele receive the most current and accurate information. Additionally, it has become increasing important for agents to review planned programs for accuracy and timeliness, and to include evaluation components to determine if information received by clientele actually solved problems or met a need. Finally, it will become increasingly important to find delivery methods that can address needs within the time period expected by our clientele.

Our data showed that Extension clientele have a high degree of education, and, the higher their education level, the greater their likelihood of satisfaction. The challenge for agents will be to identify where program improvements can be made to attract and maintain a clientele that have less formal education. Further assessments are necessary to identify the needs of this group, and additional training for Extension agents is necessary to meet these needs.

Finally, age is another important factor in overall customer satisfaction. The results showed that older Extension clientele are also more satisfied compared to younger clientele, controlling for agent attributes and service quality determinants. It will be necessary to develop strategies for recruiting younger clientele, and this will entail further studies to better understand the dynamics of this market segment.

Acknowledgements

The authors wish to thank Larry Arrington and Howard Ladewig for constructive comments on an earlier version of this article. This article is a revision of one presented at the annual meeting of the Southern Rural Sociological Association, Orlando, Florida, in February, 2002.

References

Anderson, R. A., Hsieh, P., & Su, H. (1998). Resource allocation and resident outcomes in nursing homes: Comparisons between the best and worst. Research in Nursing, 21, (4), pp. 297-313.

Bennett, C. F. (1982). Reflective appraisal of programs (RAP): An approach to studying clientele-perceived results of Cooperative Extension programs. Ithaca, NY: Media Services at Cornell University.

Cronin, J. J. Jr., & Taylor, S. A. (1992). Measuring service quality: A reexamination and Extension. Journal of Marketing, 56, pp. 55-68.

Davis, W. L., & Verma, S. (1993). Performance Appraisal: How Extension Agents View the System. Journal of Extension [On-line], 31(4). Available at: http://www.joe.org/joe/1993winter/a3.html

Florida Board of Regents. (1988, February). The relationship between research and the Cooperative Extension Service. A report to the Florida Legislature.

Geletkanycz, M. A., & Black, S. S. (2001). Bound by the past? Experience-based effects on commitment to the strategic status quo. Journal of Management, 27, pp. 3-21.

Heskett, J. L., Jones, T. O., Loveman, G., Sasser, W. E. Jr., & Schlesinger, L. A. (1994). Putting the service-profit chain to work. Harvard Business Review, March-April.

Israel, G. D. (2000). Conducting a customer satisfaction survey. Florida Cooperative Extension Service, Factsheet AEC 356, University of Florida. Available at: http://edis.ifas.ufl.edu/WC039

Kane, J. S., & Freeman, K. A. (1997). A theory of equitable performance standards. Journal of Management, 23(1), pp. 37-58.

Ladewig, H. (1999). Accountability and the Cooperative Extension System. Unpublished Paper Presented At The Cooperative Extension Program Leadership Conference. Pittsburgh, PA: 1999.

Parasuraman, A., Zeithaml, V. A., & Berry, L.L. (1985). A conceptual model of service quality and its implications for future research. Journal of Retailing, 49, pp. 41-50.

Radhakrishna, R. (2002). Measuring and benchmarking customer satisfaction: implications for organizational and stakeholder accountability. Journal of Extension [On-line], 40(1). Available at: http://www.joe.org/joe/2002february/rb2.html

Reichheld, F. F. (2000). Loyalty-based management. Harvard Business Review, November-December.

Rucci, A. J., Kirn, S. P., & Quinn, R. T. (1998). The employee-customer-profit chain at Sears. Harvard Business Review, January-February.

Improving County-Based Science Programs: Bringing Out the Science Teacher in Your Volunteer Leaders

Martin H. Smith
Assistant Cooperative Extension Specialist
Veterinary Medicine Extension
University of California - Davis
Davis, California
mhsmith@ucdavis.edu

Cheryl L. Meehan
Post Graduate Researcher
Veterinary Medicine Extension
University of California - Davis
Davis, California
clmeehan@ucdavis.edu

Richard P. Enfield
Youth Development Advisor
University of California Cooperative Extension, San Luis Obispo County
San Luis Obispo, California
rpenfield@ucdavis.edu

Jeannette L. George
Youth Development Advisor
University of California Cooperative Extension, Tehama and Glenn Counties
Red Bluff, California
jlgeorge@ucdavis.edu

Jane Chin Young
Youth Development Advisor
University of California Cooperative Extension, Marin County
Novato, California
jchin@ucdavis.edu

Introduction

There is a recognized need for improvement in science literacy among children and youth (Hiraoka, 1998; National Center for Education Statistics, 2000; Zinsmeister, 1998), and community-based education programs like 4-H can play an important role in meeting this need. The learn-by-doing approach used in 4-H provides an excellent opportunity for children and youth to gain an understanding of science concepts (Williamson & Smoak, 1999). However, in order to optimize the impact of 4-H science projects on their youth audiences, volunteer leaders must be trained effectively.

In 2001/2002, the American Honda Foundation funded a research initiative directed at improving the abilities of adult volunteers as trainers and teen volunteers as leaders of a hands-on, inquiry-based science curriculum with 4-H youth. This project built upon the "Step-Up" Incremental Training Model (Smith & Enfield, 2002) by adding a Training of Trainers component for 4-H adult volunteers.

Project goals included:

• Increasing the competence and confidence of 4-H adult volunteers with respect to training teen leaders in a science curriculum;

• Increasing the competence and confidence of 4-H teen leaders in their roles as cross-age mentors; and

• Improving the science literacy of 4-H youth through the increase of their use of the Scientific Thinking Processes (Lowery, 1992).

Project collaborators included:

• The UC-Davis School of Veterinary Medicine,

• Veterinary Medicine Extension, and

• 4-H Youth Development Programs in Glenn, Marin, San Luis Obispo, and Tehama Counties in California.

Background

Adult volunteer leaders are at the core of county-based 4-H programs throughout the nation. In 2000, there were 495,152 adult volunteer leaders working directly or indirectly with 4-H teens and youth (USDA, 2001). Supported, and often led, by these volunteers, 4-H projects engage youth through a method of "learn-by-doing." Hence, it is critical that 4-H Youth Development Programs use training models for volunteer leaders that facilitate the transfer of specific program or curriculum content and methodology.

Effective volunteer training strengthens county-based programs (Snider, 1985; Hoover & Connor, 2001), and makes volunteers better educators and trainers (Hoover & Connor, 2001). Training that has a positive effect on volunteers' abilities increases their confidence while instilling a sense of ownership and responsibility with respect to the programs they lead (Snider, 1985). Furthermore, programs with highly trained volunteer leaders require less maintenance (Snider, 1985), which increases the potential for long-term sustainability.

Elementary school students are extremely interested in the world around them, and participation in inquiry-based science programs can catalyze this innate curiosity (Rillero, 1999). Thus, a rich and engaging elementary science experience is crucial for the intellectual development of all students.

However, a recent examination of elementary school classrooms revealed that instructional practices in science might not be meeting students' educational needs. For example, classroom teachers in grades K-5 report spending an average of only 25 minutes per day in science activities, which is less than one-quarter of the time spent on reading and literacy (Fulp, 2002). In addition, elementary school science is often taught using traditional methods such as whole class discussions and lecture-based instruction; only 41% of the teachers indicated that inquiry-based learning was heavily emphasized in their science lessons (Fulp, 2002).

The