Volume 36 Number 2
|
April 1998 Volume 36 Number 2 |
|
|
Breathes there an Extension Service that hasn't worried about its financial support and the future? Probably not. That's why Roy Frederick's commentary in this issue may strike a responsive cord with many. However, he makes some positive suggestions about how Extension can work with policymakers to demonstrate our worth and help assure continuing support. Other articles in this issue of your Journal are also related to support through accountability, reporting, and evaluation. North Carolina reports about a new reporting system, while articles from Ohio and others relate to evaluation and accountability. Extension relies heavily upon volunteers to meet its mission. How do we recognize those who help us be successful? In many ways, but some ways appear to be more meaningful then others. Find out what long-time Ohio 4-H leaders had to say about recognition. For those of you concerned with nutrition education, there's a new resource now available from the Agricultural Research Service. A two-disk CD-ROM set continuing three years' microdata from the 1994-96 Continuing Survey of Food Intakes by individuals and the 1994-1996 Diet and Health Knowledge Survey. The set costs $65 and is available from the National Technical Information Service at 705-605-6000 (accession number PB98-50047). You must like your Journal! The number of manuscripts has been on a constant increase. This means it has taken longer to complete the reviews and other steps necessary for an article to appear in "print." Please be patient. We are increasing the number of reviewers and hope to improve the process in the next few months. In the meantime, keep those manuscripts coming! Leonard Calvert, editor Extension Journal, Inc. Extension Journal, Inc. is a quasi-official body of the National Association of State Universities and Land-Grant Colleges and the Extension Committee on Organization and Policy (ECOP). It is a nonprofit corporation organized for the purpose of publishing a journal for professional Extension staff, adult educators, and community developers. Board of Directors: Michael Lambur, President, Virginia, member-at-large Ex-officio:
Leonard Calvert, Oregon, Editor Editorial Committee:
Joyce Alves, Arizona Commentary
Extension Education Opportunities with Policymakers
A.L. (Roy) Frederick Readers of recent issues of the Journal of Extension undoubtedly have noticed that our profession is debating several critical, overarching issues:
-- How can Cooperative Extension be more relevant and accountable? The last question is perhaps the most unsettling of all. It implies a great deal about knowing and responding to the needs of our customers. An affirmative answer to it would go a long way toward providing satisfactory responses to the other questions. Unfortunately, we may know less about our customers than we are willing to acknowledge (King, 1993).
Among the customers that need more attention are those that provide financial resources to Cooperative Extension: county commissioners, state legislators, and perhaps even our representatives in Congress. It's not enough to provide elected officials with an overview of programs offered to the general public. We need to offer programs that directly benefit them in their role as policymakers. Hahn consistently has identified "government" as a key Extension audience, especially in issues-oriented public policy education (1990). To his credit, Hahn does not shy away from advocating both technical assistance and process help for such audiences. The point is that election to office does not automatically make a person all-knowing about government functions. Nor should it imply that all elected officials are well grounded in leadership styles and group dynamics. As Extension faculty work with decision-makers in a public policy setting, our roles will be reshaped. We must be more than transfer agents of scientific findings and other useful information. Yes, Extension professionals can still provide valuable background information to public officials. But we must always acknowledge that personal values--of both policymakers and the public--play a big role in decision-making. Because values are so important, Extension should not attempt to prescribe solutions for public problems. If we help define issues, describe potential alternative solutions and their likely consequences, and assist in developing a process for receiving public input into the decision-making process, we will have done our job.
For more than three decades, Nebraskans have been concerned about high property taxes. Nebraska Cooperative Extension has responded by offering educational programming on this issue. Historically, much of the effort has been directed to helping rank-and-file voters understand a series of citizen petitions initiated to reduce dependence on property taxes. Until recently, few attempts were made to segment the Extension education "market" into various categories of users. In early 1996, a combination of circumstances caused reevaluation of property-tax programming. One group of citizens had begun circulating a petition that would have abolished property taxes altogether. Another group's petition proposed a 30 percent reduction. Members of local governing boards (for example, school board members and county commissioners) feared that basic services would need to be cut substantially, even if the more moderate proposal was approved by voters. Meanwhile, the state legislature found itself being pressured to negotiate a middle ground between tax-cut proponents and service providers. The Extension response was to begin work on several fronts. The Extension public policy specialist initiated a weekly newsletter on property tax issues. Though the newsletter was made available to the public through the news media, a primary audience was the Legislature itself. Recent tax law changes in other states were analyzed. Relevant articles in public finance journals were reviewed. The idea was for Extension to be a player in the evolving tax debate, albeit in a somewhat less formal way than is associated with legislative hearings. Numerous presentations were made to Extension and other audiences during the legislative session. In several cases, the Extension policy specialist either made these presentations jointly with a member of the Legislature or at the invitation of the member. (The weekly newsletter was perhaps a subtle, but consistent, reminder that Extension had something to offer on the issue.) One other group -- Extension faculty at the county level -- received special attention during this period. Because reduced property tax revenue could impact on local programs, Extension administration made a special effort to 1) protect funding and 2) keep faculty informed of legislative developments. Essentially, county faculty received two streams of information: One dealt with concerns of the public at large; the other, with Extension specifically. Ultimately, the Nebraska Legislature passed legislation in 1996 that, beginning in 1998, will restrict property tax rates that may be levied by local governments -- school districts, counties, cities and miscellaneous districts. On average, these reductions will be about 15 percent. Equally important, the Legislature offered the prospect of increased state aid to local governments if they first tried to become more efficient through reorganization. The Legislature's action opened another educational opportunity for Cooperative Education. It was based on the predicament of many local governing boards. Members felt overwhelmed by the need to plan simultaneously for reduced property tax revenue and new ways of doing business through reorganization. Cooperative Extension has responded by working directly with boards, especially county commissions. Beginning in late 1996, the Extension focus has been on analyzing county budgets, projecting new (1998 and beyond) funding realities, and suggesting alternatives for the post-1998 period. The initial request for Cooperative Extension assistance came from a county commissioner who had had previous positive experience with Extension programs relating to community leadership. Subsequent Extension education opportunities, based to some degree on success in the first county, have evolved in an additional 20 counties. Extension's role with county commissioners has been customized to the expressed needs of the county. Leadership for the program has come from three specialists and two local educators. Typically, Extension's contribution begins with an analysis of how much property tax revenue will be lost under the new scheme. Then, in what can be a sensitive undertaking, each major area of spending is examined. (Costs of Cooperative Extension are included in this analysis.) Mandatory and discretionary categories of expenditures are identified. Comparisons are made with counties of similar size. Alternatives for meeting revenue shortfalls are detailed, though Cooperative Extension does not make recommendations regarding a preferred alternative. The educational opportunity offered by Cooperative Extension frequently has been in the form of a widely publicized public meeting. County commissioners, school board members, and others participate, as does the general public. In other cases, Extension has conducted a separate work session specifically for commissioners. Though the latter sessions are officially "public meetings," typically few, if any, others have attended. Those who participate often express fears about losing local control of public services or being forced to consolidate with other local governments. General distrust of state and federal mandates is another frequent theme. Still, the need to reduce local property taxes keeps most participants looking to the future. In several counties, commissioners have responded favorably to a suggestion from Extension faculty that a county-wide advisory group be formed to help provide input on future budget and restructuring decisions. An Extension specialist typically assists with the organization and initial orientation of the group. Extension educators then serve as on-going coordinators for these groups, though Extension explicitly refrains from trying to influence the decisions of the group.
County commissioners appreciate Extension's assistance in addressing budget concerns. Both formal feedback and anecdotal evidence have been supportive. Factors that have contributed to satisfactory responses are (a) attention to local budget details in Extension workshops, that is, a recognition that no two counties have exactly the same circumstances with respect to property tax levies and spending needs; (b) an appreciation for local decision-making; (c) a recognition by Extension faculty that citizens in local communities, including county commissioners, are sometimes suspicious of state laws and regulations; (d) a sense that county Extension programs are being given the same level of introspection as other county-funded programs; and (e) Extension's assistance in organizing citizen advisory groups. Left unsaid, but implied in some favorable responses, is the reality that Extension has been willing to become involved in controversial issues at the local level. Commissioners have sensed the mood of their constituents and know difficult decisions must be made. Making those decisions is facilitated by fact-finding and community consensus-building, both of which are well within Extension's capability. The perception of Extension's effectiveness may be important in the future when it comes to making funding decisions. Indeed, one study from Minnesota indicated that it was the single most important factor when state legislators appropriated funds for Extension (Kabes, 1991). How better to enhance the perception of effectiveness than to provide quality programs directly to policymakers?
Hahn, A.J. (1990, Spring). Issues-Oriented Public Policy Education: A Framework for Integrating the Process. Journal of Extension, 28(1). Kabes, D.E. (1991, Winter). Legislators' Criteria for Extension Funding. Journal of Extension, 29(4). King, D. (1993, Fall). Facing the Image Deficit. Journal of Extension, 31(3).
Creation of a Web Based Accomplishment Reporting System
John G. Richardson
Kevin J. Gamble
R. David Mustian North Carolina Cooperative Extension Service
The demands for increased levels of accountability seem to be ever-increasing for practically all societal entities. With this expectation so prevalent, it is no surprise that policy makers are increasingly focusing close attention on the relevance of organizations and their value to their constituents, as well as to society as a whole. Such emphasis on accountability led to the passage of the Government Performance and Results Act (GPRA), which is now being implemented. Reports from across the country indicate that states, counties, and others appear to have similar ideas relative to making sure their tax dollars are being expended as intended. The North Carolina Cooperative Extension Service (NCCES) has not been isolated in this age of accountability and has been focusing considerable efforts to satisfy current and anticipated organizational and programmatic accountability needs and demands. One major component of this increased focus is the development of a completely new reporting system that attempts to capture all accomplishment results emanating from programming efforts in each county. Results include contacts, measures of program progress, impacts, volunteer time, cost-benefits, success stories, and delivery strategies for all planned programs. It also includes civil rights reporting as well as successes from other program efforts not included in regularly planned programs, such as special educational efforts dealing with disasters.
It is clear that a system for reporting all required aspects of accountability functions needs to be included in a single system. With this general concept in mind, state program leadership appointed a special program Reporting and Accountability Task Force to develop the criteria needed to address all future accountability needs of the organization. The task force was charged with identifying all reporting needs; developing goals and objectives for a new reporting system; identifying the parameters of a new system; and ultimately developing a diagrammatic model that could be studied, revised, and used as the design for the new system. The Reporting and Accountability Task Force held meetings with Extension personnel and conducted interviews with key county Extension directors. Those interviewed were asked to canvas their associates by various means to secure as much input as possible as to what was and was not needed for local accountability, preferences of time-lines for required reporting, and special wishes, such as user friendliness of a new system. Throughout the entire process, the committee received direct input from agents in more than 60 of the state's 100 counties. Altogether, more than one-half of all agents in the NCCES provided input into conceptualizing and designing the new reporting system. From the initial development of goals and objectives through the final roll-out of a World Wide Web based system, agents across the state were included in review and decision making on a continuous basis.
Building on the open dialogue established throughout the state, as well as with colleagues throughout the country, the task force developed a goal and a list of objectives for the new reporting system. The goal was: To establish an effective and efficient reporting system that is user-friendly, easily accessible, and provides needed organizational accountability requirements. With the goal as the guiding principal, objectives were then developed that included:
Throughout this conceptual process, Extension administration and program leadership were continuously involved to assure that everyone was on the same conceptual plane. After creation of the objectives, a list of parameters was developed. It included such specifics as what was required and when, time-lines, items needed for adequate accountability, and things that would be useful to have but not really seen as of vital importance for organizational accountability purposes. The next step was actual development of a diagrammatic model design. The design had to include all necessary components as well as those thought to be important for future reporting needs. Also, one objective was to design a system that was all- inclusive, that is, only one reporting system rather than several different ones, all functioning slightly different. As the model was designed, all interested persons were given the opportunity to review and make suggestions. The State Major Plan Task Force spent hours during its monthly meetings intensely analyzing each component and recommending changes. Actual time-lines for required reporting probably created the greatest discussion, and ultimately, the Extension Administrative Council would have the final say in what was or was not included, and when reporting would occur. Similar long discussions were common among the administrative group as well. Often, seemingly tiny adjustments would precipitate lengthy discussion, analysis, and intensity. Ultimately, a final diagrammatic model was accepted as the blueprint for the system. The next steps included development of input and output specifications to meet the needs of the Extension Technology Services information systems group for their guidance in the design of the required computer programs. In tandem with this on- going process, parameters were being established by the State Major Plan chairs to identify specific program measures for each plan objective. Parameters indicating program progress and results would become the major component of the entire reporting system.
With key sections and components of each section identified, it was clear that the major part of the system would be the measures of progress (MOPS) and Impacts associated with each of the objectives within each of the twenty State Major Plans. Since real program outcomes rather than inputs were now the focus, the guidance and training requirements for actually developing realistic MOPS and Impacts was a daunting task. Altogether, seventy-six objectives were given MOPS and Impacts in which reports would be completed. During this process, many individual SMP task forces recognized the significance of their expected outcomes, and many decided they had been perhaps too creative in developing a large number of measures. Ultimately, due to the press of software development requirements, final decisions had to be made with the MOPS and Impacts which would be used for reporting 1996 accomplishments. An example of MOPs and Impacts is shown in Exhibit 1 for one objective in one of the twenty state major plans.
Example of Program Measures of Progress and Impacts For One of Seventy-Six State Major Program Objectives State Major Plan AGING WITH GUSTO! OBJECTIVE 1. Participants in aging issues programs will increase awareness, gain knowledge, change attitudes, develop skills, and adopt practices and behaviors to help make their later years more financially secure. Measures of Progress:
Impacts:
Other components of the SMP reports include volunteers, volunteer hours and calculated value, and cost-benefit analyses, plus a narrative description of the program progress and results. Calculation of volunteer value is automatically set at a rate of $10 per hour. The literature gives wide ranges for valuing volunteer time, so a value was selected that would be reasonably conservative, yet high enough to reflect reasonable value of one's time. The cost-benefit analyses may be most difficult to make in some circumstances and relatively easy in others. Considerable discussion was focused on whether to include a requirement for the cost-benefit information. The final decision was to include such information. A fact sheet was developed to assist agents in understanding cost-benefits. While an assessment of this component may be difficult following initial use by agents, it is clear that a new paradigm is emerging in which agents are making assessments of the value of their time and its most worthy allocation for greatest impact. Separate sections were included for success stories for planned SMP programs, and those of a non SMP focus. Often, valuable work is performed that has not been included in a plan of work. An example was the experiences in 1996 with two major hurricanes, and the need to provide all types of disaster relief information and assistance on short notice. With this new reporting system, the successes of these emergency educational efforts were reported and resulted in the Secretary's Honor Award for Emergency Response being presented to the North Carolina Cooperative Extension Service by USDA. To provide guidance for writing success stories, a special training fact sheet has been developed. Success stories are limited to no more than 150 words, and will be truncated by the computer if they contain more than the prescribed word limit. The civil rights section includes all components of previously needed information. It has only been adjusted to fit the new system, with time-lines being changed to twice a year rather than reports being required once a year. The final section is one that allows those reporting to indicate their program delivery strategies. This section is optional, but will be accumulated at the state level so that use trends can be observed. N. C. State University is currently developing a university outreach reporting system, and reporting activities is expected to be a key part of that program. Therefore, while NCCES is more interested in MOPS and Impacts of programs, the types and numbers of program delivery activities remain important to a large number of people. As a result, this optional component of the NCCES system may later become a required entity as well.
Following the design and specifications phases of the system development, emphasis was placed on developing a computer system that could accommodate all of the intricate components of the reporting system. Initial plans rested with development of a text-based system linked to all county units. Fortunately, an innovative idea for using a graphically- based user interface (GUI) in conjunction with a client-server computing model began to emerge. This idea was rendered possible by a major statewide investment in the NCCES information technology infrastructure. These improvements provided for continuous Internet connectivity to all of the NCCES county centers and allowed for the development of a modern software solution for the new reporting system. The skills required for developing a graphics based system required development of new skills by members of the Extension Technology Services group. This challenge was met, and a new World Wide Web graphics based system was completed. All components of the new system were developed, tested, and released within an eight month period during 1996. The decision to use the World Wide Web as the delivery mechanism for the new reporting system contributed greatly to the speed with which the new system could be deployed. Most states have moved to make the World Wide Web available to all of their county units. The system developed in North Carolina would be easy to adapt and modify for use by other states. Testing of each computer program component involved initial release of an Alpha version of the program to six volunteer counties. Comments were received from the testing counties, and these were compiled into a punch list for the program development team to analyze and make adjustments in the system as indicated. Following this step, the second, or Beta version, was released to the same six counties for testing and review. Only then was the system released to all reporting units. This process is time consuming, but proved to be extremely valuable in that the systems released were essentially bug-free. Training for personnel in all 100 counties was conducted to introduce them to the new system and to practice with Section A. Universal comments of user friendliness have been received. Also, many agents are now finding the task of completing their reports so easy that they are entering their own information rather than giving it to secretaries for entering. Naturally, this is an evolving process and some will continue to depend on others to make their entries. User friendliness of the system was further indicated when 97 of 102 reporting units met the accomplishment reporting deadline date, which was only one month after final release of the system.
The resources devoted to development of this new NCCES reporting system have been enormous. Yet, in order to meet the organizational accountability needs that are required and expected now and in the future, we believe the NCCES has developed a system that will accomplish the objectives that were initially developed. Thanks to rapidly emerging computer technologies, a system has been implemented that could have only been imagined just a short time ago. With the similarity in information technology infrastructure deployed throughout the Cooperative Extension System, much of the work accomplished in North Carolina could be readily modified and used in other states. Based on initial feedback, NCCES has embarked on a new system that will provide the needed focus that should be placed on achieving actual program accomplishments. It is, however, clear that the knowledge curve is quite broad as to what measures of progress really are and what really entails program impacts. While obvious refinement will need to be made in the quantity and quality of indicated program results, the steps being taken have led far along the path of improved focus on programs and their accomplishments. Quality training coupled with increased levels of knowledge and awareness of program impacts should provide a solid basis for helping the NCCES to meet all accountability needs in the future. There was much learned in constructing the NCCES system that other states could benefit from as they develop or refine their own systems.
Establishing Effective Mentoring Relationships
| |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Table 1 Water Quality Information Exchange |
||
|---|---|---|
| INFORMATION RECIPIENT |
INFORMATION TYPE | INFORMATION SOURCE |
| Farmers (60%) | Progress on HMR (5%) Awareness of BMPs (30%) Knowledge of BMPs (20%) WQ research info (5%) |
Agents/Scientists Agents Agents Scientists |
| Scientists (20%) | Farm survey data (20%) | Farmers |
| Agents/ (20%) Technicians |
Educational materials (10%) Info on HMR (10%) |
Scientists Scientists |
As shown in Table 1, farmers need more types of information than other project participants do. For the farmer to make informed decisions, he or she must be able to integrate all levels of information into crop and animal production decisions. To assist in this process, farm plans have been developed by the USDA - Natural Resources Conservation Service (USDA-NRCS) and the farmer. A waste management plan also is required for some animal production units, but changes in the farm operation may require changes to the plan.
Information regarding nutrient and pesticide management is critical to linking the effects of BMPs to water quality. Without field level information, it is difficult to correlate implementation of BMPs with monitoring data. Field level information from farmers is also important, as models are used to simulate current conditions or estimate the effects of future BMPs on water quality. In addition, field level information is used to refine BMPs and determine which BMPs work best under a given set of circumstances. If scientists prepare educational materials and information on the project with agents and technicians, it can be passed on to farmers.
Many farmers were reluctant to participate in the farm surveys and were often reluctant to schedule an appointment. Contract livestock growers often felt that animal production information should be confidential and they did not want to share the information with a government agency. The reason for non- participation of some farmers has not been determined. Fortunately, many of the farmers did participate each year of the five-year survey. Out of a total of 330 agricultural fields in the watershed, including 14 farms with animal operations, water quality technicians were able to collect survey data from approximately 40% of the fields in 1990, 17% in the years 1991- 1993, 31% in 1994, and 29% in 1995. The increase in data collection in 1994 was due to the efforts of a newly hired local technician with training in animal science. The USDA-NRCS shared BMP implementation data for the project and nutrient management plan data for 125 fields. In addition, the USDA-Food Security Administration provided information on annual cropping patterns.
Participation in the farm survey may have been improved by increasing the educational outreach to explain the survey. Farmers may have responded more positively if their participation was publicly recognized or if they were compensated. The need for the farm survey was not fully endorsed by all extension agents in the area. The survey was viewed as the responsibility of the technicians. Crop and livestock agents could have had a greater role in survey data collection had there been time and interest. The USDA-NRCS and Soil and Water Conservation District technicians often met with project area farmers and collected similar data to develop farm plans, yet these technicians often did not participate directly in farm survey data collection. The farm survey was a new idea and the direct benefits to the watershed residents or to local agricultural agencies was not clear.
The reliability of the information received from the farmers varied considerably. While some farmers kept good field records, others were unsure about the crop, tillage, and nutrient or pesticide applications on a particular field on their farm.
Technicians found the farm survey work challenging. However, their work was more satisfying when they were providing water quality education in conjunction with completing the survey.
From the level of farmer participation in the past five years, it is clear that consistently achieving a high level of in -person farm survey participation (for example, > 50%) is difficult. Information on crops and BMPs is needed for each field in the watershed; therefore, an innovative strategy for data collection is advocated. Producers were informed about the need of the survey through a farm breakfast meeting and through an informational brochure distributed to the farmers; however, there was no incentive or reward for participation. It was explained to the farmers that only tract and field number, not names, would be associated with the raw data. In addition, they were told that the data was for university usage only and that, to date, no information has ever been released. The data was used for pollutant transport modeling of the effects of BMPs and to evaluate the impact of BMPs on surface and ground water quality.
The following recommendations should improve future farm survey data collection: (a) the survey must be brief, (b) the survey should be used to request essential information, (c) survey results from other agencies should be used wherever possible, and (d) basic information should be collected from farms without requiring information from the farmer (e.g. crop type). If possible, simple field observations, called windshield surveys, should be used to gather information that does not require contact with the farmer. In a windshield survey, technicians note crop type for each field on a map as they travel in the watershed to complete summer and winter crop surveys.
Agency records and windshield surveys will not provide details on nutrient and pesticide application rates. Therefore, interviews with selected farmers are needed for more detailed field level information. While a complete survey of the watershed is needed, selective interviews may serve in obtaining a sufficient sample of information. Targeting survey efforts to representative farmsteads or study areas is recommended to increase data acquisition success and reliability if assumptions of data transferability are correct.
Observations and recommendations for field level information based on the HMR Project, USDA Water Quality Demonstration Project, are presented. Farmers need training and education on BMPs to protect and enhance water quality. They need to understand that the knowledge they have about their farm is essential for advancement of water quality research. Farmers should be encouraged to document BMPs on selected agricultural fields and the information should be readily available to researchers. Researchers need field-specific information from farms to evaluate BMPs. This process is iterative and cyclical, with both farmers and researchers relying on information exchange to facilitate water quality improvements. When researchers are privy to site-specific information about individual fields, they can learn more about the effects of BMPs on water quality and, in turn, educate farmers to use appropriate BMPs. Likewise, when farmers are knowledgeable about BMPs and their positive effect on water quality, they are motivated to implement recommended BMPs.
Babbie, E. (1983). The practice of social research (3rd ed.). Belmont, CA: Wadsworth Publishing.
Coffey, S.W., Stone, K.C., & Line, D.E. (1993). Validation of EPIC for land applied waste. Paper resented at the International Winter Meeting of American Society of Agricultural Engineers. Chicago, IL.
Constance, D.H., Rikoon, J.S., & Ma, J.C. (1996). Landlord involvement in environmental decision-making on rented Missouri cropland: pesticide use and water quality issues.
Czaja, R. & Blair, J. (1996). Designing surveys: a guide to decisions and procedures. Thousand Oaks, CA, London, New Delhi: Pine Forge Press.
Fowler, F.J. (1988). Survey research methods. (Rev.ed.). Newbury Park, London, New Delhi: Sage Publications.
Jacobson, B.M., Jennings, G.D., Feng, J., & Stone, K.C. (1995). Watershed scale non-point source model evaluation, In Heatwole (Ed.) Water quality modeling, proceedings of the International Symposium 186-191 (ASAE Pub. 05-95). St. Joseph, MI. American Society of Agricultural Engineers.
Stone, K.C., Hunt, P. G., Coffey, S. W., & Mathey, T. A. (1995). Water quality status of a USDA water quality demonstration project in the Eastern Coastal Plain. Journal of Soil and Water Conservation, 50,657-571.
Elizabeth E. Gorham
Extension Family Resource Management Specialist
Utah State University
Logan, Utah
Internet address: lizg@ext.usu.edu
Sharon A. DeVaney
Assistant Professor
Purdue University
West Lafayette, Indiana
Internet address: sdevaney@purdue.edu
Janet C. Bechman
Extension Family Resource Management Specialist
Purdue University
West Lafayette, Indiana
Internet address: jcb@purdue.edu
Financial educators promote the use of financial management practices by families and individuals to improve their financial well-being. Simply increasing knowledge of financial management principles does not insure that a person will be a more effective financial manager. Knowledge must be applied. The purpose of this study was to assess the effectiveness of the Women's Financial Information Program (WFIP) in promoting the adoption of financial management practices by participants in three states.
A few studies (DeVaney, Gorham, Bechman & Haldeman, 1995, 1996; Jeries & Allen, 1991; Titus, Fanslow & Hira, 1989; Varcoe & Wright, 1989) have shown the positive effects of using selected financial management practices. Effective financial management as defined by Schnittgrund and Baker (1983) combines financial management practices and outcome results such as the type of budget used, the frequency of saving, and the frequency of financial management problems in the family. Research shows that consumers believe financial management practices like budgeting and saving are valuable (Davis, 1987; Godwin & Carroll, 1986; Mullis & Schnittgrund, 1982; Schnittgrund & Baker, 1983). But, most studies relating to financial management practices identify the audience using recommended financial management practices rather than the results of using the practice.
Characteristics of those who adopt recommended management practices have been the topic of previous research. Beutler & Mason (1987) studied factors associated with using formal budget planning. They found that young, married, and well-educated households with high demand on available resources were more likely to adopt the practice of written budgets. Level of income did not significantly effect the practice of budgeting, according to Beutler and Mason. More families who budgeted their money, compared to families who did not budget, believed that they could increase their satisfaction with financial management by planning expenditures (Mullis & Schnittgrund, 1982)
Rosenfeld and Neese-Todd (1993) showed that most aspects of the quality of satisfaction with financial status are related to the individual's perception of their control over finances. Women, more often than men, view themselves as powerless and lacking essential resources to be able to make changes in their lives (Burman, 1994). Cox (1991) states that support groups are a viable means of empowering women who are marginalized mainly because of illiteracy. Support groups provide a social experience for sharing personal stories as a means of understanding their experiences, finding their own voices, and raising their self- esteem. Group process is a powerful means of imparting knowledge, assisting decision-making and affecting change (Yalom, 1985; Cox, 1991).
Even though financial management practices have been proven to increase net worth and satisfaction with financial resources, there is evidence of resistance and failure of consumers to adopt such practices (Beutler & Mason, 1987; Davis, 1988; Godwin & Carroll, 1986; Schnittgrund & Baker, 1983). Davis (1988, p. 47) suggests that "even affluent households do not see the balance sheet as a useful financial tool". Although audiences indicate a high interest in a topic, few take action on their beliefs (Iams, Steinfelt, & Wilhelm, 1986; McKenna & Nichols, 1988).
Davis (1987) found that lack of time and knowledge were the two reasons most often given for not using recommended practices of budgeting, record keeping, comparing records to the budget, and preparing a balance sheet. The need for budgeting financial resources and wise use of credit are most often felt by those with low personal incomes or who are in debt. To encourage adoption of financial management practices, Walker, Tremblay and Parkhurst (1984, p. 429) recommend that educational programming be "inexpensive, uncomplicated, and readily accessible". The purpose of this study was to determine those characteristics of financial management education that promote adoption of recommended financial practices.
The study model was based on systems theory where inputs in the form of resources, attitudes, and goals are transformed by one or more continual processes into outputs of achieved goals and met demands (Deacon & Firebaugh, 1988). Inputs in the study include age, marital status, education level, employment status, income, satisfaction with personal financial situation, and financial practices completed prior to program enrollment. Throughputs consist of the individual's perception of competency in managing finances, use of program workbooks, and influence of other participants in the group setting. Output is the adoption of recommended financial management practices. Examples of recommended financial practices include record keeping, goal setting, spending plans, funds for emergencies, wise use of credit, regular savings, insurance, retirement plans, and investments.
WFIP was developed by AARP and implemented by Cooperative Extension Services throughout the United States since 1987 (American Association of Retired Persons, 1992). WFIP encourages participants to take personal responsibility for financial management decisions through a series of lessons featuring speakers, workbooks, and small group discussions. At each session of the seven-week series, participants listen to invited professionals speak on topics suggested in the program materials. Then participants meet in small groups for discussion of workbook lessons and the speaker's comments. If additional information is needed, participants are encouraged to contact financial professionals to seek answers and to share findings at the next meeting. This study differs from the format suggested by AARP in the use of both a pre- and a post-assessment instead of using only a post-assessment.
Participants were self-selected and recruited through brochures, newspaper articles, advertisements, and referrals from former participants. Data were collected from participants by Extension educators in three states (Indiana, Nevada, and Utah) between Fall, 1993 and Spring, 1996. A pre-assessment was administered during the first session. Data for the post- assessment were collected by a mailed instrument three to six months after the last session using a modified form of the procedure developed by Dillman (1978). Only those who responded to both assessments were included in the sample. The instruments were adapted slightly from previous studies and were pre-tested with a small sample of WFIP program participants prior to their use with the larger sample.
The pre-assessment used eight questions to assess satisfaction with personal financial management. A Likert scale ranging from 1 to 6, Very Dissatisfied to Very Satisfied, was used to indicate satisfaction with income, money for necessities, money for emergencies, current savings, amount owed, money available for future needs, the way money is handled, and who handles the money. Other questions requested information about gender, age, marital status, ethnicity, education, employment status, and income.
The post-assessment included two questions about completion of workbook exercises during and after participation scored on a Likert scale from 1 to 5, None to All of Them. Ten statements, using a Likert scale from 1 to 4, Strongly Disagree to Strongly Agree, measured perceptions of financial competency defined as level of confidence to manage and achieve goals, anxiety about their situation, comfort about spending, ease of decision making, ability to set priorities and appropriate goals, seek assistance, solve problems, and to positively affect their financial position. Higher scores indicated increased confidence in handling financial matters since beginning WFIP. The influence of the small group was scored from 1 to 4, Not At All to Strong and Significant Influence.
The dependent variable, number of financial practices completed during or after WFIP, could range from zero to 27. Practices included setting up a record keeping system, developing or revising a spending plan, assessing the adequacy of insurance coverage, and review or establishment of an investment plan.
Data were analyzed using SPSS (Statistical Package for Social Sciences). Frequencies and means were obtained to describe the sample. Multiple regression was used to identify the determinants of financial management practices adoption. Tests for multicollinearity yielded no highly correlated variables.
The response rate was 54.4% based on 1,544 pre-assessments and 840 post-assessments. There were no significant differences between those who completed only the pre-assessments and those who completed both the pre-and post-assessments. Of the 576 usable responses (able to be matched), 38.2% were from Indiana, 37.3% from Nevada, and 24.5% from Utah. The majority (96%) were white females. Ages ranged from 20 to over 75, with almost half (48.9%) between 45 and 64 years. Almost two-thirds (63%) were married. Seventy percent had more than a high school education. Over half (56.2%) were employed full- or part-time. The majority (75.1%) had personal incomes between $10,000 and $49,999. The average number of financial practices that were adopted was 11.62.
The results of multiple regression analysis showed that age, financial practices completed prior to WFIP, workbook completion during and after WFIP, and perception of personal financial competency significantly predicted the number of financial practices adopted. These factors explained 49.6% of the variance in the number of financial practices adopted (F = 20.551; d. f. = 11, 230; p = .001). See Table 1.
| Table 1 Multiple Regression Analysis on Number of Financial Practices Adopted |
||
|---|---|---|
| Variable | Beta | P-value |
| Age | .241 | .001** |
| Married | .053 | .291 |
| Education | .046 | .363 |
| Employed | .059 | .267 |
| Personal income | .043 | .418 |
| Satisfaction with management | .007 | .902 |
| No. of practices before WFIP | -.561 | .001** |
| Influence of small group | .022 | .666 |
| Workbook during WFIP | .151 | .004* |
| Workbook after WFIP | .228 | .001** |
| Perception of competency | .186 | .001** |
| F value = 20.551 R-square = .496 * p < .01, ** p < .001 |
||
The older the participant, the greater the number of financial management practices adopted. The discretionary time, urgency, and need for adopting financial management practices are likely to be greater as one ages. Increased adoption of recommended practices was significantly affected by completion of workbook exercises during and after participation. Program coordinators should strongly encourage the use of the workbook. As perception of personal financial competency increased, the number of financial practices adopted increased. Understanding the importance of perceptions is an important contribution of this study.
Those who completed more actions prior to WFIP adopted fewer recommended financial management practices during and after WFIP. This seems reasonable. Financial management is a continual processing of information as circumstances change (Deacon & Firebaugh, 1988). The study, however, did not distinguish between an action ever completed and subsequent processing (updating) needed. In future studies, data could be collected on when the action was last completed, level of satisfaction with task, and conditions to be satisfied (that is, more knowledge, time, income) before task can be done to their satisfaction.
WFIP was designed to improve the financial well-being of individuals and families. Participants receive challenges to adopt successful financial management practices from professional speakers, use the workbook, and actively participate as a member of a small group. It is important for financial management educators to understand the conditions supportive of adoption of financial management practices. Then educators can target programs effectively, obtain the most effective speakers, and encourage participants to use the workbooks during and after the series of lessons.
Continued research is needed to determine additional factors affecting program effectiveness. It may be useful to compare the number of recommended practices adopted in a larger sample to see if findings are consistent among participants.
American Association of Retired Persons (1992). Fact sheet. Washington, D.C.
Beutler, I. F., and Mason, J. W. (1987). Family cash-flow budgeting. Home Economics Journal, 16(1), 3-12.
Burman, S. (1994). The disease concept of alcoholism: It's impact on women's treatment. Journal of Substance Abuse Treatment, 11(2), 121-126.
Cox, E. O. (1991). The critical role of social action in empowerment oriented groups. Social Work with Groups, 14(3-4), 77 -90.
Davis, E. P. (1987). Patterns of family financial management. Papers of the Western Region Home Management Family Economics Educators, 2, 108-112.
Davis, E. P. (1988). Who prepares a balance sheet? An exploratory analysis. Papers of the Western Region Home Management Family Economics Educators, 3, 47-49.
Deacon, R. E., and Firebaugh, F. M. (1988). Family resource management. Boston, MA: Allyn & Bacon.
DeVaney, S. A., Gorham, E. E., Bechman, J. C., & Haldeman, V. (1995). Saving and investing for retirement. Family Economics and Resource Management Biennial, 1, 153-158.
DeVaney, S. A., Gorham, E. E., Bechman, J. C., & Haldeman, V. (1996). Cash flow management and credit use. Financial Counseling and Planning, 7, 71-79.
Dillman, D. A. (1978). Mail and telephone surveys: The total design method. New York, NY: John Wiley and Sons.
Godwin, D. D., & Carroll, D. D. (1986). Financial management attitudes and behaviour of husbands and wives. Journal of Consumer Studies and Home Economics, 10, 77-96.
Iams, D. R., Steinfelt, V., and Wilhelm, M. S. (1986). Needs assessment for financial management programs in extension. Journal of Home Economics, (Fall), 48-50.
Jeries, N. and Allen, C. M. (1991). Wives' satisfaction with financial management of their families. Papers of the Western Region Home Management Family Economics Educators, 6, 47- 53.
McKenna, J. S., and Nichols, S. Y. (1988). Planning for retirement security: What helps or hinders women in the middle years? Home Economics Research Journal, 17(2), 153-164.
Mullis, R, J., and Schnittgrund, K. P. (1982). Budget behavior: Variance over the life cycle of low income families. Journal of Consumer Studies and Home Economics, 6, 113-120.
Rosenfield, S., and Neese-Todd, S. (1993). Elements of a psycho-social clubhouse program associated with a satisfying quality of life. Hospital and Community Psychiatry, 44(1), 76- 78.
Schnittgrund, K. P., & Baker, G. (1983). Financial management of low-income urban families. Journal of Consumer Studies and Home Economics, 7, 261-270.
Titus, P. M., Fanslow, A. M., & Hira, T. K. (1989). Net worth and financial satisfaction as a function of household managers' competencies. Home Economics Research Journal, 17, 309 -318.
Varcoe, K. P., and Wright, J. (1989). Resource management education: Measuring impact in different settings, Papers of the Western Region Home Management Family Economics Educators, 4, 27- 33.
Walker, E. S., Tremblay, K. R., Jr., and Parkhurst, A. M. (1984). Financial management and family life. Family strengths 5: Continuity and diversity. Newton, MA. Educational Development Center.
Yalom, I. D. (1985). The theory and practice of group psychotherapy. New York, NY: Harper Collins.
Cathy L. Butler
Assistant Home Economist (4-H)
Louisiana Cooperative Extension Service
Ouachita Parish
West Monroe, Louisiana
Internet address: Ouachita@agctr.lsu.edu
Janette B. Bowman
Associate Professor
Institute of Gerontology
Northeast Louisiana University
Monroe, Louisiana
Internet address: igbowman@alpha.nlu.edu
The drop in 4-H membership among junior high and high school -age students concerns parents, leaders, and 4-H agents. Studies show that teenagers who participate in 4-H club activities are more likely to develop life skills in leadership, personal development, and citizenship (Cantrell, Heinsohn, & Doebler, 1989; Meszaros 1993). However, many youth become inactive during their teenage years when leadership skills are most often developed (Acosta & Holt, 1991).
Hartley (1983) suggested designing activities to address the needs and expectations of 4-H members as a means of keeping members. Others have promoted improving 4-H programs by assessing the needs of youth. The assessments are important tools in identifying potential 4-H members and in improving 4-H programs (Jones & Smith, 1988; Myer, Crites, & Haldeman, 1996).
Lerner (1995) found that successful youth programs provide program participants with intensive and individualized attention. He reported that successful programs stress diversity and respond to the individual needs of at-risk youth.
The importance of creating equal partnerships between youth and program educators was identified by the Cooperative Extension System's Strategic Plan (1994), which emphasized the value of including youth in defining, developing, and implementing 4-H programs. The 4-H staff in Ouachita Parish has used the following strategies to recruit teenage youth into 4-H programs: (a) conducted advisory committees that included 4-H teenagers;(b) received input from Junior Leaders;(c) Conducted informal focus group meetings with community leaders; and (d) addressed special program requests from the community. However, staff had not formally surveyed 4-H members and non-members to determine program needs. Recognizing that this assessment would benefit the 4-H program, a needs assessment survey was administered. This article highlights findings from the study.
The survey questionnaire was adapted from the "4-H Senior Needs Assessment" questionnaire developed by Acosta and Holt (1991). Two hundred surveys were hand-delivered to 4-H leaders at four junior high and high schools in Ouachita Parish. The schools were selected because each had an organized 4-H club. The surveys were administered by 4-H leaders during mathematics, history, and physical education classes and in 4-H club meetings.
Each 4-H leader selected the classes to be included in the sample. One hundred fourteen students who returned a parental consent form volunteered to participate in the study. The participants were in grades 6 through 12 and ranged in age from 11 to 18 years. The sample included 47 4-H members and 67 non- members.
Thirty-four program topics were included on the questionnaire. Students were asked to rank their top seven interests from one to seven, with one being the most important. The teens also indicated topics that were not of interest to them by placing "Xs" in the blanks. Even though the youth ranked seven programs that they were interested in, we concentrated on the top three programs identified by each youth. The bottom four program preferences were treated as an "X" or not important. Please refer to Table 1 for the top eight program choices identified with a ranking of one to three by 4-H members and non- members.
| Table 1 4-H Members Rankings of Topics of Highest Interest |
||
|---|---|---|
| Topics | Member n |
Percent % |
| Coping with unemployment | 26 | 23 |
| Fashion | 13 | 11 |
| Teen pregnancy | 12 | 10 |
| Suicide | 11 | 10 |
| Drugs | 11 | 10 |
| Dating | 10 | 09 |
| Computers | 10 | 09 |
| After high school | 10 | 09 |
| Non-Member Rankings of Topics of Highest Interest | ||
| Topics | Non-members n |
Percent % |
| Coping with unemployment | 29 | 25 |
| After high school | 18 | 16 |
| Fitness | 18 | 16 |
| Dating | 18 | 16 |
| Jobs | 15 | 13 |
| Fashion | 14 | 12 |
| Computers | 13 | 11 |
| Health | 11 | 10 |
| Note: n = 114 | ||
Coping with unemployment was ranked as the number one program preference by 4-H members and non-members. We were surprised by this. Until this survey, we did not think that this would have been a topic of interest, let alone the number one program preference of the youth. This program topic was not identified as a topic of interest in Acosta & Holt's 1991 study. Even though programs are administered through the Cooperative Extension for adults, we have not addressed this issue in our 4-H program. As a result of this study, this particular issue has been brought to the attention of our 4-H staff.
Fashion ranked second among the 4-H members. However it was not as high of a priority among the non-members. This finding was not a surprise. Our 4-H program has a well established fashion board, and this program area is one of the most traditional among our programs. However, since non-members have indicated that this is also a program interest, we have undertaken new ways of increasing awareness about our fashion program.
After high school was ranked as a high priority for non- members and was also of interest to the 4-H members. This finding was also similar to Acosta & Holt's 1991 study. According to Abell and Ludwig (1997), youth who develop a sense of identity and establish autonomy are more likely to make responsible and independent decisions. Therefore, we recognize the importance of providing opportunities for growth in this program area.
Non-members ranked fitness as a strong program interest, while this was not indicated by members. In addition the non- members indicated that they were also interested in health. Health was not indicated as a program priority among the 4-H members. We have several fitness and health programs that we administer each year including a "Family Fun Walk," for the community. However, we will use these findings in planning next year's activities to try to attract more non-members to these programs and events.
Teen pregnancy, suicide, and drugs were identified by members as program areas of interest. These topics were not identified by the non-members as important areas of interest. Currently we do not address teen pregnancy or suicide in our programs. But we do have several self-esteem programs that provide pro-active content. Teen pregnancy and suicide issues could be incorporated as facets into self-esteem programs. We will keep in mind Lerner's (1995) recommendation that these topics may require intensive and individual attention. In addition we will facilitate small discussion groups on these specific topics at our Junior Leader meetings. One of our 4-H high school clubs has been involved in developing a community service project for children on "Saying No to Drugs," but from this study we see the need for providing this information to a more diverse group.
It came as no surprise that dating and computers were of interest to both groups. Our 4-H program now has project books available that address each topic. However, the 4-H staff will be exploring new ways to market these programs.
The results from this study have been useful in improving the 4-H curriculum, in helping the advisory committee set priorities, and in program development. We realize that young people are more likely to actively participate in programs that meet their needs. This study provided us with some direction in meeting the needs of youth. We recognize that the needs assessment survey could be strengthened by involving teens in the development of the instrument. This study has enabled us to see the value of on-going needs assessment surveys and the importance of involving the teenagers in each phase of the process.
Abell, E., & Ludwig, K. B. (1997). Developmental considerations in designing parenting education for adolescent parents. Journal of Family and Consumer Sciences, 89 (2), 41-44.
Acosta, D. T., & Holt, B. A. (1991). Give teens the programs they want.and need. Journal of Extension, 29, 29-30.
Cantrell, J., Heinsohn, A. L., & Doebler, M. K. (1989). Is it worth the costs? Journal of Extension, 27, 16-18.
Cooperative Extension System. (1994). Focus on the future: A strategic plan for 4-H and youth development. October, United States Department of Agriculture, Washington, DC.
Hartley, R. S. (1983). Keeping 4-H members. Journal of Extension, 21, 19-24.
Jones, D. A., & Smith, W. C. (1988). CPR: Purposeful action. Journal of Extension, 26, 26-28.
Lerner, R. M. (1995). Features and principles of effective youth programs: Promoting positive youth development through the integrative vision of family and consumer sciences. Journal of Family and Consumer Sciences, 87 (4), 16-21.
Meszaros, P. S. (1993). The 21st century imperative: A collaborative ecological investment in youth. Journal of Home Economics, 85 (3), 11-20.
Myer, P. A., Crites, A. M., & Haldeman, V. A. (1996). Assessing the need for family economic well-being programs: A comparison of two samples. Journal of Family and Consumer Sciences, 88 (3), 27-31.
James R. Lindner
Research and Extension Associate
Ohio State University Piketon Research & Extension Center
Piketon, Ohio
Internet address: lindner.16@osu.edu
Ruben D. Nieto
Special Assignment, Urban Programs
Ohio State University Extension
Internet address: nieto.1@osu.edu
Quality improvement, in particular Total Quality Management (TQM), has had a major influence on U.S. management philosophy (Ross, 1993). While not a modern idea, quality improvement has gained interest in recent years. For most of the 20th century, productivity was the primary emphasis for most profit and non- profit organizations; the emphasis, however, is now changing to quality. The reason is simple; neither price, technology, nor quantity is sufficient to differentiate products or services from the competition. One thing that attracts and keeps customers is the "extra value" of quality--as defined by the customer (Cannie, 1991).
The College of Food, Agricultural, and Environmental Sciences at The Ohio State University has begun a Continuous Quality Improvement (CQI) process. One of the goals of the CQI project is to support and ensure quality teaching and excellence throughout the college, including Ohio State University Extension (OSU Extension). Extension program personnel at OSU use program evaluation as a tool for measuring the effectiveness of their teaching. Typically program evaluation consists of designing an instrument to gather data, gathering and analyzing the data, comparing the data with standards and benchmarks, and commending accomplishments and correcting shortcomings (Buford, Bedeian, & Lindner, 1995).
Benchmarking is establishing a point of reference from which comparison can be made. One type of benchmarking is to internally or administratively set standards. Another benchmarking process is to examine records from similar organizations and set competitive standards. Once a benchmark has been established, an organization can determine whether or not a program is meeting its objectives.
While several sources of information for performance evaluation exist within OSU Extension, clientele input is highly regarded as an important source of information. A series of valid and reliable evaluation instruments were designed by OSU Extension to gather evaluation data on effective teaching for Extension program personnel (Spiegel, 1992).
This set of materials is known as the Evaluation of Effective Extension Teaching (EEET). Group Form I measures teaching effectiveness. This evaluation instrument consists of nine statements with a five-point, Likert-type scale and a space for comments. The form uses the following scale: 1=Strongly Disagree, 2=Disagree, 3=Neither Disagree nor Agree, 4=Agree, and 5=Strongly Agree. The following statements are included in Group Form I: The Instructor ... (a) was well prepared, (b) was interested in helping me, (c) showed respect for all persons attending the workshop, (d) stimulated me in wanting to learn, (e) answered questions clearly, (f) related program content to real-life situations, (g) gave clear explanations, (h) held my attention, and i) presented information that will help me. Validity of the instrument was established by a panel of experts consisting of faculty members at The Ohio State University, Extension professionals, and Extension clientele. A reliability coefficient of .93 was determined for Group Form I (Spiegel, 1992).
By using Group Form I, Extension program personnel evaluations serve two purposes: (a) to provide information to the individual Extension teacher for the improvement of teaching and (b) to assist administrators in making decisions regarding promotion/tenure and annual performance appraisal. However, sources of variation in the evaluation data have not been examined. By using CQI techniques on Group Form I data, the sources of variation reflecting the quality of Extension programming can be determined and controlled.
Continuous quality improvement (CQI) and total quality management (TQM) have similar definitions and are used interchangeably in this paper. TQM is a process used to achieve "quality." Defining the TQM process is somewhat tenuous because of the various approaches developed by academicians and practitioners (Bedeian, 1993). Most processes developed to implement TQM include the following elements: (a) strategic planning, (b) leadership, (c) quality results, (d) information and analysis, (e) quality assurance, (f) human resource utilization, and (g) customer satisfaction (Joiner, 1992; Cohen & Brand, 1993; Schmidt & Finnigan, 1993; Bedeian, 1993).
While all elements in the TQM process are important and critical to successfully achieving quality, this paper will focus on the information and analysis element and, in particular, the use of statistical quality control charts to determine sources of variation affecting the quality of Extension programming.
Program evaluation in Extension is typically a "snapshot in time" of the effectiveness of a particular program or activity. Such information is useful not only to Extension professionals, but also to Extension administrators. But if TQM principles are to be applied, more information is needed. Statistical quality control is the application of statistical techniques for measuring and analyzing deviations in manufactured materials, parts, and products for the purpose of improving the quality of the process that created such deviations. The use of these statistical techniques was one of the key factors for Japan becoming the world leader in product quality (Buford, Bedeian, & Lindner, 1995). Statistical quality control techniques work in the service sector as well and they can work in Extension (Cohen & Brand, 1993; Bedeian, 1993).
An understanding of variation in evaluation scores is needed. Data collected on any activity, series of events, or situation will usually exhibit variation (Deming, 1960). In Extension, the primary goal is to educate clientele. One of many activities in this educational organization is providing useful programs to clientele. If ten workshops are conducted over a given year, undoubtedly some sort of program evaluation similar to the example above would be conducted.
If the average evaluation rating was a four or higher, Extension educators would have confidence that the educational program was conducted reasonably well. But since no educational program is conducted without variation, assuming that all clients provided a "good" evaluation is unrealistic. Tolerance or specification limits are set to allow variation above and below the average evaluation score. In fact, most of the evaluations will be slightly above or below the average score. Specification limits are upper and lower standards set by managers or clientele in response to clientele expectations. These specification limits represent quality standards that the organization and clientele consider acceptable.
OSU Extension uses Group - Form I as one tool to assess the quality of its educational programs. For example, assume that Extension administrators expect educational programs to receive at least a mean evaluation rating of a four. Evaluation of educational programs should yield data within the specified acceptable limits. If not, educational resources will be wasted and some customer
