|
February 2003
|
       |
FeaturesIntegrating Development of Extension Materials and Formative Informal Evaluation: Land Application of Sewage Sludge as a Case ExampleUta Krogmann Virginia Gibson Rutgers University BackgroundIn New Jersey, 66% of the 252,926 metric dry tons per year (N.J., 1999) of sewage sludge (biosolids) are land applied (42% out-of state, 58% in-state) (T. Pilawski, personal communication, 1999). This follows an increasing trend to land apply sewage sludge as a primary disposal option in the U.S. Despite this, land application of sewage sludge remains controversial in the agricultural community of New Jersey, the most densely populated state in the U.S. There are a number of issues associated with land application in New Jersey:
Controversy over sewage sludge application can be due to numerous factors, including local management problems such as odors, vectors, stockpiling, and truck traffic. Other reasons include uncertainties and disagreements concerning the safety and appropriateness of sewage sludge land application among researchers, regulatory authorities, environmental groups, and the general public. Due to the controversial nature of this topic, the development of an Extension program and Extension materials to provide Rutgers Cooperative Extension (RCE) agents and farmers with information on this topic was stalled in its initial stages for several years in New Jersey. As a result, a new approach was taken to develop an RCE program and Extension materials in a manner intended to minimize dissent and meet the pressing needs of Extension agents to provide materials on the subject to farmers. This article discusses how simultaneous Extension material development and formative informal evaluation efforts can be applied to develop more responsive Extension programs for controversial issues, with land application of sewage sludge in New Jersey as a case example. It summarizes one informal approach to gaining feedback (formative evaluation) on process and materials as they are developed. The purpose in this specific case was to gain feedback from Extension agents, farmers, Rutgers faculty, and external experts (using methods of semi-structured interviews, meeting evaluations, peer- reviews and pre-testing). The feedback was used to guide and improve the start of an Extension program on the controversial topic of land application of sewage sludge and specifically the development of related Extension materials. This approach was designed to better understand stakeholder concerns and to meet the needs of Extension agents and farmers. ApproachIf formative informal evaluations are an integral part of the process, evaluation results can provide feedback to guide and improve the process of developing Extension educational materials for a new Extension program. The data that evaluations can provide are disparate, including objective data concerning Extension agents' and agricultural community's needs, reinforcement to improve decision making regarding allocation of resources, and data to buttress decision making about policies that affect farmers. In this case, the Extension materials included RCE's technical guidelines for land application of sewage sludge (RCE, 2001) and fact sheets for agricultural Extension agents and, eventually, farmers as target audiences (RCE, 2000). While RCE's guidelines provide technical information regarding land application, the fact sheets are for a lay audience, dealing with social, legal procedural, and technical issues. The process steps and the methods of gaining feedback to get a program started and to develop Extension materials on land application of sewage sludge are summarized in Table 1.
An informal evaluation approach was used because solving a problem rather than conducting rigorous qualitative research was the priority. Furthermore, funding was limited. A qualitative evaluation approach was used to guide and improve the process in iterative steps, which would have been more difficult to accomplish with less flexible, standardized quantitative evaluation methods. However, we emphasize that this is not rigorous social science research because our evaluation approach was limited not only by the sample size of the population (selected Extension agents in New Jersey) but also by the questions that were asked. Questions were focused on program development or on the clarity of materials that were developed. They did not test the targeted audience's understanding of the materials (e.g., questions about facts) or ask questions that dealt with their perceptions of risk. Also, this program did not employ quantitative methodologies. Information on more rigorous social science approaches to evaluation can be found in Patton (1997) or Morgan, Fischhoff, Bostrom, Lave, & Atman (1992). Results of This ProcessWorkshop Preparation and Telephone Interviews Five agricultural agents in field, fruit, and vegetable crop areas were interviewed using a semi-structured interview method. These interviews were not meant to be generalized to all agricultural Extension agents, but rather to develop a better in-depth understanding of agents' feelings and concerns. The interviewees were selected by so-called purposeful sampling of information rich cases (Patton, 2001). The interviewer chose the interviewees from a list of agricultural agents who were known to be involved with the topic and who specialized in different commodity areas. Because only five interviews were conducted, these results were used to develop a workshop agenda, but general conclusions could not be drawn. The interviewer was not part of RCE, was previously trained as a professional interviewer, and was not involved with this issue before. Despite differences in answers, a substantial overlap was found in these interviews (Table 2).
Previous program development efforts in this area at RCE had focused solely on technical issues. However, these initial interviews showed that social and legal issues were just as important. The interviews also showed that the interviewed agents had concerns regarding land application that they wanted to articulate and that they felt that technical guidelines were needed to enable agents to respond appropriately to farmers' inquiries. Using data results from the interviews, the following workshop agenda was developed:
Workshop and Meeting Evaluations Twenty-one agricultural agents and specialists volunteered to participate in this workshop. This was considered a large number of participants because it was the beginning of the growing season, the busiest time of the year for agents. Because all stakeholders were so anxious to resolve the issues involved, the workshop could not be scheduled at a more convenient time. At the beginning, all participants described their top three concerns regarding the use of sewage sludge, and a list of concerns was created. Extension agents' and specialists' concerns, in order of priority were:
During the discussion about the need for RCE's technical guidelines, a representative of the RCE administration emphasized how important it is that all information RCE provides is supported by sound scientific studies. During the discussion about "where to go from here," the suggestion to form an RCE Biosolids Working Group was made. The tasks of the group were 1) to develop fact sheets, including a fact sheet entitled "Questions to Ask Before Considering Application on Farmland" for farmers and Extension agents, and 2) to review and revise RCE's technical guidelines for land application of sewage sludge on agricultural land. Thirteen of the 21 participants filled out the evaluation forms after the meeting. Only 13 responded, because, due to the inconvenient time, some individuals left early. All of the evaluations indicated that the meeting was well organized, productive, and useful for their work. When asked about the usefulness of various agenda items, respondents considered the initial discussion about their concerns and the discussion with the Cornell Waste Management Institute most useful. When asked what topics they would like to have covered at future meetings, respondents mentioned:
In addition, several requested the chance to review RCE's technical guidelines as they were revised. When asked for additional comments, one respondent said, "We have waited long enough. Make this a high priority, and make a stand on what we want to say to growers and the public." The evaluation form included a list of potential fact sheet topics, which was created based on the telephone interviews. Two issues received top ranking, liability and technical guidelines for sewage sludge use. These issues were followed in importance by sewage sludge quality and variability, heavy metals in sewage sludge, regulations, and human health issues. Biosolids Working Group and Meeting Evaluations The Biosolids Working Group consisted of 11 members, mostly Extension agents and specialists. Areas of expertise included:
The diversity of the group ensured that various aspects of sewage sludge land application were covered. In order to include more Extension personnel in the discussion, members of the working group discussed relevant issues in their commodity groups. When necessary, other experts and commodity groups were contacted for factual advice. For example, an environmental law professor was contacted about legal issues. During the nine subsequent working group meetings, members presented information about certain topics, which were identified at each previous meeting, and RCE's technical guidelines and fact sheets were reviewed. After RCE's technical guidelines were revised and the fact sheets were drafted based on working group discussions, the outreach materials were discussed with the RCE administration, which would be ultimately responsible for the release of the developed materials. A lot of time was spent preparing for each meeting to ensure they would be productive and to reduce the number of meetings necessary to accomplish the task. All meetings were anonymously evaluated. The responses indicated that the meetings were considered very productive, well organized, and useful for the audience. Therefore, not many improvements from meeting to meeting were needed. There were only a few comments that needed to be addressed, such as "Keep closer control over the agenda items. Although the discussion was good, we moved around a lot." Peer-Review and Pre-Testing Based on feedback from interviews, evaluations, and meetings, the working group revised RCE's technical guidelines for land application of sewage sludge and developed eight new fact sheets. Fact sheet topics are:
RCE's guidelines and fact sheets were then peer-reviewed. In addition, the fact sheets were pre-tested with the intended audiences. Other issues that were identified at the workshop (e.g., liability, perception, sewage sludge composition) were addressed in journal articles because some of these topics required more research and others were not appropriate for a fact sheet (Goldfarb, Krogmann, & Hopkins, 1999; Krogmann, Gibson, & Chess, 2001; Krogmann & Chiang, 2002). The peer-review of RCE's technical guidelines focussed on technical accuracy. Peer-reviewers were the Biosolids Working Group, Rutgers faculty, and external experts. The peer-review of the fact sheets was more elaborate and included three steps:
Besides specific comments addressing individual fact sheets, the following general suggestions were provided by the Center of Environmental Communication:
Comments from the various peer-reviews were incorporated in RCE's technical guidelines and in the fact sheets. The next step was to pre-test the peer-reviewed fact sheets with the targeted audience. The fact sheets were pre-tested at grower meetings and County Board of Agriculture meetings. These meetings were chosen to pre-test the fact sheets, because the target audience attended these meetings and because the organizers of these meetings supported the pre-testing. As an incentive for participating in the pre-testing, homemade cookies were distributed to the participants. For materials intended to be widely disseminated, pre-testing with a larger survey encompassing a greater cross-section of people is recommended. Because we assumed limited distribution for our materials, a less rigorous, informal pre-testing was conducted. In addition, a focus group with representatives of the intended audience may provide additional insights. The main results of pre-testing of one fact sheet at a County Board of Agriculture meeting are provided below (Table 3). Each fact sheet was pre-tested to find out if it was clear and understandable and if it met the audience's needs. Filling out the pre-testing questionnaires took about 10-15 minutes.
The results showed the fact sheet was well designed and clear, although there was still work to be done. Main conclusion based on the pre-tests was to cross-reference other fact sheets because questions about issues such as liability and regulations were raised. There were many comments about the tone of the fact sheet (both positive and negative), pointing to the need to use value- and jargon-free language. To address this, the fact sheet benefited from reviewers who were sensitized to this controversial issue at the local level and could provide more neutral language. Implications and ConclusionsThe integration of the development of Extension materials and formative informal evaluations (semi-structured interviews, meeting evaluations, peer-reviews, pre-testing) addressed the controversial issue of land application of sewage sludge and successfully helped in the development of RCE's technical guidelines (RCE, 2001) and outreach materials (RCE, 2000). After not having a program for several years, these outreach materials were published. The novel aspect of our materials is that they also attempt to address social and legal issues. Key for the success of this process was the use of informal evaluative methods on an ongoing basis and the ongoing participation of the intended audience. In addition, the expertise in environmental communication was essential for the evaluation portion of this process. Without this process, the materials would still not exist. However, the process is very time consuming, requiring many hours of preparation and revisions of the materials and background information. The integration of Extension material development and formative informal evaluations tries to address the needs of the target audience. In our case, the outreach materials were designed to address RCE agents' and farmers' concerns such as long-term soil productivity, plant growth, and liability. They do not address the concerns of other groups, such as sewage sludge generators, and may be objectionable to them. Our overall goal was to provide balanced information to RCE agents so that they can help farmers make more informed decisions. ReferencesGoldfarb, W., Krogmann, U., & Hopkins, C. (1999). Unsafe sewage sludge or beneficial biosolids: Liability, planning, and management issues regarding the land application of sewage treatment residuals. Boston College Environmental Affairs Law Review 26, 687-768. Krogmann, U., & Chiang, H.-N. (2002). Selected nutrients and heavy metals in sewage sludge from New Jersey POTWs. JAWRA 38, 681-692. Krogmann, U., Gibson, V. & Chess, C. (2001). Land application of sewage sludge: Perceptions of New Jersey vegetable farmers. Waste Management & Research 19, 115-125. Morgan, M.G., Fischhoff, B., Bostrom, A., Lave, L., & Atman, C.J. (1992). Communicating risk to the public. Environmental Science and Technology, 26, 2049-2056. N.J. (1999). Sludge Quality Assurance. 31 New Jersey Register 200 from February 1, 1999. Patton, M.Q. (1997). Utilization focused evaluation: The new century text. Thousands Oaks, CA: Sage Publications. Patton, M.Q. (2001). Qualitative research & evaluation methods. Thousands Oaks, CA: Sage Publications. Rutgers Cooperative Extension (2001). Guidelines for the land application of sewage sludge for New Jersey agriculture. Edited by Krogmann, U., Boyles, L.S., & Heckman, J.R. Rutgers Cooperative Extension Bulletin #E228. New Brunswick, NJ. Available at: http://www.rce.rutgers.edu/pubs/subcategory.asp?cat=4&sub=31 Rutgers Cooperative Extension (2000). Land application of sewage sludge (biosolids). Series of 8 fact sheets. Rutgers Cooperative Extension Fact Sheet #FS951, #FS952, #FS953, #FS954, #FS955, #FS956, #FS957, and #FS958. New Brunswick, NJ. Available at: http://www.rce.rutgers.edu/pubs/subcategory.asp?cat=4&sub=31
The Oneida County 4-H Conservation Field Days ConflictStephen C. Brown Sarah Vonhof Alicia Kelley Authors' Note: This article describes a highly contentious conflict that played out over the mid-1990's. Publication of this article was purposely delayed at the request of an involved party so that it would not contribute to a worsening of the situation. The names of people involved have been changed at their request. The authors would also like to recognize the significant contributions to this article of Dr. Susan Senecah of the State University of New York College of Environmental Science and Forestry. In the fall of 1994, Cornell Cooperative Extension of Oneida County, New York, conducted its 30th annual Oneida County 4-H Conservation Education Field Days. In its long history, the Field Days program had created no known controversy in Oneida County. The biggest problems usually experienced by its organizers were logistical. The status quo was challenged when a small group of local activists came together to form the Citizens for Balanced Environmental Education (CBEE). Their goal was to change the format and content of the Oneida County program to encompass the views of CBEE. The conflict began when Bart Schnee of CBEE attended the 1994 Field Day program and shot undercover video footage of the presentations containing subject matter that the group considered inappropriate. The videotape was given to the staff at Broadcast New York, who edited its contents and aired it on television statewide. The result was a heated environmental conflict between Citizens for Balanced Environmental Education, Cornell Cooperative Extension of Oneida County, and the Federated Sportsmen's Clubs of Oneida County that lasted for several years. Conservation Education Field Days (CEFD) have been a regular part of Cornell Cooperative Extension 4-H programs statewide for 30 years (Volk, 1995). It is the single largest 4-H educational event in the state. Because no curriculum is formally instituted by Cornell University, individual counties are primarily left to their own resources to organize the program if they choose to participate. Although the lack of a statewide curriculum allows counties to address local needs, it also makes the program particularly vulnerable to local influence. At the time of the conflict described in this article, few counties conducting CEFD's had clearly stated goals and objectives for their programs. This was something CBEE would exploit to its own advantage. In 1994, more than 26,000 youth in 31 counties were involved in CEFD's. In Oneida County, approximately 1,000 children attend the program annually. The format, sponsorship, and name of CEFD's vary widely from county to county, but all share the common theme of a hands-on outdoor event involving exposure to environmental topics, concepts, careers, and/or recreation. It is important to note that different sides of the Oneida County conflict debate the definition of "environmental." For the purposes of this article, it is used to mean any topic, concept, career, or recreational pastime that involves agriculture, wildlife, ecology, biology, or the out-of-doors. At the Oneida County field days, children hear presentations from volunteers and from professionals at various stations in a local state park. It is the goal of many CEFD's to introduce students to environmental careers, and the New York State Department of Environmental Conservation (DEC) personnel are usually presenters. The DEC achieves many of its wildlife and forestry management goals through harvesting activities. Therefore, fishers, hunters, trappers, and foresters are also common presenters. This is the case in Oneida County. 4-H volunteer Ernie Boerman conducted the presentation CBEE found most objectionable. Boerman was also a politically active member of the Federated Sportsmen's Clubs of Oneida County. His presentation, "New York State Furbearers: History & Stewardship," covered the topic of animal population management practices. The trapping aspects of Boerman's program and wildlife management practices promoted by the DEC were what CBEE found objectionable. CBEE claimed that they did not seek to eliminate these programs. Instead, they wanted a "balanced" viewpoint offered. If a presentation on wildlife management focuses upon hunting and trapping, then CBEE feels it should be balanced with another program on alternative management techniques that do not use hunting or trapping. The videotape edited by Broadcast New York portrayed the CEFD as an event where youth are "indoctrinated" by sportsmen into believing that guns, trapping, and hunting are okay. Indeed, by watching the video, one would get the impression that the entire event was about trapping, hunting, and guns. In relatively conservative Oneida County, the conflict was now perceived as a polarized dispute between animal rights activists and hunters/trappers/fishers. According to the three main groups involved in the incident (Cornell Cooperative Extension, The Federated Sportsmen's Clubs, and CBEE), this was not a fair review of what actually occurs at the CEFD. It was evident the video galvanized attitudes on the issue; Cooperative Extension felt they had been "blindsided" by CBEE, having no previous indication that elements of their CEFD were objectionable. The Federated Sportsmen's Clubs felt that CBEE was attacking their practices in general. And from CBEE's viewpoint, this was not an animal rights issue but rather an issue over balanced environmental education. In the following weeks and months, concerned parties on all sides sent letters to local newspapers and public officials. Locally, Cooperative Extension was flooded with letters from national environmental and vegetarian groups via email or facsimile. At least two Freedom of Information Act requests were filed with Cooperative Extension. There was also speculation that "spies" had been employed to find out what the "other side" was planning. At the same time, staffing changes at Cooperative Extension, unrelated to the conflict, created confusion and heightened CBEE's belief that Cooperative Extension was stonewalling in addressing their demands. When Cooperative Extension invited members of CBEE to participate in the planning process for the 1995 CEFD, the Federated Sportsmen's Clubs refused to participate, believing that Cooperative Extension had caved in to CBEE demands. Fearing that without their presence Cooperative Extension would eliminate any presentation not approved by CBEE, the Federated Sportsmen's Clubs later returned to the planning table for the 1995 event. Cooperative Extension, seeing itself in the middle position of the conflict, tried to achieve consensus between the Federated Sportsmen's Clubs and CBEE. The Oneida County legislature then heavily criticized Cooperative Extension for legitimizing CBEE on this issue. Due to increasingly negative public opinion and legislative pressure, Cooperative Extension abandoned the participatory planning effort. Cornell University, realizing it knew very little about what was occurring in the counties running CEFD programs, conducted a survey of counties to find out how they prepared and implemented local CEFD's (Volk, 1995). This led Cornell to conduct a statewide workshop entitled "Increasing the Effectiveness of Conservation Field Day Programs" in June 1995. Their hope was that the workshop would prevent other counties from becoming embroiled in similar controversies by forcing them to examine the quality and objectives of their own CEFD programs. Perspectives on the Conflict from the Organizations InvolvedOne of the most interesting aspects of this conflict is the varying perspectives of the situation held by the three major organizations involved. The following sections describe the conflict over Oneida County CEFD as seen from the three primary parties involved. Citizens for Balanced Environmental Education The following perspectives were obtained through personal interviews with Bart Schnee, Bill Grinder, and Clara Palmese, all representing Citizens for Balanced Environmental Education (CBEE). Like Cornell Cooperative Extension, CBEE saw the Broadcast New York report as the initial event that sparked the conflict. They were disappointed that the tape was edited to frame the controversy as an animal rights issue. Cooperative Extension assumed that Schnee had done the editing to portray the CEFD as against his interests, but the editing was actually done by Broadcast New York. Schnee said that had he done the editing, he would have framed the report differently. CBEE feels that Cooperative Extension and the Sportsmen used the edited video to portray them as radical animal rights activists. CBEE noted several important flashpoints of the conflict. The first was the airing of the report previously mentioned. The second were the alleged incidents of harassment by members of the Federated Sportsmen's Clubs. One member of the Federated Sportsmen's Clubs discovered a nail in his car tire and accused Schnee of planting the nail. According to Grinder and Schnee, no acts of violence were perpetrated by CBEE. When Grinder discovered a nail in his tire, he made no such accusations. There were also incidents in which members of CBEE allegedly received "prank" phone calls and answering machine messages. According to members of CBEE, the source of these calls was identified through Caller ID. CBEE claims that the identified phone number matched a phone number on a presentation application belonging to a member of the Federated Sportsmen's Clubs (the same member who discovered a nail in his tire). CBEE brought this evidence before the planning committee, but felt that it was disregarded. The third flashpoint noted by CBEE was the exclusion of Schnee and Palmese from the 1995 CEFD. On the first day of the event, Schnee and Palmese were told by a park official at the gate that they were not on the list of guests allowed to observe the presentations. The media were also excluded. Schnee responded by contacting lawyers and the American Civil Liberties Union about a possible violation of the First Amendment. On the second day of the CEFD, Schnee was permitted to serve as Grinder's presentation assistant at his booth. This also gave him the opportunity to observe and take still photographs, which ended in a shoving altercation. Finally, CBEE questioned Cooperative Extension's commitment to reform CEFD through a committee planning process. Grinder thought that the committee members and the meeting agendas were unprofessional. He gives the example that meetings were not formally conducted according to Roberts Rules of Order. As time passed, the group began to suspect Cornell Cooperative Extension's promise of reform. CBEE believed the issues approach had shifted from "anti-hunting" balancing "pro-hunting" to "non-consumptive uses" balancing "consumptive uses." This shift in definition meant that presentations like "Recycling" would balance presentations like "Hunting." The Federated Sportsmen's Clubs of Oneida County The information on the Sportsmen's perspective was provided by a personal interview with Ernie Boerman. Boerman is very concerned about the environment and sees himself as a "conservationist" rather than an "environmentalist." Boerman was unaware of any objection to his "Furbearers" presentation until he saw the edited videotape aired on Broadcast New York. The way in which this tape was edited and the negative way in which it represented his presentation enhanced his belief that animal rights activists are fanatics and cannot be dealt with reasonably. He described how Schnee of CBEE requested more information from him following his presentation at the 1994 Field Days and how Schnee falsely presented himself as a person "against those People for the Ethical Treatment of Animals types." Boerman felt that he had been fooled. He associates this kind of tactic with the "surprise attack on Pearl Harbor" and says it reinforces the belief of the Federated Sportsmen's Clubs that animal rights groups and Sportsmen have absolutely no common ground. Boerman believes the refusal to attend any meeting with animal rights activists (CBEE) turned out to be a mistake on the part of the Sportsmen. Cooperative Extension had already held two CEFD planning meetings in which CBEE members were present and Federated Sportsmen's Clubs representatives were absent. When a local outdoor columnist attended one of the meetings, he was shocked at the new direction CEFD seemed to be taking under the influence of CBEE. He subsequently warned the Sportsmen that their failure to attend these meetings could drastically impact the program in favor of CBEE. At the time, the Federated Sportsmen's Clubs believed Cooperative Extension was only dealing with CBEE out of "excess political correctness." They believed that in the end, Extension would fully support the Sportsmen's viewpoint. They also believed that by allowing CBEE to participate in the planning process Cooperative Extension was displaying weakness. The June 1995 workshop, "Increasing the Effectiveness of Conservation Field Day Programs," contributed heavily to Boerman's perception of the situation. At this in-service, Cornell University refused to condemn the viewpoints of animal rights activists. This furthered Boerman's belief that Cornell and its Extension Program were rife with "political correctness." Meanwhile, local DEC (New York State Department of Environmental Conservation) personnel had assured the Federated Sportsmen's Clubs that if compromises made by Cooperative Extension were unacceptable to the Sportsmen, the DEC would withdraw support from the Oneida County program. In such a situation, the Federated Sportsmen's Clubs would also withdraw their support of the program. Such a combined action would have meant the loss of nine out of 12 core presenters and essentially ended the program for 1995. Another important event was the turnover of Extension agents in charge of CEFD: veteran agents Roy DeHope and Sally Haslip had left after accepting other jobs. In Boerman's viewpoint, had DeHope and Haslip remained with Cornell Cooperative Extension of Oneida County, all of the Sportsmen's fears would have been allayed and none of CBEE's demands would have been met. Boerman acknowledged Cooperative Extension's plight and the different directions and pressures being forced upon them, but still perceived the new agents as inexperienced and therefore vulnerable to exploitation. Ernie Boerman did see a silver lining to the controversy. He believed that the Federated Sportsmen's Clubs exposed a bad internal situation in Cooperative Extension and in Cornell University. "The animal rights people had really gotten to that place." Cornell Cooperative Extension of Oneida County Cornell Cooperative Extension's perspective on the conflict was provided by a personal interview with Richard Kragen and Bonnie Rice, Agents at Cornell Cooperative Extension of Oneida County. Neither Kragen nor Rice was involved with the 1994 Conservation Education Field Days (CEFD). Both were assigned to the 1995 CEFD after the planning process had begun. Cornell Cooperative Extension was not aware of any problems with their CEFD until the airing of the Broadcast New York report. They assumed that the video had been edited by Bart Schnee to promote an anti-hunting, pro-animal rights position. They objected to Schnee not coming to Cooperative Extension first if he had a problem with the CEFD's. The "bad press" Cooperative Extension received in newspaper editorials was noted by Kragen as the flashpoint of the conflict. Cooperative Extension felt they were being "criticized for having all sorts of ulterior motives." They responded by forming a planning committee to bring all the parties together in order to address concerns. According to Cooperative Extension, the meetings were not constructive because they found it "difficult to reason with the animal rights group." This was compounded by the fact that there was no written statement of the CEFD's educational goals and objectives Another flashpoint for Cooperative Extension was the September 1995 mailing by CBEE. According to Kragen, "several days before the CEFD they [CBEE] tried to sabotage the event and sent out flyers to all the schools with blatant misinformation and all sorts of allegations as to what was being said and done." Administrators received some phone calls, and one call was received from a concerned parent. Cooperative Extension responded with a mailing of its own. A letter was sent to school superintendents and principals, as well as county legislators, in order to refute the allegations made by CBEE. ConclusionsCornell Cooperative Extension's consensus-building approach resulted in a backlash by both Citizens for Balanced Environmental Education and the Federated Sportsmen's Clubs. In large part, the lack of a statement of goals and objectives for their CEFD forced Cooperative Extension to try to build consensus between two groups for whom consensus was virtually impossible. If CBEE and the Sportsmen could agree on anything, it was that they disliked the way in which Cooperative Extension was handling the situation. Even though it would seem that CBEE has achieved some measure of success in obtaining their goals in Oneida County, they remain disappointed with the "unbalanced" content of the presentations. Yet their inclusion provided them a measure of legitimacy and forced Cooperative Extension in other counties to re-examine their CEFD's. The Sportsmen remained unhappy because after 30 years of dedication to CEFD's, Cornell Cooperative Extension has, in their view, acquiesced to a minority group. From the beginning, Cooperative Extension felt negotiation with the animal rights group would be futile. There seemed to be no room for compromise. Unfortunately, the lack of written educational goals for the CEFD forced Cooperative Extension into a situation where it had little choice but to try to build consensus. One specific action that Cooperative Extension took in response to the conflict was the formation of a planning committee. In the past, only a Cornell Cooperative Extension agent and a member of the Federated Sportsmen's Club planned the CEFD. The new process for the 1995 CEFD entailed planning committee meetings, input from the community, and the careful screening of all subject matter. This was a substantial increase in allocated resources for the event. Both during and after the 1995 event, CBEE complained about the promotion of hunting at a youth event. However, Cooperative Extension maintains that this did not occur. They feel that after completing the lengthy planning process, the 1995 CEFD was "straight down the middle." The presentations were balanced, and the content appropriate. But Agent Richard Kragen admits that his definition of "balance" may differ from the other groups' definitions. Cooperative Extension agrees with a point raised by the Sportsmen that "conservation" is the balance between "preservationist" and "consumptionist" viewpoints. Perhaps the most important lesson to be learned from this conflict is the need for any Cooperative Extension program to have clearly stated goals and objectives. Because the CEFD in Oneida County had been such a long-running and successful program, no one gave much thought to such an important statement. Had there been such a document, it is likely much of this conflict could have been avoided. As of January 2002, Cornell Cooperative Extension of Oneida County has experienced no further problems with Conservation Education Field Day programs. ReferencesVolk, Timothy A. (1995). The current status of Conservation Field Days: Results of an e-mail survey of New York State counties." Cornell Cooperative Extension, Department of Natural Resources, Cornell University. Ithaca, New York: April.
University Extension and Urban Planning Programs: An Efficient PartnershipZenia Kotval IntroductionCommunity-university partnerships have taken many forms as higher educational institutions assess their "engagement" while at the same time communities attempt to tap academia for needed resources. Michigan State University (MSU) has developed a partnership model that fulfills both of these needs. It is built on a partnership between an academic unit, Urban and Regional Planning Program, and an outreach unit, Extension. Urban Planning Programs inherently have an outreach- and practice-oriented component in their curriculum. Experiential learning is an important aspect of these programs. Having an established relationship in urban areas would enhance the student experience. Furthermore, land grant universities in the United States have a legacy of strong Extension functions. MSU Extension is addressing urban concerns through a program that supports staff in six Michigan cities:
The Extension staff in these cities are focused on community and economic development issues in their cities' neighborhoods. Because much of their work is done with community-based organizations, obtaining needed resources is a primary task. A university can provide needed technical assistance to communities by partnering its students with University extension agents. Partnerships between urban Extension agents and planning programs have great merit. With the intent of formalizing this relationship, we developed a partnership model called Urban Planning Partnerships (UPP). This article describes this partnership that has been developed between Extension and Urban and Regional Planning Programs at Michigan State University--Urban Planning Partnerships. It is a model that provides an opportunity for a university to strengthen its links to communities throughout its state. It is a partnership that facilitates the experiential 1eaming needs of urban planning students while assisting urban Extension staff with capacity-building resources. Underlying Learning Theories and PrinciplesMany scholars have written and promoted active learning theories and the importance of experience and reflection in learning and practice. From the importance of experience and the social/cultural context in the creation of knowledge (Bruner, 1966, 1996), to the importance of individualized learning of value of personal experiences (Cross, 1992; Rogers & Feiburg 1994), to Schon's (1987) use of reflection in the development of professionals, scholars have espoused a more reflective, experience based and active pedagogy. University-Community Partnerships have been recognized as a valuable contribution to both the academic community and our cities and towns. In the words of Henry Cisneros, former HUD secretary, "The long-term futures of both the city and the university in this country are so intertwined that one cannot--or perhaps will not--survive without the other." Increasingly, colleges and university are bringing their time, energy, and resources to bear on local problems. They are using their other physical, financial, and intellectual capital to facilitate economic development, provide social services and technical assistance, and create opportunities for applied research. Despite this movement, service learning or experiential learning is seen as a departure from traditional lecture-based courses. Not all disciplines understand, accept, or acknowledge the importance and significance of this pedagogical alternative. One of the most common criticisms of service-learning courses is the lack of faculty experience in structuring a service-learning course. Professional organizations and scholars in liberal-arts-based education are beginning to illustrate best practices and principles in an effort to help design and develop courses that better link theory and practice. For example, the National Society for Experimental Education (1997) offers principles in experiential education. Fundamentals include:
One widely published study (Chickering & Gamson, 1987) sponsored by the American Association of Higher Education, the Education Commission of the States of the Johnson Foundation offers seven principles for creating service-learning courses. These principles focus on:
Kolb's (1984), experiential learning cycle requires four different learning abilities:
Four learning roles are assigned to the four learning abilities:
Learning is most effective when a student goes through all roles regardless of order. There are many more examples of guiding principles, best practices that are emerging to help study and research service-learning alternatives. Believing in the merits and need for service-learning courses and responding to the trend of increasing collaboration between universities and community partners, we crated Urban Planning Partnerships. The program draws on many of the guiding principles and reflects on creating a mutually beneficial association between the university (academic and Extension units) and the communities we serve. Urban Planning PartnershipsUrban Planning Partnerships (UPP) is an outreach initiative located within the Urban and Regional Planning Program at Michigan State University, with primary funding provided by Extension. UPP seeks to facilitate timely research and outreach on urban policy and planning issues in Michigan communities and to build meaningful and lasting ties with these communities. In order to build these long-term commitments and facilitate shared learning, UPP focuses its agenda on the six urban areas where Extension staff are working: Grand Rapids, Detroit, Saginaw, Flint, Pontiac and Lansing. UPP's working agenda is to:
Impacts of the Urban Planning Partnership (UPP) Effectively Links Scholarship with Urban Outreach Students apply their classroom learning to real community situations by testing theory in practice-oriented situations. The students gain real-life experience, and the community gains needed research and technical assistance. Facilitates Linkages Among Teaching, Research, and Outreach Urban Planning Partnerships has introduced urban outreach projects into classroom settings, provided a forum and support for applied research projects, and disseminated timely research on policy issues that affect Michigan communities. Furthers the Mission of MSU-Extension Urban Planning Partnerships has responded to urban issues identified by the six cities and conveyed by the Extension staff. Provides Support for URP Students and Faculty Urban Planning Partnerships has provided support for Urban and Regional Planning Program (URP) students and faculty through graduate assistantships, research funding and support, and matching resources for urban outreach projects. It will provide opportunities for students to gain experience while enhancing their education and promoting faculty outreach. Facilitates Cross-Disciplinary Linkages Although its primary home is Urban and Regional Planning Program, Urban Planning Partnerships is committed to making a conscious effort to seek expertise and help from Landscape Architecture, Geography, and other allied programs on campus to better serve the needs of Michigan communities and facilitate cross disciplinary applied research. The Urban and Regional Planning Program and the Practicum Course The most successful avenue, to date, for integrating teaching, research, and outreach involves the Urban Planning Practicum Course. This course lends itself to the UPP mission quite well. The Urban and Regional Planning program at Michigan State University requires all graduating undergraduate and graduate students to enroll in the Planning Practicum, a capstone course. The faculty in Urban and Regional Planning are unanimous in their support of the Practicum as an important learning tool, integrating classroom work and pragmatic planning in actual community situations. They view the experience as being essential in the progression from student to trained practitioner. It is a tremendous teaching vehicle, helping students to increase their knowledge and confidence while providing a needed service to our communities. Practicum provides a vehicle for substantive learning and the integration of techniques with theory, resulting in graduates who are more effective planners. Placed in the final semester, it is truly a capstone course. The students are asked to draw from their observations of planning history, explain where the project fits in terms of planning theory, use the tools and methods that they have learned, and apply their newly acquired research techniques. It is the responsibility of the Extension agent to identify potential projects that could utilize the expertise of the Urban Planning students. An effective strategy has been for the agents to utilize the students to further a project with a community-based organization that they are working with on a neighborhood revitalization effort. The agent works with the organization to develop the student project and then takes the project idea to the Practicum instructor, who helps develop it further into a meaningful Practicum project. This is an important process--one that requires translating community issues into academic language so that the learning objectives of the course are met as well as the true needs of the community. There are many advantages to utilizing the Practicum project. These include:
Extension agents in urban areas (and probably everywhere else, too) can always utilize students in their community and economic development work. Students lend credibility to the agents' work and provide needed research and other technical assistance that the agent alone cannot provide. Students need community experience to round out their resumes. The difficulty has always been in accessing these students. Urban Planning Partnerships has become the vehicle that has been able to match students with urban Extension work. Practicum undertakes three to six projects each year that have been developed by the urban Extension agents. Students rank the projects by their preference on the first day of class and are allocated their first or second preference as often as possible. Every project must have a community client who works with the Extension staff person. The Extension agent and community clients:
They are also required to stay in weekly contact with the Practicum instructor to ensure that the students are working to expectation. This last point is crucial because both the client and the students must be working toward the same end. For their grade, each Practicum group develops a PowerPoint presentation and a written report of the project. The PowerPoint is presented on campus to an audience of the community clients, Extension agents, and other invited guests. The presentation is also given at a community meeting organized by the community client, which gives the students practice in presenting and also gives the community client a chance to show off their work. The written report is given to the community client, Extension agent, and the Practicum instructor. Implications and Lessons LearnedThe use of practice-oriented courses in university curricula is not unusual. Building strong, sustainable partnerships between entities with different work cultures and expectations, however, could be a challenge. Our Partnership, like any new venture, has experienced successes and challenges. The following discussion is a retrospective view on what we did well and what aspects need more work. We hope the reader may learn from both our experiences. Successes A True Partnership One of the critical reasons why UPP has been highly successful is the nature of the partnership. It is truly beneficial to both MSU Extension and the Urban and Regional Planning Program. Planning faculty have a keen understanding of student capabilities and academic timeframes, course objectives, and learning styles. Extension agents know the pulse of the community and have a keen understanding of problems and issues particular and relevant at a moment in time. A coordinated effort involving both faculty and agents in project planning most certainly yields a better-defined scope of work. Students benefit from the partnership because they have a mentor in school and an advocate and liaison in the community. The agent acts as a "teacher in the field" and often mediates between students and community organizations. Stronger Relationship Over time, faculty and Extension agents build a relationship, understanding, and trust, which benefits students and community alike. By building on the partnership structure, the agent is able to tap into a wide range of student and faculty interests and expertise. This enhances their knowledge base and lends credibility within the community. Extension agents can use the Partnership and commitment developed from the program to leverage grants and funds from foundations, nonprofit and state agencies. Faculty and student time is usually considered an in-kind contribution that can leverage other funds. Partnership provides faculty and students with a constant stream of ideas and projects with community partners for practical projects or applied research. Opportunities for collaborative research on "real-time" problems are a win-win situation for faculty, students, Agents, community, university and the profession at large. Challenges The Partnership hasn't existed without challenges. Every year, these challenges are faced, learned from and a stronger relationship is ultimately built. Some of the more common challenges have been the following. A Lack of Understanding of What Makes a Valuable Student Project Requests for administering surveys or creating a database for properties are often received from agents. Although these are worthwhile tasks, they have limited learning objectives and don't make good semester-long projects. Limited Expertise and Interest Often there is a mismatch between the interests and needs of the Extension agent and the expertise and needs of program faculty and students. Some faculty have defined research agendas that have little room for compromise, or the needs of a community are better served by a discipline other than planning. Identifying and creating a project that meets the needs of the community, the agent, and the planning program is not always easy. Too often, the number and types of projects are limited by the number of faculty who wish to participate and agents who can make a commitment. Limited Time Commitments from Faculty, Students, and Agent The Partnership needs a significant commitment in terms of time and effort. Sometimes, the agent has too many other commitments and can't devote the time and ongoing guidance needed to sustain the project or research activity. Faculty have too many other courses or ongoing research projects, and students have several other courses and often a part time job to balance. Faculty don't always manage to balance our multiple tasks as efficiently and equitably as desired. Unrealistic Expectation Initially there might be unrealistic expectation on both sides. Faculty assumes that the agent knows how to scope a project, organize community support, mentor students, and have time to commit. Agents may assume that they're to bring possibilities to the attention of faculty and that there will be adequate follow through or that every project is a good project. More request are often received than faculty and students are capable of addressing. A concentrated effort is being made to have better communication between the campus and our field partners while learning from past experiences. In SummaryA strong, sustainable partnership that brings the research and expertise of our academic unit to benefit communities with the help of our Extension agents has been created. Extension agents are the true link between campus and the community and play an integral role in furthering the mission of University Outreach. The benefits are worth the effort, and the limitations are not insurmountable. With greater recognition and visibility, Urban Planing Partnership will expand and enhance the collaborative venture. Acknowledgment The author thanks friend and colleague Carol Townsend, Community Development Agent, Michigan State University Extension, for her valuable insights and perspectives on the Partnership and suggestions and edits to this paper. ReferencesBruner, J. (1966). Towards a theory of instruction. Boston, MA: Harvard University Press Bruner, J. (1996) The culture of education. Boston, MA: Harvard University Press. Chickering, A., & Gamson, Z. (1987). Seven principles for good practice in undergraduate education. Study sponsored by The Johnson Foundation. Cross, P. (1992). Adults as learners: Increasing participation and facilitating learning. San Francisco, CA: Jossey-Bass. Kolb, D. (1984). Experiential learning: Experience as the source of learning and development. Englewood Cliffs, NJ: Prentice Hall National Society for Experiential Education. (1997) Foundations of experiential education. Available at: http://www.nsee.org/found.html Rogers, C., & Feiburg, J.H. (1994). Freedom to learn (3rd Edition). Columbus, OH: Merrill. Schon, D. (1987). Educating the reflective practitioner. San Francisco, CA: Jossey-Bass.
Graduation: From National Initiative to Base ProgramSherry C. Betts Donna J. Peterson James C. Roebuck The University of Arizona The Cooperative Extension System supports base programs and national initiatives that help provide direction for educational programming across the country. Base programs and national initiatives are supported by the Cooperative State Research, Education and Extension Service (CSREES) and by the state partners (Cooperative State Research, Education, and Extension Service Base Programs, <http://www.reeusda.gov/1700/programs/baseprog.htm>). National initiatives give special programming emphasis to issues that are of national importance and attention, and have about a 5-year life. Base programs support the seven major areas in which the Cooperative Extension Service provides educational programming:
Keys to Successful GraduationA search of the Journal of Extension revealed several themes relating to the graduation of programs and initiatives as they move from one level to the next. The need to coordinate and construct strategic coalitions and partnerships, and the need for a common vision, commitment, and understanding are key factors that influence the likelihood that an Initiative will develop from infancy into base programming. Williams (1997) presents internal and external factors related to the success of the Decisions for Health Initiative. The internal factors are administrative support and specialist leadership. External factors relate to the changing societal conditions. Williams (1997) persuasively argues for the importance of strategic coalitions for efforts to develop solutions to community problems and issues. Not only do strategic coalitions represent the most effective method for achieving desired outcomes and hence positively affecting an initiative's progress towards sustainability, they also ensure that key players involved with programmatic efforts have the needed information needed to move the program forward (Banach & Gregory, 2001). These strategic partnerships, whether grassroots organizations, specialist leaderships, funding agencies, or community leaders, once linked and strategically networked together, provide valuable insights into how a programmatic response, or initiative, should be constructed or changed over time. Strategic coalitions providing valuable information at key points in an initiative's life cycle is especially important because key tasks, core questions, skills needed, and communication loops may differ according to whether a program is in infancy or has graduated to base programming (Banach & Gregory, 2001). Another broad theme involves the necessity of program designers to provide a clear vision, with strong commitment and clearly articulated goals. Banach and Gregory (2001), in one of the most clearly articulated "road maps" of initiative graduation, stress the importance of having a clear vision, a strong commitment, and well-defined roles at each stage of a programs life. These factors help ensure a "realistic optimism" necessary for continual graduation of programs to the ultimate goal of sustainability. However, it is not enough to merely enunciate a clear vision, an awareness of a program's life course must be considered as well. Kraft (2001) notes the importance of engaging key partners in a long-term vision-producing process early in the implementation process. Moreover, articulating a clear vision while incorporating valuable information from strategic partners at each stage of an initiative's life cycle ensures that it will expand and thrive. And a strong commitment by the state to implement and see the initiative through ensures that a clear vision be continually refreshed in response to an ever-changing social landscape. These broad themes stress what is necessary for a program to succeed, transition toward sustainability, and ultimately graduate into base programming in which the initiative becomes institutionalized into daily social practice. There is little discussion, however, of how the institutional landscape changes once the initiative has graduated to the final desirable level. This article builds on the literature by providing empirical support for changes that occur once a program has graduated to base programming, while providing some lessons and implications applicable to other programmatic responses in the future. A Practical ExampleEstablished in 1994, the Children, Youth and Families At Risk (CYFAR) National Initiative was Cooperative Extension's response to conditions in America that place children and their families at risk. Strategic planners recognized that significant organizational changes were needed to achieve positive outcomes for children, youth, and families at risk and to evaluate the organizational system (4-H National Strategic Business Plan, 1991; Home Economics Sub-Committee, ECOP, 1990; Sadowske & Adrian, 1990; Search Institute, 1993; Strategic Directions of the Cooperative Extension System, 1990; Strategic Planning Council, ECOP, 1990). Niche expansion seemed to be an accurate description of Extension's position at the time of the CYFAR Initiative. An organization must evolve along with the larger society to remain effective (Zammuto, 1982). Over time, effective organizations satisfy their constituents through niche expansion, a process that may involve expansion to include new constituencies. While Cooperative Extension continued to satisfy its traditional constituents, that constituency may be shrinking as a proportion of the U.S. population. Thus, a larger group had yet to benefit fully from the system's resources. There was a mixed reception within the system to these changes due to the fact that in some areas, the old niche was still a comfortable fit, while in others it was not. Such diversity of experience added richness to the organization, but may also have complicated the process of developing and communicating a shared vision at all levels in the system. The Initiative was graduated into base programs in 1999. The mission of the CYFAR Program is:
CSREES provides funding and support to the states for three objectives, all directed toward the CYFAR mission (2002 CYFAR Request For Applications, Program Announcement, <http://www.reeusda.gov/4h/cyfar/>:
Related to these objectives, CSREES staff convened a work group to write a CYFAR philosophy that included six components of organizational change. These were seen as central to the achievement of the CYFAR mission, especially in terms of strategies to build statewide Extension capacity to support this work:
This article presents the results of a study of the graduation of one National Initiative into base programs, focusing on the above six organizational components with an examination of sustainability of program efforts. Implications for the implementation and graduation of other National Initiatives are discussed. MethodThe Organizational Change Survey assesses the six components listed above. Each component was operationally defined with a number of measurable survey items. In October 1998, the University of Arizona published the report, National Results of the Organizational Change Survey (Betts, Marczak, Peterson, Sewell, & Lipinski, 1998, <http://ag.arizona.edu/fcr/fs/cyfar/>). That report included data from 42 states that were collected between September 1997 and July 1998 from all paid Extension professionals in the state who worked directly or indirectly with children, youth and families. It provided a snapshot of the state of Extension during the late 1997 and early 1998 time period with regard to its organizational ability to support programming for children, youth and families at risk. National and state trends were reported and organized by the six components. Three years later, August 2000 to May 2001, this survey was repeated. The survey was changed slightly in Round 2 to reflect feedback received after its first implementation and system changes. Specific references to the Children, Youth and Families At Risk (CYFAR) National Initiative were replaced with phrases referring to children, youth and families at risk in general or simply removed as appropriate. Eligible respondents included all paid Extension professionals in the community, county region, area, and university who work directly or indirectly with children, youth, and families. Individuals in nonpaid positions and Extension professionals who work primarily on unrelated issues were excluded. Respondents were identified by code number only. Each state collected data and forwarded it the University of Arizona, where the data sets were aggregated and analyzed. Twenty-two states submitted data that are included in this report. Analyses T-tests were performed on data from each state to detect significant differences in the mean responses between Rounds 1 and 2. If there were no statistical differences between the data from Rounds 1 and 2, the state is considered to have maintained their previous level of support on that item. If there was a significant increase at the p < .05 level, the state is considered to have significantly increased its capacity, while trend level increase (or approaching significance) is where the probability is greater than .05 but less than .10. The same procedure was used for those states that showed either a significant or approaching significant decrease in capacity. Respondent Characteristics Of the 24 states that completed Round 2, 22 states also participated in Round 1:
The 22 participating states are the unit of analysis. Response rates among the 22 states were consistently high, ranging from 63% to 98%. The number of eligible respondents in the states varied with the size of the state Extension staff. Overall, 3,051 individuals out of 3,846 returned a completed survey, resulting in a 79% response rate. The following characterizes individuals who responded across the 22 states:
ResultsComponent 1 is concerned with vision, planning, and commitment. Table 1 shows that few if any states showed a decrease in the number of respondents who indicated that:
Of particular note is the number of states that have increased in those who believe their states have a clear, long-term commitment to work with children, youth, and families at risk (17 states) and who personally understand their state's vision (13 states). Both of these issues are critical to the integration of CYFAR programming into base programs on a continuing basis.
Component 2 is concerned with training, supports and rewards for staff and volunteers. Table 2 presents data on perceived administrative and campus support. Supervisor support for allocation of time and priorities to work with at risk audiences when there are conflicting demands rose dramatically. Fourteen states reported increasesı12 states reported significant differences at the p < .05 level and an additional 2 states reported trend level increases (p < .10). The other 8 states maintained supervisor support; no states decreased in this area. All 22 states reported either maintained or increased allocation of financial resources for work with children, youth and families at-risk.
Table 3 reports the status of training received. The large majority of states maintained or increased the participation of respondents in all training topic areas. Specifically, 21 of the 22 states either maintained (13) or increased (8) the number of respondents who reported good or excellent overall training and staff development opportunities.
Questions regarding Internet use were added in Round 2, so comparison with Round 1 is not possible on many questions. However, both rounds asked about frequency of use of the CYFERnet Web site. Chi-square analysis found significant differences between the two Rounds, with definite movement from less to more frequent use (P2=160.687, df=4, p < .000). In Round 1, 76% of the respondents accessed CYFERnet once a year or never; in Round 2, this fell to 66%. The greatest increase between rounds was in the category "Every Few Months." Component 3 is concerned with the degree to which Extension professionals are recognized as critical resources in research and education for children, youth, and family issues. Table 4 presents the data for five items, all of which show most states either maintained or increased the number of respondents who reported good or excellent knowledge in principles of positive development, risk and resilience factors, programming for at-risk audiences, and obtaining resources to support programs. The fifth item documents that 19 of the 22 states either maintained (16 states) or increased (3 states) in the percentage of respondents who are called upon at least monthly for their expertise.
Table 5 presents data regarding evaluation knowledge and practice. Sixteen states showed an increase in respondents involved in evaluation. The other six states maintained this involvement from Round 1; no states reported a decrease in those doing evaluation.
Table 6 presents data regarding knowledge of and involvement with policy. Nearly 76% of the respondents believe that one of their roles is to educate policymakers on children, youth, and family issues. This rate was maintained for 16 states and increased from Round 1 for four states. However, only 26% of the respondents indicate they have good or excellent knowledge of policy and legislation, with only three states reporting improvement since Round 1, two states reporting a decrease, and 17 states maintaining the status quo.
Component 4 is concerned with diversity, inclusivity, and pluralism. Again, maintenance of or improvements in this area are evident from the data in Tables 7, 8, and 9. Diversity is treated as a critical issue in all 22 states, with most states reporting that hiring staff and volunteers from populations not historically a part of Extension was a priority. Most states also report a great deal of work with diverse audiences; this was maintained or increased since Round 1. Twenty states reported maintained or increased skill in working with diverse audiences, and all 22 states reported maintained or increased comfort in working with diverse audiences.
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||