|
October 2007
|
       |
Contents
Editor's PageAn Extension ContextAs you know, I review all submissions initially to determine whether they are suitable for blind review (Feature, Research in Brief, Ideas at Work) or publication (Commentary, Tools of the Trade). In 2006, I returned 102 submissions (36%) to their authors for revision before accepting them as suitable for review or publication, and I rejected 49 (17%) as unsuitable for JOE. There are a number of reasons why a submission can get returned for revision or rejected, but a fairly common one is lack of an Extension context. If you read the "Journal Sections/Article Categories" section of the JOE Submission Guidelines, you'll see phrases like "particular interest and significance to U.S. Extension professionals," "importance to U.S. Extension professionals," "interest to U.S. Extension professionals," and "useful to U.S. Extension professionals." JOE is a heterogeneous journal. That is, busy Extension professionals from a wide range of disciplines read JOE. JOE is unlike more specialized, discipline-specific journals in which the data presented speaks for itself to specialists in the same discipline. In JOE, the data must be placed in an Extension context so that it speaks to professionals from many disciplines. In other words, JOE articles must answer the "so what?" question for Extension. That way, as many readers as possible will learn from their fellow Extension professionals' work. It's called "Journal of Extension" for a reason. October JOEI start by calling attention to two Features.
There are also three Features that deal with health issues. They're joined by an Ideas at Work article--"Land Use and Health: What Role for Extension?"--arguing that Extension educators can "bridge the gap between planning and public health." But probably the most striking aspect of this issue--to me, anyway--is how many of the articles deal in one way or another with information technology. I count 11 out of 29, including all six Tools of the Trade articles. I hope I've whetted your appetite for an interesting issue. Laura Hoelscher, Editor
How Integrated Extension Programming Helps Market Cooperative Extension: The North Carolina Recommendation
Karen DeBord Do you ever get tired of people saying, "Extension seems to be the best kept secret?" Those of us who have described our work to others have heard this time and again. Cooperative Extension is indeed a wonderful network of professionals delivering educational information on a variety of topics to the public. However, to truly market what we do, we must take a long hard look at our organization and assure that we are meeting the contemporary needs of society. Marketing the organization is intricately linked to how we conduct our day-to-day business. It is not simply about a new logo or an updated brochure. Marketing goes to the core of who we are and how we operate. Taking a Close Look at OurselvesTo market what we do best, we must first recognize what it is that we do best. Cooperative Extension in North Carolina set out to identify and preserve the best of our rich Extension past while looking toward a renewed future. Establishing a renewed identity parallels the vision referred to as "Engaged Universities" presented the Kellogg Commission (2001). In their report, 25 university presidents collectively agreed that land-grant universities must be responsive and relevant by "bringing the resources and expertise at our institutions to bear on problems in a coherent way." Further, they indicate that within the university, the lines between disciplines must be more transparent, and faculty must be more involved with community issues. The concept of the "engaged" university is valuable in guiding the analysis of the Extension organization and how the organizational structure of Extension parlays into marketing difficulty. Analyzing Extension's Organizational StructureAnalyzing Extension's organizational structure involves understanding internal relationships, relationship dependency, and how authority is granted and managed. In empowered, credible organizations, Tjosvold (1991) recommends establishing strong cooperative links and constructive relations in organizations by developing: (a) a common direction and vision, (b) mutual tasks, (c) assessment of joint productivity, (d) shared rewards contingent upon success, (e) complementary responsibilities and roles that require collaboration, and (f) team identity and supportive culture (p. 297). These are primarily structural and cultural interventions aimed at fostering interdependent (rather than dependent) relations within the organization. That is, less dependency and more shared autonomy. In taking a long hard look at Cooperative Extension, it is apparent that there are many dependent units within Extension. Externally, Extension is no longer in a safe cocoon of county and state government. Others have become part of our traditional and comfortable arena. Over time, these transformations within community organizations have caused Extension to respond in an often undisciplined, segmented, disjointed fashion. How then can an organization be marketed that has so many aspects and that is serving so many areas? It is difficult for the public to understand and grasp. What Part of Extension Should Be Marketed?In his book From Good to Great (2001), Jim Collins advises organizations to identify "their passion, what drives them economically, and what they can be best in the world at" (this author paid no heed to ending a sentence in a preposition). The meeting of these three aspects becomes the (marketable) vortex that represents Extension work. For Extension, this can become the litmus test for decisions regarding planning, prioritization, programming, marketing, personnel functions, and accountability. A North Carolina Extension study team identified Extension's passion as enhancement of the quality of people's lives (economically, environmentally, etc). This includes all people, all classes, all races, all backgrounds. The second aspect, what drive Extension economically, is Extension-built internal and external networks (county partnerships, foundations, grants, and other opportunities). And finally, what is Extension best in the world at? The team agreed that Extension has the ability to provide a collective application of a vast network or portfolio of resources. By combining the Collins vortex with the vision of an engaged university, the North Carolina study team realized program prioritization, program development, and marketing are intricately linked organizational functions. Traditionally, Extension has viewed marketing as merely promotion. This view places Extension communications units in a reactive role. Thus, the team recommended that an integrated educational programming response organizationally must be coupled with marketing efforts in the initial stages. Revising the Extension Educational ModelIf the marketable vortex involves an internal and external application of the portfolio of Extension resources to enhance lives, then the first step in strengthening marketing is to revisit the educational programming model. The educational programming model serves as the underpinning by which new educators are oriented and experienced educators retrained, and provides the bridge to connect marketing with educational programming. Extension educators in North Carolina worked for more than a year to gain system agreement on the new model and definitions. Figure 1 depicts the new model. Figure 1. The definition of educational programming has theoretically (Boone, Safrit, & Jones, 2002) been available but the knowledge of HOW to conduct educational programs is often short-cut, with anxious public servants jumping into educational intervention as opposed to giving attention to the critical steps of intentional planning. The revised North Carolina Extension educational programming definition is: that educational programming "is a planned, comprehensive, and integrated set of educational change strategies that are based on documented, high priority needs and are designed to produce behavioral changes among targeted learners that filter throughout social systems to produce social, economic and environmental impacts" (Guion, 2007). Nested within this definition is a key component that makes this approach different--integrated programming. Integrated programming positions Extension operating as "engaged" with communities around issues as opposed to operating in rigid or narrow subject matter areas. Integrated programming is defined as a collaborative approach involving partners and various disciplines planning and implementing one or more strategies to impact micro and macro systems associated with one or more identified issues (DeBord, 2007). Issues programming has been part of Extension as a system for decades. An issue is a matter of widespread public concern. It is complex and multifaceted, and requires multiple disciplines to bring about impact. Integrated ProgrammingIntegrated programming has two critical dimensions. First, it requires multiple disciplines to bring resources to bear on the identified issue, and second it goes deep within economic, community, and social systems. In the definition of integrated programming above, "systems" refers to places and organizations where people interact, such as schools, families, organizations, government, economy, and culture (Bronfenbrenner, 1979). Learners are the smallest (or micro) aspect of the social system. However, a macro system refers to impacts made with community organizations, systems, policies, standards, or rules, including culture and the economy. In order to truly create change, we must not only work collaboratively across disciplines but we must address the issue at various social levels. Figure 2 provides a visual diagram. Figure 2. Examples of Integrated ProgrammingTo illustrate integrated programming, imagine an Extension educator who is teaching an individual how to prepare for the workforce by teaching job interviewing skills, job-related skills, and resume building. While those are valuable skills to have, integrated programming would see beyond the individual to the other systems that touch this issue of economic development. If, within that community, there are few jobs, there is not transportation for the individual to use to get to work, and no child care for that worker, then what good has it done to teach the individual? An impact has to be made in other system levels and additional partners must be part of the educational design team. Another example is the emerging issue of urbanization or uncontrolled growth in communities. This issue affects the environment, the economy, and quality of life. Multiple strategies are needed at the homeowner level (landscape practice), the youth level (understanding the environment), the family level (creating connections between parent/ child/grandparent to partake of and understand the environment), as well as the community policy level (zoning, growth policy, infrastructure). These are just a few of the partners that must be mobilized to anticipate a collective impact while bringing multiple Extension resources to bear on the issue. Tying Marketing and Program Development TogetherIn North Carolina, we envision a long process to organizationally change who we are and how we operate. One marketing firm (Carolina Public Relations) indicated that once we are ready organizationally, we will "earn" our logo, and marketing will fall into place. To strategically prepare for marketing, NC Cooperative Extension has made some organizational changes. The first step was identifying three strategic priorities: environment, economy, and youth/family/ community well-being. Issues flow from these priorities areas. New specialists in both program development and marketing have been hired to work closely together to identify multi-system and multidisciplinary efforts that address the priority areas. The marketing specialists will highlight those efforts that are truly integrated and that affect systems within the strategic priority areas. However, marketing will be incorporated at the beginning of the educational process and all along the way, not just at the end. Only time will tell if these initial organizational changes will affect the way Extension is perceived by the public. However, the renewed sense of addressing contemporary societal issues has given a facelift to an organization ever interested in providing educational intervention to meet public needs. References:Boone, E. J., Safrit, R. D., & Jones, J. (2002). Developing programs in adult education, 2nd Edition. Prospect Heights, IL: Waveland. Bronfenbrenner, U. (1979). The ecology of human development: Experiments by nature and design. Cambridge, Mass.: Harvard University Press. Collins, J. (2001). From good to great. New York: Harper Collins DeBord, K. (2007). North Carolina Extension Conference. North Carolina State University. Raleigh, NC. Gabelnick, F. MacGregor, J. Matthes, R. S. & Smither, B. L., Eds. (1990). Learning communities: Creating connections among students, faculty and disciplines. New Directions for Teaching and Learning. No 41. San Francisco:Jossey Bass. Guion, L. (2007). North Carolina Extension Conference. North Carolina State University, Raleigh, NC Holland, B. (2005). Civic and academic engagement in the multiversity Institutional trends and initiatives at the University of California. Symposium presentation. Berkeley, CA. Retrieved January 29, 2007, from: http://cshe.berkeley.edu/events/seru21symposium2005/ Kellogg Commission (2001). Returning to our roots: Executive summaries of the reports of the Kellogg Commission the Future of State and Land Grant Universities. Retrieved June 2007 from: http://www.nasulgc.org/Kellog/kellog.htm Tjosvold, D. Andrews, I. R., & Struthers, J. T. (1991). Power and interdependence in workgroups. Group and Organizational Studies, 16 (3). 285-299. Witkin, B. R., & Altschuld, J. W. (1995). Planning and conducting needs assessments: A practical guide. Thousand Oaks: Sage Publications. View reader comments for this Commentary in the JOE Discussion Forum. (This forum is no longer accepting new entries.)
Using Diffusion of Innovation Concepts for Improved Program Evaluation
William G. Hubbard
Lorilee R. Sandmann
The University of Georgia IntroductionThe diffusion of innovations theories, developed over a half century ago, have provided a popular framework to explain how new ideas and technologies are spread and adopted in a community (Rogers, 2003). The framework has been used for program planning, it has been empirically tested, and it has undergone critique from various perspectives since its inception in the 1950s (Yates, 2001). Throughout the years, it has remained instrumental to Extension professionals, scholars, and students alike and continues to be useful in countless other fields, including medicine, telecommunications, information technology, and social marketing (Rogers, 2003). Although the framework has provided practitioners in Cooperative Extension and other fields with an overview of how information is diffused and adopted, its potential as a tool in the area of program evaluation has not been fully examined. The field of program evaluation is complex and comprehensive, but one of its basic goals is to determine whether program participants adopt a particular practice promoted by the educational program. Adoption rates are often expressed as a simple percentage: for example "34% of participants adopted practice 'X' based on a follow-up survey conducted 'Y' months after the program." A number reflecting ultimate adoption of practice, however, does not indicate what role education may or may not have had in the decision to adopt or reject a particular practice. For example, what do we know about those participants who did not adopt a particular practice? Was the proposed practice too expensive to implement? Was it too complex to comprehend and implement? Was it too risky? Did the program presenter provide inadequate information? Answers to these and other questions can provide educators with a better understanding of their role in influencing the adoption of practice. Future programs could then be designed to accommodate these factors and yield higher rates of adoption. Extension educators and evaluation specialists have in fact included questions of this nature in post workshop surveys (Rollins, 1993) but not in a way that isolates the relative and absolute effects of the educational program. This article provides a theoretical argument for using classical diffusion of innovation concepts and theories along with modern multivariate statistical procedures such as regression analysis to gain a more robust understanding of the factors that influence the adoption of practice. In addition, a planned empirical test of this concept is discussed. Gaining this more detailed insight is essential for today's Extension professional who is interested in affecting change. Diffusion of Innovations Background"Diffusion is the process by which an innovation is communicated through certain channels over time among members of a social system" (Rogers, 1963). The diffusion framework is a fairly involved framework that includes several "sub-theories" or concepts. These concepts together provide insight into human and social nature, including how new information is accepted (or not accepted) by potential users. Because of this, the diffusions framework draws heavily from the fields of psychology and rural sociology (Beal & Bohlen, 1957). Components of the classic diffusion framework include the innovation-decision theory, the individual innovativeness theory, the theory of rate of adoption, and the theory of perceived attributes (Rogers, 2003). In addition, the diffusion framework includes theories relating to communication aspects and channels. Only the individual innovativeness theory, the theory of perceived attributes, and concepts relating to communication channels are covered here because these are the most relevant to adoption of practice by individuals. Finally, some factors that are external to the diffusion framework but which affect adoption of practice are discussed. The theory of individual innovativeness suggests that in most social systems there are innovators, early adopters, early majority adopters, late majority adopters and "laggards." These five categories are often visually represented as S- and bell-shaped curves (Rogers, 2003). The S-shaped curve indicates the cumulative number of adopters from innovators to laggards; the bell-shaped curve represents the resulting normal distribution. The S-shaped curve illustrates the fact that there are relatively few adopters at first but that, as the technology, concept, or practice is picked up by innovators and early adopters, their influence will have an impact on the later adopters that make up a majority of potential adopters. The resultant bell-shaped curve graphically represents the different types of adopters and roughly reflects categories corresponding to standard deviations. That is, early and late majority adopters are often time statistically shown to be one standard deviation "above" the mean (average adopter), and the innovators, and early adopters, and laggards are two to three standard deviations "below" the mean (Rogers, 2003). Figure 1 depicts this classic graph. Figure 1. Another diffusion theory, the theory of perceived attributes, focuses on how the program participant views characteristics of the practice under investigation. These have been typically categorized as those that relate to the complexity, compatibility, trialability, relative advantage, and observability of a practice or technology (Rogers, 2003). A brief description of each of these attributes follows.
In summary, all things being equal, the more profitable, understandable, personally compatible, observable, and testable the participant considers the innovation, the higher the potential for adoption. Other concepts relevant to the diffusion of innovations framework and having influence on adoption/rejection decisions include those relating to the communication channel, social networks, and external factors (Rogers, 2003). Communication channels include the change agent or agency and attributes of the communication program (for example, educational program type or means of disseminating information). Social networks and systems include support systems such as a local farmer/forest owner organization or association and the type and amount of interaction with professionals following the educational intervention (such as county agents or foresters). Finally, external factors such as markets, weather, natural disasters, policy, and unanticipated events all affect adoption of practices. Diffusion of Innovation Framework for EvaluationThe diffusion of innovations approach to program evaluation recognizes that a variety of factors influence the adoption of a practice. This approach provides insight into why educational program participants adopt or reject a practice on which they have received information or training. Diffusion of innovations research can provide information, for example, on barriers and motivations external to the educational program that may have strong influences on the decision to adopt or reject a practice. These factors may include the individuals' personality, socio-demographic characteristics, networks, and prior knowledge of the topic. Other influences may stem from the five perceived attributes associated with the practice or innovation under question (complexity, compatibility, trialability, relative advantage, and observability) or from the participant's social network and the availability of information and assistance from other sources. Empirical examples that use this framework for evaluation purposes within the U.S. Extension System are not common. Research by King and Rollins (1995) indicated that change agent's attitude, participant's economic concerns, and technical information influenced the adoption of an agricultural innovation by participants who received information from a training program. Another study investigated factors that influence the adoption of practice and the participation in educational outreach of integrated pest management (IPM) in Utah. Characteristics that influenced farmers and producers to adopt practices or participate in educational programs included major source of income (on-farm or off-farm), farm size, market destination (in-state or out-of-state), diversity of crop produced, past intensity of IPM outreach efforts, and development of commodity organizations (Alston & Reding, 1998 ). Research specifically linking the diffusion of innovations theories with program evaluation has been undertaken in the health field. Cervero and Rottet (1984) created an instrument designed to empirically test a diffusion-innovation-evaluation model hypothesized previously by Cervero Figure 2). The study sought to analyze the impact of the training program (Continuing Professional Education or CPE Program) on behavior change and performance (client outcomes) and to determine the extent of non-education program factors (proposed change, individual behavior, social system) on employee adoption of practice (measured as a change in performance). A 51-item survey instrument was designed, tested for validity, and implemented. Data collection included the review of charts, interviews with participants and supervisors, and personal observation by the researchers themselves. Results indicate that a model based on Rogers's diffusion of innovations explains between 39 and 81% of the variance of the dependent variables (Rogers, 2003). This finding suggests that this design and a modified framework could be extremely useful as an evaluation tool in Extension and other educational fields. Figure 2. From Theory to Practice in ExtensionThe Cervero model informs a way to test an evaluation model based on diffusion of innovations concepts. Such an empirical test of this model within the Extension System is underway for a regional forestry short course (The Master Tree Farmer Series). Past program participants will be surveyed to estimate the influence of various factors on the adoption of practice. Table 1 illustrates a sample of the proposed variables that will be used to study the influence of educational and other variables on the adoption of practice. These are strictly a sampling of the types of questions that will be asked. A group of professional foresters and educators will provide more input into the reliability and validity of these questions and others before the actual survey is designed and delivered. As with the Cervero model, the study covers four categories of independent variable. These include those that relate to the educational program of interest, the individual participating in the program, the proposed practice (trialability, complexity, observability, relative advantage and compatibility), and the social system or network surrounding the participant. A sampling of subvariables and questions are also listed.
Other factors that may explain the variation in the dependent variable include location (are participants in some states more likely to apply a practice than those in others?) and time elapsed since attending the course (landowners may have had time to implement practices that were problematic immediately following their participation). While some of these variables may be correlated with others, the collection of this data at this time will be useful for descriptive purposes (for example, key stakeholders may wish to know if adoption of practice occurs in their state to a greater extent than in others). To More Robust, Defensible Extension EvaluationsWhile the diffusion of innovations concepts were developed during the years of rapid agricultural innovation and dissemination, the theories have evolved over the years to incorporate user/client-based needs (Rogers, 2003). User/client needs-based models incorporate end-users into research design, implementation, and technology transfer activities. The diffusion of innovations framework for program evaluation should lead to a better understanding of any barriers or issues surrounding adoption of a practice by incorporating end user needs and obstacles. As with the diffusion of innovations framework, the area of program evaluation is vast, diverse, and still developing. In Extension, evaluation has historically been one-dimensional. Measurement has focused solely on the impact of program participation in terms of change in behavior or adoption of practice. Characteristics of the educational program and influences of external factors have not normally been accounted for in an inclusive framework such as this. Using diffusion of innovations techniques to guide an Extension program evaluation can provide an opportunity to investigate what is going on inside "the black box" of program impact determination (Bush, Mullis, & Mullis, 1995). In addition, it can also move Extension evaluation toward more causal modeling. Today's Extensionist is living in a complex world where social, economic, and environmental factors all influence adoption of practice (Clements, 1999). Applying statistical inferences to study these factors in a systematic intentional manner can yield a better understanding of the relative impact of education and information. This may be a particularly important investment in major, multi-year, or multi-state Extension programs. Such research may also uncover valuable information for providing leadership among public service providers who also may influence adoption of practice. Examples in the forestry community, for instance, include state forestry agencies and associations, private forestry consultants, and others who provide support and assistance to private owners. Finally, this type of program evaluation model can be adopted for many disciplines within Extension and can be used with stakeholders and potential funding agencies to provide for more holistic, credible evaluations. The results of applying these concepts in an actual program evaluation are currently underway and will provide insight into the practical nature of such a model. ReferencesAbrussese, R. S. (1987). The Cervero model. The Journal of Continuing Education in Nursing, 18(1), 22-23. Alston, D. G., & Reding, M. E. (1998). Factors influencing adoption and educational outreach of integrated pest management. Journal of Extension [On-line], 36(3). Available at: http://www.joe.org/joe/1998june/a3.html Andrews, M. (1983). Evaluation: An essential process. Journal of Extension [On-line], 21(5). Available at: http://www.joe.org/joe/1983september/index.html Beal, G. M., & Bohlen, J. M. (1957). The diffusion process. Ames: Iowa State University of Science and Technology. Bennett, C. (1977). Analyzing impacts of extension programs (Slightly rev. July l977. ed.). Washington: Deptartment of Agriculture Extension Service. Broderick, S. H., Snyder, L. B., & Tyson, C. B. (1996). Selling stewardship within the community: A social marketing approach. In M. J. Baughman & N. Boodman (Eds.), Proceedings: Symposium on nonindustrial private forests: Learning from the past, prospects for the future (pp. 255-263). St. Paul: University of Minnesota, Minnesota Extension Service Special Programs. Bush, C., Mullis, R., & Mullis, A. (1995). Evaluation: An afterthought or an integral part of program development. Journal of Extension [On-line], 33(2). Available at: http://www.joe.org/joe/1995april/a4.html Cervero, R. M., & Rottet, S. (1984). Analyzing the effectiveness of continuing professional educational: An exploratory study. Adult Education Quarterly, 34(3), 135-146. Clements, J. (1999). Results? Behavior change! Journal of Extension [On-line], 37(2). Available at: http://www.joe.org/joe/1999april/comm1.html King, R. N., & Rollins, T. (1995). Factors influencing the adoption decision: An analysis of adopters and nonadopters. Journal of Agricultural Education, 36(4), 39. Lamble, W., & Seaman, D. (1994). Diffusion and adoption: Basic processes for social change. In D. J. Blackburn (Ed.), Extension handbook: Processes and practices. San Francisco: Thompson Educational Publishing. Rogers, E. M. (1963). The adoption process: Part I. Journal of Extension [On-line], 1(1). Available at: http://www.joe.org/joe/1963spring/index.html Rogers, E. M. (2003). Diffusion of innovations (5th ed.). New York: Free Press. Rollins, T. (1993). Profile of farm technology adopters. Journal of Extension [On-line], 31(3). Available at: http://www.joe.org/joe/1993fall/rb1.html Yates, B. L. (2001, May 24-28, 2001). Applying diffusion theory: Adoption of media literacy programs in schools. International Communication Association Conference. Retrieved February 8, 2005, from: http://www.westga.edu/~byates/applying.htm
Use of Computer Technologies by Educators in Urban Community Science Education Programs
Alexey Kudryavtsev
Marianne Krasny
Gretchen Ferenz
Lisa Babcock IntroductionOutreach and Extension programs increasingly are using computer technologies to deliver services and resources to the public. Examples include videoconferences (Pankow, Porter, & Schuchardt, 2006), Webcasts (LGEAN 2004), electronic newsletters (Westa, Broderick, & Tyson, 2005), online communities (Kallioranta, Vlosky, & Leavengood, 2006; Schlager & Fusco, 2004), youth education programs (Mutchler, Anderson, Taylor, Hamilton, & Mangle, 2006), and curriculum and training materials on Web sites and CDs (Dunn, Thomas, Green, & Mick, 2006; Mayfield, Wingenbach, & Chalmers, 2006; Penuel, Bienkowski, Korbak, 2005; Zimmer, Shriner, & Scheer, 2006). When working in low-income urban and other under-resourced communities, Extension staff need to ensure that audiences are able to access the various digital tools. The ability to use computer technologies is often viewed through the lens of the "digital divide," or the gap between those people and communities that can effectively use information and communication technologies and those that cannot (Norris & Conceicao, 2004; Shelley & Thrane, 2004; Warschauer, 2003). Originally, the digital divide referred to the lack of access to computer technologies and Internet connectivity (Mitchell, 2003; Mossberger & Caroline, 2003). Even recently, studies that address the digital divide in communities served by Extension are based on the premise that the digital divide refers to access (Elbert & Alston, 2005). However, Cullen (2001) argued that this is a more complex issue, and identified four factors that may influence the digital divide:
Although access to computer technologies has been described in the literature (Elbert & Alston, 2005), little is known about other factors that are influencing the use of computers in community settings served by Extension. In this article, we explore the four digital divide factors in urban low-income communities and investigate how community educators are using digital materials in their after-school and other non-formal youth education programs. Research QuestionsWe examined components of the digital divide and related computer use as follows:
MethodsWe conducted two separate studies. The first was a qualitative study conducted within the context of the Garden Mosaics program in NYC, and the second was a written survey of CBOs in six US cities. Study 1. Computer Use in Garden Mosaics in NYCGarden Mosaics ProgramGarden Mosaics is a youth and community science education program, through which youth learn about environmental science in urban community gardens within an intergenerational, multicultural, and action context. Originally developed by Cornell University in collaboration with community education programs in cities across the U.S., it has recently moved to a permanent home with the American Community Gardening Association. Funders have included the USDA, NSF, and the Weed Science Society of America. The program's mission is "connecting youth and elders to investigate the mosaics of plants, people, and cultures in gardens; to learn about science; and to act together to enhance their community." From 2001-05, Garden Mosaics reached about 700 educators and 12,000 youth across the U.S. and in Canada <http://www.gardenmosaics.org>. Garden Mosaics employs computer technologies for educator training, curriculum resources, and program implementation as follows:
SampleParticipants in study 1 were eight educators from four after-school programs and four CBOs in low-income communities in NYC. In spring 2005, Cornell University Cooperative Extension-NYC created a list of about 100 CBOs in the South Bronx that work with 10-18 year-old youth, have community gardens in their neighborhoods, and have computers connected to the Internet. Educators from these CBOs were invited to participate in a Garden Mosaics training workshop, after which they were expected to implement the program with youth. Fifteen educators took part in the 2-day training workshop in May 2005, during which they learned how to implement the Garden Mosaics curriculum with youth through demonstrations and hands-on activities and were briefly introduced to the Garden Mosaics online resources and interactive DVD. In June 2006, they were invited to participate in this research project. Six educators accepted this invitation; nine others did not implement Garden Mosaics during summer 2005 and did not participate in this study. An additional two educators from community gardening organizations in Brooklyn who helped organize a Garden Mosaics workshop for garden activists in February 2005, also took part in this research. Among the eight educators, seven are women, three are immigrants, and six represent minority groups (African-American, Hispanic, and South-Asian). The size and number of staff in these organizations vary significantly, from organizations that do not have an office but have more than 10 volunteer educators, to after-school programs that are part of larger community development corporations and have several full-time educators. Interviews and ObservationsIn June-July 2005, we conducted semi-structured interviews (Denzin & Lincoln, 2000; Mason, 2002) with eight educators to determine the availability of computer technologies in their organizations and their computer skills and attitudes. We also visited their organizations and observed their facilities (computer labs, available digital technologies) and conducted observations of the use of Garden Mosaics digital materials by educators in two organizations. In addition, three educators from the NYC CBOs voluntarily participated in a Garden Mosaics online forum during June-August 2005, along with 27 other educators from 10 states. We interviewed these three educators about the benefits that they received from participation in the forum relative to learning about the Garden Mosaics curriculum, sharing ideas about environmental and science education and networking with peers. Study 2. CBO Survey of Computer UseWe conducted a written survey of educators from 21 urban CBOs serving minority and immigrant youth, which were identified as being likely to participate in future science enrichment programs by five larger educational organizations (Fairchild Tropical Botanic Garden in Miami FL, Lawrence Hall of Science in Berkeley CA, Saint Louis Science Center, University of the District of Columbia, and Cornell University). Rather than a random sample, the respondents could be considered as representative of CBOs likely to form collaborations with urban Extension programs. Only one CBO in NYC participated in both study 1 and study 2. The number of employees in the CBOs ranges from 1 to 42, and the majority of the youth and staff are from underrepresented minority groups (predominantly African-American and Hispanic). The survey included questions about Internet access and use of computers with youth. ResultsStudy 1. Computer Use in Garden Mosaics in NYCAccess to Computer TechnologiesBecause educators were chosen for this research from the population that has computers and Internet access, questions about access focused on types of computer technologies they were using. All the educators use computers and the Internet at work and/or at home at least a few times a week. Their computers are adequate to run software for education programs; only one educator had obsolete computers in her organization and thus could not download some program resources (e.g., PDF files). Four CBOs have computer classrooms with more than six computers connected to the Internet available for use by youth. All CBOs have basic computer peripherals (printers, DVD players, speakers, etc.) and other digital devices (e.g., digital cameras). Computer SkillsAll interviewed educators were able to use generic computer applications, such as Microsoft Office and Internet browsers. Two participants of this study had basic Web design skills and used graphic design programs. Only one educator used computers just for checking emails and creating simple word-processing documents and was unwilling to learn other simple computer programs. All educators developed their computer skills themselves through "trial and error" and with help from colleagues; only one educator had participated in computer workshops. AttitudesThe eight community educators were split in their attitudes toward computer technologies. Four educators enthusiastically described integration of computer technologies into their education programs; they also claimed that computers help them learn about education curricula and assist their professional development. These educators reported that they already use the Internet to download lesson plans, "find new ways to teach the same things," look for illustrations for teaching youth in their programs, and check for grant related information. One of these educators finds that the Internet is useful for locating community gardens and accessing neighborhood maps for outdoor activities with youth. Another participant posts her newsletter for members of a gardening CBO on the Web, which helps "to reach more people and reduce using paper." In contrast, four educators were not aware of any benefits from the Internet and computer technologies for their education programs other than sending emails and text editing. For example, one educator wanted to become involved in some kind of networking with peers, but did not think that computer-mediated communication could help her to do that: "What I do not like about computers is the feeling that we are so removed from each other. I feel you get so much more when you talk face-to-face with somebody, and you get a real feel of what's going on and the overall enthusiasm." Content: Use of Garden Mosaics MaterialsCurriculum materials on the Web site. Garden Mosaics curriculum materials for educators are available on the Web site in PDF format; educators also received hard copies during the training workshops. Four of eight educators had not visited the Garden Mosaics Web site following the workshop because they thought the printed handouts had everything they needed. Only one of eight community educators visited the Garden Mosaics Web site repeatedly. DVD. Of the six educators who participated in the training workshop and received the Garden Mosaics training DVD, three did not use the DVD in the 2 months following the workshop, and one stated that she learned all the important information she would need for program implementation at the workshop and that she did not learn much from the DVD afterwards. Another two educators watched the DVD soon after the workshop and found it very helpful for broadening their knowledge about Garden Mosaics. Interestingly, one educator used the DVD for educators to introduce the Garden Mosaics program to youth. Her group of 15 11-13 year-olds watched the DVD on the computer, saw other youth doing Garden Mosaics activities, and became excited about participation in this program. Web forum. Of the three educators who participated in the Web forum, one had limited experience using computers and was nervous about the idea of joining the forum. However, after a trial experience she enjoyed communication with other educators throughout the country, and, in fact, was one of the most active Web forum participants. In the beginning of the forum, educators experienced some technical problems, which nearly discouraged them from participation. However, once we assisted them in overcoming technical problems, participants stated that the forum was helpful for learning about Garden Mosaics. Their messages on the Web forum discussion boards indicate that they benefited from networking with other educators across the U.S. (Kudryavtsev, 2006). Online databases. Educators were informed at the workshops that youth can submit results of their investigations and action projects to the databases on the Garden Mosaics Web site. Two of the eight educators understood the educational value of the databases for youth. For example, one of them said, "I like the data-sharing and stories. It gives students the chance to write about and express their experience with the gardens; it allows them to read other experiences, so they do not feel that they are alone." However, six educators were not aware of benefits from using these databases with youth and did not understand how to submit the information to the databases. Study 2. CBO Survey of Computer UseOf the 21 CBOs surveyed in May-June 2006, 19 had fast Internet connections, and 18 used computers in youth programs. When asked to rate the importance of: "Using the latest 'hip' digital technologies" as a "means to get youth engaged in learning science," the educators' mean response was 2.1 on a Likert scale of 1-10, with 1 being most important. Using digital technologies ranked slightly lower than "hands-on activities" (1.4), "identifying a problem in your community and using science to help solve it" (1.8), and "communicating with scientists, including scientists who are young and culturally diverse" (2.0). For the three CBOs that do not use computers with youth, reasons given included lack of access, prohibitive costs, logistics (youth group too large for computer lab), and an emphasis on nature-based rather than classroom type programs (this CBO used hand-held GPS devices with youth in their outdoor activities). The remaining 17 CBOs cited the following uses of computers in their youth programs: Internet research (8 responses), online learning activities (4), GIS/GPS (2), presentations (2), and design projects (2). Several other uses received one response each, including for homework, as rewards, to demonstrate concepts using CDs, PDAs in field work, and for making budgets. DiscussionThe results of this study suggest that if Extension is to play a role in bridging the digital divide in low-income, urban minority communities, access to computers and the Internet may no longer be a major concern. Similarly, whereas lack of computer skills limited the use of computers among some NYC educators, this problem was readily overcome with minimal support. Although the sample sizes in this study were small, the finding that access to computers is no longer a major barrier to urban Extension programs is supported by the fact that Cornell Cooperative Extension-NYC was able to identify approximately 100 CBOs in low-income communities with Internet access, and by the observations of our colleagues in other cities that CBOs generally have access to Internet technologies (personal communication, N Stein, Lawrence Hall of Science; H Hughes, Saint Louis Science Center). Thus, whereas we cannot definitively say that most CBOs have Internet access, it appears that finding CBOs working in urban minority communities that have Internet access is not a problem. Attitude toward computer technologies and awareness of content appeared to be important factors limiting the use of digital resources in Garden Mosaics, where only half the educators used computers in their youth programs. However, the written survey conducted 9 months later suggests that computer use in CBO youth programs is much more widespread, with nearly all educators incorporating computers into their youth programs. It is possible that the difference in timing between the two studies accounts for some of this discrepancy because it may be that the use of computers in urban CBOs is relatively recent. Another possibility is that the CBOs in the second study had a longer history of collaboration with science museums and other larger organizations that supported their use of computers. Furthermore, the ways in which Garden Mosaics uses technology in its youth programs, i.e., for reporting results to the Internet, may not be aligned with CBO youth practices. The survey indicated that youth in CBO programs use computers for Internet research and learning activities, rather than for data reporting and other uses. That reporting results is not a preferred use of computers by youth programs is supported by the results of a 2005 Garden Mosaics evaluation survey of 591 CBO and Extension educators from across the U.S., which revealed that even though 83% of educators had computer access at work, only 9% of Garden Mosaics youth programs submitted reports to the online databases. Of those groups that did submit reports, the educators felt that "youth seeing their work on the Web site" was more important than "computer use being a good experience for the youth" or "youth feeling as if they were contributing to the work of scientists" (Kudryavtsev, unpublished data). ConclusionExtension educators working with urban CBOs should no longer be guided by the notion of the digital divide as limited access to computer technologies. Rather our results suggest that Extension programs that target low-income urban communities need to consider demonstrations of new and innovative computer uses in educational programs and discussions of the values of these technologies as professional development and educational tools. Finally, knowing that youth are motivated by using "hip" technologies, Extension should make a concerted attempt to engage youth and adults working closely with youth in designing the technology component of their programs. Acknowledgement This work was funded by the National Science Foundation (ESI 0125582), the Cornell Urban Scholars Program, Edmund S. Muskie/FREEDOM Support Act Graduate Fellowship Program, and USDA. Thanks to Garden Mosaics Program Leader Keith Tidball and to Ken Reardon and Ruth Sinton of the Cornell Urban Scholars Program for their contributions to this research. ReferencesCullen, R. (2001). Addressing the digital divide. Online information review. 25(5):311-320. Denzin, N., & Lincoln, Y. (2000). Handbook of qualitative research. Thousand Oaks: Sage Publications. Dunn, C., Thomas, C., Green, C., & Mick, J. (2006). The impact of interactive multimedia on nutrition and physical activity knowledge of high school students. Journal of Extension [On-line], 44(2). Article 2FEA6. Available at: http://www.joe.org/joe/2006april/a6.shtml Elbert, C., & Alston, A. (2005). An evaluation study of the United States Cooperative Extension Service's role in bridging the digital divide. Journal of Extension [On-line], 43(5). Article 5RIB1. Available at: http://www.joe.org/joe/2005october/rb1.shtml Kallioranta, S., Vlosky, R., & Leavengood, S. (2006). Web-based communities as a tool for Extension and outreach. Journal of Extension [On-line], 44(2). Article 2FEA4. Available at: http://www.joe.org/joe/2006april/a4.shtml Kudryavtsev, A. (2006). Use of computer technologies in dissemination and implementation of environmental education programs. MS Thesis. Cornell University. Available at: http://www.gardenmosaics.cornell.edu/pgs/aboutus/aboutus4.htm Local Government Environmental Assistance Network. (2004). Seeing green with trees: the economic and environmental benefits of urban forests. Available at: http://www.lgean.org/html/whatsnew.cfm?id=853 Mason, J. (2002). Qualitative research. Thousand Oaks: Sage Publications. 223 p. Mayfield, C., Wingenbach, J., & Chalmers, D. (2006). Using CD-based materials to teach turfgrass management. Journal of Extension [On-line], 44(2). Article 2FEA5. Available at: http://www.joe.org/joe/2006april/a5.shtml Mitchell, M. (2003). Possible, probable and preferable futures of the digital divide. Informing Science. June 2003. 609-627. Mossberger, K., & Caroline, J. (2003). Virtual inequality: beyond the digital divide. Washington, D.C.: Georgetown University Press. 192 p. Mutchler, M. S., Anderson, S. A., Taylor, U. R., Hamilton, W, & Mangle, H. (2006). Bridging the digital divide: an evaluation of a train-the-trainer, community computer education program for low-income youth and adults. Journal of Extension [Online] 44(3). Article 3FEA2. Available at: http://www.joe.org/joe/2006june/a2.shtml Norris, D., & Conceicao, S. (2004). Narrowing the digital divide in low-income, urban communities. New directions for adult and continuing education, 101:69-81. Pankow, D., Porter, N., & Schuchardt, J. (2006). Training educator and community collaborators using a satellite videoconference format. Journal of Extension [On-line], 44(1). Article 1TOT6. Available at: http://www.joe.org/joe/2006february/tt6.shtml Penuel, W., Bienkowski, M., Korbak, C., et al. (2005). GLOBE Year 9 evaluation: Implementation supports and student outcomes. Menlo Park, CA: SRI International. Schlager, M., & Fusco, J. (2004). Teacher professional development, technology, and communities of practice. Are we putting the cart before the horse? In: Barab S., Kling R., & Gray J., eds. Designing for virtual communities in the service of learning. Cambridge: Cambridge University Press, 120-153. Shelley, M., & Thrane, L. (2004). Digital citizenship: Parameters of the digital divide. In: Social Science Computer Review, 22(2): 256-269. Warschauer, M. (2003).Technology and Equity: A Comparative Study. Paper presented at the Annual Meeting of the American Educational Research Association, April 24, 2003, Chicago, Illinois. 37 p. Westa, S., Broderick, S., & Tyson, B. (2005). Getting the work out in the Last Green Valley: Integrating digital video, direct mail, and Web-based information for specific target audiences. In: Journal of Extension [On-line] 43(1). Available at: http://www.joe.org/joe/2005february/a7.shtml Zimmer, B., Shriner J., & Scheer, S. (2006). Use of evaluation of a statewide 4-H volunteer newsletter. Journal of Extension [On-line], 44(1). Article 1RIB8. Available at: http://www.joe.org/joe/2006february/rb8.shtml
Preferred Information Delivery Methods of North Carolina Forest Landowners
Robert E. Bardon
Dennis Hazel
Kevin Miller North Carolina State University IntroductionDynamic forestland ownership patterns and increased demands for forest products together emphasize the need to deliver relevant forestry information to a growing and changing non-industrial private forestland (NIPF) owner population. North Carolina's NIPF population is estimated at 479,000 (Brown, New, Oswalt, Johnson, & Rudis, 2006). Giving one-on-one attention to each forestland owner would best satisfy their diverse needs, but would be impossible to accomplish. Since one-on-one attention is impractical, Extension educators will need to deploy other methods of information delivery in order to reach NIPF clientele. Researchers suggest using a diversity of information delivery methods to reach clientele, but particular information delivery methods must be matched with target audiences to insure their efficacy (Egan, Welch, Page, & Sebastian, 1992; Rodewald, 2001; Londo & Gaddis, 2003; Radhakrishna, Nelson, Franklin, & Kessler, 2003; Cartmell II, Orr, & Kelemen, 2006). The choice of information delivery method used by an Extension educator may have serious consequences for program effectiveness. Some fear that using one information delivery method may alienate those who prefer another and particularly that "high-tech approaches may intimidate certain groups of clientele (e.g., older clientele)" (Rodewald, 2001). Because so many delivery methods are available today, the preference of the clientele for a particular method may be difficult to predict. The purpose of the study reported here was to identify preferences for information delivery methods among groups of North Carolina's non-industrial private forest landowners and to investigate these groups for descriptive socio-demographic, land, or management experience characteristics. If information delivery method preferences can be linked with socio-demographic, land ownership, or management characteristics, educational efforts can be directed at specific groups of landowners using the methods they prefer. MethodsData for this analysis came from a 2005 mail survey of 2600 NIPF landowners from 13 counties. The 13 counties, selected using a stratified random sample, were chosen from a population of 100 counties distributed between seven Cooperative Extension districts. A stratified random sampling of the counties was done to ensure that all regions of the state were represented (Figure 1). Figure 1. Within each county, 200 landowners were randomly selected from the 2004 present use-value tax records. Surveys were mailed to all 2,600 landowners, with a reminder postcard sent to the recipients 3 weeks after the original mailing. Late respondents were given the option to send in the original survey, request an additional survey by mail or telephone, or use a Web address on the postcard to access an identical copy of the survey that could be completed online. The survey instrument was designed based on previous studies of NIPF owners (Birch, 1996) and using Surveying the Social World: Principles and Practice in Survey Research (Aldridge & Levine, 2001). Prior to mailing the survey it was reviewed by 10 people of various backgrounds to include local landowners, graduate students, natural resource professionals who work with the public, and North Carolina State University faculty members. The survey asked participants about their preferences for information delivery methods, their socio-demographics, their land, and their forest management experience. The six information delivery methods included mail-based material, Web-based material, short programs, long programs, landowner association participation, and distance education. Mail-based material was defined as newsletters, brochures, compact discs, Extension publications, and magazine articles. Web-based material was defined as Web-site reading, downloadable publications, or streaming video. Short programs were defined as evening or less than half-day seminars or workshops at county facilities. Long programs were defined as full day/multiple day field site visits or demonstrations. Participation in a landowner association was considered self-explanatory. Distance education was defined as Web-based landowner courses, video-based landowner courses, or textbook-based correspondence courses. The options were not mutually exclusive. Respondents were asked to rank each information delivery method on a 4-point continuum somewhere between would never use and would often use. The 4-point continuum interval is by one. Socio-demographics factors included gender, age, marital status, occupation, number of children below the age of 18, income, and education. Land ownership factors included acreage owned, land ownership tenure, resident or absentee landowner, and primary residence location. Forest management factors included past forest management experience, future plans for forest management, sources from which forestry information is obtained, and income needs from their forestland. The definition of "past experience" refers to forest management practices previously undertaken, and the definition of "future plans" refers to the likelihood that a landowner will practice forest management on their land in the future. Both "past experience" and "future plans" were ranked on 10-point continuums, with "past experience" ranked somewhere between "not at all experienced" and "very experienced" and "future plans" ranked some where between "not at all likely" and "very likely". Each 10-point continuum's interval is by one. Nearly all survey questions inherently had categorical responses; the few that did not, age, land ownership tenure, and acreage owned, were categorized using Birch's (1996) classifications. A K-means cluster analysis (SAS, 1999) was performed using only respondents' preference for information delivery methods. To investigate differences among clusters with regard to socio-demographics, land characteristics, and management experience, contingency tables analysis (SAS, 1999) was used. To determine whether or not clusters were statistically significantly different with respect to a given question, Pearson's Chi-Square was used. ResultsResults of the study are based on 460 returned questionnaires, those in which respondents answered all questions about information delivery methods and claimed at least one acre of forestland. The response rate was 17.7%. K-means cluster analysis identified five groups of landowners that were cohesive with respect to preference for information delivery methods. Figure 2 shows the likelihood that a member of a given cluster will use a particular information delivery method. Each cluster has been given a memorable name that helps describe the preferred method of information delivery. The "Don't Bother Me" cluster is unlikely to use any information delivery method. The "Snail-Mailers" prefer only mail-based information delivery. The "Short-Mailers" prefer mail-based materials and short programs. The "Web-Mailers" are most likely to use mail-based information and the Internet. Finally, the "Fan Club" cluster will likely use any information delivery method. Figure 2. Contingency table analysis resulted in the identification of several socio-demographic characteristics that differed significantly across clusters (Table 1). These characteristics were retirement status, marital status, number of children under 18, age, occupation, income, and education. Gender was not significantly different among clusters at the 0.05 level (p = 0.12), indicating that each landowner cluster contained relatively the same ratio of males to females. Landowners in the "Don't Bother Me" cluster, "Snail-Mailers" cluster, and "Short-Mailers" cluster were more likely to be retired than the landowners in the "Web-Mailers" or "Fan Club" clusters. "Web-Mailers" landowners and "Fan Club" landowners were more likely to be married and have children under the age of 18. A similar pattern is seen with age class, occupation, income, and education, where "Web-Mailers" and "Fan Club" landowners dominated the lower age classes, have higher ratio of landowners in white-collar occupations, in upper income classes, and higher education levels.
Contingency table analysis revealed that clusters differed by acreage class and land ownership tenure, but not by location of primary residence (p = 0.13) or residence on their forestland (p = 0.13) (Table 2). The "Don't Bother Me" cluster and "Snail-Mailers" cluster consisted of landowners predominantly in the smaller acreage classes and dominated the longer tenure classes. Landowners in the "Short-Mailers," "Web-Mailers," and "Fan Club" clusters also owned land in the smaller acreage classes but had a greater ratio of landowners with in the larger acreage classes, 100 acres and larger. "Short-Mailers," "Web-Mailers," and "Fan Club" clusters had fewer landowners in the older tenure classes, tenure classes prior to 1960.
Analysis of the clusters' forest management experience characteristics revealed four key areas of differences: past management experience, future plans for management, the percent of the cluster's respondents who require income from their forestland, and sources from which forestry information has been obtained (Table 3). Three information sources, Forest Industry (p = 0.06), Logger/Timber Buyer (p = 0.74), and Neighbors (p = 0.13), were not significantly different among clusters at the 0.05 level. Landowners in the "Don't Bother Me" cluster and "Snail-Mailers" cluster were less likely to have past management experience, less likely to have future plans for forest management, and less likely to required income from their land. Their top four sources for information received were State Forest Service, consulting foresters, Cooperative Extension, and logger/timber buyer. The "Short-Mailers," "Web-Mailers," and "Fan Club" clusters consisted of landowners much more involved with their land. These clusters had more landowners who have past management experience, future plans for forest management, and were more likely to require income from their land. The "Short-Mailers," "Web-Mailers," and "Fan Club" top four sources for forestry information were State Forest Service, consulting foresters, Cooperative Extension and logger/timber buyer.
DiscussionKrejcie and Morgan (1970) indicate that a sample size equal to 384 is statistically representative of a population of 1 million individuals. In North Carolina, the NIPF landowner population is estimated to be 479,000 owners (Brown, New, Oswalt, Johnson, & Rudis, 2006). Based on Krejcie and Morgan (1970), the response rate of the study reported here is statistically representative of North Carolina's NIPF owners. Cluster analysis determined that there are five groups of landowners with respect to information delivery method preferences in North Carolina. One of the clusters, the "Don't Bother Me" cluster, expressed very little interest in any information delivery method or in managing their forestland. Because of this, the "Don't Bother Me" cluster is likely to be very difficult to reach. They only constitute 7% of the respondents, so expending effort to direct educational efforts at this group of people will be costly for the amount of impact that could be expected. A second cluster, the "Fan Club" cluster, expressed interest in all information delivery methods. This group represents 23% of respondents and consists of landowners in all acreage classes, with a majority in the 100-500 acre class. A majority of respondents in this cluster have received forestry information from Cooperative Extension, State Forest Service, consulting foresters, and loggers/timber buyers. These landowners will not require Extension educators to target them with a specific information delivery method in order to be reached; information delivery methods targeted at other groups will reach this group. The three remaining clusters, "Snail-Mailers," "Short-Mailers," and "Web-Mailers," which represent 21%, 24%, and 25% of the respondents, respectively, have particular preferences for methods of information delivery, and each has characteristics that allow for the identification of these target audiences. By being able to identify specific audiences among these three clusters and targeting them with their preferred delivery method, Extension educators will be most effective in delivering forestry education. "Snail-Mailers"Approximately 21% of respondents were classified as "Snail-Mailers." "Snail-Mailers" prefer mail-based information to all other delivery methods. Nearly two-thirds of this cluster is over 66 years old. More than two-thirds of this cluster is retired. Compared with "Short-Mailers" and "Web-Mailers", the "Snail-Mailers" cluster has a higher percentage of respondents (29.5%) who earned less than $40,000 in 2004, likely because many members of this cluster are retired (68.4%). To reach this cluster most effectively, Extension educators should specifically target retirees. Educational information should be developed that can be direct | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
