The Journal of Extension - www.joe.org

February 2017 // Volume 55 // Number 1 // Tools of the Trade // 1TOT2

Organizational Capacity Survey: Capturing an Extension System's Current State and Pinpointing Areas for Improvement

Abstract
An organizational capacity survey (OCS) can reveal gaps in what an organization is doing and what an organization's employees feel the organization should be doing. Mississippi State University (MSU) Extension conducted an OCS to assess perceptions of MSU Extension's vision and commitment, plan of work, working relationships, diversity and pluralism, public value, personnel knowledge and expertise, and training needs. Questions were designed to measure perceptions of how MSU Extension is currently and how it should be ideally. This design allowed discrepancy scores to be calculated to identify areas in which capacity development was needed or desired that could then be addressed through state-based professional development.


Donna J. Peterson
Evaluation Specialist
donna.peterson@msstate.edu

Laura H. Downey
Evaluation Specialist
laura.downey@msstate.edu

Alisha Hardman
Evaluation Specialist
a.hardman@msstate.edu

Mississippi State University
Mississippi State, Mississippi

Extension systems are often in transition due to the requirement to be responsive to factors that include stakeholder needs; local, state, and federal budgets; technological changes; legislative priorities; and personnel changes. Such organizational transitions provide an opportune time for organizational learning. Mississippi State University (MSU) Extension has been engaged in an intentional process to reassess system priorities, evaluate efficiency and effectiveness, and focus on resource management with two primary goals in mind: (a) enhance connections between campus and counties and (b) re-establish an integrated program planning framework and process.

When broad system changes are put into motion, several activities can help ensure success (Betts, Peterson, & Roebuck, 2003):

  • Explicitly declare the desired changes and organizational supports needed.
  • Write operational definitions of strategies and desired outcomes so that they can be measured.
  • Evaluate system change through data collection at multiple time points (e.g., baseline, mid-course, end).
  • Use the evaluation data to provide feedback, inform stakeholders, make changes during the process, and plan for sustainability.
  • Provide organizational support for staff through in-service training, administrative support, and other relevant resources.

Development and Implementation of an Organizational Capacity Survey

MSU Extension evaluation specialists recognized that an organizational capacity survey (OCS) could fill the need they had to understand MSU Extension employees' perceptions of the state of the organization. An OCS is one type of tool that can be used for documenting system changes in times of transition. The MSU Extension evaluation specialists used the OCS to explore the current state of MSU Extension as perceived by its employees in terms of (a) assumptions on which organizational support for programs are based; (b) the ways in which organizational values, vision, and strategic planning are communicated and understood across organizational levels; (c) identification of areas in which capacity needs to be developed; and (d) celebration of areas of success (Betts, Peterson, Marczak, & Richmond, 2001).

The online OCS was conducted from May through June 2014. The MSU OCS was loosely based on an OCS developed to document the current states of and changes in abilities of state Extension systems that had received Children, Youth, and Families at Risk (CYFAR) funding from the U.S. Department of Agriculture (Betts, Marczak, Peterson, Sewell, & Lipinski, 1998). This foundational OCS is referred to herein as the CYFAR OCS. (CYFAR funding was to be used to develop and sustain effective programs for children, youth, and families at risk. For more information regarding the CYFAR OCS, see "Graduation: From National Initiative to Base Programs" in the February 2003 issue of the Journal of Extension.) Whereas the CYFAR OCS was used with paid Extension professionals who worked directly or indirectly with children, youth, and families, the MSU OCS targeted MSU Extension administrators, specialists, agents, and instructors across all four program areas (agriculture and natural resources, community resource development, family and consumer sciences, and 4-H youth development).

Similar to the CYFAR OCS, domains assessed in the MSU OCS were MSU Extension's vision and commitment, plan of work, working relationships, diversity and pluralism, public value, personnel knowledge and expertise, training needs, and demographics. Plan of work and public value were not included in the CYFAR OCS, but were added as new domains to the MSU OCS. Table 1 provides sample survey questions. The full instrument is available from the first author.

Table 1.
Sample Questions for Each Domain of the Mississippi State University (MSU)
Extension Organizational Capacity Survey

Survey domain Sample questions
MSU Extension's vision and commitment
  1. Our state Extension Service has articulated a clear vision for work in:
    1. Agriculture and Natural Resources.
    2. Community Resource Development.
    3. Family and Consumer Sciences.
    4. 4-H Youth Development.
  2. Our county's vision is congruent with the state Extension Service's vision for work in:
    1. Agriculture and Natural Resources.
    2. Community Resource Development.
    3. Family and Consumer Sciences.
    4. 4-H Youth Development.
[Response options: strongly agree, agree, not sure, disagree, strongly disagree]
Plan of work
  1. Our state Extension Service has a State Plan of Work in place for expanding and strengthening programming in:
    1. Agriculture and Natural Resources.
    2. Community Resource Development.
    3. Family and Consumer Sciences.
    4. 4-H Youth Development.
  2. I personally understand my role in our State Plan of Work in:
    1. Agriculture and Natural Resources.
    2. Community Resource Development.
    3. Family and Consumer Sciences.
    4. 4-H Youth Development.
[Response options: strongly agree, agree, not sure, disagree, strongly disagree]
Working relationships
  1. In your work, mark the type of relationship you currently have with:
    1. Extension professionals from other counties.
    2. Community agencies and organizations.
[Response options: none, networking, cooperation, coordination, coalition, collaboration]
  1. Collaboration with other organizations is worth the effort.
[Response options: strongly agree, agree, not sure, disagree, strongly disagree]
Diversity and pluralism
  1. In the MSU Extension Service, we:
    1. Plan programs to address the needs and assets of audiences rather than expect participants to fit existing programs.
    2. Recruit staff from populations not historically a part of Extension.
[Response options: strongly agree, agree, not sure, disagree, strongly disagree]
Public value
  1. I engage the public to identify needs not currently being addressed by Extension programs.
  2. I have evidence of at least two reasons why Extension is valuable to the public, not just clients.
[Response options: strongly agree, agree, not sure, disagree, strongly disagree]
Personnel knowledge and expertise
  1. Please rate your knowledge of:
    1. Resolving conflict.
    2. Working with Advisory Councils.
[Response options: excellent, very good, good, fair, poor, not applicable]
Training needs
  1. Indicate whether you would like to receive training on the following topics in the next 12 months:
    1. Communicating the public value of Extension programs.
    2. Sustaining effective programs.
[Response options: yes, no]

The majority of MSU OCS questions were phrased to assess perceptions of MSU Extension currently and perceptions of how MSU Extension should be ideally. This format allowed for the calculation of discrepancy or gap scores between paired items to identify areas in which capacity development was needed and/or desired. Specifically, discrepancy scores were calculated for questions about organizational vision and commitment, plan of work, working relationships, diversity and pluralism, and public value by subtracting the "ideal" response from the "current" response to demonstrate the size and direction of the gap.

The processes for establishing validity and reliability of items in the CYFAR OCS were described in the initial report of results (Betts et al., 1998). Since the plan-of-work items in the MSU OCS were modeled after the mission and vision questions in the CYFAR OCS, validity and reliability should mirror that of the CYFAR survey. Face validity of the public value scale was established through expert review, with minor revisions made on the basis of expert recommendations. Cronbach's alpha was .782 for the "current" public value items and .865 for the "ideal" public value items.

Conclusions

The MSU OCS demonstrates the utility of the method for informing organizational learning. Findings from an OCS can reveal gaps in what an organization is doing and what an organization's employees perceive the organization should be doing. Identified areas of need can then be addressed through state-based professional development. Such professional development should address relevant structural factors (e.g., mechanisms and procedures that allow an organization to systematically collect, disseminate, and use information) and cultural factors (e.g., shared professional values, vision, and leadership) (Rowe, 2010).

Extension systems in other states could adopt and/or adapt the MSU OCS to identify areas in which their respective organization's capacity could be strengthened to effectively support staff and programs, and meet clientele needs. If multiple Extension systems used the MSU OCS questions, gaps that occur across states could be identified, perhaps resulting in regional or national capacity-building efforts. Also, if an OCS is repeated every 3–5 years, Extension systems can track changes over time and document trends in organizational capacity.

References

Betts, S. C., Marczak, M. S., Peterson, D. J., Sewell, M., & Lipinski, J. (1998). Cooperative Extension's capacity to support programs for children, youth and families at risk: National results of the Organizational Change Survey. Retrieved from http://ag.arizona.edu/sfcs/cyfernet/cyfar/

Betts, S. C., Peterson, D. J., Marczak, M. S., & Richmond, L. S. (2001). System-wide evaluation: Taking the pulse of a national organization serving children, youth, and families at risk. Children's Services: Social Policy, Research, and Practice, 4(2), 87–101. doi:10.1207/S15326918CS0402_3

Betts, S. C., Peterson, D. J., & Roebuck, J. C. (2003). Graduation: From national initiative to base programs. Journal of Extension, 41(1) Article 1FEA4. Available at: https://www.joe.org/joe/2003february/a4.php

Rowe, E. (2010). Looking at Extension as a learning organization. Journal of Extension, 48(4) Article 4RIB1. Available at: https://www.joe.org/joe/2010august/rb1.php