October 2007
Volume 45 Number 5
Article Number 5TOT1

Printer-Friendly Version

SAMMIE: Using Technology for a One-Stop Program Evaluation Resource

Thomas M. Archer
Leader, Program Development and Evaluation
Ohio State University Extension
Columbus, Ohio
archer.3@osu.edu

Karen Bruns
Leader, OSU CARES
Ohio State University Extension
Columbus, Ohio
bruns.1@osu.edu

Catherine A. Heaney
Associate Professor, Teaching
Stanford University Prevention Research Center
Stanford, California
cheaney@stanford.edu

Abstract: Whether evaluating impact of community-based programs is new to you, or you are an experienced evaluator, SAMMIE can help you expand your skills. SAMMIE represents Successful Assessment Methods and Measurement In Evaluation. It is a one-stop Web portal to valuable impact documentation resources. Through SAMMIE you can: [1] Access resources on 21 evaluation related topics; [2] Read the best literature on the Web related to program evaluation; [3] Ask an Expert your questions about program evaluation; and [4] Develop a personalized program evaluation plan. SAMMIE is available free of charge to anyone who has Web access Go to: <http://www.sammie.osu.edu>

A Timely Approach to Extension Program Evaluation Education

The day of the Extension, 2-day, statewide, evaluation in-service workshop has long passed. No, wait--lengthy, evaluation in-services have always been limited! Face-to-face professional development sessions on any topic are becoming antiquated. But the fact is that the vast majority of Extension program personnel have NEVER been overly interested in developing formal evaluation skills.

There are several reasons for this. First, most Extension employees are hired because of their discipline expertise and their interest in teaching. Evaluation is not a part of their academic arsenal, and it may not be as high of a priority when it comes to professional development.

Then there is the ever-present, and sometimes oppressive, issue of budget constraints. Traveling halfway across the state for a 5-hour professional development workshop takes financial resources. Smaller budget mean fewer people, which means those who are left do more--programming, not evaluation.

But a new era exists for those conducting programs with public or foundation funds. No longer can one expect continued financial support just by doing good work. One must now show results. Even though there may be little interest, time, or money to acquire adequate program impact evaluation skills necessary to sustain their "doing good work," more and more Extension Educators are realizing the importance, and even the necessity, of acquiring impact evaluation skills.

Distance Education and the Web have provided alternatives to face-to-face in-services for sharing knowledge and skills. There are millions of resources on the Web, but can program educators afford the time needed to search and verify the legitimacy of these Web sites? For example, a recent Web search on "How to Conduct a Focus Group Interview" yielded 1.98 million hits; a similar search on "How to Design a Questionnaire" netted more than 16.1 million hits.

Portals to the Web

SAMMIE is a one-stop Web site that gives users ready access to impact evaluation resources and tools. As an acronym, SAMMIE stands for Successful Assessment Methods and Measurement In Evaluation.

Whether evaluating the impact of educational programs is new to the potential SAMMIE user, or s/he has evaluated programs and just wants to quickly refresh knowledge on specific evaluation techniques, SAMMIE is the Web portal to help. Visitors to the site are encouraged to do the first six modules in order, but can jump from one module to another based on their specific needs.

As one goes through the modules, the SAMMIE user can:

Access resources. Web-based readings and resources are clustered around the 21 learning modules on topics ranging from ethics in evaluation to designing and using surveys, focus groups, and other evaluation methods.

Read the best and latest literature on the Web related to program planning and evaluation.

"Ask an Expert" questions about program planning and evaluation and engage in discussion forums. These tools provide added information and insight into specific questions the user may have.

Develop a personalized program with an evaluation plan. By using a wiki, colleagues can develop an evaluation plan on-line and open it for feedback and critique from other SAMMIE participants.

Use learning tools, including videos and self-assessments to help expand knowledge.

Learning Modules Available

All these features are available for each of the 21 SAMMIE modules. Below is a list of the module topics:

What Can Evaluation Do for You?

Stages of Evaluation

Getting Everybody on Board

Developing a Logic Model

Asking Evaluation Questions and Setting Standards for Success

Ethical Issues

Evaluation Design I

Evaluation Design II

Data Collection and Analysis Strategies

Quantitative Methodologies--Survey Methodology

Quantitative Methodologies--Questionnaire Design

Quantitative Methodologies--Sampling

Quantitative Methodologies--Descriptive Statistics

Quantitative Methodologies--Techniques for Comparing Groups

Qualitative Methodology--Individual Interviews

Qualitative Methodology--Focus Groups

Qualitative Methodology--Observations

Qualitative Analysis

Interpreting Your Findings

Reporting Your Results

Enhancing the Use of Your Evaluation Results

Potential Use

This Web portal has a multitude of potential uses in program planning and designing/conducting impact evaluations, as well as analyzing, reporting, and using the results of those evaluations.

SAMMIE is available free of charge to anyone who has Web access. Go to <http://www.sammie.osu.edu> to learn more. In just a few minutes potential participants can create an account. After creating an account, participants will want to go through the orientation module, which showcases all the features of the SAMMIE modules and gives an overview of how to use the various features. Check it out!

Acknowledgment

SAMMIE was made available through The Great Lakes Center for Agricultural Safety and Health, The Ohio State University Extension Program Development and Evaluation Unit, The Ohio State University Office of Outreach and Engagement, OSU CARES/OSU Extension, and as the result of 2003-2006 grant support by cooperative agreement #U50OH008108 from Centers for Disease Control & Prevention.

This article is online at http://www.joe.org/joe/2007october/tt1.shtml.

Copyright © by Extension Journal, Inc. ISSN 1077-5315. Articles appearing in the Journal become the property of the Journal. Single copies of articles may be reproduced in electronic or print form for use in educational or training activities. Inclusion of articles in other publications, electronic sources, or systematic large-scale distribution may be done only with prior electronic or written permission of the Journal Editorial Office, joe-ed@joe.org.