|
Baseline Data for Your Program?
Nancy Ellen Kiernan
Program Evaluator
Penn State University
University Park, Pennsylvania
nekiernan@psu.edu
Abstract:
Collecting baseline data is an integral part of
evaluation. Understanding how different Extension programs are designed and
delivered can suggest different sampling plans to collect baseline data.
When considering an evaluation strategy for a state or multi-county program,
one of the initial steps program planners should consider is how to collect
baseline data. Baseline data establishes a basis for:
- Comparing what was happening before the program with what happens after
and
- Extension to claim credit for impact.
When program planners have a state or multi-county perspective, they have
the opportunity to develop a broader based sampling plan to collect baseline
data. Sampling can reduce the burden on Extension educators and program participants,
yet provide a representative profile of the target audience.
How program planners should sample can depend on the design of the program
and how the program is delivered. Below are two typical Extension program models
and a sampling plan for each. Choose the model closest to your program, and
consider the sampling plan.
Extension Program Model #1
Program Characteristics
-
The target audience is typically limited in the area, perhaps 100-900
persons.
-
Over a 2-3 year period, a series of educational activities are conducted
for this target audience.
-
The educational activities feature different aspects of the subject matter.
Each educational activity in the series has new educational objectives
related to immediate impacts, but taken together across all activities,
the objectives combine to achieve an extended impact. In a calf and heifer
program, for example, the first educational activity may center on calf
housing, the second on calf nutrition, the third on calf health, but all
are focused on reducing calf mortality, the extended impact.
-
The series of educational activities is directed to the same specific
individuals in the limited target audience (for instance, dairy farmers
in a multi-county area).
Figure 1.
Extension Program Model #1. The diversity of symbols indicates various parts of program over time that, added together, comprise one large program. It indicates a the lack of repetition of the same program.
-
"Complete" learning of all skills, knowledge, and practices
within the Extension program experience by the participants does not occur
in a short time span, as in some programs, because the educational activities
are spread over time.
-
The target audience is limited enough so that there is likely to be contact
information available, for instance, swine producers or childcare providers.
This list provides the basis for sampling.
Examples of This Type of Program
-
Agronomic Production and Management Practices (Covers variety selection,
planting dates, rates and methods, plant protection practices, fertilizers
practices, harvest and storage practices)
-
Tax Reform for Elected Officials
-
4-H Volunteers
-
Dairy and many programs in agriculture
How to Collect Baseline Data
Before the program, collect baseline data by randomly sampling the target
audience and measure their knowledge, attitudes, skills, practices, and relevant
demographics. After the series of educational activities, return to the random
sample; ask the same questions to evaluate the program's impact; then generalize
to all program participants.
Advantages for Extension
-
No short-term or on-site evaluation for impact has to be conducted during
educational activities, reducing the evaluation burden on county educators
and participants.
-
Collecting baseline data before a program will highlight which topics
need emphasis and which should be dropped.
Caveats
-
Track who participates in the programs so you will have a measure of their
exposure or involvement in the program. This is not difficult. (See http://www.Extension.psu.edu/evaluation/pdf/TS68.pdf)
-
Over-sample when gathering baseline data. Some participants in the baseline
study may not participate in the program or might not be in the post-program
evaluation.
-
Over a long duration, other factors besides Extension influence participants;
a control group selected from a group not receiving the program will provide
a measure of those influences.
Extension Program Model #2
-
Program Characteristics
-
The target audience is typically a large group, perhaps thousands, in
the area. Many of these programs are for the general public.
-
The educational program will take place within a short duration (1 - 6
wks).
-
The educational program is repeated over and over again among groups
to reach a large percentage of the target audience. Over 2-3 years hundreds,
perhaps even thousands, of persons will participate in this program.
-
In Figure 2 below, the educational program is repeated 9 times within
the 2-3 year period, but could be repeated more.
Figure 2.
Extension Program Model #2. The square boxes indicate repetition of the same program over time. Each box is independent and separate from the other, just like a program that repeats itself many times.
-
"Complete" learning within the Extension program experience
takes place within a relatively short duration.
-
The target audience is so large it is unlikely there would be a contact
list, which necessitates a different sampling approach for baseline data
than in Model #1.
Examples of This Type of Program
- Water Quality Workshops
- Cancer Prevention
- Estate Planning
- Pesticide Applicator Training
- Health and Nutrition
How to Collect Baseline Data
Before the program begins, consider first, the number of times the educational
program will be repeated. Second, randomly select from which of these programs
you will collect baseline. In the model above, you might randomly select the
3rd, 6th, and 9th times the program is conducted.
Over time, you want at least 200-300 participants, depending on your analysis
and other factors. Once you figure the number of participants for an analysis,
it is easy to figure the number of programs at which you will need to collect
baseline data.
To evaluate the impact of the programs, go back to the same participants with
a post-program measurement at the appropriate time.
Advantage for Extension
- Baseline (and later, evaluation) data do not need to be collected from
everyone, but only from participants in randomly selected programs. The burden
on county educators and program participants will be reduced dramatically.
Caveats
-
If the program is conducted for diverse groups, be sure to consider this
when sampling.
-
Over a long duration, other factors besides Extension can influence participants;
a control group selected from a group not receiving the program will provide
a measurement of those influences.
Conclusion
There are many aspects to an evaluation strategy, of which collecting baseline
data is an initial step. Understanding how different Extension programs are
designed and delivered can suggest different sampling plans to collect baseline
data. Planning how to collect baseline data for a state or multi-county program
before it is launched can reduce the burden of collecting data for Extension
educators and program participants.
This article is online at http://www.joe.org/joe/2006june/tt6.shtml.
Copyright © by Extension
Journal, Inc. ISSN 1077-5315. Articles appearing in the Journal become
the property of the Journal. Single copies of articles may be reproduced
in electronic or print form for use in educational or training activities.
Inclusion of articles in other publications, electronic sources, or systematic
large-scale distribution may be done only with prior electronic or written
permission of the Journal Editorial Office, joe-ed@joe.org.
If you have difficulties viewing or printing this page, please contact JOE Technical Support.
|
