Step 5: Monitor progress and evaluate effectiveness

A coalition should begin development of an evaluation plan almost from the outset of its work, and, if not before, definitely within the logic model development and strategic planning process.  From that point, evaluation becomes an on-going process throughout the life of the coalition.

Coalitions should think broadly about evaluation and monitoring progress.  Too often, community groups assume that the only road to evaluation is an investment of thousands of dollars to a professional evaluator who does all the evaluation for the coalition.  While this is one mechanism to incorporate evaluation into a coalition’s work, many other options exist.

Two types of evaluation are relevant to coalition work and environmental prevention: process evaluation and outcome evaluation.

Process evaluation helps measure whether or not the coalition is actually doing the work it committed to in the strategic plan.  Did the coalition chair meet with the city council representative?  Did the marketing committee develop the promised awareness postcard?  Was the postcard distributed to all 5,000 residents, or just 3,000?  Did we recruit representation from law enforcement?

While process evaluation measures whether the coalition actually fulfilled the strategic plan, outcome evaluation measures whether those efforts actually worked as our logic model indicated they would.  In other words, it measures the effectiveness of the coalition’s efforts.  Was underage access to alcohol reduced?  Did drinking and driving rates go down?  Have fewer people arrived in the emergency room with drug-related injuries and illnesses?

From most perspectives – including, typically, those of funders, coalition members, and the community – outcome evaluation is key to a coalition’s ongoing existence.  Coalitions can continue to engage in a variety of activities, but if those activities are not creating the desired outcomes, then the activities lose purpose, and the coalition should revisit its logic model and strategic plan to determine how to shift gears.

While hiring a professional evaluator may be the ideal, many communities cannot or choose not to spend their funding in that way.  Some acquire professional or semi-professional evaluation services without the associated cost by utilizing university graduate students; staff within municipalities, agencies, and hospitals; or evaluation professionals who happen to be members of the community and provide their services as their contribution to the coalition.

Even without a professional evaluator, coalitions can acquire some evaluative data, both on a process and outcome basis.  Some mechanisms by which coalitions conduct evaluation without using professional services:

• Revisiting original community indicators from Step 1 to determine if change has occurred.
• Administering surveys pre- and post-activity to gauge community perception, knowledge, attitudes and behaviors.
• Contrasting change (or lack thereof) between segments of the community that were targeted by the environmental prevention efforts and those who were not.

Coalition evaluation also includes monitoring the internal progress and evaluating the coalition itself.  The coalition may need to disband or change some committees or task forces and possibly create new ones.  Perhaps interest is waning, and the coalition needs to engage in a short-term initiative to spark interest in its efforts again.  A lack of success in a certain area may be due to the lack of representation from a certain community sector in the coalition.  All these steps can result from an evaluation of the coalition and its processes.  A terrific means for this internal evaluation is through a coalition self-assessment.

Some of the tasks involved in the evaluation stage:

• Choose an evaluator capable of monitoring your efforts to make sure strategies are used according to your plan and are having an impact on the targeted substance abuse problem.
• Incorporate evaluation into the strategic plan as a means for measuring the success of coalition efforts.
• Create a feedback mechanism so that monitoring can be used to correct and strengthen your project while it is being implemented.

Continue to Step 6: Sustain efforts
Return to Ohio Center for Coalition Excellence page