This blog post is part 5 in a series exploring the use of an HRA in developing a health intervention plan using research from Wellsource student intern Kori Pitt. In this example program, Kori outlines an example workplace obesity intervention program called the Triple W Challenge. Catch up on the series:

• Part One: Process Overview
• Part Two: Assessing Your Workplace Health
• Part Three: Planning Your Program
• Part Four: Implementing Your Program

Once the assessment and planning steps are complete and the implementation plan has been selected, the final step toward creating an obesity intervention plan is deciding how you will evaluate the program activities.

Monitor and Evaluate

The evaluation of your program offers insight into what was achieved, what could be improved, and if the program made a difference. Evaluating the program will determine effectiveness, future planning efforts, and if costs should or can be modified or reduced.

The perceived value of a workplace intervention depends on the level of corporate investment, hinging on three dimensions: significance, worth, and merit. These dimensions will vary from organization to organization.

That is why the framework includes engaging stakeholders within the first steps of program development. This allows identifying the different preferences and needs of the stakeholders early on to ensure proper assessment and evaluation.

The framework for evaluation contains 6 steps:

1. Engage stakeholders: Stakeholders are individuals and organizations that have something to lose or gain as a result of the program. Any evaluation must take into consideration the needs and values of these partners so that their perspectives are understood. These individuals and organizations include those that are involved in program operations and those that are affected by the program.
2. Describe the program: Describing the program involves explaining how the activities and resources needed will bring about the outcomes of the program. This step involves creating a logic model, or flow chart, that details how the inputs (resources) and activities will accomplish the desired outcomes.
3. Focus the evaluation design: A systemic approach to deciding what the evaluation design is to focus on. This step involves identifying all the evaluation details that will lead to a successful evaluation plan and what steps are needed to get there. Focusing on why the evaluation is being done in the first place will guide your strategy for measuring success. Determine what the program effects should be, how to affect the participants of the program, how to gain insight into how program activities should be done, and how to improve those activities.
4. Gather credible evidence: Gathering evidence, or data from program and evaluation activities, strengthens the results of the evaluation and establishes credibility. Stakeholder involvement in gathering and defining data encourages them to act on evaluation recommendations and increases the chances of stakeholders accepting evaluation conclusions.
5. Justify conclusions: Evidence from the evaluation can’t be justified in isolation. Values set by the stakeholders need to be considered when justifying conclusions.
6. Ensure use and share lessons learned: Evaluation findings need to be used appropriately and dissemination of findings should be done in an appropriate and contextual manner. Transparency about all aspects of the evaluation is necessary during this step.

Triple W Challenge example:

The objectives set earlier in this blog series will guide the evaluation process, as they describe what the program is trying to accomplish. Based on the objectives, measures will need to be developed to tell what’s really going on. Assemble before and after statistics on the chosen indicators.

In this example, by the end of the 24-month intervention, 90% of participants will exhibit mindfulness skills at least 2 times per week when feeling stressed. Some measures might be to assess mindfulness before and after the intervention (using mindfulness scales), the number of times the participants felt stressed per week, and the number of times they used mindfulness techniques to combat feelings of stress.

Collect data on these measures through surveys or focus groups to see if the program is directly influencing the expected outcomes. Focus groups on the text messaging system can provide feedback on how to change or improve text messages for more effective behavior change modification. Assessing the HRA and readiness to change scales over time is another useful tool for measuring changes within your target population. Use the results to adjust the program as necessary. Are the objectives being met as planned? If not, the data should point to what changes need to be made. Evaluation is complex but is crucial to inform the success of the intervention program.

Meet Kori Pitt, MPH

Kori Pitt obtained her BS in Public Health at the University of Nevada, Reno in 2013. She recently graduated with her Master’s in Public Health from the University of Arizona. Kori’s program concentrated in health promotion, where she examined health behavior and developed skills in implementing and evaluating health interventions on a population level.

Kori's internship at Wellsource involved performing research on lifestyle behaviors affecting obesity in working females. She performed statistical analyses on health risk assessment data and created an evidence-based workplace intervention framework for obesity. Kori is passionate about chronic disease education and prevention and bridging the gap between public health research and practice.