Afterschool Focus: Using Your Local Evaluation for Continuous Improvement

Your local evaluation is more than a 21st CCLC grant requirement. It is also a powerful tool to strengthen the quality of your program, inspire staff, improve professional development, support improved student achievement, engage partners, and lay the foundation for program sustainability. 

Through your local evaluation, you can ask and answer important questions about your 21st CCLC program and the students you serve. When we evaluate a program, we want to find out what is working for our students and families and what changes we can make to produce desired outcomes. As Marion Baldwin from the Illinois Quality Afterschool team said, “Continuous quality improvement means you are constantly seeking to do a good job today and an even better one tomorrow.”

	Source: Beyond the Bell®

To prepare your team for “doing an even better job tomorrow,” think of your local evaluation as part of the continuous improvement process. 

• First, develop afterschool programming using sound processes.
• Then, implement programming by employing quality practices.
• Finally, evaluate the implementation of the afterschool programming. Use the data and recommendations from your evaluation report to improve your program offerings and assess progress toward program objectives.

 

Planning Your Evaluation

Using your evaluation for continuous improvement begins with planning. Although the evaluation template and required components are a starting point for your evaluation,¹ you should also work with your evaluator to plan an evaluation that aligns with your program goals. Doing so may mean including additional evaluation questions and collecting additional data. You may also want to consider afterschool quality standards such as the Act NOW Illinois Statewide Afterschool Quality Standards. If your 21st CCLC program has an evaluation from the previous year, review the recommendations and plan ways to track progress toward addressing those recommendations. These extra steps will yield evaluation results that provide a more detailed and robust picture of your 21st CCLC program and, in turn, you will be more prepared to make progress toward meeting program goals. 

When planning your evaluation, think about how you want to present your results. Talk to your evaluator about the report format that would work for your program. Do you plan to share your evaluation results with different partners? Your evaluator might be able to create short summaries for different audiences, like families, school leaders and staff, community partners, and potential funders. Think about what these groups might want to know about your program and work with your evaluator to find ways to present these findings in meaningful and engaging ways. Keep an open mind throughout the evaluation process as your ideas about how best to share results may shift after you have findings in hand. 

 

Using Your Evaluation for Program Improvement

Your evaluation results can help you identify and communicate your program’s strengths and reflect on what needs changing for improvement. To use your local evaluation for improvement, you will want to engage your 21st CCLC team in a continuous cycle of inquiry and reflection of all program elements. 

Plan to set aside at least 2 hours to discuss evaluation results and prioritize findings to address. Some evaluators can help present findings; however, you will want to discuss expectations early in the process. When presenting findings, be sure to stress to your staff that the recommendations are an opportunity to improve the program and not personal. Creating that expectation will help staff feel more comfortable participating in the discussion. Then ask your team to review findings and ask the following questions about what they see in the results:

• Does the challenge require a short- or long-term fix?
• Is this something our 21st CCLC can address?
• What is the level of importance? (high, medium, low) 

You may also want to divide staff into small groups to review findings by site so that continuous improvement changes are tailored to each site. After your team has prioritized findings to address, outline action steps, develop a timeline, and identify lead staff for each item. The Beyond the Bell Tool 92: Post-Evaluation Planning Tool provides additional information on leading your 21st CCLC team through this review and planning process. 

Remember to ground conversations about your evaluation in findings and data rather than speculation and stories. Sophia Mansori, a research scientist at the Education Development Center, the organization that conducts Illinois’s annual 21st CCLC evaluation and provides technical assistance to grantees on conducting evaluations, stresses the importance of focusing the conversation on the evaluation. “It is important to think about making improvements based on data, especially if a program is not seeing the outcomes or changes they hope or expect.” Likewise, Mansori suggests program stakeholders think about what the data do not tell them. “It may be that data [do] not provide a complete picture, in which case a grantee should think about what new types of data they should collect in the future to inform their evaluation,” she says. 

You will also want to share your evaluation results with external partners. “Sharing evaluation findings is a great way to engage stakeholders—including teachers or school administrators, parents, and community partners. Engaging stakeholders can help programs identify strategies for program improvement,” says Mansori. 

When you share your evaluation results, we encourage you to highlight and celebrate success but also be transparent about the challenges your program has faced and how your team plans to address them. Your partners likely know that afterschool programs faced additional challenges during the COVID-19 pandemic. Include those challenges as part of your presentation, as well as ways your program has met those challenges. As Mansori notes, your partners may have additional ideas and resources to support program improvement. Beyond the Bell Tool 94: Sharing Evaluation Results provides additional ideas on sharing evaluation findings with different stakeholder groups. 

 

Remember the Improvement Cycle

Armed with evaluation results and improvement plans, your team is ready to continue its improvement cycle by planning or revising programming, determining how you will measure success, implementing programming, collecting data, and reviewing results. Don’t wait until you receive your next evaluation results to reflect on your program’s achievements. Make this an ongoing part of your work. 

Building reflection into your program helps create a culture of continuous improvement. If staff members know that there will be reflection time at every staff meeting, it becomes an expected and natural part of team meetings. The same is true for advisory board and partner meetings as well. Continuously ask “What’s working?” and “What’s not working and why?” 

Leaders who are excited about the culture of improvement can help create an environment for staff to do their best work and build a culture of success. Leaders can foster an atmosphere in which staff members recognize data collection and analysis as opportunities to increase their marketable skills and professionalism AND result in greater success for the students they serve. 

 

References

Act Now! Afterschool for Children and Teens. (n.d.). Statewide quality afterschool standards. https://actnowillinois.org/quality-standards/

American Institutes for Research. (2021). Beyond the Bell – Live! https://beyondthebell.org/online-home (Free registration required)

Beyond the Bell. (2019). Data: Your continuous improvement reality check [Webinar]. https://www.youtube.com/watch?v=3CQi16Vl7XU

Education Development Center. (2021). Launching your evaluation. Presentation at the Illinois Quality Afterschool FY2022 Virtual New Grantee Orientation. https://www.youtube.com/watch?v=ZxT8-aSjdjQ

 

Footnotes

¹See the “Forms” tab on the Illinois State Board of Education 21st CCLC webpage for the local evaluation template for your program’s funding year.