News from the Field: ASAP @ RPS Makes the Most of Evaluation
ASAP @ RPS, or After School Achievement Program at Rockford Public Schools 21st CCLC, served over 2,600 students in kindergarten through 12th grade at 18 school sites during the 2013-14 school year. We use University of Illinois-Rockford as our external evaluator and work closely with them collecting data, contributing to reports, using the data for continuous improvement, and sharing the data with our stakeholders and the community.
We collect an amazing amount of data for our evaluation: daily attendance, reading levels, grades, assessment scores, baseline student surveys, student and parent satisfaction and opinion surveys, staff surveys, teacher surveys, discipline referrals, expulsions (both in school and out of school), excused and unexcused absences, demographic information, and more. We collect data on students enrolled in the 21st CCLC as well as students who attend the school where a site is located. This allows us to compare the results of afterschool participants with the entire school. We also compare results of afterschool attendees with those who do not attend. Finally, when we analyze data for afterschool attendees, we compare results among different attendance rates: students who attend regularly (80% of programming, four out of five days a week, or 112 days) with those who attend semi-regularly (30-79% of days or 42-111 days) and non-regular attendees (less than 30% of days or less than 42 days). This allows us to examine the relationship between attendance and the different goals that we set for our program.
Our evaluator works with our 21st CCLC’s project director to produce the final report for each grant, which is sent to the Illinois State Board of Education (ISBE) and to ISBE’s statewide 21st CCLC evaluator. We then compile a report for each individual program site. The individual site report is given to each principal, site coordinator, and the community partner that works with that site. Each partner can share the data and findings with their funders (such as United Way or the Northern Illinois Community Foundation [NICF]) and their own board of directors. Each year, we share the progress report at a meeting of our Advisory and Sustaining Group, which is made up of a representative from each of our community partners and the school district. This year, we invited school board members to attend because one board member indicated he was interested in learning more about the 21st CCLC programs and felt participating in the meeting was the best way to learn.
At the Advisory and Sustaining Group meeting, the evaluator presents a report on the progress that we have made toward our goals. We also present a progress report and a summary of results for each year we have had funding. As Rockford has often had more than one grant serving several sites, we combine all data into one sheet showing totals of students and their demographics for all grants and sites. This includes demographic data, attendance, and program retention for the past 10 years. We also provide an update on each grant, with data on demographics, student achievement, parent satisfaction, teacher survey results, and student satisfaction-followed by site-specific academic results. We include pictures from each afterschool site to help put a face to our students and our activities. Having board members participate in the progress report sessions keeps them informed and enlightened about programs serving students, progress made, and community participation. It also shows our community partners that the district is supportive of the programming.
Our activities and partnerships were even featured in the Rockford Public Schools 2013-14 annual report. As the annual report says, “We cannot foster readiness alone. It takes many partnerships and volunteers to support and nurture our students. We could not do what we do without the support of the entire Rockford community.”
To share evaluation information with parents, we include bits and pieces from our evaluation report in our monthly parent newsletters, always using the data to show parents that those who attend regularly show the most progress. We ask for feedback from parents to help improve programming. We also use the progress reports with our site coordinators, looking at data to see what improvement has been shown, and where we need further improvement, a practice that leads to discussions and planning for continuous improvement. We tackle a different area at each monthly meeting to ensure we continually pay attention to our student and family needs.
We have learned that the community leaders who attend the evaluation report meetings are sharing our findings and using the information to support our programs. We recently received a phone call from the head of youth programs from NICF, who had attended Advisory and Sustaining Group meeting. The person from NICF called with some questions about our progress reports for last year because he was presenting our evaluation information to the board of directors of his organization. We have worked with NICF in small ways for the last 14 years and are now working with them to set up an endowment fund that would support afterschool programming. When community leaders contact us to get additional information about our evaluation reports so they can share information with their organization’s leadership, we know that our strategy of sharing results is having an impact!
Opening meetings about evaluation reports to community stakeholders can be nerve wracking, but stakeholders’ questions are usually to the point and, with the report in hand, you have the answers. If not, you can always say you don’t know and will get the answers back to the questioner. You just have to be sure to keep that communication line open and be sure to respond.