Identifying Program Areas in Need of Support
From LiteracyTentWiki
Back to Assessment Information
Back to Using Data for Program Improvement
Back to Full Discussion by Threads
Identifying Program Areas in Need of Support
Larry and all,
Table 4 and Table 4b (of the NRS Reports) contain a great deal of information, which if analyzed by program, site and class, can help begin to identify strengths and areas needing additional support. For example, these two tables can help program administrators/teachers identify some of the following reasons behind low percentages of level gains:
1. Students aren't staying long enough to post-test. If the total contact hours divided by the total number of students is less than the number of hours required for post-testing, we know we have a retention problem. Now we can look at: the intake process and transition into class; whether instruction is meeting student needs/expectations; whether curriculum and materials meet the varied levels and learning styles/modalities of students; outside barriers that keep students from attending; etc.
2. Students are staying long enough to post-test, but are not being post-tested. (Compare the number of students on Table 4 to the number on Table 4b.) If this is the case, we can look into whether this is a staff problem - i.e. they have no sytem to know when a learner has enough hours to post-test; they need assessment training/support; etc. - or a student problem. In some programs we have found that when students learn they will be taking a post-test, they stop coming for a period of time. In this case, we need to help students understand that we are assessing our effectiveness, not their intelligence.
3. Students are being post-tested, but not making level gains. (Compare the number of students on Table 4b -i.e. those who have been post-tested - to the number who show a gain on that table.) If this is the case, we can begin to look at instructional strategies and curriculum/materials.
There is one other possibility that I can think of. Students may be post-tested and making gains, but in some cases there is a glitch in the paper flow and for some reason information is not being submitted for data entry or data entry is flawed. If you have a system for submitting class data back to each teacher for review, they can help you identify if this is the case and where the glitch may be taking place.
In short, I love the NRS reports because they tell you so much about your program if you take time to analyze them. I hope I haven't taken up your time telling all of you things you already knew. I'm looking forward to hearing some great new ideas this week.
Barbara Hofmeyer
Coaching Consultant for
Indiana DOE, Division of Adult Education
Barbara,
I like your thinking!
What we advise is to monitor periodically (monthly or quarterly) program pre-posttest rates and retention factors. In this way you can see if a program has had a problem in time to intervene to explore the problem and correct it.
In our Data Detective training we illustrated how to use data as the starting point for exploring what might be happening in your programs. As you note in your example below, low educational gains may be due to low retention, program staff not administering the posttest or students just not making gains. Data can help you pinpoint which of these three problems may exist. Once you know this, you can dig deeper to look for the underlying issues on which you can direct program improvement (such as changes in class schedules, better support and training for assessment or changes in instruction that may be needed).
Larry Condelli
