EbaeEffectiveModel

From LiteracyTentWiki

How to Know if a Model is Effective

• “We look at three areas [in our [program]; number of GED passed or in progress; number of accommodations for folks with disabilities and retention. Since we have been doing this, our program has become more focused and more people are graduating overall. Our ultimate measure is whether people pass the GED after being in our program. Since we really began monitoring our GEDs, our numbers have risen tremendously. The rise in GEDs is our measure, but you have to be careful. Adult education is not like K-12 where we have everyone start at 5 years old and end at 18. I think you need to look at what we can do to increase the number over the long haul. Second, since we work with learning disabilities here in San Diego, we have been maintaining about 15% of our GEDs taken with accommodations. This is measuring that we are serving those in our population who need these services. Third, retention. Students come if the program is working. They vote with their feet. “ [Message 2, Part Three]

From John Comings:

• “A specific example might help focus the discussion. One subpopulation might be ASE/GED students, whose reading skills are sufficient to pass the requirements of their ASE program or the GED and whose goal is a job with good pay and benefits. Research suggests this goal can only be met if students are encouraged to build their skills and knowledge so that they can attain a higher score (GED score predicts income) and helped to transition into postsecondary education or training programs (adults with two years of postsecondary education or training are more likely to attain good pay and benefits). A program model that would help students in this subgroup reach their goals might: Intake and Orientation: help students understand the steps necessary to reach their goal and work with them to develop a learning plan that fits with the demands of their lives. That learning plan would outline the instructional and support services they need to be successful. Instruction: Most ASE/GED classes are a combination of individual study, small group work, and whole class instruction. This approach developed as a response to mixed enrollment and open entry/open exit classes. With well-defined curriculum goals (the ASE requirements or the GED test) and an acceptance of the needs of this subpopulation that lead to mixed enrollment and open entry/open exit classes, development of an effective model of instruction that draws on the experience of good teachers should not be difficult. Adding a self-study component that utilizes the recent development of technology applications should not be difficult either. Since these students will have post-secondary education as part of their goal, this curriculum would have to be augmented with content (such as academic vocabulary) that would provide the skills and knowledge needed to be successful in postsecondary education and training. Support Services: ASE/GED programs usually cannot provide direct support services (such as day care) but they can provide counseling and assistance to students so that they can arrange those services for themselves or find ways around these life barriers to participation. Programs can help students develop the emotional support they need by establishing a program climate and procedures that build support among students and between staff and students. Transition: Programs that help ASE/GED students make the transition to postsecondary education do focus on skills and knowledge, but they also focus on establishing a support structure for students once they enter postsecondary education. The building of this support structure begins in the ASE/GED program but should continue after the transition.<br? Cost: A program like this might cost $1500 to $2500 per student. Evaluation: An evaluation of a program like this would have to include a longitudinal component that followed students for 3-5 years. A random assignment experiment or a treatment and comparison group study could look at this model vs the existing model that costs quite a bit less. I believe the evaluation would show that this more expensive model is cost-effective. With this data in hand, our field could argue for policies that support this kind of program for this population. If someone comes up with a good idea that might improve this model (a new math curriculum for example) or make it less expensive (a new technology tool for example), it could be tested against this accepted model.
Other subpopulations (adults with very low literacy skills or those seeking education to help their children in school for example) may be more difficult to fit into this approach, but starting with the easiest subpopulations might make model evaluation for these subpopulations easier. Do you think this would work? How might you do it differently? [Message 6, Part Three]

• [Reply to Message 6: Part Three]

“….all you propose here makes sense, but the funds aren't in place to implement and evaluate programs in the way you suggest, or the support for professional development and the respect for the fact that teachers do learn and want to learn is absent in many adult learning programs….. yes, you've proposed a fine model; yes, programs likely incorporate many of the elements you describe - if not all of the pieces you propose, but no, we can't seem to bridge the gap between the kinds of evaluation and analysis that Big Research seems to demand of us, while ongoing small and participatory measures tell us that people are learning.” [Message 7, Part Three]

From John Comings: • [Reply to Message 7: Part Three] “Thanks. Yes this is what good programs look like and I'm suggesting that we do the "big evaluation." Once that evaluation is done and identifies impact, which I'm confident it would, then the field would have the evidence it needs to justify funding for this subgroup and accountablity could move from measuring outcomes to measuring whether or not programs conform to this model of good practice. Future research for this subgroup would take place within programs that conform to this model.” [Message 8, Part Three]