Ebae New Additions to the Discussion
From LiteracyTentWiki
In a message posted by
Donna Martinez <Dmartin336 at aol.com>
Wednesday, January 05, 2005 7:42 AM
To: aaace-nla at lists.literacytent.org with the
Subject: Evidence on "What Works": An Argument for Extended-Term Mixed-Method,
she suggested this article by researcher, Madhabi Chatterji:
Evidence on "What Works": An Argument for Extended-Term Mixed-Method (ETMM) Evaluation Designs http://www.aera.net/uploadedFiles/Journals_and_Publications/Journals/Educational_Researcher/Volume_33_No_9/02_ERv33n9_Chatterji.pdf
The article begins:
"Federal policy tools for gathering evidence on “What Works” in education, such as the What Works Clearinghouse’s (WWC) standards, emphasize randomized field trials as the preferred method for generating scientific evidence on the effectiveness of educational programs. This article argues instead for extended-term mixed method (ETMM) designs. Emphasizing the need to consider temporal factors in gaining thorough understandings of programs as they take hold in organizational or community settings, the article asserts that formal study of contextual and site-specific variables with multiple research methods is a necessary prerequisite to designing sound field experiments for making generalized causal inferences. A theoretical rationale and five guiding principles for ETMM designs are presented, with suggested revisions to the WWC’s standards."
Posted here by
David J. Rosen
djrosen1@comcast.net
January 7, 2005
For a somewhat different take on what constitutes valid science, my essay, Postpositive Scientific Philosophy: Mediating Convergences, may be of interest. The essay can be accessed here: http://www.the-rathouse.com/Postpositivism.htm
(It looks like you have to cut and paste into your browser).
In highlighting the matter of scientific philosophy, I'm raising the issue of epistemology (how we can come to know) on the claim that how we can come to know is at least partially (if not totally?) shaped by the mental models (paradigms, if you will) that undergird our assumptions.
In positing postpositivism (and there is a scholarly school of thought upon which this is based), I'm operating out of several hypotheses:
a) That the focal point of an investigation is a problem orientation of significant enough scope worthy of the time and resources needed to work toward a reasonablly provisional resolution. Obviously, what constitutes a "worthy" problem and a "provisionanal resolution" are debatable matters. Still, the point holds.
b) A scientific-based approach to a cultural phenomenon like adult literacy education is based on a quest for truth as a regulative ideal. It is the quest for truth that separates postpositivism from constructivism and/or postmodernism. It is the ideal nature of the quest, even to the extent of being defined as a creative fiction (pace, Karl Popper), that separates it from positivism.
c) That in a problem-focused aproach to scientific investigation, methodologies are drawn upon as relevant to the study. On this assumption, there is no hierarchy (gold standard) of methodologies even as the nature of the study needs to be highly competent according to the disciplinary canons that undergird the study.
In my discussion of three representative postpositive scientific philosophers (John Dewey, Karl Popper, and Nicholas Rescher) there is a certain abstract nature, which I find unavoidable in order to discuss scientific philosophy. To help to make the paper more relevant to adult literacy, I illustrate the various theories highlighted in the essay to the discussion on the relationship between learning how to read and write and the broader development of literacy defined as meaning making and knowledge acquisition through the appropriation of print-based text. This, I contend, is a problem area of sufficiently important scope to require some type of provisional resolution. The concluding chapter includes four hypotheses and 19 supportive statements that seeks to set out a framework for an extended research project based on a blended definition of literacy as "a duly hedged syntheses" of progressive reading and writing mastery and knowledge acquisition.
No doubt the essay leaves more gaps and questions than it's able to resolve, and in trems of a specific research agenda, I am seeking to be suggestive rather than definitive. My more fundamental purpose is to establish a sounder epistemological framework for educational research via postpositivism that mediates the critical space between the postitivist and constructivist/ postmodern research traditions.
George Demetrion gdemetrion@msn.com January 7, 2005
From::dwyoho@earthlink.net
Subject: [AAACE-NLA] Toward a True Dialogue
To: aaace-nla@lists.literacytent.org
Date: February 7, 2005 11:16:56 AM EST
Andrea, et.al,
You've invited me to talk further about the "science question", so I shall offer a few ideas. But I feel this forum by its very nature severely limits discussion of something so complex.
When I was a high school principal, I learned (in fact I was told) that success lay in "finding the greatest good for the greatest number". Now this isn't a bad premise, and I think it is the premise that motivates a lot of scholarship in education. As a principal, I spent a lot of time reading published scientific studies in search of this elusive formula. In the end, the guidance I found in these studies seemed to "prove" what to me seemed more like common sense to begin with. For example, classroom climate, the power of role models, the efficiency of direct instruction, and the importance of mastery have all been well-documented by eminent researchers. Yet I already knew that these things were important even before I was trained as a teacher. I experienced them, you see, as a student. At the time I wasn't thinking about the ways I could create these things. I just looked for them as I chose my own course of study. Later as I considered published research about these things my study had a profound influence on me professionally. I think perhaps I was affected so much because I had found "legitimacy" for what I already "felt" to be true.
Now I recognize a lot of dangers were/are inherent in this process. Certainly this is not empirical science. However, as I continued to practice as a teacher I learned something else: always, in every circumstance, reserve judgement (decision) until there is enough evidence to reasonably assure me that I am on the right course. And where does this evidence come from? Why, from the students, of course. And here is where I believe I do have "overlap" (Andrea's word) with science. So long as my mind remains open and I consciously look for evidence, am I not doing the same things an empiricist does when an hypothesis is offered and then tested?
But on the other hand, I am certain that this process of study-apply-grow cannot be "science", because I also learned that I cannot ever guarantee that "what works" with one class or learner or school can be "duplicated". Here is where some scientists lose respect for me. S/he thinks that if I can't repeat one success with another, using the same tool, then whatever process I am using must not be legitimate.
Here is where values come in. Now the battle is joined! If one approach doesn't work, I do not reject it as illegitimate. I put that tool back into the toolbox and use another, saving the first tool for another application at another time. The more research I read, the more tools I add, and with experience, I become better and better at matching the right tool with any given set of circumstances, under conditions that constantly change.
In talking with doctors and medical people, it eventually dawned on me that this metaphor of a "toolbox" and this seemingly trial and error process is exactly what physicians do every day, although few would say so out loud. Because both doctors and teachers are in effect "tampering" with a human being, we don't like to think about the risks we take, much less talk about them. In fact, we spend copious energy trying to minimize the risks i.e. we research and read and talk and question and never stop learning. We are always "practicing".
How do I know I am effective? The same way doctors do. The learner/patient, if s/he can, tells me I am, in many, many ways. But sometimes I really don't know exactly what I have accomplished.
What I think is happening in this political climate is that one school of science, educational psychology and neurology, has provided, perhaps unwittingly, "legitimacy" to a set of values about teaching and learning that was already perceived as "common sense" by a group of people who now happen to have power. Well, I did/do the same thing. But the difference is I learned to RESERVE JUDGEMENT, and keep looking for more tools for my toolbox. So long as I continue to do that, I think I am on common ground with most scientists. It is not science that is the problem. It is how that science is used. And isn't that always where the rub is?
It bothers me that informed, careful trial and error is perfectly legitimate in science, but now rejected as "fads" in education. It bothers me that years and years of published research has been written off as merely self-serving gibberish.
When I was in Washington I suggested at the last NIFL meeting that their website include a prominent section (perhaps a compendium) of seminal research in education. I wonder sometimes if the current decision-makers have ever heard of Dewey or Maslow, Bloom or Mager or Hunter. I hope this will be considered.
For the Cause! Debbie
Deborah W. Yoho
Co-moderator, NIFL-Health Listserv
Executive Director, Greater Columbia Literacy Council
Past President, SC Adult Literacy Educators
2728 Devine Street, Columbia, SC 29205
803-765-2555 Fax 803-799-8417 dwyoho@earthlink.net
Scientific-based Education Research Discussion (continued) Scientific-basedEdResearch
