Scientific-basedEdResearchPart2

From LiteracyTentWiki

Discussion Continued, from Scientific-basedEdResearch

Message Thirteen
2-1-05

I want to thank Cecil Smith for shedding light on Paula and Keith Stanovich. I am pleased to see NIFL selected eminient researchers for this work. I can easily see how the document would be useful for students in training. However, it is designed for practicing educators.

My own training, more than 30 years ago, certainly included a grounding in understanding and interpreting all kinds of research, at least to the extent that I have been able to assess the value of any given report as related to my own teaching. When I don't understand, I know how to find out what I need to bridge the gap. I cannot say whether or not my training was/is typical, but as a high school principal most teachers I worked with had these important skills. I freely admit my own over-sensitivity to language, even defensiveness, any time teachers are all painted with the same brush, but I still think the overview and introduction to the document are simplistic. That is not to reject the work entirely, and as I said in my initial post, I found it useful overall.

For the Cause! Debbie

Deborah W. Yoho
Co-moderator, NIFL-Health Listserv
Executive Director, Greater Columbia Literacy Council
Past President, SC Adult Literacy Educators
2728 Devine Street, Columbia, SC 29205
803-765-2555 Fax 803-799-8417 dwyoho@earthlink.net

Message Fourteen
2-1-05

I'm a bit reluctant to engage in a George-and-Cecil discussion in this public forum, but want to offer a couple of responses to George that I hope might facilitate further discussion among the larger audience here.

First, George, I read your review of the Stanovich and Stanovich paper. Having had the benefit of reading your always interesting and insightful comments in this forum over the past years, I feel that I have a good understanding of where you stand in regards to the positivist tradition as it applies to education. I'm probably a bit more aligned with the Stanovich and Stanovich perspective, having been trained in and largely engaging in a decidedly positivist, quantitatively-oriented program of research. On the other hand, I share your concerns about alignment of education with medical science as if that field is somehow the "gold standard." Apparently, it is to Russ Whitehurst. There is certainly a strong appeal--on an emotional level--to embarce the medical model; after all, medicine deals with issues of life and death. Powerful stuff.

Nonetheless, I think that there is real value in bringing scientific rigor to the study of educational programs and practices. I tend to agree with Stanovich and Stanovich that educational practices have largely been driven by an "anything goes" mentality. By this, _I_ mean that teachers are always looking for better ways to improve their practice and increase students' performances and achievement. And there is, sometimes, a tendency to jump onto bandwagons and to follow fads. There's a pretty long history of this in American education.

Two recent examples suffice here. The first is multiple intelligences theory which has sparked an array of "innovations," how-to books, instructional practices, curricula, and even schools based on Gardner's theory. None of this stuff, to the best of my knowledge (and I am always looking for the evidence) has been subjected to any kind of empirical test. Of course, multiple intelligences theory is not a theory of education or of curriculum; it is a theory of individual differences in intellectual functioning, or qualities of thinking perhaps. It may have some implications for teaching--although, in my mind, a bit of a leap from theory to practice. This hasn't stopped an entire MI industry from springing up. I don't know of any studies that show that students in MI schools have significantly higher achievement than those in comparable non-MI schools.

Somewhat related is the more recent "brain-and-education" movement which purports to translate findings directly from the neuroscience lab into applications for classroom practices. Again, there is an industry emerging--professional development workshops, "pop" psych books, instructional materials, curricula and so on based on this loosey-goosey "translation" of neuroscience into classroom instruction. But, not much scientific evidence that instruction based on neuroscientific findings has any meaningful impact on student achievement.

I'm a little uncomfortable with the idea that procedures and curricula are adopted because they seem intuitively sensible or align with teachers' judgments that some program or practice does work (too much potential for bias). I am a parent representative to my local school district's curriculum committee and so I've seen first-hand over the past 7-8 years the kinds of activities and thinking that goes into curriculum revisions and adoptions. The teachers and administrators who make up the committee are excellent--informed, caring, intelligen--educators trying to make difficult decisions under often-trying circumstances. But, by and large, curricula are revised and adopted by upon hunches, prevailing trends, deep desires to do "something different," and--to much less extent--a solid grasp of the available scientific evidence (which often isn't available because it doesn't exist).

I'd write more but I've gone on too long and I promised to meet my wife for lunch today...

Cecil Smith

M Cecil Smith, Ph.D.
Professor of Educational Psychology
Northern Illinois University
DeKalb, IL 60115-2854
(815) 753-8448
(815) 753-8750 (FAX)
mcsmith@niu.edu
www.cedu.niu.edu/~smith/

Message Fifteen
2-1-05

I don't dare to speak on behalf of Gardner, but when I read what you wrote about MI theory below, I couldn't help but jump in! Many years ago, I organized a fireside chat for graduate students at AERA and Gardner was invited to talk to us. He shared that he was dismayed at what was being done with his theory. By publishing his theory, he was sharing his views which he believed made sense and hoped that someone would one day empirically test. He never meant for schools and instructional approaches to assume that it was fact. He felt that it was premature. He viewed himself as a theorist, and hoped others who viewed themselves as empriricsts would test his theories out before educators would jump and grab onto his theory.

Daphne Greenberg
Georgia State University

Message Sixteen
2-2-05

Although the "medical" model is often referred to when talking about the "gold standard" for research, this is misleading. There is not a single "medical model" of research. While it is true that a great deal of progress has been made through controlled studies of treatments, this is not the only type of medical research that takes place. For example, changes in knowledge about the effects of smoking have not come about because scientists gave a study group some cigars while preventing a control group from smoking, but because of correlational studies. And these studies are so powerful that at this point I do not think anybody would argue for spending time, money and lives on conducting a quantified and controlled study called "Does smoking have a negative impact on health?"

So if education is to look at medicine for research standards, we should look at the wide array of medical research techniques, and not limit it to one type of experimental design.

Erik Jacobson

Message Seventeen
2-2-05

Erik and all

Is it possible that when we rail against the terminology (medical model) we're not so much critiquing the research, per se, but the notion that adults learning language and/or literacy are somehow broken and in need of repair?

Janet Isserlis

Message Eighteen
2-2-05 Hi all, yeah, I also can't go with the medical model analogy, even with Erik's point about different types of research.

I don't really understand why this comparison seems to hold any water although it is pretty clear that our field is grasping for some sort of framework. But why medicine? The notion isn't even about "preventative measures" which at least would make a little more sense to me than comparing educational research with medical research.

Like Janet, I find this to mean that there are a bunch of people with bad literacy health out there - the deficit model.

Why are we not looking to other disciplines for some of the frameworks we're seeking - like the social sciences. It seems to me that the medical model is only a gold standard for itself - the medical model. I'm left with fruit and meat - not apples and oranges with all this. marie

Message Nineteen
2-2-05

It is in fact the medical model that was used in the 2002 ESDOE Strategic Plan in which other forms of research--no doubt those stemming from the anthropological sciences--were linked with "fads" and concerns about the "guru principle."

There's more than a little politics undergirding the Bush administration's use of the medical metaphor--the doctor is to the patient what the teacher is to the student-the undisputed expert who dispenses the right treatment in just the right dosage.

George Demetrion

Message Twenty
2-2-05

Aaace-nla colleagues: The discussion of the medical metaphor for adult literacy education reminded me of the informal research I did several years ago on metaphors for adult literacy education. The Psychotherapy metaphor, a medically related metaphor, was agreed to by 80 percent of respondents. The Medical metaphor was agreed to by 56 percent of respondents. The Research Note of 10/2/99 is re-posted below. Tom Sticht

Metaphors & Analogies In Adult Literacy Education

There seem to be several dominant metaphors and analogies underpinning work in adult literacy education, either explicitly or implicitly.

In workshops conducted in Canada and the United States participants were asked about eight metaphors/analogies that seem to be operating in adult literacy education. They were asked to rate each metaphor as to whether it was totally inappropriate, somewhat inappropriate, neither appropriate nor inappropriate, somewhat appropriate, or very appropriate for adult literacy education.

Altogether 81 adult educators rated the eight metaphors. The analysis below shows the metaphors and the percentage of adult educators rating the metaphor either somewhat or very appropriate for adult literacy education. The metaphors are listed according to their rank as most to least appropriate.

  1. The Psychotherapy Metaphor. Adult Literacy Education is an ego strength (self-esteem) developer. Analogy: The Psychotherapist is to the Depressed Client as the Adult Literacy Educator is to the Low Self Esteem Adult Learner 80%
  2. The Business Metaphor: Adult Literacy Education is a business. Analogy: The Merchant is to the Customer as the Adult Literacy Educator is to the Student. 70%
  3. (tie with #4)The Economic Metaphor. Adult Literacy Education is an investment (human capital investment). Analogy: The Business Person is to the Investment in Machinery/Facilities (to Increase Profits Through Increased Productivity) as the Public (including the adult students/learners) is to the Investment in Adult Literacy Educators, Facilities, Study Time, etc. to Increase The Productivity of Adult Students/Learners (note: this is listed as rank #3 because it seems more closely related to the second place Business metaphor than does the Public School metaphor which is tied for rank #3). 69%
  4. (tie with #3)The Public Schools Metaphor. Adult Literacy Education is a public school. Its purpose is to educate adults to their potential and to teach them to be responsible and productive citizens (or at least get their GED). Analogy: The Public Schools are to Children as Adult Literacy Providers are to Adults. 69%
  5. The Revolutionary Metaphor. Adult Literacy Education is a liberator. Analogy: The Revolutionary Leader (Liberator) is to the Oppressed as the Adult Literacy Educator is to the Learners. 58%
  6. The Medical Metaphor: Adult Literacy Education is a clinic. It cures (remediates) maladies of education. Analogy: The Doctor is to the Patient as the Adult Literacy Educator is to the Student. 56%
  7. The Military Metaphor. Adult literacy education is a war against illiteracy. Analogies: (1) The soldier "stamps out" aggression as the adult literacy educator "stamps out" illiteracy. (2) The soldier is to the enemy as the adult literacy educator is to the illiterate. 28%
  8. The Parent Metaphor. The adult literacy educator is a parent. Analogy: The Parent is to the Child as the Adult Literacy Educator is to the Adult Student. 26%

For 19 Canadians in community literacy work, the top three metaphors were: Psychotherapy, Economic, Business,. For 25 U.S. literacy educators in community based groups the top three metaphors were: Psychotherapy, Public Schools, Business,. For 37 U.S. adult educators working in corrections the top three metaphors were: Business, Economic, Public Schools,.

Across the groups, six of the eight metaphors were rated as appropriate for adult literacy education by over half the respondents. Interestingly, the Revolutionary metaphor, which might be associated with social justice and the critical literacy movement, especially the work of Paulo Freire, did not emerge in the top three metaphors or analogies thought appropriate for adult literacy education by the 81 participants in the present surveys. In contrast, the Business and/or Economic metaphors were always in the top three dominant metaphors.

The predominance of the Psychotherapy metaphor in community-based adult literacy educators in both Canada (89%) and the U.S. (90%) while the Revolutionary metaphor was ranked fourth in both groups may indicate that adult literacy workers in these settings view depression rather than oppression as a more serious problem to be overcome in their literacy work. If this is so, it is a view not shared by their colleagues who work in corrections, where Psychotherapy was tied with Medical for fourth place and the Revolutionary metaphor was ranked sixth.

This suggests that adult literacy educators working in different organizational or institutional contexts work with different orienting mindsets about their literacy education activities. Possibly this could complicate the development of policies and practices that are uniform and standardized across various organizational contexts.

I note that the phrasing of the metaphors might affect responses. The survey form has more information for each metaphor. If anyone would like a copy of the survey form send an email with a surface mailing address to tsticht@aznet.net.

Tom Sticht Applied Behavioral & Cognitive Sciences, Inc.
NLA Listserv: Research Note 10/2/99

Message Twenty-One
2-2-05

George, Marie, et al,

Perhaps it is because I sit in a different "seat" and how this leads to somewhat different perspectives, but I don't have the same reaction to comparisons with the medical research model(s). As a funding source for ABE (adult literacy, adult secondary, ESOL, family literacy, workplace education, etc.), our office is constantly bombarded with assertions and anecdotes of who and what is "good," "effective," "high quality," "best practice," etc. Absent evidence I can have confidence in, I have to take most of these claims with a grain of salt.

When I refer to medical research model(s) I am not seeking to apply a "disease prevention/treatment/cure" model to ABE. Rather, I am seeking to get as much as I/we can from the systematic approaches to answering hard and complicated questions that have been developed and refined by our colleagues in the medical community. Although the "life and death" issues are different in many regards, I believe that there is just as much at stake in education as there is in medicine (that's why I've dedicated most of my life to this work)-- hence, I also believe that we have a lot in common in terms of having to answer hard and complicated questions.

Now I am incredibly familiar with all (or at least virtually all) the arguments about shortcomings with the data we are (and are not) collecting and with ABE related research studies -- those posted here (virtually all of which I carefully read) as well as those made by practitioners, researchers, and other stakeholders here in Massachusetts and across the nation. That said, I also have to process/deal with the shortcomings of values driven, philosophically driven, and politically driven claims -- as well as claims that are driven by plain old self-interest. At the end of the day, I am struck by how few decisions we can make where we are very confident that our current and future students are getting the absolute best from the all too scarce resources dedicated to achieving their dreams and aspirations.

Is this expectation unrealistic? In many respects, the answer is surely yes -- at least at this point in human development. However, I see no reason that we shouldn't be pursuing the goal of increased confidence that our current and future students are getting the absolute best -- albeit in an informed/"smart" and realistic manner. We should be skeptical when a single study yields results that fly in the face of our own experience -- witness the "Riverside, CA" study that concluded that basic skills education made no contribution to the employability and earnings potential of adults on public assistance. It was a well constructed study and its findings were probably correct -- for that context (program, staff, students, curriculum, etc), at that point in time. It was used to justify the "work first" movement nationally and in many states. In the natural sciences, the vast majority of research and policy leaders would never alter course so dramatically based on a single study that had not been replicated several times over and with systematic variations of the many, many variables at play.

Why are some people so quick to be dismissive of others who care about our students and our field as much as they do? The director and staff at NCSALL are such people AND they see value in the medical research model(s). Guilt by association is the cheap way out and doesn't do our students justice. Why aren't we encouraging our colleagues to think this through and find the value that is there -- to engage in a dialogue that helps us learn how to use the best of it and avoid the pitfalls? We have a lot to learn -- that's what I've loved about waking up every morning for 33 years and heading into work -- that this is a field of work where I get to learn and grow every single day.

My colleagues and I who are responsible for adult education in Massachusetts will continue to listen and learn from many different people / perspectives / approaches / organizations, et al -- including systematic research and data analysis following "the medical model(s)." And we, as well as our students, will be better off for it.

take care, bob bickerton, MA director of adult ed and chair of the national council of state directors of adult ed

Message Twenty-Two
2-3-05 Yeah, but my point is that while it sometimes happens that a person afflicted with a physical ailment may feel some shame about it (those cases are going to be isolated and I would say connected to societal positions), the person who suffers from lack of education will always feel that shame, which comes directly from society.

Pain is just a term, and you can situate it in any context. I find traffic and grocery shopping to be a pain sometimes, I find my kids at the haywire after dinner hour to be a pain sometimes.

I don't associate the term only with medical stuff and so it doesn't hold water for me that the medical model can be linked to education because people feel various types of pain.

marie

Message Twenty-Three
2-3-05

I think that Bob Bickerton is basically making an argument for common sense (we shouldn’t leap to change our practice on the basis of research-- until it’s been replicated and shown to be applicable to our particular “case.”) and for open-mindedness (There may be as many reasonably effective ways to make the cat function better as there are to kill it.)

Makes sense to me.

Karen Mundie Greater Pittsburgh Literacy Council

Message Twenty-Four
2-4-05

Debbie, et al,

Thank you for your kind words AND your critically thoughtful response -- "critical" in all the best connotations of the word.

I've also been in many meetings and discussions where I've witnessed the disconnect between educators, researchers, and policy makers. For what it's worth, I agree that arrogance is the most unfortunate source of that disconnect but that there are others as well. For example, I've followed up with some of our public health partners after meetings in which it was clear that we weren't all on the same page -- this doesn't happen frequently, but it does occur from time to time. In these instances, our further discussions tend to surface two other dynamics. The first is that although we have an enormous overlap in VALUES, the vocabulary and conceptual frameworks we use can be quite different. We sometimes think we're talking about the same thing and it turns out that we've engaged in a bit of "two ships passing in the night." The second is that they've been trained to be hyper cautious about what their training tells them are "unsubstantiated claims" -- lives, after all, and lots of them are at stake; witness some past and current claims by pharmaceutical companies. It's not always clear to them why we don't approach what's "proven" and "unproven" in the same way when we discuss instruction, a.k.a. "treatment" in medical parlance. Health educators generally provide a helpful bridge but it is my belief that we also own some of the responsibility for building these bridges. For me, that means being able to see the world from their point of view and helping them to see the world from mine (ours). It also means sharing with them the best of what we have to offer AND, of course, learning from them the best of what they have to offer -- including the rigor they bring to having "confidence" about the decisions that must be made.

But we do encounter so much arrogance that it interferes with potentially beneficial reciprocal processes. I'll add "hypocrisy" to the list as well. For example, the same administration that tells us to value only scientifically proven methods also tell us to support faith based delivery models of unproven success -- "unproven" at least by that same criteria.

I'm reminded of a quote from a letter I read 20 or more years ago that Charles Lamb sent to a colleague about a mutual acquaintance -- it resonated so powerfully with me that I committed it to memory: "He uses statistics like a drunken man uses a lamp post -- for support, rather than illumination." In one graceful phrase, he captured for me both the glory and the gore of scientific inquiry -- both of which I encountered as an undergrad at MIT (first pursuing Physics, later Architecture and finally being captivated and then "captured" by my work study job in adult literacy!). Having some fluency in both worlds has been helpful.

As you point out, there is more evidence than only that which can be captured via treatment and control group methodologies -- hence, we enthusiastically support a wide range of ABE programs in our state (including some faith based programs) for which we have a wide range of evidence of effectiveness. (By the way, over 50% of ABE programs in Massachusetts are community based organizations AND their grants are not systematically less {or more} that the grants we award to school districts, community colleges, and other providers.)

At the end of the day, we can and should work together to use EVERY tool available to us to learn what works well and not let the arrogance and hypocrisy throw us off track. For me, that's a separate political battle.

take care, bob bickerton, MA director of adult ed and NCSDAE chair

End of Part 2. Continue on to Scientific-basedEdResearchPart3 or Return to Scientific-basedEdResearch

Retrieved from "http://wiki.literacytent.org/index.php/Scientific-basedEdResearchPart2"