Scientific-basedEdResearchPart3

From LiteracyTentWiki

Discussion continued from Scientific-basedEdResearchPart2 and begun with Scientific-basedEdResearch

Message Twenty-Five
2-5-05

Hi folks,

This is a general response both to Cecil's and Bob's highly thoughtful messages, with which I am not in that much disagreement. However, I would stress more than them the centrality of political culture in grounding the discussion of legitimacy over educational research--which I would rather refer to as educational scholarship that allows for equal scope both for theory and empirically-driven research, both of which are valuable in their own dimensions. The conservative political polemic is inscribed in the USdOE Strategic Plan in the following:

"Unlike medicine, agriculture, and industrial production, the field of education operates largely on the basis of ideology and professional consensus. As such, it is subjected to fads and is incapable of cumulative progress that follows from the application of the scientific method and from the systematic collection and use of objective information in policy making. We will change education to make it an evidence-based field." (U.S. Department of Education Strategic Plan, 2002-2007, p., 48).

Thus, the analogy to medicine as the surer source of evidence to educational truth is politically inscribed from the get go, as an underlying reality of the cultural politics of contemporary educational discourse. That does not make investigations based on such a model invalid, far from it, and such studies can be productive indeed. However, it does problematize the analogy from its inception in which the linkage between what is referred to as "objective" science and educational "fads" and "frills" couldn't be starker. Thus, in three unsubstantiated sentences, a century's worth of academic educational discourse is ruled as illicit by definition. In working through whatever approaches and methodologies may bear light on a given topic, and keeping an open mind, this political reality is part of the underlying dynamics on how legitimacy is being defined in the public and policy sector at this time. That the educational is political all the way down couldn't be starker in the given era, a reality that belies the alleged quest for "objectivity" that drives the current USdOE for educational truth.

On the Stanovich and Stanovitch article, in my review I identified 19 statements by the authors with which I was in essential agreement. So in that respect, I am with Cecil and others who point to the strong points made in the piece about science and educational research. My concerns were two-fold

a) A very strong polemics against other modes of educational scholarship linked too easily with "fads," political machinations, and concerns over the "guru principle." Catherine King's post on the importance of Donna Merten's text, Research Methods in Education and Psychology may have applicability here in that one person's paradigm may be viewed as another person's fad. The issue is not only research methods, but epistemology and traditions of contemporary social philosophy on how they factor in to what gets accepted as legitimate and what gets dubbed as fads. I deal with this issue of research traditions in my chapter 9 of Conflicting Paradigms (thanks Tom, for doing those reviews). In that I juxtapose the Mertens' text with that of the study by Shavelstone and Towne Scientific Research in Education, written in the spirit very much similar to the Stanovich and Stanovitch article.

b) I took some issue with the authors' interpretation of what is fundamental and what is secondary to science and emphasized the centrality of the scientific method based on whatever problem at hand a particular investigation was focused on. I develop this argument extensively in my web-based article (which expands on the research chapters in my book) Postpositivist Scientific Philosophy: Mediating Convergences: http://www.the-rathouse.com/writingsonpopper.html

I am positing this article as a counter statement of sorts to at least some of the claims in the Stanovich and Stanovitch article, even as I am more sympatico with what I take as its more reasonable statements.

In terms of the Stanovich and Stanovitch article, one of the arguments I make is to challenge the emphasis the authors place on the value of "metastatistical analysis" and the concepts of "comparison, control, and manipulation" I don't deny the importance of these, depending on the nature of a particular study, but neither do I define these as the summa cum laude of the substance of scientific research, which I link more with the scientific method that characterizes empirically-based inquiry as articulated, for example, in John Dewey's Logic: The Theory of Inquiry (esp. Ch 6, patterns of Inquiry") and in Karl Popper's Realism and the Aims of Science I've discussed these texts in various email messages and they make up the substance of my web article, so I won't elaborate here, except to make the singular point that choice of methodologies depends on the research problem at hand, and some questions remain highly qualitative in scope.

In terms of working through the knotty problem of political brokering, I've discussed in several messages the importance of a bi-lingual approach. For some, political necessity may require a close affiliation with the operative assumptions of the DOE, even when one seeks to broaden definitions regardless of what one thinks of the research the Department spawns or legitimizes--and some of it may be quite good. On that I keep an open mind. I think this is one of the realities in which NCSALL, for example, has to operate, which, in my view, needs to be honored. Even still, and I think NCSALL does this, is the need for a bi-lingual approach--working with the feds as necessary and as useful, but:

a) Not allowing DOE assumptions to define one's own house b) Legitimizing other modalities of educational scholarship however distant it may be from DOE premises on the nature of scientific research

Then there are other--many other adult literacy scholars and critical practitioners who are not politically compelled to accept DOE premises. They have the luxury, and also the responsibility, of drawing on DOE-based research to the extent, and only to the extent that they can determine that such work has validity and adds to our collective knowledge of the field. On this interpretation, the more rigorous definitions of science promulgated by Stanovich and Stanovich as well as Shavelsone and Towne would be accepted for what they contribute, but not necessarily as the standard upon which intellectual legitimacy is based. That latter quest requires a working through the academic canonical assumptions that comprise the interdisciplinary field of adult literacy studies. While in one sense there is no end to this since canons are forever potentially revisable, the reality is that the academic disciplines have developed their own revisable norms in the determination of the quality of a study in a given discipline or disciplines.

Part of the issue is the extent to which adult literacy studies belongs in the sciences or cultural studies, or if both, which seems reasonable, how the mediation is negotiated. In terms of the politics of legitimacy the issue is whether the normative standards are derived from Washington D.C., or, as an applied field, through some combination between some segment of the field and an interdisciplinary academic canon that gives intellectual structure to the field.

No doubt, these are large questions, but ones, I argue, that are essential to explore. All insight that sheds knowledge is welcome. So is an acute analysis on the ways in which political culture constrains or enhances certain ways of perceiving reality.

While the French are not much in favor these days in Washington D.C., French social philosopher Michael Foucault had more than a point in drawing our attention to how the power/knowledge nexus is construed in given historical times.

George Demetrion

Message Twenty-Six
2-5-05

Thank you, Debbie and Bob, for the dialogue.

I have read both posts fairly carefully, it is Saturday morning and the pressure is off for awhile.

Debbie, I'm wondering what is in that area where you don't overlap with scientific studies (I may not be saying it well), because it sounds like there is an area of agreement when you talk with funders, then another area thst belongs really to education and not a medical model. This area I'd call a teacher/student model.

I'll illustrate. Over my teaching years I developed (did not go in with) an informal model of "good teaching health." I can't remember how many indicators I had, I'm out of the classroom. But here's one: in the hallway before class, if a student looked maybe grumpy or out of sorts, I would praise something about the student (not hard, they were all my children), like a hat, a sweater, something. If the student smiled and looked happy I'd know everything was OK, if not, then I had to get busy and make the world better.

So that is why I am wondering if these pieces of evidence for program "success" I'd guess you'd say, are in that gray area? The teacher/student area? If so, then what might these indicators be? I hope I'm on track, here.

Thanks a lot.

Andrea

Message Twenty-Seven
2-5-05

George and anyone else.

We simply have to widen the model, widen the framework, there is no other way. We have to add in educational stuff. "Stuff" is not a very elegant way of putting it. We are teachers, not doctors.

And I do want to say that the political trimmings now draped around the science model turn it into "scientism," "the theory that investigational methods used in the natural sciences should be applied in all fields of inquiry." When I was in graduate school I was taught that "scientism" was a bad word, a measure of slipshod thinking.

I don't know how this can be done. How do we fit the woman who still comes to school after being abused by her husband into a medical model? How education benefits students is not easy and can take a long time. The Perry pre-school students have been followed now for maybe 20 years.

Andrea

Message Twenty-Eight
2-5-05

George, Tom, and others:

I would like to offer three quotes from a contemporary philosopher, Bernard Lonergan, that might shed some light on the problems that presently confront education and the trend in policy, not towards "evidence-based research," but in covertly defining that evidence on the medical model--defined further on the model of physics and the natural sciences, rather than on human development and dialogue.

First, about natural and human sciences (and education):

"First of all, then, there is a real problem set by science and especially by human science, . . . Natural science can get along pretty well by relying simply on the pragmatic criterion of success. We produce results, and everybody can see the results. They can keep going on that basis. But even so, they suffer from a neglect of basic research, of fundamental thinking, simply because it is difficult to see the necessity of fundamental research when your criteria ultimately are pragmatic, the results that everyone can appreciate. But human science cannot get along on that basis.

"Human science is involved in philosophic issues from the simple fact that the human scientist is one of his own objects. He cannot be totally detached in the science without special guidance. The history of the human sciences, where the element ‘scientific’ has been emphasized, has been a continual flight from what is truly human to what, in man, is not properly human." (Collected World of Bernard Lonergan: Phenomenology and Logic: The Boston College Lectures on Mathematical Logic and Existentialism {2001}. University of Toronto Press)

Second, about education and the scientist's horizons:

"All learning is, not a mere addition to previous learning, but rather an organic growth of it. . . . Moreover, every investigation is conducted from within some horizon. This remains true even if one does not know one operates from with a horizon, or even if one assumes that one makes no assumptions. . . . So all of our intentions, statements, deeds stand within contexts. To such contexts we appeal when we outline the reasons for our goals, when we clarify amplify, qualify our statements, or when we explain our deeds. Within such contexts must be fitted each new item of knowledge and each new factor in our attitudes. What does not fit, will not be noticed or, if forced on our attention, it will seem irrelevant or unimportant. Horizons then are the sweep of our interests and of our knowledge; they are the fertile source of further knowledge and care; but they also are the boundaries that limit our capacities for assimilating more than we already have attained." (Lonergan, 1972, Method In Theology, pp. 235-250)

Third, about the attitude and prior philosophical condition of the scientist. I have added "the data of educational studies" after the "data of consciousness" as an extrapolation of Lonergan's meaning to our field:

. . . as there are sciences of nature, so also there is a science of man. As the sciences of nature are empirical, so also the science of man is empirical; for science is the resultant of an accumulation of related insights, and scientific insights grasp ideas that are immanent not in what is imagined but in what is given. If the sciences of nature can be led astray by the blunder that the objective is not the verified, but the ‘out there’, so also can the human sciences;

but while this blunder in physics yields no more than the ineptitude of Galileo’s primary qualities and Newton’s true motion, it leads zealous practitioners of scientific method in the human field to rule out of court a major portion of the data and so deny the empirical principle. Durkheimian sociology and behaviourist psychology may have excuses for barring the data of consciousness (and the data of educational studies), for there exist notable difficulties in determining such data; but the business of the scientist is not to allege difficulties as excuses . . . (Lonergan, Insight, A Study of Human Understanding [1958], p. 235)

My note: This "difficulty" extends into many areas of education in both adult and K-12 arenas of study and application.

Catherine B. King
Adjunct Instructor
Department of Education
National University
San Diego, CA

Message Twenty-Nine
2-5-05

Hello Andrea:

The text that I referred to earlier addresses your note and draws on several fields of studies and many arguments to give us a "picture" of where the field is with regard to the "stuff" you speak of.

Here it is again: "Research and Evaluation in Education and Psychology: Integrating Diversity with Quantitative, Qualitative, and Mixed Methods (D. Mertens, 2005; Sage Publications: Thousand Oaks, CA)

Mertens gives full credence to the contributions of positivist and post-positivist models (the natural-science and statistical sides of the medical model) while showing how the human sciences and education have grown to

(1) include human data and history, and

(2) maintain critical methods from within different paradigms of study, e.g., constructivist and transformative paradigms.

These other paradigms maintain their critical-scientific edge-- rightly defined by our <methods> and not by restricting the data, or by restricting our expectations of outcomes, to those that are appropriate to the natural or the mathematical fields.

Whatever the tenets of science and the scientist are supposed to be, they are not, and never have been, dogmatic.

Regards,

Catherine King

Message Thirty
2-5-05

Hi Catherine,

It certainly is nice to hear your voice bubble up again! And I really enjoyed George's discussion with you in his book, it brought back old times.

Thanks very much for the reference, I'm glad you take cognizance of the "stuff."

The medical model just isn't good enough, it doesn't fit who we are and what we do. It doesn't capture quality, excellence.

Regards,

Andrea

Message Thirty-One
2-3-05

Hi Andrea, thanks for this, but no, it doesn't make any sense to me.

I believe there's a big difference between those types of pain. One type of pain leads the sufferer to believe that this is no fault of his own, and via the institutions that are set up for him, he can find the treatment he needs and deserves (hmmm...guess we'll leave the health care situation out of that comment for now).

The other type of pain is about shame and the feeling that somehow this is your fault and you are not good enough because those around you don't seem to have any problem. Then to compound this, there aren't droves of people falling over themselves to build institutions to provide them the services they need and deserve. Although, if that guy is lucky enough to find one of those institutions, bet he has to be on a wait list.

And that is also a painful message.

marie

Message Thirty-Two
2-3-05

There are few people whose opinions I respect as much as Bob Bickerton's, and I thank him for his balanced post about the usefulness of the medical model. Some additional comments:

from Bob: "...I believe that there is just as much at stake in education as there is in medicine (that's why I've dedicated most of my life to this work)-- hence, I also believe that we have a lot in common in terms of having to answer hard and complicated questions..."

As moderator of the NIFL Health list, it has been my pleasure to dialogue with a number of people in the medical profession, and the common ground is rich indeed. The medical model, and scientific processes in general, are absolutely and critically important. However, often the "dialogue" I yearn for with members of the scientific community goes nowhere. Also from Bob:

"...I see no reason that we shouldn't be pursuing the goal of increased confidence that our current and future students are getting the absolute best -- albeit in an informed/"smart" and realistic manner...In the natural sciences, the vast majority of research and policy leaders would never alter course so dramatically based on a single study that had not been replicated several times over and with systematic variations of the many, many variables at play... Why are some people so quick to be dismissive of others who care about our students and our field as much as they do?... Why aren't we encouraging our colleagues to think this through and find the value that is there -- to engage in a dialogue that helps us learn how to use the best of it and avoid the pitfalls?..."

I agree with every word. But I'd like to offer an answer to Bob's questions:

I am eager to "engage in a dialogue" with anyone about virtually anything so long as there is mutual respect and a true desire to come away having learned something. My problem is I have attempted to engage in a dialogue in many different ways with "scientific people" who often clearly have an agenda, aren't interested in what I have to say, and insist on defining the parameters of any discussion by absolutes arbitrarily defined. In short, I am dismissed before I even get to express any ideas. Unfortunately, some of these arrogant individuals, I believe, are in charge of a lot of adult education policy-making at the moment. NCL,NIFL and NCSALL are working hard to communicate with them, but I have seen no evidence of a genuine dialogue much less the development of conclusions because one side isn't listening. But maybe it is me who won't listen, because I refuse to accept that "scientific studies" are the ONLY legitimate source of information to guide my practice. I am certain Bob agrees that the "wisdom" we have already discussed here is also valuable, and the Stanovich report does a great job of explaining how conclusions can be reached by finding the intersection of wisdom and science.

But Bob continues:

"...My colleagues and I who are responsible for adult education in Massachusetts will continue to listen and learn from many different people / perspectives / approaches / organizations, et al -- including systematic research and data analysis following "the medical model(s)." And we, as well as our students, will be better off for it."

Thank God for Bob and all those like him. I just wish he were the one in charge of re-writing the WIA, oops, I mean the JTIA, the JOB TRAINING Improvement Act. I just have to remind myself daily, "This too will pass."

With apologies for some heat, but I have no where else to dump it...Debbie

Deborah W. Yoho
Co-moderator, NIFL-Health Listserv
Executive Director, Greater Columbia Literacy Council Past President, SC Adult Literacy Educators
2728 Devine Street, Columbia, SC 29205
803-765-2555 Fax 803-799-8417 dwyoho@earthlink.net

Message Thirty-Three
2-6-05

Perhaps what concerns me most about the use of the scientific model in any field is the complete faith that some put in it. In medicine, for example, a scientific model works well if we know that condition A is caused by B and that intervention C will create condition D, health. There are many instances where this formula works, and the quality of our lives is better as a result. But there are so many cases where this model does not work. And when the model does not work, the patient often ends up being blamed, marginalized, not believed, given inappropriate treatments, or being shunned.

When doctors have unconditional faith in medicine as a science, they do not believe patients who describe things that are not in their medical books; they cannot cope with seeing patients they cannot cure. Yet, there are known diseases for which there is no known cause. Some of these diseases have recently been "validated" as a result of the human genome project's finding their genetic basis. There are also many disease for which there is no known cure; and there are cures that cannot be explained.

A patient who has a rare disorder or a disorder that manifests itself in an unusual manner is served well by a physician who practices medicine as both a science and an art. Such physicians see things that others do not. They imagine interventions that others would not entertain.

Many of us learned in schools, so the practices being used did work for some and to some degree. But the adults we work with did not learn in school. In that way, I think they are more like those with a rare disorder. (Please do not take this to mean that I think they have a disease. This is just an analogy.)

My fear is that the reliance on a scientific model is not likely to produce results that apply well to the adults we serve. Then, to the extent that we invest in this model, when "proven" approaches do not work with our adults, they will once again be blamed, marginalized, not believed, given inappropriate interventions, or be shunned.

I believe that our students are more the exceptions than the rule. Even if scientific methodology provides evidence of clear educational cases where condition A is caused by B and with intervention C will create condition D, learning, my guess is that the adults we work with will not be well served by this approach. They will most likely be more like the patients whose conditions require practitioners who draw from both the science and the art of their practice.

Hs (Helen Schlarp)

Message Thirty-Four
2-7-05

Debbie,

Thank you very much for speaking as you have. This is how I too worked as a teacher, and I assume (cannot test this) that many teachers work this way, also.

This is what John Dewey espoused: ask questions, gather data, analyse data against the questions, then formulate more questions--it is a growth method of learning. And it is 100% scientific. This is what ANY scientist does. ANY SCIENTIST.

And if anyone on this list needs an example of ANY SCIENTIST, go to the work of neuroscientist Joseph LeDoux, known for research on the amygdala--world class guy--this is the process he went through.

I really appreciate your thoughtful response.

As you can tell, I get pretty steamed when the science of teaching is run over with a Mack truck. Some say "art of teaching" because it would be impossible to ennumerate all the variables. Just so we're all on the same page.

Andrea

Message Thirty-Five
2-7-05

Colleagues--

I guess I better follow up on what I said previously.

Quantitative research is a subheading of qualitative research.Debbie is describing qualitative research.

In teaching, the quality of the relationship between teacher and student is what counts. Maybe this could be measured by "happy classroom," v "unhappy classroom," but maybe not. The teacher is the instrument, as in any qualitative research project, where the researcher is the instrument. Debbie knows she is doing the right thing because the student tells her, in many ways. If she is "off-course," she will make a course correction.

The best example I know of this process is described in the works of Vivian Paley., but be careful, she teaches young children.

I was reading last night some Washington documents put together by people who had never taught...and it showed. I don't like to be trampled on, professionally, by people who don't know what they are talking about, it hurts to much, so I have to speak up. And I really do thank Debbie for speaking up as she did.

Andrea

Message Thirty-Six
2-7-05

Andrea, et.al,

You've invited me to talk further about the "science question", so I shall offer a few ideas. But I feel this forum by its very nature severely limits discussion of something so complex.

When I was a high school principal, I learned (in fact I was told) that success lay in "finding the greatest good for the greatest number". Now this isn't a bad premise, and I think it is the premise that motivates a lot of scholarship in education. As a principal, I spent a lot of time reading published scientific studies in search of this elusive formula. In the end, the guidance I found in these studies seemed to "prove" what to me seemed more like common sense to begin with. For example, classroom climate, the power of role models, the efficiency of direct instruction, and the importance of mastery have all been well-documented by eminent researchers. Yet I already knew that these things were important even before I was trained as a teacher. I experienced them, you see, as a student. At the time I wasn't thinking about the ways I could create these things. I just looked for them as I chose my own course of study. Later as I considered published research about these things my study had a profound influence on me professionally. I think perhaps I was affected so much because I had found "legitimacy" for what I already "felt" to be true.

Now I recognize a lot of dangers were/are inherent in this process. Certainly this is not empirical science. However, as I continued to practice as a teacher I learned something else: always, in every circumstance, reserve judgement (decision) until there is enough evidence to reasonably assure me that I am on the right course. And where does this evidence come from? Why, from the students, of course. And here is where I believe I do have "overlap" (Andrea's word) with science. So long as my mind remains open and I consciously look for evidence, am I not doing the same things an empiricist does when an hypothesis is offered and then tested?

But on the other hand, I am certain that this process of study-apply-grow cannot be "science", because I also learned that I cannot ever guarantee that "what works" with one class or learner or school can be "duplicated". Here is where some scientists lose respect for me. S/he thinks that if I can't repeat one success with another, using the same tool, then whatever process I am using must not be legitimate.

Here is where values come in. Now the battle is joined! If one approach doesn't work, I do not reject it as illegitimate. I put that tool back into the toolbox and use another, saving the first tool for another application at another time. The more research I read, the more tools I add, and with experience, I become better and better at matching the right tool with any given set of circumstances, under conditions that constantly change.

In talking with doctors and medical people, it eventually dawned on me that this metaphor of a "toolbox" and this seemingly trial and error process is exactly what physicians do every day, although few would say so out loud. Because both doctors and teachers are in effect "tampering" with a human being, we don't like to think about the risks we take, much less talk about them. In fact, we spend copious energy trying to minimize the risks i.e. we research and read and talk and question and never stop learning. We are always "practicing".

How do I know I am effective? The same way doctors do. The learner/patient, if s/he can, tells me I am, in many, many ways. But sometimes I really don't know exactly what I have accomplished.

What I think is happening in this political climate is that one school of science, educational psychology and neurology, has provided, perhaps unwittingly, "legitimacy" to a set of values about teaching and learning that was already perceived as "common sense" by a group of people who now happen to have power. Well, I did/do the same thing. But the difference is I learned to RESERVE JUDGEMENT, and keep looking for more tools for my toolbox. So long as I continue to do that, I think I am on common ground with most scientists. It is not science that is the problem. It is how that science is used. And isn't that always where the rub is?

It bothers me that informed, careful trial and error is perfectly legitimate in science, but now rejected as "fads" in education. It bothers me that years and years of published research has been written off as merely self-serving gibberish.

When I was in Washington I suggested at the last NIFL meeting that their website include a prominent section (perhaps a compendium) of seminal research in education. I wonder sometimes if the current decision-makers have ever heard of Dewey or Maslow, Bloom or Mager or Hunter. I hope this will be considered.

For the Cause! Debbie

Message Thirty-Seven
2-7-05

Debbie,

Thank you very much for speaking as you have. This is how I too worked as a teacher, and I assume (cannot test this) that many teachers work this way, also.

This is what John Dewey espoused: ask questions, gather data, analyze data against the questions, then formulate more questions--it is a growth method of learning. And it is 100% scientific. This is what ANY scientist does. ANY SCIENTIST.

And if anyone on this list needs an example of ANY SCIENTIST, go to the work of neuroscientist Joseph LeDoux, known for research on the amygdala--world class guy--this is the process he went through.

I really appreciate your thoughtful response.

Andrea

Continue on to Scientific-basedEdResearchPart4 , or to ConceptsinInquiryandLearning or Return to the beginning of this discussion at Scientific-basedEdResearch or to the previous page Scientific-basedEdResearchPart2

Retrieved from "http://wiki.literacytent.org/index.php/Scientific-basedEdResearchPart3"

Scientific-basedEdResearchPart3

From LiteracyTentWiki

Views

Personal tools

Navigation

Search

Toolbox