AccountabilityFullDiscussion
From LiteracyTentWiki
Return to Accountability
From: eileeneckert@hotmail.com
Subject: [NIFL-AALPD:1852] Re: education and learning
Date: Monday, January 03, 2005 07:38 PM
Andrea, you said it sounds like I was getting at organizational learning when I wrote about examining finance, policy, administration, etc. in light of the extent to which they are organized around and support learning as a goal and as a process. A couple of comments.
"Organizational learning" as a term becomes a kind of shorthand for a whole set of assumptions, theories, and practices. This list regularly features discussions in which someone uses a word or phrase, assuming it will mean the same thing to all the readers as it does to the writer, only to find him/herself caught in a storm of controversy because that innocuous bit of shorthand means something different to everyone and everyone assumes their meaning is "the" meaning--I exaggerate, but probably not by much. It's certainly happened to me enough, and probably will again with this message.
That said, many of the individuals and teams in an organization, or different organizations in a system, can focus their efforts on a common goal or issue and they might demonstrate "organizational learning" around that issue, but that content (the goal or issue) doesn't have to be learning. I'm talking about all the different people and departments and areas of organizations (and the systems of which they are a part) focusing on learning--organizational learning about learning--META-learning.
Let me give you an example. There are others, but this one comes to mind. In the last several years, especially since the Workforce Investment Act of 1998, but it started before that, there has been a lot of effort--and, I think--organizational learning, focused on the concept of accountability, as defined by the government and developers of the NRS and research and training activities. However, this effort around accountability is not an effort around learning; in fact, I'd argue it's detrimental to a focus on learning learning by siphoning money, time, and energy away from learning activities and into testing and record-keeping activities. A whole lot of smart and practical program leaders have become very savvy (demonstrated organizational learning) about playing the accountability game, but are they and their programs correspondingly better than they used to be at focusing on, fostering and supporting learning? Has that organizational learning led to more learning for students? Has it made instructors better at teaching? Has it made program directors more expert at setting up the conditions to support learning on the part of students, teachers, and themselves?
So no, I'm not just getting at organizational learning, I'm talking about making learning the focus of all an organization's, and a system's, efforts. If we had that at the top, if WIA and the NRS were about learning, it would make it much easier at every other level to focus on learning; but that's still not an excuse not to focus on learning.
Eileen
From: AWilder106@aol.com
Subject: [NIFL-AALPD:1854] Re: education and learning
Date: Monday, January 03, 2005 08:43 PM
Eileen,
I'm still not sure if you are talking about organizatinal learning or student learning--if student learning then I am with you all the way in your observation. Or organizational learning bent to student learning. I think that is the point of what you are saying? If so, yes,the student is being given short shrift.
Andrea
From: eileeneckert@hotmail.com
Subject: [NIFL-AALPD:1855] Re: education and learning
Date: Monday, January 03, 2005 10:07 PM
Andrea, I'm talking about learning as the <content or focus> of effort and organizational learning--certainly student learning; that's what adult basic skills, ESOL, all adult ed. programs are there for. But we shouldn't ignore the power of making our activities supportive of staff, teachers, and administrators learning, individually and together, <about learning>--that also has an effect on student learning (see the West Ed report, Teachers Who Learn, Kids Who Achieve). Learning and meta-learning.
Another example: it's kind of a cliche to say that feedback facilitates learning, but Kluger and DeNisi found that the effects of feedback can vary widely. Kluger and DeNisi’s (1996) meta-analysis examined the effects of feedback interventions (FIs), that is, “actions taken by (an) external agent(s) to provide information regarding some aspect(s) of one’s task performance” (p.255). Kluger and DeNisi found that on average, feedback had a moderate positive effect on performance (ESsm = .41). They also reported that over 38% of calculated effect sizes were negative. So in almost 40% of studies, people did worse with feedback than without it! I'm quoting my own dissertation here, which went on to review research on how feedback affects the development of proficiency.
It helps to know something about what research shows of the effects of feedback on learning, and what characteristics of feedback support learning (and what kind has a negative effect). It also helps to look at other disciplines that address learning and performance, as well as education, to find out what the research shows. And it helps to apply it, whether to student learning or to supervision of teachers or even to performance/compliance reviews of programs (there's an interesting study by Pajak and Glickman on how teachers received feedback with slight changes in wording). And to try it out, and to have time to reflect and evaluate how well it works when you try it, and revise based on that reflection...
I didn't pick that example because everyone should know about feedback--it's just one example of how the research is much more informative than the best practice of giving feedback on student work.
Eileen
P.S. Here are the references if you want to look them up: Kluger, A. N. & DeNisi, A. (1996). The effects of feedback interventions on performance: A historical review, a meta-analysis, and a preliminary feedback intervention theory. Psychological Bulletin, 119(2), 254-284.
Pajak, E. & Glickman, C. D. (1989). Informational and controlling language in simulated supervisory conferences. American Educational Research Journal, 26(1), 93-106.
From: rkmcknight@comcast.net
Subject: [NIFL-AALPD:1856] Re: education and learning
Date: Monday, January 03, 2005 10:53 PM
Good points, Eileen!
Thanks for raising these questions. I will add some more:
You are right on target asking if learning is the focus of our efforts. On a broad scale, ARE publicly funded organizations accountable to educators who work in these systems and to the public? Who monitors publicly funded programs? What results do they report? Do they even bother, or is it like insider trading - only known to a privileged few? Do the results demonstrate learning? Are they concerned with validity?
Perhaps its time to take a more proactive position - to ask for an accounting of publicly funded initiatives. Like the bills before congress, are we reading the annual reports? Where are the annual reports? Perhaps we need something like a "three day rule"? Here in Virginia, as in many states, technology and other initiatives have evaded the scrutiny of accountability. Though everyone is aware of this, nobody is willing to put their head on the chopping block by asking questions. As the saying goes, "never bite the hand that feeds you", and asking questions can have a negative effect on job security.
So I ask you now, is this how learning works? Does intimidation have a place in learning? Is it possible for learning to take place when questions are repressed? As surely as these questions need to be asked, publicly funded organizations need to be accounable, particularly those that are outsourced. An illustrative example can be found amid the clamor for charter schools. Though few results have been reported, and the integrity of some reports have come into question, a steady flow of funding and legislation continues to promote vouchers.
Who is minding the store? When funding is allocated, what results are achieved? Has public accountability fallen by the wayside? Must we wait for a crisis before we ask for public accountability? With so much banter about evidence, what evidence demonstates organizational accountability - organizational learning? Are we afraid to ask these questions? Whatever happened to learning organizations? This concept, so widely described, has not been part of my recent experience - how 'bout you? Are publicly funded organizations, learning organizations? Should they be? And, whatever happened to integrity - has it become an outmoded concept, too?
With hopes for accountablility in 2005,
Roberta McKnight Healthcare Multimedia Design http://www.hcmmdesign.net
From: rboone@vineland.org
Subject: [NIFL-AALPD:1857] Re: education and learning
Date: Tuesday, January 04, 2005 10:46 AM
>>Has public accountability fallen by the wayside? <<
Working in Adult Education an enormous amount of time is spent generating reports. The problem being, the amount of time actually "teaching" is inversely proportional to the volume of reports that must be submitted to various agencies each month. But in the name of accountability, we persevere. It seems that the reports are an end to themselves. It feels like no one even reads the reports. This is not just a problem with Adult Education, but with a ubiquitous information overload in the "computer age."
My question would be, is the accountability accountable?
What is the ultimate goal of accountability? Is it to help the student or to
build job security into an administrative layer that can demonstrate the
ability to quantify results?
richard boone
From: James.Parker@ed.gov
Subject: [NIFL-AALPD:1858] Re: education and learning
Date: Tuesday, January 04, 2005 10:54 AM
Richard, an excellent question. I think Public Accountability should travel both ways (up and down).
Parker, James
From: eileeneckert@hotmail.com
Subject: [NIFL-AALPD:1859] accountability (from the ed. and learning thread)
Date: Tuesday, January 04, 2005 01:25 PM
Roberta, Richard, James, and others, I think Roberta articulated many of my feelings and concerns, and I hope others will discuss the issues she raised (I know I'll come back to them). To pull one thread from the many in the messages, though, can we consider what we mean by accountability? It's another of those terms that has been so overused (in general, not so much on the list) and so contorted that it has lost much of its meaning. Instead of guiding and ensuring progress toward better meeting the needs of learners, it's evolved into a jumping-through-hoops set of empty formalities that often takes educators' attention away from a focus on learning. Several years ago I began trying to formulate my ideas on the subject, stemming from the thought that there's got to be a better way to structure accountability so that it is meaningful and consistent with improving learning and instruction to meet learners' needs. The result was an article with my colleague Sandy Bell of the UConn Adult Learning Program that will be published in Adult Basic Education, and I'm going to copy an excerpt here:
"Instead of outside interventions and requirements that structure accountability as a matter of responding to external standards, efforts in keeping with a complex systems view of literacy education would produce generative activity on the part of individuals and programs; that is, activity that contributes to the 'emergence of a coherent collective identity' (Davis & Sumara, 2001, p. 88). This activity weaves together learning and the documentation, assessment, and evaluation of that learning—at the individual, class, and program levels. Such activity would produce what we call 'authentic accountability.' "
We go on to describe how that could be done, and from the conclusion:
"We believe that authentic accountability efforts ... are congruent with the work of improving instruction and student learning in ways that conventional accountability efforts ... are not. However, authentic accountability activities cannot be imposed in addition to current conventional accountability requirements. Each approach requires different actions. Policymakers should not expect teachers and programs to fit the actions required for authentic accountability and those required for conventional accountability into their already overstretched budgets of time, energy, and money. Policymakers need to choose their paradigm. Educators need to analyze the factors influencing policymakers. With that analysis in mind, they can plan how best to advocate for use of an authentic accountability framework. Though we have described and explained authentic accountability from a complex systems and naturalistic research perspective, it is congruent with a Freireian approach to literacy as praxis, embodying both reflection and action (Freire, 1970/2000). It is profoundly political, and therefore, to some, dangerous. With an informed understanding of authentic accountability and the factors supporting and hindering its being understood, adopted, and implemented, educators can advocate for the approach to accountability that will meet their learners’ needs."
I think that rather than extending meaningless accountability busywork upwards (I don't think anyone was advocating for that, but I add it just to be clear), we need to have "authentic accountability" at every level.
Roberta, Richard, James, others, what would it "look like" if the agencies and organizations "above" you in the hierarchy were accountable to you and to learners? How would you know?
Thanks for the thought-provoking discussion.
Eileen
From: rkmcknight@comcast.net
Subject: [NIFL-AALPD:1860] Re: accountability (from the ed. and learning thread)
Date:Tuesday, January 04, 2005 03:02 PM
Yes Eileen,
We must clarify the distinction between assessment and accountability. Both should be free of bias.
Accountability is devised to ensure that funding is not wasted or
controlled by political agendas. This is why we have Sunshine Laws that
allow access to meeting minutes - to ensure that things are on the up
and up.
Accountability also ensures that political agendas do not usurp the
integrity of education's mission to serve the needs of learners and educators.
Assessment and accountability are not synonymous terms - they are quite distinct. I hope this helps.
My Best, Roberta McKnight Healthcare Multimedia Design http://www.hcmmdesign.net
From: rboone@vineland.org
Subject: [NIFL-AALPD:1861] Re: accountability (from the ed. and learning thread)
Date: Tuesday, January 04, 2005 03:55 PM
The assessment of Accountability is political by nature. Someone must set a "standard" by which we say there is a measure of Accountability. This standard is set politically by arbitrarily defining the assessments that measure Accountability. Ultimately accountability would just be the dependant variable applied to a function on assessment. A = f(a)
richard boone
From: eileeneckert@hotmail.com
Subject: [NIFL-AALPD:1863] Re: accountability (from the ed. and learning thread)
Date: Tuesday, January 04, 2005 09:26 PM
Roberta, I appreciate this statement that you made: "Publicly funded education is a public trust. How do we account for tax funded expenditures? What benefit has been derived from this funding?"
But I think you are boxing yourself in with your definitions of assessment and accountability. Assessment, even in conventional terms, goes well beyond the role you give it: "Assessment is devised to improve and serve instruction, teachers and learners. This is where we test learners with all the various means at our disposal - authentic, etc. - we assess what was learned in order to improve instruction." Assessment certainly includes assessment of learner performance--beginning and ending and the progress in between (although I don't <test> students at all even when I'm fortunate enough to be teaching or tutoring). Assessment can also be used to improve programming beyond instruction.
Assessment is at the heart of accountability--assessing and demonstrating the outcomes of instruction and programming, what's been learned and what's been gained. But I am <not> talking about standardized assessments. I think the outcomes should be measured first and most importantly in terms of quality-of-life and empowerment of learners, by their own perception, and how their learning in a program contributed to that. And instruction and programming should be focused on supporting that learning.
I'm not trying to operate within the conventional definitions of assessment and accountability; I'm proposing a legitimate (more legitimate, I think) alternative. It's not "free of bias" as I don't think there's any such thing; rather it tries to compensate for individual biases through triangulation of data and other naturalistic methods.
I don't think there's any reason to accept the current assessment and accountability frameworks even if the assessments in the accountability system actually met their own standards for validity and reliability. Those ideas suited the Cold War mentality that anything quantifiable was scientific and therefore better than anything that...wasn't, or didn't try to be.
I hear that the journal issue with my article is in the fall 2004 issue of Adult Basic Education (I didn't get a copy, so I didn't even know it was out!), so maybe you'll find it worth reading what I can't fully articulate in a post to the list. My point is simply that we can organize all aspects of education around learning, and we should.
Eileen
From: sgabb@bristol.mass.edu
Subject: [NIFL-AALPD:1864] Re: accountability (from the ed. and learning thread)
Date: Tuesday, January 04, 2005 10:59 PM
Roberta, Eileen (and all ) - I too thank you both for your thoughtful responses to the important questions of assessment and accountability. I too agree in spirit with your statement and questions - "Publicly funded education is a public trust. How do we account for tax funded expenditures? What benefit has been derived from this funding?"
I would remind our colloquia, however, that the Adult Basic Education delivery systems in this country are inadequately funded, therefore a betrayal of the hope that such a poorly supported system can in fact meet the exploding need for such services with quality and assurance of reasonable 'benefits' as defined by those with the power of the purse. In my 32 years with this amazing endeavor, I have learned over and over about the strength and determination of adults who choose to extend their formal education and build their basic skills. I have seen an increased awareness of this need in both the public and private sectors, and an increased willingness to provide 'tax funded expenditures'. However, the increase in funding has been meagher when compared to the expanding demand. Nevertheless, we have seen an increasing cry for 'accountability' from a field still condemning those within it to part time work and poverty wages. (Following is my longish response to this conversation. Read on at your own risk.)
I do embrace totally our responsibility, despite our limited funding, our need to be accountable to our learners. I agree with Eileen that it is our responsibility to engage in dialogue about goals, assessment and outcomes with our learners, in order to evaluate whether participation in the ABE services have indeed enabled the learners to frame their goals, identify needed skills and knowledge, and demonstrate competency and knowledge attainment in ways that are meaningful in their lives. In my experience, adult learners always know why they returned to school: each learner can spell out the reasons for entering our programs.
Each learner has defined purposes: to read better so I can enjoy a book, read to my children, feel like I am smart; to get a GED so I can complete my education, show my children it's important to finish what you start, feel like I am smart; to improve my English fluency in speaking, reading and writing so I can communicate with anyone who requires that I speak in English, fill out forms, write letters, understand what I am hearing.
The 'benefits' outlined by those who frame 'accountability to the tax payers' are listed in terms of numbers, certificates, certified employment. These 'countable outcomes' are considered by many the only legimate 'proof' of our accountability. Accountability to the learners - their evaluation of the services - is not considered legitimate.
Certainly we provide assessment to measure our own performance as well: are learners succeeding in attaining the skills and knowledge we have introduced? But as Eileen indicated, we need to assess with equal importance whether the knowledge and skills we introduce are really in line with the purposes and goals of our learners. Once we are sure our instructional content is in line with learner purposes and goals, we should indeed assess our own competency as instructors, guides for learning, partners in educational dialogue. In this context, as Eileen says, assessment with our learners enables us to evaluate and improve our entire schema for adult education programming.
I am also a realist. I recognize that currently, in order to work with and for adult learners in the current climate for funding and programming, I must comply with the limited, even destructive definitions of 'outcomes', 'benefits' and 'accountability'. I hope to continue engaging both practitioners and learners in dialogue about why this is important for survival in the short run, and about how we can - and must change and expand these definitions and practices in the long run. Through this dialogue we can elevate the voices of our learners as equal partners in advocacy for a system that is truly 'public adult education' - not the few bread crumbs we are spared, but rather the true investment in the natural resource of our learners, whose outcomes will continue to benefit us all.
Sally Gabb, SABES SE, Bristol Community College, Fall River, MA sgabb@bristol.mass.edu
From: rkmcknight@comcast.net
Subject: [NIFL-AALPD:1865] Re: Accountability (from the ed. and learning thread)
Date: Wednesday, January 05, 2005 01:07 AM
Thanks so much Eileen,
You are right on all counts. When I refer to testing, I'm referring to all forms of assessment. Assessment can take the form of asking questions or simply talking with a student. But, it gives us some place to begin and focus the learning process. I think of assessment as a microlevel process, and accountability as a macrolevel process:
Assessment let's us know what a person needs to learn - at the individual level. Its a circular concept - a feedback loop - to guide the continuous process of learning. It helps us discern, not only what needs to be learned, but also whether learning occurred from our efforts. Using myself as an example, I check (test, if you will) myself to see whether the direction of my inquiry is fruitful. If I'm not learning what I hoped to learn through a particular process, then I must decide where to direct my learning/inquiry to fill in any gaps in my knowledge/understanding.
From an accountability perspective, if I find that I'm not learning what I wanted in a particular program, then I'll scout around to find a program to help me. I might learn that the program isn't helping anyone - I don't know until I ask. Accountability exists at this macrolevel. It tells us things like - how a whole class/program is doing, whether particular teaching methods are useful. At the program/state/national level we evaluate the merits of programs and methods. By exchanging information about macrolevel approaches/issues at conferences and listservs like this one, we can make informed decisions about funding - to ensure the public trust.
For the purposes of this discussion, let's use the TABE as an example of the dynamics between assessment and accountability (micro/macro). The TABE has a limited scope - it assesses a particular range of skills for each individual (micro level). We know its validity is limited because it does not measure all that we wish to measure. Its use as a tool for accountability (macro level) is similarly limited. The magnitude of these limitations is increased when it is used as a tool for accountability to the exclusion of a broader range of measures.
We need more assessment tools to provide a better picture of what is occuring at the program level. Stites and many others have been saying this for a long time. To get at the issue of accountability and discern whether funding decisions are worthwhile, we have to ask questions like:
What is the process for approving new assessment instruments? This has not been made public by the NRS.
What content are we measuring with existing/approved instruments? This is a source of appropriate concern.
What benefit is derived from particular funding initiatives? This should be public information.
The issue here is public disclosure. Administators must address these and other questions. They must account for whether and how programs and expenditures for adult education serve learners and educators. In the process, perhaps we can link assessment and accountability in a more useful and meaningful way.
My Best, Roberta McKnight Healthcare Multimedia Design http://www.hcmmdesign.net
From: mariecora@hotspurpartners.com
Subject: [NIFL-AALPD:1866] Re: accountability (from the ed. and learning thread)
Date: Wednesday, January 05, 2005 10:03 AM
Hi all,
I agree that assessment and accountability are very different indeed. But I would also say that it is not only the political agendas that should be held accountable. I believe that accountability should be a multi-lane highway traveling in both directions and that all parties involved have levels of accountability that they should in fact meet - including the student.
For me assessment is about the progress of the individual; evaluation is about the progress of an entire group (the program, for example; or a particular project); accountability should have these other types of 'checks' embedded within it. And there should be clearly written statements about what each of these pieces means, how they are interpreted and used, for what purposes, and how they interact together.
Accountability for me presently is not spread around enough, and the
demand for it is entirely too top down.
marie cora
mariecora@hotspurpartners.com
From: rrunyon@mail.unomaha.edu
Subject: [NIFL-AALPD:1867] Re: accountability (from the ed. and learning thread)
Date: Wednesday, January 05, 2005 11:44 AM
Colleagues I think the recent contributions by Eileen and Sally have hit upon essential elements of assessment and accountability reporting. We need guidelines, procedures, checklists, etc. for naturalistic or ethnographic reporting of learning outcomes from the learners' perspectives. Last fall, I conducted interviews with 15 adult learners and recorded some fascinating interview data. These were self-reports of personal experiences and program benefits as perceived by our learners. I concur with the emphasis placed by Sally on the learner's goal/objectives framework. I wonder if there are any social scientists out there who can give us guidance or published references to formal procedures (possibly checklists and interview guides) that would assist us in colleting this sort of data from our learners?
Bob Runyon Literacy Center for the Midlands (www.midlandsliteracy.org)
From: rkmcknight@comcast.net
Subject: [NIFL-AALPD:1868] Re: accountability (from the ed. and learning thread)
Date: Wednesday, January 05, 2005 12:06 PM
Hi Bob,
I agree that Sally and Eileen have shared some excellent insights. To answer your question below, the field of nursing has used checklists and varied types of assessments for some time now. Since so much of nursing involves psychomotor skills, these types of assessments are necessary.
For example, part of the certification process for CNAs (Certified Nursing Assistants) is a demonstration of skills - performance of nursing skills such as reading a thermometer and the like. Evaluators are trained to evaluate candidates and ensure reliability (consistency of evaluation processes) prior to administering the exams.
Nursing also created the first CAT (Computerized Adaptive Test) which is used for nursing licensure. I'm sure there are similar examples in other fields. You can view some info about this at the National League for Nursing website:
http://www.nln.org/testprods/index.htm
While these process are not perfect, trememdous effort and resources have been invested in developing and continually refining them.
Cheers! Roberta McKnight Healthcare Multimedia Design http://www.hcmmdesign.net
From: AWilder106@aol.com
Subject: [NIFL-AALPD:1869] Re: accountability (from the ed. and learning thread)
Date: Wednesday, January 05, 2005 12:13 PM
Bob,
There are two NCSALL publications that might be useful to you, #13 and #2. Beth Bingham who worked on #13 can be reached at the University of Tennessee. Erik Jacobson who worked on #2 can be reached at: Jacobson@air.org.
Both Beth and Erik are experienced social scientists, and both support (through their work) the inclusion of student perspectives in evaluations.
If someone invited them, I think they might both participate in the discussion.
Andrea
From: marie.cora@hotspurpartners.com
Subject: [NIFL-AALPD:1870] Re: accountability (the learners' perspectives)
Date: Thursday, 6 January 2005 6:33 a.m.
Yes, the Bingman papers came immediately to mind:
I've come a long way: learner-identified outcomes of participation in adult literacy programs http://gseweb.harvard.edu/~ncsall/research/report13.pdf The Learner Identified Outcomes study brings learners' perspectives to the ongoing research conversation on outcomes of participation in adult literacy education.
Outcomes of participation in adult basic education: the importance of learners' perspective http://gseweb.harvard.edu/~ncsall/research/op_bingman.pdf This paper suggests that learners have a different perspective on performance than the authors of the Workforce Investment Act.
And I know this isn't from this century (wow, that sounds funny!), but Ways With Words by Shirley Brice Heath cannot be overlooked when thinking about this from the learners' perspective.
About a year ago on the NIFL assessment list, there was a discussion of capturing advancement and progress in students that is not literacy-based, but rather the social pieces - the archives have those posts and many people listed experiences, tools, and processes that they found helpful. You can check out the archives at http://www.nifl.gov/lincs/discussions/nifl-assessment/assessment.html or if people feel it's necessary, I can dig up some of them to forward.
marie cora Moderator, NIFL Assessment Discussion List, and Coordinator/Developer LINCS Assessment Special Collection at http://literacy.kent.edu/Midwest/assessment/
From: eileeneckert@hotmail.com
Subject: [NIFL-AALPD:1871] assessment, from accountability (from the ed. and learning thread)
Date: Wednesday, January 05, 2005 12:54 PM
Bob, I'd be interested in hearing your interview findings. Could you summarize some themes you found, or powerful examples?
There are many techniques in Angelo and Cross' book, "Classroom Assessment Techniques" that can be easily adapted for ABE/ESOL classes and programs of many types.
I've been exploring development and use of rubrics to make aspects of skilled performance explicit--several advantages to rubrics are: 1. They can be developed for as small or large a group as needed; they don't have to be applicable across program types but can be tailored to the individual class' needs if necessary (or they can be applied more broadly). 2. The process of creating and developing them makes the teacher(s) who are doing it think about exactly what's important, what different levels of performance look like, etc. Creating them is an ongoing learning experience for teachers (PD with immediate applicability) 3. When shared with students (as they should be!) they can be a tool to faciliatate teacher-student dialogue about learning and progress, and to focus attention on things to work on--they are a learning tool as well as assessment 4. Students who use them for self-assessment as part of an overall assessment strategy develop greater metacognitive skills and capacity for self-direction; they don't have to always wait for teacher feedback and direction 5. A rubric that's done well can be used for assessment of a broad range of tasks and situations that require use of the skill, so it can be used to assess performance on <real> real-life tasks, not just simulations carried out in the classroom or lab.
When rubrics are used to assess a range of performances of real importance to learners, then the assessments and documents (where there are written documents) can be collected in portfolios and be the subject of reflection, self-evaluation by students, and dialogue with instructors. They are tangible evidence of the outcomes of learning. Students, teachers, and anyone else can see how effective instruction and learning are--accountability!
When teachers and program leaders understand assessment as more than testing, and learn to create their own assessment tools, they don't have to rely on products created by others that may not be just right for them, or at least they can be more effective in adapting and using them. It's along the lines of giving someone a fish vs. teaching them how to fish.
I do understand Sally's reminder that adult ed. is underfunded, but there are already activities going on, and they should be continually moving toward greater effectiveness and responsiveness to learners' needs. Also, until we articulate our own vision, values, and practices consistent with those, we're always going to be in a position of reacting to and responding to, and defending against the imposition of someone else's agenda. Sally's in the position of dealing with the reality that is, and I respect that and think she's in a place where there's a community of practitioners making the best of difficult circumstances, and doing good work in those circumstances. I have decided, for now, to advocate primarily from outside the system and not experience as much pressure as some experience to not make waves, and I think both positions are needed.
OK. I'll stop now, as I've been dominating this thread for days and it's time to shut up for a while. I may have to tie my hands behind my back to keep them from the keyboard.
Eileen
From: sreid@workbase.org.nz
Subject: [NIFL-AALPD:1872] Re: accountability (the learners' perspectives)
Date: Wednesday, January 05, 2005 03:08 PM
Good morning I have been following this discussion and am thankful that at this stage we do not have standardised tests in New Zealand although I am sure there are others in the field here who would welcome them. Thank you Marie for your succinct summary of the differences between assessment, evaluation and accountability. Although these reports from NIACE( National Institute of Adult and Community Education) in the UK are not specifically literacy focussed they are useful because they acknowledge confidence of learners to do or try to do something as an acceptable outcome.
http://www.niace.org.uk/funds/aclf/Catching-Confidence-Final-Report.pdf http://www.niace.org.uk/funds/aclf/Catching-Confidence-Summary.pdf
The main report is 75 pages long so the summary might be a good place to start
Susan Reid Workbase the New Zealand Centre for Workforce Literacy Development
www.workbase.org.nz
www.nzliteracyportal.org.nz
From:bingman@utk.edu
Subject:Accountability catch-up
Date:Monday, January 17, 2005 03:27 PM
This is a somewhat belated response to a thread from a week or so ago. I’m not sure I am a social scientist, but I have thought about the questions you all have been exploring in the accountability thread ever since I decided as a county program coordinator (in the previous century) that our program should use a standardized test to measure the effectiveness of our program and then saw the effects on both students and teachers when some students did worse on a post-test. The more recent work I did with NCSALL [[1]] included the papers Marie referred to and also some work with local programs reported in NCSALL Report # 20 [[2]]. In that action research project three programs developed ways to measure or document their progress and proposed to their states that these be used for accountability reporting (this was early WIA). The states were interested, but did not in the end agree to use these approaches. We did however develop a guide for developing local outcomes documentation that can help programs be accountable to their own constituents even if the documentation is not acceptable to their primary funder. The guide is called How Are We Doing? and can be downloaded from NCSALL at http:/ /gseweb.harvard.edu/~ncsall/teach.html
I do not have or know of a checklist or tool for using student interviews for accountability purposes that will meet federal guidelines. However, the work that EFF [[3]] has done provides tools that could be used both for accountability as defined by our current system (learning gains by level) and accountability re. learners’ and programs’ varied goals. The EFF Assessment Resource Collection found at http://eff.cls.utk.edu/ describes the performance continua and level descriptors for the EFF Standards [[4]]. Performance measures based on the continua can be used to determine progress (or level gain) on a variety of literacy skills. Look at the Guides and the ARC Library for the level descriptors for eleven of the 16 EFF Standards and for information on how they were developed and how they can be used to design as well as assess learning.
Beth Bingman
Return to Accountability
