Assessment in Workplace/ESOL
From LiteracyTentWiki
Back to Assessment Information
The following is the assessment thread of a larger discussion focused on Workplace and ESOL issues that took place on the NIFL Workplace Literacy and Adult English Language Learners Discussion Lists September 18 - 22.
To view the full archives of this discussion, please go to: The NIFL Workplace Literacy Discussion List [1]
or to
The NIFL Adult English Language Learners Discussion List [2]
--- How can teachers and administrators develop curricula and materials that meet the needs and goals of the learners in class as well as the needs and expectations of employers?
I have had about six years experience in teaching and developing curricula for Working English Language Programs in Northern Virginia. Factors which improve overall success rate included: worker needs assessment, supervisor needs assessment, shadowing employees, developing task analysis (what does the worker do? When does s/he use English to do it?), and worker and supervisor incentives (paid class time, additional staffing, etc.)
When I was teaching, I worked with primarily authentic teaching materials (which often meant searching the workplace on my own time). I used written materials found on the job whenever possible (Human Resource forms, Quality control forms, safety forms, etc.) to build lessons. Problem-posing communicative activities based on supervisors' and students' needs assessment were lively and effective. Writing summaries of these and other OTJ activities helped students learn to keep more accurate records and/or formulate their own thoughts in a more organized and timely fashion.
For literacy students, I used pictures taken at the worksite and vocabulary building for dialogs first and then writing true stories.
A curriculum with a basic framework (such as introductions/small talk, customer interaction, management interaction, coworker interaction, safety and quality control) needs to be super flexible to be effective.
I would venture that many English language learners we work with in this environment thrive in non-traditional classroom settings using a wide variety of teaching strategies. Sometimes the 'how' might be more important than the 'what.'
Now for evaluation, or 'the needs and expectations of funders' --- progress is a lot more difficult to measure easily in the workplace. Many of our ESL tools for assessment are inadequate at the K-12 and adult education level. Workplace ESL throws more variables into the mix. Funders, be they government or private, must have rationale for refunding.
I'm curious about what instruments people are using in current Working English Language Programs to measure progress in English language learners.
Do you like the instruments you are using? Why or why not?
Sharon McKay
Washington, DC
In terms of assessment, I think the tools should come from the workplace itself. Have students complete necessary forms that they might need to complete in the workplace. Try mock discussions that they might have with customers. Practice telephone conversations where they are answering calls and taking messages. See how they take messages for others. One time I was working with a group of adult students whose greatest fear was answering business calls and taking messages. We worked on different scenarios of calls they could receive and how they could respond to them. I think the best thing though is once you get past the basics that are common in most places of employment (i.e. answering telephones, taking down information, etc.), the instructor must become familiar with what the company does and with what the employer wants from his/her employees. Once the instructor knows this material, he/she should use that for assessment. It's a prime use of authentic assessment, modeled for the particular business in which the adult ESL student will be working. I would also recommend speaking to the student, addressing his/her comfort level, to add to the assessment. Confidence is key. I find that many of my ESL students know the material but are so afraid of performing and think that they do not know it that they hold back. It is important to address their concerns when you are assessing them. They may also tell you of an area that they feel they need additional practice. Listening to students is the major part of teaching in any environment.
Alison Cochrane
ESL Adult Instructor
New York, NY
One quick thing – and apologies if this has been mentioned – I’m sorting through a huge pile of messages from multiple lists.
When I taught ESOL to university maintenance workers, I used a time1, time 2 survey to assess one piece of the overall progress. I’d asked supervisors what they thought workers needed to learn, where improvement was needed and then, after the first module (about 8 weeks) asked if there had been any progress in those areas. So, for example, it would look something like that
Workers write better/clearer work orders:
not at all
somewhat
very much
Fewer mistakes are made in following work orders
Based on whatever it is the learners or the funders/employers set as goals, this survey provides one piece of the larger assessment picture. We also used the BEST test to gauge language progress – insofar as a test can give that information after a relatively short instructional stretch (4 hours/week over 8 weeks).
Janet Isserlis
The most fun I ever had designing a class was when my community college got a contract with a local foundations company. It was my first experience so I knew no limits. I walked through the hiring process with all the forms; we did the swipe card for clocking in; I talked to everyone who had contact with the students--what questions did they ask, what information did they need from the students; what were emergency procedures; Then I went out on the worksites to see what the students did on the job. I learned everything I could about laying foundations. They gave me samples of all the materials the students used on the job and told me what they did with them. They provided with vocabulary lists. I worked with the office manager on organizing the units. It was powerful because it was practical.
Let me echo, however, the importance of the teacher. It must be one who catches the spirit of the workplace. This is no place for someone who is mired in a book, a grammar lesson, or the "way we've always done things."
Kathryn Quinn
Frederick County Public Schools Adult Education
Colleagues,
Over the years I have seen some great program- and teacher-made workplace-focused, contextualized English language learning curriculum materials. Some of it stays at the company. Some stays with the teacher. Most gets lost when teachers leave, and programs end. I am interested in trying to address that problem, to build workplace (English language and other) curriculum that can be more widely used by teachers in the classroom and online, and online by students. Here is the first draft of my criteria for the solution to be developed. I would be interested in having our guests' -- and others' -- reactions to the criteria.
1) Industry-specific, for example: healthcare, hospitality, built on accepted industry skill standards
2) Tailored to specific jobs within the industry, including specific vocabulary related to the job, but not limited to those jobs
3) Built on solid, adult education content standards: Perhaps someone has taken some of the best state level ESOL content standards and built a set of "meta-standards" for ESOL content which can be used nationally. Does this exist? EFF and SCANS are possibilities, too, but they are not ESOL content standards. What would you recommend here?
4) Built nationally (or internationally) but easily adaptable by teachers for local situations and for specific learners. In every industry there are standard writing tasks -- certain kinds of notes, reports, communications to the next shift. There could be small pieces of instruction on how to write a shift change note in manufacturing, but this would need to be contextualized by a local teacher to the particular manufacturing context. If the lesson were available, for example on a wiki, a teacher could easily modify it. We might also have a record of the modifications and have lots of variations that others might use: shift notes for auto manufacturing, shift notes for pharmaceuticals manufacturing, shift notes for nurses, etc. The new technology offers workplace basic skills curriculum the opportunity to develop very tailored models -- which are also available to others in the industry who want to sue them.
5) Available online as lessons which students could use directly online themselves or which teachers could use in face-to-face (classroom) learning or online learning, using learning object standards so that small pieces of instruction can easily be selected and recombined, so for example, the instruction could be used for "just-in-time" language learning as well as the longer-term language learning needed to progress to proficiency in English.
Your comments, additions, suggested changes?
David J. Rosen
djrosen@comcast.net
Janet
Can you please explain more on how you decided to use the BEST test with the university maintenance workers? How did you apply the test? Do you have to be trained in BEST applications and scoring? What does it entail?
Thank you.
Alison Cochrane
ESL Instructor
New York, NY
Janet,
I am also interested in how you used the BEST test(s). We currently use all three BEST tests in Tennessee. We have difficulty showing level gains using BEST in workplace settings since we often find that employers want short-term classes that produce unrealistic results. I would like to know which test(s) you used.
Thanks,
Sandra Fugate, ESOL PD Coordinator
UT Center for Literacy Studies
All
I used the Oral BEST; this was also in the late 80's, so there was no NRS requirement. If memory serves, the state may or may not have partially funded the work - and the employer certainly bore much of the cost, but I don't recall either entity (state office of adult ed or the employer) being particularly concerned about the test outcomes. The larger question, though, is what - if any - standardized assessment can help measure gains in very short term, content or goal-specific classes. My question would be around why a standardized test (designed to measure gains over time) would be necessary. Do we need to find ways to create yet another standardized instrument to capture shorter-term gains?
Janet Isserlis
I agree with you Janet. I question how a standardized test can measure specific content related gains, particularly when these gains are expected to develop in a relatively short course of time. My time with my students is two hours a day, four days a week, for two and one-half months. If they pass the course, I wonder how much I truly prepared them for their next transition. Did they process some skills? Did they learn? That's why I try to differentiate my assessments, but I find many of my ESL adults to be very test-oriented. They believe numbers and grades determine how well they are learning.
Alison Cochrane
If folks feel like a test helps them see their progress, one approach might be to integrate that into the classes – to ask students to develop questions that address what they’ve been learning, so that the jointly construct a test. This, in combination with other forms of ongoing assessment can help make learning visible to learners – but likely doesn’t answer the outside requirement (where it exists) for a standardized instrument.
Are many workplaces requiring these instruments?
Janet Isserlis
At many companies, we give both types of tests - a job-related assessment, and a standardized test which the state requires (whether it is relevant to that student's success on the job or not).
Mona
Mona Baker, Coordinator
Workplace Basic Skills
phone: 704 330 4554
fax: 704 330 4420
Here's how we're hoping to use the BEST in workplace situations:
To collect baseline data and measure progress if a) length of course of instruction warrants such and b) if state AE funds are being used to pay for services (contact hours are important to formula funding)
For the employer and employees, however, we advocate using multiple alternative assessments that can measure changes in work-related performance and behavior. Sandra Fugate, TN uses one of the same resources we are using here in Texas; it contains many examples of alternative assessments.
Barbara Tondre
The state of NC requires standardized testing. Additional testing is up to the discretion of the instructor and/or company.
Mona
Mona Baker, Coordinator
Workplace Basic Skills
phone: 704 330 4554
fax: 704 330 4420
