Assessment in Family Literacy

From LiteracyTentWiki

Back to Assessment Information

The following discussion took place on the NIFL-Assessment Listserv between December 20, 2004 and January 18, 2005. The topic is Assessment in Family Literacy Programming.

Summary

The discussion opens with questions pertaining to the types of assessments used in family literacy programming. An initial response notes the complexity of family literacy, and that it is very difficult to bridge early and adult education in instruction. The components of family literacy are outlined as follows: Early Education, Parenting, Parent Child Interactive Literacy (PACT), and Adult Education. Descriptions of a couple of programs and the clientele they serve are described in terms of noting the challenges inherent in serving this dual population. Various assessment tools are discussed and their drawbacks are noted. A point is made regarding the difficulty of connecting outcome data from the assessments to good instructional practice, which is required in federal funding. Links to 2 resources on family literacy work and assessments are provided.

Discussion Thread

Hi everyone,

I'm wondering what tools and processes people are using in family literacy programs?

Do you use the NRS required ones and supplement with other types of tools? Are there assessment processes/tools specifically designed forfamily literacy? Actually, do family literacy programs get the same funding stream as the ABE programs required to report to the NRS?

I've been looking around for resources in this area too. I found this one, which has some good stuff in it, but a lot of it is focused on the overall evaluation of program, rather than on individual assessment. Can anyone comment on the usefulness of this resource? Are there other resources out there that focus on assessment in fam lit?

Assessing Success in Family Literacy and Adult ESL (Revised Edition)
Carol Van Duzer and Daniel D. Holt, Editors [1]

Thanks,
marie cora
Moderator, NIFL Assessment Discussion List, and
Coordinator/Developer LINCS Assessment Special Collection [2]

Marie,

Assessment in Family Literacy is an issue that I've spent the past two hours responding to as a State Evaluator. The complex nature of family literacy makes it hard for one program coordinator to act as an instructional leader across the domains of early and adult education. Family literacy has four program components; Early Education, Parenting and Parent Child Interactive Literacy (AKA, PACT), and of course Adult Education.

Adult Education is in relatively good shape with the TABE and CASAS used routinely, in keeping with the state's NRS requirements. I'm in touch with 63 local programs as the state evaluator in two states. In Missouri, programs use either the Peabody Picture Vocabulary Test Revised (PPVT), or the Preschool Language Scales (most current). Pat McKee, USDEd Even Start program officer is quite comfortable with programs using the PPVT receptive language subscale. It's been used nationally for outcomes evaluation. As for Parenting Education, Diane DiAngelo and Doug Powell at RMC developed the PEP. It is a solid tool that informs program component design and contributes to staff development needs assessment.

In Missouri, we use an over-simplified tool call the Parent Observation Guide to assess parent-child interactive literacy (PACT). It gets at the five key components of SBRR.

The key to all of these assessments is to connect the data generated by administering them to good practice. Something sorely lacking in many venues. Further professional development in test administration and instructional implications is sorely needed if Even Start can hope to produce the outcomes evidence demanded by the administration.

Best,
Jeri Levesque, Ed.D.

Jeri,
Could you furnish more information about where to obtain the PEP and Parent Observation Guide? We are a small community based organization and are in the middle of searching for tools to evaluate PACT and Parent Time.

Thanks so much,
Christi Graff
Education Director
Learning Lab, Inc.

Our EvenStart Fam Lit program is an anomaly in Montana. It often has a preponderance of L2 English adult and child students living in mono and dual-language/culture households.

As the adult ed ESL instructor, I generally have the adult EvenStart ESL students in my classes when they are not participating required EvenStart activities. Many of the L2 English parents have attained (much) higher L1 literacy and life skills levels than have their L1 English counterparts, which adds another interesting twist.

The L2 children remain in the EvenStart classroom program throughout the day. The language/culture/L1-literacy level mix makes our EvenStart program unique in our state - a fact that leads to an official assessment dilemma.

As a state, we use the original BEST, the BEST Plus and the TABE - all of which are designed for adults. However, if one reviews the literature, you will find that the TABE is designed for L1 English students not L2 English students. Thus part of the general assessment bind.

The more significant official "bind" is that the L2 children's language/cognitive skills are, perforce, being assessed as though they were L1 English kids linguistically and cognitively.

Here are a couple of real case scenarios to illustrate what is normal for our L2 kids:

One of the five-year-old boys is operationally bilingual (English-Japanese). However, when he visits relatives in Japan for any extended period of time, he returns operationally mono-lingual (Japanese) for a while and, then, becomes E/J again.

A newly arrived, extremely bright, 4-year-old, Taiwanese child was going through both the "silent period" in second language acquisition terms and culture shock at (1) being here and (2) being separated from her Mom. Although we, as staff, were certainly aware of her personal dilemma, it was assessment time with the arrival of CLIO.

The EvenStart program is involved in a pilot curriculum/assessment study called CLIO. Although CLIO does have language/cognitive development assessment capabilities in English and in Spanish, those are only two of the many native languages spoken and understood by our EvenStart kids. Out of nine children, there were no L1 English or L1 Spanish kids in the Evenstart room the last time the CLIO assessor was on site. Such a student mix and the resulting assessment dilemma is surely normal for programs in other states. So:

How do you assess the language and corresponding cogitive and literacy development of your non-native, English-speaking EvenStart children in such manner that provides valid pictures of their development in these domains as L2 children and, of course, aligns with your mandated reporting system?

Is there much emphasis on staffing family literacy programs with instructors who are trained in child second language acquisition and literacy development?

Sincerely,
Terrence Kelley
Missoula MT ABLE

Hello all and Happy New Year.

Thanks for your post Terrance. I had a couple of comments:

Can you explain this a little more for us: "It often has a preponderance of L2 English adult and child students living in mono and dual-language/culture households." Do you mean that second language speakers in your programs live in households with Americans there who only speak English? Or they live in households where more than one language is spoken? I'm not sure how that situation is different from others around the country.

It appears from your post that many of the adults in your programs are highly educated, or at least educated as compared to.....whom? I'm not sure with what L1 group you are comparing your students with. Also, do you find that there are differences among the participants who have more education with those who have less? I wonder if that has anything to do with retention, student or program success, etc.

I have my Assessment Specialist hat on now: You should never use the TABE with any student unless he/she is a native English speaker or the person's education/literacy level is high enough (past some ESOL level) for the TABE content material to be appropriate for that person. Likewise: assessments developed for native English speakers are generally NOT appropriate for second language assessment (there are exceptions to this rule when particular adaptations are made - for example, the DAR was developed as a reading assessment for children, but it has appropriate applications for use with adults).

You must know what a particular assessment tool was developed for and how it was developed before you can decide if it might be appropriate for the population you are working with. Without this rule of thumb, I feel that assessments should be thrown right out the window (which is only my opinion). (Actually, there are a few rules of thumb with assessment that I feel MUST be adhered to or I personally wouldn't consider the assessment valid in the least; following standardized test administration to the letter is one of course).

Finally Terrance, can you tell us more about CLIO? What does it stand for and who developed it? You said that there is an assessor who comes on site - what is that like? Does your staff get to learn CLIO and administer it, or it is not that type of assessment?

Your second to last question I interpreted this way, so correct me if I'm wrong: "where is there an assessment that is equally as helpful for me and my students in the classroom as it is for the funder to see the progress of our students and program?" In other words, are you asking about the disconnect between the classroom assessments and the NRS?

Your final question is really interesting: do other famlit programs have elementary second language instructors? I would love to hear responses to that as well.

Thanks again,
marie cora
Moderator, NIFL Assessment Discussion List, and
Coordinator/Developer LINCS Assessment Special Collection [3]

Marie,

Thanks for the caveat regarding use of the TABE for ESL learners in family literacy programs. The CASAS provides more valid information and can help gauge the impact of instruction on the real world needs/goals of the learners. Our state funds a number of family literacy programs for ESOL learners. I evaluated one in a metropolitan setting. It served a diverse community of families including refugees from war torn nations. The range of communication skills was profound. The CASAS scores help define the class. Pre-test scores ranged from Beginning Literacy/Pre-beginning Literacy ESL (score 169) to a high of Professional Skills (245 - no post test). The mean pre test score of 185 indicates a class composed of High Beginning ESL suggesting very limited English proficiency. Post-test scores increased to a minimum of Low Beginning (180) to a high of Professional Skills (248). The mean post-test score of 216 indicates the class increased English proficiency to the High Intermediate ESL level. The scores demonstrate significant overall class achievement for the ten adults who regularly attended the program.

When the program began, Intake data demonstrated the primary goal of most learners was to learn enough English to secure steady employment and housing. Many parents also wanted to insure school success for their young children. The program was located in the elementary school where school aged children attended and a preschool component served the preschool children on site. Early results were a little confusing. As CASAS scores increased people, especially men, left the program. Turns out, their English language skills improved (oral) enough to get jobs. Eventually the program had to adjust the schedule to accommodate working adults. The instructor kept a balance between lessons on English speaking and learning to read and write in English. For some women, it was their first time in a school setting. Others were professionals or spouses of high level professionals recently immigrated to the country.

Our assessments included home literacy behaviors (literally none of the families owned a book at the beginning of the program) and parental involvement in schools (100% of the parents were actively engaged in school activities). The most glaring findings of the evaluation echo John Coming's study on adult education (MassINC, 2000), The need for intensive educational services in the city outweighs the availability of program with English as Second Language instruction.

  1. 1. There are a limited number of literacy programs and services available for those who demonstrate limited English proficiency. Currently, only 10 of the 79 City's Adult Education and Literacy (AEL)

programs have teachers who can instruct English as a Second Language (ESL) population. The ratio of roughly 1:8 ESL adults to programs offering ESL does not correlate with total AEL enrollment of roughly 1:4 ratio of ESL to others. The City's AEL identifies 2,309 adults of the 8,689 enrolled as ESL students. The data suggest that 25% of the City's AEL students are serviced by 12 % of the programs. Interestingly, the only AE: programs that offer classes on Saturdays or Sundays are ESL sites. Although this would seem to accommodate the variable work schedules of adults, the data collected from the ESL family literacy parents determined that no adults were willing to participate in weekend events.

  1. 2. Adults who participated in the program made significant gains on the CASAS after 100 hours of instruction.
  1. 3. All parents who engaged in family literacy demonstrated parental involvement in their children's education.

This and more from our formal (CASAS) and informal assessment data. In response to a question you posed recently, there is no single instrument that can adequately measure the impact of family literacy on the family as a whole. Family literacy is a complex literacy sytem. Infants, toddlers, school-aged, adolescents, and adults all have their own bailiwick of cognitive, language, and early literacy assessment instruments.

Best,
Jeri Levesque, Ed.D.
State Evaluator, Missouri & Kentucky Even Start Family Literacy Programs

Marie,

Here is another assessment, not my favorite, it deals more with who we serve rather than what is the effect of family literacy on language and literacy learning. He considers it a BREAKTHROUGH in assessment, but I have not endorsed it in MO.

Ron Taylor, Reading/Study Skills Coordinator for the Eastern Jackson County Campuses of the Metropolitan Community Colleges in Kansas City MO. For the last few years, has been working with several school districts and adult programs to develop an instrument to assist us in our important work with families.

THE FAMILIA INVENTORY is an easy-to-use diagnostic questionnaire on software for use in family literacy programs.

The inventory has been tested with over 2000 families from varied economic, educational, and cultural backgrounds. The results have demonstrated the inventory to be a dependable and reliable instrument in designing plans for family effectiveness in literacy development. The inventory can be used with low-literate or literate families.

THE FAMILIA INVENTORY assesses ten activities involving literacy in a family, including:

     *Family Work and Play

     *Parental Modeling of Reading

     *Shared Reading

     *Television Use

     *Writing Activities

     *Library Use

     *Practical Reading

     *Support of School

     *Verbal Interaction

     *Extended Family Interaction

The inventory is standardized for families in the following groups:

     -Families with children ages 0-5

     -Families with children ages 6-9

     -Families with children ages 10-12

     -Families with any age combination of children

     -EuroAmerican families

     -African-American families

     -Hispanic-American families

     -Total Population

The inventory is available on software in both English and Spanish with Form A (Pretest) and Form B (Posttest) for use before and after a program of intervention.

The software is covered by copyright.

An e-mail request for a sample disk including your name and school program should be sent to: taylor@pennvalley.cc.mo.us

Jeri Levesque, Ed.D.
State Evaluator, Missouri & Kentucky Even Start Family Literacy Programs

From the PEN Weekly Newsblast:

PROMOTING FAMILY INVOLVEMENT IN CHILD LEARNING & DEVELOPMENT

Harvard Family Research Project has just released the Winter 2004/2005 issue of "The Evaluation Exchange." The latest issue's topic is evaluating programs that promote families' involvement in children's learning and development. The new issue compiles the current knowledge base on family support and involvement programs and provides a continuous perspective on family processes surrounding children's learning and development, from a child's early years through adolescence. Articles in the issue address the challenges of evaluating family programs, including the need for conceptual clarity, methodological rigor, accountability, and contextual responsiveness. In an interview with Jeanne Brooks-Gunn she reflects on breakthrough findings and new directions for research, evaluation, and practice in family-focused interventions. Rounding out the issue are examples of ongoing evaluations of parent leadership and organizing programs that are working to ensure that schools serve all children at high standards. The issue is available in full online [4]

Dianna Baycich
Ohio Literacy Resource Center
Research I Bld.
1100 Summit St.
PO Box 5190
Kent State University
Kent, OH 44242
330.672.7841 330.672.4841 (fax)

Hi folks,

Several of you have requested (info on The Parent Observation Guide from Missouri). I have contacted Jeri Levesque, who is traveling right now and so it's difficult for her to reply. She let me know that she has asked LIFT-Missouri (Missouri's Literacy Resource Center) to post this information at their website. [5] (not sure if it's there yet....)

Thanks,

marie cora
Moderator, NIFL Assessment Discussion List, and
Coordinator/Developer LINCS Assessment Special Collection [6]

Retrieved from "http://wiki.literacytent.org/index.php/Assessment_in_Family_Literacy"