Questions
From LiteracyTentWiki
The following is a listing of the questions posed by both the guests and participants during the discussion. The questions have been excerpted here. For a full reading of the discussion, please go to NASDiscussion.
Back to Measuring Literacy: Performance Levels for Adults, Interim Report
Date: June 14, 2005
From: Judy Koenig and Stuart Elliot, Guests
The following were posed by the guests, Judy Koenig and Stuart Elliot, as starting points for discussion or clarification.
- Questions concerning the lowest and the highest categories of the literacy levels; specifically, questions concerning the difference between the Non-Literacy in English and Below Basic categories, and comparing the percentages of folks falling within these categories between the 1992 survey and the 2003 survey;
- An explanation of the Bookmark Standard-Setting Procedure that was utilized in the study;
- An explanation of the Quasi Contrasting Groups Procedure, that was utilized to adjust the cut scores;
- Why the Committee chose not to determine a level of performance as "necessary" or "sufficient" for effectively functioning in life; as an example of this, note that the terms used to label the levels are purely descriptive, rather than indicating a particular judgment about that level;
- An explanation of how the Committee's process differed from the original process for determining the levels and cut scores for the 1992 results;
- The recommendations that the Committee puts forth based on its research.
Date: June 20, 2005
From: Marie Cora, moderator, mariecora@hotspurpartners.com
I'm going to start us right off by asking our guests to respond to the first item that appeared in the "suggested preparations" section of the announcement, which I've copied below:
"Questions concerning the lowest and the highest categories of the literacy levels; specifically, questions concerning the difference between the Non-Literacy in English and Below Basic categories, and comparing the percentages of folks falling within these categories between the 1992 survey and the 2003 survey;"
I know that the 2003 survey included additional components to try to gauge the lower ESOL levels, so I'm interested in hearing about that. I'm also interested in hearing about the Non-Literacy versus Below Basic categories.
Date: June 22, 2005
From: Debbie Yoho, Columbia, SC, dwyoho@earthlink.net
A primary concern of many direct providers of literacy services like me is the accountability standards of the National Reporting System in relation to the lowest level learners. Do you know if the new data will be used or could be used to establish two very low levels in the NRS system?
Date: June 22, 2005
From: Tom Sticht, El Cajon, CA, tsticht@aznet.net
I'm wondering what responsibility the National Academy of Sciences has in addressing the shortcomings of the new ALL and informing policymakers, news agencies, and the general public about the misleading ALL report. This is particularly important since the ALL report also disseminates faulty data about the U.S. I'm also wondering what the responsibilities of the larger community of adult literacy researchers and scientists are in trying to bring the misleading statements of the ALL report to the attention of the general public.
Does anyone from the NAS/NRC/BOTA report on performance levels for adult literacy have any thoughts about any of this?
Date: June 22, 2005
From: Miriam Kroeger, Arizona, MKroege@ade.az.gov
Has there been any correlations of the "standard" assessments (TABE, CASAS,) and the GED Tests, including the English Proficiency test (Test 6) to the 2003 NAALS, or for that matter to the NRS?
Date: June 23, 2005
From: Marie Cora, moderator, mariecora@hotspurpartners.com
1. Please define "Demand-Side Analysis"; give us an example, perhaps one that compares demand-side analysis to other types of analyses. What sort of analysis was utilized in determining the original results of the NALS and the NAAL? Do you know why?
2. The way the test was developed does not support standards-based inferences being made about the data. However, when the data was reported, inferences were in fact based on some set of standards. Is it possible to re-interpret the data based on a different set of analyses? Do you feel that the results would be very different if they were? Would this be a useful exercise or not?
3. The NALS and NAAL are examples of assessments that try to get at evaluating program needs. So how do the results relate to the individuals that took the tests? Can interpretations be made of the individuals then, if the assessments were designed to examine the larger program needs? And if literacy is a collaborative process in many instances, and many programs structure the students in groups (groupwork), wouldn't the validity of testing individuals be problematic? What is the relationship between testing the individual and inferring that programs who serve some particular population might be adequate or not?
4. Why was the health section done differently than the other sections? Wouldn't that affect the validity and reliability of the rest of the test?
Date: June 24, 2005
From: Andrea Wilder, AWilder106@aol.com
I am interested in knowing how you came up with the definition of literacy in the first paragraph of the Executive summary. Are these "common sense" definitions, or are you citing conclusions from other work?
Also, it looks to me, going over the NAAL levels, TABE levels, NRS levels, there is an attempt to make these congruent; I hope I have been reading accurately. Is this true?
Date: June 24, 2005
From: Andrea Wilder, AWilder106@aol.com
Did the Committee discuss a Spanish language track for literacy?
Date: June 24, 2005
From: David Rosen, Jamaica Plain, MA, djrosen@comcast.net
1. Do we have any evidence that the NALS data have been used by any audiences to improve adult literacy education services? What exactly, do you think, is the added value of these studies? For example, do you think the NAAL will be more useful to practitioners and policy makers than the NALS? If so, why?
2. In addition to literacy levels, "need, for literacy in our society," there are two other related questions whose answers might be of more use to practitioners and policy makers: a) what is the actual unmet demand for services to improve basic skills? That is, how many people who need these services have stepped forward and are on waiting lists? and b) what is the "latent demand," that is, the number of people in need who -- if they knew adult literacy education services were available -- would step forward to enroll in adult literacy education programs? Have there been -- or is anyone planning -- national studies of demand or latent demand?
