Idiom
Idiom
Archive (Summer 2002):
Theme: Meeting Individual Needs
CONTENTS
The
Opportunity and Promise of the Standards Movement for Limited English Proficient
Students
by Gerald E. DeMauro
Historical Perspective
As a place of great hope and promise for immigrants to the United States,
New York has a proud history of providing access to many people through its
system of fair assessment. Standardized testing in New York has long been both
a means of overcoming lower expectations for many, and the promise for others
of a system that could enable people to demonstrate their skills through fair
and uniform methodologies.
Also traditionally, though, populations of limited English proficiency (LEP
or ELL) and students who learn English as a second language have had difficulty
with the moving definition of sufficiency of English skills, often translated
into hypothetical constructs as “proficiency,” “fluency,”
or, depending upon the application, “dominance.” Educational programs
designed to meet the needs of these populations have always had to grapple with
defining how much English was necessary for program exit. The definitions ranged
from normative applications of percentiles to periods of program participation
expressed in years.
The implementation of the Learning Standards in New York, and similar developments
in other states, provides an opportunity for LEP students, by offering the promise
of what those of us in measurement call “operational definitions”
to these constructs. Simply put, the exit criteria for programs for students
learning English as a second language can become referenced to the level of
English necessary for achieving New York State Learning Standards in a monolingual
classroom environment. Because New York State has developed tests to yield scores
that are interpretable in terms of achieving State Learning Standards, students
in a monolingual environment can be identified as having, or lacking, the requisite
English skills. Once that is established, a level of performance on the New
York State English as a Second Language Achievement Test (NYSESLAT) (currently
under development) can be determined that represents attainment of the requisite
skills.
Validity Nationally recognized professional standards for testing define validity
as “the degree to which evidence and theory support the interpretation
of test scores entailed by proposed uses of tests.”1 Generally, the interpretation
of the test score is either norm referenced or criterion referenced. Norm referenced
interpretations gauge student performance with respect to some comparison group,
usually the sample of students that comprise the test norms. Criterion referenced
interpretations gauge student performance with respect to the achievement of
some criterion—typically, achievement of a passing or cutoff score.
Note that the tests themselves, strictly speaking, are not intrinsically norm
referenced or criterion referenced. Rather, it is the interpretation of the
scores that is norm referenced or criterion referenced. The design and development
of the tests certainly must be tailored to the intended interpretation, and
it is the support that is brought to the interpretation that constitutes the
validity of the instrument.
In the past, cutoff scores were either preset and assumed when one adopted use
of the tests, or were set with respect to some expert judgment of what constitutes
competence, proficiency, fluency, or another hypothetical construct. For LEP
(ELL) students, the relationship between passing the tests and performance in
the monolingual classroom depended heavily on how well defined these constructs
were about the ability to perform in a monolingual English instructional environment.
For example, suppose a test was used to determine eligibility for mainstreaming
into the monolingual English instructional environment, and that eligibility
was decided by achieving a level of scoring defined as “proficient.”
Then the use of the test for that purpose would be valid to the extent that
there was support for the two possible interpretations of test scores: first,
that students achieving that level of scoring possess the English language skills
needed to succeed in the mainstream environment, and second, that students who
score lower than that level do not possess that required level of skills.
New Possibilities for a Criterion Referenced System With the adoption of the
New York State Learning Standards, it has become possible to operationally define
what it means to be successful in the monolingual English instructional environment:
achievement of New York State Learning Standards. This provides an unprecedented
opportunity for LEP student assessment and, as a consequence, for an instructional
program that is informed by criterion referenced interpretations of test scores.
The design of the NYSESLAT development incorporates field testing of both English
proficient students in a monolingual instructional environment and LEP students.
By referencing the students’ performance on state tests of the Learning
Standards, it is possible to identify the skills possessed by the English proficient
students that are not possessed by the LEP students and that are related to
achievement in the monolingual environment. Through careful delineation of the
skills measured by each test question on the NYSESLAT, test scaling procedures
will identify which skills are associated with the different levels of performance.
These procedures thus enable evaluation of those skills not only at a single
point in time, but also as students mature in grade level and in English proficiency.
Moreover, available standards for English as a Second Language (ESL) describe
Beginning, Intermediate, Advanced, and Transitional levels of skills for LEP
students.2 Test items can also be classified with regard to these levels, and
performance on the NYSESLAT can be gauged to progress through each of these
levels, identifying student strengths and weaknesses all along the way.
The Promise
As the state continues development of the NYSESLAT, an opportunity exists
to bring this important meaning to the test scores: what skills are needed for
performance at each level of ESL proficiency, and what English language skills
are needed to enable a student to achieve State Learning Standards. This represents
a new level of assessment, one that promises to provide considerable support
to the interpretations made from test scores. Moreover, by working with teachers
and ESL experts and going through a deliberated process involving state-of-the-art
scaling technologies, there is an opportunity to provide information to ESL
teachers that can only help improve program design and instructional intervention.
References
American Educational Research Association, American Psychological Association,
National Council on Measurement in Education. (1999). Standards for educational
and psychological testing, p. 9. Washington, DC.
Author. Office of Bilingual Education, the State Education Department, the University
of the State of New York. (2001). Learning standards for English as a second
language: Building the bridge (draft). Albany, NY.
Gerald E. DeMauro, coordinator of state assessment at the New York State
Education Department, drew up the original design for the NYSESLAT.
updated
on
October 4, 2004