NYS TESOL Publication: Idiom
Idiom
Archive (Spring 2003):
Theme: Equity and Excellence
NYS
ESL Learning Standards and NYSESLAT: Equity and Excellence—an
Opportunity for ELLs
by Carmen Perez-Hogan
I. Overview of ESL Standards/ Assessment for Limited English
Proficient/English Language Learners (LEP/ELL)
A major goal of the New York State Education Department, Office
of Bilingual Education has been the development of an instrument
to measure growth in English proficiency of limited English proficient
students statewide. Following is an outline of the most current
events leading to meeting this goal.
A. Historical Perspective
1. The New York State Board of Regents approved “Strategies
for Raising Standards” in 1996 as follows:
a. All students including LEP/ELLs must meet
standards in seven core subjects including English.
b. All students including LEP/ELLs must pass
Regents tests in five subjects including English, in order to
graduate from high school.
c. Regents tests in the content areas are translated
into the five top languages.
d. The required English as a second language
and English language arts instructional time is increased.
2. The New York State Committee on Testing of LEP Students recommended:
a. ESL Achievement Test: Develop the
New York State English as a Second Language Achievement Test (NYSESLAT)
based on the New York State English as a Second Language standards.
New York State Education Department in collaboration with ETS
develop the test.
b. LEP Identification Test: Upgrade the
New York City Language Assessment Battery (LAB) and align it with
ELA standards. New York City Department of Education completes
this revision.
B. ESL Learning Standards
1. New York ESL standards and performance indicators are developed:
· New York State Education Department
· New York City Bilingual Education Technical
Assistance Center
· Statewide steering committee and regional
workshops
· Center for Applied Linguistics
2. New York Sate ESL Standards are:
a. LEP students will listen, speak, read and
write in English for information and understanding.
b. LEP students will listen, speak, read and
write in English for literary response, enjoyment, and expression.
c. LEP students will listen, speak, read and
write in English for critical analysis and evaluation.
d. LEP students will listen, speak, read and
write in English for classroom and social interaction.
e. LEP students will demonstrate cross-cultural
knowledge and understanding.
C. Development of NYSESLAT
1. ESL and bilingual teachers were trained on how to write
items by SED and ETS (2001-2003).
2. NYSESLAT was field tested (2002-2003).
3. Districts order NYSESLAT tests from the SED at no cost to districts
(February 2003).
4. Training of trainers in districts on test administration of
NYSESLAT (February-March 2003).
5. Training of teachers in districts on scoring of the NYSESLAT
(March-April 2003).
6. Standards setting (April 2003).
7. First administration of NYSESLAT (May 2003).
8. Local scoring of the NYSESLAT (May-June 2003).
D. Description of the NYSESLAT
The NYSESLAT assessments reflect the New York State Learning
Standards for both English as a Second Language and English Language
Arts. The assessments have the following features.
NYSESLAT provides five grade-level assessments:
· PreK-1 · 2-4
· 5-6 · 7-8
· 9-12
Each grade-level assessment includes all four language modalities:
· Listening ·
Reading
· Speaking ·
Writing
NYSESLAT
Development
by Gerald DeMauro, SED
New York State’s adoption of learning standards
presents an extraordinary educational opportunity for the state’s
limited English proficient (LEP) children. Before adoption, hypothetical
constructs like fluency or proficiency served as benchmarks for
English language acquisition. The demands of each grade level
were locally determined and based on what could be identified
from state assessments and from state and federal requirements
not tied to specific content and skills domains.
The learning standards and their levels of specification
require students to achieve a specific body of knowledge and skills.
The federal No Child Left Behind legislation further strengthens
the preeminence of this body for all children, and enables the
states to define proficiency as the English language—necessary
for success in any learning environment: bilingual or monolingual.
Thus, if your English is strong enough to understand spoken classroom
interaction, participate in that interaction, read assigned materials,
and produce writing expressing your knowledge and skill, then
you have achieved proficiency. By providing a measure of these
capacities, success on the state assessment system becomes a criterion
of proficiency.
The development of the New York State English
as a Second Language Achievement Test (NYSESLAT), then, is linked
to the New York State assessment system, and degrees of success
on the NYSESLAT indicate readiness for achievement in any of the
three learning environments cited above. This is an enormous step
forward for the education of LEP children. It changes the whole
psychology of the programs and of instruction—from one of
catching up by addressing a deficit (remedial) to a program of
preparing the children to achieve the learning standards, (developmental).
As of this writing, the NYSESLAT is in production
for a May 2003 administration. Consistent with the design focusing
on achievement of the learning standards, it has been field tested
on both English proficient students and LEP students. Much of
the technical work is ongoing. This includes the vital linking
to the English Language Arts examinations in grades 4 and 8, to
eliminate the need for LEP students to take these tests. It is
also being vertically scaled to allow comparisons from year to
year of progress through the grades. Standards are also being
set for program exit, and for growth from beginning proficiency
in English as a Second Language to advanced proficiency in English
as a second language. What follows is the developmental work that
has been achieved thus far.
Scoring
of the NYSESLAT Field Test Responses in Speaking and Writing
by Doug Fiero, ETS
During the fall of 2002, K-12 students in New
York State schools took field tests in reading, writing, listening,
and speaking. The field tests were given to students at six grade
levels: 1 and 2, 3 and 4, 5 and 6, 7 and 8, 9 and 10, and 11 and
12. The reading and listening tests were in a multiple-choice
format. The writing and speaking tests were performance tests
that required scoring by teachers, including a considerable number
of New York State ESL and bilingual teachers.
There were two steps in scoring the speaking
and writing assessments: first, the range-finding process, in
which benchmark and sample responses were selected at each grade
level; then, a week later, the actual scoring of the student responses.
On January 3-February 2, 2003, in Albany, the
Educational Testing Service (ETS), which had created the NYSESLAT
assessments and were managing the range-finding and scoring sessions,
conducted training for the exam leaders and teacher scorers.
The range-finding sessions were conducted in
Albany on February 1 and 2, 2003. For these sessions, the scoring
leaders at each grade level met with a small group of teachers
to select benchmark responses and sample responses at each point
in the scoring scales.
A benchmark response solidly and definitely
reflects a particular score on the score scale. The sample responses
include a range of responses that show different ways that students
can achieve a particular score. The scoring leaders and the teachers
who helped them listened to the students’ taped speaking
responses or read their written responses to the questions in
the writing test, identified good candidates for benchmark and
sample responses, discussed these responses until firm agreement
was reached about the scores, and prepared the examples of student
responses that would be used to train scorers at the actual scoring
sessions for the NYSESLAT field tests.
On February 8 and 9, 2003, the actual field-test scoring sessions
were held in Princeton, New Jersey near ETS offices. There, larger
groups of scorers evaluated the entire set of student responses.
The scoring leaders who had led the range-finding sessions in
Albany trained the scorers for the actual scoring session, using
the benchmarks and sample responses as guiding examples. This
training was to ensure that the scorers would apply the score
scales to the responses accurately, consistently, and expeditiously.
Once the scorers were trained on the various questions, they worked
through much of the weekend, completing the scoring of 2055 speaking
tests and 2915 writing tests.
Scoring
the 1-2 Speaking Subtest of the NYSESLAT
by Julie Dziewisz, exam leader participant for
the NYSESLAT
During two recent weekends this winter, I found
myself involved in a fascinating, complex process, and an unprecedented
opportunity—as an exam leader for grade levels 1-2 of the
NYSESLAT scoring process. The NYSESLAT is the confluence of the
TESOL ESL standards and the NYS English Language Arts standards.
NYS standards for K-12 are known as the “best state standards
of all states” and the most challenging, as well.
For grade levels 1-2, there were 649 responses
available for benchmark and sample pulling and then eventually,
scoring. The 1-2 grade level speaking test consisted of 7 questions
and corresponding prompts to obtain a speech sample.
In scoring the responses, conjecture was in
no way to be a part of the decision. Consistency, accuracy, and
quality were essential in the scoring process. As Doug Fiero indicates,
consistency among test scores is critical, since we do not want
what is called “the luck of the draw,” in which the
score depends upon who scores the test. Our accuracy and speed
were also important, as Fiero notes, since we only had the weekend
to listen to the taped responses and come to a consensus on which
responses would be used for the benchmarks or samples. The group
was also required to select a range of samples for each of the
scores of 0-2 on the 7 questions (a total of 42 for grade levels
1-2). This became quite a challenge where a one-word response
was acceptable and grade-level appropriate for a few of the questions.
During the entire two-weekend process it became
evident to me that there were definitely some do’s and don’ts
for testing that could be shared.
Some tips for optimum testing situation:
· The teacher or exam administrator should
completely read the test directions before beginning the test.
· Administer the test in a quiet location
to give the student the best opportunity to be successful and
adequately present the test to the student. Be aware of outdoor
and other classroom noises and distractions that could develop
before starting the test.
· The test administrator should use a
friendly and non-threatening voice. Many teachers used the child’s
name throughout the test, going from one question to another.
This helps create a comfortable environment for the student.
· Try to keep pace with the test. Do
not engage in unnecessary conversations with the student during
the testing situation and stick to the prompts given in the test
booklet.
Scoring
the 5-6 Speaking Subtest of the NYSESLAT
by Caryn Bachar, exam leader participant for
the NYSESLAT
On January 31-February 2, 2003, I joined a number
of educators, administrators, and retirees who convened in Albany
to be trained as an exam leader by the Educational Testing Service
test development and performance scoring staff. ETS representatives
briefly described the history and reasoning of the NYSESLAT and
the responsibilities of the exam leaders. Although the reading
and listening subtests had been machine scored, the writing and
speaking were to be individually graded utilizing a predetermined
rubric.
The leaders were given a copy of the scoring
guide and the particular grade level test in the specific modality
(speaking or writing).
Each leader was to establish benchmarks for
each item, and a series of samples to be used for training at
scoring sessions for that particular grade level. Those dealing
with writing were to complete their task within one day. The speaking
test was allotted two days.
The speaking test consisted of seven items,
each of which assessed a particular skill. Items 2 and 3 were
scored as one unit. The item types included describing, explaining,
predicting, speaking persuasively, giving directions, narrating
a story, and interpreting and explaining information in a chart,
table, or graph.
The following morning each speaking modality
leader, along with two readers, was ensconced in a hotel room
complete with tape recorders and a large number of taped responses.
The participants included a group of teachers with a range of
backgrounds including ESL teachers. This ultimately proved beneficial
because ESL methodologies, experiences, understanding of L1 and
L2 acquisition and development, cultural norms, etc., were explained
and a fresh mainstream classroom perspective was gained as well.
As a result of listening to numerous students, both ELLs and native
speakers, the benchmarks were established, the samples culled
from the group, notes for changes and training were generated,
and, most important, the rubric was edited and refined. This remains
a work in progress as new situations, patterns and particularities
emerge.
The following weekend, exam leaders met in Princeton
to train readers and score the actual field tests. Of the available
levels, I scored grades 5-6, which had approximately 230 responses.
Included in the responses were ELLs with various proficiency levels,
and a relatively large number of native speakers as the control
group.
The NYSESLAT, based on the NYS ESL Standards, is an innovative
assessment that if used correctly and consistently, will present
a more comprehensive description of the knowledge and proficiency
levels of our students.
Scoring the 9-12 Speaking Subtest of
the NYSESLAT
by Joanne Beard, exam leader participant for
the NYSESLAT
I spent the greater portion of two weekends
as an exam leader for ETS, scoring the speaking section of the
NYSESLAT grades 9-12.
Based on the ESL standards, the test consists
of four parts: reading, writing, speaking, and listening. The
reading and listening sections are machine scored, while the remaining
two, writing and speaking, are scored by the administering teacher.
The first weekend of scoring, approximately
400 responses taped responses were played to select the benchmarks
and representative samples. The majority of the scorers had problems
with the rubric; so a few changes in the wording were necessary
to make it acceptable. On the second weekend, all of the remaining
responses were scored using the benchmarks set in the previous
session as guides.
Scorers can’t give out any information
that was on the test. Even talking about the answers is prohibited
since answers give insight into the questions. The questions themselves—which
were solicited from teachers around the state—are kept in
vaults, just like the Regents exams. So we were instructed not
to talk about the questions, either.
But there are some things that I can talk about,
things that we discussed during our lunch breaks. Many of the
tape recordings we listened to were perfect examples of how not
to give a test. In all fairness, the teachers knew that they were
just testing the test, not the students. Background noises, car
horns, telephones, other students, and hallway melees were prominent;
the car horn sounded like it was in the same room. When the bell
rang in another and the sounds of a crowd became obvious, you
knew the teacher was testing in the hall.
Some students were tested in the same classroom
where lessons were going on and it was obvious that there was
a roomful of people listening in. In an actual testing event,
maybe the teachers would have taken more care. But then again,
many of the ESL teachers don’t even have rooms for classes,
never mind optimum conditions for testing.
There were the teachers who wouldn’t give
up when they didn’t get enough of a response. They asked
so many questions that it was difficult to determine if the answer
belonged to the student or the teacher. Guidelines for the teachers
include suitable prompt questions.
I can’t wait to hear how the tests are
received by the ESL teachers. I hope it’s the test that they
have been waiting for. The parts that I saw were pretty much on
target for English situations in the real world.
This experience was educating, enlightening,
and fun.
Conclusion
by Gerald DeMauro
As many states call New York to understand what
has been achieved thus far with the NYSESLAT, the state continues
massive effort to develop the highest quality instrument for English
skills acquisition. This is truly the work of the whole bilingual
and English as a Second Language community in the state, along
with the continued efforts of the State Education Department Offices
of Bilingual Education and State Assessment, the Educational Testing
Service, Pearson, the various districts throughout the state,
and many others all contributing to some part of this effort.
It rests on the solid foundation of New York State’s learning
standards for English as a second language and addresses the necessary
skills for successful achievement of the state learning standards.
updated
on
October 4, 2004
