NY has trouble linking student scores to proper teachers

Originally published: August 21, 2012 8:50 PM
Updated: August 21, 2012 10:05 PM
By JOHN HILDEBRAND  john.hildebrand@newsday.com

Photo credit: iStock | Hundreds of thousands of student scores could not be linked to the proper teachers in New York State's new rating system, heightening concerns among Long Island educators that the evaluations will turn out to be imprecise and virtually impossible to explain. (August 21, 2012)

Thousands of student scores could not be linked to the proper teachers in New York State's new rating system, heightening concerns among Long Island educators that the evaluations will turn out to be imprecise and virtually impossible to explain to teachers or parents.

American Institutes for Research, or AIR, the nonprofit agency that developed the rating system for the state's portion of teacher and principal evaluations, recently reported that more than 50 percent of fourth- and fifth-graders statewide, for example, could not be matched to specific teachers in 2010-11, one of the years used for the evaluations.

About 200,000 students statewide at grade levels 3 through 8 take state tests in English and math annually. Consequently, those students' scores were used to rate principals, but not individual teachers.

AIR's report was released last week by the State Education Department at the same time the state's initial teacher ratings, known as "growth scores," were sent to local school districts.

State education authorities have insisted that the system is fair and transparent and will improve over time. Even some academic researchers who have questioned the wisdom of incorporating such scores into job evaluations concede the state's approach is technically "state of the art."

Tom Dunn, an education department spokesman, said that while every student with two consecutive years of test scores was included in ratings, more than 50 percent of fourth- and fifth-grade scores from 2010-11 had to be excluded because school districts at that time were encouraged, but not required to submit complete data matching students with teachers.

In addition, scores were excluded when students spent less than a minimum number of days in class, Dunn said.

Linkage improved greatly with the 2011-12 test results -- with 75 to 80 percent of students tied to the correct teacher -- and should improve again next year, Dunn added.

"The real world is complex and while very simple methods may be easier to understand, this does not mean simple methods are more accurate," Dunn stated.

Some leading school administrators on the Island voiced deep concern over difficulties encountered by evaluators in linking students to teachers.

"I can only say I was a little shocked by that," said Bill Johnson, superintendent of Rockville Centreschools and a former president of the New York State Council of School Superintendents. "I understand this is the first time out of the box, but there's a lot here that suggests the results aren't fair and accurate yet."

Many local educators also question the use of complex statistical formulas in a system intended ultimately to help determine whether teachers keep their jobs. Most districts won't phase in the new evaluation system until next year.

"If the goal is to provide transparency to educators and parents, then they've failed miserably," said Mike Cohen, an adjunct associate math professor at Hofstra University."There are probably about five people in the state of New York who can understand that thing."

State authorities have defended the formulas, on grounds that only sophisticated analysis can take proper account of factors influencing student scores -- for example, community poverty rates.

Sean Corcoran, an associate professor of educational economics at New York University, said the large number of students who could not be linked to teachers was "startling," though he regarded the state's approach as "state of the art" in a technical sense.