"The movement around teacher evaluation reform was pushed fairly boldly and quickly around the accountability issue," Silva says, "And that was a mistake. It left teachers behind and didn't attend to the vulnerabilities and insecurities teachers feel."

Instead of just focusing on how to punish or reward the worst or best 5 percent of teachers, she notes, "the larger solution lies with the 90 percent [in the middle], and helping that 90 percent improve and creating the conditions necessary for them to do their job well."

Testing is a blunt instrument

Most agree student learning should be at the heart of teacher evaluations – but what happens when the tests are flawed?

When scoring math tests this year, teachers were constantly calling the testing company over mistakes on the test, says Katie Zahedi, principal of Linden Avenue Middle School in Red Hook, N.Y. On a few questions, her teachers differed strongly with Pearson, the testing company, on how the answers should be scored.

Nancy Keeney, an English and reading teacher at Linden Avenue, says that the tests have also started to affect how she teaches – for the worse. The reading test, for instance, asks students to read a poem or short excerpt and then dissect the various elements into a "graphic organizer."

"This isn't something English teachers would teach if it weren't on the test," Ms. Keeney says, "but now you have to teach kids how to do that…. I have never seen any kid sit up straighter and spark to life when going over graphic organizers and learning how to identify correct answers."

A recipient of Race to the Top funds, New York has been working to roll out an evaluation system – which local unions need to sign on to, district by district – required by the Race to the Top rules. The result has been a complicated system that few understand and almost no one seems happy with. Many districts still haven't come to an agreement with their unions, and as of this spring, nearly 1,500 New York principals – a third of all public school principals in the state – had signed an open letter of concern about the state's approach.

Meanwhile, the state has been cited by the federal government for its failure to follow through on the promises it made in its application, particularly relating to putting the new evaluation system in place.

In their letter, the New York principals cite a number of concerns, but the big one comes down to test scores, and the value-added model that, in theory, isolates the teacher's contribution to a student's improvement on standardized tests.

In New York, other measures, including observations, also factor into evaluations, but the state has determined that no teacher who is deemed "ineffective" based on student test scores can be deemed "effective" overall – even though student scores account, in many districts, for only 20 percent of a teacher evaluation. The trouble, say critics, is that value-added scores are highly flawed and particularly unstable with certain groups of students, including English-language learners, special-education students, and gifted students.

"I went from being very enthusiastic about these methods to extremely worried," says Linda Darling-Hammond, a Stanford education professor who has researched teacher evaluation extensively and says she's seen value-added scores bounce wildly around even when she and other researchers believed they'd controlled for all student characteristics.

A few small anomalies, she says, can make a big difference in how a teacher appears to be doing.

Ms. Darling-Hammond cites a New York City teacher whose scores showed her to be the worst eighth-grade math teacher in the city, even though her kids – part of a gifted and talented class – all passed the state's Regents exam for integrated algebra, a test they normally wouldn't take until late in high school. The students appeared to have moved far beyond the material they were tested on.

"If you had a basket of evidence, you would have seen that she got the eighth-graders to pass the Regents, seen that the scores were high. You might have also seen classroom evidence. You would have said 'this teacher is kicking it out,' " says Darling-Hammond. "The issue is not, 'Do we look at student learning?' The issue is, 'How do you look at the data?' "

But others believe that, even while flawed, value-added data is still in many cases the best way to look at a teacher's impact on student learning – so long as it's combined with other tools.

"There are a million good, legitimate concerns, but there's also a philosophical element … which is being willing to be judged by something that's truly objective," says Mr. Weisberg at TNTP.

"We don't have the same sort of debates about whether we've squeezed all the potential for unfairness out of evaluation systems for surgeons or engineers," Weisberg says. "How long do you want to wait until we have a system that satisfies all the concerns?"

Phillips, at the Gates foundation, notes that in her organization's study, value-added scores tend to correlate highly with other, more nuanced assessments of teaching quality.

Value-added "is volatile, but not so volatile that it's not predictive," says Phillips. "But some assessments it's based on aren't as great as you'd like, either. It's why you want multiple measures."

The nuance of multiple measures

Student test scores are far less controversial when they're not the only piece of evidence being used in an evaluation – or when administrators have some discretion to interpret those scores.

That's the case at the Achievement First (AF) schools, a growing network of 20 charter schools in New York and Connecticut that has one of the most comprehensive, and harmonious, evaluation systems around – all initiated by teachers. While test scores are key (they count for 40 percent of the teacher's overall evaluation in cases where a standardized state test exists), school leaders understand their limitations and can use professional judgment to weigh those scores.

And the test scores are just one piece of a system that also includes detailed observations from in-school and out-of-school evaluators, and surveys of students, peers, administrators, and parents. Every teacher is assigned a coach, who helps them improve and hone their craft in weekly or biweekly meetings.

"We have a culture of continuous improvement, and teachers are intent on getting better," says Kate Baker, the principal at AF Bridgeport Academy Elementary School in Connecticut.

This spring, the end of his first year teaching first grade at AF Bridgeport, Ted Eckert received an 11-page evaluation summary.

It included the results of surveys from his students' families, fellow teachers, administrators, and four observation scores: three formal ones, and a fourth score based on frequent informal observations. In the fall, he'll get his students' test scores.

"It was a really complete picture of not only me as an instructor and teacher, but as a professional on a staff with lots of moving parts," says Mr. Eckert. In his five previous years teaching in theBronx and Queens, he says he got "nothing even close to as systematized as I experienced this year."

AF's evaluation system began several years ago, when teachers started asking for more support to help them improve, and also for a career path that could allow them to stay in the classroom while still moving up into leadership positions. Teachers met with school leaders, and together they came up with the current system.

"We knew what we included would help drive teacher behaviors, so we wanted to make sure the right stuff was in there," says Sarah Coon, AF's senior director of talent development.

In AF's model, teachers are grouped into five "buckets" of experience, from Stage 1 ("intern") through Stage 5 ("master teacher"). As they move up, they have the chance for increased recognition, an increased salary scale, and more leadership and professional development opportunities.

A master teacher has control over a $2,500 professional development budget, can serve as a coach and get extra coaching sessions from experts, and works with a teaching and learning team on things like curriculum development.

"Had we done this without teachers' full involvement, we might have come up with different rewards," says Ms. Coon.

This was the first year, and so far, teachers seem to be relishing the feedback.

"The rubric is so specific that you can understand where the scores are coming from," says Katie Maro, a third-grade teacher at AF Crown Heights Elementary School in Brooklyn, N.Y. Ms. Maro says she understands why so many teachers are resistant to an evaluation that is heavy on test scores. But given that these scores come in a rich context, with so many other factors, she says she feels fine with that being a piece of it.

"AF has taken a lot of time to see that I don't see myself as a number," she says.

Even more important, say other AF teachers, they have supports that help them use the feedback to improve.

"The times I've been most successful with helping teachers develop are when we identify growth areas in observation and then work on them in weekly coaching," says Ms. Baker, the Bridgeport principal, noting that she – like everyone – has a coach she works with. "It's hard to make forward momentum with just a few observations. It needs to be tied to an ongoing practice of coaching and development."

What to do with the measurement

Measuring teaching only means something if those results are put to good use: to identify – and possibly remove – the very worst teachers, if necessary, and to support teachers and help them get better.

"Can we improve struggling teachers? It's one of the things we don't know," says Weisberg. Good evaluation systems, he hopes, can help districts target their professional development wisely to help those teachers.

But, he says, when teachers don't improve they need to leave the classroom.

Likewise, many reformers feel there is untapped potential for the best teachers. They could reach more students through larger classes or smart use of technology. Or schools can give them leadership and mentoring opportunities to help fellow teachers. These possibilities could have the added benefit of attracting talented teachers to the profession, by creating new career ladders and offering more rewards, say advocates.

"Where quality systems are going in place, even teachers who are heavily skeptical and worried about it to begin with … say, 'This helped me change my practice, it helped me improve," says Phillips.

Evidence of that is mounting in Hillsborough County, where first- and second-year teachers are now assigned a mentor.

Aimee Ballans says she watched newer teachers there make big leaps this past year when she could tie her coaching to the specific observation rubric that's part of the new system.

In one case, she helped a high school reading teacher who was struggling with a group of boys in her last class of the day. The teacher was reluctant, but Ms. Ballans pushed her to try a form of structured debate in which the kids got to get up, move around, and switch sides in the debate.

"At the end of the class period, we had every kid participating, and the kids said, 'We thought we were just going to come in here and read today, but this was awesome,' " Ballans recalls. "We had a great conversation afterwards where we really dug down into why this worked."

Helping that rookie second-year teacher see that the students' previous disengagement wasn't their fault was "the best gift I could give her."