Note:  This is Dr. Joan Daly-Lewis's response to Carol Burris's blog.

Dear Carol:

Carol, here is my concern about the Hinchey article  (“ Getting Teacher Assessment Right: What Policymakers Can Learn From Research”)- the new NYS APPR seems to address ALL of the recommendations!  But as you have said in your Post blog, the devil is in the details.  There is actually much in the new APPR system that could have had the potential to improve the supervision of instruction.    I have tried to lay out the areas that I feel present obstacles to school improvement, but first I’d like to make it clear that most Long Island districts have a need to update and strengthen their teacher evaluation processes.  Moreover, they largely recognize this need.  Nothing here is meant to suggest that everything is fine the way it is.   I have spent the past six years working full time helping Long Island districts to upgrade their teacher performance assessment practices, and I know that New York State and the Federal government policies have the potential of providing an important catalyst to improvement by requiring the following:

1. linking teacher and principal evaluation to a clear set of professional standards
2. selection of an appropriate rubric
3. an emphasis on student achievement in addition to assessment of teacher actions
4. requiring multiple measures through which to gather evidence, 
5. training of evaluators
6. timely and constructive feedback 
7. establishing of links between professional performance and professional development
8. improvement plans for teachers needing improvement
9. attention to meeting the needs of English language learners and students with disabilities

Now let’s look at the specifics.  The detail that is serving as the major fatal flaw in New York is the use of a 100-point rating system.  The 100 point scoring requirement suggests that as a field (and as over 700 individual districts) we possess the methodological capability to invent a system to fairly and accurately assign some portion of these 100 points to teacher performance based on student achievement on local and state tests, and on teachers’ classroom performance.  Let’s look at each of these three components:

First, the state-test-driven component assigns absolute causality to individual teachers for student test results in a manner that is unsupportable, especially for a single year’s test administration.  Let’s be clear.  While it is essential to consider test performance in determining teacher efficacy, this needs to be done in a way that is fair, accurate, and responsive to the context.  Schools can make very accurate judgments by examining performance scores over time (three to five years).  However, the New York State regs have a numerical weighting (with the very high stakes consequences) assigned to a single test administrator.  This is unsupportable.  For one thing, the tests themselves are not as stable as we would wish; for evidence consider the recent trig/algebra exam.  Additionally, the regulations assign the tests a burden beyond their designed purpose.  More importantly, as many have noted, this “grading” of teachers on a specific student test administration will cause an even more obsessive focus on student test scores, with a further narrowing of the curriculum to full time test prep.  The effect is aggravated by the result of a last minute “deal” with the governor in which schools may now choose to count not just 20%, but up to 40% of a teacher’s rating on a single math and ELA score.  On a related “devil in the details” note, the State is not able to provide the districts with the assigned “”teacher or principal student growth percentile” scores until late in June, meaning that this component will yield a fixed score that will need to be lowered onto the (theoretically) already calculated component scores for classroom performance (“60 points-other”).  One effect of not knowing the State scores until there is no way to adjust the other scores (so that the overall picture is accurate) is likely to be artificial inflation of the local performance scores as schools attempt to ensure that teachers don’t “accidentally” fall into a performance category lower than the school perceives that they should.  This sounds so complicated as I’m trying to explain it, and it is.  Read on…

The second component of the rating system requires that districts award each teacher up to 20 points based on student performance on local growth measures (unless the local district elects to use the State test score for the full 40 point maximum, as mentioned above).   It is unclear how districts can structure a fair system for translating student performance into teacher performance points.  From a policy point of view it seems grossly inefficient (and costly in terms of human capital) to ask over 700 districts to each wrestle a unique solution to this econometric challenge without clear, viable models, especially since the task requires local collective bargaining, a further complication.  From a test and measurement point of view, it’s a mystery how to equate something like student growth in “writing,” or in “building understanding of grade level science concepts” with a teacher “grade.”  I have spend hours and hours exploring this, knowing that it’s to be done in a manner that is “transparent” and that the calculation method must be provided to all prior to the beginning of this school year.

The third component, “Other 60 Points,” is a measure of teacher competence rated in terms of the seven New York State teaching standards.  Experience shows that administrators and teachers who have been trained in the use of a standards-based rubric can develop a solid level of inter-rater reliability in assessing teaching performance against meaningful, research-based performance criteria when they are asked to assign teachers to one of four categories such as New York’s HEDI (highly effective, effective, developing/needs improvement, or ineffective).   that’s the good news.  Again, the flaw in the New York State system is its requirement that districts assign a numerical rating out of 60 points.  The more score point possibilities, the more data is required to substantiate a rating.  Many districts have helped their administrators to develop skill and accuracy in assessing teacher performance  by considering an array of clearly defined, research-based components, and collecting evidence to reach a decision as to whether a teacher is unsatisfactory (i.e., ineffective), in need of improvement (i.e., “developing”), proficient (i.e., “effective”), or distinguished (i.e., “highly effective”).  Granted, many other districts still have a need to do this important work.  However, the current statute and regulation greatly complicates this challenge by requiring each district to design a system that will meaningfully interface with the highly questionable scoring of student growth on state and local measures.  I’m reminded of the vexing experience of being given a homework problem to solve in seventh grade for which there was no solution, being reduced to tears by trying to achieve something that was just not possible to do.  I have developed multiple models, all of which require compromises in principle to achieve the goal of trying to fairly rate teachers and ensure that teachers are not punished by the scoring system. 

I am angry that precious teacher and administrator time is being swallowed up by committee meetings devoted to the seemingly impossible task of assigning numerical grades (out of 20 or out of 60) to teacher performance.  I am angry when I know that simply requiring the nine components listed above and then monitoring districts’ holistic ratings outcomes via correlation of ratings to student test scores over time would have gotten us where we need to go with much less expenditure of money and human capital.  I am angry that this Federal and State initiative represents another opportunity for publishers and software developers to siphon money from the public schools through the use of expensive third party tests and outside evaluators.  I am angry that teachers will lose sight of the purposes of schooling as they press students to get every point possible on State tests and high stakes local measures.  I am angry that this will distract from the more important work of building understanding of the Common Core and working to operationalize it across the disciplines.  I am angry that teachers and administrators are angry and demoralized, and that great teachers are discouraging their sons and daughters from going into the field of teaching.  I am angry that the numerical system will draw time and energy to game-playing with the numbers rather than a sustained focus on improving teaching and learning.  I am especially angry that this system will make teachers and administrator feel “done to,” rather than enable them to be partners in continuous improvement. Most of all, I’m angry that we have been saddled with a performance appraisal system that violates what we know about human behavior by focusing on rewards and punishment rather than recognizing that the real pathway to highly effective schools is to appeal to our desire to make a difference in the lives of children, and provide the conditions to make this possible.  I’d strongly suggest to Arnie Duncan that he read Daniel Pink’s 2009 book, Drive, to understand why this current initiative will collapse and die without achieving the critical goal of strengthening teaching and learning.   Please know that we are marching forward, working tirelessly to meet the mandate’s requirements while trying desperately to do as little damage as possible to local school improvement efforts and professional relationships. 

Joan Daly-Lewis