The dangers of building a plane in the air

This was written by Carol Corbett Burris, principal of South Side High School in New York. She was named the 2010 New York State Outstanding Educator by the School Administrators Association of New York State.

By Carol Corbett Burris

Buckle your seat belts and hold on for your life. Teachers and principals, welcome to APPR Airlines flight 2011. Your journey on the ‘plane to be built in the air’ just took off from New York’s Albany airport.

This description of the New York teacher and principal evaluation system known as APPR is not my critique of an incomplete and untested evaluation system. Rather, it is the description provided by the state Education Department itself. Across New York State, all of the school and district leaders who evaluate teachers are being pulled out of their schools for mandated, taxpayer-funded training in this APPR teacher and principal evaluation system.

The scripted curriculum given to local trainers by the Education Department begins with a bizarre video of smiling mechanics wearing unopened parachutes, building an unfinished plane in flight—all of which the trainers liken to APPR, which means Annual Professional Performance Review. As the narrator tells us, “Some people like to climb mountains. I like to build planes…in the air.” You can find the video here.

It is labeled a ‘funny video’ on YouTube. Not a school leader in the room laughed. Likening a system that is now driving professional evaluations and student testing to what could have been an episode of the Three Stooges did not amuse. I felt sad. Leaders of integrity and courage do not let unfinished planes with students and teachers aboard off the ground.

As the day progressed, we learned just how unfinished that plane is.

The question and answer session went like this:

“How can we evaluate teachers when we do not have test results before they leave for the summer?” We do not know.

“What do we say to parents when we are compelled to put a teacher publically rated ‘ineffective’, fairly or unfairly, back into the classroom for their second year?” That was labeled a ‘rhetorical question.’

“How will they determine the scores, based on test scores, of teachers who go on maternity, or of special education and reading teachers?” Reply: Dosage formulas. They will determine the ‘dosage’ (yes, the trainers told us, that is their term) of the teacher that a student got.

The plane, full of hot air, rests on thin air.

On Day 2, we learned that the plane is driven by data. To bring the point home, we were all given a copy of a book entitled Driven by Databy Paul Bambrick-Santoyo, a managing director of a charter school outfit called Uncommon Schools. [Uncommon Schools just received a $1.4 million grant from the U.S. Department of Education to expand.]

This book, which is now being given at unknown expense to thousands of school leaders at APPR trainings, is an interesting choice. Bambrick-Santoyo credits his method with creating successful 90/90/90 schools (which means 90% free or reduced-price lunch, 90% students of color and 90% proficiency).

Uncommon Schools has a solid connection with New York’s Education Commissioner, John B. King. Prior to coming to the Education Department, King held the same Uncommon School’s position that Mr. Bambrick-Santoyo now holds. What I found to be more worrisome, however, is the book itself.

Driven by Data is a guide on how to make data the driving force of instruction. It is basically a handbook for teaching to the test. Rather than cite evidence from peer-reviewed research studies, Mr. Bambrick backs his claims by pointing readers to data from the schools that he claims achieved outstanding results from adopting his practices and attending his training.

Being most familiar with New York’s database, I researched the New York charter schools that Bambrick-Santoyo holds up as models in the book. One of the model schools is South Bronx Classical. Bambrick-Santoyo praises it for its students’ outstanding performance on the kindergarten and first grade Terranova exams.

For example, he points out an above grade level rate for first-graders in 2008 on the language Terranova is 94%, thus meeting his 90% grade. Two years later, however, when that cohort of students took the state English Language Arts third-grade exam, only 51% were proficient.

Most of the time, the author uses growth in proficiency rates on the New York State exams to demonstrate charter school excellence. Those rates, however, were inflated because the tests became easier and easier, which New York State recognized and corrected the following year by racing the cut scores. After the correction in 2010 New York state elementary English Language Arts (ELA) proficiency rates dropped, on average, 20 points.

For example, the author praises Achievement First Bushwick Charter School for its excellence and data driven practices. Curiously, the book shows the test score results not from Bushwick, but from another Achievement First school, Crown Heights, which Bambrick-Santoya notes had a 2009 fourth-grade ELA proficiency rate of 98%. But in 2010, the non-inflated rate was 55% for fourth graders and 25% for seventh graders.

Achievement First Bushwick’s ELA 2010 proficiency rates range from 23% to 42%, below the state average and the author’s 90% mark.

The leader of the third school, Explore Charter, attended Bambrick-Santoyo’s Leadership Workshop as well as the New Leaders for New School’s program, of which Bambrick-Santoyo is a graduate and faculty member. He tells us that the principal’s “intense” “face to face meetings” with teachers led to success. He notes her 2009 results: Explore’s third-grade ELA test scores in 2009 reflected an impressive 85% proficiency rate. In 2010, however, the third-grade proficiency ratewas an unimpressive 33%.

The fourth and final school, True North Rochester Prep, had a decrease in ELA proficiency rates, but remained competitive when compared with state averages. However, only eighth graders made the above 90% grade. True North Rochester’s sister school, True North Troy Prep had an ELA proficiency percentage of only 40%.

I will leave it to others to speculate about the concerns raised above. Part of the drop is certainly due to the state’s change in cut scores, but this does not appear to tell the whole story. One thing is clear. The author makes his case based on achieving 90% and above proficiency rates in urban schools that follow his method. Clearly in 2010 those New York schools did not make the grade in literacy. One would have hoped that the state Education Department would have looked at the 2010 data for the book’s New York schools before distributing it in training across the state.

But such caution does not happen when you build a plane in the air. It seems that if you are determined to force your vision of ‘reform’ based on the sheer arrogance of believing you are right, you want that plane built in the air before educational professionals, researchers and reporters can peek inside.

Take a second look at that YouTube video; the passengers don’t have parachutes — the only ones who do are those who enthusiastically build the plane.

-0-

Follow The Answer Sheet every day by bookmarkinghttp://www.washingtonpost.com/blogs/answer-sheet.