The Capital Region BOCES recently sponsored a useful conference entitled Ready, Set, Grow!: School Improvement through Value-Added Analysis. Value-Added is clearly an idea whose time has come, and the conference was co-sponsored by the New York State School Boards Association, the New York State Council of School Superintendents, and the School Administrators Association of New York State. If you looked in the eyes of the attendees - it was a crowded house - there was a mixture of attentiveness and apprehension.

One can think of accountability systems in geometric terms. New York currently calculates Adequate Yearly Progress (AYP) by comparing points: the performance of a group of students at a point in time is compared to a state standard. You're either there (AYP), or you're not.

Chapter 57 of the Laws of 2007 requires New York to have a Growth Model in place by the 2008-09 school year. A growth model is a line with slope. The same group of students is measured at two or more points in time, and the slope will be positive, negative, or zero - signifying growth, regression, or stasis. Districts not yet at AYP may still be able to demonstrate that students are on trend to get to there, soon. Demonstration of growth will allow a more nuanced analysis, and may become a new form of "safe harbor" for districts whose progress is masked by the current point-based system.

This same law requires that New York implement a Value-Added Model by the 2010-11 school year, subject to all sorts of conditions and approvals. With a value-added model, we will have two lines, each with its own slope: one representing expected growth and the other observed growth. A further level of analysis will be possible: schools whose students have not made AYP (the point), nor are on a timely track to reach AYP in the future (the line), may still be able to demonstrate that observed growth has a larger positive slope than would be expected if the district/school/teacher had done nothing.

Crudely put, a value-added model is a statistical method to demonstrate "better than nothing." Schools that are considered under-performing in a point-based AYP model, and not-on-track in a line-based AYP model, may still produce student growth that exceeds expectation (i.e., has value). A value-added model will also identify high-achieving schools that produce no growth, beyond what would be expected had the district/school/teacher done nothing.

In a value-added model, the key issue becomes how to determine expected growth. How do we calculate the slope of the predicted line?

The solution is a tangle of statistics, involving multiple measures, differences, correlations, and covariates, but the basic idea is that future growth is predicted based on past achievement, controlling for student/school characteristics. Some value-added models collect multi-year data and have students serve as their own statistical control; others have fewer repeated measures but use demographic variables as covariates to reduce error.

Which brings me back to the look in the eyes of the conference attendees. The policy implications of these developments are enormous and exciting; the statistics are complex. Some vendors claim the statistics are so complex - and valuable - that they are proprietary, a secret. Thankfully, New York has declared it will not adopt a secret model!

Despite the stamp of approval proffered by legislators, professional organizations, and individual testimonials, there is still a healthy debate within the field about the conditions under which a value-added model is, or is not, valid. The concern is that those who will create, comment on, or implement the policies surrounding a value-added model may not fully understand the underlying technical issues, and will therefore be overly cautious, overly eager, or - worse - will defer key decisions to the recommendations of those who claim to understand. Explaining this to the public is another matter entirely.

We often make decisions based on complex and poorly understood calculations (e.g., stock derivatives). But the mandatory adoption of a value-added model is an example in which students, schools, and teachers will be informed - and judged - in a profoundly different manner. Although this may be a new age of data collection, reporting and analysis, transparency, curiosity, and inclusiveness are timeless virtues when we have so much at stake. The leadership at the State Education Department is in the midst of a healthy debate on growth and value-added models, with an established time-line that includes expert guidance and opportunities for public comment.

Expectation is a funny thing. We know that higher expectations produce better outcomes, so long as the expectation is not so unreasonable as to produce hopelessness and apathy. If expectation is managed properly, students, teachers, and schools tend to meet the challenge. How will value-added models maintain the balance between statistical expectations based on past behavior, and high-but-not-too-high expectations based on future goals?

What do you think about growth and value-added models, and the implications for guiding and evaluating students, teachers, schools, and districts?

This entry is also posted in the Data Group at https://schoolleadership.ning.com/group/data. Please post your comments in the Data Group so others can follow the discussion!