Dear readers.  Even if this is far too long for a blog—2000 plus words!–please, if you read it, respond.
Thanks.

The number one “habit of mind” that we based our work at Mission Hill, Central Park East schools on:  “how do we know what we know and how credible is it?”  With all the education talk about “evidence-based” and “data driven” reform you’d think we’d stop for a moment and ask ourselves how much school evidence/data we can truly count on?  Or even scarier—how do we know anything beyond our first-hand knowledge?

I just finished reading a blog by Diane Ravitch about Geoffrey Canada’s work in Harlem which, in turn, is based on a blog by Gary Rubenstein. Rubenstein gives facts and figures to  (1) prove Canada’s proclaimed graduation rates aren’t honest, (2) show that Canada’s success depends heavily on the incredible fiscal resources he has access to,  3) remind us that Canada built his rep without acknowledging that he kicked out two entire classes because they didn’t get good test scores, and (4) that he denies all the above.

But in a field in which I know a lot I no longer believe anyone’s data; thus  exposing Canada hardly matters!  Not even my own “facts”l  sometimes I don’t convince even me!  I know too much about my own temptation to pick and choose evidence that confirms my beliefs to assume that my allies—and enemies– aren’t similarly influenced.  At any one moment the temptation to lie, fudge or obscure negative data can be either trivial or critical. The higher the stakes that rely on the data the greater the temptation—like e.g. bonuses, reputation, livelihood, jail– to look for the best and hide the worst.  The GAO claims that 33 states cheat, but I believe it relies on an old-fashioned rule—no explicit prepping for a specific test.

So I wonder, is my nostalgia for a time when I “believed” most  “facts” just that—false memory?  Or even worse, stupidness on my part?  I suspect some of it was stupid and some naivete.  After all, I long ago noticed that the NY Times never got a story quite right if it was one I happened to know a lot about—where I was there, for example.  But I still kept/keep, sort of,  believing all the information they offer on what I don’t know much about.

I remember an anarchist friend of mine disputing my claim that people were living longer today than they had a hundred years earlier.  When asked why I believed it, I mentioned as one example, census data.  He lashed into me about my naiveté in believing government-sponsored data.  I felt sorry for him because how can one cope with a world where you cannot know who and what is “a fact”.  It surely makes even flawed democracy a utopian dream.

I’m in his boat now, and it feels awful.

Close to home, for example, I know how easy it is to fudge graduation data, drop-out data, class-size data, attendance data, GPA averages, test scores, and on and on.  I’ve even dabbled in a few of these myself.   It’s hard to get caught—unless someone is really after you or you’ve let too many people in on the secret.

Therefore should we stop collecting the stuff?  Maybe.  At least I feel comfortable saying we (1) shouldn’t be collecting new stuff with high stakes attached, and (2) should remain very skeptical—especially if, on the basis of ones personal knowledge, the data seems miraculous or peculiar.

I used to carefully scan the ranked test score reports in the NY Times.  (It began in the 1960s).  Schools were ranked in order of scores, and the story indicated both this year’s scores and last year’s.  What I soon noticed were occasional great leaps or declines—which seemed unlikely if we were actually comparing oranges to oranges.  Either something happened—such as the school having suddenly become the site of the District’s gifted program, or a new principal was no longer inflating scores as his/her predecessor did.  (She/he may not have even known they were inflated.)  I checked some and it confirmed my suspicions.  Others I had no way to confirm or deny.  Similarly, years ago I witnessed an enormous rise in attendance in our high schools following a new chancellor’s demand that we focus on attendance (“they can’t learn if they’re not there.”)  Until I realized we had simultaneously, and not secretly, changed the class period when attendance was officially taken—from first to third, I believe.

Drop-out figures?  They are hard to count and aren’t simply the difference between the number of 8th graders vs 10th or 9th vs 12th.   (Although big discrepancies in either requires some explaining.)   After all, kids leave one school for another—some of which can be verified, some not.  After all,  families move to other cities, states and countries.  Also some can be accounted for by hold-overs unless one looks into the 5 and 6 year graduating rate.  We’d need a team of detectives per school to follow-up and even then it’s problematic how much they could discover.  Except for rare drop-in visits to count a random sample of classes we are pretending schools are telling the truth.  Maybe there are more honest principals out there than one might think.  But even the few who are more careless, let’s say, are rarely “accused.”  Both the cost and the morale impact of being continually inspected for the truth would be beyond immense.  (Store-keepers, bankers, you name it  have reached the same conclusion and have invented annoying ways to keep “us” honest, but not themselves).

I could go on and on.  Every time we institute a new policy to catch wrong-doers most of us act just like our students, we put our minds to new ways to get around the new rules.  The last fiscal crisis being a good example.  It’s easier than improving the school  (economy) in ways that will show up on high stakes rank-order lists.

A wonderful friend of mine (and of many other school people) ran a high school that took all the kids others wanted to get rid of.  He never said “no” if there was a space.  And the kids he took were grateful because he really cared about them.  But after many years some reporter decided to expose him by noting the school’s relatively low attendance rate and relatively high drop-out data.  He was, the story suggested, a phony who had been getting away with this for years.  My friend soon retired and afterwards died under sad circumstances.  Of course, were it not for him other feeder schools would have had worse data.  And, I wonder. would he have served his students better had he been willing to fudge the data?

The world is a worse place when we feel that maybe we “should” lie in order to “do good.”

So where do I go with this?  I’ve reached a few possibly useful conclusions—to start with.   To lessen the reasons to lie the stakes must not be too high and to increase the reasons to tell the truth the consequences must be helpful.  (Campbell’s Law)  Then  we need  to make it easier for the truth to be naturally exposed—where lying would require too much collaboration from too many people to last long.  (That’s what I usually count on–truth will win out over time–when I hear outrageous conspiracy theories.)   That’s one reason I like small schools.  Assuming that people generally trust data that supports what they otherwise know first-hand, school size helps check lying too much.   If I say 100% graduated, hopefully some kids, teachers and parents simply know better because they know better; they remember.   And on and on. There was a story in the media some years ago about a speech in which the valedictorian  started off by asking the graduates to look round and think about their freshman classmates–those who were no longer with them, who hadn’t made it.

But, we have to rely on some “facts”–just to get out of bed each morning.  But how much further from our own self-knowledge can we rely on “the evidence”?  In short, not far.  Restoring confidence in “the facts” while retaining sufficient skepticism is a tough balancing act.  It’s what, ideally, schools,  the media, the courts (and friends) are there for.   I’ve come to believe that the first order of the day for any reformer is: figure this puzzle out.  The answers must, I fear,  finally rest in human judgment; but judgment can be trained, improved upon and what better place for doing this than schools..

Yes, smallness is one partial answer.  Openness is another.   Not getting so tangled up in our fear of intruders that we lock everything up would help.   (And then we get hacked, etc.)   Lots of opportunities for families and schools to share information—more and more family conferences to clarify the self-serving lies that even the best kids occasionally tell.  Especially if the kids are at such meetings too so they can check on misleading claims adults sometimes indulge in.  It also means tackling the “isms—above all racism.  It’s this—and all the small disrespectful acts that go with it,  that cannot help but undermine trust.

We discovered (from others, including good private schools) the value of visiting teams of respected colleagues and experts, who come and spend time on a regular basis—as we did at CPESS and on some level also at Mission Hill.  Let them look over our records, our curriculum, our assessment tools and interview a sample of parents, teachers and students.  Sit in classes.  Then at the end, after reading their reports, we enjoy an open free-for-all, followed often by a written faculty “response.”   These were NOT for high-stakes purposes, but ways of checking for useful and helpful feedback.   It helps also if the school culture rests on frequent teacher-to-teacher visits, drop-ins, etc.

How to shift the balance?  How much of it must be mandated from above?  How far “above”?  Who should have access to what?  What protections are needed from harmful or premature disclosures—or should there be none?   “What we say here, stays here” may at times be critical for healthy discussion—if so, how do we provide for that too?  We need to leave room for discussing those “white lies” that even the strictest truth-teller might – or might not – occasionally indulge in.  And we need to help young people sort these out too, without undue fear.  The value of making such “habits of mind” explicit and user friendly takes time and effort.

How might we try some of these ideas out on an experimental level?   It is probably the narrative that goes with them that will or will not help persuade others to follow—not the statistical part.  The primary tool of a democracy is persuasion.  The facts are part of trying to persuade. Generally we stick with what we have been believing until someone we trust a lot on a personal basis presents an eye-witness report that forces us to consider the possibility that “I’m wrong.”  We have to respect how hard it to persuade people they’re wrong.  For as Thomas Kuhn said—in discussing the search for scientific proof—sticking with one’s current viewpoint is not a bad idea.  If we have no commitment to our ideas we will never know whether they are right or wrong.  We need accommodate new “truths” to old ones for as long as we can.  But also it shouldn’t be too uncomfortable to switch “sides”–eventually one should be able to drop practices or beliefs that even you have begun to be skeptical about and try out a few that you used to shun.  It’s easier if you are also able to revert!   Watching good teachers caused me to reconsider some of my pedagogical certainties:  like the value of choral reading (and not just of music). Even about lining-up routines; although I’ve also questioned why we need to line-up so often!

It was even exciting when I came back from visiting a city(Minneapolis) that never lined kids up, to ask colleagues why we needed them.

I’m also, as I finish this, thinking about how the other four “habits of mind” serve as a partial check on the first.  Number 2 usually is something like this: how else might it appear, look to others?  The third asks about perceived patterns, the fourth asks “what if” and the last asks, “who cares and does it matter?”  There are probably dozens of other habits of mind that we use as we delve deeper and deeper into the usually unending search for knowledge.  But then the dilemma is: habits depend on frequent use in many different settings.

The crisis, so-called, in American education is a symptom of a “crisis of trust” which in turn poses a “crisis for democracy” writ large—as an idea itself.  If we are not to give up, we need schools, families and communities that start to carefully rebuild trust within their own four walls, and base it on losing the fear that we might, on occasion, be wrong.  No institution I know, alas, presently values being wrong less than our K-12 schools.  We might as well start there.  Maybe if we do we can reverse the trends of the past few decades or distrust at all levels of society.