Newspaper investigation suggests widespread cheating on school tests

Newspaper investigation suggests widespread cheating on school tests


Suspicious test scores in roughly 200 school districts nationwide resemble those that entangled Atlanta in the biggest cheating scandal in American history, an investigation by The Atlanta Journal-Constitution shows.

The newspaper analyzed test results for 68,000 public schools and found high concentrations of suspect math or reading scores in school systems from coast to coast.

The analysis doesn't prove cheating. But it reveals that test scores in hundreds of cities followed a pattern that, in Atlanta, indicated cheating in multiple schools. State investigators last year detailed fraudulent test scores in Atlanta Public Schools, including in half of the elementary and middle schools. The state's review named about 180 teachers and administrators, some of whom confessed to altering test papers. Georgia's governor started a criminal investigation.

Questions about the trustworthiness of standardized test scores come as the federal government, states and school districts place more weight on the results for teacher and school accountability. (As part of its accountability system, Texas will begin administering a new standardized test, the State of Texas Assessments of Academic Readiness, this week.)

"These findings are concerning," U.S. Secretary of Education Arne Duncan said in an emailed statement after being briefed on the Journal-Constitution's analysis. "States, districts, schools and testing companies should have sensible safeguards in place to ensure tests accurately reflect student learning."

In nine districts, scores careened so unpredictably that the odds of such dramatic shifts occurring without an intervention such as tampering were worse than 1 in 1 billion.

In Houston, for instance, test results for entire grades of students jumped two, three or more times the amount expected in one year, the analysis shows. When children moved to a new grade the next year, their scores plummeted — a finding that suggests the gains were not due to learning.

Similar discrepancies were found in test scores for Dallas. Several other Texas school districts showed smaller but suspicious variations; they included the Aldine, Fort Bend, Laredo and Socorro school districts.

No Austin-area school district was implicated in the newspaper's analysis.

Overall, 196 of the nation's 3,125 largest school districts had enough suspect tests that the odds of the results occurring by chance alone were worse than 1 in 1,000.

For 33 of those districts, the odds were worse than 1 in 1 million.

A few of the districts already face accusations of cheating. But in most, no one has challenged the scores in a broad, public way.

The newspaper's analysis suggests that tens of thousands of children may have been harmed by inflated scores that could have precluded tutoring or more drastic administrative actions.

The analysis shows that in 2010 alone, the gradewide reading scores of 24,618 children nationwide swung so improbably that the odds of it happening by chance were less than 1 in 10,000.

Cheating is one of few plausible explanations for why scores would change so dramatically for so many students in a district, said James Wollack, a University of Wisconsin-Madison expert in testing and cheating who reviewed the newspaper's analysis.

"I can say with some confidence," he said, "cheating is something you should be looking at."

The findings come as government officials, reeling from recent scandals, are beginning to acknowledge that a troubling amount of score manipulation occurs. Though the federal government requires the tests, it has not mandated screening scores for anomalies or investigating those that turn up.

Daria Hall, director of K-12 policy with the nonprofit Education Trust, said education officials should take steps to ensure the validity of test results because of the critical role they play in policy and practice.

"If we are going to make important decisions based on test results — and we ought to be doing that — we have to make important decisions about how we are going to ensure their trustworthiness," she said. "That means districts and states taking ownership of the test security issue in a way that they haven't to date."

‘Way too much pressure'

Both critics and supporters of testing said the newspaper's findings are further evidence that in the frenzy to raise scores, the nation failed to pay enough attention to what was driving the gains.

"We are putting way too much pressure on people to raise scores at a very large clip without holding them accountable for how they are doing it," said Daniel Koretz, a Harvard Graduate School of Education testing expert.

Test-score pressure is palpable in schools grappling with urban blight and poverty.

These are the schools that the 2001 No Child Left Behind Act was supposed to fix.

But at Patrick Henry Elementary in St. Louis, airy red brick towers rising above the school belie a grimmer reality on the ground. Children leaving one recent afternoon passed piles of trash and a .45-caliber bullet tucked into the curb. Inside, their classrooms are beset by mold, rats, discipline problems and scandal.

Last year, the former principal — once hailed as among the district's strongest — was accused by Missouri officials of falsifying attendance rolls to get more state money.

State investigators didn't publicly question Henry's test scores. But the newspaper analysis found suspicious scores in the school dating to 2007. In 2010, for instance, about 42 percent of fourth-graders passed the state math test. When the class took the tests as fifth-graders the next year — with state investigators looking into cheating and other fraud allegations — just 4 percent passed math.

Experts say learning doesn't typically jump backward.

Henry's scores were consistently among the lowest in the state — except for the occasional sudden leap.

Rural, city schools flagged

The Journal-Constitution used freedom of information laws to collect test scores from around the nation.

In each state, the newspaper used statistics to identify unusual score jumps and drops on state math and reading tests by grade and school. Declines can signal cheating. The calculations also sought to rule out other factors that can lead to big score shifts, such as small classes and dramatic changes in class size.

Some school leaders accused of cheating have attributed steep gains to exemplary teaching. But experts said instruction isn't likely to move scores to the degree seen in the analysis.

Through teaching alone, Wollack said, "it's going to be pretty tough to have that sort of an impact."

The newspaper developed a statistical method to identify school systems with far more unusual tests than expected, which could signal endemic cheating such as that which occurred in Atlanta. The newspaper's score analysis used conservative measures that highlighted extremes and were likely to miss many instances of cheating.

Big to medium cities and rural districts harbored the highest concentrations of suspect tests. No Child Left Behind might help explain why. The law forced districts to contend with the scores of poor and minority students in an unprecedented way, judging schools by the performance of such subgroups as well as by overall achievement.

Hence, high-poverty schools faced some of the most relentless pressure of the kind critics say increases cheating.

Improbable scores were twice as likely to appear in charter schools as regular schools. Charters, which receive public money, can face intense pressure as supposed laboratories of innovation that, in theory, live or die by their academic performance.

Common problems unite the big-city districts with the most prevalent suspicious scores: Many faced state takeovers if scores didn't improve quickly. Teachers' pay or even their continued employment sometimes depended on test performance. And their students were among those needing the most help.

The analysis, for instance, flagged more than 1 in 6 tests in St. Louis some years. In Detroit, it was 1 in 7.

Dozens of school systems in midsize cities — such as Gary, Ind.; East St. Louis, Ill., and Mobile, Ala. — exhibited high concentrations of suspicious tests too.

Though high-poverty city schools were more likely to have suspicious tests, improbable scores also showed up in an exclusive public school for the gifted on the Upper West Side of Manhattan. And they appeared in a rural district roughly 70 miles south of Chicago with one school, dirt roads and a women's prison.

Houston's example

In 2002, Houston was the first winner of the Broad Prize, which has become the most coveted award in urban education. The Eli and Edythe Broad Foundation praised Houston's intense focus on test results. More recently, Houston has been among the leaders in tying teacher pay to student test scores.

Twice in the past seven years, the newspaper found, Houston exhibited fluctuations with virtually no chance of occurring except through tampering.

In 2005, scores fell precipitously in five dozen classes in 38 schools after a statistical analysis by The Dallas Morning News suggested test-tampering in Houston. The district fired teachers and principals and improved test security.

In 2011, however, as three-fourths of Houston teachers earned performance-based bonuses, scores rose improbably in a similar number of classes in the same number of schools. In the same year, Houston confirmed nine cheating allegations and fired or took other action against 21 employees.

Through Jason Spencer, a spokesman for the district, Houston officials questioned whether cheating caused all of the unusual score changes the analysis found. He said the district doesn't think its pay-for-performance plan has made cheating more likely.

"We feel like we put a lot of safeguards in place," he said, but he added: "We know it happens. We would never pretend it's not an issue."

When no one's watching

In Dallas, officials said that when irregularities surfaced in 2004, they instituted new test security measures and started screening for anomalies. The number of suspicious gains there dropped after the 2004 allegations — but then began inching up again a few years later.

Leaders need to maintain that tough stance even after cheating disappears from the headlines, experts say.

For years, Los Angeles was among the least suspicious big-city districts. But when California stopped conducting routine erasure analysis in 2008 for budget reasons, the number of improbable score changes in L.A. climbed steeply.

States and districts find little advice when they do decide to conduct erasure or statistical screenings of test scores.

Federal education officials and testing experts have begun working on new recommendations for detecting and investigating test-score anomalies.

Wollack, the Wisconsin testing expert, said there is room to improve. "Some of the investigations that have taken place in the past have been less than thorough, have been less exhaustive than they should have been," he said. "Cheating went undetected as a result."

Daria Hall of the Education Trust said that most educators don't cheat and that testing data are essential for determining if students have basic skills — such as the ability to read.

"What parent doesn't want to know how their child is doing in reading and in math? What teacher doesn't want to know how their student is doing?" she said. "You can't take away the source of the information. We have to make the information better."


Hot spots of suspect scores

Roughly 200 school districts around the country had high concentrations of suspect test scores. For these school systems, the odds of so many suspicious score changes occurring in a single district because of chance alone were extraordinarily low – ranging from less than 1 in 1,000 to less than 1 in 1 billion. The analysis doesn't prove cheating. But experts say cheating is one of only a few explanations for why so many students' test scores would shift so dramatically.

Behind this story

To find school testing irregularities, The Atlanta Journal-Constitution collected databases of standardized test scores from 69,100 schools in 14,743 districts and 49 states. (Nebraska was excluded because it has given a statewide test only since last year.) The analysis covered grades three through eight in reading and math — the grades and subjects required for statewide testing by the federal No Child Left Behind law.

Because of privacy laws, the analysis could not look at individual students' test scores. Instead, it compared test scores achieved by a ‘cohort' of students; that is, when a third-grade class in a school moves on to fourth grade, the group is likely to remain similar and so test scores shouldn't vary much.

From there, the newspaper created a linear regression model for each state, grade cohort and year and compared the average score for a class with the score predicted by the model based on the previous year's average score.

By plotting large changes in scores for a cohort, the analysis was able to identify test results that were highly unlikely to happen without some type of outside intervention, such as cheating at some point in the system.

In some cases, test scores showed dramatic declines in the years after cheating was exposed or investigated.

The newspaper showed its methodology and results to independent experts on testing and data analysis to confirm its findings.

For more on the newspaper's investigation, including a database for district-by-district searching, go online to http://tinyurl.com/6rxnz3t.

Views: 111

Reply to This

JOIN SL 2.0

SUBSCRIBE TO

SCHOOL LEADERSHIP 2.0

School Leadership 2.0 is the premier virtual learning community for school leaders from around the globe.  Our community is a subscription based paid service ($19.95/year or only $1.99 per month for a trial membership)  which will provide school leaders with outstanding resources. Learn more about membership to this service by clicking one our links below.

 

Click HERE to subscribe as an individual.

 

Click HERE to learn about group membership (i.e. association, leadership teams)

__________________

CREATE AN EMPLOYER PROFILE AND GET JOB ALERTS AT 

SCHOOLLEADERSHIPJOBS.COM

FOLLOW SL 2.0

© 2024   Created by William Brennan and Michael Keany   Powered by

Badges  |  Report an Issue  |  Terms of Service