On Tuesday the Organization for Economic Cooperation and Developmentreleased the 2012 scores for the Program for International Student Assessment, commonly known as PISA, the international reading, math, and science exam of 15-year-olds. The United States did not do well: Compared with their peers in the 33 other OECD nations around the world, American teens ranked 17thin reading, 21st in science, and 26th in math. The top-performing region was Shanghai.

While these results always make news, this year there is an added tempest in the teapot of the education policy world: The OECD and the Obama administration worked in advance with a selected group of advocacy organizations to launch a media campaign called PISA Day. Which organizations? The College Board, ACT, America Achieves, and the Business Roundtable—all key architects of the Common Core, the new national curriculum standards whose increased rigor and standardized tests have led to a much-publicized protest movement among some parents, teachers, and kids. Groups that support the Core have an interest in calling attention to low American test scores, which today they will use to argue that the Core is the solution not only to our academic woes, but also to reviving the American economy. Happy PISA Day!

But the truth is that the lessons of PISA for our school reform movement are not as simple as they are often made out to be. PISA results aren’t just about K–12 test scores and curricula—they are also about academic ability tracking, income inequality, health care, child care, and how schools are organized as workplaces for adults.

To figure out what PISA results really tell us, let’s first look at what’s on the test. PISA is quite different from the mostly multiple-choice, fact-driven state exams American kids take annually. The idea of PISA is to test students’ ability to handle words and numbers in real-world situations. One math activity asked students to compare the value of four cars, using a chart showing the mileage, engine capacity, and price of each one. American kids were especially bad at problems like this, in which they were not provided with a formula, but had to figure out how to manipulate the numbers on their own.

A reading activity asked test takers to read a short play, and then write about what the characters were doing before the curtain went up. The challenge is that the question prompts students to envision and describe a scene not actually included in the text itself. These are good questions that most of our kids should be able to tackle—we want analytical, creative children, not just kids who are good at memorization.

Yet we shouldn’t be surprised that our 15-year-olds are stagnant on PISA. Our best American exam, the National Assessment of Educational Progress, already shows that the performance of our older teenagers has been flat since 1971, even as our elementary school kids—especially poor kids—have improved. Our kids do OK when they’re young, but then stall in high school, time and time again. This fact is backed by two other big international exams that get less airtime, TIMSS and PIRLS, which show that American fourth- and eighth-graders are improving in math, science, and reading, and are actually above average internationally.

Why do our little kids do better than our older ones? Ability tracking may have something to do with it. The PISA results show that in higher-performing nations, all students younger than 15 are exposed to the most challenging math concepts. Nations that track their math instruction by ability, like the U.S., do worse on these tests, because fewer kids—especially poor kids—are exposed to the deeper conceptual thinking that becomes more important as the grades progress and tests get harder. This helps to account for why, despite the vast privilege of our most affluent students, only 9 percent of American students perform in the top two categories in math, compared with the global average of 13 percent.

There’s another PISA result that should be heeded just as much as, if not more than, the rankings themselves: The OECD found that school systems with greater teacher leadership opportunities, like Canada’s, outperform those like ours, in which administrators and policymakers exert more top-down control over the classroom, through scripted lessons or teacher evaluation systems that heavily weigh student test scores. Yet you won’t hear about that much on PISA Day, because those have both become popular interventions during the Obama era of education reform.

Critics of the media circus surrounding PISA Day, like the Economic Policy Institute, a labor-oriented think tank, contend that politicians, business leaders, and journalists like to focus on PISA rankings because PISA is the test on which American students do the worst—and thus the results paint a portrait of failing American schools that are responsible for our economic woes. These critics point out that some nations that have historically done well on international exams, like Iceland, were especially hard hit by the recession, and thus there is little reason to believe that better performance on these tests would aid the U.S. economy. After all, we are already producing more college graduates than we can employ in good jobs.

Richard Rothstein and Martin Carnoy write:

Today, threats to the nation’s future prosperity come much less from flaws in our education system than from insufficiently stimulative fiscal policies which tolerate excessive unemployment, wasting much of the education our young people have acquired; an outdated infrastructure: regulatory and tax policies that reward speculation more than productivity; an over-extended military; declining public investment in research and innovation; a wasteful and inefficient health care system; and the fact that typical workers and their families, no matter how well educated, do not share in the fruits of productivity growth as they once did.

That statement contains a lot of sense, and gets to an important underlying point: that test results and education policies are too often considered in isolation from other social and economic realities.

Of course, one goal of education reform is to make sure that the decent opportunities that do exist in our economy are equally accessible to kids who grow up disadvantaged. PISA shows that with about 15 percent of the U.S. achievement gap attributable to poverty, we are smack in the global middle in terms of the effect of socioeconomics on educational achievement. That makes us similar to the U.K., Singapore, Shanghai, Denmark, Spain, Poland, Germany, Brazil, and Argentina, and it means we’re doing better by our poor children than France, New Zealand, Portugal, and Chile.

In Norway, Iceland, Korea, Japan, and Canada, poverty and immigration status have less of an effect on kids’ academic performance than here in the U.S. But these nations also tend to provide children with more social supports outside of K–12 schools, like universal pre-K and health care, both of which help students perform better academically. (Secretary of Education Arne Duncan will soon make that point by pushing the president’s pre-K plan, which is unlikely to attract sufficient GOP support in Congress to become law.) Maybe the takeaway from PISA shouldn’t be that Common Core is the answer, but rather that we need a comprehensive approach to educating and caring for our poorest children in order to close the achievement gap between rich and poor in this country, and between American students and their developed-nation peers.

Still, is it possible that the advocacy groups are right? Will the Common Core raise our PISA scores? Since the Common Core is focused on greater depth and less breadth in our currently rushed and overstuffed American math curriculum, it probably will help our kids do better on exams like PISA. But that’s not the be-all, end-all. It’s important not to forget the other lessons of today’s results: that out-of-school social supports matter, teachers should be empowered, and all kids ought to be exposed to the most challenging material.