(3 of 4)
The early feedback was promising. Principals were lobbying to get their schools switched out of the control group and into the treatment group. Parents began using the paychecks as progress reports, contacting teachers to find out why their kids' checks had gone up or down. In Chicago, Duncan discovered that the program affected kids in ways he'd never expected. "I remember going to schools and seeing how excited the kids were when they got their checks. They were like pep rallies but around academic success!" he says. Fryer appeared on The Colbert Report and CNN to talk about the experiment, and that's about when the death threats started. All the while, Fryer refused to speculate about what the data would reveal. He was not all that interested in whether the kids raised their grades or turned in their homework. Grades are subjective. The more objective measure would come at the end of the year, when the students took their standardized tests. Would they improve more than the kids who were not getting paid? Or would they, as the psychologists predicted, actually do worse? "If it doesn't work, we're going to stop and start doing something else," says Washington schools chancellor Michelle Rhee. "But if it does work, it should drive where we put our money."
The results began to trickle into the lab last summer. In New York City, the $1.5 million paid to 8,320 kids for good test scores did not work at least not in any way that's easy to measure. In Chicago, under a different model, the kids who earned money for grades attended class more often and got better grades, two major accomplishments. Those students did not, however, do better on their standardized tests at the end of the year.
In Washington, the kids did better on standardized reading tests. Getting paid on a routine basis for a series of small accomplishments, including attendance and behavior, seemed to lead to more learning for those kids. And in Dallas, the experiment produced the most dramatic gains of all. Paying second-graders to read books significantly boosted their reading-comprehension scores on standardized tests at the end of the year and those kids seemed to continue to do better the next year, even after the rewards stopped.
The kids had much in common. In all four cities, a majority were African American or Hispanic and from low-income families. So why did the results vary so dramatically from city to city?
One clue came out of the interviews Fryer's team conducted with students in New York City. The students were universally excited about the money, and they wanted to earn more. They just didn't seem to know how. When researchers asked them how they could raise their scores, the kids mentioned test-taking strategies like reading the questions more carefully. But they didn't talk about the substantive work that leads to learning. "No one said they were going to stay after class and talk to the teacher," Fryer says. "Not one."
We tend to assume that kids (and adults) know how to achieve success. If they don't get there, it's for lack of effort or talent. Sometimes that's true. But a lot of the time, people are just flying blind. John List, an economist at the University of Chicago, has noticed the disconnect in his own education experiments. He explains the problem to me this way: "I could ask you to solve a third-order linear partial differential equation," he says. "A what?" I ask. "A third-order linear partial differential equation," he says. "I could offer you a million dollars to solve it. And you can't do it." (He's right. I can't.) For some kids, doing better on a geometry test is like solving a third-order linear partial differential equation, no matter the incentive.
Similarly, in Chicago, kids were paid for grades a result they could not always control. There, the findings were mixed. Kids who got paid did indeed get better grades, and they also attended class more a week and a half more over the school year. That is a big deal, since nearly half of Chicago's high school kids drop out before they graduate and the kids who skip school and fail courses as freshmen tend to be the ones who drop out. We won't know until 2012 if the experiment lowered the dropout rate, but we do know that the rewards did not raise standardized-test scores.
So what happens if we pay kids to do tasks they know how to do? In Dallas, paying kids to read books something almost all of them can do made a big difference. In fact, the experiment had as big or bigger an effect on learning as many other reforms that have been tested, like lowering class size or enrolling kids in Head Start early-education programs (both of which cost thousands of dollars more per student). And the experiment also boosted kids' grades. "If you pay a kid to read books, their grades go up higher than if you actually pay a kid for grades, like we did in Chicago," Fryer says. "Isn't that cool?"
It may also help that the kids in Dallas were the youngest in the experiment, making them more receptive to reforms. It's hard to know for sure. Another caveat is that the Dallas model worked differently on different kids. Most (including Hispanic kids and poor kids) did better when they were being paid. But the ones who spoke very little English and took their standardized tests in Spanish did not benefit from the incentives, a mystery that Fryer addresses at some length in his study but cannot entirely explain.
Meanwhile, in Washington, each school got to choose three of the payment metrics, and some of the elements ended up being outcomes like test scores. But the students were also paid on the basis of attendance and behavior two actions that are under their direct control. Under this hybrid model, the kids who got paid did better on their standardized reading tests. Because of the small size of the school system, the Washington sample was less well balanced than those in the other cities. But its results contain one remarkable finding: the kids who were helped the most by the experiment were the ones who are normally among the hardest to reach. "The typical reform helps girls more than it helps boys," Fryer says. "[This] is the opposite. In D.C., all the results are being driven by the boys. That's fascinating."