If you listen to the reformsters, the answer is an unequivocal "Of course!" Not only that: anyone who questions the narrative of undeniable success in New Orleans is guilty, in the words of Campbell Brown, of "denigrat[ing] hard work & progress of LA teachers, parents & KIDS."
As if the constant droning about the "failure" of American education found at Brown's website isn't the same level of denigration. Way to elevate the conversation, Campbell...
The certainty of the reformy side in the rightness of their beliefs is, of course, a core feature of their movement. I will concede that there are those on the opposite side who rush to condemn any evidence that favors things like charter schools or merit pay or school reconstitution.
But the reformsters always seem to forget that the burden of proof is on them. That doesn't mean that every argument against reformy policies has to be rebutted beyond any doubt; it does mean, however, that reasonable critiques should be engaged beyond simple mockery.
Take Peter Cook, for example, striking back at Andrea Gabor's NY Times op-ed, which questioned the awesomeness of reform in NOLA:
It's quite clear to those of us who study these things that Gabor was talking about how the Louisiana's tests compare to other states' in terms of their difficulty. And Gabor is right: when mapped against the NAEP, the national standardized test, Louisiana's definition of proficiency is, in fact, quite low. It's perfectly appropriate to point this out when presenting a critique of the research that purports to show NOLA's reforms have led to better student outcomes. Why, then, does Cook snidely dismiss Gabor's point?
The reliably reformy Jon Chait has put together a collection of research that supports the NOLA success narrative. I'll admit that some of it is well worth considering. Doug Harris, for example, is a well-regarded and highly competent economist, and the work of the Education Alliance for New Orleans should be seriously considered in any conversation about school reform in New Orleans.
But no one should unquestioningly jump to the conclusion that we need to replicate NOLA in cities around the country on the basis of a brief Harris wrote for EducationNext. First of all, even he admits it doesn't document his full methods (Harris very graciously replied to an email I sent asking him when the "long form" of his research will be out; he said sometime within the next few months, hopefully).
More importantly, and as Harris says in a surprisingly level-headed interview with Matt Barnum for The Seventy Four*, there is all the world of difference between acknowledging some outcome gains in NOLA and imposing some of the reforms that have been implemented there on the rest of the country:
That's a point that needs to be stressed repeatedly: New Orleans is merely one example of education "reform." It doesn't necessarily "prove" the reform agenda is unfailingly successful, any more than Ohio "proves" charter schools are always doomed to failure, or Newark "proves" that school reconstitution is always a train wreck.
Social science is a messy business, and results, even in tightly controlled randomized experiments, are hard to replicate. Which means that caution is always warranted when moving from research findings to policy formation -- especially when there is substantial evidence that contradicts the portrait of New Orleans as an unbridled success:
That's Julian Vasquez-Heilig**, another well-regarded education researcher. Even if there have been significant improvements in New Orleans, by many metrics the Recovery School District and the State of Louisiana are low performing systems.
Dropout and Graduation
Further, as this brief from the National Education Policy Center points out, the cost of these test score gains may be unacceptably high:
Who's right: NEPC and Vasquez Heilig, or Harris and ERA? The answer, of course, is that they both are -- but the question is flawed. It's not a matter of the data "proving" which policy course we must pursue; it's a matter of weighing the evidence within context to inform decisions about how to improve our education systems.“Recently, the Education Research Alliance for New Orleans, an organization at Tulane University dedicated to objective analysis of the New Orleans reforms, shared preliminary results from a study that reported meaningfully higher test scores following the post-Katrina reforms, even after accounting for population changes,” said Professor Huriya Jabbar, who has studied the New Orleans reforms. “However, the authors also report that the gains were not equal across groups: white students gained more than black students from the reforms.“Furthermore, researchers have not yet determined which features of the reforms were successful (e.g., autonomy, teacher labor market reforms, and increased resources, including the influx of private philanthropic funds), as well as the role of other citywide changes in housing and employment.”Moreover, groups of students, parents, and community members remain skeptical of the reform movement and have raised concerns that the new school system remains inequitable. For example, students and parents have raised concerns with some charter schools that have been unresponsive to students and too harsh in their disciplinary policies. After years of complaints lodged by parents about the treatment of students with special needs in the charter system, including physical and emotional abuse and “counseling out,” the parties settled a lawsuit brought by the Southern Poverty Law Center, acknowledging these grievances and requiring independent monitoring and auditing of charter schools’ special education services. According to news reports, the decentralized, fragmented school system in New Orleans has also been particularly unprepared to serve the growing percentages of English Language Learners in the city.Further, within the choice system family income exerts a strong influence. A recent study found that low-income families make schooling decisions differently than affluent families. Low-income families are much more constrained in their choices because of practical considerations such as after-school care and distance, and therefore measured academic outcomes play a smaller role their decisions.Schools have also engaged in behaviors that constrain parents’ choices. A recent study revealed that school leaders in New Orleans, facing the combined pressure of recruiting more students and raising test scores, cream-skimmed the more affluent or high-achieving students through, for example, selective advertising and recruitment.
So what's the context of New Orleans?
Prior to Katrina, NOLA was one of the lowest-performing and most poorly managed school systems in the nation, suffering from the effects of segregation, inadequate funding, and chronic poverty. Then an unprecedented (at least in the modern Unites States) natural disaster displaced an enormous number of people, changing the demographics of the city:
Afterward, a large amount of public funds poured into the schools. The teacher workforce changed dramatically, as did the governance of the schools. Charter schools flourished, but arguably at the expense of special education students, and with great variation in their spending.
There appear to be some significant test score increases in New Orleans' schools -- although there continue to be questions as to whether or not researchers have adequately controlled for changes in the city's student population. However, the district still lags far behind the rest of the nation.
Further, while charterization was a significant change in New Orleans education policy, the expansion of charters was concurrent with increased spending on schools, large changes in social services spending and delivery, and, again, substantial changes in the demographics of the student and city population.
I think that's a fair reading of what's been happening in New Orleans -- and I think it isn't nearly enough evidence to charge into a wholesale dismantling of our urban school systems. If there is a lesson to be learned from New Orleans, it's that too many in the education policy arena are ready to jump on a study or two that confirm their ideological predilections without stopping to consider all of the ramifications of what they propose.
In other words: No one has come close to showing that New Orleans has "proved" anything about how to improve America's urban school districts.
So relax, reformsters: it's called The Big Easy for a reason.
God bless NOLA.
* Just to be clear: I'm not surprised that either Harris or Barnum is level-headed; rather, I'm surprised such a nuanced view showed up in The Seventy Four.
** So I see a lot of criticism out there of this brief. I won't defend it, but I will point out it shows Louisiana is a low performing state on the NAEP, which counts for something if the measure of success in NOLA is how the city does compared to the rest of the state. Also, I haven't read Lopez and Olson's NAEP study, so I'll reserve judgment on that until I do...
Hmm, looks like that hasn't been published. Which is the problem with this whole conversation, isn't it? I wish NPE had waited and given us some real depth, much as I wish EducationNext had given us the long-form results of Harris's work.