I will protect your pensions. Nothing about your pension is going to change when I am governor. - Chris Christie, "An Open Letter to the Teachers of NJ" October, 2009

Saturday, February 17, 2018

Arming Teachers Is a Bad Idea

UPDATE: A reader points me to this 2014 piece by Russ Moore, a gun owning principal. It's good. 

I know I'm going to get a lot of grief for this post, but someone's got to say it:

Arming teachers is a bad idea.

I know there are a sizable number of teachers who are licensed, responsible gun owners. No one I know wants to take away your right to have a gun, although plenty of us want greater regulation, as is demanded by the Second Amendment*, than we have now.

But bringing your gun to school so you can take a law enforcement role concurrent with your responsibilities as a teacher is a bad policy, for many reasons:

- Armed teachers are not adequately trained or experienced, and they never will be. Your proficiency with a firearm is not adequate qualification for you to be carrying one at school. You might be able to hit a target every time, but you will never have the training or experience to be entrusted to know which target to hit.

"Oh, I'll train during the summer." That's not enough. Law enforcement officials go out on the streets every day. They get experience every day. They are supervised in the use of deadly force every day. There is no conceivable scenario where you can meet your obligations as a teacher and receive adequate training and experience so you know not just how to use a deadly weapon, but when.

"But I was in the military." Thank you for your service. But military experience is not the same as civilian law enforcement experience. The rules of engagement are not the same. The environment of the battlefield and the school are completely different. And no one ever expected that you could engage the enemy while simultaneously teaching children -- two tasks with extraordinarily large cognitive loads that, if we're being honest with ourselves, no human being can do at the same time.

- We can't afford it. There is no way the American public would ever accept armed educators in the classroom unless they were regularly screened, trained, and supervised (even then, I'm betting most parents would be against it). This costs money -- money we could be spending on school facilities, salaries, curriculum, etc.

You can't expect to arm teachers and not increase their workload. You can't expect to increase their workload and not pay them more. But why would we pay teachers more to partially train them in law enforcement techniques when we could instead pay them more to train them to be better teachers?

- There's no evidence it will work. I'm not going to wade into a long battle over the evidence on gun proliferation and crime. Suffice to say there is an extraordinary amount of junk research out there**, and both-sidesism runs rampant. A large part of the problem is that the Centers for Disease Control has been prevented from engaging in research on the correlation between gun ownership and gun violence.

If you want to ignore the reams of evidence that shows gun ownership does not lead to a reduction in violent crime, I can't help you. But at least show me a body of high-quality research that supports the contention that arming teachers will make students safer. Not anecdotes; not non-peer reviewed hack junk. I'm talking serious, empirical evidence. Where is it?

- It's completely impractical. The premise for arming teachers is that they will have their weapons at the ready on a moment's notice as they will be the first-responders to a gun attack. Which means they will have to carry a loaded weapon with them at all times.

A law enforcement officer's foremost priority is making sure her weapon is secured. A teacher's foremost priority is the needs of her students. These are not the same things. It is very easy to imagine all sorts of scenarios where a teacher needs to focus completely on a student to the exclusion of everything else. But that can never happen so long as that teacher has to put the security of their firearm first.

The more you think about this, the more absurd the whole idea becomes. Are you really suggesting a gym teacher should lead his students in basketball drills with a gun strapped to his chest? A music teacher should conduct a choir with a sidearm? A kindergarten teacher should give a hug to a crying child while carefully making sure that child doesn't touch her gun?


- An armed teacher will always be suspicious. I'm sure most of the teachers who are calling to be armed are very nice folks who have the best of intentions. But are we sure all of them are?

Teachers are generally stable, moderate people. But they're people. They have bad relationships and battles with addiction and personal demons at the same rates as everyone else. Are we really confident we can screen out teachers who shouldn't be armed? One bad call is all it will take.

The reality is that an armed teacher will always be treated differently than an unarmed one. And if I'm a law enforcement officer who arrives at an active shooter scene and I see an armed civilian firing a weapon around children...

Look, we can have a serious debate about how to protect our citizens -- especially our children -- from the horrific tragedies that are plaguing this country while respecting the rights of responsible gun owners. I have no problem with hunters and marksmen and people who need a gun for their personal protection. I agree that banning all guns will not solve our problems.

But arming teachers is not a serious policy proposal; it is simply a bad idea. If we're going to create good public policy that protects students in schools, we should not waste time considering whether we should arm teachers -- that's a distraction we really can't afford right now.

ADDING: A commenter on Facebook:
I worked with a retired NJSP officer who took a security job at a HS. He said he'd NEVER bring his gun to school because it might be taken from him in a scuffle and there was a HUGE risk of collateral damage in a group situation if he had to use it. He was a senior officer with 26 years' experience.
I'm obviously not a law enforcement professional. But isn't this the reason why corrections officers don't carry weapons when they are in proximity to the prison population? Isn't this why police officers don't bring their weapons into holding areas?

I made an assertion above that I really should confirm with law enforcement officers: when you are issued a weapon, isn't your top priority making sure that weapon is always secured?

* "A well regulated militia, being necessary to the security of a free state, the right of the people to keep and bear arms, shall not be infringed."

** And if anyone knows about junk research, it's an education policy researcher...

Friday, February 16, 2018

Don't Believe The Hype: The Facts About Graduation Rates in Newark and Camden, NJ (UPDATED)

This week, the NJ Education Policy Forum published a brief of mine on graduation rates in Newark and Camden. Not more than a couple of days later, however, the NJDOE published new graduation rate data for 2017.

So this post has two purposes: to update the data, and to explain what my analysis shows. Let me give some background first:

Over the years, two cities in New Jersey have been held up as national models for education "reform": Newark and Camden. Both are urban districts with severe economic challenges, both have many children living in poverty, and both have large populations of English Language Learners.

In addition, Newark and Camden have school districts that have been under state control for years: Newark since 1995, and Camden since 2013 (although Camden was subject to regulation by a state monitor well before that). Under state control, both cities implemented similar policies for their districts, including large expansions of charter schools.

Both cities have been run for the past several years by state superintendents who were appointed by former Governor Chris Christie. Cami Anderson was superintendent in Newark from 2011 to 2015, followed Chris Cerf, the former state Commissioner of Education, who just recently resigned. Paymon Rouhanifard was appointed by Christie in 2013 and remains in his role. All three cut their teeth in the New York City DOE during the time when Joel Klein, a master of self-promotion, was the chancellor of schools. None had ever run a school district before taking their positions.

In the past year, there's been a lot of talk about whether or not Camden and Newark have seen improvements during Chris Christie's two terms. As I explain in the brief, graduation rates are often brought up by supporters of Anderson, Cerf, and Rouhanifard as proof that their policies and leadership got positive results.

Just this past week, for example, the Star-Ledger gave a glowing review of Cerf's tenure: "The graduation rate has jumped to 77 percent, up by more than 20 points."* And last month the NY Times suggested Rouhanifard would make a fine choice to return to NYC and take over as chancellor: "In Camden, which was seen as one of the most troubled school systems in the country, Mr. Rouhanifard has improved graduation rates and lowered suspension rates."

The problem with claims like these is they lack context. It's great that graduation rates have climbed in Newark and Camden... but what if they've been climbing everywhere? What if the rises in graduation rates are part of an overall trend, and have little to nothing to do with these superintendents' policies or leadership? If that's the case, there's no reason to believe state control has led to better graduation outcomes.

So that's what I set out to explore: Did Newark's and Camden's graduation rates rise faster than other districts? Or is there a universal (what researchers often call a secular) trend that explains the rise? It sounds like a simple enough analysis, right? All we have to do is look at whether these cities saw their rates increase faster than the rest of the state...

Except there's a problem.

If you compare Newark's or Camden's graduation rate changes to all the other districts in New Jersey (which is what one author does here), you're going to have to deal with something known as a ceiling effect. I've blogged about this before:

You can't have a graduation rate over 100 percent, right? So there's a "ceiling" effect: because so many suburban districts were already graduating almost all of their students, they really couldn't improve. But more disadvantaged districts, with lower graduation rates, could and did improve.
It makes no sense to judge Newark or Camden against districts that have graduation rates near 100 percent and therefore can't get any higher. What we should do instead is judge their graduation rate growth against similar districts -- which is what I've done in the brief, and now updated here with 2017 data.

To determine which districts we should compare to Newark and Camden, I use the state's District Factor Groups (DFGs). Districts in the same DFG have similar socio-economic profiles; both Camden and Newark are DFG-A districts. The state began a new way of reporting graduation rates in 2011, so that's the baseline year. You can find all the rest of the technical information in the brief.

So, what do the data show?

The rise in graduation rates for Newark and Camden simply mirror a trend for all similar districts in New Jersey.

You can see this from the green line: all other DFG-A district saw their rates rise 11 percentage points over the past seven years. Camden's rose a bit less; Newark's somewhat more. But there's no doubt an overall trend in rising graduation rates for DFG-A districts explains a great deal of the trend in Newark and Camden.

Now, one of the arguments for state control is that they've allowed for charter school growth, and that's helped improve outcomes in Newark and Camden. As Bruce Baker and I explain in great detail, there isn't much evidence to support that contention. But let's go ahead and add charter schools into the mix -- how do Newark and Camden compare when we add in local charter school graduation rates?**

Again, it's clear that the statewide rise in graduation rates explains much of the growth in rates in Newark and Camden -- even when we add in charter schools. Yes, Newark does close the gap somewhat with other DFG-A districts. But I did some econometric modeling, and the change is not statistically significant. The updated regression outputs, using 2017 data, are below.

In addition: Baker and I found in our review that there are differences in outcome growth that appear to be correlated to regional differences; in other words, test score outcomes change in different ways in different parts of the state. So let's just look at the changes in DFG-A graduation rates for districts that are close to Newark -- in this case, within the same labor market.

Newark's graduation rates grew at almost the same rate as all the other DFG-A districts in its immediate area. And Camden?

There's only one other DFG-A district in Camden's labor market (Paulsboro). The data are noisy, but it's clear that Camden is not showing outsized growth in its graduation rate; if anything, the district is slipping.

I can only conjecture at this point, but I suspect the overall rise in graduation rates may be due to a policy called credit recovery. Newark pushed it hard in 2012, which might explain the initial bump in its graduation rate. Credit recovery includes a variety of programs -- include online learning, summer school, and so on -- that give students an opportunity to make up high school credits in subjects they previously failed.

Credit recovery is controversial: there's a lot of speculation that many programs used in credit recovery (especially on-line learning programs) are low in quality. There hasn't been a lot of study of credit recovery, and I don't know of any work specific to New Jersey on the issue.

In any case, no matter why the rates are rising, it's clear that they are rising in all DFG-A districts. Which means the claim that Newark's and Camden's graduation rate growth is due to the leadership of Anderson, Cerf, or Rouhanifard is little better than hype.

Look, I don't think it's wrong for any superintendent to promote the successes of their school district. But Newark and Camden have repeatedly been held up as exemplars, and reformy folks have pointed to their graduation rates as proof that charter school expansion and state control and "great school leaders" get results.

That's just not the case -- and it's unfair to districts like Paulsboro and Elizabeth and Orange and Irvington and Dover and East Orange, which have seen their rates rise just as much, if not more. Most of these districts haven't had large growth in their charter sector, and they remain under local control. Many of their superintendents are lifelong educators, and took the traditional route to district leadership positions: classroom, building principal, central office, with a stop along the way at a traditional university or college where they earned an advanced degree in educational administration.

Where is the applause for these leaders? Why aren't newspapers interviewing them? Where are the calls from editorial boards to bring them on board in the biggest urban districts? Why aren't they proof that local control of schools can yield good results?

I've noted this before: there are a lot of good schools and good school districts out there, serving many children living in economic disadvantage, that never get their due. They may not have leaders who relentlessly promote themselves, but that doesn't mean we should ignore them. And we certainly shouldn't foist policies on them when those policies come from districts that don't perform any better.

Here are the updated regression tables:

* The piece never spells out exactly what the baseline year was for comparison.

** Aggregating the rates of charter and public district schools is tricky stuff. See the brief for how I did it.

Wednesday, January 31, 2018

Is "No Excuses" Narrowing The Curriculum? Evidence From Newark

This post is part of a series on recent research into Newark schools and education "reform."

Here's Part I.

Here's Part II.

This is Part III.

* * *

Just a few more thoughts about the recent research Bruce Baker and I did on Newark, one of the most prominent test cases of education "reform" to emerge in recent years.

We wrote our research brief, published by the National Center for Education Policy, as a response to recent research by a team of Harvard economists that (to state it briefly) purports to show how moving students from "low-performing" public schools to "high-performing" charters lifted the overall outcomes of students in Newark.

However, as Bruce and I point out: the gains were quite small, they appear to be completely tied to a change in exams (from the old NJASK to the PARCC), and other districts in Essex County, NJ showed similar results. So it's not as if there is much for the supporters of "reform" in Newark, including Mark Zuckerberg, to crow about here. 

Still, there's an interesting question that arises in studying the Newark "reforms," or any other educational intervention: what, exactly, are the (small) test score gains really indicating?

It so happens that as I was writing our report, I was also reading Daniel Koretz's excellent new book, The Testing Charade. One of the central arguments Koretz makes is that focus on test outcomes can distort educational programs to the point where:
  1. The "learning" that takes place in a test prep environment amounts to little more than pedagogical shortcuts that might increase test scores, but aren't reflective of deepening knowledge.
  2. Non-tested subjects -- history, lab sciences, civics, music, art, physical education, etc. -- are given short shrift.
As a music educator, the second point is the one that concerns me the most. Charter school marketeers like Eva Moskowitz will tell us over and over how rich and deep her schools' curriculum is ("Chess! We have chess!"), but as I showed last year, the plain fact is these schools do not come anywhere close to offering what affluent suburban schools offer.

But how do charters compare to their neighboring urban district schools? Do charters sacrifice instruction in non-tested subjects? Do they offer less programming in, say, the arts compared to their host districts -- keeping in mind that arts offerings in urban district schools are often considerably less expansive than those in more affluent communities? 

I'll be the first to say this is a big topic and the data we have isn't going to give us a definitive answer. But there are clues -- good clues -- that suggest charters may not be putting as many resources into non-tested area instruction as public district schools.

We have access to what are known as "staffing files" for all publicly-funded schools in New Jersey, both district and charter. The files list characteristics of every certificated school staff member in the state (yes, I'm in there also). One of those characteristics is the "job code," which is related to the subject area for which a teacher receives a certificate. I, for example, am listed as a "music teacher," and my certification says I have met the requirement to be considered competent to teach in that subject.*

Because we can determine the number of staff with particular certifications, and because we know the numbers of students enrolled at a school, we can calculate the "student load" for teachers in different subject areas. This gives us, arguably, an indication of how expansive programming in particular subjects could be. For example: if one school has enough art teachers that each teacher has a "load" of 400 students, we would expect the school to offer more art than a school where teachers have a "load" of 800.

There are some caveats that go along with this. Comparing schools with different grade levels, for example, becomes problematic, as school systems might rationally decide that the teachers of older students may be able to offer high-quality programming even with a greater "load" than teachers of younger students. Still, this is a good first step for our purposes.**

So let's look at the data. How, for example, does the deployment of arts teachers compare between the Newark Public Schools and the city's charter sector?

Year after year, Newark's charters provide their students with fewer art teachers per student, compared to NPS. There are some pretty wild swings in the data for the charters (NPS is quite steady). Historically, however, the charter sector just hasn't deployed as many personnel toward teaching art as the public, district schools.

Let's look at some other examples. Music:

Physical education:


Social Studies:

World Languages:

Again, there are some big swings. But the general pattern in Newark is that the charter schools historically have tended to deploy fewer teachers per pupil in non-tested subjects compared to NPS.

And when you think about it... how could they? Even the largest charter chains -- TEAM/KIPP and Uncommon/North Star -- have small enrollments compared to the size of NPS. Scale issues can make it hard for a charter school to offer all the programming a large school district can. And the mom-and-pop charters just don't have the numbers of kids to justify keeping a full-time music teacher AND an art teacher AND a PE teacher AND foreign language teachers on staff.

But more than that: as Koretz points out in his book, the leaders of the big charter chains all admit they place great value on test outcomes -- hell, they've written entire books about it. Doesn't it stand to reason that they'd sacrifice non-tested subject instruction if that meant better test scores? Given their own words, isn't this kind of staff deployment exactly what we'd expect to find?

Again: I don't pretend this is the last word on the subject. But I think this is more than enough evidence for charter regulators to start taking a close look at the entire curriculum within these schools. If we really believe that all children should have a rich, deep education in many domains of learning, shouldn't we make sure that schools funded with taxpayer monies are doing just that?

That's enough about Newark for now... time to think about some other things...

* Among other things, getting a certificate requires a certain amount of college-level coursework specific to a subject; for example, I had to have a designated number of credits in music. You also have to pass a subject-specific test, like the Praxis.

** I continue to work on this technique, which I have used before in similar research. Early indications are the grade level a charter serves doesn't have a big effect on these ratios... but this is, admittedly, work in progress.

Monday, January 15, 2018

The End of the Teacher-Bashing, Chris Christie Era


Allow me a few personal thoughts:

Eight years ago, I started this blog in direct response to what I and many other teachers around the state perceived as a climate of teacher bashing brought on by New Jersey's governor, Chris Christie.

Today is Christie's last day as governor. Somehow, we teachers survived.

I'm only being a little hyperbolic when I say this. When I look back on Christie's two terms, I see both a series of policies and a set of attitudes that were -- and are -- a threat to the teaching profession in New Jersey.

- The value of our modest pensions and health care benefits (which are less generous than those found in the private sector) continues to erode, and current retirees have lost their cost-of-living-increases. We were already paying a wage penalty for choosing to become teachers. Now, Christie's appointees want us to give up even more of our compensation, even as pension fund managers collect outrageously high fees.

- New Jersey teachers are subject to an innumerate, illogical evaluation system that uses arbitrary weights of error-prone measures of "growth" that appear to be significantly biased. In short, NJDOE under Chris Christie has created an unvalidated mess of a teacher evaluation system that wastes time and money.

- Despite these serious problems with NJ's evaluation system, Christie has worked for years to undermine tenure and other workplace protections for teachers -- which happen to also be protections for taxpayers and students.

- Christie has demeaned the professionalism of educators by consistently appointing people into leadership positions who have neither the experience, the qualifications, nor the track records necessary for success.

- Christie has promoted the expansion of charter schools, which hire less-experienced teachers at lower pay than hosting public district schools. Many of these charters have serious issues with accountability and transparency, yet Christie enthusiastically supports them. Christie's administration has also turned a blind eye toward charters that clearly do not enroll the same types of students as their hosting public district schools. He has also actively promoted policies that disproportionately affect teachers of color through pubic school "renewal" and charter school expansion.

- Perhaps most important: Christie has tarnished New Jersey's legacy as a leader in school funding reform by promoting inequitable, inadequate school funding schemes and repeatedly ignoring the state's own law regarding school funding.

Add to all this Christie's bullying, preening, sneering, dismissive, sexist attitude toward teachers -- no, not just their unions, but teachers themselves. 

When you total up all of the above, it really is remarkable New Jersey's teachers are still doing the great work they do every day on behalf of this state's children.

Does our profession need to improve? Of course. But Chris Christie's constant undermining of public schools and public educators has done nothing to improve the evaluation, compensation, work conditions, or prestige of New Jersey's teachers. He has made the profession less attractive, which inevitably means fewer of our best and brightest are considering teaching as a career path.

Phil Murphy has a very difficult job ahead. Years of fiscal irresponsibility -- and, yes, I will be the first to say this recklessness long predates Chris Christie -- put him in a very difficult spot. There's little doubt in my mind health benefits for teachers will have to be reconfigured, and it's hard to think of a scenario where we come out ahead. Our policy on state aid to schools needs an overhaul, as does our tax policy.

None of this will be easy. But at least we will have a governor who supports public schools and public school teachers. 

After eight years, I'm looking forward to going to work on Tuesday knowing the governor isn't going to publicly blame me and my fellow teachers for the many problems New Jersey faces that we didn't create and we can't be expected to fix on our own.

Best of luck, Governor Murphy.

And Mr. Christie: have a nice life...

ADDING: As I said, the whole reason for starting Jersey Jazzman was to push back on the anti-teacher rhetoric of the Christie era. I do believe this blog has evolved to become something more than that... and yet this first mission has come to an end.

What's next? Stand by...

UPDATE: You have got to be kidding me...
On his final full day in office, Gov. Chris Christie on Monday signed a controversial bill into law that will increase the pensions of former Camden Mayor Dana Redd -- a Democratic ally -- and some other elected New Jersey officials.
Christie made no statement on the measure, which was one of 150 he took action on before he's set to leave office Tuesday. His office did not immediately return a message seeking comment.
The Democratic-controlled state Legislature fast-tracked the Democratic-sponsored legislation in the final weeks before a new set of lawmakers were sworn in and Christie finished his eight-year tenure.
The new law (S3620) allows some politicians to re-enroll in the state's Public Employees' Retirement System after being kicked out because they switched positions.
Look, I really don't have a problem with anyone getting a pension. I really don't have a problem with this law. But it's utterly, totally hypocritical for Chris Christie, of all people, to sign this on his last day in office after eight years of relentlessly hammering away at the idea that public employee pensions are too generous and must be curtailed.

The excuse that this only affects a few people, and therefore doesn't have a major impact, is completely beside the point. If we can't afford to meet our obligations as a state to workers for work they have already done, how can we possibly justify additional pension spending for a lucky few, no matter how small?

See that quote at the top of this blog?
I will protect your pensions. Nothing about your pension is going to change when I am governor. - Chris Christie, "An Open Letter to the Teachers of NJ" October, 2009
We've been lied to, we've been insulted, and now this -- on Christie's very last day.

I'll tell you something, though: tomorrow, I will join with thousands of New Jersey's teachers, and our cops and firefighters and social workers and DPW workers and everyone else at the municipal, county, state, and school district levels as we head off to work and do our damn best to make this the greatest state in the nation.

Chris Christie was never good enough to lead us. He was never our equal. We are the backbone of the Garden State. We do the work that needs to be done. We are Jersey Strong.

I will always be proud to be a New Jersey public school teacher, a New Jersey union member, and a New Jersey public employee.

I know what I do every day for the people of this state. Every public employee does.

Saturday, January 6, 2018

"Miracle" School "Journalism" and Gorilla Channel Values

UPDATE: We seem to go through this every time I do one of these...

What I show here are the grade-level enrollments each year as a cohort -- the "Class of '17," for example -- passes through a school. This is not student level data; so far as I know, neither NYSED not NJDOE publishes attrition, backfilling, and retention data.

Yes, it's possible students are retained, which could make underclass grades larger than upperclass grades. But that raises a series of issues by itself -- is a school keeping students for five years at extra expense to the taxpayers?

The data is what it is. But the burden of proof is not on me -- it's on those who makes claims about the effectiveness of certain schools. 

I promise I'll get back to Newark in a bit. But before I do, let's talk a bit about education journalism and The Gorilla Channel...

By now, you've probably heard about the internet prank pulled by cartoonist Ben Ward. Ward tweeted out a completely phony "excerpt" from Michael Wolff's new book about Donald Trump, recounting a made-up story about Trump watching videos of gorillas fighting up to 17 hours a day.

It appears that quite a few people fell for the prank and passed on the tweet in ways that made it seem that they thought the story was real. One interpretation of the incident is that too many of us fall too quickly for stories that we desperately want to be true, even if, on their face, they are implausible.*

For me, the timing of all this was fortuitous, because I had just sent out a tweet that captured a paragraph from a blog by Diane Ravitch that asked journalists to be more credulous when covering "miracle" school stories:
Ahem. My take? Journalists should always question stories that involve miraculous claims about test scores and graduation rates. Skepticism should be their default attitude towards claims that sound too good to be true. If at first they take the bait, they will tend to stop digging and become defensive. Those who take the bait will look foolish, and indeed they are. When a school makes outlandish claims about test scores, ask first who graded the tests. Then check the process for excluding and selecting students. Ask whether the school has the same proportion of students with disabilities and English learners as neighborhood schools. Dig deeper. Ask whether it accepts students with cognitive disabilities, or only those with mild learning disabilities. Keep digging. It has been my experience that behind every “miracle” school there is either fraud, dubious practices (e.g., credit recovery), or careful selection and exclusion of students.**
What Diane describes here is painfully common in education journalism: some school official or PR hack (these days, usually from a charter school) calls up the op-ed page editor at a news outlet and pitches a story about a school that "beats the odds." The school almost always has some data point they are using to sell their "success." But the editor or reporter almost never follow through and put that data into proper context.

What emerges is a story that feels right to folks with reformy predilections, but isn't based on a full and proper accounting of the facts. "Miracle" school stories are like The Gorilla Channel: because people want to believe them, they suspend critical thinking and accept them without subjecting them to appropriate scrutiny.

Which brings us to this recent op-ed from the NY Daily NewsApparently, no one at the News thinks they should check their op-ed writers' claims to see if they actually make any sense:
It’s college acceptance season, and if you’re a senior at one of Democracy Prep’s high schools in New York City chances are you are very happy: No charter network has demonstrated more success getting its students into and through college.
According to the network, last year 189 of the 195 seniors in its three high schools that had graduating classes went on to college. And although the sample size is small (the network has graduated fewer than 400 students), the network estimates that 80% of its graduates either are still in college or have graduated. 
Nearly all Democracy Prep graduates are from low-income families. Nationally, less than 20% of high school graduates below the median of household income receive bachelor's degrees within six years. 
Do you want this to be true? Would it make you feel better if you knew that children from low-income families, growing up in cities whose schools have been under-resourced for years, didn't suffer a penalty in college admissions? Would it make you feel better about our nation's appalling neglect of its most vulnerable children if you could point to a few schools that "beat the odds"?

We know there is systemic, structural inequality and racism in America. We know our schools reflect this, and that educational outcomes are influenced by these ugly societal realities. But here's a story that calls these truths into question. How much do you want to set aside your skepticism? Enough to keep you from examining the facts?

Ok, then...

According to Democracy Prep's website, the network runs five high schools in New York City:

  • Democracy Prep Charter High School
  • Democracy Prep Harlem High School
  • Bronx Prep High School
  • Democracy Prep Endurance High School
  • Harlem Prep High School

I am using NY State Education Department data for this analysis. I'll start by aggregating these five schools and asking a basic question: How many students who start as freshmen at Democracy Prep stay until their senior year?

This is important: if a school sheds students as they move from year to year, it suggests that school isn't placing all of its students in college -- it's only placing those who stayed for the full four years. What does the data show?

Year after year, Democracy Prep's charter schools shed substantial numbers of students between their freshman and senior year.

Is a school really placing all of its students into college if it loses substantial numbers of those students before their senior year? Is it fair to make a comparison of college placement rates between Democracy Prep and other high schools enrolling large numbers of low-income students if:
  1. Democracy Prep only enrolls students who apply to be there?
  2. Many students leave Democracy Prep before their senior year?
Keep in mind that the author of this piece, Charles Sahm, claims that cohort attrition is not an issue at Democracy Prep:
Unlike other charters that don’t accept new students after a certain grade, Democracy Prep — which operates elementary, middle, and high schools — takes in new students whenever a new spot opens up, even in later grades. [emphasis mine]
Maybe they do -- but Democracy Prep does not come close to replacing all of the students it sheds from year to year.

Did anyone at the News think to check Sahm's claim to see if it was both accurate and put into proper context? Did they think they have an obligation to their readers to make sure that relevant facts were being presented? Did they stop to consider that anyone making a bold claim like Sahm's should be required to subject that claim to rigorous scrutiny?

Education journalism in this country is too often shaped by Gorilla Channel values: because newspapers editors and publishers desperately want to believe certain things, they are willing to suspend critical thinking and merely report whatever they are fed.

How desperately do you want to believe in The Gorilla Channel? How desperately do you want to believe in "miracle" schools? Are you willing to have a good-faith argument about education policy?

Or is that too much to ask?

Another "miracle school" op-ed, courtesy of The Gorilla Channel!

ADDING: Sahm notes that Democracy Prep runs schools in other states:
Democracy Prep’s impact is not limited to New York. The network’s high school in Camden, N.J., is graduating its first class this year. All 34 of this year’s seniors have already received at least one college acceptance. This is revolutionary change for a city that five years ago only had three students graduate high school “college ready,” according to the College Board.
As luck would have it, I happen to have some familiarity with NJ education data...

Freedom Prep, the Democracy Prep affiliate in Camden, NJ, has not yet graduated a class. But we can see the cohort sizes up until last year.

This year's seniors at Freedom Prep were in a class of 78 freshman. By the fall of their junior year, they were down to 33 students. 58% of the freshmen of the Class of 2018 at Freedom Prep had left by their junior year.

Shouldn't this be discussed?

ADDING MORE: Gary Rubinstein calls out the Gorilla Channel values of the press's coverage of Success Academy. It ain't pretty...

* OK, look: I will be the first to say Donald Trump is unfit for office. But come on...

** Emphasis mine; lightly edited for clarity.

Tuesday, January 2, 2018

Only You Can Prevent School Finance Ignorance!

We interrupt our regularly scheduled, fact-based analysis of Newark's schools for a good, old-fashioned school finance debunking...

I'm thinking about trying to convince @SchlFinance101 to start a School Finance Hall of Shame, where we would regularly acknowledge great feats of public ignorance in the field of education fiscal policy.

My first nominee for 2018 would have to be this piece in today's edition of NJ Spotlight, a monumental display of school finance ignorance. For the academy's consideration, I offer this excerpt:

Senseless school funding

Compounding the problem for middle-class taxpayers is the state’s senseless school-funding plan that shovels truck loads of money to a handful of school districts while leaving suburban taxpayers to fund more than 90 percent of their school needs while they also pay more than 80 percent of other people's school costs. More state school aid money goes to Newark than to all of Monmouth and Ocean counties combined. Morris County, in total, gets less state aid than the City of Passaic; the same goes for Bergen County. Jersey City gets more annual state aid than either Salem, Somerset, Sussex, or Warren counties. Yet, the Legislature refuses to make changes to fairly distribute state aid that comes from the very taxpayers who are paying the most taxes to Trenton.
Where to begin?

First: true greatness in school funding ignorance requires comparisons that are not only specious, but utterly lacking in any attempt at fairness. Take, for example:
Jersey City gets more annual state aid than either Salem, Somerset, Sussex, or Warren counties.
We'll leave aside the many complex problems with this comparison and ask a very, very simple question: Should a small, sparsely populated county get as much aid as a large, densely populated city?

Even if you believe in absolutely insane ideas like "fair funding," where every student gets the same amount of state aid, Jersey City would get more aid than Salem, Sussex, and Warren Counties because Jersey City has more students!

I really should stop here - this op-ed, which can't even deal with the most basic mathematical concepts, clearly doesn't deserve any more attention. But let's use this as a teachable moment and dive into another concept that appears to elude so many who opine so loudly about school finance policy -- tax capacity.

Imagine two houses that are exactly the same, but located in two different towns.
Each house is exactly the same in all details, and each costs the same (in the real world they don't*). One, however, is located in hard-scrabble Palookaville, where it's one of the most expensive properties in town. The other is the cheapest house in Hoity-Toity Village, where McMansions abound.

In other words: one house is located in a relatively property-poor town, while one is in a property-rich community.
Now, for the purposes of this example, we will do something utterly unforgivable and set aside the tons and tons of research that shows students in greater economic disadvantage require more resources to achieve equal educational opportunity. Instead -- just this once -- we will imagine that each town wants to raise equal amounts of revenue per pupil for its schools.

Again, we are setting aside boatloads of research that shows less advantaged students need more resources and, instead, simply imagining what tax rates each town needs to set to get equal funding for its students. As a matter of basic math, property-poor towns must set higher tax rates to raise the same amount of money as property-rich towns.
In Palookaville, houses cost $100K on average. To raise $10K per house, the town has to set a tax rate of 10 percent. But in Hoity-Toity Village, houses cost $1 million on average. To raise $10K per house, they only have to levy a 1 percent property tax.

Now let's go back to our two equivalent houses. How much does each pay?
Houses in property-poor communities must pay higher tax rates than similar houses in property-rich communities to raise the equivalent amount of revenue.

This disparity is exactly the reason that we have state aid to begin with. If the state didn't step in and try to equalize the different tax bases in communities with varying amounts of property wealth, poor communities would be at a perpetual disadvantage just trying to raise the same amounts of revenue.

Back in 2016, Ajay Srikanth and I broke this phenomenon down in detail when we critiqued Chris Christie's "Fairness Formula." I wish I could say we were making an original point -- we weren't. Because this is one of the most basic ideas in public policy: Wealthier communities have a greater capacity to generate revenues than less-wealthy communities. If you don't understand this, you have no business opining about... well, anything.

Now, I will be the first one to say that New Jersey, and all other states, should take a hard look at how they determine the taxing capacity of school districts before they implement their school funding formulas. In New Jersey, there is a legitimate argument to be made that some -- some -- communities getting large amounts of state aid should pitch in a greater share of local taxes.

But ignorance like this is keeping us from having the serious conversation we need to have. Stop the madness.

Sadly typical school finance roundtable.

* As one of America's best economists, Leah Platt Boustan, points out, people understand that living in a property-wealthy community can lower their tax rate. So they will pay more for their house to live there. The houses in our example wouldn't cost the same; the one in the wealthy community would cost more. And it would be worth it: better services, lower tax rates.

You get this, right? If not, I'll keep trying...

Saturday, December 30, 2017

Test Scores Gains Are Not Necessarily a Sign of Better Instruction: A Cautionary Tale From Newark

This post is part of a series on recent research into Newark schools and education "reform."

Here's Part I.

Here's Part II.

* * *

In this series, I've been breaking down recent research about Newark, NJ's schools. Reformy types have been attempting to make the case that "reforms" in Newark over the past several years -- including charter school expansion, merit pay, Common Core alignment, school closures, and universal enrollment -- have led to gains in student learning. These "reforms" are purportedly the result of Facebook CEO Mark Zuckerberg's high-profile, $100 million grant to the city's schools back in 2010.

Zuckerberg recently funded a study, published by the Center for Education Policy Research at Harvard University this past fall, that shows a gain in "value-added" on tests for Newark compared to the rest of the state. (A technical paper, published by the National Bureau of Economic Research, is found here.)

Bruce Baker and I looked carefully at this study, and added our own analysis of statewide data, to produce a review of this research. One of our most important findings is that most of the "gains" -- which are, in our opinion, educationally small anyway (more on this later) -- can be tied to a switch New Jersey made in 2015 from the NJASK statewide exams to the newer PARCC exams.

As I noted in the last post, even the CEPR researchers suggest this is the most likely explanation for the "gains."
Assuming both tests have similar levels of measurement error, this implies that the PARCC and NJASK were assessing different sets of skills and the districts that excelled in preparing students for PARCC were not necessarily the same as the districts that excelled at preparing students for NJASK. Thus, what appears to be a single-year gain in performance may have been present before 2015, but was simply undetected by earlier NJASK tests. (p. 22, NBER, emphasis mine)
As I pointed out last time, there has never been, to my knowledge, any analysis of whether the PARCC does a better job measuring things we care about compared to the NJASK. So, while the PARCC has plenty of supporters, we really don't know if it's any better than the old test at detecting "good" instructional practices, assuming we can hold things like student characteristics constant.

But even if we did have reason to believe the PARCC was a "better" test, I still would find the sentence above that I bolded to be highly problematic. Let's look again at the change in "value-added" that the CEPR researchers found (p. 35 of the NBER report, with my annotations):

"Value-added" -- ostensibly, the measure of how much the Newark schools contributed to student achievement gains -- was trending downward prior to 2014 in English language arts. It then trended upward after the change to the new test in 2015. But the CEPR authors say that the previous years may have actually been a time when Newark students would have been doing better, if they had been taking the PARCC instead of the NJASK.

The first problem with this line of thinking is that there's no way to prove it's true. But the more serious problem is that the researchers assume, on the basis of nothing, that the bump upwards in value-added represents real gains, as opposed to variations in test scores which have nothing to do with student learning.

To further explore this, let me reprint an extended quote we used in our review from a recent book by Daniel Koretz, an expert on testing and assessment at Harvard's Graduate School of Education. The Testing Charade should be required reading for anyone opining about education policy these days. Koretz does an excellent job explaining what tests are, how they are limited in what they can do, and how they've been abused by education policy makers over the years.

I was reading Koretz's book when Bruce and I started working on our review. I thought it was important to include his perspective, especially because he explicitly takes on the writings of Paul Bambrick-Santoyo and Doug Lemon, who both hold just happen to hold leadership positions at Uncommon Schools, which manages North Star Academy, one of Newark's largest charter chains.

Here's Koretz:

One of the rationales given to new teachers for focusing on score gains is that high-stakes tests serve a gatekeeping function, and therefore training kids to do well on tests opens doors for them. For example, in Teaching as Leadership[i] – a book distributed to many Teach for America trainees – Steven Farr argues that teaching kids to be successful on a high-stakes test “allows teachers to connect big goals to pathways of opportunity in their students’ future.” This theme is echoed by Paul Bambrick-Santoyo in Leverage Leadership and by Doug Lemov in Teach Like a Champion, both of which are widely read by new teachers. For example, in explaining why he used scores on state assessments to identify successful teachers, Lemov argued that student success as measured by state assessments is predictive not just of [students’] success in getting into college but of their succeeding there. 
Let’s use Lemov’s specific example to unpack this. 
To start, Lemov has his facts wrong: test scores predict success in college only modestly, and they have very little predictive power after one takes high school grades into account. Decades of studies have shown this to be true of college admissions tests, and a few more recent studies have shown that scores on states’ high-stakes tests don’t predict any better. 
However, the critical issue isn’t Lemov’s factual error; it’s his fundamental misunderstanding of the link between better test scores and later success of any sort (other than simply taking another similar test). Whether raising test scores will improve students’ later success – in contrast to their probability of admission – depends on how one raises scores. Raising scores by teaching well can increase students’ later success. Having them memorize a couple of Pythagorian triples or the rule that b is the intercept in a linear equation[ii] will increase their scores but won’t help them a whit later. 
Some of today’s educators, however, make a virtue of this mistake. The[y] often tell new teachers that tests, rather than standards or a curriculum, should define what they teach. For example, Lemov argued that “if it’s ‘on the test,’ it’s also probably part of the school’s curriculum or perhaps your state standards… It’s just possible that the (also smart) people who put it there had a good rationale for putting it there.” (Probably? Perhaps? Possible? Shouldn’t they look?) Bambrick-Santoyo was more direct: “Standards are meaningless until you define how to assess them.” And “instead of standards defining the sort of assessments used, the assessments used define the standard that will be reached.” And again: “Assessments are not the end of the teaching and learning process; they’re the starting point.” 
They are advising new teachers to put the cart before the horse.”[iii] [emphasis mine; the notes below are from our review]
Let's put this into the Newark context:

  • One of the most prominent "reforms" in Newark has been the closing of local public district schools while moving more students into charter schools like North Star.
  • By their own admission, these schools focus heavily on raising test scores.
  • The district also claims it has focused on aligning its curriculum with the PARCC (as I point out in our review, however, there is little evidence presented to back up the claim).
  • None of these "reforms," however, are necessarily indicators of improved instruction.
How did Newark get its small gains in value-added, most of which were concentrated in the year the state changed its tests? The question answers itself: the students were taught with the goal of improving their test scores on the PARCC. But those test score gains are not necessarily indicative of better instruction. 

As Koretz notes in other sections of his book, "teaching to the test" can take various forms. One of those is curricular narrowing: focusing on tested subjects at the expense of instruction in other domains of learning that aren't tested. Did this happen in Newark?

More to come...

[i] Farr, S. (2010). Teaching as leadership; The highly effective teacher’s guide to closing the achievement gap. San Francisco: Josey-Bass. We note here that Russakoff reports that Teach for America received $1 million of the Zuckerberg donation “to train teachers for positions in Newark district and charter schools.” (Russakoff, D. (2016). The Prize; Who’s in charter of America’s schools? New York, NY: Houghton, Mifflin, Harcourt. p. 224)
[ii] A “Pythagorean Triple” is a memorized ratio that conforms to the Pythagorean theorem regarding the ratio of the sides of a right triangle. Koretz critiques the linear intercept rule, noting that b is often taught as the intercept of an equation in high school, but is usually the coefficient of an equation in college courses. In both cases, Kortez contends test prep strategies keep students from gaining a full understanding of the concepts being taught. See: Koretz, D. (2017) The testing charade; Pretending to make schools better. Chicago, IL: University of Chicago Press. pp. 104-108.
[iii] Koretz, D. (2017) The testing charade; Pretending to make schools better. Chicago, IL: University of Chicago Press. p. 114-115.