Slate has an interesting interview with Leon Nayfakh speaking to John Pfaff, here is the critical excerpt from Pfaff:
What appears to happen during this time—the years I look at are 1994 to 2008, just based on the data that’s available—is that the probability that a district attorney files a felony charge against an arrestee goes from about 1 in 3, to 2 in 3. So over the course of the ’90s and 2000s, district attorneys just got much more aggressive in how they filed charges. Defendants who they would not have filed felony charges against before, they now are charging with felonies. I can’t tell you why they’re doing that. No one’s really got an answer to that yet. But it does seem that the number of felony cases filed shoots up very strongly, even as the number of arrests goes down.
You will note that district attorneys are relatively politically independent at this level. And this:
But just letting people out of prison—decarcerating drug offenders—will not reduce the prison population by as much as people think. If you released every person in prison on a drug charge today, our state prison population would drop from about 1.5 million to 1.2 million. So we’d still be the world’s largest incarcerating country; we’d still have an enormous prison population.
Keep in mind that some in prison on drug charges are actually violent offenders who did a plea bargain down to a drug charge.
The interview also offers evidence against alternative explanations of the boom in the prison population, such as putting the blame on longer sentences. Here is Pfaff’s home page and his related papers.