Here are the number: "Among sentenced prisoners under State jurisdiction in 2008, 18 percent were sentenced for drug offences and only 6 percent were incarcerated for drug possession alone."
Here is 538 on the same: "Releasing Drug Offenders Won’t End Mass Incarceration"
Nevertheless reducing state prison populations by 18% and federal prison populations by about 50% would be a big improvement. Also some crime is due to the high price of drugs that would be very cheap if they were legal.
I also think our sentences are too long for most other crimes. Restitution should be used for property crime, home imprisonment with ankle bracelets and supervision should used for those who committed violent crimes but are low risk to commit again (an argument can even be made for more corporal punishment over prison see here: http://www.overcomingbias.com/2011/06/in-favor-of-flogging.html). Long sentences are not a good replacement for good enforcement and are expensive.
Again I would completely legalize all drugs and enable the sale of medicine (except antibiotics for which I would require a prescription and/or a test to get.) with out requiring a prescription.