I'm not saying it's harmful, but it's most definitely a drug. I would say anything that has medicinal effects on your body or alters your mind in some way is a drug. Alcohol is a drug, in my opinion.
The word "drug" does not imply "badness" in any way, so what's wrong with calling marijuana a drug?