It can be aggravating for many, but laws regarding medication in Japan are extremely strict. If you want to make sure you have a trouble-free experience, you need to do your due diligence.
The last thing you want during your trip is to take a half-day going to a doctor, or problems at customs that can ruin a trip.
Bringing your own medicine to Japan
We can't give any legal advice for this, but the rule of thumb is that you shouldn't bring medication that are stimulants or opiates which are generally illegal in Japan.
You can however bring in prescribed medicine (narcotics/phychotropics) for personal if you get permission before entering Japan. Click here for more info.
Even the most basic medicines such as aspirin or ibuprofen need to be purchased from a store that has a licensed staff member. Convenience stores and supermarkets do not fall into this category, so look for a shop that has 薬局 (which means "drug store") on a sign, and be sure to go during daytime hours. Even 24 hour stores do not usually have a licensed person on staff during the night.
When you do go, it's going to be helpful to have the name of what you want written out in Japanese & English so that you can avoid any confusion. Be sure to ask the staff about dosage since the packaging/instructions will all be in Japanese. Here are some words that will come in handy.
朝 (asa = morning)
昼 (hiru = noon)
夜 (yoru = night)
食前 (shoku-zen = take before meal)
食後 （shoku-go = take after meal）
粒 (tsubu = capsule)
錠 （jo = tablet）
服用 (huku-yo = take medicine)
So if it says "昼食後２錠服用", it means take two tablets after eating lunch.
To go even further, it may typically say on the box of aspirin "成人、１日２回を限度とし、なるべく空腹時をさけて"which means that adults shouldn't take it on an empty stomach, and should only take it up to twice a day.
Filling a prescription
Japanese pharmacies won't fill a foreign prescription (and many medicines are unavailable anyway) so if you have an emergency, you need to go and see a doctor. If you're at a hotel, it's best to ask the staff for a doctor that can communicate with you, but if you're in a rural area that may be difficult so please keep that in mind. Your best bet is to find a hospital that specializes in foreign patients, though it might cost a bit more.
Once you have your prescription you'll need to go to a nearby pharmacy. Often a "drug store" is only for over-the-counter medications, so you'll need to go to an actual pharmacy (処方箋薬局), and these often have limited hours. When you get your prescription, there's typically a pharmacy very close to where you are located (different section of the building, on a different floor, across the street, etc), so be sure to ask them where you can go to get your prescription filled.