TheSword
Legend
The penalty is in taking away something that otherwise in my campaigns is present.There's no penalty to the players if a small town doesn't have a scroll shop. In 5e the default assumption is that there aren't any magic shops at all. If the DM is adding them in to some or all major cities, that's a bonus for the players, not a penalty.
You could replace scroll shop with gem merchant (as the original example), wizard with spell books to transcribe from, or an armourer selling breastplates or splint mail. There the same principles would apply.