Reynard, good post and I see your point.
Though sometimes it can be frustrating to try to get that story out for the players if they are not engaged to know the "why" of particular locations. Then no real story will emerge other than, "Let's go here, kill everything, take their gold, and level up." The players won't care about the old prison, it's history, or the tragic tale of two lovers sentence to die there. To the players, it's just blah blah blah, "Oh look, there's a goblin, kill it. How much did he have on him?"
Haha. Yes.
This ties a it back in (along with your post above, Reynard, about towns being for "downtime"...Which I also agree with!) with the problem back in previous editions of when the town was just for resupply and train (and the occasional bender at the tavern, of course! haha). More than once I remember players saying, "I only need XYZ xp for next level, I go off and kill a sheep (or whatever) on the way back to town." thing.
We often ran into the issue of certain characters "leveling" before others. It's easy enough to hand wave, "Ok, you spend two months training. Magey and Thiefy level up and deduct X,000 gp."
But I also found it fulfilling, for the players who didn't have enough XP, as much as myself as DM, to have things happening in the "home base town" for the characters to do. It didn't necessarily have to be a side adventure or fighting anything, but holidays/festivals, helping out at the local temple for Clericy, wrestling or arm wrestling tournaments for Fightery.
Just general stuff to maintain the immersion in the world...and yeah, potentially gain some RP experience or regular XP if a side "adventure" did occur. Keep the game moving/characters evolving in between treasure hunts.
So, I think one can give things for the PCs explore "in town." It isn't generally "plot" stuff or significant as a hook, though it certainly could be used as such (or better yet, come back up in the future as one). But, like I said, it keeps the PCs active when some of them might be busy doing other things...help them feel that they are still advancing vs. "Ok, it's 2 months later and you head to the ruins of Deadguy cuz you heard there was treasure there."
As a player, that always left me feeling..."jipped" I guess, for lack of a better word. Two months have gone by and I've just been lounging around the inn? How much weight did my character put on! lol.
I will also throw in my 2 coppers to concur with your thought to bring back the training costs and treasure/gp=xp. Personally, we never did away with it...but I haven't played (for any length) later editions where this wasn't the case, either. So I don't have a lot of experience on which to base a comparison.
Anyway...yeah, good thread!
--Steel Dragons