I do it in the social and exploration pillar. Its not just dungeon delving and killing things or character development.
Character development can happen in the field, sure...but only to a limited extent. Some examples of things you can't (usually) do while in the field:
--- buy or build a home for yourself and-or your family
--- interact with your family and-or non-adventuring friends in any meaningful way
--- buy or build a temple, laboratory, monastery, stronghold, etc. (though admittedly you could just take over an adventuring site and claim it for your own)
--- gain and-or use political influence in town, or run for office
--- open a school or training ground for less-experienced adventurers and-or commoners
--- invent, research, and design a new spell
--- invent and construct (or commission someone else to construct) a new never-before-seen magic item
--- get married and-or have a child and-or raise a child
--- party your furry little face off and maybe get yourself arrested
Other than the second one, everything on that list carries costs; and all of them (maybe other than getting married) are purely downtime activities.
Im not playing to simulate real life.
I am, though with fantastical elements attached. I want the fantasy world to, ideally, be as real as this one, warts and all.