I should clarify that no one is required to do anything, and I keep merrily world-building on my own. So my spouse, for example, is perfectly happy to just inhabit her character and let the events of the world unfold.
I want to inhabit my character is part of it. Since there would either be no reason for me to not invent something that would help so I would always be thinking of what I could do and feel pressure to do those things. Just knowing I have the capability to do it would take me out of immersing myself in my PC. I also see the potential for certain people overdoing it and dominating the flow of the story.
I give people a fair amount of leeway about their backgrounds and what they do in their downtime even if I assert editorial control. But being able to add things in on the fly on a regular basis? While it's always interesting to hear about other options (and I'll be curious what you think after you've done this for a while) I just don't see it working for me, no matter what side of the DM's screen I'm on.