Yeah, I can see how that's certainly divisive to party dynamics - tough to have all of those be (a) in the same location and (b) not at least somewhat in conflict with each other. And they're also far too much like full-time jobs to count as mere downtime activities. How do you justify everyone taking time out from their important leadership roles to strap on armour and go clear out a dungeon?Fighters (and similar) would become manorial nobility (not strictly feudal, as feudalism is manorialism plus vassalage, but vassalage wasn't specifically part of the original domain rules AIUI), providing military protection and legal representation in return for taxes and levies.
Clerics would become the leader of a church to their deity (Druids would get a grove, but more or less the same deal), with associated duties and benefits.
Rogues would become the head of a thieves' guild in a city or township, with missions, dues, etc.
Wizards would acquire a wizard tower, from which they could take apprentices and do other forms of wizardy stuff.
It seems like, if one were to do this at all, then the domain to be managed should be (a) something that arises naturally from the campaign, and (b) something chosen by the group as a whole, not completely different things for each character. You then find some specific roles within the broader domain for each character to look after.
If I were going to do something like this, I'd try to place a lot more emphasis on logistical issues like manpower, planning and transportation, and not on affordability. 5e doesn't really do anything to solve the fact that high-level play tends to break the economy, so in general the assumption should be that you're limited by how many things you can do at once, rather than by whether or not you can afford to do them. Keeping the gold-flow out of things also avoids having to deal with players going in the opposite direction and funnelling an entire region's taxes into their bags of holding.