Sleeple, people who know there is a whole world of adventure out there and yet someone decide they rather get into their own beds at night and pretend it does not exist. Also helps that in many editions of D&D these ordinary folk were highly vulnerable to sleep spells, ; ) .
Looking at it from the other side, I've been running a campaign set in the police force of a fantasy city. The police don't need a term for non-adventurers, apart from "people" or "local inhabitants." To them, "adventurer" means a person of unknown disposition and capabilities, possibly sensible, but possibly prone to using their power unwisely. They're treated with caution.
Populace seems to be the best fit.
Denizens can mean non-humans as can inhabitants.
Populace is not limited to particular class levels like "commoners" or the "masses" or to imply a relationship to a higher class like "subjects."
I suppose you could also use Community. Members of the Community, Community Member, etc.
This assumes that adventurers and other wanderers are not considers members of the community (or populace for that matter).
I was looking for an antonym for nomad and there really isn't many good options. "The Settled". But Settlers, doesn't fit. An "unadventurous" person is a "homebody" but the "homebodies" doesn't work in this situation.
How about "RESIDENTS" ?
Dweller may also work, but I like Resident better.
I think you have to look at it as a question of slang and not an objective description. Because what legal reason is there for there to be a distinction? After all the difference between military personnel and civilians is a legal question - different rules apply for both the conduct of and conduct towards these classes. That and military forces are also substantial in number, whereas "adventurer" is a very ad hoc kind of category. When exactly does a person even "become" an adventurer? Can not anyone who is foolhardy enough just decide to be an adventurer one day? Why do they stop being their previous distinction because of some fanciful whim? When does a person stop being an adventurer?