EzekielRaiden
Follower of the Way
By definition, some aspects of the world are less important than others. It's simply the nature of engaging in narration, which every DM ever has had to do.It's not a characterization that I find useful to my games. Indeed I find it actively harmful, because it encourages me to treat some aspects of the world as less important. My goal is to present the world impartially, so asking myself "is this important or just color" is working against that.
Or maybe color is one of those jargon terms that means something very specific to you. In this case I'm not sure what it is. If you do mean something specific, a concise definition would be helpful.
You simply cannot give equal detail to everything. You can't. It's physically impossible. Some things will be central to the ongoing unresolved situation(=conflict) and others peripheral to that situation(=color). Some things will start off as part of conflict, and become color over time, as their relevance fades. Some things will start as color and become involved in conflict over time, as their relevance grows. Some things will vacillate. It is the nature of description itself that details with less relevance get less attention.
This applies to everything, not just people. Some streets are more relevant than others in a city; most are just color, but a few are relevant enough to get stuff like names, even though (in principle) every street needs a name or number so people know where to go to get to places. Some businesses are more relevant than others. I'm sure you don't have a comprehensive employee list complete with names, backstories, hopes, dreams, etc., etc. for every tailor or even any tailoring shop in a given large city, even though you'll have rich merchants wearing fancy clothes which had to have come from such a tailoring shop at some point. Some nobles are more important than others; the party may only meet one or two of any given rank, but if it's a High Medieval European manorialism/vassalage system there will (pretty much guaranteed) be numerous barons and counts(/earls) at least, if not higher titles like marquess/marchioness or duke/duchess--but it's exceedingly unlikely that more than a small handful of the actual vassals of the King/Emperor/Grand Duke/etc. will ever get names, let alone titles and lands and retinues etc.
None of this means "color" is unimportant collectively. Color as a whole is exceedingly important. You can't neglect it. Let me repeat that: you cannot neglect color if you want a successful experience. But just because you can't neglect it doesn't mean it gets the same intensity or quantity (or quality!) of effort. In the world of storytelling proper, as in writing novels or screenplays or the like, you can almost always tell when someone has neglected the color and focused only on the so-called "important" bits. It will feel simpler, and if that becomes noticeable, it'll probably feel hollow. Sometimes minimal color is fine; little to nothing is made of the other vassals of King Claudius in Hamlet, for example, or indeed a variety of other topics that should be Very Relevant to the heir to the throne, and nobody's taking Shakespeare to task for failing to lift a finger to describe that stuff. Conversely, sometimes color really does need to be absolutely everywhere in order to make a world feel concrete and grounded, and (as always) Tolkien remains the standard by which we judge fantasy stories for that, where he poured out thousands of hours on things that really, genuinely, don't matter to the conflict of the story (like the different branches of the Elven languages and how they diverged from one another), but are a huge part of why the story feels weighty and meaningful.
Recognizing that color isn't part of the core of the story doesn't mean color is meaningless. It's not. It's supremely meaningful. It's just not the beating heart of the story. It's the periphery. Calling it such isn't an insult. It's simply a fact.