I thought on this, and yes. Though like everything, its a matter of degrees.
For example, if I create a world where there are no races beyond human, but no other rules change, is it RAW? Technically, yes. However, you've cut a large swath of content out of the PHB as well as changed the expectations of much of the game's material. I'd argue it still a major change, enough that I would hesitate to call it "Abjectiveless D&D" vs. "Homebrewed D&D".
However, a lesser example would be making Elves Lawful-Neutral and resembling Ancient Athenian Democracy in their culture. Its a shift from the flavor text for sure, and it might jar someone who expects Tolkien-elves, but your still much closer to RAW than the above example (This would even be true if you made your own subrace of elves). Of course, a radical change (like make elves small, bearded toymakers from arctic regions) or wholesale changes (replacing Neo-Vancian with Spellpoints) firmly fall into "houseruled/homebrewed" D&D. So it depends on your definition of "Fluff" I say. Its fine to change WHAT an elf is like (within reason), but banning elves seems like a shade too far, even if it is done for "fluff" reasons. Like a lot of things, its a matter of give-and-take.
Of course, lots of classic D&D settings might not jive with this definition: by my own standards, Eberron, Dark Sun, Ravenloft, or Dragonlance might not be "D&D" due to how the modify core assumptions, and I'm not going to stand here and tell anyone those settings are not D&D. However, I think the idea of "abjectiveless D&D" vs. "Flavored D&D" is a crucial distinction; I might tell my friends we're playing a new D&D game and they might expect generic core-rules D&D up to the very moment I pull out the Dark Sun box set. So "D&D", as a generic concept, constitutes its own unique flavor that other worlds and homebrews change as needed. I also think its a great disservice to try to constantly force both the core rules to be generic enough to accept any sort of flavoring without hassles AND to make the core so overpowering it changes the nature of settings and worlds trying to flavor it. A kobold should hang with dragons and worship Kurtulmak in lieu of a better, more specific idea.
I've been trying to coin (so far unsuccessfully) the term "Game as Written" to define both the mechanical "rules" unchanged and the use of the default core "fluff" that goes along with it. The stuff that if I told you "I just played D&D" you'd assume was true based on the core rules expectations. This might not be the best terminology though, and save for "Adjectiveless D&D" (vs. "Eberron D&D", "Dark Sun D&D" or "Forgotten Realms D&D") I'm at a loss to explain using just the core rules + the small amount needed to make them function (a map or adventure setting, for example).
TL;DR answer: depends on what's changing.