While the core rules are applicable to many settings, they go out of their way to fit around the Realms as a core setting, to the point that they suggest names for Humans from the various Realms cultures specifically. In addition, the "race" entries were written with Realmslore in mind, so very little of those descriptors are applicable if you're playing, say, in Eberron or Dark Sun or Krynn. They MIGHT be useful, and there are a few throw-away lines to other settings in the various options (especially for Cleric Domains, Warlock Patrons, and "races") but these are adjunct to the Realms-FIRST model of 5e. This stands in contrast to the Realms-SECOND approach of both 3e (Greyhawk-FIRST) and 4e (Nerath-FIRST).
Any particular flavour text, unless said otherwise, is to be assumed to be true for the core assumed setting of the Forgotten Realms. They would not include something that contradicts Realmslore if they didn't intend to do so (see: Firbolg depiction in VGM).