Do you want the world to feel very real and grounded and possibly even historical? Or do you like the world at large to be weird and wild and unusual? Do you prefer an anchor of believability in order to accentuate the fantasy elements, or o you prefer a world steeped in magic and mysticism and weirdness?
To be honest, "How fantastical is it?" isn't a question I ever worry about, in my gaming or in my reading.
In the game, I care more about the quality of the players and GM than the world. In my game books, I care less about how fantastical the choices are and far more about the rest of the execution, given those choices.
Whatever you're making, execute it well, and I'll probably like it.