This is a good and important point.
For myself —and I'm only speaking for myself here!— I want a (pseudo-)historic "medieval" setting because that's what I enjoy most, so the fantastic elements (magic, monsters, even other races of humanoids) HAVE to be rare/dangerous/untameable/antagonistic and...