I played Talislanta for years, and it has a full ecosystem of fantastic beasts. I once had a player who wanted to name his character "Bamboo Rat" and I had to explain to him that Talislanta had no rats, and probably no bamboo.
It does give Talislanta a far more alien feel than D&D, and there is no such thing as an elf, dwarf, or halfling either (and "men" all have different colored skins.) It didn't hurt believability at all, as many here have contended. In fact, since players would encounter the primary creatures in an ecology through an adventure, they would often put together how the ecosystem worked on their own which improved believability. Furthermore, it increased the "gee whiz" factor which is always Talislanta's strength.
So yes, you can do it, making fantastic creatures fill in niches of the ecosystem. You can make a completely fantastic ecosystem. If you were watching it as a TV show, people might get alienated, but in an RPG there is no problem with it.