The answer to how to go with monsters in the world depends on how you want to handle them....
My first suggestion goes very much opposite of your initial aims, but in terms of what you want available to the PCs could still work...
For example, if you are going to have random creatures appearing out of no where with no rhyme and reason, rarely geographically near others of their own species... well then, they aren't natural creations. In fact, perhaps they are not even of the same plane of existence. Perhaps Medussa, Minotaurs, Gnolls, Lizardfolk, etc... maybe they are literally made out of nightmares. Just this low magic aura that permeates the world, particularly in the darkest corners, and the fears and doubts, tragedies and cruelties, violent and vengeful thoughts of humans (and near-humans) just cause the darkness to take shape in the hidden parts of the world and emerge to enact dark acts upon the world and spread that darkness. Their various forms and their near-human intelligence is because they are born from the twisted thoughts in the minds of people and are born as those people's fears. As for the armor and weapons... perhaps the world has recently had many great wars, one of which caused this whole darkness to permeate the world, and it is a rather trivial task to find a functional weapon and some chain or leather armor to drape around one's body.
Such a thing would allow for the fact that monsters are just monsters, and you could use them far more creatively in odd combinations without breaking the internal logic of the world. Of course, this would be very high magic in a sense-- far more so than Forgotten Realms-- but it would be high magic that the PCs and allies are not at all in control of, it is a primal force of evil of the world itself. And it would give the world a sense of dread-- no matter how many monsters you destroy, there is no real progress, no stemming the tide unless you can create hope in the world to quell the darkness that has taken hold in the hearts of humanity... but without that hope... well... there is no hope of victory. Pessimism about the situation is a self-fulfilling prophecy.
Alternatively, you can go the route that there are no "monsters" per se, but that the things usually called "monsters" are merely just people, the same as every other people inhabiting the earth. Of course, this naturally raises the question of "well, if they are just sentient people, why aren't they living in the community? Why can't players be these races-- at least initially in the campaign?" so let's work reasonable excuses into this with the basic understanding that all races in the world have some very negative traits that cause difficulties for them to all live together and these races have some particularly nasty negative traits that are almost impossible to break them of-- at least within the world in the current state it is.
Anyway, I'll offer it up for you to use as many aspects of it as you like.
This is going to be long, as these ideas have been bouncing around in my head for a while.
History of the Races
First came the giants, they were born from the Earth itself and fought over territory with the dragons, and were generally successful. They had a tyrannical hierarchical society where the biggest and had the best of everything and the poorest, least cared for became smaller and smaller and had shorter and shorter lives. The smaller ones eventually stopped being considered true giants and instead the giants began to breed them for specialized tasks.
They bred one group short and stout so that they could dig down deep into the earth to extract the metals. Able to see in the underground caverns with an insatiable lust for gold and gems, and specializing in stone-cutting, smithing and craftsmanship-- these eventually came to be what is known as Dwarfs.
Then they had their troop servants, living weapons who could take a beating like no other, survive in the harshest of climates, able to fight equally in daylight or starlight and were filled with an unquenchable rage that they could unleash upon the enemies of the Giants. With little fear they would rush to their deaths to strike down those at whom they were directed, but bred fast enough that they would simply never run out. These eventually came to be what is known today as the Ogres and the Orcs.
They also bred those who could take to the open plains and were skilled at transforming wild into productive, fertile farmland and producing excesses of food and lumber. With the stealth and stamina to chase down game and the patience to breed the lifestock, they were also the primary producers of leather. As their services were best rendered in the daylight, they lacked the night vision of the other two stocks. These came to be what is known as humans today.
The world was quite stable during these years and the world is littered with ancient forgotten dungeons and ruins from the age of the giants. But then came the Elves from across the sea. They were a massive number of refugees from a great war between the lords of their people and an evil empire known as the Formorians. And on the heels of the Elves, came the army of the Fomorians known as the Goblins. The Giants ordered their forces to drive both from their lands, but they found little success as the Elves became adapt at hiding within the forests and their magic was something the forces of the Giant just couldn't match. The Goblin forces proved to be difficult to contend with. The common peon caste of the Goblins were able to outbreed the orcs and humans enough that they seemed an unstoppable plague of locusts while the elite caste were easily a match for one in personal combat and they even had specially bred bear-like warriors who working together could match the might of the prized Ogres.
The Giants attempted to counter these forces by using what they could from the Elves (some some believe capturing elves and forced breeding with the Dwarves) to create the Gnomes. The Giants weren't particularly happy with the result and they were too few in number to make a meaningful impact.
Still, the ranks of the Goblin armies were eventually broken and the Elves were tracked down and slain until their numbered dwindled, it seemed as though these foreign groups would eventually be eliminated completely-- but for some reason, a plot by the Elves or the Goblins, or just weariness from the war itself, the Dwarfs, Humans and Orcs rebelled against their Giant overlords. The result was devastating, at that point the Giants had become so lazy, idle and dependent upon their servants that they had no defense. They were driven out of the lands and their wealth plundered.
But almost immediately after this victory was won, a civil war erupted between the Humans, Dwarfs and Orcs. The Orcs seemed certain to win with their superior military might and understanding, but a union between the Humans and Dwarfs managed to defeat them. The Orcs were driven into the frigid plains to the north where the other races could not, and had no desire to, follow. Their descendants since have become much more animalistic, superstitious, xenophobic and violent since those days.
After the Orcs were gone, tensions began to arise between the Dwarfs and the Humans, but the Dwarfs had a monopoly on stone and metals while the humans had a monopoly on grains and fruits, including those that made the liquors that many Dwarfs were and still are highly addicted to and have perfected the art of brewing. A small group of Dwarfs took on the role of go-between merchants and became known as the "Hill Dwarfs" while the majority of the Dwarfs dug deep into their mountains and hardly ever emerged again-- many of them being agoraphobic and having odd beliefs about the effect of the sky and the sun upon people exposed to it.
The Goblins began to grow as a threat again and probably would have overrun the humans, the mountain Dwarfs were disinclined to help more than the bare amount and most Elves still retained a hatred for the Humans and Dwarfs who had nearly driven them to extinction when they first came to the nation. However, some Elves knew that if the Goblins won, then they would surely be eliminated. So while most Elves stayed in their hidden fortresses to grow their magical powers and try to reconnect with their homeland across the sea through mystical means, some of them came out and assisted the humans. They grew quite close with the humans and retain perhaps the closest of all relations of any of the races to the humans. The Gnomes, who are seen as pitiable abominations by nearly all, managed to retain diplomatic relations with all the Elves, Dwarfs and Humans and also have been instrumental in keeping a lid on the threat of the Goblins. Still, even after hundreds of years the Goblins seem impossible to entirely defeat-- at best they get scattered and rally again in some new location.
Orc Culture
They have had to develop a culture revolving around the harshest survival of the fittest mentality with either the biggest, strongest, most murderous leading the tribes or one who has survived and seems favored by the mystical spirits and has the cunning, ruthlessness and brutality to put down any challenge to their rule. Females are seen as little more than chattel and kept pregnant by the alpha Orc as often as possible with little regard to their own wishes. They have almost no educational system, all knowledge is past down through oral tradition and few Orcs live long enough to both collect and pass on much knowledge. They cannot afford to waste anything and anyone who dies, whether friend or foe, is flayed for leather, eaten and has their bones used as tools, building materials or decoration. In fact, they would consider it rude not to eat the dead.
Orcs often claim they can hear the spirits of nature or of the dead and somehow these voices they hear that no one else can usually tend to drive them towards taking violent actions. In addition, they tend to be extreme prone to losing their temper or become offended and lashing out violently. From their earliest age they know that showing even the slightest sign of weakness could mean not surviving to see tomorrow.
In order to gain rank in tribes, Orcs generally must slay many people, with extra credit given to those who show particular brutality and ruthlessness. This is often done against other Orc tribes, but non-Orcs are generally seen as easier targets. As a result, Orc war parties often descend from the north and can infiltrate quite deep into the lands and even set up camp in forgotten ruins as they work on slaying people and gathering trophies, generally body parts with which part being highly dependent upon the tribe, to bring back home as demonstration of their might and thus purchase a high rank in a tribe. Because seeing an Orc ever since those days has generally always led to a lethal encounter, it is pretty well understood that encountering an Orc means either fighting to the death or fleeing and hoping they won't catch up. The few Orcs that exist outside of Orc society are generally either slaves or 3rd class citizens that are only marginally tolerated in the most lawless of places. Even then, any time any crime occurs they are likely to be blamed and lynched regardless of the evidence. "Half-Orcs" are only marginally more tolerated, and for the most part it is often unclear whether they are actually hybrids or are just Orcs alienated from their own society who were clever enough to realize they might survive slightly longer if they claim to be half-human.
Goblin Culture
The common Goblins are primarily fixated on simply eating and breeding. They generally have both the intelligence and attitude of undisciplined, uneducated, sociopathic children. They are born in litters of typically 4 to 8 and age and develop quickly. They are generally "taken care of" communally, which generally simply means that they subsist on the scraps of the scraps that the rest of the tribe subsists on. They have a tolerance for filth and stench and general lack of hygiene more than perhaps any other people. They generally have no concept of personal property, lack any attention span and will not engage in any work unless it has an immediate reward or comes under threat of death. While they generally will not kill other Goblins, and will even assist those of their own tribe (even if only for the complaining of their fellows to cease), anyone outside the tribe is liable only to be killed and robbed unless they seem incapable of being overcome or can provide some immediate benefit.
These common Goblins will live whereever they can find refuge, even if temporary, and will gather food from whatever source available, regardless of the negative attention it attracts. They are expert at sneaking about to avoid detection, and will often use this to burglarize homes even when the owners are asleep. Goblins also give off a particular pheromone that causes wild predatory mammals such as wolves, giant bats, bears, and such to become somewhat docile towards them and fight to protect them, as if the animals believe the goblins are their cubs.
Goblins, particularly female Goblins, are even known to steal babies and children from homes, and while the rumors that they eat them are highly unsubstantiated, children who get adopted into goblin tribes either tend not to survive, usually due to being more prone to disease or slain by the dangerous animals the goblins keep around, or generally turn mad and criminal.
The result of this, and the fact that it takes only about a decade for a small group of goblins to overbreed and overrun a community consuming all their food, means that despite the fact that they are sentient, communities (particularly elf and dwarf communities) treat Goblins with roughly the same content as rats and generally exterminate them without hesitation or mercy, seeking out to make sure every last one is dead lest the infestation return soon. Human attempts to rehabilitate common Goblins has generally universally ended in failure. The short lives of Goblins usually mean that they feel compelled to take the shortest, most direct route to obtaining what they desire and that is virtually always the destructive, criminal route. Even when young Goblins can be educated to try to suppress this impulse, their bodies begin breaking down at only 20 years of age and they generally die from natural causes at only age 30, by the time they are capable of mature thinking it is too late for them to live much of a benevolent life. Even when influences conspire for there to be a goblin who might act good and fair to those outside their own tribe, the fact that so much of the world has resolved to kill or enslave them means that they usually end up dead for trying to be friendly to outsiders and their death becomes an example to the rest of the tribe to never act in such a way.
Bugbars are extraordinarily large, hairy type of goblin that could be mistaken as a bear as they tend to be around the same size, have a similar build and even somewhat similar features. They generally display even fewer signs of obvious intellect than the common Goblins. They tend not to speak much, preferring to communicate in guttural grunts and growls. They generally cannot stand to be around more than a handful of their own kind unless they can beat a group of common Goblins into serving them. Even then they tend to just want to be left to their own devices in their caves or in their forest burrows, living off the land and slaying anyone who crosses into their territory and probably eating them. They display the same tendency towards idleness as the common Gobins, perhaps to an even greater extent as they will even go without clothing at all unless they expect a fight and put on some armor to protect themselves.
Because they are so dangerous, powerful and yet, somehow, difficult to find or track when they don't wish to be, little attempt have been made over the years, little attempt to communicate with them has been made and its always been a failure. An encounter with one generally means a lethal encounter to anyone who isn't a Goblin and often enough they will sneak up on one and smash in their heads before any attempt to communicate can be done. So while they may well be more to them and their culture than it appears from the outside, they remain more of a mystery than Orcs, common Goblins or Hobgoblins.
Hobgoblins, the elite caste of goblins that was bred into its own species by the Fomorians before they ever entered these lands, is oddly quite similar to humans in many aspects. They are slightly larger, they live and mature about the same rate and have comparable natural intellect and physicality with the primary difference being their culture. Hobgoblins have a highly rigid culture, all centered around service, generally military service, to their direct superior. They universally train to perform military service from as early as they can walk and put a greater emphasis on it than whatever specialized craft they might perform for the clan. Manual labor is generally considered to be beneath them and, if possible, they will gather common goblins and, at the end of a whip or cudgel, force them to do the farming, domestic animal raising, cleaning, mining, and other dirty jobs while the Hobgoblins do jobs that are generally cleaner and require more skill such as fishing, hunting, butchering, cooking, brewing, masonry, pottery, woodworking, smithing, and so forth as well as serving as the guards and military for their settlements. In clans that cannot manage to find slaves to do the dirty work, the Hobgoblins who do engage in it and all of their offspring henceforth are marked and considered no longer true Hobgoblins. Although generally considered thugs by those who have not seen much under the surface, Hobgoblins can be surprisingly cultured and produce music, poetry and even plays-- although they all generally glorify warfare, slaughter and military sacrifice and service to the community.
Hobgoblins believe that their purpose in this life is to prepare to do eternal war in the next life. Ideally they should train themselves up to be the ultimate weapon and die in battle against a worthy foe. If that isn't possible, they should at least prove themselves in this life able to support those who do engage in such high and honorable military service. Naturally the fact that the highest aim of a Hobgoblin is to defeat foe after foe in battle until they finally go down fighting taking someone stronger with them means that they are very much inclined to engage in warfare of one kind or another-- and Humans, Dwarfs, Elfs and Orcs all make fine foes to test their mettle against. And while they are generally interest in seeing that their fights against others are fair and honorable in some sense, they tend not to respect any distinction between "civilians" and "military" as they have no distinction their own and believe that every last person should fight for their side and if they choose to turn and flee instead, they deserve a coward's death. This means it is not unusual for a Hobgoblin band to go around attacking and wiping out settlements of civilians and believing they did the rest of the people of the race a favor by doing away with such a weak, idle group of them.
Elves and Dwarfs find Hobgoblins particularly vexing, the Elves because Hobgoblins were sent to this land to slay the Elven refugees that fled here and neither race has forgotten this, and Dwarfs because Dwarfs tend not to forget or forgive the military defeats and brutality enacted upon their kind, and victories against the Hobgoblins do not erase them.
Often enough Hobgoblin clans end up antagonizing their neighbors until every allies and destroys them causing the survivors to scatter and reform new tribes in some other area that is either abandoned or can easily be overrun and captured. Towns that have attempted to take in Hobgoblins after a military victory or just take in those that have fallen out of favor of their own clans find the cultural divide too great. They tend to aim to take jobs as militia, watchmen, or town guards only to prove time and time again to be too sadistically brutal in such roles that they cause more trouble than they stop or they eventually get bored of not having an enemy to fight and decide to slaughter whichever group is most skirting the laws of the community. Others find a place in bandit groups or mercenary bands, though usually get insulted and aggravated taking orders from lesser races eventually.
Even Hobgoblins that might be totally benevolent due to personal experience or trauma attract the ire due to the actions of all the rest of their kind. At worst of times they suffer the same lethal fate as Orcs and common Goblins, at best they eventually find trying to live in the community too intolerable and decide to go live alone somewhere else.
Also gives you a "brutish" intermediary foe between human-sized Orcs and, well, giant Giants.
