When your games involve wars and kingdoms forces how do you build them?
The answer to most of these questions will be "you don't need to prep something until it comes up in game."
The PCs will likely never directly fight an army, so their stats don't matter. You want to have one army fight another, decide who wins. If that's not satisfying, come up with some way to measure the relative strengths and weaknesses of each army and make a roll to see who wins.
However I then worry when is too much too much?
It's too much when you spend time worrying about things that either won't come up in the game or don't matter, like the precise stats for the armies.
As I am typing this I wonder how much can be hand waved. Like I can call an army of paladins an army of paladins the players don't need to know if they have smite or lay on hands unless they are part of an encounter, and at least at low level those encounters need to be small or it will TPK.
Don't worry about balance. If the PCs are stupid enough to try to take on an entire army, they deserve the TPK. They've earned it. Don't wreck the verisimilitude of the world to curate the experience for the PCs. The strength of the NPCs should in no way be tied to the strength of the PCs. Fighting an ancient dragon at 1st level should be suicide. The players should know that.
Then I am wondering, do I just story point who wins if I have 2 NPC armies fight, or should I 'roll it out' in between games to be able to tell the players it was 'fair'
Players don't tend to care about things unless it directly affects them.
I also have seen many times here and elsewhere online that enough commoners can take down a dragon. How much do I need to worry about army sizes?
Only as much as you think it matters. You don't need to have a perfectly complete world before you start. Even Matt Mercer is still filling out Exandria as they're nearing a decade of livestreams. Write down a number that sounds right.
If I want to throw numbers around as intelligence about 1 feudal lord or another what are sane numbers? Like if someone owns 3 towns and 2 castle/keeps and a bunch of farm land and the DM told me they can field 200,000 troops I would be highly skeptical, but if they have 20,000 is that too much? What about 10,000?
Look at history. A village has up to 1000 people, according to the DMG. A town has up to 6000 people, according to the DMG. A city has up to 25,000 people, according to the DMG. How much of the population does the local lord mobilize? Those numbers include women and children.
You need between 1/2 and 1 full acre of farmland to feed one person for one year, depending on the time and technology. In D&D-like faux-medieval times, it's likely 1 full acre...assuming no magic farming. So that village of 1000 people would need 1000 acres of farmland just to feed the people there. The town 6000 acres and the city 25,000 acres. You get things like taxes when you have more food that absolutely necessary to feed the people working the land. The lord takes taxes in the form of excess foodstuffs or the equivalent value in coin.
Historically, kingdoms campaigned in the off season so they could mobilize the farmers to fight. Farmers tended to be not great at fighting, hence professional soldiers becoming a thing, and the farmers would leave the fight to go home and tend their crops. Look at the cost of food in the PHB. Farmers and nobles selling their grain for coin would earn that much (but ignoring the costs to grow and harvest that grain), you'd need to sell a lot of grain to buy one sword. Now what about armor and a shield? What about a spear and food to last the campaign? Etc.
So "the kingdom has 3 towns and a castle, so they can field 20,000 fully armed and armored, well-trained fighting men in good health" is way more fantastical than having dragons and spellcasters.
How does caster change this? Like If 1 lord can field 5,000 spear men and 2,000 archers with 500 light cavalry and a dozen siege weapons, but there enemy can bring 400 spear men, 100 archers, 50 light cavalry and 60 level 5+ wizards and 100 level 3-4 wizards and that side has 30 clerics/healers it seems like the casters are a major force multiplier.
Yep. If casters were a thing they'd literally take over the world in short order. That noble with the castle is great and all, but a 1st-level caster with charm person would be the king in a few days' time. To say nothing of things like attack cantrips. A 1st-level caster with fire bolt and light would be worth several dozen archers. Anything approaching coherent worldbuilding quickly falls apart once you include things that are possible because of the fantasy elements of D&D.
Also monsters. Like if one side has ogres and the other side has goblins that skews things too.
Another good example. It's really easy to become bogged down in the infinite details and never get anything done. Skip it. It honestly doesn't matter. As long as you're in the ballpark of probably, you're close enough.