First off, I am genuinely curious and trying to be helpful but if I am bothering you please tell me and I will stop.
Secondly you kind of remind me of where I was regarding encounters during 3.5
1. Since I don't run 8 encounters per adventuring day, how challenging should each encounter be.
Well, first off how challenging do you think it ought to be? I do not run 8 encounters per day either. The players always regard an encounter as deadlier than it is, in my experience. By challenging, I do not mean the DMG categories, I mean in plain language what is average, what is the top end of challenging.
On my part, some encounters are easy to make the party feel powerful and some are hard to make them feel they have earned their win.
2. Since I have a larger (or smaller) party, how should that change encounter design.
Depends on the answer to 1
3. Some characters have magic items, how does that affect the encounter balance?
I generally ignore magic items, I mostly pay attention to action economy and damage output.
4. If I want to have a horde of lower level baddies, how do I keep them challenging?
Personally I mostly take an appropriate level monster with the similar characteristics and reskin as a mob and set their size to large, huge or whatever. I have also use what was in effect environmental effects as low level mobs. That said I have watched Matt Mercer dice out dozens of NPCs in a fight.
5. Adverse: how do I make a solo boss challenging without a single deadly attack (such as the over-powered breath attack.)
What is the average damage per round from the party, multiply by the number of rounds you want the monster to last and give it that many hit points and adjust the action economy to make it a challenge to the party. The actions of the bad guys has to match or even over match the party in the early rounds and be somewhere near while they are up.
A dragon has 3 attacks and 3 legendary actions against a party or 4 with may be 5 actions per round at low water to high as 8 not including bonus actions (depending on composition). So a party of 8 will have at a minimum have 8 actions, probably average at 12 or so. That dragon will be near dead in a round and they will probably burn out its legendary resistances in a round also.
This is my suggestion, though to be honest I mostly eyeball it.
6. My party has a couple of characters with powerful builds that regularly (and vastly) overshadow the rest of the group. Will WotC ever acknowledge this?
They did, it was called fourth edition and we remember what happened.

More seriously, given the way WoTC is designed, where there is an element of public approval in the design process and the conservatism engendered by that, coupled with the propensity of some to min-max this will always be a problem.
7. Environmental hazards? How should they modify encounter design?
8. What if I want to alter monsters? How does that affect Challenge Level? (Greater HP, magic items, new special abilities, templates?)
I don't think these are even special case situations. Several of these can happen in an average session.
I agree that these are all issues but given the loose balance between classes, the way design is done and the general complexity of the system I am not sure that you can achieve the precision you appear to be looking for.
Out of curiosity, how is Pathfinder working out for you? (Pathfinder 2e, I presume)