I feel a bit surprised myself. Take the Troglodyte encounter for instance.
The writing seems to imply the party interacts with the Trogs in some other way than just obliterating them. But trogs are smelly nasty underdark vermin; why would they even bother about their squabbling when it would be easier and faster to simply wade right through them? Trogs are CR 1/4!
Even if all twenty were Champions as detailed by OotA it would still be a weak encounter, since they do pitiful damage - all those hit points would only make it a slow fight, not a dangerous one.
The later chapters are weakly designed. The high level design is seriously lacking.
At first blush, the first problem is still using too many creatures in the very low CR range and not using enough double-digit CR creatures as rank-and-file troopers.
I know bounded accuracy makes a few promises, but really, there's still a limit to what use a CR 1 humanoid can be in a level 15 fight...