I'd add inherently before adversarial, but true. However, I don't know that saying Polymorph causes you to lose your agency is RAW. The only text in the spell that covers that territory is:
Well, yeah. The DM can make just about anything adversarial.
For me, that's enough to leave the player in control. In addition, why would it call out that the creature can't cast spells due specifically to the lack of hands or speech? If the idea is animal intelligence takes over, casting Fireball is not a relevant concern.
Personality is not intelligence. Personality is the barbarian being an irritable jerk, the cleric being a jolly, peace loving person, etc. That remains, so the barbarian as a T-rex is going to be dangerous as hell to everyone, where the cleric as a T-rex isn't going to be hurting anyone unless very hungry or attacked.
The reason it calls out lack of hands and speech for spellcasting is that there are some very smart animals out there. The giant ape is at 7 and the ape is at 6. That's easily in PC range for intelligence and the rules don't set minimum stats for spellcasting any longer. If a PC rolls a 4 for intelligence, he can choose to be a wizard and cast spells.
The player is not going to have the same level of control that he had before. Even if you allow the player to act however he wishes, the simple inability to cast any spells is reduced agency if he's a spellcaster, and the lack of hands keeps the fighter from using weapons, so his agency is reduced as well. He can no longer do all the things he used to be able to do. Additionally, the spell also says in the same sentence as the lack of hands and speech affecting spellcasting, "The creature is limited in the actions it can perform by the nature of the new form
..." That portion of the sentence imposes other restrictions to agency that go above and beyond the spellcasting limitations.
There is no adversarial DMing going on by telling the player he's going to have to roleplay the intelligence of the beast he is being turned into. That isn't DM vs. Player mentality. It's just an impartial ruling that you happen to disagree with.
Now, if you want to argue the spell being dangerous is RAI, I have no slam dunk argument against that. The best I can offer is that Feeblemind, which sets Intelligence to 1, clarifies that:
If that's what giving you the mental statistics of a Lizard or a Crab allows for, I personally wouldn't read Polymorph as more dangerous, especially given that most creatures folks are turning into have a higher stat than that.
As I mentioned earlier in the thread, WotC isn't very good at judging animal stats. It gives the common house spider a strength of 2, which would allow that little sucker to drag 30 pounds across the floor. Spiders are strong for their size, but there's no daddy long legs that dragging 1 pound anywhere, let alone 30.
In my reading, I see that the first thing that polymorph mentions in regard to this aspect is the term 'game statistics', and read the intention that the changed mental statistics are to be applied as relevant to game situations: saving throws, ability checks, etc. Not personality, not memory, not choice of action.
Actually, the only thing that is unchanged in the polymorph spell is personality. It explicitly limits actions based on the nature(not form) of the new creature, and spellcasting based on lack of hands(unnecessary distinction if nature=form) and speech.
But, like I said, I can see your argument, and your table is your table.
Absolutely. I wouldn't be upset at a table that let players run free with their choices while polymorphed or a table where it's limited like I read the spell.