Fanaelialae
Legend
Like I said in another response, my PHB does not have the line designated as 2 under Polymorph, only for True Polymorph. It's possible that it was errata'd, but I was going off of what's printed in my copy.You're missing the final clause of that sentence which modifies the entire premise.
1- The creature is limited in the actions it can perform by the nature of its new form, and
a- it can't speak,
b- cast spells, or
c- take any other action that requires hands or speech;
2- unless its new form is capable of such actions.
If you polymorph a humanoid into a Bear, it can't speak, cast spells, or take any action that requires hands or speech, because the Bear isn't capable of such actions ( using hands or speaking).
If you polymorph a humanoid into another humanoid, say dwarf into goblin, the new form (goblin) IS capable of speaking, casting spells and taking other actions that require hands or speech, so the dwarf polymorphed into goblin may cast spells IF the new form has spells in its stat block.
I already stated that in the case of True Polymorph, you would gain access to the creature's spellcasting (if any). On the other hand, in the rare case that it comes up with Polymorph, I would rule that you don't. Since according to @Zubatcarteira that only beast that can cast spells is the Deep Rothe (and then only Dancing Lights) it's fairly moot.