NitesShade
Villager
I think what your table did in the moment is the right way to do it...go with the GMs call and move on without getting salty about it. In between sessions you can have a discussion about the long term implications of the ruling and go from there.
From my perspective as a GM and a Player....taking away the Stunning Strike ability (which might be the Monk's best ability entirely) from Large and Larger opponents is a BIG change, not a small flavor tweak. If the GM NEEDED that change in their campaign there should be some sort of equally BIG benefit (Like all saves against the stunning strike are at disadvantage). I'd probably take that trade-off.
I agree having fun is the first priority. Also, I do feel my PC is greatly diminished by the modification. Hence we are revisiting. Also, I like your suggestion.