The way I read silence means that around 6th or 7th level, you cannot make a significant fight against a level 1000 wizard if there is a cleric in the party w/o special considerations that are in place specifically to neuter the cleric.
That, I interpret as broken in the sense of not being able to play w/o contingency plans specifically in place to counter said spell.
Other way around, around 6th or 7th level, if the party attacks a bunch of cleric types, it means that the party casters can do nothing of value w/o assistance of their non-caster party members (which can be a good thing, depending on situation.) This I interpret as of great concern - anything that can lockout a player type is a serious consideration...
I don't understand how I'm reading this wrong, but, from my understanding, with the typical combat length of 5e, having access to 2-3 silence spells fundamentally changes how the game plays...
This spell can be directly countered by Counterspell (as it is cast), or Mislead can be used as means to facilitate escape. For the mildly higher level wizard Contingency is an even better form of escape. And of course Move and Dash can also be used to escape.
Every now and then, a party will have one or more of its members neutered, by various means. This not a serious consideration. If it occurs too often at a particular table, then it can be a concern.