If you like to use something, be it a weapon or a spell, against your opponents, is it not fair game to use against you? Does it cause problems if it is used in that way? Or should your side be the only ones able to do that to your opponents?
There's a lot here that depends on how you like to play. For some gamers, the level playing field is fine - you live by the counterspell, you die by the counterspell. It's part of the fun of the game.
But there are other gamers who really don't like to be stymied by anything - obstacles like this are frustrating and not fun. And I think that comes up in quite a few threads around here ranging from not liking being dominated/charmed/held, or not liking wasting turns trying to set up a powerful combination, or wasting turns not affecting their target.
I think the game has changed a lot over the editions to keep this style of player happy compared to earlier editions, hence some players claiming the game is easy mode. I don't agree with the easy mode assessment and appreciate many of the efforts made to make the game interesting without making taking a player completely out of the action due to a single bad die roll or two (now, a string of bad rolls can still be frustrating, but I'm OK with that because that's frustration at my luck, not some unfair rule). But there's only so far I'm really willing to entertain these concerns before my eyes start rolling. It's a game - sometimes you won't be on the winning side. Sometimes you'll get beat or take a beating. Some opponents are going to be particularly frustrating just as others will be easy or tailor made for you to beat up on them. If you're getting too many of the former because the DM has some idea in his or her head that you have to be CHALLENGED!!1! 24/7, 365.25 days a year, then maybe tell them to lay off a bit or you'll find another game because they don't know squat about good pacing.