I am one of the people who likes where they were trying to go with identifying a spell but do not like the implementation from a game mechanics or play perspective. When it comes down to it,
counterspell is a limited resource that must be used judiciously. You burn your 9th-level spell slot when the BBEG tries to teleport away. You do not use
counterspell at all when your opponent is casting
prestidigitation.
Previously, I imagine "most" groups would have said something like "I cast
fireball at 5th-level." "I cast
counterspell. My check is enough to stop it.", or they had a house-rule about hiding spells. This rule introduces a disconnect between player/DM knowledge and character knowledge that can be dealt with in a few ways, and I do not like any of the ones I have seen so far.
1.
Hide the spell being cast so the player/DM is as in the dark as the creature: As has been noted, there can be distrust added about what someone was really going to cast and whether that changes based on the enemy deciding to
counterspell.
2.
Keep everything in the open: Now there is a disconnect in player/DM and creature knowledge. The table knows what spell is being cast, but the creature does not. Let's say the DM is generous, and the creature can identify and act with the same reaction. Now there is the situation of a creature using a reaction, failing the check, and deciding to
counterspell anyway. This may be why Xanathar's has identifying a spell be a separate reaction, but then we are back to multiple creatures needing to burn their reactions, and a solo bad guy cannot know what he is countering.
3.
Decide ahead of time what spell to cast: In this case, every spell needs to be recorded in an encounter where there can be reactions to casting a spell. This can be writing the spell down or investing in spell cards. The correct spell is placed face down and is revealed when any reactions have been resolved. It resolves the problem in option 1 when playing in person but slows play down and does not work as well in online play.
The intentions of the players and DM need to be out in the open so they can collaboratively build the story together. I think there can be solutions to identifying spells for the purpose of reactions, but the description of the rule needs to meld the player/DM knowledge with the creature's knowledge of the situation. Personally, I let everyone know what spell is being cast if they have seen the spell before. It either needs to be in a spell book, in their prepared spell list, or the casting witnessed by them. This means that spells are unknown usually once at most. This has worked at our table, and I am sure there are other solutions that have worked well, but I think this is one small spot where Xanathar's missed the mark.
That being said, I think there can be ways to incorporate deception into spellcasting if the player/DM and creature are discovering the information at the same time. I posted the work in progress Spellduelist feat for
War of the Burning Sky in the other (much shorter) thread on this subject:
Spellduelist
Prerequisite: The ability to cast at least one spell.
Thanks to extensive practice with dueling other spellcasters, you gain the following benefits:
* You can pretend to cast a spell as a bonus action. The spell must be on your spell list, and any onlooker who would use a reaction to you casting a spell - such as by attempting to counterspell, taking an attack of opportunity, or performing a readied action - must make a Wisdom (Insight) check contested by your Charisma (Deception) check to realize the deception, or their reaction is used upon a failure as if you had actually cast the spell. Opportunity attacks and readied actions are taken, and the reaction for counterspell is used but the spell slot is not as there is not a valid target. You have advantage on the Charisma (Deception) check. You cannot cast any other spell on this turn other than the spell you were pretending to cast or a cantrip with a casting time of 1 action. If you actually cast a spell this turn, anyone who wants to use a reaction must have decided to contest your deception and succeeded.
* You have advantage on Wisdom (Insight) checks to determine if another spellduelist is pretending to cast a spell and Intelligence (Arcana) checks to identify a spell being cast.
This allows a spell duelist to fake out enemies. Going up against a Mage Slayer, they could pretend to cast a defensive spell, get hit, and then actually cast the spell so they do not have to worry about losing concentration. They can get someone to waste a counterspell or use their prepared action. These are the scenarios I have considered when a Spellduelist pretends to cast a spell as a bonus action:
1. If no one indicates they are going to react to the spell, they can actually cast the spell quickly, before anyone has a chance to react.
2. If creatures react because you faked them out, they burn their reaction early. Attacks of opportunity and readied actions are used. The spell slot for
counterspell is not used because there was not a valid target, but the reaction is used. The spellduelist can then cast the faked spell or a cantrip with a casting time of 1 action.
3. Any creatures who were willing to risk losing/using their reaction at the "wrong" time but figure out the deception do not have to use their reaction but are ready for the quick casting afterward if it comes. These are the only creatures who are then able to use their reaction when the Spellduelist actually casts a spell that turn.
I think this works because everything is out in the open and the risk is distributed among the players and the DM regardless of who controls the Spellduelist. Additionally, the choices of the player/DM mirror that of the creature. A Spellduelist, like the controlling player/DM, can fake out creatures to get them to commit to something and then choose to do something else. If they fail to deceive anyone, they can still go through with the spell knowing that their enemies are ready for it or choose another course of action to avoid the chance of something like
counterspell foiling their spell.
I think any sort of official option or house rule on identifying spells and reacting to them needs to try to have a similar matching of the player/DM and creature's knowledge and discovery where possible.