I added a bonus action attack (LeShay Weapons), that takes the base CR to 20.5, then the charm bumps it to 21. Does that feel better?
I had originally planned to do this, but thought it was overkill. It didn't change the numbers to much so I am happy to add it back if you feel it is needed.
Instead of that. In the attack description, I would say he gets 5 attacks, but can replace one of them with Fey Step. Its easier to remember as a DM...2 rounds in with this guy I am staring out his action block....its easy to forget the little things in other places.