Hiya.
Well, as far as feats go I think the name of the Feat should be the deciding factor. When I'm reading Crossbow Expert, I'm reading what it can do from the perspective that it was put in the book so a PC could be, you know, "an expert with
crossbows". When I read Polearm Master I get the distinct impression it's got something to do with being good with
Polearms.
Reading "Crossbow Expert" and thinking "Hey, that's great! Now I can cast cantrip spells in melee combat and not at disadvantage!"

To me that's akin to someone reading the manual to their new blender/juicer and them thinking "Awesome! Now I can type 120 words per minute!". A blender/juicer chops things up really fast...but it doesn't have anything to do with how fast you can type. Shooting a crossbow and bending the fabric of reality with a few words and gestures isn't exactly the same thing in my book.
Any player at our table would be laughed at if he/she came to the game thinking "Crossbow Master" made them better at casting Cantrips. Then, after everyone settled down and realized the player was actually serious, there would be uncomfortable silence and blank stares for a few seconds...before someone would make some other completely illogical jump in order to crack a joke about it (followed by more laughter, of course).
My suggestion: If all else fails, look at the name/title to help. And in this case, "Crossbow" does not equate to "Casting Cantrips" in any way, shape or form.
^_^
Paul L. Ming