No, it was not his answer. I found it. He did not call you phantom people.
He called the act of trying to please everyone and especially redesigning things, most people are content with "chasing phantoms".
Which is a generally good idea. Chances you annoy more people than you please if you go after each thing a random person does not like are big.
He even said, he wanted to do research to make sure more than a single person has issues. This is actually taking your concern seriously.
My prime example is always this:
4e was on a good way. Then the paladin was playtested on a con. A famous scene that found its way into the echo chamber that is the internet was the paladin using divine challenge and run away to grant disadvantage forever.
That sorted for a lot of noise and the designers added really annoying and complicated conditions so that such behavior is impossible. Resulting in a confusingly annoying ability.
In a very late update, the conditions were removed for the better.
So before liste ing to the loudest voices, they are very well advised doing research first to not chase phantoms (i. e. rules that work ok, but are willfully exploited by 1 out of 1000 people, usually in bad faith).