It had to do with the Balance card from the Deck of Many Things. I knew the card existed from previous encounters (he was one of those DM's who loved using the Deck, no matter how many times it blew up in his face, and got irritated when players got "too powerful" as a result). So since I'd picked up the item in an adventure he ran, I decided to consult it before drawing any card, and when I actually drew Balance, he refused to allow any "take backs".
In his opinion, the Phylactery is supposed to warn the wearer about actions that would cause their alignment to shift normally, like a veritable angel on your shoulder saying "oh, but if you murderhobo those goblins, you'll turn evil!", not "I put on a cursed helmet and I'm evil now". He started saying that even if it did work the way I thought it did, it was obviously too powerful, since it could apparently predict the result of any action I took with 100% accuracy, and all divination effects in the game are balanced by having failure chances or countermeasures.
Then he shifted to how long a "moment" is, saying that if I wanted to use the item it would take up my actions so anything I did would take twice as long, since obviously I need to constantly consult the item as "the future is always changing". Then he finally said "well, all of your previous draws are negated, since there was always a possibility of drawing Balance, and the Phylactery would have told you not to draw any cards".
I mean, he was the DM, so I had to abide by his rulings. I can't imagine what he would have said if I'd decided to memorize a stack of Augury spells before drawing from the Deck! I stopped using the Phylactery since apparently it didn't do anything common sense wouldn't do.