I don't have a view on the proper ruling, but I did want to say - isn't seeing the future itself a magical occurrence - even if the memory of what one saw is itself mundane.For me, nothing that affects magic would affect portent. I would rule that anything that had an effect on memory could wipe out portent. It is not a magical effect, it is simply your memory of the future you saw that is giving you the advantage of the ability.

(Dungeons & Dragons)
Rulebook featuring "high magic" options, including a host of new spells.