I don’t care for the sanity rules in the DMG, and I haven’t seen any homebrew or 3rd party sanity rules I’ve really liked. If I were to run a campaign with those kind of lovecraftian themes, and if I did decide that D&D was the system I was going to use for it, I would probably try to hack together something along the lines of Bloodborne’s Insight mechanic, where encounters with the horrific give some mix of advantages and disadvantages. But probably I would use a different rules system.
In Bloodborne you gain a point of Insight at certain predetermined points, such as when you first encounter a boss monster. There are also items you can expend for Insight. As your Insight gets high enough, certain environmental effects occur, like the music changing, or certain environmental features that were previously invisible being made visible, etc. There are also certain enemies and attacks that do more damage to you the higher your Insight. So having high Insight makes the game more difficult in certain ways but can also reveal more lore through the environmental changes. You can also spend Insight to call another player into your game to help you with a boss, and to buy certain items.I'm not familiar with that game. Would you mind sketching out how that might look roughly in D&D 5e? It sounds like an interesting hack.