I agree with you. Whether a court will judge a work is derivative or not seems to me hard to predict. Adventures and bestiaries that invoke the D&D mechanics seem particularly at risk.
That's indeed the problem. There are no clear rules you can use to ensure you are safe if you rely simply on "game rules cannot be copyrighted" because the expression can still be, and there is a huge grey murky potential pitfall if you reuse enough rules from a fairly unique and complex set (something which I believe has never been tested), and can dig yourself in further by replicating tables of data that are not based on mathematical formulas.
The OGL defined clear rules allowing people without access to expensive legal teams to reuse specific Open Game Content without having to worry about accidentally straying into that grey murky area.