I mix it up.
If it's something that the players are aware of and checking for, I roll against them in plain site.
There are some checks I don't even want them to know about (or at least what it's about)
eg. Something is out of the ordinary and i want to have a spellcaster(s) make an arcane...