For me it depends on whether the PCs are traveling the dungeon and looking for secret doors or if they stop in an area to explore more thoroughly.
When the characters enter an adventure location, I ask the players to establish their marching order, any light sources that are in effect, and what tasks they engage in while traveling. That might be keeping watch for hidden dangers (traps, monsters), mapping, foraging, tracking, navigating, or searching for secret doors. For anyone who has chosen to do anything other than keep watch for hidden dangers, they will not be able to notice traps or lurking monsters, running afoul of the traps or being automatically surprised by monsters. The trade-off is they get the benefit of the other task they are doing. If the PCs pass by a secret door, I check to see if the PC's passive Wisdom (Perception) is sufficient to notice the secret door. If it is, they notice it. If it is not, they don't.
When the players decide to explore a given area more thoroughly, I ask them to choose an area about of about 1000 square feet that they can explore over the course of 10 minutes and establish their tasks. Each task takes about 10 minutes, including searching for secret doors. If the approach to the goal of searching for secret doors in that area has an uncertain outcome and a meaningful consequence for failure, then I ask for a Wisdom (Perception) check. At the end of the 10 minutes, I may roll for wandering monsters or just count the time spent against some countdown or both.
In either case, once a secret door is located, they will usually need to spend 10 minutes figuring out how it works. This may call for an Intelligence (Investigation) check. If they succeed, they figure out how it works and can open it. As above, I then roll for wandering monsters or mark off the time spent against a countdown or both. Thus, any given interaction with a secret door takes something like 10 to 20 minutes (or more) of in-game time.
I always do two additional things: I establish ahead of time that there may be secret doors in the adventure location. This will be done through legends or rumors and then later through telegraphing clues when describing the environment. As well, I always make secret doors very valuable: They're shortcuts around dangerous traps or monsters, contain treasure, or offer a safe place to rest. This gives the players a good reason to look for them at the risk of running afoul of traps and/or being automatically surprised by monsters.