Terrible indeed!

No, I'm joking, and excuse me if I'm being pedantic:
"Say yes or roll" is assuming a game or situation without GM Veto, Necessary Prerequisites for pass/fail and the like.
It is meant also for conflict resolution in mind, but can be easily ported to task res.
Anyway, in a d&d situation it would be like: We don't wake up the dragon and steal the treasure!
Gm: Dude... Roll for initiative and prepare for combat.
Or: We use scouts and animals to open a way thru the jungle and arrive at the temple's gate.
Gm: Fine. / Not so fast: roll for every task you do, the forest is full of dangers.
In your example it'd be something like:
We don't solve the riddle and instead use a magic ritual/thief skill to overcome it.
Gm: roll your dice and let's see...
Generally speaking SYORTD was intended for games in which the "information" is not only easily obtained by PCs, but rather given in advance by the Gm to favor choices, course of action, conflicting inter-party decisions to be made, by the table.
You can also use it like :
I use streetwise to track down the sect when they go to a tea house.
Gm: fine. They go around openly, you spot them easily.
Or: Gm: they have spies around downtown that might spot you first: roll... (then anything might happen)