Opposed rolls may slow game down significantly if there is a lot of rolling. In such cases, I'd strongly prefer only players rolling. But if there is only a moderate number of rolls in a scene, the increase in handling time is negligible.
Opposed rolls are nice in many cases, because they prevent the defender from feeling that they are passive - and in my experience, it is often more important to how a conflict "feels" than tactical options that are available. I am trying to dodge, counter his arguments, stop him from hacking into our system; not just waiting to see if he succeeds. Of course, even with no defense rolls, the scene may be described this way, but what happens at the table (who rolls) create a strong bias in people's minds.
Because of this, I wouldn't use opposed rolls for opposition that is passive (like in the climbing example), unless the situation is framed as a part of bigger conflict and handled as such by game mechanics (eg. "Will we get to the villain's lair before he sacrifices paladin's sister in his summoning ritual? If not, will we get there before the whole ritual is finished?" - in this case, we are climbing, but it's the race against the villain that is important, not the climb itself).
Opposed rolls are good if there are choices to be made while rolling (eg. how much resources to spend on defense, how much on offense, or what kind of maneuver to use), or when the roll decides more than a success of a single, simple action (eg. the result of whole combat, or negotiation, with the details of roll results deciding how serious combatants' wounds are or how far they had to compromise).
As I prefer games where there are a few rolls, but each roll is meaningful and describes more than just success/failure, I like opposed rolls, in general.
If a conflict consists of a big number of simple actions, there is not much to be gained by introducing opposed rolls, and more to lose in speed - in this case, rolls against static difficulty are enough.