For some definitions of "easier", maybe.
Your account of the players needing a "Plan B" reinforces, to me at least, that essentially here we're talking about a puzzle. Attempted solution A failed, so we try solution B instead.
I call it trial and error; and see it in many ways as part of the game: see also my (probably far too) many posts in here re a) magic item identification and b) metagame knowledge vs. player knowledge, where I also support trial-and-error play rather than just having information handed to you (items) and-or pre-loaded (metagame).
Well, the sort of surprise you describe catches the players unawares and undermines or in some other undesired way reframes the experiences of the PCs. In the sort of approach to GMing, and to RPGing, that I am describing, that is a type of failure. The PC's intentions in action have not been realised. That is an appropriate consequence of failure. It's not just something for the GM to impose on the situation.
And here I disagree, which by now is probably rather obvious: I see it as well within the DM's purview to drop all sorts of surprises on the characters...some good, some bad. Where it becomes railroading is when such is done out of spite (I've seen this) or to force the party to stay on mission rather than do something else.
No! No! No!
I - the GM - did not know either. The likelihood of the brother having been evil all along was not established as an element of the fiction until I narrated the failure of the attempt to find the mace in the ruin.
OK - I didn't realize this was an element introduced on the fly.
And that makes it an
even bigger problem that it already was! Why?
Because now every interaction that has ever happened with the brother in previous play has just been invalidated. He was evil all along, it seems, but any previous interactions with him didn't have that sitting there as part of his backstory and-or personality (known only to you-as-GM as he is, I believe, an NPC) and thus couldn't be a part of driving what he did or said. In other words, the internal consistency of that character just took one hell of a beating.
This is what [MENTION=6696971]Manbearcat[/MENTION], [MENTION=16586]Campbell[/MENTION], [MENTION=99817]chaochou[/MENTION] and I are trying to get at in talking about a game in which everyone, even the GM, plays to find out what happened.
And this is why I have been asking [MENTION=37579]Jester David[/MENTION] what exactly he takes the connection to be between prior planning and depth of the shared fiction. And why I reacted so strongly to your suggestion that a player-driven game, in which the GM does not adjudicate by reference to secret backstory, cannot involve surprises.
Because when a GM is at the helm and knows the surprises ahead of time she can filter what happens in the game (outside what the players do with their own characters) through those unknown elements and thus keep things somewhat consistent.
For instance, vis-a-vis the brother: for all we know, his conduct towards his brother (which up to that point had occurred only in backstory, and mostly involved teaching him magic) was motivated by evil. Maybe he was preparing his brother to be a worthy sacrifice to the balrog!
And again, if this had been known all along there could have been some consistency; and possibilities for roleplay that are now lost: maybe the evil brother let something slip at some point, for example. Can't happen now - it's too late.
If it turns out, in play, that the brother was evil all along, well that throws some new light on all his past dealings. But clearly those past dealings don't preclude him having been evil all along (because, if they did, then finding the cursed black arrows would not be a tenable failure, because it would contradict backstory that has already been established at the table).
You're saying the arrows represent a failure; I'd say they represent a success of a sort: it's confirmed that the brother was bent all along. Now whether or not that's something the party wants to know is irrelevant - they know it now.
That said, it's hard to contradict backstory that doesn't exist even when it probably should have.
But the claim that this sort of thing can't be done unless the GM works it all out in advance is simply not true.
It can be done provided nobody cares too much about the validity of or consistency with what has gone before.
Your example of the taxed townsfolk is another one where, until we know how that was established at the table, and what its significance is to the participants, we can't say anything about it from the point of view of GM authority, player influence over the shared fiction, railroading, etc.
For instance, if the PCs return to town and the GM frames them into a scene of sullen townsfolk, with the idea in mind that the sullen-ness is due to raised taxes, is that railroading? If the adventure the PCs have just returned from is one in which they won a social conflict with the baron over the level of taxation, then probably yes - it's just fiating a failure over the top of the players' success.
If the state of the village and the wellbeing of the villagers is something the players (via their PCs) have a clear commitment to, then it doesn't look like railroading but rather framing: it's setting up the situation with which, presumably, the PCs are going to engage. (By pushing the baron to lower taxes.)
If the village is just somewhere the PCs are passing through, then it seems to be simply colour, and nothing of any significance is going to turn on it.
What I had in mind was somewhere between colour and framing, I suppose, as in my example I was thinking (but didn't type where I should have done) the PCs hadn't had much if anything to do with the Baron up till now but they had been to the town numerous times before: I'd envisioned it as their home base. It's changed while the party was out in the field but said change has nothing to do with the party's previous actions in any way.
But, it's still the DM doing something off-screen that has effects on-screen - so is it framing, railroad, or neither? (I say "neither", it's just the world moving on...)
Re: character secrets:
That's a different thing altogether. Speaking purely for myself, I find that Robin Laws gives good advice in this respect in Over the Edge: the game generally becomes more fun when the players are in on the secret even if their PCs are not.
I've never read Robin Laws but the more I hear about him (her?) the less I'm interested; this statement merely adds to that feeling as I probably couldn't disagree with it more.
Player knowledge and character knowledge should be the same, particularly with regards to the other characters! (it's the metagame business again: Laws is clearly a fan of metagame knowledge, where I am not).
Lanefan