In my above scenario, PCs were tasked to deal with the culprit who was removing arcane materials from a permanent teleportation circle in the making. It was discovered that this led back to the ruins of a nearby temple and more specifically to the catacombs. There they found a portal to the Negative Plane that had been rudimentary and temporarily shut (imagine poor stitching) using the materials stolen from the teleportation circle.
They managed to close the portal permanently, but decided to press on in the catacombs until they came across a ghostly knight who warned them not to go any further and that he was protecting further life from falling prey to great evil, just like he had once fallen. They realised that that whoever had been stealing the arcane material was undead and had only been doing so to close a portal to the Negative Plane (essentially a good thing). Nevertheless, according to the ghost they were a great evil.
They failed their social skill challenge, and the ghost did not relent in his conviction thus forcing a combat encounter. They defeated the ghost and will next in the session prepare to defeat this great evil which supposedly lurks beyond the sealed doors the ghost had been guarding.
What I did, was offered an XP (reward) in exchange for a disadvantage on a PC's action (to place goal in jeopardy), with defined stakes, if I could tie that action back to the character's TBIF (when it relates to a flaw or challenges their belief). The PC has the right to ignore the offer and thus not suffer any penalty, but then neither do they gain the XP.
Is that the only way to gain XP? (In Torchbearer, there are no XP in the most literal sense, but there is level advancement based on
spending Fate and Persona, and these are
acquired by reference to Belief and Goal, plus to a lesser extent Instinct and Creed, in the way I've described upthread.)
I ask, because that might influence how you handle the "disadvantage" XP: are they the mainstay, or more of a bonus?
In my earlier post I also mentioned simple obstacles. I want to say a little bit more about those. In Torchbearer and Burning Wheel, every obstacle will generate a check. (There's an exception that relates to pacing and thematic focus, but I'll ignore it for the moment - let's suppose all the obstacles are thematically on-point, like they seem to be in your example.)
Even if it's very easy, it still triggers a check - eg moving the materials from A to B to stop them being used by the villain might be an Ob 1 Labourer test. And the dice pool system (roll your pool of d6s, with every 4+ being a success) means that failure is always possible (if unlikely) even on the easiest task.
When a failure happens, the GM has to bring in a consequence - a change to the fictional situation, that speaks in some fashion to someone's Goals and/or Beliefs. This means that, even in rather instrumental action declarations, the thematic/character-development "weight" is still sitting there on the scales. This is a big part of how the quest to rid the temple and catacombs of undead
also promises and produces thematic development. (Another part of the engine is that, parallel to level gain in Torchbearer, there is a system for skill improvement that requires both failure and successes; and Burning Wheel has something similar though not identical; which means that players don't always hate the risk of failing on their rolls.)
A simple example: a PC had been cursed by a gem, to obsessively crave and protect it ("my precious") - mechanically, this meant writing a Belief about keeping the gem safe from theft etc. The PC was out shopping with Golin (the Dwarf PC) and Golin's new friend Gerda, whom he had recruited from some bandits the PCs had encountered not long before.
The Resources test to purchase whatever it was they were after failed, and so the vendor wouldn't sell, and then (the GM-imposed consequence) when the gem-cursed PC got back to her rooms she found her gem had been stolen! At first she assumed it had been stolen by her enemy Megloss, but once that was cleared up (via social interaction with Megloss) then she realised it was Gerda. (Which also made sense from a meta-perspective, as she was the only named character other than the two PCs present in the shopping scene.)
So even the simple, most instrumental obstacle - can I persuade this person to sell me the <whatever it was> - produced this character-relevant situation, of Golin's friend Gerda having stolen, and herself become obsessed by, the other PC's cursed Elfstone..
I don't think D&D's approach to resolving actions, and especially to determining the consequences of failure, always makes it so easy to keep the focus on these things that the players are bringing to the front and centre of the fiction, because they tend to encourage consequences to be narrated in terms of the immediate physical environment rather than the "thematic" environment that is surrounding the characters.
4e skill challenges are an exception here (or can be). I saw in your post that you're using social skill challenges: if you're wanting to increase the character discovery/development aspect in play; and if you're currently not using failures in skill challenges, and the reframings that they produce, to do that; then I'd suggest maybe giving it a try.