Random thoughts:
Maybe the new lands are "cursed." The rumor is that whoever the ruler is will die within a year (3 months, whatever timeframe you need) of taking the reins. Regardless of whether it is true or not, the last 3 appointees have all resigned before the time was up.
As a result, the commonfolk aren't too productive, as there has been a huge turnover in rulers, and they fully expect whoever to rule next proclaim a lot of changes and then be gone before even a third of them go through fully.
The "curse" doesn't really affect the lands, but the last baron appointed ended up having to be institutionalized for being paranoid and suffering from multiple personality disorder (he had been perfectly sane beforehand). The baron before simply disappeared, never to have been heard of. The baron before that one quit without stating a reason, and has refused to talk to anyone about it.
What is the curse actually? Well, it could be several things, most of which someone who knows the world better than I could suggest. However, so I don't just suggest something and then quit halfway, one possible scenario is thus:
The baron before the one just mentioned (the fourth one back) was a just and respected ruler of his people. For political reasons, his only son was kidnapped, though it was all kept quiet. The baron agreed to a ransom and a meeting place was agreed upon in the main court (as I said, it was a political kidnapping, and someone in the main court was involved) as a "safe" location.
The father arrived with the ransom ready to pay up. He flew into a rage and attempted to kill his sons' kidnappers when the kidnappers said that his son had been killed, but the main villain was too adept. As the baron lay dying, the main villain says something to the effect of "How ironic, tomorrow's rumor will be that you attempted to break into my castle and kidnap my only son. Luckily I was able to stop you, but was forced to mortally wound you. Because of my high standing in the court, my word will be trusted and there will be no inquiry. I'll suggest that my son (ooc who is of age to rule) will gain control of your lands, Your poor son is not actually dead, but will be raised as a slave (ooc: or servant etc) in my son's household."
Upon hearing these words, as he lay dying, he spat out a curse and vowed that his lands would never know peace again until the rightful ruler (ie his son) was acknowledged as the legitimate ruler.
Whether the villain's son actually took the barony or not is up to you. Either the villain's son could be the silent one who refuses to talk about it, or the villain could have been scared off and let someone else be appointed.
Since that day, whoever rules that land is haunted by nightmares, visions during waking hours, seems to be prone to more accidents, and have accidents happen around him, etc. The nightmares and visions (as well as perhaps some good information gathering) can give him clues to what really happened.
Like I said, that's only one possible idea, there are lots I'm sure. This gives you the flexibility to let your PC keep the land (and act as a reagent till the kid grows up) or have someone else appointed (perhaps the old baron's most trusted advisor) to be reagent.
As an added incentive to make him deal in some capacity, is there a rule that says if you give up barony of one land you have to give it up for all of your lands? If that is the case, that's quite an incentive to not fail.