"Relatively recent" and "all but a very few" is not terribly specific, and maybe not terribly functional when we are speaking of fictional worlds. The dynamic there is not difficult to see - courts for all but people of politically important status are rarely a high priority in monarchies.
Louis XIV turned the Bastille into a prison. From 1659 onwards, it served as a penitentiary. And there's a long history of using sanitariums as a form of imprisonment when someone was inconvenient. And, Queen Guenevere being sent to a convent is a very polite form of imprisoning her, but since Lancelot went to rescue her, we ought to call it what is was, no?
The Romans had
the Mamertine prison in 640 BC. Galley slavery had you chained to the boat, which is certainly imprisonment.
And, for crying out loud, the "dungeons" in "Dungeons and Dragons" are named for places people were imprisoned!