There already is a skill system in the game, but it's just "what would a character of your ancestry, background and class be good at? Roll with advantage using the proper attribute stat as a bonus."
And the using of the background for skills opens up that "player skill" stuff that the OSR talks about. Come up with a compelling case to the GM that because you are "banished" that you know how to do something?