I wouldn't picture the Points of light as being a small village surrounded by ravening hordes. A village of 100 people could live on as little as fifty square kilometers of farmland (unirrigated), that's a square roughly four and a half miles on a side. On open, level ground, a person can walk that far in a little over an hour. Since it's farmland, it mostly will be open and in some ways level. Unfortunately, unirrigated means that it's not a square, but instead is spread out over what good farmland is available. So it's more likely to be a squiggly line a few miles wide and maybe ten miles long, meaning that it takes a whopping two-three hours to get from one end of a PoL to another.
Irrigated farmland supports greater densities, so a village of 100 could live on as little as 8 square kilometers, or about 3 square miles. Thus the village, though necessarily located on a water supply would be much more compact and easily defended.
Note that in either case there will be a central nucleus of aristocracy (even if it's just a mayor) and specialized tradesmen supported by the farmers, while the farmers spend more time at the edge of civilization, more at risk. They'll also know the surrounding lands better because they'll be supplementing their food supply by hunting while the year's crop is growing. If they're smart, the villagers will have implemented crop rotation, too.
Now, how would they defend themselves? The nucleus of the village will be surrounded by an earthworks, which may be surmounted by a wooden palisade. The earthworks would be a manmade hill about ten feet tall and with a natural slope of about 30 degrees. It would take very little work to maintain. The palisade, which would have to be regularly repaired throughout the year and, basically, replaced every few years would be made from small tree trunks maybe 15-20 feet tall (but once placed in the ground, only 10-15 feet above the top of the berm).
The outlying farms would be at greater risk, but the farmers' homes would be very sturdily built and they'd have a means of signaling. A good signal (horn, banging on a kettle, etc) could carry for miles, and the neighbors would pick up the cry and the small local militia (which a village could support) would be able to come to their aid. The sturdily built home would hold out for the hour or so it would take the militia to get there. The crop would only be at risk when it was ripe, and the entire village would turn out to get it harvested as soon as humanly possible to protect it not just from wild animals and kobolds, but rain (which can destroy a ripe grain crop) and other nasty weather.
Human beings had to face a hostile world in our on non-magical past, though the only hostile enemy they faced was also human, and they survived well enough. The population of a medieval village would have been much denser than the farmlands of modern Kansas. Most nutrition would come from crops being treated mostly as a communal property, with meat being grown by individual families (yes, even the blacksmith in town would have a pig).
In the event of irrigated farmland it wouldn't make much sense to assume a small village. Population pressure would exert itself very rapidly and you'd end up with a small city in a few generations. It would have more serious defenses to make up for the fact that it's also a much more tempting target.
Potential enemies can never be right on a Point's doorstep; it just doesn't work from either a narrative or simulationist perspective. If a band of kobolds lived in a cave system right under the village, the villagers would never be able to sleep and one group would have killed the other long before the heroes got there... unless the kobolds arrived just before the heroes did. Something like a lich with an army of hobgoblins led by a goblin priest wearing a red cloak would have to be a constant burden rather than a threat; they've enslaved the humans and rule over them, but aren't likely to destroy them. A threat like that isn't going to be living in the next valley over, it's either in
this valley or it's an army on the march.
If your human population is scattered and diminished then your monster population must be as well, otherwise none of it works. The reason travel is dangerous isn't because you take two steps and trip over a tribe of lizardmen, it's because walking 60 miles (two days travel to get to the next village) will take you through a whole crapload of uninhabited land that has, at most, a single permanent threat and a good chance of smaller roaming threats.
Edit: my numbers on population per area came from
here. The rest comes from what I've seen of English, French, and New England farm villages that have survived into the modern era.