I agree that the challenge itself should just be the water - going down rapids on a boat made for it is hard. A barge? That's super tough!
I volunteer for the pacific Coast Guard Auxiliary fleet up here in BC, and I do a lot of calls involving distressed/lost/damaged boats. This is in an age of navigational aids, aluminium hulls, and marine radios. Imagine the difficulties in a fantasy setting!
Sandbars are a problem, but a minor one - on a barge, you'd just stop, pretty much. Rocks or reefs are going to be a bigger hazard - if you hit one, your barge could capsize, tilt, or (worse) take on water.
PCs going overboard in a river can swim to shore, most likely, but if the current is strong enough, this can be a real difficulty. Without floatation, even strong swimmers are in trouble. While rivers are a different beast than the ocean I work on, bear in mind that it's not hard to drown in almost any situation. Someone in water just trying to remain stationary can still be struck by debris and go under... or get caught in a bad current, get caught in a series of rocks and slowly drown, or whatever else.
This is where i make the obligatory coast guard comment to readers - if you're going to be on the water, wear a PFD! If you go overboard, it WILL save your life. My unit just had a call a few weeks ago to find someone who didn't have adequate flotation and died. So, yeah. Even if you're a strong swimmer, wear one. Anyways -
Visibility is a lot harder than you'd think. Because the water reflects off the water, it becomes hard to see someone even in calm water. In a thrashing sea, finding a person even fifty feet away is going to take work, unless they're wearing bright reflective clothing. And that light reflecting off the water plays tricks with your eyes. A few months back, we were trying to find a missing hiker wearing a red jacket. We had to do a full stop of the boat anytime anything red was seen - even stop signs. And again, this is on a boat with nightvision goggles, expensive imaging systems, and binoculars. Imagine being on a storm-tossed barge without any of that. Your eyes WILL play tricks on you.
Depending on the time of year, hypothermia can be an issue. In the ocean, you can get hypothermia even in July (the ocean's temperature here raises less than a degree in the summer from the coldest part of the winter, and we're a fairly mild climate... you can get hypothermia even in California!). Rivers are a different beast, because the water temperature fluctuates a bit more, but if you think it might be a factor, here's the info:
In cold water, you have what we call the 1-10-1 rule. That means you have one minute to get your breathing under control and collect yourself (harder than it sounds. I've done this, and in icy water, you'll naturally start hyperventilating). You then have ten minutes of strong motor control - this is where you start swimming to shore if you can, or doing whatever needs to be done. After that, you're hooped... and you have about one hour before hypothermia will really set in.
It's not pretty. You'll lose basic motor controls. You'll be confused, unable to think clearly, and in many cases, you'll get angry and violent if people try to help you. Your limbs will start to freeze as your body goes into shock and keeps as much blood in the core and head as possible. In bad cases, moving a hypothermic person can kill them - icy water in the veins of the arms and legs gets dislodged, goes right to the heart, and causes cardiac arrest.
(again, going into real life coast guard mode, if you're treating someone that may be hypothermic, just wrap them up in a blanket, don't let them move too much, and make sure they're wearing dry clothing. And get help immediately. Weird thing to be talking about in July, but there it is).
On a barge, footing is going to be an issue. The deck will heave, and it's easy to be tossed aside. Encourage PCs to be wearing no armour (even leather would kill you). It's not uncommon for medieval sailors to strap themselves to the boat (we still do it today, in some situations). Water on deck is going to be an occurence, without a doubt.
Anyways. Here's hoping it gives you a few ideas, and if you have any questions, feel free to ask. This stuff is a bigger hobby of mine than even D&D. There's a reason I volunteer. It's a passion, and I'm all for sharing information for a good cause (even if "a good cause" is a just a fun night of D&D!)