I think a more realistic comparison for men and women in D&D terms would be to make them climb a wall, jump a few hurdles, fight with a three pound training weapon, and haul sandbags. My prediction is that, for a group of physically active participants, men will meaningfully outperform women only in the sandbag event.
Rowing is a bad example for a number of reasons. First, it involves a very specific motion, not dynamic and varied actions. Str in D&D is very general. Second, yes, it involves only athletes. I don't know a lot about rowing, but I'm going to go out on a limb and guess men are more commonly interested in rowing, giving them a huge advantage in the talent pool department. If there are three times as many male rowers as female, the top fifty percent of men are going to blow away the ten ten percent of women. Third, it's hard to translate rowing performance into D&D units. I don't know how much damage rowing causes or how many pounds it weighs. Telling me the energy output by itself is pretty meaningless, because Str in D&D does not measure energy output except in a very aggregate and general way. It's useful information, but certainly not the last word. Fourth, rowing is very demanding and places unusual demands on its participants; competitive rowers are going to have very different characteristics from average people, and that difference itself is going to be different than the differences between practitioners of one sport and another.
Finally, glancing at actual Olympic records, I see that the best times arevery close. I don't know what a double scull is, but I do know that 6.11.49 and 6.49.00 are close enough I question whether there's a difference AT ALL at the leve of detail D&D uses. A difference of just 8% or so, assuming rowing uses a similar progress to encumbrance, translates into a single point of Str.
So even if we use rowing as our test of strength, a cursory look suggests what has already been stated: while significant, the differences between men and women aren't MEANINGFUL. The exceptions are upper body lifting and throwing (advantage men) and gymnastics (advantage women). Even though women completely dominate gymnastics in terms of what they can physically do, I would never give women +2 Dex on the basis of that, simply because Dex represents more than just your Tumble modifier. While some people are resistant to the notion, Str in D&D is far more than horsepower, far more even though work output for a group of muscles or the body as a whole.
In specific areas, very specific tasks, you might be able to demonstrate a 1-2 point ability score difference between men and women. If you aggregate a number of performances, the difference essentially disappears.