Putting numeric bonuses or penalties on gender adds a level of sexism that was never present in tales of heroic individuals, the basis of DND adventurers. Hercules was never measured against the strength of men or women, only against the strength of other people/creatures born from the gods. Joan of Arc's valor and wisdom exceeded all her peers, man or woman.
If you want to add sexism, however, do so moderately. If you believe that the average person can be represented by 10s and 11s across the board in all ability scores, then the biological difference between men and women seems pretty insignificant. In fact, I'd say if there should be any difference, it would have to be within 1 point, such as men getting 11 strength, women getting 10 strength. Applying that to DND adventurers, the scale would only look something like +1 Str for masculine men, +1 Intelligence for feminine men, +1 Charisma for feminine women, and +1 Dex for masculine women. No counterbalance penalties should be applied that makes anyone a full ability modifier better than their opposite sex/gender. If you insist on having counterbalance, -1 Wisdom for masculine men and masculine women, -1 Constitution for feminine men and feminine women. That's the only way I'd rule it. This represents actual sexism where slight differences are exaggerated into unfair suppositions. The average man and the average woman are mentally and athletically closer to each other than to every other creature on earth.