That makes it impossible for an adult human in reasonable health to cross the Channel. Which we know is not the case (plenty of normal Joes have swam 20 miles in open water before, or been stuck alone in open water for a long time - they're exhausted when found and require a ton of bedrest, but they make it.
I'd make Swimming the English Channel a skill challenge with a DC 10 Constitution (Athletics) check, with a failure causing a level of Exhaustion, and requiring 3 successes to make the crossing.
Each success consumes 2 and a half hours time.
Meaning your average adult human with a Con of 10 would possibly die attempting such a crossing without training (the first failed check imposes disadvantage on future checks due to exhaustion, and 6 levels of exhaustion, kills you).
You likely finish the swim (if you make it) in around 15 hours, with 3-4 levels of exhaustion. Passing all 3 checks makes the crossing in 7 and a half hours, which is the current record. The longest crossing was around 17 hours (meaning that person had to make 7 checks, and finished with 4 levels of exhaustion).
Rough waters would increase the DC by 5.
You're OK until you fail that first Athletics check, at which time you're in a world of trouble as you tire in open waters.
Using improvised flotation devices (your pants, a bit of driftwood) grant advantage.