True, but one way you could look at it is that it's not how often you fumble, but whether or not you recover from the fumble, or even how you recover from the fumble. That's what I'd use a dex check for.
For example, if I rolled a fumble and the result ended up being that I dropped my weapon, a...