This may seem really basic, but why not just make rapiers unavailable in your campaign? In most cases when a race or class allows proficiency with a rapier, it also allows proficiency in some other finesse weapon (shortswords, scimitars), so it shouldn't be that much of a change.