Immoralkickass
Adventurer
Your buff would be nice for my character, but not so great for a Cavalier Fighter. I could possibly one shot a strong monster if my crit smite damage get doubled. (Also, my party has a Grave Cleric, so can it get quadrupled?) But it won't solve any of the issues i mentioned above.A lot of our experiences mirror yours.
Do you think the "charging" ability gives a nice enough bump to keep it viable at higher levels? The defensive aspects for the mount are great, but as mounted combat is lack-luster in 5E we feel this helps.
Its funny because i would suggest adding advantage on saves against being knocked off the mount (because when the mount get targeted by enemies you cant force it to attack you), but that would be redundant for Cavaliers.