Whether we all leaned one way, the other way, or covered the spectrum - our opinions are irrelevant. The impacts of not following a Paladin's Oath is up to the DM. All we can tell you is how we see it.
In my game, the Oaths are like Oaths in the real world. Their value is determined by the one swearing it, and the ones holding you accountable for it. The ramifications for breaking it (from the one to which you swore) are situation specific.
They're also role playing gold. Great stories usually require great conflict, and the conflict between duty and something else (honor, love, sandwich, glory, etc...) can offer real personal drama. My favorite classes in D&D, when I DM, are Warlock, Paladin and Cleric because they inherently have a relationship at the core of their character that is built upon either duty, loyalty or obligation. I like to lull the PCs into a false sense of security that the relationship is not what their character is about, and then I place obstacles in their way that make them reconsider that position.
If you swear an Oath to Bane or Asmodeus, and then break it ... well, you're not going to have a good day. You'll likely know in advance what price you will pay for breaking that vow, and it will likely be horrific. It has been about 20 years, but the culmination of a three year campaign was salvaged from horrific die rolls by a clever PC tricking the big bad into breaking their Oath to Asmodeus and being swallowed down to hell just before they plunged the world into darkness.
If you swear an Oath to Pholtus, Bahamut or Moradin, and then break it ... you may find yourself on trial. There will be an effort to make the punishment fit the crime.
If you swear an Oath to Kord or Thrithereon and break it, you might find the focus being on whether you're living up to the spirit of your Oath, not the trappings of it. If you disappoint, they may give you a chance to redeem yourself, may just chastise you, or may start to consider whether you are worthy of the gifts they've given.
If you swear an Oath to Gruumsh, you might find he doesn't give a %$!# about pretty words - he cares about you doing exactly what he wants, when he wants it. Fail to predict the unwritten demands he has for you, and you may find that former allies in his service turn on you.
If you swear an Oath to Oberron and Titania, they're likely going to be the ones that tricked you into breaking your Oath. And your punishment might be anything on the spectrum from severe to humorous. Whatever suits their whim.
However, I will almost always position someone that was taken an Oath or signed a contract (Warlocks) in a position where they have to decide what the Oath means to them. The character's views are as important, if not more, that the views of the one to which they have sworn.
My clerics also all have an obligation. For some it is based upon written rules, for others it is just a duty of loyalty to the cause - but failing to follow the teachings of your God will have repercussions.