Blue
Ravenous Bugblatter Beast of Traal
How do you figure that? Isn't your DM having you make saves & ability checks? Do you not enjoy having a better chance to pass them? Do you not like having more HP?
Different scores have different amounts of utility. For one character Charisma might raise your saves but Str or Int does little for you.
Once you reach the cap of 20, you're not improving the ability score that gives you the most return. You've probably got a second best that gives you some returns, maybe CON or DEX or a secondary from your class. But it's still diminished usefulness compared to the primary. Then when that caps at 20, you're finding you are getting less and less out of each advance.
Yes, you are still getting SOME returns, but they are DIMINISHED vs. the best you got. That's the very definition of diminishing returns.