Lyxen
Great Old One
Between +1 everywhere (which, with rolled stats, can result in a lot of good stats) and a feat, depending on what feat you allow, I don't think it's more "powerful" to starts with one. But I usually taylor feats to tie with the settings so no -5/+10 pure mechanical feats.
In general, I agree with you, most of the feats are actually fairly reasonable, and should not cause a problem. The difficulty is that the feats are a bit of a strapped on rule (and they remain an option, therefore less play-tested), and it's only SOME of the feats that cause problems, and it's always the same ones, because they can be abused and some of those bloody builds cannot exist without them. Not only that, but feats like PAM and Shield Master are bloody annoying since they slow down the game a lot, with additional effects even at low level (and I also totally hate the visual effect of shield master, ragdolling an opponent every round).
In my soon-to-start powergaming DM's campaign, we are allowed one feat at level 1, but it's along a selected list of half-feats, and that's OK, it's not really that powerful and it gives some additional usual social benefits, but the point is that, because it's a half-feat, it still gets a +1 to a stat, and since we are rolling stats... Well, you see where it leads, it's always these combinations...