The inflation was very significant not little.
But the inflation was inflation: numbers going up without a commiserate rise in value. A first level fighter with a longsword and 18 STR is going to hit a goblin 35% of the time, and do an average of 6.5 points of damage against its average 4 hp. We can even knock the fighter's STR to 16, and the numbers change to 30% and 5.5, respectively. If he has splint male and a shield, the goblin's going to hit 15% of the time, and do an average of 3.5 points of damage vs. the fighter's average of 5.5 hp.
In 4e, the fighter starts with STR bonus (4), Weapon Talent (1), and Longsword proficiency (+3), so +8 vs. the Goblin Warrior's AC 17, he hits 55% of the time. His longsword does average 8.5 points of damage vs. the Goblin's 29 hp. The fighter can never one-shot the goblin, not even if uses his Daily Brute Strike, and rolls three 8's for the damage (such a situation could one-shot the Goblin Blackblade, who has 25 HP). And who wants to use their daily on a goblin? Meanwhile, fighter is probably going to have an AC of 19 (scale plus heavy shield) vs the Goblin's +6, so the Goblin has a 35% chance to hit, and does average 6.5 points of damage vs the fighter's 25.
So, for both fighter and goblin, it's easier to hit their opponent, but they need to hit more often to put their opponent down. This is just a quick and simple example of the fighter, but it's true of the other classes as well. Casters get a bit more spells, but on the whole they're weaker than their predecessors. To-hit percentage goes up, but so does hp, and to a greater degree than damage. That's not really a change in power level (the 1e 1st level fighter's going to one-shot many low-level enemies he successfully hits; the 4e fighter isn't, unless they're minions), it's just using math better suited for the more granular nature of 4e combat. In pre-3e versions, at least, combat was more abstract -- there were a lot more feints, attacks, and near-misses in the combat round than was represented by the attack roll. 4e goes pretty much blow-by-blow, and the numbers are just made more granular to reflect that.