It was enough for "DM to the stars" Chris Perkins to disallow the ability to affect objects.
I'm not aware of the Chris Perkis episode involving a fireball.
The one I am familiar with involved adjudication of the drow's Darkfire ability; here are the relevant mechanics:
Target: One creature
Attack: [mental stat] +4 vs. Reflex
Hit: Until the end of your next turn, all attacks against the target have combat advantage, and the target cannot benefit from invisibility or concealment.
For those familiar with pre-4e editions of the game, this is 4e's implementation of the classic drow Faerie Fire ability. It is not a damaging attack, and has no keywords other than and indication that it is an encounter power. It certainly doesn't have the fire keyword, because it produces no damage (fire or otherwise) and does not involve manipulating or creating any flames.
In the podcast I saw, one of the players tried to use Darkfire to burn down a door - I think they had misunderstood the power, perhaps being misled by its name. Perkins referred to the "target" line in explaining why the power had no effect - I assume he deemed this to be a more expedient path than taking the player through the power in detail. When you read Perkins' "DM Experience" column, there is no reason to think that he wouldn't let this power be used to (say) illuminate an object down a pit so one of the drow's allies lacking darkvision could see it and reach down for it.
A 4e fireball might light a ganary on fire, if you're not in Chris Perkins' game, or my DMs game, and if what you're looking for in D&D is a consistent and believable world, 4e doesn't promise you that. You may get it, but it's not something the game is overly concerned about delivering. It's not one of 4e's goals in the way that it is a goal for previous editions.
I personally think that what you say here is somewhat contradicted by, or at least in tension with, the following passages from the 4e DMG (pp 65-66):
Like characters, objects have hit points and defense scores (except for Will defense; see Object Immunities and Vulnerabilities, below). . .
Usually, it doesn’t matter what kind of attack you make against an object: damage is damage. However, there are a few exceptions.
All objects are immune to poison damage, psychic damage, and necrotic damage.
Objects don’t have a Will defense and are immune to attacks that target Will defense.
Some unusual materials might be particularly resistant to some or all kinds of damage. In addition, you might rule that some kinds of damage are particularly effective against certain objects and grant the object vulnerability to that damage type. For example, a gauzy curtain or a pile of dry papers might have vulnerability 5 to fire because any spark is likely to destroy it.
It seems to me that the reason objects are invulnerable to Will attacks and psyhcic damage is that they lack minds, and that the reason they are invulnerable to poison and necrotic damage is that they lack functioning biological systems. (Which has an interesting implication that plants, in 4e, are not objects.) And the passage is quite explicit that the reason a pile of paper has vulnerability to fire is because of its flammability.
In all this I see nothing different from previous editions of D&D (though there is far more detail than in AD&D, OD&D or B/X), except that 4e is more precise in its use of keywords to correlate mechanics with fiction. The description of fireball as targetting creatures is simply (i) to set up the contrast with "enemies" (ie fireball can cause friendly fire) and (ii) economy - much like B/X, in which fireball is described as a burst of fire that damage creatures. But I've never heard of anyone playing B/X D&D having difficulty extrapolating fireball to the possibility of incinerating boats and libraries as well as goblins.