Perhaps, if that was all that was said.I would say that the "neither worn nor carried" language strongly implies that items which are worn or carried are not affected.
But deciding that (say) the scroll is burned along with the wizard who was holding it isn't establishing a rule at all. It's applying a rule (or, rather, a conjunction of rules: I've cited the relevant passages of the SRD upthread).
But anything that I decide applies only to my own table.
In circumstances like that it's natural that varying interpretations and table practices will flourish. That doesn't mean that everyone is house-ruling, though. That just shows us that even reasonable minds can differ over the interpretation of natural language rules and over their application to (imagined) empirical circumstances.
At least in common law countries, judges disagree all the time over rules that are (generally) drafted with far more care than the D&D ones, that have received far more skilled analysis and commentary, and in cases where the stakes are much higher and hence much greater effort is made to get it right. They are only held to some degree of uniformity by rules establishing hierarchies of authority and rules for casting votes on the bench. Why would we expect the D&D RAW to produce any greater uniformity of interpretation and application?
Obviously I don't think it's crystal clear at all. I've quoted the text on p 87 that says that objects can be damaged by attacks, including spell attacks, in much the same way as creatures can. The same text also mentions that objects always fail their DEX saves. Given that a big chunk of the damaging spells that involve DEX saves are the fiery ones (Fireball (including the delayed version), Burning Hands, Flaming Sphere, Fire Storm, Flame Strike, Incendiary Cloud, Meteor Swarm, Wall of Fire, maybe some others I'm missing*), that text would be fairly pointless if those spells couldn't damage objects.
Of the spells I've mentioned, three have no text about damaging objects (Flame Strike, Incendiary Cloud and Wall of Fire): it would be odd if that absence of text meant that they could damage worn/carried objects though Fireball, Burning Hands and Flaming Sphere cannot. Delayed Blast Fireball, Fire Storm and Meteor Swarm all have text noting that the spell damages objects in the area, as well as igniting objects that are neither worn nor carried. Few other spells have such text: does that mean that the passage on p 87 is meant to be applied only in the context of 7th and higher level spells?
Perhaps, if that was all that was said.
The general rules for fire do not trump the fact that the fireball spell tells you how it works, and it damages items which are not worn or carried.
In the matter of attended objects, the rules deliberately leave it to GM adjudication (appropriately, in my view) so that the GM can make sensible callings like not having the held scroll be unscathed by the fireball, not having an Evading Rogue character's clothes burnt off while the Rogue was unharmed*, etc.
*Unless that's the sort of game you want, of course.![]()
[MENTION=23751]Maxperson[/MENTION] - that is not how language works.

(Dungeons & Dragons)
Rulebook featuring "high magic" options, including a host of new spells.