There is debate on this subject because the rules use non-definitive words like "most" and "some". The actual relevant rules text is quoted below from the SRD.
SRD said:
Energy Attacks
Acid and sonic attacks deal damage to most objects just as they do to creatures; roll damage and apply it normally after a successful hit. Electricity and fire attacks deal half damage to most objects; divide the damage dealt by 2 before applying the hardness. Cold attacks deal one-quarter damage to most objects; divide the damage dealt by 4 before applying the hardness.
Ineffective Weapons
Certain weapons just can’t effectively deal damage to certain objects.
Vulnerability to Certain Attacks
Certain attacks are especially successful against some objects. In such cases, attacks deal double their normal damage and may ignore the object’s hardness.
So, acid and sonic do full damage, electricity and fire deal half, and cold deals one quarter. Since Force (or other energy types like negative or positive energy) isn't specified, it deals full damage. The first paragraph clearly states you apply the hardness. I included the other two paragraphs because you could use them for energy attacks and rule per the RAW due to the non-definitive terms used. For example, you could rule that Fire is ineffective against stone. No matter how much damage you deal, it will never destroy the stone. Maybe you want to exclude red dragon breath and other really hot fires, but that's up to you. Another example is that wood is likely vulnerable to Fire, so maybe you want to ignore the hardness when applying fire damage.
Now, Force isn't specified so it deals full damage, but you could use the middle paragraph to rule differently, if you want, depending on what you think that particular Force damage means. On the other hand, a Force spell like magic missile explicitly deals no damage at all to inanimate objects. Other spells that don't have that explicit text should deal full damage, but still will not overcome hardness.
Another wrinkle to your question is the FAQ, which is the source of the confusion. I pull the lines relating to hardness:
D&D FAQ v3.5 07/28/05 said:
Hardness applies to acid and sonic attacks.
Hardness applies to electricity and fire attacks.
Hardness also applies to cold damage.
Hardness applies to force attacks.
Later on, the FAQ says:
D&D FAQ v3.5 07/28/05 said:
Acid, sonic, and force attacks ignore hardness. Hardness applies to cold, electricity, and fire attacks.
There's no resolution to this until WotC issues a revision of the FAQ to fix it.