Okay, people seem to baking this a little too far ;-)
Here's how I think it ought to be....
First of all, it is stated clearly that inherent bonuses cap at +5, so you don't need to worry about someone stacking up on a +2, a +4, and a +1 and ending up with a +7 bonus. Since the cap is there regardless, and the progression in the cost of the books/wish spells/scrolls/whatever is linear, I see no reason not to let them stack (and this is how we do it in my group, too) up to a max of +5. Also, the bonus granted is only partially magical, since it requires a certain amount of study and such...so I figure that the bonus should be noted as an increase in the base stat (ex: I use the 'temporary score' section to note pluses from items such as headbands of intellect, and I simply add the inherent bonus onto the 'ability score' section of the character sheet, since it's not really actively being caused by any magic item or anything). So poison and such should be able to take those points away, I would say (although I'm not sure about other magic items, now that it's been brought up....regardless, we've never needed to worry about that in my group, and nobody has ever really had to endure so much ability damage as to disable them).
Anyway, to recap...
bonuses should stack, up to +5
bonuses from the books and such are less magical (in that they are permanent and not actively caused by items) than other magical stat gains, and as such should be treated a little differently (esp. since they cost FIVE THOUSAND exp per +1...xp costs are nasty!)