When firing at an inanimate object, I say the arrow sticks if it does any damage to the structure. If you're shooting at a wooden wall, the hardness is 5, so you need to do 6 points. You could even stick it into a stone wall, but you'd need either a mighty bow or some kind of magic. (Stone has hardness 8; a base longbow does 1d8 damage; without a +1 from somewhere, even a max roll won't hurt the wall.)
If you fail to do at least 1 point past the hardness, the arrow bounces off or breaks. On the other hand, if you do too much damage-- if you reduce the hit points to 0 in that area-- the arrow penetrates and keeps going.
[/houserule]