Our group has always house-ruled that if you hit with an arrow, it's gone. You could take the time to pry it out (with pliers), but you're just as likely to break it in the process as remove it. For misses, the DM let us recover half the arrows. They break easily if they hit stuff, and have a nasty tendency to bury themselves into the grass and such so that you can barely see the fletching sticking out. It's why I have bright yellow fletching and nocks on my real arrows - we shoot outdoors mostly, and I've lost and broken plenty of arrows.
Now we really only tracked magical arrows in that way. Normal ones we just sort of assumed we had enough of, and bought more as needed to replace them. My character had fletching skill, so she could just make them as needed with the right materials. It's like tracking food - only do it if it seems to matter for that particular part of the campaign. If you are lost in the desert, the amount of food and water you carry is absolutely essential to know. Otherwise, it's just book-keeping.