I'm generally of the 'it has to be on your sheet' persuasion, but I also assume that any adventurer over 1st level is going to know he needs to pack rope, a hatchet, a tinderbox, his mess kit, and a bedroll if he's going further than the farms outside of town.
When I'm playing rather than DMing I like to have as detailed a list as possible of equipment carried, despite that most of the games I've played in the DM has worried little if at all about mundane eq. I like to have a list regardless just because it yields so many more options ready-to-hand. Not every situation calls for feats and spells. Having a sack with a pound or two of flour in your bag to make breakfast can come in awful handy with invisible creatures and, combined with a good whack with a mace, makes a nifty incendiary device. I played a rogue once in 2nd edition who billed himself as a wizard but just used a collection of odds and ends in a Heward's Handy Haversack for his 'spells'.
If you'd like to keep track of mundane gear but don't want to wait around for your players to sift through the price lists in the PHB, I recommend coming up with an 'adventurer's pack'. List all of the gear an adventurer is likely to have and note the average price out beside it. They can then just pay a lump sum to get the whole thing and only worry about replenishing whatever gets used up. I keep meaning to, but haven't gotten around to as yet, make up a modular version with 'cleric's pack', 'thief's pack', etc. portions.