I agree.  In my games, unless I get ambushed by the players, all downtime is done between sessions.  I encourage the players to come up with descriptions as detailed as they like of what happened during that time, including new NPCs, rumors etc.  But the shopping is done off-camera.