I agree. In my games, unless I get ambushed by the players, all downtime is done between sessions. I encourage the players to come up with descriptions as detailed as they like of what happened during that time, including new NPCs, rumors etc. But the shopping is done off-camera.