We play in my man-cave, at a kitchen table with seating for six - which works out well, as we have a DM and five players. When I'm DMing, I generally dump all of my dice on the top surface of a wheeled "kitchen island" I use as an additional table surface, and just grab whatever dice I need from there. When I'm a player (and don't need as much "table space" as I do when I'm DMing), I generally bring 5d20 (two blue, two white, one red), 6d6, 2d4, 4d8, and d%, which is generally all I need immediately within reach. The red d20 is reserved for my bite attacks (I run a lizardfolk PC), the blue and white d20s for my weapon attacks (with the color-matching d20s ready to jump in for potential crits). And, should the occasion rise, my extra dice are right there on my wheeled "kitchen island."
My brother made me a dice bag decades ago but it finally wore through, so nowadays I store my dice in a large, plastic pill bottle when not otherwise in use. I typed up a warning label for it, which reads as follows:
CAUTION: USE ONLY AS DIRECTED
May cause emotional extremes, especially when a d20 comes up as a natural "1" or "20" during especially important rolls. Fireball dice must be handled with care; if a burning sensation persists, see your physician at once. For external use only. If swallowed, induce vomiting - but nobody's going to want to count that die's result, so prepare to reroll.
Incidentally, the "fireball dice" referenced above are my 12d6 that have the 1-3 faces predominantly yellow and the 4-6 faces predominantly orange, so you can tell at a glance whether the damage is on the high end or low end.
Johnathan