I've culled my collection twice, with a third time on the hoirzon.
The first I got rid of a load of items of marginal value - orphaned sourcebooks for out of print games, loads of games I'd never played (and never expected to play), lots of low quality d20 and OGL materials. No regrets there, at least not yet.
The second time was all my Pathfinder 1e books except for adventures - when I finally got to actually play PF I very quickly decided it wasn't for me, and anyway I actually have all the books I got rid of in PDF. Most of these went to a Pathfinder GM of my acquaintance, so they're doing more good with her than with me.
The third cull, if/when it happens, will be a load of old magazines - random issues of White Dwarf, Arcane, Backstab, and others. These are just taking up space, so might as well go.
The things I expect to keep permanently are all my D&D books (of all editions), all my D&D-compatible adventures (including Pathfinder), and a few favourite games (Warhammer Fantasy Roleplay 2nd, Star Wars d6 and Saga, the core of Vampire (but not all the sourcebooks), Firefly, and a few others). That said, I have no current plans for the third cull mentioned above, never mind any beyond that.
One other thing: in the event that I lost everything (in a fire or whatever), it's perhaps worth mentioning that the collection I'd
rebuy is much smaller than that lost - probably D&D 5e core, and then other items only as I felt I was missing them. And a lot more use of PDFs instead of physical works.