When I was a kid we played 3.5 and basically everyone had a character with darkvision so I remember we essentially hand-waved over the issue of light-sources - we had a good time regardless so I wouldn't go so far as to say it ruined the experience.
When I began running games again without darkvision, it was noticeable how it introduced at a lot of fiddly 'wait, who's holding the torch' and 'no, wait, you can't see over there' moments that at times felt more like a trivial bit of upkeep, or me coming off as a pedantic DM, and less like a generator of interesting scenarios. That might be because most of my players were total newbies to TTRPG's so they don't know the standard dungeon-crawling procedures, and cause I'm still a pretty green DM, but I think we will iron it out eventually. I'd say darkvision isn't a thing that's makes the game immediately better as soon as you remove it, it takes a bit of experience to make it work without it.