Apply the existing rules.  Constant disadvantage for perception is a thing right?  It would be worse if you are on unfamiliar ground.  I know not to walk “back there” but an intruder does not.
Also, what about a mixed party?  If you walk ahead, alone to get the benefit of dark vision…cost benefit a analysis…you might get boned.
It you are a party with light sources, you are not surprising “them” but they could surprise “you.”
It’s just a feature of you apply the rules and a double edged sword possibly.
But the fact is lax rules application happens.  We get tired…and at times it reduces mystery I suppose.  But can you read with darkvision?  Make out colored words from the background?  Black and white and disadvantage are not perfect senses…