There's a few questions we need answered before we can fully weigh in. I'll throw out a couple and offer an idea that has a presumed answer.
1) How does the character tunnel?
If they tunnel with mole-hands then they need to stop what they're doing, put their weapons away to free up their claws, and start digging. If they just provide an aura of dirt-displacement and "Fly Down" that'd be a wholly different thing.
2) What are the limitations to tunneling?
In 5e, tunneling creatures can move through sand, earth, snow, mud, and ice but they cannot burrow through solid stone. On it's face, this limits your burrowing character's ability to tunnel while in dungeons, cities, or other environments where loose material is more scarce and hard-packed or stone material is much more common. Spending a round to rip up paving stones -before- you can tunnel in a city-street would be interesting.
3) What is the tunnel structure?
5e makes no presumption about how the tunneling happens, as it's a creature to creature trait. Some creatures, like Earth Elementals, just flow through the ground and leave no tunnel. Others, like a Purple Worm, leave tunnels large enough to become Adventure Areas when you have enough purple worms digging through the area to make a mish-mash of intersecting tunnels.
4) How does Difficult Terrain work for tunneling?
Does a thickly rocky medium make tunneling harder, slowing down your tunneler's movement in places without solid stone? Does particularly wet sand or soil make things more complicated? Does a thick clay just make everything a mess and slow someone using a burrow speed down? Or is it "Stone stops you and nothing else matters"? I imagine WotC never felt the need to answer this question as burrowing speeds are much rarer than fly speeds.
So... Assuming a mole-man who can't dig through solid rock and who gets slowed up by various material compositions as difficult terrain who leaves behind a tunnel? No problem at all. His enemies can always follow into the tunnel to try and keep fighting, he has to disarm himself to dig, but it gives him a unique mode of travel to overcome obstacles. Just make sure there's plenty of "We can't go over! We'll have to go... under..." moments.
The further you get from that baseline, though, the harder it's going to be.
Yeah, but if you're not choosing to leave a tunnel behind you (which only some burrowing creatures can do), are you even still visible by the time you try to leave that square?
Just apply a cover bonus. It's not significantly different than an opportunity attack as someone tries to dash around a corner or hide behind a piece of furniture. It's just "Down" instead of lateral.