I think it’s probably better to understand people calling it a lie as an expression of how they feel about how a DM who does it is treating them, rather than as an accusation targeted at anyone in particular. Maybe DMs who fudge don’t think they’re lying to their players, but if you do fudge, I think it’s best to understand that a sizable portion of players will feel like they have been lied to, should they find out. And if you try to prevent them from finding out, in order to avoid making them feel that way… Aren’t you kind of proving them right?
For this reason, I strongly believe any DM who intends to engage in dice fudging should clear it with their players in session zero. If everyone is onboard, great, fudge away. If anyone isn’t on board… Wouldn’t you rather know that ahead of time?