It was another way around for me.
I fudged quite often when I was less experienced, knew less games and couldn't really decide with my friends what style we want to play in advance.
As years passed, I improved my GMing skills, I learned communicating with my players and I gathered many different RPG systems. Now I don't fudge. If I know I wouldn't like a possible result of a roll, I don't make such roll. If a game rules would push me towards making rolls I don't like, I house rule (before the game starts) or just switch to a different game. And my players enjoy games I run with this approach.
For example, I used to fudge to save PCs from rolls that would kill them. Now I just discuss with my players if they want a lethal game or not. If they want lethal play, we select a game where character creation may be done in 10-15 minutes and we all accept that one or two characters die in a session. If they don't like their characters dying and want to focus on long-term development, we use a game that has more interesting stakes in conflicts than character death.
I think every DM has to find what they are comfortable with doing. I don't think there is anything wrong with fudging nor do I think there is anything wrong with not fudging.
The most important thing is what you mentioned being on the same page with your players.
My players want a game where it is hard to die but the chance exists. I have added things to my 3.5 game like action points and we play that no name mooks don't crit only named bad guys do.
The reason I say I will fudge is because I have had two times in this campaign seen where it was what I considered necessary to keep the players fun up.
One was the time the player lost a character in the session before and then just had terrible luck with rolls and even with the action points and other things I still killed him outright with damage. I have myself I have been in a situation like that as a player and I hated it and I could tell he was not enjoying so it so I fudged the damage and knocked to minuses.
The other time was with my good friend who hates having her character die it is her least favorite part of the game. She accepts it as part of the game and is not a baby over it. Anyway she found out a few days before the session that she was going to go blind. This was heart breaking for her she is an artist and does a lot of crafting one of her major hobbies is painting miniatures. I should say was. She no longer can do any of that.
I asked her if she wanted to play and she wanted to, to keep her mind off what was going to happen. That session she failed a reflex safe and the damage would have been enough to kill her character. I chose instead to knock her to minuses but not kill her character it just didn't feel right to do so that night.
To me people are more important than the rules and sometimes I think that it is okay to bend them. I have never found the very occasional fudging to do any harm to the game.
I am not advocating to fudge all the time and I do believe it is better to have things in your game so you don't have to fudge. But to me taking a hard stance of saying you will never do it just seems limiting to me. It is a tool you may never need it but to say I will never fudge just seems rigid.