Not really. I mean, its a human game, with human elements to it. I see no need to remove that human element. As a player you put faith in your DM to run a challenging and fun game, and as a DM you repay that faith and do just that.
Hilariously, even the 'anti fudgers' seem to be totally OK with this human element being involved to ignore charts entirely, or simply selecting results or an outcome the DM wants from a chart, as long as the DM doesnt also roll the dice at the same time.
What about if I selected my random encounter, yet still rolled the dice, but for no other reason other than to hear them hit the table, and I didnt even look at them, or register the result?
Would the anti-fudging crowd still be angry at me?