As long as you're using the encounter guidelines, choose monsters that can't one-shot PCs on average rolls, and set damage to average (or slightly lower - like 3 per d6 instead of 3.5, for instance), it should be /less/ deadly.
The reason is that it becomes more predictable, and, normal encounters in 5e are tuned to be almost-trivial resource-attrition events, not real dangers (in support of faster combat and reduced healing dependence). It's the possibility of a high damage roll, failed save, or the like at an inopportune moment that makes 5e combat as deadly as it is (and that mostly at low levels).
In the case of a very difficult encounter with a monster that rolls swingy damage, setting it's damage to average could make the encounter deadlier. I think that'd be the only time to worry.