I'm also an err-on-the-side-of-priming guy. You just want to make sure it's a fine, light coat to not obscure any detail. Try to ensure that if you're using a spray can (my preference) that you're doing it from at least 10-12" away and that it's not too humid or cold when you're doing it so it doesn't clump or get "fuzzy". Make light quick passes and rotate the minis.
I remember back in the 80s Mithril Miniatures' MERP line actually came with a fine grey primer coat on them, which was nice. Not that I ever actually painted them. The couple I had were such nice sculpts that I didn't dare. Didn't want to mess them up with my inexpert fumbling. But they still looked gorgeous just in that grey primer.
For years when I first started painting large quantities and getting any good at it in the late 90s and early 2000s the maxim was that you always needed to wash minis with soap and water to clean off mold release chemicals and then let dry completely, then prime, for best results.
Later many companies seemed to get good at pre-cleaning their models.
I remember when I got the first Reaper Bones kickstarter they were supposed to be no-prime-needed and I tried it, but had inconsistent results. Some definitely had bad/patchy coverage when I painted straight onto the white Bones material. So I went back to default of always priming first.