To me it is very important that the setting feels believable to me. Because if I have trouble believing it, how can I possibly expect my players to? That is the high bar I've set for myself.
My players did tell me recently that they really appreciate the world building, and the strong feeling that the setting is a cohesive whole. They said it really feels like a real place. It might be that my obsession with details is a part of that feeling. I'd like to think so.
So, I would disagree that it is entirely pointless to have realism in any fictional setting and on the notion of 'shoe horning'. When I extensively research what sort of fruit is common to a carribean setting for my pirate campaign, I don't feel there's any shoe horning going on. Sure, I could stick to just fruit I'm familiar with (which would be bananas and oranges), but I think there is added value in some degree of realism.
Similarly, I feel that I should at least get the facts straight when I run a campaign set in '50's London. What sort of telecommunication and modes of transport are available in this time period? What are the laws regarding guns? That sort of thing. I would consider it a pretty big blunder if I got any of those facts wrong. Yes, it may be a fictional story, with a setting that is mostly fantasy. But I think the trick to any good setting is a solid believable foundation, rooted in some research. Plus, as a DM I think I should be better informed than my players.