Actually, the offensiveness is exactly that. It's not, as far as I understand the complains about it, that it's a device that punishes in-setting atheists -- and as such, it's as offensive as an in-game device punishing practionners of any other real-life belief -- but that isn't the crux of the opposition. As an atheist myself, I have no problem with the Wall of the Faithless because, in the game, gods are demonstrably true. They pass the test. There is no reason not to select one of them (preferentially one who will get you to a nice afterlife).
But what can be found offensive by some other atheists is that the designers chose to make atheism a losing proposition in their story ; not that it makes sense for it to be a losing proposition within their narrative. They reproach the designers to use a Thermian argument because the designers chose in real life to make atheism a non-viable choice. While I can see this position, I think it also makes ANY real-life worship a losing proposition, since those gods don't exist in the setting and their followers in the realms would be sent to the Faithless Wall for worshipping someone who doesn't send a servant to take care of their soul. So, with regard to real-life philosophical choices, the FR story was "equal opportunity". But apparently, the Wall got removed. Which is bittersweet for atheism, since while it's a victory, the gods continue to be demonstrably true in the setting, so it remains offensive to those who hold the belief that there gods can't be demonstrably true. And instead of being dissolved and turning to a brick after death, the atheists will just... look like morons in the setting. Not really better if this is the beef to have to with the Wall, I guess.