Why not just eliminate the spell if you're going to nerf it? If you can't see through it, you might was well cast Wall of Stone. At least you can make a wall of stone permanent if you concentrate long enough.
I look at it like an unbreakable pane of glass. Nothing can pass through it except light ... it's effectively invisible. Whether radiant energy is just a way to say that the light is so intense that it damages is up to the DM.
Next thing, you'll be ruling that invisible creatures are blind because the light doesn't strike their corneas.
I don't think I'm nerfing it. A wall of stone can be smashed apart. It has hit points. A WoF does not. You can't destroy it. You either break the spell-caster's concentration or you wait 10 minutes. That is really, really powerful, especially in battle.
You are racing out of a dungeon with big baddies at your heals. You cast WoF to block the passage behind you. You've got 10 minutes to run and hide.
You can put a dome on or sphere around an enemy. Without misty step, blink, or teleportation, they are stuck there.
You ware down to just a few hit point. You won't be able to take another hit. Encase yourself in a WoF. Take 10 minutes to drink potions and cast buffs on yourself. Or do the same for a comrade.
All that said, I will allow non-magical light through the WoF, because it seems that is what is intended. I just blocks magic, at least any magic that requires you to target a creature, object, or space on the other side of a WoF or requires any magic force or energy to pass through it. That appears to be the intent of the RAW.
But that raises some interesting questions. So I can't send a magical light ray through it. But if natural light can go through it, can I use mirrors to focus a strong beam of light through it? Perhaps hot enough to cause burning damage to someone behind it?
What about sound? Can you hear through a WoF or will it block out sound if you have it in dome or sphere around you? If so could a beast or instrument with be loud enough to harm or deafen someone within a WoF dome/sphere?