If you do that, expect everyone to start wielding daggers, or possibly dinner forks.
I mean, why burden yourself with a heavy object like a sword when any sharp stick you find on the ground works just as well?
Sorry, just thinking like a gamer...
Not the gamers I would play with. This sort of reductio just ignores the basic soundness of the OP's idea.
Would some people choose your popsicle stick idea? They might. Who cares? If they find that fun, fine. At my table, they'd be laughed away in a second.
But if one player wants to be a twink, either the DM will say yes or no. In either case, that helps sets the tone of the game.
There are of course many possible ways to avoid this, even if one player did want to go this way:
* not give out magic popsicle sticks. The magic weapons made available to the players by the DM will be the weapons they use for most of their careers.
* distinguish weapons from found objects, which do 1d4 for anyone.
Honestly, I'd be fine with a dagger doing full damage in the hand of a professional: against unarmoured opponents they have a comparable stopping power to any other hand weapon.
The OP's suggestion is reasonable and balanced, and would be comfortable as D&D at my table. Sure it's in other games as well, but that's not a liability -- it may even be a strength.