When I've discussed it in the past in a game I made with some other players years ago, the result in playtesting was players using skills for silly reasons simply so it wouldn't degrade. In a like manner, when adventures stop adventuring for an extended period of time, hit points should also degrade IMO, but they don't. Heck, 5E even removed aging modifiers so characters would lose STR, DEX, and CON in exchange for gaining INT and possibly some WIS. I liked those ideas in AD&D.
For better or worse, D&D fails to model a lot of things that happen in real life and the best we can do is adapt with what we have to work with. In your wizard/longsword example, if the wizard has been using it without gaining his proficiency bonus, but is determined to use it any way because the player wills it so, then the DM could gradually award proficiency. Perhaps with each level, the DM decides "Has the wizard used the longsword enough to warrant improvement since last level?" If so, grant a +1 "leaning bonus". Once the learning bonus matches the character level proficiency bonus, the character is proficient.
If you prefer a purely mechanical method, something like "if during this level, the wizard has made X number of attacks with the longsword, he earns a learning bonus for this level." You can decide on the X as you feel suites your table.
For simplicity's sake, the half bonus (maybe starting at level 3?) works well IMO if you absolutely had to include it. With rounding down, the non-proficiency bonus would thus be:
Level 1: +0
2: +0
3: +1
4: +1
5: +1
6: +1
7: +1
8: +1
9: +2
10: +2
11: +2
12: +2
13: +2
14: +2
15: +2
16: +2
17: +3
18: +3
19: +3
20: +3
A simple option could be +1 at level 3, +2 at level 9, +3 at level 15 (just a +1 every six level beginning at 3rd). Another option is make it Tier-based: Tier 1, no bonus, Tier 2 = +1, Tier 3 = +2, Tier 4 = +3. It works out somewhat close to the half-proficiency bonus, but is slightly slower.
I don't know, there are tons of ways to implement it. Pick your poison.