Here are my thoughts:
1. I don't understand how it is going to be simpler or more elegant, if you're basically going to be retooling every power or feat that gives attack and defense bonuses - which is a LOT of powers and feats. If you keep situational modifiers (which are very important to a lot of strategies) you're still going to be calculating things every round, so it's not going to be significantly easier. And the parts that you don't have to calculate every round, you could have precalculated anyway.
2. The solution of making some enemies harder to hit by giving penalties to attack them against certain defenses is functionally equivalent to the existing defense system. "-2 to attack Monster X with Fortitude attacks" is the same as "Fortitude defense is 2 higher than its other defenses." If you're going to go this route, I don't see what you're gaining by scrapping the existing attack/defense system.
3. Such a drastic change to the system is going to have lots of effects that you didn't anticipate. For example, how do alchemical items (that have fixed attack bonuses depending on the items' level) work with this? In the system as written, it would be very easy for mid- to high-level characters to stock up on low-level alchemical items and use them repeatedly, because the low attack bonus of the low-level items won't hurt them at all.
---
Here is my suggestion for solving the problems you identified without such a drastic change.
Primary Stats, Enhancement Bonuses, and "Math Problems":
Currently, the way it essentially works is that over a period of 29 levels (from 1st to 30th) players get about a 28 point increase in their attack bonus:
+15 from half level
+6 from enhancement bonuses
+4 from stat bumps
+3 from Expertise feats
You want to eliminate the need for the bottom three. So a way to do this is ban Expertise feats, don't do enhancement bonuses to attack, don't give stat bonuses to attack, and give +1 per level instead of +1/2 per level. Then you get +29 from level (only one point difference from above) from 1st to 30th. Also you have to add in a flat bonus to account for the stat bonus that players initially have at 1st level - let's say +3.
So the overall conclusion is to replace the half level, enhancement bonus, stat bonus, and Expertise feats with "3 + level". Then other feats, proficiency bonuses, and situational bonuses can still work.
For defenses it is a little more complicated. The progression for AC is as follows:
+15 from half level
+6 from enhancement bonuses
+6 from either:
-> Light armor users: +4 stat bumps, +2 masterwork
-> Heavy armor users: +6 masterwork.
=27 total over 29 levels
To replace this you can get rid of enhancement bonuses, masterwork bonuses, and stat mods and replace it with a "+ level" instead of "+1/2 level". But if you're going to get rid of the stat bonus to light armor you're going to have to give light armor users a static bonus to replace it otherwise light armor users are going to be seriously gimped vis-a-vis heavy armor users. Let's say you do the same thing as before and count the initial "stat bonus" as +3. Then we have overall:
Light armor users get 13+level+armor bonus AC
Heavy armor users get 10+level+armor bonus AC
The progression for the NADs (non-AC defenses) is as follows:
+15 from half level
+6 from neck slot items
+4 from stat bumps (if one of two primary stats bumped is for this NAD; +1 otherwise)
+4 from Paragon/Epic Defense feats
Total of 28 points over 29 levels, which assumes that you bump a stat for this NAD every time you get a chance to, and you take the Epic Defense feat. (Normally this doesn't all happen for all three NADs, which is why people report that it's hard to avoid getting hit in the NADs at higher levels)
So again we can roll all of that into one "+1 per level" bonus. But we still have to decide what to do about the initial stats. One idea is to just give everyone a fixed amount, say +3 again. Then we would have the NADs just be 13+level, plus any class and race bonuses and stuff. That would eliminate the "stat polarity" problem of people not wanting to put two good stats in the same defense category. But it would also eliminate one of the incentives to put stat points in non-primary skills in the first place. Would this work?