Class would automatically grant "expertise". So a Fighter is an "expert" with swords. A Wizard is an "expert" with spells. Etcetera.So without proficiency you are just rolling the flat d20 without any modifier at all? Hmm...
I'd have to work some numbers, but that might be approaching the realm of being too limited or too small.
Also, considering how few PCs ever end up with expertise in any fashion, it means proficiency bonus will hardly ever get used. Not saying that is a bad thing necessarily, but it certainly changes the mind set!