I like the idea that there might be different ways to cast a spell, and if you have some extra-nifty materials you might be able to squeeze out a slightly better spell than normal. I think it adds flavor and it also adds one more type of treasure you can give out.
In addition to material components, the circumstances in which one casts a spell, I think, could have a great effect on it as well. Examples of some possibilities:
* a "naked" wizard (i.e. no component bag, no spellbook), trying to escape captivity, could be able to eke out a minor spell. Maybe to substitute for a component the wizard suffers fatigue or exhaustion or hit point loss; and maybe he suffers temporary Wisdom damage as he wracks his brain trying to recall a spell from memory.
* a wizard who has the luxury of spending extra time and maybe extra resources could have his spells empowered or maximized.
* a wizard who has the luxury of casting a spell within his own sanctuary/lab could be able to reshape it or maybe even "hang" the spell as a suspended rune or glowing sigil, ready to activate with a touch (a move-equivalent action instead of a standard action).
I think there are feats or other abilities that probably provide equivalent abilties in the current rules. Unfortunately sometimes these more flavorful options take a back seat to power because of the nature of adventuring. Maybe the wizard class (to use an example) could be redesigned so that the Scribe Scroll freebie feat could be, instead, a choice of a few different more flavorful feats (Eschew components, Spell Mastery, etc.).