Right after 4e came out, we ran a single session of KotS, using only the rules found there-in....
...He was extremely frustrated with the way the 4e wizard played and felt. It felt extremely de-powered to him, and not having any rules for rituals (KotS rules, only) really killed the utility aspect of wizards.
Okay, there's your first problem. The rules presented in Keep on the Shadowfell, while giving a reasonable approximation of 4E play, leave out a few critical rules and aspects... especially in regards to Wizards.
The pre-generated Wizard character, for example, only has half the number of Cantrips that should be available to him. They left out Prestidigitation and Mage Hand, arguably the two most versatile and useful of the Cantrips, if you have an inventive Wizard.
Also, Wizards are allowed to choose two powers at each level (three, if they take a feat) and after each extended rest they get to choose which of the two powers they want to use for the day -- it's a pseudo-Vancian method.
Lastly, while the list of Rituals presented in the book is disappointingly short, they do provide quite a bit of variety and flexibility outside of combat not just to Wizards, but to anyone willing to take the Ritual Casting feat. In addition, it gives me, the DM, an opportunity for a little more variety in the treasure they find... Finding a new ritual or valuable components for rituals makes my Wizard's eye's light up.
You might encourage him to look for it in different places (character/team synergies, for instance) to get the same kind of rush, but it could be that 4e just won't provide the enjoyment that he's looking for.
This is very, very important, and a big distinction between 3E and 4E. 3E was mainly based around building a group powerful characters that make up a group of adventurers. Whereas 4E is more about building a group of characters that make up a powerful group.
Much of the fiddling in 4E comes from group tactics. Instead of finding powerful combos within the scope of a single character, you have to look for powerful combos that span the abilities of the entire adventuring party.
I've also found that my players have shifted their class preferences from 3E... Players that couldn't stand playing spellcasters previously are picking Clerics and Wizards, and those that up until now detested sword-bashing are choosing Fighters and Rogues. Your player may enjoy playing another class better. As someone who has similar sensibilities about arcane spellcasters as he does, I might suggest the Warlord and also to take another look at the Wizard, once you've got a copy of the PHB with the full rules.
Otherwise, you can always go back to playing 3E. There's nothing wrong with that, so long as none of the players mind.