Good stuff.In a recent campaign the PCs had an artefact - a mirror from Open Grave. It had a personality and was pretty insane.
<snip>
I think it seemed mysterious - partially because it had a personality, partially because its history was unknown (it was tied into setting elements which the players had yet to uncover).
You could probably do that pretty easily with D&D's standard spell casting system.
<snip>
Not the only way to make magic mysterious, but it seemed to work in this one case.
In 4e I think that warlocks would particularly lend themselves well to this. I also find that the chaos sorcerer Demonskin Adept in my group has something of a mysterious feel to him - in part because of the funny things happening on random die rolls, in part because he has a range of immediate actions which mean that his magic use is never predictable, in part because of the sorts of story elements that are inherent in his source of power, which resemble in some ways what you were saying about the mirror in your game.