I don't VoP in front of me, but if you can manage the meager material components (three 5 gp items), then there is no reason you can't cast the spell. I see no problem with casters with VoP casting this spell; it's basically false luxury. You have the ability to create this fantasy mansion, but only for a short time (short time when compared to a permanent structure) and it disappears after a time.
Although, it does somewhat lead someone to question the character's vow. I would allow them to cast it, but it probably is more appropriate as a scroll...a sort of escapist thing.