Why wouldn't it work mechanically? Its not that different from a druid who focuses on fey/beast conjurations, a wizard who uses elementals, or even a cleric with their spiritual weapons (poor sorcerer, full caster and no summon options). Casters + pets is pretty common. There's some issues with the Chain and Beastmaster, but those classes act as more hybrid-caster styles than dedicated casters, which lends itself to different rules. Anyways, moving on.
Its not that odd, because the pets act as extensions of the caster's will. You had to cast a spell for their presence on the battlefield. So, its not that different than any other buffing spell - it just takes an unusual form.
Not really what else would really make them unplayable in your opinion. Clearly, its part of the game design for three different classes.