I liked the Shen prestige class a lot. I have developed a martial arts prestige class that is based in the development of chi(or ki) abilities, the Chi Master (you can check it on another thread in this same forum).
I like your idea of having martial arts feats/shen prestige classes/martial art masteries associated with fantasy creatures instead of the usual Shaolin kung fu ones.
My Oriental Adventures DM and I agreed in that the Shen prestige class would work better if instead of actual shapechanging, the different forms of each animal shen meaned the martial artist getting in a certain "stance" (if you have played any Tekken games and saw Lei character or any Shaolin kung-fu type movies you would know what I mean... each of those animal styles have a distinctive stance and way of fighting/moving).
I guess that the best way of doing what you suggest is by having a "stance" related to each mythical creature. How to make those stances available to a character remains open to debate: they could be feats, they could be special abilities related to a certain prestige class, they could be special types of Shen prestige classes, or they could even work like martial arts masteries (like the ones from OA). You could even rule them as alternatives to high level monk abilities like Abundant Step or something like that... say that if he meets some prerequisites he can swap Abundant Step or another ability of similar level with the DM consent for the ability of taking one of those stances.
My suggestions would be something along the lines of:
Displacer Beast stance: while in this stance the martial artist has a 20% miss chance (as for concealment). Also, he can make attacks of opportunity like if his reach was 5' longer by lashing around as if his whole body was a tentacle. If he does so he is unbalanced until his next action however.
Golem stance: while in this stance the martial artist gains a Damage Reduction like (Wisdom Bonus)/Adamantine. He also gains the equivalent of Light Fortification armor (25% chance of negating critical hits/sneak). He loses his extra speed and monk level AC bonus (but not the Wisdom bonus to AC) while in the stance.
Minotaur stance: while in this stance the martial artist can make an special charge attack that deals extra damage. He effectively gains the Powerful Charge feat (extra damage on a charge depending on size). Also, his blows are considered as if made with a two-handed weapon (+1.5xStr modifier, +2 to damage for each -1 with Power Attack...) while in this stance, but he can't make a flurry of blows.
Treant stance: While in this stance a martial artist gets the Improved Sunder feat and deals an extra damage against objects equal to his Wisdom bonus. He also gains Light Fortification but gets a -4 on saves vs. fire effects (but he keeps his evasion or improved evasion abilities if available).
Those are ideas right out of my head, it could be worked upon and improved...
