I don't mean the social aspect of play... that's something that depends entirely on the people involved and not the rules.
Exactly, and the social aspect represents a very large proportion (if not the entirety) of the play experience as felt by the participants.
I'm looking at play experience to be related to the play... is it challenge based play, is it collaborative, does play evoke a specific genre, and things like that.
Even there, the influence of the social side will trample any differences between, say, challenge-based play and collaborative play.
Whether mechanical play is challenge-based or collaborative, etc., is influenced by the rules to a greater or lesser extent depending on what said rules actually say, which is what the original discussion is-was about. The rules can't tell the social side what to do, however, which means their influence on the actual play experience is, at best, very limited.
And when it comes to evokation of genre the setting is going to trump the rules every time, in that if one is inappropriate to the other it'll be the rules that give way rather then the setting.
I think that it's a real challenge from a design standpoint to do what you're describing here and have any of those given playstyles provide a satisfying version of that playstyle.
If one uses a heavy hand when designing, this is true. But a light hand on the helm and a willingness to let some things be more freeform can open up a lot more space for different styles...
For instance, B/X D&D isn't well suited to courtly intrigue and diplomacy. And I think most versions of D&D will perform poorly if not outright fail at delivering satisfying versions of those types of play.
...and here's a good example. BX and 1e D&D don't have much by way of rules for social anything, and yet - ironically enough - that lack of rules IMO and IME means you can still do intrigue-style play in those systems simply by going freeform.