It is not a subtle observation, just a statement of the obvious. To argue that a structure of formal rules is not a set of restrictions would be utterly absurd.
There are two ways to look at rules. The first way is this: "If rules are restrictions, then everything that is not covered by the rules is permited."
In my opinion, having a structure of formal rules is actually enabling. I don't see rules as telling me what I can't do, as give me the DM (or player) tools for handling common situations in a fair, uniform, and comprehensible manner. Because I see the rules as a set of restrictions, the rules aren't in the way except where I want them to be, like "You can't walk through walls without a special exemption that lets you do so."
And in this sense, a set of formal rules is no more restrictive in practice than a set of informal rules. In practice, no matter how informal the DM likes his rules, there are restrictions he wants to impose. The restriction that, "You can do everything you can in real life, and nothing more.", is actually a pretty heavy restriction. Worse yet, informal leads to all sorts of arguments about what you can and can't do in real life because the player feels like he's restricted by the heavy burden of the DM's (often unjustified) opinions. Worse yet, informal rules puts a very heavy burden on a DM to be a master rulesmith capable of spinning up fair, balance, and comprehensible rulings on every subject on a momments notice. This isn't a terribly big burden to me, because I am a pretty good rules smiths (if I do say so myself), but an enormous number of campaigns have gotten derailed by bad rulings from otherwise good DMs.
The alternate approach is one I see from time to time put forth by proponents of modern gaming, and it is, "The rules are a set of permissions telling you what you can do." I've met otherwise rational and logical players that insist on this interpretation because they think it is empowering somehow, but in fact it is entirely disempowering. If the rules are a set of permissions, then everything not mentioned by the rules is forbidden. I think it is actually that interpretation of the rules that brings 'old school gamers' into contention with 'modern gamers'.
I don't think it is whether or not you have a formal rules set. One only has to puruse 1e rule books, old copies of Dragon magazine, and even some of D&D's competitors at the time to see that when it came to wanting a rule for everything, nothing quite surpassses the 'old school' style. (This is the frequent subject of humor in literature that spoofs the 'old school style, such as 'Knights of the Dinner Table'.)