Rules are what turn it into a game. Even if those rules are "try to match the storyteller's genre conventions in order to have a positive outcome".
If I swing a cardboard tube at my friend while making lightsaber sounds, we're having fun. But it's not a game. If she then dangles lifeless the arm I him her in, scoops up a cardboard tube in her non-dominant hand and swings at me (with the requisite lightsaber noise), we've now established some impromptu rules - get hit in a limb and lose the use of it. It's turning into a game.
Saying "the rules aren't important" goes one of two ways. Either what's happening it not a game -- it can be fun, like one person telling a story and others adding in. Or the rules are just hidden an the storyteller's head so they aren't obvious and shared. That can be a rewarding game, it can be a frustrating game. And potentially worse, it can be an inconsistent game.
I've played a card game where part of the rules are you can't discuss the rules and have to figure them out via play. But that doesn't mean there aren't rules. It just means you don't have access to them and need to learn them from observation.
This isn't a push for "we need heavy rules for everything, simulation is the only way to go". Part of rules for anything will always be how they are interpreted - in people's heads. Swinging cardboard tubes at each other can be a game.
For me, having a framework so everyone has a shared understanding, but not having much more than a framework that slows things down with rules of progressively less scope and/or importance, is also good. I don't need to have all of the corner cases mapped out. For others they do - or want a lot less formalized than that. That's all fine.
But don't mistake that for no rules, not if it's actually a game.