I would say that in an 'improv' method such as yours (which I have used in the past), is going to be a shallow pond. Working off the cuff, improvising as you go, you can certainly come up with memorable gaming; it can even allow for some player agency.
But for real, dynamic player agency, I believe that you need more detail and depth of plot than a GM can come up with 'on the fly'.
This is wildly incorrect. It can apply if the focus is on exploring the GM's ideas, yes, but this isn't at all the only approach to take. If the focus is on the PCs, then an improv-style can have deeply immersive and detailed play because it's the players themselves that are helping. On the other hand, as I just said, if the focus is on exploring a detailed setting not focused on the PCs, then, sure, prep helps quite a lot unless you're just very gifted.
Take Blades in the Dark, my favorite non-D&D go to game. It has a loosely defined setting with only a few hard points that act as boundaries to the game (you can't easily leave the city, so you have to deal with the consequences of your actions -- no murderhoboing). As settings go, it's got a very light touch -- almost no details past some thumbnails for some gangs and neighborhoods. The game itself is entirely driven by player actions and the resolutions of such. It's very improv -- you can't prep the game because the way the mechanics work the first few checks thrown will spiral out of any possible prep you could have done, and if you try to use prepped material, you'll break the game mechanics. But, within that, I've run games of shocking depth and detail, largely because Blades spreads that load by focusing play on PC goals and using PC actions to build the fiction, so the players are right there with you helping detail out the game. It's not the players exploring the GM's game, but rather everyone at the table discovering the game together.
And, this isn't to knock D&D -- I'm ending a Blades rotation shortly and diving right back into my 5e campaign. Covid and the fact I play with some first responders while I started in a new project at work has made things pretty hectic, so the Blades campaign game, where a consistent player group isn't much of a problem, made more sense. Still, excited to get back to my Planescape game. Although, to be fair, I don't prep too heavily there and still have a pretty detailed and intricate game going.
But at the bottom line, if your players keep showing up week after week, you're doing it right.
Now, this... this is absolutely 100% right. If everyone's having fun, you're playing the right way.