AbdulAlhazred
Legend
I can only give personal recollections, so with that proviso..
Game designers and some players were definitely aware of some of the possibilities a good bit earlier.
1st of all I would say the sandbox wasn't a RESPONSE to anything. The sandbox was the Ur State, where D&D started. I mean it might have in its most prototypical form have been just "here's the dungeon, go any direction you want" but the point was you DID always go whatever way you wanted. The DM might trick you into going a certain way or put out the bait, but no GOOD DM forced anyone to do anything. Obviously simplistic railroad was an easy outgrowth, the DM had only a certain adventure to run, so choo choo you ended up there. Even back in the earliest days that wasn't really considered tolerable DMing.
In any case IIRC Top Secret SI had some sort of meta-game stuff. Toon was releasd in 1984 by SJG and very definitely was almost ABOUT the meta-game/breaking the 4th wall, etc. Admittedly though, MOST early 80's designers were focused laser-like on better sim. I think the thought was that somehow if you could make an RPG that was a perfected enough simulation of the genre then somehow it would be qualitatively better, thus such monstrosities as Twilight 2000 which simulated every detail of your character's gun and which bullet he was using and exactly where to the inch you got shot, etc. I'd say by the mid-80's that misapprehension had been thoroughly popped.
Interestingly I do recall people using meta-game mechanics as house rules, even in the 70's. There was a Boot Hill campaign where after we'd played through a number of pointless rounds of bar fight, guns, everyone bleeds dead, we invented a plot point. Your character could invoke it to do most anything, but informally you had to use it 'fairly' to make your character's story better. You were actually expected to even use it against your character if it made his story more cool (characters were pretty disposable in that game anyway). One of the GMs got a bit carried away with it and we never quite carried that on as a general thing into other games, but the germ of the idea was definitely floating around.
It certainly is true enough IMHO that the whole thing didn't solidify and become generally understood until WW came along, and a few other similar things. I guess every idea has to percolate for a while before it ripens (ah the mixed metaphors).