These threads are great to get me thinking about how to be a better GM.
The degree to which I "make stuff up" is inherently related to the specific game. Some games require more of that than others. So I'm going to talk about how I like to do it in D&D.
I think, as others have said, that consistency is a key. If I'm going to make global changes to rules (house rules), I let the players know ahead of time. Otherwise they might create a character that doesn't function as expected. That is annoying. I always ask my DM to let me know what the house rules are ahead of time for exactly that reason.
Example from an upcoming D&DNext game:
2. Finesse: The finesse weapon property does not function if you are wearing heavy armor or using a shield (bucklers are allowed).
3. Single-Weapon Fighting: If you are hit while wielding a weapon in only one hand, you may use your reaction to increase your AC by 1 (potentially causing the attack to miss) provided you have not used your other hand to attack, defend, or cast a spell since the start of your last turn. When you propose this option, the DM will tell you if you were missed by only 1, so that you won't waste your reaction when it cannot work.
Players could be justifiably upset if they didn't know either of these rules. I have document with all 6 of the formal house rules presented to the players.
Other things that I find nice to have understood from the start are how the DM interprets and applies rules that tend to be treated differently by different DMs.
Example from general experience:
I like to secretly roll many skill checks so that players don't know if they succeeded or failed. This really adds to the sense of immersion for me, and I love it. There is no rule for or against it, so it's legal either way.
It wouldn't bother me either way on how my DM does this, although I'd personally prefer he kept some rolls secret. As a DM, I make a judgement call on whether or not listing off these sorts of stylistic techniques to the group prior to the campaign is going to be appreciated or just feel like a waste of time to the group, and then act accordingly.
Then we get to what I think many of us are talking about: creative application. This is where we take the rules as written and bend or add to, without actually invalidating the base rule.
Example from upcoming D&DNext adventure:
I'm planning to run Isle of Dread, and the players will arrive via a shipwreck. Since I've had a lot of time to work on it, I've done more prep than I usually do (I love to improvise). I've decided that the players who desire to will be able to make Intelligence checks (with proficiency bonus if they have training in and use navigator's tools) to figure out where they are on the coast based on their map, after a day of coastal travel. I will make the rolls for them in secret. If anyone who is using and trained in navigator's tools fails the check by 10 or more points, I will rule that they believe they know where they are (even though they are wrong). Those who aren't trained and fail know that they don't know. So if we have trained characters succeeding and trained character failing by 10, we will have two different sources confidently stating where they are, and none of the players will know who is right. Golden. Now the characters can discuss/argue/reason amongst themselves, or decide to go one direction or another until they find out who was right.
I've made two situational rules changes. The official rules would probably require a Wis check, but I rule Int makes more sense in context, since you are trying to figure out a puzzle from nothing but the visual clues, rather than keeping track of where you are and where you are heading. Navigator's tools still will normally use Wisdom, but I've set precedent that sometimes ability scores will work with different skills/proficiencies. I've also made proficiency or lack thereof have an effect on the results of the check.
Personally, I'd both appreciate my DM doing that sort of thing, and like to know ahead of time that he does it. As DM, I'd tell players ahead of time about these sorts of situational changes I might make, and how they might set precedents. In the specific instance, because it's more or less a one-shot with pregenerated characters designed to start up quickly, I won't bother telling them ahead of time. Instead I'll ask before starting another adventure or campaign if they liked those approaches or not. These changes set precedents, but since those precedents are purely in the hands of the DM (like setting DCs), they can't really be easily codified other than to say, "I'm ruling that I can do this. Anyone object to either precedent?"
Next comes the part that can get annoying if not handled properly: mid-session rules revisions.
Made-up Example:
In running a session of Next a PC mage blasts the heck out of opponents with Ray of Frost, so I decide that cantrips do less damage. Let's just make them scale the same as a cleric's divine strike, because I find that more pleasing.
This is a big red flag. I'd hate that as a player, and I wouldn't do it as a DM. Instead, I'd let the rules stand as they are, and after the session or adventure I'd approach the group with the proposed house rule and see how they felt about it
And probably the worst way of doing it would be the unstable rules syndrome:
Made-up Example:
Mid-session decide that the PC's rays of frost do less damage, but I let the NPCs cantrips keep doing the normal damage. When the PCs notice this, I change one or the other to match. Then I do something obnoxious like this later because I think that's how it should be.
... Needs no commentary.
Now, once you get past the rules you get into content creation. Part of being a GM is making up content. Unless you are using a published module, that is a huge part of your role. You make up NPCs, locations, story hooks, etc. Even in this though, consistency in style matters for trust.
Improvisationally adding or increasing challenges based on party effectiveness is something I personally do not like. I don't want the NPC to spontaneously gain extra hit points because we are whomping him, or a trap to appear in the next room. Players will pick up on this. For me, it reduces the trust that I have in my character's agency. It's like Elder Scrolls IV: Oblivion out of the box, where you end up with epic mudcrabs and bandits in full daedric armor (needless to say, I played it with a mod) because they just keep getting stronger to stop you from getting anymore effective. I want to know that the world is what it is, and I get to interact with it in a simulationist manner. If we are playing a narrativist game where the world is just a supporting figure to a story about the characters, I darn well better be told that from the get go (or know it is the case because we are playing an overtly narrativist system). But I'm talking D&D.
So I determine what the challenges are, and stick with it. That doesn't mean that I might not make them somewhat adjustable. For instance, if I didn't know how many players I was going to have in a particular adventure, I might decide that there is one bugbear present for each PC at a certain encounter. But IMO, this needs to be done pre-game.
Interpretation of setting is related to the above, but something that I'm not nearly as opposed to. With interpretation of setting you make judgement calls about dynamic events based on what seems most fun to the story or game.
Hypothetical Example:
For instance, if you know that there are such and so NPCs in a small village, or even that there are certain types of NPCs in a small villages, you can decide which ones the PCs run into when based on DM fiat (instead of or in addition to a random table). These are the sorts of situations where I might roll randomly on a table if present, but then may stick with the roll or change the result if I think it would be more fun in the specific situation (I do the same with random treasure tables--want the tables, but like to tweak results).
I've never heard of a player objecting to this sort of arbitrary judgement. The NPCs are all there (or potentially there by prior DM/adventure design), and the DM is simply choosing which elements to bring in when. This is a major part of being a good DM, IMO. I think part of the reason I consider this a good thing (unless used to the complete exclusion of all randomness--I like randomness in D&D events) is that it doesn't adjust the challenge. You aren't just deciding to make the game more difficult for them.
And finally, adding color is another essential part of making stuff up as DM.
Example made up right now:
Harold Uthgarden is the long-time bartender and owner of the Flaming Flagon Restaurant and Tavern. Harold is a slightly portly middle-aged man with tawny skin, dark hair, a short beard and an eye-patch. He offers a friendly grin missing a single front tooth. He always wear a white cooking apron while at work, primarily because he likes to pop into the kitchen and make food for himself. Harold is a bit prone to drink, and when drunk he seems to poor most of his brains on the floor. At least he's a happy drunk. Harold is a good-natured man, with quite a bit of wisdom to impart to those who earn his friendship.
I find that sort of thing to be a highlight of my DMing experience. I enjoy the improvisation and acting involved in making up NPCs on the spot. It doesn't bother me in the least when my DM does that either. It's a tactic that is most enjoyable from either side of the screen if the players can't tell which characters you made up on the spot and which characters were pre-designed. The only way I can imagine this being a bad thing is if a DM needs a bit of work on how well he accomplishes the goal. Perhaps he recreates the same 3 characters over and over again, or stops the game for 5 minutes to think up a name, or the characters he makes up never have any future relevance to the campaign, even if the players attempt to invest in giving them relevance.
So when it comes to content creation, unless your improvisation involves increasing challenges, I can't see that it would even need to be run past the players prior to the campaign. It's just a given and desirable practice unless you are using a highly structured adventure.