Before my game inflated to six players and a sporadic guest appearance by a seventh, it began as a three-player game (with a mostly-optimal two-trident elf ranger, a very-suboptimal human inspiring warlord, and a generally-optimal human shielding swordmage). I began by building encounters for three PCs, but that limited possible enemy synergies too much to provide a satisfactory challenge to the PCs. The budget system doesn't per se break down below a four-PC budget, but it becomes far less reliable as a gauge of challenge.
I ultimately defaulted to building as if there were four PCs, and not counting minions in the budget (but still awarding XP for them). These were pre-MM2 minions, and so were little more than challenging terrain features. To compensate for the outsize challenge, I adjudicated effects far more loosely with an eye to the cinematic. Jam two tridents into someone's ribs, and tear him in two with a fey scream? All right, any minions next to him faint, or flee, or otherwise exit the theater of battle. When they first bloodied an enemy, that enemy would typically suffer some form of lasting setback, like having an arm hacked off, dropping a weapon, acquiring a limp (and an attendant -1 to speed), etc. I also allowed (and still allow, though in a more limited fashion) an Awesome Bonus, wherein a player describing the attack entertainingly can grab +1, +2, or +3 (yielding +1 only now, with an allowance for true excellence granting +2) to the attack roll, depending how novel and entertaining the particular description is.
I also started using interesting terrain more and more. Every session, look at the character sheets, and pick two characters who can shine in the terrain, and one who can be particularly hindered. We had a melee-heavy party, with one elf with high Athletics and one character in heavy armor; difficult terrain or large chasms (for instance) helped the elf ranger shine, and the chainmail-wearing warlord to be hindered. The swordmage had armor that granted him resist 5 fire, so I would frequently scatter campfires or flammable items (including a memorable fight in a dry cornfield) around the encounter area--usually fires that would deal 5 damage for entering or for starting a turn within them. On the other hand, the warlord's numerous ways to grant allies additional movement and the swordmage's several ways to rearrange enemy positions made narrow corridors a great way to let them shine.
The "two feature, one falters" system works well for crafting the particulars of an encounter's enemy spread, as well, either on a whole-encounter level, or on a per-enemy level; that is, you can either build the entire fracas so that two characters will be especially effective vs. their opponents, or select each (or the majority) of the foes such that each enemy is "weak" to two characters' strengths, and "strong" against one character's weaknesses. The former is easier, but I found the latter created more exciting encounters for the players. Fire-using enemies, high-damage/low-mobility enemies, and enemies (Get a thesaurus, Squizzle.) with low to-hit bonuses but terrible on-hit riders made the swordmage stand out as a hero, able to keep his peers safe or to lock down and then prance away from a particular heavy-hitter. He also was especially effective against minions. The elf ranger (especially once he got a Cloak of Distortion) was the most effective in the party against ranged opponents, and also was excellent against slow-moving enemies (since he had a variety of ways to increase his own movement), and could escape grabs easily with his high Strength and Dexterity, plus trained Athletics and Acrobatics.
I'd caution against using a 2/1 spread for social or skill-based encounters, meanwhile. In a combat, everyone can deal some damage, or at least absorb it, and thereby feel like they contributed to the victory; if someone has no strength to leverage in a skill- or roleplay-based event, that person is sidelined, and doesn't get to have much fun. I made a spreadsheet with every party member's skill bonuses, highlighting ones in which they had the party's best score or lowest score (and a few other details, mostly useful to me personally). In any skill-based encounter, make sure every party member has a skill they can use that is at least above the party's average score in that skill--preferrably, one that they have the party's highest score in.
If you add more members to the party, be ready to retract, incrementally, the party-friendly bonuses you gave (like reeling in any Awesome Bonus you might implement). Do not, however, stop building encounters that spotlight various members of the party--just don't build ones that spotlight every member of the party (positively or negatively) if that's going to be difficult. Everyone should get roughly equal time to shine and to be hindered, but not everyone needs to be one or the other in every encounter. In a four-person party, aim for two excelling characters, one characcter of normal effectiveness, and one character hindered, for instance; for seven characters, try 2/3/2 or 3/2/2, or even 2/4/1. It's of course not going to kill the game if you have a few encounters that favor everyone, or hinder everyone, or otherwise alter that spread--in fact, it's good to mix such encounters in, to keep things varied--but you want to keep a rule of thumb like this (or whatever you finds works for your group) in mind, so that you're consciously creating a baseline. Make sure to rotate the "spotlight" appropriately, so that no one is shining every encounter or feeling screwed over constantly.
It might seem like a lot to think about when expressed as a wall of text, but if you just keep it in mind as a guiding principle when building encounters, I think you'll find thtat your encounters get better and better in terms of player enjoyment and your own fun, and that you'll eventually settle into your own personal encounter-building groove wherein tailoring the game to your party comes as a second nature.
I ultimately defaulted to building as if there were four PCs, and not counting minions in the budget (but still awarding XP for them). These were pre-MM2 minions, and so were little more than challenging terrain features. To compensate for the outsize challenge, I adjudicated effects far more loosely with an eye to the cinematic. Jam two tridents into someone's ribs, and tear him in two with a fey scream? All right, any minions next to him faint, or flee, or otherwise exit the theater of battle. When they first bloodied an enemy, that enemy would typically suffer some form of lasting setback, like having an arm hacked off, dropping a weapon, acquiring a limp (and an attendant -1 to speed), etc. I also allowed (and still allow, though in a more limited fashion) an Awesome Bonus, wherein a player describing the attack entertainingly can grab +1, +2, or +3 (yielding +1 only now, with an allowance for true excellence granting +2) to the attack roll, depending how novel and entertaining the particular description is.
I also started using interesting terrain more and more. Every session, look at the character sheets, and pick two characters who can shine in the terrain, and one who can be particularly hindered. We had a melee-heavy party, with one elf with high Athletics and one character in heavy armor; difficult terrain or large chasms (for instance) helped the elf ranger shine, and the chainmail-wearing warlord to be hindered. The swordmage had armor that granted him resist 5 fire, so I would frequently scatter campfires or flammable items (including a memorable fight in a dry cornfield) around the encounter area--usually fires that would deal 5 damage for entering or for starting a turn within them. On the other hand, the warlord's numerous ways to grant allies additional movement and the swordmage's several ways to rearrange enemy positions made narrow corridors a great way to let them shine.
The "two feature, one falters" system works well for crafting the particulars of an encounter's enemy spread, as well, either on a whole-encounter level, or on a per-enemy level; that is, you can either build the entire fracas so that two characters will be especially effective vs. their opponents, or select each (or the majority) of the foes such that each enemy is "weak" to two characters' strengths, and "strong" against one character's weaknesses. The former is easier, but I found the latter created more exciting encounters for the players. Fire-using enemies, high-damage/low-mobility enemies, and enemies (Get a thesaurus, Squizzle.) with low to-hit bonuses but terrible on-hit riders made the swordmage stand out as a hero, able to keep his peers safe or to lock down and then prance away from a particular heavy-hitter. He also was especially effective against minions. The elf ranger (especially once he got a Cloak of Distortion) was the most effective in the party against ranged opponents, and also was excellent against slow-moving enemies (since he had a variety of ways to increase his own movement), and could escape grabs easily with his high Strength and Dexterity, plus trained Athletics and Acrobatics.
I'd caution against using a 2/1 spread for social or skill-based encounters, meanwhile. In a combat, everyone can deal some damage, or at least absorb it, and thereby feel like they contributed to the victory; if someone has no strength to leverage in a skill- or roleplay-based event, that person is sidelined, and doesn't get to have much fun. I made a spreadsheet with every party member's skill bonuses, highlighting ones in which they had the party's best score or lowest score (and a few other details, mostly useful to me personally). In any skill-based encounter, make sure every party member has a skill they can use that is at least above the party's average score in that skill--preferrably, one that they have the party's highest score in.
If you add more members to the party, be ready to retract, incrementally, the party-friendly bonuses you gave (like reeling in any Awesome Bonus you might implement). Do not, however, stop building encounters that spotlight various members of the party--just don't build ones that spotlight every member of the party (positively or negatively) if that's going to be difficult. Everyone should get roughly equal time to shine and to be hindered, but not everyone needs to be one or the other in every encounter. In a four-person party, aim for two excelling characters, one characcter of normal effectiveness, and one character hindered, for instance; for seven characters, try 2/3/2 or 3/2/2, or even 2/4/1. It's of course not going to kill the game if you have a few encounters that favor everyone, or hinder everyone, or otherwise alter that spread--in fact, it's good to mix such encounters in, to keep things varied--but you want to keep a rule of thumb like this (or whatever you finds works for your group) in mind, so that you're consciously creating a baseline. Make sure to rotate the "spotlight" appropriately, so that no one is shining every encounter or feeling screwed over constantly.
It might seem like a lot to think about when expressed as a wall of text, but if you just keep it in mind as a guiding principle when building encounters, I think you'll find thtat your encounters get better and better in terms of player enjoyment and your own fun, and that you'll eventually settle into your own personal encounter-building groove wherein tailoring the game to your party comes as a second nature.