As a caster(cognate), yeah, the LA on a drow bites hard, especially as the abiltiy bonuses don't push the save DC's above where they would be otherwise.
Now, if you were playing, say, a Phenric Human Wilder (+4 Charisma, +2 LA), your manifester level would still be down by 2, but your save DC's would be up by 2; as most of the powers save DC's scale up at +1 DC per 2 PP's spent, that puts you at a net gain of +1 DC (at a cost of power points, manifester level, power access, saves, BAB, Hit Points, Hit Dice, and Feat Progression). For primary casters (and caster cognates, such as Psions or Wilders), Level Adjustments greater than 0 bite ... hard.
You might, for example, have been a Grey Elf (+2 Int, -2 Str + regular elf traits; LA + 0), put those extra two levels into Psion, and picked up the Touchsight power, possibly Psionic Darkvision, and Incarnate. Poof! Permanent Darkvision-60, in addition to elven low-light vision. Granted, Darkvision-60 is not as good as Darkvision-120, but you have noticed the drawbacks inherent in an LA, and with the extra powers known and power points, you could manifest an augmented Touchsight when needing a longer sight range. LA > 0 is generally not worth it for a primary caster (there are a small handful of exceptions - usually, overpowered templates, or those that add to your effective spellcasting level in a spellcasting class you were taking anyway).