I'd say these are both mildly off. Though there's a lot to not like about 4e depending on your style, but it's not a bad game. I would say that outside of 1e Oriental Adventures, 3e offers about the most non-combat options for characters; of the two, 4e is more combat oriented.
If you follow the retro-clone philosophy, Labyrinth Lord is looking like best-of-breed. The PDF is free, but I do believe a print version is available:
Labyrinth Lord
Listen to Psion. The man knows his games.
Personally, I like 3.5. Alternatively, you could check out Paizo's Pathfinder, which is described as 3.75 since Wizards of the Coast no longer support 3rd edition. I don't like 4th, and I agree that it's more focused on combat than 3rd edition is. Also, 4th took away a lot of the "superfluous" things that - for me, make 3rd edition fun.
OD&D - original D&D, 1974. Forget this one. It's basically unavailable, unless you want to scour eBay and pay upwards of $100 for it.
Basic D&D - this one is doable, but you'll have to get them all off eBay. Even then the boxed sets might be a little pricey, and you have to get multiples of them as you gain in level. There are a number of varieties of this game, and even I can't keep them all straight. I'm most familiar with the "red box" basic set. There are others. If you want to go this route, you could try locating the "Rules Cyclopedia" - it contains everything the boxed sets had.
1st edition - books are available relatively easily on eBay, and probably relatively cheaply. 1st edition isn't a bad set, but the rules are kind of wonky in places. Also, most GMs who ran 1st edition had tons of house rules to "repair" a lot of the weirdness in the game. Certain things that are in the rules were almost universally never accepted by the masses to the point that official rules often confuse and bewilder veteran players of the game. I'd suggest you find a grognard to teach you these rules if you're interested in them. 1st edition does play very loose, which sounds kind of what you're looking for. If you ever seen the D&D cartoon, it is based on this edition.
2nd edition - actually not terribly different from 1st. The bonus with 2nd edition is that there are some great campaign settings available for it like Planescape (pricey, hard to find) and Ravenloft or Dark Sun. That's the real appeal to 2nd edition. The rules set isn't too bad, though some of the rules are really weird (wrestling rules, I'm looking at you). There's tons of support material for this edition, and you could probably acquire the books on eBay very cheaply. I played this edition personally for over 10 years and had a great time with it.
3.x - this is currently my favorite system, and fixes a lot of the weird problems in 2nd edition, while at the same time expanding on what you can do in combat with rules for grappling, disarming, and bull rushing people. While these rules aren't perfect, they are at least there, which is more than what 2nd edition had. 3rd edition also simplifies the XP system, and removes racial level limits, and makes ability scores more logical. There's TONS of support material for it since 3rd party publishers could write products for this system, which is a huge bonus.
4th edition - to me, it's a completely different game than 3rd, and all previous editions. Sure, fighters and wizards are still there, but mechanically, the game is built around a completely different kind of engine - one that is tuned for combat. Some other drawbacks to this system are that it would be the most expensive since it's brand new, and there's very little in the way of support material for it, since it's only been out this year, and most 3rd party publishers won't touch it with a 10' pole.