1. Athlons do not run hotter than P4s. They run a bit cooler, but it's mostly a null issue now, as the biggest problem when they did run hot was that very few manufacturers automatically shut the CPU off if it got too hot, and that they retail fans weren't that good. Both of those issued have been successfully dealt with for most any motherboard or retail CPU from the past year.
2. P4 celerons suck. Period. Athlons, up to about the 2800+, tend to be just a bit faster than P4s...but the 800MHz P4 'C' ruined that PR scheme (Athlons still have an edge in office apps, but if you notice a 10% performance gain in Outlook, you're way too picky). The P4s are still more expensive, a but a 'C' in a dual-channel DDR motherboard will beat most XPs for memory-intensive tasks, such as games and working with video, file compression, virus scanning, etc.
3. Unfortunately AMD chipsets are hard to find, and their last for the Socket A isn't a good performer now. VIA's are currently good, but not great. Given the prices, I wouldn't bother, given how I've had amazing luck w/ SiS.
NVidia's NForce2 is mostly ironed out and is as good as it gets for performance, but I don't know as I'd trust a server with it.
SiS' 746 (and 746FX and 748) is really about as good as it gets. Not quite NForce2 performance, but within single-digit %s, and I've yet to have one with any strange issues (like VIA is so known for) that newer chipsets don't (even Intel's are having issues with memory now).The ECS L7S7A2, while amazingly cheap, is an excellent, if no-frills, board, and combined with a ~2GHz Athlon, you'd be hard-pressed to beat the value. As long as you get a decent power supply (Fortron 300w only $32 shipped, not a big deal), any SiS-based board will be happy.
4. If you're used to an Intel or want a prebuilt PC, get a real P4. Even a 400MHz FSB 2GHz will blow away any Celeron, and a 800MHz FSB 2.4C will have a nice, long life.
5. Don't get an Athlon OEM PC--let someone you know or a local shop make one. while I've seen and worked on many that are 2 and 3 years old and still running, HP/Compaq never cease to amaze me at how cheap they can get. Dell might still be cheap, but not as bad as HP/Compaq, and always have much better designs, using plastic instead of steel, where HP has a fully steel case that can't have been as cheap as Dell's cases, but sacrifices capacitors everywhere.
Bottom line: Athlons don't suck, Celerons do, and a P4 OEM PC is your best bet, especially if you can find one with HyperThreading in your price range.