Before getting into any of that, I'd first ask yourself a question: How badly do you need the money and how much do you want to get rid of your stuff? For most items you're not going to get a great return, especially when you factor in the time it takes to put stuff up online and drive to the post office. If you really need the cash and want to get rid of your stuff, read on. If not, well, just enjoy the way your books look on the shelf.
The problem with selling RPG stuff is that quite a lot of it has little or no value, especially third-party stuff from the OGL/d20 days. Trying to get rid of your copy of Diomin or Codex of Erde? Good luck.
It really depends upon what you're trying to sell. There are some basic rules of thumb, but RPG material can vary widely in price. Feel free to post a list of what you have and I'll try to let you know what has value and what is kindling.
RPG.Net has a good sales/trade forum, but the same principle applies - you might garner interest in your copy of Wilderlands of High Fantasy or Knights of the Old Republic, but no one wants your copy of Oathbound.
This is the same for used books in general - the availability of materials on the internet has both increased the price and value of truly rare items, and decreased the value of everything else. You'll see $.01 priced copies of books on Amazon; the price of shipping is $3.99, so one can only surmise that a small profit is made if the seller ships cheaply.
Ebay or Amazon are both useful, Half.com as well, but especially with the latter two you might just put your stock online and then forget about it until you get an email four months later that one book sold.
You might also want to consider selling them as lots on Ebay. You won't make much money per book, but you could sell a bunch of stuff and still make a few dollars. A "twenty book d20 lot" might make you $20-50, whereas most of them wouldn't sell individually. Of course then it depends upon how desperate you are for the money or if, in the end, you'd rather keep a given book than sell it for $2-3.
Hope that helps and good luck! Again, feel free to post your list of stuff and plenty of the folks here will be able to help you out.