It's already been mentioned, but it bears repeating... There are some things you'll get next to nothing for, and other stuff you'll probably do well on.
Selling the stuff on eBay will bring you more money than if you were to sell the stuff to a dealer. It's a heck of alot more work, though.
Things to maximize your money:
1) Accurate, detailed descriptions.
2) Photos/scans of the items.
(Condition is incredibly important, especially since you're talking about out-of-print stuff, which is - in theory - a collectible. People want to know and see what they're bidding on.)
3) Get as many key words in your title as possible - you want people using the search function to be able to find the item.
4) Make sure you list your shipping price in the description (and DON'T overcharge - a small handling fee is okay, but don't be ridiculous). If someone doesn't tell me what shipping is, I won't bid.
5) Most D&D stuff can be sent Media Rate which is cheaper. You may want to consider Priority Mail because the packing materials for that are free at the Post Office.
6) Shipping info can be found online at
www.usps.gov.
7) Unless you absolutely have to, DON'T set a reserve price. I won't bid on those auctions.
8) If you feel you must have a reserve, state what that reserve is in the description.
9) Low opening bids draw people in. High opening bids scare people away.
10) You want to get multiple people bidding on the same item. That's what makes you a profit.
11) People will bid more than they really want to if they get in a "war" with someone over an item.
(If you start with a low price, you're more apt to get multiple bidders. If you start with a high price, you're more likely to get only one bidder. If you start with a high price, make SURE that price is what you're willing to part with the item for, because you will probably only get one bidder - if that.)
12) Do a search for any item you plan to list. If there are a few others up already, don't list yours at that time.
13) Do a search for any item you plan to list, then check the completed auctions for that item. That will give you an IDEA of what that item will sell for.
14) If you sell items in lots, be sure to list everything in the lot. If someone is too lazy to type up an accurate list, I'm not gonna give them my money.
That's all I can think of off the top of my head.
). 
