I think the big differences between homebrew stuff and a product are utility and clarity, other than that it's just matter of quality and finding a publisher.
I say utility, as far as is it useful for gamers besides yourself? Obviously, a guide to the customs of a race you made up for your game is useless for other gamers, who don't have that race. A lot of guys want to publish their superhero character and his supporting cast, but what you run into is, who cares? Villains can be products, because people need bad guys to fight, but a superhero isn't a very good product, because people like to make their own hero characters. If you are going to play a superhero someone else made up, why not just play Superman or Batman instead of Captain Homebrew? If your homebrew hero is super interesting, that's great, but who wants a hero more interesting than the PCs hanging out in their neighborhood?
Clarity is just expressing your ideas in an easily communicable way. Laying out a fantasy city so that people know where everything is and how to use it in their game. Part of this is consistency, like not mentioning a mayor in one spot and having the city ruled by a council or whatever. Part of it is being clear with yourself, as to what you are writing. Don't write a book on rangers in non-forest settings and end up writing more about life at sea than marine rangers. This seems to be a problem for a lot of people.
When the writing is done, the other part of clarity is the actual expression of that writing. In pdf or print, things need to be laid out and organized, and how this is done makes all the difference between producing a professional product and looking like a hack. Some people want to form their own companies, others just partner with an existing one to package their work. This makes a lot of sense, as you don't want the first thing you write to have the added disadvantage of being the first thing you ever laid out. It doesn't take a lot of money to become a pdf publisher (as opposed to print) but it takes a lot of know-how.
This is all obvious stuff, but I hope it helps.