Rev,
What they said is generally correct, you can't remove IE from Windows.
However, you CAN lock it down so that it's not as much of a security risk. It utterly hobbles IE, but if you use a 3rd-party browser, that's fine.
My suggestion:
Go to Control Panel, Internet Options - these setting are essentially all the allowed options for IE under Windows, they won't affect any third-party browser you are using..
General Tab:
Set Home page to blank with "Use Blank" button.
Temporary Internet Files: Setting Button -> Set Amount of Disk Space to 1 MB.
Security Tab:
Select each Zone separately, click the custom button, and set every option except Use Pop-up blocker to Disabled or Requires User Name and Password.
Privacy Tab:
Block All Cookies (Slider to top)
Check the Pop-up blocker.
Pop-up clocker Settings button, make sure no sites are allowed through, and set Filter Level to High.
Hit Ok.
Even if you have a program that refuses to open links in your 3rd party browser and forces the opening in IE, you should be protected from most standard infection vectors.
Hope that helps.