Well, anchovies are a Roman "invention" (well, maybe they didn't invent the fish itself, but the kind that come cured in tins), and have long been used as the basis for sauces, including Caesar salad dressing. In fact they where the basis of a staple dish in Roman times (can't remember the name off the top of my head), and where a pretty big deal as a trade item.
Being from Italy, it's not too surprising that they ended up on pizza. They are also supposed to be an aphrodisiac, at least in their raw form. In most dishes they are used as a flavoring and not as a main ingredient, as the flavor is so strong, but I do enjoy them on pizza, myself. I find they're better cooked on, so that the oils and flavor bakes into the sauce - some places seem to put them on the pizza after it comes out of the oven. Eating them raw (and they're cured, so it's not really raw...) they are extremely salty and have a very strong oily flavor (most oily fishes do).