Let's assume that there's some kind of Marie Celeste effect: People simply vanish, leaving everything as it is right at this second. For simplicity's sake, we'll assume that at that second, there are no cars, busses, planes, trains, whatever in motion at the time.
One of the major things that will happen will be fires. Probably tens of thousands of them, everywhere people are, caused by something sparkling natural gas lines, gasoline storage, oil containers, unmaintained chemical processes, etc.
We can assume power will fail within a few days I think, since there's no-one to repair things or monitor processes or undo failsafe shutdowns, but until then you'll have falling power lines and innumerable powered items that will spark fires. If a major building fire is in progress, it will probably spread a great deal before it burns itself out. Some might not do that, if they reach chemical plants, oil refineries and the like. Depending on the time of year and the amount of rainfall, vast amounts of forest and acreage will burn uncontrolled and that will take care of a lot of the suburbs in places.
Fire should over, say, fifty years, take care of a lot of the city buildings since many that ordinarily would stand the test of time will be gutted, then fall over or fall in depending on the exact means of construction. Much of the incidental man-made pollution will also be taken care of since much of the styrofoam, rubber and plastic will also burn. It's residue will likely remain poisonous for decades.
Some cities will eventually vanish, since they require man's assistance to continue; New Orleans is probably a good example of this. The netherlands, probably. Anything below sea level that requires a wall or dam to keep back the sea can expect to flood within 50 years or so, I'd think, depending on the amount of wear from storms.
Flooding will also occur, further wiping out or burying many traces of man; without maintenance or oversight, I'm sure many of the larger dams will eventually burst. Flood control will be a thing of the past. Rivers will sweep away parts of cities and bury them in silt.
Storms will do tremendous damage. Many cities on the coast will suffer a great deal of wear and tear every year, depending on what the hurricane season is like. Tornadoes in the south and midwest will render some buildings down to splinters every year.
I don't know what the long term effects of areas that are currently built on inadequately-created landfill, but I'm guessing it's not good. Let's say that 200 years from now, everything built on landfill has sunken or fallen back into the sea.
Some areas will be rendered unlivable forever as nuclear power plants deteriorate and release waste (I doubt the pile will be exposed; unchecked, some might create a China Syndrome but some will probably just lie there dormant until some poor animal or human successor manages to crack open the unit), chemical waste disposal units crack and vent poisons into the water table, etc.
Real city-leveling earthquakes are very rare; the couple that might occur within, say, 200 years, will take care of leveling and burning some West Coast cities.
Many roads will be wide trails within two hundred years or so, as plants and weather effects crack and break apart the upper layers; poorly maintained roads deteriorate quickly. (but not as quickly as now in some cases; they won't be subjected to multiton stresses every few minutes). Many roadways in states that have regular deep winters probably will not fare as well. Bridges and overpasses will eventually fall from ice expanding and cracking the seals over and over again, a little bit more each year. Same with some buildings, once the glass breaks from shifting and the rain can get inside.
Many animals will die, of course. Many will revert to the wild. Some zoo animals will escape and breed, though it seems like many will simply die in their cages and moated areas. Some will get loose, though, and may or may not spread through the area depending on their dietary needs (I seem to remember that many zoo animals need some pretty specific things in their diet?), and the number that get loose. Most will probably die back after a few generations since there won't be enough, say, chimpanzees in an area to maintain needed genetic diversity. Some may find niches left wide open, and they will fit right in and adapt.
(I dunno, though; there are quasi-tame monkeys in Florida that got loose from movie shoots that seem to have done well for themselves; surely several monkey generations have passed since the Fifties and Sixties).