generically speaking, I'd keep the google apps since you are running a Google OS.
The carriers apps, or the extra crap they put on there, if it'll let you whack it, sure. I find my Verizon android has games and other crap I don't want that it will NOT let me delete.
At that point, you'd be in the realm of rooting your android and starting with a fresh OS built from Google.
That's one thing I like about the AT&T phones that I've seen in the past. In the pre-smartphone era, my Verizon Razor was handicapped and smeared with crap from Verizon. My friend's AT&T Razor was not. When I went from Razor to AT&T iPhone 3g, I had a nice clean phone with no Carrrier Crap on it.
I don't know what ATT does with Androids, but I really don't like Carrier Crap.