I fall into the camp of them being mostly inconsequential. For one thing, there's no overt mention of how many experience points a user has. Sure, you can hover over the green box, but I have to wonder how many people actually do that. On the other hand, I mostly don't use the XP system, so perhaps that's my own bias on the whole thing showing through.
My second thought is that, despite XP, I can still tell who among the board posters is not worth listening to (no names). On the flip side, I generally read all of the posts in a thread, and posts are judged on the merit of whether or not they contribute something useful. There are, of course, posters who I note tend to post things interesting to me, and I know who they are (again, no names). I suppose my natural tendency is to judge a post by its own merit, and not that of who posted it.
While I thought it was a bad idea, and I'm mostly indifferent to it now, there are a couple of downsides. Like StreamOfTheSky, I would be very surprised if there were no "cliques" that emerged from analyzing the XP given/received. I think the XP system is most likely used as a Digg-style "I agree" button, versus an actual receptions to a post. That lends itself to precipitating out groups of like-minded posters. I think it would be even more pronounced if there were a negative-XP button.
I also have a theory that XP is directly related to post count. At first blush, that seems like a dumb statement; of course people who post more will have more XP, because they have more chances to receive it (ie, have posted more quality posts). I don't think that is the reason though. My suspicion is that XP in reality has nothing to with the inherent quality of posts and everything to do with post count. It could possibly be due to posters basically giving XP randomly (due to XP giving limitations). I'm not sure as to the why because we don't have any data to analyze.