One small irritation of having AIM forward to my phone is having the message buffer fill up and then get some IM with breathless news that I definitely want to reply to, only to find that she's long since logged off AIM, which I can't tell until my reply bounces. The message buffer fills up fast, because people using IM often send multiple messages to the phone in a single thought--that's just the way people talk on IM--and each gets forwarded as a single SMS.
At this point, AIM, Twitter, LJ, GrandCentral, among other services, all point at my phone. The lag is often noticeable, depending on the service.
As I said, it's a small irritation, directed at the technology, not at all at those trying to reach me.