Like Harry Reid, Bill Clinton is in a little trouble. During the 2008 campaign, Clinton told Senator Kennedy "A few years ago, this guy would have been getting us coffee."
In the abstract, the phrase is not racially charged. Clinton may have, and probably has, used the phrase countless times to describe someone who has risen quickly, and, he thinks, undeservedly. That's often unfair, but not racially charged. But when he used the phrase about someone whose father is African, it is considered to be such. Not sure if that's right or wrong, or good or bad, but that's how it has played out, and how most pundits, and politicians, see it.