I'd argue that there's too much of a lopsided emphasis on all of the privileges/advantage/benefits related to being White and/or male...and not enough discussion related to additional intersectional factors.

Therefore, when people hear "privilege" -- they envision an accusation that they're somehow inherently undeserving of success...or they ruminate about all of the folks around them whom they resent.

To me, that means we need to change the connotation of the word "privilege" itself.