Also, I wonder if the culture surrounding sexual orientations and gender identities has a lot to do with it. Most of my gay male friends (and sometimes my bi male friends) feel similarly to how I do, when it comes to unsolicited comments on our weight -- we don't want the comments made!
Straight male friends, on the other hand, more often tell me I shouldn't find a reason to be offended and I should take it as the compliment it was intended to be.
The fact that I'm autistic also adds an extra layer of insecurity and self-obsession to this dynamic. I feel that most heterosexual autistic men would be able to relate to that, even though they don't share my sexual orientation.