There's another element to it: many men restrain ourselves in how we outwardly react to something *because* there's a stigma that if we show emotion and passion about the "wrong" thing, then we'll be sissy-shamed or feminized by society.
It's all definitely rooted in patriarchy and misogyny. But it hits various people from many different angles.
The question shouldn't be why people "let" society bother us (and disenchant us) with these gendered roles/expectations...it should be why we fail to claim space and speak our truths while also failing to stand up for others who do the same.