Like with any group, the reason for that anger is diverse.

With some, like you've pointed out, it's because they fear diversity and multiculturalism.

Others don't want to share power/privilege.

Others seek to be outright dominant and authoritarian, controlling the lives of others.

Others don't want to admit that the political elite have lied to them for so many decades.

Others view it as people lecturing them via various iterations of "Count Your Blessings!" whenever they bring up an issue related to adversity specifically within the context of their own life.

Others fear that the asymmetrical acceptance of double standards and hypocrisy will become normalized.

For me, I identify with the latter two fears. However, my preferred remedy/solution is to lean into how society can be more inclusive across the board...rather than justifying systems and mechanisms of perpetuated exclusion.