Regarding the people who accused you of being "racist" for capitalizing both -- they are wrong.

You've name-dropped some specific individuals here on Medium, and I'll let them speak for themselves. Yes, there is a cross-section of White people who choose to take offense to *ANY* usage of the term "White," even if it's just being used as a neutral descriptor. I would agree that those people, in those cases, are textbook examples of white fragility.

I wouldn't capitalize "olive-skinned" (or "fair-skinned") -- but I *would* capitalize "Brown" when using it as a blanket reference to someone whose racial & ethnic background gives them olive skin and comes from a culture that has been targeted by white supremacy.

I still haven't heard any logical or rational reason why capitalizing "Black" necessitates keeping "White" in all lowercase. Yes, "White" can be used as a blanket reference to people of European descent as a whole. The fact that Black people are unable to trace their lineage as far back as White people, due to the abomination of slavery between the African continent and the Western Hemisphere -- that's a consequence of the white supremacy upon which the United States was founded and built.

One doesn't "cancel out" or "negate" the other.