As a gay guy who never intends to raise children, I'd never considered this before (since it's probably never something I'll have to think about) -- but I agree with much of your perspective.
I do wonder, though, why "Brothers Day" (May 24), "Sisters Day" (Aug. 1), "Aunts & Uncles Day" (July 26), "Cousins Day" (July 24), "Grandparents Day" (Sep. 12), "Daughters Day" (Sep. 26), and "Sons Day" (Sep. 28) aren't celebrated more prominently? (Yes, I had to Google all of these dates). I also wonder how nonbinary and genderfluid people decide which, if any, holiday to celebrate when there's a reference within the holiday implying biological sex?