I guess I'm skeptical of most "studies" because I have no context for the researchers' biases going into it.

It would be premature to discount ALL biological factors when comparing the brains of male and female specimens. But, while you're right about the lack of ethics when it comes to experimenting on children in terms of brain research, that lack of data also hinders our ability to fully understand the adult brain.