There's a popular JPG meme someone created online, that shows two young women of color (one with a light brown complexion, and the other whose complexion is of a darker pigment) standing next to each other...one of them is facing the camera, and the other is facing away from it.
The meme reads: "I see color -- and I love every color I see."
One girl has the first half of that slogan printed on the front of her sweatshirt; and the other girl has the latter half of that slogan printed on the back of her sweatshirt.
That's why I refrain from using the phrase "I don't see color" -- because it can be interpreted in two different ways that, as you've pointed out, meaning totally different things.
When I see someone who is Black, Indigenous, or a Person of Color -- I assume they've probably faced racial oppression in their lives, but I reserve further analysis until I've actually had a conversation with them (or read about their personal experiences, as written from their P.O.V.). I'll admit I don't necessarily make that same assumption when I see a White person -- but I *DON'T* automatically rule it out as a possibility, from the onset.
In fact, I have a very curious mind where I constantly wonder what individuals' ethnic backgrounds happen to be, especially when the person has a skin tone that's olive, tan, or light brown. Obviously, I don't ASK them that (especially not if they're a complete stranger), but I do enjoy learning about it.