It depends what you mean by "Hispanic" -- how much of their heritage and culture comes from Latin America rather than Europe. You're right -- it *is* complicated!

Many Jews are White, but there are also Jews of Color. Most Italians (residentiallty) are White, but there are also Italians of Color (usually with mixed North African and Middle Eastern heritage in their genetics). People from Morocco are largely Brown (Arabic/Middle Eastern and African), but there are White Moroccans.

We could do "color swatch" tests against every individual's skin...but who would be the arbiter of who stays and who goes?