It seems as though two different things are being conflated, here.

Permitting discrimination against transgender people, based on secular laws and statutes, is clearly transphobia.

Being willing/unwilling to date somebody else based on their gender identity is NOT transphobia. It's sexual/bodily sovereignty.