I think the main factor should be the level of consistency of the business owner's discretion.

If they turned away a customer just because they overheard the customer say they are gay, *that* should be illegal. If they refuse to bake a cake for a same-sex wedding because they don't do specialty fare based on their "morals" (however fucked-up I might think their "morals" are), that's denying a unique privilege (rather than a basic service). Legally, discretion over range of privilege's should be at the business owner's judgment.