I'm not saying it's a great policy, but it makes sense to me.
Running a whites-only business overlaps but is different from being racist.
In this scenario, it would be weird to force your business to associate with whites-only businesses. But those are illegal, so you're not forced to do that. You're only forced to work with racists that don't run whites-only businesses.
Yes, I suppose there is a logically consistent space there, but not one that seems either appealing or consistent with the way the legal lines have been drawn in the US. For example, as it is not illegal in the US to express hatred for people of a particular ethnicity, should bakeries therefore be required to bake 'Ethnicity X sucks' cakes? That seems both absurd and inconsistent with the eventual outcome of the Masterpiece Cakeshop case.
My overall point here is that it does not make much sense to object to the freedom of association arguments being made in this thread on the grounds that similar arguments were used to support discrimination against Black Americans prior to the Civil Rights Act. The Civil Rights Act recognized racism as a unique evil that required special legal remedy; it did not recognize racists as requiring special legal protection. So, yes, freedom of association permits you to avoid associating with racists but does not (in all cases) permit you to avoid associating with people of a particular ethnicity.
Running a whites-only business overlaps but is different from being racist.
In this scenario, it would be weird to force your business to associate with whites-only businesses. But those are illegal, so you're not forced to do that. You're only forced to work with racists that don't run whites-only businesses.