This is going to come down to Justice Kennedy. Yes, Kennedy wrote the majority opinion in the gay marriage case (Obergefell v. Hodges), but his opinion also contained the following caveat: "[T]hose who adhere to religious doctrines may continue to advocate with utmost, sincere conviction that, by divine precepts, same-sex marriage should not be condoned,” and he made clear that they are protected in this mission by the First Amendment.
Note that this is NOT an issue of federal law. The issue is whether the Colorado law under which the baker was punished violates the Constitution. There is no federal law prohibiting the baker from doing what he did. Laws that classify on the basis of sexual orientation appear to receive heightened (intermediate, specifically) scrutiny by SCOTUS, although this is not clearly settled. (Ironically, this was first introduced by a case that invalidated a provision in the Colorado constitution that denied certain benefits to gays). So Colorado will need to show that the law furthers an "important" government interest by means that are "substantially related" to that interest.
But of course the Supremacy Clause in the Constitution means that the Colorado law cannot conflict with established federal law, in any case. And part of the punishment levied against the baker required that he train his staff on how to comply with the law, meaning he could be forced to instruct his staff on how his deeply held religious beliefs are discriminatory. The problem here is that the First Amendment enjoys special solicitude in the constellation of constitutional freedoms, and it could be argued that this amounts to "compelled speech" or "viewpoint discrimination," both of which are mighty oaks and big
's in free speech jurisprudence. Add in the sincerely held belief element in this case, and I think there's a path forward for Justice Kennedy to side with the baker. I think it could be possible to invalidate that portion of the Colorado law and reverse the judgment, with further proceedings possible. The sincerely held belief element has well-established doctrine that should prevent the type of arbitrary hypos that have been suggested in this thread and elsewhere. There must be a real and long-held tradition of adherence to the idea by a real and long-existing religious group.