My only issue with the letter is that #cancelling someone is a form of speech. That's how it's supposed to work. Person A says something, society analyzes it and reacts accordingly -- whether neutrally, in support, or in opposition. I really don't have any problem with people calling out or canceling or whatever. That's part of Speech too, baby.
Seen in the best light, the letter is a call for more nuanced thought and consideration of how we oppose ideas we disagree with. Which I think is a great and smart thing to consider, but keeping in mind that sometimes, reprehensible ideas don't deserve much nuance or thought or discussion.