Just wait until you get called a Nazi and punched by some clown who disagrees with something you said. What will your defense be then? “Oh, but he was wrong about me being a Nazi, I’m not and only I get to decide who is and isn’t a Nazi and who should be punched.”
Some people are too simple minded to actually examine the larger consequences of their personal stance on the acceptability of responding to speech with physical violence. Hint: you and people who think like you aren’t going to be the only ones doing the zero warning sucker punching. Is that really what you want?

That’s a bit like saying we shouldn’t send guilty people to prison, because that might be used against the innocent.

Do you fail to comprehend the difference between justice and vigilantism? Justice is an open process where the accused gets a defense. Vigilantism not just Nazi-punching, it’s also what George Zimmerman did to Trayvon Martin.

Exactly what defense do you expect Spencer to offer? “When I advocated for the mass extermination of blacks and Jews, I didn’t intend to do it myself”?
Come back to me when you understand the difference between personal ethics and government law.

That way when the thugs and brownshirts come around to beat up some lefties they think are anti-American commies and just as deserving of a beat down as this scum, we can say with honesty “We’ve always been against violent responses to free speech.”

You can put that right on your tombstone.