Freedom of speech means you can't arrest Nazis until they actually start killing Jews, for example

In point of fact, Nazis started threatening and assaulting Jews, vandalizing their businesses, and imposing weird new discriminatory rules on them, some years before the mass murder started in earnest. None of the above are generally taken to be protected by "freedom of speech".