This seems exaggerated to me (although I agree that the tendencies you mention exist and are significant). Consider, for example, the famous incident in which the American Civil Liberties Union defended a Nazi group's right to demonstrate. The ACLU was and is a non-fringe organization, and likewise, the position that "Nazism is despicable, but freedom of speech and assembly is more important than silencing Nazis" is reasonably mainstream in the United States.

There's not much danger of the Nazis convincing non-negligible numbers of people, so this is a cheap way to signal one's support for freedom of speech. Call me when the ACLU is interested in protecting the right of pro-life groups to demonstrate outside abortion clinics.

Similarly, the idea that "Drug use is bad, but marijuana should be legal" seems not-uncontroversial but also non-fringey.

My understanding is that most of these people actually hold the position that "use of certain drugs is bad, but marijuana isn't one of those drugs".