So you do not believe that others' beliefs are evidence?
It's sometimes (or even very often) evidence, but not when (1) there's not even a shred of evidence elsewhere, and (2) there's a convincing, systematic explanation for how a particular cluster of epistemic vulnerabilities in human brain hardware led to its widespread adoption.
In other words, a large portion of society believing something is evidence only if the memetic market test for the adoption of the idea at hand is intact. But our hardware and factory settings are so ridiculously mal-adapted to the epistemic environment of the modern world that this market test is extremely often utterly broken and useless.
If you want to make use of the societal thoughts on an issue, you must first appraise the health of the market test for the adoption of the ideas. Is it likely that competition in this area of the memetic environment will lead to ever more sound beliefs, or is there a wrench in the system that is bound to lead to a systematic spiral to ever more ridiculous or counterproductive dogmas?
Our hardware is just so riddled with epistemic problems that it would be a huge mistake to consider societal conclusions at face value. If the market test for meme propagation were intact, and the trial-and-error system for weeding out less useful beliefs in favor of more useful ones ran smoothly, large-scale acceptance of a position would of course be plenty of evidence--no further questions asked.
But we live in a different world--one where this trial-and-error system is in utter disrepair in an absolutely staggering number of cases. In such a world, one must always start with the question, "Is the memetic market test intact in this case, or must I go this epistemic journey myself?"
Of course the market test is better or worse from one place to the next, and I hang out here because the Less Wrong community certainly has one of the best belief propagation systems out there. If everybody on here seems to believe something with a lot of conviction, that to me is strong evidence.
In case the point was lost in the length, I should state it concisely. Whether the beliefs of others are evidence is a contextual question. It depends what the market test is, specifically for the propagation of the belief under scrutiny. If there's reason to believe that the market test is corrupted because of a particular hardware or software vulnerability, then there's reason to dismiss the widespread acceptance, and declare it no evidence at all.
If you accept all that, this of course brings us to the all-important question of why I think the memetic market test for the propagation of religion is broken enough to explain such widespread adoption despite how epistemically insane I consider it, but I don't think I need to (try to) answer that. You've probably heard it all before on here, in writing on memetics, from Dawkins, etc.
I more or less accept your reasoning as far as it goes, but:
our hardware and factory settings are so ridiculously mal-adapted to the epistemic environment of the modern world that this market test is extremely often utterly broken and useless.
If this is true, then why have so much confidence in your own personal appraisal of who to trust and who to write off as deluded? It is of course true that nearly everyone believes what they do for non-truth-tracking reasons, but "nearly everyone" isn't everyone, and there are many people, both theist an...
Are there any essays anywhere that go in depth about scenarios where AIs become somewhat recursive/general in that they can write functioning code to solve diverse problems, but the AI reflection problem remains unsolved and thus limits the depth of recursion attainable by the AIs? Let's provisionally call such general but reflection-limited AIs semi-general AIs, or SGAIs. SGAIs might be of roughly smart-animal-level intelligence, e.g. have rudimentary communication/negotiation abilities and some level of ability to formulate narrowish plans of the sort that don't leave them susceptible to Pascalian self-destruction or wireheading or the like.
At first blush, this scenario strikes me as Bad; AIs could take over all computers connected to the internet, totally messing stuff up as their goals/subgoals mutate and adapt to circumvent wireheading selection pressures, without being able to reach general intelligence. AIs might or might not cooperate with humans in such a scenario. I imagine any detailed existing literature on this subject would focus on computer security and intelligent computer "viruses"; does such literature exist, anywhere?
I have various questions about this scenario, including: