Point by point take on this:
Banish talk like "There is absolutely no evidence for that belief". But anecdotal evidence is evidence, and it ought to sway my beliefs.
The evidence can be weak enough and/or be evidence for an immense number of other things besides what evidence claims it is evidence for, as to be impossible to process qualitatively. If there's "no evidence", the effect size is usually pretty small, much smaller than the filtering that the anecdotes pass through, much smaller than can be inferred qualitatively, etc.
2.
Banish talk like "I don't know anything about that".
There's great many things that you have never even thought of, and you know nothing about those things. They have no probabilities assigned, and worse, they work as if they had probability of zero. And you can't avoid that, because there's far more things you ought to enumerate than you can enumerate, by a very very huge factor.
Having heard of something leads to quite non-Bayesian change of the belief (effective zero to non-zero). In light of this, degrees of beliefs are not probabilities, but some sort of tag values attached to the propositions that were considered (a very small subset of the totality of propositions), tags which need to be processed in a manner as to arrive at most accurate choices in the end despite absence of processing of the vast majority of relevant beliefs. (A manner which does resemble probability theory to some extent)
Treating them more like probabilities will just lead to a larger final error, even though superficially the edit distance from your algorithm to some basic understanding of probability theory can seem smaller.
3.
Banish talk of "thresholds of belief".
Imposing thresholds on both beliefs and evidence in support of the beliefs allows you to compensate for and decrease the consequences of the unavoidable errors described above. The thresholds have been established after a very long history of inferring some completely wrong conclusions based on accumulation of evidence that was weaker than what can be usefully processed qualitatively but instead requires very accurate set up and quantitative calculations.
4.
Absence of evidence is evidence of absence.
Sometimes, and sometimes it's really weak evidence that isn't statistically independent from the belief that it ought to affect.
5.
Many bits of "common sense" rationality can be precisely stated and easily proved within the austere framework of Bayesian probability.
But you threw away those that can not be easily demonstrated.
6.
You cannot expect[2] that future evidence will sway you in a particular direction.
Theorems of probability are not going to hold exactly for the optimum value that should be assigned to the beliefs in the light of what's described in 2, and working as ff they do hold can not be expected to improve outcomes.
Keep in mind that you can reasonably expect that in the future great many things that you have never thought of may be brought to your attention, without being able to actually enumerate and process a significant fraction of them right now and then.
edit: improved that some. Also, many of those limitations would hold for any physically plausible Jupiter Brains, Matroshka Brains, or other such giant objects which, while they can process great many more beliefs than you can, are still stuck with processing only a minuscule fraction of the beliefs they ought to process.
edit2: interestingly, David Chapman touches on much same points.
There's great many things that you have never even thought of, and you know nothing about those things. They have no probabilities assigned, and worse, they work as if they had probability of zero. And you can't avoid that, because there's far more things you ought to enumerate than you can enumerate, by a very very huge factor.
You don't need to enumerate beliefs to assign them nonzero probability. You can have a catch-all "stuff nothing like anything that'd ever even occur to me, unless it smacked me in the face" category, to which you can assign nonzero probability.
David Chapman criticizes "pop Bayesianism" as just common-sense rationality dressed up as intimidating math[1]:
What does Bayes's formula have to teach us about how to do epistemology, beyond obvious things like "never be absolutely certain; update your credences when you see new evidence"?
I list below some of the specific things that I learned from Bayesianism. Some of these are examples of mistakes I'd made that Bayesianism corrected. Others are things that I just hadn't thought about explicitly before encountering Bayesianism, but which now seem important to me.
I'm interested in hearing what other people here would put on their own lists of things Bayesianism taught them. (Different people would make different lists, depending on how they had already thought about epistemology when they first encountered "pop Bayesianism".)
I'm interested especially in those lessons that you think followed more-or-less directly from taking Bayesianism seriously as a normative epistemology (plus maybe the idea of making decisions based on expected utility). The LW memeplex contains many other valuable lessons (e.g., avoid the mind-projection fallacy, be mindful of inferential gaps, the MW interpretation of QM has a lot going for it, decision theory should take into account "logical causation", etc.). However, these seem further afield or more speculative than what I think of as "bare-bones Bayesianism".
So, without further ado, here are some things that Bayesianism taught me.
What items would you put on your list?
ETA: ChrisHallquist's post Bayesianism for Humans lists other "directly applicable corollaries to Bayesianism".
[1] See also Yvain's reaction to David Chapman's criticisms.
[2] ETA: My wording here is potentially misleading. See this comment thread.