First off, it seems weird to say "I have 33 bits of evidence that this bit is a 1."
It seems weird to me because the bits of "33 bits" looks like the same units as the bit of "this bit", but they aren't the same. Map/territory. From now on, I'm calling the first, A-bits, and the second, B-bits.
Why does it take an infinite number of bits of evidence to get 1 bit of information?
It takes an infinite number of A-bits to know with absolute certainty one B-bit.
But that means a one-bit evidence-giver is someone who is right 2/3 of the time. Why the 2/3? That seems weird.
What were you expecting?
If it's worth saying, but not worth its own post (even in Discussion), then it goes here.
Notes for future OT posters:
1. Please add the 'open_thread' tag.
2. Check if there is an active Open Thread before posting a new one. (Immediately before; refresh the list-of-threads page before posting.)
3. Open Threads should be posted in Discussion, and not Main.
4. Open Threads should start on Monday, and end on Sunday.