In the Alice and Bob example aren't the rules of the game part of the natural latent? Like Carol also needs to remember that neither Alice nor Bob will draw any [color that isn't purple] lines and this information isn't in the green lines.
I think there is a misunderstanding as what counts as an observation. I think your point of view is something like "Yesterday I experienced a universe consistent with the laws of physics, and the day before that I experienced a universe consistent with the laws of physics and so on. I'm about 30 years old so that's 10,950 observations" to which I ask "If you have all that knowledge, what did you have for dinner 78 days ago? Was it consistent with the laws of physics?" to make the rhetorical point that a boltsman brain does not need to fake any history, it only needs to fake your current memories of history, a much easier task. Boltzman brains are very similar to Last Thursdayism.
I think people are just choosing comparison sets wrong? Using the bag anology, there are two pieces of paper labeled "6" so upon being handed a random piece of paper labeled "6" I should be 50/50 as to whether it came from the first bag or the second bag, No?