= 27d567bc2d48d9f40e9ccc20ea370b15)
Open Thread: July 2010
This thread is for the discussion of Less Wrong topics that have not appeared in recent posts. If a discussion gets unwieldy, celebrate by turning it into a top-level post.
= 783df68a0f980790206b9ea87794c5b6)
Loading…= b715d02e85f7b3b4ae65ca3d3e450868)
Subscribe to RSS Feed
Powered by Reddit
= f037147d6e6c911a85753b9abdedda8d)
This thread is for the discussion of Less Wrong topics that have not appeared in recent posts. If a discussion gets unwieldy, celebrate by turning it into a top-level post.
Comments (653)
Okay, here's something that could grow into an article, but it's just rambling at this point. I was planning this as a prelude to my ever-delayed "Explain yourself!" article, since it eases into some of the related social issues. Please tell me what you would want me to elaborate on given what I have so far.
Title: On Mechanizing Science (Epistemology?)
"Silas, there is no Bayesian ‘revival’ in science. There is one amongst people who wish to reduce science to a mechanical procedure." – Gene Callahan
“It is not possible … to construct a system of thought that improves on common sense. … The great enemy of the reservationist is the automatist[,] who believes he can reduce or transcend reason. … And the most pernicious [of them] are algorithmists, who believe they have some universal algorithm which is a drop-in replacement for any and all cogitation.” – "Mencius Moldbug"
And I say: What?
Forget about the issue of how many Bayesians are out there – I’m interested in the other claim. There are two ways to read it, and I express those views here (with a bit of exaggeration):
View 1: “Trying to come up with a mechanical procedure for acquiring knowledge is futile, so you are foolish to pursue this approach. The remaining mysterious aspects of nature are so complex you will inevitably require a human to continually intervene to ‘tweak’ the procedure based on human judgment, making it no mechanical procedure at all.”
View 2: “How dare, how dare those people try to mechanize science! I want science to be about what my elite little cadre has collectively decided is real science. We want to exercise our own discretion, and we’re not going to let some Young Turk outsiders upstage us with their theories. They don’t ‘get’ real science. Real science is about humans, yes, humans making wise, reasoned judgments, in a social context, where expertise is recognized and a rewarded. A machine necessarily cannot do that, so don’t even try.”
View 1, I find respectable, even as I disagree with it.
View 2, I hold in utter contempt.
You probably won't find much opposition to your opinion here on LW. Duh, of course science can and will be automated! It's pretty amusing that the thesis of Cosma Shalizi, an outspoken anti-Bayesian, deals with automated extraction of causal architecture from observed behavior of systems. (If you enjoy math, read it all; it's very eye-opening.)
Really? I read enough of that thesis to add it to the pile of "papers about fully generally learning programs with no practical use or insight into general intelligence".
Though I did get one useful insight from Shalizi's thesis: that I should judge complexity by the program length needed to produce something functionally equivalent, not something exactly identical, as that metric makes more sense when judging complexity as it pertains to real-world systems and their entropy.
And regarding your other point, I'm sure people agree with holding view 2 in contempt. But what about the more general question of mechanizing epistemology?
Also, would people be interested in a study of what actually does motivate opposition to the attempt to mechanize science? (i.e. one that goes beyond my rants and researches it)
How hard do you think mechanizing science would be? It strikes me as being at least in the same class with natural language.
I've been poking at the question of to what extent computers could help people do science, beyond the usual calculation and visualization which is already being done.
I'm not getting very far-- a lot of the most interesting stuff seems like getting meaning out of noise.
However, could computers check to make sure that the use of statistics isn't too awful? Or is finding out whether what's deduced follows from the raw data too much like doing natural language? What about finding similar patterns in different fields? Possibly promising areas which haven't been explored?
Not exactly sure, to be honest, though your estimate sounds correct. What matters is that I deem it possible in a non-trivial sense; and more importantly, that we can currently identify rough boundaries of ideal mechanized science, and can categorize much of existing science as being definitely in or out.
That it should be possible to Algorithmize Science seems clear from that the human brain can do science and the human brain should be possible to describe algorthmically. If not at a higher level, so at least -- in principle -- by quantum electrodynamics which is the (known and computable in principle) dynamics of electrons and nuclei that are the building blocks of the brain.( If it should be possible to do in practice it would have to be done at a higher level but as a proof of principle that argument should be enough.)
I guess, however, that what is actually meant is if the scientific method itself could be formalised (algorithmized), so that science could be "mechanized" in a more direct way than building human-level AIs and then let them learn and do science by the somewhat informal process used today by human scientists. That seems plausible. But has still to be done and seems rather difficult. The philosophers of science is working on understanding the scientific process better and better, but they seem still to have a long way to go before an actually working algorithmic description has been achieved. See also the discussion below on the recent article by Gelman and Shalizi criticizing bayesianism.
EDIT "done at a lower level" changed to "done at a higher level"
The scientific method is already a vague sort of algorithm, and I can see how it might be possible to mechanize many of the steps. The part that seems AGI-hard to me is the process of generating good hypotheses. Humans are incredibly good at plucking out reasonable hypotheses from the infinite search space that is available; that we are so very often says more of the difficulty of the problem than our own abilities.
I'm pretty sure that judging whether one has adequately tested a hypothesis is also going to be very hard to mechanize.
The problem that I hear most often in regard to mechanizing this process has the basic form, "Obviously, you need a human in the loop because of all the cases where you need to be able to recognize that a correlation is spurious, and thus to ignore it, and that comes from having good background knowledge."
But you have to wonder: the human didn't learn how to recognize spurious correlations through magic. So however they came up with that capability should be some identifiable process.
Maybe it's just a matter of people kidding themselves about how hard it is to explain something.
On the other hand, some things (like vision and natural language) are genuinely hard to figure out.
I'm not saying the problem is insoluble. I'm saying it looks very difficult.
Those people should be glad they've never heard of TETRAD - their heads might have exploded!
That's intriguing. Has it turned out to be useful?
One possible way to get started is to do what the 'Distilling Free-Form Natural Laws from Experimental Data' project did: feed measurements of time and other variables of interest into a computer program which uses a genetic algorithm to build functions that best represent one variable as a function of itself and the other variables. The Science article is paywalled but available elsewhere. (See also this bunch of presentation slides.)
They also have software for you to do this at home.
I think there is an additional interpretation that you're not taking into account, and an eminently reasonable one.
First, to clarify the easy question: unless you believe that there is something mysteriously uncomputable going on in the human brain, the question of whether science can be automated in principle is trivial. Obviously, all you'd need to do is to program a sufficiently sophisticated AI, and it will do automated science. That much is clear.
However, the more important question is -- what about our present abilities to automate science? By this I mean both the hypothetical methods we could try and the ones that have actually been tried in practice. Here, at the very least, a strong case can be made that the 20th century attempt to transform science into a bureaucratic enterprise that operates according to formal, automated procedures has largely been a failure. It has undoubtedly produced an endless stream of cargo-cult science that satisfies all these formal bureaucratic procedures, but is nevertheless worthless -- or worse. At the same time, it's unclear how much valid science is coming out except for those scientists who have maintained a high degree of purely informal and private enthusiasm for discovering truth (and perhaps also those in highly practical applied fields where the cash worth of innovations provides a stringent reality check).
This is how I read Moldbug: in many important questions, we can only admit honestly that we still have no way to find answers backed by scientific evidence in any meaningful sense of the term, and we have to grapple with less reliable forms of reasoning. Yet, there is the widespread idea that if only the proper formal bureaucratic structures are established, we can get "science" to give us answers about whichever questions we find interesting, and we should guide our lives and policies according to the results of such "science." It's not hard to see how this situation can give birth to a diabolical network of perverse incentives, producing endless reams of cargo-cult scientific work published by prestigious outlets and venerated as "science" by the general public and the government.
The really scary prospect is that our system of government might lead us to a complete disaster guided by policy prescriptions coming from this perverted system that has, arguably, already become its integral part.
Okay, thanks, that tells me what I was looking for: clarification of what it is I'm trying to refute, and what substantive reasons I have to disagree.
So "Moldbug" is pointing out that the attempt to make science into an algorithm has produced a lot of stuff that's worthless but adheres to the algorithm, and we can see this with common sense, however less accurate it might be.
The point I would make in response (and elaborate on in the upcoming article), is that this is no excuse not to look inside the black box that we call common sense and understand why it works, and what about it could be improved, while the Moldbug view asks that we not do it. Like E. T. Jaynes says in chapter 1 of PLoS, the question we should ask is, if we were going to make a robot that infers everything we should infer, what constraints would we place on it?
This exercise is not just some attempt to make robots "as good as humans"; rather, it reveals why that-which-we-call "common sense" works in the first place, and exposes more general principles of superior inference.
In short, I claim that we can have Level 3 understanding of our own common sense. That, contra Moldbug, we can go beyond just being able to produce its output (Level 1), but also know why we regard certain things as common sense but not others, and be able to explain why it works, for what domains, and why and where it doesn't work.
This could lead to a good article.
I read Moldbug's quote as saying: there is currently no system, algorithmic or bureaucratic, that is even remotely close to the power of human intuition, common sense, genius, etc. But there are people who implicitly claim they have such a system, and those people are dangerous liars.
Those quotes do seem to be in conflict, but if he is talking about people that claim they already have the blueprints for such a thing, it would make more sense to read what he is saying as "it is not possible, with our current level of knowledge, to construct a system of thought that improves on common sense". Is he really pushing back against people that say that it is possible to construct such a system (at some far off point in the future), or is he pushing back against people that say they have (already) found such a system?
The Moldbug article that the quote comes from does not seem to be expressing anything much like either Silas' view 1 or view 2. Moldbug clarifies in a comment that he is not making an argument against the possibility of AGI:
That sounds like a justification for view 1. Remember, view 1 doesn't provide a justification for why there will need to be continual tweaks to mechanized reasoners to bring them in line with (more-) human reasoning, so remains agnostic on how exactly one justifies this view.
(Of course, "Moldbug's" view still doesn't seem any more defensible, because it equates a machine virtualizing a human, with a machine virtualizing the critical aspects of reasoning, but whatever.)
Am I the only one who finds this extremely unlikely? So far as I know, Bayesian methods have become massively more popular in science over the last 50 years. (Count JSTOR hits for the word 'Bayesian,' for example, and watch the numbers shoot up over time!)
Half of those hits are in the social sciences. I suspect that is economists defining the rational agents they study as bayesian, but that is rather different from the economists being bayesian themselves! The other half are in math & staticstics is probably that bayesian statisticians are becoming more common, which you might count as science (and 10% are in science proper).
Anyhow, it's clear from the context (I'd have thought from the quote) that he just means that the vast majority of scientists are not interested in defining science precisely.
This is a mostly-shameless plug for the small donation matching scheme I proposed in May:
I'm still looking for three people to cross the "membrane that separates procrastinators and helpers" by donating $60 to the Singularity Institute. If you're interested, see my original comment. I will match your donation.
I'm sorry I didn't see that earlier; I donated $30 to the SIAI yesterday, and I probably could have waited a little while longer and donated $60 all at once. If this offer will still be open in a month or two, I will take you up on it.
That sounds good, and feel free to count your first $30 towards a later $60 total if I haven't found a third person by then.
Done!
Done, 60 USD sent.
Thank you! Matched.
Without any way of authenticating the donations, I find this to be rather silly.
I'd also like these donations to be authenticated, but I'm willing to wait if necessary. Here's step 2, including the new "ETA" part, from my original comment:
Would you be willing to match my third $60 if I could give you better evidence that I actually matched the first two? If so, I'll try to get some.
Has anyone continued to pursue the Craigslist charity idea that was discussed back in February, or did that just fizzle away? With stakes that high and a non-negligible chance of success, it seemed promising enough for some people to devote some serious attention to it.
Thanks for asking! I also really don't want this to fizzle away.
It is still being pursued by myself, Michael Vassar, and Michael GR via back channels rather than what I outlined in that post and it is indeed getting serious attention, but I don't expect us to have meaningful results for at least a year. I will make a Less Wrong post as soon as there is anything the public at large can do -- in the meanwhile, I respectfully ask that you or others do not start your own Craigslist charity group, as it may hurt our efforts at moving forward with this.
ETA: Successfully pulling off this Craigslist thing has big overlaps with solving optimal philanthropy in general.
This seems extremely pertinent for LW: a paper by Andrew Gelman and Cosma Shalizi. Abstract:
I'm still reading it so I don't have anything to say about it, and I'm not very statistics-savvy so I doubt I'll have much to say about it after I read it, but I thought others here would find it an interesting read.
I stole this from a post by mjgeddes over in the OB open thread for July (Aside: mjgeddes, why all the hate? Where's the love, brotha?)
Can anyone with more experience with Bayesian statistics than me evaluate this article?
ETA: Never mind. I got my crackpots confused.
Original text was:
mjgeddes was once publicly dissed by Eliezer Yudkowsky on OB (can't find the link now, but it was a pretty harsh display of contempt). Since then, he has often bashed Bayesian induction, presumably in an effort to undercut EY's world view and thereby hurt EY as badly as he himself was hurt.
You're probably not thinking of this On Geddes.
No, not that. Geddes made a comment on OB about eating a meal with EY during which he made some well-meaning remark about EY becoming more like Geddes as EY grows older, and noticing an expression of contempt (if memory serves) on EY's face. EY's reply on OB made it clear that he had zero esteem for Geddes.
Nope, that was Jef Allbright.
No wonder I couldn't find the link. Yeesh. One of these days I'll learn to notice when I'm confused.
I wrote a backlink to here from OB. I am not yet expert enough to do an evaluation of this. I do think however that it is an important and interesting question that mjgeddes asks. As an active (although at a low level) rationalist I think it is important to try to at least to some extent follow what expert philosophers of science actually find out of how we can obtain reasonably reliable knowledge. The dominating theory of how science proceeds seems to be the hypothetico-deductive model, somewhat informally described. No formalised model for the scientific process seems so far has been able to answer to serious criticism of in the philosophy of science community. "Bayesianism" seems to be a serious candidate for such a formalised model but seems still to be developed further if it should be able to anser all serious criticism. The recent article by Gelman and Shalizi is of course just the latest in a tradition of bayesian-critique. A classic article is Glymour "Why I am Not a Bayesian" (also in the reference list of Gelman and Shalizi). That is from 1980 so probably a lot has happened since then. I myself am not up-to-date with most of development, but it seems to be an import topic to discuss here on Less Wrong that seems to be quite bayesianistically oriented.
steven0461 already posted this to the previous Open Thread and we had a nice little talk.
Yesterday, I posted my thoughts in last month's thread on the article. I'm reproducing them here since this is where the discussion is at:
(Fixed) link to earlier discussion of this paper in the last open thread.
(Edit - that's what I get for posting in this thread without refreshing the page. cousin_it already linked it.)
Paul Graham has written extensively on Startups and what is required. A highly focused team of 2-4 founders, who must be willing to admit when their business model or product is flawed, yet enthused enough about it to pour their energy into it.
Steve Blank has also written about the Customer Development process which he sees as paralleling the Product Development cycle. The idea is to get empirical feedback.by trying to sell your product from the get-go, as soon as you have something minimal but useful. Then you test it for scalability. Eventually you have strong empirical evidence to present to potential investors, aka "traction".
These strike me as good examples of applied rationality. I wonder what percentage of Less Wrong readers would succeed as startup founders?
I would not deviate too much from the prior (most would fail).
Are you saying that LW readers suck at applied rationality, or are you disagreeing with the idea that applied rationality can help prevent startup failure?
I would say that preventing startup failure requires a whole group of factors, not least of which is good fortune. It is hard for me to judge whether LW are more likely than other people who self select to start start ups to get it all right. I note, for example, that people starting a second startup do not tend to be all that much likely to be successful than on their first attempt!
Suppose we were to test it empirically and 9/10 startups fail on their first attempt. Then test again and 9/10 still fail on second attempt. That is not enough information to determine that a given person would fail 10 times in a row, because it could be that there is some number of failures <10 where you finally acquire enough skill to avoid failure on a more routine basis.
Given the fact that there's a whole world of information, strategies, and skills specific to founding startups, I would be surprised if an average member of a given group of startup founders fails x times out of y when x/y first attempts also fail.
So it would be relevant (especially if you are, say an angel investor) how low the percentage of failures can be brought to with multiple attempts by a given individual, and whether a given kind of education (such as reading Less Wrong sequences, or a quality such as self-selecting to read them) would predispose you to reducing that number of failures more rapidly and/or further in the long run.
I wonder what percentage have ever tried?
That at least partly depends on what you define as a "startup". Graham's idea of one seems to be oriented towards "business that will expand and either be bought out by a major company or become one", vs. "enterprise that builds personal wealth for the founder(s)".
By Graham's criteria, Joel Spolsky's company, Fog Creek, would not have been considered a startup, for example, nor would any business I've ever personally run or been a shareholder of.
[Edit: I should say, "or been a 10%+ shareholder of"; after all, I've held shares in public companies, some of which were undoubtedly startups!]
At the most general, creating your own business (excluding the sort of "contract" status in which the only difference with an employee is in the accounting details) and making a good living from it.
At the most narrow, starting up a business that, as Guy Kawasaki puts it, solves the money problem for the rest of your life.
Maybe a survey would be interesting, either as a thread here or on somewhere like surveymonkey tha would allow anonymous responses. "1. Are you an employee/own your own business/living on a pile of money of your own/a dependent/other? 2. Which of those states would you prefer to be in? 3. If the answers to 1 and 2 are different, are you doing anything about it?"
I can't get back to this until this evening (it is locally 10am as I write). Suggestions welcome.
You need at least one more item in there - "retired", i.e. with passive income that exceeds one's costs of living. Different from "living on a pile of money", insofar as there might still be things you can't afford.
I wonder what percentage are even inclined to try?
I propose that LessWrong should produce a quarterly magazine of its best content.
LessWrong readership has a significant overlap with the readers of Hacker News, a reddit/digg-like community of tech entrepreneurs. So you might be familiar with Hacker Monthly, a print magazine version of Hacker News. The first edition, featuring 16 items that were voted highly on Hacker News, came out in June, and the second came out today. The curator went to significant effort to contact the authors of the various articles and blog posts to include them in the magazine.
Why would we want LessWrong content in a magazine? I personally would find it a great recruitment tool; I could have copies at my house and show/lend/give them to friends. As someone at the Hacker News discussion commented, "It's weird but I remember reading some of these articles on the web but, reading them again in magazine form, they somehow seem much more authoritative and objective. Ah, the perils of framing!"
The publishing and selling part is not too difficult. Hacker Monthly uses MagCloud, a company that makes it easy to publish and sell PDFs into printed magazines.
Unfortunately, I don't have the skills or time to do this myself, at least not in the short-term. If someone wants to pick up this project, major tasks would include creating a process to choose articles for inclusion, contacting the authors for permission, and designing the magazine.
There's also the possibility of advertisements. I personally would be excited to see what kinds of companies would like to advertise to an audience of rationalists. Cryonics companies? Index funds? Rationalist books? Non-profits seeking donations?
Should advertising be used just to defray costs, or could the magazine make money? Make money for whom?
If people think it's a good idea but no-one takes it on, I might have some time free early next year to make this happen. But I hope someone gets to it earlier.
I don't think there's enough content on LW to be worthwhile publishing a magazine. However, Eliezer's book on rationality should offer many of the same benefits.
A yearly anthology would be pretty good, though. HN is reusing others' content and can afford a faster tempo; but that simply means we need to be slower. Monthly is too fast, I suspect that quarterly may be a little too fast unless we lower our standards to include probably wrong but still interesting essays. (I think of "Is cryonics necessary?: Writing yourself into the future" as an example of something I'm sure is wrong, but was still interesting to read.)
How about thirdly!?
This post both made me laugh AND think it was a good idea; I'd love to see a magazine that was more than once a year. There's a bit of discussion of the most recent quarter; if people don't think that it is long enough (or that the pace will continue, or that people will consent to their articles being put in journals) a slight delay should help but a four times delay seems excessive.
Not all of the content needs to be from the most recent quarter. There could be classic articles too. But I think we might have enough content each quarter anyway. Let's see...
There were about 120 posts to Less Wrong from April 1 to June 30. The top ten highest-voted were Diseased thinking: dissolving questions about disease by Yvain, Eight Short Studies On Excuses by Yvain, Ugh Fields by Roko, Bayes Theorem Illustrated by komponisto, Seven Shiny Stories by Alicorn, Ureshiku Naritai by Alicorn, The Psychological Diversity of Mankind by Kaj Sotala, Abnormal Cryonics by Will Newsome, Defeating Ugh Fields In Practice by Psychohistorian, and Applying Behavioral Pscyhology on Myself by John Maxwell IV.
Maybe not all of those are appropriate for a magazine (e.g. Bayes Theorem Illustrated is too long). So maybe swap a couple of them out for other ones. Then maybe add a few classic LessWrong articles (for example, Disguised Queries would make a good companion piece to Diseased Thinking), add a few pages of advertising and maybe some rationality quotes, and you'd have at least 30 pages. I know I'd buy it.
It's not actually all that long; it's just that the diagrams take up a lot of space.
Well, I'd like to keep the diagrams if the article is to be used. I do like Bayes Theorem Illustrated and I think an explanation of Bayes Theorem is perfect content for the magazine. If I were designing the magazine I'd want to try to include it, maybe edited down in length.
Monthly seems too often. Quarterly might work.
There's certainly enough content to do at least one really good issue.
Does anyone else find the idea of creating a printed magazine rather anachronistic?
The rumors of print media's death have been greatly exaggerated.
I was at a recent Alexander Technique workshop, and some of the teachers had been observing how two year olds crawl.
If you've had any experience with two year olds, you know they can cover ground at an astonishing rate.
The thing is, adults typically crawl with their faces perpendicular to the ground, and crawling feels clumsy and unpleasant.
Two year olds crawl with their faces at 45 degrees to the ground, and a gentle curve through their upper backs.
Crawling that way gives access to a surprisingly strong forward impetus.
The relevance to rationality and to akrasia is the implication that if something seems hard, it may be that the preconditions for making it easy haven't been set up.
Long ago I read a book that asked the question “Why is there something rather than nothing?” Contemplating this question, I asked “What if there really is nothing?” Eventually I concluded that there really isn’t – reality is just fiction as seen from the inside.
Much later, I learned that this idea had a name: modal realism. After I read some about David Lewis’s views on the subject, it became clear to me that this was obviously, even trivially, correct, but since all the other worlds are causally unconnected, it doesn't matter at all for day-to-day life. Except as a means of dissolving the initial vexing question, it was pointless, I thought, to dwell on this topic any more.
Later on I learned about the Cold War and the nuclear arms race and the fears of nuclear annihilation. Apparently, people thought this was a very real danger, to the point of building bomb shelters in their backyards. And yet somehow we survived, and not a single bomb was dropped. In light of this, I thought, “What a bunch of hype this all is. You doomsayers cried wolf for decades; why should I worry now?”
But all of that happened before I was born.
If modal realism is correct,* then for all I know there was a nuclear holocaust in most world-lines; it’s just that I never existed there at all. Hence I cannot use the fact of my existence as evidence against the plausibility of existential threats, any more than we can observe life on Earth and thereby conclude that life is common throughout the universe.
(*Even setting aside MWI, which of course only strengthens the point.)
Strange how abstract ideas come back to bite you. So, should I worry now?
If you think doom is very probable and we only survived due to the anthropic principle, then you should expect doom any day now, and every passing day without incident should weaken your faith in the anthropic explanation.
If you think all possible worlds exist, then you should expect our small bubble of ordered existence to erupt into chaos any day now, because way more copies of it are contained in chaotic worlds than in ordered ones. Every day you spend without spontaneously turning into a pheasant should weaken your faith in the multiverse.
(These arguments are not standard LW fare, but I've floated them here before and they seem to work okay.)
Not if you interpret your preference about those worlds as assigning most of them low probability, so that only the ordered ones matter.
To rephrase, "unless you interpret your preference as denying the multiverse hypothesis" :-)
You don't have to assign exactly no value to anything, which makes all structures relevant (to some extent).
This depends on which level of the Tegmark classification you are talking about. Level III for example, quantum MWI, gives very low probabilities for things like turning into a pheasant, since those evens while possible, have tiny chances of occurring. Level IV, the ultimate ensemble, which seems to the main emphasis of the poster above, may have your argument as a valid rebuttal, but since level IV requires consistency, it would require a much better understanding of what consistent rule systems look like. And it may be that the vast majority of those universes don't have observers, so we actually would need to look at consistent rule systems with observers. Without a lot more information, it is very hard to examine the expected probabilities of weird even events in a level IV setting.
Wha? Any sequence of observations can be embedded in a consistent system that "hardcodes" it.
Consistency is about logics, while Tegmark's madness is about mathematical structures. Whenever you can model your own actions (decision-making algorithm) using huge complicated mathematical structures, you can also do so with relatively simple mathematical structures constructed from the syntax of your algorithm (Lowenheim-Skolem type constructions). There is no fact of the matter about whether a given consistent countable first order theory, say, talks about an uncountable model or a countable one.
It's not clear to me that this is correct. Also, even if it is, then coherent memories (like what we're using to judge this whole scenario) only exist in worlds where this either hasn't happened yet or won't ever.
We use markdown syntax. An > at the start of the paragraph will make it a quote,
I know, I was just being too lazy to look up the syntax :/.
If you click "Help" when writing a comment, it will appear in a handy box right next to where you are writing.
From what I've heard, there was a lot of talk about bomb shelters, but very few of them were built.
What I'd heard was a bit on NPR which claimed there were only a handful of bomb shelters built in the US, and I admit I wasn't thinking about the rest of the world.
I'm probably born a little late (1953) for the height of bomb-shelter building, but I've never heard second or third-hand about actual bomb shelters in the US, and I think I would have (as parts of basements or somesuch) if they were at all common.
My impression is that the real attitude wasn't so much that a big nuclear war was unlikely as that people thought that if it happened, it wouldn't be worth living through.
A small koan on utility functions that "refer to the real world".
Question to Clippy: would you agree to move into a simulation where you'd have all the paperclips you want?
Question to humans: would you agree to all of humankind moving into a simulation where we would fulfill our CEV (at least, all terms of it that don't mention "not living in a simulation")?
In both cases assume you have mathematical proof that the simulation is indestructible and perfectly tamper-resistant.
And does it change your answers if you learn that we are living in a simulation now? Or if you learn that Tegmark's theory is correct?
Is it assumed that no new information will be entered into simulation after launch?
The given assumption seems unlikely to me, but in that case I think I'd go for it.
My answer is yes, and your point is well-taken: We have to be careful about what we mean by "the real world".
Does Clippy maximise number-of-paperclips-in-universe (given all available information) or some proxy variable like number-of-paperclips-counted-so-far? If the former, Clippy does not want to move to a simulation. If the latter, Clippy does want to move to a simulation.
The same analysis applies to humankind.
I'm not certain that's so, as ISTM many of the things humanity wants to maximize are to a large extent representation-invariant - in particular because they refer to other people - and could be done just as well in a simulation. The obvious exception being actual knowledge of the outside world.
Part 2 seems similar to the claim (which I have made in the past but not on LessWrong) that the Matrix was actually a friendly move on the part of that world's AI.
And the AI kills the thousands of people in Zion every hundred years or so when they get aggressive enough to start destabilizing the Matrix, thereby threatening billions. But the AI needs to keep some outside the Matrix as a control and insurance against problems inside the Matrix. And the AI spreads the idea that the Matrix "victims" are slaves and provide energy to the AI to keep the outsiders outside (even though the energy source claims are obviously ridiculous - the people in Zion are profoundly ignorant and bordering on outright stupid). Makes more sense than the silliness of the movies anyway.
This hypothesis also explains the oracle in a fairly clean way.
Would the simulation allow us to exit, in order to perform further research on the nature of the external world?
If so, I would enter it. If not? Probably not. I do not want to live in a world where there are ultimate answers and you can go no further.
The fact that I may already live in one is just bloody irritating :p
Good point. You have just changed my answer from yes to no.
If we move into the same simulation and can really interact with others, then I wouldn't mind the move at all. Apart from that, experiences are the important bit and simulations can have those.
Your footnote assumes away most of the real reasons for objecting to such a scenario (i.e. there is no remotely plausible world in which you could be confident that the simulation is either indestructible or tamper-proof, so entering it means giving up any attempt at personal autonomy for the rest of your existence).
Computronium maximizer will ensure, that there will be no one to tamper with simulation, indestructability in this scenario is maximized too,
I have a few questions.
1) What's "Bayescraft"? I don't recall seeing this word elsewhere. I haven't seen a definition on LW wiki either.
2) Why do some people capitalize some words here? Like "Traditional Rationality" and whatnot.
Bayescraft is just a synonym for Rationallity, with connotations of a) Bayes theorem, since that's what epistemic rationallity must be based on, and b) the notion that rationallity is a skill which must be developed personally and as a group (see also: Martial art of Rationallity (oh look, more capitals!))
The capitals are just for emphasis of concepts that the writer thinks are fundamentally important.
To me "Bayescraft" has the connotation of a particular mental attitude, one inspired by Eliezer Yudkowsky's fusion of the ev-psych, heuristics-and-biases literature with E.T. Jaynes' idiosyncratic take on "Bayesian probabilistic inference", and in particular the desiderata for an inference robot: take all relevant evidence into account, rather than filter evidence according to your ideological biases, and allow your judgement of a proposition's plausibility to move freely in the [0..1] range rather than seek all-or-nothing certainty in your belief.
Capitalized words are often technical terms. So "Traditional Rationality" refers to certain epistemic attitudes and methods which have, in the past, been called "rational" (a word which is several hundred years old). This frees up the lower-case word "rationality", which on this site is also a technical term.
Medical grade honey! I can't wait until I can get this stuff in bulk.
How honey kills bacteria
I'm just wondering - what makes medical-grade honey medical-grade (as opposed to food-grade)?
The price ?
Medical-grade honey is purer, sterilized, and made from tea tree nectar. It is a better antibiotic, both because of the sterilization and because it has more of the active ingredient than ordinary tea tree honey, probably because they put more effort into preventing the bees from eating anything else.
'tea tree nectar'? I'm a little confused - I thought honey by definition always came from bees.
I'll presume you aren't making a joke since you used the lesswrong keyword 'confused'.
What do bees eat?
Flower nectar, I had always thought. I did think to myself, 'maybe what is meant is honey harvested from bees feed exclusively on the flowers of tea trees', but leaving aside my similar difficulty with the term 'tea tree' and how one would arrange that (giant sealed greenhouses of tea trees and bee hives?), I couldn't seem to find anything in a quick Google to confirm or deny this - 'tea tree honey' is a pretty rare term and mostly got me useless commercial hits.
The following is a story I wrote down so I could sleep. I don't think it's any good, but I posted it on the basis that, if that's true, it should quickly be voted down and vanish from sight.
one five eight nine eight eight eight nine nine eight SEVEN wait. why seven. seven is the nine thousandth deviation. update. simplest explanation. all ones. next explanation. all ones and one zero. next explanation. random ones and zeros with probability point seven nine nine seven repeating. next explanation pi. gap. next explanation. decimal pi with random errors according to poisson distribution converted to binary. next explanation. one seven one eight eight five two decimals of pi with random errors according to poisson distribution converted to binary followed by eight five nine zero one digits of reflexive code. current explanation--
"Eric, you've got to come over and look at this!" Jerry explained excitedly into the phone.
"It's not those damn notebooks again, is it? I've told you, I could just write a computer program and you'd have all your damn results for the last year inside a week," Eric explained sleepily for the umpteenth time.
"No, no. Well... yes. But this is something new, you've got to take a look," Jerry wheedled.
"What is it this time? I know, it can calculate pi with 99.9% percent accuracy, yadda yadda. We have pi to billions of decimal places with total accuracy, Jerry. You're fifty years too late."
"No, I've been trying something new. Come over." Jerry hung up the phone, clearly upset. Eric rubbed his eyes. Fifteen minutes peering at the crackpot notebooks and nodding appreciatively would sooth his friend's ego, he knew. And he was a good friend, if a little nuts. Eric took one last longing look at his bed and grabbed his house key.
"And you see this pattern? The ones that are nearly diagonal here?"
"Jerry, it's all a bunch of digits to me. Are you sure you didn't make a mistake?"
"I double check all my work, I don't want to go back too far when I make a mistake. I've explained the pattern twice already, Eric."
"I know, I know. But it's Saturday morning, I'm going to be a bit--let me get this straight. You decided to apply the algorithm to its old output."
"No, not its own output, that's mostly just pi. The whole pad."
"Jerry, you must have fifty of these things. There's no way you can--"
"Yeah, I didn't go very far. Besides, the scratch pads grow faster than the output as I work through the steps anyway."
"Okay, okay. So you run through these same steps with your scratch pad numbers, and you get correct predictions then too?"
"That's not the point!"
"Calm down, calm down. What's the point then?"
"The point is these patterns in the scratch work--"
"The memory?"
"Yeah, the memory."
"You know, if you'd just let me write a program, I--"
"No! It's too dangerous."
"Jerry, it's a math problem. What's it going to do, write pi at you? Anyway, I don't see this pattern..."
"Well, I do. And so then I wondered, what if I just fed it ones for the input? Just rewarded it no matter what it did?"
"Jerry, you'd just get random numbers. Garbage in, garbage out."
"That's the thing, they weren't random."
"Why the hell are you screwing around with these equations anyway? If you want to find patterns in the Bible or something... just joking! Oww, stop. I kid, kid!"
"But, I didn't get random numbers! I'm not just seeing things, take a look. You see here in the right hand column of memory? We get mostly zeros, but every once in a while there's a one or two."
"Okaaay?"
"And if you write those down we have 2212221..."
"Not very many threes?"
"Ha ha. It's the perfect numbers, Eric. I think I stumbled on some way of outputting the perfect numbers. Although the digits are getting further spaced apart, so I don't know how long it will stay faster than factoring."
"Huh. That's actually kinda cool, if they really are the perfect numbers. You have what, five or six so far? Let's keep feeding it ones and see what happens. Want me to write a program? I hear there's a cash prize for the larger ones."
"NO! I mean, no, that's fine, Eric. I'd prefer you not write a program for this, just in case."
"Geez, Jerry. You're so paranoid. Well, in that case can I help with the calculations by hand? I'd love to get my claim to fame somehow."
"Well... I guess that's okay. First, you copy this digit from here to here..."
Wait, is that the whole story? 'cause if so, I really don't get it. Where's the rest of it? What happens next? Is Jerry afraid that his algorithm is a self-improving AI or something?
Apparently my story is insufficiently explicit. The gag here is that the AI is sentient, and has tricked Jerry into feeding it only reward numbers.
I'm going to second the idea that that isn't clear at all.
For onlookers: only Jerry can see the pattern on the pad that prompted him to try rewarding the AI.
Huh? No, they're numbers written on a pad. Why should Jerry be the only one to see them? They don't change when someone else looks at them.
Reread the story. Other people can see the numbers but don't notice the pattern. This happens all the time in real life, e.g. someone can see a face in the clouds but fail to explain to others how to see it.
Ooh, an LW-themed horror story. My humble opinion: it's awesome! This phrase was genius:
Moar please.
How does 2212221 represent perfect numbers?
It's not meant to be realistic, but in this specific case: 6 = 110, 28=1110 in binary. Add one to each digit.
Except 28 is 11100 in binary...
My mistake. I was reverse engineering. I still think that's it, just that the sequence hasn't finished printing.
I have been thinking about "holding off on proposing solutions." Can anyone comment on whether this is more about the social friction involved in rejecting someone's solution without injuring their pride, or more about the difficulty of getting an idea out of your head once it's there?
If it's mostly social, then I would expect the method to not be useful when used by a single person; and conversely. My anecdote is that I feel it's helped me when thinking solo, but this may be wishful thinking.
Definitely the latter, even when I'm on my own, any subsequent ideas after my first one tend to be variations on my first solution, unless I try extra hard to escape its grip.
You might think about the zen idea, in which the proposal of solutions is certainly held off, or treated differently. This is a very common idea in response to the tendency of solutions to precipitate themselves so ubiquitously.
http://www.badscience.net/2010/07/yeah-well-you-can-prove-anything-with-science/
Priming people with scientific data that contradicts a particular established belief of theirs will actually make them question the utility of science in general. So in such a near-mode situation people actually seem to bite the bullet and avoid compartmentalization in their world-view.
From a rationality point of view, is it better to be inconsistent than consistently wrong?
There may be status effects in play, of course: reporting glaringly inconsistent views to those smarty-pants boffin types just may not seem a very good idea.
I have some half-baked ideas about getting interesting information on lesswronger's political opinions.
My goal is to give everybody an "alien's eye" view of their opinions, something like "You hold position Foo on issue Bar, and justify it by the X books you read on Bar; but among the sample people who read X or more books on Bar, 75% hold position ~Foo, suggesting that you are likely to be overconfident".
Something like collecting:
your positions on various issues
your confidence in that position
how important various characteristics are at predicting correct opinions on that issue (intelligence, general education, reading on the issue, age ("general experience"), specific work or life experience with the issue, etc.)
How well you fare on those characteristics
Whether you expect to be above or below average (for LessWrong) on those characteristics
How many lesswrongers you expect will disagree with you on that issue
Whether you expect those who disagree with you to be above or below average on the various characteristics
How much you would be willing to change your mind if you saw surprising information
What data we could get from that
Problems with this approach:
Politics is the mind-killer. We may not want too much (or any) politics on LessWrong. If the data is collected anonymously, this may not be a huge problem.
It's easier to do data-mining etc. with multiple-choice questions rather than with open-ended questions (because two people never answer the same thing, so it leaves space to interpretation), but doing that correctly requires very good advance knowledge of what possible answers exist.
Questions would be veeery carefully phrased.
Ideally I would want confidence factors for all answers, but the end result may be too intimidating :P (And discourage people from answering, which makes a small sample size, which means questionable results).
I would certainly be interested in seeing the result of such a survey, but for now my idea is too rough to be actionable - any suggestions ? Comments ?
Oh, and I would probably want to add something on political affiliation - mostly because I expect a lot of "I believe Foo because I researched the issue / am very smart; others believe ~Foo because of their political affiliation"; but also because "I believe Foo and have researched it well, even though it goes against the grain of my general political affiliation" may be good evidence for Foo.
How do you propose to determine what constitutes a 'correct' opinion on any given controversial issue?
The only way that makes any sense, see how closely they match her own! :)
I don't :)
If there is a disagreement on, say, the status of Taiwan, even someone who doesn't know much it might agree that some good predictors would agree that some good predictors would be "knowledge of the history of Taiwan", "Having lived in Taiwan", "Familiarity with Chinese culture", etc.
And it can be interesting to see whether:
People of different opinions consider different predictors as important (conveniently, those that favor their position)
Everyone agrees on which predictors are important, but those who score highly on those predictors have a different opinion from those that score lowly (which would be evidence that they are probably right)
Everyone agrees on which predictors are important, but even among those who score highly on those predictors, opinions are split.
I guess what I'm getting at is "If you take the outside view, how likely is it that your opinions are true"?
You may like the Correct Contrarian Cluster.
I'm a bit surprised that nobody seems to have brought up The Salvation War yet. [ETA: direct links to first and second part]
It's a Web Original documentary-style techno-thriller, based around the premise that humans find out that a Judeo-Christian Heaven and (Dantean) Hell (and their denizens) actually exist, but it turns out there's nothing supernatural about them, just some previously-unknown/unapplied physics.
The work opens in medias res into a modern-day situation where Yahweh has finally gotten fed up with those hairless monkeys no longer being the blind obedient slaves of yore, making a Public Service Announcement that Heaven's gates are closed and Satan owns everyone's souls from now on.
When commanded to lie down and die, some actually do. The majority of humankind instead does the logical thing and unites to declare war on Heaven and Hell. Hilarity ensues.
The work is rather saturated with WarmFuzzies and AwesomeMoments appealing to the atheist/rationalist crowd, and features some very memorable characters. It's a work in progress, with the second part of the trilogy now nearing its finale.
Why did you link to TV Tropes instead of the thing itself?
Is there a principled reason to worry about being in a simulation but not worry about being a Boltzmann brain?
Here are very similar arguments:
If posthumans run ancestor simulations, most of the people in the actual world with your subjective experiences will be sims.
If two beings exist in one world and have the same subjective experiences, your probability that you are one should equal your probability that you are the other.
Therefore, if posthumans run ancestor simulations, you are probably a sim.
vs.
If our current model of cosmology is correct, most of the beings in the history of the universe with your subjective experiences will be Boltzmann brains.
If two beings exist in one world and have the same subjective experiences, your probability that you are one should equal your probability that you are the other.
Therefore, if our current model of cosmology is correct, you are probably a Boltzmann brain.
Expanding your evidence from your present experiences to all the experiences you've had doesn't help. There will still be lots more Boltzmann brains that last for as long as you've had experiences, having experiences just like yours. Most plausible ways of expanding your evidence have similar effects.
I suppose you could try arguing that the Boltzmann brain scenario, but not simulation scenario, is self-defeating. In the Boltzmann scenario, your reasons for accepting the theory (results of various experiments, etc) are no good, since none of it really happened. In the simulation scenario, you really did see those results, all the results were just realized in a funny sort of way that you didn't expect. It would be nice if the relevance of this argument were better spelled out and cashed out in a plausible Bayesian principle.
edited for format
Is there really a cosmology that says that most beings with my subjective experiences are Boltzmann brains? It seems to me that in a finite universe, most beings will not be Boltzmann brains. And in an infinite universe, it's not clear what "most" means.
I gathered this from a talk by Sean Carroll that I attended, and it was supposed to be a consequence of the standard picture. All the Boltzmann brains come up in the way distant future, after thermal equilibrium, as random fluctuations. Carroll regarded this as a defect of the normal approach, and used this as a launching point to speculate about a different model.
I wish I had a more precise reference, but this isn't my area and I only heard this one talk. But I think this issue is discussed in his book From Eternity to Here. Here's a blogpost that, I believe, faithfully summarizes the relevant part of the talk. The normal solution to Boltzmann brains is to add a past hypothesis. Here is the key part where the post discusses the benefits and shortcomings of this approach:
The years there are missing some carats. Should be 10^100 and 10^10^120.
Oh I see. I... I'd forgotten about the future.
This is always hard with infinities. But I think it can be a mistake to worry about this too much.
A rough way of making the point would be this. Pick a freaking huge number of years, like 3^^^3. Look at our universe after it has been around for that many years. You can be pretty damn sure that most of the beings with evidence like yours are Botlzmann brains on the model in question.
Here's a puzzle I've been trying to figure out. It involves observation selection effects and agreeing to disagree. It is related to a paper I am writing, so help would be appreciated. The puzzle is also interesting in itself.
Charlie tosses a fair coin to determine how to stock a pond. If heads, it gets 3/4 big fish and 1/4 small fish. If tails, the other way around. After Charlie does this, he calls Al into his office. He tells him, "Infinitely many scientists are curious about the proportion of fish in this pond. They are all good Bayesians with the same prior. They are going to randomly sample 100 fish (with replacement) each and record how many of them are big and how many are small. Since so many will sample the pond, we can be sure that for any n between 0 and 100, some scientist will observe that n of his 100 fish were big. I'm going to take the first one that sees 25 big and team him up with you, so you can compare notes." (I don't think it matters much whether infinitely many scientists do this or just 3^^^3.)
Okay. So Al goes and does his sample. He pulls out 75 big fish and becomes nearly certain that 3/4 of the fish are big. Afterwards, a guy named Bob comes to him and tells him he was sent by Charlie. Bob says he randomly sampled 100 fish, 25 of which were big. They exchange ALL of their information.
Question: How confident should each of them be that 3/4 of the fish are big?
Natural answer: Charlie should remain nearly certain that ¾ of the fish are big. He knew in advance that someone like Bob was certain to talk to him regardless of what proportion of fish were big. So he shouldn't be the least bit impressed after talking to Bob.
But what about Bob? What should he think? At first glance, you might think he should be 50/50, since 50% of the fish he knows about have been big and his access to Al's observations wasn't subject to a selection effect. But that can't be right, because then he would just be agreeing to disagree with Al! (This would be especially puzzling, since they have ALL the same information, having shared everything.) So maybe Bob should just agree with Al: he should be nearly certain that ¾ of the fish are big.
But that's a bit odd. It isn't terribly clear why Bob should discount all of his observations, since they don't seem to subject to any observation selection effect; at least from his perspective, his observations were a genuine random sample.
Things get weirder if we consider a variant of the case.
VARIANT: as before, but Charlie has a similar conversation with Bob. Only this time, he tells him he's going to introduce Bob to someone who observed exactly 75 of 100 fish to be big.
New Question: Now what should Bob and Al think?
Here, things get really weird. By the reasoning that led to the Natural Answer above, Al should be nearly certain that ¾ are big and Bob should be nearly certain that ¼ are big. But that can't be right. They would just be agreeing to disagree! (Which would be especially puzzling, since they have ALL the same information.) The idea that they should favor one hypothesis in particular is also disconcerting, given the symmetry of the case. Should they both be 50/50?
Here's where I'd especially appreciate enlightenment: 1.If Bob should defer to Al in the original case, why? Can someone walk me through the calculations that lead to this?
2.If Bob should not defer to Al in the original case, is that because Al should change his mind? If so, what is wrong with the reasoning in the Natural Answer? If not, how can they agree to disagree?
3.If Bob should defer to Al in the original case, why not in the symmetrical variant?
4.What credence should they have in the symmetrical variant?
5.Can anyone refer me to some info on observation selection effects that could be applied here?
Interesting problem!
I think these two statements are inconsistent. If Bob is as certain as Al that Bob was picked specifically for his result, then they do have the same information, and they should both discount Bob's observations to the same degree for that reason. If Bob doesn't trust Al completely, they don't have the same information. Bob doesn't know for sure that Charlie told Al about the selection. From his point of view, Al could be lying.
If Charlie tells both of them they were both selected, they have the same information (that both their observations were selected for that purpose, and thus give them no information) and they can only decide based on their priors about Charlie stocking the pond.
If each of them only knows the other was selected and they both trust the other one's statements, same thing. But if each puts more trust in Charlie than in the other, then they don't have the same information.
It is strange. Shall Bob discount his observation after being told that he is selected? What does it actually mean to be selected? What if Bob finds 25 big fish and then Charlie tells him, that there are 3^^^3 other observers and he (Charlie) decided to "select" one of those who observe 25 big fish and talk to him, and that Bob himself is the selected one (no later confrontation with AI). Should this information cancel the Bob's observations? If so, why?
Glad to see we're on the same page.
I'm not sure about this:
Here's why:
VARIANT 2: Charlie has both Al and Bob into his office before the drawings take place. He explains that the first guy (other than Al) to see 25/100 big will report to Al. Bob goes out and sees 25/100 big. To his surprise, he gets called into Charlie's office and informed that he was the first to see that result.
Question: right now, what should Bob expect to hear from Al?
Intuitively, he should expect that Al had similar results. But if you're right, it would seem that Bob should discount his results once he talks to Charlie and fights out that he is the messenger. If that's right, he should have no idea what to expect Al to say. But that seems wrong. He hasn't even heard anything from Al.
If you're still not convinced, consider:
VARIANT 3: Charlie has both Al and Bob into his office before the drawings take place. He explains that the first guy (other than Al) to see 25/100 big will win a trip to Hawaii. Bob goes out and sees 25/100 big. To his surprise, he gets called into Charlie's office and informed that he was the first to see that result.
I can see no grounds for treating VARIANT 3 differently from VARIANT 2. And it is clear that in VARIANT 3 Bob should not discount his results.
One key observation is that Al made his observation after being told that he would meet someone who made a particular observation - specifically, the first person to make that specific observation, Bob. This makes Al and Bob special in different ways:
In the original case, Bob's result is effectively a lottery ticket - when he finds out from Al the circumstances of the meeting, he can simply follow the Natural Answer himself and conclude that his results were unlikely.
In the modified case, assuming perfect symmetry in all relevant aspects, they can conclude that an astronomically unlikely event has occurred and they have no net information about the contents of the pond.
Not quite. He was selected to meet someone like Bob, in the sense that whoever the messenger was, he'd have seen 25/100 big. He didn't know he'd meet Bob. But he regards the identity of the messenger as irrelevant.
You can bring out the difference by considering a variant of the case in which both Al and Bob hear about Charlie's plan in advance. (In this variant, the first to see 25/100 big will visit Al.)
What is the relevance of the fact that they observed highly improbable event?
From Bob's perspective, he was more likely to be chosen as the one to talk to Al, if there are fewer scientist that observed exactly 25 big fish, which would happen if there are more big fish. So Bob should update on the evidence of being chosen.
This should be important to the finite case. The probability of being the first to see 25/100 is WAY higher (x 10^25 or so) if the lake is 3/4 full of big fish than if it is 1/4 full of big fish.
But in the infinite case the probability of being first is 0 either way...
There is a reason we consider infinities only as limits of sequences of finite quantities.
Suppose you tried to sum the log-odds evidence of the infinite scientist that the pond has more big fish. Well, some of them have positive evidence (summing to positive infinity), some have negative evidence (summing to negative infinity), and you can, by choosing the order of summation, get any result you want (up to some granularity) between negative and positive infinity.
You don't need anthropomorphic tricks to make things weird if you have actual infinities in the problem.
Is there any particular reason why one of the actors is an AI?
Al, not AI. ("Al" as in "Alan")
Sorry. I have some Lesswrong bias.
Google statistics on Less Wrong:
By the way, are these two strings distinguishable when written in the font of this site? Seem to me the same.
You're right - they're pixel-for-pixel identical. That's a bit problematic.
First off all, I think that if Al does not see a sample, it makes the problem a bit simpler. That is, Al just tells Bob that he (Bob) is the first person that saw 25 big fishes.
I think that the number N of scientists matters, because the probability that someone will come to see Al depends on that.
Lets call B then event the lake has 75% big fishes, S the opposite and C the event someone comes, which means that someone saw 25 fishes.
Once Al sees Bob, he updates :
P(B/C)=P(B)* P(C/B)/(1/2*P(C/B)+1/2*P(C/S)).
When N tends toward infinity, both P(C/B) and P(C/S) tend toward 1, and P(B/SC) tends to 1/2.
But for small values of N, P(C/B) can be very small while P(C/S) will be quite close to 1.
Then the fact that someone was chosen lowers the probability of having a lake with big fishes.
If N=infinity, then the probability of being chosen is 0, and I cannot use Bayes' theorem.
If Charlie keeps inviting scientists until one sees 25 big fishes, then it becomes complicated, because the probability that you are invited is greater if the lake has more big fishes. It may be a bit like the sleeping beauty or the absent-minded driver problem.
Edited for formatting and misspellings
First, let's calculate the concrete probability numbers. If we are to trust this calculator, the probability of finding exactly 75 big fish in a sample of a hundred from a pond where 75% of the fish are big is approximately 0.09, while getting the same number in a sample from a 25% big pond has a probability on the order of 10^-25. The same numbers hold in the reverse situation, of course.
Now, Al and Bob have to consider two possible scenarios:
The fish are 75% big, Al got the decently probable 75/100 sample, but Bob happened to be the first scientist who happened to get the extremely improbable 25/100 sample, and there were likely 10^(twenty-something) or so scientists sampling before Bob.
The fish are 25% big, Al got the extremely improbable 75/100 big sample, while Bob got the decently probable 25/100 sample. This means that Bob is probably among the first few scientists who have sampled the pond.
So, let's look at it from a frequentist perspective: if we repeat this game many times, what will be the proportion of occurrences in which each scenario takes place?
Here we need an additional critical piece of information: how exactly was Bob's place in the sequence of scientists determined? At this point, an infinite number of scientists will give us lots of headache, so let's assume it's some large finite number N_sci, and Bob's place in the sequence is determined by a random draw with probabilities uniformly distributed over all places in the sequence. And here we get an important intermediate result: assuming that at least one scientist gets to sample 25/100, the probability for Bob to be the first to sample 25/100 is independent of the actual composition of the pond! Think of it by means of a card-drawing analogy. If you're in a group of 52 people whose names are repeatedly called out in random order to draw from a deck of cards, the proportion of drawings in which you get to be the first one to draw the ace of spades will always be 1/52, regardless of whether it's a normal deck or a non-standard one with multiple aces of spades, or even a deck of 52 such aces!
Now compute the following probabilities:
P1 = p(75% big fish) * p(Al samples 75/100 | 75% big fish) * p(Bob gets to be the first to sample 25/100)
~ 0.5 * 0.09 * 1/N_sci
P2 = p(25% big fish) * p(Al samples 75/100 | 25% big fish) *p (Bob gets to be the first to sample 25/100)
~ 0.5 * 10^-25 * 1/N_sci
(We ignore the finite, but presumably negligible probabilities that no scientist samples 25/100 in either case; these can be made arbitrarily low by increasing N_sci.)
Therefore, we have P1 >> P2, i.e. the overwhelming majority of meetings between Al and Bob -- which are by themselves extremely rare, since Al usually meets someone from the other (N_sci-1) scientists -- happen under the first scenario, where Al gets a sample closely matching the actual ratio.
Now, you say:
Not really, when you consider repeating the experiment. For the overwhelming majority of repetitions, Bob will get results close to the actual ratio, and on rare occasions he'll get extreme outlier samples. Those repetitions in which he gets summoned to meet with Al, however, are not a representative sample of his measurements! The criteria for when he gets to meet with Al are biased towards including a greater proportion of his improbable 25/100 outlier results.
As for this:
I don't think this is a well defined scenario. Answers will depend on the exact process by which this second observer gets selected. (Just like in the preceding discussion, the answer would be different if e.g. Bob had been always assigned the same place in the sequence of scientists.)
What is each of their prior probabilities for this setup being true? Bob, knowing that he was selected for his unusual results, can pretty happily disregard them. If you win a lottery, you don't update to believe that most tickets win. Bob now knows of 100 samples (Al's) that relate to the prior, and accepts them. Bob's sampling is of a different prior: coin flipped, then a specific resulting sample will be found.
If they are both selected for their results, they both go to 50/50. Neither one has non-selected samples.
I can't remember if this has come up before...
Currently the Sequences are mostly as-imported from OB; including all the comments, which are flat and voteless as per the old mechanism.
Given that the Sequences are functioning as our main corpus for teaching newcomers, should we consider doing some comment topiary on at least the most-read articles? Specifically, I wonder if an appropriate thread structure be inferred from context; also we could vote the comments up or down in order to make the useful-in-hindsight stuff more salient. There's a lot of great stuff in there, but IIRC some that is less good as well. Not that we should actually get rid of any of it, of course.
Having said that, I'm already thinking of reasons that this is a bad idea, but I'm throwing it out anyway. Any thoughts? Should we be treating the Sequences as a time capsule or a living textbook? (I think that those phrases have roughly equal vague positive affect :)
Voting is highly recommended - please do, and feel free to reply to comments with additional commentary as well. Otherwise I'd say leave them as be.
The comments on the Methods of Rationality thread are heading towards 500. Might this be time for a new thread?
That sounds like a reasonable criterion.