Sock puppet accounts aren't appreciated, mwaser, especially when you keep plugging the same blog. Comments about those links have received at least 28 downvotes already, just in this Open Thread.

Do we have IP-based sitebans?

I worry about this as well when I'm reading long arguments or long works of fiction presenting ideas I disagree with. My tactic is to stop occasionally and go through a mental dialog simulating how I would respond to the author in person. This serves a double purpose, as hopefully I'll have better cached arguments in the event I ever need them.

Of course, this is a dangerous tactic as well, because you may be shutting off critical reasoning applied to your preexisting beliefs. I only apply this tactic when I'm very confident the author is wrong and is using fallacious arguments. Even then I make sure to spend some amount of time playing devil's advocate.

I would not worry overmuch about the long-term negative effects of your studying for the bar: with the possible exception of the "overly sincere" types who fall very hard for cults and other forms of indoctrination, people have a lot of antibodies to this kind of thing.

You will continue to be entagled with reality after you pass the exam, and you can do things, like read works of social science that carve reality at the joints, to speed up the rate at which your continued entaglement with reality with cancel out any falsehoods you have to cram for now. Specifically, there are works about the law that do carve reality at the joints -- Nick Szabo's online writings IMO fall in that category. Nick has a law degree, by the way, and there is certainly nothing wrong with his ability to perceive reality correctly.

ADDED. The things that are really damaging to a person's rationality, IMHO, are natural human motivations. When for example you start practicing, if you were to decide to do a lot of trials, and you learned to derive pleasure -- to get a real high -- from the combative and adversarial part of that, so that the high you got from winning with a slick and misleading angle trumped the high you get from satisfying you curiosity and from refining and finding errors in your model of reality -- well, I would worry about that a lot more than your throwing yourself fully into winning on this exam because IMHO the things we derive no pleasure from, but do to achieve some end we care about (like advancing in our career by getting a credential) have a lot less influence on who we turn out to be than things we do because we find them intrinsically rewarding.

One more thing: we should not all make our living as computer programmers. That would make the community less robust than it otherwise would be :)

I found an interesting paper on Arxiv earlier today, by the name of Closed timelike curves via post-selection: theory and experimental demonstration.

It promises such lovely possibilities as quick solutions to NP-complete problems

It won't work, as is clearly explained here.

If this worked, Harry could use it to recover any sort of answer that was easy to check but hard to find. He wouldn't have just shown that P=NP once you had a Time-Turner, this trick was more general than that. Harry could use it to find the combinations on combination locks, or passwords of every sort. Maybe even find the entrance to Slytherin's Chamber of Secrets, if Harry could figure out some systematic way of describing all the locations in Hogwarts. It would be an awesome cheat even by Harry's standards of cheating.

Harry took Paper-2 in his trembling hand, and unfolded it.

Paper-2 said in slightly shaky handwriting:

DO NOT MESS WITH TIME

Harry wrote down "DO NOT MESS WITH TIME" on Paper-1 in slightly shaky handwriting, folded it neatly, and resolved not to do any more truly brilliant experiments on Time until he was at least fifteen years old.

To put this into my own words "The more information you extract from the future, the less you are able to control the future from the past. And hence, the less understanding you can have about what those bits of future-generated information are actually going to mean."

I wrote that before actually looking at the paper you linked. I don't understand much QM either, but now that I have looked it seems to me that figure 2 of the paper backs me up on my interpretation of Harry's experiment.

Yes. There are cases where occam's razor results in a tie (or, at least, indistinguishably close).

Consider the spin on an arbitrary particle in deep space, or whether or not an arbitrary digit of pi is even.

[T]o avoid that trivial inconvenience, I've put up a mirror of it.

You're awesome.

I've previously been impressed by how social psychologists reason, especially about identity. Schemata theory is also a decent language for talking about cognitive algorithms from a less cognitive sciencey perspective. I look forward to reading this chapter. Thanks for mirroring, I wouldn't have bothered otherwise.

The method that Eliezer is referring to is known as Solomonoff induction which relies on programs as defined by Turing machines. Quantum computing doesn't come into this issue since these formulations just talk about length of specification, not efficiency of computation. There are theorems that also show that for any given Turing complete well-behaved language, the minimum size of program can't be differ by more than a constant. So changing the language won't alter the priors other than a fixed amount. Taken together with Aumann's Agreement Theorem, the level of disagreement about estimated probability should go to zero in the limiting case (disclaimer I haven't seen a proof of that last claim, but I suspect it would be a consequence of using a Solomonoff style system for your priors).

Beautiful. Matthew Yglesias, +1 point.

It is entirely possible that some social groups are experiencing the kind of changes that Flanagan describes, but as Yglesias says, she apparently is unaware that there is such a thing as scientific evidence on the question.

From that Wikipedia article:

Inside the railcar, besides the paper clips, there are the Schroeders’ book and a suitcase filled with letters of apology to Anne Frank by a class of German schoolchildren.

Apologizing for ... being German? That's really bizarre.

I think it's very noble of them to collect numerous paperclips and hold them safely out of use. c=@ I just hope they have appropriate protocols in place to ensure they don't become stolen or unbent. Anyone know if there's an insurance policy taken out against loss or destruction of the paperclips?

I think I found the study they're talking about thanks to this article. I might take a look at it - if the methodology is literally just 'smoking was banned, then the heart attack rate dropped', that sucks.

(Edit to link to the full study and not the abstract.)

Just skimmed it. The methodology is better than that. They use a regression to adjust for the pre-existing downward trend in the heart attack hospital admission rate; they represent it as a linear trend, and that looks fair to me based on eyeballing the data in figures 1 and 2. They also adjust for week-to-week variation and temperature, and the study says its results are 'more modest' than others', and fit the predictions of someone else's mathematical model, which are fair sanity checks.

I still don't know how robust the study is - there might be some confounder they've overlooked that I don't know enough about smoking to think of - but it's at least not as bad as I expected. The authors say they want to do future work with a better data set that has data on whether patients are active smokers, to separate the effect of secondhand smoke from active smoking. Sounds interesting.

I'm not necessarily arguing for this position as saying we need to address it. "Suicidal AI" is to the problem of constructing FAI as anarchism is to political theory; if you want to build something (an FAI, a good government) then, on the philosophical level, you have to at least take a stab at countering the argument that perhaps it is impossible to build it.

I'm working under the assumption that we don't really know at this point what "Friendly" means, otherwise there wouldn't be a problem to solve. We don't yet know what we want the AI to do.

What we do know about morality is that human beings practice it. So all our moral laws and intuitions are designed, in particular, for small, mortal creatures, living among other small, mortal creatures.

Egalitarianism, for example, only makes sense if "all men are created equal" is more or less a statement of fact. What should an egalitarian human make of a powerful AI? Is it a tyrant? Well, no, a tyrant is a human who behaves as if he's not equal to other humans; the AI simply isn't equal. Well, then, is the AI a good citizen? No, not really, because citizens treat each other on an equal footing...

The trouble here, I think, is that really all our notions of goodness are really "what is good for a human to do." Perhaps you could extend them to "what is good for a Klingon to do" -- but a lot of moral opinions are specifically about how to treat other people who are roughly equivalent to yourself. "Do unto others as you would have them do unto you." The kind of rules you'd set for an AI would be fundamentally different from our rules for ourselves and each other.

It would be as if a human had a special, obsessive concern and care for an ant farm. You can protect the ants from dying. But there are lots of things you can't do for the ants: be an ant's friend, respect an ant, keep up your end of a bargain with an ant, treat an ant as a brother...

I had a friend once who said, "If God existed, I would be his enemy." Couldn't someone have the same sentiment about an AI?

(As always, I may very well be wrong on the Internet.)

It seems unlikely that an FAI would commit suicide if humans need to be protected from UAI, or if there are other threats that only an FAI could handle.

Do you ever have a day when you log on and it seems like everyone is "wrong on the Internet"? (For values of "everyone" equal to 3, on this occasion.) Robin Hanson and Katja Grace both have posts (on teenage angst, on population) where something just seems off, elusively wrong; and now SarahC suggests that "the only friendly AI may be one that commits suicide". Something about this conjunction of opinions seems obscurely portentous to me. Maybe it's just a know-thyself moment; there's some nascent opinion of my own that's going to crystallize in response.

Now that my special moment of sharing is out of the way... Sarah, is the friendly AI allowed to do just one act of good before it kills itself? Make a child smile, take a few pretty photos from orbit, save someone from dying, stop a war, invent cures for a few hundred diseases? I assume there is some integrity of internal logic behind this thought of yours, but it seems to be overlooking so much about reality that there has to be a significant cognitive disconnect at work here.

It's better than mainstream Singularity articles in the past, IMO; unfortunately, Kurzweil is seen as an authority, but at least it's written with some respect for the idea.

It does seem to be about a lot of different things, some of which are just synonymous with scientific progress (I don't think it's any revelation that synthetic biology is going to become more sophisticated.)

I'm curious: Was the SIAI contacted for that article? I haven't had time to read it all, but a word-search for "Singularity Institute" and "Yudkowsky" turned up nothing.

I agree that this article isn't very good. It seems to do the standard problem of combining a lot of different ideas about what the Singularity would entail. It emphasizes Kurzweil way too much, and includes Kurzweil's fairly dubious ideas about nutrition and health. The article also uses Andrew Orlowski as a serious critic of the Singularity making unsubstantiated claims about how the Singularity will only help the rich. Given that Orlowski's entire approach is to criticize anything remotely new or weird-seeming, I'm disappointed that the NYT would really use him as a serious critic in this context. The article strongly reinforces the perception that the Singularity is just a geek-religious thing. Overall, not well done at all.

But throughout my whole life up to that point (the question even appeared in the animated series Beetlejuice that I saw when I was little), I had assumed a different question was being asked: specifically,

If a tree falls, and no human (or human-entangled[1] sensor) is around to hear it, does it still make vibrations? On what basis do you believe this, lacking a way to directly check? (Call this "Question S".)

Me too! It was actually explained that way to me by my parents as a kid, in fact. I wonder if there are two subtly different versions floating around or EY just interpreted it uncharitably.

And yet, the quantum mechanical world behaves exactly this way. Observations DO change exactly what happens. So, apparently at the quantum mechanical level, nature does have some way of knowing.

I'm not sure what effect that this has upon your argument, but it's something that I think that you're missing.

This seems worthy of a top-post. When you make it a top level post link to the relevant prior posts about complexity of hypotheses.

I think that if this post is left as it is this post would be to trivial to be a top level post. You could reframe it as a beginners' guide to Occam, or you could make it more interesting by going deeper into some of the issues (if you can think of anything more to say on the topic of differentiating between hypotheses that make the same predictions, that might be interesting, although I think you might have said all there is to say)

I believe this is the conversation you're responding to.

(upvoted)

This is great, even more so as you made it open source. I added it to References & Resources for LessWrong.

You should make a short top-level post about this so more people see this

I'd vote you up again for handing out your source code as well as the quote list, but I can't, so an encouraging reply will have to do...

Don't blogspam. Anyone who can see this on the open thread page can see the first posting of it a few comments down.

Where did Vladimir say he was "confused and dazed", as the quotation marks imply he did? I don't see any such comment here or on your blog.

I liked this more when you posted it the first time.

Huh. That's very interesting. I'm a bit confused by the claim that evolution bridges the is/ought divide which seems more like conflating different meanings of words more than anything else. But the general point seems strong.

How can I understand quantum physics?

I don't think the subject is inherently difficult. For example quantum computing and quantum cryptography can be explained to anyone with basic clue and basic math skills.

That's because quantum computing and quantum cryptography only use a subset of quantum theory. Your link says, for example, that the basics of quantum computing only require knowing how to handle 'discrete (2-state) systems and discrete (unitary) transformations,' but a full treatment of QT has to handle 'continuously infinite systems (position eigenstates) and continuous families of transformations (time development) that act on them.' The full QT that can deal with these systems uses a lot more math.

I wonder if there's a general trend for people who are interested in quantum computing and not all of QT to play down the prerequisites you need to learn QT. Your post reminded me of a Scott Aaronson lecture, where he says

The second way to teach quantum mechanics leaves a blow-by-blow account of its discovery to the historians, and instead starts directly from the conceptual core -- namely, a certain generalization of probability theory to allow minus signs. Once you know what the theory is actually about, you can then sprinkle in physics to taste, and calculate the spectrum of whatever atom you want.

Which is technically true, but if you want to know about quark colors or spin or exactly how uncertainty works, pushing around |1>s and |2>s and talking about complexity classes is not going to tell you what you want to know.

To answer your question more directly, I think the best way to understand quantum physics is to get an undergrad degree in physics from a good university, and work as hard as you can while you're getting it. Getting a degree means you have the physics-leaning math background needed to understand explanations of QT that don't dumb it down.

I might be overestimating the amount of math that's necessary - I'm basing this on sitting in on undergrad QT lectures - but I've yet to find a comprehensive QT text that doesn't use calculus, complex numbers, and linear algebra.

Try Jonathan Allday's book "Quantum Reality: Theory and Philosophy." It is technical enough that you get a quantitative understanding out of it, but nothing like a full-blown textbook.

Somewhat accepted partial answer is that huge brains are ridiculously expensive - you need a lot of high energy density food (= fire), a lot of DHA (= fish) etc. Chimp diet simply couldn't support brains like ours (and aquatic ape etc.), nor could they spend as much time as us engaging in politics as they were too busy just getting food.

Perhaps chimp brains are as big as they could possibly be given their dietary constraints.

Five-second guess: Human-level Machiavellian intelligence needs language facilities to co-evolve with, grunts and body language doesn't allow nearly as convoluted schemes. Evolving some precursor form of human-style language is the improbable part that other species haven't managed to pull off.

Seconding kodos96. As this would exonerate not only Knox and Sollecito but Guede as well, it has to be treated with considerable skepticism, to say the least.

More significant, it seems to me (though still rather weak evidence), is the Alessi testimony, about which I actually considered posting on the March open thread.

Still, the Aviello story is enough of a surprise to marginally lower my probability of Guede's guilt. My current probabilities of guilt are:

Knox: < 0.1 % (i.e. not a chance)

Sollecito: < 0.1 % (likewise)

Guede: 95-99% (perhaps just low enough to insist on a debunking of the Aviello testimony before convicting)

It's probably about time I officially announced that my revision of my initial estimates for Knox and Sollecito was a mistake, an example of the sin of underconfidence.

I of course remain willing to participate in a debate with Rolf Nelson on this subject.

Finally, I'd like to note that the last couple of months have seen the creation of a wonderful new site devoted to the case, Injustice in Perugia, which anyone interested should definitely check out. Had it been around in December, I doubt that I could have made my survey seem like a fair fight between the two sides.

That story would be consistent with Guédé's, modulo the usual eyewitness confusion.

That is certainly interesting enough to subject to further reflection. Do we have any evolutionary psychologists in the audience?

Tom_McCabe2 suggests generalizing EY's rebuttal of Pascal's Wager to Pascal's Mugging: it's not actually obvious that someone claiming they'll destroy 3^^^^3 people makes it more likely that 3^^^^3 people will die. The claim is arguably such weak evidence that it's still about equally likely that handing over the $5 will kill 3^^^^3 people, and if the two probabilities are sufficiently equal, they'll cancel out enough to make it not worth handing over the $5.

Personally, I always just figured that the probability of someone (a) threatening me with killing 3^^^^3 people, (b) having the ability to do so, and (c) not going ahead and killing the people anyway after I give them the $5, is going to be way less than 1/3^^^^3, so the expected utility of giving the mugger the $5 is almost certainly less than the $5 of utility I get by hanging on to it. In which case there is no problem to fix. EY claims that the Solomonoff-calculated probability of someone having 'magic powers from outside the Matrix' 'isn't anywhere near as small as 3^^^^3 is large,' but to me that just suggests that the Solomonoff calculation is too credulous.

(Edited to try and improve paraphrase of Tom_McCabe2.)

The unbounded utility function (in some physical objects that can be tiled indefinitely) in Pascal's mugging gives infinite expected utility to all actions, and no reason to prefer handing over the money to any other action. People don't actually show the pattern of preferences implied by an unbounded utility function.

If we make the utility function a bounded function of happy lives (or other tilable physical structures) with a high bound, other possibilities will offer high expected utility. The Mugger is not the most credible way to get huge rewards (investing in our civilization on the chance that physics allows unlimited computation beats the Mugger). This will be the case no matter how huge we make the (finite) bound.

Eliezer's original article bases its argument on the use of Solomonoff induction. He even suggests up front what the problem with it is, although the comments don't make anything of it: SI is based solely on program length and ignores computational resources. The optimality theorems around SI depend on the same assumption. Therefore I suggest:

4. Pascal's Mugging is a refutation of the Solomonoff prior.

But where a computationally bounded agent, or an unbounded one that cares how much work it does, should get its priors from instead would require more thought than a few minutes on a lunchtime break.

The traditional answer is to follow the Kelly criterion, is it not? That would imply

where n is the number of tickets. This implies you should buy n such that (€1)*n = Wf*, where W is your initial wealth.

Edit: Thanks, JoshuaZ, for pointing out that the Kelly criterion might not be the applicable one in a given situation.

It is a good blog, and it has a slightly wider topic spread than LW, so even if you're familiar with most of the standard failures of judgment there'll be a few new things worth reading. (I found the "introducing fines can actually increase a behavior" post particularly good, as I wasn't aware of that effect.)

Thanks, this looks like an excellent supplement for LW.

I am tempted to downvote this comment from -2 just for the irony, but I don't prefer to see fewer comments like this, so I won't.

Besides, the default cutoff is at -4, not -3.

I wish you wouldn't do that, and stuck instead with the generally approved norm of downvoting to mean "I'd prefer to see fewer comments like this" and upvoting "I'd like to see more like this".

You're deliberately participating in information cascades, and thereby undermining the filtering process. As an antidote, I recommend using the anti-kibitzer script (you can do that through your Preferences page).

That article makes it sound like "countersignaling" is forgoing a mandated signal - like showing up at a formal-dress occasion in street clothes.

Alicorn made a post about the tactics of countersignaling a while back.

Am I correct in reading that Longbets charges a $50 fee for publishing a prediction and they have to be a minimum of 2 years in the future? Thats a bit harsh. But these sites are pretty interesting. And they could be useful to. You could judge the accuracy of different users including how accurate they are at guessing long-term, short-term, etc predictions as well as how accurate they are in different catagories (or just how accurate they are on average if you want to be simple.) Then you can create a fairly decent picture of the future, albeit I expect many of the predictions will contradict each other. This is kind of what their already doing obviously, but they could still take it a step further.

This sounds great, I'm definitely in. I feel like I have a moderately okay intuitive grasp on Bayescraft but a chance to work through it from the ground up would be great.

In. Have the deadtree version, but I was stymied in my first crack at it.

In. If needed I can cover a few of the early chapters.

I'm in. I already read the first few chapters, but it will be nice to go over them to solidify that knowledge. The slower pace will help as well. The later chapters rely on some knowledge of statistics, maybe some member of the book club is already knowledgeable to be able to find good links to summaries of these things when they come up?

I would be interested, what is the intended time period for the reading? I have a two-week trip coming up when I will probably be busy but aside from that I would very much like to participate.

I'm in, been intending to read through some maths on my free time.

It's thesis writeup period for me, but this is extremely tempting.