Jiro comments on Siren worlds and the perils of over-optimised search - Less Wrong

27 Post author: Stuart_Armstrong 07 April 2014 11:00AM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Article Navigation

Comments (411)
Tags: constraints optimisation unconstrained deceit manipulation optimization concealment ai risk search

Comments (411)

You are viewing a single comment's thread. Show more comments above.

Comment author: Jiro 14 May 2014 03:01:59PM *  2 points [-]

I also think that evaluation by academics is a terrible test for things that don’t come with blatant overwhwelming unmistakable undeniable-even-to-humans evidence – e.g. this standard would fail MWI, molecular nanotechnology, cryonics,

I think that in saying this, Eliezer is making his opponents' case for them. Yes, of course the standard would also let you discard cryonics. One solution to that is to say that the standard is bad. Another solution is to say "yes, and I don't much care for cryonics either".

Comment author: [deleted] 14 May 2014 03:15:44PM -1 points [-]

I think that in saying this, Eliezer is making his opponents' case for him.

Nah, those are all plausibly correct things that mainstream science has mostly ignored and/or made researching taboo.

If you prefer a more clear-cut example, science was wrong about continental drift for about half a century -- until overwhelming, unmistakable evidence became available.

Comment author: Jiro 14 May 2014 08:13:46PM 2 points [-]

The main reason that scientists rejected continental drift was that there was no known mechanism which could cause it; plate tectonics wasn't developed until the late 1950's.

Continental drift is also commonly invoked by pseudoscientists as a reason not to trust scientists, and if you do so too you're in very bad company. There's a reason why pseudoscientists keep using continental drift for this purpose and don't have dozens of examples: examples are very hard to find. Even if you decide that continental drift is close enough that it counts, it's a very atypical case. Most of the time scientists reject something out of hand, they're right, or at worst, wrong about the thing existing, but right about the lack of good evidence so far.

Comment author: [deleted] 14 May 2014 08:40:52PM -2 points [-]

The main reason that scientists rejected continental drift was that there was no known mechanism which could cause it; plate tectonics wasn't developed until the late 1950's.

There was also a great deal of institutional backlash against proponents of continental drift, which was my point.

Continental drift is also commonly invoked by pseudoscientists as a reason not to trust scientists, and if you do so too you're in very bad company.

Guilt by association? Grow up.

There's a reason why pseudoscientists keep using continental drift for this purpose and don't have dozens of examples: examples are very hard to find. Even if you decide that continental drift is close enough that it counts, it's a very atypical case.

There are many, many cases of scientists being oppressed and dismissed because of their race, their religious beliefs, and their politics. That's the problem, and that's what's going on with the CS people who still think AI Winter implies AGI isn't worth studying.

Comment author: Jiro 14 May 2014 10:29:05PM 2 points [-]

There was also a great deal of institutional backlash against proponents of continental drift, which was my point.

So? I'm pretty sure that there would be backlash against, say, homeopaths in a medical association. Backlash against deserving targets (which include people who are correct but because of unlucky circumstances, legitimately look wrong) doesn't count.

I'm reminded of an argument I had with a proponent of psychic power. He asked me what if psychic powers happen to be of such a nature that they can't be detected by experiments, don't show up in double-blind tests, etc.. I pointed out that he was postulating that psi is real but looks exactly like a fake. If something looks exactly like a fake, at some point the rational thing to do is treat it as fake. At that point in history, continental drift happened to look like a fake.

Guilt by association? Grow up.

That's not guilt by association, it's pointing out that the example is used by pseudoscientists for a reason, and this reason applies to you too.

There are many, many cases of scientists being oppressed and dismissed because of their race, their religious beliefs, and their politics.

If scientists dismissed cryonics because of the supporters' race, religion, or politics, you might have a point.

Comment author: [deleted] 14 May 2014 11:21:04PM *  -2 points [-]

I'll limit my response to the following amusing footnote:

If scientists dismissed cryonics because of the supporters' race, religion, or politics, you might have a point.

This is, in fact, what happened between early cryonics and cryobiology.

EDIT: Just so people aren't misled by Jiro's motivated interpretation of the link:

However, according to the cryobiologist informant who attributes to this episode the formal hardening of the Society for Cryobiology against cryonics, the repercussions from this incident were far-reaching. Rumors about the presentation -- often wildly distorted rumors -- began to circulate. One particularly pernicious rumor, according to this informant, was that my presentation had included graphic photos of "corpses' heads being cut off." This was not the case. Surgical photos which were shown were of thoracic surgery to place cannula and would be suitable for viewing by any audience drawn from the general public.

This informant also indicates that it was his perception that this presentation caused real fear and anger amongst the Officers and Directors of the Society. They felt as if they had been "invaded" and that such a presentation given during the course of, and thus under the aegis of, their meeting could cause them to be publicly associated with cryonics. Comments such as "what if the press got wind of this," or "what if a reporter had been there" were reported to have circulated.

Also, the presentation may have brought into sharper focus the fact that cryonicists existed, were really freezing people, and that they were using sophisticated procedures borrowed from medicine, and yes, even from cryobiology, which could cause confusion between the "real" science of cryobiology and the "fraud" of cryonics in the public eye. More to the point, it was clear that cryonicists were not operating in some back room and mumbling inarticulately; they were now right there in the midst of the cryobiologists and they were anything but inarticulate, bumbling back-room fools.

Obviously political.

Comment author: Jiro 15 May 2014 02:24:11PM *  2 points [-]

Most of the history described in your own link makes it clear that scientists objected because they think cryonics is scientifically nonsense, not because of race, religion, or politics. The article then tacks on a claim that scientists reject it for ethical reasons, but that isn't supported by its own history, just by a few quotes with no evidence that these beliefs are prevalent among anyone other than the people quoted.

Furthermore, of the quotes it does give, one of them is vague enough that I have no idea if it means in context what the article claims it means. Saying that the "end result" is damaging doesn't necessarily mean that having unfrozen people walking around is damaging--it may mean that he thinks cryonics doesn't work and that having a lot of resources wasted on freezing corpses is damaging.

Comment author: Jiro 15 May 2014 03:31:54PM *  2 points [-]

You're equivocating on the term "political". When the context is "race, religion, or politics", "political" doesn't normally mean "related to human status", it means "related to government". Besides, they only considered it low status based on their belief that it is scientifically nonsensical.

My reply was steelmanning your post by assuming that the ethical considerations mentioned in the article counted as religious. That was the only thing mentioned in it that could reasonably fall under "race, religion, or politics" as that is normally understood.

Powered by Reddit Powered by Reddit