Open Thread, January 11-17, 2016

username2

If it's worth saying, but not worth its own post (even in Discussion), then it goes here.

Notes for future OT posters:

1. Please add the 'open_thread' tag.

2. Check if there is an active Open Thread before posting a new one. (Immediately before; refresh the list-of-threads page before posting.)

3. Open Threads should be posted in Discussion, and not Main.

4. Open Threads should start on Monday, and end on Sunday.

If it's worth saying, but not worth its own post (even in Discussion), then it goes here.

Notes for future OT posters:

1. Please add the 'open_thread' tag.

2. Check if there is an active Open Thread before posting a new one. (Immediately before; refresh the list-of-threads page before posting.)

3. Open Threads should be posted in Discussion, and not Main.

4. Open Threads should start on Monday, and end on Sunday.

There is a field that thinks hard about risks from unintelligent computers (computer security) that tackles very difficult problems that sometimes get written about in popular fiction (Neil Stephenson, etc.) and manages to not look silly.

I think to the extent that (U)FAI research is even a "real area," it would be closest in mindset to computer security.

manages to not look silly.

Computer security as portrayed on TV frequently does look silly.

0gjm10y

I endorse Lumifer's quibble about the field of computer security, with the caveat that often the fact that the risks happen inside computer systems is much more important than the fact that they come from people. The sort of "value alignment" questions MIRI professes (I think sincerely) to worry about seem to me a long way away from computer security, and plausibly relevant to future AI safety. But it could well be that if AI safety really depends on nailing that sort of thing down then we're unfixably screwed and we should therefore concentrate on problems there is at least some possibility of solving...

0Lumifer10y

Not quite. Computer security deals with managing risks coming from people, it's just that the universe where it has to manage risks is a weird superposition of the physical world (see hardware or physical-access attacks), the social world (see social engineering attacks), and the cyberworld (see the usual 'sploit attacks).