lukeprog comments on Welcome to Less Wrong! (5th thread, March 2013) - Less Wrong
You are viewing a comment permalink. View the original post to see all comments and the full post content.
You are viewing a comment permalink. View the original post to see all comments and the full post content.
Comments (1750)
Note also that Roman co-authored 3 of the papers on MIRI's publications page.
His paper http://cecs.louisville.edu/ry/LeakproofingtheSingularity.pdf seriously discusses ways to confine a potentially hostile superintelligence, a feat MIRI seems to consider hopeless. Did you guys have a good chat about it?
I think most everyone at MIRI and FHI thinks boxing is a good thing, even if many would say not enough on its own. I don't think you will find many who think that open internet connections are a matter of indifference for AI developers working with powerful AGI.
High-grade common sense (the sort you'd get by asking any specialist in computer security) says that you should design an AI which you would trust with an open Internet connection, then put it in the box you would use on an untrusted AI during development. (No, the AI will not be angered by this lack of trust and resent you. Thank you for asking.) I think it's safe to say that for basically everything in FAI strategy (I can't think of an exception right now) you can identify at least two things supporting any key point, such that either alone was designed to be sufficient independently of the other's failing, including things like "indirect normativity works" (you try to build in at least some human checks around this which would shut down any scary AI independently of your theory of indirect normativity being remotely correct, while also not trusting the humans to steer the AI because then the humans are your single point of failure).
See my interview with Roman here.
Thanks. Pretty depressing, though.