artemium comments on [Link] If we knew about all the ways an Intelligence Explosion could go wrong, would we be able to avoid them? - Less Wrong Discussion
You are viewing a comment permalink. View the original post to see all comments and the full post content.
Loading…
Subscribe to RSS Feed
= f037147d6e6c911a85753b9abdedda8d)
Comments (15)
Hmm..I wasn't aware of that. Is there any source for that statement? Is MIRI actually doing any general AI research? I don't think that you can easily jump from one specific field of AI research (ethics) to general AI research&design.
From here.
Also,
I get the sense that Eliezer wants to be one of the nine people in that basement, if he can be, but I might be streching the evidence little to say "Eliezer has expressed that ultimately, the goal of MIRI is not just research how to make FAI, but to be the one's to make it."
Thanks! Haven't seen that before. I still think it would be better to specialize on ethics issue and than apply its result on AGI sytsem developed by other (hopefully friendly) party. But It would be awesome if someone who is genuinely ethical develops AGI first. I'm really hoping that some big org which went furthest in AI research like google decides to cooperate with MIRI on that issue when they reach the critical point in AGI buildup.
This is something that I think is neglected (in part because it's not the relevant problem yet) in thinking about friendly AI. Even if we had solved all of the problems of stable goal systems, there could still be trouble, depending on who's goals are implemented. If it's a fast take-off, whoever cracks recursive self-improvement first, basically gets Godlike powers (in the form a genii that reshapes the world according to your wish). They define the whole future of the expanding visible universe. There are a lot of institutions who I do not trust to have the foresight to think "We can create utopia beyond anyone's wildest dreams" and instead to default to "We'll skewer the competition in the next quarter."
However, there are unsubstantiated rumors that Google has taken some ex-MIRI people for work on a project of some kind.