You're looking at Less Wrong's discussion board. This includes all posts, including those that haven't been promoted to the front page yet. For more information, see About Less Wrong.

Lumifer comments on Stupid Questions May 2015 - Less Wrong Discussion

10 Post author: Gondolinian 01 May 2015 05:28PM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (263)

You are viewing a single comment's thread. Show more comments above.

Comment author: Gondolinian 01 May 2015 06:04:30PM 13 points [-]

There is a not necessarily large, but definitely significant chance that developing machine intelligence compatible with human values may very well be the single most important thing that humans have or will ever do, and it seems very likely that economic forces will make strong machine intelligence happen soon, even if we're not ready for it.

So I have two questions about this: firstly, and this is probably my youthful inexperience talking (a big part of why I'm posting this here), but I see so many rationalists do so much awesome work on things like social justice, social work, medicine, and all kinds of poverty-focused effective altruism, but how can it be that the ultimate fate of humanity to either thrive beyond imagination or perish utterly may rest on our actions in this century, and yet people who recognize this possibility don't do everything they can to make it go the way we need it to? This sort of segues in to my second question, which is what is the most any person, more specifically, I can do for FAI? I'm still in high school, so there really isn't that much keeping me from devoting my life to helping the cause of making sure AI is friendly. What would that look like? I'm a village idiot by LW standards, and especially bad at math, so I don't think I'd be very useful on the "front lines" so to speak, but perhaps I could try to make a lot of money and do FAI-focused EA? I might be more socially oriented/socially capable than many here, perhaps I could try to raise awareness or lobby for legislation?

Comment author: Viliam 02 May 2015 05:06:35PM *  6 points [-]

people who recognize this possibility don't do everything they can to make it go the way we need it to

Despite all talking about rationality, we are still humans with all typical human flaws. Also, it is not obvious which way it needs to go. Even if we had unlimited and infinitely fast processing power, and could solve mathematically all kinds of problems related to Löb's theorem, I still would have no idea how we could start transferring human values to the AI, considering that even humans don't understand themselves, and ideas like "AI should find a way to make humans smile" can lead to horrible outcomes. So maybe the first step would be to upload some humans and give them more processing power, but humans can also be horrible (and the horrible ones are actually more likely to seize such power), and the changes caused by uploading could make even nice people go insane.

So, what is the obvious next step, other than donating some money to the research, which will most likely conclude that further research is needed? I don't want to discourage anyone who donates or does the research, just saying that the situation with the research is frustrating by its lack of feedback. On the scale where 0 is the first electronic computer and 100 is the Friendly AI, are we at least at point 1? If we happen to be there, how would we know that?

Comment author: Capla 06 May 2015 09:07:52PM 0 points [-]

So maybe the first step would be to upload some humans and give them more processing power,

I would like this plan, but there are reasons to think that the path to WBE passes through nueromorphic AI which is exceptionally likely to be unfriendly, since the principle is basically to just copy parts of the human brain without understanding how the human brain works.