Mostly stuff that's familiar if you've read enough of Eliezer's articles, but this bit was interesting:
LUKE: Well Eliezer one last question. I know you have been talking about writing a book for quite a while and a lot of people will be curious to know how that’s coming along.
ELIEZER: So, I am about to finished with the first draft. The book seems to have split into two books. One is called How to Actually Change Your Mind and it is about all the biases that stop us from changing our minds. And all these little mental skills that we invent in ourselves to prevent ourselves from changing our minds and the counter skills that you need in order to defeat this self-defeating tendency and manage to actually change your mind.
It may not sound like an important problem, but if you consider that people who win Nobel prizes typically do so for managing to change their minds only once, and many of them go on to be negatively famous for being unable to change their minds again, you can see that the vision of people being able to change their minds on a routine basis like once a week or something, is actually the terrifying Utopian vision that I am sure this book will not actually bring to pass. But, it may none the less manage to decrease some of the sand in the gears of thought.
I don't think you need to apologize for the self-promotion. I was happy to hear it, and glad to be told about it.
No, but it's quite an interesting question. Evolution does go in for sticks as well as carrots, even though punishment has non-obvious costs among humans.
When I made my comment, I hadn't read the interview. I'm not sure about Eliezer's worst case scenario from lack of boredom-- it requires that there be a best moment which the AI would keep repeating if it weren't prevented by boredom. Is there potentially a best moment to tile the universe with? Could an AI be sure it had found the best moment?
The sticks are for things that are worse than sitting there doing nothing.
I figure boredom is like that - it has to work at the hedonic baseline - so it has to be a stick.
Is there potentially a best moment to tile the universe with? Could an AI be sure it had found the best moment?
Are its goals to find the "best moment" in the first place? It seems impossible to answer such questions without reference to some kind of moral system.
It makes for good Less Wrong introductory material to point people to, since there are lots of people who won't read long article online but will listen to a podcast on the way to work: LINK.
Apologies for the self-promotion, but it could hardly be more relevant to Less Wrong...