Less Wrong is a community blog devoted to refining the art of human rationality. Please visit our About page for more information.

Comment author: lukeprog 30 September 2014 03:56:48AM 3 points [-]

This is some nice foreshadowing:

Readers of this chapter must not expect a blueprint for programming an artificial general intelligence. No such blueprint exists yet, of course. And had I been in possession of such a blueprint, I most certainly would not have published it in a book. (If the reasons for this are not immediately obvious, the arguments in subsequent chapters will make them clear.)

Comment author: lukeprog 30 September 2014 03:51:17AM 3 points [-]

The AI section is actually very short, and doesn't say much about potential AI paths to superintelligence. E.g. one thing I might have mentioned is the "one learning algorithm" hypothesis about the neocortex, and the relevance of deep learning methods. Or the arcade learning environment as a nice test for increasingly general intelligence algorithms. Or whatever.

Comment author: lukeprog 30 September 2014 03:42:18AM 2 points [-]

You mention by Randal Koene interview on WBE. I'd also mention the Ken Hayworth interview.

Comment author: lukeprog 30 September 2014 01:12:21AM 3 points [-]
Comment author: lukeprog 26 September 2014 03:30:13PM 7 points [-]

In Ideal Advisor Theories and Personal CEV, my co-author and I describe a particular (but still imprecisely specified) version of the parliamentary approach:

we determine the personal CEV of an agent by simulating multiple versions of them, extrapolated from various starting times and along different developmental paths. Some of these versions are then assigned to a parliament where they vote on various choices and make trades with one another.

We then very briefly argue that this kind of approach can overcome some objections to parliamentary models (and similar theories) made by philosopher David Sobel.

The paper is short and non-technical, but still manages to summarize some concerns that we'll likely want a formalized parliamentary model to overcome or sidestep.

Comment author: lukeprog 24 September 2014 07:06:10AM 16 points [-]

He who knows only his own side of the case, knows little of that.

J.S. Mill

Comment author: lukeprog 23 September 2014 11:17:35PM *  7 points [-]

I have yet to read a book on consciousness that I was fairly happy with. At the moment I would recommend this review article over any particular book I've read.

Example books on consciousness I've read or skimmed: Consciousness Explained, The Ego Tunnel, Consciousness: VSI, Consciousness: An Introduction, Consciousness and the Brain.

Comment author: lukeprog 23 September 2014 07:10:27PM *  22 points [-]

In case this helps and isn't obvious to everyone, I'll briefly mention that I'm the Executive Director of MIRI and I agree with what Daniel wrote above.

Also, the linked Ross Andersen piece on FHI is really good and people should read it.

Comment author: sbenthall 18 September 2014 02:39:39AM 8 points [-]

So there's some big problems of picking the right audience here. I've tried to make some headway into the community complaining about newsfeed algorithm curation (which interests me a lot, but may be more "political" than would interest you) here:


which is currently under review. It's a lot softer that would be ideal, but since I'm trying to convince these people to go from "algorithms, how complicated! Must be evil" to "oh, they could be designed to be constructive", it's a first step. More or less it's just opening up the idea that Twitter is an interesting testbed for ethically motivated algorithmic curation.

I've been concerned more generally with the problem of computational asymmetry in economic situations. I've written up something that's an attempt at a modeling framework here. It's been accepted only as a poster, because it's results are very slim. It was like a quarter of a semester's work. I'd be interested in following through on it.


The main problem I ran into was not knowing a good way to model relative computational capacity; the best tool I had was big-O and other basic computational theory stuff. I did a little sort of remote apprenticeship with David Wolpert as Los Alamos; he's got some really interesting stuff on level-K reasoning and what he calls predictive game theory.


(That's not his most recent version). It's really great work, but hard math to tackle on ones own. In general my problem is there isn't much of a community around this at Berkeley, as far as I can tell. Tell me if you know differently. There's some demand from some of the policy people--the lawyers are quite open-minded and rigorous about this sort of thing. And there's currently a ton of formal work on privacy, which is important but not quite as interesting to me personally.

My blog is a mess and doesn't get into formal stuff at all, at least not recently.

Comment author: lukeprog 18 September 2014 02:42:17AM 2 points [-]


Comment author: lukeprog 17 September 2014 07:12:57PM 4 points [-]

Seb, what kind of work do you "try to do" in this area? Do you have some blog posts somewhere or anything?

View more: Next