Less Wrong is a community blog devoted to refining the art of human rationality. Please visit our About page for more information.

Comment author: Stuart_Armstrong 21 June 2017 04:53:55PM 0 points [-]

Lloyds also has good reports on things like Coronal Mass Ejections and similar mid-range risks; eg: https://www.lloyds.com/~/media/lloyds/reports/emerging-risk-reports/solar-storm-risk-to-the-north-american-electric-grid.pdf

Comment author: Darklight 13 June 2017 05:37:40AM 1 point [-]

I might be able to collaborate. I have a masters in computer science and did a thesis on neural networks and object recognition, before spending some time at a startup as a data scientist doing mostly natural language related machine learning stuff, and then getting a job as a research scientist at a larger company to do similar applied research work.

I also have two published conference papers under my belt, though they were in pretty obscure conferences admittedly.

As a plus, I've also read most of the sequences and am familiar with the Less Wrong culture, and have spent a fair bit of time thinking about the Friendly/Unfriendly AI problem. I even came up with an attempt at a thought experiment to convince an AI to be friendly.

Alas, I am based near Toronto, Ontario, Canada, so distance might be an issue.

Comment author: Stuart_Armstrong 15 June 2017 05:06:05AM 0 points [-]

Interesting. Can we exchange email addresses?

Comment author: Lumifer 12 June 2017 03:22:53PM 0 points [-]

So if you say "imagine that you were healthy, smart, and happy..." they'll still often say they don't want to live that long.

And what makes you believe that? I doubt that you have data.

Comment author: Stuart_Armstrong 15 June 2017 05:04:45AM 0 points [-]

In fact I do. Parental data :-(

Comment author: username2 12 June 2017 08:29:51AM 0 points [-]

But you can still keep asking the "why" question and go back dozens of layers, usually suffering combinatorial explosion of causes, and even recursion in some cases. Only very, very rarely have I ever encountered a terminal, genesis cause for which there isn't a "why" -- the will to live is honestly the only one occurring to me right now. Everything else has causes upon causes as far as I'd care to look...

Comment author: Stuart_Armstrong 12 June 2017 01:33:29PM 0 points [-]

Oh, in that sense, yeah. I meant as in having articulated meta-preferences that explain lower level preferences.

Comment author: cousin_it 12 June 2017 08:50:54AM *  0 points [-]

Why? vNM utility maximization seems like a philosophical idea that's clearly on the right track. There might be other such ideas about being friendly to imperfect agents.

Comment author: Stuart_Armstrong 12 June 2017 01:32:37PM 0 points [-]

vNM is rationality - decisions.

Being friendly to imperfect agents is something I've seen no evidence for; it's very hard to even define.

Comment author: Lumifer 09 June 2017 08:41:50PM *  1 point [-]

I don't think it's false, it's more like implicitly conditioned on what you expect. I would say it unrolls into "I don't want to live past 100 given that I expect myself to be sick, feeble-minded, and maybe in permanent low-grade pain by that point".

Take away the implied condition and the preference will likely change as well.

Comment author: Stuart_Armstrong 12 June 2017 08:01:35AM 1 point [-]

Unfortunately the implied conditional is often a alief, not a belief. So if you say "imagine that you were healthy, smart, and happy..." they'll still often say they don't want to live that long. But if there were a lot of healthy, smart, happy 100 year olds, people would change their minds.

Comment author: cousin_it 09 June 2017 09:10:33PM *  2 points [-]

Many philosophical problems seem to have correct solutions, so I have some hope. For example, the Absent-Minded Driver problem is a philosophical problem with a clear correct solution. Formalizing the intuitive process that leads to solving such problems might be safer than solving them all up front (possibly incorrectly) and coding the solutions into FAI.

Comment author: Stuart_Armstrong 12 June 2017 07:43:37AM 0 points [-]

It seems that the problems to do with rationality have correct solutions, but not the problems to do with values.

Comment author: username2 09 June 2017 08:13:02PM *  0 points [-]

Is that really the standard definition of agent though? Most textbooks I've seen talk of agents working towards the achievement of a goal, but it says nothing about the permanence of that goal system. I would expect an "idealized agent" to always take actions that maximize likelihood of achieving its goals, but that is orthogonal from whether the system of goals changes.

Comment author: Stuart_Armstrong 12 June 2017 07:42:21AM 0 points [-]

Then take my definition of agent in this post as "expected utility maximiser with a clear and distinct utility that is, in practice, Cartesianianly separated from the rest of the universe", and I'll try and be clearer in subsequent posts.

Comment author: username2 09 June 2017 08:15:21PM 0 points [-]

Okay well that doesn't jive with my own introspective experience.

Comment author: Stuart_Armstrong 12 June 2017 07:40:18AM 0 points [-]

It seems to me to jive with how many people react to unexpected tensions between different parts of their values (eg Global warming vs markets solve everything, or Global warming vs nuclear power is bad). If the tension can't be ignored or justified away, they often seem to base their new decision on affect and social factors, far more than any principled meta-preference for how to resolve tensions.

Comment author: Dagon 09 June 2017 07:33:15PM 1 point [-]

Statements about preferences are not preferences. "I don't want to live past 100", for most people, is a false statement, not a contradictory desire.

Comment author: Stuart_Armstrong 12 June 2017 07:32:06AM 0 points [-]

It's a true statement, in that people will take actions that match up with that preference.

View more: Next