Less Wrong is a community blog devoted to refining the art of human rationality. Please visit our About page for more information.

Comment author: CronoDAS 22 March 2017 07:54:49PM *  5 points [-]

Home appliances have improved on measures other than durability, though, such as energy efficiency. And cars are significantly more durable, lasting for roughly twice the mileage before requiring repairs that amount to rebuilding the car...

Comment author: Luke_A_Somers 22 March 2017 10:58:51PM *  1 point [-]

Quite - I have a 10 year old car and haven't had to do anything more drastic than change the battery - regular maintenance kinds of stuff.

Comment author: Luke_A_Somers 22 March 2017 03:46:19PM 0 points [-]

This is about keeping the AI safe from being altered by bad actors before it becomes massively powerful. It is not an attempt at a Control Problem solution. It could still be useful.

Comment author: Lumifer 20 March 2017 03:20:41PM *  1 point [-]

What interesting ideas do you find here? This looks like a ranty of-course-it's-clear-in-the-rearview-mirror "wisdom" to me.

Comment author: Luke_A_Somers 20 March 2017 09:49:08PM 2 points [-]

A) the audit notion ties into having our feedback cycles nice and tight, which we all like here.

B) This would be a little more interesting if he linked to his advance predictions on the war so we could compare how he did. And of course if he had posted a bunch of other predictions so we could see how he did on those (to avoid cherry-picking). That would rule out rear-view-mirror effects.

Comment author: komponisto 20 March 2017 03:44:00AM 1 point [-]

The first link is MM saying what EY would later say in No Safe Defense, Not Even Science.

Comment author: Luke_A_Somers 20 March 2017 06:24:47PM 0 points [-]

Really? There seems a little overlap to me, but plenty of mismatch as well. Like, MM says Bayesians are on crack, as one of the main points of the article.

Comment author: Oscar_Cunningham 16 March 2017 08:49:13PM 2 points [-]

I think utility functions can produce more behaviours than you give them credit for.

  1. Humans don't have a utility function and make very incoherent decisions. Humans are also the most intelligent organisms on the planet. In fact, it seems to me that the less intelligent an organism is, the easier its behavior can be approximated with model that has a utility function!

The less intelligent organisms are certainly more predictable. But I think that the less intelligent ones actually can't be described by utility functions and are instead predictable for other reasons. A classic example is the Sphex wasp.

Some Sphex wasps drop a paralyzed insect near the opening of the nest. Before taking provisions into the nest, the Sphex first inspects the nest, leaving the prey outside. During the inspection, an experimenter can move the prey a few inches away from the opening. When the Sphex emerges from the nest ready to drag in the prey, it finds the prey missing. The Sphex quickly locates the moved prey, but now its behavioral "program" has been reset. After dragging the prey back to the opening of the nest, once again the Sphex is compelled to inspect the nest, so the prey is again dropped and left outside during another stereotypical inspection of the nest. This iteration can be repeated several times without the Sphex changing its sequence; by some accounts, endlessly.

So it looks like the wasp has a utility function "ensure the survival of its children" but in fact it's just following one of a number of fixed "programs". Whereas humans are actually capable of considering several plans and choosing the one they prefer, which I think is much closer to having a utility function. Of course humans are less predictable, but one would always expect intelligent organisms to be unpredictable. To predict an agent's actions you essentially have to mimic its thought processes, which will be longer for more intelligent organisms whether they use a utility function or not.

  1. The randomness of human decisions seems essential to human success (on top of other essentials such as speech and cooking). Humans seem to have a knack for sacrificing precious lifetime for fool's errands that very occasionally create benefit for the entire species.

If trying actions at random produces useful results then a utility maximising AI will choose this course. Utility maximisers consider all plans and pick the one with the highest expected utility, and this can turn out to be one that doesn't look like it goes directly towards the goal. Eventually of course the AI will have to turn its attention towards its main goal. The question of when to do this is known as the exploration vs. exploitation tradeoff and there are mathematical results that utility maximisers tend to begin by exploring their options and then turn to exploiting their discoveries once they've learnt enough.

To define a utility function is to define a (direction towards a) goal. So a discussion of an AI with one, single, unchanging utility function is a discussion of an AI with one, single, unchanging goal. That isn't just unlike the intelligent organisms we know, it isn't even a failure mode of intelligent organisms we know. The nearest approximations we have are the least intelligent members of our species.

Again I think that this sort of behaviour (acting towards multiple goals) can be exhibited by utility maximizers. I'll give a simple example. Consider the agent who can by any 10 fruits from a market, and suppose its utility function is sqrt(number of oranges) + sqrt(number of apples). Then it buys 5 oranges and 5 apples (rather than just buying 10 apples or 10 oranges). The important thing about the example is the the derivative of the utility function is decreasing as the number of oranges increases, and so the more it has already the more it will prefer to buy apples instead. This creates a balance. This is just a simple example but by analogy it would be totally possible to create a utility function to describe a multitude of complex values all simultaneously.

  1. Two agents with identical utility functions are arguably functionally identical to a single agent that exists in two instances. Two agents with utility functions that are not identical are at best irrelevant to each other and at worst implacable enemies.

Just like humans, two agents with different utility functions can cooperate through trade. The two agents calculate the outcome if they trade and the outcome if they don't trade, and they make the trade if the utility afterwards is higher for both of them. It's only if their utilities are diametrically opposed that they can't cooperate.

Comment author: Luke_A_Somers 17 March 2017 06:26:53PM 0 points [-]

Agreed on that last point particularly. Especially since, if they want similar enough things, they could easily cooperate without trade.

Like if two AIs supported Alice in her role as Queen of Examplestan, they would probably figure that quibbling with each other over whether Bob the gardener should have one or two buttons undone (just on the basis of fashion, not due to larger consequences) is not a good use of their time.

Also, the utility functions can differ as much as you want on matters aren't going to come up. Like, Agents A and B disagree on how awful many bad things are. Both agree that they are all really quite bad and all effort should be put forth to prevent them.

Comment author: MrMind 17 March 2017 01:13:33PM 4 points [-]

Rationalists like to live in group houses.

Do they? I personally hate sharing living spaces. Am I the weirdo? I suspect it's an American custom, not something proper of rationalists per se.

Comment author: Luke_A_Somers 17 March 2017 02:55:29PM 1 point [-]

An American Rationalist subculture question, perhaps. Certainly NOT America as a whole.

In response to Excuses and validity
Comment author: Luke_A_Somers 13 March 2017 08:13:19PM 1 point [-]

You say all excuses are equally valid and then turn around and say they're more or less valid. Do you mean that excuses people would normally think of making have a largely overlapping range of possible validities?

Comment author: Bound_up 07 March 2017 10:26:45PM 0 points [-]

Thanks, Luke

Can you clarify? The first part sounds like MWI is irrelevant to the question of fine-tuning of universal constants. Are you saying that if only one Everett branch was real, then it would be unlikely to have things like a planet under the right circumstances for life, but that is accounted for by MWI, since it explores all the permutations of a universe with constants like ours?

If I'm getting this, then that means MWI accounts for things like "why is the earth in the right place" kinds of things, but not "why is the proton this particular mass" kinds of things

Comment author: Luke_A_Somers 10 March 2017 07:50:04PM 0 points [-]

Well, if the laws of the universe were such that it were unlikely but not impossible for life to form, MWI would take care of the rest, yes.

BUT, if you combine MWI with something that sets the force laws and particle zoo of the later universe as an aspect of quantum state, then MWI helps a lot - instead of getting only one, it makes ALL† of those laws real.

† or in case of precise interference that completely forces certain sets of laws to have a perfectly zero component, nearly all. Or if half of them end up having a precisely zero component due to some symmetry, then, the other half of these rule-sets… etc. Considering the high-dimensional messiness of these proto-universe-theories, large swaths being nodal (having zero wavefunction) seems unlikely.

Comment author: Bound_up 28 February 2017 11:00:26AM 0 points [-]

Okay, but the best theory, MWI, does not suggest different constants, and the theory that does is not particularly well thought of, am I understanding this right?

So, this is a bad rebuttal to the fine-tuning argument.

Comment author: Luke_A_Somers 06 March 2017 05:55:54PM 0 points [-]

MWI is orthogonal to the question of different fundamental constants. MWI is just wavefunction realism plus no collapse plus 'that's OK'.

So, any quantum-governed system that generates local constants will do under MWI. The leading example of this would be String Theory.

MWI is important here because if only one branch is real, then you need to be just as lucky anyway - it doesn't help unless the mechanism makes an unusually high density of livable rules. That would be convenient, but also very improbable.

Comment author: sone3d 21 February 2017 07:48:21PM 1 point [-]

Sorry, I didn't expressed correctly. What I'm asking is "what should I believe"?

Comment author: Luke_A_Somers 27 February 2017 08:46:10PM 0 points [-]

Why should you believe any specific conclusion on this matter rather than remain in doubt?

View more: Next