jaime2000 comments on No Universally Compelling Arguments in Math or Science - Less Wrong

30 Post author: ChrisHallquist 05 November 2013 03:32AM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (227)

You are viewing a single comment's thread.

Comment author: jaime2000 05 November 2013 04:37:05PM *  17 points [-]

It seems obvious that people are using "universally compelling arguments" in two different senses.

In the first sense, a universally compelling argument is one that could convince even a rock, or a mind that doesn't implement modus ponens, or a mind with anti-inductive priors. In this sense, the lack of universally compelling arguments for any domain (math/physics/morality) seems sufficiently well established.

In another sense, a universally compelling argument is one that could persuade any sufficiently sane/intelligent mind. I think we can agree that all such minds will eventually conclude that relativity and quantum mechanics are correct (or at least a rough approximation to whatever the true laws of physics end up being), so in this sense we can call the arguments that lead to them universally compelling. Likewise, in this sense, we can note as interesting the non-existence of universally compelling arguments which could compel a sufficiently sane/intelligent paperclipper to value life, beauty, justice, and the American way. It becomes more interesting if we also consider the case of babyeaters, pebblesorters, or humans with values sufficiently different to our own.

You are using the term in the first sense, but the people who are bothered by it are using it in the second sense.

Comment author: SaidAchmiz 05 November 2013 06:22:42PM 8 points [-]

Except that "sufficiently sane/intelligent" here just means, it seems, "implements modus ponens, has inductive priors, etc." We can, like Nick Tarleton, simply define as "not a mind" any entity or process that doesn't implement these criteria for sufficient sanity/intelligence...

... but then we are basically saying: any mind that is not convinced by what we think should be universally compelling arguments, is not a mind.

That seems like a dodge, at best.

Are there different criteria for sufficient sanity and intelligence, ones not motivated by the matter of (allegedly) universally compelling arguments?

Comment author: Tyrrell_McAllister 05 November 2013 08:11:22PM 4 points [-]

Except that "sufficiently sane/intelligent" here just means, it seems, "implements modus ponens, has inductive priors, etc."

"Sufficiently sane/intelligent" means something like, "Has a sufficient tendency to form true inferences from a sufficiently wide variety of bodies of evidences."

Now, we believe that modus ponens yields true inferences. We also believe that a tendency to make inferences contrary to modus ponens will cause a tendency to make false inferences. From this you can infer that we believe that a sufficiently sane/intelligent agent will implement modus ponens.

But the truth of this inference about our beliefs does not mean that "sufficiently sane/intelligent" is defined to mean "implements modus ponens".

In particular, our definition of "sufficiently sane/intelligent" implies that, if A is a sufficiently sane/intelligent agent who lives in an impossible possible world that does not implement modus ponens, then A does not implement modus ponens.

Comment author: Armok_GoB 05 November 2013 11:29:57PM 2 points [-]

"sufficiently sane/intelligent" means "effective enough in the real world to pose a threat to my values". Papercillper qualifies, flue virus qualifies, anti-inductive AI does not qualify.

Comment author: Eugine_Nier 06 November 2013 05:59:29AM 7 points [-]

So, how is the project to teach mathematics to the flue virus going?

Comment author: Armok_GoB 06 November 2013 03:44:22PM 2 points [-]

Why, it hasn't been wrong about a single thing so far, thank you!

Comment author: TheAncientGeek 05 November 2013 06:29:53PM 2 points [-]

... but then we are basically saying: any mind that is not convinced by what we think should be universally compelling arguments, is not a mind.

That doesn't follow. For one thing, we can find out how the Mind works by inspecting its code, not just by black box testing it If it seems to have all that it needs and isn't convinced by arguments that convince us, it may well be we who are wrong.

Comment author: SaidAchmiz 05 November 2013 07:09:45PM *  1 point [-]

For one thing, we can find out how the Mind works by inspecting its code

We can?

So I have all these minds around me.

How do I inspect their code and thereby find out how they work? Detailed instructions would be appreciated. (Assume that I have no ethical restrictions.)

That (only slightly-joking) response aside, I think you have misunderstood me. I did not mean that we are (in the scenario I am lampooning) saying:

"Any mind that is not convinced by what we think should be universally compelling arguments, despite implementing modus ponens and having an Occamian prior, is not a mind."

Rather, I meant that we are saying:

"Any mind that is not convinced by what we think should be universally compelling arguments, by virtue of said mind not implementing modus ponens, having an Occamian prior, or otherwise having such-and-such property which would be required in order to find this argument compelling, is not a mind."

The problem I am pointing out in such reasoning is that we can apply it to any argument we care to designate as "this ought to be universally compelling". "Ah!" we say, "this mind does not agree that ice cream is delicious? Well, that's because it doesn't implement <whatever happens to be required of a mind in order for it to find ice cream delicious>, and without said property, why, we can hardly call it a mind at all."

A rationality quote of sorts is relevant here:

"Well, let's put it like this. A human has encountered an extraterrestrial lifeform. How do they each discover, that they are both intelligent?"

"I have no idea," said Valentine merrily. "All that I have read on this subject reduces to a vicious circle. If they are capable of contact, then they are intelligent. And the reverse: if they are intelligent, then they are capable of contact. And in general: if an extraterrestrial lifeform has the honor of possessing a human psychology, then it is intelligent. Like that."

(Roadside Picnic, Arkady and Boris Strugatsky)

What we have here is something similar. If a mind is sufficiently sane/intelligent, then it will be convinced by our arguments. And the reverse: if it is convinced by our arguments, then it is sane/intelligent...

In yet other words: we can hardly say "we expect all sane/intelligent minds to be convinced by these arguments" if we have in the first place defined sanity and intelligence to require the ability to be convinced by those very arguments.

Comment author: TheAncientGeek 05 November 2013 07:15:55PM -2 points [-]

No, it's not viciously circular to argue that an entity that fulfills all the criteria for being an X is an X.

Comment author: SaidAchmiz 05 November 2013 07:25:52PM -1 points [-]

That's not what is happening here. Is what I wrote actually unclear? Please reread my comment, starting with the assumption that what you responded with is not what my intended meaning was. If still unclear, I will try to clarify.

Comment author: [deleted] 05 November 2013 08:39:56PM *  2 points [-]

Yes. You can convince a sufficiently rational paperclip maximizer that killing people is Yudkowsy::evil, but you can't convince it to not take Yudkowsy::evil actions, no matter how rational it is. AKA the orthogonality thesis (when talking about other minds) and “the utility function is not up for grabs” (when talking about ourselves).