whpearson comments on My problems with Formal Friendly Artificial Intelligence work - Less Wrong

4 Post author: whpearson 06 December 2016 08:31PM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Article Navigation

Comments (26)

Comments (26)

You are viewing a single comment's thread. Show more comments above.

Comment author: whpearson 07 December 2016 08:32:38PM *  2 points [-]

2+2=4 pays rent.

Ugh can use it right away for counting days. Source code based decision theory not so much. There aren't the societies based on agents that can read each others source code, so I can't try and predict them with source code based decision theories. It seems like it is mathematically interesting thing though, so it is still interesting. I just don't want it to be a core part of our sole pathway to try and solve the AI problem.

Would a recommendation to skip solving the simplified problems, because they don't include all the messy details, more likely lead to prople solving the hard problems faster, or giving up?

Perhaps it would lead people to avoid trying to find optimal decision theories and accept the answer that the best decision theory depends on the circumstances. Then we can figure out what our circumstances are and find good decisions theories for those. And create designs that can do similarly.

Like the best search algorithm is context dependent, where even algorithms of a worst complexity class can be better due to memory locality and small size.

How specifically are you planning to use this insight in constructing a Friendly superintelligent AI?

Supposition> If answers to how decisions are made (and a whole host of other problems) are contextual and complex then it is worth trading information about what answers they have found within their context.

  1. Figure the control flows between parts of the human brain that means they manage to roughly align themselves into a coherent entity (looking at dopamine etc).
  2. Apply the same informational constraints to computers so that we can link a computer up to a human (it doesn't need to be physically as long as the control flows work). The computer should be aligned with the user as much as a part of the brain is aligned to another part (which is hopefully sufficient).
  3. While this is in an embryonic stage get as many people from all around the world to use it .
  4. Hard take off (I think this less likely due to the context sensitivity things I have described, but still possible)- it is worthwhile to trade between agents so groups of human/computers that co-operate and trade information advance quicker. than loan rogue agents. If we have done the alignment work right then it is likely the weight of them will be able to squash any human inimical systems that appear.

This pathway makes no sense from people that expect there to be winner take all optimal AI designs as any one embryonic system might find the keys to the future and take it over. But if that is not the way the world works....

Comment author: Viliam 08 December 2016 12:03:28PM *  2 points [-]

There are mathematical concepts that didn't pay rent immediately: imaginary numbers, quaternions, non-Euclidean geometry, Turing machines...

But thanks for the specific plan, it sounds like it could work.

Comment author: whpearson 08 December 2016 06:59:12PM 1 point [-]

If you have any interest in my work into a hypothesis for 1 let me know.

Comment author: The_Jaded_One 10 December 2016 01:32:49PM 1 point [-]

There aren't the societies based on agents that can read each others source code,

Human beings can probabilistically read each others' source code. That's why we use primitive versions of noncausal decision theory like getting angry, wanting to take revenge, etc.

Comment author: whpearson 10 December 2016 03:13:24PM 0 points [-]

Human beings can probabilistically read each others' source code

This seems like a weird way of say, humans can make/refine hypotheses about other agents. What does talking about source code give you?

That's why we use primitive versions of noncausal decision theory like getting angry, wanting to take revenge, etc.

Tit for tat (which seems like revenge) works in normal game theories for IPD (of infinite length) which is a closer to what we experience in everyday life. I thought Non-causal decision theories are needed for winning on one-shots?

Comment author: The_Jaded_One 10 December 2016 06:29:34PM *  2 points [-]

In the case of humans, "talking about source code" is perhaps not that useful, though we do have source code, it's written in quaternary and has a rather complex probabilistic compiler. And that source code was optimized by a purely causal process, demonstrating the fact that causal decision theory agents self modify into acausal decision theory agents in many circumstances.

Revenge and anger work for one shot problems, for example if some stranger comes and sexually assaults your wife, they cannot escape your wrath by "saying oh it's only one shot, I'm a stranger in a huge city you'll never see me again so there's no point taking revenge". You want to punch the in the face as an end in itself now, this is a simple way of our brains being a bit acausal, decision theory wise.

Comment author: whpearson 11 December 2016 09:55:38PM 0 points [-]

I thought anger and revenge (used in one shot situations) might be generalising from what to do in the iterated version which is what we had for more of our evolutionary history.

I kinda like a-causal decision theory for choosing to vote at all. I will choose to vote so that other people like me choose to vote.

Powered by Reddit Powered by Reddit