Comment Permalink

Oliver Siegel2y10

Fair point! But how do you know that this ungrounded mysticism doesn't apply to current debate about the potential capabilities of AI systems?

Why is an AI suddenly able to figure out how to break the laws of physics and be super intelligent about how to end intelligent life, but somehow incapable of comprehending the human laws of ethics and morality, and valuing life as we know it?

What makes the laws of physics easier to understand and easier to circumvent than the human laws of ethics and morality? (And also, navigating the human laws of ethics and morality must be required for ending all life. Unless software suddenly has the same energy as enriched plutonium or something like that, and one wrong bit flip causes an explosive chain reaction)

What makes it so much more difficult to understand critical thinking and "how to store human values in a computer", and in contrast what makes "accidentally ending all intelligent life" so easy, by comparison?

It seems to me that "ASI on mission to destroy the humans" is the same thing as "luminiferous aether".

We taught AI English and how to draw pictures and create art. Both pretty "fuzzy" things.

How hard can it be to train AI on a dataset of 90% of known human values and 90% of known problems and solutions with respect to those values for a neural net to have an "above average human"-grasp on the idea that "ending all intelligent life" computes as "that's a problem and it's immoral" ?

Beyond that, alignment is unsolvable anyways for AGI systems that perform at above human intelligence. Can't predict the future with a software, because there could always be software that uses the future predicting software and negates the output - aka the Halting Problem. Can't do anything about that.

See in context

-12

How to store human values on a computer

by Oliver Siegel

5th Nov 2022

1 min read

-12

One of the main obstacles to building safe and aligned AI is that we don't know how to store human values on a computer.

Why is that?

Human values are abstract feelings and intuitions that can be described with words.

For example:

Freedom or Liberty
Happiness or Welfare
Justice
Individual Sovereignty
Truth
Physical Health & Mental Health
Prosperity & Wealth

We see that AI systems like GPT-3 or other NLP based systems use Word2Vec or other node networks / knowledge graphs to encode words and their meanings as complex relationships. This then allows software developers to parameterize language so that a computer can do Math with it, and transform complicated formulas into a human readable output. Here's a great video on how computers understand language using Word2Vec . Besides that, ConceptNet another great example of a linguistic knowledge graph.

So, what we have to do is find a knowledge graph that helps us encode human values, so that computers can better understand human values and learn to operate within those parameters, not outside of them.

My suggestion for how to do this is quite simple: Categorize each "positive value goal" according to the actionable, tangible methods and systems that help fulfill this positive value goal, and then contrast that with all the negative problems that exist in the world with respect to that positive value goal.

Using the language of the instrumental convergence thesis:

Instrumental goals are the tangible, actionable methods and systems (also called solutions) that help to fulfill terminal goals.
Problems on the other hand are descriptions of how terminal goals are violated or unfulfilled.

There can be many solutions associated to each positive value goal, as well as multiple problems.

Such a system lays the foundation for a node network or knowledge graph of our human ideals and what we consider "good".

Would it be a problem if AI was instrumentally convergent on doing the most good and solving the most problems? That's something I've put up for discussion here and I'd be curious to hear your opinion in the comments!

Frontpage

-12

How to store human values on a computer

New Comment

17 comments, sorted by

top scoring

Click to highlight new comments since: Today at 6:21 PM

[-]Ruby2y61

Moderator here. I approved this post but strong downvoted it (according to the "votes are for what you want to see more/less of" approach). It's not exactly obviously super wrong, but feels perhaps too basic and I don't want a tonne more of this on the site. Perhaps it feels like it's written without building on the Sequences which touches a lot on this. Though I could be pretty wrong and we should encourage people to think about AI wherever there at. Not sure. I think engaging on all previous AI content is unnecessary, but I will downvote if i think content doesn't feel like the author has read the Sequences. Hence downvote but still approved.

[-]Oliver Siegel2y10

Thank you! Could I get a link to "The Sequences" ? I can't find it here: https://www.lesswrong.com/tags/all

[-]quanticle2y53

https://www.readthesequences.com/

Edit: Specifically, you may wish to read: https://www.readthesequences.com/The-Hidden-Complexity-Of-Wishes

[-]Oliver Siegel2y00

Thanks for sharing! Yes, is seems that the computational complexity could indeed explode at some point.

But then again, an average human brain is capable of storing common sense values and ethics, so unless there's a magic ingredient in the human brain, it's probably not impossible to rebuild it on a computer.

Then, with an artificial brain that has all the benefits of never fatiguing and such, we may come close to a somewhat useful Genie that can at least advise on the best course of action given all the possible pitfalls.

Even if it'll just be, say 25% better than the best human - all humans could get access to this Genie on their Smartphone, how cool would that be?

But I'll have to dig deeper into The Sequences, seems very comprehensive.

I found Monica Anderson's blog quite inspiring, as well. She writes about model free, holistic systems. https://experimental-epistemology.ai/

[-]quanticle2y53

But then again, an average human brain is capable of storing common sense values and ethics, so unless there’s a magic ingredient in the human brain, it’s probably not impossible to rebuild it on a computer.

Of course it's possible to rebuild human morality on a computer. There is, however, a vast unfathomable chasm between possible and easy.

[-]Oliver Siegel2y10

Yea, i agree!

But if it were easy, everyone would do it... ;p

Based on your knowledge, what do you think might be the biggest hurdles to making it possible, using a system similar like the one i described above?

[-]quanticle2y83

The biggest problem, as I see it, is that you haven't come to a thorough understanding of what is it that you mean by "all the actionable tangible methods and systems that help fulfill this positive value goal and then contrast with all the negative problems that exist in the world that exist with respect to that positive goal".

In other words, what you've written there is just, "Make the computer do good things, and also make the computer not do bad things."

Yes, it would be wonderful if we could make the computer just do good things and make the computer not do bad things. But if it were that easy, AI alignment would be a trivial problem.

Edit: Mysterious Answers To Mysterious Questions is a good sequence post that explains the issues with your approach.

[-]Oliver Siegel2y10

Interesting article! It reminds me of Monica Anderson's blog: https://experimental-epistemology.ai/

She embraces the mysticism and proposes that holistic, non-reductionist model free systems are undeniably effective.

> "The biggest problem, as I see it, is that you haven't come to a thorough understanding of what you mean"

That's another one, that Monica writes a lot about: Understanding.

What does is mean to understanding something? And what is the meaning of meaning?

Yes, they sounds like metaphysical, mystical ideas, and they might be fundamentally unsolvable (See: Hard problem of consciousness or explanatory gap).

But we already see that it must be possible for systems to exist in this universe, that are aligned within a group. Many groups of humans use their brains to figure out how to coexist peacefully.

So unless humans posses a mystical ingredient, it must be possible to recreate this sort of understanding in machines.

But we don't currently know how to teach this to this machines, in part because we lack a good dataset, and we don't understand it, ourselves.

Do you think that alignment is fundamentally an engineering problem, or is it one of the humanities and one of philosophy?

[-]quanticle2y20

It's an engineering problem. If I'm honest, I see essentially zero room for the humanities in AI alignment. The level of fuzzy-thinking and lack of rigor that characterizes the humanities is a hindrance, where alignment is concerned.

In other words, we can discuss the philosophical implications of having machines that "understand", after we've implemented the guardrails that prevent those machines from ending intelligent life.

EDIT: I read the first two articles on the blog that you linked, and I found it to be a classic example of what "Mysterious Answers To Mysterious Questions" is warning about. "Understanding" is used the same way that ancient natural philosophers used "phlogiston", or "elan vital" or "luminiferous aether".

[-]Oliver Siegel2y10

Fair point! But how do you know that this ungrounded mysticism doesn't apply to current debate about the potential capabilities of AI systems?

It seems to me that "ASI on mission to destroy the humans" is the same thing as "luminiferous aether".

We taught AI English and how to draw pictures and create art. Both pretty "fuzzy" things.

[-]quanticle2y20

Why is an AI suddenly able to figure out how to break the laws of physics and be super intelligent about how to end intelligent life, but somehow incapable of comprehending the human laws of ethics and morality, and valuing life as we know it?

Why do you think an AI would need to break the laws of physics in order to become superintelligent? As Eliezer and gwern have pointed out, the laws of physics are no bar to a machine achieving power beyond our capability to stop.

What makes it so much more difficult to understand critical thinking and “how to store human values in a computer”, and in contrast what makes “accidentally ending all intelligent life” so easy, by comparison?

"Accidentally ending all intelligent life" is the default outcome. It's what happens when you program a self-optimizing maximizing process and unleash it. As Eliezer said, once, "The AI does not hate you. The AI does not fear you. The AI merely sees that you are composed of atoms that it could use for its own purposes."

Furthermore, why do you think the comprehension is the problem? A superintelligence may fully comprehend human values, but it might be programmed in a way where it just doesn't care. A superintelligent AI tasked with maximizing the number of paperclips in the universe will of course be capable of comprehending human morality and ethics. It might even say that it agrees. But its utility function is fixed. Its goal is to maximize paperclips. It will do whatever it can to maximize the number of paperclips and if that happens to go against what it knows of human morality, well, so much the worse for human morality, then.

How hard can it be to train AI on a dataset of 90% of known human values and 90% of known problems and solutions with respect to those values for a neural net to have an “above average human”-grasp on the idea that “ending all intelligent life” computes as “that’s a problem and it’s immoral” ?

I look forward to you producing such a database.

Beyond that, alignment is unsolvable anyways for AGI systems that perform at above human intelligence. Can’t predict the future with a software, because there could always be software that uses the future predicting software and negates the output—aka the Halting Problem. Can’t do anything about that.

That is a misunderstanding of the Halting Problem.

[-]Ruby2y20

The Sequences is the original name, but it got edited down and renamed to "Rationality:A-Z"

https://www.lesswrong.com/rationality

[-]Maxwell Clarke2y11

I think you might also be interested in this: https://www.lesswrong.com/posts/Nwgdq6kHke5LY692J/alignment-by-default In general John Wentworths alignment agenda is essentially extrapolating your thoughts here and dealing with the problems in it.

It's unfortunate but I agree with Ruby- your post is fine but a top-level lesswrong post isn't really the place for it anymore. I'm not sure where the best place to get feedback on this kind of thing is (maybe publish here on LW but as a short-form or draft?) - but you're always welcome to send stuff to me! (Although busy finishing master's next couple of weeks)

[-]Oliver Siegel2y10

Thank you for the resource!

I'm planning to continue publishing more details about this concept. I believe it will address many of the things mentioned in the post you linked.

Instead of posting it all at once, I'm posting it in smaller chunks that all connect.

I have something coming up about preventing instrumental convergence with formalized critical thinking, as well as a general problem solving algorithm. It'll hopefully make sense once it's all there!

[-]Maxwell Clarke2y11

Respect for thinking about this stuff yourself. You seem new to alignment (correct me if I'm wrong) - I think it might be helpful to view posting as primarily about getting feedback rather than contributing directly, unless you have read most of the other people's thoughts on whichever topic you are thinking/writing about.

[-]Oliver Siegel2y10

Absolutely, I'm here for the feedback! No solution should go without criticism, regardless of what authority posted the idea, or how much experience the author has. :)

[-]Maxwell Clarke2y10

Oh or EA forum, I see it's crossposted

Moderation Log