All of aleph_four's Comments + Replies

If there is no solution to the alignment problem within reach of human level intelligence, then the AGI can’t foom into an ASI without risking value drift…

A human augmented by a strong narrow AIs could in theory detect deception by an AGI. Stronger interpretability tools…

What we want is a controlled intelligence explosion, where an increase in strength of the AGI leads to an increase in our ability to align, alignment as an iterative problem…

A kind of intelligence arms race, perhaps humans can find a way to compete indefinitely?

3Jeffrey Ladish
Yeah it seems possible that some AGI systems would be willing to risk value drift, or just not care that much. In theory you could have an agent that didn't care if its goals changed, right? Shoshannah pointed out to me recently that humans have a lot of variance in how much they care if they're goals are changed. Some people are super opposed to wireheading, some think it would be great. So it's not obvious to me how much ML-based AGI systems of around human level intelligence would care about this. Like maybe this kind of system converges pretty quickly to coherent goals, or maybe it's the kind of system that can get quite a bit more powerful than humans before converging, I don't know how to guess at that.

I love being accused of being GPT-x on Discord by people who don't understand scaling laws and think I own a planet of A100s

There are some hard and mean limits to explainability and there's a real issue that a person that correctly sees how to align AGI or that correctly perceives that an AGI design is catastrophically unsafe will not be able to explain it. It requires super-intelligence to cogently expose stupid designs that will kill us all. What are we going to do if there's this kind of coordination failure?

People have poor introspective access to the reasons why they like or dislike something; when they are asked for an explanation, they often literally fabricate their reasons.

omg, they literally work that way. I can't, let me off

Let’s add another Scott to our coffers.

1mako yass
The other other Scott A

Lately I’ve been requiring a higher bar than the Turing Test. I propose “Anything that can program, and can converse convincingly in natural language about what it is programming must be thinking.”

2Matt Goldenberg
I feel like this too low a bar, a programmer can still give the program the rules for what programming is, and the rules for what language is. A true AGI must make its' own rules. /s

uh... I guess cannot get around the regress involved in claiming my moral values superior to competing systems in an objective sense? I hesitate to lump together the same kind of missteps that are involved with a mistaken conception of reality (a mis-apprehension of non-moral facts) with whatever goes on internally when two people arrive at different values.

I think it’s possible to agree on all mind independent facts, without entailing perfect accord on all value propositions, and that moral reflection is fully possible without objective moral truth. Perha

... (read more)
1TAG
It looks like some people can, since the attitudes of professional philosophers break down as: 1. Meta-ethics: moral realism 56.4%; moral anti-realism 27.7%; other 15.9%. I can see how the conclusion would be difficult to reach if you make assumptions that are standard round here, such as 1. Morality is value 2. Morality is only value 3. All value is moral value. But I suppose other people are making other assumptions. Some verdicts lead to jail sentences. If Alice does something that is against Bob's subjective value system, and Bob does something that is against Alice's subjective value system, who ends up in jail? Punishments are things that occur objectively, so need an objective justification. Subjective ethics allows you to deliver a verdict in the sense of "tut-tutting", but morality is something that connects up with laws and punishments, and that where subjectivism is weak.

Well, i struggle to articulate what exactly we disagree on, because I find no real issue with this comment. Maybe i would say “high philosophical ability/sophistication causes both intergalactic civilization and moral convergence.”? I hesitate to call the result of that moral convergence “moral fact,” though I can conceive of that convergence.

uhh, it goes to sleep after a bit, but brings you back to what you were last doing.

The OCR doesn’t destroy the original

convert lines into appropriate geometric forms

Nope on this

convert text blocks to calendar entries, tickets, mails,...

Nope

I’m immensely skeptical that open individualism will ever be more than a minority position (among humans, at least) But at any rate, convergence on an ethic doesn’t demonstrate objective correctness of that ethic from outside that ethic.

Most intelligent beings in the multiverse share similar preferences.

I mean this could very well be true, but at best it points to some truths about convergent psychological evolution.

This came about because there are facts about what preferences one should have, just like there exist facts about what decision theory one should use or what prior one should have, and species that manage to build intergalactic civilizations

Sure, there are facts about what preferences would best enable the emergence of an intergalactic civilization. I struggle to see th

... (read more)
2Wei Dai
There's a misunderstanding/miscommunication here. I wasn't suggesting "what preferences would best enable the emergence of an intergalactic civilization" are moral facts. Instead I was suggesting in that scenario that building an intergalactic civilization may require a certain amount of philosophical ability and willingness/tendency to be motivated by normative facts discovered through philosophical reasoning, and that philosophical ability could eventually enables that civilization to discover and be motivated by moral facts. In other words, it's [high philosophical ability/sophistication causes both intergalactic civilization and discovery of moral facts], not [discovery of "moral facts" causes intergalactic civilization].

ReMarkable solves some of these issues. I am now at the point where I have so many notes written on traditional paper that I do not want more to accumulate more and I cannot efficiently consult them without OCR and search functionality

4Gunnar_Zarncke
Thank you. I looked into ReMarkable and it seems very alike to the Boox that I got as a present and that I'm utterly disappointed with (yes really). Can you tell me whether it has * a screensaver mode that just keeps on the screen what was last visible? * an OCR that does not throw the original notes away (the arrangement e.g. in a mindmap)

I’m not entirely sure what moral realism even gets you. Regardless of whether morality is “real” i still have attitudes towards certain behaviors and outcomes, and attitudes towards other people’s attitudes. I suspect the moral realism debate is confused altogether.

1TAG
It gets you something that error theory doesn't get you , which is that moral claims have truth values. And it gets you something that subjectivism doesn't get you, which is some people being actually wrong, and not just different to you. That's parallel to pointing out that people still have opinions when objective truth is available. People should believe the truth (this site, passim) and similarly should follow the true morality.
1Rafael Harth
To make Wei Dai's answer more concrete, suppose something like the symmetry theory of valence is true; in that case, there's a crisp, unambiguous formal characterization of all valence. Then add open individualism to the picture, and it suddenly becomes a lot more plausible that many civilizations converge not just towards similar ethics, but exactly identical ethics.
4Wei Dai
Here's what I wrote in Six Plausible Meta-Ethical Alternatives: "Most intelligent beings in the multiverse share similar preferences. This came about because there are facts about what preferences one should have, just like there exist facts about what decision theory one should use or what prior one should have, and species that manage to build intergalactic civilizations (or the equivalent in other universes) tend to discover all of these facts. There are occasional paperclip maximizers that arise, but they are a relatively minor presence or tend to be taken over by more sophisticated minds." In the above scenario, once you become intelligent enough and philosophically sophisticated enough, you'll realize that your current attitudes are wrong (or right, as the case may be) and change them to better fit the relevant moral facts.

money is the materialization of credit

woah, a marvelous inversion

Before I even got to your comment, I was thinking “You can pry my laptop out of my cold dead hands Marx!”

Thank you for this clarification on personal vs private property.

As of right now, I think that if business-as-usual continues in AI/ML, most unskilled labor in the transportation/warehousing of goods will be automatable by 2040.

Scott Anderson, Amazon’s director of Robotics puts it at over 10 years. https://www.theverge.com/2019/5/1/18526092/amazon-warehouse-robotics-automation-ai-10-years-away.

I don’t think it requires any fundamental new insights to happen by 2040, only engineering effort and currently available techniques.

I believe the economic incentives will align with this automation once it becomes achievable.

Tran

... (read more)
4mako yass
Can I infer via nominative determinism that Scott Anderson is a friend of the rationalist community?

Mathematics is more real than money certainly. If we collectively agree that money has no value, it has no value. If we collectively agree that mathematics has no use, it does not stop being an unreasonably effective abstraction for describing natural phenomena.

Well if qualia aren’t epiphenomenal then an accurate simulation must include them or deviate into errancy. Claiming that you could accuracy simulate a human but leave out consciousness is just the p-zombie argument in different robes