= 27d567bc2d48d9f40e9ccc20ea370b15)
CharlesR comments on Muehlhauser-Wang Dialogue - Less Wrong
You are viewing a comment permalink. View the original post to see all comments and the full post content.
= 783df68a0f980790206b9ea87794c5b6)
Loading…= b715d02e85f7b3b4ae65ca3d3e450868)
Subscribe to RSS Feed
Powered by Reddit
= f037147d6e6c911a85753b9abdedda8d)
You are viewing a comment permalink. View the original post to see all comments and the full post content.
Comments (284)
First, thank you for publishing this illuminating exchange.
I must say that Pei Wang sounds way more convincing to an uninitiated, but curious and mildly intelligent lay person (that would be me). Does not mean he is right, but he sure does make sense.
When Luke goes on to make a point, I often get lost in a jargon ("manifest convergent instrumental goals") or have to look up a paper that Pei (or other AGI researchers) does not hold in high regard. When Pei Wang makes an argument, it is intuitively clear and does not require going through a complex chain of reasoning outlined in the works of one Eliezer Yudkowsky and not vetted by the AI community at large. This is, of course, not a guarantee of its validity, but it sure is easier to follow.
Some of the statements are quite damning, actually: "The “friendly AI” approach advocated by Eliezer Yudkowsky has several serious conceptual and theoretical problems, and is not accepted by most AGI researchers. The AGI community has ignored it, not because it is indisputable, but because people have not bothered to criticize it." If one were to replace AI with physics, I would tend to dismiss EY as a crank just based on this statement, assuming it is accurate.
What makes me trust Pei Wang more than Luke is the common-sense statements like "to make AGI safe, to control their experience will probably be the main approach (which is what “education” is all about), but even that cannot guarantee safety." and "unless you get a right idea about what AGI is and how it can be built, it is very unlikely for you to know how to make it safe". Similarly, the SIAI position of “accelerate AI safety research and decelerate AI capabilities research so that we develop safe superhuman AGI first, rather than arbitrary superhuman AGI” rubs me the wrong way. While it does not necessarily mean it is wrong, the inability to convince outside experts that it is right is not a good sign.
This might be my confirmation bias, but I would be hard pressed to disagree with "To develop a non-trivial education theory of AGI requires a good understanding about how the system works, so if we don’t know how to build an AGI, there is no chance for us to know how to make it safe. I don’t think a good education theory can be “proved” in advance, pure theoretically. Rather, we’ll learn most of it by interacting with baby AGIs, just like how many of us learn how to educate children."
As a side point, I cannot help but wonder if the outcome of this discussion would have been different were it EY and not LM involved in it.
Edit: Yeah, this was meant as a quote.
The question is whether "AGI researchers" are experts on "AI safety". If the answer is "yes", we should update in their direction simply because they are experts. But if the situation is like mine, then Pei Wang is committing argumentum ad populum. Not only should we not pay attention, we should point this out to him.
(You may want to put "cryonics" between square brackets, I nearly missed this deviation from the original quote.)
The grandparent is a quote? That probably should be indicated somehow. I was about to reply as if it was simply his words.
Point (4) of the first reply from Pei Wang. I didn't noticed, but there are other deviations from the original phrasing, to eliminate direct references to the AGI community. It merely refers to "people" instead, making it a bit of a straw man. Charles' point may still stand however, if most of the medical profession thinks cryonics doesn't work (meaning, it is a false hope).
To make a quote, put a "
>" at the beginning of the first line of the paragraph, like you would in an e-mail:Oh, it's that simple? How do you find this sort of thing out?
LessWrong is based on Reddit code, which uses Markdown syntax. It's based on email conventions. Clik on the "Show Help button" at the bottom-right of your editing window when you write a comment, it's a good quick reference.
Your introduction style is flawless. I was expecting either a daringfireball link or a mention of the 'Help' link but you have included both as well as a given the history and explained the intuitive basis.
I hope you'll pardon me for playing along a little there. It was a novel experience to be the one receiving the quoting instructions rather than the one typing them out. I liked the feeling of anonymity it gave me and wanted to see if that anonymity could be extended as far as acting out the role of a newcomer seeking further instructions.
Pleased to meet you loup-vaillant and thank you for making my counterfactual newcomer self feel welcome!
You got me. Overall, I preffer to judge posts by their content, so I'm glad to learn of your trick.
For the record, even I expected to stop at the Daring Fireball link. I also wrote a bunch of examples, but only then noticed/remembered the "Show help" button. I also erased a sentence about how to show markdown code in markdown (it's rarely useful here, there was the Daring Fireball link, and my real reason for writing it was to show off).
I tend to heavily edit my writings. My most useful heuristic so far is "shorter is better". This very comment benefited from it (let's stop the recursion right there).
It seemed gentler than responding with a direct challenge to the inference behind the presumption.