I think you should post this as its own thread in Discussion.
This has been proposed before, and on LW is usually referred to as "Oracle AI". There's an entry for it on the LessWrong wiki, including some interesting links to various discussions of the idea. Eliezer has addressed it as well.
See also Tool AI, from the discussions between Holden Karnofsky and LW.
Count me surveyed.
Interesting. I wonder to what extent this corrects for people's risk-aversion. Success is evidence against the riskiness of the action.
Having circular preferences is incoherent, and being vulnerable to a money pump is a consequence of that.
I knew that if I had 0.95Y I would trade it for (0.95^2)Z, which I would trade for (0.95^3)X, then actually I'd be trading 1X for (0.95^3)X, which I'm obviously not going to do.
This means that you won't, in fact, trade your X for .95Y. That in turn means that you do not actually value X at .9Y, and so the initially stated exchange rates are meaningless (or rather, they don't reflect your true preferences).
Your strategy requires you to refuse all tr...
Judging from the comments this is receiving on Hacker News, this post is a mindkiller. HN is an audience more friendly to LW ideas than most, so this is a bad sign. I liked it, but unfortunately it's probably unsuitable for general consumption.
I know we've debated the "no politics" norm on LW many times, but I think a distinction should be made when it comes to the target audience of a post. In posts aimed to make a contribution to "raising the sanity waterline", I think we're shooting ourselves in the foot by invoking politics.
Reading that HN thread, the problem appears to be a troll (who also showed up on Yvain's original blog post).
I like the combination of conciseness and thoroughness you've achieved with this.
There are a couple of specific parts I'll quibble about:
Therefore the next logical step is to use science to figure out how to replace humans by a better version of themselves, artificial general intelligence.
"The Automation of Science" section seems weaker to me than the others, perhaps even superfluous. I think the line I've quoted is the crux of the problem; I highly doubt that the development of AGI will be driven by any such motivations.
...Will we be able to
Out of curiosity, what are your current thoughts on the arguments you've laid out here?
I agree. I've noticed an especially strong tendency to premature generalization (including in myself) in response to people asking for advice. Tell people what your experiences were, not (just) the general conclusions you drew from them.
Is Omega even necessary to this problem?
I would consider transferring control to staply if and only if I were sure that staply would make the same decision were our positions reversed (in this way it's reminiscent of the prisoner's dilemma). If I were so convinced, then shouldn't I consider staply's argument even in a situation without Omega?
If staply is in fact using the same decision algorithms I am, then he shouldn't even have to voice the offer. I should arrive at the conclusion that he should control the universe as soon as I find out that it can prod...
I dunno, I think it is. It took me several hours of reflection to realize that it could be framed in those terms. The show didn't do any breaking.
Yes, thanks. I wanted to use strikethrough but a) I couldn't figure out how to do it in LW's markdown and b) it wouldn't work anyway if you copy/paste to rot13.com like I do.
I mostly agree with you. In particular I really liked that Znqbxn'f jvfu jrag fb sne nf gb erjevgr gur havirefr. Gur fbhepr bs ure rzbgvbaf orvat sbe gur zntvpny tveyf naq gur pehrygl bs gur onetnva gurl znqr, V jnf npghnyyl n yvggyr jbeevrq va gur yrnq-hc gb gur svanyr gung ure jvfu jbhyqa'g or zbzragbhf rabhtu.
Ng gur fnzr gvzr, gubhtu gur jvfu raqrq hc ovt rabhtu gb or n fngvfslvat raq, V guvax vg'f cerggl rnfl gb jbaqre jul fur pbhyqa'g tb shegure. Gur arj havirefr vf arneyl vqragvpny gb gur byq bar, evtug qbja gb vaqvivqhny crbcyr. Gur zntvpny tveyf ab...
I nearly posted exactly this earlier today.
It's an excellent show, though don't expect too much rationality. Madoka is no HP:MoR, but since there is very little rationality-relevant content in anime it does stand out.
For me it was a case of two independent interests unexpectedly having some crossover. As a fan of SHAFT (the animation studio) and mahou shoujo in general, it was a given I was going to watch Madoka. Then fhcreuhzna vagryyvtraprf naq vasbezngvba-nflzzrgevp artbgvngvba?
In a classic mahou shoujo setup like this, with magical powers and wish-gra...
As one of the 83.5%, I wish to point out that you're misinterpreting the results of the poll. The question was: "Which disaster do you think is most likely to wipe out greater than 90% of humanity before the year 2100?" This is not the same as "unfriendly AI is the most worrisome existential risk".
I think that unfriendly AI is the most likely existential risk to wipe out humanity. But I think that an AI singularity is likely farther off than 2100. I voted for an engineered pandemic, because that and nuclear war were the only two risks I...
I just took the survey. I was pretty sure I remembered the decade of Newton's book, but I was gambling on the century and I lost.
I think quibbles over definitions and wording of most of the probability questions would change my answers by up to a couple of orders of magnitude.
Lastly, I really wanted some way to specify that I thought several xrisks were much more likely than the rest (for example, [nuclear weapons, engineered pandemic] >> others).
My central objection is that this feels like a very un-LessWrongish way to approach a problem. A grab bag of unrelated and unsourced advice is what I might expect to see on the average blog.
Not only is there basically no analysis of what we're trying to do and why, but the advice is a mixed bag. If one entry on the list completely dominates most of the others in terms of effectiveness (and is a prerequisite to putting the others to good use), I don't expect it to be presented as just another member of the list. A few other entries on the list I consider t...
Edit: Grouchymusicologist has already covered silly grammar-nazism, passives, and Strunk and White, complete with the Languagelog links I was looking for.
\25. Write like you talk. When possible, use small, old, Germanic words.
I think this one should be deleted. The first sentence of it is wrong as written, but the idea behind it is expressed clearly and sufficiently in #26 anyway. People do not talk in grammatical, complete sentences.
As for the second half, do you really look up etymologies as you write? I have only the vaguest sense of the origins of ...
I agree with your conclusion (this is a worthwhile pursuit), but I have some qualms.
There are a couple of general points that I think really need to be addressed before most of the individual points on this list can be considered seriously:
Following a list of prescriptions and proscriptions is a really poor way to learn any complex skill. A bad writer who earnestly tries to follow all the advice on this list will almost certainly still be bad at writing. I think the absolute best, most important advice to give to an aspiring writer is to write. A lot.
That was an awesome introduction post. I like the way you think.
Some googling lead me to the Wikipedia article on cognitive dissonance (this link should get you to the right spot on the page).
Wikipedia's citation for this is: Tavris, Carol; Elliot Aronson (2008). Mistakes were made (but not by me). Pinter and Martin. pp. 26–29. This book's first 55 pages are viewable on Google Books. I'll attempt to link directly to the relevant section here but it's an ugly URL so I'm not sure it'll work.
Citation 17 looks like just the thing you're looking for, but the viewable portion of their citations section cuts off just too ear...
The original was Eliezer himself, in How to Seem (and Be) Deep. I'm more fond of TheOtherDave's analogy, though, since I think the baseball bat analogy suffers from one weakness: you're drawing a metaphorical parallel in which death (which you disagree is bad) is replaced by something that's definitely bad. Sometimes you can't get any farther than this, since this sets off some people's BS detectors (and to be honest I think the heuristic they're using to call foul on this is a decent one).
Even if you can get them to consider the true payload of the argum...
I don't mean to rain on cousin_it's parade at all here, but I have to put in an additional plug for "After Life." Even if you didn't really find the blog post especially interesting, if you have any affinity for science fiction I really think "After Life" is worth a look. I recommend it with no reservations.
It's short, it's free, and it's the best exploration I've seen of some very LessWrong ideas. The premise of the story is based on recursive self-modification and uploading, and it's entertaining as well as interesting.
(There actually was a method for getting it, but it was an Advanced Guess Culture technique, not readily taught in one session.)
I'd love an explanation of the technique.
FWIW, among my friends--whom I might describe as "polite askers" or "assertive guessers"--it's common to ask "does anybody want to split this with me?" That way, you're both asking for what you want (more of the thing) and making an offer in a guess-culture-compatible way. It's easy for other people to accept, because now by taking it they're not preventing you from having it. If no one does, you can be reasonably confident no one else actually wanted it.
A variant on the same thing is: "Would anyone else like this?" ...
Ferd's method works, assuming you can actually manage to help with the dishes (the trick to that is to just start doing it, rather than ask... if you ask, the host is obligated to say "no, of course not," since it is understood that you don't actually want to help with the dishes), but the one I had in mind is you take a serving implement, pick up the last piece of chicken, catch the eye of someone else at the table, and offer it to them. They, of course, are obligated to say "No, you take it" (as are you, if someone offers it to you). ...
Volunteer to help with the dishes, then ask whether you can take away the plate the chicken is sitting on. If nobody else claims it, it's yours.
Clear another plate before you touch the one with the chicken on it. Clear something else after. Clear your plate when you're done eating.
Don't do more work than your hosts. You're being helpful, not trying to work off the price of dinner.
Hello, and welcome.
You are correct in your observation that this section does not have a high rate of new posts. I'm not sure, but I think you are likely correct in your guess that a flood of new posts would not be appreciated. LessWrong doesn't have a very traditional forum structure, and I'm not sure that a place exists on this site yet that quite fits your posting style. I'm commenting here in part because that puts you in the same boat as me - my first comment on this site was the opinion that the avenues of participation in LW don't seem to fit how I ...
I was going more for the point that some ambiguous questions about probabilities are disguised less-ambiguous questions about payoffs
To provide my feedback data point in the opposite direction, I found this to be well-expressed in the OP.
I have not read the original Harry Potter series. I first learned that Quirrell was Voldemort when, after finishing the 49 chapters of MoR out at that point, I followed a link from LW to the collected author's notes and read those.
I think that for those who have not read the source material (though there may not be many of us), it is basically impossible to intuit that Quirrell is Voldemort from the body of the fanfic so far.
That said, I don't feel like I missed out in any way and don't see why it necessarily needs to be any more explicit until the inevita...
I have a question about chapter 49 and was wondering if anyone else had a similar reaction. Assuming Quirrell is not lying/wrong, and Voldemort did kill Slytherin's Monster, then my first thought was how unlikely that Slytherin's Monster should have even survived long enough to make it to 1943. No prior Heir of Slytherin had had the same idea? Perhaps no prior Heir of Slytherin had been strong enough to defeat Slytherin's Monster? No prior Heir had been ruthless enough?
Maybe this constitutes weak evidence for the theory that Quirrell is lying.
Isn't your point of view precisely the one the SuperHappies are coming from? Your critique of humanity seems to be the one they level when asking why, when humans achieved the necessary level of biotechnology, they did not edit their own minds. The SuperHappy solution was to, rather than inflict disutility by punishing defection, instead change preferences so that the cooperative attitude gives the highest utility payoff.
Reddit-style posting is basically the same format as comment threads here, it's just a little easier to see the threading. One thing that feels awkward using threaded comments is conversation, and people's attempts to converse in comment threads is probably part of why comment threads balloon to the size they do. That's one area that chat/IRC can fill in well.
Another issue is that top-level posts have a feeling of permanence to them. It's like publishing something. I'd rather start with an idea and be able to discuss it and shape it. Top-level posts seem l...
Hi.
edit: to add some potentially useful information, I think the biggest reason I haven't participated is that I feel uncomfortable with the existing ways of contributing (solely, as I understand it, top-level posts and comments on those posts). I know there has been discussion on LW before on potentially adding forums, chat, or other methods of conversing. Consider me a data point in favor of opening up more channels of communication. In my case I really think having a LW IRC would help.
Is this not kosher? The minimum karma requirement seems like an anti-spam and anti-troll measure, with the unfortunate collateral damage of temporarily gating out some potential good content. The post seems clear to me as good content, and my suggestion to MazeHatter in the open thread that this deserved its own thread was upvoted.
If that doesn't justify skirting the rule, I can remove the post.