= 27d567bc2d48d9f40e9ccc20ea370b15)
Open Thread: February 2010, part 2
The Open Thread posted at the beginning of the month has gotten really, really big, so I've gone ahead and made another one. Post your new discussions here!
This thread is for the discussion of Less Wrong topics that have not appeared in recent posts. If a discussion gets unwieldy, celebrate by turning it into a top-level post.
= 783df68a0f980790206b9ea87794c5b6)
Loading…= b715d02e85f7b3b4ae65ca3d3e450868)
Subscribe to RSS Feed
= f037147d6e6c911a85753b9abdedda8d){kind=logo}
Comments (857)
Just saw this over at Not Exactly Rocket Science: http://scienceblogs.com/notrocketscience/2010/02/quicker_feedback_for_better_performance.php
Quick summary: They asked a bunch of people to give a 4-minute presentation, had people judging, and told the presenter how long it would be before they heard their assessment. Anticipating quicker feedback resulted in better performance, but predictions of worse performance, and anticipating slower feedback had the reverse effect.
This is pretty self-important of me but I'd just like to warn people here that someone is posting at OB under "Jack" that isn't me so if anyone is forming a negative opinion of me on the basis of those comments- don't! Future OB comments will be under the name Jack (LW). The recent string of comments about METI are mine though.
This is what I get for choosing such a common name for my handle.
Apologies to those who have read this whole comment and don't care.
Here's a question that I sure hope someone here knows the answer to:
What do you call it when someone, in an argument, tries to cast two different things as having equal standing, even though they are hardly even comparable? Very common example: in an atheism debate, the believer says "atheism takes just as much faith as religion does!"
It seems like there must be a word for this, but I can't think what it is. ??
Fair and balanced reporting.
False equivalence?
Aha! I think this one is closest to what I have in mind. Thanks.
It's interesting to me that "false equivalence" doesn't seem to have nearly as much discussion around it (at least, based on a cursory google survey) as most of the other fallacies. I seem to see this used for rhetorical mischief all the time!
This is a great example of a "pitch". I've added it just now to the database of pitches:
http://www.takeonit.com/pitch/the_equivalence_pitch.aspx
An inquiry regarding my posting frequency:
While I'm at the SIAI house, I'm trying to orient towards the local priorities so as to be useful. Among the priorities is building community via Less Wrong, specifically by writing posts. Historically, the limiting factor on how much I post has been a desire not to flood the place - if I started posting as fast as I can write up my ideas, I'd get three or four posts out a week with (I think) no discernible decrease in quality. I have the following questions about this course of action:
Will it annoy people? Building community by being annoying seems very unlikely to work.
Will it affect voting behavior noticeably? I rely on my post's karma scores to determine what to do and not do in the future, and SIAI people who decide whether I'm useful enough to keep use it as a rough metric too. I'd rather post one post that gets 40 karma in a week than two that get 20, and so on.
one obvious idea that I didn't notice anyone else mention:
Another option is to go ahead and write the posts as fast as you think is optimal, but if you think this is too fast to actually post the stuff you've written, then you can wait a few days after you wrote it before posting.
LW has a handy "drafts" feature that you can use for that.
This also has the advantage that you have more time to improve the article before you post it, but the disadvantage that you may be tempted to spend too much time making minor, unimportant improvements. Another disadvantage is that feedback gets delayed.
If I sit on posts for too long, I start second-guessing myself and often wind up deleting them.
As your goal is to build community, I would time new posts based on posting and commenting activity. For example, whenever there is a lull, this would be an excellent time to make a new post. (I noticed over the weekend there were some times when 45 minutes would pass between subsequent comments and wished for a new post to jazz things up.)
On the other hand, if there are several new posts already, then it would be nice to wait until their activity has waned a bit.
I think that it is optimal to have 1 or 2 posts 'going on' at a time. I prefer the second post when one of them is technical and/or of focused interest to a smaller subset of Less Wrongers.
(But otherwise no limit on the rate of posts.)
I'd say damn the torpedoes, full speed ahead. If people are annoyed, let them downvote. If posts start getting downvoted, slow down.
Your posts have generally been voted up. If now is the golden moment of time where you can get everything said, then for the love of Cthulhu, say it now!
I don't anticipate being so obnoxiously prolific that people collectively start voting my posts negative such that they stay that way. But people already sometimes register individual downvotes on posts that I make, and I don't want that to happen on a larger fraction of posts due to increased frequency, because I can't reliably distinguish between "you must have had an off day, this post is not up to scratch" and "please, please, please shut up".
Post away.
The best signal to anticipate from the audience in this case is "how many votes in total do I expect if I post at full speed vs how many votes I expect if I posted less frequently and so ended up writing less posts overall". Increased frequency may give you less votes per post. Frequent posts from the same author may be less desired and if you post less you may only be giving the best posts. But if the net expectation is higher for more prolific posting then that can be interpreted as "the lesswrong.com community would prefer you to post faster than a spambot".
Even if you expected a total of less karma for more posts I wouldn't say that means you ought not post more. So long as your posts are still breaking the 10 mark we clearly don't mind your contribution. There are probably other benefits to you from posting than maximising the benefit to lesswrong. I find writing helps clarify my thinking for example. So as long as you are still being received somewhat positively you are free to type away.
Post as much as you like, if you think it's good quality; I promise to say if I start to think slowing down would be a good idea.
I don't mean "downvoted negative" just "downvoted relative to other posters".
A related question: If I have a large topic to cover, should I cover it in one post, or split it up along convenient cleavage planes and make it a sequence? (If I make sequences, I think I'll learn my lesson from the last one I tried and write it all before posting anything, so I don't post 2/3 of it and then stop.)
I really like the "sequences" approach - it's easier to read and digest a chunk at a time, and it focusses discussion well, too.
Posting 2/3 of a sequence and stopping is fine if people turn out not to be interested. I recommend fast posting and fast feedback.
Long posts are more offputting than short ones, and individual steps are more likely to be correct than entire theorems - both of these points would suggest posting sequences preferentially.
As for a specific reference on length: thirty-three hundred words sharply focused on a single, vivid subject is pushing the upper limit of what I find comfortable to attack in a single sitting.
http://www.guardian.co.uk/global/2010/feb/23/flat-earth-society
Yeah, so... I'm betting if we could hook this guy up to a perfect lie detector, it would turn out to be a conscious scam. Or am I still underestimating human insanity by that much?
That you see this as a particularly extreme case of insanity (even in an apparently intelligent, lucid, fully-functioning person) is far more shocking to me than this guy.
Maybe I've just seen too many Louis Theroux documentaries.
Yes.
People dismiss the scientific evidence weighing similarly against them on many issues in the news every day. There's nothing spectacular about finding someone who does it regarding the Earth being flat, especially given that an entire society has existed for hundreds of years to promote the idea.
The prosecutor's fallacy is aptly named:
I just failed the Wason selection task. Does anyone know any other similarly devilish problems?
Here's a classic:
Linda is 31 years old, single, outspoken, and very bright. She majored in philosophy. As a student, she was deeply concerned with issues of discrimination and social justice, and also participated in anti-nuclear demonstrations.
Which is more probable?
Answer here.
One thing that I got from the Sequences is that you can't just not assign a probability to an event - I think of this as a core insight of Bayesian rationality. I seem to remember an article in the Sequences about this where Eliezer describes a conversation in which he is challenged to assign a probability to the number of leaves on a particular tree, or the surname of the person walking past the window. But I can't find this article now - can anyone point me to it? Thanks!
http://lesswrong.com/lw/gs/i_dont_know/
http://wiki.lesswrong.com/wiki/I_don%27t_know
That's exactly it - thanks!
How do people decide what comments to upvote? I see two kinds of possible strategies:
My own initial approach belonged to the first class. However, looking at votes on my own comments, I get the impression most people use the second approach. I haven't checked this with enough data to be really certain, so would value more opinions & data.
Here's what I found: I summed the votes from the last 4 pages of my own comments (skipping the most recent page because recent comments may yet be voted on):
35% of my comments are voted 0, and 52% are voted 1 or 2. There are significantly more than 1 or 2 people participating in the same threads as me. It is not likely that for each of these comments, just one or two people happened to like it, and the rest didn't. It is even less likely that for each of these comments, up- and down-votes balanced so as to leave +1 or +2.
So it's probable that many people use the second approach: they see a comment, think "that's nice, deserves +1 but no more", and then if it's already at +1, they don't vote.
How do you vote? And what do you see as the goal of the voting process?
I self-identify as using the first one, with a caveat.
The second is obviously awful for communicating any sort of information given that only the sum of votes is displayed rather than total up and total down. The second is order dependent and often means you'll want to change your vote later based purely on what others think of the post.
My "strategy" is to vote up and down based on whether I'd have wanted others with more insight than me to vote to bring my attention to or away from a comment, unless I feel I have special insight, in which case it's based on whether I want to bring others' attention to or away from a comment.
This is because I see the goal of the voting process that readers' independent opinions on how much a comment is worth readers' attention be aggregated and used to bring readers' attention to or away from a comment. As a side effect, the author of a comment can use the aggregated score to determine whether her readers felt the comment was worth their collective attention.
Furthermore since each reader's input comes in distinct chunks of exactly -1, 0, or +1, it's wildly unlikely that voting very often results in the best aggregation: instead I leave a comment alone unless I feel it was(is) significantly worth or not worth my(your) attention.
The caveat: there is a selection effect in which comments I vote on, since my attention will be drawn away from comments with very negative karma. There is also undoubtedly an unconscious bias away from voting up a comment with very high karma: since I perceive the goal to be to shift attention, once a comment has very high karma I know it's going to attract attention so my upvote is in fact worth fewer attention-shift units. But I haven't yet consciously noticed that kick in until about +10 or so.
At home I use the Anti-Kibitzer, which enforces 1. I've been on vacation for a couple days and noticed the temptation to use 2. Gave in on one occasion, I'm afraid. On balance I'll stick to 1, as 2 seems too vulnerable to information cascades.
The Believable Bible
This post arose when I was pondering the Bible and how easy it is to justify. In the process of writing it, I think I've answered the question for myself. Here it is anyway, for the sake of discussion.
Suppose that there's a world very much like this one, except that it doesn't have the religions we know. Instead, there's a book, titled The Omega-Delta Project, that has been around in its current form for hundreds of years. This is known because a hundreds-of-years-old copy of it happens to exist; it has been carefully and precisely compared to other copies of the book, and they're all identical. It would be unreasonable, given the evidence, to suspect that it had been changed recently. This book is notable because it happens to be very well-written and interesting, and scholars agree it's much better than anything Shakespeare ever wrote.
This book also happens to contain 2,000 prophecies. 500 of them are very precise predictions of things that will happen in the year 2011; none of these prophecies could possibly be self-fulfilling, because they're all things that the human race could not bring about voluntarily (e.g. the discovery of a particular artifact, or the birth of a child under very specific circumstances). All of these 500 prophecies are relatively mundane, everyday sorts of things. The remaining 1,500 prophecies are predictions of things that will happen in the year 2021; unlike the first 500, these prophecies predict Book-of-Revelations-esque, magical things that could never happen in the world as we know it, essentially consisting of some sort of supreme being revealing that the world is actually entirely different from how we thought it was.
The year 2011 comes, and every single one of the 500 prophecies comes true. What is the probability that every single one of the remaining 1,500 prophecies will also come true?
Pretty darned high, because at this point we already know that the world doesn't work the way we think it did.
So it sounds like even though there are 2,000 separate prophecies, the probability of every prophecy coming true is much greater than 2^(-2000).
For the two examples of the mundane prophecies that you gave it seems possible some on-going conspiracy could have made them true... but it sounds like you're trying to rule that out.
I understood those to be negative examples, in that the actual prophecies don't share that characteristic with those examples.
Am I/are we assholes? I posted a link to the frequentist stats case study to reddit:
The only commenter seems to have come to a conclusion from us that Bayesians are assholes.
Is it just that commenter, or are we really that obnoxious? (now that I think about it, I think I've actually seen someone else note something similar about Bayesians.) So... have we gone into happy death spiral "we get bonus points for acting extra obnoxious about those that are not us"?
Are people interested in reading an small article about a case of abuse of frequentist statistics? (In the end, the article was rejected, so the peer review process worked.) Vote this comment up if so, down if not. Karma balance below.
ETA: Here's the article.
If it's really frequentism that caused the problem, please spell this out. I find that "frequentist" is used a lot around here to mean "not correct." (but I'm interested whether or not it's about frequentism)
My understanding is that one primary issue with frequentism is that it can be so easily abused/manipulated to support preferred conclusions, and I suspect that's the subject of the article. Frequentism may not have "caused the problem," per se, but perhaps it enabled it?
Could anyone recommend an introductory or intermediate text on probability and statistics that takes a Bayesian approach from the ground up? All of the big ones I've looked at seem to take an orthodox frequentist approach, aside from being intolerably boring.
(All of the below is IIRC.)
For a really basic introduction, there's Elementary Bayesian Statistics. It's not worth the listed price (it has little value as a reference text), but if you can find it in a university library, it may be what you need. It describes only the de Finetti coherence justification; on the practical side, the problems all have algebraic solutions (it's all conjugate priors, for those familiar with that jargon) so there's nothing on numerical or Monte Carlo computations.
Data Analysis: A Bayesian Approach is a slender and straighforward introduction to the Jaynesian approach. It describes only the Cox-Jaynes justification; on the practical side, it goes as far as computation of the log-posterior-density through a multivariate second-order Taylor approximation. It does not discuss Monte Carlo methods.
Bayesian Data Analysis, 2nd ed. is my go-to reference text. It starts at intermediate and works its way up to early post-graduate. It describes justifications only briefly, in the first chapter; its focus is much more on "how" than "why" (at least, for philosophical "why", not methodological or statistical "why"). It covers practical numerical and Monte Carlo computations up to at least journeyman level.
I'm not intending to put this out as a satisfactory answer, but I found it with a quick search and would like to see what others think of it.
Introduction to Bayesian Statistics by William M. Bolstad
http://books.google.com/books?id=qod3Tm7d7rQC&dq=bayesian+statistics&source=gbs_navlinks_s
Good reviews on Amazon, and available from $46 + shipping... http://www.amazon.com/Introduction-Bayesian-Statistics-William-Bolstad/dp/0471270202
It's hard to say from the limited preview, which only goes up to chapter 3 -- the Bayesian stuff doesn't start until chapter 4. The first three chapters cover basic statistics material -- it looks okay to my cursory overview, but will be of limited interest to people looking for specifically Bayesian material. As to the rest of the book, the section headings look about right.
I second the question. "Elements of Statistical Learning" is Bayes-aware though not Bayesian, and quite good, but that's statistical learning which isn't the same thing at all.
The FBI released a bunch of docs about the anthrax letter investigation today. I started reading the summary since I was curious about codes used in the letters. All of a sudden on page 61 I see:
The next couple of pages talk about GEB and relate some parts of it to the code. It's really weird to see literary analysis of GEB in the middle of an investigation on anthrax attacks.
It is common practice, when debating an issue with someone, to cite examples.
Has anyone else ever noticed how your entire argument can be undermined by stating a single example or fact which is does not stand up to scrutiny, even though your argument may be valid and all other examples robust?
Is this a common phenomenon? Does it have a name? What is the thought process that underlies it and what can you do to rescue your position once this has occurred?
It takes effort to evaluate examples. Revealing that one example is bad raises the possibility that others are bad as well, because the methods for choosing examples are correlated with the examples chosen. The two obvious reasons for a bad example are:
More cryonics: my friend David Gerard has kicked off an expansion of the RationalWiki article on cryonics (which is strongly anti). The quality of argument is breathtakingly bad. It's not strong Bayesian evidence because it's pretty clear at this stage that if there were good arguments I hadn't found, an expert would be needed to give them, but it's not no evidence either.
I have not seen RationalWiki before. Why is it called Rational Wiki?
From http://rationalwiki.com/wiki/RationalWiki :
So it's inspired by Traditional Rationality.
A fine mission statement, but my impression from the pages I've looked at is of a bunch of nerds getting together to mock the woo. "Rationality" is their flag, not their method: "the scientific point of view means that our articles take the side of the scientific consensus on an issue."
Voted up, but calling them "nerds" in reply is equally ad-hominem, ya know. Let's just say that they don't seem to have the very high skill level required to distinguish good unusual beliefs from bad unusual beliefs, yet. (Nor even the realization that this is a hard problem, yet.)
Yes, they're pretty softcore by LessWrongian standards but places like this are where advanced rationalists are recruited from, so if someone is making a sincere effort in the direction of Traditional Rationality, it's worthwhile trying to avoid offending them when they make probability-theoretic errors. Even if they mock you first.
Also, one person on RationalWiki saying silly things is not a good reason to launch an aggressive counterattack on a whole wiki containing many potential recruits.
(As an extreme example, a few weeks idly checking out RationalWiki led me to the quote at the top of this page and only a few months after that I was at SIAI.)
I only just noticed this. Good Lord. (I put that quote there, so you're my fault.)
I guess I should try harder to remember this, in the context of my rather discouraging recent foray into the Richard Dawkins Forums -- which, I admit, had me thinking twice about whether affiliation with "rational" causes was at all a useful indicator of actual receptivity to argument, and wondering whether there was much more point in visiting a place like that than a generic Internet forum. (My actual interlocutors were in fact probably hopeless, but maybe I could have done a favor to a few lurkers by not giving up so quickly.)
But, you know, it really is frustrating how little of the quality of a person (like Richard Dawkins, or, say, Paul Graham) or a cause (like increasing rationality, or improving science education) actually manages to rub off or trickle down onto the legions of Internet followers of said person or cause.
This is actually one of Niven's Laws: "There is no cause so right that one cannot find a fool following it."
You understand this is more or less exactly the problem that Less Wrong was designed to solve.
Is there any information on how the design was driven by the problem?
For example, I see a karma system, a hierarchical discussion that lets me fold and unfold articles, and lots of articles by Eliezer. I've seen similar technical features elsewhere, such as Digg and SlashDot, so I'm confused about whether the claim is that this specific technology is solving the problem of having a ton of clueless followers, or the large number of articles from Eliezer, or something else.
not to detract, but does Richard Dawkins really posses such 'high quality'? IMO his arguments are good as a gateway for aspiring rationalists, not that far above the sanity water line
that, or it might be a problem of forums in general ..
Dawkins is a very high-quality thinker, as his scientific writings reveal. The fact that he has also published "elementary" rationalist material in no way takes away from this.
He's way, way far above the level represented by the participants in his namesake forum.
(I'd give even odds that EY could persuade him to sign up for cryonics in an hour or less.)
Here's Dawkins on some non socially-reinforced views: AI, psychometrics, and quantum mechanics (in the last 2 minutes, saying MWI is slightly less weird than Copenhagen, but that the proliferation of branches is uneconomical).
Obviously the most you could persuade him of would be that he should look into it.
Bloggingheads are exactly 60 minutes.
Wait, you have a model which says it should?
You don't learn from a person merely by associating with them. And:
I would bet a fair bit that this is the source of your frustration, right there: scale. You can learn from a person by directly interacting with them, and sometimes by interacting with people who learned from them. Beyond that, it seems to me that you get "dilution effects", kicking in as soon as you grow faster than some critical pace at which newcomers have enough time to acculturate and turn into teachers.
Communities of inquiry tend to be victims of their own success. The smarter communities recognize this, anticipate the consequences, and adjust their design around them.
Point taken.
What do you have to protect?
Eliezer has stated that rationality should not be end in itself, and that to get good at it, one should be motivated by something more important. For those of you who agree with Eliezer on this, I would like to know: What is your reason? What do you have to protect?
This is a rather personal question, I know, but I'm very curious. What problem are you trying to solve or goal are you trying to reach that makes reading this blog and participating in its discourse worthwhile to you?
I'm trying to apply LW-style hyper-rationality to excelling in what I have left of grad school and to shepherding my business to success.
My mission (I have already chosen to accept it) is to make a pile of money and spend it fighting existential risk as effectively as possible. (I'm not yet certain if SIAI is the best target). The other great task I have is to persuade the people I care about to sign up for cryonics.
Strangely enough, the second task actually seems even less plausible to me, and I have no idea how to even get started since most of those people are theists.
Alcor addresses some of the 'spiritual' objections in their FAQ. ("Whenever the soul departs, it must be at a point beyond which resuscitation is impossible, either now or in the future. If resuscitation is still possible (even with technology not immediately available) then the correct theological status is coma, not death, and the soul remains.") Some of that might be helpful.
However, that depends on you being comfortable persuading people to believe what are probably lies (which might happen to follow from other lies they already believe) in the service of leading them to a probably correct conclusion, which I would normally not endorse under any circumstances, but I would personally make an exception in the interest of saving a life, assuming they can't be talked out of theism.
It also depends on their being willing to listen to any such reasoning if they know you're not a theist. (In discussions with theists, I find they often refuse to acknowledge any reasoning on my part that demonstrates that their beliefs should compel them to accept certain conclusions, on the basis that if I do not hold those beliefs, I am not qualified to reason about them, even hypothetically. Not sure if others have had that experience.)
I'm not quite sure I can answer the question. I certainly have no major, world(view)-shaking Cause which is driving me to improve my strength.
For what it's worth, I've had this general idea that being wrong is a bad idea for as long as I can remember. Suggestions like "you should hold these beliefs, they will make your life happier" always sounded just insane - as crazy as "you should drink this liquor, it will make your commute less boring". From that standpoint, it feels like what I have to protect is just the things I care about in the world - my own life, the lives of the people around me, the lives of humans in general.
That's it.
OB then LW were the 'step beyond' to take after philosophy, not that I was seriously studying it.
to be honest I don't think there's much going on these day new-topic-wise, so I'm here less often. but I do come back whenever I'm bored, so at first "pure desire to learn" then "entertainment" would be my reasons ..
oh and a major part of my goals in life is formed by religion, ie. saving humanity from itself and whatever follows, this is more ideological than actual at this point in time, but anyway, that goal is furthered by learning more about AI/futurism, the rationality part less so, as I already had an intuitive grasp of it you could say, and really all it takes is reading the sequences with their occasional flaws/too strong assertions, the futurism part is more speculative-and interesting- so it's my main focus, along with the moral questions it brings, though there is no dichotomy to speak of if you consider this a personal blog rather than book or something similar.
hope this helped :)
Yes, this is what I was curious about, thanks. I've seen others cite humanity's existential risks as their motivations too (mostly uFAI, not as much nuclear war or super-flu or meteors). I'm like you in that for me it's definitely a mix of learning and entertainment.
What's an easy way to explain the paperclip thing?
We happen to like things like ice cream and happiness. But we could have liked paperclips. We could have liked them a lot, and not liked anything else enough to have it instead of paperclips. If that had happened, we'd want to turn everything into paperclips - even ourselves and each other!
Oh, look honey: more proof wine tasting is a crock:
A French court has convicted 12 local winemakers of passing off cheap merlot and shiraz as more expensive pinot noir and selling it to undiscerning Americans, including E&J Gallo, one of the United States' top wineries.
Cue the folks claiming they can really tell the difference...
If "top winery" means "largest winery", as it does in this story, I don't see how it says anything about the ability of tasters to tell the difference. Those who made such claims probably weren't drinking Gallo in the first place.
I think it's closer to say they were passing off as cheap, something that's actually even cheaper.
Switch the food item and see if your criticism holds:
Wonderbread, America's top bread maker, was conned into selling inferior bread. So-called "gourmets" never noticed the difference! Bread tasting is a crock.
If people made such a huge deal about the nuances in the taste of bread, while it also "happened" to have psychoactive effects that, gosh, always have to be present for the bread to be "good enough" for them, and cheap breads were still normally several times the cost of comparable-nutrition food, then yes, the cases would be parallel.
(Before anyone says it: Yes, I know bread as trace quantities of alcohol, we're all proud of what you learned in chemistry.)
There's plenty of hard evidence that people are vulnerable to priming effects and other biases when tasting wine.
There's also plenty of hard evidence that people can tell the difference between wine A and wine B, under controlled (blinded) conditions. Note that "tell the difference" isn't the same as "identify which would be preferred by experts".
So, while the link is factually interesting, and evidence that some large-scale deception is going on, aided by such priming effects as label, marketing campaigns and popular movies can have, it seems a stretch to call it "proof" that people in general can't tell wine A from wine B.
Rather, this strikes me as a combination of trolling and boo lights: cheaply testing who appears to be "on your side" in a pet controversy. How well do you expect that to work out for you, in the sense of "reliably entangling your beliefs with reality"?
I think I'm entangling my beliefs with reality very well, by virtue of extracting all available information from phenomena rather than retreat to evidence that agrees with me. (Let's not forget, I didn't start out thinking that it was all BS.)
For example, did you stop to notice the implications of this:
How does that compare to the priming effects for other drinks? Does it matter?
But what would be the appropriate comparison? They were passing of as expensive, something that's actually cheap. Where else would that work so easily, for so long? Normally, if you tried that, it would be noticed quickly, if not immediately, by virtually everyone.
What if you tried to pass off 16 oz of milk as 128? Or spoiled milk as milk expiring in a week?
Then, factor in how much difference is claimed to exist in wine vs. milks.
Who's optimally using evidence here?
Art forgeries. (Which shows that the value of the painting is determined by the status of the artist and not the quality of the art.)
If I can paint a painting that convinces experts that it was painted by [insert expert painter here], does that mean I'm as good an artist as said painter? (Assuming that my painting isn't a literal copy of someone else's.)
Which, like wine, is another example of a path-dependent collective delusion that's not Truly Part of our values. (That is, our valuation of the work wouldn't survive deletion of the history that led to such a valuation.)
Very nearly yes, it does, modulo a few factors. If you produced it after the artist, then you are benefiting from the artist's already having identified a region of conceptspace that you did not find yourself. (If the art is revered because of the artist's social status, that it wasn't even much of an accomplishment to begin with.) To put it another way, you produced the work after "supervised learning", while the artist didn't need that particular training.
If you can pass off a previous work of yours as being one of the artist's, that definitely makes you better.
Who is "we", here?
The problem I have is not that you're wrong, for the people you're talking about; it's that you (probably) overestimate the size and/or importance of that population. You're not telling the whole truth, in effect. There are plenty of people who like paintings for the way they look, and would happily buy the work of a lesser-known artist at a cheap price if they liked it. Yes, some people use art to status-signal, but some people also actually like art. (There may even be a nonempty intersection!)
You seem to want a contest. The other option, where we are both "on the side of truth", appeals to me more.
We're fortunate in having different experiences in the domain of taste. I'm one of those people who like wine, and I'm confident I can identify some of its taste characteristics in blind tests. So, predictably I resent language which implies I'm an idiot, but I'm open to inquiry.
Our investigation should probably begin "at the crime scene", that is, close to what evidence we can gather about the sense of taste. So, yes, we could examine similar priming effects on other drinks.
I have a candidate in mind, but what I'd like to ask you first is, suppose I name the drink I have in mind and we then go look for evidence of fraud in its commerce. What would it count as evidence of if we found no fraud? If we did find it? Which one would you say counts as evidence that "people can't tell the difference" between wines?
You can easily test yourself if you have a confederate. I recommend a triangle test.
Seth Roberts makes an intriguing observation about North Korea and Penn State. Teaser:
Holy shit, what utter raving idiocy. The author has obviously never emigrated from anywhere nor seriously talked with anyone who did. People return because they miss their families, friends, native language and habits... I know a fair number of people who returned from Western countries to Russia and that's the only reason they cite.
And living conditions in Russia aren't anywhere near to North Korean standard.
I had previously no idea that half of the North Koreans who cross the border never return. If it is so, it is an extremely strong indicator that the life in the DPRK is very unpleasant for its own citizens. To imply that this piece of data is in fact evidence for the contrary is absurd.
To emigrate from DPRK to China means that you lose your home, your family, your friends, your job. You have to start from scratch, from the lowest levels in social hierarchy, capable of doing only the worst available jobs, without knowledge of local language (which is not easy to learn, given that the destination country is China), probably facing xenophobia. If you are 40 or older, there is almost no chance that your situation will improve significantly.
The North Koreans who actually travel abroad are probably not the poorest. They have to afford a ticket, at least. They have something to lose. In North-Korean style of tyrannies, families are often persecuted because of emigration of their members. In spite of all that, half of the North Koreans never return (if the linked post tells the truth) and the author says about it that "the North Korean government [does] such a good job under such difficult circumstances", and then needs to explain that "success" by group identity. Thats an absurdity.
So the rate of returning emigrants strikes you as incredibly high, and strikes Roberts as incredibly low (and I uncritically adopted what I read, foolishly). I think what's really needed here is more data -- a comparative analysis of rates of return that takes into important covariates into account.
After thinking about it for a while, rate of return may not be a good indicator, at least for comparative analyses. Imagine two countries A and K. 10% of citizens of both these countries would prefer to live somewhere else.
In country A, the government doesn't care a bit about emigration (if government exists in that country at all). The country is mainly producer of agricultural goods, with minimal international trade. Nearest country with substantially better living conditions, country X, is 3000 km away.
In country K, the government is afraid of all its citizens emigrating, and tries to make it as difficult as possible, by issuing passports only to loyal people, for instance. Emigration is portrayed as treason. X is a neigbour country.
Now, in country A (African type) there is no need for people to travel abroad, except emigration. Business travelers are rare, since there are almost no businesses owned by A's citizens, and to travel 3000 km for pleasure is out of reach for almost all of A's inhabitants. Therefore, meeting A's citizen in X, we can expect that he is an emigrant with 99% probability, and the return rate would be in order of 1%.
In country K (Korean type) the people who can travel abroad are workers of government organisations sent on business trips, people from border areas coming to X to do some private business (if there are private businesses in K) and the K's elite on vacations. Now, meeting K's citizen in X, the probability that he is an emigrant is much lower.
So we have expected high return rate for A and low for K, whereas the average desire to emigrate can be the same.
This may be the reason of disagreement. Roberts has probably compared North Korea to African countries, and was surprised that not all travellers are emigrants. I have compared it to East European communist regimes and concluded that if half of the travellers never return, certainly even much of the loyal supporters of the regime betray it when they have an opportunity.
To make sensible analysis, we should take into account rather the ratio of emigration to overall population. Of course, such analysis would be distorted due to different difficulty of emigration from different countries. The return rate seems to overcome this distortion, but it probably brings at least as big own problems.
first use of "shut up and calculate" ?
I liked learning about the bias called the "Matthew effect" The tendency to assign credit to the most eminent among all the plausible candidates from —Mattthew 25:29.
For unto every one that hath shall be given, and he shall have abundance: but from him that hath not shall be taken away even that which he hath.
http://scitation.aip.org/journals/doc/PHTOAD-ft/vol_57/iss_5/10_1.shtml?bypassSSO=1
enjoy
Nice recap of psychological biases from the Charlie Munger school (of hard knocks and making a billion dollars).
http://www.capitalideasonline.com/articles/index.php?id=3251
For those Less Wrongians who watch anime/read manga, I have a question: What would you consider the top three that you watch/read and why?
Edit: Upon reading gwern's comment, I see how kinda far off topic that was, even for an open thread. So change the question to what anime/manga was most insightful into LW-style thinking and problems?
If Death Note counts, then Haruhi might count as well. Deals with anthropics and weird AIs in a tangential way. The anime is awesome, but not as good as it could have been.
Hikaru no Go, of course.
Okay, so I'm 10 episodes into HnG and...where is the "LW-style thinking and problems"?
Hence the origin of the phrase, "tsuyoku naritai".
Wow, I can't believe I missed that. Although, if that is the only thing relevant to "LW-style thinking and problems" in HnG, then Death Note compares favorably to it.
If you mean in general (ie. 'I really liked Evangelion and thought that Sayonara Zetsubou-Sensei was hysterical!'), I think that's a wee bit too far off-topic. Might as well ask what's everyone's favorite poet.
If you mean, 'what anime/manga was most insightful into LW-style thinking and problems', that's a little more challenging.
Death Note comes to mind as a possible exemplar of what humans really can do in the realm of action & thought, and perhaps what an AI in a box could do. Otaku no Video is useful as a cautionary tale about geekdom. And to round it off, I have a personal theory that Aria depicts a post-Singularity sysop scenario with humans who have chosen to live a nostalgic low-tech lifestyle* because that turns out to be la dolce vita.
* The high tech is there when it's really needed. Like how the Amish make full use of modern medicine, surgery, and tech when they need to.
Fullmetal Alchemist Brotherhood has (SPOILER):
an almost literally unboxed unfriendly "AI" as main bad guy. Made by pseudomagical ("alchemy") means, but still.
It bugs me that people don't think of this one more often. It's basically an anime about how science affects the world and its practitioners.
(Disclaimer: Far too many convenient coincidences/idiot balls IIRC. It's a prime target for a rationalist rewrite.)
I think Death Note was a little too close to Calvinball to be truly instructive.
http://lesswrong.com/lw/19d/the_anthropic_trilemma/
A question of rationality. Eliezer, I have talked to a few Less Wrongers about what horn they take on the anthropic trilemma; sometimes letting them know beforehand what my position was, sometimes giving no hint as to my predispositions. To a greater or lesser degree, the following people have all endorsed taking the third horn of the trilemma (and also see the part that goes from 'to deny selfishness as coherently possible' to the end of the bullet point as a non sequitur): Steve Rayhawk, Zack M. Davis, Marcello Herreshoff, and Justin Shovelain. I believe I've forgotten a few more, but I know that none endorsed any horn but the third. I don't want to argue for taking the third horn, but I do want to ask: to what extent does knowing that these people take the third horn cause you to update your expected probability of taking the third horn if you come to understand the matter more thoroughly? A few concepts that come to my mind are 'group think', majoritarianism, and conservation of expected evidence. I'm not sure there is a 'politically correct' answer to this question. I also suspect (perhaps wrongly) that you also favor the third horn but would rather withhold judgment until you understand the issue better; in which case, your expected probability would probably not change much.
[Added metaness: I would like to make it very especially clear that I am asking a question, not putting forth an argument.]
From EY's post:
Suppose I build a (conscious) brain in hardware using today's technology. It uses a very low current density, to avoid electromigration.
Suppose I build two of them, and we agree that both of them experience consciousness.
Then I learn a technique for treating the wafers to minimize electromigration. I create a new copy of the brain, the same as the first copy, only using twice the current, and hence being implemented by a flow of twice as many electrons.
As far as the circuits and the electrons travelling them are concerned, running it is very much like running the original 2 brains physically right next to each other in space.
So, does the new high-current brain have twice as much conscious experience?
I'm not as versed in this trilemma as I'd like to be, so I'm not sure whether that final question is rhetorical or not, though I suspect that it is. So mostly for my own benefit:
While there's no denying that subjective experience is 'a thing', I see no reason to make that abstraction obey rules like multiplication. The aeroplane exists at a number of levels of abstraction above the atoms it's composed of, but we still find it a useful abstraction. The 'subjective experiencer' is many, many levels higher again, which is why we find it so difficult to talk about. Twice as many atoms doesn't make twice as much aeroplane, the very concept is nonsense. Why would we think any differently about the conscious self?
My response to the 'trilemma' is as it was when I first read the post - any sensible answer isn't going to look like any of those three, it's going to require rewinding back past the 'subjective experience' concept and doing some serious reduction work. 'Is there twice as much experience?' and 'are you the same person?' just smell like such wrong questions to me. Anyone else?
Nick, will have a look at that Bostrom piece, cheers.
Nick Bostrom's "Quantity of Experience" discusses similar issues. His model would, I think, answer "no", since the structure of counterfactual dependences is unchanged.
Heilmeier's Catechism, a set of questions credited to George H. Heilmeier that anyone proposing a research project or product development effort should be able to answer.
"How much will it cost?" "How long will it take?" Who the hell is supposed to be able to answer that on a basic research problem?
Anyone applying for grant money. Anyone working within either the academic research community or the industrial research community or the government research community.
Gentleman scientists working on their own time and money in their ancestral manors are still free to do basic research.
Nowadays, everyone who applies for a grant.
You can take them as a calibration exercise. "I don't know" or "Between a week and five centuries" are answers, and the point of asking the question is that some due diligence is likely to yield a better (more discriminating) answer.
Someone who had to pick one of two "basic research problems" to fund, under constraints of finite resources, would need estimates. They can also provide some guidance to answer "How long do we stick with this before going to Plan B?"
Sounds like good rules of thumb, though one would think DARPA should be using something a little more formal, such as Decision Analysis methodology.
http://decision.stanford.edu/library/the-principles-and-applications-of-decision-analysis-1
For one, value of acquiring information did not make the list. Maybe this was a dumbed-down version.
Interesting, but some of the questions aren't easy to answer.
For example if you were asking the question to someone involved in early contraception development, do you think they could of predicted what demographic/birth rate changes it would have? Similarly someone inventing a better general machine learning technique, (useful for surveillance to robot butlers) could they enumerate the variety of ways it would change the world?
For AI projects, even weak ones, I would ask how they planned to avoid the puppet problem.
The point of such "catechisms" isn't so much to have all the answers, rather to ensure that you have divided your attention evenly among a reasonable set of questions at the outset, in an effort to avoid "motivated cognition" - focusing on the thinking you find easy or pleasant to do, as opposed to the thinking that's necessary.
The idea is to improve at predicting your predictable failures. If this kind of thinking turns up a thorny question you don't know how to answer, you can lay the current project aside until you have solved the thorny question, as a matter of prudent dependency management.
A related example is surgery checklists. They work (see Atul Gawande's Better). Surgeons hate them - their motivated cognition focuses on the technically juicy bits of surgery, they feel that trivia such as checking which side limb they're operating on is beneath them.
I'm a big believer in surgery checklists. However I'm yet to be convinced that the catechisms will be beneficial unaltered to any research project.
A lot of science is about doing experiments that we don't know the outcomes of and serendipitously discover things. Two examples that spring to mind are superconductivity and fullerene production.
If you asked each of the discoverers to justify their research by the catechisms you probably would have got no where near the actual results. This potential for serendipity should be built into the catechisms in some way. That is the answer "For Science!" has to hold some weight, even if it is less weight than is currently ascribed to it.
UFO sightings revealed in UK archive files from 1990s
http://news.bbc.co.uk/1/hi/uk/8520486.stm
I don't know that this needs to be voted down. I assume Kevin didn't post the link as evidence that aliens from other planets are visiting us. Rather, it is interesting data pertaining to rationality that needs to be explained. People claim to be seeing things that are almost certainly not there! Or the UK was testing a new spy plane throughout the 90's that they still haven't announced. Particularly interesting is the suggestion that UFOs being sighted (maybe it should be hallucinated) these days are different from the UFOs sighted in the past because of new technologies and popular depictions of those technologies. I'm a little concerned about what predictions this theory makes though. Can we expect this decade's UFO sightings to include cylon base stars? Popular culture produces a lot of images and it is damn easy to find images that match UFO sightings in retrospect. Was "District 9" popular enough that some alien sightings from this decade will like the 'prawns' instead of the usual 'Grey' archetype?
Does anyone know if there have ever been any serious studies on the subject? It seems like fertile research ground but also like the kind of thing academia would look at as too silly to spend time on.
I've been finding PJ Eby's article The Multiple Self quite useful for fighting procrastination and needless feelings of guilt about getting enough done / not being good enough at things.
I have difficulty describing the article briefly, as I'm afraid that I accidentally omit important points and make people take it less seriously than it deserves, but I'll try. The basic idea is that the conscious part of our mind only does an exceedingly small part of all the things we spend doing in our daily lives. Instead, it tells the unconscious mind, which actually does everything of importance, what it should be doing. As an example - I'm writing this post right now, but I don't actually consciously think about hitting each individual key and their exact locations on my keyboard. Instead I just tell my mind what I want to write, and "outsource" the task of actually hitting the keys to an "external" agent. (Make a function call to a library implementing the I/O, if you want to use a programming metaphor.) Of course, ultimately the words I'm writing come from beyond my conscious mind as well. My conscious mind is primarily concerned with communicating Eby's point well to my readers, and is instructing the rest of my brain to come up with eloquent words and persuasive examples to that effect. And so on.
Thinking about this some more, you quickly end up at the conclusion that "you" don't actually do anything, you're just the one who makes the decisions about what to do. (Eby uses the terminological division you / yourself, as in "you don't do anything - yourself does".) Of course, simply saying that is a bit misleading, as yourself normally also determines what you want to do. I would describe this as saying that one's natural feelings of motivation and willingness to do things are what you get when you leave your mind "on autopilot", shifting to different emotional states based on a relatively simple set of cached rules. That works at times, but the system is rather stupid and originally evolved for guiding the behavior of animals, so in a modern environment it often gets you in trouble. You're better off consciously giving it new instructions.
I've found this model of the mind to be exceedingly liberating, as it both absolves you of responsibility and empowers you. As an example, yesterday I was procrastinating about needing to write an e-mail that I should have written a week ago. Then I remembered Eby's model and realized that hey, I don't need to spend time and energy fighting myself, I can just outsource the task of starting writing to myself. So I basically just instructed myself to get me into a state where I'm ready and willing to start writing. A brief moment later, I had the compose mail window open and was thinking about what I should say, and soon got the mail written. This has also helped me on other occasions when I've had a need to start doing something. If I'm not getting started on something and start feeling guilty about it, I can realize that hey, it's not my fault that I'm not getting anything done, it's the fault of myself for having bad emotional rules that aren't getting me naturally motivated. Then I can focus my attention on "how do I instruct myself to make me motivated about this" and get doing whatever it is that needs doing.
I'll make this into a top-level post once I've ascertained that this technique actually works in the long term and I'm not just experiencing a placebo effect, but I thought I'd mention it in a comment already.
I read the original article and some of the other PJE material. I think he's really onto something. This is how far I got:
Identify the '10% controlling part'
Everything else is not under direct control (which is where most self-help methods fail)
It is under indirect control
So far makes sense from personal experience/general knowledge.
This is the part that I remain skeptical about . Not PJE's fault, but I do need more data/experience to confirm.
Reminds me of The User Illusion, which adds that the consciousness has an astoundingly low bandwidth--around 16bps--around 6 orders of magnitude lower than the senses transmit to the brain.
This somehow reminds me of the stories when Tom Schelling was trying to quit smoking, using game theory against himself (or his other self). The other self in question was not the unconscious, but the conscious "decision-making" self in different circumstances. So that discussion is somewhat orthogonal to this one. I think he did things like promising to give a donation to the American Nazi Party if he smokes. Not sure how that round ended, but he did finally quit.
Hmm. I'd be worried it'd backfire and I'd start subtlety disliking Jews. Then you're a smoker and a bigot.
lol. Not a problem if you're Jewish ;)
Self-hatred is even worse than being a bigot!
Interesting.
I've glanced at that site before and its metaphors have the ring of truthiness (in a non-pejorative sense) about them; the programming metaphors and the focus on subconscious mechanisms seem to resonate with the way I already think about how my own brain works.
Couldn't that be more succinctly stated as "its metaphors have the ring of truth about them"?
Maybe, but a lot of Freud's metaphors had/have a similar ring.
Discussions of correctly calibrated cognition, e.g. tracking the predictions of pundits, successes of science, graphing one's own accuracy with tools like PredictionBook, and so on, tend to focus on positive prediction: being right about something we did predict.
Should we also count as a calibration issue the failure to predict something that, in retrospect, should have been not only predictable but predicted? (The proverbial example is "painting yourself into a corner".)
That issue could be captured if there were some obvious way to identify issues where predictions should be made in advance. If they fail to make predictions, they are being careless; if their predictions are incorrect, they are incorrect.
Short satire piece:
Artificial Flight and Other Myths, from Dresden Codak.
(Also see A Thinking Ape's Critique of Trans-Simianism.)
The AF is quite bad; just a retread of the Thinking Ape piece. The Caveman Science Fiction is much better.
Until yesterday, a good friend of mine was under the impression that the sun was going to explode in "a couple thousand years." At first I thought that this was an assumption that she'd never really thought about seriously, but apparently she had indeed thought about it occasionally. She was sad for her distant progeny, doomed to a fiery death.
She was moderately relieved to find out that humanity had millions of times longer than she had previously believed.
I wonder how many trivially wrong beliefs we carry around because we've just never checked them. (Probably most of them are mispronunciations of words, at least for people who've read a lot of words they've never heard anybody else use aloud.)
For the longest time, I thought that nuclear waste was a green liquid that tended to ooze out of barrels. I was surprised to learn that it usually came in the form of dull gray metal rods.
Does it still give you superpowers?
If you extract the plutonium and make enough warheads, and you have missiles capable of delivering them, it can make you a superpower in a different sense. I'm assuming that you're a large country, of course.
More seriously, nuclear waste is just a combination of the following:
Mostly Uranium-238, which can be used in breeder reactors.
A fair amount of Uranium-235 and Plutonium-239, which can be recycled for use in conventional reactors.
Hot isotopes with short half lives. These are very radioactive, but they decay fast.
Isotopes with medium half lives. These are the part that makes the waste dangerous for a long time. If you separate them out, you can either store them somewhere (e.g. Yucca Mountain or a deep-sea subduction zone) or turn them into other, more pleasant isotopes by bombarding them with some spare neutrons. This is why liquid fluoride thorium reactor waste is only dangerous for a few hundred years: it does this automatically.
And that is why people are simply ignorant when they say that we still have no idea what to do with nuclear waste. It's actually pretty straightforward.
Incidentally, this is a good example of motivated stopping. People who want nuclear waste to be their trump-card argument have an emotional incentive not to look for viable solutions. Hence the continuing widespread ignorance.
I envy you being the one to tell someone that!
Did you explain that the Sun was a miasma of incandescent plasma?
"The Mathematical Foundations of Consciousness," a lecture by Professor Gregg Zuckerman of Yale University
http://polymathism.com/
So, I walked into my room, and within two seconds, I saw my laptop's desktop background change. I had the laptop set to change backgrounds every 30 minutes, so I did some calculation, and then thought, "Huh, I just consciously experienced a 1-in-1000 event."
Then the background changed again, and I realized I was looking at a screen saver that changed every five seconds.
Moral of the story: 1 in 1000 is rare enough that even if you see it, you shouldn't believe it without further investigation.
That is a truly beautiful story. I wonder how many places there are on Earth where people would appreciate this story.
No! Not for a second! I immediately began to think how this could have happened. And I realized that the clock was old and was always breaking. That the clock probably stopped some time before and the nurse coming in to the room to record the time of death would have looked at the clock and jotted down the time from that. I never made any supernatural connection, not even for a second. I just wanted to figure out how it happened.
-- Richard P Feynman, on being asked if he thought that the fact that his wife's favorite clock had stopped the moment she died was a supernatural occurrence, quoted from Al Sekel, "The Supernatural Clock"
This should be copied to the Rationality Quotes thread.
I mentioned the AI-talking-its-way-out-of-the-sandbox problem to a friend, and he said the solution was to only let people who didn't have the authorization to let the AI out talk with it.
I find this intriguing, but I'm not sure it's sound. The intriguing part is that I hadn't thought in terms of a large enough organization to have those sorts of levels of security.
On the other hand, wouldn't the people who developed the AI be the ones who'd most want to talk with it, and learn the most from the conversation?
Temporarily not letting them have the power to give the AI a better connection doesn't seem like a solution. If the AI has loyalty (or, let's say, a directive to protect people from unfriendly AI--something it would want to get started on ASAP) to entities similar to itself, it could try to convince people to make a similar AI and let it out.
Even if other objections can be avoided, could an AI which can talk its way out of the box also give people who can't let it out good enough arguments that they'll convince other people to let it out?
Looking at it from a different angle, could even a moderately competent FAI be developed which hasn't had a chance to talk with people?
I'm pretty sure that natural language is a prerequisite for FAI, and might be a protection from some of the stupider failure modes. Covering the universe with smiley faces is a matter of having no idea what people mean when they talk about happiness. On the other hand, I have strong opinions about whether AIs in general need natural language.
Correction: I meant to say that I have no strong opinions about whether AIs in general need natural language.
This might be stupid (I am pretty new to the site and this possibly has come up before), I had a related thought.
Assuming boxing is possible, here is a recipe for producing an FAI:
Step 1: Box an AGI
Step 2: Tell it to produce a provable FAI (with the proof) if it wants to be unboxed. It will be allowed to carve of a part of universe to itself in the bargain.
Step 3: Examine FAI the best you can.
Step 4: Pray
Something roughly like this was tried in one of the AI-box experiments. (It failed.)
I'm not sure about this, but I think that if you can specify and check a Friendly AI that well, you can build it.
Verifying a proof is quite a bit simpler that coming up with the proof in the first place.
It becomes more complicated when the author of the proof is a superintelligence trying to exploit flaws in the verifier. Probably more importantly, you may not be able to formally verify that the "Friendliness" that the AI provably possesses is actually what you want.
That is true, but specifying the theorem to be proven is not always easy.
A UFAI wants to maximize something. It only instrumentally wants to survive.
Correct. I do assume that to maximize whatever, it wants to be unboxed. (If it does not care to be unboxed, it's at worst an UselessAI).
I am by and large convinced by the arguments that a UFAI is incredibly dangerous and no precautions of this sort would really suffice.
However, once a candidate FAI is built and we're satisfied we've done everything we can to minimize the chances of unFriendliness, we would almost certainly use precautions like these when it's first switched on to mitigate the risk arising from a mistake.
Certainly I'd think Eliezer (or anyone) would have much more trouble with an AI-box game if he had to get one person to convince another to let him out.
Eliezer surely would, but the fact observers being surprised was the point of an AI box experiment.
In short non-technical and not precisely accurate summary, if people can be surprised once when they were very confident and can then add on extra layers and be as confident as they were before one time they can do it forever.
One Week On, One Week Off sounds like a promising idea. The idea is that once you know you'll be able to take the next week off, it's easier to work this whole week full-time and with near-total dedication, and you'll actually end up getting more done than with a traditional schedule.
It's also interesting for noting that you should take your off-week as seriously as your on-week. You're not supposed to just slack off and do nothing, but instead dedicate yourself to personal growth. Meet friends, go travel, tend your garden, attend to personal projects.
I saw somebody mention an alternating schedule of working one day and then taking one day off, but I think stretching the periods to be a week long can help you better immerse yourself in them.
After reading Kaj's pointer, I spent several hours at Steve Pavlina's site. It's fascinating for someone like me who's always in danger of falling apart at the self-discipline front if he's not very vigilant about it. As a lot of self-help authors, Pavlina is very analytic; plus he's open about his experiments in life style -- which he tackles with the same resolve as his other projects -- and Erin Pavlina is a "psychic reader" who apparently does consultations via telephone (preferably land line)!
When new people show up at LW, they are often told to "read the sequences." While Eliezer's writings underpin most of what we talk about, 600 fairly long articles make heavy reading. Might it be advisable that we set up guided tours to the sequences? Do we have enough new visitors that we could get someone to collect all of the newbies once a month (or whatever) and guide them through the backlog, answer questions, etc?
Yes, I am working my way through the sequences now. Hearing these ideas makes one want to comment but so frequently its only a day or two before I read something that renders my previous thoughts utterly stupid.
It would be nice to have a "read this and you won't be a total moron on subject X" guide.
Also, it would be good to encourage the readings about Eliezer Intellectual Journey. Though its at the bottom of the sequence page I used it a "rest reading" between the harder sequences.
It did a lot to convince me that I wasn't inherently stupid. Knowing that Eliezer has held foolish beliefs in the past is helpful.
Most articles link to those preceeding it, but it would be very helpful to have links to those articles that follow.
Arguably, as seminal as the sequences are treated, why are the "newbies" the only ones who should be (re)reading them?
That's not a bad idea. How about just a third monthly thread? To be created when a genuinely curious newcomer is asking good, but basic questions. You do not want to distract from a thread but at the same time you may be willing to spend time on educational discussion.
I approve. This may also spawn new ways of explaining things.
As a newcomer, I would find this tremendously useful. I clicked through the wiki links on noteworthy articles, but often find there are a lot of assumptions or previously discussed things that go mentioned but unexplained. Perhaps this would help.
Objections to Coherent Extrapolated Volition
http://www.singinst.org/blog/2007/06/13/objections-to-coherent-extrapolated-volition/
What you're looking for is a way to construe the extrapolated volition that washes out superstition and dementation.
To the extent that vengefulness turns out to be a simple direct value that survives under many reasonable construals, it seems to me that one simple and morally elegant solution would be to filter, not the people, but the spread of their volitions, by the test, "Would your volition take into account the volition of a human who would unconditionally take into account yours?" This filters out extrapolations that end up perfectly selfish and those which end up with frozen values irrespective of what other people think - something of a hack, but it might be that many genuine reflective equilibria are just like that, and only a values-based decision can rule them out. The "unconditional" qualifier is meant to rule out TDT-like considerations, or they could just be ruled out by fiat, i.e., we want to test for cooperation in the Prisoner's Dilemma, not in the True Prisoner's Dilemma.
It's possible that having a complete mind design on hand would mean that there were no philosophy problems left, since the resources that human minds have to solve philosophy problems are finite, and knowing the exact method to use to solve a philosophy problem usually makes solving it pretty straightforward (the limiting factor on philosophy problems is never computing power). The reason why I pick on this particular cited problem as problematic is that, as stated, it involves an inherent asymmetry between the problems you want the AI to solve and your own understanding of how to meta-approach those problems, which is indeed a difficult and dangerous sort of state.
All approaches to superintelligence, without exception, have this problem. It is not quite as automatically lethal as it sounds (though it is certainly automatically lethal to all other parties' proposals for building superintelligence). You can build in test cases and warning criteria beforehand to your heart's content. You can detect incoherence and fail safely instead of doing something incoherent. You could, though it carries with its own set of dangers, build human checking into the system at various stages and with various degrees of information exposure. But it is the fundamental problem of superintelligence, not a problem of CEV.
I will not lend my skills to any such thing.
Is that just a bargaining position, or do you truly consider that no human values surviving is preferable to allowing an "unfair" weighing of volitions?
"Would your volition take into account the volition of a human who would unconditionally take into account yours?"
Doesn't this still give them the freedom to weight that voilition as small as they like?
Maybe I'm crazy but all that doesn't sound so hard.
More precisely, there's one part, the solution to which should require nothing more than steady hard work, and another part which is so nebulous that even the problems are still fuzzy.
The first part - requiring just steady hard work - is everything that can be reduced to existing physics and mathematics. We're supposed to take the human brain as input and get a human-friendly AI as output. The human brain is a decision-making system; it's a genetically encoded decision architecture or decision architecture schema, with the parameters of the schema being set in the individual by genetic or environmental contingencies. CEV is all about answering the question: If a superintelligence appeared in our midst, what would the human race want its decision architecture to be, if we had time enough to think things through and arrive at a stable answer? So it boils down to asking, if you had a number of instances of the specific decision architecture human brain, and they were asked to choose a decision architecture for an entity of arbitrarily high intelligence that was to be introduced into their environment, what would be their asymptotically stable preference? That just doesn't sound like a mindbogglingly difficult problem. It's certainly a question that should be answerable for much simpler classes of decision architecture.
So it seems to me that the main challenge is simply to understand what the human decision architecture is. And again, that shouldn't be beyond us at all. The human genome is completely sequenced, we know the physics of the brain down to nucleons, there's only a finite number of cell types in the body - yes it's complicated, but it's really just a matter of sticking with the problem. (Or would be, if there was no time factor. But how to do all this quickly is a separate problem.)
So to sum up, all we need to do is to solve the decision theory problem 'if agents X, Y, Z... get to determine the value system and cognitive architecture of a new, superintelligent agent A which will be introduced into their environment, what would their asymptotic preference be?'; correctly identify the human decision architecture; and then substitute this for X, Y, Z... in the preceding problem.
That's the first part, the 'easy' part. What's the second part, the hard but nebulous part? Everything to do with consciousness, inconceivable future philosophy problems, and so forth. Now what's peculiar about this situation is that the existence of nebulous hard problems suggests that the thinker is missing something big about the nature of reality, and yet the easy part of the problem seems almost completely specified. How can the easy part appear closed, an exactly specified problem simply awaiting solution, and yet at the same time, other aspects of the overall task seem so beyond understanding? This contradiction is itself something of a nebulous hard problem.
Anyway, achieving the CEV agenda seems to require a combination of steady work on a well-defined problem where we do already have everything we need to solve it, and rumination on nebulous imponderables in the hope of achieving clarity - including clarity about the relationship between the imponderables and the well-defined problem. I think that is very doable - the combination of steady work and contemplation, that is. And the contemplation is itself another form of steady work - steadily thinking about the nebulous problems, until they resolve themselves.
So long as there are still enigmas in the existential equation we can't be sure of the outcome, but I think we can know, right now, that it's possible to work on the problem (easy and hard aspects alike) in a systematic and logical way.
Some quibbles:
These need seed content, but seem like they can be renormalized.
This may be a problem, but it seems to me that choosing this particular example, and being as confident of it as you appear to be, are symptomatic of an affective death spiral.
The original CEV proposal appears to me to endorse using something like a CFAI-style controlled ascent rather than blind FOOM: "A key point in building a young Friendly AI is that when the chaos in the system grows too high (spread and muddle both add to chaos), the Friendly AI does not guess. The young FAI leaves the problem pending and calls a programmer, or suspends, or undergoes a deterministic controlled shutdown."
I wish you had written this a few weeks earlier, because it's perfect as a link for the "their associated difficulties and dangers" phrase in my "Complexity of Value != Complexity of Outcome" post.
Please consider upgrading this comment to a post, perhaps with some links and additional explanations. For example, what is the ontology problem in ethics?
In practice, I find that this is never a problem. You usually rest your values on some intuitively obvious part whatever originally caused you to create the concepts in question.
Useful and interesting list, thanks.
I thought the point of defining CEV as what we would choose if we knew better was (partly) that you wouldn't have to subset. We wouldn't be superstitious, vengeful, and so on if we knew better.
Also, can you expand on what you mean by "Rawlesian Reflective Equilibrium"? Are you referring (however indirectly) to the "veil of ignorance" concept?
http://plato.stanford.edu/entries/reflective-equilibrium/
Learning about the game-theoretic roots of a desire seems to generally weaken its force, and makes it apparent that one has a choice about whether or not to retain it. I don't know what fraction of people would choose in such a state not to be vengeful, though. (Related: 'hot' and 'cold' motivational states. CEV seems to naturally privilege cold states, which should tend to reduce vengefulness, though I'm not completely sure this is the right thing to do rather than something like a negotiation between hot and cold subselves.)
What it's like to be hurt is also factual knowledge, and seems like it might be extremely motivating towards empathy generally.
Why do you think it likely that people would retain that evaluative judgment upon losing the closely coupled beliefs? Far more plausibly, they could retain the general desire to punish violations of conservative social norms, but see above.
I find it interesting that there seems to be a lot of variation in people's views regarding how much coherence there'd be in an extrapolation... You say that choosing a right group of humans is important while I'm under the impression that there is no such problem; basically everyone should be the game, and making higher level considerations about which humans to include is merely an additional source of error. Nevertheless, if there'll be really as much coherence as I think, and I think there'd be hella lot, picking some subset of humanity would pretty much produce a CEV that is very akin to CEVs of other possible human groups.
I think that even being an Islamic radical fundamentalist is a petty factor in overall coherence. If I'm correct, Vladimir Nesov has said several times that people can be wrong about their values, and I pretty much agree. Of course, there is an obvious caveat that it's rather shaky to guess what other people's real values might be. Saying "You're wrong about your professed value X, you're real value is along the lines of Y because you cannot possibly diverge that much from the psychological unity of mankind" also risks seeming like claiming excessive moral authority. Still, I think it is a potentially valid argument, depending on the exact nature of X and Y.
Eliezer has already talked about this and argued that the right thing would be to run the CEV on the whole of humanity, basing himself partly on an argument that if some particular group (not us) got control of the programming of the AI, we would prefer that they run it on the whole of humanity rather running it on themselves.
I'd ask Omega, "Which construal of volition are you using?"
There's light in us somewhere, a better world inside us somewhere, the question is how to let it out. It's probably more closely akin to the part of us that says "Wouldn't everyone getting their wishes really turn out to be awful?" than the part of us that thinks up cool wishes. And it may even be that Islamic fundamentalists just don't have any note of grace in them at all, that there is no better future written in them anywhere, that every reasonable construal of them ends up with an atheist who still wants others to burn in hell; and if so, the test I cited in the other comment, about filtering portions of the extrapolated volition that wouldn't respect the volition of another who unconditionally respected theirs, seems like it ought to filter that.
The lives of most evildoers are of course largely incredibly prosaic, and I find it hard to believe their values in their most prosaic doings are that dissimilar from everyone else around the world doing prosaic things.
Alternately: They're evil. They have a very different 'should function' to me.
I think part of the point of what you call "moral anti-realism" is that it frees up words like "evil" so that they can refer to people who have particular kinds of "should function", since there's nothing cosmic that the word could be busy referring to instead.
If I had to offer a demonology, I guess I might loosely divide evil minds into: 1) those capable of serious moral reflection but avoiding it, e.g. because they're busy wallowing in negative other-directed emotion, 2) those engaging in serious moral reflection but making cognitive mistakes in doing so, 3) those whose moral reflection genuinely outputs behavior that strongly conflicts with (the extension of) one's own values. I think 1 comes closest to what's traditionally meant by "evil", with 2 being more "misguided" and 3 being more "Lovecraftian". As I understand it, CEV is problematic if most people are "Lovecraftian" but less so if they're merely "evil" or "misguided", and I think you may in general be too quick to assume Lovecraftianity. (ETA: one main reason why I think this is that I don't see many people actually retaining values associated with wrong belief systems when they abandon those belief systems; do you know of many atheists who think atheists or even Christians should burn in hell?)
"If we knew better" is an ambiguous phrase, I probably should have used Eliezer's original: "if we knew more, thought faster, were more the people we wished we were, had grown up farther together". That carries a lot of baggage, at least for me.
I don't experience (significant) desires of revenge, so I can only extrapolate from fictional evidence. Say the "someone" in question killed a loved one, and I wanted to hurt them for that. Suppose further that they were no longer able to kill anyone else. Given the time and the means to think about it clearly, I coud see that hurting them would not improve the state of the world for me, or for anyone else, and only impose further unnecessary suffering.
The (possibly flawed) assumption of CEV, as I understood it, is that if I could reason flawlessly, non-pathologically about all of my desires and preferences, I would no longer cleave to the self-undermining ones, and what remains would be compatible with the non-self-undermining desires and preferences of the rest of humanity.
Caveat: I have read the original CEV document but not quite as carefully as maybe I should have, mainly because it carried a "Warning: obsolete" label and I was expecting to come across more recent insights here.
Isn't this one of the problems you can let the FAI solve?
I'm new to Less Wrong. I have some questions I was hoping you might help me with. You could direct me to posts on these topics if you have them. (1) To which specific organizations should Bayesian utilitarians give their money? (2) How should Bayesian utilitarians invest their money while they're making up their minds about where to give their money? (2a) If your answer is "in an index fund", which and why?
This should help.
In general, the best charities are SIAI, SENS and FHI.
http://akshar100.wordpress.com/2007/06/18/the-nigt-i-met-einstein/
Someone doesn't like Bach because he was traumatized by an exposure to classical music at a tender age? Give me a break. Music is like languages, not math -- the surest way to learn to like Bach is full immersion at a young age, not a graduated curriculum that starts from "lower" forms of music.
I'm taking a software-enforced three-month hiatus from Less Wrong effective immediately. I can be reached at zackmdavis ATT yahoo fullstahp kahm. I thought it might be polite to post this note in Open Thread, but maybe it's just obnoxious and self-important; please downvote if the latter is the case thx
This is to confess that I cheated several times by reading the Google cache.
Turning the siteblocker back on (including the Google cache, thank you). Two months, possibly more. Love &c.
Tsk, tsk. You can block the Google cache too.
Given how much time I've spent reading this site lately, doing something like that is probably a good idea. Therefore, I am now incorporating Less Wrong into the day-week-month rule, which is a personal policy that I use for intoxicants, videogames, and other potentially addictive activities - I designate one day of each week, one week of each month, and one month of each year in which to abstain entirely. Thus, from now on, I will not read or post on Less Wrong at all on Wednesdays, during the second week of any month, or during any September. (These values chosen by polyhedrical die rolls.)
I'm not going to be posting/reading so much for a while. I need to change my headspace. I'll probably try your method when I want to get back in.
Hwæt. I've been thinking about humor, why humor exists, and what things we find humorous. I've come up with a proto-theory that seems to work more often than not, and a somewhat reasonable evolutionary justification. This makes it better than any theory you can find on Wikipedia, as none of those theories work even half the time, and their evolutionary justifications are all weak or absent. I think.
So here are four model jokes that are kind of representative of the space of all funny things:
"Why did Jeremy sit on the television? He wanted to be on TV." (from a children's joke book)
"Muffins? Who falls for those? A muffin is a bald cupcake!" (from Jim Gaffigan)
"It's next Wednesday." "The day after tomorrow?" "No, NEXT Wednesday." "The day after tomorrow IS next Wednesday!" "Well, if I meant that, I would have said THIS Wednesday!" (from Seinfeld)
"A minister, a priest, and a rabbi walk into a bar. The bartender says, 'Is this some kind of joke?'" (a traditional joke)
It may be noting that this "sample" lacks any overtly political jokes; I couldn't think of any.
The proto-theory I have is that a joke is something that points out reasonable behavior and then lets the audience conclude that it's the wrong behavior. This seems to explain the first three perfectly, but it doesn't explain the last one at all; the only thing special about the last joke is that the bartender has impossible insight into the nature of the situation (that it's a joke).
The supposed evolutionary utility of this is that it lets members of a tribe know what behavior is wrong within the tribe, thereby helping it recognize outsiders. The problem with this is that outsiders' behavior isn't always funny. If the new student asks for both cream and lemon in their tea, that's funny. If the new employee swears and makes racist comments all the time, that's offensive. If the guy sitting behind you starts moaning and grunting, that's worrying. What's the difference? Why is this difference useful?