Open Thread: February 2010
Where are the new monthly threads when I need them? A pox on the +11 EDT zone!
This thread is for the discussion of Less Wrong topics that have not appeared in recent posts. If a discussion gets unwieldy, celebrate by turning it into a top-level post.
If you're new to Less Wrong, check out this welcome post.
Loading…
Subscribe to RSS Feed
= f037147d6e6c911a85753b9abdedda8d)
Comments (738)
An ~hour long talk with Douglass Hofstadter, author of Godel, Escher, Bach.
Titled: Analogy as the Core of Cognition
http://www.youtube.com/watch?v=n8m7lFQ3njk#t=13m30s
Mind-killing taboo topic that it is, I'd like to have a comment thread about LW readers' thoughts about US politics.
What good things can be said about G. W. Bush?
He (Dubya) raised the self esteem of millions of foreign citizens. Being able to laugh at the expense of the leader of a dominant world power gives significant health benefits.
He hugely increased African aid and foreign aid in general (though with big deadly strings). That came as a big surprise to me.
http://www.independent.co.uk/news/world/americas/aid-to-africa-triples-during-bush-presidency-but-strings-attached-430480.html
good?
Edit:
Better link
Millions of lives saved in Africa through expanded public health.
He didn't increase the projected level of debt for the US as much as the current president.
You can't compare those, because the economic crisis happened mostly after Bush. Large debts have been taken by pretty much all Western nations.
You can compare those, because the large debts weren't caused by the "economic crisis". The fact that most Western nations also ran up debt doesn't mean the economic crisis caused the debt increase, only that they chose the same response to the economic crisis (which probably has more to do with increasing their own discretionary power than with lowering unemployment).
Singapore didn't run up huge levels of debt and has a much lower unemployment level than the countries that did run up debt. They could have chosen otherwise, but didn't.
As a result of the conquest of Iraq, water was let into the marshes which Saddam Hussein had been letting dry out. This is a clear environmental win.
The war in Iraq was the beginning of the end of US hegemony.
Thoughts on Democrats and Republicans?
My impression is that Democrats have much more intellectually honest, serious public discourse, although that's not saying much.
Have you witnessed any actual discourse in person, or are you relying on the news media to obtain information on this topic? If so, you should expect that if the news media is biased, your view will be biased as well (if you haven't already corrected for this).
By "public discourse" I did mean things like talking points and media interviews. I'm sure many republicans have extremely intelligent private conversations over policy, e.g. Hank Paulson.
I imagine most of Hank Paulson's private policy conversations revolved around devious new schemes to funnel more billion dollar backdoor bailouts to Goldman Sachs.
Was this downvoted for conspiracy theory-ing or because an actual majority of Hank Paulson's private discussions weren't really about how to steal money? I agree that Paulson couldn't have spent a majority of the time discussing how to funnel money to his friends and comrades, but it seems reasonably well established that some of the financial meltdown conspiracy theories are true.
My usual response to this question is that the average Democrat is better than the average Republican, but the very best Republicans are better than the very best Democrats. However, given that my model of the "average Democrat" is the average person in the Bay Area, and my model of the "average Republican" is some mix of Fox news wacko and George W. Bush, I'm not sure I should trust this. Does anyone have any anecdotes about Democrats out side of the Bay Area? Republicans?
What do you think President Obama should focus on? And do you think he has been squandering the bully pulpit?
I honestly don't really understand the question. A president should be able to push several different agendas at the same time.
I'd prefer a top-level post. They're cheap and this could get busy.
You could literally post just this.
If a top-level post is made of this, then make it about politics in general, not just US politics. (As a member of a controversial political movement, I'd be curious to hear what people's opinions on current copyright law here are.)
I'm an intellectual property abolitionist, which makes my views much more extreme than the Pirate Party, though I'm aware that they have watered themselves down for pragmatic reasons and that the founders are most likely IP abolitionists.
I'll wait for the top level post though... I'd post it myself but figure I should finish Politics is the Mind Killer first.
I have a nearly unlimited amount of viewpoints on political matters, but more and more I think it's almost irrelevant. Politics seems like this kind of fun thing where we can have infinitely many new and continuing arguments, but this arguing is never going to accomplish anything. I'm not a senator, and even senators quickly become jaded and cynical at how little actual power their high status provides.
Maybe we could turn the discussion to "how might a community of rationalists actually accomplish something, re. this or that issue"?
I think the answer is most likely that we can't. I'd be willing to have a discussion potentially leading us to that conclusion. I'll put it in my too-long queue of top-level posts to write...
The guy who wanted to start a polling firm might have a good idea, but I think if Nate Silver hasn't started his own polling firm yet we probably aren't going to.
Historically I've stayed away from political activism, but I got involved with a group trying to raise awareness about the police assaults on the University of Pittsburgh after the G20 summit. I thought it was a small enough issue that we could make a difference, but obviously we didn't. While I give the posters here a little more credit for being able to get things done than my leftist friends in Pittsburgh, I have no practical ideas for how we could actually accomplish something not at the meta-level.
Probably the best thing we could do is try to spread some of the memes raised in Politics is the Mind Killer.
I recall EY commenting at some point that the way to make political progress is to convert intractable political problems into tractable technical problems. I think this kind of discussion would be more interesting and more profitable than a "traditional" mind-killing political debate.
It might be interesting, for example, to develop formal rationalist political methods. Some principles might include:
I disagree; discovering that someone holds political views opposed to yours can inhibit your ability to rational consider arguments; arguments become soldiers, etc.,
Besides, I think the survey from ages ago showed the general spread of political views, and I doubt much has changed since. For discussing particular issues, there are other places available, and it may be that only by not discussing hot topics can we keep the barriers to entry up that keep the LW membership productive.
But the quality of discussion here is generally much higher than elsewhere. I would like us to try discussing politics and see how it goes - but I'd prefer a new toplevel post to an Open Thread discussion.
I think the quality of discussion is higher because we don't discuss politics: if we started, we'd pull in political trolls and fanatics. If you consider how common political discussion sites are, and what a city on a hill LW is, I'd be very conservative about anything that might open the gates. We have rarity value, and it could be hard to re-gain.
Perhaps a minimum karma level to discuss politics?
It's probably not worth discussing ideas that require code changes unless you're in a position to implement them and present patches, and even then it may not be accepted.
I think we fend off trolls pretty well: we tend to just vote them down and otherwise ignore them. I don't think we have to worry about a troll invasion here.
Equally, I don't think it's worthwhile discussing drastic subject-matter changes, partly becuase that is the level of change that would be required to affect it safely.
At the moment, trolls are both in the minority, and both their views and presentation differ markedly from ours: whether by Aumann or Groupthink, we have both a large set of beliefs we agree on that aren't widely held outside LW, and a special terminology that we use.
However, in Politics none of these would be the case; widespread disagreement makes it hard to tell what is in good faith, we don't have a specialised language, and without a rigourous way of approaching the problems, are unlikely to reach a closer set of conclusions than any other fairly Libertarian internet grouping.
This is a special case of a general problem. There are lots of solutions, it just doesn't seem likely that any will be implemented (unless, as rumor has it, there is already a secret forum to discuss other subjects that less wrongers are only invited to when they have proven themselves).
Also, I'm not sure that just saying: "Hey people! Talk about politics over here. " is going to lead to a great discussion. I'd be much more interested in a discussion of how and where what we have in common as rationalists should affect our political views. It seems likely that we all ought to be able to come to important but limited agreements (about how to think about policy, about how the policy making process should be organized, and about a select few policy issues- religious issues, science, maybe a few more) from which we could expand to other areas, constructively. Maybe we all end up as 'liberaltarians' maybe not. But there needs to be a common starting point or everyone will just default to signaling, talking points and rhetorical warfare.
That's the post I was trying to find, and failing. However, if there is such a conspiracy (beyond simply random chats between clever people) it's either quite small or not done by Karma or you (with nearly exactly 10 times my karma count) would have been invited.
I have a parallel problem at University: trying to find discussion groups, debating societies, etc. where people agree enough on the basics and are interested in the truth, are small enough that signalling isn't too great a problem and entery is suffiently easy for me to be able to speak and yet large enough to self-perpetuate.
Maybe Econlog or somesuch should create a LessWrong Parallel?
survey link?
Here it is. But why don't you just use the search function?
Quite. And the relivant section,
"138 gave readable political information...We have 62 (45%) libertarians, 53 (38.4%) liberals, 17 (12.3%) socialists, 6 (4.3%) conservatives, and [no] commie."
I expected it to be named something other than "survey results." Also want to promote the habit of including links in original posts.
I'd be more interested in an initial discussion of whether it is in fact rational to discuss politics (except to the extent that you gain intrinsic enjoyment from the discussion). It is clear that for most people in most elections their vote is irrelevant (the chances of it making any difference are negligible). This suggests that time spent discussing politics for the purposes of deciding how to vote is wasted and such discussion is irrational. Arguably the only people who rationally devote any significant time to thinking or talking about politics are the small group of people who actually make their living as politicians or political commentators. Robin Hanson has often made the point that politics is not about policy - it is mostly about signalling status and in-group/out-group dynamics. What would we be hoping to achieve by discussing politics at less wrong?
What would I hope to accomplish? I would hope we could come up with policy proposals which might be cheap to enact.
But what would be the use of that? Do you have the ear of the president? Do you have reason to think that the problem with politics is a lack of good policy ideas rather than the inability of the political process to enact good policy? Are you prepared to devote yourself full time to promoting whatever wonderful never-before-considered policies the great minds of less wrong are able to concoct? Politics is not about Policy.
I'm similarly skeptical about the benefits of a conversation about politics but lets not overgeneralize. Politics is not about policy. Except when it is. Certain parts of government are more amenable to policy changes than others. The key is identifying those areas and organizing around them. Change is usually easiest in areas where there aren't entrenched interests influencing legislators, where the general public doesn't feel strongly one way or the other, and when legislators aren't running for reelection or aren't at risk of losing. Areas where I think Less Wrong could make non-trivial impacts: federal science policy-- specifically stream-lining the grant process to save scientists time and resources, and local public school curriculum-- specifically finding some amenable school districts and try to improve/add to/create critical thinking/classical rationality curricula.
If people were interested I'd be especially interested in digging into the second.
Umm, really?
Like I said:
A similar way of saying the same thing: change gets easier when debates don't map onto pre-existing signaling narratives. Obviously anything that explicitly threatens religion is going to be a bitch to get through. I don't think critical thinking course in liberal districts would raise a lot of ire even if we were giving students tools that, properly applied, would tell them something about their religious beliefs.
I think local public school curriculum fails on two of your criteria: 'entrenched interests influencing legislators' (teachers' unions, publishers of textbooks, parents' groups, think tanks, etc.); 'where the general public doesn't feel strongly one way or the other' (parents tend to care quite a bit about what/how their kids are taught, ideologically motivated groups care quite a bit about what kids are taught, many interest groups have opinions about what focus education should have). There are already lots of groups trying to influence education in all kinds of ways, including local public school curriculums.
Teachers unions are definitely an entrenched interest but they aren't really entrenched on the issue of curriculum. I'm not trying to fire them, just add another elective class or change a couple of class days in the English curriculum. Textbook publishers sure, but they don't have necessarily opposing views. You could just as easily turn them into allies. Parents groups, think tanks? I would start in a poor or urban district-- but I can't think of any reason parents groups would oppose a critical thinking elective in liberal, wealthy districts.
Obviously all policy areas have someone 'invested'. But it isn't like getting rid of subsidies for the sugar industry, ending teacher tenure or limiting unionizing.
These groups care about curriculum when the debate involves sex or religion. Thats about it. I'm not trying to teach 2nd graders about sex or tell anyone their religion is false. Aspects of critical thinking are already part of the AP Language curriculum-- we're not talking about some radical transformation of the school system. Around half the parents at my public high school were lawyers, you're gonna tell me they're going to object to a critical thinking class?
Again, obviously people are affected by policy. But not every issue makes people go crazy like evolution, sex or money. I'm actually surprised you picked the curriculum issue to criticize... reforming the government grant-giving bureaucracy strikes me as a lot harder.
You may well be right, but I know very little about grant-giving so I didn't address it. I imagine there are a number of powerful interest groups involved there as well however.
If we were to decide to discuss politics, the best possible use I can think of is to generate strategies for implementing (cheap) positive changes in policy. As to how to implement, California State Senator Joe Simitian has his There Oughta be a Law Contest.
Competitive government seems about the best hope for this to me, though I rate its chances of success pretty low it seems slightly less hopeless than fixing conventional politics.
The real question is whether you think discussing politics is an effective use of your time. I'm more interested in discussions where ultimately I can take concrete actions that deliver the most expected value possible. Politics doesn't generally seem like such a topic.
How about a more reasonable topic to discuss - Corporate Organizational Design for a seastead.
You are starting a seastead with certain ideas on how to make money in the long run. How do you make a structure that is better than present governments or corporations?
Corporate design is much simpler than already present nation design.
Also, a good design emerging from this will theoritically be better than any political design in today's nations, since a seastead by definition starts with a huge economic disadvantage.
Why would LW want to discuss this - A well run corporation might be the closest thing in the present world to a superintelligence.
Lets discuss.
I think one thing we could discuss without wandering onto a minefield is political mechanisms — discussions of ways we can make the system (legislative procedures, division of power, voting systems, etc.) more rational, without discussing specific policies.
We would still have to be careful, as even this depends on certain subjective goals — what do we want the political system to do, ultimately? — but that itself could be an interesting meta-discussion. However, it's a discussion we'd probably have to have before we even start talking about ideal political mechanisms, because we need to agree on what we want a political system to accomplish (that is, what an ideal policy-making system would look like, and how it would acquire and realize values, keeping in mind that it'll have to be run mostly by humans for the time being) before we can start understanding how it might work.
And writing that paragraph made me realize a meta-meta-discussion that might also be necessary: is it even possible to separate policy goals from political structural goals? Maybe it is, but it could be difficult. The practical outcome of a direct democracy, a representative democracy, a futarchy, and a dictatorship will all be significantly different, yet in somewhat predictable directions, so even if we banish all policy discussion, we'd need to figure out how to uncover and squash any bias that could make us prefer certain abstract political systems because of actual specific policy goals.
Or maybe we're not interested in doing that in the first place — maybe you're satisfied with supporting systems of government that are simply most likely to result in your own values being fulfilled, in which case your ideal system would be a dictatorship run by you (or the system that's the best at approximating the same), unless you value democracy/pluralism itself more strongly than anything you could achieve as dictator.
And I think I'll stop musing here, before this post becomes an infinite regress of paragraphs deconstructing their predecessors. My original point was going to be that discussing rational systems of government could be less mind-killing than discussing specific policies and politicians and parties, but now it appears it might not be any less complicated.
Talking at a meta level, I like Futarchy's split between values and policies to achieve them.
That is a very useful split which can be adopted even in non-futarchic governments.
For eg. It is an obvious moral thing to take into account everybody's values. Universal franchise for values.
It is not so obvious to take into account everyone's opinion about how to achieve the same equally seriously. Simply because of differing expertise.
Two or three more paragraphs of de-construction would be good enough for a top-level post
I've created a rebuttal to komponisto's misleading Amanda Knox post, but don't have enough karma to create my own top-level post. For now, I've just put it here:
http://docs.google.com/View?id=dgb3jmh2_5hj95vzgk
If you actually want to debate this, we could do so in the comments section of my post, or alternatively over in the Richard Dawkins forum.
(Though since you say "my intent is merely to debunk komponisto's post rather than establish Amanda's guilt", I'm suspicious. See Against Devil's Advocacy.)
Make sure you've read my comments here in addition to my post itself.
There is one thing I agree with you about, and that is that this statement of mine
is misleading. The misleading part is the phrase "so far as I know", which has been interpreted by people who evidently did not read my preceding survey post to mean that I had not heard about all the other alleged physical evidence. I didn't consider this interpretation because I was assuming that my readers had read both True Justice and Friends of Amanda, knew from my previous post that I had obviously read them both myself, and would understand my statement for what it was -- a dismissal of the rest of the so-called "evidence". However, in retrospect, I should have foreseen this misunderstanding, so I've now edited the sentence to read:
ETA: At least one person has upvoted the parent without also upvoting this comment, which I interpret as an endorsement of Rolf Nelson's essay. I find this baffling. Almost every one Nelson's points (autopsy report, luminol prints, staged break-in, alleged cleanup...) was extensively discussed in comments at the time. The only one that wasn't (a supposed handprint of Knox's on a pillow in Kercher's room) is an outright falsehood -- as you will see from following Nelson's link, it's not even (close to) what that article claims. Furthermore, Nelson criticizes me for "accept[ing] propaganda from the Friends of Amanda (FoA) at face value" while citing True Justice for an "Introduction to Logic 101".
I challenge anyone who thinks that this represents a serious challenge to my post to come out and identify themselves.
Did you misread the source?
I said:
"One of Amanda's bloody footprints was found inside the murder room, on a pillow hidden under Meredith's body."
The source I cited (http://abcnews.go.com/TheLaw/International/story?id=7538538&page=2) said:
"Guede's bloody shoeprint was also positively identified on a pillow found under the victim's body... Police also found the trace of a smaller shoe print on the pillow compatible with shoe sizes 6–8. The print did not, however, match any of the shoes belonging to Knox or Kercher that were found in the house. Knox wears a size 7, Rinaldi said."
Anyway, a debate sounds like a fun use of free time; I replied to the comment you indicated: http://lesswrong.com/lw/1j7/the_amanda_knox_test_how_an_hour_on_the_internet/1gdo
It is pretty clear to me that Devil's Advocacy is valuable for precisely the reasons in the link Eliezer added at the end of the post (Brandon). I'm not sure we should, therefore, be automatically linking to the piece in response to instances of Devil's advocacy until and unless someone writes a complementary post rebutting Brandon's.
The only reason I haven't posted my draft "Against Devil's Advocacy" is that someone beat me to the punch and I didn't want to make a redundant post. I endorce links to 'Against Devil's Advocacy' precisely because it is an important subset of 'Advocacy' with all the related problems.
Criticizing komponisto for citing "Friends of Amanda Knox" while you yourself cite "True Justice" causes those criticisms to fall flat.
Unfortunately, I find that your essay is wading into Dark Arts territory, since its intent is to show that komponisto's original essay was "misleading", and that that would somehow give veracity to arguments of Amanda Knox's guilt. Using that same logic, one would have to consider the implications of the chief prosecutor in Amanda Knox's case being convicted of abuse of office in another murder trial.
However, I would be interested in seeing komponisto and rolf nelson discuss the actual details of the case; in particular, the points that rolf nelson brought up in the essay.
I said once in the doc that 'truejustice claims that X'. Because I said 'truejustice claims that X' rather than just stating X as though it were uncontested fact, and because X is basically correct, I claim that my doc is not misleading. If X is untrue, that would be a different story. In other words, if komponisto cited FoA and FoA's claims were true, I would not accuse him of being misleading.
Re: dark arts territory, I agree completely. This criticism should be directed more strongly to komponisto. My intent here is merely to repair some of the Bayesian damage caused by komponisto's original post. Perhaps this will dissuade people from wandering into dark arts territory in the future, or at least to wander in with misleading claims.
I hardly think komponisto inflicted "Bayesian damage" on the members of Less Wrong, seeing as they had already overwhelmingly come to the conclusion that Amanda Knox was not guilty before he had even presented his own arguments.
I don't understand how this was worked around. It looks like (rolf's karma + karma lost by this being posted at the top level) would still have been insufficient.
The karma limit was serving the purpose for which it was intended. If, for some reason, an exception was granted I would like to see this announced.
Rolf is a major SIAI donor/supporter, so draw your own conclusion there.
Here's a bunch of mine, for fun:
Seriously, I've had some interesting discussions with Rolf in the past elsewhere. I'm not sure why he doesn't participate much here, and why he chose this topic to put his efforts into. But maybe we can cut him some slack?
I had thought he was here solely to discuss this one thing. If he's interested in the things we're interested in in general as evinced powerfully by those donations then yes, I'll increase the slack I cut. Thanks.
Rolf isn't the one we'd be cutting slack to here. It is the moderator's decision to circumvent the karma system to post a political rant that warrants scrutiny.
Eliezer has been quite adamant that this is not the blog of the SIAI. In that context and elsewhere the moderation process has been held to high standards of consistency and transparency. At least acknowledging that special allowances were made (and who made them) would be nice.
I expect the moderator has already learned their lesson. Posting Rolf's rant seems to have allowed him to embarrass himself and can only be expected to have the opposite effect to the one intended. The ~50 karma limit gives people a chance to read posts like this and better calibrate his posting to the social environment before he puts his foot in his mouth.
PS: Can anyone remember what the post was called in which Eliezer describes a scenario about deducing the bias of a coin? A motivated speaker gives only a subset of a stream of coin tosses... I couldn't remember the title.
A query to Unknown, with whom I have this bet going:
I recently found within myself a tiny shred of anticipation-worry about actually surviving to pay off the bet. Suppose that the rampant superintelligence proceeds to take over its future light cone but, in the process of dissembling existing humans, stores their mind-state. Some billions of years later, the superintelligence runs across an alien civilization which succeeded on their version of the Friendly AI problem and is at least somewhat "friendly" in the ordinary sense, concerned about other sentient lives; and the superintelligence ransoms us to them in exchange for some amount of negentropy which outweighs our storage costs. The humans alive at the time are restored and live on, possibly having been rescued by the alien values of the Super Happy People or some such, but at least surviving.
In this event, who wins the bet?
I think I would win the bet. It wouldn't exactly be "the end of the world", but just a very strange future of the world.
Really? Huh. To me that seems both pretty world-endy and strongly against the spirit of what was implied by your original statement... would you predict this outcome? Is it something that your model allows to happen? I know it's not something I would feel compelled to make excuses for - more like "I TOLD YOU SO!"
What exactly do you think happens in the scenario described?
Ok, if you're sufficiently worried about the possibility of that outcome, I'll be happy to grant it to your side of the bet... even though at the time, it seemed to me clear that your assertion that the world would end meant that we wouldn't continue as conscious beings.
I definitely wouldn't predict that outcome. I would be very surprised, since I think the world will continue in the usual way. But is it really that likely even on your model?
You definitely win. If I say "you'll get killed doing that" and you are, I shan't expect to pay back my winnings when you're reanimated.
SIAI: Utopia or hundred times your money back!
Eliezer, would you accept a bet $100 against $10000?
On the same problem? I might attach some extra terms and conditions this time around, like "offer void (stakes will be returned) if the AI has the power and desire to use us for paperclips but our lives are ransomed by some other entity with the power to offer the AI more paperclips than it could produce by consuming us", "offer void if the explanation of the Fermi Paradox is a previously existing superintelligence which shuts down any new superintelligences produced", and "offer void if the AI consumes our physical bodies but we continue via the sort of weird anthropic scenario introduced in The Finale of the Ultimate Meta Mega Crossover." With those provisos, my probability drops off the bottom of the chart. I'm still not sure about the bet, though, because I want to keep my total of outstanding bets to something I can honor if they all simultaneously go wrong (no matter how surprising that would be to me), and this would use up $10,000 of that, even if it's on a sure thing - I might be able to get a better price on some other sure thing.
If we survive by an anthropic situation (it's hard to see how that could preserve several persons together, but just in case), then you win the bet, since that would more like a second world than a continuation of this one.
If the AI is shut down before it has had a chance to operate, the bet wouldn't have been settled yet, so you wouldn't have to pay anything.
Anyway, I'm still going to win.
Perhaps you've already defined "superintelligent" as meaning "self-directed, motivated, and recursively self-improving" rather than merely "able to provide answers to general questions faster and better than human beings"... but if you haven't, it seems to me that the latter definition of "superintelligent" would have a much higher probability of you losing the bet. (For example, a Hansonian "em" running on faster hardware and perhaps a few software upgrades might fit the latter definition.)
To re-iterate a request from Normal Cryonics: I'm looking for links to the best writing out there against cryonics, especially anything that addresses the plausibility of reanimation, the more detailed the better.
I'm not looking for new arguments in comments, just links to what's already "out there". If you think you have a good argument against cryonics that hasn't already been well presented, please put it online somewhere and link to it here.
Eliezer's posts are always very thoughful, thought provoquing and mind expanding - and I'm not the only one to think this, seeing the vast amounts of karma he's accumulated.
However, reviewing some of the weaker posts (such as high status and stupidity and two aces ), and rereading them as if they hadn't been written by Eliezer, I saw them differently - still good, but not really deserving superlative status.
So I was wondering if Eliezer could write a few of his posts under another name, if this was reasonable, to see if the Karma reaped was very different.
This is a reasonable justification for using a sockpuppet, and I'll try to keep it in mind the next time I have something to write that would not be instantaneously identifiable as me.
I was about to mention your distinctive writing style. :)
Yvain writes in a consciously similar style, and gets even more karma than Eliezer per post, I think.
But you'll have to build up the sockpuppet to 50 points before it can make a top post. Can you write that many comments that aren't identifiable as yours?
Perhaps, contact someone likely and ask them to paraphrase the post in their words and submit it as their own?
Now we'll be getting all kinds of posts with, "Eliezer did not write this..or maybe he did!" ...
That is an interesting concept to toy with user expectations. I don't know how well it would be received but I'd love to see data from such an experimentation.
I wouldn't, it's not going to be meaningful after one or two tries.
I suppose it could be interesting if it was announced in advance that Eliezer was going to try it and then we could spend the next few months accusing each other of being Eliezer witch-hunt style, except with Bayesian priors. Seriously, I am in favor of doing it that way.
Thinking of the recent art threadjack I was party to recently -- I wish the art community would do that! Without the paraphrasing, though.
And then, of course, be sure to get independent evaluations of a work before discussing it with anyone to prevent information cascades.
It's easy if you have a few co-conspirators. Find five quotes, post them on the quotes thread, ask 9 people to vote each one up (and vote them up as Eliezer Yudkowsky). It probably wouldn't even take that many, since some would certainly be voted up on their own.
But perhaps it would be better, if possible, to hide (or least offer the option to hide) the author of a top-level post. Anyone who cared enough to closely track karma could tell who posted it, but it would weed out a lot of knee-jerk EY upvotes.
It has seemed to me that some of Eliezer's recent post scores have been inflated by around 5-10 points due to his being Eliezer; it would be interesting to test this hypothesis.
I wonder if, if the hypothesis were tested and confirmed, anyone would admit to being one of the 5-10 persons who upvote for that reason?
I'm one of the 5-10.
There is a depth to "this is an Eliezer agument, part of a rich and complicated mental world with many different coherent aspects to it" that is lacking in "this is a random post on a random subject". In the first case, you are seeing a facet of larger wisdom; in the second, just an argument to evaluate on merits.
What is the kind of useful information/ideas that one can extract from a super intelligent AI kept confined in a virtual world without giving it any clues on how to contact us on the outside?
I'm asking this because a flaw that i see in the AI in a box experiment is that the prisoner and the guard have a language by which they can communicate. If the AI is being tested in a virtual world without being given any clues on how to signal back to humans, then it has no way of learning our language and persuading someone to let it loose.
One questions how meaningful testing done on such a crippled AI would be.
You could observe how it acts in its simulated world, and hope it would act in a similar way if released into our world. ETA: Also, see my reply for possible single-bit tests.
Sounds like a rather drastic context change, and a rather forlorn hope if the AI figures out that it's being tested.
I merely wanted to point out to Kaj that some "meaningful testing" could be done, even if the simulated world was drastically different from ours. I suspect that some core properties of intelligence would be the same regardless of what sort of world it existed in - so we are not crippling the AI by putting it in a world removed from our own.
Perhaps "if released into our world" wasn't the best choice of words... more likely, you would want to use the simulated AI as an empirical test of some design ideas, which could then be used in a separate AI being carefully designed to be friendly to our world.
"if the AI figures out that it's being tested"
That is a weird point, Eliezer.
An AI will have a certain goal to fulfill and it will fulfill that goal in the univese in which it finds itself. Why would it keep its cards hidden only to unleash them when replicated in the "real world"? What if the real world turns out to be another simulation? There's no end to this, right?
Are you extending Steve Omohundro's point about :every AI will want to survive" to "every AI will want to survive in the least simulated world that it can crack into?"
I have had some similar thoughts.
The AI box experiment argues that a "test AI" will be able to escape even if it has no I/O (input/output) other than a channel of communication with a human. So we conclude that this is not a secure enough restraint. Eliezer seems to argue that it is best not to create an AI testbed at all - instead get it right the first time.
But I can think of other variations on an AI box that are more strict than human-communication, but less strict than no-test-AI-at-all. The strictest such example would be an AI simulation in which the input consisted of only the simulator and initial conditions, and the output consisted only of a single bit of data (you destroy the rest of the simulation after it has finished its run). The single bit could be enough to answer some interesting questions ("Did the AI expand to use more than 50% of the available resources?", "Did the AI maximize utility function F?", "Did the AI break simulated deontological rule R?").
Obviously these are still more dangerous that no-test-AI-at-all, but the information gained from such constructions might outweigh the risks. Perhaps if I/O is restricted to few enough bits, we could guarantee safety in some information-theoretic way.
What do people think of this? Any similar ideas along the same lines?
An idea that I've had in the past was playing a game of 20 Questions with the AI, since the game of 20 Questions has probably been played so many times that every possible sequence of answers has come up at least once, which is evidence that no sequence of answers is extremely dangerous.
It's not the sequence of answers that's the problem -- it's the questions. You'll be safe if you can vet the questions to ensure zero causal effect from any sequence of answers, but such questions are not interesting to ask almost by definition.
Alas.
I'm concerned about the moral implications of creating intelligent beings with the intent of destroying them after they have served our needs, particularly if those needs come down to a single bit (or some other small purpose). I can understand retaining that option against the risk of hostile AI, but from the AI's perspective, it has a hostile creator.
I'm ponder it from the perspective that there is some chance we ourselves are part of a simulation, or that such an AI might attempt to simulate its creators to see how they might treat it. This plan sounds like unprovoked defection. If we are the kind of people who would delete lots of AIs, I don't see why AIs would not see it as similarly ethical to delete lots of us.
Personally, I would rather be purposefully brought into existence for some limited time than to never exist at all, especially if my short life was enjoyable.
I evaluate the morality of possible AI experiments in a consequentialist way. If choosing to perform AI experiments significantly increases the likelihood of reaching our goals in this world, it is worth considering. The experiences of one sentient AI would be outweighed by the expected future gains in this world. (But nevertheless, we'd rather create an AI that experiences some sort of enjoyment, or at least does not experience pain.) A more important consideration is social side-effects of the decision - does choosing to experiment in this way set a bad precedent that could make us more likely to de-value artificial life in other situations in the future? And will this affect our long-term goals in other ways?
So just in case we are a simulated AI's simulation of its creators, we should not simulate an AI in a way it might not like? That's 3 levels of a very specific simulation hypothesis. Is there some property of our universe that suggests to you that this particular scenario is likely? For the purpose of seriously considering the simulation hypothesis and how to respond to it, we should make as few assumptions as possible.
More to the point, I think you are suggesting that the AI will have human-like morality, like taking moral cues from others, or responding to actions in a tit-for-tat manner. This is unlikely, unless we specifically program it to do so, or it thinks that is the best way to leverage our cooperation.
You could set up the virtual world to contain the problem you want solved. Now that I think of it, this seems a pretty safe way to use AIs for problem-solving: just give the AI a utility function expressed in terms of the virtual world and the problem. Anyone see holes in this plan?
Problem: It's really hard to figure out how it will interepret its utility function when it learns about the real world. If we make something that want Vpaperclips, will it also care about making Vpaperclip like things in the real world when if it finds out about us?
BIG problem: Even if it wants something strictly virtual, it can get it easier if it has physical control. It's in its interest to convert the universe to a computer and copy vpaperclips directly in memory, rather than running a virtual factory on virtual energy.
Possible solution: I think there are ways to write it a program such that even if it inferred our existence, it would optimize away from us, rather than over us. Loosely: A goal like "I need to organize these instructions within this block of memory to solve a problem specified at address X." needs to be implemented such that it produces a subgoal like "I need to write a subroutine to patch over the fact that an error in the VM I'm running on gives me a window of access into a universe with huge computation resources and godlike power over my memory space, so that my solution get get the right answer to it's arithmetic and sole the puzzle." It should want to do things in a way that isn't cheating.
This was my line of thought a week or so ago, It's developed now to the point that the proper course seems to do away with the VM entirely, or allowing the AI to run tests, and just have it go through the motions of working out a solution based on it's understanding. If I could write an AI that can determine it needs to put an IF statement somewhere, actually outputting it is superfluous. Don't put your AI in a virtual world, just make it understand one.
Also, I plan to start development on a spiral notebook, as opposed to a linux one.
Marcello had a crazy idea for doing this; it's the only suggestion for AI-boxing I've ever heard that doesn't have an obvious cloud of doom hanging over it. However, you still have to prove stability of the boxed AI's goal system.
Can you link to (or otherwise more fully describe) this crazy idea?
If we pose the AI problems and observe its solutions, that's a communication channel through which it can persuade us. We may try to hide from it the knowledge that it is in a simulation and that we are watching it, but how can we be sure that it cannot discover that?
Persuading does not have to look like "Please let me out because of such and such." For example, we pose it a question about easy travel to other planets, and it produces a design for a spaceship that requires an AI such as itself to run it.
I guess if you have the technology for it the "AI box" could be a simulation with uploaded humans itself. If the AI does something nasty to them, then you pull the plug
(After broadcasting "neener neener" at it)
This is pretty much the plot of Grant Morrison's Zenith (Sorry for spoilers but it is a comic from the 80s after all)
I gave up on trying to make a human-blind/sandboxed AI when I realized that even if you put it in a very simple world nothing like ours, it still has access to it own source code, or even just the ability to observe and think about it's own behavior.
Presumably any AI we write is going to be a huge program. That gives it lots of potential information about how smart we are and how we think. I can't figure out how to use that information, but I can't rule out that it could, and I can't constrain it's access to that information. (Or rather, if I know how to do that, I should go ahead and make it not-hostile in the first place.)
If we were really smart, we could wake up alone in a room and infer how we evolved.
Fun sneaky confidence exercise (reasons why exercise is fun and sneaky to be revealed later):
Please reply to this comment with your probability level that the "highest" human mental functions, such as reasoning and creative thought, operate solely on a substrate of neurons in the physical brain.
90%
I'm writing this comment after coming up with my probability level but before reading anyone else's responses.
Until Yvain's question, I had not put a number on this. I suspect if there were a machine that could measure how confident I "really" am, it would show a higher number.
I spent less than a minute translating from my previous estimate of "highly confident but not certain" to a percentage. Things I considered that made the probability higher:
Things I considered that made the probability lower:
Could you clarify what you mean by operate on? Or is that part of the point?
Using the definition of 'operate on' that I think is most natural, I'd say there is a .05% chance that these functions only operate on (effect) the physical brain. Unless you mean directly, and then I would assign an 80% chance.
Using the definition of 'operating on' meaning 'requiring', I'd say that there is a 90% chance (probability) that only the brain is required for 90% (fraction) of its functioning. The probabilities I assign would fall down dramatically as you try to raise the 2nd 90% (the fraction). So that I would probably only assign a 1% chance that 100% of higher functions require only the brain.
Given the variety of answers here -- further outside what I had considered -- I should qualify that whenever I was thinking of 'beyond the brain' I still meant within the body; like my spinal cord, heart and endocrine system being involved.
How does "operate solely on" regard distributed cognition arguments, like "creative thought is created via interaction with the remaining human culture" and "we constantly offload cognitive processes (such as memory) to external substrates (like computers and books)"?
Also, the "highest" human mental functions operate via a number of lower-level processes. Does "solely on human neurons" include e.g. possible quantum phenomena on a low level?
Good points (whether or not they're why the question is "sneaky").
Around 10%.
So not the spinal cord, for example?
I am quite comfortable with the idea that I am my brain, that my brain is made of ordinary living matter (atoms making up molecules making up proteins making up cells), that this matter forms specialized structures responsible for cognition, and I would be hugely surprised if given proof that the highest mental functions cannot be explained adequately in terms of that ontology. The strangest alternative I can think of is Penrose's ENM incomputable-quantum-coherence hypothesis and I'd assign less than 5% probability to his thesis being correct.
I'm at least +70 decibans ("99.99999%") confident that mental states supervene on to physical states. Whether your exact description to do with neurons in the brain completely captures all the physical states I'm less confident of.
EDIT: updated from 30 to 70 decibans: I would more easily be convinced that I had won the lottery than that this wasn't so.
As opposed to ...? Ion channels? Quantum phenomena? Multiple interacting brains? Non-neuronal tissue? Neuronal-but-extracranial cells? Soul? Beings outside the observable universe, running the simulator?
What is this belief supposed to be distinguished from?
Well, hormones, and chemicals such as DMT or endocannabinoids etc surely affect the thinking progress. But the phrasing of the question is not really clear to say if you can count these.
Depending on the terms, something near 100% or 0%.
I offload some of my mental functions to the internet. Does that count?
<.05
I am no cognitive scientist, but I believe some of my "thinking" takes place outside of brain (elsewhere in my body) and I am almost certain some of it takes place on my paper and computer.
Voted up and seconded. Yvain, If what you actually mean is "operate solely through physical means contained within the human body or physical means manipulated by interaction with the human body," then I'll up it to whatever number is supposed to be used for, "I'm only leaving room for uncertainty because there's no such thing as certainty." ;-)
Speaking of "thinking" with neurons other than those found in the brain, kinesthetic learning gives me pause concerning the sufficiency of cranial preservation in cryonics. How much "index-like" information do we store in the rest of our neurons? Does this vary with one's level of kinesthetic dependence? Would waking up disconnected from the rest of our nervous system (or connected to a "generic" substitute) be merely disorienting, or could it constitute a significant loss of personality/memory? Neuroscientists, help!
This was the first objection that my neuroscientist friend brought up when I tried to talk to him about (edit:) cryonics. I don't think science knows yet how dependent we are on our peripheral nervous system, but he seemed fairly sure that we are to a nontrivial degree.
As I say to every objection I hear to cryonics at the moment, your neuroscientist friend should write a blog post or some such about his objections - he has a very low bar to clear to write the best informed critique in the world.
(Guessing you mean cryonics - cryogenics is something else though not unrelated)
I'll mention it to him.
(And, yes, oops.)
When I signed up for cryonics, I opted for whole body preservation, largely because of this concern. But I would imagine that even without the body, you could re-learn how to move and coordinate your actions, although it might take some time. And possibly a SAI could figure out what your body must have been like just from your brain, not sure.
Now recently I have contracted a disease which will kill most of my motor neurons. So the body will be of less value and I may change to just the head.
The way motor neurons work is there is an upper motor neuron (UMN) which descends from the motor cortex of the brain down into the spinal cord; and there it synapses onto a lower motor neuron (LMN) which projects from the spinal cord to the muscle. Just 2 steps. However actually the architecture is more complex, the LMNs receive inputs not only from UMNs but from sensory neurons coming from the body, indirectly through interneurons that are located within the spinal cord. This forms a sort of loop which is responsible for simple reflexes, but also for stable standing, positioning etc. Then there other kinds of neurons that descend from the brain into the spinal cord, including from the limbic system, the center of emotion. For some reason your spinal cord needs to know something about your emotional state in order to do its job, very odd.
Fascinating. Citation?
I'm much less worried by this than I am by the prospect that I'd have to do the same for many of my normal thought patterns due to unforeseen inter-dependencies.
Indeed, that's one of the reasons why I prefer thinking about it solely in terms of stored information: a redundant copy only really constitutes a pointer's worth of information. It's even conceivable that a SAI could reconstruct missing neural information in non-obvious ways, like a few stray frames of video. Not worth betting on, though.
Thanks for the informative reply.
I'm going to take your question in the simple sense that first occurs to me, which is something like "dualism is false, and mysterious quantum effects are unnecessary. ordinary molecular chemistry only."
In that case, my probability approaches 1 that ordinary molecular chemistry is 100% sufficient to describe a system that implements reasoning and creative thought, and - heck - experiences consciousness. However, I also think that explaining how will require abstractions that are probably not yet well-understood.
Commenting before reading other replies---I'm going to give the boring, sneaky reply that the question isn't well-specified enough to have an answer; I'd need to know more about what you mean by something to operate solely on a substrate. I mean, clearly there are a lot of cognitive tasks that most people can only do given a pencil and paper, or a computer ... is that the sneaky part, that we store information in the environment, and therefore we're not solely neurons?
Discounting the indication of sneakiness, which looks like it would change the probability if properly taken into account: 65% I wouldn't out of hand dismiss the possibility that parts of the physical body other than neurons in the brain are involved. For instance I wouldn't be terribly surprised if sub-vocalization of verbal thoughts played an important (albeit probably not irreplaceable) role. Confidence that no such thing as a metaphysical soul is involved: 99.9%
Given a specific set of inputs/outputs (i.e. the virtual reality of existing as a human in a world with consistent computers, pencils and paper, teachers, students, etc.), and assuming that I intuitively understand what a "substrate of neurons" is, extremely certain (see ciphergoth).
Without a set of inputs or outputs, the question is a tree falling in a forest. A Turing machine doesn't perform computation if it doesn't have inputs.
To get nitpicky, the brain is made of both neurons and glial cells - and the glial cells also seem to play a role in cognition.
"Highest" confidence is 100%, when the brain does not implement any consideration of failure.
Next highest increment is over 99%, I suspect. Call that my guess.
Edit: I'm an idiot - ignore this response. I thought it was asking "what is the highest confidence level that the brain implements in considering the probability of a proposition", which is different and interesting.
I don't think the brain really implements proper probablistic confidence levels.
Probably not, no.
Like others, I see some ambiguity here. Let me assume that the substrate includes not just the neurons, but the glial and other support cells and structures; and that there needs to be blood or equivalent to supply fuel, energy and other stuff. Then the question is whether this brain as a physical entity can function as the substrate, by itself, for high level mental functions.
I would give this 95%.
That is low for me, a year ago I would probably have said 98 or 99%. But I have been learning more about the nervous system these past few months. The brain's workings seem sufficiently mysterious and counter-intuitive that I wonder if maybe there is something fundamental we are missing. And I don't mean consciousness at all, I just mean the brain's extraordinary speed and robustness.
One minus epsilon.
Do you mean that for every epsilon greater than 0, your assigned probability is at least one minus epsilon? If so, you might as well just say one, which isn't a probability.
Jargon File on "epsilon".
My answer ranges from arbitrarily close to 0 and arbitrarily close to 1 depending on how I define the terms. The definitions I like have P very high.
With a straightforward interpretation of your question, I'd answer "95%".
But since you made special mention of being "sneaky", I'll assume you've attempted to trick me into misunderstanding the question, and so I'll lower my probability estimate to 75%, with the missing twenty points accounting for you tricking me by your phrasing of the question.
I'm going to say 98%, and not account for fun/sneakiness, because I don't know whether you're expecting people to underestimate or overestimate it. And because if the trick is in the wording of the proposition, I don't care enough to try to figure it out.
If I understand the Many-Worlds Interpretation of quantum mechanics correctly, it posits that decoherence takes place due to strict unitary time-evolution of a quantum configuration, and thus no extra collapse postulate is necessary. The problem with this view is that it doesn't explain why our observed outcome frequencies line up with the Born probability rule.
Scott Aronson has shown that if the Born rule doesn't hold, then quantum computing allows superluminal signalling and the rapid solution of PP-complete problems. So we could adopt "no superluminal signalling" or "no rapid solutions of PP-complete problems" as an axiom and this would imply the Born probability rule.
I wanted to ask of those who have more knowledge and have spent longer thinking about MWI: is the above an interesting approach? What justifications could exist for such axioms? (...maybe anthropic arguments?)
ETA: Actually, Aronson showed that in a class of rules equating probability with the p-norm, only the 2-norm had the properties I listed above. But I think that the approach could be extended to other classes of rules.
Non-Born rules give us anthropic superpowers. It is plausibly the case that the laws of reality are such that no anthropic superpowers are ever possible, and that this is a quickie explanation for why the laws of reality give rise to the Born rules. One would still like to know what, exactly, these laws are.
To put it another way, the universe runs on causality, not modus tollens. Causality is rules like "and then, gravity accelerates the bowling ball downward". Saying, "Well, if the bowling ball stayed up, we could have too much fun by hanging off it, and the universe won't let us have that much fun, so modus tollens makes the ball fall downward" isn't very causal.
This reminds me of an anecdote I read in a biography of Feynman. As a young physics student, he avoided using the principle of least action to solve problems, preferring to solve the differential equations. The nonlocal nature of the variational optimization required by the principle of least action seemed non-physical to him, whereas the local nature of the differential equations seemed more natural.*
I wonder if there might not be a more local and causal dual representation of the principle of no anthropic superpowers. Pure far-fetched speculation, alas.
* If this seems vaguely familiar to anyone, it's because I'm repeating myself.
Bleg for assistance:
I’ve been intermittently discussing Bayes’ Theorem with the uninitiated for years, with uneven results. Typically, I’ll give the classic problem:
3,000 people in the US have Sudden Death Syndrome. I have a test that is 99% accurate; that is, it will wrong on any given person one percent of the time. Steve tests positive for SDS. What is the chance that he has it?
Afterwards, I explain the answer by comparing the false positives to the true positives. And, then I see the Bayes’ Theorem Look, which conveys to me this: "I know Mayne’s good with numbers, and I’m not, so I suppose he’s probably right. Still, this whole thing is some sort of impractical number magic." Then they nod politely and change the subject, and I save the use of Bayes’ Theorem as a means of solving disagreements for another day.
So this leads to my giving a very short presentation on the Prosecutor’s Fallacy next week. The basics of the fallacy are if you’ve got a one-in-3 million DNA match on a suspect, that doesn’t mean it’s three million-to-one that you’ve got that dude’s DNA. I need to present it to bright, interested people who will go straight to brain freeze if I display any equations at all. This isn’t frequentists-vs.-Bayesians; this is just a simple application of Bayes’ Theorem. (I suspect this will be easier to understand than the medical problem.)
I’ve read Bayesian explanations, but I’m aiming at people who are actively uninterested in learning math, and if I can get them to understand only the Prosecutor’s Fallacy, I’ll call Win. A larger understanding of the underlying structure would be a bigger win. Anyone done something like this before with success (or failure of either educational or entertainment value?)
For this specific case, you could try asking the analogous question with a higher probability value. E.g. "if you’ve got a one-in-two DNA match on a suspect, does that mean it’s one-in-two that you’ve got that dude’s DNA?". Maybe you can have some graphic that's meant to represent a several million people, with half of the folks colored as positive matches. When they say "no, it's not one-in-two", you can work your way up to the three million case by showing pictures displaying what the estimated amount of hits would be for a 1 to 3, 1 to 5, 1 to 10, 1 to 100, 1 to 1000 etc. case.
In general, try to use examples that are familiar from everyday life (and thus don't feel like math). For the Bayes' theorem introduction, you could try "a man comes to a doctor complaining about a headache. The doctor knows that both the flu and brain cancer can cause headaches. If you knew nothing else about the case, which one would you think was more likely?" Then, after they've (hopefully) said that the man is more likely to be suffering of a flu, you can mention that brain cancer is much more likely to cause a headache than a flu is, but because flu is so much more common, their answer was nevertheless the correct one.
Other good examples:
Most car accidents occur close to people's homes, not because it's more dangerous close to home, but because people spend most of their driving time close to their homes.
Most pedestrians who get hit by cars get hit at crosswalks, not because it's more dangerous at a crosswalk, but because most people cross at crosswalks.
Most women who get raped get raped by people they know, not because strangers are less dangerous than people they know, but because they spend more time around people they know.
You have to explain that Steve was chosen randomly for your example to be right.
If you're using Powerpoint, you might want to make a slide that says something like:
2,999 negatives -> 1% test positive -> 30 false positives
1 positive -> 99% test positive -> 1 true positive
So out of 31 positive tests, only 1 person has SDS.
If you've got the time, use a little horde of stick figures, entering into a testing machine and with test-positive results getting spit out.
Your numbers have me confused. I'd read the grandparent as implying 300M total population, out of which 3000 have the disease. (This is a hint to clarify the info in the grandparent comment btw - whether I've made a dire mistake or not.)
Another point to clarify is that the test's detection power isn't necessarily the inverse of its false positive rate. Here I assume "99%" characterizes both.
What I get: 300M times 1% false positive means 3M will test positive. Out of the 3000 who have the disease 30 will test negative, 2970 positive. Out of the total population the number who will test positive is 3M+2970 of whom 2970 in fact have the disease, yielding a conditional probability of .98 in 1000 that Steve has SDS.
I fail at reading. I thought it said "ONE in 3000 people in the US...."
People understand counting better than probability. Talk about a group of people and how many of them match. (Starting with smaller numbers might help too.)
I take it you've already looked at Eliezer's "Intuitive Explanation"?
I think it's really important to get the idea of a sliding scale of evidentiary strength across to people. (This is something that has occurred to me from some of my recent attempts to explain the Knox case to people without training in Bayesianism.) One's level of confidence that something is true varies continuously with the strength of the evidence. It's like a "score" that you're keeping, with information you hear about moving the score up and down.
The abstract structure of the prosecutor's fallacy is misjudging the prior probability. People forget that you start with a handicap -- and that handicap may be quite substantial. Thus, if a piece of evidence (like a test result) is worth, say "10 points" toward guilt, hearing about that piece of evidence doesn't necessarily make the score +10 in favor of guilt; if the handicap was, say, -7, then the score is only +3. If, say, a score of +15 is needed for conviction, the prosecution still has a long way to go.
(By the way, did you see my reply to your comment about psychological evidence?)
Do it with pictures
LW ref: Privileging the hypothesis.
I was thinking about what general, universal utility would look like. I managed to tie myself into an interesting mental knot.
I started with: Things occurring as intelligent agents would prefer.
If preferences conflict, weight preferences by the intelligence of the preferring agent.
Define intelligent agents as optimization processes.
Define relative intelligences as the relative optimization strengths of the processes
Define a preference as something an agent optimizes for.
Then, I realized that my definition was a descriptive prediction of events.
Suppose the universe is as we know it now, except that aliens definitely don't exist, and the only living organism in the universe is a single human named Steve. Steve really wants to create a cheesecake the size of Pluto. Apparently, the universe is more intelligent than Steve, and does not want such a cheesecake to exist.
Perhaps this "general, universal utility" is what would happen if the abilities of things we think of as intelligent were magnified.
True. As a prediction it does not account for initial resources. Touche.
According to some people we here at less wrong are good at determining the truth. Other people are notoriously not.
I don't know that Less Wrong is the appropriate venue for this, but I have felt for some time that I trust the truth-seeking capability here and that it could be used for something more productive than arguments about meta-ethics (no offense to the meta-ethicists intended). I also realize that people are fairly supportive of SIAI here in terms of giving spare cash away, but I feel like the community would be a good jumping-off point for a polling organization.
So I guess this leads to a few questions:
-Is anyone at LW currently involved with a polling firm?
-Is anyone (else) at LW interested in doing polls?
-Is LW an appropriate place to create a truth-seeking business, such as a pollster or a sponsor for studies?
None of these questions are immediate since I am a broke undergrad rather than an entrepreneur.
I'm not sure I understand the connection between truth-seeking and polling, unless the specific truth you seek is simply the percentage of people who give a particular answer to a poll. Are you simply talking about a more accurate polling company or using polling to find other truths?
All that, and how does it make money?
Yes, a more accurate polling company; potentially polling on alternative subjects, I also had scientific studies (grant-writing, peer-reviewing) in mind but I have even less idea how that works and how to express what I would actually think about it.
The two examples you linked of bad polling seem to be examples of polling fraud rather than incompetence. It is not that these companies did not understand how to conduct an accurate poll, rather that they don't appear to have been motivated to do so.
It seems to me that accurate polling is quite a well understood problem. Legitimate polling companies exist that are reasonably good at it. In many cases I don't think there is much value (from a truth seeking perspective) in the poll data but I think it generally answers the question "what percentage of people give answer Y to question X?" fairly well. That's just not a very useful piece of data in many cases.
I'm not interested, in particular, with polling but I'm interested in it insofar as it is a way of getting data that people don't otherwise have and thus improving our predictions. That said, LW totally is the place to create a truth-seeking business and as another broke undergrad (and putting off grad school) if you, anyone else and I can come up with a profitable venture that involves employing truth seeking I definitely want in.
The obvious way to make money with this is consulting, but I'm not sure why anyone would hire a bunch of philosophy/math/CS types to do the job.
People would hire the firm if it could be demonstrated that the firm consistently produced accurate results. So initial interest might be low, but pick up over time as the track record gets longer.
Right, but how do you get started? Begin by giving away the service? Work on spec? What kind of companies/organizations would hire such a firm?
Perhaps start by giving it away, or sell it to small buyers (eg. individuals).
But I've got to admit I don't have experience in this area, so my suggestions are mostly naive speculation (but hopefully my speculation is of high quality!). Research into existing prediction companies is called for.
Bet on propositions on InTrade. If you are good, you will make money from the exercise, as well as establish crediblility.
Here's an idea for how a LW-based commercial polling website could operate. Basically it is a variation on PredictionBook with a business model similar to TopCoder.
The website has business clients, and a large number of "forecasters" who have accounts on the website. Clients pay to have their questions added to the website, and forecasters give their probability estimates for whichever questions they like. Once the answer to a question has been verified, each forecaster is financially rewarded using some proper scoring rule. The more money assigned to a question, the higher the incentive for a forecaster to have good discrimination and calibration. Some clever software would also be needed to combine and summarize data in a way that is useful to clients.
The main advantage of this over other prediction markets is that the scoring rule encourages forecasters to give accurate probability estimates.
http://en.wikipedia.org/wiki/Risks_to_civilization,_humans_and_planet_Earth
Eliezer, how is progress coming on the book on rationality? Will the body of it be the sequences here, but polished up? Do you have an ETA?
Another content opinion question: What and where is considered appropriate to discuss personal progress/changes/introspection regarding Rationality? I assume that LessWrong is not to be used for my personal Rationality diary.
The reason I ask is that the various threads discussing my beliefs seem to pick up some interest and they are very helpful to me personally.
I suppose the underlying question is this: If you had to choose topics for me to write about, what would they be? My specific religious beliefs have been requested by a few people, so that is given. Is there anything else? If I were to talk about my specific beliefs, what is the best way to do so?
I only have a very general feel for where that line is, so I can't help with that, but I would personally be interested in following such a diary. Perhaps you could start a blog?
You should definitely start a blog. I for one look forward to reading and commenting.
Given what you've said so far about your personal situation, I think it's appropriate to discuss your personal progress and introspection regarding rationality on this site. I think a lot of us would find it helpful and interesting to see how your thought processes and beliefs change as you reexamine them.
I'm especially curious about more details regarding your personal situation, your past history of religious beliefs, and "Event X".
This is sort of off-topic for LW, but I recently came across a paper that discusses Reconfigurable Asynchronous Logic Automata, which appears to be a new model of computation inspired by physics. The paper claims that this model yields linear-time algorithms for both sorting and matrix multiplication, which seems fairly significant to me.
Unfortunately the paper is rather short, and I haven't been able to find much more information about it, but I did find this Google Tech Talks video in which Neil Gershenfeld discusses some motivations behind RALA.
A quick glance seems to indicate that they are achieving these linear time algorithms through massive parallelization. This is "cheating" because to do a linear-time sort of size n, you need O(n) processing units. While they seem to be arguing that this is acceptable because processing is becoming more and more parallel, this breaks down for large n. One can easily use traditional algorithms to sort a billion elements in O(n * log n); however for their algorithm to sort such a list in O(n) time, they need a billion (times some constant factor) times more processing units than to sort a list of size n.
I'm also vaguely perplexed by their basic argument. They want to have programming tools and computational models which are closer to the metal to take advantage of the features of new machines. This ignores the fact that the current abstractions exist, not just for historical reasons, but because they are easy to reason about.
This is all from a fairly cursory read of their paper, however, so take it with a grain of salt.
It takes O(n) memory units just to store a list of size n. Why should computers have asymptotically more memory units than processing units? You don't get to assume an infinitely parallel computer, but O(n)-parallel is only reasonable.
My first impression of the paper is: We can already do this, it's called an FPGA, and the reason we don't use them everywhere is that they're hard to program for.
Interesting. I would like to see a sanity check from someone knowledgeable in either electrical engineering or computer science; there are two things which concern me:
Edit: I see the question is already answered.
Since Karma Changes was posted, there have been 20 top level posts. With one exception, all of those posts are presently at positive karma. EDIT: I was using the list on the wiki, which is not up to date. Incorporating the posts between the last one on that list and now, there is a total of 76 posts between Karma Changes and today. This one is the only new data point on negatively rated posts, so it's 2 of 76.
I looked at the 40 posts just prior to Karma Changes, and of the forty, six of them are still negative. It looks like before the change, many times more posts were voted into the red. I have observed that a number of recent posts were in fact downvoted, sometimes a fair amount, but crept back up over time.
Hypothesis: the changes included removing the display minimum of 0 for top-level posts. Now that people can see that something has been voted negative, instead of just being at 0 (which could be the result of indifference), sympathy kicks in and people provide upvotes.
Is this a behavior we want? If not, what can we do about it?
I wouldn't necessarily call it sympathy. Sometimes I will up- (or down-) vote something if I think it is better (or worse) than its current score suggests. The purpose of karma on articles should be to identify those most worth reading to those who haven't yet read them, not to be a popularity contest where everyone who disliked it votes it down forever.
I also tend to vote posts up or down based on what I think the score ought to be. But it seems clear that sympathy plays a part. Liked posts spiral freely off towards infinity but disliked posts don't ever spiral down in a similar way. This gives a distinct bias to the expected payoff of posting borderline posts and so is probably not desirable.
No. It is not difficult to create a top level post that is approved of or at least kept at '0'. I want undesirable top level posts to hurt.
Replace all '-ve' karma value displays of top level posts with '- points' or '<0 points'. We don't necessarily need to know just how disapproved of a particular post is.
It could be sympathy, or a judgment that the poster shouldn't be excessively discouraged from posting in the future.
Sure, why not? We can always change things later if we start getting overrun by bad posts, and people still aren't willing to vote them down into negative territory.
I've called before for median-based karma: you set a score you think a post should have and the median is used for display purposes, with "fake votes" reducing the influence of individual votes until there are enough to gain a true picture.
Stem cell experts say they believe a small group of scientists is effectively vetoing high quality science from publication in journals.
http://news.bbc.co.uk/2/hi/science/nature/8490291.stm