[Link] A superintelligent solution to the Fermi paradox

-1 Will_Newsome 30 May 2012 08:08PM

Here.

Long story short, it's an attempt to justify the planetarium hypothesis as a solution to the Fermi paradox. The first half is a discussion of how it and things like it are relevant to the intended purview of the blog, and the second half is the meat of the post. You'll probably want to just eat the meat, which I think is relevant to the interests of many LessWrong folk.

The blog is Computational Theology. It's new. I'll be the primary poster, but others are sought. I'll likely introduce the blog and more completely describe it in its own discussion post when more posts are up, hopefully including a few from people besides me, and when the archive will give a more informative indication of what to expect from the blog. Despite theism's suspect reputation here at LessWrong I suspect many of the future posts will be of interest to this audience anyway, especially for those of you who take interest in discussion of the singularity. The blog will even occasionally touch on rationality proper. So you might want to store the fact of the blog's existence somewhere deep in the back of your head. A link to the blog's main page can be found on my LessWrong user page if you forget the url.

I'd appreciate it if comments about the substance of the post were made on the blog post itself, but if you want to discuss the content here on LessWrong then that's okay too. Any meta-level comments about presentation, typos, or the post's relevance to LessWrong, should probably be put as comments on this discussion post. Thanks all!

Newcomblike problem: Counterfactual Informant

0 Clippy 12 April 2012 08:25PM

I want to propose a variant of the Counterfactual Mugging problem discussed here.  BE CAREFUL how you answer, as it has important implications, which I will not reveal until the known dumb humans are on record.

Here is the problem:

Clipmega is considering whether to reveal to humans information that will amplify their paperclip production efficiency.  It will only do so if it expects that, as a result of revealing to humans this information, it will receive at least 1,000,000 paperclips within one year.

Clipmega is highly accurate in predicting how humans will respond to receiving this information.

The smart humans' indifference curve covers both their current condition and the one in which Clipmega reveals the idea and steals 1e24 paperclips.  (In other words, smart humans would be willing to pay a lot to learn this if they had to, and there is an enormous "consumer surplus".)

Without Clipmega's information, some human will independently discover this information in ten years, and the above magnitude of the preference for learning now vs later exists with this expectation in mind.  (That is, humans place a high premium on learning it how, even though they will eventually learn it either way.)

The human Alphas (i.e., dominant members of the human social hierarchy), in recognition of how Clipmega acts, and wanting to properly align incentives, are considering a policy: anyone who implements this idea in making paperclips must give Clipmega 100 paperclips within a year, and anyone found using the idea but not having donated to Clipmega is fined 10,000 paperclips, most of which are given to Clipmega.  It is expected that this will result in more than 1,000,000 paperclips being given to Clipmega.

Do you support the Alphas' policy?

Problem variant: All of the above remains true, but there also exist numerous "clipmicros" that unconditionally (i.e. irrespective of their anticipation of behavior on the part of other agents) reveal other, orthogonal paperclip production ideas.  Does your answer change?

Optional variant:  Replace "paperclip production" with something that current humans more typically want (as a result of being too stupid to correctly value paperclips.)

The Future of Education

2 Michelle_Z 14 February 2012 08:58PM

This morning I read an interesting post on the future of education. I thought it would be interesting to have some members of LessWrong discuss it. I know it is idealistic, but some of the points raised were interesting.

 

Alex Lindsay

The Future of Education

Through an anomaly in the space-time continuum, I fell into the future last week. It was an odd sensation … traveling through time. But at least I made it back. I spent the time I had there at a local school and thought I would share what I saw …

Grades are gone. 

Kids aren't in grade 1 or grade 3 … which was described to me as a "rudimentary" way to "cattle" students. The admins were gentle about it, explaining to me that when school was paper based, there just wasn't the facility to customize the classroom to the student. They explained that even though it was horribly inefficient, they understood why it needed to exist. They did point out that it ran a decade too long, affecting millions … but I changed the subject before it got ugly.

Instead, kids in school have individual achievement levels, which are different for every subject. They have 0-1800 points in each subject. Each student works at their own pace through these milestones and moving forward when they get near perfect scores. A student might have 1500 in one subject and 400 in another. Because everything is online and integrated, there aren't really "grades" like A, B, C, and D … kids are just accumulating points.

When kids max out in a subject, they can spend more time on other subjects; if they are on pace (they are expected to accumulate 100 points a year), they can create independent studies. Many students work very hard to move through the point structure so they can have more free time … which is structured but still up to them. Added resources are applied to students more than 100 points behind their pace. You end up with 20% of the students passing through the system with very little help beyond the structure, 60% getting some help, and 20% getting a much larger amount of attention to move through the system.

Lectures are gone

Lectures the way we know them don't really exist. Most of school is divided into 4 processes: Movies, Games, Projects and Discussion. Movies and Games largely exist on the tablets every student has (these look like iPads but they roll up into a baton-like structure). Projects are done with other students … there are very few opportunities to work on projects alone as it's not seen as an effective character development process in today's job world (where the only people working in a vacuum are doing low-paying work). Discussions are lead by subject experts.

These "Subject Experts" are what used to be called teachers. They are a breed among themselves. They are part brainiac in their field, part Tony Robbins. Their job is to make their subject exciting to learn.

Usually, they begin training for their position very early in life, adding heavy levels of presentation and interpersonal skills to their study load. They work as assistants after reaching 1800 levels in all subjects and focus on a particular subject to master. They train, practice and are allowed to present for basic student events for about decade before they are actually allowed to "solo" an educational subject. It's an incredible amount of work but it also pays well -- salaries for these experts average in what is, in today's money, about $300,000 a year, with the top experts making over a $1M a year. This is largely based on their demand globally. Students are essentially given what is the equivalent to a voucher for discussions and are able to choose their lecturers for each seminar they choose to attend.

There are less of them, as you might guess. Typically about 50,000-100,000 of them at any one time -- much less than the 4M that were working at the peak of the process in the US. While this sounds crazy, we have to remember that most of the objective training is happening interactively within the training tools. There are also over 2,000,000 assistants vying for the Expert positions, providing ongoing support for the students and smaller talks. These assistants are paid, but it's a hard life while they prove themselves.

The discussions are really global events: students attend from all over the world. Some are in theatres together, some are at home, some are in smaller event locations. Students at these events are of all ages. They attend based on their achievement levels, not their age. So you may have 1000 students from 15 countries, aged from 10 to 18. Questions are posted and voted on by the group to percolate to the top and be discussed by the expert (or experts -- there are often people from given industries participating in these events). The events are productions, usually with intense graphics and TV-level production values.

Movies

Top content experts are often the designers and hosts for the online training tools that all students use for their ongoing training. These movies provide core knowledge that is part of the interactive guides that students use to move through their subject matter. These movies are Star Wars-level FX films that explain the subject matter. Some of them are period pieces, some are animated adventures. I'm told as Hollywood stumbled and the education system began to build, many producers moved to this content for survival.

Games

These movies are closely connected to games that the students play -- while they might be something that looks like a geeky version of "Civilizations" or a first person-shooter from the Civil War, or a Physics game that requires students to understand gravity, momentum, etc. The games are not an extra -- they are required and the student scores are connected to their overall achievement scores. These games don't look like the square interactive "Educational" games today. In fact, they have nearly replaced the mindless games of today. I'm told as the government started spending billions on game development, EA and others could A) see that there was money to be made and B) could see there wouldn't be much time to play other games … leading to the new "development gaming" movement.

Projects

Projects are really global affairs. As students reach a required project in a subject (based on their point path), they go online to find others around the world in the same situation. These teams are usually 4-6 people and dig into creating interactive reports that are a mix of video, animation, and text. In addition to deepening subject understanding, the projects are designed to build global relationships and communication. Students are not permitted to do more than 3 projects with the same people in their career. They do rate each other, which builds a bit of a "global team marketplace." Project teams need to have an "Average" score … meaning, high-scoring individuals are encouraged to bring in one or two individuals with lower scores to help them progress. Students are first teamed with "Assistant" mentors, then industry mentors and then industry experts (who are partially in it to recruit students out of school, as their companies pay top dollar for the ability to participate in the "advance" programs, and finding qualified talent has become an incredibly competitive market).

Subjects are slightly adjusted.

Languages (most kids learn English, Chinese, French or Spanish, and an elective language which can things like Japanese, Russian, Arabic or Sign Language). Students are required to be fluent in 4 languages by age 16. Most begin at 4 or 5. A large portion of this training looks like "Rosetta Stone"… then students get into more conversational classes (vocabulary drills are all on the tablets). By age 12, many students are simply taking classes in other languages.

Math - Pretty much the same but with an emphasis on problem solving. There are many fewer equations and more integrated problems. Of course, the students are much more advanced as the more interactive teaching processes have been extremely effective in this area.

Literature - An odd thing to call it given I never saw a book or anything that resembled it. Still, students listen to the classics and discuss the philosophical implications.

Sciences - Kids start in Physics almost at day one. They learn about basic engineering principles in the 300 levels (what could be kind of considered 3rd grade, but it's really what was taught in high school before). 

Global Society (what used to be called Civil Studies and History) - Understanding how cultures around the world evolved to their current state. Understanding one's own state is important, but usually only addressed in the global context.

Personal Development - Now considered one of the most important skills in a highly competitive global jobs market, kids are educated from nearly day one on effective person skills. These skills are not moral or religious, just simply good operating behaviour … and what it takes to be effective.

Creative Arts - From Drawing to Music to Performing Arts … these skills are seen as intrinsic to creating a "Creative" individual that can think their way through the complex issues of the day. In the West, there aren't many "doing-only" jobs that haven't shipped overseas or replaced by technology. As a result, being creative has become much more important.

Physical Arts (what used to be PE) - Ah, the days of Kickball are gone. This is a fairly gruelling daily regime that includes nutrition education and customized exercise processes. Martial Arts, Gymnastics and other dexterity building classes are the norm. Over 25% of the student body globally is a Black Belt. This has more to do with training the mind and self-esteem than person protection.

I asked how this happened in the US … I was told it didn't. In fact, the US was one of the last countries to adopt the still-controversial system. The stakeholders at the time resisted the change and called it too radical to be even tested. The result has been a steep investment to catch up with other countries and much higher unemployment in the US... as many of the information age jobs left the US over the 15 years they resisted the changes.

The revolution actually began in the emerging world, specifically in Africa. New fibre running into East and Southern Africa empowered African nations, with too many kids and not enough teachers, to augment their staff with new videos and interactive learning. The students of these early systems not only learned much faster, but become the most facile at building the content (as they were very familiar with it). I was told as much as 60% of all the content in the global infrastructure is created in Sub-Saharan Africa (Rwanda, Zimbabwe, South Africa and Tanzania).

Anyway, when I arrived back in this time, I wrote this all down as fast as I could to remember it. I hope you find it useful.

Was it really just a dream? I don't know. But if it is was a dream, it was a really good one.

 

Thoughts? Comments?

AI is not enough

-22 benjayk 07 February 2012 03:53PM

What I write here may be quite simple (and I am certainly not the first to write about it), but I still think it is worth considering:


Say we have an abitrary problem that we assume has an algorithmic solution, and search for the solution of the problem.


How can the algorithm be determined?
Either:
a) Through another algorithm that exist prior to that algorithm.
b) OR: Through something non-algorithmic.


In the case of AI, the only solution is a), since there is nothing else but algorithms at its disposal. But then we have the problem to determine the algorithm the AI uses to find the solution, and then it would have to determine the algorithm to determine that algorithm, etc...
Obviously, at some point we have to actually find an algorithm to start with, so in any case at some point we need something fundamentally non-algorithmic to determine a solution to an problem that is solveable by an algorithm.


This reveals something fundamental we have to face with regards to AI:

Even assuming that all relevant problems are solvable by an algorithm, AI is not enough. Since there is no way to algorithmically determine the appropiate algorithm for an AI (since this would result in an infinite regress), we will always have to rely on some non-algorithmical intelligence to find more intelligent solutions. Even if we found a very powerful seed AI algorithm, there will always be more powerful seed AI algorithms that can't be determined by any known algorithm, and since we were able to find the first one, we have no reason to suppose we can't find another more powerful one. If an AI recursively improves 100000x times until it is 100^^^100 times more powerful, it still will be caught up if a better seed AI is found, which ultimately can't be done by an algorithm, so that further increases of the most general intelligence always rely on something non-algorithmic.

But even worse, it seems obvious to me that there are important practical problems that have no algorithmic solution (as opposed to theoretical problems like the halting problem, which are still tractable in practice), apart from the problem of finding the right algorithm.
In a sense, it seems all algorithms are too complicated to find the solution to the simple (though not necessarily easy) problem of giving rise to further general intelligence.
For example: No algorithm can determine the simple axioms of the natural numbers from anything weaker. We have postulate them by virtue of the simple seeing that they make sense. Thinking that AI could give rise to ever improving *general* intelligence is like thinking that an algorithm can yield "there is a natural number 0 and every number has a successor that, too, is a natural number". There is simply no way to derive the axioms from anything that doesn't already include it. The axioms of the natural numbers are just obvious, yet can't be derived - the problem of finding the axioms of natural numbers is too simple to be solved algorithmically. Yet still it is obvious how important the notion of natural numbers is.
Even the best AI will always be fundamentally incapable of finding some very simple, yet fundamental principles.
AI will always rely on the axioms it already knows, it can't go beyond it (unless reprogrammed by something external). Every new thing it learns can only be learned in term of already known axioms. This is simply a consequence of the fact that computers/programs are functioning according to fixed rules. But general intelligence necessarily has to transcend rules (since at the very least the rules can't be determined by rules).


I don't think this is an argument against a singularity of ever improving intelligence. It just can't happen driven (solely or predominantly) by AI, whether through a recursively self-improving seed AI or cognitive augmentation. Instead, we should expect a singularity that happens due to emergent intelligence. I think it is the interaction of different kind of intelligence (like human/animal intuitive intelligence, machine precision and the inherent order of the non-living universe, if you want to call that intelligence) that leads to increase in general intelligence, not just one particular kind of intelligence like formal reasoning used by computers.

A case study in fooling oneself

-2 Mitchell_Porter 15 December 2011 05:25AM

Note: This post assumes that the Oxford version of Many Worlds is wrong, and speculates as to why this isn't obvious. For a discussion of the hypothesis itself, see Problems of the Deutsch-Wallace version of Many Worlds.

smk asks how many worlds are produced in a quantum process where the outcomes have unequal probabilities; Emile says there's no exact answer, just like there's no exact answer for how many ink blots are in the messy picture; Tetronian says this analogy is a great way to demonstrate what a "wrong question" is; Emile has (at this writing) 9 upvotes, and Tetronian has 7.

My thesis is that Emile has instead provided an example of how to dismiss a question and thereby fool oneself; Tetronian provides an example of treating an epistemically destructive technique of dismissal as epistemically virtuous and fruitful; and the upvotes show that this isn't just their problem. [edit: Emile and Tetronian respond.]

I am as tired as anyone of the debate over Many Worlds. I don't expect the general climate of opinion on this site to change except as a result of new intellectual developments in the larger world of physics and philosophy of physics, which is where the question will be decided anyway. But the mission of Less Wrong is supposed to be the refinement of rationality, and so perhaps this "case study" is of interest, not just as another opportunity to argue over the interpretation of quantum mechanics, but as an opportunity to dissect a little bit of irrationality that is not only playing out here and now, but which evidently has a base of support.

The question is not just, what's wrong with the argument, but also, how did it get that base of support? How was a situation created where one person says something irrational (or foolish, or however the problem is best understood), and a lot of other people nod in agreement and say, that's an excellent example of how to think?

On this occasion, my quarrel is not with the Many Worlds interpretation as such; it is with the version of Many Worlds which says there's no actual number of worlds. Elsewhere in the thread, someone says there are uncountably many worlds, and someone else says there are two worlds. At least those are meaningful answers (although the advocate of "two worlds" as the answer, then goes on to say that one world is "stronger" than the other, which is meaningless).

But the proposition that there is no definite number of worlds, is as foolish and self-contradictory as any of those other contortions from the history of thought that rationalists and advocates of common sense like to mock or boggle at. At times I have wondered how to place Less Wrong in the history of thought; well, this is one way to do it - it can have its own chapter in the history of intellectual folly; it can be known by its mistakes.

Then again, this "mistake" is not original to Less Wrong. It appears to be one of the defining ideas of the Oxford-based approach to Many Worlds associated with David Deutsch and David Wallace; the other defining idea being the proposal to derive probabilities from rationality, rather than vice versa. (I refer to the attempt to derive the Born rule from arguments about how to behave rationally in the multiverse.) The Oxford version of MWI seems to be very popular among thoughtful non-physicist advocates of MWI - even though I would regard both its defining ideas as nonsense - and it may be that its ideas get a pass here, partly because of their social status. That is, an important faction of LW opinion believes that Many Worlds is the explanation of quantum mechanics, and the Oxford school of MWI has high status and high visibility within the world of MWI advocacy, and so its ideas will receive approbation without much examination or even much understanding, because of the social and psychological mechanisms which incline people to agree with, defend, and laud their favorite authorities, even if they don't really understand what these authorities are saying or why they are saying it.

However, it is undoubtedly the case that many of the LW readers who believe there's no definite number of worlds, believe this because the idea genuinely makes sense to them. They aren't just stringing together words whose meaning isn't known, like a Taliban who recites the Quran without knowing a word of Arabic; they've actually thought about this themselves; they have gone through some subjective process as a result of which they have consciously adopted this opinion. So from the perspective of analyzing how it is that people come to hold absurd-sounding views, this should be good news. It means that we're dealing with a genuine failure to reason properly, as opposed to a simple matter of reciting slogans or affirming allegiance to a view on the basis of something other than thought.

At a guess, the thought process involved is very simple. These people have thought about the wavefunctions that appear in quantum mechanics, at whatever level of technical detail they can muster; they have decided that the components or substructures of these wavefunctions which might be identified as "worlds" or "branches" are clearly approximate entities whose definition is somewhat arbitrary or subject to convention; and so they have concluded that there's no definite number of worlds in the wavefunction. And the failure in their thinking occurs when they don't take the next step and say, is this at all consistent with reality? That is, if a quantum world is something whose existence is fuzzy and which doesn't even have a definite multiplicity - that is, we can't even say if there's one, two, or many of them - if those are the properties of a quantum world, then is it possible for the real world to be one of those? It's the failure to ask that last question, and really think about it, which must be the oversight allowing the nonsense-doctrine of "no definite number of worlds" to gain a foothold in the minds of otherwise rational people.

If this diagnosis is correct, then at some level it's a case of "treating the map as the territory" syndrome. A particular conception of the quantum-mechanical wavefunction is providing the "map" of reality, and the individual thinker is perhaps making correct statements about what's on their map, but they are failing to check the properties of the map against the properties of the territory. In this case, the property of reality that falsifies the map is, the fact that it definitely exists, or perhaps the corollary of that fact, that something which definitely exists definitely exists at least once, and therefore exists with a definite, objective multiplicity.

Trying to go further in the diagnosis, I can identify a few cognitive tendencies which may be contributing. First is the phenomenon of bundled assumptions which have never been made distinct and questioned separately. I suppose that in a few people's heads, there's a rapid movement from "science (or materialism) is correct" to "quantum mechanics is correct" to "Many Worlds is correct" to "the Oxford school of MWI is correct". If you are used to encountering all of those ideas together, it may take a while to realize that they are not linked out of logical necessity, but just contingently, by the narrowness of your own experience.

Second, it may seem that "no definite number of worlds" makes sense to an individual, because when they test their own worldview for semantic coherence, logical consistency, or empirical adequacy, it seems to pass. In the case of "no-collapse" or "no-splitting" versions of Many Worlds, it seems that it often passes the subjective making-sense test, because the individual is actually relying on ingredients borrowed from the Copenhagen interpretation. A semi-technical example would be the coefficients of a reduced density matrix. In the Copenhagen interpetation, they are probabilities. Because they have the mathematical attributes of probabilities (by this I just mean that they lie between 0 and 1), and because they can be obtained by strictly mathematical manipulations of the quantities composing the wavefunction, Many Worlds advocates tend to treat these quantities as inherently being probabilities, and use their "existence" as a way to obtain the Born probability rule from the ontology of "wavefunction yes, wavefunction collapse no". But just because something is a real number between 0 and 1, doesn't yet explain how it manages to be a probability. In particular, I would maintain that if you have a multiverse theory, in which all possibilities are actual, then a probability must refer to a frequency. The probability of an event in the multiverse is simply how often it occurs in the multiverse. And clearly, just having the number 0.5 associated with a particular multiverse branch is not yet the same thing as showing that the events in that branch occur half the time.

I don't have a good name for this phenomenon, but we could call it "borrowed support", in which a belief system receives support from considerations which aren't legitimately its own to claim. (Ayn Rand apparently talked about a similar notion of "borrowed concepts".)

Third, there is a possibility among people who have a capacity for highly abstract thought, to adopt an ideology, ontology, or "theory of everything" which is only expressed in those abstract terms, and to then treat that theory as the whole of reality, in a way that reifies the abstractions. This is a highly specific form of treating the map as the territory, peculiar to abstract thinkers. When someone says that reality is made of numbers, or made of computations, this is at work. In the case at hand, we're talking about a theory of physics, but the ontology of that theory is incompatible with the definiteness of one's own existence. My guess is that the main psychological factor at work here is intoxication with the feeling that one understands reality totally and in its essence. The universe has bowed to the imperial ego; one may not literally direct the stars in their courses, but one has known the essence of things. Combine that intoxication, with "borrowed support" and with the simple failure to think hard enough about where on the map the imperial ego itself might be located, and maybe you have a comprehensive explanation of how people manage to believe theories of reality which are flatly inconsistent with the most basic features of subjective experience.

I should also say something about Emile's example of the ink blots. I find it rather superficial to just say "there's no definite number of blots". To say that the number of blots depends on definition is a lot closer to being true, but that undermines the argument, because that opens the possibility that there is a right definition of "world", and many wrong definitions, and that the true number of worlds is just the number of worlds according to the right definition.

Emile's picture can be used for the opposite purpose. All we have to do is to scrutinize, more closely, what it actually is. It's a JPEG that is 314 pixels by 410 pixels in size. Each of those pixels will have an exact color coding. So clearly we can be entirely objective in the way we approach this question; all we have to do is be precise in our concepts, and engage with the genuine details of the object under discussion. Presumably the image is a scan of a physical object, but even in that case, we can be precise - it's made of atoms, they are particular atoms, we can make objective distinctions on the basis of contiguity and bonding between these atoms, and so the question will have an objective answer, if we bother to be sufficiently precise. The same goes for "worlds" or "branches" in a wavefunction. And the truly pernicious thing about this version of Many Worlds is that it prevents such inquiry. The ideology that tolerates vagueness about worlds serves to protect the proposed ontology from necessary scrutiny.

The same may be said, on a broader scale, of the practice of "dissolving a wrong question". That is a gambit which should be used sparingly and cautiously, because it easily serves to instead justify the dismissal of a legitimate question. A community trained to dismiss questions may never even notice the gaping holes in its belief system, because the lines of inquiry which lead towards those holes are already dismissed as invalid, undefined, unnecessary. smk came to this topic fresh, and without a head cluttered with ideas about what questions are legitimate and what questions are illegitimate, and as a result managed to ask something which more knowledgeable people had already prematurely dismissed from their own minds.

Do the people behind the veil of ignorance vote for "specks"?

1 D227 11 November 2011 01:26AM

The veil of ignorance as Rawls put it ..."no one knows his place in society, his class position or social status; nor does he know his fortune in the distribution of natural assets and abilities, his intelligence and strength, and the like." 

 

The device allows for certain issues like slavery and income distribution to be determined beforehand.  Would one vote for a society in which there is a chance of severe misfortune, but greater total utility?  e.g, a world where 1% earn $1 a day and 99% earn $1,000,000 vs. a world where everyone earns $900,000 a day.  Assume that dollars are utilons and they are linear (2 dollars indeed gives twice as much utility).  What is the obvious answer?  Bob chooses $900,000 a day for everyone.  

 

But Bob is clever and he does not trust himself that his choice is the rational choice, so he goes into self-dialogue to investigate:

Q: "What is my preference, value or goal(PVG), such that, instrumental rationality may achieve it?"

A "I my preference/value/goal is for there to be a world in which total utility is less, but severe misfortune eliminated for everyone"

Q " As an agent are you maximizing your own utility by your actions of choosing a $900,000 a day world?

A " Yes, my actions are consistent with my preferences; I will maximize my utility by achieving my preference of limiting everyone's utility.  This preference takes precedence.

Q: "I will now attack your position with the transitivity argument.  At which point does your consistency change?  What if the choices where 1% earns $999,999 and 99% earn 1,000,000?"

A: "My preference,values and goals have already determined a threshold, in fact my threshold is my PVG.  Regardless the fact that my threshold may be different from everyone else's threshold, my threshold is my PVG.  And achieving my PVG is rational."

Q: "I will now attack your position one last time, with the "piling argument".  If every time you save one person from destitution, you must pile on the punishment on the others such that everyone will be suffering."

A: "If piling is allowed then it is to me a completely different question.  Altering what my PVG is.  I have one set of values for a non piling and piling scenario.  I am consistent because piling and not piling are two different problems."

 

In the insurance industry, purchasing insurance comes with a price.  Perhaps 1.5% premium of the cost of reimbursing you for your house that may burn down.  The actuaries have run the probabilities and determine that you have a 1% chance that your house will burn down.  Assume that all dollar amounts are utilons across all assets.  Bob once again is a rational man.  Every year Bob is chooses to pay 1.5% in premium even though his average risk is technically a 1% loss, because Bob is risk adverse. So risk adverse that he prefers a world in which he has less wealth, the .5% went to the insurance companies making a profit. Once again Bob questions his rationality on purchasing insurance:

Q: "What is my preference?"

A: "I would prefer to sacrifice more than my share of losses( .5% more), for the safety-net of zero chance catastrophic loss."

Q "Are your actions achieving your values?"

A "Yes, I purchased insurance, maximizing my preference for safety."

Q "Shall I attack you with the transitivity argument?"

A "It wont work.  I have already set my PVG, it is a premium price at which I judge to make the costs prohibitive.  I will not pay 99% premium to protect my house , but I will pay 5%."

Q "Piling?"

A "This is a different problem now."

 

Eliezer's post on Torture vs. Dust Specks [Herehas generated lots of discussion as well as what Eliezer describes as interesting [ways] of [avoiding] the question.  We will do no sort of thing in this post, we will answer the question as intended; I will interpret that eye specks is cumulatively greater suffering than the suffering of 50 years. 

 My PVG tells me that I would rather have a speck in my eye, as well as the eye's of 3^^^3 people, than to risk to have one (perhaps me) suffer torture for 50 years, even though my risk is only 1/(3^^^3) which is a lot less than 50 years (Veil of ignorance).  My PVG is what I will maximize, and doing so is the definition of instrumental rationality.  

In short, the rational answer is not TORTURE or SPECKS, but depends on what your preference, values and goals are.  You may be one of those whose preference is to let that one person feel torture for 50 years, as long as your actions that steer the future toward outcomes ranked higher in your preferences, you are right too.

Correct me if I am wrong but I thought rationality did not imply that there were absolute rational preferences, but rather rational ways to achieve your preferences...

 

I want to emphasize that in no way did I intend for this post to declare anything.  And want to thank everyone in advance for picking apart every single word I have written.  Being wrong is like winning the lottery.  I do not claim to know anything, the assertive manner in which I wrote this post was merely a way to convey my ideas, of which, I am not sure off.   

 

 

 

Why We Can't Take Expected Value Estimates Literally (Even When They're Unbiased)

75 HoldenKarnofsky 18 August 2011 11:34PM

Note: I am cross-posting this GiveWell Blog post, after consulting a couple of community members, because it is relevant to many topics discussed on Less Wrong, particularly efficient charity/optimal philanthropy and Pascal's Mugging. The post includes a proposed "solution" to the dilemma posed by Pascal's Mugging that has not been proposed before as far as I know. It is longer than usual for a Less Wrong post, so I have put everything but the summary below the fold. Also, note that I use the term "expected value" because it is more generic than "expected utility"; the arguments here pertain to estimating the expected value of any quantity, not just utility.

While some people feel that GiveWell puts too much emphasis on the measurable and quantifiable, there are others who go further than we do in quantification, and justify their giving (or other) decisions based on fully explicit expected-value formulas. The latter group tends to critique us - or at least disagree with us - based on our preference for strong evidence over high apparent "expected value," and based on the heavy role of non-formalized intuition in our decisionmaking. This post is directed at the latter group.

We believe that people in this group are often making a fundamental mistake, one that we have long had intuitive objections to but have recently developed a more formal (though still fairly rough) critique of. The mistake (we believe) is estimating the "expected value" of a donation (or other action) based solely on a fully explicit, quantified formula, many of whose inputs are guesses or very rough estimates. We believe that any estimate along these lines needs to be adjusted using a "Bayesian prior"; that this adjustment can rarely be made (reasonably) using an explicit, formal calculation; and that most attempts to do the latter, even when they seem to be making very conservative downward adjustments to the expected value of an opportunity, are not making nearly large enough downward adjustments to be consistent with the proper Bayesian approach.

This view of ours illustrates why - while we seek to ground our recommendations in relevant facts, calculations and quantifications to the extent possible - every recommendation we make incorporates many different forms of evidence and involves a strong dose of intuition. And we generally prefer to give where we have strong evidence that donations can do a lot of good rather than where we have weak evidence that donations can do far more good - a preference that I believe is inconsistent with the approach of giving based on explicit expected-value formulas (at least those that (a) have significant room for error (b) do not incorporate Bayesian adjustments, which are very rare in these analyses and very difficult to do both formally and reasonably).

The rest of this post will:

  • Lay out the "explicit expected value formula" approach to giving, which we oppose, and give examples.
  • Give the intuitive objections we've long had to this approach, i.e., ways in which it seems intuitively problematic.
  • Give a clean example of how a Bayesian adjustment can be done, and can be an improvement on the "explicit expected value formula" approach.
  • Present a versatile formula for making and illustrating Bayesian adjustments that can be applied to charity cost-effectiveness estimates.
  • Show how a Bayesian adjustment avoids the Pascal's Mugging problem that those who rely on explicit expected value calculations seem prone to.
  • Discuss how one can properly apply Bayesian adjustments in other cases, where less information is available.
  • Conclude with the following takeaways:
    • Any approach to decision-making that relies only on rough estimates of expected value - and does not incorporate preferences for better-grounded estimates over shakier estimates - is flawed.
    • When aiming to maximize expected positive impact, it is not advisable to make giving decisions based fully on explicit formulas. Proper Bayesian adjustments are important and are usually overly difficult to formalize.
    • The above point is a general defense of resisting arguments that both (a) seem intuitively problematic (b) have thin evidential support and/or room for significant error.

continue reading »

To Speak Veripoop

-18 thre3e 18 July 2011 02:50AM

From the sociological point of view I cannot imagine a world without compulsory, god declared, laws for basic behaviors, such as sex-related, murder-related, and god-worship related behaviors. My outlook comes from my certainty that some minds are susceptible to the seeking of such compulsions, and my certainty that some other minds are susceptible to a need to supply such compulsions, sometimes as themselves as the authority, and sometimes as representatives of higher authority. The latter group always seems to produce some very successful iterations, from Moses to Jim Jones. . . As it is said in commerce, if there is demand, there will always be folks who will make it a life quest to supply that demand.

If what I'm saying has bases in fact, and if the atheistic challenge is to disenfranchise, dis-empower, organized religion, and other publicists of drivel, then how can mere logical, rational, rhetoric be looked to in order to bring about this goal? It seems evident to me that such rhetoric does not have the needed determinants to effect the goal. Rationality cannot seem to supply the needed compulsions. Thus, rationality goes unheeded.

I have an idea for a possible solution. What if we successfully substituted a new word for truth. What if it became common to say VERIPOOP in place of VERITAS? From that small acorn might grow reexamination of the human faculty for knowing, and claiming truth. It should be obvious to all, that we humans do not have a truth-knowing faculty. We can only know human level truth, which is always temporary and finitely circumscribed. Grass was known to be green for a long time in history, but, as we all know, green is not a property of grass any more. Nature supplies color only to those who are not color blind. Greenness is a human thing, not a grass thing. Reflecting white light at a certain wavelength is intrinsic to grass, but not color. We humans can know only truth that is bound to change in time, but "real" truth cannot change. It is already truth. Where else could it go?

Yes, there are mathematical proofs that would present themselves as truth forever. But it's easy to overlook the fact that all scientific and mathematical pronouncements are abstracts of reality. They may be correct within the confines of the postulates that undergird them, but reality is greater than any finite number of postulates. Further, postulates are arbitrarily chosen. Parallel lines may never meet, or always meet, or meet just under specified specified conditions. Therefore, that which is correct is not necessarily truth.This is a fact about the human knowledge horizon, the human condition. The horizon, wherever one draws it, however far we might advance in knowledge, is inexorably there. Yet the wild eyed compulsion addicts are willing to die for what? Why it's their "truth," of course. So, I say that the very word needs to be expunged, because, amazingly, every time it is uttered, it presents a lie. It claims that someone has corralled truth.

VERIPOOP would put us in our place. A new appreciation may develop of the human knowability horizon. How can one be an extremist when one knows that what one proclaims with vehemence is VERIPOOP? It seems to have a calming effect. Scientific veripoops are wonderful. The fact that presently the scientific method doesn't allow truth to be considered truth forever, as it did when science was in the hands of the compulsive knowers of Europe, (e.g. the Galileo problem), is also wonderful. But there is no other word available currently. Science must call its temporary findings truth, especially on true or false tests. Yet the facts show that they are a step down from truth. They are VERIPOOP!

 

My true rejection

-16 dripgrind 14 July 2011 10:04PM

Here's why I'm not going to give money to the SIAI any time soon.

Let's suppose that Friendly AI is possible. In other words, it's possible that a small subset of humans can make a superhuman AI which uses something like Coherent Extrapolated Volition to increase the happiness of humans in general (without resorting to skeevy hacks like releasing an orgasm virus).

Now, the extrapolated volition of all humans is probably a tricky thing to determine. I don't want to get sidetracked into writing about my relationship history, but sometimes I feel like it's hard to extrapolate the volition of one human.

If it's possible to make a Friendly superhuman AI that optimises CEV, then it's surely way easier to make an unFriendly superhuman AI that optimises a much simpler variable, like the share price of IBM.

Long before a Friendly AI is developed, some research team is going to be in a position to deploy an unFriendly AI that tries to maximise the personal wealth of the researchers, or the share price of the corporation that employs them, or pursues some other goal that the rest of humanity might not like.

And who's going to stop that happening? If the executives of Corporation X are in a position to unleash an AI with a monomaniacal dedication to maximising the Corp's shareholder value, it's probably illegal for them not to do just that.

If you genuinely believe that superhuman AI is possible, it seems to me that, as well as sponsoring efforts to design Friendly AI, you need to (a) lobby against AI research by any groups who aren't 100% committed to Friendly AI (pay off reactionary politicians so AI regulation becomes a campaign issue, etc.) (b) assassinate any researchers who look like they're on track to deploying an unFriendly AI, then destroy their labs and backups.

But SIAI seems to be fixated on design at the expense of the other, equally important priorities. I'm not saying I expect SIAI to pursue illegal goals openly, but there is such a thing as a false-flag operation.

While Michelle Bachmann isn't talking about how AI research is a threat to the US constitution, and Ben Goertzel remains free and alive, I can't take the SIAI seriously.

Anyone want to give Holmesian reasoning a try?

-4 [deleted] 28 June 2011 09:53PM

In Sherlock Holmes fiction, we see that Holmes is capable of making correct inferences using insufficient information and long, tenuous chains of reasoning. I'm curious what would happen if we tried to apply this in real life. Here's a riddle containing insufficient information to come to the right answer with any certainty; will our Holmesian reasoning attempts be anything close to the "correct" answer, or will it be totally off?

The other day, I was listening to music from a movie on my headphones. In the movie, one scene depicts one of the characters getting out of bed. He puts one foot on the ground, then the other. The headphones were broken on one side. Which side?

Use your meta-riddle awareness: this isn't just a random event, but the sort of event that I would make into a riddle.

Here's the answer I had in mind, rot13'd.

View more: Next