The Domain of Your Utility Function

24 Peter_de_Blanc 23 June 2009 04:58AM

Unofficial Followup to: Fake Selfishness, Post Your Utility Function

A perception-determined utility function is one which is determined only by the perceptual signals your mind receives from the world; for instance, pleasure minus pain. A noninstance would be number of living humans. There's an argument in favor of perception-determined utility functions which goes like this: clearly, the state of your mind screens off the state of the outside world from your decisions. Therefore, the argument to your utility function is not a world-state, but a mind-state, and so, when choosing between outcomes, you can only judge between anticipated experiences, and not external consequences. If one says, "I would willingly die to save the lives of others," the other replies, "that is only because you anticipate great satisfaction in the moments before death - enough satisfaction to outweigh the rest of your life put together."

Let's call this dogma perceptually determined utility. PDU can be criticized on both descriptive and prescriptive grounds. On descriptive grounds, we may observe that it is psychologically unrealistic for a human to experience a lifetime's worth of satisfaction in a few moments. (I don't have a good reference for this, but) I suspect that our brains count pain and joy in something like unary, rather than using a place-value system, so it is not possible to count very high.

The argument I've outlined for PDU is prescriptive, however, so I'd like to refute it on such grounds. To see what's wrong with the argument, let's look at some diagrams. Here's a picture of you doing an expected utility calculation - using a perception-determined utility function such as pleasure minus pain.

continue reading »

With whom shall I diavlog?

9 Eliezer_Yudkowsky 03 June 2009 03:20AM

Bloggingheads.tv can't exactly call up, say, the President of France and get him to do a diavlog, but they have some street cred with mid-rank celebrities and academics.  With that in mind, how would you fill in this blank?

"I would really love to see a diavlog between Yudkowsky and ____________."

Open Thread: June 2009

4 Cyan 01 June 2009 06:46PM

I provide our monthly place to discuss Less Wrong topics that have not appeared in recent posts. Work your brain and gain prestige by doing so in E-prime (or not, as you please).

The Frontal Syndrome

18 Annoyance 01 June 2009 04:10PM

Neuroscientists have a difficult time figuring out which parts of the brain are involved in different functions.  Naturally-occurring lesions to the brain are rarely specific to a particular anatomical region, the complications involved with the injury and treatment act as a smokescreen, and finding a patient who's damaged the particular spot you want to learn about is frustrating at best and nigh-impossible at worst.

Fortunately for researchers, inappropriate surgical interventions of the past can shed light on neurological questions.

The strange and horrifying history of psychosurgery is a topic beyond the scope of this site, and certainly beyond this post.  Interested readers can easily find a great wealth of relevant discussion on the Net and in libraries, even (in more extensive collections) works written by the physicians involved in such surgeries during the era in which they were popular.  Even a casually-curious individual can find lots of non-technical analysis and history to read - for such people, I particularly recommend Great and Desperate Cures by Elliot Valenstein.

Of especial relevance is the prefrontal leukotomy, more commonly (if somewhat imprecisely) known as the lobotomy.  There are several features in particular that are of interest to people interested in the nature of effective thought:

continue reading »

A social norm against unjustified opinions?

11 Kaj_Sotala 29 May 2009 11:25AM

A currently existing social norm basically says that everyone has the right to an opinion on anything, no matter how little they happen to know about the subject.

But what if we had a social norm saying that by default, people do not have the right to an opinion on anything? To earn such a right, they ought to have familiarized themselves on the topic. The familiarization wouldn't necessarily have to be anything very deep, but on the topic of e.g. controversial political issues, they'd have to have read at least a few books' worth of material discussing the question (preferrably material from both sides of the political fence). In scientific questions where one needed more advanced knowledge, you ought to at least have studied the field somewhat. Extensive personal experience on a subject would also be a way to become qualified, even if you hadn't studied the issue academically.

The purpose of this would be to enforce epistemic hygiene. Conversations on things such as public policy are frequently overwhelmed by loud declarations of opinion from people who, quite honestly, don't know anything on the subject they have a strong opinion on. If we had in place a social norm demanding an adequate amount of background knowledge on the topic before anyone voiced an opinion they expected to be taken seriously, the signal/noise ratio might be somewhat improved. This kind of a social norm does seem to already be somewhat in place in many scientific communities, but it'd do good to spread it to the general public.

At the same time, there are several caveats. As I am myself a strong advocate on freedom of speech, I find it important to note that this must remain a *social* norm, not a government-advocated one or anything that is in any way codified into law. Also, the standards must not be set *too* high - even amateurs should be able to engage in the conversation, provided that they know at least the basics. Likewise, one must be careful that the principle isn't abused, with "you don't have a right to have an opinion on this" being a generic argument used to dismiss any opposing claims.

Can we create a function that provably predicts the optimization power of intelligences?

-7 whpearson 28 May 2009 11:35AM

Follow up to  Efficient Cross-domain Optimization

When I am skeptical that we will ever understand intelligence, I am skeptical that we will ever be able to reliably map a systems description onto its optimization power. This has implications for how well we will create intelligences and how well intelligences will be at self-improving.

Obviously we can't predict the effectiveness of an arbitrary program, due to rice's theorem and intelligence being a non-trivial property. So the best we can hope for is predicting the effectiveness of a set of programs. Is such a function possible? This is my take on the subject.

continue reading »

This Failing Earth

19 Eliezer_Yudkowsky 24 May 2009 04:09PM

Suppose I told you about a certain country, somewhere in the world, in which some of the cities have degenerated into gang rule.  Some such cities are ruled by a single gang leader, others have degenerated into almost complete lawlessness.  You would probably conclude that the cities I was talking about were located inside what we call a "failed state".

So what does the existence of North Korea say about this Earth?

No, it's not a perfect analogy.  But the thought does sometimes occur to me, to wonder if the camel has two humps.  If there are failed Earths and successful Earths, in the great macroscopic superposition popularly known as "many worlds" - and we're not one of the successful.  I think of this as the "failed Earth" hypothesis.

Of course the camel could also have three or more humps, and it's quite easy to imagine Earths that are failing much worse than this, epic failed Earths ruled by the high-tech heirs of Genghis Khan or the Catholic Church.  Oh yes, it could definitely be worse...

...and the "failed state" analogy is hardly perfect; "failed state" usually refers to failure to integrate into the global economy, but a failed Earth is not failing to integrate into anything larger...

...but the question does sometimes haunt me, as to whether in the alternative Everett branches of Earth, we could identify a distinct cluster of "successful" Earths, and we're not in it.  It may not matter much in the end; the ultimate test of a planet's existence probably comes down to Friendly AI, and Friendly AI may come down to nine people in a basement doing math.  I keep my hopes up, and think of this as a "failing Earth" rather than a "failed Earth".

But it's a thought that comes to mind, now and then.  Reading about the ongoing Market Complexity Collapse and wondering if this Earth failed to solve one of the basic functions of global economics, in the same way that Rome, in its later days, failed to solve the problem of orderly transition of power between Caesars.

continue reading »

Inhibition and the Mind

7 Annoyance 21 May 2009 05:34PM

Babies have a curious set of reflexes: lightly brush their palms, or the soles of their feet, and they will immediately grasp whatever caused the contact. In the case of feet, it’s more of an attempt than a successful grasping; human feet, while far more flexible and manipulative than most creatures’, are no longer the virtual hands possessed by our tree-dwelling ancestors and relatives.

These and a few other basic responses are commonly called the “primitive, or infantile, reflexes“, and are unusual for a variety of reasons. For one thing, they’re not permanent. As babies age, the reflexes disappear.

But they’re not gone.

continue reading »

Share Your Anti-Akrasia Tricks

20 Vladimir_Golovin 15 May 2009 07:06PM

People have been encouraging me to share my anti-akrasia tricks, but it feels inappropriate to dedicate a top-level post solely to unproven techniques that work for some person and may not work for others, so:

Go ahead and share your anti-akrasia tricks!

Let's make it an open thread where we just share what works and what doesn't, without worrying (yet) about having to explain tricks with deep theories, or designing proper experiments to verify them. However, if you happen to have a theory or a proposed experiment in mind, please share.

Bragging is fine, but please share the failures of your techniques as well – they are just as valuable, if not more.

Note to readers – before you read the comments and try the tricks, keep in mind that the techniques below are not yet proven supported or explained by proper experiments, and are not yet backed by theory. They may work for their authors, but are not guaranteed to work for you, so try them at your own risk. It would be even better to read the following posts before rushing to try the tricks:

Essay-Question Poll: Voting

3 Alicorn 15 May 2009 05:04AM

There has been a considerable amount of discussion scattered around Less Wrong about voting, what software features having to do with voting should be added or subtracted, what purpose voting should serve, etc.  It seems as though it would be useful to have conveniently consolidated information on how people are actually voting, so we know what habits that we want to encourage or discourage are actually in use and how prevalently.

1. About what percentage of comments do you vote on at all?  What percentage of top-level posts?

2. Do you use the boo vote or the anti-kibitzer extensions?  Why or why not?

3. What karma threshold do you use to filter what you see, if any?

4. When you vote on a post, or read it and decide not to vote on it, what features of the post are you occurrently conscious of that influence your decision either way?  (Submitter, current post score, length, style, topic, spelling, whatever.)  What about comments?

continue reading »

View more: Prev | Next