I suppose you could say that it's equivalent to "total utilitarianism that only takes into account the utility of already extant people, and only takes into account their current utility function [at the time the decision is made] and not their future utility function".
(Under mere "total utilitarianism that only takes into account the utility of already extant people", the government could wirehead its constituency.)
Yes, this is explicitly inconsistent over time. I actually would argue that the utility function for any group of peopl...
My intended solution was that, if you check the utility of your constituents from creating more people, you're explicitly not taking the utility of the new people into account. I'll add a few sentences at the end of the article to try to clarify this.
Another thing I can say is that, if you assume that everyone's utility is zero at the decision point, it's not clear why you would see a utility gain from adding more people.
...Followup: Holy crap! I know exactly one person who wants Hermione to be defeated by Draco when Lucius is watching. Could H&C be Dumbledore?
My theory is that Lucius trumped up these charges against Hermione entirely independent of the midnight duel. He was furious that Hermione defeated Draco in combat, and this is his retaliation.
I doubt that Hermione attended the duel; or, if she did attend it, I doubt that anything bad happened.
My theory does not explain why Draco isn't at breakfast. So maybe my theory is wrong.
I am confused about why H&C wanted Hermione to be defeated by Draco during the big game when Lucius was watching. If you believe H&C is Quirrell (and I do): did Quirrell go to all that trouble just to impress Lucius with how his son was doing? That seems like an awful risk for not much reward.
The new Update Notifications features (http://hpmor.com/notify/) is pretty awesome but I have a feature request. Could we get some sort of privacy policy for that feature?
Like, maybe just a sentence at the bottom saying "we promise to only use your email address to send you HPMOR notifications, and we promise never to share your email address with a third party"?
It's not that I don't trust you guys (and in fact I have already signed up) but I like to check on these things.
I think the issue was that Harry was constantly, perpetually, invariably reacting to everything with shock and outrage. It got... tiresome.
But I went back much later and read it again, and there wasn't nearly as much outrage as I remembered.
Good story!
Ouch! I -- I actually really enjoyed Ender's Game. But I have to admit there's a lot of truth in that review.
Now I feel vaguely guilty...
I found this series much harder to enjoy than Eliezer's other works -- for example the Super Happy People story, the Brennan stories, or the Sword of Good story.
I think the issue was that Harry was constantly, perpetually, invariably reacting to everything with shock and outrage. It got... tiresome.
At first, before I knew who the author was, I put this down to simple bad writing. Comments in Chapter 6 suggest that maybe Harry has some severe psychological issues, and that he's deliberately being written as obnoxious and hyperactive in order to meet plot...
I think the issue was that Harry was constantly, perpetually, invariably reacting to everything with shock and outrage. It got... tiresome.
I suspect that a main inspiration for writing the story was Eliezer's constant shock and outrage over the fact that Rowling's characters show absolutely no interest in the inner workings of their weird Universe. I vividly remember how outrageous this was for me when I read the originals. Actually, I have only read the first two books, so when I read Eliezer's time-turner scene, I first believed that he invented the a...
I think some of Harry's annoyingness is due to the fact that he's modeled after young Eliezer. He's a mix of wish-fulfillment for young Eliezer and an opportunity for older Eliezer to criticize his younger self. This is really apparent with the chapters involving the Sorting Hat.
This isn't visible, right? I will feel very bad if it turns out I am spamming the community with half-finished drafts of the same article.
Hi! I'd like to suggest two other methods of counting readers: (1) count the number of usernames which have accessed the site in the past seven days (2) put a web counter (Google Analytics?) on the main page for a week (embed it in your post?) It might be interesting to compare the numbers.
The good news is that this pruning heuristic will probably be part of any AI we build. (In fact, early forms of this AI will have to use a much stronger version of this heuristic if we want to keep them focused on the task at hand.)
So there is no danger of AIs having existential Boltzmann crises. (Although, ironically, they actually are brains-in-a-jar, for certain definitions of that term...)
The anthropic principle lets you compute the posterior probability of some value V of the world, given an observable W. The observable can be the number of humans who have lived so far, and the value V can be the number of humans who will ever live. The probability of a V where 100W < V is smaller than the probability of a V only a few times larger than W.
This argument could have been made by any intelligent being, at any point in history, and up to 1500AD or so we have strong evidence that it was wrong every time. If this is the main use of the anth...
Personally it bothers me that the explanation asks a question which is numerically unanswerable, and then asserts that rationalists would answer it in a given way. Simple explanations are good, but not when they contain statements which are factually incorrect.
But, looking at the karma scores it appears that you are correct that this is better for many people. ^_^;
A brain tumor always causes a headache, but exceedingly few people have a brain tumor. In contrast, a headache is rarely a symptom for cold, but most people manage to catch a cold every single year. Given no other information, do you think it more likely that the headache is caused by a tumor, or by a cold?
Given no other information, we don't know which is more likely. We need numbers for "rarely", "most", and "exceedingly few". For example, if 10% of humans currently have a cold, and 1% of humans with a cold have a heada...
You're missing the point. This post is suitable for an audience whose eyes would glaze over if you threw in numbers, which is wonderful (I read the "Intuitive Explanation of Bayes' Theorem" and was ranting for days about how there was not one intuitive thing about it! it was all numbers! and graphs!). Adding numbers would make it more strictly accurate but would not improve anyone's understanding. Anyone who would understand better if numbers were provided has their needs adequately served by the "Intuitive" explanation.
I would like to know more about your statement "50,000 users would surely count as a critical mass". How many users does Craigslist have in total?
I especially think it's unlikely that Craigslist would be motivated by the opinions of 50,000 Facebook users, especially if you had not actually conducted a poll but merely collected the answers of those that agree with you.
You should contact Craigslist and ask them what criteria would actually convince them that Craigslist users want for-charity ads.
each person could effectively cause $20,000 to be generated out of nowhere
As a rationalist, when you see a strange number like this, you have to ask yourself: Did I really just discover a way to make lots of money very efficiently? Or could it be that there was a mistake in my arithmetic somewhere?
That one billion dollars is not being generated out of nowhere.
It is being generated as payment for ad clicks.
Let's check your assumptions:
How much money will the average user generate from banner ad clicks in five years?
How many users does Craigslist have?...
So, people who have a strong component of "just be happy" in their utility function might choose to wirehead, and people in which other components are dominant might choose not to.
That sounds reasonable.
Well, I said most existing humans are opposed to wireheading, not all. ^_^;
Addiction might occur because: (a) some people suffer from the bug described above; (b) some people's utility function is naturally "I want to be happy", as in, "I want to feel the endorphin rush associated with happiness, and I do not care what causes it", so wireheading does look good to their current utility function; or (c) some people underestimate an addictive drug's ability to alter their thinking.
It's often difficult to think about humans' utility functions, because we're used to taking them as an input. Instead, I like to imagine that I'm designing an AI, and think about what its utility function should look like. For simplicity, let's assume I'm building a paperclip-maximizing AI: I'm going to build the AI's utility function in a way that lets it efficiently maximize paperclips.
This AI is self-modifying, meaning it can rewrite its own utility function. So, for example, it might rewrite its utility function to include a term for keeping its pro...
Humans evaluate decisions using their current utility function, not their future utility function as a potential consequence of that decision. Using my current utility function, wireheading means I will never accomplish anything again ever, and thus I view it as having very negative utility.
It's often difficult to think about humans' utility functions, because we're used to taking them as an input. Instead, I like to imagine that I'm designing an AI, and think about what its utility function should look like. For simplicity, let's assume I'm building a paperclip-maximizing AI: I'm going to build the AI's utility function in a way that lets it efficiently maximize paperclips.
This AI is self-modifying, meaning it can rewrite its own utility function. So, for example, it might rewrite its utility function to include a term for keeping its pro...
I think I am happy with how these rules interact with the Anthropic Trilemma problem. But as a simpler test case, consider the following:
An AI walks into a movie theater. "In exchange for 10 utilons worth of cash", says the owner, "I will show you a movie worth 100 utilons. But we have a special offer: for only 1000 utilons worth of cash, I will clone you ten thousand times, and every copy of you will see that same movie. At the end of the show, since every copy will have had the same experience, I'll merge all the copies of you back into one."
Note that, although AIs can be cloned, cash cannot be. ^_^;
I claim that a "sane" AI is one that declines the special offer.
(I'm not sure what the rule is here for replying to oneself. Apologies if this is considered rude; I'm trying to avoid putting TLDR text in one comment.)
Here is a set of utility-rules that I think would cause an AI to behave properly. (Would I call this "Identical Copy Decision Theory"?)
Suppose that an entity E clones itself, becoming E1 and E2. (We're being agnostic here about which of E1 and E2 is the "original". If the clone operation is perfect, the distinction is meaningless.) Before performing the clone, E calculates its ex
It's difficult to answer the question of what our utility function is, but easier to answer the question of what it should be.
Suppose we have an AI which can duplicate itself at a small cost. Suppose the AI is about to witness an event which will probably make it happy. (Perhaps the AI was working to get a law passed, and the vote is due soon. Perhaps the AI is maximizing paperclips, and a new factory has opened. Perhaps the AI's favorite author has just written a new book.)
Does it make sense that the AI would duplicate itself in order to witness this event in greater multiplicity? If not, we need to find a set of utility rules that cause the AI to behave properly.
In modern times, some people have started to see nature more as an enemy to be conquered than as a god to be worshiped.
I've seen people argue the opposite. In ancient times, nature meant wolves and snow and parasites and drought, and you had to kill it before it killed you. Only recently have we developed the idea that nature is something to be conserved. (Because until recently, we weren't powerful enough that it mattered.)
Note that when the trillion were told they won, they were actually being lied to - they had won a trillionth part of the prize, one way or another.
Suppose that, instead of winning the lottery, you want your friend to win the lottery. (Or you want your random number generator to crack someone's encryption key, or you want a meteor to fall on your hated enemy, etc.) Then each of the trillion people would experience the full satisfaction from whatever random result happened.
I deny that increasing the number of physical copies increases the weight of an experience. If I create N copies of myself, there is still just one of me, plus N other agents running my decision-making algorithms. If I then merge all N copies back into myself, the resulting composite contains the utility of each copy weighted by 1/(N+1).
My feeling about the Boltzmann Brain is: I cheerfully admit that there is some chance that my experience has been produced by a random experience generator. However, in those cases, nothing I do matters anyway. Thus I d...
Also: it seems like a really poor plan, in the long term, for the fate of the entire plane to rest on the sanity of one dude. If Hirou kept the sword, he could maybe try to work with the wizards -- ask them to spend one day per week healing people, make sure the crops do okay, etc. Things maybe wouldn't be perfect, but at least he wouldn't be running the risk of everybody-dies.
I think my concern about "power corrupts" is this: humans have a strong drive to improve things. We need projects, we need challenges. When this guy gets unlimited power, he's going to take two or three passes over everything and make sure everybody's happy, and then I'm worried he's going to get very, very bored. With an infinite lifespan and unlimited power, it's sort of inevitable.
What do you do, when you're omnipotent and undying, and you realize you're going mad with boredom?
Does "unlimited power" include the power to make yourself not bored?
I think the key difference is that delta utilitarianism handles it better when the group's utility function changes. For example, if I create a new person and add it to the group, that changes the group's utility function. Under delta utilitarianism, I explicitly don't count the preferences of the new person when making that decision. Under total utilitarianism, [most people would say that] I do count the preferences of that new person.