The error bars on the result make this result meaningless. You should, at minimum, also consider how hot/successful she is relative to you (can she do better?) and whether she and you have shared goals (what are her dreams/expectations? are they compatible with yours, and are you both making progress?). It would be extraordinarily improbable for her to forget you overnight, but if she broke up with you she has likely wanted to do better for some time. The usual post-breakup advice is to focus on your own goals for awhile.

That the Death Eaters were bad guys was not in question. The question was whether they were the bad guys; whether there was one villain in the story, or two... - http://hpmor.com/chapter/47

For me it's low confidence and speculative, but t could be foreshadowing. Dumbledore could be bad. Dumbledore and QQ could even be on the same side. The "one villain in the story, or two" at the time and in context was Harry thinking there were 2 sides and both were bad. But as a meta-hint / foreshadowing it could be saying that there was 1 side with 2 villains.

One big piece of evidence for was that Voldemort and Dumbledore fought a war for several years and neither killed the other. Harry initially thought this was evidence for Voldemort being dumb, but by 94 Harry has updated to the enemy being smart. So Voldemort could have wiped out the Order but didn't.

Dumbledore is soft, but he could have killed all the death eaters. Just gone to their houses and killed them, none of them, nor all of them together (excluding Voldie) are going to be on par with Grindelwald + elderwand + blood sacrifice. Dumbledore could have wiped out the death eaters but didn't.

You can explain that with two factors each handicapping one of them, Dumbledore is soft and Voldemort has a cunning plan. But you can also explain it with coordination. Coordination certainly fits some of the facts better.

Weighing against this of course, is that it really doesn't seem like Dumbledore's style.

How magic really works in HPMOR, my guess: Spells are like functions in a computer program -- ways to manipulate data (the world) without understanding the underlying implementation (how the spell actually makes changes happen). The next level up from the magical world is an enormous computer such as the one described in Permutation City, except with quantum hardware to continuously and seamlessly recompute the present and the preceding six hours. The machine's creator and friends copied themselves into this Universe, and gave themselves magic, implemented through spells/functions that change the physical world when triggered. One program provides a terminal or other way to create new spells, and appears to wizards who have become sufficiently experienced with magic to meet the terminal spell's requirements. The Interdict of Merlin was created by one of these learned wizards, who decided the terminal was too easy to get access to, after a newly ascended wizard made a programming error and erased Atlantis. In this Universe, the solution to the hard problem of consciousness is that NPCs are philosophical zombies and PCs are game-players from a universe one or more levels below the magical universe. That is, Harry is fully conscious but is actually an alien sitting in a virtual-reality console and suppressing part of their mind so as to fully experience only Harry's in-universe experience. In the alien's universe, there is a satisfying and provable answer to the hard problem of consciousness.

EDIT: I'm seeing a lot of negative votes. I will argue my case if you tell me what's wrong.

In all seriousness, how do you know that you're not simply brainwashed into believing cookies are making you happy?

For example, during my religious years, attending a 5-hour prayer meeting made me feel happier -- even ones where not much English was spoken. Much of this was a learned association between attendance and the feeling of "doing the right thing," in retrospect. Once I no longer thought of it as "the right thing," the happiness I derived from it waned.

Write an expert system for making philosophical statements about itself.

Make it scientific articles instead. Thus MIRI will get more publications. :D

You can also make different expect systems compete with each other by trying to get most publications and citations.

Stay in London, and study in the evenings if you want. Benjamin Franklin said "three removes is as bad as a fire", meaning there's a high cost to rebuilding your social network. I'd guess it would take you about 18 months to fully build new friendships. I moved to a non-ideal city for work (twice!) and it set my career back by a couple of years. The cost of living in Glasgow is lower because people are happier living in London.

If you want to fully maximize utility, you're making a false choice by just looking at the two jobs. Get back in grad school and work as hard as possible until you're in the top half of the class at a top school (or otherwise meet the "great hacker" criteria that Paul Graham describes on his website). Then, start a fast-growing startup with one or two other outstanding hackers. When it stops growing after two to ten years, sell out. Then, you should be in a really high-utility space where you can do massive good and/or enjoy novel luxuries.

I think we're weaker on acting according to our predictions than on the predictions themselves.

Your arguments would be much more convincing if you showed results from actual code. In engineering fields, including control theory and computer science, papers that contain mathematical arguments but no test data are much more likely to have errors than papers that include test data, and most highly-cited papers include test data. In less polite language, you appear to be doing philosophy instead of science (science requires experimental data, while philosophy does not).

What would you want this code to do? What code (short of a full-functioning AGI) would be at all useful here?

At minimum, I would like to see you try to test your theories by examining the actual performance of real-world computer systems, such as search engines, as they perform tasks analogous to making high-level ethical decisions.

Can you expand on this, possibly with example tasks, because I'm not sure what you are requesting here.

Your examples about predicting the future are only useful if you can identify trends by also considering past predictions that turned out to be inaccurate. The most exciting predictions about the future tend to be wrong, and the biggest advances tend to be unexpected.

This is a trenchant critique, but it ultimately isn't that strong: having trouble predicting should be a reason to if anything be more worried rather than less.

It appears that as technology improves, human lives become better and safer. I expect this trend to continue. I am not convinced that AI is fundamentally different -- in current societies, individuals with greatly differing intellectual capabilities and conflicting goals already coexist, and liberal democracy seems to work well for maintaining order and allowing incremental progress. If current trends continue, I would expect competing AIs to become unimaginably wealthy, while non-enhanced humans enjoy increasing welfare benefits. The failure mode I am most concerned about is a unified government turning evil (in other words, evolution stopping because the entire population becomes one unchanging organism), but it appears that this risk is minimized by existing antitrust laws (which provide a political barrier to a unified government) and by the high likelihood of space colonization occurring before superhuman AI appears (which provides a spatial barrier to a unified government).

This is missing the primary concern of people at MIRI and elsewhere. The concern isn't anything like gradually more and more competing AI coming online that are slightly smarter than baseline humans. The concern is that the first true AGI will self-modify itself to become far smarter and more capable of controlling the environment around it than anything else. In that scenario, issues like anti-trust or economics aren't relevant. It is true that on balance human lives have become better and safer, but that isn't by itself a strong reason to think that trend will continue, especially when considering hypothetical threats such the AGI threat whose actions are fundamentally discontinuous to prior human trends for standards of living.

Ringmione seems to be the most popular hypothesis at the moment. It strikes me as extremely careless a plan for Harry to attempt; recall Quirrell's comments after the battle with the transfigured armor, and the first battle where passing out ended the transfiguration on the marshmallow he was practicing with.

However, I took "Harry's parents come to Hogwarts" as a completely insane move that Dumbledore/Macgonnagle would be highly unlikely to pull, and yet the elder wizards thought rather differently from me and did it anyway. I still think using the ring involves Harry assigning way too little cleverness to Dumbledore, but in light of the ordeal with his parents I'll give it a little more probability weight.

(Note also that Harry did something under the covers when Flitwick showed up.)

Your arguments would be much more convincing if you showed results from actual code. In engineering fields, including control theory and computer science, papers that contain mathematical arguments but no test data are much more likely to have errors than papers that include test data, and most highly-cited papers include test data. In less polite language, you appear to be doing philosophy instead of science (science requires experimental data, while philosophy does not).

I imagine you have not actually written code because it seems too hard to do anything useful -- after 50 years of Moore's law, computers will execute roughly 30 million times as many operations per unit time as present-day computers. That is, a 2063 computer will do in 1 second what my 2013 computer can do in 1 year. You can close some of this gap by using time on a high-powered computing cluster and running for longer times. At minimum, I would like to see you try to test your theories by examining the actual performance of real-world computer systems, such as search engines, as they perform tasks analogous to making high-level ethical decisions.

Your examples about predicting the future are only useful if you can identify trends by also considering past predictions that turned out to be inaccurate. The most exciting predictions about the future tend to be wrong, and the biggest advances tend to be unexpected.

I agree that this seems like an important area of research, though I can't confidently speculate about when human-level general AI will appear. As far as background reading, I enjoyed Marshall Brain's "Robotic Nation", an easy-to-read story intended to popularize the societal changes that expert systems will cause. I share his vision of a world where the increased productivity is used to deliver a very high minimum standard of living to everyone.

It appears that as technology improves, human lives become better and safer. I expect this trend to continue. I am not convinced that AI is fundamentally different -- in current societies, individuals with greatly differing intellectual capabilities and conflicting goals already coexist, and liberal democracy seems to work well for maintaining order and allowing incremental progress. If current trends continue, I would expect competing AIs to become unimaginably wealthy, while non-enhanced humans enjoy increasing welfare benefits. The failure mode I am most concerned about is a unified government turning evil (in other words, evolution stopping because the entire population becomes one unchanging organism), but it appears that this risk is minimized by existing antitrust laws (which provide a political barrier to a unified government) and by the high likelihood of space colonization occurring before superhuman AI appears (which provides a spatial barrier to a unified government).