Basically, the busy beaver function tells us the maximum number of steps that a Turing machine with a given number of states and symbols can run for. If we know the busy beaver of, for example, 5 states and 5 symbols, then we can tell you if any 5 state 5 symbol Turing machine will eventually halt.

However, you can see why it's impossible to in general find the busy beaver function- you'd have to know which Turing machines of a given size halted, which is in general impossible.

Are you aware of the busy beaver function? Read this.

Basically, it's impossible to write down numbers large enough for that to work.

It's a kind of utilitarianism. I'm including act utilitarianism and desire utilitarianism and preference utilitarianism and whatever in utilitarianism.

What do you mean by "utilitarianism"? The word has two different common meanings around here: any type of consequentialism, and the specific type of consequentialism that uses "total happiness" as a utility function. This sentence appears to be designed to confuse the two meanings.

Yeah, my mistake. I'd never run across any other versions of consequentialism apart from utilitarianism (except for Clippy, of course). I suppose caring only for yourself might count? But do you seriously think that the majority of those consequentialists aren't utilitarian?

I edited my comment to include a tiny bit more evidence.

This seems like it has makings of an interesting poll question.

I agree. Let's do that. You're consequentialist, right?

I'd phrase my opinion as "I have terminal value for people not suffering, including people who have done something wrong. I acknowledge that sometimes causing suffering might have instrumental value, such as imprisonment for crimes."

How do you phrase yours? If I were to guess, it would be "I have a terminal value which says that people who have caused suffering should suffer themselves."

I'll make a Discussion post about this after I get your refinement of the question?

Here's an old Eliezer quote on this:

4.5.2: Doesn't that screw up the whole concept of moral responsibility?

Honestly? Well, yeah. Moral responsibility doesn't exist as a physical object. Moral responsibility - the idea that choosing evil causes you to deserve pain - is fundamentally a human idea that we've all adopted for convenience's sake. (23).

The truth is, there is absolutely nothing you can do that will make you deserve pain. Saddam Hussein doesn't deserve so much as a stubbed toe. Pain is never a good thing, no matter who it happens to, even Adolf Hitler. Pain is bad; if it's ultimately meaningful, it's almost certainly as a negative goal. Nothing any human being can do will flip that sign from negative to positive.

So why do we throw people in jail? To discourage crime. Choosing evil doesn't make a person deserve anything wrong, but it makes ver targetable, so that if something bad has to happen to someone, it may as well happen to ver. Adolf Hitler, for example, is so targetable that we could shoot him on the off-chance that it would save someone a stubbed toe. There's never a point where we can morally take pleasure in someone else's pain. But human society doesn't require hatred to function - just law.

Besides which, my mind feels a lot cleaner now that I've totally renounced all hatred.

It's pretty hard to argue about this if our moral intuitions disagree. But at least, you should know that most people on LW disagree with you on this intuition.

EDIT: As ArisKatsaris points out, I don't actually have any source for the "most people on LW disagree with you" bit. I've always thought that not wanting harm to come to anyone as an instrumental value was a pretty obvious, standard part of utilitarianism, and 62% of LWers are consequentialist, according to the 2012 survey. The post "Policy Debates Should Not Appear One Sided" is fairly highly regarded, and it esposes a related view, that people don't deserve harm for their stupidity.

Also, what those people would prefer isn't nessecarily what our moral system should prefer- humans are petty and short-sighted.

Harry's failing pretty badly to update sufficiently on available evidence. He already knows that there are a lot of aspects of magic that seemed nonsensical to him: McGonagall turning into a cat, the way broomsticks work, etc. Harry's dominant hypothesis about this is that magic was intelligently designed (by the Atlanteans?) and so he should expect magic to work the way neurotypical humans expect it to work, not the way he expects it to work.

I disagree. It seems to me that individual spells and magical items work in the way neurotypical humans expect them to work. However, I don't think that we have any evidence that the process of creating new magic or making magical discoveries works in an intuitive way.

Consider by analogy the Internet. It's not surprising that there exist sites such as Facebook which are really well designed and easy to use for humans, rendering in pretty colors instead of being plain HTML. However, these websites were created painstakingly by experts dealing with irritating low level stuff. It would be surprising that the same website had a surpassingly brilliant data storage system as well as an ingenius algorithm for something else.

Yeah, I'm pretty sure I (and most LWers) don't agree with you on that one, at least in the way you phrased it.