Behind every Steve Jobs are thousands of very intelligent and hard-working employees and millions of smart people who have created a larger ecosystem. If one only pays attention to Steve Jobs they will leave out most of the work. They will praise Steve Jobs far too highly and disregard the importance of unglamorous labor.

I think that Steve Jobs is a bad example here, since his specific genius is not in designing things himself but in wringing as much productive work as possible out of intelligent and hard-working employees doing unglamorous labor. (Consider Edison, whose primary invention was the modern R&D lab, vs. Tesla, who was a good inventor but terrible businessman or manager.)

I was one of the people who expressed opinion against the LW content. In general I liked the event, but found those parts off-putting. I'm really surprised that people new to it seemed so oblivious.

Perhaps one reason why people who were familiar with that content were hesitant about showing it to others, was that they were afraid it would reflect poorly on them. If I brought a bunch of 'regular' friends to a 'transhumanist' meetup that I told them I was somewhat involved in, I would be really be afraid of them getting a poor impression of transhumanism.

It's kind of like taking your significant other to meet your parents. You're significant other may not mind your parents quirks (or vice versa), but you notice every one and horrified for them.

Another thing that comes to mind is that they some of the 'serious' talk was controversial even among this crowd. Personally I really don't believe that humans should live forever, for example. Here the people who care the most about it would also care the most about discrepancies. For instance, a very devout Catholic would be the first to get angered by what they feel to be a wrong or mistaken representation of Catholicism at what seems like a very sacred event.

Overall though, thanks for getting feedback and writing this all up! I'm really interested in how it progresses.

I studied engineering, but looking back Computer Science seems like it would have been a lot better.

I'd be very interested in hearing more – you're just the sort of person who I was hoping would comment (graduate from a top program several years out who switched fields and so has had exposure to both). In what respects would majoring in computer science have been better for you?

I want to note that engineering degrees can be more work than computer science degrees. This definitely true at Harvey Mudd College.

I studied engineering, but looking back Computer Science seems like it would have been a lot better. I've headed there since, but I definitely feel like I'm playing catch-up in comparison.

If all 'moral worth' meant was the consequences of what happened, I just wouldn't deem 'moral worth' to be that relevant towards judging. It would seem to me like we're just making 'moral worth' into something kind of irrelevant except from a completely pragmatic point.

Not sure if saying 'making the best decision you could is al you can do' is that much of a shortcut. I mean, I would imagine that a lot of smart people would realize that 'making the best decision you can' is still really, really difficult. If you act as your only judge (not just all of you, but only you at any given moment), then you may have less motivation; however, it would seem strange to me if 'fear of being judged' is the one thing that keeps us moral, even if it happens to become apparent that judging is technically impossible.

One would be ethical if their actions end up with positive outcomes, disregarding the intentions of those actions. For instance, a terrorist who accidentally foils an otherwise catastrophic terrorist plan would have done a very ‘morally good’ action.

This seems intuitively strange to many, it definitely is to me. Instead, ‘expected value’ seems to be a better way of both making decisions and judging the decisions made by others.

If the actual outcome of your action was positive, it was a good action. Buying the winning lottery ticket, as per your example, was a good action. Buying a losing lottery ticket was a bad action. Since we care about just the consequences of the action, the goodness of an action can only be evaluated after the consequences have been observed - at some point after the action was taken (I think this is enforced by the direction of causality, but maybe not).

So we don't know if an action is good or not until it's in the past. But we can only choose future actions! What's a consequentialist to do? (Equivalently, since we don't know whether a lottery ticket is a winner or a loser until the draw, how can we choose to buy the winning ticket and choose not to buy the losing ticket?) Well, we make the best choice under uncertainty that we can, which is to use expected values. The probability-literate person is making the best choice under uncertainty they can; the lottery player is not.

The next step is to say that we want as many good things to happen as possible, so "expected value calculations" is a correct way of making decisions (that can sometimes produce bad actions, but less often than others) and "wishful thinking" is an incorrect way of making decisions.

So the probability-literate used a correct decision procedure to come to a bad action, and the lottery player used an incorrect decision procedure to come to a good action.

The last step is to say that judging past actions changes nothing about the consequences of that action, but judging decision procedures does change something about future consequences (via changing which actions get taken). Here is the value in judging a person's decision procedures. The terrorist used a very morally wrong decision procedure to come up with a very morally good action: the act is good and the decision procedure is bad, and if we judge the terrorist by their decision procedure we influence future actions.

--

I think it's very important for consequentialists to always remember that an action's moral worth is evaluated on its consequences, and not on the decision theory that produced it. This means that despite your best efforts, you will absolutely make the best decision possible and still commit bad acts.

If you let it collapse - if you take the shortcut and say "making the best decision you could is all you can do", then every decision you make is good, except for inattentiveness or laziness, and you lose the chance to find out that expected value calculations or Bayes' theorem needs to go out the window.

Yes. "Good" can mean desirable outcomes, or responsible decision making. The first obviously matches consequentialism. It appears not to be obvious to Lesswrongians that the second matches deontology. When we judge whether someone behaved culpably or not, we want to know whether they applied the rules and heuristic appropriate to their reference class (doctor, CEO, ships captain...). The consequences of their decision may have landed them in a tribunal, but we don't hold people to blame for applying the rules and getting the wrong results.

Optimizing Future Decisions: Actual vs. Expected Value

Not sure what you mean here. Future is never actual, only expected (or, more often, unexpected).

Right, it seems kind of strange to declare that you're considering only states of the world in your decisions, but then to treat judgments of right and wrong as an deontological layer on top of that where you consider whether the consequentialist rule was followed correctly. But that does seem to be a mainstream version of consequentialism. As far as I can tell, it mostly leads to convoluted, confused-sounding arguments like the above and the linked talk by Neiladri Sinhababu, but maybe I'm missing something important.

You come to what is more or less the right consequentialist answer in the end, but it seems to me that your path is needlessly convoluted. Why are we judging past actions? Generally, the reason is to give us insight into and perhaps influence future decisions. So we don't judge the lottery purchase to have been good, because it wouldn't be a good idea to imitate it (we have no way to successfully imitate "buy a winning lottery ticket" behavior, and imitating "buy a lottery ticket" behavior has poor expected utility, and similarly for many broader or narrower classes of similar actions), and so we want to discourage people from imitating it, not encourage them. If we're being good consequentialists, what other means could it possibly be appropriate to use in deciding how to judge other than basing it on the consequences of judging in that way?