When I was a teenager, I imagined that if you had just a tiny infinitesimally small piece of a curve - there would only be one moral way to extend it. Obviously, an extension would have to be connected to it, but also, you would want it to connect without any kinks. And just having straight-lines connected to it wouldn't be right, it would have to be curved in the same sort of way - and so on, to higher-and-higher orders. Later I realized that this is essentially what a Taylor series is.

I also had this idea when I was learning category theory that objects were points, morphisms were lines, composition was a triangle, and associativity was a tetrahedron. It's not especially sophisticated, but it turns out this idea is useful for n-categories.

Recently, I have been learning about neural networks. I was working on implementing a fairly basic one, and I had a few ideas for improving neural networks: making them more modular - so neurons in the next layer are only connected to a certain subset of neurons in the previous layer. I read about V1, and together, these led to the idea that you arrange things so they take into account the topology of the inputs - so for image processing, having neurons connected to small, overlapping, circles of inputs. Then I realized you would want multiple neurons with the same inputs that were detecting different features, and that you could reuse training data for neurons with different inputs detecting the same feature - saving computation cycles. So for the whole network, you would build up from local to global features as you applied more layers - which suggested that sheaf theory may be useful for studying these. I was planning to work out details, and try implementing as much of this as I could (and still intend to as an exercise), but the next day I found that this was essentially the idea behind convolutional neural networks. I'm rather pleased with myself since CNNs are apparently state-of-the-art for many image recognition tasks (some fun examples). The sheaf theory stuff seems to be original to me though, and I hope to see if applying Gougen's sheaf semantics would be useful/interesting.

I really wish I was better at actually implementing/working out the details of my ideas. That part is really hard.

I came up with the idea of a Basic Income by myself, by chaining together some ideas:

• Capitalism is the most efficient economic system for fulfilling the needs of people, provided they have money.

• The problem is that if lots of people have no money, and no way to get money (or no way to get it without terrible costs to themselves), then the system does not fulfill their needs.

• In the future, automation will both increase economic capacity, while also increase the barrier to having a 'valuable skill' allowing you to get money. Society will have improved capacity to fulfill the needs of people with money, yet the barrier to having useful skills and being able to get money will increase. This leads to a scenario where the society could easily produce the items needed by everyone, yet does not because many of those people have no money to pay for them.

• If X% of the benefits accrued from ownership of the capital were taken and redistributed evenly among all humans, then the problem is averted. Average people still have some source of money with which they can purchase the fulfillment of their needs, which are pretty easy to supply in this advanced future society.

• X=100%, as in a strict socialism, is not correct, as then we get the economic failures we saw in the socialist experiments of the past century.

• X = 0%, as in a strict libertarianism, is not correct, as then everyone whose skills are automated starve.

• At X = some reasonable number, capitalism still functions correctly (that is, it works today with our current tax rate levels, and hopefully in our economically progressed future society, it provides sufficient money to everyone to supply basic needs).

Eventually I found out that my idea was pretty much a Basic Income system.

This is not a direct answer: Every time I come up with an idea in a field I am not very deeply involved in sooner or later I will realise that the phenomenon is either trivial, a misperception or very well studied. Most recently this happened with pecuniary externalities.

For as long as I can remember, I had the idea of a computer upgrading its own intelligence and getting powerful enough to make the world a utopia.

Oh, another thing: I remember thinking that it didn't make sense to favour either the many worlds interpretation or the copenhagen interpretation, because no empirical fact we could collect could point towards one or the other, being as we are stuck in just one universe and unable to observe any others. Whichever one was true, it couldn't possibly impact on one's life in any way, so the question should be discarded as meaningless, even to the extent that it didn't really make sense to talk about which one is true.

This seems like a basically positivist or postpositivist take on the topic, with shades of Occam's Razor. I was perhaps around twelve. (For the record, I haven't read the quantum mechanics sequence and this remains my default position to this day.)

Derivatives. I imagined tangent lines traveling along a function curve and thinking 'I wonder what it looks like when we measure that?' And so I would try to visualize the changing slopes of the tangent lines at the same time. I also remembering wondering how to reverse it. Obviously didn't get farther than that, but I remember being very surprised when I took calculus and realizing that the mind game I had been playing was hugely important and widespread, and could in fact be calculated.

At school my explanation for the existence of bullies was that it was (what I would later discover was called) a Nash equilibrium.

I had drawn up some rather detailed ideas for an atomic powered future: The idea was to solve two major problems. The first was the inherent risk of an over pressure causing such a power plant to explode. The second problem to solve was the looming water shortage facing many nations.

The idea was a power plant that used internal sterling technology so as to operate at atmospheric pressures. Reinforcing this idea was basically a design for the reactor to "entomb" itself if it reached temperatures high enough to melt its shell. The top of the sterling engine would have a salt water reservoir that would be boiled off. The water then would be collected and directed in a piping system to a reservoir. The plant would then both produce electricity AND fresh water.

Of course then while researching thorium power technology in school I discovered that the South Korean SMART micro reactor does in fact desalinate water. On one level I was depressed that my idea was not "original" however, overall I'm exited that I came up with an idea that apparently had enough merit for people actually go through and make a finished design based upon it. The fact that my idea had merit at all gives me hope for my future as an engineer.

I'm another independent discoverer of something like utilitarianism, I think when I was in elementary school. My earliest written record of it is from when I was 15, when I wrote: "Long ago (when I was 8?), I said that the purpose of life was to enjoy yourself & to help others enjoy themselves - now & in the future."

In high school I did a fair amount of thinking (with relatively little direct outside influence) about Goodhart's law, social dilemmas, and indirect utilitarianism. My journal from then include versions of ideas like the "one thought too many" argument, decision procedures vs. criterion for good, tradeoffs between following an imperfect system and creating exceptions to do better in a particular case, and expected value reasoning about small probabilities of large effects (e.g. voting).

On religion, I thought of the problem of evil (perhaps with outside influence on that one) and the Euthyphro argument against divine command theory.

16-year-old me also came up with various ideas related to rationality / heuristics & biases, like sunk costs ("Once you’re in a place, it doesn’t matter how you got there (except in mind - BIG exception)"), selection effects ("Reason for coincidence, etc. in stories - interesting stories get told, again & again"), and the importance of epistemic rationality ("Greatest human power - to change ones mind").

In 6th or 7th grade I told my class that it was obvious that purchasing expensive sneakers is mostly just a way to show how cool you are or that you can afford something that not everyone else could. Many years latter I would read about signalling http://en.wikipedia.org/wiki/Signalling_(economics)

The following are not ideas as much as questions I had while growing up, and I was surprised/relieved/happy to find out that other people much smarter than me, spent a lot of time thinking about and is "a thing". For example I really wanted to know if there was a satisfactory way to figure out if Christianity was the one true religion and it bothered me very much that I could not answer that question. Also, I was concerned that the future might not be what I want it to be, and I am not sure that I know what I even want. It turns out that this isn't a unique problem and there are many people thinking about it. Also, what the heck is consciousness? Is there one correct moral theory? Well, someone is working on it.

I've found that ideas that affect me most fall into two major categories: either they will be ideas that hit me completely unprepared or they are ideas that I knew all along but had not formalized. Many-Worlds and and timelessness were the former for me. Utilitarianism and luminosity were the latter.

Once a Christian friend asked me why I cared so much about what he believed. Without thinking, I came up with, "What you think determines what you choose. If your idea of the world is inaccurate, your choices will fail."

This was years before I found LW and learned about the connection between epistemic and instrumental rationality.

P.S. My friend deconverted himself some years afterwards.

After learning the very basics of natural selection, I started thinking about goal systems and reward circuits and ethics. I thought that all of our adaptations were intended to allow us to meet our survival needs so we could pass on our genes. But what should people do once survival needs are met? What's the next right and proper goal to pursue? That line of reasoning's related Googling led me to Eliezer's Levels of Intelligence paper, which in turn led me to Less Wrong.

Reading through the sequences, I found so many of the questions that I'd thought about in vague philosophical terms explained and analyzed rigorously, like personal identity vs continuity of subjective experience under things like teleportation. Part of the reason LW appealed to me so much back then is, I suspect, that I had already thought about so many of the same questions but just wasn't able to frame them correctly.

This made me curious enough to skim through my childhood writing. Convergent and divergent infinite series, quicksort, public choice theory, pulling the rope sideways, normative vs positive statements, curiosity stoppers, the overton window.

My Moloch moment is what led me to seek out Overcomingbias.

Tangent thread: What sophisticated idea are you holding on to that you are sure has been formalized somewhere but haven't been able to find?

I'll go first: When called to explain and defend my ethics I explained I believe in "Karma, NO not the that BS mysticism Karma, but plain old actions have consequences in our very connected world kind of Karma." If you treat people in a manner of honesty and integrity in all things, you will create a community of cooperation. The world is strongly interconnected and strongly adaptable so the benefits will continue outside your normal community, or if you frequently change communities. The lynchpin assumption of these beliefs is that if I create One Unit of Happiness for others, it will self propagate, grow and reflect, returning me more that One Unit of Happiness over the course of my lifetime. The same applies for One Unit of Misery.

I've only briefly studied ethics and philosophy, can someone better read point my to the above in formal context.

I figured out utilitarianism aged ~10 or so.

I had some thoughts about the "power" of mathematical proof techniques that I now recognize as pointing towards turing completeness.

Came up with the RNA-world hypothesis on my own when reading about the structure and function of ribosomes in middle school.

Decided long ago that there was a conflict between the age of the universe and the existence of improvements in space travel meaning that things such as we would never be able to reach self-replicating interstellar travel. Never came to the conclusion that it meant extinction at all and am still quite confused by people who assume its interstellar metastasis or bust.

Well, this isn't quite what you were asking for, but, as a young teenager a few days after 9/11, I was struck with a clear thought that went something like: "The American people are being whipped into a blood frenzy, and we are going to massively retaliate against somebody, perpetuating the endless cycle of violence that created the environment which enabled this attack to occur in the first place."

But I think it's actually common for young people to be better at realpolitik and we get worse at it as we absorb the mores of our culture.

In middle school I heard a fan theory that Neo had powers over the real world because it was a second layer of the matrix-- the idea of simulations inside simulations was enough for me to come to Bostrom's simulation argument.

Also during the same years I ended up doing an over the top version of comfort zone expansion by being really silly publicly.

In high school I think I basically argued a crude version of compatibilism before learning the term, although my memory of the conversation is a bit vague

1. This happened when I was 12 years old. I was trying to solve a problem at a mathematical contest which involved proving some identity with the nth powers of 5 and 7. I recall thinking vaguely "if you go to n+1 what is added in the left hand side is also in the right hand side" and so I discovered mathematical induction. In ten minutes I had a rigorous proof. Though, I didn't find it so convincing, so I ended with an unsure-of-myself comment "Hence, it is also valid for 3, 4, 5, 6 and so on..."

2. When I was in high school, creationism seemed unsatisfying in the sense of a Deus Ex Machina narrative (I often wonder how theists reconcile the contradiction between the feeling of religious wonder and the feeling of disappointment when facing Deus Ex Machina endings). The evolution "story" fascinated me with its slow and semi-random progression over billions of years. I guess this was my first taste of reductionism. (This is also an example of how optimizing for interestingness instead of truth has led me to the correct answer.)

I independently conceived of determinism and a vague sort of compatibilism when I was twelveish.

Cartesian skepticism and egoism, when I was maybe eleven. I eventually managed to argue myself out of both -- Cartesian skepticism fell immediately, but egoism took a few years.

(In case it isn't obvious from that, I did not have a very good childhood.)

I remember coming close to rediscovering pseudoformalism and the American caste system, but I discovered those concepts before I got all the way there.

When I was first learning about neural networks, I came up with the idea of de-convolutional networks: http://www.matthewzeiler.com/

Also, I think this is not totally uncommon. I think this suggests that there is low-hanging fruit in crowd-sourcing ideas from non-experts.

Another related thing that happens is that I'll be reading a book, and I'll have a question/thought that gets talked about later in the book.

I rediscovered most of the more widely agreed upon ontological categories (minus one that I still don't believe to adhere to the definition) before I knew they were called that, at about the age of 17. The idea of researching them came to me after reading a question from some stupid personality quiz they gave us in high school, something like "If you were a color, which color would you be?" -- and something about it rubbed me the wrong way, it just felt ontologically wrong, conflating entities with properties like that. (Yes, I did get the intended meaning of the question, I wasn't that much of an Aspie even back then, but I could also see it in the other, more literal way.)

I remember it was in the same afternoon that I also split up the verb "to be" into its constituent meanings, and named them. It seemed related.

Maybe these aren't so sophisticated, but I figured out determinism + a form of compatibilism, and the hard problem of consciousness in 10th grade.

In second or third grade, I noticed that (n+1) * (n+1) = (n * n) + n + (n+1).

I came up with a basic version of Tegmark's level 4 multiverse in high school and wrote an essay about it in English class. By that time though I think I'd already read Permutation City which involves similar ideas.

I think I was a de facto utilitarian from a very young age; perhaps eight or so.

I independently constructed algebra (of the '3*x+7=49. Solve for x.' variety) while being given 'guess and check' word problems in second grade. That's a slightly different variety than most of the other examples here, though.

It seems to me like something that can be solved by a community driven website where users can vote on questions.

There are concept inventories in a lot of fields, but these vary in quality and usefulness. The most well-known of these is the Force Concept Inventory for first semester mechanics, which basically aims to test how Aristotelian/Newtonian a student's thinking is. Any physicist can point out a dozen problems with it, but it seems to very roughly measure what it claims to measure.

Russ Roberts (host of the podcast EconTalk) likes to talk about the "economic way of thinking" and has written and gathered links about ten key ideas like incentives, markets, externalities, etc. But he's relatively libertarian, so the ideas he chose and his exposition will probably not provide a very complete picture. Anyway, EconTalk has started asking discussion questions after each podcast, some of which aim to test basic understanding along these lines.

I've often considered a self-assessment system where the sitter is prompted with a series of terms from the topic at hand, and asked to rate their understanding on a scale of 0-5, with 0 being "I've never heard of this concept", and 5 being "I could build one of these myself from scratch".

The terms are provided in a random order, and include red-herring terms that have nothing to do with the topic at hand, but sound plausible. Whoever provides the dictionary of terms should have some idea of the relative difficulty of each term, but you could refine it further and calibrate it against a sample of known diverse users, (novices, high-schoolers, undergrads, etc.)

When someone sits the test, you report their overall score relative to your calibrated sitters ("You scored 76, which puts you at undergrad level"), but you also report something like the Spearman rank coefficient of their answers against the difficulty of the terms. This provides a consistency check for their answers. If they frequently claim greater understanding of advanced concepts than basic ones, their understanding of the topic is almost certainly off-kilter (or they're lying). The presence of red-herring terms (which should all have canonical score of 0) means the rank coefficient consistency check is still meaningful for domain experts or people hitting the same value for every term.

Actually, this seems like a very good learning-a-new-web-framework dev project. I might give this a go.

See Mr.Money Mustache's 50 Jobs over $50,000 without a degree and SSC's Floor Employment for a number of suggestions.

I assume you don't consider going to a good school as a good path?

I am not aware of an existing one, although it is related to Moloch, as described in SSC when applied to the state:

although from a god’s-eye-view everyone knows that eliminating corporate welfare is the best solution, each individual official’s personal incentives push her to maintain it.

What Munger describes as The State, SSC calls Moloch. What your link calls the Munger test, may as well be called the Moloch test:

The Munger test:

In debates, I have found that it is useful to describe this problem as the "unicorn problem," precisely because it exposes a fatal weakness in the argument for statism. If you want to advocate the use of unicorns as motors for public transit, it is important that unicorns actually exist, rather than only existing in your imagination. People immediately understand why relying on imaginary creatures would be a problem in practical mass transit. But they may not immediately see why "the State" that they can imagine is a unicorn. So, to help them, I propose what I (immodestly) call "the Munger test."

Go ahead, make your argument for what you want the State to do, and what you want the State to be in charge of. Then, go back and look at your statement. Everywhere you said "the State" delete that phrase and replace it with "politicians I actually know, running in electoral systems with voters and interest groups that actually exist."

If you still believe your statement, then we have something to talk about.

Another possibility is that Americans are more individualistic. Maintaining a family fortune means subordinating yourself enough that it isn't spent down.

What I'd like to know is how the Brits are doing it.

I think there's a couple of fairly simple reasons contributing to Americans not having a culture of inheritance: first, that we live a long time by historical standards; and second, that we have a norm of children moving out after maturity. The first means that estates are generally released after children are well into their careers, and sometimes after they're themselves retired. The second means that all but the very wealthiest have to establish their own careers rather than living off the family dime.

This wouldn't directly affect actual inheritance, but it does take a lot of the urgency out of establishing a legacy. That lack of urgency might in turn contribute to reductions in real inheritance, given that you can sink a more or less arbitrary amount of money (by middle-class standards) into things like travel and expensive hobbies.

It is widespread. In other words, your thesis is not supported by facts. There is just nothing to explain here, except how the illusion of self-made fortunes is perpetuated in the teeth of the facts. The US has appallingly low social mobility, and a ever rising share of very rich americans got that way by inheritance. This isn't obvious because flaunting the fact that you got born with a silver spoon for every day of the week in your mouth runs against the american mythos - Members of the British old money crowd are proud of the fact that they personally did nothing to create their wealth and flaunt it. A proper 'murican in the exact same situation is slightly embarrassed by it and might at least show up for board meetings in the family business so that they can maintain some pretense that they work for a living.

Reading the comments... one commenter objects to WMI in a way which I would summarize as: "MWI provides identical experimental predictions to CI, which makes it useless, and also MWI provides wrong experimental predictions (unlike CI), which makes it wrong".

The author immediately detects the contradiction:

You know more about it than me. But if it's just equations and you can't empirically test which interpretation is true, then why does the Casimir force make MWI less likely?

Another commenter says that MWI has a greater complexity of thought, and while it is more useful to explore algorithmic possibilities on quantum computers abstractly, CI wins because it is about the real world.

Then the former commenter says (in reaction to the author) that MWI didn't provide useful predictions, and that Casimir force can only be explained by quantum equations and not by classical physics.

(Why exactly is that supposed to be an argument against MWI? No idea. Also, if MWI doesn't provide useful predictions, how can it be useful for studying quantum computers? Does it mean that quantum computers are never going to work in, you know, the real life?)

Finally, yet another commenter explains things from MWI point of view, saying that "observers" must follow the same fundamental physics as rocks.

Maybe he'll grow up to be a mathematician.

If I only had three wishes, I would still spend one of them on having enough sense to make good wishes. I'd probably do that if I only had two wishes.

I might even use my only wish on having significantly better sense. My current situation isn't desperate-- if I only had one wish and were desperate, the best choice might well be to use the wish on dealing with the desperate circumstance as thoroughly as possible.

For me it is obviously "I want a friendly artificial general intelligence" - seems like the safest bet.

But the AI would still be constrained by the laws of physics. Intelligence can't beat thermodynamics. You need to wish for an omnipotent friendly AI.

Here is a list of papers about the talks, if you want to get an idea without attending.

It doesn't seem to be clear whether that's just people of different cultures grouping faces differently, like how they might group colors differently even though their eyes work the same, or if their face/emotion correspondence is different.

Whole life via Rudi Hoffman for Alcor.

I've not personally finished my own arrangements, but I'll likely be using whole life of some kind. I do know that Rudi Hoffman is an agent well recommended by people who've gone the insurance route, so talking to him will likely get you a much better idea of what choices people make (A small warning, his sight is not the prettiest thing). You could also contact the people recommended on Alcor's Insurance Agents page, if you so desire.

Have you ever clicked on the grey envelope icon found at the bottom right of every post and comment? If you do, then immediate replies to it show up in your inbox also. Look at the parent of one of these mysterious messages and see if its envelope is green. If it is, you can click it again to turn it off.

If a reply to one of your comments is deleted before you read it, you'll be alerted but won't get the message. I believe the alert should go away once you check your messages, though.

I think if someone is responding to you in a very downvoted thread it might not show up in the your replies?

I find it hard to think of ISIS members as human, or at least I don't want to belong to the same species as them.

Beware of refusing to believe undeniable reality just because it's not nice.

How morally different are ISIS fighters from us? If we had a similar upbringing would we think it morally correct to kill Yazidi children for having the "wrong" religion?

A relevant factor which is (intentionally or not) ignored by American media is that, from the point of view of pious Muslims, Yazidis are satanists.

To quote Wikipedia (Taus Melek is basically the chief deity for Yazidis, God the Creator being passive and uninvolved with the world):

As a demiurge figure, Tawûsê Melek is often identified by orthodox Muslims as a Shaitan (Satan), a Muslim term denoting a devil or demon who deceives true believers. The Islamic tradition regarding the fall of "Shaitan" from Grace is in fact very similar to the Yazidi story of Malek Taus – that is, the Jinn who refused to submit to God by bowing to Adam is celebrated as Tawûsê Melek by Yazidis, but the Islamic version of the same story curses the same Jinn who refused to submit as becoming Satan.[38] Thus, the Yazidi have been accused of devil worship.

So, what's the Christianity's historical record for attitude towards devil worshippers?

or at least I don't want to belong to the same species as them

Any particular reason you feel this way about the Sunni armed groups, but not about, say, Russian communists, or Mao's Chinese, or Pol Pot's Cambodians, or Rwandans, or... it's a very long list, y'know?

I find it hard to think of ISIS members as human

That's how the ISIS fighters feel about the Yazidi.

Or might genetics play a role in our differing moral views?

It's possible that more inbred clannish societies have smaller moral circles than Western outbreeders.

I against my brother, my brothers and I against my cousins, then my cousins and I against strangers

• Bedouin proverb

It's a little harder to say about the ISIS guys, but I think personality wise many of us are a lot like the Al Qaeda leadership. Ideology and Jihad for it appeals.

Most people don't take ideas too seriously. We do. And I think it's largely genetic.

I find it hard to think of ISIS members as human

Human, All Too Human.

Historically, massacring The Other is the rule, not the exception. You don't even need to be particularly ideological for that. People who just go with the flow of their community will set The Other on fire in a public square, and have a picnic watching. Bring their kids. Take grandma out for the big show.

How morally different are ISIS fighters from us?

Uhm, taboo "morally different"?

Are their memes repulsive to me? Yes, they are.

Do they have terminal value as humans (ignoring their instrumental value)? Yes, they do.

How about their instrumental value? Uhm, probably negative, since they seem to spend a lot of time killing other humans.

If we had a similar upbringing would we think it morally correct to kill Yazidi children for having the "wrong" religion? Or might genetics play a role in our differing moral views?

Probably yes. I think there can be a genetic influence, but there is much more of "monkey see, monkey do" in humans.

First you might want to consider propaganda.

http://www.revleft.com/vb/ten-commandments-war-t52907/index.html?s=8387131b8a98f6ee7e6ba74cce570d8e

http://home.cc.umanitoba.ca/~mkinnear/16_Falsehood_in_wartime.pdf

1. We do not want war.

2. The opposite party alone is guilty of war

3. The enemy is the face of the devil.

4. We defend a noble cause, not our own interest.

5. The enemy systematically commits cruelties; our mishaps are involuntary.

6. The enemy uses forbidden weapons.

7. We suffer small losses, those of the enemy are enormous.

8. Artists and intellectuals back our cause.

9. Our cause is sacred.

10. All who doubt our propaganda, are traitors.

Part of "us" is our culturally transmitted values.

My impression is that ISIS is mostly a new thing-- it's a matter of relatively new memes taken up by adolescents and adults rather than generational transmission.

I don't think it's practical to see one's enemies, even those who behave vilely and are ideologically committed to continuing to do so, as non-human. To see them as non-human is to commit oneself as framing them as incomprehensible. More exactly, the usual outcomes seems to be "all they understand is force" or "there's nothing to do but kill them". which makes it difficult to think of how to deal with them if victory by violence isn't a current option.

If we had a similar upbringing would we think it morally correct to kill Yazidi children for having the "wrong" religion?

The question is irrelevant. If it is wrong to behead children for having the "wrong" religion, that is not affected by fictional scenarios in which "we" believed differently. (It's not clear what "we" actually means there, but that's a separate philosophical issue.) Truth is not found by first seeing what you believe, and then saying, "I believe this, therefore it is true."

Or might genetics play a role in our differing moral views?

This question is also irrelevant.

I find it hard to think of ISIS members as human

Well, they are. Start from there.

Here is a Vice documentary posted today about ISIS: https://news.vice.com/video/the-islamic-state-full-length

To put it as simply as I could, LessWrongers like to quantify stuff. A more specific instance of this is the fact that, since this website started off as the brainchild of an AI researcher, the prevalent intellectual trends will be those with applicability in AI research. Computers work easily with quantifiable data. As such, if you want to instill human morality into an AI, chances are you'll at least consider conceptualizing morality in utilitarian terms.

The confluence of a number of ideas.

Cox's theorem shows that degree of belief can be expressed as probabilities.

The VNM theorem shows that preferences can be expressed as numbers (up to an additive constant), usually called utilities.

Consequentialism, the idea that actions are to be judged by their consequences, is pretty much taken as axiomatic.

Combining these gives the conclusion that the rational action to take in any situation is the one that maximises the resulting expected utility.

Your morality is your utility function: your beliefs about how people should live are preferences about they should live.

Add the idea of actually being convinced by arguments (except arguments of the form "this conclusion is absurd, therefore there is likely to be something wrong with the argument", which are merely the absurdity heuristic) and you get LessWrong utilitarianism.

To the extent that lesswrong has an official ethical system, that system is definitely not utilitarianism.

In general this site focuses on the friendly AI problem, a nihilistic or a hedonistic AI might not be friendly to humans. The notion of an existentialist AI seems to be largely unexplored as far as I know.

is there something else I should do?

Find out how your brain went wrong, with a view to not going so wrong again.

It seems I was unclear. I have no intention of attempting to kill all humans. I'm not posting the argument publicly because I don't want to run the (admittedly small) risk that someone else will read it and take it seriously. I'm just wondering if there's anything I can do with this argument that will make the world a slightly better place, instead of just not sharing it (which is mildly negative to me and neutral to everyone else - unless I've sparked anyone's curiousity, for which I apologise).

What values could possibly lead to such a choice?

You should consider that the problem may not be in the argument, but in your beliefs about the values you think you have.

I'd say not to worry about it unless it's a repetitive thought.

Reform yourself. Killing all humans is axiomatically evil in my playbook, so eithar (a) you are reasoning from principles which permit Mark!evil (which makes you Mark!evil, and on my watch-list), or (b) you made a mistake. It's probably the latter.

Do you care about it? It sounds like you're responding appropriately (though IMO it's better that such arguments be public and be refuted publicly, as otherwise they present a danger to people who are smart or lucky enough to think up the argument but not the refutation). If the generation of that argument, or what it implies about your brain, is causing trouble with your life then it's worth investigating, but if it's not bothering you then such investigation might not be worth the cost.

Since you won't be able to kill all humans and will eventually get caught and imprisoned, the best move is to abandon your plan, accordingo to utilitarian logic.

The description: "Philosophy in Video Games [F]: A discussion of philosophical themes present in many different video games. Topics will include epistemology, utilitarianism, philosophy of science, ethics, logic, and metaphysics. All topics will be explained upon introduction and no prior knowledge is necessary to participate!"

Did they record all panels?

See Coupon collector's problem, particularly "tail estimates".