Filter Last three months

Less Wrong is a community blog devoted to refining the art of human rationality. Please visit our About page for more information.

LW 2.0 Strategic Overview

46 Habryka 15 September 2017 03:00AM

Update: We're in open beta! At this point you will be able to sign up / login with your LW 1.0 accounts (if the latter, we did not copy over your passwords, so hit "forgot password" to receive a password-reset email).

Hey Everyone! 

This is the post for discussing the vision that I and the rest of the LessWrong 2.0 team have for the new version of LessWrong, and to just generally bring all of you up to speed with the plans for the site. This post has been overdue for a while, but I was busy coding on LessWrong 2.0, and I am myself not that great of a writer, which means writing things like this takes quite a long time for me, and so this ended up being delayed a few times. I apologize for that.

With Vaniver’s support, I’ve been the primary person working on LessWrong 2.0 for the last 4 months, spending most of my time coding while also talking to various authors in the community, doing dozens of user-interviews and generally trying to figure out how to make LessWrong 2.0 a success. Along the way I’ve had support from many people, including Vaniver himself who is providing part-time support from MIRI, Eric Rogstad who helped me get off the ground with the architecture and infrastructure for the website, Harmanas Chopra who helped build our Karma system and did a lot of user-interviews with me, Raemon who is doing part-time web-development work for the project, and Ben Pace who helped me write this post and is basically co-running the project with me (and will continue to do so for the foreseeable future).

We are running on charitable donations, with $80k in funding from CEA in the form of an EA grant and $10k in donations from Eric Rogstad, which will go to salaries and various maintenance costs. We are planning to continue running this whole project on donations for the foreseeable future, and legally this is a project of CFAR, which helps us a bunch with accounting and allows people to get tax benefits from giving us money. 

Now that the logistics is out of the way, let’s get to the meat of this post. What is our plan for LessWrong 2.0, what were our key assumptions in designing the site, what does this mean for the current LessWrong site, and what should we as a community discuss more to make sure the new site is a success?

Here’s the rough structure of this post: 

  • My perspective on why LessWrong 2.0 is a project worth pursuing
  • A summary of the existing discussion around LessWrong 2.0 
  • The models that I’ve been using to make decisions for the design of the new site, and some of the resulting design decisions
  • A set of open questions to discuss in the comments where I expect community input/discussion to be particularly fruitful 

Why bother with LessWrong 2.0?  

I feel that independently of how many things were and are wrong with the site and its culture, overall, over the course of its history, it has been one of the few places in the world that I know off where a spark of real discussion has happened, and where some real intellectual progress on actually important problems was made. So let me begin with a summary of things that I think the old LessWrong got right, that are essential to preserve in any new version of the site:

On LessWrong…


  • I can contribute to intellectual progress, even without formal credentials 
  • I can sometimes have discussions in which the participants focus on trying to convey their true reasons for believing something, as opposed to rhetorically using all the arguments that support their position independent of whether those have any bearing on their belief
  • I can talk about my mental experiences in a broad way, such that my personal observations, scientific evidence and reproducible experiments are all taken into account and given proper weighting. There is no narrow methodology I need to conform to to have my claims taken seriously.
  • I can have conversations about almost all aspects of reality, independently of what literary genre they are associated with or scientific discipline they fall into, as long as they seem relevant to the larger problems the community cares about
  • I am surrounded by people who are knowledgeable in a wide range of fields and disciplines, who take the virtue of scholarship seriously, and who are interested and curious about learning things that are outside of their current area of expertise
  • We have a set of non-political shared goals for which many of us are willing to make significant personal sacrifices
  • I can post long-form content that takes up as much space at it needs to, and can expect a reasonably high level of patience of my readers in trying to understand my beliefs and arguments
  • Content that I am posting on the site gets archived, is searchable and often gets referenced in other people's writing, and if my content is good enough, can even become common knowledge in the community at large
  • The average competence and intelligence on the site is high, which allows discussion to generally happen on a high level and allows people to make complicated arguments and get taken seriously
  • There is a body of writing that is generally assumed to have been read by most people  participating in discussions that establishes philosophical, social and epistemic principles that serve as a foundation for future progress (currently that body of writing largely consists of the Sequences, but also includes some of Scott’s writing, some of Luke’s writing and some individual posts by other authors) 


When making changes to LessWrong, I think it is very important to preserve all of the above features. I don’t think all of them are universally present on LessWrong, but all of them are there at least some of the time, and no other place that I know of comes even remotely close to having all of them as often as LessWrong has. Those features are what motivated me to make LessWrong 2.0 happen, and set the frame for thinking about the models and perspectives I will outline in the rest of the post. 

I also think Anna, in her post about the importance of a single conversational locus, says another, somewhat broader thing, that is very important to me, so I’ve copied it in here: 

1. The world is locked right now in a deadly puzzle, and needs something like a miracle of good thought if it is to have the survival odds one might wish the world to have.

2. Despite all priors and appearances, our little community (the "aspiring rationality" community; the "effective altruist" project; efforts to create an existential win; etc.) has a shot at seriously helping with this puzzle.  This sounds like hubris, but it is at this point at least partially a matter of track record.

3. To aid in solving this puzzle, we must probably find a way to think together, accumulatively. We need to think about technical problems in AI safety, but also about the full surrounding context -- everything to do with understanding what the heck kind of a place the world is, such that that kind of place may contain cheat codes and trap doors toward achieving an existential win. We probably also need to think about "ways of thinking" -- both the individual thinking skills, and the community conversational norms, that can cause our puzzle-solving to work better.

4. One feature that is pretty helpful here, is if we somehow maintain a single "conversation", rather than a bunch of people separately having thoughts and sometimes taking inspiration from one another.  By "a conversation", I mean a space where people can e.g. reply to one another; rely on shared jargon/shorthand/concepts; build on arguments that have been established in common as probably-valid; point out apparent errors and then have that pointing-out be actually taken into account or else replied-to).

5. One feature that really helps things be "a conversation" in this way, is if there is a single Schelling set of posts/etc. that people (in the relevant community/conversation) are supposed to read, and can be assumed to have read.  Less Wrong used to be a such place; right now there is no such place; it seems to me highly desirable to form a new such place if we can.

6. We have lately ceased to have a "single conversation" in this way.  Good content is still being produced across these communities, but there is no single locus of conversation, such that if you're in a gathering of e.g. five aspiring rationalists, you can take for granted that of course everyone has read posts such-and-such.  There is no one place you can post to, where, if enough people upvote your writing, people will reliably read and respond (rather than ignore), and where others will call them out if they later post reasoning that ignores your evidence.  Without such a locus, it is hard for conversation to build in the correct way.  (And hard for it to turn into arguments and replies, rather than a series of non sequiturs.)

The Existing Discussion Around LessWrong 2.0

Now that I’ve given a bit of context on why I think LessWrong 2.0 is an important project, it seems sensible to look at what has been said so far, so we don’t have to repeat the same discussions over and over again. There has already been a lot of discussion about the decline of LessWrong, the need for a new platform and the design of LessWrong 2.0, and I won’t be able to summarise it all here, but I can try my best to summarize the most important points, and give a bit of my own perspective on them.

Here is a comment by Alexandros, on Anna’s post I quoted above:

Please consider a few gremlins that are weighing down LW currently:

1. Eliezer's ghost -- He set the culture of the place, his posts are central material, has punctuated its existence with his explosions (and refusal to apologise), and then, upped and left the community, without actually acknowledging that his experiment (well kept gardens etc) has failed. As far as I know he is still the "owner" of this website, retains ultimate veto on a bunch of stuff, etc. If that has changed, there is no clarity on who the owner is (I see three logos on the top banner, is it them?), who the moderators are, who is working on it in general. I know tricycle are helping with development, but a part-time team is only marginally better than no-team, and at least no-team is an invitation for a team to step up.


...I consider Alexei's hints that Arbital is "working on something" to be a really bad idea, though I recognise the good intention. Efforts like this need critical mass and clarity, and diffusing yet another wave of people wanting to do something about LW with vague promises of something nice in the future... is exactly what I would do if I wanted to maintain the status quo for a few more years.

Any serious attempt at revitalising should focus on defining ownership and plan clearly. A post by EY himself recognising that his vision for lw 1.0 failed and passing the batton to a generally-accepted BDFL would be nice, but i'm not holding my breath. Further, I am fairly certain that LW as a community blog is bound to fail. Strong writers enjoy their independence. LW as an aggregator-first (with perhaps ability to host content if people wish to, like hn) is fine. HN may have degraded over time, but much less so than LW, and we should be able to improve on their pattern.

I think if you want to unify the community, what needs to be done is the creation of a hn-style aggregator, with a clear, accepted, willing, opinionated, involved BDFL, input from the prominent writers in the community (scott, robin, eliezer, nick bostrom, others), and for the current to be archived in favour of that new aggregator. But even if it's something else, it will not succeed without the three basic ingredients: clear ownership, dedicated leadership, and as broad support as possible to a simple, well-articulated vision. Lesswrong tried to be too many things with too little in the way of backing.

I think Alexandros hits a lot of good points here, and luckily these are actually some of the problems I am most confident we have solved. The biggest bottleneck – the thing that I think caused most other problems with LessWrong – is simply that there was nobody with the motivation, the mandate and the resources to fight against the inevitable decline into entropy. I feel that the correct response to the question of “why did LessWrong decline?” is to ask “why should it have succeeded?”. 

In the absence of anyone with the mandate trying to fix all the problems that naturally arise, we should expect any online platform to decline. Most of the problems that will be covered in the rest of this post are things that could have been fixed many years ago, but simply weren’t because nobody with the mandate put much resources into fixing them. I think the cause for this was a diffusion of responsibility, and a lot of vague promises of problems getting solved by vague projects in the future. I myself put off working on LessWrong for a few months because I had some vague sense that Arbital would solve the problems that I was hoping to solve, even though Arbital never really promised to solve them. Then Arbital’s plan ended up not working out, and I had wasted months of precious time. 

Since this comment was written, Vaniver has been somewhat unanimously declared benevolent dictator for life of LessWrong. He and I have gotten various stakeholders on board, received funding, have a vision, and have free time – and so we have the mandate, the resources and the motivation to not make the same mistakes. With our new codebase, link posts are now something I can build in an afternoon, rather than something that requires three weeks of getting permissions from various stakeholders, performing complicated open-source and confidentiality rituals, and hiring a new contractor who has to first understand the mysterious Reddit fork from 2008 that LessWrong is based on. This means at least the problem of diffusion of responsibility is solved. 

Scott Alexander also made a recent comment on Reddit on why he thinks LessWrong declined, and why he is somewhat skeptical of attempts to revive the website: 

1. Eliezer had a lot of weird and varying interests, but one of his talents was making them all come together so you felt like at the root they were all part of this same deep philosophy. This didn't work for other people, and so we ended up with some people being amateur decision theory mathematicians, and other people being wannabe self-help gurus, and still other people coming up with their own theories of ethics or metaphysics or something. And when Eliezer did any of those things, somehow it would be interesting to everyone and we would realize the deep connections between decision theory and metaphysics and self-help. And when other people did it, it was just "why am I reading this random bulletin board full of stuff I'm not interested in?"

2. Another of Eliezer's talents was carefully skirting the line between "so mainstream as to be boring" and "so wacky as to be an obvious crackpot". Most people couldn't skirt that line, and so ended up either boring, or obvious crackpots. This produced a lot of backlash, like "we need to be less boring!" or "we need fewer crackpots!", and even though both of these were true, it pretty much meant that whatever you posted, someone would be complaining that you were bad.

3. All the fields Eliezer wrote in are crackpot-bait and do ring a bunch of crackpot alarms. I'm not just talking about AI - I'm talking about self-help, about the problems with the academic establishment, et cetera. I think Eliezer really did have interesting things to say about them - but 90% of people who try to wade into those fields will just end up being actual crackpots, in the boring sense. And 90% of the people who aren't will be really bad at not seeming like crackpots. So there was enough kind of woo type stuff that it became sort of embarassing to be seen there, especially given the thing where half or a quarter of the people there or whatever just want to discuss weird branches of math or whatever.

4. Communities have an unfortunate tendency to become parodies of themselves, and LW ended up with a lot of people (realistically, probably 14 years old) who tended to post things like "Let's use Bayes to hack our utility functions to get superfuzzies in a group house!". Sometimes the stuff they were posting about made sense on its own, but it was still kind of awkward and the sort of stuff people felt embarassed being seen next to.

5. All of these problems were exacerbated by the community being an awkward combination of Google engineers with physics PhDs and three startups on one hand, and confused 140 IQ autistic 14 year olds who didn't fit in at school and decided that this was Their Tribe Now on the other. The lowest common denominator that appeals to both those groups is pretty low.

6. There was a norm against politics, but it wasn't a very well-spelled-out norm, and nobody enforced it very well. So we would get the occasional leftist who had just discovered social justice and wanted to explain to us how patriarchy was the real unfriendly AI, the occasional rightist who had just discovered HBD and wanted to go on a Galileo-style crusade against the deceptive establishment, and everyone else just wanting to discuss self-help or decision-theory or whatever without the entire community becoming a toxic outcast pariah hellhole. Also, this one proto-alt-right guy named Eugene Nier found ways to exploit the karma system to mess with anyone who didn't like the alt-right (ie 98% of the community) and the moderation system wasn't good enough to let anyone do anything about it.

7. There was an ill-defined difference between Discussion (low-effort random posts) and Main (high-effort important posts you wanted to show off). But because all these other problems made it confusing and controversial to post anything at all, nobody was confident enough to post in Main, and so everything ended up in a low-effort-random-post bin that wasn't really designed to matter. And sometimes the only people who didpost in Main were people who were too clueless about community norms to care, and then their posts became the ones that got highlighted to the entire community.

8. Because of all of these things, Less Wrong got a reputation within the rationalist community as a bad place to post, and all of the cool people got their own blogs, or went to Tumblr, or went to Facebook, or did a whole bunch of things that relied on illegible local knowledge. Meanwhile, LW itself was still a big glowing beacon for clueless newbies. So we ended up with an accidental norm that only clueless newbies posted on LW, which just reinforced the "stay off LW" vibe.

I worry that all the existing "resurrect LW" projects, including some really high-effort ones, have been attempts to break coincidental vicious cycles - ie deal with 8 and the second half of 7. I think they're ignoring points 1 through 6, which is going to doom them.

At least judging from where my efforts went, I would agree that I have spent a pretty significant amount of resources on fixing the problems that Scott described in point 6 and 7, but I also spent about equal time thinking about how to fix 1-5. The broader perspective that I have on those latter points is I think best illustrated in an analogy: 

When I read Scott’s comments about how there was just a lot of embarrassing and weird writing on LessWrong, I remember my experiences as a Computer Science undergraduate. When the median undergrad makes claims about the direction of research in their field, or some other big claim about their field that isn't explicitly taught in class, or if you ask an undergraduate physics student what they think about how to do physics research, or what ideas they have for improving society, they will often give you quite naive sounding answers (I have heard everything from “I am going to build a webapp to permanently solve political corruption” to “here’s my idea of how we can transmit large amounts of energy wirelessly by using low-frequency tesla-coils”.) I don’t think we should expect anything different on LessWrong. I actually think we should expect it to be worse here, since we are actively encouraging people to have opinions, as opposed to the more standard practice of academia, which seems to consist of treating undergraduates as slightly more intelligent dogs that need to be conditioned with the right mixture of calculus homework problems and mandatory class attendance, so that they might be given the right to have any opinion at all if they spend 6 more years getting their PhD. 

So while I do think that Eliezer’s writing encouraged topics that were slightly more likely to attract crackpots, I think a large chunk of the weird writing is just a natural consequence of being an intellectual community that has a somewhat constant influx of new members. 

And having undergraduates go through the phase where they have bad ideas, and then have it explained to them why their ideas are bad, is important. I actually think it’s key to learning any topic more complicated than high-school mathematics. It takes a long time until someone can productively contribute to the intellectual progress of an intellectual community (in academia it’s at least 4 years, though usually more like 8), and during all that period they will say very naive and silly sounding things (though less and less so as time progresses). I think LessWrong can do significantly better than 4 years, but we should still expect that it will take new members time to acclimate and get used to how things work (based on user-interviews of a lot of top commenters it usually took something like 3-6 months until someone felt comfortable commenting frequently and about 6-8 months until someone felt comfortable posting frequently. This strikes me as a fairly reasonable expectation for the future). 

And I do think that we have many graduate students and tenured professors of the rationality community who are not Eliezer, and who do not sound like crackpots, that can speak reasonably about the same topics Eliezer talked about, and who I feel are acting with a very similar focus to what Eliezer tried to achieve. Luke Muehlhauser, Carl Shulman, Anna Salamon, Sarah Constantin, Ben Hoffman, Scott himself and many more, most of whose writing would fit very well on LessWrong (and often still ends up there). 

But all of this doesn’t mean what Scott describes isn’t a problem. It’s still a bad experience for everyone to constantly have to read through bad first year undergrad essays, but I think the solution can’t involve those essays not getting written at all. Instead it has to involve some kind of way of not forcing everyone to see those essays, while still allowing them to get promoted if someone shows up who does write something insightful from day one. I am currently planning to tackle this mostly with improvements to the karma system, as well as changes to the layout of the site, where users primarily post to their own profiles and can get content promoted to the frontpage by moderators and high-karma members. A feed consisting solely of content of the quality of the average Scott, Anna, Ben or Luke post would be an amazing read, and is exactly the kind of feed I am hoping to create with LessWrong, while still allowing users to engage with the rest of the content on the site (more on that later).

I would very very roughly summarize what Scott says in the first 5 points as two major failures: first a failure of separating the signal from the noise, and second a failure of enforcing moderation norms when people did turn out to be crackpots or just unable to productively engage with the material on the site. Both of which are natural consequences of the abandonment of promoting things to main, the fact that discussion is ordered by default by recency and not by some kind of scoring system, and the fact that the moderation tools were completely insufficient (but more on the details of that in the next section)

My models of LessWrong 2.0

I think there are three major bottlenecks that LessWrong is facing (after the zeroth bottleneck, which is just that no single group had the mandate, resources and motivation to fix any of the problems): 

  1. We need to be able to build on each other’s intellectual contributions, archive important content and avoid primarily being news-driven
  2. We need to improve the signal-to-noise ratio for the average reader, and only broadcast the most important writing
  3. We need to actively moderate in a way that is both fun for the moderators, and helps people avoid future moderation policy violations


The first bottleneck for our community, and the biggest I think, is the ability to build common knowledge. On facebook, I can read an excellent and insightful discussion, yet one week later I forgot it. Even if I remember it, I don’t link to the facebook post (because linking to facebook posts/comments is hard) and it doesn’t have a title so I don’t casually refer to it in discussion with friends. On facebook, ideas don’t get archived and built upon, they get discussed and forgotten. To put this another way, the reason we cannot build on the best ideas this community had over the last five years, is because we don’t know what they are. There’s only fragments of memories of facebook discussions which maybe some other people remember. We have the sequences, and there’s no way to build on them together as a community, and thus there is stagnation.

Contrast this with science. Modern science is plagued by many severe problems, but of humanity’s institutions it has perhaps the strongest record of being able to build successfully on its previous ideas. The physics community has this system where the new ideas get put into journals, and then eventually if they’re new, important, and true, they get turned into textbooks, which are then read by the upcoming generation of physicists, who then write new papers based on the findings in the textbooks. All good scientific fields have good textbooks, and your undergrad years are largely spent reading them. I think the rationality community has some textbooks, written by Eliezer (and we also compiled a collection of Scott’s best posts that I hope will become another textbook of the community), but there is no expectation that if you write a good enough post/paper that your content will be included in the next generation of those textbooks, and the existing books we have rarely get updated. This makes the current state of the rationality community analogous to a hypothetical state of physics, had physics no journals, no textbook publishers, and only one textbook that is about a decade old. 

This seems to me what Anna is talking about - the purpose of the single locus of conversation is the ability to have common knowledge and build on it. The goal is to have every interaction with the new LessWrong feel like it is either helping you grow as a rationalist or has you contribute to lasting intellectual progress of the community. If you write something good enough, it should enter the canon of the community. If you make a strong enough case against some existing piece of canon, you should be able to replace or alter that canon. I want writing to the new LessWrong to feel timeless. 

To achieve this, we’ve built the following things: 

  • We created a section for core canon on the site that is prominently featured on the frontpage and right now includes Rationality: A-Z, The Codex (a collection of Scott’s best writing, compiled by Scott and us), and HPMOR. Over time I expect these to change, and there is a good chance HPMOR will move to a different section of the site (I am considering adding an “art and fiction” section) and will be replaced by a new collection representing new core ideas in the community.
  • Sequences are now a core feature of the website. Any user can create sequences of their own and other users posts, and those sequences themselves can be voted and commented on. The goal is to help users compile the best writing on the site, and make it so that good timeless writing gets read by users for a long time, as opposed to disappearing into the void. Separating creative and curatorial effort allows the sort of professional specialization that you see in serious scientific fields.
  • Of those sequences, the most upvoted and most important ones will be chosen to be prominently featured on other sections of the site, allowing users easy access to read the best content on the site and get up to speed with the current state of knowledge of the community.
  • For all posts and sequences the site keeps track of how much of them you’ve read (including importing view-tracking from old LessWrong, so you will get to see how much of the original sequences you’ve actually read). And if you’ve read all of a sequence you get a small badge that you can choose to display right next to your username, which helps people navigate how much of the content of the site you are familiar with.
  • The design of the core content of the site (e.g. the Sequences, the Codex, etc.) tries to communicate a certain permanence of contributions. The aesthetic feels intentionally book-like, which I hope gives people a sense that their contributions will be archived, accessible and built-upon.
    One important issue with this is that there also needs to be a space for sketches on LessWrong. To quote PaulGraham: “What made oil paint so exciting, when it first became popular in the fifteenth century, was that you could actually make the finished work from the prototype. You could make a preliminary drawing if you wanted to, but you weren't held to it; you could work out all the details, and even make major changes, as you finished the painting.”
  • We do not want to discourage sketch-like contributions, and want to build functionality that helps people build a finished work from a prototype (this is one of the core competencies of Google Docs, for example).

And there are some more features the team is hoping to build in this direction, such as: 

  • Easier archiving of discussions by allowing discussions to be turned into top-level posts (similar to what Ben Pace did with a recent Facebook discussion between Eliezer, Wei Dai, Stuart Armstrong, and some others, which he turned into a post on LessWrong 2.0
  • The ability to continue reading the content you’ve started reading with a single click from the frontpage. Here's an example logged-in frontpage:




The second bottleneck is improving the signal-to-noise ratio. It needs to be possible for someone to subscribe to only the best posts on LessWrong, and only the most important content needs to turned into common-knowledge. 

I think this is a lot of what Scott was pointing at in his summary about the decline of LessWrong. We need a way for people to learn from their mistakes, while also not flooding the inboxes of everyone else, and while giving people active feedback on how to improve in their writing. 

The site structure: 

To solve this bottleneck, here is the rough content structure that I am currently planning to implement on LessWrong: 

The writing experience: 

If you write a post, it first shows up nowhere else but your personal user page, which you can basically think of being a medium-style blog. If other users have subscribed to you, your post will then show up on their frontpages (or only show up after it hit a certain karma threshold, if users who subscribed to you set a minimum karma threshold). If you have enough karma you can decide to promote your content to the main frontpage feed (where everyone will see it by default), or a moderator can decide to promote your content (if you allowed promoting on that specific post). The frontpage itself is sorted by a scoring system based on the HN algorithm, which uses a combination of total karma and how much time has passed since the creation of the post. 

If you write a good comment on a post a moderator or a high-karma user can promote that comment to the frontpage as well, where we will also feature the best comments on recent discussions. 


Meta will just be a section of the site to discuss changes to moderation policies, issues and bugs with the site, discussion about site features, as well as general site-policy issues. Basically the thing that all StackExchanges have. Karma here will not add to your total karma and will not give you more influence over the site. 

Featured posts

In addition to the main thread, there is a promoted post section that you can subscribe to via email and RSS, that has on average three posts a week, which for now are just going to be chosen by moderators and editors on the site to be the posts that seem most important to turn into common-knowledge for the community. 

Meetups (implementation unclear)

There will also be a separate section of the site for meetups and event announcements that will feature a map of meetups, and generally serve as a place to coordinate the in-person communities. The specific implementation of this is not yet fully figured out. 

Shortform (implementation unclear)

Many authors (including Eliezer) have requested a section of the site for more short-form thoughts, more similar to the length of an average FB post. It seems reasonable to have a section of the site for that, though I am not yet fully sure how it should be implemented. 


The goal of this structure is to allow users to post to LessWrong without their content being directly exposed to the whole community. Their content can first be shown to the people who follow them, or the people who actively seek out content from the broader community by scrolling through all new posts. Then, if a high-karma users among them finds their content worth posting to the frontpage, it will get promoted. The key to this is a larger userbase that has the ability to promote content (i.e. many more than have the ability to promote content to main on the current LessWrong), and the continued filtering of the frontpage based on the karma level of the posts. 

The goal of all of these is to allow users to see good content at various levels of engagement with the site, while giving some personalization options so that people can follow the people they are particularly interested and while also ensuring that this does not sabotage the attempt at building common knowledge by having the best posts from the whole ecosystem be featured and promoted on the frontpage. 

The karma system:

Another thing I’ve been working on to fix the signal-to-noise ratio is to improve the karma system. It’s important that the people having the most significant insights are able to shape a field more. If you’re someone who regularly produces real insights, you’re better able to notice and bring up other good ideas. To achieve this we’ve built a new karma system, where your upvotes and downvotes weight more if you have a lot of karma already. So far the current weighting is a very simple heuristic, whereby your upvotes and downvotes count for log base 5 of your total karma. Ben and I will post another top-level post to discuss just the karma system at some point in the next few weeks, but feel free to ask any questions now, and we will just include those in that post.

(I am currently experimenting with a karma system based on the concept of eigendemocracy by Scott Aaronson, which you can read about here, but which basically boils down to applying Google’s PageRank algorithm to karma allocation. How trusted you are as a user (your karma) is based on how much trusted users upvote you, and the circularity of this definition is solved using linear algebra.)

I am also interested in having some form of two-tiered voting, similarly to how Facebook has a primary vote interaction (the like) and a secondary interaction that you can access via a tap or a hover (angry, sad, heart, etc.). But the implementation of that is also currently undetermined. 


The third and last bottleneck is an actually working moderation system that is fun to use by moderators, while also giving people whose content was moderated a sense of why, and how they can improve. 

The most common, basic complaint currently on LessWrong pertains to trolls and sockpuppet accounts that the reddit fork’s mod tools are vastly inadequate for dealing with (Scott's sixth point refers to this). Raymond Arnold and I are currently building more nuanced mod tools, that include abilities for moderators to set the past/future votes of a user to zero, to see who upvoted a post, and to know the IP address that an account comes from (this will be ready by the open beta). 

Besides that, we are currently working on cultivating a moderation group we are calling “Sunshine Regiment.” Members of the sunshine regiment that will have the ability to take various smaller moderation actions around the site (such as temporarily suspending comment threads, making general moderating comments in a distinct font and promoting content), and so will have the ability to generally shape the culture and content of the website to a larger degree.

The goal is moderation that goes far beyond dealing with trolls, and actively makes the epistemic norms a ubiquitous part of the website. Right now Ben Pace is thinking about moderation norms that encourage archiving and summarizing good discussion, as well as other patterns of conversation that will help the community make intellectual progress. He’ll be posting to the open beta to discuss what norms the site and moderators should have in the coming weeks. We're both in agreement that moderation can and should be improved, and that moderators need better tools, and would appreciate good ideas about what else to give them.

How you can help and issues to discuss:

The open beta of the site is starting in a week, and so you can see all of this for yourself. For the duration of the open beta, we’ll continue the discussion on the beta site. At the conclusion of the open beta, we plan to have a vote open to those who had a thousand karma or more on 9/13 to determine whether we should move forward with the new site design, which would move to the url from its temporary beta location, or leave LessWrong as it is now. (As this would represent the failure of the plan to revive LW, this would likely lead to the site being archived rather than staying open in an unmaintained state.) For now, this is an opportunity for the current LessWrong community to chime in here and object to anything in this plan.

During the open beta (and only during that time) the site will also have an Intercom button in the bottom right corner that allows you to chat directly with us. If you run into any problems, or notice any bugs, feel free to ping us directly on there and Ben and I will try to help you out as soon as possible.

Here are some issues where I discussion would be particularly fruitful: 

  • What are your thoughts about the karma system? Does an eigendemocracy based system seem reasonable to you? How would you implement the details? Ben and I will post our current thoughts on this in a separate post in the next two weeks, but we would be interested in people’s unprimed ideas.
  • What are your experiences with the site so far? Is anything glaringly missing, or are there any bugs you think I should definitely fix? 
  • Do you have any complaints or thoughts about how work on LessWrong 2.0 has been proceeding so far? Are there any worries or issues you have with the people working on it? 
  • What would make you personally use the new LessWrong? Is there any specific feature that would make you want to use it? For reference, here is our current feature roadmap for LW 2.0.
  • And most importantly, do you think that the LessWrong 2.0 project is doomed to failure for some reason? Is there anything important I missed, or something that I misunderstood about the existing critiques?
The closed beta can be found at

Ben, Vaniver, and I will be in the comments!

HOWTO: Screw Up The LessWrong Survey and Bring Great Shame To Your Family

24 ingres 08 October 2017 03:43AM

Let's talk about the LessWrong Survey.

First and foremost, if you took the survey and hit 'submit', your information was saved and you don't have to take it again.

Your data is safe, nobody took it or anything it's not like that. If you took the survey and hit the submit button, this post isn't for you.

For the rest of you, I'll put it plainly: I screwed up.

This LessWrong Survey had the lowest turnout since Scott's original survey in 2009. I'll admit I'm not entirely sure why that is, but I have a hunch and most of the footprints lead back to me. The causes I can finger seem to be the diaspora, poor software, poor advertising, and excessive length.

The Diaspora

As it stands, this years LessWrong survey got about 300 completed responses. This can be compared with the previous one in 2016 which got over 1600. I think one critical difference between this survey and the last was its name. Last year the survey focused on figuring out where the 'Diaspora' was and what venues had gotten users now that LessWrong was sort of the walking dead. It accomplished that well I think, and part of the reason why is I titled it the LessWrong Diaspora Survey. That magic word got far off venues to promote it even when I hadn't asked them to. The survey was posted by Scott Alexander, Ozy Frantz, and others to their respective blogs and pretty much everyone 'involved in LessWrong' to one degree or another felt like it was meant for them to take. By contrast, this survey was focused on LessWrong's recovery and revitalization, so I dropped the word Diaspora from it and this seemed to have caused a ton of confusion. Many people I interviewed to ask why they hadn't taken the survey flat out told me that even though they were sitting in a chatroom dedicated to SSC, and they'd read the sequences, the survey wasn't about them because they had no affiliation with LessWrong. Certainly that wasn't the intent I was trying to communicate.

Poor Software

When I first did the survey in 2016, taking over from Scott I faced a fairly simple problem: How do I want to host the survey? I could do it the way Scott had done it, using Google Forms as a survey engine, but this made me wary for a few reasons. One was that I didn't really have a Google account set up that I'd feel comfortable hosting the survey from, another was that I had been unimpressed with what I'd seen from the Google Forms software up to that point in terms of keeping data sanitized on entry. More importantly, it kind of bothered me that I'd be basically handing your data over to Google. This dataset includes a large number of personal questions that I'm not sure most people want Google to have definitive answers on. Moreover I figured: Why the heck do I need Google for this anyway? This is essentially just a webform backed by a datastore, i.e some of the simplest networking technology known to man in 2016. But I didn't want to write it myself, didn't need to write it myself this is the sort of thing there should be a dozen good self hosted solutions for.

There should be, but there's really only LimeSurvey. If I had to give this post an alternate title, it would be "LimeSurvey: An anti endorsement".

I could go on for pages about what's wrong with LimeSurvey, but it can probably be summed up as "the software is bloated and resists customization". It's slow, it uses slick graphics but fails to entirely deliver on functionality, its inner workings are kind of baroque, it's the sort of thing I probably should have rejected on principle and written my own. However at that time the survey was incredibly overdue, so I felt it would be better to just get out something expedient since everyone was already waiting for it anyway. And the thing is, in 2016 it went well. We got over 3000 responses including both partial and complete. So walking away from that victory and going into 2017, I didn't really think too hard about the choice to continue using it.

A couple of things changed between 2016 and our running the survey in 2017:

Hosting - My hosting provider, a single individual who sets up strong networking architectures in his basement, had gotten a lot busier since 2016 and wasn't immediately available to handle any issues. The 2016 survey had a number of birthing pains, and his dedicated attention was part of the reason why we were able to make it go at all. Since he wasn't here this time, I was more on my own in fixing things.

Myself - I had also gotten a lot busier since 2016. I didn't have nearly as much slack as I did the last time I did it. So I was sort of relying on having done the whole process in 2016 to insulate me from opening the thing up to a bunch of problems.

Both of these would prove disastrous, as when I started the survey this time it was slow, it had a variety of bugs and issues I had only limited time to fix, and the issues just kept coming, even more than in 2016 like it had decided now when I truly didn't have the energy to spare was when things should break down. These mostly weren't show stopping bugs though, they were minor annoyances. But every minor annoyance reduced turnout, and I was slowly bleeding through the pool of potential respondents by leaving them unfixed.

The straw that finally broke the camels back for me was when I woke up to find that this message was being shown to most users coming to take the survey:

Message Shown To Survey Respondents Telling Them Their Responses 'cannot be saved'.

"Your responses cannot be saved"? This error meant for when someone had messed up cookies was telling users a vicious lie: That the survey wasn't working right now and there was no point in them taking it.

Looking at this in horror and outrage, after encountering problem after problem mixed with low turnout, I finally pulled the plug.

Poor Advertising

As one email to me mentioned, the 2017 survey didn't even get promoted to the main section of the LessWrong website. This time there were no links from Scott Alexander, nor the myriad small stakeholders that made it work last time. I'm not blaming them or anything, but as a consequence many people who I interviewed to ask about why they hadn't taken the survey had not even heard it existed. Certainly this had to have been significantly responsible for reduced turnout compared to last time.

Excessive Length

Of all the things people complained about when I interviewed them on why they hadn't taken the survey, this was easily the most common response. "It's too long."

This year I made the mistake of moving back to a single page format. The problem with a single page format is that it makes it clear to respondents just how long the survey really is. It's simply too long to expect most people to complete it. And before I start getting suggestions for it in the comments, the problem isn't actually that it needs to be shortened, per se. The problem is that to investigate every question we might want to know about the community, it really needs to be broken into more than one survey. Especially when there are stakeholders involved who would like to see a particular section added to satisfy some questions they have.

Right now I'm exploring the possibility of setting up a site similar to yourmorals so that the survey can be effectively broken up and hosted in a way where users can sign in and take different portions of it at their leisure. Further gamification could be added to help make it a little more fun for people. Which leads into...

The Survey Is Too Much Work For One Person

What we need isn't a guardian of the survey, it's really more like a survey committee. I would be perfectly willing (and plan to) chair such a committee, but I frankly need help. Writing the survey, hosting it without flaws, theming it so that it looks nice, writing any new code or web things so that we can host it without bugs, comprehensively analyzing the thing, it's a damn lot of work to do it right and so far I've kind of been relying on the generosity of my friends for it. If there are other people who really care about the survey and my ability to do it, consider this my recruiting call for you to come and help. You can mail me here on LessWrong, post in the comments, or email me at If that's something you would be interested in I could really use the assistance.

What Now?

Honestly? I'm not sure. The way I see it my options look something like:

Call It A Day And Analyze What I've Got - N=300 is nothing to sneeze at, theoretically I could just call this whole thing a wash and move on to analysis.

Try And Perform An Emergency Migration - For example, I could try and set this up again on Google Forms. Having investigated that option, there's no 'import' button on Google forms so the survey would need to be reentered manually for all hundred-and-a-half questions.

Fix Some Of The Errors In LimeSurvey And Try Again On Different Hosting - I considered doing this too, but it seemed to me like the software was so clunky that there was simply no reasonable expectation this wouldn't happen again. LimeSurvey also has poor separation between being able to edit the survey and view the survey results, I couldn't delegate the work to someone else because that could theoretically violate users privacy.

These seem to me like the only things that are possible for this survey cycle, at any rate an extension of time would be required for another round. In the long run I would like to organize a project to write a new software from scratch that fixes these issues and gives us a site multiple stakeholders can submit surveys to which might be too niche to include in the current LessWrong Survey format.

I'm welcome to other suggestions in the comments, consider this my SOS.


LW 2.0 Open Beta starts 9/20

24 Vaniver 15 September 2017 02:57AM

Two years ago, I wrote Lesswrong 2.0. It’s been quite the adventure since then; I took up the mantle of organizing work to improve the site but was missing some of the core skills, and also never quite had the time to make it my top priority. Earlier this year, I talked with Oliver Habryka and he joined the project and has done the lion’s share of the work since then, with help along the way from Eric Rogstad, Harmanas Chopra, Ben Pace, Raymond Arnold, and myself. Dedicated staff has led to serious progress, and we can now see the light at the end of the tunnel.


So what’s next? We’ve been running the closed beta for some time at with an import of the old LW database, and are now happy enough with it to show it to you all. On 9/20, next Wednesday, we’ll turn on account creation, making it an open beta. (This will involve making a new password, as the passwords are stored hashed and we’ve changed the hashing function from the old site.) If you don't have an email address set for your account (see here), I recommend adding it by the end of the open beta so we can merge accounts. For the open beta, just use the Intercom button in the lower right corner if you have any trouble. 


Once the open beta concludes, we’ll have a vote of veteran users (over 1k karma as of yesterday) on whether to change the code at over to the new design or not. It seems important to look into the dark and have an escape valve in case this is the wrong direction for LW. If the vote goes through, we’ll import the new LW activity since the previous import to the new servers, merging the two, and point the url to the new servers. If it doesn’t, we’ll likely turn LW into an archive.


Oliver Habryka will be posting shortly with his views on LW and more details on our plans for how LW 2.0 will further intellectual progress in the community.

2017 LessWrong Survey

21 ingres 13 September 2017 06:26AM

The 2017 LessWrong Survey is here! This year we're interested in community response to the LessWrong 2.0 initiative. I've also gone through and fixed as many bugs as I could find reported on the last survey, and reintroduced items that were missing from the 2016 edition. Furthermore new items have been introduced in multiple sections and some cut in others to make room. You can now export your survey results after finishing by choosing the 'print my results' option on the page displayed after submission. The survey will run from today until the 15th of October.

You can take the survey below, thanks for your time. (It's back in single page format, please allow some seconds for it to load):

Click here to take the survey

Bi-weekly Rational Feed

20 deluks917 08 August 2017 01:56PM

===Highly Recommended Articles:

Skills Most Employable by 80,000 Hours - Metrics: Satisfaction, risk of automation, and breadth of applicability. Leadership and social skills will gain the most in value. The least valuable skills involve manual labor. Tech skills may not be the most employable but they are straightforward to improve at. The most valuable skills are the hardest to automate and useful in the most situations. Data showing a large oversupply of some tech skills, though others are in high demand. A chart of which college majors add the most income.

Something Was Wrong by Zvi Moshowitz - Zvi visits a 'stepford pre-school'. He can't shake the feeling that something is wrong. He decides not to send his son to the place where kid's souls go to die.

Ems Evolve by Bayesian Investor - Will the future we dominated by entities that lack properties we consider important (such as 'have fun' or even 'sentient'). Will agents lacking X-value outcompete other agents. What counter-measures could society take and how effective would they be.

Housing Price Bubble Revisited by Tyler Cowen - "Over the entire 20th century real home prices averaged an index value of about 110 (and were quite close to this value over the the entire 1950-1997 period). Over the entire 20th century, housing prices never once roce above 131, the 1989 peak. But beginning around 2000 house prices seemed to reach for an entirely new equilibrium. In fact, even given the financial crisis, prices since 2000 fell below the 20th century peak for only a few months in late 2011. Real prices today are now back to 2004 levels and rising. As I predicted in 2008, prices never returned to their long-run 20th century levels."

Tyler Cowen On Stubborn Attachments by EconTalk - "Cowen argues that economic growth--properly defined--is the moral key to maintaining civilization and promoting human well-being. Along the way, the conversation also deals with inequality, environmental issues, and education"


Contra Grant On Exaggerated Differences by Scott Alexander - "Hyde found moderate or large gender differences in aggressiveness, horniness, language abilities, mechanical abilities, visuospatial skills, mechanical ability, tendermindness, assertiveness, comfort with body, various physical abilities, and computer skills. Perhaps some peeople might think that finding moderate-to-large-differences in mechanical abilities, computer skills, etc supports the idea that gender differences might play a role in gender balance in the tech industry. But because Hyde’s meta-analysis drowns all of this out with stuff about smiling-when-not-observed, Grant is able to make it sound like this proves his point. It’s actually worse than this, because Grant misreports the study findings in various ways."

Links: On The Site Of The Angels by Scott Alexander - Standard SSC links post.

Mildly Condescending Advice by SlateStarScratchpad - Ten mildly condescending but useful pieces of advice Scott recommends.

Communism by SlateStarScratchpad - Scott thinks he would have been a communist in 1910.

What Are The Median Psychiatrists Scores On The by SlateStarScratchpad - Psychiatrists are very mentally well adjusted on average. "I think you get way more illness in the therapists, counselors, etc, especially the ones that are kind of low-status and don’t require a lot of training." Doctor's recovery rates from alcoholism are very good.

Why Not More Excitement About Prediction Aggregation by Scott Alexander - Prediction markets and aggregation methods work. Superforecasters proved some groups can consistently make good predictions. Why isn't there more interest? Wouldn't investors pay for predictions? Do theories about signaling and prestige explain the situation?

Where The Falling Einstein Meets The Rising Mouse by Scott Alexander - Eliezer/Scott's model of intelligence suggests that the gap between 'village idiot' and Einstein is tiny relative to the difference between 'village idiot' and a chimp. This suggests that once AI reaches human levels it will almost immediately pass the best human. This happened in Go. But in other fields progress was gradual throughout the approximately human level skill range. Scott looks at possible explanations.

Stem vs The Humanities by SlateStarScratchpad - A long and intelligent thread about "STEM" vs "The Humanities". What are the natural categories? Should we consider math part of the humanities? Should we groups careful humanities scholars with careful STEM scholars? So-called-autistics. Other topics.

Gender Imbalances Are Mostly Not Due To Offensive Attitudes by Scott Alexander - Men and women massively differ in terms of interest in things vs people. Libertarians are about 5% women. r/MRAand the gamergate subreddit have twice this percentage. Trump voters are close to gender parity and the Catholic Church has more women than men. Why this matters.

Is It Possible To Have Coherent Principles Around Free Speech Norms by Scott Alexander - The doctrine of the preferred first speaker. Separating having an opinion, signaling a propensity, and committing a speech act. Self selected communities. "Don’t try to destroy people in order to enforce social norms that only exist in your head"

Book Review Raise A Genius by Scott Alexander - Scott quote-mines Polgar's book on raising genius. Many of the quotes are concerned with the importance of instilling a love of learning in children. Polgar gives some detail on how to do this but not as much as Scott hoped. Summary: "Get those four things right – early start, single-subject focus, 1:1 home schooling, and a great parent/teacher – and the rest is just common-sense advice."

Against Signal Boosting As Doxxing by Scott Alexander - Free speech did not come into existence ex nihilo when the First Amendment was ratified. People need to be free from mobs as well as kings.

Open Djed by Scott Alexander - Bi-Weekly public open thread. Meetup Tab. Updates on rationalist houses in Berkeley. Selected comments on Griggs. A comment on Democratic strategy and Georgia.

Why Is Clozapine So Great by Scott Alexander - Clozapine is a very effect anti-psychotic but it has large serious side effects. NDMA agonists and a proposed mechanism for Clozapine. Maybe you could just give patients a normal anti-psychotic plus glycine.

Djoser Joseph Osiris by Scott Alexander - "The other day a few people (including Ben Hoffman of Compass Rose) tried to convince me that Pharaoh Djoser was the inspiration for the god Osiris and the Biblical Joseph. The short summary is that the connection between Djoser and Osiris is probably meaningless, but there’s a very small chance there might be some tiny distant scrap of a connection to Joseph."

Don't Blame Griggs by Scott Alexander - Griggs vs. Duke Power Co is commonly cited as making it prohibitively hard for companies to use intelligence tests in hiring. Scott argues this doesn't explain the rise in credentialism. You can still ask about SAT scores. Fields with easily available test scores (LSAT, MCAT) are still credentialist. Other countries lack equivalents of Griggs vs Duke.

Highlights From The Comment Thread On Meritocracy by Scott Alexander - Real merit vs credentials. Which merits do we reward? Meritocracy causes high ability people to concentrate into one class. Just rid rid of ruler and structural divisions between people. Scott finds the later idea utopian. " The most salient alternative to meritocracy isn’t perfect equality, it’s cronyism."


Why So Few Women In Cs: The Google Memo Is Right by Artir (Nintil) - Sampler: Lots of data and graphs covering multiple countries. "Occupational segregation by gender is stronger in egalitarian countries. This is a fatal blow to the sexism theory." In the 1980s demand for the CS major far outstripped capacity. This lead to severe limits on who could major in CS. These limits occurred at the same time female enrollment percentage dropped.

Double Crux Web App by mindlevelup - Double Crux is a rationalist technique for resolving and understanding disagreements. It involves identifying facts/statements, called cruxes, that would cause you to change your mind if you changed your mind about the crux. The author built software to facilitate double crux during the Google CSSI 3 week web dev camp. Links to the site and an explanation of Double Crux.

Compare Institutions To Institutions Not To Perfection by Robin Hanson - Hanson responds to criticisms of prediction markets. Short term accuracy is always easier to incentivize. Its always easier to find surface as opposed to deep connections.

Thank You For Listening by Ben Hoffman (Compass Rose) - Zvi's post above starts with a reference to a previous Ben Quo post. If you have hurt your child via school you aren't the enemy. Society taught you that you were helping. If you are still sending your child to a harmful system you aren't the enemy either, you are doing what you think is best.

Something Was Wrong by Zvi Moshowitz - Zvi visits a 'stepford pre-school'. He can't shake the feeling that something is wrong. He decides not to send his son to the place where kid's souls go to die.

Inscrutable Ideas by Gordon (Map and Territory) - The author describes 'holonic' thinking and why its hard to explain. Postmodernism as a flawed holonic tradition. Buddhism as a better holonic tradition. Fundamental incompatibility with system-relationship epistemology.

Body Pleasure by Sarah Perry (ribbonfarm) - "As non-human intelligences get more sophisticated, it may be the case that human work remains extremely important; however, it may also be that humans are faced with increasing leisure. If that is the case, the critical problem facing humanity will be how to enjoy ourselves. If that seems silly, consider your favorite dystopian images of the future: only humans who understand how to enjoy themselves can demand living conditions in which they are able to do so."

Erisology Of Self And Will The Need And The Reasons by Everything Studies - "Here in part 6 I discuss the reasons why the traditional view persists when prescientific thinking on other topics often doesn’t."

Confidence And Patience Dont Feel Like Anything In Particular by Kaj Sotala - Being confident doesn't feel like anything. 'Feeling confident' is really just the lack of feeling unconfident.

Foom Justifies Ai Risk Efforts Now by Robin Hanson - Organizations and corporations are already much smarter and more powerful than individuals, yet they remain mostly under control. Despite setbacks (Wars, revolutions, famines) the organization ecosystem is mostly functional. The only reason to be preemptively worried about AI is if AI takeoff will be very fast.

Skills Most Employable by 80,000 Hours - Metrics: Satisfaction, risk of automation, and breadth of applicability. Leadership and social skills will gain the most in value. The least valuable skills involve manual labor. Tech skills may not be the most employable but they are straightforward to improve at. The most valuable skills are the hardest to automate and useful in the most situations. Data showing a large oversupply of some tech skills, though others are in high demand. A chart of which college majors add the most income.

A Tactics by protokol2020 - Why its very hard to argue against the scientific consensus on fields such as Relativity or Quantum Mechanics. The Earth's temperature was hotter when it rotated fast, despite a fainter sun. Many physicists failed to grasp this fact. What does that imply?

Hedonic Model by Jeff Kaufman - "Happiness is having how things are going be closer to how you think things could be going." Some interesting implications including that both inequality and social mobility are bad.

Link Blog: Broadcom Broadpwn Gender Signal by Name and Nature - Links: History of Atheism. Evolution of Trust. Graphics depicting the Fast Fourier Transform. Remotely Compromising Android and iOS via a Bug in Broadcom’s Wi-Fi Chipsets.

Lying On The Ground by mindlevelup - "A rambling look at how rewards, distractions, and attention interact. Starts with the idea of lying on the ground as an interesting break-time activity and goes from there to talk about Saturation and Feeling, two concepts that I’ve been thinking about lately."

Ems Evolve by Bayesian Investor - Will the future we dominated by entities that lack properties we consider important (such as 'have fun' or even 'sentient'). Will agents lacking X-value outcompete other agents. What counter-measures could society take and how effective would they be.

Models Of Human Relationships Tools To Understand People by Elo (BearLamp) - Brief Model descriptions: Crucial Conversations. 4 Difficult Conversations, 4 Behaviors that kill relationships, How to Win Friends and Influence People (Detailed review), Non-Judgmental conversations, Emotional Intelligence. Circling, The Game (PUA), Apologies, Emotional Labor and others.

Inefficiencies In The Social Value Market by Julia Galef - Add liquidity where needed. Solve coordination problems. Pool risks. Provide resource allocation information. Make biases work for you. Remove rent-seeking. Reduce transaction costs.

Erisology Of Self And Will Campbellian Thinking In The Wild by Everything Studies - "In this section I’ll show some examples of casual conversation revealing Campbellian ideas. Comment threads attached to online newspaper articles are excellent sources of such casual conversation. Written down in a neat and accessible form, their existence makes it practical to do this kind of research for the first time."

The Future: Near Zero Growth Rates by The Foundational Research Institute - Moore's law cannot possibly go one for more than ~400 years, we will hit physical limits to computation. At 2.3% growth in energy use we would need to coat the Earth in Solar panels to get enough solar energy in only 400 years. If we captured all the energy from the sun we would run out in 1350 years. The universe can only support so much economic activity. We will in a very unusual part of humanity's timeline in terms of growth rates.

How I Found Fixed The Root Problem Behind My Depression And Anxiety After 20 Years by Kaj Sotala- Finding the root cause: self-concept. How to cultivate lovable self-concepts (ex: bravery). Consider memories where you lived up to the concept of being brave. Also consider cases where you failed. Integrating the positives and negatives into a healthy whole. Positive benefits the author experienced: professional success, emotional landscape improvement, negative emotions disappeared. Expected relationship changes. Lots of personal history details throughout.

Taking Integrity Literally by Ben Hoffman (Compass Rose) - Defending Kant. Fight the murderer or shut the door but don't become the sort of person who considers lying. Honesty is optimal in healthy environments. Thoughts on unhealthy environments. How Ben started to become honest about how late he would be. Not lying to yourself or others.

People Dont Have Beliefs Anymore Than They Have by Bound_up (lesswrong) - Actions are not deduced form goals. Beliefs are not deduced from models of the world. Maybe nerds have real beliefs but most people do not. Less nerdy people will probably interpret in your stated beliefs as social moves and will respond in turn.

Complexity Is Bad by Zvi Moshowitz - People can only think about ~3 things and store ~7 pieces of information in working memory. People will simplify in unexpected ways or fail to engage. Some concepts that help you manage complexity (ex: Resonance, Chunking). A link to the MtG head of R&D's podcast about why complexity is a cost.

Write Down Your Process by Zvi Moshowitz - Writing down your thought process helps you improve. Magic R&D's openness. Zvi's success as a MtG player and writer.


July 2017 Newsletter by The MIRI Blog - News and Links: Open AI, Deepmind, AI Impacts, EA global, 80K hours, etc

Yudkowsky And Miri by Jeff Kaufman - Elizier once wrote an extremely embarrassing article called 'So you want to be a Seed AI Programmer'. A ML researcher showed it to Jeff Kaufman and said it implied Elizier was a crank. Elizier wrote it in 2003, when he was 24. What does this imply about MIRI?


Medical Research Cancer Is Hugely Overfunded by Sanjay (EA forum) - Chart of disease burden vs research share. Six reasons you might disagree with the conclusion including cause tractability and methodology.

Blood Donation Generally Not That Effective On by Grue_Slinky (EA forum) - Having a supply of blood is very important. However the marginal value of blood donation is too low to recommend it as an efficient intervention.

How We Banned Fur In Berkeley by jayquigly (EA forum) - Fur sales banned. Main strategies: cultivating relationships with sympathetic council members, utilizing a proven template for the bill. Background. Strategy Details. Advice

Links: Our Main Goal is to Learn by GiveDirectly - Eight media links on Give Directly, Basic Income, Cash Transfers and Development Aid.

Funding Constraints For Ea Orgs by Jeff Kaufman - Value of direct work vs donation. Jeff argues EA organizations could make use of more resources. For example EA-Global could hire non-EA professional conference organizers.

===Politics and Economics:

Rise And Fall Of Rawlsianism by Artir (Nintil) - "I will introduce street Rawlsianism, a simplified version of Rawls’s Theory of Justice to get an idea of what this is all about. Then, I will explain how that came to be, including some extra details about Rawls’s justification for his theory. This story itself, the development of Rawls’s own philosophical views, is a good enough criticism of his original theory, but I will add at the end what I think are the strongest critiques I know."

Hazlett's Political Spectrum by Robin Hanson - "Not only would everything have been much better without FCC regulation, it actually was much better before the FCC! Herbert Hoover, who was head of the US Commerce Department at the time, broke the spectrum in order to then “save” it, a move that probably helped him rise to the presidency."

Another Point Of View by Simon Penner (Status 451) - The author was raised working class in semi-rural Canada and moved to Silicon valley. He experienced a ton of culture shock and significant cultural discrimination. This causes him to have less sympathy for people who quit software because of relatively minor pressure saying they don't fit in. The author overcame this stuff and so should other people.

Rapid Onset Gender Dysphoria Is Bad Science by Ozy (Thing of Things) - Ozy cites two commonly misinterpreted but good studies about suicide rates among transgender individuals. Ozy then discusses a very shoddy study about people rapidly becoming trans after meeting trans friends. However the study got its information by asking the parents of trans teens and young adults. Ozy explains how and why young adults hide much of their feelings from their parents, especially if they are neurodiverse.

Housing Price Bubble Revisited by Tyler Cowen - "Over the entire 20th century real home prices averaged an index value of about 110 (and were quite close to this value over the the entire 1950-1997 period). Over the entire 20th century, housing prices never once roce above 131, the 1989 peak. But beginning around 2000 house prices seemed to reach for an entirely new equilibrium. In fact, even given the financial crisis, prices since 2000 fell below the 20th century peak for only a few months in late 2011. Real prices today are now back to 2004 levels and rising. As I predicted in 2008, prices never returned to their long-run 20th century levels."

Reinventing The Wheel Of Fortune by sam[]zdat - Two definitions of democracy. A key idea: "Lasch is an external commentary using this rough model. At some point, the combined apparatus of American culture (the state, capital, media, political agitation) tried to make things “better”. To better its citizens required new social controls (paternalism). The taylorism employed makes things more focused on image, and this results in a more warlike society. Happened with “authenticity” last time and also [everything below]. To deal with this invasion, society turns to narcissistic defenses. Narcissism is self-centered, but it’s an expression of dependence on others, and specifically on the others’ validation of the narcissist’s image."


Prime Towers Problem by protokol2020 - Prime height towers. From which tower is the most tower tops visible.

The Unyoga Manifesto by SquirrelInHell - Yoga has a sort of 'competitive' ethos baked in. There is alot of pressure to do the postures 'correctly'. Instead you should listen to your body and follow the natural incentive gradients that lead to maintaining one's body well. Four practical pieces of advice.

Clojure The Perfect Language To Expand Your Brain by Eli Bendersky - Clojure will almost certainly change how you think about programming. Clojure is a fully modern and useable Lisp. The designers of Clojure are extremely pragmatic, building upon decades of industry experience. Sequences and laziness for powerful in-language data processing. Right approach to OOP. Built-in support for concurrency and parallelism.

A Physics Problem Once Again by protokol2020 - Discussion of n-dimensional mating. Approximate the sum of all gravitational forces between pairs of atoms inside the earth.

Meta Contrarian Typography by Tom Bartleby - The author is a self-described meta-contrarian. Supporting two spaces after a period. The three reasons for single spaces and why they don't hold up. Double spaces makes writing easier to skim, periods are over-worked in English.

I Cant Be Your Hero Im Too Busy Being Super by Jim Stone (ribbonfarm) - "But people don’t generally take on the burdens of inauthenticity without good reason. Often it’s because they want to occupy social roles that allow them to get their physical and psychological needs met, and other people won’t let them play those roles unless they are the right kind of person. Sometimes people put on masks simply to secure the role of “community member” or “citizen” or “human being”."


Physical Training Dating Strategies And Stories From The Early Days by Tim Feriss - Tim answers viewer questions. Physical training, interview prep, the art of networking, education reform, dream guests on the show.

Living With Violence by Waking Up with Sam Harris - "Gavin de Becker about the primacy of human intuition in the prediction and prevention of violence."

Amanda Askell On Pascals Wager And Other Low Risks Wi by Rational Speaking - Pascal's Wager: It's rational to believe in God, because if you're wrong it's no big deal, but if you're right then the payoff is huge. Amanda Askell argues that it's much trickier to rebut Pascal's Wager than most people think. Handling low probability but very high impact possibilities: should you round them down to zero? Does it matter how measurable the risk is? And should you take into account the chance you're being scammed?"

Tyler Cowen On Stubborn Attachments by EconTalk - "Cowen argues that economic growth--properly defined--is the moral key to maintaining civilization and promoting human well-being. Along the way, the conversation also deals with inequality, environmental issues, and education"

40 Making Humans Legible by The Bayesian Conspiracy - Seeing like a State. Scott and Sam[]zdat's posts. Green Revolution. Age of Em. Chemtrails and invasive species. Friendship is Optimal.

Dave Rubin by Tyler Cowen - "Comedy and political correctness, which jokes should not be told, the economics of comedy, comedy in Israel and Saudi Arabia, comedy on campus, George Carlin, and the most underrated Star Wars installment"

Yascha Mounk by The Ezra Klein Show - Trump's illiberalism is catalyzed by his failures. Recently Trump has been more illiberal. Support for Trump remains at around 40 percent. What does this imply about the risk of an illiberal Trump successor with more political competence.

Alex Guarnasche by EconTalk - Food network star. "What it's like to run a restaurant, the challenges of a career in cooking, her favorite dishes, her least favorite dishes, and what she cooked to beat Bobby Flay."

On Becoming A Better Person by Waking Up with Sam Harris - "David Brooks. His book The Road to Character, the importance of words like “sin” and "virtue," self-esteem vs. self-overcoming, the significance of keeping promises, honesty, President Trump."

Julia Galef On How To Argue Better And Change Your Mind More by The Ezra Klein Show - Thinking more clearly and arguing better, Ezra's concerns that the traditional paths toward a better discourse. Signaling is turtles all the way down, motivated reasoning, probabilistic debating, which identities help us find truth, making online arguments less terrible. Julia heavily emphasizes the importance of good epistemic communities. Being too charitable can produce wrong predictions. Seeing like a State.

Ten small life improvements

16 paulfchristiano 20 August 2017 07:09PM

I've accumulated a lot of small applications and items that make my life incrementally better. Most of these ultimately came from someone else's recommendation, so I thought I'd pay it forward by posting ten of my favorite small improvements.

(I've given credit where I remember who introduced the item into my life. Obviously the biggest part of the credit goes to the creator.)

Video speed

Video Speed Controller lets you speed up HTML 5 video; it gives a nicer interface than the YouTube speed adjustment and works for most videos displayed in a browser (including e.g. netflix/amazon).

(Credit: Stephanie Zolayvar?)


Spectacle on OSX provides keyboard shortcuts to snap windows to any half or third of the screen (or full screen).

Pinned tabs + tab wrangler

I use tab wrangler to automatically close tabs (and save a bookmark) after 10m. I keep gmail and vimflowy pinned so that they don't close. For me, closing tabs after 10m is usually the right behavior.

Aggressive AdBlock

I use AdBlock for anything that grabs attention even if isn't an ad. I usually block "related content," "next stories," the whole youtube sidebar, everything on Medium other than the article, the gmail sidebar, most comment sections, etc. Similarly, I use kill news feed to block my Facebook feed.

Avoiding email inbox

I often need to write or look up emails during the day, which would sometimes lead me to read/respond to new emails and switch contexts. I've mostly fixed the problem by leaving gmail open to my list of starred emails rather than my inbox, ad-blocked the "Inbox (X)" notification, and pin gmail so that I can't see the "Inbox (X)" title.

Christmas lights

I prefer the soft light from christmas lights to white overhead lights or even softer lamps. My favorite are multicolored lights, though soft white lights also seem OK.

(Credit: Ben Hoffman)


Karabiner remaps keys in a very flexible way. (Unfortunately, it only works on OSX pre-Sierra. Would be very interested if there is any similarly flexible software that )

Some changes have helped me a lot:

  • While holding s: hjkl move the cursor. (Turn on "Simple Vi Mode v2") I find this way more convenient than the arrow keys.
  • While holding d: hjkl move the mouse. (Turn on "Mouse Keys Mode v2") I find this slightly more convenient than a mouse most of the time, but the big win is that I can use my computer when a bluetooth mouse disconnects.
  • Other stuff while holding s: (add this gist to your private.xml):
    • While holding s: u/o move to the previous and next word, n is backspace. 
    • While holding s+f: key repeat is 10x faster.
    • While holding s+a: hold shift (so cursor selects whatever it moves over, e.g. I can quickly select last ten words by holding a+s+f and then holding u for 1 second).

I'd definitely pay > a minute a day for these changes.


I find split+tented keyboards much nicer than usual keyboards. I use a Kinesis Freestyle 2 with this to prop it up. I put my touchpad on a raised platform between the keyboard halves. Alternatively, you might prefer the wire cutter's recommendations.

(Credit: Emerald Yang)


Vimflowy is similar to Workflowy, with a few changes: it lets you "clone" bullets so they appear in multiple places in your document, has marks that you can jump to easily, and has much more flexible motions / macros / etc. I find all of these very helpful. The biggest downside for most people is probably modal editing (keystrokes issue commands rather than inserting text).

The biggest value add for me is the time tracking plugin. I use vimflowy essentially constantly, so this gives me extremely fine-grained time tracking for free.

Running locally (download from github) lets you use vimflowy offline, and using the SQLite backend scales to very large documents (larger than workflowy can handle).

(Credit: Jeff Wu and Zachary Vance.)

ClipMenu [hard to get?]

Keeps a buffer of the last 20 things you've copied, so that you can paste any one of them. Source for OSX is on github here, I'm not sure if it can be easily compiled/installed (binaries used to be available). Would be curious if anyone knows a good alternative or tries to compile it.

(Credit: Jeff Wu.)

[Link] How I [Kaj] found & fixed the root problem behind my depression and anxiety after 20+ years

16 Kaj_Sotala 26 July 2017 12:56PM

Models of human relationships - tools to understand people

15 Elo 29 July 2017 03:31AM

This post will not teach you the models here.  This post is a summary of the models that I carry in my head.  I have written most of the descriptions without looking them up (See Feynman notebook method).  If you have read a book on every one of these points they will make sense, as if you were shaking hands with an old acquaintance.  If you are seeing them for the first time, they won't make very much sense or they will feel like a surface trivial truth.

I can't make you read all the books but maybe I can offer you that the answer to social problems is surprisingly simple.  After reading enough books you start to see the overlap and realise they often are trying to talk about the same thing.  (i.e. NVC + Gottman go together well).

In fact if you were several independent dragon hunters trying to model an invisible beast and all of various people's homemade sensors kept going "ping" at similar events you would probably start to agree you were chasing the same monster.  Models should start to agree when they are talking about the same thing.  The variety of models should make it easier for different minds to connect to different parts of the answer.

All models are wrong, some models are useful.  Try to look at where the models converge.  That's where I find the truth.

1. The book Crucial Confrontations - Kerry Patterson
(without explaining how) If you can navigate to a place of safety in a conversation you can say pretty much anything.  Which is not to say "here is how to be a jerk" but if you know something is going to come across negative you can first make sure to be in a positive/agreeable/supportive conversation before raising the hard thing.

In the middle of a yelling match is maybe not the best time to bring up something that has bugged you for years.  However a few sentences about growth mindset, supporting people being a better person and trying to help (and getting a feel that the person is ready to hear the thing) and you could tell anyone they are a lazy bum who needs to shape up or ship out.

The conversation needs to be safe.  For example - "I want to help you as a person and I know how hard it can be to get feedback from other people and I want to make you into a better person.  I have an idea for how you might like to improve.  Before I tell you I want to reassure you that even though this might come across abrasive I want to help you grow and be better in the future..."

(some people will be easier than others to navigate a safe conversation and that's where there are no hard and fast rules for how to do this.  Go with your gut)

The crux of this model is "have a model of the other person" [15]

2. The partner book "Difficult conversations"

There are 4 types of difficult conversations around communicating a decision:
a. Consultation (Bob asks Alice for ideas for the decision he is going to make on his own)
b. Collaboration (Bob and Alice make a decision together)
c. Declaration (Bob tells alice the decision he has made)
d. Delegation (Bob tells alice to make the decision)

As someone's boss you may sometimes have to pass on bad news in the form of a declaration.  It's up to you which conversation this is going to be but being clear about what conversation this is will be helpful to a person to understand their place in responding or interacting with you.  It becomes difficult where there is a misunderstanding about what is going on.

It's also important when you are on the receiving end to be on the same page about what conversation this is.  (you don't want to be negotiating in a collaborative manner when they are trying to give you a declaration of their decision, and the same when you are leading the conversation).

Among other details in the book.

3. Getting the 3rd story.

linking back to -
(from one of those books [1] or [2])

Bob knows what happened from his perspective and Alice knows her version of events.  Where there is a disagreement of what follows from different versions of events it is possible to construct a 3rd person story.  This may be hard to do when you are involved and an actual 3rd person can help but is not crucial in constructing the story.  If you can step outside of your own story and construct a 3rd version together this can resolve misunderstandings.
Something like; "I thought you said we should meet here, even though I said I wanted ice-cream, you thought that meant we should meet at the ice-cream place next door and we each waited 30mins for the other one to turn up to where we were.".  By constructing a 3rd story it's possible that no one was at fault.  It's also possible that it can become clear what went wrong and how to learn from that or what can be done differently.

(cue business management After-Action-Review activities {what did we do well, what could we have done better, what would we do differently}, now SWOT)

4. The Gottman Institute research (and book)

The 4 horsemen of divorce (but just because that's what the research is about doesn't mean we can't apply it elsewhere) (yes Gottman is limited in value because of bad use of statistics we can't be sure the models are accurate, I still find it's a good model at explaining things).

Don't do these things.  When you see these things, recognise them for what they are and don't engage with them.  If necessary acknowledge people are feeling certain angry feelings and let them get them out (not everyone can efficiently drop how they are feeling and get on with talking about it, especially not without practice).

Each one has an antidote, usually in the form of an attitude or strategy that can leave you thinking about the same thing differently and relating to it differently.

I. Criticism
I would rename to "inherent criticism".  Comes in the form of an inherent descriptor like, "you are a lazy person", "you always run late".  "you are the type of person who forgets my birthday"[see 5].  Try to replace inherent criticism with *[6] concrete descriptions of actions.

To counter this - try descriptions like [6a]:  "I can see you are sitting on the couch right now and I would like you to offer help when you can see me cleaning".  "yesterday I saw you try to do a few extra tasks and that caused us to run late", "you forgot my birthday last year".

The important thing about the change here is that an inherent label comes in the form of an unchangeable belief.  It's equivalent to saying, "you are a tall person".  It's fixed in time, space and attitude.  You don't want to give someone a fixed negative trait.  Not in your head and especially not out of your head either to that person or to anyone else.  You set someone up for failure if you do.  As soon as someone is "the lazy one" you give them the ticket to "always be lazy" and if they are half smart they will probably take it.  Besides - you don't change people's actions by using criticism.  You maybe relieve some frustration but then you have created some open frustration and the problem still exists.

II. Defensiveness
Probably easiest to understand by the description of reactive defensiveness.  It usually comes as a reaction to an accusation.  If two people are yelling, chances are neither is listening.  In response to "you are always making us run late", a defensive reaction would be, "I make us run late because you always stress me out".

It does two things:
1. claim to not be responsible
2. make a second accusation (can be irrelevant to the subject at hand).

First of all if you are bringing up several problems at once you are going to confuse matters.  Try to deal with one problem at a time.  It doesn't really matter which so long as you are not yelling about being late while they are yelling about you forgetting the laundry. (and so long as you deal with all the problems)

The second part is that you can't shift blame.  Absorbing some blame does not make you a bad person.  Nor does it make you inherently terrible.  You can have both done a wrong thing and not be a bad person.  After all you had your reasons for doing what you did.

The antidote to defensiveness is to acknowledge [6] what they have said and move forward without reacting.

III. Contempt
This is about an internal state as much as an external state.  Contempt is about the story we tell ourselves about the other person (see NVC) and is a state of negative intent.  I hold you contemptuously.  For example, "a good person would not run late", "if you were smarter you would just...", "I work so hard on this relationship and you just...", Some examples of displays of contempt include when a person uses sarcasm, cynicism, name-calling, eye-rolling, sneering, mockery, and hostile humour [see 7 - emotional intelligence about physiological events].  This overlaps with Inherent criticism and makes more sense with [6 NVC].
Contempt has two antidotes, Teacher mindset and curiosity.  Teacher mindset can change an attitude of, "He should know what he did wrong" to, "I need to explain to him how to do it right".  Curiosity [See NVC, also [3] the 3rd story] can take you to a place of trying to understand what is going on and take you away from the place of the stories we tell ourselves.[10]

IV. Stonewalling
This is a physiological state of going silent.  It is used when you are being lectured (for example) and you go silent, possibly start thinking about everything else while you wait for someone to finish.  It's like holding your breath when you go underwater, waiting for it to pass.  If you are doing this what you need to do is take a break from whatever is going on and do something different, for example go for a walk and calm down.
There was a classic joke, they asked a 110 year old why he lived so long and he said, every time I got into an argument with my wife I used to go for a walk.  I went on a lot of walks in my life.
Because this is a physiological state it's so easy to fix so long as you remember to pay attention to your internal state [see NVC what is most alive in you, and 11. what does that look like in practice]

5. How to win friends and influence people

I always recommend this book to people starting the journey because it's a great place to start.  These days I have better models but when I didn't know anything this was a place to begin.  Most of my models are now more complicated applications of the ideas initially presented.  You still need weak models before replacing them with more complicated ones which are more accurate.
The principles and (in brackets) what has superseded them for me:

1. Don't criticize, condemn or complain. (There are places and methods to do this.  Criticism can be done as [1] from a place of safety or in [4] from a teacher/mentor/growth mindset.  Definitely don't do it from a place of criticism.  Condemnation is more about [10] and is an inherent trait.  Progress doesn't usually happen when we use inherent traits, From Saul Alinsky's rules for radicals - don't complain unless you have the right answer - "I have a problem and you have to figure out how to fix it for me" is not a good way to get your problem solved.)
2. Give honest, sincere appreciation. (so long as you are doing this out of the goodness of your heart good.  If you are using it for manipulation you can just not bother.  NVC supersedes this.  By keeping track of what is most alive in you, you can do better than this)
3. Arouse in the other person an eager want. (Work out what people want, work out how to get both your needs met - superceded by NVC.)
4. Become genuinely interested in other people. (depends what for.  Don't bother if you don't want to.  That would not be genuine.  You need to find the genuine interest inside yourself first.)
5. Smile. (um.  Hard to disagree with but a default smiling state is a good one to cultivate - from [7] physiological states are linked two ways.  Smiling will make you happy just as being happy will make you smile)
6. Remember that a person's name is to that person the most important sound in any language. (I don't know about most important but I would say that anyone can remember names with practice.

7. Be a good listener. Encourage others to talk about themselves. (NVC - pay attention to what is most alive in you when you do. Make sure you know about the spectrum of )
8. Talk in terms of the other person's interest. (Sure why not.  Sales are a lot easier when you are selling what people want. See [15] and NVC to supersede how and why this works)
9. Make the other person feel important - and do so sincerely. (I guess?  I don't do this actively.)
10 The only way to get the best of an argument is to avoid it. ([9] if you are in an argument something already went wrong)
11. Show respect for the other person's opinions. Never say, "You're wrong." (NVC, instead of saying no, say what gets in the way.  "here is evidence that says otherwise" can be better than "durr WRONGGG" but I have seen people use "you are wrong" perfectly fine.)
12. If you are wrong, admit it quickly and emphatically. (hard to disagree with, but holding onto grudges and guity things is not useful.  [4] gottman talks about defensiveness, avoid defensiveness and acknowledge the fact that someone feels you are at fault first.  It will satisfy the psychological need arising in an offended person [14])
13. Begin in a friendly way. (as opposed to what?  Sure I guess.)
14. Get the other person saying, "Yes, yes" immediately. (Yes ladders are important and valuable.  You see bits of this creeping into Gottman [4], NVC [6], The game [13] and other practices but no one as yet explains it as well as I would like.  The game probably has the best commentary on it, short of business books that escape my memory right now)
15. Let the other person do a great deal of the talking. (not really important who talks so long as you are on the same page and in agreement.  If you want someone else to do the emotional labour [15] for you, then you can let them.  If you want to do it for them you can.  Implications of EL are not yet clear to me in full.  Some places it will be good to do EL for people, other places they need to do it for themselves to feel ownership of the problems and solutions)
16. Let the other person feel that the idea is his or hers. (sure I guess.  A good idea is it's own champion.  Ideas that are obviously better will win out.  You can't make a turd beat a diamond but you can employ tricks to polish certain diamonds over others.  This technique is battling over little bits.  can be useful but I would not rely on it alone.)
17. Try honestly to see things from the other person's point of view. (NVC [6] and EL [15] should help do that better.  Imagining that you are that person in a way that is hard to impart in words because it's about having the experience of being that other person (see and not "just thinking about it". needs a longer description and is an effective technique.)
18. Be sympathetic with the other person's ideas and desires. (NVC supercedes.  Everyone has basic feelings and needs that you can understand, like the need for safety)
19. Appeal to the nobler motives. (giving people a reputation to live up to is a valuable technique that I would say only works for qualified people - but does not work so well if you put pressure on people who are less skilled.  Probably relates to the things going through our head at the time - see also book - the inner game of tennis, NVC, judgement model)
20. Dramatize your ideas. (I don't know?  Try it.  It could work.  will not work by virtue of it being a good model of things, might work by luck/breaking people out of their habits)
21. Throw down a challenge. (can work if people are willing to rise to a challenge can work against you and create cognitive dissonance if people are not willing.  Need more information to make it work)
22. Begin with praise and honest appreciation. (Don't give people a shit sandwich - slices of compliments surrounding shit.  That's not respectful of them.  Instead using [1] navigate to a place of safety to talk about things)
23. Call attention to people's mistakes indirectly. (there are correct and incorrect ways to do this.  You can be passive agressive about it.  I don't see a problem with being blunt - in private, in safe conversations [1] - about what is going on)
24. Talk about your own mistakes before criticizing the other person. (don't yammer on, but it can help to connect you and them and the problem.  NVC would be better than just this)
25. Ask questions instead of giving direct orders. (socratic method, can be a drain, need more advanced skills and [15] EL to know if this is appropriate )
26. Let the other person save face. (better described in I agree with this, but [15] EL might describe it better)
27. Praise the slightest and every improvement. Be "lavish in your praise." (NVC disagrees, praise only what is relevant, true and valid.  Be a teacher [4] but deliver praise when praise is due.)
28. Give the other person a fine reputation to live up to. (This is 19/26 again.  I agree with it.  I could use it more)
29. Use encouragement. Make the fault seem easy to correct. (agree, solve the "problem" for someone else, make it easy to move forward)
30. Make the other person happy about doing the thing you suggest. (NVC gives a better model of doing what other people want, "with the joy of a small child feeding a hungry duck")

* Giving people a positive reputation to live up to.  "I trust that you won't forget my birthday again".  Don't be silly with this, "I have confidence that you will give me a million dollars" will not actually yield you a million dollars unless you have reason to believe that will work.

6. NVC - Non-Judgemental communication 

I can't yet do justice to NVC but I am putting together the pieces.  Best to watch the youtube talk in the title link but here are some short points.  Also this helps -
a. Concrete descriptions -
In agreement with Gottman, be concrete and specific -  The objective test of whether the description is concrete is whether the description can be followed by an anonymous person to produce the same experience.  "you are a lazy person" VS "you are sitting on the couch"
b. Acknowledge feelings -
people have huge psychological needs to be heard and understood.  Anyone can fulfill that need
c. Connect that to a need
See the NVC video.
d.  Making a request
See NVC video.
e. Saying no by passing your goals forward
Instead of saying no, Consider what it is that gets in the way of you saying no and say that instead.  Keep in mind vulnerability [16].  This also allows people to plan around your future intentions.  If someone asks you to buy a new car and you say, "no I plan to save money towards buying a house" they can choose to be mindful of that in the future and they can act accordingly (not offering you a different car for sale next week).
f. Connect with what is most alive in you right now
See video for best description.

7. Emotional intelligence

There is a two way path between physiological states and emotional states.

Try these:
a. Hold a pencil/pen in your mouth and go back and read the joke about the old man [4]. (expect to find it funnier than you did the first time)
b. furrow your brow while reading the first paragraph of this page again (expect to either feel confused or the cognitive dissonance version if you know it very well - "I know this too well")
The two way path means that you can feel better about emotional pain by taking a paracetamol, but more specifically, if you take a break from a situation and come back to it the emotions might have improved.  This can include getting a glass of water, going for a walk, getting some fresh air.  And for more complicated decisions - sleeping on it (among other things).

Everyone can train emotional intelligence, they need practice.  This includes holding an understanding of your own states as well as being able to notice emotional states in other people.

I had an ex who had particularly visible physiological states, it was a very valuable experience to me to see the state changes and it really trained my guessing mind to be able to notice changes.  These days I can usually see when things change but I can't always pick the emotion that has come up.  This is where NVC and curiosity become valuable (also circling).

EI is particularly important when it is particularly deficient.  In the book it talks about anger as a state that (to an untrained person) can cause a reaction before someone knows that they were angry.  Make sure to fix that first before moving to higher levels of emotional management.

8. model of arguments

(see also NVC)

If you view disagreements or misunderstandings as a venn diagram of what you know and what the other person knows.  You have full rights to make comment on anything you know but only have limited rights to make comment on what the other person knows.  Instead you can comment on the information they have given you.  "you said 'X', I know Y about what you said 'X'".  To say X is wrong, is not going to yield progress.  Instead to acknowledge that they described 'X' and their description does not make sense to me, or leaves me feeling confused [6].

9. The argument started earlier

From Gavin: "If I ever find myself in a position of saying - well officer, let me explain what happened...", Something already went wrong well and truly before now.
When you start the journey you will start getting to "Aha" moments about where arguments start.  As you get more and more experience you realise the argument started well and truly earlier than you ever first realised.  When you get really good at it, you can stop and say [6] "I am confused"  well and truly before a yelling match.

10. The stories we tell ourselves

NVC based, Judgement model, There is a lot of people who are thinking in stories.  Related -

Their entire existence is the story and narrative they tell about themselves (see also Jordan Peterson - maps of meaning).  The constant narrative about how "the world hates me" is going to give you a particular world experience compared to the constant narrative, "I am a lucky person".  You see this in gamblers who are searching for "the prevailing wind" or "winning streaks".

You also see this in social pressure - when people think and get fixated on, "what will people think of me?", sometimes the social pressure does not even have to be there to cause the thoughts and the actions that would be "social pressure".
Several models of thinking advocate removing the story telling in your head to relieve the psychological pain.  See books, "search inside yourself", NVC, Gateless gatecrashers, some information in the Persistent Non Symbolic Experience Article.

I am not sure what is the best practice, but mindfulness seems to help as well, since these thoughts are all theoretical, grounding yourself in the concrete [6a] and observing those thoughts seems to alleviate the anxieties it can cause.  But this can explain a lot of people's actions (they are telling themselves a particular story in their head).

11. Polling your internal states
[related to 6 NVC]. Any time you are disconnected to what is going on, try asking yourself an internal question of "what is going on?" to connect with what is most alive in you right now.  This might be a feeling of boredom.  It could be anything, but if it's not a good and strong connection with what is presently happening you have a chance to fix it.  (See also the book "The Charisma myth")

12. circling (The circling handbook)

[6 built on NVC] is a practice of living in the current and present experience.  You can focus on another person or focus on yourself.  Perpetually answering the question of "what is most alive in you right now?" and sharing that with other people.

Some examples include:I am feeling nervous sharing this experienceI just closed my eyes and put my head back trying to think of a good example.I am distracted by the sound of birds behind me.I can feel air going past my nostrils as I think about this question.

The creators of cicling find it a very connecting experience to either share what is going on inside you or to guess at what is going on inside someone else and ask if that's an accurate guess.  Or to alternate experiences, each sharing one and one.  or each guessing of each other - one and one.

I find it valuable because everyone can understand present experience, and get a glimpse of your current experience in the process of sharing experience with you.  This method can also work as a form of [15] and [7].

13. The game

(From the book The Game) This concept receives equal part condemnation and praise from various parties.

The basic concept of the game is that life is a game.  Specifically social interactions are a game that you can try out.  You can iterate on and repeat until success.  In the book it follows the journey of a pick up artist as he generally disregards other people's agency and works out how to get what he wants (regularly bed people) through some stages of practicing certain methods of interaction, and iterating until he sees a lot of success.

I see a lot of this concept at kegan stage 3[18].  Everything is about social, and the only thing that matters is social relationships.

Most of the condemnations comes from the failure of this model to treat other people as human, worthy of moral weight, thought or anything other than to be used to your own purposes.  If you don't like dehumanising people the book can still teach you a lot about social interaction, and practicing towards incremental improvement.

If you feel uncomfortable with Pick up, you should examine that belief closely, it's probably to do with feeling uncomfortable with people using manipulation to pursue sex.  That's fine, there is a lot to learn about social and a lot of social systems before you turn into "literally the devil" for knowing about it.  There are also other social goals other than sex that you can pursue.

If you are cautious about turning into a jerk - you are probably not likely to ever even get close to actions that paint you as a jerk because your filters will stop you.  It's the people who have no filter on actions that might want to be careful - herein lies dark arts and being a jerk.  And as much as no one will stop you, no one will really enjoy your presence either if you are a jerk.

The biggest problem I have with game and game methodology is that we all play a one-shot version.  With high stakes of failure.  Which means some of the iteration and having to fail while you learn how to not be terrible - will permanently damage your reputation.  There is no perfect "retry" - a reputation will follow you basically to the ends of the earth and back.  As much as game will teach you some things, the other models in this list have better information for you and are going to go further than game.

14. what an apology must do from Aaron Lazare, M.D.- on apology

1. A valid acknowledgement of the offence that makes clear who the offender is and who is the offended. The offender must clearly and completely acknowledge the offence.
2. An effective explanation, which shows an offence was neither intentional nor personal, and is unlikely to recur.
3. Expressions of remorse, shame, and humility, which show that the offender recognises the suffering of the offended.
4. A reparation of some kind, in the form of a real or symbolic compensation for the offender’s transgression.
An effective apology must also satisfy at least one of seven psychological needs of an offended person.
1. The restoration of dignity in the offended person.
2. The affirmation that both parties have shared values and agree that the harm committed was wrong.
3. Validation that the victim was not responsible for the offense.
4. The assurance that the offended party is safe from a repeat offense.
5. Reparative justice, which occurs when the offended sees the offending party suffer through some type of punishment.
6. Reparation, when the victim receives some form of compensation for his pain.
7. A dialogue that allows the offended parties to express their feelings toward the offenders and even grieve over their losses.

These are not my notes from the book but they are particularly valuable when trying to construct an understanding of apologising and making up for misdeeds.  I don't have them in memory but I know when I need to make a serious apology I can look them up.  They fit quite well with [6], but are more specific to apology and not all interactions.

15. Emotional labour

A relatively new concept.  This is roughly the ability to:
I. Model someone else's emotional state
II. Get it right
III. act on their emotional state

For example:
I. I notice my partners eyes are droopy and they do not appear to be concentrating very well.  Is rubbing eyes and checking their watch a lot.
II. I suspect they are sleepy
III. I make them a coffee, or I offer to make them coffee.  (as a downgraded form I mention they look tired and ask if this is the case)

From Erratio:

Emotional labour is essentially a name for a managerial role in a relationship. This takes on a few different concrete forms.

The first is management of the household, appointments, shopping, and other assorted tasks that are generally shared across couples and/or housemates. Sweeping a floor or cooking dinner is not emotional labour, but being the person who makes sure that those things are accomplished is. It doesn't matter whether you get the floor swept by doing it yourself, asking your partner to do it, firing up a Roomba, or hiring a cleaning service; what matters is that you are taking on responsibility for making sure the task is done. This is why people who say that they would be happy to help with the housework if you would just tell them what needs doing are being a lot less helpful than they think. They're taking the physical labour component of the task but explicitly sticking the other person with the emotional labour component.

The second is taking responsibility for the likes, dislikes, feelings, wants and needs of other people who you are in a relationship with (and to be clear, it doesn't have to be a romantic relationship). Stereotypical scenarios that are covered by this kind of emotional labour include: the hysterical girlfriend who demands that her boyfriend drop everything he's doing to comfort her, the husband who comes home tense and moody after a long day at the office and expects to be asked how his day went and listened to and have validating noises made at him, noticing that the other person in a conversation is uncomfortable and steering the conversation to a more pleasant topic without having to be asked, helping a confused friend talk through their feelings about a potential or former partner, reminding your spouse that it's so-and-so's birthday and that so-and-so would appreciate being contacted, remembering birthdays and anniversaries and holidays and contacting people and saying or doing the right things on each of those dates.

This overlaps with [7].  Commentary on this concept suggest that it's a habit that women get into doing more than men.  Mothers are good at paying attention to their kids and their needs (as the major caregiver from early on), and stemming from this wives also take care of their husbands.  While it would not be fair to suggest that all wives do anything I would be willing to concede that these are habits that people get into and are sometimes socially directed by society.

I am not sure of the overall value of this model but it's clear that it has some implications about how people organise themselves - for better or worse.

16. Vulnerability - Brene brown
In order to form close connections with people a certain level of vulnerability is necessary.  This means that you need to share about yourself in order to give people something to connect to.  In the other direction people need to be a certain level of vulnerable to you in order to connect.  If you make sure to be open and encouraging and not judge you will enable people to open up to you and connect with you.
Sometimes being vulnerable will get you hurt and you need to be aware of that and not shut down future experiences (continue to be open with people).  I see this particularly in people who "take time" to get over relationships.  Being vulnerable is a skill that can be practiced.  Vulnerability replaced a lot of my ideas about [13 The game].  And would have given me a lot of ideas of how to connect with people, combined with [15] and [12]. (I have not read her books but I expect them to be useful)

17. More Than Two (book)

This is commonly known as the polyamory bible.  It doesn't have to be read as a polyamory book, but in the world of polyamory emotional intelligence and the ability to communicate is the bread and butter of every day interactions.  If you are trying to juggle two or three relationships and you don't know how to talk about hard things then you might as well quit now.  If you don't know how to handle difficult feelings or experiences you might as well quit polyamory now.

Reading about these skills and what you might gain from the insight that polyamorous people have learnt is probably valuable to anyone.

18. Kegan stages of development

Other people have summarised this model better than me.  I won't do it justice but if I had to be brief about it - there are a number of levels that we pass through as we grow from very small to more mature.  They include the basic kid level where we only notice inputs and outputs.  Shortly after - when we are sad "the whole world is sad" because we are the whole world.  Eventually we grow out of that and recognise other humans and that they have agency.  At around teenager we end up caring a lot about what other people think about us.  classic teenagers are scared of social pressure and say things like, "I would die if she saw me in this outfit" (while probably being hyperbolic, there is a bit of serious concern present).  Eventually we grow out of that and into system thinking (Libertarian, Socialist, among other tribes).  And later above tribalism into more nuanced varieties of tribes.

It's hard to describe and you are better off reading the theories to get a better idea.  I find the model limited in application but I admit I need to read more about the theories to get my head around it better.

I have a lot more books on the topic to read but I am publishing this list because I feel like I have a good handle on the whole "how people work" and, "how relationships work" thing.  It's rare that anyone does any actions that surprise me (socially) any more.  In fact I am getting so good at it that I trust my intuition [11] more than what people will say sometimes.

When something does not make sense I know what question to ask [6] to get answers.  Often enough it happens that people won't answer the first time, this can represent people not feeling Safe [1] enough to be vulnerable [16].  That's okay.  That represents it's my job to get them to a comfortable place to open up if I want to get to the answers.

I particularly like NVC, Gottman, EL, EI, Vulnerability all of them and find myself using them fortnightly.  Most of these represent a book or more of educational material.  Don't think you know them enough to dismiss them if you have not read the books.  If you feel you know them and already employ the model then it's probably not necessary to look into it further, but if you are ready to dismiss any of these models because they "sound bad" or "don't work" then I would encourage you to do your homework and understand them inside and out before you reject them.

The more models I find the more I find them converging on describing reality.  I am finding less and less I can say, "this is completely new to me" and more and more, "oh that's just like [6] and [7]

Meta: this is something around 6000 words and took a day to write ~12 hours.  I did this in one sitting because everything was already in my head.  I am surprised I could sit still for this long.  (I took breaks for food and a nap but most of today was spent at my desk)

Originally posted on my blog:

Cross posted to Medium:

[Link] Game Theory & The Golden Rule (From Reddit)

15 Brillyant 28 July 2017 01:54PM

The dark arts: Examples from the Harris-Adams conversation

15 Stabilizer 20 July 2017 11:42PM

Recently, James_Miller posted a conversation between Sam Harris and Scott Adams about Donald Trump. James_Miller titled it "a model rationalist disagreement". While I agree that the tone in which the conversation was conducted was helpful, I think Scott Adams is a top practitioner of the Dark Arts. Indeed, he often prides himself on his persuasion ability. To me, he is very far from a model for a rationalist, and he is the kind of figure we rationalists should know how to fight against.


Here are some techniques that Adams uses:


  1. Changing the subject: (a) Harris says Trump is unethical and cites the example of Trump gate-crashing a charity event to falsely get credit for himself. Adams responds by saying that others are equally bad—that all politicians do morally dubious things. When Harris points out that Obama would never do such a thing, Adams says Trump is a very public figure and hence people have lots of dirt on him. (b) When Harris points out that almost all climate scientists agree that climate change is happening and that it is wrong for Trump to have called climate change a hoax, Adams changes the subject to how it is unclear what economic policies one ought to pursue if climate change is true.
  2. Motte-and-bailey: When Harris points out that the Trump University scandal and Trump's response to it means Trump is unethical, Adams says that Trump was not responsible for the university because it was only a licensing deal. Then Harris points out that Trump is unethical because he shortchanged his contractors. Adams says that that’s what happens with big construction projects. Harris tries to argue that it’s the entirety of Trump’s behavior that makes it clear that he is unethical—i.e., Trump University, his non-payment to contractors, his charity gate-crashing, and so on. At this points Adams says we ought to stop expecting ethical behavior from our Presidents. This is a classic motte-and-bailey defense. Try to defend an indefensible position (the bailey) for a while, but then once it becomes untenable to defend it, then go to the motte (something much more defensible).
  3. Euphemisation: (a) When Harris tells Adams that Trump lies constantly and has a dangerous disregard for the truth, Adams says, I agree that Trump doesn’t pass fact checks. Indeed, throughout the conversation Adams never refers to Trump as lying or as making false statements. Instead, Adams always says, Trump “doesn’t pass the fact checks”. This move essentially makes it sound as if there’s some organization whose arbitrary and biased standards are what Trump doesn’t pass and so downplays the much more important fact that Trump lies. (b) When Harris call Trump's actions morally wrong, Adams makes it seem as if he is agreeing with Harris but then rephrases it as: “he does things that you or I may not do in the same situation”. Indeed, that's Adams's constant euphemism for a morally wrong action. This is a very different statement compared to saying that what Trump did was wrong, and makes it seem as if Trump is just a normal person doing what normal people do. 
  4. Diagnosis: Rather than debate the substance of Harris’s claims, Adams will often embark on a diagnosis of Harris’s beliefs or of someone else who has that belief. For example, when Harris says that Trump is not persuasive and does not seem to have any coherent views, Adams says that that's Harris's "tell" and that Harris is "triggered" by Trump's speeches. Adams constantly diagnoses Trump critics as seeing a different movie, or as being hypnotized by the mainstream media. By doing this, he moves away from the substance of the criticisms.
  5. Excusing: (a) When Harris says that it is wrong to not condemn, and wrong to support, the intervention of Russia in America’s election, Adams says that the US would extract revenge via its intelligence agencies and we would never know about it. He provides no evidence for the claim that Trump is indeed extracting revenge via the CIA. He also says America interferes in other elections too. (b) When Harris says that Trump degraded democratic institutions by promising to lock up his political opponent after the election, Adams says that was just a joke. (c) When Harris says Trump is using the office of the President for personal gain, Adams tries to spin the narrative as Trump trying to give as much as possible late in his life for his country. 
  6. Cherry-picking evidence: (a) When Harris points out that seventeen different intelligence agencies agreed that Russia’s government interfered in the US elections, Adams says that the intelligence agencies have been known to be wrong before. (b) When Harris points out that almost all climate scientists agree on climate change, Adams points to some point in the 1970s where (he claims) climate scientists got something wrong, and therefore we should be skeptical about the claims of climate scientists.

Overall, I think what Adams is doing is wrong. He is an ethical and epistemological relativist: he does not seem to believe in truth or in morality. At the very least, he does not care about what is true and false and what is right and wrong. He exploits his relativism to push his agenda, which is blindingly clear: support Trump.


(Note: I wanted to work on this essay more carefully, and find out all the different ways in which Adams subverts the truth and sound reasoning. I also wanted to cite more clearly the problematic passages from the conversations. But I don't have the time. So I relied on memory and highlighted the Dark Arts moves that struck me immediately. So please, contribute in the comments with your own observations about the Dark Arts involved here.)

Prediction should be a sport

13 chaosmage 10 August 2017 07:55AM

So, I've been thinking about prediction markets and why they aren't really catching on as much as I think they should.

My suspicion is that (beside Robin Hanson's signaling explanation, and the amount of work it takes to get to the large numbers of predictors where the quality of results becomes interesting) the basic problem of prediction markets is that they look and feel like gambling. Or at best like the stock market, which for the vast majority of people is no less distasteful.

Only a small minority of people are neither disgusted by nor terrified of gambling. Prediction markets right now are restricted to this small minority.

Poker used to have the same problem.

But over the last few decades Poker players have established that Poker is (also) a sport. They kept repeating that winning isn't purely a matter of luck, they acquired the various trappings of tournaments and leagues, they developed a culture of admiration for the most skillful players that pays in prestige rather than only money and makes it customary for everyone involved to show their names and faces. For Poker, this has worked really well. There are much more Poker players, more really smart people are deciding to get into Poker and I assume the art of game probably improved as well.

So we should consider re-framing prediction the same way.

The calibration game already does this to a degree, but sport needs competition, so results need to be comparable, so everyone needs to make predictions on the same events. You'd need something like standard cards of events that players place their predictions on.

Here's a fantasy of what it could look like.

  • Late in the year, a prediction tournament starts with the publication of a list of events in the coming year. Everybody is invited to enter the tournament (and maybe pay a small participation fee) by the end of the year, for a chance to be among the best predictors and win fame and prizes.
  • Everyone who enters plays the calibration game on the same list of events. All predictions are made public as soon as the submission period is over and the new year begins. Lots of discussion of each event's distribution of predictions.
  • Over the course of the year, events on the list happen or fail to happen. This allows for continually updated scores, a leaderboard and lots of blogging/journalistic opportunities.
  • Near the end of the year, as the leaderboard turns into a shortlist of potential winners, tension mounts. Conveniently, this is also when the next tournament starts.
  • At new year's, the winner is crowned (and I'm open to having that happen literally) at a big celebration which is also the end of the submission period for the next tournament and the revelation of what everyone is predicting for this next round. This is a big event that happens to be on a holiday, where more people have time for big events.
Of course this isn't intended to replace existing prediction markets. It is an addition to those, a fun and social thing with lots of PR potential and many opportunities to promote rationality. It should attract people to prediction who are not attracted to prediction markets. And it could be prototyped pretty cheaply, and developed further if it is as much fun as I think it would be.

Rational Feed

12 deluks917 09 September 2017 07:48PM

=== Updates:

I have been a little more selective about which articles make it onto the feed. I have not been overly selective and all of the obviously general interest rationalsit articles still make it.

Unless people object I am going to try a "weekly feed". The bi-weekly feed is pretty long. I currently post on the SSC reddit and lesswrong. Weekly seems fine for the SSC reddit but lesswrong is a lower activity forum. I will see how it goes. Obviously on a weekly feed there will about half as many recommended articles.

===Highly Recommended Articles:

Object, Subjects and Gender by The Baliocene Apocrypha - "Under modern post-industrial bureaucratized high-tech capitalism, it is less rewarding than ever before to be a subject. Under modern post-industrial bureaucratized high-tech capitalism, it is more rewarding than ever before to be an object. This alone accounts for a lot of the widespread weird stuff going on with gender these days."

Winning Is For Losers by Putanumonit (ribbonfarm) - Zero vs Positive Sum Games. The strong have room to cooperate. Rene Girard's theory of mimetics and competition. College Admissions. Tit for Tat. Spiked dicks in nature. Short and long term strategies in dating. Quirky dating profiles. Honesty on the first date. Beating Moloch with a transhuman God.

Premium Mediocre by Jacob Falkovich - Being 30% wrong is better than being 5% wrong. Consumption: Signaling vs genuine enjoyment. Dating other PM people. Venkat is wrong about impressing parents. He is more wrong, or joking, about cryptocurrencies. Fear of missing out.

Ten New 80000 Hours Articles Aimed At The by 80K Hours (EA forum) - Ten recent articles and descriptions from 80K hours. Over and underpaid jobs relative to their social impact, the most employable skills, learning ML, whether most social programs work and other topics.

Minimizing Motivated Beliefs by Entirely Useless - The tradeoffs between epistemic and instrumental rationality. Yudkowsky's argument such tradeoffs either very stupid or don't exist. Issues with Yudkowsky: Denial that belief is voluntary, thinking that trading away the truth requires being blind to consequences. Horror victims and transcendent meaning. Interesting things are usually false.


How Do We Get Breasts Out Of Bayes Theorem by Scott Alexander - "But evolutionary psychologists make claims like 'Men have been evolutionarily programmed to like women with big breasts, because those are a sign of fertility.' Forget for a second whether this is politically correct, or cross-culturally replicable, or anything like that. From a neurological point of view, how could this possibly work?"

Predictive Processing And Perceptual Control by Scott Alexander - "predictive processing attributes movement to strong predictions about proprioceptive sensations. Because the brain tries to minimize predictive error, it moves the limbs into the positions needed to produce those sensations, fulfilling its own prophecy." Connections with Will Power's 'Behavior: The Control of Perception' which Scott already reviewed.

Book Review: Surfing Uncertainty by Scott Alexander - Scott finds a real theory of how the brain works. "The key insight: the brain is a multi-layer prediction machine. All neural processing consists of two streams: a bottom-up stream of sense data, and a top-down stream of predictions. These streams interface at each level of processing, comparing themselves to each other and adjusting themselves as necessary."

Links: Exsitement by Scott Alexander - Slatestarcodex links post. A Nootropics survey, gene editing, AI, social norms, Increasing profit margins, politics, and other topics.

Highlights From The Comments On My Irb Nightmare by Scott Alexander - Tons of hilarious IB stories. A subreddit comment about getting around irb. Whether the headaches are largely institutional rather than dictated by government fiat. Comments argue in favor of the irb and Scott responds.

My IRB Nightmare by Scott Alexander - Scott tries to run a study to test the Deck Depression Inventory. The institutional review board makes this impossible. They not only make tons of capricious demands they also attempt to undermine the study's scientific validity.

Slippery Slopen Thread by Scott Alexander - Public open thread. The slippery slope to rationalist catgirl. Selected top comments. Update on Trump and crying wolf.

Contra Askell On Moral Offsets by Scott Alexander - Axiology is the study of what’s good. Morality is the study of what the right thing to do is. You can offset axiological effects but you can't offset moral transgressions.


MRE Futures To Not Starve by Robin Hanson - Emergency food sources as a way to mitigate catastrophic risk. The Army's 'Meals Ready to Eat'. Food insurance. Incentives for producers to deliver food in emergencies. Incentives for researchers to find new sources. Sharing information.

Book Reviews: Zoolitude And The Void by Jacob Falkovich - Seven Surrenders the sequel to 'Too like the Lightning' mercilessly cuts the bad parts and focuses on the politics, personalities, and philosophy that made TLTL great. The costs of adding too much magic to a setting, don't make the mundane irrelevant. One Hundred Years of Solitude: Shit just happens. Zoo City: Realistic Magic: "The Zoo part is the magic: some people who commit crimes mysteriously acquire an animal familiar and a low-key magical talent." The Mark and the Void: "Technically, there’s no magic in The Mark and the Void. But there’s investment banking, which takes the role of the mysterious force that decides the fate of individuals and nations but remains beyond the ken of mere mortals."

The World As If by Sarah Perry (ribbonfarm) - "This is an account of how magical thinking made us modern." Magical thinking as a confusing of subjective and objective. Useful fictions. Hypothetical thinking. Pre-modern concrete thinking and categorization schemes relative to modern abstract ones. As if thinking. Logic and magic.

To Save The World Make Sure To Go Beyond Academia by Kaj Sotala - Academic research often fails to achieve real change. Lots of economic research concerns the optimal size of a carbon tax but we currently lack any carbon tax. Academic research on x-risk from nuclear winter doesn't change the motivations of politicians very much.

Introducing Mindlevelup The Book by mindlevelup - MLU compiled and edited their work from 2017 into a 30K word, 150 page book. Most of the material appeared on the blog but some of it is new and the pre-existing posts have been edited for clarity.

Expanding Premium Mediocrity by Zvi Moshowitz - "This is (much of) what I think Rao is trying to say in the second section of his post, the part about Maya but before Molly and Max, translated into DWATV-speak. Proceed if and only if you want that."

Simple Affection And Deep Truth by Particular Virtue - "Simple Affection is treating someone like a child: they will forget about bad things, as long as you give them something good to think about instead. Deep Truth is treating someone like an elephant: they never forget, and they forgive only with deep deliberation."

Are People Innately Good by Sailor Vulcan - SV got into two arguments that went badly. One was on all lives matter. The other occurred when SV tried to defend Glen of Intentional Insights on the SSC discord. Terminal values aren't consistent. SV was abused as a child.

Metapost September 5th by sam[]zdat - Plans for the blog. Next series will be on epistemology and the ''internal' side of nhilism. Revised introduction. Sam will probably write fiction. Site reorganization. History section. Current reading list. Patreon.

Minimizing Motivated Beliefs by Entirely Useless - The tradeoffs between epistemic and instrumental rationality. Yudkowsky's argument such tradeoffs either very stupid or don't exist. Issues with Yudkowsky: Denial that belief is voluntary, thinking that trading away the truth requires being blind to consequences. Horror victims and transcendent meaning. Interesting things are usually false.

Exploring Premium Mediocrity by Zvi Moshowitz - Defining premium mediocre. Easy and hard mode related to Rao's theories of losers, sociopaths and heroes. The Real Thing. A 2x2 ribbonfarm style graph. Restaurants.

Tegmarks Book Of Foom by Robin Hanson - Tegmark's recent book basically described Yudkowsky's intelligence explosion. Tegmark is worried the singularity might be soon and we need to have figured out big philosophical issues by then. Hanson thinks Tegmark overestimates the generality of intelligence. AI weapons and regulations.

The Doomsday Argument In Anthropic Decision Theory by Stuart Armstrong (lesswrong) - "In Anthropic Decision Theory (ADT), behaviors that resemble the Self Sampling Assumption (SSA) derive from average utilitarian preferences. However, SSA implies the doomsday argument. This post shows there is a natural doomsday-like behavior for average utilitarian agents within ADT."

Forager Vs Farmer Elaborated by Robin Hanson - Early humans collapsed Machiavellian dynamics down to a reverse-dominance-hierarchy. Group norm enforcement and its failure modes. Safety leads to collective play and art, threat leads to a return to Machiavellianisn and suspicion. Individuals greatly differ as to what level of threat causes the switch, often for self-serving reasons. Left vs right. "The first and primary political question is how much to try to resolve issues via a big talky collective, or to let smaller groups decide for themselves."

Critiquing Other Peoples Plans Politely by Katja Grace - Three failure modes: The attack, The polite sidestep, The inadvertent personal question. A plan to avoid these issues: debate beliefs, not actions.

Gleanings From Double Crux On The Craft Is Not The Community by Sarah Constantin - Results from Sarah's public double crux. Sarah initially did not think the rationalist intellectual project was worth preserving. She wants to see results, even though she concedes that formal results can be very difficult to get. What is the value of introspection and 'navel grazing'?

Intrinsic Properties And Eliezers Metaethics by Tyrrell_McAllister (lesswrong) - Intuitions of intrinsicness. Is goodness intrinsic? Seeing intrinsicness in simulations. Back to goodness.

Winning Is For Losers by Putanumonit (ribbonfarm) - Zero vs Positive Sum Games. The strong have room to cooperate. Rene Girard's theory of mimetics and competition. College Admissions. Tit for Tat. Spiked dicks in nature. Short and long term strategies in dating. Quirky dating profiles. Honesty on the first date. Beating Moloch with a transhuman God.

Dangers At Dilettante Point by Everything Studies - Its relatively easy to know a little about alot of topics. But its dangerous to find yourself playing the social role of the knowledgeable person too often. The percentage fo people with a given level of knowledge goes to zero quickly.

Entrenchment Happens by Robin Hanson - Many systems degrade, collapse and our replaced. However other systems, even somewhat arbitrary ones, are very stable over time. Many current systems in programming, language and law are likely to remain in the future.

Premium Mediocre by Jacob Falkovich - Being 30% wrong is better than being 5% wrong. Consumption: Signaling vs genuine enjoyment. Dating other PM people. Venkat is wrong about impressing parents. He is more wrong, or joking, about cryptocurrencies. Fear of missing out.


Ideological Engineering And Social Control by Geoffrey Miller (EA forum) - China is trying hard to develop advanced AI. A major goal is to use AI to monitor both physical space and social media. Supressing wrong-think doesn't require radically advanced AI.

Incorrigibility In Cirl by The MIRI Blog - Paper. Goal: Incentivize a value learning system to follow shut down instructions. Demonstration that some assumptions are not stable with respect to model mis-specification (ex programmer error). Weaker sets of assumptions: difficulties and simple strategies.

Nothing Wrong With Ai Weapons by kbog (EA forum) - Death by AI is no more intrinsically bad than death by conventional weapons. Some consequenitoualist issues the author addresses: Civilian deaths, AI arms race, vulnerability to hacking.


Can Outsourcing Improve Liberias Schools Preliminary RCT Results by Innovations for Poverty - "Last summer, the Liberian government delegated management of 93 public elementary schools to eight different private contractors. After one year, public schools managed by private operators raised student learning by 60 percent compared to standard public schools. But costs were high, performance varied across operators, and contracts authorized the largest operator to push excess pupils and under-performing teachers into other government schools."

Ten New 80000 Hours Articles Aimed At The by 80K Hours (EA forum) - Ten recent articles and descriptions from 80K hours. Over and underpaid jobs relative to their social impact, the most employable skills, learning ML, whether most social programs work and other topics.

Is Ea Growing Some Ea Growth Metrics For 2017 by Peter Hurford (EA forum) - Activity metrics for EA website, donations data, additional Facebook data, commentary that EA seems to be growing but there is substantial uncertainty.

Ea Survey 2017 Series Cause Area Preferences by Tee (EA forum) - Top Cause Area, near-top areas, areas which should not have EA resources, cause area correlated with demographics, donations by cause area.

Looking At How Superforecasting Might Improve AI Predictions by Will Pearson (EA forum) - Good Judgement Project: What they did, results, relevance. Lessons: Focus on concrete issues, focus on AI with no intelligence augmentation, learn a diverse range of subjects, breakdown the open questions, publicly update.

Why Were Allocating Discretionary Funds To The Deworm The World Initiative by The GiveWell Blog - "Why Deworm the World has a pressing funding need. The benefits and risks of granting discretionary funds to Deworm the World today. Why we’re continuing to recommend that donors give 100% of their donation to AMF."

Ea Survey 2017 Series Community Demographics by Katie Gertsch (EA forum) - Some results: Mostly young and male, slight increase in female participation. Highest concentration cities. Atheism/Agnostic rate fell from 87% to 80%. Increase in the proportion of EA who see EA as a duty or opportunity as opposed to an obligation.

Effective Altruism Survey 2017 Distribution And by Ellen McGeoch and Peter Hurford (EA forum) - EA 2017 Study results are in. Details about distribution abd data analysis techniques. Discussion of whether the subpopulation is a representative sample of EA and its subpopulations.

Six Tips Disaster Relief Giving by The GiveWell Blog - Practical advice for effective disaster relief charity. Give Cash, give to proven effective charities and allow charities significant freedom in how they use your donation.

===Politics and Economics:

Harvard Admit Legacy Students by Marginal Revolution - Demand for Ivy league admissions far outstrips supply. The main constraint is that the Ivy League depends on donations. One way to scale up, while maintaining high donation rates, is to increase legacy admissions. Teaching quality is unlikely to suffer, qualified students are easy to find.

Object, Subjects and Gender by The Baliocene Apocrypha - "Under modern post-industrial bureaucratized high-tech capitalism, it is less rewarding than ever before to be a subject. Under modern post-industrial bureaucratized high-tech capitalism, it is more rewarding than ever before to be an object. This alone accounts for a lot of the widespread weird stuff going on with gender these days."

Links 11 by Artir - Psychology, Politics, Economics, Philosophy, Other. Several links related to the Google memo.

Unpopular Ideas About Crime And Punishment by Julia Galef - Thirteen opinions on prison abolition, the death penalty, corporal punishment, rehabilitation, redistribution and more.

Intangible Investment and Monopoly Profits by Marginal Revolution - "Intangible capital used to be below 30 percent of the S&P 500 in the 70s, now it is about 84 percent. " Seven implications about profit, monopoly, spillover, etc.

What You Cant Say To A Sympathetic Ear by Katja Grace - Sharing socially unacceptable views with your friends is putting them in a bad situation, regardless of whether they agree with those ideas. If they don't punish you society will hold them complicit. Socially condemning views is worse than commonly thought "To successfully condemn a view socially is to lock that view in place with a coordination problem."

A I Bias Doesnt Mean What Journalists Want You To Think It Means by Chris Stucchio And Lisa Mahapatra (Jacobite) - What is data science and AI? What is bias? How do we identify bias? The fallout of the author's algorithm. Predicting Creditworthiness. Understanding Language. Predicting Criminal Behavior. Journalists and Wishful Thinking.

Four Decades of the Middle East by Bryan Caplan - "Almost all of the Middle East's disasters over the past four decades can be credibly traced back to a single highly specific major event: the Iranian Revolution. Let me chronicle the tragic trail of dominoes."

The Thresher by sam[]zdat - "Still, if what makes 'modernity' modernity is partially in technology, then the Uruk Machine will be updated and whirring at unfathomable speeds, the thresher to Gilgamesh’s sacred club."

The Uruk Machine by sam[]zdat - Sam's fundamental framework: Seeing like a State, The Great Transformation, The True Believer, The Culture of Narcissism.


Into The Gray Zone by Bayesian Investor - Book Review. A modest fraction of people diagnosed as being in a persistent vegetative state have locked in syndrome. People misjudge when they would want to die. Alzheimer's.


What You Need To Know About Climate Change by Waking Up with Sam Harris - "How the climate is changing and how we know that human behavior is the primary cause. They discuss why small changes in temperature matter so much, the threats of sea-level rise and desertification, the best and worst case scenarios, the Paris Climate Agreement, the politics surrounding climate science."

Dan Rather by The Ezra Klein Show - "Rather and I discuss the Trump presidency and what it means for the Republican Party's future, our fractured media landscape, and Rather's own evolving career in media."

Caplan Family by Bryan Caplan - "For the last two years, I homeschooled my elder sons, Aidan and Tristan, rather than send them to traditional middle school. Now they've been returned to traditional high school. We decided to mark our last day with a father-son/teacher-student podcast on how we homeschooled, why we homeschooled, and what we achieved in homeschool."

Rob Reich On Foundations by EconTalk - "The power and effectiveness of foundations--large collections of wealth typically created and funded by a wealthy donor. Is such a plutocratic institution consistent with democracy? Reich discusses the history of foundations in the United States and the costs and benefits of foundation expenditures in the present."

Jesse Singal On The Problems With Implicit Bias Tests by Rational Speaking - "The IAT has been massively overhyped, and that in fact there's little evidence that it's measuring real-life bias. Jesse and Julia discuss how to interpret the IAT, why it became so popular, and why it's still likely that implicit bias is real, even if the IAT isn't capturing it."

Emotionally Charged Discussion by The Bayesian Conspiracy - Conversations where one party thinks the other sides position is stupid/evil/etc. Debate vs truth seeking. Julia Galef's lists of unpopular ideas. Agenty Duck's thoughts on introspection. Double Crux.

The Future Of Intelligence by Waking Up with Sam Harris - "Max Tegmark. His new book Life 3.0: Being Human in the Age of Artificial Intelligence. They talk about the nature of intelligence, the risks of superhuman AI, a nonbiological definition of life, the substrate independence of minds, the relevance and irrelevance of consciousness for the future of AI, near-term breakthroughs in AI."

Benedict Evans by EconTalk - "Two important trends for the future of personal travel--the increasing number of electric cars and a world of autonomous vehicles. Evans talks about how these two trends are likely to continue and the implications for the economy, urban design, and how we live."

The Life Of A Quant Trader by 80,000 Hours - What do quant traders do. Compensation. Is quant trading harmful? Who is a good fit and how to break into quant trading. Work environment and motivation. Variety of available positions.

Online discussion is better than pre-publication peer review

12 Wei_Dai 05 September 2017 01:25PM

Related: Why Academic Papers Are A Terrible Discussion Forum, Four Layers of Intellectual Conversation

During a recent discussion about (in part) academic peer review, some people defended peer review as necessary in academia, despite its flaws, for time management. Without it, they said, researchers would be overwhelmed by "cranks and incompetents and time-card-punchers" and "semi-serious people post ideas that have already been addressed or refuted in papers already". I replied that on online discussion forums, "it doesn't take a lot of effort to detect cranks and previously addressed ideas". I was prompted by Michael Arc and Stuart Armstrong to elaborate. Here's what I wrote in response:

My experience is with systems like LW. If an article is in my own specialty then I can judge it easily and make comments if it’s interesting, otherwise I look at its votes and other people’s comments to figure out whether it’s something I should pay more attention to. One advantage over peer review is that each specialist can see all the unfiltered work in their own field, and it only takes one person from all the specialists in a field to recognize that a work may be promising, then comment on it and draw others’ attentions. Another advantage is that nobody can make ill-considered comments without suffering personal consequences since everything is public. This seem like an obvious improvement over standard pre-publication peer review, for the purpose of filtering out bad work and focusing attention on promising work, and in practice works reasonably well on LW.

Apparently some people in academia have come to similar conclusions about how peer review is currently done and are trying to reform it in various ways, including switching to post-publication peer review (which seems very similar to what we do on forums like LW). However it's troubling (in a "civilizational inadequacy" sense) that academia is moving so slowly in that direction, despite the necessary enabling technology having been invented a decade or more ago.

Feedback on LW 2.0

11 Viliam 01 October 2017 03:18PM

What are your first impressions of the public beta?

Rational Feed

11 deluks917 27 August 2017 03:49AM

===Highly Recommended Articles:

What Is Rationalist Berkleys Community Culture by Zvi Moshowitz - The original rationalist community mission was to save the world, not to be nice to each other. Sarah recently suggested the later is currently the actual goal. Zvi reinterprets this as sounding an alarm. The rationalists should not become just another Berkeley community of bohemians and weirdos.

Cthugha The Living Flame by Exploring Egregores - Rationalists as worshippers of an Eldritch Star God. Valuing knowledge and ideas above all else. Bonobos and transhumanists. Yudkowsky's argument about distributed vs concentrated intellect. The AI box experiment. Nerds as the true extraverts. "What do you think the singularity will actually look like?" The site maps eight other Eldritch Gods to different philosophical dispositions.

Internet Explorers Not Exploiters by Nostalgebraist - Exploit vs explore tradeoffs. Attention spans. How long should you try a math problem before you give up? Exploring new options can be uncomfortable since it might lead nowhere. Addictive games and the internet. Academic research.

Diversity And Team Performance What The Research Says by Eukaryote - Opens with several links about diversity and inclusion in EA. The pros and cons of different types of diversity in terms of group cohesion and information processing. Practical ways to minimize the costs of diversity and magnify the benefits. Lots of references.

The Market Power Story by Noah Smith - Many issues in the American economy are blamed on the increasing market power of a small number of firms. Analysis: Monopolistic competition. Profits. Market Concentration. Output restriction. Three updates. Lots of citations and references to papers.

The Anti Slip Slope by samuelthefifth (Status 451) - An analogy between workplace noise and workplace sexism. How efforts to stamp out 'workplace noise' can get out of control.

Dota 2 by Open Ai - Open AI codes a 1v1 Dota-2 bot that defeaated top players. The bots actions per minute were comparable to many humans. The bot learned the game from scratch by self-play, and does not use imitation learning or tree search. The game involves hidden information and the bot's strategies were complicated.

Stop Caring So Much About Technical Problems by Particular Virtue - Links to an article describing what attributes actually get developers jobs (other than technical skill). Caring about making great products is much more desirable than caring about technical problems. Developer interviews are highly random. Experience matters alot. Enterprise programmers are disliked. Practical advice.


Partial Credit by Scott Alexander - Blotting out the Sun. Short story.

Moral Reflective Equilibrium and the Absurdity Principle by SlateStarScratchpad - A long discussion about the nature of morality. The absurdity heuristic. Reflective equilibrium of moral values. The feedback loop between intuition and logic.

Advertising by SlateStarScratchpad - Nostalgebraist muses about advertising. Scott briefly explains how advertising works on SSC.

Fear And Loathing At Effective Altruism Global 2017 by Scott Alexander - EA Global was well run and impressive. The deep weirdness of EA. The fundamental goodness of effective altruists. The yoke is light and everyone is welcome.

Community History by Scott (r/SSC) - Scott answers: "What happened to Lesswrong? When (and more importantly why) did the spread out to other blogs happen?"

Threado Quia Absurdum by Scott Alexander - Bi-Weekly public open thread. Recommended comments on: how organizations change over time, self-driving car progress, gun laws in the Czech Republic, why comments are closed on some posts here. Scot may be choosing a SSC moderator.

Brief Cautionary Notes On Branded Combination Nootropics by Scott Alexander - Many 'Xbrain' pills contain ineffectively low doses of ingredients. Nootropics, like many drugs, effect people differently; you need to isolate which nootropics work for you. Drug interactions are very poorly understood, even for well studied drugs.

The Lizard People Of Alpha Draconis 1 Decided To Build An Ansible by Scott Alexander - Faster than light communication via negative average preference utilitarianism.

Sparta by SlateStarScratchpad - A historian claims that Sparta's military renown was developed during a period when Sparta's actual military ability was declining. Scott disagrees and cites sources showing that the earliest records all claim Sparta was very powerful.


Internal Dialogue About End Of World by Sailor Vulcan - Short Story. Keep living, maybe we will win the lottery.

My Tedtedx Talks by Robin Hanson - Ted talks by Robin about his books "The Age of Em" and "The Elephant in the Brain". Talks are short ~12 minutes.

Paranoia Testing by Elo - Experiments to test if you have paranoia. Costs. Notes and some graphs.

Theres Always Subtext by Robin Hanson - Mostly a quote about subtext in film.

Play In Hard Mode by Zvi Moshowitz - "Hard mode is harder. The reason to Play in Hard Mode is because it is the only known way to become stronger, and to defend against Goodhart’s Law." Zvi revists the eleven examples from 'easy mode' and shows how to approach them from a hard mode perspective.

Play In Easy Mode by Zvi Moshowitz - Eleven examples of 'selling out' and taking the path of least resistance. Interestingly in several examples taking the easy path is quite defensible.

Emotional Labour by Elo - "I wanted to save you the effort of thinking about the thing and so I decided not to tell/ask you before it was resolved." VS "I wanted to not have to withhold a thing from you so I told you as soon as it was bothering me so that I didn’t have to lie/cheat/withhold/deceive you even if I thought it was in your best interest"

Paths Forward On Berkeley Culture Discussion by Zvi Moshowitz - Follow up to Zvi's post on the Berkeley rationalist community. A long sketch of the arguments Zvi would make and the article he would write if he had time to respond in depth.

How Social Is Reason by Robin Hanson - Humans alone have a logical reasoning module. 'Logical Fallacies' evolved because they are adaptive for persuasion. Unschooled populations often cannot solve logical problems. Epistemic learned helplessness. Impressive complex arguments are preferred over simple ones.

Cthugha The Living Flame by Exploring Egregores - Rationalists as worshippers of an Eldritch Star God. Valuing knowledge and ideas above all else. Bonobos and transhumanists. Yudkowsky's argument about distributed vs concentrated intellect. The AI box experiment. Nerds as the true extraverts. "What do you think the singularity will actually look like?" The site maps eight other Eldritch Gods to different philosophical dispositions.

Self Fulfilling Prophecy by Entirely Useless - The author analyzes various edge cases about intention and choice. They discuss how to modify their theories and whether they are on the right track.

Decisions As Predictions by Entirely Useless - "Consider the hypothesis that both intention and choice consist basically in beliefs: intention would consist in the belief that one will in fact obtain a certain end, or at least that one will come as close to it as possible. Choice would consist in the belief that one will take, or that one is currently taking, a certain temporally immediate action for the sake of such an end."

Bathtime by The Unit of Caring - Bath time play with a baby. Things are compelling when they have the right balance of surprise and predictability.

Internet Explorers Not Exploiters by Nostalgebraist - Exploit vs explore tradeoffs. Attention spans. How long should you try a math probem before you give up? Exploring new options can be uncomfortable since it might lead nowhere. Addictive games and the internet. Academic research.

Embracing Metamodernism by Gordon (Map and Territory) - "Metamodernism believes in reconstructing things that have been deconstructed with a view toward reestablishing hope and optimism in the midst of a period (the postmodern period) marked by irony, cynicism, and despair."

Why Ethnicity Ideology by Robin Hanson - "he more life decisions a feature influences, the more those who share this feature may plausibly share desired policies, policies that their coalition could advocate. So you might expect political coalitions to be mostly based on individual features that are very useful for predicting individual behavior. But you’d be wrong."

A Village Is Better Than Group House by Particular Virtue - More private space. Non-shared legal ownership. More people means much more social space and stability.

A Flaw In The Way Smart People Think About Robots And Job Loss by Tom Bartleby - Considering jobs one at a time causes smart people to think no one will lose their job from automation. However small incremental advances reduce the number of needed workers. A history of secretaries. Personal experience of saving time via programming.

More Brain Lies by Aceso Under Glass - "But sometimes it helps to take the gap between is and ought as a sign of how high your standards are, rather than how bad you are at a thing."

Ems In Walkaway by Robin Hanson - A review of the science fiction book 'Walkaway' which features brain emulation. Robin describes what he finds realistic and unrealistic.

Take My Job by Jacob Falkovich - "I want to tell you about the job I’m leaving, why you should think about applying for it, and what it has taught me in the last four years about company culture, diversity, and the makings of a good workplace." Cool jobs have work environments. Keep company identity small if you want real diversity.

The Parliamentary Model As The Correct Ethical Model by Kaj Sotala - An explanation of how the 'parliamentary' model of morality resolves uncertainty around which model of morality is correct. Why the parliamentary model is itself the correct model.

The Problem With Prestige by Robin Hanson - Small fields such as academic disciplines often use prestige to reward people. A mathematical model of how effort is allocated to maximize prestige. Why prestige doesn't scale and what is under-incentivized by prestige.

How I Think About Free Speech Four Categories by Julia Galef - Descriptions of the following categories: No consequences, Individual social consequences, Official social consequences, Legal consequences. Disagreements about categories.

Choices Are Really Bad by Zvi Moshowitz - Exercising willpower is a cost in the short term. Decision fatigue. Reasons why people, including you, WILL choose wrong. People justify their choices. Choices create blame and responsibility. Choices cause paralysis. Choice are communication. Choices require justification. Choices let people defect and destroy cooperation.

What Is Rationalist Berkleys Community Culture by Zvi Moshowitz - The original rationalist community mission was to save the world, not to be nice to each other. Sarah recently suggested the later is currently the actual goal. Zvi reinterprets this as sounding an alarm. The rationalists should not become just another Berkeley community of bohemians and weirdos.

Repairing Anxiety Using Internal And External Locus Of Control Models by Elo - Two variable model. Locus of Control: Internal or External. Feeling: Good or bad. The four combinations. Moving diagonally, for example from internal-bad to external-good.

Social Insight When A Lie Is Not A Lie When A by Bound_Up (lesswrong) - If you merely speak the truth as you see then you will be misunderstood. Example of saying you are an atheist. Many people are incapable of understanding your real arguments.

Multiverse Wide Cooperation Via Correlated Decision Making by The Foundational Research Institute - "If we care about what happens in civilizations located elsewhere in the multiverse, we can superrationally cooperate with some of the their inhabitants. That is, if we take their values into account, this makes it more likely that they do the same for us. In this paper, I attempt to assess the practical implications of this idea"

Questions Are Not Just For Asking by Malcom Ocean (ribbonfarm) - Hazards of asking questions. Hold your Questions. Reveal your questions. Un-ask your questions. Question your questions. Using Questions to Organize Attention. Letting the question ask you; becoming the answer.

Happiness Is Not Coherent Concept by Particular Virtue - A social science concept is 'real' if and only if it represents reality well and you have ruled out alternatives. "If a thing can be measured several different ways, and a causal factor can push one in a direction but not the other, then you start to worry that the thing is not actually one thing, but several things." Why should you care that happiness isn't a single thing.

The Craft Is Not The Community by Sarah Constantin (Otium) - The Berkeley Rationalists are building a true community: Sharehouses, Plans for an unschooling center, etc. However many rationalist companies/projects have failed. Sarah doesn't think it makes sense to tackle 'external facing' projects as a community. Tesla Motors and MIT aren't run as community projects, they are run meriotocratically. Lots of analysis on the meaning of community and what makes organizations effective. Personal.


More On Dota 2 by Open Ai - Timeline of the DOTA-bot's rapid improvement. Bot Exploits. Physical Infrastructure. What needs to be done to play 5x5.

Openai Bots Were Defeated At Least 50 Times - People could play against the openAI Dota bot. Several people found strategies to beat the bot. One of the human victors explains their strategy.

Dota 2 by Open Ai - Open AI codes a 1v1 Dota-2 bot that defeaated top players. The bots actions per minute were comparable to many humans. The bot learned the game from scratch by self-play, and does not use imitation learning or tree search. The game involves hidden information and the bot's strategies were complicated.


Things I Have Gotten Wrong by Aceso Under Glass - Mistaken evaluations: Animal Charity Evaluators, Raising for Effective Giving, Charity Science, Tostan.

We Have No Idea If There Are Cost Effective Interventions Into Wild Animal Suffering by Ozy - Many people are confident there are no effective ways to reduce wild animal suffering, Ozy disagrees. Ecosystems are complex but we aren't completely uncertain. Wild Animal Suffering is a tiny field staffed by non-experts working part time.

Altruism Is Incomplete by Zvi Moshowitz - "I worry many in EA are looking at life like a game where giving money to charity is how the world scores victory points." Controls in psychology are often motivated by researcher bias. Amazon is the world's most effective charity. Life is about getting things done, often for selfish reasons. Veganism. Zvi doesn't believe the official EA party line.

Let Them Decide by GiveDirectly - Eight media articles about Basic Income, Give Directly, Cash Transfer and Development Aid.

High Time For Drug Policy Reform Part 44 by MichaelPlant (EA forum) - "This is the fourth of four posts on DPR. In this part I provide some simplistic but illustrative cost-effectiveness estimates comparing an imaginary campaign for DPR against current interventions for poverty, physical health and mental health; I also consider what EAs should do next."

High Time For Drug Policy Reform Policy by MichaelPlant (EA forum) - "This is the third of four posts on DPR. In this part I look at what a better approach to drug policy might be and then discuss how neglected and tractable this problem is as cause area of EAs to work on."

Drug Policy Reform 1 by MichaelPlant (EA forum) - 9300 Words. Six Mechanisms for drug reform to do good: Fighting mental illness. Reducing pain. Improving public health. Reducing crime, violence, corruption and instability (including international scale). Raising revenue for governments. Recreational use. Five major objections and the Author's response.

===Politics and Economics:

Diversity And Team Performance What The Research Says by Eukaryote - Opens with several links about diversity and inclusion in EA. The pros and cons of different types of diversity in terms of group cohesion and information processing. Practical ways to minimize the costs of diversity and magnify the benefits. Lots of references.

Unpopular Ideas About Social Norms by Julia Galef - Twenty-four ideas, many with references explaining the ideas. As an example: "Overall it would be a good thing to have a totally transparent society with no privacy"

Unpopular Ideas About Political And Economic Systems by Julia Galef - Twenty-three ideas, many with references explaining the ideas. As an example "Many people have a moral duty not to vote".

The Market Power Story by Noah Smith - Many issues in the American economy are blamed on the increasing market power of a small number of firms. Analysis: Monopolistic competition. Profits. Market Concentration. Output restriction. Three updates. Lots of citations and references to papers.

The Courage To Stand Up And Do The Wrong Thing by Tom Bartleby - According to Supreme Court Justice Black, applying Brown vs Board of Education to DC schools was an unprincipled but correct decision. Have principles. Don't follow them over a cliff. Acknowledge deviations. Charlottesville. Cloduflare suspends service to the daily stormer.

Many Topics by Scott Aaronson - Misc Topics: HTTPS / Kurtz / eclipse / Charlottesville / Blum / P vs. NP

The Muted Signal Hypothesis Of Online Outrage by Kaj Sotala - "People want to feel respected, loved, appreciated, etc. When we interact physically, you can easily experience subtle forms of these feelings... Online, most of these messages are gone: a thousand people might read your message, but if nobody reacts to it, then you don’t get any signal indicating that you were seen... . So if you want to consistently feel anything, you may need to ramp up the intensity of the signals."

Marching Markups by Robin Hanson - "Holding real productivity constant, if firms move up their demand curves to sell less at a higher prices, then total output, and measured GDP, get smaller. Their numerical estimates suggest that, correcting for this effect, there has been no decline in US productivity growth since 1965. That’s a pretty big deal."

Greater Gender Parity Economics Suggests Reform Tenure Systems by Marginal Revolution - Biological clocks conflict with the tenure system timeline. Tyler recommends a much more flexible system with a variety of roles. The leaders in the economics profession have been 'punching down' at an infamous anonymous economics forum.

Moral Precepts And Suicide Pacts by Perfecting Dated Visions - "To be trusted to remain peaceful, you must be the kind of person who remains peaceful. And to be a peaceful person and earn the trust placed in you, you must be peaceful even when you have every right to fight. It’s the same with tolerance. If you want to shut up your argumentative opponents and vigorously retaliate when your opponents show signs of intolerance, you will not be trusted to be tolerant to others who are tolerant, even those who basically agree with you." The constitution, World War 1, Nazi's today.

The Anti Slip Slope by samuelthefifth (Status 451) - An analogy between workplace noise and workplace sexism. How efforts to stamp out 'workplace noise' can get out of control.

Seattle Minimum Wage Study Part 3 Tell Me Why Im Wrong Please by Zvi Moshowitz - Most writers thought the Seattle minimum wage study showed that low wage workers were hurt. Zvi found a fundamental flaw in their analysis. If you correct for raising wages in Seattle then the study seems to show low wage workers weren't hurt or perhaps benefitted.

Theory Vs Data In Statistics by Noah Smith - Theory heavy vs minimal theory models in Economics. Machine learning as the extreme of a "no model required" paradigm.

Thats Amore by sam[]zdat - Epistocracy, democracy with limits on who can vote. Competency and incompetency and pizza. Politics is the strongest identity. Trading power for the image of power. Morlocks and Eloi. Replication crisis. Google guy. The Left's support for the powerful. Nhilism.

Contra Sadedin Varinsky: The Google Memo Is Still Right Again by Artir - Detailed refutation of two criticisms of the google memo. Lots of long quotations and citation of counter evidence.

Indian Feminism And The Role Of The Environment: Why The Google Memo Is Still Right by Artir - A very detailed cross-country look at female enrollment in CS and various technology fields. A focus on countries where women are well represented in tech (many in Asia). Lots of discussion.

Brief Thoughts On The Google Memo by Julia Galef - "So as far as I can see, there are only two intellectually honest ways to respond to the memo: 1. Acknowledge gender differences may play some role, but point out other flaws in his argument (my preference) 2. Say “This topic is harmful to people and we shouldn’t discuss it” (a little draconian maybe, but at least intellectually honest)"

The Kolmogorov Option by Scott Aaronson - Kolmogorov was a brilliant mathematician as well as a sensitive and kind man. However he cooperated with the Soviets. An option for living in a society where many falsehoods are 'official truth': Build a bubble of truth and wait for the right time to take down the Orthodoxy. Don't charge headfirst and get killed. There are no 'good heretics' in the eyes of the Inquisition.


Can Atheists be Jewish by Brute Reason - Reasons MIRI can be an atheist Jew: Judaism is a religion, but being Jewish isn’t necessarily. Belief in god isn’t particularly central in most Jewish communities and practices. Because I fucking said so.

Ten Small Life Improvements by Paul Christiano (lesswrong) - Nine tech tips. Christmas lights all year round.

Extremely Easy Problem by protokol2020 - How much water per second do you need to raise the sea level 6 meters in 100 years.

The Premium Mediocre Life Of Maya Millennial by venkat (ribbonfarm) - Venkat - "Yes, ribbonfarm is totally premium mediocre. We are a cut above the new media mediocrityfests that are Vox and Buzzfeed, and we eschew low-class memeing and listicles. But face it: actually enlightened elite blog readers read Tyler Cowen and Slatestarcodex."

Right And Left Folds Primitive Recursion Patterns In Python And Haskell by Eli Bendersky - "In this article I'll present how left and right folds work and how they map to some fundamental recursive patterns. The article starts with Python, which should be (or at least look) familiar to most programmers. It then switches to Haskell for a discussion of more advanced topics like the connection between folding and laziness, as well as monoids."

Meta Contrarian Typography Part 2 by Tom Bartleby - You should use two spaces after your sentences whn drafting. Why to use a plaintext editor. Why to write a resume in plaintext. Flexibility is power. Two spaces is more much more machine readable.

Stop Caring So Much About Technical Problems by Particular Virtue - Links to an article describing what attributes actually get developers jobs (other than technical skill). Caring about making great products is much more desirable than caring about technical problems. Developer interviews are highly random. Experience matters alot. Enterprise programmers are disliked. Practical advice.

Trip Sitting Tips And Tricks by AellaGirl - Thirteen practical tips for trip sitting someone on a high dose of acid. Focuses on accepting their experiences, treating them similarly to a small child and keeping yourself safe.

Erisology Of Self And Will Closing Thoughts by Everything Studies - "Here in Part 7 I’ll end with a summary and some thoughts on how to deal with the problems described in the series."


We Are Not Worried Enough About The Next Pandemic by 80,000 Hours - "We spend the first 20 minutes covering his work as a foundation grant-maker, then discuss how bad the pandemic problem is, why it’s probably getting worse, and what can be done about it. In the second half of the interview we go through what you personally could study and where you could work to tackle one of the worst threats facing humanity."

Identity Terror by Waking Up with Sam Harris - "Douglas Murray. Identity politics, the rise of white nationalism, the events in Charlottesville, guilt by association, the sources of western values, the problem of finding meaning in a secular world."

Seth Stephens Davidowitz On What The Internet Can Tell Us by Rational Speaking - "New research gives us into which parts of the USA are more racist, what kinds of strategies reduce racism, whether the internet is making political polarization worse, and the sexual fetishes and insecurities people will only admit to their search engine."

John McWhorter on the Evolution of Language and Words on the Move by EconTalk - "The unplanned ways that English speakers create English, an example of emergent order. Topics discussed include how words get short (but not too short), the demand for vividness in language, and why Shakespeare is so hard to understand."

The Limits Of Persuasion by Waking Up with Sam Harris - "David Pizarro and Tamler Sommers. Free speech on campus, the Scott Adams podcast, the failings of the mainstream media, moral persuasion, moral certainty, the ethics of abortion, Buddhism, the illusion of the self."

Conversation: Comedian Dave Barry by Marginal Revolution - "What makes Florida special, why business writing is so terrible, Eddie Murphy, whether social conservatives can be funny (in public), the weirdness of Peter Pan, how he is so productive, playing guitar with Roger McGuinn, DT, the future of comedy."

Ritual And Spirituality by The Bayesian Conspiracy - Rationalist ritual. Witchcraft. Welcome to Nightvale. Concerts. What makes something ritual? Is rationalist ritual psychologically safe?

Chris Hayes by The Ezra Klein Show - Chris Hayes. Should Trump be removed from office. "Infighting between different factions of the Democratic Party, the signs that congressional Republicans are growing some backbone, and the reports that Trump’s closest aides are conspiring to keep him from doing too much damage to the country."

The Biology Of Good And Evil by Waking Up with Sam Harris - "Robert Sapolsky. His work with baboons, the opposition between reason and emotion, doubt, the evolution of the brain, the civilizing role of the frontal cortex, the illusion of free will, justice and vengeance, brain-machine interface, religion, drugs"

Senator Michael Bennet by The Ezra Klein Show - Senator Michael Bennet. "This is a conversation about why Congress is broken, and what broke it. We discuss money, partisanship, the media, the rules, the leadership, and much more. We talk about what Bennet thinks House of Cards gets right (hint: it’s the sociopathy) and whether President Trump’s antics are creating some hope of institutional renewal."

Book Review: Mathematics for Computer Science (Suggestion for MIRI Research Guide)

11 richard_reitz 22 July 2017 07:26PM

tl;dr: I read Mathematics for Computer Science (MCS) and found it excellent. I sampled Discrete Mathematics and Its Applications (Rosen)—currently recommended in MIRI's research guide—as well as Concrete Mathematics and Discrete Mathematics with Applications (Epp), which appear to be MCS's competition. Based on these partial readings, I found MCS to be the best overall text. I therefore recommend MIRI change the recommendation in its research guide.


MCS is used at MIT for their introductory discrete math course, 6.042, which appears to be taken primarily by second-semester freshman and sophomores. You can find OpenCourseWare archives from 2010 and 2015, although the book is self-contained; I never had occasion to use them throughout my reading. 

If you liked Computability and Logic (review), currently in the MIRI research guide, you'll like MCS:

MCS is a wonderful book. It's well written. It's rigorous, but does a nice job of motivating the material. It efficiently proves a number of counterintuitive results and then helps you see them as intuitively obvious. Freed from the constraint of printing cost, it contains many diagrams which are generally useful. You can find the pdf here or, if that link breaks, by googling "Mathematics for Computer Science". (See section 21.2 for why this works.)

MCS is regularly updated during the semester. Based on the dates of revision given to the cover, I suspect that the authors attempt to update it within a week of the last update during the semester. The current version is 87 pages longer than the 2015 version, suggesting ~40 pages of material is added a year. My favorite thing about the constant updates was that I never needed to double check statements about our current state of knowledge to see if anything had changed since publication.

MCS is licensed under a Creative Commons attribution share-alike license: it is free in the sense of both beer and freedom. I'm a big fan of such copyleft licenses, so I give MIT major props. I've tried to remain unbiased in my review, but halo effect suggests my views on the text might be affected by the text's license: salt accordingly.


The only prerequisite is single-variable calculus. In particular, I noted integration, differentiation, and convergence/infinite sums coming up. That said, I don't remember seeing them coming up in sections that provided a lot of dependencies: with just a first course in algebra, I feel a smart 14-year-old could get through 80–90% of the book, albeit with some help, mostly in places where "do a bunch of algebra" steps are omitted. An extra 4–5 years of practice doing algebraic manipulations makes a difference.

MCS is also an introduction to proofwriting. In my experience, writing mathematical proofs is a skill complex enough to require human feedback to get all the nuances of why something works and why something else doesn't work and why one approach is better than another. If you've never written proofs before and would like a human to give you feedback, please pm me.

Comparison to Other Discrete Math Texts


I randomly sampled section 4.3 of Rosen, on primes and greatest common divisors and was very unimpressed. Rosen states the fundamental theorem of arithmetic without a proof. The next theorem had a proof which was twice as long and half as elegant as it could have been. The writing was correct but unmotivating and wordy. For instance, Rosen writes "If n is a composite integer", which is redundant, since all composite numbers are integers, so he could have just said "If n is composite".

In the original Course Recommendations for Friendliness Researchers, Louie responded to Rosen's negative reviews:

people taking my recommendations would be geniuses by-and-large and that the harder book would be better in the long-run for the brightest people who studied from it.

Based on the sample I read, Rosen is significantly dumbed-down relative to MCS. Rosen does not prove the fundamental theorem of arithmetic whereas MCS proves it in section 9.4. For the next theorem, Rosen gives an inelegant proof when a much sleeker—but reasonably evident!—proof exists, making it feel like Rosen expected the reader to not be able to follow the sleeker proof. Rosen's use of "composite integer" instead of "composite" seems like he assumes the reader doesn't understand that the only objects one describes as composite are integers; MCS does not contain the string "composite integer".

In the section I read, Rosen has worked examples for finding gcd(24, 36) and gcd(17, 22), something I remember doing when I was 12. It's almost like Rosen was spoon-feeding how to guess the teacher's password for the student to regurgitate on an exam instead of building insight.

Concrete Mathematics

There are probably individuals who would prefer Concrete Mathematics to MCS. These people are probably into witchcraft.

I explain by way of example. In section 21.1.1, MCS presents a very sleek, but extremely nonobvious, proof of gambler's ruin using a clever argument courtesy of Pascal. In section 21.1.2, MCS gives a proof that doesn't require the reader to be "as ingenuious Pascal [sic]". As an individual who is decidedly not as ingenious as Pascal was, I appreciate this.

More generally, say we want to prove a theorem that looks something like "If A, then B has property C." You start at A and, appealing to the definition of C, show that B has it. There's probably some cleverness involved in doing so, but you start at the obvious place (A), end in the obvious place (B satisfies the definition of C), and don't rely on any crazy, seemingly-unrelated insights. Let's call this sort of proof mundane.

(Note that mundane is far from mechanical. Most of the proofs in Baby Rudin are mundane, but require significant cleverness and work to generate independently.)

There is a virtue in mundane proofs: a smart reader can usually generate them after they read the theorem but before they read its proof. Doing is beneficial, since proof-generating makes the theorem more memorable. It also gives the reader practice building intuition by playing around with the mathematical objects and helps them improve their proofwriting by comparing their output to a maximally refined proof.

On the end of the spectrum opposing mundane is witchcraft. Proofs that use witchcraft typically have a step where you demonstrate you're as ingenious as Pascal by having a seemingly-unrelated insight that makes everything easier. Notice that, even if you are as ingenious as Pascal, you won't necessarily be able to generate these insights quickly enough to get through the text at any reasonable pace.

For the reasons listed above, I prefer mundane proofs. This isn't to say MCS is devoid of witchcraft: sometimes it's the best or only way of getting a proof. The difference is that MCS uses mundane proofs whenever possible whereas Concrete Mathematics invokes witchcraft left and right. This is why I don't recommend it.

Individuals who are readily as ingenious as Pascal, don't want the skill-building benefits of mundane proofs, or prefer the whimsy of witchcraft may prefer Concrete Mathematics.


I randomly sampled section 12.2 of Epp and found it somewhat dry but wholly unobjectionable. Unlike Rosen, I felt like Epp was writing for an intelligent human being (though I was reading much further along in the book, so maybe Rosen assumed the reader was still struggling with the idea of proof). Unlike Concrete Mathematics, I detected no witchcraft. However, I felt that Epp had inferior motivation and was written less engagingly. Epp is also not licensed under Creative Commons.


Epp, Rosen, and MCS are all ~1000 pages long, whereas Concrete Mathematics is ~675. To determine what these books covered that might not be in MCS, I looked through their table of contents' for things I didn't recognize. The former three have the same core coverage, although Epp and Rosen go into material you would find in Computability and Logic or Sipser (also part of the research guide), whereas MCS spends more time developing discrete probability. Based on the samples I read, Epp and MCS have about the same density, whereas Rosen spends little time building insight and a lot of time showing how to do really basic, obvious stuff. I would expect Epp and MCS to have roughly the same amount of content covering mostly (but not entirely) the same stuff and Rosen to offer a mere shadow of the insight of the other two.

Concrete Mathematics seems to contain a subset of MCS's topics, but from the sections I read, I expect the presentation to be wildly different.


My only substantial complaint about MCS is that, to my knowledge, the source LaTeX is not available. Contrast this to SICP, which has the HTML available. This resulted in a proliferation of PDFs tailored for different use cases. It'd be nice, for instance, to have a print-friendly version of MCS (perhaps with fewer pages), plus a version that fit nicely onto the small screen of an ereader or mobile device, plus a version with the same aspect ratio as my monitor. This all would be extremely easy to generate given the source. It would also facilitate crowdsourcing proofreading: there are more than a few typos, although they don't preempt comprehension. At the very least, I wish there were somewhere to submit errata.

Some parts of MCS were notation-heavy. To quote what a professor once wrote on a problem set of mine:

I'm not sure all the notation actually serves the goal of clarifying the argument for the reader. Of course, such notation is sometimes needed. But when it is not needed, it can function as a tool with which to bludgeon the reader…

I found myself referring to Wikipedia's glossary of graph theory terms more than a few times when I was making definitions to put into Anki. Not sure if this is measuring a weak section or a really good glossary or something else.

A Note on Printing

A lot of people like printed copies of their books. One benefit of MCS I've put forward is that it's free (as in beer), so I investigated how much printing would cost.

I checked the local print shops and Kinko's online was unable to find printing under $60, a typical price around $70, with the option to burn $85 if I wanted nicer paper. This was more than I had expected and between ⅓ and ½ (ish) the price of Rosen or Epp.

Personally, I think printing is counterproductive, since the PDF has clickable links.

Final Thoughts

Despite sharing first names, I am not Richard Stallman. I prefer the license on MCS to the license on its competitors, but I wouldn't recommend it unless I thought the text itself was superior. I would recommend baby Rudin (nonfree) over French's Introduction to Real Analysis; Hoffman and Kunze's Linear Algebra (nonfree) over Jim Hefferson's Linear Algebra; and Epp over 2010!MCS. The freer the better, but that consideration is trumped by the quality of the text. When you're spending >100 hours working out of a book that provides foundational knowledge for the rest of your life, ~$150 and a loss of freedom is a price many would pay for better quality.

Eliezer writes:

Tell a real educator about how Earth classes are taught in three-month-sized units, and they would’ve sputtered and asked how you can iterate fast enough to learn how to teach that.

Rosen is in its seventh edition. Epp is in its fourth edition and Concrete Mathematics its second. The earliest copy of MCS I've happened across comes from 2004. Near as I can tell, it is improved every time the authors go through the material with their students, which would put it in its 25th edition.

And you know what? It's just going to keep getting better faster than anything else.


Thank you to Gram Stone for reviewing drafts of this review.

David C Denkenberger on Food Production after a Sun Obscuring Disaster

9 JenniferRM 17 September 2017 09:06PM

Having paid a moderate amount of attention to threats to the human species for over a decade, I've run across an unusually good thinker with expertise unusually suited to helping with many threats to the human species, that I didn't know about until quite recently.

I think he warrants more attention from people thinking seriously about X-risks.

David C Denkenberger's CV is online and presumably has a list of all his X-risks relevant material mixed into a larger career that seems to have been focused on energy engineering.

He has two technical patents (one for a microchannel heat exchanger and another for a compound parabolic concentrator) and interests that appear to span the gamut of energy technologies and uses.

Since about 2013 he has been working seriously on the problem of food production after a sun obscuring disaster, and he is in Lesswrong's orbit basically right now.

This article is about opportunities for intellectual cross-pollination!

continue reading »

AI Summer Fellows Program

9 abramdemski 06 August 2017 07:35AM

CFAR is running a free two-week program this September, aimed at increasing participant's ability to do technical research in AI alignment. Like the MIRI Summer Fellows Program which ran the past two years, this will include CFAR course material, plus content on AI alignment research and time to collaborate on research with other participants and researchers such as myself! It will be located in the SF Bay area, September 8-25. See more information and apply here.

People don't have beliefs any more than they have goals: Beliefs As Body Language

9 Bound_up 25 July 2017 05:41PM

Many people, anyway.


There is a common mistake in modeling humans, to think that they have actual goals, and that they deduce their actions from those goals. First there is a goal, then there is an action which is born from the goal. This is wrong.

More accurately, humans have a series of adaptations they execute. A series of behaviors which, under certain circumstances, kinda-sorta-maybe aim at the same-ish thing (like inclusive genetic fitness), but which will be executed regardless of whether or not they actually hit that thing.

Actions are not deduced from goals. The closest thing we have to "goals" are inferred from a messy conglomerate of actions, and are only an approximation of the reality that is there: just a group of behaviors.


I've come to see beliefs as very much the same way. Maybe some of us have real beliefs, real models. Some of us may in fact choose our statements about the world by deducing them from a foundational set of beliefs.

The mistake is repeated when we model most humans as having actual beliefs (nerds might be an exception). To suppose that their statements about reality, their propositions about the world, or their answers to questions are deduced from some foundational belief. First there is a belief, then there is a report on that belief, provided if anyone inquires about the belief they're carrying around. This is wrong.

More accurately, humans have a set of social moves/responses that they execute. Some of those moves APPEAR (to the naive nerd such as I) to be statements about how and what reality is. Each of these "statements" was probably vetted and accepted individually, without any consideration for the utterly foreign notion that the moves should be consistent or avoid contradiction. This sounds as tiresome to them as suggesting that their body language, or dance moves should be "consistent," for to them, the body language, dance moves, and "statements about reality" all belong to the same group of social moves, and thinking a social move is "consistent" is like thinking a certain posture/gesture is consistent or a color is tasty.

And missing the point like a nerd and taking things "literally" is exactly the kind of thing that reveals low social acuity.

Statements about individual beliefs are not deduced from a model of the world, just like actions are not deduced from goals. You can choose to interpret "I think we should help poor people" as a statement about the morality of helping poor people, if you want to miss the whole point, of course. We can suppose that "XYZ would be a good president" is a report on their model of someone's ability to fulfill a set of criteria. And if we interpret all their statement as though they were actual, REAL beliefs, we might be able to piece them together into something sort of like a model of the world.

All of which is pointless, missing the point, and counter-productive. Their statements don't add up to a model like ours might, anymore than our behaviors really add up to a goal. The "model" that comes out of aggregating their social learned behaviors will likely be inconsistent, but if you think that'll matter to them, you've fundamentally misunderstood what they're doing. You're trying to find their beliefs, but they don't HAVE any. There IS nothing more. It's just a set of cached responses. (Though you might find, if you interpret their propositions about reality as signals about tribal affiliation and personality traits, that they're quite consistent).

"What do you think about X" is re-interpreted and answered as though you had said "What do good, high-status groups (that you can plausibly be a part of) think about X?"

"I disagree" doesn't mean they think your model is wrong; they probably don't realize you have a model. Just as you interpret their social moves as propositional statements and misunderstand, so they interpret your propositional statements as social moves and misunderstand. If you ask how their model differs from yours, it'll be interpreted as a generic challenge to their tribe/status, and they'll respond like they do to such challenges. You might be confused by their hostility, or by how they change the subject. You think you're talking about X and they've switched to Y. While they'll think you've challenged them, and respond with a similar challenge, the "content" of the sentences need not be considered; the important thing is to parry the social attack and maybe counter-attack. Both perspectives make internal sense.

As far as they're concerned, the entire meaning of your statement was basically equivalent to a snarl, so they snarled back. Beliefs As Body Language.

Despite the obvious exceptions and caveats, this has been extremely useful for me in understanding less nerdy people. I try not to take what to them are just the verbal equivalent of gestures/postures or dance moves, and interpret them as propositional statements about the nature of reality (even though they REALLY sound like they're making propositional statements about the nature of reality), because that misunderstands what they're actually trying to communicate. The content of their sentences is not the point. There is no content. (None about reality, that is. All content is social). They do not HAVE beliefs. There's nothing to report on.

MILA gets a grant for AI safety research

9 Dr_Manhattan 21 July 2017 03:34PM

The really good news is that Yoshua Bengio is leading this (he is extremely credible in modern AI/deep learning world), and this is a pretty large change of mind for him. When I spoke to him at a conference 3 years ago he was pretty dismissive of the whole issue; this year's FLI conference seems to have changed his mind (kudos to them)

Of course huge props to OpenPhil for pursuing this

Publication of "Anthropic Decision Theory"

8 Stuart_Armstrong 20 September 2017 03:41PM

My paper "Anthropic decision theory for self-locating beliefs", based on posts here on Less Wrong, has been published as a Future of Humanity Institute tech report. Abstract:

This paper sets out to resolve how agents ought to act in the Sleeping Beauty problem and various related anthropic (self-locating belief) problems, not through the calculation of anthropic probabilities, but through finding the correct decision to make. It creates an anthropic decision theory (ADT) that decides these problems from a small set of principles. By doing so, it demonstrates that the attitude of agents with regards to each other (selfish or altruistic) changes the decisions they reach, and that it is very important to take this into account. To illustrate ADT, it is then applied to two major anthropic problems and paradoxes, the Presumptuous Philosopher and Doomsday problems, thus resolving some issues about the probability of human extinction.

Most of these ideas are also explained in this video.

To situate Anthropic Decision Theory within the UDT/TDT family: it's basically a piece of UDT applied to anthropic problems, where the UDT approach can be justified by using generally fewer, and more natural, assumptions than UDT does.

[Link] Stanislav Petrov has died (2017-05-19)

8 fortyeridania 18 September 2017 03:13AM

New business opportunities due to self-driving cars

8 chaosmage 06 September 2017 08:07PM

This is a slightly expanded version of a talk presented at the Less Wrong European Community Weekend 2017.

Predictions about self-driving cars in the popular press are pretty boring. Truck drivers are losing their jobs, self-driving cars will be more rented than owned, transport becomes cheaper, so what. The interesting thing is how these things change the culture and economy and what they make possible.

I have no idea about most of this. I don't know if self-driving cars accelerate or decelerate urbanization, I don't know how public transport responds, I don't even care which of the old companies survive. What I do think is somewhat predictable is some of the business opportunities become economical that previously weren't. I disregard retail, which would continue moving to online retail at the expense of brick and mortar stores even if the FedEx trucks would continue to be driven by people.

Diversification of vehicle types

A family car that you own has to be somewhat good at many different jobs. It has to get you places fast. It has to be a thing that can transport lots of groceries. It has to take your kid to school.

With self-driving cars that you rent for each seperate job, you want very different cars. A very fast one to take you places. A roomy one with easy access for your groceries. And a tiny, cute, unicorn-themed one that takes your kid to school.

At the same time, the price of autonomy is dropping faster than the price of batteries, so you want the lowest mass car that can do the job. So a car that is very fast and roomy and unicorn-themed at the same time isn't economical.

So if you're an engineer or a designer, consider going into vehicle design. There's an explosion of creativity about to happen in that field that will make it very different from the subtle iterations in car design of the past couple of decades.

Who wins: Those who design useful new types of autonomous vehicles for needs that are not, or badly, met by general purpose cars.

Who loses: Owners of general purpose cars, which lose value rapidly.

Services at home

If you have a job where customers come to visit you, say you're a doctor or a hairdresser or a tattoo artist, your field of work is about to change completely. This is because services that go visit the customer outcompete ones that the customer has to go visit. They're more convenient and they can also easily service less mobile customers. This already exists for rich people: If you have a lot of money, you pay for your doctor's cab and have her come to your mansion. But with transport prices dropping sharply, this reaches the mass market.

This creates an interesting dynamic. In this kind of job, you have some vague territory - your customers are mostly from your surrounding area and your number of competitors inside this area is relatively small. With services coming to the home, everyone's territories become larger, so more of them overlap, creating competition and discomfort. I believe the typical solution, which reinstates a more stable business situation and requires no explicit coordination, is increased specialization within your profession. So a doctor might be less of her district's general practitioner and more of her city's leading specialist in one particular illness within one particular demographic. A hairdresser might be the city's expert for one particular type of haircut for one particular type of hair. And so on.

Who wins: Those who adapt quickly and steal customers from stationary services.

Who loses: Stationary services and their landlords.

Rent anything

You will not just rent cars, you will rent anything that a car can bring to your home and take away again. You don't go to the gym, you have a mobile gym visit you twice a week. You don't own a drill that sits unused 99,9% of the time, you have a little drone bring you one for an hour for like two dollars. You don't buy a huge sound system for your occasional party, you rent one that's even huger and on wheels.

Best of all, you can suddenly have all sort of absurd luxuries, stuff that previously only millionaires or billionaires would afford, provided you only need it for an hour and it fits in a truck. The possibilities for business here are dizzying.

Who wins: People who come up with clever business models and the vehicles to implement them.

Who loses: Owners and producers of infrequently used equipment.

Self-driving hotel rooms

This is a special case of the former but deserves its own category. Self-driving hotel rooms replace not just hotel rooms, but also tour guides and your holiday rental car. They drive you to all the tourist sites, they stop at affiliated restaurants, they occasionally stop at room service stations. And on the side, they do overnight trips from city to faraway city, competing with airlines.

Who wins: The first few companies who perfect this.

Who loses: Stationary hotels and motels.

Rise of alcoholism and drug abuse

Lots of people lack intrinsic motivation to be sober. They basically can't decide against taking something. Many of them currently make do with extrinsic motivation: They manage to at least not drink while driving. In other words, for a large number of people, driving is their only reason not to drink or do drugs. That reason is going away and consumption is sure to rise accordingly.

Hey I didn't say all the business opportunities were particularly ethical. But if you're a nurse or doctor, if you go into addiction treatment you're probably good.

Who wins: Suppliers of mind-altering substances and rehab clinics.

Who loses: The people who lack intrinsic motivation to be sober, and their family and friends.

Autonomous boats and yachts

While there's a big cost advantage to vehicle autonomy in cars, it is arguably even bigger in boats. You don't need a sailing license, you don't need to hire skilled sailors, you don't need to carry all the room and food those sailors require. So the cost of going by boat drops a lot, and there's probably a lot more traffic in (mostly coastal) waters. Again very diverse vehicles, from the little skiff that transports a few divers or anglers to the personal yacht that you rent for your honeymoon. This blends into the self-driving hotel room, just on water.

Who wins: Shipyards, especially the ones that adapt early.

Who loses: Cruise ships and marine wildlife.

Mobile storage

The only reason we put goods in warehouses is that it is too expensive to just leave them in the truck all the way from the factory to the buyer. That goes away as well, although with the huge amounts of moved mass involved this transition is probably slower than the others. Shipping containers on wheels already exist.

Who wins: Manufacturers, and logistics companies that can provide even better just in time delivery.

Who loses: Intermediate traders, warehouses and warehouse workers.

That's all I got for now. And I'm surely missing the most important innovation that self-driving vehicles will permit. But until that one becomes clear, maybe work with the above. All of these are original ideas that I haven't seen written down anywhere. So if like one of these and would like to turn it into a business, you're a step ahead of nearly everybody right now and I hope it makes you rich. If it does, you can buy me a beer. :-)

Heuristics for textbook selection

8 John_Maxwell_IV 06 September 2017 04:17AM

Back in 2011, lukeprog posted a textbook recommendation thread.  It's a nice thread, but not every topic has a textbook recommendation.  What are some other heuristics for selecting textbooks besides looking in that thread?

Amazon star rating is the obvious heuristic, but it occurred to me that Amazon sales rank might actually be more valuable: It's an indicator that profs are selecting the textbook for their classes.  And it's an indicator that the textbook has achieved mindshare, meaning you're more likely to learn the same terminology that others use.  (But there are also disadvantages of having the same set of mental models that everyone else is using.)

Somewhere I read that Elements of Statistical Learning was becoming the standard machine learning text partially because it's available for free online.  That creates a wrinkle in the sales rank heuristic, because people are less likely to buy a book if they can get it online for free.  (Though Elements of Statistical Learning appears to be a #1 bestseller on Amazon, in bioinformatics.)

Another heuristic is to read the biographies of the textbook authors and figure out who has the most credible claim to expertise, or who seems to be the most rigorous thinker (e.g. How Brands Grow is much more data-driven than a typical marketing book).  Or try to figure out what text the most expert professors are choosing for their classes.  (Oftentimes you can find the syllabi of their classes online.  I guess the naive path would probably look something like: go to US News to see what the top ranked universities are for the subject you're interested in.  Look at the university's course catalog until you find the course that covers the topic you want to learn.  Do course_id on Google in order to find the syllabus for the most recent time that course was taught.)

The Reality of Emergence

8 DragonGod 19 August 2017 09:58PM

Social Insight: When a Lie Is Not a Lie; When a Truth Is Not a Truth

8 Bound_up 11 August 2017 06:28PM

//The point has already been made, that if you wish to truly be honest, it is not enough to speak the truth.

I generally don't tell people I'm an atheist (I describe my beliefs without using any common labels). Why? I know that if I say the words "I am an atheist," that they will hear the following concepts:

- I positively believe there is no God

- I cannot be persuaded by evidence any more than most believers can be persuaded by evidence, ie, I have a kind of faith in my atheism

- I wish to distance myself from members of religious tribes

As I said, the point has already been made; If I know that they will hear those false ideas when I say a certain phrase, how can I say I am honest in speaking it, knowing that I will cause them to have false beliefs? Hence the saying, if you wish to protect yourself, speak the truth. If you wish to be honest, speak so that truth will be heard.

Many a politician convincingly lies with truths by saying things that they know will be interpreted in a certain positive (and false) way, but which they can always defend as having been intended to convey some other meaning.


The New

There is a counterpart to this insight, come to me as I've begun to pay more attention to the flow of implicit social communication. If speaking the truth in a way you know will deceive is a lie, then perhaps telling a lie in a way that you know will communicate a true concept is not a lie.

I've relaxed my standards of truth-telling as I've come to understand this. "You're the best" and "You can do this" statements have been opened to me, no qualifiers needed. If I know that everyone in a group has to say "I have XYZ qualification," but I also know that no one actually believes anybody when they say it, I can comfortably recite those words, knowing that I'm not actually leading anybody to believe false things, and thus, am not being dishonest.

Politicians use this method, too, and I think I'm more or less okay with it. You see, we have a certain problem that arises from intellectual inequality. There are certain truths which literally cannot be spoken to some people. If someone has an IQ of 85, you literally cannot tell them the truth about a great number of things (or they cannot receive it). And there are a great many more people who have the raw potential to understand certain difficult truths, but whom you cannot reasonably tell these truths (they'd have to want to learn, put in effort, receive extensive teaching, etc).

What if some of these truths are pertinent to policy? What do you do, say a bunch of phrases that are "true" in a way you will interpret them, but which will only be heard as...

As what? What do people hear when you explain concepts they cannot understand? If I had to guess, very often they interpret this as an attack on their social standing, as an attempt by the speaker to establish themselves as a figure of superior ability, to whom they should defer. You sound uppity, cold, out-of-touch, maybe nerdy or socially inept.

So, then...if you're socially capable, you don't say those things. You give up. You can't speak the truth, you literally cannot make a great many people hear the real reasons why policy Z is a good idea; they have limited the vocabulary of the dialogue by their ability and willingness to engage. 

Your remaining moves are to limit yourself to their vocabulary, or say something outside of that vocabulary, all the nuance of which will evaporate en route to their ears, and which will be heard as a monochromatic "I think I'm better than you."

The details of this dynamic at play go on and on, but for now, I'll just say that this is the kind of thing Scott Adams is referring to when he says that what Trump has said is "emotionally true" even if it "doesn't pass the fact checks" (see dialogue with Sam Harris).

In a world of inequality, you pick your poison. Communicate what truths can be received by your audience, a nerd, and stay out of elections.

Bi-Weekly Rational Feed

8 deluks917 24 July 2017 09:56PM

===Highly Recommended Articles:

Superintelligence Risk Project Update II by Jeff Kaufman - Jeff's thoughts and the sources he found most useful. Project is wrapping up in a few day. Topics: Technical Distance to AI. Most plausible scenarios of Superintelligence risk. OpenPhil's notes on how progress was potentially stalled in Cryonics and Nanotech.

Superintelligence Risk Project Update by Jeff Kaufman - Links to the three most informative readings on AI risk. Details on the large number of people Jeff has talked to. Three fundamental points of view on AI-Safety. Three Fundamental points of disagreement. An update on the original questions Jeff was trying to answer.

Podcast The World Needs Ai Researchers Heres How To Become One by 80,000 Hours - "OpenAI’s latest plans and research progress. Concrete Papers in AI Safety, which outlines five specific ways machine learning algorithms can act in dangerous ways their designers don’t intend - something OpenAI has to work to avoid. How listeners can best go about pursuing a career in machine learning and AI development themselves."

Radical Book Club The Decentralized Left by davidzhines (Status 451) - The nature of leftwing organizing and what righties can learn from it. An exposition of multiple books on radical left organization building. Major themes are "doing the work" and "decentralized leadership".

Study Of The Week To Remediate Or Not To Remediate by Freddie deBoer - Should low math proficiency students take remedial algebra or credit bearing statistics. The City University of New York ran an actual randomized study to test this. The study had pretty good controls. For example students were randomly assigned to three groups, participating professors taught one section of each group.

Kenneth Arrow On The Welfare Economics Of Medical Care A Critical Assessment by Artir (Nintil) - "Kenneth Arrow wrote a paper in 1963, Uncertainty and the Welfare Economics of Medical Care. This paper tends to appear in debates regarding whether healthcare can be left to the market (like bread), or if it should feature heavy state involvement. Here I explain what the paper says, and to what extent it is true."

Becoming Stronger Together by b4yes (lesswrong) - "About a year ago, a secret rationalist group was founded. This is a report of what the group did during that year."

The Destruction Of American Cuisine by Small Truths - America used to have a tremendous number of regional cuisines, most are dead. They were killed by supermarkets and frozen food. This has been costly both in terms of culture and health (antibiotic resistance, crop monoculture risk)


Targeting Meritocracy by Scott Alexander - Education and merit are different. Programming is one of the last meritocracies, this lets disadvantaged people get into the field. If a job is high impact we want to hire on merit. The original, literal meaning of meritocracy is important.

Classified Thread 2 Best In Classified by Scott Alexander - Scott is promoting a project to accelerate the trend of rationalists living near each other. There are four houses available for rent near Ward Street in Berkeley. Ward street is currently the rationalist hub in the Bay. Commenters can advertise other projects and services.

Url Of Sandwich by Scott Alexander - Standard links post, somewhat longer than usual.

Opec Thread by Scott Alexander - Bi-weekly open thread. Update on Scott and Katja's travels. Salt Lake City Meetup highlight. Topher Brennan is running for Senate.

Can We Link Perception And Cognition by Scott Alexander - SSC survey optical illusions. "So there seems to be a picture where high rates of perceptual ambiguity are linked to being weirder and (sometimes, in a very weak statistical way) lower-functioning." Speculation about fundamental connections between perception and cognitive style. Ideas for further research.

Change Minds Or Drive Turnout by Scott Alexander - Extreme candidates lower turnout among their own party. Is base turnout really the only thing that matters? Lots of quotes from studies.


Learning From Past Experiences by mindlevelup - "This is about finding ways to quickly learn from past experiences to inform future actions. We briefly touch upon different learning models." Model-based and Model-Free reinforcement learning. Practical advice and examples.

How Long Has Civilization Been Going by Elo (BearLamp) - Human agricultural society is only 342-1000 generations old. "Or when you are 24 years old you have lived one day for every year humans have had written records." Human civilization is only a few hundred lifetimes old.

Choices Are Bad by Zvi Moshowitz - Choices reduce perceived value. Choices require time and energy. Making someone choose is imposing a real cost.

Erisology Of Self And Will: The Gulf by Everything Studies - "Part 4 will discuss some scientific disciplines with bearing on the self, and how their results are interpreted differently by the traditional paradigm vs. the scientific."

Philosophy Vs Duck Tests by Robin Hanson - Focusing on deep structure vs adding up weak cues. If it looks like an x... More discussion of whether most people will consider ems people and/or conscious.

Knowing How To Define by AellaGirl - "These are three ways in which a word can be ‘defined’ – the role it plays in the world around it (the up-definition), synonyms (lateral-definition), and the parts which construct the thing (down-definition)." Applications to morality and free-will.

Change Is Bad by Zvi Moshowitz - "Change space, like mind space, is deep and wide. Friendly change space isn’t quite to change space what friendly mind space is to mind space, but before you apply any filters of common sense, it’s remarkably close." A long list of conditions that mean change has lower expected value. Why we still need to make changes. Keep your eyes open.

Meditation Insights Suffering And Pleasure Are Intrinsically Bound Together by Kaj Sotala - The concrete goal of meditation is to train your peripheral awareness. Much suffering comes from false promises of pleasure. Procrastinating to play a videogame won't actually make you feel better. Temptation losses its power once you truly see the temptations for what they truly are.

Be My Neighbor by Katja Grace - Katja lives in a rationalist house on ward street in Berkeley and its great. The next step up is a rationalist neighborhood. Katja is promoting the same four houses as Scott. Be her neighbor?

What Value Subagents by G Gordan (Map and Territory) - Splitting the mind into subagents is a common rationalist model (links to Alicorn, Briene Yudkowsky, etc). However the author preferred model is a single process with inconsistent preferences. Freud. System 1 and System 2. The rider and the Elephant become one. Subagents as masks. Subagents as epicycles.

The Order Of The Soul by Ben Hoffman (Compass Rose) - The philosophy of accepting things vs the impulse to reshape them. Many philosophical and psychological models split the soul into three. Internalized authority vs seeing the deep structure of moral reality. In some sense math is the easiest thing in the world to learn. School poisons the enjoyment of rational thought. Lockhart's lament. Feynman. Eichmann and thinking structurally.

Aliens Merely Sleeping by Tyler Cowen - The universe is currently too hot for artificial life to be productive. Advanced civilizations might be freezing themselves until the universe cools. "They could achieve up to 1030 times more than if done today" [short]

Book Reviews by Torello (lesswrong) - Rationalist Adjacent. Each book has an interesting 'ideas per page' rating. Homo Deus, Sapiens, Super-intelligence, Surfaces and Essences, What Technology Wants, Inside Jokes, A Skeptic's Guide to the Mind.

Geometers Scribes Structure Intelligence by Ben Hoffman (Compass Rose) - "How does spatial reasoning lead to formal, logical reasoning?" Fluid and crystalized intelligence. Some history of Philosophy. How social dynamics lead to the evolution of reasoning. Talmudic and Western law, and their oddities. Universal Grammar and connecting with the divine. FizzBuzz.

High Dimensional Societies by Robin Hanson - In high dimensional space the distance between points varies less. What implications does this have for 'spatial' social science models (ex analogues of 1D spectrums and 2D graphs).

Feelings In The Map by Elo (BearLamp) - Confusion is not a property of the external world. The same holds for many emotions. Non-violent communication and speaking from your own perspective.

Lesswrong Is Not About Forum Software by enye-word (lesswrong) - The best way to increase activity on lesswrong is to get back the top posters, especially Scott and Eliezer.

Explication by mindlevelup - "This essay is about explication, the notion of making things specific. I give some examples involving Next Actions and systematization. This might also just be obvious to many people. Part of it is also a rehash of Act Into Uncertainty. Ultimately, explication is about changing yourself."

Concrete Instructions by Elo (BearLamp) - "The objective test of whether the description is concrete is whether the description can be followed by an anonymous person to produce the same experience." Some examples including the 'paper folding game'.

Human Seems Low Dimensional by Robin Hanson - 'Humanness' seems to be a one dimensional variable. Hence people are likely to consider ems conscious and worthy of decent treatment since ems are human-like on many important factors. Some discussion of a study where people rated how human-like various entities were.

Erisology Of Self And Will: A Natural Offering by Everything Studies - A description of naturalism and it relation to science. Daniel Dennet. Many philosophers are still dualists about the self. The self as a composite. Freedom as emergent.

The Hungry Brain by Bayesian Investor - A short review that focuses on the basics of Guynet's ideas and meta-discussion of why Guynet included so much neuroscience. "Guyenet provides fairly convincing evidence that it’s simple to achieve a healthy weight while feeling full. (E.g. the 20 potatoes a day diet)."

Boost From The Best by Robin Hanson - [Age of Em] How many standard deviations above the mean will be the best em be? How much better will they be than the second best em? How much of a wage/leisure premium will the best em receive.

Becoming Stronger Together by b4yes (lesswrong) - "About a year ago, a secret rationalist group was founded. This is a report of what the group did during that year."

In Praise Of Fake Frameworks by Valentine (lesswrong) - "I use a lot of fake frameworks — that is, ways of seeing the world that are probably or obviously wrong in some important way. I think this is an important skill. There are obvious pitfalls, but I think the advantages are more than worth it. In fact, I think the "pitfalls" can even sometimes be epistemically useful."

Letter To Future Layperson by Sailor Vulcan (BYS) - A letter from someone in our age to someone post singularity. Description of the hardships and terrors of pre-singularity life. Emotional and poetic. ~5K words.


Conversation With An Ai Researcher by Jeff Kaufman - The anonymous researcher thinks AI progress is almost entirely driven by hardware and data. Back propagation has existed for a long time. Go would have taken at least 10 more years if go-aI work had remained constrained by academic budgets.

Openai Baselines PPO by Open Ai - "We’re releasing a new class of reinforcement learning algorithms, Proximal Policy Optimization (PPO), which perform comparably or better than state-of-the-art approaches while being much simpler to implement and tune. PPO has become the default reinforcement learning algorithm at OpenAI because of its ease of use and good performance."

Superintelligence Risk Project Update II by Jeff Kaufman - Jeff's thoughts and the sources he found most useful. Project is wrapping up in a few day. Topics: Technical Distance to AI. Most plausible scenarios of Superintelligence risk. OpenPhil's notes on how progress was potentially stalled in Cryonics and Nanotech.

Real Debate Robots Education by Tyler Cowen - Robots are already becoming part of the classroom. K-12 is an artificially creation anyway. Robots can help autistic or disabled children. Children sometimes trust robots too much.

Robust Adversarial Inputs by Open Ai - "We’ve created images that reliably fool neural network classifiers when viewed from varied scales and perspectives. This challenges a claim from last week that self-driving cars would be hard to trick maliciously since they capture images from multiple scales, angles, perspectives, and the like."

What Is Overfitting Exactly by Andrew Gelman - "If your model is correct, “overfitting” is impossible. In its usual form, “overfitting” comes from using too weak of a prior distribution."

Conversation With Bryce Wiedenbeck by Jeff Kaufman - "AGI is possible, it could be a serious problem, but we can't productively work on it now." AGI will look very different from current technologies. Utility functions are a poor model of human behavior.

Examples Of Superintelligence Risk by Jeff Kaufman - A series of extended quotes describing ways AI with innocent seeming goals can destroy the world. Authors: Nick Bostrom, Eliezer (and collaborators), Luke M, 80K hours, Tim Urban. Jeff finds them unpersuasive and asks for better ones. Lots of interesting comments. Eleizer himself comments describing how 'paperclip maximizers' might realistically occur.

Superintelligence Risk Project Update by Jeff Kaufman - Links to the three most informative readings on AI risk. Details on the large number of people Jeff has talked to. Three fundamental points of view on AI-Safety. Three Fundamental points of disagreement. An update on the original questions Jeff was trying to answer.

Conversation With Michael Littman by Jeff Kaufman - CS Professor at Brown's opinions: Deep Learning is surprisingly brittle in his experience. General Intelligence will require large fundamental advances. The AI risk community isn't testing their ideas so they probably aren't making real progress.


EAGX Relaunch by Roxanne_Heston (EA forum) - The EA global satellite EAGA-X conferences have been low activity. Changes: More funding and flexibility. Standardized formats. Fewer groups approved. Stipends to primary organizers.

Uncertainty Smoothes Out Differences In Impact by The Foundational Research Institute - Many inside view evaluations conclude that one intervention is orders of magnitude more effective than another. Uncertainty significantly reduces these ratios.

Autonomy: A Search For A Measure Will Pearson (EA forum) - "I shall introduce a relatively formal measure of autonomy, based on the intuition that it is the ability to do things by yourself with what you have. The measure introduced allows you to move from less to more autonomy, without being black and white about it. Then I shall talk about how increasing autonomy fits in with the values of movements such as poverty reduction, ai risk reduction and the reduction of suffering."

Eight media articles on GiveDirectly, Cash Transers and Basic Income.- A world where 8 men own as much wealth as 3.6 billion people by GiveDirectly -

More Giving Vs Doing by Jeff Kaufman - EA is moving far more money than it used to and the ramp up will continue. This means direct work has become relatively more valuable. Nonetheless giving money is still useful, capacity isn't being filled. Jeff plans on earning to give based on his personal constraints.

Why I Think The Foundational Research Institute by Mike Johnson (EA forum) - A description of the FRI. Good things about FRI. FRI's research framework and why the author is worried. Eight long objections. TLDR: "functionalism ("consciousness is the sum-total of the functional properties of our brains") sounds a lot better than it actually turns out to be in practice. In particular, functionalism makes it impossible to define ethics & suffering in a way that can mediate disagreements."

Tranquilism by The Foundational Research Institute - A paper arguing that reducing suffering is more important than promoting happiness. Axiology. Non-consciousness. Common Objections. Conclusion.

An Argument For Why The Future May Be Good by Ben West (EA forum) - Factory farming shows that humans are deeply cruel. Technology enabled this cruelty, perhaps the future will be even darker. Counterargument: Humans are lazy, not evil. Humans as a group will spend at least small amounts altruistically. In the future the cost of reducing suffering will go down low enough that suffering will be rare or non-existent.

Arguments Moral Advocacy by The Foundational Research Institute - "What does moral advocacy look like in practice? Which values should we spread, and how? How effective is moral advocacy compared to other interventions such as directly influencing new technologies? What are the most important arguments for and against focusing on moral advocacy?"

An Argument For Broad And Inclusive by Kaj Sotala (EA forum) - "I argue for a very broad, inclusive EA, based on the premise that the culture of a region is more important than any specific group within that region... As a concrete strategy, I propose a division into low-level and high-level EA"

Not Everybody wants a Goat by GiveDirectly - Eight links on GiveDirectly, Cash Transfers, Effective Altruism and Basic Income.

Mid Year Update by The GiveWell Blog - Encouraging more charities to apply. More research of potential interventions. Short operations recap. GiveWell is focusing more on outreach.

===Politics and Economics:

College Tuition by Tom Bartleby - Sticker prices for college have gone up 15K in twenty years, but the average actual cost has only gone up 2.5K. High prices are almost compensated by high aid. Advantage: more equitable access to education. Disadvantages: Not everyone knows about the aid, financial aid is large enough it can seriously distort family financial decisions.

War Of Wages Part 1 Apples And Walmarts by Jacob Falkovich (Put A Number On It!) - The Author thinks minimum wage hurts the poor. Walmart can't afford higher wages. Copenhagan Interpretation of Ethics: Walmart helps the poor and gets blamed, Apple does nothing for the poor but avoids blame.

Links 10 by Artir (Nintil) - Tons of links. Economics, Psychology, AI, Philosophy, Misc.

Pretend Ask Answer by Ben Hoffman (Compass Rose) - A short dialogue about Patriarchy and the meaning of oppression. Defensive actions are often a response to bad faith from the other side. Its not ok to explicitly say you think your partner is arguing in bad faith.

Cultural Studies Ironically Is Something Of A Colonizer by Freddie deBoer - An origin story for Writing Studies. The fields initial methodological diversity. Cultural studies took over the field, empirical work has been pushed out. Evidence that some cultural studies professors really do believe its fundamentally bigoted to do science and empirical research endangers marginalized students. The field has become insular.

The Dark Arts Examples From The Harris Adams Debate by Stabilizer (lesswrong) - The author accuses Scott Adams of using various dark Arts: Changing the subject, Motte-and-bailey, Euphemisation, Diagnosis, Excusing, Cherry-picking evidence.

Study Of The Week Modest But Real Benefits From Lead Exposure Interventions by Freddie deBoer - Freddie reviews a survey he found via SSC. The study had very good controls. Methodology is explained and key graphs are posted and discussed. Scott and Freddie seem to agree on the facts but have a different opinion on how large to consider the effects.

Descriptive And Prescriptive Standards by Simon Penner (Status 451) - Leadership means winning the Keynesian Beauty Contest. Public opinion doesn't exist as a stable reality. Prescribing public opinion. Dangers of social reform and leaders twisting the facts to promote noble outcomes.

A Taylorism For All Seasons by Lou (sam[]zdat) - "Christopher Lasch – The Culture of Narcissism, part 1/X, current essay being more of an overview." A Masquerade where you must act out the mask you choose.

Mechanism Agnostic Low Plasticity Educational Realism by Freddie deBoer - Freddie's educational philosophy. People sort into persistent academic strata. Educational attainment is heavily determined by factors outside of school's control. The mechanism differences in academic ability is unknown. Social and political implications.

Kin Aesthetics Excommunicate Me From The Church Of Social Justice by Frances Lee - A SJ-insider's critical opinion of SJ. Fear of being impure. Original Sin. Reproducing colonial structures of power and domination within social justice. Everyday Feminism's belittling articles. More humility. Bringing humanity to everyone, even those who have been inhumane.

Study Of The Week To Remediate Or Not To Remediate by Freddie deBoer - Should low math proficiency students take remedial algebra or credit bearing statistics. The City University of New York ran an actual randomized study to test this. The study had pretty good controls. For example students were randomly assigned to three groups, participating professors taught one section of each group.

Should We Build Lots More Housing In San Francisco: Three Reasons People Disagree by Julia Galef - For each of the three reasons Julia describes multiple sub-reasons. More housing might not lower prices much. More housing won't help the poor. NIMBY objections might be legitimate.

Kenneth Arrow On The Welfare Economics Of Medical Care A Critical Assessment by Artir (Nintil) - "Kenneth Arrow wrote a paper in 1963, Uncertainty and the Welfare Economics of Medical Care. This paper tends to appear in debates regarding whether healthcare can be left to the market (like bread), or if it should feature heavy state involvement. Here I explain what the paper says, and to what extent it is true."

Thoughts On Doxxing by Ozy (Thing of Things) - CNN found the identity of the guy who made the video of Trump beating up CNN. They implied they would dox him if he continued being racist. Is doxxing him ok? What about doxxing someone who runs r/jailbait? Ozy discusses the practical effect of doxxing and unleashing hate mobs.

On The Seattle Minimum Wage Study Part 2 by Zvi Moshowitz - Several relevant links are included. Seattle's economic boom and worker composition changes are important factors. Zvi dives deep into the numbers and tries to resolve an apparent contradiction.

Radical Book Club The Decentralized Left by davidzhines (Status 451) - The nature of leftwing organizing and what righties can learn from it. An exposition of multiple books on radical left organization building. Major themes are "doing the work" and "decentralized leadership".

On The Seattle Minimum Wage Study Part 1 by Zvi Moshowitz - The claimed effect sizes are huge. Zvi's priors about the minimum wage. Detailed description of some of the paper's methods and how it handle potential issues. Discussion of the raw data. More to come in part 2.


Childcare II by Jeff Kaufman - A timeline of childcare for Jeff's two children. Methods: Staying at home, Daycare, Au pair, Nanny.

Easier Chess Problem by protokol2020 - How many pieces do you need to capture a black queen?

Book Review Mathematics For Computer Science by richard_reitz (lesswrong) - Why the text should be in the MIRI research guide. Intro. Prereqs. Detailed comparisons to similar texts. Complaints.

Information is Physical by Scott Aaronson - Is information is physical a contentful expression? Why 'physics is information' is tautological. A proposed definition. Double slit experiment. Observation in Quantum Mechanics. Information takes up a minimum amount of space. Entropy. Information has nowhere to go.

Book Review Working Effectively With Legacy Code By Michael C Feathers by Eli Bendersky - To improve code we must refactor, to refactor we have to test, making code testable may take heroic efforts. "The techniques described by the author are as terrible as the code they're up against."

The Ominouslier Roar Of The Bitcoin Wave by Artem and Venkat (ribbonfarm) - A video visualizing and audiolizing the bitcoin blockchain. A related dialogue.

From Monkey Neurons To The Meta Brain by Hal Morris (ribbonfarm) - Neurons that only fire in response to Jennifer Anniston. Mirror Neurons. Theory of Mind. The path from copying movement to human-level empathy. Infant development. Dreams as social simulator. Communicating with our models of other people. He rapidly accelerating and dangerous future. We need to keep our mind open to possibilities.

Newtonism Question by protokol2020 - Balancing Forces. Gravity problem.

Short Interview Writing by Tyler Cowen - Tyler Cowen's writing habits. Many concrete details such as when he writes and what program he uses. Some more general thoughts on writing such as Tyler's surprising answer to which are his favorite books on writing.

Unexpected by protokol2020 - Discussion of gaps between primes. "Say, that you have just sailed across some recordly wide composite lake and you are on a prime island again. What can you expect, how much wider will the next record lake be?"

Interacting With A Long Running Child Process In Python by Eli Bendersky - Using the subprocess module to run an http server. Solutions and analysis of common use cases. Lots of code.

4d Mate Problem by protokol2020 - How many queens do you need to get a checkmate in 4D chess.

The Destruction Of American Cuisine by Small Truths - America used to have a tremendous number of regional cuisines, most are dead. They were killed by supermarkets and frozen food. This has been costly both in terms of culture and health (antibiotic resistance, crop monoculture risk)


Sally Satel On Organ Donation by EconTalk - "The challenges of increasing the supply of donated organs for transplantation and ways that public policy might increase the supply." Tax Credits. The ethics of donor compensation.

Podcast The World Needs Ai Researchers Heres How To Become One by 80,000 Hours - "OpenAI’s latest plans and research progress. Concrete Papers in AI Safety, which outlines five specific ways machine learning algorithms can act in dangerous ways their designers don’t intend – something OpenAI has to work to avoid. How listeners can best go about pursuing a career in machine learning and AI development themselves."

88 Must We Accept A Nuclear North Korea by Waking Up with Sam Harris - "Mark Bowden and the problem of a nuclear-armed North Korea."

Triggered by Waking Up with Sam Harris - "Sam Harris and Scott Adams debate the character and competence of President Trump."

Conversation Atul Gawande by Tyler Cowen - The marginal value of health care, AI progress in medicine, fear of genetic engineering, whether the checklist method applies to marriage, FDA regulation, surgical regulation, Michael Crichton and Stevie Wonder, wearables, what makes him weep, Knausgaard and Ferrante, why surgeons leave sponges in patients.

Nneka Jones Tapia by The Ezra Klein Show - The first psychologist to run a prison. 30% of inmates have diagnosed mental health problems. Mental heath view of the penal system, balancing punishment and treatment, responsibility versus mental instability, the tension between what we use jail for and what we should use jail for.

Tamar Haspel by EconTalk - "Why technology helps make some foods inexpensive, how animals are treated, the health of the honey bee, and whether eggs from your backyard taste any better than eggs at the grocery."

From Cells To Cities by Waking Up with Sam Harris - "Biological and social systems scale, the significance of fractals, the prospects of radically extending human life, the concept of “emergence” in complex systems, the importance of cities, the necessity for continuous innovation"

Inside The World Of Supertraining: Mark Bell by Tim Feriss - "Mark’s most important lessons for building strength. How to avoid injury and breakdown. Lesser-known training techniques that nearly everyone overlooks. How Mark became a millionaire by offering his gym memberships for free."

Eddie Izzard by The Ezra Klein Show - 27 marathons in 27 days, process for writing jokes, why he wants to run for parliament, inspiration from Al Franken's, borrowing confidence from his future self. What he learned as a street performer, routines are based on history and anthropology, World War I, 'cake or death?'. His gender identity, and how he integrated it into his act early on, etc.

Martha Nussbaum by EconTalk - "The tension between acquiring power and living a life of virtue. Topics discussed include Hamilton's relationship with Aaron Burr, Burr's complicated historical legacy, and the role of the humanities in our lives."

Rs 188 Robert Kurzban On Being Strategically Wrong by Rationally Speaking - Why Everyone (Else) is a Hypocrite." The "modular mind" hypothesis, and how it explains hypocrisy, self-deception, and other seemingly irrational features of human nature.

Toy model of the AI control problem: animated version

7 Stuart_Armstrong 10 October 2017 11:12AM

Crossposted at LessWrong 2.0.

A few years back, I came up with a toy model of the AI control problem. It has a robot moving boxes into a hole, with a slightly different goal than it's human designers, and a security camera to check that it's behaving as it should. The robot learns to block the camera to get its highest reward.

I've been told that the model is useful for explaining the control problem quite a few people, and I've always wanted to program the "robot" and get an animated version of it. Gwern had a live demo, but it didn't illustrate all the things I wanted to.

So I programmed the toy problem in python, and generated a video with commentary.

In this simplified version, the state space is sufficiently small that you can explicitly generate the whole table of Q-values (expected reward for taking an action in a certain state, assuming otherwise optimal policy). Since behaviour is deterministic, this can be updated in dynamical programming, using a full width backup. The number of such backups essentially measures the depth of the robot's predictive ability.

The most interesting depths of planning are:

  • At depth 0, the robot behaves randomly.
  • At depth 3, the robot crashes itself efficiently, to avoid loosing too much reward.
  • At depth 6, the robot figures out how to get one box into the hole. The automated camera turns it off.
  • At depth 17, the robot finally starts to deceive/conceal/manipulate, by blocking the camera and pushing two boxes into the hole. Note that the robot's deception comes from the fact that its incentives are misaligned, and that humans tried to control it.
  • At depth 18, the robot efficiently does the plan from depth 17.
  • At depth 20, the robot does the maximally efficient plan: blocking the camera, and pushing all boxes into the hole.
  • At depth 32, the robot has the correct Q-values for the maximally efficient plan.
  • At depth 45, finally, the Q-value table is fully updated, and the robot will take maximally efficient, and, if need be, deceptive plans from any robot/box starting positions.

The code and images can be found here.

Rational Feed

7 deluks917 17 September 2017 10:03PM

Note: I am trying out a weekly feed. 

===Highly Recommended Articles:

Superintelligence Risk Project: Conclusion by Jeff Kaufman - "I'm not convinced that AI risk should be highly prioritized, but I'm also not convinced that it shouldn't. Highly qualified researchers in a position to have a good sense the field have massively different views on core questions like how capable ML systems are now, how capable they will be soon, and how we can influence their development." There are links to all the previous posts. The final write up goes into some detail about MIRI's research program and an alternative safety paradigm connected to openAI.

On Bottlenecks To Intellectual Progress In The by Habryka (lesswrong) - Why LessWrong 2.0 is a project worth pursuing. A summary of the existing discussion around LessWrong 2.0. The models used to design the new page. Open questions.

Patriarchy Is The Problem by Sarah Constantin - Dominance hierarchies and stress in low status monkeys. Serotnonin levels and the abuse cycles. Complex Post Traumatic Stress Disorder. Submission displays. Morality-As-Submission vs. Morality-As-Pattern. The biblical God and the Golden Calf.

Ea Survey 2017 Series Donation Data by Tee (EA forum) - How Much are EAs Donating? Percentage of Income Donated. Donations Data Among EAs Earning to Give (who donated 57% of the total). Comparisons to 2014 and 2015. Donations totals were very heavily skewed by large donors.


Classified Thread 3 Semper Classifiedelis by Scott Alexander - " Post advertisements, personals, and any interesting success stories from the last thread". Scott's notes: Community member starting tutoring company, homeless community member gofundme, data science in North Carolina.

Toward A Predictive Theory Of Depression by Scott Alexander - "If the brain works to minimize prediction error, isn’t its best strategy to sit in a dark room and do nothing forever? After all, then it can predict its sense-data pretty much perfectly – it’ll always just stay “darkened room”." But why would low confidence cause sadness? Well, what, really, is emotion?

Promising Projects for Open Science To by SlateStarScratchpad - Scott answers what the most promising projects are in the field of transparent and open science and meta-science.

Ot84 Threadictive Processing by Scott Alexander - New sidebar ad for social interaction questions. Sidebar policy and feedback. Selected Comments: Animal instincts, the connectome, novel concepts encoded in the same brain areas across animals, hard coded fear of snakes, kitten's who can't see horizontal lines.


Peer Review Younger Think by Marginal Revolution - Peer Review as a concept only dates to the early seventies.

The Wedding Ceremony by Jacob Falkovich - Jacob gets married. Marriage is really about two agents exchanging their utility functions for the average utility function of the pair. Very funny.

Fish Oil And The Self Critical Brain Loop by Elo - Taking fish oil stopped ELO from getting distracted by a critical feedback loop.

Against Facebook The Stalking by Zvi Moshowitz - Zvi removes Facebook from his phone. Facebook proceeds to start emailing him and eventually starts texting him.

Postmortem: Mindlevelup The Book by mindlevelup - Estimates vs reality. Finishing both on-target and on-time. Finished product vs expectations. Took more time to write than expected. Going Against The Incentive Gradient. Impact evaluation. What Even is Rationality? Final Lessons.

Prepare For Nuclear Winter by Robin Hanson - Between nuclear war and natural disaster Robin estimates there is about a 1 in 10K chance per year that most sunlight is blocked for 5-10 years. This aggregates to about 1% per century. We have the technology to survive this as a species. But how do we preserve social order?

Nonfiction Ive Been Reading Lately by Particular Virtue - Selfish Reasons to Have More Kids. Eating Animals. Your Money Or Your Life. The Commitment.

Dealism by Bayesian Investor - "Under dealism, morality consists of rules / agreements / deals, especially those that can be universalized. We become more civilized as we coordinate better to produce more cooperative deals." Dealism is similar to contractualism with a larger set of agents and less dependence on initial conditions.

On Bottlenecks To Intellectual Progress In The by Habryka (lesswrong) - Why LessWrong 2.0 is a project worth pursuing. A summary of the existing discussion around LessWrong 2.0. The models used to design the new page. Open questions.

Lw 20 Open Beta Starts 920 by vanier (lesswrong) - The new site goes live on September 20th.

2017 Lesswrong Survey by ingres (lesswrong) - Take the survey! Community demographics, politics, Lesswrong 2.0 and more!

Contra Yudkowsky On Quidditch And A Meta Point by Tom Bartleby - Eliezer criticizes Quiditch in HPMOR. Why the snitch makes Quiditch great. Quidditch is not about winning matches its about scoring points over a series of games. Harry/Eliezer's mistake is the Achilles heel of rationalists. If lots of people have chosen not to tear down a fence you shouldn't either, even if you think you understand why the fence went up.

Whats Appeal Anonymous Message Apps by Brute Reason - Fundamental lack of honesty. Western culture is highly hostile to the idea that some behaviors (ex lying) might be ok in some contexts but not in others. Compliments. Feedback. Openness.

Meritocracy Vs Trust by Particular Virtue - "If I know you can reject me for lack of skill, I may worry about this and lose confidence. But if I know you never will, I may phone it in and stop caring about my actual work output." Trust Improves Productivity But So Does Meritocracy. Minimum Hiring Bars and Other Solutions.

Is Feedback Suffering by Gordan (Map and Territory) - The future will probably have many orders of magnitude more entities than today, and those entities may be very weird. How do we determine if the future will have order of magnitude more suffering? Phenomenology of Suffering. Panpsychism and Suffering. Feedback is desire but necessarily suffering. Contentment wraps suffering in happiness. Many things may be able to suffer.

Epistemic Spot Check Exercise For Mood And Anxiety by Aceso Under Glass - Outline: Evidence that exercise is very helpful and why, to create motivation. Setting up an environment where exercise requires relatively little will power to start. Scripts and advice to make exercise as unmiserable as possible. Scripts and advice to milk as much mood benefit as possible. An idiotic chapter on weight and food. Spit Check: Theory is supported, advice follows from theory, no direct proof the methods work.

Best Of Dont Worry About The Vase by Zvi Moshowitz - Zvi's best posts. Top5 posts for Marginal Revolution Readers. Top5 in general. Against Facebook Series. Choices are Bad series. Rationalist Culture and Ideas (for outsiders and insiders). Decision theory. About Rationality.


Superintelligence Risk Project: Conclusion by Jeff Kaufman - "I'm not convinced that AI risk should be highly prioritized, but I'm also not convinced that it shouldn't. Highly qualified researchers in a position to have a good sense the field have massively different views on core questions like how capable ML systems are now, how capable they will be soon, and how we can influence their development." There are links to all the previous posts. The final write up goes into some detail about MIRI's research program and an alternative safety paradigm connected to openAI.

Understanding Policy Gradients by Squirrel In Hell - Three perspectives on mathematical thinking: engineering/practical, symbolic/formal and deep understanding/above. Application of the theory to understanding policy gradients and reinforcement learning.

Learning To Model Other Minds by Open Ai - "We’re releasing an algorithm which accounts for the fact that other agents are learning too, and discovers self-interested yet collaborative strategies like tit-for-tat in the iterated prisoner’s dilemma."

Hillary Clinton On Ai Risk by Luke Muehlhauser - A quote by Hilary Clinton showing that she is increasingly concerned about AI risk. She thinks politicians need to stop playing catch-up with technological change.


Welfare Differences Between Cage And Cage Free Housing by Open Philosophy - OpenPhil funded several campaigns to promote cage free eggs. They now believe they were overconfident in their claims that a cage free system would be substantially better. Hen welfare, hen mortality, transition costs and other issues are discussed.

Ea Survey 2017 Series Donation Data by Tee (EA forum) - How Much are EAs Donating? Percentage of Income Donated. Donations Data Among EAs Earning to Give (who donated 57% of the total). Comparisons to 2014 and 2015. Donations totals were very heavily skewed by large donors.

===Politics and Economics:

Men Not Earning by Marginal Revolution - Decline in lifetime wages is rooted in lower wages at early ages, around 25. "I wonder sometimes if a Malthusian/Marxian story might be at work here. At relevant margins, perhaps it is always easier to talk/pay a woman to do a quality hour’s additional work than to talk/pay a man to do the same."

Great Wage Stagnation is Over by Marginal Revolution - Median household incomes rose by 5.2 percent. Gains were concentrated in lower income households. Especially large gains for hispanics, women living alone and immigrants. Some of these increases are the largest in decades.

There Is A Hot Hand After All by Marginal Revolution - Paper link and blurb. "We test for a “hot hand” (i.e., short-term predictability in performance) in Major League Baseball using panel data. We find strong evidence for its existence in all 10 statistical categories we consider. The magnitudes are significant; being “hot” corresponds to between one-half and one standard deviation in the distribution of player abilities."

Public Shaming Isnt As Bad As It Seems by Tom Bartleby - Online mobs are like shark attacks. Damore's economic prospects. Either targets are controversial and get support or uncontroversial and the outrage quickly abates. Justine Sacco. Success of public shaming is orthogonal to truth.

Hoe Cultures A Type Of Non Patriarchal Society by Sarah Constantin - Cultures that farmed with the plow developed classical patriarchy. Hoe cultures that practiced horticulture or large scale gardening developed different gender norms. In plow cultures women are economically dependent on me, in how cultures its the reverse. How cultures had more leisure but less material abundance. Hoe cultures aren't feminist.

Patriarchy Is The Problem by Sarah Constantin - Dominance hierarchies and stress in low status monkeys. Serotnonin levels and the abuse cycles. Complex Post Traumatic Stress Disorder. Submission displays. Morality-As-Submission vs. Morality-As-Pattern. The biblical God and the Golden Calf.

Three Wild Speculations From Amateur Quantitative Macro History by Luke Muehlhauser - Measuring the impact of the industrial revolution: Physical health, Economic well-being, Energy capture, Technological empowerment, Political freedom. Three speculations: Human wellbeing was terrible into the Industrial Revolution then rapidly improved. Most variance in wellbeing is captured by productivity and political freedom. It would take at least 15% of the world to die to knock the world off its current trajectory.

Whats Wrong With Thrive/Survive by Bryan Caplan - Unless you cherry-pick the time and place, it is simply not true that society is drifting leftward. A standard leftist view is that free-market "neoliberal" policies now rule the world. Radical left parties almost invariably ruled countries near the "survive" pole, not the "thrive" pole. You could deny that Communist regimes were "genuinely leftist," but that's pretty desperate. Many big social issues that divide left and right in rich countries like the U.S. directly contradict Thrive/Survive. Major war provides an excellent natural experiment for Thrive/Survive.

Gender Gap Stem by Marginal Revolution - Discussion of a recent paper. "Put (too) simply the only men who are good enough to get into university are men who are good at STEM. Women are good enough to get into non-STEM and STEM fields. Thus, among university students, women dominate in the non-STEM fields and men survive in the STEM fields."

Too Much Of A Good Thing by Robin Hanson - Global warming poll. Are we doing too much/little. Is it possible to do too little/much. "When people are especially eager to show allegiance to moral allies, they often let themselves be especially irrational."


Tim Schafer Videogame Roundup by Aceso Under Glass - Review and discussion of Psychonauts and Massive Chalice. Light discussion of other Schafer games.

Why Numbering Should Start At One by Artir - the author responds to many well known arguments in favor of 0-indexing.

Still Feel Anxious About Communication Every Day by Brute Reason - Setting boundaries. Telling people they hurt you. Doing these things without anxiety might be impossible, you have to do it anyway.

Burning Man by Qualia Computing - Write up of a Burning Man trip. Very long. Introduction. Strong Emergence. The People. Metaphysics. The Strong Tlön Hypothesis. Merging with Other Humans. Fear, Danger, and Tragedy. Post-Darwinian Sexuality and Reproduction. Economy of Thoughts about the Human Experience. Transcending Our Shibboleths. Closing Thoughts.

The Big List Of Existing Things by Everything Studies - Existence of fictional and possible people. Heaps and the Sorites paradox. Categories and basic building blocks. Relational databases. Implicit maps and territories. Which maps and concepts should we use?

Times To Die Mental Health I by (Status 451) - Personal thoughts on depression and suicide. "The depressed person is not seem crying all the time. It is in this way that the depressed person becomes invisible, even to themselves. Yet, positivity culture and the rise of progressive values that elude any conversation about suicide that is not about saving, occlude the unthinkable truth of someone’s existence, that they simply should not be living anymore."

Astronomy Problem by protokol2020 - Star-star occultation probability.


The Impossible War by Waking Up with Sam Harris - " Ken Burns and Lynn Novick about their latest film, The Vietnam War."

Is It Time For A New Scientific Revolution Julia Galef On How To Make Humans Smarter by 80,000 Hours - How people can have productive intellectual disagreements. Urban Design. Are people more rational than 200 years ago? Effective Altruism. Twitter. Should more people write books, run podcasts, or become public intellectuals? Saying you don't believe X won't convince people. Quitting an econ phd. Incentives in the intelligence community. Big institutions. Careers in rationality.

Parenting As A Rationalist by The Bayesian Conspiracy - Desire to protect kids is as natural as the need for human contact in general. Motivation to protect your children. Blackmail by threatening children. Parenting is a new sort of positive qualia. Support from family and friends. Complimenting effort and specific actions not general properties. Mindfulness. Treating kids as people. Handling kid's emotions. Non-violent communication.

The Nature Of Consciousness by Waking Up with Sam Harris - "The scientific and experiential understanding of consciousness. The significance of WWII for the history of ideas, the role of intuition in science, the ethics of building conscious AI, the self as an hallucination, how we identify with our thoughts, attention as the root of the feeling of self, the place of Eastern philosophy in Western science, and the limitations of secular humanism."

A16z Podcast On Trade by Noah Smith - Notes on a podcast Noah appeared on. Topics: Cheap labor as a substitute for automation. Adjustment friction. Exports and productivity.

Gillian Hadfiel by EconTalk - " Hadfield suggests the competitive provision of regulation with government oversight as a way to improve the flexibility and effectiveness of regulation in the dynamic digital world we are living in."

The Turing Test by Ales Fidr (EA forum) - Harvard EA podcast: "The first four episodes feature Larry Summers on his career, economics and EA, Irene Pepperberg on animal cognition and ethics, Josh Greene on moral cognition and EA, Adam Marblestone on incentives in science, differential technological development"

How long has civilisation been going?

7 Elo 22 July 2017 06:41AM

I didn't realise how short human history was.  Somewhere around 130,000 years ago we were standing upright as we are today.  Somewhere around 50,000 years ago we broadly arrived at:

the fully modern capacity for Culture *

That's roughly when we started, "routine use of bone, ivory, and shell to produce formal (standardized) artifacts".  Agriculture and humans staying still to grow plants happened at about 10,000BCE (or 12,000 years ago).

Writing started happening around 6600BCE* (8600 or so years ago).  

This year is 5777 in the Hebrew calendar.  So someone has been counting for roughly that long.

The pyramids are estimated to have been built at around 2600 BCE (4600 years ago)

Somewhere between then and zero by the christian calendar we sorted out a lot of metals and how to use them.

And somewhere between then and now we finished up all the technological advances that lead to present day.

But it's hard to get a feel for that.  Those are just some numbers of years.  Instead I want to relate that to our lives.  And our generations.

12,000 years ago is a good enough point to start paying attention to.

If a human generation is normally between 12* and 35* years.  Considering that further back the generations would have been closer to 12 years apart and today they are shifting to being more like 30 years apart (and up to 35 years apart).  That means the bounds are:

12,000/35 = 342

342-1000 generations.  That's all we have.  In all of humanity.  We are SO YOUNG!

(if you take the 8600 year number as a starting point you get a range of 717-242.)

Let's make it personal

I know my grandparents which means I am a not-negligible chance to also know my grandchildren and maybe even more (depending on medical technology).  I already have a living niece so I have already experienced 4 generations.  Without being unreasonable I can expect to see 5 and dream to see 6, 7 or infinite.  

(5/1000)->(7/342) = between a half a percent and two percent of human history.  I will have lived through 1/2% - 2% of human generations (ignoring longevity escape for a moment) to date.

Compared to other life numbers:

Days in a year * 100 year = 36,500 days in a 100 year lifespan.

52 weeks *100 = 5200.  Or one week of a 100 year lifespan is equivalent to one generation of humans.

12,000 years / 365 days = 32.8 years.  Or when you are 32 years old you have lived more days than humans have been collecting artefacts of worth.

8600 years/365 = 23.5 years.  Or when you are 24 years old you have lived one day for every year humans have had written records.

Discrete human lives

If you put an olden day discrete human life at 25 years - maybe more, and a modern day discrete life at 90 years and compare those to the numbers above

12,000/25 = 480 discrete human lifetimes

12,000/90=133 discrete human lifetimes

8600/25=344 discrete human lifetimes

8600/90=95 discrete human lifetimes

That's to say the entire of recorded history is only about 350 independent human lives stacked end on end.

Everything we know in history has been done on somewhere less than 480 discrete lifetime runthroughs.

Humanity is so young.  And we forget so easily that 500 lifetimes ago we were nothing.

Meta:  Thanks billy for hanging out and thinking about the numbers with me.  This idea came up on a whim and took a day of thinking about and about an hour to write up

Original post:

The Outside View isn't magic

6 Stuart_Armstrong 27 September 2017 02:37PM

Crossposted at Less Wrong 2.0.

The planning fallacy is an almost perfect example of the strength of using the outside view. When asked to predict the time taken for a project that they are involved in, people tend to underestimate the time needed (in fact, they tend to predict as if question was how long things would take if everything went perfectly).

Simply telling people about the planning fallacy doesn't seem to make it go away. So the outside view argument is that you need to put your project into the "reference class" of other projects, and expect time overruns as compared to your usual, "inside view" estimates (which focus on the details you know about the project.

So, for the outside view, what is the best way of estimating the time of a project? Well, to find the right reference class for it: the right category of projects to compare it with. You can compare the project with others that have similar features - number of people, budget, objective desired, incentive structure, inside view estimate of time taken etc... - and then derive a time estimate for the project that way.

That's the outside view. But to me, it looks a lot like... induction. In fact, it looks a lot like the elements of a linear (or non-linear) regression. We can put those features (at least the quantifiable ones) into a linear regression with a lot of data about projects, shake it all about, and come up with regression coefficients.

At that point, we are left with a decent project timeline prediction model, and another example of human bias. The fact that humans often perform badly in prediction tasks is not exactly new - see for instance my short review on the academic research on expertise.

So what exactly is the outside view doing in all this?


The role of the outside view: model incomplete and bias human

The main use of the outside view, for humans, seems to be to point out either an incompleteness in the model or a human bias. The planning fallacy has both of these: if you did a linear regression comparing your project with all projects with similar features, you'd notice your inside estimate was more optimistic than the regression - your inside model is incomplete. And if you also compared each person's initial estimate with the ultimate duration of their project, you'd notice a systematically optimistic bias - you'd notice the planning fallacy.

The first type of errors tend to go away with time, if the situation is encountered regularly, as people refine models, add variables, and test them on the data. But the second type remains, as human biases are rarely cleared by mere data.


Reference class tennis

If use of the outside view is disputed, it often develops into a case of reference class tennis - where people with opposing sides insist or deny that a certain example belongs in the reference class (similarly to how, in politics, anything positive is claimed for your side and anything negative assigned to the other side).

But once the phenomena you're addressing has an explanatory model, there are no issues of reference class tennis any more. Consider for instance Goodhart's law: "When a measure becomes a target, it ceases to be a good measure". A law that should be remembered by any minister of education wanting to reward schools according to improvements to their test scores.

This is a typical use of the outside view: if you'd just thought about the system in terms of inside facts - tests are correlated with child performance; schools can improve child performance; we can mandate that test results go up - then you'd have missed several crucial facts.

But notice that nothing mysterious is going on. We understand exactly what's happening here: schools have ways of upping test scores without upping child performance, and so they decided to do that, weakening the correlation between score and performance. Similar things happen in the failures of command economies; but again, once our model is broad enough to encompass enough factors, we get decent explanations, and there's no need for further outside views.

In fact, we know enough that we can show when Goodhart's law fails: when no-one with incentives to game the measure has control of the measure. This is one of the reasons central bank interest rate setting has been so successful. If you order a thousand factories to produce shoes, and reward the managers of each factory for the number of shoes produced, you're heading to disaster. But consider GDP. Say the central bank wants to increase GDP by a certain amount, by fiddling with interest rates. Now, as a shoe factory manager, I might have preferences about the direction of interest rates, and my sales are a contributor to GDP. But they are a tiny contributor. It is not in my interest to manipulate my sales figures, in the vague hope that, aggregated across the economy, this will falsify GDP and change the central bank's policy. The reward is too diluted, and would require coordination with many other agents (and coordination is hard).

Thus if you're engaging in reference class tennis, remember the objective is to find a model with enough variables, and enough data, so that there is no more room for the outside view - a fully understood Goodhart's law rather than just a law.


In the absence of a successful model

Sometimes you can have a strong trend without a compelling model. Take Moore's law, for instance. It is extremely strong, going back decades, and surviving multiple changes in chip technology. But it has no clear cause.

A few explanations have been proposed. Maybe it's a consequence of its own success, of chip companies using it to set their goals. Maybe there's some natural exponential rate of improvement in any low-friction feature of a market economy. Exponential-type growth in the short term is no surprise - that just means growth in proportional to investment - so maybe it was an amalgamation of various short term trends.

Do those explanations sound unlikely? Possibly, but there is a huge trend in computer chips going back decades that needs to be explained. They are unlikely, but they have to be weighed against the unlikeliness of the situation. The most plausible explanation is a combination of the above and maybe some factors we haven't thought of yet.

But here's an explanation that is implausible: little time-travelling angels modify the chips so that they follow Moore's law. It's a silly example, but it shows that not all explanations are created equal, even for phenomena that are not fully understood. In fact there are four broad categories of explanations for putative phenomena that don't have a compelling model:

  1. Unlikely but somewhat plausible explanations.
  2. We don't have an explanation yet, but we think it's likely that there is an explanation.
  3. The phenomenon is a coincidence.
  4. Any explanation would go against stuff that we do know, and would be less likely than coincidence.

The explanations I've presented for Moore's law fall into category 1. Even if we hadn't thought of those explanations, Moore's law would fall into category 2, because of the depth of evidence for Moore's law and because a "medium length regular technology trend within a broad but specific category" is something that has is intrinsically likely to have an explanation.

Compare with Kurzweil's "law of time and chaos" (a generalisation of his "law of accelerating returns") and Robin Hanson's model where the development of human brains, hunting, agriculture and the industrial revolution are all points on a trend leading to uploads. I discussed these in a previous post, but I can now better articulate the problem with them.

Firstly, they rely on very few data points (the more recent part of Kurzweil's law, the part about recent technological trends, has a lot of data, but the earlier part does not). This raises the probability that they are a mere coincidence (we should also consider selection bias in choosing the data points, which increases the probability of coincidence). Secondly, we have strong reasons to suspect that there won't be any explanation that ties together things like the early evolution of life on Earth, human brain evolution, the agricultural revolution, the industrial revolution, and future technology development. These phenomena have decent local explanations that we already roughly understand (local in time and space to the phenomena described), and these run counter to any explanation that would tie them together.


Human biases and predictions

There is one area where the outside view can still function for multiple phenomena across different eras: when it comes to pointing out human biases. For example, we know that doctors have been authoritative, educated, informed, and useless for most of human history (or possibly much worse than useless). Hence authoritative, educated, and informed statements or people are not to be considered of any value, unless there is some evidence the statement or person is truth tracking. We now have things like expertise research, some primitive betting markets, and track records to try and estimate their experience; these can provide good "outside views".

And the authors of the models of the previous section have some valid points where bias is concerned. Kurzweil's point that (paraphrasing) "things can happen a lot faster than some people think" is valid: we can compare predictions with outcomes. Robin has similar valid points in defense of the possibility of the em scenario.

The reason these explanations are more likely valid is because they have a very probable underlying model/explanation: humans are biased.



  • The outside view is a good reminder for anyone who may be using too narrow a model.
  • If the model explains the data well, then there is no need for further outside views.
  • If there is a phenomena with data but no convincing model, we need to decide if it's a coincidence or there is an underlying explanation.
  • Some phenomena have features that make it likely that there is an explanation, even if we haven't found it yet.
  • Some phenomena have features that make it unlikely that there is an explanation, no matter how much we look.
  • Outside view arguments that point at human prediction biases, however, can be generally valid, as they only require the explanation that humans are biased in that particular way.

Rational Feed: Last Week's Community Articles and Some Recommended Posts

6 deluks917 25 September 2017 01:41PM

===Highly Recommended Articles:

Why I Am Not A Quaker Even Though It Often Seems As Though I Should Be by Ben Hoffman - Quakers have consistently gotten to the right answers faster than most people, or the author. Arbitrage strategies to beat the quakers. An incomplete survey of alternatives.

Could A Neuroscientist Understand A Microprocessor by Rationally Speaking - "Eric Jonas, discussing his provocative paper titled 'Could a Neuroscientist Understand a Microprocessor?' in which he applied state-of-the-art neuroscience tools, like lesion analysis, to a computer chip. By applying neuroscience's tools to a system that humans fully understand he was able to reveal how surprisingly uninformative those tools actually are."

Reasonable Doubt New Look Whether Prison Growth Cuts Crime by Open Philosophy - Part1 of a four part, in depth, series on Criminal Justice reform. The remaining posts are linked below. "I estimate, that at typical policy margins in the United States today, decarceration has zero net impact on crime. That estimate is uncertain, but at least as much evidence suggests that decarceration reduces crime as increases it. The crux of the matter is that tougher sentences hardly deter crime, and that while imprisoning people temporarily stops them from committing crime outside prison walls, it also tends to increase their criminality after release. As a result, “tough-on-crime” initiatives can reduce crime in the short run but cause offsetting harm in the long run. Empirical social science research—or at least non-experimental social science research—should not be taken at face value. Among three dozen studies I reviewed, I obtained or reconstructed the data and code for eight. Replication and reanalysis revealed significant methodological concerns in seven and led to major reinterpretations of four. These studies endured much tougher scrutiny from me than they did from peer reviewers in order to make it into academic journals. Yet given the stakes in lives and dollars, the added scrutiny was worth it. So from the point of view of decision makers who rely on academic research, today’s peer review processes fall well short of the optimal."

Deterrence De Minimus by Open Philosophy - Part 2.

Incapacitation How Much Does Putting People Inside Prison Cut Crime Outside by Open Philosophy - Part 3.

Aftereffects Us Evidence Says Doing More Time Typically Leads More Crime After by Open Philosophy - Part 4.


L Dopen Thread by Scott Alexander - Bi-weekly public open thread. Berkeley SSC meetup. New ad for the Greenfield Guild, an online network of software consultants. Reasons to respect the society of friends.

Meditative States As Mental Feedback Loops by Scott Alexander - the main reason we don't see emotional positive feedback loops is that people get distracted. If you do not get distracted you can experience a bliss feedback look.

Book Review Mastering The Core Teachings Of The Buddha by Scott Alexander - "Buddhism For ER Docs. ER docs are famous for being practical, working fast, and thinking everyone else is an idiot. MCTB delivers on all three counts." Practical buddhism with a focus on getting things done. buddhism is split into morality concentration and wisdom. Discussion of "the Dark Night of the Soul" which is a sort of depression occurs when you have had some but not enough spiritual experience.


Impression Track Records by Katja Grace - Three reasons its better to keep impression track records and belief track records separate.

Why I Am Not A Quaker Even Though It Often Seems As Though I Should Be by Ben Hoffman - Quakers have consistently gotten to the right answers faster than most people, or the author. Arbitrage strategies to beat the quakers. An incomplete survey of alternatives.

The Best Self Help Should Be Self Defeating by mindlevelup - "Self-help is supposed to get people to stop needing it. But typical incentives in any medium mean that it’s possible to get people hooked on your content instead. A musing on how the setup for writing self-help differs from typical content."

Nobody Does The Thing That They Are Supposedly Doing by Kaj Sotala - "In general, neither organizations nor individual people do the thing that their supposed role says they should do." Evolutionary incentives. Psychology of motivation. Very large number of links.

Out To Get You by Zvi Moshowitz - "Some things are fundamentally Out to Get You. They seek resources at your expense. Fees are hidden. Extra options are foisted upon you." You have four responses: Get Gone, Get Out (give up), Get Compact (limit what it wants) or Get Ready for Battle.

In Defense Of Unreliability by Ozy - Zvi claims that when he makes plan with friends in the bay he never assumes the plan will actually occur. Ozy depends on unreliable transport. Getting places 10-15 early is also costly. Flaking and agoraphobia.

Strategic Goal Pursuit And Daily Schedules by Rossin (lesswrong) - The author benefitted from Anna Salamon’s goal-pursuing heuristics and daily schedules.

Why Attitudes Matter by Ozy - Focusing on attitudes can be bad for some people. Two arguments: "First, for any remotely complicated situation, it would be impossible to completely list out all the things which are okay or not okay. Second, an attitude emphasis prevents rules-lawyering."

Humans Cells In Multicellular Future Minds by Robin Hanson - In general humans replace specific systems with more general adaptive systems. Seeing like a State. Most biological and cultural systems are not general. Multi-cellular organisms re tremendously inefficient. The power of entrenched systems. Human brains are extremely general. Human brains may win for a long time vs other forms of intelligence.

Recognizing Vs Generating An Important Dichotomy For Life by Gordon (Map and Territory) - Bullet Points -> Essay vs Essay -> Bullet Points. Generating ideas vs critique. Most advice is bad since it doesn't convey the reasons clearly. Let the other person figure out the actual advice for themselves.

Prediction Markets Update by Robin Hanson - Prediction markets provide powerful information but they challenge powerful entrenched interests, Hanson compares them to "a knowledgeable Autist in the C-suite". Companies selling straight prediction market tech mostly went under. Blockchain platforms for prediction markets. Some discussion of currently promising companies.


Focus Areas Of Worst Case Ai Safety by The Foundational Research Institute - Redundant safety measures. Tripwires. Adversarial architectures. Detecting and formalizing suffering. Backup utility functions. Benign testing environments.

Srisk Faq by Tobias Baumann (EA forum) - Quite detailed responses to questions about suffering risks and their connection to AGI. sections: General questions, The future, S-risks and x-risks, Miscellaneous.


Reasonable Doubt New Look Whether Prison Growth Cuts Crime by Open Philosophy - Part1 of a four part, in depth, series on Criminal Justice reform. The remaining posts are linked below. "I estimate, that at typical policy margins in the United States today, decarceration has zero net impact on crime. That estimate is uncertain, but at least as much evidence suggests that decarceration reduces crime as increases it. The crux of the matter is that tougher sentences hardly deter crime, and that while imprisoning people temporarily stops them from committing crime outside prison walls, it also tends to increase their criminality after release. As a result, “tough-on-crime” initiatives can reduce crime in the short run but cause offsetting harm in the long run. Empirical social science research—or at least non-experimental social science research—should not be taken at face value. Among three dozen studies I reviewed, I obtained or reconstructed the data and code for eight. Replication and reanalysis revealed significant methodological concerns in seven and led to major reinterpretations of four. These studies endured much tougher scrutiny from me than they did from peer reviewers in order to make it into academic journals. Yet given the stakes in lives and dollars, the added scrutiny was worth it. So from the point of view of decision makers who rely on academic research, today’s peer review processes fall well short of the optimal."

Deterrence De Minimus by Open Philosophy - Part 2.

Incapacitation How Much Does Putting People Inside Prison Cut Crime Outside by Open Philosophy - Part 3.

Aftereffects Us Evidence Says Doing More Time Typically Leads More Crime After by Open Philosophy - Part 4.

Paypal Giving Fund by Jeff Kaufman - The PayPal giving fund lets you batch donations and PayPal covers the fees if you use it. Jeff thought there must be a catch but it seems legit.

What Do Dalys Capture by Danae Arroyos (EA forum) - How Disability Adjusted life years computed. DALYs misrepresent mental health. DALY's Miss Indirect Effects. Other issues.

Against Ea Pr by Ozy - The EA community is the only large entity trying to produce accurate and publicly available assessments of charities. Hence the EA community should not trade away any honesty. EAs should simply say which causes and organizations are most effective, they should not worry about PR concerns.

Ea Survey 2017 Series Qualitative Comments Summary by tee (EA forum) - Are you an EA, how welcoming is EA, local EA meetup attendance, concerns with not being 'EA enough', improving the survey.

Demographics Ii by tee (EA forum) - Racial breakdown. Percent white in various geographic locations. Political spectrum. Politics correlated with cause area, diet and geography, employment, fields of study, year joining EA.

===Politics and Economics:

Raj Chetty Course Using Big Data Solve Economic Social Problems by Marginal Revolution - Link to an eleven lecture course. "Equality of opportunity, education, health, the environment, and criminal justice. In the context of these topics, the course provides an introduction to basic statistical methods and data analysis techniques, including regression analysis, causal inference, quasi-experimental methods, and machine learning."

Speech On Campus Reply To Brad Delong by Noah Smith - The safeguard put in place to exclude the small minority of genuinely toxic people will be overused. Comparison to the war on terror. Brad's exclusions criteria are incredibly vague. The speech restriction apparatus is patchwork and inconsistent. Cultural Revolution.

Deontologist Envy by Ozy - The behavior of your group is highly unlikely to effect the behavior of your political opponents. Many people respond to proposed tactics by asking "What if everyone did that". Ozy claims these responses show an implicit Kantian or deontological point of view.

Peak Fossil Fuel by Bayesian Investor - Electric cars will have a 99% market share by 2035. "Electric robocars run by Uber-like companies will be cheap enough that you’ll have trouble giving away a car bought today. Uber’s prices will be less than your obsolete car’s costs of fuel, maintenance, and insurance."

What We Didn't Get by Noah Smith - We are currently living in a world envisioned by the cyberpunk writers. the early industrial sci-fi writers also predicted many inventions. Why didn't mid 1900s sci-fi come true? We ran out of theoretical physics and we ran out of energy. Energy density of fuel sources. Some existing or plausible technology is just too dangerous. Discussion of whether strong AI, personal upload, nanotech and/or the singularity will come true.

Unpopular Ideas About Children by Julia Galef - Julia's thoughts on why she is collecting these lists. Parenting styles, pro and anti-natalism, sexuality, punishment, etc. Happiness studies. Some other studies finding extreme results.

The Margin Of Stupid by Noah Smith - Can we trust studies showing that millennials are as racist as their parents, except for the ones in college who are extreme leftists?

Role of Allies in Queer Spaces by Brute Reason - The main purpose of having allies in LBGTQA spaces is providing cover for closeted or questioning members. Genuinely cis-straight allies are ok in some spaces like LBGTQA bands. But straight allies cause problems when they are present in queer support spaces.

The Wonder Of International Adoption by Bryan Caplan - Benefits of international adoption of third world children. Adoptees are extremely stunted physically on arrival but make up some of the difference post adoption. International adoptions raises IQ by at least 4 points on average and perhaps as much as 8.


Coin Flipping Problem by protokol2020 - Flipping coins until you get a pre-committed sequence. You re-start whenever your flip doesn't match the sequence. Relationship between the expected number of flips and the length of the sequence.

Seek Not To Be Entertained by Mr. Money Mustache - Don't be normal, normal people need constant entertainment. You can get enjoyment and satisfaction from making things. Advice for people less abnormal than MMM. What you enjoy doesn't matter, what matters is what is good for you.

Propositions On Immortality by sam[]zdat - Fiction. A man digresses about philosophy, the nature of time, the soul, consciousness and mortality.

Comments For Ghost by Tom Bartleby - Ghost is a blog platform hat doesn't natively support comments. Three important use cases and why they all benefit from comments: Ex-Wordpress blogger who wants things to 'just work', Power suers care about privacy and don't want to use third party comments, The Static-Site Fence-Sitter since the main dynamic content you want is comments.

Prime Crossword by protokol2020 - Can you create a grid larger than [3,7],[1,1] where all the rows and columns are primes? (37, 11, 31 and 71 are prime).


Reihan Salam by The Ezra Klein Show - Remaking the Republican party, but not the way Donald Trump did it. "The future of the Republican Party, the healthcare debate, and how he would reform our immigration system (and upend the whole way we talk about it). "

Into The Dark Land by Waking Up with Sam Harris - "Siddhartha Mukherjee about his Pulitzer Prize winning book, The Emperor of All Maladies: A Biography of Cancer."

Conversation with Larry Summers by Marginal Revolution - "Mentoring, innovation in higher education, monopoly in the American economy, the optimal rate of capital income taxation, philanthropy, Hermann Melville, the benefits of labor unions, Mexico, Russia, and China, Fed undershooting on the inflation target, and Larry’s table tennis adventure in the summer Jewish Olympics."

Hilary Clinton by The Ezra Klein Show - Hilary's dream of paying for basic income with revenue from shared national resources. Why she scrapped the plan. Hilary thinks she should perhaps have thrown caution to the wind. Hilary isn't a radical, she is proud of the American political system and is annoyed other's don't share her enthusiasm for incremental progress.

David Remnick by The Ezra Klein Show - New Yorker editor. "Russia’s meddling in the US election, Russia’s transformation from communist rule to Boris Yeltsin and Vladimir Putin, his magazine’s coverage of President Donald Trump, how he chooses his reporters and editors, and how to build a real business around great journalism."

Gabriel Zucman by EconTalk - "Research on inequality and the distribution of income in the United States over the last 35 years. Zucman finds that there has been no change in income for the bottom half of the income distribution over this time period with large gains going to the top 1%. The conversation explores the robustness of this result to various assumptions and possible explanations for the findings."

Could A Neuroscientist Understand A Microprocessor by Rationally Speaking - "Eric Jonas, discussing his provocative paper titled 'Could a Neuroscientist Understand a Microprocessor?' in which he applied state-of-the-art neuroscience tools, like lesion analysis, to a computer chip. By applying neuroscience's tools to a system that humans fully understand he was able to reveal how surprisingly uninformative those tools actually are."

Intrinsic properties and Eliezer's metaethics

6 Tyrrell_McAllister 29 August 2017 11:26PM


I give an account for why some properties seem intrinsic while others seem extrinsic. In light of this account, the property of moral goodness seems intrinsic in one way and extrinsic in another. Most properties do not suffer from this ambiguity. I suggest that this is why many people find Eliezer's metaethics to be confusing.

Section 1: Intuitions of intrinsicness

What makes a particular property seem more or less intrinsic, as opposed to extrinsic?

Consider the following three properties that a physical object X might have:

  1. The property of having the shape of a regular triangular. (I'll call this property "∆-ness" or "being ∆-shaped", for short.)
  2. The property of being hard, in the sense of resisting deformation.
  3. The property of being a key that can open a particular lock L (or L-opening-ness).

To me, intuitively, ∆-ness seems entirely intrinsic, and hardness seems somewhat less intrinsic, but still very intrinsic. However, the property of opening a particular lock seems very extrinsic. (If the notion of "intrinsic" seems meaningless to you, please keep reading. I believe that I ground these intuitions in something meaningful below.)

When I query my intuition on these examples, it elaborates as follows:

(1) If an object X is ∆-shaped, then X is ∆-shaped independently of any consideration of anything else. Object X could manifest its ∆-ness even in perfect isolation, in a universe that contained no other objects. In that sense, being ∆-shaped is intrinsic to X.

(2) If an object X is hard, then that fact does have a whiff of extrinsicness about it. After all, X's being hard is typically apparent only in an interaction between X and some other object Y, such as in a forceful collision after which the parts of X are still in nearly the same arrangement.

Nonetheless, X's hardness still feels to me to be primarily "in" X. Yes, something else has to be brought onto the scene for X's hardness to do anything. That is, X's hardness can be detected only with the help of some "test object" Y (to bounce off of X, for example). Nonetheless, the hardness detected is intrinsic to X. It is not, for example, primarily a fact about the system consisting of X and the test object Y together.

(3) Being an L-opening key (where L is a particular lock), on the other hand, feels very extrinsic to me. A thought experiment that pumps this intuition for me is this: Imagine a molten blob K of metal shifting through a range of key-shapes. The vast majority of such shapes do not open L. Now suppose that, in the course of these metamorphoses, K happens to pass through a shape that does open L. Just for that instant, K takes on the property of L-opening-ness. Nonetheless, and here is the point, an observer without detailed knowledge of L in particular wouldn't notice anything special about that instant.

Contrast this with the other two properties: An observer of three dots moving in space might notice when those three dots happen to fall into the configuration of a regular triangle. And an observer of an object passing through different conditions of hardness might notice when the object has become particularly hard. The observer can use a generic test object Y to check the hardness of X. The observer doesn't need anything in particular to notice that X has become hard.

But all that is just an elaboration of my intuitions. What is really going on here? I think that the answer sheds light on how people understand Eliezer's metaethics.

Section 2: Is goodness intrinsic?

I was led to this line of thinking while trying to understand why Eliezer's metaethics is consistently confusing.

The notion of an L-opening key has been my personal go-to analogy for thinking about how goodness (of a state of affairs) can be objective, as opposed to subjective. The analogy works like this: We are like locks, and states of affairs are like keys. Roughly, a state is good when it engages our moral sensibilities so that, upon reflection, we favor that state. Speaking metaphorically, a state is good just when it has the right shape to "open" us. (Here, "us" means normal human beings as we are in the actual world.) Being of the right shape to open a particular lock is an objective fact about a key. Analogously, being good is an objective fact about a state of affairs.

Objective in what sense? In this important sense, at least: The property of being L-opening picks out a particular point in key-shape space1. This space contains a point for every possible key-shape, even if no existing key has that shape. So we can say that a hypothetical key is "of an L-opening shape" even if the key is assumed to exist in a world that has no locks of type L. Analogously, a state can still be called good even if it is in a counterfactual world containing no agents who share our moral sensibilities.

But the discussion in Section 1 made "being L-opening" seem, while objective, very extrinsic, and not primarily about the key K itself. The analogy between "L-opening-ness" and goodness seems to work against Eliezer's purposes. It suggests that goodness is extrinsic, rather than intrinsic. For, one cannot properly call a key "opening" in general. One can only say that a key "opens this or that particular lock". But the analogous claim about goodness sounds like relativism: "There's no objective fact of the matter about whether a state of affairs is good. There's just an objective fact of the matter about whether it is good to you."

This, I suppose, is why some people think that Eliezer's metaethics is just warmed-over relativism, despite his protestations.

Section 3: Seeing intrinsicness in simulations

I think that we can account for the intuitions of intrinsicness in Section 1 by looking at them from the perspective simulations. Moreover, this account will explain why some of us (including perhaps Eliezer) judge goodness to be intrinsic.

The main idea is this: In our minds, a property P, among other things, "points to" the test for its presence. In particular, P evokes whatever would be involved in detecting the presence of P. Whether I consider a property P to be intrinsic depends on how I would test for the presence of P — NOT, however, on how I would test for P "in the real world", but rather on how I would test for P in a simulation that I'm observing from the outside.

Here is how this plays out in the cases above.

(1) In the case of being ∆-shaped, consider a simulation (on a computer, or in your mind's eye) consisting of three points connected by straight lines to make a triangle X floating in space. The points move around, and the straight lines stretch and change direction to keep the points connected. The simulation itself just keeps track of where the points and lines are. Nonetheless, when X becomes ∆-shaped, I notice this "directly", from outside the simulation. Nothing else within the simulation needs to react to the ∆-ness. Indeed, nothing else needs to be there at all, aside from the points and lines. The ∆-shape detector is in me, outside the simulation. To make the ∆-ness of an object X manifest, the simulation needs to contain only the object X itself.

In summary: A property will feel extremely intrinsic to X when my detecting the property requires only this: "Simulate just X."

(2) For the case of hardness, imagine a computer simulation that models matter and its motions as they follow from the laws of physics and my exogenous manipulations. The simulation keeps track of only fundamental forces, individual molecules, and their positions and momenta. But I can see on the computer display what the resulting clumps of matter look like. In particular, there is a clump X of matter in the simulation, and I can ask myself whether X is hard.

Now, on the one hand, I am not myself a hardness detector that can just look at X and see its hardness. In that sense, hardness is different from ∆-ness, which I can just look at and see. In this case, I need to build a hardness detector. Moreover, I need to build the detector inside the simulation. I need some other thing Y in the simulation to bounce off of X to see whether X is hard. Then I, outside the simulation, can say, "Yup, the way Y bounced off of X indicates that X is hard." (The simulation itself isn't generating statements like "X is hard", any more than the 3-points-and-lines simulation above was generating statements about whether the configuration was a regular triangle.)

On the other hand, crucially, I can detect hardness with practically anything at all in addition to X in the simulation. I can take practically any old chunk of molecules and bounce it off of X with sufficient force.

A property of an object X still feels intrinsic when detecting the property requires only this: "Simulate just X + practically any other arbitrary thing."

Indeed, perhaps I need only an arbitrarily small "epsilon" chunk of additional stuff inside the simulation. Given such a chunk, I can run the simulation to knock the chunk against X, perhaps from various directions. Then I can assess the results to conclude whether X is hard. The sense of intrinsicness comes, perhaps, from "taking the limit as epsilon goes to 0", seeing the hardness there the whole time, and interpreting this as saying that the hardness is "within" X itself.

In summary: A property will feel very intrinsic to X when its detection requires only this: "Simulate just X + epsilon."

(3) In this light, L-opening keys differ crucially from ∆-shaped things and from hard things.

An L-opening key differs from an ∆-shaped object because I myself do not encode lock L. Whereas I can look at a regular triangle and see its ∆-ness from outside the simulation, I cannot do the same (let's suppose) for keys of the right shape to open lock L. So I cannot simulate a key K alone and see its L-opening-ness.

Moreover, I cannot add something merely arbitrary to the simulation to check K for L-opening-ness.  I need to build something very precise and complicated inside the simulation: an instance of the lock L. Then I can insert K in the lock and observe whether it opens.

I need, not just K, and not just K + epsilon: I need to simulate K + something complicated in particular.

Section 4: Back to goodness

So how does goodness as a property fit into this story?

There is an important sense in which goodness is more like being ∆-shaped than it is like being L-opening. Namely, goodness of a state of affairs is something that I can assess myself from outside a simulation of that state. I don't need to simulate anything else to see it. Putting it another way, goodness is like L-opening would be if I happened myself to encode lock L. If that were the case, then, as soon as I saw K take on the right shape inside the simulation, that shape could "click" with me outside of the simulation.

That is why goodness seems to have the same ultimate kind of intrinsicness that ∆-ness has and which being L-opening lacks. We don't encode locks, but we do encode morality.



1. Or, rather, a small region in key-shape space, since a lock will accept keys that vary slightly in shape.

P: 0 <= P <= 1

6 DragonGod 27 August 2017 09:57PM

Part of The Contrarian Sequences.                

Reply to infinite certainty and 0 and 1 are not probabilities.



In infinite certainty, Eliezer makes the argument that you can't ever be absolutely sure of a proposition. That is an argument I disagreed with for a long time, but due to Akrasia acedia, I never got around to writing it. I think I have a more coherent counter argument now, and would present it below. Because the post I am replying to and infinite certainty are linked, I address both of them in this post.


This doesn't mean, though, that I have absolute confidence that 2 + 2 = 4.  See the previous discussion on how to convince me that 2 + 2 = 3, which could be done using much the same sort of evidence that convinced me that 2 + 2 = 4 in the first place.  I could have hallucinated all that previous evidence, or I could be misremembering it.  In the annals of neurology there are stranger brain dysfunctions than this.

This is true. That a statement is true does not mean that you have absolute confidence in the veracity of the statement. It is possible that you may have hallucinated everything.


Suppose you say that you're 99.99% confident that 2 + 2 = 4.  Then you have just asserted that you could make 10,000 independent statements, in which you repose equal confidence, and be wrong, on average, around once.

I am not so sure of this. If I have X% confidence in a belief, and I am well calibrated, then if there were K statements for which I said I have X% confidence in, then you expect that ((100-X)/100)*K of those statements would be wrong, and the remainder would be right. It does not follow that if I have X% confidence in a belief that I can make K statements in which I repose equal confidence, and be wrong only ((100-X)/100)*K  times. 

It's something like X% confidence (implies) if you made K statements then ((100-X)/100)*K of those statements would be wrong.

A well calibrated agent does not have to be able to make K with only ((100-X)/100)*K  wrong those statements for them to possess X% confidence in the proposition. It only indicates that in a hypothetical world in which they did make K statements, if they were well calibrated, only ((100-X)/100)*K  of those statements would be wrong. To assert that a well calibrated agent must be able to make those statements before they can have X% confidence, is to establish the hypothetical as a given fact—either a honest mistake, or deliberate malice.                                


As for the notion that you could get up to 100% confidence in a mathematical proposition—well, really now!  If you say 99.9999% confidence, you're implying that you could make one million equally fraught statements, one after the other, and be wrong, on average, about once.  That's around a solid year's worth of talking, if you can make one assertion every 20 seconds and you talk for 16 hours a day.

Assert 99.9999999999% confidence, and you're taking it up to a trillion.  Now you're going to talk for a hundred human lifetimes, and not be wrong even once?

Assert a confidence of (1—1/googolplex) and your ego far exceeds that of mental patients who think they're God.

And a googolplex is a lot smaller than even relatively small inconceivably huge numbers like 3^^^3.

All based on the same flawed premise, and equally flawed.


I am Infinitely Certain

There is one proposition that I would start with and assign a probability of 1, not 1-1/googolplex. Not 1 - 1/3^^^^3, Not 1 - epsilon (where epsilon is an arbitrarily small number), but a probability of 1.              


I exist.


Rene Descartes presents a very wonderful argument for the veracity of this statement:

Accordingly, seeing that our senses sometimes deceive us, I was willing to suppose that there existed nothing really such as they presented to us; And because some men err in reasoning, and fall into Paralogisms, even on the simplest matters of Geometry, I, convinced that I was as open to error as any other, rejected as false all the reasonings I had hitherto taken for Demonstrations; And finally, when I considered that the very same thoughts (presentations) which we experience when awake may also be experienced when we are asleep, while there is at that time not one of them true, I supposed that all the objects (presentations) that had ever entered into my mind when awake, had in them no more truth than the illusions of my dreams. But immediately upon this I observed that, whilst I thus wished to think that all was false, it was absolutely necessary that I, who thus thought, should be something; And as I observed that this truth, I think, therefore I am,[c] was so certain and of such evidence that no ground of doubt, however extravagant, could be alleged by the Sceptics capable of shaking it, I concluded that I might, without scruple, accept it as the first principle of the philosophy of which I was in search


Eliezer quotes Rafal Smigrodski:

"I would say you should be able to assign a less than 1 certainty level to the mathematical concepts which are necessary to derive Bayes' rule itself, and still practically use it.  I am not totally sure I have to be always unsure.  Maybe I could be legitimately sure about something.  But once I assign a probability of 1 to a proposition, I can never undo it.  No matter what I see or learn, I have to reject everything that disagrees with the axiom.  I don't like the idea of not being able to change my mind, ever."


I am alright with accepting as an axiom that I exist. I see no reason why I should be cautious of assigning a probability of 1 to this statement. I am infinitely certain that I exist.               

If you accept Descartes argument, then this is very important. You're accepting that we can be infinitely certain about a proposition—and not just that—that it is sensible to be infinitely certain about a proposition. Usually, only one counterexample is necessary, but there are several other statements which you may assign a probability of 1 to.                     


I believe that I exist.                      


I believe that I believe that I exist. 


I believe that I believe that I believe that I exist.

And so on and so forth, ad infinitum. An infinite chain of statements, all of which are exactly true. I have satisfied Eliezer's (fatuous) requirements for assigning a certain level of confidence to a proposition. If you feel that it is not sensible to assign probability 1 to the first statement, then consider this argument. I assign a probability 1 to the proposition "I exist". This means that the proposition "I exist" exists (pun intended) in my mental map of the world, and is therefore a belief of mine. By deduction, if I assign a probability of 1 to the statement "I exist", then I must assign a probability of 1 to the proposition "I believe that I exist". By induction, I must assign a probability of 1 to all the infinite statements, and all of them are true.                     

(I assign a probability of 1 to deduction being true).


Generally, using the power of recursion, we can pick any statement, to which we assign a probability of 1 and generate infinite more statements to which we (by deduction) also assign a probability of 1.               

Let X be a proposition to which we assign a probability of 1.                 

define f(var, n=0)

      if n < 0 or type(n) != int

      return -1

      end if

      if var == X and n == 0

            var = ("I believe " + var + ".")

            print var

      end if

      n = (n < 2)?2:n

      str = ("I believe that " + var + ".")

      print str

      i = 0

      while i < n

            str += "I believe that " + str + "."       

            print str

      end while       

      end if else

      f(str, n**n)


f(f(X, n)) for any X (to which we assign a probability of 1 and some valid n) prints an infinite number of statements to which we also assign a probability of 1.                  


While I'm at it, I can show that there are an uncountably infinite number of such statements with a probability of 1.

Let S be the array of all propositions produced by f(f(X, n)) (for some valid X to which we assigned a probability of 1, and a valid n).

define g(var) 

      k = rand(#S)

      i = 0

      j = rand(#S)

      str = "I believe " + S[j] 


      while i < k

            j = rand(#S)

            str += " and " + S[j]



      end while


      f(g(var), 2)


Assuming #S = Aleph_null, there are 2^#S possible values for str, and each of them can be used to generate an infinite sequence of true propositions. By Cantor's diagonal argument the number of propositions to which we assign a probability of 1 are uncountable. For each of those propsitions, we assign a probability of 0 to their negation. That is if you accept Descartes argument, or accept any single proposition has having a probability of 1 (or 0), then you accept uncountably infinite many propositions as having a probability of 1 (or 0). Either we can never be certain of any propositions ever, or we can be certain of uncountably infinite many propositions (you can also use the outlined method to construct K statements with arbitrary accuracy).           

Personally, I see no problem with accepting "I exist" (and deduction) as having P of 1.


When you work in log odds, the distance between any two degrees of uncertainty equals the amount of evidence you would need to go from one to the other.  That is, the log odds gives us a natural measure of spacing among degrees of confidence.

Using the log odds exposes the fact that reaching infinite certainty requires infinitely strong evidence, just as infinite absurdity requires infinitely strong counterevidence.

This ignores the fact that you can assign priors of 0 and 1—in fact, it is for this very reason that I argue that 0 and 1 are probabilities—Eliezer is right in that we can never update upwards (or downwards as the case may be) to 1 or 0 (without using priors of 0 or 1), but we can (and I argue we should) sometimes start with priors of 0 and 1.


0 and 1 as priors.

Consider Pascal's Mugging. Pascal's Mugging is a breaker (breakers are a name I coined for decision problems which break decision theories). Let us reconceive the problem such that the person doing the mugging is me. 

I walk up to Eliezer and tell him that he should pay me a $10,000 or I would grant him infinite negative utility. 

Now, I cannot (as a matter of fundamental physical law) inflict infinite negative utility on Eliezer. However, if Eliezer is rational (maximising his expected utility), then Eliezer must pay me the money. No matter how much money I demand from Eliezer, Eliezer must pay me, because Eliezer does not assign a probability of 0 to me carrying out my threat, and no matter how small the probability is, as long as it's not 0, paying me the ransom I demanded is the choice which maximises expected utility.                


(If you claim that it is impossible for me to grant you infinite negative utility/infinite negative utility is incoherent/return a category error on infinite negative utility, then you are assigning a probability of 0 to the existence of infinite negative utility, and (implicitly (because P(A) >= P(A and B). A here is "infinite negative utility exists". B is "I can grant infinite negative utility".) assigning a probability of 0 to me granting you infinite negative utility).

I have no problems with decision problems which break decision theories, but when a problem breaks the very formulation of rationality itself, then I'm pissed. There is a trivial solution to resolving Pascal's mugging using classical decision theory (accept the objective definition of probability; once you do so, the probability of me carrying out my threat becomes zero and the problem disappears). Only the insistence to cling to (unfounded) subjective probability that forbids 0 and 1 as probabilities leads to this mess.                

If anything, Pascal's mugging should be a definitive proof demonstrating that indeed 0 and 1 are perfectly legitimate priors (if you accept a prior of 0 that I will grant you infinite negative utility, then trivially, you accept a prior of 1 that I do not grant you infinite negative utility). Pascal's mugging only "breaks" Expected utility theory if you forbid priors of 0 and 1—an inane commandment.    


 I'll expand more on breakers, rationality, etc. in my upcoming several ten pages+ paper.



So I propose that it makes sense to say that 1 and 0 are not in the probabilities; just as negative and positive infinity, which do not obey the field axioms, are not in the real numbers.

The main reason this would upset probability theorists is that we would need to rederive theorems previously obtained by assuming that we can marginalize over a joint probability by adding up all the pieces and having them sum to 1.

However, in the real world, when you roll a die, it doesn't literally have infinite certainty of coming up some number between 1 and 6.  The die might land on its edge; or get struck by a meteor; or the Dark Lords of the Matrix might reach in and write "37" on one side.

If you made a magical symbol to stand for "all possibilities I haven't considered", then you could marginalize over the events including this magical symbol, and arrive at a magical symbol "T" that stands for infinite certainty.

But I would rather ask whether there's some way to derive a theorem without using magic symbols with special behaviors.  That would be more elegant.  Just as there are mathematicians who refuse to believe in double negation or infinite sets, I would like to be a probability theorist who doesn't believe in absolute certainty.

Eliezer presents a shaky basis for rejecting 0 and 1 as probabilities. His model leads to absurd conclusion(s) (a proof by contradiction that 0 and 1 are indeed probabilities), he offers no benefits to rejecting the standard model and replacing it with his (only multiple demerits), and he doesn't formalise an alternative model of probability that is free of absurdities and has more benefits than the standard model.                  

0 and 1 are not probabilities is a solution in search of a problem.



Epistemic Hygiene

This article may have come across as overly vicious and confrontational; I adopted such an attitude to minimise the bias in my perception of the original article based on the halo effect.


The Contrarian Sequences

6 DragonGod 27 August 2017 07:33PM

A series of posts wherein I outline my disagreements with Shishou (Eliezer Yudkowsky). 


When I discovered the sequences was my second epiphany (the first was when I became atheist, and my world was turned upside down). Eliezer's charisma, his arrogance, the force of his personality, and his eloquence all combined to make a deadly drug. I was hooked, and seized with a fervour greater than when I first gave my life to Jesus Christ. Eliezer became my new Jesus, and I was drinking the koolaid pretty badly. Some months later (in light of criticism from both friend and foe), I realised I was a cultist, and began trying to sanitise myself. Due to the Halo effect, I accepted everything Eliezer said unconditionally, and never bothered trying to ascertain for myself the veracity of his claims.                  


I am stronger now than I was then, and aspiring higher still. This is a project in raising my epistemic hygiene. I will only react to posts that I feel were wrong in the overarching thesis, and develop my counterarguments in them. I expect I should write counter posts for at least 1% of all posts (if I don't reach the target, I'm probably not being critical enough), and at most 5% (if I exceed that, then I've probably just biased myself in the opposite direction, and/or I'm getting a kick out of disagreeing with Eliezer). Posts in the contrarian sequences would be in chronological order of how I wrote them, and not in the order of how they appear in the sequences, or in any ontological order.  


 Table of Contents


  1. The Reality of Emergence                              
  2. P: 0 <= P <= 1


Could the Maxipok rule have catastrophic consequences? (I argue yes.)

6 philosophytorres 25 August 2017 10:00AM

Here I argue that following the Maxipok rule could have truly catastrophic consequences.

Here I provide a comprehensive list of actual humans who expressed, often with great intensity, omnicidal urges. I also discuss the worrisome phenomenon of "latent agential risks."

And finally, here I argue that a superintelligence singleton constitutes the only mechanism that could neutralize the "threat of universal unilateralism" and the consequent breakdown of the social contract, resulting in a Hobbesian state of constant war among Earthians.

I would genuinely welcome feedback on any of these papers! The first one seems especially relevant to the good denizens of this website. :-)

[Link] Nasas ambitious plan to save earth from a supervolcano

6 ChristianKl 24 August 2017 10:01AM

[Link] The Unyoga Manifesto

6 SquirrelInHell 04 August 2017 09:24PM

[Link] We've failed: paid publication , pirates win.

5 morganism 16 September 2017 09:53PM

Instrumental Rationality Sequence Finished! (w/ caveats)

5 lifelonglearner 09 September 2017 01:49AM

Hey everyone,

Back in April, I said I was going to start writing an instrumental rationality sequence.

It's...sort of done.

I ended up collecting the essays into a sort of e-book. It's mainly content that I've put here (Starting Advice, Planning 101, Habits 101, etc.), but there's also quite a bit of new content.

It clocks in at about 150 pages and 30,000 words, about 15,000 of which I wrote after the April announcement post. (Which beats my estimate of 10,000 words before burnout!!!)

However, the editor for LW 1.0 editor isn't making it easy to port the stuff here from my Google Drive.

As LW 2.0 enters actual open beta, I'll repost / edit the essays and host them there. 

In the meantime, if you want to read the whole compiled book, the direct Google Doc link is here. That's where the real-time updates will happen, so it's what I'd recommend using to read it for now.

(There's also an online version on my blog if for some reason you want to read it there.)

It's my hope that this sequence becomes a useful reference for newcomers looking to learn more about instrumental rationality, which is more specialized than The Sequences (which really are more for epistemics).

Unfortunately, I didn't manage to write the book/sequence I set out to write. The actual book as it is now is about 10% as good as what I actually wanted. There's stuff I didn't get to write, more nuances I'd have liked to cover, more pictures I wanted to make, etc.

After putting in many hours of research and writing, I think I've learned more about the sort of effort that would need to go into making the actual project I'd outlined at the start.

There'll be a postmortem essay analyzing my expectations vs reality coming soon.

As a result of this project and a few other things, I'm feeling burned out. There probably won't be any major projects from me for a little bit, while I rest up.

View more: Next