Less Wrong is a community blog devoted to refining the art of human rationality. Please visit our About page for more information.

Comment author: JenniferRM 17 June 2017 09:31:36AM *  3 points [-]

Three places similar ideas have occurred that spring to mind:

FIRST Suarez's pair of novels Daemon and Freedom(tm) are probably the most direct analogue, because it is a story of taking over the world via software, with an intensely practical focus.

The essential point for this discussion here and now is that prior to launching his system, the character who takes over the world first tests the quality of the goal state that he's aiming at by implementing it first as a real world MMORP. Then the takeover of the world proceeds via trigger-response software scripts running on the net, but causing events in the real world via: bribes, booby traps, contracted R&D, and video game like social engineering.

The MMORP start not only functions as his test bed for how he wants the world to work at the end... it also gives him starting cash, a suite of software tools for describing automated responses to human decisions, code to script the tactics of swarms of killer robots, and so on.

SECOND Nozick's Experience Machine thought experiment is remarkably similar to your thought experiment, and yet aimed at a totally different question.

Nozick was not wondering "can such a machine be described in detail and exist" (this was assumed) but rather "would people enter any such machine and thereby give up on some sort of atavistic connection to an unmediated substrate reality, and if not what does this mean about the axiological status of subjective experience as such?"

Personally I find the specifics of the machine to matter an enormous amount to how I feel about it... so much so that Nozick's thought experiment doesn't really work for me in its philosophically intended manner. There has been a lot of play with the concept in fiction that neighbors on the trope where the machine just gives you the experience of leaving the machine if you try to leave it. This is probably some kind of archetypal response to how disgusting it is in practice for people to be pure subjective hedonists?

THIRD Greg Egan's novel Diaspora has most of the human descended people living purely in and as software.

In the novel any common environment simulator and interface (which has hooks into the sensory processes of the software people) is referred to as a "scape" and many of the software people's political positions revolve around which kinds of scapes are better or worse for various reasons.

Konishi Polis produces a lot of mathematicians, and has a scape that supports "gestalt" (like vision) and "linear" (like speech or sound) but it does not support physical contact between avatars (their relative gestalt positions just ghost around and through each other) because physical contact seems sort of metaphysically coercive and unethical to them. By contrast Carter-Zimmerman produces the best physicists, and it has relatively high quality physics simulations built into their scape, because they think that high quality minds with powerful intuitions require that kind of low level physical experience embedded into their everyday cognitive routines. There are also flesh people (who think flesh gives them authenticity or something like that) and robots (who think "fake physics" is fake, even though having flesh bodies is too dangerous) and so on.

All of the choices matter personally to the people... but there is essentially no lock in, in the sense that people are forced to do one thing or another by an overarching controller that settles how things will work for everyone for all time.

If you want to emmigrate from Konishi to Carter Zimmerman you just change which server you're hosted on (for better latency) and either have mind surgery (to retrofit your soul with the necessary reflexes for navigating the new kind of scape) or else turn on a new layer of exoself (that makes your avatar in the new place move according to a translation scheme based on your home scape's equivalent reflexes).

If you want to, you can get a robot body instead (the physical world then becomes like a very very slow scape and you run into the question of whether to slow down your clocks and let all your friends and family race ahead mentally, or keep your clock at a normal speed and have the robot body be like a slow moving sculpture you direct to do new things over subjectively long periods of time). Some people are still implemented in flesh, but if they choose they can get scanned into software and run as a biology emulation. Becoming biologically based is the only transformation rarely performed because... uh... once you've been scanned (or been built from software from scratch) why would you do this?!

Interesting angles:

Suarez assumes physical coercion and exponential growth as the natural order, and is mostly interested in the details of these processes as implemented in real political/economic systems. He doesn't care about 200 years from now and he uses MMORP simulations as simply a testbed for practical engineering in intensely human domains.

Nozick wants to assume utopia, and often an objection is "who keeps the Experience Machine from breaking down?"

Egan's novel has cool posthuman world building, but the actual story revolves around the question of keeping the experience machine from breaking down... eventually stars explode or run down... so what should be done in the face of a seemingly inevitable point in time where there will be no good answer to the question of "how can we survive this new situation?"

In response to Bring up Genius
Comment author: JenniferRM 09 June 2017 07:28:10AM *  9 points [-]

I was really impressed by the chess curriculum in two ways:

Remember to let the child win most of the time.

The surprise I had here is that when my dad taught me chess when I was about 5, something that stands out in my memory is that he emphasized that he would always play to win, and if I won it would be a really victory, and if I lost I did not have a right to complain. I did not win much, but I really liked playing. I remember wanting to play more and him not having time, rather than having a problem with motivation.

Some tips for teaching chess to 4 or 5 years old children. First, I made a blank square divided into 8x8 little squares, with named rows and columns. I named a square, my daughter had to find it; then she named a square and I had to find it. Then we used the black-and-white version, and we were guessing the color of the named square without looking.

Then we introduced kings, in a "king vs king" combat; the task was to reach the opposing row of the board with your king. Then we added a pawn; the goal remained to reach the opposing row. After a month of playing, we introduced the queen, and the concept of checkmate. Later we gradually added the remaining pieces (knights were the most difficult).

Then we solved about thousand "checkmate in one move" puzzles. Then two moves, three moves, four moves. That took another 3 or 4 months. And only afterwards we started really playing against each other.

That is a really thoughtfully structured curriculum! Maybe it is a standard way to do it in some countries but all through my childhood when I taught people chess or saw it taught by others the basic process was just to explain the moves of all the pieces by demonstration and then just jump in. Rows? Columns? Notation? Endgames? That didn't come until way later.

Comment author: darius 02 June 2017 09:12:35PM *  2 points [-]

Small correction: Law's Order is by David Friedman, the middle generation. It's an excellent book.

I had a similar reaction to the sequences. Some books that influenced me the most as a teen in the 80s: the Feynman Lectures and Drexler's Engines of Creation. Feynman modeled scientific rationality, thinking for yourself, clarity about what you don't know or aren't explaining, being willing to tackle problems, ... it resists a summary. Drexler had many of the same virtues, plus thinking carefully and boldly about future technology and what we might need to do in advance to steer to an acceptable outcome. (I guess it's worth adding that seemingly a lot of people misread it as gung-ho promotion of the wonders of Tomorrowland that we could all look forward to by now, more like Kurzweil. For one sad consequence, Drexler seems to have become a much more guarded writer.)

Hofstadter influenced me too, and Egan and Szabo.

Comment author: JenniferRM 03 June 2017 05:22:18AM 1 point [-]

Thanks for the correction! I'll leave the Milton/David error in, so your correction reads naturally :-)

Comment author: JenniferRM 02 June 2017 06:02:03PM 0 points [-]

However, the conclusions are likely wrong, because it's rational for "sleeping" civilizations to still want to round up stars that might be ejected from galaxies, collect cosmic dust, and so on.

Stray thought this inspired: what if "rounding up the stars" is where galaxies come from in the first place?

Comment author: JenniferRM 02 June 2017 05:04:51PM 1 point [-]

I have a hunch that the Grue Bleen Problem for Reward Functions will start to become less complicated as "reward function inference" becomes more tractable.

At an aggregate level, the need to model preferences is already happening on a large scale via personalization of recommendation engines, and at a small scale it comes up with text interpretation (like reference co-resolution) in the paradigm where words are taken to be moves in a cooperative game where two communicating agents "want to pay attention to the same things" but also "want to pay attention to the things they already care about".

As practical methods for modeling human preferences evolve under engineering constraints, I suspect that it will become easier to talk about how these preferences change (or do not change) in very concrete ways.

Comment author: username2 01 June 2017 11:12:57PM 3 points [-]

Is there any book that had a bigger inpact on your life than the sequences ? What is it and why ?

Comment author: JenniferRM 02 June 2017 04:44:03PM *  9 points [-]

The sequences for me were like "yeah, but everyone smart already knows this stuff" and then I was more interested in the community and commenters from the golden era who were often slightly "crazy smart". However, when I try to imagine the effect the sequences had on people who had read less, and seek similar things in my own life, the first things that stands out is GEB.

Part of the value was in dissolving a bunch of youthful philosophical confusions about the relationship between meaningful artifacts (like word or programs or whatever) and the formal or semi-formal contexts that interpret and reveal their meaning within limits imposed by the Incompleteness Theorem.

Another chunk of the value was that GEB pointed to and paired with related books like Metamagical Themas (also by Hofstadter), The Night Is Large (by Gardner, who Hofstadter replaced at Scientific American), and Fluid Concepts and Creative Analogies (about Hofstadter's grad student's projects).

In the same general vein, but probably more fun to read, Diaspora, Permutation City, and Kiln People seem like touchstones of novelistically concrete philosophy of mind to me. My understanding is that Diaspora and to a lesser extent Permutation City were highly influential for Eliezer as well.

Also, James Gleick has had a remarkable run of non-fiction books, better than the normal pop science stuff, at least for me. If I see a book of his on a shelf that I haven't read yet I'll buy it just based on his name.

Another book that stands out for me as highly influential, but separate from the LW/extropian memeplex, is Milton Friedman's Law's Order. Admittedly it isn't that far from the memeplex around here, given that it was written by Patri's grandfather.

In the same vein, Nick Szabo's blog archives are dense reading, but worth it. He is not exactly local, but was friends with Hal Finney and was widely suspected of being Satoshi.

Comment author: Duncan_Sabien 26 May 2017 09:25:15AM 5 points [-]

Currently there are both men and women interested (though many more men than women).

All of your points above seem sound at first glance, and yes, it's on the docket to be sorted out. I don't think I want to go full monastery, but there's a decent chance the house itself will end up being activity-restricted in some way.

Thanks for the detailed model-sharing.

Comment author: JenniferRM 30 May 2017 06:50:00AM *  2 points [-]

I'm glad the model was deemed useful :-) Good luck.

Comment author: JenniferRM 26 May 2017 08:37:52AM *  43 points [-]

The most common cause of the collapse of high investment intentional communities is romantic drama.

(Maybe the Dragon Barracks are so obviously a boy thing that you're taking for granted that there will be no girls in the house, but all the weird non-gendered pronouns like "a Dragon will brush its teeth" imply either an attempt to have a team composed of both men or women, or else a hilarious level of contempt for the agency of your space monkeys. I'm going to assume that you're imagining mixed gender living arrangements rather than already starting with verbal de-personalization of presumed uniformly male space monkeys...)

So anyway, assuming men and women in the house at the same time, that's what usually causes things to collapse in the long run.

The two standard failure modes are Bonobo egalitarianism that collapses due to the accumulation of residual jealousies over time or else a harem forms around the charismatic cult leader (which isn't necessarily a failure mode... it is just a sign of a cult leader whose stated community goals are a load of hypocritical baloney compared to the real goal of getting more than his "fair share" of tail -- cue the Limp Bizkit song).

There are lots of patches for this sort of thing that have historically worked for various kinds of communities. Requiring celibacy is an obvious one that monasteries often use. Disallowing any romantic statuses except "single" and "closed dyadic marriage" (with a managed "courting" status to mediate the one way transition) is another standard trick.

Whatever the rule is, the standard enforcement mechanism is "ostracism" because the real problem from a social engineering perspective is the accumulation of complicated feelings that slow and redirect the workings of the social machine away from its stated purposes and towards managing the wreckage of new and old love triangles. If you throw away the cogs that are liable to have "complicated feelings" and replace them with non-complicated cogs... then the machine should continue to run as designed?

(I think maybe the romantic mores that were junked in the US in the 1960's arose in the first place because villages are kinda like auto-poetic intentional communities. The pragmatically useful norms of village romance, that kept the village from exploding, could be semi-safely junked because (well, obviosuly "the pill" but also because) cities are anonymous and moderately well mixed... essentially everyone in a city is already pre-ostrasized by everyone else, and we each are desperately struggling to create a synthetic village-like community despite the isolating forces of urban mixing. In an already chaotic urban romantic economy a divorce causing additional minor lesioning of the local social graph is like a dust devil in a hurricane. There might actually be a lot of dust devils caused by hurricane turbulence for all I know, but I'm pretty sure no one cares much because the actual hurricane make them irrelevant.)

Anyway, for the above reasons, you might want to just say "this is a fraternity and if women want to start a rationalist sorority that can be a separate thing". Alternatively, think about romantic norms up front.

Comment author: chaosmage 03 April 2017 02:18:01PM 8 points [-]

One of the skills to talk about would be the skill of actively proselytizing and getting people into rationality. I don't mean onboarding people who are already interested, I mean actually going up to people who you wish were rationalists and trying to make them.

Successful communities do this, although the specifics vary widely. EA does it, which I think is why EA is growing while LW isn't. We've been largely coasting on Eliezer's wave.

Thus is difficult because LW rationality arose in the tech culture of California, I.e. an unusually individualistic culture within an unusually individualistic part of the most individualistic country ever. Only in California could one be called a "cult" for seeking a consensus philosophy. Any active proselytizing would definitely encounter the "cult" charge again.

But proselytizing works. It keeps a movement young - we're already noticably older on average than we were ten years ago and we're starting to look like a cohort of tech nerds who were in their impressionable college age when Eliezer wrote the sequences. And it keeps a movement dynamic - if new people are coming in all the time, you don't have to suffer the ossification that it takes to retain people as they get older. LW rationality has no less need of this than other movements.

And there are definitely people who are much better at it than others, so a systematic study of what works is eminently doable. I think this fits squarely into Project Hufflepuff.

Comment author: JenniferRM 07 April 2017 05:49:35PM *  8 points [-]

Personally, I think cohorts happen automatically, and LW is "yet another cohort" and if we want to be part of a movement with inter-generational significance then maybe we should pause to consider why we think we should be "the first generation" in a movement that lasts forever...

In this vein, I appreciate previous places and groups like:

If I was going to name the entire thing, I think I might call it "Outsider Science" (taking a cue from "Outsider Art" and contrasting it with "Vannevarian Science").

So if you wanted to be so Hufflepuff that you sacrificed the whole group on the altar of being social (rather than just sacrificing yourself for the group) I'd argue that it would be a natural progression to work on reconnecting, resuscitating, archiving, and generally paying attention to these older places and communities, and putting yourself in service to their long term goals.

The hard thing here is that the diagnostic criteria looking backwards seems to be having a certain mindset towards physical reality and being a kind of a cultural orphan at the same time. The standoffishness and founding a tiny little institutes is part of what this movement seems to do by default?

Thus, projecting forward, you would predict that new instances of "the outsider science movement" would form on their own, start their own thing, and reject the notion of intellectual parentage, as much as we (the hypothetical intellectual parents) try to bring them into the loose confederation of previous attempts at self organized scientific research aiming at eternal intellectual challenges.

A lot of the future people you'd be trying to bring into the fold might very well prefer to struggle on alone.

Arguably, Vanevarian Science (with government credentialed universities doing government funded research) is already doing what you would evolve into anyway, and has succeeded so far and so thoroughly that its "highest mid level hierarchs" have become members of the deep government of the world? So maybe the right thing to do is just let all the various orphans struggle on by themselves, and just go try to get a job at NSF while retaining fond feelings for the strugglers?

So my guess is that Bacon's Effecting Of All Things Possible has run for a long while now, and maybe "the orphans" who might have belonged to the high church version (but somehow never connected with the central culture) were never really noticed until the internet came along and then could start to find each other and form little clumps and clusters.

So maybe the most Hufflepuff thing possible would be to somehow be encourage a larger internet culture that finds and welcomes these autonomous orphan clusters, while also extending an olive branch to the high church "Heirs of Bacon" who exist in the deep government, and see if there is some way to establish a state of communion between the main tree and all the little saplings :-)

Comment author: ChristianKl 12 February 2017 04:40:11PM *  5 points [-]

In some sense bans on lead are mandatory treatments for stupidity. The same goes for government-mandated addition of iodine to salt.

Comment author: JenniferRM 26 February 2017 08:13:44AM *  1 point [-]

It makes me sad to see non-iodized sea salt become trendy in the kinds of circles where vaccines are considered "unnatural" and kids get whooping cough.

I think there is a general issue here where "libertarianism" and "paternalism" come into conflict.

My preference in nearly all such cases is to default people into the thing that seems to honestly be the best policy, and let people opt out in a way that involves some larger or smaller trivial inconviences if they want to be contrarian for some reason.

View more: Next