Less Wrong is a community blog devoted to refining the art of human rationality. Please visit our About page for more information.

John_Maxwell_IV comments on CFAR’s new focus, and AI Safety - Less Wrong

30 Post author: AnnaSalamon 03 December 2016 06:09PM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (87)

You are viewing a single comment's thread. Show more comments above.

Comment author: John_Maxwell_IV 03 December 2016 12:23:31PM *  16 points [-]

There's a difference between optimizing for truth and optimizing for interestingness. Interestingness is valuable for truth in the long run because the more hypotheses you have, the better your odds of stumbling on the correct hypothesis. But naively optimizing for truth can decrease creativity, which is critical for interestingness.

I suspect "having ideas" is a skill you can develop, kind of like making clay pots. In the same way your first clay pots will be lousy, your first ideas will be lousy, but they will get better with practice.

...creation is embarrassing. For every new good idea you have, there are a hundred, ten thousand foolish ones, which you naturally do not care to display.

Source.

If this is correct, this also gives us clues about how to solve Less Wrong's content problem.

Online communities do not have a strong comparative advantage in compiling and presenting facts that are well understood. That's the sort of thing academics and journalists are already paid to do. If online communities have a comparative advantage, it's in exploring ideas that are neglected by the mainstream--things like AI risk, or CFARish techniques for being more effective.

Unfortunately, LW's culture has historically been pretty antithetical to creativity. It's hard to tell in advance whether an idea you have is a good one or not. And LW has often been hard on posts it considers bad. This made the already-scary process of sharing new ideas even more fraught with the possibility of embarrassment.

If a single individual present is unsympathetic to the foolishness that would be bound to go on at such a [brainstorming] session, the others would freeze. The unsympathetic individual may be a gold mine of information, but the harm he does will more than compensate for that. It seems necessary to me, then, that all people at a session be willing to sound foolish and listen to others sound foolish.

Same source.

I recommend recording ideas in a private notebook. I've been doing this for a few years, and I now have way more ideas than I know what to do with.

Elon Musk

Relevant: http://waitbutwhy.com/2015/11/the-cook-and-the-chef-musks-secret-sauce.html

Comment author: Lumifer 07 December 2016 03:32:02PM 1 point [-]

There's a difference between optimizing for truth and optimizing for interestingness.

Oh yes. For example, Physical Review Letters is mostly interested in the former, while HuffPo -- in the latter.

the more hypotheses you have, the better your odds of stumbling on the correct hypothesis

That's not true because you must also evaluate all these hypotheses and that's costly. For a trivial example, given a question X, would you find it easier to identify a correct hypothesis if I presented you with five candidates or with five million candidates?

I suspect "having ideas" is a skill you can develop

Yes, subject to native ability. I suspect it's more like music than like clay pots: some people find it effortless, most can improve with training, and some won't do well regardless of how much time they spend practicing.

That's the sort of thing academics and journalists are already paid to do.

Kinda. On the one hand, pop-sci continues to be popular. On the other hand, journalists are very very bad at it.

the already-scary process of sharing new ideas

I would like to suggest attaching less self-worth and less status to ideas you throw out. Accept that it's fine that most of them will be shot down.

I don't like the kindergarten alternative: Oh, little Johnny said something stupid, like he usually does! He is such a creative child! Here is a gold star!

I recommend recording ideas in a private notebook.

I concur. Note that LW is not that private notebook.

Comment author: John_Maxwell_IV 08 December 2016 02:17:16AM *  2 points [-]

OK, so I told you the other day that I find you a difficult person to have discussions with. I think I might find your comments less frustrating if you made an effort to think of things I would say in response to your points, and then wrote in anticipation of those things. If you're interested in trying this, I converted all my responses using rot13 so you can try to guess what they will be before reading them.

Oh yes. For example, Physical Review Letters is mostly interested in the former, while HuffPo -- in the latter.

UhssCb vf gelvat gb znkvzvmr nq erirahr ol jevgvat negvpyrf gung nccrny gb gur fbeg bs crbcyr jub pyvpx ba nqf. Gur rkvfgrapr bs pyvpxonvg gryyf hf onfvpnyyl abguvat nobhg ubj hfrshy vg jbhyq or sbe lbhe nirentr Yrff Jebatre gb fcraq zber gvzr trarengvat ulcbgurfrf. Vg'f na nethzrag ol nanybtl, naq gur nanybtl vf dhvgr ybbfr.

V jbhyq thrff Culfvpny Erivrj Yrggref cevbevgvmrf cncref gung unir vagrerfgvat naq abiry erfhygf bire cncref gung grfg naq pbasvez rkvfgvat gurbevrf va jnlf gung nera'g vagrerfgvat. Shegurezber, V fhfcrpg gung gur orfg culfvpvfgf gel gb qb erfrnepu gung'f vagrerfgvat, naq crre erivrj npgf nf n zber gehgu-sbphfrq svygre nsgrejneqf.

That's not true because you must also evaluate all these hypotheses and that's costly. For a trivial example, given a question X, would you find it easier to identify a correct hypothesis if I presented you with five candidates or with five million candidates?

Gur nafjre gb lbhe dhrfgvba vf gung V jbhyq cersre svir zvyyvba pnaqvqngrf. Vs svir ulcbgurfrf jrer nyy V unq gvzr gb rinyhngr, V pbhyq fvzcyl qvfpneq rirelguvat nsgre gur svefg svir.

Ohg ulcbgurfvf rinyhngvba unccraf va fgntrf. Gur vavgvny fgntr vf n onfvp cynhfvovyvgl purpx juvpu pna unccra va whfg n srj frpbaqf. Vs n ulcbgurfvf znxrf vg cnfg gung fgntr, lbh pna vairfg zber rssbeg va grfgvat vg. Jvgu n ynetre ahzore bs ulcbgurfrf, V pna or zber fryrpgvir nobhg juvpu barf tb gb gur evtbebhf grfgvat fgntr, naq erfgevpg vg gb ulcbgurfrf gung ner rvgure uvtuyl cynhfvoyr naq/be ulcbgurfvf gung jbhyq pnhfr zr gb hcqngr n ybg vs gurl jrer gehr.

Gurer frrzf gb or cerggl jvqrfcernq nterrzrag gung YJ ynpxf pbagrag. Jr qba'g frrz gb unir gur ceboyrz bs gbb znal vagrerfgvat ulcbgurfrf.

I would like to suggest attaching less self-worth and less status to ideas you throw out. Accept that it's fine that most of them will be shot down.

I don't like the kindergarten alternative: Oh, little Johnny said something stupid, like he usually does! He is such a creative child! Here is a gold star!

V pvgrq fbzrbar V pbafvqre na rkcreg ba gur gbcvp bs perngvivgl, Vfnnp Nfvzbi, ba gur fbeg bs raivebazrag gung ur guvaxf jbexf orfg sbe vg. Ner gurer ernfbaf jr fubhyq pbafvqre lbh zber xabjyrqtrnoyr guna Nfvzbi ba guvf gbcvp? (Qvq lbh gnxr gur gvzr gb ernq Nfvzbi'f rffnl?)

Urer'f nabgure rkcreg ba gur gbcvp bs perngvivgl: uggcf://ivzrb.pbz/89936101

V frr n ybg bs nterrzrag jvgu Nfvzbi urer. Lbhe xvaqretnegra nanybtl zvtug or zber ncg guna lbh ernyvmr--V guvax zbfg crbcyr ner ng gurve zbfg perngvir jura gurl ner srryvat cynlshy.

uggc://jjj.birepbzvatovnf.pbz/2016/11/zlcynl.ugzy

Lbh unir rvtugrra gubhfnaq xnezn ba Yrff Jebat. Naq lrg lbh unira'g fhozvggrq nalguvat ng nyy gb Qvfphffvba be Znva. Lbh'er abg gur bayl bar--gur infg znwbevgl bs Yrff Jebat hfref nibvq znxvat gbc-yriry fhozvffvbaf. Jul vf gung? Gurer vf jvqrfcernq nterrzrag gung YJ fhssref sebz n qrsvpvg bs pbagrag. V fhttrfg perngvat n srj gbc-yriry cbfgf lbhefrys orsber gnxvat lbhe bja bcvavba ba gurfr gbcvpf frevbhfyl.

Comment author: Lumifer 09 December 2016 03:53:23PM *  1 point [-]

I told you the other day that I find you a difficult person to have discussions with

Yes. This is unfortunate, but I cannot help you here.

if you made an effort to think of things I would say in response to your points, and then wrote in anticipation of those things

I think it's a bad idea. I can't anticipate your responses well enough (in other words, I don't have a good model of you) -- for example, I did not expect you to take five million candidate hypotheses. And if I want to have a conversation with myself, why, there is no reason to involve you in the process.

The existence of clickbait tells us basically nothing about how useful it would be for your average Less Wronger to spend more time generating hypotheses.

We didn't get to an average Lesswronger generating hypotheses yet. You've introduced a new term -- "interestingness" and set it in opposition to truth (or should it have been truthiness?) As far as I can see, clickbait is just a subtype of "interestingness" -- and if you want to optimize for "interestingness", you would tend to end up with clickbait of some sort. And I'm not quite sure what does it have to do with the propensity to generate hypotheses.

If five hypotheses were all I had time to evaluate, I could simply discard everything after the first five.

If a correct hypothesis was guaranteed to be included in your set, you would discard the true one in 99.9999% of the cases, then.

The initial stage is a basic plausibility check which can happen in just a few seconds.

Let's try it. "Earth rotates around the Sun" -- ha-ha, what do I look like, an idiot? Implausible. Next!

...Isaac Asimov, on the sort of environment that he thinks works best for it.

Where "it" is "writing fiction"?

Your kindergarten analogy might be more apt than you realize--I think most people are at their most creative when they are feeling playful.

LOL. Kids are naturally playful -- the don't need a kindergarten for it. In fact, kindergartens tend to use their best efforts to shut down kids creativity and make them "less disruptive", "respectful", "calm", and all the things required of a docile shee... err... member of society.

I suggest creating a few top-level posts yourself before taking your own opinion on these topics seriously.

I neither see much reason to do so, nor do I take my own opinion seriously, anyway :-P

Do you want playfulness or seriousness? Pick a side.

Comment author: John_Maxwell_IV 10 December 2016 01:51:50AM *  2 points [-]

I told you the other day that I find you a difficult person to have discussions with

Yes. This is unfortunate, but I cannot help you here.

Is this due to lack of ability or lack of desire? If lack of ability, why do you think you lack this ability?

Comment author: Lumifer 10 December 2016 05:24:49AM 1 point [-]

I lack the ability to change you and I lack the desire to change myself.

Comment author: plethora 07 December 2016 06:34:22AM 0 points [-]

Online communities do not have a strong comparative advantage in compiling and presenting facts that are well understood. That's the sort of thing academics and journalists are already paid to do.

But academics write for other academics, and journalists don't and can't. (They've tried. They can't. Remember Vox?)

AFAIK, there isn't a good outlet for compilations of facts intended for and easily accessible by a general audience, reviews of books that weren't just written, etc. Since LW isn't run for profit and is run as outreach for, among other things, CFAR, whose target demographic would be interested in such an outlet, this could be a valuable direction for either LW or a spinoff site; but, given the reputational risk (both personally and institutionally) inherent in the process of generating new ideas, we may be better served by pivoting LW toward the niche I'm thinking of -- a cross between a review journal, SSC, and, I don't know, maybe CIA (think World Factbook) or RAND -- and moving the generation and refinement of ideas into a separate container, maybe an anonymous blog or forum.

Comment author: satt 07 December 2016 11:17:14PM *  1 point [-]

But academics write for other academics, and journalists don't and can't. (They've tried. They can't. Remember Vox?)

Would that be Vox, Vox, or Vox?

Edit, 5 minutes later: a bit more seriously, I'm not sure I'd agree that "academics write for other academics" holds as a strong generalization. Many academics focus on writing for academics, but many don't. I think the (relatively) low level of information flow from academia to general audiences is at least as much a demand-side phenomenon as a supply-side one.

Comment author: Good_Burning_Plastic 08 December 2016 09:47:33AM *  1 point [-]

Many academics focus on writing for academics, but many don't.

Given "publish or perish", usually the latter won't stay in academia for long.

Comment author: satt 10 December 2016 03:58:06PM 0 points [-]

I'd be reluctant to go as far as "usually", but yes, publish-or-perish norms are playing a role here too.

Comment author: John_Maxwell_IV 08 December 2016 08:22:00AM *  0 points [-]

Academics write textbooks, popular books, and articles that are intended for a lay audience.

Nevertheless, I think it's great if LW users want to compile & present facts that are well understood. I just don't think we have a strong comparative advantage.

LW already has a reputation for exploring non-mainstream ideas. That attracts some and repels others. If we tried to sanitize ourselves, we probably would not get back the people who have been repulsed, and we might lose the interest of some of the people we've attracted.