Less Wrong is a community blog devoted to refining the art of human rationality. Please visit our About page for more information.

Comment author: Vladimir_Nesov 06 March 2015 09:09:53AM *  6 points [-]

He doesn't "diss UFAI concerns", he says "I don’t know how to productively work on that", which seems accurate for the conventional meaning of "productive" even after looking into the issue. He doesn't address the question of whether it's worthwhile to work on the issue unproductively, illustrating his point with the analogy (overpopulation on Mars) where it's clearly not.

Comment author: Kaj_Sotala 06 March 2015 01:54:53PM 8 points [-]

His article commentary on G+ seems to get more into the "dissing" territory:

Enough thoughtful AI researchers (including Yoshua Bengio​, Yann LeCun) have criticized the hype about evil killer robots or "superintelligence," that I hope we can finally lay that argument to rest. This article summarizes why I don't currently spend my time working on preventing AI from turning evil.

Comment author: Kaj_Sotala 03 March 2015 06:41:39AM *  12 points [-]

Kevin Simler's Prickles and Goo is also related: he talks about the advantages and disadvantages of having "prickly" (strong, solid) and "gooey" (weak, flexible) identities:

There are at least two significant benefits to being prickly:

  1. Having a large prickly identity makes you more socially salient. By standing up for yourself, you stand out from the crowd. You create a strong identity, and just as important, a legible identity. This stability and legibility make you a reliable platform for others to build on.
  2. George Bernard Shaw famously wrote: "The reasonable man adapts himself to the world; the unreasonable one persists in trying to adapt the world to himself. Therefore all progress depends on the unreasonable man." The unreasonable man here is the prickly one. People who are gooey conform themselves too easily to their surroundings, and don't seek to impose themselves on the world.

The prickly people I know are all extremely useful in these two ways.

But clearly there are also benefits to being gooey:

  1. Having a gooey identity makes you more flexible. We speak of people who "go with the flow," or who are "easy-going." Flexibility makes you easier to work with, more amenable to compromise, more of a "team player." Whenever these things are useful, so is having a small-prickly/large-gooey identity.
  2. Being gooey leads to having a smaller ego. When ego gets in the way of making good decisions, gooey has the advantage.
  3. Finally, if Buddhism teaches us anything, it's that having a smaller prickly identity and a larger gooey identity makes us happier. The second of the Four Noble Truths holds that "the origin of suffering is attachment," i.e., prickles. "Let go," the Buddha invites us, "and you will be happy."


Luckily we don't need to make a single, categorical choice between having a prickly self and having a gooey self. We can make separate choices for each component of our identities. So the really important question becomes:

Which parts of your identity should be prickly, vs. which parts gooey?

Deciding where to put something — in the prickly part or the gooey part — depends on what game you're playing and how much you want to win.

The game of morality requires that you put your moral principles in the prickly, uncompromising part of your identity. If you don't, we'll accuse you of having loose (i.e. gooey) morals.

The game of science asks you to put all of your scientific beliefs in the gooey part of your identity, while holding fast and prickly to the principle of truth (and nothing else). To do otherwise — to maintain a scientific belief as part of your core, inviolable, prickly identity — is to sacrifice truth on the altar of ego: the cardinal sin of science.

The game of political debate is played by placing your political beliefs and values in the prickly parts of your identity. Scientifically this is reprehensible, but it seems necessary for winning arguments. (It may also be why science and politics go together like oil and water.) On the other hand, to operate successfully as a politician — to actually get things done, in a very messy world — requires the ability to become gooey at strategic moments.

The game of art — i.e., being an artist, especially of the "high art" variety — requires maintaining a hard (and unique) aesthetic sense. "Selling out" is the accusation we level at an artist who turns gooey in service of commercial success.

The game of art appreciation, on the other hand, is played (optimally) by treating all your aesthetic sensibilities as flexible and gooey. The more you hold fast to liking certain types of things (which, by implication, means you dislike other types of things), the narrower your appreciative range.

We could analyze other games this way (finance, happiness, marriage, acting, etc.), but I think you get the idea. Winning requires strategic identity construction — deciding when to yield and when to prickle.

Comment author: jaime2000 02 March 2015 03:06:29PM *  11 points [-]

365tomorrows recently published a hard science-fiction story of mine called "Procrastination", which was inspired by the ideas of Robin Hanson. I believe LessWrong will find it enjoyable.

Comment author: Kaj_Sotala 02 March 2015 07:19:05PM 1 point [-]

I thought that the ideas seemed awfully familiar, when the story popped up on 365!

Comment author: Mark_Friedenbach 11 February 2015 06:54:33PM 1 point [-]

I'm not sure whether this field is actually where my comparative advantage lies.

What else are you considering?

I would advise that's only half the the equation though. You should also weight by how unique that contribution would be. We simply don't have enough people doing AGI work like concept formation. Not to place too much pressure, but if you don't work on this then it's not clear who would. It's an underfunded area academically (hence these grants are a great opportunity), and too long term to be a part of industrial research efforts...

Comment author: Kaj_Sotala 01 March 2015 06:41:54PM 0 points [-]

What else are you considering?

Rationality training and community-building, basically.

But I just submitted my FLI grant application for the concept learning project anyway. :-)

Comment author: Kaj_Sotala 23 February 2015 03:09:29PM 4 points [-]

Posts related to this topic: Kurzban et al. on opportunity cost models of mental fatigue and resource-based models of willpower, Why Self-Control Seems (but may not be) Limited. Both discuss the "cognitive load depletes willpower" hypothesis.

Comment author: dxu 19 February 2015 03:36:26AM *  0 points [-]

Just checking, but verbal and mathematical reasoning skills are positively correlated, right? This assertion seems to be supported by the fact that many (I'd go so far as to say nearly all) LW users have high verbal intelligence (as evidenced by the general quality of the comments here) and most of them seem to have high mathematical intelligence as well (as evidenced by the many posts on decision theory, game theory, and other fields of mathematics). If the two are correlated, do you know the coefficient of correlation?

Comment author: Kaj_Sotala 19 February 2015 01:24:08PM *  2 points [-]

One reference that also comes to mind is this box from Deary 2001. If we assume "verbal intelligence" to correspond to the "verbal comprehension" group factor in the diagram, and "mathematical reasoning" to correspond to its "perceptual organization" factor (since perceptual organization's associated subtests of picture completion, block design, matrix reasoning, and picture arrangement sound the most similar to Raven's matrices; though "arithmetic" is in the working memory factor) then if I'm thinking about this correct, those two group factors would share 65% (100 * 0.86^2 * 0.94^2) of their variance.

Comment author: Luke_A_Somers 18 February 2015 06:20:42PM *  3 points [-]

I'm pretty sure it's 2 (same as Vaniver, gwillen, and Alicorn). Was that what popped out at you?

It didn't take me less than 10 seconds to come up with this (I'd be surprised if it was less than 20 or more than 40 to find it and check, but I didn't check the clock). I tried to figure out the pattern without priming by looking at the possible answers, so there wasn't even really a chance to have the right answer pop out in this fashion.

ETA: I have taken Raven's Matrices before, so I was ready.

Comment author: Kaj_Sotala 18 February 2015 09:55:25PM 2 points [-]

Was that what popped out at you?

Nope: I got the fourth one. Guess it was just my brain playing tricks at me, then. :)

(I tried to do it using basically just unthinking pattern recognition, looking at the sequence of patterns as a sequence of movement: somehow, using that criteria, the fourth one seemed to display "the most similar kind of motion" as compared to the above examples, even though a more conscious analysis suggested that it seemed to be breaking some of the rules of the above sequences, and I couldn't come up with any verbal summary of the rule. But it still just felt so right somehow.)

Comment author: Kaj_Sotala 18 February 2015 03:49:08PM 5 points [-]

Out of curiosity, what is the correct answer to the example Raven's item? One of the answer candidates popped out to me immediately as the most likely one, and I'm interested to know whether that's a sign of me having superior pattern recognition ability or whether a part of me just wants to believe that.

Comment author: JonahSinick 13 February 2015 07:56:31PM *  6 points [-]

And I think that anyone who makes even the slightest substantial contribution to homotopy type theory is doing interesting, original work.

I partially respond to this here.

I think the Low-Hanging Fruit Complaint is more often a result of not knowing where there's a hot, productive research frontier than of the universe actually lacking interesting new mathematics to uncover.

There's a lot of potential for semantic differences here, and risk of talking past each other. I'll try to be explicit. I believe that:

  • There are very few people who have a nontrivial probability of discovering statements about the prime numbers that are both true, that people didn't already believe to be true, and that people find fascinating.
  • The same is not far from being true for all areas of math that have been mainstream for 100+ years: algebraic topology, algebraic geometry, algebraic number theory, analytic number theory, partial differential equations, Lie Groups, functional analysis, etc.
  • There is a lot of rich math to be discovered outside of the areas that pure mathematicians have focused on historically, and that people might find equally fascinating. In particular, I believe this to be true within the broad domain of machine learning.
  • There are few historical examples of mathematicians discovering interesting new fields of math without being motivated by applications.
Comment author: Kaj_Sotala 14 February 2015 07:39:30PM 1 point [-]

Upvoted for being specific.

Comment author: Kaj_Sotala 12 February 2015 08:36:18PM *  0 points [-]

Interface Determinism: Assuming that all that counts in analyzing the nature and behavior of a system is what comes or goes across the system-environment interface.

While this is somewhat different from what I take the author to mean, this reminded me of the two possible mental models of the self-environment relationship that Kevin Simler pictures under the "inhabitance" heading of Ethology and Personal Identity.

View more: Next