Less Wrong is a community blog devoted to refining the art of human rationality. Please visit our About page for more information.

Open thread, Apr. 03 - Apr. 09, 2017

1 Post author: Elo 03 April 2017 06:58AM

Notes for future OT posters:

1. Please add the 'open_thread' tag.

2. Check if there is an active Open Thread before posting a new one. (Immediately before; refresh the list-of-threads page before posting.)

3. Open Threads should start on Monday, and end on Sunday.

4. Unflag the two options "Notify me of new top level comments on this article" and "

Comments (76)

Comment author: CellBioGuy 07 April 2017 07:30:56PM *  6 points [-]

Final version of thesis going out within 4 days. Getting back into a semi-regular schedule after PhD defense, death in the family, and convergence of job-search on a likely candidate in quick succession. Astrobiology writing likely to restart soon. Possible topics include:

  • schools of thought in life origins research
  • the nature of LUCA
  • recent work on the evolution of potentiated smart animal lineages on Earth
  • WTF are eukaryotes anyway
  • the fallacies of the Fermi paradox/ 'great filter' concepts
  • the fallacies of SETI as it is currently performed
Comment author: MaryCh 07 April 2017 03:35:29PM 4 points [-]

I'm thinking on writing a post on doing 'lazy altruism', meaning 'something having a somewhat lasting effect that costs the actor only a small inconvenience, and is not specifically calculated to do the most amount of good - only the most amount per this exact effort."

Not sure I'm not too lazy to expand on it, though.

Comment author: Elo 08 April 2017 09:13:07PM 2 points [-]

write it!

Comment author: Viliam 11 April 2017 04:37:41PM 0 points [-]

Potentially very useful, considering how many lazy people are out there.

Comment author: Lumifer 03 April 2017 05:04:30PM 4 points [-]

Tyler Cowen and Ezra Klein discuss things. Notably:

Ezra Klein: The rationality community.

Tyler Cowen: Well, tell me a little more what you mean. You mean Eliezer Yudkowsky?

Ezra Klein: Yeah, I mean Less Wrong, Slate Star Codex. Julia Galef, Robin Hanson. Sometimes Bryan Caplan is grouped in here. The community of people who are frontloading ideas like signaling, cognitive biases, etc.

Tyler Cowen: Well, I enjoy all those sources, and I read them. That’s obviously a kind of endorsement. But I would approve of them much more if they called themselves the irrationality community. Because it is just another kind of religion. A different set of ethoses. And there’s nothing wrong with that, but the notion that this is, like, the true, objective vantage point I find highly objectionable. And that pops up in some of those people more than others. But I think it needs to be realized it’s an extremely culturally specific way of viewing the world, and that’s one of the main things travel can teach you.

Comment author: 9eB1 04 April 2017 04:06:36PM *  5 points [-]

I think no one would argue that the rationality community is at all divorced from the culture that surrounds it. People talk about culture constantly, and are looking for ways to change the culture to better address shared goals. It's sort of silly to say that that means it should be called the "irrationality community." Tyler Cowen is implicitly putting himself at the vantage point of a more objective observer with the criticism, which I find ironic.

Where Tyler is wrong is that it's not JUST another kind of culture. It's a culture with a particular set of shared assumptions, and it's nihilistic to imply that all cultures are equal no matter from what shared assumptions they issue forth. Cultures are not interchangeable. Tyler would also have to admit (and I'm guessing he likely would admit if pressed directly) that his culture of mainstream academic thought is "just another kind of religion" to exactly the same extent that rationality is, it's just less self-aware about that fact.

As an aside, I think Lumifer is a funny name. I always associate it with Lumiere from Beauty and the Beast, and with Lucifer. Basically I always picture your posts as coming from a cross between a cartoon candle and Satan.

Comment author: Lumifer 04 April 2017 05:30:46PM *  2 points [-]

It's sort of silly to say that that means it should be called the "irrationality community."

Notice the name of this website. It is not "The Correct Way To Do Everything".

it's not JUST another kind of culture. It's a culture with a particular set of shared assumptions

Don't ALL cultures have their own particular set of shared assumptions? Tyler's point is that the rationalist culture, says Tyler, sets itself above all others as it claims to possess The Truth (or at least know the True Paths leading in that general direction) -- and yet most cultures have similar claims.


Lucifer is the bringer of light (Latin: lux). Latin also has another word for light: lumen (it's the same root but with the -men suffix). Just sayin' :-P

But I will also admit that the idea of an all-singing all-dancing candelabra has merit, too :-)

Comment author: Brillyant 04 April 2017 04:47:54PM 1 point [-]

It's sort of silly to say that that means it should be called the "irrationality community." Tyler Cowen is implicitly putting himself at the vantage point of a more objective observer with the criticism, which I find ironic.

It did seem to be a pretty bold and frontal critique. And "irrationality community" is probably silly. But I agree LW, et al has at times a religious and dogmatic feel to it. In this way the RC becomes something like the opposite of the label it carries. That seems to be his point.

As an aside, I think Lumifer is a funny name. I always associate it with Lumiere from Beauty and the Beast, and with Lucifer. Basically I always picture your posts as coming from a cross between a cartoon candle and Satan.

Yes. Yes.

If this wasn't exactly the mental image I had of Lumifer before, then it is now.

Maybe a bit more Satan than cartoon

Comment author: Viliam 04 April 2017 11:46:06AM *  4 points [-]


(posted mostly for the challenge of writing this properly in markdown syntax. actually, it was quite easy)

Comment author: Lumifer 10 April 2017 04:45:51PM 1 point [-]

Razib Khan also offers a comment.

Comment author: Nate_Rausch 05 April 2017 04:35:56AM 1 point [-]

A bit tongue-in-cheeck, but how about taking Tyler's unfair label as a proposal?

We could start the rationality religion, without the metaphysics or ideology of ordinary religion. Our God could be everything we do not know. We worship love. Our savior is the truth. We embrace forgiveness as the game-theoretical optimal modified tit-for-tat solution to a repeated game. And so on.

We thoroughly investigate and aggregate the best knowledge humanity currently has on how to live. And we create Rationality Temples worldwide. There will be weekly congregations, with talks on a sequence, with following discussions, on topics such as signalling, bayesian thinking, cognitive biases. We propose a three step way to heaven on earth: identifying worthwhile causes, charting effective solutions and taking actions to achieve it. Lifetime goal is writing a sequence. Compassion meditation and visualisation prayer once per day. Haha, okay perhaps I'm overdoing it.

Using the well-established concepts, rituals and memes of religion is easy to mock, but what if it is also an effective way to build our community and reach our goals?

Comment author: MrMind 05 April 2017 08:02:11AM 2 points [-]

what if it is also an effective way to build our community and reach our goals?

It surely is an effective way, since by this mean all kinds of silly causes have been pursued. But creating a religion out of rationality (lowercase) would defeat its purpose: in the span of a year, rationality would become the password to learn by memory and the beginning structures will solidify as an attire.
Religions are appealing exactly because they exempt their members from thinking on their own and accepting hard truths: rationality has instead more in common with martial arts, they are mostly a question of training and learning to take many hits.

Comment author: Nate_Rausch 05 April 2017 08:15:25PM 0 points [-]

Well, yes, but couldn't one just make a new religion without those attributes. For example the first of the 10 commandments could be: Question everything, including these texts. Be a student, not a follower. Finding fault in ourself is the highest virtue, free speech etc. ? :-)

Comment author: Lumifer 05 April 2017 03:04:38PM *  0 points [-]

Our God could be everything we do not know.

Literally God of the Gaps! :-)

We worship love

Which love?

Comment author: gjm 05 April 2017 04:52:05PM 2 points [-]

Tangential: I think the "four loves" thing is a bit of a cheat, like the "fifty Eskimo words for snow" meme. The Greeks had different words for describing different kinds of positive interpersonal affect -- but so do we! We have "affection" and "friendship" and "devotion" and "lust" and "loyalty" and "benevolence" and so on.

That doesn't mean there's anything wrong with noticing when one of those words (in this case "love") gets used very broadly, and it doesn't change the fact that "see how other people classify things" is a useful heuristic. But I am entirely unconvinced that there's anything very special about the Ancient Greeks, or the notion of love, in this regard -- other than various historical contingencies involving C S Lewis, Christianity, and the history of the New Testament.

Comment author: Lumifer 05 April 2017 05:29:53PM 0 points [-]

But I am entirely unconvinced that there's anything very special about the Ancient Greeks

Well, they were pretty special, being the cradle of the Western civilization 'n'all, but in this specific case all I intended was to give the OP a possible list of specific meanings of the word "love" to consider.

Comment author: Nate_Rausch 05 April 2017 08:13:12PM 0 points [-]

Fair point, well I don't think romantic love is worthy of sacred status in the irrationality religion. Tough those four neither seemed quite to fit the love I had in mind.

Perhaps something closer the buddhist concept of bodhisattva, meaning altruistic love for all sentient beings?

Comment author: Lumifer 06 April 2017 12:57:47AM 1 point [-]

altruistic love for all sentient beings?

Sounds like the plain old Christian love, but with a new cool label :-/

Comment author: bogus 06 April 2017 01:11:59AM *  1 point [-]

Ah, Christian love... the kind of altruistic love that makes people tell you that you're a fallen and depraved creature due to the sin of Adam and thus will be burning in Hell forever unless you "accept Jesus as your savior" by becoming a Christian ASAP. (If precedent is any guide, I can already guess that Roko's basilisk will be featured prominently in the new "rationality religion"!)

Comment author: Lumifer 06 April 2017 02:09:30AM *  0 points [-]

Well, our baseline is what, Buddhist love? There is Buddhist hell as well and surprise! it doesn't sound like a pleasant place. You get there through being enslaved by your lusts and desires -- unless, of course, you accept the teachings of Buddha ASAP :-P

Comment author: Elo 04 April 2017 09:00:00PM 1 point [-]

irrationality community

No one is more critical of us than ourselves. "LessWrong" is lesswrong for being humble about it. Hopefully that humility sticks around for a very long time.

Comment author: philh 05 April 2017 09:31:22AM 6 points [-]

No one is more critical of us than ourselves.

This seems untrue. For example, RationalWiki.

In the past I could also have pointed to some individuals (who AFAIK were not associated with RW, but they could have been) who I think would have counted. I can't think of any right now, but I expect they still exist.

Comment author: dxu 05 April 2017 09:38:17PM 0 points [-]

That's fair, but I also think it largely misses the point of Elo's comment. Here, have (an attempt at) a rephrase:

No community is as prone to self-criticism as the rationalist community.

Comment author: philh 06 April 2017 09:22:50AM 1 point [-]

I think I would rather say, less superlatively, that we're unusually good at self-criticism.

(I do note that I'm comparing my inside view of the rationalist community with my outside view of other communities, so I shouldn't put too much confidence in this.)

(But yes, I agree that I was ignoring the thing that Elo was actually trying to point at.)

Comment author: Elo 06 April 2017 12:34:55AM 0 points [-]

yes. the fact that we know of RW, and this mention, and try to understand what is going on here. is a good start.

Comment author: MrMind 05 April 2017 08:03:26AM 4 points [-]

Humility is good, but calibration is better.

Comment author: 9eB1 04 April 2017 10:15:50PM 0 points [-]

Bryan Caplan responded to this exchange here

Comment author: tristanm 05 April 2017 04:55:13PM 1 point [-]

many well-known, devastating counter-examples.

I would object to calling these "devastating counter-examples", they're more like unsolved problems. It seems overly dramatic. I'm not a perfect Bayesian agent, I use my intuitions a lot, but that is not grounds on which to reject Bayesianism, and I think we could say something similar about consequentialism. I may not know how to perfectly measure relative happiness, or perfectly predict the future, but it doesn't seem like that should be grounds to reject consequentialism entirely, in favor of alternatives which don't cope with those issues either.

Comment author: dxu 05 April 2017 10:01:26PM 1 point [-]

One very common error people make is to treat "utilitarianism" and "consequentialism" as if they were one and the same thing. Utilitarianism makes claims about what is moral and what is not. Consequentialism makes claims about what sort of properties a moral criterion should have. Criticisms about utilitarianism, therefore, are often taken also as criticisms of consequentialism, when in fact the two are distinct concepts!

Comment author: Good_Burning_Plastic 05 April 2017 11:07:28PM 0 points [-]

On the other hand, utilitarianism is a subset of consequentialism, so whereas a criticism of utilitarianism is not necessarily a criticism of consequentialism, the converse is true.

Comment author: Elo 03 April 2017 07:04:24AM 4 points [-]

Curious about if this is worth making into it's own weekly thread. Curious as to what's being worked on, in personal life, work life or just "cool stuff". I would like people to share, after all we happen to have similar fields of interest and similar fields we are trying to tackle.

Projects sub-thread:

  • What are you working on this week (a few words or a serious breakdown)(if you have a list feel free to dump it here)?
  • What do you want to be asked about next week? What do you expect to have done by then?
  • Have you noticed anything odd or puzzling to share with us?
  • Are you looking for someone with experience in a specific field to save you some search time?
  • What would you describe are your biggest bottlenecks?
Comment author: Elo 04 April 2017 09:58:47AM 2 points [-]

I am working on:

  • learning colemak
  • vulnerability, circling (relational therapy stuff) investigating the ideas around it.
  • trialling supplements: Creatine, Protein, Citrulline malate. Adding in Vitamins: C, D, Fish oil, Calcium, Magnesium, Iron, Distant future trials: 5HTP, SAMe. preliminary SAMe and 5HTP were that they make me feel like crap.
  • promoting the voluntary euthanasia party of NSW (Australia) (political reform of the issue)
  • emptying my schedule to afford me more "free" time in which to write up posts.
  • contemplating book topic ideas.
  • trying to get better routines going, contemplating things like "I only sleep at home", and cancelling all meeting and turning off all connected media for a week.

my biggest bottlenecks are myself (sometimes focus) and being confident at what will have a return VS not have a return. (hence many small experiements)

Comment author: WalterL 05 April 2017 03:51:15PM 0 points [-]

You are working on vulnerability? I'm not sure I understand.

Comment author: Elo 06 April 2017 12:38:39AM 0 points [-]


yes. This talk seems to imply value from being vulnerable in social settings to connect with people. I am experimenting to see if it's worth it or better.

Comment author: username2 05 April 2017 01:59:30PM 0 points [-]

Where else do you sleep?

Comment author: Elo 06 April 2017 12:40:26AM 0 points [-]

This may seem obvious to me but I sleep at the houses of partners, my parents house, a property I was doing some work for ~1.5 hrs away from home. The point is that I have less control over the bed, bedroom setup, my sleep and my morning routine if I am not in the bed that I engineered the routine around.

Comment author: MaryCh 03 April 2017 05:48:08PM *  2 points [-]

Translating a novel (really, a collection of essays) about WWII massacres of Jews in Kyiv & the rise of neo-nazism in post-Soviet republics (and much in between). It will take me a few months, probably, since this is a side job.

Overall impression: the past is easier to deal with, because it is too horrible. Imagine 10^5 deaths. Although I unfortunately know the place where it happened, & he includes personal stories (more like tidbits), so the suspension of disbelief takes some effort to maintain. But the 'present' part - a series of the author's open letters to mass-media and various officials about pogroms and suchlike that went unpunished - is hard: he keeps saying the same over and over. (Literally. And his style is heavy on the reader.) After a while the eye glazes over & notices only that the dates and the addresses change, but the content doesn't, except for the growing list of people who had not answered.

Just had not answered.

Now this is - easy to imagine.

Maybe this isn't odd, but I had thought it would be the other way around.

Comment author: Lumifer 03 April 2017 06:15:37PM 1 point [-]

the past is easier to deal with, because it is too horrible. Imagine 10^5 deaths.

" A single death is a tragedy; a million deaths is a statistic" -- a meme

Comment author: MaryCh 03 April 2017 06:18:04PM 1 point [-]

Ah, but what if you have walked above their bones?

Comment author: Lumifer 03 April 2017 06:57:09PM 1 point [-]

You always walk over bones, it's just that you know of some and don't know of others.

Comment author: MaryCh 04 April 2017 09:24:07AM 1 point [-]

I don't know how you do it, but you seldom fail to cheer me up. Even a little bit.


Comment author: philh 03 April 2017 09:44:24AM *  2 points [-]

I'm working on

  • a graphing library for python (ggplot2's conceptual model with a pythonic API)
  • writing cliffs notes for Order Without Law (a book about how people settle disputes without turning to the legal system)
  • learning ukelele with Yousician (currently making extremely slow progress on the "double stops" lesson, I find it really hard to consistently strum two strings at once)
  • trying to write an essay about how heuristics can sometimes be anti-inductive (they become less accurate the more they're applied), and how we don't really seem to have any cultural awareness of this problem even though it seems important

I might have the last one complete by next week, but the others are fairly long-term projects.

Comment author: Viliam 03 April 2017 08:50:50AM 2 points [-]

Reminds me, we didn't have a bragging thread for some time.

What would you describe are your biggest bottlenecks?

I think I'd like to see this as a separate topic (probably monthly, because such things take time).

However, just do as you wish, and then we'll see how it works.

Comment author: moridinamael 06 April 2017 10:13:28PM 1 point [-]

I'm working on a podcast read-through of the web serial Worm. We've just put out the fifth episode of the series. It's becoming pretty popular by our standards.

Comment author: Darklight 06 April 2017 10:22:33AM 1 point [-]

I recently made an attempt to restart my Music-RNN project:


Basically went and made the dataset five times bigger and got... a mediocre improvement.

The next step is to figure out Connectionist Temporal Classification and attempt to implement Text-To-Speech with it. And somehow incorporate pitch recognition as well so I can create the next Vocaloid. :V

Also, because why not brag while I'm here, I have an attempt at an Earthquake Predictor in the works... right now it only predicts the high frequency, low magnitude quakes, rather than the low frequency, high magnitude quakes that would actually be useful... you can see the site where I would be posting daily updates if I weren't so lazy...


Other than that... I was recently also working on holographic word vectors in the same vein as Jones & Mewhort (2007), but shelved that because I could not figure out how to normalize/standardize the blasted things reliably enough to get consistent results across different random initializations.

Oh, also was working on a Visual Novel game with an artist friend who was previously my girlfriend... but due to um... breaking up, I've had trouble finding the motivation to keep working on it.

So many silly projects... so little time.

Comment author: turchin 05 April 2017 08:09:08AM 3 points [-]

Our article about using nuclear submarines as refuges in case of a global catastrophe has been accepted for the Futures journal and its preprint is available online.


Recently many methods for reducing the risk of human extinction have been suggested, including building refuges underground and in space. Here we will discuss the perspective of using military nuclear submarines or their derivatives to ensure the survival of a small portion of humanity who will be able to rebuild human civilization after a large catastrophe. We will show that it is a very cost-effective way to build refuges, and viable solutions exist for various budgets and timeframes. Nuclear submarines are surface independent, and could provide energy, oxygen, fresh water and perhaps even food for their inhabitants for years. They are able to withstand close nuclear explosions and radiation. They are able to maintain isolation from biological attacks and most known weapons. They already exist and need only small adaptation to be used as refuges. But building refuges is only “Plan B” of existential risk preparation; it is better to eliminate such risks than try to survive them.

Full text: http://www.sciencedirect.com/science/article/pii/S0016328716303494?np=y&npKey=6dcc6d35057e4c51bfd8d6933ab62c6d4a1604b5b71a40f060eb49dc7f42c9a1

Comment author: morganism 08 April 2017 11:05:08PM 2 points [-]

Back when SeaSteading forum was going, one of the folks was working on building cement submarines, as a pre-design to growing them like coral.

Comment author: morganism 10 April 2017 11:40:33PM 2 points [-]

"Modafinil-Induced Changes in Functional Connectivity in the Cortex and Cerebellum of Healthy Elderly Subjects"


"CEDs may also help to maintain optimal brain functioning or compensate for subtle and or subclinical deficits associated with brain aging or early-stage dementia."

"In the modafinil group, in the post-drug period, we found an increase of centrality that occurred bilaterally in the BA17, thereby suggesting an increase of the FC of the visual cortex with other brain regions due to drug action." FC analysis revealed connectivity increase within the cerebellar Crus I, Crus II areas, and VIIIa lobule, the right inferior frontal sulcus (IFS), and the left middle frontal gyrus (MFG)."

"These frontal areas are known to modulate attention levels and some core processes associated with executive functions, and, specifically, inhibitory control and working memory (WM). These functions depend on each other and co-activate frontal areas along with the posterior visual cortex to re-orient attention toward visual stimuli and also enhance cognitive efficiency.

Data on behavioral effects of modafinil administration in our study group are missing and, at this stage, we can only provide theoretical speculations for the functional correlates of the regional activations that we have found to be promoted by the drug."

Comment author: MrMind 07 April 2017 10:43:40AM *  2 points [-]

I'm still mulling over the whole "rationalism as a religion". I've come to the conclusion that there are indeed two axioms that are shared by the rational-sphere that we-cannot-quite-prove, and whose variations produce different cultures.
I call them underlying reality and people are perfect.
"Underlying reality" (U): refers to the existence of a stratum of reality that is independent from our senses and our thoughts, whose configurations gives the notion of truth as correspondence.
"People are perfect" (P): instead refers to the truth of mental ideation that people might have, whether they are (or there's a subset that is) always right.
Here's a rough scheme:
U, P: religion. Our feelings reflect directly the inspiration of a higher source of truth.
U, not P: rationalism. We are imperfect hardware in a vast and mostly unknowable world.
not U, P: the most uncertain category. Perhaps magic? There's no fixed, underlying truth but our thoughts can influnce it.
not U, not P: postmodernism. Nothing is true and everything's debatable.

I might make this a little more precise in a proper post.

Comment author: tristanm 06 April 2017 11:51:23PM 2 points [-]

In "Strong AI Isn't Here Yet", Sarah Constantin writes that she believes AGI will require another major conceptual breakthrough in our understanding before it can be built, and it will not simply be scaled up or improved versions of the deep learning algorithms that already exist.

To argue this, she makes the case that current deep learning algorithms have no way to learn "concepts" and only operate on "percepts." She says:

I suspect that, similarly, we’d have to have understanding of how concepts work on an algorithmic level in order to train conceptual learning.

However, I feel that her argument was lacking in terms of tangible evidence for the claim that deep-learning algorithms do not learn any high-level concepts. It seems to be based on the observation that we currently do not know how to explicitly represent concepts in mathematical or algorithmic terms. But I think if we are to take this as a belief, we should try to predict how the world would look differently if deep-learning algorithms could learn concepts entirely on their own, without us understanding how.

So what kind of problems, if solved by neural networks, would surprise us if this belief was held? Well, to name a couple of experiments that surprise me, I would probably point out DCGAN and InfoGAN. In the former, they are able to extract visual "concepts" out of the generator network by taking the latent vectors of all the examples that share one kind of attribute of their choosing (in the paper they take "smiling" / "not smiling" and "glasses" / "no glasses") and averaging them. Then they are able to construct new images by doing vector arithmetic in the latent space using this vector and passing them through the generator, so you can take a picture of someone without glasses and add glasses to them without altering the rest of their face, for example. In the second paper, their network learns a secondary latent variable vector that extracts disentagled features from the data. Most surprisingly, their network seems to learn such concepts as "rotation" (among other things) from a data set of 2D faces, even though there is no way to express the concept of three dimensions in this network or have that encoded as prior knowledge somehow.

Just this morning in fact, OpenAI revealed that they had done a very large scale deep-learning experiment using multiplicative LSTMs on Amazon review data. What was more surprising than just the fact they had beaten the benchmark accuracy on sentiment analysis, was that they had done it in an unsupervised manner by using the LSTMs to predict the next character in a given sequence of characters. They discovered that a single neuron in the hidden layer of this LSTM seemed to extract the overall sentiment of the review, and was somehow using this knowledge to get better at predicting the sequence. I would find this very surprising if I believed it were unlikely or impossible for neural networks to extract high-level "concepts" out of data without explicitly encoding it into the network structure or the data.

What I'm getting at here is that we should be able to set benchmarks on certain well-defined problems and say "Any AI that solves this problem has done concept learning and does concept-level reasoning", and update based on what types of algorithms solve those problems. And when that list of problems gets smaller and smaller, we really need to watch out to see if we have redefined the meaning of "concept" or drawn to the tautological conclusion that the problem really didn't require concept level reasoning after all. I feel like that has already happened to a certain degree.

Comment author: bogus 07 April 2017 11:25:24AM *  0 points [-]

The problem with AGI is not that AIs have no ability to learn "concepts", it's that the G in 'AGI' is very likely ill-defined. Even humans are not 'general intelligences', they're just extremely capable aggregates of narrow intelligences that collectively implement the rather complex task we call "being a human". Narrow AIs that implement 'deep learning' can learn 'concepts' that are tailored to their specific task; for instance. the DeepDream AI famously learns a variety of 'concepts' that relate to something looking like a dog. And sometimes these concepts turn out to be usable in a different task, but this is essentially a matter of luck. In the Amazon reviews case, the 'sentiment' of a review turned out to be a good predictor of what the review would say, even after controlling for the sorts of low-order correlations in the text that character-based RNNs can be expected to model most easily. I don't see this as especially surprising, or as having much implication about possible 'AGI'.

Comment author: entirelyuseless 07 April 2017 02:25:22PM 0 points [-]

Humans are general intelligences, and that is exactly about having completely general concepts. Is there something you cannot think about? Suppose there is. Then let's think about that thing. There is now nothing you cannot think about. No current computer AI can do this; when they can, they will in fact be AGIs.

Comment author: Thomas 03 April 2017 04:44:36PM *  1 point [-]
Comment author: Lumifer 03 April 2017 05:07:53PM *  2 points [-]

Your link is broken.

ETA: Thomas fixed the problem.

Comment author: Thomas 03 April 2017 06:02:41PM 0 points [-]

I don't know. I can't reproduce the error, it just works when I click it.

It is strange however, that this number 9056 has been given by the Wordpress. Usually it is the title of the post.

Comment author: Lumifer 03 April 2017 06:08:36PM *  2 points [-]

Your link leads me to a Wordpress login page. To reproduce, try deleting your cookies (or try another browser).

Specifically, your link resolves to this: https://wordpress.com/wp-login.php?redirect_to=https%3A%2F%2Fwordpress.com%2Fpost%2Fprotokol2020.wordpress.com%2F9056

Is that -- https://protokol2020.wordpress.com/2017/04/02/another-intermezzo-intermezzo-problem/ -- the link you mean?

Comment author: Thomas 03 April 2017 07:05:09PM 1 point [-]

Thank you very much! Now, should be fixed!

Comment author: Jordan2386 06 April 2017 09:18:28PM 0 points [-]

I have an idea about how to make rationality great again (bad joke, but I'm serious). The term "theoretical rationality" may be coined by me - idk, but the new meanings to "theoretical and applied rationality" are mine and include 1)optimisation 2)fast completion of goals and ideals 3)updating the list of desirable goals and ideals 4) repeat. Any comments?

Comment author: Elo 06 April 2017 10:28:13PM 1 point [-]

how does this compare to the instrumental and epistemic rationality separation?

Comment author: Jordan2386 18 April 2017 09:30:59AM 0 points [-]

The distinctions are in the application. In my proposed system, the theoretic (epistemic) rationality should mostly be a byproduct of the applied (instrumental) rationality. My view puts a huge emphasis on the knowledge being derived mostly and predominantly derived through (many fast) interactions with the environment, avoiding the pitfalls of fixing scientific "laws" (which themselves are products of many observations). This is not the Bayesian view with priors on what one could expect from looking into some unexpected phenomenon. If it works, it works. If the theory says it can't work, it would still work.

Comment author: MrMind 07 April 2017 08:39:40AM 0 points [-]

the new meanings to "theoretical and applied rationality" are mine


Comment author: Jordan2386 15 April 2017 05:42:31AM 0 points [-]

The conference looks cool, but does not follow the main framework of "theoretical and applied rationality", as defined by me. More specifically, I can use the term "whatever works (most) effectively" not to cause confusion with the otherwise defined "theoretical and applied rationality".What I do with my approach is to completely avoid theorization, unless the interactions with the environment uncover a pattern that needs recording. The main focus of the approach is on the applied end. If one needs to, say, improve wealth for the poorest I can't think of a more efficient way to do so than fast experimentation and coming up with new and better way to do so. The cool bit is that it can be sustainable and empowering, blah, blah, blah (not the focus of this conversation). The idea is that one interacts with the environment to acquire new knowledge. Otherwise one can become detached from the environment as is most of modern maths theory (I study maths in Oxford btw). Collecting theory is very very backwards to the second pillar of rationality - achieving one's goals and ideals.

Comment author: whpearson 06 April 2017 11:13:25AM 0 points [-]

Is there a place in the existential risk community for a respected body/group to evaluate peoples ideas and put them on a danger scale? Or dangerous given assumptions.

If this body could give normal machine learning a stamp of safety then people might not have to worry about death threats etc?

Comment author: MaryCh 05 April 2017 06:12:15PM *  0 points [-]

In some situations my thinking becomes much more structured, I throw out the syntax and the remaining words come in very clear hierarchy and kind of seem to echo shortly. It lasts, perhaps, less than a second.

Examples: "snake - nonvenomous (snake, snake) - dead, where's the head, somebody struck it with something, probably a stick, curse them, ought to collect it, where's vodka"; "snake - viper (viper, viper) - back off, where's camera, damn, it's gone, ought to GPS the place"; "orchid - Epipactis (...pactis) - why not something rarer, again this weed".

Has it been like that for you?

Comment author: whpearson 05 April 2017 10:24:06AM 0 points [-]

I'm wondering what people would think about adopting the term Future Super Intelligences or FSI, rather than AGI or SAGI.

This would cover more scenarios (e.g. uploads/radical augments) where the motivational systems of super powerful actors may not be what we are used to. It would also show that we are less worried about current tech than talking about AIs does, there is that moment when you have to explain you are not worried about backprop.

Comment author: g_pepper 05 April 2017 12:04:01PM 0 points [-]

I never really thought that AGI implied any specific technology; certainly the "G" in AGI rules out the likelihood of AGI referring to current technology since current technology is not generally intelligent. AGI seems to capture what it is we are talking about quite well, IMO - Artificial (i.e. man-made) General Intelligence.

Do you really find yourself explaining that AGI is not the same as backpropagation very often?

Comment author: whpearson 06 April 2017 09:38:07AM *  0 points [-]

AGI seems to capture what it is we are talking about quite well, IMO - Artificial (i.e. man-made) General Intelligence.

I think AGI gets compressed to AI by the mainstream media and then people working on current ML think that you people are worried about their work (which they find ridiculous and so don't want to engage).

An example of the compression is here.

We don't help ourselves calling it airisk.

Comment author: g_pepper 06 April 2017 10:35:39PM 1 point [-]

I think AGI gets compressed to AI by the mainstream media

Actually, the term AI has traditionally encompassed both strong AI (a term which has been mostly replaced by LW and others with "AGI") and applied (or narrow) AI, which includes expert systems, current ML, game playing, natural language processing, etc. It is not clear to me that the mainstream media is compressing AGI into AI; instead I suspect that many mainstream media writers simply have not yet adopted the term AGI, which is a fairly recent addition to the AI jargon. The mainstream media's use of "AI" and the term "AIRISK" are not so much wrong as they are imprecise.

I suspect that the term AGI was coined specifically to differentiate strong AI from narrow AI. If the mainstream media has been slow in adopting the term AGI, I don't see how adding yet another, newer term will help - in fact, doing so will probably just engender confusion (e.g. people will wonder what if anything is the difference between AGI and FSI).

Comment author: whpearson 06 April 2017 11:22:06PM 0 points [-]

AGI has been going back over 10 years? Longer than the term AIrisk has been around, as far as I can tell. We had strong vs weak before that.

AGIrisk seems like a good compromise? Who runs comms for the AGIrisk community?

Imprecision matters when you are trying to communicate and build communities.

Comment author: g_pepper 07 April 2017 01:02:06AM 0 points [-]

AGIrisk seems like a good compromise?

I certainly prefer it to FSIrisk.

Who runs comms for the AGIrisk community?

I doubt anyone does. Terms catch on or fail to catch on organically (or memetically, to be precise).

Imprecision matters when you are trying to communicate and build communities.

Perhaps. But, I doubt that a significant amount of the reluctance to take the unfriendly AGI argument seriously is due to confusion over terminology. Nor is changing terminology likely to cause a lot of people who previously did not take the argument seriously to begin to take it seriously. For example, there are some regulars here on LW who do not think that unfriendly AGI is a significant risk. But I doubt that any LW regular is confused about the distinction between AGI and narrow AI.

Comment author: ChristianKl 03 April 2017 09:48:37AM 0 points [-]

Are there any studies that determine whether regular coffein consumption has a net benefit? Or does the body produce enough additional receptors to counteract it?

Comment author: Dagon 03 April 2017 11:01:05PM 1 point [-]

I like "coffein" as a portmanteau. I will try to introduce it into conversation and see who notices.

Comment author: PotterChrist 03 April 2017 12:52:46PM *  0 points [-]

Google is your friend. Here is an example link:

http://www.sciencedirect.com/science/article/pii/0278691595000933 I have smird'(proof)#regular question, did this just safe it but only for me?

Comment deleted 03 April 2017 12:55:08PM [-]
Comment author: PotterChrist 03 April 2017 09:05:22AM *  0 points [-]

Normal "introducing myself", normal I am of faith.

Comment author: PotterChrist 03 April 2017 10:29:57AM 0 points [-]

Regular, "dont think about the irony"