Less Wrong is a community blog devoted to refining the art of human rationality. Please visit our About page for more information.

New discussion platform for Less Wrong community (Mastodon instance)

5 rayalez 17 April 2017 04:00PM

Hey, everyone! I have created a new discussion platform for Less Wrong:


My goal for this project is to revitalize our community, and create a platform for rational conversations about the current events and new ideas. I'm hoping that in this format, we will have more active discussion, and get to know each other better.

The site is built using Mastodon - a new, open source, decentralized social networking platform that is currently taking off.

Comment author: Viliam 09 March 2017 10:02:53AM 2 points [-]

If this is supposed to be a description of how actual human brains work, I guess we naturally don't have any "useful metrics we want to optimize for". Instead we are driven by various impulses, which historically appeared by random mutations, and if they happened to contribute to human survival and reproduction, they were preserved and promoted by natural selection. At this moment, the impulses that sometimes make us (want to) optimize for some useful metrics are a part of that set. But they are just one among many desires, not some essential building block of the human brain.

There is some problem even with having seemingly finite goals. For example, if the machine has a probabilistic model of the world, and you ask it to make 100 paperclips, there is a potential risk -- depending on the specific architecture -- that the machine would recognize that it doesn't have literally 100% certainty of having already created 100 paperclips, and will try to optimize for making this certainty as high as possible (destroying humanity as a side effect). For example, the machine may think "maybe humans are messing with my memory and visual output to make me falsely believe that I have 100 paperclips, when in reality maybe I have none; I guess it would be safer to kill them all". So maybe the goal should instead be something like "make 100 paperclips with probability at least 99%", but... you know, the general idea is that there may be some unnoticed way how the supposedly finite goal might spawn an infinite subtask.

Otherwise... this seems like a nice high-level view of the things, but the devil is in the details. You could write thousands of scientific papers merely on how to correctly implement things like "picture of the world", "concept of a cat", etc. That is, the heavy work is hidden behind these seemingly innocent words.

Comment author: rayalez 09 March 2017 10:27:44AM *  0 points [-]

Thank you for your reply!

For a long time, the way ANNs work kinda made sense to me, and seemed to map nicely onto my (shallow) understanding of how human brain works. But I could never imagine how could the values/drives/desires be implemented in terms of ANN.

The idea that you can just quantify something you want as a metric, feed it as an input, and see if the output is closer to what we want is new to me. It was a little epiphany, that seems to make sense, so it prompted me to write this post.

Evolutionary, I guess human/animal utility function would be something like "How many copies of myself have I made? Let's maximize that." But from the subjective perspective, it's probably more like "Am I receiving the pleasure from the reward system my brain happened to develop?"

For sure there are a bunch of different impulses/drives, but they all are just little rewards for transforming the current state of the world into the one our brain prefers, right? Maybe they have appeared randomly, but if you were to design one intentionally, is that how you would go about it?


  • Get inputs from eyes/ears.
  • Recognize patterns, make predictions.
  • Compare predictions to how things turned out, update the beliefs, improve the model of the world.
  • Repeat.

General intelligence taking actions towards it's values

  • Perceive the difference between the state of world, and the state I want.
  • Use the model of the world that I've learned to predict the outcomes of possible actions.
  • If I predict that applying action to the world will lead to rewards - take action.
  • See how it turned out, update the model, repeat.

I agree that specific goals can also have unintended consequences. It just occurred to me that this kind of problem would be much easier to solve than trying to align the abstract values, and the outcome is the same - we get what we want.

Oh, and I totally agree that there's probably a ton of complexity when it comes to the implementation. But it would be pretty cool to figure out at least the general idea of what intelligence and consciousness are, what things we need to implement, and how they fit together.

How AI/AGI/Consciousness works - my layman theory

0 rayalez 09 March 2017 09:17AM

This is just my layman theory. Maybe it’s obvious to experts, probably has flaws. But it seems to make sense to me, perhaps will give you some ideas. I would love to hear your thoughts/feedback!


Consume input

The data you need from the world(like video), and useful metrics we want to optimize for, like number of paperclips in the world.


Make predictions and take action

Like deep learning does.

How do human brains convert their structure into action?

Maybe like:

- Take the current picture of the world as an input.

- Come up with random action.

- “Imagine” what will happen.

Take the current world + action, and run it through the ANN. Predict the outcome of the action applied to the world.

- Does the output increase the metrics we want? If yes — send out the signals to take action. If no — come up with another random action and repeat.


Update beliefs

Look at the outcome of the action. Does the picture of the world correspond to the picture we’ve imagined? Did this action increase the good metrics? Did the number of paperclips in the world increase? If it did — positive reinforcement. Backpropagation, and reinforce the weights.



Take current picture of the world=> Imagine applying an action to it => Take action => Positive/Negative reinforcement to improve our model => Repeat until the metrics we want equal to the goal we have set.




Consciousness is neurons observing/recognizing patterns of other neurons.

When you see the word “cat”— photons from the page come to your retina and are converted to neural signal. A network of cells recognizes the shape of letters C, A, and T. And then a higher level, more abstract network recognizes that these letters together form the concept of a cat.

You can also recognize signals coming from the nerve cells within your body, like feeling a pain when stabbing a toe.

The same way, neurons in the brain recognize the signals coming from the other neurons within the brain. So the brain “observes/feels/experiences” itself. Builds a model of itself, just like it builds a map of the world around, “mirrors” itself(GEB).


Sentient and self-improving

So the structure of the network itself is fed as one of it’s inputs, along with the video and metrics we want to optimize for. It can see itself as a part of the state of the world it bases predictions on. That’s what being sentient means.

And then one of the possible actions it can take is to modify it’s own structure. “Imagine” modifyng the structure a certain way, if you predict that it leads to the better predictions/outcomes —modify it. If it did lead to more paperclips — reinforce the weights to do more of that. So it keeps continually self improving.



We don’t want this to lead to the infinite amount of paperclips, and we don’t know how to quantify the things we value as humans. We can’t turn the “amount of happiness” in the world into a concrete metrics without the unintended consequences(like all human brains being hooked up to wires that stimulate our pleasure centers).

That’s why instead of trying to encode the abstract values to maximize for, we encode very specific goals.

- Make 100 paperclips (utility function is “Did I make 100 paperclips?”)

- Build 1000 cars

- Write a paper on how to cure cancer

Humans remain in charge, determine the goals we want, and let AI figure out how to accomplish them. Still could go wrong, but less likely.

(originally published on my main blog)

Comment author: Alexandros 27 November 2016 10:40:52AM *  66 points [-]

Hi Anna,

Please consider a few gremlins that are weighing down LW currently:

  1. Eliezer's ghost -- He set the culture of the place, his posts are central material, has punctuated its existence with his explosions (and refusal to apologise), and then, upped and left the community, without actually acknowledging that his experiment (well kept gardens etc) has failed. As far as I know he is still the "owner" of this website, retains ultimate veto on a bunch of stuff, etc. If that has changed, there is no clarity on who the owner is (I see three logos on the top banner, is it them?), who the moderators are, who is working on it in general. I know tricycle are helping with development, but a part-time team is only marginally better than no-team, and at least no-team is an invitation for a team to step up.

  2. the no politics rule (related to #1) -- We claim to have some of the sharpest thinkers in the world, but for some reason shun discussing politics. Too difficult, we're told. A mindkiller! This cost us Yvain/Scott who cited it as one of his reasons for starting slatestarcodex, which now dwarfs LW. Oddly enough I recently saw it linked from the front page of realclearpolitics.com, which means that not only has discussing politics not harmed SSC, it may actually be drawing in people who care about genuine insights in this extremely complex space that is of very high interest.

  3. the "original content"/central hub approach (related to #1) -- This should have been an aggregator since day 1. Instead it was built as a "community blog". In other words, people had to host their stuff here or not have it discussed here at all. This cost us Robin Hanson on day 1, which should have been a pretty big warning sign.

  4. The codebase, this website carries tons of complexity related to the reddit codebase. Weird rules about responding to downvoted comments have been implemented in there, nobody can make heads or tails with it. Use something modern, and make it easy to contribute to. (telescope seems decent these days).

  5. Brand rust. Lesswrong is now kinda like myspace or yahoo. It used to be cool, but once a brand takes a turn for the worse, it's really hard to turn around. People have painful associations with it (basilisk!) It needs burning of ships, clear focus on the future, and as much support as possible from as many interested parties, but only to the extent that they don't dillute the focus.

In the spirit of the above, I consider Alexei's hints that Arbital is "working on something" to be a really bad idea, though I recognise the good intention. Efforts like this need critical mass and clarity, and diffusing yet another wave of people wanting to do something about LW with vague promises of something nice in the future (that still suffers from problem #1 AFAICT) is exactly what I would do if I wanted to maintain the status quo for a few more years.

Any serious attempt at revitalising lesswrong.com should focus on defining ownership and plan clearly. A post by EY himself recognising that his vision for lw 1.0 failed and passing the batton to a generally-accepted BDFL would be nice, but i'm not holding my breath. Further, I am fairly certain that LW as a community blog is bound to fail. Strong writers enjoy their independence. LW as an aggregator-first (with perhaps ability to host content if people wish to, like hn) is fine. HN may have degraded over time, but much less so than LW, and we should be able to improve on their pattern.

I think if you want to unify the community, what needs to be done is the creation of a hn-style aggregator, with a clear, accepted, willing, opinionated, involved BDFL, input from the prominent writers in the community (scott, robin, eliezer, nick bostrom, others), and for the current lesswrong.com to be archived in favour of that new aggregator. But even if it's something else, it will not succeed without the three basic ingredients: clear ownership, dedicated leadership, and as broad support as possible to a simple, well-articulated vision. Lesswrong tried to be too many things with too little in the way of backing.

Comment author: rayalez 27 November 2016 10:08:19PM *  9 points [-]

I am working on a project with this purpose, and I think you will find it interesting:


It is intended to be a community for intelligent discussion about rationality and related subjects. It is still a beta version, and has not launched yet, but after seeing this topic, I have decided to share it with you now.

It is based on the open source platform that I'm building:


This platform will address most of the issues discussed in this thread. It can be used both like a publishing/discussion platform, and as a link aggregator, because it supports both twitter-like discussion, reddit-like communities, and medium-like long form articles.

This platform is in active development, and I'm very interested in your feedback. If LessWrong community needs any specific functionality that is not implemented yet - I will be happy to add it. Let me know what you think!

Comment author: rayalez 27 November 2016 09:59:36PM *  0 points [-]

I am working on a project with the similar purpose, and I think you will find it interesting:


It is intended to be a community for intelligent discussion about rationality and related subjects. It is still a beta version, and has not launched yet, but after seeing this topic, I have decided to share it with you now.

If you find it interesting and can offer some feedback - I would really appreciate it!

Comment author: rayalez 15 June 2016 12:20:49AM *  1 point [-]

Hey, everyone! Author of rationalfiction.io here.

I am actively building and improving our website, and I would be happy to offer it as a new platform for LW community, if there's interest.

I can take care of the hosting, and build all the necessary features.

I've been thinking about creating a LW-like website for a while now, but I wasn't sure that it will work. After reading this post I have decided that I'm going launch and see where it goes.

If there's any ideas or suggestions about how such platform can be improved or what features we'll need - let's discuss them.

By the way, the platform is open source(though I will probably fork it as a separate project and develop it in a new repo).

Comment author: root 31 May 2016 12:54:15PM *  0 points [-]

Not going to use it but:

  1. Good job on not having a javascript hell

  2. Some people might like a mobile view (if there isn't one already)

  3. No RSS feeds?

Comment author: rayalez 31 May 2016 01:23:46PM *  0 points [-]
  1. Thanks!

  2. It works well on my iPad, haven't tested it on the phones yet. I will.

  3. There are links to author's RSS feed in the post footer and on the profile pages.

Is there a reason you don't want to use the site? I'd appreciate any feedback or ideas on how I can make it better.

rationalfiction.io - publish, discover, and discuss rational fiction

7 rayalez 31 May 2016 12:02PM

Hey, everyone! I want to share with you a project I've been working on for a while - http://rationalfiction.io.

I want it to become the perfect place to publish, discover, and discuss rational fiction.

We already have a lot of awesome stories, and I invite you to join and post more! =)

Comment author: RowanE 11 April 2016 01:38:22PM 0 points [-]

Guy who doesn't know much about startups here - "launched the first version" and "want [it] to become" sound indicative of something more "outline of a novel" - can you elaborate on how big of an accomplishment it was to get it off the ground in the first place?

Comment author: rayalez 12 April 2016 06:09:49PM *  1 point [-]

In startups, it is so called "MVP" - minimal viable product, a simplest version that you can show users to get some feedback and see if it works. It is the first step to building a startup.

To me it's a pretty huge accomplishment, I'm really proud of myself =) Most of the work went not into coding the website, but into figuring out what it is. I needed a thing that would be valuable, and that I would be excited to work on for the following few years.

A competent programmer could probably create something like that in a week, but because I'm just learning web development(along with writing, producing videos, and other stuff) it took me longer. At the moment it's the best thing I've created, so I'm really happy about it.

Also it's actually the 3rd iteration of my startup idea(first one was a platform for publishing fiction, 2nd - platform for publishing webcomics.)

Comment author: rayalez 08 April 2016 08:48:12PM *  7 points [-]

I've launched the first version of my startup, lumiverse:


I want lumiverse to become the perfect place for people to publish, discover and discuss great educational videos. I want to build a friendly and intelligent community, make it easy for video creators to find an audience, and make it easy for viewers to discover awesome videos.

I also have finaly made the first few episodes of Orange Mind - my video series about rationality.

View more: Next