Less Wrong is a community blog devoted to refining the art of human rationality. Please visit our About page for more information.

Comment author: JohnEPaton 30 July 2012 05:34:22AM 0 points [-]

If a belief turns deadbeat, evict it.

This might be challenging because our beliefs tend to shape the world we live in thus masking their error. Does anyone have any practical tips for discovering erroneous beliefs?

Comment author: Risto_Saarelma 30 July 2012 03:31:59AM 1 point [-]

Yes you could probably use some machine learning algorithm to build a brain with the input of a video feed. But this says relatively little about how the brain actually develops in nature.

That's just the thing. It makes a big difference whether we're talking about a (not necessarily human) brain in general, or a specific, particular brain. Artificial intelligence research is concerned about being able to find any brain design it can understand and work with, while neuroscience is concerned with the particulars of human brain anatomy and often the specific brains of specific people.

Also, I'd be kinda hesitant to dismiss anything that involves being able to build a brain as "saying relatively little" about anything brain-related.

Comment author: JohnEPaton 30 July 2012 04:00:43AM 1 point [-]

Thanks for the clarification. You're right that artificial intelligence and neuroscience are two different fields.

Comment author: JohnEPaton 30 July 2012 03:07:49AM 0 points [-]

I'm just wondering whether it's true that the Markov property holds for minds. I'm thinking that a snapshot of the world is not enough, but you also need to know something about the rate at which the world is changing. Presumably this information would require the knowledge of states further back.

Also, isn't there an innate element of randomness when it comes to decision making and how our minds work. Neurons are so small that presumably there are some sort of quantum effects, and wouldn't this mean again that information from one step previous wasn't enough.

I don't know, but just some thoughts.

Comment author: JohnEPaton 30 July 2012 02:55:39AM 0 points [-]

What is the tradeoff between average utility and total utility? Presumably a world with only ten people who all have tremendous utility would be just as repugnant as Parfit's world.

Comment author: Risto_Saarelma 24 July 2012 07:53:52AM *  7 points [-]

The argument from the small size of the genome is more plausible, especially if Eliezer is thinking in terms of Kolmogorov complexity, which is based on the size of the smallest computer program needed to build something. However, it does not follow that if the genome is not very complex, the brain must not be very complex, because the brain may be built not just based on the genome, but also based on information from the outside environment.

I think the question in the original debate could be formulated as something like: How big a solution, in the amount of program code we need to write, do we need to find to be able to implement a self-improving artificial intelligence that will, given an amount of sensory input and opportunities to interact with its environment comparable to that of a human growing up, grow up to human-level cognition.

I don't see how the other sources of information needed for brain development is a counterargument here. Once you have a machine learning system that will bootstrap itself to sentience given a few years of video feed, you've done pretty well indeed.

I don't also see how the compressibility argument is supposed to work without further qualifiers. You might drop the size of Windows Vista to a third or less if you compressed aggressively at every possible opportunity, but would you get out of the order-of-magnitude ballpark which is about the level of detail the argument operates on?

(I wanted to say here that you'd never get Vista to fit on a 360 kB floppy disk, but on second thought that would also require something like "with any kind of sane human engineering effort you could only spend a millennium or so with". If you have an arbitrary amount of cleverness for composing the program and an arbitrary amount of time running it, you get into Busy Beaver Land, and a floppy disk suddenly has the potential to be an extremely scary artifact. On the other hand, compared to eldritch computational horrors from Busy Beaver Land the human genome has had a very limited amount of evolutionary design going in it, and has a very limited time to execute itself.

(Well you would probably run into the pigeonhole principle if you wanted to get the actual Vista Vista out of the ultra-compressed thing, but for the purposes of the argument, it's enough to get a something that does all the sort of things Windows Vista does, no matter what the exact bit pattern.))

Comment author: JohnEPaton 30 July 2012 02:49:16AM 0 points [-]

I think he's saying that the brain is not just the genome. What you see as an adult brain also represents a host of environmental factors. Since these environmental factors are complex, so then is the brain.

Yes you could probably use some machine learning algorithm to build a brain with the input of a video feed. But this says relatively little about how the brain actually develops in nature.

Comment author: JohnEPaton 30 July 2012 02:40:07AM 0 points [-]

You make a good point that the genome does not completely determine how the brain is set up. Environment is hugely influential in how things develop. I recently read that things the expression of our genes can be influence by things called transcription factors, as well as process called slicing and transposition. Each of these things is effected by the environment. For example, if your a small rat pup and your Mom licks you then this will trigger a cascade of hormones that will end up changing your DNA and your amygdala so that you release less stress hormone as an adult. Indeed, when you pass on your genes you will also produce kids that are less prone to stress.

Since environment is so important in development this exponentially increases the complexity of the brain. You don't only need to know the genome, but you also need to know all the relevant environmental factors from the moment of conception. There simply is no blueprint for the brain and any attempt to decode the genome is erroneous.

In response to Chaotic Inversion
Comment author: JohnEPaton 30 July 2012 02:01:33AM 0 points [-]

I'm not sure if I completely get this post. Are you saying that the amount of time when you cannot work is chaotic or not? If I get it correctly, you're saying that when you feel like you cannot work you can often look at various things that are going on (such as when you last had a meal, what time it is, whether you recently exercised etc...) and then use these factors to add an element of predictability into your down time. Is this right?

Comment author: lisa 07 February 2012 10:17:24PM *  17 points [-]


I'm a 20 year old student at Georgia Tech, double majoring in Industrial Engineering and Psychology, and am spending the current semester studying abroad at the University of Leeds in the UK.

I read HPMOR this weekend on a bus trip to London and as soon as I returned I found this site and have been enthralled by the Sequences, which I am slowly working my way through.

All of my life I have loved to read and learn new things and think through them, but last year I came to the realization that my curiosity had started to die in my late high school years. I found myself caring about getting a good grade and then abruptly forgetting the information. Much of what I was "learning" I never truly understood and yet I was still getting praise from teachers for my good grades, so I saw no reason to invest more effort. Early last year, I realized that this was happening and attempt to rededicate myself to finding things that again made me passionate about learning. This was a major contribution to adding Psychology as a second major.

This semester of new classes in a new educational system combined with the past few days of reading the Sequences have sparked my interest in many subjects. I'm itching to go to the school library and start picking up anything that catches my interest now that the the thirst to learn has been reawakened. I'm especially interested in Evolutionary Psychology, Social Psychology, and Statistics. I have absolutely no idea what I would like to do as a future career, but have this reoccurring thought that I would love to do some sort of work which involved restructuring the education system. (Every person at my University that I have mentioned that thought to gives me a strange look and says either "Education? You???' or "But then you wouldn't make any money!')

Anyways, I am extremely glad to have found this site and community.

Comment author: JohnEPaton 30 July 2012 01:49:55AM 0 points [-]

That's cool that your studying a combination of Psychology and Engineering. I'm doing something similar and it seems to be very rare to find someone who is working in both of those fields. I'm sure that in the UK people would be even less understanding of this. It seems like over there you just choose one subject and that's all you do for the next three years. Keep on looking at those library books. I think the most important thing as an undergrad is to follow your interests even if this means dialling back on the effort you put into class work.

Comment author: JohnEPaton 30 July 2012 01:44:38AM 7 points [-]


My name is John Paton. I'm an Operations Research and Psychology major at Cornell University. I'm very interested in learning about how to improve the quality of my thinking.

Honestly, I think that a lot of my thoughts about how the world works are muddled at the moment. Perhaps this is normal and will never go away, but I want to at least try and decrease it.

At first glance, this community looks awesome! The thinking seems very high quality, and I certainly want to contribute to the discussion here.

I also write at my own blog, optimizethyself.com

See you in the discussion!


Comment author: JohnEPaton 30 July 2012 01:37:29AM 1 point [-]

I'm twenty and I feel like I missed the space conquest spirit completely. When I think about the scientific interests that appeal to my generation, I think that advances in health, computing, and sustainable energy seem far more important. I believe that we haven't really lost our need for conquest it's just moved into domain closer to home. The question now is, how can we improve life on earth? How can we extend life? How can we collect and analyze data in better ways than before?

I think that the decline of violence in the world may have something to do with this. Since there is less violence between nation states, there is less of an incentive for countries to show their dominance on an international level. This is just a guess though.

View more: Next