• I self diagnosed excoriation as an addiction. I've stopped licking or biting my lips under all circumstances and have started applying a cream at night to both my lips and my finger tips. Although, I still haven't figured out a solution to supplement my willpower in scratching the skin inside my nose :( I'm scared I'll end up like those people who snort lots of cocaine and lose the gap between their nostrils.

• Discovered that trigger warnings are keeping me from overcoming hyper-vigilence (actually diagnosed for this one!), not protecting me!

I made serious progress on a system for generating avatar animations based on the motion of a VR headset. It still needs refinement, but I'm extremely proud of what I've got so far.

https://www.youtube.com/watch?v=klAsxamqkkU

For Omnivores:

• Do you think the level of meat consumption in America is healthy for individuals? Do you think it's healthy for the planet?

Meat is obviously healthy for individuals. We evolved to eat as much of it as we could get. Many nutrients seem to be very difficult to obtain in sufficient, bio-available form from an all-vegetable diet. I just suspect most observant vegans are substantially malnourished.

On the planet side of things, meat is an environmental disaster. The methane emissions are horrifying, as is the destruction of rainforest. Hopefully, lab-grown meat allows us to switch to an eco-friendly alternative.

• How do you feel about factory farming? Would you pay twice as much money for meat raised in a less efficient (but "more natural") way?

Factory farming is necessary to continue to feed the world. I don't care about "natural", but I'd pay extra for food from animals that had been genetically engineered to be happy and extremely stupid/near-comatose, to reduce total suffering-per-calorie. This would be more effective and less costly than switching to free-range.

• Are there any animals you would (without significantly changing your mind) never say it was okay to hunt/farm and eat? If so, what distinguishes these animals from the animals which are currently being hunted/farmed?

Great apes. cetaceans, and a few birds. The range of animal intelligence is extremely broad. I find it extremely unlikely that chickens have anything recognizable as a human-like perception of the world. I think the odds are better than not that dolphins, chimps, and parrots do.

If you're interested, the animal I'm most on the fence about is pigs.

• If all your friends were vegetarians, and you had to go out of your way to find meat in a similar way to how vegans must go out of their way right now, do you think you'd still be an omnivore?

Yes. I cook most of my own meals, and my meat consumption would continue even in the absence of social eating.

For Vegetarians:

• If there was a way to grow meat in a lab that was indistinguishable from normal meat, and the lab-meat had never been connected to a brain, do you expect you would eat it? Why/why not?

I obviously no moral problem with that. That would be fantastic. However, I probably wouldn't eat the lab meat. I find the texture / mouth-feel of most meat pretty gross, and lab-grown meat would be significantly more expensive than my current diet. Since microbiome acclimation means that resuming eating meat could make me very sick for a while, I'm not sure I see the profit in it.

I am very interested in synthetic milk, cheese, and eggs, however.

• Indigenous hunter gatherers across the world get around 30 percent of their annual calories from meat. Chimpanzees, our closest non-human relatives, eat meat. There are arguments that humans evolved to eat meat and that it's natural to do so. Would you disagree? Elaborate.

Obviously, humans evolved to be omnivorous. However, the paleo people are lunatics if they think we ate as much meat as they do (much less of the hyper-fatty livestock we've bred over the last couple of millenia). Meat was a most likely a rare supplement to the largely-vegetarian diets of ancestral peoples.

Regardless, none of this is the point. Today, it's perfectly possible to eat a vegan diet and be healthy (see: Soylent). You can't avoid the obvious moral horror of eating the flesh of semi-sentient animals like pigs by shouting the word 'natural' and running away.

• Do you think it's any of your business what other people eat? Have you ever tried (more than just suggesting it or leading by example) to get someone to become a vegetarian or vegan?

Only if they bring it up first. I do think we have a moral obligation to try to reduce animal suffering, but harassing my friends isn't actually helping the cause in any way, and might be hurting. I do try to corrupt my meat-eating friends who are having seconds thoughts about it, but, you know, in a friendly way.

• What do you think is the primary health risk of eating meat (if any)?

Parasites probably. Meat in moderation clearly isn't especially bad for you. It's just, you know, wrong.

What is the largest living thing that has been successfully pulled out of cryo? A single cell? Disconnected tissue? An organ?

I'd find a single cell not terribly convincing, a working organ much more so.

(context: I haven't signed up and don't currently intend to, but it's mentally marked as "keep an eye on this, additional information may change my mind.")

The issue is that crashing the mosquito population doesn't work if even a few of them survive to repopulate - the plan needs indefinite maintenance, and the mosquitoes will eventually evolve to avoid our lab-bred dud males.

I wonder if you could breed a version of the mosquito that's healthy but has an aversion to humans, make your genetic change dominant, and then release a bunch of THOSE mosquitoes. There'd be less of a fitness gap between the modified mosquitoes and the original species, so if we just kept dumping modified males every year for a decade or two, we might be able to completely drive the original human-seeking genes out of the ecosystem.

I'd strongly suggest the movie project nim, if you haven't seen it. In some respects chimpanzee intelligence develops faster than that of a human child, but it also planes off much earlier. Their childhood development period is much shorter.

To first approximation, general intelligence in animals can be predicted by number of neurons/synapses in general learning modules, but this isn't the only factor. I don't have an exact figure, but that poster article suggests parrots have perhaps 1-3 billion ish cortical neuron equivalent.

The next most important factor is probably degree of neotany or learning window. Human intelligence develops over the span of 20 years. Parrots seem exceptional in terms of lifespan and are thus perhaps more human like - where they maintain a childlike state for much longer. We know from machine learning that the 'learning rate' is a super important hyperparameter - learning faster has a huge advantage, but if you learn too fast you get inferior long term results for your capacity. Learning slowly is obviously more costly, but it can generate more efficient circuits in the long term.

I inferred/guessed that parrots have very long neotenic learning windows, and the articles on Alex seem to confirm this.

Alex reached a vocabulary of about 100 words by age 29, a few year's before his untimely death. The trainer - Irene Pepperberg - claims that Alex was still learning and had not reached peak capability. She rated Alex's intelligence as roughly equivalent to that of a 5 year old. This about makes sense if the parrot has roughly 1/6th our number of cortical neurons, but has similar learning efficiency and long learning window.

To really compare chimp vs parrot learning ability, we'd need more than a handful of samples. There is also a large selection effect here - because parrots make reasonably good pets, whereas chimps are terrible dangerous pets. So we haven't tested chimps as much. Alex is more likely to be a very bright parrot, whereas the handful of chimps we have tested are more likely to be average.

It also does not explain why birds are better at language tasks than cats. Cat brains are much larger. The training rewards in the lab are the same. And, yet, cats significantly underperform parrots at every single language-related task we can come up with. Why? Because the parrots have had a greater evolutionary pressure to be good at language-style tasks - and, as a result, they have evolved task-specific neurological algorithms to make it easier.

Cat brains are much larger, but physical size is irrelevant. What matters is neuron/synapse count.

According to my ULM theory - the most likely explanation for the superior learning ability of parrots is a larger number of neurons/synapses in their general learning modules - (whatever the equivalent of the cortex is in birds) and thus more computational power available for general learning.

Stop right now, and consider this bet - I will bet that parrots have more neurons/synapses in their cortex-equivalent brain regions than cats.

Now a little google searching leads to this blog article which summarizes this recent research - Complex brains for complex cognition - neuronal scaling rules for bird brains,

From the abstract:

We show that in parrots and songbirds the total brain mass as well as telencephalic mass scales approximately linearly with the total number of neurons, i.e. neuronal density does not change significantly as brains get larger. The neuronal densities in the telencephalon exceed those observed in the cerebral cortex of primates by a factor of 2-8. As a result, the numbers of telencephalic neurons in the brains of the largest birds examined (raven, kea and macaw) equal or exceed those observed in the cerebral cortex of many species of monkeys.

Finally, our findings of comparable numbers of neurons in the cerebral cortex of medium-sized primates and in the telencephalon of large parrots and songbirds (particularly corvids) strongly suggest that large numbers of forebrain neurons, and hence a large computational capacity, underpin the behavioral and cognitive complexity reported for parrots and songbirds, despite their small brain size.

The telencephalon is believed to be the equivalent of the cortex in birds. The cortex of the smallest monkeys have about 400 million neurons, whereas the cat's cortex has about 300 million neurons. A medium sized monkey such as a night monkey has more than 1 billion cortical neurons.

The deepmind's team's solution to this is the neural Turing machine model, which is a hybrid system between a neural network and a database. It's not a pure ANN.

Yes it is a pure ANN - according to my use of the term ANN (arguing over definitions is a waste of time). ANNs are fully general circuit models, which obviously can re-implement any module from any computer - memory, database, whatever. The defining characteristics of an ANN are - simulated network circuit structure based on analog/real valued nodes, and some universal learning algorithm over the weights - such as SGD.

Your position is not as mainstream as you like to present it.

You don't understand my position. I don't believe DL as it exists today is somehow the grail of AI. And yes I'm familiar with Hinton's 'Capsule' proposals. And yes I agree there is still substantial room for improvement in ANN microarchitecture, and especially for learning invariances - and unsupervised especially.

This is far from the mainstream linguistic perspective.

For any theory of anything the brain does - if it isn't grounded in computational neuroscience data, it is probably wrong - mainstream or not.

No, it really isn't. I don't update based on forum posts on topics I don't understand, because I have no way to distinguish experts from crackpots.

You don't update on forum posts? Really? You seem pretty familiar with MIRI and LW positions. So are you saying that you arrived at those positions all on your own somehow? Then you just showed up here, thankfully finding other people who just happened to have arrived at all the same ideas?

Yes, I've read your big universal learner post, and I'm not convinced.

Do you actually believe that evolved modularity is a better explanation of the brain then the ULM hypothesis? Do you have evidence for this belief or is it simply that which you want to be true? Do you understand why the computational neuroscience and machine learning folks are moving away from the latter towards the former? If you do have evidence please provide it in a critique in the comments for that post where I will respond.

First off, you're seriously misrepresenting the success of deep learning as support for your thesis. Deep learning algorithms are extremely powerful, and probably have a role to play in building AGI, but they aren't the end-all, be-all of AI research.

Make some specific predictions for the next 5 years about deep learning or ANNs. Let us see if we actually have significant differences of opinion. If so I expect to dominate you in any prediction market or bets concerning the near term future of AI.

First off the bat, you absolutely can create an AGI that is a pure ANN. In fact the most successful early precursor AGI we have - the atari deepmind agent - is a pure ANN. Your claim that ANNs/Deep Learning is not the end of all AGI research is quickly becoming a minority position.

Humans can learn echolocation, but they can't learn echolocation the way bats and dolphins can learn echolocation

What the scottsman!

The real test here would be to take a brain and give it an entirely new sense

Done and done. Next!

Notably, the general learner hypothesis does not explain why non-surgically-modified brains are so standardized in structure and functional layout. Something that you yourself bring up in your article.

I discussed this in the comments - it absolutely does explain neurotypical standardization. It's a result of topographic/geometric wiring optimization. There is an exactly optimal location for every piece of functionality, and the brain tends to find those same optimal locations in each human. But if you significantly perturb the input sense or the brain geometry, you can get radically different results.

Consider the case of extreme hydrocephaly - where fluid fills in the center of the brain and replaces most of the brain and squeezes the remainder out to a thin surface near the skull. And yet, these patients can have above average IQs. Optimal dynamic wiring can explain this - the brain is constantly doing global optimization across the wiring structure, adapting to even extreme deformations and damage. How does evolved modularity explain this?

It also obviously has hard-coded specialized modules, to some degree, which is why (for example) all human cultures develop language and music, which isn't something you'd expect if we were all starting from zero.

This is nonsense - language processing develops in general purpose cortical modules, there is no specific language circuitry.

There is a small amount of innate circuit structures - mainly in the brainstem, which can generate innate algorithms especially for walking behavior.

The question is which aspect dominates brain performance.

This is rather obvious - it depends on the ratio of pure learning structures (cortex, hippocampus, cerebellum) to innate circuit structures (brainstem, some midbrain, etc). In humans 95% or more of the circuitry is general purpose learning.

What about Watson?

Not an AGI.

Finally, I don't have the background to refute your argument on the efficiency of the brain (although I know clever people who do who disagree with you).

The correct thing to do here is update. Instead you are searching for ways in which you can ignore the evidence.

But, taking it as a given that you're right, it sounds like you're assuming all future AIs will draw the same amount of power as a real brain and fit in the same spatial footprint.

Obviously not - in theory given a power budget you can split it up into N AGIs or one big AGI. In practice due to parallel scaling limitations, there is always some optimal N. Even on a single GPU today, you need N about 100 or more to get good performance.

You can't just invest all your energy into one big AGI and expect better performance - that is a mind numbingly naive strategy.

To sum up: yes, I've read your thing. No, it's not as convincing as you seem to believe.

Update, or provide counter evidence, or stop wasting my time.

So, to sum up, your plan is to create an arbitrarily safe VM, and use it to run brain-emulation-style denovo AIs

No. I said:

Stop thinking of AGI as some wierd mathy program. Instead think of brain emulations - and then you have obvious answers to all of these questions.

I used brain emulations as analogy to help aid your understanding. Because unless you have deep knowledge of machine learning and computational neuroscience, there are huge inferential distances to cross.

Human beings are not just general optimizers.

Yes we are. I have made a detailed, extensive, citation-full, and well reviewed case that human minds are just that.

All of our understanding about the future of AGI is based ultimately on our models of the brain and AI in general. I am claiming that the MIRI viewpoint is based on an outdated model of the brain, and a poor understanding of the limits of computation and intelligence.

I will summarize for one last time. I will then no longer repeat myself because it is not worthy of my time - any time spent arguing this is better spent preparing another detailed article, rather than a little comment.

There is extensive uncertainty concerning how the brain works and what types of future AI are possible in practice. In situations of such uncertainty, any good sane probabilistic reasoning agent should come up with a multimodal distribution that spreads belief across several major clusters. If your understanding of AI comes mainly from reading LW - you are probably biased beyond hope. I'm sorry, but this is true. You are stuck in box and don't even know it.

Here are the main key questions that lead to different belief clusters:

• Are the brain's algorithms for intelligence complex or simple?
• And related - are human minds mainly software or mainly hardware?
• At the practical computational level, does the brain implement said algorithms efficiently or not?

If the human mind is built out of a complex mess of hardware specific circuits, and the brain is far from efficient, than there is little to learn from the brain. This is Yudkowsky/MIRI's position. This viewpoint leads to a focus on pure math and avoidance of anything brain-like (such as neural nets). In this viewpoint hard takeoff is likely, AI is predicted to be nothing like human minds, etc.

If you believe that the human is complex and messy hardware, but the brain is efficient, than you get Hanson's viewpoint where the future is dominated by brain emulations. The brain ems win over brain inspired AI because scanning real brain circuitry is easier than figuring out how it works.

Now what if the brain's algorithms are not complex, and the brain is efficient? Then you get my viewpoint cluster.

These questions are empirical - and they can be answered today. In fact, I realized all this years ago and spent a huge amount of time learning more about the future of computer hardware, the limits of computation, machine learning, and computational neuroscience.

Yudkowsky, Hanson, and to some extent Bostrom - were all heavily inspired by the highly influential evolved modularity hypothesis in ev psych from Tooby and Cosmides. In this viewpoint, the brain is complex, and most of our algorithmic content is hardware based rather than software. I have argued that this viewpoint has been tested empirically and now disproven. The brain is built out of relatively simple universal learning algorithms. It will essentially be almost impossible to build practical AGI that is very different from the brain (remember, AGI is defined as software which can do everything the brain does).

Bostrom/Yudkowksky have also argued that the brain is very far from efficient. For example, from true sources of disagreement:

Human neurons run at less than a millionth the speed of transistors, transmit spikes at less than a millionth the speed of light, and dissipate around a million times the heat per synaptic operation as the thermodynamic minimum for a one-bit operation at room temperature. Physically speaking, it ought to be possible to run a brain at a million times the speed without shrinking it, cooling it, or invoking reversible computing or quantum computing.

The first two statements are true, the third statement is problematic, and the thrust of the conclusion is incorrect. The minimum realistic energy for a brain-like circuit is probably close to what the brain actually uses:

• the landauer bound depends on speed and reliability. The 10^-21 J/bit bound only applies to a signal of infinitely low frequency. For realistic fast reliable signals, the bound is 100 times higher: around 10^-19 J/bit.
• the landauer bound applies to single 1 bit ops. The fundamental bound for a 32 bit flop is around 10^5 or 10^6 times higher. Moore's Law is ending and we are actually close to these bounds already. Synapses perform analog ops which have lower cost than a 32 bit flop, but still a much higher cost than a single bit op.
• most of the energy consumption in any advanced computer comes from wire dissipation, not switch dissipation. Signaling in the brain uses roughly 0.5x10^-14 J/bit/mm (5 fJ/bit/mm) 2, which appears to be within an order of mag or two of optimal, and is perhaps one order of magnitude more efficient than current computers. Wire signal energy in computers is not improving significantly. For example, for 40nm tech in 2010, the wire energy is 240 fj/bit/mm, and is predicted to be around 150 to 115 by 2017 3. The practical limit is perhaps around 1 fJ/bit/mm, but that would probably require much lower speeds.

These errors add up to around 6 orders of magnitude or so. The brain is near the limits of energy efficiency for what it does in terms of irreversible computation. No practical machine we will ever build in the near future is going to be many orders of magnitude more efficient than the brain. Yes, eventually reversible and quantum computing could perhaps result in large improvements, but those technologies are far and will come long after neuromorphic AGI.