Here are some relevant blockquotes of Bostrom's reasoning on brain-computer interfaces, from Superintelligence chapter 2:

It is sometimes proposed that direct brain–computer interfaces, particularly implants, could enable humans to exploit the fortes of digital computing—perfect recall, speedy and accurate arithmetic calculation, and high-bandwidth data transmission—enabling the resulting hybrid system to radically outperform the unaugmented brain.64 But although the possibility of direct connections between human brains and computers has been demonstrated, it seems unlikely that such interfaces will be widely used as enhancements any time soon.65

To begin with, there are significant risks of medical complications—including infections, electrode displacement, hemorrhage, and cognitive decline—when implanting electrodes in the brain. ... One study of Parkinson patients who had received deep brain implants showed reductions in verbal fluency, selective attention, color naming, and verbal memory compared with controls. Treated subjects also reported more cognitive complaints.66 Such risks and side effects might be tolerable if the procedure is used to alleviate severe disability. But in order for healthy subjects to volunteer themselves for neurosurgery, there would have to be some very substantial enhancement of normal functionality to be gained.

Futhermore:

enhancement is likely to be far more difficult than therapy. Patients who suffer from paralysis might benefit from an implant that replaces their severed nerves or activates spinal motion pattern generators.67 Patients who are deaf or blind might benefit from artificial cochleae and retinas.68 Patients with Parkinson’s disease or chronic pain might benefit from deep brain stimulation that excites or inhibits activity in a particular area of the brain.69 What seems far more difficult to achieve is a high-bandwidth direct interaction between brain and computer to provide substantial increases in intelligence of a form that could not be more readily attained by other means. Most of the potential benefits that brain implants could provide in healthy subjects could be obtained at far less risk, expense, and inconvenience by using our regular motor and sensory organs to interact with computers located outside of our bodies. We do not need to plug a fiber optic cable into our brains in order to access the Internet.

Not only can the human retina transmit data at an impressive rate of nearly 10 million bits per second, but it comes pre-packaged with a massive amount of dedicated wetware, the visual cortex, that is highly adapted to extracting meaning from this information torrent and to interfacing with other brain areas for further processing.70 Even if there were an easy way of pumping more information into our brains, the extra data inflow would do little to increase the rate at which we think and learn unless all the neural machinery necessary for making sense of the data were similarly upgraded. Since this includes almost all of the brain, what would really be needed is a “whole brain prosthesis–—which is just another way of saying artificial general intelligence. Yet if one had a human-level AI, one could dispense with neurosurgery: a computer might as well have a metal casing as one of bone. So this limiting case just takes us back to the AI path, which we have already examined.

I'm confused by the framing of the Anvil problem. For humans, a lot of learning is learning from observing others, seeing their mistakes and their consequences. We can predict various events that will result in other's deaths based on previous observation of what happened to yet other people. If we're above a certain level of solipsism, we can extrapolate to ourselves.

Does the AIXI not have the ability to observe other agents? Is it correct to be a solipsist? Seems like a tough learning environment if you have to discover all consequences yourself.

It's still possible to extrapolate from stubbing your toe, burning your fingers on the stove, and mashing your thumb with a hammer. Is there some reason to expect that AIXI will start out its interactions with the world by picking up an anvil rather than playing with rocks and eggs?

I don't answer survey questions that ask about race, but if you met me you'd think of me as white male.

I'm more strongly libertarian (but less party affiliated) than the survey allowed me to express.

I have reasonably strong views about morality, but had to look up the terms "Deontology", "Consequentialism", and "Value Ethics" in order to decide that of these "consequentialism" probably matches my views better than the others.

Probabilities: 50,30,20,5,0,0,0,10,2,1,20,95.

On "What is the probability that significant global warming is occurring or will soon occur, and is primarily caused by human actions?", I had to parse several words very carefully, and ended up deciding to read "significant" as "measureable" rather than "consequential". For consequential, I would have given a smaller value.

I answered all the way to the end of the super bonus questions, and cooperated on the prize question.

I've always found interrupt mode incredibly frustrating to deal with, because it takes me longer to come up with appropriate responses than most people's Maximum Wait Time, and by the time a monologue is finished, whatever I thought of is no longer on-topic. I do find it interesting that you associate interruption with nerddom, though. As a tech geek, interruption is a Very Bad Thing because it disrupts mental state. If I want someone's attention, I stand near them, look attentive, and wait for them to swap out. I expect others to do the same. Sometimes I have to ask. But I don't think I've had anyone competent refuse to wait or express confusion about why.

It's possible nerd communities make a distinction between work-interruption and conversation-interruption. Most of my communication is textual so I may not have noticed. One of the things I've always loved about communication on the Internet is that interruption is effectively impossible; both parties can get their say without being dependent on the other party to shut up for a moment.

I believe that emotions play a big part in thinking clearly, and understanding our emotions would be a helpful step. Would you mind saying more about the time you spend focused on emotions? Are you paying attention to your concrete current or past emotions (i.e. "this is how I'm feeling now", or "this is how I felt when he said X"), or more theoretical discussions "when someone is in fight-or-flight mode, they're more likely to Y than when they're feeling curiosity"?

You also mentioned exercises about exploiting emotional states; would you say more about what CFAR has learned about mindfully getting oneself in particular emotional states?

Progress is reduction of expected work remaining.

No it isn't. Those things are often correlated but not equivalent. New information can be gained that increases the expected work remaining despite additional valuable work having been done.

I like the phrase "precedent utilitarianism". It sounds to utilitarians like you're joining their camp, while actually pointing out that you're taking a long-term view of utility, which they usually refuse to do

On what basis would you say it's the case that utilitarians usually refuse to take a long-term view of utility?

… if anyone asks, I did not tell you it was ok to do math like this.

Agreed. Though of course, I don't really see Faramir as disagreeing -- it was, after all, the Rangers of Ithilien who ambushed the Haradrim and killed the soldier they're talking about.

I'm a little bit proud that I don't know who all these people are.

Some non-nitwit (actual-economic-value-generating) startups I've heard proposed lately by people in this or related communities:

• Kevin Fischer is interested in identifying useful sub-chemicals in certain legal psychoactive plants. Anyone with biotech, chemical-identifying training would be useful to him.
• Mike Darwin (not LW-style rationalist, but cryonicist) says that his research and numerous other papers show that melatonin, among some other chemicals, is very effective at preventing cerebral-reperfusion ischemic injury which is the real killer in heart attacks and strokes, and for which there are apparently not currently approved medications.
• Zvi Mowshowitz is now trying to refound a startup to provide evidence-based, rationalist-filtered medical care - evidence-based doctors as opposed to just evidence-based medical research that often gets ignored by actual doctors.
• John Schloendorn is the most competent biotech guy I know. He was literally trying to cure cancer - by trying to duplicate the abilities of a 100%-cancer-immune strain of mice, in humans - when his startup ran out of money; and he has a lot of other low-hanging fruits on his list as well.