Maybe "still as consciously engaged in it as [...] in the beginning" was too strong, but compare programming to driving. If you're like most drivers you can do basically anything while driving (in normal traffic and weather conditions), as long as it doesn't require you to take your limbs from the car's controls or your eyes from the road.

The programming equivalent of this would be that you can write a program (let's say a binary tree implementation in C for the sake of argument) while having a conversation, and

• this would not make you take noticeably longer to write the program
• nor would it mess up your conversational ability.

A test of messing up your conversational ability would be whether the friend you're talking to could tell over the phone that you're doing something else at the same time.

(I'm guessing you can't do this but I'd be interested to hear otherwise.)

Here is my understanding of the issue.

Traffic works well because driving is one of the activities that is relatively easy to internalize and perform as a matter of habit, like walking or riding a bicycle. System 1 thinking is usually fast, predictable and reliable. You don't need to "focus intently". It is true that you cannot take your eyes off the road (or, more accurately, shift your passive attention (see http://www.cdl.org/resource-library/articles/attention2.php) away) for longer than a brief instant, otherwise the subconscious feedback loop which includes visual and auditory inputs, your brain, your extremities and the vehicle breaks down, not because you have to consciously "focus intently". The "magnets in the road" are built into the area of your brain that controls your reflexes. That's why learning safe driving habits is so important.

Not everyone is equally capable of internalizing the driving process, or put in enough time to do so. These people white-knuckle it, constantly engaging their full slow and unsuitable System 2 in the loop, and consequently they find the normal driving activity exhausting, rather than relaxing. It is often easy to notice these drivers by their slow reaction to traffic lights, overly careful driving style and by generally overcompensating for road hazards. After all, System 2 is taxing, slow and unreliable. Ironically, these people may be safer drivers overall, since they never let their attention wander.

This passive attention escalates to active when something unusual happens, like when you hear emergency vehicles and have to make decisions which are not internalized. Or when some other car wanders into your lane, or if the traffic stops unexpectedly. Anything that either breaks the passive attention loop or causes this escalation from passive to active attention to slip is dangerous: texting, arguing, listening to a child fussing in the back, concentrating on a phone call.

In contrast, there are activities which do not naturally lend themselves to internalizing. Programming is one of them. Even after doing it for decades, people are still as consciously engaged in it as they did in the beginning. Thus your hope for a safe AGI "by analogy with safe driving", seems misplaced to me.

Yes, that is exactly what they are saying. It happens to be the case that this thing works for you. That is only very weak evidence that it works for anyone else at all. All humans are not the same.

We recommend getting over being insulted and frustrated when things that work for you specifically turn out to be flukes, it's not a surprising thing and sufficiently internalizing how many actual studies turn out to be flukes would make it the obvious result. Reality shouldn't be strange or surprising or insulting!

My argument had nothing to do with conscientiousness.

There is currently no convenient way to accurately track your caloric intake and caloric expenditure. Attempts to do so have all the usual sources of error that this thread already covered.

In particular, adding up the numbers on the labels of the things you eat is not a sufficiently accurate method of determining caloric intake, see for example the FDA's Food Labeling Guide, questions N30-37. That's before we begin to talk about preparation loss ratios, nutrient bioavailability and the vagaries of the human metabolic system.

So when somebody recommends CI:CO, they're recommending either 1) vapid numerology or 2) a time-sink that is also largely numerology. It's unreliable, and therefore not something I would ever suggest to somebody else.

All your self-congratulatory narcissism is also largely off-topic. In particular,

I don't know anyone else who brings this level of dedication to their diet. I know people who don't, and so they're fat.

is such a bizarre source of evidence that I can't imagine why you bothered stating it. Is it really so controversial that pseudoscientific magic is bad advice?

I was right in the grandparent. I'm tapping out.

Well, no wonder. The way the hypothetical scenario is presented evokes a giant array of ways it could go wrong.

Please don't fight the hypothetical.

I think it likely that the people Luke spoke with were likely intelligent people who knew that hypotheticals are supposed to test your values and priorities and responded in the spirit of the question.

I suspect that, with these much safer sounding provisions, most people would opt to have access to the machine rather than not, and would eventually use it for 100% of the time.

Many people become addicted to drugs, and end up using them nearly 100% of the time. That doesn't mean that's what they really want, it just means they don't have enough willpower to resist.

How humans would behave if encountered with a pleasure machine is not a reliable guide to how humans would want to behave if they were encountered with it, in the same way that the way humans would behave if encountered with heroin is not a reliable guide to how humans would want to behave when encountered with heroin. There are lots of regretful heroin users.

Personal example: The greatest feeling of bliss I have experienced is dozing off in a naturally doped up state after highly satisfying sex. This state is utterly passive, but so thoroughly pleasant that I could see myself opting for an eternity in it.

Wouldn't it be even better to constantly be feeling this bliss, but also still mentally able to pursue non-pleasure related goals? I might not mind engineering the human race to feel pleasure more easily, as long as we were still able to pursue other worthwhile goals.

If someone sounds crazy to you, maybe you have misinterpreted them.

The original post said that nutrition was "fairly easy", and that one should follow the rule of "calories in, calories out" and that one should eat "micronutrient dense food." CI:CO is broken because it's difficult to measure CI with any accuracy and intractably hard to measure CO. It ignores all sorts of subtleties like getting enough protein in your diet and the difference between bulking and cutting.

For example, good luck seeing gains if you're eating 20% fat, 70% carb, and 10% protein on a 10-15% caloric deficit. All the micronutrients in the world won't save that diet. But you're still following CI:CO!

Sigh. If this point is really so hard for people to get, maybe it's just not worth making. It's probably easier to let LW devolve into a bunch of badly-sourced self help advice than to continue tilting at windmills.

Of course.

Presumably the company in question could easily manufacture a whole bunch for itself and get a significant portion of the bitcoin market (although it would have to be careful about destroying the currency's value).

Even if everything you say is true (and, e.g., you weren't recording completely bogus fat percentage numbers, you measured your weight consistently, the internet didn't mislead you on calorie counts, etc. etc.), this gives you extremely weak evidence to expect that other people would benefit from doing the same.

Not that this argument hasn't been tried before.

Then they will blast you and the pleasure machine into deep space at near light-speed so that you could never be interfered with. Would you let them do this for you?

Most people say they wouldn't choose the pleasure machine.

Well, no wonder. The way the hypothetical scenario is presented evokes a giant array of ways it could go wrong.

What if the pleasure machine doesn't work? What if it fails after a month? What if it works for 100 years, followed by 1,000 years of loneliness and suffering?

Staying on Earth sounds a lot safer.

Suppose the person you are asking is religious. Will they be forfeiting an eternity in heaven by opting for passive pleasure in this life? They would say no, yet they are ultimately after eternal pleasure (heaven).

If you want to be fair to the person to whom you're talking, propose a pleasure machine they can activate at their convenience, and deactivate at any time. In addition, phrase it so that the person will remain healthy as long as they're in the machine, and they'll receive a minimum-wage income for spent time.

I suspect that, with these much safer sounding provisions, most people would opt to have access to the machine rather than not, and would eventually use it for 100% of the time.

Religious people may still not, if they fear losing access to heaven.

Personal example: The greatest feeling of bliss I have experienced is dozing off in a naturally doped up state after highly satisfying sex. This state is utterly passive, but so thoroughly pleasant that I could see myself opting for an eternity in it. I would still, however, only opt for this if I knew it could not end in suffering, e.g. by becoming immune to pleasant states of mind in the end

You can't volunteer; it's a job for specialists.

In some Greek myth there's a fleet heading off to war - an important endeavor, involving a group of more than 50 people - but they get held up by some bad weather. After exhausting all the usual remedies, the fleet's leadership determines that the gods have to be appeased by some extreme measure, so he summons his daughter from home and sacrifices her. It's all very sad, but it works; the storm abates and the war can proceed.

Have we considered encouraging people to donate to science in a similar way? Not ritual murder, of course. Sacrificing a child in a more figurative sense. Produce more children than you otherwise planned to, send the spares off to be trained (from a very young age, making best use of that precious neural plasticity) as specialists in whatever field will be most needful 20 years down the road.