LESSWRONG
LW

Six months ago I thought CFAR was probably a bad idea. Now I think it's worth the investment, and have been positively surprised in three major ways in the last two months about the positive effects of already-done CFAR work.
Three months ago I thought Amy + some professionals could organize the Summit without much work from the rest of SingInst; I no longer think that's true.
Due to updates about simulation shutdown risk and the difficulty of FAI philosophy (I think it's easier than I used to believe, though still very hard), I think an FAI team is a better idea than I thought four months ago.
I've downgraded my estimation of my own rationality several times in the past three months.
In the past two weeks I switched from thinking CFAR should be a "standard-template" non-profit to thinking it should be a non-profit that acts almost entirely like a for-profit company (it's very complicated, but basically: it will scale better and faster that way).
I've updated several times in favor of thinking I can grow into a pretty good long-term executive.

[-]PECOS-913y160

Due to updates about simulation shutdown risk and the difficulty of FAI philosophy (I think it's easier than I used to believe, though still very hard), I think an FAI team is a better idea than I thought four months ago.

Can you elaborate on this? Specifically, what did you learn about simulation shutdown risk, what do you mean by FAI team, and what does one have to do with the other?

3lukeprog12y

Kawoomba bumped your comment to my attention, but unfortunately I don't now recall the details of the updates you're asking for more info about. (I don't recall the "three major ways" I was positively surprised by CFAR, either.)

[-]wedrifid13y140

Six months ago I thought CFAR was probably a bad idea. Now I think it's worth the investment, and have been positively surprised in three major ways in the last two months about the positive effects of already-done CFAR work.

I just updated in favor of CFAR being a misleading acronym. Took me a while to work out that this means Center For Applied Rationality, not this. That may become less significant once google actually knows about it.

0Kawoomba12y

Bumping this comment.

[-]David Althaus13y240

Meta-Note: This is great! We should make this into a monthly or bi-monthly recurring thread like "Rationality Quotes" or "What are you working on?".

Back to the topic: I overestimated the efficacy of my anti-depressant and now believe that it was mainly placebo.

7magfrump13y

What evidence was it that led you to this conclusion? DISCLAIMER: I'm curious about this because I had the opposite experience; right now I'm just curious but this is a reminder to myself not to get argumentative about the fact that people have different subjective experiences and chemical reactions to things.

7David Althaus13y

My average happiness and productivity declined over the last two months from around 8 to 5 on a scale of 10 (I measure those things every day through introspection and write it down). Previously the average was almost never below 6-7 in any given week. I also didn't change my diet, exercise regimen, etc. and there were no difficult experiences I had to handle. Heck, I even doubled the dose of my anti-depressant - with no effect. I still think that this anti-depressant has some effect only that it isn't nearly as great as I imagined and that a changing worldview or belief-system can have a far higher impact.

1magfrump13y

That's a pretty intense effect! In fact that makes it look like the medication was having a negative effect. That's a little unsettling, although I guess antidepressants are highly variable in how they work with different people? That seems like a really huge change though, especially considering you say you didn't change your diet/exercise/stress level/etc. Rather than saying "mainly placebo" I would have said "had a significant negative effect on my mood" with that data!

1David Althaus13y

Sorry, I probably haven't expressed myself clearly. I still believe (~60%) bupropion has some, small positive effects for me and that my mood would be even worse if I didn't take it. Here's a more detailed story: I've taken this anti-depressant for quite some time, for around 8-9 months. At first I noticed a definite increase in mood and productivity (or so it seemed). I also tried to eliminate confounding effects and so stopped to take it for a week or so - and my happiness declined significantly! However, I was still concerned about placebo-effects so I made some "blind" tests (i.e. took capsules either containing bupropion or not, but without my knowledge which containing which). The next day I guessed if I had taken a mere placebo or not, and I was right about 85% of the time (n was only 7, but still...) 'cause my mood was worse when I only took a placebo. I also took two pills on some days and it felt like I had much more energy and was way happier. So Bupropion seemed like a true panacea to me. Unfortunately, my mood started to decline around two months ago therefore I went to my psychiatrist and he doubled my dose. This was around a month ago but didn't really help. I hope this clears things up! ;)

1tadrinth13y

Are you sure you're not just building up a resistance/dependence? I tried anti-depressants but they eventually stopped really doing anything, I believe somewhere between 6 months to a year after starting them. I think resistance is pretty common. Also, most anti-depressants take a while to kick in, so I suspect any day-to-day dosage changes are going to be more about withdrawal symptoms than anything else.

0David Althaus13y

Yeah, increasing tolerance is probably one of the main factors. But I thought I could counter that with the doubling of the dosage. Yup, I guess it's likely that the negative day-to-day effects were mostly withdrawal symptoms. What do you recommend? Just not taking antidepressants for a while?

3MixedNuts13y

My first tought was poop-out, but that doesn't happen with bupropion, only with SSRIs, right? You shouldn't stop and start again the same antidepressant (and, to a lesser extent, antidepressants in the same family, or that target the same neurotransmitter), this will build resistance. With SSRIs, poop-out tends to be resistance at any dose, not tolerance. I've had poop-out from placebo (WTF?) and larger doses don't work there either. A week is a bit short for a test, but bupropion is unusually fast so that's a decent test. Why are you trying to correct for the placebo effect at all? Improvement from placebo is real and desirable, and there's probably a feedback loop between placebo- and drug-related improvement that helps even further. How many antidepressants have you tried? You might just be on the wrong one. How large is your dose? If you're at 300mg/day trying going up to 450 is possible, though I'm not very optimistic. Does your depression have a seasonal pattern at all, and if so could the weather be responsible for the mood drop? You seem oddly trusting in self-experiment and advice from strangers; why are you seeking recommendations besides your psychiatrist's? Cost? Disclaimer: I'm a nutjob who reads Crazy Meds and Neuroskeptic, not a psychiatrist.

2David Althaus13y

Yeah, poop-out could be the culprit. However, this would also suggest that the positive effects were mainly placebo because, as you mention in your second comment, the most reasonable account of poop-out is that the placebo effect wears off. I was correcting for placebo cause I wanted to try lots of different antidepressants and other drugs and see which "really" work. But I guess you're right; the placebo effect is pretty amazing and one shouldn't risk eliminating it by over-zealous self-experimentation. I haven't tried any other antidepressants since I fear the side-effects of SSRIs/SNRIs and MAOIs seem like too much of a hassle (although selegiline sounds pretty good). But I'm rationalizing. I definitely should try some new stuff. In the past I was fairly happy during the summer or at least happier than in the winter so the weather has probably nothing to do with my current low. I also didn't change the brand of my antidepressant. Nah, it has nothing to do with money. My psychiatrist costs me basically nothing. So, why do I prefer self-experimentation and seek out advice from strangers? Well, I don't think very highly of my psychiatrist or doctors in general to begin with (not especially eager for a long discussion about the reasons). Furthermore, I learned a huge deal about my body through self-experimentation that I almost certainly couldn't have learned otherwise.

1MixedNuts13y

Do you know more than I do about models of poop-out? That thing is annoying and I want to know about it. You can get some interactions from trying too many antidepressants. For example, having been on SSRIs may make you build a tolerance to other SSRIs. This appears to be a big reason why psychiatrists like to stop early, along with side effects and convenience. Still there is some value in exploration. For comparing antidepressants you probably want open-label tests, and comparing meds directly against each other rather than with placebo. SSRIs are pretty damn tame; which side effects are you afraid of? Sexual side effects can be a major pain but they'll go away when you go off the meds. Vanilla-ice-cream side effects (nausea, headache, somnolence, insomnia, diarrhea, constipation, dry mouth, sweating, weight gain) are common to the majority of meds, and they all go away except the weight gain. So you should at least try them unless you have some very unusual reason. If you're worried about weight gain, try fluoxetine (which is the usual first resort, and combines well with bupropion) or sertraline. MAOIs are freaky shit. I hear they're very effective, but they have so many side effects and contraindications that they're often not worth it. TCAs have a reputation for being effective. Apparently the reason they fell out of style is that they don't tolerate small lapses (unlike e.g. fluoxetine with its absurdly long half-life) and that a couple's week worth is a fatal dose, which is very bad for suicidal patients. "Throw lots of things at depression" is an area of expertise of psychiatrists, so consider trusting them more.

2[anonymous]13y

There has been talk about side effects actually enhance the placebo effect. From Efficacy and Effectiveness of Antidepressants: Current Status of Research:

0MixedNuts13y

More ideas: There's some evidence that poop-out can affect any antidepressant, perhaps any med. The dominant theory is "When a med doesn't work, it can work at first due to placebo effect, but then be conditioned out of working". Have you been taking the exact same form of bupropion? Different brands of the same med can work differently for some people. Also, if you're taking sustained/extended-release pills, this will have affected your experiments.

0tadrinth13y

I was on an SSRI so I'm not sure any of my experience is actually relevant to bupropion. If your depression has an obvious cause, fix that instead. I was depressed because of grad school, and I got better when I graduated.

0magfrump13y

Yes, it does, thank you!

1[anonymous]13y

I assume you're telling all of this to your psychiatrist and following their advice. In addition, speaking with a psychologist about your concerns. Ask yourself if you might have bipolar disorder or seasonal depression rather than just depression. Anti-depressants don't work great in a lot of those cases, and in the case of bipolar disorder, it makes your mood swings worse. Also try multiple drugs, they don't all target the same mechanisms. As for non-professional/medication steps, you should probably follow some of lukeprogs advice on happiness (find it from the sequences page). He covers the literature on what has empirically been shown to make people happier. It's not intuitive. This illustrates how your intuition about what makes you happy is wrong. It's not having a certain model of reality or willing your mind to do something from the inside, it's about finicky lizard-brain mood machinery and daily doses of beating it into happiness by feeding it tiny doses of positive external experiences.

[-]Kaj_Sotala13y220

I read Shalizi's post on the difficulties of central planning, where he noted that even using something as simple as linear optimization to organize things becomes impossible if you need to do it on the scale of a national economy. This made me significantly reduce my belief in the proposition that something like CEV would be anywhere near computationally tractable, at least in the form that it's usually discussed.

That made me consider something like Goertzel & Pitt's human-assisted CBV approach, where much of the necessary computation gets outsourced to humans, as an approach that's more likely to work. Of course, their approach pretty much requires a slow takeoff in order to work, and I consider a hard takeoff pretty likely. Logically I should then have updated more towards considering that we'll end of losing our complexity of value during the Singularity but I didn't, possibly because I was already giving that a very high probability anyway and I can't perceive my intuitive probability estimates in sufficiently high precision for the difference to register. However I did update considerably towards thinking that Goertzel's ideas on Friendliness have more merit than I'd previously presumed, and that people should be looking in a direction like the one Goerzel & Pitt propose.

9gwern13y

Shalizi's post also points out that if you relax any of the requirements, you can get answers much more quickly, and also notice that modern computers & algorithms run vastly faster. As a matter of fact, linear optimization is one of the best examples of progress: (1988 is, incidentally, way after the cited short paper pointing out the impossibility of computing in time, IIRC.) Given that CEV is all about extrapolating, making consistent, simplifying and unifying aggregate preferences, I wouldn't take linear programming as much more relevant to CEV as, say, various ruminations about NP or EXP-time.

5Vaniver13y

The best requirement to relax, in my opinion, is that of optimality (which, incidentally, is a strong reason to be an adaptation executor rather than a utility maximizer!). My professional research is into optimization heuristics that just focus on getting good solutions and not worrying if they're the best ones- which allows tackling problems that are immensely larger. For many problems, it's simply not worth the time to ensure that no better solution exists- it's a lot of effort for little payout.

1Kaj_Sotala13y

Yes, but he still considers it impossible even with modern computers and algorithms. I'm not sure how extrapolating, making consistent, simplifying, unifying, and implementing the aggregate preferences-in-general of everybody on Earth would be easier than simply implementing the resource-related preferences of everybody in a single nation.

1gwern13y

To followup from http://cscs.umich.edu/~crshalizi/weblog/919.html

1gwern13y

His fundamental conclusion (see the section on non-convexity) is that it's as hard for capitalism as planning, which isn't really an issue for CEV. ('OK, so fine, we'll go with either system as convenient, and apply optimization to as large subunits as we can manage before it breaks down.') I thought it was obvious. The difficulty is related to the number of arbitrary distinct constraints being enforced. Reduce the number of constraints, and you reduce the difficulty. Whether CEV is actually possible - whether the reduction in constraints happens and a Parfitian convergence of ethical criteria happens - is the fundamental question and doubted by many, but also unaffected by what kind of complexity linear optimization may be!

5Mitchell_Porter13y

The principal challenge of CEV is not the part where you take into account all the specific, contingent individuals who exist on the earth, in the course of following some recipe for extrapolating, aggregating, etc. The important step is figuring out the "recipe" itself. If we had a recipe which we knew to perfectly reflect true human ideals, but it proved to be computationally infeasible, we (or a FAI) could then think about feasible approximations to this ideal. The important questions are: What sort of decision system is a typical human being? What are the values and value templates with which it comes equipped? These questions should have exact answers just as objective as the facts about how many limbs or how many bones a typical human has, and those answers should in turn imply that "friendliness" and "CEV" ought to work in some specific way.

[-]Sarokrae13y210

I changed my mind about my own effectiveness at relationships, and downgraded my confidence in being actually in control of my brain. I've upgraded my estimation of the extent to which I am like a typical female.

Specifically, what I've learned is that in dealing with things that are, to a large extent, affected by my unconscious, it is helpful to treat my conscious and unconscious as separate agents, only one of which I am in control of. In doing this, I noticed that the proportion of my decision making affected by unconscious "blips" was higher than I thought, and furthermore that my unconscious reacts to stimuli in a way which is predicted by pua-Game to a far greater extent than I believed (despite me being a very atypical female).

Concrete predictions which have changed as a result of this experience: I've increased my confidence in being able to deal with future relationship problems. If future problems do arise, I plan to use trusted sources on LTR-Game (to decipher my unconscious) as well as conscious reasoning. I've also massively decreased my confidence that polyamory is a worthwhile relationship model for me to attempt at any point (while my conscious thinks it's... (read more)

6Raemon13y

I had a very similar experience a few months ago (replacing "typical female" with "typical male"). Or at least an experience that could have outputted a nearly identical post. The experience felt incredibly crippling and dehumanizing. Towards the beginning of my experience I predicted ways in which I was likely to make bad decisions, and ways I was likely to be emotionally affected. For a few weeks I made an effort (which felt near-herculean) NOT to make those errors and avoiding those emotional consequences. Eventually I ran out of willpower, and spent a month watching myself making the bad decisions I had predicted. I came out this with a very different mental model of myself. I'm not sure I consider it a positive yet. I make better predictions about myself but am not noticeably better at actually acting on the information.

[-]Sarokrae13y120

This experience has definitely been a positive for me, because I now have a more accurate model of my own behaviour which does allow me to more successfully solve problems. (Solving this particular problem has cause relationship satisfaction to shoot up from an albeit quite low slump to a real high point for both me and my OH.)

I'll just share the main specific technique I learned from the experience, just in case it might also work for you. When I treat my conscious and unconscious as separate agents, I accept that I cannot control my unconscious thinking (so just trying really hard not to do it won't help much), but I can /model/ how my unconscious reacts to different stimuli. If you draw the diagram with "Me", "Hamster" (what I call my unconscious) and "World" (including, specifically to here, the behaviour of my OH), and use arrows to represent what can affect what, then it's a two-way arrow between Me and World, a one-way arrow from World to Hamster, and a one-way arrow from Hamster to Me. After I drew that diagram it became pretty bloody obvious that I needed to affect the world in such a way as to cause positive reactions in Hamster (and for which I need accurate models of both World and Hamster), and most ideally, kickstart a Me -> World -> Hamster -> Me positive feedback loop.

1khafra13y

Would the Litany of Tarski and a hug from nyan_sandwich help? I'm interested in the ways you and Sarokrae actually noticed these "blips." I usually don't notice myself making decisions, when I make them; perhaps if I did spend some time predicting how a person in my circumstances would make bad decisions, I could notice them afterwards.

4Sarokrae13y

I'm not sure if describing what the blip feels like would help without going through the process of discovery, but I'll have a go anyway: it's noticing that you have a thought in your head without remembering the process you got through to reach it. When there's a new thought formed that's within easy mental grasp distance, especially when it's a judgement of a person or an emotion e.g. attraction, and the reason for it is not within an easy grasp distance, then that's a sign for me that it's an unconscious conclusion. Basically if a new thought that feels "near" appears, but when I ask myself why, the answer feels "far", that's a sign that if I did retrieve the answer it would be a rationalisation rather than the actual explanation, and I attempt to abort the retrieval process (or at least proceed with many mental warning signs).

0GLaDOS13y

I may have read too much Roissy but I've grown used to calling that part of me my hamster. (^_^) Edit: Heh. It seems I'm not alone in using this terminology. Also overall my personal experience has been very similar, except I haven't yet tried poly.

[-]komponisto13y190

My probability that cryonics will work has gone down after reading this.

[-]Paul Crowley13y140

I expect to read a dashed-off blog comment that makes half a maybe-plausible technical argument against cryonics but is not turned into a blog post that sets the argument out in detail roughly once every three months. Every day I don't read one, my confidence goes up. Reading that comment returns my confidence down to where it was about three months ago.

1gwern13y

Excellent - now apply a hope function and be even more precise!

0CasioTheSane13y

I think kalla724 might have something there, but I really hope he/she posts something with more details, and some specific examples. However, there is enough information in that post for someone with enough free time, a background in biochemistry, and literature access to begin researching the idea on their own...

[-]gwern13y190

I've continued to research iodine's effect on IQ in adults & children, and the more null studies I manage to find the more pessimistic I get. I think I'm down to 5% from 50% (when I had only Fitzgerald 2012 as a null). The meta-analysis reports, as expected, a very small estimated effect size.

0CarlShulman13y

Do you think the large scale rollout studies were wrong too?

2gwern13y

No, I think they were right; at least, your question suggests you think I am pessimistic on an issue different from the actual issue I am pessimistic on.

5CarlShulman13y

My understanding was that iodine helped with early brain development in children for areas with severe deprivation and poverty diets, and that this was easily vanquished by iodized salt (even if you buy sea salt, you would get iodine from meat and iodized salt in prepared foods). So I haven't worried about going out of my way for it myself, but have been keen on promoting it in regions with widespread goiter and non-iodized salt in Africa and South Asia (and hostile to rich country non-iodized salt for effects on children and pregnant women).

6gwern13y

My current understanding is that 'early' here is turning out to be very early indeed; the window of opportunity is basically pregnancy. This is very bad for my hopes of using it in an adult (myself).

[-]bramflakes13y160

I met a relativist postmodern-type that also understood evolutionary psychology and science in general.

9Jayson_Virissimo13y

What, you hadn't heard of Paul Feyerabend?

9CarlShulman13y

More detail, please.

3[anonymous]13y

2[anonymous]13y

Seconding the call for more details.

5bramflakes13y

Well when they made claims like "logic/math is only one way of knowing things, along with metaphor and [something else I forgot]" and some claims about culture being distinct from individuals, I readied myself for explaining how human intuition and culture ultimately reduces to deterministic firing of neurons. When I started on that, they jumped ahead and said that they already knew and accepted a soul-free, lawful universe. I was a bit stumped, until I realised that I'd just pattern-matched those claims onto a stereotype of the kind of people that would write in The Social Text. Upon reflection, I had been talking more in the vein of "The Universe is lawful, even when we don't know the laws, so math ultimately drives everything", and they had been more along the lines of "We can't go around calculating probabilities all the time, so we might as well go by intuition most of the time". It was a failure to cross inferential gaps on my part.

0athingtoconsider13y

I think you just broke LW's new commenter CSS.

0bramflakes13y

Looks fine to me?

[-]Solvent13y150

Young adult male here.

I've come to the conclusion that I'm nowhere near as attractive or good with girls as I thought I was.

I got my first girlfriend pretty much by accident last year. It was so incredibly amazing that I decided that romantic success was something I needed to become very good at. I spent quite a while reading about it, and thinking about how to be attractive and successful with women. I broke up with my girlfriend as I moved to a small town for two months at the beginning of this year, during which time I practiced approaching girls and flirting with them.

Then I moved to college, and the first attractive, smart girl I saw, I went up to her and got her as a girlfriend pretty much immediately. I thought that I must have been very good and attractive to have gotten such a gorgeous girlfriend so quickly. She broke up with me after a month or two. She immediately moved through two or three boyfriends over the space of a month or two. Meanwhile, I've been looking for a new girlfriend, but haven't had any success.

So I thought I was attractive and good with girls, and then it turned out that I just had a wild stroke of luck. So it goes.

I'm suspicious that I was simply arrogant about how good I was, and if I had thought more dispassionately, I wouldn't have been so wrong in my assessment of my own attractiveness.

[-]CasioTheSane13y210

Might I suggest that you may be looking at this all wrong- women are more attracted to your confidence than your looks. I suspect that your physical attractiveness is just fine, but the event of being dumped by this smart and beautiful woman hurt your self-confidence, and caused you to seem less attractive to other women afterwards.

The sort of guy who thinks a girl broke up with him because of his unattractiveness is very unattractive to most women, whereas the sort of guy who thinks "it's her loss, I was out of her league anyways" is highly attractive. If you get (or learn to fake) more self confidence, I predict that your success will return. Ironically, being arrogant about how good you are is both necessary and almost sufficient to actually be good.

5Solvent13y

I don't mean attractiveness just in the sense of physical looks. I mean the whole thing of my social standing, confidence and perceived coolness. But thanks for the advice.

[-]Nectanebo13y120

Austrian Economics.

I was fairly convinced too, so I am now very worried about how many other more blatant silly things I believe and may believe in the future. I've definitely been more at least a bit more wary than usual after realising this particular mistake.

I initially didn't really want to make this post, but I recognised that it was for reasons perhaps relating to status ( I was embarrassed to admit I believed something comparatively less trivial and more obviously rubbish compared to others in this thread) but it was pretty was easy to get over it once I thought about it and also thanks to the fact that this kind of thing is exactly what the thread was probably looking for.

6Jayson_Virissimo13y

"Austrian Economics" is a very large set of propositions. Which ones have you significantly changed your degree of belief in?

5Nectanebo13y

The kinds of writing produced based on 'praxeology', by people like Ludwig von Mises and Murray Rothbard, I have changed my mind to think as being not very reliable. I don't think that the arguments put forth by these Austrian Economists trying to justify when they reject empiricism and make weird assumptions are very good. This actually hurts a sizable amount of writing by these people, most of which I haven't read but any that I did end up having believed I now place very little trust in. The wikipedia synopsis of one of Mises' works should give you a fairly good idea of the kinds of assumptions he makes.. The links are fairly broad because I was admittedly trying to get a better grip on this stuff after having accepted much of it for a while, having given the whole school much benefit of the doubt early because the kinds of ideas it concluded with (stuff like lassiez-faire econ, not to regulate) fitted in with my preconcieved, often political, beliefs. Politics is the mind-killer, don'tcha know. Sorry for the lack of specific propositions. I've read some Hayek too, he seems better, but I don't really know. edit: changed a word to make a sentence make sense

1syzygy13y

Austrian-minded people definitely have some pretty crazy methods, but their economic conclusions seem pretty sound to me. The problem arises when they apply their crazy methods to areas other than economics (see any libertarian theory of ethics. Crazy stuff)

1bramflakes13y

There's lots of different libertarian theories of ethics. Can you be more specific?

2syzygy13y

"Universally Preferable Behavior" by Stefan Molyneux, "Argumentation Ethics" by Hans Hermann Hoppe, and of course Objectivism, to name the most famous ones. Generally the ones I'm referring to all try to deduce some sort of Objective Ethics and (surprise) it turns out that property rights are an inherent property of the universe and capitalism is a moral imperative. Forgive me if you're thinking of some other libertarians who don't have crazy ethical theories. I didn't mean to make gross generalizations. I've just observed that libertarian philosophers who consciously promote their theories of ethics tend to be of this flavor.

[-]wedrifid13y110

I just changed my mind in this direction:

Due to updates about simulation shutdown risk and the difficulty of FAI philosophy (I think it's easier than I used to believe, though still very hard), I think an FAI team is a better idea than I thought four months ago.

... and slightly upgraded my expectation of human non-extinction.

Damn it is easy to let other people do (some of) your thinking for you.

[-]WrongBot13y100

Up until a month or so ago, I was convinced I'd landed my dream job. If I had a soul, it would be crushed now.

Which is not to say that it's awful, not by any means. I've just gained a new perspective on the value of best practices in software development.

9komponisto13y

Can you explain in more detail? I'm interested in learning about the downsides of programming jobs (which have been strongly promoted around here).

[-]Viliam_Bur13y100

Seconded, more explanation is needed.

My experience with the best software practices is the following:

When a deadline is near, all best software practices are thrown out of the window. Later in the project, a deadline is always near.
While the spirit of the best software practices is ignored, it is still possible to follow their letter religiously, and be proud of it. This frequently leads to promotion.

1WrongBot13y

This is precisely the problem. Not really much more to add.

2WrongBot13y

I work in video games, so my experience isn't at all typical of programming more generally. The big issues are that: * Development priorities and design are driven by marketing. * Lots of time is spent doing throwaway work for particular demos. I (and many others) wasted a couple weeks hacking together a scripted demo for E3 that will never be seen again. * The design for my portion of the project has changed directions numerous times, and each new version of the feature has been implemented in a rush, so we still have bits of code from five iterations ago hanging around, causing bugs. * Willingness to work extremely long hours (70+/week) is a badge of pride. I'm largely exempt from this because I'm a contractor and paid hourly, but my salaried coworkers frequently complain about not seeing enough of their families. On the other hand, some of the grateful to have an excuse to get away from their families. * The downside of being a contractor is that I don't get benefits like health insurance, sick days, paid time off, etc. Many of these issues are specific to the games industry and my employer particularly, and shouldn't be considered representative of programming in general. Quality of life in the industry varies widely.

0wedrifid13y

Which is to say, much less than getting paid a lot and not working with dickheads?

3Morendil13y

I'm guessing "rewarded much less than knowing your way around office politics".

1WrongBot13y

The people I work with are mostly not dickheads and the pay is reasonable. It's the mountain of ugly spaghetti code I'm expected to build on top of that kills me. There's no time to do refactors, of course.

[-]Filipe13y80

I've changed my mind on whether Cosma Shalizi believes that P=NP. I thought he did, upon reading "Whether there are any such problems, that is whether P=NP, is not known, but it sure seems like it.", at his blog, only to discover after emailing him that he made a typo. I've also learned not to bet with people with such PredictionBook stats, and specially not as much as $100.00.

4[anonymous]13y

Never bet against a Gwern, good advice.

[-]tgb13y80

I have lowered my estimation of how hard it is to write rhyming, rhythmically satisfying poetry (no regard to the literary quality of the product). It has become my hobby on my walk to work. Read some Lewis Carroll and try to think along the beat pattern he uses - just garble words to yourself filling in the occasional blank that sounds good so long as you get the rhythm right. Do that for a while and you can start piecing phrases into that mold with some work and iteration. It's much more fun to just feel the beats than to count.

[-]private_messaging13y70

Changed my mind from lack of particular opinion on SI to SI being complete cranks. Changed my mind from opinion that AIs may be dangerous, to much lower estimate of potential danger. (I have post history on Dmytry account to prove changing mind).

0jsalvatier13y

What changed your mind on the latter?

1private_messaging13y

When I seen circulating notion of serious AI danger, without details, I guess I assumed it originated from better/more relevant arguments. What I see instead is arguments of general difficulty of some aspects of AI (such as real world motivation) crafted as to suggest update of unlikelihood of only "friendly AI that genuinely cares about mankind" but not the general unlikelihood of real world motivation on AI, because the one making the arguments tells you that you should update the former but doesn't tell about the latter. This is combined with some theoretical notion of "rationality" that would work for updates on a complete inference graph, but which is about as rational on an incomplete inference graph such as the one above, as it is true that due to law of inertia after walking off the top of 10 story building, you'll just keep on walking.

[-]beriukay13y60

I've changed my mind on the persuasiveness of a specific argument. I used to hold a high degree of confidence in the line of reasoning that "since nobody can agree on just about anything about god, it is likely that god doesn't exist". But then, in an unrelated conversation, someone pointed out that it would be foolish to say that "since nobody can agree on the shape of the earth, earth has no shape." I must be question-begging!

4syzygy13y

I think the correct comparison would be, "since no one can agree on the nature of Earth/Earth's existence, Earth must not exist" but this is ridiculous since everyone agrees on at least one fact about Earth: we live on it. The original argument still stands. Denying the existence of god(s) doesn't lead to any ridiculous contradictions of universally experienced observations. Denying Earth's geometry does.

0beriukay13y

That's the conclusion I came to as well, but I was worried that I was rationalizing, so I had to downgrade my confidence in the argument.

[-]jsalvatier13y60

After reading the comments on my post on Selfish Reasons to Have More Kids, I think it's somewhat less likely that it's substantially correct. I think this might be largely a social effect rather than a evidence effect, though.

[-]Daniel_Burfoot13y30

I wouldn't say I changed my mind, but I substantially increased my p-estimate that the following recipe could produce something very close to intelligence:

1) a vast unlabeled data set (think 1e8 hours of video and audio/speech data plus the text of every newspaper article and novel ever written)

2) a simple unsupervised learning rule (e.g. the reduced Boltzmann machine rule)

3) a huge computer network capable of applying many iterations of the rule to the data set.

I previously believed that such an approach would fail because it would be very difficult to "debug" the resulting networks. Now I think that might just not matter.

[-]Shmi13y20

I expected that my intuitive preference for any number of dust specks over torture would be easy to formalize without stretching it too far. Does not seem like it.

On the other hand, given the preference for realism over instrumentalism on this forum, I'm still waiting for a convincing (for an instrumentalist) argument for this preference.

7Mitchell_Porter13y

If you want a reason to prefer dust specks for others over torture for yourself, consistently egocentric values can do it. That will also lead you to prefer torture for others over torture for yourself. What about preferring torture for others over dust speck for yourself? It's psychologically possible, but the true threshold (beyond which one would choose torture for others) seems to lie somewhere between inconvenience for oneself and torture for oneself. It seems that LW has never had a serious discussion about the likely fact that the true human value system is basically egocentric, with altruism being sharply bounded by the personal costs experienced; nor has there been a discussion about the implications of this for CEV and FAI. ETA: OK, I see I didn't say how a person would choose between dust specks for 3^^^3 others versus torture for one other. Will recently mentioned that you should take the preferences of the 3^^^3 into account: would they want someone to be tortured for fifty years, so that none of them got a dust speck in the eye? "Renormalizing" in this way is probably the best way to get a sensible and consistent decision procedure here, if one employs the model of humans as "basically egocentric but with a personal threshold of cost below which altruism is allowed".

1tenlier13y

Do you really have that preference? For example, if every but one of trillions of humans was being tortured and had dust specks, would you feel like trading the torture-free human's freedom from torture for the removal of specks from the tortured. If that were so, then you just are showing a fairly usual preference (inequality is bad!) which is probably fine as an approximation of stuff you could formalize consequentially. But that's just an example. Often there's some context in which your moral intuition is reversed, which is a useful probe. (usual caveat: haven't read the sequences) Topic for discussion: Less Wrongians are frequentists to a greater extent than most folk who are intuitively Bayesian. The phrase "I must update on" is half code for (p<0.05) and half signalling, since presumably you're "updating" a lot, just like regular humanssssssssssssssssssssssssssssssssss.

3NoSignalNoNoise13y

1lessdazed13y

It's a psychological trick to induce more updating than is normal. Normal human updating tends to be insufficient).

1magfrump13y

If I recall correctly Alicorn made a reference to reversing the utilities in this argument... would you think it better for someone to give up a life of the purest and truest happiness, if in exchange they created all of the 10 second or less cat videos that will ever be on youtube throughout all of history and the future? My intuitions here say yes; it can be worth sacrificing your life (i.e. torturing yourself working at a startup) to create a public good which will do a small amount for a lot of people (i.e. making standard immunization injections also give people immunity to dust specks in their eyes)

7tenlier13y

Manipulative phrasing. Of course, it will always seem worth torturing yourself, yadda yadda, when framed as a volitional sacrifice. Does your intuition equally answer yes when asked if it is worth killing somebody to do etc etc? Doubt it (and not a deontological phrasing issue)

0magfrump13y

Certainly there's a difference between what I said and the traditional phrasing of the dilemma; certainly the idea of sacrificing oneself versus another is a big one. But the OP was asking for an instrumentalist reason to choose torture over dust specks. It is pretty far-fetched to imagine that literally torturing someone will actually accomplish... well, almost anything, unless they're a supervillain creating a contrived scenario in which you have to torture them. When you will actually be trading quality of life for barely-tangible benefit on a large scale is torturing yourself working at a startup. This is an actual decision that people make to make lives miserable in exchange for minor but widespread public goods. And I fully support the actual trades of this sort that people actually make. That's my instrumentalist argument for, as a human being, accepting the metaphor of dust specks versus torture, not my philosophical argument for a decision theory that selects it.

-1tenlier13y

Was there any reason to think I didn't understand exactly what you said the first time? You agree with me and then restate. Fine, but pointless. Additionally, unimaginative re: potential value of torture. Defending lack of imagination in that statement by claiming torture defined in part by primary intent would be inconsistent.

0magfrump13y

The reason I thought you didn't understand what I was talking about was that I was calling on examples from day to day life, this is what I took "instrumentalist" to mean, and you starting talking about killing people, which is not an event from day to day life. If you are interested in continuing this discussion (which if not I won't object) let's take this one step at a time; does that difference seem reasonable to you?

0tenlier13y

The day to day life bit is irrelevant. The volitional aspect is not at all. Take the exact sacrifice you described but make it non-volitional. "torturing yourself working at a startup" becomes slavery when non-volitional. Presumably you find that trade-off less acceptable. The volitional aspect is the key difference. The fact that your life is rich with examples of volitional sacrifice and poor in examples of forced sacrifice of this type is not some magic result that has something to do with how we treat "real" examples in day to day life. It is entirely because "we" (humans) have tried to minimize the non-volitional sacrifices because they are what we find immoral!

0magfrump13y

Point number one is: I don't understand how you can say, when I am making an argument explicitly restricted to instrumental decision theory, how day to day life is irrelevant. Instrumentalism should ONLY care about day to day life. With respect to forced sacrifice, my intuitions say I should just do the math, and that the reason volition is so important is that the reasonable expectation that one won't be forced to make sacrifices is a big-ticket public good, meaning the math almost always comes out on its side. I think that you're saying these choices have been screened off, but I think non-volitional choices have been screened off because they are in general bad trades rather than because "volition" is a magic word that lets you get whatever you want. Point three, let's turn this around... say someone is about to spend their entire life being tortured. Would you rescue them, if you knew it meant throwing a harmless dust speck into the eye of everyone ever to exist or be emulated? This should be equivalent, but both of the sacrifices here are forced since, at a minimum, some human beings are sociopaths and wouldn't agree to take the dust speck. If you want me to consider volition more closely, can you come up with some forced sacrifice choices that are reasonable exchanges that I might come across if I lived in a different world?

2magfrump13y

One possible idea: if I was the son of an African warlord, and I had the ability to make my parents' political decrees more compassionate if I talked to them after they blew off steam torturing people, but I could instead make them torture fewer people by talking to them beforehand. Here my intuitions say I should let the individuals be tortured in exchange for effecting large scale policy decisions.

1Manfred13y

Well, preferences are pretty easy to fit. Utility(world) = e^-(# of specks) - 1000*(# of people getting tortured) However, note that this still requires that there is some probability of someone being tortured that you would trade a dust speck for.

0Shmi13y

It doesn't work if you continuously increase the the severity of the minor inconvenience/reduce the severity of torture and try to find where the two become qualitatively comparable, as pointed out in this reply. The only way I see it work is to assign zero disutility to specks (I advocated it originally to be at the noise level). Then I thought that it is possible to have the argument work reasonably well even with a non-zero disutility, but at this point I don't see how.

-2Manfred13y

Utility(world) = e^-(# of specks) + X*e^-(# of people getting tortured), where X is some constant larger than 1/(1-1/e) in the incommensurate case, and less than that in the commensurate limit. Of course, this assumes some stuff about the number of people getting tortured / specked already - but that can be handled with a simple offset.

1Shmi13y

I don't think this addresses the point in the link. What happens when you go from specks to something slightly more nasty, like a pinch? Or slightly increase the time it takes to get rid of the speck? You ought to raise the disutility limit. Or if you reduce the length of torture, you have to lower the disutility amount from torturing one person. Eventually, the two intersect, unless you are willing to make a sharp qualitative boundary between two very similar events.

1Manfred13y

Yes, the two intersect. That's what happens when you make things quantitative. Just because we are uncertain about where two things should, morally, intersect, does not mean that the intersection itself should be "fuzzy."

0Shmi13y

The point is that without arbitrarily drawing the specks/torture boundary somewhere between x stabbed toes and x+epsilon stabbed toes the suggested utility function does not work.

0Manfred13y

Hm, how can I help you see why I don't think this is a problem? How about this. The following two sentences contain exactly the same content to me: "Without arbitrarily drawing the specks/torture boundary somewhere, the suggested utility function does not work." "Without drawing the specks/torture boundary somewhere, the suggested utility function does not work." Why? Because morality is already arbitrary. Every element is arbitrary. The question is not "can we tolerate an arbitrary boundary," but "should this boundary be here or not?"

0Shmi13y

Are you saying that you are OK with having x stabbed toes being incommensurate with torture, but x+1 being commensurate ? This would be a very peculiar utility function.

1Manfred13y

Yes, that is what I am saying. One can deduce from this that I don't find it so peculiar. To be clear, this doesn't reflect at all what goes on in my personal decision-making process, since I'm human. However, I don't find it any stranger than, say, having torture be arbitrarily 3^3^2 times worse than a dust speck, rather than 3^3^2 + 5. Sarcasm time: I mean, seriously - are you honestly saying that at 3^3^2 + 1 dust specks, it's worse than torture, but at 3^3^2 - 1, it's better? That's so... arbitrary. What's so special about those two dust specks? That would be so... peculiar. As soon as you allow the arbitrary size of a number to be "peculiar," there is no longer any such thing as a non-peculiar set of preferences. That's just how consistent preferences work. Discounting sets of preferences on account of "strangeness and arbitrariness" isn't worth the effort, really.

0Shmi13y

I don't mean peculiar in any negative sense, just that it would not be suitable for goal optimization.

0Manfred13y

Is that really what you meant? Huh. Could you elaborate?

[-]RolfAndreassen13y10

Two minutes ago I changed my mind on the source of the problem I was having: It wasn't an off-by-one error in my indexing but rather just failing to supply multiple work spaces for the case of multiple convolutions in the same PDF. D'oh! I must confess, though, that I wasn't convinced by anyone's argument, as such. So perhaps it doesn't count properly for this thread.

[-]Thomas13y10

I was wrong about Neanderthals and us. I was sure that they were much more alien to us, as now has appeared they were. Now we see that some even grandfathered some of us.

I was politically correct, I guess. Australian Aborigines and Europeans are nearly too distant cousins for this correctness.

1ahartell13y

Sorry, but could you elaborate on that last line?

Moderation Log