= 27d567bc2d48d9f40e9ccc20ea370b15)
gwern comments on Open Thread, November 23-30, 2013 - Less Wrong Discussion
You are viewing a comment permalink. View the original post to see all comments and the full post content.
= 783df68a0f980790206b9ea87794c5b6)
Loading…= b715d02e85f7b3b4ae65ca3d3e450868)
Subscribe to RSS Feed
Powered by Reddit
= f037147d6e6c911a85753b9abdedda8d)
You are viewing a comment permalink. View the original post to see all comments and the full post content.
Comments (295)
Absolutely. When I look at my own projects, they go like 'gathering and cleaning data: 2 months. Figuring out the right analysis the first time: 2 days. Runtime of analysis: 2 hours.'
The first time this happened to me, it drove me nuts. It reminded me of writing my first program, where it took maybe 20 minutes to write it even looking everything up, and then 2 hours to debug. That was when the true horror of programming struck me. Years later, when I came across the famous quote by Wilkes that "I can remember the exact instant when I realized that a large part of my life from then on was going to be spent in finding mistakes in my own programs." I instantly knew it for truth.
(Same for http://lesswrong.com/lw/j8f/anonymous_feedback_forms_revisited/ - downloading the data and figuring out how to join up the two CSVs in just the right way, an irritating hour. The logistic regression? Maybe 3 minutes playing around with different predictor variables.)
This realization caused one of my big life mistakes, I think. It struck me in high school, and so I foolishly switched my focus from computer science to physics (I think, there might have been a subject or two in between) because I disliked debugging. Later, I realized that programming was both powerful and inescapable, and so I'd have to get over how much debugging sucked, which by then I had the emotional maturity to do (and I suspect I could have done back then, if I had also had the realization that much of my intellectual output would be programming in basically any field).
I think the whole experience is also interesting on a meta-level. Since programming is essentially the same as logical reasoning, it goes to show that humans are very nearly incapable of creating long chains of reasoning without making mistakes, often extremely subtle ones. Sometimes finding them provides insight (especially in multi-threaded code or with memory manipulation), although most often it's just you failing to pay attention.
Threading is not normally part of logical reasoning. Compare with mathematics, where even flawed proofs are usually (though not always) of correct results. I think a large part of the difficulty of correct programming is the immaturity of our tools.