Morendil comments on Diseased disciplines: the strange case of the inverted chart - Less Wrong

47 Post author: Morendil 07 February 2012 09:45AM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (150)

You are viewing a single comment's thread. Show more comments above.

Comment author: Morendil 05 February 2012 10:45:38AM 3 points [-]

seems that both charts support the same conclusion: the longer problem goes undetected, the more problems it brings

Yes, there's a sort of vague consistency between the charts. That's precisely where the problem is: people already believe that "the longer a bug is undetected, the harder it is to fix" and therefore they do not look closely at the studies anymore.

In this situation, the studies themselves become suspect: this much carelessness should lead you (or at least, it leads me) to doubt the original data; and in fact the original data appears to be suspect.

As Software Engineering is too far from being a science anyway

Yes, that's where I end up. :)

Comment author: vi21maobk9vp 05 February 2012 11:44:48AM 1 point [-]

In this situation, the studies themselves become suspect: this much carelessness should lead you (or at least, it leads me) to doubt the original data; and in fact the original data appears to be suspect.

Of course, by 1989 both experience and multiple cause-and-effect explanations told people this is the case. And the two graphs are actually different data sets with the same conclusion, so it looks like people just took whatever graph they found quickly.

Comparing early quickly-found bugs and late quicky-found bugs is still impossible with this quality of data, but it is for the better. The real problem is not citing graph correctly - it is about what affects both bug severity and bug detection. Like having any semblance of order in the team.

As Software Engineering is too far from being a science anyway Yes, that's where I end up.

Are there people that claim this is about true science and not set of best practices? Maybe they are the real problem for now...

Comment author: Morendil 05 February 2012 12:03:28PM 5 points [-]

Typical quote: "Software engineering is defined as the systematic application of science, mathematics, technology and engineering principles to the analysis, development and maintenance of software systems, with the aim of transforming software development from an ad hoc craft to a repeatable, quantifiable and manageable process."

And the publications certainly dress up software engineering in the rhetoric of science: the style of citation where you say something and then add "(Grady 1999)" as if that was supposed to be authoritative.

It will be impossible to make progress in this field (and I think this has implications for AI and even FAI) until such confusions are cleared away.