Bugmaster comments on How to Fix Science - Less Wrong

50 Post author: lukeprog 07 March 2012 02:51AM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (141)

You are viewing a single comment's thread. Show more comments above.

Comment author: Bugmaster 05 March 2012 06:28:21PM 0 points [-]

If I collect a set of totally useless methods, followed by method A which is not totally useless, followed by method B which is also not totally useless.

Perhaps I misinterpreted lukeprog's words. It sounded to me like he was saying, "all these other methods were useless, and science is the only method we know of that isn't useless". This makes science the best thing we've got, by definition, though it doesn't preclude the existence of better methods that are yet unknown to us.

Comment author: TheOtherDave 05 March 2012 10:16:28PM 2 points [-]

Ah, I see.

Sure, if by "first" Luke meant /only/, then yes, Luke's statement (properly interpreted) means science as we know it the best method currently available, and essentially contradicts the rest of his post. Agreed.

Can you clarify your reasons for thinking he meant /only/ rather than /first/?

Comment author: Bugmaster 05 March 2012 10:25:54PM 0 points [-]

I interpreted "first" to mean "first in human history". Since science is the method we currently use to understand the world, I assumed that no other methods were as good -- otherwise, we'd be using those. Human history is ongoing, though, so we could find better methods in the future.

Luke goes on to discuss different ways of fixing science, which led me to believe that he doesn't know of any other methods that are superior to science, either. If he did, presumably he'd be advocating that we drop science altogether, and replace it with these other methods.

Comment author: TheOtherDave 06 March 2012 04:04:00AM 2 points [-]

It's by no means clear that we're generally using the best methods ever discovered. In fact, it seems unlikely on the face of it. It's not especially uncommon for an earlier method to become popular enough that later superior methods fail to displace it in the popular mind.

That aside, at this point it kind of sounds like the main thing we're disagreeing about is whether the two different things under discussion in Luke's post are both properly labelled "science" or not. In which case I'm happy to concede the point and adopt your labels. In which case everyone involved agrees that we should use science and the interesting discussion is about what techniques science ought to comprise.

Comment author: Bugmaster 06 March 2012 04:55:10AM 0 points [-]

It's not especially uncommon for an earlier method to become popular enough that later superior methods fail to displace it in the popular mind.

Can you list some examples ?

That aside, at this point it kind of sounds like the main thing we're disagreeing about is whether the two different things under discussion in Luke's post are both properly labelled "science" or not.

Good point. I guess it all depends on whether the changes Luke proposes should count as reforming science, or as replacing it with an entirely new methodology. I don't think that his changes go far enough to constitute a total replacement; after all, he even titled the article "How to Fix Science", not "Science is Dead, Let's Replace It".

In addition, I think that the changes he proposes to fix publication bias and NHST are incremental rather than entirely orthogonal to the way science is done now (I admit that I'm not sure what to make of the "experimenter bias" section). But it sounds like you might disagree...

Comment author: TheOtherDave 06 March 2012 05:23:05PM 0 points [-]

Can you list some examples ?

Just to pick one we're discussing elsethread, billions of people around the world continue to embrace traditional religious rituals as a mechanism for improving their personal lot in life and interpreting events around them, despite the human race's discovery of superior methods for achieving those goals.

I guess it all depends on whether the changes Luke proposes should count as reforming science, or as replacing it with an entirely new methodology.

Yes, agreed, that's basically what our disagreement boils down to. As I said before, if we can agree about what changes ought to be made, I simply don't care whether we call the result "reforming science" or "replacing science with something better."

So I'm happy to concede the point and adopt your labels: we're talking about incrementally reforming science.

Comment author: Bugmaster 06 March 2012 06:45:11PM 0 points [-]

Just to pick one we're discussing elsethread, billions of people around the world continue to embrace traditional religious rituals as a mechanism for improving their personal lot in life...

Fair enough, I think I misinterpreted what you meant by "popular mind". That said, though, all the people whose primary job description is to understand the natural world are currently using science as their method of choice; at least, all the successful ones are.

we're talking about incrementally reforming science.

In that case, we have no fundamental disagreements; incremental improvement is always a good thing (as long as it's actually an improvement, of course). The one thing I would disagree with Luke (and, presumably, yourself) about is the extent to which "science is broken". I think that science works reasonably well in its current form -- though there's room for improvement -- whereas Luke seems to believe that science has hit a dead end. On the other hand, the changes he proposes are fairly minor, so perhaps I am simply misinterpreting his tone.

Comment author: TheOtherDave 06 March 2012 11:40:16PM 0 points [-]

Fair enough. I agree that if by "popular mind" we mean successful professional understanders of the natural world, then my assertion that science has not displaced religion in the popular mind as a preferred mechanism for understanding the world is at the very least non-obvious, and likely false. That seems an unjustified reading of that phrase to me, but that doesn't matter much.

I can't decide if we disagree on whether "science works reasonably well in its current form", as I don't really know what that phrase means. Even less can I decide whether you and Luke disagree on that.

Comment author: Bugmaster 06 March 2012 11:50:46PM 1 point [-]

I can't decide if we disagree on whether "science works reasonably well in its current form", as I don't really know what that phrase means.

I wanted to express something like this: let's imagine that we (we humans, that is) implement none of Luke's reforms, nor any other reforms at all. Scientific journals continue to exist in their current form; peer review keeps working the same way as it does now, everyone uses p-values, etc. Under this scenario, what percentage of currently unsolved scientific problems (across all disciplines) can we expect to become solved within the next 100 years or so ?

If science is truly broken, then the answer would be, "close to zero". If science works very well, we can expect an answer similar like, "twice the percentage that were solved during the last 100 years", or possibly "all of them (though many new problems will be discovered)". If science works ok, but is not as efficient as it could be, we could expect an answer somewhere between these two extremes. I personally believe that this latter scenario is closest to the truth.

Comment author: TheOtherDave 07 March 2012 01:33:21AM 1 point [-]

Well, I certainly agree that in the absence of adopting any reforms to how science is done, we should expect some percentage between "close to zero" and "all" currently unsolved problems to be solved in the next century.

If I adopt your stricter measure of between epsilon and 2N where N is the %age of problems solved in the last century, I still agree that we should expect some such threshold to be met or exceeded in the next century.

If that implies that science is not broken, then I agree that science is not broken.

Comment author: Eugine_Nier 06 March 2012 02:29:01AM 2 points [-]

I interpreted "first" to mean "first in human history". Since science is the method we currently use to understand the world, I assumed that no other methods were as good -- otherwise, we'd be using those.

Note necessarily since humans aren't perfectly rational.