= 27d567bc2d48d9f40e9ccc20ea370b15)
V_V comments on How about testing our ideas? - Less Wrong
You are viewing a comment permalink. View the original post to see all comments and the full post content.
= 783df68a0f980790206b9ea87794c5b6)
Loading…= b715d02e85f7b3b4ae65ca3d3e450868)
Subscribe to RSS Feed
Powered by Reddit
= f037147d6e6c911a85753b9abdedda8d)
You are viewing a comment permalink. View the original post to see all comments and the full post content.
Comments (113)
It seems that you are suggesting turning LW into some sort of alternative circuit for scientific pubblication.
That seems an inefficient thing to attempt. If you do publishable research, publish it to a peer-reviewed academic journal or conference.
You might use LW as a place to exchange preliminary ideas and drafts. Personally I'm a bit skeptical that it would yield a significant increase of productivity: scientific research typically requires an high level of domain specialization, it's unlikely that you would find many people with the relevant expertise on LW, and the discussion will be uninteresting for anybody else, but you might want to give it a try.
You can call it that. I call it refining the art of human rationality. I don't think building new knowledge is something that magically only happens in a box designated Academia. Remember SI did years of research basically outside it, they only started publishing so they could attract more talent and as a general PR move, not because it was the most efficient way to do it. We are already an alternative circuit for scientific publication. This is exactly what we do every time we publish an article carrying some novel take on human rationality or some instrumentally useful advice. We are just bad at it.
You don't seem to have read the related articles I cited. I strongly suggest you do.
I would also recommend you read Why Academic Papers Are A Terrible Discussion Forum. As to your invoaction of the somewhat broken formal peer review process that came into existence in the 20th century and is sadly still with us (I recommend you search Vladimir_M's comment history for more information on arguments against it) and the even more broken journal system I don't feel like attacking those particular applause lights right now.
Science is much more than its current flawed implementation. As I said if you can get this research, that we need to figure out if our original thinking and speculation is non-sense or not, done inside academia, great job! More power to you. But I do think we should be open to doing it ourselves when this is needed. The better ones clearly should also be posted on preprint sites like ArXiv. Indeed the very best work probably would be worth paying the price of the significant effort needed to craft papers respected journals are likely to accept.
Good enough so it counts as strong Bayesian evidence =/= Will be published by a peer reviewed journal
No, but Academia is optimized for that and has hundreds of years of demonstrated effectiveness and accumulated experience.
Is it perfect? No.
Can you build something better from ground up? I don't think so, at least not at a cost smaller than the cost needed to improve it. Certainly the present LW doesn't look remotely like a superior alternative.
And what did they accomplish? Pretty much nothing, AFAIK.
If that's their main reason to publish then it seems that they are doing the right thing for the wrong reason.
Are they so deluded to seriously believe they will be the first to build an AGI? I hope not. Therefore, if they want to have a chance to influence the development of AI projects, the best they can do is to disseminate the results of their research in a way that will maximize the number of AI researchers who will notice it and take it seriously. And this way is not Less Wrong, or an Harry Potter fanfiction or meetups or minicamps or all the other stuff they do. It's academic publishing. Academic publishing should be SI's raison d'etre, not a PR move.
So far, AFAIK (I'm not a SI historian, so I might be mistaken) they published a few papers on philosophy, on the same sort of topics the FHI people do (some of these papers are co-authored with them, IIUC). I didn't read all of them, but my impression is that they didn't contribute particularly novel insights.
We have yet to see what results their current research on program equilibrium will yield.
I've skimmed them. The "Science: Do It Yourself" uses an examples that is flawed by a glaring methodological error from the start. That speaks lots of why scientific research, like many other activities, is something best left to professionals.
Actually the features of academia I'm criticizing are much younger than that. The modern peer review system is something Einstein didn't have to deal with for example. If you think hundreds of years of scientific progress are a good track record for a system I have some news for you...
Obviously none of his great papers could have survived peer review. Some people argue that this was merely because of trivial stylistic issues, and could have been fixed by giving citations in correct format, and so on and so forth, so that they read like modern peer reviewed papers.
Perhaps
But the fact that he got his degree with a boring trivial paper, when he had several of his greatest papers in hand, suggests that there was no fixing them. If they could not be submitted for a degree, probably could never pass peer review. Peer review is in a sense a form of committee, and committees tend to be dumber than their dumbest member. The primary job of a committee, whatever its ostensible job, is to make sure that no one rocks the boat, just as the primary job of a bureaucracy, whatever its ostensible job, is to breed red tape with red tape to generate more red tape.
Getting a group of people to function together so that their output is smarter than any one of them is hard, a deep and unsolved problem. The normal outcome is that their output is dumber than any one of them.
The scientific community solved this problem from the late seventeenth century to late nineteenth or early twentieth century. Although engineering continues to advance, and more powerful tools such as DNA readers continue to advance science, science itself seemed to run out of puff after Einstein.
I'm guessing this post was down voted because of author not content because I can't find anything wrong with the latter.
Yes this is evidence towards him not being sure those papers could be fixed.
Exactly, coordination is hard. Perverse incentives, Goodhart's law, agency dilemma, etc.
See most non-profit organizations ever.
Stagnation in our time.
While I think you are right for most fields, I would argue we see a relatively healthy culture and even functional institutions when it comes mathematics since they have been making considerable progresses. I'm continually shocked at just how many of say recent advancements in evolutionary biology are basically rediscoveries of what Darwin himself said! In general reading and taking seriously the best of old thinkers is an excellent use of spare time for the intellectually curious.
Peter Thiel makes the case that outside of computers we aren't seeing much advancement in engineering either since at least the 1970s. He cites cultural reasons but also notes we've effectively outlawed innovation in the world of "stuff" but not the world of "bits", so those who like innovation go to Wall Street or Silicon Valley. Arguably the net impact of more people going to Wall Street to practice financial voodoo have been decidedly negative. If it was worth paying the opportunity costs for all those people to go to Sillicon Valley also may not be as clear cut as we may like to imagine, especially if you take Eliezer's arguments about the dangers of AI seriously and remember that the area of "stuff" includes such fields such as energy and medicine which radically alter quality of life.
Not only do many confuse progress of technology for scientific progress, they are used to thinking about building up knowledge about the world and healthy institutions of science as being basically the same thing. Which isn't true at all. We've had millennia of gains in naturalistic knowledge before we ever came up with the scientific method let alone the culture of science! Westerners from about 1700-1950 did something remarkably right to do so much with so little. What could they have done with the tools available today! I hope no one will bring up a low hanging fruit counterargument here, as it is hard to argue that what they picked hadn't been low hanging fruit for a respectable and advanced civilizations like that of the Chinese as well.
No. I don't know the author but downvoted the naive understanding of progress implied with "ran out of puff after Einstein". It was cheap cynicism signalling that seemed misleading to me (especially since the earlier parts of the comment came across as authorative.)
Thank you for explaining this.
I also downvoted, and I actually considered not doing so because it was so far above the usual standards of what I've come to expect from sam0345, but I decided that if it were written by somebody else, I would have downvoted, for pretty much the same reasons wedrifid gave.
That doesn't really follow. You could be missing a problem with the content, or someone else could mistakenly observe same. (Or both!) (Or there's some third reason to downvote comments that is neither author nor content!)
I will edit that sentence since I see your point about it.
This is not something I've overlooked. I did say I didn't find anything wrong with it, I expected to be corrected if wrong or change some minds if the down voters where wrong.
You have to admit that considering author in question my hypothesis is very likely.
Edit: Can down voters please explain why you've down voted this comment?
My guess differs from your own: Criticizing academia is as political, indeed more political, than criticizing women, blacks, gays, Islam, the poor, Mexicans, the underclass, the fatherless, and so on and so forth, but because academia does not proclaim itself as a victim group, but rather a victimizing group, no one can leap forth with outraged cries of racist sexist homobophobic Islamophobia. ("homophobic" intentionally misspelled)
Those who find my postings offensive are allowed to outraged on behalf of the poor victimized oppressed victims of victimization, but if outraged by unkind accounts of academia, have to find some new rationalization for outrage.
One may, of course, criticize academia for racism sexism etc, and be much loved for doing so, but criticizing academia for intellectual misconduct, accusing academia of ignorance, closed mindedness, dogmatism, and just plain not caring about the truth does not get one loved.
You seem to be under the impression that Einstein's papers were not reviewed by professional physicists. That's incorrect: They were reviewed by journal editors who were professional physicists.
The modern peer review system was invented because during the 20th century the submissions to journals greatly increased both in number and in sub-field specialization. While journals also increased in number and specialization, they couldn't keep up with that and had to "outsource" the review process.
This is quite wrong.
Even Einstein's field, theoretical physics, had significant progress until at least the mid-70s, when the Standard Model was completed. Subsequent stagnation was probably largely due to the difficulty of obtaining experimental data: testing all the features of the Standard Model required an enormous effort culminating in the LHC, and presently we can't do experiments on Plank scale phenomena.
Other areas of science greatly progressed. Biology, for instance, is still far from stagnation.
The stagnation of theoretical physics in the last few decades is also due to information cascades and other sociological/political effects; see The Trouble with Physics by Lee Smolin.
There was no such stagnation. This is the period which saw M-theory, the holographic principle, and the twistor revival, three of the great theoretical advances of all time, and in general there was an enormous elevation in the level of technical knowledge. Smolin is just peeved that string theory is where all the action was.
The question is whether these theories correspond to reality.
Holography likely does, in some form, given that it pops up from every direction of research. The rest -- who knows.
I understand M theory sufficiently well to be seriously underwhelmed.
M theory and the holographic principle suspiciously resemble postmodernism: insiders talking to each other in ways that supposedly demonstrate their erudition, without any external check to verify that they are actually erudite, or even understand each other, or even understand what they themselves are saying. Twistors are valid and erudite mathematics, but don't seem to get us any closer to anything interesting.
M theory is just string theory only more so. The trouble with string theory as a theory of spacetime is that it takes place in a fixed space time background, thus inherently makes no sense whatever. If you start with a contradiction, you can deduce anything you please. The central problem in any quantum theory of spacetime is that you have no fixed spacetime to stand upon, and string theory just blithely ignores the problem. That is not an advance in theoretical physics, that is finding weak excuses to publish meaningless papers.
That's just an approximation. Those situations (flat space, hyperbolic space) are really just asymptotically fixed - the form of the space-time in the infinite past or the infinite future is fixed. But in between, you can have topology change.
String theory in positively curved space may even allow for topologically distinct asymptotic outcomes, but that is still a topic of great confusion.
There is a standard paradigm for applying string theory to the real world - grand unification, broken supersymmetry, compactification. I'd give that about a 50% chance of being correct. Then there are increasingly unfamiliar scenarios, the extreme of which would be a theory in which you don't even have strings or branes, but in which some of the abstract properties of string theory (e.g. the algebraic structure of the amplitudes) still hold. The twistors could swing either way here: twistorial variables may exist for an orthodox string scenario, but there may also be twistorial theories way outside the usual M-theoretic synthesis.
Are those on par with Einstein's work? (Maybe they're close enough -- though I'd wait a bit longer before saying that; but if you count how many physicists are working today and how many were working in the early 20th century...)
They are not. Einstein took some existing math and the hints of new physics and built a beautiful model on top of it, with marvelous new predictions, all of which pan out. Then he did it again 10 years later.
Nothing like it has been repeated since. Creation of QM was close, but it was a collective effort over decades, still quite unfinished.
But Einstein only needed one journal editor to decide that his paper was good stuff that would rock the boat, whereas under peer review, he would in practice need every peer reviewer to agree that his papers did not rock the boat.
Under the old system, he needed one of n to get published. Under the new system, it tends to be closer to n of n.
Consensus, as Galileo argued, produces bad science.
And, pretty obviously, we are getting bad science.
Recall the recent study reported in nature that only three of fifty results in cancer research were replicable.
The background to this replication study is that biomedical companies pick academic research to try to develop new medications - and they decided they needed to do some quality assurance.
The exact rules of peer review vary between different journals and conferences, but in general no single referee has veto power. If there is major disagreement between referees, they will discuss, and if they fail to form a consensus the journal editors / conference chairmen will step in and make the final decision, after possibly recruiting additional referees.
This seems to be a more accurate process than having a single editor making a decision based on only their own expertise.
That's a false positive problem, while you seemed to be arguing that peer review generated too many false negatives.
Anyway, neither referees nor editors try to replicate experimental results while reviewing a paper. That's not the goal of the review process.
The review process is not intended to be a scientific "truth" certification. It is intended to ensure that a paper is innovative, clearly written, easy to place in the context of the research in its field, doesn't contain glaring methodological errors and is described in sufficient detail to allow experimental replication. Replication is something that is done by independent researchers after the paper is published.
Is there any way to tell whether a lack of major new science is the result of institutional problems or if it's caused by an absence of major discoveries which could be made with current tools?
Never mind DNA itself, which was discovered post Einstein
What I've inferred from statements of key SI folk (most especially Luke) is that they don't think this likely, but they think the possible futures in which it happens are vastly superior to the ones in which it doesn't, so they're working towards it anyway.
Yeah, this seems pretty plausible to me as well. (Though also pretty unlikely.)
FWIW, my understanding of SI's original chosen strategy for making AI researchers take LW's ideas about Friendliness seriously was to publicize the Sequences, which would improve the general rationality of people everywhere (aka "raise the sanity waterline"), which would improve the rationality of AI researchers (and those who fund them, etc), which would increase the chances of AI researchers embracing the importance of Friendliness, which would increase the chances of FAI being developed before UFAI, which would save the world.
From what I can tell, SI has since them moved on to other strategies for saving the world, like publishing the Sequences in book form, publishing popular fiction, holding minicamps, etc., but all built on the premise that "raising the sanity waterline" among the most easily reached people is a more viable approach than attempting to reach specific audiences like professional researchers.
That's seems to be an inefficient approach.
Even if you accept the premise that you can "teach" rationality to AI researchers capable of building an AGI (who probably would not be idiots, but they might be indeed affected by biases), doing so it's still an extremenly unfocused way to accomplish the task of advancing the state of the art on machine ethics.
If you want to advance the state of the art on machine ethics, then the most efficient way of doing it is to do actual research on machine ethics. If AI researchers don't take machine ethics as seriously as you think they should, then the most efficient way to convince them is to put forward your arguments in forms and media accessible and salient to them.
Once you go for peer review, you may receive negative feedback, of course. That might mean two things: That your core claims are wrong, in which case you should recognize that, stop wasting your efforts and move to something else, or that your arguments are uncompelling or unclear, in which case you should improve them, since it is your responsibility to make yourself understood.
My admittedly incomplete understanding is that "raising the sanity waterline" activities have now been spun off to the Center for Applied Rationality, which is either planning to incorporate as a non-profit or already incorporated. This would then leave SIAI as focusing on the strictly AGI- and Friendliness-related stuff.
Ah. I'm aware of the SI/CFAR split, but haven't paid much attention to what activities are owned by which entity, or how separate their staffs and resources actually are. E.g., I haven't a clue which entity sponsors LW, if either, or even whether it's possible to distinguish one condition from the other.
From the information available on their websites, it seems that LW is still operated by SI.
I suggest splitting it off an operating it as a charity separate from both SI and CFAR.