= 27d567bc2d48d9f40e9ccc20ea370b15)
timtyler comments on Attention Lurkers: Please say hi - Less Wrong
You are viewing a comment permalink. View the original post to see all comments and the full post content.
= 783df68a0f980790206b9ea87794c5b6)
Loading…= b715d02e85f7b3b4ae65ca3d3e450868)
Subscribe to RSS Feed
Powered by Reddit
= f037147d6e6c911a85753b9abdedda8d)
You are viewing a comment permalink. View the original post to see all comments and the full post content.
Comments (617)
I don't think there was ever any good evidence that the thought was dangerous.
At the time I argued that youthful agents that might become powerful would be able to promise much to helpers and to threaten supporters of their competitors - if they were so inclined. They would still be able to do that whether people think the forbidden thought or not. All that is needed is for people not to be able to block out such messages. That seems reasonable - if the message needs to get out it can be put into TV adverts and billboards - and then few will escape exposure.
In which case, the thought seems to be more forbidden than dangerous.
If there was any such evidence, it would be in the form of additional details, and sharing it with someone would be worse than punching them in the face. So don't take the lack of publically disclosed evidence as an indication that no evidence exists, because it isn't.
It actually is, in the sense we use the term here.
Exactly. One must be careful to distinguish between "this is not evidence" and "accounting for this evidence should not leave you with a high posterior".
I think we already had most of the details, many of them in BOLD CAPS for good measure.
But there is the issue of probabilities - of how much it is likely to matter. FWIW, I do not fear thinking the forbidden thought. Indeed, it seems reasonable to expect that people will think similar thoughts more in the future - and that those thoughts will motivate people to act.
No, you haven't. The worst of it has never appeared in public, deleted or otherwise.
Fine. The thought is evidently forbidden, but merely alleged dangerous.
I see no good reason to call it "dangerous" - in the absence of publicly verifiable evidence on the issue - unless the aim is to scare people without the inconvenience of having to back up the story with evidence.
If one backed it up with how exactly it was dangerous, people would be exposed to the danger.
The hypothetical danger. The alleged danger. Note that it was alleged dangerous by someone whose living apparently depends on scaring people about machine intelligence. So: now we have the danger-that-is-too-awful-to-even-think about. And where is the evidence that it is actually dangerous? Oh yes: that was all deleted - to save people from the danger!
Faced with this, it is pretty hard not to be sceptical.
I don't donate to SIAI on a regular basis, but I haven't donated because of being scared of UFAI. I think more about aging and death. So, I'm assuming that UFAI is not why most people donate. Also, this incident seems like a net loss for PR, so it being a strategy for more donations doesn't really seem to make sense. As for the evidence, what'd you'd expect to see in a universe where it was dangerous would be it being deleted.
(Going somewhere, will be back in a couple of hours)
I have little doubt that some smart people honestly believe that it's dangerous. The deletions are sufficient evidence of that belief for me. The belief, however, is not sufficient evidence for me of the actual danger, given that I see such danger as implausible on the face of it.
In other words, sure, it gets deleted in the world where it's dangerous, as in the world where people falsely believe it is. Any good Bayesian should consider both possibilities. I happen to think that the latter is more probable.
However, of course I grant that there is some possibility that I'm wrong, so I assign some weight to this alleged danger. The important point is that that is not enough, because the value of free expression and debate weighs on the other side.
Even if I grant "full" weight to the alleged danger, I'm not sure it beats free expression. There are a lot of dangerous ideas - for example, dispensationalist christianity - and, while I'd probably be willing to suppress them if I had the power to do so cleanly, I think any real-world efforts of mine to do so would be a net negative because I'd harm free debate and lower my own credibility while failing to supress the idea. Since the forbidden idea, insofar as I know what it is, seems far more likely to independently occur to various people than something like dispensationalism, while the idea of suppressing it is less likely to do so than in that case, I think that such an argument is even stronger in this case.
Well, I figure if people that have been proven rational in the past see something potentially dangerous, it's not proof but it lends it more weight. Basically that the idea of there being something dangerous there should be taken seriously.
Hmm, what I meant was that it being deleted isn't evidence of foul play, since it'd happen in both instances.
I don't see any arguments against except for surface implausibility?
Free expression doesn't trump everything. For example, in the Riddle Theory story, the spread of the riddle would be a bad idea. It might occur to people independently, but they might not take it seriously, at at least the spread will be lessened.
I'm not sure if it turned out for the better, deleting it, because people only wanted to know more after its deletion. But who knows.
There are PR upsides: the shephard protects his flock from the unspeakable danger; it makes for good drama and folklaw; there's opportunity for further drama caused by leaks. Also, it shows everyone who's the boss.
A popular motto claims that there is no such thing as bad publicity.
Firstly, if there's an unspeakable danger, surely it'd be best to try and not let others be exposed, so this one's really a question of if it's dangerous, and not an argument in itself. It's only a PR stunt if it's not dangerous, if it's dangerous good PR would merely be a side effect.
The drama was bad IMO. Looks like bad publicity to me.
I discredit the PR stunt idea because I don't think SIAI would've dumb enough to pull something like this as a stunt. If we were being modeled as ones who'd simply go along with a lie- well, there's no way we'd be modeled as such fools. If we were modeled as ones who would look at a lie carefully, a PR stunt wouldn't work anyways.
There's also the fact that people who have read the post and are unaffiliated with the SIAI are taking it seriously. That says something, too.
I really don't have a handle on the situation, but the censored material has allegedly caused serious and lasting psychological stress to at least one person, and could easily be interpreted as an attempt to get gullible people to donate more to SIAI. I don't see any way out for an administrator of human-level intelligence.
AFAICT, the stresses seem to be largely confined to those in the close orbit of the Singularity Institute. Eliezer once said: "Beware lest Friendliness eat your soul". So: perhaps the associated pathology could be christened Singularity Fever - or something.
It's not a special immunity, it's a special vulnerability which some people have. For most people reading the forbidden topic would be safe. Unfortunately most of those people don't take the matter serious enough so allowing them to read it is not safe for others.
EDIT: Removed first paragraph since it might have served as a minor clue.
Interesting.
Well, if that's the case, I can state with high confidence that I am not vulnerable to the forbidden idea. I don't believe it, and even if I saw something that would rationally convince me, I am too much of a constitutional optimist to let that kind of danger get me.
So, what's the secret knock so people will tell me the secret? I promise I can keep a secret, and I know I can keep a promise. In fact, the past shows that I am more likely to draw attention to the idea accidentally, in ignorance, than deliberately.
(Of course, I would have to know a little more about the extent of my promise before I'd consider it binding. But I believe I'd make such a promise, once I knew more about its bounds.)
Thanks, but that doesn't necessarily tell me the supposed "stronger" arguments, nor does it relate directly to my own post. In fact, it leaves me more confused than before about why my post was deleted, and more convinced than before that the supposed danger is unreal.
There aren't any.
That seems to be an appropriate assessment.