Currently most X-risk reduction resources are directed by a presumption that AGI is coming in less than a decade. I think this "consensus" is somewhat overconfident, and also somewhat unreal (i.e. it's less of a consensus than it seems). That's a very usual state of affairs, so I don't want to be too melodramatic about it, but it still has concrete bad effects. I wish people would say "I don't have additional clearly-expressible reasons to think AGI is coming very soon, that I'll defend in a debate, beyond that it seems like everyone else thinks that.". I also wish people would say "I'm actually mainly thinking that AGI is coming soon because thoughtleaders Alice and Bob say so.", if that's the case. Then I could critique Alice's and/or Bob's stated position, rather than taking potshots at an amorphous unaccountable ooze.
I'm a bit confused about whether it's actually good. I think I often run a heuristic counter to it... something like:
"When you act in accordance with a position and someone challenges you on it, it's healthy for the ecosystem and culture to give the best arguments for it, and find out whether they hold up to snuff (i.e. whether the other person has good counterarguments). You don't have to change your mind if you lose the argument—because often we hold reasons for illegible but accurate intuitions—but it's good to help people figure out the state of the best arguments at the time."
I guess this isn't in conflict, if you just separately give the cause for your belief? e.g. "I believe it for cause A. But that's kind of hard to discuss, so let me volunteer the best argument I can think of, B."
Added the content warning; thx.
I thought this was going to take the tack that it's still okay to birth people who are definitely going to die soon. I think on the margin I'd like to lose a war with one more person on my team, one more child I love. I reckon it's a valid choice to have a child you expect to die at like 10 or 20. In some sense, every person born dies young (compared to a better society where people live to 1,000).
I'm not having a family because I'm busy and too poor to hire lots of childcare, but I'd strongly consider doing it if I had a million dollars.
1I think Eliezer has oft-made the meta observation you are making now, that simple logical inferences take shockingly long to find in the space of possible inferences. I am reminded of him talking about how long backprop took.
In 1969, Marvin Minsky and Seymour Papert pointed out that Perceptrons couldn't learn the XOR function because it wasn't linearly separable. This killed off research in neural networks for the next ten years.
[...]
Then along came this brilliant idea, called "backpropagation":
You handed the network a training input. The network classified it incorrectly. So you took the partial derivative of the output error (in layer N) with respect to each of the individual nodes in the preceding layer (N - 1). Then you could calculate the partial derivative of the output error with respect to any single weight or bias in the layer N - 1. And you could also go ahead and calculate the partial derivative of the output error with respect to each node in the layer N - 2. So you did layer N - 2, and then N - 3, and so on back to the input layer. (Though backprop nets usually had a grand total of 3 layers.) Then you just nudged the whole network a delta - that is, nudged each weight or bias by delta times its partial derivative with respect to the output error.
It says a lot about the nonobvious difficulty of doing math that it took years to come up with this algorithm.
I find it difficult to put into words just how obvious this is in retrospect. You're just taking a system whose behavior is a differentiable function of continuous paramaters, and sliding the whole thing down the slope of the error function. There are much more clever ways to train neural nets, taking into account more than the first derivative, e.g. conjugate gradient optimization, and these take some effort to understand even if you know calculus. But backpropagation is ridiculously simple. Take the network, take the partial derivative of the error function with respect to each weight in the network, slide it down the slope.
If I didn't know the history of connectionism, and I didn't know scientific history in general - if I had needed to guess without benefit of hindsight how long it ought to take to go from Perceptrons to backpropagation - then I would probably say something like: "Maybe a couple of hours? Lower bound, five minutes - upper bound, three days."
"Seventeen years" would have floored me.
I am starting to get something from these posts.
I feel confused about the notion that people only want to donate to a thing if they will be on the hook for needing to donate every year forevermore to keep it afloat, as opposed to donating to cause it to get its business in order and then it can sustain itself.
He just learned that keeping secrets is bad in general, and so he doesn’t by default, unless explicitly agrees to.
This is not true! My policy is simply that you should not assume that I will promise to keep your secrets after you tell me, if you didn't check with me first.
I can confirm; Oliver keeps many secrets from me, that he has agreed to others, and often keeps information secret based on implicit communication (i.e. nobody explicitly said that it was secret, but his confident read of the situation is that it was communicated with that assumption). I sometimes find this frustrating because I want to know things that Oliver knows :P
1And he sent the message in a way that somehow implied that I was already supposed to have signed up for that policy, as if it's the most normal thing in the world, and with no sense that this is a costly request to make (or that it was even worth making a request at all, and that it would be fine to prosecute someone for violating this even if it had never been clarified at all as an expectation from the other side).
Speaking generally, many parties get involved in zero-sum resource conflicts, and sometimes form political alliances to fight for their group to win zero-sum resource conflicts. For instance, if Alice and Bob are competing to get the same job, or Alice is trying to buy a car for a low price and Bob is trying to sell it to her for a high price, then if Charlie is Alice's ally, she might hope that Charlie all will take actions that help her get more/all of the resources in these conflicts.
Allies of this sort also expect that they can share information that is easy to use adversarially against them between each other, with the expectation it will be consistently used either neutrally or in their favor by the allies.
Now, figuring out who your allies are is not a simple process. There are no forms involved, there are no written agreements, it can be fluid, and picked up in political contexts by implicit signals. Sometimes you can misread it. You can think someone is allied, tell them something sensitive, then realize you tricked yourself and just gave sensitive information to someone. (The opposite error also occurs, where you don't realize someone is your ally and don't share info and don't pick up all the value on the table.)
My read here is that Mikhail told Habryka some sensitive information about some third party "Jackson", assuming that Habryka and Jackson were allied. Habryka, who was not allied with Jackson in this way, was simply given a scoop, and felt free to share/use that info in ways that would cause problems for Jackson. Mikhail said that Habryka should treat it as though they were allies, whereas Habryka felt that he didn't deserve it and that Mikhail was saying "If I thought you would only use information in Jackson's favor when telling you the info, then you are obligated to only use information in Jackson's favor when using the info." Habryka's response is "Uh, no, you just screwed up."
(Also, after finding out who "Jackson" is from private comms with Mikhail, I am pretty confused why Mikhail thought this, as I think Habryka has a pretty negative view of Jackson. Seems to me simply like a screw-up on Mikhail's part.)
Second one seems reasonable.
Clarifying in the first case: If Bob signs up and DMs 20 users, and one reports spam, are you saying that we can only check his DM, or that at this time we can then check a few others (if we wish to)?
1
Love the shout out; I will repeat myself once more, it’s important to distinguish between WordPress (the open-source software) and WordPress.com (the commercial hosting service run by Automattic). Automattic was founded by Matt Mullenweg, who co-founded the open-source WordPress project, and the company continues to contribute to WordPress, but they’re separate entities.