I've posted on LW before, but I posted again here after a long hiatus because of recent AI news, and entirely unaware of the good heart thing; then made several comments after reading the original post, but thinking it was a joke. Now I understand why the site was so strangely active.
"An animal looking curiously in the mirror, but the reflection is a different kind of animal; in digital style."
"A cat looking curiously in the mirror, but the reflection is a different kind of animal; in digital style."
"A cat looking curiously in the mirror, but the reflection is a dog; in digital style."
Curious to see how it handles modified-reflection and lack-of-specificity.
Another thing whose True Name is probably a key ingredient for alignment (and which I've spent a lot of time trying to think rigorously about): collective values.
Which is interesting, because most of what we know so far about collective values is that, for naive definitions of "collective" and "values", they don't exist. Condorcet, Arrow, Gibbard and Satterthwaite, and (crucially) Sen have all helped show that.
I personally don't think that means that the only useful things one can say about "collective values" are negative results like the ones above. I th...
I think this post makes sense given the premises/arguments that I think many people here accept: that AG(S)I is either amazingly good or amazingly bad, and that getting the good outcome is a priori vastly improbable, and that the work needed to close the gap between that prior and a good posterior is not being done nearly fast enough.
I don't reject those premises/arguments out of hand, but I definitely don't think they're nearly as solid as I think many here do. In my opinion, the variance in goodness of reasonably-thinkable post-AGSI futures is mind-boggl...
Sure, humans are effectively ruthless in wiping out individual ant colonies. We've even wiped out more than a few entire species of ant. But our ruthfulness about our ultimate goals — well, I guess it's not exactly ruthfulness that I'm talking about...
...The fact that it's not in our nature to simply define an easy-to-evaluate utility function and then optimize, means that it's not mere coincidence that we don't want anything radical enough to imply the elimination of all ant-kind. In fact, I'm pretty sure that for a large majority of people, there's no ut...
I guess we're using different definitions of "friendly/unfriendly" here. I mean something like "ruthlessly friendly/unfriendly" in the sense that humans (neurotic as they are) aren't. (Yes, some humans appear ruthless, but that's just because their "ruths" happen not to apply. They're still not effectively optimizing for future world-states, only for present feels.)
I think many of the arguments about friendly/unfriendly AI, at least in the earlier stages of that idea (I'm not up on all the latest) are implicitly relying on that "ruthless" definition of (un...
Why does the AI even "want" failure mode 3? If it's a RL agent, it's not "motivated to maximize its reward", it's "motivated to use generalized cognitive patterns that in its training runs would have marginally maximized its reward". Failure mode 3 is the peak of an entirely separate mountain than the one RL is climbing, and I think a well-designed box setup can (more-or-less "provably") prevent any cross-peak bridges in the form of cognitive strategies that undermine this.
That is to say: yes, it can (or at least, it it's not provable that it can't) ...
One way of dividing up the options is: fix the current platform, or find new platform(s). The natural decay process seems to be tilting towards the latter, but there are downsides: the diaspora loses cohesion, and while the new platforms obviously offer some things the current one doesn't, they are worse than the current one in various ways (it's really hard to be an occasional lurker on FB or tumblr, especially if you are more interested in the discussion than the "OP").
If the consensus is to fix the current platform, I suggest trying the simple...
I disagree. I think the issue is whether "pro-liberty" is the best descriptive term in this context. Does it point to the key difference between things it describes and things it doesn't? Does it avoid unnecessary and controversial leaps of abstraction? Are there no other terms which all discussants would recognize as valid, if not ideal? No, no, and no.
Whether something is a defensible position, and whether it should be embedded in the very terms you use when more-neutral terms are available, are separate questions.
If you say "I'm pro-liberty", and somebody else says "no you're not, and I think we could have a better discussion if you used more specific terms", you don't get to say "why won't you accept me at face value".
When you say "Nothing short of X can get you to Y", the strong implication is that it's a safe bet that X will at least not move you away from Y, and sometimes move you toward it. So OK, I'll rephrase:
The OP suggests that colonization is in fact a proven way to turn at least some poor countries into more productive ones.
Note that my post just above was basically an off-the-cuff response to what I felt was a ludicrously wrong assumption buried in the OP. I'm not an expert on African history, and I could be wrong. I think that I gave the OP's idea about the level of refutation it deserved, but I should have qualified my statements more ("I'd guess..."), so I certainly didn't deserve 5 upvotes for this (5 points currently; I deserve 1-3 at most).
I think that it's worth being more explicit in your critique here.
The OP suggests that colonization is in fact a proven way to turn poor countries into productive ones. But in fact, it does the opposite. Several parts of Africa were at or above average productivity before colonization¹, and well below after; and this pattern has happened at varied enough places and times to be considered a general rule. The examples of successful transitions from poor countries to rich ones—such as South Korea—do not involve colonization.
¹Note that I'm considering the tria...
I think you can make this critique more pointed. That is: "pro-liberty" is flag-waving rhetoric which makes us all stupider.
I dislike the "politics is a mind-killer" idea if it means we can't talk about politically touchy subjects. But I entirely agree with it if it means that we should be careful to keep our language as concrete and precise as possible when we approach these subjects. I could write several paragraphs about all the ways that the term "pro-liberty" takes us in the wrong direction, but I expect that most of you can figure all that out for yourselves.
It appears that you need to be logged in from FB or twitter to be fully non-guest. That seems like a... strange... choice for an anti-akrasia tool.
(Tangentially related to above, not really a reply)
Fair enough. Thanks. Again, I agree with some of your points. I like blemish-picking as long as it doesn't require open-ended back-and-forth.
You're raising some valid questions, but I can't respond to all of them. Or rather, I could respond (granting some of your arguments, refining some, and disputing some), but I don't know if it's worth it. Do you have an underlying point to make, or are you just looking for quibbles? If it's the latter, I still thank you for responding (it's always gratifying to see people care about issues that I think are important, even if they disagree); but I think I'll disengage, because I expect that whatever response I give would have its own blemishes for you to find.
In other words: OK, so what?
Full direct democracy is a bad idea because it's incredibly inefficient (and thus also boring/annoying, and also subject to manipulation by people willing to exploit others' boredom/annoyance). This has little or nothing to do with whether people's preferences correlate with their utilities, which is the question I was focused on. In essence, this isn't a true Goldilocks situation ("you want just the right amount of heat") but rather a simple tradeoff ("you want good decisions, but don't want to spend all your time making them").
As to t...
(small note: the sentence you quote from me was unclear. "because" related to "presume", not "saying". But your response to what I accidentally said is still largely cogent in relation to what I meant to say, so the miscommunication isn't important. Still, I've corrected the original. Future readers: lumifer quoted me correctly.)
The model is not easy to subject to full, end-to-end testing. It seems reasonable to test it one part at a time. I'm doing the best I can to do so:
I've run an experiment on Amazon Mechanical Turk involving hundreds of experimental subjects voting in dozens of simulated elections to probe my strategy model.
I'm working on getting survey data and developing statistical tools to refine my statistical model (mostly, posterior predictive checks; but it's not easy, given that this is a deeper hierarchical model than most).
In terms of the utilitarian assumpt
I presume you're saying that utility-based simulations are not credible. I don't think you're actually trying to say that they're not numerical estimates. So let me explain what I'm talking about, then say what parts I'm claiming are "credible".
I'm talking about monte-carlo simulations of voter satisfaction efficiency. You use some statistical model to generate thousands of electorates (that is, voters with numeric utilities for candidates); a media model to give the voters information about each other; and a strategy model to turn information, u...
[ ] Wow, these people are smart. [ ] Wow, these people are dumb. [ ] Wow, these people are freaky. [ ] That's a good way of putting it, I'll remember that.
(For me, it's all of the above. "Insight porn" is probably the biggest, but it doesn't dominate.)
Electology is an organization dedicated to improving collective decision making — that is, voting. We run on a shoestring; somewhere in the lowish 5 digits $ per year. We've helped get organizations such as the German Pirate Party and the various US stat Libertarian Parties to use approval voting, and gotten bills brought up in several states (no major victories so far, but we're just starting.)
Is a better voting system worth it, even if most people still vote irrationally? I'd say emphatically yes. Plurality voting is just a disaster as a system, filled w...
In terms of “saving throws” one can buy for a humanity that may be navigating tricky situations in an unknown future, improvements to thinking skill seem to be one of the strongest and most robust.
Improvements to collective decision making seem to be potentially an even bigger win. I mean, voting reform; the kind of thing advocated by Electology. Disclaimer: I'm a board member.
Why do I think that? Individual human decisionmaking has already been optimized by evolution. Sure, that optimization doesn't fit perfectly with a modern need for rationality, but...
One idea for measurement in a randomized trial:
In order to apply, you have to list 4 people who would definitely know how awesome you're being a year from now, and give their contact info. Then, choose 1 of those people 6 months later and 1 person a year later and ask them how awesome the person is being. When you ask, include a "rubric" of various stories of various awesomeness levels, in which the highest levels are not always just $$$ but sometimes are. Ask the people you're asking to please not contact the person specifically to check awesome...
I think you've misunderstood the question. As I understand it, it's not "is the distribution of startup values a power law" but "do startups distribute their profits to employees according to a power law".
Wish I could both up- and down- vote this comment. +1 for interesting, cogent observation; -1 for followinng that up with facile beakering. So instead I upvoted this comment and downvoted your reply below ( which deserves the downvote in its own right)
(I just made up the word "beakering". It means doing TV science, with beakers and bafflegab, in real life. A lot of amateur evo-something and neuro-something involve beakering.)
Would be better if you didn't say whom you ended up agreeing with. Most people here have either a halo or horns on Eliezer, and discounting that is distracting.
That's simpler to say, but not at all simpler to do.
Bump.
(I realize you're busy, this is just a friendly reminder.)
Also, I added one clause to my comment above: the bit about "imperfectly measured", which is of course usually the case in the real world.
Great article overall. Regression to the mean is a key fact of statistics, and far too few people incorporate it into their intuition.
But there's a key misunderstanding in the second-to-last graph (the one with the drawn-in blue and red "outcome" and "factor"). The black line, indicating a correlation of 1, corresponds to nothing in reality. The true correlation is the line from the vertical tangent point at the right (marked) to the vertical tangent point at the left (unmarked). If causality indeed runs from "factor" (height)...
No argument here. It's hard to build a good social welfare function in theory (ie, even if you can assume away information limitations), and harder in practice (with people actively manipulating it). My point was that it is a mistake to think that Arrow showed it was impossible.
(Also: I appreciate the "thank you", but it would feel more sincere if it came with an upvote.)
I think you've done better than CarlShulman and V_V at expressing what I see as the most fundamental problem with EA: the fact that it is biased towards the easily- and short-term- measurable, while (it seems to me) the most effective interventions are often neither.
In other words: how do you avoid the pathologies of No Child Left Behind, where "reform" becomes synonymous with optimizing to a flawed (and ultimately, costly) metric?
This issue is touched by the original post, but not at all deeply.
Note: Arrow's Impossibility Theorem is not actually a serious philosophical hurdle for a utilitarian (though related issues such as the Gibbard-Satterthwaite theorem may be). That is to say: it is absolutely trivial to create a social utility function which meets all of Arrow's "impossible" criteria, if you simply allow cardinal instead of just ordinal utility. (Arrow's theorem is based on a restriction to ordinal cases.)
Upvoted because I think this is a real issue, though I'm far from sure whether I'd put it at "worst".
... And that is not a new idea either. "Allow me to play the devil's advocate for a moment" is a thing people say even when they are expressing support before and after that moment.
Can anyone explain why the parent was downvoted? I don't get it. I hope there's a better reason than the formatting fail.
This is a key question. The general answer is:
For realistic cases, there is no such theorem, and so the task of choosing a good system is a lot about choosing one which doesn't reward strategy in realistic cases.
Roughly speaking, my educated intuition is that strategic payoffs grow insofar as you know that the distinctions you care about are orthogonal to what the average/modal/median voter cares about. So insofar as you are average/modal/median, your strategic incentive should be low; which is a way of saying that a good voting system can have low str
Yup. That's what people say. I don't know what the general rule is, but it's definitely right for this case.
I, too, hope that our disagreement will soon disappear. But as far as I can see, it's clearly not a semantic disagreement; one of us is just wrong. I'd say it's you.
So. Say there are 3 voters, and without loss of generality, voter 1 prefers A>B>C. Now, for every one of the 21 distinct combinations for the other two, you have to write down who wins, and I will find either an (a priori, determinative; not mirror) dictator or a non-IIA scenario.
ABC ABC: A
ABC ACB: A
ABC BAC: ?... you fill in these here
ABC BCA: ?
ABC CAB: .
ABC CBA: .
ACB ACB: .
ACB BAC:
ACB BC...
I'm sorry, you really are wrong here. You can't make up just one scenario and its result and say that you have a voting rule; a rule must give results for all possible scenarios. And once you do, you'll realize that the only ones which pass both unanimity and IIA are the ones with an a priori dictatorship. I'm not going to rewrite Arrow's whole paper here but that's really what he proved.
Under Arrow's terms, this still counts as a dictator, as long as the other ballots have no effect. (Not "no net effect", but no effect at all.)
In other words: if I voted for myself, and everyone else voted for Kanye, and my ballot happened to get chosen, then I would win, despite being 1 vote against 100 million.
It may not be the traditional definition of dictatorship, but it sure ain't democracy.
Again, you're simply not understanding the theorem. If a system fails non-dictatorship, that really does mean that there is an a priori dictator. That could be that one vote is chosen by lot after the ballots are in, or it could be that everybody (or just some special group or person) knows beforehand that Mary's vote will decide it. But it's not that Mary just happens to turn out to be the pivotal voter between a sea of red on one side and blue on the other.
I realize that this is counterintuitive. Do you think I have to be clearer about it in the post?
Wait until I get to explaining SODA; a voting system where you can vote for one and still get better results.
As for comparing different societies: there are of course societies with different electoral systems, and I think some systems do tend to lead to better governance than in the US/UK, but the evidence is weak and VERY confounded. It's certainly impossible to clearly demonstrate a causal effect; and would be, even assuming such an effect existed and were sizeable. I will talk about this more as I finish this post.
Thanks, I'll work on that.
Your probability theory here is flawed. The question is not about P(A&B), the probability that both are true, but about P(A|B), the probability that A is true given that B is true. If A is "has cancer" and B is "cancer test is positive", then we calculate P(A|B) as P(B|A)P(A)/P(B); that is, if there's a 1/1000 chance of cancer and and the test is right 99/100, then P(A|B) is .99.001/(.001.99+.999.01) which is about 1 in 10.
I'll certainly have more content that addresses these questions as the post develops. For now, I'll simply respond to your misunderstanding about Arrow. The problem is not that there will always be an a posteori pivotal voter, but that (to satisfy the other criteria) there must be an a priori dictator. In other words, you would get the same election result by literally throwing away all ballots but one without ever looking at them. This is clearly not democracy.
This is still in-progress, and I'm going to get to some of that later. Here's my defense of the current summary:
It's easy, but not helpful, to use "postmodern" as a shorthand for "bad ideas" of some kind. Something like Sturgeon's law ("90% of everything is crap") applies to postmodernism as to everything else, and I'd even agree that it's a kind of thinking that is more likely than average to come unmoored from reality, but that doesn't mean that it's barren of all insight. Especially today, at least 20 years after its heydey, and considering that even in its heyday it was a very rare academic department indeed where drinking the kool-...
Is "do whatever action you predict to maximize the electricity in this particular piece of wire" really "general"? You're basically claiming that the more intelligent someone is, the more likely they are to wirehead. With humans, in my experience, and for a loose definition of "wirehead", the pattern seems to be the opposite; and that seems to me to be solid enough in terms of how RL works that I doubt it's worth the work to dig deep enough to resolve our disagreement here.