All of nerzhin's Comments + Replies

Have you tried this? Does it work?

A: worked on me. I thought, "Okay, I realized I don't know that much about my religion. What's the deal?" So during church I'd actually read the bible. It didn't take long.

B: Well, I once was on a discussion board that was primarily for evangelical Christians. In the natural course of discussion, I mentioned the tribe of Benjamin. You know, the one with clearly-God-sanctioned mass murder and rape.

Some of them came up with some pretzel logic justifications. The rest of them backed away quietly.

Incidents like that were a big contributing factor to why the site was shut down.

I don't understand much of this, and I want to, so let me start by asking basic questions in a much simpler setting.

We are playing Conway's game of life with some given initial state. An disciple AI is given a 5 by 5 region of the board and allowed to manipulate its entries arbitrarily - information leaves that region according to the usual rules for the game.

The master AI decides on some algorithm for the disciple AI to execute. Then it runs the simulation with and without the disciple AI. The results can be compared directly - by, for example, count... (read more)

2JenniferRM
First: by the holographic principle the meaningful things to pay attention to are the boundary cells. Second... this is cool. Did you invent this AI research paradigm just now off the top of your head, or have you already seen research where software was given arbitrary control of part of the board and given board-wide manipulation goals. If the latter, could you give me a research key word to drop into google scholar, or maybe a URL?
0Stuart_Armstrong
The game of life is interesting, because it's not reversible. It would then be possible to design an AI that does something (brings happiness to a small child or whatever) such that in a million iterations, the board is exactly as it would have been had the AI not existed. But yes, counting the squares different might work in theory, though it might be too chaotic to be much use in practice. In our world, we use 'chaos' to get non-reversiblity, and coarse graining to measure the deviation.
0skepsci
Exactly. If you have determinism in the sense of a function from AI action to result world, you can directly compute some measure of the difference between worlds X and X', where X is the result of AI inaction, and X' is the result of some candidate AI action. As nerzhin points out, you can run into similar problems even in deterministic universes, including life, if the AI doesn't have perfect knowledge about the initial configuration or laws of the universe, or if the AI cares about differences between configurations that are so far into the future they are beyond the AI's ability to calculate. In this case, the universe might be deterministic, but the AI must reason in probabilities.
0endoself
A direct measurement of the number of squares that are different isn't very informative, especially due to chaos theory. The issue of determinism is less important, I think. Unless the AI has unlimited computing power and knowledge of the entire state of the board, it will have to use probabilities to understand the world.

The first requirement:

Even as chicks, geese cannot be handled by a human, or encounter other geese who have been.

suggests that a FAI would not tell us that it exists. In other words, the singularity may already have happened.

7TheOtherDave
Agreed with the first part of that. The second part I'm less sure of. It may already have happened iff the world as it is is consistent with what we'd expect such an entity to create, which is precisely what I'm wondering about. My instinct is to say "nonsense; it could certainly create a world where irrevocable suffering and death aren't quite so common, without violating that constraint"... a point that the Fun Theory Sequence also makes at some length, IIRC. But on thinking about it I'm not at all sure that it could without spoiling the scenario altogether. For example: even in the world as it is, we are remarkably willing as a species to behave as though we were being taken care of by an all-powerful supernatural force; it might be that any significant further reduction in our suffering that wasn't visibly our own doing would bring us over a tipping point where that willingness became a literally irresistable temptation. Dunno. It seems as though it ought to be possible to make more specific predictions based on that speculation, perhaps even falsifiable ones, but thus far I've failed to get traction.

You can pay someone $8/hr to do menial tasks 20 hrs/ week, for a total of about $8000 / year.

With payroll taxes and insurance, I would expect this to cost at least $12000 a year.

3daenerys
Good point! I would still say it is worth it, though.

Sure, but you could have a limit on how many rounds back they remember, or you could fill in the history with some rule.

Or just prohibit the bots from knowing which round they are playing.

3JoshuaZ
If they are allowed to know the history of the bot they are playing against (that is, what it has done to them) then they have to know how many rounds they've played.

It is not at all clear that the people resistant to addictive drugs are reproducing at a higher rate than those who aren't.

-12[anonymous]

I think the sub-proposal is too complex and involves too many trivial inconveniences. I up-voted the original proposal.

0lessdazed
Thank you for the feedback.

almost every researcher in CS flaunts copyright, posting their papers on their own websites

Many journals explicitly allow you to distribute a "preprint" of your journal articles on your personal website. For example, the Elsevier policy states that authors retain:

the right to post a pre-print version of the journal article on Internet websites including electronic pre-print servers, and to retain indefinitely such version on such servers or sites for scholarly purposes

Another way of saying this (I think - Vladimir_M can correct me):

You only have two choices. You can be the kind of person who kills the fat mat in order to save four other lives and kills the fat man in order to get a million dollars for yourself. Or you can be the kind of person who refuses to kill the fat man in both situations. Because of human hardware, those are your only choices.

4lessdazed
Spreading this meme, even by a believing virtue ethicist, would seem to reduce the lifespan of fat men with bounties on their heads much faster than it would spare the crowds tied to the train tracks. U: "Ooo look, a way to rationalize killing for profit!" VE: "No no no, the message is that you shouldn't kill the fat man in either ca-" U: "Shush you!" Of course, one may want to simply be the sort who tells the truth, consequences to fat men be damned.
3Bongo
This seems obviously false.

I don't mean to imply that the kind of person who would kill the fat man would also kill for profit. The only observation that's necessary for my argument is that killing the fat man -- by which I mean actually doing so, not merely saying you'd do so -- indicates that the decision algorithms in your brain are sufficiently remote from the human standard that you can no longer be trusted to behave in normal, cooperative, and non-dangerous ways. (Which is then correctly perceived by others when they consider you scary.)

Now, to be more precise, there are actu... (read more)

There are three things you could want:

  1. You could want the extra dollar. ($6 instead of $5)

  2. You could want to feel like someone who care about others.

  3. You could genuinely care about others.

The point of the research in the post, if I understand it, is that (many) people want 1 and 2, and often the best way to get both those things is to be ignorant of the actual effects of your behavior. In my view a rationalist should decide either that they want 1 (throwing 2 and 3 out the window) or that they want 3 (forgetting 1). Either way you can know the truth and still win.

3atucker
The problem with strategic ignorance is if the situation is something like 6/1 vs. 5/1000. Most people care more about themselves than others, but I think that at that level most people would just choose to lose a dollar and give 999 more. If you choose to not learn something, then you don't know what you're causing to happen, even if it would entirely change what you would want to do.

Here is a presentation that was used in a similar setting before.

I recommend trying to cover less than you currently plan. Just one or two big ideas should be more than enough.

0falenas108
Thanks, that link will probably be incredibly useful.

Donating to VillageReach signals philanthropic intention and affords networking opportunities with other people who care about global welfare who might be persuaded to work against x-risk

Also, donating to VillageReach saves people's lives, and those people will have agency and abilities and may very well contribute to existential risk reduction.

0ArisKatsaris
But they may also contribute to existential risk increase. What sort of calculation have you made that makes you think these people are more likely to contribute to existential risk increase? I don't think I've seen any reasonable argument that can be made that simply having more random people around will help deal with existential risk. Most likely existential risks (UFAI, grey goo, bioterrorism, whatever) will be caused by people afterall.
1jhuffman
Should we also work to boost birth rates in all areas of the world? Because we are working against that goal in some key ways. It is hard to control all the variables but there is very convincing evidence that modernization affects birth rates in developing countries in a number of ways. Including influences of cost of children, productivity of children, and education of women.
2jsteinhardt
They will still come from a very poorly educated area of the world. I think the effect is overall a little unclear (it might stabilize that area of the world somewhat, which would have positive spillover for everyone else; or the population increase will spark additional conflict that has negative spillover).

How did this go over with your advisor? (Serious question.)

1mwengler
He was a tenured professor that ran large observatories. He was always more interested in what I could do technically than in whether I approved of him or not. I don't think he paid any attention to my comment about the whale, although he did give me good comments on my thesis. I did include quotes from "Moby Dick" at the tops of some of the chapters, including "...I hold that the Whale is a fish [not an animal]..." whcih I had at the top of a chapter which was the quantum theory of the SIS device I was working with. So there were whale quotes in there for him :)

Nowadays, however, the class system has become far harsher and the distribution of status much more skewed. The better-off classes view those beneath them with frightful scorn and contempt, and the underclass has been dehumanized to a degree barely precedented in human history.

How do you measure this kind of thing? Do you have a citation?

5Dustin
I too would be interested in sources for this assertion. It goes contrary to what I would say if I were asked to guess about classes of today compared to classes of fifty years ago. edit: Oops, I should have refreshed page before commenting, as I now see Vladimir_M responded. Leaving comment for content about my state of mind on this issue.
0Vladimir_M
No, it's a conclusion from common sense and observation, though I could find all kinds of easily cited corroboration. Unfortunately, as I said, a more detailed analysis of these trends and their components and causes would get us far into various controversial and politicized topics, which are probably best left alone here. I stated these opinions only because they seemed pertinent to the topic of the original post and the subsequent comments, i.e. the reasons for broad dissatisfaction with life in today's developed world, and their specific relation to the issues of work.

I'm not sure I believe you. By "non-wage costs and risks" do you mean things like health benefits, or lawsuit liability, or what? I can think of a lot of productive uses for cheap labor.

There's a bunch of trash and graffiti in my city. There's lots of unemployed people whose labor cleaning it up would be worth, say, a euro an hour.

Your (funny) comment made me realize that ubiquitous smartphones and the right software might actually make something like a karma system for in-person conversations possible.

If you have to pick one of the above ideas as most useful, which would it be?

1Morendil
Number one (explicit "end of message" signals) is really interesting, because it changes the dynamics of conversation completely: for the listeners, because they are not allowed to interrupt; for the speaker, because they must form complete thoughts more deliberately. Some people comically ramble for some time and then peter out. Now it falls on some gentle soul to give them a polite reminder. (Suggested: "is that all?" - but really wait until it's clear that was all.) But when that has happened to you once, you pay more attention to the next time. If there is a moderator in the conversation, this convention makes things much easier for them, giving them natural occasions to redirect the flow of conversation and to balance "air time" between people who ramble and those who tend to stick to the point. It may be possible to combine this convention with the OP's suggestion, e.g. a hand signal with the previously and explicitly agreed meaning of "I think you are rambling, or otherwise not adding much".

Parties do this. Religions do this. Universities do this. Parents do this. Lovers do this. Why should we be any different?

Um. Because we want to be different from political parties and religions?

2Raw_Power
How about the other two? And we don't just want to be different, we want to change them. Rationalists should win, even if our winning conditions might not be what our adversaries expect.

I'm pretty sure we've talked about similar things before - the closest thing I could find after a quick search is a list of math prerequisites.

A suggestion: to make this concrete, we could identify specific courses on MIT's open coarse project. Then people could actually "get" this degree in some sense.

I'll start with the obvious: Math 18.05, Introduction to Probability and Statistics.

9Zetetic
It seems obvious that street fighting mathematics would be highly appropriate.

What do you study at Stanford? Why?

6gwern
From his blog:

This looks very much like an ultimatum game, with Player A playing the proposer (how much of her $500,000 will she share?) and Player B as the responder (his role is to either accept or decline).

0Psychohistorian
As has been pointed out, it is not, because there is continuous interaction, bargaining, and perhaps the potential for pre-commitments, though as I mentioned, those could be risky.

I'm not sure it's controversial, but I disagree very slightly on the margin. All your points are good. However, if

  1. I have already read some papers from an author, and

  2. I trust that their abstracts are honest representations of their work, and

  3. I am not relying on their work as a basis for my own, but just pointing it out to my readers

I will cite it after just reading the abstract.

A perfectly clear, logical, honest, and readable account of your work is often ipso facto unpublishable: what is required is writing according to unofficial, tacitly acknowledged rules that are extremely hard to figure out on your own.

This has not been my experience. My experience with journal editors and reviewers has been that they want a clear and readable account, but it probably varies a great deal from field to field.

Your point about brand names and networks, however, is very well taken.

6Vladimir_M
In many fields -- but not all, as you note, and it's hard to speculate on the exact proportion -- there's an evil arms race in fundamentally dishonest self-promotion. Basically, you must employ every imaginable spin short of outright lying to blow up your contribution out of all proportion and minimize the perceived shortcomings of your work. If instead you write up a complete, straightforward, and honest account that will leave the reader informed as accurately as possible, there's no way you're getting published unless it's a very extraordinary breakthrough. Of course, even in such circumstances, you want your paper to be clear and readable in the sense that the reviewers will read it easily and end up convinced by your claims and impressed by its high-status qualities, without too may unpleasant questions occurring to them. But this is mostly about Dark Arts, not real clarity of exposition.

Because you don't want to pad your bibliography and give the impression you know more about the topic than you really do. Also, the body may not match the abstract, the body may poorly substantiate its claims in the abstract, you should understand any document you're relying on to make your point, etc.

Why is this controversial?

Edit: In fairness, some people do intend "efficient scholarship" to mean "cite any paper with an abstract that looks like it agrees with you and hope no one asks questions", but I don't think that's what lukeprog means.

This is interesting because my initial response is to disagree, but I don't think I have good reasons or evidence.

To drastically oversimplify: You seem to be saying that intelligence is primary and social skills are learned. You're born smart or dumb, and if you're smart, you over-analyze social situations and become afraid.

My initial reaction is the opposite: Social skills are primary, intelligence is learned. You are born with or without good social skills, and if you don't have them, you read a lot (by yourself) and hack computers or whatever, so that you become smart.

0[anonymous]
.

Why? Does this attract alumni donations? Prospective students? Why exactly do you have to project the university website image?

1Manfred
I'd guess increasing your apparent value to prospective students and their parents is the main value of having shiny fluff on your front page. But I may not be separating the fluff-ass-marketing from the fluff-as-cultural-norm : there is an expectation of what a university page looks like, and people are uncomfortable when their expectations are violated too radically. It's possible that there's no intrinsic value to sticking a slideshow with pretty pictures and links to press releases on your website, but because of our current expectations it just looks bad not to have something like that.

There are several colleges that do this, calling it the block plan. The ones I know of are Cornell College, Colorado College, and Quest University.

People are fundamentally unsolvable to me

This might be your point, but the above statement is probably not true.

Not to say it's easy to begin learning to solve people, or even that it's worth it. But it's probably possible.

1a363
I hope it's true in the sense that I won't one day start thinking that I somehow understand ("grok") humanity and know what it means to be human (or just a sentient being) in a general sense. In the specific sense, individual people are not that mysterious in their behaviour most of the time. But their motivations can be hard to understand from their own point of view. I guess it's mostly because I can't be bothered to find out...

Can I ask how you find "random blogs"? Is it truly random, or do you have a method for finding new stuff?

A medium-level exercise would be to find such an example, in some news article perhaps, quote it, and ask what is going on.

Chain prediction mechanisms to obtain faster feedback

I suspect that with each link in the chain, you get a much less accurate and stable result.

2Strange7
Yes, but if the cost reduction and speed improvement is enough it can still be worthwhile. "A good answer now is better than a perfect answer next week" and so forth. It doesn't take a doctorate in physics to know that an initially stationary, unsupported rock less than ten meters above the earth's surface will hit that surface within the next second or two. For that matter, consider some sort of craftsmanship where masters traditionally employ apprentices. The apprentice can't do master-level work (yet) but is still in training, and can improve the master's performance by performing necessary menial tasks and allowing the boss to concentrate on things that demand mastery.

Overt is the key word.

When you buy a car that's cheaper than a Volvo, or drive over the speed limit, or build a house that cannot withstand a magnitude 9 earthquake, you are making a life and death decision.

2drethelin
no. The phrase "life or death decision" does not mean this and this is not how it's used.

I don't really know anything about your situation, your wife, your relationship. So please don't take anything I say very seriously. Desrtopa may be right, and I certainly didn't want to imply that you weren't already a good husband.

I'm really glad to hear you're in marriage counseling. That will be more helpful than anything I say.

As far as not trying to change her: you've got lots of time. If she gets thirsty, she'll let you know. What I'm advising against is trying to deconvert her so that you feel better, which is what I read (rightly or wrongly) in the line I quoted in the grandparent.

0jwhendy
No problem, and I didn't take your statement as at all implying that I wasn't a good husband. I did say: That is how I see things, but would not say that this fact means I'm actively pursuing bringing this outcome to pass. I do generally leave all of this alone. It's come to the surface more lately due to discussions about children, but most of the time we just leave it be and that seems to help us do as you suggested -- rebuild around other common interests, activities, and the like. Also, even though me feeling better would be a byproduct, I only want that to be a byproduct. That is, I'd very much like her to come to her own understanding of what I now see, not that she would deconvert specifically for my feelings. That's a good reminder.

I think my emotional satisfaction would dramatically increase if she were to deconvert.

This is hard for her, too.

Your roles and responsibilities to your wife are entirely different from the responsibility you've described to your own conscience to be true and follow the evidence and so on. The strategies we're discussing on this thread, though interesting and maybe useful, are probably not things you want to use with your wife, who already knows you well and knows the story.

My advice is pretty much the opposite of Desrtopa's. Don't talk about your que... (read more)

0Desrtopa
Do you think he's not being a good husband now? I'm not advising him (not trying to advise him anyway) to become more confrontational or put more pressure on his wife and friends. But I think he needs social support from people who accept him, and in order to feel accepted, he needs to have people who think, if not that he's right in not believing, at least that he's doing something respectable and intellectually honest. jwhendy knows best what he wants out of a relationship, and if he decides that he's comfortable sweeping disagreements under the rug and simply not talking about what he believes anymore, then that may be the way for him to be happiest. But it sounds to me like he's not satisfied with that, and if he wants to be accepted while remaining open and honest, he needs to be able to influence how others think.
1jwhendy
I'm well aware. It absolutely is. I agree, and tend to abide by that advice. I think when I provide rantish outbursts trying to justify myself, it's usually because: * The discussion of our kids comes up and she thinks she has more of a right to raise them to believe Catholicism, citing as her primary reason that it's really important to her. That's quite challenging and usually leads me to want to stand up for myself regarding the amount of work and research I've put into this and how I think that counts as a valid reason that I have an equal say as well. * When I feel proselytized. She brings up leading topics, in my opinion. She came back from a retreat and told the valiant story of a man who doubted but said to himself, "Well, if there's a heaven, even though I doubt, I'm going to spend my entire life trying to believe anyway so that I can go." Given the pertinence to my own story and the fact that she shared nothing else about the retreat other than that, my bet was on her trying to defend my stance and why that logic isn't sound (which belief is the right one to get into heaven?). (She verified later that it was, in fact, a conversion-directed comment.) * When I feel challenged about my process, like if she poo-poo's what I've read, chalks it all up to bias, or something similar. I guess I could go on a little bit, but just wanted to cite some of the items that have a tendency to draw out the "defensive me." Other than that, I can say with near certainty that our best times have always correlated with our longer-ish periods of just not talking about religion/my quest at all. I had this thought several months ago, actually. I realized that I tend to talk pretty openly about anything and everything -- what I read, things I find interesting, work, etc., but that she doesn't always do this and that I missed "knowing her." We also began marriage counseling and I stated as one of my goals at our first session that I think finding new "common ground" will be

There was a very interesting discussion of exactly this question (at least as it relates to the mathematics community) on mathoverflow recently.

This is a much more useful way to think about faith than just thinking of it as somehow the opposite of reason.

0[anonymous]
Faith is belief that unconditionally prevents understanding views inconsistent with it. A simple, illuminating definition, rendering faith a defined variety of viral meme. For the mechanism, based on psychologist Daniel T. Gilbert's "Spinozan" theory of comprehension and belief, see "Unraveling the mystery of morality: The unity of comprehension and belief explains moralism and faith"

A couple nitpicks: The author of the book of Hebrews is not known, and this book is not normally attributed to Paul even in Christian tradition.

Also, your second quote appears to be from the New Revised Standard Version, not the New International Version that you cite.

0PhilGoetz
Hebrews is "sort of" attributed to Paul in Christian tradition (it doesn't say it's written by Paul, but it's very Paul-like); but this authorship is controversial. I wasn't aware of how controversial it was. I'll revise my reference. New Revised Standard Version: "Now faith is the assurance of things hoped for, the conviction of things not seen."

Many teachers, in my experience, don't notice when students are confused or bored -- or maybe they notice but don't care

Or they notice, and care to some extent, but have other things to worry about. Like a pressure to cover a certain amount of material, or a fear of boring one group of students while they slow down for another, or a (maybe partly justified) belief that the students who are confused and bored just aren't trying.

They are ideas for allaying fears that SIAI is incompetent or worse. Which, since it is devoted to building an AI, would tend to allay fears that it is building an evil one.

9Eliezer Yudkowsky
Basically incompetent organizations that try to build AI just won't do anything.

Impostor syndrome is pretty common among men too, in my experience. It may still be more common in women, but I'm not sure.

2Vladimir_M
Yes, this should definitely be taken into account. In fact, given the present state of many fields, I'm sure that for many people in academia the "impostor syndrome" is just a true realization that their work is worthless.

I can accomplish a great many things, especially unfamiliar things, only when there is no help available.

This sounds like my experience, at least some of the time. I am male, for what it's worth.

Looking at your list of backgrounds, the missing thing that jumps out at me is discrete math. You might also want to think about learning some differential equations, if it wasn't included in your calculus sequence.

0Perplexed
Really? If you were to take the four top-selling Macroeconomics textbooks for undergrad Econ majors, or the four top-selling Macroeconomics textbooks for economics graduate schools, those books would be presenting different models? That would surprise me, though I have to admit, I haven't looked at a modern econ textbook in thirty years.
2Barry_Cotter
Hell Yes! Macroeconomics is either a construction site or a graveyard

Don't trust anyone but a mathematician

I'm pretty sure you're at least half-joking. But just in case, I need to point out that mathematicians are not immune to this kind of thing.

2[anonymous]
yep, joke.

There is an enormous (far too enormous for its value to the world, in my opinion) literature on the unexpected hanging paradox (also known as the surprise exam paradox) in the philosophy and mathematics literature. The best treatments are:

Timothy Y. Chow, The surprise examination or unexpected hanging paradox, American Mathematical Monthly 105 (1998) pp. 41-51. (ungated)

Elliot Sober, To give a surprise exam, use game theory, Synthese 115 (1998) pp. 355-373. (ungated)

3SilasBarta
The paradox actually has practical implications. It shows a general mechanism by which you can "surprise" someone despite a predictable outcome. It goes like this: 1) You tell someone they will be "surprised" by an upcoming event (e.g., what gift you will buy them). 2) They start to suspect it will be one of a number of unusual outcomes. 3) The event actually has its regular, boring, predictable outcome. 4) But the other person is still surprised, since they did not expect the boring outcome (when before your statement, they did)! I know of a major case where this reasoning was applied: on one season of the TV show The Apprentice. (The show where people try to get a job with Donald Trump and one person is eliminated from consideration ["fired"] each week.) During the second episode/competition, one contestant walked out due to frustration, and she didn't come back until evaluation time. Then, in ads for the next episode, they said, "Next time, on The Apprentice, one candidate will quit the competition -- and you'll never guess who it is!" This, of course, prompted speculation that someone other than the last episode's quitter would be the one to quit ... but no, it was the same woman who left, this time permanently, rather than being fired. Well, it was certainly a suprise by that point!
2cousin_it
Yeah, came here to say the same. Thanks. When I tried to solve this problem about 10 years ago, I came up with the equivalent of Fitch's "Goedelized" solution, described on pages 5-6 of Chow's article. I'm still not sure why many people consider it wrong; it seems to utterly dissolve the "paradox" for me.

I think I disagree, but I'm not sure what elitism means here.

Elitism might help prevent this error. But can it lead to other errors?

5komponisto
Yes. For any method of correcting an error, there's always a possibility of overcorrecting.

Can you give examples, either of emotion driven behaviors becoming dark arts when raised to awareness, or of dark arts being necessary to healthy interaction? I think we are using different definitions of what dark arts are.

Load More