While reading a collection of Tom Wayman's poetry, suddenly a poem came to me about Hal Finney ("Dying Outside"); since we're contributing poems, I don't feel quite so self-conscious. Here goes:

He will die outside, he says.
Flawed flesh betrayed him,
it has divorced him -
for the brain was left him,
but not the silverware
nor the limbs nor the car.
So he will take up
a brazen hussy,
tomorrow's eve,
a breather-for-him,
a pisser-for-him.
He will be letters,
delivered slowly;
deliberation
his future watch-word.
He would not leave until he left this world.
I try not to see his mobile flesh,
how it will sag into eternal rest,
but what he will see:
symbol and symbol, in their endless braids,
and with them, spread over strange seas of thought
mind (not body), forever voyaging.

http://www.gwern.net/fiction/Dying%20Outside

I wrote this poem yesterday in an unusual mood. I don't entirely agree with it today. Or at least, I would qualify it.

What is meant by computation? When I wrote that intelligence is not computation, I must have meant a certain sort of computation because of course all thought is some kind of computation.

To what extent has distinction been made between systematic/linear/deductive thought (which I am criticizing as obviously limited in the poem) and intelligent pattern-based thought? Has there been any progress in characterizing the latter?

For example, consider the canonical story about Gauss. To keep him busy with a computation, his math teacher told him to add all the numbers from 1 to 100. Instead, according to the story, Gauss added the first number and the last number, multiplied by 100 and divided by 2. Obviously, this is a computation. But yet a different sort. To what extent do you suppose he logically deduced the pattern of the lowest number and highest number combining always to single value or just guessed/observed it was a pattern that might work? And then found that it did work inductively?

I'm very interested in characterizing the difference between these kinds of computation. Intelligent thinking seems to really be guesses followed by verification, not steady linear deduction.

No, that's not the case. A one-boxer in Newcomb's problems is acting with entirely correct beliefs. All agree that the one-boxer will get more money than the two-boxer. That correct belief is what motivates the one-boxer.

The scenarios that you describe sound somewhat (but not exactly) like Gettier problems to me.

(I wasn't the downvoter.)

I am completely baffled why this would be downvoted. I guess asking a question in genuine pursuit of knowledge, in an open thread, is wasting someone's time, or is offensive.

I like to think someone didn't have the time to write "No, that's not the case," and wished, before dashing off, leaving nothing but a silhouette of dust, that their cryptic, curt signal would be received as intended; that as they hurry down the underground tunnel past red, rotating spotlights, they hoped against hope that their seed of truth landed in fertile ground -- godspeed, downvote.

Whether or not a singleton is the best outcome or not, I'm way too uncomfortable with the idea to be singing songs of praise about it.

I strongly suspect that lesswrong.com has an ideological bias in favor of "morality."

It does.

There has been no proof that rationality requires morality. Yet I suspect that posts coming from a position of moral nihilism would not be welcomed.

In general they are not. But I find that a high quality clearly rational reply that doesn't adopt the politically correct morality will hover around 0 instead of (say) 8. You can then post a couple of quotes to get your karma fix if desired.

Gur qrngu qvqa'g srry irel qrngu-yvxr. Vg frrzrq yvxr gur rzbgvba fheebhaqvat vg jnf xvaq bs pbzcerffrq vagb bar yvggyr ahttrg gung V cerggl zhpu fxvzzrq bire. V jnf nyfb rkcrpgvat n jbeyq jvgubhg qrngu, juvpu yrsg zr fhecevfrq. Va gur erny jbeyq, qrngu vf bsgra n fhecevfr, ohg yvxr va gur erny jbeyq, fhqqra qrngu va svpgvba yrnirf hf jvgu n srryvat bs qvforyvrs. Lbh pbhyq unir orra uvagvat ng gung qrngu sebz gur svefg yvar.

Nyfb, lbh xvaq bs qevir evtug cnfg gur cneg nobhg birecbchyngvba. V guvax birecbchyngvba vf zl zbgure'f znva bowrpgvba gb pelbavpf.

I enjoyed it. I made an effort to read Key genderlessly. This didn't work at first, probably because I found the Spivak pronouns quite hard to get used to, and "ey" came out as quite male to me, then fairly suddenly flipped to female somewhere around the point where ey was playing on the swing with Trellis. I think this may have been because Trellis came out a little more strongly male to me by comparison (although I was also making a conscious effort to read ey genderlessly). But as the story wore on, I improved at getting rid of the gender and by the end I no longer felt sure of either Key or Trellis.

Point of criticism: I didn't find the shift between what was (to me) rather obviously the two halves of the story very smooth. The narrative form appeared to take a big step backwards from Key after the words "haze of flour" and never quite get back into eir shoes. Perhaps that was intentional, because there's obviously a huge mood shift, but it left me somewhat dissatisfied about the resolution of the story. I felt as though I still didn't know what had actually happened to the original Key character.

You traded off a lot of readability for the device of making the protagonist's gender indeterminate. Was this intended to serve some literary purpose that I'm missing? On the whole the story didn't seem to be about gender.

I also have to second DanArmak's comment that if there was an overall point, I'm missing that also.

Looks like I'm in the minority for reading Key as slightly male. I didn't get a gender for Trellis. I also read the librarian as female, which I'm kind of sad about.

I loved the story, found it very touching, and would like to know more about the world it's in. One thing that confused me: the librarian's comments to Key suggested that some actual information was withheld from even the highest levels available to "civilians". So has someone discovered immortality, but some ruling council is keeping it hidden? Or is it just that they're blocking research into it, but not hiding any actual information? Are they hiding the very idea of it? And what's the librarian really up to?

Were you inspired by Nick Bostrom's "Fable of the Dragon"? It also reminded me a little of Lois Lowry's "The Giver".

Thanks so much for sharing it with us!

For me, both of the characters appeared female.

Sbe zr gur fgbel fbeg bs oebxr qbja whfg nf Xrl'f sevraq jnf xvyyrq. Vg frrzrq gbb fbba vagb gur aneengvir gb znxr fhpu n znwbe punatr. Nyfb, jvgu erfcrpg gb gur zbeny, vg frrzrq vafhssvpvragyl fubja gung lbh ernyyl ertneq cnva nf haqrfvenoyr - vg frrzrq nf gubhtu lbh pbhyq or fnlvat fbzrguvat nybat gur yvarf bs "gurl whfg qba'g haqrefgnaq." Orpnhfr bs gung nf jryy nf gur ehfurq srry bs gur raqvat, vg fbeg bs pnzr bss nf yrff rzbgvbanyyl rssrpgvir guna vg pbhyq.

I love the new gloss on "What do you want to be when you grow up?"

or switch to a different genderless pronoun system.

Don't. Spivak is easy to remember because it's just they/them/their with the ths lopped off. Nonstandard pronouns are difficult enough already without trying to get people to remember sie and hir.

I liked it. :)

Part of the problem that I had, though, was the believability of the kids: kids don't really talk like that: "which was kind of not helpful in the not confusing me department, so anyway"... or, in an emotionally painful situation:

Key looked suspiciously at the librarian. "You sound like you're trying not to say something."

Improbably astute, followed by not seeming to get the kind of obvious moral of the story. At times it felt like it was trying to be a story for older kids, and at other times like it was for adults.

The gender issue didn't seem to add anything to the story, but it only bothered me at the beginning of the story. Then I got used to it. (But if it doesn't add to the story, and takes getting used to... perhaps it shouldn't be there.)

Anyway, I enjoyed it, and thought it was a solid draft.

Finally got around reading the story. I liked it, and finishing it gave me a wild version of that "whoa" reaction you get when you've doing something emotionally immersive and then switch to some entirely different activity.

I read Key as mostly genderless, possibly a bit female because the name sounded feminine to me. Trellis, maybe slightly male, though that may also have been from me afterwards reading the comments about Trellis feeling slightly male and those contaminating the memory.

I do have to admit that the genderless pronouns were a bit distracting. I think it was the very fact that they were shortened version of "real" pronouns that felt so distracting - my mind kept assuming that it had misread them and tried to reread. In contrast, I never had an issue with Egan's use of ve / ver / vis / vis / verself.

I got used to the Spivak after a while, and while it'd be optimal for an audience used to it, it did detract a little at first. On the whole I'd say it's necessary though (if you were going to use a gender'd pronoun, I'd use female ones)

I read Key as mainly female, and Trellis as more male- it would be interesting to know how readers' perceptions correlated with their own gender.

The children seemed a little mature, but I thought they'd had a lot better education, or genetic enhancement or something. I think spending a few more sentences on the important events would be good though- otherwise one can simply miss them.

I think you were right to just hint at the backstory- guessing is always fun, and my impression of the world was very similar to that which you gave in one of the comments.

Great story!

I kept thinking of Key as female. This may be because I saw some comments here that saw em as female, or because I know that you're female.

The other character I didn't assign a sex to.

I enjoyed the story-- it was an interesting world. By the end of the story, you were preaching to a choir I'm in.

None of the characters seemed strongly gendered to me.

I was expecting opposition to anesthesia to include religiously based opposition to anesthesia for childbirth, and for the whole idea of religion to come as a shock. On the other hand, this might be cliched thinking on my part. Do they have religion?

The neuro couldn't be limited to considered reactions-- what about the very useful fast reflexive reaction to pain?

Your other two story links didn't open.

This made me say "Awwwwwwww..."

Did LW play a part, or was she just browsing the Internet?

Thanks for posting this (I didn't know the videos were up), though you've posted it in the December 2009 open thread.

(The current open thread is in the Discussion section, though this may be worth its own Discussion post.)

Awesome, thanks ^_^

Huh. Looking at that wiki drove home how much I ReallyHate CamelCase.

I'd say go ahead and post a link to this - if they've got any especially good sample techniques, be sure to include a pointer there.

If I recall correctly this request is already on the implementation queue.

I think that kind of a question is fine in the Open Thread.

More generally: LW is about developing human rationality, but is it appropriate for questions about already solved aspects of rationality?

Most posts here are written by someone who understands an aspect of rationality, to explain it to those who don't. I see no reason not to ask questions in the open thread. I think they should be top-level posts only if you anticipate a productive discussion around them; most already-solved questions can be answered with a single comment and that would be that, so no need for a separate post.

Well, if we modified what we wanted, we wouldn't get what we originally wanted because we wouldn't want to...

Can't think of anything else off the top of my head.

tut's quip holds the key:

But in general the answer is another question: Why would you want to do that?

But to expound on it a bit further, if I want to drive to Dallas to see a band play I can (a) figure out a strategy to get there or (b) stop wanting to go. Assuming that (b) is even possible, it isn't actually a solution to the problem of how to get to Dallas. Applying the same principle to all Wants does not provide you with a way to always get what you want. Instead, it helps you avoid not getting what you want.

If you wanted nothing more than to avoid disappointment by not getting what you want than the safest route is to never desire anything that isn't a sure thing. Or simply not want anything at all. But a simpler route to this whole process is to ditch that particular Want first. The summation is a bit wordy and annoying, but it ends like this:

Instead of wanting to avoid not getting what you want by not wanting anything else, simply forgo the want that is pushing you to avoid not getting what you want.

With parens:

Instead of (wanting to avoid {not getting what you want} by not wanting anything else), simply forgo (the want that is pushing you to avoid {not getting what you want}).

In other words, you can achieve the same result that modifying your wants would create by not getting too disappointed if you don't get what you want. Or, don't take it so hard when things don't go your way.

Hopefully that made some sense (and I got it right.)

Why optimize our strategies to achieve what we happen to want, instead of just modifying what we want?

For some things it is probably wise to change your desires to something you can actually do. But in general the answer is another question: Why would you want to do that?

One thing that has worked for me lately is the following: whenever I do something and don't really know why I did it (or am uncomfortable with the validity of my rationalizations), I try and think of the action in Outside View terms. I think of (or better, write out) a short external description of what I did, in its most basic form, and its probable consequences. Then I ask what goal this action looks optimized for; it's usually something pretty simple, but which I might not be happy consciously acknowledging (more selfish than usual, etc).

That being said, even more helpful than this has been discussing my actions with a fairly rational friend who has my permission to analyze it and hypothesize freely. When they come up with a hypothesis that I don't like, but which I have no good counterarguments against, we've usually hit paydirt.

well, clearly something has gone horribly wrong with your brain, and you should figure out the problem and fix it.

I don't think of this as something wrong with my brain, so much as it functioning properly in maintaining a conscious/unconscious firewall, even though this isn't as adaptive in today's world as it once was. It's really helped me in introspection to not judge myself, to not get angry with my revealed preferences.

People who watch me talking about myself sometimes say I'm good at introspection, but I think about half of what I do is making up superstitions so I have something doable to trick myself into making some other thing, previously undoable, doable. ("Clearly, the only reason I haven't written my paper is that I haven't had a glass of hot chocolate, when I'm cold and thirsty and want refined sugar." Then I go get a cup of cocoa. Then I write my paper. I have to wrap up the need for cocoa in a fair amount of pseudoscience for this to work.) This is very effective at mood maintenance for me - I was on antidepressants and in therapy for a decade as a child, and quit both cold turkey in favor of methods like this and am fine - but I don't know which (if, heck, any) of my conclusions that I come to this way are "really true" (that is, if the hot chocolate is a placebo or not). They're just things that pop into my head when I think about what my brain might need from me before it will give back in the form of behaving itself.

You have to take care of you brain for it to be able to take care of you. If it won't tell you what it wants, you have to guess. (Or have your iron levels checked :P)

I tend to think of my brain as a thing with certain needs. Companionship, recognition, physical contact, novelty, etc. Activities that provide these tend to persist. Figure out what your dysfunctional actions provide you in terms of your needs. Then try and find activities that provide these but aren't so bad and try and replace the dysfunctional bits. Also change the situation you are in so that the dysfunctional default actions don't automatically trigger.

My dream is to find a group of like minded people that I can socialise and work with. SIAI is very tempting in that regard.

Does anyone here think they're particularly good at introspection or modeling themselves, or have a method for training up these skills? It seems like it would be really useful to understand more about the true causes of my behavior, so I can figure out what conditions lead to me being good and what conditions lead to me behaving poorly, and then deliberately set up good conditions. But whenever I try to analyze my behavior, I just hit a brick wall---it all just feels like I chose to do what I did out of my magical free will. Which doesn't explain anything.

My suggestion is focussing your introspection on working out what you really want. That is, keep investigating what you really want until such time as the phrase 'me behaving poorly' and 'being good' sound like something that is in a foreign language, that you can understand only by translating.

You may be thinking "clearly something has gone horribly wrong with my brain" but your brain is thinking "Something is clearly wrong with my consciousness. It is trying to make me do all this crazy shit. Like the sort of stuff we're supposed to pretend we want because that is what people 'Should' want. Consciousnesses are the kind of things that go around believing in God and sexual fidelity. That's why I'm in charge, not him. But now he's thinking he's clever and is going to find ways to manipulate me into compliance. F@#@ that s#!$. Who does he think he is?"

When trying to work effectively with people empathy is critical. You need to be able to understand what they want and be able to work with each other for mutual benefit. Use the same principle with yourself. Once your brain believes you actually know what it (ie. you) want and are on approximately the same page it may well start trusting you and not feel obliged to thwart your influence. Then you can find a compromise that allows you to get that 'simple thing' you want without your instincts feeling that some other priority has been threatened.

If someone thought we lived in an internally consistent simulation that is undetectable and inescapable, is it even worth discussing?

Not particularly.

Wouldn't the practical implications of such a simulation imply the same things as the material world/reality/whatever you call it?

Absolutely.

Would it matter if we dropped "undetectable" from the proposed simulation? At what point would it begin to matter?

The point where you find a way to hack it, escape from the simulation and take over (their) world.

Firefox (or maybe one of the million extensions that I've downloaded and forgotten about) has a feature where, if you mouseover a link, the URL linked to will appear in the lower bar of the window. A different color would be easier, though.

What are the implications to FAI theory of Robin's claim that most of what we do is really status-seeking? If an FAI were to try to extract or extrapolate our values, would it mostly end up with "status" as the answer and see our detailed interests, such as charity or curiosity about decision theory, as mere instrumental values?

If it went that far it would also go the next step. It would end up with "getting laid".

I think it's kinda like inclusive genetic fitness: It's the reason you do things, but you're (usually) not conciously striving for an increased amount of it. So I don't think it could be called a terminal value, as such...

An independent piece of evidence moves the log-odds a constant additive amount regardless of the prior. Averaging log-odds amounts to moving 2/3 of that distance if 2/3 of the people have the particular piece of evidence. It may behave badly if the evidence is not independent, but if all you have are posteriors, I think it's the best you can do.

Fascinating. As a child, I also detested praise, and I have always had something bordering on an obsession for symmetry and an aversion to asymmetry.

I hadn't heard of the Thue-Morse sequence until now, but it is quite similar to a sequence I came up with as a child and have tapped out (0 for left hand/foot/leg, 1 for right hand/foot/leg) or silently hummed (or just thought) whenever I was bored or was nervous.

My sequence is:

[0, 1, 1, 0] [1001, 0110, 0110, 1001] [0110 1001 1001 0110, 1001 0110 0110 1001, 1001 0110 0110 1001, 0110 1001 1001 0110] ...

[commas and brackets added to make the pattern obvious]

As a kid, I would routinely get the pattern up into the thousands as I passed the time imagining sounds or lights very quickly going off on either the left (0) or right (1) side.

Incidentally, my childhood dislike of asymmetry led me to invent the Thue-Morse sequence

I love this community.

I spent much of my childhood obsessing over symmetry. At one point I wanted to be a millionaire solely so I could buy a mansion, because I had never seen a symmetrical suburban house.

Just thought I'd mention this: as a child, I detested praise.

++MeToo;

Your first example sounds related to the broken window theory, but I've never seen a name for the underlying bias. (The broken window fallacy is something else altogether.)

It seems like a type of apathy.

Bee-sting theory of poverty is the closest I've heard. You're right, this is real and deserves a name, but I don't know what it would be.

This seems like a special case of the more general "just one can't hurt" (whatever the current level) way of thinking. I don't know any name for this but I guess you could call it something like the "non-Archimedean bias"?

"Lost Cause Fallacy" or the "Fallacy of Futility" [...] Does this have a standard name

"Sunk cost fallacy"

Informally, "throwing good money after bad"? I agree that this is a real and interesting phenomenon.

actual number that you can plug into Baye's rule

Bayes' rule.

I found Rob Zhara's comment helpful.

Yes, there is a proof.

http://lesswrong.com/lw/s0/where_recursive_justification_hits_bottom/ljr

Try Solomonoff Induction

Not only is there no proof, there isn't even any evidence for it. Any effort to collect evidence for it leaves you assuming what you're trying to prove. This is the "problem of induction" and there is no solution; however, since you are built to be incapable of not applying induction and you couldn't possibly make any decisions without it.

Occam's razor is dependent on a descriptive language / complexity metric (so there are multiple flavours of the razor).

Unless a complexity metric is specified, the first question seems rather vague.

I'm confused. Have you never seen long-term goal directed behavior?

So, I've been thinking about this for some time now, and here's what I've got:

First, the point here is to self-reflect to want what you really want. This presumably converges to some specific set of first degree desires for each one of us. However, now I'm a bit lost on what do we call "values", are they the set of first degree desires we have(not?), set of first degree desires we would reach after infinity of self-reflection, or set of first degree desires we know we want to have at any given time?

As far as I can tell, akrasia would be a subproblem of this.

So, this should be about right. However, I think it's weird that here people talk a bit about akrasia, and how to achieve those n-degree desires, but I haven't seen anything about actually reflecting and updating what you want. Seems to me that people trust a tiny bit too much to the power of cognitive dissonance fixing the problem between wanting to want and actually wanting, this resulting to the lack of actual desire in achieving what you know you should want(akrasia).

I really dunno how to overcome this, but this gap seems worth discussing.

Also, since we need eternity of self-reflection to reach what we really want, this looks kinda bad for FAI: Figuring out where our self-reflection would converge in infinity seems pretty much impossible to compute, and so, we're left with compromises that can and probably will eventually lead to something we really don't want.

The problem with the specific scenario given, with experimental modification/duplication, rather than careful proof based modification, is that is liable to have the same problem that we have with creating systems this way. The copies might not do what the agent that created them want.

Which could lead to a splintering of the AI, and in-fighting over computational resources.

It also makes the standard assumptions that AI will be implemented on and stable on the von Neumann style computing architecture.

Let me point out that we (humanity) does actually have some experience with this scenario. Right now, mobile code that spreads across a network without effective controls on the bounds of its expansion by the author is worms. If we have experience, we should mine it for concrete predictions and countermeasures.

General techniques against worms might include: isolated networks, host diversity, rate-limiting, and traffic anomaly detection.

Are these low-cost/high-return existential reduction techniques?

Congratulations! I guess people will believe everything you say now.

Congratulations!

Why not post an introduction to your thesis research on LW?

With a doctorate in ...?

The conversion techniques page is fascinating. I'll put this to use good in further spreading the word of Bayes.

d) should be changed to the sparseness of intelligent aliens and limits to how fast even a superintelligence can extend its sphere of influence.

Restructuring since the fact that the universe hasn't been noticeably paperclipped can't possible be considered evidence for (c).

The universe has either been paperclipped (1) or it hasn't been (2).

If (1):

(A) we have observed paperclipping and not realized it (someone was really into stars, galaxies and dark matter)

(B) Our universe is the result of paperclipping (theists were right, sort of)

(C) Superintelligences tend not to optimize things that are astronomically visible.

If (2)

(D) Super-intelligences are impossible.

(E) Quantum immortality true.

(F) No intelligent aliens.

(G) Some variety of simulation hypothesis is true.

(H) Galactic aliens exist but have never constructed a super-intelligence do to a well enforced prohibition on AI construction/research, an evolved deficiency in thinking about minds as a physical object (substance dualism is far more difficult for them to avoid than it is for us), or some other reason that we can't fathom.

(I) Friendliness is easy + Alien ethics doesn't include any values that lead to us noticing them.

You acquired a second-order desire, which is a preference about preferences.

Tyler Cowen offering a reason why the CRU hacked emails should raise our confidence in AGW...

That's not what he is saying. His argument is not that the hacked emails actually should raise our confidence in AGW. His argument is that there is a possible scenario under which this should happen, and the probability that this scenario is true is not infinitesimal. The alternative possibility - that the scientists really are smearing the opposition with no good reason - is far more likely, and thus the net effect on our posteriors is to reduce them - or at least keep them the same if you agree with Robin Hanson.

Here's (part of) what Tyler actually said:

Another response, not entirely out of the ballpark, is: 2. "These people behaved dishonorably. They must have thought this issue was really important, worth risking their scientific reputations for. I will revise upward my estimate of the seriousness of the problem."

I am not saying that #2 is correct, I am only saying that #2 deserves more than p = 0. Yet I have not seen anyone raise the possibility of #2.

The second link doesn't load; should be this.

I have some advice.

1. Pay attention to the timing of your edit/compile/test cycle time. Efforts to get this shorter pay off both in more iterations and in your personal motivation (interacting with a more-responsive system is more rewarding). Definitely try to get it under a minute.

2. A good dataset is incredibly valuable. When starting to attack a problem - both the whole thing, and subproblems that will arise - build a dataset first. This would be necessary if you are doing any machine learning, but it is still incredibly helpful even if you personally are doing the learning.

3. Succeed "instantaneously" - and don't break it. Make getting to "victory" - a complete entry - your first priority and aim to be done with it in a day or a week. Often, there's temptation to do a lot of "foundational" work before getting something complete working, or a "big refactoring" that will break lots of things for a while. Do something (continuous integration or nightly build-and-test) to make sure that you're not breaking it.

Great! That competition looks like a lot of fun, and I wish you the best of luck with it.

As for advice, perhaps the best I can give you is to explain the characteristics the winning program will have.

It will make no, or minimal, use of game tree search. It will make no, or minimal, use of machine learning (at best it will do something like tuning a handful of scalar parameters with a support vector machine). It will use pathfinding, but not full pathfinding; corners will be cut to save CPU time. It will not know the rules of the game. Its programmer will probably not know the exact rules either, just an approximation discovered by trial and error. In short, it will not contain very much AI.

One reason for this is that it will not be running on a supercomputer, or even on serious commercial hardware; it will have to run in real time on a dinky beige box PC with no more than a handful of CPU cores and a few gigabytes of RAM. Even more importantly, only a year of calendar time is allowed. That is barely enough time for nontrivial development. It is not really enough time for nontrivial research, let alone research and development.

In short, you have to decide whether your priority is Starcraft or AI. I think it should be the latter, because that's what has actual value at the end of the day, but it's a choice you have to make. You just need to understand that the reward from the latter choice will be in long-term utility, not in winning this competition.