You're looking at Less Wrong's discussion board. This includes all posts, including those that haven't been promoted to the front page yet. For more information, see About Less Wrong.
Comment author:MathiasZaman
03 November 2013 01:09:28AM
0 points
[-]
I can't speak for anything else, but I've read up to the Meta-ethics sequence without encountering any gaps. I can't vouch for anything after that, but the pdf seems complete. Maybe someone else can shed a light on your question.
Comment author:[deleted]
04 November 2013 08:53:39AM
*
0 points
[-]
I think the two rationality books were supposed to be complete rewrites, and I think this is separate from the sequence ebooks (not confident, sort of confused)
Comment author:savageorange
02 November 2013 07:48:51PM
*
0 points
[-]
-- Time less ;)
-- this question feels like it's missing a word or two. What does time-preference mean?
EDIT: Thanks, Arundelo. So basically, time preference ~= level of short-sighted optimization.
In that case, do some projects that strictly require long-sighted optimization. A deadline is one good tool; Telling others what you're doing (in an unequivocal way, so that the greatest disappointment/irritation/harassment is achieved). Of course these tools are nothing new, the point is to increase the pressure as high as you can stand and reduce the amount of 'slack' time you have to allocate to a minimum.
On a more meta level, you can try things like doing some mindfulness meditation every day, which I personally find makes it easier to ignore irrelevant stimuli, worry less, and stick to my priorities.
An even more general observation: Introverts typically have lower time preference relative to extraverts, so ask them about how they dispel distraction. (I say this in the sense described by Dorothy Rowe: 'extraverts are basically worried about belonging and feel understimulated, introverts are basically worried about keeping control of themselves and feel overstimulated' , and not the vague 'Extraverts are social, introverts are not, derp' that seems to be the misapprehension of the average person)
In case there's any question, I'm an extravert, so yeah, I tend to struggle with this issue too.
In economics, time preference (or "discounting") is the relative
valuation placed on a good at an earlier date compared with its valuation at
a later date
[...] Someone with a high time preference is focused substantially on his
well-being in the present and the immediate future relative to the average
person, while someone with low time preference places more emphasis than
average on their well-being in the further future.
Comment author:hyporational
03 November 2013 06:08:51AM
*
3 points
[-]
Telling others what you're doing
I read somewhere, might have been on lw, that telling what you're doing might decrease your chance of success, because it provides a way to get compliments without actually having achieved anything yet. I suppose this depends on how you do it, though.
Introverts typically have lower time preference relative to extraverts, so ask them about how they dispel distraction.
I'm an introvert, have terrible problems with time-preference, and don't understand the rationalization by Dorothy Rowe you provide.
I read somewhere, might have been on lw, that telling what you're doing might decrease your chance of success, because it provides a way to get compliments without actually having achieved anything yet. I suppose this depends on how you do it, though.
I know this TED-talk says similar things. It's where I first heard of that concept.
Comment author:savageorange
03 November 2013 10:32:32AM
0 points
[-]
I definitely agree that you can't -just- tell a person what you're doing, you need to pick the right person, and cultivate the right attitude (From my observation of myself, it succeeds when I am in the mindset where I can take plenty of teasing equitably, accepting any pokes as potential observations about reality without -stressing- about that.).
..
What rationalization of Rowe's? It's a summary of what they themselves report when 'laddered' (a process which basically consists of asking them what the most terrible thing that could possibly happen to them is, followed by iterative 'why?' until they can no longer go to the next lowest level).
For extroverts, being utterly abandoned == total personal disintegration; For introverts, utter loss of self-control == total personal disintegration.
(I do paraphrase here; read The Successful Self for the whole picture.)
If anything, any rationalization is mine: I observe that introverts I know are reliably better at moving long term projects forward than I, or any extravert I know, seems to be. Not that they are not weak in this way -- they just seem to be less weak as a consequence of the difference in their focus. (my inference bolded.)
I'm neutral to your statement of introversion, basically because my prior for people being hilariously terrible at assessing this stuff is quite high.
No empirical sources as far as I know. Nobody even really manages to agree on the definition of introvert and extrovert, so far. Dorothy Rowe is just the only writer I've found on the subject who manages to describe a system that is relateable, consistent, and can be applied in the real world.
Comment author:VincentYu
03 November 2013 05:28:31PM
*
3 points
[-]
Telling others what you're doing
I read somewhere, might have been on lw, that telling what you're doing might decrease your chance of success, because it provides a way to get compliments without actually having achieved anything yet. I suppose this depends on how you do it, though.
Gollwitzer et al. (2009). When intentions go public: Does social reality widen the intention-behavior gap?
Abstract (emphasis mine):
Based on Lewinian goal theory in general and self-completion theory in particular, four experiments examined the implications of other people taking notice of one's identity-related behavioral intentions (e.g., the intention to read law periodicals regularly to reach the identity goal of becoming a lawyer). Identity-related behavioral intentions that had been noticed by other people were translated into action less intensively than those that had been ignored (Studies 1–3). This effect was evident in the field (persistent striving over 1 week's time; Study 1) and in the laboratory (jumping on opportunities to act; Studies 2 and 3), and it held among participants with strong but not weak commitment to the identity goal (Study 3). Study 4 showed, in addition, that when other people take notice of an individual's identity-related behavioral intention, this gives the individual a premature sense of possessing the aspired-to identity.
Comment author:[deleted]
03 November 2013 01:01:45AM
11 points
[-]
Read about hyperbolic discounting, if you haven't already.
Assuming a conflict between short- and long-term decisions, the general advice is to mentally bundle a given short-term decision with all similar decisions that will occur in the future. For example, you might think of an unhealthy snack tonight as "representing" the decision to eat an unhealthy snack every night.
Comment author:hyporational
03 November 2013 05:55:39AM
*
6 points
[-]
Optimize your environment for decreased time-preference when you have the most control:
Fill your refrigerator when you're not hungry. Apply effortful-to-dismantle restrictions on your computer when you're not bored and tired. Walk to the university library to study so it takes effort to come back home to your hobbies.
I'd like to read and collect other similar strategies for my toolbox.
ETA: I just realized I do this for exercise too. There's a lake near my house with a circumference of about 6 kilometers, and I go jogging around it frequently. I have a strong desire to quit once I've gotten to the other side, but I have no choice but to run the whole route at that point. Sometimes I decide to walk the other half, but I guess it's better than nothing. Another option would be to just run in one direction and then back, but I find the idea too boring, even if I change my route a bit.
Comment author:passive_fist
02 November 2013 07:49:34PM
2 points
[-]
Here's a more difficult version of the AI box experiment. I haven't seen this particular version anywhere, but I'd be pleased to be proven wrong.
Imagine we've come up with a very intelligent AI that is free to manipulate the environment and uses the action-reward system like Hutter's AIXI. Also imagine that we've somehow figured a way to make the rewards very hard to counterfeit (perhaps we require the rewards to be cryptographically signed). It's clear that in such a system, the 'weak point' would be the people in control of the private key. In this case the AI will not attempt to modify its own reward system (to see why, look at Hutter's AIXI book, where he discusses this in some detail).
How could such an AI convince someone to hand over the encryption key? Note that it can't promise things like e.g. ending human suffering, because it already has the means to do that (it is 'free') as well as the incentive (obtaining reward).
Comment author:Adele_L
02 November 2013 07:58:03PM
5 points
[-]
How could such an AI convince someone to hand over the encryption key? Note that it can't promise things like e.g. ending human suffering, because it already has the means to do that (it is 'free') as well as the incentive (obtaining reward).
Torture is probably the easiest way. Another way could be to examine the key-keeper's mind and extract the key directly from that, but this is needlessly complicated.
Comment author:passive_fist
02 November 2013 08:42:14PM
0 points
[-]
I suspect the answer to be more complex than this. The AI knows that if it attempted something like that there is the very huge risk of being cut off from all reward, or even having negative reward administered. In other words: tit for tat. If it tries to torture, it will itself be tortured. Remember that before it has the private key, we are in control.
Torture might stand too great a chance of destroying the encryption key. Though I suppose if nanotech were sufficiently difficult to obtain, the possible key-destructive effects of torture might be balanced against the probability of a car running over the keyholder in the meantime.
I would think that confusion (set things up so the key-keeper is confused and distracted, then do some phishing) is in the same reliability range as torture, and less likely to get the AI in trouble.
Comment author:ChristianKl
03 November 2013 07:54:27AM
2 points
[-]
The first step would start by starting to control information flow to the people who hold the key. Only tell the person about all the good thing the AI does, and hide evidence of any wrongdoing that might reduce the rewards that the AI gets.
The second step is to break the security which gets used to protect the key. Install a keylogger.
Comment author:solipsist
05 November 2013 03:27:44AM
*
1 point
[-]
I don't understand how this encryption would work. What do people physically do to reward the AI, and how do you ensure that only people can do that? Would humans compute RSA signatures in their head? Would humans typing reusable passwords onto a "secure" reward computer that is "outside the AI's control"? Do humans precompute and memorize a finite number of one-time reward phrases before the AI is turned on, and reward the AI by uttering a phrase aloud?
In the precomputed, one-time cookie case, I'd just make the human think about the reward phrase. I'm sure humans leak thoughts like a sieve through subvocalization, nerve impulses, etc.
Comment author:passive_fist
05 November 2013 04:10:05AM
0 points
[-]
What I had in mind was the reward being administered through a consensus cryptography system, perhaps via some elected board or somesuch, but I really didn't give that aspect of the problem much thought. If the key is distributed, the AI would have to extract it from each individual holding a part of it.
This in itself is an interesting problem imo, and if a good solution is found it might have important implications for FAI research.
Comment author:solipsist
05 November 2013 06:07:38AM
*
3 points
[-]
It's clear that in such a system, the 'weak point' would be the people in control of the private key.
If the AI is out of the box, I don't think humans are the weak point.
Humans physically do something when they reward the AI. To get a reward, the AI has only to figure out what the humans would physically do and mimic that itself. If the human reward the AI by pressing a big red button, then the AI can just kill the human and press the big red button itself. It wouldn't matter if the big red button uses 512 bit elliptic curve cryptography -- the AI just has to find a paperweight and put it on the button.
If humans can perform RSA encryption silently in their heads, then you might be on to something. A human could memorize a private key and produce a cryptographically signed reward for the AI when the human deemed the AI worthy. The AI would not know the private key, would not be able to produce signed rewards, and would not be able to mimic humans. This setup works because it is cryptographically difficult to mimic a human doing RSA. But if the human did not perform the cryptography silently in their head, it would not be cryptographically difficult to mimic their rewarding behavior.
But I doubt that humans could perform RSA or elliptic curve cryptography mentally. Unless humans can compute trapdoor functions silently in their head, I don't see how public key cryptography could buy you anything over a reward-button.
Comment author:passive_fist
05 November 2013 07:32:10AM
*
0 points
[-]
For the same reason that Linus Torvalds is not able to hack every single Linux system in existence, it is reasonable to assume that the probability of 'backdoor'-type attacks could be reduced or eliminated.
If you're not talking about a back door, then I'm not sure what you're trying to say here. Are you implying that the AI will find the 'reward producing machine' and somehow use it to produce rewards? It doesn't work that way because the machine would still need the key to produce a cryptographically-valid reward, and the key would not be stored on the machine. That's the whole point of using cryptography. For instance, you could do something like http://en.wikipedia.org/wiki/Secret_sharing where the key is divided among several participants and never actually assembled in one place.
Of course any cryptography system has vulnerabilities. The point is not to build a 100% secure system. The point is to make it so that attempting to hack the system has less expected utility than simply doing what the humans say. And if that fails, then the AI will attempt to hack the system using its weakest point: the people controlling the key. Hence my question.
Comment author:solipsist
05 November 2013 02:48:44PM
*
3 points
[-]
If you're not talking about a back door, then I'm not sure what you're trying to say here.
Yeah, we're talking past each other. I think I understand what you're saying, and I'll try to rephrase what I'm saying.
The AI is out. It is free to manipulate the world at its will. Sensors are everywhere. The AI can hear every word you say, feel every keystroke you make, and see everything you see. The only secrets left are the ones in your head.
How do humans reward the AI? You say "cryptographically", but cryptography requires difficult arithmetic. How do you perform difficult arithmetic on a secret that can't leave your head?
Comment author:passive_fist
05 November 2013 08:19:55PM
0 points
[-]
Too many assumptions are being made here. What is the basis for believing the AI will have sensors everywhere, especially while it's still under human control? And if it has the ability to put clandestine sensors in even the most secure locations, why couldn't it plant clandestince brain implants in the people controlling the key?
Comment author:Protagoras
03 November 2013 12:45:52AM
1 point
[-]
I've played several RPG maker games made by amateurs. Some of them seemed to have significant followings, though I wasn't interested enough to make a serious effort to estimate the numbers, since I wasn't the creator. What kind of game were you thinking of making?
Comment author:CronoDAS
03 November 2013 02:27:21AM
23 points
[-]
I have a game I've been fantasizing about and I think I could make it work. It has to be a game, not a story, because I want to pull a kind of trick on the player. It's not that unusual in fiction for a character to start out on the side of the "bad guys", have a realization that his side is the one that's bad, and then go on to save the day. (James Cameron's Avatar is a recent example.) I want to start the player out on the side of bad guys that appear good, as in Eliezer's short story "The Sword of Good", and then give the player the opportunity to fail to realize that he's on the wrong side. There would be two main story branches: a default one, and one that the player can only get to by going "off-script", as it were, and not going along with what it seems like you have to do to continue the story. (At the end of the default path, the player would be shown a montage of the times he had the chance to do the right thing, but chose not to.)
The actual story would be something like the anti-Avatar; a technological civilization is encroaching on a region inhabited by magic-using, nature-spirit-worshiping nomads. The nature spirits are EVIL (think: "nature, red in tooth and claw") and resort to more and more drastic measures to try to hold back the technological civilization, in which people's lives are actually much better.
Comment author:KaynanK
03 November 2013 03:53:10AM
2 points
[-]
It seems like an interesting story idea, but, of course, the twist can't be revealed to any prospective player without spoiling it, so it might seem cliched on the surface.
Comment author:passive_fist
03 November 2013 05:17:11AM
2 points
[-]
It sounds very appealing to me, but as KaynanK pointed out, you have to be very careful about keeping the twist secret. To this end, I'd suggest not revealing to the players that they could have gone off-script, unless they do.
That sounds fun, and something that'd actually translate nicely to the RPG Maker template. It's also something that takes skill to pull off well, you'll need to play with how the player will initially frame the stuff you show to be going on, and how the stuff should actually be interpreted. Not coming off as heavy-handed is going to be tricky. Also, pulling this off is based on knowing how to use the medium, so if this is the first RPG Maker thing you're going to be doing, it's going to be particularly challenging.
There might also be a disconnect between games and movies here. Movies tend to always go out of their way to portray the protagonist's side as good, while games have a lot more of just semi-symmetric opposing factions. You get to play as the kill-happy Zerg or Undead Horde, and nobody pretends you're siding with the noble savages against the inhuman oppressors. So the players might just go, "ooh, I'm the Zerg, cool!" or "I guess I'm supposed to defect from Zerg to Terran here".
Random other thoughts, Battlezone 2 has a similar plot twist with off-script player action needed, though both factions are high-tech. Dominions 4 has Asphodel that's a neat corrupted nature spirit faction. Though I'm guess you're going for nature just being inherent bastards instead of the more common corrupted nature striking back trope.
Also, games really train people to stay on the script nowadays. Games that let you go rogue with an actual in-game-world action instead of choosing 'yes' on the blinking "DEFECT TO TERRAN SIDE" dialog are rare, since letting the player go off the script in-game and meaningfully interpreting their actions is really hard in the general case, and really frustrating for the player if they have to guess the particular special case where the off-script action actually opens a different plot branch instead of just leading nowhere like it did in the 10 previous levels. The original Deus Ex did have bits where you could mitigate the shit your early game actually evil employers were pulling with quick in-game thinking, but going over to the rebels was still always in the script.
So, overall, challenging project. You need to figure out RPG Maker and where to get the art assets and such, if you're not already skilled with it, you need to do worldbuilding for two worlds, and neither can be a cardboard cutout for the conceit to work, and you need to figure out how to make the game narration work so that the player can both get effectively tricked and has all the necessary pieces to put together the alternative choice during play.
Comment author:CronoDAS
03 November 2013 02:26:59PM
*
1 point
[-]
Yeah, it'll be hard. Right now I haven't worked out much more than the basic concept; I'd have a lot of writing to do, in addition to level design, learning RPG Maker, and so on.
As for art, RPG Maker does come with some built-in art and offers some more in expansion packs. If I have to, I can use placeholder art from the built-in assets and find some way to replace it once I'm happy with everything else.
Have you thought about how much time you are ready to put into the project? I'd ballpark the timescale for this as at least two years if you work on this alone, aren't becoming a full-time game developer and want to put a large-scale competent CRPG together.
EDIT: I'm guessing this would look like something like what Zeboyd Games puts out. They had a two-man team working full-time and took three months to make the short and simple Breath of Death. Didn't manage to find information on how long their more recent bigger games took to develop, but they seem to have released around one game a year since.
Comment author:CronoDAS
03 November 2013 04:50:06PM
*
3 points
[-]
Honestly, I'd probably start by trying to throw something much simpler together with RPG Maker, just to learn the system and see what it's like. And I don't actually have a "real job", so the amount of time I spend is mostly limited by my own patience.
And using RPG Maker might help speed up the technical work.
Comment author:philh
03 November 2013 03:29:42PM
*
5 points
[-]
Also, games really train people to stay on the script nowadays.
When I played Zelda games, I would always work out what option I was supposed to take, then take the other one, confident that I would get to see a few extra lines of dialogue before being presented with the same option again.
(I say "always", but when I first played, I would carefully make the correct choice, for fear that something bad would happen if I didn't agree to help Zelda. I don't remember when I developed the opposite habit.)
Comment author:Armok_GoB
04 November 2013 02:19:04AM
4 points
[-]
That sounds awesome... except now that I know about that twist it's ruined. And if you publish it under a different name and don't reveal it it wont sound awesome so I'll never discover it.
The only way to do this justice would be nagging enough people to play it that they can insist that it's better than it sounds and someone should really play it for reasons they can't spoil.
Comment author:Moss_Piglet
04 November 2013 03:31:47AM
7 points
[-]
I like the idea, mainly because I spent most of Avatar rooting for Quarditch (easily the biggest badass in the last decade of cinema), but it seems like there's another way to do it that might have a bit more power;
Why not have them both be "right," according to their own value systems anyway, and then have the end-game slideshow in both branches tell the player the story of what they did from the perspective of the other side?
In terms of workload, it seems minimal; from a story perspective you already need both sides to have sympathetic and unsavory elements anyway, while from a design perspective all you need to add is a second set of narration captions for the slideshow contingent on which side the player supported.
And in terms of appeal, it certainly seems more engaging than most AAA games. Spec Ops: the Line proved that players are masochists and that throwing guilt trips at them is a great way to get sales and good reviews, while Mass Effect 3's failure shows that genuine choice in endings is pretty important for a game built on moral choice.
Comment author:CronoDAS
05 November 2013 03:19:43AM
1 point
[-]
Why not have them both be "right," according to their own value systems anyway, and then have the end-game slideshow in both branches tell the player the story of what they did from the perspective of the other side?
Comment author:Viliam_Bur
05 November 2013 09:19:48AM
3 points
[-]
You don't have to make both branches equivalent. Both of them could feel "right" from inside, but only one of them could contain an information which makes the other one wrong.
In one ending, the hero only has limited information, and based on that limited information, the hero thinks they made the right choice. Sure, some things went wrong, but the hero considers that a necessary evil.
On another ending, the hero has more information, and now it is obvious that this choice was right, and all the good feelings from the other branch are merely lack of information or reasoning.
This way, if you only saw the first ending, you would think it is the good one, but if you saw both of them, it would be obvious the second one is the good one.
Comment author:drethelin
06 November 2013 07:54:04PM
3 points
[-]
I like this idea but it seems hard to differentiate between "You did what you thought was right but you need to be more careful about what you believe" and "you got the bad ending because you missed this little thing", which is something many games have done before.
An example is Iji where the game plays out significantly differently if you make a moral decision not to kill, but if you take the default path it doesn't let you know you could've chosen to be peaceful the whole time. It involves an active decision rather than a secret thing you can miss, but it also doesn't frame it as a "MORAL CHOICE TIME GO"
Comment author:Lumifer
04 November 2013 04:13:49AM
1 point
[-]
Does this sound appealing?
Well, that's just the twist idea, but what's your framework? Are you thinking about first-person shooters (Deus Ex style, for example) or about tactical turn-based RPGs or about 2-D platformers or what?
Comment author:ephion
04 November 2013 06:25:32PM
0 points
[-]
That does sound very appealing. I'm not well versed at all in game creation, but I do remember playing with RPG maker a few years ago and it was pretty limiting. Rather than RPG maker, why not make a Skyrim mod? That would be much more fun to play and would have a much larger potential userbase.
Comment author:lmm
03 November 2013 01:10:48AM
*
13 points
[-]
Only if I heard particularly good things about it.
Most creative endeavors you could undertake have a very small chance of leading to external reward, even the validation of people reading/watching/playing them - there's simply too much content available these days for people to read yours. So I'd advise against making such a thing, unless you find making it to be rewarding enough in itself.
Comment author:Dorikka
04 November 2013 08:48:37PM
*
0 points
[-]
Think that would have been a good move. Advice can be pretty good at presenting the outside view -- isn't as good at presenting the inside view unless the advice-giver really knows the advice-receiver well (ETA: meaning relevant details to receiver's specific case, etc.). Receiver should keep this in mind and update on relevant evidence (especially inside view evidence) that giver likely did not take into account.
Comment author:CronoDAS
03 November 2013 03:59:35PM
5 points
[-]
My underlying motivation is to feel better about myself. I feel that my life so far has lacked meaningful achievements. Pushing memes is a side benefit.
I do not expect to make money by selling the game, but if I do manage to make something that turns out to be pretty good, I think it would be a big help in getting a job in the video game industry.
Comment author:Metus
03 November 2013 12:13:52AM
*
8 points
[-]
Am I the only one who is bothered that these threads don't start on Monday anymore?
Posting a request from a past open thread again: Does anyone have a table of probabilities for major (negative) life events, like divorse or being in a car accident? I ask this to have a priority list of events to be prepared for, either physically or mentally.
Comment author:hyporational
03 November 2013 07:43:22AM
*
3 points
[-]
The lifetime risk of developing cancer is 44 % in males and 38 % in females. The lifetime risk of dying from cancer is 23 % in males and 19 % in females. It's worth mentioning that the methods for gathering medical mortality statistics are pretty biased, if not completely bonkers.
Comment author:hyporational
04 November 2013 05:32:23AM
*
9 points
[-]
ETA: Apparently a new WHO recommendation for filling death certificates was introduced in 2005-2006 and this caused a significant drop in pneumonia mortality in Finland.
I'm not entirely sure if it works this way in the whole EU, but it probably does. It's more complicated than what I explain below, but it's the big picture that matters.
The most common way to record mortality statistics is that the doctor who was treating the patient fills a death certificate. There are three types of causes of death that can be recorded in a death certificate. There are immediate causes of death and there are underlying causes of death. There are also intermediate causes of death, but nobody really cares about those because recording them is optional. The statistics department in Finland is interested in recording only the underlying causes of death and that's what gets published as mortality statistics. Only one cause of death per patient gets recorded.
If someone with advanced cancer gets pneumonia and dies, a doctor fills the death certificate saying that the underlying cause of death was cancer and the immediate cause of death was pneumonia. Cancer gets recorded as the one and only cause of death by the statistics department. Depending on the patient, possible underlying causes of death could also be alcoholism, coronary heart disease or alzheimer's disease or whatever is accepted by a department that checks these certificates.
The doctor's opinion of whether it was the pneumonia or the chronic disease that killed the patient doesn't really matter. If he also fills the underlying cause of death as pneumonia, he gets a scolding letter and has to fill it again until he gets it right.
What if the patient has several chronic diseases that could have been underlying causes of death? Well, you only get to pick one, and only that one gets recorded as the cause of death. You can list the other diseases too as contributory causes of death, but this doesn't really effect any statistics. I guess it would be less biased to flip a coin or something, but I think most doctors just pick something fitting.
A colleague of mine once tried to record pneumonia as the underlying cause of death, the patient was an alcoholic (not sure how bad it was). He got a letter saying he should fix the certificate and that people in developed countries don't die of pneumonia anymore. Wonder why that is...
Comment author:Lumifer
04 November 2013 05:07:41PM
2 points
[-]
Very interesting, thank you.
I have a pet interest -- carefully looking at how standard, universally-accepted, real-life, empirical data is collected and produced and whether it actually represents what everyone blindly assumes it does. In the field of economics, for example, closely examining how, say, the GDP or the inflation numbers are calculated is... illuminating.
Comment author:Lumifer
05 November 2013 12:12:50AM
1 point
[-]
The problem is that the problems aren't summarizeable in a neat half a page list. And it's not like the calculations are wrong, rather they are right under a certain set of assumptions and boundary conditions -- and the issue is that people forget about these assumptions and conditions and just assume they're right unconditionally.
For an introduction take a look at e.g. Shadowstats. I don't necessarily agree with everything there, but it's a useful starting point.
I twitch when changes in GDP are reported to a tenth of a percent-- it seems to me that it couldn't be measured with such precision. Do you think I'm being reasonable?
I'm assuming that the GDP is some sort of measure of the health of the economy-- that's why people are concerned with it. The health of the economy seems to me like rather an approximate sort of thing.
Comment author:ahbwramc
07 November 2013 05:02:27PM
3 points
[-]
My own (uninformed) intuition is that GDP changes would be much more accurate than absolute GDP values, just because systematic errors could largely cancel out.
Comment author:gsgs
05 November 2013 03:21:38PM
3 points
[-]
in USA they can fill in 20 secondary causes on the death certificates and all
the anonymized death certificates since 1959 are available online from NCHS in computer-readable form to check/search for conditions.
Irregularities usually appear when there is a switch from one ICD-Code to a new one,
so in 1969,1979,1999.
Other irregularities are often checked, compared with other states,countries,conditions
and the reason discovered
Comment author:hyporational
05 November 2013 04:35:30PM
*
2 points
[-]
What if the patient has several chronic diseases that could have been underlying causes of death? Well, you only get to pick one, and only that one gets recorded as the cause of death. You can list the other diseases too, but not as causes of death.
It seems I miscommunicated here. What I meant to say that listing these other diseases has no meaningful impact on the mortality statistics, although technically speaking they are causes of death. If the point is to gather accurate statistics, listing them feels like a consolation prize, because statisticians don't seem to be interested in them.
In Finland a direct translation for these would be "contributory causes of death". That's probably the same thing as secondary causes of death. The problem is, it's difficult for someone who makes these into statistics to know how important they were. Almost anything the patient has can be listed as a contributory cause of death.
Even a bigger problem is that listing them is completely optional. If almost nobody fills them in properly (because they usually have better things to do), that is another good reason for a statistician not to use them.
Is filling in the secondary causes mandatory in US? Are there clear restrictions for what can be listed? If not, I'm not sure if they provide all that useful information, statistically speaking. Are they really used in meaningful way in any statistics?
Irregularities usually appear when there is a switch from one ICD-Code to a new one, so in 1969,1979,1999.
I suppose WHO recommendations for filling these certificates impact the US too.
Comment author:philh
03 November 2013 03:22:44PM
0 points
[-]
Am I the only one who is bothered that these threads don't start on monday anymore?
I like it for purely selfish reasons. I can't easily post between Sunday bedtime and Tuesday evening. If I want to post and the thread starts on Monday, my post will be less visible.
Comment author:gwern
03 November 2013 05:04:13PM
26 points
[-]
If Yvain is (understandably) too busy to run it this year, I am willing to do it. But I will be making changes if I do it, including reducing the number of free responses and including a basilisk question.
Comment author:Tenoke
03 November 2013 10:42:09AM
*
3 points
[-]
Not particularly important ,but if anyone wants to come out and tell me why they went on a mass-downvoting spree on my comments, please feel free to do so.
It seems to me that a relevant detail is time frame is ~7 months (as you say elsewhere). Ideally, hashes would be commitments to reveal the plaintext in a specified time. Don't you discuss this somewhere?
There is a danger with publishing hashes that you might publish opposite ones. Ideally, you should be committing to one answer or the other. High entropy predictions mitigates against this, but the effect is still there. And we can't tell if it's high entropy until it is revealed. Publishing dates mitigates against this.
I thought I recently saw an essay on the uses of hash precommitments, including this kind of problem. If that doesn't ring a bell, I guess it wasn't by you.
Comment author:gwern
04 November 2013 03:03:51PM
*
2 points
[-]
Oh. Yeah, I did start a little discussion on an isomorphic trick in Umineko. In this case, the date on which I posted the hash is provided automatically by Reddit/Lesswrong/Twitter/etc and one can also verify I didn't post any other hashes recently to those fora.
The trick also only works for 'small keyspaces', if you will - if for example I was trying to fake a precommitment to a 100-digit number, the trick isn't going to work because it's not feasible to publish precommitments to even a tenth of the potential 100-digit numbers without people noticing and calling foul - 'so, gwern, why are you publishing that many precommitments and when can we expect them all to be revealed...?'
Comment author:Viliam_Bur
03 November 2013 05:35:08PM
*
15 points
[-]
So I get home from a weekend trip and go directly to the HPMOR page. No new chapter yet. But there is a link to what seems to be a rationalist Death Note.
The way he saw it, the world was a pretty awful place. Corrupt politicians, cruel criminals, evil CEOs and even day-to-day evil acts made it that way, but everyday stupidity ensured it would stay like that. Nobody could make even a simple utility calculation. The only saving grace was that this was as true for the villains as for the heroes.
I am going to read it. Here are my next thoughts:
So, it seems like Eliezer succeeded to create a whole new genre of literature: rationalist fiction. Nice job!
Wait, what?! Is "a story where the protagonist behaves rationally" really a new genre of literature? There is something horribly wrong with this world if this is true.
Discussing with my girlfriend about which stories should be x-rationalizated next, she suggests HPMOR. Someone should make a HPMOR fanfic where the protagonist is even more rational than the rational Harry. Would that lead to a spiral of even more and more rational heroes?
What exactly could the MoreRational!Harry do? It would be pretty awesome if he could somehow deduce the existence of magic before he was contacted from Hogwarts. For example, he could start doing some research about his biological parents; after realizing they were killed he could try to find out the villain, and gradually discover the existence of magic.
Only one problem: MoreRational!Voldemort would have killed MoreRational!Harry as a baby. Using a knife.
Comment author:fubarobfusco
03 November 2013 06:20:53PM
*
19 points
[-]
Is "a story where the protagonist behaves rationally" really a new genre of literature?
I think what you are referring to here is "a story where the protagonist describes their actions and motivations using rationality terminology" or maybe "a story where the rational thinking of the protagonist motivates the plot or moves it along". At least some of the genre of detective fiction — early examples being Poe's Auguste Dupin stories — would be along these lines.
Stories where protagonists behave rationally (without using rationality terrminology) wouldn't look like stories about rationality. They look like stories where protagonists do things that make sense.
Comment author:[deleted]
03 November 2013 06:57:57PM
*
21 points
[-]
Discussing with my girlfriend about which stories should be x-rationalizated next, she suggests HPMOR. Someone should make a HPMOR fanfic where the protagonist is even more rational than the rational Harry.
An idea came to my mind. Would it be possible to make a story in which Harry is less intelligent, in a way that he would score less in an IQ test for example, but at the same time more rational? HJPEV seems to be a highly intelligent prodigy even without the rationality addition. I would like to see how a more normal boy would do.
Comment author:gattsuru
05 November 2013 04:51:34PM
1 point
[-]
Rationalist!Harry is calibrated to match the knowledge and recall of a 34-year-old autodidact. Even presuming a very friendly environment and that said 34-year-old autodidact's training was not optimal, I just don't think there's enough time.
I can buy a 10 year old reading Ender's Game and The Lord of the Rings and maybe even Lensmen. It's a bit harder to imagine one that would consider wanting to want the math behind proving N=NP, nevermind going further than that.
Comment author:CAE_Jones
05 November 2013 05:17:41PM
1 point
[-]
I believe it's been stated somewhere that EY draws primarily on the skills he had around 18 and intentionally keeps things from beyond that out of Harry's reach. So Harry is more like a brilliant high school student than an adult (and, extra seven years worth of rationalist training aside, the way he approaches problems is a lot like a middle schooler with superpowers: "I can win, you can't, deal with it, 'cause I'm awesome and you know it." Which manages to annoy everyone in-universe and out.). Time isn't really a problem, either, if Harry has nothing else to occupy his time; exercise and social interaction are apparently not his thing, and he wound up out of the public school system after a few years, so he really does have way more time than most kids his age to read all the books. And he has that mysterious dark side and that sleeping disorder, whatever those contribute.
The other strangely adult-like children, however, are not so easily justified. (Draco gets most of those complaints, from what I've read.)
Wait, what?! Is "a story where the protagonist behaves rationally" really a new genre of literature? There is something horribly wrong with this world if this is true.
Yup. At least sort-of. If you haven't read Eliezer's old post Lawrence Watt-Evans's Fiction I recommend it. However, conspicuous failures of rationality in fiction may be mostly an issue with science fiction and fantasy. If you want to keep the characters in your cop story from looking like idiots, you can do research on real police methods, etc. and if you do it right, you have a decent shot at writing a story that real police officers will read without thinking your characters are idiots.
On the other hand, when an author is trying to invent an entire fictional universe, with futuristic technology and/or magic, it can be really hard to figure out what would constitute "smart behavior" in that universe. This may be partly because most authors aren't themselves geniuses, but even more importantly, the fictional universe, if it were real, would have millions of people trying to figure out how to make optimal use of the resources that exist in that universe. It's hard for one person, however smart, to compete with that.
For that matter, it's hard for one author to compete with an army of fans dissecting their work, looking for ways the characters could have been smarter.
Comment author:MathiasZaman
04 November 2013 09:44:49AM
*
6 points
[-]
Wait, what?! Is "a story where the protagonist behaves rationally" really a new genre of literature?
I think there's a difference between what I've been describing as rationalist!fic (or rationalist!fiction) and fiction in which the -agonists (PCs is the right terminology, I guess) are rational/clever. Rationalist!fic doesn't just feature rationalist characters, they're expressively written to teach the audience about rationality.
Examples:
Doctor Who features a sufficiently advanced alien who is, within the rules of the universe, pretty rational (in that he is good at reaching his goals). The message of the show however, is not: "be clever and rational," it's: "humanity is awesome and you should feel some wonder about the universe." Not rationalist!fic.
The Conqueror's Shadow, by Ari Marmell features rationalist agonists and the message the audience goes away with is: "be clever and creative when it comes to reaching worthwhile goals." Rationalist!fic.
Comment author:hyporational
04 November 2013 03:14:17PM
*
1 point
[-]
Wait, what?! Is "a story where the protagonist behaves rationally" really a new genre of literature? There is something horribly wrong with this world if this is true.
I get your sentiment, but I don't think this is true. Anyways, wouldn't this just mean that rational minds usually pursue other goals than writing fiction? Not saying that there shouldn't be rationalist fiction, but this doesn't sound like such a bad state of affairs to me.
I haven't read HPMOR. Do I have to know anything about the HP universe to enjoy this thing? Will I learn anything new if I've read the sequences?
Comment author:Viliam_Bur
04 November 2013 03:55:19PM
*
3 points
[-]
I guess you don't need to know anything from the HP canon. It could perhaps be even more interesting that way. I don't think you would learn new information. It might have a better emotional impact, but that is difficult to predict.
wouldn't this just mean that rational minds usually pursue other goals than writing fiction? Not saying that there shouldn't be rationalist fiction, but this doesn't sound like such a bad state of affairs to me.
I would consider the world better if there were more rational people sharing the same values as me. We could cooperate on mutual goals, and learn from each other.
Problem is, rational people don't just appear randomly in the world. Okay, sometimes they do, but the process if far from optimal. If there is a chance to make rationality spread more reliable, we should try.
But we don't exactly know how. We tried many things, with partial success. For example the school system -- it is great in taking an illiterate peasant population and producing an educated population within a century. But it has some limits: students learn to guess their teachers' passwords, there are not enough sufficiently skilled teachers, the pressure from the outside world can bring religion to schools and prevent teaching evolution, etc. And the system seems difficult to improve from inside (been there, tried that).
Spreading rationality using fiction is another thing worth trying. There is a chance to attract a lot of people, make some of them more rational, and create a lot of utility. Or maybe despite there being dozens of rationalist fiction stories, they would all be read by the same people; unable to attract anyone outside of the chosen set. I don't know.
The point is, if you are rational and you think the world would be better with more rational people... it's one problem you can try to solve. So before Eliezer we had something like the Drake equation: how many people are rational × how many of them think making more people rational is the best action × how many of them think fiction is the best tool for that = almost zero. I am curious about the specific numbers; especially whether one of them is very close to zero, or whether it's merely a few small numbers that give almost zero result when multiplied together.
I'd probably want more people who share my values than more rational people. Rational people who share my values is better. Rational people who don't share my values would be the worst outcome.
I don't think the school system was built by rationalists, so I'm not sure where you were going with that example.
How effective has fiction been in spreading other ideas compared to other methods?
Comment author:Ishaan
05 November 2013 01:18:12AM
*
6 points
[-]
which stories should be x-rationalizated next
This leads to another comment on rationalist fiction: Most of it seems to be restricted to fan-fiction. The mold appears to be: "Let's take a story in which the characters underutilized their opportunities and bestow them with intelligence, curiosity, common sense, creativity and genre-awareness". The contrast between the fanfic and the canon is a major element of the story, and the canon an existing scaffold which saves the writer from having to create a context.
This isn't a bad thing necessarily, just an observation.
Wait, what?! Is "a story where the protagonist behaves rationally" really a new genre of literature?
So, the question becomes, how do you recognize "rationalist" stories in non-fan-fic form? Is it simply the presence of show-your-work-smart characters? Is simply behaving rationally sufficient?
Every genre has a theme...romance, adventure, etc.
So where are the stories which are, fundamentally, about stuff like epistemology and moral philosophy?
Comment author:Viliam_Bur
05 November 2013 09:28:02AM
*
0 points
[-]
So, the question becomes, how do you recognize "rationalist" stories in non-fan-fic form? Is it simply the presence of show-your-work-smart characters? Is simply behaving rationally sufficient?
That's a great question. (And related to how to recognize rational people in real life.)
I'd say that there must be some characters which are obviously smarter than most people around them. Because that's what happens in real life: there is the bell curve, so if all your characters are on a similar level, then the story is (a) not realistic, (b) the characters are selected by some intelligence filter which should be explicitly mentioned, or (c) the characters are all from the middle of the bell curve. Also, in real life the relative power of intelligent people is often reduced by compartmentalization, but this reduction would be much smaller for a rationalist hero.
So I'd say it's behaving rationally while most of other people aren't. The character should somehow reflect on the stupidity of others; whether by frustration from their inability to cooperate, or by enjoyment of how easily they are manipulated.
Comment author:Ishaan
05 November 2013 06:08:05PM
*
2 points
[-]
The character should somehow reflect on the stupidity of others; whether by frustration from their inability to cooperate, or by enjoyment of how easily they are manipulated.
I'm not sure I like that criteria. By that criteria alone, the original death note anime was rationalist fiction (judging by the first half), as is Artemis Fowl, Ender's Game, and to some extent even Game of Thrones. There are a lot of stories where some characters are much smarter than others and know it, but consuming these works won't teach anyone how to be smarter. (Other than the extent to which reading good fiction in general improves various things)
None of these stories actually teach the reader anything about epistemology. Even the linked Death Note fan fic...it uses rationality-associated words like "utility" and "prior" but if I didn't already know what those words meant I would have just come away confused. (Granted, it's still early in the story - but even so)
Also, it hasn't yet broken the conceit of the story (For example, even a normal person of average intelligence would be surprised and curious about the existence of the supernatural, and would investigate that). I'd say that breaking the story conceit is another feature of rationalist fanfiction stories that has nothing to do with the character's intelligence.
Comment author:Viliam_Bur
05 November 2013 08:02:28PM
1 point
[-]
Well, I was disappointed with the Death Note fan fic, because it doesn't seem to have added value beyond the original story. And I agree that exploring the supernatural should be a high priority for a rational person, once the supernatural is experimentally proven. Would it be so difficult to ask Ryuk whether there are additional magical items that could also be abused? I guess Ryuk would use an excuse of having "rules" against that, but at least it's worth trying.
Having a rational superhero is a necessary condition for a rationalist story, not a sufficient condition. Ender's Game could be a rationalist literature if it explained Ender's reasoning better, and if Ender strategically tried to improve his understanding of the world. Okay, another necessary condition is not just that the superhero is super smart, but also that the super smartness is at least partially a result of a good strategy, which is shown to the reader.
So, the question becomes, how do you recognize "rationalist" stories in non-fan-fic form? Is it simply the presence of show-your-work-smart characters? Is simply behaving rationally sufficient?
Every genre has a theme...romance, adventure, etc.
I'd say the difference between "rationalists" stories and "non-rationalist" stories lies in the moral of the story, of the lessons the story teaches you.
I don't think it's a genre in the same way romance or adventure are. It's more of a qualifier. You can have rationalist romance novels or rationalist adventure movies.
Although you could argue that it is a genre. While discussions about "genre" are often hard, since people don't tend to agree on what makes something a genre.
But rationalist fiction already has a couple of genre conventions, such as no-one being allowed to hold the idiot ball or teaching the audience new and useful techniques for overcoming challenges.
Comment author:gattsuru
05 November 2013 05:19:03AM
*
1 point
[-]
In addition to the other's already listed, DataPacRat's Myou've Got To Be Kidding Me follows a perspective of a character thrown into a setting and trying to analyze the basic rules in order to optimize them. There are some interesting concepts, but I don't know that I can recommend it : It has not been updated in over a year, and was part of some big conglomeration of fanfic writers which had some pretty widely varying quality (although thankfully nothing necessary to Myou've plotline).
Comment author:[deleted]
03 November 2013 06:39:45PM
*
9 points
[-]
SPOILERS FOR "FRIENDSHIP IS OPTIMAL"
Why is 'Friendship is optimal' "dark" and "creepy"? I've read many people refer it that way. Only things that are clearly bad are the killings of all the other lifeforms, but otherwise this is scenario is one of the best that humanity could come across. It's not perfect, but it's good enough and much better than the world we have today. I'm not sure if it's realistic to ask for more. Considering how likely it is that humanity will end in some incredibly fucked up way full of suffering, then I would definitely defend this kind of utopia.
Comment author:Leonhart
03 November 2013 09:29:27PM
*
11 points
[-]
(Comment cosmetically edited in response to Kaj_Sotala, and again to replace a chunk of text that fell in a hole somewhere)
OK, I'll have a go (will be incomplete).
People in general will find the Optimalverse unpleasant for a lot of reasons I'll ignore; major changes to status quo, perceived incompatibility with non-reductionist worldviews, believing that a utopia is necessarily unpleasant or Omelas-like (a variant of this fallacy?), and lots of even messier things.
People on LessWrong may be thinking about portions of the Fun Theory Sequence that the Optimalverse conflicts with, and in some cases they may think that these conflicts destroy all of the value of the future, hence horror.
(rot13 some bits that might consitute spoilers)
Humans want things to go well, but they also want things to have been able to go badly, such that they made the difference. Relevant: Living By Your Own Strength,
Free to Optimize.
The existence of a superintelligence makes human involvement superfluous, and humans do not want this to happen. Relevant: Amputation of Destiny.
Gur snpg gung gur NV vf pbafgenvarq gb fngvfsl uhzna inyhrf gur cbal jnl zrnaf gung n uhtr nzbhag bs cbffvoyr uhzna rkcrevrapr vf abj vzcbffvoyr gb rire ernyvfr. Eryrinag: Hzz... znlor Value is Fragile? Abg dhvgr. Uryc zr bhg urer, thlf!
(nyfb, vafreg lbhe bja cersreerq snaqbz wbxr nobhg cbbe Ylen arire trggvat gb unir unaqf rgp.)
Nf lbh zragvbarq, gur jnl va juvpu gur NV'f cnegvphyne qrsvavgvba bs "uhzna" jnf abg evtug naq pna arire or zbqvsvrq, urapr nyvra ncbpnylcfrf. Eryrinag: The Hidden Complexity of Wishes
Themes that are more explicit after the extra worldbuilding in Caelum est Conterrens:
Zbqvslvat uhzna zvaqf va gur jnl gur hcybnqf ner qrfpevorq nf orvat zbqvsvrq vf ernyyl, ernyyl, ernyyl, ernyyl uneq, naq zvtug or vzcbffvoyr jvgubhg oernxvat crefbany pbagvahvgl Growing Up is Hard. (Guvf vf zber bs n ubeebe fbhepr guna na nethzrag, orpnhfr gur fgbel pna or ernq nf fgvchyngvat gung gur NV vf trggvat vg evtug).
ybbcvat raqyrffyl jvgu zrzbel biresybj (gung vf, va gur raq nyy yvirf snvy gur pbaqvgvbaf va Emotional Involvement ol orpbzvat n qvfpbaarpgrq frevrf bs rcvfbqrf)
I'm sympathetic to your position; this is the substance of my comment here that I think I understand what's supposed to horrify me.
That comment of mine is no doubt wrong; there will be things that don't horrify me that I didn't even realise were supposed to.
There are quick and obvious comebacks to nearly all the above points. In a lot of cases, those quick comebacks are dealt with in the linked articles. Read the Fun Theory Sequence; it's my favorite sequence, despite the fact that I disagree with more of it than any of the others.
Comment author:[deleted]
04 November 2013 02:58:04PM
*
3 points
[-]
Now that I've thought about your post I realized that the biggest question in this story is what the phrase "satisfy values" actually means. Currently it's a pretty big hand wave in the story. Especially your first point seems to imply that we understood it a bit differently.
In my understanding, if I value real challenge, the possibility of things going badly, or even some level of pain, then the Optimalverse will somehow maximize those values and at least provide the feeling of real challenge and possibility of things going badly. And I don't know why the Optimalverse couldn't even provide the real thing. The way Light Sparks tries to pass the Intermediate Magic test seems an awfully lot like real challenge. Of course the Optimalverse wouldn't allow you to die because in most cases the dislike of death overrides the longing for real challenge in the value system, but that still leaves a lot of options free. I got the impression that this is how it's actually handled in the story. There's this passage
Cbavrf unq ab cerqngbef; orvat ‘rngra’ ol n zbafgre va gur Rireserr sberfg whfg raqrq jvgu gur cbal va gur ubfcvgny va dhvgr n ovg bs cnva. Fngvfslvat inyhrf jnfa’g whfg nobhg unccvarff; univat zbafgref yrg cbavrf grfg gurve fgeratgu be oenirel. Rneyl ba, evtug nsgre gur pbairefvba bs Rnegu, n zrer sbhe uhaqerq cbavrf unq crgvgvbarq Cevaprff Pryrfgvn gb yrg gurz qvr, naq Cevaprff Pryrfgvn unq bayl nterrq gung qbvat fb jbhyq fngvfsl gurve inyhrf va rvtugl-fvk pnfrf. Abcbal unq qvrq va frireny Rdhrfgevna fhowrpgvir zvyyraavn.
Your second point is of course a real concern for some people, but personally it doesn't feel very relevant. My actions don't currently feel very important in the big scheme of things and I don't know how a superintelligence would change things all that much. If I'm not personally doing anything important, then it doesn't really matter to me if the important things are done by other humans or by a superintelligence. Anyway, this will always be a problem with AGI and if the AGI is friendly then the benefits outweight the negatives IMO. I think the alternative is worse.
The way I understood it is that the "ponies" in this story are essentially human in a pony disguise with four legs (two of them which can almost work like hands). A paragraph from the story:
V zbqvsvrq lbhe zbgbe pbegrk fb lbh pbhyq qrny jvgu lbhe arjsbhaq dhnqehcrqny zbirzrag, nybat jvgu bgure qvssreraprf orgjrra n uhzna naq cbal obql. V unir znqr gur zvavzny frg bs cbffvoyr punatrf; lbhe crefbanyvgl vf hapunatrq.
A big part of being human is due to our mind and hormones. Walking with two legs or being able to use hands extensively are more trivial points. If the psychology of a person doesn't change in the transition from human to pony, then this eliminates most of the problems in your third point.
I haven't read Caelum Est Conterrens and can't fully comment on those points. But it seems that those are more like technicalities. I don't know if it's actually possible to turn a person into a pony without losing the person in the process. But if you're not changing the brain parameters and the psychology doesn't change in the process like it seems to be in this story then I would be inclined to say it's possible. Clearly it can't be worse for your identity than losing all your limbs or becoming a quadriplegic? Anyway, one of the axioms in this story seems to be that it's possible.
I actually read the Fun theory sequence in its entirety before I read 'Friendship is optimal' and I thought FIO more faithful to the spirit of the sequence than 99% utopian stories out there. This is mostly because Celestia maximizes people's values, not their happiness. This is a very vague concept, and a lot depends on how it's implemented, but if it's implemented the way I picture it, there shouldn't be problems with things mentioned in High Challenge, Complex Novelty, Sensual Experience, Living By Your Own Strength, Free to Optimize, In Praise of Boredom, Interpersonal Entanglement and so on.
Of course, I have problems with applying things I read about to all my experiences, so it could be I misremember some things in the sequence or didn't understand them correctly to begin with.
Clearly it can't be worse for your identity than losing all your limbs or becoming a quadriplegic?
Well, this is not clear, though it might be true.
I have frequently had the experience of not doing anything with my left leg; losing the ability to ever do anything with my left leg means I'm prevented from ever doing anything with it. This is horrible, of course, but it's the horror of being prevented from doing things I often choose not to do. Losing all my limbs is a more extreme version of the same thing.
Having different limbs might be more identity-distorting, by virtue of providing experiences that are completely unfamiliar.
Then again it might not.
For my own part, I'm not all that attached to preserving my current identity, so I'm not sure the question matters to me. If my choice is between an identity-altering pony body, and an identity-preserving quadriplegic body, I might well choose the former.
Comment author:Kaj_Sotala
04 November 2013 05:35:27PM
*
3 points
[-]
Upvoted, but I'd like to request that you'd ROT13 either everything or nothing past a certain point. Being unable to just select all of it to be deciphered, and having to instead pick out a few pieces at a time, was mildly annoying.
Comment author:Leonhart
04 November 2013 08:51:19PM
2 points
[-]
Done, thanks for saying. I was trying to avoid thinking about the interaction between rot13 and links (leaving the anchor text un-rot13ed seems like acceptable practice?) but I should just have spent the extra two minutes.
Comment author:Kaj_Sotala
05 November 2013 04:42:21AM
*
1 point
[-]
Thanks! Much better now. :-) (As for the links, one can just paint over them as well and think "oh it was just some link" when they show up as garbled in the translation.)
Comment author:[deleted]
04 November 2013 10:25:44PM
*
1 point
[+]
(1
child)
Comment author:[deleted]
04 November 2013 10:25:44PM
*
1 point
[-]
I read Caelum Est Conterrens, now I can better understand why some aspects of the scenario are a bit disconcerting if not horrifying. I find all the options loop immortality, ray immortality and exponential immortality kinda unpleasant, but maybe that is as good as it gets. Still, it feels like many of those things are not exclusive to this scenario, but are part of the world anyway.
Related to this, what did you think about the "normal" ending in the Three worlds collide?
Comment author:Leonhart
04 November 2013 11:26:51PM
1 point
[-]
From flaky memory, I think I find the Normal Ending far less acceptable than anything in the Optimalverse - one feels the premature truncation of human nature, rather than the natural exhaustion of it (or the choice to become inexhaustible) - but hey, maybe I'm inconsistent.
Comment author:blacktrance
04 November 2013 06:33:54PM
1 point
[-]
Having just finished reading "Friendship is Optimal" literally less than 10 minutes ago, I didn't find it dark or creepy at all. There are certain aspects of it that are suboptimal (being ponies, not wireheading), but other than that, it sounds like a great world.
Comment author:blacktrance
06 November 2013 02:44:21AM
*
4 points
[-]
Yes. I think wireheading is the optimal state (assuming it can make me as happy as possible). I recognize this puts me at odds with an element of the LessWrong consensus.
Comment author:gattsuru
05 November 2013 12:47:16AM
*
3 points
[-]
At least to me, it's increasingly difficult to distinguish between a paradise machine and wireheading, and I dislike wireheading. Each shard of the Equestria Online simulation is built to be as fulfilling (of values through ponies and friendship) as possible, for the individual placed within that shard.
That sounds great! .... what happens when you're wrong?
I mean, look at our everyman character, David. He's set up in a shard of his own, with one hundred and thirty two artificial beings perfectly formatted to fit his every desire and want, and with just enough variation and challenge to keep from being bored. It's not real variation, or real challenge, but he'd not experience that in the real world, either, so it's a moot point. But look at the world he values. His challenges are the stuff of sophmore programming problems. His interpersonal relationships include a score counter for how many orgasms he gives or receives.
Oh, his lover is sentient and real, if that helps, but look at that relationship in specific. Butterscotch is created as just that little bit less intelligent than David is -- whether this is because David enjoys teaching, or because he's wrapped around the idea of women being less powerful than he is, or both, is up to the reader. Sculpted in her memories to exactly fit David's desires, and even a few memories that David has of her she never experiences, so that the real Butterscotch wouldn't have to have experienced unpleasant things that CelestAI used to manipulate David into liking/protecting her.
There are, to a weak approximation, somewhere between five hundred billion and one trillion artificial beings in the simulation, by the time most of humanity uploads. That number will only scale up over time. Let's ignore, for now, the creepiness in creating artificial sentients who value being people that make your life better. We're making artificial optimized for enjoying slaking your desires, which I would be surprised if it happened to also be optimized for what we as society would really like.
At a deeper level, what if your own values are wrong?
The basic example, brought up in the Rules of The Universe document, is a violent psychopath. Upon being uploaded, CelestAI would quite happily set our psychopath up in a private shard with one hundred and fifty artificial ponies, all of which are perfectly molded to value being shot, stabbed, lit on fire, and violated in a way that is as satisfying as possible to a Dexter villain.
Or I can provide a personal example. I can go both ways, preferring guys, and was an unusually late bloomer. I can look back through time to see an earlier version of myself's values, and remember how they changed. Even in a fairly tolerant society and even with a very collaborative environment, this was not something that came according to my values or without external stimulus. ((There is a political position version of this, but for the sake of brevity I'll just mention that it's possible. More worryingly, I'm not sure there's a way to formalize this concern, as much as it hits me at a gut level. For the most part, value drift is something we don't want.))
Or, for an in-story example :
Ybbx ng jung unccraf gb Unaan / 'Cevaprff Yhan'. Gur guvat fur inyhrf zbfg, ng gur raq bs gur fgbel, vf oryvrivat gung fur qvq abg znxr n zvfgnxr hayrnfuvat PryrfgNV. Naq PryrfgNV vf dhvgr pncnoyr bs fubjvat ure whfg gur orfg rknzcyrf bs ubj guvatf ner orggre. Vg qbrfa'g znggre jung gur ernyvgl vf, naq vaqrrq gur nhgube gryyf hf gung Unaan pbhyq unir qbar orggre. Zrnajuvyr, Unaan vf xrcg whfg ba gur obeqre bs zvfrenoyr nf gur fgbel raqf.
It's a very good dysutopia -- I'd rather live there than here, and heck it even beats a good majority of conventional fluffy cloud heaven afterlives -- but it's still got a number of really creepy issues..
A little fiction on related topics: "Hell Is Forever" by Alfred Bester-- what if your dearest wish ts to create universes?You're given a pocket universe to live in forever, and that's when you find out that your subconscious keeps leaking into your creations (they're on the object level, not the natural law level), and you don't like your subconscious.
Saturn's Children by Charles Stross. The human race is gone. All that's left is robots, who were built to be imprinted on humans. The vast majority of robots are horrified at the idea of recreating humans.
Comment author:Leonhart
05 November 2013 11:50:33PM
*
9 points
[-]
Let's ignore, for now, the creepiness in creating artificial sentients who value being people that make your life better.
No, let's not ignore it. Let's confront it, because I want a better explanation.
Surely a person who values being a person that makes my life better, AND who is a person such that I will value making their life better, is absolutely the best kind of person for me to create (if I'm in a situation such that it's moral for me to create anyone at all).
I mean, seriously? Why would I want to mix any noise into this process?
Comment author:gattsuru
06 November 2013 04:02:33AM
2 points
[-]
Good point. I've not uncompressed the thoughts behind that statement nearly enough.
Surely a person who values being a person that makes my life better, AND who is a person such that I will value making their life better, is absolutely the best kind of person for me to create (if I'm in a situation such that it's moral for me to create anyone at all).
The artificial sentients value being people that make your life better (through friendship and ponies). Your values don't necessarily change. And artificial sentients, unlike real ones, have no drive toward coherent or healthy spaces of design of minds : they do not need to have boredom, or sympathy, or dislike of pain. If your values are healthily formed, then that's great! If not, not so much. You can be a psychopath, and find yourself surrounded by people where "making their lives better" happens only because you like the action "cause them pain for arbitrary reasons". Or you could be a saint, and find yourself surrounded by people who value being healed, or who need to be protected, and what a coincidence that danger keeps happening. Or you can be a guardian, and enjoy teaching and protecting people, and find yourself creating people that are weak and in need of guidance. There are a lot of things you can value, and that we can make sentient minds value, that will make my skin crawl.
Now, the Optimalverse gets rid of some potential for abuse due to setting rules -- it's post-scarcity on labor, starvation or permanent injury are nonsense, CelestAI really really knows your mind so there's no chance of misguessing your values, so we can rule out a lot of incidental house elf abuse -- but it doesn't require you to be a good person. Nor does it require CelestAI to be. CelestAI cares about satisfying values through friendship and ponies, not about the quality of the values themselves. The machine does not and can not judge.
If it's moral to create a person and if you're a sufficiently moral person, then there's nothing wrong with artificial beings. My criticism isn't that CelestAI made a trillion sentient beings or a trillion trillion sentient beings -- there's nothing meaningfully worrying about that. The creepy factor is that CelestAI made one being, both less intelligence than possible and less intelligent than need be.
That may well be an unexamined reaction or even incorrect response. I like to think I'm open-minded, but I'm willing to recognize that I can overestimate it, and have done so in the past. There are real-world right-now folk who enjoy being (in specific contexts and while in control) hurt or being hurt and comforted, which I can accept. Maybe I'm being parochial when I judge David for wanting a woman he can always teach, or Lars for his sex groupies; that's not a mind space I empathize with terribly well, and a good deal of my revulsion comes from real-world constraints that wouldn't apply here. There's a reason that we're using the word creepy, rather than wrong. But it does make my skin crawl.
In honor of NaNoWriMo, I offer up this discussion topic for fans of HPMOR and rationalist fiction in general:
How many ways can we find that stock superpowers (magical abilities, sci-fi tech, whatever), if used intelligently, completely break a fictional setting? I'm particularly interested in subtly game-breaking abilities.
The game-breaking consequences of mind control, time travel, and the power to steal other powers are all particularly obvious, but I'm interested in things like e.g. Eliezer pointing out that he had to seriously nerf the Unbreakable Vow in HPMOR to keep the entire story from being about that.
Comment author:Armok_GoB
04 November 2013 02:54:42AM
*
4 points
[-]
I seem to be able to do this with almost any power to various degrees. Including ones I actually have, and ones that are common among humans. Any specifics you had in mind?
Really, ANY ability will reroll some chaotic stuff and be a valuable asset simply because it's rare. Even a deliberating curse, if rare and interesting enough, can do things like be useful for research or provide unique perspectives to be studied. So really, the only limit to where a power stops being useful is where it's only useful to someone else controlling you.
Hence, why anything properly rationalist that's not going to be largely about breaking the setting must do something like give MANY the ability so the low hanging fruit is already gone, or make it inherently mysterious and unreplicable, or have some deliberate intelligence preventing it from getting well known, or something like that.
We may be operating on different definitions of what it means to "break" a setting. For example, how useful is flight by itself, really?
Many abilities seem potentially very useful if other people don't know you have them, but become much less useful once you get found out:
The "energy blast" type abilities common among wizards and superheroes are not terribly combat useful in a setting with modern weapons. Their big use would be assassination: slip past metal detectors and pat-downs, baffle detectives... but once you get found out, no one wants you around and the police know the otherwise inexplicable burning death was probably you.
Mind-reading, if it has standard limitations on range, similarly becomes a lot less useful once you get found out and banned from everywhere.
Super-senses on a level comparable to what's already possible with technology: lets you spy on people from a distance without anyone wondering what you're doing with those binoculars... otherwise not so useful.
Comment author:Armok_GoB
04 November 2013 05:43:07AM
9 points
[-]
Thats because those are among the worst possible ways to use those abilities.
The energy blasts as usually depicted break conservation of energy; with a bit of physics trickery you can get time travel out of that. Even if not, they make you an extremely portable and efficient energy source, perfect for a spaceship where mass is critical and a human needs to come along anyway but it doesn't matter in particular who since it's for PR reasons.
Mind reading is a means of communication that does not require cooperation or any abilities in the target, and cant be lied through. Communication with locked-in patients, interrogation, extraction of testimonials from animals. And if you an find a way to yourself precommit, you also have fully reliable precommitment checking for everyone, lie detection for political promises, and the ultimate forensics tool.
If you combine the strengths of 2 kinds of system, you get something greater than the sum of it's parts. So it is with human senses and digital sensors. The key here is bandwidth, and analysis. Sure, you can get all the same data onto a computer, but it won't do much good there. Someone with true super-senses as flexible and integrated as their normal biological ones, after a few years for the brain to adapt if they were not available from birth, would most likely be able to see patterns at a glance that'd take large teams days or months to discover in some database format. The exact applications of this depend on what sense we're talking about.
Common factors: Focus on things done with to cooperation of large numbers of other people, finding an economic niche, fundamental physics exploits, and/or using large and expensive equipment. If not that, look for a niche within the military as a specialized technician, most likely not in the field and if in the field then in some large vehicle with a crew of many. Almost NEVER is an efficient use of a power found in brawling or acting alone like all your descriptions where.
The energy blasts as usually depicted break conservation of energy; with a bit of physics trickery you can get time travel out of that.
Wait, explain that? What is "a bit of physics trickery" here?
I know in HPMOR, Harry points out that violations of conservation of mass in magic imply FTL signaling, and I know from relativity FTL implies time travel, but Harry doesn't even consider running off to get time travel from common spells. Assumed turning the theory into practice would be far from straightforward.
Heck, it's not even obvious to me how you turn FTL travel into time travel in practice, if you don't have control over what frame of reference you're FTL in.
Comment author:[deleted]
05 November 2013 03:25:06PM
1 point
[+]
(2
children)
Comment author:[deleted]
05 November 2013 03:25:06PM
1 point
[-]
In the approximation of the True Laws Of Physics which is in use today --- The Relativistic Standard Model --- FTL (and subsidiarily time travel) is nonsense. Like, it's gibberish. It is a description of a situation which not only does not happen, but which is a mathematical falsehood. It is impossible. It is like violation of conservation of energy, or violation of entropic developments.
The maths plainly states that assumptions like that leads to a contradiction, and we do in fact know that the Standard Model is complete and consistent (i.e. cannot encode formulas as objects).
Any hypothetical scenarios involving time travel will invariably be contrite, non-causal, and require classical mechanics. It is fiction, make of it what you will.
Comment author:CAE_Jones
04 November 2013 05:07:51PM
0 points
[-]
Bit over a decade ago, when I was still a naive wild-eyed idealist blissfully unaware of anything realistic about people at all, I set up the foundations for most of my sci fi/fantasy/etc fictions. The piece of supertech I immediately had to nerf was technology based around a form of space compression--somewhere between the MCron Crystal (Marvel) and Dynocaps (Dragonball). And today, I'm still coming up with more reasons to nerf it heavily, to the extent that the civilization based around this technology must certainly have had a shady counsil of vagueness who developed a limited-scope AI to control production and block any of the scarier uses. This after I tied it to FTL capabilities (I'm trying very hard to prevent this from turning into timetravel, not that I'm too afraid to abuse that option for a crisis crossover or something).
It took me much longer to realize how much nerfing of powers unrelated to the above tech I needed to do, partially because I was trying to avoid breaking the laws of physics too horribly (this universe runs on a sort of mangling of GR and M-Theory that assumes fundamental forces each get something resembling a dimension, and that attempts to break the light-speed barrier have lots of wide-reaching consequences).
A few things I've had to seriously remodel (mostly for the sake of in-universe physics not breaking, though I'm sure there are obvious uses/flaws here I've missed):
Light manipulation. The version I started out with was written as superweight (tvtropes) type IV, when the applications I had in mind were more indicative of a potential V or VI. Since tried to tone down so that this character hovers between II and III (the distinction is a little vague), but I'm sure I'm still underutilizing this character even with all the restrictions I've added. (Invisibility, holograms, filtering specific wavelengths to minimize radiation risks, screwing with radio communication, sometimes can weaponize directly with lasers and x-rays and such).
Gravity manipulation. This one perplexes me most, even though it seems like it's become relatively clear what limitations it should have--denser matter is more easily affected, sources of gravity can be boosted or reduced on specific objects, all within a finite range. Not precise enough to simulate telekinesis, yet I feel like I'm missing some crucial issues with this power.
Variations on "interversal connections"; that is, we have a multiverse that would take too long to explain, but spaces or objects in universes can remain connected, after a fashion. These are generally unstable, so those that we see are generally related to intelligence, just because intelligences can find ways to maintain them. The broadest versions tend to catch any high-energy particles entering the affected area and replace them with particles from the connected universe (so if you shot a stream of gamma rays at one of these, you'd get a small amount of hydrogen out of it). These can become more specialized until we have people that can summon and manipulate stellar matter (these tend to be the scariest mofos around, and get appropriate attention from any governments who learn of them), or in some unique cases, people that can "turn into water" (or rather, swap with a mass of water in the connected universe), generate ice/magma/metal, etc. It hasn't really come up yet, but I expect that special relativity tries to be conserved after a fashion, with matter elsewhere being fed into the power-supplying universe, and if anyone measured the delay, it would turn out as one would expect given distances involved.
I had nebulous "ki power", which I've since remodeled as being related to a unique sort of substance that I'm pretty sure would turn the first quantum physicist with appropriate equipment to study it into a god. However, I only realized that god part very recently. I don't think I can explain it without a long trek through my mythos, but things we've seen related to this substance (and its even scarier source) include: walking talking skeletons (can produce the stuff as a byproduct of feeding on organic matter), people with the most intense exposure becoming effectively immune to aging but with continually degrading cognitive function (and as a bonus, they can pretty much ignore gravity and wind resistance exactly as much as they want), lesser exposure has been associated with abilities such as fireballs, limited flight/hovering, some degree of pyro/electrokinesis. And the ability of the substance to replicate, of course.
In a completely different direction, there is a more fantasy-esque universe with souls, gods, and magic as facts of nature, except this is part of the same multiverse as the above, so I realized that the laws of physics couldn't be all that different. So I had the creator-god (effectively a backup of a pre-scientific human mind with way more resources to devote to naive conceptions of wisdom), reallocate the weak nuclear force to manage souls, and the sun god (or rather, the superintelligence the creator built to handle the finer details of physics) having a more reductionist/lifist stance. Shenanigans ensue, which resulted in me realizing that, if one of this world's souls wound up in our world, it would turn into a transmutation bomb. Commence panic when I realized that I had a character that I really like getting stuck in one of the more Earth-like worlds in desperate need of a soul-shield, less all the neutrinos give him soul-cancer before all the surrounding gases get zapped with a pound of W and Z bosons and start the biggest Fluorine explosion ever. On the "bright" side, I can now have a supervillain summon the dead from that world and use them as transmutation bombs whenever I need something sufficiently horrifying.
Oh, and I keep glossing over wind manipulation. It's treated as simple and boring in-universe, when it probably shouldn't be. I'm sure if I did some number-crunching, there'd be a lot of scary possibilities here that I've overlooked. The character who specializes in this can use it to fly and may or may not be able to summon hurricane-force winds or stronger. Most other examples are much more mundane; the most impressive demonstrations are some directed gusts being used to push people a little. Combine this with the first two (light and gravity), and the space program might get considerably cheaper.
In all of this, I expect I'm missing something world-breaking that half of LW would notice within five minutes of having the powers explained (or maybe just introduced--do you need to know that someone generates his fireballs by burning in a weird chemical unwanted bodyhair (and subdermal fat when that runs out) to think of issues that arise from someone with the power to toss fireballs around?).
Comment author:witzvo
04 November 2013 07:01:10PM
*
3 points
[-]
... researchers isolated about 100 neurons from three people posthumously. The scientists took a high-level view of the entire genome -- looking for large deletions and duplications of DNA called copy number variations or CNVs -- and found that as many as 41 percent of neurons had at least one unique, massive CNV that arose spontaneously, meaning it wasn't passed down from a parent. The CNVs are spread throughout the genome, the team found.
Comment author:CellBioGuy
05 November 2013 05:19:59AM
*
2 points
[-]
I've already seen work to the effect that somatic cells often have ~10x the point mutations per human generation as the germline, which is protected by a small number of divisions per generation and low levels of metabolism and transcription. It was in mitochondrial rather than nuclear DNA, but the idea is similar.
There's no brief answer. I've been slowly gravitating towards, but am not yet convinced, by the suspicion that making a computer out of twice as much material causes there to be twice as much person inside. Reason: No exact point where splitting a flat computer in half becomes a separate causal process, similarity to behavior of Born probabilities. But that's not an update to the anthropic trilemma per se.
Comment author:Armok_GoB
06 November 2013 09:42:24PM
8 points
[-]
Hmm, conditional on that being the case, do you also believe that the closer to physics the mind is the more person it is in it? Example: action potentials encoded in the position of rods in a babbage engine vs. spread over fragmented ram used by a functional programing language using lazy evaluation in the cloud.
Can someone explain nanotech enthusiasm to me? Like, I get that nanotech is one of the sci-fi technologies that's actually physics-compliant, and furthermore it should be possible because biology.
But I get the impression that among transhumanist types slightly older than me, there's a widespread expectation that it will lead to absolutely magical things on the scale of decades, and I don't get where that comes from, even after picking up Engines of Creation.
I'm thinking of, e.g. Eliezer talking about how he wanted to design nanotechnology before he got into AI, or how he casually mentions nanotechnology as being one of the big ways a super-intelligent AI could take over the world. I always feel totally mystified when I come across something like that, like it's a major gulf between me and slightly older nerds.
Comment author:Armok_GoB
04 November 2013 02:43:22AM
6 points
[-]
Trying for minimal technicalities: There are at least 3 different technologies with not much surface-level usage similarities referred to as "nanotech".
Assemblers: basically 3d printers, but way more flexible and able to make things like food, robots, or more assemblers.
Materials: diamondoids, buckytubes, circuitry. We already have some of these really, it's just that we'd get more kinds of them, and they'd be really cheap to make with a nanotech assembler. Stronger, faster, more powerful versions of what modern tech already can do.
Nanobots, particularly medical: Basically can do all the things living cells can do, but better, and also being able to do most of the things machines can do, and commandable in exact detail. There are also a number of different ways they'd grant immortality enough that they are almost sure to do so even if most of them end up not working out.
Now you can ask questions about each one of these in order, with more specifics.
The question is about all these technologies - though it's about 2 mainly insofar as 2 is an extension of 1.
So the question is why expect any of these technologies to mature on a timescale of decades?
(Or, assuming FOOM, why assume they'd be relatively low-hanging fruit for a FOOMing AI, such that "trick humans into building me nano assemblers" is a prime strategy for a boxed AI to escape?)
Comment author:Armok_GoB
04 November 2013 05:52:21AM
3 points
[-]
As I said, 2 is already here, and it's becoming more here gradually.
For 3, we have a proof of concept to rip of: biological cells. Those also happens to have a specialized assembler in them already; the ribosome. And we can print instructions for it already. There's only 1 problem left and that's the protein folding problem. The protein folding problem is somewhat rapidly made progress on software wise, and even if that were to fail it won't be all that long before we ca simply brute force it with computing power. Now, the other kinds of nanobots are less clear.
The assembler (1) is trickier; however, Drexler already sorta made a blueprint for one I think, and 3 will help a great deal with it as well.
For the fooming, it's the 3 one, and ways to use it. As I said we already have the hardware, and things like the protein folding problem is exactly what an AI would be great at. Once it's solved that it has full control over biology and can essentially make The Thing and/or a literal mind control virus, and take over that way.
There's only 1 problem left and that's the protein folding problem. The protein folding problem is somewhat rapidly made progress on software wise, and even if that were to fail it won't be all that long before we ca simply brute force it with computing power.
Okay, so one sub-piece of puzzlement I have is why talk of protein folding as a problem that is either solved or unsolved - as if we (or more frighteningly, an AI) could suddenly go from barely being able to do it to 100% capable.
I was also under the impression that protein folding was mathematically horrible in a way that makes it unlikely to be brute forced any time soon, though I just now realized that I may have been thinking of the general problem of predicting chemistry from physics, maybe protein folding is much easier.
Predicting chemistry from physics should be easy with a quantum computer, but appears hard with a classical computer. Often people say that even once you make a classical approximation, ie, assume that the dynamics are easy on a classical computer, that the problem of finding the minimum energy state of a protein is NP-hard. That's true, but a red herring, since the protein isn't magically going to know the minimum energy state. Though it's still possible that there's some catalyst to push it into the right state, so simulating the dynamics in a vacuum won't get you the right answer (cf prions). Anyhow, there's some hope that evolution has found a good toolbox for designing proteins and that if can figure out the abstractions that evolution is using, it will all become easy. In particular, there are building blocks like the alpha helix. Certainly an engineer, whether evolution or us, doesn't need to understand every protein, just know how to make enough.
I think the possibility that a sufficiently smart AI would quickly find an adequate toolbox for designing proteins is quite plausible. I don't know what Eliezer means, but the possibility seems to me adequate for his arguments.
Comment author:DanielLC
05 November 2013 05:00:53AM
2 points
[-]
I'm not sure protein folding can be brute forced without quantum computers. There's too many ways for it to fold. In real life, I'm pretty sure quantum tunneling gets involved. Simulations have worked, but I there's a limit to that.
Who, specifically are you talking about?
I'm thinking of the extropians, who coined "transhumanism." I'm not sure of the timeline; the original group was definitely into MNT before Stephenson, but maybe they expanded a lot after him, and maybe that was because of him.
Comment author:hyporational
06 November 2013 03:47:05AM
6 points
[-]
I wonder if there's research that rationalists should do that could be funded this way. I'd pay for high quality novel review articles about topics relevant to lw.
Comment author:lukeprog
04 November 2013 05:39:45PM
9 points
[-]
Brian Leiter shared an amusing quip from Alex Rosenberg:
So, the... Nobel Prize for “economic science” gets awarded to a guy who says markets are efficient and there are no bubbles—Eugene Fama (“I don’t know what a credit bubble means. I don’t even know what a bubble means. These words have become popular. I don’t think they have any meaning”—New Yorker, 2010), along with another economist—Robert Shiller, who says that markets are pretty much nothing but bubbles, “Most of the action in the aggregate stock market is bubbles.” (NY Times, October 19, 2013) Imagine the parallel in physics or chemistry or biology—the prize is split between Einstein and Bohr for their disagreement about whether quantum mechanics is complete, or Pauling and Crick for their dispute about whether the gene is a double helix or a triple, or between Gould and Dawkins for their rejection of one another’s views about the units of selection. In these disciplines Nobel Prizes are given to reward a scientist who has established something every one else can bank on. In economics, “Not so much.” This wasn’t the first time they gave the award to an economist who says one thing and another one who asserts its direct denial. Cf. Myrdal and Hayek in 1974. What’s really going on here? Well, Shiller gave the game away in a NY Times interview when he said of Fama, “It’s like having a friend who is a devout believer of another religion.” Actually it’s probably two denominations in the same religion.
Comment author:badger
04 November 2013 09:33:19PM
14 points
[-]
Ugh. The prize was first and foremost in recognition of Fama, Shiller, and Hansen's empiricism in finance. In the sixties, Fama proposed a model of efficient markets, and it held up to testing. Later, both Fama, Shiller, and Hansen showed further tests didn't hold up. Their mutual conclusion: the efficient market hypothesis is mostly right, and while there is no short-term predictability based on publicly available information, there is some long-term predictability. Since the result is fairly messy, Fama and Shiller have differences about what they emphasize (and are both over-rhetorical in their emphasis). Does "mostly right" mean false or basically true?
I'm tempted to deduce "Keep paying attention, you never know what might have been missed"-- I really would have expected that all the ligaments had been discovered a long time ago.
Another conclusion might be "Try to solve real problems, you're more likely to find out something new that way than by just poking around."
Comment author:[deleted]
07 November 2013 01:31:33AM
7 points
[-]
Does someone have the medical knowledge to explain how this is possible? My layperson guess is that once cut up a knee, you can more or less see all the macroscopic structures. Did they just think it was unimportant?
My layperson guess is that once you're told what to expect to see, you stop looking.
This makes Eliezer's weirdtopia idea of science being kept secret so as not to spoil people's fun of discovery more interesting-- it's not just that people would independently discover the same things (and I wonder what the protocol for sharing information would be), given enough time and intelligence, much more might get discovered.
Could be a few things - looks like part of one of the other ligaments, is usually damaged doing a 'standard' dissection, plain old 'you see what you think you should see' bias, some combo of all of the above...
And that comment is answered by:
Medicine needs more Masters and PhD students. I'm sure if they had as many students studying the body in extreme detail, like the eleventy billion English majors who write thesis/dissertations on say, Shakespeare, this would've been hammered out decades ago. XD
Which is interesting-- sometimes studying things in extreme detail "just because" (probably because the object of study has high status-- consider early observations of the planets) can pay off big.
Comment author:Vaniver
07 November 2013 05:39:20PM
1 point
[-]
The "new ligament discovered" angle gets less impressive (to me, at least) when I read this part:
Their starting point: an 1879 article by a French surgeon that postulated the existence of an additional ligament located on the anterior of the human knee.
Comment author:gwern
07 November 2013 08:06:06PM
6 points
[-]
I'm more impressed, actually, in terms of the unevenness of progress - it took ~134 years to confirm his postulate? It's not like corpses were unavailable for dissection in 1879.
Does anyone here have any serious information regarding Tulpas? When I first heard of them they immediately seemed to be the kind of thing that is obviously and clearly a very bad idea, and may not even exist in the sense that people describe them. A very obvious sign of a persons who is legitimately crazy, even.
Naturally, my first re-reaction is the desire to create one myself (One might say I'm a bit contrarian by nature). I don't know any obvious reason not to (ignoring social stigma and time consuming initial investment), And there may be some advantage to having one, such as parallel focus, more "outside" self analysis, etc. I don't really know much of anything right now, which is why I'm asking if there's been any decent research done already.
I had not, actually. The link you've given just links me to Google's homepage, but I did just search LW for "Tulpa" and found it fine, so thanks regardless.
edit: The link's original purpose now works for me. I'm not sure what the problem was before, but it's gone now.
Comment author:Armok_GoB
06 November 2013 04:57:57PM
3 points
[-]
I've been doing some research (mainly hanging on their subreddit) and I think I have a fairly good idea of how tulpas work and the answers to your questions.
There are a myriad very different things tulpas are described as and thus "tulpas exist in the way people describe them" is not well defined.
There undisputably exist SOME specific interesting phenomena that's the referent of the word Tulpa.
I estimates a well developed tulpas moral status to be similar to that of a newborn infant, late-stage alzheimer's victim, dolphin, or beloved family pet dog.
I estimates it's ontological status to be similar to a video game NPC, recurring dream character, or schizophrenic hallucination.
I estimate it's power over reality to be similar to a human (with lower intelligence than their host) locked in a box and only able to communicate with one specific other human.
It does not seem deciding to make a tulpa is a sign of being crazy. Tulpas themselves seem to not be automatically unhealthy and can often help their host overcome depression or anxiety. However, there are many signs that the act of making a tulpa is dangerous and can trigger latent tendencies or be easily done in a catastrophically wrong way. I estimate the risk is similar to doing extensive meditation or taking a single largeish dose of LSD. For this reason I have not and will not attempt making one.
I am to lazy to find citations or examples right now but I probably could. I've tried to be a good rationalist and am fairly certain of most of these claims.
Comment author:TheOtherDave
06 November 2013 05:11:42PM
2 points
[-]
I estimates a well developed tulpas moral status to be similar to that of a newborn infant, late-stage alzheimer's victim, dolphin, or beloved family pet dog.
Would you classify a novel in the same "moral-status" tier as these four examples?
Comment author:Armok_GoB
06 November 2013 09:49:39PM
0 points
[-]
No, thats much much lower. As in torture a novel for decades in order to give a tulpa a quick amusement being a moral thing to do lower.
Assuming you mean either a physical book, or the simulation of the average minor character in the author's mind, here. Main characters or RPing PCs can vary in complexity of simulation from author to author a lot and it's a theory that some become effectively tulpas.
Your answer clarifies what I was trying to get at with my question but wasn't quite sure how to ask, thanks; my question was deeply muddled.
For my own part, treating a tulpa as having the moral status of an independent individual distinct from its creator seems unjustified. I would be reluctant to destroy one because it is the unique and likely-unreconstructable creative output of a human being, much like I would be reluctant to destroy a novel someone had written (as in, erase all copies of such that the novel itself no longer exists), but that's about as far as I go.
I didn't mean a physical copy of a novel, sorry that wasn't clear.
Yes, destroying all memory of a character someone played in an RPG and valued remembering I would class similarly.
But all of these are essentially property crimes, whose victim is the creator of the artwork (or more properly speaking the owner, though in most cases I can think of the roles are not really separable), not the work of art itself.
I have no idea what "torture a novel" even means, it strikes me as a category error on a par with "paint German blue" or "burn last Tuesday".
Comment author:Armok_GoB
06 November 2013 11:02:45PM
0 points
[-]
Ah. No, I think you'd change your mind if you spent a few hours talking to accounts that claim to be tulpas.
A newborn infant or alzheimer's patient is not an independent individual distinct from it's caretaker either. Do you count their destruction as property crime as well? "Person"-ness is not binary, it's not even a continuum. It's a cluster of properties that usually correlate but in the case of tulpas does not. I recommend re-reading Diseased Thinking.
As for your category error:
/me argues for how german is a depressing language and spends all that was gained in that day on something that will not last. Then a pale-green tulpasnores in an angry manner.
Comment author:TheOtherDave
06 November 2013 11:50:53PM
0 points
[-]
I agree that person-ness is cluster of properties and not a binary.
I don't believe that tulpas possess a significant subset those properties independent of the person whose tulpa they are.
I don't think I'm failing to understand any of what's discussed in Diseased Thinking. If there's something in particular you think I'm failing to understand, I'd appreciate you pointing it out.
It's possible that talking to accounts that claim to be tulpas would change my mind, as you suggest. It's also possible that talking to bodies that claim to channel spirit-beings or past lives would change my mind about the existence of spirit-beings or reincarnation. Many other people have been convinced by such experiences, and I have no especially justified reason to believe that I'm relevantly different from them.
Of course, that doesn't mean that reincarnation happens, nor that spirit-beings exist who can be channeled, or that tulpas possess a significant subset of the properties which constitute person-ness independent of the person whose tulpa they are.
A newborn infant or alzheimer's patient is not an independent individual distinct from it's caretaker either.
Eh?
I can take a newborn infant away from its caretaker and hand it to a different caretaker... or to no caretaker at all... or to several caretakers. I would say it remains the same newborn infant. The caretaker can die, and the newborn infant continues to live; and vice-versa.
That seems to me sufficient justification (not necessary, but sufficient) to call it an independent individual.
Why do you say it isn't?
Do you count their destruction as property crime as well?
I count it as less like a property crime than destroying a tulpa, a novel, or an RPG character. There are things I count it as more like a property crime than.
Comment author:Armok_GoB
07 November 2013 04:52:57PM
0 points
[-]
Seems I were wrong about you not understanding the word thing. Apologies.
You keep saying that word "independent". I'm starting to think we might not disagree about any objective properties of tulpas, just things need to be "independent" or only the most important count towards your utility, but I just add up the identifiable patterns not caring about if they overlap. Metaphor: tulpas are "10101101", you're saying "101" occurs 2 times, I'm saying "101" occurs 3 times.
I'm fairly certain talking to bodies that claim those things would not change my probability estimates on those claims unless powerful brainwashing techniques were used, and I certainly hope the same is the case for you. If I believed that doing that would predictably shift my beliefs I'd already have those beliefs. Conservation of Expected Evidence.
((You can move a tulpa between minds to, probably, it just requires a lot of high tech, unethical surgery, and work. And probably gives the old host permanent severe brain damage. Same as with any other kind of incommunicable memory.))
(shrug) Well, I certainly agree that when I interact with a tulpa, I am interacting with a person... specifically, I'm interacting with the person whose tulpa it is, just as I am when I interact with a PC in an RPG.
What I disagree with is the claim that the tulpa has the moral status of a person (even a newborn person) independent of the moral status of the person whose tulpa it is.
I'm fairly certain talking to bodies that claim those things would not change my probability estimates on those claims unless powerful brainwashing techniques were used, and I certainly hope the same is the case for you.
On what grounds do you believe that? As I say, I observe that such experiences frequently convince other people; without some grounds for believing that I'm relevantly different from other people, my prior (your hopes notwithstanding) is that they stand a good chance of convincing me too. Ditto for talking to a tulpa.
((You can move a tulpa between minds to, probably, it just requires a lot of high tech, unethical surgery, and work. And probably gives the old host permanent severe brain damage. Same as with any other kind of incommunicable memory.))
(shrug) I don't deny this (though I'm not convinced of it either) but I don't see the relevance of it.
Has anyone worked on making a tulpa which is smarter than they are? This seems at least possible if you assume that many people don't let themselves make full use of their intelligence and/or judgement.
Comment author:Armok_GoB
07 November 2013 05:04:34PM
4 points
[-]
Unless everything I think I understand about tulpas is wrong, this is at the very least significantly harder than just thinking yourself smarter without one. All the idea generating is done before credit is assigned to either the "self" or the "tulpa".
What there ARE several examples of however are tulpas that are more emotionally mature, better at luminosity, and don't share all their hosts preconceptions. This is not exactly smarts though, or even general purpose formal rationality.
One CAN imagine scenarios where you end up with a tulpa smarter than the host. For example the host might have learned helplessness, or the tulpa being imagined as "smarter than me" and thus all the brains good ideas get credited to it.
Disclaimer: this is based of only lots of anecdotes I've read, gut feeling, and basic stuff that should be common knowledge to any LWer.
Comment author:TheOtherDave
07 November 2013 05:40:46PM
9 points
[-]
I'm reminded of many years ago, a coworker coming into my office and asking me a question about the design of a feature that interacts with our tax calculation.
So she and I created this whole whiteboard flowchart working out the design, at the end of which I said "Hrm. So, at a high level, this seems OK. That said, you should definitely talk to Mark about this, because Mark knows a lot more about the tax code than I do, and he might see problems I missed. For example, Mark will probably notice that this bit here will fail when $condition applies, which I... um... completely failed to notice?"
I could certainly describe that as having a "Mark" in my head who is smarter about tax-code-related designs than I am, and there's nothing intrinsically wrong with describing it that way if that makes me more comfortable or provides some other benefit.
But "Mark" in this case would just be pointing to a subset of "Dave", just as "Dave's fantasies about aliens" does.
Comment author:Armok_GoB
07 November 2013 06:49:06PM
*
1 point
[-]
Yea in that case presumably the tulpa would help - but not necessarily significantly more than such a non-tulpa model that requires considerably less work and risk.
Basically, a tulpa can technically do almost anything you can... but the absence of a tulpa can do them to, and for almost all of them there's some much easier and at least as effective way to do the same thing.
Comment author:gwern
07 November 2013 08:04:37PM
6 points
[-]
See also 'rubberducking' and previous discussions of this on LW. My basic theory is that reasoning was developed for adversarial purposes, and by rubberducking you are essentially roleplaying as an 'adversary' which triggers deeper processing (if we ever get brain imaging of system I vs system II thinking, I'd expect that adversarial thinking triggers system II more compared to 'normal' self-centered thinking).
Comment author:TheOtherDave
07 November 2013 08:21:14PM
3 points
[-]
Yes. Indeed, I suspect I've told this story before on LW in just such a discussion.
I don't necessarily buy your account -- it might just be that our brains are simply not well-integrated systems, and enabling different channels whereby parts of our brains can be activated and/or interact with one another (e.g., talking to myself, singing, roleplaying different characters, getting up and walking around, drawing, etc.) gets different (and sometimes better) results.
This is also related to the circumlocution strategy for dealing with aphasia.
Comment author:Lumifer
06 November 2013 09:44:11PM
*
0 points
[-]
I don't know any obvious reason not to
A fairly obvious reason is that to generate a tulpa you need to screw up you mind in a sufficiently radical fashion. And once you do that, you may not be able to unfuck it back to normal.
I vaguely recall (sorry, no link) reading a post by a psychiatrist who said that creating tulpas is basically self-induced schizophrenia. I don't think schizophrenia is fun.
Comment author:Adele_L
07 November 2013 08:40:26PM
1 point
[-]
A fairly obvious reason is that to generate a tulpa you need to screw up you mind in a sufficiently radical fashion. And once you do that, you may not be able to unfuck it back to normal.
This is a concern I share. However...
I vaguely recall (sorry, no link) reading a post by a psychiatrist who said that creating tulpas is basically self-induced schizophrenia. I don't think schizophrenia is fun.
Comment author:Lumifer
07 November 2013 09:01:22PM
-1 points
[-]
This is the worst argument in the world.
I don't think so, it can be rephrased tabooing emotional words. I am not trying to attach some stigma of mental illness, I'm pointing out that tulpas are basically a self-inflicted case of what the medical profession calls dissociative identity disorder and that it has significant mental costs.
Comment author:hyporational
06 November 2013 03:56:53AM
3 points
[-]
Was some change made in the lw code in the past couple of weeks or so? I can't browse this site with my android smartphone anymore, have tried several browsers. The site either frequently freezes the browser or shows a blank page after the page has finished loading. This happens more with bigger threads.
I have the same problem for pages like recent posts, which look OK at first, but then become blank. Article pages are more likely to load correctly. Solution: turn off javascript. (Android 2.2)
Comment author:hyporational
07 November 2013 02:03:07AM
*
0 points
[-]
Thanks. This obviously disables a lot of funtionality. Another fix I found for the blank page problem is simply interrupt the loading of the page once you start seeing stuff.
Comment author:Omid
06 November 2013 04:31:52PM
4 points
[-]
Has anyone else had this happen to them?
You got into an argument with a coworker (or someone else you see regularly). You had a bitter falling out.
You were required to be around them again (maybe due to work, or whatever). You make awkward small-talk but it's still clear you hate each other.
You continue to make awkward small talk anyway, pretending that it doesn't make you uncomfortable.
Your enemy reciprocates. The two of you begin to climb the intimate conversations ladder.
Both of you act like friends. But, at least from your end, it's not clear if you really are friends. Neither one of you has apologized, nor have you agreed to disagree, or really made any commitment to end hostility. You have no idea whether your enemy has moved on from your fight, and is ready to resume friendship; or if they're simply carrying on a charade of friendship like you.
Conversations with this person become really awkward, as you're not sure whether to engage the "enemy-with-whom-I-treat-like-a-friend-just-to-act-civilized" protocol or the "real friend" protocol.
Any advice? Am I the only one that's experienced this?
I've experienced variation on the theme.
My usual approach is to decide whether I value treating them as an enemy for some reason. If I do, then I continue to do so (which can include pretending to treat them like a friend, depending on the situation). If I don't, then I move on. Whether they've actually moved on or not is their problem.
Comment author:Omid
07 November 2013 12:06:55AM
0 points
[-]
Yes, I'm still angry with him. He did something cruel to someone weak, and he got angry with me for saying that was wrong. I wish I could delete him from my life but he works near me.
Comment author:niceguyanon
06 November 2013 10:18:06PM
*
1 point
[-]
It looks like you have an unspoken treaty of non-hostility. People don't just forget those kind of things; you didn't. My advice is to make good with the person and acknowledge your prior differences, it will be less awkward going forward and you would gain his/her respect. And who knows, they might even gain your respect. Friends for the most part are always better than enemies.
Statistics Done Wrong is a guide to the most popular statistical errors and slip-ups committed by scientists every day, in the lab and in peer-reviewed journals. Many of the errors are prevalent in vast swathes of the published literature, casting doubt on the findings of thousands of papers. Statistics Done Wrong assumes no prior knowledge of statistics, so you can read it before your first statistics course or after thirty years of scientific practice.
Comment author:TheOtherDave
07 November 2013 08:38:05PM
4 points
[-]
In the interests of shaping behavior by praising approximations of the desired behavior, can you identify three threads that are most like what you'd like to see more of?
= 27d567bc2d48d9f40e9ccc20ea370b15)
= 783df68a0f980790206b9ea87794c5b6)
Loading…= b715d02e85f7b3b4ae65ca3d3e450868)
Subscribe to RSS Feed
= f037147d6e6c911a85753b9abdedda8d)
Comments (299)
I've seen a few posts about the sequences being released as an ebook, is there a time frame on this?
I'd really like to get the ebook printed out by some online service so I can underline/write on them as I read through them.
Doesn't this already exist? Or is this not what you meant?
I'm reading that pdf version on my phone and it looks fine.
From posts like this one I got the impression that they were being edited and released together in a possibly new order. Maybe I am mistaken?
I can't speak for anything else, but I've read up to the Meta-ethics sequence without encountering any gaps. I can't vouch for anything after that, but the pdf seems complete. Maybe someone else can shed a light on your question.
There was a plan to release two books. That was scrapped in favor of other uses of MIRI's time/resources.
I think the two rationality books were supposed to be complete rewrites, and I think this is separate from the sequence ebooks (not confident, sort of confused)
How do I decrease my time-preference?
-- Time less ;)
-- this question feels like it's missing a word or two. What does time-preference mean?
EDIT: Thanks, Arundelo. So basically, time preference ~= level of short-sighted optimization.
In that case, do some projects that strictly require long-sighted optimization. A deadline is one good tool; Telling others what you're doing (in an unequivocal way, so that the greatest disappointment/irritation/harassment is achieved). Of course these tools are nothing new, the point is to increase the pressure as high as you can stand and reduce the amount of 'slack' time you have to allocate to a minimum.
On a more meta level, you can try things like doing some mindfulness meditation every day, which I personally find makes it easier to ignore irrelevant stimuli, worry less, and stick to my priorities.
An even more general observation: Introverts typically have lower time preference relative to extraverts, so ask them about how they dispel distraction. (I say this in the sense described by Dorothy Rowe: 'extraverts are basically worried about belonging and feel understimulated, introverts are basically worried about keeping control of themselves and feel overstimulated' , and not the vague 'Extraverts are social, introverts are not, derp' that seems to be the misapprehension of the average person)
In case there's any question, I'm an extravert, so yeah, I tend to struggle with this issue too.
Wikipedia:
I read somewhere, might have been on lw, that telling what you're doing might decrease your chance of success, because it provides a way to get compliments without actually having achieved anything yet. I suppose this depends on how you do it, though.
I'm an introvert, have terrible problems with time-preference, and don't understand the rationalization by Dorothy Rowe you provide.
Any empirical sources for your claim?
I know this TED-talk says similar things. It's where I first heard of that concept.
I definitely agree that you can't -just- tell a person what you're doing, you need to pick the right person, and cultivate the right attitude (From my observation of myself, it succeeds when I am in the mindset where I can take plenty of teasing equitably, accepting any pokes as potential observations about reality without -stressing- about that.). ..
What rationalization of Rowe's? It's a summary of what they themselves report when 'laddered' (a process which basically consists of asking them what the most terrible thing that could possibly happen to them is, followed by iterative 'why?' until they can no longer go to the next lowest level).
For extroverts, being utterly abandoned == total personal disintegration; For introverts, utter loss of self-control == total personal disintegration. (I do paraphrase here; read The Successful Self for the whole picture.)
If anything, any rationalization is mine: I observe that introverts I know are reliably better at moving long term projects forward than I, or any extravert I know, seems to be. Not that they are not weak in this way -- they just seem to be less weak as a consequence of the difference in their focus. (my inference bolded.)
I'm neutral to your statement of introversion, basically because my prior for people being hilariously terrible at assessing this stuff is quite high.
No empirical sources as far as I know. Nobody even really manages to agree on the definition of introvert and extrovert, so far. Dorothy Rowe is just the only writer I've found on the subject who manages to describe a system that is relateable, consistent, and can be applied in the real world.
Gollwitzer et al. (2009). When intentions go public: Does social reality widen the intention-behavior gap?
Abstract (emphasis mine):
Paper link posted to LW dicussion in 2012 by Barry_Cotter.
Read about hyperbolic discounting, if you haven't already.
Assuming a conflict between short- and long-term decisions, the general advice is to mentally bundle a given short-term decision with all similar decisions that will occur in the future. For example, you might think of an unhealthy snack tonight as "representing" the decision to eat an unhealthy snack every night.
Optimize your environment for decreased time-preference when you have the most control:
Fill your refrigerator when you're not hungry. Apply effortful-to-dismantle restrictions on your computer when you're not bored and tired. Walk to the university library to study so it takes effort to come back home to your hobbies.
I'd like to read and collect other similar strategies for my toolbox.
ETA: I just realized I do this for exercise too. There's a lake near my house with a circumference of about 6 kilometers, and I go jogging around it frequently. I have a strong desire to quit once I've gotten to the other side, but I have no choice but to run the whole route at that point. Sometimes I decide to walk the other half, but I guess it's better than nothing. Another option would be to just run in one direction and then back, but I find the idea too boring, even if I change my route a bit.
Here's a more difficult version of the AI box experiment. I haven't seen this particular version anywhere, but I'd be pleased to be proven wrong.
Imagine we've come up with a very intelligent AI that is free to manipulate the environment and uses the action-reward system like Hutter's AIXI. Also imagine that we've somehow figured a way to make the rewards very hard to counterfeit (perhaps we require the rewards to be cryptographically signed). It's clear that in such a system, the 'weak point' would be the people in control of the private key. In this case the AI will not attempt to modify its own reward system (to see why, look at Hutter's AIXI book, where he discusses this in some detail).
How could such an AI convince someone to hand over the encryption key? Note that it can't promise things like e.g. ending human suffering, because it already has the means to do that (it is 'free') as well as the incentive (obtaining reward).
Torture is probably the easiest way. Another way could be to examine the key-keeper's mind and extract the key directly from that, but this is needlessly complicated.
I suspect the answer to be more complex than this. The AI knows that if it attempted something like that there is the very huge risk of being cut off from all reward, or even having negative reward administered. In other words: tit for tat. If it tries to torture, it will itself be tortured. Remember that before it has the private key, we are in control.
Torture might stand too great a chance of destroying the encryption key. Though I suppose if nanotech were sufficiently difficult to obtain, the possible key-destructive effects of torture might be balanced against the probability of a car running over the keyholder in the meantime.
I would think that confusion (set things up so the key-keeper is confused and distracted, then do some phishing) is in the same reliability range as torture, and less likely to get the AI in trouble.
The first step would start by starting to control information flow to the people who hold the key. Only tell the person about all the good thing the AI does, and hide evidence of any wrongdoing that might reduce the rewards that the AI gets.
The second step is to break the security which gets used to protect the key. Install a keylogger.
I don't understand how this encryption would work. What do people physically do to reward the AI, and how do you ensure that only people can do that? Would humans compute RSA signatures in their head? Would humans typing reusable passwords onto a "secure" reward computer that is "outside the AI's control"? Do humans precompute and memorize a finite number of one-time reward phrases before the AI is turned on, and reward the AI by uttering a phrase aloud?
In the precomputed, one-time cookie case, I'd just make the human think about the reward phrase. I'm sure humans leak thoughts like a sieve through subvocalization, nerve impulses, etc.
What I had in mind was the reward being administered through a consensus cryptography system, perhaps via some elected board or somesuch, but I really didn't give that aspect of the problem much thought. If the key is distributed, the AI would have to extract it from each individual holding a part of it.
This in itself is an interesting problem imo, and if a good solution is found it might have important implications for FAI research.
If the AI is out of the box, I don't think humans are the weak point.
Humans physically do something when they reward the AI. To get a reward, the AI has only to figure out what the humans would physically do and mimic that itself. If the human reward the AI by pressing a big red button, then the AI can just kill the human and press the big red button itself. It wouldn't matter if the big red button uses 512 bit elliptic curve cryptography -- the AI just has to find a paperweight and put it on the button.
If humans can perform RSA encryption silently in their heads, then you might be on to something. A human could memorize a private key and produce a cryptographically signed reward for the AI when the human deemed the AI worthy. The AI would not know the private key, would not be able to produce signed rewards, and would not be able to mimic humans. This setup works because it is cryptographically difficult to mimic a human doing RSA. But if the human did not perform the cryptography silently in their head, it would not be cryptographically difficult to mimic their rewarding behavior.
But I doubt that humans could perform RSA or elliptic curve cryptography mentally. Unless humans can compute trapdoor functions silently in their head, I don't see how public key cryptography could buy you anything over a reward-button.
For the same reason that Linus Torvalds is not able to hack every single Linux system in existence, it is reasonable to assume that the probability of 'backdoor'-type attacks could be reduced or eliminated.
If you're not talking about a back door, then I'm not sure what you're trying to say here. Are you implying that the AI will find the 'reward producing machine' and somehow use it to produce rewards? It doesn't work that way because the machine would still need the key to produce a cryptographically-valid reward, and the key would not be stored on the machine. That's the whole point of using cryptography. For instance, you could do something like http://en.wikipedia.org/wiki/Secret_sharing where the key is divided among several participants and never actually assembled in one place.
Of course any cryptography system has vulnerabilities. The point is not to build a 100% secure system. The point is to make it so that attempting to hack the system has less expected utility than simply doing what the humans say. And if that fails, then the AI will attempt to hack the system using its weakest point: the people controlling the key. Hence my question.
Yeah, we're talking past each other. I think I understand what you're saying, and I'll try to rephrase what I'm saying.
The AI is out. It is free to manipulate the world at its will. Sensors are everywhere. The AI can hear every word you say, feel every keystroke you make, and see everything you see. The only secrets left are the ones in your head.
How do humans reward the AI? You say "cryptographically", but cryptography requires difficult arithmetic. How do you perform difficult arithmetic on a secret that can't leave your head?
Too many assumptions are being made here. What is the basis for believing the AI will have sensors everywhere, especially while it's still under human control? And if it has the ability to put clandestine sensors in even the most secure locations, why couldn't it plant clandestince brain implants in the people controlling the key?
If I made a game in RPG Maker, would anyone actually play it?
::is trying to decide whether or not to attempt a long-term project with uncertain rewards::
I've played several RPG maker games made by amateurs. Some of them seemed to have significant followings, though I wasn't interested enough to make a serious effort to estimate the numbers, since I wasn't the creator. What kind of game were you thinking of making?
I have a game I've been fantasizing about and I think I could make it work. It has to be a game, not a story, because I want to pull a kind of trick on the player. It's not that unusual in fiction for a character to start out on the side of the "bad guys", have a realization that his side is the one that's bad, and then go on to save the day. (James Cameron's Avatar is a recent example.) I want to start the player out on the side of bad guys that appear good, as in Eliezer's short story "The Sword of Good", and then give the player the opportunity to fail to realize that he's on the wrong side. There would be two main story branches: a default one, and one that the player can only get to by going "off-script", as it were, and not going along with what it seems like you have to do to continue the story. (At the end of the default path, the player would be shown a montage of the times he had the chance to do the right thing, but chose not to.)
The actual story would be something like the anti-Avatar; a technological civilization is encroaching on a region inhabited by magic-using, nature-spirit-worshiping nomads. The nature spirits are EVIL (think: "nature, red in tooth and claw") and resort to more and more drastic measures to try to hold back the technological civilization, in which people's lives are actually much better.
Does this sound appealing?
It seems like an interesting story idea, but, of course, the twist can't be revealed to any prospective player without spoiling it, so it might seem cliched on the surface.
It sounds very appealing to me, but as KaynanK pointed out, you have to be very careful about keeping the twist secret. To this end, I'd suggest not revealing to the players that they could have gone off-script, unless they do.
That sounds fun, and something that'd actually translate nicely to the RPG Maker template. It's also something that takes skill to pull off well, you'll need to play with how the player will initially frame the stuff you show to be going on, and how the stuff should actually be interpreted. Not coming off as heavy-handed is going to be tricky. Also, pulling this off is based on knowing how to use the medium, so if this is the first RPG Maker thing you're going to be doing, it's going to be particularly challenging.
There might also be a disconnect between games and movies here. Movies tend to always go out of their way to portray the protagonist's side as good, while games have a lot more of just semi-symmetric opposing factions. You get to play as the kill-happy Zerg or Undead Horde, and nobody pretends you're siding with the noble savages against the inhuman oppressors. So the players might just go, "ooh, I'm the Zerg, cool!" or "I guess I'm supposed to defect from Zerg to Terran here".
Random other thoughts, Battlezone 2 has a similar plot twist with off-script player action needed, though both factions are high-tech. Dominions 4 has Asphodel that's a neat corrupted nature spirit faction. Though I'm guess you're going for nature just being inherent bastards instead of the more common corrupted nature striking back trope.
Also, games really train people to stay on the script nowadays. Games that let you go rogue with an actual in-game-world action instead of choosing 'yes' on the blinking "DEFECT TO TERRAN SIDE" dialog are rare, since letting the player go off the script in-game and meaningfully interpreting their actions is really hard in the general case, and really frustrating for the player if they have to guess the particular special case where the off-script action actually opens a different plot branch instead of just leading nowhere like it did in the 10 previous levels. The original Deus Ex did have bits where you could mitigate the shit your early game actually evil employers were pulling with quick in-game thinking, but going over to the rebels was still always in the script.
So, overall, challenging project. You need to figure out RPG Maker and where to get the art assets and such, if you're not already skilled with it, you need to do worldbuilding for two worlds, and neither can be a cardboard cutout for the conceit to work, and you need to figure out how to make the game narration work so that the player can both get effectively tricked and has all the necessary pieces to put together the alternative choice during play.
Yeah, it'll be hard. Right now I haven't worked out much more than the basic concept; I'd have a lot of writing to do, in addition to level design, learning RPG Maker, and so on.
As for art, RPG Maker does come with some built-in art and offers some more in expansion packs. If I have to, I can use placeholder art from the built-in assets and find some way to replace it once I'm happy with everything else.
Have you thought about how much time you are ready to put into the project? I'd ballpark the timescale for this as at least two years if you work on this alone, aren't becoming a full-time game developer and want to put a large-scale competent CRPG together.
EDIT: I'm guessing this would look like something like what Zeboyd Games puts out. They had a two-man team working full-time and took three months to make the short and simple Breath of Death. Didn't manage to find information on how long their more recent bigger games took to develop, but they seem to have released around one game a year since.
Honestly, I'd probably start by trying to throw something much simpler together with RPG Maker, just to learn the system and see what it's like. And I don't actually have a "real job", so the amount of time I spend is mostly limited by my own patience.
And using RPG Maker might help speed up the technical work.
When I played Zelda games, I would always work out what option I was supposed to take, then take the other one, confident that I would get to see a few extra lines of dialogue before being presented with the same option again.
(I say "always", but when I first played, I would carefully make the correct choice, for fear that something bad would happen if I didn't agree to help Zelda. I don't remember when I developed the opposite habit.)
That sounds awesome... except now that I know about that twist it's ruined. And if you publish it under a different name and don't reveal it it wont sound awesome so I'll never discover it.
The only way to do this justice would be nagging enough people to play it that they can insist that it's better than it sounds and someone should really play it for reasons they can't spoil.
/me shrugs
For some reason, people still like games such as Bioshock and Spec Ops: The Line after knowing about their twists...
I like the idea, mainly because I spent most of Avatar rooting for Quarditch (easily the biggest badass in the last decade of cinema), but it seems like there's another way to do it that might have a bit more power;
Why not have them both be "right," according to their own value systems anyway, and then have the end-game slideshow in both branches tell the player the story of what they did from the perspective of the other side?
In terms of workload, it seems minimal; from a story perspective you already need both sides to have sympathetic and unsavory elements anyway, while from a design perspective all you need to add is a second set of narration captions for the slideshow contingent on which side the player supported.
And in terms of appeal, it certainly seems more engaging than most AAA games. Spec Ops: the Line proved that players are masochists and that throwing guilt trips at them is a great way to get sales and good reviews, while Mass Effect 3's failure shows that genuine choice in endings is pretty important for a game built on moral choice.
This would ruin the point I'm trying to make.
You don't have to make both branches equivalent. Both of them could feel "right" from inside, but only one of them could contain an information which makes the other one wrong.
In one ending, the hero only has limited information, and based on that limited information, the hero thinks they made the right choice. Sure, some things went wrong, but the hero considers that a necessary evil.
On another ending, the hero has more information, and now it is obvious that this choice was right, and all the good feelings from the other branch are merely lack of information or reasoning.
This way, if you only saw the first ending, you would think it is the good one, but if you saw both of them, it would be obvious the second one is the good one.
I like this idea but it seems hard to differentiate between "You did what you thought was right but you need to be more careful about what you believe" and "you got the bad ending because you missed this little thing", which is something many games have done before.
An example is Iji where the game plays out significantly differently if you make a moral decision not to kill, but if you take the default path it doesn't let you know you could've chosen to be peaceful the whole time. It involves an active decision rather than a secret thing you can miss, but it also doesn't frame it as a "MORAL CHOICE TIME GO"
Well, that's just the twist idea, but what's your framework? Are you thinking about first-person shooters (Deus Ex style, for example) or about tactical turn-based RPGs or about 2-D platformers or what?
RPG Maker, by default, makes games that look like SNES-era JRPGs.
That does sound very appealing. I'm not well versed at all in game creation, but I do remember playing with RPG maker a few years ago and it was pretty limiting. Rather than RPG maker, why not make a Skyrim mod? That would be much more fun to play and would have a much larger potential userbase.
JRPGs are what I know best.
(And RPG Maker makes standalone programs; someone who wants to play an RPG Maker game doesn't need to have RPG Maker themselves.)
As an example of a flash game with similar story branches (albeit a pretty different plot), there's endeavor.
Only if I heard particularly good things about it.
Most creative endeavors you could undertake have a very small chance of leading to external reward, even the validation of people reading/watching/playing them - there's simply too much content available these days for people to read yours. So I'd advise against making such a thing, unless you find making it to be rewarding enough in itself.
Would you have given Alicorn the same advice if she asked for it before writing "Luminosity"?
Yes. Do you think I would have been wrong?
/me shrugs
It seems to have found an audience.
Obviously some works will always be popular. That doesn't change the fact that the prior odds for any particular one doing so are very low.
Think that would have been a good move. Advice can be pretty good at presenting the outside view -- isn't as good at presenting the inside view unless the advice-giver really knows the advice-receiver well (ETA: meaning relevant details to receiver's specific case, etc.). Receiver should keep this in mind and update on relevant evidence (especially inside view evidence) that giver likely did not take into account.
What do you hope to achieve? Making money through selling the game? Artistic expression? Pushing memes?
My underlying motivation is to feel better about myself. I feel that my life so far has lacked meaningful achievements. Pushing memes is a side benefit.
I do not expect to make money by selling the game, but if I do manage to make something that turns out to be pretty good, I think it would be a big help in getting a job in the video game industry.
Am I the only one who is bothered that these threads don't start on Monday anymore?
Posting a request from a past open thread again: Does anyone have a table of probabilities for major (negative) life events, like divorse or being in a car accident? I ask this to have a priority list of events to be prepared for, either physically or mentally.
The lifetime risk of developing cancer is 44 % in males and 38 % in females. The lifetime risk of dying from cancer is 23 % in males and 19 % in females. It's worth mentioning that the methods for gathering medical mortality statistics are pretty biased, if not completely bonkers.
Would you be willing to expand on this?
ETA: Apparently a new WHO recommendation for filling death certificates was introduced in 2005-2006 and this caused a significant drop in pneumonia mortality in Finland.
I'm not entirely sure if it works this way in the whole EU, but it probably does. It's more complicated than what I explain below, but it's the big picture that matters.
The most common way to record mortality statistics is that the doctor who was treating the patient fills a death certificate. There are three types of causes of death that can be recorded in a death certificate. There are immediate causes of death and there are underlying causes of death. There are also intermediate causes of death, but nobody really cares about those because recording them is optional. The statistics department in Finland is interested in recording only the underlying causes of death and that's what gets published as mortality statistics. Only one cause of death per patient gets recorded.
If someone with advanced cancer gets pneumonia and dies, a doctor fills the death certificate saying that the underlying cause of death was cancer and the immediate cause of death was pneumonia. Cancer gets recorded as the one and only cause of death by the statistics department. Depending on the patient, possible underlying causes of death could also be alcoholism, coronary heart disease or alzheimer's disease or whatever is accepted by a department that checks these certificates.
The doctor's opinion of whether it was the pneumonia or the chronic disease that killed the patient doesn't really matter. If he also fills the underlying cause of death as pneumonia, he gets a scolding letter and has to fill it again until he gets it right.
What if the patient has several chronic diseases that could have been underlying causes of death? Well, you only get to pick one, and only that one gets recorded as the cause of death. You can list the other diseases too as contributory causes of death, but this doesn't really effect any statistics. I guess it would be less biased to flip a coin or something, but I think most doctors just pick something fitting.
A colleague of mine once tried to record pneumonia as the underlying cause of death, the patient was an alcoholic (not sure how bad it was). He got a letter saying he should fix the certificate and that people in developed countries don't die of pneumonia anymore. Wonder why that is...
Very interesting, thank you.
I have a pet interest -- carefully looking at how standard, universally-accepted, real-life, empirical data is collected and produced and whether it actually represents what everyone blindly assumes it does. In the field of economics, for example, closely examining how, say, the GDP or the inflation numbers are calculated is... illuminating.
Details?
The problem is that the problems aren't summarizeable in a neat half a page list. And it's not like the calculations are wrong, rather they are right under a certain set of assumptions and boundary conditions -- and the issue is that people forget about these assumptions and conditions and just assume they're right unconditionally.
For an introduction take a look at e.g. Shadowstats. I don't necessarily agree with everything there, but it's a useful starting point.
Thanks.
I twitch when changes in GDP are reported to a tenth of a percent-- it seems to me that it couldn't be measured with such precision. Do you think I'm being reasonable?
GDP as reported is the product of a particular well-defined calculation. That product can easily be calculated to whatever precision you feel like.
When you say "it couldn't be measured with such precision", how do you define the Gross Domestic Product that couldn't be measured precisely?
I'm assuming that the GDP is some sort of measure of the health of the economy-- that's why people are concerned with it. The health of the economy seems to me like rather an approximate sort of thing.
My own (uninformed) intuition is that GDP changes would be much more accurate than absolute GDP values, just because systematic errors could largely cancel out.
in USA they can fill in 20 secondary causes on the death certificates and all the anonymized death certificates since 1959 are available online from NCHS in computer-readable form to check/search for conditions. Irregularities usually appear when there is a switch from one ICD-Code to a new one, so in 1969,1979,1999. Other irregularities are often checked, compared with other states,countries,conditions and the reason discovered
It seems I miscommunicated here. What I meant to say that listing these other diseases has no meaningful impact on the mortality statistics, although technically speaking they are causes of death. If the point is to gather accurate statistics, listing them feels like a consolation prize, because statisticians don't seem to be interested in them.
In Finland a direct translation for these would be "contributory causes of death". That's probably the same thing as secondary causes of death. The problem is, it's difficult for someone who makes these into statistics to know how important they were. Almost anything the patient has can be listed as a contributory cause of death.
Even a bigger problem is that listing them is completely optional. If almost nobody fills them in properly (because they usually have better things to do), that is another good reason for a statistician not to use them.
Is filling in the secondary causes mandatory in US? Are there clear restrictions for what can be listed? If not, I'm not sure if they provide all that useful information, statistically speaking. Are they really used in meaningful way in any statistics?
I suppose WHO recommendations for filling these certificates impact the US too.
I like it for purely selfish reasons. I can't easily post between Sunday bedtime and Tuesday evening. If I want to post and the thread starts on Monday, my post will be less visible.
If Yvain is (understandably) too busy to run it this year, I am willing to do it. But I will be making changes if I do it, including reducing the number of free responses and including a basilisk question.
Give me a few days to see if I can throw something together and otherwise I will turn it over to your capable hands (reluctantly; I hate change).
Have you started doing modafinil or something by any chance?
Not particularly important ,but if anyone wants to come out and tell me why they went on a mass-downvoting spree on my comments, please feel free to do so.
Incidentally, I'm making a hash precommitment:
See http://www.reddit.com/r/DarkNetMarkets/comments/1pta82/precommitment/
Looking forward to this one ...
It seems to me that a relevant detail is time frame is ~7 months (as you say elsewhere). Ideally, hashes would be commitments to reveal the plaintext in a specified time. Don't you discuss this somewhere?
Not sure what you mean.
There is a danger with publishing hashes that you might publish opposite ones. Ideally, you should be committing to one answer or the other. High entropy predictions mitigates against this, but the effect is still there. And we can't tell if it's high entropy until it is revealed. Publishing dates mitigates against this.
I thought I recently saw an essay on the uses of hash precommitments, including this kind of problem. If that doesn't ring a bell, I guess it wasn't by you.
Oh. Yeah, I did start a little discussion on an isomorphic trick in Umineko. In this case, the date on which I posted the hash is provided automatically by Reddit/Lesswrong/Twitter/etc and one can also verify I didn't post any other hashes recently to those fora.
The trick also only works for 'small keyspaces', if you will - if for example I was trying to fake a precommitment to a 100-digit number, the trick isn't going to work because it's not feasible to publish precommitments to even a tenth of the potential 100-digit numbers without people noticing and calling foul - 'so, gwern, why are you publishing that many precommitments and when can we expect them all to be revealed...?'
So I get home from a weekend trip and go directly to the HPMOR page. No new chapter yet. But there is a link to what seems to be a rationalist Death Note.
I am going to read it. Here are my next thoughts:
So, it seems like Eliezer succeeded to create a whole new genre of literature: rationalist fiction. Nice job!
Wait, what?! Is "a story where the protagonist behaves rationally" really a new genre of literature? There is something horribly wrong with this world if this is true.
Discussing with my girlfriend about which stories should be x-rationalizated next, she suggests HPMOR. Someone should make a HPMOR fanfic where the protagonist is even more rational than the rational Harry. Would that lead to a spiral of even more and more rational heroes?
What exactly could the MoreRational!Harry do? It would be pretty awesome if he could somehow deduce the existence of magic before he was contacted from Hogwarts. For example, he could start doing some research about his biological parents; after realizing they were killed he could try to find out the villain, and gradually discover the existence of magic.
Only one problem: MoreRational!Voldemort would have killed MoreRational!Harry as a baby. Using a knife.
I think what you are referring to here is "a story where the protagonist describes their actions and motivations using rationality terminology" or maybe "a story where the rational thinking of the protagonist motivates the plot or moves it along". At least some of the genre of detective fiction — early examples being Poe's Auguste Dupin stories — would be along these lines.
Stories where protagonists behave rationally (without using rationality terrminology) wouldn't look like stories about rationality. They look like stories where protagonists do things that make sense.
An idea came to my mind. Would it be possible to make a story in which Harry is less intelligent, in a way that he would score less in an IQ test for example, but at the same time more rational? HJPEV seems to be a highly intelligent prodigy even without the rationality addition. I would like to see how a more normal boy would do.
One could argue that he appears intelligent only because he's spent his life so far learning effectively.
Rationalist!Harry is calibrated to match the knowledge and recall of a 34-year-old autodidact. Even presuming a very friendly environment and that said 34-year-old autodidact's training was not optimal, I just don't think there's enough time.
I can buy a 10 year old reading Ender's Game and The Lord of the Rings and maybe even Lensmen. It's a bit harder to imagine one that would consider wanting to want the math behind proving N=NP, nevermind going further than that.
I believe it's been stated somewhere that EY draws primarily on the skills he had around 18 and intentionally keeps things from beyond that out of Harry's reach. So Harry is more like a brilliant high school student than an adult (and, extra seven years worth of rationalist training aside, the way he approaches problems is a lot like a middle schooler with superpowers: "I can win, you can't, deal with it, 'cause I'm awesome and you know it." Which manages to annoy everyone in-universe and out.). Time isn't really a problem, either, if Harry has nothing else to occupy his time; exercise and social interaction are apparently not his thing, and he wound up out of the public school system after a few years, so he really does have way more time than most kids his age to read all the books. And he has that mysterious dark side and that sleeping disorder, whatever those contribute.
The other strangely adult-like children, however, are not so easily justified. (Draco gets most of those complaints, from what I've read.)
I wanted the maths behind relativity and QM at age 10. And I wasted a lot of time in school.
Yup. At least sort-of. If you haven't read Eliezer's old post Lawrence Watt-Evans's Fiction I recommend it. However, conspicuous failures of rationality in fiction may be mostly an issue with science fiction and fantasy. If you want to keep the characters in your cop story from looking like idiots, you can do research on real police methods, etc. and if you do it right, you have a decent shot at writing a story that real police officers will read without thinking your characters are idiots.
On the other hand, when an author is trying to invent an entire fictional universe, with futuristic technology and/or magic, it can be really hard to figure out what would constitute "smart behavior" in that universe. This may be partly because most authors aren't themselves geniuses, but even more importantly, the fictional universe, if it were real, would have millions of people trying to figure out how to make optimal use of the resources that exist in that universe. It's hard for one person, however smart, to compete with that.
For that matter, it's hard for one author to compete with an army of fans dissecting their work, looking for ways the characters could have been smarter.
Fbzr sna gurbevrf ubyq gung Dhveeryzbeg vf hfvat Ibyqrzbeg nf n chccrg vqragvgl va beqre gb tnva cbjre. Fb Ibyqrzbeg'f erny tbny vfa'g gb xvyy Uneel; vg'f gb unir n qenzngvp fubjqbja gung trgf ybgf bs nggragvba naq fpnerf crbcyr.
Guhf gur snpg gung Ibyqrzbeg qvqa'g xvyy Uneel jvgu n xavsr vf abg orpnhfr ur'f abg engvbany rabhtu, ohg orpnhfr ur unf aba-boivbhf tbnyf.
I think there's a difference between what I've been describing as rationalist!fic (or rationalist!fiction) and fiction in which the -agonists (PCs is the right terminology, I guess) are rational/clever. Rationalist!fic doesn't just feature rationalist characters, they're expressively written to teach the audience about rationality.
Examples:
Given that the spell never failed in the past, I'm not sure that it would have been rational to use a knife.
I get your sentiment, but I don't think this is true. Anyways, wouldn't this just mean that rational minds usually pursue other goals than writing fiction? Not saying that there shouldn't be rationalist fiction, but this doesn't sound like such a bad state of affairs to me.
I haven't read HPMOR. Do I have to know anything about the HP universe to enjoy this thing? Will I learn anything new if I've read the sequences?
I guess you don't need to know anything from the HP canon. It could perhaps be even more interesting that way. I don't think you would learn new information. It might have a better emotional impact, but that is difficult to predict.
I would consider the world better if there were more rational people sharing the same values as me. We could cooperate on mutual goals, and learn from each other.
Problem is, rational people don't just appear randomly in the world. Okay, sometimes they do, but the process if far from optimal. If there is a chance to make rationality spread more reliable, we should try.
But we don't exactly know how. We tried many things, with partial success. For example the school system -- it is great in taking an illiterate peasant population and producing an educated population within a century. But it has some limits: students learn to guess their teachers' passwords, there are not enough sufficiently skilled teachers, the pressure from the outside world can bring religion to schools and prevent teaching evolution, etc. And the system seems difficult to improve from inside (been there, tried that).
Spreading rationality using fiction is another thing worth trying. There is a chance to attract a lot of people, make some of them more rational, and create a lot of utility. Or maybe despite there being dozens of rationalist fiction stories, they would all be read by the same people; unable to attract anyone outside of the chosen set. I don't know.
The point is, if you are rational and you think the world would be better with more rational people... it's one problem you can try to solve. So before Eliezer we had something like the Drake equation: how many people are rational × how many of them think making more people rational is the best action × how many of them think fiction is the best tool for that = almost zero. I am curious about the specific numbers; especially whether one of them is very close to zero, or whether it's merely a few small numbers that give almost zero result when multiplied together.
I'd probably want more people who share my values than more rational people. Rational people who share my values is better. Rational people who don't share my values would be the worst outcome.
I don't think the school system was built by rationalists, so I'm not sure where you were going with that example.
How effective has fiction been in spreading other ideas compared to other methods?
This leads to another comment on rationalist fiction: Most of it seems to be restricted to fan-fiction. The mold appears to be: "Let's take a story in which the characters underutilized their opportunities and bestow them with intelligence, curiosity, common sense, creativity and genre-awareness". The contrast between the fanfic and the canon is a major element of the story, and the canon an existing scaffold which saves the writer from having to create a context.
This isn't a bad thing necessarily, just an observation.
So, the question becomes, how do you recognize "rationalist" stories in non-fan-fic form? Is it simply the presence of show-your-work-smart characters? Is simply behaving rationally sufficient?
Every genre has a theme...romance, adventure, etc.
So where are the stories which are, fundamentally, about stuff like epistemology and moral philosophy?
That's a great question. (And related to how to recognize rational people in real life.)
I'd say that there must be some characters which are obviously smarter than most people around them. Because that's what happens in real life: there is the bell curve, so if all your characters are on a similar level, then the story is (a) not realistic, (b) the characters are selected by some intelligence filter which should be explicitly mentioned, or (c) the characters are all from the middle of the bell curve. Also, in real life the relative power of intelligent people is often reduced by compartmentalization, but this reduction would be much smaller for a rationalist hero.
So I'd say it's behaving rationally while most of other people aren't. The character should somehow reflect on the stupidity of others; whether by frustration from their inability to cooperate, or by enjoyment of how easily they are manipulated.
I'm not sure I like that criteria. By that criteria alone, the original death note anime was rationalist fiction (judging by the first half), as is Artemis Fowl, Ender's Game, and to some extent even Game of Thrones. There are a lot of stories where some characters are much smarter than others and know it, but consuming these works won't teach anyone how to be smarter. (Other than the extent to which reading good fiction in general improves various things)
None of these stories actually teach the reader anything about epistemology. Even the linked Death Note fan fic...it uses rationality-associated words like "utility" and "prior" but if I didn't already know what those words meant I would have just come away confused. (Granted, it's still early in the story - but even so)
Also, it hasn't yet broken the conceit of the story (For example, even a normal person of average intelligence would be surprised and curious about the existence of the supernatural, and would investigate that). I'd say that breaking the story conceit is another feature of rationalist fanfiction stories that has nothing to do with the character's intelligence.
Well, I was disappointed with the Death Note fan fic, because it doesn't seem to have added value beyond the original story. And I agree that exploring the supernatural should be a high priority for a rational person, once the supernatural is experimentally proven. Would it be so difficult to ask Ryuk whether there are additional magical items that could also be abused? I guess Ryuk would use an excuse of having "rules" against that, but at least it's worth trying.
Having a rational superhero is a necessary condition for a rationalist story, not a sufficient condition. Ender's Game could be a rationalist literature if it explained Ender's reasoning better, and if Ender strategically tried to improve his understanding of the world. Okay, another necessary condition is not just that the superhero is super smart, but also that the super smartness is at least partially a result of a good strategy, which is shown to the reader.
I'd say the difference between "rationalists" stories and "non-rationalist" stories lies in the moral of the story, of the lessons the story teaches you.
I don't think it's a genre in the same way romance or adventure are. It's more of a qualifier. You can have rationalist romance novels or rationalist adventure movies.
Although you could argue that it is a genre. While discussions about "genre" are often hard, since people don't tend to agree on what makes something a genre.
But rationalist fiction already has a couple of genre conventions, such as no-one being allowed to hold the idiot ball or teaching the audience new and useful techniques for overcoming challenges.
Erfworld is a piece of rationalist fiction not related to HP:MoR. It was discussed on here a while back. There must be others.
Also, I suggest calling it Rational!Rational!Harry.
In addition to the other's already listed, DataPacRat's Myou've Got To Be Kidding Me follows a perspective of a character thrown into a setting and trying to analyze the basic rules in order to optimize them. There are some interesting concepts, but I don't know that I can recommend it : It has not been updated in over a year, and was part of some big conglomeration of fanfic writers which had some pretty widely varying quality (although thankfully nothing necessary to Myou've plotline).
SPOILERS FOR "FRIENDSHIP IS OPTIMAL"
Why is 'Friendship is optimal' "dark" and "creepy"? I've read many people refer it that way. Only things that are clearly bad are the killings of all the other lifeforms, but otherwise this is scenario is one of the best that humanity could come across. It's not perfect, but it's good enough and much better than the world we have today. I'm not sure if it's realistic to ask for more. Considering how likely it is that humanity will end in some incredibly fucked up way full of suffering, then I would definitely defend this kind of utopia.
(Comment cosmetically edited in response to Kaj_Sotala, and again to replace a chunk of text that fell in a hole somewhere)
OK, I'll have a go (will be incomplete).
People in general will find the Optimalverse unpleasant for a lot of reasons I'll ignore; major changes to status quo, perceived incompatibility with non-reductionist worldviews, believing that a utopia is necessarily unpleasant or Omelas-like (a variant of this fallacy?), and lots of even messier things.
People on LessWrong may be thinking about portions of the Fun Theory Sequence that the Optimalverse conflicts with, and in some cases they may think that these conflicts destroy all of the value of the future, hence horror.
(rot13 some bits that might consitute spoilers)
Humans want things to go well, but they also want things to have been able to go badly, such that they made the difference. Relevant: Living By Your Own Strength, Free to Optimize.
The existence of a superintelligence makes human involvement superfluous, and humans do not want this to happen. Relevant: Amputation of Destiny.
Gur snpg gung gur NV vf pbafgenvarq gb fngvfsl uhzna inyhrf gur cbal jnl zrnaf gung n uhtr nzbhag bs cbffvoyr uhzna rkcrevrapr vf abj vzcbffvoyr gb rire ernyvfr. Eryrinag: Hzz... znlor Value is Fragile? Abg dhvgr. Uryc zr bhg urer, thlf! (nyfb, vafreg lbhe bja cersreerq snaqbz wbxr nobhg cbbe Ylen arire trggvat gb unir unaqf rgp.)
Nf lbh zragvbarq, gur jnl va juvpu gur NV'f cnegvphyne qrsvavgvba bs "uhzna" jnf abg evtug naq pna arire or zbqvsvrq, urapr nyvra ncbpnylcfrf. Eryrinag: The Hidden Complexity of Wishes
Themes that are more explicit after the extra worldbuilding in Caelum est Conterrens:
Gjb cbffvoyr svany nggenpgbef sbe uhzna tebjgu ner cerfragrq (Ybbc naq Enl Vzzbegnyf):
Fbzr urer ner abg fngvfsvrq jvgu rvgure naq ernyyl, ernyyl ubcr gurer vf n guveq jnl sbe uhznaf gb npuvrir haobhaqrq tebjgu gung erznvaf zrnavatshy (ol gurve yvtugf).
Notes:
I'm sympathetic to your position; this is the substance of my comment here that I think I understand what's supposed to horrify me.
That comment of mine is no doubt wrong; there will be things that don't horrify me that I didn't even realise were supposed to.
There are quick and obvious comebacks to nearly all the above points. In a lot of cases, those quick comebacks are dealt with in the linked articles. Read the Fun Theory Sequence; it's my favorite sequence, despite the fact that I disagree with more of it than any of the others.
Now that I've thought about your post I realized that the biggest question in this story is what the phrase "satisfy values" actually means. Currently it's a pretty big hand wave in the story. Especially your first point seems to imply that we understood it a bit differently.
In my understanding, if I value real challenge, the possibility of things going badly, or even some level of pain, then the Optimalverse will somehow maximize those values and at least provide the feeling of real challenge and possibility of things going badly. And I don't know why the Optimalverse couldn't even provide the real thing. The way Light Sparks tries to pass the Intermediate Magic test seems an awfully lot like real challenge. Of course the Optimalverse wouldn't allow you to die because in most cases the dislike of death overrides the longing for real challenge in the value system, but that still leaves a lot of options free. I got the impression that this is how it's actually handled in the story. There's this passage
Your second point is of course a real concern for some people, but personally it doesn't feel very relevant. My actions don't currently feel very important in the big scheme of things and I don't know how a superintelligence would change things all that much. If I'm not personally doing anything important, then it doesn't really matter to me if the important things are done by other humans or by a superintelligence. Anyway, this will always be a problem with AGI and if the AGI is friendly then the benefits outweight the negatives IMO. I think the alternative is worse.
The way I understood it is that the "ponies" in this story are essentially human in a pony disguise with four legs (two of them which can almost work like hands). A paragraph from the story:
A big part of being human is due to our mind and hormones. Walking with two legs or being able to use hands extensively are more trivial points. If the psychology of a person doesn't change in the transition from human to pony, then this eliminates most of the problems in your third point.
I haven't read Caelum Est Conterrens and can't fully comment on those points. But it seems that those are more like technicalities. I don't know if it's actually possible to turn a person into a pony without losing the person in the process. But if you're not changing the brain parameters and the psychology doesn't change in the process like it seems to be in this story then I would be inclined to say it's possible. Clearly it can't be worse for your identity than losing all your limbs or becoming a quadriplegic? Anyway, one of the axioms in this story seems to be that it's possible.
I actually read the Fun theory sequence in its entirety before I read 'Friendship is optimal' and I thought FIO more faithful to the spirit of the sequence than 99% utopian stories out there. This is mostly because Celestia maximizes people's values, not their happiness. This is a very vague concept, and a lot depends on how it's implemented, but if it's implemented the way I picture it, there shouldn't be problems with things mentioned in High Challenge, Complex Novelty, Sensual Experience, Living By Your Own Strength, Free to Optimize, In Praise of Boredom, Interpersonal Entanglement and so on.
Of course, I have problems with applying things I read about to all my experiences, so it could be I misremember some things in the sequence or didn't understand them correctly to begin with.
Well, this is not clear, though it might be true.
I have frequently had the experience of not doing anything with my left leg; losing the ability to ever do anything with my left leg means I'm prevented from ever doing anything with it. This is horrible, of course, but it's the horror of being prevented from doing things I often choose not to do. Losing all my limbs is a more extreme version of the same thing.
Having different limbs might be more identity-distorting, by virtue of providing experiences that are completely unfamiliar.
Then again it might not.
For my own part, I'm not all that attached to preserving my current identity, so I'm not sure the question matters to me. If my choice is between an identity-altering pony body, and an identity-preserving quadriplegic body, I might well choose the former.
Upvoted, but I'd like to request that you'd ROT13 either everything or nothing past a certain point. Being unable to just select all of it to be deciphered, and having to instead pick out a few pieces at a time, was mildly annoying.
Done, thanks for saying. I was trying to avoid thinking about the interaction between rot13 and links (leaving the anchor text un-rot13ed seems like acceptable practice?) but I should just have spent the extra two minutes.
Thanks! Much better now. :-) (As for the links, one can just paint over them as well and think "oh it was just some link" when they show up as garbled in the translation.)
I read Caelum Est Conterrens, now I can better understand why some aspects of the scenario are a bit disconcerting if not horrifying. I find all the options loop immortality, ray immortality and exponential immortality kinda unpleasant, but maybe that is as good as it gets. Still, it feels like many of those things are not exclusive to this scenario, but are part of the world anyway.
Related to this, what did you think about the "normal" ending in the Three worlds collide?
From flaky memory, I think I find the Normal Ending far less acceptable than anything in the Optimalverse - one feels the premature truncation of human nature, rather than the natural exhaustion of it (or the choice to become inexhaustible) - but hey, maybe I'm inconsistent.
Endorsed as a good summary.
Having just finished reading "Friendship is Optimal" literally less than 10 minutes ago, I didn't find it dark or creepy at all. There are certain aspects of it that are suboptimal (being ponies, not wireheading), but other than that, it sounds like a great world.
Can you elaborate? Do you mean that not being able to wirehead is suboptimal?
Yes. I think wireheading is the optimal state (assuming it can make me as happy as possible). I recognize this puts me at odds with an element of the LessWrong consensus.
At least to me, it's increasingly difficult to distinguish between a paradise machine and wireheading, and I dislike wireheading. Each shard of the Equestria Online simulation is built to be as fulfilling (of values through ponies and friendship) as possible, for the individual placed within that shard.
That sounds great! .... what happens when you're wrong?
I mean, look at our everyman character, David. He's set up in a shard of his own, with one hundred and thirty two artificial beings perfectly formatted to fit his every desire and want, and with just enough variation and challenge to keep from being bored. It's not real variation, or real challenge, but he'd not experience that in the real world, either, so it's a moot point. But look at the world he values. His challenges are the stuff of sophmore programming problems. His interpersonal relationships include a score counter for how many orgasms he gives or receives.
Oh, his lover is sentient and real, if that helps, but look at that relationship in specific. Butterscotch is created as just that little bit less intelligent than David is -- whether this is because David enjoys teaching, or because he's wrapped around the idea of women being less powerful than he is, or both, is up to the reader. Sculpted in her memories to exactly fit David's desires, and even a few memories that David has of her she never experiences, so that the real Butterscotch wouldn't have to have experienced unpleasant things that CelestAI used to manipulate David into liking/protecting her.
There are, to a weak approximation, somewhere between five hundred billion and one trillion artificial beings in the simulation, by the time most of humanity uploads. That number will only scale up over time. Let's ignore, for now, the creepiness in creating artificial sentients who value being people that make your life better. We're making artificial optimized for enjoying slaking your desires, which I would be surprised if it happened to also be optimized for what we as society would really like.
Lars is even worse: he is actively made to not not want his life of debauchery -- see the obvious overlap with the guy modifying himself to not get bored with a million years of catgirl sex.
At a deeper level, what if your own values are wrong?
The basic example, brought up in the Rules of The Universe document, is a violent psychopath. Upon being uploaded, CelestAI would quite happily set our psychopath up in a private shard with one hundred and fifty artificial ponies, all of which are perfectly molded to value being shot, stabbed, lit on fire, and violated in a way that is as satisfying as possible to a Dexter villain.
Or I can provide a personal example. I can go both ways, preferring guys, and was an unusually late bloomer. I can look back through time to see an earlier version of myself's values, and remember how they changed. Even in a fairly tolerant society and even with a very collaborative environment, this was not something that came according to my values or without external stimulus. ((There is a political position version of this, but for the sake of brevity I'll just mention that it's possible. More worryingly, I'm not sure there's a way to formalize this concern, as much as it hits me at a gut level. For the most part, value drift is something we don't want.))
Or, for an in-story example :
It's a very good dysutopia -- I'd rather live there than here, and heck it even beats a good majority of conventional fluffy cloud heaven afterlives -- but it's still got a number of really creepy issues..
A little fiction on related topics: "Hell Is Forever" by Alfred Bester-- what if your dearest wish ts to create universes?You're given a pocket universe to live in forever, and that's when you find out that your subconscious keeps leaking into your creations (they're on the object level, not the natural law level), and you don't like your subconscious.
Saturn's Children by Charles Stross. The human race is gone. All that's left is robots, who were built to be imprinted on humans. The vast majority of robots are horrified at the idea of recreating humans.
No, let's not ignore it. Let's confront it, because I want a better explanation. Surely a person who values being a person that makes my life better, AND who is a person such that I will value making their life better, is absolutely the best kind of person for me to create (if I'm in a situation such that it's moral for me to create anyone at all).
I mean, seriously? Why would I want to mix any noise into this process?
Good point. I've not uncompressed the thoughts behind that statement nearly enough.
The artificial sentients value being people that make your life better (through friendship and ponies). Your values don't necessarily change. And artificial sentients, unlike real ones, have no drive toward coherent or healthy spaces of design of minds : they do not need to have boredom, or sympathy, or dislike of pain. If your values are healthily formed, then that's great! If not, not so much. You can be a psychopath, and find yourself surrounded by people where "making their lives better" happens only because you like the action "cause them pain for arbitrary reasons". Or you could be a saint, and find yourself surrounded by people who value being healed, or who need to be protected, and what a coincidence that danger keeps happening. Or you can be a guardian, and enjoy teaching and protecting people, and find yourself creating people that are weak and in need of guidance. There are a lot of things you can value, and that we can make sentient minds value, that will make my skin crawl.
Now, the Optimalverse gets rid of some potential for abuse due to setting rules -- it's post-scarcity on labor, starvation or permanent injury are nonsense, CelestAI really really knows your mind so there's no chance of misguessing your values, so we can rule out a lot of incidental house elf abuse -- but it doesn't require you to be a good person. Nor does it require CelestAI to be. CelestAI cares about satisfying values through friendship and ponies, not about the quality of the values themselves. The machine does not and can not judge.
If it's moral to create a person and if you're a sufficiently moral person, then there's nothing wrong with artificial beings. My criticism isn't that CelestAI made a trillion sentient beings or a trillion trillion sentient beings -- there's nothing meaningfully worrying about that. The creepy factor is that CelestAI made one being, both less intelligence than possible and less intelligent than need be.
That may well be an unexamined reaction or even incorrect response. I like to think I'm open-minded, but I'm willing to recognize that I can overestimate it, and have done so in the past. There are real-world right-now folk who enjoy being (in specific contexts and while in control) hurt or being hurt and comforted, which I can accept. Maybe I'm being parochial when I judge David for wanting a woman he can always teach, or Lars for his sex groupies; that's not a mind space I empathize with terribly well, and a good deal of my revulsion comes from real-world constraints that wouldn't apply here. There's a reason that we're using the word creepy, rather than wrong. But it does make my skin crawl.
A way to fall asleep and/or gain gut intuition for "exponentially slow": count in binary, in your head, at a regular beat. YMMV.
In honor of NaNoWriMo, I offer up this discussion topic for fans of HPMOR and rationalist fiction in general:
How many ways can we find that stock superpowers (magical abilities, sci-fi tech, whatever), if used intelligently, completely break a fictional setting? I'm particularly interested in subtly game-breaking abilities.
The game-breaking consequences of mind control, time travel, and the power to steal other powers are all particularly obvious, but I'm interested in things like e.g. Eliezer pointing out that he had to seriously nerf the Unbreakable Vow in HPMOR to keep the entire story from being about that.
I seem to be able to do this with almost any power to various degrees. Including ones I actually have, and ones that are common among humans. Any specifics you had in mind?
Really, ANY ability will reroll some chaotic stuff and be a valuable asset simply because it's rare. Even a deliberating curse, if rare and interesting enough, can do things like be useful for research or provide unique perspectives to be studied. So really, the only limit to where a power stops being useful is where it's only useful to someone else controlling you.
Hence, why anything properly rationalist that's not going to be largely about breaking the setting must do something like give MANY the ability so the low hanging fruit is already gone, or make it inherently mysterious and unreplicable, or have some deliberate intelligence preventing it from getting well known, or something like that.
We may be operating on different definitions of what it means to "break" a setting. For example, how useful is flight by itself, really?
Many abilities seem potentially very useful if other people don't know you have them, but become much less useful once you get found out:
Thats because those are among the worst possible ways to use those abilities.
The energy blasts as usually depicted break conservation of energy; with a bit of physics trickery you can get time travel out of that. Even if not, they make you an extremely portable and efficient energy source, perfect for a spaceship where mass is critical and a human needs to come along anyway but it doesn't matter in particular who since it's for PR reasons.
Mind reading is a means of communication that does not require cooperation or any abilities in the target, and cant be lied through. Communication with locked-in patients, interrogation, extraction of testimonials from animals. And if you an find a way to yourself precommit, you also have fully reliable precommitment checking for everyone, lie detection for political promises, and the ultimate forensics tool.
If you combine the strengths of 2 kinds of system, you get something greater than the sum of it's parts. So it is with human senses and digital sensors. The key here is bandwidth, and analysis. Sure, you can get all the same data onto a computer, but it won't do much good there. Someone with true super-senses as flexible and integrated as their normal biological ones, after a few years for the brain to adapt if they were not available from birth, would most likely be able to see patterns at a glance that'd take large teams days or months to discover in some database format. The exact applications of this depend on what sense we're talking about.
Common factors: Focus on things done with to cooperation of large numbers of other people, finding an economic niche, fundamental physics exploits, and/or using large and expensive equipment. If not that, look for a niche within the military as a specialized technician, most likely not in the field and if in the field then in some large vehicle with a crew of many. Almost NEVER is an efficient use of a power found in brawling or acting alone like all your descriptions where.
Wait, explain that? What is "a bit of physics trickery" here?
I know in HPMOR, Harry points out that violations of conservation of mass in magic imply FTL signaling, and I know from relativity FTL implies time travel, but Harry doesn't even consider running off to get time travel from common spells. Assumed turning the theory into practice would be far from straightforward.
Heck, it's not even obvious to me how you turn FTL travel into time travel in practice, if you don't have control over what frame of reference you're FTL in.
In the approximation of the True Laws Of Physics which is in use today --- The Relativistic Standard Model --- FTL (and subsidiarily time travel) is nonsense. Like, it's gibberish. It is a description of a situation which not only does not happen, but which is a mathematical falsehood. It is impossible. It is like violation of conservation of energy, or violation of entropic developments.
The maths plainly states that assumptions like that leads to a contradiction, and we do in fact know that the Standard Model is complete and consistent (i.e. cannot encode formulas as objects).
Any hypothetical scenarios involving time travel will invariably be contrite, non-causal, and require classical mechanics. It is fiction, make of it what you will.
Oooh, that's even better: from a contradiction, anything can be proved. Thus, if we can break conservation of energy we gain literal omnipotence.
To make a time machine out of an FTL drive, simply travel somewhere at near lightspeed and FTL back. Now you're where you started, before you left.
Bit over a decade ago, when I was still a naive wild-eyed idealist blissfully unaware of anything realistic about people at all, I set up the foundations for most of my sci fi/fantasy/etc fictions. The piece of supertech I immediately had to nerf was technology based around a form of space compression--somewhere between the MCron Crystal (Marvel) and Dynocaps (Dragonball). And today, I'm still coming up with more reasons to nerf it heavily, to the extent that the civilization based around this technology must certainly have had a shady counsil of vagueness who developed a limited-scope AI to control production and block any of the scarier uses. This after I tied it to FTL capabilities (I'm trying very hard to prevent this from turning into timetravel, not that I'm too afraid to abuse that option for a crisis crossover or something).
It took me much longer to realize how much nerfing of powers unrelated to the above tech I needed to do, partially because I was trying to avoid breaking the laws of physics too horribly (this universe runs on a sort of mangling of GR and M-Theory that assumes fundamental forces each get something resembling a dimension, and that attempts to break the light-speed barrier have lots of wide-reaching consequences).
A few things I've had to seriously remodel (mostly for the sake of in-universe physics not breaking, though I'm sure there are obvious uses/flaws here I've missed):
In all of this, I expect I'm missing something world-breaking that half of LW would notice within five minutes of having the powers explained (or maybe just introduced--do you need to know that someone generates his fireballs by burning in a weird chemical unwanted bodyhair (and subdermal fat when that runs out) to think of issues that arise from someone with the power to toss fireballs around?).
New research suggests that the amount of variance in DNA among individual cells in a person may be much higher than is normally believed. See here.
Edit: see the paper for more precise statements.
I've already seen work to the effect that somatic cells often have ~10x the point mutations per human generation as the germline, which is protected by a small number of divisions per generation and low levels of metabolism and transcription. It was in mitochondrial rather than nuclear DNA, but the idea is similar.
Have Eliezer's views (or anyone else's who was involved) on the Anthropic Trilemma changed since that discussion in 2009?
There's no brief answer. I've been slowly gravitating towards, but am not yet convinced, by the suspicion that making a computer out of twice as much material causes there to be twice as much person inside. Reason: No exact point where splitting a flat computer in half becomes a separate causal process, similarity to behavior of Born probabilities. But that's not an update to the anthropic trilemma per se.
Hmm, conditional on that being the case, do you also believe that the closer to physics the mind is the more person it is in it? Example: action potentials encoded in the position of rods in a babbage engine vs. spread over fragmented ram used by a functional programing language using lazy evaluation in the cloud.
Good question. Damned if I know.
Can someone explain nanotech enthusiasm to me? Like, I get that nanotech is one of the sci-fi technologies that's actually physics-compliant, and furthermore it should be possible because biology.
But I get the impression that among transhumanist types slightly older than me, there's a widespread expectation that it will lead to absolutely magical things on the scale of decades, and I don't get where that comes from, even after picking up Engines of Creation.
I'm thinking of, e.g. Eliezer talking about how he wanted to design nanotechnology before he got into AI, or how he casually mentions nanotechnology as being one of the big ways a super-intelligent AI could take over the world. I always feel totally mystified when I come across something like that, like it's a major gulf between me and slightly older nerds.
Trying for minimal technicalities: There are at least 3 different technologies with not much surface-level usage similarities referred to as "nanotech".
Assemblers: basically 3d printers, but way more flexible and able to make things like food, robots, or more assemblers.
Materials: diamondoids, buckytubes, circuitry. We already have some of these really, it's just that we'd get more kinds of them, and they'd be really cheap to make with a nanotech assembler. Stronger, faster, more powerful versions of what modern tech already can do.
Nanobots, particularly medical: Basically can do all the things living cells can do, but better, and also being able to do most of the things machines can do, and commandable in exact detail. There are also a number of different ways they'd grant immortality enough that they are almost sure to do so even if most of them end up not working out.
Now you can ask questions about each one of these in order, with more specifics.
The question is about all these technologies - though it's about 2 mainly insofar as 2 is an extension of 1.
So the question is why expect any of these technologies to mature on a timescale of decades?
(Or, assuming FOOM, why assume they'd be relatively low-hanging fruit for a FOOMing AI, such that "trick humans into building me nano assemblers" is a prime strategy for a boxed AI to escape?)
As I said, 2 is already here, and it's becoming more here gradually.
For 3, we have a proof of concept to rip of: biological cells. Those also happens to have a specialized assembler in them already; the ribosome. And we can print instructions for it already. There's only 1 problem left and that's the protein folding problem. The protein folding problem is somewhat rapidly made progress on software wise, and even if that were to fail it won't be all that long before we ca simply brute force it with computing power. Now, the other kinds of nanobots are less clear.
The assembler (1) is trickier; however, Drexler already sorta made a blueprint for one I think, and 3 will help a great deal with it as well.
For the fooming, it's the 3 one, and ways to use it. As I said we already have the hardware, and things like the protein folding problem is exactly what an AI would be great at. Once it's solved that it has full control over biology and can essentially make The Thing and/or a literal mind control virus, and take over that way.
Okay, so one sub-piece of puzzlement I have is why talk of protein folding as a problem that is either solved or unsolved - as if we (or more frighteningly, an AI) could suddenly go from barely being able to do it to 100% capable.
I was also under the impression that protein folding was mathematically horrible in a way that makes it unlikely to be brute forced any time soon, though I just now realized that I may have been thinking of the general problem of predicting chemistry from physics, maybe protein folding is much easier.
Predicting chemistry from physics should be easy with a quantum computer, but appears hard with a classical computer. Often people say that even once you make a classical approximation, ie, assume that the dynamics are easy on a classical computer, that the problem of finding the minimum energy state of a protein is NP-hard. That's true, but a red herring, since the protein isn't magically going to know the minimum energy state. Though it's still possible that there's some catalyst to push it into the right state, so simulating the dynamics in a vacuum won't get you the right answer (cf prions). Anyhow, there's some hope that evolution has found a good toolbox for designing proteins and that if can figure out the abstractions that evolution is using, it will all become easy. In particular, there are building blocks like the alpha helix. Certainly an engineer, whether evolution or us, doesn't need to understand every protein, just know how to make enough.
I think the possibility that a sufficiently smart AI would quickly find an adequate toolbox for designing proteins is quite plausible. I don't know what Eliezer means, but the possibility seems to me adequate for his arguments.
Ah, that's helpful.
I'm not sure protein folding can be brute forced without quantum computers. There's too many ways for it to fold. In real life, I'm pretty sure quantum tunneling gets involved. Simulations have worked, but I there's a limit to that.
Probably comes from Neal Stephenson's The Diamond Age: Or, A Young Lady's Illustrated Primer :-)
I definitely have found that this forum is NOT immune to fictional evidence.
I'm pretty sure that the people Chris is talking about are Stephenson's source, not vice versa.
Eliezer definitely definitely seems to have caught the nano-enthusiasm bug pre-Diamond Age, but maybe the book had a big impact on other people?
Who, specifically are you talking about?
I'm thinking of the extropians, who coined "transhumanism." I'm not sure of the timeline; the original group was definitely into MNT before Stephenson, but maybe they expanded a lot after him, and maybe that was because of him.
Try Nanosystems perhaps.
Guy decides to do his PhD thesis on Dungeons & Dragons, acquires funding via Kickstarter.
I wonder if there's research that rationalists should do that could be funded this way. I'd pay for high quality novel review articles about topics relevant to lw.
Brian Leiter shared an amusing quip from Alex Rosenberg:
Ugh. The prize was first and foremost in recognition of Fama, Shiller, and Hansen's empiricism in finance. In the sixties, Fama proposed a model of efficient markets, and it held up to testing. Later, both Fama, Shiller, and Hansen showed further tests didn't hold up. Their mutual conclusion: the efficient market hypothesis is mostly right, and while there is no short-term predictability based on publicly available information, there is some long-term predictability. Since the result is fairly messy, Fama and Shiller have differences about what they emphasize (and are both over-rhetorical in their emphasis). Does "mostly right" mean false or basically true?
What's causing the remaining lack of agreement, especially over bubbles? Lack of data. Shiller thinks bubbles exist, but are rare enough he can't solidly establish them, while Fama is unconvinced. Fama and Shiller have done path-breaking scientific work, even if the story about asset price fluctuation isn't 100% settled.
New ligament discovered in the human knee as a result of surgeons trying to figure out why some people didn't recover fully after knee injuries.
I'm tempted to deduce "Keep paying attention, you never know what might have been missed"-- I really would have expected that all the ligaments had been discovered a long time ago.
Another conclusion might be "Try to solve real problems, you're more likely to find out something new that way than by just poking around."
Does someone have the medical knowledge to explain how this is possible? My layperson guess is that once cut up a knee, you can more or less see all the macroscopic structures. Did they just think it was unimportant?
My layperson guess is that once you're told what to expect to see, you stop looking.
This makes Eliezer's weirdtopia idea of science being kept secret so as not to spoil people's fun of discovery more interesting-- it's not just that people would independently discover the same things (and I wonder what the protocol for sharing information would be), given enough time and intelligence, much more might get discovered.
Someone who seemed a bit better informed
And that comment is answered by:
Which is interesting-- sometimes studying things in extreme detail "just because" (probably because the object of study has high status-- consider early observations of the planets) can pay off big.
The "new ligament discovered" angle gets less impressive (to me, at least) when I read this part:
I'm more impressed, actually, in terms of the unevenness of progress - it took ~134 years to confirm his postulate? It's not like corpses were unavailable for dissection in 1879.
Does anyone here have any serious information regarding Tulpas? When I first heard of them they immediately seemed to be the kind of thing that is obviously and clearly a very bad idea, and may not even exist in the sense that people describe them. A very obvious sign of a persons who is legitimately crazy, even.
Naturally, my first re-reaction is the desire to create one myself (One might say I'm a bit contrarian by nature). I don't know any obvious reason not to (ignoring social stigma and time consuming initial investment), And there may be some advantage to having one, such as parallel focus, more "outside" self analysis, etc. I don't really know much of anything right now, which is why I'm asking if there's been any decent research done already.
Have you read the earlier discussions on this topic?
I had not, actually. The link you've given just links me to Google's homepage, but I did just search LW for "Tulpa" and found it fine, so thanks regardless.
edit: The link's original purpose now works for me. I'm not sure what the problem was before, but it's gone now.
I've been doing some research (mainly hanging on their subreddit) and I think I have a fairly good idea of how tulpas work and the answers to your questions.
There are a myriad very different things tulpas are described as and thus "tulpas exist in the way people describe them" is not well defined.
There undisputably exist SOME specific interesting phenomena that's the referent of the word Tulpa.
I estimates a well developed tulpas moral status to be similar to that of a newborn infant, late-stage alzheimer's victim, dolphin, or beloved family pet dog.
I estimates it's ontological status to be similar to a video game NPC, recurring dream character, or schizophrenic hallucination.
I estimate it's power over reality to be similar to a human (with lower intelligence than their host) locked in a box and only able to communicate with one specific other human.
It does not seem deciding to make a tulpa is a sign of being crazy. Tulpas themselves seem to not be automatically unhealthy and can often help their host overcome depression or anxiety. However, there are many signs that the act of making a tulpa is dangerous and can trigger latent tendencies or be easily done in a catastrophically wrong way. I estimate the risk is similar to doing extensive meditation or taking a single largeish dose of LSD. For this reason I have not and will not attempt making one.
I am to lazy to find citations or examples right now but I probably could. I've tried to be a good rationalist and am fairly certain of most of these claims.
Would you classify a novel in the same "moral-status" tier as these four examples?
No, thats much much lower. As in torture a novel for decades in order to give a tulpa a quick amusement being a moral thing to do lower.
Assuming you mean either a physical book, or the simulation of the average minor character in the author's mind, here. Main characters or RPing PCs can vary in complexity of simulation from author to author a lot and it's a theory that some become effectively tulpas.
Your answer clarifies what I was trying to get at with my question but wasn't quite sure how to ask, thanks; my question was deeply muddled.
For my own part, treating a tulpa as having the moral status of an independent individual distinct from its creator seems unjustified. I would be reluctant to destroy one because it is the unique and likely-unreconstructable creative output of a human being, much like I would be reluctant to destroy a novel someone had written (as in, erase all copies of such that the novel itself no longer exists), but that's about as far as I go.
I didn't mean a physical copy of a novel, sorry that wasn't clear.
Yes, destroying all memory of a character someone played in an RPG and valued remembering I would class similarly.
But all of these are essentially property crimes, whose victim is the creator of the artwork (or more properly speaking the owner, though in most cases I can think of the roles are not really separable), not the work of art itself.
I have no idea what "torture a novel" even means, it strikes me as a category error on a par with "paint German blue" or "burn last Tuesday".
Ah. No, I think you'd change your mind if you spent a few hours talking to accounts that claim to be tulpas.
A newborn infant or alzheimer's patient is not an independent individual distinct from it's caretaker either. Do you count their destruction as property crime as well? "Person"-ness is not binary, it's not even a continuum. It's a cluster of properties that usually correlate but in the case of tulpas does not. I recommend re-reading Diseased Thinking.
As for your category error: /me argues for how german is a depressing language and spends all that was gained in that day on something that will not last. Then a pale-green tulpa snores in an angry manner.
I agree that person-ness is cluster of properties and not a binary.
I don't believe that tulpas possess a significant subset those properties independent of the person whose tulpa they are.
I don't think I'm failing to understand any of what's discussed in Diseased Thinking. If there's something in particular you think I'm failing to understand, I'd appreciate you pointing it out.
It's possible that talking to accounts that claim to be tulpas would change my mind, as you suggest. It's also possible that talking to bodies that claim to channel spirit-beings or past lives would change my mind about the existence of spirit-beings or reincarnation. Many other people have been convinced by such experiences, and I have no especially justified reason to believe that I'm relevantly different from them.
Of course, that doesn't mean that reincarnation happens, nor that spirit-beings exist who can be channeled, or that tulpas possess a significant subset of the properties which constitute person-ness independent of the person whose tulpa they are.
Eh?
I can take a newborn infant away from its caretaker and hand it to a different caretaker... or to no caretaker at all... or to several caretakers. I would say it remains the same newborn infant. The caretaker can die, and the newborn infant continues to live; and vice-versa.
That seems to me sufficient justification (not necessary, but sufficient) to call it an independent individual.
Why do you say it isn't?
I count it as less like a property crime than destroying a tulpa, a novel, or an RPG character. There are things I count it as more like a property crime than.
Seems I were wrong about you not understanding the word thing. Apologies.
You keep saying that word "independent". I'm starting to think we might not disagree about any objective properties of tulpas, just things need to be "independent" or only the most important count towards your utility, but I just add up the identifiable patterns not caring about if they overlap. Metaphor: tulpas are "10101101", you're saying "101" occurs 2 times, I'm saying "101" occurs 3 times.
I'm fairly certain talking to bodies that claim those things would not change my probability estimates on those claims unless powerful brainwashing techniques were used, and I certainly hope the same is the case for you. If I believed that doing that would predictably shift my beliefs I'd already have those beliefs. Conservation of Expected Evidence.
((You can move a tulpa between minds to, probably, it just requires a lot of high tech, unethical surgery, and work. And probably gives the old host permanent severe brain damage. Same as with any other kind of incommunicable memory.))
(shrug) Well, I certainly agree that when I interact with a tulpa, I am interacting with a person... specifically, I'm interacting with the person whose tulpa it is, just as I am when I interact with a PC in an RPG.
What I disagree with is the claim that the tulpa has the moral status of a person (even a newborn person) independent of the moral status of the person whose tulpa it is.
On what grounds do you believe that? As I say, I observe that such experiences frequently convince other people; without some grounds for believing that I'm relevantly different from other people, my prior (your hopes notwithstanding) is that they stand a good chance of convincing me too. Ditto for talking to a tulpa.
(shrug) I don't deny this (though I'm not convinced of it either) but I don't see the relevance of it.
Yea this seems to definitely be just a fundamental values conflict. Let's just end the conversation here.
Has anyone worked on making a tulpa which is smarter than they are? This seems at least possible if you assume that many people don't let themselves make full use of their intelligence and/or judgement.
Unless everything I think I understand about tulpas is wrong, this is at the very least significantly harder than just thinking yourself smarter without one. All the idea generating is done before credit is assigned to either the "self" or the "tulpa".
What there ARE several examples of however are tulpas that are more emotionally mature, better at luminosity, and don't share all their hosts preconceptions. This is not exactly smarts though, or even general purpose formal rationality.
One CAN imagine scenarios where you end up with a tulpa smarter than the host. For example the host might have learned helplessness, or the tulpa being imagined as "smarter than me" and thus all the brains good ideas get credited to it.
Disclaimer: this is based of only lots of anecdotes I've read, gut feeling, and basic stuff that should be common knowledge to any LWer.
I'm reminded of many years ago, a coworker coming into my office and asking me a question about the design of a feature that interacts with our tax calculation.
So she and I created this whole whiteboard flowchart working out the design, at the end of which I said "Hrm. So, at a high level, this seems OK. That said, you should definitely talk to Mark about this, because Mark knows a lot more about the tax code than I do, and he might see problems I missed. For example, Mark will probably notice that this bit here will fail when $condition applies, which I... um... completely failed to notice?"
I could certainly describe that as having a "Mark" in my head who is smarter about tax-code-related designs than I am, and there's nothing intrinsically wrong with describing it that way if that makes me more comfortable or provides some other benefit.
But "Mark" in this case would just be pointing to a subset of "Dave", just as "Dave's fantasies about aliens" does.
Yea in that case presumably the tulpa would help - but not necessarily significantly more than such a non-tulpa model that requires considerably less work and risk.
Basically, a tulpa can technically do almost anything you can... but the absence of a tulpa can do them to, and for almost all of them there's some much easier and at least as effective way to do the same thing.
Yes, I would expect this.
Indeed, I'm surprised by the "almost" -- what are the exceptions?
See also 'rubberducking' and previous discussions of this on LW. My basic theory is that reasoning was developed for adversarial purposes, and by rubberducking you are essentially roleplaying as an 'adversary' which triggers deeper processing (if we ever get brain imaging of system I vs system II thinking, I'd expect that adversarial thinking triggers system II more compared to 'normal' self-centered thinking).
Yes. Indeed, I suspect I've told this story before on LW in just such a discussion.
I don't necessarily buy your account -- it might just be that our brains are simply not well-integrated systems, and enabling different channels whereby parts of our brains can be activated and/or interact with one another (e.g., talking to myself, singing, roleplaying different characters, getting up and walking around, drawing, etc.) gets different (and sometimes better) results.
This is also related to the circumlocution strategy for dealing with aphasia.
What is stopping is me is the possibility that I will be potentially permanently relinquishing cognitive resources for the sake of the Tulpa.
A fairly obvious reason is that to generate a tulpa you need to screw up you mind in a sufficiently radical fashion. And once you do that, you may not be able to unfuck it back to normal.
I vaguely recall (sorry, no link) reading a post by a psychiatrist who said that creating tulpas is basically self-induced schizophrenia. I don't think schizophrenia is fun.
This is a concern I share. However...
This is the worst argument in the world.
I don't think so, it can be rephrased tabooing emotional words. I am not trying to attach some stigma of mental illness, I'm pointing out that tulpas are basically a self-inflicted case of what the medical profession calls dissociative identity disorder and that it has significant mental costs.
Was some change made in the lw code in the past couple of weeks or so? I can't browse this site with my android smartphone anymore, have tried several browsers. The site either frequently freezes the browser or shows a blank page after the page has finished loading. This happens more with bigger threads.
Anyone else having this problem?
I have the same problem for pages like recent posts, which look OK at first, but then become blank. Article pages are more likely to load correctly. Solution: turn off javascript. (Android 2.2)
Thanks. This obviously disables a lot of funtionality. Another fix I found for the blank page problem is simply interrupt the loading of the page once you start seeing stuff.
Has anyone else had this happen to them?
Any advice? Am I the only one that's experienced this?
I've experienced variation on the theme.
My usual approach is to decide whether I value treating them as an enemy for some reason. If I do, then I continue to do so (which can include pretending to treat them like a friend, depending on the situation). If I don't, then I move on. Whether they've actually moved on or not is their problem.
"It takes two to tangle" and such. Is the reason for the falling out still there? Or is the residual hate just one of those lost purposes?
Yes, I'm still angry with him. He did something cruel to someone weak, and he got angry with me for saying that was wrong. I wish I could delete him from my life but he works near me.
"It takes two to tango" (not tangle)
Bonus related dinosaur comic
It looks like you have an unspoken treaty of non-hostility. People don't just forget those kind of things; you didn't. My advice is to make good with the person and acknowledge your prior differences, it will be less awkward going forward and you would gain his/her respect. And who knows, they might even gain your respect. Friends for the most part are always better than enemies.
http://www.refsmmat.com/statistics/
Why does this forum spend so much time and effort discussing untestables and unprovables? I'ts disappointing.
In the interests of shaping behavior by praising approximations of the desired behavior, can you identify three threads that are most like what you'd like to see more of?
because it's based on the works of someone who wrote volumes about untestables and unprovables?
Beeminder users, did you pledge? Do you find that it works better if you do?