All of emmab's Comments + Replies

What does "unbounded" mean?

Without bound, as in, without there existing some specific bound you will never surpass.

What does it mean to "retrocausally compress 'self'"

Make yourself easier for your past self to index on. e.g. for an evil version, if Horde Prime wants Horde clones to work to benefit Horde Prime, Horde Prime can work to place himself in the center of the universe, which he previously programmed other Hordes to care about.

 Are you postulating that: [...]

I'm saying something closest to #3. In order to specify an individual, you have to be ... (read more)

So what does "is magic real?", as a child might ask the question, correspond to?

3kithpendragon
Depends on the context. If the child is asking... * ... about mythological stories being true, the answer is that they are not factual, but rather correspond to important aspects of the culture. * ... about the ability to affect things without apparently interacting with them, the answer is to point at the TV remote. [My kid still doesn't believe me that talking and writing are psychic powers, but that's the answer I'm giving on that front.] * ... if things happen in the world that are not trivial to understand or control, the answer is simply "Yes". [If my kid asked "Is lightning magic," I would have to answer "Yes" because I cannot make lightning do what I want: as far as I'm concerned it "just happens" when conditions are exactly right. As far as I've been able to find out, nobody fully understands it.] * ... if actions can have strongly unexpected results, of course they can! * ... if waving my hand was the proximate cause for X happening, I'd have to admit most of the time that it was just good timing on my part. But I can wave my hand at my cell phone to have it snap a selfie, and if I have an NFC chip I can sometimes wave my hand to tell a computer what to do. In general, I think "magic" is best used to point at a state of non-understanding. I don't think the "is it real" query is really thinking about it in the right category. It's a lot like asking, "Is a binary search tree real?" or "Is the color red real?" All three are data processing questions of one sort or another; but the color red, binary search trees, and magic all don't directly talk about anything in the territory. Rather: "red" is a perception, "binary search tree" is a data structure, and "magic" indicates non-understanding. In terms of decision theory, either the "magic" process gives known or unknown outputs for specific inputs. If the outputs are known, as in "When I twist the fluff in this particular way I get thread", the gears might not matter for your purposes. If unknown, as in
-1Teerth Aloke
Define evil. 
Do these things feel logically impossible per se?

Yes, same qualia as looking at an Escher staircase IRL but feels more fundamental.

Do they feel impossible because they contradict other things that you believe are true?

No. I can't break down why they feel impossible.

Do you draw the conclusion that the impossible-seeming things genuinely cannot exist or (in the case of self-perception?) genuinely do not exist, despite appearances?

Kinda but I can't maintain that because milliseconds later I perceive the "impossible" qualia again.

Free will ... (read more)

standard model of PA = schelling model of PA?

5. A second machine, designed solely to neutralize an evil super-intelligent machine will win every time, if given similar amounts of computing resources (because specialized machines always beat general ones).

This implies you have some resource you didn't fully imbue to the first AI, that you still have available to imbue to the second. What is that resource?

The claim that specialized machines always beat general ones seems questionable in the context of an AGI. Actually, I'm not sure I understand the claim in the first place. Maybe he means by analogy to a supervised learning system--if you take a network trained to recognize cat pictures, and also train it to recognize dog pictures, then given a fixed number of parameters you can expect it will get less good at recognizing cat pictures.

I feel a sense of impossibility that "anything could exist at all".

I feel a sense of impossibility when I contemplate the recursive nature of perceiving myself perceiving thoughts.

I feel a sense of impossibility about something unspeakable that comes before and is outside anything else.

How are we sure we mean the same thing by the word consciousness though? All I can tell for sure is that ppl think consciousness is "impossible" (cus they try to invent quantum phlogiston to explain it), and something about consciousness engendering moral... (read more)

4Mitchell_Porter
This is an intriguing comment, but it might take time and care to determine what it is that you are talking about. For example, the "sense of impossibility" that you "get... about lots of things": what kind of sense of impossibility is it? Do these things feel logically impossible per se? Do they feel impossible because they contradict other things that you believe are true? Do you draw the conclusion that the impossible-seeming things genuinely cannot exist or (in the case of self-perception?) genuinely do not exist, despite appearances?

spectrum of qualia of rapid muscle movement:

1. wiggle back and forth with no delay

2. effort required to make individual movements one in a row

some actions are only sometimes available to me in form #2 e.g. glossolalia, patterns of rapid eye movement

sometimes it seems like a matter of training e.g. learning to wiggle my ears

I've been experimenting a bit with using vimwiki: https://github.com/vimwiki/vimwiki

This topic (affordance/encoding) is one of the universal entry points to systemization of fully general agency.

Submission: low bandwidth oracle, ask:

IFF I'm going to die with P>80% in the next 10 years while >80% (modulo natural death rate) of the rest of humanity survives for at least 5 more years then, was what killed me in the reference class:

  • disease
  • mechanical/gross-physical accident
  • murdered
  • other

Repeat to drill down and know the most important hedges for personal survival.

The "rest of humanity survives" condition reduces the chance the question becomes entangled with the eschaton.

i.e. I'm pointing out that selfish utility functions are ... (read more)

2Stuart_Armstrong
See the edit, and make sure you "decide on the length of each episode, and how the outcome is calculated. The Oracle is run once an episode only (and other Oracles can't generally be used on the same problem; if you want to run multiple Oracles, you have to justify why this would work), and has to get objective/loss/reward by the end of that episode, which therefore has to be estimated in some way at that point."

Decided to upload source to github now that I know arbital's license: https://github.com/emma-borhanian/arbital-scrape

Licensed under MIT and Unlicense. Updated the drive/mega links.

Thanks for hosting, added link to post.

Please do not re-download the pages from arbital.com without good reason. I've added a single line of code to disable this. This is why I'm not uploading the source code to github, but did include it in the zip file you can download.

Running the code as-is will simply regenerate the HTML using the already-downloaded raw json.

Edit: This is being downvoted. I'm happy to reevaluate this and upload to github instead of merely including the source in the zip file. Please comment if this is what you wish.

3emmab
Decided to upload source to github now that I know arbital's license: https://github.com/emma-borhanian/arbital-scrape
4Said Achmiz
Regardless of whether you upload to GitHub, would you mind adding a license for the source code?

I'll wait to open source mine to see if yours is better then :)

3emmab
Nvm. Arbital Scrape V2

Simulacra as free-floating schelling points could actually be good if they represent mathematical truth about coordination between agents within a reference class, intended to create better outcomes in the world?

But if a simulacrum corresponds to truth because people conform their future behavior to its meaning in the spirit of cooperation does it still count as a simulacrum?

It feels like you're trying to implicitly import all of good intent, in its full potential, stuff it into the word "truth", and claim it's incompatible with the use of schelling points

... (read more)
1Kenny
I don't think [2] is accurate. Certainly some people are using simulacrums "cooperatively" but only as a larger defection – that's the whole point: to receive benefits that are unearned per object-level reality. I agree that all of this behavior isn't a good (central) example of 'immoral behavior', but it's certainly not good. It might be to some degree inevitable, but so are lots of bad things.
Answer by emmab20

Do you ever get the feeling that you're unsure what was true until the moment you said it? Like on the inside you're this highly contextual malleable thing but when you act it resolves and then you become consistent with something for a time?

Do you ever feel like you're writing checks you can't quite cash, running ahead, saying as true what you plan to *make* true, what becomes true in the saying it. Do you ever experience imposter syndrome?

Do you ever feel like we're all playing a game of pretend and nobody can quite step out of character?

> From the inside, this is an experience that in-the-moment is enjoyable/satisfying/juicy/fun/rewarding/attractive to you/thrilling/etc etc.

people’s preferences change in different contexts since they are implicitly always trying to comply with what they think is permissible/safe before trying to get it, up to some level of stake outweighing this, along many different axes of things one can have a stake in

to see people’s intrinsic preferences we have to consider that people often aren’t getting what they want and are tricked into wanting suboptimal thi... (read more)

1the gears to ascension
I wouldn't say so, no. good point about lots of level 1 things being distorted or obscured by level 3. I think the model needs to be restructured to not have a privileged instrinsicness to level 1, but rather initialize moment to moment preferences with one thing, then update that based on pressures from the other things