All of ckai's Comments + Replies

ckai10

If I understand the sort of thing you're talking about correctly, I like Miles Vorkosigan's solution (from Memory, by Lois McMaster Bujold):

"The one thing you can't trade for your heart's desire is your heart."

ckai30

I was not commenting on goodness or badness, just simulacraness.

Are you saying that assigning titles according to a system that you think is how titles should work makes the title a more accurate truthful signal about actual social reality than assigning titles according to the way titles are generally assigned industry wide?

It seems to me that this is not the case, because assigning titles that are inaccurate (according to industry standards) in either direction causes confusion.  In the case of title deflation, the excerpt above suggests that it mak... (read more)

ckai30

It seems to me that they're both doing exactly the same thing:  assigning titles according to a method that's not industry standard for reasons of their own.  Maybe industry standard is more meaningfully simulacra level 3 (in the social truth sense) than either of them?

5Ben Pace
One of them is trying to build an internally consistent social reality, and one of them is trying to take advantage of other people making an internally consistent social reality (by making false claims within the others' social reality).
ckai60

The efficiency of a process is the rate with which it turns what you have into what you want.

This definition seems to imply that what you have is not what you want (fair enough, I guess), and that the process is a means to get what you want, and the process itself is also not a part of what you want.

Since life is a process (I do not believe that the one who dies with the most toys wins), efficiency under this definition is trading away a part of life for some end result that you want.  (And I think that people tend to object to efficiency in human thi... (read more)

ckai10

I'd also say it's kind of sweet that you assume that people who are pursuing the arts find it to be rewarding, or that the camaraderie that keeps these communities knit together is a pleasant experience.

Actually, that was an element that you introduced.  "We don't want people working a job primarily because it's fun and they like their coworkers."  and "the occasional bright spots of camaraderie when they do manage to get some sort of project going"  

But given your description of "a slowly-developing psychological exaggeration of how meaning... (read more)

ckai90

I know a successful author who says he started writing because he noticed authors get more attention at cons.  FWIW.  (I'm pretty sure he gets something out of the work itself, and yet, this is the story he tells.)

Observing this discussion, your point seems to be that there are some people in the world who do things because they are fun rather than because they are worthwhile (IYO), and that the world as a whole would be better if these people were less able to have fun so that they would be more motivated to do worthwhile (IYO) things.  Giv... (read more)

2DirectedEvolution
I'd add one more piece, which came out of my discussion with ryan_b above. In addition to losing positive social contact (camaraderie), we're losing negative social contact (bullying, obnoxious people, etc). Most people think that you need more than one positive interaction to "cancel out" a negative one. So even small reductions in negative social contact might make up for large losses in positive social contact. So we're losing: * Entire projects for which camaraderie is necessary for them to exist (which I posit is a sign that they may be relatively lacking in social value). * Aspects of social experience in projects that do survive, including both in-person bullying and obnoxiousness, along with camaraderie (which I posit isn't too hard to imagine being neutral-to-positive on net). You and others here are arguing that there exist jobs that are of great social value, but that also depend on camaraderie to get started or survive. Examples given here include startups and this author that you speak of. Surely there are others we could give. If we lose even one project of great social value, along with many unnecessary projects fueled by camaraderie alone, that might still be a net loss. To broaden and take this literally, the loss of any X potentially diminishes all X. When an artist pursues their art rather than becoming a shoe salesman, the shoe industry is diminished. I guess, but who cares? On the level of the economy, everything is a tradeoff. RE you're "I also think it's really... sweet" bit, I'd also say it's kind of sweet that you assume that people who are pursuing the arts find it to be rewarding, or that the camaraderie that keeps these communities knit together is a pleasant experience. From what I've encountered, a lot of that "camaraderie" looks like FOMO, jealousy, inferiority complexes, extreme competition for scarce resources, and a sense of identity defined by victory in a zero-sum status competition, and to top it all off, it has to come w
ckai10

You could see it as a portrait of a man who's stuck at simulacra level 3 (in the masking the lack of reality sense, not so much the signaling sense, though you can also watch him deciding which signals to send and which he's just going to ignore even though he knows he should send them).  He not only lacks the epistemic tools to see reality except in glimpses before retreating to his fortress of banality, but is scared of reality and so doesn't want to, despite his near-constant boredom and misery.  And so he dies in confusion, a banal, narcissis... (read more)

ckai30

Yes, I agree with that.  Of course it's meaningful!  It wouldn't be a reflection of reality if it wasn't.  But meaningful isn't the same as complete or undistorted.

For example, I think it's meaningful (maybe not the most insightful thing that could possibly be said, but meaningful) to talk about the original Star Trek in terms of head, heart, and gut as reflected in the characters of Spock, McCoy, and Kirk.  I don't think this covers everything that Star Trek is, or everything that those characters are, or everything that real people ca... (read more)

ckai10

But The Gervais Principle is a model of a tv show, not directly of reality.  I haven't seen the particular show, but most tv shows are not trying to model reality, but reflect it, and distorting it is fair and even expected.  There's an argument to be made that the distortions are what makes it interesting.

Do you see this differently?

2Gordon Seidoh Worley
I think Rao is clearly trying to take a model from the show and present it as saying something meaningful about the world we live in and not just the world of the show.
2Matt Goldenberg
Yeah, I think that Rao is using the Office to illustrate what he sees as a real world pattern.
ckai10

I'm reminded of this:
https://www.antipope.org/charlie/blog-static/2018/01/dude-you-broke-the-future.html

(And no worries.)

ckai30

That's a really different scenario from the historical one.

I'd like to note that I didn't say the governors were stupid (nor do I believe that people in the past were stupid), just that they were likely to be very different in outlook and understanding about the world and likely to act on these ways of thinking (and I also think that the world was also different and some of their understanding may be more accurate for their time).  I was trying to question the notion of control, which I think is a question that still holds in the AGI scenario.

When you... (read more)

2Daniel Kokotajlo
I'm not saying it's the same, just that it's similar in the ways relevant to my argument. Questioning the notion of control is fine. I agree that colonial kings had limited control over their governors, etc. and I agree that an AGI takeover would involve the AGI having limited control over the humans it commands, at least at first. For more of what I mean on "game over," see this post. Basically, for purposes of timelines, we care about our loss of influence. Even if the AIs that wrest influence from us are themselves not in control, and lose influence to other AIs later, or whatever, it doesn't matter much from our perspective. Moreover it's not total loss of influence that matters, but relative loss of influence. If some event happens that reduces our influence by 90%, i.e. makes it ten times less likely for us to achieve a given amount of improvement in the world by our lights, then for planning purposes we should focus on plans that finish before that event occurs, or ideally plans that prevent that event from occurring. There's a related issue of "How bad will things get, by our lights, once we lose control?" I do think that in this sense, unfortunately, what we are facing is historically (mostly) unprecedented. The Aztecs were conquered by humans, after all. In the grand scheme of things the colonizers and colonized weren't that different, and so things turned out badly for the colonized but not as badly as they could have been. I think unfortunately that the divergence in values between AIs and humans is likely to be bigger than the divergence between historical colonized and colonizers. (EDIT: Oh, and I don't think the governors or people in the past were stupid either. And I didn't interpret you that way; my apologies if it sounded like I did.)
ckai30

I'm not sure that works when you're not starting with a unified culture.  Long term long distance delegation may only work if everyone shares the same doctrine, more or less, went to the same schools, has similar ideas about what the goal is, has similar loyalties, etc.  You install a governor in India and you have some idea of what sorts of things they may do, because you know how they were brought up.

But if you write to a subordinate "Sounds like we need to increase moral" and the next thing you hear about they've killed a lot of people and put... (read more)

4Daniel Kokotajlo
I think I agree with the picture you paint here; these are real difficulties. However, in context, I don't think it undermines my position. The context is, we are talking about how easy it would be for advanced agenty AGI to take over the world; my position is that it wouldn't need to start out with a large amount of resources and power, and that it would need to have an intelligence/tech advantage over its rivals but not a huge one. So yeah, I think from the AGIs perspective it will be very frustrating, its stupid human (and lesser-AGI) proxies and subordinates will be misunderstanding its intentions constantly, making bad decisions generally, and occasionally outright rebelling against its rule. But over time these issues will be resolved. Meanwhile from the perspective of the losers, well, we've lost. Compare to what happened with colonialism: All the things you describe about the difficulty of the king controlling his governor of India etc. actually happened quite frequently, but nevertheless from the perspective of the conquered peoples it was still game over.
ckai30

What level of communication technology do you consider necessary?  It seems that in order to control the whole world, you'd need a pretty high level of communication technology.

2Daniel Kokotajlo
I'm not sure. I wouldn't be surprised if it turns out that to control the whole world you need a certain level of communication technology, and that level wasn't reached until recently. However, my current guess is that communication tech would not be a constraint, because colonialism. European empires were able to maintain colonies all around the globe despite not being able to send messenges faster than sailing ships. I think having some form of writing/reading is probably crucial though. And probably clay tablets aren't enough; you probably need paper or something like it. Maybe even a printing press, but I'm not sure about that.
ckai20

Okay, but what is a bicycle?  

Is this?  
https://upload.wikimedia.org/wikipedia/commons/thumb/4/4e/Michauxjun.jpg/682px-Michauxjun.jpg

Is this?
https://en.wikipedia.org/wiki/File:1886_Starley_%27Rover%27_Safety_Cycle_British_Motor_Museum_09-2016_(29928044262).jpg

What about this?
https://en.wikipedia.org/wiki/File:Draisine_or_Laufmaschine,_around_1820._Archetype_of_the_Bicycle._Pic_01.jpg

Except for the pneumatic tire, about which I really have no idea, it seems pretty likely that a blacksmith and/or clockmaker in the past <i>could</i> make... (read more)

1Dirichlet-to-Neumann
I doubt bicycle would be very useful without good roads and latex tire. Synthetic latex is way above my knowledge - but natural latex is easy to find in Southern East Asia.
ckai80

You may find that this is a cycle.  At some points you may need to process for yourself and give more weight to what you think, but at some points you may feel the need for more information or an outside point of view.

ckai10

Yeah, I don't feel like that really covers it.  Maybe what I meant wasn't really utility per se, but rather an intuition about people ragequitting this system not just because of the moving issue but because of a lot of little unpredictable annoyances adding up, with moving into a new space being one example of how this could happen.  It seems like the more things change, the more unexpected annoyances are likely to pop up, whether within a living space, a neighborhood, the whole city, or whatever.

Like, a lot of people move one month (it could ev... (read more)

2mako yass
The word "Antifragile" springs annoyingly to mind. Constant weak shocks with lots of survivors. Maybe congestion prediction systems will tend to emerge? (Maybe those responses could be incorporated back into the utility function!?) Good thoughts. I think I can see a resolution. Moves could planned to take place throughout the month after the solution is proposed. It occurs to me now that the optimizers are going to be a bit more complex than I'd imagined. They can't just produce a mapping from residents to locations. Moves have to be ordered. The chains have to start at an unoccupied location and end before the month is out. Move plans could take the capacity of the moving contracters into account, if those are known quantities. Yeah. While this project is making me realize that going without firm guarantees is sometimes really useful (saying, "I don't know how good it will be" enables it to become unexpectedly good. Giving it broad latitude lets it compromise on things that turn out to be more costly than was anticipated.), and I think the optimizer might end up being pretty reliable, I think there will need to be quite a few firm guarantees.
ckai20

I think you're underestimating the utility of living in the same place for periods longer than a month.  Most real places have some problems that are annoying at first but easy (or at least possible) to work around once you've had a chance to figure out what the work arounds are.

You should probably also consider that some people will want to keep their preferences private, so giving everyone access to all preferences for the purposes of distributed optimization doesn't seem reasonable.

2mako yass
Then just increase the number in the utility function that I mentioned represents the utility of living in the same place for periods longer than a month. Are you asking about periods longer than that? I guess there's no reason the system couldn't look even further back. A move penalty that changes the longer they've been in that position. I've considered that, yeah, public incremental improvement processes wouldn't work for everyone. Likely it would have to go away after the initial stages. I'd originally planned for a process where the preference expression data would never leave city hardware, where solvers would instead send their optimization programs into city hardware, which would pass the data to the program, run for an hour or whatever, pass out the resultant solution, then reset.
ckai60

It seems pretty important to know who are the people who get to vote later on.  Since Twitter is a social network, is it people who are networked in some way, or is it random Twitter users, or...?  It seems like this could work very differently depending on exactly how it's implemented.  And how it works will influence whether anyone is willing to stake their cred on the results of a vote of this group of people.

 

What incentivizes anyone to vote at all, or to vote accurately?

ckai20

Ah, I was looking at your comment in the other direction, do ads cause prosperity or are ads correlated with prosperity (and having a hard time forming a model that ads cause prosperity while not finding ads being correlated with prosperity significant, really).  But I guess you were saying more that ads are a sign of prosperity and so even if they're annoying it's better than living somewhere that isn't prosperous enough for ads?

2aa.oswald
Yeah, that's exactly what I'm saying. Ads are, at the very most basic, just broadcasts of information with the intention of changing some behavior. Prosperity requires different actors in the economy to do what is most efficient, but often it isn't efficient for actors to actually go and track down their own suppliers. Instead, suppliers go and track down their buyers by broadcasting ads. The Internet has certainly changed that: my search history and browsing history effectively broadcast to suppliers, "I need a new car", and then the algorithms present me with information concerning that need (ads on instagram for Jeeps). Yet, the vast majority of people live in a pre-Internet situation for their most basic needs, where broadcasting from the supplier is still important.  Without this mechanism of "supplier broadcast", a lot of buyers would simply not be able to meet their needs because they don't have the bandwidth to go and fulfill their needs. The problem that Katja really should be getting at is when the supplier broadcast goes wrong- generating need when there isn't any, or broadcasting so much that it swamps out more important signals, etc. 
ckai20

Do you consider this connection to be correlation or causation?

1aa.oswald
Haha, it is definitely both. There's nothing to advertise! But there's nothing to advertise because the activities that benefit from advertising are illegal! 
ckai120

I am an outsider/lurker, so maybe I just don't get it, but it seems to me that even if the messaging around this event is changed to make it more clearly serious rather than there being a possible interpretation of all in fun no particular outcome is better than any other, there is a very real (not symbolic) mixed message going on with the way things are currently set up.  The first message is hey, we're doing this really cool ritual and you are invited to participate.  The second message is we don't want our website to go down so don't do anythi... (read more)

3habryka
Yeah, I think this is a real problem. I do think there is something a bit interesting about the ambiguity "The only way to win is not to play", but there are also a bunch of costs associated with the weird ambiguity, and I am not sure how things weigh up. The current thing and framing is the best we could come up with in a few dozen hours of collective thinking, but we can probably do better, and maybe the weird ambiguity is too costly, though overall I think we got the basic idea across to most people, despite the ambiguity, so I am hopeful that we can iterate on that and get us all the way everyone getting it.