All of derefr's Comments + Replies

In such cases, it more-often-than-not seems to me that the arguer has arrived at their conclusion through intuition, and is now attempting to work back to defensible arguments without those arguments being ones that would convince them, if they didn't first have the intuition.

Indeed, even knowing that in general I'm not a very jealous person, I was surprised at my own reaction to this thread: I upvoted a far greater proportion of the comments here than I usually do. I guess I'm more compersive than I thought!

There's a specific failure-mode related to this that I'm sure a lot of LW has encountered: for some reason, most people lose 10 "agency points" around their computers. This chart could basically be summarized as "just try being an agent for a minute sheesh."

I wonder if there's something about the way people initially encounter computers that biases them against trying to apply their natural level of agency? Maybe, to coin an isomorphism, an "NPC death spiral"? It doesn't quite seem to be learned helplessness, since they still ... (read more)

A continuum is still a somewhat-unclear metric for agency, since it suggests agency is a static property.

I'd suggest modelling a sentience as a colony of basic Agents, each striving toward a particular utility-function primitive. (Pop psychology sometimes calls these "drives" or "instincts.") These basic Agents sometimes work together, like people do, toward common goals; or override one-another for competing goals.

Agency, then, is a bit like magnetism--it's a property that arises from your Agent-colony when you've got them all pointing... (read more)

This seems to suggest that modelling people (who may be agents) as non-agents has only positive consequences. I would point out one negative consequence, which I'm sure anyone who has watched some schlock sci-fi is familiar with: you will only believe someone when they tell you you are caught in a time-loop if you already model them as an agent. Substitute anything else sufficiently mind-blowing and urgent, of course.

Since only PCs can save the world (nobody else bothers trying, after all), then nobody will believe you are currently carrying the world on your shoulders if they think you're an NPC. This seems dangerous somehow.

I note that this suggests that an AI that was as smart as an average human, but also as agenty as an average human, would still seem like a rather dumb computer program (it might be able to solve your problems, but it would suffer akrasia just like you would in doing so.) The cyberpunk ideal of the mobile exoself AI-agent, Getting Things Done for you without supervision, would actually require something far beyond equivalent to an average human to be considered "competent" at its job.

Not wanting to give anything away, I would remind you that what we have seen of Harry so far in the story was intended to resemble the persona of an 18-year-old Eliezer. Whatever Harry has done so far that you would consider to be "Beyond The Impossible", take measure of Eliezer's own life before and after a particular critical event. I would suggest that everything Harry has wrought until this moment has been the work of a child with no greater goal--and that, whatever supporting beams of the setting you feel are currently impervious to being kn... (read more)

Would you want to give the reader closure for the arc of a character who is, as the protagonist states, going to be coming back to life?

Personally, this reminds me more than anything of Crono's death in Chrono Trigger. Nobody mourns him--mourning is something to do when you don't have control over space and time and the absolute resolve to harness that control. And so the audience, also, doesn't get a break to stop and think about the death. They just hurl themselves, and their avatar, face-first into solving it.

Why not? Sure, you might start to recurse and distract yourself if you try to picture the process as a series of iterative steps, just as building any other kind of infinite data structure would—but that's what declarative data structure definitions were made for. :)

Instead of actually trying to construct each new label as you experience it, simply picture the sum total of your current attention as a digraph. Then, when you experience something, you add a label to the graph (pointing to the "real" experience, which isn't as easily visualized as t... (read more)

In the sociological "let's all decide what norms to enforce" sense, sure, a lack of "morality" won't kill anyone. But in the more speculative-fictional "let's all decide how to self-modify our utility functions" sense, throwing away our actual morality—the set of things we do or do not cringe about doing—in ourselves, or in our descendants, is a very real possibility, and (to some people) a horrible idea to be fought with all one's might.

What I find unexpected about this is that libertarians (the free-will kind) tend to think ... (read more)

Er, yes, edited.

He's quite prepared in a Hero's Journey sense, though. In Harry's own mind, he has lost his mentor. Thus, he is now free to be a mentor. And what better way to grow, as a Hero and über-rationalist, than to teach others to do what you do?

Of course, Harry would say that he's already doing that with Draco—but in the same way that he usually holds back his near-mode instrumental-rationalist dark side, he's holding back the kind of insights that Draco would need to think the way Harry thinks; Harry is training Draco to be a scientist, but not an instrumental ra... (read more)

this kind of question-dissolving is not the standard, evolution-provided brain pathway.

Hawkins would agree.

Whatever substrate supports the computation inscribing your consciousness would be necessarily real, under whatever sense the word "real" could possibly have useful meaning. ("I think; thinking is an algorithm; therefore something is, in order to execute that algorithm.")

Interestingly, proposing a Tegmark multiverse makes the deepest substrate of consciousness "mathematics."

We're built to play games. Until we hit the formal operational stage (at puberty), we basically have a bunch of individual, contextual constraint solvers operating mostly independently in our minds, one for each "game" we understand how to play—these can be real games, or things like status interactions or hunting. Basically, each one is a separately-trained decision-theoretical agent.

The formal operational psychological stage signals a shift where these agents become unified under a single, more general constraint-solving mechanism. We begin to ... (read more)

I would say that it is not that we want essences in our sexuality, but that gender and sexuality are essentialist by nature: the sexual drive is built on top of the parts of our brains that essentialize/abstract/encapsulate, and so reducing the concept would involve modifying the human utility function to desire the parts, rather than the pretended whole.

Or, to put it another way: a heterosexual blegg is not 50% attracted to something with 50% blegg features and 50% rube features; it is attracted only to pure rubes, and the closer something is to being a r... (read more)

It's pretty common, though. You wanted the other people reading to think of you as clever, and considered that to be "worth" making the author feel a bit bad. This is what the proxy-value of karma, as implemented by the Reddit-codebase discussion engine of this site, reflects: the author can only downvote once (and even then they are discouraged from doing so, unlike with, say, a Whuffie system), but the audience can upvote numerous times.

Thinking back, I've had many discussions on the Internet that devolved into arguments, where, although my int... (read more)

The errors of others, or the errors of those of superior social ranking? Do Korean teachers refrain from correcting students?

This is an example of Conservation of Detail, which is just another way to say that the contrapositive of your statement is true: if you don't need to take something in a game, then the designer won't have bothered to make it take-able (or even to include it.)

I always assume that there's all sorts of stuff lying around in an RPG house that you can't see, because your viewpoint character doesn't bother to take notice of it. It might just be because it's irrelevant, but it might also be for ethical reasons: your viewpoint character only "reports" things to you that his system of belief allows him to act upon.

This seems to track with the Eliezer's fictional "conspiracies of knowledge": if we don't want our politicians to get their hands on our nuclear weapons (or the theory for their operation), then why should they be allowed a say in what our FAI thinks?

In software development, this is known as being "Agile." Originally, software was designed mostly in head-land (a "Big Design Up Front"), but gradually a different process was pushed wherein a smaller, prototype design would first be constructed, then evaluated for its effects in real-land, and then improved upon, repeatedly. I find it interesting that unlike in the world of sports, where "one step at a time" can be almost universally agreed upon, software development is rife with controversy over whether "Agile software development methods" have any real advantages.

Here's a theory as to why: the experience may indeed be painful in the psychosocial context of our present society, but perhaps only in that context, or more specifically, because of that context.

That is, we have ideas of shame—that certain things are, or are not shameful—that are culturally based, and when we do things that offend our (learned) sense of shame, we feel, and remember, the associated negative emotions, without necessarily remembering their cause. We associate the negative emotions with the circumstance, instead of the long-gone prior that ca... (read more)