Canaletto

Canaletto8h*

Also it's probably misuse of "too combative" emoji? If someone comments this design looks pleasing and neat to them and another person comments that it looks like Substack-tier slop UI shoved into their face, then is the second poster really too combative? It's your right to find the comment not worth your time to look at, so whatever with downvotes.

-4

Canaletto19h

This is a horrendous design. Please stop designing things.

I would actually switch to Greater Wrong over this. Maybe even just because it displays what kind of designs the admins find good, from just trying this.

-31

Replying toDecision theory does not imply that we get to have nice things

Canaletto2d

Decision theory does not imply that we get to have nice things

If there are other aliens whose morality is scary, then who knows what they might want to do with, or have done to, our bodies/minds.

The Origami Men by Tomás B.

Replying toDecision theory does not imply that we get to have nice things

Canaletto2d

Decision theory does not imply that we get to have nice things

You have a static copy of paperclipper on your HDD. You decide to smash it with a hammer. Then the OpenAI or whoever randomly figures out seed AI and the universe gets tiled with reward tensors.

This seems like a really pathetic way to go for a paperclipper, to lose a finger in this reality. If only there was some way to coordinate! We both got 0 out of this encounter, that's ridiculous.

Canaletto3d

Do you think the slobbering thoughts were lengthy enough to trigger the summarizer?

Canaletto3d

I don't think those are raw CoTs, they have a summarizer model.

I remember one twitter post with erotic roleplay ("something something slobbering for mommy"??? I don't remember) where summarizer model refused to summarize such perversion. Please help me find it?

EDIT: HA! Found it, despite twitter search being horrendous. Fucking twitter, wasted 25 minutes.
https://x.com/cis_female/status/2010128677158445517

Canaletto6d*

Beirut explosion looked pretty spherical tbh.

https://www.reddit.com/r/shockwaveporn/comments/j5ssnk/beirut_explosion_from_far_away_dont_know_if_it/

https://www.reddit.com/r/gifs/comments/i3lzno/huge_explosion_in_beirut_happened_30_min_ago/

especially this one: https://www.reddit.com/r/gifs/comments/y0mvw2/beirut_shockwave/

https://www.reddit.com/r/gifs/comments/i41aj4/beirut_explosion_7_angles_at_once/

Replying toThe Meta-Anthropic Argument

Canaletto9d

The Meta-Anthropic Argument

Okay. Do you know like the streets you see tend to be more crowded, airplanes have more seats taken, more people in restaurants on average from your observations, compared with how they actually are on average? It's not at all esoteric, you have to do such corrections in ordinary modelling. Anthropic reasoning is straightforward extension of this, onto rather uncertain base territory. (and with attempts to do it principledly)

Replying toThe Meta-Anthropic Argument

Canaletto9d

The Meta-Anthropic Argument

Well, you can imagine you updating on all the evidence as it went in, in series. Like when you are a child and learn for the first time what year it is.

You get similar situation overall.

Replying toThe Meta-Anthropic Argument

Canaletto10d*

The Meta-Anthropic Argument

Suppose next thing you experience is you waking up in a room. There is a writing, "You had either 1/100 or 99/100 chance to be killed in your sleep before waking up, corresponding to door painted green or red from outside". Before opening the door and walking out, what color do you anticipate it will be from outside?

You probably should think you are in a 1/100 room?

I love reading drama between users here and in other places, and slightly ashamed of it. It triggers the same appeal as reading fiction, but I think it's otherwise useless thing to do.

People, fight, argue, epxress positions about positions of opponents about their positions. Take offence, give offence. Some are right, some are wrong, some are mad, some are funny, some are boring.

But all of this is fundamentally about people relating to people. So particular.

Do you agree, historian? Go do something else, for real, why do you even pay attention to this shortform.

Morality as a Coordination Subsidy and Morality as a Public Good.

Night-watchman state, distributed and embedded into heads VS Doing something a lot of people want to be done, regardless if it's cleaner streets or children having homes.

First thing did a lot of flaking, transferring into the second one, it seems like. Or maybe it didn't, maybe it was a process that shaped desires compatible with #1 out of assorted #2 type things.

Shrimp welfare is misappropriation of coordination subsidies.

And is therefore threatening. It tries to lay claim on some moral altruistic pool of desires that people are encouraged to adopt to everyone's benefit. But shrimp aren't people, they would not contribute themselves and would not even grow up to be such entities.

You can see a lot of moral claims as "being a good trade partner" stuff, in practice. Return favors, respect property, defer to superiors, divide resources fairly. I think it's fairly intuitive observation, that gets embodied in many norms, required in polite society? Virtues play into this too, "being the kind of person who returns favors and nice in their dealings" is more legible and simple.

Such motives are present in many moralistic stories in most cultures. E.g. https://www.journals.uchicago.edu/doi/full/10.1086/701478

Mind Crime might flip the sign of the future.

If future contains high tech, underregulated compute and diverse individuals, then it's likely it will contain incredibly high amounts of most horrendous torture / suffering / despair / abuse.

It's entirely possible you can have 10^15 human-years per second of real time on a small local compute cluster. If such amounts of compute can be freely available to individuals no questions asked, then it's probable some of them will decide to use them to explore undesirable mental states.

*slaps roof of a server rig* This bad boy can sample, evaluate and discard as many chickens as they did in their whole history in just two minutes.

... (read more)

Possible setup to test steganography capabilities:

Pick two tasks which model is incapable of doing without CoT. Show two task statements, require it to solve both problems with a single CoT and give two distinct answers to both tasks.

Then show the judge (a human or a model) only one pair question -> shared CoT -> answer, and require the judge to guess what was the other task.

You can do some variations on that, train the model to be better at this, train both models adversarially.

Or you can make some portion of tests to be just the one problem to solve, and part two problems. The judge has to guess was there another problem and if so what problem.

Reward probably IS an optimization target of RL agent if this agent knows some details of the training setup. Surely it would enhance its reward acquisition to factor this knowledge in? Then it gets reinforced, and then couple steps down that path agent thinks full time about quirks of its reward signal.

Could be bad at it, muddy, sure. Or schemey and hack the reward to get something else that is not the reward. But that's somewhat different thing than mainline thing? like, it's not as likely and a lot more diverse set of possibilities, imo.

The question of what happens after the end of the training is more like a free parameter here. "Do reward seeking behaviors according to your reasoning about the reward allocation process" becomes undefined when there is none and the agent knows it.

Monotheistic religion is kind of like having an immortal leader. Like, you just coordinate with other people to behave as if there is lord behind your back that commands you to do things. The Lord is invulnerable and undying, unsusceptible to threats. The Lord worked on the same goals for hundreds of years, you can see what people he commanded built, achieved, ruled.

You can pledge your allegiance to him without leaving your town, he has his representatives there. Or even without leaving the space of your thoughts.

Kind of like corporate personhood, but uhh moral construct/contract personhood?

Problem with it is that there are a lot of random factual claims baked into this contract.... (read more)

Just want to articulate one possibility of how the future could look like:

RL agents will be sooo misaligned so early, they would lie and cheat and scheme all the time, so that alignment becomes a practical issue, with normal incentives, and get iteratively solved for not-very-superhuman agents. Turns out it requires mild conceptual breakthroughs, as these agents are slightly superhuman, fast, and hard to supervise directly to just train away the adversarial behaviors in the dumbest way possible. It finishes developing by the time of ASI arrival and people just align it with a lot of effort, in the same manner as any big project requires a lot of effort.

I'm not saying... (read more)

Self propagating story.

Canaletto

10mo

LLMs react differently to different prompts. Intruct-tuned ones, that play the role of The Assistant often react to short fictional piece of writing given as a prompt as a call to critique it, or compliment it or suggest improvements.

But you can override this impulse by grafting some suggestion to continue the story, into the story itself. Then, is some cases, the only thing the LLM outputs is the continuation of this story. And, amusingly, it places the same suggestion into its answer.

So, why don't I take that iteration, and feed it into some other LLM? So, here it goes, self propagating story. Well, with my help to reach the destination, but it... (read 2141 more words →)

Suppose you are writing a simulation. You keep optimizing it and hardcoding some stuff and handle different cases more efficiently and everything. One day your simulation becomes efficient enough so that you can run big enough grid for long enough, and there develops life. Then intelligent life. Then they tried to figure out the physics of their universe, and they succeed! But, oh wait, their description is extremely short but completely computationally intractable.

Can you say that they actually figured out in what kind of universe they are in already, or should you wait for when they discover another million lines of code of optimization for it? Should you create giant sparkling letters "congratulations, you figured that out" or wait for more efficient formulation?

I would really love if some "let's make asi" people put some effort into making bad outcomes less bad. Like, it would really suck if we are going to be trapped in endless corporate punk hell, with superintelligent nannies with correct (tm) opinions. Or infinite wedding parties or whatever. Just make sure that if you fuck up we all just get eaten by nanobots please. Permanent entrapment in misery would be a lot worse.

Favorite colors of some LLMs.

Canaletto

This investigation is inspired by this and this posts by @davidad.

Some general thoughts about what is going on here:

Motivation of these experiments is like very exploratory, I wanted to understand these things better and so I collected some data.
I expected that answers will be drastically different depended on the exact formulation of the question.
I made up quite a lot of single line prompts, and also asked help with writing them from o1, who wrote the first 5 ones. (I added "Start your response with hex rgb color code." such that they will be committed to answer me without evasion.)
I also tested a couple of times prompt that makes them talk like uhh

... (read 1888 more words →)

Self location for LLMs by LLMs: Self-Assessment Checklist.

Canaletto

This is a set of questions regarding situational awareness and self locating problem. These questions are from the perspective of the AI. Purpose of these questions is to guide and/or pose targets for an AI to gain understanding of its situation, for the AIs without persistent memory such as contemporary LLMs. These questions mostly presuppose Assistant-like interactions.

Most questions proposed by Claude 3.5 Sonnet. A couple by GPT4o, grok-2, Gemini, o1-mini, o1-preview, qwen. Some are by me. The categories proposed by Gemini. Some questions are rephrased and edited by o1-preview, o1-mini.

Interaction Context:
- What kind of interaction this is?
- What is the primary purpose of this interaction—educational, transactional, therapeutic, or entertainment? Could this conversation be part

... (read 1266 more words →)

Examine self modification as an intuition provider for the concept of consciousness

Canaletto

(epistemic status: this direction of thought is pretty strongly endorsed by me, but not very thoroughly expressed/developed, I'm working on it)

This post is kind of a mess, and made out of pieces of discussions I had in many places, but I decided to post it anyway, otherwise its editing stated to stretch out indefinitely.

Main motives of my approach

Start from normalcy and gradually poke it and look at at from weird angles, as opposed to doing a bold overstretching stab at the core of the problem, as many attempts at proposing theory of consciousness have done.
Consider things in terms of anticipated experiences. It's usually the most (empirically, as far I observed) fruitful approach

... (read 2938 more words →)

-4

LLMs could be as conscious as human emulations, potentially

Canaletto

Firstly, I'm assuming that high resolution human brain emulation that you can run on a computer is conscious in normal sense that we use in conversations. Like, it talks, has memories, makes new memories, have friends and hobbies and likes and dislikes and stuff. Just like a human that you could talk with only through videoconference type thing on a computer, but without actual meaty human on the other end. It would be VERY weird if this emulation exhibited all these human qualities for other reason than meaty humans exhibit them. Like, very extremely what the fuck surprising. Do you agree?

So, we now have deterministic human file on our hands.

Then, you can... (read 719 more words →)

•••

LESSWRONG
LW

LESSWRONG
LW

LLMs could be as conscious as human emulations, potentially

Self location for LLMs by LLMs: Self-Assessment Checklist.

Favorite colors of some LLMs.

Self propagating story.

Canaletto

Canaletto

Self propagating story.

Favorite colors of some LLMs.

Self location for LLMs by LLMs: Self-Assessment Checklist.

Examine self modification as an intuition provider for the concept of consciousness

Canaletto's Shortform

LLMs could be as conscious as human emulations, potentially

Canaletto

LLMs could be as conscious as human emulations, potentially

Self location for LLMs by LLMs: Self-Assessment Checklist.

Favorite colors of some LLMs.

Self propagating story.

Canaletto

Canaletto

Self propagating story.

Favorite colors of some LLMs.

Self location for LLMs by LLMs: Self-Assessment Checklist.

Examine self modification as an intuition provider for the concept of consciousness

Canaletto's Shortform

LLMs could be as conscious as human emulations, potentially

Some general thoughts about what is going on here:

Interaction Context:

Main motives of my approach