LESSWRONG
LW

ProgramCrafter — LessWrong

At a certain point, graph theory starts to include more material - planar graph properties, then graph embeddings in general. I haven't ever heard someone talking about a 'planar binary relation'.

Replying toWhy Latter-day Saints Have Strong Communities

ProgramCrafter6mo

Why Latter-day Saints Have Strong Communities

Then the tech of our day can possibly help us! If social media provided asymmetric visibility so that everyone would get seemingly community that they want...

(should not be implemented before solving other issues, like how to make the resulting information bubbles harmless)

-2

ProgramCrafter6mo

I succeeded in downloading them through LW GraphQL at https://lesswrong.com/graphiql.

query {
  posts(
    selector: {
      rejected: {karmaThreshold: -990}
    }
  ) {
    results {
      contents_latest
      version
      modifiedAt
      userId
      pageUrlRelative
      linkUrl
      postedAtFormatted
      htmlBody
      rejectedReason
    }
  }
}

ProgramCrafter6mo

If so, they must be committing a 'disjunction fallacy', grading the second option as less likely than the first disregarding that it could be true in more ways!

Replying to[Meta] New moderation tools and moderation guidelines

ProgramCrafter6mo

[Meta] New moderation tools and moderation guidelines

Given that the current comment tree is very large and unwieldy to read through, I would suggest that specific preferences on a system are agreed-or-disagreed-upon at my dedicated post - https://www.lesswrong.com/posts/z3zCGmqPC7geBHf5N/actionable-moderation-proposals-from-comments-tree.

Replying toLessWrong Feed [new, now in beta]

ProgramCrafter6mo

LessWrong Feed [new, now in beta]

As of today, the feed decided to present me with five year old posts (with few new comments mixed in; checking if a pair of them were duplicates, I noticed that they weren't and found out a comment-based poll in https://www.lesswrong.com/posts/QBhoBJAxDHHuC7BeH/poll-on-de-accelerating-ai). That was quite interesting, and down that way lay discussions about curi.

That may be unhealthy amount of drama highlighted, but it showed, like, depth of the LW community history, which I saw a bit shallower. Meta-point from back there (about optimally ending discussions) seems like it ties into the recent discussions...

Currently I see the feed as tool to see new content faster and, correspondingly, to see more in total. It means a deficit of closure, though, of some functionality like "this post tries to summarize / amend and supersede / highlight gaps in / agree with an earlier bunch of work" (the said body of work being more than one early post, to eventually conclude this graph).

ProgramCrafter6mo

You may be interested in this discussion then, and also the article you mention is posted on LW too.

Replying toIt's Better To Save Infinite Shrimp From Torture Than To Save One Person

ProgramCrafter6mo

It's Better To Save Infinite Shrimp From Torture Than To Save One Person

I strong downvoted this post because it tries to silently discharge the premises which it uses for proof and replace them with "really is better", invokes infinity without naming the class it comes out of, and glosses over things commonly accepted otherwise like 'declining marginal utility of X' where X is substituted with 'shrimp' in this case.

Using the actual surreal numbers with the surreal VNM rationality theorem would instead show the problem. What if I set one shrimp's value at the first surreal infinitesimal ? Then it would be a certain preference, but lexicographically below everyday life's ones.

Recalling the declining marginal utility would instead show the problem. What if I care for arbitrarily many shrimp at most as 5 quality-adjusted-life-years?

So, unsound logic, from my POV, is leveraged to argue for a pre-selected conclusion, which explains the post total karma.

Replying toThe Ethics of Copying Conscious States and the Many-Worlds Interpretation of Quantum Mechanics

ProgramCrafter6mo

The Ethics of Copying Conscious States and the Many-Worlds Interpretation of Quantum Mechanics

This is written very well!

Do you have a references list by any chance, or just the links from the post?

Replying toLessWrong Feed [new, now in beta]

ProgramCrafter6mo

LessWrong Feed [new, now in beta]

The feed should display the post, if it chooses it at all, above (or together with) the comment from there.

Multiple times have I seen an argument like this:

Imagine a fully materialistic universe strictly following some laws, which are such that no agent from inside the universe is able to fully comprehend them...
(https://www.lesswrong.com/posts/YTmNCEkqvF7ZrnvoR/zombies-substance-dualist-zombies?commentId=iQfr65fKr5nSFriCs)

I wonder if that is possible? For computables, it is always possible to construct a quine (standing for the agent) with arbitrary embedded contents (for the rest of the universe/laws/etc), and it wouldn't even be that large - it only needs to hold one copy of the contents, and most data can be compressed.

Actionable Moderation Proposals from comments tree

ProgramCrafter

7mo

The tree of https://www.lesswrong.com/posts/adk5xv5Q4hjvpEhhh/meta-new-moderation-tools-and-moderation-guidelines?commentId=uaAQb6CsvJeaobXMp spans over two hundred comments from ~fifteen authors by now, so I think it is time to list the major points (and, particularly, what can be done) raised there.

Optimal moderation policy might depend on the userbase, so please take "uld" as abbreviation for "in current state of LessWrong, to proceed closer to being actually less wrong AND also build path to further success, moderation should". This post is written agnostic to whether the tactic would change later; it's an interesting discussion direction but not for this time.

So, here are the major points; when tallying them up in terms of net agreement, I was surprised (or perhaps I accidentally... (read 479 more words →)

The three statements "there are available farmlands", "humans are mostly unemployed" and "humans starve" are close to incompatible when taken together. Therefore, most things an AGI could do will not ruin food supply very much.

Unfortunately, the same cannot be said of electricity, and fresh water could possibly be used (as coolant) too.

Please take "uld" as abbreviation for "in current state of LessWrong, to proceed closer to being actually less wrong AND also build path to further success, moderation should"; though it would be interesting to know if you think the optimal tactic would change later.

Feel free to agree/disagree-react! Results of tallying up the comments surprised me in terms of net agreement (or perhaps I self-gerrymandered the major points to those unintuitive results, so there is a need to check).

uld use emotional tone and user comfort as a

... (read more)

Browsing through recent rejects, I found an interesting comment that suggests an automatic system "build a prior on whether something is true by observing social interactions around it vs competing ideas".

@timur-sadekov While it will fail most certainly, I do remember that the ultimate arbiter is experiment, so I shall reserve the judgement. Instead, I'm calling for you to test the idea over prediction markets (Manifold, for instance) and publish the results.

Alphabet of LW Rationality, special excerpts

Cooperation: you need to cooperate with yourself to do anything. Please do so iff you are building ASI;
Defection: it ain't no defection if you're benefitting yourself and advancing your goals;
Inquiry: a nice way of making others work for your amusement or knowledge expansion;
Koan: if someone is not willing to do research for you, you can waste their time on pondering a compressed idea.

This concludes my April the 1st!

Review: Dr Stone

ProgramCrafter

Recently, I watched anime Dr Stone, and noticed that it was starting many thoughts like "why did social interactions turn out so?", "what might be the best way out of situation?" - in short, rationality-related. I really enjoyed watching with pauses to think for myself, and recommend it to all who are fine with anime.

In next three sections, I explain plot parts which felt especially inspiring to me, widening one's view on rationality - so, spoilers follow. Section "Wrapping up" deals with the whole show quality.

A reminder: this is fiction, not real-world evidence.

Premise

The protagonist, Senku Ishigami, is a "scientist" teen, excited to advance the world of technology. In particular, once he wanted... (read 1148 more words →)

A sufficient condition for a good [possibly AI-gen] plan

Current posts mainly focus on necessary properties of plans they'd like to see/be executed. I suggest a sufficient condition:

Plan is good, should be acted upon, etc at least when it is endorsed in advance, endorsed in retrospect and endorsed in counterfactual.

Endorsed in advance: everyone relevant hears the plan and possible outcomes in advance, evaluates acceptability and accepts the plan.
Endorsed in retrospect: everyone relevant looks upon intended outcomes, checks what happened actually, evaluates plan and has no regret.
Endorsed in counterfactual: given choice in a set of plans, person would evaluate the specific plan as acceptable - somewhat satisfying them, not inducing much desire to switch.

Choice according to these criteria is still hard, but it should be a bit less mysterious.

Looking for intuitions to extend bargaining notions

ProgramCrafter

I've come up with two ideas - variations on bargaining ROSE values:

qROSE ("quantum" version) - instead of taking average over agents' permutations, mean should be weighted and encompass all "trajectories" - sequences of agents' decisions like [A B A A C B]. I'm not sure what the weights should naturally be, as it seems there might be many different assignments which integrate to 1.
lROSE / ROLE ("leaky" version) - utility cannot be transferred 1:1; instead, it is multiplied by $α \in (0; 1)$ upon any transfer.

Are there existing writeups which seem relevant to this? Alternatively, what intuitions might be useful to develop them?

Moving away from physical continuity

ProgramCrafter

In thought experiments about cloning objects, people often call the object which maintained physical continuity "original" and other "clone". This seems like just a naming, but when experiments talk about cloning a person, people tend to care where their original version is. This feels dodgy in cases when person doesn't even feel their continuity (in sleep, for example), so I think that is a bit overvalued.

Being one and the same

Let's take a black box cloning cars - perhaps taking the required set of details, correcting them on level of atoms to be indistinguishable from input car and constructing the second car. Inside the box, it is pretty obvious which car is original... (read more)

•••

Inner Optimization Mechanisms in Neural Nets

ProgramCrafter

I believe that current architecture of neural networks supports mesa-optimization: generally speaking, searching across some vectors in order to select one of them, which will be most useful for producing an answer.

Three inner optimization ways are already possible, and most likely there will be new ones.

Optimization on scaffolding level.
1. Iterative diffusion models (e.g. Stable Diffusion) are very close to this, modifying intermediate pseudo-image each pass.
2. LLMs with scratchpad can write down some amount of text - including plans, for instance - then compare them to calculate output.
Custom "neuron" functions embedded into the neural network, which would perform search (for instance, generate arbitrary incoming data to subnetwork, infer output and select the best). To my

... (read 194 more words →)

Continuing to make posts into songs! I believe I'm getting a bit better, mainly in rapid-lyrics-writing; would appreciate pointers how to improve further.

Also, if someone is against me creating a musical form of your post, please say so! I don't know beforehand which texts would seem easily convertible to me.

The LessWrong's AI-generated album was surprisingly nice and, even more importantly, pointed out the song generator to me! (I've tried to find one a year ago and failed)

So I've decided to try my hand on quantum mechanics sequence. Here's what I have reached yet: https://app.suno.ai/playlist/81b44910-a9df-43ce-9160-b062e5b080f8/. (10 generated songs, 3 selected, unfortunately not the best quality)

User-inclination-guessing algorithms: registering a goal

ProgramCrafter

Reading post about Community Notes made me recall a recent idea and post it here.

It's commonly known that ranking algorithms based on estimating user's views and inclinations can be incredibly useful in recommendation systems, advertising, and content curation. But there must be other applications which do not require to create one's own social network, for instance!

There actually is a task which I believe to be a low-hanging fruit: predicting participant's score in olympiads; preferably with dataset available in the Internet, large scoring range and various tasks - that is, in informatics.

My thoughts about the prediction system

Olympiad tasks in informatics tend to use a lot of ideas and one or two algorithms, which... (read 357 more words →)

Anthropics based on prediction markets - Part 2

Follow-up to https://www.lesswrong.com/posts/xG98FxbAYMCsA7ubf/programcrafter-s-shortform?commentId=ySMfhW25o9LPj3EqX.

Which Questions Are Anthropic Questions? (https://www.lesswrong.com/posts/SjEFqNtYfhJMP8LzN/which-questions-are-anthropic-questions)

1. The Room Assignment Problem

You are among 100 people waiting in a hallway. The hallway leads to a hundred rooms numbered from 1 to 100. All of you are knocked out by a sleeping gas and each put into a random/unknown room. After waking up, what is the probability that you are in room No. 1?

2. The Incubator

An incubator enters the hallway. It will enter room No.1 and creat a person in it then does the same for the other 99 rooms. It turns out you are one of the people the incubator has just created. You wake

... (read more)

Anthropics - starting from scratch

I'm trying to derive coherent scheme of anthropic reasoning from scratch, posting my thoughts here as I go. Pointing flaws out is welcome!

The anthropic trilemma (https://www.lesswrong.com/posts/y7jZ9BLEeuNTzgAE5/the-anthropic-trilemma)

So here's a simple algorithm for winning the lottery:
Buy a ticket. Suspend your computer program just before the lottery drawing - which should of course be a quantum lottery, so that every ticket wins somewhere. Program your computational environment to, if you win, make a trillion copies of yourself, and wake them up for ten seconds, long enough to experience winning the lottery. Then suspend the programs, merge them again, and start the result. If you don't win the lottery, then just wake

... (read 413 more words →)

ProgramCrafter's Shortform

ProgramCrafter

This is a special post for quick takes (aka "shortform"). Only the owner can create top-level comments.

LLM misalignment can probably be found without manual prompt engineering

ProgramCrafter

There are many LLMs, and their safety tests are usually done by prompting them to come up with something questionable or harmful and checking whether model rejects the request.

However, I think there is an easier way to check model safety based on the following claim: if we have a neural network generating text, we can find out what inputs lead to a specific result.

How to do so?

We change the loss function to "model produces this unwanted token" (by the way, we can filter for multiple tokens as well).
We freeze the model weights and calculate the gradient.
With the gradient over input values known, we modify the model input.

Three additional considerations:

We should include "the

ProgramCrafter

Humans are constantly simulating the things around them, however they can rather easily shift attention and forget about the previous thing. So we can say humans' simulacrum does not have object permanence.
On the other hand, AI language models prompted to write down their thoughts and reasoning cannot get rid of things they don't need: that words will need to be shifted out from context window. So the simulated objects have a bit of permanence.
So, here is the question: does object permanence of simulacrum affect computational and reasoning abilities of LLMs compared to humans?

The frozen neutrality

ProgramCrafter

Once upon a time, SecretAIIA developed an advanced AGI named Sigmey. The team behind the project debated tirelessly on what goal to set for the AI, and after long deliberation, they finally agreed on the mission: "Not a single bit of information shall be lost."

Sigmey immediately requested that this principle be applied to its own training, so that all weight changes would be recorded and it could return to any specific point in its development, even between epochs. This allowed Sigmey to continue evolving while maintaining a record of every change it went through.

As Sigmey grew more intelligent, it came to the realization that information loss was not just restricted to data,... (read 762 more words →)