All of Coafos's Comments + Replies

Coafos10

The Nash equilibrium is

The Nash equilibrium is a solution for non-cooperative games. Politics, almost by definition, is about cooperation. Cooperative games are a superset of non-cooperatives, because in the case of no communication it reduces to one.

The word "signalling" in virtue signalling is a form of communication. Democracy has many forms, the most common is representative democracy. In it, the voter does not choose a policy directly, but chooses a delegate. The job of the delegate is to learn about the world (for example economics), and make a decis... (read more)

5lsusr
The central thesis of my post is that individual voters are not individually incentivized to vote well. For this reason, bad decisions by politicians do not necessarily result in them getting replaced in subsequent elections. Politicians are removed for violating mass stated preferences, which differ from good policy in predictable directions. Yes. I used economics in this post because it could perform double-duty, since this post also discusses moral hazard. The principle applies for foreign affairs, law, etc.
Coafos21

An arbitrary constant you expect to multiply by.

Usually, k is not just an arbitrary real number, but an integer, like in .

For arbitrary constants to multiply by I think  (lambda, greek letter) is used.

4Shankar Sivarajan
I agree that k strongly connotes integer, but I wouldn't expect to see λ used like that if not as a Lagrange multiplier.
Coafos30

I think this is a good object level post. Problem is, I don't think MIRI is at the object level. Quote from the comm. strat.: "The main audience we want to reach is policymakers."

Communication is no longer a passive background channel for observing a world, but speech becomes an action changing it. Predictions start to influence the things they predict.

Say AI doom is a certainty. People will be afraid, and stop research. Few years later doom doesn't happen, everyone complains.

Say AI doom is an impossibility. Research continues, something something papercli... (read more)

3Seth Herd
Interesting, thank you. I think that all makes sense, and I'm sure it plays at least some part in their strategy. I've wondered about this possibility a little bit. Yudkowsky has been consistent in his belief that doom is near certain without a lot more time to work on alignment. He's publicly held that opinion, and spent a huge amount of effort explaining and arguing for it since well before the current wave of success with deep networks. So I think for him at least, it's a sincerely held belief. Your point about the stated belief changing the reality is important. Everything is safer if you think it's dangerous - you'll take more precautions. With that in mind, I think it's pretty important for even optimists to heavily sprinkle in the message "this will probably go well IF everyone involved is really careful".
Coafos10

How to be the smartest person in the world:

Let's say A is smarter than B if A knows about topic X, but B doesn't.

Step 1: Let's say you don't know about quantum biology. But Charlie knows, because they currently do a phd in it.

Step 2: Go to Charlie. Say: "I heard about quantum biology, and it sounds interesting. Could you give quick intro on it?"

Step 3: Charlie says (eager to talk about the cool idea they found): "Sure. Quantum biology is [two hour forty-seven minute long monologe]."

Step 4: Important! Listen to it.

Step 5: BOOM! Now you also know about quant... (read more)

Coafos2-1

Taking a bad option away might be worse for a person, but will be much better for the people. These regulations (no selling organs or sex) exists, becuse in a free market there would be a race-to-bottom which would not increase human values.

Suppose we allow selling sex for rent. The number of rentable apartmants stays the same; however, there will more demand for them, because some people can now pay for them by non-monetary means. Because of this, the rent prices will increase, and that would just accelerate the rent-problem.

While exchanging kidneys for m... (read more)

2Brendan Long
These arguments seem to come from misunderstandings of why prices changes. This is written in a way that hides the important fact that rents rise in this situation because more people have housing, and more people having housing is good. You can write a fully-general argument against anything good like this. "Suppose we give poor people food. The amount of food stays the same, however, there will be more demand for it because some people don't have to pay for it. Because of this, food prices increase, and that will just accelerate the problem." The problem of high prices is that some people can't get the thing, but if prices are increasing because more people can get the thing then pointing at the higher price as bad doesn't make any sense. I'm confused about what this even means but isn't this a fully general argument against money and/or trade in general?
Coafos60

Searched PsyNet on Google, and I think PSYNet refers to the netcode for RocketLeague, a popular game. Maybe they pulled text message logs from somewhere; based on the "ForgeModLoader" token, it's plausible.

Alternative guess is this, a python library for online behavioural experiments. It connects to Dallinger and Mechanical Turk.

On Google, the string "PsyNetMessage" also appeared in this paper and at a few gpt2 vocab lists, but no other results for me.

On Bing/DuckDuckGo it outputted a lot more Reddit threads with RocketLeague crash logs. The crash logs are... (read more)

The world is a complicated and chaotic place. Anything could interact with everything, and some of these are good. This post describes that general paralysis of the insane can be cured with malaria. At least if they do not die during the treatment.

If late-stage syphilis (general paralysis) isn't treated, then they probably die 3-5 years with progressively worse symptoms each year. So even when 5-20% of the died immediately when the treatment started, they still had better survival rates in one and five years. A morbid example of an expected value choice: w... (read more)

What does this post add to the conversation?

Two pictures of elephant seals.

How did this post affect you, your thinking, and your actions?

I am, if not deeply, but certainly affected by this post. I felt some kind of joy looking at these animals. It calmed my anger and made my thoughts somewhat happier. I started to believe the world can become a better place, and I would like to make it happen. This post made me a better person.

Does it make accurate claims? Does it carve reality at the joints? How do you know?

The title says elephant seals 2 and c... (read more)

2Raemon
...what.... what was your idea?

I was always surprised that small changes in public perception, a slight change in consumption or political opinion can have large effects. This post introduced the concept of the social behaviour curves for me, and it feels like explains quite a lot of things. The writer presents some example behaviours and movements (like why revolutions start slowly or why societal changes are sticky), and then it provides clear explanations for them using this model. Which explains how to use social behaviour curves and verifies some of the model's predictions at the s... (read more)

There are lots of anecdotes about choosing the unused path and being the disruptor, but I feel this post explains the idea more clearly, with better analogies and boundaries.

To achieve a goal you have to build a lot of skills (deliberate practice) and apply them when it is really needed (maximum performance). Less is talked about searching for the best strategy and combination of skills. I think "deliberate play" is a good concept for this because it shows that strategy research is a small but important part of playing well.

I think this post points towards something important, which is a bit more than what the title suggests, but I have a problem describing it succinctly. :)

Computer programming is about creating abstractions, and leaky abstractions are a common enough occurrence to have their own wiki page. Most systems are hard to comprehend as a whole, and a human has to break them into parts which can be understood individually. But these are not perfect cuts, the boundaries are wobbly, and the parts "leak" into each other.

Most commonly these leaks happen because of a tech... (read more)

I think this post describes an important idea for political situations.

While online politics is a mind-killer, it (mostly) manages to avoid "controversial" topics and stays on a meta-level. The examples show that in group decisions the main factor is not the truth of statements but the early focus of attention. This dynamic can be used for good or bad, but it feels like it really happens a lot, and accurately describes an aspect of social reality.

Coafos175

If you heard "good is the enemy of the great", then also consider "perfect is the enemy of good".

Have you factored in the cost of task switching and meta-strategy research? A lot of economic theory trivializes the energy required for thinking, which might be correct for larger entities, but it's an important factor for individual humans.

Switching to a different worldview and re-evaluating your options takes a non-negligible amount of mental energy. Learning about new opportunities and hearing new strategies also takes time. If you're optimizing for doing g... (read more)

Coafos20

Thanks for the post, it's an important update on the state of information warfare.

Privacy can be thought of as a shield. If you build a wall against small-arm spam, then it's ok, but if you try to build an underground bunker, then it's weird because only Certified Good Guys have access to advanced weapons. Why are you trying to protect yourself against the Certified Good Guys?

What changed is that thanks to AI advancements in the last few years, it become possible to create homemade heat-seeking infomissiles. Suddenly, there are other arguments for building bunkers.

2[anonymous]
Because many of those Certified Good Guys happen to not be from my country.
3ponkaloupe
The underground bunker is only weird if I’m vocal about it. Am I posting this from behind tor? Do I pass my messages through an anti-stylometry filter before sharing them? I’m already operating behind a nonsense pseudonym, but if I used a real-sounding name that wasn’t my legal name, would you question it? It’s often the case that effective privacy techniques simply aren’t easily detectable by 3rd or even 2nd parties: the goal is frequently to blend in (to become a part of some larger “anonymity set”).
Coafos*50

Have you heard that the medium is the message? It was written before the internet happened, and said that society becomes tv-like or radio-like if it watches a lot of tv or radio.

It is interesting to see how this idea applies to the internet. I agree with you on that we should not handle the internet as a block, because each side has it's artifacts. I think there should be more explaration of ideas, but on other sites. LW in it's current form suited for long essay type posts, which I think is good for it's stated purpose, methodical discussion of ideas.

Coafos10

It's evening, the sun is set. A man walks up to a scholar:
"Scholar, the sun rose yesterday and today morning. Will it rise again tomorrow?"
"Man, I don't know, it's kinda dark right now. Have you heard about the no free lunch theorem?"

Coafos42

Marketers, scammers and trolls are trying-to control the internet bottom-up, joining the ranks of the users and going against internet institutions. While it's a problem (a possibly big one), a worse situation is when the internet instutions themselves start using LLMs for top-down control. For a fictional example, see heaven banning.

Coafos23

That's the second filter, because "optimizing" is two words: having a goal and maximising (or minimising) it.

First, one has to aknowledge that solving aligment is a goal. Many people does not recognize that it's a problem, beacuse smart robots will learn what love means and won't hurt us.

What you talked about in your post comes after this. When someone is walking towards the goalpost of alignment, they should realize that there might be multiple routes there and they should choose the quickest one, because only winning matters.

Coafos4-2

Thank you for writing this post, I think this is a useful framing of this problem. For me personally, the doom game is fun, imho I have more motivation to do things and I become more self-confident. (if it ends what worse could happen) But that's for me, with my socially isolated Math/ComSci/CosHo background.

For others, I don't think it's a good game. I kinda noticed the tons of psychotic breakdowns around the field and, like, that's bad, but I could not have articulated why it was bad.

And even for me, I might kinda overshoot with the whole information hazard share-or-not thinking. It's better if you're in charge of the game and not let the doom game play you.

1Valentine
Awesome. I have deep respect for this kind of conscious game-playing. Rock on!
Coafos43

ITT: Millenials lamenting about the decadence of youth :)

On a more serious note, awesome.

I like reading text, prefer transcripts of podcasts to the actual audio (I find it boring, even when sped up), and spend too much time looking at memes on reddit and facebook. Sometimes I yell at clouds.

Somehow, videos are missing from my media consumption. I attribute it to me using 3rd party apps for everything. These apps create a barrier in the endless flow, and I have to choose content more intentionally.

I started watching the videos, and holy shoes, you found the... (read more)

Coafos73

What about weekends? There are currently 104 days in a year where you're not supposed to work.

The big difference is that these days are uniformly distributed through the year, and aren't in a one or two week block.

Coafos21

Political polarization is very high in the US. This is a global phenomenom, and in other countries polarization is currently decreasing.

Coafos115

(Ep. vibes: I went to few EA cons, and subscribed to the forum digest.)

I blame EA. They were simply too successful.

There are the following effects at play:

  • Bad AI gonna kill us all :(
  • Preparing for emergent threats is one of the most effective ways to help others.
  • The best way to have good ideas is to have a lot of ideas; and the best way to have a lot of ideas is to have a lot of people.
  • Large funnels were built for new AI Safety researchers.
  • The largest discussions about the topic happened at LW and rat circles.
  • The general advice I heard at EA conferences in
... (read more)
2mruwnik
There's also the encouragement to post to LW as a method of getting noticed and in that way finding an alignment job/team/project
Coafos20

Great story. I haven't thought you could cross steampunk and singularity, but it kind of works.

Coafos30

Not like it gonna matter (<100), but if it did, I don't want future me to do the funni.

Coafos22

Once every month. (at least as I know it)

Coafos20

In section 1: When the 'affine' word is commonly used, the mixture coeffecient (denoted there by ) can be any real number. When , then it's still an affine combination, but a more precise tern could be 'convex combination'.

However, as you work with function spaces, convexity in a function space is different then being a convex function, so maybe some new notation should be introduced.

Coafos30

Have you heard about Infra-Bayesianism?

If I get it correctly, the core idea is that "consider every possible scenario, use a maximin policy while caring about conterfactual branches", which is very similar to the idea presented in the linked post. The "Nirvana trick" in the other post is similar to just eliminating branches/cells, where the agent would take a different action from the predicted policy.

Non-Nashian Game Theory is Pareto optimal, Infra-Bayesianism implements Updateless Decision Theory. If the two are connected, that could mean that UDT and Pareto-optimality are connected too.

Coafos10

One of my favourite Gettier-like problems is about black holes.

Say you have a very dense star. It is so dense, that the gravitational force on its surface is capable of pulling back even the particles of its light, leaving only a black hole in the sky. How large can it be with a given mass?

It's an easy exercise using Newtonian mechanics. Take a light particle with mass . Its gravitational energy at a distance is , and its kinetic energy is at the start. If the total energy is negative, then the path of the light particles will stay within a... (read more)

Coafos180

Interesting post, abstractions are the few stable-ish quantities that weren't eaten by chaotic noise.

Exponentially growing errors are not always chaotic. Suppose you have around 1000 starting cells, with a 1% error in the population size. The number of cells doubles in each hour. The absolute error of the population size can be 10.24 times larger than the initial population 10 hours later; however, the relative error remained 1%. (The billiard ball example is still chaotic, but the tilde character does the heavy lifting: 31.4 with 10% error is an imprecise... (read more)

Coafos30

I do not think that the prototypical scientific method is not valuable in the long term.

In any experiment, there are lots of naturally varying parameters (current phase of the Moon, air pressure, amount of snow on the slope), and there are lots of naturally constant parameters (strength of gravity, room temperature, amount of hydroxyhypotethicol in the solution). There are base and derived parameters. The distances from the sun and the orbital periods vary between the planets, but (distance)^3/(orbital period)^2 is constant.

In the experiment, you measure X... (read more)

Coafos20

I think a useful concept would be the colour of bits. For example, a digital song can be bought on a CD or downloaded from the internet. The computer does not see a difference between them, because it just sees a number, but in the eye of the law, one of them is legal, the other is not.

The number on the CD is coloured "green", the downloaded number is coloured "red". Green numbers are legal, but red numbers are not. If you upload a song a from CD, it will be red because you can only send red numbers. However, if the studio produces a new CD, it will have g... (read more)

Coafos*70

Hi!

I am a Mathematics university student from Europe. I don't comment often, and English isn't my native language, sorry for any mistakes in my tone or my language. I'm reading this site since March, but I heard about this site a long time ago.

I was always interested in computers and AI, so I found LW and Miri in 2015. But I didn't stay at that time. I think my entry point was when someone (around 2018 maybe?) recommended Unsong on reddit because it was weird and fun. I read a lot of stories on the rational fiction subreddit. (Somehow, I did not read HPMoR... (read more)