All of RationalDino's Comments + Replies

Hypothetically this is possible.

But, based on current human behavior, I would expect such simulations to focus on the great and famous, or on situations which represent fun game play.

My life does not qualify as any of those. So I heavily discount this possibility.

I agree with your caveats.

However I'm not egocentric enough to imagine myself as particularly interesting to potential simulators. And so that hypothetical doesn't significantly change my beliefs.

This line of reasoning is a lot like the Doomsday argument.

I am well aware that my assumptions are my assumptions. They work for me, but you may want to assume something different.

I've personally interpreted the Doomsday argument that way since I ran across it about 30 years ago. Honestly AI x-risk is pretty low on my list of things to worry about.

The simulation argument never impressed me. Every simulation that I've seen ran a lot more slowly than the underlying reality. Therefore even if you do get a lengthy regress of simulations stacked on on simulations, most of the experience to have is in the underlying reality, and not in the simulations. Therefore I've concluded that I probably exist in reality, not a simulation.

If all beliefs in a Bayesian network are bounded away from 0 and 1, then an approximate update can be done to arbitrary accuracy in polynomial time.

The pathological behavior shows up here because there are two competing but mutually exclusive belief systems. And it is hard to determine when your world view should flip.

I hope that this makes it more interesting to you.

I like your second argument. But to be honest, there is a giant grey area between "non-replicable" and "fraudulent". It is hard to draw the line between, "Intellectually dishonest but didn't mean to deceive" and "fraudulent". And even if you could define the line, we lack the data to identify what falls on either side.

It is worth reading Time to assume that health research is fraudulent until proven otherwise? I believe that this case is an exception to Betteridge's law - I think that the answer is yes. Given the extraordinary efforts that the editor of An... (read more)

The effect isn't large, but you'd lose that bet. See Nonreplicable publications are cited more than replicable ones.

I have a simple solution for Pascal muggers. I assume that, unless I have good and specific reason to believe otherwise, the probability of achieving an extreme utility u is bounded above by O(1/u). And therefore after some point, I can replace whatever extreme utility is quoted in an argument with a constant. Which might be arbitrarily close to 0.

In some contexts this is obvious. For example if someone offers you $1000 to build a fence in their yard, you might reasonably believe them. You might or might not choose to do it. If they offered you $10,000, th... (read more)

It depends on subject matter.

For math, it is already here. Several options exist, Coq is the most popular.

For philosophy, the language requirements alone need AI at the level of reasonably current LLMs. Which brings their flaws as well. Plus you need knowledge of human experience. By the time you put it together, I don't see how a mechanistic interpreter can be anything less than a (hopefully somewhat limited) AI.

Which again raises the question of how we come to trust in it enough for it not to be a leap of faith.

Nobody ever read the 1995 proof.

Instead they wound up reading the program. This time it was written in C - which is easier to follow. And the fact that there were now two independent proofs in different languages that ran on different computers greatly reduced the worries that one of them might have a simple bug.

I do not know that any human has ever tried to properly read any proof of the 4 color theorem.

Now to the issue. The overall flow and method of argument were obviously correct. Spot checking individual points gave results that were also correct. The... (read more)

You may be missing context on my reference to the 4 color problem. The original 1976 proof, by Appel and Haken, took over 1000 hours of computer time to check. A human lifetime is too short to verify that proof. This eliminates your first option. The Objectivist cannot, even in principle, check the proof. Life is too short.

Your first option is therefore, by hypothesis, not an option. You can believe the AI or not. But you can't actually check its reasoning.

The history of the 4 color problem proof shows this kind of debate. People argued for nearly 20 years... (read more)

Concrete example.

Let's presuppose that you are an Objectivist. If you don't know about Objectivism, I'll just give some key facts.

1. Objectivists place great value in rationality and intellectual integrity.
2. Objectivists believe that they have a closed philosophy. Meaning that there is a circle of fundamental ideas set out by Ayn Rand that will never change, though the consequences of those ideas certainly are not obvious and still needs to be worked out.
3. Objectivists believe that there is a single objective morality that can be achieved from Ayn Rand's idea

What you want sounds like Próspera. It is too early to say how that will work out.

They took some inspiration from Singapore. When Singapore became independent in 1965, it was a poverty-stricken third world place. It now has a better GDP/capita than countries like the USA. And also did things like come up with the best way of teaching math to elementary school students.

But Singapore is only libertarian in some ways. They are also a dictatorship who does not believe in, for instance, free speech. Their point is that when you cram immigrants from many culture... (read more)

As a life strategy I would recommend something I call "tit for tat with forgiveness and the option of disengaging".

Most of the time do tit for tat.

When we seem to be in a negative feedback loop, we try to reset with forgiveness.

When we decide that a particular person is not worth having in our life, we walk them out of our life in the most efficient way possible. If this requires giving them a generous settlement in a conflict, that's generally better than continuing with the conflict to try for a more even settlement.

The first three are things most social... (read more)

Honest question about a hypothetical.

How would you respond if you set this up, and then your personal GPT concluded, from your epistemology and the information available to it, that your epistemology is fundamentally flawed and you should adopt a different one. Suppose further than when it tried to explain it to you, everything that it said made sense but you could not follow the full argument.

What should happen then? Should you no longer be the center of the prosthetic enabled "you" that has gone beyond your comprehension? Should the prosthetics do their ... (read more)

The reason why variance matters is that high variance increases your odds of going broke. In reality, gamblers don't simply get to reinvest all of their money. They have to take money out for expenses. That process means that you can go broke in the short run, despite having a great long-term strategy.

Therefore instead of just looking at long-term returns you should also look at things like, "What are my returns after 100 trials if I'm unlucky enough to be at the 20th percentile?" There are a number of ways to calculate that. The simplest is to say that if... (read more)

I'm sorry that you are confused. I promise that I really do understand the math.

In repeated addition of random variables, all of these have a close relationship. The sum is approximately normal. The normal distribution has identical mean, median, and mode. Therefore all three are the same.

What makes Kelly tick is that the log of net worth gives you repeated addition. So with high likelihood the log of your net worth is near the mean of an approximately normal distribution, and both median and mode are very close to that. But your net worth is the exponent ... (read more)

Dang it. I meant to write that as,

If you bet more than Kelly, you'll experience lower returns on average and higher variance.

That said, both median and mode are valid averages, and Kelly wins both.

I believe that AI safety is a real issue. There are both near term and long term issues.

I believe that the version of AI safety that will get traction is regulatory capture.

I believe that the AI safety community is too focused on what fascinating technology can do, and not enough on the human part of the equation.

On Andrew Ng, his point is that he doesn't see how exactly AI is realistically going to kill all of us. Without a concrete argument that is worth responding to, what can he really say? I disagree with him on this, I do think that there are realistic scenarios to worry about. But I do agree with him on what is happening politically with AI safety.

I am using Trump merely as an illustrative example of techniques.

My more immediate concern is actually the ability of China to shape US opinion through TikTok.

The simple reason to use Kelly is this.

With 100% odds, any other strategy will lose to Kelly in the long run.

This can be shown by applying the strong law of large numbers to the random walk that is the log of your net worth.

Now what about a finite game? It takes surprisingly few rounds before Kelly, with median performance, pulls ahead of alternate strategies. It takes rather more rounds before, say, you have a 90% chance of beating another strategy. So in the short to medium run, Kelly offers the top of a plateau for median returns. You can deviate f... (read more)

Social media has proven more than capable of creating effective social contexts for persuading people.

LLMs are perfectly capable of operating in these social contexts. Particularly if they have (as in the case of TikTok and China) the support of the owner of the site.

Do you have specific cause to believe that LLMs will fail to persuade in these social contexts?

You make a good point that Cruz et al may have different beliefs than they portray publicly. But if so, then Cruz must have had a good acting coach in late 2018.

About 70 million followers, you're right to call me out for overstating it. But according to polling, that's how many people believe that the 2020 election was stolen at the ballot box. So far he has lost dozens of election challenge cases, key members of his inner circle have admitted in court that there was no evidence, and he's facing multiple sets of felony charges in multiple jurisdictions.

I t... (read more)

With all due respect, I see no evidence that elites are harder to fool now than they were in the past. For concrete examples, look at the ones who flipped to Trump over several years. The Corruption of Lindsey Graham gives an especially clear portrayal about how one elite went from condemning Trump to becoming a die-hard supporter.

I dislike a lot about Mr. Graham. But there is no question that he was smart and well aware of how authoritarians gain power. He saw the risk posed by Trump very clearly. However he knew himself to be smart, and thought he could ... (read more)

I disagree that people who do ML daily would be in a good position to judge the risks here. The key issue is not the capabilities of AI, but rather the level of vulnerability of the brain. Since they don't study that, they can't judge it.

It is like how scientists proved to be terrible at unmasking charlatans like Uri Geller. Nature doesn't actively try to fool us, charlatans do. The people with actual relevant expertise were people who studied how people can be fooled. Which meant magicians like James Randi. Similarly, to judge this risk, I think you shoul... (read more)

Sorry, but you're overthinking what's required. Simply being able to reliably use existing techniques is more than enough to hack the minds of large groups of people, no complex new research needed.

Here is a concrete example.

First, if you want someone's attention, just make them feel listened to. ELIZA could already successfully do this in the 1970s, ChatGPT is better. The result is what therapists call transference, and causes the person to wish to please the AI.

Now the AI can use the same basic toolkit mastered by demagogues throughout history. Use simpl... (read more)

You would be amazed at what lengths many go to never learn.

Ever heard the saying (variously attributed) that A level people want to be around other A level people while B level people want to be around C level people?

A lot of those B level people are ones who stop getting better because they believe themselves to already be good. And they would prefer to surround themselves with people who confirm that belief than risk challenging themselves.

Furthermore, it is easier to maintain illusions of superior competency when it isn't competitive. It was a lot easie... (read more)

One example is the kind of person who began to learn something, worked at it, and became good at it compared to their friends. Without context for what "good" really means in the outside world, it is easy to believe that you are good.

In my blog I gave the example of myself as a teenager in chess. I could usually beat everyone in my school except my brother, so I felt like a good player.

But my competitive rating would have probably been about 1200-1400. I still remember my first encounter with a good chess player. A master was sitting in public, playing sim... (read more)

I agree that when you feel sure of your reasoning, you are generally more likely right than when you aren't sure.

But when you cross into feeling certain, you should suspect cognitive bias. And when you encounter other people who are certain, you should question whether they might also have cognitive bias. Particularly when they are certain on topics that other smart and educated people disagree with them on.

This is not a 100% rule. But I've found it a useful guideline.