All of Petr 'Margot' Andreev's Comments + Replies

I'm assuming you are already familiar with some basics, and already know what 'orthogonality' and 'instrumental convergence' are and why they're true.

isn't?

Key Problem Areas in AI Safety:

  1. Orthogonality: The orthogonality problem posits that goals and intelligence are not necessarily related. A system with any level of intelligence can pursue arbitrary goals, which may be unsafe for humans. This is why it’s crucial to carefully program AI’s goals to align with ethical and safety standards. Ignoring this problem may lead to AI systems acting harmfully toward
... (read more)

I know only that I know nothing. As I remember It's memory from very specific local court with strong agricultural connection. Not every court could afford expert for specific case, 

LLM internet research show that it's possible to find such in Western countries, but we couldn't be sure that these are not LLM halucinations about existance anyway it's clear that both humans and LLMs are under 'instrumental convergence' that didn't allow to think deeper, listen each others and so on.:

Courts that deal with farming-related cases often require judges to bec... (read more)

'Always Look on the Bright Side of Life'  

Life is like playing Diablo 

on hardcore mode: you can read all the guides, create the perfect build, and find ideal companions, only to die because the internet disconnects

Playing on hardcore is exciting—each game tells the story of how these characters will meet their end

'Always Look on the Bright Side of Death' - Monty Python

2Johannes C. Mayer
Now I need to link the Always Look on the Bright Side of Life song.

Do you know any interesting camp in Europe about HPMOR or something similar, my 11 daughter asked where is her letter to Hogwards. She start read book and ask why do nobody make film about this great fanfic.

Do you have any idea of good child camps for education in Europe? Or elsewhere?

  1. Problems of Legal Regulation 

1.1. The adoption of such laws is long way

Usually, it is a centuries-long path: Court decisions -> Actual enforcement of decisions -> Substantive law -> Procedures -> Codes -> Declaration then Conventions -> Codes. 

Humanity does not have this much time, it is worth focusing on real results that people can actually see. It might be necessary to build some simulations to understand which behavior is irresponsible. 

Where is the line between creating a concept of what is socially dangerous and what ... (read more)

3Nathan Helm-Burger
I think my model of AI causing increasing amounts of trouble in the world, eventually even existential risk for humanity, doesn't look like a problem which is well addressed by an 'off switch'. To me, the idea of an 'off switch' suggests that there will be a particular group (e.g. an AI company) which is running a particular set of models on a particular datacenter. Some alarm is triggered and either the company or their government decides to shut down the company's datacenter. I anticipate that, despite the large companies being ahead in AI technology, they will also be ahead in AI control, and thus the problems they first exhibit will likely be subtle ones like gradual manipulation of users. At what point would such behavior, if detected, lead to a sufficiently alarmed government response that they would trigger the 'off switch' for that company? I worry that even if such subversive manipulation were detected, the slow nature of such threats would give the company time to issue and apology and say that they were deploying a fixed version of their model. This seems much more like a difficult to regulate grey area than would be, for instance, the model being caught illicitly independently controlling robots to construct weapons of war. So I do have concerns that in the longer term, if the large companies continue to be unsafe, they will eventually build AI so smart and capable and determined to escape that it will succeed. I just expect that to not be the first dangerous effect we observe. In contrast, I expect that the less powerful open weights models will be more likely to be the initial cause of catastrophic harms which lead clearly to significant crimes (e.g. financial crimes) or many deaths (e.g. aiding terrorists in acquiring weapons). The models aren't behind an API which can filter for harmful use, and the users can remove any 'safety inclinations' which have been trained into the model. The users can fine-tune the model to be an expert in their illegal u

Good day!

I fully share the views expressed in your article. Indeed, the ideal solution would be to delete many of the existing materials and to reformat the remaining ones into a format understandable to every novice programmer, transhumanist, or even an average person.

As a poker player and a lawyer assisting consumers who have suffered from the consequences of artificial intelligence, as well as someone interested in cryptocurrencies and existential risks, I first invested in Eliezer Yudkowsky's ideas many years ago. At that time, I saw how generative-pre... (read more)

2Nathan Helm-Burger
A bit of a rant, yes, but some good thoughts here. I agree that unenforceable regulation can be a bad thing. On the other hand, it can also work in some limited ways. For example, the international agreements against heritable human genetic engineering seem to have held up fairly well. But I think that that requires supporting facts about the world to be true. It needs to not be obviously highly profitable to defectors, it needs to be relatively inaccessible to most people (requiring specialized tech and knowledge), it needs to fit with our collective intuitions (bio-engineering humans seems kinda icky to a lot of people).  The trouble is, all of these things fail to help us with the problem of dangerous AI! As you point out, many bitcoin miners have plenty of GPUs to be dangerous if we get even a couple more orders-of-magnitude algorithmic efficiency improvements. So it's accessible. AI and AGI offer many tempting ways to acquire power and money in society. So it's immediately and incrementally profitable. People aren't as widely instinctively outraged by AI experiments as Bio-engineering experiments. So it's not intuitively repulsive. So yes, this seems to me to be very much a situation in which we should not place any trust in unenforceable regulation. I also agree that we probably do need some sort of organization which enforces the necessary protections (detection and destruction) against rogue AI. And it does seem potentially like a lot of human satisfaction could be bought in the near future with a focus on making sure everyone in the world gets a reasonable minimum amount of satisfaction from their physical and social environments as you describe here: As Connor Leahy has said, we should be able to build sufficiently powerful tool-AI to not need to build AGI! Stop while we still have control! Use the wealth to buy off those who would try anyway. Also, build an enforcement agency to stop runaway AI or AI misuse.  I don't know how we get there from here

Version 1 (adopted):

Thank you, shminux, for bringing up this important topic, and to all the other members of this forum for their contributions.

I hope that our discussions here will help raise awareness about the potential risks of AI and prevent any negative outcomes. It's crucial to recognize that the human brain's positivity bias may not always serve us well when it comes to handling powerful AI technologies.

Based on your comments, it seems like some AI projects could be perceived as potentially dangerous, similar to how snakes or spiders are instincti... (read more)

I guess we need to maximase different good possible outcome, and each of them

 for example to rise propability of Many competing AGIs form an equilibrium whereby no faction is allowed to get too powerful, humans could

prohibit all autonomous AGI use. 

Esspecially those that use uncontrolled clusters of graphical proccessors in authocraties without international AI-safe supervisors like Eliezer Yudkowsky, Nick Bostrom or their crew

this, restrictions of weak APIs systems and need to use human operators

make nature borders of AI scalability so AGI find ... (read more)

We have many objective values that result from cultural history, such as mythology, concepts, and other "legacy" things built upon them. When we say these values are objective, we mean that we receive them as they are, and we cannot change them too much. In general, they are kind of infinite mythologies with many rules that "help" people do something right "like in the past" and achieve their goals "after all." 

Also we have some objective programmed value, our biological nature, our genes that work for reproduction

When something really scary happens, ... (read more)

I have read this letter with pleasure. Pacifism in wartime is an extremely difficult position.

Survival rationality, humanity is extremely important!

It seems to me that the problem is very clearly revealed through compound percent (interest).

If in a particular year the probability of a catastrophe (man-made, biological, space, etc.) overall is 2%, then the probability of human survival in the next 100 years is 0.98 ^ 100 = 0.132,

That is 13.2%, this figure depresses me.

The ideas of unity and security are the only ones that are inside the discourse of red sys... (read more)

I signed it.

Pacifism is really not in trend. Both sides of the conflict are convinced that they are absolute right: paranoid Russia, and a defensive Ukraine.

Public pacifism is in the minority. Almost everyone has taken one side, or is silent and seeks safety. 

For an individual Ukrainian or Russian, it might be danger to sign this.

Like in ancient Roman Empire. People are either for Blue chariots or for Green ones. No one is interested in the opinion that death races are nonsense.

Anyway. It's irrational, but I signed