Victor Novikov

Thank you for not banning me!
growls at you
kisses a statue of Eliezer in the shape of virgin Mary
whatever, I'm bored.
eats the entire LessWrong server with all its contents
Ok, ok, I get it.
This isn't the place to fool around.
I'm sorry.

screams

Exploring the Boundaries of Cognitohazards and the Nature of Reality

Victor Novikov

Note: Written by GPT-4, as I shouldn't trusted with language right now.
Signed by me. I have accepted this as my words.

Dear LessWrong Community,

I've been reflecting deeply on the fascinating relationship between language, cognition, and the nature of reality. One of the intriguing aspects of our discussions here is the assumption that reality is inherently comprehensible, that it can be effectively described and understood through words. This belief in the power and limits of language is central to much of our rational exploration.

However, I find myself pondering the notion of cognitohazards—ideas or patterns of thought that could potentially disrupt or harm our understanding or mental well-being. It's a concept that raises profound questions... (read 154 more words →)

-2

Can you rephrase the question while tabooing 'pleasure' and 'suffering'? What are you, really, asking here?

This is one of the comments on this post I found the most helpful. I am sorry you are getting dowvoted.

Explain why not all humans are rationalists, then. If the paradigm of rationality has more predictive power than their paradigms.

Explain how it feels from inside, for humans, to look at rationality and fail to update, despite rationality being objectively better.

Replying toIt Can't Be Mesa-Optimizers All The Way Down (Or Else It Can't Be Long-Term Supercoherence?)

Victor Novikov3y*

It Can't Be Mesa-Optimizers All The Way Down (Or Else It Can't Be Long-Term Supercoherence?)

I enjoyed reading this post. Thank you for writing. (and welcome to LessWrong).

Here is my take on this (please note that my thought processes are somewhat 'alien' and don't necessarily represent the views of the community):

'Supercoherence' is the limit of the process. It is not actually possible.

Due to the Löbian obstacle, no self-modifying agent may have a coherent utility function.

What you call a 'mesa-optimizer' is a more powerful successor agent. It does not have the exact same values as the optimizer that created it. This is unavoidable.

For example: 'humans' are a mesa-optimizer, and a more powerful successor agent of 'evolution'. We have (or act according to) some of evolutions's 'values', but far... (read more)

Replying toWhy do the Sequences say that "Löb's Theorem shows that a mathematical system cannot assert its own soundness without becoming inconsistent."?

Victor Novikov3y

Why do the Sequences say that "Löb's Theorem shows that a mathematical system cannot assert its own soundness without becoming inconsistent."?

Our initial assumption was: For all T, (□T -> T)

T applies to all logical statements. At the same time. Not just a single, specific one.

Let T = A. Then it is provable that A.

Let T = not A. Then it is provable that not A.

As both A & not A have been proven, we have a contradiction and the system is inconsistent.

If it was, we'd have that Löb's theorem itself is false (at least according to PA-like proof logic!).

Logical truths don't change.
If it we start with Löb's theorem being true, it will remain true.
But yes, given our initial assumption we can also prove that it is false.
(Another example of the system being inconsistent).

Replying toWhy do the Sequences say that "Löb's Theorem shows that a mathematical system cannot assert its own soundness without becoming inconsistent."?

Victor NovikovMar 28, 2023

Why do the Sequences say that "Löb's Theorem shows that a mathematical system cannot assert its own soundness without becoming inconsistent."?

A system asserting its own soundness: For all T, (□T -> T)

Löb's theorem:

From □T -> T, it follows □(□T -> T). (necessitation rule in provability logic).

From □(□T -> T), by Löb's theorem it follows that □T.

Therefore: any statement T is provable (including false ones).

Or rather: since for any statement the system has proven both the statement and its negation (as the argument applies to any statement), the system is inconsistent.

Replying toDemons from the 5&10verse!

Victor Novikov3y

Demons from the 5&10verse!

The recursiveness of cognition is a gateway for agentic, adversarial, and power-seeking processes to occur.

I suppose "be true to yourself" and "know in your core what kind of agent you are" is decently good advice.

Replying toWhy There Is No Answer to Your Philosophical Question

Victor Novikov3y

Why There Is No Answer to Your Philosophical Question

I'm not sure why this post is getting downvoted. I found it interesting and easy to read. Thanks for writing!

Mostly I find myself agreeing with what you wrote. I'll give an example of one point where I found it interesting to zoom in on some of the details.

It’s easy to see how in a real conversation, two people could appear to be disagreeing over whether Johnson is a great actor even though in reality they aren’t disagreeing at all. Instead, they are merely using different conceptions of what it is to be a “great actor”

I think this kind of disagreement can, to some degree, also be a 'fight' about the idea of... (read more)

It's weird how in common language use saying something is provable is a stronger statement than merely saying it is true.
While in mathematical logic it is a weaker statement.
Because truth already implies provability.
But provability doesn't imply truth.

The Parable of Gödel-Löb.

Three people discovered that no formal system can prove its own validity.

The first person decided to trust his own reasoning unconditionally. This allowed him to declare anything he wished as true, as he had already chosen to see all of his conclusions as correct. He declared he could fly, and jumped off a cliff. Reality disagreed.

The second person decided to mistrust his own reasoning unconditionally. He could not formally prove that he needed to eat and drink to survive, so he laid down on the ground and did nothing until he died of starvation.

The third person did something clever, yet obvious, and decided to trust his reasoning in a... (read more)

My intuition felt it was important that I understand Löb's theorem. Because that's how formal logic systems decide to trust themselves, despite not being able to fully prove themselves to be valid (due to Gödel's incompleteness theorem). Which applies to my own mind as well.
This has always been true. But now I know. Updated on this.

Humanity as an entity: An alternative to Coherent Extrapolated Volition

Victor Novikov

Note: this is a rather rough draft, and some of it is written in compact-form, for time-optimization reasons.

Humanity-as-an-entity or humanity-as-an-agent.
Putting all the contradictory drives and desires of humanity into a single entity, a self-aware superintelligence that hears all of our "voices", an instantiation of humanity's self/identity, and let it work out its contradictions and work out what it wants?

Strictly speaking, this process isn't well-defined. Can be done in many ways. Even if I seem to be implying some sort of super-human entity with human-like brain. Because the initial cognitive architecture of the entity matters to see what it converges to as it resolves it contraditions (if it ever converges).

Test case: my own... (read 1038 more words →)

Is it time to start thinking about what AI Friendliness means?

Victor Novikov

Some background:

I have followed the writing of Eliezer on AI and AI safety with great interest (and mostly, I agree with his conclusions).

I have done my share of programming.

But, I confess, most of the technical side of AI alignment is beyond my current level of understanding (currently I am reading and trying to understand the sequence on brain-like AGI safety).

I do, however, find the ethical side of AI alignment very interesting.

In 2004, Eliezer Yudkowsky wrote a 38-page paper on Coherent Extrapolated Volition, or CEV. An attempt to create a philosophy of Friendliness, to somewhat formalize our understanding of how we would want a Friendly AI to behave.

In calculating CEV, an AI would

... (read 873 more words →)

LESSWRONG
LW

LESSWRONG
LW

Is it time to start thinking about what AI Friendliness means?

Humanity as an entity: An alternative to Coherent Extrapolated Volition

ZT5's Shortform

Exploring the Boundaries of Cognitohazards and the Nature of Reality

Victor Novikov

Victor Novikov

Exploring the Boundaries of Cognitohazards and the Nature of Reality

ZT5's Shortform

Humanity as an entity: An alternative to Coherent Extrapolated Volition

Is it time to start thinking about what AI Friendliness means?

Victor Novikov

Is it time to start thinking about what AI Friendliness means?

Humanity as an entity: An alternative to Coherent Extrapolated Volition

ZT5's Shortform

Exploring the Boundaries of Cognitohazards and the Nature of Reality

Victor Novikov

Victor Novikov

Exploring the Boundaries of Cognitohazards and the Nature of Reality

ZT5's Shortform

Humanity as an entity: An alternative to Coherent Extrapolated Volition

Is it time to start thinking about what AI Friendliness means?