Epistemic status: very unsure, just got a possibly interesting philosophical concept to share from my thoughts. Confidence: seems possible but hard to evaluate Might not be a very original thought but never heard of it. Traditional views see nothingness as the simplest possible state - a complete absence of everything....
I think there are multiple moral worldviews that are rational and based on some values. Likely the whole continuum.
The thing is that we have values that are in conflict in edge cases, and those conflicts need to be taken into account and resolved when building a worldview as a whole. You can resolve them in many ways. Some might be simple like "always prefer X", some might be more complex like "in such and such circumstances or precoditions prefer X over Y, in some other preconditions prefer Z over Y, in some other ...". It might be threshold-based when you try to measure the levels of things and weight them mathematically or... (read 483 more words →)
Part of the animal nature, including humans, is to crave novelty and surprise and avoid boredom. This is pretty crucial to the learning process in a changing and complex environment. Humans have multi-level drives, and not all of them are well-targeted on specific goals or needs.
It is very visible in small children. Some people with ADHD, like me, have a harder time regulating themself well and this is also especially visible for us, even when being adult. I know exactly what I should be doing. This is one thing. I also may feel hungry. That's another thing. But still, I may indulge in doing a third thing instead - something that satiates... (read more)
I think there are only two likely ways how the future can go with AGI replacing human labor - if we somehow solve other hard problems and won't get killed or wireheaded or get a dystopian future right away.
My point of view is based on observations of how different countries work and their past directions. However, things can go differently in different parts of the world. They can also devolve into bad scenarios, even in parts that you would think are well-posed to be good.
I think it might be reformulated the other way around: Capabilities scaling tends to increase existing alignment problems. It is not clear to me that any new alignment problem was added when capabilities scaled up in humans. The problem with human design, which is also visible in animals, is that we don't have direct, stable high-level goals. We are mostly driven by metric-based goodharting prone goals. There are direct feelings - if you feel cold or pain you do something that will make you not feel that. If you feel good, you do things that lead to that. There are emotions that are kind of similar but about internal state. Those are... (read 565 more words →)
Right now I think you can replace junior programmers with Claude 3.5 Sonnet or even better with one of the agents based on a looped chain of thoughts + access to tools.
On the other hand, it does not yet go in that direction for being a preferred way to work with models for more advanced devs. Not for me, and not for many others.
Models still have strange moments of "brain farts" or gaps in their cognition. It sometimes makes them do something wrong and cannot figure out how to do that correctly until told exactly how. They also often miss something.
When writing code if you make such an error and build on... (read more)
Thought on short timelines. Opinionated.
I think that AGI timelines might be very short based on an argument taken from a different side of things.
We all can agree that humans have general intelligence. If we look at how our general intelligence evolved from simpler forms of specific intelligence typical for animals - it wasn't something that came from complex interactions and high evolutional pressure. Basically there were two aspects of that progress. The first one is the ability to pass on knowledge through generations (culture). Something that we share with some other animals including our cousins chimpanzee. The second one is intersexual selection - at some moment in the past, our species started... (read more)
I think that in exchange:
the person asking "what do you mean" got confused about the nuances of verbal and non-verbal communication.
Nearly all people understand that "good morning" does not state the fact of the current morning being good, but a greeting with a wish for your morning to be good.
The answer "mornings aren't good" is an intended pun using the too-literal meaning to convey the message that the person does not like mornings at all. Depending on intonation it might be a cheeky comment or suggestion that they are not good because of the person greeting (f.ex. if they need to wake up early because of them every day).
Epistemic status: very unsure, just got a possibly interesting philosophical concept to share from my thoughts.
Confidence: seems possible but hard to evaluate
Might not be a very original thought but never heard of it.
Traditional views see nothingness as the simplest possible state - a complete absence of everything. This is a very specific state - it lacks everything, starting from basic math concepts and ending with complex things including physics. We might consider an alternative perspective: perhaps nothingness actually contains a superposition of all logically possible states, models, and systems, with their probability weighted by the inverse of their complexity. In this view, simpler models have a higher probability in this superposition, but... (read 344 more words →)
There is a practical reason to subscribe more to the Camp 1 research, even if you are in Camp 2.
I might be wrong, but I think the hard problem of qualia won't be solvable in the near future, if at all. To research something you need N > 1 of that phenomenon. We, in some sense, have N = 1. We have it ourselves to observe subjectively and can't observe anyone else qualia. We think other humans have it based on the premise they say they have qualia and they are built similarly so it's likely.
We are not sure if animals have it as they don't talk and can't tell us so.... (read more)
In many publications, posts, and discussions about AI, I can see an unsaid assumption that intelligence is all about prediction power.
In articles that I read, I often see a case made for optimization processes that tend to sacrifice as much as possible of the value on dimensions that the agent/optimizer does not care about for a very minuscule increase on dimensions that change the perceived total value. For example, AI that creates a dystopia that is very good on some measures, but really bad on some other just to refine those that matter for it.
What I don't see analyzed that much is that agents need to be self-referencing in their thought process, and on a meta level, also take their thought process itself and its limits and consequences as part of their... (read more)