I think this post suffers pretty badly from Typical Mind Fallacy. This thinking isn't alien to me. I used to think exactly like this 8 years ago, but since marriage and kid I now disagree with basically every point.
One claim that is hopefully uncontroversial: Humans are not literally optimizing for IGF,
I think this is controverisial because it's basically wrong :)
First, its not actually obvious what "definition" of IGF you are using. If you talk about animals, the definition that might fit is "number of genes in the next generation". However if you talk ab...
I generally don't think LLMs today are conscious, as far as i can tell neither does Sam Altman, but there is some disagreement. They could acquire some characteristics that could be considered conscious as scale increases. However merely having "qualia" and being conscious is not the same thing as being functionally equivalent a new human, let alone a specific human. The term "upload" as commonly understood is a creation of a software construct functionally and qualia-equivalent to a specific human.
Thanks for the first part of the comment.
As mentioned in my above comment, the reason for mixing "can" and "should" problems is that they form a "stack" of sorts, where attempting to approximately solve the bottom problems makes the above problems harder and verification is important. How many people would care about the vision if one could never be certain the process succeeds?
Fixed the link formatting and added a couple more sources, thanks for the heads up. The temperature claim does not seem unusual to me in the slightest. I have personally tried to do a relatively cold bath and noticed my "perception" alter pretty significantly.
The organ claim does seem more unusual, but I have heard various forms of it from many sources at this point. It does not however seem in any way implausbile. Even if you maintain that the brain is the "sole" source of cognition, the brain is still an organ and is heavily affected by the operation of other organs.
There is a lot to unpack. I have definitely heard from leaders of the community claims to the tune of "biology is over," without further explanation of what exactly that means or what specific steps are expected to happen when the majority of people disagree with this. The lack of clarity here makes it hard to find a specific claim of "I will forcefully do stuff to people they don't like," but me simply saying "I and others want to actually have what we think of as "humans" keep on living" is met with some pushback.
You seem to be saying that the "I" or "Se...
The general vibe of the first two parts seems correct to me. Also, an additional point is that evolution's utility function of inclusive genetic fitness didn't completely disappear and is likely still a sub-portion of the human utility function. I suspect there is going to be disagreement on this, but it would also be interesting to do a poll on this question and break it down by people who do or do not have kids.
Yes I think we understand each other. One thing to keep in mind is that different stakeholders in AI are NOT utilitarians, they have local incentives they individually care about. Given the fact that COVID didn't stop gain-of-function research, this means that getting EVERYONE to care would require a death toll larger than COVID. However, getting someone like CEO of google to care would "only" require a half - a - trillion dollar lawsuit against Microsoft for some issue relating to their AIs.
And I generally expect those - types of warning shots to be pretty likely given how gun-ho the current approach is.
I am mostly agreeing with you here, so I am not sure you understood my original point. Yes Reality is giving us things that for a set of reasonable people such as you and me should be warning shots.
Since a lot of other people don't react to them, you might become pessimistic and extrapolate that NO warning shot is going to be good enough. However I posit that SOME warning shots are going to be good enough. An AI - driven bank run followed by an economic collapse is one example, but there could be others. Generally I expect that when warning shots reach "nation-level" socio-economic problems, people will pay attention.
However, this will happen before doom.
Yes, I agree. I have ideas how to fix it as well, but I seriously doubt they will gain much traction
I am also familiar with Paul Christiano, I think his arguments for slower, more continous take off are broadly on the right track as well.
Given that the extreme positions have strong stake-outs on twitter, I am once again claiming that there needs to be a strong stake-out of the more reasonable centrism. This isn't the first post in this direction, there were ones before and there will be ones after.
Just trying to keep this particular ball rolling.
I have skimmed the Alignment Forum side and read most of MIRI's work before 2015. While it's hard to know about the "majority of people," it does seem that the public reporting is around two polarized camps. However in this particular case, I don't think it's just the media. The public figures for both sides (EY and Yann Lecunn) seem pretty consistent with their messaging and talking past each other.
Also if the majority of people in the field agree with the above, that's great news and also means that reasonable centrism needs to be more prominently ...
Your comment is a little hard to understand. You seem to be saying that "scaling" is going to make it harder to align, which I agree with. I am not sure what "deliberate reasoning" means in this context. I also agree that having a new kind of training process is definitely required to keep GPT aligned either vis-a-vis OpenAI's rules or actually good rules.
I agree that the current model breaks down into "shoggoth" and "mask." I suspect future training, if it's any good would need to either train both simultaneously with similar levels of complexity fo...
While I agree that Twitter is a bad site, I expect some of Musk's actions to make it better (but not fully fix it). Your attempt to tie personality-based critiques (stem / white / male) isn't helpful. Addiction to social platforms is a general issue and needs to be solved in a general way.
However, the solutions you outline are in fact some of the ways that the situation will proceed. I don't think 1. [government] is likely or will sit well either.
However, 2 [fix] is plausible. Companies would not "admin" problems, but they could fix without "ad...
Yeah defining "harm", or more formally, a "non-beneficial modification to a human being" is a hard task and is in many ways the core problem I am pointing at. Allowing people to take part in defining what is "harmful" to themselves is both potentially helpful as it brings local information and tricky because people may have already been ensnared by a hostile narrow AI to misunderstand "harm."
Thanks for the comment! I look into some of the philosophy required in part 4, x-risk relationships in part 5 and todos in part 6. Understanding consciousness is an important sub-component and might be needed sooner than we think. I think an important piece is understanding what modifications to consciousness are harmful or beneficial. This would have a sub-problem of what chemicals or organisms alter it and in what ways as well as what ideas and experiences seem to have a lasting effect on people. It's possible this is more doable than understanding consciousness as a whole, but it certainly touches a lot of the core problem.
Mostly agree. A sophisticated user might have some great feedback on how a website ranks its products, but shouldn't and doesn't want to have access to the internals of the algorithm. So giving some users a slightly better surface area for interaction aside from "being part of the training data" seems like an important problem to solve. Could be useful all the way toward AGI as we would need a story of how a particular person still has some capacity for future influence.