All of xarkn's Comments + Replies

xarkn43

We can try to select for AIs that outwardly seem friendly, but on anything close to our current ignorance about their cognition, we cannot be nearly confident that an AI going through the intelligence explosion will be aligned to human values.

This bolded part is a bit difficult to understand. Or at least I can't understand what exactly is meant by it.

It would then go about optimizing the lightcone according to its values

"lightcone" is an obscure term, and even within Less Wrong I don't see why the word is clearer than using "the future" or "the universe". I would not use the term with a lay audience.



 

1Towards_Keeperhood
Thank you for your feedback! Feedback is great. It means that we have only very little understanding of how and why AIs like ChatGPT work. We know almost nothing about what's going on inside them that they are able to give useful responses. Basically all I'm saying here is that we know so little that it's hard to be confident of any nontrivial claim about future AI systems, including that they are aligned. A more detailed argument for worry would be: We are restricted to training AIs through giving feedback on their behavior, and cannot give feedback on their thoughts directly. For almost any goal an AI might have, it is in the interest of the AI to do what the programmers want it to do, until it is robustly able to escape and without being eventually shut down (because if it does things people don't like while it is not yet powerful enough, people will effectively replace it with another AI which will then likely have different goals, and thus this ranks worse according to the AI's current goals). Thus, we basically cannot behaviorally distinguish friendly AIs from unfriendly AIs, and thus training for friendly behavior won't select for friendly AIs. (Except in the early phases where the AIs are still so dumb that they cannot realize very simple instrumental strategies, but just because a dumb AI starts out with some friendly tendencies, doesn't mean this friendliness will generalize to the grown-up superintelligence pursuing human values. E.g. there might be some other inner optimizers with other values cropping up during later training.) (An even more detailed introduction would try to concisely explain why AIs that can achieve very difficult novel tasks will be optimizers, aka trying to achieve some goal. But empirically it seems like this part is actually somewhat hard to explain, and I'm not going to write this now.) Yep, true.
8khafra
It's very standard advice to notice when a sense of urgency is being created by a counterparty in some transaction; and to reduce your trust in that counterparty as well as pausing. It feels like a valuable observation, to me, that the counterparty could be internal--some unendorsed part of your own values, perhaps.
xarkn1214

You are not directly vouching for anyone here, but as a general point I'd like to argue that friendship is a poor predictor of ethical behavior. 

It may be tempting to consider positive social experiences and friendship as evidence that someone behaves generally ethically and with high standards, but when dealing with more capable people, it's not. Maintaining ethical behavior and building trust in low-stakes settings like friendship with few temptations to try and exploit for profit is trivially easy. Especially if you are socially skilled and capable... (read more)

Viliam2419

Following this tangent, I would say that judging other people is a skill. Some people are better at it, some are worse, and the Dunning–Kruger effect very likely applies. Learning this skill is both explicit (what to notice) and implicit (you get burned -- you learn what to fear).

Examples of explicit lessons:

  • Notice how the person treats people other than you -- very likely, they will treat you the same in the future, when they no longer need to impress you. Similarly, if the person tells you to treat other people badly, in the future they will probably do
... (read more)
xarkn136104

Instead of creating new ingroups and outgroups and tribal signifiers for enforcing such, we should focus on careful truth-seeking. Some mythologies and terms that engage our more primal instincts can be useful, like when Scott introduced "Moloch", but others are much more likely harmful. "Orthodox vs Reform" seems like a purely harmful one, that is only useful for enforcing further division and tribal warfare.

To summarize, in this post Aaronson,

  1. Enforces the idea that AI safety is religion.
  2. Creates new terminology to try and split the field into two ideologi
... (read more)
AnonAcc*2213

This is wonderfully written, thank you.

I do worry that "further division and tribal warfare" seems the default, unless there's an active effort at reconciliation.

Sam Altman (CEO of OpenAI) tweets things like:

 

LessWrong compares OpenAI with Phillip Morris and, in general, seems very critical of OpenAI. "I've seen a lot more public criticism lately".

I doubt that it's actually good to have this strong division, and it might have positive EV to try to move into a more cooperative direction, and try to lower the temperature and divisiveness.

Answer by xarkn50

Historically it seems likely that there will be several weeks or even months of warning where a reasonably prepared person can react. What sort of crisis would immediately or surprisingly close all borders? Coronavirus closed many borders, but we had many weeks to react before that. A significant challenge might not be leaving, but being accepted to another country.

Some practical and mostly cost-effective ideas that should increase the speed and odds of escape:

  • Connections: Having friends abroad. Having friends you can discuss and analyze the risks with.
  • E
... (read more)
xarkn50
inoculating via the GI tract, which may lead to weaker symptoms than the same load in the respiratory system.

A critical care doctor speculated in the This Week in Virology -podcast that getting the virus gastrointestinally might result in worse outcomes. They had observed that in hospitalized patients, those with GI symptoms tended to have worse outcomes, and one theory for why was that the GI system has the widest surface area for the virus to multiply in before spreading to the rest of the body. I don't have the expertise to judge how plausible this is.

xarkn10

Regarding point 2, how sure are you? Why are we even trying to disinfect N95 masks if that's true? I think your point is plausible but the filter technology in these masks isn't entirely trivial. Most filter materials actually depend on a static electric charge in the polypropylene to filter properly. Does the charge actually release the active virus particles after some time, and then you breathe them in? I have no idea. I was already surprised to find out that masks simply aren't just a dense material that filters particles, but a bit more complicated.

1EGI
Pretty sure. You should not get your filter wet though since this may allow diffusion across the filter, which is why it is unsafe to wear (N 95 or other) fleece masks for extended periods. Also stuff that is bound in the filter is also attracted via Van-der-Waals forces which are really strong on this scale.
xarkn20

I bought a half-mask and several filters almost two months ago, and it's definitely easier and safer for my once-per-week shopping trip than a single use mask. I don't think that it's a particularly effective general solution though, for the following reasons:

1. All of these are sold out. To make more, you need to manufacture both masks and filters.

2. All the replaceable filters are sold out. The filters need to be sequentially rotated or otherwise disinfected.

3. Wearing one for anything more than 30 minutes is still quite awful.

4. Most of them have exhaust valves which still spread the virus to others.

1EGI
1. This was the point of my original post. States should begin stimulating large scale production of masks and filters to provide most people with such masks. 2. No they don't. Virus particles on the filter stay there as long as the filter does not get wet and decay quite quickly 3. Your mask either does not fit right or is low quality or has some kind of gas combination filter with very high flow resistance. 4. Yes, see discussion above.
1Yandong Zhang
I bought 3M respirators and filters several months ago. I did not use it since I worked from home and did not go shopping. Those devices are more cheaper than N95 mask. https://www.3m.com/3M/en_US/company-us/all-3m-products/~/All-3M-Products/Safety/Personal-Safety/Personal-Protective-Equipment/Reusable-Respirators/?N=5002385+8709322+8711017+8711405+8720539+8720550+3294857497&rt=r3
1Mike Bishop
You have experience wearing a mask like above and are telling us it's awful to wear more than 30 minutes?