Replying toHypothetical: what would you do?

My hypothetical self thanks you for your input and has punted the issues to the real me.

I feel like I need to dig a little bit into this

If you actually understand why it's not useful

Honestly I don't know for sure I do, how can I when everything is so ill-defined and we have so few scraps of solid fact to base things on.

That said, there is a couple of issue, and the major one is grounding, or rather the lack of grounding.

Grounding is IMO a core problem, although people rarely talk about, I think that mainly comes about because we (humans) seemingly have solved it.

I don't think that's the case at all,... (read more)

Hypothetical: what would you do?

JNS

Let’s pretend I have a semi rigors model that lays out why RLHF is doomed to fail and also that it negatively affects model performance (including why it does so)

Let’s go further into lala land and pretend that I have an architectural plan that does much better, very transparent, steerable and corrigible, can be deployed and used without changing or retraining the base LLM.

There are some downsides like requires more compute at inference time, not provable bulletproof, likely breaks in SI regime and definitely breaks under self improvement (so very definitely NOT an alignment proposal).

Short term this looks beneficial, also looks like shortening timelines, and extremely unlikely to advance the AI safety... (read more)

Replying toWhat works for ADHD and/or related things?

JNS3y

What works for ADHD and/or related things?

Yes could be ADHD, but I am not at a professional.

As for your therapist, that is not conclusive and by no means a sign the person does not have ADHD.

10 years before my diagnosis, my doctor had a feeling I might have ADHD, so he presented my file in conference and EVERYONE there reasoned as your therapist, so nothing further happened for 10 years.

Intelligence can and often does a lot of work that compensate for executive deficiency’s in people with ADHD.

Anyway, do the assessment by the book, be objective and hold off on knee jerk calls based on singular things like has a job, has an education.

I have what almost looks like a career with massive responsibility, academic education, married, kids, never in trouble with the law, no abuse of drugs or alcohol - traditional thinking says I cannot possible have ADHD, and yet the by the book assessment was crystal clear.

Replying toWhat works for ADHD and/or related things?

JNSAug 02, 2023

What works for ADHD and/or related things?

Problems with attention can come from many places, and from your post I can see you know that.

As for ADHD, the attention thing is not even close to the main thing, but it is unfortunately a main external observable (like hyperactivity), and so that why its so grossly misnamed.

Having ADHD means lots of things, like:

- You either get nothing done all day, manage 3 times 5 minutes of productivity, or do 40 hours of work in 5 wall clock hours.

- Doing on thing, you notice another "problem", start working on that and then repeat, do that for 8 hours and all of a sudden you have started 20 things, and no progress... (read 378 more words →)

Replying toWhat The Lord of the Rings Teaches Us About AI Alignment

JNS3y

What The Lord of the Rings Teaches Us About AI Alignment

Prison guards don’t seem to voluntarily let people go very often, even when the prisoners are more intelligent than them.

That is true, however I don't think it serves as a good analogy for intuitions about AI boxing.

The "size" of you stick and carrot matters, and most humans prisoners have puny sticks and carrots.

Prison guard also run a enormous risk, in fact straight up just letting someone go is bound to fall back on them 99%+ of the time, which implies a big carrot or stick is the motivator. Even considering that they can hide their involvement, they still run a risk with a massive cost associated.

And from the prisoners point of view... (read more)

JNS3y

How does pausing change much of anything?

Lets say we manage to implement a world wide ban/pause on large training runs, what happens next?

Well obviously smaller training runs, up to whatever limit has been imposed, or no training runs for some time.^[1]

The next obvious thing that happens, and btw is already happening in the open source community, would be optimizing algorithms. You have a limit on compute? Well then you OBVIOUSLY will try and make the most of the compute you have.

Non of that fixes anything.

What we should do:^[2]

Pour tons of money into research, first order of business is to make the formal case for x-risk is a thing and must actively be... (read more)

JNS3y

Completely off the cuff take:

I don't think claim 1 is wrong, but it does clash with claim 2.

That means any system that has to be corrigible cannot be a system that maximizes a simple utility function (1 dimension), or put another way "whatever utility function is maximizes must be along multiple dimensions".

Which seems to be pretty much what humans do, we have really complex utility functions, and everything seems to be ever changing and we have some control over it ourselves (and sometimes that goes wrong and people end up maxing out a singular dimension at the cost of everything else).

Note to self: Think more about this and if possible write up something more coherent and explanatory.

JNS3y

Reasonably we need both, but most of all we need some way to figure out what happened in the situation where we have conflicting experiments, so as to be able to say "these results are invalid because XXX".

Probably more of an adversarial process, where experiments and their results must be replicated*. Which means experiments must be documented way more detailed, and also data has to be much more clear and especially the steps that happen in clean up etc.

Personally I think science is in crisis, people are incentivized to write lots of papers, publish results fast, and there is zero incentive to show a paper is false / bad, or replicate an experiment.

*If possible, redoing some experiment is going to be very hard, especially if we would like the experiments to have as little in common as possible (building another collider to does what LHC does is not happening any time soon).

Replying toMetaAI: less is less for alignment.

JNS3y

MetaAI: less is less for alignment.

Thanks for the write-up, that was very useful for my own calibration.

Fair warning: Colorful language ahead.

Why is it whenever Meta AI does a presentation, YC posts something, they release a paper, I go:

Jeez guys, that's it? With all the money you have, and all the smart guys (including YC), this is really it?

What is going on? You come off as a bunch of people who have your heads so far up your own ass, sniffing rose smelling (supposedly but not really) farts, to realize that you come across as amateur's with way too much money.

Its sloppy, half baked, not even remotely thought through, and the only reason you build and release anything... (read more)

-1

JNS3yQuick Take

I got my entire foundation torn down, and with it came everything else.

It all came crashing down in one giant heap of rubble.

I’ll just rebuild, I thought - not realizing you can’t build without a foundation plan.

So all I’ve ended up doing was shift through the rubble, searching for things that feel right.

Now I am back, in a very literal sense, to where I all began, so much was built, so many things destroyed and corrupted, and a major piece ended and got buried.

And all I got is “what the eff am I doing here?”

The obvious answer is “yelling at the sky demanding answers” and being utterly ignored.

I guess as per usual it is all up to me, except I don’t know how to rebuild myself……again.

F…..

Replying toMark for follow up?

JNS3y*

Mark for follow up?

Sure, I often browse LW casually and whenever I come across an interesting post, or a comment or whatever, and I go "hmm right I might have sometime to contribute / say here, let me get back to it when I have time to think about it and write something maybe relevant"

My specific problem, is that I am a massive scatterbrain, so I hardly ever do come back to it, and even if I do it usually eludes me what the momentary insight I wanted to get into was.

On top of that I do this from a lot of different devices, and whatever I am looking for to help me quickly go... (read more)

Mark for follow up?

JNS

Is there way to mark a post, shortform and comment, such that I can get back to it when I have time and write a comment/reply?

I know I can bookmark post, but what I really need is a mark for follow up with the option for leaving myself a note as to why.

I am giant scatterbrain, and during the day I spend downtime browsing LW, and go spelunking - and usually I stumble into things that make me go “hmmm I need to get back to this”, and then I promptly forget all about it.

Edit: I am pasting in a reply I made to a question about clarification, I am pasting it... (read 429 more words →)

Alone, wandering the endless hallways of this massive temple of healing.

Feels empty and eerily quiet, and yet I know there are 100’s of people around, most sleeping, some watching, a few dying, and close by someone being born.

Yesterday feels like ages ago, orbiting Saturn on morphine, billions of miles away from the excruciating pain that brought me here.

The daze is gone, and so is the morphine induced migraine, I feel fine, great even, and guilty.

But home I may not go, so I wander these deserted hallways, pondering the future, will it be there for my kids?

AI x-risk is convergent.

Believing otherwise is like hurling yourself at the ground, convinced you'll miss and start flying.

I recently came across a post on LinkedIn, and I have to admit the brilliance of the arguments, the coherent and frankly bulletproof ontology displayed, I was blown away and immediately had to do a major update to p(doom).

I think that the magnitude of the AI alignment problem has been ridiculously overblown & our ability to solve it widely underestimated.
I've been publicly called stupid before, but never as often as by the "AI is a significant existential risk" crowd.
That's OK, I'm used to it.
-Yann LeCun, March 20 2023

An idealized Oracle is equivalent to a universal Turing machine (UTM).

A self-improving Oracle approaches UTM-like behavior in the limit.

What about a (self-improving) token predictor under iteration? It appears Oracle-like, but does it tend toward UTM behavior in the limit, or is it something distinct?

Maybe, just maybe, the model does something that leads it to not be UTM like in the limit, and maybe (very much maybe) that would allow us to imbue it with some desirable properties.

/end shower thought

Are we heading towards an new financial crisis?

Mark to market changes since 2009, combined with the recent significant interest hikes, seems to make bank balance sheets "unreliable".

Mark to market changes broadly means that banks can have certain assets on their balance sheet, and the value of the asset is set via mark to model (usually meaning its marked down as worth face value).

Banks traditionally have a ton of bonds on their balance sheet, and a lot of those are governed by mark to model and not mark to market.

Interest rates go up a lot, which leads to bonds dropping in value by a lot (20% or atm more depending on duration).

However due... (read more)

JNS's Shortform

JNS

This is a special post for quick takes (aka "shortform"). Only the owner can create top-level comments.

The idea

JNS

Preamble

Preamble is longish, but also de facto mandatory, if you want to keep reading that is.

I am looking for feedback on the story "The idea".

The purpose of the story is to act as a hook and get, some, people to continue on to the epilogue.

The epilogue is not include here, but will, depending on the feedback, be publish in a similar manner later on. Most of it has been written at this point, and what is left to do is restructuring and rewrites.

The story itself is set in an alternate timeline, where we did not solve any fundamental problems, but the frontrunners did manage to channel an epic amount of security mindset,... (read 2678 more words →)

LESSWRONG
LW

LESSWRONG
LW

JNS

Mark for follow up?

Hypothetical: what would you do?

The idea

JNS's Shortform

JNS

JNS

Hypothetical: what would you do?

Mark for follow up?

JNS's Shortform

The idea

JNS

Mark for follow up?

Hypothetical: what would you do?

The idea

JNS's Shortform

JNS

JNS

Hypothetical: what would you do?

Mark for follow up?

JNS's Shortform

The idea

Preamble