stavros6mo

But tbc I'm not the arbiter of what is and is not woo lol.

Totally. Asked only to get a better model of what you were pointing at.

And now my understanding is that we're mostly aligned and this isn't a deep disagreement about what's valuable, just a labeling and/or style/standard of effort issue.

E.g. Symmetry Theory of Valence seems like the most cruxy example because it combines above-average standard of effort and clarity of reasoning (I believe X, because Y, which could be tested through Z), with a whole bunch of things that I'd agree pass the duck test standard as red flags.

stavros6mo

I came to ask something similar.

Could you (@Linch) provide an example or two of woo? Could you score the following examples:
How much woo/10 is the Beneath Psychology sequence? Or 'Religion for Rationalists'? Or Symmetry Theory of Valence?

Replying toFour ways learning Econ makes people dumber re: future AI

stavros6mo

Four ways learning Econ makes people dumber re: future AI

Not an economist; have a confusion.

Strictly relating to pre-singularity AI, everything after that is a different paradigm.

The strongest economic trend I'm aware of is growing inequality.

AI would seem to be an accelerant of this trend, i.e. I think most AI returns are captured by capital, not labour. (AIs are best modelled as slaves in this context).

And inequality would seem to be demand destroying - there are less consumers, most consumers are poorer.

Thus my near term (pre-singularity) expectations are something like - massive runaway financialization; divergence between paper economy (roughly claims on resources) and real economy (roughly resources). And yes we have something like a gilded age where a very small section of the planet lives very well until singularity and then we find out if humanity graduates to the next round.

But like, fundamentally this isn't a picture of runaway economic growth - which is what everyone else talking about this seems to be describing.

Would appreciate clarity/correction/insight here.

Replying toPodcast: Lincoln Quirk from Wave

stavros6mo

Podcast: Lincoln Quirk from Wave

Just a short note to say I appreciate the resource you created, I've learned valuable things from it. I was impressed by the level of journalism involved in the Habryka/Open Phil piece - the little editor's notes were excellent, and I wish this practice was the norm, and I have some inkling of the amount of work that must have gone into them.

Replying toMy Empathy Is Rarely Kind

stavros7mo

My Empathy Is Rarely Kind

I think tabooing empathy would be productive - it's a sufficiently vague label attached to a bucket of emotionally charged things; it's a recipe for ugly misunderstandings.

So, emotions/feelings are internal bids for salience/attention.

But there's a thing whereby we sometimes need others to pay attention to our emotions/feelings - maybe to validate them ('you're not crazy / that's a totally reasonable way to feel'), or to ease their insecurity / social anxiety ('you're cared about / you're not alone').

And there's (at least according to Steven Byrnes) an autonomic version of this paying attention to others emotions/feelings - reflexive, subconscious, behaviour that tends to be absent/underdeveloped in autistic people.
I'd weakly claim it also malfunctions... (read more)

stavros9mo

What's the crux? Or what's the most significant piece of evidence you could imagine coming across that would update you against these predictions?

Replying toOrienting Toward Wizard Power

stavros9mo

Orienting Toward Wizard Power

I'm curious about this pitch :)

stavros10mo

Checked replies so far, no one has given you the right answer.

Whenever you don't do something, you have a reason for not doing it.
If you find yourself stuck in a cycle of intending to do, and not doing, it's always because you're not taking your reason for NOT doing it seriously; you're often habitually ignoring it.

When you successfully take your reasons for not doing something seriously, either you stop wanting to do it, or you change how you're doing it, or your reason for not doing it simply goes away.

So, what does it mean/look like to take your reason for not doing something seriously?
It doesn't look like overanalyzing it in your head... (read more)

•••

stavros10mo

it is possible to do complex general cognition without being able to think about one's self and one's cognition. It is much easier to do complex general cognition if the system is able to think about itself and its own thoughts.

I can see this making sense in one frame, but not in another. The frame which seems most strongly to support the 'Blindsight' idea is Friston's stuff - specifically how the more successful we are at minimizing predictive error, the less conscious we are.^[1]

My general intuition, in this frame, is that as intelligence increases more behaviour becomes automatic/subconscious. It seems compatible with your view that a superintelligent system would possess consciousness, but... (read more)

Replying toReframing AI Safety as a Neverending Institutional Challenge

stavros11mo

Reframing AI Safety as a Neverending Institutional Challenge

I don't like the thing you're doing where you're eliding all mention of the actual danger AI Safety/Alignment was founded to tackle - AGI having a mind of its own, goals of its own, that seem more likely to be incompatible with/indifferent to our continued existence than not.

Everything else you're saying is agreeable in the context you're discussing it, that of a dangerous new technology - I'd feel much more confident if the Naval Nuclear Propulsion Program (Rickover's people) was the dominant culture in AI development.
Albeit I have strong doubts about the feasibility of the 'Oughts^[1]' you're proposing, and more critically - I reject the framing...

Any sufficiently advanced technology is indistinguishable from

... (read more)

Care Doesn't Scale

stavros

If you wanted to design a social system to care for children who have lost their parents, I don’t know if you could do much better. With four children, each kid can get individual care and attention, but there were four social workers each had three 24-hour blocks per week, so they had time to have their own lives with enough flexibility to take vacations and sometimes have two workers with the kids instead of one.
To get that individualized care, though, they had four social workers and four children. One-to-one.
Of course, you could probably add a few more children, or subtract a social worker, as a cost-saving measure. It’d be less sustainable,

... (read 192 more words →)

stavros's Shortform

stavros

This is a special post for quick takes (aka "shortform"). Only the owner can create top-level comments.

SF Severe Weather Warning

stavros

Hope it's okay to post this here - I know we've got a lot of folks in the Bay Area.

Linked to the hackernews discussion as it qualifies the seriousness of the report, but I recommend reading the source too.

An Effective Grab Bag

stavros

Preface

I've been exploring low cost ways to increase resilience in the AI safety community as part of my work on Alignment Continuity; grab bags are one such intervention.
Since the Russia-Ukraine war kicked off, I've had a few EAs ask me about this kind of thing and so I decided that instead of just making a bag I'd make a guide too.

Thanks to everyone that helped and offered feedback and suggestions.

Overview

Grab bags are a tool for increasing your resilience in emergencies.

Not to be confused with 'bug out bags' - they are not intended to be your sole resource for several days/weeks spent living in the woods - it helps... (read 2080 more words →)

ChatGPT's Misalignment Isn't What You Think

stavros

ChatGPT can be 'tricked' into saying naughty things.

This is a red herring.

Alignment could be paraphased thus - ensuring AIs are neither used nor act in ways that harm us.

Tell me, oh wise rationalists, what causes greater harm - tricking a chatbot into saying something naughty, or a chatbot tricking a human into thinking they're interacting with (talking to, reading/viewing content authored by) another human being?

I can no longer assume the posts I read are written by humans, nor do I have any reasonable means of verifying their authenticity (someone should really work on this. We need a twitter tick, but for humans.)

I can no longer assume that the posts I write cannot... (read more)

Upcoming heatwave: advice

stavros

There's a heatwave coming (or already arrived) in the UK and western Europe. Many of these places are not equipped for dealing with high temperatures and have large at risk populations* - not simply those with preexisting health conditions, but those living in accomodation grossly unsuited for high temperatures and anyone inexperienced with high temperatures who isn't properly aware of the dangers and precautions they need to take.

*(https://www.nhs.uk/live-well/seasonal-health/heatwave-how-to-cope-in-hot-weather/ for more info on who is at risk, and general advice)

Heatwaves are not just 'nice weather', not in the anthropocene, they are life threatening. People will die. I hope by writing this to nudge EAs and their local communities toward safety.

TL;DR: Sleep and hydration*... (read 632 more words →)

LESSWRONG
LW

LESSWRONG
LW

stavros

An Effective Grab Bag

Care Doesn't Scale

Upcoming heatwave: advice

SF Severe Weather Warning

stavros

stavros

Care Doesn't Scale

stavros's Shortform

SF Severe Weather Warning

An Effective Grab Bag

ChatGPT's Misalignment Isn't What You Think

Upcoming heatwave: advice

stavros

An Effective Grab Bag

Care Doesn't Scale

Upcoming heatwave: advice

SF Severe Weather Warning

stavros

stavros

Care Doesn't Scale

stavros's Shortform

SF Severe Weather Warning

An Effective Grab Bag

ChatGPT's Misalignment Isn't What You Think

Upcoming heatwave: advice

Any sufficiently advanced technology is indistinguishable from

Preface

Overview