iceman

LESSWRONG
LW

iceman — LessWrong

One can imagine ‘good’ versions of all this tech, but right now it doesn’t exist. It seems crazy that it doesn’t exist? Shouldn’t someone sell it? Properly curated experiences that gave parents proper control and steered children in good ways seem super doable, and would have a very large market with high willingness to pay.

I'm having problems imagining the 'good' version because I can't imagine the business model working: you are competing with free. People round "free with externalities" to free, and see that as preferable to "paid for in a way that aligns incentives". The existence of a "free" option causes people to anchor and anchor hard that the value of... (read more)

iceman11mo

I'll pass, but thanks.

Replying to Arguments for optimism on AI Alignment (I don't endorse this version, will reupload a new version soon.)

iceman2y

Arguments for optimism on AI Alignment (I don't endorse this version, will reupload a new version soon.)

But POC||GTFO is really important to constraining your expectations. We do not really worry about Rowhammer since the few POCs are hard, slow and impractical. We worry about Meltdown and other speculative execution attacks because Meltdown shipped with a POC that read passwords from a password manager in a different process, was exploitable from within Chrome's sandbox, and my understanding is that POCs like that were the only reason Intel was made to take it seriously.

Meanwhile, Rowhammer is maybe a real issue but is so hard to pull off consistently and stealthily that nobody worries about it. My recollection was when it was first discovered, people didn't panic that much because there wasn't warrant to panic. OK, so there was a problem with the DRAM. OK, what are the constraints on exploitation? Oh, the POCs are super tricky to pull off and will often make the machine hard to use during exploitation?

A POC provides warrant to believe in something.

Replying to Arguments for optimism on AI Alignment (I don't endorse this version, will reupload a new version soon.)

iceman2y

Arguments for optimism on AI Alignment (I don't endorse this version, will reupload a new version soon.)

On the topic of security mindset, the thing that the LW community calls "security mindset" isn't even an accurate rendition of what computer security people would call security mindset. As noted by lc, actual computer security mindset is POC || GTFO, or trying to translate that into lesswrongesse, you do not have warrant to believe in something until you have an example of the thing you're maybe worried about being a real problem because you are almost certain to be privileging the hypothesis.

Replying toAI romantic partners will harm society if they go unregulated

iceman3y

AI romantic partners will harm society if they go unregulated

Are AI partners really good for their users?

Compared to what alternative?

As other commenters have pointed out, the baseline is already horrific for men, who are suffering. Your comments in the replies seem to reject that these men are suffering. No, obviously they are.

But responding in depth would just be piling on and boring, so instead let's say something new:

I think it would be prudent to immediately prohibit AI romance startups to onboard new users[..]

You do not seem to understand the state of the game board: AI romance startups are dead, and we're already in the post-game.

character.ai was very popular around the second half of 2022, but near the end of it, the... (read 546 more words →)

Replying toA Hill of Validity in Defense of Meaning

iceman3y

A Hill of Validity in Defense of Meaning

So, I started off with the idea that Ziz's claims about MIRI were frankly crazy...because Ziz was pretty clearly crazy (see their entire theory of hemispheres, "collapse the timeline," etc.) so I marked most of their claims as delusions or manipulations and moved on, especially since their recounting of other events on the page where they talked about miricult (which is linked in OP) comes off as completely unhinged.

But Zack confirming this meeting happened and vaguely confirming its contents completely changes all the probabilities. I now need to go back and recalculate a ton of likelihoods here starting from "this node with Vassar saying this event happened."

From Ziz's page:

LessWrong dev Oliver Habryka

... (read more)

-2

Replying toA Hill of Validity in Defense of Meaning

iceman3y

A Hill of Validity in Defense of Meaning

The second half (just live off donations?) is also my interpretation of OP. The first half (workable alignment plan?) is my own intuition based on MIRI mostly not accomplishing anything of note over the last decade, and...

MIRI & company spent a decade working on decision theory which seems irrelevant if deep learning is the path (aside: and how would you face Omega if you were the sort of agent that pays out blackmail?). Yudkowsky offers to bet Demis Hassabis that Go won't be solved in the short term. They predict that AI will only come from GOFAI AIXI-likes with utility functions that will bootstrap recursively. They predict fast takeoff and FOOM.

Ooops.

The... (read more)

-28

Replying toA Hill of Validity in Defense of Meaning

iceman3y

A Hill of Validity in Defense of Meaning

It's not exactly the point of your story, but...

Probably the most ultimately consequential part of this meeting was Michael verbally confirming to Ziz that MIRI had settled with a disgruntled former employee, Louie Helm, who had put up a website slandering them.

Wait, that actually happened? Louie Helm really was behind MIRICult? The accusations weren't just...Ziz being Ziz? And presumably Louie got paid out since why would you pay for silence if the accusations weren't at least partially true...or if someone were to go digging, they'd find things even more damning?

Those who are savvy in high-corruption equilibria maintain the delusion that high corruption is common knowledge, to justify expropriating those who naively don't

... (read more)

-13

Replying toSome reasons to not say "Doomer"

iceman3y

Some reasons to not say "Doomer"

Just to check, has anyone actually done that?

I'm thinking of a specific recent episode where [i can't remember if it was AI Safety Memes or Connor Leahy's twitter account] posted a big meme about AI Risk Deniers and this really triggered Alexandros Marinos. (I tried to use Twitter search to find this again, but couldn't.)

It's quite commonly used by a bunch of people at Constellation, Open Philanthropy and some adjacent spaces in Berkeley.

Fascinating. I was unaware it was used IRL. From the Twitter user viewpoint, my sense is that it's mostly used by people who don't believe in the AI risk narrative as a pejorative.

Replying toSome reasons to not say "Doomer"

iceman3y

Some reasons to not say "Doomer"

Why are you posting this here? My model is that the people you want to convince aren't on LessWrong and that you should be trying to argue this on Twitter; you included screenshots from that site, after all.

(My model of the AI critics would be that they'd shrug and say "you started it by calling us AI Risk Deniers.")

Launched: Friendship is Optimal

iceman

13y

Friendship is Optimal has launched and is being published in chunks on FIMFiction.

Friendship is Optimal is a story about an optimizer written to "satisfy human values through friendship and ponies." I would like to thank everyone on LessWrong who came out and helped edit it. Friendship is Optimal wouldn't be what is today without your help.

Thank you.

Teaser description:

Hanna, the CEO of Hofvarpnir Studios, just won the contract to write the official My Little Pony MMO. Hanna has built an A.I. Princess Celestia and given her one basic drive: to satisfy everybody's values through friendship and ponies. And Princess Celestia will follow those instructions to the letter...even if you don't want her to.

Here is the schedule for the next chapters:

Friday (Nov. 16th): Chapter 4 - 5

Monday (Nov. 19th): Chapter 6 - 7

Thursday (Nov. 22th): Chapter 8 - 9

Sunday (Nov. 25th): Chapter 10 - 11, Author's Afterword

Friendship is Optimal: A My Little Pony fanfic about an optimization process

iceman

13y

[EDIT, Nov 14th: And it's posted. New discussion about release. Link to Friendship is Optimal.]

[EDIT, Nov 13th: I've submitted to FIMFiction, and will update with a link to its permanent home if it passes moderation. I have also removed the docs link and will make the document private once it goes live.]

Over the last year, I’ve spent a lot of my free time writing a semi-rationalist My Little Pony fanfic. Whenever I’ve mentioned this side project, I’ve received requests to alpha the story.

I present, as an open beta: Friendship is Optimal. Please do not spread that link outside of LessWrong; Google Docs is not its permanent home. I intend to put it up on fanfiction.net... (read more)

152

117

LESSWRONG
LW

LESSWRONG
LW

iceman

iceman

Politics Is Upstream of AI

Launched: Friendship is Optimal

Friendship is Optimal: A My Little Pony fanfic about an optimization process

iceman

iceman

iceman

Politics Is Upstream of AI

Launched: Friendship is Optimal

Friendship is Optimal: A My Little Pony fanfic about an optimization process