Dr_Manhattan

Replying toAn alternative way to browse LessWrong 2.0

An alternative way to browse LessWrong 2.0

Any updates on the API? (thinking of) Playing around with interesting ways to index LW, figure there should be something better than scraping

Replying toAGI will drastically increase economies of scale

Dr_Manhattan2y

AGI will drastically increase economies of scale

So how does one invest in China, as a country?

Replying toDouglas Hofstadter changes his mind on Deep Learning & AI risk (June 2023)?

Dr_Manhattan3y

Douglas Hofstadter changes his mind on Deep Learning & AI risk (June 2023)?

He was only a de facto mysterian: thought mind is so complicated that it may as well be mysterious (but ofc he believed it's ultimately just physics). This position is updateable, and he clearly updated.

Replying toBing Chat is blatantly, aggressively misaligned

Dr_Manhattan3y*

Bing Chat is blatantly, aggressively misaligned

A net saying "I'm thinking about ways to kill you" does not necessarily imply anything whatsoever about the net actually planning to kill you

Since these nets are optimized for consistency (as it makes textual output more likely), wouldn't outputting text that is consistent with this "thought" be likely? E.g. convincing the user to kill themselves, maybe giving them a reason (by searching the web)?

Replying toHow it feels to have your mind hacked by an AI

Dr_Manhattan3y

How it feels to have your mind hacked by an AI

I've been wishing for someone to write AI-singularity parallel of Bardbury's Martian Chronicles (which are pretty much independent sample/ simulations of how living on Mars could go)

Replying toReplacing Karma with Good Heart Tokens (Worth $1!)

Dr_Manhattan4y

Replacing Karma with Good Heart Tokens (Worth $1!)

Replying toMental nonsense: my anti-insomnia trick

Dr_Manhattan4y

Mental nonsense: my anti-insomnia trick

Sharing a personal weird trick why not. I like falling asleep to light TV (via iPad). I watch short shows that a) I like and don't think are boring b) I have seen before. Usually 10 minutes into a 20 min show I'm ready (Futurama is my favorite for this + my meme game is much improved by this)

Replying toI left Russia on March 8

Dr_Manhattan4y

I left Russia on March 8

Was thinking about you! Glad you made it out. Feel free to DM if I can be of assistance

Replying to(briefly) RaDVaC and SMTM, two things we should be doing

Dr_Manhattan4y

(briefly) RaDVaC and SMTM, two things we should be doing

MIRI is bottlenecked more on ideas worth pursuing and people who can pursue them, than on funding

Ideas come from (new) people, and you mentioned seed planting which should contribute to having such people in 4-6 years, seems like still a worthy thing to do for AGI if anything is worth doing for any cause at all (given your short timelines). If you agree what's the bottleneck for that effort?

Replying toVisible Thoughts Project and Bounty Announcement

Dr_Manhattan4y

Visible Thoughts Project and Bounty Announcement

Related work:
Show Your Work: Scratchpads for Intermediate Computation with Language Models
https://arxiv.org/abs/2112.00114

(from very surface-level perusal) Prompting the model resulted in
1) Model outputting intermediate thinking "steps"

2) Capability gain

What are we predicting for Neuralink event?

Dr_Manhattan

Dr_Manhattan, Vaniver

Interesting exercise in AI-adjacent forecasting area (brain-computer interfaces). Curious if people want to specify some possible reveals+probabilities. https://twitter.com/neuralink/status/1149133717048188929

(if in the somewhat likely scenario you're relying on inside info please mention it)

LW Dev question: FB-style tagging?

Dr_Manhattan

As LW is an intellectual **community** where people get to know each other and know "who knows what, who's working on what" it seems like something like tagging people FB-style in specific posts/comments is a very low-hanging fruit. (Of course you should be able to silence people from tagging you or sort taggers by some score). Is this in the pipeline, or has been ruled out for some reason?

Using rationality to debug Machine Learning

Dr_Manhattan

Interesting case study making rounds in ML social media, supporting the thesis that rationality techniques are useful in doing actual ML research. Several implicit references and explicit reference to CFAR at the bottom

Bayesian statistics as epistemic advantage

Dr_Manhattan

Interesting Talking Machines episode quote about Bayesian stats being used at Bletchley and GCHQ (its successor). Seems like they held on to a possibly significant advantage (crypto ppl would be better to comment on this) for years, owing largely to Turing. (The rest of the episode is about AI safety and also interesting.)

Source:

http://www.thetalkingmachines.com/blog/2016/2/26/ai-safety-and-the-legacy-of-bletchley-park

GCHQ in the ’70s, we thought of ourselves as completely Bayesian statisticians. All our data analysis was completely Bayesian, and that was a direct inheritance from Alan Turing. I’m not sure this has ever really been published, but Turing, almost as a sideline during his cryptoanalytic work, reinvented Bayesian statistics for himself. The work against Enigma and other German ciphers... (read more)

MILA gets a grant for AI safety research

Dr_Manhattan

http://www.openphilanthropy.org/focus/global-catastrophic-risks/potential-risks-advanced-artificial-intelligence/montreal-institute-learning-algorithms-ai-safety-research

The really good news is that Yoshua Bengio is leading this (he is extremely credible in modern AI/deep learning world), and this is a pretty large change of mind for him. When I spoke to him at a conference 3 years ago he was pretty dismissive of the whole issue; this year's FLI conference seems to have changed his mind (kudos to them)

Of course huge props to OpenPhil for pursuing this

NIPS 2015

Dr_Manhattan

10y

There is a fairly large contingent of safety research oriented people at NIPS this year. I'm unfortunately not among them, but if you're there and interested in connecting with others on AI safety topics or other LW issues - general rationality, EA, etc. I welcome you to make this thread a Schelling point to create meeting opportunities :). You can also PM me I can connect you to people I know are there.

Velocity of behavioral evolution

Dr_Manhattan

11y

This suggested a major update on the velocity of behavioral trait evolution.

Basically mice transmitted fear of cherry smell reliably into the very next generation (via epigenetics).

www.newscientist.com/article/dn24677-fear-of-a-smell-can-be-passed-down-several-generations.html#.VJRgr8ADo

This seems pretty important.

What Peter Thiel thinks about AI risk

Dr_Manhattan

11y

This is probably the clearest statement from him on the issue:

http://betaboston.com/news/2014/12/10/audio-peter-thiel-visits-boston-university-to-talk-entrepreneurship-and-backing-zuck/

25:30 mins in

TL;DR: he thinks its an issue but also feels AGI is very distant and hence less worried about it than Musk.

I recommend the rest of the lecture as well, it's a good summary of "Zero to One" and a good QA afterwards.

Cognitive distortions of founders

Dr_Manhattan

11y

Interesting take on entrepreneurial success:

http://www.harrisonmetal.com/cognitive-distortions-of-founders/

LESSWRONG
LW

LESSWRONG
LW

NES-game playing AI [video link and AI-boxing-related comment]

Gunshot victims to be suspended between life and death [link]

What are we predicting for Neuralink event?

Hawking/Russell/Tegmark/Wilczek on dangers of Superintelligent Machines [link]

Dr_Manhattan

Dr_Manhattan

What are we predicting for Neuralink event?

LW Dev question: FB-style tagging?

Using rationality to debug Machine Learning

Bayesian statistics as epistemic advantage

MILA gets a grant for AI safety research

Learning from Human Preferences - from OpenAI (including Christiano, Amodei & Legg)

NIPS 2015

Dr_Manhattan

NES-game playing AI [video link and AI-boxing-related comment]

Gunshot victims to be suspended between life and death [link]

What are we predicting for Neuralink event?

Hawking/Russell/Tegmark/Wilczek on dangers of Superintelligent Machines [link]

Dr_Manhattan

Dr_Manhattan

What are we predicting for Neuralink event?

LW Dev question: FB-style tagging?

Using rationality to debug Machine Learning

Bayesian statistics as epistemic advantage

MILA gets a grant for AI safety research

Learning from Human Preferences - from OpenAI (including Christiano, Amodei & Legg)

NIPS 2015