Part I. Superintelligence and Security: Contingencies in the Securitisation of AGI
Part II. Power and Polarity: The Limits of US Hegemony
Part III. Lessons and Learnings: Takeaways for the Governance of Emerging Technologies
...Thank you!
I don't think I understand. Let's work through an example.
The AI is being told to write good code, and wants to write subtly-vulnerable code instead. If it just were to honestly try to write good code, it would take (let's say) 100 serial steps of reasoning to succeed. To write subtly-vulnerable code, let's say, requires 300 serial steps of reasoning. The model's strategy is to write subtly-vulnerable code but in a way that looks like it's trying to write good code and just getting a bit unlucky and taking longer than usual (let's suppose the humans ...
They made a protest; the rationalists called cops on them.
I don't think that claim is true. As far as I remember the owner of the venue that was rented in by the rationalist in question called the cops.
Was there any other possible mistake that I missed?
Coliving communities that are focused on doing strong experimentation in changing human cognition are risky. That goes for both Ziz's Rationalist Fleet and for Leverage.
There are questions about how to deal with the topic of trans-identity. You could say that especially ten years ago, the d...
As the creator of the linked market, I agree it's definitional. I think it's still interesting to speculate/predict what definition will eventually be considered most natural.
Epistemic status: very speculative
Content warning: if true this is pretty depressing
This came to me when thinking about Eliezer's note on Twitter that he didn't think superintelligence could do FTL, partially because of Fermi Paradox issues. I think Eliezer made a mistake, there; superintelligent AI with (light-cone-breaking, as opposed to within-light-cone-of-creation) FTL, if you game it out the whole way, actually mostly solves the Fermi Paradox.
I am, of course, aware that UFAI cannot be the Great Filter in a normal sense; the UFAI itself is a potentially-expanding technological civilisation.
But. If a UFAI is expanding at FTL, then it conquers and optimises the entire universe within a potentially-rather-short timeframe (even potentially a negative timeframe at long distances, if the only cosmic-censorship limit is closing a loop). That means the...
You're encouraged to write a self-review, exploring how you think about the post today. Do you still endorse it? Have you learned anything new that adds more depth? How might you improve the post? What further work do you think should be done exploring the ideas here?
Still endorse. Learning about SIA/SSA from the comments was interesting. Timeless but not directly useful, testable or actionable.
Note: Probably reinventing the wheel here. Heavily skewed to my areas of interest. Your results may vary.
If you want [abstractapplic]'s feedback on anything, let me know.
I have received creative feedback from abstractapplic. It was useful and made me happy. This is an endorsement.
I'm a new undergraduate student essentially taking a gap year(it's complicated). I'm looking for someone that would be interested in studying various fields of science and mathematics with me. I've taken through Calculus 2, know how to program in Python, and know a smattering about mechanics/probability/statistics/biology(though probably more at an introductory undergraduate level if that)
The curriculum would probably roughly follow John S Wentworth's study guide, although of course any deviations based on curiosity or interest would certainly be welcome.
I'm thinking of having weekly video calls to discuss/dissolve confusions/set goals for next time as well as time tracking with Toggl, and maybe body doubling study sessions if they're helpful?
Let me know if you're interested, and we can set up a time to meet!
I am down to some level of tagging along and learning together, but not a full commitment. You probably want to find someone that can make a stronger commitment as an actual study partner.
I am a year 3 student (which means I may already know some of the stuff, and that I have other courses) and timezones likely suck (UTC+8 here). We can discuss on discord @papetoast if you like.
I thought this quote was nice and oddly up-to-the-minute, from Iris Murdoch's novel The Philosopher's Pupil (1983), spoken by the character William Eastcote at a Quaker meeting:
...My dear friends, we live in an age of marvels. Men among us can send machines far out into space. Our homes are full of devices which would amaze our forebears. At the same time our beloved planet is ravaged by suffering and threatened by dooms. Experts and wise men give us vast counsels suited to vast ills. I want only to say something about simple good things which are as it were
All quotes, unless otherwise marked, are Tolkien's words as printed in The Letters of J.R.R.Tolkien: Revised and Expanded Edition. All emphases mine.
Writing to his son Michael in the RAF:
...[here is] the tragedy and despair of all machinery laid bare. Unlike art which is content to create a new secondary world in the mind, it attempts to actualize desire, and so to create power in this World; and that cannot really be done with any real satisfaction. Labour-saving machinery only creates endless and worse labour. And in addition to this fundamental disability of a creature, is added the Fall, which makes our devices not only fail of their desire but turn to new and horrible evil. So we come inevitably from Daedalus and Icarus
How could you possibly know something like that?
For example, I’m sure I’ve looked up what “rostral” means 20 times or more since I started in neuroscience a few years ago. But as I write this right now, I don’t know what it means. (It’s an anatomical direction, I just don’t know which one.) Perhaps I’ll look up the definition for the 21st time, and then surely forget it yet again tomorrow. :)
What else? Umm, my attempt to use Anki was kinda a failure. There were cards that I failed over and over and over, and then eventually got fed up and stopped trying. (...