All of Robbo's Comments + Replies

Robbo110

That poem was not written by Hitler.

According to this website and other reputable-seeming sources, the German poet Georg Runsky published that poem, "Habe Geduld", around 1906.

On 14 May 1938 a copy of this poem was printed in the Austrian weekly Agrarische Post, under the title 'Denke es'. It was then falsely attributed to Adolf Hitler.

In the Hitler biography of John Toland (1976) it appeared for the first time in English translation. Toland made the mistake in identifying it as a true Hitler poem, supposedly written in 1923.

Robbo20

There was a rush to deontology that died away quickly, mostly retreating back into its special enclave of veganism.

Can you explain what you mean by the second half of that sentence?

Zvi172

Vegans believe that they should follow a deontological rule, to never eat meat, rather than weighing the costs and benefits of individual food choices. They don't consume meat even when it is expensive (in various senses) to not do so. And they advocate for others to commit to doing likewise.

Whereas EA thinking in other areas instead says to do the math.

Robbo31

To clarify, what question were you thinking that is more interesting than? I see that as one of the questions that is raised in the post. But perhaps you are contrasting "realize it is conscious by itself" with the methods discussed in "Could we build language models whose reports about sentience we can trust?"

5Gunnar_Zarncke
I was skimming the Cog parts too quickly. I have reread and found the parts that you probably refer to: and So yes, I see this question as a crucial one.
Robbo10

I think I'd need to hear more about what you mean by sapience (the link didn't make it entirely clear to me) and why that would ground moral patienthood. It is true in my opinion that there are other plausible grounds for moral patienthood besides sentience (which, its ambiguity notwithstanding, I think can be used about as precisely as sapience, see my note on usage), most notably desires, preferences, and goals. Perhaps those are part of what you mean by 'sapience'?

Robbo10

Great, thanks for the explanation. Just curious to hear your framework, no need to reply:

-If you do have some notion of moral patienthood, what properties do you think are important for moral patienthood? Do you think we face uncertainty about whether animals or AIs have these properties? -If you don't, are there questions in the vicinity of "which systems are moral patients" that you do recognize as meaningful?

1Signer
I don't know. If I need to decide, I would probably use some "similarity to human mind" metrics. Maybe I would think about complexity of thoughts in the language of human concepts or something. And I probably could be persuaded in the importance of many other things. Also I can't really stop on just determining who is moral patient - I start thinking what exactly to value about them and that is complicated by me being (currently interested in counterarguments against being) indifferent to suffering and only counting good things. Yes for "similarity to human mind" - we don't have precise enough knowledge about AI's or animals mind. But now it sounds like I've only chosen these properties to not be certain. In the end I think moral uncertainty plays more important role than factual uncertainty here - we already can be certain that very high-level low-resolution models of human consciousness generalize to anything from animals to couple lines of python.
Robbo10

Very interesting! Thanks for your reply, and I like your distinction between questions:

Positive valence involves attention concentration whereas negative valence involves diffusion of attention / searching for ways to end this experience.

Can you elaborate on this? What is do attention concentration v. diffusion mean? Pain seems to draw attention to itself (and to motivate action to alleviate it). On my normal understanding of "concentration", pain involves concentration. But I think I'm just unfamiliar with how you / 'the literature' use these terms.

2Jacob Pfau
The relationship between valence and attention is not clear to me, and I don't know of a literature which tackles this (though imperativist analyses of valence are related). Here are some scattered thoughts and questions which make me think there's something important here to be clarified: * There's a difference between a conscious stimulus having high saliency/intensity and being intrinsically attention focusing. A bright light suddenly strobing in front of you is high saliency, but you can imagine choosing to attend or not to attend to it. It seems to me plausible that negative valence is like this bright light. * High valence states in meditation are achieved via concentration of attention * Positive valence doesn't seem to entail wanting more of that experience (c.f. there existing non-addictive highs etc.), whereas negative valence does seem to always entail wanting less. That is all speculative, but I'm more confident that positive and negative valence don't play the same role on the high-level functional level. It seems to me that this is strong (but not conclusive) evidence that they are also not symmetric at the fine-grained level. I'd guess a first step towards clarifying all this would be to talk to some researchers on attention.
Robbo10

I'm trying to get a better idea of your position. Suppose that, as TAG also replied, "realism about phenomenal consciousness" does not imply that consciousness is somehow fundamentally different from other forms of organization of matter. Suppose I'm a physicalist and a functionalist, so I think phenomenal consciousness just is a certain organization of matter. Do we still then need to replace "theory" with "ideology" etc?

1Signer
It's basically what is in that paper by Kammerer: "theory of the difference between reporatable and unreportable perceptions" is ok, but calling it "consciousness" and then concluding from reasonable-sounding assumption "consciousness agents are moral patients" that generalizing theory about presense of some computational process in humans to universlal ethics is arbitrariness-free inference - that I don't like. Because reasonableness of "consciousness agents are moral patients" decrease than you substitute theory's content into it. It's like theory of beauty, when "precise computational theory that specifies what it takes for a biological or artificial system to have various kinds of conscious, valenced experiences" feels like more implied objectivity.
Robbo10

to say that [consciousness] is the only way to process information

I don't think anyone was claiming that. My post certainly doesn't. If one thought consciousness were the only way to process information, wouldn't there not even be an open question about which (if any) information-processing systems can be conscious?

1superads91
I never said you claimed such either, but Charbel did. "It is possible that this [consciousness] is the only way to effectively process information" I was replying to his reply to my comment, hence I mentioned it.
Robbo10

A few questions:

  1. Can you elaborate on this?

Suffering seems to need a lot of complexity

and also seems deeply connected to biological systems.

I think I agree. Of course, all of the suffering that we know about so far is instantiated in biological systems. Depends on what you mean by "deeply connected." Do you mean that you think that the biological substrate is necessary? i.e. you have a biological theory of consciousness?

AI/computers are just a "picture" of these biological systems.

What does this mean?

Now, we could someday crack con

... (read more)
1superads91
1.) Suffering seems to need a lot of complexity, because it demands consciousness, which is the most complex thing that we know of. 2.) I personally suspect that the biological substrate is necessary (of course that I can't be sure.) For reasons, like I mentioned, like sleep and death. I can't imagine a computer that doesn't sleep and can operate for trillions of years as being conscious, at least in any way that resembles an animal. It may be superintelligent but not conscious. Again, just my suspicion. 3.) I think it's obvious - it means that we are trying to recreate something that biological systems do (arithmetics, imagine recognition, playing games, etc) on these electronic systems called computers or AI. Just like we try to recreate a murder scene with pencils and paper. But the murder drawing isn't remotely a murder, it's only a basic representation of a person's idea of a murder. 4.) Correct. I'm not completely excluding that possibility, but like I said, it would be a great luck to get there not on purpose. Maybe not "winning the lottery" luck as I've mentioned, but maybe 1 to 5% probability. We must understand that suffering takes consciousness, and consciousness takes a nervous system. Animals without one aren't conscious. The nature of computers is so drastically different from that of a biological nervous system (and, at least until now, much less complex) that I think it would be quite unlikely that we eventually unwillingly generate this very complex and unique and unknown property of biological systems that we call consciousness. I think it would be a great coincidence.
Robbo10

Thanks for this great comment! Will reply to the substantive stuff later, but first - I hadn't heard of the The Welfare Footprint Project! Super interesting and relevant, thanks for bringing to my attention

Robbo20

A third (disconcerting) possibility is that the list of demands amounts to saying “don’t ever build AGIs”

That would indeed be disconcerting. I would hope that, in this world, it's possible and profitable to have AGIs that are sentient, but which don't suffer in quite the same way / as badly as humans and animals do. It would be nice - but is by no means guaranteed - if the really bad mental states we can get are in a kinda arbitrary and non-natural point in mind-space. This is all very hard to think about though, and I'm not sure what I think.

I’m hope

... (read more)
Robbo20

Thanks, I'll check it out! I agree that the meta-problem is a super promising way forward

Robbo20

The whole field seems like an extreme case of anthropomorphizing to me.

Which field? Some of these fields and findings are explicitly about humans; I take it you mean the field of AI sentience, such as it is?

Of course, we can't assume that what holds for us holds for animals and AIs, and have to be wary of anthropomorphizing. That issue also comes up in studying, e.g., animal sentience and animal behavior. But what were you thinking is anthropomorphizing exactly? To be clear, I think we have to think carefully about what will and will not carry over from... (read more)

Robbo10

This author is: https://fantasticanachronism.com/2021/03/23/two-paths-to-the-future/

"I believe the best choice is cloning. More specifically, cloning John von Neumann one million times"

Robbo100

I guess even though I don't disagree that knowledge accumulation has been a bottleneck for humans dominating all other species, I don't see any strong reason to think that knowledge accumulation will be a bottleneck for an AGI dominating humans, since the limits to human knowledge accumulation seem mostly biological. Humans seem to get less plastic with age, mortality among other things forces us to specialize our labor, we have to sleep, we lack serial depth, we don't even approach the physical limits on speed, we can't run multiple instances of our own

... (read more)
2Rohin Shah
(To be clear, the thing you quoted was commenting on the specific argument presented in that post. I do expect that in practice AI will need social learning, simply because that's how an AI system could make use of the existing trove of knowledge that humans have built.)
Robbo190

The core part of Ajeya's model is a probability distribution over how many OOMs of compute we'd need with today's ideas to get to TAI / AGI / APS-AI / AI-PONR / etc.

I didn't know the last two acronyms despite reading a decent amount of this literature, so thought I'd leave this note for other readers. Listing all of them for completeness (readers will of course know the first two):

TAI: transformative AI

AGI: artificial general intelligence

APS-AI: Advanced, Planning, Strategically aware AI [1]

AI-PONR: AI point of no return [2]

[1] from Carlsmith, which Dan... (read more)

4Daniel Kokotajlo
Sorry! I'll go back and insert links + reference your comment
Robbo190

In general, I don't yet see a strong reason to think that our general brain architecture is the sole, or potentially even primary reason why we've developed civilization, discontinuous with the rest of the animal kingdom. A strong requirement for civilization is the development of cultural accumulation via language, and more specifically, the ability to accumulate knowledge and technology over generations.

In The Secrets of Our Success, Joe Henrich argues that without our stock of cultural knowledge, individual humans are not particularly more generally... (read more)

6Matthew Barnett
I 75% agree with this, but I do think that individual humans are smarter than individual chimpanzees. A big area of disagreement is distinguishing between "intrinsic ability to innovate" vs. "ability to process culture", and whether it's even possible to distinguish the two. I wrote a post about this two years ago.  This is the big crux for me on the evolution of humans and its relevance to the foom debate. Roughly, I think Henrich's model is correct. I think his model provides a simple, coherent explanation for why humans dominate the world, and why it happened on such a short timescale, discontinuously with other animals. Of course, intelligence plays a large role on his model: you can't get ants who can go to the moon, no matter how powerful their culture. But the the great insight is that our power does not come from our raw intelligence: it comes from our technology/culture, which is so powerful because it was allowed to accumulate.  Cultural accumulation is a zero-to-one discontinuity. That is, you can go a long time without any of it, and then something comes along that's able to do it just a little bit and then shortly after, it blows up. But after you've already reached one, going from "being able to accumulate culture at all" to "being able to accumulate it slightly faster" does not give you the same discontinuous foom as before. We could, for example, imagine that an AI that can accumulate culture slightly faster than other humans. Since this AI is only slightly better than humans, however, it doesn't go and create its own culture on its own. Unlike the humans -- who actually did go and create their own culture completely on their own, separate from other animals -- the AI will simply be one input to the human economy. This AI would be important input to our economy for sure, but not a completely separate entity producing its own distinct civilization, like the prototypical AI that spins up nanobot factories and kills us all within 3 minutes. It wil
Robbo20

Thanks so much for this!

  1. Curious about

For example, I was aiming to pursue a PhD in machine learning, partly because I thought it would make me worthwhile. When I felt worthwhile I stopped that; I was able to think more freely about which strategy looked best according to my values.

If you have a chance I’d love to hear more about what this process looked like. What did thinking something would make you worthwhile feel like? Do you think that self love helped you care less about the status of a PhD? Or was it some other mechanisms? In general, how self... (read more)

Robbo20

I'm curious about this passage:

Yes, yes, all of that is good, that is an excellent list of some of the downsides one should measure. It reminds me of nothing so much as an Effective Altruist trying to measure the advantages of an intervention to find things they can measure and put into a report. Yes, they will say, here is a thing called ‘development’ so it counts as something that can be put into the utility function and we can attach a number, excellent, very good. Then we can pretend that this is a full measure of how things actually work, and still

... (read more)
1The Scary Black Hundreder
Moreover, it is unclear to me what point the OP is making about the list of potential downsides of wearing masks. Perhaps it helps to already know his position on that matter?
Robbo10

Thanks for your thoughts! I think I'm having a bit of trouble unpacking this. Can you help me unpack this sentence:

"But I our success rides on overcoming these arguments and designing AI where more is better."

What is "more"? And what are "these arguments"? And how does this sentence relate to the question of whether explain data makes us put place more or less weight on similar-to-introspection hypotheses?

2Charlie Steiner
Whoops, I accidentally a word there. I've edited that sentence to "But I think our success rides on overcoming these arguments and designing AI where more is better." Where "more" means more data about humans, or more ability to process the information it already has. And "these arguments" means the arguments for why too much data might lead the AI to do things we don't want (maybe the most mathematically clear example is how CIRL stops being corrigible if it can accurately predict you). So to rephrase: there are some reasons why adding brain activity data might cause current AI designs to do things we don't want. That's bad; we want value learning schemes that come with principled arguments that more data will lead to better outcomes.
Robbo10

You might be interested in this post by Harri Besceli, which argues that "the best and worst experiences you had last week probably happened when you were dreaming".

Eric Schwitzgebel has also written that philosophical hedonists, if consistent, would care more about the quality of dream experiences: https://schwitzsplinters.blogspot.com/2012/04/how-much-should-you-care-about-how-you.html

Robbo30

Even if we were able to get good readings from insula & cingulate cortex & amygdala et alia, do you have thoughts on how and whether we could "ground" these readings? Would we calibrate on someone's cringe signal, then their gross signal, then their funny signal - matching various readings to various stimuli and subjective reports?

4Steven Byrnes
In principle, I think different reactions should project to different subcortical structures (e.g. the hypothalamus has lots of little cell groups that look different and do different things, I think). In practice, I dunno, I guess what you said sounds about right.
Robbo30

Hi Steven, thanks!

  1. On terminology, I agree.

Wait But Why, which of course is not an authoritative neuroscience source, uses "scale" to mean "how many neurons can be simultaneously recorded". But then it says fMRI and EEG have "high scale", but "low spatial resolution" - somewhat confusing since low spatial resolution means that fMRI and EEG don't record any individual neurons. So, my gloss on "scale" is more like WBW actually is talking about, and probably is better called "coverage". And then it's best to just talk about "number of simultaneously record... (read more)

Robbo170

I scheduled a conversation with Evan based on this post and it was very helpful. If you're on the fence, do it! For me, it was helpful as a general career / EA strategy discussion, in addition to being useful for thinking about specifically Long-Term Future Fund concerns.

And I can corroborate that Evan is indeed not that intimidating.

Robbo40

"I'm tempted to recommend this book to people who might otherwise be turned away by Rationality: From A to Z."

Within the category of "recent accessible introduction to rationality", would you recommend this Pinker book, or Julia Galef's "Scout Mindset"? Do thoughts on the pros and cons of each, or who would benefit more from each?

2Matthew Barnett
I only skimmed Julia Galef's book, which is why I didn't compare the two. I suspect her book would be a better fit for newcomers, but I'm not sure.
Robbo*80

Thanks for collecting these things! I have been looking into these arguments recently myself, and here are some more relevant things:

  1. EA forum post "A New X-Risk Factor: Brain-Computer Interfaces" (August 2020) argues for BCI as a risk factor for totalitarian lock-in.
  2. In a comment on that post, Kaj Sotala excerpts a section of Sotala and Yampolskiy (2015), "Responses to catastrophic AGI risk: a survey". This excerpts contains links to many other relevant discussions:
    1. "De Garis [82] argues that a computer could have far more processing power than a human brain
... (read more)
1niplav
That's a pretty impressive list of resources! I hadn't done a lot of research on the topic beforehand, but I'll definitely look into these when expanding on the arguments in this post. Especially the Shulman comment in 4. seems quite relevant, I wonder why FHI hasn't published their work on it (too much of a hassle to format & prepare?)
Robbo70

Thank you for writing about this. It's a tremendously interesting issue. 

I feel qualitatively more conscious, which I mean in the "hard problem of consciousness" sense of the word. "Usually people say that high-dose psychedelic states are indescribably more real and vivid than normal everyday life." Zen practitioners are often uninterested in LSD because it's possible to reach states that are indescribably more real and vivid than (regular) real life without ever leaving real life. (Zen is based around being totally present for real life. A Zen master

... (read more)
2lsusr
Though I wrote "while meditating", that language is misleading. The effects persist after meditation. They are often most salient immediately after meditation since, while meditating, I am too focused on meditating to appreciate the effects. When I have a consistent mediation practice, I am more conscious along the intensity, complexity and access dimensions. I feel more conscious along the experiential repertoire too, but that might be more subjective. What do you mean by "determinacy"? I don't understand your definition. I would be surprised if there weren't other ways I am more conscious after meditation that isn't included under your terms, but this is a notoriously difficult experience to describe.
Robbo20

Tons of handy stuff here, thanks!

I love the sound of Cold Turkey. I use Freedom for my computer, and I use it less than I otherwise would because of this anxious feeling, almost certainly exaggerated but still with a basis in reality, that whenever I start a full block it is a Really Big Deal and I might accidentally screw myself over - for example, if I suddenly remember I have to do something else. (Say, I'm looking for houses and it turns out I actually need to go look something up). But Cold Turkey, I'd just block stuff a lot more freely without the an... (read more)

Robbo60

I enjoyed reading this and skimming through your other shortforms. I’m intrigued by this idea of using the short form as something like a journal (albeit a somewhat public facing one).

Any tips, if I might want to start doing this? How helpful have you found it? Any failure modes?

2Willa
Cheers :) copied from bottom of post: "My #1 tip is to start writing shortforms, whatever you can do, give it a go :) try different strategies, write about different types of things, be more personal or less personal, fail publicly, and so on so you can see what works well for you and grow in the ways you want to grow!" Were there any parts you found particularly enjoyable, interesting, or even enlightening in some way? I'll share my experiences thus far writing these shortforms: * I definitely use these shortforms as a public journal or log, and I find that really helpful in several ways. * When I post something on the public internet (i.e. public http/https sites, not walled platforms like Facebook, Twitter, Discord, or otherwise) I find that it gives me more self-confidence, feels very "real" and tangible in a way that writing privately or posting on a private / walled platform does not (that realness and tangible "in the world" feel is good for me), and feels like a good avenue for "leveling up" in a variety of ways. * I value feedback, critique, suggestions, discussion, and so on from other people. I believe that a person becomes the best version of theirself, "levels up" more quickly and in a better way, and so on when practising their arts amongst a community of other practitioners, publicly. I don't do very well when it's just me by myself and while there are some forms of deliberate practice for some skills and arts that work okay-ish solo or require being solo occasionally, I'm at my very best when I'm learning and operating publicly in a community. So these shortforms help me learn and operate publicly in a community I care about and like being a part of, strengthen my efforts to learn and practise, and more. * These shortforms feel like a great way for me to practise writing publicly before doing the honestly scarier and more intimidating thing of writing a regular post here on LW that could go on main / the frontpage. Since I'm not that wo
Answer by Robbo100

Jonathan Simon is working on such a project: "What is it like to be AlphaGo"? 

Robbo30

[disclaimer: not an expert, possibly still confused about the Baldwin effect]

A bit of feedback on this explanation: as written, it didn’t make clear to me what makes it a special effect. “Evolution selects for genome-level hardcoding of extremely important learned lessons.” As a reader I was like, what makes this a special case? If it’s useful lesson then of course evolution would tend to select for knowing it innately - that does seem handy for an organism.

As I understand it, what is interesting about the Baldwin effect is that such hard coding is selecte... (read more)

3TurnTrout
Right, I wondered this as well. I had thought its significance was that the effect seemed Lamarckian, but it wasn't. (And, I confess, I made the parent comment partly hoping that someone would point out that I'd missed the key significance of the Baldwin effect. As the joke goes, the fastest way to get your paper spell-checked is to comment it on a YouTube video!) Thanks for this link. One part which I didn't understand is why closeness in learning-space (given your genotype, you're plastic enough to learn to do something) must imply that you're close in genotype-space (evolution has a path of local improvements which implement genetic assimilation of the plastic advantage). I can learn to program computers. Does that mean that, given the appropriate selection pressures, my descendents would learn to program computers instinctively? In a reasonable timeframe? It's not that I can't imagine such evolution occurring. It just wasn't clear why these distance metrics should be so strongly related. Reading the link, Dennett points out this assumption and discusses why it might be reasonable, and how we might test it.
Robbo20

I'm very intrigued by "prosthetic human voice meant for animal use"! Not knowing much about animal communication or speech in general, I don't even know what this mean. Could you say a bit more about what that would be?

Robbo30

Welcome, David! What sort of math are you looking to level up on? And do you know what AI safety/related topics you might explore? 

4David Udell
My medium-term math goal is to pick up some algebra and analysis.  I've heard from some people with math backgrounds that those are good basics to pick up if you're interested in modern math.  My roadmap from here to there is to finish off David Lay's Linear Algebra textbook plus an equivalent textbook for calculus (which I haven't done any of since high school), and then move on to intro real analysis and intro abstract algebra textbooks.  So far, I've found self-studying math very rewarding, and so self-motivating as long as I'm not starved for time. Lately I've been reading up on some of the stuff on persuasion tools/AI "social superpowers."  It's an intrinsically interesting idea that in the medium-term future, following the best arguments you can find given that you read around broadly enough could cease to be a reliable route to holding the most accurate possible views -- if we get widespread proliferation of accessible and powerful persuasion tools.  If GPT-n gets really good at generating arguments that convince people, it might become dangerous (with regard to preserving your terminal values and sanity) to read around on the unfiltered internet.  So this seems like a cool thing to think more about.
Robbo40

Thanks for this! People interested in the claim (which Korsgaard takes to be a deficiency of utilitarianism) that for utilitarians "people and animals don’t really matter at all; they are just the place where the valuable things happen", might be interested in Richard Yetter Chappell's [1] paper "Value Receptacles" (pdf). It's an exploration of what this claim could even mean, and a defense of utilitarianism in light of it. 

[1] Not incidentally, a long-time effective altruist. Whose blog is great.

3Erich_Grunewald
I know the author and the blog but didn't know the paper, thanks!
Robbo10

Interesting - what sort of thing do you use this for? what sort of thing have you done after rolling a 2? 

I imagine it must be things that are in some sense 'optional' since (quite literally) odds are you will not end up doing it.