= 27d567bc2d48d9f40e9ccc20ea370b15)
V_V comments on Debunking Fallacies in the Theory of AI Motivation - Less Wrong
You are viewing a comment permalink. View the original post to see all comments and the full post content.
= 783df68a0f980790206b9ea87794c5b6)
Loading…= b715d02e85f7b3b4ae65ca3d3e450868)
Subscribe to RSS Feed
Powered by Reddit
= f037147d6e6c911a85753b9abdedda8d)
You are viewing a comment permalink. View the original post to see all comments and the full post content.
Comments (343)
Richard, I know (real!) people who think that wireheading is the correct approach to life, who would do it to themselves if it were feasible, and who would vote for political candidates if they pledged to legalize or fund research into wireheading. (I realize this is different from forcible wireheading, but unless I've misjudged your position seriously I don't think you see the lack of consent as the only serious issue with that proposal.)
I disagree with those people; I don't want to wirehead myself. But I notice that I am uncertain about many issues:
Should they be allowed to wirehead? Relatedly, is it cruel of me to desire that they not wirehead themselves? Both of these issues are closely related to the issue of suicide--I do, at present, think it should be legal for others to kill themselves, and that it would be cruel of me to desire that they not kill themselves, rather than desiring that they not want to kill themselves.
Are there philosophical foundations that I could rely on to convince them that, while they might want to wirehead now, after reflecting on those foundations, they would not want to wirehead? Do I endorse those foundations for myself, or am I mistaken in not wanting to wirehead?
How should my feelings on wireheading impact my feelings on related issues, like video gaming? I definitely became much more ambivalent about my video gaming hobby when I saw the parallels to wireheading, and the generalized case of superstimuli deserves philosophical attention. But it could be instead that, by endorsing video gaming (at least for others), that means I should also endorse wireheading, as just the best possible video game.
It seems to me that, rather than acknowledging that these questions are serious and unanswered, and thus seeking to find answers, you instead seek to dismiss those questions as not worth asking, and thus not worth answering. I feel like this is the wrong approach to philosophy, for many reasons.
I also suspect that this attitude generalizes: there are many places where I see Yudkowsky and similar thinkers saying "hold on a minute, what feature of reality can we point to that generates X?" and you seem to be responding with "but clearly X will be generated," or "but superintelligence will generate X." Is it actually clear? Will it be obvious in prospect that an AI design is superintelligent at determining what we want, rather than just obvious in retrospect? What features of the AI design should we be looking for?
For example, you point out that Swarm Relaxation Intelligences (SRIs) lack some dangerous features of CLAIs. This seems like a move in the right direction, but a proof of safety is not just the absence of known errors. It seems wise to let safety and verifiability drive our design choices, and to develop a strong understanding of what features lead to what guarantees (or, at least, what evidence in favor of safety or alignment).
I agree with everything in this comment up to:
This doesn't appear to be correct given that you can always transform functional programs into imperative programs and vice versa.
I've never heard that you can program in functional languages without doing testing and relying only on type checking to ensure correct behavior.
In fact, AFAIK, Haskell, the most popular pure functional programming language, is bad enough that you actually have to test all non-trivial programs for memory leaks, since it is not possible to reason except for special cases about the memory allocation behavior of a program from its source code and the language specification: the allocation behavior depends on implementation-specific and largely undocumented details of the compiler and the runtime.
Anyway, this memory allocation issue may be specific to Haskell, but in general, as I understand, there is nothing in the functional paradigm that guarantees a higher level of correctness than the imperative paradigm.
"Certain classes of errors" is meant to be read as a very narrow claim, and I'm not sure that it's relevant to AI design / moral issues. Many sorts of philosophical errors seem to be type errors, but it's not obvious that typechecking is the only solution to that. I was primarily drawing on this bit from Programming in Scala, and in rereading it I realize that they're actually talking about static type systems, which is an entirely separate thing. Editing.
Ok, sorry for being nitpicky.
In case it wasn't clear, thanks for nitpicking, because I was confused and am not confused about that anymore.
The relevant difference is in isolation and formulation of side effects, which encourages formulation of more pieces of code whose behavior can be understood precisely in most situations. The toolset of functional programming is usually better for writing higher order code that keeps the sources of side effects abstract, so that they are put back separately, without affecting the rest of the code. As a result, a lot of code can have well-defined behavior that's not disrupted by context in which it's used.
This works even without types, but with types the discipline can be more systematically followed, sometimes enforced. It also becomes possible to offload some of the formulation-checking work to a compiler (even when the behavior of a piece of code is well-defined and possible to understand precisely, there is the additional step of making sure it's used appropriately).
See Why Haskell just works. It's obviously not magic, the point is that enough errors can be ruled out by exploiting types and relying on spare use of side effects to make a difference in practice. This doesn't ensure correct behavior (for example, Haskell programs can always enter an infinite loop, while promising to eventually produce a value of any type, and Standard ML programs can use side effects that won't be reflected in types). It's just a step in the right direction, when correctness is a priority. There is also a prospect that more steps in this direction might eventually get you closer to correctness.
tangent:
I used to love functional programming and the elegance of e.g. Haskell, until I realized functional programming has the philosophy exactly backwards. You want to make it easy for humans and hard for machines, not vice versa.
Human think causally, e.g. imperatively and statefully. When humans debug functional/lazy programs, they generally smuggle in stateful/causal thinking to make progress. This is a sign something is going wrong with the philosophy.
Hm. The primary reason I got interested in fp is that I really like SQL, I think it is very easy for the human mind. And LINQ is built on top of functional programming, the Gigamonkeys book buils a similar query language on top of functional programming and macros, so it seems perhaps fp should be used that way, taking it as far as possible towards making query languages in it.
But I guess it always depends on what you want to do. My philosophy of programming is automation based. That means, if I need to do something once, I do it by hand, if a thousand times I write code. This, the ability to repeat operations many times, is what makes automating human work possible and from this I derived that the most important imperative structure is the loop. The loop is what turns something that was a mere set of rules into a powerfol data processing machinery, doing an operation many more times than I care to do it. With SQL, LINQ and other queries, we are essentially optimizing the loop as such. For example the generator expression in Python is a neat little functional loop-replacement, mini-query language.
Yes, that's how it was intended to be and how they spin it, but in practice the abstraction is leaky and it leaks in bad, difficult to predict ways therefore, as I said, you end up with things like having to test for memory leaks, something that is usually not an issue in "imperative" languages like Java, C# or Python.
I like the functional paradigm inside a good multi-paradigm language: passing around closures as first-class objects is much cleaner and concise than fiddling with subclasses and virtual methods, but forcing immutability and lazy evaluation as the main principles of the language doesn't seem to be a good design choice. It forces you to jump through hoops to implement common functionality like interaction, logging or configuration, and in return it doesn't deliver the higher modularity and intelligibility that were promised.
Anyway, we are going OT.
Agreed. Abstractions are still leaky, and where some pathologies in abstraction (i.e. human-understandable precise formulation) can be made much less of an issue by using the functional tools and types, others tend to surface that are only rarely a problem for more concrete code. In practice, the tradeoff is not one-sided, so its structure is useful for making decisions in particular cases.