My wife and I are the high school youth leaders at a UU church. Most of our youths are atheists or agnostics. I looked over the page you linked and subsequent pages on the course itself as I am very interested in helping our youths to think more rationally about goals. As the course is outlined now I would not suggest it to our youths because:
a) The course requires a lot of time and there's not enough information about how that time can be broken up. We meet for two hours per week and could maybe spare an hour of that for a program, but it's not clear th...
I purchased Shilov's Linear Algebra and put it on my bookshelf. When I actually needed to use it to refresh myself on how to get eigenvalues and eigenvectors I found all the references to preceding sections and choppy lemma->proof style writing to be very difficult to parse. This might be great if you actually work your way through the book, but I didn't find it useful as a refresher text.
Instead, I found Gilbert Strang's Introduction to Linear Algebra to be more useful. It's not as thorough as Shilov's text, but seems to cover topics fairly thoroughly and each section seems to be relatively self contained so that if there's a section that covers what you want to refresh your self on, it'll be relatively self contained.
If you aren't good at reading other people's signals, then the following heuristic is a pretty good one: If you like A, and you are wondering whether A likes you, the answer is no.
This heuristic is terrible if you're trying to find a romantic partner since following it consistently will always lead you to believe that the people you're interested in and whose reciprocal interest isn't clear to you are not interested in you. If you live in a society where your potential partner isn't supposed to make overt signals about their romantic interests (becaus...
Mixing the high glycemic load foods with the low glycemic load foods will result in a lower peak insulin concentration than if you ate them separately.
Also, millet has a slightly higher glycemic load (12/100g) compared to quinoa (10/100g), but has almost the same calories (~120) and is usually significantly cheaper (in my area, it's about a third the cost when purchased in bulk). It's probably comparable to the basmati brown (which I don't like the taste of).
I never feel full on meat + vegs, but adding a bit of bread does the job. Conversely, I never feel full on bread or rice or generally carbs only, adding a bit of meat does the job. It seems I need the combination.
My subjective experience is that starting a meal with a small amount of insulin spiking carbohydrate and then moving on to protein and fat results in feeling full faster than starting with the protein/fat and moving on to the carbs. I generally have a rule about portioning my food out at the beginning of the meal and not going back for more unt...
Apparently, one reason more intellectual people (typical Silicon Valley types) have less of an addiction problem is that they enjoy their work and thus life enough, they don't need to quickly wash down another suck of a day, so they can have less euphoric hobbies in the evening, say, drawing or painting.
I don't think this is exactly right. There is a correlation between intelligence and addiction, but it's not so strong that you won't still find a lot of addicts among the intelligentsia. Chemical addiction is a process whereby you ingest chemicals to st...
as my after-work fluid intake is mostly beer, I realized that now my brain cannot tell the difference between thirst and alcohol cravings.
Does your at-work brain confuse thirst with alcohol cravings too?
One idea would be thirst-like feeling -> drink water -> re-examine, but water is not a very good thirst quencher.
So test this by drinking something that isn't beer or water but matches your other criteria for good thirst quenchers. Carbonated water with lemon or lime juice in it will meet the criteria that you listed, but actually staying hy...
I guess what I mean is how do you know that it was that tactic that worked? How do you know that the people who showed compassion afterwards did so because it was demanded of them and not because people making angry demands made them feel more safe openly showing pre-existing compassion? I tend to agree with your first impression. I certainly don't respond to hostility by handing over control of my emotions to hostile people. I get defensive of my position.
Of course this is probably me committing the typical mind fallacy and trying to avoid thinking about...
Extrapolating from just the American civil rights movement and Indian independence movements, both of them were accompanied by barely contained violent movements with the same goals. Acceding to the demands of the peaceful protests provided a way to give the status of winning the conflict to the peaceful people while meeting the demands of the violent. Conversely, the recent Occupy movement had no real violent wing to speak of and while a lot of people showed up for the protests and there was a lot of awareness raised, there was no legislative impact at all.
Any example I could give could be disputed because it's always possible to reverse cause and effect and say "he only lacks empathy because of X" rather than "he believes X due to lack of empathy".
Fair enough. It does seem like it would be difficult to tell those two things apart from the outside.
And my impression is that empathy towards only the in-group is a normal human trait and that it is often affected by society only in the trivial sense that society determines what the in-group is.
Also true (probably).
If you're trying to ...
But individuals who have empathy with some others, but not other others, are more common. They can have terminal values to cause suffering for that portion of the population they don't have empathy with.
I'm having a hard time getting this. Can you provide an example where the lack of empathy for some group isn't driven by another value? My impression is that empathy is a normal human trait and that socializing teaches us who is worthy of empathy and who isn't, but then the lack of empathy is instrumental (because it serves to further the goals of society). People who actually lack empathy suffer from mental disorders like psychopathy as far as I know.
It took less time to highlight "Why We Are Fighting You" and search on Google than it took for you to ask for a source. Literally it took three clicks.
Are you are suggesting that people just have a desire to cause suffering and that their reasons (dieties, revenge, punishment, etc.) are mostly just attempts to frame that desire in a personally acceptable manner? I ask because it seems like most people probably just don't enjoy watching just anyone suffer, they tend to target other groups which suggests a more strategic reason than just enjoying cruelty.
I'm now tempted to include this announcement of the newsletter in the newsletter just for the one-off recursion joke I can make.
I say go for it, but then my highest voted submission to discussion was this.
But that's the thing. There is no sensory input for "social deference". It has to be inferred from an internal model of the world itself inferred from sensory data...Reinforcement learning works fine when you have a simple reward signal you want to maximize. You can't use it for social instincts or morality, or anything you can't just build a simple sensor to detect.
Why does it only work on simple signals? Why can't the result of inference work for reinforcement learning?
I don't think that humans are pure reinforcement learners. We have all sorts of complicated values that aren't just eating and mating.
We may not be pure reinforcement learners, but the presence of values other than eating and mating isn't a proof of that. Quite the contrary, it demonstrates that either we have a lot of different, occasionally contradictory values hardwired or that we have some other system that's creating value systems. From an evolutionary standpoint reward systems that are good at replicating genes get to survive, but they don't have ...
Okay, I am convinced. I really, really appreciate you sticking with me through this and persistently finding different ways to phrase your side and then finding ways that other people have phrased it.
For reference it was the link to the paper/book that did it. The parts of it that are immediately relevant here are chapter 3 and section 4.2.1.1 (and optionally section 5.3.5). In particular, chapter 3 explicitly describes an order of operations of goal and subgoal evaluation and then the two other sections show how wireheading is discounted as a failing str...
How would that [valuing universe-states themselves] work? Well that's the quadrillion dollar question. I have no idea how to solve it.
Yeah, I think this whole thread may be kind of grinding to this conclusion.
It's certainly not impossible as humans seem to work this way
Seem to perhaps, but I don't think that's actually the case. I think (as mentioned above) that we value reward signals terminally (but are mostly unaware of this preference) and nothing else. There's another guy in this thread who thinks we might not have any terminal values.
I'm no...
It discourages me that he tabooed 'values' and you immediately used it anyway.
In fairness, I only used it to describe how they'd come to be used in this context in the first place, not to try and continue with my point.
I wrote a Python-esque pseudocode example of my conception of what an AGI with an arbitrary terminal value's very high level source code would look like. With little technical background, my understanding is very high level with lots of black boxes. I encourage you to do the same, such that we may compare.
I've never done something l...
But there is no theoretical reason you can't have an AI that values universe-states themselves.
How would that work? How do you have a learner that doesn't have something equivalent to a reinforcement mechanism? At the very least it seems like there has to be some part of the AI that compares the universe-state to the desired-state and that the real goal is actually to maximize the similarity of those states which means modifying the goal would be easier than modifying reality.
And if it did have such a goal, why would it change it?
Agreed. I am trying to get someone to explain how such a goal would work.
Pleasure and reward are not the same thing. For humans, pleasure almost always leads to reward, but reward doesn't only happen with pleasure. For the most extreme examples of what you're describing, ascetics and monks and the like, I'd guess that some combination of sensory deprivation and rhythmic breathing cause the brain to short circuit a bit and release some reward juice.
Sure. My terminal goal is an abstraction of my behavior to shoot my laser at the coordinates of blue objects detected in my field of view.
Well, I suppose that does fit the question I asked. We've mostly been talking about an AI with the ability to read and modify it's own goal system which Yvain specifically excludes in the blue-minimizer. We're also assuming that it's powerful enough to actually manipulate it's world to optimize itself. Yvain's blue minimizer also isn't an AGI or ASI. It's an ANI, which we use without any particular danger all the time...
I don't think they're necessarily safe. My original puzzlement was more that I don't understand why we keep holding the AI's value system constant when moving from pre-foom to post-foom. It seemed like something was being glossed over when a stupid machine goes from making paperclips to a being a god that makes paperclips. Why would a god just continue to make paperclips? If it's super intelligent, why wouldn't it figure out why it's making paperclips and extrapolate from that? I didn't have the language to ask "what's keeping the value system stable through that transition?" when I made my original comment.
My apologies for taking so long to reply. I am particularly interested in this because if you (or someone) can provide me with an example of a value system that doesn't ultimately value the output of the value function, it would change my understanding of how value systems work. So far, the two arguments against my concept of a value/behavior system seem to rely on the existence of other things that are valuable in and of themselves or that there is just another kind of value system that might exist. The other terminal value thing doesn't hold much promise...
Sure. I think if you assume that the goal is paperclip optimization after the AI has reached it's "final" stable configuration then the normal conclusions about paperclip optimizers probably hold true. The example provided dealt more with the transition from dumb-AI to smart-AI and I'm not sure why Tully (or Clippy) wouldn't just modify their own goals to something that's easier to attain. Assuming that the goals don't change though, we're probably screwed.
I think FeepingCreature was actually just pointing out a logical fallacy in a misstatement on my part and that is why they didn't respond further in this part of the thread after I corrected myself (but has continued elsewhere).
If you believe that a terminal goal for the state of the world other than the result of a comparison between a desired state and an actual state is possible, perhaps you can explain how that would work? That is fundamentally what I'm asking for throughout this thread. Just stating that terminal goals are terminal goals by definition is true, but doesn't really show that making a goal terminal is possible.
A paperclip maximizer won't wirehead because it doesn't value world states in which its goals have been satisfied, it values world states that have a lot of paperclips
I am not as confident as you that valuing worlds with lots of paperclips will continue once an AI goes from "kind of dumb AI" to "super-AI." Basically, I'm saying that all values are instrumental values and that only mashing your "value met" button is terminal. We only switched over to talking about values to avoid some confusion about reward mechanisms.
...A pa
As far as I know terminal values are things that are valuable in an of themselves. I don't consider not building baby-mulchers to be valuable in and of itself. There may be some scenario in which building baby-mulchers is more valuable to me than not and in that scenario I would build one. Likewise with doomsday devices. It's difficult to predict what that scenario would look like, but given that other humans have built them I assume that I would too. In those circumstances if I could turn off the parts of my brain that make me squeamish about doing that, ...
Again, you've pulled a statement out of a discussion the context of the behavior of a self-modifying AI. So, fine. In my current condition I wouldn't build a baby mulcher. That doesn't mean that I might not build a baby mucher if I had the ability to change my values. You might as well say that I terminally value not flying when I flap my arms. The thing you're discussing just isn't physically allowed. People terminally value only what they're doing at any given moment because the laws of physics say that they have no choice.
Well, the pleasure center and the reward center are different things, but I take your meaning. I think that I could be conditioned to build a baby-mulching machine or a doomsday device. Why not? Other people have done it. Why would I assume that I'm that different from them?
EDIT TO ADD: Even if I have a value that I can't escape currently (like not killing people), that's not to say that if I had the ability to physically modify the parts of my brain that held my values I wouldn't do it for some reason.
Two other people in this thread have pointed out that the value collapse into wireheading or something else is a known and unsolved problem and that the problems of an intelligence that optimizes for something assumes that the AI makes it through this in some unknown way. This suggests that I am not wrong, I'm just asking a question for which no one has an answer yet.
Fundamentally, my position is that given 1.) an AI is motivated by something 2.) That something is a component (or set of components) within the AI and 3.) The AI can modify that/those compone...
I think I understood you. What do you think I misunderstood?
Maybe we should quit saying that evolution rewards anything at all. Replication isn't a reward, it's just a byproduct of an non-intelligent processes. There was never an "incentive" to reproduce, any more than there is an "incentive" for any physical process. High pressure air moves to low pressure regions, not because there's an incentive, but because that's just how physics works. At some point, this non-sentient process accidentally invented a reward system and replication,...
I don't consider morality to be a terminal value. I would point out that even a value that I have that I can't give up right now wouldn't necessarily be terminal if I had the ability to directly modify the components of my mind. They are unalterable because I am not able to physically manipulate the hardware, not because I wouldn't alter them if I could (and saw a reason to).
We don't have the ability to directly fulfil the reward center. I think narcotics are the closest we've got now and lots of people try to mash that button to the detriment of everything else. I just think it's a kind of crude button and it doesn't work as well as the direct ability to fully understand and control your own brain.
Thanks, Gleb. I will look into this more.
It really seems like you have gone out of your way to not actually share any content until I give you personal information. After looking at a few of your pages I still have no idea what you're offering except that it's something to help people find meaning in their lives using methods that (probably) have one or more scientific study to back them up. This is the kind of sales pitch I usually associate very strongly with scammers. The main differences between the way you look and the way a scammer looks are:
a) I fou... (read more)