I disagree to an extent. The examples provided seem to me to be examples of "being stupid" which agents generally have an incentive to do something about, unless they're too stupid for that to occur to them. That doesn't mean that their underling values will drift towards a basin of attraction.
The corrigibility thing is a basin of attraction specifically because a corrigible agent has preferences over itself and it's future preferences. Humans do that too sometimes, but the examples provided are not that.
In general, I think you should expect dynamic preferences (cycles, attractors, chaos, etc...) anytime an agent has preferences over it's own future preferences, and the capability to modify it's preferences.
If you have access to the episode rewards, you should be able to train an ensemble of NNs using bayes + MCMC, with the final reward as output and the entire episode as input. Maybe using something like this: http://people.ee.duke.edu/~lcarin/sgnht-4.pdf
This get's a lot more difficult if you're trying to directly learn behaviour from rewards or vise-versa because now you need to make assumptions to derive "P(behaviour | reward)" or "P(reward | behaviour)".
Edit: Pretty sure OAI used a reward ensemble in https://openai.com/blog/deep-reinforcement-learning-fro...
I'm not convinced. Especially if this sort of underpay is a common policy across multiple orgs across the rationalist and EA communities. In a closed system with 2 people a "fair" price will balance the opportunity cost to the person doing the work and the value both parties assign to the fence.
But this isn't a closed system. I expect that low balling pay has a whole host of higher order negative effects. Off the top of my head:
Lightcone Infrastructure is competing for talent not only with industry, but also with the rest of the nonprofit sector. You could also frame this pay rate as a way to attract people from other forms of lower-paying nonprofit work, where they may be getting, say, industry - 40% rather than Lightcone's pay rate of industry - 30%.
From that point of view, I think LI's approach makes more sense. First, attract talent from even lower-paying nonprofits, along with a view enthusiasts from industry. Once that resource is tapped out, then you start increasing your ...
Can I encourage you to write up a top-level post detailing what you think the ideal salary algorithm for non-profits is?
I think you raise some valid points, and also can viscerally feel the punishment being inflicted on habryka for his forthrightness here (which is useful to everyone regardless of the specific salary), which is going to reduce similar efforts in the future in ways we will all be poorer for. My general solution for problems like this is to suggest people write up a top level post making their general case (which may link to the motiva...
Our current salary policy is to pay rates competitive with industry salary minus 30%.
What was the reasoning behind this? To me this would make sense if there was a funding constraint, but I was under the impression that EA is flush with cash.
If the following are the stated stakes:
If things go right, we can shape almost the full light cone of humanity to be full of flourishing life. Billions of galaxies, billions of light years across, for some 10^36 (or so) years until the heat death of the universe.
Then I would strongly advise against low balling ...
I feel like a lot of this depends on what Oliver meant by "competitive", and people are making different assumptions in the comments. I indeed think 70% of average local programmer wage would be too low, because I expect the people LC hires to be better than that average. OTOH, if it means "30% off literally the highest offer you can get", which this comment implies, that seems pretty reasonable to me (contingent on market rate coming into it at all). The highest offer you can get probably comes with a bunch of unpleasantness they have to pay people to tol...
One part of the reasoning goes something like this. Suppose you and your neighbor would like a nice hedge placed between your properties. This is something that both of you want. Also, your neighbor is a landscaper, with an hourly rate of $100. You propose to pay your neighbor to do the work.
What price is fair? One answer is $100/hour, given that's their standard rate. But I think this is wrong, because this isn't just a job for them, it is also something they personally care about.
To actually figure out the price we'd need some estimate of the value of th...
Here's a legitimate application, buying PornHub Premium. https://news.bitcoin.com/pornhubs-premium-services-crypto-payments-13-digital-assets-supported/.
Online payment processors are an oligopoly and can at any moment revoke a businesses ability to receive online payment even if they're not breaking the law. Thus what business is and is not permissible online is entirely up to the whims of this oligopoly and the law. Crypto provides a way around this.
I'm liking where this story is going.
IMO 2020 wasn't a turning point, and Facebook is not special. The events that happend lately have been a predictable development in a steadily escalating trend toward censorship. I'll note that these censorship policies are widespread across every social media platform, and infact extend well beyond social media and apply to the entire infrastructure stack. Everything from DDoS protection services, to cloud service providers, to payment processors have all been getting more bold over the course of several years about pulling plugs on people saying the wron...
I wouldn't look too deeply into that. The selection process for moderators on reddit is essentially first come first serve + how good are you at convincing existing moderators you should join the team. As far as I can tell this process doesn't usually select for "good" moderation, especially once a sub gets big enough that network effects make a subreddit grow despite "bad" moderation. This applies for most values of "good" and "bad".
That's great and all, but with all due respect:
Fuck. That. Noise.
Regardless of the odds of success and what the optimal course of action actually is, I would be very hard pressed to say that I'm trying to "help humanity die with dignity". Regardless of what the optimal action should be given that goal, on an emotional level, it's tantamount to giving up.
Before even getting into the cost/benefit of that attitude, in the worlds where we do make it out alive, I don't want to look back and see a version of me where that became my goal. I also don't think that ... (read more)
Agreed. Also here’s the poem that goes with that comment:
... (read more)Makes me think of the following quote. I'm not sure how much I agree with or endorse it, but it's something to think about.
"To win any battle, you must fight as if you are already dead.” — Miyamoto Musashi.
I don't in fact personally know we won't make it. This may be because I'm more ignorant than Eliezer, or may be because he (or his April first identity, I guess) is overconfident on a model, relative to me; it's hard to tell.
Regardless, the bit about "don't get psychologically stuck having-to-(believe/pretend)-it's-gonna-work seems really sane and healthy to me. Like falling out of an illusion and noticing your feet on the ground. The ground is a more fun and joyful place... (read more)
Exquisitely based
I think there's an important point about locus of control and scope. You can imagine someone who, early in life, decides that their life's work will be to build a time machine, because the value of doing so is immense (turning an otherwise finite universe into an infinite one, for example). As time goes on, they notice being more and more pessimistic about their prospects of doing that, but have some block against giving up on an emotional level. The stakes are too high for doomerism to be entertained!
But I think they overestimated their locus of control w... (read more)
As cheesy as it is, this is the correct response. I'm a little disappointed that Eliezer would resort to doomposting like this, but at the same time it's to be expected from him after some point. The people with remaining energy need to understand his words are also serving a personal therapeautic purpose and press on.