I am interested in this line of research, I feel it needs a lot more work than one paper, though.

A key question is whether we can dig down into the relationship between environments and learning agents. Are there low complexity environments that neural networks do badly in?

What is really essential about our laws of physics to create a world that neural networks do relatively well in?

I have another theory on how Deep Learning works: http://lesswrong.com/lw/m9p/approximating_solomonoff_induction/

The idea is that neural networks are a (somewhat crude) approximation of solomonoff induction.

I don't see why this article is on -1 karma at the moment. It's an interesting topic.

Here's the problem with talking x-risk with cynics who believe humanity is a net negative, and also a couple possible solutions.

Frequently, when discussing the great filter, or averting nuclear war, someone will bring up the notion that it would be a good thing. Humanity has such a bad track record with environmental responsibility or human rights abuses toward less advanced civilizations, that the planet, and by extension the universe, would be better off without us. Or so the argument goes. I've even seen some countersignaling severe enough to argue, somewhat seriously, in favor of building more nukes and weapons, out of a vague but general hatred for our collective insanity, politics, pettiness, etc.

Obviously these aren't exactly careful, step by step arguments, where if I refute some point they'll reverse their decision and decide we should spread humanity to the stars. It's a very general, diffuse dissatisfaction, and if I were to refute any one part, the response would be "ok sure, but what about [lists a thousand other things that are wrong with the world]". It's like fighting fog, because it's not their true objection, at least not quite. It's not like either of us feels like we're on opposite sides of a debate or anything though, so usually pointing out a few simple facts is enough to get a concession that there are exceptions to the rule "humanity sucks". However, obviously refuting all thousand things, one by one, isn't a sound strategy. There really is a lot of bad stuff that humanity has done, and will continue to do I'm sure.

Usually, I try to point at broad improving trends like infant mortality, war, extreme poverty, etc. I'll argue that the media biases our fears by magnifying all the problems that remain. I paint a rosy future of people fighting debater's prisons in the past, debating universal healthcare today, and in the future arguing fiercely over whether money and work are needed at all in their post-scarcity Star Trek economy. Political rights for minorities yesterday, social justice today, argue over any minor inconveniences tomorrow. Starvation yesterday, healthy food for all today, gourmet delicacies free next to drinking fountains tomorrow. I figure they're more likely to accept a future where we never stop arguing, but do so over progressively more petty things, and never realize we're in a utopia.

However, I think I might have better luck trying to counter-counter signal. "Yeah, humanity is pretty messed up, but why do you want to put us out of our misery? Shouldn't we be made to suffer through climate change and everything else we've brought on ourselves, instead of getting off easy? Imagine another thousand years of inane cubical work and a dozen more Trump presidencies. Maybe we'll learn our lesson." [Obviously, I'm joking here.]

I think this might have the advantage of aligning their cynicism with their more charitable impulses, at least the way my conversations tend to go. And there's no impulse to counter-counter-counter-signal, because I've gone up a meta-level and made the counter-signaling game explicit, which releases all the fun available from being contrarian, and moves the conversation toward new sources of amusement. I'll bet we could then proceed to have interesting discussions on how to solve the world's problems. If whoever I'm musing with comes up with a few ideas of their own, maybe they'll even take ownership of the ideas, and start to actually care about saving the world in their own way. I can dream, I suppose.

By how many orders of magnitude? Would you play Russian Roulette for $10/day? It seemed to me that implicit in your argument was that even if someone disagrees with you about the expected value, an order of magnitude or so wouldn't invalidate it. There's a rather narrow set of circumstances where your argument doesn't apply to your own situation. Simply asserting that you will sign up soon is far from sufficient. And note that many conditions necessitate further conditions; for instance, if you claim that your current utility/dollar ratio is ten times what it will be in a year, then you'd better not have turned down any loans with APY less than 900%.

And how does the value of cryonics go up as your mortality rate does? Are you planning on enrolling in a program with a fixed monthly fee?

Then you have the problem that I'm not in the USA (I plan to eventually move, once my career is strong enough to score the relevant visa); being in the US is the best way to ensure a successful, timely suspension. If you are in Europe you have to both pay more for transport and you will be damaged more by the long journey, assuming you die unexpectedly in Europe.

OTOH it looks like the mortality in your late 20s in the EU is less than half that in the US.

By how many orders of magnitude? Would you play Russian Roulette for $10/day? It seemed to me that implicit in your argument was that even if someone disagrees with you about the expected value, an order of magnitude or so wouldn't invalidate it. There's a rather narrow set of circumstances where your argument doesn't apply to your own situation. Simply asserting that you will sign up soon is far from sufficient. And note that many conditions necessitate further conditions; for instance, if you claim that your current utility/dollar ratio is ten times what it will be in a year, then you'd better not have turned down any loans with APY less than 900%.

And how does the value of cryonics go up as your mortality rate does? Are you planning on enrolling in a program with a fixed monthly fee?

They should have some statistics, even if they're not completely conclusive.

As I understand it, the costs are:

$1400 for lodging (commuting would cost even more) $2500 deposit (not clear on the refund policy) 10% of next year's income (with deposit going towards this)

I wouldn't characterize that as "very little". It's enough to warrant asking a lot of questions.

How would you characterize the help you got getting a job? Getting an interview? Knowing what to say in an interview? Having verifiable skills?

They should have some statistics, even if they're not completely conclusive.

As I understand it, the costs are:

$1400 for lodging (commuting would cost even more) $2500 deposit (not clear on the refund policy) 10% of next year's income (with deposit going towards this)

I wouldn't characterize that as "very little". It's enough to warrant asking a lot of questions.

How would you characterize the help you got getting a job? Getting an interview? Knowing what to say in an interview? Having verifiable skills?

Just a quick update, I signed the contract today and am now employed in the role of senior machine learning scientist at a company in Europe.