Less Wrong is a community blog devoted to refining the art of human rationality. Please visit our About page for more information.

Comment author: advancedatheist 28 February 2015 01:21:09AM *  0 points [-]

I hate to spoil the mood for nerd grieving and geek hermeneutics, but Star Trek made a certain kind of sense in the late 1960's (nearly 50 years ago!) when the U.S. and the Soviet Union had real space programs which tried to do new things, one after another. But because astronautics has regressed since then, despite all accelerationist propaganda you hear from transhumanists, this genre of mythological framework for thinking about "the future" makes less and less sense. Given the failure of the "space age," would people 50 years from now, in a permanently Earth-bound reality, bother to watch these ancient shows and obsess over the characters?

Comment author: V_V 28 February 2015 10:08:53AM 2 points [-]

but Star Trek made a certain kind of sense in the late 1960's (nearly 50 years ago!) when the U.S. and the Soviet Union had real space programs which tried to do new things, one after another.

I haven really watched more than a few episodes of ToS, but IIUC it never even bothered to be a realistic depiction of how space exploration would look like. It was more e metaphor of the Cold War, in Space!

would people 50 years from now, in a permanently Earth-bound reality, bother to watch these ancient shows and obsess over the characters?

They will probably idolize some dude who played a vampire. Or zombie. Or BDSM vampire zombie...

Comment author: skeptical_lurker 26 February 2015 12:56:16PM 3 points [-]

I saw this paper before, and maybe I'm being an idiot but I didn't understand this:

Training Deep Neural Networks is complicated by the fact that the distribution of each layer's inputs changes during training, as the parameters of the previous layers change.

I thought one generally trained the networks layer by layer, so layer n would be completely finished training before layer n+1 starts. Then there is no problem of "the distribution of each layer's inputs changes" because the inputs are fixed once training starts.

Admittedly, this is a problem if you don't have all the training data to start of with and want to learn incrementally, but AFAICT that is not generally the case in these benchmarking contests.

Regardless, its amazing how simple DNNs are. People have been working on computer vision and AI for about 60 years, and then a program like this comes along which is only around 500 lines of code, conceptually simple enough to explain to anyone with a reasonable mathematical background, but can nevertheless beat humans at a reasonable range of tasks.

Comment author: V_V 27 February 2015 04:13:43PM *  0 points [-]

Regardless, its amazing how simple DNNs are. People have been working on computer vision and AI for about 60 years, and then a program like this comes along which is only around 500 lines of code, conceptually simple enough to explain to anyone with a reasonable mathematical background, but can nevertheless beat humans at a reasonable range of tasks.

Beware, there is a lot of non-obvious complexity in these models:
"Traditional" machine learning models (i.e. logistic regression, SVM, random forests) only have few hyperparameters and they are not terribly sensitive to their values, hence you can usually tune them coarsely and quickly.
These fancy deep neural networks can easily have tens, if not hundreds of hyperparameters, and they are often quite sensitive to them. A bad choice can easily make your training procedure quickly stop making progress (insufficient capacity/vanishing gradients) or diverge (exploding gradients) or converge to something which doesn't generalize well on unseen data (overfitting).
Finding a good choice of hyperparameters can be really a non-trivial optimization problem on its own (and a combinatorial one, since many of these hyperparameters are discrete and you can't really expect the model performances to depend monotonically on their values).
Unfortunately, in these DNN papers, especially the "better than humans" ones, hyperparameters values often seem to appear out of nowhere.
There is some research and tools to do that systematically, but it is not often discussed in the papers presenting novel architectures and results.

Comment author: skeptical_lurker 26 February 2015 08:32:36PM 3 points [-]

Really? I was under the impression that training the whole network with gradient decent was impossible, because the propagated error becomes infinitesimally small. In fact, I thought that training layers individually was the insight that made DNNs possible.

Do you have a link about how they managed to train the whole network?

Comment author: V_V 27 February 2015 03:48:40PM 0 points [-]

I was under the impression that training the whole network with gradient decent was impossible, because the propagated error becomes infinitesimally small.

If you do it naively, yes. But researches figured out how to attack that problem from multiple angles: from the choice of the non-linear activation function, to specifics of the optimization algorithm, to the random distribution used to sample the initial weights.

Do you have a link about how they managed to train the whole network?

The batch normalization paper cited above is one example of that.

Comment author: Jonathan_Lee 18 February 2015 10:50:12AM 4 points [-]

That sounds a rather odd argument to make, even at the time. Astronomy from antiquity was founded on accurate observations.

Astronomy and epistemology aren't quite the same. Predicting where Saturn would be on a given date requires accurate observation, and nobody objected to Coperniucus as a calculational tool. For example, the Jesuits are teaching Copernicus in China in Chinese about 2 years after he publishes, which implies they translated and shipped it with some alacrity.

The heavens were classically held to be made of different stuff; quintessense (later called aether) was not like regular matter -- this is obvious from the inside, because it maintains perpetual motion where normal matter does not. A lot of optical phenomena (eg. twinkling stars, the surface of the moon) were not seen as properties of the objects in question but properties of regular 4-elements matter between us and them.

By a modern standard, the physics is weird and disjointed... but that is historically how it was seen.

Comment author: V_V 18 February 2015 03:44:17PM 1 point [-]

By a modern standard, the physics is weird and disjointed... but that is historically how it was seen.

I wonder how current physics will look like if/when GR and QM will be finally unified...

Comment author: alienist 17 February 2015 06:28:07AM 6 points [-]

Before Newton unified terrestial and celestial mechanics, you needed to keep them separate whether you were using a geocentric or a heliocentric model. You still needed a sublunary sphere sphere around the Earth where things slow down and fall and break and decay on their way towards the End of Time, while God and the Angels watch us from their perfect and immutable Heaven. Neither the geocentric nor the heliocentric model had an advantage in terms of explanatory power here.

If the earth has a sublunary sphere, that suggests the earth is 'special', which is certainly more parsimonious in a geocentric universe. Also why doesn't earth's sublunary sphere cause it to fall into the sun?

Comment author: V_V 17 February 2015 04:38:10PM 0 points [-]

If the earth has a sublunary sphere, that suggests the earth is 'special', which is certainly more parsimonious in a geocentric universe.

Either way the Earth has to be special.

Also why doesn't earth's sublunary sphere cause it to fall into the sun?

Because nobody figured out that the Sun had gravity before Newton.

Comment author: V_V 15 February 2015 10:15:43PM *  5 points [-]

the pre-modern Catholic Church was opposed to the concept of the Earth orbiting the Sun with the deliberate purpose of hindering scientific progress and to keep the world in ignorance.

Are you sure you are not attacking a strawman/nut picking? I mean, there are certainly people who believe that, but is it really a representative position among atheists (*)?

(* Here I assume we are talking about atheists who don't partecipate to a secular/political religion, as these ones lend towards fanaticism, therefore I suppose they are more likely to hold false and inflammatory beliefs as long as they support their ideology and demonize competing ideologies)

Gravity. Why do the objects have weight, and why are they all pulled towards the center of the Earth? Why don't objects fall off the Earth on the other side of the planet? Remember, Newton wasn't even born yet! The geocentric view had a very simple explanation, dating back to Aristotle: it is the nature of all objects that they strive towards the center of the world, and the center of the spherical Earth is the center of the world.

So why don't the Sun and the planets fall on the Earth?

In the Aristotelic model you still needed a distinction between terrestial mechanics, ruling the sublunary sphere where gravity but also friction, drag, decay, and all kinds of irreversible processes occur, and celestial mechanics, ruling the celestial spheres, where everything moves like clockwork rather than "falling down" without any apparent energy source and doesn't show any signs of decay and irreversibility that 17th century people could have observed with instruments of their time.

Before Newton unified terrestial and celestial mechanics, you needed to keep them separate whether you were using a geocentric or a heliocentric model. You still needed a sublunary sphere sphere around the Earth where things slow down and fall and break and decay on their way towards the End of Time, while God and the Angels watch us from their perfect and immutable Heaven.
Neither the geocentric nor the heliocentric model had an advantage in terms of explanatory power here.

Comment author: James_Miller 12 February 2015 02:51:54PM *  4 points [-]

I also have a high variance in my intellectual abilities . I got a perfect score on the math section of the GRE, but received a C+ in my high school geometry class despite putting a massive amount of effort into it. A big challenge I have repeatedly faced is convincing people that my inability to accomplish certain things isn't due to laziness.

Comment author: V_V 13 February 2015 03:44:28PM 3 points [-]

If I understand correctly, there are various forms of dyscalculia which can selectively impact performance in arithmetic or geometry, and dyscalculia in general occurs over a wide range of IQ.

This suggests that skills in specific math fields may be related to specific neural circuits in the brain.

Comment author: Transfuturist 09 February 2015 05:49:28PM 2 points [-]

That really seems against the spirit of the experiment. If you categorically refuse to let the AI out, then you're contravening the entire purpose that the AI was created for. It might as well be destroyed. The implicit cost in refusing to determine whether the AI is Friendly is enormous.

Comment author: V_V 10 February 2015 06:40:33PM 2 points [-]

So what? You are not talking to a real AI, and the "experiment" is a poor model for a real AI safety assessment scenario.

Keep in mind that the rules states that the "AI" player gets to determine all the context of the fictional setting and the results of all tests. It's basically the "Game Master" in RPG terminology.
Can you beat a sufficiently smart and motivated GM who is determined to screw you player character? Seems pretty hard ("Rocks fall, Everyone Dies").

But in this game the "AI" player needs the specific approval of the "Gatekeeper" player in order to win, and the rules allow for the "Gatekeeper" player to step out of character or play an irrational character, which is exactly what you have to do to infallibly counter any machination the "AI" player can devise.

Comment author: asd 09 February 2015 12:00:28PM 0 points [-]

My strategy was that there would always be a default position in which I could switch if the opponent's argument started to get too convincing, and for me that was the "there's a 100% chance that all AIs are dangerous" position.

Comment author: V_V 09 February 2015 02:07:37PM 1 point [-]

"there's a 100% chance that all AIs are dangerous"

It seems to me that the default position of the Gatekeeper should be "I don't give a shit about AIs, I'm just playing to win."

Comment author: eli_sennesh 05 February 2015 09:45:14PM 1 point [-]

/u/JoshuaFox was telling me to wait until I actually recorded the voiced lecture before posting to LW, but oh well, here it is. I'll make a full and proper Discussion post when I've gotten better from my flu, taken tomorrow's exam, submitted tomorrow's abstracts to MSR, and thus fully done my full-time job before taking time to just record a lecture in an empty room somewhere.

Comment author: V_V 06 February 2015 12:16:32PM 0 points [-]

Thanks!

View more: Next