JohnEPaton comments on Reinforcement Learning: A Non-Standard Introduction (Part 1) - Less Wrong

20 Post author: royf 29 July 2012 12:13AM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (18)

You are viewing a single comment's thread.

Comment author: JohnEPaton 30 July 2012 03:07:49AM 0 points [-]

I'm just wondering whether it's true that the Markov property holds for minds. I'm thinking that a snapshot of the world is not enough, but you also need to know something about the rate at which the world is changing. Presumably this information would require the knowledge of states further back.

Also, isn't there an innate element of randomness when it comes to decision making and how our minds work. Neurons are so small that presumably there are some sort of quantum effects, and wouldn't this mean again that information from one step previous wasn't enough.

I don't know, but just some thoughts.

Comment author: harshhpareek 03 August 2012 07:49:52PM *  0 points [-]

(Assuming Mind=Brain, i.e. the entire mind is just the physical brain and no "soul" is involved. Also, Neurons aren't really all that small, they're quite macroscopic -- though the processes in the neurons like chemical interactions need quantum mechanics for their description)

In Newtonian Mechanics, it is sufficient to know the positions and velocities (i.e. derivaties of position) of particles to determine future states. So, the world is Markov given this informatio.

In Schrodinger's equation, you again only need to know \Psi and it's time derivative to know all future states. I think the quantum properties of the brain are adequately described just with Schodinger's equation. You do need to include nuclear forces etc in a description of the brain. You may need quantum electrodynamics, but I think Schrodinger's equation is sufficient.

My physics education stopped before I got here, but Dirac's equation which may be necessary to model the brain seems to require the second time-derivative of the wavefunction -- so you may need the second order time-derivatives to make the model Markov. Can someone who knows a bit more quantum physics chime in here?

EDIT: Reading the wiki article more carefully, it seems Dirac's equation is also first order

Comment author: royf 30 July 2012 06:28:33AM 0 points [-]

In the model there's the distribution p, which determines how the world is changing. In the chess example this would include: a) how the agent's action changes the state of the game + b) some distribution we assume (but which we may or may not actually know) about the opponent's action and the resulting state of the game. In a physics example, p should include the relevant laws of physics, together with constants which tell the rate (and manner) in which the world is changing. Any changing parameters should be part of the state.

It seems that you're saying that it may be difficult to know what p is. Then you are very much correct. You probably couldn't infer the laws of physics from the current wave function of the universe, or the rules of chess from the current state of the game. But at this point we're only assuming that such laws exist, not that we know how to learn them.

p and q are probability distributions, which is where we allow for randomness in the process. But note that randomness becomes a tricky concept if you go deep enough into physics.

As for the "quantum mind" theory, as far as I can tell it's fringe science at best. Personally, I'm very skeptical. Regardless, such a model can still have the Markov property, if you include the wave function in your state.