How to Disentangle the Past and the Future

[-]kpreid13y80

When a light bulb observes voltage, it emits light, regardless of whether it did so a second earlier. When the light bulb's internal attributes entangle with the voltage, they lose all information of what came before.

This example is false. An incandescent light bulb has a memory: its temperature. The temperature both determines the amount of light currently emitted by the bulb, and also the electrical resistance of the filament (higher when hot), which means that even the connected electrical circuit is affected by the state of the bulb — turning on the bulb produces a high “inrush current”.

A much better example would be a LED (not an LED light bulb, which likely contains a stateful power supply circuit), which is stateless for most practical purposes. (For example, once upon a time, there was networking hardware which could be snooped optically — the activity indicator LEDs were simply connected to the data lines and therefore transmitted them as visible light. Modern equipment typically uses programmed blinking intervals instead.)

[-]royf13y10

Fixed. Thanks!

[-]V_V13y10

That seems to be a convoluted way of defining a Markov process .

It would preferable if you attempted to use standard terminology and provide references frame the discourse within the theory.

[-]royf13y20

I explained this in my non-standard introduction to reinforcement learning.

We can define the world as having the Markov property, i.e. as a Markov process. But when we split the world into an agent and its environment, we lose the Markov property for each of them separately.

I'm using non-standard notation and terminology because they are needed for the theory I'm developing in these posts. In future posts I'll try to link more to the handful of researchers who do publish on this theory. I did publish one post relating the terminology I'm using to more standard research.

[-][anonymous]13y00

On the other hand, the future is independent of the past given the present. That's not profound metaphysics, that's simply how we define the state of a system: it's everything one needs to know of the past of the system to compute its future.

There is some metaphysics involved, particularly in positing that such a "state" exists, i.e. the question of whether the universe is causal.

For example, in classical mechanics, simply the positions of every particle do not constitute the state. You need both the positions and velocities. The velocity is a function of the current configuration of the world, and also some instantaneous information (time derivative) about how its moving through time. However, that you need only the first time derivative (and not the second and further) is a fact of physics, and whether a finite description of the "state" exists, or if not even an infinite description would suffice, is a metaphysics question.

In a probabilistic model (like quantum mechanics), you're asking for the conditional independence of the present and the past, and similar considerations hold.

[This comment is no longer endorsed by its author]Reply

[-]amcknight13y00

This seems to me like a major spot where the dualistic model of self-and-world gets introduced into reinforcement learning AI design (which leads to the Anvil Problem). It seems possible to model memory as part of the environment by simply adding I/O actions to the list of actions available to the agent. However, if you want to act upon something read, you either need to model this by having atomic read-and-if-X-do-Y actions, or you still need some minimal memory to store the previous item(s) read in.

[-][anonymous]13y-20

Intelligent agents need to have memory? You are otherwise right though, to a bayesian, priors are certainly important whether updating or not (also, the future doesn't have entirely mutual information with the past, as the future has more entropy).

[-]Vladimir_Nesov13y130

To perform actions, a UDT agent needs knowledge about the world to the same extent as any other agent does. The distinction lies in when (or what for) this knowledge is needed: an updateful agent would use this knowledge early in its decision making process, while an updateless agent can be more naturally modeled as first deciding on a general strategy chosen once for all possible situations, and only then enacting the action prescribed by that strategy for the present circumstances. This last step benefits from having accurate knowledge about the world. (Of course, in practice this knowledge would be needed for efficient decision making from the start, updatelessness is a decision making algorithm design principle, not a decision making algorithm execution principle.)

[-]Kratoklastes13y-30

The idea of entanglement of present and future states is what makes me think that dogs invest (albeit with a strategy that has binding constraints on rate of return): they know that by burying a bone, the probability that it will be available for them at some later date, is higher than it would be if the bone was left in the open.

In other words, the expected rate of return from burying, is greater than for not-burying. (Both expected rates of return are negative, and E[RoR|not-burying]= -100% for relatively short investment horizons).

It also opens up the idea that the dog knows that the bone is being stored for 'not-now', and that 'not-now' is 'after-now' in some important sense: that is, that dogs understand temporal causality in ways other than simple Pavlovian torture-silliness.

I've also watched a crow diversify: I was tossing him pieces of bread from my motel balcony while on a hiking holiday. He ate the first five or six pieces of bread, then started caching excess bread under a rock. After he'd put a couple of pieces under the rock, he cached additional pieces elsewhere, and did this for several different locations.

E[Pr(loss=100%)] diminishes when you bury the bone, but it diminishes even more if you bury multiple bones at multiple locations (i.e., you diversify).

And yet there are still educated people who will tell you that animals' heads are full of little more than "[white noise]...urge to have sex...[white noise]...urge to eat... [white noise]... PREDATOR! RUN... [white noise]..." - again, the tendency to run from a predator indicates that the animal is conscious of its (future) fate if it fails to escape, and it knows that it will 'cease-to-be' if the predator catches it.

The same sort of people believe that animals see in monochrome (in which case, why waste scarce evolutionary resources on developing, e.g., bright plumage?)

Surprise... I'm a vegetarian.

[-]MugaSofer13y-30

Of course, it's hard to be sure how much of those behaviors is "programmed in" by evolution.

The same sort of people believe that animals see in monochrome (in which case, why waste scarce evolutionary resources on developing, e.g., bright plumage?)

In fairness, some animals do see in monochrome. Others can see into the ultraviolet or have even more exotic senses. I guess these people (who I don't seem to have encountered, are you sure you're not generalizing?) are treating all animals as dogs, which isn't uncommon in fiction.

[-]Kratoklastes13y30

Not generalising in the least: I'm a man of the people who interacts often with the common man - particularly the rustic and bucolic variety (from the Auvergne in Deepest Darkest France to the dusty hinterland of rural Victora and New South Wales).

Everywhere I've ever lived, I've had conversations about animals (most of which I've initiated, I admit - and most of them before I went veggie), with folks ranging from French eleveurs de boeuf to Melbourne barristers and stock analysts: their lack of awareness of the complexity of animal sense organs (and their ignorance of animal awareness research generally) is astounding.

It may well be that you've never met anybody who thinks that all animals see in monochrome - maybe you're young, maybe you don't get out much, or maybe you don't have discussions about animals much. Fortunately, the 'animals see in black and white' trope is dying (as bad ideas should), but it's not dead.

To give you some context: I'm so old that when I went to school we were not allowed to use calculators (mine was the last generation to use trig tables). If you polled people my age (especially outside metropolitan areas) I reckon you would get >50% of them declaring that animals see in "black and white" - that's certainly my anecdotal experience.

Lastly: what makes you think that dogs see in monochrome? As far as we can tell dogs see the visual spectrum in the same way as a red-green colour-blind human does - they have both rods and cones in their visual apparatus, but with different sensitivities than humans' (same for cats, but carts lack cones that filter for red).

Of course we are only using "We can do this, and we have these cells" methods to make that call: as with some migratory birds that can 'see' magnetic fields, dogs and cats may have senses of which we are not yet aware. Cats certainly act as if they know something we don't.

[-]MugaSofer13y-10

Not generalising in the least: I'm a man of the people who interacts often with the common man - particularly the rustic and bucolic variety (from the Auvergne in Deepest Darkest France to the dusty hinterland of rural Victora and New South Wales).

Everywhere I've ever lived, I've had conversations about animals (most of which I've initiated, I admit - and most of them before I went veggie), with folks ranging from French eleveurs de boeuf to Melbourne barristers and stock analysts: their lack of awareness of the complexity of animal sense organs (and their ignorance of animal awareness research generally) is astounding.

Fair enough.

It may well be that you've never met anybody who thinks that all animals see in monochrome - maybe you're young, maybe you don't get out much, or maybe you don't have discussions about animals much.

Probably.

Lastly: what makes you think that dogs see in monochrome? As far as we can tell dogs see the visual spectrum in the same way as a red-green colour-blind human does - they have both rods and cones in their visual apparatus, but with different sensitivities than humans' (same for cats, but carts lack cones that filter for red).

Whoops, you're right. I was misremembering.

Cats certainly act as if they know something we don't.

... they do? I have four cats, and I've seen them do some pretty stupid things, but nothing that seemed to suggest they have any senses we haven't discovered yet . What kind of thing did you have in mind?

[-]Desrtopa12y10

... they do? I have four cats, and I've seen them do some pretty stupid things, but nothing that seemed to suggest they have any senses we haven't discovered yet . What kind of thing did you have in mind?

Well, I've often seen my own cats suddenly go into an alert state, whip their heads around, and stare at nothing in particular, for no reason I could determine. But I've always supposed this was a consequence of having hearing outside a human range of frequency and intensity.

LESSWRONG
LW

LESSWRONG
LW

22

How to Disentangle the Past and the Future

22

22