All of Ian Televan's Comments + Replies

from random import *

runs = 100000
S = runs
for _ in range(runs):
   while(randint(1,20) != 1):
       S += 1
print(S/runs)

>>> 20.05751

In my experience teachers tend to only give examples of typical members of a category. I wish they'd also give examples along the category border, both positive and negative. Something like: "this seems to have nothing to do with quadratic equations, but it actually does, this is why" and "this problem looks like it can be solved using quadratic equations but this is misleading because XYZ". This is obvious in subjects like geography, (when you want to describe where China is, don't give a bunch of points around Beijing as examples, but instead d... (read more)

Thank you very much for posting this! I've been thinking about this topic for a while now and feel like this is criminally overlooked. There are so many resources on how to teach other people effectively, but virtually none on how to learn things effectively from other people (not just from textbooks). Yet we are often surrounded by people who know something that we currently don't and who might not know much about teaching or how to explain things well. Knowing what questions to ask and how to ask them makes these people into great teachers - while you reap the benefits! - this feels like a superpower. 

While I agree that the algorithm might output 5, I don't share the intuition that it's something that wasn't 'supposed' to happen, so I'm not sure what problem it was meant to demonstrate. I thought of a few ways to interpret it, but I'm not sure which one, if any, was the intended interpretation:

a) The algorithm is defined to compute argmax, but it doesn't output argmax because of false antecedents. 

- but I would say that it's not actually defined to compute argmax, therefore the fact that it doesn't output argmax is not a problem.

b) Regardless of th... (read more)

3abramdemski
OK, this makes sense to me. Instead of your (A) and (B), I would offer the following two useful interpretations: 1: From a design perspective, the algorithm chooses 5 when 10 is better. I'm not saying it has "computed argmax incorrectly" (as in your A); an agent design isn't supposed to compute argmax (argmax would be insufficient to solve this problem, because we're not given the problem in the format of a function from our actions to scores), but it is supposed to "do well". The usefulness of the argument rests on the weight of "someone might code an agent like this on accident, if they're not familiar with spurious proofs". Indeed, that's the origin of this code snippet -- something like this was seriously proposed at some point. 2: From a descriptive perspective, the code snippet is not a very good description of how humans would reason about a situation like this (for all the same reasons). Right, this makes sense to me, and is an intuition which I many people share. The problem, then, is to formalize how to be "selectively blind" in an appropriate way such that you reliably get good results.

I don't quite follow why 5/10 example presents a problem.

Conditionals with false antecedents seem nonsensical from the perspective of natural language, but why is this a problem for the formal agent? Since the algorithm as presented doesn't actually try to maximize utility, everything seems to be alright. In particular, there are 4 valid assignments: 

The algorithm doesn't try to select an assignment with largest , but ... (read more)

3abramdemski
Hmm. I'm not following. It seems like you follow the chain of reasoning and agree with the conclusion: This is exactly the point: it outputs 5. That's bad! But the agent as written will look perfectly reasonable to anyone who has not thought about the spurious proof problem. So, we want general tools to avoid this kind of thing. For the case of proof-based agents, we have a pretty good tool, namely MUDT (the strategy of looking for the highest-utility such proof rather than any such proof). (However, this falls prey to the Troll Bridge problem, which looks pretty bad.) More generally, the problem is that for formal agents, false antecedents cause nonsensical reasoning. EG, for the material conditional (the usual logical version of conditionals), everything is true when reasoning from a false antecedent. For Bayesian conditionals (the usual probabilistic version of conditionals), probability zero events don't even have conditionals (so you aren't allowed to ask what follows from them). Yet, we reason informally from false antecedents all the time, EG thinking about what would happen if  So, false antecedents cause greater problems for formal agents than for natural language. The problem is also "solved" if the agent thinks only about the environment, ignoring its knowledge about its own source code. So if the agent can form an agent-environment boundary (a "cartesian boundary") then the problem is already solved, no need to try reversed outputs. The point here is to do decision theory without such a boundary. The agent just approaches problems with all of its knowledge, not differentiating between "itself" and "the environment".

Could someone explain why this doesn't degenerate into an entirely circular concept when we postulate a stronger compiler; or why it doesn't become entirely dependent on the choice of the compiler?

  1. There are many programs that output identical sequences. That's a waste. Make it so that no two different programs have the same output.
  2. There are many sequences that when fed into the compiler don't result in valid programs. That's a waste. Make it so that every binary sequence represents a valid program.

Now we have a set of sequences that we'd like to encode: S ... (read more)

I thought of a slightly different exception for the use of "rational": when we talk about conclusions that someone else would draw from their experiences, which are different from ours. "It's rational for Truman Burbank to believe that he has a normal life." 

Or if I had an extraordinary experience which I couldn't communicate with enough fidelity to you, then it might be rational for you not to believe me. Conversely, if you had the experience and tried to tell me, I might answer with "Based only on the information that I received from you, which is p... (read more)

Richard Feynman once said that if you really understand something in physics you should be able to explain it to your grandmother.  I believed him.

Curiously enough, there is a recording of an interview with him where he argues almost exactly the opposite, namely that he can't explain something in sufficient detail to laypeople because of the long inferential distance.

It seems that the mistake that people commit is imagining the the second scenario is a choice between 0.34*24000 = 8160 and 0.33*27000 = 8910. Yes, if that was the case, then you could imagine a utility function that is approximately linear in the region 8160 to 8910, but sufficiently concave in the region 24000 to 27000  s.t. the difference between 8160 and 8910 feels greater than between 24000 and 27000... But that's not the actual scenario with which we are presented. We don't actually get to see 8160 or 8910. The slopes of the ... (read more)

But is the Occam's Razor really circular? The hypothesis "there is no pattern" is strictly simpler than "there is this particular pattern", for any value of 'this particular'.. Occam's Razor may expect simplicity in the world, but it is not the simplest strategy itself.  

Edit: I'm talking about the hypothesis itself, as a logic sequence of some kind, not that, which the hypothesis asserts. It asserts maxentropy - the most complex world. 

Originally I thought of an exception where the thing that we don't know was a constructive question. e.g. given more or less complete knowledge or material science, how to we construct a decent bridge? But it's an obvious limitation, no self-proclaimed reductionist would actually try to apply reductionism in such situation.

It seems to me that you're describing a reverse scenario: suppose we have an already constructed object, and want to figure out how works - can reductionism still be used? I'd still say yes.

Take an airplane, for example. Knowing relevant... (read more)

2TAG
You shouldn't ignore computational tractability, it's important. It's not necessary to strenuously defend reductionism in order to "exclude the supernatural'.

Something felt off about this example and I think I can put my finger on it now. 

My model of the world gives the event with the blue tentacle probability ~0. So when you ask me to imagine it, and I do so, what it feels like to me like I'm coming up with a new model to explain it, which gives a higher probability to that outcome than my current model does. This seems to be the root of the apparent contradiction, it appears that I'm violating the invariant. But I don't think that that's what actually happening. Consider this fictional exchange:

EY: Imagi

... (read more)

Care to elaborate? Also, that's not really an exception, but a boundary - it's exactly what you would expect if there are finitely many layers of composition i.e. the world is not like an infinite fractal. 

1TAG
Social construction is an exception to reductionism. A lot of things could be used as physical currency. Leaves are a bad choice, but things ranging from cowrie shells to obsidian shards have been used. You can't tell what money is by examining it microscopically...in fact that';s an problem in archaelogy, where some ancient artifacts remain mysterious despite high-tech investigation. But you can tell what money is by looking outward, at its function, at how it's used ... money is the thing that can be exchanged for any other thing. And that kind of non-reductionism doesn't imply anything spooky ..banknotes arent immaterial entities.. and that is very much the point: you don't have to believe that in strict reductionism in order to be broadly reductionist or materialist.

Of course it doesn't work for problems where the objects in question are already fundamental and cannot be reduces any further. But that's what I meant in the original post - reductionist frameworks would fail to produce any new insights if we were already at the fundamental level.

2TAG
That's not the only exception.

If reductionism was wrong then I would expect reductionist approaches to be ineffective. Every attempt at gaining knowledge using a reductionist framework would fail do discover anything new, except by accident on very rare occasions. Or experiments would fail to replicate because the conservation of energy was routinely violated in unpredictable ways. 

2TAG
Reductionism isnt something that has to be 100% true or 0% true. It can be something that works for some problems but not others.

Conservation laws or not, you ought to believe in the existence of the photon because you continue having the evidence of its existence - it's your memory of having fired the photon! Your memory is entangled with the state of the universe, not perfectly, but still, it's Bayesian evidence. And if your memory got erased, then indeed, you'd better stop believing that the photon exists.

That seems unlikely. There is already a certain difficulty in showing that illusion of free will is an illusion. "It seems like you have free will, but actually, it doesn't seem." - The seeming is self-evident, so what does it mean to say that something actually doesn't seem if it feels like it seems. As far as I understand it, it's not like it doesn't really seem so, but you're mistaken about it and think that it actually seems so, and then mindfulness meditation clears up that mistake for you and you stop thinking that it seems that you have free will. I... (read more)

As Sam Harris points out, the illusion of free will is itself an illusion. It doesn't actually feel like you have free will if you look closely enough. So then why are we mistaken about things when we don't examine them closely enough? Seems like a too-open-ended question. 

2Yoav Ravid
Is the illusion of the illusion of free will also an illusion? Is it a recursive illusion?

Update:  a) is just wrong and b) is right, but unsatisfying because it doesn't address the underlying intuition which says that the stopping criterion ought to matter. I'm very glad that I decided to investigate this issue in full detail and run my own simulations instead of just accepting some general principle from either side.

MacKay presents it as a conflict between frequentism vs bayesianism and argues why frequentism is wrong. But I started out with a bayesian model and still felt that motivated stopping would have some influence. I'm going ... (read more)

Fixing my predictions now, before going to investigate this issue further (I have Mackay's book within the hand's reach and would also like to run some Monte-Carlo simulations to check the results; going to post the resolution later): 

a) It seems that we ought to treat the results differently, because the second researcher in effect admits to p-hacking his results. b) But on the other hand, what if we modify the scenario slightly: suppose we get the results from both researchers 1 patient at a time. Surely we ought to update the priors by the same amo... (read more)

2Ian Televan
Update:  a) is just wrong and b) is right, but unsatisfying because it doesn't address the underlying intuition which says that the stopping criterion ought to matter. I'm very glad that I decided to investigate this issue in full detail and run my own simulations instead of just accepting some general principle from either side. MacKay presents it as a conflict between frequentism vs bayesianism and argues why frequentism is wrong. But I started out with a bayesian model and still felt that motivated stopping would have some influence. I'm going to try to articulate the best argument why the stopping criterion must matter and then explain why it fails. First of all the scenario doesn't describe exactly what the stopping criterion was. So I made up one: The (second) researcher treats patients and gets the results one at a time. He has some particular threshold for the probability that the treatment is >60% effective and he is going to stop and report the results the moment the probability reaches the threshold. He derives this probability by calculating a beta distribution for the data and integrating it from 0.6 to 1. (for those who are unfamiliar with the beta distribution, I recommend this excellent video by 3Blue1Brown) In this case the likelihood of seeing the data given underlying probability x is given by beta f(x)=(10070)x70(1−x)30, and the probability that treatment is >60% effective is α:=∫10.6f(x)dx. Now the argument: motivated stopping ensures that we don't just get 70 successes and 30 failures. We have an additional constraint that after each of the 99 outcomes for treatment the probability is strictly <α and only after the 100th patient it reaches α. Surely then, we must modify f(x) to reflect this constraint. And if the true probability was really >60%, then surely there are many Everett branches where the probability reaches α before we ever get to the 100th patient. If it really took so long, then it must be because it's actually less likely that

Fascinating subject indeed!

  1. I wonder how one would need to modify this principle to take into account risk-benefit analysis. What if quickly identifying wiggins meant incurring great benefit or avoiding great harm, then you would still need a nice short word for them. This seems obvious, the question is only how much shorter would the word need to be.
  2. Labels that are both short and phonetically consistent with a given language are in short supply, therefore we would predict that sometimes even unrelated things shared labels - if they occupied sufficiently di
... (read more)

I tried to reason through the riddles, before reading the rest and I made the same mistake as the jester did. It is really obvious in hindsight; I thought about this concept earlier and I really thought I had understood it. Did not expect to make this mistake at all, damn.

I even invented some examples on my own, like in the programming language Python a statement like print("Hello, World!") is an instruction to print "Hello, World!" on the screen, but "print(\"Hello, World!\")" is merely a string, that represents the first string, it's completely inert. (i... (read more)

(Of course I don't know how the authors actually come up with the hypothesis and I could be wrong, and the conclusions seem very plausible anyway, but..) The study seem to be susceptible to stopping bias. 

If the correlation was very strong right away, they could've said "Parental grief directly correlates with reproductive potential, Q.E.D!"

It wasn't, but they found a group resembling early hunter-gatherers; with the conclusion "Parental grief directly correlates with reproductive potential from back then, Q.E.D!"

If this didn't turn out either, and th... (read more)

I'm not sure whether the explanation at the end was right, but this is a very powerful technique nonetheless. I observed a similar problem many times, but couldn't quite put my finger on it.

Arguing against consistency itself. "I was trying to be consistent when I was younger, but now I'm more wise than that."

This feels very important.

Suppose that something *was* deleted. What was it? What am I failing to notice? 

Maybe learning to 'regenerate' the knowledge that I currently possess is going to help me 'regenerate' the knowledge that 'was deleted'.