LESSWRONG
LW

All of Troof's Comments + Replies

Influence functions - why, what and how
Troof1111

Thanks for this! One thing I don't understand about influence functions is: why should I care about the proximal Bregman objective? To interpret a model, I'm really interested in in the LOO retraining, right? Can we still say things like "it seems that the model relied on this training sample for producing this output" with the PBO interpretation?

Reply
6Nina Panickssery
I agree that approximating the PBO makes this method more lossy (not all interesting generalization phenomena can be found). However, I think we can still glean useful information about generalization by considering "retraining" from a point closer to the final model than random initialization. The downside is if, for example, some data was instrumental in causing a phase transition at some point in training, this will not be captured by the PBO approximation.  Indeed, the paper concedes: Purely empirically, I think Anthropic's results indicate there are useful things that can be learnt, even via this local approximation: My intuition here is that even if we are not exactly measuring the counterfactual "what if this datum was not included in the training corpus?", we could be estimating "what type of useful information is the model extracting from training data that looks like this?". 
The Dictatorship Problem
Troof66

Mostly agree, but I think it's wrong to imagine that you need a popular ideology to get a dictatorship. In Mao's or Pol Pot's case for instance, it was really driven by a minority of "fanatics" at first. Then a larger number of people can (at least pretend to) join the rank, and even become enforcers, until a stable equilibrium is reached: you don't know who is pretending and who is a true believer, and trying to guess can be very risky.

Reply
2dr_s
I don't remember the details of the history, but if I'm not wrong Mao's faction took power taking advantage of the chaos at the end of a long period of strife and civil war that originally was sparked by the overthrowing of the imperial government. I'd say having popular support isn't the only way for a dictatorship to start, but it's the main way to get there from a democracy. Other obvious ways are if a foreign power installs it somehow, or if, as in Mao's case, it's simply the result of the distillation process that takes place during a revolution, in which the most ruthless and fanatical bastards keep rising on top until they're the only ones left, regardless of what was the original impetus behind the uprising. So I think for our current trajectory in western powers, popularity remains the main road to power.
PredictIt is closing due to CFTC changing its mind
Troof81

I wonder if this has anything to do with Kalshi. The CFTC seems much more eager to go after weirder predictions markets (like Polymarket some time ago) now that it has a legit one.

Reply
Human values & biases are inaccessible to the genome
TroofΩ010

I like the tree example, and I think it's quite useful (and fun) to think of dumb and speculative way for the genome to access world concept. For instance, in response to "I infer that the genome cannot directly specify circuitry which detects whether you’re thinking about your family", the genome could:

  • Hardcode a face detector, and store the face most seen during early childhood (for instance to link them to the reward center). 
  • Store faces of people with an odor similar to amniotic fluid odor or with a weak odor (if you're insensitive to yo
... (read more)
Reply
2TurnTrout
I totally buy that the genome can do those things, but think that that it will probably not be locating the "family" concept in your learned world model.
A bicycle for your memory
Troof10

You might be interested in this blog post, which develops similar ideas and use the same cache analogy.

Reply
The Goldbach conjecture is probably correct; so was Fermat's last theorem
Troof30

For a more precise probabilistic approach to Fermat's last theorem by Feynman, which take into account the repartition of the nth powers, see this amazing article http://www.lbatalha.com/blog/feynman-on-fermats-last-theorem

Reply
1Stuart_Armstrong
Thanks! It's cool to see his approach.