All of angmoh's Comments + Replies

If your opponent makes bad assumptions or bad decisions, your decisions won't be rewarded properly, and it can take you a very long time indeed to figure out from first principles that that is happening. If you are playing with a player who thinks that "all reds" is a strong hand, it can take you many, many hands to figure out that they're overestimating their hands instead of just getting anomalously lucky with their hidden cards while everyone else folds!

(Is someone who knows more about poker than I do going to tell me that this specific example is wrong

... (read more)
1rossry
I basically agree that it will be obvious to you (a reasonable poker player) or even to me (an interested and over-theorized amateur), but as I said in a cousin comment, what actually matters is whether it'll be obvious to the student making the mistake, which is a taller order. I think that "all reds" is overstated as literally written (I mean, you'll eventually go to showdown and have it explained to you), but I mean it to gesture at a broader point, and because the scene in Eleven is too good not to quote.

You're right but I like the chef example anyway. Even if cherry picked, it does get at a core truth - this kind of intuition evolves in every field. I love the stories of old hands intuitively seeing things a mile away.

Sutskever's response to Dwarkesh in their interview was a convincing refutation of this argument for me:

Dwarkesh Patel
So you could argue that next-token prediction can only help us match human performance and maybe not surpass it? What would it take to surpass human performance?

Ilya Sutskever
I challenge the claim that next-token prediction cannot surpass human performance. On the surface, it looks like it cannot. It looks like if you just learn to imitate, to predict what people do, it means that you can only copy people. But here is a counter argument f... (read more)

1tangerine
I respect Sutskever a lot, but if he believed that he could get an equivalent world model by spending an equivalent amount of compute learning from next-token prediction using any other set of real-world data samples, why would they go to such lengths to specifically obtain human-generated text for training? They might as well just do lots of random recordings (e.g., video, audio, radio signals) and pump it all into the model. In principle that could probably work, but it’s very inefficient. Human language is a very high density encoding of world models, so by training on human language models get much of their world model “for free“, because humanity has already done a lot of pre-work by sampling reality in a wide variety of ways and compressing it into the structure of language. However, our use of language still doesn’t capture all of reality exactly and I would argue it’s not even close. (Saying otherwise is equivalent to saying we’ve already discovered almost all possible capabilities, which would entail that AI actually has a hard cap at roughly human ability.) In order to expand its world model beyond human ability, AI has to sample reality itself, which is much less sample-efficient than sampling human behavior, hence the “soft cap”.

"Dwarkesh chose excellent questions throughout, displaying an excellent sense of when to follow up and how, and when to pivot."

This is the basic essence of why Dwarkesh does such good interviews. He does the groundwork to be able to ask relevant and interesting questions, ie. actually reading their books/works, and seems to consistently have put actual thought into analysing the worldview of his subjects.

The unambitiousness of modern geoengineering in general is dismaying.

For my Australian perspective: in the early 1900s there were people discussing how make use of the massive tracts of desert wasteland than make up most of the outback (ie: https://en.wikipedia.org/wiki/Bradfield_Scheme). None of this stuff could be considered today - one tree getting chopped down is liable to make the news: https://www.bbc.com/news/world-australia-54700074

Hard to escape the conclusion that we should all go lie in a ditch so as to guarantee that no impact to anything occurs ever.

This seems about right. Sam is a bit of a cowboy and probably doesn't bother involving the board more than he absolutely has to.

Stefánsson's "The Fat of the Land" is not really worth reading for any scientific insight today, but it's entertaining early 1900s anthropology. 

I don't have much of an opinion on any specific diet approach, but I can tell you my own experience with weight loss: I've always been between 15-25% bodyfat, yoyoing around. This routine isn't ideal, so I too am a 'victim' of the weight gain phenomenon. 

I have no satisfying answers for "why are we getting fatter" or "what makes caloric deficits so hard to maintain". I appreciate the diet blogging community that tries to tackle these questions with citizen science.

5Lev Protter
You may be interested in the what Tucker Goodrich is doing, he's been reviewing the literature, and it's probably the Linoleic acid. He's pointed at the research on the direct stimulation of the endocanabinoid system by omega6. He's interviewed someone who studied Tributyl tin, an obesogen present at relevant doses in all of our environments, it also happens to agonize the same receptors omega6s do, and also has canabinoid activity. Imagine trying to lose weight while smoking weed all day every day.

I assume you're familiar with Vilhjálmur Stefánsson's work if you are interested in low protein carnivore diets, but I was really was surprised to see how similar the 'ex150' sounds compared to the classic ~80:20 fat:protein experiments. These aren't really new ideas - although I'm sure there's a lot more information available on the details.

Anyway, dieting seems like something where while people on average fail, you do see some individual successes, so it's worth poking around the edges and giving things a go. It's always nice to see results from the coal... (read more)

3johnlawrenceaspden
I'm a complete innocent in all this. I've never needed to lose weight before, hence appealing for help here. And I don't know anything about Vilhjálmur Stefánsson or ketogenic diets in general. I do know that sloth and gluttony aren't the explanation, because I have been a slothful glutton for most of my life and I never gained much weight, nor lost it in the long periods when I was a sporty glutton. That's gone wrong recently, hence my search for reasons and techniques. Wikipedia seems to imply that Vilhjálmur Stefánsson was interested in eskimo-style all-meat diets.  exfatloss seems to be deliberately holding the amount of protein low, and that does seem to be a load-bearing part of his approach. Also the anti-polyunsaturated fats bit, which I find intriguing because it's such a good theory, and yet it makes predictions which don't seem to be true. I'm irritatingly fat, not dying of morbid obesity. I wouldn't touch such things with someone else's bargepole, absent twenty years of widespread use and researchers motivated to find the unintended consequences. Chemical patches as a remedy for chemical poisoning is ok, if it's the best we can do, but unless the problem is some permanent environmental contaminant, I'm sure we can do better than that!

Good post. 

For Westerners looking to get a palatable foothold on the priorities and viewpoints of the CCP (and Xi), I endorse "The Avoidable War" written last year by Kevin Rudd (former Prime Minister of Australia, speaks mandarin, has worked in China, loads of diplomatic experience - imo about as good of a person that exists to interpret Chinese grand strategy and explain it from a Western viewpoint). The book is (imo, for a politician), impressively objective in its analysis.

Some good stuff in there explaining the nature of Chinese cynicism about foreign motivations that echoes some of what is written in this post, but with a bit more historical background and strategic context. 

Yeah - it's odd, but TC is a self-professed contrarian after all.

I think the question here is: why doesn't he actually address the fundamentals of the AGI doom case? The "it's unlikely / unknown" position is really quite a weak argument which I doubt he would make if he actually understood EY's position. 

Seeing the state of the discourse on AGI risk just makes it more and more clear that the AGI risk awareness movement has failed to express its arguments in terms that non-rationalists can understand. 

People like TC should the first type of public intellectual to grok it, because EY's doom case is is highly analogous to market dynamics. And yet.

For example, a major point of disagreement between me and Eliezer is that Eliezer often dismisses plans as “too complicated to work in practice,” but that dismissal seems divorced from experience with getting things to work in practice (e.g. some of the ideas that Eliezer dismisses are not much more complex than RLHF with AI assistants helping human raters). In fact I think that you can implement complex things by taking small steps—almost all of these implementation difficulties do improve with empirical feedback.

EY's counter to this?

@Gerald Monroe On the question of Japan's unique lack of variation, I think it's unlikely to be decisive here. The 'monoculture' argument may have some merit, but even a genetically 'homogenous' population has plenty of variation - especially one 125m strong like the Japanese. 

Fertility related traits are just so fundamental to genetic fitness that selection is guaranteed to wring out the higher fertility alleles where the environment allows.

1[anonymous]
I agree, but in a less diverse environment the necessary genes might be rarer/there might be less choices for recombinations.  I agree that if there are subgroups in Japan's populations that under the current conditions are breeding above replacement, and if somehow Japan's conditions stayed static for centuries, eventually their 'problem' would self correct.   Ultimately it doesn't matter because the conditions are likely to change rapidly in the next ~5-50 years.   (leaving a wide window for the event "AGI + follow up developments")  I mean even ignoring broader scale changes from AGI, the "salaryman working to death" model doesn't make any sense when those kind of somewhat rote tasks can all be automated.  

Imo Japan is one of the more illuminating examples on this topic:

  • Japan had a TFR of 5 in the 1930s. It's been only 3 generations since Japan's TFR began to fall, and France took 5 to stablise around the current level (1830s-1980s). I agree that the trend since 2005 is too short term to be sure, but it's interesting to note! The above modelling suggests that a faster fertility transition should result in a faster bounceback - the lower the TFR the more adaptive high-TFR genes + cultures will be relatively.
  • The fertility transition hit East Asia harder and fa
... (read more)

There's good reason to believe the fertility transition as a general phenomenon is subject to negative feedback, thus almost certain to be self correcting. Selection self-evidently favours high fertility culture and genes.

 

See this study for an attempt to model the effects:

Correlations in fertility across generations: can low fertility persist?
Martin Kolk, Daniel Cownden and Magnus Enquist
Published:22 March 2014

https://royalsocietypublishing.org/doi/10.1098/rspb.2013.2561#d3e788,

"... Our models suggest a mechanism in which the recent fertility decline... (read more)

3[anonymous]
So other than the sheer resource model I mention - where a declining population frees up job slots and housing and education slots - cultural and genetic selection. Would this possibly explain countries with extreme long term decline, like Japan?  Japan seems to have closer to a true cultural/genetic monoculture, where there may simply not be enough diverse subgroups for ones with the right (cultural and genetic) alleles.

Agree - Gell-Mann amnesia sums up my experience with trying to get ChatGPT to be useful for a professional context so far. It is weak on accuracy and details.

My questions:

  • Is this something that can be overcome through skilful prompting?
  • Is there something about LLMs that makes these issues difficult to overcome?
  • ChatGPT is very conservative with providing factual information even when it's possible to tease out the relevant information (i.e. topics like law or commercial activities), is this purposeful throttling?

The speed issue is the #1 factor stopping me from trying audiobooks. A book might take me 4-8 hours to read but the internet tells me audio is 2-3x slower. I have a lot of other prejudices against audiobooks (flipping / skimming less easy, less focus on the task etc) but that's the main one.

1mikbp
How can you read 2-3x faster than a person speaks (1x)? Do you mean that when you "read" you just skim most of the time and really read only the parts you are interested in? As others mention, most readers allow you to increase the speed of the the audio. Up to 2x, for light content and with headphones is usually alright if you can concentrate on the audio. Faster than that, I find it really difficult to follow, so you are probably still faster.
3Brendan Long
I largely agree, but most apps will let you listen to things at faster speeds if you want to (my max speed is 1.25x - 1.5x depending on content dryness though).
3rsaarelm
Still makes sense if you listen when walking or driving when you couldn't read a book anyway. I mostly listen to podcasts instead of audiobooks though, a book is a really long commitment compared to a podcast episode.
3Phil Scadden
Couldnt agree more. I have no patience for audio and video. Too slow. Might watch instructional on video if I cant find decent manual. Not much into conferences either - just let me see the papers.

Seconded - I'd like to see more of this angle of analysis too. I assume the reason why the 'soft take-off' is underdiscussed is that tech people a) pay more attention to the news on AI, and b) see the future of this stuff viscerally and the endgame is what looms. I think that's not wrong, because the endgame is indeed transformative. But how we get there and how quickly it happens is a completely open question.

I work in the AEC industry (Architecture, Engineering, Construction) - 90%+ of people have zero idea about recent advances in AI. But on the other h... (read more)

I can anecdotally report that when I started consistently getting up immediately upon my alarm going off the subjective feeling of the first 5-10 minutes was far superior and I didn't feel much tiredness even with a relatively short night of sleep. I started doing this through setting my alarm to the maximum latest time possible and still allow me to get to work on time, and then noticed how much better I felt while doing this (previously I was a chronic snooze-button masher and felt pretty groggy waking up).

I have noticed in the WFH/office phases of the p... (read more)

So I looked into this, and the 'refine win condition' seems to be an actual technique that some of the best 3pt shooters do employ! I looked up some interviews with some of the best basketball shooters (Steph Curry and Ray Allen) and they mention playing little games with themselves while practice. They will do things such a define win condition as a swish instead of just making the shot, or employing some particular type of rhythm or footwork, or slightly altering the angle of their shot.

But on the other hand, they also mention plenty of repetitive drilli... (read more)

Thanks for the post - skill acquisition does seem like an area lacking attention from the rationalist community in general. 

I wonder if these learnings would apply to sports. "Stop after win" doesn't seem like a productive idea if you are trying to get better at, say, shooting a 3pt shot in basketball - the traditional approach is "reps reps reps". Thoughts?

1Nicole Dieker
BTW I could learn something even more useful than "stop after win" with another month of metrics; maybe "immediately redefine more sophisticated win condition and work towards that" is the real key. But the data I've got now suggests that just repeating something you've already practiced to a defined win condition is counterproductive.
1Nicole Dieker
If you define your win condition and achieve it, your next step is to define a new win condition and achieve it as well. That means you could go from "play passage all notes accurate from memory" to "play passage all notes accurate from memory without curling 5th finger." I'm going to write a post on Tuesday about reps reps reps vs. mindful repetition, and why a rep where you pay attention to why you're failing is just as valuable as a win. I think the real question is whether the traditional approach to shooting a 3-pointer works. Do people who shoot shoot shoot shoot land more 3-pointers than people who prep, define, shoot, evaluate, prep with new information, etc.? And do the best players do all of that so quickly (and so integratedly) that it looks like rep rep rep?

Ctrl+F "CFR" & "IFR" = 0 results. Apologies if you used different terminology here but I feel it is worth pointing out that CFR has cratered in many nations, even those with high case counts and high testing rates. The old assumption of ~.5-1% IFR seems to be thoroughly passed its use-by date. For highly vaxed populations current IFR is probably sub 0.1% overall, with the same qualifiers about age/morbidity profile etc.

Particularly of interest is Australia, which could be said to have had a lot of 'dry tinder' given the lack of pre-omicron deaths overa... (read more)

Apparently the issue was subject to a quite a bit of activist opposition anyway - but the timing is suspicious, especially since the Serbian PM was personally involving herself in the issue.

I don't like listening to Rogan, but I do like hearing him translated into rationalist-speak.

Huh, sort of strong +1 this. I do like listening to Rogan talk shit with friends, and I do like the way he brings ideas and knowledge out of interesting people, but I do not like him debating.

Yet Zvi made the debating eminently readable and both actually sounded sensible, which is a skill I do not have myself. If I'd written it then it would have been filled with the irritation I felt while listening to it. Zvi wrote it really well.

Four, he has uniquely powerful ideas about how to do project management well...

I am interested in this. Any suggestions for posts that focus on project management specifically?

3ryan_b
The unrecognized simplicities of effective action series of posts; in particular #2(b) linked above. The dominant examples are the Manhattan Project, Atlas, and Apollo. He also spends quite a bit of time on ARPA and Xerox/PARC. Included in the blog posts are the relevant books he was reading at the time, if I recall. 

Other examples of drugs that also seem to work for weight loss are GLP-1 agonists, which are relatively new and potentially promising.

I keep seeing references to 'long covid' by smart people, usually in the context of "we need to confirm what the effects are" but have never seen any real evidence from obscure or mainstream sources that it's something worth thinking about much. Surely if it were a real and important thing it would be identified and proven as such by now?

At this moment it strikes me as something that's probably either bullshit or a minor consideration until further evidence emerges to the contrary, but smart people continue to mention it. What am I missing?

3Lukas_Gloor
BBC: https://www.bbc.co.uk/news/health-57584295 The Times: https://www.thetimes.co.uk/article/why-is-britain-now-the-capital-of-long-covid-grjpvzfvw  There's a good chance that the numbers are overstated because of methodological difficulties, but it's not obvious, and the numbers are high enough that it would be worth concern even if they're 50% overstated.  Also, reddit is full with scary stories about Long Covid. Obviously that doesn't mean much because you get scary stories in any data set with tens of millions of people, but I think it's quite telling that a lot of people who self-declare as having Long Covid report having messed up senses of smell (e.g., strawberries tasting like burnt tyres).  That's not usually a sign of normal depression or burnout. It's clearly Covid related, and it seems to happen frequently enough to produce reddit communities filled with such posts every day, and the media did report on it, and the symptoms seem to be connected (e.g., the people who report the long-lasting loss of sense/taste also have other fatigue-related symptoms, which seem to have gone up in people with past Covid as well). 

I think it can be argued that in the West that practical achievement and directed pursuit probably is like this at a core level - humans are capable extraordinary effort if the conditions (and pressures) are right. Parenthood is one of the things that is still like everything used to be - pushed to the edge of human capability because there isn't another option. I think for many other modern pursuits a common failure mode is indulging in the eases and luxuries of modern existence - no such option with parenthood or language immersion.

Related thought: sloga... (read more)

AFAIK the main effect from the PM's policy change seems to be around relaxing indemnity rules for GPs so that they could hand out AZ if they wanted to without getting sued by people who develop the blood clot disorder. Previously this was an issue due to the current ATAGI advice recommending against it.

I thought the PM's statement on this wasn't too crazy - the blood clot risks are objectively still very low and the ATAGI report contemplates the then near-zero covid in Aus as you note. I assume somebody in govt realised that at current and projected v... (read more)

Thanks for the article - very informative and exactly the kind of content I enjoy!

In 2017 Australia held a public survey on whether same-sex marriage should be allowed, the results of which were pledged informally to be enacted by the government. Public votes on specific issues are relatively rare in Australia, so the debate around the procedural merits of voting on this particular issue were quite active.

I recall the main arguments against conducting a survey were mostly procedural criticisms, that it is wastefully expensive to hold a postal survey when ... (read more)

3Raoul Audouin
The the high rejection rate for referendums is explained by a few factors, among which: * If a referendum appears to be popular, the legislatures will pass it into a law before the referendum vote, and the initiators of the referendum will remove their initiative. Therefore the only initiatives that go into referendum are the ones that are either obviously unpopular among the majority of citizens, or the ones that the majority in the parliament strongly opposes. * Because citizens can initiate a referendum, the legislature is compelled into passing laws on the topics that could be brought through a referendum, therefore what is left for initiatives is more divisive topics.
6Martin Sustrik
The thing about the cost is that it's already paid. Voting happens four times a year in any case and adding one more initiative doesn't change much. There's certainly a cost associated with government and parliament processing the initiative, but again, that's what they are expected to do, it can't be really thought of as an extra cost.