LESSWRONG
LW

All of FL33TW00D's Comments + Replies

Firefox Focus on iPhone is useful here.

Delete all social media apps from your phone, and hide safari.

Only access any platforms via Firefox Focus - browsing sessions are ephemeral, so you need to login each time. This added friction basically solved it for me.

2gilch3mo

It's also available on Android.

Do humans really learn from "little" data?

FL33TW00D3mo10

The linked video says so at 30:45

Do humans really learn from "little" data?

FL33TW00D3mo30

I think you're missing a trick here purely focusing on waking seconds.

Demis explains this beautifully in this talk here:

TLDW:

The brain replays memories stochastically during sleep, and replays them OOMs faster than they were experienced. This "multi epoch training" allows the brain to learn much more from the environment, and it can prioritise salient experiences.

3Alice Wanderland2mo

Perhaps? I'm not fully understanding your point, could you explain a bit more what I'm missing - how does accounting for sleep and memory replay add to the point of comparing the pretraining dataset sizes between human brains and LLMs? At first glance, my understanding of your point is that adding in sleep seconds would increase the training set size for humans by a third or more. I wanted to make my estimate conservative so I didn't add in sleep seconds, but I'm sure there's a case for an adjustment adding it in.

2PeterMcCluskey3mo

"OOMs faster "? Where do you get that idea? Dreams indicate a need for more processing than what happens when we're awake, but likely less than 2x waking time.