All of Eric Askelson's Comments + Replies

Am I just inexperienced or confused, or is this paper using a lot of words to say effectively very little? Sure, this functional form works fine for a given set of regimes of scaling, but it effectively gives you no predictive ability to determine when the next break will occur. 

Sorry if this is overly confrontational, but I keep seeing this paper on Twitter and elsewhere and I'm not sure I understand why.

4Ethan Caballero
When f (in equation 1 of the paper ( https://arxiv.org/abs/2210.14891 ) not the video) of next break is sufficiently large, it gives you predictive ability to determine when that next break will occur; also, the number of seeds needed to get such predictive ability is very large. When f of next break is sufficiently small (& nonnegative), it does not give you predictive ability to determine when that next break will occur. Play around with fi in this code to see what I mean:  https://github.com/ethancaballero/broken_neural_scaling_laws/blob/main/make_figure_1__decomposition_of_bnsl_into_power_law_segments.py#L25-L29

It's perhaps somewhat morally cowardly to worry about it, but what about legal concerns? In general offering bounties for illegal acts is illegal, and often prosecuted if someone takes you up on the bounty. I don't think eliminating a mosquito species is explicitly illegal in any jurisdictions, but then again, one could certainly imagine prosecution in some scenarios.

You could add a clause like "This bounty will only be paid out if you did not break any laws of the United States in the process." Uncertainty in payouts reduces the value of a bounty, but it ... (read more)

This certainly seems like a way to increase the likelihood that I associate a response with a flashcard in Anki, but will this actually help recall? 

For example, one of my Anki cards might ask how to recursively copy all files with a certain extension out of a directory tree; by adding an image I might strengthen the association between the front and back of the Anki card, but not increase the probability that I actually remember the correct invocation to move files when I really need to. 

This is a failure mode I've noticed before with spaced rep... (read more)

29eB1
For language flashcards, you can have the front card be something like "Biblioteca" and the back card "Library [generated picture of a library]." This helps recall even though it's not part of the prompt-- it just seems to stick better in your memory.