Top postsTop post
The best language models are still getting smarter and more capable. To an increasing degree, this is because they are trained by Reinforcement Learning with Verifiable Rewards. Chain of thought reasoning allows models to evade the finite depth restriction on information flow by passing (relatively little) information back into the...
If Bayesian stats is so great, it should be able to do everything that ordinary stats can do. Indeed, as a fan of Bayesian stats, I have not made much of an attempt to learn normal stats with its p-values and t-tests outside what was required for my highschool statistics...
Logical Share Splitting I previously wrote a post based on the following law in probability theory: This law relates the probabilities of given individual statements to the probabilities of logical combinations of them. Prediction market prices should approximate probabilities, and by combining this with the above equation, we produce a...
An ideal machine learning model trained on an independent identically distributed dataset would end up making the same predictions after seeing all data, no matter what order it updated on the data. This is not true of neural nets trained by gradient descent. To first order in the learning rate,...
Airships are pretty dang cool. Airplanes need a continuous expenditure of energy to stay in the air, but if you just fill a bag with a light gas, you can stay up in the air with no energy expenditure at all. The two lightest gases are hydrogen and helium. Though...
Epistimic status: Pure speculation; I have 1 GPU, not dozens, so I have no reason to try and implement this stuff myself. When training large models, we might have many GPUs we'd like to use to speed the process up. This requires a way to parallelize training somehow. Some options...
Muireall and DaemonicSigil trade physics problems. Answers and discussion of the answers have been spoilered so you can try the problems yourself. Please also use spoiler formatting (type ">!") in the comments. DaemonicSigil Smeared Out Sun Okay, so the first problem is from Thinking Physics. (I promise I'll have some...