As uncertainty grows around how AI development will affect culture and society, it becomes more valuable to compare track records of predictions about technological progress.
I've recently been working on automating parts of the methodology from Arb's Scoring The Big 3's Predictive Performance report[1], and have had some promising preliminary results. I hope to try to automate most of the steps in the original report, making it feasible to analyse many more track records and publish the results.
I am particularly interested in the following questions:
See also original Cold Takes post explaining why such evaluations are valuable
Who’s track record of AI predictions would you like to see evaluated?
Whoever has the best track record :)
As uncertainty grows around how AI development will affect culture and society, it becomes more valuable to compare track records of predictions about technological progress.
I've recently been working on automating parts of the methodology from Arb's Scoring The Big 3's Predictive Performance report[1], and have had some promising preliminary results. I hope to try to automate most of the steps in the original report, making it feasible to analyse many more track records and publish the results.
I am particularly interested in the following questions:
See also original Cold Takes post explaining why such evaluations are valuable