"GPT-4's release was delayed by ~8 months because they wanted to do safety testing"
I have heard this claim before (with 6 months). This could be understood as "GPT-4 was ready to go 6 month earlier, they simply did a lot of testing to go the extra mile."
Alternatively this is how long it took to make the foundational model useful, and while they did spend extra resources for red teaming etc. in parallel, this didn't come with a great cost of releasing it later.
Are we sure they didn't just count the time to RLHF it? Seems plausible to me that it always takes ~ 20% of dev time to RLHF a model. (epistemic status: spitballing)
Thanks! I contacted them.
So far, each generation of GPT has brought significant improvements. Thinking about the timeline for the next iteration, I noticed that there is a striking difference between extrapolating the past trend and what prediction markets seem to believe.
Forecasting short term AI Capabilities is important as current trends might continue to lead us to bigger changes. Also, it is nice to have testable predictions that we can use to calibrate our predictions and figure out who is worth listening to.
Rule 1 of forecasting: stop thinking too much and look at some historical data.
We don't know how long it took to develop each GPT, but we can look at how much... (read 674 more words →)
Awesome, they didn't mind.