All of Lawrence Phillips's Comments + Replies
220
Thanks Neel, we agree that we misinterpreted this. We've removed the claim.
13
Thanks for making the correction!
40
For anyone who'd like to see questions of this type on Metaculus as well, there's this thread. For certain topics (alignment very much included), we'll often do the legwork of operationalizing suggested questions and posting them on the platform.
Side note: we're working on spinning up what is essentially an AI forecasting research program; part of that will involve predicting the level of resources allocated to, and the impact of, different approaches to alignment. I'd be very glad to hear ideas from alignment researchers as to how to best go about this, a...
120
Nice work. A few comments/questions:
- I think you're being harsh on yourselves by emphasising the cost/benefit ratio. For one, the forecasters were asked to predict Elizabeth van Norstrand's distributions rather than their mean, right? So this method of scoring would actually reward them for being worse at their jobs, if they happened to put all their mass near the resolution's mean as opposed to predicting the correct distribution. IMO a more interesting measure is the degree of agreement between the forecasters' predictions and Elizabeth's distributions,
5
Great questions! I'll try to respond to the points in order.
Question 1
The distinction between forecasters/Elizabeth making predictions of her initial distributions or the final mean, was one that was rather confusing. I later wrote some internal notes to think through some implications in more detail. You can see them here.
I have a lot of uncertainty in how to best structure these setups. I think though that for cost effectiveness, Elizabeth's initial distributions should be seen as estimates given of the correct value, which is what she occasionally later gave. As such, for cost effectiveness we are interested in how well the forecasters did and estimating this correct value, vs. how well she did at estimating this correct value.
Separately, it's of course apparent that that correct value itself is an estimate, and there's further theoretical work to be done to best say what it should have been estimating, and empiricle work to be done to get a sense of how well it holds up against even more trustworthy estimates.
I personally don't regard the cost effectiveness here as that crucial, I'd instead treat much of this experiment as a set of structures that could apply to more important things in other cases. Elizabeth's time was rather inexpensive compared to other people/procedures we may want to use in the future, and we could also spend fixed costs improving the marginal costs of such a setup.
Question 2
We haven't talked about this specific thing, but I could definitely imagine it. The general hope is that even without such a split, many splits would happen automatically. One big challenge is to get the splits right. One may initially think that forecaster work should be split by partitions of questions, but this may be pretty suboptimal. It may be that some forecasters have significant comparative advantages to techniques that span across questions; for instance, some people are great at making mathematical models, and others are great at adjusting the
We'd probably try something along the lines you're suggesting, but there are some interesting technical challenges to think through.
For example, we'd want to train the model to be good at predicting the future, not just knowing what happened. Under a naive implementation, weight updates would probably go partly towards better judgment and forecasting ability, but also partly towards knowing how the world played out after the initial training cutoff.
There are also questions around IR; it seems likely that models will need external retrieval mechanisms to fo... (read more)