All of gugu's Comments + Replies

gugu10

Secondly, OpenAI had complete access to the problems and solutions to most of the problems. This means they could have actually trained their models to solve it. However, they verbally agreed not to do so, and frankly I don't think they would have done that anyway, simply because this is too valuable a dataset to memorize.

 

Now, nobody really knows what goes on behind o3, but if they follow the kind of "thinking", inference-scaling of search-space models published by other frontier labs that possibly uses advanced chain-of-thought and introspection com

... (read more)
57vik
  I definitely don't have a problem with this as well - just that this needs to be much more transparent and carefully though-out than how it happened here.   My concern is that "verbally agreeing to not use it for training" leaves a lot of opportunities to still use it as a significant advantage. For instance, do we know that they did not use it indirectly to validate a PRM that could in turn help a lot? I don't think making a validation set out of their training data would be as effective. Re: "maybe it would have took OpenAI a bit more time to contract some mathematicians, but realistically, how much more time?": Not much, they might have done this indepently as well. (assuming the mathematicians they'd contact would be equally willing to contribute directly to OpenAI)
gugu40

(indeed the politics of our era is moving towards greater acceptance of inequality)

How certain are you of this, and how much do you think it comes down more to something like "to what extent can disempowered groups unionise against the elite?".

To be clear, by default I think AI will make unionising against the more powerful harder, but it might depend on the governance structure. Maybe if we are really careful, we can get something closer to "Direct Democracy", where individual preferences actually matter more!

4Noosphere89
I am focused here on short-term politics in the US, which ordinarily would matter less, if it wasn't likely that world-changing AI would be built in the US, but given that it might, it becomes way more important than normal.
gugu40

[sorry, have only skimmed the post, but I feel compelled to comment.]


I feel like unless we make a lot of progress on some sort of "Science of Generalisation of Preferences", for more abstract preferences (non-biological needs mostly fall into this), even if certain individuals have, on paper, much more power than others, at the end of the day, they likely rely on vastly superintelligent AI advisors to realise those preferences, and at that point, I think it is the AI advisor _really_ in control.
I'm not super certain of this, like, the Catholic Church defin... (read more)

gugu10

I have read through most of this post and some of the related discussion today. I just wanted to write that it was really interesting, and as far as I can tell, useful, to think through Paul's reasoning and forecasts about strategy-related questions.
In case he believes this is a good idea, I would be very glad to read through a longer, more comprehensive document describing his views on strategic considerations.

gugu1714

Sooo this was such an intriguing idea that I did some research -- but reality appears to be more boring:

In a recent informal discussion I believe said OPP CEO remarked he had to give up the OpenAI board seat as his fiancée joining Anthropic creates a conflict of interest. Naively this is much more likely, and I think is much better supported by the timelines.
According to LinkedIn of the mentioned fiancée joined in already as VP in 2018 and was promoted to a probably more serious position in 2020, and her sibling was promoted to VP in 2019.
The Anthropic spl... (read more)

habryka153

The "conflict of interest" explanation also matches my understanding of the situation better.

gugu10

I think the point was less about a problem with refugees (which should be solved in time with European coordination), maybe more that the whole invasion is "good news" for conservative parties, as most crises are. 

gugu20

A lot of people brought up sanctions, and they could indeed influence European economy/politics.

I would be curious about what sanctions in particular are likely to be implemented, and what are their implications - a major economic setback/energy prices soaring could radicalize European politics perhaps?

My guess would be that overall the whole event increases support for conservative/nationalist/populist parties - for example, even though Hungary's populist government was trying to appear to be balancing "between the West and Russia" (thus now being in an uncomfortable situation), I think they can probably actually spin it around to their advantage. (Perhaps even more so, if they can fearmonger about refugees.)

4Mateusz Bagiński
The current Polish government is very much conservative, right-wing, and populist but they clearly voice support for Ukraine and criticize Putin's actions (which does not necessarily mean they're going to do anything substantial about it).