The most promising AI model afaik right now is LLMs. My understanding is that progress in LLMs is slowing down as the gains we can make from more data and run times dry up, and that opportunity right now lies with smaller, more efficient models and creative applications of the existing technology.

I also can't see how LLMs could FOOM when creating them takes a lot of time and processing power. If an LLM understood themselves well enough to alter individual nodes and expect improvements then that would make sense, but that sounds hard enough that I would expect they're already basically superintelligence?

All that being said, I have not finished Nick Bostrom's Superintelligence, so I'd be very interested to hear what I might be missing here.