(I really like how gears-y your comment is, many thanks and strong-upvoted.)

This is helpful, thanks. Bummer though...

Claude.ai has web search! Woo-hoo! You have to enable it in the settings.

It mystifies me that as a Pro user my feature settings don't include the web search option, only the analysis tool. I wonder if it's a geographic location thing (I'm in Southeast Asia).

I like the optimal forager take, seems intuitively correct. I'd add that Dwarkesh struck gold by getting you on his podcast too. (Tangentially: this grand theory of intelligence video snippet reminds me of a page-ish-long writeup on that I stumbled upon deep in the bowels of https://gwern.net/ which I've annoyingly never been able to find again.)

Also thanks for the pointer to Werbos, his website Welcome to the Werbos World! funnily enough struck me as crackpot-y and I wouldn't have guessed just from the landing page that he's the discoverer of backprop, respected former program director at the NSF, etc. 

Just to clarify, your post's bottomline is that AIs won't be omnipotent, and this matters for AI because a lot of common real-life problems are NP-hard, but also that this doesn't really matter (for us?) because there are ways around NP-hardness through cleverness and solving a different problem, or else by scaling hardware and writing programs more efficiently, or (referencing James) by just finding a good-enough solution instead of an optimal one? 

I have 1050W of solar, ~10kWh of batteries, a 3kW hybrid inverter, and a 5.5kW gasoline generator. In spring and fall I can easily go a week without needing shore power or the generator. In summer and winter, I can't

Sorry naive question, I get that you can't do it in winter, but why not summer? Isn't that when solar peaks?

On a more substantive note: 

Aside from the normal cognitive benefits of being bilingual or multilingual, would learning some new language (or a conlang for this purpose) specifically to have conscious thought with be useful?

Not sure if this is exactly what you had in mind, since it's fictional transhumanist tech, but I was reminded of this passage from Richard Ngo's recent short story The Gentle Romance:

Almost everyone he talks to these days consults their assistant regularly. There are tell-tale signs: their eyes lose focus for a second or two before they come out with a new fact or a clever joke. He mostly sees it at work, since he doesn’t socialize much. But one day he catches up with a college friend he’d always had a bit of a crush on, who’s still just as beautiful as he remembers. He tries to make up for his nervousness by having his assistant feed him quips he can recite to her. But whenever he does, she hits back straight away with a pitch-perfect response, and he’s left scrambling.

“You’re good at this. Much faster than me,” he says abruptly.

“Oh, it’s not skill,” she says. “I’m using a new technique. Here.” With a flick of her eyes she shares her visual feed, and he flinches. Instead of words, the feed is a blur of incomprehensible images, flashes of abstract color and shapes, like a psychedelic Rorschach test.

“You can read those?”

“It’s a lot of work at first, but your brain adapts pretty quickly.”

He makes a face. “Not gonna lie, that sounds pretty weird. What if they’re sending you subliminal messages or something?”

Back home, he tries it, of course. The tutorial superimposes images and their text translations alongside his life, narrating everything he experiences. Having them constantly hovering on the side of his vision makes him dizzy. But he remembers his friend’s effortless mastery, and persists. Slowly the images become more comprehensible, until he can pick up the gist of a message from the colors and shapes next to it. For precise facts or statistics, text is still necessary, but it turns out that most of his queries are about stories: What’s in the news today? What happened in the latest episode of the show everyone’s watching? What did we talk about last time we met? He can get a summary of a narrative in half a dozen images: not just the bare facts but the whole arc of rising tension and emotional release. After a month he rarely needs to read any text.

That last link goes to Kurt Vonnegut on the 8 “shapes” of stories. The story is that Vonnegut wrote a master’s thesis on the shapes of stories that he submitted to the anthropology department at the University of Chicago, which rejected it. Here's a YouTube video of him talking about it; below is an infographic from that article: 

That said, Richard's Vonnegut-inspired fictional tech is about communicating narratives efficiently, not precise facts or statistics. For that, Gwern's On the Existence of Powerful Natural Languages persuaded me that you can't really have powerful general-purpose conlangs that boost cognition across a wide variety of domains.

(You mention Mandarin having compact grammar but in the table you grade it a ❌ at compact grammar.)

D'oh, you're obviously right, thanks! 

This remark at 16:10 by Dwarkesh Patel on his most recent podcast interview AMA: Career Advice Given AGI, How I Research ft. Sholto & Trenton was pretty funny: 

... big guests just don't really matter that much if you just look at what are the most popular episodes, or what in the long run helps a podcast grow. By far my most popular guest is Sarah Paine, and she, before I interviewed her, was just a scholar who was not publicly well-known at all, and I just found her books quite interesting—so my most popular guests are Sarah Paine and then Sarah Paine, Sarah Paine, Sarah Paine because I have electric chairs(?) a lecture series with her. And by the way, from a viewer-a-minute adjusted basis, I host the Sarah Paine podcast where I occasionally talk about AI.

(After Sarah Paine comes geneticist David Reich, then Satya Nadella and Mark Zuckerberg, "then [Sholto & Trenton] or Leopold (Aschenbrenner) or something, then you get to the lab CEOs or something")