I think the article undersells the problems of ChatGPT's hallucinations. One example from the article where ChatGPT is said to win is a recipe for risotto. However, I wouldn't follow a risotto recipe for ChatGPT just because I can't be confident it hasn't hallucinated some portion of the recipe but would happily follow one from Google, even if the format is a bit more annoying. Same issue with calculating load bearing capacity for a beam only more serious!

Having said that, it does seem like there are definitely specific areas where ChatGPT will be more useful. Coding is a good example as verifying the code is usually straightforward and/or would need to be done anyway. In many cases ChatGPT for an overview followed by Google for more detail/verification is probably a good way to go - I think this would be a good idea for the load bearing wall example.

1. It would be helpful to compare Google to ChatGPT on the actual distribution of queries to Google. I'd guess only a small % of queries to Google are about algorithms (e.g. doubly-linked lists question). IIRC many queries are not really searches but just looking for specific URLs (e.g. people search "facebook" or "wordle"). Also very common is naural-language translation (ChatGPT can't translate whole websites right now). Finally, many searches are for local information that ChatGPT can't offer at all right now ("weather", "food near me"). Programmers and other power users are probably have different usage distributions than the average user.

2. That said, I think less internet-savvy people could find web-enabled ChatGPT better for many queries because having a back-and-forth conversation with ChatGPT about some issue (e.g. tech support, medical, how to cook something) is easier than doing a search, opening a few tabs with the results and skimming them, modifying the search if the results weren't great (e.g. adding "reddit"), collating information from multiple pages, etc.

have you tried metaphor.systems? personally I find that neither chatgpt or google can hold a candle to metaphor for the queries metaphor works well on. let me know if I've mentioned it too many times and should shut up yet, but I've been bringing it up at every opportunity because no other ai system has as much of a positive impact on my daily workflow. I've found so many other awesome tools with it.

Reading the article, and painting it with my own experiences with Google failing me, it looks like Google is best when you want to understand a specific, named (possibly niche) object, but that chat GPT potentially has an advantage when you want to understand two things with a specific relationship, or if you can't name the specific object you want but can only describe it in relation to other objects.

An example from my own life. I few years ago I wanted to know if it was possible to put a latex table in a footnote. I still don't know if it is possible, because Google buried me in websties telling me how to attach a footnote to a table - a completely different formatting problem that shares the same two key words. I suspect chat GPT would be good for this.

One major factor is that Google serves more than a hundred thousand searches per second. A reasonably detailed response by ChatGPT (comparable to reading a top search result) is on the order of a thousand inferences, and so scaling to a Google replacement would require on the order of a hundred million inferences per second. Given the size of the model, I'm not sure that even Google would have enough compute to support that rate.

There is also the hallucination problem, as well as the fact that ChatGPT can't actually link to anything on the Internet, it can only tell you something about it.