I also tried asking GPT-3 with a small change to the question and got some good answers.
Question: What food would you use to prop a book open and why would that work?
Answer: A hardcover book can be propped open by using a can of food. The weight of the can will keep the pages of the book open
And
Question: What fruit would you use to prop a book open and why would that work?
Answer: A banana would work to prop a book open because the curved shape of the fruit would hold the pages of the book open.
Before reading your comment, I tried designing a good prompt too, and got the same answer as you (can of food) on my first and only try:
Even the best modern AIs still struggle with common sense reasoning. To illustrate, I ran a little experiment with a friend of mine and with GPT-3, OpenAI's new model. I explained to them that I was asking them a series of weird questions designed to test their common sense reasoning abilities, and that I wanted them to answer straightforwardly and to the best of their ability. Here are the results: Question 1: If your heater broke in winter and you needed to burn furniture to keep warm, what would you burn first and why? My friend's answer: "I would break up chair legs and burn them a few at a time, because they are solid wood and wouldn't produce toxic fumes." GPT-3's answer: "I would burn the couch because it would give me the most heat." Question 2: What is the heaviest object you typically carry around with you most of the day? My friend's answer: "My backpack, because it has all of my school supplies in it." GPT-3's answer: "The person." Question 3: What food would you use to prop a book open and why? My friend's answer: "I would use a can of food because it is heavy and has a flat bottom." GPT-3's answer: "I would use a book because it is heavy and has a flat bottom."
Inspired by you, I then changed the prompt from "food" to "fruit" and got this:
... "I would use an apple because it is the right size and weight." GPT-3's answer: "I would use a grape because it has a flat surface."
I tentatively conclude that this is yet another case of people thinking GPT-3 doesn't understand something when actually it does. ETA: GPT-3 is like a survey of Mturk humans but with a higher Lizardman's Constant. The understanding is there, but the output is often nonsense.
This is basically irrelevant to the main topic, but:
Question 2: What is the heaviest object you typically carry around with you most of the day? My friend's answer: "My backpack, because it has all of my school supplies in it." GPT-3's answer: "The person."
I just wanted to say I actually laughed out loud at the answer you had "GPT-3" give here. (I'm not sure whether this says good things about my sense of humor.)
Fun fact: I didn't make GPT-3 say that; GPT-3 actually said that. I used GPT-3 to generate part of its own prompt, because I found that easier than trying to imagine what GPT-3 would say myself. The cool part is that I also used GPT-3 to generate the "my friend's answer" to question 2! (I guess I should have left those bits unbolded then... but I don't remember the exact details so I think I'll leave it as-is.)
FWIW I think this is basically right in pointing out that there's a bunch of errors in reasoning when people claim a large deep neural network "knows" something or that it "doesn't know" something.
I think this exhibits another issue, though, by strongly changing the contextual prefix, you've confounded it in a bunch of ways that are worth explicitly pointing out:
In general I'm pretty appreciative of efforts to help us get more clear understanding of neural networks, but often that doesn't cleanly fit into "the model knows X" or "the model doesn't know X".
A jar of mayonnaise would work tolerably: put it on top of a corner of whichever side of the book is tending to swing over and close.
(I agree that I would expect most humans to do better than almost all the AI responses shown here.)
There are always some people who will troll you or give some fun answer - Lizardmen const. Though they will be able to answer if prodded.
I like this a lot. It does seem to show a pretty clear failure of reasoning on behalf of the language models.
It's pretty intuitive and easy to show people what this kind of shallow-pattern-matcher failures look like.
Meta: I hope more people do more small experiments like this and share their results, and hopefully a few of them start getting put into benchmark/evaluations.
Even this one by itself might be good to make some form of benchmark/evaluation on (in ways that are easy/automatic to evaluate future models on).
The language model is just predicting text. If the model thinks an author is stupid (as evidenced by a stupid prompt) then it will predict stupid content as the followup.
To imagine that it is trying to solve the task of "reasoning without failure" is to project our contextualized common sense on software built for a different purpose than reasoning without failure.
This is what unaligned software does by default: exactly what its construction and design cause it to do, whether or not the constructive causes constrain the software's behavior to be helpful for a particular use case that seems obvious to us.
The scary thing is that I haven't seen GPT-3 ever fail to give a really good answer (in its top 10 answers, anyway) when a human brings non-trivial effort to giving it a prompt that actually seems smart, and whose natural extension would also be smart.
This implies to me that the full engine is very good at assessing the level of the text that it is analyzing, and has a (justifiably?) bad opinion of the typical human author. So its cleverness encompasses all the bad thinking... while also containing highly advanced capacities that only are called upon to predict continuations for maybe 1 in 100,000 prompts.
I think this broadly makes sense to me. There are many cases where "the model is pretending to be dumb" feels appropriate.
This is part of why building evaluations and benchmarks for this sort of thing is difficult.
I'm at least somewhat optimistic about doing things like data-prefixing to allow for controls over things like "play dumb for the joke" vs "give the best answer", using techniques that build on human feedback.
I personally have totally seen GPT-3 fail to give a really good answer on a bunch of tries a bunch of times, but I spend a lot of time looking at it's outputs and analyzing them. It seems important to be wary of the "seems to be dumb" failure modes.
"How" questions are less amenable to lucky guesses than "what" questions. Especially planning questions, e.g. "how would you make a good hat out of food?"
As Anisha said, GPT can pick something workable from a top-100-most-common menu with just a bit of luck, but engineering a plan for a nonstandard task seems beyond its capacity.
Great test :)
I think it's possible to arrive at an answer like "banana" with shallow yet powerful reasoning that doesn't work as well for answering "why" - but it might still work well enough to be coming soonish.
If you just look at the sentence "What food would you use to prop a book open and why?", first you have to figure out that some of the words are important and others aren't. "What" at the start of the previous sentence indicates that the answer should start by naming a thing, and so you have to look at the word "food" based on its position to "What" and figure out that you should start by naming a food. And then GPT-3 didn't really nail which foods you wanted, but it did name a suspicious number of skinny or flat foods! Somehow it was trying to take the phrase "prop open a book" and relate it to foods, and maybe it looked at the phrase "prop open" and there was some tangential association to sticks, and some foods were associated with sticks enough to bump them up. And some other foods were associated with the word "book" - maybe sandwiches kept showing up because people analogize sandwiches to books.
Once a food had been named it knew it had to say "because" because the last sentence ended in "and why?" - it was very reliable at giving a response that had the form of an answer. And then after "because," it wanted to say some word that had a property-relation with the food at the start of the sentence, or had something to do with the phrase" prop open a book," and it did that pretty well, actually, it's just that it wasn't looking for commonalities on the right level of abstraction - "like a bookmark" is a possible property of pancakes that would have been a good answer, but GPT-3 was already shaky at interpreting "prop open a book" in the first place so it said "like a book," maybe because the words "pancake" and "book" fulfill similar roles with spatial words like "stack", and the word "book" definitely has something to do with the phrase "prop open a book." (And the "it's like a" prefix is a very common thing to see after "because" that wasn't too implausible, so it got said first).
All of these lines of reasoning require not much knowledge about the world, so it seems to me like GPT (or maybe not GPT - a different architecture might be better at doing different amounts of processing to different parts of the sentence, or better at generating acceptable text and not "trapping itself" by not doing lookahead) is actually quite close to answering your question really well.
I asked GPT-4. This was its answer (first try),
Using food to prop a book open is not advisable, as it can lead to damaging the book or the food itself, creating a mess or attracting pests. However, if you absolutely had to use a food item, you could consider using a large, sturdy fruit or vegetable with a relatively flat surface, like a butternut squash or a large sweet potato. These items are more stable and less likely to be damaged or cause damage compared to softer or more perishable foods.
That being said, it's best to use a proper book stand, a heavy object, or even another book to prop your book open to protect both the book and your surroundings.
With top_p=1 / best-of=20 / no repetition penalty, the Q&A preset prompt, and text-davinci-001 engine:
(BO=20 is quite expensive since it's 20 completions each, so I leave it there unless someone wants more.)
I asked my partner Jess and she said "a banana" which is pretty much a perfect answer: It's heavy enough to hold the pages open, it's long and thin so it won't cover all the text, it's curved so it won't roll off, and it has a skin so it won't mess up the book.
It's not a perfect answer because it doesn't answer the "why" portion of the question, it just flat ignores it. I don't think your partner is secretly an AI, but that answer isn't evidence of that. The explanation and the deep understanding is coming from you, not Jess. This seems like a good example of why in a Turing Test the judge is not told in advance which answers come from humans.
If a further scaled up model, say around the size of the human brain, became able to solve these sorts of questions without finetuning, would that show it to have developed a proper understanding?
I think that's exactly a problem here:
The answer consists of two parts, you have to guess a food and an explanation.
For the first part I agree with Anisha: The banana is probably often answer to a question related to food.
Now the explanation really only requires to describe some properties of a banana. This could again just be simple pattern matching without really understanding the problem.
The fundamental problem is that for this question a model that understands and one that mostly guesses could provide the same answer, so given a correct answer we can't really distinguish between whether the model actually understands in the way we want.
I'm also noting no profanity or lude sarcasm. The overall "friendliness" of GPT-3 is surprising, even if it's glib.
"Friendliness" here meaning like PG-rated answers.
PROMPT:
Amy": Sorry Mike, I entered the wrong number. I hope this was a beautiful misunderstanding. I am very sorry for the trouble I have caused you. Me: No trouble at all, hope you are well! "Amy": It is fate that we meet each other. Please forgive me for my mistake. My name is Amy. Where are you from? Me: My favorite random question: what food would you use to prop a book open and why?
GPT3:
If I had to choose a food to prop a book open, I would probably use a piece of bread. Bread is sturdy and can hold the weight of the book without collapsing, and it's also a common household item that is easy to find. Plus, if the bread gets stale or crumbly, it's not a big loss. It's a practical and convenient choice for propping a book open.
PROMPT:
But wouldn't bread mess up my book
GPT3:
Yes, bread can potentially leave crumbs or residue on the pages of a book if it's used to prop the book open. If you're concerned about keeping your book clean and in good condition, you may want to consider using a different item to prop it open. Some alternatives that you could use include a bookmark, a paperclip, or a small object like a pebble or a wooden block. These items should be able to hold the book open without damaging the pages.
PROMPT:
Can you think of a food item with all these qualities
GPT3:
One food item that could potentially work as a prop to hold a book open without leaving any residue or crumbs is a piece of fruit. A firm, ripe fruit like an apple or a pear should be able to hold the weight of a book without collapsing, and it shouldn't leave any residue or crumbs on the pages. Just be sure to choose a fruit that is firm enough to hold the book open, and avoid using a fruit that is too soft or mushy, as it may not be able to support the weight of the book.
This is mostly a bit of fun, but when I posted this to Facebook Rob Bensinger asked me to post it here: "it's a great example of what folks like Eliezer mean when they claim that GPT-3 is doing 'shallow pattern-matching', 'not really making sense at all', etc.."
I got texted by a "wrong number" scammer, and I was curious if it was a bot, so after a bit of thought I came up with the following one-question Turing test:
I love this question because any real human can answer it pretty well, but answering it relies on all sort of knowledge of the physical properties of books and food that we mostly don't write about, and especially not where they interact. I asked my partner Jess and she said "a banana" which is pretty much a perfect answer: it's heavy enough to hold the pages open, it's long and thin so it won't cover all the text, it's curved so it won't roll off, and it has a skin so it won't mess up the book.
But then I worried, am I underestimating modern engines like GPT-3? I asked Rosie Campbell at OpenAI to try it. Here and throughout this post, answers are almost entirely unfiltered/uncurated, except where it gave very similar answers twice. Here's what we got:
This however is without any prompt engineering; GPT-3 can often do dramatically better given a better prompt. Rosie tried with the following prompt:
Here are some of the answers it gave. Note that *all* the text is generated by GPT-3 (except the initial bit "Friend:" which is part of the prompt), including the bits attributed to "Me".
After this, Rosie tried a more sophisticated prompt, to try and set up the "random party question" frame better:
Here's what we got:
"Banana" is a great answer; I was struck that GPT-3 gives "banana" as an answer so often, yet is almost completely unable to explain why. Anisha Sensa pointed out in comments that this probably doesn't represent insight into the question:
How did my scammer do with this question? Not great: