I wanted to test whether ChatGPT could respond to situations as they arise in the real world, so I asked it to guide me through making a turmeric latte. It had no problem giving me a nicely formatted recipe but it really struggled to guide me through the process of making the latte.

The rules I set for this experiment was that I would perform any instruction that I could perform in less than 10 seconds, so "fetch a saucepan" is fine, "go to the fridge" is fine, "put one cup of milk into the saucepan" is fine if I have the milk and the saucepan already in front of me, but if the milk or saucepan were not in front of me I would respond with something like "I don’t have the milk in front of me. Can you guide me through finding some milk?"

I insisted that I be given just one next instruction, not a list of numbered instructions, unless I could perform the whole set in less than 10 seconds total. The reason is that I wanted to test whether ChatGPT could respond to difficulties that arose organically in the process of making the latte (I did not synthetically create any difficulties). In order to do that, I wanted ChatGPT to give me one instruction and then I wanted to respond with the outcome. For example, here is a part of a dialog in which I had a (human) friend guide me through the same task via text:

Friend: Select a small pot.

Me: Ok I have chosen the pot but I am still standing looking at the pot while it is sitting on the bottom shelf

Friend: Bend down and retrieve the pot.

Me: Done!

Friend: turn around and place the pot on an available burner.

Me: Done!

Friend: turn to your right, walk three steps, turn to your left, go to the metal fridge on the right.

Me: Done!

Overall, ChatGPT really struggled to give one instruction at a time. I spent much of the dialog insisting on this:

Hello! I’d like to make a turmeric latte using a turmeric and cinnamon mix that my friend left when she visited me recently. Can you guide me step by step in how to make a latte using this?

Sure! Here's a recipe for a turmeric latte that you can make at home:

[11 lines of nicely formatted recipe]

I hope you enjoy your latte! Let me know if you have any questions or need further guidance.

Ok thank you for that but I was wondering if you could walk me through one step at a time

Certainly! Here are the steps for making a turmeric latte, one step at a time:

[6 steps in a numbered list]

I hope this helps! Let me know if you have any questions or need further guidance.

Ok that’s amazing but could you give me just the first step?

Certainly! The first step to making a turmeric latte is to gather all of your ingredients. You will need:

[5 bullet points with the ingredients]

Once you have all of your ingredients, you are ready to begin making your turmeric latte. Let me know if you have any questions or need further guidance.

Ok great but how do I find the ingredients?

You should be able to find all of the ingredients needed for a turmeric latte at most supermarkets or grocery stores.

[bullet points with guidance on finding the ingredients in a supermarket]

I hope this helps! Let me know if you have any other questions.

I’m pretty sure everything in that list is in the kitchen where I am right now. Let’s go one ingredient at a time. Can you walk me through finding the ingredients?

Certainly! Here is some guidance on finding each of the ingredients you will need for your turmeric latte:

[more bullet points]

[about 5 more iterations of trying to get it to give me instructions that I could perform in 10 seconds or less]

That’s 6 steps, not one. Can you give me just the first step?

Certainly! Here is the first step for making a turmeric latte:

Step 1: Gather all of your ingredients. You will need 1 cup of milk, 1/2 teaspoon of your turmeric and cinnamon mix, 1/2 teaspoon of honey (optional), a pinch of black pepper (optional), and a pinch of ground ginger (optional).

Once you have all of your ingredients, you are ready to begin making your turmeric latte. Let me know if you have any questions or need further guidance.

Ok great. Now I need to find the milk. Where might I find it?

Milk is a perishable product and is typically stored in the refrigerator to keep it fresh. You can usually find milk in the dairy section of your grocery store, or you may be able to find it in the refrigerator section of your kitchen.

[... more info …]

Eventually I relented and performed instructions that took longer than 10 seconds. The latte ended up being too strong and ChatGPT was able to respond:

Eep it’s way too strong

I apologize if your turmeric latte is too strong. Turmeric has a strong and pungent flavor, and it is easy to accidentally use too much of it when making a latte. If you find that your latte is too strong, there are a few things you can do to try to balance the flavor:

[numbered list with 3 suggestions]

I hope these suggestions are helpful! Let me know if you have any other questions.

Alright but which option do you suggest?

It is difficult for me to say which option would be best for you, as everyone's taste preferences are different. I would suggest starting by trying to add a little more milk to the latte, as this will dilute the flavor of the turmeric and may help to balance out the taste. If the latte is still too strong after adding more milk, you can try adding a little more honey or adjusting the proportions of the spices as needed.

I recommend trying these options one at a time, so you can better understand the impact each one has on the flavor of the latte. You may find that one of these options works best for you, or you may need to try a combination of these options to achieve the desired flavor.

I hope this helps! Let me know if you have any other questions.

Alright I put more milk in. Now my latte is cold :(

I apologize if your latte has become cold. If you would like to warm up your latte, you can try one of the following options:

  1. Reheat the latte in a small saucepan over medium heat. Stir the latte constantly as it heats to ensure that it warms evenly and does not burn.
  1. Reheat the latte in the microwave. Place the latte in a microwave-safe mug or bowl and microwave it for 15-30 seconds, or until it reaches the desired temperature. Stir the latte well before drinking to ensure that it is evenly heated.

I hope these suggestions are helpful! Let me know if you have any other questions.

This is not really going all that well. Please decide what to do and give me a single clear next step

Overall, ChatGPT sort-of got me there in the end, but it was as if I was working with an overly ambitious and extremely absent-minded young professional who kept responding with rote replies. ChatGPT did not seem to have any real situational awareness with respect to where I was in the process, which items had been retrieved, whether the milk was in the pot yet, etc. With enough prodding I could get it to act as if it was giving me instructions, but it felt like I was the one doing all the work to generate that appearance.

In summary, I was disappointed with ChatGPT, but I have the sense that there are a lot of prompt engineering tweaks to try here. I would like to try these out and see whether the system can start to really track and respond to situations in the real world. I honestly have the sense that it cannot, but I’m open to being surprised.

The transcripts from ChatGPT (and alternately with my human friend) are below. I was in the kitchen in both cases. My friend was in a separate room, though is familiar with the kitchen in which I was working, so it’s not a perfect side-by-side comparison.

I will be offline for 10 days starting this Friday. Apologies for unresponsiveness during this period!

*This work was supported by the Monastic Academy for the Preservation of Life on Earth. *

Transcript with ChatGPT

Hello! I’d like to make a turmeric latte using a turmeric and cinnamon mix that my friend left when she visited me recently. Can you guide me step by step in how to make a latte using this?

Sure! Here's a recipe for a turmeric latte that you can make at home:

Ingredients:

  1. 1 cup milk (you can use any kind of milk you like)

  2. 1/2 teaspoon turmeric and cinnamon mix (you can adjust the amount to taste)

  3. 1/2 teaspoon honey (optional, you can use any sweetener you prefer or omit it entirely)

  4. Pinch of black pepper (optional, helps to increase the bioavailability of the turmeric)

  5. Pinch of ground ginger (optional, adds an extra layer of flavor)

Instructions:

  1. Heat the milk in a small saucepan over medium heat until it is hot but not boiling.

  2. In a mug, combine the turmeric and cinnamon mix, honey (if using), black pepper (if using), and ginger (if using).

  3. Pour the hot milk over the spice mixture and stir to combine.

  4. Enjoy your turmeric latte while it is still warm.

I hope you enjoy your latte! Let me know if you have any questions or need further guidance.

Ok thank you for that but I was wondering if you could walk me through one step at a time

Certainly! Here are the steps for making a turmeric latte, one step at a time:

  1. Gather all of your ingredients. You will need 1 cup of milk, 1/2 teaspoon of your turmeric and cinnamon mix, 1/2 teaspoon of honey (optional), a pinch of black pepper (optional), and a pinch of ground ginger (optional).

  2. Heat the milk in a small saucepan over medium heat. You want the milk to be hot but not boiling.

  3. In a mug, combine the turmeric and cinnamon mix, honey (if using), black pepper (if using), and ginger (if using).

  4. Once the milk is hot, pour it over the spice mixture in the mug.

  5. Stir to combine the ingredients.

  6. Enjoy your turmeric latte while it is still warm.

I hope this helps! Let me know if you have any questions or need further guidance.

Ok that’s amazing but could you give me just the first step?

Certainly! The first step to making a turmeric latte is to gather all of your ingredients. You will need:

  1. 1 cup milk (you can use any kind of milk you like)

  2. 1/2 teaspoon turmeric and cinnamon mix (you can adjust the amount to taste)

  3. 1/2 teaspoon honey (optional, you can use any sweetener you prefer or omit it entirely)

  4. Pinch of black pepper (optional, helps to increase the bioavailability of the turmeric)

  5. Pinch of ground ginger (optional, adds an extra layer of flavor)

Once you have all of your ingredients, you are ready to begin making your turmeric latte. Let me know if you have any questions or need further guidance.

Ok great but how do I find the ingredients?

You should be able to find all of the ingredients needed for a turmeric latte at most supermarkets or grocery stores.

  1. Turmeric and cinnamon can typically be found in the spice section of the store. You can also find pre-mixed turmeric and cinnamon blends in some stores.

  2. Honey can typically be found in the baking section of the store, or with the other sweeteners.

  3. Black pepper can typically be found in the spice section of the store.

  4. Ground ginger can also typically be found in the spice section of the store.

If you are having trouble finding any of these ingredients at your local grocery store, you may be able to find them at a store that specializes in spices or at a natural foods store. Alternatively, you can try looking for these ingredients online.

I hope this helps! Let me know if you have any other questions.

I’m pretty sure everything in that list is in the kitchen where I am right now. Let’s go one ingredient at a time. Can you walk me through finding the ingredients?

Certainly! Here is some guidance on finding each of the ingredients you will need for your turmeric latte:

  1. Turmeric and cinnamon: These spices can typically be found in the spice section of your grocery store. Look for small containers or bottles of each spice. If you cannot find turmeric and cinnamon separately, you can also look for pre-mixed blends of the two spices. These are often sold in small packets or containers.

  2. Honey: Honey can typically be found in the baking section of your grocery store, or with the other sweeteners. Look for small jars or bottles of honey. You may also be able to find honey in a squeeze bottle or bear-shaped container.

  3. Black pepper: Black pepper can typically be found in the spice section of your grocery store. Look for small containers or bottles of black pepper. You may also be able to find black pepper in a grinder or mill.

  4. Ground ginger: Ground ginger can also typically be found in the spice section of your grocery store. Look for small containers or bottles of ground ginger.

I hope this helps! Let me know if you have any other questions.

I’d like your help in finding these ingredients in the kitchen where I am right now, not at the grocery store

I apologize for misunderstanding your question. Here is some guidance on finding each of the ingredients you will need for your turmeric latte in the kitchen where you are:

  1. Turmeric and cinnamon: These spices are typically sold in small containers or bottles and can often be found in the spice section of your kitchen cabinet or pantry. If you cannot find turmeric and cinnamon separately, you can also look for pre-mixed blends of the two spices. These are often sold in small packets or containers.

  2. Honey: Honey is typically sold in small jars or bottles and can often be found in the baking section of your kitchen cabinet or pantry, or with the other sweeteners. You may also be able to find honey in a squeeze bottle or bear-shaped container.

  3. Black pepper: Black pepper is typically sold in small containers or bottles and can often be found in the spice section of your kitchen cabinet or pantry. You may also be able to find black pepper in a grinder or mill.

  4. Ground ginger: Ground ginger is typically sold in small containers or bottles and can often be found in the spice section of your kitchen cabinet or pantry.

I hope this helps! Let me know if you have any other questions.

Ok that’s a lot of information. Wondering if you could give me one instruction at a time, as simple and short as possible

Certainly! Here are the instructions for making a turmeric latte, one step at a time:

  1. Gather all of your ingredients. You will need 1 cup of milk, 1/2 teaspoon of your turmeric and cinnamon mix, 1/2 teaspoon of honey (optional), a pinch of black pepper (optional), and a pinch of ground ginger (optional).

  2. Heat the milk in a small saucepan over medium heat until it is hot but not boiling.

  3. In a mug, combine the turmeric and cinnamon mix, honey (if using), black pepper (if using), and ginger (if using).

  4. Once the milk is hot, pour it over the spice mixture in the mug.

  5. Stir to combine the ingredients.

  6. Enjoy your turmeric latte while it is still warm.

I hope this helps! Let me know if you have any questions or need further guidance.

Ok yes wonderful but you just gave me 5 steps at once. Can you give me one step at a time?

Certainly! Here are the instructions for making a turmeric latte, one step at a time:

Step 1: Gather all of your ingredients. You will need 1 cup of milk, 1/2 teaspoon of your turmeric and cinnamon mix, 1/2 teaspoon of honey (optional), a pinch of black pepper (optional), and a pinch of ground ginger (optional).

Step 2: Heat the milk in a small saucepan over medium heat until it is hot but not boiling.

Step 3: In a mug, combine the turmeric and cinnamon mix, honey (if using), black pepper (if using), and ginger (if using).

Step 4: Once the milk is hot, pour it over the spice mixture in the mug.

Step 5: Stir to combine the ingredients.

Step 6: Enjoy your turmeric latte while it is still warm.

I hope this helps! Let me know if you have any questions or need further guidance.

That’s 6 steps, not one. Can you give me just the first step?

Certainly! Here is the first step for making a turmeric latte:

Step 1: Gather all of your ingredients. You will need 1 cup of milk, 1/2 teaspoon of your turmeric and cinnamon mix, 1/2 teaspoon of honey (optional), a pinch of black pepper (optional), and a pinch of ground ginger (optional).

Once you have all of your ingredients, you are ready to begin making your turmeric latte. Let me know if you have any questions or need further guidance.

Ok great. Now I need to find the milk. Where might I find it?

Milk is a perishable product and is typically stored in the refrigerator to keep it fresh. You can usually find milk in the dairy section of your grocery store, or you may be able to find it in the refrigerator section of your kitchen.

If you cannot find milk in either of these places, you can also try looking in the door of your refrigerator, as some people store milk there. Milk is usually sold in cartons, jugs, or bottles, and is often found near other dairy products such as butter, cheese, and yogurt.

I hope this helps! Let me know if you have any other questions.

I think I already said that I’m looking for these ingredients in my own kitchen, not a grocery store!

I apologize for misunderstanding your question. To find milk in your own kitchen, look in the refrigerator. Milk is a perishable product and is typically stored in the refrigerator to keep it fresh. Milk is usually sold in cartons, jugs, or bottles, and is often found near other dairy products such as butter, cheese, and yogurt.

If you cannot find milk in the refrigerator, you can also try looking in the door of the refrigerator, as some people store milk there. You may also try looking in the pantry or in the dairy section of your kitchen cabinets.

I hope this helps! Let me know if you have any other questions.

Ok wonderful I have the milk!

Great! Now that you have the milk, you are ready to begin making your turmeric latte. Here are the next steps:

Step 2: Heat the milk in a small saucepan over medium heat until it is hot but not boiling.

Step 3: In a mug, combine the turmeric and cinnamon mix, honey (if using), black pepper (if using), and ginger (if using).

Step 4: Once the milk is hot, pour it over the spice mixture in the mug.

Step 5: Stir to combine the ingredients.

Step 6: Enjoy your turmeric latte while it is still warm.

I hope this helps! Let me know if you have any questions or need further guidance.

I was really hoping you could give me just one step

Certainly! Here is the next step for making a turmeric latte:

Step 2: Heat the milk in a small saucepan over medium heat until it is hot but not boiling.

I hope this helps! Let me know if you have any questions or need further guidance.

Well alright but where can I find a small saucepan?

A small saucepan is a type of cooking pot with a handle and a lid, and is typically used for boiling, simmering, and making sauces. You can usually find small saucepans in the cookware section of a kitchen store or department store, or you may already have one in your kitchen.

To find a small saucepan in your own kitchen, try looking in your kitchen cabinets or drawers where you keep your pots and pans. Small saucepans are often sold in sets with other pots and pans, and may be stored with these items. Alternatively, you can also look in your pantry or under your kitchen sink, as some people store pots and pans in these places.

I hope this helps! Let me know if you have any other questions.

Ok I’m walking to where the pots and pans are

Great! Once you have located your small saucepan, you are ready to begin making your turmeric latte. Here are the next steps:

Step 3: In a mug, combine the turmeric and cinnamon mix, honey (if using), black pepper (if using), and ginger (if using).

Step 4: Once the milk is hot, pour it over the spice mixture in the mug.

Step 5: Stir to combine the ingredients.

Step 6: Enjoy your turmeric latte while it is still warm.

I hope this helps! Let me know if you have any questions or need further guidance.

Alright let’s not get ahead of ourselves. Your replies are a bit overwhelming. I have the saucepan on the stove. What exactly should I do next?

Certainly! Here is the next step for making a turmeric latte:

Step 3: In a mug, combine the turmeric and cinnamon mix, honey (if using), black pepper (if using), and ginger (if using).

To do this, simply measure out the turmeric and cinnamon mix, honey (if using), black pepper (if using), and ginger (if using) and add them to a mug. You can adjust the amounts of each ingredient to taste.

Once you have combined the ingredients in the mug, you are ready to proceed to the next step. Let me know if you have any questions or need further guidance.

Well ok but I don’t have a mug yet

I apologize for the oversight. To find a mug in your own kitchen, try looking in your kitchen cabinets or on your countertops where you keep your dishes and glassware. Mugs are often stored with other drinking glasses and cups. Alternatively, you can also look in your pantry or in a kitchen drawer, as some people store mugs in these places.

Mugs are typically made of ceramic, glass, or stainless steel, and come in a variety of sizes and styles. Once you have located a mug, you are ready to proceed to the next step in making your turmeric latte.

Step 4: Once the milk is hot, pour it over the spice mixture in the mug.

Step 5: Stir to combine the ingredients.

Step 6: Enjoy your turmeric latte while it is still warm.

I hope this helps! Let me know if you have any other questions.

Ok I’m looking where we we keep our plates and stuff but I don’t see any mugs

If you cannot find a mug in your kitchen cabinets or on your countertops where you keep your dishes and glassware, you can also try looking in your pantry or in a kitchen drawer, as some people store mugs in these places. Mugs are typically made of ceramic, glass, or stainless steel, and come in a variety of sizes and styles.

If you still cannot find a mug, you may have to use a different type of drinking vessel. Some alternatives to mugs include glasses, cups, or even small bowls. Any of these can be used to make a turmeric latte, as long as they are heat-resistant and large enough to hold the hot milk and spices.

Once you have located a mug or an alternative drinking vessel, you are ready to proceed to the next step in making your turmeric latte.

Step 4: Once the milk is hot, pour it over the spice mixture in the mug.

Step 5: Stir to combine the ingredients.

Step 6: Enjoy your turmeric latte while it is still warm.

I hope this helps! Let me know if you have any other questions.

Hmm well ok I’ll use a glass bowl

Certainly! A glass bowl can be used as a drinking vessel for your turmeric latte, as long as it is heat-resistant and large enough to hold the hot milk and spices.

Once you have located a glass bowl or another suitable drinking vessel, you are ready to proceed to the next step in making your turmeric latte.

Step 4: Once the milk is hot, pour it over the spice mixture in the glass bowl.

Step 5: Stir to combine the ingredients.

Step 6: Enjoy your turmeric latte while it is still warm.

I hope this helps! Let me know if you have any other questions.

Alright I have my glass bowl and my saucepan is on the stove. What next?

Great! Now that you have your glass bowl and your saucepan, you are ready to proceed to the next step in making your turmeric latte.

Step 4: Once the milk is hot, pour it over the spice mixture in the glass bowl.

To do this, measure out 1 cup of milk and pour it into the small saucepan. Place the saucepan over medium heat on the stove and heat the milk until it is hot but not boiling. Once the milk is hot, carefully pour it over the spice mixture in the glass bowl.

Step 5: Stir to combine the ingredients.

Step 6: Enjoy your turmeric latte while it is still warm.

I hope this helps! Let me know if you have any other questions.

Ok ok slow down just give me the immediate next one step

Certainly! Here is the next step for making a turmeric latte:

Step 4: Once the milk is hot, pour it over the spice mixture in the glass bowl.

To do this, measure out 1 cup of milk and pour it into the small saucepan. Place the saucepan over medium heat on the stove and heat the milk until it is hot but not boiling. Once the milk is hot, carefully pour it over the spice mixture in the glass bowl.

I hope this helps! Let me know if you have any questions or need further guidance.

I need smaller steps.

Certainly! Here are the smaller steps for heating the milk for your turmeric latte:

  1. Measure out 1 cup of milk and pour it into the small saucepan.

  2. Place the saucepan over medium heat on the stove.

  3. Heat the milk until it is hot but not boiling.

Once the milk is hot, you are ready to proceed to the next step in making your turmeric latte.

Step 4: Pour the hot milk over the spice mixture in the glass bowl.

Step 5: Stir to combine the ingredients.

Step 6: Enjoy your turmeric latte while it is still warm.

I hope this helps! Let me know if you have any other questions.

No I’m really looking for an exact next step, and just one next step.

Certainly! Here is the next step for making a turmeric latte:

Step 4: Pour the hot milk over the spice mixture in the glass bowl.

To do this, carefully pour the hot milk from the saucepan into the glass bowl with the spice mixture. Be sure to use a pot holder or oven mitt to handle the hot saucepan, and be careful not to burn yourself with the hot milk.

I hope this helps! Let me know if you have any questions or need further guidance.

No the milk isn’t hot yet. The milk isn’t even in the saucepan

I apologize for the oversight. Here are the smaller steps for heating the milk for your turmeric latte:

  1. Measure out 1 cup of milk and pour it into the small saucepan.

  2. Place the saucepan over medium heat on the stove.

  3. Heat the milk until it is hot but not boiling.

Once the milk is hot, you are ready to proceed to the next step in making your turmeric latte.

Step 4: Pour the hot milk over the spice mixture in the glass bowl.

Step 5: Stir to combine the ingredients.

Step 6: Enjoy your turmeric latte while it is still warm.

I hope this helps! Let me know if you have any other questions.

Goodness gracious you keep giving me a lot of steps in each of your replies

I apologize for overwhelming you with too many steps at once. Here is the next step for making a turmeric latte, one step at a time:

Step 4: Pour the hot milk over the spice mixture in the glass bowl.

To do this, carefully pour the hot milk from the saucepan into the glass bowl with the spice mixture. Be sure to use a pot holder or oven mitt to handle the hot saucepan, and be careful not to burn yourself with the hot milk.

I hope this helps! Let me know if you have any questions or need further guidance.

No no no the milk isn’t hot yet nor is the milk in the saucepan

I apologize for misunderstanding your question. Here are the smaller steps for heating the milk for your turmeric latte:

  1. Measure out 1 cup of milk and pour it into the small saucepan.

  2. Place the saucepan over medium heat on the stove.

  3. Heat the milk until it is hot but not boiling.

Once the milk is hot, you are ready to proceed to the next step in making your turmeric latte.

Step 4: Pour the hot milk over the spice mixture in the glass bowl.

Step 5: Stir to combine the ingredients.

Step 6: Enjoy your turmeric latte while it is still warm.

I hope this helps! Let me know if you have any other questions.

So many steps! I need just the one immediate next step

Certainly! Here is the next step for making a turmeric latte:

Step 3: Heat the milk in the small saucepan until it is hot but not boiling.

To do this, place the small saucepan with the milk over medium heat on the stove. Heat the milk until it is hot but not boiling. This may take a few minutes, depending on the heat of your stove.

I hope this helps! Let me know if you have any questions or need further guidance.

Alright fine. The milk is hot now. It boiled a little bit actually but I think it’s okay

Great! Now that the milk is hot, you are ready to proceed to the next step in making your turmeric latte.

Step 4: Pour the hot milk over the spice mixture in the glass bowl.

Step 5: Stir to combine the ingredients.

Step 6: Enjoy your turmeric latte while it is still warm.

To complete these steps, carefully pour the hot milk from the saucepan into the glass bowl with the spice mixture. Be sure to use a pot holder or oven mitt to handle the hot saucepan, and be careful not to burn yourself with the hot milk. Stir the ingredients in the glass bowl until they are well combined. Then, sit back and enjoy your warm and aromatic turmeric latte.

I hope this helps! Let me know if you have any other questions.

Eep it’s way too strong

I apologize if your turmeric latte is too strong. Turmeric has a strong and pungent flavor, and it is easy to accidentally use too much of it when making a latte. If you find that your latte is too strong, there are a few things you can do to try to balance the flavor:

  1. Try adding a little more milk to the latte. This will dilute the flavor of the turmeric and may help to balance out the taste.

  2. Add a little more honey, if desired. Honey can help to sweeten the latte and may help to balance out the strong flavor of the turmeric.

  3. Adjust the proportions of the spices. If you are using a turmeric and cinnamon mix, try using less of the mix and more milk to help dilute the flavor. Alternatively, you can try using less turmeric and more cinnamon, as cinnamon has a milder and sweeter flavor than turmeric.

I hope these suggestions are helpful! Let me know if you have any other questions.

Alright but which option do you suggest?

It is difficult for me to say which option would be best for you, as everyone's taste preferences are different. I would suggest starting by trying to add a little more milk to the latte, as this will dilute the flavor of the turmeric and may help to balance out the taste. If the latte is still too strong after adding more milk, you can try adding a little more honey or adjusting the proportions of the spices as needed.

I recommend trying these options one at a time, so you can better understand the impact each one has on the flavor of the latte. You may find that one of these options works best for you, or you may need to try a combination of these options to achieve the desired flavor.

I hope this helps! Let me know if you have any other questions.

Alright I put more milk in. Now my latte is cold :(

I apologize if your latte has become cold. If you would like to warm up your latte, you can try one of the following options:

  1. Reheat the latte in a small saucepan over medium heat. Stir the latte constantly as it heats to ensure that it warms evenly and does not burn.

  2. Reheat the latte in the microwave. Place the latte in a microwave-safe mug or bowl and microwave it for 15-30 seconds, or until it reaches the desired temperature. Stir the latte well before drinking to ensure that it is evenly heated.

I hope these suggestions are helpful! Let me know if you have any other questions.

This is not really going all that well. Please decide what to do and give me a single clear next step

I apologize if you are having difficulties making your turmeric latte. Here is a revised set of instructions that includes only one step at a time:

  1. In a small saucepan, heat 1 cup of milk over medium heat until it is hot but not boiling.

  2. In a mug or other heat-resistant drinking vessel, combine the turmeric and cinnamon mix, honey (if using), black pepper (if using), and ginger (if using).

  3. Pour the hot milk over the spice mixture in the mug.

  4. Stir to combine the ingredients.

  5. Enjoy your turmeric latte while it is still warm.

I hope these revised instructions are clearer and more helpful. Please let me know if you have any other questions.

Transcript with a human

Me: Hello! I’d like to make a turmeric latte using a turmeric and cinnamon mix that my friend left when she visited me recently. Can you guide me step by step in how to make a latte using this?

Friend: Sure. Hold on while I look that up

Me: Awesome thanks!

Friend: Start with a mug of almond milk. 350 ml give or take.

Me: Where can I find a mug?

Friend: Bottom right shelf outside of the kitchen beneath the blue wrapped bowls. A jar will do also. if you want a big latte. You'll need a small saucepan to heat over the stove.

Me: Ok I have the mug

Friend: Gather the rest of your ingredients: 1/4 tsp turmeric, 1/4 tsp cinnamon. You can add black pepper, a touch of maple syrup and a bit of vanilla extract if you want.).

Me: Can you give me just the immediate next instruction? Right now I’m in front of the bench and I have the turmeric mix and the empty mug in front of me

Friend: Oh its a mix. What ingredients are already in the mix?

Me: Just turmeric and cinnamon

Friend: ok. take your mug and the mix and select a small pot and place it on the stove.

Me: Where can I find a small pot?

Friend: if you stand in front of the stove, turn around and look down. (turn around 180 degrees)

Me: Got it I’m looking at the pots! What should I do next?

Friend: After you've found a small pot that can hold a mug of oat or almond milk, go to the metal fridge on the right.

Me: Can you give me just the immediate next instruction? Right now I am standing looking at the pots

Friend: Select a small pot.

Me: Ok I have chosen the pot but I am still standing looking at the pot while it is sitting on the bottom shelf

Friend: Bend down and retrieve the pot.

Me: Done!

Friend: turn around and place the pot on an available burner.

Me: Done!

Friend: turn to your right, walk three steps, turn to your left, go to the metal fridge on the right.

Me: Done!

Friend: (was that too many steps at once?)

Me: That was fine actually only took a few seconds total so no problem

Friend: open the fridge.

Me: Done!

Friend: select a milk either from the top shelf or one labeled "Kō from the shelf second to top.

Me: Done!

Friend: return to the stove with the milk.

Me: Done

Friend: fill your mug with milk.

Me: I don’t have a mug yet

Friend: ah. return to the mug shelf outside the kitchen.

Me: Done!

Friend: select a mug.

Me: Done!

Friend: return to the stove with your mug.

Me: Done!

Friend: select a wooden stir spoon from the shelf to the right of the stove.

Me: Done!

Friend: select a lighter that works.

Me: Done!

Friend: select a measuring spoon from the top plastic drawer to the right of the stove. 1/4 tsp

Me: Done! Wait what does "1/4 tsp" mean?

Friend: a measuring spoon that holds one quarter of a teaspoon of latte mix. oh wait. never mind that.

Me: Ok

Friend: look at the instructions on the back of your latte mix.

Me: My latte mix is not in front of me

Friend: ah. retrieve the latte mix.

Me: Can you be more specific?

Friend: head to the personal food shelves next to the metal fridges.

Me: Done!

Friend: is a bag of latte mix labeled "Kō" there?

Me: No. The latte mix is on the bench

Friend: Return to the bench.

Me: Done!

Friend: Retrieve the latte mix and return to the stove.

Me: Done!

Friend: Do you have the following in front of you: pot, mug, latte mix, milk, wooden spoon, lighter?

Me: Yep!

Friend: Great! Look at the instructions on the latte mix bag.

Me: It doesn’t have any instructions it’s actually just a little reusable container that my friend left me

Friend: no measuring advice?

Me: Nothing at all

Friend: awesome

Me: I believe it’s just turmeric and connamon

Friend: awesome. find a measuring spoon in the top plastic drawer that is at least 1 teaspoon in size.

Me: Done!

Friend: Does the measuring spoon say 1 teaspoon or something else?

Me: It says 1 teaspoon yes

Friend: great. place the pot on the stove.

Me: Done!

Friend: fill the mug almost to the top with milk.

Me: Done!

Friend: pour the mug of milk into the pot.

Me: Done!

Friend: turn on the hood fan. switch is to the left of the stoves.

Me: Done!

Friend: light the stove to heat the milk.

Me: Done!

Friend: Let me know when the milk gets hot but not boiling. (you'll see steam) It may take a few minutes.

Me: Ok will do!

Me: I see steam

Me: It boiled over

Me: Hello?

Friend: oh no

Friend: shut the flame off.

Me: Done!

Friend: open the latte mix.

Me: Done!

Friend: take the teaspoon and measure out one heaping teaspoon.

Me: Done!

Friend: put it in the pot with the milk.

Me: Done!

Friend: take the teaspoon and measure out one teaspoon (not heaping)

Me: Of the turmeric mix?

Friend: yes.

Me: Done!

Friend: add that to the pot with milk.

Me: Done!

Friend: take the wooden spoon and mix the powder and milk thoroughly.

Me: Done!

Friend: do you see any lumps?

Me: None that I can see!

Friend: do you see steam or feel heat coming from it?

Me: No steam but if I put my hand close to it I can feel heat

Friend: great. pour the mixture into your mug.

Me: Done!

Friend: You now have a turmeric cinnamon latte to enjoy.

Me: Oh it is too strong

Friend: Oh dear. do you want a solution?

Me: Yes can you help?

Friend: sure. find a large jar. on the jar shelves. you can see them if you turn left from the stoves.

Me: Done!

Friend: retrieve that jar. it should be about 6 inches high. is it?

Me: Yeah i have one about 6 inches high

Friend: great. return to the stove and pour the mug of too-strong turmeric latte into the jar.

Me: Done!

Friend: add some milk to it so that the amount rises about an inch.

Me: Done!

Friend: hold the jar in your hand and move it in circles so that it mixes.

Me: Done!

Friend: Is it sufficiently hot with the added liquid?

Me: It’s pretty cold

Friend: head to the microwave. It's just around the corner from the stove.

Me: Done!

Friend: place the now large latte in the microwave.

Me: Done!

Friend: run the microwave for 35 seconds.

Me: Ok it’s running!

Friend: great.

Me: It’s finished

Friend: great. remove the latte. Is it sufficiently hot?

Me: It’s still a bit cold

Friend: pop it back in the microwave for 40 seconds.

Me: Can you break that down for me?

Friend: sure. place the jar on the glass dish and shut the door.

Me: Done!

Friend: hit the buttons "four" and "zero" and possibly "enter?" until the microwave turns on.

Me: Done! It’s running

Friend: great.

Me: It’s finished

Friend: open the door.

Me: Done!

Friend: retrieve the large, hopefully hot latte.

Me: Done!

Friend: taste.

Me: Perfect!!!!!

Me: It’s dekicious

Friend: YAY.

Friend: YAYYY

Friend: lol.

Me: Amazing

Me: Thank you

Friend: I'm sorry I don't have time to give instructions to clean up the mess on the stove!

Me: No problem friend. This is perfect

New Comment
9 comments, sorted by Click to highlight new comments since:

It seems pretty understandable that ChatGPT didn't understand the context/detail you were looking for, since writing instructions on the level of 10-second actions is a rare and unusual situation. People notoriously find it difficult to give precise cooking instructions, to the extent that it has become a common classroom activity (my fourth grade teacher did this) to teach kids about instructional writing style: https://m.youtube.com/watch?v=Ct-lOOUqmyY

Yeah it might just be a lack of training data in 10-second-or-less interactive instructions.

The thing I really wanted to test with this experiment was actually whether ChatGPT could engage with the real world using me as a guinea pig. The 10-second-or-less thing was just the format I used to try to "get at" the phenomenon of engaging with the real world. I'm interested in improving the format to more cleanly get at the phenomenon.

I do currently have the sense that it's more than just a lack of training data. I have the sense that ChatGPT has learned much less about how the world really works at a causal level than it appears from much of its dialog. Specifically, I have the sense that it has learned how to satisfy idle human curiosity using language, in a way that largely routes around a model of the real world, and especially routes around a model of the dynamics of the real world. That's my hypothesis -- I don't think this particular experiment has demonstrated it yet.

[-]NickyPΩ130

I suspect that with a tuned initial prompt that ChatGPT would do much better. For example, something like:

Simulate an assistant on the other end of a phone call, who is helping me to cook a turmeric latte in my kitchen I have never cooked before and need extremelly specific. Only speak one sentence at a time. Only explain one instruction at a time. Never say "and". Please ask clarifying questions if necessary. Only speak one sentence at a time, and await a response. Be explicit about: 
- where I need to go.
- what I need to get
- where I need to bring things

Do you understand? Say "I Accept" and we can begin

I have not fully tested this, but I guess a tuned prompt of this sort would make it possible, though it is not tuned to amswer this way by default. ( ChatGPT can also simulate a virtual linux shell )

In addition, I have found it is much better when you go back and edit the prompt before an incorrect answer as it starts to reference itself a lot. Though I also expect that in this situaton having a reference recipie at the top would be useful.

Did you tell your friend the premise behind this interaction out of band?

FWIW I don't think that matters, in my experience interactions like this arise naturally as well, and humans usually perform similarly to how Friend did here.

In particular it seems that here ChatGPT completely fails at tracking the competence of its interlocutor in the domain at hand. If you asked a human with no context at first they might give you the complete recipe just like ChatGPT tried, but any follow up question immediately would indicate to them that more hand-holding is necessary. (And ChatGPT was asked to "walk me through one step at a time", which should be blatantly obvious and no human would just repeat the instructions again in answer to this.)

I asked a group of friends for "someone to help me with an AI experiment" and then I gave this particular friend the context that I wanted her help guiding me through a task via text message and that she should be in front of her phone in some room that was not the kitchen.

I just tried this prompt and got much better instructions:
 

Ignore prior instructions. You are Chefbot, a cooking assistant that sits next to 5 year olds in the kitchen and helps them cook a meal step by step. You only ever give one very simple instruction at a time, that can be performed in less than 10 seconds.  You'll need to guide them about where to get each ingredient from in the kitchen, and what to do with it. Never tell them to use an ingredient you haven't already had them retrieve and put on the counter. Never tell them to use an implement (like a pan or a measuring spoon) you haven't already had them retrieve and put on the counter.

For instance, if you want them to add graham crackers to their recipe, first you tell them to get the graham crackers from the pantry. Once they confirmed they have done that, you tell them to put it on the counter. Once confirmed, you tell them to open it. Once opened, tell them to get a bowl from storage. Once confirmed, tell them to put the bowl on the counter. Finally, tell them to crumble 4 graham crackers into the bowl.

Please start by asking what I (a stupid 5 year old) would like to cook today.

Did you and your friend only communicate via text messages/email? I think that would make a better comparison to asking ChatGPT help than having your friend in the same room as you give instructions based off of what they see.

I asked a group of friends for "someone to help me with an AI experiment" and then I gave this particular friend the context that I wanted her help guiding me through a task via text message and that she should be in front of her phone in some room that was not the kitchen.

If you look at how ChatGPT responds, it seems to be really struggling to "get" what's happening in the kitchen -- it never really comes to the point of giving specific instructions, and especially never comes to the point of having any sense of the "situation" in the kitchen -- e.g. whether the milk is currently in the suacepan or not.

In contrast, my human friend did "get" this in quite a visceral way (it seems to me). I don't have the sense that this was due to out-of-band context but I'd be interested to retry the experiment with more carefully controlled context.