This is a linkpost for https://twitter.com/shminux/status/1602140709204955142
Prodding ChatGPT to solve a basic algebra problem
6Unnamed
2Shmi
4Unnamed
1spkoc
2Unnamed
4the gears to ascension
New Comment
yeah, looks like "steb by step" is a magic incantation:
https://twitter.com/ESYudkowsky/status/1602177149401989120
... which makes it even more human. I used to almost yell at the students I used to tutor: DON'T SKIP STEPS!
Didn't seem to work for me. It still seems to get confused trying to match similar words together even when they shouldn't be. Again quite, dumb/young human.
It tries to continue its dialogue in ways that fit with its previous replies, so prompts like "think step by step" might fail to get it back on track mid-conversation even when they would've worked in the initial prompt.
Also, there is some randomness.
TL;DR: It behaves amazingly like a human would!
It understands the word problem, but struggles to put it in equations, takes multiple tries, screws up again, eventually muddles through to get the right answer. Given a similar problem it does slightly better, but still messes up. On the third problem it messes up again, then realizes that something went wrong... and promptly blames the problem setup, unable to question its own reasoning.