This has been done for a while. A few years ago there was some noise about a russian chatbot which impersonated a good-looking girl and tried to scam people to give personal information and/or money.

Every time it succeeded, it passed the turing test.