Judea Pearl proposes a mini-Turing test for seeing if a machine understands causality.
The test is mini, because before the conversation begins, you encode the causal relations in advance, perhaps through fine-tuning, or through prompts. Then you ask associational, causal, and counterfactual questions to see if the LLM gives the right answers. Causal knowledge is claimed by PaLM, but the examples they provide were hardly rigorous: https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhLmQjS3gOQ2x7ru3xovYjVw-Yr2fKDCqhDHByQZitD92Yu4L-v2B... (read more)
In the past it's only been the third Sunday, but I'd be up for making it more frequent if others are also interested. Make it both and we'll see how the attendance is.
Judea Pearl proposes a mini-Turing test for seeing if a machine understands causality.
The test is mini, because before the conversation begins, you encode the causal relations in advance, perhaps through fine-tuning, or through prompts. Then you ask associational, causal, and counterfactual questions to see if the LLM gives the right answers. Causal knowledge is claimed by PaLM, but the examples they provide were hardly rigorous:
https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhLmQjS3gOQ2x7ru3xovYjVw-Yr2fKDCqhDHByQZitD92Yu4L-v2B... (read more)