Michael Liu
20
3
Michael Liu has not written any posts yet.
Michael Liu has not written any posts yet.
Agreed. I highly recommend this blog post (https://sander.ai/2024/09/02/spectral-autoregression.html) for concretely understanding why autoregressive and diffusion models are so similar, despite seeming so different.
According to the creator the "Claude plays Pokemon", the internal knowledge in Claude can often be more harmful than good for successfully navigating the game. In the system prompt, Claude is specifically told not to trust it's instincts and to rely on the memories in its context. See (starting @ 20:28):
Really good and well written post. While I think it always good to provide vigorous and strong evidence against Gary Marcus (and others) when he defaults to promoting claims that confirm his bias, I wonder if there is are beter long term solutions to this issue. For normal everyday people that don't follow AI or people that show modest interest in learning more about AI, they will likely never see this posts or have even heard of LW/Alignment Forum/80k/etc. I do think that the lack of public mainstream content is part of the issue, but I'm sure that there are lots of difficulties that I am naive to. Also, I sense that... (read more)