There's been some recent work in this direction which seems quite interesting: https://arxiv.org/abs/2302.08582