I work for a leading private statistical research company and think this is a wonderful post. I heartily agree with all the takeaways. I may expand on data leakage examples I've seen "in the wild" in a follow-up post if there's demand for more stories, but your second "time-travelling" example brought back wonderful memories of a large company-wide debate, since your initial suggestion was our modus operandi, and there was likewise "tolerant skepticism" when it was questioned.
"I looked into it and found . . . that the conventional approach worked fine." So... (read more)
I work for a leading private statistical research company and think this is a wonderful post. I heartily agree with all the takeaways. I may expand on data leakage examples I've seen "in the wild" in a follow-up post if there's demand for more stories, but your second "time-travelling" example brought back wonderful memories of a large company-wide debate, since your initial suggestion was our modus operandi, and there was likewise "tolerant skepticism" when it was questioned.
"I looked into it and found . . . that the conventional approach worked fine." So... (read more)