The Internet is a great invention. Just about everything in humanity’s knowledge can be found on the Internet: with just a few keystrokes, you can find dozens of excellent quality textbooks, YouTube videos and blog posts about any topic you want to learn. The information is accessible to a degree...
We are now almost two years into the COVID-19 pandemic: international travel has slowed to a trickle and all machine learning conferences have moved online. By now, most of my PhD research has taken place during the pandemic, which I’ve presented at four online conferences (ACL ’20, EMNLP ’20, NAACL...
A classic economics joke goes like this: > Two economists are walking down a road, when one of them notices a $20 bill on the ground. He turns to his friend and exclaims: "Look, a $20 bill!" The other replies: "Nah, if there's a $20 on the bill on the...
The Internet is a great invention. Just about everything in humanity’s knowledge can be found on the Internet: with just a few keystrokes, you can find dozens of excellent quality textbooks, YouTube videos and blog posts about any topic you want to learn. The information is accessible to a degree that scholars and inventors could have hardly dreamed of even a few decades ago. With the popularity of remote work and everything moving online this decade, it is not surprising that many people are eager to learn new skills from the Internet.
However, despite the abundance of resources, self-studying over the Internet is harder than you think. You can easily find a list... (read 1031 more words →)
We are now almost two years into the COVID-19 pandemic: international travel has slowed to a trickle and all machine learning conferences have moved online. By now, most of my PhD research has taken place during the pandemic, which I’ve presented at four online conferences (ACL ’20, EMNLP ’20, NAACL ’21, and ACL ’21). I’ve also had the fortune to attend a few in-person conferences before the pandemic, so here I'll share some of my thoughts on the advantages of each format.
Travel
One of the perks of choosing grad school for me was the chance to travel to conferences to present my work. Typically the first author of each paper gets funded by... (read 762 more words →)
To be fair, almost nobody considered a pandemic to be a serious possibility prior to 2020, so it is understandable that pandemic preparedness research was a low-priority area. There may be lots of open and answerable questions in unpopular topics, but if the topic is obscure, the payoff for making a discovery is small (in terms of reputation and recognition).
Of course, COVID-19 has proven to us that pandemic research is important, and immediately researchers poured in from everywhere to work on various facets of the problem (e.g., I even joined in an effort to build a ventilator simulator). The payoff increased, so the inefficiencies quickly disappeared.
Now you can argue that pandemic research should've been more prioritized before. That is obvious in hindsight but was not at all obvious in 2019. Out of the zillions of low-priority research areas that nobody cares about now, how will you decide which one will become important? Unless you have a time machine to see into the future, it remains a low-payoff endeavor.
Yeah, I agree that the EMH holds true more for incremental research than for truly groundbreaking ideas. I'm not too familiar with MCMC or Bayesian inference so correct me if I'm wrong, but I'm guessing these advancements required combining of ideas that nobody expected would work? The deep learning revolution could probably have happened sooner (in the sense that all the prerequisite tools existed), but few people before 2010 expected neural networks to work so consequently the inefficiencies there remained undiscovered.
At the same time, I wouldn't denigrate research that you might view as "incremental", because most research is of that nature. By this I mean, for every paper published in the ACL / EMNLP conferences, if the authors hadn't published it, someone else would almost certainly have published something very similar within 1-2 years. Exceptions to this are few and far between -- science advances via an accumulation of many small contributions.
True, I guess a more precise statement is "most problems that are important and solvable are already solved". There are lots of small gaps in my research as well, like "what if we make a minor adjustment to method X" -- whatever the outcome, it's below the bar for a publication so they're generally left untouched.
A classic economics joke goes like this:
Two economists are walking down a road, when one of them notices a $20 bill on the ground. He turns to his friend and exclaims: "Look, a $20 bill!" The other replies: "Nah, if there's a $20 on the bill on the ground, someone would've picked it up already."
The economists in the joke believe in the Efficient Market Hypothesis (EMH), which roughly says that financial markets are efficient and there's no way to "beat the market" by making intelligent trades.
If the EMH was true, then why is there still a trillion-dollar finance industry with active mutual funds and hedge funds? In reality, the EMH is not... (read 617 more words →)