The article doesn't give examples, but the paper does. Here are a few examples of question overlap, for context.
Test Question: who plays max voice in a goofy movie
Train Question: who does max voice in a goofy movie
Answer: Jason Marsden
Test Question: when will the 2018 oscar nominations be announced
Train Question: when are the oscar nominations for 2018 announced
Answer: January 23 2018
Test Question: who has scored more goals in the premier league
Train Question: most goals scored by a premier league player
Answer: Alan Shearer
Test Question: where are the cones in the eye located
Train Question: where are cone cells located in the eye
Answer: retina
Test Question: who led the conquest of the incas in south america
Train Question: conquistador who defeated the incan empire in peru
Answer: francisco pizarro
It makes sense that questions like these are especially susceptible to brute memorization. These are certainly bugs in the benchmark—and, in fact, about a third of questions in each dataset have this issue.
The paper also considers answer overlap; that is, when the answer to the question also occurs in the training set. This alone does not imply memorization, but it does open the door for some shortcuts. Some examples from the paper are:
Open Natural Questions
Duplicated: Phil Simms, Brian Johnson, 8, the Indians, the 1830s
Unique: Cloves, Matt Monro, 1,020 – 1,080 kg, Hermann Ebbinghaus, Matt Flinders
TriviaQA
Duplicated: David Bowie, Battle of camlann, Heligoland, Henry VII, Niagra Falls
Unique: Death in the afternoon, Clash of the Titans, ice-cream sundae, Camshaft, Cumberland
WebQuestions
Duplicated: Harvard, Alderaan, India, 2011, Zeus
Unique: Queen Victoria, Braslia, Paddington, Tom Corbett, Gary
It's less obvious how to treat answer overlap. In particular, removing answer overlap might bias the dataset towards harder questions. Some of the reduction in model scores is presumably because models have to work harder to understand the question, as merely looking for topical similarities will be ineffective, but it also removes questions whose answers are general enough to apply to many questions, like ‘8’, ‘the 1930s’, ‘Harvard’, ‘2011’, etc. This means that reductions in the score don't clearly say how much cheating happened. However, the success of the BERT-based Nearest Neighbor model, which retrieves the answer of the most semantically similar fine-tuning sample, scores similarly to BART, which seems much too high for comfort.
It should be noted that a lot of the discussions about causal knowledge and intelligence aren't too relevant to these questions, as they are largely tests of memory or retrieval. It is expected that these models answer the questions by searching their training data or their document index. The issue is that their training data isn't meant to contain copies of the test questions, just the information sufficient to answer them.
Thanks for the comment and for adding examples, and for your nuanced comments of the answer overlap split.
My position is that these datasets are still useful for QA, but what was lacking was an analysis of how easy/hard the questions in them were, and what kind of modelling was needed to do well. These overlap phenomena are less like "bugs" maybe, but more like poorly understood features.
We need models that can accurately recall QA pairs they have seen before, so being able to score well on "memorizable" QA pairs is still important to do well, but we also want models that can do more than that.
One single accuracy number on a leaderboard cannot capture all the behavioural information we need to properly understand the capabilities of these models.
That's useful for chatbots, where you have a big FAQ and people ask questions. That's exactly what RASA does, using TensorFlow. It's reasonably good at matching user questions with stored questions. But that's all the AI part does.
The paper also considers answer overlap; that is, when the answer to the question also occurs in the training set. This alone does not imply memorization, but it does open the door for some shortcuts. Some examples from the paper are:
It's less obvious how to treat answer overlap. In particular, removing answer overlap might bias the dataset towards harder questions. Some of the reduction in model scores is presumably because models have to work harder to understand the question, as merely looking for topical similarities will be ineffective, but it also removes questions whose answers are general enough to apply to many questions, like ‘8’, ‘the 1930s’, ‘Harvard’, ‘2011’, etc. This means that reductions in the score don't clearly say how much cheating happened. However, the success of the BERT-based Nearest Neighbor model, which retrieves the answer of the most semantically similar fine-tuning sample, scores similarly to BART, which seems much too high for comfort.It should be noted that a lot of the discussions about causal knowledge and intelligence aren't too relevant to these questions, as they are largely tests of memory or retrieval. It is expected that these models answer the questions by searching their training data or their document index. The issue is that their training data isn't meant to contain copies of the test questions, just the information sufficient to answer them.