The Perils of Using Quotations to Authenticate NLG Content
...
Old News and Fake Facts
Natural Language Generation (NLG) models are capable of producing convincing and plausible output because they have learned semantic architecture, rather than more abstractly assimilating the actual history, science, economics, or any other topic on which they might be required to opine, which are effectively entangled as ‘passengers’ in the source data.
The factual accuracy of the information that NLG models generate assumes that the input on which they are trained is in itself reliable and up-to-date, which presents an extraordinary burden in terms of pre-processing and further human-based verification – a costly stumbling block that the NLP research sector is currently addressing on many fronts. ...
The real danger of obtaining quotes from default GPT-3 (for instance) is that it sometimes produces correct quotes, leading to a false confidence in this facet of its capabilities...
GopherCite
Hoping to address this general shortcoming in NLG models, Google’s DeepMind recently proposed GopherCite, a 280-billion parameter model that’s capable of citing specific and accurate evidence in support of its generated responses to prompts. ...
Quoting Falsehoods
However, when tested against Oxford University’s TruthfulQA benchmark, GopherCite’s responses were rarely scored as truthful, in comparison to the human-curated ‘correct’ answers.
The authors suggest that this is because the concept of ‘supported answers’ does not in any objective way help to define truth in itself, since the usefulness of source quotes may be compromised by other factors, such as the possibility that the author of the quote is themselves ‘hallucinating’ (i.e. writing about fictional worlds, producing advertising content, or otherwise fantasticating inauthentic material. ...
See the full story here: https://www.unite.ai/the-perils-of-using-quotations-to-authenticate-nlg-content/
