Google Envisages A GPT-3-like Query System, Without Search Results
A new paper from four Google researchers proposes an ‘expert’ system capable of authoritatively answering users’ questions without presenting a list of possible search results, similar to the Q&A paradigm that has come to public attention through the advent of GPT-3 over the past year.
The paper, entitled Rethinking Search: Making Experts out of Dilettantes, suggests that the current standard of presenting the user with a list of search results in response to an inquiry is a ‘cognitive burden’, and proposes improvements in the ability of a natural language processing system (NLP) to provide an authoritative and definitive response.
Under the proposed model of an ‘expert’, cross-domain oracle, the thousands of possible search result sources will be baked into a language model instead of being explicitly available as an exploratory resource for users to evaluate and navigate for themselves. Source: https://arxiv.org/pdf/2105.02274.pdf
The paper, led by Donald Metzler at Google Research, proposes improvements in the type of multi-domain oracle responses that can currently be obtained from deep learningautoregressive language models such as GPT-3. The main improvements envisaged are a) that the model would be capable of accurately citing the sources that informed the response, and b) that the model would be prevented from ‘hallucinating‘ responses or inventing non-existent source material, which is currently an issue with such architectures.
Multi-Domain Training And Capabilities
Additionally, the proposed language model, characterized in the paper as ‘A Single Model for all Information Retrieval Tasks’, would be trained on a variety of domains, including images and text. It would also need an understanding about the provenance of knowledge, which is lacking in GPT-3 style architectures.
‘To replace indexes with a single, unified model, it must be possible for the model itself to have knowledge about the universe of document identifiers, in the same way that traditional indexes do. One way to accomplish this is to move away from traditional LMs and towards corpus models that jointly model term-term, term-document, and document-document relationships.’
In the image above, from the paper, three approaches in response to a user inquiry: left, the language models implicit in Google’s algorithmic search results have chosen and prioritized a ‘best answer’, but have left it as the top result of many. Center, a GPT-3 style conversational response, which speaks with authority, but does not justify its claims or cite sources. Right, the proposed expert system incorporates the ‘best response’ from the ranked search results directly into a didactic answer, with academic-style footnote citations (not depicted in the original image) indicating the sources that inform the response.
See the full story here: https://www.unite.ai/google-envisages-a-gpt-3-like-query-system-without-search-results/