4 Things GPT-4 Will Improve From GPT-3
... GPT-3 was brute-force trained in most of the Internet’s available text data. But it wasn’t explicitly taught to do these tasks. The system is so powerful that it became a meta-learner. It learned how to learn. And users could communicate with it in plain natural language; GPT-3 would receive the description and recognize the task it had to do. ...
Disclaimer: GPT-4 doesn’t exist (yet). ...
GPT-3 is big. GPT-4 will be bigger. Here’s the reason... GPT-3 isn’t just more powerful than GPT-2, it is differently more powerful. There’s a qualitative leap between both models. ...
GPT-4 will perform better few-shot multitasking... GPT-4 could be the proof that language models can learn multitasking from a few examples almost as good as we can. ...
GPT-4 won’t depend as much on good prompting ... Tech blogger Gwern Branwen ... calls this way of interacting with GPT-3 prompt programming. We can give GPT-3 a written input and it’ll know which task it has to perform. ... However, Gwern warns that the results of these tests can vary in quality. The reason is that prompt programming involves sampling. And sampling can “prove the presence of knowledge but not the absence”....
GPT-4 will have a larger context window...The beta API allows the user to input a text 500–1000 words long — the context window — for GPT-3 to do something with it. This means the system can’t continue a half-written novel or complete the code for a large program. ...
See the full story here: https://towardsdatascience.com/4-things-gpt-4-will-improve-from-gpt-3-2b1e7a6da49f
