...
How Does Sora Work?
Sora combines features of text and image generating tools in what is called a “diffusion transformer model.”
Transformers are a type of neural network first introduced by Google in 2017. They are best known for their use in large language models such as ChatGPT and Google Gemini.
Diffusion models, on the other hand, are the foundation of many AI image generators. They work by starting with random noise and iterating towards a “clean” image that fits an input prompt. ...
Sora uses the transformer architecture to handle how frames relate to one another. While transformers were initially designed to find patterns in tokens representing text, Sora instead uses tokens representing small patches of space and time. ...
...Sora makes videos up to 60 seconds. ...
Lumiere cannot make videos composed of multiple shots, while Sora can. ...
OpenAI’s technical paper about Sora is titled “Video generation models as world simulators.” ...
See the full story here: https://amplify.nabshow.com/articles/ic-what-is-openai-sora