Disney, Rutgers Scientists Use AI to Generate Storyboards
The system first “determines if a given snippet contains a particular syntactic structure and subsequently splits and assembles it into simpler sentences, recursively processing it until no further simplification is possible,” then applies a “coordination step and a lexical simplifier, which “matches actions in the simplified sentences with 52 animations (expanded to 92 by a dictionary of synonyms) in a predefined library.”
The content is fed to the pipeline, dubbed Cardinal, which turns the actions into previsualizations in the Unreal video game engine. The system has been trained with “scene descriptions from 996 screenplays drawn from over 1,000 scripts scraped from freely available sources including IMSDb, SimplyScripts, and ScriptORama5.” The system is made up of “525,708 descriptions containing 1,402,864 sentences, 920,817 (over 40 percent) of which had at least one action verb.”
See the full story here: http://www.etcentric.org/disney-rutgers-scientists-use-ai-to-generate-storyboards/