Computer Graphics Laboratory ETH Zurich


Generating Animations from Screenplays

Y. Zhang, E. Tsipidi, S. Schriber, M. Kapadia, M. Gross, A. Modi

Proceedings of the Eighth Joint Conference on Lexical and Computational Semantics (*{SEM} 2019) (Minneapolis, Minnesota, USA, June, 2019), pp. 292--307


Automatically generating animation from natural language text finds application in a number of areas e.g. movie script writing, instructional videos, and public safety. However, translating natural language text into animation is a challenging task. Existing text-to-animation systems can handle only very simple sentences, which limits their applications. In this paper, we develop a text-to-animation system which is capable of handling complex sentences. We achieve this by introducing a text simplification step into the process. Building on an existing animation generation system for screenwriting, we create a robust NLP pipeline to extract information from screenplays and map them to the system's knowledge base. We develop a set of linguistic transformation rules that simplify complex sentences. Information extracted from the simplified sentences is used to generate a rough storyboard and video depicting the text. Our sentence simplification module outperforms existing systems in terms of BLEU and SARI metrics.We further evaluated our system via a user study: 68\% participants believe that our system generates reasonable animation from input screenplays.


Download Paper
Download Video
Download Paper