skip to main content skip to footer

Systems and Methods for Generating Facial Expressions in a User Interface

Huang, Yuchi; Khan, Saad
Patent Issued:
Dec 10, 2019
Patent Number:
ETS Patent
Document Type:
Family ID:
Subject/Key Words:
Patent, Active Patent, Facial Expressions, Avatars (Computer graphics), Human Computer Interaction, User Computer Interface, Dyads, Interviews


Example systems and methods are disclosed for generating facial expressions in an avatar-based user interface for a computing device. An example system may include a digital camera, a display device and a user interface application. The digital camera may be configured to capture image data of a user over time. The user interface application may be stored on a non-transitory computer-readable medium and executable by a processor, and may include: an expression feature extractor configured to process the image data of the user to generate a plurality of facial expression descriptor vectors, a sketch generation module configured to use a first conditional deep convolutional generative adversarial network (DC-GAN) model to generate an expressive facial sketch image of an avatar based on the plurality of facial expression descriptor vectors, and an image generation module configured to use a second conditional DC-GAN model to generate a facial expression image from the expressive facial sketch image. The display device may be configured to display the facial expression image as a user interface avatar.

Read More