Day 65 – AI Converts Your Photo Or Video Selfie To Animation – GANsNRoses

Image Source: GANsNRoses

This is one of the cool features I admired to write on this blog, how Artificial Intelligence converts your photo or video selfie to animation.

Thus library based on a research paper GANs N’ Roses: Stable, Controllable, Diverse Image to Image Translation (works for videos too!).

Abstract:

We show how to learn a map that takes a content code, derived from a face image, and a randomly chosen style code to an anime image. We derive an adversarial loss from our simple and effective definitions of style and content. This adversarial loss guarantees the map is diverse — a very wide range of anime can be produced from a single content code. Under plausible assumptions, the map is not just diverse, but also correctly represents the probability of an anime, conditioned on an input face. In contrast, current multimodal generation procedures cannot capture the complex styles that appear in anime. Extensive quantitative experiments support the idea the map is correct. Extensive qualitative results show that the method can generate a much more diverse range of styles than SOTA comparisons. Finally, we show that our formalization of content and style allows us to perform video to video translation without ever training on videos.

This GitHub library uses a dataset for the UGATIT name called selfie2anime. You can also use your own dataset.

We can go to Google Colab to run this code directly and upload your photo to get your own animation. However, the model is trained on the female version and provides animation in a female version.

If you don't want to run the Google Colab, we have an option to see how it works on Web Demo.

Citation

If you use this code or ideas from our paper, please cite our paper:

@misc{chong2021gans,
title={GANs N’ Roses: Stable, Controllable, Diverse Image to Image Translation (works for videos too!)},
author={Min Jin Chong and David Forsyth},
year={2021},
eprint={2106.06561},
archivePrefix={arXiv},
primaryClass={cs.CV}
}