This is my 99th-day blog on my #100daychallenge. I want to check some awesome libraries that give real effect on our photos.
Today, we will check 3D Ken Burns, how it works, and its features.
What is 3D Ken Burns?
This is a reference implementation of 3D Ken Burns Effect from a Single Image [1] using PyTorch. Given a single input image, it animates this still image with a virtual camera scan and zoom subject to motion parallax.
Source: 3D Ken Burns
Research Paper:
3D Ken Burns Effect from a Single Image
How does it work?
Overview of our depth estimation pipeline. Given a high-resolution image, we start by estimating a coarse depth based on a low-resolution input
image. This depth estimation network is guided by semantic information extracted using VGG-19 [Simonyan and Zisserman 2014] and supervised on a computer-generated dataset with accurate ground truth depth in order to facilitate geometrically sound predictions. To avoid semantic distortions, we then adjust the depth map according to the segmentation of Mask R-CNN [He et al. 2017] and make sure that each salient object is mapped to a coherent plane. Lastly, we utilize a depth refinement network that, guided by the input image, upsamples the coarse depth and ensures accurate depth boundaries.
Source: 3D Ken Burns Effect from a Single Image
Let’s check the code how it works that was created Manuel Romero.
Note: If you copy the code to local Google Colab, it would give a module error not found for Chainer and Gevent. I have installed both separately.
I have copied the code, and you can check on Google Colab.
Sample Animation Videos
Example 1 – Original Photo and 3D Video
Example 2 – Original Photo and 3D Video
Example 3 – Original Photo and 3D Video
What you like about 3D Kens Burns library?
Please comment your thoughts.
Further Reading
Posts on Artificial Intelligence, Deep Learning, Machine Learning, and Design Thinking articles:
Cartoonize Your Photo or Video Using Cartoonize Artificial Intelligence
Artificial Intelligence Chatbot Using Neural Network and Natural Language Processing
Further Reading
Posts on Artificial Intelligence, Deep Learning, Machine Learning, and Design Thinking articles:
Artificial Intelligence Chatbot Using Neural Network and Natural Language Processing
Thanks for a very informative article & providing the code for folks like me to use the 3D Ken Burns effect on our own pics.
I have been tweaking the autozoom function, with varying levels of success, in order to see if it can produce different angles or longer videos. Just wondering if you’ve tried this also & could suggest some values / tips?
Thanks
No, I haven’t tried in-depth on this.
If you have anything to share with our AI community, I will look forward.
Thanks for a very informative article & providing the code for folks like me to use the 3D Ken Burns effect on our own pics.
I have also been tweaking the autozoom function, with varying levels of success, in order to see if it can produce different angles or longer videos. Just wondering if you’ve tried this also & could suggest some values / tips?
Thanks
I really want this to work, but I am having a problem at the very end:
Traceback (most recent call last):
File “autozoom.py”, line 7, in
import cupy
ModuleNotFoundError: No module named ‘cupy’
Any ideas as to how I can get this to work? Thanks so much!
James,
If you use a virtual environment then the code will run without any issue.
However, if you run through Colab or an external environment then we need to import manually missing packages before running the code.
I have tested again today on Google Colab today and looks good. You can check on this link https://colab.research.google.com/drive/1HRNaIaTf7CE4nlJBlUS94zLhIV3Dg-vI?usp=sharing.
You can find a solution here in StackOverflow to https://stackoverflow.com/questions/56563973/no-module-named-cupy-on-google-colab.
Try to install with this command if you are running on Google Colab !pip install cupy.
Let me know how it goes.
P.S added !pip install cupy in the blog code after seeing comments.