Animating static AI Art using Thin-Plate Spline Motion Model

This had my mind blown last night as I was playing with it.  You take an image
of a face [any human face, dogs don't work, I tried], use a driving video, and the model will turn your static image into a video.

So, thinking outside the box, I wondered if I took a random this person doesn't exist picture and then animate it with the default Jackie Chan. Shockingly it WORKED!

Go to https://thispersondoesnotexist.com and generate a face [right click save it].

Go to the demo:

https://replicate.com/yoyo-nb/thin-plate-spline-motion-model

Use your image, Jackie Chan, and the Vox dataset, then hit the submit button. After a few moments you will get an animated video.

Some more examples:

Generated with MidJourney, then animated.

There are some artifacts left over, but overall, it was pretty good.

Since this image had the forehead and above blocked, there is a weird effect in the video.

Here is another MidJourney generated image:

This one was about the coolest of the bunch.

Final Thoughts

While it may not be picture perfect today, I can't wait to see it get better and better in the future.  It has me thinking of a bunch of use cases.

If you have any questions or comments, please feel free to reach out to me on Mastodon.

Jeremy