Using the type of style transfer above, you can take any of your videos and apply an art video filter to make it look like a famous painting or movie! Style transfer usually takes really long, but with our pretrained filters you can convert your video in minutes. You can look at our Guide on how to convert a single photo.
Why the Art Video Filter Works
One of the biggest issues with style transfer is that it takes a LONG time for certain manifestations. Transforming a single image can take over an hour to get something of good quality. This stems from the fact that images are really big – a 1080p image contains over 2 million individual pixels, each with three values for red, green, and blue. So imagine your favorite YouTube video and let’s do some quick maths.
A regular old 1080p video has well over 300 billion pixels every second. Each pixel is a byte, but you don’t need to download 300 gigabytes every second because of data compression. Style transfer isn’t as lucky though. Any style transfer method will have to transform every single one of those 300 billion pixels to convert just one second of video. Now assume we want to transform a 5 minute video with a method that takes 1 minute per 1080p frame – pretty fast for most methods.
This post would end here if there wasn’t an answer to this problem. But there is! Justin Johnson et. al. at Stanford devised a technique that is fast enough to run on a live webcam feed! It takes under 1 second to run on a single frame. This brings the time to transform a short video down to a few minutes.
There is a catch though. The model that takes <1 second per frame has to be trained first. And that takes hours.
The technique boils down to performing an exhaustive style transfer on a few thousand images, and then training a neural network to do the exhaustive style transfer efficiently. This means that a new network has to be trained for every desired style. In my experience, the model training takes around 8 hours, and it still takes another hour or two to run it on something like a 5 minute video. With this method, you should train 4-5 networks for styles that you like, apply them to short clips, and then pick the one you like best for a longer video.
This paper isn’t the holy grail of speed and beauty we are all hoping for, but it is a huge step in the right direction.
How to Do It Yourself
If you have no coding ability, I am happy to convert videos for you. Just send any requests to firstname.lastname@example.org, and I can help!