New research could make it easier to capture slow-motion on your smartphone. To capture slo-mo footage it's all about capturing a large number of frames per second.
If you do not record enough, it becomes choppy and unwatchable as soon as you slow down your video – unless that is, you use artificial intelligence to imagine the extra frames.
Chip designer Nvidia does exactly that. The company uses deep learning to turn 30 frames-per-second videos into breathtaking, 240 frames-per-second slow motion. Essentially the AI system looks at two different frames and then it creates intermediary footage by tracking the movement of objects from one frame to the next.
It is not quite the same as actual imagining footage like a human brain does, but it produces accurate (though not perfect) results.
The process will need refinement before it can be used commercially, but when the system is improved, it will be used to add slow-motion effects to smartphone footage after it's been recorded.
"While it is possible to take 240-frame-per-second videos with a cell phone, recording everything at high frame rates is impractical, as it requires large memories and is power-intensive for mobile devices," write the researchers in a paper, describing their work on pre-print server arXiv. "For these reasons and others, it is of great interest to generate high-quality slow-motion video from existing videos."
But with every new system on the market, there are limitations. For starters, the researchers had to train their deep learning system to handle specific types of footage. In order to slow down the video of a car skidding through water (as seen in the video), Nvidia had to first train it on similar footage. This might limit how widely a consumer version of the research could apply slow-motion to target video. It is also worth noting that such a technique can't produce an infinite number of intermediary frames.
Without necessary data, it becomes impossible for computers to predict motion and behaviour of real-world objects. However, Nvidia's system can produce up to seven intermediary frames, which is more than enough to create decent slow-motion.
Nvidia's techniques look faster and easier than existing methods to fake slow-motion footage. Expect to see features like this appearing in your phone's camera in the near future.