Home » Posts » A few ways to detect loops...

A few ways to detect loops in video clips

This post briefly talks about a few ways to detect loops in video clips using ffmpeg or python, experienced when I was making seamless loops and extended versions of video and audio clips. — My need for detecting loops in clips arises from my sudden interest in making seamless loops and extended versions of video clips. The multimedia franchise, Uma Musume, just released the second series of their remixed music video. Uma Musume is known for its high quality music and video production. Even though it has been more than a year since I last logged into the game, I still keep an eye on the latest news and updates, especially for their lives and newly released music.

The remix music series is a collection of remixed versions of the original songs from the game accompanied by delicate (seemingly) looping animations. While each clip only lasts for about one minute, it is possible to extend both the video and audio clips to make them last for a longer period of time, i.e. creating a seamless loop. The final result looks like this:

---

Detecting loops in video clips is actually detecting identical frames. Identical frames allow you to connect two clips together without any noticeable jump. However, there is no built-in function in AE or PR for detecting identical frames. Thus, I tried using python scripts and ffmpeg to achieve this goal.

Similar to image comparison, the common way to compare two frames is to create a mask/filter based on the first frame. Then, apply the mask to the second frame and compare the two frames. If the two frames are identical, the result should be black. The percentage of black pixels in the result image is the similarity between the two frames.

ffmpeg

ffmpeg uses the blend filter to compare two frames. The blend filter takes two frames as input and outputs a single frame. The output frame is the result of blending, which is the comparison result.

This is a sample code that compares the first frame with all the other frames.

ffmpeg -i {videoname.mp4} -filter_complex "[0]trim=start_frame=:end_frame=1[c];[c][0]blend=all_mode=difference,
blackframe" -f null -

The output should be like this:

[Parsed_blackframe_2 @ 0000021471ac5040] frame:1 pblack:100 pts:658658 t:21.955267 type:P last_keyframe:0
[Parsed_blackframe_2 @ 0000021471ac5040] frame:90 pblack:98 pts:747747 t:24.924900 type:P last_keyframe:0
[Parsed_blackframe_2 @ 0000021471ac5040] frame:91 pblack:98 pts:748748 t:24.958267 type:P last_keyframe:0

where frame is the frame number starting from the ‘start_frame’ specified, and ‘pblack’, the percentage of black, is the similarity. ‘t’ is the time stamp counting from the start of the video.

Reference: https://video.stackexchange.com/questions/19869/finding-the-place-where-a-video-loops

python

The basic idea of detecting loop in python is similar to the one in ffmpeg, that is to compare a selected frame with the remaining frame to see if they are identical. There are various implementations using different libraries including opencv and PIL. To make the process more efficient, many implementations computes and calculates a hash for each frame in a video. This is faster than comparing each frames in the whole video.

Some implementations can be found here.