Saving Key-frames as Image¶

DeFFcode's FFdecoder API provide effortless and precise Frame Seeking with -ss FFmpeg parameter that enable us to save any frame from a specific part of our input source.

We'll discuss aboout it briefly in the following recipes:

DeFFcode APIs requires FFmpeg executable

DeFFcode APIs MUST requires valid FFmpeg executable for all of its core functionality, and any failure in detection will raise RuntimeError immediately. Follow dedicated FFmpeg Installation doc ➶ for its installation.

Additional Python Dependencies for following recipes

Following recipes requires additional python dependencies which can be installed easily as below:

OpenCV: OpenCV is required for saving video frames. You can easily install it directly via pip:

OpenCV installation from source

You can also follow online tutorials for building & installing OpenCV on Windows, Linux, MacOS and Raspberry Pi machines manually from its source.

Make sure not to install both pip and source version together. Otherwise installation will fail to work!

Other OpenCV binaries

OpenCV maintainers also provide additional binaries via pip that contains both main modules and contrib/extra modules opencv-contrib-python, and for server (headless) environments like opencv-python-headless and opencv-contrib-python-headless. You can also install any one of them in similar manner. More information can be found here.
```
pip install opencv-python       
```
Pillow: Pillow is a Imaging Library required for saving frame as Image. You can easily install it directly via pip:
```
pip install Pillow     
```
Matplotlib: Matplotlib is a comprehensive library for creating static, animated, and interactive visualizations, also required for saving frame as Image. You can easily install it directly via pip:
```
pip install matplotlib   
```
Imageio: Imageio is a Library for reading and writing a wide range of image, video, scientific, and volumetric data formats, also required for saving frame as Image. You can easily install it directly via pip:
```
pip install imageio      
```

Always use FFdecoder API's terminate() method at the end to avoid undesired behavior.

Never name your python script deffcode.py

When trying out these recipes, never name your python script deffcode.py otherwise it will result in ModuleNotFound error.

Extracting Key-frames as PNG image¶

In this example we will seek to 00:00:01.45(or 1045msec) in time and decode one single frame in FFdecoder API, and thereby saving it as PNG image using few prominent Image processing python libraries by providing valid filename (e.g. foo_image.png).

Time unit syntax in -ss FFmpeg parameter

You can use two different time unit formats with -ss FFmpeg parameter:

Sexagesimal(in seconds): Uses (HOURS:MM:SS.MILLISECONDS) format, such as in 01:23:45.678.
Fractional: such as in 02:30.05. This is interpreted as 2 minutes, 30 and a half a second, which would be the same as using 150.5 in seconds.

Using PillowUsing OpenCVUsing MatplotlibUsing Imageio

In Pillow, the fromarray() function can be used to create an image memory from an RGB frame:

# import the necessary packages
from deffcode import FFdecoder
from PIL import Image

# define the FFmpeg parameter to seek to 00:00:01.45(or 1s and 45msec)
# in time and get one single frame
ffparams = {"-ss": "00:00:01.45", "-frames:v": 1}

# initialize and formulate the decoder with suitable source
decoder = FFdecoder("foo.mp4", **ffparams).formulate()

# grab the RGB24(default) frame from the decoder
frame = next(decoder.generateFrame(), None)

# check if frame is None
if not (frame is None):
    # Convert to Image
    im = Image.fromarray(frame)
    # Save Image as PNG
    im.save("foo_image.png")
else:
    raise ValueError("Something is wrong!")

# terminate the decoder
decoder.terminate()

In OpenCV, the imwrite() function can export BGR frame as an image file:

# import the necessary packages
from deffcode import FFdecoder
import cv2

# define the FFmpeg parameter to seek to 00:00:01.45(or 1s and 45msec) 
# in time and get one single frame
ffparams = {"-ss": "00:00:01.45", "-frames:v":1}

# initialize and formulate the decoder for BGR24 outputwith suitable source
decoder = FFdecoder("foo.mp4", frame_format="bgr24", **ffparams).formulate()

# grab the BGR24 frame from the decoder
frame = next(decoder.generateFrame(), None)

# check if frame is None
if not(frame is None):
    # Save our image as PNG
    cv2.imwrite('foo_image.png', frame)
else:
    raise ValueError("Something is wrong!")

# terminate the decoder
decoder.terminate()

In Matplotlib, the imsave() function can save an RGB frame as an image file:

# import the necessary packages
from deffcode import FFdecoder
import matplotlib.pyplot as plt

# define the FFmpeg parameter to seek to 00:00:01.45(or 1s and 45msec) 
# in time and get one single frame
ffparams = {"-ss": "00:00:01.45", "-frames:v":1}

# initialize and formulate the decoder with suitable source
decoder = FFdecoder("foo.mp4", **ffparams).formulate()

# grab the RGB24(default) frame from the decoder
frame = next(decoder.generateFrame(), None)

# check if frame is None
if not(frame is None):
    # Save our image as PNG
    plt.imsave('foo_image.png', frame)
else:
    raise ValueError("Something is wrong!")

# terminate the decoder
decoder.terminate()

In Imageio, the imwrite() function can be used to create an image memory from an RGB frame:

# import the necessary packages
from deffcode import FFdecoder
import imageio

# define the FFmpeg parameter to seek to 00:00:01.45(or 1s and 45msec) 
# in time and get one single frame
ffparams = {"-ss": "00:00:01.45", "-frames:v":1}

# initialize and formulate the decoder with suitable source
decoder = FFdecoder("foo.mp4", **ffparams).formulate()

# grab the RGB24(default) frame from the decoder
frame = next(decoder.generateFrame(), None)

# check if frame is None
if not(frame is None):
    # Save our output
    imageio.imwrite('foo_image.jpeg', frame)
else:
    raise ValueError("Something is wrong!")

# terminate the decoder
decoder.terminate()

Generating Thumbnail with a Fancy filter¶

`fancy_thumbnail.jpg` (Courtesy - BigBuckBunny)

In this example we first apply FFmpeg’s tblend filter with an hardmix blend mode (cool stuff) and then seek to 00:00:25.917(or 25.917sec) in time to retrieve our single frame thumbnail, and thereby save it as JPEG image with valid filename (e.g. fancy_thumbnail.jpg) using Pillow library.

Time unit syntax in -ss FFmpeg parameter

You can use two different time unit formats with -ss FFmpeg parameter: - [x] Sexagesimal(in seconds): Uses (HOURS:MM:SS.MILLISECONDS), such as in 01:23:45.678 - [x] Fractional: such as in 02:30.05, this is interpreted as 2 minutes, 30 seconds, and a half a second, which would be the same as using 150.5 in seconds.

Available blend mode options

Other blend mode options for tblend filter include: addition, addition128, grainmerge, and, average, burn, darken, difference, difference128, grainextract, divide, dodge, freeze, exclusion, extremity, glow, hardlight, hardmix, heat, lighten, linearlight, multiply, multiply128, negation, normal, or, overlay, phoenix, pinlight, reflect, screen, softlight, subtract, vividlight, xor

# import the necessary packages
from deffcode import FFdecoder
from PIL import Image

# define the FFmpeg parameter to
ffparams = {
    "-vf": "tblend=all_mode='hardmix'",  # trim and reverse
    "-ss": "00:00:25.917",  # seek to 00:00:25.917(or 25s 917msec)
    "-frames:v": 1,  # get one single frame
}

# initialize and formulate the decoder with suitable source
decoder = FFdecoder("BigBuckBunny.mp4", **ffparams).formulate()

# grab the RGB24(default) frame from the decoder
frame = next(decoder.generateFrame(), None)

# check if frame is None
if not (frame is None):
    # Convert to Image
    im = Image.fromarray(frame)
    # Save Image as JPEG
    im.save("fancy_thumbnail.jpg")
else:
    raise ValueError("Something is wrong!")

# terminate the decoder
decoder.terminate()

Last update: January 10, 2023