Hardware-Accelerated Video Decoding¶

FFmpeg offer access to dedicated GPU hardware with varying support on different platforms for performing a range of video-related tasks to be completed faster or using less of other resources (particularly CPU).

By default, DeFFcode's FFdecoder API uses the Input Source's video-decoder (extracted using Sourcer API) itself for decoding its input. However, you could easily change the video-decoder to your desired specific supported Video-Decoder using FFmpeg options by way of its ffparams dictionary parameter. This means easy access to GPU Accelerated Hardware Decoder to get better playback and accelerated video decoding on GPUs that will generate equivalent output to software decoders, but may use less power and CPU to do so.

Use ffmpeg -decoders terminal command to lists all FFmpeg supported decoders.

We'll discuss its Hardware-Accelerated Video Decoding capabilities briefly in the following recipes:

DeFFcode APIs requires FFmpeg executable

DeFFcode APIs MUST requires valid FFmpeg executable for all of its core functionality, and any failure in detection will raise RuntimeError immediately. Follow dedicated FFmpeg Installation doc ➶ for its installation.

Always use FFdecoder API's terminate() method at the end to avoid undesired behavior.

Never name your python script deffcode.py

When trying out these recipes, never name your python script deffcode.py otherwise it will result in ModuleNotFound error.

GPU-accelerated Hardware-based Video Decoding¶

Example Assumptions

Please note that following recipe explicitly assumes:

You're running Windows operating system with a supported NVIDIA GPU.
You're using FFmpeg 4.4 or newer, configured with atleast --enable-nonfree --enable-libx264 --enable-cuda --enable-cuvid --enable-cuda-nvcc options during compilation. For manual compilation follow these instructions ➶
You already have appropriate Nvidia video drivers and related softwares installed on your machine.

These assumptions MAY/MAY NOT suit your current setup. Kindly use suitable parameters based your system platform and hardware settings only.

In this example, we will be using Nvidia's Hardware Accerlated CUDA Video-decoder(cuda) in FFdecoder API to automatically detect NV-accelerated video codec and achieve GPU-accelerated hardware video decoding of YUV420p frames from a given Video file (say foo.mp4) on Windows Machine.

More information on Nvidia's CUVID can be found here ➶

YUV video-frames decoded with DeFFcode APIs are not yet supported by OpenCV methods.

Currently, there's no way to use DeFFcode APIs decoded YUV video-frames in OpenCV methods, and also you cannot change pixel format to any other due to NV-accelerated video codec supporting only few pixel-formats.

To learn about exclusive -ffprefixes parameter. See Exclusive Parameters ➶

# import the necessary packages
from deffcode import FFdecoder
import json

# define suitable FFmpeg parameter
ffparams = {
    "-vcodec": None,  # skip any decoder and let FFmpeg chose
    "-ffprefixes": [
        "-vsync",
        "0",
        "-hwaccel", # chooses appropriate HW accelerator
        "cuda",
        "-hwaccel_output_format", # keeps the decoded frames in GPU memory
        "cuda",
    ],
    "-custom_resolution": "null",  # discard `-custom_resolution`
    "-framerate": "null",  # discard `-framerate`
    "-vf": "scale_npp=format=yuv420p,hwdownload,format=yuv420p,fps=30.0",  # define your filters
}

# initialize and formulate the decoder with params and custom filters
decoder = FFdecoder(
    "foo.mp4", frame_format="null", verbose=True, **ffparams  # discard frame_format
).formulate()

# grab the YUV420 frame from the decoder
for frame in decoder.generateFrame():

    # check if frame is None
    if frame is None:
        break

    # {do something with the frame here}

# terminate the decoder
decoder.terminate()

Last update: September 21, 2022