Transcoding Live frames¶

What exactly is Transcoding?

Before heading directly into recipes we have to talk about Transcoding:

Transcoding is the technique of transforming one media encoding format into another.

This is typically done for compatibility purposes, such as when a media source provides a format that the intended target is not able to process; an in-between adaptation step is required:

Decode media from its originally encoded state into raw, uncompressed information.
Encode the raw data back, using a different codec that is supported by end user.

While decoding media into video frames is purely managed by DeFFcode's FFdecoder API, you can easily encode those video frames back into multimedia files using any well-known video processing library such as OpenCV and VidGear.

We'll discuss transcoding using both these libraries briefly in the following recipes:

DeFFcode APIs requires FFmpeg executable

DeFFcode APIs MUST requires valid FFmpeg executable for all of its core functionality, and any failure in detection will raise RuntimeError immediately. Follow dedicated FFmpeg Installation doc ➶ for its installation.

Additional Python Dependencies for following recipes

Following recipes requires additional python dependencies which can be installed easily as below:

OpenCV: OpenCV is required for previewing and encoding video frames. You can easily install it directly via pip:

OpenCV installation from source

You can also follow online tutorials for building & installing OpenCV on Windows, Linux, MacOS and Raspberry Pi machines manually from its source.

Make sure not to install both pip and source version together. Otherwise installation will fail to work!

Other OpenCV binaries

OpenCV maintainers also provide additional binaries via pip that contains both main modules and contrib/extra modules opencv-contrib-python, and for server (headless) environments like opencv-python-headless and opencv-contrib-python-headless. You can also install any one of them in similar manner. More information can be found here.
```
pip install opencv-python       
```
VidGear: VidGear is required for lossless encoding of video frames into file/stream. You can easily install it directly via pip:
```
pip install vidgear[core]       
```

Always use FFdecoder API's terminate() method at the end to avoid undesired behavior.

Never name your python script deffcode.py

When trying out these recipes, never name your python script deffcode.py otherwise it will result in ModuleNotFound error.

Transcoding Video using OpenCV VideoWriter API¶

OpenCV's VideoWriter() class can be used directly with DeFFcode's FFdecoder API to encode video frames into a multimedia file. However, it lacks fine-grained control over output quality, bitrate, compression, and other advanced parameters—features that are readily available with VidGear's WriteGear API.

In this example, we will decode video frames with different pixel formats from a given video file (e.g., foo.mp4) using the FFdecoder API, and then encode them in real time using OpenCV's VideoWriter() method..

OpenCV's VideoWriter() class requires a valid Output filename (e.g. output_foo.avi), FourCC code, framerate, and resolution as input.

You can use FFdecoder's metadata property object that dumps source Video's metadata information (as JSON string) to retrieve output framerate and resolution.

BGR framesRGB framesGRAYSCALE framesYUV frames

By default, OpenCV expects BGR format frames in its cv2.write() method.

# import the necessary packages
from deffcode import FFdecoder
import json, cv2

# initialize and formulate the decoder for BGR24 pixel format output
decoder = FFdecoder("foo.mp4", frame_format="bgr24").formulate()

# retrieve JSON Metadata and convert it to dict
metadata_dict = json.loads(decoder.metadata)

# prepare OpenCV parameters
FOURCC = cv2.VideoWriter_fourcc("M", "J", "P", "G")
FRAMERATE = metadata_dict["output_framerate"]
FRAMESIZE = tuple(metadata_dict["output_frames_resolution"])

# Define writer with parameters and suitable output filename for e.g. `output_foo.avi`
writer = cv2.VideoWriter("output_foo.avi", FOURCC, FRAMERATE, FRAMESIZE)

# grab the BGR24 frame from the decoder
for frame in decoder.generateFrame():

    # check if frame is None
    if frame is None:
        break

    # {do something with the frame here}

    # writing BGR24 frame to writer
    writer.write(frame)

     # let's also show output window
    cv2.imshow("Output", frame)

    # check for 'q' key if pressed
    key = cv2.waitKey(1) & 0xFF
    if key == ord("q"):
        break

# close output window
cv2.destroyAllWindows()

# terminate the decoder
decoder.terminate()

# safely close writer
writer.release()

Since OpenCV expects BGR format frames in its cv2.write() method, therefore we need to convert RGB frames into BGR before encoding as follows:

# import the necessary packages
from deffcode import FFdecoder
import json, cv2

# initialize and formulate the decoder for RGB24 pixel format output
decoder = FFdecoder("foo.mp4").formulate()

# retrieve JSON Metadata and convert it to dict
metadata_dict = json.loads(decoder.metadata)

# prepare OpenCV parameters
FOURCC = cv2.VideoWriter_fourcc("M", "J", "P", "G")
FRAMERATE = metadata_dict["output_framerate"]
FRAMESIZE = tuple(metadata_dict["output_frames_resolution"])

# Define writer with parameters and suitable output filename for e.g. `output_foo.avi`
writer = cv2.VideoWriter("output_foo.avi", FOURCC, FRAMERATE, FRAMESIZE)

# grab the RGB24 frame from the decoder
for frame in decoder.generateFrame():

    # check if frame is None
    if frame is None:
        break

    # {do something with the frame here}

    # converting RGB24 to BGR24 frame
    frame_bgr = cv2.cvtColor(frame, cv2.COLOR_RGB2BGR)

    # writing BGR24 frame to writer
    writer.write(frame_bgr)

# terminate the decoder
decoder.terminate()

# safely close writer
writer.release()

OpenCV also directly consumes GRAYSCALE frames in its cv2.write() method.

# import the necessary packages
from deffcode import FFdecoder
import json, cv2

# initialize and formulate the decoder for GRAYSCALE output
decoder = FFdecoder("foo.mp4", frame_format="gray", verbose=True).formulate()

# retrieve JSON Metadata and convert it to dict
metadata_dict = json.loads(decoder.metadata)

# prepare OpenCV parameters
FOURCC = cv2.VideoWriter_fourcc("M", "J", "P", "G")
FRAMERATE = metadata_dict["output_framerate"]
FRAMESIZE = tuple(metadata_dict["output_frames_resolution"])

# Define writer with parameters and suitable output filename for e.g. `output_foo_gray.avi`
writer = cv2.VideoWriter("output_foo_gray.avi", FOURCC, FRAMERATE, FRAMESIZE)

# grab the GRAYSCALE frame from the decoder
for frame in decoder.generateFrame():

    # check if frame is None
    if frame is None:
        break

    # {do something with the frame here}

    # writing GRAYSCALE frame to writer
    writer.write(frame)

# terminate the decoder
decoder.terminate()

# safely close writer
writer.release()

With FFdecoder API, frames extracted with YUV pixel formats (yuv420p, yuv444p, nv12, nv21 etc.) are generally incompatible with OpenCV APIs. But you can make them easily compatible by using exclusive -enforce_cv_patch boolean attribute of its ffparam dictionary parameter.

Let's try encoding YUV420p pixel-format frames with OpenCV's write() method in following python code:

You can also use other YUV pixel-formats such yuv422p(4:2:2 subsampling) or yuv444p(4:4:4 subsampling) etc. instead for more higher dynamic range in the similar manner.

# import the necessary packages
from deffcode import FFdecoder
import json, cv2

# enable OpenCV patch for YUV frames
ffparams = {"-enforce_cv_patch": True}

# initialize and formulate the decoder for YUV420p output
decoder = FFdecoder(
    "input_foo.mp4", frame_format="yuv420p", verbose=True, **ffparams
).formulate()

# retrieve JSON Metadata and convert it to dict
metadata_dict = json.loads(decoder.metadata)

# prepare OpenCV parameters
FOURCC = cv2.VideoWriter_fourcc("M", "J", "P", "G")
FRAMERATE = metadata_dict["output_framerate"]
FRAMESIZE = tuple(metadata_dict["output_frames_resolution"])

# Define writer with parameters and suitable output filename for e.g. `output_foo_gray.avi`
writer = cv2.VideoWriter("output_foo_gray.avi", FOURCC, FRAMERATE, FRAMESIZE)

# grab the yuv420p frame from the decoder
for frame in decoder.generateFrame():

    # check if frame is None
    if frame is None:
        break

    # convert it to `BGR` pixel format,
    # since imshow() method only accepts `BGR` frames
    bgr = cv2.cvtColor(yuv, cv2.COLOR_YUV2BGR_I420)

    # {do something with the BGR frame here}

    # writing BGR frame to writer
    writer.write(frame)

# terminate the decoder
decoder.terminate()

# safely close writer
writer.release()

Transcoding lossless video using WriteGear API¶

High CPU Usage when chaining FFdecoder with WriteGear

When chaining FFdecoder with WriteGear, both FFmpeg processes (decoding + encoding) run as fast as your hardware allows with no artificial pacing between them. This causes the pipeline to max out your CPU to process the video in the shortest time possible, which may be undesirable.

You can mitigate this in two ways depending on your use case:

Throttle to Real-Time SpeedLimit FFmpeg Threads

Pass the -re flag via FFdecoder's -ffprefixes parameter to force FFmpeg to read the input at its native framerate. This naturally paces the pipeline to real-time speed and drastically reduces CPU usage:

# force input to be read at native framerate
decoder = FFdecoder("foo.mp4", frame_format="bgr24", **{"-ffprefixes": ["-re"]}).formulate()

Pass -threads to both FFdecoder and WriteGear to cap the number of CPU threads each FFmpeg process may use. This leaves headroom for other system tasks:

# limit decoder to 2 threads
decoder = FFdecoder("foo.mp4", frame_format="bgr24", **{"-threads": 2}).formulate()

# limit encoder to 2 threads
writer = WriteGear(output="output_foo.mp4", **{"-input_framerate": fps, "-threads": 2})

Hardware Acceleration

If your machine has a dedicated GPU, you can offload encoding to the GPU entirely — for example by passing "-vcodec": "h264_nvenc" to WriteGear (NVIDIA) — shifting the heavy lifting off the CPU.

VidGear's WriteGear API provides a flexible and robust wrapper over FFmpeg (compression mode) for encoding real-time video frames into lossless multimedia files or streams.

Combined with DeFFcode's FFdecoder API, it enables a high-level lossless FFmpeg transcoding pipeline (decoding + encoding) with full control over FFmpeg parameters and real-time frame manipulation.

In this example, we will decode video frames with different pixel formats from a given video file (e.g., foo.mp4) using the FFdecoder API, and then encode them into a lossless video file with a controlled framerate using the WriteGear API in real time.

Additional Parameters in WriteGear API

WriteGear API only requires a valid Output filename (e.g. output_foo.mp4) as input, but you can easily control any output specifications (such as bitrate, codec, framerate, resolution, subtitles, etc.) supported by FFmpeg (in use).

You can use FFdecoder's metadata property object that dumps source Video's metadata information (as JSON string) to retrieve source framerate.

BGR framesRGB framesGRAYSCALE framesYUV frames

WriteGear API by default expects BGR format frames in its write() class method.

# import the necessary packages
from deffcode import FFdecoder
from vidgear.gears import WriteGear
import json

# initialize and formulate the decoder for BGR24 output
decoder = FFdecoder("foo.mp4", frame_format="bgr24", verbose=True).formulate()

# retrieve framerate from source JSON Metadata and pass it as `-input_framerate` 
# parameter for controlled framerate
output_params = {
    "-input_framerate": json.loads(decoder.metadata)["source_video_framerate"]
}

# Define writer with default parameters and suitable
# output filename for e.g. `output_foo.mp4`
writer = WriteGear(output="output_foo.mp4", **output_params)

# grab the BGR24 frame from the decoder
for frame in decoder.generateFrame():

    # check if frame is None
    if frame is None:
        break

    # {do something with the frame here}

    # writing BGR24 frame to writer
    writer.write(frame)

# terminate the decoder
decoder.terminate()

# safely close writer
writer.close()

In WriteGear API, you can use rgb_mode parameter in write() class method to write RGB format frames instead of default BGR as follows:

# import the necessary packages
from deffcode import FFdecoder
from vidgear.gears import WriteGear
import json

# initialize and formulate the decoder
decoder = FFdecoder("foo.mp4", verbose=True).formulate()

# retrieve framerate from source JSON Metadata and pass it as `-input_framerate` 
# parameter for controlled framerate
output_params = {
    "-input_framerate": json.loads(decoder.metadata)["source_video_framerate"]
}

# Define writer with default parameters and suitable
# output filename for e.g. `output_foo.mp4`
writer = WriteGear(output="output_foo.mp4", **output_params)

# grab the BGR24 frame from the decoder
for frame in decoder.generateFrame():

    # check if frame is None
    if frame is None:
        break

    # {do something with the frame here}

    # writing RGB24 frame to writer
    writer.write(frame, rgb_mode=True)

# terminate the decoder
decoder.terminate()

# safely close writer
writer.close()

WriteGear API also directly consumes GRAYSCALE format frames in its write() class method.

# import the necessary packages
from deffcode import FFdecoder
from vidgear.gears import WriteGear
import json

# initialize and formulate the decoder for GRAYSCALE output
decoder = FFdecoder("foo.mp4", frame_format="gray", verbose=True).formulate()

# retrieve framerate from source JSON Metadata and pass it as `-input_framerate` parameter
# for controlled output framerate
output_params = {
    "-input_framerate": json.loads(decoder.metadata)["source_video_framerate"]
}

# Define writer with default parameters and suitable
# output filename for e.g. `output_foo_gray.mp4`
writer = WriteGear(output="output_foo_gray.mp4", **output_params)

# grab the GRAYSCALE frame from the decoder
for frame in decoder.generateFrame():

    # check if frame is None
    if frame is None:
        break

    # {do something with the frame here}

    # writing GRAYSCALE frame to writer
    writer.write(frame)

# terminate the decoder
decoder.terminate()

# safely close writer
writer.close()

WriteGear API also directly consume YUV (or basically any other supported pixel format) frames in its write() class method with its -input_pixfmt attribute in compression mode. For its non-compression mode, see above example.

You can also use yuv422p(4:2:2 subsampling) or yuv444p(4:4:4 subsampling) instead for more higher dynamic ranges.

In WriteGear API, the support for -input_pixfmt attribute in output_params dictionary parameter was added in v0.3.0.

# import the necessary packages
from deffcode import FFdecoder
from vidgear.gears import WriteGear
import json

# initialize and formulate the decoder for YUV420 output
decoder = FFdecoder("foo.mp4", frame_format="yuv420p").formulate()

# retrieve framerate from source JSON Metadata and pass it as 
# `-input_framerate` parameter for controlled framerate
# and add input pixfmt as yuv420p also
output_params = {
    "-input_framerate": json.loads(decoder.metadata)["output_framerate"],
    "-input_pixfmt": "yuv420p"
}

# Define writer with default parameters and suitable
# output filename for e.g. `output_foo_yuv.mp4`
writer = WriteGear(output="output_foo_yuv.mp4", logging=True, **output_params)

# grab the YUV420 frame from the decoder
for frame in decoder.generateFrame():

    # check if frame is None
    if frame is None:
        break

    # {do something with the frame here}

    # writing YUV420 frame to writer
    writer.write(frame)

# terminate the decoder
decoder.terminate()

# safely close writer
writer.close()