Frame Compression for NetGear API⚓

Overview⚓

NetGear API enables real-time JPEG Frame Compression capabilities for optimizing performance significantly while sending frames over the network.

For enabling Frame Compression, NetGear uses powerful simplejpeg library at its backend, which is based on recent versions of libjpeg-turbo JPEG image codec, to accelerate baseline JPEG compression and decompression on all modern systems. NetGear API employs its exposed decode_jpeg and encode_jpeg methods to encode video-frames to JFIF format before sending it at Server, and cleverly decode it at the Client(s) all in real-time, thereby leveraging performance at cost of minor loss in frame quality.

Frame Compression is enabled by default in NetGear, and can be easily controlled through jpeg_compression_quality, jpeg_compression_fastdct, jpeg_compression_fastupsample like attributes of its options dictionary parameter during initialization.

Useful Information about Frame Compression

Frame Compression is enabled by default in NetGear along with fast dct and compression-quality at 90%.
Exclusive jpeg_compression attribute can also be used to disable Frame Compression.
Frame Compression can leverage performance up-to 5-10% with exclusive performance attributes.
Frame Compression is compatible with all messaging pattern and modes.

Frame Compression is primarily controlled by Server end. That means, if Frame Compression is enabled at Server, then Client(s) will automatically enforce the Frame Compression with defined performance attributes. Otherwise if it is disabled, then Client(s) disables it too.

Exclusive Attributes⚓

For implementing Frame Compression, NetGear API currently provide following exclusive attribute for its options dictionary parameter to leverage performance with Frame Compression:

jpeg_compression: (bool/str) This internal attribute is used to activate/deactivate JPEG Frame Compression as well as to specify incoming frames colorspace with compression. Its usage is as follows:
- For activating JPEG Frame Compression (Boolean):
  
  In this case, colorspace will default to BGR.
  
  You can set jpeg_compression value to False at Server end to completely disable Frame Compression.
```
# enable jpeg encoding
options = {"jpeg_compression": True}
```
- For specifying Input frames colorspace (String):
  
  In this case, JPEG Frame Compression is activated automatically.
  
  Supported colorspace values are RGB, BGR, RGBX, BGRX, XBGR, XRGB, GRAY, RGBA, BGRA, ABGR, ARGB, CMYK. More information can be found here ➶
```
# Specify incoming frames are `grayscale`
options = {"jpeg_compression": "GRAY"}
```
Performance Attributes ⚓
- jpeg_compression_quality: (int/float) This attribute controls the JPEG quantization factor. Its value varies from 10 to 100 (the higher is the better quality but performance will be lower). Its default value is 90. Its usage is as follows:
```
# activate jpeg encoding and set quality 95%
options = {"jpeg_compression": True, "jpeg_compression_quality": 95}
```
- jpeg_compression_fastdct: (bool) This attribute if True, NetGear API uses fastest DCT method that speeds up decoding by 4-5% for a minor loss in quality. Its default value is also True, and its usage is as follows:
```
# activate jpeg encoding and enable fast dct
options = {"jpeg_compression": True, "jpeg_compression_fastdct": True}
```
- jpeg_compression_fastupsample: (bool) This attribute if True, NetGear API use fastest color upsampling method. Its default value is False, and its usage is as follows:
```
# activate jpeg encoding and enable fast upsampling
options = {"jpeg_compression": True, "jpeg_compression_fastupsample": True}
```

Usage Examples⚓

Bare-Minimum Usage⚓

Following is the bare-minimum code you need to get started with Frame Compression in NetGear API:

Server End⚓

Open your favorite terminal and execute the following python code:

You can terminate both sides anytime by pressing Ctrl+C on your keyboard!

# import required libraries
from vidgear.gears import VideoGear
from vidgear.gears import NetGear
import cv2

# open any valid video stream(for e.g `test.mp4` file)
stream = VideoGear(source="test.mp4").start()

# activate jpeg encoding and specify other related parameters
options = {
    "jpeg_compression": True,
    "jpeg_compression_quality": 90,
    "jpeg_compression_fastdct": True,
    "jpeg_compression_fastupsample": True,
}

# Define NetGear Server with defined parameters
server = NetGear(pattern=1, logging=True, **options)

# loop over until KeyBoard Interrupted
while True:

    try:
        # read frames from stream
        frame = stream.read()

        # check for frame if None-type
        if frame is None:
            break

        # {do something with the frame here}

        # send frame to server
        server.send(frame)

    except KeyboardInterrupt:
        break

# safely close video stream
stream.stop()

# safely close server
server.close()

Client End⚓

Then open another terminal on the same system and execute the following python code and see the output:

You can terminate client anytime by pressing Ctrl+C on your keyboard!

If compression is enabled at Server, then Client will automatically enforce Frame Compression with its performance attributes.

# import required libraries
from vidgear.gears import NetGear
import cv2

# define NetGear Client with `receive_mode = True` and defined parameter
client = NetGear(receive_mode=True, pattern=1, logging=True)

# loop over
while True:

    # receive frames from network
    frame = client.recv()

    # check for received frame if Nonetype
    if frame is None:
        break

    # {do something with the frame here}

    # Show output window
    cv2.imshow("Output Frame", frame)

    # check for 'q' key if pressed
    key = cv2.waitKey(1) & 0xFF
    if key == ord("q"):
        break

# close output window
cv2.destroyAllWindows()

# safely close client
client.close()

Bare-Minimum Usage with Variable Colorspace⚓

Frame Compression also supports specify incoming frames colorspace with compression. In following bare-minimum code, we will be sending GRAY frames from Server to Client:

New in v0.2.2

This example was added in v0.2.2.

This example works in conjunction with Source ColorSpace manipulation for VideoCapture Gears ➶

Supported colorspace values are RGB, BGR, RGBX, BGRX, XBGR, XRGB, GRAY, RGBA, BGRA, ABGR, ARGB, CMYK. More information can be found here ➶

Server End⚓

Open your favorite terminal and execute the following python code:

You can terminate both sides anytime by pressing Ctrl+C on your keyboard!

# import required libraries
from vidgear.gears import VideoGear
from vidgear.gears import NetGear
import cv2

# open any valid video stream(for e.g `test.mp4` file) and change its colorspace to grayscale
stream = VideoGear(source="test.mp4", colorspace="COLOR_BGR2GRAY").start()

# activate jpeg encoding and specify other related parameters
options = {
    "jpeg_compression": "GRAY", # set grayscale
    "jpeg_compression_quality": 90,
    "jpeg_compression_fastdct": True,
    "jpeg_compression_fastupsample": True,
}

# Define NetGear Server with defined parameters
server = NetGear(pattern=1, logging=True, **options)

# loop over until KeyBoard Interrupted
while True:

    try:
        # read grayscale frames from stream
        frame = stream.read()

        # check for frame if None-type
        if frame is None:
            break

        # {do something with the frame here}

        # send grayscale frame to server
        server.send(frame)

    except KeyboardInterrupt:
        break

# safely close video stream
stream.stop()

# safely close server
server.close()

Client End⚓

Then open another terminal on the same system and execute the following python code and see the output:

You can terminate client anytime by pressing Ctrl+C on your keyboard!

If compression is enabled at Server, then Client will automatically enforce Frame Compression with its performance attributes.

Client's end also automatically enforces Server's colorspace, there's no need to define it again.

# import required libraries
from vidgear.gears import NetGear
import cv2

# define NetGear Client with `receive_mode = True` and defined parameter
client = NetGear(receive_mode=True, pattern=1, logging=True)

# loop over
while True:

    # receive grayscale frames from network
    frame = client.recv()

    # check for received frame if Nonetype
    if frame is None:
        break

    # {do something with the grayscale frame here}

    # Show output window
    cv2.imshow("Output Grayscale Frame", frame)

    # check for 'q' key if pressed
    key = cv2.waitKey(1) & 0xFF
    if key == ord("q"):
        break

# close output window
cv2.destroyAllWindows()

# safely close client
client.close()

Using Frame Compression with Variable Parameters⚓

Client's End⚓

Open a terminal on Client System (where you want to display the input frames received from the Server) and execute the following python code:

Note down the local IP-address of this system(required at Server's end) and also replace it in the following code. You can follow this FAQ for this purpose.

If compression is enabled at Server, then Client will automatically enforce Frame Compression with its performance attributes.

You can terminate client anytime by pressing Ctrl+C on your keyboard!

# import required libraries
from vidgear.gears import NetGear
import cv2


# Define NetGear Client at given IP address and define parameters 
# !!! change following IP address '192.168.x.xxx' with yours !!!
client = NetGear(
    address="192.168.x.xxx",
    port="5454",
    protocol="tcp",
    pattern=1,
    receive_mode=True,
    logging=True,
    **options
)

#  loop over
while True:

    # receive frames from network
    frame = client.recv()

    # check for received frame if Nonetype
    if frame is None:
        break

    # {do something with the frame here}

    # Show output window
    cv2.imshow("Output Frame", frame)

    # check for 'q' key if pressed
    key = cv2.waitKey(1) & 0xFF
    if key == ord("q"):
        break

# close output window
cv2.destroyAllWindows()

# safely close client
client.close()

Server End⚓

Now, Open the terminal on another Server System (with a webcam connected to it at index 0), and execute the following python code:

Replace the IP address in the following code with Client's IP address you noted earlier.

You can terminate stream on both side anytime by pressing Ctrl+C on your keyboard!

# import required libraries
from vidgear.gears import VideoGear
from vidgear.gears import NetGear
import cv2

# activate jpeg encoding and specify other related parameters
options = {
    "jpeg_compression": True,
    "jpeg_compression_quality": 90,
    "jpeg_compression_fastdct": True,
    "jpeg_compression_fastupsample": True,
}

# Open live video stream on webcam at first index(i.e. 0) device
stream = VideoGear(source=0).start()

# Define NetGear server at given IP address and define parameters 
# !!! change following IP address '192.168.x.xxx' with client's IP address !!!
server = NetGear(
    address="192.168.x.xxx",
    port="5454",
    protocol="tcp",
    pattern=1,
    logging=True,
    **options
)

# loop over until KeyBoard Interrupted
while True:

    try:
        # read frames from stream
        frame = stream.read()

        # check for frame if Nonetype
        if frame is None:
            break

        # {do something with the frame here}

        # send frame to server
        server.send(frame)

    except KeyboardInterrupt:
        break

# safely close video stream
stream.stop()

# safely close server
server.close()

Using Bidirectional Mode for Video-Frames Transfer with Frame Compression ⚓

NetGear now supports Dual Frame Compression for transferring video-frames with its exclusive Bidirectional Mode for achieving unmatchable performance bidirectionally. You can easily enable Frame Compression with its performance attributes at both ends to boost performance bidirectionally.

In this example we are going to implement a bare-minimum example, where we will be sending video-frames (3-Dimensional numpy arrays) of the same Video bidirectionally at the same time for testing the real-time performance and synchronization between the Server and Client using Bidirectional Mode. Furthermore, we're going to use optimal Dual Frame Compression Setting for Sending and Receiving frames at both Server and Client end.

This example is great for building applications like Real-time Video Chat System.

This Dual Frame Compression feature also available for Multi-Clients Mode.

We're also using reducer() Helper method for reducing frame-size on-the-go for additional performance.

Remember to define Frame Compression's performance attributes both on Server and Client ends in Dual Frame Compression to boost performance bidirectionally!

Server End⚓

Open your favorite terminal and execute the following python code:

You can terminate both side anytime by pressing Ctrl+C on your keyboard!

# import required libraries
from vidgear.gears import NetGear
from vidgear.gears.helper import reducer
import numpy as np
import cv2

# open any valid video stream(for e.g `test.mp4` file)
stream = cv2.VideoCapture("test.mp4")

# activate Bidirectional mode and Frame Compression
options = {
    "bidirectional_mode": True,
    "jpeg_compression": True,
    "jpeg_compression_quality": 95,
    "jpeg_compression_fastdct": True,
    "jpeg_compression_fastupsample": True,
}

# Define NetGear Server with defined parameters
server = NetGear(pattern=1, logging=True, **options)

# loop over until KeyBoard Interrupted
while True:

    try:
        # read frames from stream
        (grabbed, frame) = stream.read()

        # check for frame if not grabbed
        if not grabbed:
            break

        # reducer frames size if you want even more performance, otherwise comment this line
        frame = reducer(frame, percentage=20)  # reduce frame by 20%

        # {do something with the frame here}

        # prepare data to be sent(a simple text in our case)
        target_data = "Hello, I am a Server."

        # send frame & data and also receive data from Client
        recv_data = server.send(frame, message=target_data) # (1)

        # check data just received from Client is of numpy datatype
        if not (recv_data is None) and isinstance(recv_data, np.ndarray):

            # {do something with received numpy array here}

            # Let's show it on output window
            cv2.imshow("Received Frame", recv_data)
            key = cv2.waitKey(1) & 0xFF

    except KeyboardInterrupt:
        break

# safely close video stream
stream.release()

# safely close server
server.close()

Everything except numpy.ndarray datatype data is accepted as target_data in message parameter.

Client End⚓

Then open another terminal on the same system and execute the following python code and see the output:

You can terminate client anytime by pressing Ctrl+C on your keyboard!

# import required libraries
from vidgear.gears import NetGear
from vidgear.gears.helper import reducer
import cv2

# activate Bidirectional mode and Frame Compression
options = {
    "bidirectional_mode": True,
    "jpeg_compression": True,
    "jpeg_compression_quality": 95,
    "jpeg_compression_fastdct": True,
    "jpeg_compression_fastupsample": True,
}

# again open the same video stream
stream = cv2.VideoCapture("test.mp4")

# define NetGear Client with `receive_mode = True` and defined parameter
client = NetGear(receive_mode=True, pattern=1, logging=True, **options)

# loop over
while True:

    # read frames from stream
    (grabbed, frame) = stream.read()

    # check for frame if not grabbed
    if not grabbed:
        break

    # reducer frames size if you want even more performance, otherwise comment this line
    frame = reducer(frame, percentage=20)  # reduce frame by 20%

    # receive data from server and also send our data
    data = client.recv(return_data=frame)

    # check for data if None
    if data is None:
        break

    # extract server_data & frame from data
    server_data, frame = data

    # again check for frame if None
    if frame is None:
        break

    # {do something with the extracted frame and data here}

    # lets print extracted server data
    if not (server_data is None):
        print(server_data)

    # Show output window
    cv2.imshow("Output Frame", frame)

    # check for 'q' key if pressed
    key = cv2.waitKey(1) & 0xFF
    if key == ord("q"):
        break

# close output window
cv2.destroyAllWindows()

# safely close video stream
stream.release()

# safely close client
client.close()

Last update: February 17, 2022

Frame Compression for NetGear API⚓

Overview⚓

Exclusive Attributes⚓

Performance Attributes ⚓

Usage Examples⚓

Bare-Minimum Usage⚓

Server End⚓

Client End⚓

Bare-Minimum Usage with Variable Colorspace⚓

Server End⚓

Client End⚓

Using Frame Compression with Variable Parameters⚓

Client's End⚓

Server End⚓

Using Bidirectional Mode for Video-Frames Transfer with Frame Compression ⚓

Server End⚓

Client End⚓