Skip to content

Bidirectional Mode for NetGear API

Bidirectional Mode
NetGear's Bidirectional Mode

Overview

Bidirectional Mode enables seamless support for Bidirectional data transmission between Client/Consumer and Sender/Publisher along with video-frames through its synchronous messaging patterns such as zmq.PAIR (ZMQ Pair Pattern) & zmq.REQ/zmq.REP (ZMQ Request/Reply Pattern).

In Bidirectional Mode, we utilizes the NetGear API's message parameter of send() method for sending data from Server-to-Client, and return_data parameter of recv() method to return data back from Client-to-Server all while transferring frames in real-time.

This mode can be easily activated in NetGear through bidirectional_mode attribute of its options dictionary parameter during initialization.

 

Important Information regarding Bidirectional Mode

  • In Bidirectional Mode, zmq.PAIR(ZMQ Pair) & zmq.REQ/zmq.REP(ZMQ Request/Reply) are ONLY Supported messaging patterns. Accessing this mode with any other messaging pattern, will result in ValueError.

  • Bidirectional Mode enables you to send data of ANY1 Data-type along with frame bidirectionally.

  • Bidirectional Mode may lead to additional LATENCY depending upon the size of data being transfer bidirectionally. User discretion is advised!

  • Bidirectional Mode is smart enough to sense data (if available or not), and DOES NOT interfere with transferring of video-frames (unless data is huge), as both mechanisms works independently.

  • Bidirectional Mode is now compatibile with both Multi-Servers mode and Multi-Clients mode exclusive modes.

 

 

Features of Bidirectional Mode

  • Enables easy-to-use seamless Bidirectional data transmission between two systems.

  • Supports zmq.PAIR & zmq.REQ/zmq.REP messaging patterns.

  • Support for sending data of almost any1 datatype.

  • Auto-enables reconnection if Server or Client disconnects prematurely.

 

 

Exclusive Parameters

To send data bidirectionally, NetGear API provides two exclusive parameters for its methods:

  • message: It enables user to send data to Client, directly through send() method at Server's end.

  • return_data: It enables user to send data back to Server, directly through recv() method at Client's end.

 

 

Usage Examples

Bare-Minimum Usage

Following is the bare-minimum code you need to get started with Bidirectional Mode in NetGear API:

Server End

Open your favorite terminal and execute the following python code:

You can terminate both sides anytime by pressing Ctrl+C on your keyboard!

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
# import required libraries
from vidgear.gears import VideoGear
from vidgear.gears import NetGear

# open any valid video stream(for e.g `test.mp4` file)
stream = VideoGear(source="test.mp4").start()

# activate Bidirectional mode
options = {"bidirectional_mode": True}

# Define NetGear Server with defined parameters
server = NetGear(logging=True, **options)

# loop over until KeyBoard Interrupted
while True:

    try:
        # read frames from stream
        frame = stream.read()

        # check for frame if Nonetype
        if frame is None:
            break

        # {do something with the frame here}

        # prepare data to be sent(a simple text in our case)
        target_data = "Hello, I am a Server."

        # send frame & data and also receive data from Client
        recv_data = server.send(frame, message=target_data) # (1)

        # print data just received from Client
        if not (recv_data is None):
            print(recv_data)

    except KeyboardInterrupt:
        break

# safely close video stream
stream.stop()

# safely close server
server.close()
  1. âš  Everything except numpy.ndarray datatype data is accepted as target_data in message parameter.

Client End

Then open another terminal on the same system and execute the following python code and see the output:

You can terminate client anytime by pressing Ctrl+C on your keyboard!

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
# import required libraries
from vidgear.gears import NetGear
import cv2

# activate Bidirectional mode
options = {"bidirectional_mode": True}

# define NetGear Client with `receive_mode = True` and defined parameter
client = NetGear(receive_mode=True, logging=True, **options)

# loop over
while True:

    # prepare data to be sent
    target_data = "Hi, I am a Client here."

    # receive data from server and also send our data
    data = client.recv(return_data=target_data)

    # check for data if None
    if data is None:
        break

    # extract server_data & frame from data
    server_data, frame = data

    # again check for frame if None
    if frame is None:
        break

    # {do something with the extracted frame and data here}

    # lets print extracted server data
    if not (server_data is None):
        print(server_data)

    # Show output window
    cv2.imshow("Output Frame", frame)

    # check for 'q' key if pressed
    key = cv2.waitKey(1) & 0xFF
    if key == ord("q"):
        break

# close output window
cv2.destroyAllWindows()

# safely close client
client.close()

 

 

Using Bidirectional Mode with Variable Parameters

Client's End

Open a terminal on Client System (where you want to display the input frames received from the Server) and execute the following python code:

Note down the local IP-address of this system(required at Server's end) and also replace it in the following code. You can follow this FAQ for this purpose.

You can terminate client anytime by pressing Ctrl+C on your keyboard!

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
# import required libraries
from vidgear.gears import NetGear
import cv2

# activate Bidirectional mode
options = {"bidirectional_mode": True}

# Define NetGear Client at given IP address and define parameters 
# !!! change following IP address '192.168.x.xxx' with yours !!!
client = NetGear(
    address="192.168.x.xxx",
    port="5454",
    protocol="tcp",
    pattern=1,
    receive_mode=True,
    logging=True,
    **options
)

# loop over
while True:

    # prepare data to be sent
    target_data = "Hi, I am a Client here."

    # receive data from server and also send our data
    data = client.recv(return_data=target_data)

    # check for data if None
    if data is None:
        break

    # extract server_data & frame from data
    server_data, frame = data

    # again check for frame if None
    if frame is None:
        break

    # {do something with the extracted frame and data here}

    # lets print received server data
    if not (server_data is None):
        print(server_data)

    # Show output window
    cv2.imshow("Output Frame", frame)

    # check for 'q' key if pressed
    key = cv2.waitKey(1) & 0xFF
    if key == ord("q"):
        break

# close output window
cv2.destroyAllWindows()

# safely close client
client.close()

 

Server End

Now, Open the terminal on another Server System (a Raspberry Pi with Camera Module), and execute the following python code:

Replace the IP address in the following code with Client's IP address you noted earlier.

You can terminate stream on both side anytime by pressing Ctrl+C on your keyboard!

Backend PiGear API now fully supports the newer picamera2 python library under the hood for Raspberry Pi camera modules. Follow this guide ➶ for its installation.

Make sure to complete Raspberry Pi Camera Hardware-specific settings prior using this backend, otherwise nothing will work.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
# import required libraries
from vidgear.gears import VideoGear
from vidgear.gears import NetGear
from vidgear.gears import PiGear
from libcamera import Transform

# add various Picamera2 API tweaks
options = {
    "queue": True,
    "buffer_count": 4,
    "controls": {"Brightness": 0.5, "ExposureValue": 2.0},
    "transform": Transform(hflip=1),
    "auto_align_output_config": True,  # auto-align camera configuration
}

# open pi video stream with defined parameters
stream = PiGear(resolution=(640, 480), framerate=60, logging=True, **options).start()

# activate Bidirectional mode
options = {"bidirectional_mode": True}

# Define NetGear server at given IP address and define parameters 
# !!! change following IP address '192.168.x.xxx' with client's IP address !!!
server = NetGear(
    address="192.168.x.xxx",
    port="5454",
    protocol="tcp",
    pattern=1,
    logging=True,
    **options
)

# loop over until KeyBoard Interrupted
while True:

    try:
        # read frames from stream
        frame = stream.read()

        # check for frame if Nonetype
        if frame is None:
            break

        # {do something with the frame here}

        # prepare data to be sent(a simple text in our case)
        target_data = "Hello, I am a Server."

        # send frame & data and also receive data from Client
        recv_data = server.send(frame, message=target_data) # (1)

        # print data just received from Client
        if not (recv_data is None):
            print(recv_data)

    except KeyboardInterrupt:
        break

# safely close video stream
stream.stop()

# safely close server
server.close()
  1. âš  Everything except numpy.ndarray datatype data is accepted as target_data in message parameter.
Under the hood, Backend PiGear API (version 0.3.3 onwards) prioritizes the new picamera2 API backend.

However, the API seamlessly switches to the legacy picamera backend, if the picamera2 library is unavailable or not installed.

It is advised to enable logging(logging=True) to see which backend is being used.

The picamera library is built on the legacy camera stack that is NOT (and never has been) supported on 64-bit OS builds.

You could also enforce the legacy picamera API backend in PiGear by using the enforce_legacy_picamera user-defined optional parameter boolean attribute.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
# import required libraries
from vidgear.gears import VideoGear
from vidgear.gears import NetGear
from vidgear.gears import PiGear

# add various Picamera tweak parameters to dictionary
options = {
    "hflip": True,
    "exposure_mode": "auto",
    "iso": 800,
    "exposure_compensation": 15,
    "awb_mode": "horizon",
    "sensor_mode": 0,
}

# open pi video stream with defined parameters
stream = PiGear(resolution=(640, 480), framerate=60, logging=True, **options).start()

# activate Bidirectional mode
options = {"bidirectional_mode": True}

# Define NetGear server at given IP address and define parameters 
# !!! change following IP address '192.168.x.xxx' with client's IP address !!!
server = NetGear(
    address="192.168.x.xxx",
    port="5454",
    protocol="tcp",
    pattern=1,
    logging=True,
    **options
)

# loop over until KeyBoard Interrupted
while True:

    try:
        # read frames from stream
        frame = stream.read()

        # check for frame if Nonetype
        if frame is None:
            break

        # {do something with the frame here}

        # prepare data to be sent(a simple text in our case)
        target_data = "Hello, I am a Server."

        # send frame & data and also receive data from Client
        recv_data = server.send(frame, message=target_data) # (1)

        # print data just received from Client
        if not (recv_data is None):
            print(recv_data)

    except KeyboardInterrupt:
        break

# safely close video stream
stream.stop()

# safely close server
server.close()
  1. âš  Everything except numpy.ndarray datatype data is accepted as target_data in message parameter.

 

 

Using Bidirectional Mode for Video-Frames Transfer

In this example we are going to implement a bare-minimum example, where we will be sending video-frames (3-Dimensional numpy arrays) of the same Video bidirectionally at the same time, for testing the real-time performance and synchronization between the Server and the Client using this(Bidirectional) Mode.

This example is useful for building applications like Real-Time Video Chat.

We're also using reducer() method for reducing frame-size on-the-go for additional performance.

Remember, Sending large HQ video-frames may required more network bandwidth and packet size which may lead to video latency!

Server End

Open your favorite terminal and execute the following python code:

You can terminate both side anytime by pressing Ctrl+C on your keyboard!

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
# import required libraries
from vidgear.gears import NetGear
from vidgear.gears.helper import reducer
import numpy as np
import cv2

# open any valid video stream(for e.g `test.mp4` file)
stream = cv2.VideoCapture("test.mp4")

# activate Bidirectional mode
options = {"bidirectional_mode": True}

# Define NetGear Server with defined parameters
server = NetGear(pattern=1, logging=True, **options)

# loop over until KeyBoard Interrupted
while True:

    try:
        # read frames from stream
        (grabbed, frame) = stream.read()

        # check for frame if not grabbed
        if not grabbed:
            break

        # reducer frames size if you want more performance, otherwise comment this line
        frame = reducer(frame, percentage=30)  # reduce frame by 30%

        # {do something with the frame here}

        # prepare data to be sent(a simple text in our case)
        target_data = "Hello, I am a Server."

        # send frame & data and also receive data from Client
        recv_data = server.send(frame, message=target_data) # (1)

        # check data just received from Client is of numpy datatype
        if not (recv_data is None) and isinstance(recv_data, np.ndarray):

            # {do something with received numpy array here}

            # Let's show it on output window
            cv2.imshow("Received Frame", recv_data)
            key = cv2.waitKey(1) & 0xFF

    except KeyboardInterrupt:
        break

# safely close video stream
stream.release()

# safely close server
server.close()
  1. âš  Everything except numpy.ndarray datatype data is accepted as target_data in message parameter.

 

Client End

Then open another terminal on the same system and execute the following python code and see the output:

You can terminate client anytime by pressing Ctrl+C on your keyboard!

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
# import required libraries
from vidgear.gears import NetGear
from vidgear.gears.helper import reducer
import cv2

# activate Bidirectional mode
options = {"bidirectional_mode": True}

# again open the same video stream
stream = cv2.VideoCapture("test.mp4")

# define NetGear Client with `receive_mode = True` and defined parameter
client = NetGear(receive_mode=True, pattern=1, logging=True, **options)

# loop over
while True:

    # read frames from stream
    (grabbed, frame) = stream.read()

    # check for frame if not grabbed
    if not grabbed:
        break

    # reducer frames size if you want more performance, otherwise comment this line
    frame = reducer(frame, percentage=30)  # reduce frame by 30%

    # receive data from server and also send our data
    data = client.recv(return_data=frame)

    # check for data if None
    if data is None:
        break

    # extract server_data & frame from data
    server_data, frame = data

    # again check for frame if None
    if frame is None:
        break

    # {do something with the extracted frame and data here}

    # lets print extracted server data
    if not (server_data is None):
        print(server_data)

    # Show output window
    cv2.imshow("Output Frame", frame)

    # check for 'q' key if pressed
    key = cv2.waitKey(1) & 0xFF
    if key == ord("q"):
        break

# close output window
cv2.destroyAllWindows()

# safely close video stream
stream.release()

# safely close client
client.close()

 

Using Bidirectional Mode for Video-Frames Transfer with Frame Compression 🔥

This usage examples can be found here ➶

 

 

  1. Additional data of numpy.ndarray data-type is ONLY SUPPORTED at Client's end with its return_data parameter.

    ↩↩