How can you effectively pull an Amazon Kinesis Video Stream for custom python processing?

I have an RTSP stream streaming to Amazon Kinesis Video Streams. I want to be able to run custom image processing on each frame of the stream. I have the custom image processing algorithm written in python, so naturally I want to be able to pull the KVS with python, and input it to the image processing algorithm. To start, I've tried making a python app which just displays the KVS stream by following advice from this post (however this post uses Amazon Rekognition for custom processing, my use case (custom python processing) is slightly different).

My questions are:

1. How come my HLS python KVS player is laggy / choppy (randomly pauses, then plays the past few seconds very quickly)? However, the HLS streamer available here looks quite good.

2. How can you effectively pull an Amazon Kinesis Video Stream for custom python processing? What are the pros/cons of using HLS vs Amazon's GetMedia API?

   • I also looked into using Amazon's GetMedia API, and found this post where someone attempted to use GetMedia but then ended up going with HLS.
   • I found the approach to using GetMedia for custom processing explained here (python) and here (c++).

3. Is it worth using Sagemaker for this? For my image processing, I only want to perform inference, I don't need to train the network, it is already trained.

Code for kvs_player.py:

import boto3
import cv2

STREAM_NAME = "ExampleStream"
STREAM_ARN = "MY_STREAM_ARN"
AWS_REGION = 'us-east-1'


def hls_stream():

    kv_client = boto3.client("kinesisvideo", region_name=AWS_REGION)
    endpoint = kv_client.get_data_endpoint(
        StreamName=STREAM_NAME,
        APIName="GET_HLS_STREAMING_SESSION_URL"
    )['DataEndpoint']

    print(endpoint)

    # # Grab the HLS Stream URL from the endpoint
    kvam_client = boto3.client("kinesis-video-archived-media", endpoint_url=endpoint, region_name=AWS_REGION)
    url = kvam_client.get_hls_streaming_session_url(
        StreamName=STREAM_NAME,
        PlaybackMode="LIVE"
    )['HLSStreamingSessionURL']

    vcap = cv2.VideoCapture(url)

    while(True):
        # Capture frame-by-frame
        ret, frame = vcap.read()

        if frame is not None:
            # Display the resulting frame
            cv2.imshow('frame', frame)

            # Press q to close the video windows before it ends if you want
            if cv2.waitKey(22) & 0xFF == ord('q'):
                break
        else:
            print("Frame is None")
            break

    # When everything done, release the capture
    vcap.release()
    cv2.destroyAllWindows()
    print("Video stop")

if __name__ == '__main__':
    hls_stream()

Some generic answers

1. You might want to do some debugging into your application to understand what "laggy" means. It could simply be the network latency, errors while transmitting or simply performance issues running decoding, python processing and later re-encoding.
2. GetMedia is a fast API and the Parser library https://github.com/aws/amazon-kinesis-video-streams-parser-library can be used to do real-time parsing. You might need to decode the video stream which is two orders of magnitude more CPU and IO intensive operation so you might want to use some hardware accelerated decoder (or at least a good software based decoder that takes advantages of SIMD/SSE2 instruction set). JCodec is certainly not the one.
3. It depends on your application really. At any rate, before the actual Sagemaker run, you need to retrieve the fragments, parse and decode them. There is a KVS Sagemaker KIT sample you can use for more serious inference applications.

For a more scoped and targeted questions you can go directly to GitHub page for the appropriate project/asset in question and cut an issue including detailed description and the verbose logs.