I'm trying to upload a large video (around 1.5 GB) through Video Indexer API. My machine however takes up lot of RAM to do so. The deployment system has quite a small amount of RAM. I want to use the API so that the video is uploaded in multiple parts without using up too much memory (around 100MB would suffice).

I've tried to use ffmpeg to split the video in chunks and upload it piece by piece but Video Indexer recognizes them as different videos and gives separate insights for each. It would be better if the video is aggregated online.

How can I do chunked video upload to MS Video Indexer?

Let my guess. Previously, you followed the offical tutorial Tutorial: Use the Video Indexer API and the Upload Video API reference (the Python sample code at the end of API reference page as the figure below) to upload your large video.

It cost a lot of memory because the code below send the data block {body} read from memory, and its value comes from the code open("<your local file name>").read().

conn.request("POST", "/{location}/Accounts/{accountId}/Videos?name={name}&accessToken={accessToken}&%s" % params, "{body}", headers)

However, if you read the subsection videoUrl of the document Upload and index your videos and the following C# code carefully, even the explaination for videoUrl in API reference, you will see the video file passed as a multipart/form body content is not the only way.

videoUrl

A URL of the video/audio file to be indexed. The URL must point at a media file (HTML pages are not supported). The file can be protected by an access token provided as part of the URI and the endpoint serving the file must be secured with TLS 1.2 or higher. The URL needs to be encoded.

If the videoUrl is not specified, the Video Indexer expects you to pass the file as a multipart/form body content.

The screenshot for the C# code with videoUrl

The screenshot for the videoUrl parameter in API reference

You can first upload a large video file to Azure Blob Storage or other online services satisfied the videoUrl requirement via Python streaming upload code or other tools like azcopy or Azure Storage Explorer, then using Azure Blob Storage as example to generate a blob url with sas token (Python code as below) to pass it as videoUrl to API request for uploading.

Python code for generating a blob url with sas token

from azure.storage.blob.baseblobservice import BaseBlobService
from azure.storage.blob import BlockBlobService, BlobPermissions
from datetime import datetime, timedelta

account_name = '<your account name>'
account_key = '<your account key>'
container_name = '<your container name>'
blob_name = '<your blob name>'
service = BaseBlobService(account_name=account_name, account_key=account_key)

token = service.generate_blob_shared_access_signature(container_name, blob_name, BlobPermissions.READ, datetime.utcnow() + timedelta(hours=1),)
blobUrlWithSas = f"https://{account_name}.blob.core.windows.net/{container_name}/{blob_name}?{token}"

Hope it helps.