[SOLVED] Load Testing video streaming HLS/MPEG-DASH/Smooth : What metrics do I need?

Load Testing video streaming HLS/MPEG-DASH/Smooth : What metrics do I need?

At my company we need to load test video streaming delivering Live and VOD.

Video are delivered following 3 encoding formats:

We'd like to use Apache JMeter in order to do that. We've found UbikLoadPack plugin which is a commercial plugin.

We've been able to successfully test the 3 types of streams.

Now comes the Load Testing report, what metrics do we need to take into account and what does it provide ?

Solution

Our solution provides the following metrics besides the other metrics provided by Apache JMeter:

Buffer Fill time:This is the time user waits before video starts playing. During this time he gets a progress roller. During this time the first seconds of the video are downloaded. It’s an important metric to check as users may not wait minutes for your video.
Lag Time: When you face issues, network contention or overloaded server, at some time during the play of the video, there won’t be enough data to play by the player, so it will stop playing and wait for data to be buffered. This negatively impacts your user experience so you need to ensure that this lag time is acceptable regarding your SLA. In ideal case, lag time should be equal to buffer fill time.
Play Time : the real time video took to play provided user waits for lags
Lag ratio: is lag time / video total duration. So if user waits 30s for a video of 5 minutes. Lag ratio would be 10%
Download Time : Total time taken to download video
Hits: provides the number of HTTP request that were made when reading 1 video stream.
Average Chunk Time : provides the average chunk download time when reading 1 video stream.
Average Manifest Time : provides the average manifest download time when reading 1 video stream.

For more details you can read:

Don't hesitate to contact our support by mail for such questions, they'll be happy to help.