windowsvideovideo-capturems-media-foundationuvc

MS Media Foundation Session default presenter - video aspect ratio is wrong

Learning (painfully) how to use the Topology and Session to present a captured video. Nothing fancy - just select a web cam, list its modes, choose a video format and hit "go". In general these are the steps I'm taking to present the video capture:

  1. List the available devices with MFEnumDeviceSources() filtered by MF_DEVSOURCE_ATTRIBUTE_SOURCE_TYPE_VIDCAP_GUID and let user select one
  2. List the streams based on GetStreamDescriptorCount() after activating the source and creating Presentation Descriptors with CreatePresentationDescriptor() and let the user select a stream (if multiple streams are available)
  3. Display all the supported video formats based on the list of IMFMediaTypes available through GetMediaTypeByIndex() and let the user choose one

Once the exact format is selected I build the Topology this way:

  1. call MFCreateTopology() to create a new IMFTopology object
  2. create media sink activation with MFCreateVideoRendererActivate()
    1. making sure to call SetCurrentMediaType() on the IMFMediaTypeHandler object of the currently selected IMFStreamDescriptor
  3. create the source node with MFCreateTopologyNode(), setting its presentation and stream descriptors with the call to SetUnknown() and adding that node to the Topology
    1. made sure to set its current IMFMediaType to the one chosen by user with SetCurrentMediaType()
  4. create an output node and call SetObject() of it providing the previously created media sink activation object (from step 5 above)
  5. connect the source to output with ConnectOutput() providing it with the node ID 0

When the "preview" button is clicked a session IMFMediaSession object (created at the app's startup) is set with the new Topology

m_session->SetTopology(MFSESSION_SETTOPOLOGY_IMMEDIATE, pTopo);

And this is where I see something strange. I set the size of the preview video based on the frame dimensions provided by the IMFMediaType that the user chose and the source seems to be producing the video in that format. However the renderer is handling the pixel aspect ratio incorrectly and is letterboxing/pillarboxing the video while presenting the picture as being stretched either vertically of horizontally.

For the life of me I am unable to find the way to tell the renderer to adjust for the proper pixel aspect ratio (which was set to the proper value in step 5.1 above)

The MS's SDK example only shows how to present captured video for uncompressed formats - it doesn't use the IMFMediaSession object and works fine for the YUV2 format but not for the MJPG. In fact it worked so well that I thought moving onto using the session would be easy :)

Using the IMFMediaSession seems to be the least cumbersome way of supporting compressed video formats, especially the H.264 that's in the newer UVC 1.5 standard and any new (H.265?) formats that are coming up in the future.

Solution

  • Further research has shown that one can access the renderer object by querying the session to get the service and then use that service to get the video display control. That IMFVideoDisplayControl is the object that lets you control various aspects of the rendered video, including the aspect ratio of the image.

    Here's how to get to that IMFVideoDisplayControl object:

    1. get the IMFGetService:

      hr = m_session->QueryInterface<IMFGetService>(&service);

    2. get the IMFVideoDisplayControl:

      hr = service->GetService(MR_VIDEO_RENDER_SERVICE, IID_IMFVideoDisplayControl, (void**)&control);

    Once you have the display control object you can achieve the rest. In my case (since I knew the actual true geometry of the expected video) the solution was to set the aspect ratio mode to MFVideoARMode_None