c++uwphololensc++-cxc++-winrt

C++/WinRT - How to get data from IBuffer


I am doing an Hololens project in C++/WinRT and I want to get usable data from an IBuffer, given by VertexPositions() of a SpatialSurfaceMesh.

I use this project sample as a reference, because it more or less does what I want (it takes the IBuffers of the meshes and renders them) The difference in my project is that I want to get the positions of meshes, not render them.

The function they use to get data from IBuffer is GetDataFromIBuffer, which is :

template <typename t = byte>
    t* GetDataFromIBuffer(Windows::Storage::Streams::IBuffer^ container)
    {
        if (container == nullptr)
        {
            return nullptr;
        }

        unsigned int bufferLength = container->Length;

        if (!(bufferLength > 0))
        {
            return nullptr;
        }

        HRESULT hr = S_OK;

        ComPtr<IUnknown> pUnknown = reinterpret_cast<IUnknown*>(container);
        ComPtr<IBufferByteAccess> spByteAccess;
        hr = pUnknown.As(&spByteAccess);
        if (FAILED(hr))
        {
            return nullptr;
        }

        byte* pRawData = nullptr;
        hr = spByteAccess->Buffer(&pRawData);
        if (FAILED(hr))
        {
            return nullptr;
        }

        return reinterpret_cast<t*>(pRawData);
    }

However this is in C++/CX, and I need to use C++/WinRT. I tried to translate it (as it is said in this Microsoft documentation, which explains how to do it and is based on the same project sample) But I can't make it work, here is how it is after translating it :

template <typename t = byte>
t* GetDataFromIBuffer(IBuffer container)
{
    if (container == nullptr)
    {
        return nullptr;
    }

    unsigned int bufferLength = container.Length();

    if (!(bufferLength > 0))
    {
        return nullptr;
    }

    HRESULT hr = S_OK;

    Microsoft::WRL::ComPtr<IUnknown> pUnknown = reinterpret_cast<IUnknown*>(&container);
    Microsoft::WRL::ComPtr<Windows::Storage::Streams::IBufferByteAccess> spByteAccess;
    hr = pUnknown.As(&spByteAccess);
    if (FAILED(hr))
    {
        return nullptr;
    }

    byte* pRawData = nullptr;
    hr = spByteAccess->Buffer(&pRawData);
    if (FAILED(hr))
    {
        return nullptr;
    }

    return reinterpret_cast<t*>(pRawData);
}

The error is Unhandled exception at 0x00007FF985543848 (ntdll.dll) in Project.exe: RangeChecks instrumentation code detected an out of range array access.

It happens at the line Microsoft::WRL::ComPtr<IUnknown> pUnknown = reinterpret_cast<IUnknown*>(&container);, more explicitely during the conversion from the IUnknown return by the reinterpret_cast to Microsoft::WRL::ComPtr<IUnknown>.

I found no way to make this work.

I am really stuck here, do you have any ideas ? Thank you !

I also found a stackoverflow post asking a somewhat similar question, but when I tried to do what the accepted answer gives :

byte* GetDataFromIBuffer(IBuffer container)
{
    if (container == nullptr)
    {
        return nullptr;
    }

    unsigned int bufferLength = container.Length();

    if (!(bufferLength > 0))
    {
        return nullptr;
    }

    HRESULT hr = S_OK;

    IUnknown* pUnk = reinterpret_cast<IUnknown*>(&container);
    Windows::Storage::Streams::IBufferByteAccess * spByteAccess = nullptr;
    hr = pUnk->QueryInterface(IID_PPV_ARGS(&spByteAccess));
    if (FAILED(hr))
    {
        return nullptr;
    }

    byte* pRawData = nullptr;
    hr = spByteAccess->Buffer(&pRawData);
    if (FAILED(hr))
    {
        return nullptr;
    }

    return reinterpret_cast<byte*>(pRawData);
}

I have the exact same error, but this time on the line : 'hr = pUnk->QueryInterface(IID_PPV_ARGS(&spByteAccess));'

EDIT (after IInspectable answer):

I don't have errors anymore, so thanks for that ! However, the result I get are incoherent, I don't get logical positions in space. What I do after getting the data from the IBuffer :

auto position = surfaceMesh.second->VertexPositions();
auto stride = position.Stride();
auto scale = surfaceMesh.second->VertexPositionScale();
auto positionCoordinateSystem = surfaceMesh.second->CoordinateSystem();

auto transform = positionCoordinateSystem.TryGetTransformTo(currentCoordinateSystem).Value();

...
// In a loop :

auto rawData = GetDataFromIBuffer(position.Data());

float x = BytesToShort(rawData + i) / scale.x;
i += stride;
float y = BytesToShort(rawData + i) / scale.y;
i += stride;
float z = BytesToShort(rawData + i) / scale.z;
i += stride * 2;

float3 vertex = MultVectByMat({ x, y, z }, transform);

Here are some of the results : See the image

It doesn't make sense to have z = 5 as it would mean the device detects a surface 5m behind itself


Solution

  • There are well-hidden extension functions to the IBuffer interface for the C++/WinRT projection, that aren't otherwise (to my knowledge) documented.

    They live a special life in, in that they get injected by the cppwinrt.exe code generator, on the fly, and don't have any machine-readable representation (in metadata, or otherwise) to support automated documentation generation.

    The one of specific interest is data():

    uint8_t* data() const;
    

    This would allow you to rewrite GetDataFromIBuffer as

    byte* GetDataFromIBuffer(IBuffer container)
    {
        return reinterpret_cast<byte*>(container ? container.data() : nullptr);
    }
    

    Internally, it does essentially the same thing you were trying to do. I think the issue in your implementation is

    IUnknown* pUnk = reinterpret_cast<IUnknown*>(&container);
    

    This reinterprets the address of a com_ptr as if it were a COM pointer. A com_ptr, however, is a struct that contains a COM pointer (as opposed to deriving from it)1.

    Applying the reinterpret_cast to the expression container.get() might work, but a better solution would be:

    auto const unk = container.as<IUnknown>();
    

    In case container derives from multiple interfaces, the as function template picks the canonical IUnknown.

    As long as you're using the C++/WinRT-supplied extension function data(), you don't have to worry about any of those intricacies.


    1 WRL's ComPtr is modeled pretty much the same way, so the reinterpret_cast in the WRL-based implementation runs into the same UB (which may or may not result in the same observable behavior).