CUVID Video Codec SDK RTP packets decoding

I'm decoding each RTP packet to see if it contains IDR NALU and when it does I pass NALUs (PPS, SPS, I-frame in the case) to decoding pipeline where I run NVIDIA Video Codec SDK

RTSP stream is h264, SVC off. So format selection for the decoder is out of question I believe

NvDecoder dec(cuContext, true, cudaVideoCodec_H264);
nFrameReturned = dec.Decode(nalu.data, nalu.len);

Decode function:

int NvDecoder::Decode(const uint8_t *pData, int nSize, int nFlags, int64_t nTimestamp)
{
    m_nDecodedFrame = 0;
    m_nDecodedFrameReturned = 0;
    CUVIDSOURCEDATAPACKET packet = { 0 };
    packet.payload = pData;
    packet.payload_size = nSize;
    packet.flags = nFlags | CUVID_PKT_TIMESTAMP;
    packet.timestamp = nTimestamp;
    if (!pData || nSize == 0) {
        packet.flags |= CUVID_PKT_ENDOFSTREAM;
    }
    NVDEC_API_CALL(cuvidParseVideoData(m_hParser, &packet));

    return m_nDecodedFrame;
}

However nFrameReturned is always zero, thus no frames are decoded

I suppose NALUs should be formatted in some way before being passed to the decoder?

I've tried flattening NALUs into 1d array, sending I-frame NALU only - the result is the same, no frames are decoded

I've figured it out.

Apparently I've been missing NALU markers

Adding the markers and flattening the data to 1d array works as expected

For anyone stumbling on this, this is how I append markers in go:

nalu = append([]uint8{0x00, 0x00, 0x00, 0x01}, []uint8(nalu)...)