EDITS ADDED AT END***
I am looking for the 'simplest' way to configure an aac encoder (FDK-AAC library), sdp header, and RTP headers so that the minimum viable RTP audio stream can be played. (To avoid subjectivity, by simplest, I mean requires the least amount of manual overhead/config and setup work to get streaming to a player like ffplay or vlc)
I am getting single channel 16b PCM data as input. I have already confirmed I am able to encode that and dump the bitstream to an aac file which plays fine with ffplay using the following configuration options for fdk-aac and a pretty standard encoding loop:
aacEncoder_SetParam(aac_handle, AACENC_AOT, AOT_AAC_LC);
aacEncoder_SetParam(aac_handle, AACENC_BITRATE, 96000);
aacEncoder_SetParam(aac_handle, AACENC_SAMPLERATE, 16000);
aacEncoder_SetParam(aac_handle, AACENC_CHANNELMODE, MODE_1);
aacEncoder_SetParam(aac_handle, AACENC_BANDWIDTH, 0);
aacEncoder_SetParam(aac_handle, AACENC_CHANNELORDER, 1);
aacEncoder_SetParam(aac_handle, AACENC_AFTERBURNER, 1);
aacEncoder_SetParam(aac_handle, AACENC_SBR_MODE, 0);
aacEncoder_SetParam(aac_handle, AACENC_SBR_RATIO, 0);
aacEncoder_SetParam(aac_handle, AACENC_TRANSMUX, TT_MP4_LOAS);
aacEncoder_SetParam(aac_handle, AACENC_SIGNALING_MODE, 0);
I have also confirmed that I am able to stream the raw PCM data to ffplay using the following SDP and RTP header setup:
char sdp[] = "v=0\r\n"
"o=- 0 0 IN IP4 127.0.0.1\r\n"
"s=Unnamed\r\n"
"c=IN IP4 127.0.0.1\r\n"
"a=recvonly\r\n"
"a=charset:UTF-8\r\n"
"m=audio 50040 RTP/AVP 100\r\n"
"a=rtpmap:100 L16/16000\r\n";
// Header setup (in send loop):
hdr.flags = 0x80;
hdr.mk_pt = 0x80 | 100;
hdr.sq_nb = htons(rtp_packet_count);
hdr.ts = htonl(rtp_timestamp_audio);
hdr.ssrc = htonl(10);
...
// packet count and timestamp updated later
I have not had any success trying to stream AAC with RT(S)P however, despite different encoder configurations (mostly changing the transmux param to try TT_MP4_LATM_MCP1 and others), SDP packets, and RTP headers. Most recent example of an SDP/RTP header I have tried to use is:
char sdp_aac[] =
"v=0\r\n"
"o=- 0 0 IN IP4 0.0.0.0\r\n"
"s=Unnamed\r\n"
"c=IN IP4 0.0.0.0\r\n"
"a=recvonly\r\n"
"a=charset:UTF-8\r\n"
"m=audio 0 RTP/AVP 97\r\n"
"a=rtpmap:97 MP4A-LATM/16000/1\r\n"
"a=fmtp:97 cpresent=1; config=400028100000\r\n";
// Header setup (in send loop):
hdr.flags = 0x80;
hdr.mk_pt = 0x80 | 97;
hdr.sq_nb = htons(rtp_packet_count);
hdr.ts = htonl(rtp_timestamp_audio);
hdr.ssrc = htonl(10);
This results in ffplay outputting a long dump of errors as long as I leave the stream trying to play:
[rtsp @ 0x7f79e4000bc0] RTP MP4A-LATM with in-band configuration is not implemented. Update your FFmpeg version to the newest one from Git. If the problem still occurs, it means that your file has a feature which has not been implemented.
[rtsp @ 0x7f79e4000bc0] If you want to help, upload a sample of this file to ftp://upload.ffmpeg.org/incoming/ and contact the ffmpeg-devel mailing list. (ffmpeg-devel@ffmpeg.org)
[aac @ 0x7f79e40046c0] Number of bands (31) exceeds limit (16)./0
Input #0, rtsp, from 'rtsp://10.66.171.29':
Metadata:
title : Unnamed
Duration: N/A, start: 0.000000, bitrate: N/A
Stream #0:0: Audio: aac (LC), 16000 Hz, mono, fltp
[rtsp @ 0x7f79e4000bc0] Malformed LATM packet
Last message repeated 1 times
[aac @ 0x7f79e40b6680] Number of bands (31) exceeds limit (16).
[rtsp @ 0x7f79e4000bc0] Malformed LATM packet
Last message repeated 1 times
[aac @ 0x7f79e40b6680] channel element 3.4 is not allocated
[rtsp @ 0x7f79e4000bc0] Malformed LATM packet
[aac @ 0x7f79e40b6680] channel element 3.4 is not allocated
Last message repeated 2 times
[aac @ 0x7f79e40b6680] Sample rate index in program config element does not match the sample rate index configured by the container.
[aac @ 0x7f79e40b6680] decode_pce: Input buffer exhausted before END element found
[rtsp @ 0x7f79e4000bc0] Malformed LATM packet 0KB sq= 0B f=0/0
Last message repeated 1 times
[rtsp @ 0x7f79e4000bc0] Malformed LATM packet 0KB sq= 0B f=0/0
Last message repeated 1 times
[rtsp @ 0x7f79e4000bc0] Malformed LATM packet 0KB sq= 0B f=0/0
Last message repeated 1 times
[aac @ 0x7f79e40b6680] Number of bands (31) exceeds limit (16).
[aac @ 0x7f79e40b6680] channel element 2.10 is not allocated
[aac @ 0x7f79e40b6680] skip_data_stream_element: Input buffer exhausted before END element found
[aac @ 0x7f79e40b6680] SBR was found before the first channel element.
[aac @ 0x7f79e40b6680] Reserved bit set.
...
And On
and vlc reports simply main decoder error: buffer deadlock prevented.
I am not sure if I am messing up the transport type or something else in the encoder (see the first ffplay error message) or the header/SDP OR the way I packetize and send the data (likely some combination). I am essentially brand new to AAC/streaming/RTP/RTSP and in the "throw things at the wall" stage so I am hoping someone can help me figure out the correct encoder settings and SDP params required for the simplest audio stream ffplay or vlc will accept.
I have read through RFC 5691, 6416, and 3550 but there is so much extra info I am a bit overwhelmed by the configurability of all this as well.
Happy to edit the question if there is any more info I can provide!
I am now able to 'stream' (albeit poorly/with gaps and glitches but recognizable) using the following configs:
// Encoder config: (same as above if not listed)
aacEncoder_SetParam(aac_handle, AACENC_TRANSMUX, TT_MP4_ADTS)
// SDP:
char sdp_aac[] =
"v=0\r\n"
"o=- 0 0 IN IP4 0.0.0.0\r\n"
"s=Unnamed\r\n"
"c=IN IP4 0.0.0.0\r\n"
"a=recvonly\r\n"
"a=charset:UTF-8\r\n"
"m=audio 50040 RTP/AVP 97\r\n"
"a=rtpmap:97 mpeg4-generic/16000/1\r\n"
"a=fmtp:97 config=1408\r\n"; // config hex from encoder config binary
// RTP Packet Headers:
hdr.flags = 0x80;
hdr.mk_pt = 0x80 | 97;
hdr.sq_nb = htons(rtp_packet_count);
hdr.ts = htonl(rtp_timestamp_audio);
hdr.ssrc = htonl(10);
However, with ffplay, I now get a single repeated error and no audio: [rtsp @ 0x7fc3d8000bc0] Error parsing AU headers
The main issue I was encountering was the lack of 'AU Headers' in the stream (Explained here in RFC 3640). Overall, the settings I needed to get the stream working were:
Encoder config:
aacEncoder_SetParam(aac_handle, AACENC_AOT, AOT_AAC_LC) != AACENC_OK)
aacEncoder_SetParam(aac_handle, AACENC_BITRATE, 96000) != AACENC_OK)
aacEncoder_SetParam(aac_handle, AACENC_BITRATEMODE, 0) != AACENC_OK)
aacEncoder_SetParam(aac_handle, AACENC_SAMPLERATE, 16000) != AACENC_OK)
aacEncoder_SetParam(aac_handle, AACENC_CHANNELMODE, MODE_1) != AACENC_OK)
aacEncoder_SetParam(aac_handle, AACENC_BANDWIDTH, 0) != AACENC_OK)
aacEncoder_SetParam(aac_handle, AACENC_CHANNELORDER, 1) != AACENC_OK)
aacEncoder_SetParam(aac_handle, AACENC_AFTERBURNER, 0) != AACENC_OK)
aacEncoder_SetParam(aac_handle, AACENC_SBR_MODE, 0) != AACENC_OK)
aacEncoder_SetParam(aac_handle, AACENC_SBR_RATIO, 0) != AACENC_OK)
aacEncoder_SetParam(aac_handle, AACENC_TRANSMUX, TT_MP4_ADTS) != AACENC_OK)
aacEncoder_SetParam(aac_handle, AACENC_SIGNALING_MODE, 0) != AACENC_OK)
SDP (note added *Length params for AU header):
"v=0\r\n"
"o=- 0 0 IN IP4 0.0.0.0\r\n"
"s=Unnamed\r\n"
"c=IN IP4 0.0.0.0\r\n"
"a=recvonly\r\n"
"a=charset:UTF-8\r\n"
"m=audio 50040 RTP/AVP 98\r\n"
"a=rtpmap:98 mpeg4-generic/16000/1\r\n"
"a=fmtp:98 mode=AAC-hbr; config=1408; sizeLength=13; indexLength=3; IndexDeltaLength=3;\r\n";
RTP Header:
hdr.flags = 0x80;
hdr.mk_pt = 0x80 | 98;
hdr.sq_nb = htons(stream->cnt);
hdr.ts = htonl(timestamp);
hdr.ssrc = htonl(10);
AU Header: (The missing piece):
typedef struct{
uint16_t header_len = 0x1000; // Size of this structure not counting this field
uint16_t au_size_idx; // First 13b are the size of the AU in bytes, last 3b are apparently ignored (RFC 3640 s3.3.6)
} au_header_aac_t;
uint16_t au_size = this_len << 3;
au_hdr.au_size_idx = ntohs(au_size);