RTP and TS Demuxer

Jan 18, 2013 at 1:00 PM
Edited Jan 18, 2013 at 1:03 PM


I'm looking for a good H.264Framer. I want to make a RTP Packet from H.264 encoded file like .ts, .mp4. And transfer RTP Packets to RTSP/RTP Player like a VLC.

Is it possible to use your TSUnit(set of TranportPackets) for making a RTP Packet?

Is the TSUnit kinds of NAL Unit?

Please give me a some advice.


Jan 18, 2013 at 4:59 PM

BaseMedia can be used to extract the h.264 bitstream (a collection of Network Abstraction Layer Units) from mp4 and TS.  The format is slightly different between TS and MP4.  TS uses start code prefixed NAL Units which begin with hex bytes 00 00 01 (extra leading 0s are ok) and MP4 uses length prefixed which begin with a 32bit bigendian length for the following NAL Unit.

Audio (typically AAC though MP3 and AC3 are popular as well) can be extracted as well.  The TS sample already gives you access to the streams and I'll be writing a sample this weekend hopefully to demonstrate extracting the streams from MP4.

Packaging them for transport over RTP is outside the scope of my project but Julius Friendman's Managed Media Aggregation project might be able to do that.  We were planning on doing a sample application in the near future that shows how.

Note that if your source is TS, there are two other options.  There is an RTP profile for encapsulating TS directly in a single stream.  For that you'd just need to read the 188 byte TS packets that start with 0x47 and send them to an RTP muxer.  You might need to parse the packets a bit as well to get out timing information so it isn't transmitted too fast, though, and my library can do that.

Also, TS can be sent raw unicast or multicast UDP.  You just stuff up to 7 188byte TS packets into a UDP packet and send it.  Again, you'll want to look at the timestamps in the TS file and make sure you aren't sending too fast.

For timing, you'll have to read enough of the TS file to get the Program Map Table which will tell you the PID of the Program Clock Reference (PCR) stream and then parse those packets to get timestamps that you can correlate to the current system time in order to throttle your send.  If there's no PCR stream, you'll need to look at the audio or video stream and pull timestamps out of the TS headers.

MP4 has timing information as well but it's a bit trickier since the file stores them as sample durations of an arbitrary timescale.  You'd need the TimeScale from the stream's moov\trak\mdia\mdhd box and the SampleDelta from the moov\trak\mdia\minf\stbl\stsd\stts box's table.  As you get each sample, you'd add its delta to your running count of time and divide that total by TimeScale to get the current stream time in seconds.  Correlate that stream time to the current system time and then you can decide whether or not to wait and how long.

All the building blocks are there between the two libraries but it'll take some work.

Jan 19, 2013 at 1:17 AM

Actually Julius Friendman introduce your project. And my project is based on his project. So my project can transfer RTP/RTSP Packet to remote VLC player.

But the source of stream is a JPEG image, so it is very slow and inefficient. So I changed the source to MP4 or TS from JPEG. Julius Friendman's MMA has a pcketizing class of JPEG, but not MP4 . So he told me about your project.

Anyway I'm so happy to your comment. "All the building blocks are there between the two libraries."


Jan 19, 2013 at 4:38 AM

Ok, so you still need a way to frame h.264 (and probably AAC audio) as RTP packets.  Neither of our projects do that at present.  I started writing some code for it before so I'll see if I can find that and make it workable for you.

Feb 7, 2013 at 8:19 AM
jclary, in the post above you mentioned that you might write "a sample to demonstrate extracting the streams from MP4". Did you have time to write the sample, and if yes, where can I find it?

Thanks for your great work!
Feb 7, 2013 at 5:07 PM
Sorry, I haven't finished it yet. I've been swamped with work related to the FCC deadline for Closed Captioning.