Windows Phone MP4 recorded video

Jun 23, 2013 at 7:13 AM
Edited Jun 23, 2013 at 7:14 AM
Hi!

Your Library is great! I have a question:

how can I merge two mp4 video files with your libary which were recorded with Windows Phone?

Could you give me some points on it, how is it possible?

I would be very grateful if you could answer!

Thanks!

Krisztián
Coordinator
Jun 23, 2013 at 8:30 PM
If you are trying to connect them end-to-end, so long as the tracks are the same format, the contents of the two mdat boxes can be combined into a new mdat box and the track information can be combined at the table level (the boxes under moov/trak/mdia/minf/stbl all have arrays of information about the samples) but all the offsets and sample numbers would have to be updated very carefully similar to the way the FastStart command does in BmffViewer.

The durations in moov/mvhd and moov/trak/tkhd will have to be added together as well and if the timescale in tkhd is different, all the sample durations in the tables for one of the files will have to be recalculated using the new base. Basically you'd multiply each sample duration by the original timescale then divide by the new timescale, I believe.

The one real gotcha you might run into is if the audio tracks aren't nearly exactly the same length as the video. In that case you might have to create and insert some extra samples (or repeat/stretch some existing ones) so the audio doesn't go out of sync in the second half.

If the formats and encoding parameters of the tracks differ in any significant way, however, you'd have to transcode it to match somehow and that's well outside the scope of this library.
Jun 23, 2013 at 10:09 PM
Hi!

Thank You for Your answer!

How can I put two mdat files together?
  • Should I create a new BaseMedia object, and insert the two MovieDataBox as a children into it? Need I recalculate the offsets after it? How can I do it?
  • Is it enough if I use / modify the duration of MovieBox object, which belongs to the first file?
Thank You!

Krisztián
Coordinator
Jun 23, 2013 at 10:29 PM
It depends. You might be able to just put both mdat's in one file. All the offsets of the samples are relative to the beginning of the file, not the beginning of the mdat, though, which is what makes updating the track information rather tedious.

I am not certain all players will like this, however. It might just be better to combine the contents of both mdats into a single new mdat. There's not really much to an mdat box -- a length and a type then the data.

You can just clone the track information off the first file and then modify the durations and tables and write that out but unless the moov is at the end of the file, you'll have to update all the offsets anyway because the moov will be larger forcing you to relocate the start of the mdat. It's a lot simpler if the moov boxes are at the end since you can leave the ftyp and mdat from one file where it is, append to the contents of the mdat (updating its length, or add a second mdat), then rewrite the moov on the end and you'll only need to change the offsets for one of the files.
Jun 24, 2013 at 2:37 PM
Hi!

Thank You for Your answer!

I could create a new MovieDataBox and I could modify the duration inside the TrackBox (i had to modify the constructor of the MovieDataBox class, because the offset property has to be set).

However when the generated mp4 file is playing, I can only see black screen :-(. The duration of the movie and the content size is ok.

Yesterday you wrote, that i should modify the samples inside the TrackBox at the table level as well (moov/trak/mdia/minf/stbl), but i don't know, how and what should i do it. I have used the code from the FastStart method, but it didn't work.

Here is my code:
    public bool FastStart(string path, bool removeFreeSpace = true)
    {
        FileInfo info = new FileInfo(@"C:\Users\krisztian_yoga\Desktop\videos\CameraMovie2.mp4");
        BaseMediaFile clone2 = new BaseMediaFile(info);

        using (BaseMediaFile clone = new BaseMediaFile(this._FileInfo))
        {
            clone.DecompressMovieHeader();
            clone2.DecompressMovieHeader();

            MovieBox moov = clone.Children.OfType<MovieBox>().SingleOrDefault();
            MovieBox moov2 = clone2.Children.OfType<MovieBox>().SingleOrDefault();

            if (moov == null) return false;
            MovieDataBox mdat = clone.Children.OfType<MovieDataBox>().SingleOrDefault();
            MovieDataBox mdat2 = clone2.Children.OfType<MovieDataBox>().SingleOrDefault();
            if (mdat == null) return false;

            Stream s1 = mdat.GetContentStream();
            byte[] data1 = s1.ReadBytes((int)s1.Length);

            Stream s2 = mdat2.GetContentStream();
            byte[] data2 = s2.ReadBytes((int)s2.Length);

            MemoryStream mStream = new MemoryStream();
            mStream.WriteBytes(data1);
            mStream.WriteBytes(data2);
            mStream.Flush();
            s1.Dispose();
            s2.Dispose();

            mStream.Seek(0, SeekOrigin.Begin);

            MovieDataBox newMdat = new MovieDataBox(mStream, mdat.ContentOffset.Value, mdat.Offset.Value, mdat.ContentSize.Value + mdat2.ContentSize.Value);

            int mdatIndex = clone.Children.IndexOf(mdat);
            clone.Children.Remove(mdat);
            clone.Children.Insert(mdatIndex, newMdat);

            Debug.WriteLine("Calculating new 'mdat' offset.");
            ulong mdatOffset = 0;
            foreach (Box box in clone.Children)
            {
                if (box is MovieDataBox) break;
                else mdatOffset += GetBoxWriteSize(box);
            }

            bool isPositiveAdjustment = mdatOffset > mdat.Offset.Value;
            ulong chunkAdjustment = mdatOffset > mdat.Offset.Value ? mdatOffset - mdat.Offset.Value : mdat.Offset.Value - mdatOffset;

            List<TrackBox> traks = new List<TrackBox>( moov.Children.OfType<TrackBox>() );
            var duration = moov.Children.OfType<MovieHeaderBox>().SingleOrDefault();
            var duration2 = moov2.Children.OfType<MovieHeaderBox>().SingleOrDefault();
            duration.Duration = duration.Duration + duration2.Duration;

            List<TrackBox> traks2 = new List<TrackBox>( moov2.Children.OfType<TrackBox>() );

            foreach (TrackBox track in traks)
            {
                TrackHeaderBox th = track.Children.OfType<TrackHeaderBox>().SingleOrDefault();

                TrackBox track2 = traks2[traks.IndexOf(track)];

                if (th != null)
                {
                    th.Duration = duration.Duration;
                }

                MediaBox mdia = track.Children.OfType<MediaBox>().SingleOrDefault();
                MediaBox mdia2 = track2.Children.OfType<MediaBox>().SingleOrDefault();

                if (mdia != null && mdia2 != null)
                {
                    MediaInformationBox minf = mdia.Children.OfType<MediaInformationBox>().SingleOrDefault();
                    MediaInformationBox minf2 = mdia2.Children.OfType<MediaInformationBox>().SingleOrDefault();

                    if (minf != null && minf2 != null)
                    {
                        SampleTableBox stbl = minf.Children.OfType<SampleTableBox>().Single();
                        SampleTableBox stbl2 = minf2.Children.OfType<SampleTableBox>().Single();

                        if (stbl != null && stbl2 != null)
                        {
                            Debug.WriteLine("Updating 'stco' offsets.");
                            bool upgradeToChunkLongOffsetBox = false;
                            ChunkOffsetBox stco = stbl.Children.OfType<ChunkOffsetBox>().SingleOrDefault();
                            ChunkOffsetBox stco2 = stbl2.Children.OfType<ChunkOffsetBox>().SingleOrDefault();
                            if (stco != null && stco2 != null)
                            {
                                for (int i = 0; i < stco.Entries.Count; i++)
                                {
                                    try
                                    {
                                        if (isPositiveAdjustment)
                                            stco.Entries[i] = checked((uint)((ulong)stco.Entries[i] + (uint)((ulong)stco2.Entries[i] + chunkAdjustment)));
                                        else
                                            stco.Entries[i] = checked((uint)((ulong)stco.Entries[i] - (uint)((ulong)stco2.Entries[i] - chunkAdjustment)));
                                    }
                                    catch (OverflowException)
                                    {
                                        upgradeToChunkLongOffsetBox = true;
                                        break;
                                    }
                                }
                            }

                            if (upgradeToChunkLongOffsetBox)
                            {
                                Debug.WriteLine("Overflow encountered.  Converting 'stco' to 'co64' to compensate.");
                                // TODO: Copy and adjust stco values
                            }

                            ChunkLargeOffsetBox co64 = stbl.Children.OfType<ChunkLargeOffsetBox>().SingleOrDefault();
                            ChunkLargeOffsetBox co642 = stbl2.Children.OfType<ChunkLargeOffsetBox>().SingleOrDefault();
                            if (co64 != null)
                            {
                                Debug.WriteLine("Updating 'co64' offsets.");
                                for (int i = 0; i < co64.Entries.Count; i++)
                                {
                                    if (isPositiveAdjustment)
                                        co64.Entries[i] = checked(co64.Entries[i] + co642.Entries[i] + chunkAdjustment);
                                    else
                                        co64.Entries[i] = checked(co64.Entries[i] - co642.Entries[i] - chunkAdjustment);
                                }
                            }
                        }
                    }
                }
            }

            clone.SaveAs(path);
            return true;
        }
What can be the problem? Thank you for your help!

Krisztián
Jun 25, 2013 at 8:07 AM
Hi!

Could You help me?

I would be very grateful if you could answer!

Thanks!

Krisztián
Coordinator
Jun 25, 2013 at 5:16 PM
FastStart shows how to modify the offsets in the stco or co64 box, whichever is present. That's only part of it. You have to combine the tables from both files as I mentioned originally. Append all the table Entries from one file to the table Entries in the other file AND update the ChunkOffsets in stco/co64's table Entries.

So for each of the ITableBox boxes in moov\trak\mdia\minf\stbl... stts, stss, stsc, stcz and either stco or co64, do something like following pseudocode:

foreach(var entry in stts_file2.Entries) stts_file1.Entries.Add(entry);

That'll make the file1 version of stts have the entries from both files. Do that for the others as well.

stco/co64 is special case because the ChunkOffsets in its table are relative to the start of the file so you'll have to update the entries from one or both.

Don't mess with TrackBox's or MovieDataBox's offset. You said it has to be set but it shouldn't. When you save it, it writes it in the order they appear in the document tree. The offset stored in the box itself is just tracking of where it was in the original file. It uses that on save so it knows where to copy any payload data from. If you change it, it'll copy the wrong things from the source file. It doesn't load the whole file into memory, just all of the box header info.

You will need to know the new offset of the mdat(s) or where the contents are going to go in the file, at least. First thing I'd try is with TWO mdats. Just append the mdat from file 2 to the root of file 1. The tricky bit is if the moov is near the start of the file, it'll be longer which will move BOTH mdats in the new file. That means you'll need to combine the tables, recalculate the size of the ftyp and moov in order to deduce where the first mdat is. Then add the length of the first mdat and that should be the start of the second one. You'll need to remember how long the stco/co64 table was to begin with because you'll have to adjust the offsets for each differently.

Once you've updated all of that you can save the file1 document tree to a new file and it should copy over all the box payloads from the two source files. So long as those ChunkOffset values are correct, it should be playable.

I don't have time right now to do a sample for you, I'm sorry. I'll see if I can get to it in the next few days.
Jun 26, 2013 at 5:02 PM
Thank you for your help! Thank you, thank you, thank you!

Today I could create a merged mp4 file, which plays correctly after the operation.

Krisztián
Coordinator
Jun 26, 2013 at 5:27 PM
Glad to hear it.

For the completeness, you might also want to look at combining the tables from moov\trak\edts\elst like you did with the others. I think all you need to do as far as adjustment is add the duration of the first file's trak to the MediaTime field of the second file's but I'm not 100% certain.

Most of the time it's used to offset the video trak because capture cards can rarely start the audio and video capture simultaneously. Not merging this could result in a slight desynchronization between audio and video.

Some players ignore elst, however.
Jul 3, 2013 at 8:08 AM
Edited Jul 3, 2013 at 8:12 AM
I'm sorry to write again, but I have a problem. Sometimes the merged video is going to be corrupted but sometimes don't. As I can see, a video which was recorded with a windows phone, doesn't have moov\trak\edts\elst.

I share with you the relevant part of my code (without any try-catch blocks etc.).

Do you see anything, which could cause the problem?

Thank You for Your help!

using (Stream rootStream = await files[0].OpenStreamForReadAsync())
            {
                using (BaseMediaFile rootFile = new BaseMediaFile(rootStream))
                {
                        rootMoov = rootFile.Children.OfType<MovieBox>().SingleOrDefault();
                        rootMdat = rootFile.Children.OfType<MovieDataBox>().SingleOrDefault();
                        rootTraks = new List<TrackBox>(rootMoov.Children.OfType<TrackBox>());
                        rootDuration = rootMoov.Children.OfType<MovieHeaderBox>().SingleOrDefault();

                    for (int n = 1; n < files.Count; n++)
                    {
                            Stream chunkStream = await files[n].OpenStreamForReadAsync();
                            BaseMediaFile chunkFile = new BaseMediaFile(chunkStream);

                            chunkStreams.Add(chunkStream);
                            chunkFiles.Add(chunkFile);

                            MovieBox chunkMoov = chunkFile.Children.OfType<MovieBox>().SingleOrDefault();
                            MovieDataBox chunkMdat = chunkFile.Children.OfType<MovieDataBox>().SingleOrDefault();

                            rootFile.Children.Insert(rootFile.Children.Count - 1, chunkMdat);

                            ulong mdatOffset = 0;

                            foreach (Box box in rootFile.Children)
                            {
                                if (box is MovieDataBox && rootFile.Children.IndexOf(box) == rootFile.Children.Count - 2) break;
                                else mdatOffset += rootFile.CalculateBoxWriteSize(box);
                            }

                            ulong chunkAdjustment = mdatOffset - chunkMdat.Offset.Value;

                            MovieHeaderBox chunkDuration = chunkMoov.Children.OfType<MovieHeaderBox>().SingleOrDefault();
                            rootDuration.Duration += chunkDuration.Duration;

                            List<TrackBox> chunkTraks = new List<TrackBox>(chunkMoov.Children.OfType<TrackBox>());

                            foreach (TrackBox rootTrack in rootTraks)
                            {
                                TrackBox chunkTrack = chunkTraks[rootTraks.IndexOf(rootTrack)];
                                TrackHeaderBox rootTrackHeader = rootTrack.Children.OfType<TrackHeaderBox>().SingleOrDefault();

                                if (rootTrackHeader != null)
                                {
                                    rootTrackHeader.Duration = rootDuration.Duration;
                                }

                                MediaBox rootMdia = rootTrack.Children.OfType<MediaBox>().SingleOrDefault();
                                MediaBox chunkMdia = chunkTrack.Children.OfType<MediaBox>().SingleOrDefault();

                                if (rootMdia != null && chunkMdia != null)
                                {
                                    MediaInformationBox rootMinf = rootMdia.Children.OfType<MediaInformationBox>().SingleOrDefault();
                                    MediaInformationBox chunkMinf = chunkMdia.Children.OfType<MediaInformationBox>().SingleOrDefault();

                                    MediaHeaderBox rootMhb = rootMdia.Children.OfType<MediaHeaderBox>().SingleOrDefault();
                                    MediaHeaderBox chunkMhb = chunkMdia.Children.OfType<MediaHeaderBox>().SingleOrDefault();
                                    rootMhb.Duration += chunkMhb.Duration;

                                    if (rootMinf != null && chunkMinf != null)
                                    {
                                        SampleTableBox rootStbl = rootMinf.Children.OfType<SampleTableBox>().SingleOrDefault();
                                        SampleTableBox chunkStbl = chunkMinf.Children.OfType<SampleTableBox>().SingleOrDefault();

                                        if (rootStbl != null && chunkStbl != null)
                                        {
                                            //stts 
                                            TimeToSampleBox rootStts = rootStbl.Children.OfType<TimeToSampleBox>().SingleOrDefault();
                                            TimeToSampleBox chunkStts = chunkStbl.Children.OfType<TimeToSampleBox>().SingleOrDefault();

                                            if (rootStts != null && chunkStts != null)
                                            {
                                                for (int i = 0; i < chunkStts.Entries.Count; i++)
                                                {
                                                    rootStts.Entries.Add(chunkStts.Entries[i]);
                                                }
                                            }

                                            //stsc
                                            SampleToChunkBox rootStsc = rootStbl.Children.OfType<SampleToChunkBox>().SingleOrDefault();
                                            SampleToChunkBox chunkStsc = chunkStbl.Children.OfType<SampleToChunkBox>().SingleOrDefault();

                                            if (rootStsc != null && chunkStsc != null)
                                            {
                                                var lastStsc = rootStsc.Entries.Last();

                                                for (int i = 0; i < chunkStsc.Entries.Count; i++)
                                                {
                                                    chunkStsc.Entries[i].FirstChunk += lastStsc.FirstChunk;

                                                    rootStsc.Entries.Add(chunkStsc.Entries[i]);
                                                }
                                            }

                                            //stco
                                            ChunkOffsetBox rootStco = rootStbl.Children.OfType<ChunkOffsetBox>().SingleOrDefault();
                                            ChunkOffsetBox chunkStco = chunkStbl.Children.OfType<ChunkOffsetBox>().SingleOrDefault();

                                            if (rootStco != null && chunkStco != null)
                                            {
                                                for (int i = 0; i < chunkStco.Entries.Count; i++)
                                                {
                                                    chunkStco.Entries[i] = checked((uint)((ulong)chunkStco.Entries[i] + chunkAdjustment));

                                                    rootStco.Entries.Add(chunkStco.Entries[i]);
                                                }
                                            }

                                            //stsz
                                            SampleSizeBox rootStsz = rootStbl.Children.OfType<SampleSizeBox>().SingleOrDefault();
                                            SampleSizeBox chunkStsz = chunkStbl.Children.OfType<SampleSizeBox>().SingleOrDefault();

                                            if (rootStsz != null && chunkStsz != null)
                                            {
                                                for (int i = 0; i < chunkStsz.Entries.Count; i++)
                                                {
                                                    rootStsz.Entries.Add(chunkStsz.Entries[i]);
                                                }
                                            }

                                            //stss 
                                            SyncSampleBox rootStss = rootStbl.Children.OfType<SyncSampleBox>().SingleOrDefault();
                                            SyncSampleBox chunkStss = chunkStbl.Children.OfType<SyncSampleBox>().SingleOrDefault();

                                            if (rootStss != null && chunkStss != null)
                                            {
                                                for (int i = 0; i < chunkStss.Entries.Count; i++)
                                                {
                                                    rootStss.Entries.Add(chunkStss.Entries[i]);
                                                }
                                            }
                                        }
                                    }
                                }
                            }
                    string fileName = String.Format(mergedVideoFileName, DateTime.Now.Year, DateTime.Now.Month, DateTime.Now.Day);
                    StorageFile mergedFile = await rootFile.SaveAs(fileName);

                    foreach (var item in chunkFiles)
                    {
                        item.Dispose();
                    }

                    foreach (var item in chunkStreams)
                    {
                        item.Dispose();
                    }

                    return mergedFile;
                }
Jul 3, 2013 at 9:30 AM
I can send you a corrupted merged video file if it helps you.
Jul 4, 2013 at 8:06 AM
Edited Jul 4, 2013 at 9:55 AM
I've observed that if the length of a video in the "merge-list" is less than or around 1 second, the merged video is going to be corrupted. What do You think? What can be the problem?

Thank You for Your help!
Jul 5, 2013 at 7:07 AM
Edited Jul 5, 2013 at 7:08 AM
Yesterday I've tried, what happens if I skip the videos from the "merge-list", which are less than 2 seconds. I didn't help. So I think the problem can be inside the merge-algorithm.

Do you have any idea, what can be the problem?

Thank You for Your help!
Jul 7, 2013 at 11:18 AM
Hi!

I would be very grateful if you could answer!

Maybe the problem is that I append the mdat files after each other?! I've tried to create a new single MovieDataBox object, but in the created file the content of the mdat box is always 0.

Thanks!

Krisztián
Coordinator
Jul 8, 2013 at 5:42 PM
Sorry, I'm not sure what your issue is exactly. I'll take a look if you put a couple of sample files somewhere I can download them.

You'll probably have to customize the MovieDataBox to pull from multiple files/offsets/lengths. Internally it uses the file it was created from and the offset and length information. There's an alternate constructor that takes a stream, offset and length so you can point it to any portion of any file. Making it support multiple, however, won't be trivial.

The simplest thing would be to copy the contents of all the mdats into one file (just the payload, not the mdat itself) and then create a new MovieDataBox referencing that file, 0 offset and the file length. Then when you write the mdat, it'll pull from that temporary file. I'd suggest doing that initially to see if it fixes your problem before we start looking at adding multiple payload source support to the MovieDataBox or Box classes.
Jul 9, 2013 at 6:57 AM
Edited Jul 9, 2013 at 7:03 AM
Hi!

Thank You for Your answer!

You can download here a zip file, which contains some videos as follows:

/temp folder: the merged video contains these records.

MyVideo_2013_7_9.mp4: merges the TemporaryCameraMovie.mp4 and TemporaryCameraMovie (2).mp4 files. It looks and works good.

MyVideo_2013_7_9 (2).mp4: merges the TemporaryCameraMovie.mp4, TemporaryCameraMovie (2).mp4 and TemporaryCameraMovie (3).mp4 files. After ~2 seconds the video getting stuck.

MyVideo_2013_7_9 (3).mp4: merges the TemporaryCameraMovie.mp4, TemporaryCameraMovie (2).mp4, TemporaryCameraMovie (3).mp4 and TemporaryCameraMovie (4).mp4 files. The video crashes immediately at the beginning (some players, like the VLC Player can play the half of the video, but the others (Media Player on Windows Phone, Media Player on Windows 8 etc.) don't).

TemporaryCameraMovie.mp4-joined.mp4: merges the TemporaryCameraMovie.mp4, TemporaryCameraMovie (2).mp4, TemporaryCameraMovie (3).mp4 and
TemporaryCameraMovie (4).mp4 files, but with a desktop app on windows, called My MP4BoxGui. I just used it to trying to figure what can be the problem with my merged video compared to the MP4Box version. I noticed that (maybe) the stsc SampleToChunkBox box is wrong in my video, because the MP4Box version contains different entries.

I've tried to create a single mdat file. The problem is, that when I create a new MovieDataBox object, the payload in the created file is going to be empty. Here is my example code, how i merge the payloads and create the object:

Stream s1 = mdat.GetContentStream();
byte[] data1 = s1.ReadBytes((int)s1.Length);

Stream s2 = mdat2.GetContentStream();
byte[] data2 = s2.ReadBytes((int)s2.Length);

MemoryStream mStream = new MemoryStream();
mStream.WriteBytes(data1);
mStream.WriteBytes(data2);
mStream.Flush();

mStream.Seek(0, SeekOrigin.Begin);

MovieDataBox newMdat = new MovieDataBox(mStream, 0, (ulong)mStream.Length);

Here you can see the merged mdat component in the BmffViewer:

Image

Thanks!

Krisztián
Coordinator
Jul 9, 2013 at 8:19 AM
Hrm, there's a bug somewhere in my code but it's not jumping out at me. Probably either in the mdat constructor or the Box.ToStream method. I'm not certain I ever tested loading from a different file but I'll see if I can make it work. I added it in there thinking people might find it useful but I usually just do mdat's manually since they are so simple -- 4 byte length, 4 byte FOURCC and then the payload.

I think you are right about SampleToChunkBox. I'm pretty sure you have to add the total number of chunks from previous files to the ones you are adding. FirstChunk specified the first chunk with a given SamplesPerChunk value. So total up the stco table's count from previous files and add it to the values in FirstChunk for the next file.
Coordinator
Jul 9, 2013 at 8:24 AM
Oh, when you update FirstChunk, check it against the file from that other tool. I'm not 100% certain if those are 1 or 0 based but I'm pretty sure they are 1 based. Likely be an off-by-one error that increases by 1 for each file added.. easy fix.
Jul 9, 2013 at 9:17 AM
Edited Jul 9, 2013 at 9:21 AM
Thank you, thank you, thank you!

The problem was with the SampleToChunkBox. I increased the value of FirstChunk not the stco table's count, but with the last FirstChunk value of the previous stsc table. So it looks like, it works. I'm going to test again the process.

Thanks!

Krisztián
Coordinator
Jul 9, 2013 at 9:26 AM
That'll work in some cases but not all. The entries in that box mark the beginning of a series of unspecified length. It's slightly more likely that the last chunk in the file will have a different number of samples from the rest but that won't always be the case. You really should get the actual total number of chunks for the file from stco/co64 if possible.
Oct 21, 2013 at 9:51 PM
Hello

I'm trying to do the same thing as kfekete up here - join mp4 files. And I'm stuck. It's using your method of merging the files with one extension. I'm creating a new mdat from memory stream where I put all the mdats.
Merged video can be seen but you can only watch 5s. I've checked everything and all seems to be correct in the respect of the file I get from ffmpeg concat...
If you're willing to take a look here is my source code with some test movies http://sdrv.ms/15YtLFn

That part of my code is different than the one posted by kfekete at Jul 3 at 10:08 AM
MemoryStream mStream = new MemoryStream();
          Stream sRoot = rootMdat.GetContentStream();
          byte[] data = new byte[sRoot.Length];
//write some dummy data for mdat header
          mStream.Write(data, 0, 8);
          sRoot.Read(data, 0, (int)sRoot.Length);
          mStream.Write(data, 0, (int)sRoot.Length);
          ulong mdatOffset = (ulong)sRoot.Length;
          adjust.Add(mdatOffset + rootMdat.Offset.Value-16);
          for (int n = 1; n < files.Count; n++)
          {
            //Stream chunkStream = new IsolatedStorageFileStream(files[n], FileMode.Open, IsolatedStorageFile.GetUserStoreForApplication());
            Stream chunkStream = new FileStream(files[n], FileMode.Open);
            BaseMediaFile chunkFile = new BaseMediaFile(files[n], chunkStream);

            chunkStreams.Add(chunkStream);
            chunkFiles.Add(chunkFile);

            MovieBox chunkMoov = chunkFile.Children.OfType<MovieBox>().SingleOrDefault();
            MovieDataBox chunkMdat = chunkFile.Children.OfType<MovieDataBox>().SingleOrDefault();

            //rootFile.Children.Insert(rootFile.Children.Count - 1, chunkMdat);
            Stream sTmp = chunkMdat.GetContentStream();
            data = new byte[sTmp.Length];
            sTmp.Read(data, 0, (int)sTmp.Length);
            mStream.Write(data, (int)chunkMdat.ContentOffset.Value, (int)sTmp.Length - (int)chunkMdat.ContentOffset.Value);
            mdatOffset += (ulong)sTmp.Length;
            adjust.Add((ulong)sTmp.Length);
          }

          mStream.Seek(0, SeekOrigin.Begin);
          mStream.WriteBEUInt32((uint)mdatOffset);
          mStream.WriteBEUInt32((uint)1835295092);
          mStream.Seek(0, SeekOrigin.Begin);
          rootMdat = (MovieDataBox)Box.FromStream(mStream);
          for (int i = 0; i < rootFile.Children.Count; i++)
          {
            if (rootFile.Children[i] is MovieDataBox)
            {
              rootFile.Children[i] = rootMdat;
            }
          }
Thanks!
Coordinator
Oct 21, 2013 at 11:22 PM
First thing that strikes me is the ContentSize of the combined MDAT is larger than the ContentSize of both added together by 56 bytes. Depending on where that extra space is and whether or not the offset adjustments compensate for it, it could definitely be causing the problem. Since the first half plays, I expect it's in the middle or half in the middle and half at the end. Unlike Annex-B bitstreams, there's no way to synchronize on the data in an MP4 track if the chunk offsets aren't correct. All the frames with bad offsets will just be unusable.

I haven't gotten to poking around the code for it yet so that's just one possibility. I figured you could take a look while I am -- you might spot it quicker since you've been working with the code recently.
Coordinator
Oct 21, 2013 at 11:58 PM
Ok... Definitely extra 0x00s in the middle and on the end. If you account for them in the offset adjustments it's fine for the extra space to be there but otherwise it's a problem.

Still trying to sort through what all you are trying to do in your extra code but first thing I'd suggest is not using the length of the stream you get back from GetContentStream(). It returns the original file stream seeked to the right offset and wrapped in a ConstrainedStream so you can't read past the end of the content. The stream length, however, is still the length of the file not the length of the box content. That way you can seek around in it relative to the start of the file which is what a player has to do.
Oct 22, 2013 at 7:01 AM
Thanks for the tips. Will look into that.
Oct 22, 2013 at 11:25 AM
I've changed code a bit to use rootMdat.ContentSize.Value instead of the length of the stream. Which solves the problem with the 0s in the middle of mdat.
One thing that worries me is that now my mdat has following structure
mdat header|encoder info|1st file data|encoder info|2nd file data

In the files joined by ffmpeg you don't have this second encoder info (check -3.mp4-joined.mp4) I wonder how you can handle that. I was looking over the internet about the mdat structure but every site claims it's simple store for chunks.
          byte[] data = new byte[rootMdat.ContentSize.Value];
          mStream.Write(data, 0, 8);
          sRoot.Read(data, 0, (int)rootMdat.ContentSize.Value);
          mStream.Write(data, 0, (int)rootMdat.ContentSize.Value);
          ulong mdatOffset = rootMdat.ContentSize.Value;
          adjust.Add(mdatOffset + rootMdat.Offset.Value);
          for (int n = 1; n < files.Count; n++)
          {
            //Stream chunkStream = new IsolatedStorageFileStream(files[n], FileMode.Open, IsolatedStorageFile.GetUserStoreForApplication());
            Stream chunkStream = new FileStream(files[n], FileMode.Open);
            BaseMediaFile chunkFile = new BaseMediaFile(files[n], chunkStream);

            chunkStreams.Add(chunkStream);
            chunkFiles.Add(chunkFile);

            MovieBox chunkMoov = chunkFile.Children.OfType<MovieBox>().SingleOrDefault();
            MovieDataBox chunkMdat = chunkFile.Children.OfType<MovieDataBox>().SingleOrDefault();

            //rootFile.Children.Insert(rootFile.Children.Count - 1, chunkMdat);
            Stream sTmp = chunkMdat.GetContentStream();
            data = new byte[chunkMdat.ContentSize.Value];
            sTmp.Read(data, 0, (int)chunkMdat.ContentSize.Value);
            mStream.Write(data, (int)chunkMdat.ContentOffset.Value, (int)chunkMdat.ContentSize.Value - (int)chunkMdat.ContentOffset.Value);
            mdatOffset += chunkMdat.ContentSize.Value;
            adjust.Add((ulong)chunkMdat.ContentSize.Value);
          }
Coordinator
Oct 22, 2013 at 2:41 PM
The encoder info is basically a comment although it should be valid bitstream. I think it's an odd combination of inconsistent behavior in ffmpeg (the first file gets its whole mdat content, the second gets just the bits pointed to in the stco offsets) and the MS encoder strangely not having the first chunk offset point at the comment.

You could mimic the behavior by looking at the first stco offset of all files after the first one and skipping a bit of their mdat's but it'd be kinda messy. I really don't see the point unless you are really worried about the file size and in that case I'd handle all the source files the same which would likely drop both of them. It's really handy having them there for diagnostic purposes, though.
Coordinator
Oct 22, 2013 at 2:47 PM
Edited Oct 22, 2013 at 2:48 PM
Oh, one further thing to consider. I don't know about Microsoft's encoder but with the more commonly used x264 library, you are required to leave the encoder info block in there as part of the terms of VideoLAN's x264 license. So before considering dropping them, I'd comb through Microsoft's license to make sure they don't have similar terms.
Coordinator
Oct 22, 2013 at 3:05 PM
Oh, and MDAT content can be anything you like really as far as the Base Media File Format (MPEG-4 Part 14) is concerned. The format of the content is codec specific and tied to the particular track that points to it. You'd have to look at the MPEG-4 Part 10 for h.264's bitstream structure and MPEG-2 Part 7 for AAC's structure.

Delving into the actual bitstreams gets a lot more difficult because they tend to use lots of variable bit length values with complex encodings (like Exponential Golomb codes) which makes reading it very challenging. Generally there shouldn't be much reason for it unless you are writing an actual codec. I've had to do it both for custom codecs I've written and injecting captioning data and other tweaks. Unless you just really enjoy low level bit-fiddling and Information Theory maths, I'd avoid it if possible.
Oct 22, 2013 at 3:47 PM
Thanks again for the tips. I think there is still something with the tables data that I miss but for now I don't know what is that :( Anyway will be looking into that.
Oct 23, 2013 at 9:15 PM
Thanks. I've found the problem. The ctts atom wasn't updated with the chunk entries so players wouldn't know they need to decode rest of the mdat.
Oct 25, 2013 at 9:21 PM
Hello

Sorry to bother you again. Could I ask do me a favor?
If you could take a look at the following file http://sdrv.ms/17jhz57
It's merged according to the previous code but the possibility to play it depends on the player. Ones like MS Zune plays correctly first part than 2nd part is played very fast like there is some sync problem. Others like skydrive player in IE play it correctly almost to the end and then complains that they cannot decode the file. Some don't play it at all. FFmpeg based players seems to play it correctly.
The parts http://sdrv.ms/169J26m and http://sdrv.ms/17jjVRq they both play correctly everywhere.
Have you ever seen something like this and do you have any thoughts what could be a reason? If you want I can also post a code of merge function.