MP4/AVC: Parsed dimensions wrong



When parsing mpeg-4 files (H264 - MPEG-4 AVC - part 10 - avc1) the video dimensions (TrackHeaderBox: "width", "height") are wrong. I tried this for different video samples.



jclary wrote Feb 7, 2013 at 3:35 PM

The folks in the MPEG group decided to use 16.16 fixed-point for those. There's no equivalent precision data type in C# so anything you convert it to in C# would result in changing the underlying value when you try to save the box again.

In most cases they will be whole numbers so you can just do >>16. A double or decimal conversion would just divide by UInt32.MaxValue but it'll still round it badly.

Note that this may not be the size of the underlying h.264 anyway. It's supposed to tell the player what to scale to. Figuring out what the actual size is turns out to be exceedingly complicated. It involves parsing the Sequence Parameter Set NALU and getting out the width and height in macroblocks then subtracting left, top, right and bottom cropping values... something like:

width = ((pic_width_in_mbs_minus1 +1)16) - frame_crop_left_offset2 - frame_crop_right_offset2;
height= ((2 - frame_mbs_only_flag)
(pic_height_in_map_units_minus1 +1) * 16) - (frame_crop_top_offset * 2) - (frame_crop_bottom_offset * 2)

Sadly I don't have an PPS parser I can open-source at the moment. It's quite involved as it is a bit-oriented format with variable-length values.

jclary wrote Feb 7, 2013 at 4:32 PM

Oops, that should have been divide by UInt16.MaxValue.

I'm looking at doing a fixed point type struct for this. I've already got a fraction type (MatrixIO.Fraction in the MatrixIO.IO project) that was inspired by this problem but that's not ideal either. I'd have done a FixedPoint<TUpper, TLower> but the lack of any way to restrict generics to integer types and no easy access to arithmetic with generics makes that a bad solution. I think I'll just have to do the various fixed point formats in the spec as separate types. I'd probably have anyway since I think there's a 4.4 fixed point somewhere else in the spec and no 4 bit type in C#.

Any way you look at it there will either be manual bit fiddling or loss of precision. :/

wrote Feb 14, 2013 at 6:18 PM