Todo

-- Bug fixes

Prove correct:
- decode_update_progress() in h264.c
h264_race_checking branch has some work on h264,
but not that function. It might be worth putting
the branch under #ifdef DEBUG in mainline, but
the code would have to be cleaner.
- MPV_lowest_referenced_row() and co in mpegvideo.c
- Same in vp3.

h264:
- Files split at the wrong NAL unit don't (and can't)
be decoded with threads (e.g. TS split so PPS is after
the frame, PAFF with two fields in a packet). Scan the
packet at the start of decode and don't finish_thread_setup
until all PPS/SPS have been encountered.
- Some files with harmless header errors decode
differently.

mpeg4:
- Packed B-frames need to be explicitly split up
when frame threading is on. Currently disabled for
this reason.
- Support interlaced.

mpeg1:
- Seeking always prints "first frame not a keyframe"
with threads on. Currently disabled for this reason.

other:
- ffmpeg with -vsync != 0 breaks A/V sync because
it can't handle more than 1 frame of decoder delay.
This can be seen with H.264 on mainline.
- Decoding single-frame files with 3+ threads never
outputs anything in ffmpeg. It should stop the initial
delay and just return a frame immediately if one of the
first N frames is an EOF packet.
- Error resilience has to run before ff_report_frame_progress()
is called.
- 'make test' must pass. This means pthread emulation
must work the same way without --enable-pthreads.
There also seems to be some problem related to
draw_edges and the mpeg4 encoder.

-- Optimization

- await_references() in h264 spends too much time
skipping zeros in refs[]. Add a next_array_nonzero
macro or something to use write-combining checks;
this would probably be useful somewhere else too.
- fill_edges() is disabled for h264 PAFF+MT
because it writes into the other field's
thread's pixels.
- Check update_thread_context() functions and make
sure they only copy what they need to.
- Support frame+slice threading.

-- Style

- See if AVCodec init_thread_copy and update_thread_context
can be merged cleanly.

-- Features

- Support streams with width/height changing. This
requires flushing all current frames (and buffering
the input in the meantime), closing the codec and
reopening it. Or don't support it.
- Support encoding. Might need more threading primitives
for good ratecontrol; would be nice for audio and libavfilter too.
- After merging to mainline, deprecate avcodec_thread_init
and just set thread_count.

-- Samples

http://astrange.ithinksw.net/ffmpeg/mt-samples/

See yuvcmp.c in this directory to compare decoded samples.
Try commenting out ff_thread_finish_setup calls so
only one thread runs at once, and then binary search+
scattering printfs to look for differences in codec contexts.
