Signed-off-by: Jean-Baptiste Kempf <jb@videolan.org>

close #14464

The mutex was allocated in Open() and destroyed in Setup()...

The configuration ID must exist in this code path.

(The reference count was updated without the mutex.)

Allocating a picture does not mean much w.r.t. buffering. The decoder
may just have started.

This function was intended to work around a certain class of bugs
inside video decoders whereby the a reference picture was never
dereferenced, i.e. missing decoder_UnlinkPicture() call.

There are however some situations where this hack would release a
still referenced picture:

- If the video output or decoder has a high latency or if a video
  filter holds pictures (deinterlace), there may be zero free pictures
  even though the total number of pictures is sufficient overall.
  In that case, waiting for a picture buffer to be released normally
  is the right and safe approach.

- If the byte stream is invalid or corrupt such that the number of
  required pictures (DPB size) is underestimated, dropping frames is
  acceptable. Frames would be corrupt anyway due to missing references.
  (This case could be better worked around by allocating extra
   pictures on-the-fly, though this would require memory copying in the
   video output.)

- Even if the decoder indeed leaks pictures, the oldest referenced
  picture is not necessarily among those leaked.

If the picture was not truly leaked, vout_FixLeaks() would cause an
extraneous picture release. This should lead to an assertion failure in
picture_Release(). Without assertions, it would lead to undefined
behaviour, especially invalid pointer use in case of hardware surfaces.

In any case, picture leaks are still recovered when resetting the video
output with vout_Reset(), after the decoder is destroyed. That might
turn out to be a problem too though.

VLC itself has no submodules

More comments, try to make it clearer, release p_block and p_pic in only one
place. It also fixes a p_block leak if we were not able to create a p_pic.

According to ISO 14496-10 7.4.1.2.4,
next first NAL of next picture appears when
nal_ref_ids differs with one being equal to 0.

The decoder can now detect and avoid this case. Retaining the
conversion is counter-productive (especially for transcode).

This partially reverts commit 84b23f96,
which is no longer necessary.

This makes checking for video output support before activation a lot
simpler (especially avoiding callbacks).

This effectively reverts commit b667100b.

The decoder can request another format, or even retry the same format
with better luck next time. So do not set b_error. b_error is for
unrecoverable failures, such as if the decoder module is upset.