Upon failure of vlc_va_Initialize() the error path jumped to the label 'error',
which would call sys->conn->unlock(). However the error path was taken before
sys->conn->lock() was called and even worse sys->conn = NULL in this case. This
would result into crash obviously. Therefor move the 'error' label one line down,
so it only calls the cleanup routions and add a lable 'unlock' for the error paths
that need to unlock first before cleaning up.

If no memory could be allocated, then return 0 in Read() for this stream_filter.
VLC core will then neatly handle this error situation. Otherwise a buffer overflow
is triggered.

Make sure cancellation does not happen in these routines.

If creating a subpicture fails, then the picture is not going to carry a
reference to the cached subpicture structure. In this case the cached
subpicture was being freed anyway. Which resulted in a crash.

Instead check if there is an subpicture cache id and if it is > 0,
before attempting to release it.

fixup: a47b657e

The selected profile wasn't checked against card's ones.
(ex: resulting in sending MPEG2 to a VC1/h264 only card)
Signed-off-by: Jean-Paul Saman <jpsaman@videolan.org>

- remove debugging
- add locking around refcounting where it was missing

- rework refcounting
- rename functions to have cache_ prefix
- fix resetting of sys->render.i_cache in failure paths

Subpictures are already detached from the surface and can de free'd independently.

Refactor the subpicture handling code to be more consistent.

Treat subpicture_cache_t::i_id as unsigned int as it should have been. Use 0 as the special value indicating an invalid subpicture_cache_t definition.

Set p_base to NULL before passing it to vaMapBuffer.

The idea is to get the  number of pictures needed according to core for direct rendering.
So that the decoder knows how many surfaces it needs to allocated. However the vout is instantiated,
after the codec is. Luckily the number needed is rather static, so if there is no vout yet, then return
the static value.

The PictureRelease function destroyed the Surface as it needed to do when there was no
shared pool of surfaces. However now a shared pool of surfaces is used and the PictureRelease
function should not destroy it anymore.

Do refcounting when the subpicture is actually attached to the picture,
thus only in SubpictureRegionsLink() and SubpictureRegionsUnlink() functions

Conflicts:
	configure.ac

don't run aclocal for those (we only patch Makefile.am)

- remove stuff we don't need
- replace prefix by template (.pc/.la/-config)
- include script to replace prefix template by $PWD
- use widely available bzip2 format so xz is not needed

Use:
$ make package-win32
$ make package-win64
$ make package-osx
$ make package-whatever

Reported-by: Joseph Atkinson <jsa@freebsd.org>

This reverts commit ec76c41b.

Setting the translation mode is the responsibility of the program. A
library must not interfere with that. It could perhaps be added to VLC.
However it should not be required as we now force binary mode by
default in vlc_fopen() and always specify an explicit mode when calling
fopen() directly. Also it would make VLC behave differently than other
LibVLC applications, which may raise really weird bugs.