Rename vlc_va_conn_t::p_display_x11 to vlc_va_conn_t::p_x11.

Simplify check for valid surface and chroma.

Cleanup no longer needed variables.

The video output is now responsible for the surfaces and the pictures. It
needs a special path in modules/codec/avcodec/video.c to make it happen.
This simplifies the code and locking considerably.

This reverts commit cede8daf.

The struct picture_pool_configuration_t has 2 functions called Lock and
Unlock, which are called when a picture is taken from the pool and when
a picture is given back to the pool.

The surface i_refcount and i_order counter are no longer sufficient
to tell which surface is free to use.

The function VoutDisplayEventKeyDispatch() missed a return statement.

Add private release function, which is not going to be overridden by
Picture Poool. It is not allowed to alter the refcount from the picture
in this callback function. Its purpose is to provide a callback to
release private resource (picture_sys_t) when needed.
Feeing of picture_sys_t itself is done inside the pf_release() callback.
Which will be called only on picture_Delete() when used inside a picture
Pool.

Force refcount to one in vlc_va_Get().

surface->i_order is not the display order, but more the age of the
surface in use.

Small cleanup of error message so it shows more usefull information.

Use a static mutex to protect access for libva.

Setting the palette does not work and the function always returns with
status VA_STATUS_ERROR_OPERATION_FAILED. Re-enable when figure out how
it should be used then.

Move setting of VAImage properties to the function where the image is
created.

Create a palette for a VAImage when our region picture has one.

vout_display_NewWindow() expects that wnd_cfg.width and wnd_cfg.height are the
fullscreen size of current display. Otherwise scaling to larger sizes then the
decoded picture size is not going to work.

The subtitle subpictures are regenerated for each call to vd->Prepare.
This is not very efficient and results in flickering display of subtitles.
Implement a VASubpicture's cache to solve the flickering.

Remove debug leftover.

Share SubpictureDestroy() between vaapi.c and vaapi_x11.c.

Remove displaying of non VLC_CODEC_VAAPI_SURFACE pictures in VAAPI X11 vout output.

Increase the number of surfaces requested to 30. VAAPI X11 vout
creates enough pictures to take the direct rendering code path
in vout core. Which means there are about 30 pictures in flight.

With a low number of surfaces these will be reused before they
are being displayed and this results in out of order display of
pictures. The viewer will see a stuttering image.

These conditions should not happen.

Add some more debugging code.

The surface mutex is no longer needed now that VASubpictures remember
to which VASurfaceID they were attached. The subpictures are now remembered
inside picture_sys_t directly iso at the vlc_va_surface_t structure.

Release subpicture association when picture is released. Note that picture_pool_New() overrides this
pictures pf_release() function and restores it before cleaning up the picture pool.

When surfaces are reclaimed or released before they are being displayed,
then the subpicture association still exists. This code tries to catch
such pictures and will release the subpictures association.

- do picture refcounting from pf_release() functions.

- Renamed PictureReleaseLocked to ReleaseSurfaceLocked
- Cleanup code paths

Introduce VLC_CODEC_VAAPI_SURFACE to determine what to do with the
passed picture. If VLC_CODEC_VAAPI_SURFACE, then the picture caries
a VASurface which must be used to display. Else it is a normal
picture and should be treated as such.

The video output could also get picture_t * that were not decoded
on the GPU. Still use the GPU to display them.

DBus, XOSD and Audio Scrobbler interfaces depend on this.

Also protect concurrent access to p_sys between main thread and libvlc callbacks

fixes: #4827