Instead just set the flag to disabled, since this obviously
isn't used at the moment.

Also fix a case in DecodeVideo where the b_direct flag for the
wrong port was checked.

This simplifies enabling the direct buffer behaviour for the
cases where it actually is desired.
Signed-off-by: Martin Storsjö <martin@martin.st>