Don't mess up deinterleaving if there is leftover-samples and keep
leftover samples in interleaved format. Fixes #8570