avcodec: cleanup encodeAudio

Handle better cases when we don't have input frame size
same as encoder input frame size. Also gather input samples
untill we get full frame fillet and not just next block.

Set length based on used samples and not frame_size

Also try to make memory calculations cleaner so they don't
overshoot and write stuff all over.

Refs #8265