This patch fixes the setup/destroy of several modules to make
the cleanup stage run to completion.
Signed-off-by: Simon Que <sque@ti.com>