spu_forget will do mmput on the DMA address space,
which can lead to lots of other stuff getting triggered.
We better not hold a semaphore here that we might
need in the process.

Noticed by Al Viro.
Signed-off-by: Arnd Bergmann <arndb@de.ibm.com>
Signed-off-by: Paul Mackerras <paulus@samba.org>