This avoids a BUG_ON in the enter_vt path due to objects being in the GTT
when we shouldn't have ever let them be (as we're not supposed to touch the
device during that time).

This was triggered by a change in the 2D driver to use the GTT mapping of
objects after pinning them to improve software fallback performance.
Signed-off-by: Eric Anholt <eric@anholt.net>
Signed-off-by: Dave Airlie <airlied@linux.ie>