This reverts commit 2c81ce4c.

It caused several new troubles (eg suspend slowdown bisected down to
this patch by Pavel Machek), so just revert it for now.
Signed-off-by: Alexey Starikovskiy <astarikovskiy@suse.de>
Cc: Pavel Machek <pavel@suse.cz>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>