enable/disable_msi_mode have several side effects which keeps them from being
generally useful.  So this patch replaces them with with two much more
targeted functions: msi_set_enable and msix_set_enable.

This patch makes pci_dev->msi_enabled and pci_dev->msix_enabled the definitive
way to test if linux has enabled the msi capability, and has the appropriate
msi data structures set up.

This patch ensures that while writing the msi messages in save/restore and
during device initialization we have the msi capability disabled so we don't
get into races.  The pci spec requires that we do not have the msi capability
enabled and the msi messages unmasked while we write the messages.  Completely
disabling the capability is overkill but it is easy :)

Care has been taken so we never have both a msi capability and intx enabled
simultaneously.  We haven't run into a problem yet but better safe then sorry.
Signed-off-by: Eric W. Biederman <ebiederm@xmission.com>
Cc: Michael Ellerman <michael@ellerman.id.au>
Cc: Paul Mackerras <paulus@samba.org>
Cc: Benjamin Herrenschmidt <benh@kernel.crashing.org>
Cc: Greg KH <greg@kroah.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>