Special QPs are not allocated using the regular QP number bitmap, so
when they are destroyed, their QP number should not be freed in the
bitmap.

Found by Dotan Barak of Mellanox.
Signed-off-by: Roland Dreier <rolandd@cisco.com>