Skip to content

L
linux-davinci
An error occurred fetching the project authors.
Switch branch/tag
  1. 07 May, 2007 1 commit
    • Roland Dreier's avatar
      IB: Return "maybe missed event" hint from ib_req_notify_cq() · ed23a727
      Roland Dreier authored 
      The semantics defined by the InfiniBand specification say that
completion events are only generated when a completions is added to a
completion queue (CQ) after completion notification is requested.  In
other words, this means that the following race is possible:

	while (CQ is not empty)
		ib_poll_cq(CQ);
	// new completion is added after while loop is exited
	ib_req_notify_cq(CQ);
	// no event is generated for the existing completion

To close this race, the IB spec recommends doing another poll of the
CQ after requesting notification.

However, it is not always possible to arrange code this way (for
example, we have found that NAPI for IPoIB cannot poll after
requesting notification).  Also, some hardware (eg Mellanox HCAs)
actually will generate an event for completions added before the call
to ib_req_notify_cq() -- which is allowed by the spec, since there's
no way for any upper-layer consumer to know exactly when a completion
was really added -- so the extra poll of the CQ is just a waste.

Motivated by this, we add a new flag "IB_CQ_REPORT_MISSED_EVENTS" for
ib_req_notify_cq() so that it can return a hint about whether the a
completion may have been added before the request for notification.
The return value of ib_req_notify_cq() is extended so:

	 < 0	means an error occurred while requesting notification
	== 0	means notification was requested successfully, and if
		IB_CQ_REPORT_MISSED_EVENTS was passed in, then no
		events were missed and it is safe to wait for another
		event.
	 > 0	is only returned if IB_CQ_REPORT_MISSED_EVENTS was
		passed in.  It means that the consumer must poll the
		CQ again to make sure it is empty to avoid the race
		described above.

We add a flag to enable this behavior rather than turning it on
unconditionally, because checking for missed events may incur
significant overhead for some low-level drivers, and consumers that
don't care about the results of this test shouldn't be forced to pay
for the test.
Signed-off-by: default avatarRoland Dreier <rolandd@cisco.com>
      ed23a727
  3. 04 Feb, 2007 1 commit
    • Michael S. Tsirkin's avatar
      IB: Return qp pointer as part of ib_wc · 062dbb69
      Michael S. Tsirkin authored 
      struct ib_wc currently only includes the local QP number: this matches
the IB spec, but seems mostly useless. The following patch replaces
this with the pointer to qp itself, and updates all low level drivers
and all users.

This has the following advantages:
- Ability to get a per-qp context through wc->qp->qp_context
- Existing drivers already have the qp pointer ready in poll cq, so
  this change actually saves a tiny bit (extra memory read) on data path
  (for ehca it would actually be expensive to find the QP pointer when
  polling a CQ, but ehca does not support SRQ so we can leave wc->qp as
  NULL for ehca)
- Users that need the QP number can still get it through wc->qp->qp_num

Use case:

In IPoIB connected mode code, I have a common CQ shared by multiple
QPs.  To track connection usage, I need a way to get at some per-QP
context upon the completion, and I would like to avoid allocating
context object per work request just to stick a QP pointer into it.
With this code, I can just use wc->qp->qp_context.
Signed-off-by: default avatarMichael S. Tsirkin <mst@mellanox.co.il>
Signed-off-by: default avatarRoland Dreier <rolandd@cisco.com>
      062dbb69
  5. 08 Jan, 2007 1 commit
  7. 29 Nov, 2006 1 commit
    • Roland Dreier's avatar
      IB/mthca: Fix section mismatches · f4f3d0f0
      Roland Dreier authored 
      Commit b3b30f5e ("IB/mthca: Recover from catastrophic errors")
introduced some section mismatch breakage, because the error recovery
code tears down and reinitializes the device, which calls into lots of
code originally marked __devinit and __devexit from regular .text.

Fix this by getting rid of these now-incorrect section markers.

Reported by Randy Dunlap <randy.dunlap@oracle.com>.
Signed-off-by: default avatarRoland Dreier <rolandd@cisco.com>
      f4f3d0f0
  9. 17 Oct, 2006 1 commit
    • Arthur Kepner's avatar
      IB/mthca: Use mmiowb after doorbell ring · 1f5c23e2
      Arthur Kepner authored 
      We discovered a problem when running IPoIB applications on multiple
CPUs on an Altix system. Many messages such as:

ib_mthca 0002:01:00.0: SQ 000014 full (19941644 head, 19941707 tail, 64 max, 0 nreq)

appear in syslog, and the driver wedges up.

Apparently this is because writes to the doorbells from different CPUs
reach the device out of order. The following patch adds mmiowb() calls
after doorbell rings to ensure the doorbell writes are ordered.
Signed-off-by: default avatarArthur Kepner <akepner@sgi.com>
Signed-off-by: default avatarRoland Dreier <rolandd@cisco.com>
      1f5c23e2
  11. 22 Sep, 2006 1 commit
  13. 18 Jun, 2006 2 commits
  15. 09 May, 2006 1 commit
    • Roland Dreier's avatar
      IB/mthca: Fix race in reference counting · a3285aa4
      Roland Dreier authored 
      Fix races in in destroying various objects.  If a destroy routine
waits for an object to become free by doing

	wait_event(&obj->wait, !atomic_read(&obj->refcount));
	/* now clean up and destroy the object */

and another place drops a reference to the object by doing

	if (atomic_dec_and_test(&obj->refcount))
		wake_up(&obj->wait);

then this is susceptible to a race where the wait_event() and final
freeing of the object occur between the atomic_dec_and_test() and the
wake_up().  And this is a use-after-free, since wake_up() will be
called on part of the already-freed object.

Fix this in mthca by replacing the atomic_t refcounts with plain old
integers protected by a spinlock.  This makes it possible to do the
decrement of the reference count and the wake_up() so that it appears
as a single atomic operation to the code waiting on the wait queue.

While touching this code, also simplify mthca_cq_clean(): the CQ being
cleaned cannot go away, because it still has a QP attached to it.  So
there's no reason to be paranoid and look up the CQ by number; it's
perfectly safe to use the pointer that the callers already have.
Signed-off-by: default avatarRoland Dreier <rolandd@cisco.com>
      a3285aa4
  17. 29 Mar, 2006 1 commit
  19. 20 Mar, 2006 3 commits
    • Roland Dreier's avatar
      IB/mthca: Add device-specific support for resizing CQs · 4885bf64
      Roland Dreier authored 
      Add low-level driver support for resizing CQs (both kernel and
userspace) to mthca.
Signed-off-by: default avatarRoland Dreier <rolandd@cisco.com>
      4885bf64
    • Roland Dreier's avatar
      IB/mthca: Get rid of might_sleep() annotations · 399d7921
      Roland Dreier authored 
      The might_sleep() annotations in mthca are silly -- they all occur
shortly before calls that will end up in core functions like kmalloc()
that will print the same warning in an unsafe context anyway.  In
fact, beyond cluttering the source, we're actually bloating text with
CONFIG_DEBUG_SPINLOCK_SLEEP and/or CONFIG_PREEMPT_VOLUNTARY set.

With both options set, getting rid of the might_sleep()s saves a lot:
add/remove: 0/0 grow/shrink: 0/7 up/down: 0/-171 (-171)
function                                     old     new   delta
mthca_pd_alloc                               132     109     -23
mthca_init_cq                                969     946     -23
mthca_mr_alloc                               592     568     -24
mthca_pd_free                                 67      42     -25
mthca_free_mr                                219     194     -25
mthca_free_cq                                570     545     -25
mthca_fmr_alloc                              742     716     -26
Signed-off-by: default avatarRoland Dreier <rolandd@cisco.com>
      399d7921
    • Roland Dreier's avatar
      IB/mthca: Make functions that never fail return void · d9b98b0f
      Roland Dreier authored 
      The function mthca_free_err_wqe() can never fail, so get rid of its
return value.  That means handle_error_cqe() doesn't have to check
what mthca_free_err_wqe() returns, which means it can't fail either
and doesn't have to return anything either.  All this results in
simpler source code and a slight object code improvement:

add/remove: 0/0 grow/shrink: 0/2 up/down: 0/-10 (-10)
function                                     old     new   delta
mthca_free_err_wqe                            83      81      -2
mthca_poll_cq                               1758    1750      -8
Signed-off-by: default avatarRoland Dreier <rolandd@cisco.com>
      d9b98b0f
  21. 06 Jan, 2006 1 commit
  23. 15 Dec, 2005 1 commit
  25. 10 Nov, 2005 1 commit
  27. 29 Oct, 2005 1 commit
  29. 27 Aug, 2005 6 commits
  31. 27 Jul, 2005 1 commit
  33. 08 Jul, 2005 1 commit
  35. 27 Jun, 2005 5 commits
  37. 16 Apr, 2005 4 commits