• Herbert Xu's avatar
    [PATCH] nbd: fix TX/RX race condition · 4b2f0260
    Herbert Xu authored
    Janos Haar of First NetCenter Bt.  reported numerous crashes involving the
    NBD driver.  With his help, this was tracked down to bogus bio vectors
    which in turn was the result of a race condition between the
    receive/transmit routines in the NBD driver.
    
    The bug manifests itself like this:
    
    CPU0				CPU1
    do_nbd_request
    	add req to queuelist
    	nbd_send_request
    		send req head
    		for each bio
    			kmap
    			send
    				nbd_read_stat
    					nbd_find_request
    					nbd_end_request
    			kunmap
    
    When CPU1 finishes nbd_end_request, the request and all its associated
    bio's are freed.  So when CPU0 calls kunmap whose argument is derived from
    the last bio, it may crash.
    
    Under normal circumstances, the race occurs only on the last bio.  However,
    if an error is encountered on the remote NBD server (such as an incorrect
    magic number in the request), or if there were a bug in the server, it is
    possible for the nbd_end_request to occur any time after the request's
    addition to the queuelist.
    
    The following patch fixes this problem by making sure that requests are not
    added to the queuelist until after they have been completed transmission.
    
    In order for the receiving side to be ready for responses involving
    requests still being transmitted, the patch introduces the concept of the
    active request.
    
    When a response matches the current active request, its processing is
    delayed until after the tranmission has come to a stop.
    
    This has been tested by Janos and it has been successful in curing this
    race condition.
    
    From: Herbert Xu <herbert@gondor.apana.org.au>
    
      Here is an updated patch which removes the active_req wait in
      nbd_clear_queue and the associated memory barrier.
    
      I've also clarified this in the comment.
    Signed-off-by: default avatarHerbert Xu <herbert@gondor.apana.org.au>
    Cc: <djani22@dynamicweb.hu>
    Cc: Paul Clements <Paul.Clements@SteelEye.com>
    Signed-off-by: default avatarHerbert Xu <herbert@gondor.apana.org.au>
    Signed-off-by: default avatarAndrew Morton <akpm@osdl.org>
    Signed-off-by: default avatarLinus Torvalds <torvalds@osdl.org>
    4b2f0260
nbd.c 19.3 KB