1. 07 Oct, 2009 3 commits
    • Paul E. McKenney's avatar
      rcu: Place root rcu_node structure in separate lockdep class · 978c0b88
      Paul E. McKenney authored
      Before this patch, all of the rcu_node structures were in the same lockdep
      class, so that lockdep would complain when rcu_preempt_offline_tasks()
      acquired the root rcu_node structure's lock while holding one of the leaf
      rcu_nodes' locks.
      
      This patch changes rcu_init_one() to use a separate
      spin_lock_init() for the root rcu_node structure's lock than is
      used for that of all of the rest of the rcu_node structures, which
      puts the root rcu_node structure's lock in its own lockdep class.
      Suggested-by: default avatarPeter Zijlstra <peterz@infradead.org>
      Signed-off-by: default avatarPaul E. McKenney <paulmck@linux.vnet.ibm.com>
      Cc: laijs@cn.fujitsu.com
      Cc: dipankar@in.ibm.com
      Cc: akpm@linux-foundation.org
      Cc: mathieu.desnoyers@polymtl.ca
      Cc: josh@joshtriplett.org
      Cc: dvhltc@us.ibm.com
      Cc: niv@us.ibm.com
      Cc: rostedt@goodmis.org
      Cc: Valdis.Kletnieks@vt.edu
      Cc: dhowells@redhat.com
      LKML-Reference: <12548908983277-git-send-email->
      Signed-off-by: default avatarIngo Molnar <mingo@elte.hu>
      978c0b88
    • Paul E. McKenney's avatar
      rcu: Make hot-unplugged CPU relinquish its own RCU callbacks · e74f4c45
      Paul E. McKenney authored
      The current interaction between RCU and CPU hotplug requires that
      RCU block in CPU notifiers waiting for callbacks to drain.
      
      This can be greatly simplified by having each CPU relinquish its
      own callbacks, and for both _rcu_barrier() and CPU_DEAD notifiers
      to adopt all callbacks that were previously relinquished.
      
      This change also eliminates the possibility of certain types of
      hangs due to the previous practice of waiting for callbacks to be
      invoked from within CPU notifiers.  If you don't every wait, you
      cannot hang.
      Signed-off-by: default avatarPaul E. McKenney <paulmck@linux.vnet.ibm.com>
      Cc: laijs@cn.fujitsu.com
      Cc: dipankar@in.ibm.com
      Cc: akpm@linux-foundation.org
      Cc: mathieu.desnoyers@polymtl.ca
      Cc: josh@joshtriplett.org
      Cc: dvhltc@us.ibm.com
      Cc: niv@us.ibm.com
      Cc: peterz@infradead.org
      Cc: rostedt@goodmis.org
      Cc: Valdis.Kletnieks@vt.edu
      Cc: dhowells@redhat.com
      LKML-Reference: <1254890898456-git-send-email->
      Signed-off-by: default avatarIngo Molnar <mingo@elte.hu>
      e74f4c45
    • Paul E. McKenney's avatar
      rcu: Move rcu_barrier() to rcutree · d0ec774c
      Paul E. McKenney authored
      Move the existing rcu_barrier() implementation to rcutree.c,
      consistent with the fact that the rcu_barrier() implementation is
      tied quite tightly to the RCU implementation.
      
      This opens the way to simplify and fix rcutree.c's rcu_barrier()
      implementation in a later patch.
      Signed-off-by: default avatarPaul E. McKenney <paulmck@linux.vnet.ibm.com>
      Cc: laijs@cn.fujitsu.com
      Cc: dipankar@in.ibm.com
      Cc: akpm@linux-foundation.org
      Cc: mathieu.desnoyers@polymtl.ca
      Cc: josh@joshtriplett.org
      Cc: dvhltc@us.ibm.com
      Cc: niv@us.ibm.com
      Cc: peterz@infradead.org
      Cc: rostedt@goodmis.org
      Cc: Valdis.Kletnieks@vt.edu
      Cc: dhowells@redhat.com
      LKML-Reference: <12548908982563-git-send-email->
      Signed-off-by: default avatarIngo Molnar <mingo@elte.hu>
      d0ec774c
  2. 06 Oct, 2009 2 commits
  3. 05 Oct, 2009 6 commits
    • Thomas Gleixner's avatar
      futex: Fix locking imbalance · eaaea803
      Thomas Gleixner authored
      Rich reported a lock imbalance in the futex code:
      
         http://bugzilla.kernel.org/show_bug.cgi?id=14288
      
      It's caused by the displacement of the retry_private label in
      futex_wake_op(). The code unlocks the hash bucket locks in the
      error handling path and retries without locking them again which
      makes the next unlock fail.
      
      Move retry_private so we lock the hash bucket locks when we retry.
      Reported-by: default avatarRich Ercolany <rercola@acm.jhu.edu>
      Signed-off-by: default avatarThomas Gleixner <tglx@linutronix.de>
      Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
      Cc: Darren Hart <dvhltc@us.ibm.com>
      Cc: stable-2.6.31 <stable@kernel.org>
      LKML-Reference: <new-submission>
      Signed-off-by: default avatarIngo Molnar <mingo@elte.hu>
      eaaea803
    • Aaro Koskinen's avatar
      panic: Fix panic message visibility by calling bust_spinlocks(0) before dying · d014e889
      Aaro Koskinen authored
      Commit ffd71da4 ("panic: decrease oops_in_progress only after
      having done the panic") moved bust_spinlocks(0) to the end of the
      function, which in practice is never reached.
      
      As a result console_unblank() is not called, and on some systems
      the user may not see the panic message.
      
      Move it back up to before the unblanking.
      Signed-off-by: default avatarAaro Koskinen <aaro.koskinen@nokia.com>
      Reviewed-by: default avatarFrederic Weisbecker <fweisbec@gmail.com>
      LKML-Reference: <1254483680-25578-1-git-send-email-aaro.koskinen@nokia.com>
      Signed-off-by: default avatarIngo Molnar <mingo@elte.hu>
      d014e889
    • Paul E. McKenney's avatar
      rcu: Replace the rcu_barrier enum with pointer to call_rcu*() function · 135c8aea
      Paul E. McKenney authored
      The rcu_barrier enum causes several problems:
      
        (1) you have to define the enum somewhere, and there is no
            convenient place,
      
        (2) the difference between TREE_RCU and TREE_PREEMPT_RCU causes
            problems when you need to map from rcu_barrier enum to struct
            rcu_state,
      
        (3) the switch statement are large, and
      
        (4) TINY_RCU really needs a different rcu_barrier() than do the
            treercu implementations.
      
      So replace it with a functionally equivalent but cleaner function
      pointer abstraction.
      Signed-off-by: default avatarPaul E. McKenney <paulmck@linux.vnet.ibm.com>
      Acked-by: default avatarMathieu Desnoyers <mathieu.desnoyers@polymtl.ca>
      Cc: laijs@cn.fujitsu.com
      Cc: dipankar@in.ibm.com
      Cc: akpm@linux-foundation.org
      Cc: josh@joshtriplett.org
      Cc: dvhltc@us.ibm.com
      Cc: niv@us.ibm.com
      Cc: peterz@infradead.org
      Cc: rostedt@goodmis.org
      Cc: Valdis.Kletnieks@vt.edu
      Cc: dhowells@redhat.com
      LKML-Reference: <12541998232366-git-send-email->
      Signed-off-by: default avatarIngo Molnar <mingo@elte.hu>
      135c8aea
    • Paul E. McKenney's avatar
      rcu: Clean up code based on review feedback from Josh Triplett, part 4 · a0b6c9a7
      Paul E. McKenney authored
      These issues identified during an old-fashioned face-to-face code
      review extending over many hours.  This group improves an existing
      abstraction and introduces two new ones.  It also fixes an RCU
      stall-warning bug found while making the other changes.
      
      o	Make RCU_INIT_FLAVOR() declare its own variables, removing
      	the need to declare them at each call site.
      
      o	Create an rcu_for_each_leaf() macro that scans the leaf
      	nodes of the rcu_node tree.
      
      o	Create an rcu_for_each_node_breadth_first() macro that does
      	a breadth-first traversal of the rcu_node tree, AKA
      	stepping through the array in index-number order.
      
      o	If all CPUs corresponding to a given leaf rcu_node
      	structure go offline, then any tasks queued on that leaf
      	will be moved to the root rcu_node structure.  Therefore,
      	the stall-warning code must dump out tasks queued on the
      	root rcu_node structure as well as those queued on the leaf
      	rcu_node structures.
      Signed-off-by: default avatarPaul E. McKenney <paulmck@linux.vnet.ibm.com>
      Cc: laijs@cn.fujitsu.com
      Cc: dipankar@in.ibm.com
      Cc: akpm@linux-foundation.org
      Cc: mathieu.desnoyers@polymtl.ca
      Cc: josh@joshtriplett.org
      Cc: dvhltc@us.ibm.com
      Cc: niv@us.ibm.com
      Cc: peterz@infradead.org
      Cc: rostedt@goodmis.org
      Cc: Valdis.Kletnieks@vt.edu
      Cc: dhowells@redhat.com
      LKML-Reference: <12541491934126-git-send-email->
      Signed-off-by: default avatarIngo Molnar <mingo@elte.hu>
      a0b6c9a7
    • Paul E. McKenney's avatar
      rcu: Clean up code based on review feedback from Josh Triplett, part 3 · 3d76c082
      Paul E. McKenney authored
      Whitespace fixes, updated comments, and trivial code movement.
      
      o	Fix whitespace error in RCU_HEAD_INIT()
      
      o	Move "So where is rcu_write_lock()" comment so that it does
      	not come between the rcu_read_unlock() header comment and
      	the rcu_read_unlock() definition.
      
      o	Move the module_param statements for blimit, qhimark, and
      	qlowmark to immediately follow the corresponding
      	definitions.
      
      o	In __rcu_offline_cpu(), move the assignment to rdp_me
      	inside the "if" statement, given that rdp_me is not used
      	outside of that "if" statement.
      Signed-off-by: default avatarPaul E. McKenney <paulmck@linux.vnet.ibm.com>
      Cc: laijs@cn.fujitsu.com
      Cc: dipankar@in.ibm.com
      Cc: akpm@linux-foundation.org
      Cc: mathieu.desnoyers@polymtl.ca
      Cc: josh@joshtriplett.org
      Cc: dvhltc@us.ibm.com
      Cc: niv@us.ibm.com
      Cc: peterz@infradead.org
      Cc: rostedt@goodmis.org
      Cc: Valdis.Kletnieks@vt.edu
      Cc: dhowells@redhat.com
      LKML-Reference: <12541491931164-git-send-email->
      Signed-off-by: default avatarIngo Molnar <mingo@elte.hu>
      3d76c082
    • Paul E. McKenney's avatar
      rcu: Fix rcu_lock_map build failure on CONFIG_PROVE_LOCKING=y · 162cc279
      Paul E. McKenney authored
      Move the rcu_lock_map definition from rcutree.c to rcupdate.c so that
      TINY_RCU can use lockdep.
      Reported-by: default avatarIngo Molnar <mingo@elte.hu>
      Signed-off-by: default avatarPaul E. McKenney <paulmck@linux.vnet.ibm.com>
      Signed-off-by: default avatarIngo Molnar <mingo@elte.hu>
      162cc279
  4. 23 Sep, 2009 3 commits
    • Paul E. McKenney's avatar
      rcu: Clean up code to address Ingo's checkpatch feedback · 9b2619af
      Paul E. McKenney authored
      Move declarations and update storage classes to make checkpatch happy.
      Signed-off-by: default avatarPaul E. McKenney <paulmck@linux.vnet.ibm.com>
      Cc: laijs@cn.fujitsu.com
      Cc: dipankar@in.ibm.com
      Cc: akpm@linux-foundation.org
      Cc: mathieu.desnoyers@polymtl.ca
      Cc: josh@joshtriplett.org
      Cc: dvhltc@us.ibm.com
      Cc: niv@us.ibm.com
      Cc: peterz@infradead.org
      Cc: rostedt@goodmis.org
      Cc: Valdis.Kletnieks@vt.edu
      Cc: dhowells@redhat.com
      LKML-Reference: <12537246441701-git-send-email->
      Signed-off-by: default avatarIngo Molnar <mingo@elte.hu>
      9b2619af
    • Paul E. McKenney's avatar
      rcu: Clean up code based on review feedback from Josh Triplett, part 2 · 1eba8f84
      Paul E. McKenney authored
      These issues identified during an old-fashioned face-to-face code
      review extending over many hours.
      
      o	Add comments for tricky parts of code, and correct comments
      	that have passed their sell-by date.
      
      o	Get rid of the vestiges of rcu_init_sched(), which is no
      	longer needed now that PREEMPT_RCU is gone.
      
      o	Move the #include of rcutree_plugin.h to the end of
      	rcutree.c, which means that, rather than having a random
      	collection of forward declarations, the new set of forward
      	declarations document the set of plugins.  The new home for
      	this #include also allows __rcu_init_preempt() to move into
      	rcutree_plugin.h.
      
      o	Fix rcu_preempt_check_callbacks() to be static.
      Suggested-by: default avatarJosh Triplett <josh@joshtriplett.org>
      Signed-off-by: default avatarPaul E. McKenney <paulmck@linux.vnet.ibm.com>
      Cc: laijs@cn.fujitsu.com
      Cc: dipankar@in.ibm.com
      Cc: akpm@linux-foundation.org
      Cc: mathieu.desnoyers@polymtl.ca
      Cc: josh@joshtriplett.org
      Cc: dvhltc@us.ibm.com
      Cc: niv@us.ibm.com
      Cc: peterz@infradead.org
      Cc: rostedt@goodmis.org
      Cc: Valdis.Kletnieks@vt.edu
      Cc: dhowells@redhat.com
      LKML-Reference: <12537246443924-git-send-email->
      Signed-off-by: default avatarIngo Molnar <mingo@elte.hu>
      Peter Zijlstra <peterz@infradead.org>
      1eba8f84
    • Paul E. McKenney's avatar
      rcu: Clean up code based on review feedback from Josh Triplett · fc2219d4
      Paul E. McKenney authored
      These issues identified during an old-fashioned face-to-face code
      review extended over many hours.
      
      o	Bury various forms of the "rsp->completed == rsp->gpnum"
      	comparison into an rcu_gp_in_progress() function, which has
      	the beneficial side-effect of forcing consistent use of
      	ACCESS_ONCE().
      
      o	Replace hand-coded arithmetic with DIV_ROUND_UP().
      
      o	Bury several "!list_empty(&rnp->blocked_tasks[rnp->gpnum & 0x01])"
      	instances into an rcu_preempted_readers() function, as this
      	expression indicates that there are no readers blocked
      	within RCU read-side critical sections blocking the current
      	grace period.  (Though there might well be similar readers
      	blocking the next grace period.)
      
      o	Remove a dangling rcu_restart_cpu() declaration that has
      	been dangling for almost 20 minor releases of the kernel.
      Signed-off-by: default avatarPaul E. McKenney <paulmck@linux.vnet.ibm.com>
      Acked-by: default avatarPeter Zijlstra <peterz@infradead.org>
      Cc: laijs@cn.fujitsu.com
      Cc: dipankar@in.ibm.com
      Cc: akpm@linux-foundation.org
      Cc: mathieu.desnoyers@polymtl.ca
      Cc: josh@joshtriplett.org
      Cc: dvhltc@us.ibm.com
      Cc: niv@us.ibm.com
      Cc: rostedt@goodmis.org
      Cc: Valdis.Kletnieks@vt.edu
      Cc: dhowells@redhat.com
      LKML-Reference: <12537246442687-git-send-email->
      Signed-off-by: default avatarIngo Molnar <mingo@elte.hu>
      fc2219d4
  5. 22 Sep, 2009 5 commits
    • Darren Hart's avatar
      futex: Fix wakeup race by setting TASK_INTERRUPTIBLE before queue_me() · 0729e196
      Darren Hart authored
      PI futexes do not use the same plist_node_empty() test for wakeup.
      It was possible for the waiter (in futex_wait_requeue_pi()) to set
      TASK_INTERRUPTIBLE after the waker assigned the rtmutex to the
      waiter. The waiter would then note the plist was not empty and call
      schedule(). The task would not be found by any subsequeuent futex
      wakeups, resulting in a userspace hang.
      
      By moving the setting of TASK_INTERRUPTIBLE to before the call to
      queue_me(), the race with the waker is eliminated. Since we no
      longer call get_user() from within queue_me(), there is no need to
      delay the setting of TASK_INTERRUPTIBLE until after the call to
      queue_me().
      
      The FUTEX_LOCK_PI operation is not affected as futex_lock_pi()
      relies entirely on the rtmutex code to handle schedule() and
      wakeup.  The requeue PI code is affected because the waiter starts
      as a non-PI waiter and is woken on a PI futex.
      
      Remove the crusty old comment about holding spinlocks() across
      get_user() as we no longer do that. Correct the locking statement
      with a description of why the test is performed.
      Signed-off-by: default avatarDarren Hart <dvhltc@us.ibm.com>
      Acked-by: default avatarPeter Zijlstra <peterz@infradead.org>
      Cc: Steven Rostedt <rostedt@goodmis.org>
      Cc: Eric Dumazet <eric.dumazet@gmail.com>
      Cc: Dinakar Guniguntala <dino@in.ibm.com>
      Cc: John Stultz <johnstul@us.ibm.com>
      LKML-Reference: <20090922053038.8717.97838.stgit@Aeon>
      Signed-off-by: default avatarIngo Molnar <mingo@elte.hu>
      0729e196
    • Darren Hart's avatar
      futex: Correct futex_q woken state commentary · d8d88fbb
      Darren Hart authored
      Use kernel-doc format to describe struct futex_q.
      
      Correct the wakeup definition to eliminate the statement about
      waking the waiter between the plist_del() and the q->lock_ptr = 0.
      
      Note in the comment that PI futexes have a different definition of
      the woken state.
      Signed-off-by: default avatarDarren Hart <dvhltc@us.ibm.com>
      Acked-by: default avatarPeter Zijlstra <peterz@infradead.org>
      Cc: Steven Rostedt <rostedt@goodmis.org>
      Cc: Eric Dumazet <eric.dumazet@gmail.com>
      Cc: Dinakar Guniguntala <dino@in.ibm.com>
      Cc: John Stultz <johnstul@us.ibm.com>
      LKML-Reference: <20090922053029.8717.62798.stgit@Aeon>
      Signed-off-by: default avatarIngo Molnar <mingo@elte.hu>
      d8d88fbb
    • Darren Hart's avatar
      futex: Make function kernel-doc commentary consistent · d96ee56c
      Darren Hart authored
      Make the existing function kernel-doc consistent throughout
      futex.c, following Documentation/kernel-doc-nano-howto.txt as
      closely as possible.
      
      When unsure, at least be consistent within futex.c.
      Signed-off-by: default avatarDarren Hart <dvhltc@us.ibm.com>
      Acked-by: default avatarPeter Zijlstra <peterz@infradead.org>
      Cc: Steven Rostedt <rostedt@goodmis.org>
      Cc: Eric Dumazet <eric.dumazet@gmail.com>
      Cc: Dinakar Guniguntala <dino@in.ibm.com>
      Cc: John Stultz <johnstul@us.ibm.com>
      LKML-Reference: <20090922053022.8717.13339.stgit@Aeon>
      Signed-off-by: default avatarIngo Molnar <mingo@elte.hu>
      d96ee56c
    • Darren Hart's avatar
      futex: Correct queue_me and unqueue_me commentary · d40d65c8
      Darren Hart authored
      The queue_me/unqueue_me commentary is oddly placed and out of date.
      Clean it up and correct the inaccurate bits.
      Signed-off-by: default avatarDarren Hart <dvhltc@us.ibm.com>
      Acked-by: default avatarPeter Zijlstra <peterz@infradead.org>
      Cc: Steven Rostedt <rostedt@goodmis.org>
      Cc: Eric Dumazet <eric.dumazet@gmail.com>
      Cc: Dinakar Guniguntala <dino@in.ibm.com>
      Cc: John Stultz <johnstul@us.ibm.com>
      LKML-Reference: <20090922053015.8717.71713.stgit@Aeon>
      Signed-off-by: default avatarIngo Molnar <mingo@elte.hu>
      d40d65c8
    • Darren Hart's avatar
      futex: Correct futex_wait_requeue_pi() commentary · 56ec1607
      Darren Hart authored
      Correct various typos and formatting inconsistencies in the
      commentary of futex_wait_requeue_pi().
      Signed-off-by: default avatarDarren Hart <dvhltc@us.ibm.com>
      Acked-by: default avatarPeter Zijlstra <peterz@infradead.org>
      Cc: Steven Rostedt <rostedt@goodmis.org>
      Cc: Eric Dumazet <eric.dumazet@gmail.com>
      Cc: Dinakar Guniguntala <dino@in.ibm.com>
      Cc: John Stultz <johnstul@us.ibm.com>
      LKML-Reference: <20090922052958.8717.21932.stgit@Aeon>
      Signed-off-by: default avatarIngo Molnar <mingo@elte.hu>
      56ec1607
  6. 21 Sep, 2009 21 commits