- 29 Oct, 2009 13 commits
-
-
Peter Zijlstra authored
Hopefully a more readable version of the same. Signed-off-by:
Peter Zijlstra <a.p.zijlstra@chello.nl> Signed-off-by:
Dinakar Guniguntala <dino@in.ibm.com> Cc: John Stultz <johnstul@us.ibm.com> Cc: Darren Hart <dvhltc@us.ibm.com> Cc: John Kacur <jkacur@redhat.com> Signed-off-by:
Thomas Gleixner <tglx@linutronix.de>
-
Peter Zijlstra authored
APERF/MPERF support for cpu_power. APERF/MPERF is arch defined to be a relative scale of work capacity per logical cpu, this is assumed to include SMT and Turbo mode. APERF/MPERF are specified to both reset to 0 when either counter wraps, which is highly inconvenient, since that'll give a blimp when that happens. The manual specifies writing 0 to the counters after each read, but that's 1) too expensive, and 2) destroys the possibility of sharing these counters with other users, so we live with the blimp - the other existing user does too. Signed-off-by:
-
Peter Zijlstra authored
[ dino: backport to 31-rt ] Signed-off-by:
-
Dinakar Guniguntala authored
Move some of the aperf/mperf code out from the cpufreq driver thingy so that other people can enjoy it too. Signed-off-by:
-
Peter Zijlstra authored
Move the APERFMPERF capacility into a X86_FEATURE flag so that it can be used outside of the acpi cpufreq driver. [ dino: backport to 31-rt ] Signed-off-by:
-
Peter Zijlstra authored
Its a source of fail, also, now that cpu_power is dynamical, its a waste of time. before: <idle>-0 [000] 132.877936: find_busiest_group: avg_load: 0 group_load: 8241 power: 1 after: bash-1689 [001] 137.862151: find_busiest_group: avg_load: 10636288 group_load: 10387 power: 1 [ dino: backport to 31-rt ] Signed-off-by:
-
Peter Zijlstra authored
When the capacity drops low, we want to migrate load away. Allow the load-balancer to remove all tasks when we hit rock bottom. [ dino: backport to 31-rt ] Signed-off-by:
-
Peter Zijlstra authored
Keep an average on the amount of time spend on RT tasks and use that fraction to scale down the cpu_power for regular tasks. Signed-off-by:
-
Peter Zijlstra authored
Recompute the cpu_power for each cpu during load-balance Signed-off-by:
-
Peter Zijlstra authored
The idea is that multi-threading a core yields more work capacity than a single thread, provide a way to express a static gain for threads. [ dino: backport to 31-rt ] Signed-off-by:
-
Peter Zijlstra authored
In order to prepare for a more dynamic cpu_power, update the group sum while walking the sched domains during load-balance. Signed-off-by:
-
Peter Zijlstra authored
Do the placement thing using SD flags XXX: consider degenerate bits [ dino: backport to 31-rt ] Signed-off-by:
-
Peter Zijlstra authored
cpu_power is supposed to be a representation of the process capacity of the cpu, not a value to randomly tweak in order to affect placement. Remove the placement hacks. [ dino: backport to 31-rt ] Signed-off-by:
-
- 28 Oct, 2009 11 commits
-
-
Darren Hart authored
The queue_me/unqueue_me commentary is oddly placed and out of date. Clean it up and correct the inaccurate bits. Signed-off-by:
Darren Hart <dvhltc@us.ibm.com> Acked-by:
Peter Zijlstra <peterz@infradead.org> Cc: Steven Rostedt <rostedt@goodmis.org> Cc: Eric Dumazet <eric.dumazet@gmail.com> Cc: Dinakar Guniguntala <dino@in.ibm.com> Cc: John Stultz <johnstul@us.ibm.com> LKML-Reference: <20090922053015.8717.71713.stgit@Aeon> Signed-off-by:
Ingo Molnar <mingo@elte.hu>
-
Darren Hart authored
When requeuing tasks from one futex to another, the reference held by the requeued task to the original futex location needs to be dropped eventually. Dropping the reference may ultimately lead to a call to "iput_final" and subsequently call into filesystem- specific code - which may be non-atomic. It is therefore safer to defer this drop operation until after the futex_hash_bucket spinlock has been dropped. Originally-From: Helge Bahmann <hcb@chaoticmind.net> Signed-off-by:
-
Darren Hart authored
The memory barrier semantics of futex_wait_queue_me() are non-obvious. Add some commentary to try and clarify it. Signed-off-by:
-
Darren Hart authored
The state machine described in the comments wasn't updated with a follow-on fix. Address that and cleanup the corresponding commentary in the function. Signed-off-by:
-
Thomas Gleixner authored
Rich reported a lock imbalance in the futex code: http://bugzilla.kernel.org/show_bug.cgi?id=14288 It's caused by the displacement of the retry_private label in futex_wake_op(). The code unlocks the hash bucket locks in the error handling path and retries without locking them again which makes the next unlock fail. Move retry_private so we lock the hash bucket locks when we retry. Reported-by:
Rich Ercolany <rercola@acm.jhu.edu> Signed-off-by:
-
Darren Hart authored
Correct various typos and formatting inconsistencies in the commentary of futex_wait_requeue_pi(). Signed-off-by:
-
Darren Hart authored
Make the existing function kernel-doc consistent throughout futex.c, following Documentation/kernel-doc-nano-howto.txt as closely as possible. When unsure, at least be consistent within futex.c. Signed-off-by:
-
Darren Hart authored
Use kernel-doc format to describe struct futex_q. Correct the wakeup definition to eliminate the statement about waking the waiter between the plist_del() and the q->lock_ptr = 0. Note in the comment that PI futexes have a different definition of the woken state. Signed-off-by:
-
Darren Hart authored
If userspace tries to perform a requeue_pi on a non-requeue_pi waiter, it will find the futex_q->requeue_pi_key to be NULL and OOPS. Check for NULL in match_futex() instead of doing explicit NULL pointer checks on all call sites. While match_futex(NULL, NULL) returning false is a little odd, it's still correct as we expect valid key references. Signed-off-by:
-
Thomas Gleixner authored
The requeue_pi path doesn't use unqueue_me() (and the racy lock_ptr == NULL test) nor does it use the wake_list of futex_wake() which where the reason for commit 41890f24 (futex: Handle spurious wake up) See debugging discussing on LKML Message-ID: <4AD4080C.20703@us.ibm.com> The changes in this fix to the wait_requeue_pi path were considered to be a likely unecessary, but harmless safety net. But it turns out that due to the fact that for unknown $@#!*( reasons EWOULDBLOCK is defined as EAGAIN we built an endless loop in the code path which returns correctly EWOULDBLOCK. Spurious wakeups in wait_requeue_pi code path are unlikely so we do the easy solution and return EWOULDBLOCK^WEAGAIN to user space and let it deal with the spurious wakeup. Cc: Darren Hart <dvhltc@us.ibm.com> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Eric Dumazet <eric.dumazet@gmail.com> Cc: John Stultz <johnstul@linux.vnet.ibm.com> Cc: Dinakar Guniguntala <dino@in.ibm.com> LKML-Reference: <4AE23C74.1090502@us.ibm.com> Cc: stable@kernel.org Signed-off-by:
-
Thomas Gleixner authored
There is currently no check to ensure that userspace uses the same futex requeue target (uaddr2) in futex_requeue() that the waiter used in futex_wait_requeue_pi(). A mismatch here could very unexpected results as the waiter assumes it either wakes on uaddr1 or uaddr2. We could detect this on wakeup in the waiter, but the cleanup is more intense after the improper requeue has occured. This patch stores the waiter's expected requeue target in a new requeue_pi_key pointer in the futex_q which futex_requeue() checks prior to attempting to do a proxy lock acquistion or a requeue when requeue_pi=1. If they don't match, return -EINVAL from futex_requeue, aborting the requeue of any remaining waiters. Signed-off-by:
-
- 20 Oct, 2009 2 commits
-
-
Thomas Gleixner authored
-
John Kacur authored
commit 3c96a2 (ipc: Make the ipc code -rt aware) introduced a imbalance of preempt_disable_rt vs. preempt_enable_nort. That results in preempt count leak. Make it symetric. Reported-by:
Joerg Abraham <Joerg.Abraham@alcatel-lucent.de> Signed-off-by:
John Kacur <jkacur@redhat.com> Signed-off-by:
-
- 13 Oct, 2009 6 commits
-
-
Thomas Gleixner authored
commit e31b7991 (x86: highmem: Remove leftover function prototypes) removed kmap_atomic_prot_pfn() which is not a leftover, but should have been left where it was. Restore it. Signed-off-by:
-
Peter Zijlstra authored
The numa aliens tear down is not covered by the per cpu locked changes which we did to slab. Fix that. Signed-off-by:
-
Thomas Gleixner authored
The futex code does not handle spurious wake up in futex_wait and futex_wait_requeue_pi. The code assumes that any wake up which was not caused by futex_wake / requeue or by a timeout was caused by a signal wake up and returns one of the syscall restart error codes. In case of a spurious wake up the signal delivery code which deals with the restart error codes is not invoked and we return that error code to user space. That causes applications which actually check the return codes to fail. Blaise reported that on preempt-rt a python test program run into a exception trap. -rt exposed that due to a built in spurious wake up accelerator :) Solve this by checking signal_pending(current) in the wake up path and handle the spurious wake up case w/o returning to user space. Reported-by:
Blaise Gassend <blaise@willowgarage.com> Debugged-by:
-
Thomas Gleixner authored
Remy Bohmer pointed out that we create the hrtimer softirq thread even when CONFIG_HIGH_RES_TIMERS is off. That results in a softirq-NULL name for the thread. The thread is needed on -rt Signed-off-by: -
Thomas Gleixner authored
This reverts commit d69c5d37. The softirq is necessary even in the CONFIG_HIGH_RES_TIMERS=n case. Signed-off-by:
-
Thomas Gleixner authored
This reverts commit 928686b7. The patch was an optimization of the old futex wake code where we woke the waiter and then set q->lock_ptr to NULL. When the waiter preempted the waker then we run into lock contention on q->lock_ptr aka. hb->lock. commit f1a11e (futex: remove the wait queue) changes the wakeup logic by setting q->lock_ptr to NULL _before_ waking the task. It keeps a reference on the task struct of the to be woken task to avoid an exit race. The combination of both patches resulted in different race on -RT: A is blocked on futex B calls futex_wake B sets q(A)->lock_ptr to NULL and puts A on the wake list B is preempted ... A wakes up (e.g. timer, signal) A detects q->lock_ptr = NULL and returns A waits on a different futex B is scheduled back in B wakes A A sees a spurious wake up Reported-by:
-
- 12 Oct, 2009 1 commit
-
-
Darren Hart authored
PI futexes do not use the same plist_node_empty() test for wakeup. It was possible for the waiter (in futex_wait_requeue_pi()) to set TASK_INTERRUPTIBLE after the waker assigned the rtmutex to the waiter. The waiter would then note the plist was not empty and call schedule(). The task would not be found by any subsequeuent futex wakeups, resulting in a userspace hang. By moving the setting of TASK_INTERRUPTIBLE to before the call to queue_me(), the race with the waker is eliminated. Since we no longer call get_user() from within queue_me(), there is no need to delay the setting of TASK_INTERRUPTIBLE until after the call to queue_me(). The FUTEX_LOCK_PI operation is not affected as futex_lock_pi() relies entirely on the rtmutex code to handle schedule() and wakeup. The requeue PI code is affected because the waiter starts as a non-PI waiter and is woken on a PI futex. Remove the crusty old comment about holding spinlocks() across get_user() as we no longer do that. Correct the locking statement with a description of why the test is performed. Signed-off-by:
-
- 09 Oct, 2009 1 commit
-
-
Thomas Gleixner authored
-RT replaces kmap_atomic* functions with macros, but we kept the function prototypes around. Remove them. Signed-off-by:
-
- 08 Oct, 2009 3 commits
-
-
Thomas Gleixner authored
exit_pi_state() is called from do_exit() but not from do_execve(). Move it to release_mm() so it gets called from do_execve() as well. Signed-off-by: -
Peter Zijlstra authored
The robust list pointers of user space held futexes are kept intact over an exec() call. When the exec'ed task exits exit_robust_list() is called with the stale pointer. The risk of corruption is minimal, but still it is incorrect to keep the pointers valid. Actually glibc should uninstall the robust list before calling exec() but we have to deal with it anyway. Nullify the pointers after [compat_]exit_robust_list() has been called. Reported-by:
Anirban Sinha <ani@anirban.org> Signed-off-by:
-
Eero Nurkkala authored
Commit f2e21c96 had unfortunate side effects with cpufreq governors on some systems. If the system did not switch into NOHZ mode ts->inidle is not set when tick_nohz_stop_sched_tick() is called from the idle routine. Therefor all subsequent calls from irq_exit() to tick_nohz_stop_sched_tick() fail to call tick_nohz_start_idle(). This results in bogus idle accounting information which is passed to cpufreq governors. Set the inidle flag unconditionally of the NOHZ active state to keep the idle time accounting correct in any case. [ tglx: Added comment and tweaked the changelog ] Reported-by:
Steven Noonan <steven@uplinklabs.net> Signed-off-by:
Eero Nurkkala <ext-eero.nurkkala@nokia.com> Cc: Rik van Riel <riel@redhat.com> Cc: Venkatesh Pallipadi <venkatesh.pallipadi@intel.com> Cc: Greg KH <greg@kroah.com> Cc: Steven Noonan <steven@uplinklabs.net> Cc: stable@kernel.org LKML-Reference: <1254907901.30157.93.camel@eenurkka-desktop> Signed-off-by:
-
- 07 Oct, 2009 1 commit
-
-
Darren Hart authored
If futex_wait_requeue_pi() wakes prior to requeue, we drop the reference to the source futex_key twice, once in handle_early_requeue_pi_wakeup() and once on our way out. Remove the drop from the handle_early_requeue_pi_wakeup() and keep the get/drops together in futex_wait_requeue_pi(). Reported-by:
Helge Bahmann <hcb@chaoticmind.net> Signed-off-by:
-
- 06 Oct, 2009 1 commit
-
-
Thomas Gleixner authored
Remy Bohmer pointed out that we create the hrtimer softirq thread even when CONFIG_HIGH_RES_TIMERS is off. That results in a softirq-NULL name for the thread. Skip the thread creation/wakeup/teardown when CONFIG_HIGH_RES_TIMERS=n Signed-off-by:
-
- 04 Oct, 2009 1 commit
-
-
Thomas Gleixner authored
commit 21ece08c (net: fix the xtables smp_processor_id assumptions for -rt) fixed only half of the problem. The filter functions might run in thread context and can be preempted and migrated on -RT. Signed-off-by:
-
