- 29 Jul, 2009 40 commits
-
-
Thomas Gleixner authored
KVM expects the notifier call with irqs enabled. It's necessary due to a possible IPI call. Make the preempt-rt version behave the same way as mainline. Signed-off-by:Thomas Gleixner <tgxl@linutronix.de>
-
Gregory Haskins authored
We will use this later in the series to eliminate the need for a function call. [ Steven Rostedt: added task_is_current function ] Signed-off-by:
Gregory Haskins <ghaskins@novell.com> Signed-off-by:
Ingo Molnar <mingo@elte.hu> Signed-off-by:
Thomas Gleixner <tglx@linutronix.de>
-
Ingo Molnar authored
Signed-off-by:
-
Thomas Gleixner authored
Signed-off-by:
-
Ingo Molnar authored
Here we are in the CPU_DEAD notifier, and we must not sleep nor enable interrupts. Signed-off-by:
-
Ingo Molnar authored
Signed-off-by:
-
Ingo Molnar authored
Signed-off-by:
-
Ingo Molnar authored
Signed-off-by:
-
Thomas Gleixner authored
Idle task boosting is a nono in general. There is one exception, when NOHZ is active: The idle task calls get_next_timer_interrupt() and holds the timer wheel base->lock on the CPU and another CPU wants to access the timer (probably to cancel it). We can safely ignore the boosting request, as the idle CPU runs this code with interrupts disabled and will complete the lock protected section without being interrupted. So there is no real need to boost. Signed-off-by:
-
Ingo Molnar authored
Signed-off-by:
-
Ingo Molnar authored
Signed-off-by:
-
Ingo Molnar authored
Signed-off-by:
-
Ingo Molnar authored
Signed-off-by:
-
Ingo Molnar authored
Signed-off-by:
-
Ingo Molnar authored
Signed-off-by: -
Ingo Molnar authored
Signed-off-by:
-
Steven Rostedt authored
Argh, cut and paste wasn't enough... Use this patch instead. It needs an irq disable. But, believe it or not, on SMP this is actually better. If the irq is shared (as it is in Mark's case), we don't stop the irq of other devices from being handled on another CPU (unfortunately for Mark, he pinned all interrupts to one CPU). Signed-off-by:
Steven Rostedt <rostedt@goodmis.org> Signed-off-by:
-
Thomas Gleixner authored
Waking the thread even when no timers are scheduled is useless. Signed-off-by: -
Arnaldo Carvalho de Melo authored
Shorten the softirq kernel thread names because they always overflow the limited comm length, appearing as "posix_cpu_timer" CPU# times. Done on 2.6.24.7, but probably applicable to later kernels. Signed-off-by:
Arnaldo Carvalho de Melo <acme@redhat.com> Signed-off-by:
-
John Stultz authored
posix-cpu-timer code takes non -rt safe locks in hard irq context. Move it to a thread. Signed-off-by:
John Stultz <johnstul@us.ibm.com> Signed-off-by:
-
Thomas Gleixner authored
add RT stats to /proc/stat Signed-off-by:
-
Ingo Molnar authored
Creates long latencies for no value Signed-off-by:
-
Ingo Molnar authored
Signed-off-by:
-
Ingo Molnar authored
Signed-off-by:
-
Ingo Molnar authored
Signed-off-by:
-
Ingo Molnar authored
Allows that code to be preemtible Signed-off-by:
-
Thomas Gleixner authored
Add the missing function Signed-off-by: -
Steven Rostedt authored
The current code of rt_downgrade_write simply does a BUG(). There are places in the kernel that uses this code, and will crash a runnning preempt-rt kernel. The rt_downgrade_write converts a rwsem held for write into a rwsem held for read without ever releasing the semaphore. In -rt, the rwsems are simply a mutex. There is nothing different between a rwsem held for write, and one held for read. The difference is that one held for read can nest. This patch changes the BUG_ON() to simply BUG if the caller is not the owner of the semaphore. This patch comes from my rt-git repo, and has been tested there. Signed-off-by:
-
Thomas Gleixner authored
recursive rwlocks are only allowed for recursive reads. recursive rwsems are not allowed at all. Follow up to Jan Blunks fix. Signed-off-by: -
Jan Blunck authored
This patch removes the stupid "Read locks within the self-held write lock succeed" behaviour. This is breaking in mm_take_all_locks() since it is quite common to ensure that a lock is taken with BUG_ON(down_read_trylock(&mm->mmap_sem)). Signed-off-by:
Jan Blunck <jblunck@suse.de> Signed-off-by:
-
Thomas Gleixner authored
Map spinlocks, rwlocks, rw_semaphores and semaphores to the rt_mutex based locking functions for preempt-rt. Signed-off-by: -
john stultz authored
So if I enable CONFIG_DEBUG_RT_MUTEXES with 2.6.24.7-rt14, I tend to quickly see a number of BUG warnings when running Java tests: BUG: jxeinajar/3383: lock count underflow! Pid: 3383, comm: jxeinajar Not tainted 2.6.24-ibmrt2.5john #3 Call Trace: [<ffffffff8107208d>] rt_mutex_deadlock_account_unlock+0x5d/0x70 [<ffffffff817d6aa5>] rt_read_slowunlock+0x35/0x550 [<ffffffff8107173d>] rt_mutex_up_read+0x3d/0xc0 [<ffffffff81072a99>] rt_up_read+0x29/0x30 [<ffffffff8106e34e>] do_futex+0x32e/0xd40 [<ffffffff8107173d>] ? rt_mutex_up_read+0x3d/0xc0 [<ffffffff81072a99>] ? rt_up_read+0x29/0x30 [<ffffffff8106f370>] compat_sys_futex+0xa0/0x110 [<ffffffff81010a36>] ? syscall_trace_enter+0x86/0xb0 [<ffffffff8102ff04>] cstar_do_call+0x1b/0x65 INFO: lockdep is turned off. --------------------------- | preempt count: 00000001 ] | 1-level deep critical section nesting: ---------------------------------------- ... [<ffffffff817d8e42>] .... __spin_lock_irqsave+0x22/0x60 ......[<ffffffff817d6a93>] .. ( <= rt_read_slowunlock+0x23/0x550) After some debugging and with Steven's help, we realized that with rwlocks, rt_mutex_deadlock_account_lock can be called multiple times in parallel (where as in most cases the mutex must be held by the caller to to call the function). This can cause integer lock_count value being used to be non-atomically incremented. The following patch converts lock_count to a atomic_t and resolves the warnings. Signed-off-by:
-
Thomas Gleixner authored
The sleeping locks implementation based on rtmutexes can miss wakeups for two reasons: 1) The unconditional usage TASK_UNINTERRUPTIBLE for the blocking state Results in missed wakeups from wake_up_interruptible*() state = TASK_INTERRUPTIBLE; blocks_on_lock() state = TASK_UNINTERRUPTIBLE; schedule(); .... acquires_lock(); restore_state(); Until the waiter has restored its state wake_up_interruptible*() will fail. 2) The rtmutex wakeup intermediate state TASK_RUNNING_MUTEX Results in missed wakeups from wake_up*() waiter is woken by mutex wakeup waiter->state = TASK_RUNNING_MUTEX; .... acquires_lock(); restore_state(); Until the waiter has restored its state wake_up*() will fail. Solution: Instead of setting the state to TASK_RUNNING_MUTEX in the mutex wakeup case we logically OR TASK_RUNNING_MUTEX to the current waiter state. This keeps the original bits (TASK_INTERRUPTIBLE / TASK_UNINTERRUPTIBLE) intact and lets wakeups succeed. When a task blocks on a lock in state TASK_INTERRUPTIBLE and is woken up by a real wakeup, then we store the state = TASK_RUNNING for the restore and can safely use TASK_UNINTERRUPTIBLE from that point to avoid further wakeups which just let us loop in the lock code. This also removes the extra TASK_RUNNING_MUTEX flags from the wakeup_process*() functions as they are not longer necessary. Signed-off-by: -
Thomas Gleixner authored
The adaptive spin patches introduced an overdesigned optimization for the adaptive path. This avoidance of those code pathes is not worth the extra conditionals and furthermore it is inconsistent in itself. Remove it and use the same mechanism as the other lock pathes. That way we have a consistent state manipulation scheme and less extra cases. Signed-off-by:
-
Thomas Gleixner authored
The manipulation of the waiter task state is copied all over the place with slightly different details. Use one set of functions to reduce duplicated code and make the handling consistent for all instances. Signed-off-by:
-
Thomas Gleixner authored
Signed-off-by:
-
Steven Rostedt authored
Lock stealing and non cmpxchg will always go into the slow path. This patch detects the fact that we didn't go through the work of blocking and will exit early. Signed-off-by:
Steven Rostedt <srostedt@redhat.com> Signed-off-by:
-
Gregory Haskins authored
[ The following text is in the "utf-8" character set. ] [ Your display is set for the "iso-8859-1" character set. ] [ Some characters may be displayed incorrectly. ] From: Peter W. Morreale <pmorreale@novell.com> Remove the redundant attempt to get the lock. While it is true that the exit path with this patch adds an un-necessary xchg (in the event the lock is granted without further traversal in the loop) experimentation shows that we almost never encounter this situation. Signed-off-by:Peter W. Morreale <pmorreale@novell.com> Signed-off-by:
-
Gregory Haskins authored
[ The following text is in the "utf-8" character set. ] [ Your display is set for the "iso-8859-1" character set. ] [ Some characters may be displayed incorrectly. ] From: Peter W.Morreale <pmorreale@novell.com> In wakeup_next_waiter(), we take the pi_lock, and then find out whether we have another waiter to add to the pending owner. We can reduce contention on the pi_lock for the pending owner if we first obtain the pointer to the next waiter outside of the pi_lock. Signed-off-by: -
Gregory Haskins authored
[ The following text is in the "utf-8" character set. ] [ Your display is set for the "iso-8859-1" character set. ] [ Some characters may be displayed incorrectly. ] It is redundant to wake the grantee task if it is already running, and the call to wake_up_process is relatively expensive. If we can safely skip it we can measurably improve the performance of the adaptive-locks. Credit goes to Peter Morreale for the general idea. Signed-off-by:
-
