• Peter Zijlstra's avatar
    generic-ipi: remove kmalloc() · 8969a5ed
    Peter Zijlstra authored
    Remove the use of kmalloc() from the smp_call_function_*()
    calls.
    
    Steven's generic-ipi patch (d7240b98: generic-ipi: use per cpu
    data for single cpu ipi calls) started the discussion on the use
    of kmalloc() in this code and fixed the
    smp_call_function_single(.wait=0) fallback case.
    
    In this patch we complete this by also providing means for the
    _many() call, which fully removes the need for kmalloc() in this
    code.
    
    The problem with the _many() call is that other cpus might still
    be observing our entry when we're done with it. It solved this
    by dynamically allocating data elements and RCU-freeing it.
    
    We solve it by using a single per-cpu entry which provides
    static storage and solves one half of the problem (avoiding
    referencing freed data).
    
    The other half, ensuring the queue iteration it still possible,
    is done by placing re-used entries at the head of the list. This
    means that if someone was still iterating that entry when it got
    moved, he will now re-visit the entries on the list he had
    already seen, but avoids skipping over entries like would have
    happened had we placed the new entry at the end.
    
    Furthermore, visiting entries twice is not a problem, since we
    remove our cpu from the entry's cpumask once its called.
    
    Many thanks to Oleg for his suggestions and him poking holes in
    my earlier attempts.
    Signed-off-by: default avatarPeter Zijlstra <a.p.zijlstra@chello.nl>
    Cc: Oleg Nesterov <oleg@redhat.com>
    Cc: Linus Torvalds <torvalds@linux-foundation.org>
    Cc: Nick Piggin <npiggin@suse.de>
    Cc: Jens Axboe <jens.axboe@oracle.com>
    Cc: "Paul E. McKenney" <paulmck@linux.vnet.ibm.com>
    Cc: Rusty Russell <rusty@rustcorp.com.au>
    Signed-off-by: default avatarIngo Molnar <mingo@elte.hu>
    8969a5ed
smp.c 12.9 KB