• Davide Libenzi's avatar
    epoll keyed wakeups: add __wake_up_locked_key() and __wake_up_sync_key() · 4ede816a
    Davide Libenzi authored
    This patchset introduces wakeup hints for some of the most popular (from
    epoll POV) devices, so that epoll code can avoid spurious wakeups on its
    waiters.
    
    The problem with epoll is that the callback-based wakeups do not, ATM,
    carry any information about the events the wakeup is related to.  So the
    only choice epoll has (not being able to call f_op->poll() from inside the
    callback), is to add the file* to a ready-list and resolve the real events
    later on, at epoll_wait() (or its own f_op->poll()) time.  This can cause
    spurious wakeups, since the wake_up() itself might be for an event the
    caller is not interested into.
    
    The rate of these spurious wakeup can be pretty high in case of many
    network sockets being monitored.
    
    By allowing devices to report the events the wakeups refer to (at least
    the two major classes - POLLIN/POLLOUT), we are able to spare useless
    wakeups by proper handling inside the epoll's poll callback.
    
    Epoll will have in any case to call f_op->poll() on the file* later on,
    since the change to be done in order to have the full event set sent via
    wakeup, is too invasive for the way our f_op->poll() system works (the
    full event set is calculated inside the poll function - there are too many
    of them to even start thinking the change - also poll/select would need
    change too).
    
    Epoll is changed in a way that both devices which send event hints, and
    the ones that don't, are correctly handled.  The former will gain some
    efficiency though.
    
    As a general rule for devices, would be to add an event mask by using
    key-aware wakeup macros, when making up poll wait queues.  I tested it
    (together with the epoll's poll fix patch Andrew has in -mm) and wakeups
    for the supported devices are correctly filtered.
    
    Test program available here:
    
    http://www.xmailserver.org/epoll_test.c
    
    This patch:
    
    Nothing revolutionary here.  Just using the available "key" that our
    wakeup core already support.  The __wake_up_locked_key() was no brainer,
    since both __wake_up_locked() and __wake_up_locked_key() are thin wrappers
    around __wake_up_common().
    
    The __wake_up_sync() function had a body, so the choice was between
    borrowing the body for __wake_up_sync_key() and calling it from
    __wake_up_sync(), or make an inline and calling it from both.  I chose the
    former since in most archs it all resolves to "mov $0, REG; jmp ADDR".
    Signed-off-by: default avatarDavide Libenzi <davidel@xmailserver.org>
    Cc: Alan Cox <alan@lxorguk.ukuu.org.uk>
    Cc: Ingo Molnar <mingo@elte.hu>
    Cc: David Miller <davem@davemloft.net>
    Cc: William Lee Irwin III <wli@movementarian.org>
    Signed-off-by: default avatarAndrew Morton <akpm@linux-foundation.org>
    Signed-off-by: default avatarLinus Torvalds <torvalds@linux-foundation.org>
    4ede816a
sched.c 246 KB