• Simon Derr's avatar
    [PATCH] __vm_enough_memory() signedness fix · 2f60f8d3
    Simon Derr authored
    We have found what seems to be a small bug in __vm_enough_memory() when
    sysctl_overcommit_memory is set to OVERCOMMIT_NEVER.
    
    When this bug occurs the systems fails to boot, with /sbin/init whining
    about fork() returning ENOMEM.
    
    We hunted down the problem to this:
    
    The deferred update mecanism used in vm_acct_memory(), on a SMP system,
    allows the vm_committed_space counter to have a negative value.
    
    This should not be a problem since this counter is known to be inaccurate.
    
    But in __vm_enough_memory() this counter is compared to the `allowed'
    variable, which is an unsigned long.  This comparison is broken since it
    will consider the negative values of vm_committed_space to be huge positive
    values, resulting in a memory allocation failure.
    
    Signed-off-by: <Jean-Marc.Saffroy@ext.bull.net>
    Signed-off-by: <Simon.Derr@bull.net>
    Signed-off-by: default avatarAndrew Morton <akpm@osdl.org>
    Signed-off-by: default avatarLinus Torvalds <torvalds@osdl.org>
    2f60f8d3
nommu.c 27.7 KB