• Lee Schermerhorn's avatar
    Fix NUMA Memory Policy Reference Counting · 480eccf9
    Lee Schermerhorn authored
    This patch proposes fixes to the reference counting of memory policy in the
    page allocation paths and in show_numa_map().  Extracted from my "Memory
    Policy Cleanups and Enhancements" series as stand-alone.
    
    Shared policy lookup [shmem] has always added a reference to the policy,
    but this was never unrefed after page allocation or after formatting the
    numa map data.
    
    Default system policy should not require additional ref counting, nor
    should the current task's task policy.  However, show_numa_map() calls
    get_vma_policy() to examine what may be [likely is] another task's policy.
    The latter case needs protection against freeing of the policy.
    
    This patch adds a reference count to a mempolicy returned by
    get_vma_policy() when the policy is a vma policy or another task's
    mempolicy.  Again, shared policy is already reference counted on lookup.  A
    matching "unref" [__mpol_free()] is performed in alloc_page_vma() for
    shared and vma policies, and in show_numa_map() for shared and another
    task's mempolicy.  We can call __mpol_free() directly, saving an admittedly
    inexpensive inline NULL test, because we know we have a non-NULL policy.
    
    Handling policy ref counts for hugepages is a bit trickier.
    huge_zonelist() returns a zone list that might come from a shared or vma
    'BIND policy.  In this case, we should hold the reference until after the
    huge page allocation in dequeue_hugepage().  The patch modifies
    huge_zonelist() to return a pointer to the mempolicy if it needs to be
    unref'd after allocation.
    
    Kernel Build [16cpu, 32GB, ia64] - average of 10 runs:
    
    		w/o patch	w/ refcount patch
    	    Avg	  Std Devn	   Avg	  Std Devn
    Real:	 100.59	    0.38	 100.63	    0.43
    User:	1209.60	    0.37	1209.91	    0.31
    System:   81.52	    0.42	  81.64	    0.34
    Signed-off-by: default avatarLee Schermerhorn <lee.schermerhorn@hp.com>
    Acked-by: default avatarAndi Kleen <ak@suse.de>
    Cc: Christoph Lameter <clameter@sgi.com>
    Acked-by: default avatarMel Gorman <mel@csn.ul.ie>
    Signed-off-by: default avatarAndrew Morton <akpm@linux-foundation.org>
    Signed-off-by: default avatarLinus Torvalds <torvalds@linux-foundation.org>
    480eccf9
hugetlb.c 21.4 KB