• Mel Gorman's avatar
    page-allocator: change migratetype for all pageblocks within a high-order page... · 2f66a68f
    Mel Gorman authored
    page-allocator: change migratetype for all pageblocks within a high-order page during __rmqueue_fallback
    
    When there are no pages of a target migratetype free, the page allocator
    selects a high-order block of another migratetype to allocate from.  When
    the order of the page taken is greater than pageblock_order, all
    pageblocks within that high-order page should change migratetype so that
    pages are later freed to the correct free-lists.
    
    The current behaviour is that pageblocks change migratetype if the order
    being split matches the pageblock_order.  When pageblock_order <
    MAX_ORDER-1, ownership is not changing correct and pages are being later
    freed to the incorrect list and this impacts fragmentation avoidance.
    
    This patch changes all pageblocks within the high-order page being split
    to the correct migratetype.  Without the patch, allocation success rates
    for hugepages under stress were about 59% of physical memory on x86-64.
    With the patch applied, this goes up to 65%.
    Signed-off-by: default avatarMel Gorman <mel@csn.ul.ie>
    Cc: Andy Whitcroft <apw@shadowen.org>
    Cc: KOSAKI Motohiro <kosaki.motohiro@jp.fujitsu.com>
    Cc: KAMEZAWA Hiroyuki <kamezawa.hiroyu@jp.fujitsu.com>
    Signed-off-by: default avatarAndrew Morton <akpm@linux-foundation.org>
    Signed-off-by: default avatarLinus Torvalds <torvalds@linux-foundation.org>
    2f66a68f
page_alloc.c 137 KB