- 14 Aug, 2009 1 commit
-
-
Balbir Singh authored
root cgroup. This is a part of the several patches to reduce mem cgroup overhead. I had posted other approaches earlier (including using percpu counters). Those patches will be a natural addition and will be added iteratively on top of these. The patch stops resource counter accounting for the root cgroup. The data for display is derived from the statisitcs we maintain via mem_cgroup_charge_statistics (which is more scalable). What happens today is that, we do double accounting, once using res_counter_charge() and once using memory_cgroup_charge_statistics(). For the root, since we don't implement limits any more, we don't need to track every charge via res_counter_charge() and check for limit being exceeded and reclaim. The main mem->res usage_in_bytes can be derived by summing the cache and rss usage data from memory statistics (MEM_CGROUP_STAT_RSS and MEM_CGROUP_STAT_CACHE). However, for memsw->res usage_in_bytes, we need additional data about swapped out memory. This patch adds a MEM_CGROUP_STAT_SWAPOUT and uses that along with MEM_CGROUP_STAT_RSS and MEM_CGROUP_STAT_CACHE to derive the memsw data. This data is computed recursively when hierarchy is enabled. The tests results I see on a 24 way show that 1. The lock contention disappears from /proc/lock_stats 2. The results of the test are comparable to running with cgroup_disable=memory. Here is a sample of my program runs Without Patch Performance counter stats for '/home/balbir/parallel_pagefault': 7192804.124144 task-clock-msecs # 23.937 CPUs 424691 context-switches # 0.000 M/sec 267 CPU-migrations # 0.000 M/sec 28498113 page-faults # 0.004 M/sec 5826093739340 cycles # 809.989 M/sec 408883496292 instructions # 0.070 IPC 7057079452 cache-references # 0.981 M/sec 3036086243 cache-misses # 0.422 M/sec 300.485365680 seconds time elapsed With cgroup_disable=memory Performance counter stats for '/home/balbir/parallel_pagefault': 7182183.546587 task-clock-msecs # 23.915 CPUs 425458 context-switches # 0.000 M/sec 203 CPU-migrations # 0.000 M/sec 92545093 page-faults # 0.013 M/sec 6034363609986 cycles # 840.185 M/sec 437204346785 instructions # 0.072 IPC 6636073192 cache-references # 0.924 M/sec 2358117732 cache-misses # 0.328 M/sec 300.320905827 seconds time elapsed With this patch applied Performance counter stats for '/home/balbir/parallel_pagefault': 7191619.223977 task-clock-msecs # 23.955 CPUs 422579 context-switches # 0.000 M/sec 88 CPU-migrations # 0.000 M/sec 91946060 page-faults # 0.013 M/sec 5957054385619 cycles # 828.333 M/sec 1058117350365 instructions # 0.178 IPC 9161776218 cache-references # 1.274 M/sec 1920494280 cache-misses # 0.267 M/sec 300.218764862 seconds time elapsed Data from Prarit (kernel compile with make -j64 on a 64 CPU/32G machine) For a single run Without patch real 27m8.988s user 87m24.916s sys 382m6.037s With patch real 4m18.607s user 84m58.943s sys 50m52.682s With config turned off real 4m54.972s user 90m13.456s sys 50m19.711s NOTE: The data looks counterintuitive due to the increased performance with the patch, even over the config being turned off. We probably need more runs, but so far all testing has shown that the patches definitely help. Signed-off-by:Balbir Singh <balbir@linux.vnet.ibm.com> Cc: Prarit Bhargava <prarit@redhat.com> Cc: Andi Kleen <andi@firstfloor.org> Reviewed-by:
KAMEZAWA Hiroyuki <kamezawa.hiroyu@jp.fujitsu.com> Reviewed-by:
Daisuke Nishimura <nishimura@mxp.nes.nec.co.jp> Cc: KOSAKI Motohiro <kosaki.motohiro@jp.fujitsu.com> Cc: Paul Menage <menage@google.com> Cc: Li Zefan <lizf@cn.fujitsu.com> Signed-off-by:
Andrew Morton <akpm@linux-foundation.org>
-
- 09 Sep, 2009 1 commit
-
-
Daisuke Nishimura authored
Signed-off-by:
-
- 05 Sep, 2009 1 commit
-
-
KAMEZAWA Hiroyuki authored
be visited if reclaimd==0. But css'refcnt handling for next_mz is not enough and it makes css->refcnt leak. Signed-off-by:
-
- 31 Jul, 2009 1 commit
-
-
Andrew Morton authored
Cc: Balbir Singh <balbir@linux.vnet.ibm.com> Cc: KAMEZAWA Hiroyuki <kamezawa.hiroyu@jp.fujitsu.com> Cc: KOSAKI Motohiro <kosaki.motohiro@jp.fujitsu.com> Cc: Li Zefan <lizf@cn.fujitsu.com> Signed-off-by:
-
- 24 Aug, 2009 2 commits
-
-
Balbir Singh authored
Permit reclaim from memory cgroups on contention (via the direct reclaim path). memory cgroup soft limit reclaim finds the group that exceeds its soft limit by the largest number of pages and reclaims pages from it and then reinserts the cgroup into its correct place in the rbtree. Add additional checks to mem_cgroup_hierarchical_reclaim() to detect long loops in case all swap is turned off. The code has been refactored and the loop check (loop < 2) has been enhanced for soft limits. For soft limits, we try to do more targetted reclaim. Instead of bailing out after two loops, the routine now reclaims memory proportional to the size by which the soft limit is exceeded. The proportion has been empirically determined. Signed-off-by:
KOSAKI Motohiro <kosaki.motohiro@jp.fujitsu.com> Signed-off-by:
-
Balbir Singh authored
Refactor the arguments passed to mem_cgroup_hierarchical_reclaim() into flags, so that new parameters don't have to be passed as we make the reclaim routine more flexible Signed-off-by:
-
- 04 Aug, 2009 1 commit
-
-
Hugh Dickins authored
Kernel panic - not syncing: No init found; after lots of scheduling while atomics, starting from when async_thread does sd_probe_async. mem_cgroup_soft_limit_check() was doing an unbalanced get_cpu(): don't get_cpu if we won't need it, and put_cpu if we did get_cpu. And fix the silliness of passing it an "over_soft_limit" argument that just tells it to return false when false. Signed-off-by:
Hugh Dickins <hugh.dickins@tiscali.co.uk> Cc: Li Zefan <lizf@cn.fujitsu.com> Cc: KOSAKI Motohiro <kosaki.motohiro@jp.fujitsu.com> Tested-by:
Jiri Slaby <jirislaby@gmail.com> Signed-off-by:
-
- 21 Jul, 2009 3 commits
-
-
Balbir Singh authored
Introduce an RB-Tree for storing memory cgroups that are over their soft limit. The overall goal is to 1. Add a memory cgroup to the RB-Tree when the soft limit is exceeded. We are careful about updates, updates take place only after a particular time interval has passed 2. We remove the node from the RB-Tree when the usage goes below the soft limit The next set of patches will exploit the RB-Tree to get the group that is over its soft limit by the largest amount and reclaim from it, when we face memory contention. Signed-off-by:
-
Balbir Singh authored
plus swap controller (memsw) is currently not supported. Resource counters have been enhanced to support soft limits and new type RES_SOFT_LIMIT has been added. Unlike hard limits, soft limits can be directly set and do not need any reclaim or checks before setting them to a newer value. Kamezawa-San raised a question as to whether soft limit should belong to res_counter. Since all resources understand the basic concepts of hard and soft limits, it is justified to add soft limits here. Soft limits are a generic resource usage feature, even file system quotas support soft limits. Signed-off-by:
-
Balbir Singh authored
similar has existed in the group scheduler in the form of shares. The CPU controllers interpretation of shares is very different though. Soft limits are the most useful feature to have for environments where the administrator wants to overcommit the system, such that only on memory contention do the limits become active. The current soft limits implementation provides a soft_limit_in_bytes interface for the memory controller and not for memory+swap controller. The implementation maintains an RB-Tree of groups that exceed their soft limit and starts reclaiming from the group that exceeds this limit by the maximum amount. This patch: Add documentation for soft limits Signed-off-by:
-
- 26 Jun, 2009 2 commits
-
-
Andrew Morton authored
#22: FILE: mm/memcontrol.c:1138: + ^I * We access a page_cgroup asynchronously without lock_page_cgroup().$ ERROR: code indent should use tabs where possible #23: FILE: mm/memcontrol.c:1139: + ^I * Especially when a page_cgroup is taken from a page, pc->mem_cgroup$ ERROR: code indent should use tabs where possible #24: FILE: mm/memcontrol.c:1140: + ^I * is accessed after testing USED bit. To make pc->mem_cgroup visible$ ERROR: code indent should use tabs where possible #25: FILE: mm/memcontrol.c:1141: + ^I * before USED bit, we need memory barrier here.$ ERROR: code indent should use tabs where possible #26: FILE: mm/memcontrol.c:1142: + ^I * See mem_cgroup_add_lru_list(), etc.$ ERROR: code indent should use tabs where possible #27: FILE: mm/memcontrol.c:1143: + ^I */$ total: 6 errors, 0 warnings, 13 lines checked ./patches/memcg-add-comments-explaining-memory-barriers.patch has style problems, please review. If any of these errors are false positives report them to the maintainer, see CHECKPATCH in MAINTAINERS. Please run checkpatch prior to sending patches Cc: KAMEZAWA Hiroyuki <kamezawa.hiroyu@jp.fujitsu.com> Signed-off-by: -
KAMEZAWA Hiroyuki authored
Cc: Daisuke Nishimura <nishimura@mxp.nes.nec.co.jp> Cc: Balbir Singh <balbir@linux.vnet.ibm.com> Signed-off-by:
-
- 29 Jun, 2009 1 commit
-
-
Balbir Singh authored
The PageAcctLRU bit itself does not mean a lot without checking if the mem cgroup is the same as the root cgroup. This patch fixes a left over from the previous versions. Reported-by:
-
- 23 Jun, 2009 1 commit
-
-
Andrew Morton authored
Cc: Balbir Singh <balbir@linux.vnet.ibm.com> Cc: Daisuke Nishimura <nishimura@mxp.nes.nec.co.jp> Cc: KAMEZAWA Hiroyuki <kamezawa.hiroyu@jp.fujitsu.com> Cc: Li Zefan <lizf@cn.fujitsu.com> Cc: Paul Menage <menage@google.com> Signed-off-by:
-
- 24 Aug, 2009 1 commit
-
-
Balbir Singh authored
all pages in the root cgroup. As a side-effect, we can no longer set a memory hard limit in the root cgroup. A new flag to track whether the page has been accounted or not has been added as well. Flags are now set atomically for page_cgroup, pcg_default_flags is now obsolete and removed. Signed-off-by:
-
- 20 Aug, 2009 2 commits
-
-
Ben Blum authored
at once to a cgroup by writing the tgid to the 'cgroup.procs' file. This current implementation makes use of a per-threadgroup rwsem that's taken for reading in the fork() path to prevent newly forking threads within the threadgroup from "escaping" while the move is in progress. Cgroups subsystems that need to perform per-thread actions in their "attach" callback are (currently) responsible for doing their own synchronization, since this occurs outside of the critical section that locks against cloning within a thread group. Signed-off-by:
Ben Blum <bblum@google.com> Signed-off-by:
Paul Menage <menage@google.com> Acked-by:
Li Zefan <lizf@cn.fujitsu.com> Cc: Matt Helsley <matthltc@us.ibm.com> Cc: "Eric W. Biederman" <ebiederm@xmission.com> Cc: Oleg Nesterov <oleg@redhat.com> Cc: "Paul E. McKenney" <paulmck@us.ibm.com> Signed-off-by:
-
Andrew Morton authored
Cc: Ben Blum <bblum@google.com> Cc: Li Zefan <lizf@cn.fujitsu.com> Cc: Matt Helsley <matthltc@us.ibm.com> Cc: Oleg Nesterov <oleg@redhat.com> Cc: Paul Menage <menage@google.com> Signed-off-by:
-
- 24 Aug, 2009 3 commits
-
-
Ben Blum authored
sighand's atomic count, to piggyback on its cacheline), and two functions in kernel/cgroup.c (for now) for easily+safely obtaining and releasing it. If another part of the kernel later wants to use such a locking mechanism, the CONFIG_CGROUPS ifdefs should be changed to a higher-up flag that CGROUPS and the other system would both depend on, and the lock/unlock functions could be moved to sched.c or so. Signed-off-by:
-
Dave Young authored
kernel/sched.c:10536: error: 'cpu_cgroup_attach' undeclared here (not in a function) Cc: Ben Blum <bblum@google.com> Cc: Paul Menage <menage@google.com> Cc: Li Zefan <lizf@cn.fujitsu.com> Cc: Matt Helsley <matthltc@us.ibm.com> Cc: "Eric W. Biederman" <ebiederm@xmission.com> Cc: Oleg Nesterov <oleg@redhat.com> Cc: Peter Zijlstra <a.p.zijlstra@chello.nl> Cc: Ingo Molnar <mingo@elte.hu> Signed-off-by: -
Ben Blum authored
a whole threadgroup at a time, for use in cgroup_attach_proc. (This is a pre-patch to cgroup-procs-writable.patch.) Currently, new mode of the attach function can only tell the subsystem about the old cgroup of the threadgroup leader. No subsystem currently needs that information for each thread that's being moved, but if one were to be added (for example, one that counts tasks within a group) this bit would need to be reworked a bit to tell the subsystem the right information. Signed-off-by:
Matt Helsley <matthltc@us.ibm.com> Cc: "Eric W. Biederman" <ebiederm@xmission.com> Cc: Oleg Nesterov <oleg@redhat.com> Cc: Peter Zijlstra <a.p.zijlstra@chello.nl> Cc: Ingo Molnar <mingo@elte.hu> Signed-off-by:
-
- 20 Aug, 2009 5 commits
-
-
Ben Blum authored
This is a prepatch for making the procs file writable. In order to free the old css_sets for each task to be moved as they're being moved, the freeing mechanism must be RCU-protected, or else we would have to have a call to synchronize_rcu() for each task before freeing its old css_set. Signed-off-by:
-
Ben Blum authored
judges based on the requested size whether or not the array needs to be vmalloced or can be gotten via kmalloc, and similar for kfree/vfree. Signed-off-by:
-
Ben Blum authored
namespaces reading the tasks or procs file could result in one process seeing results from the other's namespace. Rather than one pidlist for each file in a cgroup, we now keep a list of pidlists keyed by namespace and file type (tasks versus procs) in which entries are placed on demand. Each pidlist has its own lock, and that the pidlists themselves are passed around in the seq_file's private pointer means we don't have to touch the cgroup or its master list except when creating and destroying entries. Signed-off-by:
-
Ben Blum authored
pidlist for the tasks file. Those are now separated into a new struct cgroup_pidlist, of which two are had, one for procs and one for tasks. The way the seq_file operations are set up is changed so that just the pidlist struct gets passed around as the private data. Interface example: Suppose a multithreaded process has pid 1000 and other threads with ids 1001, 1002, 1003: $ cat tasks 1000 1001 1002 1003 $ cat cgroup.procs 1000 $ Signed-off-by:
-
Paul Menage authored
reports unique tgids rather than pids, and allows all threads in a threadgroup to be atomically moved to a new cgroup. The subsystem "attach" interface is modified to support attaching whole threadgroups at a time, which could introduce potential problems if any subsystem were to need to access the old cgroup of every thread being moved. The attach interface may need to be revised if this becomes the case. Also added is functionality for read/write locking all CLONE_THREAD fork()ing within a threadgroup, by means of an rwsem that lives in the sighand_struct, for per-threadgroup-ness and also for sharing a cacheline with the sighand's atomic count. This scheme should introduce no extra overhead in the fork path when there's no contention. The final patch reveals potential for a race when forking before a subsystem's attach function is called - one potential solution in case any subsystem has this problem is to hang on to the group's fork mutex through the attach() calls, though no subsystem yet demonstrates need for an extended critical section. This patch: Revert commit 096b7fe0 Author: Li Zefan <lizf@cn.fujitsu.com> AuthorDate: Wed Jul 29 15:04:04 2009 -0700 Commit: Linus Torvalds <torvalds@linux-foundation.org> CommitDate: Wed Jul 29 19:10:35 2009 -0700 cgroups: fix pid namespace bug This is in preparation for some clashing cgroups changes that subsume the original commit's functionaliy. The original commit fixed a pid namespace bug which Ben Blum fixed independently (in the same way, but with different code) as part of a series of patches. I played around with trying to reconcile Ben's patch series with Li's patch, but concluded that it was simpler to just revert Li's, given that Ben's patch series contained essentially the same fix. Signed-off-by:
-
- 30 Jul, 2009 4 commits
-
-
Paul Menage authored
least one bound subsystem. The mount option "none" is treated as an explicit request for no bound subsystems. A hierarchy with no subsystems can be useful for plain task tracking, and is also a step towards the support for multiply-bindable subsystems. As part of this change, the hierarchy id is no longer calculated from the bitmask of subsystems in the hierarchy (since this is not guaranteed to be unique) but is allocated via an ida. Reference counts on cgroups from css_set objects are now taken explicitly one per hierarchy, rather than one per subsystem. Example usage: mount -t cgroup -o none,name=foo cgroup /mnt/cgroup Based on the "no-op"/"none" subsystem concept proposed by kamezawa.hiroyu@jp.fujitsu.com Signed-off-by:
-
Paul Menage authored
pointers in a struct css_set uniquely identify the hierarchy->cgroup mappings associated with the css_set; and there's no way to directly identify the associated set of cgroups other than by indirecting through the appropriate subsystem state pointers. This patch removes the need for that assumption by adding a back-pointer from struct cg_cgroup_link object to its associated cgroup; this allows the set of cgroups to be determined by traversing the cg_links list in the struct css_set. Signed-off-by:
-
Paul Menage authored
completely independent of the cgroups framework, it limits its usefulness for debugging the contents of internal data structures. Move the debug subsystem code into the scope of all the cgroups data structures to make more detailed debugging possible. Signed-off-by:
-
Paul Menage authored
allow disambiguation in the presence of empty hierarchies and multiply-bindable subsystems this patch adds support for naming a new cgroup hierarchy via the "name=" mount option A pre-existing hierarchy may be specified by either name or by subsystems; a hierarchy's name cannot be changed by a remount operation. Example usage: # To create a hierarchy called "foo" containing the "cpu" subsystem mount -t cgroup -oname=foo,cpu cgroup /mnt/cgroup1 # To mount the "foo" hierarchy on a second location mount -t cgroup -oname=foo cgroup /mnt/cgroup2 Signed-off-by:
-
- 24 Jul, 2009 1 commit
-
-
Xiaotian Feng authored
Acked-by:
Xiaotian Feng <dfeng@redhat.com> Signed-off-by:
-
- 13 Sep, 2009 1 commit
-
-
Randy Dunlap authored
Signed-off-by:
Randy Dunlap <randy.dunlap@oracle.com> Reviewed-by:
Jesper Juhl <jj@chaosbits.net> Signed-off-by:
-
- 09 Sep, 2009 2 commits
-
-
Andrew Morton authored
#98: FILE: Documentation/vm/page-types.c:193: + __max1 > __max2 ? __max1: __max2; }) ^ total: 1 errors, 0 warnings, 309 lines checked ./patches/page-types-add-feature-for-walking-process-address-space.patch has style problems, please review. If any of these errors are false positives report them to the maintainer, see CHECKPATCH in MAINTAINERS. Please run checkpatch prior to sending patches Cc: Wu Fengguang <fengguang.wu@intel.com> Signed-off-by: -
Wu Fengguang authored
default action is to walk raw memory PFNs. Both the virtual address and physhcal address of each present pages will be listed: # ./tools/vm/page-types -lp $$ | head -3 voffset offset len flags 400 11bebe 1 __RU_lA____M______________________ 402 11bebc 1 __RU_lA____M______________________ Note that voffset/offset/len are now showed as hex numbers. Cc: Andi Kleen <andi@firstfloor.org> Signed-off-by:
Wu Fengguang <fengguang.wu@intel.com> Signed-off-by:
-
- 18 Aug, 2009 1 commit
-
-
Josh Triplett authored
Cc: Wu Fengguang <fengguang.wu@intel.com> Signed-off-by:
-
- 14 Aug, 2009 1 commit
-
-
Jaswinder Singh Rajput authored
Documentation/auxdisplay/cfag12864b-example.c: string.h is included more than once. Signed-off-by:
Jaswinder Singh Rajput <jaswinderrajput@gmail.com> Acked-by:
-
- 04 Aug, 2009 1 commit
-
-
Xiaotian Feng authored
fs.txt and Documentation/filesystem/proc.txt, but stale file-nr definition is remained. This patch add back the right fs-nr definition for 2.6 kernel. Signed-off-by: Xiaotian Feng<dfeng@redhat.com> Cc: Randy Dunlap <randy.dunlap@oracle.com> Signed-off-by:
-
- 31 Jul, 2009 3 commits
-
-
Peng Tao authored
mount command in util-linux package is well aware of shared subtree features now. The patch also fixes two typos in sharedsubtree.txt. Signed-off-by:
Peng Tao <bergwolf@gmail.com> Signed-off-by:
-
Randy Dunlap authored
from Documentation/filesystems/sharedsubtree.txt. Fix annoying "Lets" -> "Let's". Insert space between '#' prompt and "mount" command. Signed-off-by:
Miklos Szeredi <miklos@szeredi.hu> Cc: Christoph Hellwig <hch@infradead.org> Signed-off-by:
-
Zhaolei authored
broken-down time. time_to_tm() is more generic to do that. Signed-off-by:
Zhao Lei <zhaolei@cn.fujitsu.com> Cc: OGAWA Hirofumi <hirofumi@mail.parknet.co.jp> Cc: Thomas Gleixner <tglx@linutronix.de> Cc: Ingo Molnar <mingo@elte.hu> Cc: Pavel Machek <pavel@ucw.cz> Cc: Andi Kleen <andi@firstfloor.org> Signed-off-by:
-
- 24 Aug, 2009 1 commit
-
-
Zhaolei authored
calendar time and broken-down time. Here is some source I found: fs/ncpfs/dir.c fs/smbfs/proc.c fs/fat/misc.c fs/udf/udftime.c fs/cifs/netmisc.c net/netfilter/xt_time.c drivers/scsi/ips.c drivers/input/misc/hp_sdc_rtc.c drivers/rtc/rtc-lib.c arch/ia64/hp/sim/boot/fw-emu.c arch/m68k/mac/misc.c arch/powerpc/kernel/time.c arch/parisc/include/asm/rtc.h ... We can make a common function for this type of conversoion, At least we can get following benefit: 1: Make kernel simple and unify 2: Easy to fix bug in converting code 3: Reduce clone of code in future For example, I'm trying to make ftrace display walltime, this patch will make me easy. This code is based on code from glibc-2.6 Signed-off-by:
-
