- 03 Apr, 2009 40 commits
-
-
David Howells authored
Add the API for a generic facility (FS-Cache) by which caches may declare them selves open for business, and may obtain work to be done from network filesystems. The header file is included by: #include <linux/fscache-cache.h> Documentation for the API is also added to: Documentation/filesystems/caching/backend-api.txt This API is not usable without the implementation of the utility functions which will be added in further patches. Signed-off-by:
David Howells <dhowells@redhat.com> Acked-by:
Steve Dickson <steved@redhat.com> Acked-by:
Trond Myklebust <Trond.Myklebust@netapp.com> Acked-by:
Al Viro <viro@zeniv.linux.org.uk> Tested-by:
Daire Byrne <Daire.Byrne@framestore.com>
-
David Howells authored
Add the API for a generic facility (FS-Cache) by which filesystems (such as AFS or NFS) may call on local caching capabilities without having to know anything about how the cache works, or even if there is a cache: +---------+ | | +--------------+ | NFS |--+ | | | | | +-->| CacheFS | +---------+ | +----------+ | | /dev/hda5 | | | | | +--------------+ +---------+ +-->| | | | | | |--+ | AFS |----->| FS-Cache | | | | |--+ +---------+ +-->| | | | | | | +--------------+ +---------+ | +----------+ | | | | | | +-->| CacheFiles | | ISOFS |--+ | /var/cache | | | +--------------+ +---------+ General documentation and documentation of the netfs specific API are provided in addition to the header files. As this patch stands, it is possible to build a filesystem against the facility and attempt to use it. All that will happen is that all requests will be immediately denied as if no cache is present. Further patches will implement the core of the facility. The facility will transfer requests from networking filesystems to appropriate caches if possible, or else gracefully deny them. If this facility is disabled in the kernel configuration, then all its operations will trivially reduce to nothing during compilation. WHY NOT I_MAPPING? ================== I have added my own API to implement caching rather than using i_mapping to do this for a number of reasons. These have been discussed a lot on the LKML and CacheFS mailing lists, but to summarise the basics: (1) Most filesystems don't do hole reportage. Holes in files are treated as blocks of zeros and can't be distinguished otherwise, making it difficult to distinguish blocks that have been read from the network and cached from those that haven't. (2) The backing inode must be fully populated before being exposed to userspace through the main inode because the VM/VFS goes directly to the backing inode and does not interrogate the front inode's VM ops. Therefore: (a) The backing inode must fit entirely within the cache. (b) All backed files currently open must fit entirely within the cache at the same time. (c) A working set of files in total larger than the cache may not be cached. (d) A file may not grow larger than the available space in the cache. (e) A file that's open and cached, and remotely grows larger than the cache is potentially stuffed. (3) Writes go to the backing filesystem, and can only be transferred to the network when the file is closed. (4) There's no record of what changes have been made, so the whole file must be written back. (5) The pages belong to the backing filesystem, and all metadata associated with that page are relevant only to the backing filesystem, and not anything stacked atop it. OVERVIEW ======== FS-Cache provides (or will provide) the following facilities: (1) Caches can be added / removed at any time, even whilst in use. (2) Adds a facility by which tags can be used to refer to caches, even if they're not available yet. (3) More than one cache can be used at once. Caches can be selected explicitly by use of tags. (4) The netfs is provided with an interface that allows either party to withdraw caching facilities from a file (required for (1)). (5) A netfs may annotate cache objects that belongs to it. This permits the storage of coherency maintenance data. (6) Cache objects will be pinnable and space reservations will be possible. (7) The interface to the netfs returns as few errors as possible, preferring rather to let the netfs remain oblivious. (8) Cookies are used to represent indices, files and other objects to the netfs. The simplest cookie is just a NULL pointer - indicating nothing cached there. (9) The netfs is allowed to propose - dynamically - any index hierarchy it desires, though it must be aware that the index search function is recursive, stack space is limited, and indices can only be children of indices. (10) Indices can be used to group files together to reduce key size and to make group invalidation easier. The use of indices may make lookup quicker, but that's cache dependent. (11) Data I/O is effectively done directly to and from the netfs's pages. The netfs indicates that page A is at index B of the data-file represented by cookie C, and that it should be read or written. The cache backend may or may not start I/O on that page, but if it does, a netfs callback will be invoked to indicate completion. The I/O may be either synchronous or asynchronous. (12) Cookies can be "retired" upon release. At this point FS-Cache will mark them as obsolete and the index hierarchy rooted at that point will get recycled. (13) The netfs provides a "match" function for index searches. In addition to saying whether a match was made or not, this can also specify that an entry should be updated or deleted. FS-Cache maintains a virtual index tree in which all indices, files, objects and pages are kept. Bits of this tree may actually reside in one or more caches. FSDEF | +------------------------------------+ | | NFS AFS | | +--------------------------+ +-----------+ | | | | homedir mirror afs.org redhat.com | | | +------------+ +---------------+ +----------+ | | | | | | 00001 00002 00007 00125 vol00001 vol00002 | | | | | +---+---+ +-----+ +---+ +------+------+ +-----+----+ | | | | | | | | | | | | | PG0 PG1 PG2 PG0 XATTR PG0 PG1 DIRENT DIRENT DIRENT R/W R/O Bak | | PG0 +-------+ | | 00001 00003 | +---+---+ | | | PG0 PG1 PG2 In the example above, two netfs's can be seen to be backed: NFS and AFS. These have different index hierarchies: (*) The NFS primary index will probably contain per-server indices. Each server index is indexed by NFS file handles to get data file objects. Each data file objects can have an array of pages, but may also have further child objects, such as extended attributes and directory entries. Extended attribute objects themselves have page-array contents. (*) The AFS primary index contains per-cell indices. Each cell index contains per-logical-volume indices. Each of volume index contains up to three indices for the read-write, read-only and backup mirrors of those volumes. Each of these contains vnode data file objects, each of which contains an array of pages. The very top index is the FS-Cache master index in which individual netfs's have entries. Any index object may reside in more than one cache, provided it only has index children. Any index with non-index object children will be assumed to only reside in one cache. The FS-Cache overview can be found in: Documentation/filesystems/caching/fscache.txt The netfs API to FS-Cache can be found in: Documentation/filesystems/caching/netfs-api.txt Signed-off-by: -
David Howells authored
Recruit a page flag to aid in cache management. The following extra flag is defined: (1) PG_fscache (PG_private_2) The marked page is backed by a local cache and is pinning resources in the cache driver. If PG_fscache is set, then things that checked for PG_private will now also check for that. This includes things like truncation and page invalidation. The function page_has_private() had been added to make the checks for both PG_private and PG_private_2 at the same time. Signed-off-by:Rik van Riel <riel@redhat.com> Acked-by:
-
David Howells authored
The attached patch causes read_cache_pages() to release page-private data on a page for which add_to_page_cache() fails. If the filler function fails, then the problematic page is left attached to the pagecache (with appropriate flags set, one presumes) and the remaining to-be-attached pages are invalidated and discarded. This permits pages with caching references associated with them to be cleaned up. The invalidatepage() address space op is called (indirectly) to do the honours. Signed-off-by:
-
David Howells authored
Document the slow work thread pool. Signed-off-by:
-
David Howells authored
Make the slow work pool configurable through /proc/sys/kernel/slow-work. (*) /proc/sys/kernel/slow-work/min-threads The minimum number of threads that should be in the pool as long as it is in use. This may be anywhere between 2 and max-threads. (*) /proc/sys/kernel/slow-work/max-threads The maximum number of threads that should in the pool. This may be anywhere between min-threads and 255 or NR_CPUS * 2, whichever is greater. (*) /proc/sys/kernel/slow-work/vslow-percentage The percentage of active threads in the pool that may be used to execute very slow work items. This may be between 1 and 99. The resultant number is bounded to between 1 and one fewer than the number of active threads. This ensures there is always at least one thread that can process very slow work items, and always at least one thread that won't. Signed-off-by:Serge Hallyn <serue@us.ibm.com> Acked-by:
-
David Howells authored
Make the slow-work thread pool actually dynamic in the number of threads it contains. With this patch, it will both create additional threads when it has extra work to do, and cull excess threads that aren't doing anything. Signed-off-by:
-
David Howells authored
Create a dynamically sized pool of threads for doing very slow work items, such as invoking mkdir() or rmdir() - things that may take a long time and may sleep, holding mutexes/semaphores and hogging a thread, and are thus unsuitable for workqueues. The number of threads is always at least a settable minimum, but more are started when there's more work to do, up to a limit. Because of the nature of the load, it's not suitable for a 1-thread-per-CPU type pool. A system with one CPU may well want several threads. This is used by FS-Cache to do slow caching operations in the background, such as looking up, creating or deleting cache objects. Signed-off-by:
-
git://git.kernel.org/pub/scm/linux/kernel/git/viro/vfs-2.6Linus Torvalds authored
* 'for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/viro/vfs-2.6: Remove two unneeded exports and make two symbols static in fs/mpage.c Cleanup after commit 585d3bc0 Trim includes of fdtable.h Don't crap into descriptor table in binfmt_som Trim includes in binfmt_elf Don't mess with descriptor table in load_elf_binary() Get rid of indirect include of fs_struct.h New helper - current_umask() check_unsafe_exec() doesn't care about signal handlers sharing New locking/refcounting for fs_struct Take fs_struct handling to new file (fs/fs_struct.c) Get rid of bumping fs_struct refcount in pivot_root(2) Kill unsharing fs_struct in __set_personality()
-
git://git.kernel.org/pub/scm/linux/kernel/git/airlied/drm-2.6Linus Torvalds authored
* 'drm-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/airlied/drm-2.6: (21 commits) drm/radeon: load the right microcode on rs780 drm: remove unused "can_grow" parameter from drm_crtc_helper_initial_config drm: fix EDID backward compat check drm: sync the mode validation for INTERLACE/DBLSCAN drm: fix typo in edid vendor parsing. DRM: drm_crtc_helper.h doesn't actually need i2c.h drm: fix missing inline function on 32-bit powerpc. drm: Use pgprot_writecombine in GEM GTT mapping to get the right bits for !PAT. drm/i915: Add a spinlock to protect the active_list drm/i915: Fix SDVO TV support drm/i915: Fix SDVO CREATE_PREFERRED_INPUT_TIMING command drm/i915: Fix error in SDVO DTD and modeline convert drm/i915: Fix SDVO command debug function drm/i915: fix TV mode setting in property change drm/i915: only set TV mode when any property changed drm/i915: clean up udelay usage drm/i915: add VGA hotplug support for 945+ drm/i915: correctly set IGD device's gtt size for KMS. drm/i915: avoid hanging on to a stale pointer to raw_edid. drm/i915: check for -EINVAL from vm_insert_pfn ...
-
git://git.kernel.org/pub/scm/linux/kernel/git/davem/net-2.6Linus Torvalds authored
* git://git.kernel.org/pub/scm/linux/kernel/git/davem/net-2.6: (54 commits) glge: remove unused #include <version.h> dnet: remove unused #include <version.h> tcp: miscounts due to tcp_fragment pcount reset tcp: add helper for counter tweaking due mid-wq change hso: fix for the 'invalid frame length' messages hso: fix for crash when unplugging the device fsl_pq_mdio: Fix compile failure fsl_pq_mdio: Revive UCC MDIO support ucc_geth: Pass proper device to DMA routines, otherwise oops happens i.MX31: Fixing cs89x0 network building to i.MX31ADS tc35815: Fix build error if NAPI enabled hso: add Vendor/Product ID's for new devices ucc_geth: Remove unused header gianfar: Remove unused header kaweth: Fix locking to be SMP-safe net: allow multiple dev per napi with GRO r8169: reset IntrStatus after chip reset ixgbe: Fix potential memory leak/driver panic issue while setting up Tx & Rx ring parameters ixgbe: fix ethtool -A|a behavior ixgbe: Patch to fix driver panic while freeing up tx & rx resources ...
-
Jack Steiner authored
Fix slab corruption caused by alloc_cpumask_var_node() overwriting the tail end of an off-stack cpumask. The function zeros out cpumask bits beyond the last possible cpu. The starting point for zeroing should be the beginning of the mask offset by a byte count derived from the number of possible cpus. The offset was calculated in bits instead of bytes. This resulted in overwriting the end of the cpumask. Signed-off-by:
Jack Steiner <steiner@sgi.com> Acked-by: Mike Travis <travis.sgi.com> Acked-by:
Ingo Molnar <mingo@elte.hu> Cc: Rusty Russell <rusty@rustcorp.com.au> Cc: Stephen Rothwell <sfr@canb.auug.org.au> Cc: <stable@kernel.org> [2.6.29.x] Signed-off-by:
Andrew Morton <akpm@linux-foundation.org> Signed-off-by:
Linus Torvalds <torvalds@linux-foundation.org>
-
Robin Holt authored
Implement __raw_read_lock_flags and __raw_write_lock_flags for the ia64 architecture. [kosaki.motohiro@jp.fujitsu.com: typo fix] Signed-off-by:
Petr Tesarik <ptesarik@suse.cz> Signed-off-by:
Robin Holt <holt@sgi.com> Cc: <linux-arch@vger.kernel.org> Cc: Ingo Molnar <mingo@elte.hu> Cc: Peter Zijlstra <a.p.zijlstra@chello.nl> Acked-by:
Tony Luck <tony.luck@intel.com> Signed-off-by:
KOSAKI Motohiro <kosaki.motohiro@jp.fujitsu.com> Signed-off-by:
-
Robin Holt authored
Pass the original flags to rwlock arch-code, so that it can re-enable interrupts if implemented for that architecture. Initially, make __raw_read_lock_flags and __raw_write_lock_flags stubs which just do the same thing as non-flags variants. Signed-off-by:
Peter Zijlstra <a.p.zijlstra@chello.nl> Cc: <linux-arch@vger.kernel.org> Acked-by:
-
Robin Holt authored
SGI has observed that on large systems, interrupts are not serviced for a long period of time when waiting for a rwlock. The following patch series re-enables irqs while waiting for the lock, resembling the code which is already there for spinlocks. I only made the ia64 version, because the patch adds some overhead to the fast path. I assume there is currently no demand to have this for other architectures, because the systems are not so large. Of course, the possibility to implement raw_{read|write}_lock_flags for any architecture is still there. This patch: The new macro LOCK_CONTENDED_FLAGS expands to the correct implementation depending on the config options, so that IRQ's are re-enabled when possible, but they remain disabled if CONFIG_LOCKDEP is set. Signed-off-by: -
Coly Li authored
Make ufs return f_fsid info for statfs(2). Signed-off-by:
Coly Li <coly.li@suse.de> Cc: Evgeniy Dushistov <dushistov@mail.ru> Signed-off-by:
-
Coly Li authored
Make sysv file system return f_fsid info for statfs(2). Signed-off-by:
-
Coly Li authored
Make squashfs return f_fsid info for statfs(2). Signed-off-by:
-
Coly Li authored
Make reiserfs3 return f_fsid info for statfs(2). By Andreas' suggestion, this patch populates a persistent f_fsid between boots/mounts with help of on-disk uuid record. Randy Dunlap reported a compiling error from v2 patch like: fs/built-in.o: In function `reiserfs_statfs': super.c:(.text+0x7332b): undefined reference to `crc32_le' super.c:(.text+0x7333f): undefined reference to `crc32_le' Also he provided helpful solution to fix this error. The modification of v3 patch is based on Randy's suggestion, add 'select CRC32' in fs/reiserfs/Kconfig. Signed-off-by: -
Coly Li authored
Make qnx4 file system return f_fsid info for statfs(2). Signed-off-by:
Anders Larsen <al@alarsen.net> Signed-off-by:
-
Coly Li authored
Make omfs return f_fsid info for statfs(2). Signed-off-by:
Bob Copeland <me@bobcopeland.com> Signed-off-by:
-
Coly Li authored
Make minix file system return f_fsid info for statfs(2). Signed-off-by:
-
Coly Li authored
Make isofs return f_fsid info for statfs(2). Signed-off-by:
-
Coly Li authored
Make hpfs return f_fsid info for statfs(2). Signed-off-by:
-
Coly Li authored
Make hfsplus return f_fsid info for statfs(2). Signed-off-by:
-
Coly Li authored
Make hfs return f_fsid info for statfs(2). Signed-off-by:
-
Coly Li authored
Make fat return f_fsid info for statfs(2). Signed-off-by:
OGAWA Hirofumi <hirofumi@mail.parknet.co.jp> Signed-off-by:
-
Coly Li authored
Make efs return f_fsid info for statfs(2), and do a little variable renaming in efs_statfs(). Signed-off-by:
-
Coly Li authored
Make cramfs return f_fsid info for statfs(2). Signed-off-by:
-
Coly Li authored
Make befs return f_fsid info for statfs(2). Signed-off-by:
-
Coly Li authored
Make affs return f_fsid info for statfs(2). Signed-off-by:
-
Coly Li authored
Currently many file systems in Linux kernel do not return f_fsid in statfs info, the value is set as 0 in vfs layer. Anyway, in some conditions, f_fsid from statfs(2) is useful, especially being used as (f_fsid, ino) pair to uniquely identify a file. Basic idea of the patches is generating a unique fs ID by huge_encode_dev(sb->s_bdev->bd_dev) during file system mounting life time (no endian consistent issue). sb is a point of struct super_block of current mounted file system being accessed by statfs(2). This patch: Make adfs return f_fsid info for statfs(2), and do a little variable renaming in adfs_statfs(). Signed-off-by:
-
Gerd Hoffmann authored
Signed-off-by:
Gerd Hoffmann <kraxel@redhat.com> Cc: Arnd Bergmann <arnd@arndb.de> Cc: Al Viro <viro@zeniv.linux.org.uk> Cc: <linux-api@vger.kernel.org> Cc: <linux-arch@vger.kernel.org> Cc: Ralf Baechle <ralf@linux-mips.org> Signed-off-by:
-
Ralf Baechle authored
Signed-off-by:
Ralf Baechle <ralf@linux-mips.org> Signed-off-by:
-
Gerd Hoffmann authored
This patch adds preadv and pwritev system calls. These syscalls are a pretty straightforward combination of pread and readv (same for write). They are quite useful for doing vectored I/O in threaded applications. Using lseek+readv instead opens race windows you'll have to plug with locking. Other systems have such system calls too, for example NetBSD, check here: http://www.daemon-systems.org/man/preadv.2.html The application-visible interface provided by glibc should look like this to be compatible to the existing implementations in the *BSD family: ssize_t preadv(int d, const struct iovec *iov, int iovcnt, off_t offset); ssize_t pwritev(int d, const struct iovec *iov, int iovcnt, off_t offset); This prototype has one problem though: On 32bit archs is the (64bit) offset argument unaligned, which the syscall ABI of several archs doesn't allow to do. At least s390 needs a wrapper in glibc to handle this. As we'll need a wrappers in glibc anyway I've decided to push problem to glibc entriely and use a syscall prototype which works without arch-specific wrappers inside the kernel: The offset argument is explicitly splitted into two 32bit values. The patch sports the actual system call implementation and the windup in the x86 system call tables. Other archs follow as separate patches. Signed-off-by:
-
Gerd Hoffmann authored
Factor out some code from compat_sys_writev() which can be shared with the upcoming compat_sys_pwritev(). Signed-off-by:
-
Gerd Hoffmann authored
This patch series: Implement the preadv() and pwritev() syscalls. *BSD has this syscall for quite some time. Test code: #if 0 set -x gcc -Wall -O2 -o preadv $0 exit 0 #endif /* * preadv demo / test * * (c) 2008 Gerd Hoffmann <kraxel@redhat.com> * * build with "sh $thisfile" */ #include <stdio.h> #include <stdlib.h> #include <unistd.h> #include <errno.h> #include <inttypes.h> #include <sys/uio.h> /* ----------------------------------------------------------------- */ /* syscall windup */ #include <sys/syscall.h> #if 0 /* WARNING: Be sure you know what you are doing if you enable this. * linux syscall code isn't upstream yet, syscall numbers are subject * to change */ # ifndef __NR_preadv # ifdef __i386__ # define __NR_preadv 333 # define __NR_pwritev 334 # endif # ifdef __x86_64__ # define __NR_preadv 295 # define __NR_pwritev 296 # endif # endif #endif #ifndef __NR_preadv # error preadv/pwritev syscall numbers are unknown #endif static ssize_t preadv(int fd, const struct iovec *iov, int iovcnt, off_t offset) { uint32_t pos_high = (offset >> 32) & 0xffffffff; uint32_t pos_low = offset & 0xffffffff; return syscall(__NR_preadv, fd, iov, iovcnt, pos_high, pos_low); } static ssize_t pwritev(int fd, const struct iovec *iov, int iovcnt, off_t offset) { uint32_t pos_high = (offset >> 32) & 0xffffffff; uint32_t pos_low = offset & 0xffffffff; return syscall(__NR_pwritev, fd, iov, iovcnt, pos_high, pos_low); } /* ----------------------------------------------------------------- */ /* demo/test app */ static char filename[] = "/tmp/preadv-XXXXXX"; static char outbuf[11] = "0123456789"; static char inbuf[11] = "----------"; static struct iovec ovec[2] = {{ .iov_base = outbuf + 5, .iov_len = 5, },{ .iov_base = outbuf + 0, .iov_len = 5, }}; static struct iovec ivec[3] = {{ .iov_base = inbuf + 6, .iov_len = 2, },{ .iov_base = inbuf + 4, .iov_len = 2, },{ .iov_base = inbuf + 2, .iov_len = 2, }}; void cleanup(void) { unlink(filename); } int main(int argc, char **argv) { int fd, rc; fd = mkstemp(filename); if (-1 == fd) { perror("mkstemp"); exit(1); } atexit(cleanup); /* write to file: "56789-01234" */ rc = pwritev(fd, ovec, 2, 0); if (rc < 0) { perror("pwritev"); exit(1); } /* read from file: "78-90-12" */ rc = preadv(fd, ivec, 3, 2); if (rc < 0) { perror("preadv"); exit(1); } printf("result : %s\n", inbuf); printf("expected: %s\n", "--129078--"); exit(0); } This patch: Factor out some code from compat_sys_readv() which can be shared with the upcoming compat_sys_preadv(). Signed-off-by: -
David VomLehn authored
Decompression errors can arise due to corruption of compressed blocks on flash or in memory. This patch propagates errors detected during decompression back to the block layer. Signed-off-by:
David VomLehn <dvomlehn@cisco.com> Signed-off-by:
-
Robin Holt authored
Fix a minor compile warning when building on ia64. drivers/misc/sgi-xp/xpc_sn2.c: In function `xpc_clear_remote_msgqueue_flags_sn2': drivers/misc/sgi-xp/xpc_sn2.c:1746: warning: comparison of distinct pointer types lacks a cast Signed-off-by:
-
Robin Holt authored
Some current configs turn on GRU for ia64. The GRU code does not correctly load on boot on ia64 (GRU does continue to work for x86-64), so changing the IA64 Kconfig to not select GRU on ia64 configs for now until we have time to fix. Signed-off-by:
-
