aboutsummaryrefslogtreecommitdiff
path: root/fs/xfs/xfs_mount.h
AgeCommit message (Collapse)Author
2007-07-14[XFS] Concurrent Multi-File Data StreamsDavid Chinner
In media spaces, video is often stored in a frame-per-file format. When dealing with uncompressed realtime HD video streams in this format, it is crucial that files do not get fragmented and that multiple files a placed contiguously on disk. When multiple streams are being ingested and played out at the same time, it is critical that the filesystem does not cross the streams and interleave them together as this creates seek and readahead cache miss latency and prevents both ingest and playout from meeting frame rate targets. This patch set creates a "stream of files" concept into the allocator to place all the data from a single stream contiguously on disk so that RAID array readahead can be used effectively. Each additional stream gets placed in different allocation groups within the filesystem, thereby ensuring that we don't cross any streams. When an AG fills up, we select a new AG for the stream that is not in use. The core of the functionality is the stream tracking - each inode that we create in a directory needs to be associated with the directories' stream. Hence every time we create a file, we look up the directories' stream object and associate the new file with that object. Once we have a stream object for a file, we use the AG that the stream object point to for allocations. If we can't allocate in that AG (e.g. it is full) we move the entire stream to another AG. Other inodes in the same stream are moved to the new AG on their next allocation (i.e. lazy update). Stream objects are kept in a cache and hold a reference on the inode. Hence the inode cannot be reclaimed while there is an outstanding stream reference. This means that on unlink we need to remove the stream association and we also need to flush all the associations on certain events that want to reclaim all unreferenced inodes (e.g. filesystem freeze). SGI-PV: 964469 SGI-Modid: xfs-linux-melb:xfs-kern:29096a Signed-off-by: David Chinner <dgc@sgi.com> Signed-off-by: Barry Naujok <bnaujok@sgi.com> Signed-off-by: Donald Douwsma <donaldd@sgi.com> Signed-off-by: Christoph Hellwig <hch@infradead.org> Signed-off-by: Tim Shimmin <tes@sgi.com> Signed-off-by: Vlad Apostolov <vapo@sgi.com>
2007-07-14[XFS] Lazy Superblock CountersDavid Chinner
When we have a couple of hundred transactions on the fly at once, they all typically modify the on disk superblock in some way. create/unclink/mkdir/rmdir modify inode counts, allocation/freeing modify free block counts. When these counts are modified in a transaction, they must eventually lock the superblock buffer and apply the mods. The buffer then remains locked until the transaction is committed into the incore log buffer. The result of this is that with enough transactions on the fly the incore superblock buffer becomes a bottleneck. The result of contention on the incore superblock buffer is that transaction rates fall - the more pressure that is put on the superblock buffer, the slower things go. The key to removing the contention is to not require the superblock fields in question to be locked. We do that by not marking the superblock dirty in the transaction. IOWs, we modify the incore superblock but do not modify the cached superblock buffer. In short, we do not log superblock modifications to critical fields in the superblock on every transaction. In fact we only do it just before we write the superblock to disk every sync period or just before unmount. This creates an interesting problem - if we don't log or write out the fields in every transaction, then how do the values get recovered after a crash? the answer is simple - we keep enough duplicate, logged information in other structures that we can reconstruct the correct count after log recovery has been performed. It is the AGF and AGI structures that contain the duplicate information; after recovery, we walk every AGI and AGF and sum their individual counters to get the correct value, and we do a transaction into the log to correct them. An optimisation of this is that if we have a clean unmount record, we know the value in the superblock is correct, so we can avoid the summation walk under normal conditions and so mount/recovery times do not change under normal operation. One wrinkle that was discovered during development was that the blocks used in the freespace btrees are never accounted for in the AGF counters. This was once a valid optimisation to make; when the filesystem is full, the free space btrees are empty and consume no space. Hence when it matters, the "accounting" is correct. But that means the when we do the AGF summations, we would not have a correct count and xfs_check would complain. Hence a new counter was added to track the number of blocks used by the free space btrees. This is an *on-disk format change*. As a result of this, lazy superblock counters are a mkfs option and at the moment on linux there is no way to convert an old filesystem. This is possible - xfs_db can be used to twiddle the right bits and then xfs_repair will do the format conversion for you. Similarly, you can convert backwards as well. At some point we'll add functionality to xfs_admin to do the bit twiddling easily.... SGI-PV: 964999 SGI-Modid: xfs-linux-melb:xfs-kern:28652a Signed-off-by: David Chinner <dgc@sgi.com> Signed-off-by: Christoph Hellwig <hch@infradead.org> Signed-off-by: Tim Shimmin <tes@sgi.com>
2007-07-14[XFS] Don't grow filesystems past the size they can index.Nathan Scott
When growing a filesystem we don't check to see if the new size overflows the page cache index range, so we can do silly things like grow a filesystem page 16TB on a 32bit. Check new filesystem sizes against the limits the kernel can support. SGI-PV: 957886 SGI-Modid: xfs-linux-melb:xfs-kern:28563a Signed-Off-By: Nathan Scott <nscott@aconex.com> Signed-off-by: David Chinner <dgc@sgi.com> Signed-off-by: Tim Shimmin <tes@sgi.com>
2007-02-10[XFS] Re-initialize the per-cpu superblock counters after recovery.Lachlan McIlroy
After filesystem recovery the superblock is re-read to bring in any changes. If the per-cpu superblock counters are not re-initialized from the superblock then the next time the per-cpu counters are disabled they might overwrite the global counter with a bogus value. SGI-PV: 957348 SGI-Modid: xfs-linux-melb:xfs-kern:27999a Signed-off-by: Lachlan McIlroy <lachlan@sgi.com> Signed-off-by: David Chinner <dgc@sgi.com> Signed-off-by: Tim Shimmin <tes@sgi.com>
2007-02-10[XFS] Fix block reservation changes for non-SMP systems.Kevin Jamieson
SGI-PV: 956323 SGI-Modid: xfs-linux-melb:xfs-kern:27940a Signed-off-by: Kevin Jamieson <kjamieson@bycast.com> Signed-off-by: David Chatterton <chatz@sgi.com> Signed-off-by: David Chinner <dgc@sgi.com> Signed-off-by: Tim Shimmin <tes@sgi.com>
2007-02-10[XFS] Fix block reservation mechanism.David Chinner
The block reservation mechanism has been broken since the per-cpu superblock counters were introduced. Make the block reservation code work with the per-cpu counters by syncing the counters, snapshotting the amount of available space and then doing a modifcation of the counter state according to the result. Continue in a loop until we either have no space available or we reserve some space. SGI-PV: 956323 SGI-Modid: xfs-linux-melb:xfs-kern:27895a Signed-off-by: David Chinner <dgc@sgi.com> Signed-off-by: Christoph Hellwig <hch@infradead.org> Signed-off-by: Tim Shimmin <tes@sgi.com>
2007-02-10[XFS] Make growfs work for amounts greater than 2TBDavid Chinner
The free block modification code has a 32bit interface, limiting the size the filesystem can be grown even on 64 bit machines. On 32 bit machines, there are other 32bit variables in transaction structures and interfaces that need to be expanded to allow this to work. SGI-PV: 959978 SGI-Modid: xfs-linux-melb:xfs-kern:27894a Signed-off-by: David Chinner <dgc@sgi.com> Signed-off-by: Christoph Hellwig <hch@infradead.org> Signed-off-by: Tim Shimmin <tes@sgi.com>
2007-02-10[XFS] remove unused xflags parameter from sync routinesLachlan McIlroy
SGI-PV: 959137 SGI-Modid: xfs-linux-melb:xfs-kern:27710a Signed-off-by: Lachlan McIlroy <lachlan@sgi.com> Signed-off-by: Eric Sandeen <sandeen@sandeen.net> Signed-off-by: Tim Shimmin <tes@sgi.com>
2007-02-10[XFS] Fix UP build breakage due to undefined m_icsb_mutex.David Chinner
SGI-PV: 952227 SGI-Modid: xfs-linux-melb:xfs-kern:27692a Signed-off-by: David Chinner <dgc@sgi.com> Signed-off-by: Lachlan McIlroy <lachlan@sgi.com> Signed-off-by: Tim Shimmin <tes@sgi.com>
2007-02-10[XFS] Reduction global superblock lock contention near ENOSPC.David Chinner
The existing per-cpu superblock counter code uses the global superblock spin lock when we approach ENOSPC for global synchronisation. On larger machines than this code was originally tested on this can still get catastrophic spinlock contention due increasing rebalance frequency near ENOSPC. By introducing a sleeping lock that is used to serialise balances and modifications near ENOSPC we prevent contention from needlessly from wasting the CPU time of potentially hundreds of CPUs. To reduce the number of balances occuring, we separate the need rebalance case from the slow allocate case. Now, a counter running dry will trigger a rebalance during which counters are disabled. Any thread that sees a disabled counter enters a different path where it waits on the new mutex. When it gets the new mutex, it checks if the counter is disabled. If the counter is disabled, then we _know_ that we have to use the global counter and lock and it is safe to do so immediately. Otherwise, we drop the mutex and go back to trying the per-cpu counters which we know were re-enabled. SGI-PV: 952227 SGI-Modid: xfs-linux-melb:xfs-kern:27612a Signed-off-by: David Chinner <dgc@sgi.com> Signed-off-by: Lachlan McIlroy <lachlan@sgi.com> Signed-off-by: Tim Shimmin <tes@sgi.com>
2006-09-28[XFS] Fix kmem_zalloc_greedy warnings on 64 bit platforms.Nathan Scott
SGI-PV: 955302 SGI-Modid: xfs-linux-melb:xfs-kern:26907a Signed-off-by: Nathan Scott <nathans@sgi.com> Signed-off-by: Tim Shimmin <tes@sgi.com>
2006-09-28[XFS] remove bhv_lookup, _range version works aswell and has more usefulChristoph Hellwig
semantics. SGI-PV: 954580 SGI-Modid: xfs-linux-melb:xfs-kern:26563a Signed-off-by: Christoph Hellwig <hch@lst.de> Signed-off-by: Nathan Scott <nathans@sgi.com> Signed-off-by: Tim Shimmin <tes@sgi.com>
2006-06-20[XFS] Remove version 1 directory code. Never functioned on Linux, justNathan Scott
pure bloat. SGI-PV: 952969 SGI-Modid: xfs-linux-melb:xfs-kern:26251a Signed-off-by: Nathan Scott <nathans@sgi.com>
2006-06-09[XFS] Resolve a namespace collision on remaining vtypes for FreeBSDNathan Scott
porters. SGI-PV: 953338 SGI-Modid: xfs-linux-melb:xfs-kern:26108a Signed-off-by: Nathan Scott <nathans@sgi.com>
2006-06-09[XFS] Resolve a namespace collision on vnode/vnodeops for FreeBSD porters.Nathan Scott
SGI-PV: 953338 SGI-Modid: xfs-linux-melb:xfs-kern:26107a Signed-off-by: Nathan Scott <nathans@sgi.com>
2006-06-09[XFS] Resolve a namespace collision on vfs/vfsops for FreeBSD porters.Nathan Scott
SGI-PV: 9533338 SGI-Modid: xfs-linux-melb:xfs-kern:26106a Signed-off-by: Nathan Scott <nathans@sgi.com>
2006-06-09[XFS] Shutdown the filesystem if all device paths have gone. MadeNathan Scott
shutdown vop flags consistent with sync vop flags declarations too. SGI-PV: 939911 SGI-Modid: xfs-linux-melb:xfs-kern:26096a Signed-off-by: Nathan Scott <nathans@sgi.com>
2006-06-09[XFS] Add parameters to xfs_bmapi() and xfs_bunmapi() to have them reportOlaf Weber
the range spanned by modifications to the in-core extent map. Add XFS_BUNMAPI() and XFS_SWAP_EXTENTS() macros that call xfs_bunmapi() and xfs_swap_extents() via the ioops vector. Change all calls that may modify the in-core extent map for the data fork to go through the ioops vector. This allows a cache of extent map data to be kept in sync. SGI-PV: 947615 SGI-Modid: xfs-linux-melb:xfs-kern:209226a Signed-off-by: Olaf Weber <olaf@sgi.com> Signed-off-by: Nathan Scott <nathans@sgi.com>
2006-03-31[XFS] Implement the silent parameter to fill_super, previously ignored.Nathan Scott
SGI-PV: 951299 SGI-Modid: xfs-linux-melb:xfs-kern:25632a Signed-off-by: Nathan Scott <nathans@sgi.com>
2006-03-29[XFS] We really suck at spulling. Thanks to Chris Pascoe for fixing allNathan Scott
these typos. SGI-PV: 904196 SGI-Modid: xfs-linux-melb:xfs-kern:25539a Signed-off-by: Nathan Scott <nathans@sgi.com>
2006-03-14[XFS] Remove a couple of no-longer-used macros/types from XFS.Nathan Scott
SGI-PV: 950556 SGI-Modid: xfs-linux-melb:xfs-kern:25377a Signed-off-by: Nathan Scott <nathans@sgi.com>
2006-03-14[XFS] using a spinlock per cpu for superblock counter exclusion results inDavid Chinner
a preēmpt counter overflow at 256p and above. Change the exclusion mechanism to use atomic bit operations and busy wait loops to emulate the spin lock exclusion mechanism but without the preempt count issues. SGI-PV: 950027 SGI-Modid: xfs-linux-melb:xfs-kern:25338a Signed-off-by: David Chinner <dgc@sgi.com> Signed-off-by: Nathan Scott <nathans@sgi.com>
2006-03-14[XFS] Add support for hotplug CPUs to the per-CPU superblock counters byDavid Chinner
registering a notifier callback that listens to CPU up/down events to modify the counters appropriately. SGI-PV: 949726 SGI-Modid: xfs-linux-melb:xfs-kern:25214a Signed-off-by: David Chinner <dgc@sgi.com> Signed-off-by: Nathan Scott <nathans@sgi.com>
2006-03-14[XFS] On machines with more than 8 cpus, when running parallel I/ODavid Chinner
threads, the incore superblock lock becomes the limiting factor for buffered write throughput. Make the contended fields in the incore superblock use per-cpu counters so that there is no global lock to limit scalability. SGI-PV: 946630 SGI-Modid: xfs-linux-melb:xfs-kern:25106a Signed-off-by: David Chinner <dgc@sgi.com> Signed-off-by: Nathan Scott <nathans@sgi.com>
2006-03-14[XFS] XFS propagates MS_NOATIME through two levels internally but doesn'tNathan Scott
actually use it. Kill this dead code. Signed-off-by: Christoph Hellwig <hch@lst.de> SGI-PV: 904196 SGI-Modid: xfs-linux-melb:xfs-kern:25086a Signed-off-by: Nathan Scott <nathans@sgi.com>
2006-01-11[XFS] Complete the pagebuf -> xfs_buf naming convention transition,Nathan Scott
finally. SGI-PV: 947038 SGI-Modid: xfs-linux-melb:xfs-kern:24866a Signed-off-by: Nathan Scott <nathans@sgi.com>
2006-01-11[XFS] Reverse the sense of COMPAT_ATTR and ATTR2, keeps it simple andNathan Scott
consistent. SGI-PV: 941645 SGI-Modid: xfs-linux-melb:xfs-kern:202961a Signed-off-by: Nathan Scott <nathans@sgi.com>
2006-01-09[PATCH] mutex subsystem, semaphore to mutex: XFSJes Sorensen
This patch switches XFS over to use the new mutex code directly as opposed to the previous workaround patch I posted earlier that avoided the namespace clash by forcing it back to semaphores. This falls in the 'works for me<tm>' category. Signed-off-by: Jes Sorensen <jes@trained-monkey.org> Signed-off-by: Ingo Molnar <mingo@elte.hu>
2005-11-02[XFS] Rework fid encode/decode wrt 64 bit inums interacting with NFS.Nathan Scott
SGI-PV: 937127 SGI-Modid: xfs-linux:xfs-kern:24201a Signed-off-by: Nathan Scott <nathans@sgi.com>
2005-11-02[XFS] Rework the final mount options flag bit to make room for more.Nathan Scott
SGI-PV: 943866 SGI-Modid: xfs-linux:xfs-kern:24030a Signed-off-by: Nathan Scott <nathans@sgi.com>
2005-11-02[XFS] Update license/copyright notices to match the prefered SGINathan Scott
boilerplate. SGI-PV: 913862 SGI-Modid: xfs-linux:xfs-kern:23903a Signed-off-by: Nathan Scott <nathans@sgi.com>
2005-11-02[XFS] Remove xfs_macros.c, xfs_macros.h, rework headers a whole lot.Nathan Scott
SGI-PV: 943122 SGI-Modid: xfs-linux:xfs-kern:23901a Signed-off-by: Nathan Scott <nathans@sgi.com>
2005-11-02[XFS] Track external log/realtime device names for correct reporting inNathan Scott
/proc/mounts. SGI-PV: 942984 SGI-Modid: xfs-linux:xfs-kern:23862a Signed-off-by: Nathan Scott <nathans@sgi.com>
2005-11-02[XFS] Ondisk format extension for extended attributes (attr2). Basically,Nathan Scott
the data/attr forks now grow up/down from either end of the literal area, rather than dividing the literal area into two chunks and growing both upward. Means we can now make much more efficient use of the attribute space, incl. fitting DMF attributes inline in 256 byte inodes, and large jumps in dbench3 performance numbers. It is self enabling, but can be forced on/off via the attr2/noattr2 mount options. SGI-PV: 941645 SGI-Modid: xfs-linux:xfs-kern:23837a Signed-off-by: Nathan Scott <nathans@sgi.com>
2005-11-02[XFS] Introduce two new mount options (nolargeio/largeio) to allowDavid Chinner
filesystems to expose the filesystem stripe width in stat(2) rather than the page cache size. This allows applications requiring high bandwidth to easily determine the optimum I/O size for the underlying filesystem. The default is to report the page cache size (i.e. "nolargeio"). SGI-PV: 942818 SGI-Modid: xfs-linux:xfs-kern:23830a Signed-off-by: David Chinner <dgc@sgi.com> Signed-off-by: Nathan Scott <nathans@sgi.com>
2005-11-02[XFS] Provide a mechiansm for flushing delalloc before quota reporting.Nathan Scott
SGI-PV: 942815 SGI-Modid: xfs-linux:xfs-kern:23829a Signed-off-by: Nathan Scott <nathans@sgi.com>
2005-11-02[XFS] write barrier support Issue all log sync operations as orderedChristoph Hellwig
writes. In addition flush the disk cache on fsync if the sync cached operation didn't sync the log to disk (this requires some additional bookeping in the transaction and log code). If the device doesn't claim to support barriers, the filesystem has an extern log volume or the trial superblock write with barriers enabled failed we disable barriers and print a warning. We should probably fail the mount completely, but that could lead to nasty boot failures for the root filesystem. Not enabled by default yet, needs more destructive testing first. SGI-PV: 912426 SGI-Modid: xfs-linux:xfs-kern:198723a Signed-off-by: Christoph Hellwig <hch@sgi.com> Signed-off-by: Nathan Scott <nathans@sgi.com>
2005-06-21[XFS] Add support for project quota, based on Dan Knappes earlier work.Nathan Scott
SGI-PV: 932952 SGI-Modid: xfs-linux:xfs-kern:22805a Signed-off-by: Nathan Scott <nathans@sgi.com>
2005-06-21[XFS] mark various symbols static Patch from Adrian BunkChristoph Hellwig
SGI-PV: 936255 SGI-Modid: xfs-linux:xfs-kern:192760a Signed-off-by: Christoph Hellwig <hch@sgi.com> Signed-off-by: Nathan Scott <nathans@sgi.com>
2005-05-05[XFS] Cleanup use of loff_t vs xfs_off_t in the core code.Nathan Scott
SGI Modid: xfs-linux:xfs-kern:22378a Signed-off-by: Nathan Scott <nathans@sgi.com> Signed-off-by: Christoph Hellwig <hch@sgi.com>
2005-05-05[XFS] Use the right offset when ensuring a delayed allocate conversion has ↵Nathan Scott
covered the offset originally requested. Can cause data corruption when multiple processes are performing writeout on different areas of the same file. Quite difficult to hit though. SGI Modid: xfs-linux:xfs-kern:22377a Signed-off-by: Nathan Scott <nathans@sgi.com> Signed-off-by: Christoph Hellwig <hch@sgi.com> .
2005-05-05[XFS] Allow initial XFS delayed allocation size to be increased beyond 64KB.Nathan Scott
SGI Modid: xfs-linux:xfs-kern:22261a Signed-off-by: Nathan Scott <nathans@sgi.com> Signed-off-by: Christoph Hellwig <hch@sgi.com>
2005-04-16Linux-2.6.12-rc2Linus Torvalds
Initial git repository build. I'm not bothering with the full history, even though we have it. We can create a separate "historical" git archive of that later if we want to, and in the meantime it's about 3.2GB when imported into git - space that would just make the early git days unnecessarily complicated, when we don't have a lot of good infrastructure for it. Let it rip!