| commit | cf1b3fafd1f08bd3a18fc9af3c431c48f600eb32 | |
| author | Matthew Dillon <dillon@apollo.backplane.com> | |
| Sun, 19 Jun 2016 05:01:00 +0000 (18 22:01 -0700) | ||
| committer | Matthew Dillon <dillon@apollo.backplane.com> | |
| Sun, 19 Jun 2016 05:01:00 +0000 (18 22:01 -0700) | ||
| tree | f0eefa26c1f3b88dbe9ac423b2bd92084d9c5baf | treesnapshot (tar.gz zip) |
| parent | de240216d6f53717f64b182e6226f25a93e9f7b9 | commitdiff |
hammer2 - Start work on inode indexing - MAJOR CHANGE
A major change to how inodes work is required to allow NFS exports to be
supported and also to make mirroring operations optimal. Both needs are
met by indexing most inodes so they can be looked up by inode number. I've
tried to avoid having to do this for well over 2 years now but I finally
came to the conclusion that necessary features and efficiencies are
impossible without it.
+ For NFS exports we have to be able to lookup a file by inode number in
order to be able to translate NFS file handles.
+ Mirroring, Multi-Master, and other Multi-Node operations are extremely
inefficient when a large file or (even worse) some high-level directory
is renamed, because the synchronization code cannot be made aware of the
rename. The synchronizer winds up making a copy of the file or the
directory subhierarchy and that can be disaster if it winds up being
terrabytes.
To solve these problems we treat nearly ALL directory entries as hardlink
targets and place the hardlink target at the root of the PFS. This puts
nearly all inodes in a readily accessible place indexed by inode number.
This means we can now implement inode number lookups for NFS, and it also
means that the synchronizer does not have to copy anything when a file or
directory is renamed.
* Implement these changes by using an abbreviated version of the hardlink
target code that we already have. Get rid of the common-parent-directory
code (we will use the PFS iroot), and force almost everything to be a
hardlink.
* device nodes and softlinks are excepted. These work as they did before,
created in the directory entry itself and not as a hardlinked inode.
I might have to make these hardlink targets in the future too but at
the moment it looks avoidable.
* Unfortunately, this change creates a number of REGRESSIONS:
(1) There are now two inodes per file instead of one. The real inode
indexed in the PFS root, and the hardlink pointer in the directory
entry. This eats another 1KB.
(2) We lose a bunch of sequential layout optimizations (for the moment),
particularly when stat()ing directory entries.
(3) Inode creation and deletion ops will unfortunately cause more SMP
conflicts due to all the inodes being indexed in one place.
A major change to how inodes work is required to allow NFS exports to be
supported and also to make mirroring operations optimal. Both needs are
met by indexing most inodes so they can be looked up by inode number. I've
tried to avoid having to do this for well over 2 years now but I finally
came to the conclusion that necessary features and efficiencies are
impossible without it.
+ For NFS exports we have to be able to lookup a file by inode number in
order to be able to translate NFS file handles.
+ Mirroring, Multi-Master, and other Multi-Node operations are extremely
inefficient when a large file or (even worse) some high-level directory
is renamed, because the synchronization code cannot be made aware of the
rename. The synchronizer winds up making a copy of the file or the
directory subhierarchy and that can be disaster if it winds up being
terrabytes.
To solve these problems we treat nearly ALL directory entries as hardlink
targets and place the hardlink target at the root of the PFS. This puts
nearly all inodes in a readily accessible place indexed by inode number.
This means we can now implement inode number lookups for NFS, and it also
means that the synchronizer does not have to copy anything when a file or
directory is renamed.
* Implement these changes by using an abbreviated version of the hardlink
target code that we already have. Get rid of the common-parent-directory
code (we will use the PFS iroot), and force almost everything to be a
hardlink.
* device nodes and softlinks are excepted. These work as they did before,
created in the directory entry itself and not as a hardlinked inode.
I might have to make these hardlink targets in the future too but at
the moment it looks avoidable.
* Unfortunately, this change creates a number of REGRESSIONS:
(1) There are now two inodes per file instead of one. The real inode
indexed in the PFS root, and the hardlink pointer in the directory
entry. This eats another 1KB.
(2) We lose a bunch of sequential layout optimizations (for the moment),
particularly when stat()ing directory entries.
(3) Inode creation and deletion ops will unfortunately cause more SMP
conflicts due to all the inodes being indexed in one place.