1 This is arla.info, produced by makeinfo version 4.0 from arla.texi.
5 * Arla: (arla). Arla - A Free AFS Implementation
9 File: arla.info, Node: Top, Prev: (dir), Up: (dir)
14 Arla is a free AFS implementation from KTH.
16 Please send comments (and fixes) on this manual and the arla programs to
17 <arla-drinkers@stacken.kth.se>.
21 * Introduction:: Introduction to Arla.
22 * AFS infrastructure:: A description of the afs infrastructure.
23 * Organization of data:: How different data is organized in AFS.
24 * AFS and the real world:: Common problems and their solutions.
25 * Parts of Arla:: Description of different parts of arla.
26 * Debugging:: How to debug arla when its not working.
27 * Porting:: That you need to know to port arla.
28 * Programming:: Programming documentation of arla.
29 * Oddities:: Strange things that did happen to us.
30 * Themis:: Tool to keep your files system up-to-date.
31 * Arla timeline:: Short timeline of arla.
32 * Authors:: The authors of arla.
33 * Acknowledgments:: People that have helped us.
37 --- The Detailed Node Listing ---
40 * AFS infrastructure::
42 How data and servers are organized in AFS.
48 * Relationship between pts uid and unix uid::
50 How to cope with reality
54 * Integration with Kerberos::
55 * Kerberos tickets and AFS tokens::
60 * The relation between Arlad and NNPFS::
61 * The life of a file::
63 * The files in arlad/::
66 How to debug arla when its not working
69 * Debugging LWP with GDB::
72 * Debugging techniques::
79 * Porting user-space::
88 Odd stuff you find when looking around
101 File: arla.info, Node: Introduction, Next: AFS infrastructure, Prev: Top, Up: Top
106 *Caution:* Parts of this package are not yet stable software. If
107 something doesn't work, it's probably because it doesn't. If you
108 don't have backup of your data, take backup.
113 Arla is a free AFS implementation. Some of the goals are:
115 * to have an implementation that is free and can be used for adding
116 and playing with cool stuff, like support for disconnected-mode.
117 Implement features you can't get from commercial AFS.
119 * to provide an alternative to Transarc's and OpenAFS AFS-clients and
120 server implementations.
122 * to add support for platfroms that don't have AFS support from
123 Transarc or OpenAFS today.
125 This release is known to work on the following platforms: NetBSD,
126 OpenBSD, FreeBSD, Linux, Solaris, Darwin/MacOS X.
128 Earlier releases are known to work on current or earlier versions of the
129 following platforms: SunOS, AIX, IRIX, Digital UNIX. Some fixes might
130 be necessary to make Arla work.
132 There is or has been done work to support the following platforms: HPUX,
133 Fujitsu UXP/V. Some development is necessary to make Arla work.
135 There is work going on to support the following platform: Windows
136 NT/2000. Contributions are very welcome.
141 Arla has the following features (quality varies between stable and not
144 * a rxgen implementation called ydr (stable).
146 * a cache manager replacing Transarc's afsd. The cache managers
147 quality depends on platform: *BSD, Linux i386 and Solaris are
148 stable, others platforms are not as tested ane therefore not as
151 * partly implemented fs, vos, pts commands. Commands typically issued
152 by users are stable, commands issued by administrators may return
153 unmotivated errors or are not implemented yet.
155 * an implementaion of rxkad written outside USA without any export
156 restrictions (stable).
158 * a server implementation called milko, containing file server,
159 volume server and protection server. The file server has an API to
160 the storage backend(s). Milko is still unstable and not fit for
166 If you find bugs in this software, make sure it is a genuine bug and not
167 just a part of the code that isn't implemented.
169 Bug reports should be sent to <arla-drinkers@stacken.kth.se>. Please
170 include information on what machine and operating system (including
171 version) you are running, what you are trying to do, what happens, what
172 you think should have happened, an example for us to repeat, the output
173 you get when trying the example, and a patch for the problem if you have
174 one. Please make any patches with `diff -u' or `diff -c'.
176 Suggestions, comments and other non bug reports are also welcome.
181 There are two mailing lists with talk about Arla.
182 <arla-announce@stacken.kth.se> is a low-volume announcement list, while
183 <arla-drinkers@stacken.kth.se> is for general discussion.
185 There is also commit list <arla-commit@stacken.kth.se>. Send a message
186 to <LIST-request@stacken.kth.se> to subscribe.
188 The list are achived on <http://www.stacken.kth.se/lists/>.
190 Please note that the mailinglists have a policy of subscriber only
191 posting. So if you want to mail a message to the list, subscribe to it
192 first, otherwise it wont be posted on the list.
195 File: arla.info, Node: AFS infrastructure, Next: Organization of data, Prev: Introduction, Up: Top
200 This is an overview of the AFS infrastructure as viewed from a Transarc
201 perspective, since most people still run Transarc cells.
206 AFS filespace is split up in smaller parts called cells. These cells are
207 usually listed under `/afs'. A cell is usually a whole organization or
208 an adminstative unit within an organization. An example is e.kth.se
209 (with the path `/afs/e.kth.se'), that is the department of electrical
210 engineering at KTH, which obviously has the `e.kth.se' domain in DNS.
211 Using DNS domains for cell names is the typical and most convenient way.
213 Note that cell names are written in lowercase by convention.
218 All cells (and their db-servers) in the AFS world are listed in a file
219 named `CellServDB'. There is a central copy that is maintained by
220 Transarc at `/afs/transarc.com/service/etc/CellServDB'.
222 In spite of being organized in IPnumber - name pairs, where the name
223 parts resemble comments, both values are used by Transarc software and
224 confusion may arise if they are not synchronized with each other.
227 >e.kth.se # Royal Institute of Technology, Elektro
228 130.237.48.8 #sonen.e.kth.se.
229 130.237.48.7 #anden.e.kth.se.
230 130.237.48.244 #fadern.e.kth.se.
232 Again, please note that the text after the # in the cell-name is a
233 comment, *but* the hostnames after the # on the rows of an IP-address
234 is *not* a comment. The host and the ip-address needs to point at the
240 In addition Arla can use DNS to find the db-servers of a cell. The DNS
241 resource record that is used is the `AFSDB'. The resourcerecord was
242 created by Transarc but have never been implemeted in released software.
244 `AFSDB' tells you what machines are db servers for a particular cell.
245 The `AFSDB' resourcerecord is also used for DCE/DFS. An example (the 1
246 means AFS, 2 is used for DCE):
248 e.kth.se. IN AFSDB 1 fadern.e.kth.se.
249 e.kth.se. IN AFSDB 1 sonen.e.kth.se.
250 e.kth.se. IN AFSDB 1 anden.e.kth.se.
255 Some cells use the abbreviated version `/afs/<word-before-first-dot>'
256 (in the example above that would be `/afs/e/'. This might be
257 convenient when typing them, but is a bad idea, because it does not
258 create the same name space everywhere. If you create a symbolic link
259 to `/afs/e/foo/bar', it will not work for people in other cells.
264 There are several servers running in an AFS cell. For performance and
265 redundancy reasons, these servers are often run on different hosts.
266 There is a built in hierarchy within the servers (in two different
269 There is one server that keeps track of the other servers within a host,
270 restart them when they die, make sure they run in the correct order,
271 save their core-files when they crash, and provide an interface for the
272 sysadmin to start/stop/restart the servers. This server is called
273 bos-server (Basic Overseer Server).
275 Another hierarchy is the one who keeps track of data (volumes, users,
276 passwords, etc) and who is performing the real hard work (serving files)
277 There is the the database server that keeps the database (obviously),
278 and keeps several database copies on different hosts relpicated with
279 Ubik (see below). The fileserver and the client software (like the
280 afsd/arlad, pts and, vos) are pulling meta-data out of the dbserver to
281 find where to find user-privileges and where volumes resides.
283 Basic overseer - boserver
284 =========================
286 The Bos server is making sure the servers are running. If they crash, it
287 saves the corefile, and starts a new server. It also makes sure that
288 servers/services that are not supposted to run at the same time do not.
289 An example of this is the fileserver/volserver and salvage. It would be
290 devastating if salvage tried to correct data that the fileserver is
291 changing. The salvager is run before the fileserver starts. The
292 administrator can also force a file server to run through salvage again.
297 Ubik is a distributed database. It is really a (distributed) flat file
298 that you can perform read/write/lseek operation on. The important
299 property of Ubik is that it provides a way to make sure that updates are
300 done once (transactions), and that the database is kept consistent. It
301 also provides read-only access to the database when there is one (or
302 more) available database-server(s).
304 This works the following way: A newly booted server sends out a message
305 to all other servers that tells them that it believes that it is the new
306 master server. If the server gets a notice back from an other server
307 that tells it that the other server believes that it (or a third server)
308 is the master, depending on how long it has been masterserver it will
309 switch to the new server. If they can't agree, the one with the lowest
310 ip-address is supposed to win the argument. If the server is a slave it
311 still updates the database to the current version of the database.
313 A update to the database can only be done if more than half of the
314 servers are available and vote for the master. A update is first
315 propaged to all servers, then after that is done, and if all servers
316 agree with the change, a commit message is sent out from the server, and
317 the update is written to disk and the serial number of the database is
320 All servers in AFS use Ubik to store their data.
322 Volume Location database server - vlserver
323 ==========================================
325 The vldb-server is resposible for the information on what fileserver
326 every volume resides and of what kind of volumes exists on each
329 To confuse you even more there are three types of support for the
330 clients. Basically there is AFS 3.3, 3.4, and 3.6 support. The different
331 interfaces look the same for the system administrator, but there are
332 some important differences.
334 AFS 3.3 is the classic interface. 3.4 adds the possibility of multihomed
335 servers for the client to talk to, and that introduces the N interface.
336 To deal with multihomed clients AFS 3.5 was introduced. This is called
337 call the U interface. The name is due to how the functions are named.
339 The N interface added more replication-sites in the database-entry
340 structure. The U interface changed the server and clients in two ways.
342 When a 3.5 server boot it registers all its ip-addresses. This means
343 that a server can add (or remove) an network interface without
344 rebooting. When registering at the vldb server, the file server presents
345 itself with an UUID, an unique identifier. This UUID will be stored in a
346 file so the UUID keeps constant even when network addresses are changed,
349 Protection server - ptserver
350 ============================
352 The protection server keeps track of all users and groups. It's used a
353 lot by the file servers. Users can self create, modify and delete
356 When a fileserver is access they are durring the authentication giving
357 the name of the client. This name if looked up in the
358 protection-database via the protection server that returns the id of the
359 user and all the groups that the user belongs too.
361 This information is used when to check if the user have access to a
362 particular file or directory. All files created by the user are assigned
363 the user id that the protectionserver returned.
365 Kerberos server - kaserver
366 ==========================
368 The kaserver is a Kerberos server, but in other clothes. There is a new
369 RPC interface to get tickets (tokens) and administer the server. The
370 old Kerberos v4 interface is also implemented, and can be used by
371 ordinary Kerberos v4 clients.
373 You can replace this server with an Heimdal kdc, since it provides a
374 superset of the functionality.
376 Backup server - buserver
377 ========================
379 The backup server keeps the backup database that is used when backing up
380 and restoring volumes. The backup server is not used by other servers,
383 Update server - upserver
384 ========================
386 With the update server its possible to automagicly update configuration
387 files, server binaries. You keep masters that are supposed to contain
388 the correct copy of all the files and then other servers can fetch them
391 Fileserver and Volume Server - fs and volser
392 ============================================
394 The file server serves data to the clients, keeps track of callbacks,
395 and breaks callbacks when needed. Volser is the administative interface
396 where you add, move, change, and delete volumes from the server.
398 The volume server and file server are ran at the same time and they sync
399 with each other to make sure that fileserver does not access a volume
400 that volser is about to modify.
402 Every time a fileserver is started it registers it IP addresses with the
403 vldbserserver using the VL_RegisterAddrs rpc-call. As the unique
404 identifier for itself it uses its afsUUID.
406 The afsUUID for a fileserver is stored in /usr/afs/local/sysid. This is
407 the reson you must not clone a server w/o removing the sysid file.
408 Otherwise the new filserver will register as the old one and all
409 volumes on the old fileserver are pointed to the new one (where the
410 probably doesn't exist).
412 The fileserver doesn't bind to a specific interface (read address), gets
413 all packets that are destined for port 7000 (afs-fileserver/udp). All
414 outgoing packets are send on the same socket, and means that your
415 operatingsystem will choose the source-address of the udp datagram.
417 This have the side-effect that you will have asymmetric routing on
418 mulithomed fileserver for 3.4 (and older) compatible clients if they
419 don't use the closest address when sorting the vldb entry. Arla avoids
425 Salvage is not a real server. It is run before the fileserver and volser
426 are started to make sure the partitions are consistent.
428 It's imperative that salvager is NOT run at the same time as the
429 fileserver/volser is running.
431 Things that milko does differently.
432 ===================================
434 Fileserver, volumeserver, and salvage are all in one program.
436 There is no bu nor ka-server. The ka-server is replaced by kth-krb or
437 Heimdal. Heimdal's kdc even implements a ka-server readonly interface,
438 so your users can keep using programs like klog.
441 File: arla.info, Node: Organization of data, Next: AFS and the real world, Prev: AFS infrastructure, Up: Top
446 This chapter describes how data is stored and how AFS is different from,
447 for example, NFS. It also describes how data is kept consistent and what
448 the requirements were and how that inpacted on the design.
453 * Data organization::
455 * Volume management::
456 * Relationship between pts uid and unix uid::
459 File: arla.info, Node: Requirements, Next: Data organization, Prev: Organization of data, Up: Organization of data
466 It should be possible to use AFS with hundred-thousands of users
469 Writes that are done to different parts of the filesystem should
470 not affect each other. It should be possible to distribute out the
471 reads and writes over many fileservers. If you have a file that is
472 accessed by many clients, it should be possible to distribute the
475 * Transparent to users
477 Users should not need to know where their files are stored. It
478 should be possible to move their files while they are using their
483 It should be easy for a administrator to make changes to the
484 filesystem. For example to change quota for a user or project. It
485 should also be possible to move the users data for a fileserver to
486 a less loaded one, or one with more diskspace available.
488 Some benefits of using AFS are:
490 * user-transparent data migration
492 * an ability for on-line backups;
494 * data replication that provides both load balancing and
495 robustness of critical data
497 * global name space without automounters and other add-ons;
499 * @sys variables for platform-independent paths to binary
504 * client-side caching;
511 AFS isn't constructed for storing databases. It would be possible
512 to use AFS for storing a database if a layer above for locking and
513 synchronizing data would be provided.
515 One of the problems is that AFS doesn't include mandatory
516 byte-range locks. AFS uses advisory locking on whole files.
518 If you need a real database, use one, they are much more efficent
519 on solving a database problem. Don't use AFS.
523 File: arla.info, Node: Data organization, Next: Callbacks, Prev: Requirements, Up: Organization of data
528 A volume is a unit that is smaller than a partition. It is usually (or
529 should be) a well defined area, like a user's home directory, a project
530 work area, or a program distribution.
532 Quota is controlled on volume-level. All day-to-day management is done
538 In AFS a partition is what normally is named a partition. All partions
539 that afs is using are named a special way, `/vicepNN', where NN is
540 ranged from a to z, continuing with aa to zz. The fileserver (and
541 volser) automaticly picks upp all partitions starting with `/vicep'
543 Volumes are stored in a partition. Volumes can't span several
544 partitions. Partitions are added when the fileserver is created or when
545 a new disk is added to a filesystem.
547 Volume cloning and read-only clones
548 ===================================
550 A clone of a volume is often needed for volume operations. A clone is a
551 copy-on-write copy of a volume, the clone is the read-only version.
553 Two special versions of a clone are the read-only volume and the backup
554 volume. The read-only volume is a snapshot of a read-write volume (that
555 is what a clone is) that can be replicated to several fileservers to
556 distribute the load. Each fileserver plus partition where a read-only
557 clone is located is called a replication-site. It usually does not make
558 sense to have more than one read-only clone on each fileserver.
560 The backup volume is a clone that typically is made (with `vos
561 backupsys') each night to enable the user to retrieve yesterday's data
562 when they happen to remove a file. This is a very useful feature, since
563 it lessens the load on the system-administrators to restore files from
564 backup. The volume is usually mounted in the root user's home directory
565 under the name OldFiles. A special feature of the backup volume is that
566 you can't follow mountpoints out of a backup volume.
571 Volumes are independent of each other. To glue together the file tree
572 there are `mountpoint's. Mountpoints are really symlinks that are
573 formated in a special way so that they point out a volume and an
574 optional cell. An AFS-cache-manager will show a mountpoint as directory
575 and in fact it will be the root directory of the target volume.
578 File: arla.info, Node: Callbacks, Next: Volume management, Prev: Data organization, Up: Organization of data
583 Callbacks are messages that enable the AFS-cache-manager to keep the
584 files without asking the server if there is newer version of the file.
586 A callback is a promise from the fileserver that it will notify the
587 client if the file (or directory) changes within the timelimit of the
590 For contents of read-only volumes there is only one callback per volume
591 called a volume callback and it will be broken when the read-only volume
594 The time range of callbacks is from 5 to 60 minutes depending on how
595 many users of the file exist.
598 File: arla.info, Node: Volume management, Next: Relationship between pts uid and unix uid, Prev: Callbacks, Up: Organization of data
603 All volume managment is done with the `vos' command. To get a list of
604 all commands `vos help' can be used. For help on a specific vos
605 subcommand, `vos subcommand -h' can be used.
609 vos create mim c HO.staff.lha.fluff -quota 400000
613 Volumes can be moved from a server to another, even when users are
618 Read-only volumes can be replicated over several servers, they are
619 first added with `vos addsite', and the replicated with `vos
620 release' over the servers.
624 When you want to distribute the changes in the readwrite volume to
625 the read-only clones.
629 Volumes can be removed
631 Note that you shouldn't remove the last readonly volume since this
632 makes clients misbehave. If you are moving the volume you should
633 rather add a new RO to the new server and then remove it from the
636 * Backup and restoration of volumes.
638 `vos backup' and `vos backupsys' creates the backup volume.
640 To stream a volume out to a `file' or `stdout' you use `vos dump'.
641 The opposite command is named `vos restore'.
645 File: arla.info, Node: Relationship between pts uid and unix uid, Prev: Volume management, Up: Organization of data
647 Relationship between pts uid and unix uid
648 =========================================
650 Files in AFS are created with the pts uid of the token that was valid at
651 the time. The pts uid number is then by commands like `ls -l'
652 interpreted as a unix uid and translated into a username. If the pts and
653 the unix uids differ, this might confuse the user as it looks like as
654 her files are owned by someone else. This is however not the case.
655 Complications can occur if programs impose further access restrictions
656 based on these wrongly interpreted uids instead of using the `access()'
657 system call for that purpose. Graphical file browsers are typically
658 prone to that problem with the effect that the users are not able to
659 see their own files in these tools.
662 File: arla.info, Node: AFS and the real world, Next: Parts of Arla, Prev: Organization of data, Up: Top
664 AFS and the real world
665 **********************
667 This chapter tries to describe problems that you see in the real (not
668 that perfect) world and show possible solutions to these problems.
672 * NAT:: Truly evil stuff.
673 * Samba:: Export AFS to Windows computers.
674 * Integration with Kerberos:: How to integrate Kerberos with AFS.
675 * Kerberos tickets and AFS tokens:: History and tools
678 File: arla.info, Node: NAT, Next: Samba, Prev: AFS and the real world, Up: AFS and the real world
683 There's something evil out there that's called NAT, which stands for
684 Network Address Translation. For whatever reasons, people are using it
685 and will continue doing so.
687 First of all, it seemed like AFS should work just fine through NAT, you
688 just appear to be coming from the NAT-address and some random port
689 instead. Looking closer at different NAT implementations it seems like
690 they have a rather short timeout:
704 If the client doesn't transmit any traffic to a particular host for that
705 amount of time, it will get mapped to one of the IP address of the
706 NAT-server (if you happen to run PAT, the port will be randomized too).
708 The authors of Rx realized that keeping a Rx connection associated with
709 (IP-address,port) pair was a bad idea. One example is that you get
710 problems with multi-homed hosts. So Rx keeps its own connection id data
711 in the packet. With this feature client and server should be able to
712 detect address changes.
714 Unfortunately, the use of the orignal Rx-code stops this from happening
715 in Transarc/OpenAFS code. The code keeps track of incoming packets and
716 keeps track of the right peer (client). But it never updates the
717 IP-address,port pair in its data structure, so the answer packet will go
718 to the old IP-address,port pair.
720 If you can control your NAT machine you can have static mapping for your
721 AFS hosts (Transarc/OpenAFS uses source port 7000 and Arla uses source
722 port 4711). You can try to use Natkeep
723 <http://mit.edu/fredette/www/natkeep/> if you run an old Arla or
724 Transarc/OpenAFS client. From version 0.36 arla will have support for
725 polling the servers at the right interval to prevent NAT from dropping
726 information about your session.
729 File: arla.info, Node: Samba, Next: Integration with Kerberos, Prev: NAT, Up: AFS and the real world
734 The major problem when exporting the AFS filespace read-write to SMB
735 (Windows fileshareing) using Samba is the transfer of the user token to
736 the smb-server. The simple may is to use clear-text password between the
737 Windows client and the samba-server, and then to get tokens for the user
738 with this password. This solution is clearly not acceptable for security
739 aware AFS administrators.
741 Describe here how to make AFS work "securely" with samba.
743 On solution is to use `kimpersonate' + store afs key on fileserver
747 File: arla.info, Node: Integration with Kerberos, Next: Kerberos tickets and AFS tokens, Prev: Samba, Up: AFS and the real world
749 Integration with Kerberos
750 =========================
752 Kerberos 4 and 5 can be integrated quite well with AFS. This is mainly
753 due to the fact that the security model used in AFS is Kerberos. The
754 kaserver is a Kerberos 4 server with pre-authentication. The kaserver
755 also provides a feature that limits the number of password retries, and
756 after that you are locked out for half an hour. This feature can only be
757 used in the ka interface as it requires pre-authentication, but since
758 the kaserver provides a Kerberos 4 interface (without pre-authentication
759 and without this limitation) it is quite worthless.
761 Many sites indeed use a kerberosserver instead of a kaserver. One of
762 the reasons is that they want to use Kerberos 5 (which is required for
765 To use a kerberosserver, you have to put the same key into the AFS
766 KeyFile and the principal named afs of your kerberos realm. If you have
767 a cell which has another name than lowercase of your realmname, the
768 instance is the cellname. As the cellname often contains dots, this can
769 be quite confusing in Kerberos 4: afs.stacken.kth.se@STACKEN.KTH.SE. The
770 first dot is the seperator between principal and instance, the other
771 dots are part of the name. The simplest way to do create a KeyFile on
772 your AFS server is to use heimdal's kadmin to generate and extract a
773 KeyFile. It has an option designed to do so. Be aware that the serial
774 number of the afs principal must match on all kerberos servers and all
775 KeyFiles on all AFS servers.
777 The default cellname to get a kerberos ticket for is contained in the
778 configuration file `ThisCell'. If you got more than one cell, you want
779 to list all cells to get tickets for in the `TheseCells' file.
782 File: arla.info, Node: Kerberos tickets and AFS tokens, Prev: Integration with Kerberos, Up: AFS and the real world
784 Kerberos tickets and AFS tokens
785 ===============================
787 To further confuse the poor user, AFS and Kerberos programmers decided
788 that they wanted to store their credentials at different places. In AFS,
789 the kernel was a natural place to store the credentials (named token)
790 since the CMU/Transarc AFS/OpenAFS implementation lives in the kernel.
791 The Kerberos people on the other hand thought that storing the
792 credentials (named ticket) in a file would be a good idea.
794 So know you have to synchronize the credentials if you just want to
795 enter your password once. There are several tools that can do that for
796 you. The question is what tools to use for what problem.
798 To add to the confusion not all tools talk to both Kerberos and
799 kaservers. There is also a bogus user-id in the token that is supposed
800 to be the same as your pts-user-id. Not that it makes any difference,
801 but some people get confused when unknown numbers show up in the token.
802 The easily confused people are often the ones that have more than one
803 principal in the same realm/cell (read sysadmins).
805 If you want to get your ticket from your Kerberos server, you use
806 `kinit', and then use `afslog' or `aklog' to get AFS tokens and push
807 them to the kernel (and AFS daemon). Some `kinit' (and `kauth') can do
808 both for you, use `kinit --afslog' or simply `kauth'. Note that `kinit'
809 and `kauth' don't get set your AFS-token user-id right, and thus can be
810 confusing for people who think that this is important.
812 The `klog' program that you get with Transarc/OpenAFS talks to the
813 kaserver and behaves just-right in the sense that it talks to the pts
814 server to get the AFS-token user-id right, but `klog' talks only to the
815 kaserver which will not work for people with a Kerberos server. `Klog'
816 in Arla was written by Chris Wing <wingc@engin.umich.edu> as a part of
817 a packet called `afsutils', they did the right thing and talked to the
818 pts-server to get the user-id. However, it uses Kerberos libs to talk
819 to the server. These libraries require the files `/etc/krb.conf' and
820 `/etc/krb.realms' to be setup correctly for their cell/realm. Not that
823 A long time ago Ken Hornstein <kenh@cmf.nrl.navy.mil> wrote the AFS
824 Migration Kit that helped you to migrate from AFS to MIT Kerberos 5.
825 It included a tool named aklog that could convert a Kerberos tickets to
826 tokens. This tool was also rewritten in Arla by Brandon S. Allbery
827 <allbery@ece.cmu.edu>. `aklog' can't get you new credentials, it just
828 converts old ones to new ones.
830 Then Transarc decided that they needed to fix a security hole in their
831 kaserver, and while doing that, they managed to break a part in the
832 kaserver so it ceased to work for kerberos requests.
834 First the defect existed unnoticed for a long time, later Transarc has
835 not managed to distribute a working version of the kaserver. Due to
836 this, a lot of sites run a kaserver with this defect. Instead of
837 installing working authentification servers from another sources,
838 people started to whine again and Love <lha@stacken.kth.se> wrote the
839 tool `kalog' that talked the ka-protocol (but didn't do the AFS user-id
840 right) to work around that problem.
842 All tools that use Kerberos 4 need a working `/etc/krb.conf' and
843 `/etc/krb.realms'. Kerberos 5 programs need `/etc/krb5.conf'. AFS
844 aware tools need `/usr/arla/etc/CellServDB' or
845 `/usr/vice/etc/CellServDB'.
847 Also the Kerberos implementations from KTH (kth-krb and Heimdal) include
848 AFS support to make your life more pleasant. One thing is that you can
849 have a file `$HOME/.TheseCells' that lists the cells you use and the
850 Kerberos tools will try to get tickes and tokens for those cells.
851 Heimdal contains support for converting a Kerberos 4 srvtab to an AFS
854 Below is a table that describes what tools does what, what
855 inheritance(s) they have, and what protocol(s) they speak. From the
856 inheritance (also in a table below) it is possible to deduct what
857 configuration files the tools use.
859 Tool Inheritance Protocol Produces
860 Transarc/OpenAFS klog afs authlib KA Ticket and tokens
861 Arla klog Kerberos and Kerberos Ticket and tokens
863 AFS Migration kit's MIT Kerberos Kerberos Converts Kerberos
864 aklog and Ken tickets to tokens
867 Arla's aklog Kerberos and Kerberos Converts Kerberos
868 libkafs tickets to tokens
869 kth-krb's and Kerberos and Kerberos Converts Kerberos
870 Heimdal's afslog libkafs tickets to tokens
871 kalog arla and KA Get initial
872 libkafs ticket, store
878 Reads `CellServDB' and `ThisCell' in `/usr/vice/etc'
880 `Kerberos and libkafs'
881 Kerberos 4 uses `/etc/krb.conf' and `/etc/krb.realms'. Kerberos 5
882 uses `/etc/krb5.conf'.
885 Arla uses `CellServDB' and `ThisCell' in `/usr/vice/etc' and
889 Libkafs uses DNS and `CellServDB' in `/usr/vice/etc' and
890 `/usr/arla/etc' to figure out what ticket it should convert into
891 tables. The file `TheseCells' and `$HOME/.TheseCells' is used to
895 File: arla.info, Node: Parts of Arla, Next: Debugging, Prev: AFS and the real world, Up: Top
900 *Caution:* This text just tries to give a general picture. For
901 real info read the code. If you have any questions, mail
902 <arla-drinkers@stacken.kth.se>.
907 * The relation between Arlad and NNPFS::
908 * The life of a file::
910 * The files in arlad/::
914 File: arla.info, Node: How arla works, Next: The relation between Arlad and NNPFS, Prev: Parts of Arla, Up: Parts of Arla
919 Arla consists of two parts, a userland process (arlad) and the
920 kernel-module (nnpfs).
922 Arlad is written in user-space for simpler debugging (and less
923 rebooting). As a uset space program arlad does not have the same
924 limitations as if it would be written in the kernel. To avoid
925 performance loss as much as possible, nnpfs is caching data.
927 nnpfs and arlad communicate with each other via a char-device-driver.
928 There is a rpc-protocol currenly used specially written for this
931 nnpfs is written to be as simple as possible. Theoretically, nnpfs could be
932 used by other user-space daemons to implement a file system. Some
933 parts, such as syscalls, are arla-specific. These parts are designed to
934 be as general as possible.
936 For example, nnpfs does not recognize which pioctl the user-level program
937 calls, it just passes this information on to arlad.
940 File: arla.info, Node: The relation between Arlad and NNPFS, Next: The life of a file, Prev: How arla works, Up: Parts of Arla
942 The relation between Arlad and NNPFS
943 ==================================
948 Edit file | Arlad | ------> Network
950 ----|-----------------|[1]----
952 Kernel | VFS | <--[2]--> | NNPFS |
955 [1] A char device (/dev/nnpfs0)
957 [2] Xfs provides a filesystem for the vfs-layer in
958 the operating system.
961 File: arla.info, Node: The life of a file, Next: Tools and libs, Prev: The relation between Arlad and NNPFS, Up: Parts of Arla
966 Step by step description of what happens during the creation of a file.
967 The names are inspired of BSD-style VFS-layer but the idea is the same
968 in most operating systems.
970 * The user decides to open a file.
972 * open(2) syscall is issued.
974 * The vfslayer sends a VOP_LOOKUP to nnpfs that is forwarded to arlad
975 with a getnode() (seq-num 1).
977 * arlad tries to find the requested file and then, if found, sends an
978 install_node to nnpfs by writing to the nnpfs character device.
980 * nnpfs inserts the node into the cache and returns from the device
983 * arlad sends a wakeup rpc message (seq-num 1) to nnpfs. If the
984 return value is zero nnpfs tries to find the node in the cache, if
985 not found it might have been flushed out of the cache and the whole
988 * If a none-zero return value is returned, this value is sent as
989 reply to the user. This way arla can decide what error message is
990 returned, without nnpfs having support for each error.
992 * nnpfs now checks if it has the valid attributes. If the attributes
993 are invalid, nnpfs will send a rpc message to arlad to refresh it.
995 * Since the user wanted to open the file, a getdata rpc message is
996 sent from nnpfs to arlad. Now arlad fetches the files from the afs
999 * Arlad stores the file in the file cache. All vnode operations will
1000 be done on this file. Now arlad sends a installdata to nnpfs.
1002 * When nnpfs recives the installdata it looks up the node in the cache,
1003 and then it does a VOP_LOOKUP to find a vnode to the cachefile
1004 (and store it to keep it for future use).
1006 * The same thing is done when the file is a directory, except that
1007 the directory is converted from the afs directory format to an
1008 operating system dependent format and stored in a file. nnpfs reads
1011 * If the directory is modified locally, write operations are done on
1012 the file obtained from the afs-server, and when done the newly
1013 changed file is converted and reinstalled.
1015 * Now the user wants to read a file.
1017 * read(2) system call is issued.
1019 * A VOP_READ is sent to the from the vfs-layer to nnpfs.
1021 * nnpfs checks if it has valid attributes/and data (and updates if
1022 needed). Now VOP_READ is simply performed on the stored vnode of
1027 File: arla.info, Node: Tools and libs, Next: The files in arlad/, Prev: The life of a file, Up: Parts of Arla
1032 What other tools does the arla suite consists of
1034 libutil: `util/libutil.a' - A library for the most often used
1035 modules like hashtable, double-linked list, logging functions,
1038 rx: `rx/librx.a' - The library for the rx protocol
1039 (*note Rx protocol::).
1041 lwp: `lwp/liblwp.a' - The library for the lwp thread-package
1044 ydr: `ydr/ydr' - A stub generator that replaces rxgen.
1046 rxkad: `rxkad/librxkad.a' - The rx Kerberos authentication package.
1048 roken: `lib/roken/libroken.a' - The library that will unbreak
1049 things that are missing or broken.
1051 ko: `lib/ko/libko.a' - A library of functions that are arlad-core
1052 related but also are useful for programs like vos, pts, fs, etc.
1054 arlalib: `appl/lib/libarlalib.a' - A broken library that does all
1055 the hard work with connections etc.
1057 fs: `appl/fs/fs' - The fs util, extra feature
1058 (amongst others): getfid.
1060 vos: `appl/vos/vos' - The vos util.
1062 pts: `appl/pts/pts' - The pts util, extra feature: dump.
1064 udebug: `appl/udebug/udebug' - Debug your ubik server.
1066 File: arla.info, Node: Rx protocol, Next: LWP, Up: Tools and libs
1073 One of rxgen or ydr is used to generate stub-files, ydr is better since
1074 it generates prototypes, too.
1076 The current implemetation of rx it not that beautiful.
1079 File: arla.info, Node: LWP, Prev: Rx protocol, Up: Tools and libs
1084 LWP is a preepmtive thread package. It does it's context-switching by
1085 creating a private stack for each thread. The heart of the package is
1088 The stack is checked for overruns in context-switches, but that is often
1089 too late. It might be an idea to add a `red zone' at the top of the
1090 stack to be able to detect overruns.
1093 File: arla.info, Node: The files in arlad/, Next: pioctl and kafs, Prev: Tools and libs, Up: Parts of Arla
1098 This is a short describtion of the files to bring new deveplopers up to
1104 `adir.c' - contains all functions needed to to operations
1105 on afs-directory files.
1107 `afsdir_check.c' - check if an AFS-directory looks sane.
1109 `arla.c' - The startup and the standalone (-t) code.
1111 `arladeb.c' - The logging code specific to arla, like aliases
1112 for debugging masks.
1114 `cmcb.c' - The callback-server that is contacted by the
1115 server when a callback expires or a server wants to send an
1118 `conn.c' - The connection cache, responsible for caching connection
1119 based on pag and security index. It will also create new
1120 connection when needed.
1122 `cred.c' - Keep track of all credentials that all users have
1123 inserted. Indexed on pag.
1125 `fbuf.c' - An interface between rx and filedescriptors. It is also
1126 used to mmap files. Used by `adir.c'.
1128 `fcache.c' - Responsible for keeping track of files in the cache.
1129 Also fetches files from the afs-server.
1131 `fprio.c' - Tries to give files priority. These files are
1132 therefore not garbarge-collected as fast as they would be
1133 otherwise. If you wonder what this is for, think of the
1136 `inter.c' - An interface to hide all junk in fcache, just give
1137 the items a VenusFid and you can access them this way.
1139 `kernel.c' - The interface between arlad and the char-device.
1141 `messages.c' - The rpc interface between arlad and nnpfs.
1143 `volcache.c' - Cache for all volumes.
1144 Operating system specific files
1145 -------------------------------
1147 These are the files that contain operating specific functions. Today
1148 it's just conv_dir().
1152 `bsd-subr.c' - FreeBSD 2.2.6, OpenBSD 2.2, 2.3, NetBSD 1.3.x
1154 `hpux-subr.c' - HPUX
1156 `irix-subr.c' - Irix
1158 `linux-subr.c' - Linux 2.0.x, 2.1.x, 2.2
1160 `solaris-subr.c' - Solaris 2.5.x, 2.6, 7
1162 `sunos-subr.c' - SunOS
1164 `unknown-subr.c' - Stub used when compiled on a unknown OS.
1166 File: arla.info, Node: pioctl and kafs, Prev: The files in arlad/, Up: Parts of Arla
1171 The pioctl interface is the only part of nnpfs that is afs related.
1173 pioctl is a ioctl but called with a path instead of a filedescriptor.
1174 When you probe if there is a live afsclient you first run `k_hasafs()'
1175 that probes if there is an afsclient around. It also sets up some
1176 static variables in the library. So if you start to do `pioctl()' w/o
1177 running `k_hasafs()', you're up to funny errors, and/or get a corefile.
1179 `k_hasafs()' does an `AFSCALL_PIOCTL' with opcode `VIOCSETTOK' and
1180 insize == 0, ie you try to set a token (ticket) that is 0 bytes long.
1181 This is cleary invalid and kafs expects to find an `EINVAL' returned
1184 The pioctl is used more then just for `AFSCALL_PIOCTL', an other use is
1185 `AFSCALL_SETPAG' (setting pag). It has also been in use for setting nnpfs
1188 When nnpfs discovers that a path is given in the `pioctl()' it does a
1189 `VOP_LOOKUP' on the path and if the returned value is a vnode that
1190 resides in afs then it extracts the nnpfs-handle for that node (that just
1191 happens to be the VenusFid) and passes that on to arlad.
1193 The only ugly thing about the current implentation is that the syscall
1194 code assumes that the arlad on "nnpfs-fd" is the arlad that should get
1197 An example of using `pioctl()':
1200 fs_getfilecellname(char *path, char *cell, size_t len)
1202 struct ViceIoctl a_params;
1205 a_params.out_size=len;
1209 if (k_pioctl(path,ARLA_VIOC_FILE_CELL_NAME,&a_params,1) == -1)
1216 main (int argc, char **argv)
1221 errx (1, "there is no afs");
1223 if (fs_getfilecellname (".", cell, sizeof(cell)))
1224 errx (1, "fs_getfilecellname failed");
1226 printf ("cell for `.' is %s", cell);
1231 File: arla.info, Node: Debugging, Next: Arlad, Prev: Parts of Arla, Up: Top
1236 This chapter of the manual includes tips that are useful when debugging
1239 Arla and nnpfs contains logging facilities that is quite useful when
1240 debugging when something goes wrong. This and some kernel debugging tips
1246 * Debugging LWP with GDB::
1249 * Debugging techniques::
1250 * Kernel debuggers::
1254 File: arla.info, Node: Arlad, Next: Debugging LWP with GDB, Prev: Debugging, Up: Debugging
1259 If arlad is run without any arguments arlad will fork(2) and log to
1260 syslog(3). To disable forking use the -no-fork (-n) switch. In the
1261 current state of the code, arlad is allways to be started with the
1262 recover (-z) switch. This will invalidate your cache at startup. This
1263 restriction may be dropped in the future.
1265 To enable more debuggning run arla with the switch
1266 -debug=module1,module2,... One useful combination is
1267 --debug=all,-cleaner
1268 The cleaner output is usully not that intresting and can be ignored.
1270 A convenient way to debug arlad is to start it inside gdb.
1271 datan:~# gdb /usr/arla/libexec/arlad
1273 This gives you the opportunity to examine a crashed arlad.
1275 The arla team appreciates cut and paste information from the beginning
1276 to the end of the bt output from such a gdb run.
1278 To set the debugging with a running arlad use `fs arladeb' as root.
1282 datan:~# fs arladeb almost-all
1285 By default, arlad logs through syslog if running as a daemon and to
1286 stderr when running in the foreground (with `--no-fork').
1289 File: arla.info, Node: Debugging LWP with GDB, Next: nnpfs, Prev: Arlad, Up: Debugging
1291 Debugging LWP with GDB
1292 ======================
1294 For easy tracing of threads we have patch
1295 (<http://www.stacken.kth.se/projekt/arla/gdb-4.18-backfrom.diff>) for
1296 gdb 4.18 (a new command) and a gdb sequence (think script).
1298 The sequence only works for i386, but its just matter of choosing
1299 different offset in topstack to find $fp and $pc in the lwp_ps_internal
1300 part of the sequence.
1302 You should copy the `.gdbinit' (that you can find in the arlad
1303 directory in the source-code) to your home-directory, the directory from
1304 where you startat the patched gdb or use flag -x to gdb.
1306 Your debugging session might look like this:
1321 Print backtrace of FRAMEPOINTER and PROGRAMCOUNTER.
1323 (gdb) backfrom 0x8083b40 0x8083b40
1324 #0 0x8083b40 in ?? ()
1325 #1 0x8049e2f in LWP_MwaitProcess (wcount=1, evlist=0x8083b70)
1326 at /afs/e.kth.se/home/staff/lha/src/cvs/arla-foo/lwp/lwp.c:567
1327 #2 0x8049eaf in LWP_WaitProcess (event=0x8048b00)
1328 at /afs/e.kth.se/home/staff/lha/src/cvs/arla-foo/lwp/lwp.c:585
1329 #3 0x8048b12 in Producer (foo=0x0)
1330 at /afs/e.kth.se/home/staff/lha/src/cvs/arla-foo/lwp/testlwp.c:76
1331 #4 0x804a00c in Create_Process_Part2 ()
1332 at /afs/e.kth.se/home/staff/lha/src/cvs/arla-foo/lwp/lwp.c:629
1333 #5 0xfffefdfc in ?? ()
1334 #6 0x8051980 in ?? ()
1336 There also the possibility to run arla with pthreads (run configure with
1340 File: arla.info, Node: nnpfs, Next: nnpfs on linux, Prev: Debugging LWP with GDB, Up: Debugging
1345 NNPFS debugging does almost look the same on all platforms. They all
1346 share same debugging flags, but not all are enabled on all platforms.
1348 Change the debugging with the `fs nnpfsdebug' command.
1350 datan:~# fs nnpfsdebug
1352 datan:~# fs nnpfsdebug almost-all
1355 If it crashes before you have an opportunity to set the debug level, you
1356 will have to edit `nnpfs/YOUR-OS/nnpfs_deb.c' and recompile.
1358 The logging of nnpfs ends up in your syslog. Syslog usully logs to
1359 /var/log or /var/adm (look in /etc/syslog.conf).
1362 File: arla.info, Node: nnpfs on linux, Next: Debugging techniques, Prev: nnpfs, Up: Debugging
1367 There is a problem with klogd, it's too slow. Cat the `/proc/kmsg' file
1368 instead. Remember to kill klogd, since the reader will delete the text
1369 from the ring-bufer, and you will only get some of the message in your
1373 File: arla.info, Node: Debugging techniques, Next: Kernel debuggers, Prev: nnpfs on linux, Up: Debugging
1375 Debugging techniques
1376 ====================
1378 Kernel debugging can sometimes force you to exercise your imagination.
1379 We have learned some different techniques that can be useful.
1384 On operatingsystems with kernel debugger that you can use probably find
1385 where in the kernel a user-program live, and thus deadlocks or trigger
1386 the bad event, that later will result in a bug. This is a problem, how
1387 do you some a process to find where it did the intresting thing when you
1388 can't set a kernel breakpoint ?
1390 One way to be notified is to send a signal from the kernel module
1391 (psignal() on a BSD and force_sig() on linux). SIGABRT() is quite useful
1392 if you want to force a coredump. If you want to continue debugging, use
1395 Recreateable testcases
1396 ----------------------
1398 Make sure bugs don't get reintroduced.
1401 File: arla.info, Node: Kernel debuggers, Next: Darwin/MacOS X, Prev: Debugging techniques, Up: Debugging
1406 Kernel debuggers are the most useful tool when you are trying to figure
1407 out what's wrong with nnpfs. Unfortunately they also seem to have their
1408 own life and do not always behave as expected.
1413 Kernel debugging on NetBSD, OpenBSD, FreeBSD and Darwin are almost the
1414 same. You get the idea from the NetBSD example below:
1417 (gdb) target kcore netbsd.N.core
1418 (gdb) symbol-file /sys/arch/i386/compile/NUTCRACKER/netbsd.gdb
1420 This example loads the kernel symbols into gdb. But this doesn't show
1421 the nnpfs symbols, and that makes your life harder.
1423 Getting all symbols loaded at the same time
1424 -------------------------------------------
1426 If you want to use the symbols of nnpfs, there is a gdb command called
1427 `add-symbol-file' that is useful. The symbol file is obtained by
1428 loading the kernel module nnpfs with `kmodload -o /tmp/nnpfs-sym' (Darwin)
1429 or `modload' (NetBSD and OpenBSD). FreeBSD has a linker in the kernel
1430 that does the linking instead of relying on `ld'. The symbol address
1431 where the module is loaded get be gotten from `modstat', `kldstat' or
1432 `kmodstat' (it's in the `area' field).
1434 If you forgot the to run modstat/kldstat/kmodstat, you can extract the
1435 information from the kernel. In Darwin you look at the variable kmod
1436 (you might have to case it to a (kmod_info_t *). We have seen gdb loose
1437 the debugging info). kmod is the start of a linked list. Other BSDs have
1438 some variant of this.
1440 You should also source the commands in /sys/gdbscripts (NetBSD), or
1441 System/xnu/osfmk/.gdbinit (Darwin) to get commands like ps inside gdb.
1443 datan:~# modstat Type Id Off Loadaddr Size Info Rev Module
1444 Name DEV 0 29 ce37f000 002c ce388fc0 1 nnpfs_mod [...]
1446 (gdb) add-symbol-table nnpfs.sym ce37f000
1448 Debugging processes, examine their stack, etc
1449 ---------------------------------------------
1451 One of diffrencies between the BSD's are the `proc', a command that
1452 enables you do to a backtrace on all processes. On FreeBSD you give the
1453 `proc' command a `pid', but on NetBSD and OpenBSD you give a pointer to
1456 After you have ran `proc' to set the current process, you can examine
1457 the backtrace with the regular `backtrace' command.
1459 Darwin does't have a `proc' command. Instead you are supposed to use
1460 gdb sequences (System/xnu/osfmk/.gdbinit) to print process stacks,
1461 threads, activations, and other information.
1466 You can't get a crashdump for linux with patching the kernel. There are
1467 two projects have support for this. Mission Critical Linux
1468 <http://www.missioncritiallinux.com> and SGI <http://oss.sgi.com/>.
1470 Remember save the context of /proc/ksyms before you crash, since this is
1471 needed to figure out where the nnpfs symbols are located in the kernel.
1473 But you can still use the debugger (or objdump) to figure out where in
1474 the binary that you crashed.
1476 `ksymoops' can be used to create a backtrace.
1481 Adb is not a symbolic debugger, this means that you have to read the
1482 disassembled object-code to figure out where it made the wrong turn and
1483 died. You might be able to use GNU objdump to list the assembler and
1484 source-code intertwined (`objdump -S -d mod_nnpfs.o'). Remember that GNU
1485 binutils for sparc-v9 isn't that good yet.
1487 You can find the script that use use for the adb command `$<' in
1488 `/usr/lib/adb' and `/usr/platform/PLATFORNAME/adb'.
1490 Debugging a live kernel
1491 -----------------------
1493 An important thing to know is that you can debug a live kernel too, this
1494 can be useful to find dead-locks. To attach to a kernel you use a
1495 command like this on a BSD system (that is using gdb):
1498 (gdb) target kcore /dev/mem
1499 (gdb) symbol-file /sys/arch/i386/compile/NUTCRACKER/netbsd.gdb
1503 # adb -k /dev/ksyms /dev/mem
1505 Other useful debugging tools
1506 ----------------------------
1508 Most diagnosics tools like ps, dmesg, and pstat on BSD systems used to
1509 look in kernel memory to extract information (and thus earned the name
1510 kmem-groovlers). On some systems they have been replaced with other
1511 method of getting their data, like /proc and sysctl.
1513 But due to their heritage they can still be used in with a kernel and
1514 coredump to extract information on some system.
1517 File: arla.info, Node: Darwin/MacOS X, Next: Porting, Prev: Kernel debuggers, Up: Debugging
1522 You'll need two computers to debug arlad/nnpfs on darwin since the common
1523 way to debug is to use a remote kernel-debugger over IP/UDP.
1525 First you need to publish the arp-address of the computer that you are
1528 We have not found any kernel symbols in MacOSX Public Beta, so you
1529 should probably build your own kernel. Use Darwin xnu kernel source
1530 with cvs-tag: Apple-103-0-1 (not xnu-103).
1534 add-symbol-table ...
1538 File: arla.info, Node: Porting, Next: Programming, Prev: Darwin/MacOS X, Up: Top
1543 The largest part of the work needed to port Arla to a new operating
1544 system is in porting nnpfs, as kernel programming always is harder, less
1545 portable and messier than user-space dito. Arla in test mode
1546 (`arla-cli') should work without any porting on any system that's not
1547 very far away from Unix and that provides berkeley sockets (including
1548 cygwin32). The hard part is porting the NNPFS kernel module, and we will
1549 spent most of this text on how to do that.
1553 * Porting user-space::
1557 File: arla.info, Node: Porting user-space, Next: Porting NNPFS, Prev: Porting, Up: Porting
1562 The user-space parts should work on basically any system that is
1563 reasonably Posix and has berkeley sockets. The build uses autoconf and
1564 should adapt itself to most forseeable circumstances. If it fails to
1565 consider something that is missing or not working on the particular OS
1566 you are porting to, hard-code it to make sure that is what is missing
1567 and then try to create an autoconf test for it. If you fail to do so,
1568 or have no autoconf experience, send us the patches anyway and tell us
1569 where you are having the problem.
1574 The only thing that might take a little bit more effort in porting is
1575 the context-switch in the LWP user-level threads package. There are
1576 assembler versions for most of the common architectures in `lwp'. Part
1577 of the problem is getting this code assembled properly. There is
1578 unfortunately no easy and portable way of preprocessing and assembling
1579 code. There is a script `lwp/make-process.o.sh' that tries to do in
1580 some different ways, but it may fail for you. Next problem is that
1581 assembler syntax can vary a lot even on the same CPU. The source files
1582 are written in such a way that they should be acceptable to almost any
1583 syntax, but if it fails you have to find out what particular syntax has
1584 to be used and adapt the source file for that.
1586 The more interesting problem is if there is no support for your CPU.
1587 The first thing to try then is the `--with-pthreads' option that uses
1588 the pthreads library. If that fails or you want LWP working you have
1589 to figure out enough details on your CPU to write two functions in
1590 assembler, `savecontext' and `returnto' that save and restore the
1594 File: arla.info, Node: Porting NNPFS, Prev: Porting user-space, Up: Porting
1599 1. It helps to have source code for your operating system.
1601 In theory, if stuff was documented well enough, you wouldn't need
1602 it. In practice it never is, so you find out interfaces specs and
1603 how stuff works by reading the source code. If you're unable to
1604 find source code for your OS, try finding source for the closest
1605 match. If your OS is based on BSD, try the appropriate version of
1608 2. If you don't have source, try second best, include files.
1610 You can usually gather quite a lot of information on the workings
1611 of the kernel by reading the includes files in `<sys/*.h>'.
1615 Try to find out what other NNPFS port is most similar to your OS and
1616 start with that code.
1618 4. Figure out how your kernel works.
1620 You need to figure out how a few things work in your kernel:
1622 1. Loading/unloading kernel modules
1624 That varies quite a lot but it's probably easy to figure out
1625 if you have the source code for some other loadable module.
1626 Sometimes you can get the kernel to add your cdev, system
1627 call and file system automatically but usually you have to
1628 write code in your `entry-point' to add these to the
1631 2. Adding a new character device driver
1633 The kernel has a table of all known device drivers, ordered
1634 by major number. Some kernels have one for block devices and
1635 one for character devices and some have a common one. That
1636 entry usually consists of a number of function pointers that
1637 perform the operations (open, close, read, write, ...), and
1638 possible a name and some flags. It could look something like
1647 struct cdevsw cdevsw[];
1649 These are then usually stored in a table `cdevsw' indexed by
1650 the major device number. If you're really lucky there's a new
1651 way to get the kernel to add your `struct cdevsw' to the
1652 global table when loading the module or a function that does
1653 the addition for you. Otherwise there might be functions for
1654 adding/removing devices to the global table. If not, you'll
1655 have to fallback on looking for a free slot in the table and
1656 putting your struct cdevsw there. In some cases, this is not
1657 stored in a table but then there'll be a way of adding
1658 entries to the new data structure so you don't need to worry
1661 3. Adding a new system call
1663 This is quite similar to adding a new cdev but the table is
1664 usually called `sysent' instead.
1666 4. Adding a new file system
1668 Once again, quite similar in principle. The names of the
1669 structures tend to vary quite a lot more.
1671 5. Finding out how the VFS/Vnode switch works
1673 The structure vfsops contains function pointers for all of
1674 the file system operations. You need to figure out what
1675 operations you need to implement (usually at least mount,
1676 unmount, root, sync, and statfs).
1678 The operations that are performed on files are vnode
1679 operations (usually stored in a struct vnodeops), and you
1680 need to figure which of these you need and how they should
1681 work. Also, which is not as explicit, how vnodes are
1682 supposed to be allocated and freed and such.
1685 5. Suggested plan of action
1687 1. Start by writing a minimal hello-world module and make sure
1688 you can load and unload it properly.
1690 2. Then add a device driver to the module which dummy functions
1691 and verify that works.
1693 3. Try to fit the device driver functions in `nnpfs_dev.c' into the
1696 4. Do a dummy module with a system call and verify that you can
1699 5. Start trying to add enough of the vfs/vnode operations from
1700 `nnpfs_vfsops.c' and `nnpfs_vnodeops.c' so that you can build it.
1709 File: arla.info, Node: Programming, Next: Oddities, Prev: Porting, Up: Top
1714 This chapter is programming documentation of arla's internal parts.
1716 This is just to cover the ideas of the implemation, but documentation
1717 of the actual code, please see commends in the source code for that.
1721 * Arla pioctl's:: Documentation of arla pioctl's
1722 * Disco with arla:: Disconnected mode
1723 * afsUUID:: afs UUID
1726 File: arla.info, Node: Arla pioctl's, Next: Disco with arla, Prev: Programming, Up: Programming
1731 This chaper documents arla pioctl's. Arla's pioct are in the `A' range
1732 of the pioctls. OpenAFS uses `O' and orignal Transarc uses `V'.
1738 1. `opcode STATISTICS_OPCODE_LIST'
1740 Get a specific host/partition entry.
1742 2. `opcode STATISTICS_OPCODE_GETENTRY'
1744 Get a specific entry. The 5 values that are passed in are:
1745 opcode (STATISTICS_OPCODE_GETENTRY), ipv4-host-address,
1746 partition, type-of-data, slot in the histogram.
1748 Possible types of data are:
1750 1. `STATISTICS_REQTYPE_FETCHSTATUS'
1752 2. `STATISTICS_REQTYPE_FETCHDATA'
1754 3. `STATISTICS_REQTYPE_BULKSTATUS'
1756 4. `STATISTICS_REQTYPE_STOREDATA'
1758 5. `STATISTICS_REQTYPE_STORESTATUS'
1764 Return a pts name to id for a cell, the cell is referenced by the
1765 file that is passed into the request, or by name.
1771 File: arla.info, Node: Disco with arla, Next: afsUUID, Prev: Arla pioctl's, Up: Programming
1776 Disconnected operation of arla.
1781 The binary log is written to a file. All entries are of variable size.
1782 A entry is never removed from the log *note Log entry optimization::.
1787 One log entry consists of a log header with magic cookie, opcode,
1788 checksum, and length. The entry is is always 4 byte aligned in the
1789 logfile. The checksum is a simple one, its just to verify that for
1790 data corruption hasn't occured and to make sure the entry is a valid
1793 Disconnected vs connected nodes
1794 -------------------------------
1796 A FCacheNode is either a "disconnected node" or a "connected node". The
1797 later means that the node was created when arla was in connected mode
1798 and thus (potentially) exist on the file server.
1800 A disconnected node on the other hand was created when is disconnected
1801 operation. A disconnected node always have one or more entries in the
1807 The offset of the entry, a unsigned 32 bit number, is called
1808 "disco_id". Its stored in the FCacheNode so it can be updated when
1809 there is a operation on this node. All nodes for a FCacheEntry are
1810 single linked list (from the newest log entry to the oldest), the
1811 optimizer uses this to modify all previous entries from a FCacheNode.
1813 A FCacheNode with disco_id equal to 0 is a connected node that there
1814 have been no disconnected operation made on.
1816 The first entry in the log is a nop entry to make sure not a log-offset
1817 that is for a "real" entry.
1819 The limit of number of entries in the log are 2^32 / the size of the
1820 largest entry since the offset is a 32 bit number.
1822 Log entry optimization
1823 ======================
1825 To try to preserve bandwith when reinterating there are dependencies
1826 between entries. First we try to avoid storing data to the fileserver
1827 that was never meant to got there. For example a file that was created
1828 and then removed in disconnected mode, ie `DISCO_HEADER_NOP' is set in
1829 the flags field in the header.
1834 When a node is removed and the node is a disconnected node, the all
1835 previous entries are optizmied away by setting a flags in their entry
1836 headers. This make this entry/nodes be ignored by the reintergration
1837 code and never commited to the fileserver.
1839 Moveing a disconnected node
1840 ---------------------------
1842 If a disconnected node is moved, it will be created in the target
1843 directory instead of first being created and then moved.
1845 Storestatus and storedata
1846 -------------------------
1848 Also, all entries for one node storestate/storestatus are compressed to
1849 one createnode (and if needed storedata).
1857 fid - VenusFid equivalent storestatus - AFSStoreStatus equivalent
1861 needs to be smaller or equal in size then the rest
1863 data storage: header flags fid
1867 (truncation is a storedata)
1869 data storage: header fid storestatus size
1873 data storage: header fid storestatus
1877 data storage: header parentfid fid storestatus
1882 data storage: header parentfid fid storestatus
1883 name[AFSNAMEMAX] targetname[AFSNAMEMAX]
1887 data storage: header parentfid fid storestatus
1888 name[AFSNAMEMAX] targetfid
1892 data storage: header parentfid fid /* dup info */
1897 data storage: header parentfid fid /* dup info */
1898 storestatus name[AFSNAMEMAX]
1902 data storage: header parentfid fid /* dup info */
1912 1. make sure first entry in the log is a nop entry
1914 2. if nop entry or `DISCO_HEADER_NOP' is set, continue to next
1916 3. the parent fid is transformed to a connected fid (if needed) it
1917 this failes, we are unhappy and save this node for collision
1920 4. the fid is transformed to a connected fid (if needed) it this
1921 failes, we are unhappy and save this node for collision recovery
1923 5. operation is performed
1925 6. if there is change on a fid
1927 1. update kernelhandle
1929 2. update fcachehandle
1931 3. update directory fid for this name (if needed)
1933 4. transformed fids are stored in the transformation table
1938 File: arla.info, Node: afsUUID, Prev: Disco with arla, Up: Programming
1943 `AFS' uses a `DCE UUID' (Microsoft GUID is a another version of a DCE
1944 UUID) is identify file servers and callback managers. The callback
1945 manager regenerate its `uuid' every time it restarts.
1947 The fileserver stores its `uuid' on disk and uses it to identify it
1948 self when registering its addresses with the VLDB-server.
1950 The `afsUUID' is special in some implementations since it uses the
1951 ip-address as the node identifier instead of the mac-address (IEEE OUI
1952 + node address). Also the time in DCE is based on Unix epoch instead
1953 of DCE epoch. This will cause problems in about 287 years when the old
1954 generated afs UUID till start to collide with new genrated DCE UUIDs.
1955 Hopefully people have a solution to this problem then.
1958 File: arla.info, Node: Oddities, Next: Themis, Prev: Programming, Up: Top
1966 * Directories - UnixModeBits are ignored when the vnode is a
1969 * Errnos are sent over the network. Like Solaris ENOTEMPTY(93)
1970 doesn't even map to an error on sunos4 where ENOTEMPTY is 66.
1972 * Mountpoints have the mode-bits 0644, if they don't they are
1973 symlinks (and have the mode-bits 0755).
1979 * On Irix 6.5 you have to build the dirents depending on what ABI
1980 of the binary you are currently running.
1982 * . and .. need to be first in directories, this is needed since some
1983 programs (like make) "knows" that the two first entries are . and
1984 .. and thus can be skiped.
1986 * Reclen (in struct dirent) shouldn't be too large. When its larger
1987 then the buffer used in opendir/readdir/closedir, you loose.
1991 File: arla.info, Node: Themis, Next: Arla timeline, Prev: Oddities, Up: Top
1996 `Themis' is a tool for keeping local file systems up-to-date with a
1997 file system description.
1999 All lines in the input file consist of white-space separated fields. For
2000 all directives except the `exclude' directive consist of one uppercase
2001 letter, and flags can be added directly after the directive letter.
2002 Flags can be specified in any order, and consist of uppercase letters.
2004 If owner, group or mode is not specified, implicit values are used.
2005 The default implicit values are `root' for owner, `wheel' for group,
2006 `755' for directory mode bits and `644' for file mode bits.
2008 These are the directives:
2010 * F<flags> <destination> <source> [<owner> <group>] [<mode>]
2012 Indicates the presence of a file.
2014 * D<flags> <destination> [<owner> <group>] [<mode>]
2016 Indicates the presence of a directory.
2018 * T<flags> <destination> <source> [<owner> <group>] [<mode>]
2020 Indicates the presence of a directory tree.
2022 * L<flags> <destination> <link data>
2024 Indicates the presence of a symbolic link.
2026 * N<flags> <destination>
2028 Indicates that themis should ignore that file/directory. If the
2029 file exists, it is left alone.
2031 * X<flags> <destination>
2033 Indicates that themis should remove the file/directory.
2035 * M<flags> <destination> [<owner> <group>] [<mode>]
2037 Modifies the flags, owner, group and mode of the (already
2038 specified) destination.
2042 In a tree traversal, excludes all files that match <pattern>.
2043 Normal shell-style globbing can be used. Default exclution
2044 patterns that are included are: `.' and `..'. Also, when running
2045 with -macosx flag `._*' files are excluded.
2047 Common pattern to start to exclude are `*~', `RCS', `.OldFiles',
2048 `*,v', and `*.core'.
2051 These are the flags:
2053 * `A' The source path is absolute. When this flag is not specified,
2054 the destination path is appended to the source path.
2056 * `I' The file should not be overwritten if it exists.
2058 * `O' Rename existing files to .old, instead of overwriting them.
2060 * `Q' Exit with error level 4 to indicate that the file has changed.
2061 This can be used to check if a reboot is needed when certain
2062 critical files change.
2064 * `R' Only valid on directories (`D' and `T' directives). Specifies
2065 that the directory should be cleaned of any files that are not in
2066 the description file.
2068 * `P' Replaces the implicit owner, group and mode bits with the
2069 values from the source file.
2072 Normally, the input file should be pre-processed by mpp, m4, or some
2073 other pre-processor, to enable conditional processing and include files.
2076 File: arla.info, Node: Arla timeline, Next: Authors, Prev: Themis, Up: Top
2081 Arla have existed for quite some years.
2083 Development started in fall 1993 by Björn Grönvall <bg@nada.kth.se>
2084 (with an rxkad implantation), he had a working read-only implementation
2085 in winter 1994. Quick followers was Assar <assar@sics.se> (at that time
2086 <assar@pdc.kth.se>>) and Johan Danielsson <<joda@pdc.kth.se>>. The
2087 platform that was chosen was Sparc SunOS4 (the OS that NADA, KTH was
2090 Some work was being done by Patrik Stymne <patriks@e.kth.se> in porting
2091 arla to Ultrix, but this work was never finished.
2093 At this time there was no free rx, lwp or rxkad. A basic rx
2094 implementation was written, and the threading problem was solved by
2097 The Arla development started to slow down around 11 April 1995.
2099 In about Mar-Jun 1996 rx and lwp was released by Transarc, this was made
2100 possible by Jim Doyle <jrd@bu.edu>, and Derrick J. Brashear
2101 <shadow@dementia.org>.
2103 In September 1997, an rxkad implementation was written by Björn. At the
2104 same time, a need for an AFS client for OpenBSD rose at the Stacken,
2105 the local computer club at KTH. Other free OS:es, as NetBSD, FreeBSD
2106 and Linux(primarily sparc) were also in need of AFS clients.
2108 In TokKOM, a local communications system using LysKOM
2109 (<http://www.lysator.liu.se/lyskom/>), Assar suggested to some club
2110 members that it would be a nice thing to resume the arla development.
2112 Some people suggested that it would be less trouble having someone with
2113 access to the Transarc AFS source code port the code to the relevent
2114 platforms. Assar then ported nnpfs to FreeBSD 2.2.x in notime (over the
2115 night), just to show the high portability.
2117 People started to understand that arla was a concept that would work,
2118 and first out was Love Hörnqvist-Åstrand <lha@stacken.kth.se> to join.
2119 Development was primarily aimed at OpenBSD and NetBSD at the moment,
2120 and Arla lived for at least 2-3 weeks in /var/tmp on a host named
2121 yakko.stacken.kth.se.
2123 Magnus Ahltorp <map@stacken.kth.se> joined shortly thereafter, spending
2124 the rest of the year reading about the Linux VFS, and after a while,
2125 Artur Grabowski <art@stacken.kth.se> also started to work on arla,
2126 concentrating on OpenBSD kernel stuff.
2128 The first entry in ChangeLog is dated Fri Oct 24 17:20:40 1997. Around
2129 this time arla was given a CVS tree, to ease development. Now you could
2130 also mount the nnpfs-device and get the root-directory out of it.
2132 The Linux port was done in a few weeks in the beginning of 1998. Only
2133 the Linux 2.0 kernel was supported at this time.
2135 In April 1998 Assar hade a Arla paper presented at Freenix. Linux 2.1
2136 support was written also written around this time. This was a major
2137 work since there was a lot of stuff that had changed (namely the
2140 The first milko entry is dated Thu Oct 30 01:46:51 1997. Note that this
2141 milko in a sense "worked". You could get files out from it and store
2144 There was from this point a lot of work being done and quite a lot of
2145 studies was "wasted". We learned a lot, but not the stuff we were
2148 We added support for `dynroot' and `fake-mp' to prepare for Windows and
2149 Darwin/MacOSX support.
2151 In Mars 2000 preliminary support for MacOS X/Darwin 1.0 was merged in
2152 by Magnus and Assar.
2154 Around the same time there we hacked in support for Solaris 8 (beta2)
2155 There was also some work being done on Windows 2000 native driver at
2158 In June 2000 there was a presentation on MADE2000 in Gothenburg, Sweden.
2160 In September 2000 MacOS X Beta was working.
2162 This just includes some milestones, for more information se Changelog.*
2163 and NEWS files in the distribution.
2166 File: arla.info, Node: Authors, Next: Acknowledgments, Prev: Arla timeline, Up: Top
2171 Currently writing on arla are
2173 Assar Westerlund, Everything
2175 Magnus Ahltorp, Everything
2177 Artur Grabowski, BSD nnpfs, OpenBSD maintainer
2179 Love Hörnquist-Åstrand, Everything
2181 Robert Burgess, fs, Solaris nnpfs
2183 Johan Danielsson, OSF/1 nnpfs
2185 Hans Insulander, pts, OpenBSD maintainer
2187 Mattias Amnefelt, vos, milko
2193 Mikael Vidstedt (userland, some milko stuff)
2195 Jimmy Engelbercht (bos)
2196 Rhapsody nnpfs port was contributed by Alexandra Ellwood <lxs@MIT.EDU>
2197 Later, Rhapsody was renamed Darwin.
2199 Disconnected code is written by:
2201 WUWEI SHEN <wwshen@engin.umich.edu>
2203 Cheng Jin <chengjin@eecs.umich.edu>
2205 Paul Callaway <pcallawa@umich.edu>
2206 For contributors, see *Note Acknowledgments::.
2209 File: arla.info, Node: Acknowledgments, Next: Index, Prev: Authors, Up: Top
2214 lwp and rx are copyrighted by IBM. We're grateful to Derrick J
2215 Brashear <shadow@dementia.org> and Jim Doyle <jrd@bu.edu> for making
2218 the `rxkad' implementation was written by Björn Grönvall <bg@sics.se>
2219 and is also part of the kth-krb distribution.
2221 Some of the files in `libroken' come from Berkeley by the way of
2224 `editline' was written by Simmule Turner and Rich Salz.
2226 The code for gluing these together were written by ourselves.
2228 Bugfixes, documentation, encouragement, and code has been contributed
2231 Aaron M. Ucko <amu@MIT.EDU>
2232 Alec Wolman <wolman@cs.washington.edu>
2233 Alexandra Ellwood <lxs@MIT.EDU>
2234 Brad Keryan <keryan@andrew.cmu.edu>
2235 Constantine Sapuntzakis <csapuntz@openbsd.org>
2236 Dan Winship <danw@MIT.EDU>
2237 Derrick J Brashear <shadow@dementia.org>
2238 Harald Barth <haba@pdc.kth.se>
2239 Jim Doyle <jrd@bu.edu>
2240 John Davison <davisoja@clarkson.edu>
2241 John Hawkinson <jhawk@MIT.EDU>
2242 Karl Ramm <kcr@MIT.EDU>
2243 Mark Eichin <eichin@kitten.gen.ma.us>
2244 Per Boussard T/ED <Per.Boussard@era-t.ericsson.se>
2245 Dima Ruban <dima@best.net>
2246 Max <davros@cyclone.Stanford.EDU>
2247 Andrzej Filinski <andrzej@daimi.aau.dk>
2248 Chuck Lever <chuckl@netscape.com>
2249 WUWEI SHEN <wwshen@engin.umich.edu>
2250 Cheng Jin <chengjin@eecs.umich.edu>
2251 Paul Callaway <pcallawa@umich.edu>
2252 Brandon S. Allbery <allbery@ece.cmu.edu>
2253 Ken Raeburn <raeburn@raeburn.org>
2254 Jeffrey Hutzelman <jhutz+@cmu.edu>
2255 Hee-Seok Heo <hsheo@postech.ac.kr>
2256 Paul Ewing Jr. <ewing@ima.umn.edu>
2257 Niklas Hallqvist <niklas@appli.se>
2258 Marko Asplund <aspa@hip.fi>
2259 Chris Wing <wingc@engin.umich.edu>
2260 Simon Josefsson <jas@pdc.kth.se>
2261 Magnus Lindström <magnus.lindstrom@s3.kth.se>
2262 Greg Stark <gsstark@mit.edu>
2263 Matt <deberg@mit.edu>
2264 Björn Grönvall <bg@sics.se>
2265 Tino Schwarze <tino.schwarze@informatik.tu-chemnitz.de>
2266 David Sanderson <david@transarc.ibm.com>
2267 Roman Hodek <Roman.Hodek@informatik.uni-erlangen.de>
2268 Michael Sperber <sperber@informatik.uni-tuebingen.de>
2269 Dr. Lex Wennmacher <wennmach@geo.Uni-Koeln.DE>
2270 Janne Johansson <jj@dynarc.se>
2271 Dr A V Le Blanc <LeBlanc@mcc.ac.uk>
2272 Dave Morrison <dave@bnl.gov>
2273 Jochen Saile <saile@sfs.nphil.uni-tuebingen.de>
2274 Chris Kuklewicz <chrisk@MIT.EDU>
2275 Nickolai Zeldovich <kolya@MIT.EDU>
2276 Adam Thornton <adam@sinenomine.net>
2277 Neale Ferguson <Neale.Ferguson@SoftwareAG-usa.com>
2278 Hidvegi <hzoli@austin.ibm.com>
2279 Todd T. Fries <todd@fries.net>
2280 Andrea Campi <andrea@webcom.it>
2281 William Uther <will+@cs.cmu.edu>
2283 If you have done something and are not mentioned here, please send mail
2284 to <arla-drinkers@stacken.kth.se>.
2286 If you are mentioned here and have not contributed, that's because we
2290 File: arla.info, Node: Index, Prev: Acknowledgments, Up: Top
2297 * adb: Kernel debuggers.
2298 * AFS Filespace: AFS infrastructure.
2299 * AFSDB: AFS infrastructure.
2300 * afslog: Kerberos tickets and AFS tokens.
2302 * aklog: Kerberos tickets and AFS tokens.
2304 * Archives: Introduction.
2305 * Arlad debugging: Arlad.
2306 * Backup server: AFS infrastructure.
2307 * Bos server: AFS infrastructure.
2308 * Bug reports: Introduction.
2309 * Buserver: AFS infrastructure.
2310 * Cell: AFS infrastructure.
2311 * CellServDB: AFS infrastructure.
2312 * Comments: Introduction.
2313 * DCE UUID: afsUUID.
2314 * Debugging: Debugging.
2315 * Debugging arlad: Arlad.
2316 * Debugging techniques: Debugging techniques.
2317 * Debugging NNPFS: nnpfs.
2318 * Disconected operation: Disco with arla.
2319 * DNS: AFS infrastructure.
2320 * Fileserver: AFS infrastructure.
2321 * Filespace: AFS infrastructure.
2322 * Fsserver: AFS infrastructure.
2323 * Gdb: Debugging LWP with GDB.
2324 * kalog: Kerberos tickets and AFS tokens.
2325 * kaserver: Integration with Kerberos.
2326 * kauth: Kerberos tickets and AFS tokens.
2327 * kerberosserver: Integration with Kerberos.
2328 * Kernel debugging: Kernel debuggers.
2329 * Kernel debuging on linux: Kernel debuggers.
2330 * KeyFile: Integration with Kerberos.
2331 * kinit: Kerberos tickets and AFS tokens.
2332 * klog: Kerberos tickets and AFS tokens.
2333 * Ksymoops: Kernel debuggers.
2334 * Linux kernel debugging: Kernel debuggers.
2335 * Live kernel: Kernel debuggers.
2336 * Mail archives: Introduction.
2337 * Mailing list: Introduction.
2338 * Masquerading: NAT.
2341 * pts: Relationship between pts uid and unix uid.
2342 * Ptserver: AFS infrastructure.
2343 * Salvage: AFS infrastructure.
2346 * TheseCells: Integration with Kerberos.
2347 * ThisCell: Integration with Kerberos.
2348 * Tickets: Kerberos tickets and AFS tokens.
2349 * Tokens: Kerberos tickets and AFS tokens.
2350 * Ubik: AFS infrastructure.
2351 * uid: Relationship between pts uid and unix uid.
2352 * Upserver: AFS infrastructure.
2353 * Vldbserver: AFS infrastructure.
2354 * Vlserver: AFS infrastructure.
2355 * Volser: AFS infrastructure.
2356 * Volumeserver: AFS infrastructure.
2357 * NNPFS debugging: nnpfs.
2363 Node: Introduction
\x7f2110
2364 Node: AFS infrastructure
\x7f5546
2365 Node: Organization of data
\x7f15964
2366 Node: Requirements
\x7f16460
2367 Node: Data organization
\x7f18431
2368 Node: Callbacks
\x7f20761
2369 Node: Volume management
\x7f21446
2370 Node: Relationship between pts uid and unix uid
\x7f22789
2371 Node: AFS and the real world
\x7f23658
2373 Node: Samba
\x7f26081
2374 Node: Integration with Kerberos
\x7f26732
2375 Node: Kerberos tickets and AFS tokens
\x7f28564
2376 Node: Parts of Arla
\x7f34115
2377 Node: How arla works
\x7f34567
2378 Node: The relation between Arlad and NNPFS
\x7f35586
2379 Node: The life of a file
\x7f36245
2380 Node: Tools and libs
\x7f38767
2381 Node: Rx protocol
\x7f39982
2383 Node: The files in arlad/
\x7f40671
2384 Node: pioctl and kafs
\x7f42796
2385 Node: Debugging
\x7f44895
2386 Node: Arlad
\x7f45371
2387 Node: Debugging LWP with GDB
\x7f46614
2388 Node: nnpfs
\x7f48369
2389 Node: nnpfs on linux
\x7f49016
2390 Node: Debugging techniques
\x7f49361
2391 Node: Kernel debuggers
\x7f50293
2392 Node: Darwin/MacOS X
\x7f54706
2393 Node: Porting
\x7f55323
2394 Node: Porting user-space
\x7f55934
2395 Node: Porting NNPFS
\x7f57697
2396 Node: Programming
\x7f62046
2397 Node: Arla pioctl's
\x7f62481
2398 Node: Disco with arla
\x7f63589
2399 Ref: Log entry optimization
\x7f65373
2400 Node: afsUUID
\x7f68205
2401 Node: Oddities
\x7f69023
2402 Node: Themis
\x7f69929
2403 Node: Arla timeline
\x7f72653
2404 Node: Authors
\x7f76325
2405 Node: Acknowledgments
\x7f77145
2406 Node: Index
\x7f81013