added; Do lots of parallel volume lookups, exercising volcache

[arla.git] / doc / arla.info

This is arla.info, produced by makeinfo version 4.0 from arla.texi.

INFO-DIR-SECTION Arla
START-INFO-DIR-ENTRY
* Arla: (arla).           Arla - A Free AFS Implementation
END-INFO-DIR-ENTRY

\x1f
File: arla.info,  Node: Top,  Prev: (dir),  Up: (dir)

Arla
****

Arla is a free AFS implementation from KTH.

Please send comments (and fixes) on this manual and the arla programs to
<arla-drinkers@stacken.kth.se>.

* Menu:

* Introduction::                Introduction to Arla.
* AFS infrastructure::          A description of the afs infrastructure.
* Organization of data::        How different data is organized in AFS.
* AFS and the real world::      Common problems and their solutions.
* Parts of Arla::               Description of different parts of arla.
* Debugging::                   How to debug arla when its not working.
* Porting::                     That you need to know to port arla.
* Programming::                 Programming documentation of arla.
* Oddities::                    Strange things that did happen to us.
* Themis::                      Tool to keep your files system up-to-date.
* Arla timeline::               Short timeline of arla.
* Authors::                     The authors of arla.
* Acknowledgments::             People that have helped us.
* Index::                       Index.


 --- The Detailed Node Listing ---

* Introduction::
* AFS infrastructure::

How data and servers are organized in AFS.

* Requirements::
* Data organization::
* Callbacks::
* Volume management::
* Relationship between pts uid and unix uid::

How to cope with reality

* NAT::
* Samba::
* Integration with Kerberos::
* Kerberos tickets and AFS tokens::

The parts of arla

* How arla works::
* The relation between Arlad and NNPFS::
* The life of a file::
* Tools and libs::
* The files in arlad/::
* pioctl and kafs::

How to debug arla when its not working

* Arlad::
* Debugging LWP with GDB::
* nnpfs::
* nnpfs on linux::
* Debugging techniques::
* Kernel debuggers::
* Darwin/MacOS X::

Porting arla

* Porting::
* Porting user-space::
* Porting NNPFS::

Programming

* Programming::
* Disco with arla::
* afsUUID::

Odd stuff you find when looking around

* Oddities::

Miscellaneous

* Arla timeline::
* Authors::
* Acknowledgments::

* Index::

\x1f
File: arla.info,  Node: Introduction,  Next: AFS infrastructure,  Prev: Top,  Up: Top

Introduction
************

     *Caution:* Parts of this package are not yet stable software.  If
     something doesn't work, it's probably because it doesn't. If you
     don't have backup of your data, take backup.

What is Arla?
=============

Arla is a free AFS implementation. Some of the goals are:

   * to have an implementation that is free and can be used for adding
     and playing with cool stuff, like support for disconnected-mode.
     Implement features you can't get from commercial AFS.

   * to provide an alternative to Transarc's and OpenAFS AFS-clients and
     server implementations.

   * to add support for platfroms that don't have AFS support from
     Transarc or OpenAFS today.

This release is known to work on the following platforms: NetBSD,
OpenBSD, FreeBSD, Linux, Solaris, Darwin/MacOS X.

Earlier releases are known to work on current or earlier versions of the
following platforms: SunOS, AIX, IRIX, Digital UNIX. Some fixes might
be necessary to make Arla work.

There is or has been done work to support the following platforms: HPUX,
Fujitsu UXP/V. Some development is necessary to make Arla work.

There is work going on to support the following platform: Windows
NT/2000. Contributions are very welcome.

Status
======

Arla has the following features (quality varies between stable and not
implemented):

   * a rxgen implementation called ydr (stable).

   * a cache manager replacing Transarc's afsd. The cache managers
     quality depends on platform: *BSD, Linux i386 and Solaris are
     stable, others platforms are not as tested ane therefore not as
     stable.

   * partly implemented fs, vos, pts commands. Commands typically issued
     by users are stable, commands issued by administrators may return
     unmotivated errors or are not implemented yet.

   * an implementaion of rxkad written outside USA without any export
     restrictions (stable).

   * a server implementation called milko, containing file server,
     volume server and protection server. The file server has an API to
     the storage backend(s). Milko is still unstable and not fit for
     production yet.

Bug reports
===========

If you find bugs in this software, make sure it is a genuine bug and not
just a part of the code that isn't implemented.

Bug reports should be sent to <arla-drinkers@stacken.kth.se>. Please
include information on what machine and operating system (including
version) you are running, what you are trying to do, what happens, what
you think should have happened, an example for us to repeat, the output
you get when trying the example, and a patch for the problem if you have
one. Please make any patches with `diff -u' or `diff -c'.

Suggestions, comments and other non bug reports are also welcome.

Mailing list
============

There are two mailing lists with talk about Arla.
<arla-announce@stacken.kth.se> is a low-volume announcement list, while
<arla-drinkers@stacken.kth.se> is for general discussion.

There is also commit list <arla-commit@stacken.kth.se>.  Send a message
to <LIST-request@stacken.kth.se> to subscribe.

The list are achived on <http://www.stacken.kth.se/lists/>.

Please note that the mailinglists have a policy of subscriber only
posting. So if you want to mail a message to the list, subscribe to it
first, otherwise it wont be posted on the list.

\x1f
File: arla.info,  Node: AFS infrastructure,  Next: Organization of data,  Prev: Introduction,  Up: Top

AFS infrastructure
******************

This is an overview of the AFS infrastructure as viewed from a Transarc
perspective, since most people still run Transarc cells.

AFS Filespace
=============

AFS filespace is split up in smaller parts called cells. These cells are
usually listed under `/afs'. A cell is usually a whole organization or
an adminstative unit within an organization. An example is e.kth.se
(with the path `/afs/e.kth.se'), that is the department of electrical
engineering at KTH, which obviously has the `e.kth.se' domain in DNS.
Using DNS domains for cell names is the typical and most convenient way.

Note that cell names are written in lowercase by convention.

CellServDB
==========

All cells (and their db-servers) in the AFS world are listed in a file
named `CellServDB'. There is a central copy that is maintained by
Transarc at `/afs/transarc.com/service/etc/CellServDB'.

In spite of being organized in IPnumber - name pairs, where the name
parts resemble comments, both values are used by Transarc software and
confusion may arise if they are not synchronized with each other.


     >e.kth.se                  # Royal Institute of Technology, Elektro
     130.237.48.8                       #sonen.e.kth.se.
     130.237.48.7                    #anden.e.kth.se.
     130.237.48.244                  #fadern.e.kth.se.

Again, please note that the text after the # in the cell-name is a
comment, *but* the hostnames after the # on the rows of an IP-address
is *not* a comment. The host and the ip-address needs to point at the
same computer.

AFSDB
=====

In addition Arla can use DNS to find the db-servers of a cell. The DNS
resource record that is used is the `AFSDB'. The resourcerecord was
created by Transarc but have never been implemeted in released software.

`AFSDB' tells you what machines are db servers for a particular cell.
The `AFSDB' resourcerecord is also used for DCE/DFS.  An example (the 1
means AFS, 2 is used for DCE):

     e.kth.se.               IN AFSDB     1 fadern.e.kth.se.
     e.kth.se.               IN AFSDB     1 sonen.e.kth.se.
     e.kth.se.               IN AFSDB     1 anden.e.kth.se.

Shortcut names
==============

Some cells use the abbreviated version `/afs/<word-before-first-dot>'
(in the example above that would be `/afs/e/'.  This might be
convenient when typing them, but is a bad idea, because it does not
create the same name space everywhere.  If you create a symbolic link
to `/afs/e/foo/bar', it will not work for people in other cells.

Server organization
===================

There are several servers running in an AFS cell. For performance and
redundancy reasons, these servers are often run on different hosts.
There is a built in hierarchy within the servers (in two different
dimensions).

There is one server that keeps track of the other servers within a host,
restart them when they die, make sure they run in the correct order,
save their core-files when they crash, and provide an interface for the
sysadmin to start/stop/restart the servers. This server is called
bos-server (Basic Overseer Server).

Another hierarchy is the one who keeps track of data (volumes, users,
passwords, etc) and who is performing the real hard work (serving files)
There is the the database server that keeps the database (obviously),
and keeps several database copies on different hosts relpicated with
Ubik (see below). The fileserver and the client software (like the
afsd/arlad, pts and, vos) are pulling meta-data out of the dbserver to
find where to find user-privileges and where volumes resides.

Basic overseer - boserver
=========================

The Bos server is making sure the servers are running. If they crash, it
saves the corefile, and starts a new server. It also makes sure that
servers/services that are not supposted to run at the same time do not.
An example of this is the fileserver/volserver and salvage. It would be
devastating if salvage tried to correct data that the fileserver is
changing. The salvager is run before the fileserver starts. The
administrator can also force a file server to run through salvage again.

Ubik
====

Ubik is a distributed database. It is really a (distributed) flat file
that you can perform read/write/lseek operation on. The important
property of Ubik is that it provides a way to make sure that updates are
done once (transactions), and that the database is kept consistent. It
also provides read-only access to the database when there is one (or
more) available database-server(s).

This works the following way: A newly booted server sends out a message
to all other servers that tells them that it believes that it is the new
master server. If the server gets a notice back from an other server
that tells it that the other server believes that it (or a third server)
is the master, depending on how long it has been masterserver it will
switch to the new server. If they can't agree, the one with the lowest
ip-address is supposed to win the argument. If the server is a slave it
still updates the database to the current version of the database.

A update to the database can only be done if more than half of the
servers are available and vote for the master. A update is first
propaged to all servers, then after that is done, and if all servers
agree with the change, a commit message is sent out from the server, and
the update is written to disk and the serial number of the database is
increased.

All servers in AFS use Ubik to store their data.

Volume Location database server - vlserver
==========================================

The vldb-server is resposible for the information on what fileserver
every volume resides and of what kind of volumes exists on each
fileserver.

To confuse you even more there are three types of support for the
clients. Basically there is AFS 3.3, 3.4, and 3.6 support. The different
interfaces look the same for the system administrator, but there are
some important differences.

AFS 3.3 is the classic interface. 3.4 adds the possibility of multihomed
servers for the client to talk to, and that introduces the N interface.
To deal with multihomed clients AFS 3.5 was introduced. This is called
call the U interface. The name is due to how the functions are named.

The N interface added more replication-sites in the database-entry
structure. The U interface changed the server and clients in two ways.

When a 3.5 server boot it registers all its ip-addresses. This means
that a server can add (or remove) an network interface without
rebooting. When registering at the vldb server, the file server presents
itself with an UUID, an unique identifier. This UUID will be stored in a
file so the UUID keeps constant even when network addresses are changed,
added, or removed.

Protection server - ptserver
============================

The protection server keeps track of all users and groups. It's used a
lot by the file servers. Users can self create, modify and delete
groups.

When a fileserver is access they are durring the authentication giving
the name of the client. This name if looked up in the
protection-database via the protection server that returns the id of the
user and all the groups that the user belongs too.

This information is used when to check if the user have access to a
particular file or directory. All files created by the user are assigned
the user id that the protectionserver returned.

Kerberos server - kaserver
==========================

The kaserver is a Kerberos server, but in other clothes. There is a new
RPC interface to get tickets (tokens) and administer the server.  The
old Kerberos v4 interface is also implemented, and can be used by
ordinary Kerberos v4 clients.

You can replace this server with an Heimdal kdc, since it provides a
superset of the functionality.

Backup server - buserver
========================

The backup server keeps the backup database that is used when backing up
and restoring volumes. The backup server is not used by other servers,
only operators.

Update server - upserver
========================

With the update server its possible to automagicly update configuration
files, server binaries.  You keep masters that are supposed to contain
the correct copy of all the files and then other servers can fetch them
from there.

Fileserver and Volume Server - fs and volser
============================================

The file server serves data to the clients, keeps track of callbacks,
and breaks callbacks when needed. Volser is the administative interface
where you add, move, change, and delete volumes from the server.

The volume server and file server are ran at the same time and they sync
with each other to make sure that fileserver does not access a volume
that volser is about to modify.

Every time a fileserver is started it registers it IP addresses with the
vldbserserver using the VL_RegisterAddrs rpc-call. As the unique
identifier for itself it uses its afsUUID.

The afsUUID for a fileserver is stored in /usr/afs/local/sysid. This is
the reson you must not clone a server w/o removing the sysid file.
Otherwise the new filserver will register as the old one and all
volumes on the old fileserver are pointed to the new one (where the
probably doesn't exist).

The fileserver doesn't bind to a specific interface (read address), gets
all packets that are destined for port 7000 (afs-fileserver/udp). All
outgoing packets are send on the same socket, and means that your
operatingsystem will choose the source-address of the udp datagram.

This have the side-effect that you will have asymmetric routing on
mulithomed fileserver for 3.4 (and older) compatible clients if they
don't use the closest address when sorting the vldb entry. Arla avoids
this problem.

Salvage
=======

Salvage is not a real server. It is run before the fileserver and volser
are started to make sure the partitions are consistent.

It's imperative that salvager is NOT run at the same time as the
fileserver/volser is running.

Things that milko does differently.
===================================

Fileserver, volumeserver, and salvage are all in one program.

There is no bu nor ka-server. The ka-server is replaced by kth-krb or
Heimdal. Heimdal's kdc even implements a ka-server readonly interface,
so your users can keep using programs like klog.

\x1f
File: arla.info,  Node: Organization of data,  Next: AFS and the real world,  Prev: AFS infrastructure,  Up: Top

Organization of data
********************

This chapter describes how data is stored and how AFS is different from,
for example, NFS. It also describes how data is kept consistent and what
the requirements were and how that inpacted on the design.

* Menu:

* Requirements::
* Data organization::
* Callbacks::
* Volume management::
* Relationship between pts uid and unix uid::

\x1f
File: arla.info,  Node: Requirements,  Next: Data organization,  Prev: Organization of data,  Up: Organization of data

Requirements
============

   * Scalability

     It should be possible to use AFS with hundred-thousands of users
     without problems.

     Writes that are done to different parts of the filesystem should
     not affect each other. It should be possible to distribute out the
     reads and writes over many fileservers. If you have a file that is
     accessed by many clients, it should be possible to distribute the
     load.

   * Transparent to users

     Users should not need to know where their files are stored. It
     should be possible to move their files while they are using their
     files.

   * Easy to admin

     It should be easy for a administrator to make changes to the
     filesystem. For example to change quota for a user or project. It
     should also be possible to move the users data for a fileserver to
     a less loaded one, or one with more diskspace available.

     Some benefits of using AFS are:

        * user-transparent data migration

        * an ability for on-line backups;

        * data replication that provides both load balancing and
          robustness of critical data

        * global name space without automounters and other add-ons;

        * @sys variables for platform-independent paths to binary
          location;

        * enhanced security;

        * client-side caching;

Anti-requirements
=================

   * No databases

     AFS isn't constructed for storing databases. It would be possible
     to use AFS for storing a database if a layer above for locking and
     synchronizing data would be provided.

     One of the problems is that AFS doesn't include mandatory
     byte-range locks. AFS uses advisory locking on whole files.

     If you need a real database, use one, they are much more efficent
     on solving a database problem. Don't use AFS.


\x1f
File: arla.info,  Node: Data organization,  Next: Callbacks,  Prev: Requirements,  Up: Organization of data

Volume
======

A volume is a unit that is smaller than a partition. It is usually (or
should be) a well defined area, like a user's home directory, a project
work area, or a program distribution.

Quota is controlled on volume-level. All day-to-day management is done
on volumes.

Partition
=========

In AFS a partition is what normally is named a partition. All partions
that afs is using are named a special way, `/vicepNN', where NN is
ranged from a to z, continuing with aa to zz. The fileserver (and
volser) automaticly picks upp all partitions starting with `/vicep'

Volumes are stored in a partition. Volumes can't span several
partitions. Partitions are added when the fileserver is created or when
a new disk is added to a filesystem.

Volume cloning and read-only clones
===================================

A clone of a volume is often needed for volume operations. A clone is a
copy-on-write copy of a volume, the clone is the read-only version.

Two special versions of a clone are the read-only volume and the backup
volume. The read-only volume is a snapshot of a read-write volume (that
is what a clone is) that can be replicated to several fileservers to
distribute the load. Each fileserver plus partition where a read-only
clone is located is called a replication-site. It usually does not make
sense to have more than one read-only clone on each fileserver.

The backup volume is a clone that typically is made (with `vos
backupsys') each night to enable the user to retrieve yesterday's data
when they happen to remove a file. This is a very useful feature, since
it lessens the load on the system-administrators to restore files from
backup. The volume is usually mounted in the root user's home directory
under the name OldFiles. A special feature of the backup volume is that
you can't follow mountpoints out of a backup volume.

Mountpoints
===========

Volumes are independent of each other. To glue together the file tree
there are `mountpoint's. Mountpoints are really symlinks that are
formated in a special way so that they point out a volume and an
optional cell. An AFS-cache-manager will show a mountpoint as directory
and in fact it will be the root directory of the target volume.

\x1f
File: arla.info,  Node: Callbacks,  Next: Volume management,  Prev: Data organization,  Up: Organization of data

Callbacks
=========

Callbacks are messages that enable the AFS-cache-manager to keep the
files without asking the server if there is newer version of the file.

A callback is a promise from the fileserver that it will notify the
client if the file (or directory) changes within the timelimit of the
callback.

For contents of read-only volumes there is only one callback per volume
called a volume callback and it will be broken when the read-only volume
is updated.

The time range of callbacks is from 5 to 60 minutes depending on how
many users of the file exist.

\x1f
File: arla.info,  Node: Volume management,  Next: Relationship between pts uid and unix uid,  Prev: Callbacks,  Up: Organization of data

Volume management
=================

All volume managment is done with the `vos' command. To get a list of
all commands `vos help' can be used. For help on a specific vos
subcommand, `vos subcommand -h' can be used.

   * Create

          vos create mim c HO.staff.lha.fluff -quota 400000

   * Move

     Volumes can be moved from a server to another, even when users are
     using the volume.

   * Replicate

     Read-only volumes can be replicated over several servers, they are
     first added with `vos addsite', and the replicated with `vos
     release' over the servers.

   * Release

     When you want to distribute the changes in the readwrite volume to
     the read-only clones.

   * Remove

     Volumes can be removed

     Note that you shouldn't remove the last readonly volume since this
     makes clients misbehave. If you are moving the volume you should
     rather add a new RO to the new server and then remove it from the
     old server.

   * Backup and restoration of volumes.

     `vos backup' and `vos backupsys' creates the backup volume.

     To stream a volume out to a `file' or `stdout' you use `vos dump'.
     The opposite command is named `vos restore'.


\x1f
File: arla.info,  Node: Relationship between pts uid and unix uid,  Prev: Volume management,  Up: Organization of data

Relationship between pts uid and unix uid
=========================================

Files in AFS are created with the pts uid of the token that was valid at
the time. The pts uid number is then by commands like `ls -l'
interpreted as a unix uid and translated into a username. If the pts and
the unix uids differ, this might confuse the user as it looks like as
her files are owned by someone else. This is however not the case.
Complications can occur if programs impose further access restrictions
based on these wrongly interpreted uids instead of using the `access()'
system call for that purpose. Graphical file browsers are typically
prone to that problem with the effect that the users are not able to
see their own files in these tools.

\x1f
File: arla.info,  Node: AFS and the real world,  Next: Parts of Arla,  Prev: Organization of data,  Up: Top

AFS and the real world
**********************

This chapter tries to describe problems that you see in the real (not
that perfect) world and show possible solutions to these problems.

* Menu:

* NAT::                         Truly evil stuff.
* Samba::                       Export AFS to Windows computers.
* Integration with Kerberos::   How to integrate Kerberos with AFS.
* Kerberos tickets and AFS tokens:: History and tools

\x1f
File: arla.info,  Node: NAT,  Next: Samba,  Prev: AFS and the real world,  Up: AFS and the real world

NAT
===

There's something evil out there that's called NAT, which stands for
Network Address Translation. For whatever reasons, people are using it
and will continue doing so.

First of all, it seemed like AFS should work just fine through NAT, you
just appear to be coming from the NAT-address and some random port
instead.  Looking closer at different NAT implementations it seems like
they have a rather short timeout:

`FreeBSD natd'
     60 seconds

`Cisco IOS'
     300 seconds

`NetBSD ipf (ipnat)'
     600 seconds

`Linux Nat (masq)'
     300 seconds

If the client doesn't transmit any traffic to a particular host for that
amount of time, it will get mapped to one of the IP address of the
NAT-server (if you happen to run PAT, the port will be randomized too).

The authors of Rx realized that keeping a Rx connection associated with
(IP-address,port) pair was a bad idea. One example is that you get
problems with multi-homed hosts. So Rx keeps its own connection id data
in the packet. With this feature client and server should be able to
detect address changes.

Unfortunately, the use of the orignal Rx-code stops this from happening
in Transarc/OpenAFS code. The code keeps track of incoming packets and
keeps track of the right peer (client). But it never updates the
IP-address,port pair in its data structure, so the answer packet will go
to the old IP-address,port pair.

If you can control your NAT machine you can have static mapping for your
AFS hosts (Transarc/OpenAFS uses source port 7000 and Arla uses source
port 4711). You can try to use Natkeep
<http://mit.edu/fredette/www/natkeep/> if you run an old Arla or
Transarc/OpenAFS client. From version 0.36 arla will have support for
polling the servers at the right interval to prevent NAT from dropping
information about your session.

\x1f
File: arla.info,  Node: Samba,  Next: Integration with Kerberos,  Prev: NAT,  Up: AFS and the real world

Samba
=====

The major problem when exporting the AFS filespace read-write to SMB
(Windows fileshareing) using Samba is the transfer of the user token to
the smb-server. The simple may is to use clear-text password between the
Windows client and the samba-server, and then to get tokens for the user
with this password. This solution is clearly not acceptable for security
aware AFS administrators.

Describe here how to make AFS work "securely" with samba.

On solution is to use `kimpersonate' + store afs key on fileserver
(talk to Love).

\x1f
File: arla.info,  Node: Integration with Kerberos,  Next: Kerberos tickets and AFS tokens,  Prev: Samba,  Up: AFS and the real world

Integration with Kerberos
=========================

Kerberos 4 and 5 can be integrated quite well with AFS. This is mainly
due to the fact that the security model used in AFS is Kerberos. The
kaserver is a Kerberos 4 server with pre-authentication. The kaserver
also provides a feature that limits the number of password retries, and
after that you are locked out for half an hour. This feature can only be
used in the ka interface as it requires pre-authentication, but since
the kaserver provides a Kerberos 4 interface (without pre-authentication
and without this limitation) it is quite worthless.

Many sites indeed use a kerberosserver instead of a kaserver. One of
the reasons is that they want to use Kerberos 5 (which is required for
Windows 2000).

To use a kerberosserver, you have to put the same key into the AFS
KeyFile and the principal named afs of your kerberos realm. If you have
a cell which has another name than lowercase of your realmname, the
instance is the cellname. As the cellname often contains dots, this can
be quite confusing in Kerberos 4: afs.stacken.kth.se@STACKEN.KTH.SE. The
first dot is the seperator between principal and instance, the other
dots are part of the name. The simplest way to do create a KeyFile on
your AFS server is to use heimdal's kadmin to generate and extract a
KeyFile. It has an option designed to do so. Be aware that the serial
number of the afs principal must match on all kerberos servers and all
KeyFiles on all AFS servers.

The default cellname to get a kerberos ticket for is contained in the
configuration file `ThisCell'. If you got more than one cell, you want
to list all cells to get tickets for in the `TheseCells' file.

\x1f
File: arla.info,  Node: Kerberos tickets and AFS tokens,  Prev: Integration with Kerberos,  Up: AFS and the real world

Kerberos tickets and AFS tokens
===============================

To further confuse the poor user, AFS and Kerberos programmers decided
that they wanted to store their credentials at different places. In AFS,
the kernel was a natural place to store the credentials (named token)
since the CMU/Transarc AFS/OpenAFS implementation lives in the kernel.
The Kerberos people on the other hand thought that storing the
credentials (named ticket) in a file would be a good idea.

So know you have to synchronize the credentials if you just want to
enter your password once. There are several tools that can do that for
you. The question is what tools to use for what problem.

To add to the confusion not all tools talk to both Kerberos and
kaservers. There is also a bogus user-id in the token that is supposed
to be the same as your pts-user-id. Not that it makes any difference,
but some people get confused when unknown numbers show up in the token.
The easily confused people are often the ones that have more than one
principal in the same realm/cell (read sysadmins).

If you want to get your ticket from your Kerberos server, you use
`kinit', and then use `afslog' or `aklog' to get AFS tokens and push
them to the kernel (and AFS daemon). Some `kinit' (and `kauth') can do
both for you, use `kinit --afslog' or simply `kauth'. Note that `kinit'
and `kauth' don't get set your AFS-token user-id right, and thus can be
confusing for people who think that this is important.

The `klog' program that you get with Transarc/OpenAFS talks to the
kaserver and behaves just-right in the sense that it talks to the pts
server to get the AFS-token user-id right, but `klog' talks only to the
kaserver which will not work for people with a Kerberos server.  `Klog'
in Arla was written by Chris Wing <wingc@engin.umich.edu> as a part of
a packet called `afsutils', they did the right thing and talked to the
pts-server to get the user-id. However, it uses Kerberos libs to talk
to the server. These libraries require the files `/etc/krb.conf' and
`/etc/krb.realms' to be setup correctly for their cell/realm. Not that
easy.

A long time ago Ken Hornstein <kenh@cmf.nrl.navy.mil> wrote the AFS
Migration Kit that helped you to migrate from AFS to MIT Kerberos 5.
It included a tool named aklog that could convert a Kerberos tickets to
tokens. This tool was also rewritten in Arla by Brandon S. Allbery
<allbery@ece.cmu.edu>. `aklog' can't get you new credentials, it just
converts old ones to new ones.

Then Transarc decided that they needed to fix a security hole in their
kaserver, and while doing that, they managed to break a part in the
kaserver so it ceased to work for kerberos requests.

First the defect existed unnoticed for a long time, later Transarc has
not managed to distribute a working version of the kaserver. Due to
this, a lot of sites run a kaserver with this defect. Instead of
installing working authentification servers from another sources,
people started to whine again and Love <lha@stacken.kth.se> wrote the
tool `kalog' that talked the ka-protocol (but didn't do the AFS user-id
right) to work around that problem.

All tools that use Kerberos 4 need a working `/etc/krb.conf' and
`/etc/krb.realms'. Kerberos 5 programs need `/etc/krb5.conf'.  AFS
aware tools need `/usr/arla/etc/CellServDB' or
`/usr/vice/etc/CellServDB'.

Also the Kerberos implementations from KTH (kth-krb and Heimdal) include
AFS support to make your life more pleasant. One thing is that you can
have a file `$HOME/.TheseCells' that lists the cells you use and the
Kerberos tools will try to get tickes and tokens for those cells.
Heimdal contains support for converting a Kerberos 4 srvtab to an AFS
KeyFile.

Below is a table that describes what tools does what, what
inheritance(s) they have, and what protocol(s) they speak. From the
inheritance (also in a table below) it is possible to deduct what
configuration files the tools use.

Tool                    Inheritance    Protocol   Produces
Transarc/OpenAFS klog   afs authlib    KA         Ticket and tokens
Arla klog               Kerberos and   Kerberos   Ticket and tokens
                        libkafs                   
AFS Migration kit's     MIT Kerberos   Kerberos   Converts Kerberos
aklog                   and Ken                   tickets to tokens
                        Hornstein's               
                        afslib                    
Arla's aklog            Kerberos and   Kerberos   Converts Kerberos
                        libkafs                   tickets to tokens
kth-krb's and           Kerberos and   Kerberos   Converts Kerberos
Heimdal's afslog        libkafs                   tickets to tokens
kalog                   arla and       KA         Get initial
                        libkafs                   ticket, store
                                                  tokens and tickets

Inheritance table

`afs authlib'
     Reads `CellServDB' and `ThisCell' in `/usr/vice/etc'

`Kerberos and libkafs'
     Kerberos 4 uses `/etc/krb.conf' and `/etc/krb.realms'. Kerberos 5
     uses `/etc/krb5.conf'.

`arla'
     Arla uses `CellServDB' and `ThisCell' in `/usr/vice/etc' and
     `/usr/arla/etc'

`libkafs'
     Libkafs uses DNS and `CellServDB' in `/usr/vice/etc' and
     `/usr/arla/etc' to figure out what ticket it should convert into
     tables. The file `TheseCells' and `$HOME/.TheseCells' is used to
     get extra tokens.

\x1f
File: arla.info,  Node: Parts of Arla,  Next: Debugging,  Prev: AFS and the real world,  Up: Top

Parts of Arla
*************

     *Caution:* This text just tries to give a general picture.  For
     real info read the code. If you have any questions, mail
     <arla-drinkers@stacken.kth.se>.

* Menu:

* How arla works::
* The relation between Arlad and NNPFS::
* The life of a file::
* Tools and libs::
* The files in arlad/::
* pioctl and kafs::

\x1f
File: arla.info,  Node: How arla works,  Next: The relation between Arlad and NNPFS,  Prev: Parts of Arla,  Up: Parts of Arla

How does arla work
==================

Arla consists of two parts, a userland process (arlad) and the
kernel-module (nnpfs).

Arlad is written in user-space for simpler debugging (and less
rebooting).  As a uset space program arlad does not have the same
limitations as if it would be written in the kernel. To avoid
performance loss as much as possible, nnpfs is caching data.

nnpfs and arlad communicate with each other via a char-device-driver.
There is a rpc-protocol currenly used specially written for this
(`arlad/message.c')

nnpfs is written to be as simple as possible. Theoretically, nnpfs could be
used by other user-space daemons to implement a file system. Some
parts, such as syscalls, are arla-specific. These parts are designed to
be as general as possible.

For example, nnpfs does not recognize which pioctl the user-level program
calls, it just passes this information on to arlad.

\x1f
File: arla.info,  Node: The relation between Arlad and NNPFS,  Next: The life of a file,  Prev: How arla works,  Up: Parts of Arla

The relation between Arlad and NNPFS
==================================

     Userland
     
                                  ---------
                 Edit file        | Arlad |  ------> Network
                    |             ---------
                ----|-----------------|[1]----
                 -------           -------
     Kernel      | VFS | <--[2]--> | NNPFS |
                 -------           -------

[1] A char device (/dev/nnpfs0)

[2] Xfs provides a filesystem for the vfs-layer in
     the operating system.

\x1f
File: arla.info,  Node: The life of a file,  Next: Tools and libs,  Prev: The relation between Arlad and NNPFS,  Up: Parts of Arla

The life of a file
==================

Step by step description of what happens during the creation of a file.
The names are inspired of BSD-style VFS-layer but the idea is the same
in most operating systems.

   * The user decides to open a file.

   * open(2) syscall is issued.

   * The vfslayer sends a VOP_LOOKUP to nnpfs that is forwarded to arlad
     with a getnode() (seq-num 1).

   * arlad tries to find the requested file and then, if found, sends an
     install_node to nnpfs by writing to the nnpfs character device.

   * nnpfs inserts the node into the cache and returns from the device
     write.

   * arlad sends a wakeup rpc message (seq-num 1) to nnpfs.  If the
     return value is zero nnpfs tries to find the node in the cache, if
     not found it might have been flushed out of the cache and the whole
     thing is repeated.

   * If a none-zero return value is returned, this value is sent as
     reply to the user. This way arla can decide what error message is
     returned, without nnpfs having support for each error.

   * nnpfs now checks if it has the valid attributes. If the attributes
     are invalid, nnpfs will send a rpc message to arlad to refresh it.

   * Since the user wanted to open the file, a getdata rpc message is
     sent from nnpfs to arlad. Now arlad fetches the files from the afs
     file server.

   * Arlad stores the file in the file cache. All vnode operations will
     be done on this file. Now arlad sends a installdata to nnpfs.

   * When nnpfs recives the installdata it looks up the node in the cache,
     and then it does a VOP_LOOKUP to find a vnode to the cachefile
     (and store it to keep it for future use).

   * The same thing is done when the file is a directory, except that
     the directory is converted from the afs directory format to an
     operating system dependent format and stored in a file. nnpfs reads
     this file instead.

   * If the directory is modified locally, write operations are done on
     the file obtained from the afs-server, and when done the newly
     changed file is converted and reinstalled.

   * Now the user wants to read a file.

   * read(2) system call is issued.

   * A VOP_READ is sent to the from the vfs-layer to nnpfs.

   * nnpfs checks if it has valid attributes/and data (and updates if
     needed).  Now VOP_READ is simply performed on the stored vnode of
     the cachefile.


\x1f
File: arla.info,  Node: Tools and libs,  Next: The files in arlad/,  Prev: The life of a file,  Up: Parts of Arla

Tools and libs
==============

What other tools does the arla suite consists of

libutil: `util/libutil.a' - A library for the most often used
     modules like hashtable, double-linked list, logging functions,
     date-parsing, etc

rx: `rx/librx.a' - The library for the rx protocol
     (*note Rx protocol::).

lwp: `lwp/liblwp.a' - The library for the lwp thread-package
     (*note LWP::).

ydr: `ydr/ydr' - A stub generator that replaces rxgen.

rxkad: `rxkad/librxkad.a' - The rx Kerberos authentication package.

roken: `lib/roken/libroken.a' - The library that will unbreak
     things that are missing or broken.

ko: `lib/ko/libko.a' - A library of functions that are arlad-core
     related but also are useful for programs like vos, pts, fs, etc.

arlalib: `appl/lib/libarlalib.a' - A broken library that does all
     the hard work with connections etc.

fs: `appl/fs/fs' - The fs util, extra feature
     (amongst others): getfid.

vos: `appl/vos/vos' - The vos util.

pts: `appl/pts/pts' - The pts util, extra feature: dump.

udebug: `appl/udebug/udebug' - Debug your ubik server.
\x1f
File: arla.info,  Node: Rx protocol,  Next: LWP,  Up: Tools and libs

Rx protocol
===========

Rx is run over UDP.

One of rxgen or ydr is used to generate stub-files, ydr is better since
it generates prototypes, too.

The current implemetation of rx it not that beautiful.

\x1f
File: arla.info,  Node: LWP,  Prev: Rx protocol,  Up: Tools and libs

LWP
===

LWP is a preepmtive thread package. It does it's context-switching by
creating a private stack for each thread. The heart of the package is
select(2).

The stack is checked for overruns in context-switches, but that is often
too late. It might be an idea to add a `red zone' at the top of the
stack to be able to detect overruns.

\x1f
File: arla.info,  Node: The files in arlad/,  Next: pioctl and kafs,  Prev: Tools and libs,  Up: Parts of Arla

The files in arlad/
===================

This is a short describtion of the files to bring new deveplopers up to
speed.

The core of arlad
-----------------

`adir.c' - contains all functions needed to to operations
     on afs-directory files.

`afsdir_check.c' - check if an AFS-directory looks sane.

`arla.c' - The startup and the standalone (-t) code.

`arladeb.c' - The logging code specific to arla, like aliases
     for debugging masks.

`cmcb.c' - The callback-server that is contacted by the
     server when a callback expires or a server wants to send an
     InitCallBackState.

`conn.c' - The connection cache, responsible for caching connection
     based on pag and security index. It will also create new
     connection when needed.

`cred.c' - Keep track of all credentials that all users have
     inserted. Indexed on pag.

`fbuf.c' - An interface between rx and filedescriptors. It is also
     used to mmap files. Used by `adir.c'.

`fcache.c' - Responsible for keeping track of files in the cache.
     Also fetches files from the afs-server.

`fprio.c' - Tries to give files priority. These files are
     therefore not garbarge-collected as fast as they would be
     otherwise.  If you wonder what this is for, think of the
     disconnected mode.

`inter.c' - An interface to hide all junk in fcache, just give
     the items a VenusFid and you can access them this way.

`kernel.c' - The interface between arlad and the char-device.

`messages.c' - The rpc interface between arlad and nnpfs.

`volcache.c' - Cache for all volumes.
Operating system specific files
-------------------------------

These are the files that contain operating specific functions.  Today
it's just conv_dir().

`aix-subr.c' - AIX

`bsd-subr.c' - FreeBSD 2.2.6, OpenBSD 2.2, 2.3, NetBSD 1.3.x

`hpux-subr.c' - HPUX

`irix-subr.c' - Irix

`linux-subr.c' - Linux 2.0.x, 2.1.x, 2.2

`solaris-subr.c' - Solaris 2.5.x, 2.6, 7

`sunos-subr.c' - SunOS

`unknown-subr.c' - Stub used when compiled on a unknown OS.
\x1f
File: arla.info,  Node: pioctl and kafs,  Prev: The files in arlad/,  Up: Parts of Arla

pioctl and kafs
===============

The pioctl interface is the only part of nnpfs that is afs related.

pioctl is a ioctl but called with a path instead of a filedescriptor.
When you probe if there is a live afsclient you first run `k_hasafs()'
that probes if there is an afsclient around.  It also sets up some
static variables in the library. So if you start to do `pioctl()' w/o
running `k_hasafs()', you're up to funny errors, and/or get a corefile.

`k_hasafs()' does an `AFSCALL_PIOCTL' with opcode `VIOCSETTOK' and
insize == 0, ie you try to set a token (ticket) that is 0 bytes long.
This is cleary invalid and kafs expects to find an `EINVAL' returned
from `syscall(2)'.

The pioctl is used more then just for `AFSCALL_PIOCTL', an other use is
`AFSCALL_SETPAG' (setting pag). It has also been in use for setting nnpfs
debugging levels.

When nnpfs discovers that a path is given in the `pioctl()' it does a
`VOP_LOOKUP' on the path and if the returned value is a vnode that
resides in afs then it extracts the nnpfs-handle for that node (that just
happens to be the VenusFid) and passes that on to arlad.

The only ugly thing about the current implentation is that the syscall
code assumes that the arlad on "nnpfs-fd" is the arlad that should get
this syscall.

An example of using `pioctl()':

     int
     fs_getfilecellname(char *path, char *cell, size_t len)
     {
         struct ViceIoctl a_params;
     
         a_params.in_size=0;
         a_params.out_size=len;
         a_params.in=NULL;
         a_params.out=cell;
     
         if (k_pioctl(path,ARLA_VIOC_FILE_CELL_NAME,&a_params,1) == -1)
             return errno;
     
         return 0;
     }
     
     int
     main (int argc, char **argv)
     {
         char cell[100];
     
         if (!k_hasafs())
            errx (1, "there is no afs");
     
         if (fs_getfilecellname (".", cell, sizeof(cell)))
            errx (1, "fs_getfilecellname failed");
     
         printf ("cell for `.' is %s", cell);
         return 0;
     }

\x1f
File: arla.info,  Node: Debugging,  Next: Arlad,  Prev: Parts of Arla,  Up: Top

Debugging
*********

This chapter of the manual includes tips that are useful when debugging
arla.

Arla and nnpfs contains logging facilities that is quite useful when
debugging when something goes wrong. This and some kernel debugging tips
are described.

* Menu:

* Arlad::
* Debugging LWP with GDB::
* nnpfs::
* nnpfs on linux::
* Debugging techniques::
* Kernel debuggers::
* Darwin/MacOS X::

\x1f
File: arla.info,  Node: Arlad,  Next: Debugging LWP with GDB,  Prev: Debugging,  Up: Debugging

Arlad
=====

If arlad is run without any arguments arlad will fork(2) and log to
syslog(3). To disable forking use the -no-fork (-n) switch. In the
current state of the code, arlad is allways to be started with the
recover (-z) switch. This will invalidate your cache at startup.  This
restriction may be dropped in the future.

To enable more debuggning run arla with the switch
-debug=module1,module2,...  One useful combination is
        --debug=all,-cleaner
The cleaner output is usully not that intresting and can be ignored.

A convenient way to debug arlad is to start it inside gdb.
     datan:~# gdb /usr/arla/libexec/arlad
     (gdb) run -z -n
This gives you the opportunity to examine a crashed arlad.
     (gdb) bt
The arla team appreciates cut and paste information from the beginning
to the end of the bt output from such a gdb run.

To set the debugging with a running arlad use `fs arladeb' as root.

     datan:~# fs arladeb
     arladebug is: none
     datan:~# fs arladeb almost-all
     datan:~#

By default, arlad logs through syslog if running as a daemon and to
stderr when running in the foreground (with `--no-fork').

\x1f
File: arla.info,  Node: Debugging LWP with GDB,  Next: nnpfs,  Prev: Arlad,  Up: Debugging

Debugging LWP with GDB
======================

For easy tracing of threads we have patch
(<http://www.stacken.kth.se/projekt/arla/gdb-4.18-backfrom.diff>) for
gdb 4.18 (a new command) and a gdb sequence (think script).

The sequence only works for i386, but its just matter of choosing
different offset in topstack to find $fp and $pc in the lwp_ps_internal
part of the sequence.

You should copy the `.gdbinit' (that you can find in the arlad
directory in the source-code) to your home-directory, the directory from
where you startat the patched gdb or use flag -x to gdb.

Your debugging session might look like this:

     (gdb) lwp_ps
     Runnable[0]
      name: IO MANAGER
       eventlist:
       fp: 0x806aac4
       pc: 0x806aac4
      name: producer
       eventlist: 8048b00
       fp: 0x8083b40
       pc: 0x8083b40
     Runnable[1]
     [...]
     (gdb) help backfrom
     Print backtrace of FRAMEPOINTER and PROGRAMCOUNTER.
     
     (gdb) backfrom 0x8083b40 0x8083b40
     #0  0x8083b40 in ?? ()
     #1  0x8049e2f in LWP_MwaitProcess (wcount=1, evlist=0x8083b70)
         at /afs/e.kth.se/home/staff/lha/src/cvs/arla-foo/lwp/lwp.c:567
     #2  0x8049eaf in LWP_WaitProcess (event=0x8048b00)
         at /afs/e.kth.se/home/staff/lha/src/cvs/arla-foo/lwp/lwp.c:585
     #3  0x8048b12 in Producer (foo=0x0)
         at /afs/e.kth.se/home/staff/lha/src/cvs/arla-foo/lwp/testlwp.c:76
     #4  0x804a00c in Create_Process_Part2 ()
         at /afs/e.kth.se/home/staff/lha/src/cvs/arla-foo/lwp/lwp.c:629
     #5  0xfffefdfc in ?? ()
     #6  0x8051980 in ?? ()

There also the possibility to run arla with pthreads (run configure with
-with-pthreads).

\x1f
File: arla.info,  Node: nnpfs,  Next: nnpfs on linux,  Prev: Debugging LWP with GDB,  Up: Debugging

nnpfs
===

NNPFS debugging does almost look the same on all platforms. They all
share same debugging flags, but not all are enabled on all platforms.

Change the debugging with the `fs nnpfsdebug' command.

     datan:~# fs nnpfsdebug
     nnpfsdebug is: none
     datan:~# fs nnpfsdebug almost-all
     datan:~#

If it crashes before you have an opportunity to set the debug level, you
will have to edit `nnpfs/YOUR-OS/nnpfs_deb.c' and recompile.

The logging of nnpfs ends up in your syslog. Syslog usully logs to
/var/log or /var/adm (look in /etc/syslog.conf).

\x1f
File: arla.info,  Node: nnpfs on linux,  Next: Debugging techniques,  Prev: nnpfs,  Up: Debugging

nnpfs on linux
============

There is a problem with klogd, it's too slow. Cat the `/proc/kmsg' file
instead.  Remember to kill klogd, since the reader will delete the text
from the ring-bufer, and you will only get some of the message in your
cat.

\x1f
File: arla.info,  Node: Debugging techniques,  Next: Kernel debuggers,  Prev: nnpfs on linux,  Up: Debugging

Debugging techniques
====================

Kernel debugging can sometimes force you to exercise your imagination.
We have learned some different techniques that can be useful.

Signals
-------

On operatingsystems with kernel debugger that you can use probably find
where in the kernel a user-program live, and thus deadlocks or trigger
the bad event, that later will result in a bug. This is a problem, how
do you some a process to find where it did the intresting thing when you
can't set a kernel breakpoint ?

One way to be notified is to send a signal from the kernel module
(psignal() on a BSD and force_sig() on linux). SIGABRT() is quite useful
if you want to force a coredump. If you want to continue debugging, use
SIGSTOP.

Recreateable testcases
----------------------

Make sure bugs don't get reintroduced.

\x1f
File: arla.info,  Node: Kernel debuggers,  Next: Darwin/MacOS X,  Prev: Debugging techniques,  Up: Debugging

Kernel debuggers
================

Kernel debuggers are the most useful tool when you are trying to figure
out what's wrong with nnpfs. Unfortunately they also seem to have their
own life and do not always behave as expected.

Using GDB
---------

Kernel debugging on NetBSD, OpenBSD, FreeBSD and Darwin are almost the
same.  You get the idea from the NetBSD example below:

       (gdb) file netbsd.N
       (gdb) target kcore netbsd.N.core
       (gdb) symbol-file /sys/arch/i386/compile/NUTCRACKER/netbsd.gdb

This example loads the kernel symbols into gdb. But this doesn't show
the nnpfs symbols, and that makes your life harder.

Getting all symbols loaded at the same time
-------------------------------------------

If you want to use the symbols of nnpfs, there is a gdb command called
`add-symbol-file' that is useful. The symbol file is obtained by
loading the kernel module nnpfs with `kmodload -o /tmp/nnpfs-sym' (Darwin)
or `modload' (NetBSD and OpenBSD).  FreeBSD has a linker in the kernel
that does the linking instead of relying on `ld'. The symbol address
where the module is loaded get be gotten from `modstat', `kldstat' or
`kmodstat' (it's in the `area' field).

If you forgot the to run modstat/kldstat/kmodstat, you can extract the
information from the kernel. In Darwin you look at the variable kmod
(you might have to case it to a (kmod_info_t *). We have seen gdb loose
the debugging info). kmod is the start of a linked list. Other BSDs have
some variant of this.

You should also source the commands in /sys/gdbscripts (NetBSD), or
System/xnu/osfmk/.gdbinit (Darwin) to get commands like ps inside gdb.

       datan:~# modstat Type Id Off Loadaddr Size Info Rev Module
       Name DEV 0 29 ce37f000 002c ce388fc0 1 nnpfs_mod [...]
       [...]
       (gdb) add-symbol-table nnpfs.sym ce37f000

Debugging processes, examine their stack, etc
---------------------------------------------

One of diffrencies between the BSD's are the `proc', a command that
enables you do to a backtrace on all processes. On FreeBSD you give the
`proc' command a `pid', but on NetBSD and OpenBSD you give a pointer to
a `struct proc'.

After you have ran `proc' to set the current process, you can examine
the backtrace with the regular `backtrace' command.

Darwin does't have a `proc' command. Instead you are supposed to use
gdb sequences (System/xnu/osfmk/.gdbinit) to print process stacks,
threads, activations, and other information.

Debugging Linux
---------------

You can't get a crashdump for linux with patching the kernel. There are
two projects have support for this. Mission Critical Linux
<http://www.missioncritiallinux.com> and SGI <http://oss.sgi.com/>.

Remember save the context of /proc/ksyms before you crash, since this is
needed to figure out where the nnpfs symbols are located in the kernel.

But you can still use the debugger (or objdump) to figure out where in
the binary that you crashed.

`ksymoops' can be used to create a backtrace.

Using adb
---------

Adb is not a symbolic debugger, this means that you have to read the
disassembled object-code to figure out where it made the wrong turn and
died. You might be able to use GNU objdump to list the assembler and
source-code intertwined (`objdump -S -d mod_nnpfs.o'). Remember that GNU
binutils for sparc-v9 isn't that good yet.

You can find the script that use use for the adb command `$<' in
`/usr/lib/adb' and `/usr/platform/PLATFORNAME/adb'.

Debugging a live kernel
-----------------------

An important thing to know is that you can debug a live kernel too, this
can be useful to find dead-locks. To attach to a kernel you use a
command like this on a BSD  system (that is using gdb):

       (gdb) file /netbsd
       (gdb) target kcore /dev/mem
       (gdb) symbol-file /sys/arch/i386/compile/NUTCRACKER/netbsd.gdb

And on Solaris:

       # adb -k /dev/ksyms /dev/mem

Other useful debugging tools
----------------------------

Most diagnosics tools like ps, dmesg, and pstat on BSD systems used to
look in kernel memory to extract information (and thus earned the name
kmem-groovlers). On some systems they have been replaced with other
method of getting their data, like /proc and sysctl.

But due to their heritage they can still be used in with a kernel and
coredump to extract information on some system.

\x1f
File: arla.info,  Node: Darwin/MacOS X,  Next: Porting,  Prev: Kernel debuggers,  Up: Debugging

Darwin/MacOS X
==============

You'll need two computers to debug arlad/nnpfs on darwin since the common
way to debug is to use a remote kernel-debugger over IP/UDP.

First you need to publish the arp-address of the computer that you are
going to crash.

We have not found any kernel symbols in MacOSX Public Beta, so you
should probably build your own kernel. Use Darwin xnu kernel source
with cvs-tag: Apple-103-0-1 (not xnu-103).

     gdb nnpfs.out
     target remote-kdp
     add-symbol-table ...
     attach <host>

\x1f
File: arla.info,  Node: Porting,  Next: Programming,  Prev: Darwin/MacOS X,  Up: Top

Porting
*******

The largest part of the work needed to port Arla to a new operating
system is in porting nnpfs, as kernel programming always is harder, less
portable and messier than user-space dito.  Arla in test mode
(`arla-cli') should work without any porting on any system that's not
very far away from Unix and that provides berkeley sockets (including
cygwin32).  The hard part is porting the NNPFS kernel module, and we will
spent most of this text on how to do that.

* Menu:

* Porting user-space::
* Porting NNPFS::

\x1f
File: arla.info,  Node: Porting user-space,  Next: Porting NNPFS,  Prev: Porting,  Up: Porting

user-space
==========

The user-space parts should work on basically any system that is
reasonably Posix and has berkeley sockets.  The build uses autoconf and
should adapt itself to most forseeable circumstances.  If it fails to
consider something that is missing or not working on the particular OS
you are porting to, hard-code it to make sure that is what is missing
and then try to create an autoconf test for it.  If you fail to do so,
or have no autoconf experience, send us the patches anyway and tell us
where you are having the problem.

LWP
---

The only thing that might take a little bit more effort in porting is
the context-switch in the LWP user-level threads package.  There are
assembler versions for most of the common architectures in `lwp'.  Part
of the problem is getting this code assembled properly.  There is
unfortunately no easy and portable way of preprocessing and assembling
code.  There is a script `lwp/make-process.o.sh' that tries to do in
some different ways, but it may fail for you.  Next problem is that
assembler syntax can vary a lot even on the same CPU.  The source files
are written in such a way that they should be acceptable to almost any
syntax, but if it fails you have to find out what particular syntax has
to be used and adapt the source file for that.

The more interesting problem is if there is no support for your CPU.
The first thing to try then is the `--with-pthreads' option that uses
the pthreads library.  If that fails or you want LWP working you have
to figure out enough details on your CPU to write two functions in
assembler, `savecontext' and `returnto' that save and restore the
processor context.

\x1f
File: arla.info,  Node: Porting NNPFS,  Prev: Porting user-space,  Up: Porting

NNPFS
===

  1. It helps to have source code for your operating system.

     In theory, if stuff was documented well enough, you wouldn't need
     it.  In practice it never is, so you find out interfaces specs and
     how stuff works by reading the source code.  If you're unable to
     find source code for your OS, try finding source for the closest
     match.  If your OS is based on BSD, try the appropriate version of
     BSD, for example.

  2. If you don't have source, try second best, include files.

     You can usually gather quite a lot of information on the workings
     of the kernel by reading the includes files in `<sys/*.h>'.

  3. Be lazy

     Try to find out what other NNPFS port is most similar to your OS and
     start with that code.

  4. Figure out how your kernel works.

     You need to figure out how a few things work in your kernel:

       1. Loading/unloading kernel modules

          That varies quite a lot but it's probably easy to figure out
          if you have the source code for some other loadable module.
          Sometimes you can get the kernel to add your cdev, system
          call and file system automatically but usually you have to
          write code in your `entry-point' to add these to the
          appropriate tables.

       2. Adding a new character device driver

          The kernel has a table of all known device drivers, ordered
          by major number.  Some kernels have one for block devices and
          one for character devices and some have a common one.  That
          entry usually consists of a number of function pointers that
          perform the operations (open, close, read, write, ...), and
          possible a name and some flags.  It could look something like
          the following:

               struct cdevsw {
                int (*d_open)();
                int (*d_close)();
                ...
               };
               
               struct cdevsw cdevsw[];

          These are then usually stored in a table `cdevsw' indexed by
          the major device number. If you're really lucky there's a new
          way to get the kernel to add your `struct cdevsw' to the
          global table when loading the module or a function that does
          the addition for you.  Otherwise there might be functions for
          adding/removing devices to the global table.  If not, you'll
          have to fallback on looking for a free slot in the table and
          putting your struct cdevsw there. In some cases, this is not
          stored in a table but then there'll be a way of adding
          entries to the new data structure so you don't need to worry
          about it.

       3. Adding a new system call

          This is quite similar to adding a new cdev but the table is
          usually called `sysent' instead.

       4. Adding a new file system

          Once again, quite similar in principle. The names of the
          structures tend to vary quite a lot more.

       5. Finding out how the VFS/Vnode switch works

          The structure vfsops contains function pointers for all of
          the file system operations.  You need to figure out what
          operations you need to implement (usually at least mount,
          unmount, root, sync, and statfs).

          The operations that are performed on files are vnode
          operations (usually stored in a struct vnodeops), and you
          need to figure which of these you need and how they should
          work.  Also, which is not as explicit, how vnodes are
          supposed to be allocated and freed and such.


  5. Suggested plan of action

       1. Start by writing a minimal hello-world module and make sure
          you can load and unload it properly.

       2. Then add a device driver to the module which dummy functions
          and verify that works.

       3. Try to fit the device driver functions in `nnpfs_dev.c' into the
          device driver.

       4. Do a dummy module with a system call and verify that you can
          call it.

       5. Start trying to add enough of the vfs/vnode operations from
          `nnpfs_vfsops.c' and `nnpfs_vnodeops.c' so that you can build it.

       6. Debug it.

       7. Send us patches



\x1f
File: arla.info,  Node: Programming,  Next: Oddities,  Prev: Porting,  Up: Top

Programming
***********

This chapter is programming documentation of arla's internal parts.

This is just to cover the ideas of the implemation, but documentation
of the actual code, please see commends in the source code for that.

* Menu:

* Arla pioctl's::       Documentation of arla pioctl's
* Disco with arla::     Disconnected mode
* afsUUID::             afs UUID

\x1f
File: arla.info,  Node: Arla pioctl's,  Next: Disco with arla,  Prev: Programming,  Up: Programming

Arla pioctl's
*************

This chaper documents arla pioctl's. Arla's pioct are in the `A' range
of the pioctls. OpenAFS uses `O' and orignal Transarc uses `V'.

     `AIOC_STATISTICS'

     XXX max constants

       1. `opcode STATISTICS_OPCODE_LIST'

          Get a specific host/partition entry.

       2. `opcode STATISTICS_OPCODE_GETENTRY'

          Get a specific entry. The 5 values that are passed in are:
          opcode (STATISTICS_OPCODE_GETENTRY), ipv4-host-address,
          partition, type-of-data, slot in the histogram.

          Possible types of data are:

            1. `STATISTICS_REQTYPE_FETCHSTATUS'

            2. `STATISTICS_REQTYPE_FETCHDATA'

            3. `STATISTICS_REQTYPE_BULKSTATUS'

            4. `STATISTICS_REQTYPE_STOREDATA'

            5. `STATISTICS_REQTYPE_STORESTATUS'



     `AIOC_PTSNAMETOID'

     Return a pts name to id for a cell, the cell is referenced by the
     file that is passed into the request, or by name.

     XXX document format


\x1f
File: arla.info,  Node: Disco with arla,  Next: afsUUID,  Prev: Arla pioctl's,  Up: Programming

Disco with arla
***************

Disconnected operation of arla.

The Log
=======

The binary log is written to a file. All entries are of variable size.
A entry is never removed from the log *note Log entry optimization::.

One log entry
-------------

One log entry consists of a log header with magic cookie, opcode,
checksum, and length. The entry is is always 4 byte aligned in the
logfile.  The checksum is a simple one, its just to verify that for
data corruption hasn't occured and to make sure the entry is a valid
one.

Disconnected vs connected nodes
-------------------------------

A FCacheNode is either a "disconnected node" or a "connected node". The
later means that the node was created when arla was in connected mode
and thus (potentially) exist on the file server.

A disconnected node on the other hand was created when is disconnected
operation. A disconnected node always have one or more entries in the
log.

Log entry offset
----------------

The offset of the entry, a unsigned 32 bit number, is called
"disco_id". Its stored in the FCacheNode so it can be updated when
there is a operation on this node. All nodes for a FCacheEntry are
single linked list (from the newest log entry to the oldest), the
optimizer uses this to modify all previous entries from a FCacheNode.

A FCacheNode with disco_id equal to 0 is a connected node that there
have been no disconnected operation made on.

The first entry in the log is a nop entry to make sure not a log-offset
that is for a "real" entry.

The limit of number of entries in the log are 2^32 / the size of the
largest entry since the offset is a 32 bit number.

Log entry optimization
======================

To try to preserve bandwith when reinterating there are dependencies
between entries. First we try to avoid storing data to the fileserver
that was never meant to got there. For example a file that was created
and then removed in disconnected mode, ie `DISCO_HEADER_NOP' is set in
the flags field in the header.

Removal operations
------------------

When a node is removed and the node is a disconnected node, the all
previous entries are optizmied away by setting a flags in their entry
headers. This make this entry/nodes be ignored by the reintergration
code and never commited to the fileserver.

Moveing a disconnected node
---------------------------

If a disconnected node is moved, it will be created in the target
directory instead of first being created and then moved.

Storestatus and storedata
-------------------------

Also, all entries for one node storestate/storestatus are compressed to
one createnode (and if needed storedata).

data storage
============

common data types
-----------------

fid - VenusFid equivalent   storestatus - AFSStoreStatus equivalent

  1. `nop'

     needs to be smaller or equal in size then the rest

     data storage:     header     flags     fid

  2. `storedata'

     (truncation is a storedata)

     data storage:     header     fid     storestatus     size

  3. `storestatus'

     data storage:     header     fid     storestatus

  4. `createfile'

     data storage:     header     parentfid     fid     storestatus
     name[AFSNAMEMAX]

  5. `createsymlink'

     data storage:     header     parentfid     fid     storestatus
     name[AFSNAMEMAX]     targetname[AFSNAMEMAX]

  6. `createlink'

     data storage:     header     parentfid     fid     storestatus
     name[AFSNAMEMAX]     targetfid

  7. `unlink'

     data storage:     header     parentfid     fid /* dup info */
     name[AFSNAMEMAX]

  8. `createdir'

     data storage:     header     parentfid     fid /* dup info */
     storestatus     name[AFSNAMEMAX]

  9. `removedir'

     data storage:     header     parentfid     fid /* dup info */
     name[AFSNAMEMAX]


reintegration
=============

Cook-book
---------

  1.  make sure first entry in the log is a nop entry

  2.  if nop entry or `DISCO_HEADER_NOP' is set, continue to next

  3.  the parent fid is transformed to a connected fid (if needed)   it
     this failes, we are unhappy and save this node for collision
     recovery

  4.  the fid is transformed to a connected fid (if needed)   it this
     failes, we are unhappy and save this node for collision   recovery

  5.  operation is performed

  6.  if there is change on a fid

       1.  update kernelhandle

       2.  update fcachehandle

       3.  update directory fid for this name (if needed)

       4.  transformed fids are stored in the transformation table



\x1f
File: arla.info,  Node: afsUUID,  Prev: Disco with arla,  Up: Programming

afsUUID
*******

`AFS' uses a `DCE UUID' (Microsoft GUID is a another version of a DCE
UUID) is identify file servers and callback managers. The callback
manager regenerate its `uuid' every time it restarts.

The fileserver stores its `uuid' on disk and uses it to identify it
self when registering its addresses with the VLDB-server.

The `afsUUID' is special in some implementations since it uses the
ip-address as the node identifier instead of the mac-address (IEEE OUI
+ node address).  Also the time in DCE is based on Unix epoch instead
of DCE epoch. This will cause problems in about 287 years when the old
generated afs UUID till start to collide with new genrated DCE UUIDs.
Hopefully people have a solution to this problem then.

\x1f
File: arla.info,  Node: Oddities,  Next: Themis,  Prev: Programming,  Up: Top

Oddities
********

AFS
===

   * Directories - UnixModeBits are ignored when the vnode is a
     directory.

   * Errnos are sent over the network. Like Solaris ENOTEMPTY(93)
     doesn't  even map to an error on sunos4 where ENOTEMPTY is 66.

   * Mountpoints have the mode-bits 0644, if they don't they are
     symlinks (and have the mode-bits 0755).


Operating systems
=================

   * On Irix 6.5 you have to build the dirents depending on what ABI
     of the binary you are currently running.

   * . and .. need to be first in directories, this is needed since some
     programs (like make) "knows" that the two first entries are . and
     .. and thus can be skiped.

   * Reclen (in struct dirent) shouldn't be too large. When its larger
     then the buffer used in opendir/readdir/closedir, you loose.


\x1f
File: arla.info,  Node: Themis,  Next: Arla timeline,  Prev: Oddities,  Up: Top

Themis
******

`Themis' is a tool for keeping local file systems up-to-date with a
file system description.

All lines in the input file consist of white-space separated fields. For
all directives except the `exclude' directive consist of one uppercase
letter, and flags can be added directly after the directive letter.
Flags can be specified in any order, and consist of uppercase letters.

If owner, group or mode is not specified, implicit values are used.
The default implicit values are `root' for owner, `wheel' for group,
`755' for directory mode bits and `644' for file mode bits.

These are the directives:

   * F<flags> <destination> <source> [<owner> <group>] [<mode>]

     Indicates the presence of a file.

   * D<flags> <destination> [<owner> <group>] [<mode>]

     Indicates the presence of a directory.

   * T<flags> <destination> <source> [<owner> <group>] [<mode>]

     Indicates the presence of a directory tree.

   * L<flags> <destination> <link data>

     Indicates the presence of a symbolic link.

   * N<flags> <destination>

     Indicates that themis should ignore that file/directory. If the
     file exists, it is left alone.

   * X<flags> <destination>

     Indicates that themis should remove the file/directory.

   * M<flags> <destination> [<owner> <group>] [<mode>]

     Modifies the flags, owner, group and mode of the (already
     specified) destination.

   * exclude <pattern>

     In a tree traversal, excludes all files that match <pattern>.
     Normal shell-style globbing can be used. Default exclution
     patterns that are included are: `.' and `..'. Also, when running
     with -macosx flag `._*' files are excluded.

     Common pattern to start to exclude are `*~', `RCS', `.OldFiles',
     `*,v', and `*.core'.


These are the flags:

   * `A' The source path is absolute. When this flag is not specified,
     the destination path is appended to the source path.

   * `I' The file should not be overwritten if it exists.

   * `O' Rename existing files to .old, instead of overwriting them.

   * `Q' Exit with error level 4 to indicate that the file has changed.
     This can be used to check if a reboot is needed when certain
     critical files change.

   * `R' Only valid on directories (`D' and `T' directives). Specifies
     that the directory should be cleaned of any files that are not in
     the description file.

   * `P' Replaces the implicit owner, group and mode bits with the
     values from the source file.


Normally, the input file should be pre-processed by mpp, m4, or some
other pre-processor, to enable conditional processing and include files.

\x1f
File: arla.info,  Node: Arla timeline,  Next: Authors,  Prev: Themis,  Up: Top

Arla timeline
*************

Arla have existed for quite some years.

Development started in fall 1993 by Björn Grönvall <bg@nada.kth.se>
(with an rxkad implantation), he had a working read-only implementation
in winter 1994. Quick followers was Assar <assar@sics.se> (at that time
<assar@pdc.kth.se>>) and Johan Danielsson <<joda@pdc.kth.se>>. The
platform that was chosen was Sparc SunOS4 (the OS that NADA, KTH was
using).

Some work was being done by Patrik Stymne <patriks@e.kth.se> in porting
arla to Ultrix, but this work was never finished.

At this time there was no free rx, lwp or rxkad. A basic rx
implementation was written, and the threading problem was solved by
using pthreads.

The Arla development started to slow down around 11 April 1995.

In about Mar-Jun 1996 rx and lwp was released by Transarc, this was made
possible by Jim Doyle <jrd@bu.edu>, and Derrick J. Brashear
<shadow@dementia.org>.

In September 1997, an rxkad implementation was written by Björn. At the
same time, a need for an AFS client for OpenBSD rose at the Stacken,
the local computer club at KTH. Other free OS:es, as NetBSD, FreeBSD
and Linux(primarily sparc) were also in need of AFS clients.

In TokKOM, a local communications system using LysKOM
(<http://www.lysator.liu.se/lyskom/>), Assar suggested to some club
members that it would be a nice thing to resume the arla development.

Some people suggested that it would be less trouble having someone with
access to the Transarc AFS source code port the code to the relevent
platforms. Assar then ported nnpfs to FreeBSD 2.2.x in notime (over the
night), just to show the high portability.

People started to understand that arla was a concept that would work,
and first out was Love Hörnqvist-Åstrand <lha@stacken.kth.se> to join.
Development was primarily aimed at OpenBSD and NetBSD at the moment,
and Arla lived for at least 2-3 weeks in /var/tmp on a host named
yakko.stacken.kth.se.

Magnus Ahltorp <map@stacken.kth.se> joined shortly thereafter, spending
the rest of the year reading about the Linux VFS, and after a while,
Artur Grabowski <art@stacken.kth.se> also started to work on arla,
concentrating on OpenBSD kernel stuff.

The first entry in ChangeLog is dated Fri Oct 24 17:20:40 1997. Around
this time arla was given a CVS tree, to ease development. Now you could
also mount the nnpfs-device and get the root-directory out of it.

The Linux port was done in a few weeks in the beginning of 1998. Only
the Linux 2.0 kernel was supported at this time.

In April 1998 Assar hade a Arla paper presented at Freenix. Linux 2.1
support was written also written around this time. This was a major
work since there was a lot of stuff that had changed (namely the
dcache).

The first milko entry is dated Thu Oct 30 01:46:51 1997. Note that this
milko in a sense "worked". You could get files out from it and store
them.

There was from this point a lot of work being done and quite a lot of
studies was "wasted". We learned a lot, but not the stuff we were
expected to.

We added support for `dynroot' and `fake-mp' to prepare for Windows and
Darwin/MacOSX support.

In Mars 2000 preliminary support for MacOS X/Darwin 1.0 was merged in
by Magnus and Assar.

Around the same time there we hacked in support for Solaris 8 (beta2)
There was also some work being done on Windows 2000 native driver at
same time.

In June 2000 there was a presentation on MADE2000 in Gothenburg, Sweden.

In September 2000 MacOS X Beta was working.

This just includes some milestones, for more information se Changelog.*
and NEWS files in the distribution.

\x1f
File: arla.info,  Node: Authors,  Next: Acknowledgments,  Prev: Arla timeline,  Up: Top

Authors
*******

Currently writing on arla are

Assar Westerlund, Everything

Magnus Ahltorp, Everything

Artur Grabowski, BSD nnpfs, OpenBSD maintainer

Love Hörnquist-Åstrand, Everything

Robert Burgess, fs, Solaris nnpfs

Johan Danielsson, OSF/1 nnpfs

Hans Insulander, pts, OpenBSD maintainer

Mattias Amnefelt, vos, milko

Harald Barth, doc

Tomas Olsson, milko

Mikael Vidstedt (userland, some milko stuff)

Jimmy Engelbercht (bos)
Rhapsody nnpfs port was contributed by Alexandra Ellwood <lxs@MIT.EDU>
Later, Rhapsody was renamed Darwin.

Disconnected code is written by:

WUWEI SHEN <wwshen@engin.umich.edu>

Cheng Jin <chengjin@eecs.umich.edu>

Paul Callaway <pcallawa@umich.edu>
For contributors, see *Note Acknowledgments::.

\x1f
File: arla.info,  Node: Acknowledgments,  Next: Index,  Prev: Authors,  Up: Top

Acknowledgments
***************

lwp and rx are copyrighted by IBM.  We're grateful to Derrick J
Brashear <shadow@dementia.org> and Jim Doyle <jrd@bu.edu> for making
them available.

the `rxkad' implementation was written by Björn Grönvall <bg@sics.se>
and is also part of the kth-krb distribution.

Some of the files in `libroken' come from Berkeley by the way of
NetBSD/FreeBSD

`editline' was written by Simmule Turner and Rich Salz.

The code for gluing these together were written by ourselves.

Bugfixes, documentation, encouragement, and code has been contributed
by:

Aaron M. Ucko                        <amu@MIT.EDU>
Alec Wolman                          <wolman@cs.washington.edu>
Alexandra Ellwood                    <lxs@MIT.EDU>
Brad Keryan                          <keryan@andrew.cmu.edu>
Constantine Sapuntzakis              <csapuntz@openbsd.org>
Dan Winship                          <danw@MIT.EDU>
Derrick J Brashear                   <shadow@dementia.org>
Harald Barth                         <haba@pdc.kth.se>
Jim Doyle                            <jrd@bu.edu>
John Davison                         <davisoja@clarkson.edu>
John Hawkinson                       <jhawk@MIT.EDU>
Karl Ramm                            <kcr@MIT.EDU>
Mark Eichin                          <eichin@kitten.gen.ma.us>
Per Boussard T/ED                    <Per.Boussard@era-t.ericsson.se>
Dima Ruban                           <dima@best.net>
Max                                  <davros@cyclone.Stanford.EDU>
Andrzej Filinski                     <andrzej@daimi.aau.dk>
Chuck Lever                          <chuckl@netscape.com>
WUWEI SHEN                           <wwshen@engin.umich.edu>
Cheng Jin                            <chengjin@eecs.umich.edu>
Paul Callaway                        <pcallawa@umich.edu>
Brandon S. Allbery                   <allbery@ece.cmu.edu>
Ken Raeburn                          <raeburn@raeburn.org>
Jeffrey Hutzelman                    <jhutz+@cmu.edu>
Hee-Seok Heo                         <hsheo@postech.ac.kr>
Paul Ewing Jr.                       <ewing@ima.umn.edu>
Niklas Hallqvist                     <niklas@appli.se>
Marko Asplund                        <aspa@hip.fi>
Chris Wing                           <wingc@engin.umich.edu>
Simon Josefsson                      <jas@pdc.kth.se>
Magnus Lindström                     <magnus.lindstrom@s3.kth.se>
Greg Stark                           <gsstark@mit.edu>
Matt                                 <deberg@mit.edu>
Björn Grönvall                       <bg@sics.se>
Tino Schwarze                        <tino.schwarze@informatik.tu-chemnitz.de>
David Sanderson                      <david@transarc.ibm.com>
Roman Hodek                          <Roman.Hodek@informatik.uni-erlangen.de>
Michael Sperber                      <sperber@informatik.uni-tuebingen.de>
Dr. Lex Wennmacher                   <wennmach@geo.Uni-Koeln.DE>
Janne Johansson                      <jj@dynarc.se>
Dr A V Le Blanc                      <LeBlanc@mcc.ac.uk>
Dave Morrison                        <dave@bnl.gov>
Jochen Saile                         <saile@sfs.nphil.uni-tuebingen.de>
Chris Kuklewicz                      <chrisk@MIT.EDU>
Nickolai Zeldovich                   <kolya@MIT.EDU>
Adam Thornton                        <adam@sinenomine.net>
Neale Ferguson                       <Neale.Ferguson@SoftwareAG-usa.com>
Hidvegi                              <hzoli@austin.ibm.com>
Todd T. Fries                        <todd@fries.net>
Andrea Campi                         <andrea@webcom.it>
William Uther                        <will+@cs.cmu.edu>

If you have done something and are not mentioned here, please send mail
to <arla-drinkers@stacken.kth.se>.

If you are mentioned here and have not contributed, that's because we
expect you to.

\x1f
File: arla.info,  Node: Index,  Prev: Acknowledgments,  Up: Top

Index
*****

* Menu:

* adb:                                   Kernel debuggers.
* AFS Filespace:                         AFS infrastructure.
* AFSDB:                                 AFS infrastructure.
* afslog:                                Kerberos tickets and AFS tokens.
* afsUUID:                               afsUUID.
* aklog:                                 Kerberos tickets and AFS tokens.
* ALG:                                   NAT.
* Archives:                              Introduction.
* Arlad debugging:                       Arlad.
* Backup server:                         AFS infrastructure.
* Bos server:                            AFS infrastructure.
* Bug reports:                           Introduction.
* Buserver:                              AFS infrastructure.
* Cell:                                  AFS infrastructure.
* CellServDB:                            AFS infrastructure.
* Comments:                              Introduction.
* DCE UUID:                              afsUUID.
* Debugging:                             Debugging.
* Debugging arlad:                       Arlad.
* Debugging techniques:                  Debugging techniques.
* Debugging NNPFS:                         nnpfs.
* Disconected operation:                 Disco with arla.
* DNS:                                   AFS infrastructure.
* Fileserver:                            AFS infrastructure.
* Filespace:                             AFS infrastructure.
* Fsserver:                              AFS infrastructure.
* Gdb:                                   Debugging LWP with GDB.
* kalog:                                 Kerberos tickets and AFS tokens.
* kaserver:                              Integration with Kerberos.
* kauth:                                 Kerberos tickets and AFS tokens.
* kerberosserver:                        Integration with Kerberos.
* Kernel debugging:                      Kernel debuggers.
* Kernel debuging on linux:              Kernel debuggers.
* KeyFile:                               Integration with Kerberos.
* kinit:                                 Kerberos tickets and AFS tokens.
* klog:                                  Kerberos tickets and AFS tokens.
* Ksymoops:                              Kernel debuggers.
* Linux kernel debugging:                Kernel debuggers.
* Live kernel:                           Kernel debuggers.
* Mail archives:                         Introduction.
* Mailing list:                          Introduction.
* Masquerading:                          NAT.
* NAT:                                   NAT.
* PAT:                                   NAT.
* pts:                                   Relationship between pts uid and unix uid.
* Ptserver:                              AFS infrastructure.
* Salvage:                               AFS infrastructure.
* Samba:                                 Samba.
* Themis:                                Themis.
* TheseCells:                            Integration with Kerberos.
* ThisCell:                              Integration with Kerberos.
* Tickets:                               Kerberos tickets and AFS tokens.
* Tokens:                                Kerberos tickets and AFS tokens.
* Ubik:                                  AFS infrastructure.
* uid:                                   Relationship between pts uid and unix uid.
* Upserver:                              AFS infrastructure.
* Vldbserver:                            AFS infrastructure.
* Vlserver:                              AFS infrastructure.
* Volser:                                AFS infrastructure.
* Volumeserver:                          AFS infrastructure.
* NNPFS debugging:                         nnpfs.


\x1f
Tag Table:
Node: Top\x7f191
Node: Introduction\x7f2110
Node: AFS infrastructure\x7f5546
Node: Organization of data\x7f15964
Node: Requirements\x7f16460
Node: Data organization\x7f18431
Node: Callbacks\x7f20761
Node: Volume management\x7f21446
Node: Relationship between pts uid and unix uid\x7f22789
Node: AFS and the real world\x7f23658
Node: NAT\x7f24159
Node: Samba\x7f26081
Node: Integration with Kerberos\x7f26732
Node: Kerberos tickets and AFS tokens\x7f28564
Node: Parts of Arla\x7f34115
Node: How arla works\x7f34567
Node: The relation between Arlad and NNPFS\x7f35586
Node: The life of a file\x7f36245
Node: Tools and libs\x7f38767
Node: Rx protocol\x7f39982
Node: LWP\x7f40259
Node: The files in arlad/\x7f40671
Node: pioctl and kafs\x7f42796
Node: Debugging\x7f44895
Node: Arlad\x7f45371
Node: Debugging LWP with GDB\x7f46614
Node: nnpfs\x7f48369
Node: nnpfs on linux\x7f49016
Node: Debugging techniques\x7f49361
Node: Kernel debuggers\x7f50293
Node: Darwin/MacOS X\x7f54706
Node: Porting\x7f55323
Node: Porting user-space\x7f55934
Node: Porting NNPFS\x7f57697
Node: Programming\x7f62046
Node: Arla pioctl's\x7f62481
Node: Disco with arla\x7f63589
Ref: Log entry optimization\x7f65373
Node: afsUUID\x7f68205
Node: Oddities\x7f69023
Node: Themis\x7f69929
Node: Arla timeline\x7f72653
Node: Authors\x7f76325
Node: Acknowledgments\x7f77145
Node: Index\x7f81013
\x1f
End Tag Table