Don't mention fetchmail in the TUTORIAL since fetchmail 7.0.0 hasn't been

[libpwmd.git] / doc / TUTORIAL

What is PWMD?
-------------
Password Manager Daemon (pwmd[1]) is a universal data server. Applications
connect to a unix domain socket and send commands to manipulate the data.  It
started out as a way to keep track of my passwords for accounts like email,
websites, etc. but can really be used to store anything you want.

There are other password management tools but PWMD has distinguishing
features:

- It does not depend on a desktop environment but has the ability for
  applications to connect to it like a desktop solution provides.

- Some portion of a stored data can be "shared" with, or "linked to",
  another data in the same data file. It's a lot like XML entities if
  you're familiar with that, but implemented in a different way.


Element paths
-------------
The document is saved as an XML file and manipulated by commands sent from a
client. Commands that access elements take what I call an "element path" as an
argument. An element path is a character string that contains XML element
names representing branches of an element tree. The branches are separated
with a TAB (ASCII 0x09) character. Why a TAB? So other characters can be used
in the element name. If for example a '/' were used as the separator, then a
URL (i.e., http://sf.net/) could not be used as an element name.  In fact, the
only restriction of an element name is that it not contain any whitespace
characters and not begin with an exclamation point (ASCII 0x21 or !).

For the rest of this tutorial, when you see <TAB>, replace it with an actual
TAB character. For example, the element path "a<TAB>child<TAB>last" has the
following element tree:

        <a>
            <child>
                <last>
                    Content or CDATA of the "last" element.
                </last>
            </child>
        </a>

I should say that the XML document that pwmd uses is a little more
complicated. It actually looks like the following internally but we will
use the above format in this tutorial for simplicity:

        <element _name="a">
            <element _name="child">
                <element _name="last">
                    Content or CDATA of the "last" element.
                </element>
            </element>
        </element>

Every element created has an element named "element" with an attribute "_name"
associated with it. The value of the "_name" attribute is the what pwmd
commands use as the element name. It is done this way so that a wider range of
characters can be used in the element name while maintaining valid XML.

There is a difference from your normal XML document in that only the first
match of an element is considered for the current tree depth. When found, the
next element in the element path is searched for beginning at the child node
of the matched element. From the example above:

        <a>
            <child>
                <last>
                    Content or CDATA of the "last" element.
                </last>
            </child>
            <child>
                This element will never be reached because its name
                "child" is the same as the previous or sibling element which
                is matched first.
            </child>
        </a>

The second "child" element is never reached, although an element of a
different name may be accessed. The exception to this are the XPATH and
XPATHATTR commands which process the entire document and do not use element
paths but XPath expressions.


Applications
------------
Now that the basics of pwmd have been described, you will need a client
to send commands to the server to try it all out.

If you are making your own application and want to use pwmd or need a simple
client to connect to the server, a library called libpwmd[2] is available. It
has few dependencies and - as mentioned - includes a client named pwmc. The
rest of this tutorial uses pwmc for connecting. It is command-line based and
there are not any fancy graphics, but it is good for understanding how
commands are processed.

If you want a more user friendly client that kind of looks like a file
manager and has a point and click interface, try pwmdEdit[3] which is
based on the Qt4 toolkit library.


Connecting
----------
We will be using pwmc as the client which is included with libpwmd. pwmc has
two modes that commands are read from: interactive and stdin. The interactive
mode uses the readline library and has command history. The default mode can
read commands from a pipe or standard input. For the following examples we
will be using interactive mode and connecting to the default local unix domain
socket that pwmd waits for connections on.  Remote connections are possible
over an SSH channel but that is not covered here.  Read the pwmc(1) manual
page for how to do that.

The example filename we will use is "datafile". Initially it is a non-existent
file but it will be created once we have saved to it. Now let us connect to the
server and open the new file in interactive mode.

$ pwmc --interactive datafile
Connected.
pwm>

The "pwm>" prompt is a readline prompt that has command history and can do
filename completion. It also is annoying because as mentioned above, elements
in element paths are TAB delimited but command completion in readline uses the
TAB character too. So to insert a real TAB character in interactive mode
you can do one of two things: either first press CTRL-V then TAB, or disable
the readline tab completion entirely by adding the following line to your
~/.inputrc file:

        set disable-completion on

This has the problem of disabling tab completion in all programs that use the
readline library. To fix this you can create a special inputrc that only pwmc
uses by setting the INPUTRC environment variable to a file that contains only
that line before running pwmc. For example:

$ INPUTRC=/path/to/pwmc-inputrc pwmc --interactive datafile

Now you can insert a TAB character without the readline library interfering.


Commands
--------
To store some data in an element path you will need to think about how you
want your data organized as elements in the document. If for example you will
be storing account information, then it may be a good idea to categorize what
the account is for: email, instant messaging, blogging, etc.

In the following example we will create element paths for an email account. An
email account usually needs a few details to be useful such as the hostname,
port, username and password elements.

Let us set up the email server information first:

pwm> INQUIRE STORE

A note before we continue: The INQUIRE command is a pwmc specific command and
only available in interactive mode. It is not a pwmd protocol command. It
takes an argument which is the protocol command that uses a server inquire,
and in this case, the STORE command. The equivalent to this when not in
interactive mode is to use the --inquire command line option. Refer to the
documentation of the protocol command to determine if it uses a server inquire
or not.

pwmc is now waiting for data after entering the above command. It's reading
from the standard input and not from the readline prompt so your TAB character
should work correctly:

email<TAB>isp<TAB>IMAP<TAB>hostname<TAB>imap.server.com<CTRL-D><CTRL-D>

Pressing <CTRL-D> sends the current line when non-empty and terminates the
INQUIRE when on an empty line. If you were to have pressed <RETURN> before
pressing any <CTRL-D> then the entire line would have been sent including the
newline character. And since a hostname containing a newline is not a valid
hostname, that is not what we want.

We have just created an element path whose structure looks like this:

        <email>
            <isp>
                <IMAP>
                        <hostname>imap.server.com</hostname>
                </IMAP>
            </isp>
        </email>

It has good structure: the root element tells you what it's for, email in this
case. The next element, isp, could be replaced with something that shows what
email account this relates to. There are different email protocols like POP3,
IMAP and SMTP so creating an IMAP protocol tree makes sense too.

When a client needs to retrieve the value for the element path we have just
created, it sends the GET command with the element path as the argument:

pwm> GET email<TAB>isp<TAB>IMAP<TAB>hostname
imap.server.com

Remember that in interactive mode you will need to press <CTRL-V> before
pressing <TAB> unless you have configured an inputrc. The XFER lines are
status messages and can be disabled with the --no-status command line option.
They show the current and total length of data transfered in bytes. There are
other status messages too and are documented in pwmd. The last line is the
value of the element path that we just stored.

Let us create the rest of the needed elements:

pwm> INQUIRE STORE
email<TAB>isp<TAB>IMAP<TAB>port<TAB>993<CTRL-D><CTRL-D>
pwm> INQUIRE STORE
email<TAB>isp<TAB>username<TAB>myusername<CTRL-D><CTRL-D>
pwm> INQUIRE STORE
email<TAB>isp<TAB>password<TAB>mypassword<CTRL-D><CTRL-D>

Now the element structure for the root "email" element looks like this:

        <email>
            <isp>
                <IMAP>
                        <hostname>imap.server.com</hostname>
                        <port>993</port>
                </IMAP>
                <username>myusername</username>
                <password>mypassword</password>
            </isp>
        </email>

If you wanted to change your password (after changing it on the mail server of
course) just do as you did when initially creating the password element path.
The new content will overwrite the existing content:

pwm> INQUIRE STORE
email<TAB>isp<TAB>password<TAB>newpassword<CTRL-D><CTRL-D>

Lets save what we have stored so far. You could send the SAVE protocol command
but that may not work correctly depending on a few different things like file
cache status, pinentry settings or a remote connection. The best way is to let
pwmc do it by sending an EOL (end-of-line) character on an empty line:

pwm> <CTRL-D>

pwmc uses a pinentry program to ask for a passphrase when the passphrase
is not cached in pwmd. If you are using a GUI and have pinentry installed
then things are easy. A dialog box will pop up asking for a passphrase
then asking you to confirm it. Once done, the data file is encrypted and
written to disk and the passphrase is cached in pwmd so you do not need to
enter it the next time your data file is opened or saved. There are a
few pwmd configuration parameters that affect how this operates. See the
pwmd(1) manual page for details.

There is a pinentry program for the console, too. It uses the ncurses
interface and should work fine when not using the 'screen' program. When
in a screen session the pinentry will usually not behave correctly. The
solution is to make the pinentry use the terminal that screen was
started on rather than the current tty, which is the default. This is
done by setting TTYNAME in ~/.pwmd/pinentry.conf. See the pwmc(1) manual
page for details.

Applications that connect to pwmd by using the libpwmd API will more than
likely have different element path requirements. Refer to each application's
documentation that supports pwmd for the required elements.


Linking elements
----------------
One distinguishing feature of pwmd is the ability to share or link to data of
one section of the XML document, to another. For example, if your ISP lets you
host a blog on their server and you use a blogging client (which uses libpwmd)
to edit your posts, you can share authentication information with the email
example above provided that the account details are the same.  When element
linking is used this avoids the need to change the password for both elements:
the email password and the blogging client password.

How this is done is by setting a special attribute "target" for an element. If
you know anything about HTML then you might know that a "target" attribute can
have an anchor pointing to a unique ID which will make the browser "jump" to
that position when selected. This is where the attribute name came from,
because it reminded me of HTML. :)

OK, let's create an example blogging element path:

pwm> INQUIRE STORE
blog<TAB>isp<TAB>hostname<TAB>blog.myisp.com<CTRL-D><CTRL-D>
pwm> ATTR SET target blog<TAB>isp<TAB>username email<TAB>isp<TAB>username
pwm> ATTR SET target blog<TAB>isp<TAB>password email<TAB>isp<TAB>password

Now each access of the "blog/isp/username" and "blog/isp/password" element
paths (which are created if they do not exist) will point to
"email/isp/username" and "email/isp/password" respectively. To get the value
or content of an element that contains a "target" attribute without following
any links, prefix the element with a '!' when specifying it in an element
path. For example:

pwm> GET blog<TAB>isp<TAB>!password
ERR 26: No value

Here, it is an error because we have not assigned a value to the literal
element. A literal element is an element that either contains no "target"
attribute or ignores the "target" attribute by prefixing the element with the
mentioned literal '!' character. Let us try storing an example password to the
literal element to show usage:

pwmd> INQUIRE STORE
blog<TAB>isp<TAB>!password<TAB>newpassword<CTRL-D><CTRL-D>

Now when we redo the previous GET command:

pwm> GET blog<TAB>isp<TAB>!password
XFER 0 11
XFER 11 11
newpassword

This is the value of the literal password element of the element path. To
follow the element path stored in the "target" attribute, omit the '!' in the
password element:

pwm> GET blog<TAB>isp<TAB>password
XFER 0 10
XFER 10 10
mypassword

"target" attributes may also reference other elements with a "target"
attribute. Every "target" attribute will be followed until there are no other
links to resolve. To get the real element path and resolve all "target"
attributes, use the REALPATH command:

pwm> REALPATH blog<TAB>isp<TAB>password
!email<TAB>!isp<TAB>!password

As you can see each element is prefixed with the literal '!' character meaning
that no "target" attribute exists in any element of the resulting element
path.

Use the LIST command to list the root elements of the document or the entire
element tree of an element path:

pwm> LIST
!email
!blog

pwm> LIST !blog
!blog
!blog<TAB>!isp
!blog<TAB>!isp<TAB>!username
!blog<TAB>!isp<TAB>username
!blog<TAB>!isp<TAB>!password
!blog<TAB>!isp<TAB>password

Notice that there are two username and password elements: each with and
without a literal prefix. This shows that the username and password elements
have a "target" attribute. If there were child elements of the non-literal
username and password elements then they would be shown as children of these
elements using the current element path as the prefix.

For the final example, we will remove the link or target of the blogging
password element:

pwm> ATTR DELETE target blog<TAB>isp<TAB>!password

If we were to have omitted the '!' from the password element then pwmd would
try to remove a "target" attribute of the "email/isp/password" element path
and would return an error because that attribute has not been set for that
element. Note that the "blog" and "isp" elements have no literal element
character prepended to them. It is not needed because they have no link or
target attribute set.

The best way to get the hang of linking elements is to experiment. Use the
LIST and DUMP commands to show the document structure and remember to use the
literal prefix '!' when you do not want to follow any links or "target"
attributes for element that have them.


Ben Kibbey <bjk@luxsci.net>

1. http://bjk.sourceforge.net/pwmd/
2. http://bjk.sourceforge.net/pwmd/libpwmd/
3. http://bjk.sourceforge.net/pwmd/pwmdedit/