2006-11-29 James Livingston <doclivingston@gmail.com>

[rhythmbox.git] / README.daap

How to build and use Rhythmbox with DAAP music sharing support

* configure Rhythmbox with --enable-daap
        Requires libsoup & either Howl or Avahi
* Start Howl's mDNSResponder or Avahi's avahi-daemon
* Start Rhythmbox, watch as remote music shares appear in your source list
* Click on a remote source, play their music.
* Check Edit->Preferences->Sharing to share your music library.


How Music Sharing Works

0. Definitions
1. mDNS/DNS-SD support 
2. Discovering new music shares
3. Connecting to a music share
4. Playing remote music
5. Publishing your music
6. RBDAAPStructure - structure parsing & building

0. Definitions / Fundamentals

DAAP stands for Digital Audio Access Protocol.  It is the communications
protocol that music sharing uses.  It is a structured (like xml) query/response
language that works over HTTP 1.1.  A DAAP server is nothing more than an 
HTTP server that responds to certain URI requests with a structured DAAP
response.  DAAP is the same music sharing protocol used by Apple iTunes.  So,
a copy of Rhythmbox that was compiled with DAAP will be able to browse iTunes
shares, and iTunes will be able to browse Rhythmbox's music.

mDNS is multicast DNS.  DNS Service Discovery (DNS-SD) sits on top of it.  
mDNS/DNS-SD is what Apple calls Bonjour or Rendezvous.  The fundamentals aren't especially important, suffice it to say it allows for automatic discovery of 
services on your local network.  You send out an mDNS/DNS-SD request for DAAP 
servers, it responds with all the servers that have published their information 
on your network.  mDNS/DNS-SD services have types.  The DAAP service type is 
_daap.  It is transported over tcp, so in the code you'll often see _daap._tcp, 
which is the full name for a DAAP service.  

iTunes hashing is an "authentication" method that appeared in later versions
of Apple iTunes.  It is a modified md5 hash of the URI being requested from a
DAAP server.  The hash is sent along in the HTTP headers to show an iTunes
server that the connecting client is "authenticated" (i.e. "is an iTunes
client").  The hashing code was borrowed with many thanks from libopendaap.

1. mDNS/DNS-SD support

For DAAP music sharing you'll need an mDNS/DNS-SD implementation.  

Howl (http://www.porchdogsoft.com/products/howl/) 
or
Avahi (http://www.freedesktop.org/Software/Avahi) 

are supported.  Either way, the mDNS/DNS-SD routines live in 
daapsharing/rb-daap-mdns.c/h.

There are three routines in the mDNS/DNS-SD libraries that we need.  Browsing to
discover remote shares, Resolving to figure out those shares' IP addresses, and
Publishing to make our own share known on the network.  There are three opaque
datatypes, RBDAAPmDNSBrowser, RBDAAPmDNSResolver, RBDAAPmDNSPublisher.  If
Rhythmbox has been compiled with Howl support, these will will be used as
sw_discovery_oid structures, which are used to identify the specific mDNS/DNS-SD
call.  If Rhythmbox is using Avahi, these datatypes represent 
AvahiServiceBrowser, AvahiServiceResolver and AvahiEntryGroup variables, 
which, again, are used to uniquely idenfity the specific mDNS/DNS-SD call.

Browsing for remote shares is started via rb_daap_mdns_browse() which takes a
callback that is run anytime a share is discovered or goes away.  Browsing will
continue until rb_daap_mdns_browse_cancel() is called.

Resolving is done asychronously via rb_daap_mdns_resolve() and a callback which
runs when the host is resolved or a timeout reached.  Either way, when the
the callback is run, the resolving stops.  If you wish to stop resolving early,
you can use rb_daap_mdns_resolve_cancel().

Pubsishing is done via rb_daap_mdns_publish().  The callback for this function
is used to notify you of success or failure due to name collision.  Two servers
cannot have the same name.  Publishing will continue until 
rb_daap_mdns_publish_cancel() is called.

3. Discovering new music shares

Discovery is started in shell/rb-shell.c:rb_shell_constructor().
rb_daap_sources_init() is called, which starts the discovery.  When new shares
are discovered, an RBDAAPSource is created and added to the source list.  When
shares disappear from the network, the source is removed from the list.  No
attempt is made to resolve or connect to the share until the user clicks on the
source.  This reduces unneccessary computations and network traffic.

4. Connecting to a music share

When a user clicks on an music share source in the list, 
rb-shell.c:rb_shell_select_source() runs the virtual function 
rb_source_activate().  This tells the RBDAAPSource to resolve the server via
RBDAAPmDNSResolver.  In the resolver's callback function, if the resolve was
successful, an RBDAAPConnection is created to the remote share.

RBDAAPConnection takes care of all the dirty work establishing a connection and
recieving a list of the share's music & playlists.  The HTTP communication is
done via libsoup.  There are 5 steps to establishing a new connection.  In each
several HTTP requests are made, and a DAAP structure returned.  The structure
is parsed (see RBDAAPStructure) and the appropriate information read out of
the parsed structure.

1. Get the server's version (/server-info)
        Here we request /server-info to figure out what version of DAAP the
server is using.  Different versions requiring different iTunes hashing 
techniques.

2. Login (/login & /update)
        Here we request /login to recieve a session-id.  All subsequent requests
must include the session-id as part of the URI (?session-id=X).  We also request
the initial revision-number.  Revision numbers are used to track changes in
the server's music library.  RHYTHMBOX DOES NOT YET SUPPORT TRACKING THESE
CHANGES.  The session-id is used by the server to identify your connection.

3. Get the database information (/databases)
        We request /databases to get a list of databases on the server.  A list
is returned, but it is always only 1 element long, containing the ID number
of the music library on the server.

4. Get the song listing (/databases/ID/items)
        We request /database/ID/items with a list of the metadata (artist,
album, title, etc) that we're interested in.  A lengthy list of all the songs in
the database and their metadata is returned.  Each song has a unique ITEM-ID 
number in the database.  The URI for requesting the actual song is 
/databases/ID/items/ITEM-ID.ITEM-FORMAT.  We create a RhythmDBEntry for each 
song, and add it to the RhythmDB.  A local hash table translating ITEM-ID to
uri is also populated (this will be used in step 5).

5. Get the playlists (/databases/ID/containers & 
/databases/ID/containers/CONTAINER-ID/items)
        We request /databases/ID/containers to recieve a list of playlists. Each
playlist has a name and a unique CONTAINER-ID number.  For each palylist, a 
request of /databases/ID/containers/CONTAINER-ID/items is made, asking for the 
ITEM-IDs of all the songs in the playlist.  Each of these ITEM-IDs is resolved
into the uri using the hash table populated in step 4, and the uri add to the 
RBDAAPPlaylist.

After the connection is made and all the data transfered, a new 
RBDAAPPlaylistSource is created for each RBDAAPPlaylist in the connection.
These are added underneath the original RBDAAPSource.

4. Playing remote music

A URI for a DAAP music file looks like 
daap://HOST:PORT/databases/DATABASE-ID/items/ITEM-ID.ITEM-FORMAT?session-id=SESSION-ID

A new GStreamer source, RBDAAPSrc (existing in daapsharing/rb-daap-src.c/h) 
handles playing daap:// URIs.  It uses simple TCP sockets to open a connection
to the host, request the file (with the appropriate headers) and push the
recieved data through the pipeline.  The data recieved (the song) is in the
original format (be it mp3, ogg, aac, wav, whatever).  So GStreamer needs to
figure out how to decode the file before it pushes it the rest of the way
through the pipeline.  To do this, GStreamer uses its decodebin & typefind
elements. 

Every request for a song (even re-requests) must include several special 
headers, including the iTunes hash and a DAAP-Request-ID.  The Request-ID is 
simply a running tally of the number of songs we've requested.  It starts at 1,
and every time there is a new request it is incremented.  To determine the 
proper headers to send, RBDAAPsrc uses rb_daap_source_find_for_uri() and 
rb_daap_source_get_headers() which find the RBDAAPSource that contains a 
specific URI and fetch the headers needed for that URI.  
rb_daap_source_get_headers() hands the work off to 
rb_daap_connection_get_headers(), which keeps track of the Request-ID and
determines the iTunes hash.

Seeking in a DAAP stream is done by closing the existing connection and
establishing a new one, sending a Range: HTTP header with the appropriate
starting point.  However, we do not support standard GStreamer seeking.
This is because, if we were to, GStreamer's decodebin & typefind elements
abuse it, seeking several times in succession to determine the type of data
coming through the pipeline.  These requests will overwhelm an iTunes server,
prompting it to return an error after several of them.  To work around this,
RBPlayer handles seeking in DAAP streams via rb_daap_src_[set/get]_time().
This is a WORKAROUND until GStreamer is fixed or proper seeking can be figured
out.

5. Publishing your music

Publishing is started in shell/rb-shell.c:rb_shell_constructor() via the
rb_daap_sharing_init() call.  This creates an RBDAAPShare structure, which, in 
turn, creates a SoupServer structure to handle HTTP requests.  The server binds
on an available port, and this port along with the machine's IP address are
published to mDNS/DNS-SD via RBDAAPmDNSPublisher.

RBDAAPShare does several things.  It assigns unique ID numbers to every song
in the RhythmDB, and keeps an up to date mapping of RhythmDBEntry to ID and ID
to RhythmDBEntry via entry-added & entry-deleted signals from the RhythmDB.
THIS WILL EVENTUALLY BE USED TO SUPPORT SENDING MUSIC LIBRARY UPDATES.
For each request on the SoupServer, an appropriate DAAP response is built (see
RBDAAPStructure) and returned.  If the request is for a song file itself, the
file is opened via GnomeVFS and its contents returned.  Rhythmbox's DAAP server
implementation does not require iTunes hashing headers nor does it require
unique session-ids.

6. RBDAAPStructure - structure parsing & building

RBDAAPStructure is a data structure that handles parsing and building DAAP 
responses.  It lives in daapsharing/rb-daap-structure.c/h.

Useful reading:

http://www.deleet.de/projekte/daap/  The section titled 'DAAPD technicals'
http://molelog.molehill.org/blox/Computers/Macintosh/DAAP3.writeback (#1&2 also)

A DAAP response is of the form:

Byte:   1234            | 5678                  | length bytes
Data:   content code    | content length        | content

Codes are 4 byte character strings like 'msrv', 'mstt', 'mpro', and so on.
A list of codes is available in the RBDAAPContentCode enumeration in 
rb-daap-structure.h.  They are defined in the RBDAAPContentCodeDefintion array 
in rb-daap-structure.c.  There are several DAAP types.  You can ask the
DAAP server for /content-codes to retrieve a dictionary mapping codes to types,
or you can "just know."

Type       | C data type | Size
------------------------------
Byte       | gchar       | 1
Signed Int | gchar       | 1
Short      | gint16      | 2
Int        | gint32      | 4
Date       | gint32      | 4
Int64      | gint64      | 8
Version    | gdouble     | 4
String     | gchar *     | Whatever the DAAP response says
Container  | GNode *     | Whatever the DAAP response says

Containers hold other data.  For example, the response from /server-info is:

dmap.serverinforesponse                 Container
        dmap.status                     Int
        dmap.protocolversion            Version
        daap.protocolversion            Version
        daap.itemname                   String
        dmap.loginrequired              Int
        ...
dmap.databasescoutn                     Int


The serverinforesponse is a Container holding the other content.

The bulk of the parsing work is done in 
rb_daap_structure_parse_container_buffer() The data & the length are passed in,
as well as a GNode * for the parent container.  The first pass is provided an
empty container to add all the "toplevel" content to.  The 4 byte content code 
is read and the daap type looked up accordingly.  The 4 byte data length 
(code size) is read.  The data is read into a GValue, with its type set 
accordingly.  The GValue & content code are added to the GNode * as new 
children.  If type is a container, rb_daap_structure_parse_container_buffer() is
called recursively.

There are find functions rb_daap_structure_find_item() & 
rb_daap_structure_find_node() which take a GNode * structure and content code
to be located.  The structure is walked until a node matching the content code
is found.  The node or the item is returned, depending on the function being
used.

To build a DAAP structure you use rb_daap_structure_add().  It creates a new 
node containing the data you add and appends it to the parent node you pass.  
You pass NULL for the parent to create a toplevel node.  The new node is 
returned.  The data you add is passed in as a content code and then the data.  
If the content type is Container, you don't need to pass any data.  Once the 
DAAP structure is build, use rb_daap_structure_serialize() to convert it to
a string appropriate for sending over a socket.