Remove lazy loading of deltas

[pacman-ng.git] / README

ALPM library overview & internals
=================================

Here is a list of the main objects and files from the ALPM (i.e. Arch Linux
Package Management) library. This document, while not exhaustive, also
indicates some limitations (on purpose, or sometimes due to its poor design) of
the library at the present time.

There is one special file,"alpm.h", which is the public interface that
should be distributed and installed on systems with the library. Only
structures, data and functions declared within this file are made available to
the frontend. Lots of structures are of an opaque type and their fields are
only accessible in read-only mode, through some clearly defined functions.

In addition to "alpm.h", the interfaces of "alpm_list.h" have also been made
available to the frontend, for allowing it to manipulate the lists returned by
the backend.

Several structures and functions have been renamed compared to pacman 2.9 code.
This was done at first for the sake of naming scheme consistency, and then
primarily because of potential namespace conflicts between library and frontend
spaces. Indeed, it is not possible to have two different functions with the
same name declared in both spaces. To avoid such conflicts, internal function
names have been prepended with "_alpm_".

In a general manner, public library functions are named "alpm_<type>_<action>"
(examples: alpm_trans_commit(), alpm_release(), alpm_pkg_get_name(), ...).
Internal (and thus private) functions should be named "_alpm_XXX" for instance
(examples: _alpm_needbackup(), _alpm_runscriplet(), ...). Functions defined and
used inside a single file should be defined as "static".


[Initialization]

alpm_initialize() is used to initialize library internals and to create
a transparent handle object. Before its call, the library can't be used.

alpm_release() just does the opposite (memory used by the library, and the
handle is freed). After its call, the library is no longer available.


[Options]

The library does not use any configuration file. It is up to the front end to
configure the library as needed; the handle holds a number of configuration
options instead.

All of the following options have a alpm_option_get_* and alpm_option_set_*
function for getting and setting the value. They cannot be set before the
library is initialized.

* logcb: The callback function for "log" operations.
* dlcb: The callback function for download progress of each package.
* fetchcb: Callback for custom download function.
* totaldlcb: The callback function for overall download progress.
* root: The root directory for pacman to install to (Default: /)
* dbpath: The toplevel database directory (Default: /var/lib/pacman)
* logfile: The base path to pacman's log file (Default: /var/log/pacman.log)
* usesyslog: Log to syslog instead of `logfile` for file-base logging.

The following options also have `alpm_option_{add,remove}_*` functions, as the
values are list structures.
NOTE: The add and remove functions are NOT plural, as they are in English:
alpm_option_{get,set}_noupgrades -> alpm_option_{add,remove}_noupgrade.

* cachedirs: Paths to pacman's download caches (Default: /var/cache/pacman/pkg)
* noupgrades: Files which will never be touched by pacman (extracted as .pacnew)
* noextracts: Files which will never be extracted at all (no .pacnew file)
* ignorepkgs: Packages to ignore when upgrading.
* ignoregrps: Groups to ignore when upgrading.

The following options are read-only, having ONLY alpm_option_get_* functions:

* lockfile: The file used for locking the database
  (Default: <dbpath>/db.lck)
* localdb: A pmdb_t structure for the local (installed) database
* syncdbs: A list of pmdb_t structures to which pacman can sync from.

The following options are write-only, having ONLY alpm_option_set_* functions:

* usedelta: Download delta files instead of complete packages if possible.

[Transactions]

The transaction structure permits easy manipulations of several packages
at a time (i.e. adding, upgrade and removal operations).

A transaction can be initiated with a type (SYNC, UPGRADE or REMOVE),
and some flags (NODEPS, FORCE, CASCADE, ...).

Note: there can only be one type at a time: a transaction is either
created to add packages to the system, or either created to remove packages.
The frontend can't request for mixed operations: it has to run several
transactions, one at a time, in such a case.

The flags allow to tweak the library behaviour during its resolution.
Note, that some options of the handle can also modify the behavior of a
transaction (NOUPGRADE, IGNOREPKG, ...).

Note: once a transaction has been initiated, it is not possible anymore
to modify its type or its flags.

One can also add some targets to a transaction (alpm_trans_addtarget()).
These targets represent the list of packages to be handled.

Then, a transaction needs to be prepared (alpm_trans_prepare()). It
means that the various targets added, will be inspected and challenged
against the set of already installed packages (dependency checking, etc...)

Last, a callback is associated with each transaction. During the
transaction resolution, each time a new step is started or done (i.e
dependency or conflict checking, package adding or removal, ...), the
callback is called, allowing the frontend to be aware of the progress of
the resolution. Can be useful to implement a progress bar.


[Package Cache]

libalpm maintains two caches for each DB. One is a general package cache, the
other is a group cache (for package groups). These caches are loaded on demand,
and freed when the library is.

It is important to note that, as a general rule, package structures should NOT
be freed manually, as they SHOULD be part of the cache.  The cache of a
database is always updated by the library after an operation changing the
database content (adding and/or removal of packages).  Beware frontends ;)


[Package]

The package structure maintains all information for a package. In general,
packages should never be freed from front-ends, as they should always be part
of the package cache.

The 'origin' data member indicates whether the package is from a file (i.e. -U
operations) or from the package cache. In the case of a file, all data members
available are present in the structure. Packages indicated as being from the
cache have data members filled on demand. For this reason, the alpm_pkg_get_*
functions will load the data from the DB as needed.


[Errors]

The library provides a global variable pm_errno.
It aims at being to the library what errno is for C system calls.

Almost all public library functions are returning an integer value: 0
indicating success, -1 indicating a failure.
If -1 is returned, the variable pm_errno is set to a meaningful value
Wise frontends should always care for these returned values.

Note: the helper function alpm_strerror() can also be used to translate one
specified error code into a more friendly sentence, and alpm_strerrorlast()
does the same for the last error encountered (represented by pm_errno).


[List - alpm_list_t] 

The alpm_list_t structure is a doubly-linked list for use with the libalpm
routines. This type is provided publicly so that frontends are free to use it
if they have no native list type (C++, glib, python, etc all have list types).
See the proper man pages for alpm_list_t references.



PACMAN frontend overview & internals
====================================

Here are some words about the frontend responsibilities.
The library can operate only a small set of well defined operations and
dummy operations.

High level features are left to the frontend ;)

For instance, during a sysupgrade, the library returns the whole list of
packages to be upgraded, without any care for its content.
The frontend can inspect the list and perhaps notice that "pacman"
itself has to be upgraded. In such a case, the frontend can choose to
perform a special action.


[MAIN] (see pacman.c)

Calls for alpm_initialize(), and alpm_release().
Read the configuration file, and parse command line arguments.
Based on the action requested, it initiates the appropriate transactions
(see pacman_upgrade(), pacman_remove(), pacman_sync() in files upgrade.c,
remove.c and sync.c).


[CONFIGURATION] (see conf.h)

The frontend is using a configuration file, usually "/etc/pacman.conf".  Some
of these options are only useful for the frontend only (mainly the ones used to
control the output like showsize or totaldownload, or the behavior with
cleanmethod and syncfirst).  The rest is used to configure the library.


[UPGRADE/REMOVE/SYNC]

The file pacman.c has been divided into several smaller files, namely
upgrade.c, remove.c, sync.c and query.c, to hold the big parts: pacman_upgrade,
pacman_remove, pacman_sync.

These 3 functions have been split to ease the code reading.



API CHANGES BETWEEN 3.1 AND 3.2
===============================

[REMOVED]
- alpm_db_whatprovides()
- alpm_splitdep (no longer public)
- trans->targets was removed, so alpm_trans_get_targets() as well
- error codes:
    PM_ERR_OPT_*, PM_ERR_PKG_INSTALLED, PM_ERR_DLT_CORRUPTED,
    PM_ERR_LIBARCHIVE_ERROR
- event: PM_TRANS_EVT_EXTRACT_DONE
- PM_TRANS_TYPE_ADD pmtranstype_t (add transaction)
- PM_TRANS_FLAG_DEPENDSONLY pmtransflag_t

[CHANGED]
- alpm_grp_get_pkgs returns with pmpkg_t list, not package-name list
- Swap parameters on PM_TRANS_CONV_INSTALL_IGNOREPKG callback function
- download callback API changed: alpm_cb_download, alpm_cb_totaldl split
  (+ new alpm_option_get_totaldlcb(), alpm_option_set_totaldlcb() functions)
- unsigned long->off_t changes where size is used
- pmsyncpkg_t struct changes:
  - pmsynctype_t and alpm_sync_get_type() were removed
  - alpm_sync_get_data() was removed
  - alpm_sync_get_removes() was added

[ADDED]
- alpm_delta_get_from_md5sum(), alpm_delta_get_to_md5sum()
- alpm_miss_get_causingpkg() (new causingpkg field in pmdepmissing_t)
- alpm_checkdbconflicts()
- alpm_sync_newversion()
- alpm_deptest()
- error codes :
    PM_ERR_DLT_INVALID, PM_ERR_LIBARCHIVE, PM_ERR_LIBDOWNLOAD and
    PM_ERR_EXTERNAL_DOWNLOAD
- flags:
    PM_TRANS_FLAG_ALLEXPLICIT, PM_TRANS_FLAG_UNNEEDED and
    PM_TRANS_FLAG_RECURSEALL


API CHANGES BETWEEN 3.2 AND 3.3
===============================

[REMOVED]
- pmsyncpkg_t struct (pmpkg_t is used for all types of transaction targets):
  - alpm_sync_get_pkg()
  - alpm_sync_get_removes() (use alpm_pkg_get_removes() instead)
- HoldPkg handling (it is the front-end's task):
  - alpm_option_get_holdpkgs()
  - alpm_option_add_holdpkg()
  - alpm_option_set_holdpkgs()
  - alpm_option_remove_holdpkg()
  - PM_TRANS_CONV_REMOVE_HOLDPKG conversation
- Print URIs feature (it is the front-end's task):
  - flag: PM_TRANS_FLAG_PRINTURIS
  - event: PM_TRANS_EVT_PRINTURI
- alpm_delta_get_from_md5sum() and alpm_delta_get_to_md5sum()
- alpm_sync_sysupgrade()
- error codes:
    PM_ERR_TRANS_COMMITING, PM_ERR_TRANS_DOWNLOADING, PM_ERR_PKG_LOAD,
    PM_ERR_PKG_CANT_FRESH, PM_ERR_GRP_NOT_FOUND, PM_ERR_USER_ABORT,
    PM_ERR_INTERNAL_ERROR, PM_ERR_DB_SYNC, PM_ERR_PKG_HOLD and
    PM_ERR_LIBDOWNLOAD

[CHANGED]
- XferCommand support was removed, any fetch callback function can be defined:
  - alpm_option_get_xfercommand() and alpm_option_set_xfercommand() were removed
  - alpm_option_get_fetchcb() and alpm_option_set_fetchcb() were added
- function renames:
  - alpm_db_getpkgcache() -> alpm_db_get_pkgcache()
  - alpm_db_getgrpcache() -> alpm_db_get_grpcache()
  - alpm_dep_get_string() -> alpm_dep_compute_string()
  - alpm_get_md5sum() -> alpm_compute_md5sum()
  - alpm_checkdbconflicts() -> alpm_checkconflicts()
- alpm_trans_sysupgrade() has a new enable_downgrade parameter
- alpm_checkdeps() and alpm_checkconflicts() require local package list instead
  of local database
- the to-be-upgraded package is passed to the callback function with
  PM_TRANS_EVT_UPGRADE_START (as the second parameter)
- the "requiredby" package is never passed to the callback function with
  PM_TRANS_CONV_INSTALL_IGNOREPKG (the second parameter is always NULL)

[ADDED]
- alpm_pkg_get_db()
- alpm_pkg_get_removes()
- conversation: PM_TRANS_CONV_REMOVE_PKGS (remove unresolvable targets)
- flag: PM_TRANS_FLAG_NOLOCK (do not lock database)
- error codes:
    PM_ERR_SERVER_NONE, PM_ERR_TRANS_NOT_LOCKED, PM_ERR_PKG_IGNORED and
    PM_ERR_LIBFETCH


API CHANGES BETWEEN 3.3 AND 3.4
===============================

[REMOVED]
- pmtranstype_t struct (transaction type), alpm_trans_get_type()
- alpm_option_get_nopassiveftp(), alpm_option_set_nopassiveftp()

[CHANGED]
- interface for target loading:
  - alpm_trans_addtarget() and alpm_trans_sysupgrade() were removed
  - alpm_sync_target() and alpm_sync_dbtarget() can be used to add a sync target
  - alpm_sync_sysupgrade() can be used to add outdated packages (for sysupgrade)
  - alpm_add_target() can be used to add an add/upgrade target
  - alpm_remove_target() can be used to add a remove target
- interface for target listing:
  - alpm_trans_get_pkgs() was removed
  - alpm_pkg_get_removes() was removed
  - alpm_trans_get_add() can be used to list add/upgrade/sync targets
  - alpm_trans_get_remove() can be used to list to-be-removed packages
- the type parameter of alpm_trans_init() was removed
- the type of alpm_db_fetch callback function: mtimeold and mtimenew parameters
  were replaced by force parameter
- unsigned short -> int changes for Boolean variables

[ADDED]
- alpm_db_set_pkgreason()
- alpm_option_get_arch(), alpm_option_set_arch()
- alpm_option_get_usedelta()
- alpm_pkg_unused_deltas()
- alpm_conflict_get_reason()
- error code: PM_ERR_PKG_INVALID_ARCH