Add repro-finish command

[cygport.git] / README

cygport is the standard method for building and maintaining packages for
the Cygwin distribution.


1. BACKGROUND

In the past, the sanctioned way of building a Cygwin package was by
using a build script, the template for which is commonly known as the
generic-build-script, or g-b-s[2].

However, the g-b-s has many drawbacks:

1) Even the simplest changes, such as adding an argument to configure,
   requires wading through the entire 500+ line script; many beginners 
   have struggled with their first packages.
   
2) Having a history as a ash script, it is nearly impossible to read, with
   run-on commands, && continuations, and backslash-escaped line breaks.

3) It is aimed primarily at autotooled packages, and adapting it to
   other packaging systems (e.g. perl MakeMaker, python Distutils,
   stand-alone Makefiles, etc.) requires having several "branches" of the
   script for each purpose.
   
4) Updating to a newer script requires either merging package-specific
   changes into a new template (and not forgetting anything), or
   back-porting g-b-s changes into an existing package script.
   
5) There is no simple method of dealing with an original source package
   whose name or top directory do not conform with the PKGNAME-VERSION
   form, or where the Cygwin package name differs from the upstream name
   (e.g. gtk2.0 from gtk+).
   
6) It is not designed to create more than one binary package per source
   package -- e.g.  whereby the foo source is separated into foo, libfoo1,
   and libfoo-devel binary packages -- despite the fact that is often
   desirable to do so.

Having used the g-b-s in various forms for a couple of years, during
which I've packaged hundreds of unique source packages (not counting
versions), I have found various ways of 'getting by'.  I forked the
g-b-s a while back, and created a few templates for various purposes. 
This solution, however, is still severely limiting, as mentioned above.

In the meantime, I also had exposure to a Linux distribution and its
packaging system, namely Gentoo Linux and Portage.  Being a source-based
distribution, the entire packaging database consists of a collection of
scripts (ebuilds), which contain basic information about the package,
and the steps necessary to compile and install (into a DESTDIR) the
package.  With the repository available through WebCVS, it is indeed an
invaluable resource to anyone building a non-trivial package from
scratch.

In fact, Portage answers many of the drawbacks of the g-b-s:

1) Ebuilds are small, containing only package-specific information,
   making them easy to read, and even to create with little effort.

2) Ebuilds are written in a clean bash syntax.

3) The Eclass system allows ebuilds to be easily extended to any number
   of different build systems (Perl, Python, Ruby, etc.) and/or categorical
   templates (e.g. GNOME or KDE).
   
4) Updates to Portage take affect immediately, without updating all
   ebuilds, and ebuilds can be created without even looking at the
   internals of Portage itself.
   
5) Ebuilds anyway define the source package name, and can easily
   override the assumed source top directory when necessary.

Despite it's advantages, implementing Portage on Cygwin is impractical,
because:

1) Portage is primarily a package management system, while Cygwin
   already uses setup.exe, and is impractical to use just for building
   packages.

2) Portage is mainly Linux/BSD specific in a number of ways which would
   not work for Cygwin, based on its Win32 limitations (such as replacing
   in-use files).

3) There is no way to create multiple binary packages from a single
   source package, without building the source multiple times.
   
4) Most importantly, setup.exe provides a GUI which makes installation
   easier for the uninitiated.

(It should be noted that, there were renewed attempts to run Portage
on Cygwin, and this was ITP'd and rejected[3] in 2006.  And, the project
"Gentoo on Cygwin", which contains Portage, is unmaintained as of 2008.
But, there is another project "Gentoo Prefix on Cygwin", as of 2020.
It's a try to run well-maintained "Gentoo Prefix" sources on Cygwin[4].)


2. CONCEPT

The conclusion, therefore, was that a compromise was required: take the
g-b-s, and separate the package-specific (Ebuild) and
package-independent (Portage) sections into two parts.  The
package-independent sections would become the build system, into which
the package-specific information would be fed to create a package.  The
package-independent system could then be cleaned up and expanded without
affecting package-specific information.

From this idea, cygport was created.

The cygport build system contains all the package-independent functions
of the g-b-s, rewritten in a modern, easy-to-read bash syntax.  It
provides commonly-used build-time functions for the package .cygport
files, which contain only the compiling, testing, and installation
instructions.

In addition, cygport is modularized.  Support for various build systems
is provided through separate cygclasses, which are 'inherit'ed by the
package .cygport as necessary.
   
The public functions and syntax (those used by a package .cygport) are
closer to those of Portage then of the g-b-s (it is extremely unlikely,
however, that a Gentoo ebuild will work as a .cygport with a simple
rename); internal syntax is still, to some degree, similar to the g-b-s.


3. USAGE

Similar to a g-b-s source package, a cygport-generated -src tarball will
contain the package .cygport, one or two patchfiles, and the original source
package, for example:

    foo-2.3.7.tar.gz
    foo2.cygport
    foo2-2.3.7-1.cygwin.patch
    foo2-2.3.7-1.src.patch     (will be absent if package builds OOTB)

GPG .sig files for any of the above may be present as well.  All these
files must remain in the same directory.

The general format of a cygport command is:

    cygport CYGPORT_FILE COMMAND [COMMAND2]

The first argument is the (relative or absolute) path to the .cygport file
to be processed.  All other arguments are interpreted as a COMMAND, which may be:

    prep      - create working directory, unpack the source and apply patches
    compile   - run all compilation steps
    test      - run the package's test suite, if one exists
    install   - install into a DESTDIR, and run post-installation steps
    package   - create binary and source packages
    upload    - upload finished packages to cygwin.com
    finish    - delete the working directory
    all       - run prep, compile, install and package

Other COMMANDs are meant primarily for maintainers:

    fetch    - download any missing upstream sources
    fetchall - download any all upstream sources
    check    - run the testsuite
    postinst - re-run post-installation steps
    list     - create a file listing suitable for the Cygwin README
    deps     - list direct dependencies of all executables
    info     - show a summary of package information
    diff     - write a patch file capturing changes to source in the working directory
    stage    - as upload, but don't request processing of uploaded packages
    announce - compose and send a package announcement
    repro-build  - rebuild from created source package to temp directory
    repro-diff   - check whether packages from original and rebuild differ
    repro-check  - run repro-build and repro-diff
    repro-finish - delete the temp directory used for rebuild

The standard arguments --help or --version may also be passed to cygport.


4. CREATING A CYGPORT PACKAGE

See the data/sample.cygport file, included in the Cygwin package in
${prefix}/share/cygport, for a simple example.  Many more examples
are available in the Cygwin package git repositories[5].

Please see the Cygport Reference Manual, included with this package, for
documentation on the cygport API.

A Cygwin README file may be included in the CYGWIN-PATCHES directory,
named README.  A standard template for this purpose is available from
the Cygwin distribution website[2].

If .hint file auto-generation isn't being used, setup.hint files may also be
included in CYGWIN-PATCHES.
    
Custom postinstall and preremove commands may be included in the
CYGWIN-PATCHES directory as postinstall.sh and preremove.sh; these
scripts should be written as stubs, without the shebang header.


5. REQUIREMENTS

The following packages are required to build packages with cygport, in
addition to the packages own dependencies:

    cygwin
    autoconf (wrapper) and autoconf2.*
    automake (wrapper) and automake1.*
    bash
    binutils
    bzip2
    coreutils
    diffstat
    diffutils
    dos2unix
    file
    gawk
    grep
    gzip
    lftp
    libtool
    lndir
    make
    openssh
    patch
    rsync
    sed
    tar
    unzip
    util-linux
    wget
    which
    xz

6. DOWNLOAD AND INSTALLATION

Cygwin binary and source packages of cygport are available as part of
the Cygwin distribution, under the 'Devel' category.  Installing cygport 
with setup.exe will automatically install all mandatory dependencies.

cygport is hosted on Git.  Those interested in helping with cygport
development, or testing the newest features should use branch 'master'.
The sources and history can be browsed at either:

  https://cygwin.com/git/gitweb.cgi?p=cygwin-apps/cygport.git
  https://github.com/cygwin/cygport

Patches for cygport should be created through 'git format-patch' and 
based on the 'master' branch.  The source repository can be cloned from:

  https://cygwin.com/git/cygwin-apps/cygport.git
  https://github.com/cygwin/cygport.git

To build cygport from git, meson and ninja are required. The following
packages are required for the documentation: groff, help2man, and robodoc.
The testsuite will use a number of packages if they are present; at a
minimum, you should have git-archive-all for the bootstrap test.


7. SUPPORT

Discussion on cygport should occur on the Cygwin-apps list
<cygwin-apps@cygwin.com>.  Do NOT, for any reason, email the author directly.


[1] https://www.cygwin.com/
[2] https://cygwin.com/setup-packaging-historical.html  Method Two
[3] https://sourceware.org/legacy-ml/cygwin-apps/2006-03/msg00000.html
[4] https://wiki.gentoo.org/wiki/Prefix/Cygwin
[5] https://cygwin.com/git-cygwin-packages/