Improve release process.

[tails/vicves.git] / wiki / src / contribute / release_process.mdwn

[[!meta title="Release process"]]

[[!toc levels=2]]

See the [[release_schedule]].

Environment
===========

Export the following environment variables to be able to copy'n'paste
the scripts snippets found on this page:

* `VERSION`: the version number of the release being prepared.
* `export TAG=$(echo "$VERSION" | sed -e 's,~,-,')`
* `PREVIOUS_VERSION`: the version number of the previous release.
* `ISOS`: the directory where one stores `tails-i386-*`
  sub-directories like the ones downloaded with BitTorrent.
* `MASTER_CHECKOUT`: a checkout of the `master` branch of the main
  Tails Git repository.

Pre-freeze
==========

upgrade i2p
-----------

See [[contribute/design/I2P]].

Going through the usual [[review and merge
process|contribute/merge_policy]] is welcome.

Upgrade Torbutton
-----------------

Since Torbutton is not maintained in Debian anymore,
upgrade our Debian package to the latest release if needed.

Going through the usual [[review and merge
process|contribute/merge_policy]] is welcome.

Update iceweasel preferences
----------------------------

* Torbutton does not handle `general.*` and
  `browser.startup.homepage_override.*` anymore, so we have to keep in
  sync manually, in our `/etc/iceweasel/pref/iceweasel.js`, with TBB's
  `pound_tor.js`
* update `extensions.adblockplus.currentVersion`

Coordinate with Debian security updates
---------------------------------------

See [[release_process/Debian_security_updates]].

Select the right branch
=======================

For a major release, the `devel` branch should be merged into the `testing`
branch and changes should be made from there.

From minor releases, work should happen in `stable`.

Update included files
=====================

AdBlock patterns
----------------

Patterns are stored in
`config/chroot_local-includes/etc/iceweasel/profile/adblockplus/`.

1. Boot Tails
2. Open *Tools* → *Addons*
3. Select *Adblock Plus* in extensions
4. Open *Preferences* → *Filter preferences…*
5. For each filters, click *Actions* → *Update filters*
6. Close Iceweasel
7. Copy the `.mozilla/firefox/default/adblockplus/patterns.ini` from
   this Iceweasel instance to the
   `config/chroot_local-includes/etc/iceweasel/profile/adblockplus`
   directory in the Tails Git checkout.
8. Commit:
       git commit -m 'Update AdBlock Plus patterns.' \
          config/chroot_local-includes/etc/iceweasel/profile/adblockplus/patterns.ini`

Upgrade bundled binary Debian packages
--------------------------------------

That is: make sure the bundled binary Debian packages contain
up-to-date localization files.

For each bundled Debian package, get updated PO files from the
Transifex branches of `git://git.torproject.org/translation.git` (e.g.
`whisperback_completed`). Verify and commit.

Then check the PO files [[using i18nspector|contribute/l10n_tricks]].
Correct all the errors that are not in the ignored list of
[[`check_po.sh`|contribute/l10n_tricks]]. Commit the changes if any.

Then see the relevant release processes:

* [[iceweasel]]
* [[liveusb-creator]]
* [[tails-greeter]]
* [[persistence-setup]]
* [[tails-iuk]]
* whisperback:
  * see release process on
    <https://git-tails.immerda.ch/whisperback/plain/HACKING>
  * build a debian package
  * upload it to [[APT repository]]

Update PO files
---------------

Copy `*.po` from the `tails-misc_completed` branch in
`https://git.torproject.org/translation.git` into `po/`, **but** skip
[[languages that are maintained in
Git|contribute/how/translate#translate]].

Refresh the code PO files, commit the result:

        ./refresh-translations && git commit po -m 'Update PO files.'

Freeze
------

If we are at freeze time for a major release:

* Merge the `master` Git branch into `devel`.
* Merge the `devel` Git branch into `testing`.
* Reset the `testing` APT suite to the state of the `devel` one, as
  documented on [[contribute/APT_repository#workflow-freeze]].

Else, if we are at freeze time for a point-release:

* Merge the `master` Git branch into `stable`.

Changelog
---------

Remove the placeholder entry for next release in `debian/changelog`,
and then:

        ./release $VERSION $PREVIOUS_VERSION

This populates the Changelog with the Git log entries.

Then, launch an editor for the needed cleanup of the result:

        dch -e

Then, gather other useful information from every custom bundled
package's own Changelog (Greeter, Persistent Volume Assistant, etc.).

Finally, commit:

        git commit debian/changelog -m "Update changelog for $VERSION."

Included website
----------------

### Merge master

Merge `master` into the branch used for the release:

        git fetch origin && git merge origin/master

### version number

If preparing a RC, skip this part.

In the branch used to build the release, update the `inc/*` files to
match the *version number* and *date* of the new release. Set the date
at least 24 hours in the future! Between tests and mirror synchronization,
the build will not be released on the same day. Try to make sure it
matches the date of the future signature.

        RELEASE_DATE='June 26, 2013'

        echo "$VERSION"      > wiki/src/inc/stable_i386_version.html
        echo "$RELEASE_DATE" > wiki/src/inc/stable_i386_date.html
        $EDITOR wiki/src/inc/*.html
        ./build-wiki
        git commit wiki/src/inc/ -m "Update version and date for $VERSION."

### features and design documentation

Read the Changelog carefully, and update [[doc/about/features]]
pages accordingly.

Also:

        git grep PENDING wiki/src/contribute/design*

... and remove the `PENDING-FOR-N.M` warnings.

Website translations
--------------------

Refresh the website PO files and commit the ones corresponding to
pages that were added or changed accordingly to changes coming with
the new release. This e.g. ensures that the RC call for translation
points translators to up-to-date PO files:

        ./build-wiki && git add wiki/src && git commit -m 'Update website PO files.'

If at freeze time, ask on tails-l10n that someone checks the PO files
of `po/` and of the website [[using
`check_po.sh`|contribute/l10n_tricks]], and corrects all the errors.

Tag the release in Git
======================

        git tag -u 1202821CBE2CD9C1 -m "tagging version ${VERSION}" "${TAG}"
        git push --tags

(Pushing the tag is needed so that the APT repository is updated, and
the Tails APT configuration works at build and boot time. It might be
premature, as testing might reveal critical issues, but this is
a signed tag, so it can be overridden later. Yes, there is room for
improvement here.)

Prepare the versioned APT suite
===============================

Follow the [[post-tag|contribute/APT_repository#workflow-post-tag]] APT
repository documentation.

Build images
============

Build the almost-final image
----------------------------

[[Build images|contribute/build]] and carefully read the build logs to
make sure nothing bad happened.

SquashFS file order
-------------------

1. Build an ISO image.
1. Burn a DVD.
1. Boot this DVD **on bare metal**.
1. Add `profile` to the kernel command-line.
1. Three minutes after `iceweasel` has been loaded, retrieve the new sort
   file from `/var/log/boot-profile`.
1. Copy the new sort file to `config/binary_rootfs/squashfs.sort`.
1. Cleanup a bit.
1. Inspect the Git diff (including diff stat), apply common sense.
1. `git commit -m 'Updating SquashFS sort file' config/binary_rootfs/squashfs.sort`

Build the final image
---------------------

Then all included files should be up-to-date and the versioned APT
suite should be ready, so it is time to:

* tag the release *again*, with all included files in
* `git push --tags`
*  build the final image!

Build the Incremental Update Kit
--------------------------------

Example:

    $ sudo tails-create-iuk --squashfs-diff-name 0.14.squashfs \
        --old-iso tails-i386-0.14\~rc2.iso \
        --new-iso tails-i386-0.14.iso      \
        --outfile Tails_i386_0.14-rc2_to_0.14.iuk

Prepare update-description files
--------------------------------

1. Prepare update-description files (see [[!tails_todo incremental_upgrades]]
   for details):
   * update (create if needed) an update-description file for every
     *previous* supported release (e.g. N-1, N-1~rc2) that describes
     the path to the one being released,
   * create a new update-description for the version being released,
     that expresses that no update is available for that one yet
1. Detach-sign the update-description files.
1. Rename the detached signatures from `.asc` to `.sig`.

Generate the OpenPGP signatures and Torrents
============================================

First, create a directory with a suitable name and go there:

        mkdir "$ISOS/tails-i386-$VERSION" && cd "$ISOS/tails-i386-$VERSION"

Second, copy the built image to this brand new directory.
Then, rename it:

        mv *.iso "tails-i386-$VERSION.iso"

Third, generate detached OpenPGP signatures for the image to be
published, in the same directory as the image and with a `.sig`
extension; e.g.

        gpg --armor --default-key 1202821CBE2CD9C1 --detach-sign *.iso
        rename 's,\.asc$,.sig,' *.asc

Fourth, go up to the parent directory, create a `.torrent` file and
check the generated `.torrent` files metainfo:

        cd .. && \
        mktorrent -a 'http://torrent.gresille.org/announce' "tails-i386-${VERSION}" && \
        btshowmetainfo tails-i386-$VERSION.torrent

Fifth, generate detached OpenPGP signatures for every published
`.torrent` file:

        gpg --armor --default-key 1202821CBE2CD9C1 --detach-sign \
          tails-i386-$VERSION.torrent && \
        mv tails-i386-$VERSION.torrent.{asc,sig}

Upload images
=============

Sanity check
------------

Verify that the current source for Firefox is still the same we've
used when preparing our custom Iceweasel package: e.g. FF17.0.8 got
re-tagged and re-uploaded at the last minute, due to a test failure.

Better catch this before people spend time doing manual tests.

## Announce, seed and test the Torrents

Announce and seed the Torrents.

Test them with a BitTorrent client running in a different place.

## Download and seed image from lizard

    scp tails-i386-$VERSION.torrent bittorrent.lizard:
    ssh bittorrent.lizard transmission-remote --add tails-i386-$VERSION.torrent

Publish the ISO over HTTP
-------------------------

Upload the images to the primary rsync mirror. Best practice is to first
let bittorrent.lizard download the image, and then copy it from there to
rsync.lizard:

    ssh lizard.tails.boum.org \
        scp -3 -r \
            bittorrent.lizard:/var/lib/transmission-daemon/downloads/tails-i386-$VERSION \
            rsync.lizard:
    # set DIST to either 'testing' (for RC:s) or 'stable' (for actual releases)
    ssh rsync.lizard << EOF
      chown -R root:rsync_tails tails-i386-$VERSION
      chmod -R u=rwX,go=rX tails-i386-$VERSION
      sudo mv tails-i386-$VERSION /srv/rsync/tails/tails/$DIST/
    EOF

Update the time in `project/trace` file on the primary rsync mirror
and on the live wiki (even for a release candidate):

        TRACE_TIME=$(date +%s) &&
        echo $TRACE_TIME | ssh rsync.lizard "cat > /srv/rsync/tails/tails/project/trace" && \
        [ -n "$MASTER_CHECKOUT" ] && \
        echo $TRACE_TIME > "$MASTER_CHECKOUT/wiki/src/inc/trace" &&
        (
           cd "$MASTER_CHECKOUT" && \
           git commit wiki/src/inc/trace -m "Updating trace file after uploading $VERSION."
        )

Wait for the HTTP mirrors to catch up
-------------------------------------

Wait for the next rsync pull.

Make sure every webserver listed in the `dl.amnesia.boum.org` round
robin pool has the new version. Drop those that are lagging behind and
notify their administrators.

Test downloading the ISO over HTTP.

Testing
=======

1. Set up a Gobby document and copy the [[test
   suite|contribute/release_process/test]] in it.
1. Email <tails@boum.org> to announce that tests may start:
   - make it clear what's the deadline
   - make it clear where and how you expect to get feedback
   - attach the Torrent
   - attach the `.packages` file
1. Make sure someone is committed to run the automated test suite.
1. Make sure that enough people are here to run the tests, that they
   report their results in due time, and that they make it clear when
   they're leaving for good.

While this happens, the release manager should prepare for the next
steps. Please read on!

Update the website and Git repository
=====================================

In order to get any new documentation into the website, merge either
`stable` or `testing` (depending on which release you just did) into
`master`.

Rename the `.packages` file to remove the `.iso` part of its name.

Copy the `.iso.sig`, `.packages`, `.torrent` and `.torrent.sig` files
into the Git repository's `wiki/src/torrents/files/` directory and
remove the one for the previous release (including any RCs). If
preparing an RC, only copy the `.iso.sig`.

Generate the SHA-256 hash of every image to be released in `inc/*`; e.g.

        sha256sum $ISOS/tails-i386-$VERSION/tails-i386-$VERSION.iso | \
          cut -f 1 -d ' ' | tr -d '\n' \
          > wiki/src/inc/stable_i386_hash.html

Update the [[support/known_issues]] page.

Write the announcement for the release in
`news/version_$VERSION.mdwn`, including:

- Update the `meta title` directive.
- Make sure there's an `announce` tag to have an email sent to the
  news mailing-list.
- Document important config changes that persistence users have to do
  themselves (e.g. the Pidgin proxy settings change in
  [[!tails_gitweb_commit 9925321]] breaks all existing persistent
  profiles).

Write an announcement listing the security bugs affecting the previous
version in `security/` in order to let the users of the old versions
know that they have to upgrade. Date it a few days before the ISO
image to be released was *built*. Including:

- the list of CVE fixed in Linux since the one shipped in the previous
  release of Tails:
  <http://ftp-master.metadata.debian.org/changelogs/main/l/linux/unstable_changelog>
- the list of DSA fixed in packages we ship since those that were in
  the previous release of Tails: <http://security.debian.org/>
- the list of MFSA fixed by the iceweasel update:
  <https://www.mozilla.org/security/announce/>

Generate PO files for the release and security announcements with
`./build-wiki`.
Then, send them to <tails-l10n@boum.org> so that they get translated
shortly, perhaps even soon enough to integrate them before pushing the
release out officially.

Import the update description files and their detached OpenPGP
signature into the `updates/` tree. See [[!tails_todo incremental_upgrades]]
for the file naming conventions.

Then, record the last commit before putting the release out for real:

        git commit -m "releasing version ${VERSION}"

Go wild!
========

Sanity check
------------

Verify once more that the current source for Firefox is still the same
we've used when preparing our custom Iceweasel packages.

Push
----

### Git

Push the last commits to our Git repository:

        git push

... and ask <root@boum.org> to refresh the ikiwiki wrappers for
our website.

Bug tracker
-----------

Mark all issues fixed in this release as `Status: Resolved` in our bug
tracker. For a list of candidates, see the [issues in *Fix committed*
status](https://labs.riseup.net/code/projects/tails/issues?query_id=111).

IRC
---

Update the topic in our [[chatroom|chat]].

Twitter
-------

Announce the release by tweeting a link to the "news" page.

Tor blog
--------

We announce *major* releases on the Tor blog:

- [login to their Drupal](https://blog.torproject.org/user)
- *Add a New Blog Post*
- add the same tags as the previous release announce had
- choose *Filtered HTML* as the *Input format*
- paste the HTML generated by ikiwiki from the announce in `news/`
  into the textarea in the blog post editor
- cleanup a bit to make it shorter
- add a link to our [[download page|download]]
- change the internal links into external links
- turn `<h1>` into `<strong>`
- direct users to [[our communication channels|support/talk]] for
  comments and feedback
- disable comments

Tor weekly news
---------------

Write a short announcement for the Tor weekly news, or find someone
who's happy to do it.

Prepare for the next release
============================

1. Move the previous stable release to `obsolete` on the mirrors.
1. Remove any remaining RC for the just-published release from
   the mirrors.
1. Pull `master` back and merge it into `devel`.
1. Follow the
   [[post-release|contribute/APT_repository#workflow-post-release]] APT
   repository documentation.
1. Push the resulting `devel` branch.
1. If this was a major release, then reset experimental:
   - take note of branches merged into `experimental`, but not into
     `devel`:

        git log --pretty=oneline --color=never --merges devel..experimental \
          | /bin/grep 'into experimental$' \
          | sed -e 's,^[a-f0-9]\+\s\+,,' | sort -u

   - `git checkout experimental && git reset --hard devel`
   - [[hard reset|contribute/APT_repository#workflow-reset]] the `experimental`
     APT suite to the state of the `devel` one.
   - merge additional branches into experimental (both at the Git and
     APT levels)

        for branch in $UNMERGED_BRANCHES ; do
           suite=$(echo $branch | sed -e 's,[/_],-,g')
           ssh reprepro@incoming.deb.tails.boum.org tails-merge-suite $suite experimental
        done

     Note that the merge will fail for suites that contain no package.

   - `git push --force origin experimental`
1. Push the release tag to lizard for Jenkins' consumption: `git push --tags lizard`
1. Manually trigger Jenkins builds of experimental, devel and stable:
   <https://jenkins.tails.boum.org/>.
1. Bonus points if you care about Tails 1.1: update the Iceweasel
   packages for feature/wheezy, and trigger a Jenkins build thereof.

Related pages
=============

[[!map pages="contribute/release_process/*"]]