Add more statistics in the automated builds blueprint.

[tails/matsa.git] / wiki / src / blueprint / automated_builds_and_tests / autobuild_specs.mdwn

[[!meta title="Automated builds specification"]]


This blueprint helps to keep track of the discussion on the mailing
list, and is attached to [[!tails_ticket 8655]]
to specify how to implement [[!tails_ticket 6196]] ("Build all active
branches").

Some metrics about the number of branches merged per releases are
available on the [[dedicated statistics page|automated_stats]].

[[!toc levels=2]]

# Question to discuss

## Which branches we want to build?

We already build the base branches (_stable_, _testing_, _devel_ and
_experimental_) + _feature/jessie_.

The questions raised is mostly concern the _feature/*_ and _bugfix/*_ branches
(so _topic branches_)

Given an ISO build takes around 45 minutes on lizard (worst case),
given two builders lizard will be able to build something like 64 ISOs a
day.

Developers should be able to trigger automatic builds for a branch whose build
was dropped (eg. last commit too old) by pushing a dumb commit on a
timestamp file in that branch.


Proposal1:

* branches which are not merged into devel, stable and testing
* but had new commits since the previous release


## When to build it

Define the regularity we want to build topic branches, apart from being build
on Git push.

Note that we will have to plug that in automatic tests when they will be
deployed.

Proposal 1: A build a day.


## Notifications

When to notify who? And how to notify them?

Proposal 1: Notify by email the author of the offending commit on failure.


# Scenarios

In the following scenario:

0. topic branches are named branch F
0. base branches are named branch B
0. builds are ran on merges which don't raise a conflict. If the merge raises a conflict, then the topic branch's developer should take care of resolving it.


## Scenario 1 : reviewer

    As a reviewer
    When I'm asked to review branch F into branch B
    Then I need to know if branch F builds fine
      once merged into branch B (fresh results!)
    And I should be notified of the results
    And if the build succeeded
      I might want to download the resulting ISO
      I might want to get the pkg list
      I want the Redmine ticket to be notified (optional)
    Otherwise if it fails the developer who proposed the merge should be notified
      And the developer *needs* to see the build logs
      And the ticket should be reassigned to the branch submitter
      And QA check should be set to "Dev needed"


## Scenario 2 : developer

    As a developer who has the commit bit
    When I'm working on branch F
    Then I need to know if my branch builds after I've pushed
    And I need to know if my branch build is broken by something else
       possibly weeks after my last commit (by e.g Debian changes,
       changes in branch B, ...)
    And if the build succeeded
      I might want to download the resulting ISO
      I might want to get the pkg list
      I want the Redmine ticket to be notified (optional)
    Otherwise if it fails I *need* to see the build logs
      And the developer who proposed the merge should be notified
      And the ticket should be reassigned to the branch submitter
      And QA check should be set to "Dev needed"


## Scenario 3 : RM

    As the current RM
    When working the full dev release cycle
    Then I need to know when a branch FTBFS
    And when this happens I need to see the build logs.


# Future ideas

This list other scenarios not part of the first deployement iteration, but we
might want to consider it in the future.

## Scenario 10

    As a Tails developer working on branch B
    When I upload a package to APT suite B
    Then I want to know if it broke the build ASAP

(same responsiveness as when pushing to git)
(acceptable workaround: being able to manually trigger a build.)


## Scenario 11

    As the current RM
    When I push new tag T on branch B
    Then I want the APT suite for tag T to be created
    And I want the APT suite B to be copied into the APT suite T
    And once this is done, I want a build from the checkout of tag T to be
      triggered
    And I want the squashfs sort file to be generated, and the diff sent to me


## Scenario 12

    As a Tails developer
    When the test suite is ran on the ISO build from my last commit
    I want to watch TV and see the test video in HTML5 from Tor Browser


## Scenario 13

    As a Tails developer
    When an ISO is build from my last commit
    I want to access it through remote desktop (VNC/Spice/...) over Tor

# Statistics

As of 2015-02-02, there are 26 branches that would be automatically
build as part of the next 1.3 release, following the for now defined
criterias (above in this blueprint):

* feature/7779-revisit-touchpad-settings
* feature/6992-whisperback-email-address
* bugfix/8714-tor-is-ready-robustness
* bugfix/8680-git-without-polipo
* feature/8719-mount-output-in-bug-reports
* feature/6241-gnupg2
* feature/8725-remove-vagrant-bootstrap-cache
* bugfix/8715-build-system-independent-APT-sources
* feature/7756-reintroduce-whisperback
* bugfix/8699-only-create-timestamps-in-Jenkins
* feature/8740-new-signing-key-phase-2
* feature/8665-remove-adblock
* bugfix/8756-repair-local-packages
* feature/7530-docker_anonym
* feature/7530-docker-with-apt-cacher-ng
* feature/7963-background-color
* feature/8491-live-additional-software-in-whisperback
* feature/7530-docker
* feature/linux-3.18
* feature/torbrowser-alpha
* bugfix/8747-update-tails-apt-repo-signing-key
* feature/8726-use-homogenous-Debian-mirrors-at-build-time
* feature/5525-sandbox-web-browser
* feature/7752-keyringer
* feature/6739-install-electrum
* bugfix/quote-wrappers-arguments