4 *Setup instructions for a Gitian build of Bitcoin Core using a Debian VM or physical system.*
6 Gitian is the deterministic build process that is used to build the Bitcoin
7 Core executables. It provides a way to be reasonably sure that the
8 executables are really built from the source on GitHub. It also makes sure that
9 the same, tested dependencies are used and statically built into the executable.
11 Multiple developers build the source code by following a specific descriptor
12 ("recipe"), cryptographically sign the result, and upload the resulting signature.
13 These results are compared and only if they match, the build is accepted and uploaded
16 More independent Gitian builders are needed, which is why this guide exists.
17 It is preferred you follow these steps yourself instead of using someone else's
18 VM image to avoid 'contaminating' the build.
23 - [Create a new VirtualBox VM](#create-a-new-virtualbox-vm)
24 - [Connecting to the VM](#connecting-to-the-vm)
25 - [Setting up Debian for Gitian building](#setting-up-debian-for-gitian-building)
26 - [Installing Gitian](#installing-gitian)
27 - [Setting up the Gitian image](#setting-up-the-gitian-image)
28 - [Getting and building the inputs](#getting-and-building-the-inputs)
29 - [Building Bitcoin Core](#building-bitcoin-core)
30 - [Building an alternative repository](#building-an-alternative-repository)
31 - [Signing externally](#signing-externally)
32 - [Uploading signatures](#uploading-signatures)
34 Preparing the Gitian builder host
35 ---------------------------------
37 The first step is to prepare the host environment that will be used to perform the Gitian builds.
38 This guide explains how to set up the environment, and how to start the builds.
40 Debian Linux was chosen as the host distribution because it has a lightweight install (in contrast to Ubuntu) and is readily available.
41 Any kind of virtualization can be used, for example:
42 - [VirtualBox](https://www.virtualbox.org/) (covered by this guide)
43 - [KVM](http://www.linux-kvm.org/page/Main_Page)
44 - [LXC](https://linuxcontainers.org/), see also [Gitian host docker container](https://github.com/gdm85/tenku/tree/master/docker/gitian-bitcoin-host/README.md).
46 You can also install Gitian on actual hardware instead of using virtualization.
48 Create a new VirtualBox VM
49 ---------------------------
50 In the VirtualBox GUI click "New" and choose the following parameters in the wizard:
52 ![](gitian-building/create_new_vm.png)
54 - Type: Linux, Debian (64-bit)
56 ![](gitian-building/create_vm_memsize.png)
58 - Memory Size: at least 3000MB, anything less and the build might not complete.
60 ![](gitian-building/create_vm_hard_disk.png)
62 - Hard Disk: Create a virtual hard disk now
64 ![](gitian-building/create_vm_hard_disk_file_type.png)
66 - Hard Disk file type: Use the default, VDI (VirtualBox Disk Image)
68 ![](gitian-building/create_vm_storage_physical_hard_disk.png)
70 - Storage on physical hard disk: Dynamically Allocated
72 ![](gitian-building/create_vm_file_location_size.png)
74 - File location and size: at least 40GB; as low as 20GB *may* be possible, but better to err on the safe side
77 After creating the VM, we need to configure it.
79 - Click the `Settings` button, then go to `System` tab and `Processor` sub-tab. Increase the number of processors to the number of cores on your machine if you want builds to be faster.
81 ![](gitian-building/system_settings.png)
83 - Go to the `Network` tab. Adapter 1 should be attached to `NAT`.
85 ![](gitian-building/network_settings.png)
87 - Click `Advanced`, then `Port Forwarding`. We want to set up a port through which we can reach the VM to get files in and out.
88 - Create a new rule by clicking the plus icon.
90 ![](gitian-building/port_forwarding_rules.png)
92 - Set up the new rule the following way:
97 - Leave Guest IP empty
100 - Click `Ok` twice to save.
102 Get the [Debian 8.x net installer](http://cdimage.debian.org/mirror/cdimage/archive/8.5.0/amd64/iso-cd/debian-8.5.0-amd64-netinst.iso) (a more recent minor version should also work, see also [Debian Network installation](https://www.debian.org/CD/netinst/)).
103 This DVD image can be [validated](https://www.debian.org/CD/verify) using a SHA256 hashing tool, for example on
104 Unixy OSes by entering the following in a terminal:
106 echo "ad4e8c27c561ad8248d5ebc1d36eb172f884057bfeb2c22ead823f59fa8c3dff debian-8.5.0-amd64-netinst.iso" | sha256sum -c
109 Then start the VM. On the first launch you will be asked for a CD or DVD image. Choose the downloaded ISO.
111 ![](gitian-building/select_startup_disk.png)
116 This section will explain how to install Debian on the newly created VM.
118 - Choose the non-graphical installer. We do not need the graphical environment; it will only increase installation time and disk usage.
120 ![](gitian-building/debian_install_1_boot_menu.png)
122 **Note**: Navigating in the Debian installer:
123 To keep a setting at the default and proceed, just press `Enter`.
124 To select a different button, press `Tab`.
126 - Choose locale and keyboard settings (doesn't matter, you can just go with the defaults or select your own information)
128 ![](gitian-building/debian_install_2_select_a_language.png)
129 ![](gitian-building/debian_install_3_select_location.png)
130 ![](gitian-building/debian_install_4_configure_keyboard.png)
132 - The VM will detect network settings using DHCP, this should all proceed automatically
133 - Configure the network:
135 - Leave domain name empty.
137 ![](gitian-building/debian_install_5_configure_the_network.png)
138 ![](gitian-building/debian_install_6_domain_name.png)
140 - Choose a root password and enter it twice (remember it for later)
142 ![](gitian-building/debian_install_6a_set_up_root_password.png)
144 - Name the new user `debian` (the full name doesn't matter, you can leave it empty)
145 - Set the account username as `debian`
147 ![](gitian-building/debian_install_7_set_up_user_fullname.png)
148 ![](gitian-building/debian_install_8_set_up_username.png)
150 - Choose a user password and enter it twice (remember it for later)
152 ![](gitian-building/debian_install_9_user_password.png)
154 - The installer will set up the clock using a time server; this process should be automatic
155 - Set up the clock: choose a time zone (depends on the locale settings that you picked earlier; specifics don't matter)
157 ![](gitian-building/debian_install_10_configure_clock.png)
160 - Partitioning method: Guided - Use the entire disk
162 ![](gitian-building/debian_install_11_partition_disks.png)
164 - Select disk to partition: SCSI1 (0,0,0)
166 ![](gitian-building/debian_install_12_choose_disk.png)
168 - Partition Disks -> *All files in one partition*
170 ![](gitian-building/all_files_in_one_partition.png)
172 - Finish partitioning and write changes to disk -> *Yes* (`Tab`, `Enter` to select the `Yes` button)
174 ![](gitian-building/debian_install_14_finish.png)
175 ![](gitian-building/debian_install_15_write_changes.png)
177 - The base system will be installed, this will take a minute or so
178 - Choose a mirror (any will do)
180 ![](gitian-building/debian_install_16_choose_a_mirror.png)
182 - Enter proxy information (unless you are on an intranet, leave this empty)
184 ![](gitian-building/debian_install_18_proxy_settings.png)
186 - Wait a bit while 'Select and install software' runs
187 - Participate in popularity contest -> *No*
188 - Choose software to install. We need just the base system.
189 - Make sure only 'SSH server' and 'Standard System Utilities' are checked
190 - Uncheck 'Debian Desktop Environment' and 'Print Server'
192 ![](gitian-building/debian_install_19_software_selection.png)
194 - Install the GRUB boot loader to the master boot record? -> Yes
196 ![](gitian-building/debian_install_20_install_grub.png)
198 - Device for boot loader installation -> ata-VBOX_HARDDISK
200 ![](gitian-building/debian_install_21_install_grub_bootloader.png)
202 - Installation Complete -> *Continue*
203 - After installation, the VM will reboot and you will have a working Debian VM. Congratulations!
205 ![](gitian-building/debian_install_22_finish_installation.png)
210 The next step in the guide involves logging in as root via SSH.
211 SSH login for root users is disabled by default, so we'll enable that now.
213 Login to the VM using username `root` and the root password you chose earlier.
214 You'll be presented with a screen similar to this.
216 ![](gitian-building/debian_root_login.png)
221 sed -i 's/^PermitRootLogin.*/PermitRootLogin yes/' /etc/ssh/sshd_config
223 and press enter. Then,
225 /etc/init.d/ssh restart
227 and enter to restart SSH. Logout by typing 'logout' and pressing 'enter'.
230 ----------------------
232 After the VM has booted you can connect to it using SSH, and files can be copied from and to the VM using a SFTP utility.
233 Connect to `localhost`, port `22222` (or the port configured when installing the VM).
234 On Windows you can use [putty](http://www.chiark.greenend.org.uk/~sgtatham/putty/download.html) and [WinSCP](http://winscp.net/eng/index.php).
236 For example, to connect as `root` from a Linux command prompt use
238 $ ssh root@localhost -p 22222
239 The authenticity of host '[localhost]:22222 ([127.0.0.1]:22222)' can't be established.
240 RSA key fingerprint is ae:f5:c8:9f:17:c6:c7:1b:c2:1b:12:31:1d:bb:d0:c7.
241 Are you sure you want to continue connecting (yes/no)? yes
242 Warning: Permanently added '[localhost]:22222' (RSA) to the list of known hosts.
243 root@localhost's password: (enter root password configured during install)
245 The programs included with the Debian GNU/Linux system are free software;
246 the exact distribution terms for each program are described in the
247 individual files in /usr/share/doc/*/copyright.
249 Debian GNU/Linux comes with ABSOLUTELY NO WARRANTY, to the extent
250 permitted by applicable law.
253 Replace `root` with `debian` to log in as user.
255 Setting up Debian for Gitian building
256 --------------------------------------
258 In this section we will be setting up the Debian installation for Gitian building.
260 First we need to log in as `root` to set up dependencies and make sure that our
261 user can use the sudo command. Type/paste the following in the terminal:
264 apt-get install git ruby sudo apt-cacher-ng qemu-utils debootstrap lxc python-cheetah parted kpartx bridge-utils make ubuntu-archive-keyring curl
268 Then set up LXC and the rest with the following, which is a complex jumble of settings and workarounds:
271 # the version of lxc-start in Debian needs to run as root, so make sure
272 # that the build script can execute it without providing a password
273 echo "%sudo ALL=NOPASSWD: /usr/bin/lxc-start" > /etc/sudoers.d/gitian-lxc
274 echo "%sudo ALL=NOPASSWD: /usr/bin/lxc-execute" >> /etc/sudoers.d/gitian-lxc
275 # make /etc/rc.local script that sets up bridge between guest and host
276 echo '#!/bin/sh -e' > /etc/rc.local
277 echo 'brctl addbr br0' >> /etc/rc.local
278 echo 'ifconfig br0 10.0.3.2/24 up' >> /etc/rc.local
279 echo 'iptables -t nat -A POSTROUTING -o eth0 -j MASQUERADE' >> /etc/rc.local
280 echo 'echo 1 > /proc/sys/net/ipv4/ip_forward' >> /etc/rc.local
281 echo 'exit 0' >> /etc/rc.local
282 # make sure that USE_LXC is always set when logging in as debian,
283 # and configure LXC IP addresses
284 echo 'export USE_LXC=1' >> /home/debian/.profile
285 echo 'export GITIAN_HOST_IP=10.0.3.2' >> /home/debian/.profile
286 echo 'export LXC_GUEST_IP=10.0.3.5' >> /home/debian/.profile
290 At the end the VM is rebooted to make sure that the changes take effect. The steps in this
291 section only need to be performed once.
296 Re-login as the user `debian` that was created during installation.
297 The rest of the steps in this guide will be performed as that user.
299 There is no `python-vm-builder` package in Debian, so we need to install it from source ourselves,
302 wget http://archive.ubuntu.com/ubuntu/pool/universe/v/vm-builder/vm-builder_0.12.4+bzr494.orig.tar.gz
303 echo "76cbf8c52c391160b2641e7120dbade5afded713afaa6032f733a261f13e6a8e vm-builder_0.12.4+bzr494.orig.tar.gz" | sha256sum -c
304 # (verification -- must return OK)
305 tar -zxvf vm-builder_0.12.4+bzr494.orig.tar.gz
306 cd vm-builder-0.12.4+bzr494
307 sudo python setup.py install
311 **Note**: When sudo asks for a password, enter the password for the user *debian* not for *root*.
313 Clone the git repositories for bitcoin and Gitian.
316 git clone https://github.com/devrandom/gitian-builder.git
317 git clone https://github.com/bitcoin/bitcoin
318 git clone https://github.com/bitcoin-core/gitian.sigs.git
321 Setting up the Gitian image
322 -------------------------
324 Gitian needs a virtual image of the operating system to build in.
325 Currently this is Ubuntu Trusty x86_64.
326 This image will be copied and used every time that a build is started to
327 make sure that the build is deterministic.
328 Creating the image will take a while, but only has to be done once.
330 Execute the following as user `debian`:
334 bin/make-base-vm --lxc --arch amd64 --suite trusty
337 There will be a lot of warnings printed during the build of the image. These can be ignored.
339 **Note**: When sudo asks for a password, enter the password for the user *debian* not for *root*.
341 Getting and building the inputs
342 --------------------------------
344 At this point you have two options, you can either use the automated script (found in [contrib/gitian-build.sh](/contrib/gitian-build.sh)) or you could manually do everything by following this guide. If you're using the automated script, then run it with the "--setup" command. Afterwards, run it with the "--build" command (example: "contrib/gitian-build.sh -b signer 0.13.0"). Otherwise ignore this.
346 Follow the instructions in [doc/release-process.md](release-process.md#fetch-and-create-inputs-first-time-or-when-dependency-versions-change)
347 in the bitcoin repository under 'Fetch and create inputs' to install sources which require
348 manual intervention. Also optionally follow the next step: 'Seed the Gitian sources cache
349 and offline git repositories' which will fetch the remaining files required for building
352 Building Bitcoin Core
355 To build Bitcoin Core (for Linux, OS X and Windows) just follow the steps under 'perform
356 Gitian builds' in [doc/release-process.md](release-process.md#perform-gitian-builds) in the bitcoin repository.
358 This may take some time as it will build all the dependencies needed for each descriptor.
359 These dependencies will be cached after a successful build to avoid rebuilding them when possible.
361 At any time you can check the package installation and build progress with
364 tail -f var/install.log
365 tail -f var/build.log
368 Output from `gbuild` will look something like
370 Initialized empty Git repository in /home/debian/gitian-builder/inputs/bitcoin/.git/
371 remote: Counting objects: 57959, done.
372 remote: Total 57959 (delta 0), reused 0 (delta 0), pack-reused 57958
373 Receiving objects: 100% (57959/57959), 53.76 MiB | 484.00 KiB/s, done.
374 Resolving deltas: 100% (41590/41590), done.
375 From https://github.com/bitcoin/bitcoin
376 ... (new tags, new branch etc)
377 --- Building for trusty amd64 ---
378 Stopping target if it is up
379 Making a new image copy
382 Checking if target is up
383 Preparing build environment
384 Updating apt-get repository (log in var/install.log)
385 Installing additional packages (log in var/install.log)
386 Grabbing package manifest
388 Creating build script (var/build-script)
389 lxc-start: Connection refused - inotify event with no name (mask 32768)
390 Running build script (log in var/build.log)
392 Building an alternative repository
393 -----------------------------------
395 If you want to do a test build of a pull on GitHub it can be useful to point
396 the Gitian builder at an alternative repository, using the same descriptors
401 URL=https://github.com/laanwj/bitcoin.git
402 COMMIT=2014_03_windows_unicode_path
403 ./bin/gbuild --commit bitcoin=${COMMIT} --url bitcoin=${URL} ../bitcoin/contrib/gitian-descriptors/gitian-linux.yml
404 ./bin/gbuild --commit bitcoin=${COMMIT} --url bitcoin=${URL} ../bitcoin/contrib/gitian-descriptors/gitian-win.yml
405 ./bin/gbuild --commit bitcoin=${COMMIT} --url bitcoin=${URL} ../bitcoin/contrib/gitian-descriptors/gitian-osx.yml
408 Building fully offline
409 -----------------------
411 For building fully offline including attaching signatures to unsigned builds, the detached-sigs repository
412 and the bitcoin git repository with the desired tag must both be available locally, and then gbuild must be
413 told where to find them. It also requires an apt-cacher-ng which is fully-populated but set to offline mode, or
414 manually disabling gitian-builder's use of apt-get to update the VM build environment.
416 To configure apt-cacher-ng as an offline cacher, you will need to first populate its cache with the relevant
417 files. You must additionally patch target-bin/bootstrap-fixup to set its apt sources to something other than
418 plain archive.ubuntu.com: us.archive.ubuntu.com works.
423 export PATH="$PATH":/path/to/gitian-builder/libexec
425 cd /path/to/gitian-builder
426 ./libexec/make-clean-vm --suite trusty --arch amd64
428 LXC_ARCH=amd64 LXC_SUITE=trusty on-target -u root apt-get update
429 LXC_ARCH=amd64 LXC_SUITE=trusty on-target -u root \
430 -e DEBIAN_FRONTEND=noninteractive apt-get --no-install-recommends -y install \
431 $( sed -ne '/^packages:/,/[^-] .*/ {/^- .*/{s/"//g;s/- //;p}}' ../bitcoin/contrib/gitian-descriptors/*|sort|uniq )
432 LXC_ARCH=amd64 LXC_SUITE=trusty on-target -u root apt-get -q -y purge grub
433 LXC_ARCH=amd64 LXC_SUITE=trusty on-target -u root -e DEBIAN_FRONTEND=noninteractive apt-get -y dist-upgrade
436 And then set offline mode for apt-cacher-ng:
439 /etc/apt-cacher-ng/acng.conf
444 service apt-cacher-ng restart
447 Then when building, override the remote URLs that gbuild would otherwise pull from the Gitian descriptors::
451 git clone https://github.com/bitcoin-core/bitcoin-detached-sigs.git
453 BTCPATH=/some/root/path/bitcoin
454 SIGPATH=/some/root/path/bitcoin-detached-sigs
456 ./bin/gbuild --url bitcoin=${BTCPATH},signature=${SIGPATH} ../bitcoin/contrib/gitian-descriptors/gitian-win-signer.yml
462 If you want to do the PGP signing on another device, that's also possible; just define `SIGNER` as mentioned
463 and follow the steps in the build process as normal.
465 gpg: skipped "laanwj": secret key not available
467 When you execute `gsign` you will get an error from GPG, which can be ignored. Copy the resulting `.assert` files
468 in `gitian.sigs` to your signing machine and do
471 gpg --detach-sign ${VERSION}-linux/${SIGNER}/bitcoin-linux-build.assert
472 gpg --detach-sign ${VERSION}-win/${SIGNER}/bitcoin-win-build.assert
473 gpg --detach-sign ${VERSION}-osx-unsigned/${SIGNER}/bitcoin-osx-build.assert
476 This will create the `.sig` files that can be committed together with the `.assert` files to assert your
480 ---------------------
482 After building and signing you can push your signatures (both the `.assert` and `.assert.sig` files) to the
483 [bitcoin-core/gitian.sigs](https://github.com/bitcoin-core/gitian.sigs/) repository, or if that's not possible create a pull
484 request. You can also mail the files to Wladimir (laanwj@gmail.com) and he will commit them.