Update documentation for 6.3.0
[egit/eclipse.git] / org.eclipse.egit.doc / help / EGit / Contributor_Guide / Builds.html
blobef967d27858f324497d648d29cfd2672f56d73a2
1 <?xml version='1.0' encoding='utf-8' ?><!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
2 <html xmlns="http://www.w3.org/1999/xhtml">
3 <head>
4 <meta http-equiv="Content-Type" content="text/html; charset=utf-8"/>
5 <title>EGit Contributor Guide - Builds</title>
6 <link type="text/css" rel="stylesheet" href="../../book.css"/>
7 </head>
8 <body>
9 <table class="navigation" style="width: 100%;" border="0" summary="navigation">
10 <tr>
11 <th style="width: 100%" align="center" colspan="3">Builds</th>
12 </tr>
13 <tr>
14 <td style="width: 20%" align="left">
15 <a href="Running-EGit-from-Eclipse.html" title="Running EGit from Eclipse">
16 <img alt="Previous" border="0" src="../../images/prev.gif"/>
17 </a>
18 </td>
19 <td style="width: 60%" align="center"></td>
20 <td style="width: 20%" align="right">
21 <a href="Documentation.html" title="Documentation">
22 <img alt="Next" border="0" src="../../images/next.gif"/>
23 </a>
24 </td>
25 </tr>
26 <tr>
27 <td style="width: 20%" align="left" valign="top">Running EGit from Eclipse</td>
28 <td style="width: 60%" align="center"></td>
29 <td style="width: 20%" align="right" valign="top">Documentation</td>
30 </tr>
31 </table><hr class="navigation-separator"/>
32 <h1 id="Builds">Builds</h1>
33 <p>The central EGit and JGit builds run on the Jenkins build infrastructure provided by the Eclipse foundation.</p>
34 <ul>
35 <li>
36 <a href="https://ci.eclipse.org/egit/" target="egit_external">EGit Jenkins instance</a>
37 </li>
38 <li>
39 <a href="https://ci.eclipse.org/jgit/" target="egit_external">JGit Jenkins instance</a>
40 </li>
41 </ul>
42 <p>Prerequisites for the Maven build are</p>
43 <ul>
44 <li>since 6.0 Java 11, older versions need Java 8 </li>
45 <li>
46 <a href="https://maven.apache.org/download.html" target="egit_external">at least Maven 3.5.2</a>
47 </li>
48 <li>see
49 <a href="https://maven.apache.org/settings.html" target="egit_external">settings.xml reference</a> on how to do basic Maven configuration
50 </li>
51 <li>if you want to learn how Maven works start reading
52 <a href="https://maven.apache.org/guides/getting-started/index.html" target="egit_external">the Maven Getting Started Guide</a>
53 </li>
54 </ul>
55 <h2 id="JGit_2">JGit</h2>
56 <ul>
57 <li>JGit can be built using Maven or Bazel</li>
58 <li>use Java 11 to run the JGit build</li>
59 <li>JGit packaging projects (Eclipse features and p2 repository) are built using Maven and Tycho.</li>
60 </ul>
61 <h2 id="EGit_2">EGit</h2>
62 <ul>
63 <li>EGit is built using Maven and Tycho.</li>
64 </ul>
65 <h2 id="Mailing_Lists">Mailing Lists</h2>
66 <p>If you're interested in following builds, please check out the following mailing lists:</p>
67 <ul>
68 <li>
69 <a href="https://dev.eclipse.org/mailman/listinfo/jgit-build" target="egit_external">Subscribe to jgit-build@eclipse.org</a>
70 </li>
71 <li>
72 <a href="https://dev.eclipse.org/mailman/listinfo/egit-build" target="egit_external">Subscribe to egit-build@eclipse.org</a>
73 </li>
74 </ul>
75 <h2 id="Maven_Build">Maven Build</h2>
76 <p>To build JGit or EGit versions &lt;= 5.12.0 maven must be run using Java 1.8. Ensure that environment variable <tt>JAVA_HOME</tt> points to a Java 1.8 installation.</p>
77 <p>EGit versions &gt;= 5.12.1 and &lt; 6.0.0 can be built with any Java &gt;= 1.8, but the build uses
78 <a href="https://maven.apache.org/guides/mini/guide-using-toolchains.html" target="egit_external">maven toolchain</a> definitions. To run a local EGit build for these versions (including the 'master' branch, i.e. EGit nightly), you need to have a local file <tt>
79 <b>~/.m2/toolchains.xml</b></tt> providing at least a JDK toolchain definition for "JavaSE-1.8":
80 </p>
81 <pre style="width: 55em;">
82 &lt;?xml version="1.0" encoding="UTF8"?&gt;
83 &lt;toolchains&gt;
84 &lt;toolchain&gt;
85 &lt;type&gt;jdk&lt;/type&gt;
86 &lt;provides&gt;
87 &lt;id&gt;JavaSE-1.8&lt;/id&gt;
88 &lt;version&gt;1.8&lt;/version&gt;
89 &lt;/provides&gt;
90 &lt;configuration&gt;
91 &lt;jdkHome&gt;/absolute/path/to/jdk1.8&lt;/jdkHome&gt;
92 &lt;/configuration&gt;
93 &lt;/toolchain&gt;
94 &lt;toolchain&gt;
95 &lt;type&gt;jdk&lt;/type&gt;
96 &lt;provides&gt;
97 &lt;id&gt;JavaSE-11&lt;/id&gt;
98 &lt;version&gt;11&lt;/version&gt;
99 &lt;/provides&gt;
100 &lt;configuration&gt;
101 &lt;jdkHome&gt;/absolute/path/to/jdk11&lt;/jdkHome&gt;
102 &lt;/configuration&gt;
103 &lt;/toolchain&gt;
104 &lt;/toolchains&gt;
105 </pre>
106 <p>The <tt>jdkHome</tt> directory is the one that contains the <tt>bin</tt> directory in which there is the <tt>java</tt> executable.</p>
107 <p>Also include a definition for a Java 11 toolchain as shown above: EGit versions &gt;= 6.0.0 require at least Java 11 to build, and require a Java 11 toolchain definition for running the tests.</p>
108 <p>Building JGit versions &lt; 6.0.0 with maven requires running maven with Java 1.8 in all cases.</p>
109 <p>Building JGit versions &gt;= 6.0.0 with maven requires running maven and compiling with at least Java 11.</p>
110 <ul>
111 <li>Due to a
112 <a href="https://wiki.eclipse.org/Tycho/How_Tos/Dependency_on_pom-first_artifacts" target="egit_external">limitation of Tycho</a> it is not possible to mix pom-first and manifest-first builds in the same reactor build hence the pom-first JGit build has to run separately before the build for the manifest-first JGit packaging project.
113 </li>
114 <li>The local maven builds must share the same local Maven repository otherwise dependencies between these builds cannot be resolved.</li>
115 <li>To run the build behind a firewall follow
116 <a href="https://maven.apache.org/guides/mini/guide-proxies.html" target="egit_external">https://maven.apache.org/guides/mini/guide-proxies.html</a>
117 </li>
118 <li>To run the JGit maven build on a
119 <i>case-insensitive</i> file system, you might need to suppress javadoc generation if you get an error "class IO clashes with package of same name". Use "<tt>mvn clean install -Dmaven.javadoc.skip=true</tt>" instead of plain "<tt>mvn clean install</tt>".
120 </li>
121 </ul>
122 <p>Complete build sequence for a clean build (assuming $M2_HOME/bin is on the path and local Maven repository at ~/.m2/repository):</p>
123 <pre style="width: 55em;">
124 [~/src/jgit] $ mvn clean install
125 [INFO] Scanning for projects...
128 [~/src/jgit] $ mvn -f org.eclipse.jgit.packaging/pom.xml clean install
129 [INFO] Scanning for projects...
132 [~/src/jgit] $ cd ../egit
134 [~/src/egit] $ mvn clean install
135 [INFO] Scanning for projects...
138 [~/src/jgit] $ cd ../egit-github
140 [~/src/egit-github] $ mvn clean install
141 [INFO] Scanning for projects...
143 </pre>
144 <p>The EGit build uses the JGit p2 repository to resolve jgit dependencies. For local builds the build assumes
145 that egit and jgit source trees are located under a common parent folder. If this is not the case the path
146 to the jgit p2 repository has to be injected via system property:</p>
147 <pre>[~/src/egit] $ mvn clean install -Djgit-site=file:/path/to/jgit/org.eclipse.jgit.packaging/org.eclipse.jgit.repository/target/repository
148 </pre>
149 <p>in the same way you can configure a custom path for the build of egit-github to the egit p2 repository</p>
150 <pre>[~/src/egit-github] $ mvn clean install -Degit-site=file:/path/to/egit/org.eclipse.egit.repository/target/repository
151 </pre>
152 <p>The Jenkins build uses (for SNAPSHOT builds):</p>
153 <pre>[~/src/egit] $ mvn clean install -Djgit-site=<a href="https://repo.eclipse.org/content/unzip/snapshots.unzip/" target="egit_external">https://repo.eclipse.org/content/unzip/snapshots.unzip/</a>
154 org/eclipse/jgit/org.eclipse.jgit.repository/${JGIT_VERSION}/org.eclipse.jgit.repository-${JGIT_VERSION}.zip-unzip/
155 </pre>
156 <p>If you wan to build EGit for a specific Eclipse platform use the corresponding EGit target platform. For instance, to build for Eclipse 4.19 (2021-03), use the <code>egit-4.19</code> target platform:</p>
157 <pre>[~/src/egit] $ mvn -Dtarget-platform=egit-4.19 clean install
158 </pre>
159 <p>See the contents of bundle <code>org.eclipse.egit.target</code> for the available target platforms.</p>
160 <p>Upon a successful build, a p2 update site should be generated inside
161 <i>egit/org.eclipse.egit.repository/target/repository</i>. If not, make sure the target platform has been downloaded from within Eclipse (Windows&gt;Preferences&gt;Plug-in Development&gt;Target Platform). The default target platform defined in the maven build is currently Eclipse 4.6 (Neon). If you skip setting the system property <code>target-platform</code> the target platform for Eclipse 4.6 will be used. EGit built with that target platform can run on any Eclipse &gt;= 4.6. If you choose a higher target platform, the EGit produced may not run on Eclipses older than the target platform version.
162 </p>
163 <h2 id="Dependencies_and_License_Check">Dependencies and License Check</h2>
164 <p>In order to update the list of dependencies run the dash-licenses Maven plugin</p>
165 <pre style="width: 55em;">
166 mvn org.eclipse.dash:license-tool-plugin:license-check -Ddash.summary=DEPENDENCIES -Ddash.projectId=technology.jgit
167 mvn org.eclipse.dash:license-tool-plugin:license-check -Ddash.summary=DEPENDENCIES -Ddash.projectId=technology.egit
168 </pre>
169 <p>for jgit and egit and then sort the entries in the file <code>DEPENDENCIES</code> and check for changes against the previous version.
170 If necessary file CQs for new dependencies which are flagged
171 <i>restricted</i> by the tool.
172 See the
173 <a href="https://www.eclipse.org/projects/handbook/#ip-license-tool" target="egit_external">Eclipse projects handbook</a>.
174 </p>
175 <p>This tool requires Java 11.</p>
176 <h2 id="JGit_Bazel_Build">JGit Bazel Build</h2>
177 <p>Since Gerrit is built using
178 <a href="https://www.bazel.io/" target="egit_external">Bazel</a> a Bazel build was also implemented for JGit.
179 This simplifies working on Gerrit features which also require changes in JGit.
180 </p>
181 <ul>
182 <li>
183 <a href="https://www.bazel.io/versions/master/docs/install.html" target="egit_external">Install Bazel</a>
184 </li>
185 <li>To build all libraries run</li>
186 </ul>
187 <pre>bazel build :all
188 </pre>
189 <ul>
190 <li>The following test labels are supported: api, attributes, dfs, diff, http, lfs, lfs-server, nls, notes, pack, patch, pgm, reftree, revplot, revwalk, storage, submodule, symlinks, transport, treewalk, util</li>
191 <li>To run all tests execute</li>
192 </ul>
193 <pre>bazel test //...
194 </pre>
195 <ul>
196 <li>To run specific tests, using labels:</li>
197 </ul>
198 <pre>bazel test --test_tag_filters=api,dfs,revplot,treewalk //...
199 </pre>
200 <ul>
201 <li>to rerun all tests ignoring cached test results execute</li>
202 </ul>
203 <pre>bazel test //... --cache_test_results=NO
204 </pre>
205 <ul>
206 <li>to set number of concurrent test runs</li>
207 </ul>
208 <pre>bazel test //... --jobs=4
209 </pre>
210 <ul>
211 <li>to debug a test run</li>
212 </ul>
213 <pre>bazel test --test_output=streamed --test_filter=&lt;fully qualified test method&gt; &lt;test target&gt;
214 </pre>
215 <p>e.g.</p>
216 <pre>bazel test --test_output=streamed --test_filter=org.eclipse.jgit.api.GitConstructionTest.testClose //org.eclipse.jgit.test:org_eclipse_jgit_api_GitConstructionTest
217 </pre>
218 <ul>
219 <li>to configure loggers for test runs edit org.eclipse.jgit.test/tst-rsrc/simplelogger.properties, see the
220 <a href="https://www.slf4j.org/api/org/slf4j/impl/SimpleLogger.html" target="egit_external">slf4j SimpleLogger documentation</a>
221 </li>
222 <li>to run tests repeatedly use</li>
223 </ul>
224 <pre>bazel test --runs_per_test=3 &lt;test target&gt;
225 </pre>
226 <ul>
227 <li>since 5.4.0 builds run with
228 <a href="https://github.com/google/error-prone" target="egit_external">the errorprone static analyzer</a> by default. If you want to enable it for older JGit versions execute
229 </li>
230 </ul>
231 <pre>bazel build --java_toolchain //tools:error_prone_warnings_toolchain :all
232 </pre>
233 <p>Note that the Bazel build does not yet support building JGit OSGi bundles, Eclipse features and the p2 repository which are required to install JGit in Eclipse.</p>
234 <h2 id="Updating_build_version">Updating build version</h2>
235 <p>use the tools/version.sh script to update the build version in all build files, e.g.</p>
236 <pre style="width: 55em;">
237 ./tools/version.sh --snapshot=6.2.0-SNAPSHOT
238 </pre>
239 <h2 id="FindBugs_and_PMD">FindBugs and PMD</h2>
240 <p>As part of the build, JGit and EGit run FindBugs and PMD to find issues.</p>
241 <ul>
242 <li>
243 <a href="https://ci.eclipse.org/jgit/job/jgit/findbugs" target="egit_external">JGit FindBugs Results</a>
244 </li>
245 <li>
246 <a href="https://ci.eclipse.org/jgit/job/jgit/dry" target="egit_external">JGit DRY (PMD) Results</a>
247 </li>
248 <li>
249 <a href="https://ci.eclipse.org/egit/job/egit/findbugs" target="egit_external">EGit FindBugs Results</a>
250 </li>
251 <li>
252 <a href="https://ci.eclipse.org/egit/job/egit/dry" target="egit_external">EGit DRY (PMD) Results</a>
253 </li>
254 </ul>
255 <h2 id="Checking_for_JGit_API_Changes_using_API_Baseline">Checking for JGit API Changes using API Baseline</h2>
256 <p>The JGit projects have API tooling enabled. In order to use PDE API tools to get assistance with maintaining API changes and additions you need to set an API baseline:</p>
257 <ul>
258 <li>download the p2 repository for the latest EGit release (which includes the JGit artifacts) to a local folder, e.g. <code>~/egit-releases/updates-4.9.1</code>, find the p2 repository URLs
259 <a href="https://wiki.eclipse.org/EGit/FAQ#Where_can_I_find_older_releases_of_EGit.3F" target="egit_external">here</a> and download the p2 repository of the latest minor release (service releases don't change API) using the corresponding link in the last column of that table
260 </li>
261 <li>in Eclipse click "Preferences &gt; Plug-In Development &gt; API Baselines", click "Add Baseline..." and define a new baseline (e.g. egit-4.9.1) and point it to the local copy of the corresponding EGit p2 repository.</li>
262 <li>the API tools will then raise warning/errors for all detected problems and provide quick fixes helping to resolve these problems</li>
263 <li>see the
264 <a href="https://wiki.eclipse.org/PDE/API_Tools/User_Guide" target="egit_external">PDE API Tools User Guide</a> for more details.
265 </li>
266 </ul>
267 <h2 id="Signing_and_Publishing">Signing and Publishing</h2>
268 <p>EGit and JGit builds running on the JGit/EGit Jenkins instances are automatically signed
269 (using the
270 <a href="https://wiki.eclipse.org/Common_Build_Infrastructure#Signing_tool" title="Common_Build_Infrastructure#Signing_tool" target="egit_external">CBI eclipse-jarsigner-plugin</a>) and published to the folder
271 </p>
272 <pre>
273 master branch: /home/data/httpd/download.eclipse.org/egit/updates-nightly
274 latest stable branch: /home/data/httpd/download.eclipse.org/egit/updates-stable-nightly
275 </pre>
276 <ul>
277 <li>To enable signing the maven profile <code>eclipse-sign</code> must be enabled via the option <code>-P eclipse-sign</code> in the respective build jobs running at
278 <a href="https://ci.eclipse.org/egit/" target="egit_external">https://ci.eclipse.org/egit/</a>
279 </li>
280 </ul>
281 <h2 id="Creating_a_release">Creating a release</h2>
282 <p>Use the tools/release.sh script to create a new release, e.g.</p>
283 <pre style="width: 55em;">
284 ./tools/release.sh v6.1.0.202203080745-r
285 </pre>
286 <ul>
287 <li>release versions have 5 parts <pre>major.minor.patch.buildTimestamp-qualifier</pre>
288 <ul>
289 <li>Traditionally we use local time in EST timezone for the buildTimestamp part of the version number since our build servers run in that timezone.</li>
290 <li>qualifier is <pre>m1,m2, ...</pre> for milestones, <pre>rc1, rc2, ...</pre> for release candidates and <pre>r</pre> for releases</li>
291 </ul>
292 </li>
293 <li>We sign release tags, follow
294 <a href="https://git-scm.com/book/en/v2/Git-Tools-Signing-Your-Work" target="egit_external">Git Tools - Signing Your Work</a> to set this up on your computer before creating the first release.
295 </li>
296 <li>We create all milestones and releases on a dedicated stable branch to avoid interference with ongoing development on master. E.g. use the stable-6.1 branch for releasing 6.1.0 and subsequent patch releases like 6.1.1.</li>
297 <li>push the locally created release commit to git.eclipse.org for review</li>
298 <li>wait for the verification build to succeed and vote +1 on verified</li>
299 <li>review and submit the release change, then push the release tag to git.eclipse.org</li>
300 <li>the CI job will build the release version and deploy it to the Eclipse Maven repo at repo.eclipse.org</li>
301 </ul>
302 <h3 id="Release_Notes">Release Notes</h3>
303 <p>We create a release record for each release in the Eclipse project portal and publish release notes there.</p>
304 <ul>
305 <li>
306 <a href="https://projects.eclipse.org/projects/technology.jgit/governance" target="egit_external">JGit releases</a>
307 </li>
308 <li>
309 <a href="https://projects.eclipse.org/projects/technology.egit/governance" target="egit_external">EGit releases</a>
310 </li>
311 </ul>
312 <h3 id="New_and_Noteworthy">New and Noteworthy</h3>
313 <p>For major and minor releases (e.g. 6.0.0, 6.3.0) we create a New and Noteworthy page</p>
314 <ul>
315 <li>
316 <a href="https://wiki.eclipse.org/JGit/New_and_Noteworthy" target="egit_external">JGit New and Noteworthy</a>
317 </li>
318 <li>
319 <a href="https://wiki.eclipse.org/EGit/New_and_Noteworthy" target="egit_external">EGit New and Noteworthy</a>
320 </li>
321 </ul>
322 <h2 id="Contribution_to_Release_Train">Contribution to Release Train</h2>
323 <p>We participate in the
324 <a href="https://wiki.eclipse.org/Simultaneous_Release" target="egit_external">Eclipse simultaneous release</a> and its
325 <a href="https://wiki.eclipse.org/Category:SimRel-2022-09" target="egit_external">schedule</a>.
326 </p>
327 <p>The release train contribution for JGit and EGit is maintained in the git repository </p>
328 <pre>ssh://git.eclipse.org/gitroot/simrel/org.eclipse.simrel.build.git
329 </pre>
330 <p>in the files</p>
331 <pre>
332 simrel.aggr
333 egit.aggrcon
334 </pre>
335 <p>Checkout the master branch and update the versions of jgit and egit to be contributed and the URL of the new version.</p>
336 <p>Milestones (version ends with -m1, -m2, -m3) and release candidates (version ends with -rc1) are deployed on the download server to egit/staging/&lt;tag name&gt;, the final release (version ends with -r) goes to egit/updates-&lt;major version&gt;.&lt;minor version&gt;[.&lt;micro version&gt;]. The releases on the download server can be browsed
337 <a href="https://download.eclipse.org/justj/?file=egit" target="egit_external">here</a>. Old releases are moved to the
338 <a href="https://archive.eclipse.org/justj/?file=egit" target="egit_external">archive server</a>. Committers can trigger archiving
339 <a href="https://download.eclipse.org/egit/" target="egit_external">on this page</a>.
340 </p>
341 <p>All these builds are deployed to the corresponding release repositories on nexus.
342 Maven artefacts:</p>
343 <ul>
344 <li>
345 <a href="https://repo.eclipse.org/content/groups/releases/org/eclipse/jgit/" target="egit_external">jgit maven artefacts</a>
346 </li>
347 <li>
348 <a href="https://repo.eclipse.org/content/groups/releases/org/eclipse/egit/" target="egit_external">egit maven artefacts</a>
349 </li>
350 </ul>
351 <p>Corresponding p2 repositories are exposed dynamically using the Nexus Unzip Plugin:</p>
352 <ul>
353 <li>
354 <a href="https://repo.eclipse.org/content/unzip/releases.unzip/org/eclipse/jgit/" target="egit_external">jgit p2 repos</a>
355 </li>
356 <li>
357 <a href="https://repo.eclipse.org/content/unzip/releases.unzip/org/eclipse/egit/" target="egit_external">egit p2 repos</a>
358 </li>
359 </ul>
360 <p>Final releases of jgit are also deployed to Maven central using Sonatype's OSS Nexus using some scripts located in the jgit repository under tools/maven-central.</p>
361 <p>Open simrel.aggr using the
362 <a href="https://wiki.eclipse.org/CBI/aggregator" target="egit_external">CBI aggregator editor</a> to edit these files.
363 This editor knows the structure of these files and can check their validity.
364 </p>
365 <p>Push the change for review, review and submit it when the verification build job has voted +1 on verified.
366 Note that the simultaneous release repository only accepts fast-forward submits. This means if another project's change has been submitted you may have to rebase your change.</p>
367 <p>The release train build is coordinated on the
368 <a href="https://dev.eclipse.org/mailman/listinfo/cross-project-issues-dev" target="egit_external">cross-project-issues-dev mailing list</a>.
369 </p>
371 <br/>
372 </p><hr class="navigation-separator"/>
373 <table class="navigation" style="width: 100%;" border="0" summary="navigation">
374 <tr>
375 <td style="width: 20%" align="left">
376 <a href="Running-EGit-from-Eclipse.html" title="Running EGit from Eclipse">
377 <img alt="Previous" border="0" src="../../images/prev.gif"/>
378 </a>
379 </td>
380 <td style="width: 60%" align="center">
381 <a href="Contributor-Guide.html" title="EGit Contributor Guide">
382 <img alt="EGit Contributor Guide" border="0" src="../../images/home.gif"/>
383 </a>
384 </td>
385 <td style="width: 20%" align="right">
386 <a href="Documentation.html" title="Documentation">
387 <img alt="Next" border="0" src="../../images/next.gif"/>
388 </a>
389 </td>
390 </tr>
391 <tr>
392 <td style="width: 20%" align="left" valign="top">Running EGit from Eclipse</td>
393 <td style="width: 60%" align="center"></td>
394 <td style="width: 20%" align="right" valign="top">Documentation</td>
395 </tr>
396 </table>
397 </body>
398 </html>