gnulib-tool.py: Handle module dependencies that cannot be found.
[gnulib.git] / HACKING
blob964b880d204b51578cbf0dd4af45a64796cbdfeb
1 Information for GNU Gnulib maintainers and contributors
2 *******************************************************
4 Using git
5 =========
7 * We don't use topic branches.  Changes are usually small enough that
8   they can be committed directly to the master branch, after appropriate
9   testing.
11 * We maintain stable branches, though, as described in the documentation:
12   https://www.gnu.org/software/gnulib/manual/html_node/Stable-Branches.html
13   When backporting a commit to a stable branch of the last year, be sure
14   to update the copyright year of each modified file (since we don't run
15   "make update-copyright" on the stable branches).
17 * We use a linear git history — no merges. To work in this setting, it's
18   recommended that you configure git with 'git config pull.rebase = true'.
20 * Before pushing a commit, it is highly recommended that you review it in
21   its entirety. The easiest ways to do so are
22     * to run
23         $ git format-patch -1
24       and then read the patch in an editor that has syntax-colouring of patch
25       files, or
26     * to run
27         $ gitk
29 * We update the ChangeLog by hand.  The commit message is usually identical
30   to the ChangeLog entry, with the date and author line removed, with
31   the leading tabs removed, and with a blank line after the commit's
32   summary line.
33   In order to work efficiently with ChangeLog files, it is recommended that
34   you configure git to use the git-merge-changelog driver; see the instructions
35   in the lib/git-merge-changelog.c file.
36   Note: This driver reasonably keeps the ChangeLog entries together; however,
37   it does not always keep them in the order you would desire. For example,
38   when you had prepared a commit, you try to "git push" it but that fails due
39   to someone else's commit that came earlier, what you need to do is:
40     1. $ git pull
41     2. Verify that your ChangeLog entry is still the top-most one.
42     3. If it is not, then edit ChangeLog to move it to the top, and
43        $ git commit --amend ChangeLog
44     4. $ gitk
45     5. $ git push
47 * When you commit a contributor's patch, please
48   - add a reasonable ChangeLog entry in the usual style (meaningful
49     summary line and detailed change list),
50   - if the contribution is so small that it does not require a
51     copyright assignment (cf.
52     https://www.gnu.org/prep/maintain/html_node/Legally-Significant.html )
53     add a line:
54     Copyright-paperwork-exempt: Yes
55   - use the 'git commit' option --author="Contributor Name <email@address>"
58 License Notices
59 ===============
61 In *.m4 files, use a notice like this:
62   dnl Copyright (C) YEARS Free Software Foundation, Inc.
63   dnl This file is free software; the Free Software Foundation
64   dnl gives unlimited permission to copy and/or distribute it,
65   dnl with or without modifications, as long as this notice is preserved.
67 In lib/, tests/, build-aux/ files, except those that are shared with glibc,
68 use the license notices from etc/license-notices/ . This avoids gratuitous
69 differences in wording, as well misunderstandings when a license notice
70 would say "This program ...".
73 Test Suite
74 ==========
76 When adding a module, add a unit test module as well.  This is our best
77 chance to catch portability problems.
80 Maintaining high quality
81 ========================
83 It is a good idea to occasionally create a testdir of all of Gnulib:
84   $ rm -rf ../testdir-all; ./gnulib-tool --create-testdir --dir=../testdir-all --with-c++-tests --without-privileged-tests --single-configure `./all-modules`
85 and test this directory on various platforms:
86   - Linux/glibc systems,
87   - Linux/musl systems,
88   - macOS,
89   - FreeBSD,
90   - NetBSD,
91   - OpenBSD,
92   - AIX,
93   - Solaris 10 and 11,
94   - Cygwin,
95   - Haiku,
96   - Android,
97   - and other platforms of your choice.
99 There is a continuous integration that regularly performs this testing
100 on a Linux/glibc system: https://gitlab.com/gnulib/gnulib-ci
101 But this will catch only the most blatant mistakes.
104 Warning Options
105 ===============
107 For packages that use Gnulib, we recommend to use the 'warnings' or
108 'manywarnings' module, as documented in
109 https://www.gnu.org/software/gnulib/manual/html_node/warnings.html
110 https://www.gnu.org/software/gnulib/manual/html_node/manywarnings.html
112 When building Gnulib testdirs, e.g. when preparing a Gnulib patch,
113 there are three possible approaches:
115 * The simplest approach, which warns about the most common mistakes, is to
116   use GCC's -Wall option, both for C and C++ compilation units. Just set
117     $ ./configure CPPFLAGS="-Wall"
118     $ make
119   You should generally fix all compiler warnings that you see from this
120   approach.
122 * If you are developing on a glibc system and have GCC version 13 binaries
123   available, here's a recipe that will find more mistakes, but is nearly
124   as easy to use. Here, different warning options are needed for C and
125   for C++:
126     $ WARN_GCC13=`echo '
127       -fanalyzer
128       -Wall
129       -Warith-conversion
130       -Wcast-align=strict
131       -Wdate-time
132       -Wdisabled-optimization
133       -Wduplicated-cond
134       -Wextra
135       -Wformat-signedness
136       -Winit-self
137       -Winvalid-pch
138       -Wlogical-op
139       -Wmissing-include-dirs
140       -Wopenmp-simd
141       -Woverlength-strings
142       -Wpacked
143       -Wpointer-arith
144       -Wstrict-overflow
145       -Wsuggest-final-methods
146       -Wsuggest-final-types
147       -Wsync-nand
148       -Wsystem-headers
149       -Wtrampolines
150       -Wuninitialized
151       -Wunknown-pragmas
152       -Wunsafe-loop-optimizations
153       -Wvariadic-macros
154       -Wvector-operation-performance
155       -Wwrite-strings
156       -Warray-bounds=2
157       -Wattribute-alias=2
158       -Wformat-overflow=2
159       -Wformat-truncation=2
160       -Wshift-overflow=2
161       -Wunused-const-variable=2
162       -Wvla-larger-than=4031
163       -Wno-empty-body
164       -Wno-analyzer-allocation-size
165       -Wno-analyzer-fd-double-close
166       -Wno-analyzer-double-fclose
167       -Wno-analyzer-double-free
168       -Wno-analyzer-fd-leak
169       -Wno-analyzer-fd-use-after-close
170       -Wno-analyzer-fd-use-without-check
171       -Wno-analyzer-free-of-non-heap
172       -Wno-analyzer-malloc-leak
173       -Wno-analyzer-mismatching-deallocation
174       -Wno-analyzer-null-argument
175       -Wno-analyzer-null-dereference
176       -Wno-analyzer-out-of-bounds
177       -Wno-analyzer-possible-null-argument
178       -Wno-analyzer-possible-null-dereference
179       -Wno-analyzer-use-after-free
180       -Wno-analyzer-use-of-pointer-in-stale-stack-frame
181       -Wno-analyzer-use-of-uninitialized-value
182       -Wno-analyzer-va-arg-type-mismatch
183       -Wno-attribute-warning
184       -Wno-cast-align
185       -Wno-clobbered
186       -Wno-dangling-pointer
187       -Wno-format
188       -Wno-implicit-fallthrough
189       -Wno-maybe-uninitialized
190       -Wno-missing-field-initializers
191       -Wno-restrict
192       -Wno-sign-compare
193       -Wno-switch
194       -Wno-type-limits
195       -Wno-unused-parameter
196     ' | tr -d '\n' | sed -e 's/  */ /g'`
197     $ WARN_CFLAGS_GCC13="$WARN_GCC13 -Wnested-externs -Wshadow=local -Wno-discarded-qualifiers"
198     $ WARN_CXXFLAGS_GCC13="$WARN_GCC13 -Wno-cpp"
199     $ ./configure CFLAGS="-O2 -g $WARN_CFLAGS_GCC13" CXXFLAGS="-O2 -g $WARN_CXXFLAGS_GCC13"
200     $ make
201   You should generally fix all compiler warnings that you see from this
202   approach, or report when this approach produced a pointless warning
203   (so that we can fix the value of WARN_GCC13 above).
205 * If you are developing on a glibc system and have GCC version 13 binaries
206   available: Here's a recipe that will find even more mistakes, but it
207   requires that you are willing to filter out and ignore pointless warnings.
208     $ WARN_GCC13=`echo '
209       -fanalyzer
210       -Wall
211       -Warith-conversion
212       -Wcast-align=strict
213       -Wdate-time
214       -Wdisabled-optimization
215       -Wduplicated-cond
216       -Wextra
217       -Wformat-signedness
218       -Winit-self
219       -Winvalid-pch
220       -Wlogical-op
221       -Wmissing-include-dirs
222       -Wnull-dereference
223       -Wopenmp-simd
224       -Woverlength-strings
225       -Wpacked
226       -Wpointer-arith
227       -Wstrict-overflow
228       -Wsuggest-attribute=format
229       -Wsuggest-final-methods
230       -Wsuggest-final-types
231       -Wsync-nand
232       -Wsystem-headers
233       -Wtrampolines
234       -Wuninitialized
235       -Wunknown-pragmas
236       -Wunsafe-loop-optimizations
237       -Wvariadic-macros
238       -Wvector-operation-performance
239       -Wwrite-strings
240       -Warray-bounds=2
241       -Wattribute-alias=2
242       -Wformat-overflow=2
243       -Wformat=2
244       -Wformat-truncation=2
245       -Wimplicit-fallthrough=5
246       -Wshift-overflow=2
247       -Wunused-const-variable=2
248       -Wvla-larger-than=4031
249       -Wno-empty-body
250       -Wno-analyzer-double-fclose
251       -Wno-analyzer-double-free
252       -Wno-analyzer-free-of-non-heap
253       -Wno-analyzer-malloc-leak
254       -Wno-analyzer-null-argument
255       -Wno-analyzer-null-dereference
256       -Wno-analyzer-use-after-free
257       -Wno-attribute-warning
258       -Wno-cast-align
259       -Wno-clobbered
260       -Wno-format-nonliteral
261       -Wno-sign-compare
262       -Wno-type-limits
263       -Wno-unused-parameter
264     ' | tr -d '\n' | sed -e 's/  */ /g'`
265     $ WARN_CFLAGS_GCC13="$WARN_GCC13 -Wnested-externs -Wshadow=local"
266     $ WARN_CXXFLAGS_GCC13="$WARN_GCC13 -Wno-cpp"
267     $ ./configure CFLAGS="-O2 -g $WARN_CFLAGS_GCC13" CXXFLAGS="-O2 -g $WARN_CXXFLAGS_GCC13"
268     $ make
269   With this approach, use your own judgement whether to fix warnings
270   arising from your new code or not.
271   Do *not* submit patches to silence warnings from existing code:
272     - For these warnings, often the cure will be worse than the disease.
273     - Some of the warnings are false positives. Rather than silencing
274       these warnings, we prefer to report them in the GCC bug tracker
275       and wait until they are fixed in a future GCC release.
277 Similarly, for clang version 16 you can use the following recipe, that uses
278 selected warning options from
279 https://releases.llvm.org/16.0.0/tools/clang/docs/DiagnosticsReference.html :
280   $ WARN_CLANG16=`echo '
281     -Wall
282     -Wanon-enum-enum-conversion
283     -Warc-repeated-use-of-weak
284     -Warray-bounds-pointer-arithmetic
285     -Warray-parameter
286     -Watomic-properties
287     -Wbinary-literal
288     -Wbit-int-extension
289     -Wbitfield-enum-conversion
290     -Wbitwise-op-parentheses
291     -Wbool-operation
292     -Wc++-compat
293     -Wc2x-compat
294     -Wc2x-extensions
295     -Wc99-compat
296     -Wc99-designator
297     -Wc99-extensions
298     -Wcalled-once-parameter
299     -Wcast-function-type
300     -Wchar-subscripts
301     -Wcomment
302     -Wcompletion-handler
303     -Wcomplex-component-init
304     -Wcompound-token-split
305     -Wconsumed
306     -Wconversion
307     -Wcstring-format-directive
308     -Wcuda-compat
309     -Wdate-time
310     -Wdelimited-escape-sequence-extension
311     -Wdeprecated
312     -Wdeprecated-dynamic-exception-spec
313     -Wdeprecated-implementations
314     -Wdeprecated-this-capture
315     -Wdeprecated-writable-strings
316     -Wdirect-ivar-access
317     -Wdocumentation
318     -Wdocumentation-deprecated-sync
319     -Wdocumentation-html
320     -Wdocumentation-pedantic
321     -Wdocumentation-unknown-command
322     -Wdollar-in-identifier-extension
323     -Wduplicate-decl-specifier
324     -Wduplicate-enum
325     -Wduplicate-method-arg
326     -Wduplicate-method-match
327     -Wdynamic-exception-spec
328     -Wembedded-directive
329     -Wempty-init-stmt
330     -Wempty-translation-unit
331     -Wenum-compare-conditional
332     -Wenum-conversion
333     -Wenum-enum-conversion
334     -Wenum-float-conversion
335     -Wexit-time-destructors
336     -Wexpansion-to-defined
337     -Wexplicit-ownership-type
338     -Wextra
339     -Wextra-semi
340     -Wfixed-enum-extension
341     -Wflexible-array-extensions
342     -Wfloat-overflow-conversion
343     -Wfloat-zero-conversion
344     -Wfor-loop-analysis
345     -Wformat
346     -Wformat-pedantic
347     -Wformat-type-confusion
348     -Wformat=2
349     -Wfour-char-constants
350     -Wframe-address
351     -Wfuse-ld-path
352     -Wfuture-attribute-extensions
353     -Wgcc-compat
354     -Wgnu
355     -Wgnu-anonymous-struct
356     -Wgnu-auto-type
357     -Wgnu-case-range
358     -Wgnu-complex-integer
359     -Wgnu-compound-literal-initializer
360     -Wgnu-conditional-omitted-operand
361     -Wgnu-designator
362     -Wgnu-empty-initializer
363     -Wgnu-empty-struct
364     -Wgnu-flexible-array-initializer
365     -Wgnu-flexible-array-union-member
366     -Wgnu-folding-constant
367     -Wgnu-imaginary-constant
368     -Wgnu-label-as-value
369     -Wgnu-line-marker
370     -Wgnu-null-pointer-arithmetic
371     -Wgnu-offsetof-extensions
372     -Wgnu-pointer-arith
373     -Wgnu-redeclared-enum
374     -Wgnu-statement-expression
375     -Wgnu-statement-expression-from-macro-expansion
376     -Wgnu-union-cast
377     -Wgnu-zero-line-directive
378     -Wgnu-zero-variadic-macro-arguments
379     -Wheader-hygiene
380     -Widiomatic-parentheses
381     -Wignored-qualifiers
382     -Wimplicit
383     -Wimplicit-fallthrough
384     -Wimplicit-fallthrough-per-function
385     -Wimplicit-function-declaration
386     -Wimplicit-int
387     -Wimplicit-retain-self
388     -Wimport-preprocessor-directive-pedantic
389     -Wincomplete-module
390     -Winconsistent-missing-destructor-override
391     -Winfinite-recursion
392     -Wint-in-bool-context
393     -Winvalid-or-nonexistent-directory
394     -Winvalid-utf8
395     -Wkeyword-macro
396     -Wlanguage-extension-token
397     -Wlocal-type-template-args
398     -Wlogical-op-parentheses
399     -Wlong-long
400     -Wloop-analysis
401     -Wmain
402     -Wmax-tokens
403     -Wmethod-signatures
404     -Wmicrosoft
405     -Wmicrosoft-anon-tag
406     -Wmicrosoft-charize
407     -Wmicrosoft-comment-paste
408     -Wmicrosoft-cpp-macro
409     -Wmicrosoft-end-of-file
410     -Wmicrosoft-enum-value
411     -Wmicrosoft-exception-spec
412     -Wmicrosoft-fixed-enum
413     -Wmicrosoft-flexible-array
414     -Wmicrosoft-redeclare-static
415     -Wmisleading-indentation
416     -Wmismatched-tags
417     -Wmissing-braces
418     -Wmissing-method-return-type
419     -Wmost
420     -Wmove
421     -Wnested-anon-types
422     -Wnewline-eof
423     -Wnon-gcc
424     -Wnon-modular-include-in-framework-module
425     -Wnon-modular-include-in-module
426     -Wnon-virtual-dtor
427     -Wnonportable-system-include-path
428     -Wnull-pointer-arithmetic
429     -Wnull-pointer-subtraction
430     -Wnullability-extension
431     -Wnullable-to-nonnull-conversion
432     -Wopenmp
433     -Wover-aligned
434     -Woverlength-strings
435     -Woverloaded-virtual
436     -Woverriding-method-mismatch
437     -Wpacked
438     -Wpacked-non-pod
439     -Wparentheses
440     -Wpedantic
441     -Wpedantic-core-features
442     -Wpessimizing-move
443     -Wpointer-arith
444     -Wpoison-system-directories
445     -Wpragma-pack
446     -Wpragma-pack-suspicious-include
447     -Wpragmas
448     -Wpre-c2x-compat
449     -Wpre-openmp-51-compat
450     -Wprofile-instr-missing
451     -Wquoted-include-in-framework-header
452     -Wrange-loop-analysis
453     -Wrange-loop-bind-reference
454     -Wrange-loop-construct
455     -Wreceiver-forward-class
456     -Wredundant-move
457     -Wredundant-parens
458     -Wreorder
459     -Wreorder-ctor
460     -Wreserved-user-defined-literal
461     -Wretained-language-linkage
462     -Wselector
463     -Wselector-type-mismatch
464     -Wself-assign
465     -Wself-assign-overloaded
466     -Wself-move
467     -Wsemicolon-before-method-body
468     -Wshadow-all
469     -Wshadow-field
470     -Wshadow-field-in-constructor
471     -Wshadow-field-in-constructor-modified
472     -Wshadow-uncaptured-local
473     -Wshift-sign-overflow
474     -Wsigned-enum-bitfield
475     -Wsometimes-uninitialized
476     -Wsource-uses-openmp
477     -Wspir-compat
478     -Wstatic-in-inline
479     -Wstrict-potentially-direct-selector
480     -Wstrict-selector-match
481     -Wstring-concatenation
482     -Wstring-conversion
483     -Wsuggest-destructor-override
484     -Wsuggest-override
485     -Wsuper-class-method-mismatch
486     -Wtautological-bitwise-compare
487     -Wtautological-compare
488     -Wtautological-constant-in-range-compare
489     -Wtautological-overlap-compare
490     -Wtautological-type-limit-compare
491     -Wtautological-unsigned-char-zero-compare
492     -Wtautological-unsigned-enum-zero-compare
493     -Wtautological-unsigned-zero-compare
494     -Wtautological-value-range-compare
495     -Wthread-safety
496     -Wthread-safety-analysis
497     -Wthread-safety-attributes
498     -Wthread-safety-beta
499     -Wthread-safety-negative
500     -Wthread-safety-precise
501     -Wthread-safety-reference
502     -Wthread-safety-verbose
503     -Wtype-limits
504     -Wunaligned-access
505     -Wundeclared-selector
506     -Wundef-prefix
507     -Wundefined-func-template
508     -Wundefined-internal-type
509     -Wundefined-reinterpret-cast
510     -Wunguarded-availability
511     -Wuninitialized
512     -Wuninitialized-const-reference
513     -Wunknown-pragmas
514     -Wunnamed-type-template-args
515     -Wunneeded-internal-declaration
516     -Wunneeded-member-function
517     -Wunreachable-code-fallthrough
518     -Wunreachable-code-loop-increment
519     -Wunsupported-dll-base-class-template
520     -Wunused
521     -Wunused-but-set-parameter
522     -Wunused-but-set-variable
523     -Wunused-const-variable
524     -Wunused-exception-parameter
525     -Wunused-function
526     -Wunused-label
527     -Wunused-lambda-capture
528     -Wunused-local-typedef
529     -Wunused-member-function
530     -Wunused-private-field
531     -Wunused-property-ivar
532     -Wunused-template
533     -Wunused-variable
534     -Wvariadic-macros
535     -Wvector-conversion
536     -Wweak-template-vtables
537     -Wweak-vtables
538     -Wzero-length-array
539     -Wno-bitwise-instead-of-logical
540     -Wno-c11-extensions
541     -Wno-cast-function-type-strict
542     -Wno-float-conversion
543     -Wno-format-nonliteral
544     -Wno-gnu-include-next
545     -Wno-implicit-float-conversion
546     -Wno-implicit-int-conversion
547     -Wno-implicit-int-float-conversion
548     -Wno-include-next-absolute-path
549     -Wno-missing-field-initializers
550     -Wno-reserved-macro-identifier
551     -Wno-shadow
552     -Wno-shorten-64-to-32
553     -Wno-sign-compare
554     -Wno-sign-conversion
555     -Wno-strict-prototypes
556     -Wno-switch
557     -Wno-unused-parameter
558     -Wno-tautological-constant-out-of-range-compare
559     -Wno-tautological-type-limit-compare
560     -Wno-tautological-unsigned-zero-compare
561     -Wno-tautological-value-range-compare
562     -Wno-unused-command-line-argument
563     -Wno-user-defined-warnings
564   ' | tr -d '\n' | sed -e 's/  */ /g'`
565   $ ./configure CFLAGS="-O2 -g $WARN_CLANG16" CXXFLAGS="-O2 -g $WARN_CLANG16"
566   $ make
567 Again, use your own judgement to determine whether to fix or ignore a
568 specific warning.
571 Information for gnulib-tool maintenance
572 ***************************************
574 Running the unit tests
575 ======================
577 The unit tests of gnulib-tool reside in the maint-tools repository, that is a
578 satellite repository of the main gnulib repository. Instructions how to obtain
579 it are in https://savannah.gnu.org/git/?group=gnulib .
581 To run the unit tests of gnulib-tool.sh:
582   $ cd maint-tools/gnulib-tool-tests/
583   $ GNULIB_TOOL_IMPL=sh ./test-all.sh
585 To run the unit tests of gnulib-tool.py:
586   $ cd maint-tools/gnulib-tool-tests/
587   $ GNULIB_TOOL_IMPL=py ./test-all.sh
589 It is *mandatory* that you run the test suite before pushing any change to
590 gnulib-tool.sh or gnulib-tool.py! If you fail to do so, and your change contains
591 a bug, it will start to affect users immediately.
594 Debugging the Python implementation of gnulib-tool
595 ==================================================
597 With Eclipse and PyDev as IDE
598 -----------------------------
600 * Download and configuration:
601   - Eclipse IDE from https://www.eclipse.org/downloads/packages/
602     (either the build for Java or for C/C++ should work fine;
603     either use the Eclipse Installer program at the top of the page,
604     or one of the individual download links below).
605   - PyDev from https://www.pydev.org/download.html
606     section "Install as Plugin".
607     (Don't use LiClipse, since the license costs money.)
608   - Follow https://www.pydev.org/manual_101_install.html,
609     replacing http://www.pydev.org/updates
610     with      https://www.pydev.org/updates
611   - Once installed, restart the IDE.
612   - Window > Preferences > PyDev > Interpreters > Python Interpreter:
613     New > path: /usr/bin/python3
614   - Window > Preferences > PyDev > Editor > Code Analysis:
615     Others > Redefinition of builtin symbols: Ignore
617 * Create a project:
618   Let GNULIB_DIR be my gnulib checkout.
619   - File > New > Project... > PyDev > PyDev Project. Call it 'gnulib-tool'.
620     Note that this is a project inside the Eclipse workspace, so far
621     unrelated to the GNULIB_DIR.
622   - Popup menu > New > Link to Existing Source
623     Name: gnulib
624     Path: <GNULIB_DIR>
626 * Create a run configuration:
627   - Run > Run Configurations... > Python Run
628     Popup menu > New configuration
629       Name: gnulib-tool.py
630       Main > Project: gnulib-tool
631       Main > Main Module: ${workspace_loc:gnulib-tool/gnulib/.gnulib-tool.py}
632       Arguments > Program arguments: --help
633   - Test it: Run this configuration.
635 * Create a debug configuration:
636   - Run > Debug Configurations... > gnulib-tool.py
637     Popup menu > Duplicate
638   - In the duplicate, set
639     Arguments > Working directory: /tmp/oath-toolkit/lib
640     Arguments > Program arguments: --add-import
642 * Debug it:
643   - Open GLImport.py.
644   - On the left-hand border of this editor, do "Add breakpoint".
645   - Run > Debug Configurations... > pick the duplicate. Press Debug.
648 Maintaining high quality
649 ========================
651 There is a continuous integration of gnulib-tool.py that runs the unit tests,
652 at https://gitlab.com/gnulib/gnulib-tool-ci/ .