10 To save on build time, the build system and automation collaborate to allow
11 downloading a packaged en-US Firefox, performing some locale-specific
12 post-processing, and re-packaging a locale-specific Firefox. Such artifacts
13 are termed "single-locale language repacks". There is another concept of a
14 "multi-locale language build", which is more like a regular build and less
15 like a re-packaging post-processing step.
19 These builds rely on make targets that don't work for
20 `artifact builds <https://bugzilla.mozilla.org/show_bug.cgi?id=1387485>`_.
22 Instructions for single-locale repacks for developers
23 -----------------------------------------------------
25 This assumes that ``$AB_CD`` is the locale you want to repack with; you
26 find the available localizations on `l10n-central <https://hg.mozilla.org/l10n-central/>`_.
28 #. You must have a built and packaged object directory, or a pre-built
36 #. Repackage using the locale-specific changes.
40 ./mach build installers-$AB_CD
42 You should find a re-packaged build at ``OBJDIR/dist/``, and a
43 runnable binary in ``OBJDIR/dist/l10n-stage/``.
44 The ``installers`` target runs quite a few things for you, including getting
45 the repository for the requested locale from
46 https://hg.mozilla.org/l10n-central/. It will clone them into
47 ``~/.mozbuild/l10n-central``. If you have an existing repository there, you
48 may want to occasionally update that via ``hg pull -u``. If you prefer
49 to have the l10n repositories at a different location on your disk, you
50 can point to the directory via
54 ac_add_options --with-l10n-base=/make/this/a/absolute/path
56 This build also packages a language pack.
58 Instructions for language packs
59 -------------------------------
61 Language packs are extensions that contain just the localized resources. Building
62 them doesn't require an actual build, but they're only compatible with the
63 ``mozilla-central`` source they're built with.
68 ./mach build langpack-$AB_CD
70 This target shares much of the logic of the ``installers-$AB_CD`` target above,
71 and does the check-out of the localization repository etc. It doesn't require
72 a package or a build, though. The generated language pack is in
73 ``OBJDIR/dist/$(MOZ_PKG_PLATFORM)/xpi/``.
77 Despite the platform-dependent location in the build directory, language packs
78 are platform independent, and the content that goes into them needs to be
79 built in a platform-independent way.
81 Instructions for multi-locale builds
82 ------------------------------------
84 If you want to create a single build with multiple locales, you will do
86 #. Create a build and package
93 #. Create the multi-locale package:
97 ./mach package-multi-locale --locales de it zh-TW
99 On Android, this produces a multi-locale GeckoView AAR and multi-locale APKs,
100 including GeckoViewExample. You can test different locales by changing your
101 Android OS locale and restarting GeckoViewExample. You'll need to install with
102 the ``MOZ_CHROME_MULTILOCALE`` variable set, like:
104 .. code-block:: shell
106 env MOZ_CHROME_MULTILOCALE=en-US,de,it,zh-TW ./mach android install-geckoview_example
108 Multi-locale builds without compiling
109 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
111 For deep technical reasons, artifact builds do not support multi-locale builds.
112 However, with a little work, we can achieve the same effect:
114 #. Arrange a ``mozconfig`` without a compilation environment but with support
115 for the ``RecursiveMake`` build backend, like:
117 .. code-block:: shell
119 ac_add_options --disable-compile-environment
120 export BUILD_BACKENDS=FasterMake,RecursiveMake
121 ... other options ...
125 .. code-block:: shell
129 #. Manually provide compiled artifacts.
131 .. code-block:: shell
133 ./mach artifact install [-v]
137 .. code-block:: shell
141 #. Produce a multi-locale package.
143 .. code-block:: shell
145 ./mach package-multi-locale --locales de it zh-TW
147 This build configuration is fragile and not generally useful for active
148 development (for that, use a full/compiled build), but it certainly speeds
149 testing multi-locale packaging.
151 General flow of repacks
152 -----------------------
154 The general flow of the locale repacks is controlled by
155 ``$MOZ_BUILD_APP/locales/Makefile.in`` and ``toolkit/locales/l10n.mk``, plus
156 the packaging build system. The three main entry points above all trigger
159 #. Get the localization repository, if needed
160 #. Run l10n-merge with a prior clobber of the merge dir
161 #. Copy l10n files to ``dist``, with minor differences here between ``l10n-%`` and ``chrome-%``
162 #. Repackage and package
164 Details on l10n-merge are described in its own section below.
165 The copying of files is mainly controlled by ``jar.mn``, in the few source
166 directories that include localizable files. ``l10n-%`` is used for repacks,
167 ``chrome-%`` for multi-locale packages. The repackaging is dedicated
168 Python code in ``toolkit/mozapps/installer/l10n-repack.py``, using an existing
169 package. It strips existing ``chrome`` l10n resources, and adds localizations
172 Language packs don't require repackaging. The windows installers are generated
173 by merely packaging an existing repackaged zip into to an installer.
178 The localization flow handles a few file formats in well-known locations in the
181 Alongside being built by including the directory in ``$MOZ_BUILD_APP/locales/Makefile.in``
182 and respective entries in a ``jar.mn``, we also have configuration files tailored
183 to localization tools and infrastructure. They're also controlling which
184 files l10n-merge handles, and how.
186 These configurations are TOML files. They're part of the bigger
187 localization ecosystem at Mozilla, and `the documentation about the
188 file format <http://moz-l10n-config.readthedocs.io/en/latest/fileformat.html>`_
189 explains how to set them up, and what the entries mean. In short, you find
194 reference = browser/locales/en-US/**
197 to add a directory for all localizations. Changes to these files are best
198 submitted for review by :Pike or :flod.
200 These configuration files are the future, and right now, we still have
201 support for the previous way to configuring l10n, which is described below.
203 The locations are commonly in directories like
205 :file:`browser/`\ ``locales/en-US/``\ :file:`subdir/file.ext`
207 The first thing to note is that only files beneath :file:`locales/en-US` are
208 exposed to localizers. The second thing to note is that only a few directories
209 are exposed. Which directories are exposed is defined in files called
210 ``l10n.ini``, which are at a
211 `few places <https://searchfox.org/mozilla-central/search?q=path%3Al10n.ini&redirect=true>`_
214 An example looks like this
223 browser/branding/official
226 toolkit = toolkit/locales/l10n.ini
228 This tells the l10n infrastructure three things:
230 * resolve the paths against the directory two levels up
231 * include files in :file:`browser/locales/en-US` and
232 :file:`browser/branding/official/locales/en-US`
233 * load more data from :file:`toolkit/locales/l10n.ini`
235 For projects like Thunderbird and SeaMonkey in ``comm-central``, additional
236 data needs to be provided when including an ``l10n.ini`` from a different
243 mozilla = mozilla-central
244 repo = https://hg.mozilla.org/
245 l10n.ini = toolkit/locales/l10n.ini
247 This tells the l10n infrastructure where to find the repository, and where inside
248 that repository the ``l10n.ini`` file is. This is needed because for local
249 builds, :file:`mail/locales/l10n.ini` references
250 :file:`mozilla/toolkit/locales/l10n.ini`, which is where the comm-central
251 build setup expects toolkit to be.
253 Now that the directories exposed to l10n are known, we can talk about the
254 supported file formats.
259 The following file formats are known to the l10n tool chains:
262 Used in Firefox UI, both declarative and programmatically.
264 Used from JavaScript and C++. When used from js, also comes with
265 plural support (avoid if possible).
267 Used by the crashreporter and updater, avoid if possible.
269 Adding new formats involves changing various different tools, and is strongly
274 Generally, anything that exists in ``en-US`` needs a one-to-one mapping in
275 all localizations. There are a few cases where that's not wanted, notably
276 around locale configuration and locale-dependent metadata.
278 For optional strings and files, l10n-merge won't add ``en-US`` content if
279 the localization doesn't have that content.
281 For the TOML files, the
282 `[[filters]] documentation <https://moz-l10n-config.readthedocs.io/en/latest/fileformat.html#filters>`_
283 is a good reference. In short, filters match the localized source code, optionally
284 a ``key``, and an action. An example like
289 path = "{l}calendar/chrome/calendar/calendar-event-dialog.properties"
290 key = "re:.*Nounclass[1-9].*"
293 indicates that the matching messages in ``calendar-event-dialog.properties`` are optional.
295 For the legacy ini configuration files, there's a Python module
296 ``filter.py`` next to the main ``l10n.ini``, implementing :py:func:`test`, with the following
299 .. code-block:: python
301 def test(mod, path, entity = None):
304 if show_but_do_not_merge:
306 # default behavior, localizer or build need to do something
309 For any missing file, this function is called with ``mod`` being
310 the *module*, and ``path`` being the relative path inside
311 :file:`locales/en-US`. The module is the top-level dir as referenced in
314 For missing strings, the :py:data:`entity` parameter is the key of the string
320 The chrome registry in Gecko doesn't support fallback from a localization to ``en-US`` at runtime.
321 Thus, the build needs to ensure that the localization as it's built into
322 the package has all required strings, and that the strings don't contain
323 errors. To ensure that, we're *merging* the localization and ``en-US``
324 at build time, nick-named l10n-merge.
326 For Fluent, we're also removing erroneous messages. For many errors in Fluent,
327 that's cosmetic, but when a localization has different values or attributes
328 on a message, that's actually important so that the DOM bindings of Fluent
329 can apply the translation without having to load the ``en-US`` source to
332 The process can be manually triggered via
336 $> ./mach build merge-$AB_CD
338 It creates another directory in the object dir, :file:`browser/locales/merge-dir/$AB_CD`, in
339 which the sanitized files are stored. The actual repackaging process only looks
340 in the merged directory, so the preparation steps of l10n-merge need to ensure
341 that all files are generated or copied.
343 l10n-merge modifies a file if it supports the particular file type, and there
344 are missing strings which are not filtered out, or if an existing string
345 shows an error. See the Checks section below for details. If the files are
346 not modified, l10n-merge copies them over to the respective location in the
352 As part of the build and other localization tool chains, we run a variety
353 of source-based checks. Think of them as linters.
355 The suite of checks is usually determined by file type, i.e., there's a
356 suite of checks for Fluent files and one for properties files, etc.
361 Now that we talked in-depth about how to expose content to localizers,
362 where are the localizations?
364 We host a mercurial repository per locale. All of our
365 localizations can be found on https://hg.mozilla.org/l10n-central/.
367 You can search inside our localized files on
368 `Transvision <https://transvision.mozfr.org/>`_.