4 1. [Building the lib and applications](#building-the-library-and-applications)
5 - [Prerequisites](#prerequisites)
6 - [Get the code](#get-the-code)
7 - [Basics](#basic-build)
8 - [Configuration options](#configuration-options)
9 - [Dylib builds](#dylib-builds)
10 - [Debugging](#debugging)
11 - [Cross compiling](#cross-compiling)
12 - [Sanitizer support](#sanitizers)
13 - [MSVC builds](#microsoft-visual-studio-builds)
14 - [Xcode builds](#xcode-builds)
15 - [Emscripten builds](#emscripten-builds)
16 - [Extra Build Flags](#extra-build-flags)
17 2. [Testing the library](#testing-the-av1-codec)
18 - [Basics](#testing-basics)
19 - [Unit tests](#1_unit-tests)
20 - [Example tests](#2_example-tests)
21 - [Encoder tests](#3_encoder-tests)
22 - [IDE hosted tests](#ide-hosted-tests)
23 - [Downloading test data](#downloading-the-test-data)
24 - [Adding a new test data file](#adding-a-new-test-data-file)
25 - [Additional test data](#additional-test-data)
26 - [Sharded testing](#sharded-testing)
27 - [Running tests directly](#1_running-test_libaom-directly)
28 - [Running tests via CMake](#2_running-the-tests-via-the-cmake-build)
29 3. [Coding style](#coding-style)
30 4. [Submitting patches](#submitting-patches)
31 - [Login cookie](#login-cookie)
32 - [Contributor agreement](#contributor-agreement)
33 - [Testing your code](#testing-your-code)
34 - [Commit message hook](#commit-message-hook)
35 - [Upload your change](#upload-your-change)
36 - [Incorporating Reviewer Comments](#incorporating-reviewer-comments)
37 - [Submitting your change](#submitting-your-change)
38 - [Viewing change status](#viewing-the-status-of-uploaded-changes)
39 5. [Support](#support)
40 6. [Bug reports](#bug-reports)
42 ## Building the library and applications
46 1. [CMake](https://cmake.org) version 3.5 or higher.
47 2. [Git](https://git-scm.com/).
48 3. [Perl](https://www.perl.org/).
49 4. For x86 targets, [yasm](http://yasm.tortall.net/), which is preferred, or a
50 recent version of [nasm](http://www.nasm.us/).
51 5. Building the documentation requires [doxygen](http://doxygen.org).
52 6. Building the unit tests requires [Python](https://www.python.org/).
53 7. Emscripten builds require the portable
54 [EMSDK](https://kripken.github.io/emscripten-site/index.html).
58 The AV1 library source code is stored in the Alliance for Open Media Git
62 $ git clone https://aomedia.googlesource.com/aom
63 # By default, the above command stores the source in the aom directory:
69 CMake replaces the configure step typical of many projects. Running CMake will
70 produce configuration and build files for the currently selected CMake
71 generator. For most systems the default generator is Unix Makefiles. The basic
72 form of a makefile build is the following:
79 The above will generate a makefile build that produces the AV1 library and
80 applications for the current host system after the make step completes
81 successfully. The compiler chosen varies by host platform, but a general rule
82 applies: On systems where cc and c++ are present in $PATH at the time CMake is
83 run the generated build will use cc and c++ by default.
85 ### Configuration options
87 The AV1 codec library has a great many configuration options. These come in two
90 1. Build system configuration options. These have the form `ENABLE_FEATURE`.
91 2. AV1 codec configuration options. These have the form `CONFIG_FEATURE`.
93 Both types of options are set at the time CMake is run. The following example
94 enables ccache and disables the AV1 encoder:
97 $ cmake path/to/aom -DENABLE_CCACHE=1 -DCONFIG_AV1_ENCODER=0
101 The available configuration options are too numerous to list here. Build system
102 configuration options can be found at the top of the CMakeLists.txt file found
103 in the root of the AV1 repository, and AV1 codec configuration options can
104 currently be found in the file `build/cmake/aom_config_defaults.cmake`.
108 A dylib (shared object) build of the AV1 codec library can be enabled via the
109 CMake built in variable `BUILD_SHARED_LIBS`:
112 $ cmake path/to/aom -DBUILD_SHARED_LIBS=1
116 This is currently only supported on non-Windows targets.
120 Depending on the generator used there are multiple ways of going about
121 debugging AV1 components. For single configuration generators like the Unix
122 Makefiles generator, setting `CMAKE_BUILD_TYPE` to Debug is sufficient:
125 $ cmake path/to/aom -DCMAKE_BUILD_TYPE=Debug
128 For Xcode, mainly because configuration controls for Xcode builds are buried two
129 configuration windows deep and must be set for each subproject within the Xcode
130 IDE individually, `CMAKE_CONFIGURATION_TYPES` should be set to Debug:
133 $ cmake path/to/aom -G Xcode -DCMAKE_CONFIGURATION_TYPES=Debug
136 For Visual Studio the in-IDE configuration controls should be used. Simply set
137 the IDE project configuration to Debug to allow for stepping through the code.
139 In addition to the above it can sometimes be useful to debug only C and C++
140 code. To disable all assembly code and intrinsics set `AOM_TARGET_CPU` to
141 generic at generation time:
144 $ cmake path/to/aom -DAOM_TARGET_CPU=generic
149 For the purposes of building the AV1 codec and applications and relative to the
150 scope of this guide, all builds for architectures differing from the native host
151 architecture will be considered cross compiles. The AV1 CMake build handles
152 cross compiling via the use of toolchain files included in the AV1 repository.
153 The toolchain files available at the time of this writing are:
156 - arm64-linux-gcc.cmake
157 - arm64-mingw-gcc.cmake
159 - armv7-linux-gcc.cmake
160 - armv7-mingw-gcc.cmake
162 - mips32-linux-gcc.cmake
163 - mips64-linux-gcc.cmake
164 - x86-ios-simulator.cmake
167 - x86-mingw-gcc.cmake
168 - x86\_64-ios-simulator.cmake
169 - x86\_64-mingw-gcc.cmake
171 The following example demonstrates use of the x86-macos.cmake toolchain file on
172 a x86\_64 MacOS host:
175 $ cmake path/to/aom \
176 -DCMAKE_TOOLCHAIN_FILE=path/to/aom/build/cmake/toolchains/x86-macos.cmake
180 To build for an unlisted target creation of a new toolchain file is the best
181 solution. The existing toolchain files can be used a starting point for a new
182 toolchain file since each one exposes the basic requirements for toolchain files
183 as used in the AV1 codec build.
185 As a temporary work around an unoptimized AV1 configuration that builds only C
186 and C++ sources can be produced using the following commands:
189 $ cmake path/to/aom -DAOM_TARGET_CPU=generic
193 In addition to the above it's important to note that the toolchain files
194 suffixed with gcc behave differently than the others. These toolchain files
195 attempt to obey the $CROSS environment variable.
199 Sanitizer integration is built-in to the CMake build system. To enable a
200 sanitizer, add `-DSANITIZE=<type>` to the CMake command line. For example, to
201 enable address sanitizer:
204 $ cmake path/to/aom -DSANITIZE=address
208 Sanitizers available vary by platform, target, and compiler. Consult your
209 compiler documentation to determine which, if any, are available.
211 ### Microsoft Visual Studio builds
213 Building the AV1 codec library in Microsoft Visual Studio is supported. The
214 following example demonstrates generating projects and a solution for the
218 # This does not require a bash shell; command.exe is fine.
219 $ cmake path/to/aom -G "Visual Studio 15 2017"
224 Building the AV1 codec library in Xcode is supported. The following example
225 demonstrates generating an Xcode project:
228 $ cmake path/to/aom -G Xcode
231 ### Emscripten builds
233 Building the AV1 codec library with Emscripten is supported. Typically this is
234 used to hook into the AOMAnalyzer GUI application. These instructions focus on
235 using the inspector with AOMAnalyzer, but all tools can be built with
238 It is assumed here that you have already downloaded and installed the EMSDK,
239 installed and activated at least one toolchain, and setup your environment
240 appropriately using the emsdk\_env script.
242 1. Download [AOMAnalyzer](https://people.xiph.org/~mbebenita/analyzer/).
244 2. Configure the build:
247 $ cmake path/to/aom \
249 -DAOM_TARGET_CPU=generic \
252 -DCONFIG_ACCOUNTING=1 \
253 -DCONFIG_INSPECTION=1 \
254 -DCONFIG_MULTITHREAD=0 \
255 -DCONFIG_RUNTIME_CPU_DETECT=0 \
257 -DCMAKE_TOOLCHAIN_FILE=path/to/emsdk-portable/.../Emscripten.cmake
260 3. Build it: run make if that's your generator of choice:
269 # inspect.js is in the examples sub directory of the directory in which you
271 $ path/to/AOMAnalyzer path/to/examples/inspect.js path/to/av1/input/file
274 ### Extra build flags
276 Three variables allow for passing of additional flags to the build system.
278 - AOM\_EXTRA\_C\_FLAGS
279 - AOM\_EXTRA\_CXX\_FLAGS
280 - AOM\_EXTRA\_EXE\_LINKER\_FLAGS
282 The build system attempts to ensure the flags passed through the above variables
283 are passed to tools last in order to allow for override of default behavior.
284 These flags can be used, for example, to enable asserts in a release build:
287 $ cmake path/to/aom \
288 -DCMAKE_BUILD_TYPE=Release \
289 -DAOM_EXTRA_C_FLAGS=-UNDEBUG \
290 -DAOM_EXTRA_CXX_FLAGS=-UNDEBUG
293 ## Testing the AV1 codec
297 There are several methods of testing the AV1 codec. All of these methods require
298 the presence of the AV1 source code and a working build of the AV1 library and
303 The unit tests can be run at build time:
306 # Before running the make command the LIBAOM_TEST_DATA_PATH environment
307 # variable should be set to avoid downloading the test files to the
308 # cmake build configuration directory.
310 # Note: The AV1 CMake build creates many test targets. Running make
311 # with multiple jobs will speed up the test run significantly.
315 #### 2. Example tests:
317 The example tests require a bash shell and can be run in the following manner:
320 # See the note above about LIBAOM_TEST_DATA_PATH above.
323 # It's best to build the testdata target using many make jobs.
324 # Running it like this will verify and download (if necessary)
325 # one at a time, which takes a while.
327 $ path/to/aom/test/examples.sh --bin-path examples
330 #### 3. Encoder tests:
332 When making a change to the encoder run encoder tests to confirm that your
333 change has a positive or negligible impact on encode quality. When running these
334 tests the build configuration should be changed to enable internal encoder
338 $ cmake path/to/aom -DCONFIG_INTERNAL_STATS=1
342 The repository contains scripts intended to make running these tests as simple
343 as possible. The following example demonstrates creating a set of baseline clips
344 for comparison to results produced after making your change to libaom:
347 # This will encode all Y4M files in the current directory using the
348 # settings specified to create the encoder baseline statistical data:
349 $ cd path/to/test/inputs
350 # This command line assumes that run_encodes.sh, its helper script
351 # best_encode.sh, and the aomenc you intend to test are all within a
352 # directory in your PATH.
353 $ run_encodes.sh 200 500 50 baseline
356 After making your change and creating the baseline clips, you'll need to run
357 encodes that include your change(s) to confirm that things are working as
361 # This will encode all Y4M files in the current directory using the
362 # settings specified to create the statistical data for your change:
363 $ cd path/to/test/inputs
364 # This command line assumes that run_encodes.sh, its helper script
365 # best_encode.sh, and the aomenc you intend to test are all within a
366 # directory in your PATH.
367 $ run_encodes.sh 200 500 50 mytweak
370 After creating both data sets you can use `test/visual_metrics.py` to generate a
371 report that can be viewed in a web browser:
374 $ visual_metrics.py metrics_template.html "*stt" baseline mytweak \
378 You can view the report by opening mytweak.html in a web browser.
383 By default the generated projects files created by CMake will not include the
384 runtests and testdata rules when generating for IDEs like Microsoft Visual
385 Studio and Xcode. This is done to avoid intolerably long build cycles in the
386 IDEs-- IDE behavior is to build all targets when selecting the build project
387 options in MSVS and Xcode. To enable the test rules in IDEs the
388 `ENABLE_IDE_TEST_HOSTING` variable must be enabled at CMake generation time:
391 # This example uses Xcode. To get a list of the generators
392 # available, run cmake with the -G argument missing its
394 $ cmake path/to/aom -DENABLE_IDE_TEST_HOSTING=1 -G Xcode
397 ### Downloading the test data
399 The fastest and easiest way to obtain the test data is to use CMake to generate
400 a build using the Unix Makefiles generator, and then to build only the testdata
404 $ cmake path/to/aom -G "Unix Makefiles"
405 # 28 is used because there are 28 test files as of this writing.
409 The above make command will only download and verify the test data.
411 ### Adding a new test data file
413 First, add the new test data file to the `aom-test-data` bucket of the
414 `aomedia-testing` project on Google Cloud Platform. You may need to ask someone
415 with the necessary access permissions to do this for you.
417 NOTE: When a new test data file is added to the `aom-test-data` bucket, its
418 "Public access" is initially "Not public". We need to change its
419 "Public access" to "Public" by using the following
420 [`gsutil`](https://cloud.google.com/storage/docs/gsutil_install) command:
422 $ gsutil acl ch -g all:R gs://aom-test-data/test-data-file-name
424 This command grants the `AllUsers` group READ access to the file named
425 "test-data-file-name" in the `aom-test-data` bucket.
427 Once the new test data file has been added to `aom-test-data`, create a CL to
428 add the name of the new test data file to `test/test_data_util.cmake` and add
429 the SHA1 checksum of the new test data file to `test/test-data.sha1`. (The SHA1
430 checksum of a file can be calculated by running the `sha1sum` command on the
433 ### Additional test data
435 The test data mentioned above is strictly intended for unit testing.
437 Additional input data for testing the encoder can be obtained from:
438 https://media.xiph.org/video/derf/
442 The AV1 codec library unit tests are built upon gtest which supports sharding of
443 test jobs. Sharded test runs can be achieved in a couple of ways.
445 #### 1. Running test\_libaom directly:
448 # Set the environment variable GTEST_TOTAL_SHARDS to control the number of
450 $ export GTEST_TOTAL_SHARDS=10
451 # (GTEST shard indexing is 0 based).
452 $ seq 0 $(( $GTEST_TOTAL_SHARDS - 1 )) \
453 | xargs -n 1 -P 0 -I{} env GTEST_SHARD_INDEX={} ./test_libaom
456 To create a test shard for each CPU core available on the current system set
457 `GTEST_TOTAL_SHARDS` to the number of CPU cores on your system minus one.
459 #### 2. Running the tests via the CMake build:
462 # For IDE based builds, ENABLE_IDE_TEST_HOSTING must be enabled. See
463 # the IDE hosted tests section above for more information. If the IDE
464 # supports building targets concurrently tests will be sharded by default.
466 # For make and ninja builds the -j parameter controls the number of shards
467 # at test run time. This example will run the tests using 10 shards via
472 The maximum number of test targets that can run concurrently is determined by
473 the number of CPUs on the system where the build is configured as detected by
474 CMake. A system with 24 cores can run 24 test shards using a value of 24 with
475 the `-j` parameter. When CMake is unable to detect the number of cores 10 shards
476 is the default maximum value.
480 We are using the Google C Coding Style defined by the
481 [Google C++ Style Guide](https://google.github.io/styleguide/cppguide.html).
483 The coding style used by this project is enforced with clang-format using the
484 configuration contained in the
485 [.clang-format](https://chromium.googlesource.com/webm/aom/+/master/.clang-format)
486 file in the root of the repository.
488 You can download clang-format using your system's package manager, or directly
489 from [llvm.org](http://llvm.org/releases/download.html). You can also view the
490 [documentation](https://clang.llvm.org/docs/ClangFormat.html) on llvm.org.
491 Output from clang-format varies by clang-format version, for best results your
492 version should match the one used on Jenkins. You can find the clang-format
493 version by reading the comment in the `.clang-format` file linked above.
495 Before pushing changes for review you can format your code with:
498 # Apply clang-format to modified .c, .h and .cc files
499 $ clang-format -i --style=file \
500 $(git diff --name-only --diff-filter=ACMR '*.[hc]' '*.cc')
503 Check the .clang-format file for the version used to generate it if there is any
504 difference between your local formatting and the review system.
506 Some Git installations have clang-format integration. Here are some examples:
509 # Apply clang-format to all staged changes:
512 # Clang format all staged and unstaged changes:
513 $ git clang-format -f
515 # Clang format all staged and unstaged changes interactively:
516 $ git clang-format -f -p
519 ## Submitting patches
521 We manage the submission of patches using the
522 [Gerrit](https://www.gerritcodereview.com/) code review tool. This tool
523 implements a workflow on top of the Git version control system to ensure that
524 all changes get peer reviewed and tested prior to their distribution.
528 Browse to [AOMedia Git index](https://aomedia.googlesource.com/) and login with
529 your account (Gmail credentials, for example). Next, follow the
530 `Generate Password` Password link at the top of the page. You’ll be given
531 instructions for creating a cookie to use with our Git repos.
533 ### Contributor agreement
535 You will be required to execute a
536 [contributor agreement](http://aomedia.org/license) to ensure that the AOMedia
537 Project has the right to distribute your changes.
539 ### Testing your code
541 The testing basics are covered in the [testing section](#testing-the-av1-codec)
544 In addition to the local tests, many more (e.g. asan, tsan, valgrind) will run
545 through Jenkins instances upon upload to gerrit.
547 ### Commit message hook
549 Gerrit requires that each submission include a unique Change-Id. You can assign
550 one manually using git commit --amend, but it’s easier to automate it with the
551 commit-msg hook provided by Gerrit.
553 Copy commit-msg to the `.git/hooks` directory of your local repo. Here's an
557 $ curl -Lo aom/.git/hooks/commit-msg https://chromium-review.googlesource.com/tools/hooks/commit-msg
559 # Next, ensure that the downloaded commit-msg script is executable:
560 $ chmod u+x aom/.git/hooks/commit-msg
564 [documentation](https://gerrit-review.googlesource.com/Documentation/user-changeid.html)
565 for more information.
567 ### Upload your change
569 The command line to upload your patch looks like this:
572 $ git push https://aomedia-review.googlesource.com/aom HEAD:refs/for/master
575 ### Incorporating reviewer comments
577 If you previously uploaded a change to Gerrit and the Approver has asked for
578 changes, follow these steps:
580 1. Edit the files to make the changes the reviewer has requested.
581 2. Recommit your edits using the --amend flag, for example:
584 $ git commit -a --amend
587 3. Use the same git push command as above to upload to Gerrit again for another
590 In general, you should not rebase your changes when doing updates in response to
591 review. Doing so can make it harder to follow the evolution of your change in
594 ### Submitting your change
596 Once your change has been Approved and Verified, you can “submit” it through the
597 Gerrit UI. This will usually automatically rebase your change onto the branch
600 Sometimes this can’t be done automatically. If you run into this problem, you
601 must rebase your changes manually:
605 $ git rebase origin/branchname
608 If there are any conflicts, resolve them as you normally would with Git. When
609 you’re done, reupload your change.
611 ### Viewing the status of uploaded changes
613 To check the status of a change that you uploaded, open
614 [Gerrit](https://aomedia-review.googlesource.com/), sign in, and click My >
619 This library is an open source project supported by its community. Please
620 please email aomediacodec@jointdevelopment.kavi.com for help.
624 Bug reports can be filed in the Alliance for Open Media
625 [issue tracker](https://bugs.chromium.org/p/aomedia/issues/list).