Add cmake macro for downloading third-party sources, and caching them. (#8681)
commitc1aa9d223a4d361218b0bc72ff87cbc48f59eaf1
authorFred Emmott <fe@fb.com>
Wed, 8 Apr 2020 16:20:20 +0000 (8 09:20 -0700)
committerFacebook GitHub Bot <facebook-github-bot@users.noreply.github.com>
Wed, 8 Apr 2020 16:24:28 +0000 (8 09:24 -0700)
tree402a5ee2c4942b296aeae4897a73b63cddcfb91e
parent521d54475cf1c125023eee80b7cfa9a16855929b
Add cmake macro for downloading third-party sources, and caching them. (#8681)

Summary:
Just doing one project for now to keep this diff small.

Can run in three modes:
- always download from origin (default)
- try cache, fall back to origin
- only use cache, never try origin

Also added an option to generate a text file containing a list of all
the origin URLs; this could be useful for populating a cache.
Pull Request resolved: https://github.com/facebook/hhvm/pull/8681

Test Plan:
Locally built `third-party/fmt` in a bunch of configures

# Forcing cache

Configured with

```
-DHHVM_THIRD_PARTY_SOURCE_CACHE_PREFIX="http://127.0.0.1/nopenopenope/"
-DHHVM_THIRD_PARTY_SOURCE_ONLY_USE_CACHE=ON`
```

```
-- Using src='http://127.0.0.1:8080/nopenopenope/fmt-6.1.2.zip
```

... fails after lots of retries ...

# Optional cache

Rebuild with `-DHHVM_THIRD_PARTY_SOURCE_CACHE_PREFIX="http://127.0.0.1/nopenopenope/" -DHHVM_THIRD_PARTY_SOURCE_ONLY_USE_CACHE=OFF`

```
-- Using src='http://127.0.0.1:8080/nopenopenope/fmt-6.1.2.zip'
-- Using src='https://github.com/fmtlib/fmt/releases/download/6.1.2/fmt-6.1.2.zip'
-- [download 32% complete]
```

Succeeds.

# Default build

No cmake args, no attempt at using a cache

```
-- Using src='https://github.com/fmtlib/fmt/releases/download/6.1.2/fmt-6.1.2.zip'
```

succeeds

# Generating a list of URLs to cache

Re-ran cmake with

`-DHHVM_THIRD_PARTY_SOURCE_URL_LIST_OUTPUT=/tmp/urls.txt"`

Got

```
% cat /tmp/urls.txt
https://github.com/fmtlib/fmt/releases/download/6.1.2/fmt-6.1.2.zip
%
```

Reviewed By: jjergus

Differential Revision: D20873623

fbshipit-source-id: a683066e02d922fb71ca434fbea5f2062d39e279
CMake/HPHPFunctions.cmake
third-party/fmt/CMakeLists.txt