| blob | 832d3b0a97884886dbbcc74f22c992f4b7e48b31 |
1 <html>
2 <head>
7 td {
9 }
10 </style>
11 </head>
12 <body>
14 <!--#include virtual="../menu.html.incl"-->
20 <p>This document first describes the low-level
21 interface for using PTH and then briefly elaborates on its design and
22 implementation. If you are interested in the end-user view, please see the
29 line options for generating and using PTH files.<p>
37 files. Similarly, PTH files can be used as prefix headers using the
40 <pre>
41 $ clang-cc -include-pth test.h.pth test.c -o test.s
42 </pre>
46 file. This means that the contents of the PTH file are searched as substitutes
49 option:</p>
51 <pre>
52 $ cat test.h
54 $ clang-cc -emit-pth test.h -o test.h.pth
55 $ cat test.c
56 #include "test.h"
57 $ clang-cc test.c -o test -token-cache test.h.pth
58 </pre>
63 interface used to both implement the high-level PTH interface as well as to
64 provide alternative means to use PTH-style caching.</p>
68 <p>Unlike GCC's precompiled headers, which cache the full ASTs and preprocessor
69 state of a header file, Clang's pretokenized header files mainly cache the raw
71 source file into keywords, identifiers, and operators. Consequently, PTH serves
72 to mainly directly speed up the lexing and preprocessing of a source file, while
73 parsing and type-checking must be completely redone every time a PTH file is
74 used.</p>
78 <p>In the long term there are plans to provide an alternate PCH implementation
79 for Clang that also caches the work for parsing and type checking the contents
80 of header files. The current implementation of PCH in Clang as pretokenized
81 header files was motivated by the following factors:<p>
83 <ul>
86 Clang's lexer can handle, including C, Objective-C, and (in the early stages)
87 C++. This means development on language features at the parsing level or above
88 (which is basically almost all interesting pieces) does not require PTH to be
89 modified.</p></li>
92 implementation, making it easy to test. Further, because the machinery for PTH
93 resides at the lower-levels of the Clang library stack it is fairly
94 straightforward to profile and optimize.</li>
95 </ul>
97 <p>Further, compared to GCC's PCH implementation (which is the dominate
98 precompiled header file implementation that Clang can be directly compared
99 against) the PTH design in Clang yields several attractive features:</p>
101 <ul>
104 those of several other compilers), Clang's PTH files are architecture
105 independent, requiring only a single PTH file when building an program for
106 multiple architectures.</p>
108 <p>For example, on Mac OS X one may wish to
110 (i386), and 64-bit Intel architectures. In contrast, GCC requires a PCH file for
111 each architecture, as the definitions of types in the AST are
112 architecture-specific. Since a Clang PTH file essentially represents a lexical
113 cache of header files, a single PTH file can be safely used when compiling for
114 multiple architectures. This can also reduce compile times because only a single
115 PTH file needs to be generated during a build instead of several.</p></li>
119 Clang, however, memory maps PTH files as read-only, meaning that multiple
121 memory-mapped PTH file. In comparison, GCC also memory maps its PCH files but
122 also modifies those pages in memory, incurring the copy-on-write costs. The
123 read-only nature of PTH can greatly reduce memory pressure for builds involving
124 multiple cores, thus improving overall scalability.</p></li>
127 of the time needed to generate GCC's PCH files. Since PTH/PCH generation is a
128 serial operation that typically blocks progress during a build, faster
129 generation time leads to improved processor utilization with parallel builds on
130 multicore machines.</p></li>
132 </ul>
134 <p>Despite these strengths, PTH's simple design suffers some algorithmic
135 handicaps compared to other PCH strategies such as those used by GCC. While PTH
136 can greatly speed up the processing time of a header file, the amount of work
137 required to process a header file is still roughly linear in the size of the
138 header file. In contrast, the amount of work done by GCC to process a
139 precompiled header is (theoretically) constant (the ASTs for the header are
140 literally memory mapped into the compiler). This means that only the pieces of
141 the header file that are referenced by the source file including the header are
142 the only ones the compiler needs to process during actual compilation. While
143 GCC's particular implementation of PCH mitigates some of these algorithmic
144 strengths via the use of copy-on-write pages, the approach itself can
145 fundamentally dominate at an algorithmic level, especially when one considers
146 header files of arbitrary size.</p>
148 <p>There are plans to potentially implement an complementary PCH implementation
149 for Clang based on the lazy deserialization of ASTs. This approach would
150 theoretically have the same constant-time algorithmic advantages just mentioned
151 but would also retain some of the strengths of PTH such as reduced memory
152 pressure (ideal for multi-core builds).</p>
156 <p>While the main optimization employed by PTH is to reduce lexing time of
157 header files by caching pre-lexed tokens, PTH also employs several other
158 optimizations to speed up the processing of header files:</p>
160 <ul>
165 involved in context-switching to the kernel to resolve included files.</p></li>
168 PTH files record the basic structure of nested preprocessor blocks. When the
169 condition of the preprocessor block is false, all of its tokens are immediately
170 skipped instead of requiring them to be handled by Clang's
171 preprocessor.</p></li>
173 </ul>
175 </div>
176 </body>
177 </html>