| blob | 3c2e5d85db135418347a47fe59de0fc28f89ff1c |
2 <!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">
4 XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
5 This file is generated from xml source: DO NOT EDIT
6 XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
7 -->
9 <link href="./style/css/manual.css" rel="stylesheet" media="all" type="text/css" title="Main stylesheet" />
10 <link href="./style/css/manual-loose-100pc.css" rel="alternate stylesheet" media="all" type="text/css" title="No Sidebar - Default font size" />
14 <p class="menu"><a href="./mod/">Modules</a> | <a href="./mod/directives.html">Directives</a> | <a href="./faq/">FAQ</a> | <a href="./glossary.html">Glossary</a> | <a href="./sitemap.html">Sitemap</a></p>
19 <a href="http://www.apache.org/">Apache</a> > <a href="http://httpd.apache.org/">HTTP Server</a> > <a href="http://httpd.apache.org/docs/">Documentation</a> > <a href="./">Version 2.3</a></div><div id="page-content"><div id="preamble"><h1>Log Files</h1>
21 <p><span>Available Languages: </span><a href="./en/logs.html" title="English"> en </a> |
26 </div>
28 <p>In order to effectively manage a web server, it is necessary
29 to get feedback about the activity and performance of the
30 server as well as any problems that may be occurring. The Apache
31 HTTP Server provides very comprehensive and flexible logging
32 capabilities. This document describes how to configure its
33 logging capabilities, and how to understand what the logs
34 contain.</p>
35 </div>
36 <div id="quickview"><ul id="toc"><li><img alt="" src="./images/down.gif" /> <a href="#security">Security Warning</a></li>
43 </ul></div>
49 <p>Anyone who can write to the directory where Apache is
50 writing a log file can almost certainly gain access to the uid
51 that the server is started as, which is normally root. Do
53 are stored in without being aware of the consequences; see the
55 for details.</p>
57 <p>In addition, log files may contain information supplied
58 directly by the client, without escaping. Therefore, it is
59 possible for malicious clients to insert control-characters in
60 the log files, so care must be taken in dealing with raw
61 logs.</p>
67 <table class="related"><tr><th>Related Modules</th><th>Related Directives</th></tr><tr><td /><td><ul><li><code class="directive"><a href="./mod/core.html#errorlog">ErrorLog</a></code></li><li><code class="directive"><a href="./mod/core.html#loglevel">LogLevel</a></code></li></ul></td></tr></table>
69 <p>The server error log, whose name and location is set by the
71 most important log file. This is the place where Apache httpd
72 will send diagnostic information and record any errors that it
73 encounters in processing requests. It is the first place to
74 look when a problem occurs with starting the server or with the
75 operation of the server, since it will often contain details of
76 what went wrong and how to fix it.</p>
78 <p>The error log is usually written to a file (typically
81 is also possible to have the server send errors to
85 <p>The format of the error log is relatively free-form and
86 descriptive. But there is certain information that is contained
87 in most error log entries. For example, here is a typical
88 message.</p>
92 client denied by server configuration:
93 /export/home/live/ap/htdocs/test
94 </code></p></div>
96 <p>The first item in the log entry is the date and time of the
97 message. The second item lists the severity of the error being
99 directive is used to control the types of errors that are sent
100 to the error log by restricting the severity level. The third
101 item gives the IP address of the client that generated the
102 error. Beyond that is the message itself, which in this case
103 indicates that the server has been configured to deny the
104 client access. The server reports the file-system path (as
105 opposed to the web path) of the requested document.</p>
107 <p>A very wide variety of different messages can appear in the
108 error log. Most look similar to the example above. The error
109 log will also contain debugging output from CGI scripts. Any
111 be copied directly to the error log.</p>
113 <p>It is not possible to customize the error log by adding or
114 removing information. However, error log entries dealing with
115 particular requests have corresponding entries in the <a href="#accesslog">access log</a>. For example, the above example
116 entry corresponds to an access log entry with status code 403.
117 Since it is possible to customize the access log, you can
118 obtain more information about error conditions using that log
119 file.</p>
121 <p>During testing, it is often useful to continuously monitor
122 the error log for any problems. On Unix systems, you can
123 accomplish this using:</p>
126 tail -f error_log
127 </code></p></div>
133 <table class="related"><tr><th>Related Modules</th><th>Related Directives</th></tr><tr><td><ul><li><code class="module"><a href="./mod/mod_log_config.html">mod_log_config</a></code></li><li><code class="module"><a href="./mod/mod_setenvif.html">mod_setenvif</a></code></li></ul></td><td><ul><li><code class="directive"><a href="./mod/mod_log_config.html#customlog">CustomLog</a></code></li><li><code class="directive"><a href="./mod/mod_log_config.html#logformat">LogFormat</a></code></li><li><code class="directive"><a href="./mod/mod_setenvif.html#setenvif">SetEnvIf</a></code></li></ul></td></tr></table>
135 <p>The server access log records all requests processed by the
136 server. The location and content of the access log are
137 controlled by the <code class="directive"><a href="./mod/mod_log_config.html#customlog">CustomLog</a></code>
138 directive. The <code class="directive"><a href="./mod/mod_log_config.html#logformat">LogFormat</a></code>
139 directive can be used to simplify the selection of
140 the contents of the logs. This section describes how to configure the server
141 to record information in the access log.</p>
143 <p>Of course, storing the information in the access log is only
144 the start of log management. The next step is to analyze this
145 information to produce useful statistics. Log analysis in
146 general is beyond the scope of this document, and not really
147 part of the job of the web server itself. For more information
148 about this topic, and for applications which perform log
149 analysis, check the <a href="http://dmoz.org/Computers/Software/Internet/Site_Management/Log_analysis/">
150 Open Directory</a> or <a href="http://dir.yahoo.com/Computers_and_Internet/Software/Internet/World_Wide_Web/Servers/Log_Analysis_Tools/">
153 <p>Various versions of Apache httpd have used other modules and
154 directives to control access logging, including
155 mod_log_referer, mod_log_agent, and the
156 <code>TransferLog</code> directive. The <code class="directive"><a href="./mod/mod_log_config.html#customlog">CustomLog</a></code> directive now subsumes
157 the functionality of all the older directives.</p>
159 <p>The format of the access log is highly configurable. The format
160 is specified using a format string that looks much like a C-style
161 printf(1) format string. Some examples are presented in the next
162 sections. For a complete list of the possible contents of the
163 format string, see the <code class="module"><a href="./mod/mod_log_config.html">mod_log_config</a></code> <a href="mod/mod_log_config.html#formats">format strings</a>.</p>
168 <p>A typical configuration for the access log might look as
169 follows.</p>
173 CustomLog logs/access_log common
174 </code></p></div>
177 associates it with a particular log format string. The format
178 string consists of percent directives, each of which tell the
179 server to log a particular piece of information. Literal
180 characters may also be placed in the format string and will be
181 copied directly into the log output. The quote character
183 it to prevent it from being interpreted as the end of the
184 format string. The format string may also contain the special
188 <p>The <code class="directive"><a href="./mod/mod_log_config.html#customlog">CustomLog</a></code>
189 directive sets up a new log file using the defined
190 <em>nickname</em>. The filename for the access log is relative to
191 the <code class="directive"><a href="./mod/core.html#serverroot">ServerRoot</a></code> unless it
192 begins with a slash.</p>
194 <p>The above configuration will write log entries in a format
195 known as the Common Log Format (CLF). This standard format can
196 be produced by many different web servers and read by many log
197 analysis programs. The log file entries produced in CLF will
198 look something like this:</p>
201 127.0.0.1 - frank [10/Oct/2000:13:55:36 -0700] "GET
203 </code></p></div>
205 <p>Each part of this log entry is described below.</p>
207 <dl>
208 <dt><code>127.0.0.1</code> (<code>%h</code>)</dt>
210 <dd>This is the IP address of the client (remote host) which
211 made the request to the server. If <code class="directive"><a href="./mod/core.html#hostnamelookups">HostnameLookups</a></code> is
212 set to <code>On</code>, then the server will try to determine
213 the hostname and log it in place of the IP address. However,
214 this configuration is not recommended since it can
215 significantly slow the server. Instead, it is best to use a
216 log post-processor such as <code class="program"><a href="./programs/logresolve.html">logresolve</a></code> to determine
217 the hostnames. The IP address reported here is not
218 necessarily the address of the machine at which the user is
219 sitting. If a proxy server exists between the user and the
220 server, this address will be the address of the proxy, rather
221 than the originating machine.</dd>
223 <dt><code>-</code> (<code>%l</code>)</dt>
226 piece of information is not available. In this case, the
227 information that is not available is the RFC 1413 identity of
228 the client determined by <code>identd</code> on the clients
229 machine. This information is highly unreliable and should
230 almost never be used except on tightly controlled internal
231 networks. Apache httpd will not even attempt to determine
232 this information unless <code class="directive"><a href="./mod/core.html#identitycheck">IdentityCheck</a></code> is set
233 to <code>On</code>.</dd>
235 <dt><code>frank</code> (<code>%u</code>)</dt>
237 <dd>This is the userid of the person requesting the document
238 as determined by HTTP authentication. The same value is
239 typically provided to CGI scripts in the
240 <code>REMOTE_USER</code> environment variable. If the status
241 code for the request (see below) is 401, then this value
242 should not be trusted because the user is not yet
243 authenticated. If the document is not password protected,
245 one.</dd>
247 <dt><code>[10/Oct/2000:13:55:36 -0700]</code>
248 (<code>%t</code>)</dt>
250 <dd>
251 The time that the request was received.
252 The format is:
255 <code>[day/month/year:hour:minute:second zone]<br />
256 day = 2*digit<br />
257 month = 3*letter<br />
258 year = 4*digit<br />
259 hour = 2*digit<br />
260 minute = 2*digit<br />
261 second = 2*digit<br />
262 zone = (`+' | `-') 4*digit</code>
263 </p>
264 It is possible to have the time displayed in another format
265 by specifying <code>%{format}t</code> in the log format
266 string, where <code>format</code> is as in
267 <code>strftime(3)</code> from the C standard library.
268 </dd>
273 <dd>The request line from the client is given in double
274 quotes. The request line contains a great deal of useful
275 information. First, the method used by the client is
276 <code>GET</code>. Second, the client requested the resource
277 <code>/apache_pb.gif</code>, and third, the client used the
278 protocol <code>HTTP/1.0</code>. It is also possible to log
279 one or more parts of the request line independently. For
281 the method, path, query-string, and protocol, resulting in
284 <dt><code>200</code> (<code>%>s</code>)</dt>
286 <dd>This is the status code that the server sends back to the
287 client. This information is very valuable, because it reveals
288 whether the request resulted in a successful response (codes
289 beginning in 2), a redirection (codes beginning in 3), an
290 error caused by the client (codes beginning in 4), or an
291 error in the server (codes beginning in 5). The full list of
292 possible status codes can be found in the <a href="http://www.w3.org/Protocols/rfc2616/rfc2616.txt">HTTP
293 specification</a> (RFC2616 section 10).</dd>
295 <dt><code>2326</code> (<code>%b</code>)</dt>
297 <dd>The last part indicates the size of the object returned
298 to the client, not including the response headers. If no
299 content was returned to the client, this value will be
301 <code>%B</code> instead.</dd>
302 </dl>
308 <p>Another commonly used format string is called the Combined
309 Log Format. It can be used as follows.</p>
314 CustomLog log/access_log combined
315 </code></p></div>
317 <p>This format is exactly the same as the Common Log Format,
318 with the addition of two more fields. Each of the additional
319 fields uses the percent-directive
320 <code>%{<em>header</em>}i</code>, where <em>header</em> can be
321 any HTTP request header. The access log under this format will
322 look like:</p>
325 127.0.0.1 - frank [10/Oct/2000:13:55:36 -0700] "GET
328 (Win98; I ;Nav)"
329 </code></p></div>
331 <p>The additional fields are:</p>
333 <dl>
338 site that the client reports having been referred from. (This
339 should be the page that links to or includes
340 <code>/apache_pb.gif</code>).</dd>
345 <dd>The User-Agent HTTP request header. This is the
346 identifying information that the client browser reports about
347 itself.</dd>
348 </dl>
354 <p>Multiple access logs can be created simply by specifying
355 multiple <code class="directive"><a href="./mod/mod_log_config.html#customlog">CustomLog</a></code>
356 directives in the configuration
357 file. For example, the following directives will create three
358 access logs. The first contains the basic CLF information,
359 while the second and third contain referer and browser
360 information. The last two <code class="directive"><a href="./mod/mod_log_config.html#customlog">CustomLog</a></code> lines show how
361 to mimic the effects of the <code>ReferLog</code> and <code>AgentLog</code> directives.</p>
365 CustomLog logs/access_log common<br />
368 </code></p></div>
370 <p>This example also shows that it is not necessary to define a
371 nickname with the <code class="directive"><a href="./mod/mod_log_config.html#logformat">LogFormat</a></code> directive. Instead,
372 the log format can be specified directly in the <code class="directive"><a href="./mod/mod_log_config.html#customlog">CustomLog</a></code> directive.</p>
378 <p>There are times when it is convenient to exclude certain
379 entries from the access logs based on characteristics of the
380 client request. This is easily accomplished with the help of <a href="env.html">environment variables</a>. First, an
381 environment variable must be set to indicate that the request
382 meets certain conditions. This is usually accomplished with
383 <code class="directive"><a href="./mod/mod_setenvif.html#setenvif">SetEnvIf</a></code>. Then the
384 <code>env=</code> clause of the <code class="directive"><a href="./mod/mod_log_config.html#customlog">CustomLog</a></code> directive is used to
385 include or exclude requests where the environment variable is
386 set. Some examples:</p>
389 # Mark requests from the loop-back interface<br />
391 # Mark requests for the robots.txt file<br />
393 # Log what remains<br />
394 CustomLog logs/access_log common env=!dontlog
395 </code></p></div>
397 <p>As another example, consider logging requests from
398 english-speakers to one log file, and non-english speakers to a
399 different log file.</p>
403 CustomLog logs/english_log common env=english<br />
404 CustomLog logs/non_english_log common env=!english
405 </code></p></div>
407 <p>Although we have just shown that conditional logging is very
408 powerful and flexible, it is not the only way to control the
409 contents of the logs. Log files are more useful when they
410 contain a complete record of server activity. It is often
411 easier to simply post-process the log files to remove requests
412 that you do not want to consider.</p>
419 <p>On even a moderately busy server, the quantity of
420 information stored in the log files is very large. The access
421 log file typically grows 1 MB or more per 10,000 requests. It
422 will consequently be necessary to periodically rotate the log
423 files by moving or deleting the existing logs. This cannot be
424 done while the server is running, because Apache will continue
425 writing to the old log file as long as it holds the file open.
427 moved or deleted so that it will open new log files.</p>
429 <p>By using a <em>graceful</em> restart, the server can be
430 instructed to open new log files without losing any existing or
431 pending connections from clients. However, in order to
432 accomplish this, the server must continue to write to the old
433 log files while it finishes serving old requests. It is
434 therefore necessary to wait for some time after the restart
435 before doing any processing on the log files. A typical
436 scenario that simply rotates the logs and compresses the old
437 logs to save space is:</p>
440 mv access_log access_log.old<br />
441 mv error_log error_log.old<br />
442 apachectl graceful<br />
443 sleep 600<br />
444 gzip access_log.old error_log.old
445 </code></p></div>
447 <p>Another way to perform log rotation is using <a href="#piped">piped logs</a> as discussed in the next
448 section.</p>
454 <p>Apache httpd is capable of writing error and access log
455 files through a pipe to another process, rather than directly
456 to a file. This capability dramatically increases the
457 flexibility of logging, without adding code to the main server.
458 In order to write logs to a pipe, simply replace the filename
460 of the executable which should accept log entries on its
461 standard input. Apache will start the piped-log process when
462 the server starts, and will restart it if it crashes while the
463 server is running. (This last feature is why we can refer to
466 <p>Piped log processes are spawned by the parent Apache httpd
467 process, and inherit the userid of that process. This means
468 that piped log programs usually run as root. It is therefore
469 very important to keep the programs simple and secure.</p>
471 <p>One important use of piped logs is to allow log rotation
472 without having to restart the server. The Apache HTTP Server
473 includes a simple program called <code class="program"><a href="./programs/rotatelogs.html">rotatelogs</a></code>
474 for this purpose. For example, to rotate the logs every 24 hours, you
475 can use:</p>
478 CustomLog "|/usr/local/apache/bin/rotatelogs
480 </code></p></div>
482 <p>Notice that quotes are used to enclose the entire command
483 that will be called for the pipe. Although these examples are
484 for the access log, the same technique can be used for the
485 error log.</p>
487 <p>A similar but much more flexible log rotation program
489 is available at an external site.</p>
491 <p>As with conditional logging, piped logs are a very powerful
492 tool, but they should not be used where a simpler solution like
493 off-line post-processing is available.</p>
500 hosts</a>, there are several options for dealing with log
501 files. First, it is possible to use logs exactly as in a
502 single-host server. Simply by placing the logging directives
503 outside the <code class="directive"><a href="./mod/core.html#virtualhost"><VirtualHost></a></code> sections in the
504 main server context, it is possible to log all requests in the
505 same access log and error log. This technique does not allow
506 for easy collection of statistics on individual virtual
507 hosts.</p>
509 <p>If <code class="directive"><a href="./mod/mod_log_config.html#customlog">CustomLog</a></code>
511 directives are placed inside a
513 section, all requests or errors for that virtual host will be
514 logged only to the specified file. Any virtual host which does
515 not have logging directives will still have its requests sent
516 to the main server logs. This technique is very useful for a
517 small number of virtual hosts, but if the number of hosts is
518 very large, it can be complicated to manage. In addition, it
520 descriptors</a>.</p>
522 <p>For the access log, there is a very good compromise. By
523 adding information on the virtual host to the log format
524 string, it is possible to log all hosts to the same log, and
525 later split the log into individual files. For example,
526 consider the following directives.</p>
530 comonvhost<br />
531 CustomLog logs/access_log comonvhost
532 </code></p></div>
534 <p>The <code>%v</code> is used to log the name of the virtual
535 host that is serving the request. Then a program like <a href="programs/other.html">split-logfile</a> can be used to
536 post-process the access log in order to split it into one file
537 per virtual host.</p>
543 <table class="related"><tr><th>Related Modules</th><th>Related Directives</th></tr><tr><td><ul><li><code class="module"><a href="./mod/mod_logio.html">mod_logio</a></code></li><li><code class="module"><a href="./mod/mod_log_forensic.html">mod_log_forensic</a></code></li><li><code class="module"><a href="./mod/mod_cgi.html">mod_cgi</a></code></li><li><code class="module"><a href="./mod/mod_rewrite.html">mod_rewrite</a></code></li></ul></td><td><ul><li><code class="directive"><a href="./mod/mod_log_config.html#logformat">LogFormat</a></code></li><li><code class="directive"><a href="./mod/mod_log_forensic.html#forensiclog">ForensicLog</a></code></li><li><code class="directive"><a href="./mod/mpm_common.html#pidfile">PidFile</a></code></li><li><code class="directive"><a href="./mod/mod_rewrite.html#rewritelog">RewriteLog</a></code></li><li><code class="directive"><a href="./mod/mod_rewrite.html#rewriteloglevel">RewriteLogLevel</a></code></li><li><code class="directive"><a href="./mod/mod_cgi.html#scriptlog">ScriptLog</a></code></li><li><code class="directive"><a href="./mod/mod_cgi.html#scriptlogbuffer">ScriptLogBuffer</a></code></li><li><code class="directive"><a href="./mod/mod_cgi.html#scriptloglength">ScriptLogLength</a></code></li></ul></td></tr></table>
545 <h3>Logging actual bytes sent and received</h3>
548 <p><code class="module"><a href="./mod/mod_logio.html">mod_logio</a></code> adds in two additional
549 <code class="directive"><a href="./mod/mod_log_config.html#logformat">LogFormat</a></code> fields
550 (%I and %O) that log the actual number of bytes received and sent
551 on the network.</p>
554 <h3>Forensic Logging</h3>
557 <p><code class="module"><a href="./mod/mod_log_forensic.html">mod_log_forensic</a></code> provides for forensic logging of
558 client requests. Logging is done before and after processing a
559 request, so the forensic log contains two log lines for each
560 request. The forensic logger is very strict with no customizations.
561 It can be an invaluable debugging and security tool.</p>
567 <p>On startup, Apache httpd saves the process id of the parent
568 httpd process to the file <code>logs/httpd.pid</code>. This
569 filename can be changed with the <code class="directive"><a href="./mod/mpm_common.html#pidfile">PidFile</a></code> directive. The
570 process-id is for use by the administrator in restarting and
571 terminating the daemon by sending signals to the parent
572 process; on Windows, use the -k command line option instead.
574 and Restarting</a> page.</p>
580 <p>In order to aid in debugging, the
582 allows you to record the input to and output from CGI scripts.
583 This should only be used in testing - not for live servers.
590 <p>When using the powerful and complex features of <a href="mod/mod_rewrite.html">mod_rewrite</a>, it is almost
591 always necessary to use the <code class="directive"><a href="./mod/mod_rewrite.html#rewritelog">RewriteLog</a></code> to help
592 in debugging. This log file produces a detailed analysis of how
593 the rewriting engine transforms requests. The level of detail
594 is controlled by the <code class="directive"><a href="./mod/mod_rewrite.html#rewriteloglevel">RewriteLogLevel</a></code> directive.</p>
596 </div></div>
598 <p><span>Available Languages: </span><a href="./en/logs.html" title="English"> en </a> |
604 <p class="apache">Copyright 2009 The Apache Software Foundation.<br />Licensed under the <a href="http://www.apache.org/licenses/LICENSE-2.0">Apache License, Version 2.0</a>.</p>
605 <p class="menu"><a href="./mod/">Modules</a> | <a href="./mod/directives.html">Directives</a> | <a href="./faq/">FAQ</a> | <a href="./glossary.html">Glossary</a> | <a href="./sitemap.html">Sitemap</a></p></div>
606 </body></html>