2 @setfilename ../info/url
3 @settitle URL Programmer's Manual
8 @c @setchapternewpage odd
13 %\global\baselineskip 30pt % for printing in double space
15 @dircategory World Wide Web
16 @dircategory GNU Emacs Lisp
18 * URL: (url). URL loading package.
22 This file documents the URL loading package.
24 Copyright @copyright{} 1996, 1997, 1998, 1999, 2002, 2004,
25 2005, 2006 Free Software Foundation, Inc.@*
26 Copyright @copyright{} 1993, 1994, 1995, 1996 William M. Perry
28 Permission is granted to copy, distribute and/or modify this document
29 under the terms of the GNU Free Documentation License, Version 1.2 or
30 any later version published by the Free Software Foundation; with the
31 Invariant Sections being
32 ``GNU GENERAL PUBLIC LICENSE''. A copy of the
33 license is included in the section entitled ``GNU Free Documentation
40 @center @titlefont{URL}
41 @center @titlefont{Programmer's Manual}
43 @center First Edition, URL Version 2.0
45 @c @center December 1999
47 @center William M. Perry
48 @center @email{wmperry@@gnu.org}
50 @center @email{fx@@gnu.org}
52 @vskip 0pt plus 1filll
53 Copyright @copyright{} 1993, 1994, 1995, 1996 William M. Perry@*
54 Copyright @copyright{} 1996, 1997, 1998, 1999, 2002, 2003, 2004,
55 2005, 2006 Free Software Foundation, Inc.
57 Permission is granted to copy, distribute and/or modify this document
58 under the terms of the GNU Free Documentation License, Version 1.2 or
59 any later version published by the Free Software Foundation; with the
60 Invariant Sections being
61 ``GNU GENERAL PUBLIC LICENSE''. A copy of the
62 license is included in the section entitled ``GNU Free Documentation
72 * Getting Started:: Preparing your program to use URLs.
73 * Retrieving URLs:: How to use this package to retrieve a URL.
74 * Supported URL Types:: Descriptions of URL types currently supported.
75 * Defining New URLs:: How to define a URL loader for a new protocol.
76 * General Facilities:: URLs can be cached, accessed via a gateway
77 and tracked in a history list.
78 * Customization:: Variables you can alter.
85 @chapter Getting Started
86 @cindex URLs, definition
89 @dfn{Uniform Resource Locators} (URLs) are a specific form of
90 @dfn{Uniform Resource Identifiers} (URI) described in RFC 2396 which
91 updates RFC 1738 and RFC 1808. RFC 2016 defines uniform resource
94 URIs have the form @var{scheme}:@var{scheme-specific-part}, where the
95 @var{scheme}s supported by this library are described below.
96 @xref{Supported URL Types}.
98 FTP, NFS, HTTP, HTTPS, @code{rlogin}, @code{telnet}, tn3270,
99 IRC and gopher URLs all have the form
102 @var{scheme}://@r{[}@var{userinfo}@@@r{]}@var{hostname}@r{[}:@var{port}@r{]}@r{[}/@var{path}@r{]}
105 where @samp{@r{[}} and @samp{@r{]}} delimit optional parts.
106 @var{userinfo} sometimes takes the form @var{username}:@var{password}
107 but you should beware of the security risks of sending cleartext
108 passwords. @var{hostname} may be a domain name or a dotted decimal
109 address. If the @samp{:@var{port}} is omitted then the library will
110 use the `well known' port for that service when accessing URLs. With
111 the possible exception of @code{telnet}, it is rare for ports to be
112 specified, and it is possible using a non-standard port may have
113 undesired consequences if a different service is listening on that
114 port (e.g., an HTTP URL specifying the SMTP port can cause mail to be
115 sent). @c , but @xref{Other Variables, url-bad-port-list}.
116 The meaning of the @var{path} component depends on the service.
120 * Parsed URLs:: URLs are parsed into vector structures.
124 @section Configuration
126 @defvar url-configuration-directory
127 @cindex @file{~/.url}
128 @cindex configuration files
129 The directory in which URL configuration files, the cache etc.,
130 reside. Default @file{~/.url}.
136 The library functions typically operate on @dfn{parsed} versions of
137 URLs. These are actually vectors of the form:
140 [@var{type} @var{user} @var{password} @var{host} @var{port} @var{file} @var{target} @var{attributes} @var{full}]
146 is the type of the URL scheme, e.g., @code{http}
148 is the username associated with it, or @code{nil};
150 is the user password associated with it, or @code{nil};
152 is the host name associated with it, or @code{nil};
154 is the port number associated with it, or @code{nil};
156 is the `file' part of it, or @code{nil}. This doesn't necessarily
157 actually refer to a file;
159 is the target part, or @code{nil};
161 is the attributes associated with it, or @code{nil};
163 is @code{t} for a fully-specified URL, with a host part indicated by
164 @samp{//} after the scheme part.
174 @findex url-attributes
178 @findex url-set-password
182 @findex url-set-target
183 @findex url-set-attributes
185 These attributes have accessors named @code{url-@var{part}}, where
186 @var{part} is the name of one of the elements above, e.g.,
187 @code{url-host}. Similarly, there are setters of the form
188 @code{url-set-@var{part}}.
190 There are functions for parsing and unparsing between the string and
193 @defun url-generic-parse-url url
194 Return a parsed version of the string @var{url}.
197 @defun url-recreate-url url
198 @cindex unparsing URLs
199 Recreates a URL string from the parsed @var{url}.
202 @node Retrieving URLs
203 @chapter Retrieving URLs
205 @defun url-retrieve-synchronously url
206 Retrieve @var{url} synchronously and return a buffer containing the
207 data. @var{url} is either a string or a parsed URL structure. Return
208 @code{nil} if there are no data associated with it (the case for dired,
209 info, or mailto URLs that need no further processing).
212 @defun url-retrieve url callback &optional cbargs
213 Retrieve @var{url} asynchronously and call @var{callback} with args
214 @var{cbargs} when finished. The callback is called when the object
215 has been completely retrieved, with the current buffer containing the
216 object and any MIME headers associated with it. @var{url} is either a
217 string or a parsed URL structure. Returns the buffer @var{url} will
218 load into, or @code{nil} if the process has already completed.
221 @node Supported URL Types
222 @chapter Supported URL Types
225 * http/https:: Hypertext Transfer Protocol.
226 * file/ftp:: Local files and FTP archives.
227 * info:: Emacs `Info' pages.
228 * mailto:: Sending email.
229 * news/nntp/snews:: Usenet news.
230 * rlogin/telnet/tn3270:: Remote host connectivity.
231 * irc:: Internet Relay Chat.
232 * data:: Embedded data URLs.
233 * nfs:: Networked File System
240 * ldap:: Lightweight Directory Access Protocol
241 * imap:: IMAP mailboxes.
242 * man:: Unix man pages.
246 @section @code{http} and @code{https}
248 The scheme @code{http} is Hypertext Transfer Protocol. The library
249 supports version 1.1, specified in RFC 2616. (This supersedes 1.0,
250 defined in RFC 1945) HTTP URLs have the following form, where most of
251 the parts are optional:
253 http://@var{user}:@var{password}@var{host}:@var{port}/@var{path}?@var{searchpart}#@var{fragment}
255 @c The @code{:@var{port}} part is optional, and @var{port} defaults to
256 @c 80. The @code{/@var{path}} part, if present, is a slash-separated
257 @c series elements. The @code{?@var{searchpart}}, if present, is the
258 @c query for a search or the content of a form submission. The
259 @c @code{#fragment} part, if present, is a location in the document.
261 The scheme @code{https} is a secure version of @code{http}, with
262 transmission via SSL. It is defined in RFC 2069. Its default port is
263 443. This scheme depends on SSL support in Emacs via the
264 @file{ssl.el} library and is actually implemented by forcing the
265 @code{ssl} gateway method to be used. @xref{Gateways in general}.
267 @defopt url-honor-refresh-requests
268 This controls honouring of HTTP @samp{Refresh} headers by which
269 servers can direct clients to reload documents from the same URL or a
270 or different one. @code{nil} means they will not be honoured,
271 @code{t} (the default) means they will always be honoured, and
272 otherwise the user will be asked on each request.
278 * HTTP language/coding::
280 * Dealing with HTTP documents::
286 @defopt url-cookie-file
287 The file in which cookies are stored, defaulting to @file{cookies} in
288 the directory specified by @code{url-configuration-directory}.
291 @defopt url-cookie-confirmation
292 Specifies whether confirmation is require to accept cookies.
295 @defopt url-cookie-multiple-line
296 Specifies whether to put all cookies for the server on one line in the
297 HTTP request to satisfy broken servers like
298 @url{http://www.hotmail.com}.
301 @defopt url-cookie-trusted-urls
302 A list of regular expressions matching URLs from which to accept
306 @defopt url-cookie-untrusted-urls
307 A list of regular expressions matching URLs from which to reject
311 @defopt url-cookie-save-interval
312 The number of seconds between automatic saves of cookies to disk.
317 @node HTTP language/coding
318 @subsection Language and Encoding Preferences
320 HTTP allows clients to express preferences for the language and
321 encoding of documents which servers may honour. For each of these
322 variables, the value is a string; it can specify a single choice, or
323 it can be a comma-separated list.
325 Normally this list ordered by descending preference. However, each
326 element can be followed by @samp{;q=@var{priority}} to specify its
327 preference level, a decimal number from 0 to 1; e.g., for
328 @code{url-mime-language-string}, @w{@code{"de, en-gb;q=0.8,
329 en;q=0.7"}}. An element that has no @samp{;q} specification has
332 @defopt url-mime-charset-string
333 @cindex character sets
334 @cindex coding systems
335 This variable specifies a preference for character sets when documents
336 can be served in more than one encoding.
338 HTTP allows specifying a series of MIME charsets which indicate your
339 preferred character set encodings, e.g., Latin-9 or Big5, and these
340 can be weighted. The default series is generated automatically from
341 the associated MIME types of all defined coding systems, sorted by the
342 coding system priority specified in Emacs. @xref{Recognize Coding, ,
343 Recognizing Coding Systems, emacs, The GNU Emacs Manual}.
346 @defopt url-mime-language-string
347 @cindex language preferences
348 A string specifying the preferred language when servers can serve
349 files in several languages. Use RFC 1766 abbreviations, e.g.,
350 @samp{en} for English, @samp{de} for German.
352 The string can be @code{"*"} to get the first available language (as
353 opposed to the default).
356 @node HTTP URL Options
357 @subsection HTTP URL Options
359 HTTP supports an @samp{OPTIONS} method describing things supported by
362 @defun url-http-options url
363 Returns a property list describing options available for URL. The
364 property list members are:
368 A list of symbols specifying what HTTP methods the resource
373 A list of numbers specifying what DAV protocol/schema versions are
378 A list of supported DASL search types supported (string form).
381 A list of the units available for use in partial document fetches.
385 The @dfn{Platform For Privacy Protection} description for the resource.
386 Currently this is just the raw header contents.
391 @node Dealing with HTTP documents
392 @subsection Dealing with HTTP documents
394 HTTP URLs are retrieved into a buffer containing the HTTP headers
395 followed by the body. Since the headers are quasi-MIME, they may be
396 processed using the MIME library. @xref{Top,, Emacs MIME,
397 emacs-mime, The Emacs MIME Manual}. The URL package provides a
398 function to do this in general:
400 @defun url-decode-text-part handle &optional coding
401 This function decodes charset-encoded text in the current buffer. In
402 Emacs, the buffer is expected to be unibyte initially and is set to
403 multibyte after decoding.
404 HANDLE is the MIME handle of the original part. CODING is an explicit
405 coding to use, overriding what the MIME headers specify.
406 The coding system used for the decoding is returned.
408 Note that this function doesn't deal with @samp{http-equiv} charset
409 specifications in HTML @samp{<meta>} elements.
413 @section file and ftp
416 @cindex File Transfer Protocol
417 @cindex compressed files
421 ftp://@var{user}:@var{password}@@@var{host}:@var{port}/@var{file}
422 file://@var{user}:@var{password}@@@var{host}:@var{port}/@var{file}
425 These schemes are defined in RFC 1808.
426 @samp{ftp:} and @samp{file:} are synonymous in this library. They
427 allow reading arbitrary files from hosts. Either @samp{ange-ftp}
428 (Emacs) or @samp{efs} (XEmacs) is used to retrieve them from remote
429 hosts. Local files are accessed directly.
431 Compressed files are handled, but support is hard-coded so that
432 @code{jka-compr-compression-info-list} and so on have no affect.
433 Suffixes recognized are @samp{.z}, @samp{.gz}, @samp{.Z} and
436 @defopt url-directory-index-file
437 The filename to look for when indexing a directory, default
438 @samp{"index.html"}. If this file exists, and is readable, then it
439 will be viewed instead of using @code{dired} to view the directory.
446 @findex Info-goto-node
449 info:@var{file}#@var{node}
452 Info URLs are not officially defined. They invoke
453 @code{Info-goto-node} with argument @samp{(@var{file})@var{node}}.
454 @samp{#@var{node}} is optional, defaulting to @samp{Top}.
461 A mailto URL will send an email message to the address in the
462 URL, for example @samp{mailto:foo@@bar.com} would compose a
463 message to @samp{foo@@bar.com}.
465 @defopt url-mail-command
466 @vindex mail-user-agent
467 The function called whenever url needs to send mail. This should
468 normally be left to default from @var{mail-user-agent}. @xref{Mail
469 Methods, , Mail-Composition Methods, emacs, The GNU Emacs Manual}.
472 An @samp{X-Url-From} header field containing the URL of the document
473 that contained the mailto URL is added if that URL is known.
475 RFC 2368 extends the definition of mailto URLs in RFC 1738.
476 The form of a mailto URL is
478 @samp{mailto:@var{mailbox}[?@var{header}=@var{contents}[&@var{header}=@var{contents}]]}
480 @noindent where an arbitrary number of @var{header}s can be added. If the
481 @var{header} is @samp{body}, then @var{contents} is put in the body
482 otherwise a @var{header} header field is created with @var{contents}
483 as its contents. Note that the URL library does not consider any
484 headers `dangerous' so you should check them before sending the
488 Email messages are defined in @sc{rfc}822.
490 @node news/nntp/snews
491 @section @code{news}, @code{nntp} and @code{snews}
498 @c draft-gilman-news-url-01
499 The network news URL scheme take the following forms following RFC
500 1738 except that for compatibility with other clients, host and port
501 fields may be included in news URLs though they are properly only
502 allowed for nntp an snews.
505 @item news:@var{newsgroup}
506 Retrieves a list of messages in @var{newsgroup};
507 @item news:@var{message-id}
508 Retrieves the message with the given @var{message-id};
510 Retrieves a list of all available newsgroups;
511 @item nntp://@var{host}:@var{port}/@var{newsgroup}
512 @itemx nntp://@var{host}:@var{port}/@var{message-id}
513 @itemx nntp://@var{host}:@var{port}/*
514 Similar to the @samp{news} versions.
517 @samp{:@var{port}} is optional and defaults to :119.
519 @samp{snews} is the same as @samp{nntp} except that the default port
522 (It is tunneled through SSL.)
524 An @samp{nntp} URL is the same as a news URL, except that the URL may
525 specify an article by its number.
527 @defopt url-news-server
528 This variable can be used to override the default news server.
529 Usually this will be set by the Gnus package, which is used to fetch
531 @cindex environment variable
533 It may be set from the conventional environment variable
537 @node rlogin/telnet/tn3270
538 @section rlogin, telnet and tn3270
542 @cindex terminal emulation
543 @findex terminal-emulator
545 These URL schemes from RFC 1738 for logon via a terminal emulator have
548 telnet://@var{user}:@var{password}@@@var{host}:@var{port}
550 but the @code{:@var{password}} component is ignored.
552 To handle rlogin, telnet and tn3270 URLs, a @code{rlogin},
553 @code{telnet} or @code{tn3270} (the program names and arguments are
554 hardcoded) session is run in a @code{terminal-emulator} buffer.
555 Well-known ports are used if the URL does not specify a port.
560 @cindex Internet Relay Chat
564 @c Fixme: reference (was http://www.w3.org/Addressing/draft-mirashi-url-irc-01.txt)
565 @dfn{Internet Relay Chat} (IRC) is handled by handing off the @sc{irc}
566 session to a function named in @code{url-irc-function}.
568 @defopt url-irc-function
569 A function to actually open an IRC connection.
571 must take five arguments, @var{host}, @var{port}, @var{channel},
572 @var{user} and @var{password}. The @var{channel} argument specifies the
573 channel to join immediately, this can be @code{nil}. By default this is
574 @code{url-irc-rcirc}.
576 @defun url-irc-rcirc host port channel user password
577 Processes the arguments and lets @code{rcirc} handle the session.
579 @defun url-irc-erc host port channel user password
580 Processes the arguments and lets @code{ERC} handle the session.
582 @defun url-irc-zenirc host port channel user password
583 Processes the arguments and lets @code{zenirc} handle the session.
591 data:@r{[}@var{media-type}@r{]}@r{[};@var{base64}@r{]},@var{data}
594 Data URLs contain MIME data in the URL itself. They are defined in
597 @var{media-type} is a MIME @samp{Content-Type} string, possibly
598 including parameters. It defaults to
599 @samp{text/plain;charset=US-ASCII}. The @samp{text/plain} can be
600 omitted but the charset parameter supplied. If @samp{;base64} is
601 present, the @var{data} are base64-encoded.
606 @cindex Network File System
610 nfs://@var{user}:@var{password}@@@var{host}:@var{port}/@var{file}
613 The @samp{nfs:} scheme is defined in RFC 2224. It is similar to
614 @samp{ftp:} except that it points to a file on a remote host that is
615 handled by the automounter on the local host.
617 @defvar url-nfs-automounter-directory-spec
619 A string saying how to invoke the NFS automounter. Certain @samp{%}
620 sequences are recognized:
624 The hostname of the NFS server;
626 The port number of the NFS server;
628 The username to use to authenticate;
630 The password to use to authenticate;
632 The filename on the remote server;
637 Each can be used any number of times.
651 @cindex Lightweight Directory Access Protocol
653 The LDAP scheme is defined in RFC 2255.
663 @cindex @command{man}
664 @cindex Unix man pages
668 @samp{man:@var{page-spec}}
671 This is a non-standard scheme. @var{page-spec} is passed directly to
672 the Lisp @code{man} function.
674 @node Defining New URLs
675 @chapter Defining New URLs
678 * Naming conventions::
679 * Required functions::
680 * Optional functions::
681 * Asynchronous fetching::
682 * Supporting file-name-handlers::
685 @node Naming conventions
686 @section Naming conventions
688 @node Required functions
689 @section Required functions
691 @node Optional functions
692 @section Optional functions
694 @node Asynchronous fetching
695 @section Asynchronous fetching
697 @node Supporting file-name-handlers
698 @section Supporting file-name-handlers
700 @node General Facilities
701 @chapter General Facilities
706 * Gateways in general::
711 @section Disk Caching
713 @cindex Persistent Cache
716 The disk cache stores retrieved documents locally, whence they can be
717 retrieved more quickly. When requesting a URL that is in the cache,
718 the library checks to see if the page has changed since it was last
719 retrieved from the remote machine. If not, the local copy is used,
720 saving the transmission over the network.
721 @cindex Cleaning the cache
722 @cindex Clearing the cache
723 @cindex Cache cleaning
724 Currently the cache isn't cleared automatically.
725 @c Running the @code{clean-cache} shell script
726 @c fist is recommended, to allow for future cleaning of the cache. This
727 @c shell script will remove all files that have not been accessed since it
728 @c was last run. To keep the cache pared down, it is recommended that this
729 @c script be run from @i{at} or @i{cron} (see the manual pages for
730 @c crontab(5) or at(1) for more information)
732 @defopt url-automatic-caching
733 Setting this variable non-@code{nil} causes documents to be cached
737 @defopt url-cache-directory
738 This variable specifies the
739 directory to store the cache files. It defaults to sub-directory
740 @file{cache} of @code{url-configuration-directory}.
743 @c Fixme: function v. option, but neither used.
744 @c @findex url-cache-expired
745 @c @defopt url-cache-expired
746 @c This is a function to decide whether or not a cache entry has expired.
747 @c It takes two times as it parameters and returns non-@code{nil} if the
748 @c second time is ``too old'' when compared with the first time.
751 @defopt url-cache-creation-function
752 The cache relies on a scheme for mapping URLs to files in the cache.
753 This variable names a function which sets the type of cache to use.
754 It takes a URL as argument and returns the absolute file name of the
755 corresponding cache file. The two supplied possibilities are
756 @code{url-cache-create-filename-using-md5} and
757 @code{url-cache-create-filename-human-readable}.
760 @defun url-cache-create-filename-using-md5 url
761 Creates a cache file name from @var{url} using MD5 hashing.
763 This is creates entries with very few cache collisions and is fast if
764 you have the @code{md5} function as a primitive (Emacs 21 and XEmacs).
766 (url-cache-create-filename-using-md5 "http://www.example.com/foo/bar")
767 @result{} "/home/fx/.url/cache/fx/http/com/example/www/b8a35774ad20db71c7c3409a5410e74f"
771 @defun url-cache-create-filename-human-readable url
772 Creates a cache file name from @var{url} more obviously connected to
773 @var{url} than for @code{url-cache-create-filename-using-md5}, but
774 more likely to conflict with other files.
776 (url-cache-create-filename-human-readable "http://www.example.com/foo/bar")
777 @result{} "/home/fx/.url/cache/fx/http/com/example/www/foo/bar"
781 @c Fixme: never actually used currently?
782 @c @defopt url-standalone-mode
783 @c @cindex Relying on cache
784 @c @cindex Cache only mode
785 @c @cindex Standalone mode
786 @c If this variable is non-@code{nil}, the library relies solely on the
787 @c cache for fetching documents and avoids checking if they have changed
788 @c on remote servers.
791 @c With a large cache of documents on the local disk, it can be very handy
792 @c when traveling, or any other time the network connection is not active
793 @c (a laptop with a dial-on-demand PPP connection, etc). Emacs/W3 can rely
794 @c solely on its cache, and avoid checking to see if the page has changed
795 @c on the remote server. In the case of a dial-on-demand PPP connection,
796 @c this will keep the phone line free as long as possible, only bringing up
797 @c the PPP connection when asking for a page that is not located in the
798 @c cache. This is very useful for demonstrations as well.
801 @section Proxies and Gatewaying
803 @c fixme: check/document url-ns stuff
804 @cindex proxy servers
806 @cindex environment variables
808 Proxy servers are commonly used to provide gateways through firewalls
809 or as caches serving some more-or-less local network. Each protocol
810 (HTTP, FTP, etc.)@: can have a different gateway server. Proxying is
811 conventionally configured commonly amongst different programs through
812 environment variables of the form @code{@var{protocol}_proxy}, where
813 @var{protocol} is one of the supported network protocols (@code{http},
814 @code{ftp} etc.). The library recognizes such variables in either
815 upper or lower case. Their values are of one of the forms:
817 @item @code{@var{host}:@var{port}}
819 @item Simply a host name.
823 The @code{NO_PROXY} environment variable specifies URLs that should be
824 excluded from proxying (on servers that should be contacted directly).
825 This should be a comma-separated list of hostnames, domain names, or a
826 mixture of both. Asterisks can be used as wildcards, but other
827 clients may not support that. Domain names may be indicated by a
828 leading dot. For example:
830 NO_PROXY="*.aventail.com,home.com,.seanet.com"
832 @noindent says to contact all machines in the @samp{aventail.com} and
833 @samp{seanet.com} domains directly, as well as the machine named
834 @samp{home.com}. If @code{NO_PROXY} isn't defined, @code{no_PROXY}
835 and @code{no_proxy} are also tried, in that order.
837 Proxies may also be specified directly in Lisp.
839 @defopt url-proxy-services
840 This variable is an alist of URL schemes and proxy servers that
841 gateway them. The items are of the form @w{@code{(@var{scheme}
842 . @var{host}:@var{portnumber})}}, says that the URL @var{scheme} is
843 gatewayed through @var{portnumber} on the specified @var{host}. An
844 exception is the pseudo scheme @code{"no_proxy"}, which is paired with
845 a regexp matching host names not to be proxied. This variable is
846 initialized from the environment as above.
849 (setq url-proxy-services
850 '(("http" . "proxy.aventail.com:80")
851 ("no_proxy" . "^.*\\(aventail\\|seanet\\)\\.com")))
855 @node Gateways in general
856 @section Gateways in General
860 The library provides a general gateway layer through which all
861 networking passes. It can both control access to the network and
862 provide access through gateways in firewalls. This may make direct
863 connections in some cases and pass through some sort of gateway in
864 others.@footnote{Proxies (which only operate over HTTP) are
865 implemented using this.} The library's basic function responsible for
866 making connections is @code{url-open-stream}.
868 @defun url-open-stream name buffer host service
869 @cindex opening a stream
870 @cindex stream, opening
871 Open a stream to @var{host}, possibly via a gateway. The other
872 arguments are as for @code{open-network-stream}. This will not make a
873 connection if @code{url-gateway-unplugged} is non-@code{nil}.
876 @defvar url-gateway-local-host-regexp
877 This is a regular expression that matches local hosts that do not
878 require the use of a gateway. If @code{nil}, all connections are made
882 @defvar url-gateway-method
883 This variable controls which gateway method is used. It may be useful
884 to bind it temporarily in some applications. It has values taken from
885 a list of symbols. Possible values are:
889 @cindex @command{telnet}
890 Use this method if you must first telnet and log into a gateway host,
891 and then run telnet from that host to connect to outside machines.
894 @cindex @command{rlogin}
895 This method is identical to @code{telnet}, but uses @command{rlogin}
896 to log into the remote machine without having to send the username and
897 password over the wire every time.
901 Use if the firewall has a @sc{socks} gateway running on it. The
902 @sc{socks} v5 protocol is defined in RFC 1928.
905 @c This probably shouldn't be documented
906 @c Fixme: why not? -- fx
909 This method uses Emacs's builtin networking directly. This is the
910 default. It can be used only if there is no firewall blocking access.
914 The following variables control the gateway methods.
916 @defopt url-gateway-telnet-host
917 The gateway host to telnet to. Once logged in there, you then telnet
918 out to the hosts you want to connect to.
920 @defopt url-gateway-telnet-parameters
921 This should be a list of parameters to pass to the @command{telnet} program.
923 @defopt url-gateway-telnet-password-prompt
924 This is a regular expression that matches the password prompt when
927 @defopt url-gateway-telnet-login-prompt
928 This is a regular expression that matches the username prompt when
931 @defopt url-gateway-telnet-user-name
932 The username to log in with.
934 @defopt url-gateway-telnet-password
935 The password to send when logging in.
937 @defopt url-gateway-prompt-pattern
938 This is a regular expression that matches the shell prompt.
941 @defopt url-gateway-rlogin-host
942 Host to @samp{rlogin} to before telnetting out.
944 @defopt url-gateway-rlogin-parameters
945 Parameters to pass to @samp{rsh}.
947 @defopt url-gateway-rlogin-user-name
948 User name to use when logging in to the gateway.
950 @defopt url-gateway-prompt-pattern
951 This is a regular expression that matches the shell prompt.
955 This specifies the default server, it takes the form
956 @w{@code{("Default server" @var{server} @var{port} @var{version})}}
957 where @var{version} can be either 4 or 5.
959 @defvar socks-password
960 If this is @code{nil} then you will be asked for the password,
961 otherwise it will be used as the password for authenticating you to
962 the @sc{socks} server.
964 @defvar socks-username
965 This is the username to use when authenticating yourself to the
966 @sc{socks} server. By default this is your login name.
968 @defvar socks-timeout
969 This controls how long, in seconds, to wait for responses from the
970 @sc{socks} server; it is 5 by default.
972 @c fixme: these have been effectively commented-out in the code
973 @c @defopt socks-server-aliases
974 @c This a list of server aliases. It is a list of aliases of the form
975 @c @var{(alias hostname port version)}.
977 @c @defopt socks-network-aliases
978 @c This a list of network aliases. Each entry in the list takes the form
979 @c @var{(alias (network))} where @var{alias} is a string that names the
980 @c @var{network}. The networks can contain a pair (not a dotted pair) of
981 @c @sc{ip} addresses which specify a range of @sc{ip} addresses, an @sc{ip}
982 @c address and a netmask, a domain name or a unique hostname or @sc{ip}
985 @c @defopt socks-redirection-rules
986 @c This a list of redirection rules. Each rule take the form
987 @c @var{(Destination network Connection type)} where @var{Destination
988 @c network} is a network alias from @code{socks-network-aliases} and
989 @c @var{Connection type} can be @code{nil} in which case a direct
990 @c connection is used, or it can be an alias from
991 @c @code{socks-server-aliases} in which case that server is used as a
994 @defopt socks-nslookup-program
995 @cindex @command{nslookup}
996 This the @samp{nslookup} program. It is @code{"nslookup"} by default.
1000 * Suppressing network connections::
1002 @c * Broken hostname resolution::
1004 @node Suppressing network connections
1005 @subsection Suppressing Network Connections
1007 @cindex network connections, suppressing
1008 @cindex suppressing network connections
1011 In some circumstances it is desirable to suppress making network
1012 connections. A typical case is when rendering HTML in a mail user
1013 agent, when external URLs should not be activated, particularly to
1014 avoid `bugs' which `call home' by fetch single-pixel images and the
1015 like. To arrange this, bind the following variable for the duration
1018 @defvar url-gateway-unplugged
1019 If this variable is non-@code{nil} new network connections are never
1020 opened by the URL library.
1023 @c @node Broken hostname resolution
1024 @c @subsection Broken Hostname Resolution
1026 @c @cindex hostname resolver
1027 @c @cindex resolver, hostname
1028 @c Some C libraries do not include the hostname resolver routines in
1029 @c their static libraries. If Emacs was linked statically, and was not
1030 @c linked with the resolver libraries, it will not be able to get to any
1031 @c machines off the local network. This is characterized by being able
1032 @c to reach someplace with a raw ip number, but not its hostname
1033 @c (@url{http://129.79.254.191/} works, but
1034 @c @url{http://www.cs.indiana.edu/} doesn't). This used to happen on
1035 @c SunOS4 and Ultrix, but is now probably now rare. If Emacs can't be
1036 @c rebuilt linked against the resolver library, it can use the external
1037 @c @command{nslookup} program instead.
1039 @c @defopt url-gateway-broken-resolution
1040 @c @cindex @code{nslookup} program
1041 @c @cindex program, @code{nslookup}
1042 @c If non-@code{nil}, this variable says to use the program specified by
1043 @c @code{url-gateway-nslookup-program} program to do hostname resolution.
1046 @c @defopt url-gateway-nslookup-program
1047 @c The name of the program to do hostname lookup if Emacs can't do it
1048 @c directly. This program should expect a single argument on the command
1049 @c line---the hostname to resolve---and should produce output similar to
1050 @c the standard Unix @command{nslookup} program:
1052 @c Name: www.cs.indiana.edu
1053 @c Address: 129.79.254.191
1060 The library can maintain a global history list tracking URLs accessed.
1061 URL completion can be done from it. The history mechanism is set up
1062 @findex url-do-setup
1063 automatically via @code{url-do-setup} when it is configured to be on.
1064 Note that the size of the history list is currently not limited.
1066 @vindex url-history-hash-table
1067 The history `list' is actually a hash table,
1068 @code{url-history-hash-table}. It contains access times keyed by URL
1069 strings. The times are in the format returned by @code{current-time}.
1071 @defun url-history-update-url url time
1072 This function updates the history table with an entry for @var{url}
1073 accessed at the given @var{time}.
1076 @defopt url-history-track
1077 If non-@code{nil}, the library will keep track of all the URLs
1078 accessed. If it is @code{t}, the list is saved to disk at the end of
1079 each Emacs session. The default is @code{nil}.
1082 @defopt url-history-file
1083 The file storing the history list between sessions. It defaults to
1084 @file{history} in @code{url-configuration-directory}.
1087 @defopt url-history-save-interval
1088 @findex url-history-setup-save-timer
1089 The number of seconds between automatic saves of the history list.
1090 Default is one hour. Note that if you change this variable directly,
1091 rather than using Custom, after @code{url-do-setup} has been run, you
1092 need to run the function @code{url-history-setup-save-timer}.
1095 @defun url-history-parse-history &optional fname
1096 Parses the history file @var{fname} (default @code{url-history-file})
1097 and sets up the history list.
1100 @defun url-history-save-history &optional fname
1101 Saves the current history to file @var{fname} (default
1102 @code{url-history-file}).
1105 @defun url-completion-function string predicate function
1106 You can use this function to do completion of URLs from the history.
1110 @chapter Customization
1112 @section Environment Variables
1114 @cindex environment variables
1115 The following environment variables affect the library's operation at
1121 @vindex url-temporary-directory
1122 If this is defined, @var{url-temporary-directory} is initialized from
1126 @section General User Options
1128 The following user options, settable with Customize, affect the
1129 general operation of the package.
1133 Specifies the types of debug messages the library which are logged to
1134 the @code{*URL-DEBUG*} buffer.
1135 @code{t} means log all messages.
1136 A number means log all messages and show them with @code{message}.
1137 If may also be a list of the types of messages to be logged.
1139 @defopt url-personal-mail-address
1141 @defopt url-privacy-level
1143 @defopt url-uncompressor-alist
1145 @defopt url-passwd-entry-func
1147 @defopt url-standalone-mode
1149 @defopt url-bad-port-list
1151 @defopt url-max-password-attempts
1153 @defopt url-temporary-directory
1155 @defopt url-show-status
1157 @defopt url-confirmation-func
1158 The function to use for asking yes or no functions. This is normally
1159 either @code{y-or-n-p} or @code{yes-or-no-p}, but could be another
1160 function taking a single argument (the prompt) and returning @code{t}
1161 only if an affirmative answer is given.
1163 @defopt url-gateway-method
1164 @c fixme: describe gatewaying
1165 A symbol specifying the type of gateway support to use for connections
1166 from the local machine. The supported methods are:
1170 Run telnet in a subprocess to connect;
1172 Rlogin to another machine to connect;
1174 Connect through a socks server;
1182 @node Function Index
1183 @unnumbered Command and Function Index
1186 @node Variable Index
1187 @unnumbered Variable Index
1191 @unnumbered Concept Index
1194 @setchapternewpage odd
1199 arch-tag: c96be356-7e2d-4196-bcda-b13246c5c3f0