Merge commit 'origin/master'

[libwww-perl-eserte.git] / doc / examples.html

<title>libwww-perl-5</title>

<h1 align=center>LIBWWW-PERL-5</h1>

<h2>Introduction</h2>

<p>The libwww-perl-5 library is a collection of perl modules that
provide a simple and consistent programming interface to the
World-Wide Web. The library also contain modules that are of more
general use.

<p>This article gives an introduction to the library and ...

<p>The main focus of the library is to provide functions that allow
you to write WWW clients, thus libwww-perl is often presented as a WWW
client library.  The main features of the library are:

<ul>

  <li> Contains various reuseable components (modules) that can be
       used separately.

  <li> Provides an object oriented model of HTTP-style communication.
       Within this framework we currently support access to

          http,
          gopher,
          ftp,
          file, and
          mailto

       resources.

  <li> Support basic authorization

  <li> Transparent redirect handling

  <li> Supports proxy

  <li> URL handling (both absolute &amp; relative)

  <li> RobotRules (a parser for robots.txt files)

  <li> MailCap handling

  <li> HTML parser and formatter (PS and plain text)

  <li> The library be used through the full object oriented interface
       or through a very simple procedural interface.

  <li> A simple command line client application that is called
       <em>request</em>.

  <li> The library can cooperate with Tk.
       A simple Tk-based GUI browser is distributed with the Tk
       extention for perl.
</ul>




<h2>HTTP style communication</h2>

The libwww-perl library is based on HTTP style communication. What
does that mean? This is a quote from the <a
href="http://www.w3.org/pub/WWW/Protocols/">HTTP specification</a>
document:

<blockquote>
<p>The HTTP protocol is based on a request/response paradigm. A client
establishes a connection with a server and sends a request to the
server in the form of a request method, URI, and protocol version,
followed by a MIME-like message containing request modifiers, client
information, and possible body content. The server responds with a
status line, including the message's protocol version and a success or
error code, followed by a MIME-like message containing server
information, entity metainformation, and possible body content.
</blockquote>

<p>What this means to libwww-perl is that communcation always take
place by creating and configuring a <em>request</em> object. This
object is then passed to a server and we get a <em>response</em>
object in return that we can examine. The same simple model is used
for any kind of service we want to access.

<p>If we want to fetch a document from a remote file server we send it
a request that contains a name for that document and the response
contains the document itself. If we want to send a mail message to
somebody then we send the request object which contains our message to
the mail server and the response object will contain an acknowledgment
that tells us that the message has been accepted and will be forwarded
to the receipients.

<p>It is as simple as that!

<h3>Request object</h3>

The request object has the class name <em>HTTP::Request</em> in
libwww-perl. The fact that the class name use <em>HTTP::</em> as a
name prefix only implies that we use this model of communication. It
does not limit the kind of services we can try to send this
<em>request</em> to.

The main attributes of <em>HTTP::Request</em> objects are:

<ul>

  <li> The <b>method</b> is a short string that tells what kind of
       request this is.  The most usual methods are <em>GET</em>,
       <em>POST</em> and <em>HEAD</em>.

  <li> The <b>url</b> is a string denoting the protocol, server and
       the name of the "document" we want to access.  The url might
       also encode various other parameters. This is the name of the
       resource we want to access.

  <li> The <b>headers</b> contain additional information about the
       request and can also used to describe the content.  The headers
       is a set of keyword/value pairs.

  <li> The <b>content</b> is an arbitrary amount of binary data.

</ul>

  

<h3>Response object</h3>

The request object has the class name <em>HTTP::Response</em> in
libwww-perl.  The main attributes of objects of this class are:

<ul>
  <li> The <b>code</b> is a numerical value that encode the overall
       outcome of the request.

  <li> The <b>message</b> is a short (human readable) string that
       corresponds to the <em>code</em>.
       
  <li> The <b>headers</b> contain additional information about the
       response and they describe the content.

  <li> The <b>content</b> is an arbitrary amount of binary data.

</ul>

Since we don't want to handle the <em>code</em> directly in our
programs the libwww-perl response object have methods that can be used
to query the kind of code present:

<ul>

  <li> <b>isSuccess</b>
  <li> <b>isRedirect</b>
  <li> <b>isError</b>
  
</ul>


<h3>User Agent</h3>

Ok, I have created this nice <em>request</em> object. What do I do
with it?

<p>The answer is that you pass it on to the <em>user agent</em> object
and it will take care of all the things that need to be done
(low-level communcation and error handling) and the user agent will
give you back a <em>response</em> object. The user agent represents
your application on the network and it provides you with an interface
that can accept <em>requests</em> and will return <em>responses</em>.

<p><i>There should be a nice figure here explaining this. It should
show the UA as an interface layer between the application code and the
network.</i>

<p>The libwww-perl class name for the user agent is
<em>LWP::UserAgent</em>. Every libwww-perl application that wants to
communicate should create at least one object of this kind. The main
method provided by this object is <em>request()</em>. This method
takes an <em>HTTP::Request</em> object as argument and will return a
<em>HTTP::Response</em> object.

<p>The <em>LWP::UserAgent</em> has many other attributes that lets you
configure how it will interact with the network and with your
application code.

<ul>

  <li> The <b>timeout</b> specify how much time we give remote servers
       in creating responses before the library creates an internal
       <em>timeout</em> response.
  <li> The <b>agent</b> specify the name that your application should
       present itself as on the network.
  <li> The <b>useAlarm</b> specify if it is ok for the user agent to
       use the alarm(3) system to implement timeouts.
  <li> The <b>useEval</b> specify if the agent should raise an
       expection (<em>die</em> in perl) if an error condition occur.
       
  <li> The <b>proxy</b> and <b>noProxy</b> specify when communication
       should go through a <a
       href="http://www.w3.org/pub/WWW/Proxies/">proxy server</a>.
       
  <li> The <b>credentials</b> provide a way to set up usernames and
       passwords that is needed to access certain services.

</ul>

<p>Many applications would want even more control over how they
interact with the network and they get this by specializing the
<em>LWP::UserAgent</em> by sub-classing.

<!--  I don't want to describe these!!!
<ul>

  <li> simpleRequest()
  <li> redirectOK()
  <li> credentials()
  <li> getBasicCredentials()
  <li> mirror
  
</ul>
-->

<h1>Examples</h1>

Let's turn to a few examples to illustrate the concepts described
above. You should be able to run these examples directly given that
you have both perl and libwww-perl <a
href="install.html">installed</a> on your system. If you store the
examples in files you might want to change the first line (#!....) to
reflect the location of the perl interpreter on your system.

<a name="ex1"><h3>Example 1</h3></a>
<hr>
<pre>
#!/local/bin/perl -w

require LWP;

$ua = new LWP::UserAgent;

$request = new HTTP::Request 'GET', 'http://www.perl.com/perl/';
$request->header('Accept', 'text/html');

$response = $ua->request($request);

if ($response->isSuccess) {
   die "This is bad" if $response->header('Content-Type') ne 'text/html';
   print $response->content;
} else {
   die "Request failed: " . $response->code . " " . $response->message . "\n";
}

</pre>
<hr>

This example show a simple application that fetch an HTML document
with the name <a
href="http://www.perl.com/perl/">http://www.perl.com/perl/</a> from
the network and then prints it out (without reformatting). What is
going on is the following:

<ul>

  <li> First the statement <em>"require LWP;"</em> is needed to make the
       libwww-perl classes available to the application.

  <li> The next thing that happens is that we create an user agent
       object and assigns the reference to this object to the variable
       <em>$ua</em>.

  <li> Then we create a request object and assing it to the
       <em>$request</em> variable. The request object is initialized
       with the method <em>GET</em> and the URL <em>http://www....</em>.

  <li> The next thing that happens it that we configure the request by
       adding an <em>Accept</em> header to it. This header informs the
       server serving this request that we want an HTML document back.

  <li> Then we hand the request object over to the user agent and we
       receive a response object in return. We assign the response
       object to the $response variable.

  <li> Then we check the response to see that it really was a
       successful response and if it is we print the content (i.e. the
       document) that comes with the response.

  <li> If it was not a successful response we print an error message
       and die. <em>You might want to try to change the URL so that
       you get an unsuccessful response back!</em>
  
</ul>

Was this complicated for something as simple as retrieving a simple
file from a network server? Let's take a look at how we can make the
same thing much simpler.

<a name="ex2"><h3>Example 2</h3></a>

<hr>
<pre>
#!/local/bin/perl -w
use LWP::Simple;
getprint 'http://www.perl.com/perl/';
</pre>
<hr>

In this example we have used a module called <em>LWP::Simple</em>.
This two-line program essentially does the same as the code in <a
href="#ex1">example 1</a>. The <em>LWP::Simple</em> module provide a
very simplied procedural interface to the libwww-perl library. After
you have executed the <em>use LWP::Simple;</em> statement you have
access to the following routines:

<ul>
  <li> <em>get($url)</em><br> Takes an URL as an argument and returns the
       content.  Returns <em>undef</em> if an error occured.

  <li> <em>getprint($url)</em>
  <li> <em>head($url)</em>  --&gt;
       ($content_type,
       $document_size,
       $modified_time, $expires, $server)
  <li> <em>mirror($url, $file)</em>
       
  
</ul>

<p>The LWP::Simple module is also suitable for direct invocation from
the command line. The following command is equivalent with the script
above:

<pre>
   perl -MLWP::Simple -e "getprint 'http://www.perl.com/perl/'"
</pre>

<p>Let's write a more "complete" web browser...
<hr>
<pre>
#!/local/bin/perl -w
use LWP::Simple;
getprint shift || die "Usage: $0 &lt;url&gt;\n";
</pre>
<hr>


<a name="ex3"><h3>Example 3</h3></a>

Process data as it arrives from the network. Use a callback routine.

<h3>Example 4</h3>

Let's try to reformat the document using the HTML formatter.

<p>Let's write an even more "complete" web browser...
<hr>
<pre>
#!/local/bin/perl -w
use LWP::Simple;
use HTML::Parse;
print parse_html(get shift || die "Usage : $0 &lt;url&gt;\n")->format;
<pre>
</hr>

<p>Invoke a viewer for the document (MailCap)

<h3>Example 5</h3>

<ul>

  <li> A robot

  <li> Postscript output using the font metrics modules

  <li> Base64/Quoted printable

  <li> URL handling

  <li> HTTP headers
       
</ul>