Fix bug #11035 with cursor positioning on overlay strings with newlines.

[emacs.git] / doc / lispref / streams.texi

@c -*-texinfo-*-
@c This is part of the GNU Emacs Lisp Reference Manual.
@c Copyright (C) 1990-1994, 1998-1999, 2001-2012  Free Software Foundation, Inc.
@c See the file elisp.texi for copying conditions.
@setfilename ../../info/streams
@node Read and Print, Minibuffers, Debugging, Top
@comment  node-name,  next,  previous,  up
@chapter Reading and Printing Lisp Objects

  @dfn{Printing} and @dfn{reading} are the operations of converting Lisp
objects to textual form and vice versa.  They use the printed
representations and read syntax described in @ref{Lisp Data Types}.

  This chapter describes the Lisp functions for reading and printing.
It also describes @dfn{streams}, which specify where to get the text (if
reading) or where to put it (if printing).

@menu
* Streams Intro::     Overview of streams, reading and printing.
* Input Streams::     Various data types that can be used as input streams.
* Input Functions::   Functions to read Lisp objects from text.
* Output Streams::    Various data types that can be used as output streams.
* Output Functions::  Functions to print Lisp objects as text.
* Output Variables::  Variables that control what the printing functions do.
@end menu

@node Streams Intro
@section Introduction to Reading and Printing
@cindex Lisp reader
@cindex printing
@cindex reading

  @dfn{Reading} a Lisp object means parsing a Lisp expression in textual
form and producing a corresponding Lisp object.  This is how Lisp
programs get into Lisp from files of Lisp code.  We call the text the
@dfn{read syntax} of the object.  For example, the text @samp{(a .@: 5)}
is the read syntax for a cons cell whose @sc{car} is @code{a} and whose
@sc{cdr} is the number 5.

  @dfn{Printing} a Lisp object means producing text that represents that
object---converting the object to its @dfn{printed representation}
(@pxref{Printed Representation}).  Printing the cons cell described
above produces the text @samp{(a .@: 5)}.

  Reading and printing are more or less inverse operations: printing the
object that results from reading a given piece of text often produces
the same text, and reading the text that results from printing an object
usually produces a similar-looking object.  For example, printing the
symbol @code{foo} produces the text @samp{foo}, and reading that text
returns the symbol @code{foo}.  Printing a list whose elements are
@code{a} and @code{b} produces the text @samp{(a b)}, and reading that
text produces a list (but not the same list) with elements @code{a}
and @code{b}.

  However, these two operations are not precisely inverse to each other.
There are three kinds of exceptions:

@itemize @bullet
@item
Printing can produce text that cannot be read.  For example, buffers,
windows, frames, subprocesses and markers print as text that starts
with @samp{#}; if you try to read this text, you get an error.  There is
no way to read those data types.

@item
One object can have multiple textual representations.  For example,
@samp{1} and @samp{01} represent the same integer, and @samp{(a b)} and
@samp{(a .@: (b))} represent the same list.  Reading will accept any of
the alternatives, but printing must choose one of them.

@item
Comments can appear at certain points in the middle of an object's
read sequence without affecting the result of reading it.
@end itemize

@node Input Streams
@section Input Streams
@cindex stream (for reading)
@cindex input stream

  Most of the Lisp functions for reading text take an @dfn{input stream}
as an argument.  The input stream specifies where or how to get the
characters of the text to be read.  Here are the possible types of input
stream:

@table @asis
@item @var{buffer}
@cindex buffer input stream
The input characters are read from @var{buffer}, starting with the
character directly after point.  Point advances as characters are read.

@item @var{marker}
@cindex marker input stream
The input characters are read from the buffer that @var{marker} is in,
starting with the character directly after the marker.  The marker
position advances as characters are read.  The value of point in the
buffer has no effect when the stream is a marker.

@item @var{string}
@cindex string input stream
The input characters are taken from @var{string}, starting at the first
character in the string and using as many characters as required.

@item @var{function}
@cindex function input stream
The input characters are generated by @var{function}, which must support
two kinds of calls:

@itemize @bullet
@item
When it is called with no arguments, it should return the next character.

@item
When it is called with one argument (always a character), @var{function}
should save the argument and arrange to return it on the next call.
This is called @dfn{unreading} the character; it happens when the Lisp
reader reads one character too many and wants to ``put it back where it
came from.''  In this case, it makes no difference what value
@var{function} returns.
@end itemize

@item @code{t}
@cindex @code{t} input stream
@code{t} used as a stream means that the input is read from the
minibuffer.  In fact, the minibuffer is invoked once and the text
given by the user is made into a string that is then used as the
input stream.  If Emacs is running in batch mode, standard input is used
instead of the minibuffer.  For example,
@example
(message "%s" (read t))
@end example
will read a Lisp expression from standard input and print the result
to standard output.

@item @code{nil}
@cindex @code{nil} input stream
@code{nil} supplied as an input stream means to use the value of
@code{standard-input} instead; that value is the @dfn{default input
stream}, and must be a non-@code{nil} input stream.

@item @var{symbol}
A symbol as input stream is equivalent to the symbol's function
definition (if any).
@end table

  Here is an example of reading from a stream that is a buffer, showing
where point is located before and after:

@example
@group
---------- Buffer: foo ----------
This@point{} is the contents of foo.
---------- Buffer: foo ----------
@end group

@group
(read (get-buffer "foo"))
     @result{} is
@end group
@group
(read (get-buffer "foo"))
     @result{} the
@end group

@group
---------- Buffer: foo ----------
This is the@point{} contents of foo.
---------- Buffer: foo ----------
@end group
@end example

@noindent
Note that the first read skips a space.  Reading skips any amount of
whitespace preceding the significant text.

  Here is an example of reading from a stream that is a marker,
initially positioned at the beginning of the buffer shown.  The value
read is the symbol @code{This}.

@example
@group

---------- Buffer: foo ----------
This is the contents of foo.
---------- Buffer: foo ----------
@end group

@group
(setq m (set-marker (make-marker) 1 (get-buffer "foo")))
     @result{} #<marker at 1 in foo>
@end group
@group
(read m)
     @result{} This
@end group
@group
m
     @result{} #<marker at 5 in foo>   ;; @r{Before the first space.}
@end group
@end example

  Here we read from the contents of a string:

@example
@group
(read "(When in) the course")
     @result{} (When in)
@end group
@end example

  The following example reads from the minibuffer.  The
prompt is: @w{@samp{Lisp expression: }}.  (That is always the prompt
used when you read from the stream @code{t}.)  The user's input is shown
following the prompt.

@example
@group
(read t)
     @result{} 23
---------- Buffer: Minibuffer ----------
Lisp expression: @kbd{23 @key{RET}}
---------- Buffer: Minibuffer ----------
@end group
@end example

  Finally, here is an example of a stream that is a function, named
@code{useless-stream}.  Before we use the stream, we initialize the
variable @code{useless-list} to a list of characters.  Then each call to
the function @code{useless-stream} obtains the next character in the list
or unreads a character by adding it to the front of the list.

@example
@group
(setq useless-list (append "XY()" nil))
     @result{} (88 89 40 41)
@end group

@group
(defun useless-stream (&optional unread)
  (if unread
      (setq useless-list (cons unread useless-list))
    (prog1 (car useless-list)
           (setq useless-list (cdr useless-list)))))
     @result{} useless-stream
@end group
@end example

@noindent
Now we read using the stream thus constructed:

@example
@group
(read 'useless-stream)
     @result{} XY
@end group

@group
useless-list
     @result{} (40 41)
@end group
@end example

@noindent
Note that the open and close parentheses remain in the list.  The Lisp
reader encountered the open parenthesis, decided that it ended the
input, and unread it.  Another attempt to read from the stream at this
point would read @samp{()} and return @code{nil}.

@node Input Functions
@section Input Functions

  This section describes the Lisp functions and variables that pertain
to reading.

  In the functions below, @var{stream} stands for an input stream (see
the previous section).  If @var{stream} is @code{nil} or omitted, it
defaults to the value of @code{standard-input}.

@kindex end-of-file
  An @code{end-of-file} error is signaled if reading encounters an
unterminated list, vector, or string.

@defun read &optional stream
This function reads one textual Lisp expression from @var{stream},
returning it as a Lisp object.  This is the basic Lisp input function.
@end defun

@defun read-from-string string &optional start end
@cindex string to object
This function reads the first textual Lisp expression from the text in
@var{string}.  It returns a cons cell whose @sc{car} is that expression,
and whose @sc{cdr} is an integer giving the position of the next
remaining character in the string (i.e., the first one not read).

If @var{start} is supplied, then reading begins at index @var{start} in
the string (where the first character is at index 0).  If you specify
@var{end}, then reading is forced to stop just before that index, as if
the rest of the string were not there.

For example:

@example
@group
(read-from-string "(setq x 55) (setq y 5)")
     @result{} ((setq x 55) . 11)
@end group
@group
(read-from-string "\"A short string\"")
     @result{} ("A short string" . 16)
@end group

@group
;; @r{Read starting at the first character.}
(read-from-string "(list 112)" 0)
     @result{} ((list 112) . 10)
@end group
@group
;; @r{Read starting at the second character.}
(read-from-string "(list 112)" 1)
     @result{} (list . 5)
@end group
@group
;; @r{Read starting at the seventh character,}
;;   @r{and stopping at the ninth.}
(read-from-string "(list 112)" 6 8)
     @result{} (11 . 8)
@end group
@end example
@end defun

@defvar standard-input
This variable holds the default input stream---the stream that
@code{read} uses when the @var{stream} argument is @code{nil}.
The default is @code{t}, meaning use the minibuffer.
@end defvar

@defvar read-circle
If non-@code{nil}, this variable enables the reading of circular and
shared structures.  @xref{Circular Objects}.  Its default value is
@code{t}.
@end defvar

@node Output Streams
@section Output Streams
@cindex stream (for printing)
@cindex output stream

  An output stream specifies what to do with the characters produced
by printing.  Most print functions accept an output stream as an
optional argument.  Here are the possible types of output stream:

@table @asis
@item @var{buffer}
@cindex buffer output stream
The output characters are inserted into @var{buffer} at point.
Point advances as characters are inserted.

@item @var{marker}
@cindex marker output stream
The output characters are inserted into the buffer that @var{marker}
points into, at the marker position.  The marker position advances as
characters are inserted.  The value of point in the buffer has no effect
on printing when the stream is a marker, and this kind of printing
does not move point (except that if the marker points at or before the
position of point, point advances with the surrounding text, as
usual).

@item @var{function}
@cindex function output stream
The output characters are passed to @var{function}, which is responsible
for storing them away.  It is called with a single character as
argument, as many times as there are characters to be output, and
is responsible for storing the characters wherever you want to put them.

@item @code{t}
@cindex @code{t} output stream
The output characters are displayed in the echo area.

@item @code{nil}
@cindex @code{nil} output stream
@code{nil} specified as an output stream means to use the value of
@code{standard-output} instead; that value is the @dfn{default output
stream}, and must not be @code{nil}.

@item @var{symbol}
A symbol as output stream is equivalent to the symbol's function
definition (if any).
@end table

  Many of the valid output streams are also valid as input streams.  The
difference between input and output streams is therefore more a matter
of how you use a Lisp object, than of different types of object.

  Here is an example of a buffer used as an output stream.  Point is
initially located as shown immediately before the @samp{h} in
@samp{the}.  At the end, point is located directly before that same
@samp{h}.

@cindex print example
@example
@group
---------- Buffer: foo ----------
This is t@point{}he contents of foo.
---------- Buffer: foo ----------
@end group

(print "This is the output" (get-buffer "foo"))
     @result{} "This is the output"

@group
---------- Buffer: foo ----------
This is t
"This is the output"
@point{}he contents of foo.
---------- Buffer: foo ----------
@end group
@end example

  Now we show a use of a marker as an output stream.  Initially, the
marker is in buffer @code{foo}, between the @samp{t} and the @samp{h} in
the word @samp{the}.  At the end, the marker has advanced over the
inserted text so that it remains positioned before the same @samp{h}.
Note that the location of point, shown in the usual fashion, has no
effect.

@example
@group
---------- Buffer: foo ----------
This is the @point{}output
---------- Buffer: foo ----------
@end group

@group
(setq m (copy-marker 10))
     @result{} #<marker at 10 in foo>
@end group

@group
(print "More output for foo." m)
     @result{} "More output for foo."
@end group

@group
---------- Buffer: foo ----------
This is t
"More output for foo."
he @point{}output
---------- Buffer: foo ----------
@end group

@group
m
     @result{} #<marker at 34 in foo>
@end group
@end example

  The following example shows output to the echo area:

@example
@group
(print "Echo Area output" t)
     @result{} "Echo Area output"
---------- Echo Area ----------
"Echo Area output"
---------- Echo Area ----------
@end group
@end example

  Finally, we show the use of a function as an output stream.  The
function @code{eat-output} takes each character that it is given and
conses it onto the front of the list @code{last-output} (@pxref{Building
Lists}).  At the end, the list contains all the characters output, but
in reverse order.

@example
@group
(setq last-output nil)
     @result{} nil
@end group

@group
(defun eat-output (c)
  (setq last-output (cons c last-output)))
     @result{} eat-output
@end group

@group
(print "This is the output" 'eat-output)
     @result{} "This is the output"
@end group

@group
last-output
     @result{} (10 34 116 117 112 116 117 111 32 101 104
    116 32 115 105 32 115 105 104 84 34 10)
@end group
@end example

@noindent
Now we can put the output in the proper order by reversing the list:

@example
@group
(concat (nreverse last-output))
     @result{} "
\"This is the output\"
"
@end group
@end example

@noindent
Calling @code{concat} converts the list to a string so you can see its
contents more clearly.

@node Output Functions
@section Output Functions

  This section describes the Lisp functions for printing Lisp
objects---converting objects into their printed representation.

@cindex @samp{"} in printing
@cindex @samp{\} in printing
@cindex quoting characters in printing
@cindex escape characters in printing
  Some of the Emacs printing functions add quoting characters to the
output when necessary so that it can be read properly.  The quoting
characters used are @samp{"} and @samp{\}; they distinguish strings from
symbols, and prevent punctuation characters in strings and symbols from
being taken as delimiters when reading.  @xref{Printed Representation},
for full details.  You specify quoting or no quoting by the choice of
printing function.

  If the text is to be read back into Lisp, then you should print with
quoting characters to avoid ambiguity.  Likewise, if the purpose is to
describe a Lisp object clearly for a Lisp programmer.  However, if the
purpose of the output is to look nice for humans, then it is usually
better to print without quoting.

  Lisp objects can refer to themselves.  Printing a self-referential
object in the normal way would require an infinite amount of text, and
the attempt could cause infinite recursion.  Emacs detects such
recursion and prints @samp{#@var{level}} instead of recursively printing
an object already being printed.  For example, here @samp{#0} indicates
a recursive reference to the object at level 0 of the current print
operation:

@example
(setq foo (list nil))
     @result{} (nil)
(setcar foo foo)
     @result{} (#0)
@end example

  In the functions below, @var{stream} stands for an output stream.
(See the previous section for a description of output streams.)  If
@var{stream} is @code{nil} or omitted, it defaults to the value of
@code{standard-output}.

@defun print object &optional stream
@cindex Lisp printer
The @code{print} function is a convenient way of printing.  It outputs
the printed representation of @var{object} to @var{stream}, printing in
addition one newline before @var{object} and another after it.  Quoting
characters are used.  @code{print} returns @var{object}.  For example:

@example
@group
(progn (print 'The\ cat\ in)
       (print "the hat")
       (print " came back"))
     @print{}
     @print{} The\ cat\ in
     @print{}
     @print{} "the hat"
     @print{}
     @print{} " came back"
     @result{} " came back"
@end group
@end example
@end defun

@defun prin1 object &optional stream
This function outputs the printed representation of @var{object} to
@var{stream}.  It does not print newlines to separate output as
@code{print} does, but it does use quoting characters just like
@code{print}.  It returns @var{object}.

@example
@group
(progn (prin1 'The\ cat\ in)
       (prin1 "the hat")
       (prin1 " came back"))
     @print{} The\ cat\ in"the hat"" came back"
     @result{} " came back"
@end group
@end example
@end defun

@defun princ object &optional stream
This function outputs the printed representation of @var{object} to
@var{stream}.  It returns @var{object}.

This function is intended to produce output that is readable by people,
not by @code{read}, so it doesn't insert quoting characters and doesn't
put double-quotes around the contents of strings.  It does not add any
spacing between calls.

@example
@group
(progn
  (princ 'The\ cat)
  (princ " in the \"hat\""))
     @print{} The cat in the "hat"
     @result{} " in the \"hat\""
@end group
@end example
@end defun

@defun terpri &optional stream
@cindex newline in print
This function outputs a newline to @var{stream}.  The name stands
for ``terminate print.''
@end defun

@defun write-char character &optional stream
This function outputs @var{character} to @var{stream}.  It returns
@var{character}.
@end defun

@defun prin1-to-string object &optional noescape
@cindex object to string
This function returns a string containing the text that @code{prin1}
would have printed for the same argument.

@example
@group
(prin1-to-string 'foo)
     @result{} "foo"
@end group
@group
(prin1-to-string (mark-marker))
     @result{} "#<marker at 2773 in strings.texi>"
@end group
@end example

If @var{noescape} is non-@code{nil}, that inhibits use of quoting
characters in the output.  (This argument is supported in Emacs versions
19 and later.)

@example
@group
(prin1-to-string "foo")
     @result{} "\"foo\""
@end group
@group
(prin1-to-string "foo" t)
     @result{} "foo"
@end group
@end example

See @code{format}, in @ref{Formatting Strings}, for other ways to obtain
the printed representation of a Lisp object as a string.
@end defun

@defmac with-output-to-string body@dots{}
This macro executes the @var{body} forms with @code{standard-output} set
up to feed output into a string.  Then it returns that string.

For example, if the current buffer name is @samp{foo},

@example
(with-output-to-string
  (princ "The buffer is ")
  (princ (buffer-name)))
@end example

@noindent
returns @code{"The buffer is foo"}.
@end defmac

@defun pp object &optional stream
This function outputs @var{object} to @var{stream}, just like
@code{prin1}, but does it in a more ``pretty'' way.  That is, it'll
indent and fill the object to make it more readable for humans.
@end defun

@node Output Variables
@section Variables Affecting Output
@cindex output-controlling variables

@defvar standard-output
The value of this variable is the default output stream---the stream
that print functions use when the @var{stream} argument is @code{nil}.
The default is @code{t}, meaning display in the echo area.
@end defvar

@defvar print-quoted
If this is non-@code{nil}, that means to print quoted forms using
abbreviated reader syntax, e.g.@: @code{(quote foo)} prints as
@code{'foo}, and @code{(function foo)} as @code{#'foo}.
@end defvar

@defvar print-escape-newlines
@cindex @samp{\n} in print
@cindex escape characters
If this variable is non-@code{nil}, then newline characters in strings
are printed as @samp{\n} and formfeeds are printed as @samp{\f}.
Normally these characters are printed as actual newlines and formfeeds.

This variable affects the print functions @code{prin1} and @code{print}
that print with quoting.  It does not affect @code{princ}.  Here is an
example using @code{prin1}:

@example
@group
(prin1 "a\nb")
     @print{} "a
     @print{} b"
     @result{} "a
b"
@end group

@group
(let ((print-escape-newlines t))
  (prin1 "a\nb"))
     @print{} "a\nb"
     @result{} "a
b"
@end group
@end example

@noindent
In the second expression, the local binding of
@code{print-escape-newlines} is in effect during the call to
@code{prin1}, but not during the printing of the result.
@end defvar

@defvar print-escape-nonascii
If this variable is non-@code{nil}, then unibyte non-@acronym{ASCII}
characters in strings are unconditionally printed as backslash sequences
by the print functions @code{prin1} and @code{print} that print with
quoting.

Those functions also use backslash sequences for unibyte non-@acronym{ASCII}
characters, regardless of the value of this variable, when the output
stream is a multibyte buffer or a marker pointing into one.
@end defvar

@defvar print-escape-multibyte
If this variable is non-@code{nil}, then multibyte non-@acronym{ASCII}
characters in strings are unconditionally printed as backslash sequences
by the print functions @code{prin1} and @code{print} that print with
quoting.

Those functions also use backslash sequences for multibyte
non-@acronym{ASCII} characters, regardless of the value of this variable,
when the output stream is a unibyte buffer or a marker pointing into
one.
@end defvar

@defvar print-length
@cindex printing limits
The value of this variable is the maximum number of elements to print in
any list, vector or bool-vector.  If an object being printed has more
than this many elements, it is abbreviated with an ellipsis.

If the value is @code{nil} (the default), then there is no limit.

@example
@group
(setq print-length 2)
     @result{} 2
@end group
@group
(print '(1 2 3 4 5))
     @print{} (1 2 ...)
     @result{} (1 2 ...)
@end group
@end example
@end defvar

@defvar print-level
The value of this variable is the maximum depth of nesting of
parentheses and brackets when printed.  Any list or vector at a depth
exceeding this limit is abbreviated with an ellipsis.  A value of
@code{nil} (which is the default) means no limit.
@end defvar

@defopt eval-expression-print-length
@defoptx eval-expression-print-level
These are the values for @code{print-length} and @code{print-level}
used by @code{eval-expression}, and thus, indirectly, by many
interactive evaluation commands (@pxref{Lisp Eval,, Evaluating
Emacs-Lisp Expressions, emacs, The GNU Emacs Manual}).
@end defopt

  These variables are used for detecting and reporting circular
and shared structure:

@defvar print-circle
If non-@code{nil}, this variable enables detection of circular and
shared structure in printing.  @xref{Circular Objects}.
@end defvar

@defvar print-gensym
If non-@code{nil}, this variable enables detection of uninterned symbols
(@pxref{Creating Symbols}) in printing.  When this is enabled,
uninterned symbols print with the prefix @samp{#:}, which tells the Lisp
reader to produce an uninterned symbol.
@end defvar

@defvar print-continuous-numbering
If non-@code{nil}, that means number continuously across print calls.
This affects the numbers printed for @samp{#@var{n}=} labels and
@samp{#@var{m}#} references.

Don't set this variable with @code{setq}; you should only bind it
temporarily to @code{t} with @code{let}.  When you do that, you should
also bind @code{print-number-table} to @code{nil}.
@end defvar

@defvar print-number-table
This variable holds a vector used internally by printing to implement
the @code{print-circle} feature.  You should not use it except
to bind it to @code{nil} when you bind @code{print-continuous-numbering}.
@end defvar

@defvar float-output-format
This variable specifies how to print floating point numbers.  Its
default value is @code{nil}, meaning use the shortest output
that represents the number without losing information.

To control output format more precisely, you can put a string in this
variable.  The string should hold a @samp{%}-specification to be used
in the C function @code{sprintf}.  For further restrictions on what
you can use, see the variable's documentation string.
@end defvar