1 :mod:`io` --- Core tools for working with streams
2 =================================================
5 :synopsis: Core tools for working with streams.
6 .. moduleauthor:: Guido van Rossum <guido@python.org>
7 .. moduleauthor:: Mike Verdone <mike.verdone@gmail.com>
8 .. moduleauthor:: Mark Russell <mark.russell@zen.co.uk>
9 .. sectionauthor:: Benjamin Peterson
12 The :mod:`io` module provides the Python interfaces to stream handling. The
13 builtin :func:`open` function is defined in this module.
15 At the top of the I/O hierarchy is the abstract base class :class:`IOBase`. It
16 defines the basic interface to a stream. Note, however, that there is no
17 seperation between reading and writing to streams; implementations are allowed
18 to throw an :exc:`IOError` if they do not support a given operation.
20 Extending :class:`IOBase` is :class:`RawIOBase` which deals simply with the
21 reading and writing of raw bytes to a stream. :class:`FileIO` subclasses
22 :class:`RawIOBase` to provide an interface to files in the machine's
25 :class:`BufferedIOBase` deals with buffering on a raw byte stream
26 (:class:`RawIOBase`). Its subclasses, :class:`BufferedWriter`,
27 :class:`BufferedReader`, and :class:`BufferedRWPair` buffer streams that are
28 readable, writable, and both readable and writable.
29 :class:`BufferedRandom` provides a buffered interface to random access
30 streams. :class:`BytesIO` is a simple stream of in-memory bytes.
32 Another :class:`IOBase` subclass, :class:`TextIOBase`, deals with
33 streams whose bytes represent text, and handles encoding and decoding
34 from and to strings. :class:`TextIOWrapper`, which extends it, is a
35 buffered text interface to a buffered raw stream
36 (:class:`BufferedIOBase`). Finally, :class:`StringIO` is an in-memory
39 Argument names are not part of the specification, and only the arguments of
40 :func:`open` are intended to be used as keyword arguments.
46 .. data:: DEFAULT_BUFFER_SIZE
48 An int containing the default buffer size used by the module's buffered I/O
49 classes. :func:`open` uses the file's blksize (as obtained by
50 :func:`os.stat`) if possible.
52 .. function:: open(file[, mode[, buffering[, encoding[, errors[, newline[, closefd=True]]]]]])
54 Open *file* and return a stream. If the file cannot be opened, an
55 :exc:`IOError` is raised.
57 *file* is either a string giving the name (and the path if the file isn't in
58 the current working directory) of the file to be opened or a file
59 descriptor of the file to be opened. (If a file descriptor is given,
60 for example, from :func:`os.fdopen`, it is closed when the returned
61 I/O object is closed, unless *closefd* is set to ``False``.)
63 *mode* is an optional string that specifies the mode in which the file is
64 opened. It defaults to ``'r'`` which means open for reading in text mode.
65 Other common values are ``'w'`` for writing (truncating the file if it
66 already exists), and ``'a'`` for appending (which on *some* Unix systems,
67 means that *all* writes append to the end of the file regardless of the
68 current seek position). In text mode, if *encoding* is not specified the
69 encoding used is platform dependent. (For reading and writing raw bytes use
70 binary mode and leave *encoding* unspecified.) The available modes are:
72 ========= ===============================================================
74 --------- ---------------------------------------------------------------
75 ``'r'`` open for reading (default)
76 ``'w'`` open for writing, truncating the file first
77 ``'a'`` open for writing, appending to the end of the file if it exists
79 ``'t'`` text mode (default)
80 ``'+'`` open a disk file for updating (reading and writing)
81 ``'U'`` universal newline mode (for backwards compatibility; should
82 not be used in new code)
83 ========= ===============================================================
85 The default mode is ``'rt'`` (open for reading text). For binary random
86 access, the mode ``'w+b'`` opens and truncates the file to 0 bytes, while
87 ``'r+b'`` opens the file without truncation.
89 Python distinguishes between files opened in binary and text modes, even when
90 the underlying operating system doesn't. Files opened in binary mode
91 (including ``'b'`` in the *mode* argument) return contents as ``bytes``
92 objects without any decoding. In text mode (the default, or when ``'t'`` is
93 included in the *mode* argument), the contents of the file are returned as
94 strings, the bytes having been first decoded using a platform-dependent
95 encoding or using the specified *encoding* if given.
97 *buffering* is an optional integer used to set the buffering policy. By
98 default full buffering is on. Pass 0 to switch buffering off (only allowed
99 in binary mode), 1 to set line buffering, and an integer > 1 for full
102 *encoding* is the name of the encoding used to decode or encode the file.
103 This should only be used in text mode. The default encoding is platform
104 dependent, but any encoding supported by Python can be used. See the
105 :mod:`codecs` module for the list of supported encodings.
107 *errors* is an optional string that specifies how encoding and decoding
108 errors are to be handled. Pass ``'strict'`` to raise a :exc:`ValueError`
109 exception if there is an encoding error (the default of ``None`` has the same
110 effect), or pass ``'ignore'`` to ignore errors. (Note that ignoring encoding
111 errors can lead to data loss.) ``'replace'`` causes a replacement marker
112 (such as ``'?'``) to be inserted where there is malformed data. When
113 writing, ``'xmlcharrefreplace'`` (replace with the appropriate XML character
114 reference) or ``'backslashreplace'`` (replace with backslashed escape
115 sequences) can be used. Any other error handling name that has been
116 registered with :func:`codecs.register_error` is also valid.
118 *newline* controls how universal newlines works (it only applies to text
119 mode). It can be ``None``, ``''``, ``'\n'``, ``'\r'``, and ``'\r\n'``. It
122 * On input, if *newline* is ``None``, universal newlines mode is enabled.
123 Lines in the input can end in ``'\n'``, ``'\r'``, or ``'\r\n'``, and these
124 are translated into ``'\n'`` before being returned to the caller. If it is
125 ``''``, universal newline mode is enabled, but line endings are returned to
126 the caller untranslated. If it has any of the other legal values, input
127 lines are only terminated by the given string, and the line ending is
128 returned to the caller untranslated.
130 * On output, if *newline* is ``None``, any ``'\n'`` characters written are
131 translated to the system default line separator, :data:`os.linesep`. If
132 *newline* is ``''``, no translation takes place. If *newline* is any of
133 the other legal values, any ``'\n'`` characters written are translated to
136 If *closefd* is ``False`` and a file descriptor rather than a
137 filename was given, the underlying file descriptor will be kept open
138 when the file is closed. If a filename is given *closefd* has no
139 effect but must be ``True`` (the default).
141 The type of file object returned by the :func:`open` function depends
142 on the mode. When :func:`open` is used to open a file in a text mode
143 (``'w'``, ``'r'``, ``'wt'``, ``'rt'``, etc.), it returns a
144 :class:`TextIOWrapper`. When used to open a file in a binary mode,
145 the returned class varies: in read binary mode, it returns a
146 :class:`BufferedReader`; in write binary and append binary modes, it
147 returns a :class:`BufferedWriter`, and in read/write mode, it returns
148 a :class:`BufferedRandom`.
150 It is also possible to use a string or bytearray as a file for both reading
151 and writing. For strings :class:`StringIO` can be used like a file opened in
152 a text mode, and for bytearrays a :class:`BytesIO` can be used like a
153 file opened in a binary mode.
156 .. exception:: BlockingIOError
158 Error raised when blocking would occur on a non-blocking stream. It inherits
161 In addition to those of :exc:`IOError`, :exc:`BlockingIOError` has one
164 .. attribute:: characters_written
166 An integer containing the number of characters written to the stream
170 .. exception:: UnsupportedOperation
172 An exception inheriting :exc:`IOError` and :exc:`ValueError` that is raised
173 when an unsupported operation is called on a stream.
181 The abstract base class for all I/O classes, acting on streams of bytes.
182 There is no public constructor.
184 This class provides empty abstract implementations for many methods
185 that derived classes can override selectively; the default
186 implementations represent a file that cannot be read, written or
189 Even though :class:`IOBase` does not declare :meth:`read`, :meth:`readinto`,
190 or :meth:`write` because their signatures will vary, implementations and
191 clients should consider those methods part of the interface. Also,
192 implementations may raise a :exc:`IOError` when operations they do not
195 The basic type used for binary data read from or written to a file is
196 :class:`bytes`. :class:`bytearray`\s are accepted too, and in some cases
197 (such as :class:`readinto`) required. Text I/O classes work with
200 Note that calling any method (even inquiries) on a closed stream is
201 undefined. Implementations may raise :exc:`IOError` in this case.
203 IOBase (and its subclasses) support the iterator protocol, meaning that an
204 :class:`IOBase` object can be iterated over yielding the lines in a stream.
206 IOBase is also a context manager and therefore supports the
207 :keyword:`with` statement. In this example, *file* is closed after the
208 :keyword:`with` statement's suite is finished---even if an exception occurs::
210 with open('spam.txt', 'w') as file:
211 file.write('Spam and eggs!')
213 :class:`IOBase` provides these data attributes and methods:
217 Flush and close this stream. This method has no effect if the file is
220 .. attribute:: closed
222 True if the stream is closed.
226 Return the underlying file descriptor (an integer) of the stream if it
227 exists. An :exc:`IOError` is raised if the IO object does not use a file
232 Flush the write buffers of the stream if applicable. This does nothing
233 for read-only and non-blocking streams.
237 Return ``True`` if the stream is interactive (i.e., connected to
238 a terminal/tty device).
240 .. method:: readable()
242 Return ``True`` if the stream can be read from. If False, :meth:`read`
243 will raise :exc:`IOError`.
245 .. method:: readline([limit])
247 Read and return one line from the stream. If *limit* is specified, at
248 most *limit* bytes will be read.
250 The line terminator is always ``b'\n'`` for binary files; for text files,
251 the *newlines* argument to :func:`open` can be used to select the line
252 terminator(s) recognized.
254 .. method:: readlines([hint])
256 Read and return a list of lines from the stream. *hint* can be specified
257 to control the number of lines read: no more lines will be read if the
258 total size (in bytes/characters) of all lines so far exceeds *hint*.
260 .. method:: seek(offset[, whence])
262 Change the stream position to the given byte *offset*. *offset* is
263 interpreted relative to the position indicated by *whence*. Values for
266 * ``0`` -- start of the stream (the default); *offset* should be zero or positive
267 * ``1`` -- current stream position; *offset* may be negative
268 * ``2`` -- end of the stream; *offset* is usually negative
270 Return the new absolute position.
272 .. method:: seekable()
274 Return ``True`` if the stream supports random access. If ``False``,
275 :meth:`seek`, :meth:`tell` and :meth:`truncate` will raise :exc:`IOError`.
279 Return the current stream position.
281 .. method:: truncate([size])
283 Truncate the file to at most *size* bytes. *size* defaults to the current
284 file position, as returned by :meth:`tell`.
286 .. method:: writable()
288 Return ``True`` if the stream supports writing. If ``False``,
289 :meth:`write` and :meth:`truncate` will raise :exc:`IOError`.
291 .. method:: writelines(lines)
293 Write a list of lines to the stream. Line separators are not added, so it
294 is usual for each of the lines provided to have a line separator at the
300 Base class for raw binary I/O. It inherits :class:`IOBase`. There is no
303 In addition to the attributes and methods from :class:`IOBase`,
304 RawIOBase provides the following methods:
306 .. method:: read([n])
308 Read and return all the bytes from the stream until EOF, or if *n* is
309 specified, up to *n* bytes. Only one system call is ever made. An empty
310 bytes object is returned on EOF; ``None`` is returned if the object is set
311 not to block and has no data to read.
313 .. method:: readall()
315 Read and return all the bytes from the stream until EOF, using multiple
316 calls to the stream if necessary.
318 .. method:: readinto(b)
320 Read up to len(b) bytes into bytearray *b* and return the number of bytes
325 Write the given bytes or bytearray object, *b*, to the underlying raw
326 stream and return the number of bytes written (This is never less than
327 ``len(b)``, since if the write fails, an :exc:`IOError` will be raised).
333 .. class:: FileIO(name[, mode])
335 :class:`FileIO` represents a file containing bytes data. It implements
336 the :class:`RawIOBase` interface (and therefore the :class:`IOBase`
339 The *mode* can be ``'r'``, ``'w'`` or ``'a'`` for reading (default), writing,
340 or appending. The file will be created if it doesn't exist when opened for
341 writing or appending; it will be truncated when opened for writing. Add a
342 ``'+'`` to the mode to allow simultaneous reading and writing.
344 In addition to the attributes and methods from :class:`IOBase` and
345 :class:`RawIOBase`, :class:`FileIO` provides the following data
346 attributes and methods:
350 The mode as given in the constructor.
354 The file name. This is the file descriptor of the file when no name is
355 given in the constructor.
357 .. method:: read([n])
359 Read and return at most *n* bytes. Only one system call is made, so it is
360 possible that less data than was requested is returned. Use :func:`len`
361 on the returned bytes object to see how many bytes were actually returned.
362 (In non-blocking mode, ``None`` is returned when no data is available.)
364 .. method:: readall()
366 Read and return the entire file's contents in a single bytes object. As
367 much as immediately available is returned in non-blocking mode. If the
368 EOF has been reached, ``b''`` is returned.
372 Write the bytes or bytearray object, *b*, to the file, and return
373 the number actually written. Only one system call is made, so it
374 is possible that only some of the data is written.
376 Note that the inherited ``readinto()`` method should not be used on
377 :class:`FileIO` objects.
383 .. class:: BufferedIOBase
385 Base class for streams that support buffering. It inherits :class:`IOBase`.
386 There is no public constructor.
388 The main difference with :class:`RawIOBase` is that the :meth:`read` method
389 supports omitting the *size* argument, and does not have a default
390 implementation that defers to :meth:`readinto`.
392 In addition, :meth:`read`, :meth:`readinto`, and :meth:`write` may raise
393 :exc:`BlockingIOError` if the underlying raw stream is in non-blocking mode
394 and not ready; unlike their raw counterparts, they will never return
397 A typical implementation should not inherit from a :class:`RawIOBase`
398 implementation, but wrap one like :class:`BufferedWriter` and
399 :class:`BufferedReader`.
401 :class:`BufferedIOBase` provides or overrides these methods in addition to
402 those from :class:`IOBase`:
404 .. method:: read([n])
406 Read and return up to *n* bytes. If the argument is omitted, ``None``, or
407 negative, data is read and returned until EOF is reached. An empty bytes
408 object is returned if the stream is already at EOF.
410 If the argument is positive, and the underlying raw stream is not
411 interactive, multiple raw reads may be issued to satisfy the byte count
412 (unless EOF is reached first). But for interactive raw streams, at most
413 one raw read will be issued, and a short result does not imply that EOF is
416 A :exc:`BlockingIOError` is raised if the underlying raw stream has no
419 .. method:: readinto(b)
421 Read up to len(b) bytes into bytearray *b* and return the number of bytes
424 Like :meth:`read`, multiple reads may be issued to the underlying raw
425 stream, unless the latter is 'interactive.'
427 A :exc:`BlockingIOError` is raised if the underlying raw stream has no
432 Write the given bytes or bytearray object, *b*, to the underlying raw
433 stream and return the number of bytes written (never less than ``len(b)``,
434 since if the write fails an :exc:`IOError` will be raised).
436 A :exc:`BlockingIOError` is raised if the buffer is full, and the
437 underlying raw stream cannot accept more data at the moment.
440 .. class:: BytesIO([initial_bytes])
442 A stream implementation using an in-memory bytes buffer. It inherits
443 :class:`BufferedIOBase`.
445 The argument *initial_bytes* is an optional initial bytearray.
447 :class:`BytesIO` provides or overrides these methods in addition to those
448 from :class:`BufferedIOBase` and :class:`IOBase`:
450 .. method:: getvalue()
452 Return ``bytes`` containing the entire contents of the buffer.
456 In :class:`BytesIO`, this is the same as :meth:`read`.
458 .. method:: truncate([size])
460 Truncate the buffer to at most *size* bytes. *size* defaults to the
461 current stream position, as returned by :meth:`tell`.
464 .. class:: BufferedReader(raw[, buffer_size])
466 A buffer for a readable, sequential :class:`RawIOBase` object. It inherits
467 :class:`BufferedIOBase`.
469 The constructor creates a :class:`BufferedReader` for the given readable
470 *raw* stream and *buffer_size*. If *buffer_size* is omitted,
471 :data:`DEFAULT_BUFFER_SIZE` is used.
473 :class:`BufferedReader` provides or overrides these methods in addition to
474 those from :class:`BufferedIOBase` and :class:`IOBase`:
476 .. method:: peek([n])
478 Return 1 (or *n* if specified) bytes from a buffer without advancing the
479 position. Only a single read on the raw stream is done to satisfy the
480 call. The number of bytes returned may be less than requested since at
481 most all the buffer's bytes from the current position to the end are
484 .. method:: read([n])
486 Read and return *n* bytes, or if *n* is not given or negative, until EOF
487 or if the read call would block in non-blocking mode.
491 Read and return up to *n* bytes with only one call on the raw stream. If
492 at least one byte is buffered, only buffered bytes are returned.
493 Otherwise, one raw stream read call is made.
496 .. class:: BufferedWriter(raw[, buffer_size[, max_buffer_size]])
498 A buffer for a writeable sequential RawIO object. It inherits
499 :class:`BufferedIOBase`.
501 The constructor creates a :class:`BufferedWriter` for the given writeable
502 *raw* stream. If the *buffer_size* is not given, it defaults to
503 :data:`DEAFULT_BUFFER_SIZE`. If *max_buffer_size* is omitted, it defaults to
504 twice the buffer size.
506 :class:`BufferedWriter` provides or overrides these methods in addition to
507 those from :class:`BufferedIOBase` and :class:`IOBase`:
511 Force bytes held in the buffer into the raw stream. A
512 :exc:`BlockingIOError` should be raised if the raw stream blocks.
516 Write the bytes or bytearray object, *b*, onto the raw stream and return
517 the number of bytes written. A :exc:`BlockingIOError` is raised when the
521 .. class:: BufferedRWPair(reader, writer[, buffer_size[, max_buffer_size]])
523 A combined buffered writer and reader object for a raw stream that can be
524 written to and read from. It has and supports both :meth:`read`, :meth:`write`,
525 and their variants. This is useful for sockets and two-way pipes.
526 It inherits :class:`BufferedIOBase`.
528 *reader* and *writer* are :class:`RawIOBase` objects that are readable and
529 writeable respectively. If the *buffer_size* is omitted it defaults to
530 :data:`DEFAULT_BUFFER_SIZE`. The *max_buffer_size* (for the buffered writer)
531 defaults to twice the buffer size.
533 :class:`BufferedRWPair` implements all of :class:`BufferedIOBase`\'s methods.
536 .. class:: BufferedRandom(raw[, buffer_size[, max_buffer_size]])
538 A buffered interface to random access streams. It inherits
539 :class:`BufferedReader` and :class:`BufferedWriter`.
541 The constructor creates a reader and writer for a seekable raw stream, given
542 in the first argument. If the *buffer_size* is omitted it defaults to
543 :data:`DEFAULT_BUFFER_SIZE`. The *max_buffer_size* (for the buffered writer)
544 defaults to twice the buffer size.
546 :class:`BufferedRandom` is capable of anything :class:`BufferedReader` or
547 :class:`BufferedWriter` can do.
553 .. class:: TextIOBase
555 Base class for text streams. This class provides a character and line based
556 interface to stream I/O. There is no :meth:`readinto` method because
557 Python's character strings are immutable. It inherits :class:`IOBase`.
558 There is no public constructor.
560 :class:`TextIOBase` provides or overrides these data attributes and
561 methods in addition to those from :class:`IOBase`:
563 .. attribute:: encoding
565 The name of the encoding used to decode the stream's bytes into
566 strings, and to encode strings into bytes.
568 .. attribute:: newlines
570 A string, a tuple of strings, or ``None``, indicating the newlines
575 Read and return at most *n* characters from the stream as a single
576 :class:`str`. If *n* is negative or ``None``, reads to EOF.
578 .. method:: readline()
580 Read until newline or EOF and return a single ``str``. If the stream is
581 already at EOF, an empty string is returned.
585 Write the string *s* to the stream and return the number of characters
589 .. class:: TextIOWrapper(buffer[, encoding[, errors[, newline[, line_buffering]]]])
591 A buffered text stream over a :class:`BufferedIOBase` raw stream, *buffer*.
592 It inherits :class:`TextIOBase`.
594 *encoding* gives the name of the encoding that the stream will be decoded or
595 encoded with. It defaults to :func:`locale.getpreferredencoding`.
597 *errors* is an optional string that specifies how encoding and decoding
598 errors are to be handled. Pass ``'strict'`` to raise a :exc:`ValueError`
599 exception if there is an encoding error (the default of ``None`` has the same
600 effect), or pass ``'ignore'`` to ignore errors. (Note that ignoring encoding
601 errors can lead to data loss.) ``'replace'`` causes a replacement marker
602 (such as ``'?'``) to be inserted where there is malformed data. When
603 writing, ``'xmlcharrefreplace'`` (replace with the appropriate XML character
604 reference) or ``'backslashreplace'`` (replace with backslashed escape
605 sequences) can be used. Any other error handling name that has been
606 registered with :func:`codecs.register_error` is also valid.
608 *newline* can be ``None``, ``''``, ``'\n'``, ``'\r'``, or ``'\r\n'``. It
609 controls the handling of line endings. If it is ``None``, universal newlines
610 is enabled. With this enabled, on input, the lines endings ``'\n'``,
611 ``'\r'``, or ``'\r\n'`` are translated to ``'\n'`` before being returned to
612 the caller. Conversely, on output, ``'\n'`` is translated to the system
613 default line seperator, :data:`os.linesep`. If *newline* is any other of its
614 legal values, that newline becomes the newline when the file is read and it
615 is returned untranslated. On output, ``'\n'`` is converted to the *newline*.
617 If *line_buffering* is ``True``, :meth:`flush` is implied when a call to
618 write contains a newline character.
620 :class:`TextIOWrapper` provides these data attributes in addition to those of
621 :class:`TextIOBase` and its parents:
623 .. attribute:: errors
625 The encoding and decoding error setting.
627 .. attribute:: line_buffering
629 Whether line buffering is enabled.
632 .. class:: StringIO([initial_value[, encoding[, errors[, newline]]]])
634 An in-memory stream for text. It in inherits :class:`TextIOWrapper`.
636 Create a new StringIO stream with an inital value, encoding, error handling,
637 and newline setting. See :class:`TextIOWrapper`\'s constructor for more
640 :class:`StringIO` provides this method in addition to those from
641 :class:`TextIOWrapper` and its parents:
643 .. method:: getvalue()
645 Return a ``str`` containing the entire contents of the buffer.
648 .. class:: IncrementalNewlineDecoder
650 A helper codec that decodes newlines for universal newlines mode. It
651 inherits :class:`codecs.IncrementalDecoder`.