1 :mod:`io` --- Core tools for working with streams
2 =================================================
5 :synopsis: Core tools for working with streams.
6 .. moduleauthor:: Guido van Rossum <guido@python.org>
7 .. moduleauthor:: Mike Verdone <mike.verdone@gmail.com>
8 .. moduleauthor:: Mark Russell <mark.russell@zen.co.uk>
9 .. sectionauthor:: Benjamin Peterson
12 The :mod:`io` module provides the Python interfaces to stream handling. The
13 builtin :func:`open` function is defined in this module.
15 At the top of the I/O hierarchy is the abstract base class :class:`IOBase`. It
16 defines the basic interface to a stream. Note, however, that there is no
17 seperation between reading and writing to streams; implementations are allowed
18 to throw an :exc:`IOError` if they do not support a given operation.
20 Extending :class:`IOBase` is :class:`RawIOBase` which deals simply with the
21 reading and writing of raw bytes to a stream. :class:`FileIO` subclasses
22 :class:`RawIOBase` to provide an interface to files in the machine's
25 :class:`BufferedIOBase` deals with buffering on a raw byte stream
26 (:class:`RawIOBase`). Its subclasses, :class:`BufferedWriter`,
27 :class:`BufferedReader`, and :class:`BufferedRWPair` buffer streams that are
28 readable, writable, and both readable and writable.
29 :class:`BufferedRandom` provides a buffered interface to random access
30 streams. :class:`BytesIO` is a simple stream of in-memory bytes.
32 Another :class:`IOBase` subclass, :class:`TextIOBase`, deals with
33 streams whose bytes represent text, and handles encoding and decoding
34 from and to strings. :class:`TextIOWrapper`, which extends it, is a
35 buffered text interface to a buffered raw stream
36 (:class:`BufferedIOBase`). Finally, :class:`StringIO` is an in-memory
39 Argument names are not part of the specification, and only the arguments of
40 :func:`open` are intended to be used as keyword arguments.
46 .. data:: DEFAULT_BUFFER_SIZE
48 An int containing the default buffer size used by the module's buffered I/O
49 classes. :func:`open` uses the file's blksize (as obtained by
50 :func:`os.stat`) if possible.
52 .. function:: open(file[, mode[, buffering[, encoding[, errors[, newline[, closefd=True]]]]]])
54 Open *file* and return a stream. If the file cannot be opened, an
55 :exc:`IOError` is raised.
57 *file* is either a string giving the name (and the path if the file isn't in
58 the current working directory) of the file to be opened or a file
59 descriptor of the file to be opened. (If a file descriptor is given,
60 for example, from :func:`os.fdopen`, it is closed when the returned
61 I/O object is closed, unless *closefd* is set to ``False``.)
63 *mode* is an optional string that specifies the mode in which the file is
64 opened. It defaults to ``'r'`` which means open for reading in text mode.
65 Other common values are ``'w'`` for writing (truncating the file if it
66 already exists), and ``'a'`` for appending (which on *some* Unix systems,
67 means that *all* writes append to the end of the file regardless of the
68 current seek position). In text mode, if *encoding* is not specified the
69 encoding used is platform dependent. (For reading and writing raw bytes use
70 binary mode and leave *encoding* unspecified.) The available modes are:
72 ========= ===============================================================
74 --------- ---------------------------------------------------------------
75 ``'r'`` open for reading (default)
76 ``'w'`` open for writing, truncating the file first
77 ``'a'`` open for writing, appending to the end of the file if it exists
79 ``'t'`` text mode (default)
80 ``'+'`` open a disk file for updating (reading and writing)
81 ``'U'`` universal newline mode (for backwards compatibility; should
82 not be used in new code)
83 ========= ===============================================================
85 The default mode is ``'rt'`` (open for reading text). For binary random
86 access, the mode ``'w+b'`` opens and truncates the file to 0 bytes, while
87 ``'r+b'`` opens the file without truncation.
89 Python distinguishes between files opened in binary and text modes, even when
90 the underlying operating system doesn't. Files opened in binary mode
91 (including ``'b'`` in the *mode* argument) return contents as ``bytes``
92 objects without any decoding. In text mode (the default, or when ``'t'`` is
93 included in the *mode* argument), the contents of the file are returned as
94 strings, the bytes having been first decoded using a platform-dependent
95 encoding or using the specified *encoding* if given.
97 *buffering* is an optional integer used to set the buffering policy. By
98 default full buffering is on. Pass 0 to switch buffering off (only allowed
99 in binary mode), 1 to set line buffering, and an integer > 1 for full
102 *encoding* is the name of the encoding used to decode or encode the file.
103 This should only be used in text mode. The default encoding is platform
104 dependent, but any encoding supported by Python can be used. See the
105 :mod:`codecs` module for the list of supported encodings.
107 *errors* is an optional string that specifies how encoding and decoding
108 errors are to be handled. Pass ``'strict'`` to raise a :exc:`ValueError`
109 exception if there is an encoding error (the default of ``None`` has the same
110 effect), or pass ``'ignore'`` to ignore errors. (Note that ignoring encoding
111 errors can lead to data loss.) ``'replace'`` causes a replacement marker
112 (such as ``'?'``) to be inserted where there is malformed data. When
113 writing, ``'xmlcharrefreplace'`` (replace with the appropriate XML character
114 reference) or ``'backslashreplace'`` (replace with backslashed escape
115 sequences) can be used. Any other error handling name that has been
116 registered with :func:`codecs.register_error` is also valid.
118 *newline* controls how universal newlines works (it only applies to text
119 mode). It can be ``None``, ``''``, ``'\n'``, ``'\r'``, and ``'\r\n'``. It
122 * On input, if *newline* is ``None``, universal newlines mode is enabled.
123 Lines in the input can end in ``'\n'``, ``'\r'``, or ``'\r\n'``, and these
124 are translated into ``'\n'`` before being returned to the caller. If it is
125 ``''``, universal newline mode is enabled, but line endings are returned to
126 the caller untranslated. If it has any of the other legal values, input
127 lines are only terminated by the given string, and the line ending is
128 returned to the caller untranslated.
130 * On output, if *newline* is ``None``, any ``'\n'`` characters written are
131 translated to the system default line separator, :data:`os.linesep`. If
132 *newline* is ``''``, no translation takes place. If *newline* is any of
133 the other legal values, any ``'\n'`` characters written are translated to
136 If *closefd* is ``False`` and a file descriptor rather than a
137 filename was given, the underlying file descriptor will be kept open
138 when the file is closed. If a filename is given *closefd* has no
139 effect but must be ``True`` (the default).
141 The type of file object returned by the :func:`open` function depends
142 on the mode. When :func:`open` is used to open a file in a text mode
143 (``'w'``, ``'r'``, ``'wt'``, ``'rt'``, etc.), it returns a
144 :class:`TextIOWrapper`. When used to open a file in a binary mode,
145 the returned class varies: in read binary mode, it returns a
146 :class:`BufferedReader`; in write binary and append binary modes, it
147 returns a :class:`BufferedWriter`, and in read/write mode, it returns
148 a :class:`BufferedRandom`.
150 It is also possible to use a string or bytearray as a file for both reading
151 and writing. For strings :class:`StringIO` can be used like a file opened in
152 a text mode, and for bytearrays a :class:`BytesIO` can be used like a
153 file opened in a binary mode.
156 .. exception:: BlockingIOError
158 Error raised when blocking would occur on a non-blocking stream. It inherits
161 In addition to those of :exc:`IOError`, :exc:`BlockingIOError` has one
164 .. attribute:: characters_written
166 An integer containing the number of characters written to the stream
170 .. exception:: UnsupportedOperation
172 An exception inheriting :exc:`IOError` and :exc:`ValueError` that is raised
173 when an unsupported operation is called on a stream.
181 The abstract base class for all I/O classes, acting on streams of bytes.
182 There is no public constructor.
184 This class provides empty abstract implementations for many methods
185 that derived classes can override selectively; the default
186 implementations represent a file that cannot be read, written or
189 Even though :class:`IOBase` does not declare :meth:`read`, :meth:`readinto`,
190 or :meth:`write` because their signatures will vary, implementations and
191 clients should consider those methods part of the interface. Also,
192 implementations may raise a :exc:`IOError` when operations they do not
195 The basic type used for binary data read from or written to a file is
196 :class:`bytes`. :class:`bytearray`\s are accepted too, and in some cases
197 (such as :class:`readinto`) required. Text I/O classes work with
200 Note that calling any method (even inquiries) on a closed stream is
201 undefined. Implementations may raise :exc:`IOError` in this case.
203 IOBase (and its subclasses) support the iterator protocol, meaning that an
204 :class:`IOBase` object can be iterated over yielding the lines in a stream.
206 IOBase is also a context manager and therefore supports the
207 :keyword:`with` statement. In this example, *file* is closed after the
208 :keyword:`with` statement's suite is finished---even if an exception occurs::
210 with open('spam.txt', 'w') as file:
211 file.write('Spam and eggs!')
213 :class:`IOBase` provides these data attributes and methods:
217 Flush and close this stream. This method has no effect if the file is
218 already closed. Once the file is closed, any operation on the file
219 (e.g. reading or writing) will raise an :exc:`IOError`. The internal
220 file descriptor isn't closed if *closefd* was False.
222 .. attribute:: closed
224 True if the stream is closed.
228 Return the underlying file descriptor (an integer) of the stream if it
229 exists. An :exc:`IOError` is raised if the IO object does not use a file
234 Flush the write buffers of the stream if applicable. This does nothing
235 for read-only and non-blocking streams.
239 Return ``True`` if the stream is interactive (i.e., connected to
240 a terminal/tty device).
242 .. method:: readable()
244 Return ``True`` if the stream can be read from. If False, :meth:`read`
245 will raise :exc:`IOError`.
247 .. method:: readline([limit])
249 Read and return one line from the stream. If *limit* is specified, at
250 most *limit* bytes will be read.
252 The line terminator is always ``b'\n'`` for binary files; for text files,
253 the *newlines* argument to :func:`open` can be used to select the line
254 terminator(s) recognized.
256 .. method:: readlines([hint])
258 Read and return a list of lines from the stream. *hint* can be specified
259 to control the number of lines read: no more lines will be read if the
260 total size (in bytes/characters) of all lines so far exceeds *hint*.
262 .. method:: seek(offset[, whence])
264 Change the stream position to the given byte *offset*. *offset* is
265 interpreted relative to the position indicated by *whence*. Values for
268 * ``0`` -- start of the stream (the default); *offset* should be zero or positive
269 * ``1`` -- current stream position; *offset* may be negative
270 * ``2`` -- end of the stream; *offset* is usually negative
272 Return the new absolute position.
274 .. method:: seekable()
276 Return ``True`` if the stream supports random access. If ``False``,
277 :meth:`seek`, :meth:`tell` and :meth:`truncate` will raise :exc:`IOError`.
281 Return the current stream position.
283 .. method:: truncate([size])
285 Truncate the file to at most *size* bytes. *size* defaults to the current
286 file position, as returned by :meth:`tell`.
288 .. method:: writable()
290 Return ``True`` if the stream supports writing. If ``False``,
291 :meth:`write` and :meth:`truncate` will raise :exc:`IOError`.
293 .. method:: writelines(lines)
295 Write a list of lines to the stream. Line separators are not added, so it
296 is usual for each of the lines provided to have a line separator at the
302 Base class for raw binary I/O. It inherits :class:`IOBase`. There is no
305 In addition to the attributes and methods from :class:`IOBase`,
306 RawIOBase provides the following methods:
308 .. method:: read([n])
310 Read and return all the bytes from the stream until EOF, or if *n* is
311 specified, up to *n* bytes. Only one system call is ever made. An empty
312 bytes object is returned on EOF; ``None`` is returned if the object is set
313 not to block and has no data to read.
315 .. method:: readall()
317 Read and return all the bytes from the stream until EOF, using multiple
318 calls to the stream if necessary.
320 .. method:: readinto(b)
322 Read up to len(b) bytes into bytearray *b* and return the number of bytes
327 Write the given bytes or bytearray object, *b*, to the underlying raw
328 stream and return the number of bytes written (This is never less than
329 ``len(b)``, since if the write fails, an :exc:`IOError` will be raised).
335 .. class:: FileIO(name[, mode])
337 :class:`FileIO` represents a file containing bytes data. It implements
338 the :class:`RawIOBase` interface (and therefore the :class:`IOBase`
341 The *mode* can be ``'r'``, ``'w'`` or ``'a'`` for reading (default), writing,
342 or appending. The file will be created if it doesn't exist when opened for
343 writing or appending; it will be truncated when opened for writing. Add a
344 ``'+'`` to the mode to allow simultaneous reading and writing.
346 In addition to the attributes and methods from :class:`IOBase` and
347 :class:`RawIOBase`, :class:`FileIO` provides the following data
348 attributes and methods:
352 The mode as given in the constructor.
356 The file name. This is the file descriptor of the file when no name is
357 given in the constructor.
359 .. method:: read([n])
361 Read and return at most *n* bytes. Only one system call is made, so it is
362 possible that less data than was requested is returned. Use :func:`len`
363 on the returned bytes object to see how many bytes were actually returned.
364 (In non-blocking mode, ``None`` is returned when no data is available.)
366 .. method:: readall()
368 Read and return the entire file's contents in a single bytes object. As
369 much as immediately available is returned in non-blocking mode. If the
370 EOF has been reached, ``b''`` is returned.
374 Write the bytes or bytearray object, *b*, to the file, and return
375 the number actually written. Only one system call is made, so it
376 is possible that only some of the data is written.
378 Note that the inherited ``readinto()`` method should not be used on
379 :class:`FileIO` objects.
385 .. class:: BufferedIOBase
387 Base class for streams that support buffering. It inherits :class:`IOBase`.
388 There is no public constructor.
390 The main difference with :class:`RawIOBase` is that the :meth:`read` method
391 supports omitting the *size* argument, and does not have a default
392 implementation that defers to :meth:`readinto`.
394 In addition, :meth:`read`, :meth:`readinto`, and :meth:`write` may raise
395 :exc:`BlockingIOError` if the underlying raw stream is in non-blocking mode
396 and not ready; unlike their raw counterparts, they will never return
399 A typical implementation should not inherit from a :class:`RawIOBase`
400 implementation, but wrap one like :class:`BufferedWriter` and
401 :class:`BufferedReader`.
403 :class:`BufferedIOBase` provides or overrides these methods in addition to
404 those from :class:`IOBase`:
406 .. method:: read([n])
408 Read and return up to *n* bytes. If the argument is omitted, ``None``, or
409 negative, data is read and returned until EOF is reached. An empty bytes
410 object is returned if the stream is already at EOF.
412 If the argument is positive, and the underlying raw stream is not
413 interactive, multiple raw reads may be issued to satisfy the byte count
414 (unless EOF is reached first). But for interactive raw streams, at most
415 one raw read will be issued, and a short result does not imply that EOF is
418 A :exc:`BlockingIOError` is raised if the underlying raw stream has no
421 .. method:: readinto(b)
423 Read up to len(b) bytes into bytearray *b* and return the number of bytes
426 Like :meth:`read`, multiple reads may be issued to the underlying raw
427 stream, unless the latter is 'interactive.'
429 A :exc:`BlockingIOError` is raised if the underlying raw stream has no
434 Write the given bytes or bytearray object, *b*, to the underlying raw
435 stream and return the number of bytes written (never less than ``len(b)``,
436 since if the write fails an :exc:`IOError` will be raised).
438 A :exc:`BlockingIOError` is raised if the buffer is full, and the
439 underlying raw stream cannot accept more data at the moment.
442 .. class:: BytesIO([initial_bytes])
444 A stream implementation using an in-memory bytes buffer. It inherits
445 :class:`BufferedIOBase`.
447 The argument *initial_bytes* is an optional initial bytearray.
449 :class:`BytesIO` provides or overrides these methods in addition to those
450 from :class:`BufferedIOBase` and :class:`IOBase`:
452 .. method:: getvalue()
454 Return ``bytes`` containing the entire contents of the buffer.
458 In :class:`BytesIO`, this is the same as :meth:`read`.
460 .. method:: truncate([size])
462 Truncate the buffer to at most *size* bytes. *size* defaults to the
463 current stream position, as returned by :meth:`tell`.
466 .. class:: BufferedReader(raw[, buffer_size])
468 A buffer for a readable, sequential :class:`RawIOBase` object. It inherits
469 :class:`BufferedIOBase`.
471 The constructor creates a :class:`BufferedReader` for the given readable
472 *raw* stream and *buffer_size*. If *buffer_size* is omitted,
473 :data:`DEFAULT_BUFFER_SIZE` is used.
475 :class:`BufferedReader` provides or overrides these methods in addition to
476 those from :class:`BufferedIOBase` and :class:`IOBase`:
478 .. method:: peek([n])
480 Return 1 (or *n* if specified) bytes from a buffer without advancing the
481 position. Only a single read on the raw stream is done to satisfy the
482 call. The number of bytes returned may be less than requested since at
483 most all the buffer's bytes from the current position to the end are
486 .. method:: read([n])
488 Read and return *n* bytes, or if *n* is not given or negative, until EOF
489 or if the read call would block in non-blocking mode.
493 Read and return up to *n* bytes with only one call on the raw stream. If
494 at least one byte is buffered, only buffered bytes are returned.
495 Otherwise, one raw stream read call is made.
498 .. class:: BufferedWriter(raw[, buffer_size[, max_buffer_size]])
500 A buffer for a writeable sequential RawIO object. It inherits
501 :class:`BufferedIOBase`.
503 The constructor creates a :class:`BufferedWriter` for the given writeable
504 *raw* stream. If the *buffer_size* is not given, it defaults to
505 :data:`DEAFULT_BUFFER_SIZE`. If *max_buffer_size* is omitted, it defaults to
506 twice the buffer size.
508 :class:`BufferedWriter` provides or overrides these methods in addition to
509 those from :class:`BufferedIOBase` and :class:`IOBase`:
513 Force bytes held in the buffer into the raw stream. A
514 :exc:`BlockingIOError` should be raised if the raw stream blocks.
518 Write the bytes or bytearray object, *b*, onto the raw stream and return
519 the number of bytes written. A :exc:`BlockingIOError` is raised when the
523 .. class:: BufferedRWPair(reader, writer[, buffer_size[, max_buffer_size]])
525 A combined buffered writer and reader object for a raw stream that can be
526 written to and read from. It has and supports both :meth:`read`, :meth:`write`,
527 and their variants. This is useful for sockets and two-way pipes.
528 It inherits :class:`BufferedIOBase`.
530 *reader* and *writer* are :class:`RawIOBase` objects that are readable and
531 writeable respectively. If the *buffer_size* is omitted it defaults to
532 :data:`DEFAULT_BUFFER_SIZE`. The *max_buffer_size* (for the buffered writer)
533 defaults to twice the buffer size.
535 :class:`BufferedRWPair` implements all of :class:`BufferedIOBase`\'s methods.
538 .. class:: BufferedRandom(raw[, buffer_size[, max_buffer_size]])
540 A buffered interface to random access streams. It inherits
541 :class:`BufferedReader` and :class:`BufferedWriter`.
543 The constructor creates a reader and writer for a seekable raw stream, given
544 in the first argument. If the *buffer_size* is omitted it defaults to
545 :data:`DEFAULT_BUFFER_SIZE`. The *max_buffer_size* (for the buffered writer)
546 defaults to twice the buffer size.
548 :class:`BufferedRandom` is capable of anything :class:`BufferedReader` or
549 :class:`BufferedWriter` can do.
555 .. class:: TextIOBase
557 Base class for text streams. This class provides a character and line based
558 interface to stream I/O. There is no :meth:`readinto` method because
559 Python's character strings are immutable. It inherits :class:`IOBase`.
560 There is no public constructor.
562 :class:`TextIOBase` provides or overrides these data attributes and
563 methods in addition to those from :class:`IOBase`:
565 .. attribute:: encoding
567 The name of the encoding used to decode the stream's bytes into
568 strings, and to encode strings into bytes.
570 .. attribute:: newlines
572 A string, a tuple of strings, or ``None``, indicating the newlines
577 Read and return at most *n* characters from the stream as a single
578 :class:`str`. If *n* is negative or ``None``, reads to EOF.
580 .. method:: readline()
582 Read until newline or EOF and return a single ``str``. If the stream is
583 already at EOF, an empty string is returned.
587 Write the string *s* to the stream and return the number of characters
591 .. class:: TextIOWrapper(buffer[, encoding[, errors[, newline[, line_buffering]]]])
593 A buffered text stream over a :class:`BufferedIOBase` raw stream, *buffer*.
594 It inherits :class:`TextIOBase`.
596 *encoding* gives the name of the encoding that the stream will be decoded or
597 encoded with. It defaults to :func:`locale.getpreferredencoding`.
599 *errors* is an optional string that specifies how encoding and decoding
600 errors are to be handled. Pass ``'strict'`` to raise a :exc:`ValueError`
601 exception if there is an encoding error (the default of ``None`` has the same
602 effect), or pass ``'ignore'`` to ignore errors. (Note that ignoring encoding
603 errors can lead to data loss.) ``'replace'`` causes a replacement marker
604 (such as ``'?'``) to be inserted where there is malformed data. When
605 writing, ``'xmlcharrefreplace'`` (replace with the appropriate XML character
606 reference) or ``'backslashreplace'`` (replace with backslashed escape
607 sequences) can be used. Any other error handling name that has been
608 registered with :func:`codecs.register_error` is also valid.
610 *newline* can be ``None``, ``''``, ``'\n'``, ``'\r'``, or ``'\r\n'``. It
611 controls the handling of line endings. If it is ``None``, universal newlines
612 is enabled. With this enabled, on input, the lines endings ``'\n'``,
613 ``'\r'``, or ``'\r\n'`` are translated to ``'\n'`` before being returned to
614 the caller. Conversely, on output, ``'\n'`` is translated to the system
615 default line seperator, :data:`os.linesep`. If *newline* is any other of its
616 legal values, that newline becomes the newline when the file is read and it
617 is returned untranslated. On output, ``'\n'`` is converted to the *newline*.
619 If *line_buffering* is ``True``, :meth:`flush` is implied when a call to
620 write contains a newline character.
622 :class:`TextIOWrapper` provides these data attributes in addition to those of
623 :class:`TextIOBase` and its parents:
625 .. attribute:: errors
627 The encoding and decoding error setting.
629 .. attribute:: line_buffering
631 Whether line buffering is enabled.
634 .. class:: StringIO([initial_value[, encoding[, errors[, newline]]]])
636 An in-memory stream for text. It in inherits :class:`TextIOWrapper`.
638 Create a new StringIO stream with an inital value, encoding, error handling,
639 and newline setting. See :class:`TextIOWrapper`\'s constructor for more
642 :class:`StringIO` provides this method in addition to those from
643 :class:`TextIOWrapper` and its parents:
645 .. method:: getvalue()
647 Return a ``str`` containing the entire contents of the buffer.
650 .. class:: IncrementalNewlineDecoder
652 A helper codec that decodes newlines for universal newlines mode. It
653 inherits :class:`codecs.IncrementalDecoder`.