3 :mod:`tarfile` --- Read and write tar archive files
4 ===================================================
7 :synopsis: Read and write tar-format archive files.
12 .. moduleauthor:: Lars Gustäbel <lars@gustaebel.de>
13 .. sectionauthor:: Lars Gustäbel <lars@gustaebel.de>
16 The :mod:`tarfile` module makes it possible to read and write tar
17 archives, including those using gzip or bz2 compression.
18 (:file:`.zip` files can be read and written using the :mod:`zipfile` module.)
20 Some facts and figures:
22 * reads and writes :mod:`gzip` and :mod:`bz2` compressed archives.
24 * read/write support for the POSIX.1-1988 (ustar) format.
26 * read/write support for the GNU tar format including *longname* and *longlink*
27 extensions, read-only support for the *sparse* extension.
29 * read/write support for the POSIX.1-2001 (pax) format.
33 * handles directories, regular files, hardlinks, symbolic links, fifos,
34 character devices and block devices and is able to acquire and restore file
35 information like timestamp, access permissions and owner.
37 * can handle tape devices.
40 .. function:: open(name[, mode[, fileobj[, bufsize]]], **kwargs)
42 Return a :class:`TarFile` object for the pathname *name*. For detailed
43 information on :class:`TarFile` objects and the keyword arguments that are
44 allowed, see :ref:`tarfile-objects`.
46 *mode* has to be a string of the form ``'filemode[:compression]'``, it defaults
47 to ``'r'``. Here is a full list of mode combinations:
49 +------------------+---------------------------------------------+
51 +==================+=============================================+
52 | ``'r' or 'r:*'`` | Open for reading with transparent |
53 | | compression (recommended). |
54 +------------------+---------------------------------------------+
55 | ``'r:'`` | Open for reading exclusively without |
57 +------------------+---------------------------------------------+
58 | ``'r:gz'`` | Open for reading with gzip compression. |
59 +------------------+---------------------------------------------+
60 | ``'r:bz2'`` | Open for reading with bzip2 compression. |
61 +------------------+---------------------------------------------+
62 | ``'a' or 'a:'`` | Open for appending with no compression. The |
63 | | file is created if it does not exist. |
64 +------------------+---------------------------------------------+
65 | ``'w' or 'w:'`` | Open for uncompressed writing. |
66 +------------------+---------------------------------------------+
67 | ``'w:gz'`` | Open for gzip compressed writing. |
68 +------------------+---------------------------------------------+
69 | ``'w:bz2'`` | Open for bzip2 compressed writing. |
70 +------------------+---------------------------------------------+
72 Note that ``'a:gz'`` or ``'a:bz2'`` is not possible. If *mode* is not suitable
73 to open a certain (compressed) file for reading, :exc:`ReadError` is raised. Use
74 *mode* ``'r'`` to avoid this. If a compression method is not supported,
75 :exc:`CompressionError` is raised.
77 If *fileobj* is specified, it is used as an alternative to a file object opened
78 for *name*. It is supposed to be at position 0.
80 For special purposes, there is a second format for *mode*:
81 ``'filemode|[compression]'``. :func:`open` will return a :class:`TarFile`
82 object that processes its data as a stream of blocks. No random seeking will
83 be done on the file. If given, *fileobj* may be any object that has a
84 :meth:`read` or :meth:`write` method (depending on the *mode*). *bufsize*
85 specifies the blocksize and defaults to ``20 * 512`` bytes. Use this variant
86 in combination with e.g. ``sys.stdin``, a socket file object or a tape
87 device. However, such a :class:`TarFile` object is limited in that it does
88 not allow to be accessed randomly, see :ref:`tar-examples`. The currently
91 +-------------+--------------------------------------------+
93 +=============+============================================+
94 | ``'r|*'`` | Open a *stream* of tar blocks for reading |
95 | | with transparent compression. |
96 +-------------+--------------------------------------------+
97 | ``'r|'`` | Open a *stream* of uncompressed tar blocks |
99 +-------------+--------------------------------------------+
100 | ``'r|gz'`` | Open a gzip compressed *stream* for |
102 +-------------+--------------------------------------------+
103 | ``'r|bz2'`` | Open a bzip2 compressed *stream* for |
105 +-------------+--------------------------------------------+
106 | ``'w|'`` | Open an uncompressed *stream* for writing. |
107 +-------------+--------------------------------------------+
108 | ``'w|gz'`` | Open an gzip compressed *stream* for |
110 +-------------+--------------------------------------------+
111 | ``'w|bz2'`` | Open an bzip2 compressed *stream* for |
113 +-------------+--------------------------------------------+
118 Class for reading and writing tar archives. Do not use this class directly,
119 better use :func:`open` instead. See :ref:`tarfile-objects`.
122 .. function:: is_tarfile(name)
124 Return :const:`True` if *name* is a tar archive file, that the :mod:`tarfile`
128 .. class:: TarFileCompat(filename[, mode[, compression]])
130 Class for limited access to tar archives with a :mod:`zipfile`\ -like interface.
131 Please consult the documentation of the :mod:`zipfile` module for more details.
132 *compression* must be one of the following constants:
137 Constant for an uncompressed tar archive.
140 .. data:: TAR_GZIPPED
142 Constant for a :mod:`gzip` compressed tar archive.
145 .. exception:: TarError
147 Base class for all :mod:`tarfile` exceptions.
150 .. exception:: ReadError
152 Is raised when a tar archive is opened, that either cannot be handled by the
153 :mod:`tarfile` module or is somehow invalid.
156 .. exception:: CompressionError
158 Is raised when a compression method is not supported or when the data cannot be
162 .. exception:: StreamError
164 Is raised for the limitations that are typical for stream-like :class:`TarFile`
168 .. exception:: ExtractError
170 Is raised for *non-fatal* errors when using :meth:`extract`, but only if
171 :attr:`TarFile.errorlevel`\ ``== 2``.
174 .. exception:: HeaderError
176 Is raised by :meth:`frombuf` if the buffer it gets is invalid.
178 .. versionadded:: 2.6
180 Each of the following constants defines a tar archive format that the
181 :mod:`tarfile` module is able to create. See section :ref:`tar-formats` for
185 .. data:: USTAR_FORMAT
187 POSIX.1-1988 (ustar) format.
197 POSIX.1-2001 (pax) format.
200 .. data:: DEFAULT_FORMAT
202 The default format for creating archives. This is currently :const:`GNU_FORMAT`.
207 Module :mod:`zipfile`
208 Documentation of the :mod:`zipfile` standard module.
210 `GNU tar manual, Basic Tar Format <http://www.gnu.org/software/tar/manual/html_node/tar_134.html#SEC134>`_
211 Documentation for tar archive files, including GNU tar extensions.
219 The :class:`TarFile` object provides an interface to a tar archive. A tar
220 archive is a sequence of blocks. An archive member (a stored file) is made up of
221 a header block followed by data blocks. It is possible to store a file in a tar
222 archive several times. Each archive member is represented by a :class:`TarInfo`
223 object, see :ref:`tarinfo-objects` for details.
226 .. class:: TarFile(name=None, mode='r', fileobj=None, format=DEFAULT_FORMAT, tarinfo=TarInfo, dereference=False, ignore_zeros=False, encoding=None, errors=None, pax_headers=None, debug=0, errorlevel=0)
228 All following arguments are optional and can be accessed as instance attributes
231 *name* is the pathname of the archive. It can be omitted if *fileobj* is given.
232 In this case, the file object's :attr:`name` attribute is used if it exists.
234 *mode* is either ``'r'`` to read from an existing archive, ``'a'`` to append
235 data to an existing file or ``'w'`` to create a new file overwriting an existing
238 If *fileobj* is given, it is used for reading or writing data. If it can be
239 determined, *mode* is overridden by *fileobj*'s mode. *fileobj* will be used
244 *fileobj* is not closed, when :class:`TarFile` is closed.
246 *format* controls the archive format. It must be one of the constants
247 :const:`USTAR_FORMAT`, :const:`GNU_FORMAT` or :const:`PAX_FORMAT` that are
248 defined at module level.
250 .. versionadded:: 2.6
252 The *tarinfo* argument can be used to replace the default :class:`TarInfo` class
253 with a different one.
255 .. versionadded:: 2.6
257 If *dereference* is ``False``, add symbolic and hard links to the archive. If it
258 is ``True``, add the content of the target files to the archive. This has no
259 effect on systems that do not support symbolic links.
261 If *ignore_zeros* is ``False``, treat an empty block as the end of the archive.
262 If it is *True*, skip empty (and invalid) blocks and try to get as many members
263 as possible. This is only useful for reading concatenated or damaged archives.
265 *debug* can be set from ``0`` (no debug messages) up to ``3`` (all debug
266 messages). The messages are written to ``sys.stderr``.
268 If *errorlevel* is ``0``, all errors are ignored when using :meth:`extract`.
269 Nevertheless, they appear as error messages in the debug output, when debugging
270 is enabled. If ``1``, all *fatal* errors are raised as :exc:`OSError` or
271 :exc:`IOError` exceptions. If ``2``, all *non-fatal* errors are raised as
272 :exc:`TarError` exceptions as well.
274 The *encoding* and *errors* arguments control the way strings are converted to
275 unicode objects and vice versa. The default settings will work for most users.
276 See section :ref:`tar-unicode` for in-depth information.
278 .. versionadded:: 2.6
280 The *pax_headers* argument is an optional dictionary of unicode strings which
281 will be added as a pax global header if *format* is :const:`PAX_FORMAT`.
283 .. versionadded:: 2.6
286 .. method:: TarFile.open(...)
288 Alternative constructor. The :func:`open` function on module level is actually a
289 shortcut to this classmethod. See section :ref:`tarfile-mod` for details.
292 .. method:: TarFile.getmember(name)
294 Return a :class:`TarInfo` object for member *name*. If *name* can not be found
295 in the archive, :exc:`KeyError` is raised.
299 If a member occurs more than once in the archive, its last occurrence is assumed
300 to be the most up-to-date version.
303 .. method:: TarFile.getmembers()
305 Return the members of the archive as a list of :class:`TarInfo` objects. The
306 list has the same order as the members in the archive.
309 .. method:: TarFile.getnames()
311 Return the members as a list of their names. It has the same order as the list
312 returned by :meth:`getmembers`.
315 .. method:: TarFile.list(verbose=True)
317 Print a table of contents to ``sys.stdout``. If *verbose* is :const:`False`,
318 only the names of the members are printed. If it is :const:`True`, output
319 similar to that of :program:`ls -l` is produced.
322 .. method:: TarFile.next()
324 Return the next member of the archive as a :class:`TarInfo` object, when
325 :class:`TarFile` is opened for reading. Return ``None`` if there is no more
329 .. method:: TarFile.extractall([path[, members]])
331 Extract all members from the archive to the current working directory or
332 directory *path*. If optional *members* is given, it must be a subset of the
333 list returned by :meth:`getmembers`. Directory information like owner,
334 modification time and permissions are set after all members have been extracted.
335 This is done to work around two problems: A directory's modification time is
336 reset each time a file is created in it. And, if a directory's permissions do
337 not allow writing, extracting files to it will fail.
341 Never extract archives from untrusted sources without prior inspection.
342 It is possible that files are created outside of *path*, e.g. members
343 that have absolute filenames starting with ``"/"`` or filenames with two
346 .. versionadded:: 2.5
349 .. method:: TarFile.extract(member[, path])
351 Extract a member from the archive to the current working directory, using its
352 full name. Its file information is extracted as accurately as possible. *member*
353 may be a filename or a :class:`TarInfo` object. You can specify a different
354 directory using *path*.
358 Because the :meth:`extract` method allows random access to a tar archive there
359 are some issues you must take care of yourself. See the description for
360 :meth:`extractall` above.
364 See the warning for :meth:`extractall`.
367 .. method:: TarFile.extractfile(member)
369 Extract a member from the archive as a file object. *member* may be a filename
370 or a :class:`TarInfo` object. If *member* is a regular file, a file-like object
371 is returned. If *member* is a link, a file-like object is constructed from the
372 link's target. If *member* is none of the above, ``None`` is returned.
376 The file-like object is read-only and provides the following methods:
377 :meth:`read`, :meth:`readline`, :meth:`readlines`, :meth:`seek`, :meth:`tell`.
380 .. method:: TarFile.add(name[, arcname[, recursive[, exclude]]])
382 Add the file *name* to the archive. *name* may be any type of file (directory,
383 fifo, symbolic link, etc.). If given, *arcname* specifies an alternative name
384 for the file in the archive. Directories are added recursively by default. This
385 can be avoided by setting *recursive* to :const:`False`. If *exclude* is given
386 it must be a function that takes one filename argument and returns a boolean
387 value. Depending on this value the respective file is either excluded
388 (:const:`True`) or added (:const:`False`).
390 .. versionchanged:: 2.6
391 Added the *exclude* parameter.
394 .. method:: TarFile.addfile(tarinfo[, fileobj])
396 Add the :class:`TarInfo` object *tarinfo* to the archive. If *fileobj* is given,
397 ``tarinfo.size`` bytes are read from it and added to the archive. You can
398 create :class:`TarInfo` objects using :meth:`gettarinfo`.
402 On Windows platforms, *fileobj* should always be opened with mode ``'rb'`` to
403 avoid irritation about the file size.
406 .. method:: TarFile.gettarinfo([name[, arcname[, fileobj]]])
408 Create a :class:`TarInfo` object for either the file *name* or the file object
409 *fileobj* (using :func:`os.fstat` on its file descriptor). You can modify some
410 of the :class:`TarInfo`'s attributes before you add it using :meth:`addfile`.
411 If given, *arcname* specifies an alternative name for the file in the archive.
414 .. method:: TarFile.close()
416 Close the :class:`TarFile`. In write mode, two finishing zero blocks are
417 appended to the archive.
420 .. attribute:: TarFile.posix
422 Setting this to :const:`True` is equivalent to setting the :attr:`format`
423 attribute to :const:`USTAR_FORMAT`, :const:`False` is equivalent to
426 .. versionchanged:: 2.4
427 *posix* defaults to :const:`False`.
430 Use the :attr:`format` attribute instead.
433 .. attribute:: TarFile.pax_headers
435 A dictionary containing key-value pairs of pax global headers.
437 .. versionadded:: 2.6
445 A :class:`TarInfo` object represents one member in a :class:`TarFile`. Aside
446 from storing all required attributes of a file (like file type, size, time,
447 permissions, owner etc.), it provides some useful methods to determine its type.
448 It does *not* contain the file's data itself.
450 :class:`TarInfo` objects are returned by :class:`TarFile`'s methods
451 :meth:`getmember`, :meth:`getmembers` and :meth:`gettarinfo`.
454 .. class:: TarInfo([name])
456 Create a :class:`TarInfo` object.
459 .. method:: TarInfo.frombuf(buf)
461 Create and return a :class:`TarInfo` object from string buffer *buf*.
463 .. versionadded:: 2.6
464 Raises :exc:`HeaderError` if the buffer is invalid..
467 .. method:: TarInfo.fromtarfile(tarfile)
469 Read the next member from the :class:`TarFile` object *tarfile* and return it as
470 a :class:`TarInfo` object.
472 .. versionadded:: 2.6
475 .. method:: TarInfo.tobuf([format[, encoding [, errors]]])
477 Create a string buffer from a :class:`TarInfo` object. For information on the
478 arguments see the constructor of the :class:`TarFile` class.
480 .. versionchanged:: 2.6
481 The arguments were added.
483 A ``TarInfo`` object has the following public data attributes:
486 .. attribute:: TarInfo.name
488 Name of the archive member.
491 .. attribute:: TarInfo.size
496 .. attribute:: TarInfo.mtime
498 Time of last modification.
501 .. attribute:: TarInfo.mode
506 .. attribute:: TarInfo.type
508 File type. *type* is usually one of these constants: :const:`REGTYPE`,
509 :const:`AREGTYPE`, :const:`LNKTYPE`, :const:`SYMTYPE`, :const:`DIRTYPE`,
510 :const:`FIFOTYPE`, :const:`CONTTYPE`, :const:`CHRTYPE`, :const:`BLKTYPE`,
511 :const:`GNUTYPE_SPARSE`. To determine the type of a :class:`TarInfo` object
512 more conveniently, use the ``is_*()`` methods below.
515 .. attribute:: TarInfo.linkname
517 Name of the target file name, which is only present in :class:`TarInfo` objects
518 of type :const:`LNKTYPE` and :const:`SYMTYPE`.
521 .. attribute:: TarInfo.uid
523 User ID of the user who originally stored this member.
526 .. attribute:: TarInfo.gid
528 Group ID of the user who originally stored this member.
531 .. attribute:: TarInfo.uname
536 .. attribute:: TarInfo.gname
541 .. attribute:: TarInfo.pax_headers
543 A dictionary containing key-value pairs of an associated pax extended header.
545 .. versionadded:: 2.6
547 A :class:`TarInfo` object also provides some convenient query methods:
550 .. method:: TarInfo.isfile()
552 Return :const:`True` if the :class:`Tarinfo` object is a regular file.
555 .. method:: TarInfo.isreg()
557 Same as :meth:`isfile`.
560 .. method:: TarInfo.isdir()
562 Return :const:`True` if it is a directory.
565 .. method:: TarInfo.issym()
567 Return :const:`True` if it is a symbolic link.
570 .. method:: TarInfo.islnk()
572 Return :const:`True` if it is a hard link.
575 .. method:: TarInfo.ischr()
577 Return :const:`True` if it is a character device.
580 .. method:: TarInfo.isblk()
582 Return :const:`True` if it is a block device.
585 .. method:: TarInfo.isfifo()
587 Return :const:`True` if it is a FIFO.
590 .. method:: TarInfo.isdev()
592 Return :const:`True` if it is one of character device, block device or FIFO.
600 How to extract an entire tar archive to the current working directory::
603 tar = tarfile.open("sample.tar.gz")
607 How to create an uncompressed tar archive from a list of filenames::
610 tar = tarfile.open("sample.tar", "w")
611 for name in ["foo", "bar", "quux"]:
615 How to read a gzip compressed tar archive and display some member information::
618 tar = tarfile.open("sample.tar.gz", "r:gz")
620 print tarinfo.name, "is", tarinfo.size, "bytes in size and is",
622 print "a regular file."
623 elif tarinfo.isdir():
626 print "something else."
629 How to create a tar archive with faked information::
632 tar = tarfile.open("sample.tar.gz", "w:gz")
633 for name in namelist:
634 tarinfo = tar.gettarinfo(name, "fakeproj-1.0/" + name)
637 tarinfo.uname = "johndoe"
638 tarinfo.gname = "fake"
639 tar.addfile(tarinfo, file(name))
642 The *only* way to extract an uncompressed tar stream from ``sys.stdin``::
646 tar = tarfile.open(mode="r|", fileobj=sys.stdin)
654 Supported tar formats
655 ---------------------
657 There are three tar formats that can be created with the :mod:`tarfile` module:
659 * The POSIX.1-1988 ustar format (:const:`USTAR_FORMAT`). It supports filenames
660 up to a length of at best 256 characters and linknames up to 100 characters. The
661 maximum file size is 8 gigabytes. This is an old and limited but widely
664 * The GNU tar format (:const:`GNU_FORMAT`). It supports long filenames and
665 linknames, files bigger than 8 gigabytes and sparse files. It is the de facto
666 standard on GNU/Linux systems. :mod:`tarfile` fully supports the GNU tar
667 extensions for long names, sparse file support is read-only.
669 * The POSIX.1-2001 pax format (:const:`PAX_FORMAT`). It is the most flexible
670 format with virtually no limits. It supports long filenames and linknames, large
671 files and stores pathnames in a portable way. However, not all tar
672 implementations today are able to handle pax archives properly.
674 The *pax* format is an extension to the existing *ustar* format. It uses extra
675 headers for information that cannot be stored otherwise. There are two flavours
676 of pax headers: Extended headers only affect the subsequent file header, global
677 headers are valid for the complete archive and affect all following files. All
678 the data in a pax header is encoded in *UTF-8* for portability reasons.
680 There are some more variants of the tar format which can be read, but not
683 * The ancient V7 format. This is the first tar format from Unix Seventh Edition,
684 storing only regular files and directories. Names must not be longer than 100
685 characters, there is no user/group name information. Some archives have
686 miscalculated header checksums in case of fields with non-ASCII characters.
688 * The SunOS tar extended format. This format is a variant of the POSIX.1-2001
689 pax format, but is not compatible.
696 The tar format was originally conceived to make backups on tape drives with the
697 main focus on preserving file system information. Nowadays tar archives are
698 commonly used for file distribution and exchanging archives over networks. One
699 problem of the original format (that all other formats are merely variants of)
700 is that there is no concept of supporting different character encodings. For
701 example, an ordinary tar archive created on a *UTF-8* system cannot be read
702 correctly on a *Latin-1* system if it contains non-ASCII characters. Names (i.e.
703 filenames, linknames, user/group names) containing these characters will appear
704 damaged. Unfortunately, there is no way to autodetect the encoding of an
707 The pax format was designed to solve this problem. It stores non-ASCII names
708 using the universal character encoding *UTF-8*. When a pax archive is read,
709 these *UTF-8* names are converted to the encoding of the local file system.
711 The details of unicode conversion are controlled by the *encoding* and *errors*
712 keyword arguments of the :class:`TarFile` class.
714 The default value for *encoding* is the local character encoding. It is deduced
715 from :func:`sys.getfilesystemencoding` and :func:`sys.getdefaultencoding`. In
716 read mode, *encoding* is used exclusively to convert unicode names from a pax
717 archive to strings in the local character encoding. In write mode, the use of
718 *encoding* depends on the chosen archive format. In case of :const:`PAX_FORMAT`,
719 input names that contain non-ASCII characters need to be decoded before being
720 stored as *UTF-8* strings. The other formats do not make use of *encoding*
721 unless unicode objects are used as input names. These are converted to 8-bit
722 character strings before they are added to the archive.
724 The *errors* argument defines how characters are treated that cannot be
725 converted to or from *encoding*. Possible values are listed in section
726 :ref:`codec-base-classes`. In read mode, there is an additional scheme
727 ``'utf-8'`` which means that bad characters are replaced by their *UTF-8*
728 representation. This is the default scheme. In write mode the default value for
729 *errors* is ``'strict'`` to ensure that name information is not altered