Lines Matching +full:tar +full:- +full:stream
1 :mod:`tarfile` --- Read and write tar archive files
5 :synopsis: Read and write tar-format archive files.
12 --------------
14 The :mod:`tarfile` module makes it possible to read and write tar
17 higher-level functions in :ref:`shutil <archiving-operations>`.
24 * read/write support for the POSIX.1-1988 (ustar) format.
26 * read/write support for the GNU tar format including *longname* and *longlink*
27 extensions, read-only support for all variants of the *sparse* extension
30 * read/write support for the POSIX.1-2001 (pax) format.
44 allowed, see :ref:`tarfile-objects`.
49 +------------------+---------------------------------------------+
54 +------------------+---------------------------------------------+
57 +------------------+---------------------------------------------+
59 +------------------+---------------------------------------------+
61 +------------------+---------------------------------------------+
63 +------------------+---------------------------------------------+
68 +------------------+---------------------------------------------+
72 +------------------+---------------------------------------------+
76 +------------------+---------------------------------------------+
80 +------------------+---------------------------------------------+
83 +------------------+---------------------------------------------+
85 +------------------+---------------------------------------------+
87 +------------------+---------------------------------------------+
89 +------------------+---------------------------------------------+
91 +------------------+---------------------------------------------+
110 object that processes its data as a stream of blocks. No random seeking will
116 not allow random access, see :ref:`tar-examples`. The currently
119 +-------------+--------------------------------------------+
122 | ``'r|*'`` | Open a *stream* of tar blocks for reading |
124 +-------------+--------------------------------------------+
125 | ``'r|'`` | Open a *stream* of uncompressed tar blocks |
127 +-------------+--------------------------------------------+
128 | ``'r|gz'`` | Open a gzip compressed *stream* for |
130 +-------------+--------------------------------------------+
131 | ``'r|bz2'`` | Open a bzip2 compressed *stream* for |
133 +-------------+--------------------------------------------+
134 | ``'r|xz'`` | Open an lzma compressed *stream* for |
136 +-------------+--------------------------------------------+
137 | ``'w|'`` | Open an uncompressed *stream* for writing. |
138 +-------------+--------------------------------------------+
139 | ``'w|gz'`` | Open a gzip compressed *stream* for |
141 +-------------+--------------------------------------------+
142 | ``'w|bz2'`` | Open a bzip2 compressed *stream* for |
144 +-------------+--------------------------------------------+
145 | ``'w|xz'`` | Open an lzma compressed *stream* for |
147 +-------------+--------------------------------------------+
153 The *name* parameter accepts a :term:`path-like object`.
159 Class for reading and writing tar archives. Do not use this class directly:
160 use :func:`tarfile.open` instead. See :ref:`tarfile-objects`.
165 Return :const:`True` if *name* is a tar archive file, that the :mod:`tarfile`
166 module can read. *name* may be a :class:`str`, file, or file-like object.
169 Support for file and file-like objects.
182 Is raised when a tar archive is opened, that either cannot be handled by the
194 Is raised for the limitations that are typical for stream-like :class:`TarFile`
200 Is raised for *non-fatal* errors when using :meth:`TarFile.extract`, but only if
211 Base class for members :ref:`refused <tarfile-extraction-refuse>` by
217 as :ref:`TarInfo <tarinfo-objects>`.
245 The default character encoding: ``'utf-8'`` on Windows, the value returned by
249 Each of the following constants defines a tar archive format that the
250 :mod:`tarfile` module is able to create. See section :ref:`tar-formats` for
256 POSIX.1-1988 (ustar) format.
261 GNU tar format.
266 POSIX.1-2001 (pax) format.
283 :ref:`archiving-operations`
284 Documentation of the higher-level archiving facilities provided by the
287 …`GNU tar manual, Basic Tar Format <https://www.gnu.org/software/tar/manual/html_node/Standard.html…
288 Documentation for tar archive files, including GNU tar extensions.
291 .. _tarfile-objects:
294 ---------------
296 The :class:`TarFile` object provides an interface to a tar archive. A tar
298 a header block followed by data blocks. It is possible to store a file in a tar
300 object, see :ref:`tarinfo-objects` for details.
306 :ref:`tar-examples` section for a use case.
316 *name* is the pathname of the archive. *name* may be a :term:`path-like object`.
357 See section :ref:`tar-unicode` for in-depth information.
369 The *name* parameter accepts a :term:`path-like object`.
386 to be the most up-to-date version.
405 similar to that of :program:`ls -l` is produced. If optional *members* is
435 See :ref:`tarfile-extraction-filter` for details.
436 It is recommended to set this explicitly depending on which *tar* features
447 and read the :ref:`tarfile-extraction-filter` section for details.
453 The *path* parameter accepts a :term:`path-like object`.
464 directory using *path*. *path* may be a :term:`path-like object`.
480 and read the :ref:`tarfile-extraction-filter` section for details.
489 The *path* parameter accepts a :term:`path-like object`.
514 :exc:`FilterError` exceptions. If ``2``, all *non-fatal* errors are raised
520 Custom :ref:`extraction filters <tarfile-extraction-filter>`
522 and :exc:`ExtractError` for *non-fatal* ones.
531 The :ref:`extraction filter <tarfile-extraction-filter>` used
553 it is best practice to only do so in top-level applications or
569 archive. See :ref:`tar-examples` for an example.
591 *name* may be a :term:`path-like object`. If
606 The *name* parameter accepts a :term:`path-like object`.
617 A dictionary containing key-value pairs of pax global headers.
621 .. _tarinfo-objects:
624 ---------------
645 - The :meth:`~TarFile.extract` or :meth:`~TarFile.extractall` methods will
647 - :meth:`~TarFile.addfile` will fail.
648 - :meth:`~TarFile.list` will print a placeholder string.
783 A dictionary containing key-value pairs of an associated pax extended header.
849 .. _tarfile-extraction-filter:
852 ------------------
856 The *tar* format is designed to capture all details of a UNIX-like filesystem,
858 Unfortunately, the features make it easy to create tar files that have
859 unintended -- and possibly malicious -- effects when extracted.
860 For example, extracting a tar file can overwrite arbitrary files in various
881 * the string ``'tar'``: Honor most *tar*-specific features (i.e. features of
882 UNIX-like filesystems), but block features that are very likely to be
885 * the string ``'data'``: Ignore or block most features specific to UNIX-like
886 filesystems. Intended for extracting cross-platform data archives.
900 :ref:`TarInfo <tarinfo-objects>` describing the member and the destination
904 filter(/, member: TarInfo, path: str) -> TarInfo | None
910 - return a :class:`TarInfo` object which will be used instead of the metadata
912 - return ``None``, in which case the member will be skipped, or
913 - raise an exception to abort the operation or skip the member,
921 The pre-defined, named filters are available as functions, so they can be
932 Implements the ``'tar'`` filter.
934 - Strip leading slashes (``/`` and :attr:`os.sep`) from filenames.
935 - :ref:`Refuse <tarfile-extraction-refuse>` to extract files with absolute
939 - :ref:`Refuse <tarfile-extraction-refuse>` to extract files whose absolute
942 - Clear high mode bits (setuid, setgid, sticky) and group/other write bits
952 - :ref:`Refuse <tarfile-extraction-refuse>` to extract links (hard or soft)
961 - :ref:`Refuse <tarfile-extraction-refuse>` to extract device files
965 - For regular files, including hard links:
967 - Set the owner read and write permissions
969 - Remove the group & other executable permission
973 - For other files (directories), set ``mode`` to ``None``, so
975 - Set user and group info (``uid``, ``gid``, ``uname``, ``gname``)
981 .. _tarfile-extraction-refuse:
998 Among other issues, the pre-defined filters do not prevent denial-of-service
1004 to prevent e.g. exploiting pre-existing links, and to make it easier to
1006 * When working with untrusted data, use external (e.g. OS-level) limits on
1008 * Check filenames against an allow-list of characters
1012 execute when you “click on them”, or extension-less files like Windows special device names).
1015 * Check for files that would be shadowed on case-insensitive filesystems.
1019 * Tar files may contain multiple versions of the same file.
1079 tar.extractall(path, filter=filter_func)
1098 .. _tarfile-commandline:
1102 Command-Line Interface
1103 ----------------------
1107 The :mod:`tarfile` module provides a simple command-line interface to interact
1108 with tar archives.
1110 If you want to create a new tar archive, specify its name after the :option:`-c`
1113 .. code-block:: shell-session
1115 $ python -m tarfile -c monty.tar spam.txt eggs.txt
1119 .. code-block:: shell-session
1121 $ python -m tarfile -c monty.tar life-of-brian_1979/
1123 If you want to extract a tar archive into the current directory, use
1124 the :option:`-e` option:
1126 .. code-block:: shell-session
1128 $ python -m tarfile -e monty.tar
1130 You can also extract a tar archive into a different directory by passing the
1133 .. code-block:: shell-session
1135 $ python -m tarfile -e monty.tar other-dir/
1137 For a list of the files in a tar archive, use the :option:`-l` option:
1139 .. code-block:: shell-session
1141 $ python -m tarfile -l monty.tar
1144 Command-line options
1147 .. cmdoption:: -l <tarfile>
1148 --list <tarfile>
1152 .. cmdoption:: -c <tarfile> <source1> ... <sourceN>
1153 --create <tarfile> <source1> ... <sourceN>
1157 .. cmdoption:: -e <tarfile> [<output_dir>]
1158 --extract <tarfile> [<output_dir>]
1162 .. cmdoption:: -t <tarfile>
1163 --test <tarfile>
1167 .. cmdoption:: -v, --verbose
1171 .. cmdoption:: --filter <filtername>
1173 Specifies the *filter* for ``--extract``.
1174 See :ref:`tarfile-extraction-filter` for details.
1175 Only string names are accepted (that is, ``fully_trusted``, ``tar``,
1180 .. _tar-examples:
1183 --------
1185 How to extract an entire tar archive to the current working directory::
1188 tar = tarfile.open("sample.tar.gz")
1189 tar.extractall()
1190 tar.close()
1192 How to extract a subset of a tar archive with :meth:`TarFile.extractall` using
1203 tar = tarfile.open("sample.tar.gz")
1204 tar.extractall(members=py_files(tar))
1205 tar.close()
1207 How to create an uncompressed tar archive from a list of filenames::
1210 tar = tarfile.open("sample.tar", "w")
1212 tar.add(name)
1213 tar.close()
1218 with tarfile.open("sample.tar", "w") as tar:
1220 tar.add(name)
1222 How to read a gzip compressed tar archive and display some member information::
1225 tar = tarfile.open("sample.tar.gz", "r:gz")
1226 for tarinfo in tar:
1234 tar.close()
1244 tar = tarfile.open("sample.tar.gz", "w:gz")
1245 tar.add("foo", filter=reset)
1246 tar.close()
1249 .. _tar-formats:
1251 Supported tar formats
1252 ---------------------
1254 There are three tar formats that can be created with the :mod:`tarfile` module:
1256 * The POSIX.1-1988 ustar format (:const:`USTAR_FORMAT`). It supports filenames
1261 * The GNU tar format (:const:`GNU_FORMAT`). It supports long filenames and
1263 standard on GNU/Linux systems. :mod:`tarfile` fully supports the GNU tar
1264 extensions for long names, sparse file support is read-only.
1266 * The POSIX.1-2001 pax format (:const:`PAX_FORMAT`). It is the most flexible
1268 files and stores pathnames in a portable way. Modern tar implementations,
1269 including GNU tar, bsdtar/libarchive and star, fully support extended *pax*
1278 All the data in a pax header is encoded in *UTF-8* for portability reasons.
1280 There are some more variants of the tar format which can be read, but not
1283 * The ancient V7 format. This is the first tar format from Unix Seventh Edition,
1286 miscalculated header checksums in case of fields with non-ASCII characters.
1288 * The SunOS tar extended format. This format is a variant of the POSIX.1-2001
1291 .. _tar-unicode:
1294 --------------
1296 The tar format was originally conceived to make backups on tape drives with the
1297 main focus on preserving file system information. Nowadays tar archives are
1301 example, an ordinary tar archive created on a *UTF-8* system cannot be read
1302 correctly on a *Latin-1* system if it contains non-*ASCII* characters. Textual
1305 pax format was designed to solve this problem. It stores non-ASCII metadata
1306 using the universal character encoding *UTF-8*.
1318 converted. Possible values are listed in section :ref:`error-handlers`.
1320 file system calls, see :ref:`os-filenames`.
1323 because all the metadata is stored using *UTF-8*. *encoding* is only used in