> [!TIP] > 欢迎加入“Xget 开源与 AI 交流群”，一起交流开源项目、AI 应用、工程实践、效率工具和独立开发；如果你也在做产品、写代码、折腾项目或者对开源和 AI 感兴趣，欢迎[**进群**](https://file.xi-xu.me/QR%20Codes/%E7%BE%A4%E4%BA%8C%E7%BB%B4%E7%A0%81.png)认识更多认真做事、乐于分享的朋友。


[![codecov](https://codecov.io/gh/xixu-me/tzst/graph/badge.svg?token=2AIN1559WU)](https://codecov.io/gh/xixu-me/tzst) [![CodeQL](https://github.com/xixu-me/tzst/actions/workflows/github-code-scanning/codeql/badge.svg)](https://github.com/xixu-me/tzst/actions/workflows/github-code-scanning/codeql) [![CI/CD](https://github.com/xixu-me/tzst/actions/workflows/ci.yml/badge.svg)](https://github.com/xixu-me/tzst/actions/workflows/ci.yml) [![PyPI - Version](https://img.shields.io/pypi/v/tzst)](https://pypi.org/project/tzst/) [![PyPI - Downloads](https://img.shields.io/pypi/dm/tzst)](https://pypistats.org/packages/tzst) [![GitHub License](https://img.shields.io/github/license/xixu-me/tzst)](LICENSE) [![Sponsor](https://img.shields.io/badge/Sponsor-violet)](https://xi-xu.me/#sponsorships) [![Documentation](https://img.shields.io/badge/Documentation-blue)](https://tzst.xi-xu.me) **🇺🇸 English** | [🇨🇳 汉语](./README.zh.md) | [🇪🇸 español](./README.es.md) | [🇯🇵 日本語](./README.ja.md) | [🇦🇪 العربية](./README.ar.md) | [🇷🇺 русский](./README.ru.md) | [🇩🇪 Deutsch](./README.de.md) | [🇫🇷 français](./README.fr.md) | [🇰🇷 한국어](./README.ko.md) | [🇧🇷 português](./README.pt.md) **tzst** is a Python 3.12+ library and CLI for creating, extracting, listing, and testing `.tzst` and `.tar.zst` archives. It combines tar compatibility, Zstandard compression, streaming support, atomic writes, and safe-by-default extraction in a compact, production-ready interface. > [!NOTE] > In-depth technical article: **[Deep Dive into tzst: A Modern Python Archiving Library Based on Zstandard](https://blog.xi-xu.me/2025/11/01/deep-dive-into-tzst-en.html)**. ## Features - **High Compression**: Zstandard compression for excellent compression ratios and speed - **Tar Compatibility**: Creates standard tar archives compressed with Zstandard - **Command Line Interface**: Intuitive CLI with streaming support and comprehensive options - **Python API**: Clean, Pythonic API for programmatic use - **Cross-Platform**: Works on Windows, macOS, and Linux - **Multiple Extensions**: Supports both `.tzst` and `.tar.zst` extensions - **Memory Efficient**: Streaming mode for handling large archives with minimal memory usage - **Atomic Operations**: Safe file operations with automatic cleanup on interruption - **Secure by Default**: Uses the 'data' filter for maximum security during extraction - **Enhanced Error Handling**: Clear error messages with helpful alternatives ## Installation ### From GitHub Releases Download standalone executables that don't require Python installation: #### Supported Platforms | Platform | Architecture | File | |----------|-------------|------| | **Linux** | x86_64 | `tzst-{version}-linux-amd64.zip` | | **Linux** | ARM64 | `tzst-{version}-linux-arm64.zip` | | **Windows** | x64 | `tzst-{version}-windows-amd64.zip` | | **Windows** | ARM64 | `tzst-{version}-windows-arm64.zip` | | **macOS** | Intel | `tzst-{version}-darwin-amd64.zip` | | **macOS** | Apple Silicon | `tzst-{version}-darwin-arm64.zip` | #### Installation Steps 1. **Download** the appropriate archive for your platform from the [latest releases page](https://github.com/xixu-me/tzst/releases/latest) 2. **Extract** the archive to get the `tzst` executable (or `tzst.exe` on Windows) 3. **Move** the executable to a directory in your PATH: - **Linux/macOS**: `sudo mv tzst /usr/local/bin/` - **Windows**: Add the directory containing `tzst.exe` to your PATH environment variable 4. **Verify** installation: `tzst --help` #### Benefits of Binary Installation - **No Python required** - Standalone executable - **Faster startup** - No Python interpreter overhead - **Easy deployment** - Single file distribution - **Consistent behavior** - Bundled dependencies ### From PyPI Using pip: ```bash pip install tzst ``` Or using uv (recommended): ```bash uv tool install tzst ``` ### From Source ```bash git clone https://github.com/xixu-me/tzst.git cd tzst pip install . ``` ### Development Installation This project uses modern Python packaging standards: ```bash git clone https://github.com/xixu-me/tzst.git cd tzst pip install -e .[dev] ``` ## Quick Start ### Command Line Usage ```bash # Create an archive tzst a archive.tzst file1.txt file2.txt directory/ # Extract an archive tzst x archive.tzst # List archive contents tzst l archive.tzst # Test archive integrity tzst t archive.tzst ``` ### Python API Usage ```python from tzst import create_archive, extract_archive, list_archive # Create an archive create_archive("archive.tzst", ["file1.txt", "file2.txt", "directory/"]) # Extract an archive extract_archive("archive.tzst", "output_directory/") # List archive contents contents = list_archive("archive.tzst", verbose=True) for item in contents: print(f"{item['name']}: {item['size']} bytes") ``` ## Command Line Interface ### Archive Operations #### Create Archive ```bash # Basic usage tzst a archive.tzst file1.txt file2.txt # With compression level (1-22, default: 3) tzst a archive.tzst files/ -l 15 # Alternative commands tzst add archive.tzst files/ tzst create archive.tzst files/ ``` #### Extract Archive ```bash # Extract with full directory structure tzst x archive.tzst # Extract to specific directory tzst x archive.tzst -o output/ # Extract specific files tzst x archive.tzst file1.txt dir/file2.txt # Extract without directory structure (flat) tzst e archive.tzst -o output/ # Use streaming mode for large archives tzst x archive.tzst --streaming -o output/ ``` #### List Contents ```bash # Simple listing tzst l archive.tzst # Verbose listing with details tzst l archive.tzst -v # Use streaming mode for large archives tzst l archive.tzst --streaming -v ``` #### Test Integrity ```bash # Test archive integrity tzst t archive.tzst # Test with streaming mode tzst t archive.tzst --streaming ``` ### Command Reference | Command | Aliases | Description | Streaming Support | |---------|---------|-------------|-------------------| | `a` | `add`, `create` | Create or add to archive | N/A | | `x` | `extract` | Extract with full paths | ✓ `--streaming` | | `e` | `extract-flat` | Extract without directory structure | ✓ `--streaming` | | `l` | `list` | List archive contents | ✓ `--streaming` | | `t` | `test` | Test archive integrity | ✓ `--streaming` | ### CLI Options - `-v, --verbose`: Enable verbose output - `-o, --output DIR`: Specify output directory (extract commands) - `-l, --level LEVEL`: Set compression level 1-22 (create command) - `--streaming`: Enable streaming mode for memory-efficient processing - `--filter FILTER`: Security filter for extraction (data/tar/fully_trusted) - `--no-atomic`: Disable atomic file operations (not recommended) ### Security Filters ```bash # Extract with maximum security (default) tzst x archive.tzst --filter data # Extract with standard tar compatibility tzst x archive.tzst --filter tar # Extract with full trust (dangerous - only for trusted archives) tzst x archive.tzst --filter fully_trusted ``` **Security Filter Options:** - `data` (default): Most secure. Blocks dangerous files, absolute paths, and paths outside extraction directory - `tar`: Standard tar compatibility. Blocks absolute paths and directory traversal - `fully_trusted`: No security restrictions. Only use with completely trusted archives ## Python API ### TzstArchive Class ```python from tzst import TzstArchive # Create a new archive with TzstArchive("archive.tzst", "w", compression_level=5) as archive: archive.add("file.txt") archive.add("directory/", recursive=True) # Read an existing archive with TzstArchive("archive.tzst", "r") as archive: # List contents contents = archive.list(verbose=True) # Extract with security filter archive.extract("file.txt", "output/", filter="data") # Test integrity is_valid = archive.test() # For large archives, use streaming mode with TzstArchive("large_archive.tzst", "r", streaming=True) as archive: archive.extract(path="output/") ``` **Important Limitations:** - **Append Mode Not Supported**: Create multiple archives or recreate the entire archive instead ### Convenience Functions #### create_archive() ```python from tzst import create_archive # Create with atomic operations (default) create_archive( archive_path="backup.tzst", files=["documents/", "photos/", "config.txt"], compression_level=10 ) ``` #### extract_archive() ```python from tzst import extract_archive # Extract with security (default: 'data' filter) extract_archive("backup.tzst", "restore/") # Extract specific files extract_archive("backup.tzst", "restore/", members=["config.txt"]) # Flatten directory structure extract_archive("backup.tzst", "restore/", flatten=True) # Use streaming for large archives extract_archive("large_backup.tzst", "restore/", streaming=True) ``` #### list_archive() ```python from tzst import list_archive # Simple listing files = list_archive("backup.tzst") # Detailed listing files = list_archive("backup.tzst", verbose=True) # Streaming for large archives files = list_archive("large_backup.tzst", streaming=True) ``` #### test_archive() ```python from tzst import test_archive # Basic integrity test if test_archive("backup.tzst"): print("Archive is valid") # Test with streaming if test_archive("large_backup.tzst", streaming=True): print("Large archive is valid") ``` ## Advanced Features ### File Extensions The library automatically handles file extensions with intelligent normalization: - `.tzst` - Primary extension for tar+zstandard archives - `.tar.zst` - Alternative standard extension - Auto-detection when opening existing archives - Automatic extension addition when creating archives ```python # These all create valid archives create_archive("backup.tzst", files) # Creates backup.tzst create_archive("backup.tar.zst", files) # Creates backup.tar.zst create_archive("backup", files) # Creates backup.tzst create_archive("backup.txt", files) # Creates backup.tzst (normalized) ``` ### Compression Levels Zstandard compression levels range from 1 (fastest) to 22 (best compression): - **Level 1-3**: Fast compression, larger files - **Level 3** (default): Good balance of speed and compression - **Level 10-15**: Better compression, slower - **Level 20-22**: Maximum compression, much slower ### Streaming Mode Use streaming mode for memory-efficient processing of large archives: **Benefits:** - Significantly reduced memory usage - Better performance for archives that don't fit in memory - Automatic cleanup of resources **When to Use:** - Archives larger than 100MB - Limited memory environments - Processing archives with many large files ```python # Example: Processing a large backup archive from tzst import extract_archive, list_archive, test_archive large_archive = "backup_500gb.tzst" # Memory-efficient operations is_valid = test_archive(large_archive, streaming=True) contents = list_archive(large_archive, streaming=True, verbose=True) extract_archive(large_archive, "restore/", streaming=True) ``` ### Atomic Operations All file creation operations use atomic file operations by default: - Archives created in temporary files first, then atomically moved - Automatic cleanup if process is interrupted - No risk of corrupted or incomplete archives - Cross-platform compatibility ```python # Atomic operations enabled by default create_archive("important.tzst", files) # Safe from interruption # Can be disabled if needed (not recommended) create_archive("test.tzst", files, use_temp_file=False) ``` ### Error Handling ```python from tzst import TzstArchive from tzst.exceptions import ( TzstError, TzstArchiveError, TzstCompressionError, TzstDecompressionError, TzstFileNotFoundError ) try: with TzstArchive("archive.tzst", "r") as archive: archive.extract() except TzstDecompressionError: print("Failed to decompress archive") except TzstFileNotFoundError: print("Archive file not found") except KeyboardInterrupt: print("Operation interrupted by user") # Cleanup handled automatically ``` ## Performance and Comparison ### Performance Tips 1. **Compression levels**: Level 3 is optimal for most use cases 2. **Streaming**: Use for archives larger than 100MB 3. **Batch operations**: Add multiple files in single session 4. **File types**: Already compressed files won't compress much further ### vs Other Tools **vs tar + gzip:** - Better compression ratios - Faster decompression - Modern algorithm **vs tar + xz:** - Significantly faster compression - Similar compression ratios - Better speed/compression trade-off **vs zip:** - Better compression - Preserves Unix permissions and metadata - Better streaming support ## Requirements - Python 3.12 or higher (tested on 3.12-3.14) - zstandard >= 0.19.0 ## Development ### Setting up Development Environment This project uses modern Python packaging standards: ```bash git clone https://github.com/xixu-me/tzst.git cd tzst pip install -e .[dev] ``` ### Running Tests ```bash # Run tests with coverage pytest --cov=tzst --cov-report=html # Or use the simpler command (coverage settings are in pyproject.toml) pytest ``` ### Code Quality ```bash # Check code quality ruff check src tests # Format code ruff format src tests ``` ## Contributing We welcome contributions! Please read our [Contributing Guide](CONTRIBUTING.md) for: - Development setup and project structure - Code style guidelines and best practices - Testing requirements and writing tests - Pull request process and review workflow ### Quick Start for Contributors ```bash git clone https://github.com/xixu-me/tzst.git cd tzst pip install -e .[dev] python -m pytest tests/ ``` ### Types of Contributions Welcome - **Bug fixes** - Fix issues in existing functionality - **Features** - Add new capabilities to the library - **Documentation** - Improve or add documentation - **Tests** - Add or improve test coverage - **Performance** - Optimize existing code - **Security** - Address security vulnerabilities ## Acknowledgments - [Meta Zstandard](https://github.com/facebook/zstd) for the excellent compression algorithm - [python-zstandard](https://github.com/indygreg/python-zstandard) for Python bindings - The Python community for inspiration and feedback ## License Copyright © [Xi Xu](https://xi-xu.me). All rights reserved. Licensed under the [BSD 3-Clause](LICENSE) license.