Update documentation and add new guides
This commit updates multiple documentation files to improve clarity, remove emojis, and add new sections. Key changes include the addition of 'development.md' and 'performance.md', updates to the quickstart guide, and enhancements to the API and CLI documentation. These changes aim to provide better guidance for users and contributors.
This commit is contained in:
xixu-me committed
1 parent
42180498e0
commit
44843e0035
8 files changed
+916
-137
No files matched your search
+123
-87
@@ -19,7 +19,7 @@ This guide will get you up and running with tzst in just a few minutes.
|
||||
|
||||
Choose your preferred installation method:
|
||||
|
||||
### Option 1: PyPI (Recommended)
|
||||
### Option 1: PyPI
|
||||
|
||||
```bash
|
||||
pip install tzst
|
||||
@@ -31,16 +31,33 @@ Download the appropriate executable from [GitHub Releases](https://github.com/xi
|
||||
|
||||
| Platform | Architecture | Download |
|
||||
|----------|--------------|----------|
|
||||
| **🐧 Linux** | x86_64 | `tzst-v{version}-linux-x86_64.zip` |
|
||||
| **🐧 Linux** | ARM64 | `tzst-v{version}-linux-aarch64.zip` |
|
||||
| **🪟 Windows** | x64 | `tzst-v{version}-windows-amd64.zip` |
|
||||
| **🪟 Windows** | ARM64 | `tzst-v{version}-windows-arm64.zip` |
|
||||
| **🍎 macOS** | Intel | `tzst-v{version}-macos-x86_64.zip` |
|
||||
| **🍎 macOS** | Apple Silicon | `tzst-v{version}-macos-arm64.zip` |
|
||||
| **Linux** | x86_64 | `tzst-v{version}-linux-x86_64.zip` |
|
||||
| **Linux** | ARM64 | `tzst-v{version}-linux-aarch64.zip` |
|
||||
| **Windows** | x64 | `tzst-v{version}-windows-amd64.zip` |
|
||||
| **Windows** | ARM64 | `tzst-v{version}-windows-arm64.zip` |
|
||||
| **macOS** | Intel | `tzst-v{version}-macos-x86_64.zip` |
|
||||
| **macOS** | Apple Silicon | `tzst-v{version}-macos-arm64.zip` |
|
||||
|
||||
Extract the archive and add the executable to your PATH.
|
||||
|
||||
### Option 3: From Source
|
||||
### Option 3: Using uvx (No Installation)
|
||||
|
||||
Run tzst directly without installation using [uvx](https://docs.astral.sh/uv/):
|
||||
|
||||
```bash
|
||||
uvx tzst --help
|
||||
uvx tzst a archive.tzst file1.txt file2.txt directory/
|
||||
uvx tzst x archive.tzst
|
||||
```
|
||||
|
||||
This option is perfect for:
|
||||
|
||||
- **One-time usage** - No permanent installation needed
|
||||
- **Testing** - Try tzst without committing to installation
|
||||
- **CI/CD pipelines** - Use tzst in automated workflows
|
||||
- **Isolated environments** - Avoid dependency conflicts
|
||||
|
||||
### Option 4: From Source
|
||||
|
||||
```bash
|
||||
git clone https://github.com/xixu-me/tzst.git
|
||||
@@ -54,6 +71,8 @@ pip install .
|
||||
|
||||
### Command Line Interface
|
||||
|
||||
> **Note**: Download the [standalone binary](installation) for the best performance and no Python dependency. Alternatively, use `uvx tzst` for running without installation. See [uv documentation](https://docs.astral.sh/uv/) for details.
|
||||
|
||||
The CLI provides four main operations:
|
||||
|
||||
```bash
|
||||
@@ -70,6 +89,25 @@ tzst l archive.tzst
|
||||
tzst t archive.tzst
|
||||
```
|
||||
|
||||
### Command Reference
|
||||
|
||||
| Command | Aliases | Description | Streaming Support |
|
||||
|---------|---------|-------------|-------------------|
|
||||
| `a` | `add`, `create` | Create or add to archive | N/A |
|
||||
| `x` | `extract` | Extract with full paths | `--streaming` |
|
||||
| `e` | `extract-flat` | Extract without directory structure | `--streaming` |
|
||||
| `l` | `list` | List archive contents | `--streaming` |
|
||||
| `t` | `test` | Test archive integrity | `--streaming` |
|
||||
|
||||
### CLI Options
|
||||
|
||||
- `-v, --verbose`: Enable verbose output
|
||||
- `-o, --output DIR`: Specify output directory (extract commands)
|
||||
- `-l, --level LEVEL`: Set compression level 1-22 (create command)
|
||||
- `--streaming`: Enable streaming mode for memory-efficient processing
|
||||
- `--filter FILTER`: Security filter for extraction (data/tar/fully_trusted)
|
||||
- `--no-atomic`: Disable atomic file operations (not recommended)
|
||||
|
||||
#### Create Archives
|
||||
|
||||
```bash
|
||||
@@ -183,6 +221,29 @@ extract_archive("untrusted.tzst", "safe-output/", filter="data")
|
||||
extract_archive("trusted.tzst", "output/", filter="tar")
|
||||
```
|
||||
|
||||
### Security Filters
|
||||
|
||||
tzst provides three security filter options for extraction:
|
||||
|
||||
```python
|
||||
from tzst import extract_archive
|
||||
|
||||
# Extract with maximum security (default)
|
||||
extract_archive("archive.tzst", "output/", filter="data")
|
||||
|
||||
# Extract with standard tar compatibility
|
||||
extract_archive("archive.tzst", "output/", filter="tar")
|
||||
|
||||
# Extract with full trust (dangerous - only for trusted archives)
|
||||
extract_archive("archive.tzst", "output/", filter="fully_trusted")
|
||||
```
|
||||
|
||||
**Security Filter Options:**
|
||||
|
||||
- `data` (default): Most secure. Blocks dangerous files, absolute paths, and paths outside extraction directory
|
||||
- `tar`: Standard tar compatibility. Blocks absolute paths and directory traversal
|
||||
- `fully_trusted`: No security restrictions. Only use with completely trusted archives
|
||||
|
||||
### Conflict Resolution
|
||||
|
||||
```python
|
||||
@@ -211,6 +272,56 @@ create_archive("best.tzst", files, compression_level=22) # Best compression
|
||||
extract_archive("huge-archive.tzst", "output/", streaming=True)
|
||||
```
|
||||
|
||||
### Streaming Mode
|
||||
|
||||
For large archives (>100MB), use streaming mode to reduce memory usage:
|
||||
|
||||
```python
|
||||
# Memory-efficient operations
|
||||
with TzstArchive("large-archive.tzst", "r", streaming=True) as archive:
|
||||
contents = archive.list()
|
||||
archive.extractall("output/")
|
||||
is_valid = archive.test()
|
||||
```
|
||||
|
||||
**Note**: Streaming mode has limitations - you cannot extract specific files or use random access operations.
|
||||
|
||||
### File Extensions
|
||||
|
||||
The library automatically handles file extensions with intelligent normalization:
|
||||
|
||||
- `.tzst` - Primary extension for tar+zstandard archives
|
||||
- `.tar.zst` - Alternative standard extension
|
||||
- Auto-detection when opening existing archives
|
||||
- Automatic extension addition when creating archives
|
||||
|
||||
```python
|
||||
from tzst import create_archive
|
||||
|
||||
# These all create valid archives
|
||||
create_archive("backup.tzst", files) # Creates backup.tzst
|
||||
create_archive("backup.tar.zst", files) # Creates backup.tar.zst
|
||||
create_archive("backup", files) # Creates backup.tzst
|
||||
create_archive("backup.txt", files) # Creates backup.tzst (normalized)
|
||||
```
|
||||
|
||||
### Atomic Operations
|
||||
|
||||
All file creation operations use atomic file operations by default:
|
||||
|
||||
- Archives created in temporary files first, then atomically moved
|
||||
- Automatic cleanup if process is interrupted
|
||||
- No risk of corrupted or incomplete archives
|
||||
- Cross-platform compatibility
|
||||
|
||||
```python
|
||||
# Atomic operations enabled by default
|
||||
create_archive("important.tzst", files) # Safe from interruption
|
||||
|
||||
# Can be disabled if needed (not recommended)
|
||||
create_archive("test.tzst", files, use_temp_file=False)
|
||||
```
|
||||
|
||||
## Error Handling
|
||||
|
||||
```python
|
||||
@@ -250,80 +361,6 @@ with TzstArchive("data.tzst", "r") as archive:
|
||||
members = archive.getmembers()
|
||||
```
|
||||
|
||||
## Important Concepts
|
||||
|
||||
### Compression Levels
|
||||
|
||||
tzst supports compression levels from 1 to 22:
|
||||
|
||||
- **Level 1-3**: Fast compression, larger files (good for temporary archives)
|
||||
- **Level 4-6**: Balanced compression and speed (recommended for most use cases)
|
||||
- **Level 7-15**: Higher compression, slower (good for long-term storage)
|
||||
- **Level 16-22**: Maximum compression, much slower (for size-critical applications)
|
||||
|
||||
```python
|
||||
# Fast compression
|
||||
create_archive("temp.tzst", files, compression_level=1)
|
||||
|
||||
# Balanced (default)
|
||||
create_archive("backup.tzst", files, compression_level=3)
|
||||
|
||||
# High compression
|
||||
create_archive("archive.tzst", files, compression_level=9)
|
||||
|
||||
# Maximum compression
|
||||
create_archive("minimal.tzst", files, compression_level=22)
|
||||
```
|
||||
|
||||
### Security Filters
|
||||
|
||||
tzst provides extraction filters to protect against malicious archives:
|
||||
|
||||
```python
|
||||
# Safe data extraction (default, recommended)
|
||||
extract_archive("archive.tzst", "output/", filter="data")
|
||||
|
||||
# Preserve more tar features but still secure
|
||||
extract_archive("archive.tzst", "output/", filter="tar")
|
||||
|
||||
# Full trust mode (use only with trusted archives)
|
||||
extract_archive("archive.tzst", "output/", filter="fully_trusted")
|
||||
```
|
||||
|
||||
### Streaming Mode
|
||||
|
||||
For large archives (>100MB), use streaming mode to reduce memory usage:
|
||||
|
||||
```python
|
||||
# Memory-efficient operations
|
||||
with TzstArchive("large-archive.tzst", "r", streaming=True) as archive:
|
||||
contents = archive.list()
|
||||
archive.extractall("output/")
|
||||
is_valid = archive.test()
|
||||
```
|
||||
|
||||
**Note**: Streaming mode has limitations - you cannot extract specific files or use random access operations.
|
||||
|
||||
### Handling File Conflicts
|
||||
|
||||
Handle file conflicts during extraction:
|
||||
|
||||
```python
|
||||
from tzst import ConflictResolution
|
||||
|
||||
# Skip existing files
|
||||
extract_archive("archive.tzst", "output/",
|
||||
conflict_resolution=ConflictResolution.SKIP)
|
||||
|
||||
# Replace all existing files
|
||||
extract_archive("archive.tzst", "output/",
|
||||
conflict_resolution=ConflictResolution.REPLACE_ALL)
|
||||
|
||||
# Auto-rename conflicting files
|
||||
extract_archive("archive.tzst", "output/",
|
||||
conflict_resolution=ConflictResolution.AUTO_RENAME_ALL)
|
||||
```
|
||||
|
||||
## Common Patterns
|
||||
|
||||
### Backup Script
|
||||
@@ -359,17 +396,16 @@ def verify_archive(archive_path):
|
||||
|
||||
# Test integrity
|
||||
if not test_archive(archive_path):
|
||||
print("❌ Archive is corrupted!")
|
||||
print("Archive is corrupted!")
|
||||
return False
|
||||
|
||||
# List contents
|
||||
contents = list_archive(archive_path, verbose=True)
|
||||
total_size = sum(item['size'] for item in contents if item['is_file'])
|
||||
file_count = sum(1 for item in contents if item['is_file'])
|
||||
|
||||
print(f"✅ Archive is valid")
|
||||
print(f"📁 Files: {file_count}")
|
||||
print(f"📦 Total size: {total_size / 1024 / 1024:.1f} MB")
|
||||
print(f"Archive is valid")
|
||||
print(f"Files: {file_count}")
|
||||
print(f"Total size: {total_size / 1024 / 1024:.1f} MB")
|
||||
|
||||
return True
|
||||
```
|
||||
|
||||
Reference in new issue
Block a user