Compare commits
| Author | SHA1 | Date | |
|---|---|---|---|
 xixu-me
|
e65a56c00b | ||
|
xixu-me
|
d300760799 | ||
|
xixu-me
|
a5f342f556 | ||
|
xixu-me
|
cfb8290dea | ||
|
xixu-me
|
654d9f1666 | ||
|
xixu-me
|
172ec6cb29 | ||
|
xixu-me
|
94b7a5b31a | ||
|
xixu-me
|
1d98650d3b | ||
|
xixu-me
|
1cad021539 | ||
|
xixu-me
|
aef5d90a3a | ||
|
xixu-me
|
8aa893827b | ||
|
xixu-me
|
0ecbdb3ea0 | ||
|
xixu-me
|
a4ddd106f8 | ||
|
xixu-me
|
35df0a4a44 | ||
|
xixu-me
|
4e0ec638cc | ||
|
xixu-me
|
da2baa46c5 | ||
|
xixu-me
|
b1c165f473 | ||
|
xixu-me
|
028bb3f1b9 | ||
|
xixu-me
|
86582c9d49 | ||
|
xixu-me
|
44c8b20e9f | ||
|
xixu-me
|
b83d320afc | ||
|
xixu-me
|
de7d3d735e | ||
|
xixu-me
|
260ba1f66e | ||
|
xixu-me
|
ed4ac6c9fb | ||
|
xixu-me
|
0216fcc128 | ||
|
xixu-me
|
8cfc1308d6 | ||
 xixu-meandCopilot Autofix powered by AI
|
8e41740e0f | ||
|
xixu-me
|
4b59c6cad5 | ||
|
xixu-me
|
17c530ef21 | ||
|
xixu-me
|
3d99d7a6db | ||
|
xixu-me
|
d2095bb7b4 | ||
|
xixu-me
|
ec04f2935e | ||
|
xixu-me
|
22dc90259b | ||
|
xixu-me
|
97243e6a27 | ||
|
xixu-me
|
dc12d122c4 | ||
|
xixu-me
|
6693713143 | ||
|
xixu-me
|
459b695af0 | ||
|
xixu-me
|
2a3ccefb21 | ||
|
xixu-me
|
acd9ced29e | ||
|
dependabot[bot]
|
47a515536f | ||
|
xixu-me
|
79d204fc92 | ||
|
xixu-me
|
07ea3092e0 | ||
|
xixu-me
|
003ade93d3 | ||
|
dependabot[bot]
|
8c7f5e8673 |
No files matched your search
@@ -0,0 +1,33 @@
|
||||
coverage:
|
||||
status:
|
||||
project:
|
||||
default:
|
||||
target: auto
|
||||
threshold: 1%
|
||||
informational: true
|
||||
patch:
|
||||
default:
|
||||
target: auto
|
||||
threshold: 1%
|
||||
informational: true
|
||||
|
||||
# Report coverage for all files, even if not touched in PR
|
||||
report:
|
||||
exclude_labels:
|
||||
- "skip-coverage"
|
||||
|
||||
comment:
|
||||
layout: "reach,diff,flags,tree"
|
||||
behavior: default
|
||||
require_changes: false
|
||||
|
||||
# Files to ignore in coverage
|
||||
ignore:
|
||||
- "tests/*"
|
||||
- "**/__pycache__/*"
|
||||
- "**/htmlcov/*"
|
||||
- "setup.py"
|
||||
|
||||
# GitHub Checks integration
|
||||
github_checks:
|
||||
annotations: true
|
||||
@@ -1,38 +0,0 @@
|
||||
---
|
||||
name: Bug report
|
||||
about: Create a report to help us improve
|
||||
title: ''
|
||||
labels: ''
|
||||
assignees: ''
|
||||
|
||||
---
|
||||
|
||||
**Describe the bug**
|
||||
A clear and concise description of what the bug is.
|
||||
|
||||
**To Reproduce**
|
||||
Steps to reproduce the behavior:
|
||||
1. Go to '...'
|
||||
2. Click on '....'
|
||||
3. Scroll down to '....'
|
||||
4. See error
|
||||
|
||||
**Expected behavior**
|
||||
A clear and concise description of what you expected to happen.
|
||||
|
||||
**Screenshots**
|
||||
If applicable, add screenshots to help explain your problem.
|
||||
|
||||
**Desktop (please complete the following information):**
|
||||
- OS: [e.g. iOS]
|
||||
- Browser [e.g. chrome, safari]
|
||||
- Version [e.g. 22]
|
||||
|
||||
**Smartphone (please complete the following information):**
|
||||
- Device: [e.g. iPhone6]
|
||||
- OS: [e.g. iOS8.1]
|
||||
- Browser [e.g. stock browser, safari]
|
||||
- Version [e.g. 22]
|
||||
|
||||
**Additional context**
|
||||
Add any other context about the problem here.
|
||||
@@ -0,0 +1,180 @@
|
||||
name: Bug Report
|
||||
description: Report a bug or unexpected behavior in tzst
|
||||
title: "[BUG] "
|
||||
labels: ["bug", "needs-triage"]
|
||||
assignees: []
|
||||
body:
|
||||
- type: markdown
|
||||
attributes:
|
||||
value: |
|
||||
Thank you for taking the time to report a bug! Please provide as much detail as possible to help us reproduce and fix the issue.
|
||||
|
||||
- type: checkboxes
|
||||
id: search
|
||||
attributes:
|
||||
label: Search for existing issues
|
||||
description: Please search existing issues before creating a new one.
|
||||
options:
|
||||
- label: I have searched for existing issues and didn't find a duplicate
|
||||
required: true
|
||||
|
||||
- type: input
|
||||
id: version
|
||||
attributes:
|
||||
label: tzst Version
|
||||
description: What version of tzst are you using?
|
||||
placeholder: "e.g., 1.0.0"
|
||||
validations:
|
||||
required: true
|
||||
|
||||
- type: input
|
||||
id: python-version
|
||||
attributes:
|
||||
label: Python Version
|
||||
description: What version of Python are you using?
|
||||
placeholder: "e.g., 3.12.0"
|
||||
validations:
|
||||
required: true
|
||||
|
||||
- type: dropdown
|
||||
id: platform
|
||||
attributes:
|
||||
label: Operating System
|
||||
description: What operating system are you using?
|
||||
options:
|
||||
- Windows
|
||||
- macOS
|
||||
- Linux (Ubuntu)
|
||||
- Linux (CentOS/RHEL)
|
||||
- Linux (Arch)
|
||||
- Linux (Other)
|
||||
- Other
|
||||
validations:
|
||||
required: true
|
||||
|
||||
- type: dropdown
|
||||
id: usage-type
|
||||
attributes:
|
||||
label: Usage Type
|
||||
description: How are you using tzst?
|
||||
options:
|
||||
- Command Line Interface (CLI)
|
||||
- Python API
|
||||
- Both CLI and Python API
|
||||
validations:
|
||||
required: true
|
||||
|
||||
- type: textarea
|
||||
id: description
|
||||
attributes:
|
||||
label: Bug Description
|
||||
description: A clear and concise description of what the bug is.
|
||||
placeholder: Describe what happened and what you expected to happen.
|
||||
validations:
|
||||
required: true
|
||||
|
||||
- type: textarea
|
||||
id: reproduce
|
||||
attributes:
|
||||
label: Steps to Reproduce
|
||||
description: Detailed steps to reproduce the bug.
|
||||
placeholder: |
|
||||
1. Create a file/directory structure...
|
||||
2. Run the command `tzst a archive.tzst ...`
|
||||
3. Observe the error...
|
||||
validations:
|
||||
required: true
|
||||
|
||||
- type: textarea
|
||||
id: expected
|
||||
attributes:
|
||||
label: Expected Behavior
|
||||
description: What did you expect to happen?
|
||||
placeholder: Describe the expected behavior.
|
||||
validations:
|
||||
required: true
|
||||
|
||||
- type: textarea
|
||||
id: actual
|
||||
attributes:
|
||||
label: Actual Behavior
|
||||
description: What actually happened? Include any error messages or stack traces.
|
||||
placeholder: |
|
||||
Include full error messages, stack traces, or unexpected output here.
|
||||
Use code blocks (```) for formatting.
|
||||
validations:
|
||||
required: true
|
||||
|
||||
- type: textarea
|
||||
id: archive-details
|
||||
attributes:
|
||||
label: Archive Details (if applicable)
|
||||
description: Information about the archive or files involved.
|
||||
placeholder: |
|
||||
- Archive size:
|
||||
- Number of files:
|
||||
- File types:
|
||||
- Compression level used:
|
||||
- Special file types (symlinks, executables, etc.):
|
||||
validations:
|
||||
required: false
|
||||
|
||||
- type: textarea
|
||||
id: environment
|
||||
attributes:
|
||||
label: Environment Details
|
||||
description: Additional environment information that might be relevant.
|
||||
placeholder: |
|
||||
- Available disk space:
|
||||
- Memory (RAM):
|
||||
- File system type:
|
||||
- Any special permissions or restrictions:
|
||||
- Antivirus software (if relevant):
|
||||
validations:
|
||||
required: false
|
||||
|
||||
- type: dropdown
|
||||
id: workaround
|
||||
attributes:
|
||||
label: Workaround Available
|
||||
description: Have you found any workaround for this issue?
|
||||
options:
|
||||
- "No workaround found"
|
||||
- "Partial workaround available"
|
||||
- "Full workaround available"
|
||||
validations:
|
||||
required: false
|
||||
|
||||
- type: textarea
|
||||
id: workaround-details
|
||||
attributes:
|
||||
label: Workaround Details
|
||||
description: If you found a workaround, please describe it here.
|
||||
placeholder: Describe any workaround you've found.
|
||||
validations:
|
||||
required: false
|
||||
|
||||
- type: checkboxes
|
||||
id: additional-info
|
||||
attributes:
|
||||
label: Additional Information
|
||||
description: Please check any that apply to your situation.
|
||||
options:
|
||||
- label: This is a regression (worked in a previous version)
|
||||
- label: This affects large files (>100MB)
|
||||
- label: This affects archives with many files (>1000)
|
||||
- label: This involves Unicode/international characters
|
||||
- label: This involves symbolic links or special files
|
||||
- label: This affects atomic file operations
|
||||
- label: This affects streaming mode
|
||||
- label: This involves compression level edge cases
|
||||
- label: This is platform-specific behavior
|
||||
|
||||
- type: textarea
|
||||
id: additional-context
|
||||
attributes:
|
||||
label: Additional Context
|
||||
description: Any other context, screenshots, or information that might help.
|
||||
placeholder: Add any other context about the problem here.
|
||||
validations:
|
||||
required: false
|
||||
@@ -0,0 +1,14 @@
|
||||
blank_issues_enabled: false
|
||||
contact_links:
|
||||
- name: 📖 Documentation
|
||||
url: https://github.com/xixu-me/tzst#readme
|
||||
about: Check the README for usage instructions and examples
|
||||
- name: 🔒 Security Vulnerabilities (Private)
|
||||
url: mailto:security@xi-xu.me
|
||||
about: For critical security vulnerabilities, please report privately via email
|
||||
- name: 💬 Discussions
|
||||
url: https://github.com/xixu-me/tzst/discussions
|
||||
about: Ask questions and discuss ideas with the community
|
||||
- name: 📊 PyPI Package
|
||||
url: https://pypi.org/project/tzst/
|
||||
about: Download and install tzst from PyPI
|
||||
@@ -1,10 +0,0 @@
|
||||
---
|
||||
name: Custom issue template
|
||||
about: Describe this issue template's purpose here.
|
||||
title: ''
|
||||
labels: ''
|
||||
assignees: ''
|
||||
|
||||
---
|
||||
|
||||
|
||||
@@ -0,0 +1,233 @@
|
||||
name: Documentation Issue
|
||||
description: Report issues with documentation, examples, or request documentation improvements
|
||||
title: "[DOCS] "
|
||||
labels: ["documentation", "needs-triage"]
|
||||
assignees: []
|
||||
body:
|
||||
- type: markdown
|
||||
attributes:
|
||||
value: |
|
||||
Thank you for helping improve tzst documentation! Clear documentation is essential for user experience.
|
||||
|
||||
- type: checkboxes
|
||||
id: search
|
||||
attributes:
|
||||
label: Search for existing issues
|
||||
description: Please search existing issues before creating a new one.
|
||||
options:
|
||||
- label: I have searched for existing documentation issues and didn't find a duplicate
|
||||
required: true
|
||||
|
||||
- type: dropdown
|
||||
id: doc-type
|
||||
attributes:
|
||||
label: Documentation Type
|
||||
description: What type of documentation issue is this?
|
||||
options:
|
||||
- README.md
|
||||
- CONTRIBUTING.md
|
||||
- API documentation (docstrings)
|
||||
- CLI help text
|
||||
- Code examples
|
||||
- Installation instructions
|
||||
- Error messages
|
||||
- Security documentation
|
||||
- Performance guidelines
|
||||
- Platform-specific notes
|
||||
- Other
|
||||
validations:
|
||||
required: true
|
||||
|
||||
- type: dropdown
|
||||
id: issue-category
|
||||
attributes:
|
||||
label: Issue Category
|
||||
description: What type of documentation issue are you reporting?
|
||||
options:
|
||||
- Missing documentation
|
||||
- Incorrect/outdated information
|
||||
- Unclear or confusing explanation
|
||||
- Missing examples
|
||||
- Broken links or references
|
||||
- Formatting issues
|
||||
- Typos or grammar errors
|
||||
- Translation issues
|
||||
- Accessibility improvements
|
||||
- New documentation request
|
||||
validations:
|
||||
required: true
|
||||
|
||||
- type: textarea
|
||||
id: description
|
||||
attributes:
|
||||
label: Issue Description
|
||||
description: Describe the documentation issue in detail.
|
||||
placeholder: |
|
||||
Clearly describe the documentation problem:
|
||||
- What information is missing, incorrect, or unclear?
|
||||
- Where did you encounter this issue?
|
||||
- What were you trying to accomplish?
|
||||
validations:
|
||||
required: true
|
||||
|
||||
- type: textarea
|
||||
id: location
|
||||
attributes:
|
||||
label: Documentation Location
|
||||
description: Where is this documentation located?
|
||||
placeholder: |
|
||||
Specify the location:
|
||||
- File name: README.md, CONTRIBUTING.md, etc.
|
||||
- Section: "Installation", "API Reference", etc.
|
||||
- Line numbers (if applicable):
|
||||
- URL (if online documentation):
|
||||
validations:
|
||||
required: true
|
||||
|
||||
- type: textarea
|
||||
id: current-content
|
||||
attributes:
|
||||
label: Current Content
|
||||
description: What does the current documentation say? (copy the relevant text)
|
||||
placeholder: |
|
||||
Copy the current text that is problematic:
|
||||
```
|
||||
[Paste current documentation text here]
|
||||
```
|
||||
validations:
|
||||
required: false
|
||||
|
||||
- type: textarea
|
||||
id: expected-content
|
||||
attributes:
|
||||
label: Suggested Improvement
|
||||
description: What should the documentation say instead?
|
||||
placeholder: |
|
||||
Provide your suggested improvement:
|
||||
- Corrected information
|
||||
- Clearer explanation
|
||||
- Additional examples
|
||||
- Better formatting
|
||||
validations:
|
||||
required: true
|
||||
|
||||
- type: textarea
|
||||
id: examples-needed
|
||||
attributes:
|
||||
label: Examples Needed
|
||||
description: What examples would help illustrate this documentation?
|
||||
placeholder: |
|
||||
Suggest specific examples:
|
||||
- Code examples showing usage
|
||||
- Command-line examples
|
||||
- Common use cases
|
||||
- Error handling examples
|
||||
validations:
|
||||
required: false
|
||||
|
||||
- type: dropdown
|
||||
id: user-level
|
||||
attributes:
|
||||
label: Target User Level
|
||||
description: What level of user would benefit from this documentation improvement?
|
||||
options:
|
||||
- Beginner (first-time users)
|
||||
- Intermediate (some experience)
|
||||
- Advanced (power users)
|
||||
- Developer (contributing to tzst)
|
||||
- All users
|
||||
validations:
|
||||
required: false
|
||||
|
||||
- type: checkboxes
|
||||
id: documentation-areas
|
||||
attributes:
|
||||
label: Related Documentation Areas
|
||||
description: Which areas of documentation are related to this issue?
|
||||
options:
|
||||
- label: Installation and setup
|
||||
- label: Basic usage and getting started
|
||||
- label: CLI commands and options
|
||||
- label: Python API reference
|
||||
- label: Performance and optimization
|
||||
- label: Security considerations
|
||||
- label: Platform-specific information
|
||||
- label: Troubleshooting and error handling
|
||||
- label: Advanced usage patterns
|
||||
- label: Integration with other tools
|
||||
- label: Contributing guidelines
|
||||
- label: Development setup
|
||||
|
||||
- type: dropdown
|
||||
id: discovery-context
|
||||
attributes:
|
||||
label: How did you discover this issue?
|
||||
description: What were you doing when you found this documentation problem?
|
||||
options:
|
||||
- Reading documentation to learn tzst
|
||||
- Following installation instructions
|
||||
- Looking for specific API information
|
||||
- Trying to solve a problem
|
||||
- Reviewing code examples
|
||||
- Contributing to the project
|
||||
- Teaching/helping others
|
||||
- Other
|
||||
validations:
|
||||
required: false
|
||||
|
||||
- type: textarea
|
||||
id: user-journey
|
||||
attributes:
|
||||
label: User Journey Context
|
||||
description: Describe what you were trying to accomplish when you encountered this issue.
|
||||
placeholder: |
|
||||
Help us understand the context:
|
||||
- What task were you trying to complete?
|
||||
- What information were you looking for?
|
||||
- What confusion or frustration did you experience?
|
||||
- How did this impact your ability to use tzst?
|
||||
validations:
|
||||
required: false
|
||||
|
||||
- type: checkboxes
|
||||
id: improvement-type
|
||||
attributes:
|
||||
label: Improvement Type
|
||||
description: What type of improvement would help most?
|
||||
options:
|
||||
- label: More detailed explanations
|
||||
- label: Step-by-step tutorials
|
||||
- label: More code examples
|
||||
- label: Better organization/structure
|
||||
- label: Visual aids (diagrams, screenshots)
|
||||
- label: Cross-references to related topics
|
||||
- label: Troubleshooting guides
|
||||
- label: FAQ section
|
||||
- label: Video or interactive content
|
||||
- label: Beginner-friendly explanations
|
||||
- label: Advanced technical details
|
||||
|
||||
- type: textarea
|
||||
id: additional-context
|
||||
attributes:
|
||||
label: Additional Context
|
||||
description: Any other context that would help improve the documentation.
|
||||
placeholder: |
|
||||
Additional information:
|
||||
- Your experience level with similar tools
|
||||
- Documentation from other projects that you found helpful
|
||||
- Specific use cases that need better coverage
|
||||
- Any other relevant details
|
||||
validations:
|
||||
required: false
|
||||
|
||||
- type: checkboxes
|
||||
id: contribution-interest
|
||||
attributes:
|
||||
label: Contribution Interest
|
||||
description: Would you be interested in helping improve this documentation?
|
||||
options:
|
||||
- label: I would be interested in writing/improving this documentation
|
||||
- label: I would be interested in reviewing documentation changes
|
||||
- label: I can provide feedback on documentation drafts
|
||||
- label: I can help with testing documentation examples
|
||||
@@ -1,20 +0,0 @@
|
||||
---
|
||||
name: Feature request
|
||||
about: Suggest an idea for this project
|
||||
title: ''
|
||||
labels: ''
|
||||
assignees: ''
|
||||
|
||||
---
|
||||
|
||||
**Is your feature request related to a problem? Please describe.**
|
||||
A clear and concise description of what the problem is. Ex. I'm always frustrated when [...]
|
||||
|
||||
**Describe the solution you'd like**
|
||||
A clear and concise description of what you want to happen.
|
||||
|
||||
**Describe alternatives you've considered**
|
||||
A clear and concise description of any alternative solutions or features you've considered.
|
||||
|
||||
**Additional context**
|
||||
Add any other context or screenshots about the feature request here.
|
||||
@@ -0,0 +1,147 @@
|
||||
name: Feature Request
|
||||
description: Suggest a new feature or enhancement for tzst
|
||||
title: "[FEATURE] "
|
||||
labels: ["enhancement", "needs-triage"]
|
||||
assignees: []
|
||||
body:
|
||||
- type: markdown
|
||||
attributes:
|
||||
value: |
|
||||
Thank you for suggesting a new feature! Please provide as much detail as possible about your request.
|
||||
|
||||
- type: checkboxes
|
||||
id: search
|
||||
attributes:
|
||||
label: Search for existing requests
|
||||
description: Please search existing issues and discussions before creating a new feature request.
|
||||
options:
|
||||
- label: I have searched for existing feature requests and didn't find a duplicate
|
||||
required: true
|
||||
|
||||
- type: dropdown
|
||||
id: feature-type
|
||||
attributes:
|
||||
label: Feature Type
|
||||
description: What type of feature are you requesting?
|
||||
options:
|
||||
- New CLI command or option
|
||||
- New Python API functionality
|
||||
- Performance improvement
|
||||
- New compression options
|
||||
- Enhanced error handling
|
||||
- Security enhancement
|
||||
- Cross-platform compatibility
|
||||
- Integration with other tools
|
||||
- Documentation improvement
|
||||
- Other
|
||||
validations:
|
||||
required: true
|
||||
|
||||
- type: textarea
|
||||
id: problem
|
||||
attributes:
|
||||
label: Problem Description
|
||||
description: What problem would this feature solve? What use case would it enable?
|
||||
placeholder: |
|
||||
Describe the problem or limitation you're experiencing.
|
||||
What use case would this feature enable?
|
||||
validations:
|
||||
required: true
|
||||
|
||||
- type: textarea
|
||||
id: solution
|
||||
attributes:
|
||||
label: Proposed Solution
|
||||
description: Describe the solution you'd like to see implemented.
|
||||
placeholder: |
|
||||
Describe your proposed solution in detail.
|
||||
How would users interact with this feature?
|
||||
validations:
|
||||
required: true
|
||||
|
||||
- type: textarea
|
||||
id: examples
|
||||
attributes:
|
||||
label: Usage Examples
|
||||
description: Provide examples of how this feature would be used.
|
||||
placeholder: |
|
||||
# CLI example:
|
||||
tzst new-command archive.tzst --new-option
|
||||
|
||||
# Python API example:
|
||||
from tzst import TzstArchive
|
||||
with TzstArchive("archive.tzst") as archive:
|
||||
archive.new_method()
|
||||
validations:
|
||||
required: false
|
||||
|
||||
- type: textarea
|
||||
id: alternatives
|
||||
attributes:
|
||||
label: Alternatives Considered
|
||||
description: What alternatives have you considered? Why isn't the current functionality sufficient?
|
||||
placeholder: |
|
||||
Describe any alternative solutions you've considered.
|
||||
Why don't existing features meet your needs?
|
||||
validations:
|
||||
required: false
|
||||
|
||||
- type: dropdown
|
||||
id: priority
|
||||
attributes:
|
||||
label: Priority Level
|
||||
description: How important is this feature to you?
|
||||
options:
|
||||
- Critical (blocks important workflows)
|
||||
- High (would significantly improve workflow)
|
||||
- Medium (nice to have)
|
||||
- Low (minor convenience)
|
||||
validations:
|
||||
required: false
|
||||
|
||||
- type: checkboxes
|
||||
id: compatibility
|
||||
attributes:
|
||||
label: Compatibility Considerations
|
||||
description: Please check any that apply to your feature request.
|
||||
options:
|
||||
- label: This feature should maintain backward compatibility
|
||||
- label: This feature may require breaking changes
|
||||
- label: This feature should work on all supported platforms
|
||||
- label: This feature is platform-specific
|
||||
- label: This feature affects file format compatibility
|
||||
- label: This feature affects performance significantly
|
||||
- label: This feature requires new dependencies
|
||||
|
||||
- type: textarea
|
||||
id: implementation
|
||||
attributes:
|
||||
label: Implementation Ideas
|
||||
description: Do you have any ideas about how this could be implemented?
|
||||
placeholder: |
|
||||
If you have technical knowledge about how this could be implemented,
|
||||
please share your ideas here. This is optional but helpful.
|
||||
validations:
|
||||
required: false
|
||||
|
||||
- type: checkboxes
|
||||
id: contribution
|
||||
attributes:
|
||||
label: Contribution Interest
|
||||
description: Would you be interested in contributing to this feature?
|
||||
options:
|
||||
- label: I would be interested in implementing this feature
|
||||
- label: I would be interested in testing this feature
|
||||
- label: I would be interested in writing documentation for this feature
|
||||
- label: I can provide ongoing feedback during development
|
||||
|
||||
- type: textarea
|
||||
id: additional-context
|
||||
attributes:
|
||||
label: Additional Context
|
||||
description: Any other context, links, or information that would help understand this request.
|
||||
placeholder: |
|
||||
Add any other context about the feature request here.
|
||||
Links to similar features in other tools, academic papers, etc.
|
||||
validations:
|
||||
required: false
|
||||
@@ -0,0 +1,219 @@
|
||||
name: Performance Issue
|
||||
description: Report performance problems, memory usage, or speed issues
|
||||
title: "[PERFORMANCE] "
|
||||
labels: ["performance", "needs-triage"]
|
||||
assignees: []
|
||||
body:
|
||||
- type: markdown
|
||||
attributes:
|
||||
value: |
|
||||
Thank you for reporting a performance issue! Performance problems can be tricky to diagnose, so please provide as much detail as possible.
|
||||
|
||||
- type: checkboxes
|
||||
id: search
|
||||
attributes:
|
||||
label: Search for existing issues
|
||||
description: Please search existing issues before creating a new one.
|
||||
options:
|
||||
- label: I have searched for existing performance issues and didn't find a duplicate
|
||||
required: true
|
||||
|
||||
- type: input
|
||||
id: version
|
||||
attributes:
|
||||
label: tzst Version
|
||||
description: What version of tzst are you using?
|
||||
placeholder: "e.g., 1.0.0"
|
||||
validations:
|
||||
required: true
|
||||
|
||||
- type: input
|
||||
id: python-version
|
||||
attributes:
|
||||
label: Python Version
|
||||
description: What version of Python are you using?
|
||||
placeholder: "e.g., 3.12.0"
|
||||
validations:
|
||||
required: true
|
||||
|
||||
- type: dropdown
|
||||
id: platform
|
||||
attributes:
|
||||
label: Operating System
|
||||
description: What operating system are you using?
|
||||
options:
|
||||
- Windows
|
||||
- macOS
|
||||
- Linux (Ubuntu)
|
||||
- Linux (CentOS/RHEL)
|
||||
- Linux (Arch)
|
||||
- Linux (Other)
|
||||
- Other
|
||||
validations:
|
||||
required: true
|
||||
|
||||
- type: dropdown
|
||||
id: performance-type
|
||||
attributes:
|
||||
label: Performance Issue Type
|
||||
description: What type of performance problem are you experiencing?
|
||||
options:
|
||||
- Slow compression speed
|
||||
- Slow decompression speed
|
||||
- High memory usage
|
||||
- High CPU usage
|
||||
- Slow file I/O
|
||||
- Poor scaling with large files
|
||||
- Poor scaling with many files
|
||||
- Streaming mode performance
|
||||
- CLI responsiveness
|
||||
- Other
|
||||
validations:
|
||||
required: true
|
||||
|
||||
- type: textarea
|
||||
id: description
|
||||
attributes:
|
||||
label: Performance Issue Description
|
||||
description: Describe the performance problem you're experiencing.
|
||||
placeholder: |
|
||||
Describe what performance issue you're seeing.
|
||||
Be specific about what operations are slow or using too much memory.
|
||||
validations:
|
||||
required: true
|
||||
|
||||
- type: textarea
|
||||
id: benchmark-data
|
||||
attributes:
|
||||
label: Benchmark Data
|
||||
description: Provide timing, memory usage, or other measurable data.
|
||||
placeholder: |
|
||||
Operation: Creating 1GB archive
|
||||
Time taken: 45 seconds
|
||||
Memory usage: 2GB RAM
|
||||
CPU usage: 100% for entire duration
|
||||
|
||||
Expected: ~15 seconds, <500MB RAM
|
||||
validations:
|
||||
required: true
|
||||
|
||||
- type: textarea
|
||||
id: data-characteristics
|
||||
attributes:
|
||||
label: Data Characteristics
|
||||
description: Describe the files/data you're working with.
|
||||
placeholder: |
|
||||
- Total size:
|
||||
- Number of files:
|
||||
- File types:
|
||||
- Average file size:
|
||||
- Directory structure depth:
|
||||
- Compression level used:
|
||||
- Archive format (.tzst or .tar.zst):
|
||||
validations:
|
||||
required: true
|
||||
|
||||
- type: textarea
|
||||
id: system-specs
|
||||
attributes:
|
||||
label: System Specifications
|
||||
description: Provide details about your system hardware.
|
||||
placeholder: |
|
||||
- CPU:
|
||||
- RAM:
|
||||
- Storage type (SSD/HDD):
|
||||
- Available disk space:
|
||||
- Network storage (if applicable):
|
||||
validations:
|
||||
required: true
|
||||
|
||||
- type: dropdown
|
||||
id: usage-pattern
|
||||
attributes:
|
||||
label: Usage Pattern
|
||||
description: How are you using tzst?
|
||||
options:
|
||||
- Command Line Interface (CLI)
|
||||
- Python API with small archives
|
||||
- Python API with large archives
|
||||
- Python API with many small files
|
||||
- Streaming mode
|
||||
- Batch processing multiple archives
|
||||
- Interactive usage
|
||||
validations:
|
||||
required: true
|
||||
|
||||
- type: textarea
|
||||
id: reproduce
|
||||
attributes:
|
||||
label: Steps to Reproduce
|
||||
description: Detailed steps to reproduce the performance issue.
|
||||
placeholder: |
|
||||
1. Create test data of size X...
|
||||
2. Run command `tzst a archive.tzst ...`
|
||||
3. Monitor performance with [tool]...
|
||||
4. Observe slow performance...
|
||||
validations:
|
||||
required: true
|
||||
|
||||
- type: textarea
|
||||
id: comparison
|
||||
attributes:
|
||||
label: Performance Comparison
|
||||
description: How does tzst performance compare to other tools or previous versions?
|
||||
placeholder: |
|
||||
Comparison with other tools (tar, gzip, etc.):
|
||||
- Tool A: X seconds, Y MB memory
|
||||
- Tool B: X seconds, Y MB memory
|
||||
- tzst: X seconds, Y MB memory
|
||||
|
||||
Comparison with previous tzst versions (if applicable):
|
||||
- Version X.X.X: timing/memory
|
||||
- Current version: timing/memory
|
||||
validations:
|
||||
required: false
|
||||
|
||||
- type: checkboxes
|
||||
id: performance-factors
|
||||
attributes:
|
||||
label: Performance Factors
|
||||
description: Please check any that apply to your situation.
|
||||
options:
|
||||
- label: Issue occurs with atomic file operations enabled
|
||||
- label: Issue occurs with streaming mode enabled
|
||||
- label: Issue occurs with high compression levels (15+)
|
||||
- label: Issue occurs with low compression levels (1-5)
|
||||
- label: Issue occurs with Unicode filenames
|
||||
- label: Issue occurs with symbolic links
|
||||
- label: Issue occurs with many small files
|
||||
- label: Issue occurs with few large files
|
||||
- label: Issue occurs on network storage
|
||||
- label: Issue occurs during extraction
|
||||
- label: Issue occurs during compression
|
||||
- label: Issue is reproducible consistently
|
||||
|
||||
- type: textarea
|
||||
id: profiling-data
|
||||
attributes:
|
||||
label: Profiling Data
|
||||
description: If you've done any profiling, please include the results.
|
||||
placeholder: |
|
||||
Include any profiling data you've collected:
|
||||
- Python profiler output
|
||||
- Memory profiler results
|
||||
- System monitoring data
|
||||
- etc.
|
||||
validations:
|
||||
required: false
|
||||
|
||||
- type: textarea
|
||||
id: additional-context
|
||||
attributes:
|
||||
label: Additional Context
|
||||
description: Any other context that might help diagnose the performance issue.
|
||||
placeholder: |
|
||||
- Are you running other resource-intensive programs?
|
||||
- Any specific system configurations?
|
||||
- Any other relevant details?
|
||||
validations:
|
||||
required: false
|
||||
@@ -0,0 +1,239 @@
|
||||
name: Platform-Specific Issue
|
||||
description: Report issues specific to Windows, macOS, Linux, or other platforms
|
||||
title: "[PLATFORM] "
|
||||
labels: ["platform-specific", "needs-triage"]
|
||||
assignees: []
|
||||
body:
|
||||
- type: markdown
|
||||
attributes:
|
||||
value: |
|
||||
Thank you for reporting a platform-specific issue! These issues help us ensure tzst works correctly across all supported platforms.
|
||||
|
||||
- type: checkboxes
|
||||
id: search
|
||||
attributes:
|
||||
label: Search for existing issues
|
||||
description: Please search existing issues before creating a new one.
|
||||
options:
|
||||
- label: I have searched for existing platform-specific issues and didn't find a duplicate
|
||||
required: true
|
||||
|
||||
- type: input
|
||||
id: version
|
||||
attributes:
|
||||
label: tzst Version
|
||||
description: What version of tzst are you using?
|
||||
placeholder: "e.g., 1.0.0"
|
||||
validations:
|
||||
required: true
|
||||
|
||||
- type: input
|
||||
id: python-version
|
||||
attributes:
|
||||
label: Python Version
|
||||
description: What version of Python are you using?
|
||||
placeholder: "e.g., 3.12.0"
|
||||
validations:
|
||||
required: true
|
||||
|
||||
- type: dropdown
|
||||
id: platform
|
||||
attributes:
|
||||
label: Primary Platform (where issue occurs)
|
||||
description: What operating system are you experiencing the issue on?
|
||||
options:
|
||||
- Windows 11
|
||||
- Windows 10
|
||||
- Windows (other version)
|
||||
- macOS (latest)
|
||||
- macOS (older version)
|
||||
- Ubuntu (latest LTS)
|
||||
- Ubuntu (other version)
|
||||
- CentOS/RHEL
|
||||
- Arch Linux
|
||||
- Alpine Linux
|
||||
- Debian
|
||||
- Other Linux distribution
|
||||
- FreeBSD
|
||||
- Other Unix-like OS
|
||||
validations:
|
||||
required: true
|
||||
|
||||
- type: input
|
||||
id: platform-details
|
||||
attributes:
|
||||
label: Platform Details
|
||||
description: Specific version, architecture, or other platform details.
|
||||
placeholder: "e.g., Windows 10 Pro 22H2, x64 / Ubuntu 22.04.3 LTS, arm64 / macOS 13.5, x86_64"
|
||||
validations:
|
||||
required: true
|
||||
|
||||
- type: dropdown
|
||||
id: cross-platform-test
|
||||
attributes:
|
||||
label: Cross-Platform Testing
|
||||
description: Have you tested this on other platforms?
|
||||
options:
|
||||
- "No, only tested on the affected platform"
|
||||
- "Yes, works correctly on other platforms"
|
||||
- "Yes, same issue occurs on other platforms"
|
||||
- "Partially tested on other platforms"
|
||||
validations:
|
||||
required: true
|
||||
|
||||
- type: textarea
|
||||
id: other-platforms
|
||||
attributes:
|
||||
label: Other Platform Results
|
||||
description: If you tested on other platforms, what were the results?
|
||||
placeholder: |
|
||||
Platform 1: Working correctly / Same issue / Different issue / Not tested
|
||||
Platform 2: Working correctly / Same issue / Different issue / Not tested
|
||||
Platform 3: Working correctly / Same issue / Different issue / Not tested
|
||||
validations:
|
||||
required: false
|
||||
|
||||
- type: dropdown
|
||||
id: issue-category
|
||||
attributes:
|
||||
label: Issue Category
|
||||
description: What type of platform-specific issue is this?
|
||||
options:
|
||||
- File path handling (Windows path limits, Unix paths)
|
||||
- File permissions and attributes
|
||||
- Symbolic links and special files
|
||||
- Character encoding and Unicode
|
||||
- File system limitations
|
||||
- Archive compatibility across platforms
|
||||
- CLI behavior differences
|
||||
- Installation or packaging issues
|
||||
- Performance differences
|
||||
- Security or permissions model
|
||||
- Other
|
||||
validations:
|
||||
required: true
|
||||
|
||||
- type: textarea
|
||||
id: description
|
||||
attributes:
|
||||
label: Issue Description
|
||||
description: Describe the platform-specific issue you're experiencing.
|
||||
placeholder: |
|
||||
Describe the issue in detail.
|
||||
Be specific about what behavior you're seeing that differs from expected or from other platforms.
|
||||
validations:
|
||||
required: true
|
||||
|
||||
- type: textarea
|
||||
id: reproduce
|
||||
attributes:
|
||||
label: Steps to Reproduce
|
||||
description: Steps to reproduce the issue on the affected platform.
|
||||
placeholder: |
|
||||
1. Create files with [specific characteristics]...
|
||||
2. Run `tzst a archive.tzst ...`
|
||||
3. Observe platform-specific behavior...
|
||||
validations:
|
||||
required: true
|
||||
|
||||
- type: textarea
|
||||
id: expected-behavior
|
||||
attributes:
|
||||
label: Expected Behavior
|
||||
description: What behavior did you expect? How does it work on other platforms?
|
||||
placeholder: |
|
||||
Expected behavior (or behavior on other platforms):
|
||||
- Should create archive successfully
|
||||
- Should preserve file attributes
|
||||
- etc.
|
||||
validations:
|
||||
required: true
|
||||
|
||||
- type: textarea
|
||||
id: actual-behavior
|
||||
attributes:
|
||||
label: Actual Behavior
|
||||
description: What actually happens on this platform?
|
||||
placeholder: |
|
||||
Actual behavior on [platform]:
|
||||
- Error message:
|
||||
- Unexpected results:
|
||||
- Different file handling:
|
||||
validations:
|
||||
required: true
|
||||
|
||||
- type: checkboxes
|
||||
id: platform-features
|
||||
attributes:
|
||||
label: Platform-Specific Features Involved
|
||||
description: Check any platform-specific features that are involved in this issue.
|
||||
options:
|
||||
- label: Windows long paths (>260 characters)
|
||||
- label: Windows reserved filenames (CON, PRN, AUX, etc.)
|
||||
- label: Windows file attributes (hidden, system, etc.)
|
||||
- label: Windows case-insensitive filesystem
|
||||
- label: Unix/Linux file permissions (chmod)
|
||||
- label: Unix/Linux symbolic links
|
||||
- label: Unix/Linux hard links
|
||||
- label: Unix/Linux special files (devices, pipes, etc.)
|
||||
- label: macOS resource forks or extended attributes
|
||||
- label: macOS case-insensitive but case-preserving filesystem
|
||||
- label: Unicode normalization differences
|
||||
- label: Path separator differences (/ vs \)
|
||||
- label: Network filesystem behaviors
|
||||
- label: Container or virtualized environment
|
||||
|
||||
- type: textarea
|
||||
id: file-examples
|
||||
attributes:
|
||||
label: Problematic File Examples
|
||||
description: Provide examples of files or paths that cause issues.
|
||||
placeholder: |
|
||||
Examples of files that cause problems:
|
||||
- "CON.txt" (Windows reserved name)
|
||||
- "file with very long path/..." (path length issues)
|
||||
- "файл.txt" (Unicode characters)
|
||||
- Symbolic link: link -> target
|
||||
validations:
|
||||
required: false
|
||||
|
||||
- type: textarea
|
||||
id: environment-details
|
||||
attributes:
|
||||
label: Environment Details
|
||||
description: Additional environment information specific to your platform.
|
||||
placeholder: |
|
||||
- Filesystem type: NTFS/ext4/APFS/etc.
|
||||
- Case sensitivity: case-sensitive/case-insensitive
|
||||
- Python installation method: system/conda/pyenv/etc.
|
||||
- Admin/root privileges: yes/no
|
||||
- Antivirus software (if Windows):
|
||||
- Container environment: Docker/WSL/etc.
|
||||
validations:
|
||||
required: false
|
||||
|
||||
- type: textarea
|
||||
id: workaround
|
||||
attributes:
|
||||
label: Workaround
|
||||
description: Have you found any platform-specific workarounds?
|
||||
placeholder: |
|
||||
Any workarounds you've discovered for this platform:
|
||||
- Using different file names
|
||||
- Different command options
|
||||
- Platform-specific configuration
|
||||
validations:
|
||||
required: false
|
||||
|
||||
- type: textarea
|
||||
id: additional-context
|
||||
attributes:
|
||||
label: Additional Context
|
||||
description: Any other platform-specific context that might be relevant.
|
||||
placeholder: |
|
||||
- Related to company/enterprise policies?
|
||||
- Specific hardware configurations?
|
||||
- Network or security restrictions?
|
||||
- Other relevant platform details?
|
||||
validations:
|
||||
required: false
|
||||
@@ -0,0 +1,206 @@
|
||||
name: Question or Support
|
||||
description: Ask a question about using tzst or get help with a specific use case
|
||||
title: "[QUESTION] "
|
||||
labels: ["question", "support"]
|
||||
assignees: []
|
||||
body:
|
||||
- type: markdown
|
||||
attributes:
|
||||
value: |
|
||||
Welcome! We're happy to help you with tzst. Please provide as much detail as possible so we can give you the best assistance.
|
||||
|
||||
**Note:** For general questions, you might also consider:
|
||||
- Checking the [README.md](../../README.md) for common usage patterns
|
||||
- Looking at existing issues for similar questions
|
||||
- Reviewing the Python API documentation in the code
|
||||
|
||||
- type: checkboxes
|
||||
id: search
|
||||
attributes:
|
||||
label: Search for existing answers
|
||||
description: Please search existing issues and documentation before asking.
|
||||
options:
|
||||
- label: I have searched existing issues and documentation for an answer
|
||||
required: true
|
||||
|
||||
- type: dropdown
|
||||
id: question-type
|
||||
attributes:
|
||||
label: Question Type
|
||||
description: What type of question do you have?
|
||||
options:
|
||||
- How to use a specific feature
|
||||
- Best practices recommendation
|
||||
- Performance optimization help
|
||||
- Platform-specific usage
|
||||
- Integration with other tools
|
||||
- Troubleshooting help
|
||||
- Feature availability/roadmap
|
||||
- Contributing guidance
|
||||
- Other
|
||||
validations:
|
||||
required: true
|
||||
|
||||
- type: textarea
|
||||
id: question
|
||||
attributes:
|
||||
label: Your Question
|
||||
description: What would you like to know about tzst?
|
||||
placeholder: |
|
||||
Ask your question clearly and specifically:
|
||||
- What are you trying to accomplish?
|
||||
- What specific help do you need?
|
||||
- What have you already tried?
|
||||
validations:
|
||||
required: true
|
||||
|
||||
- type: textarea
|
||||
id: context
|
||||
attributes:
|
||||
label: Context and Use Case
|
||||
description: Provide context about your situation and what you're trying to accomplish.
|
||||
placeholder: |
|
||||
Help us understand your situation:
|
||||
- What are you building or working on?
|
||||
- What constraints or requirements do you have?
|
||||
- Why are you using tzst for this task?
|
||||
- What's your experience level with similar tools?
|
||||
validations:
|
||||
required: true
|
||||
|
||||
- type: textarea
|
||||
id: attempted-solutions
|
||||
attributes:
|
||||
label: What You've Tried
|
||||
description: What have you already attempted? Include code, commands, or approaches you've tried.
|
||||
placeholder: |
|
||||
Describe what you've already tried:
|
||||
```bash
|
||||
# Command you tried
|
||||
tzst a archive.tzst files/
|
||||
```
|
||||
|
||||
```python
|
||||
# Python code you tried
|
||||
from tzst import create_archive
|
||||
create_archive("test.tzst", ["files/"])
|
||||
```
|
||||
|
||||
- Approach 1: [what happened]
|
||||
- Approach 2: [what happened]
|
||||
validations:
|
||||
required: false
|
||||
|
||||
- type: input
|
||||
id: tzst-version
|
||||
attributes:
|
||||
label: tzst Version
|
||||
description: What version of tzst are you using?
|
||||
placeholder: "e.g., 1.0.0"
|
||||
validations:
|
||||
required: false
|
||||
|
||||
- type: dropdown
|
||||
id: platform
|
||||
attributes:
|
||||
label: Platform
|
||||
description: What platform are you using?
|
||||
options:
|
||||
- Windows
|
||||
- macOS
|
||||
- Linux
|
||||
- Not platform-specific
|
||||
- Multiple platforms
|
||||
validations:
|
||||
required: false
|
||||
|
||||
- type: dropdown
|
||||
id: usage-context
|
||||
attributes:
|
||||
label: Usage Context
|
||||
description: How are you planning to use tzst?
|
||||
options:
|
||||
- Command line tool for personal use
|
||||
- Command line tool in scripts/automation
|
||||
- Python library in application
|
||||
- Python library for data processing
|
||||
- Integration with existing workflow
|
||||
- Learning/educational purposes
|
||||
- Contributing to tzst development
|
||||
- Other
|
||||
validations:
|
||||
required: false
|
||||
|
||||
- type: textarea
|
||||
id: data-characteristics
|
||||
attributes:
|
||||
label: Data Characteristics (if relevant)
|
||||
description: If your question involves specific types of data, describe it.
|
||||
placeholder: |
|
||||
If relevant to your question:
|
||||
- Types of files you're working with:
|
||||
- Approximate data size:
|
||||
- Number of files:
|
||||
- Special requirements (performance, security, etc.):
|
||||
validations:
|
||||
required: false
|
||||
|
||||
- type: textarea
|
||||
id: expected-outcome
|
||||
attributes:
|
||||
label: Desired Outcome
|
||||
description: What would be the ideal solution or outcome for your question?
|
||||
placeholder: |
|
||||
Describe what you're hoping to achieve:
|
||||
- Specific functionality you want
|
||||
- Performance characteristics needed
|
||||
- Integration requirements
|
||||
- etc.
|
||||
validations:
|
||||
required: false
|
||||
|
||||
- type: checkboxes
|
||||
id: help-areas
|
||||
attributes:
|
||||
label: Areas Where You Need Help
|
||||
description: Check the areas where you need assistance.
|
||||
options:
|
||||
- label: Understanding basic tzst concepts
|
||||
- label: Choosing the right commands/API methods
|
||||
- label: Optimizing performance
|
||||
- label: Handling large files or archives
|
||||
- label: Platform-specific considerations
|
||||
- label: Error handling and troubleshooting
|
||||
- label: Security considerations
|
||||
- label: Integration with other tools/workflows
|
||||
- label: Python API usage
|
||||
- label: CLI usage and scripting
|
||||
- label: Compression settings and trade-offs
|
||||
- label: File format compatibility
|
||||
|
||||
- type: dropdown
|
||||
id: urgency
|
||||
attributes:
|
||||
label: Urgency Level
|
||||
description: How urgent is this question for you?
|
||||
options:
|
||||
- Not urgent (general learning)
|
||||
- Somewhat urgent (current project)
|
||||
- Urgent (blocking my work)
|
||||
- Very urgent (production issue)
|
||||
validations:
|
||||
required: false
|
||||
|
||||
- type: textarea
|
||||
id: additional-info
|
||||
attributes:
|
||||
label: Additional Information
|
||||
description: Any other information that might help us answer your question.
|
||||
placeholder: |
|
||||
Anything else that might be relevant:
|
||||
- Links to similar questions or resources you've found
|
||||
- Specific error messages (if applicable)
|
||||
- Related tools or technologies you're using
|
||||
- Any constraints or limitations you're working with
|
||||
validations:
|
||||
required: false
|
||||
@@ -0,0 +1,217 @@
|
||||
name: Security Vulnerability
|
||||
description: Report a security vulnerability in tzst (please follow responsible disclosure)
|
||||
title: "[SECURITY] "
|
||||
labels: ["security", "needs-triage"]
|
||||
assignees: []
|
||||
body:
|
||||
- type: markdown
|
||||
attributes:
|
||||
value: |
|
||||
# Security Vulnerability Report
|
||||
|
||||
**⚠️ IMPORTANT: For serious security vulnerabilities, please consider using private reporting instead of a public issue.**
|
||||
|
||||
If this is a critical security vulnerability that could be exploited, please:
|
||||
1. Email the maintainers privately first
|
||||
2. Allow time for a security patch before public disclosure
|
||||
3. Follow responsible disclosure practices
|
||||
|
||||
For less critical security issues or general security improvements, this public issue form is appropriate.
|
||||
|
||||
- type: checkboxes
|
||||
id: disclosure
|
||||
attributes:
|
||||
label: Responsible Disclosure
|
||||
description: Please confirm your approach to reporting this security issue.
|
||||
options:
|
||||
- label: I understand the importance of responsible disclosure for security vulnerabilities
|
||||
required: true
|
||||
- label: This is not a critical vulnerability that requires private reporting
|
||||
required: true
|
||||
- label: I have considered the potential impact of public disclosure
|
||||
required: true
|
||||
|
||||
- type: dropdown
|
||||
id: severity
|
||||
attributes:
|
||||
label: Severity Level
|
||||
description: How severe do you consider this security issue?
|
||||
options:
|
||||
- Critical (immediate action required)
|
||||
- High (significant security risk)
|
||||
- Medium (moderate security concern)
|
||||
- Low (minor security improvement)
|
||||
- Informational (security-related best practice)
|
||||
validations:
|
||||
required: true
|
||||
|
||||
- type: dropdown
|
||||
id: vulnerability-type
|
||||
attributes:
|
||||
label: Vulnerability Type
|
||||
description: What type of security issue is this?
|
||||
options:
|
||||
- Path traversal / Directory traversal
|
||||
- Archive bomb / Zip bomb
|
||||
- Arbitrary file overwrite
|
||||
- Symlink attack
|
||||
- Information disclosure
|
||||
- Denial of Service (DoS)
|
||||
- Memory corruption
|
||||
- Injection vulnerability
|
||||
- Privilege escalation
|
||||
- Cryptographic weakness
|
||||
- Input validation bypass
|
||||
- Other
|
||||
validations:
|
||||
required: true
|
||||
|
||||
- type: textarea
|
||||
id: description
|
||||
attributes:
|
||||
label: Vulnerability Description
|
||||
description: Describe the security vulnerability in detail.
|
||||
placeholder: |
|
||||
Provide a clear description of the security issue:
|
||||
- What component is affected?
|
||||
- What is the vulnerability?
|
||||
- What are the potential consequences?
|
||||
validations:
|
||||
required: true
|
||||
|
||||
- type: textarea
|
||||
id: attack-scenario
|
||||
attributes:
|
||||
label: Attack Scenario
|
||||
description: Describe how this vulnerability could be exploited.
|
||||
placeholder: |
|
||||
Describe a realistic attack scenario:
|
||||
1. Attacker creates malicious archive with...
|
||||
2. Victim extracts archive using tzst...
|
||||
3. Result: files written outside extraction directory / code execution / etc.
|
||||
validations:
|
||||
required: true
|
||||
|
||||
- type: textarea
|
||||
id: impact
|
||||
attributes:
|
||||
label: Impact Assessment
|
||||
description: What is the potential impact of this vulnerability?
|
||||
placeholder: |
|
||||
Describe the potential impact:
|
||||
- Confidentiality: Can sensitive data be exposed?
|
||||
- Integrity: Can files be modified unexpectedly?
|
||||
- Availability: Can the system be made unavailable?
|
||||
- Scope: Who would be affected?
|
||||
validations:
|
||||
required: true
|
||||
|
||||
- type: textarea
|
||||
id: affected-versions
|
||||
attributes:
|
||||
label: Affected Versions
|
||||
description: Which versions of tzst are affected by this vulnerability?
|
||||
placeholder: |
|
||||
- tzst version tested:
|
||||
- Likely affected versions:
|
||||
- Known safe versions (if any):
|
||||
validations:
|
||||
required: true
|
||||
|
||||
- type: textarea
|
||||
id: reproduce
|
||||
attributes:
|
||||
label: Proof of Concept
|
||||
description: Provide steps to reproduce or demonstrate the vulnerability.
|
||||
placeholder: |
|
||||
**WARNING: Please ensure your PoC is safe and doesn't cause actual harm**
|
||||
|
||||
Steps to reproduce:
|
||||
1. Create test archive with specific structure...
|
||||
2. Run tzst command...
|
||||
3. Observe security issue...
|
||||
|
||||
Or provide code that demonstrates the issue safely.
|
||||
validations:
|
||||
required: true
|
||||
|
||||
- type: textarea
|
||||
id: mitigation
|
||||
attributes:
|
||||
label: Suggested Mitigation
|
||||
description: Do you have suggestions for how to fix this vulnerability?
|
||||
placeholder: |
|
||||
Suggested fixes or mitigations:
|
||||
- Input validation improvements
|
||||
- Security filter enhancements
|
||||
- API changes needed
|
||||
- Configuration options
|
||||
validations:
|
||||
required: false
|
||||
|
||||
- type: checkboxes
|
||||
id: security-features
|
||||
attributes:
|
||||
label: Related Security Features
|
||||
description: Which tzst security features are involved?
|
||||
options:
|
||||
- label: Extraction filters (data, tar, etc.)
|
||||
- label: Path validation
|
||||
- label: Symlink handling
|
||||
- label: Archive format validation
|
||||
- label: Memory limits
|
||||
- label: File size limits
|
||||
- label: Atomic operations
|
||||
- label: Temporary file handling
|
||||
- label: Error handling and cleanup
|
||||
|
||||
- type: textarea
|
||||
id: references
|
||||
attributes:
|
||||
label: References
|
||||
description: Any relevant CVEs, security advisories, or research papers.
|
||||
placeholder: |
|
||||
Related security research:
|
||||
- CVE numbers:
|
||||
- Security advisories:
|
||||
- Research papers:
|
||||
- Similar vulnerabilities in other tools:
|
||||
validations:
|
||||
required: false
|
||||
|
||||
- type: textarea
|
||||
id: environment
|
||||
attributes:
|
||||
label: Environment Details
|
||||
description: Platform and environment details relevant to this vulnerability.
|
||||
placeholder: |
|
||||
- Operating System:
|
||||
- Python version:
|
||||
- tzst version:
|
||||
- Specific environment factors:
|
||||
validations:
|
||||
required: false
|
||||
|
||||
- type: checkboxes
|
||||
id: disclosure-timeline
|
||||
attributes:
|
||||
label: Disclosure Timeline
|
||||
description: What is your intended disclosure timeline?
|
||||
options:
|
||||
- label: I plan to disclose this publicly immediately
|
||||
- label: I can wait for a security fix before public disclosure
|
||||
- label: I am requesting coordinated disclosure
|
||||
- label: This has already been disclosed elsewhere
|
||||
|
||||
- type: textarea
|
||||
id: additional-info
|
||||
attributes:
|
||||
label: Additional Information
|
||||
description: Any other relevant security information.
|
||||
placeholder: |
|
||||
Additional context:
|
||||
- How did you discover this vulnerability?
|
||||
- Are you aware of any exploitation in the wild?
|
||||
- Any other relevant details?
|
||||
validations:
|
||||
required: false
|
||||
@@ -0,0 +1,180 @@
|
||||
---
|
||||
name: Pull Request
|
||||
about: Submit a pull request to improve tzst
|
||||
title: ''
|
||||
labels: ''
|
||||
assignees: ''
|
||||
---
|
||||
|
||||
## Summary
|
||||
|
||||
<!-- Provide a brief summary of your changes -->
|
||||
|
||||
## Type of Change
|
||||
|
||||
<!-- Mark with an "x" all that apply -->
|
||||
|
||||
- [ ] 🐛 Bug fix (non-breaking change that fixes an issue)
|
||||
- [ ] ✨ New feature (non-breaking change that adds functionality)
|
||||
- [ ] 💥 Breaking change (fix or feature that would cause existing functionality to change)
|
||||
- [ ] 📚 Documentation update (changes to documentation only)
|
||||
- [ ] 🧪 Test improvements (adding or updating tests)
|
||||
- [ ] 🔧 Build/CI changes (changes to build process or CI configuration)
|
||||
- [ ] ♻️ Refactoring (code changes that neither fix bugs nor add features)
|
||||
- [ ] ⚡ Performance improvement
|
||||
- [ ] 🔒 Security fix
|
||||
|
||||
## Related Issues
|
||||
|
||||
<!-- Link to related issues using keywords -->
|
||||
<!-- Examples: -->
|
||||
<!-- Fixes #123 -->
|
||||
<!-- Closes #456 -->
|
||||
<!-- Related to #789 -->
|
||||
|
||||
## Changes Made
|
||||
|
||||
<!-- Describe the changes you made in detail -->
|
||||
|
||||
### Core Changes
|
||||
<!-- List main functionality changes -->
|
||||
-
|
||||
|
||||
### API Changes
|
||||
<!-- List any API changes (breaking or non-breaking) -->
|
||||
-
|
||||
|
||||
### CLI Changes
|
||||
<!-- List any command-line interface changes -->
|
||||
-
|
||||
|
||||
## Testing
|
||||
|
||||
<!-- Describe the testing you performed -->
|
||||
|
||||
### Test Coverage
|
||||
|
||||
- [ ] Added tests for new functionality
|
||||
- [ ] Updated existing tests as needed
|
||||
- [ ] All tests pass locally (`python -m pytest`)
|
||||
- [ ] Code coverage is maintained or improved
|
||||
|
||||
### Manual Testing
|
||||
<!-- Describe manual testing performed -->
|
||||
- [ ] Tested on Windows
|
||||
- [ ] Tested on macOS
|
||||
- [ ] Tested on Linux
|
||||
- [ ] Tested with Python 3.12
|
||||
- [ ] Tested with Python 3.13
|
||||
|
||||
### Test Scenarios
|
||||
<!-- List specific scenarios you tested -->
|
||||
-
|
||||
-
|
||||
-
|
||||
|
||||
## Code Quality
|
||||
|
||||
<!-- Confirm code quality checks -->
|
||||
|
||||
- [ ] Code follows project style guidelines (`ruff check .`)
|
||||
- [ ] Code is properly formatted (`ruff format --check .`)
|
||||
- [ ] Type hints are added for new public APIs
|
||||
- [ ] Docstrings are added/updated for public methods
|
||||
- [ ] No new security vulnerabilities introduced
|
||||
|
||||
## Documentation
|
||||
|
||||
<!-- Mark documentation changes -->
|
||||
|
||||
- [ ] README.md updated (if needed)
|
||||
- [ ] Code comments added for complex logic
|
||||
- [ ] Docstrings updated for modified functions/classes
|
||||
|
||||
## Backwards Compatibility
|
||||
|
||||
<!-- Address compatibility concerns -->
|
||||
|
||||
- [ ] Changes are backwards compatible
|
||||
- [ ] Breaking changes are documented and justified
|
||||
- [ ] Migration guide provided (if applicable)
|
||||
|
||||
## Performance Impact
|
||||
|
||||
<!-- Describe any performance implications -->
|
||||
|
||||
- [ ] No performance regression
|
||||
- [ ] Performance improvement (describe below)
|
||||
- [ ] Performance impact assessed and documented
|
||||
|
||||
<!-- If there are performance changes, describe them -->
|
||||
|
||||
## Security Considerations
|
||||
|
||||
<!-- Address security aspects -->
|
||||
|
||||
- [ ] No security implications
|
||||
- [ ] Security review completed
|
||||
- [ ] Input validation added/updated
|
||||
- [ ] Secure defaults maintained
|
||||
|
||||
## Platform Compatibility
|
||||
|
||||
<!-- Confirm platform testing -->
|
||||
|
||||
- [ ] Windows compatibility verified
|
||||
- [ ] macOS compatibility verified
|
||||
- [ ] Linux compatibility verified
|
||||
- [ ] Cross-platform file path handling tested
|
||||
|
||||
## Dependencies
|
||||
|
||||
<!-- Note any dependency changes -->
|
||||
|
||||
- [ ] No new dependencies added
|
||||
- [ ] New dependencies are justified and minimal
|
||||
- [ ] Dependency versions are appropriately constrained
|
||||
- [ ] Optional dependencies are properly marked
|
||||
|
||||
## Deployment
|
||||
|
||||
<!-- For maintainers - deployment considerations -->
|
||||
|
||||
- [ ] Version number updated (if applicable)
|
||||
- [ ] Release notes prepared (if applicable)
|
||||
- [ ] Migration steps documented (if applicable)
|
||||
|
||||
## Checklist
|
||||
|
||||
<!-- Final review checklist -->
|
||||
|
||||
- [ ] PR title follows conventional commit format
|
||||
- [ ] All CI checks are passing
|
||||
- [ ] Code has been self-reviewed
|
||||
- [ ] Changes have been tested thoroughly
|
||||
- [ ] Documentation is complete and accurate
|
||||
- [ ] Ready for code review
|
||||
|
||||
## Additional Notes
|
||||
|
||||
<!-- Add any additional context, screenshots, or notes for reviewers -->
|
||||
|
||||
## Reviewer Notes
|
||||
|
||||
<!-- For reviewers - any specific areas to focus on -->
|
||||
|
||||
### Review Focus Areas
|
||||
<!-- What should reviewers pay special attention to? -->
|
||||
-
|
||||
-
|
||||
-
|
||||
|
||||
### Questions for Reviewers
|
||||
<!-- Any specific questions for the review team -->
|
||||
-
|
||||
-
|
||||
|
||||
---
|
||||
|
||||
<!-- Thank you for contributing to tzst! -->
|
||||
<!-- Please ensure all items above are completed before requesting review -->
|
||||
@@ -3,13 +3,25 @@ name: CI/CD
|
||||
on:
|
||||
push:
|
||||
branches: [main, develop]
|
||||
paths-ignore:
|
||||
- "*.md"
|
||||
- "LICENSE"
|
||||
- "docs/**"
|
||||
- ".gitignore"
|
||||
pull_request:
|
||||
branches: [main]
|
||||
paths-ignore:
|
||||
- "*.md"
|
||||
- "LICENSE"
|
||||
- "docs/**"
|
||||
- ".gitignore"
|
||||
release:
|
||||
types: [published]
|
||||
|
||||
jobs:
|
||||
test:
|
||||
permissions:
|
||||
contents: read
|
||||
runs-on: ${{ matrix.os }}
|
||||
strategy:
|
||||
matrix:
|
||||
@@ -39,18 +51,19 @@ jobs:
|
||||
|
||||
- name: Run tests
|
||||
run: |
|
||||
pytest -v --cov=tzst --cov-report=xml
|
||||
pytest --cov=tzst --cov-branch --cov-report=xml
|
||||
|
||||
- name: Upload coverage to Codecov
|
||||
if: matrix.os == 'ubuntu-latest' && matrix.python-version == '3.12'
|
||||
uses: codecov/codecov-action@v3
|
||||
uses: codecov/codecov-action@v5
|
||||
with:
|
||||
file: ./coverage.xml
|
||||
flags: unittests
|
||||
name: codecov-umbrella
|
||||
token: ${{ secrets.CODECOV_TOKEN }}
|
||||
slug: xixu-me/tzst
|
||||
|
||||
build:
|
||||
needs: test
|
||||
permissions:
|
||||
contents: read
|
||||
runs-on: ubuntu-latest
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
@@ -142,7 +155,7 @@ jobs:
|
||||
git config --local user.name "GitHub Action"
|
||||
|
||||
- name: Commit and push changes
|
||||
uses: stefanzweifel/git-auto-commit-action@v4
|
||||
uses: stefanzweifel/git-auto-commit-action@v5
|
||||
with:
|
||||
commit_message: "Update badges [skip ci]"
|
||||
file_pattern: README.md
|
||||
|
||||
@@ -0,0 +1,99 @@
|
||||
name: Publish Documentation
|
||||
|
||||
on:
|
||||
push:
|
||||
branches:
|
||||
- main
|
||||
pull_request:
|
||||
branches:
|
||||
- main
|
||||
|
||||
jobs:
|
||||
build-and-deploy-docs:
|
||||
runs-on: ubuntu-latest
|
||||
permissions:
|
||||
contents: write
|
||||
pages: write
|
||||
id-token: write
|
||||
|
||||
steps:
|
||||
- name: Checkout code
|
||||
uses: actions/checkout@v4
|
||||
with:
|
||||
fetch-depth: 0
|
||||
|
||||
- name: Set up Python
|
||||
uses: actions/setup-python@v5
|
||||
with:
|
||||
python-version: "3.12"
|
||||
|
||||
- name: Cache pip dependencies
|
||||
uses: actions/cache@v4
|
||||
with:
|
||||
path: ~/.cache/pip
|
||||
key: ${{ runner.os }}-pip-docs-${{ hashFiles('docs/requirements.txt') }}
|
||||
restore-keys: |
|
||||
${{ runner.os }}-pip-docs-
|
||||
${{ runner.os }}-pip-
|
||||
|
||||
- name: Install dependencies
|
||||
run: |
|
||||
python -m pip install --upgrade pip
|
||||
pip install -e .
|
||||
pip install -r docs/requirements.txt
|
||||
|
||||
- name: Build Sphinx documentation
|
||||
run: |
|
||||
cd docs
|
||||
python -m sphinx -b html . _build -W --keep-going
|
||||
|
||||
- name: Upload documentation artifacts
|
||||
uses: actions/upload-artifact@v4
|
||||
with:
|
||||
name: documentation
|
||||
path: docs/_build/
|
||||
retention-days: 30
|
||||
|
||||
- name: Deploy to GitHub Pages
|
||||
if: github.ref == 'refs/heads/main' && github.event_name == 'push'
|
||||
uses: peaceiris/actions-gh-pages@v4
|
||||
with:
|
||||
github_token: ${{ secrets.GITHUB_TOKEN }}
|
||||
publish_dir: docs/_build
|
||||
force_orphan: true
|
||||
user_name: "github-actions[bot]"
|
||||
user_email: "github-actions[bot]@users.noreply.github.com"
|
||||
commit_message: "Deploy documentation from ${{ github.sha }}"
|
||||
|
||||
docs-quality-check:
|
||||
runs-on: ubuntu-latest
|
||||
steps:
|
||||
- name: Checkout code
|
||||
uses: actions/checkout@v4
|
||||
|
||||
- name: Set up Python
|
||||
uses: actions/setup-python@v5
|
||||
with:
|
||||
python-version: "3.12"
|
||||
|
||||
- name: Install dependencies
|
||||
run: |
|
||||
python -m pip install --upgrade pip
|
||||
pip install -e .
|
||||
pip install -r docs/requirements.txt
|
||||
|
||||
- name: Check documentation links
|
||||
run: |
|
||||
cd docs
|
||||
sphinx-build -b linkcheck . _build/linkcheck -W --keep-going
|
||||
continue-on-error: true
|
||||
|
||||
- name: Check documentation coverage
|
||||
run: |
|
||||
cd docs
|
||||
sphinx-build -b coverage . _build/coverage
|
||||
if [ -f _build/coverage/python.txt ]; then
|
||||
echo "Documentation coverage report:"
|
||||
cat _build/coverage/python.txt
|
||||
fi
|
||||
continue-on-error: true
|
||||
+372
@@ -0,0 +1,372 @@
|
||||
# Contributing to tzst
|
||||
|
||||
Thank you for your interest in contributing to tzst! This document provides guidelines and information for contributors.
|
||||
|
||||
## Table of Contents
|
||||
|
||||
- [Code of Conduct](#code-of-conduct)
|
||||
- [Getting Started](#getting-started)
|
||||
- [Development Setup](#development-setup)
|
||||
- [Project Structure](#project-structure)
|
||||
- [Making Changes](#making-changes)
|
||||
- [Testing](#testing)
|
||||
- [Code Style](#code-style)
|
||||
- [Submitting Changes](#submitting-changes)
|
||||
- [Release Process](#release-process)
|
||||
- [Getting Help](#getting-help)
|
||||
|
||||
## Code of Conduct
|
||||
|
||||
This project follows the principles of respectful collaboration. Please be kind, constructive, and professional in all interactions.
|
||||
|
||||
## Getting Started
|
||||
|
||||
### Prerequisites
|
||||
|
||||
- Python 3.12 or higher
|
||||
- Git
|
||||
- Basic knowledge of tar archives and compression
|
||||
|
||||
### Development Setup
|
||||
|
||||
1. **Fork and clone the repository:**
|
||||
|
||||
```bash
|
||||
git clone https://github.com/xixu-me/tzst.git
|
||||
cd tzst
|
||||
```
|
||||
|
||||
2. **Create a virtual environment:**
|
||||
|
||||
```bash
|
||||
python -m venv venv
|
||||
# On Windows
|
||||
venv\Scripts\activate
|
||||
# On Unix/macOS
|
||||
source venv/bin/activate
|
||||
```
|
||||
|
||||
3. **Install the package in development mode:**
|
||||
|
||||
```bash
|
||||
pip install -e .[dev]
|
||||
```
|
||||
|
||||
4. **Verify the installation:**
|
||||
|
||||
```bash
|
||||
python -m pytest tests/
|
||||
```
|
||||
|
||||
## Project Structure
|
||||
|
||||
```
|
||||
tzst/
|
||||
├── src/tzst/ # Main package source code
|
||||
│ ├── __init__.py # Package initialization and exports
|
||||
│ ├── __main__.py # CLI entry point
|
||||
│ ├── cli.py # Command-line interface
|
||||
│ ├── core.py # Core archive functionality
|
||||
│ └── exceptions.py # Custom exceptions
|
||||
├── tests/ # Test suite
|
||||
│ ├── conftest.py # Pytest configuration and fixtures
|
||||
│ ├── test_core.py # Core functionality tests
|
||||
│ ├── test_cli.py # CLI tests
|
||||
│ └── test_*.py # Additional test modules
|
||||
├── .github/ # GitHub workflows and templates
|
||||
├── pyproject.toml # Project configuration
|
||||
├── README.md # Project documentation
|
||||
├── LICENSE # BSD 3-Clause License
|
||||
└── CONTRIBUTING.md # This file
|
||||
```
|
||||
|
||||
## Making Changes
|
||||
|
||||
### Types of Contributions
|
||||
|
||||
We welcome various types of contributions:
|
||||
|
||||
- **Bug fixes**: Fix issues in existing functionality
|
||||
- **Features**: Add new capabilities to the library
|
||||
- **Documentation**: Improve or add documentation
|
||||
- **Tests**: Add or improve test coverage
|
||||
- **Performance**: Optimize existing code
|
||||
- **Security**: Address security vulnerabilities
|
||||
|
||||
### Branch Naming
|
||||
|
||||
Use descriptive branch names:
|
||||
|
||||
- `feature/add-streaming-mode`
|
||||
- `fix/handle-corrupted-archives`
|
||||
- `docs/improve-api-documentation`
|
||||
- `test/add-compression-tests`
|
||||
|
||||
### Commit Messages
|
||||
|
||||
Follow conventional commit format:
|
||||
|
||||
```
|
||||
type(scope): description
|
||||
|
||||
[optional body]
|
||||
|
||||
[optional footer]
|
||||
```
|
||||
|
||||
Types:
|
||||
|
||||
- `feat`: New feature
|
||||
- `fix`: Bug fix
|
||||
- `docs`: Documentation changes
|
||||
- `test`: Adding or modifying tests
|
||||
- `refactor`: Code refactoring
|
||||
- `perf`: Performance improvements
|
||||
- `chore`: Build process or auxiliary tool changes
|
||||
|
||||
Examples:
|
||||
|
||||
```
|
||||
feat(core): add streaming compression support
|
||||
|
||||
fix(cli): handle invalid archive paths gracefully
|
||||
|
||||
docs(readme): update installation instructions
|
||||
```
|
||||
|
||||
## Testing
|
||||
|
||||
### Running Tests
|
||||
|
||||
```bash
|
||||
# Run all tests
|
||||
python -m pytest
|
||||
|
||||
# Run with coverage
|
||||
python -m pytest --cov
|
||||
|
||||
# Run specific test file
|
||||
python -m pytest tests/test_core.py
|
||||
|
||||
# Run with verbose output
|
||||
python -m pytest -v
|
||||
|
||||
# Run integration tests only
|
||||
python -m pytest -m integration
|
||||
```
|
||||
|
||||
### Test Structure
|
||||
|
||||
- Unit tests: Test individual functions and methods
|
||||
- Integration tests: Test component interactions
|
||||
- CLI tests: Test command-line interface
|
||||
- Platform-specific tests: Test OS-specific functionality
|
||||
|
||||
### Writing Tests
|
||||
|
||||
1. **Use descriptive test names:**
|
||||
|
||||
```python
|
||||
def test_create_archive_with_compression_level_9():
|
||||
```
|
||||
|
||||
2. **Use fixtures for common test data:**
|
||||
|
||||
```python
|
||||
def test_extract_archive(sample_archive_path, temp_dir):
|
||||
```
|
||||
|
||||
3. **Test edge cases:**
|
||||
- Empty files
|
||||
- Large files
|
||||
- Invalid inputs
|
||||
- Corrupted archives
|
||||
|
||||
4. **Add markers for test categorization:**
|
||||
|
||||
```python
|
||||
@pytest.mark.integration
|
||||
def test_full_archive_workflow():
|
||||
```
|
||||
|
||||
## Code Style
|
||||
|
||||
This project uses [Ruff](https://docs.astral.sh/ruff/) for linting and formatting.
|
||||
|
||||
### Configuration
|
||||
|
||||
Settings are defined in `pyproject.toml`:
|
||||
|
||||
- Line length: 88 characters
|
||||
- Target Python version: 3.12+
|
||||
- Import sorting with isort
|
||||
- Quote style: double quotes
|
||||
|
||||
### Running Code Style Tools
|
||||
|
||||
```bash
|
||||
# Check code style
|
||||
ruff check .
|
||||
|
||||
# Fix auto-fixable issues
|
||||
ruff check --fix .
|
||||
|
||||
# Format code
|
||||
ruff format .
|
||||
|
||||
# Check formatting without making changes
|
||||
ruff format --check .
|
||||
```
|
||||
|
||||
### Code Style Guidelines
|
||||
|
||||
1. **Follow PEP 8** with project-specific modifications
|
||||
2. **Use type hints** for all public APIs
|
||||
3. **Write docstrings** for classes and public methods
|
||||
4. **Keep functions focused** and reasonably sized
|
||||
5. **Use meaningful variable names**
|
||||
6. **Add comments** for complex logic
|
||||
|
||||
### Documentation Strings
|
||||
|
||||
Use Google-style docstrings:
|
||||
|
||||
```python
|
||||
def create_archive(
|
||||
archive_path: str | Path,
|
||||
files: Sequence[str | Path],
|
||||
compression_level: int = 3,
|
||||
) -> None:
|
||||
"""Create a new tzst archive containing the specified files.
|
||||
|
||||
Args:
|
||||
archive_path: Path where the archive will be created.
|
||||
files: Sequence of file/directory paths to include.
|
||||
compression_level: Zstandard compression level (1-22).
|
||||
|
||||
Raises:
|
||||
TzstArchiveError: If archive creation fails.
|
||||
FileNotFoundError: If input files don't exist.
|
||||
"""
|
||||
```
|
||||
|
||||
## Submitting Changes
|
||||
|
||||
### Pull Request Process
|
||||
|
||||
1. **Create a feature branch:**
|
||||
|
||||
```bash
|
||||
git checkout -b feature/your-feature-name
|
||||
```
|
||||
|
||||
2. **Make your changes** following the guidelines above
|
||||
|
||||
3. **Add tests** for new functionality
|
||||
|
||||
4. **Update documentation** if needed
|
||||
|
||||
5. **Run the test suite:**
|
||||
|
||||
```bash
|
||||
python -m pytest
|
||||
ruff check .
|
||||
ruff format --check .
|
||||
```
|
||||
|
||||
6. **Commit your changes:**
|
||||
|
||||
```bash
|
||||
git add .
|
||||
git commit -m "feat: add your feature description"
|
||||
```
|
||||
|
||||
7. **Push to your fork:**
|
||||
|
||||
```bash
|
||||
git push origin feature/your-feature-name
|
||||
```
|
||||
|
||||
8. **Create a pull request** using the provided template
|
||||
|
||||
### Pull Request Guidelines
|
||||
|
||||
- **Fill out the PR template** completely
|
||||
- **Link related issues** using keywords (fixes #123)
|
||||
- **Keep PRs focused** - one feature/fix per PR
|
||||
- **Ensure all CI checks pass**
|
||||
- **Respond to review feedback** promptly
|
||||
|
||||
### Review Process
|
||||
|
||||
1. Automated checks must pass (CI/CD, code style)
|
||||
2. Code review by maintainers
|
||||
3. Address any feedback or requested changes
|
||||
4. Final approval and merge by maintainers
|
||||
|
||||
## Release Process
|
||||
|
||||
Releases are handled by maintainers:
|
||||
|
||||
1. Update version in `src/tzst/__init__.py`
|
||||
2. Create a release tag
|
||||
3. Automated CI/CD publishes to PyPI
|
||||
|
||||
## Getting Help
|
||||
|
||||
### Resources
|
||||
|
||||
- **Issues**: [GitHub Issues](https://github.com/xixu-me/tzst/issues)
|
||||
- **Discussions**: Use GitHub Discussions for questions
|
||||
- **Documentation**: Check the README and code comments
|
||||
|
||||
### Reporting Issues
|
||||
|
||||
When reporting bugs:
|
||||
|
||||
1. **Use the bug report template**
|
||||
2. **Provide a minimal reproduction case**
|
||||
3. **Include system information** (OS, Python version)
|
||||
4. **Attach relevant files** if possible (archives, logs)
|
||||
|
||||
### Suggesting Features
|
||||
|
||||
When suggesting features:
|
||||
|
||||
1. **Use the feature request template**
|
||||
2. **Explain the use case** and motivation
|
||||
3. **Consider backwards compatibility**
|
||||
4. **Provide implementation ideas** if you have them
|
||||
|
||||
## Development Tips
|
||||
|
||||
### Performance Considerations
|
||||
|
||||
- Use streaming for large files
|
||||
- Consider memory usage patterns
|
||||
- Profile code for bottlenecks
|
||||
- Test with various file sizes
|
||||
|
||||
### Security Considerations
|
||||
|
||||
- Validate all user inputs
|
||||
- Use secure defaults (e.g., 'data' filter)
|
||||
- Handle malicious archives safely
|
||||
- Be cautious with file paths
|
||||
|
||||
### Compatibility
|
||||
|
||||
- Support Python 3.12+
|
||||
- Test on multiple platforms (Windows, macOS, Linux)
|
||||
- Consider different filesystem behaviors
|
||||
- Maintain backwards compatibility when possible
|
||||
|
||||
## Recognition
|
||||
|
||||
Contributors are recognized in several ways:
|
||||
|
||||
- Listed in release notes for significant contributions
|
||||
- Mentioned in README acknowledgments
|
||||
- GitHub contributor statistics
|
||||
|
||||
Thank you for contributing to tzst! Your efforts help make this library better for everyone.
|
||||
@@ -1,26 +1,26 @@
|
||||
# tzst
|
||||
|
||||
[](https://codecov.io/gh/xixu-me/tzst)
|
||||
[](https://github.com/xixu-me/tzst/actions/workflows/github-code-scanning/codeql)
|
||||
[](https://github.com/xixu-me/tzst/actions/workflows/ci.yml)
|
||||
[](https://pypi.org/project/tzst/)
|
||||
[](LICENSE)
|
||||
[](https://xi-xu.me/#sponsorships)
|
||||
|
||||
A Python library for creating and manipulating `.tzst`/`.tar.zst` archives using Zstandard compression.
|
||||
**tzst** is a next-generation Python library engineered for modern archive management, leveraging cutting-edge Zstandard compression to deliver superior performance, security, and reliability. Built exclusively for Python 3.12+, this enterprise-grade solution combines atomic operations, streaming efficiency, and a meticulously crafted API to redefine how developers handle `.tzst`/`.tar.zst` archives in production environments.
|
||||
|
||||
## Features
|
||||
|
||||
- **High Compression**: Uses Zstandard compression for excellent compression ratios and speed
|
||||
- **High Compression**: Zstandard compression for excellent compression ratios and speed
|
||||
- **Tar Compatibility**: Creates standard tar archives compressed with Zstandard
|
||||
- **Command Line Interface**: Easy-to-use CLI with intuitive commands and streaming support
|
||||
- **Command Line Interface**: Intuitive CLI with streaming support and comprehensive options
|
||||
- **Python API**: Clean, Pythonic API for programmatic use
|
||||
- **Cross-Platform**: Works on Windows, macOS, and Linux
|
||||
- **Multiple Extensions**: Supports both `.tzst` and `.tar.zst` extensions
|
||||
- **Flexible Extraction**: Extract with full paths or flatten directory structure
|
||||
- **Memory Efficient**: Streaming mode for handling large archives with minimal memory usage
|
||||
- **Atomic Operations**: Safe file operations with automatic cleanup on interruption
|
||||
- **Enhanced Error Handling**: Clear error messages with helpful alternatives and suggestions
|
||||
- **Secure by Default**: Uses the 'data' filter for maximum security during extraction
|
||||
- **Enhanced Error Handling**: Clear error messages with helpful alternatives
|
||||
|
||||
## Installation
|
||||
|
||||
@@ -40,7 +40,7 @@ pip install .
|
||||
|
||||
### Development Installation
|
||||
|
||||
This project uses [Hatch](https://hatch.pypa.io/) as the build system, configured in `pyproject.toml`. For development:
|
||||
This project uses [Hatch](https://hatch.pypa.io/) as the build system:
|
||||
|
||||
```bash
|
||||
git clone https://github.com/xixu-me/tzst.git
|
||||
@@ -48,11 +48,9 @@ cd tzst
|
||||
pip install -e .[dev]
|
||||
```
|
||||
|
||||
Alternatively, if you have [Hatch](https://hatch.pypa.io/) installed:
|
||||
Alternatively, with [Hatch](https://hatch.pypa.io/) installed:
|
||||
|
||||
```bash
|
||||
git clone https://github.com/xixu-me/tzst.git
|
||||
cd tzst
|
||||
hatch env create
|
||||
hatch shell
|
||||
```
|
||||
@@ -61,7 +59,7 @@ hatch shell
|
||||
|
||||
### Command Line Usage
|
||||
|
||||
> **Recommended**: The `uvx tzst` command is highly recommended for running tzst without installation and with significantly better performance. [uv](https://github.com/astral-sh/uv) provides faster package resolution and execution compared to standard pip/python approaches. See the [uv's documentation](https://docs.astral.sh/uv/) for more details.
|
||||
> **Recommended**: Use `uvx tzst` for running without installation and better performance. See [uv documentation](https://docs.astral.sh/uv/) for details.
|
||||
|
||||
```bash
|
||||
# Create an archive
|
||||
@@ -96,8 +94,6 @@ for item in contents:
|
||||
|
||||
## Command Line Interface
|
||||
|
||||
The `tzst` command provides a comprehensive CLI for archive operations:
|
||||
|
||||
### Archive Operations
|
||||
|
||||
#### Create Archive
|
||||
@@ -109,9 +105,6 @@ tzst a archive.tzst file1.txt file2.txt
|
||||
# With compression level (1-22, default: 3)
|
||||
tzst a archive.tzst files/ -l 15
|
||||
|
||||
# Disable atomic file operations (not recommended)
|
||||
tzst a archive.tzst files/ --no-atomic
|
||||
|
||||
# Alternative commands
|
||||
tzst add archive.tzst files/
|
||||
tzst create archive.tzst files/
|
||||
@@ -132,7 +125,7 @@ tzst x archive.tzst file1.txt dir/file2.txt
|
||||
# Extract without directory structure (flat)
|
||||
tzst e archive.tzst -o output/
|
||||
|
||||
# Use streaming mode for large archives (reduces memory usage)
|
||||
# Use streaming mode for large archives
|
||||
tzst x archive.tzst --streaming -o output/
|
||||
```
|
||||
|
||||
@@ -155,7 +148,7 @@ tzst l archive.tzst --streaming -v
|
||||
# Test archive integrity
|
||||
tzst t archive.tzst
|
||||
|
||||
# Test with streaming mode for large archives
|
||||
# Test with streaming mode
|
||||
tzst t archive.tzst --streaming
|
||||
```
|
||||
|
||||
@@ -171,37 +164,14 @@ tzst t archive.tzst --streaming
|
||||
|
||||
### CLI Options
|
||||
|
||||
#### Global Options
|
||||
|
||||
- `-v, --verbose`: Enable verbose output
|
||||
- `-o, --output DIR`: Specify output directory (extract commands)
|
||||
- `-l, --level LEVEL`: Set compression level 1-22 (create command)
|
||||
- `--streaming`: Enable streaming mode for memory-efficient processing of large archives
|
||||
- `--filter FILTER`: Security filter for extraction (extract commands only)
|
||||
- `--no-atomic`: Disable atomic file operations (create command only, not recommended)
|
||||
- `--streaming`: Enable streaming mode for memory-efficient processing
|
||||
- `--filter FILTER`: Security filter for extraction (data/tar/fully_trusted)
|
||||
- `--no-atomic`: Disable atomic file operations (not recommended)
|
||||
|
||||
#### Streaming Mode
|
||||
|
||||
The `--streaming` flag is available for extract, list, and test operations:
|
||||
|
||||
```bash
|
||||
# Memory-efficient operations on large archives
|
||||
tzst x large_archive.tzst --streaming
|
||||
tzst l large_archive.tzst --streaming -v
|
||||
tzst t large_archive.tzst --streaming
|
||||
```
|
||||
|
||||
**Benefits of streaming mode:**
|
||||
|
||||
- Significantly reduced memory usage for large archives
|
||||
- Better performance when processing archives that don't fit in memory
|
||||
- Automatic cleanup of resources
|
||||
|
||||
**Note:** Some advanced operations may be limited in streaming mode.
|
||||
|
||||
#### Security Filters
|
||||
|
||||
For enhanced security when extracting archives from untrusted sources, tzst provides extraction filters:
|
||||
### Security Filters
|
||||
|
||||
```bash
|
||||
# Extract with maximum security (default)
|
||||
@@ -216,18 +186,14 @@ tzst x archive.tzst --filter fully_trusted
|
||||
|
||||
**Security Filter Options:**
|
||||
|
||||
- `data` (default, recommended): Most secure option. Blocks dangerous files like device files, absolute paths, and paths outside the extraction directory. Sets safe permissions and clears user/group metadata.
|
||||
- `tar`: Standard tar compatibility. Blocks absolute paths and directory traversal but allows more file types and metadata.
|
||||
- `fully_trusted`: No security restrictions. Only use with completely trusted archives as it can be exploited for path traversal attacks.
|
||||
|
||||
**Security Warning:** Always use the default `data` filter when extracting archives from untrusted sources. Never use `fully_trusted` unless you completely trust the archive source.
|
||||
- `data` (default): Most secure. Blocks dangerous files, absolute paths, and paths outside extraction directory
|
||||
- `tar`: Standard tar compatibility. Blocks absolute paths and directory traversal
|
||||
- `fully_trusted`: No security restrictions. Only use with completely trusted archives
|
||||
|
||||
## Python API
|
||||
|
||||
### TzstArchive Class
|
||||
|
||||
The main class for working with tzst archives:
|
||||
|
||||
```python
|
||||
from tzst import TzstArchive
|
||||
|
||||
@@ -241,34 +207,20 @@ with TzstArchive("archive.tzst", "r") as archive:
|
||||
# List contents
|
||||
contents = archive.list(verbose=True)
|
||||
|
||||
# Extract specific file with security filter
|
||||
# Extract with security filter
|
||||
archive.extract("file.txt", "output/", filter="data")
|
||||
|
||||
# Extract all files (uses 'data' filter by default for security)
|
||||
archive.extract(path="output/")
|
||||
|
||||
# Extract with different security levels
|
||||
archive.extract(path="output/", filter="tar") # Standard tar compatibility
|
||||
archive.extract(path="output/", filter="data") # Maximum security (default)
|
||||
# archive.extract(path="output/", filter="fully_trusted") # Only for trusted archives!
|
||||
|
||||
# Test integrity
|
||||
is_valid = archive.test()
|
||||
|
||||
# For large archives, use streaming mode to reduce memory usage
|
||||
# For large archives, use streaming mode
|
||||
with TzstArchive("large_archive.tzst", "r", streaming=True) as archive:
|
||||
# Streaming mode is more memory efficient but may limit some operations
|
||||
contents = archive.list(verbose=True)
|
||||
archive.extract(path="output/")
|
||||
```
|
||||
|
||||
**Important Limitations:**
|
||||
|
||||
- **Append Mode Not Supported**: The `TzstArchive` class does not support append mode (`"a"`). If you need to add files to an existing archive, you must either:
|
||||
1. Create multiple separate archives
|
||||
2. Recreate the entire archive with all files
|
||||
3. Use standard tar and compress separately with external tools
|
||||
4. Extract the existing archive, add new files, and recompress
|
||||
- **Append Mode Not Supported**: Create multiple archives or recreate the entire archive instead
|
||||
|
||||
### Convenience Functions
|
||||
|
||||
@@ -277,29 +229,20 @@ with TzstArchive("large_archive.tzst", "r", streaming=True) as archive:
|
||||
```python
|
||||
from tzst import create_archive
|
||||
|
||||
# Create archive with atomic file operations (default behavior)
|
||||
# Create with atomic operations (default)
|
||||
create_archive(
|
||||
archive_path="backup.tzst",
|
||||
files=["documents/", "photos/", "config.txt"],
|
||||
compression_level=10
|
||||
)
|
||||
|
||||
# Disable atomic operations if needed (not recommended)
|
||||
create_archive(
|
||||
archive_path="backup.tzst",
|
||||
files=["documents/"],
|
||||
use_temp_file=False
|
||||
)
|
||||
```
|
||||
|
||||
**Atomic File Operations**: By default, `create_archive()` uses atomic file operations to prevent incomplete archives if the process is interrupted. The archive is first created in a temporary file, then atomically moved to the final location upon successful completion.
|
||||
|
||||
#### extract_archive()
|
||||
|
||||
```python
|
||||
from tzst import extract_archive
|
||||
|
||||
# Extract with directory structure (uses 'data' filter by default for security)
|
||||
# Extract with security (default: 'data' filter)
|
||||
extract_archive("backup.tzst", "restore/")
|
||||
|
||||
# Extract specific files
|
||||
@@ -308,11 +251,7 @@ extract_archive("backup.tzst", "restore/", members=["config.txt"])
|
||||
# Flatten directory structure
|
||||
extract_archive("backup.tzst", "restore/", flatten=True)
|
||||
|
||||
# Extract with different security filters
|
||||
extract_archive("backup.tzst", "restore/", filter="data") # Maximum security (default)
|
||||
extract_archive("backup.tzst", "restore/", filter="tar") # Standard tar compatibility
|
||||
|
||||
# For large archives, use streaming mode for memory efficiency
|
||||
# Use streaming for large archives
|
||||
extract_archive("large_backup.tzst", "restore/", streaming=True)
|
||||
```
|
||||
|
||||
@@ -323,17 +262,12 @@ from tzst import list_archive
|
||||
|
||||
# Simple listing
|
||||
files = list_archive("backup.tzst")
|
||||
for file_info in files:
|
||||
print(file_info["name"])
|
||||
|
||||
# Detailed listing
|
||||
files = list_archive("backup.tzst", verbose=True)
|
||||
for file_info in files:
|
||||
print(f"{file_info['name']}: {file_info['size']} bytes, "
|
||||
f"modified: {file_info['mtime_str']}")
|
||||
|
||||
# Use streaming mode for large archives
|
||||
files = list_archive("large_backup.tzst", verbose=True, streaming=True)
|
||||
# Streaming for large archives
|
||||
files = list_archive("large_backup.tzst", streaming=True)
|
||||
```
|
||||
|
||||
#### test_archive()
|
||||
@@ -344,15 +278,15 @@ from tzst import test_archive
|
||||
# Basic integrity test
|
||||
if test_archive("backup.tzst"):
|
||||
print("Archive is valid")
|
||||
else:
|
||||
print("Archive is corrupted")
|
||||
|
||||
# Test large archive with streaming mode
|
||||
# Test with streaming
|
||||
if test_archive("large_backup.tzst", streaming=True):
|
||||
print("Large archive is valid")
|
||||
```
|
||||
|
||||
## File Extensions
|
||||
## Advanced Features
|
||||
|
||||
### File Extensions
|
||||
|
||||
The library automatically handles file extensions with intelligent normalization:
|
||||
|
||||
@@ -361,13 +295,6 @@ The library automatically handles file extensions with intelligent normalization
|
||||
- Auto-detection when opening existing archives
|
||||
- Automatic extension addition when creating archives
|
||||
|
||||
**Extension Behavior:**
|
||||
|
||||
- If no extension is provided, `.tzst` is automatically added
|
||||
- Inconsistent extensions (e.g., `.txt`) are normalized to `.tzst`
|
||||
- Both `.tzst` and `.tar.zst` are treated as valid and equivalent
|
||||
- Opening archives automatically detects the correct format regardless of extension
|
||||
|
||||
```python
|
||||
# These all create valid archives
|
||||
create_archive("backup.tzst", files) # Creates backup.tzst
|
||||
@@ -376,7 +303,7 @@ create_archive("backup", files) # Creates backup.tzst
|
||||
create_archive("backup.txt", files) # Creates backup.tzst (normalized)
|
||||
```
|
||||
|
||||
## Compression Levels
|
||||
### Compression Levels
|
||||
|
||||
Zstandard compression levels range from 1 (fastest) to 22 (best compression):
|
||||
|
||||
@@ -385,20 +312,52 @@ Zstandard compression levels range from 1 (fastest) to 22 (best compression):
|
||||
- **Level 10-15**: Better compression, slower
|
||||
- **Level 20-22**: Maximum compression, much slower
|
||||
|
||||
### Streaming Mode
|
||||
|
||||
Use streaming mode for memory-efficient processing of large archives:
|
||||
|
||||
**Benefits:**
|
||||
|
||||
- Significantly reduced memory usage
|
||||
- Better performance for archives that don't fit in memory
|
||||
- Automatic cleanup of resources
|
||||
|
||||
**When to Use:**
|
||||
|
||||
- Archives larger than 100MB
|
||||
- Limited memory environments
|
||||
- Processing archives with many large files
|
||||
|
||||
```python
|
||||
# Fast compression
|
||||
create_archive("fast.tzst", files, compression_level=1)
|
||||
# Example: Processing a large backup archive
|
||||
from tzst import extract_archive, list_archive, test_archive
|
||||
|
||||
# Balanced (default)
|
||||
create_archive("balanced.tzst", files, compression_level=3)
|
||||
large_archive = "backup_500gb.tzst"
|
||||
|
||||
# Maximum compression
|
||||
create_archive("compressed.tzst", files, compression_level=22)
|
||||
# Memory-efficient operations
|
||||
is_valid = test_archive(large_archive, streaming=True)
|
||||
contents = list_archive(large_archive, streaming=True, verbose=True)
|
||||
extract_archive(large_archive, "restore/", streaming=True)
|
||||
```
|
||||
|
||||
## Error Handling and Recovery
|
||||
### Atomic Operations
|
||||
|
||||
The library provides comprehensive error handling with specific exception types and helpful error messages:
|
||||
All file creation operations use atomic file operations by default:
|
||||
|
||||
- Archives created in temporary files first, then atomically moved
|
||||
- Automatic cleanup if process is interrupted
|
||||
- No risk of corrupted or incomplete archives
|
||||
- Cross-platform compatibility
|
||||
|
||||
```python
|
||||
# Atomic operations enabled by default
|
||||
create_archive("important.tzst", files) # Safe from interruption
|
||||
|
||||
# Can be disabled if needed (not recommended)
|
||||
create_archive("test.tzst", files, use_temp_file=False)
|
||||
```
|
||||
|
||||
### Error Handling
|
||||
|
||||
```python
|
||||
from tzst import TzstArchive
|
||||
@@ -415,161 +374,41 @@ try:
|
||||
archive.extract()
|
||||
except TzstDecompressionError:
|
||||
print("Failed to decompress archive")
|
||||
except TzstArchiveError:
|
||||
print("Archive operation failed")
|
||||
except TzstFileNotFoundError:
|
||||
print("Archive file not found")
|
||||
except KeyboardInterrupt:
|
||||
print("Operation interrupted by user")
|
||||
# Cleanup is handled automatically
|
||||
# Cleanup handled automatically
|
||||
```
|
||||
|
||||
### Enhanced Error Messages
|
||||
## Performance and Comparison
|
||||
|
||||
The library now provides enhanced error messages with clear alternatives:
|
||||
### Performance Tips
|
||||
|
||||
```python
|
||||
# Append mode is not supported, but errors provide helpful alternatives
|
||||
try:
|
||||
with TzstArchive("archive.tzst", "a") as archive:
|
||||
archive.add("newfile.txt")
|
||||
except NotImplementedError as e:
|
||||
print(e) # Detailed message with alternatives:
|
||||
# "Append mode is not supported for tzst archives.
|
||||
# Alternatives: 1) Create multiple archives, 2) Recreate the archive,
|
||||
# 3) Use standard tar and compress separately."
|
||||
```
|
||||
1. **Compression levels**: Level 3 is optimal for most use cases
|
||||
2. **Streaming**: Use for archives larger than 100MB
|
||||
3. **Batch operations**: Add multiple files in single session
|
||||
4. **File types**: Already compressed files won't compress much further
|
||||
|
||||
## Safety and Recovery
|
||||
### vs Other Tools
|
||||
|
||||
### Atomic File Operations
|
||||
**vs tar + gzip:**
|
||||
|
||||
All file creation operations use atomic file operations by default to ensure data integrity:
|
||||
- Better compression ratios
|
||||
- Faster decompression
|
||||
- Modern algorithm
|
||||
|
||||
- **Archive Creation**: Archives are created in temporary files first, then atomically moved to final location
|
||||
- **Interruption Safety**: Automatic cleanup if process is interrupted (Ctrl+C, system shutdown)
|
||||
- **No Partial Files**: No risk of corrupted or incomplete archives in the final location
|
||||
- **Cross-Platform**: Works reliably on Windows, macOS, and Linux file systems
|
||||
**vs tar + xz:**
|
||||
|
||||
**Which Operations Are Atomic:**
|
||||
- Significantly faster compression
|
||||
- Similar compression ratios
|
||||
- Better speed/compression trade-off
|
||||
|
||||
- `create_archive()` function (when `use_temp_file=True`, which is default)
|
||||
- Creating new archives via `TzstArchive` class in write mode
|
||||
- CLI archive creation commands (`tzst a`, `tzst add`, `tzst create`)
|
||||
**vs zip:**
|
||||
|
||||
**Which Operations Are Not Atomic:**
|
||||
|
||||
- Extraction operations (files are written directly to destination)
|
||||
- Reading operations (no file modifications)
|
||||
- Operations with `use_temp_file=False` (not recommended)
|
||||
|
||||
```python
|
||||
# Atomic operations are enabled by default
|
||||
create_archive("important.tzst", files) # Safe from interruption
|
||||
|
||||
# Can be disabled if needed (not recommended)
|
||||
create_archive("test.tzst", files, use_temp_file=False)
|
||||
|
||||
# Archive class also uses atomic operations
|
||||
with TzstArchive("backup.tzst", "w") as archive:
|
||||
archive.add("documents/") # Safe from interruption
|
||||
```
|
||||
|
||||
### Enhanced Error Messages
|
||||
|
||||
The library provides comprehensive error handling with specific exception types and helpful error messages:
|
||||
|
||||
```python
|
||||
# Append mode example with helpful alternatives
|
||||
try:
|
||||
with TzstArchive("archive.tzst", "a") as archive:
|
||||
archive.add("newfile.txt")
|
||||
except NotImplementedError as e:
|
||||
print(e) # Detailed message with alternatives:
|
||||
# "Append mode is not supported for tzst archives.
|
||||
# Alternatives: 1) Create multiple archives, 2) Recreate the archive,
|
||||
# 3) Use standard tar and compress separately."
|
||||
```
|
||||
|
||||
### Recovery and Cleanup
|
||||
|
||||
The library automatically handles cleanup in various failure scenarios:
|
||||
|
||||
- **Process Interruption**: Temporary files are automatically cleaned up
|
||||
- **Disk Space Issues**: Partial files are removed if creation fails
|
||||
- **Permission Errors**: No incomplete archives are left behind
|
||||
- **Memory Errors**: Resources are properly released
|
||||
|
||||
## Performance Tips
|
||||
|
||||
1. **Choose appropriate compression levels**: Level 3 is usually optimal for most use cases
|
||||
2. **Use streaming for large archives**: Enable streaming mode (`streaming=True`) for archives larger than 100MB to reduce memory usage significantly
|
||||
3. **Atomic file operations**: The library uses atomic file operations by default to prevent incomplete archives on interruption - archives are created in temporary files first, then moved atomically
|
||||
4. **Batch operations**: Add multiple files in a single archive session when possible
|
||||
5. **Consider file types**: Already compressed files (images, videos) won't compress much further
|
||||
6. **CLI streaming options**: Use `--streaming` flag in CLI commands for memory-efficient processing of large archives
|
||||
7. **Compression level selection**: Higher levels (15-22) provide better compression but take significantly longer
|
||||
|
||||
## Memory Usage and Streaming
|
||||
|
||||
### Memory Usage Guidelines
|
||||
|
||||
- **Small archives (<10MB)**: Standard mode is recommended for simplicity
|
||||
- **Medium archives (10MB-100MB)**: Either mode works well, consider file count and system resources
|
||||
- **Large archives (>100MB)**: Strongly recommend streaming mode to prevent memory exhaustion
|
||||
- **Very large archives (>1GB)**: Always use streaming mode; standard mode may cause system instability
|
||||
- **Limited memory environments**: Use streaming mode regardless of archive size
|
||||
|
||||
### Streaming Mode Benefits
|
||||
|
||||
- **Reduced Memory Usage**: Process archives without loading entire contents into memory
|
||||
- **Large File Support**: Handle archives larger than available RAM
|
||||
- **Better Performance**: Improved performance for sequential access patterns
|
||||
- **Resource Management**: Automatic cleanup of file handles and temporary resources
|
||||
|
||||
### When to Use Streaming
|
||||
|
||||
- Archives larger than 100MB
|
||||
- Limited memory environments
|
||||
- Processing archives with many large files
|
||||
- Automated backup/restore operations
|
||||
|
||||
```python
|
||||
# Example: Processing a large backup archive
|
||||
from tzst import extract_archive, list_archive, test_archive
|
||||
|
||||
# Memory-efficient operations
|
||||
large_archive = "backup_500gb.tzst"
|
||||
|
||||
# Test integrity with minimal memory usage
|
||||
is_valid = test_archive(large_archive, streaming=True)
|
||||
|
||||
# List contents without loading entire archive
|
||||
contents = list_archive(large_archive, streaming=True, verbose=True)
|
||||
|
||||
# Extract with streaming for large archives
|
||||
extract_archive(large_archive, "restore/", streaming=True)
|
||||
```
|
||||
|
||||
## Comparison with Standard Tools
|
||||
|
||||
### vs tar + gzip
|
||||
|
||||
- **Better compression**: Zstandard typically achieves better compression ratios than gzip
|
||||
- **Faster decompression**: Zstandard decompresses faster than gzip
|
||||
- **Modern algorithm**: Zstandard is a more modern compression algorithm
|
||||
|
||||
### vs tar + xz
|
||||
|
||||
- **Faster compression**: Zstandard is significantly faster than xz at similar compression levels
|
||||
- **Comparable compression**: Similar compression ratios to xz
|
||||
- **Better balance**: Better speed/compression trade-off
|
||||
|
||||
### vs zip
|
||||
|
||||
- **Better compression**: Generally better compression than zip
|
||||
- **Preserves permissions**: Maintains Unix file permissions and metadata
|
||||
- **Streaming support**: Better support for large files and streaming
|
||||
- Better compression
|
||||
- Preserves Unix permissions and metadata
|
||||
- Better streaming support
|
||||
|
||||
## Requirements
|
||||
|
||||
@@ -580,9 +419,7 @@ extract_archive(large_archive, "restore/", streaming=True)
|
||||
|
||||
### Setting up Development Environment
|
||||
|
||||
This project uses **Hatch** as the build system and dependency manager, with configuration in `pyproject.toml`. Choose one of the following setup methods:
|
||||
|
||||
#### Using pip (Traditional approach)
|
||||
This project uses **Hatch** as the build system:
|
||||
|
||||
```bash
|
||||
git clone https://github.com/xixu-me/tzst.git
|
||||
@@ -590,107 +427,60 @@ cd tzst
|
||||
pip install -e .[dev]
|
||||
```
|
||||
|
||||
#### Using Hatch (Recommended for development)
|
||||
Or with Hatch:
|
||||
|
||||
```bash
|
||||
pip install hatch
|
||||
hatch env create
|
||||
hatch shell
|
||||
```
|
||||
|
||||
### Running Tests
|
||||
|
||||
```bash
|
||||
# Using pytest
|
||||
pytest --cov=tzst --cov-report=html
|
||||
|
||||
# Using Hatch
|
||||
hatch run pytest --cov=tzst --cov-report=html
|
||||
```
|
||||
|
||||
### Code Quality
|
||||
|
||||
```bash
|
||||
# Check code quality
|
||||
ruff check src tests
|
||||
|
||||
# Format code
|
||||
ruff format src tests
|
||||
```
|
||||
|
||||
## Contributing
|
||||
|
||||
We welcome contributions! Please read our [Contributing Guide](CONTRIBUTING.md) for:
|
||||
|
||||
- Development setup and project structure
|
||||
- Code style guidelines and best practices
|
||||
- Testing requirements and writing tests
|
||||
- Pull request process and review workflow
|
||||
|
||||
### Quick Start for Contributors
|
||||
|
||||
```bash
|
||||
git clone https://github.com/xixu-me/tzst.git
|
||||
cd tzst
|
||||
pip install hatch # Install Hatch if not already installed
|
||||
hatch env create # Create development environment
|
||||
hatch shell # Activate development environment
|
||||
pip install -e .[dev]
|
||||
python -m pytest tests/
|
||||
```
|
||||
|
||||
The `pyproject.toml` file configures the entire build process, including:
|
||||
### Types of Contributions Welcome
|
||||
|
||||
- Build system (hatchling)
|
||||
- Dependencies and optional development dependencies
|
||||
- Project metadata and entry points
|
||||
- Tool configurations (pytest, ruff, black)
|
||||
|
||||
### Running Tests
|
||||
|
||||
#### Using pytest directly
|
||||
|
||||
```bash
|
||||
# Run all tests
|
||||
pytest
|
||||
|
||||
# Run with coverage
|
||||
pytest --cov=tzst --cov-report=html
|
||||
|
||||
# Run specific test file
|
||||
pytest tests/test_core.py
|
||||
```
|
||||
|
||||
#### Using Hatch
|
||||
|
||||
```bash
|
||||
# Run tests in development environment
|
||||
hatch run pytest
|
||||
|
||||
# Run with coverage
|
||||
hatch run pytest --cov=tzst --cov-report=html
|
||||
```
|
||||
|
||||
### Code Quality Tools
|
||||
|
||||
#### Using tools directly
|
||||
|
||||
```bash
|
||||
# Check code quality with Ruff
|
||||
ruff check src tests
|
||||
|
||||
# Format code with Black
|
||||
black src tests
|
||||
|
||||
# Type checking (if mypy is installed)
|
||||
mypy src
|
||||
```
|
||||
|
||||
#### Using Hatch
|
||||
|
||||
```bash
|
||||
# Check code quality with Ruff
|
||||
hatch run ruff check src tests
|
||||
|
||||
# Format code with Black
|
||||
hatch run black src tests
|
||||
```
|
||||
|
||||
### Building and Distribution
|
||||
|
||||
```bash
|
||||
# Install build dependencies
|
||||
pip install build
|
||||
|
||||
# Build wheel and source distribution
|
||||
python -m build
|
||||
|
||||
# Using Hatch for building
|
||||
hatch build
|
||||
```
|
||||
|
||||
### Project Documentation
|
||||
|
||||
The `pyproject.toml` file serves as the central configuration for the entire project:
|
||||
|
||||
```toml
|
||||
[build-system]
|
||||
requires = ["hatchling"]
|
||||
build-backend = "hatchling.build"
|
||||
|
||||
[project]
|
||||
name = "tzst"
|
||||
description = "A Python library for creating and manipulating .tzst/.tar.zst archives"
|
||||
# ... additional metadata
|
||||
```
|
||||
|
||||
Key configuration sections:
|
||||
|
||||
- **Build system**: Uses Hatchling for modern Python packaging
|
||||
- **Dependencies**: Runtime and optional development dependencies
|
||||
- **Entry points**: CLI command registration
|
||||
- **Tool configurations**: pytest, ruff, black, and other development tools
|
||||
- 🐛 **Bug fixes** - Fix issues in existing functionality
|
||||
- ✨ **Features** - Add new capabilities to the library
|
||||
- 📚 **Documentation** - Improve or add documentation
|
||||
- 🧪 **Tests** - Add or improve test coverage
|
||||
- ⚡ **Performance** - Optimize existing code
|
||||
- 🔒 **Security** - Address security vulnerabilities
|
||||
|
||||
## Acknowledgments
|
||||
|
||||
|
||||
@@ -0,0 +1,21 @@
|
||||
# Sphinx build outputs
|
||||
_build/
|
||||
_build_simple/
|
||||
|
||||
# Sphinx auto-generated files
|
||||
_autosummary/
|
||||
|
||||
# Editor files
|
||||
.vscode/
|
||||
*.swp
|
||||
*.swo
|
||||
*~
|
||||
|
||||
# OS files
|
||||
.DS_Store
|
||||
Thumbs.db
|
||||
|
||||
# Python cache
|
||||
__pycache__/
|
||||
*.pyc
|
||||
*.pyo
|
||||
@@ -0,0 +1,33 @@
|
||||
# Minimal makefile for Sphinx documentation
|
||||
#
|
||||
|
||||
# You can set these variables from the command line, and also
|
||||
# from the environment for the first two.
|
||||
SPHINXOPTS ?=
|
||||
SPHINXBUILD ?= sphinx-build
|
||||
SOURCEDIR = .
|
||||
BUILDDIR = _build
|
||||
|
||||
# Put it first so that "make" without argument is like "make help".
|
||||
help:
|
||||
@$(SPHINXBUILD) -M help "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O)
|
||||
|
||||
.PHONY: help Makefile
|
||||
|
||||
# Catch-all target: route all unknown targets to Sphinx using the new
|
||||
# "make mode" option. $(O) is meant as a shortcut for $(SPHINXOPTS).
|
||||
%: Makefile
|
||||
@$(SPHINXBUILD) -M $@ "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O)
|
||||
|
||||
# Custom targets for development
|
||||
clean-all:
|
||||
rm -rf $(BUILDDIR)/*
|
||||
|
||||
livehtml:
|
||||
sphinx-autobuild "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O)
|
||||
|
||||
linkcheck:
|
||||
@$(SPHINXBUILD) -b linkcheck "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O)
|
||||
|
||||
coverage:
|
||||
@$(SPHINXBUILD) -b coverage "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O)
|
||||
+171
@@ -0,0 +1,171 @@
|
||||
# Documentation
|
||||
|
||||
This directory contains the Sphinx documentation for the tzst library.
|
||||
|
||||
## Setup
|
||||
|
||||
1. Install documentation dependencies:
|
||||
|
||||
```bash
|
||||
pip install -r requirements.txt
|
||||
```
|
||||
|
||||
2. Install the tzst package in development mode (required for autodoc):
|
||||
|
||||
```bash
|
||||
pip install -e ..
|
||||
```
|
||||
|
||||
## Building Documentation
|
||||
|
||||
### Local Development
|
||||
|
||||
Build the documentation locally:
|
||||
|
||||
```bash
|
||||
# On Unix/macOS
|
||||
make html
|
||||
|
||||
# On Windows
|
||||
make.bat html
|
||||
```
|
||||
|
||||
The built documentation will be in `_build/html/`. Open `_build/html/index.html` in your browser.
|
||||
|
||||
### Live Reload (Recommended for Development)
|
||||
|
||||
For automatic rebuilding when files change:
|
||||
|
||||
```bash
|
||||
# Install sphinx-autobuild if not already installed
|
||||
pip install sphinx-autobuild
|
||||
|
||||
# Start live reload server
|
||||
make livehtml
|
||||
# or
|
||||
sphinx-autobuild . _build/html
|
||||
```
|
||||
|
||||
This will start a local server (usually at <http://localhost:8000>) that automatically rebuilds and refreshes when you save changes.
|
||||
|
||||
### Other Build Targets
|
||||
|
||||
```bash
|
||||
# Check documentation coverage
|
||||
make coverage
|
||||
|
||||
# Check for broken links
|
||||
make linkcheck
|
||||
|
||||
# Build PDF (requires LaTeX)
|
||||
make latexpdf
|
||||
|
||||
# Clean build directory
|
||||
make clean
|
||||
```
|
||||
|
||||
## Documentation Structure
|
||||
|
||||
- `index.md` - Main documentation homepage
|
||||
- `quickstart.md` - Quick start guide for new users
|
||||
- `examples.md` - Practical examples and use cases
|
||||
- `api/` - API reference documentation
|
||||
- `index.md` - API overview
|
||||
- `core.md` - Core functionality documentation
|
||||
- `cli.md` - CLI documentation
|
||||
- `exceptions.md` - Exception classes documentation
|
||||
|
||||
## Writing Documentation
|
||||
|
||||
### Markdown vs reStructuredText
|
||||
|
||||
This documentation uses MyST parser, which allows you to write in Markdown with some reStructuredText features. You can use either `.md` or `.rst` files.
|
||||
|
||||
### Adding New Pages
|
||||
|
||||
1. Create a new `.md` file in the appropriate directory
|
||||
2. Add it to the relevant `toctree` directive in the parent index file
|
||||
3. Use proper Markdown headers and cross-references
|
||||
|
||||
### API Documentation
|
||||
|
||||
API documentation is automatically generated from docstrings using Sphinx autodoc. To document a new module:
|
||||
|
||||
1. Add the module to the appropriate API file (e.g., `api/core.md`)
|
||||
2. Use autodoc directives like `automodule`, `autoclass`, `autofunction`
|
||||
|
||||
### Code Examples
|
||||
|
||||
Use fenced code blocks with language specification:
|
||||
|
||||
````markdown
|
||||
```python
|
||||
from tzst import TzstArchive
|
||||
|
||||
with TzstArchive("example.tzst", "w") as archive:
|
||||
archive.add("file.txt")
|
||||
```
|
||||
````
|
||||
|
||||
### Cross-References
|
||||
|
||||
Link to other documentation pages:
|
||||
|
||||
```markdown
|
||||
See the {doc}`quickstart` guide for more information.
|
||||
```
|
||||
|
||||
Link to API documentation:
|
||||
|
||||
```markdown
|
||||
Use the {class}`tzst.TzstArchive` class.
|
||||
```
|
||||
|
||||
## Automated Deployment
|
||||
|
||||
Documentation is automatically built and deployed to GitHub Pages when changes are pushed to the main branch. The workflow is defined in `.github/workflows/publish_docs.yml`.
|
||||
|
||||
### Local Testing of Deployment
|
||||
|
||||
To test the deployment process locally:
|
||||
|
||||
1. Build the documentation: `make html`
|
||||
2. Serve the built files: `python -m http.server 8000 -d _build/html`
|
||||
3. Visit <http://localhost:8000>
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### Import Errors
|
||||
|
||||
If you get import errors when building documentation:
|
||||
|
||||
1. Make sure the tzst package is installed: `pip install -e ..`
|
||||
2. Check that all dependencies are installed: `pip install -r requirements.txt`
|
||||
3. Verify your Python path includes the src directory
|
||||
|
||||
### Theme Issues
|
||||
|
||||
If the RTD theme isn't working:
|
||||
|
||||
1. Install the theme: `pip install sphinx-rtd-theme`
|
||||
2. Check that it's listed in `requirements.txt`
|
||||
3. Verify the theme configuration in `conf.py`
|
||||
|
||||
### Build Warnings
|
||||
|
||||
Address all Sphinx warnings to ensure high-quality documentation:
|
||||
|
||||
- Fix broken cross-references
|
||||
- Add missing docstrings
|
||||
- Resolve autodoc import issues
|
||||
- Fix malformed markup
|
||||
|
||||
## Contributing
|
||||
|
||||
When contributing to documentation:
|
||||
|
||||
1. Follow the existing style and structure
|
||||
2. Test your changes locally before submitting
|
||||
3. Add examples for new features
|
||||
4. Update the changelog if appropriate
|
||||
5. Ensure all links work correctly
|
||||
@@ -0,0 +1,44 @@
|
||||
# CLI API
|
||||
|
||||
The command-line interface module provides functions for the tzst CLI tool.
|
||||
|
||||
```{eval-rst}
|
||||
.. automodule:: tzst.cli
|
||||
:members:
|
||||
:undoc-members:
|
||||
:show-inheritance:
|
||||
```
|
||||
|
||||
## Main Functions
|
||||
|
||||
### main
|
||||
|
||||
```{eval-rst}
|
||||
.. autofunction:: tzst.cli.main
|
||||
```
|
||||
|
||||
### create_parser
|
||||
|
||||
```{eval-rst}
|
||||
.. autofunction:: tzst.cli.create_parser
|
||||
```
|
||||
|
||||
## Utility Functions
|
||||
|
||||
### print_banner
|
||||
|
||||
```{eval-rst}
|
||||
.. autofunction:: tzst.cli.print_banner
|
||||
```
|
||||
|
||||
### format_size
|
||||
|
||||
```{eval-rst}
|
||||
.. autofunction:: tzst.cli.format_size
|
||||
```
|
||||
|
||||
### validate_compression_level
|
||||
|
||||
```{eval-rst}
|
||||
.. autofunction:: tzst.cli.validate_compression_level
|
||||
```
|
||||
@@ -0,0 +1,50 @@
|
||||
# Core API
|
||||
|
||||
The core module provides the main functionality for working with tzst archives.
|
||||
|
||||
```{eval-rst}
|
||||
.. automodule:: tzst.core
|
||||
:members:
|
||||
:undoc-members:
|
||||
:show-inheritance:
|
||||
```
|
||||
|
||||
## TzstArchive Class
|
||||
|
||||
The main class for handling `.tzst`/`.tar.zst` archives.
|
||||
|
||||
```{eval-rst}
|
||||
.. autoclass:: tzst.TzstArchive
|
||||
:members:
|
||||
:undoc-members:
|
||||
:show-inheritance:
|
||||
:special-members: __init__, __enter__, __exit__
|
||||
```
|
||||
|
||||
## Convenience Functions
|
||||
|
||||
High-level functions for common archive operations.
|
||||
|
||||
### create_archive
|
||||
|
||||
```{eval-rst}
|
||||
.. autofunction:: tzst.create_archive
|
||||
```
|
||||
|
||||
### extract_archive
|
||||
|
||||
```{eval-rst}
|
||||
.. autofunction:: tzst.extract_archive
|
||||
```
|
||||
|
||||
### list_archive
|
||||
|
||||
```{eval-rst}
|
||||
.. autofunction:: tzst.list_archive
|
||||
```
|
||||
|
||||
### test_archive
|
||||
|
||||
```{eval-rst}
|
||||
.. autofunction:: tzst.test_archive
|
||||
```
|
||||
@@ -0,0 +1,22 @@
|
||||
# Exceptions
|
||||
|
||||
Custom exception classes used by tzst.
|
||||
|
||||
```{eval-rst}
|
||||
.. automodule:: tzst.exceptions
|
||||
:members:
|
||||
:undoc-members:
|
||||
:show-inheritance:
|
||||
```
|
||||
|
||||
## Exception Hierarchy
|
||||
|
||||
```{eval-rst}
|
||||
.. autoexception:: tzst.exceptions.TzstArchiveError
|
||||
:members:
|
||||
:show-inheritance:
|
||||
|
||||
.. autoexception:: tzst.exceptions.TzstDecompressionError
|
||||
:members:
|
||||
:show-inheritance:
|
||||
```
|
||||
@@ -0,0 +1,58 @@
|
||||
# API Reference
|
||||
|
||||
This section contains the complete API documentation for tzst.
|
||||
|
||||
```{toctree}
|
||||
:maxdepth: 2
|
||||
|
||||
core
|
||||
cli
|
||||
exceptions
|
||||
```
|
||||
|
||||
## Overview
|
||||
|
||||
The tzst library provides both high-level convenience functions and a comprehensive class-based API for working with `.tzst`/`.tar.zst` archives.
|
||||
|
||||
### Main Components
|
||||
|
||||
- **{doc}`core`**: Core functionality including `TzstArchive` class and convenience functions
|
||||
- **{doc}`cli`**: Command-line interface functions and utilities
|
||||
- **{doc}`exceptions`**: Custom exception classes for error handling
|
||||
|
||||
### Quick Reference
|
||||
|
||||
#### Core Classes
|
||||
|
||||
```{eval-rst}
|
||||
.. currentmodule:: tzst
|
||||
|
||||
.. autosummary::
|
||||
:nosignatures:
|
||||
|
||||
TzstArchive
|
||||
```
|
||||
|
||||
#### Convenience Functions
|
||||
|
||||
```{eval-rst}
|
||||
.. autosummary::
|
||||
:nosignatures:
|
||||
|
||||
create_archive
|
||||
extract_archive
|
||||
list_archive
|
||||
test_archive
|
||||
```
|
||||
|
||||
#### Exceptions
|
||||
|
||||
```{eval-rst}
|
||||
.. currentmodule:: tzst.exceptions
|
||||
|
||||
.. autosummary::
|
||||
:nosignatures:
|
||||
|
||||
TzstArchiveError
|
||||
TzstDecompressionError
|
||||
```
|
||||
@@ -0,0 +1,157 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Development script for building and serving documentation locally."""
|
||||
|
||||
import argparse
|
||||
import shutil
|
||||
import subprocess
|
||||
import sys
|
||||
import webbrowser
|
||||
from pathlib import Path
|
||||
|
||||
|
||||
def run_command(cmd, cwd=None):
|
||||
"""Run a shell command and return the result.""" try:
|
||||
result = subprocess.run(
|
||||
cmd, shell=True, check=True, cwd=cwd,
|
||||
capture_output=True, text=True
|
||||
)
|
||||
return result.returncode == 0, result.stdout, result.stderr
|
||||
except subprocess.CalledProcessError as e:
|
||||
return False, e.stdout, e.stderr
|
||||
|
||||
|
||||
def clean_build(build_dir):
|
||||
"""Clean the build directory."""
|
||||
if build_dir.exists():
|
||||
print(f"Cleaning {build_dir}")
|
||||
shutil.rmtree(build_dir)
|
||||
|
||||
|
||||
def build_docs(source_dir, build_dir, watch=False):
|
||||
"""Build the documentation."""
|
||||
if watch:
|
||||
print("Starting live reload server...")
|
||||
print("Visit http://localhost:8000 to view the documentation")
|
||||
print("Press Ctrl+C to stop the server")
|
||||
|
||||
cmd = f"sphinx-autobuild {source_dir} {build_dir} --host 0.0.0.0 --port 8000"
|
||||
success, stdout, stderr = run_command(cmd)
|
||||
|
||||
if not success:
|
||||
print("Failed to start live reload server.")
|
||||
print("Make sure sphinx-autobuild is installed: pip install sphinx-autobuild")
|
||||
return False
|
||||
else:
|
||||
print(f"Building documentation: {source_dir} -> {build_dir}")
|
||||
cmd = f"python -m sphinx -b html {source_dir} {build_dir}"
|
||||
success, stdout, stderr = run_command(cmd)
|
||||
|
||||
if success:
|
||||
print("Documentation built successfully!")
|
||||
index_file = build_dir / "index.html"
|
||||
print(f"Open {index_file} in your browser to view the documentation")
|
||||
return True
|
||||
else:
|
||||
print("Build failed!")
|
||||
print("STDOUT:", stdout)
|
||||
print("STDERR:", stderr)
|
||||
return False
|
||||
|
||||
|
||||
def serve_docs(build_dir, port=8000):
|
||||
"""Serve the built documentation locally."""
|
||||
if not build_dir.exists():
|
||||
print(f"Build directory {build_dir} does not exist. Build the docs first.")
|
||||
return False
|
||||
|
||||
print(f"Serving documentation at http://localhost:{port}")
|
||||
print("Press Ctrl+C to stop the server")
|
||||
|
||||
cmd = f"python -m http.server {port}"
|
||||
success, stdout, stderr = run_command(cmd, cwd=build_dir)
|
||||
|
||||
return success
|
||||
|
||||
|
||||
def check_dependencies():
|
||||
"""Check if required dependencies are installed."""
|
||||
try:
|
||||
import sphinx
|
||||
print(f"Sphinx version: {sphinx.__version__}")
|
||||
except ImportError:
|
||||
print("Sphinx is not installed. Install with: pip install sphinx")
|
||||
return False
|
||||
|
||||
try:
|
||||
import tzst
|
||||
print(f"tzst version: {tzst.__version__}")
|
||||
except ImportError:
|
||||
print("tzst package is not installed. Install with: pip install -e ..")
|
||||
return False
|
||||
|
||||
return True
|
||||
|
||||
|
||||
def main():
|
||||
parser = argparse.ArgumentParser(description="Build and serve tzst documentation")
|
||||
parser.add_argument(
|
||||
"command",
|
||||
choices=["build", "clean", "serve", "watch", "check"],
|
||||
help="Command to execute"
|
||||
)
|
||||
parser.add_argument(
|
||||
"--port", "-p",
|
||||
type=int,
|
||||
default=8000,
|
||||
help="Port for serving documentation (default: 8000)"
|
||||
)
|
||||
parser.add_argument(
|
||||
"--open", "-o",
|
||||
action="store_true",
|
||||
help="Open documentation in browser after building/serving"
|
||||
)
|
||||
|
||||
args = parser.parse_args()
|
||||
|
||||
# Get directories
|
||||
script_dir = Path(__file__).parent
|
||||
source_dir = script_dir
|
||||
build_dir = script_dir / "_build"
|
||||
|
||||
if args.command == "check":
|
||||
success = check_dependencies()
|
||||
sys.exit(0 if success else 1)
|
||||
|
||||
elif args.command == "clean":
|
||||
clean_build(build_dir)
|
||||
|
||||
elif args.command == "build":
|
||||
if not check_dependencies():
|
||||
sys.exit(1)
|
||||
|
||||
success = build_docs(source_dir, build_dir)
|
||||
|
||||
if success and args.open:
|
||||
index_file = build_dir / "index.html"
|
||||
webbrowser.open(f"file://{index_file.absolute()}")
|
||||
|
||||
sys.exit(0 if success else 1)
|
||||
|
||||
elif args.command == "watch":
|
||||
if not check_dependencies():
|
||||
sys.exit(1)
|
||||
|
||||
success = build_docs(source_dir, build_dir, watch=True)
|
||||
sys.exit(0 if success else 1)
|
||||
|
||||
elif args.command == "serve":
|
||||
success = serve_docs(build_dir, args.port)
|
||||
|
||||
if args.open:
|
||||
webbrowser.open(f"http://localhost:{args.port}")
|
||||
|
||||
sys.exit(0 if success else 1)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
@@ -0,0 +1,90 @@
|
||||
# Configuration file for the Sphinx documentation builder.
|
||||
|
||||
import os
|
||||
import sys
|
||||
|
||||
# Add the source directory to the Python path
|
||||
sys.path.insert(0, os.path.abspath("../src"))
|
||||
|
||||
# Import version from the package
|
||||
from tzst import __version__
|
||||
|
||||
# -- Project information -----------------------------------------------------
|
||||
project = "tzst"
|
||||
copyright = "2025, Xi Xu"
|
||||
author = "Xi Xu"
|
||||
release = __version__
|
||||
version = __version__
|
||||
|
||||
# -- General configuration ---------------------------------------------------
|
||||
extensions = [
|
||||
"sphinx.ext.autodoc",
|
||||
"sphinx.ext.napoleon",
|
||||
"sphinx.ext.viewcode",
|
||||
"sphinx.ext.intersphinx",
|
||||
"sphinx.ext.autosummary",
|
||||
"myst_parser",
|
||||
]
|
||||
|
||||
templates_path = ["_templates"]
|
||||
exclude_patterns = ["_build", "Thumbs.db", ".DS_Store"]
|
||||
|
||||
# -- Options for HTML output -------------------------------------------------
|
||||
html_theme = "sphinx_rtd_theme"
|
||||
html_static_path = ["_static"]
|
||||
html_title = f"tzst {version} Documentation"
|
||||
html_short_title = "tzst"
|
||||
|
||||
# Theme options
|
||||
html_theme_options = {
|
||||
"canonical_url": "https://xixu-me.github.io/tzst/",
|
||||
"logo_only": False,
|
||||
"display_version": True,
|
||||
"prev_next_buttons_location": "bottom",
|
||||
"style_external_links": False,
|
||||
"style_nav_header_background": "#2980B9",
|
||||
"collapse_navigation": True,
|
||||
"sticky_navigation": True,
|
||||
"navigation_depth": 4,
|
||||
"includehidden": True,
|
||||
"titles_only": False,
|
||||
}
|
||||
|
||||
# -- Extension configuration -------------------------------------------------
|
||||
autodoc_default_options = {
|
||||
"members": True,
|
||||
"undoc-members": True,
|
||||
"show-inheritance": True,
|
||||
"special-members": "__init__",
|
||||
"exclude-members": "__weakref__",
|
||||
}
|
||||
|
||||
napoleon_google_docstring = True
|
||||
napoleon_numpy_docstring = True
|
||||
napoleon_include_init_with_doc = False
|
||||
napoleon_include_private_with_doc = False
|
||||
napoleon_include_special_with_doc = True
|
||||
napoleon_use_param = True
|
||||
napoleon_use_rtype = True
|
||||
|
||||
# Intersphinx mapping
|
||||
intersphinx_mapping = {
|
||||
"python": ("https://docs.python.org/3", None),
|
||||
"zstandard": ("https://python-zstandard.readthedocs.io/en/latest/", None),
|
||||
}
|
||||
|
||||
# Autosummary
|
||||
autosummary_generate = True
|
||||
|
||||
myst_enable_extensions = [
|
||||
"colon_fence",
|
||||
"deflist",
|
||||
"fieldlist",
|
||||
"html_admonition",
|
||||
"linkify",
|
||||
"replacements",
|
||||
"smartquotes",
|
||||
"strikethrough",
|
||||
"substitution",
|
||||
"tasklist",
|
||||
]
|
||||
@@ -0,0 +1,523 @@
|
||||
# Examples
|
||||
|
||||
This page provides practical examples of using tzst in various scenarios.
|
||||
|
||||
## Basic Operations
|
||||
|
||||
### Creating Your First Archive
|
||||
|
||||
```python
|
||||
from tzst import TzstArchive
|
||||
|
||||
# Create a simple archive
|
||||
with TzstArchive("my_first_archive.tzst", "w") as archive:
|
||||
archive.add("important_file.txt")
|
||||
archive.add("documents/", recursive=True)
|
||||
|
||||
print("Archive created successfully!")
|
||||
```
|
||||
|
||||
### Extracting an Archive
|
||||
|
||||
```python
|
||||
from tzst import TzstArchive
|
||||
|
||||
# Extract everything safely
|
||||
with TzstArchive("my_first_archive.tzst", "r") as archive:
|
||||
archive.extract("extracted_files/", filter="data")
|
||||
|
||||
print("Files extracted to extracted_files/")
|
||||
```
|
||||
|
||||
## Advanced Usage
|
||||
|
||||
### High-Compression Backup
|
||||
|
||||
```python
|
||||
from tzst import create_archive
|
||||
import os
|
||||
|
||||
# Create a highly compressed backup
|
||||
def create_backup(source_dirs, backup_name):
|
||||
create_archive(
|
||||
archive_path=f"{backup_name}.tzst",
|
||||
files=source_dirs,
|
||||
compression_level=15, # High compression
|
||||
)
|
||||
|
||||
# Check the compression ratio
|
||||
original_size = sum(
|
||||
os.path.getsize(os.path.join(dirpath, filename))
|
||||
for directory in source_dirs
|
||||
if os.path.exists(directory)
|
||||
for dirpath, dirnames, filenames in os.walk(directory)
|
||||
for filename in filenames
|
||||
)
|
||||
|
||||
compressed_size = os.path.getsize(f"{backup_name}.tzst")
|
||||
ratio = (1 - compressed_size / original_size) * 100
|
||||
|
||||
print(f"Backup created: {backup_name}.tzst")
|
||||
print(f"Compression ratio: {ratio:.1f}%")
|
||||
print(f"Original size: {original_size:,} bytes")
|
||||
print(f"Compressed size: {compressed_size:,} bytes")
|
||||
|
||||
# Usage
|
||||
create_backup(["documents/", "photos/", "projects/"], "full_backup")
|
||||
```
|
||||
|
||||
### Processing Large Archives with Streaming
|
||||
|
||||
```python
|
||||
from tzst import TzstArchive
|
||||
|
||||
def process_large_archive(archive_path, output_dir):
|
||||
"""Process a large archive efficiently using streaming mode."""
|
||||
|
||||
# Use streaming to handle large archives
|
||||
with TzstArchive(archive_path, "r", streaming=True) as archive:
|
||||
# First, list contents to understand what we're dealing with
|
||||
print("Analyzing archive contents...")
|
||||
contents = archive.list(verbose=True)
|
||||
|
||||
total_files = sum(1 for item in contents if item['is_file'])
|
||||
total_size = sum(item['size'] for item in contents if item['is_file'])
|
||||
|
||||
print(f"Archive contains {total_files} files ({total_size:,} bytes)")
|
||||
|
||||
# Extract only specific file types
|
||||
text_files = [item['name'] for item in contents
|
||||
if item['name'].endswith(('.txt', '.md', '.py'))]
|
||||
|
||||
if text_files:
|
||||
print(f"Extracting {len(text_files)} text files...")
|
||||
archive.extract(output_dir, members=text_files, filter="data")
|
||||
|
||||
print("Processing complete!")
|
||||
|
||||
# Usage
|
||||
process_large_archive("large_dataset.tzst", "extracted_text_files/")
|
||||
```
|
||||
|
||||
### Batch Archive Operations
|
||||
|
||||
```python
|
||||
from tzst import test_archive, list_archive
|
||||
import os
|
||||
from pathlib import Path
|
||||
|
||||
def verify_archive_collection(archive_dir):
|
||||
"""Verify integrity of all archives in a directory."""
|
||||
|
||||
archive_dir = Path(archive_dir)
|
||||
archives = list(archive_dir.glob("*.tzst"))
|
||||
|
||||
print(f"Found {len(archives)} archives to verify...")
|
||||
|
||||
results = []
|
||||
for archive_path in archives:
|
||||
print(f"Testing {archive_path.name}...")
|
||||
|
||||
try:
|
||||
# Test integrity
|
||||
is_valid = test_archive(str(archive_path))
|
||||
|
||||
if is_valid:
|
||||
# Get archive info
|
||||
contents = list_archive(str(archive_path), verbose=True)
|
||||
file_count = sum(1 for item in contents if item['is_file'])
|
||||
total_size = sum(item['size'] for item in contents if item['is_file'])
|
||||
|
||||
results.append({
|
||||
'name': archive_path.name,
|
||||
'status': 'Valid',
|
||||
'file_count': file_count,
|
||||
'total_size': total_size
|
||||
})
|
||||
else:
|
||||
results.append({
|
||||
'name': archive_path.name,
|
||||
'status': 'Corrupted',
|
||||
'file_count': 0,
|
||||
'total_size': 0
|
||||
})
|
||||
|
||||
except Exception as e:
|
||||
results.append({
|
||||
'name': archive_path.name,
|
||||
'status': f'Error: {e}',
|
||||
'file_count': 0,
|
||||
'total_size': 0
|
||||
})
|
||||
|
||||
# Print summary
|
||||
print("\n" + "="*60)
|
||||
print("VERIFICATION SUMMARY")
|
||||
print("="*60)
|
||||
|
||||
for result in results:
|
||||
status = result['status']
|
||||
if status == 'Valid':
|
||||
print(f"✓ {result['name']}: {result['file_count']} files, "
|
||||
f"{result['total_size']:,} bytes")
|
||||
else:
|
||||
print(f"✗ {result['name']}: {status}")
|
||||
|
||||
valid_count = sum(1 for r in results if r['status'] == 'Valid')
|
||||
print(f"\nSummary: {valid_count}/{len(results)} archives are valid")
|
||||
|
||||
# Usage
|
||||
verify_archive_collection("backup_archives/")
|
||||
```
|
||||
|
||||
## Security Examples
|
||||
|
||||
### Safe Archive Extraction
|
||||
|
||||
```python
|
||||
from tzst import TzstArchive
|
||||
from tzst.exceptions import TzstArchiveError
|
||||
|
||||
def safe_extract(archive_path, output_dir, max_size_mb=100):
|
||||
"""Safely extract an archive with size limits and security filters."""
|
||||
|
||||
try:
|
||||
with TzstArchive(archive_path, "r") as archive:
|
||||
# First, analyze the archive
|
||||
contents = archive.list(verbose=True)
|
||||
|
||||
# Check total uncompressed size
|
||||
total_size = sum(item['size'] for item in contents if item['is_file'])
|
||||
max_size_bytes = max_size_mb * 1024 * 1024
|
||||
|
||||
if total_size > max_size_bytes:
|
||||
print(f"Warning: Archive is {total_size:,} bytes when uncompressed")
|
||||
print(f"This exceeds the limit of {max_size_bytes:,} bytes")
|
||||
response = input("Continue anyway? (y/N): ")
|
||||
if response.lower() != 'y':
|
||||
return False
|
||||
|
||||
# Check for suspicious files
|
||||
suspicious_files = []
|
||||
for item in contents:
|
||||
name = item['name']
|
||||
# Check for directory traversal attempts
|
||||
if '..' in name or name.startswith('/'):
|
||||
suspicious_files.append(name)
|
||||
# Check for executable files
|
||||
if name.endswith(('.exe', '.bat', '.sh', '.com')):
|
||||
suspicious_files.append(name)
|
||||
|
||||
if suspicious_files:
|
||||
print(f"Warning: Found {len(suspicious_files)} suspicious files:")
|
||||
for file in suspicious_files[:5]: # Show first 5
|
||||
print(f" - {file}")
|
||||
if len(suspicious_files) > 5:
|
||||
print(f" ... and {len(suspicious_files) - 5} more")
|
||||
|
||||
response = input("Continue extraction? (y/N): ")
|
||||
if response.lower() != 'y':
|
||||
return False
|
||||
|
||||
# Extract with the safest filter
|
||||
print(f"Extracting {len(contents)} items to {output_dir}")
|
||||
archive.extract(output_dir, filter="data")
|
||||
print("Extraction completed safely!")
|
||||
return True
|
||||
|
||||
except TzstArchiveError as e:
|
||||
print(f"Archive error: {e}")
|
||||
return False
|
||||
except Exception as e:
|
||||
print(f"Unexpected error: {e}")
|
||||
return False
|
||||
|
||||
# Usage
|
||||
safe_extract("untrusted_archive.tzst", "safe_output/", max_size_mb=50)
|
||||
```
|
||||
|
||||
### Archive Validation Pipeline
|
||||
|
||||
```python
|
||||
from tzst import TzstArchive, test_archive
|
||||
import hashlib
|
||||
import json
|
||||
from pathlib import Path
|
||||
|
||||
def create_archive_manifest(archive_path):
|
||||
"""Create a manifest of archive contents for validation."""
|
||||
|
||||
manifest = {
|
||||
'archive_path': str(archive_path),
|
||||
'files': [],
|
||||
'created_at': str(Path(archive_path).stat().st_mtime)
|
||||
}
|
||||
|
||||
with TzstArchive(archive_path, "r") as archive:
|
||||
contents = archive.list(verbose=True)
|
||||
|
||||
for item in contents:
|
||||
if item['is_file']:
|
||||
manifest['files'].append({
|
||||
'name': item['name'],
|
||||
'size': item['size'],
|
||||
'mtime': item['mtime']
|
||||
})
|
||||
|
||||
# Save manifest
|
||||
manifest_path = Path(archive_path).with_suffix('.manifest.json')
|
||||
with open(manifest_path, 'w') as f:
|
||||
json.dump(manifest, f, indent=2)
|
||||
|
||||
print(f"Manifest created: {manifest_path}")
|
||||
return manifest
|
||||
|
||||
def validate_archive_with_manifest(archive_path):
|
||||
"""Validate an archive against its manifest."""
|
||||
|
||||
manifest_path = Path(archive_path).with_suffix('.manifest.json')
|
||||
|
||||
if not manifest_path.exists():
|
||||
print("No manifest found, creating new one...")
|
||||
create_archive_manifest(archive_path)
|
||||
return True
|
||||
|
||||
# Load manifest
|
||||
with open(manifest_path, 'r') as f:
|
||||
manifest = json.load(f)
|
||||
|
||||
print("Validating archive integrity...")
|
||||
if not test_archive(archive_path):
|
||||
print("❌ Archive integrity check failed!")
|
||||
return False
|
||||
|
||||
print("Validating against manifest...")
|
||||
with TzstArchive(archive_path, "r") as archive:
|
||||
contents = archive.list(verbose=True)
|
||||
current_files = {item['name']: item for item in contents if item['is_file']}
|
||||
|
||||
manifest_files = {item['name']: item for item in manifest['files']}
|
||||
|
||||
# Check for missing files
|
||||
missing = set(manifest_files.keys()) - set(current_files.keys())
|
||||
if missing:
|
||||
print(f"❌ Missing files: {', '.join(missing)}")
|
||||
return False
|
||||
|
||||
# Check for extra files
|
||||
extra = set(current_files.keys()) - set(manifest_files.keys())
|
||||
if extra:
|
||||
print(f"⚠️ Extra files: {', '.join(extra)}")
|
||||
|
||||
# Check file sizes
|
||||
size_mismatches = []
|
||||
for name, manifest_file in manifest_files.items():
|
||||
if name in current_files:
|
||||
if current_files[name]['size'] != manifest_file['size']:
|
||||
size_mismatches.append(name)
|
||||
|
||||
if size_mismatches:
|
||||
print(f"❌ Size mismatches: {', '.join(size_mismatches)}")
|
||||
return False
|
||||
|
||||
print("✅ Archive validation passed!")
|
||||
return True
|
||||
|
||||
# Usage
|
||||
archive_path = "important_backup.tzst"
|
||||
if validate_archive_with_manifest(archive_path):
|
||||
print("Archive is valid and matches manifest")
|
||||
else:
|
||||
print("Archive validation failed!")
|
||||
```
|
||||
|
||||
## Performance Examples
|
||||
|
||||
### Compression Level Comparison
|
||||
|
||||
```python
|
||||
from tzst import create_archive
|
||||
import time
|
||||
import os
|
||||
from pathlib import Path
|
||||
|
||||
def compression_benchmark(files, output_prefix="test"):
|
||||
"""Compare different compression levels for the same files."""
|
||||
|
||||
results = []
|
||||
levels = [1, 3, 6, 9, 15, 22] # Representative levels
|
||||
|
||||
for level in levels:
|
||||
archive_path = f"{output_prefix}_level_{level}.tzst"
|
||||
|
||||
print(f"Testing compression level {level}...")
|
||||
start_time = time.time()
|
||||
|
||||
create_archive(
|
||||
archive_path=archive_path,
|
||||
files=files,
|
||||
compression_level=level
|
||||
)
|
||||
|
||||
compression_time = time.time() - start_time
|
||||
archive_size = os.path.getsize(archive_path)
|
||||
|
||||
results.append({
|
||||
'level': level,
|
||||
'time': compression_time,
|
||||
'size': archive_size,
|
||||
'path': archive_path
|
||||
})
|
||||
|
||||
print(f" Time: {compression_time:.2f}s, Size: {archive_size:,} bytes")
|
||||
|
||||
# Print comparison table
|
||||
print("\n" + "="*70)
|
||||
print("COMPRESSION LEVEL COMPARISON")
|
||||
print("="*70)
|
||||
print(f"{'Level':<6} {'Time (s)':<10} {'Size (MB)':<12} {'Ratio':<8} {'Speed'}")
|
||||
print("-" * 70)
|
||||
|
||||
baseline_size = results[0]['size'] # Level 1 as baseline
|
||||
baseline_time = results[0]['time']
|
||||
|
||||
for result in results:
|
||||
size_mb = result['size'] / (1024 * 1024)
|
||||
ratio = result['size'] / baseline_size
|
||||
speed_factor = baseline_time / result['time']
|
||||
|
||||
print(f"{result['level']:<6} {result['time']:<10.2f} {size_mb:<12.1f} "
|
||||
f"{ratio:<8.2f} {speed_factor:<.2f}x")
|
||||
|
||||
# Clean up test files
|
||||
for result in results:
|
||||
os.remove(result['path'])
|
||||
|
||||
return results
|
||||
|
||||
# Usage
|
||||
benchmark_files = ["large_directory/", "data_files/"]
|
||||
results = compression_benchmark(benchmark_files, "benchmark")
|
||||
```
|
||||
|
||||
## Error Handling Examples
|
||||
|
||||
### Robust Archive Processing
|
||||
|
||||
```python
|
||||
from tzst import TzstArchive, extract_archive
|
||||
from tzst.exceptions import TzstArchiveError, TzstDecompressionError
|
||||
import logging
|
||||
|
||||
# Set up logging
|
||||
logging.basicConfig(level=logging.INFO)
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
def robust_archive_processor(archive_paths, output_base_dir):
|
||||
"""Process multiple archives with comprehensive error handling."""
|
||||
|
||||
results = {
|
||||
'success': [],
|
||||
'failed': [],
|
||||
'skipped': []
|
||||
}
|
||||
|
||||
for archive_path in archive_paths:
|
||||
try:
|
||||
logger.info(f"Processing {archive_path}")
|
||||
|
||||
# Create output directory for this archive
|
||||
archive_name = Path(archive_path).stem
|
||||
output_dir = Path(output_base_dir) / archive_name
|
||||
output_dir.mkdir(parents=True, exist_ok=True)
|
||||
|
||||
# First, test the archive
|
||||
logger.info(f"Testing integrity of {archive_path}")
|
||||
with TzstArchive(archive_path, "r") as archive:
|
||||
if not archive.test():
|
||||
raise TzstArchiveError(f"Archive {archive_path} failed integrity test")
|
||||
|
||||
# Get archive info
|
||||
contents = archive.list(verbose=True)
|
||||
file_count = sum(1 for item in contents if item['is_file'])
|
||||
total_size = sum(item['size'] for item in contents if item['is_file'])
|
||||
|
||||
logger.info(f"Archive contains {file_count} files ({total_size:,} bytes)")
|
||||
|
||||
# Extract with error handling
|
||||
logger.info(f"Extracting to {output_dir}")
|
||||
archive.extract(str(output_dir), filter="data")
|
||||
|
||||
results['success'].append({
|
||||
'path': archive_path,
|
||||
'file_count': file_count,
|
||||
'total_size': total_size,
|
||||
'output_dir': str(output_dir)
|
||||
})
|
||||
|
||||
logger.info(f"Successfully processed {archive_path}")
|
||||
|
||||
except TzstArchiveError as e:
|
||||
logger.error(f"Archive error processing {archive_path}: {e}")
|
||||
results['failed'].append({
|
||||
'path': archive_path,
|
||||
'error': str(e),
|
||||
'error_type': 'TzstArchiveError'
|
||||
})
|
||||
|
||||
except TzstDecompressionError as e:
|
||||
logger.error(f"Decompression error processing {archive_path}: {e}")
|
||||
results['failed'].append({
|
||||
'path': archive_path,
|
||||
'error': str(e),
|
||||
'error_type': 'TzstDecompressionError'
|
||||
})
|
||||
|
||||
except FileNotFoundError:
|
||||
logger.warning(f"Archive not found: {archive_path}")
|
||||
results['skipped'].append({
|
||||
'path': archive_path,
|
||||
'reason': 'File not found'
|
||||
})
|
||||
|
||||
except PermissionError as e:
|
||||
logger.error(f"Permission error processing {archive_path}: {e}")
|
||||
results['failed'].append({
|
||||
'path': archive_path,
|
||||
'error': str(e),
|
||||
'error_type': 'PermissionError'
|
||||
})
|
||||
|
||||
except Exception as e:
|
||||
logger.error(f"Unexpected error processing {archive_path}: {e}")
|
||||
results['failed'].append({
|
||||
'path': archive_path,
|
||||
'error': str(e),
|
||||
'error_type': 'UnexpectedError'
|
||||
})
|
||||
|
||||
# Print summary
|
||||
print("\n" + "="*60)
|
||||
print("PROCESSING SUMMARY")
|
||||
print("="*60)
|
||||
print(f"✅ Successfully processed: {len(results['success'])}")
|
||||
print(f"❌ Failed: {len(results['failed'])}")
|
||||
print(f"⏭️ Skipped: {len(results['skipped'])}")
|
||||
|
||||
if results['failed']:
|
||||
print("\nFailures:")
|
||||
for failure in results['failed']:
|
||||
print(f" - {failure['path']}: {failure['error_type']}")
|
||||
|
||||
return results
|
||||
|
||||
# Usage
|
||||
archive_list = [
|
||||
"backup1.tzst",
|
||||
"backup2.tzst",
|
||||
"backup3.tzst",
|
||||
"missing_file.tzst" # This will be skipped
|
||||
]
|
||||
|
||||
results = robust_archive_processor(archive_list, "extracted_archives/")
|
||||
```
|
||||
@@ -0,0 +1,61 @@
|
||||
# tzst Documentation
|
||||
|
||||
Welcome to **tzst**, the next-generation Python library engineered for modern archive management, leveraging cutting-edge Zstandard compression to deliver superior performance, security, and reliability.
|
||||
|
||||
```{toctree}
|
||||
:maxdepth: 2
|
||||
:caption: Contents:
|
||||
|
||||
quickstart
|
||||
api/index
|
||||
examples
|
||||
changelog
|
||||
```
|
||||
|
||||
## What is tzst?
|
||||
|
||||
**tzst** is a Python library built exclusively for Python 3.12+ that provides enterprise-grade solutions for handling `.tzst`/`.tar.zst` archives. It combines atomic operations, streaming efficiency, and a meticulously crafted API to redefine how developers handle compressed archives in production environments.
|
||||
|
||||
## Key Features
|
||||
|
||||
- **🚀 High Performance**: Leverages Zstandard compression for superior speed and compression ratios
|
||||
- **🔒 Security First**: Built-in extraction filters protect against malicious archives
|
||||
- **⚡ Streaming Support**: Memory-efficient handling of large archives
|
||||
- **🛡️ Atomic Operations**: Ensures data integrity with fail-safe file operations
|
||||
- **🎯 Modern API**: Clean, intuitive interface designed for Python 3.12+
|
||||
- **📦 CLI Tools**: Comprehensive command-line interface for everyday tasks
|
||||
|
||||
## Quick Example
|
||||
|
||||
```python
|
||||
from tzst import TzstArchive
|
||||
|
||||
# Create a new archive
|
||||
with TzstArchive("backup.tzst", "w", compression_level=5) as archive:
|
||||
archive.add("documents/")
|
||||
archive.add("photos/", recursive=True)
|
||||
|
||||
# Extract with security
|
||||
with TzstArchive("backup.tzst", "r") as archive:
|
||||
archive.extract("documents/", filter="data")
|
||||
```
|
||||
|
||||
## Installation
|
||||
|
||||
Install tzst from PyPI:
|
||||
|
||||
```bash
|
||||
pip install tzst
|
||||
```
|
||||
|
||||
## Getting Started
|
||||
|
||||
For a quick introduction to using tzst, see the {doc}`quickstart` guide.
|
||||
|
||||
For detailed API documentation, browse the {doc}`api/index` section.
|
||||
|
||||
## Indices and tables
|
||||
|
||||
- {ref}`genindex`
|
||||
- {ref}`modindex`
|
||||
- {ref}`search`
|
||||
@@ -0,0 +1,35 @@
|
||||
@ECHO OFF
|
||||
|
||||
pushd %~dp0
|
||||
|
||||
REM Command file for Sphinx documentation
|
||||
|
||||
if "%SPHINXBUILD%" == "" (
|
||||
set SPHINXBUILD=sphinx-build
|
||||
)
|
||||
set SOURCEDIR=.
|
||||
set BUILDDIR=_build
|
||||
|
||||
%SPHINXBUILD% >NUL 2>NUL
|
||||
if errorlevel 9009 (
|
||||
echo.
|
||||
echo.The 'sphinx-build' command was not found. Make sure you have Sphinx
|
||||
echo.installed, then set the SPHINXBUILD environment variable to point
|
||||
echo.to the full path of the 'sphinx-build' executable. Alternatively you
|
||||
echo.may add the Sphinx directory to PATH.
|
||||
echo.
|
||||
echo.If you don't have Sphinx installed, grab it from
|
||||
echo.https://sphinx-doc.org/
|
||||
exit /b 1
|
||||
)
|
||||
|
||||
if "%1" == "" goto help
|
||||
|
||||
%SPHINXBUILD% -M %1 %SOURCEDIR% %BUILDDIR% %SPHINXOPTS% %O%
|
||||
goto end
|
||||
|
||||
:help
|
||||
%SPHINXBUILD% -M help %SOURCEDIR% %BUILDDIR% %SPHINXOPTS% %O%
|
||||
|
||||
:end
|
||||
popd
|
||||
@@ -0,0 +1,222 @@
|
||||
# Quick Start Guide
|
||||
|
||||
This guide will help you get started with tzst quickly and efficiently.
|
||||
|
||||
## Installation
|
||||
|
||||
Install tzst using pip:
|
||||
|
||||
```bash
|
||||
pip install tzst
|
||||
```
|
||||
|
||||
## Basic Usage
|
||||
|
||||
### Creating Archives
|
||||
|
||||
Use the `TzstArchive` class or convenience functions to create archives:
|
||||
|
||||
```python
|
||||
from tzst import TzstArchive, create_archive
|
||||
|
||||
# Using TzstArchive class
|
||||
with TzstArchive("my_archive.tzst", "w", compression_level=5) as archive:
|
||||
archive.add("file.txt")
|
||||
archive.add("directory/", recursive=True)
|
||||
|
||||
# Using convenience function
|
||||
create_archive(
|
||||
archive_path="backup.tzst",
|
||||
files=["documents/", "photos/", "config.txt"],
|
||||
compression_level=10
|
||||
)
|
||||
```
|
||||
|
||||
### Extracting Archives
|
||||
|
||||
Extract archives safely with built-in security filters:
|
||||
|
||||
```python
|
||||
from tzst import TzstArchive, extract_archive
|
||||
|
||||
# Using TzstArchive class
|
||||
with TzstArchive("my_archive.tzst", "r") as archive:
|
||||
# Extract all files with security filter
|
||||
archive.extract("output/", filter="data")
|
||||
|
||||
# Extract specific files
|
||||
archive.extract("output/", members=["file.txt"], filter="data")
|
||||
|
||||
# Using convenience function
|
||||
extract_archive("backup.tzst", "restore/")
|
||||
```
|
||||
|
||||
### Listing Archive Contents
|
||||
|
||||
View what's inside an archive:
|
||||
|
||||
```python
|
||||
from tzst import TzstArchive, list_archive
|
||||
|
||||
# Using TzstArchive class
|
||||
with TzstArchive("my_archive.tzst", "r") as archive:
|
||||
contents = archive.list(verbose=True)
|
||||
for item in contents:
|
||||
print(f"{item['name']} - {item['size']} bytes")
|
||||
|
||||
# Using convenience function
|
||||
files = list_archive("backup.tzst", verbose=True)
|
||||
```
|
||||
|
||||
### Testing Archive Integrity
|
||||
|
||||
Verify that an archive is valid:
|
||||
|
||||
```python
|
||||
from tzst import TzstArchive, test_archive
|
||||
|
||||
# Using TzstArchive class
|
||||
with TzstArchive("my_archive.tzst", "r") as archive:
|
||||
is_valid = archive.test()
|
||||
print(f"Archive is {'valid' if is_valid else 'corrupted'}")
|
||||
|
||||
# Using convenience function
|
||||
if test_archive("backup.tzst"):
|
||||
print("Archive is valid")
|
||||
```
|
||||
|
||||
## Command Line Interface
|
||||
|
||||
tzst provides a comprehensive CLI for archive operations:
|
||||
|
||||
### Creating Archives
|
||||
|
||||
```bash
|
||||
# Create an archive with multiple files
|
||||
tzst a backup.tzst documents/ photos/ config.txt
|
||||
|
||||
# Create with high compression
|
||||
tzst a -l 15 backup.tzst large_files/
|
||||
|
||||
# Create without atomic operations (faster, less safe)
|
||||
tzst a --no-atomic backup.tzst files/
|
||||
```
|
||||
|
||||
### Extracting Archives
|
||||
|
||||
```bash
|
||||
# Extract all files (default: safe extraction)
|
||||
tzst x backup.tzst
|
||||
|
||||
# Extract to specific directory
|
||||
tzst x backup.tzst -o restore/
|
||||
|
||||
# Extract specific files only
|
||||
tzst x backup.tzst config.txt documents/
|
||||
|
||||
# Extract with streaming (memory efficient)
|
||||
tzst x backup.tzst --streaming
|
||||
```
|
||||
|
||||
### Listing Contents
|
||||
|
||||
```bash
|
||||
# Simple listing
|
||||
tzst l backup.tzst
|
||||
|
||||
# Detailed listing with file info
|
||||
tzst l backup.tzst -v
|
||||
|
||||
# Streaming mode for large archives
|
||||
tzst l backup.tzst --streaming
|
||||
```
|
||||
|
||||
### Testing Archives
|
||||
|
||||
```bash
|
||||
# Test archive integrity
|
||||
tzst t backup.tzst
|
||||
|
||||
# Test with streaming
|
||||
tzst t backup.tzst --streaming
|
||||
```
|
||||
|
||||
## Security Considerations
|
||||
|
||||
tzst includes built-in security features to protect against malicious archives:
|
||||
|
||||
### Extraction Filters
|
||||
|
||||
Always use appropriate filters when extracting archives from untrusted sources:
|
||||
|
||||
- **`data`** (default): Safest option, only extracts regular files and directories
|
||||
- **`tar`**: Honors most tar features but still secure
|
||||
- **`fully_trusted`**: No restrictions (only use with completely trusted archives)
|
||||
|
||||
```python
|
||||
# Safe extraction (recommended)
|
||||
archive.extract("output/", filter="data")
|
||||
|
||||
# Command line
|
||||
tzst x archive.tzst --filter=data
|
||||
```
|
||||
|
||||
### Best Practices
|
||||
|
||||
1. **Always use the default `data` filter** for untrusted archives
|
||||
2. **Enable atomic operations** (default) for data integrity
|
||||
3. **Use streaming mode** for very large archives to save memory
|
||||
4. **Validate archives** with `test()` before processing
|
||||
5. **Specify output directories** explicitly to avoid overwrites
|
||||
|
||||
## Performance Tips
|
||||
|
||||
### Memory Efficiency
|
||||
|
||||
For large archives, use streaming mode:
|
||||
|
||||
```python
|
||||
# Streaming mode uses less memory
|
||||
with TzstArchive("large.tzst", "r", streaming=True) as archive:
|
||||
archive.extract("output/")
|
||||
```
|
||||
|
||||
### Compression Levels
|
||||
|
||||
Choose appropriate compression levels based on your needs:
|
||||
|
||||
- **Level 1-3**: Fast compression, larger files
|
||||
- **Level 3-6**: Balanced (default: 3)
|
||||
- **Level 7-15**: Better compression, slower
|
||||
- **Level 16-22**: Maximum compression, much slower
|
||||
|
||||
```python
|
||||
# Fast compression for temporary files
|
||||
TzstArchive("temp.tzst", "w", compression_level=1)
|
||||
|
||||
# Maximum compression for long-term storage
|
||||
TzstArchive("backup.tzst", "w", compression_level=15)
|
||||
```
|
||||
|
||||
## Error Handling
|
||||
|
||||
tzst provides specific exceptions for different error conditions:
|
||||
|
||||
```python
|
||||
from tzst import TzstArchive
|
||||
from tzst.exceptions import TzstArchiveError, TzstDecompressionError
|
||||
|
||||
try:
|
||||
with TzstArchive("archive.tzst", "r") as archive:
|
||||
archive.extract("output/")
|
||||
except TzstArchiveError as e:
|
||||
print(f"Archive error: {e}")
|
||||
except TzstDecompressionError as e:
|
||||
print(f"Decompression error: {e}")
|
||||
```
|
||||
|
||||
## Next Steps
|
||||
|
||||
- Explore the complete {doc}`api/index` documentation
|
||||
- Check out more {doc}`examples` and use cases
|
||||
- Read about advanced features in the full documentation
|
||||
@@ -0,0 +1,17 @@
|
||||
# Documentation requirements for Sphinx
|
||||
sphinx>=7.1.0
|
||||
sphinx-rtd-theme>=2.0.0
|
||||
myst-parser>=3.0.0
|
||||
sphinxcontrib-napoleon>=0.7
|
||||
|
||||
# Additional Sphinx extensions
|
||||
sphinx-autobuild>=2021.3.14
|
||||
sphinx-copybutton>=0.5.2
|
||||
sphinxext-opengraph>=0.9.0
|
||||
sphinx-autodoc-typehints>=1.25.0
|
||||
|
||||
# Alternative modern theme (optional)
|
||||
furo>=2024.1.29
|
||||
|
||||
# Main package dependencies (needed for autodoc to import modules)
|
||||
zstandard>=0.19.0,<1.0.0
|
||||
+3
-3
@@ -5,7 +5,7 @@ build-backend = "hatchling.build"
|
||||
[project]
|
||||
name = "tzst"
|
||||
dynamic = ["version"]
|
||||
description = "A Python library for creating and manipulating .tzst/.tar.zst archives"
|
||||
description = "The next-generation Python library engineered for modern archive management, leveraging cutting-edge Zstandard compression to deliver superior performance, security, and reliability"
|
||||
readme = "README.md"
|
||||
license = "BSD-3-Clause"
|
||||
requires-python = ">=3.12"
|
||||
@@ -47,7 +47,6 @@ include = [
|
||||
"tests",
|
||||
"README.md",
|
||||
"LICENSE",
|
||||
"CHANGELOG.md",
|
||||
"CONTRIBUTING.md",
|
||||
]
|
||||
|
||||
@@ -56,7 +55,8 @@ testpaths = ["tests"]
|
||||
python_files = ["test_*.py"]
|
||||
python_classes = ["Test*"]
|
||||
python_functions = ["test_*"]
|
||||
addopts = "--cov=tzst --cov-report=term-missing --cov-report=html"
|
||||
addopts = "--cov --cov-report=term-missing --cov-report=html"
|
||||
markers = ["integration: marks tests as integration tests"]
|
||||
|
||||
[tool.ruff]
|
||||
line-length = 88
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
"""tzst - A Python library for creating and manipulating .tzst/.tar.zst archives."""
|
||||
"""tzst - The next-generation Python library engineered for modern archive management, leveraging cutting-edge Zstandard compression to deliver superior performance, security, and reliability."""
|
||||
|
||||
__version__ = "1.0.0"
|
||||
__version__ = "1.1.1"
|
||||
|
||||
from .core import (
|
||||
TzstArchive,
|
||||
|
||||
@@ -1,6 +1,8 @@
|
||||
"""Main entry point for tzst package when run as a module."""
|
||||
|
||||
import sys
|
||||
|
||||
from .cli import main
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
sys.exit(main())
|
||||
+450
-96
@@ -3,6 +3,7 @@
|
||||
import argparse
|
||||
import sys
|
||||
from pathlib import Path
|
||||
from typing import Literal, cast
|
||||
|
||||
from . import __version__
|
||||
from .core import create_archive, extract_archive, list_archive, test_archive
|
||||
@@ -20,6 +21,7 @@ def print_banner() -> None:
|
||||
"""
|
||||
print()
|
||||
print(f"tzst {__version__} : Copyright (c) 2025 Xi Xu")
|
||||
print()
|
||||
|
||||
|
||||
def format_size(size: int) -> str:
|
||||
@@ -50,6 +52,168 @@ def format_size(size: int) -> str:
|
||||
return f"{size_float:6.1f} PB"
|
||||
|
||||
|
||||
def validate_compression_level(value: str) -> int:
|
||||
"""Validate and return compression level.
|
||||
|
||||
Args:
|
||||
value: String value from command line
|
||||
|
||||
Returns:
|
||||
int: Valid compression level (1-22)
|
||||
|
||||
Raises:
|
||||
argparse.ArgumentTypeError: If value is not a valid compression level
|
||||
"""
|
||||
try:
|
||||
level = int(value)
|
||||
if not 1 <= level <= 22:
|
||||
raise argparse.ArgumentTypeError(
|
||||
f"Invalid compression level: {level}. Must be between 1 and 22."
|
||||
)
|
||||
return level
|
||||
except ValueError:
|
||||
raise argparse.ArgumentTypeError(
|
||||
f"Invalid compression level: '{value}'. "
|
||||
f"Must be an integer between 1 and 22."
|
||||
) from None
|
||||
|
||||
|
||||
def _process_file_paths(file_args: list[str]) -> list[Path]:
|
||||
"""Process file arguments into resolved Path objects.
|
||||
|
||||
Args:
|
||||
file_args: List of file path strings from command line
|
||||
|
||||
Returns:
|
||||
list[Path]: List of resolved Path objects
|
||||
"""
|
||||
files: list[Path] = []
|
||||
for file_arg in file_args:
|
||||
file_path = Path(file_arg).resolve()
|
||||
files.append(file_path)
|
||||
return files
|
||||
|
||||
|
||||
def _validate_files(files: list[Path]) -> list[Path]:
|
||||
"""Validate that files exist and are accessible.
|
||||
|
||||
Args:
|
||||
files: List of Path objects to validate
|
||||
|
||||
Returns:
|
||||
list[Path]: List of missing files (empty if all files exist)
|
||||
|
||||
Raises:
|
||||
OSError: If a file cannot be accessed due to permissions or invalid characters
|
||||
"""
|
||||
missing_files = []
|
||||
for f in files:
|
||||
try:
|
||||
if not f.exists():
|
||||
missing_files.append(f)
|
||||
except OSError as e:
|
||||
# Handle path issues (invalid characters, permissions, etc.)
|
||||
print(f"Error: Cannot access file '{f}' - {e}", file=sys.stderr)
|
||||
raise
|
||||
return missing_files
|
||||
|
||||
|
||||
def _extract_add_params(args) -> tuple[int, bool]:
|
||||
"""Extract compression parameters from arguments.
|
||||
|
||||
Args:
|
||||
args: Parsed command line arguments
|
||||
|
||||
Returns:
|
||||
tuple[int, bool]: (compression_level, use_temp_file)
|
||||
"""
|
||||
compression_level = getattr(args, "compression_level", 3)
|
||||
use_temp_file = not getattr(args, "no_atomic", False)
|
||||
return compression_level, use_temp_file
|
||||
|
||||
|
||||
def _prepare_archive_creation(args) -> tuple[Path, list[Path], int, bool] | int:
|
||||
"""Prepare and validate inputs for archive creation.
|
||||
|
||||
Args:
|
||||
args: Parsed command line arguments
|
||||
|
||||
Returns:
|
||||
tuple or int: Either (archive_path, files, compression_level, use_temp_file)
|
||||
or error code if validation fails
|
||||
"""
|
||||
archive_path = Path(args.archive)
|
||||
files = _process_file_paths(args.files)
|
||||
|
||||
# Validate files
|
||||
missing_files = _validate_files(files)
|
||||
if missing_files:
|
||||
print(
|
||||
f"Error: Files not found - {', '.join(map(str, missing_files))}",
|
||||
file=sys.stderr,
|
||||
)
|
||||
return 1
|
||||
|
||||
compression_level, use_temp_file = _extract_add_params(args)
|
||||
return archive_path, files, compression_level, use_temp_file
|
||||
|
||||
|
||||
def _execute_archive_creation(
|
||||
archive_path: Path, files: list[Path], compression_level: int, use_temp_file: bool
|
||||
) -> int:
|
||||
"""Execute the archive creation process.
|
||||
|
||||
Args:
|
||||
archive_path: Path where archive will be created
|
||||
files: List of files to add to archive
|
||||
compression_level: Compression level to use
|
||||
use_temp_file: Whether to use atomic file operations
|
||||
|
||||
Returns:
|
||||
int: Exit code (0 for success, non-zero for failure)
|
||||
"""
|
||||
print(f"Creating archive: {archive_path}")
|
||||
for file_path in files:
|
||||
print(f" Adding: {file_path}")
|
||||
|
||||
# Use atomic file operations by default for better reliability
|
||||
# This creates the archive in a temporary file first, then moves it
|
||||
create_archive(archive_path, files, compression_level, use_temp_file=use_temp_file)
|
||||
print(f"Archive created successfully - {archive_path}")
|
||||
return 0
|
||||
|
||||
|
||||
def _handle_archive_creation_exceptions(func, *args, **kwargs) -> int:
|
||||
"""Handle exceptions during archive creation.
|
||||
|
||||
Args:
|
||||
func: Function to execute
|
||||
*args: Positional arguments for func
|
||||
**kwargs: Keyword arguments for func
|
||||
|
||||
Returns:
|
||||
int: Exit code based on exception type
|
||||
"""
|
||||
try:
|
||||
return func(*args, **kwargs)
|
||||
except OSError:
|
||||
return 1 # Error already printed in _validate_files
|
||||
except ValueError as e:
|
||||
print(f"Error: Invalid parameter - {e}", file=sys.stderr)
|
||||
return 1
|
||||
except TzstArchiveError as e:
|
||||
print(f"Error: Archive operation failed - {e}", file=sys.stderr)
|
||||
return 1
|
||||
except KeyboardInterrupt:
|
||||
print("\nOperation interrupted by user", file=sys.stderr)
|
||||
# Clean up any partial files - the atomic operations in create_archive
|
||||
# handle this
|
||||
return 130 # Standard exit code for SIGINT
|
||||
except Exception as e:
|
||||
print(f"Error: Failed to create archive - {e}", file=sys.stderr)
|
||||
return 1
|
||||
|
||||
|
||||
def cmd_add(args) -> int:
|
||||
"""Command handler for creating/adding to archives.
|
||||
|
||||
@@ -79,51 +243,18 @@ def cmd_add(args) -> int:
|
||||
:func:`tzst.create_archive`: The underlying function for archive creation
|
||||
:meth:`TzstArchive.add`: The core method for adding files to archives
|
||||
"""
|
||||
try:
|
||||
archive_path = Path(args.archive)
|
||||
files: list[Path] = [Path(f) for f in args.files]
|
||||
|
||||
# Check if files exist
|
||||
missing_files = [f for f in files if not f.exists()]
|
||||
if missing_files:
|
||||
print(
|
||||
f"Error: Files not found - {', '.join(map(str, missing_files))}",
|
||||
file=sys.stderr,
|
||||
)
|
||||
return 1
|
||||
def _create_archive_workflow():
|
||||
preparation_result = _prepare_archive_creation(args)
|
||||
if isinstance(preparation_result, int):
|
||||
return preparation_result
|
||||
|
||||
compression_level = getattr(args, "compression_level", 3)
|
||||
use_temp_file = not getattr(args, "no_atomic", False)
|
||||
|
||||
print(f"Creating archive: {archive_path}")
|
||||
for file_path in files:
|
||||
print(f" Adding: {file_path}")
|
||||
|
||||
# Use atomic file operations by default for better reliability
|
||||
# This creates the archive in a temporary file first, then moves it
|
||||
create_archive(
|
||||
archive_path, files, compression_level, use_temp_file=use_temp_file
|
||||
archive_path, files, compression_level, use_temp_file = preparation_result
|
||||
return _execute_archive_creation(
|
||||
archive_path, files, compression_level, use_temp_file
|
||||
)
|
||||
print(f"Archive created successfully - {archive_path}")
|
||||
return 0
|
||||
|
||||
except FileNotFoundError as e:
|
||||
print(f"Error: File not found - {e}", file=sys.stderr)
|
||||
return 1
|
||||
except ValueError as e:
|
||||
print(f"Error: Invalid parameter - {e}", file=sys.stderr)
|
||||
return 1
|
||||
except TzstArchiveError as e:
|
||||
print(f"Error: Archive operation failed - {e}", file=sys.stderr)
|
||||
return 1
|
||||
except KeyboardInterrupt:
|
||||
print("\nOperation interrupted by user", file=sys.stderr)
|
||||
# Clean up any partial files - the atomic operations in create_archive
|
||||
# handle this
|
||||
return 130 # Standard exit code for SIGINT
|
||||
except Exception as e:
|
||||
print(f"Error: Failed to create archive - {e}", file=sys.stderr)
|
||||
return 1
|
||||
return _handle_archive_creation_exceptions(_create_archive_workflow)
|
||||
|
||||
|
||||
def cmd_extract_full(args) -> int:
|
||||
@@ -164,7 +295,9 @@ def cmd_extract_full(args) -> int:
|
||||
output_dir = Path(args.output) if args.output else Path.cwd()
|
||||
members = args.files if hasattr(args, "files") and args.files else None
|
||||
streaming = getattr(args, "streaming", False)
|
||||
filter_type = getattr(args, "filter", "data")
|
||||
filter_type = cast(
|
||||
Literal["data", "tar", "fully_trusted"], getattr(args, "filter", "data")
|
||||
)
|
||||
|
||||
print(f"Extracting from: {archive_path}")
|
||||
print(f"Output directory: {output_dir}")
|
||||
@@ -240,7 +373,9 @@ def cmd_extract_flat(args) -> int:
|
||||
output_dir = Path(args.output) if args.output else Path.cwd()
|
||||
members = args.files if hasattr(args, "files") and args.files else None
|
||||
streaming = getattr(args, "streaming", False)
|
||||
filter_type = getattr(args, "filter", "data")
|
||||
filter_type = cast(
|
||||
Literal["data", "tar", "fully_trusted"], getattr(args, "filter", "data")
|
||||
)
|
||||
|
||||
print(f"Extracting from: {archive_path}")
|
||||
print(f"Output directory: {output_dir}")
|
||||
@@ -371,6 +506,9 @@ def cmd_list(args) -> int:
|
||||
except TzstArchiveError as e:
|
||||
print(f"Error: Archive operation failed - {e}", file=sys.stderr)
|
||||
return 1
|
||||
except KeyboardInterrupt:
|
||||
print("\nOperation interrupted by user", file=sys.stderr)
|
||||
return 130
|
||||
except Exception as e:
|
||||
print(f"Error: Failed to list archive - {e}", file=sys.stderr)
|
||||
return 1
|
||||
@@ -430,11 +568,24 @@ def cmd_test(args) -> int:
|
||||
except TzstArchiveError as e:
|
||||
print(f"Error: Archive operation failed - {e}", file=sys.stderr)
|
||||
return 1
|
||||
except KeyboardInterrupt:
|
||||
print("\nOperation interrupted by user", file=sys.stderr)
|
||||
return 130
|
||||
except Exception as e:
|
||||
print(f"Error: Failed to test archive - {e}", file=sys.stderr)
|
||||
return 1
|
||||
|
||||
|
||||
def cmd_version(args) -> int:
|
||||
"""Command handler for version display.
|
||||
|
||||
Returns:
|
||||
int: Exit code (always 0)
|
||||
"""
|
||||
# Version is already printed in print_banner(), so just exit
|
||||
return 0
|
||||
|
||||
|
||||
def create_parser() -> argparse.ArgumentParser:
|
||||
"""Create and configure the command-line argument parser.
|
||||
|
||||
@@ -461,65 +612,63 @@ def create_parser() -> argparse.ArgumentParser:
|
||||
:func:`main`: The main entry point that uses this parser
|
||||
"""
|
||||
epilog = """
|
||||
Command Reference:
|
||||
Archive:
|
||||
command reference:
|
||||
archive:
|
||||
a, add, create tzst a archive.tzst files... [-l LEVEL] [--no-atomic]
|
||||
|
||||
Extract:
|
||||
x, extract tzst x archive.tzst [files...] [-o DIR] [--streaming] \\
|
||||
[--filter FILTER]
|
||||
e, extract-flat tzst e archive.tzst [files...] [-o DIR] [--streaming] \\
|
||||
[--filter FILTER]
|
||||
extract:
|
||||
x, extract tzst x archive.tzst [files...] [-o DIR] [--streaming] [--filter FILTER]
|
||||
e, extract-flat tzst e archive.tzst [files...] [-o DIR] [--streaming] [--filter FILTER]
|
||||
|
||||
Manage:
|
||||
manage:
|
||||
l, list tzst l archive.tzst [-v] [--streaming]
|
||||
t, test tzst t archive.tzst [--streaming]
|
||||
|
||||
Arguments:
|
||||
-l, --level LEVEL Compression level (1-22, default: 3)
|
||||
-o, --output DIR Output directory (default: current directory)
|
||||
-v, --verbose Show detailed information
|
||||
--streaming Use streaming mode for memory efficiency with large \\
|
||||
archives
|
||||
--filter FILTER Security filter for extraction: data (safest, default), \\
|
||||
tar, fully_trusted
|
||||
--no-atomic Disable atomic file operations (not recommended)
|
||||
arguments:
|
||||
-l, --level LEVEL compression level (1-22, default: 3)
|
||||
-o, --output DIR output directory (default: current directory)
|
||||
-v, --verbose show detailed information
|
||||
--streaming use streaming mode for memory efficiency with large archives
|
||||
--filter FILTER security filter for extraction: data (safest, default), tar, fully_trusted
|
||||
--no-atomic disable atomic file operations (not recommended)
|
||||
|
||||
Security Note:
|
||||
Always use --filter=data (default) when extracting archives from untrusted sources.
|
||||
Never use --filter=fully_trusted unless you completely trust the archive source.
|
||||
security note:
|
||||
always use --filter=data (default) when extracting archives from untrusted sources
|
||||
never use --filter=fully_trusted unless you completely trust the archive source
|
||||
|
||||
Documentation:
|
||||
documentation:
|
||||
https://github.com/xixu-me/tzst#readme
|
||||
"""
|
||||
|
||||
parser = argparse.ArgumentParser(
|
||||
prog="tzst",
|
||||
epilog=epilog,
|
||||
formatter_class=argparse.RawDescriptionHelpFormatter,
|
||||
add_help=False,
|
||||
add_help=True,
|
||||
)
|
||||
parser.add_argument(
|
||||
"--version", action="store_true", help="show version information and exit"
|
||||
)
|
||||
|
||||
# Add global arguments
|
||||
subparsers = parser.add_subparsers(
|
||||
dest="command", title="Commands", help="Available commands", metavar="COMMAND"
|
||||
dest="command", title="commands", metavar="COMMAND"
|
||||
)
|
||||
|
||||
# Add/Create command
|
||||
parser_add = subparsers.add_parser(
|
||||
"a", aliases=["add", "create"], help="Add files to archive"
|
||||
"a", aliases=["add", "create"], help="add files to archive"
|
||||
)
|
||||
parser_add.add_argument("archive", help="Archive file path")
|
||||
parser_add.add_argument("files", nargs="+", help="Files/directories to add")
|
||||
parser_add.add_argument("archive", help="archive file path")
|
||||
parser_add.add_argument("files", nargs="+", help="files/directories to add")
|
||||
parser_add.add_argument(
|
||||
"-c",
|
||||
"-l",
|
||||
"--level",
|
||||
dest="compression_level",
|
||||
type=int,
|
||||
type=validate_compression_level,
|
||||
default=3,
|
||||
choices=range(1, 23),
|
||||
metavar="LEVEL",
|
||||
help="Compression level (1-22, default: 3)",
|
||||
help="compression level (1-22, default: 3)",
|
||||
)
|
||||
parser_add.add_argument(
|
||||
"--no-atomic",
|
||||
@@ -535,15 +684,15 @@ Documentation:
|
||||
parser_extract = subparsers.add_parser(
|
||||
"x", aliases=["extract"], help="eXtract files with full paths"
|
||||
)
|
||||
parser_extract.add_argument("archive", help="Archive file path")
|
||||
parser_extract.add_argument("files", nargs="*", help="Specific files to extract")
|
||||
parser_extract.add_argument("archive", help="archive file path")
|
||||
parser_extract.add_argument("files", nargs="*", help="specific files to extract")
|
||||
parser_extract.add_argument(
|
||||
"-o", "--output", help="Output directory (default: current directory)"
|
||||
"-o", "--output", help="output directory (default: current directory)"
|
||||
)
|
||||
parser_extract.add_argument(
|
||||
"--streaming",
|
||||
action="store_true",
|
||||
help="Use streaming mode for memory efficiency with large archives",
|
||||
help="use streaming mode for memory efficiency with large archives",
|
||||
)
|
||||
parser_extract.add_argument(
|
||||
"--filter",
|
||||
@@ -561,19 +710,19 @@ Documentation:
|
||||
parser_extract_flat = subparsers.add_parser(
|
||||
"e",
|
||||
aliases=["extract-flat"],
|
||||
help="Extract files from archive (without using directory names)",
|
||||
help="extract files from archive (without using directory names)",
|
||||
)
|
||||
parser_extract_flat.add_argument("archive", help="Archive file path")
|
||||
parser_extract_flat.add_argument("archive", help="archive file path")
|
||||
parser_extract_flat.add_argument(
|
||||
"files", nargs="*", help="Specific files to extract"
|
||||
"files", nargs="*", help="specific files to extract"
|
||||
)
|
||||
parser_extract_flat.add_argument(
|
||||
"-o", "--output", help="Output directory (default: current directory)"
|
||||
"-o", "--output", help="output directory (default: current directory)"
|
||||
)
|
||||
parser_extract_flat.add_argument(
|
||||
"--streaming",
|
||||
action="store_true",
|
||||
help="Use streaming mode for memory efficiency with large archives",
|
||||
help="use streaming mode for memory efficiency with large archives",
|
||||
)
|
||||
parser_extract_flat.add_argument(
|
||||
"--filter",
|
||||
@@ -589,34 +738,240 @@ Documentation:
|
||||
|
||||
# List command
|
||||
parser_list = subparsers.add_parser(
|
||||
"l", aliases=["list"], help="List contents of archive"
|
||||
"l", aliases=["list"], help="list contents of archive"
|
||||
)
|
||||
parser_list.add_argument("archive", help="Archive file path")
|
||||
parser_list.add_argument("archive", help="archive file path")
|
||||
parser_list.add_argument(
|
||||
"-v", "--verbose", action="store_true", help="Show detailed information"
|
||||
"-v", "--verbose", action="store_true", help="show detailed information"
|
||||
)
|
||||
parser_list.add_argument(
|
||||
"--streaming",
|
||||
action="store_true",
|
||||
help="Use streaming mode for memory efficiency with large archives",
|
||||
help="use streaming mode for memory efficiency with large archives",
|
||||
)
|
||||
parser_list.set_defaults(func=cmd_list)
|
||||
|
||||
# Test command
|
||||
parser_test = subparsers.add_parser(
|
||||
"t", aliases=["test"], help="Test integrity of archive"
|
||||
"t", aliases=["test"], help="test integrity of archive"
|
||||
)
|
||||
parser_test.add_argument("archive", help="Archive file path")
|
||||
parser_test.add_argument("archive", help="archive file path")
|
||||
parser_test.add_argument(
|
||||
"--streaming",
|
||||
action="store_true",
|
||||
help="Use streaming mode for memory efficiency with large archives",
|
||||
help="use streaming mode for memory efficiency with large archives",
|
||||
)
|
||||
parser_test.set_defaults(func=cmd_test)
|
||||
|
||||
return parser
|
||||
|
||||
|
||||
def _validate_compression_level_in_argv(argv: list[str]) -> bool:
|
||||
"""Check for compression level validation errors in argv.
|
||||
|
||||
Args:
|
||||
argv: Command line arguments
|
||||
|
||||
Returns:
|
||||
bool: True if an error was found and handled, False otherwise
|
||||
"""
|
||||
if "-c" not in argv and "-l" not in argv and "--level" not in argv:
|
||||
return False
|
||||
|
||||
try:
|
||||
level_index = -1
|
||||
if "-c" in argv:
|
||||
level_index = argv.index("-c")
|
||||
elif "-l" in argv:
|
||||
level_index = argv.index("-l")
|
||||
else:
|
||||
level_index = argv.index("--level")
|
||||
if level_index + 1 < len(argv):
|
||||
level_value = argv[level_index + 1]
|
||||
try:
|
||||
level = int(level_value)
|
||||
if not 1 <= level <= 22:
|
||||
print(
|
||||
f"Invalid compression level: {level}. "
|
||||
f"Must be between 1 and 22.",
|
||||
file=sys.stderr,
|
||||
)
|
||||
return True
|
||||
except ValueError:
|
||||
print(
|
||||
f"Invalid compression level: '{level_value}'. "
|
||||
f"Must be an integer between 1 and 22.",
|
||||
file=sys.stderr,
|
||||
)
|
||||
return True
|
||||
except (ValueError, IndexError):
|
||||
pass
|
||||
return False
|
||||
|
||||
|
||||
def _validate_filter_in_argv(argv: list[str]) -> bool:
|
||||
"""Check for filter validation errors in argv.
|
||||
|
||||
Args:
|
||||
argv: Command line arguments
|
||||
|
||||
Returns:
|
||||
bool: True if an error was found and handled, False otherwise
|
||||
"""
|
||||
if "--filter" not in argv:
|
||||
return False
|
||||
|
||||
try:
|
||||
filter_index = argv.index("--filter")
|
||||
if filter_index + 1 < len(argv):
|
||||
filter_value = argv[filter_index + 1]
|
||||
valid_filters = ["data", "tar", "fully_trusted"]
|
||||
if filter_value not in valid_filters:
|
||||
print(
|
||||
f"Invalid filter specified: {filter_value}. "
|
||||
f"Must be one of: {', '.join(valid_filters)}",
|
||||
file=sys.stderr,
|
||||
)
|
||||
return True
|
||||
except (ValueError, IndexError):
|
||||
pass
|
||||
return False
|
||||
|
||||
|
||||
def _validate_command_in_argv(argv: list[str]) -> bool:
|
||||
"""Check for invalid command errors in argv.
|
||||
|
||||
Args:
|
||||
argv: Command line arguments
|
||||
|
||||
Returns:
|
||||
bool: True if an invalid command was found and handled, False otherwise
|
||||
"""
|
||||
if not argv:
|
||||
return False
|
||||
|
||||
# Valid commands and their aliases
|
||||
valid_commands = {
|
||||
"a",
|
||||
"add",
|
||||
"create",
|
||||
"x",
|
||||
"extract",
|
||||
"e",
|
||||
"extract-flat",
|
||||
"l",
|
||||
"list",
|
||||
"t",
|
||||
"test",
|
||||
}
|
||||
|
||||
# Find the first argument that's not a flag (doesn't start with -)
|
||||
for arg in argv:
|
||||
if not arg.startswith("-"):
|
||||
if arg not in valid_commands:
|
||||
print(
|
||||
f"Invalid command: '{arg}'. "
|
||||
f"Valid commands are: {', '.join(sorted(valid_commands))}",
|
||||
file=sys.stderr,
|
||||
)
|
||||
return True
|
||||
break
|
||||
|
||||
return False
|
||||
|
||||
|
||||
def _is_extreme_compression_level_in_argv(argv: list[str]) -> bool:
|
||||
"""Check for extreme compression level values that warrant special handling.
|
||||
|
||||
Args:
|
||||
argv: Command line arguments
|
||||
|
||||
Returns:
|
||||
bool: True if an extreme compression level value is found, False otherwise
|
||||
"""
|
||||
if "-c" not in argv and "-l" not in argv and "--level" not in argv:
|
||||
return False
|
||||
|
||||
try:
|
||||
level_index = -1
|
||||
if "-c" in argv:
|
||||
level_index = argv.index("-c")
|
||||
elif "-l" in argv:
|
||||
level_index = argv.index("-l")
|
||||
else:
|
||||
level_index = argv.index("--level")
|
||||
if level_index + 1 < len(argv):
|
||||
level_value = argv[level_index + 1]
|
||||
try:
|
||||
level = int(level_value)
|
||||
# Only consider extreme values (>=1000) for special handling
|
||||
return level >= 1000
|
||||
except ValueError:
|
||||
pass
|
||||
except (ValueError, IndexError):
|
||||
pass
|
||||
return False
|
||||
|
||||
|
||||
def _handle_parsing_errors(e: SystemExit, argv: list[str] | None) -> int:
|
||||
"""Handle SystemExit exceptions from argument parsing.
|
||||
|
||||
Args:
|
||||
e: SystemExit exception from argparse
|
||||
argv: Command line arguments, if any
|
||||
|
||||
Returns:
|
||||
int: Appropriate exit code
|
||||
""" # Help was requested
|
||||
if e.code == 0:
|
||||
return 0
|
||||
elif e.code == 2 and argv:
|
||||
# For now, keep standard argparse behavior (exit code 2)
|
||||
# Future versions may convert specific validation errors to exit code 1
|
||||
pass
|
||||
|
||||
# Return the original exit code for other cases
|
||||
return int(e.code) if e.code is not None else 1
|
||||
|
||||
|
||||
def _parse_arguments(parser, argv: list[str] | None):
|
||||
"""Parse command line arguments with error handling.
|
||||
|
||||
Args:
|
||||
parser: The argument parser
|
||||
argv: Command line arguments
|
||||
|
||||
Returns:
|
||||
tuple: (args, error_code) where error_code is None for success
|
||||
"""
|
||||
try:
|
||||
args = parser.parse_args(argv)
|
||||
return args, None
|
||||
except SystemExit as e:
|
||||
return None, _handle_parsing_errors(e, argv)
|
||||
|
||||
|
||||
def _execute_command(args, parser) -> int:
|
||||
"""Execute the parsed command.
|
||||
|
||||
Args:
|
||||
args: Parsed command line arguments
|
||||
parser: The argument parser for help display
|
||||
|
||||
Returns:
|
||||
int: Exit code from command execution
|
||||
"""
|
||||
# Handle --version flag
|
||||
if hasattr(args, "version") and args.version:
|
||||
return cmd_version(args)
|
||||
|
||||
if not hasattr(args, "func"):
|
||||
parser.print_help()
|
||||
return 1
|
||||
|
||||
return args.func(args)
|
||||
|
||||
|
||||
def main(argv: list[str] | None = None) -> int:
|
||||
"""Main entry point for the tzst command-line interface.
|
||||
|
||||
@@ -631,7 +986,8 @@ def main(argv: list[str] | None = None) -> int:
|
||||
Returns:
|
||||
int: Exit code for the program
|
||||
- 0: Success
|
||||
- 1: No command specified (help displayed)
|
||||
- 1: Invalid compression level, filter, or command error
|
||||
- 2: Argument parsing error (help, unknown options)
|
||||
- Other codes: Specific to individual command handlers
|
||||
|
||||
Note:
|
||||
@@ -643,16 +999,14 @@ def main(argv: list[str] | None = None) -> int:
|
||||
:func:`create_parser`: Creates the argument parser used by this function
|
||||
"""
|
||||
print_banner()
|
||||
print()
|
||||
|
||||
parser = create_parser()
|
||||
args = parser.parse_args(argv)
|
||||
args, error_code = _parse_arguments(parser, argv)
|
||||
|
||||
if not hasattr(args, "func"):
|
||||
parser.print_help()
|
||||
return 1
|
||||
if error_code is not None:
|
||||
return error_code
|
||||
|
||||
return args.func(args)
|
||||
return _execute_command(args, parser)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
|
||||
+151
-26
@@ -78,7 +78,11 @@ class TzstArchive:
|
||||
|
||||
def __exit__(self, exc_type, exc_val, exc_tb):
|
||||
"""Exit context manager."""
|
||||
self.close()
|
||||
try:
|
||||
self.close()
|
||||
except Exception:
|
||||
# Suppress exceptions during cleanup to avoid masking original exceptions
|
||||
pass
|
||||
|
||||
def open(self):
|
||||
"""Open the archive.
|
||||
@@ -191,7 +195,10 @@ class TzstArchive:
|
||||
if not path.exists():
|
||||
raise FileNotFoundError(f"File not found: {name}")
|
||||
|
||||
self._tarfile.add(str(path), arcname=arcname, recursive=recursive)
|
||||
try:
|
||||
self._tarfile.add(str(path), arcname=arcname, recursive=recursive)
|
||||
except PermissionError as e:
|
||||
raise TzstArchiveError(f"Failed to add {name}: {e}") from e
|
||||
|
||||
def extract(
|
||||
self,
|
||||
@@ -242,16 +249,23 @@ class TzstArchive:
|
||||
raise RuntimeError(
|
||||
"Extracting specific members is not supported in streaming mode. "
|
||||
"Please use non-streaming mode for selective extraction, or extract all files."
|
||||
)
|
||||
|
||||
# Prepare extraction arguments - filters are always supported in Python 3.12+
|
||||
extract_kwargs = {"filter": filter}
|
||||
|
||||
) # Prepare extraction arguments - different parameters for extract vs extractall
|
||||
try:
|
||||
if member:
|
||||
# extract() accepts set_attrs, numeric_owner, and filter
|
||||
extract_kwargs = {
|
||||
"set_attrs": set_attrs,
|
||||
"numeric_owner": numeric_owner,
|
||||
"filter": filter,
|
||||
}
|
||||
self._tarfile.extract(member, path=extract_path, **extract_kwargs)
|
||||
else:
|
||||
self._tarfile.extractall(path=extract_path, **extract_kwargs)
|
||||
# extractall() only accepts numeric_owner and filter (no set_attrs)
|
||||
extractall_kwargs = {
|
||||
"numeric_owner": numeric_owner,
|
||||
"filter": filter,
|
||||
}
|
||||
self._tarfile.extractall(path=extract_path, **extractall_kwargs)
|
||||
except (tarfile.StreamError, OSError) as e:
|
||||
if self.streaming and (
|
||||
"seeking" in str(e).lower() or "stream" in str(e).lower()
|
||||
@@ -281,6 +295,81 @@ class TzstArchive:
|
||||
|
||||
return self._tarfile.extractfile(member)
|
||||
|
||||
def extractall(
|
||||
self,
|
||||
path: str | Path = ".",
|
||||
members: list[tarfile.TarInfo] | None = None,
|
||||
*,
|
||||
numeric_owner: bool = False,
|
||||
filter: str | Callable | None = "data",
|
||||
):
|
||||
"""
|
||||
Extract all members from the archive.
|
||||
|
||||
Args:
|
||||
path: Destination directory (default: current directory)
|
||||
members: Specific members to extract (None for all)
|
||||
numeric_owner: Whether to use numeric owner IDs
|
||||
filter: Extraction filter for security. Can be:
|
||||
- 'data': Safe filter for cross-platform data archives
|
||||
- 'tar': Honor most tar features but block dangerous ones
|
||||
- 'fully_trusted': Honor all metadata (trusted archives only)
|
||||
- None: Use default behavior (may show deprecation warning)
|
||||
- callable: Custom filter function
|
||||
|
||||
Warning:
|
||||
Never extract archives from untrusted sources without proper filtering.
|
||||
The 'data' filter is recommended for most use cases as it prevents
|
||||
dangerous security issues like path traversal attacks.
|
||||
|
||||
Note:
|
||||
In streaming mode, extracting specific members is not supported.
|
||||
Some extraction operations may be limited due to the sequential
|
||||
nature of streaming mode.
|
||||
|
||||
See Also:
|
||||
:meth:`extract`: Extract a single member from the archive
|
||||
:func:`extract_archive`: Convenience function for extracting archives
|
||||
"""
|
||||
if not self._tarfile:
|
||||
raise RuntimeError("Archive not open")
|
||||
if not self.mode.startswith("r"):
|
||||
raise RuntimeError("Archive not open for reading")
|
||||
|
||||
if self.streaming and members is not None:
|
||||
# Specific member extraction not supported in streaming mode
|
||||
raise RuntimeError(
|
||||
"Extracting specific members is not supported in streaming mode. "
|
||||
"Please use non-streaming mode for selective extraction, "
|
||||
"or extract all files."
|
||||
)
|
||||
|
||||
extract_path = Path(path)
|
||||
extract_path.mkdir(parents=True, exist_ok=True)
|
||||
|
||||
try:
|
||||
# extractall() accepts numeric_owner, filter, and members parameters
|
||||
extractall_kwargs = {
|
||||
"numeric_owner": numeric_owner,
|
||||
"filter": filter,
|
||||
}
|
||||
if members is not None:
|
||||
extractall_kwargs["members"] = members
|
||||
|
||||
self._tarfile.extractall(path=extract_path, **extractall_kwargs)
|
||||
except (tarfile.StreamError, OSError) as e:
|
||||
if self.streaming and (
|
||||
"seeking" in str(e).lower() or "stream" in str(e).lower()
|
||||
):
|
||||
raise RuntimeError(
|
||||
"Extraction failed in streaming mode due to archive "
|
||||
"structure limitations. This archive may require "
|
||||
"non-streaming mode for extraction. "
|
||||
f"Original error: {e}"
|
||||
) from e
|
||||
else:
|
||||
raise
|
||||
|
||||
def getmembers(self) -> list[tarfile.TarInfo]:
|
||||
"""Get list of all members in the archive."""
|
||||
if not self._tarfile:
|
||||
@@ -461,25 +550,61 @@ def _create_archive_impl(
|
||||
# Find common parent directory for relative paths
|
||||
if files:
|
||||
file_paths = [Path(f) for f in files if Path(f).exists()]
|
||||
if file_paths:
|
||||
# Find the common parent directory
|
||||
try:
|
||||
common_parent = Path(os.path.commonpath([p.parent for p in file_paths]))
|
||||
except ValueError:
|
||||
# No common path, use parent of first file
|
||||
common_parent = file_paths[0].parent
|
||||
|
||||
# Change to common parent directory to get relative paths
|
||||
original_cwd = Path.cwd()
|
||||
try:
|
||||
os.chdir(common_parent)
|
||||
if file_paths: # Special handling for current directory "."
|
||||
current_dir = Path.cwd().resolve()
|
||||
if len(file_paths) == 1 and (
|
||||
str(file_paths[0]) == "." or file_paths[0].resolve() == current_dir
|
||||
): # When adding current directory, add its contents without "./" prefix
|
||||
with TzstArchive(archive_path, "w", compression_level) as archive:
|
||||
for file_path in file_paths:
|
||||
# Calculate relative path from common parent
|
||||
relative_path = file_path.relative_to(common_parent)
|
||||
archive.add(str(relative_path))
|
||||
finally:
|
||||
os.chdir(original_cwd)
|
||||
# Add all items in current directory, excluding archive and temp files
|
||||
archive_abs_path = archive_path.resolve()
|
||||
archive_name = archive_path.name
|
||||
for item in Path(".").iterdir():
|
||||
item_abs_path = item.resolve()
|
||||
# Skip archives and temp files for consistency
|
||||
if (
|
||||
item_abs_path == archive_abs_path
|
||||
or item.name == archive_name
|
||||
or (
|
||||
item.name.startswith(".") and item.name.endswith(".tmp")
|
||||
)
|
||||
or item.suffix.lower() in [".tzst", ".zst"]
|
||||
or item.name.lower().endswith(".tar.zst")
|
||||
):
|
||||
continue
|
||||
# Use item name as archive name to avoid "./" prefix
|
||||
item_name = str(item.name).replace("\\", "/")
|
||||
archive.add(str(item), arcname=item_name)
|
||||
else:
|
||||
# Find the common parent directory
|
||||
try:
|
||||
common_parent = Path(
|
||||
os.path.commonpath([p.parent for p in file_paths])
|
||||
)
|
||||
except ValueError:
|
||||
# No common path, use parent of first file
|
||||
common_parent = file_paths[
|
||||
0
|
||||
].parent # Change to common parent directory to get relative paths
|
||||
original_cwd = Path.cwd()
|
||||
# Convert archive path to absolute to avoid issues when changing working directory
|
||||
absolute_archive_path = archive_path.resolve()
|
||||
try:
|
||||
os.chdir(common_parent)
|
||||
with TzstArchive(
|
||||
absolute_archive_path, "w", compression_level
|
||||
) as archive:
|
||||
for file_path in file_paths:
|
||||
# Calculate relative path from common parent
|
||||
relative_path = file_path.relative_to(common_parent)
|
||||
# Normalize path separators and remove Windows prefixes
|
||||
path_str = str(relative_path).replace("\\", "/")
|
||||
if path_str.startswith("./") or path_str.startswith(".\\"):
|
||||
path_str = path_str[2:]
|
||||
# Use arcname to control the name in the archive
|
||||
archive.add(str(relative_path), arcname=path_str)
|
||||
finally:
|
||||
os.chdir(original_cwd)
|
||||
else:
|
||||
raise FileNotFoundError("No valid files found")
|
||||
else:
|
||||
|
||||
@@ -0,0 +1 @@
|
||||
"""CLI interface tests for tzst."""
|
||||
File diff suppressed because it is too large.
Load diff
+138
-1
@@ -1,5 +1,6 @@
|
||||
"""Test configuration and fixtures."""
|
||||
|
||||
import os
|
||||
import tempfile
|
||||
from pathlib import Path
|
||||
|
||||
@@ -9,8 +10,37 @@ import pytest
|
||||
@pytest.fixture
|
||||
def temp_dir():
|
||||
"""Create a temporary directory for tests."""
|
||||
with tempfile.TemporaryDirectory() as tmpdir:
|
||||
tmpdir = tempfile.mkdtemp()
|
||||
try:
|
||||
yield Path(tmpdir)
|
||||
finally:
|
||||
# Robust cleanup for Windows
|
||||
def cleanup_dir(path):
|
||||
import shutil
|
||||
import stat
|
||||
|
||||
def handle_remove_readonly(func, path, exc):
|
||||
"""Error handler for Windows readonly files."""
|
||||
try:
|
||||
os.chmod(path, stat.S_IWRITE)
|
||||
func(path)
|
||||
except Exception:
|
||||
pass # If we still can't delete it, ignore
|
||||
|
||||
try:
|
||||
shutil.rmtree(path, onerror=handle_remove_readonly)
|
||||
except (PermissionError, OSError):
|
||||
# On Windows, sometimes files are still locked
|
||||
# Try to clean up what we can
|
||||
import time
|
||||
|
||||
time.sleep(0.1) # Brief pause
|
||||
try:
|
||||
shutil.rmtree(path, onerror=handle_remove_readonly)
|
||||
except Exception:
|
||||
pass # Final fallback - ignore cleanup errors
|
||||
|
||||
cleanup_dir(tmpdir)
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
@@ -49,3 +79,110 @@ def sample_files(temp_dir):
|
||||
def sample_archive_path(temp_dir):
|
||||
"""Get path for a sample archive."""
|
||||
return temp_dir / "test.tzst"
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def comprehensive_test_files(temp_dir):
|
||||
"""Create comprehensive test files covering edge cases found during testing."""
|
||||
files = []
|
||||
|
||||
# Empty file
|
||||
empty_file = temp_dir / "empty_file.txt"
|
||||
empty_file.touch()
|
||||
files.append(empty_file)
|
||||
|
||||
# File with only whitespace
|
||||
whitespace_file = temp_dir / "whitespace_only.txt"
|
||||
whitespace_file.write_text(" \n\t\n \n")
|
||||
files.append(whitespace_file)
|
||||
|
||||
# File with only newlines
|
||||
newlines_file = temp_dir / "newlines_only.txt"
|
||||
newlines_file.write_text("\n\n\n\n\n")
|
||||
files.append(newlines_file)
|
||||
|
||||
# Binary file with null bytes
|
||||
null_bytes_file = temp_dir / "null_bytes.bin"
|
||||
null_bytes_file.write_bytes(b"\x00\x00\x00\x01\x00\x02\x00\x00\x03")
|
||||
files.append(null_bytes_file)
|
||||
|
||||
# Large text file
|
||||
large_file = temp_dir / "large_file.txt"
|
||||
large_content = "This is a large file for testing compression.\n" * 10000
|
||||
large_file.write_text(large_content)
|
||||
files.append(large_file)
|
||||
|
||||
# Binary data file
|
||||
binary_data_file = temp_dir / "binary_data.bin"
|
||||
binary_content = bytes(range(256)) * 100 # 25.6KB of binary data
|
||||
binary_data_file.write_bytes(binary_content)
|
||||
files.append(binary_data_file)
|
||||
|
||||
# File with spaces and special characters in name
|
||||
special_chars_file = temp_dir / "file with spaces.txt"
|
||||
special_chars_file.write_text("File with special characters in name")
|
||||
files.append(special_chars_file)
|
||||
|
||||
# Unicode content file (should work on all platforms)
|
||||
unicode_content_file = temp_dir / "unicode_content.txt"
|
||||
unicode_content_file.write_text("Hello 世界! 🌍", encoding="utf-8")
|
||||
files.append(unicode_content_file)
|
||||
|
||||
# Create nested directory structure
|
||||
nested_dir = temp_dir / "nested" / "very" / "deep" / "directory"
|
||||
nested_dir.mkdir(parents=True, exist_ok=True)
|
||||
nested_file = nested_dir / "deepest_file.txt"
|
||||
nested_file.write_text("File in deeply nested structure")
|
||||
files.append(nested_file)
|
||||
|
||||
return files
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def platform_specific_files(temp_dir):
|
||||
"""Create platform-specific test files."""
|
||||
files = []
|
||||
|
||||
if os.name == "posix": # Unix/Linux
|
||||
# Create executable file
|
||||
script_file = temp_dir / "test_script.sh"
|
||||
script_file.write_text("#!/bin/bash\necho 'Hello World'\n")
|
||||
try:
|
||||
script_file.chmod(0o755)
|
||||
files.append(script_file)
|
||||
except OSError:
|
||||
pass
|
||||
|
||||
# Create symlink if possible
|
||||
target_file = temp_dir / "symlink_target.txt"
|
||||
target_file.write_text("Symlink target content")
|
||||
files.append(target_file)
|
||||
|
||||
symlink_file = temp_dir / "test_symlink.txt"
|
||||
try:
|
||||
symlink_file.symlink_to(target_file)
|
||||
files.append(symlink_file)
|
||||
except OSError:
|
||||
pass
|
||||
|
||||
return files
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def compression_test_files(temp_dir):
|
||||
"""Create files for testing different compression levels."""
|
||||
files = []
|
||||
|
||||
# Highly compressible file (repetitive content)
|
||||
compressible_file = temp_dir / "highly_compressible.txt"
|
||||
compressible_content = "A" * 5000 + "B" * 5000 + "C" * 5000
|
||||
compressible_file.write_text(compressible_content)
|
||||
files.append(compressible_file)
|
||||
|
||||
# Poorly compressible file (pseudo-random data)
|
||||
random_file = temp_dir / "poorly_compressible.bin"
|
||||
random_data = bytes((i * 7 + 23) % 256 for i in range(15000))
|
||||
random_file.write_bytes(random_data)
|
||||
files.append(random_file)
|
||||
|
||||
return files
|
||||
@@ -0,0 +1 @@
|
||||
"""Integration tests for tzst library functionality."""
|
||||
@@ -0,0 +1,228 @@
|
||||
"""Integration tests for tzst complex scenarios."""
|
||||
|
||||
import os
|
||||
|
||||
import pytest
|
||||
|
||||
from tzst import create_archive, extract_archive, list_archive
|
||||
from tzst import test_archive as tzst_test_archive
|
||||
|
||||
|
||||
class TestCurrentDirectoryArchiving:
|
||||
"""Test current directory archiving bug fixes."""
|
||||
|
||||
def test_current_directory_dot_contents_without_wrapper(self, temp_dir):
|
||||
"""Test that archiving '.' includes directory contents without wrapper folder."""
|
||||
# Create test structure
|
||||
work_dir = temp_dir / "work_space"
|
||||
work_dir.mkdir()
|
||||
|
||||
# Create test files
|
||||
file1 = work_dir / "file1.txt"
|
||||
file1.write_text("Content 1")
|
||||
file2 = work_dir / "file2.txt"
|
||||
file2.write_text("Content 2")
|
||||
|
||||
# Create subdirectory with file
|
||||
subdir = work_dir / "subdir"
|
||||
subdir.mkdir()
|
||||
file3 = subdir / "file3.txt"
|
||||
file3.write_text("Content 3")
|
||||
|
||||
# Change to work directory and create archive
|
||||
original_cwd = os.getcwd()
|
||||
try:
|
||||
os.chdir(work_dir)
|
||||
archive_path = work_dir / "test.tzst"
|
||||
|
||||
# Create archive with current directory
|
||||
create_archive(archive_path, ["."])
|
||||
|
||||
# List archive contents
|
||||
contents = list_archive(archive_path)
|
||||
content_names = [item["name"] for item in contents]
|
||||
|
||||
# Should NOT contain a wrapper directory, just file contents
|
||||
assert "file1.txt" in content_names
|
||||
assert "file2.txt" in content_names
|
||||
assert "subdir/file3.txt" in content_names
|
||||
|
||||
# Should NOT contain the work directory name itself
|
||||
assert work_dir.name not in content_names
|
||||
|
||||
# Extract and verify structure
|
||||
extract_dir = temp_dir / "extracted"
|
||||
extract_archive(archive_path, extract_dir)
|
||||
|
||||
# Verify content
|
||||
assert (extract_dir / "file1.txt").read_text() == "Content 1"
|
||||
assert (extract_dir / "file2.txt").read_text() == "Content 2"
|
||||
assert (extract_dir / "subdir" / "file3.txt").read_text() == "Content 3"
|
||||
|
||||
finally:
|
||||
os.chdir(original_cwd)
|
||||
|
||||
def test_current_directory_vs_normal_archiving(self, temp_dir):
|
||||
"""Test difference between current directory and normal directory archiving."""
|
||||
# Create test structure
|
||||
work_dir = temp_dir / "work_dir"
|
||||
work_dir.mkdir()
|
||||
other_dir = temp_dir / "other_dir"
|
||||
other_dir.mkdir()
|
||||
|
||||
# Create identical files in both directories
|
||||
(work_dir / "shared.txt").write_text("Shared content")
|
||||
(other_dir / "shared.txt").write_text("Shared content")
|
||||
|
||||
original_cwd = os.getcwd()
|
||||
try:
|
||||
# Archive current directory with "."
|
||||
os.chdir(work_dir)
|
||||
current_archive = work_dir / "current.tzst"
|
||||
create_archive(current_archive, ["."])
|
||||
current_contents = list_archive(current_archive)
|
||||
current_names = [item["name"] for item in current_contents]
|
||||
|
||||
# Archive other directory normally
|
||||
normal_archive = work_dir / "normal.tzst"
|
||||
create_archive(normal_archive, [str(other_dir)])
|
||||
normal_contents = list_archive(normal_archive)
|
||||
normal_names = [item["name"] for item in normal_contents]
|
||||
|
||||
# Current directory archiving should have file at root
|
||||
assert "shared.txt" in current_names
|
||||
|
||||
# Normal directory archiving should have directory wrapper
|
||||
assert f"{other_dir.name}/shared.txt" in normal_names
|
||||
assert "shared.txt" not in normal_names # Should not be at root
|
||||
|
||||
finally:
|
||||
os.chdir(original_cwd)
|
||||
|
||||
|
||||
class TestTemporaryFileExclusion:
|
||||
"""Test temporary file exclusion bug fixes."""
|
||||
|
||||
def test_exclude_temp_files_with_tmp_in_middle(self, temp_dir):
|
||||
"""Test that temporary files with .tmp in the middle are excluded."""
|
||||
work_dir = temp_dir / "temp_test"
|
||||
work_dir.mkdir()
|
||||
|
||||
# Regular files that should be included
|
||||
regular_file = work_dir / "regular.txt"
|
||||
regular_file.write_text("Regular content")
|
||||
|
||||
# Temporary files that should be excluded
|
||||
temp_file1 = work_dir / ".a.tzst.9uztm81l.tmp"
|
||||
temp_file2 = work_dir / ".backup.abc123.tmp"
|
||||
temp_file3 = work_dir / ".test.random.tmp"
|
||||
|
||||
temp_file1.write_text("Temp content 1")
|
||||
temp_file2.write_text("Temp content 2")
|
||||
temp_file3.write_text("Temp content 3")
|
||||
|
||||
# Files that look like temp files but shouldn't be excluded
|
||||
not_temp1 = work_dir / "something.tmp" # Doesn't start with .
|
||||
not_temp2 = work_dir / ".config" # Starts with . but no .tmp
|
||||
not_temp3 = work_dir / ".tmpfile" # Has tmp but not as separate component
|
||||
|
||||
not_temp1.write_text("Not temp 1")
|
||||
not_temp2.write_text("Not temp 2")
|
||||
not_temp3.write_text("Not temp 3")
|
||||
|
||||
original_cwd = os.getcwd()
|
||||
try:
|
||||
os.chdir(work_dir)
|
||||
archive_path = work_dir / "temp_exclusion.tzst"
|
||||
|
||||
# Create archive with current directory
|
||||
create_archive(archive_path, ["."])
|
||||
|
||||
# List archive contents
|
||||
contents = list_archive(archive_path)
|
||||
content_names = [item["name"] for item in contents]
|
||||
|
||||
# Regular file should be included
|
||||
assert "regular.txt" in content_names
|
||||
|
||||
# Files that look like temp but aren't should be included
|
||||
assert "something.tmp" in content_names
|
||||
assert ".config" in content_names
|
||||
assert ".tmpfile" in content_names
|
||||
|
||||
# Actual temp files should be excluded
|
||||
assert ".a.tzst.9uztm81l.tmp" not in content_names
|
||||
assert ".backup.abc123.tmp" not in content_names
|
||||
assert ".test.random.tmp" not in content_names
|
||||
|
||||
finally:
|
||||
os.chdir(original_cwd)
|
||||
|
||||
|
||||
@pytest.mark.integration
|
||||
class TestLargeFileOperations:
|
||||
"""Integration tests for large file operations."""
|
||||
|
||||
def test_large_archive_creation_and_extraction(self, temp_dir):
|
||||
"""Test creating and extracting archives with multiple large files."""
|
||||
# Create multiple large files
|
||||
large_files = []
|
||||
for i in range(3):
|
||||
large_file = temp_dir / f"large_{i}.txt"
|
||||
content = f"Large file {i} content line.\n" * 50000 # ~1.2MB each
|
||||
large_file.write_text(content)
|
||||
large_files.append(large_file)
|
||||
|
||||
# Create archive with high compression
|
||||
archive_path = temp_dir / "large_files.tzst"
|
||||
create_archive(archive_path, large_files, compression_level=22)
|
||||
|
||||
assert archive_path.exists()
|
||||
assert tzst_test_archive(archive_path) is True
|
||||
|
||||
# Extract and verify
|
||||
extract_dir = temp_dir / "large_extracted"
|
||||
extract_archive(archive_path, extract_dir)
|
||||
|
||||
for i, original_file in enumerate(large_files):
|
||||
extracted_file = extract_dir / f"large_{i}.txt"
|
||||
assert extracted_file.exists()
|
||||
assert extracted_file.read_text() == original_file.read_text()
|
||||
|
||||
|
||||
@pytest.mark.integration
|
||||
class TestComplexDirectoryStructures:
|
||||
"""Integration tests for complex directory structures."""
|
||||
|
||||
def test_deeply_nested_directories(self, temp_dir):
|
||||
"""Test archiving deeply nested directory structures."""
|
||||
# Create deep nested structure
|
||||
current_dir = temp_dir / "deep"
|
||||
current_dir.mkdir()
|
||||
|
||||
for level in range(10): # 10 levels deep
|
||||
current_dir = current_dir / f"level_{level}"
|
||||
current_dir.mkdir()
|
||||
|
||||
# Add a file at each level
|
||||
test_file = current_dir / f"file_at_level_{level}.txt"
|
||||
test_file.write_text(f"Content at level {level}")
|
||||
|
||||
# Archive the entire structure
|
||||
archive_path = temp_dir / "deep_structure.tzst"
|
||||
create_archive(archive_path, [temp_dir / "deep"])
|
||||
|
||||
assert archive_path.exists()
|
||||
assert tzst_test_archive(archive_path) is True
|
||||
|
||||
# Extract and verify structure is preserved
|
||||
extract_dir = temp_dir / "deep_extracted"
|
||||
extract_archive(archive_path, extract_dir)
|
||||
|
||||
# Verify deep structure
|
||||
test_deep_file = extract_dir / "deep"
|
||||
for level in range(10):
|
||||
test_deep_file = test_deep_file / f"level_{level}"
|
||||
file_at_level = test_deep_file / f"file_at_level_{level}.txt"
|
||||
assert file_at_level.exists()
|
||||
assert file_at_level.read_text() == f"Content at level {level}"
|
||||
@@ -1,291 +0,0 @@
|
||||
"""Tests for the CLI interface."""
|
||||
|
||||
import pytest
|
||||
|
||||
from tzst.cli import create_parser, main
|
||||
|
||||
|
||||
class TestCLIParser:
|
||||
"""Test CLI argument parsing."""
|
||||
|
||||
def test_parser_creation(self):
|
||||
"""Test that parser can be created."""
|
||||
parser = create_parser()
|
||||
assert parser is not None
|
||||
|
||||
def test_add_command_parsing(self):
|
||||
"""Test parsing of add command."""
|
||||
parser = create_parser()
|
||||
args = parser.parse_args(["a", "test.tzst", "file1.txt", "file2.txt"])
|
||||
|
||||
assert args.command == "a"
|
||||
assert args.archive == "test.tzst"
|
||||
assert args.files == ["file1.txt", "file2.txt"]
|
||||
assert args.compression_level == 3 # default
|
||||
|
||||
def test_extract_command_parsing(self):
|
||||
"""Test parsing of extract command."""
|
||||
parser = create_parser()
|
||||
args = parser.parse_args(["x", "test.tzst", "-o", "output"])
|
||||
|
||||
assert args.command == "x"
|
||||
assert args.archive == "test.tzst"
|
||||
assert args.output == "output"
|
||||
|
||||
def test_list_command_parsing(self):
|
||||
"""Test parsing of list command."""
|
||||
parser = create_parser()
|
||||
args = parser.parse_args(["l", "test.tzst", "-v"])
|
||||
|
||||
assert args.command == "l"
|
||||
assert args.archive == "test.tzst"
|
||||
assert args.verbose is True
|
||||
|
||||
def test_test_command_parsing(self):
|
||||
"""Test parsing of test command."""
|
||||
parser = create_parser()
|
||||
args = parser.parse_args(["t", "test.tzst"])
|
||||
|
||||
assert args.command == "t"
|
||||
assert args.archive == "test.tzst"
|
||||
|
||||
|
||||
class TestCLICommands:
|
||||
"""Test CLI command execution."""
|
||||
|
||||
def test_add_command(self, sample_files, temp_dir):
|
||||
"""Test add command functionality."""
|
||||
archive_path = temp_dir / "test.tzst"
|
||||
file_paths = [str(f) for f in sample_files if f.is_file()]
|
||||
|
||||
# Run add command
|
||||
result = main(["a", str(archive_path), *file_paths])
|
||||
|
||||
assert result == 0
|
||||
assert archive_path.exists()
|
||||
|
||||
def test_list_command(self, sample_files, temp_dir):
|
||||
"""Test list command functionality."""
|
||||
archive_path = temp_dir / "test.tzst"
|
||||
file_paths = [str(f) for f in sample_files if f.is_file()]
|
||||
|
||||
# Create archive first
|
||||
main(["a", str(archive_path), *file_paths])
|
||||
|
||||
# Run list command
|
||||
result = main(["l", str(archive_path)])
|
||||
|
||||
assert result == 0
|
||||
|
||||
def test_extract_command(self, sample_files, temp_dir):
|
||||
"""Test extract command functionality."""
|
||||
archive_path = temp_dir / "test.tzst"
|
||||
file_paths = [str(f) for f in sample_files if f.is_file()]
|
||||
extract_dir = temp_dir / "extracted"
|
||||
|
||||
# Create archive first
|
||||
main(["a", str(archive_path), *file_paths])
|
||||
|
||||
# Run extract command
|
||||
result = main(["x", str(archive_path), "-o", str(extract_dir)])
|
||||
|
||||
assert result == 0
|
||||
assert extract_dir.exists()
|
||||
|
||||
def test_test_command(self, sample_files, temp_dir):
|
||||
"""Test test command functionality."""
|
||||
archive_path = temp_dir / "test.tzst"
|
||||
file_paths = [str(f) for f in sample_files if f.is_file()]
|
||||
|
||||
# Create archive first
|
||||
main(["a", str(archive_path), *file_paths])
|
||||
|
||||
# Run test command
|
||||
result = main(["t", str(archive_path)])
|
||||
|
||||
assert result == 0
|
||||
|
||||
|
||||
class TestCLIErrorHandling:
|
||||
"""Test CLI error handling."""
|
||||
|
||||
def test_missing_archive(self, temp_dir):
|
||||
"""Test handling of missing archive file."""
|
||||
fake_archive = temp_dir / "fake.tzst"
|
||||
|
||||
result = main(["l", str(fake_archive)])
|
||||
assert result == 1
|
||||
|
||||
def test_missing_files_to_add(self, temp_dir):
|
||||
"""Test handling of missing files to add."""
|
||||
archive_path = temp_dir / "test.tzst"
|
||||
fake_file = temp_dir / "fake.txt"
|
||||
|
||||
result = main(["a", str(archive_path), str(fake_file)])
|
||||
assert result == 1
|
||||
|
||||
def test_no_command(self):
|
||||
"""Test handling of no command provided."""
|
||||
result = main([])
|
||||
assert result == 1
|
||||
|
||||
def test_keyboard_interrupt_handling(self, temp_dir):
|
||||
"""Test that KeyboardInterrupt is handled properly."""
|
||||
# This test is more conceptual since we can't easily simulate KeyboardInterrupt
|
||||
# in a unit test, but we can verify the error handling structure exists
|
||||
from tzst.cli import cmd_add
|
||||
|
||||
# Create a mock args object
|
||||
class MockArgs:
|
||||
def __init__(self):
|
||||
self.archive = str(temp_dir / "interrupt_test.tzst")
|
||||
self.files = ["non_existent_file.txt"]
|
||||
self.compression_level = 3
|
||||
|
||||
# Test that the function handles FileNotFoundError properly
|
||||
result = cmd_add(MockArgs())
|
||||
assert result == 1 # Should return error code for missing files
|
||||
|
||||
|
||||
class TestCLIAliases:
|
||||
"""Test CLI command aliases."""
|
||||
|
||||
def test_add_aliases(self, sample_files, temp_dir):
|
||||
"""Test add command aliases."""
|
||||
archive_path = temp_dir / "test.tzst"
|
||||
file_paths = [str(f) for f in sample_files if f.is_file()]
|
||||
|
||||
# Test 'add' alias
|
||||
result = main(["add", str(archive_path), *file_paths])
|
||||
assert result == 0
|
||||
|
||||
# Test 'create' alias
|
||||
archive_path2 = temp_dir / "test2.tzst"
|
||||
result = main(["create", str(archive_path2), *file_paths])
|
||||
assert result == 0
|
||||
|
||||
def test_extract_aliases(self, sample_files, temp_dir):
|
||||
"""Test extract command aliases."""
|
||||
archive_path = temp_dir / "test.tzst"
|
||||
file_paths = [str(f) for f in sample_files if f.is_file()]
|
||||
extract_dir = temp_dir / "extracted"
|
||||
|
||||
# Create archive first
|
||||
main(["a", str(archive_path), *file_paths])
|
||||
|
||||
# Test 'extract' alias
|
||||
result = main(["extract", str(archive_path), "-o", str(extract_dir)])
|
||||
assert result == 0
|
||||
|
||||
def test_list_aliases(self, sample_files, temp_dir):
|
||||
"""Test list command aliases."""
|
||||
archive_path = temp_dir / "test.tzst"
|
||||
file_paths = [str(f) for f in sample_files if f.is_file()]
|
||||
|
||||
# Create archive first
|
||||
main(["a", str(archive_path), *file_paths])
|
||||
|
||||
# Test 'list' alias
|
||||
result = main(["list", str(archive_path)])
|
||||
assert result == 0
|
||||
|
||||
|
||||
@pytest.mark.integration
|
||||
class TestCLIIntegration:
|
||||
"""Integration tests for CLI."""
|
||||
|
||||
def test_full_workflow(self, sample_files, temp_dir):
|
||||
"""Test complete workflow: create, list, test, extract."""
|
||||
archive_path = temp_dir / "workflow.tzst"
|
||||
file_paths = [str(f) for f in sample_files if f.is_file()]
|
||||
extract_dir = temp_dir / "workflow_extracted"
|
||||
|
||||
# Create archive
|
||||
result = main(["a", str(archive_path), *file_paths])
|
||||
assert result == 0
|
||||
assert archive_path.exists()
|
||||
|
||||
# List contents
|
||||
result = main(["l", str(archive_path)])
|
||||
assert result == 0
|
||||
|
||||
# Test integrity
|
||||
result = main(["t", str(archive_path)])
|
||||
assert result == 0
|
||||
|
||||
# Extract
|
||||
result = main(["x", str(archive_path), "-o", str(extract_dir)])
|
||||
assert result == 0
|
||||
assert extract_dir.exists()
|
||||
|
||||
# Verify extracted files exist and have correct content
|
||||
for file_path in sample_files:
|
||||
if file_path.is_file():
|
||||
relative_path = file_path.relative_to(sample_files[0].parent)
|
||||
extracted_file = extract_dir / relative_path
|
||||
assert extracted_file.exists()
|
||||
|
||||
# Compare content
|
||||
original_content = file_path.read_bytes()
|
||||
extracted_content = extracted_file.read_bytes()
|
||||
assert original_content == extracted_content
|
||||
|
||||
|
||||
class TestCLIStreamingOptions:
|
||||
"""Test CLI streaming options."""
|
||||
|
||||
def test_extract_streaming_flag(self, sample_files, temp_dir):
|
||||
"""Test extract command with streaming flag."""
|
||||
archive_path = temp_dir / "cli_streaming_test.tzst"
|
||||
file_paths = [f for f in sample_files if f.is_file()]
|
||||
|
||||
# Create archive first
|
||||
from tzst import create_archive
|
||||
|
||||
create_archive(archive_path, file_paths)
|
||||
|
||||
# Test extract with streaming
|
||||
parser = create_parser()
|
||||
args = parser.parse_args(
|
||||
["x", str(archive_path), "--streaming", "-o", str(temp_dir / "extracted")]
|
||||
)
|
||||
|
||||
assert args.command == "x"
|
||||
assert hasattr(args, "streaming")
|
||||
assert args.streaming is True
|
||||
|
||||
def test_list_streaming_flag(self, sample_files, temp_dir):
|
||||
"""Test list command with streaming flag."""
|
||||
archive_path = temp_dir / "cli_list_streaming.tzst"
|
||||
file_paths = [f for f in sample_files if f.is_file()]
|
||||
|
||||
# Create archive first
|
||||
from tzst import create_archive
|
||||
|
||||
create_archive(archive_path, file_paths)
|
||||
|
||||
# Test list with streaming
|
||||
parser = create_parser()
|
||||
args = parser.parse_args(["l", str(archive_path), "--streaming"])
|
||||
|
||||
assert args.command == "l"
|
||||
assert hasattr(args, "streaming")
|
||||
assert args.streaming is True
|
||||
|
||||
def test_test_streaming_flag(self, sample_files, temp_dir):
|
||||
"""Test test command with streaming flag."""
|
||||
archive_path = temp_dir / "cli_test_streaming.tzst"
|
||||
file_paths = [f for f in sample_files if f.is_file()]
|
||||
|
||||
# Create archive first
|
||||
from tzst import create_archive
|
||||
|
||||
create_archive(archive_path, file_paths)
|
||||
|
||||
# Test test command with streaming
|
||||
parser = create_parser()
|
||||
args = parser.parse_args(["t", str(archive_path), "--streaming"])
|
||||
|
||||
assert args.command == "t"
|
||||
assert hasattr(args, "streaming")
|
||||
assert args.streaming is True
|
||||
@@ -0,0 +1,405 @@
|
||||
"""Tests to cover missing lines in CLI and improve overall coverage."""
|
||||
|
||||
import sys
|
||||
from unittest.mock import patch
|
||||
|
||||
import pytest
|
||||
|
||||
from tzst.cli import _validate_files, main
|
||||
|
||||
|
||||
class TestCLIMissingLines:
|
||||
"""Test specific missing lines in CLI for improved coverage."""
|
||||
|
||||
def test_validate_files_os_error_handling(self, temp_dir):
|
||||
"""Test OSError handling in validate_files function."""
|
||||
# Create a test file
|
||||
test_file = temp_dir / "test.txt"
|
||||
test_file.write_text("test content") # Mock Path.exists to raise OSError
|
||||
with patch(
|
||||
"pathlib.Path.exists", side_effect=OSError("Permission denied")
|
||||
): # Should handle OSError gracefully and continue
|
||||
try:
|
||||
_validate_files([test_file])
|
||||
except OSError:
|
||||
pass # Expected to be caught and handled
|
||||
|
||||
def test_main_function_edge_cases(self, temp_dir):
|
||||
"""Test main function edge cases for missing line coverage."""
|
||||
# Test with minimal arguments that might hit edge cases
|
||||
test_file = temp_dir / "test.txt"
|
||||
test_file.write_text("test content")
|
||||
archive_path = temp_dir / "test.tzst" # Create archive
|
||||
result = main(["a", str(archive_path), str(test_file)])
|
||||
assert result == 0
|
||||
|
||||
# Test version command through main
|
||||
with patch("sys.exit"):
|
||||
try:
|
||||
main(["--version"])
|
||||
except SystemExit:
|
||||
pass
|
||||
|
||||
# Test help command variations
|
||||
with patch("sys.exit"):
|
||||
try:
|
||||
main(["--help"])
|
||||
except SystemExit:
|
||||
pass
|
||||
|
||||
def test_command_line_argument_edge_cases(self, temp_dir):
|
||||
"""Test command line argument edge cases."""
|
||||
test_file = temp_dir / "test.txt"
|
||||
test_file.write_text("test content")
|
||||
|
||||
# Test with various argument combinations that might hit missing lines
|
||||
archive_path = temp_dir / "test.tzst"
|
||||
|
||||
# Create archive with specific compression level
|
||||
result = main(["a", str(archive_path), str(test_file), "-c", "1"])
|
||||
assert result == 0
|
||||
|
||||
# Test list with streaming
|
||||
result = main(["l", str(archive_path), "--streaming"])
|
||||
assert result == 0 # Test extract with specific options
|
||||
extract_dir = temp_dir / "extracted"
|
||||
result = main(["x", str(archive_path), "-o", str(extract_dir)])
|
||||
assert result == 0
|
||||
|
||||
def test_error_handling_edge_cases(self, temp_dir):
|
||||
"""Test error handling edge cases in CLI."""
|
||||
# Test with invalid archive path
|
||||
invalid_path = temp_dir / "nonexistent" / "test.tzst"
|
||||
|
||||
result = main(["l", str(invalid_path)])
|
||||
assert result == 1
|
||||
|
||||
# Test with invalid compression level - should return argparse error code 2
|
||||
test_file = temp_dir / "test.txt"
|
||||
test_file.write_text("test content")
|
||||
archive_path = temp_dir / "test.tzst"
|
||||
|
||||
result = main(["a", str(archive_path), str(test_file), "-c", "50"])
|
||||
assert result == 2
|
||||
|
||||
def test_filter_option_edge_cases(self, temp_dir):
|
||||
"""Test filter option edge cases."""
|
||||
test_file = temp_dir / "test.txt"
|
||||
test_file.write_text("test content")
|
||||
archive_path = temp_dir / "test.tzst"
|
||||
|
||||
# Create archive
|
||||
result = main(["a", str(archive_path), str(test_file)])
|
||||
assert result == 0
|
||||
|
||||
# Test extract with different filters
|
||||
for filter_type in ["data", "tar", "fully_trusted"]:
|
||||
extract_dir = temp_dir / f"extracted_{filter_type}"
|
||||
result = main(
|
||||
[
|
||||
"x",
|
||||
str(archive_path),
|
||||
"-o",
|
||||
str(extract_dir),
|
||||
"--filter",
|
||||
filter_type,
|
||||
]
|
||||
)
|
||||
assert result == 0
|
||||
|
||||
def test_atomic_operation_edge_cases(self, temp_dir):
|
||||
"""Test atomic operation edge cases."""
|
||||
test_file = temp_dir / "test.txt"
|
||||
test_file.write_text("test content")
|
||||
archive_path = temp_dir / "test.tzst"
|
||||
|
||||
# Test with --no-atomic flag
|
||||
result = main(["a", str(archive_path), str(test_file), "--no-atomic"])
|
||||
assert result == 0
|
||||
|
||||
# Verify archive was created
|
||||
assert archive_path.exists()
|
||||
|
||||
def test_verbose_output_edge_cases(self, temp_dir, capsys):
|
||||
"""Test verbose output edge cases."""
|
||||
test_file = temp_dir / "test.txt"
|
||||
test_file.write_text("test content")
|
||||
archive_path = temp_dir / "test.tzst" # Create archive
|
||||
result = main(["a", str(archive_path), str(test_file)])
|
||||
assert result == 0
|
||||
|
||||
# Test verbose list
|
||||
result = main(["l", str(archive_path), "-v"])
|
||||
assert result == 0
|
||||
|
||||
captured = capsys.readouterr()
|
||||
assert len(captured.out) > 0
|
||||
|
||||
def test_command_validation_edge_cases(self):
|
||||
"""Test command validation edge cases."""
|
||||
# Test with empty arguments
|
||||
result = main([])
|
||||
assert result == 1
|
||||
|
||||
# Test with invalid command - should return argparse error code 2
|
||||
result = main(["invalid_command"])
|
||||
assert result == 2
|
||||
|
||||
@pytest.mark.skipif(sys.platform != "win32", reason="Windows-specific test")
|
||||
def test_windows_specific_functionality(self, temp_dir):
|
||||
"""Test Windows-specific functionality."""
|
||||
# Test Windows reserved names
|
||||
test_file = temp_dir / "test.txt"
|
||||
test_file.write_text("test content")
|
||||
archive_path = temp_dir / "test.tzst"
|
||||
|
||||
# Create archive
|
||||
result = main(["a", str(archive_path), str(test_file)])
|
||||
assert result == 0
|
||||
|
||||
# Test with Windows path separators
|
||||
windows_style_path = str(archive_path).replace("/", "\\")
|
||||
result = main(["l", windows_style_path])
|
||||
assert result == 0
|
||||
|
||||
def test_streaming_mode_edge_cases(self, temp_dir):
|
||||
"""Test streaming mode edge cases."""
|
||||
# Create a larger file for streaming tests
|
||||
large_file = temp_dir / "large.txt"
|
||||
large_file.write_text("x" * 10000) # 10KB file
|
||||
|
||||
archive_path = temp_dir / "streaming.tzst"
|
||||
|
||||
# Create archive
|
||||
result = main(["a", str(archive_path), str(large_file)])
|
||||
assert result == 0
|
||||
|
||||
# Test all commands with streaming
|
||||
result = main(["l", str(archive_path), "--streaming"])
|
||||
assert result == 0
|
||||
|
||||
result = main(["t", str(archive_path), "--streaming"])
|
||||
assert result == 0
|
||||
|
||||
extract_dir = temp_dir / "extracted_streaming"
|
||||
result = main(["x", str(archive_path), "-o", str(extract_dir), "--streaming"])
|
||||
assert result == 0
|
||||
|
||||
def test_compression_level_boundary_values(self, temp_dir):
|
||||
"""Test compression level boundary values."""
|
||||
test_file = temp_dir / "test.txt"
|
||||
test_file.write_text("test content")
|
||||
|
||||
# Test minimum compression level
|
||||
archive_path_min = temp_dir / "min_compression.tzst"
|
||||
result = main(["a", str(archive_path_min), str(test_file), "-c", "1"])
|
||||
assert result == 0
|
||||
|
||||
# Test maximum compression level
|
||||
archive_path_max = temp_dir / "max_compression.tzst"
|
||||
result = main(["a", str(archive_path_max), str(test_file), "-c", "22"])
|
||||
assert result == 0
|
||||
|
||||
def test_output_directory_creation_edge_cases(self, temp_dir):
|
||||
"""Test output directory creation edge cases."""
|
||||
test_file = temp_dir / "test.txt"
|
||||
test_file.write_text("test content")
|
||||
archive_path = temp_dir / "test.tzst"
|
||||
|
||||
# Create archive
|
||||
result = main(["a", str(archive_path), str(test_file)])
|
||||
assert result == 0
|
||||
|
||||
# Test extraction to nested directory that doesn't exist
|
||||
nested_extract_dir = temp_dir / "level1" / "level2" / "level3"
|
||||
result = main(["x", str(archive_path), "-o", str(nested_extract_dir)])
|
||||
assert result == 0
|
||||
|
||||
# Verify directory was created
|
||||
assert nested_extract_dir.exists()
|
||||
|
||||
def test_special_file_handling_edge_cases(self, temp_dir):
|
||||
"""Test special file handling edge cases."""
|
||||
# Create files with special characteristics
|
||||
empty_file = temp_dir / "empty.txt"
|
||||
empty_file.touch()
|
||||
|
||||
binary_file = temp_dir / "binary.bin"
|
||||
binary_file.write_bytes(b"\x00\x01\x02\x03\xff")
|
||||
|
||||
unicode_file = temp_dir / "unicode.txt"
|
||||
unicode_file.write_text("Hello 世界 🌍", encoding="utf-8")
|
||||
|
||||
archive_path = temp_dir / "special.tzst"
|
||||
|
||||
# Create archive with special files
|
||||
result = main(
|
||||
[
|
||||
"a",
|
||||
str(archive_path),
|
||||
str(empty_file),
|
||||
str(binary_file),
|
||||
str(unicode_file),
|
||||
]
|
||||
)
|
||||
assert result == 0
|
||||
|
||||
# Test list and extract
|
||||
result = main(["l", str(archive_path)])
|
||||
assert result == 0
|
||||
|
||||
extract_dir = temp_dir / "extracted_special"
|
||||
result = main(["x", str(archive_path), "-o", str(extract_dir)])
|
||||
assert result == 0
|
||||
|
||||
|
||||
class TestPlatformSpecificMissingLines:
|
||||
"""Test platform-specific functionality to improve coverage."""
|
||||
|
||||
@pytest.mark.skipif(
|
||||
sys.platform != "win32", reason="Windows-specific functionality"
|
||||
)
|
||||
def test_windows_long_path_edge_cases(self, temp_dir):
|
||||
"""Test Windows long path handling edge cases."""
|
||||
# Create a very deep directory structure
|
||||
deep_dir = temp_dir
|
||||
for i in range(10):
|
||||
deep_dir = deep_dir / f"very_long_directory_name_{i}"
|
||||
deep_dir.mkdir(parents=True, exist_ok=True)
|
||||
|
||||
deep_file = deep_dir / "deep_file.txt"
|
||||
deep_file.write_text("Content in deeply nested file")
|
||||
|
||||
archive_path = temp_dir / "deep.tzst"
|
||||
|
||||
# Test archiving deep structure
|
||||
result = main(["a", str(archive_path), str(deep_file)])
|
||||
assert result == 0
|
||||
|
||||
# Test extraction
|
||||
extract_dir = temp_dir / "extracted_deep"
|
||||
result = main(["x", str(archive_path), "-o", str(extract_dir)])
|
||||
assert result == 0
|
||||
|
||||
@pytest.mark.skipif(
|
||||
sys.platform != "win32", reason="Windows-specific functionality"
|
||||
)
|
||||
def test_windows_reserved_names_edge_cases(self, temp_dir):
|
||||
"""Test Windows reserved names edge cases."""
|
||||
# Test with files that have problematic names on Windows
|
||||
normal_file = temp_dir / "normal.txt"
|
||||
normal_file.write_text("normal content")
|
||||
|
||||
# File with trailing space (problematic on Windows)
|
||||
space_file = temp_dir / "file_with_space .txt"
|
||||
space_file.write_text("space content")
|
||||
|
||||
archive_path = temp_dir / "reserved.tzst"
|
||||
|
||||
# Create archive
|
||||
result = main(["a", str(archive_path), str(normal_file), str(space_file)])
|
||||
assert result == 0
|
||||
|
||||
def test_unicode_handling_edge_cases(self, temp_dir):
|
||||
"""Test unicode handling edge cases."""
|
||||
# Create files with various unicode content
|
||||
files_to_create = [
|
||||
("chinese.txt", "你好世界"),
|
||||
("emoji.txt", "🎉🌟💫"),
|
||||
("mixed.txt", "Hello 世界! 🌍 Мир"),
|
||||
("special_chars.txt", "àáâãäåæçèéêë"),
|
||||
]
|
||||
|
||||
created_files = []
|
||||
for filename, content in files_to_create:
|
||||
file_path = temp_dir / filename
|
||||
file_path.write_text(content, encoding="utf-8")
|
||||
created_files.append(file_path)
|
||||
|
||||
archive_path = temp_dir / "unicode.tzst" # Create archive
|
||||
file_args = [str(f) for f in created_files]
|
||||
result = main(["a", str(archive_path), *file_args])
|
||||
assert result == 0
|
||||
|
||||
# Test extraction
|
||||
extract_dir = temp_dir / "extracted_unicode"
|
||||
result = main(["x", str(archive_path), "-o", str(extract_dir)])
|
||||
assert result == 0
|
||||
|
||||
# Verify unicode content is preserved
|
||||
for filename, original_content in files_to_create:
|
||||
extracted_file = extract_dir / filename
|
||||
assert extracted_file.exists()
|
||||
extracted_content = extracted_file.read_text(encoding="utf-8")
|
||||
assert extracted_content == original_content
|
||||
|
||||
def test_performance_edge_cases(self, temp_dir):
|
||||
"""Test performance-related edge cases."""
|
||||
# Create many small files
|
||||
files = []
|
||||
for i in range(50): # Create 50 small files
|
||||
file_path = temp_dir / f"small_{i:03d}.txt"
|
||||
file_path.write_text(f"Content of file {i}")
|
||||
files.append(file_path)
|
||||
|
||||
archive_path = temp_dir / "many_files.tzst" # Create archive with many files
|
||||
file_args = [str(f) for f in files]
|
||||
result = main(["a", str(archive_path), *file_args])
|
||||
assert result == 0
|
||||
|
||||
# Test listing (should handle many files efficiently)
|
||||
result = main(["l", str(archive_path)])
|
||||
assert result == 0
|
||||
|
||||
# Test extraction
|
||||
extract_dir = temp_dir / "extracted_many"
|
||||
result = main(["x", str(archive_path), "-o", str(extract_dir)])
|
||||
assert result == 0
|
||||
|
||||
def test_error_recovery_edge_cases(self, temp_dir):
|
||||
"""Test error recovery edge cases."""
|
||||
test_file = temp_dir / "test.txt"
|
||||
test_file.write_text("test content")
|
||||
archive_path = temp_dir / "test.tzst"
|
||||
|
||||
# Create archive
|
||||
result = main(["a", str(archive_path), str(test_file)])
|
||||
assert result == 0
|
||||
|
||||
# Test with readonly archive
|
||||
archive_path.chmod(0o444) # Make read-only
|
||||
|
||||
try:
|
||||
# Should handle read-only archive gracefully
|
||||
result = main(["l", str(archive_path)])
|
||||
assert result == 0
|
||||
finally:
|
||||
# Restore write permissions for cleanup
|
||||
archive_path.chmod(0o644)
|
||||
|
||||
def test_cross_platform_compatibility(self, temp_dir):
|
||||
"""Test cross-platform compatibility features."""
|
||||
# Create files with various characteristics
|
||||
text_file = temp_dir / "text.txt"
|
||||
text_file.write_text("Cross-platform text content\n")
|
||||
|
||||
binary_file = temp_dir / "binary.dat"
|
||||
binary_file.write_bytes(bytes(range(256)))
|
||||
|
||||
archive_path = temp_dir / "cross_platform.tzst"
|
||||
|
||||
# Create archive
|
||||
result = main(["a", str(archive_path), str(text_file), str(binary_file)])
|
||||
assert result == 0
|
||||
|
||||
# Test with different compression levels
|
||||
for level in [1, 11, 22]:
|
||||
archive_path_level = temp_dir / f"cross_platform_level_{level}.tzst"
|
||||
result = main(
|
||||
["a", str(archive_path_level), str(text_file), "-c", str(level)]
|
||||
)
|
||||
assert result == 0
|
||||
|
||||
# Verify can be read back
|
||||
result = main(["t", str(archive_path_level)])
|
||||
assert result == 0
|
||||
@@ -1,186 +0,0 @@
|
||||
"""Tests for CLI security filter options."""
|
||||
|
||||
from pathlib import Path
|
||||
from unittest.mock import MagicMock, patch
|
||||
|
||||
import pytest
|
||||
|
||||
from tzst.cli import create_parser
|
||||
|
||||
|
||||
class TestCLISecurityFilters:
|
||||
"""Test CLI security filter options."""
|
||||
|
||||
def test_extract_filter_argument_parsing(self):
|
||||
"""Test that --filter argument is parsed correctly for extract commands."""
|
||||
parser = create_parser()
|
||||
|
||||
# Test x/extract command with filter
|
||||
args = parser.parse_args(["x", "test.tzst", "--filter", "data"])
|
||||
assert args.command == "x"
|
||||
assert args.filter == "data"
|
||||
|
||||
args = parser.parse_args(["extract", "test.tzst", "--filter", "tar"])
|
||||
assert args.command == "extract"
|
||||
assert args.filter == "tar"
|
||||
|
||||
args = parser.parse_args(["x", "test.tzst", "--filter", "fully_trusted"])
|
||||
assert args.filter == "fully_trusted"
|
||||
|
||||
# Test e/extract-flat command with filter
|
||||
args = parser.parse_args(["e", "test.tzst", "--filter", "data"])
|
||||
assert args.command == "e"
|
||||
assert args.filter == "data"
|
||||
|
||||
args = parser.parse_args(["extract-flat", "test.tzst", "--filter", "tar"])
|
||||
assert args.command == "extract-flat"
|
||||
assert args.filter == "tar"
|
||||
|
||||
def test_filter_default_value(self):
|
||||
"""Test that filter defaults to 'data'."""
|
||||
parser = create_parser()
|
||||
|
||||
# Test default for x command
|
||||
args = parser.parse_args(["x", "test.tzst"])
|
||||
assert args.filter == "data"
|
||||
|
||||
# Test default for e command
|
||||
args = parser.parse_args(["e", "test.tzst"])
|
||||
assert args.filter == "data"
|
||||
|
||||
def test_filter_invalid_choices(self):
|
||||
"""Test that invalid filter choices are rejected."""
|
||||
parser = create_parser()
|
||||
|
||||
# Invalid filter should raise SystemExit (argparse error)
|
||||
with pytest.raises(SystemExit):
|
||||
parser.parse_args(["x", "test.tzst", "--filter", "invalid"])
|
||||
|
||||
def test_filter_with_other_options(self):
|
||||
"""Test filter option combined with other options."""
|
||||
parser = create_parser() # Test with streaming and output directory
|
||||
args = parser.parse_args(
|
||||
[
|
||||
"x",
|
||||
"test.tzst",
|
||||
"file1.txt",
|
||||
"file2.txt",
|
||||
"--filter",
|
||||
"tar",
|
||||
"--streaming",
|
||||
"-o",
|
||||
"output_dir",
|
||||
]
|
||||
)
|
||||
assert args.filter == "tar"
|
||||
assert args.streaming is True
|
||||
assert args.output == "output_dir"
|
||||
assert args.files == ["file1.txt", "file2.txt"]
|
||||
|
||||
def test_cli_help_includes_security_info(self):
|
||||
"""Test that CLI help includes security information."""
|
||||
parser = create_parser()
|
||||
help_text = parser.format_help()
|
||||
|
||||
# Check that security-related help text is present
|
||||
assert "--filter" in help_text
|
||||
assert "data" in help_text
|
||||
assert "tar" in help_text
|
||||
assert "fully_trusted" in help_text
|
||||
assert "Security Note:" in help_text
|
||||
assert "untrusted sources" in help_text
|
||||
|
||||
def test_extract_command_uses_filter(self):
|
||||
"""Test that extract commands actually use the filter parameter."""
|
||||
# This is an integration test that would require actual CLI execution
|
||||
# For now, we'll test the argument parsing and ensure the functions receive the filter
|
||||
|
||||
from tzst.cli import cmd_extract_full
|
||||
|
||||
# Mock args object
|
||||
mock_args = MagicMock()
|
||||
mock_args.archive = "test.tzst"
|
||||
mock_args.output = None
|
||||
mock_args.files = None
|
||||
mock_args.streaming = False
|
||||
mock_args.filter = "tar"
|
||||
|
||||
# Test that the functions would use the filter (we can't easily test the actual
|
||||
# extraction without creating real archives, but we can verify the code path)
|
||||
with patch("tzst.cli.Path") as mock_path:
|
||||
mock_path.return_value.exists.return_value = True
|
||||
mock_path.return_value.cwd.return_value = Path(".")
|
||||
mock_path.side_effect = lambda x: Path(x)
|
||||
|
||||
with patch("tzst.cli.extract_archive") as mock_extract:
|
||||
# This will fail because archive doesn't exist, but we can see the call
|
||||
try:
|
||||
cmd_extract_full(mock_args)
|
||||
except Exception:
|
||||
pass
|
||||
|
||||
# Verify extract_archive was called with filter parameter
|
||||
if mock_extract.called:
|
||||
call_kwargs = mock_extract.call_args[1]
|
||||
assert "filter" in call_kwargs
|
||||
assert call_kwargs["filter"] == "tar"
|
||||
|
||||
|
||||
class TestCLISecurityDocumentation:
|
||||
"""Test CLI security documentation and help."""
|
||||
|
||||
def test_security_warning_in_help(self):
|
||||
"""Test that security warnings are present in help text."""
|
||||
parser = create_parser()
|
||||
help_text = parser.format_help()
|
||||
|
||||
# Security-related text should be present
|
||||
assert "Security Note:" in help_text
|
||||
assert "data" in help_text
|
||||
assert "filter" in help_text
|
||||
|
||||
def test_filter_help_text_contains_descriptions(self):
|
||||
"""Test that filter help text contains basic descriptions."""
|
||||
parser = create_parser()
|
||||
help_text = parser.format_help()
|
||||
|
||||
# Basic filter information should be present
|
||||
assert "data" in help_text
|
||||
assert "tar" in help_text
|
||||
assert "fully_trusted" in help_text
|
||||
assert "safest" in help_text
|
||||
|
||||
|
||||
class TestCLISecurityIntegration:
|
||||
"""Integration tests for CLI security features."""
|
||||
|
||||
def test_cli_filter_argument_flow(self):
|
||||
"""Test the complete flow of filter arguments through CLI."""
|
||||
parser = create_parser()
|
||||
|
||||
# Test all valid filter values
|
||||
for filter_value in ["data", "tar", "fully_trusted"]:
|
||||
args = parser.parse_args(["x", "test.tzst", "--filter", filter_value])
|
||||
assert args.filter == filter_value
|
||||
|
||||
args = parser.parse_args(["e", "test.tzst", "--filter", filter_value])
|
||||
assert args.filter == filter_value
|
||||
|
||||
def test_security_help_content_completeness(self):
|
||||
"""Test that security help content is comprehensive."""
|
||||
parser = create_parser()
|
||||
help_text = parser.format_help()
|
||||
|
||||
# Essential security information should be present
|
||||
security_keywords = [
|
||||
"Security Note:",
|
||||
"untrusted sources",
|
||||
"data",
|
||||
"tar",
|
||||
"fully_trusted",
|
||||
"safest",
|
||||
"filter",
|
||||
]
|
||||
|
||||
for keyword in security_keywords:
|
||||
assert keyword in help_text, f"Missing security keyword: {keyword}"
|
||||
@@ -0,0 +1,216 @@
|
||||
"""Tests to exercise conftest.py fixtures and improve coverage."""
|
||||
|
||||
from tzst import create_archive, extract_archive, list_archive
|
||||
from tzst.core import TzstArchive
|
||||
|
||||
|
||||
class TestConftestFixtures:
|
||||
"""Test all conftest.py fixtures to improve coverage."""
|
||||
|
||||
def test_comprehensive_test_files_fixture(self, comprehensive_test_files, temp_dir):
|
||||
"""Test the comprehensive_test_files fixture."""
|
||||
# Ensure we have the expected file types
|
||||
assert len(comprehensive_test_files) >= 9
|
||||
|
||||
file_names = [f.name for f in comprehensive_test_files]
|
||||
|
||||
# Check for specific files that should be created
|
||||
assert "empty_file.txt" in file_names
|
||||
assert "whitespace_only.txt" in file_names
|
||||
assert "newlines_only.txt" in file_names
|
||||
assert "null_bytes.bin" in file_names
|
||||
assert "large_file.txt" in file_names
|
||||
assert "binary_data.bin" in file_names
|
||||
assert "file with spaces.txt" in file_names
|
||||
assert "unicode_content.txt" in file_names
|
||||
assert "deepest_file.txt" in file_names
|
||||
|
||||
# Create archive with these comprehensive test files
|
||||
archive_path = temp_dir / "comprehensive.tzst"
|
||||
file_paths = [str(f) for f in comprehensive_test_files if f.is_file()]
|
||||
create_archive(archive_path, file_paths)
|
||||
|
||||
# Verify archive was created and contains expected files
|
||||
assert archive_path.exists()
|
||||
contents = list_archive(archive_path)
|
||||
assert len(contents) >= 9
|
||||
|
||||
def test_platform_specific_files_fixture(self, platform_specific_files, temp_dir):
|
||||
"""Test the platform_specific_files fixture."""
|
||||
# This fixture may return empty list on Windows, non-empty on Unix
|
||||
# Just ensure it doesn't crash and returns a list
|
||||
assert isinstance(platform_specific_files, list)
|
||||
|
||||
if platform_specific_files:
|
||||
# If we have platform-specific files, create an archive with them
|
||||
archive_path = temp_dir / "platform_specific.tzst"
|
||||
file_paths = [str(f) for f in platform_specific_files if f.is_file()]
|
||||
if file_paths:
|
||||
create_archive(archive_path, file_paths)
|
||||
assert archive_path.exists()
|
||||
|
||||
def test_compression_test_files_fixture(self, compression_test_files, temp_dir):
|
||||
"""Test the compression_test_files fixture."""
|
||||
assert len(compression_test_files) == 2
|
||||
|
||||
file_names = [f.name for f in compression_test_files]
|
||||
assert "highly_compressible.txt" in file_names
|
||||
assert "poorly_compressible.bin" in file_names
|
||||
|
||||
# Test different compression levels with these files
|
||||
for level in [1, 11, 22]:
|
||||
archive_path = temp_dir / f"compression_level_{level}.tzst"
|
||||
with TzstArchive(
|
||||
archive_path, mode="w", compression_level=level
|
||||
) as archive:
|
||||
for file_path in compression_test_files:
|
||||
if file_path.is_file():
|
||||
archive.add(str(file_path), arcname=file_path.name)
|
||||
|
||||
assert archive_path.exists()
|
||||
contents = list_archive(archive_path)
|
||||
assert len(contents) == 2
|
||||
|
||||
def test_combined_fixtures_workflow(
|
||||
self, comprehensive_test_files, compression_test_files, temp_dir
|
||||
):
|
||||
"""Test using multiple fixtures together."""
|
||||
all_files = comprehensive_test_files + compression_test_files
|
||||
file_paths = [str(f) for f in all_files if f.is_file()]
|
||||
|
||||
# Create archive with all files
|
||||
archive_path = temp_dir / "combined.tzst"
|
||||
create_archive(archive_path, file_paths)
|
||||
|
||||
# Extract and verify
|
||||
extract_dir = temp_dir / "extracted"
|
||||
extract_archive(archive_path, extract_dir)
|
||||
# Verify archive was created and extracted directory exists
|
||||
assert archive_path.exists()
|
||||
assert extract_dir.exists()
|
||||
|
||||
# Count files instead of checking exact names (due to nested structure)
|
||||
extracted_files = list(extract_dir.rglob("*"))
|
||||
extracted_file_count = len([f for f in extracted_files if f.is_file()])
|
||||
original_file_count = len([f for f in all_files if f.is_file()])
|
||||
|
||||
# Should have extracted at least some files
|
||||
assert extracted_file_count > 0
|
||||
assert extracted_file_count <= original_file_count
|
||||
|
||||
def test_unicode_content_file_handling(self, comprehensive_test_files, temp_dir):
|
||||
"""Test handling of unicode content specifically."""
|
||||
unicode_files = [f for f in comprehensive_test_files if "unicode" in f.name]
|
||||
assert len(unicode_files) >= 1
|
||||
|
||||
unicode_file = unicode_files[0]
|
||||
content = unicode_file.read_text(encoding="utf-8")
|
||||
assert "世界" in content
|
||||
assert "🌍" in content
|
||||
|
||||
# Create archive and verify unicode handling
|
||||
archive_path = temp_dir / "unicode.tzst"
|
||||
create_archive(archive_path, [str(unicode_file)])
|
||||
|
||||
# Extract and verify content is preserved
|
||||
extract_dir = temp_dir / "extracted_unicode"
|
||||
extract_archive(archive_path, extract_dir)
|
||||
|
||||
extracted_file = extract_dir / unicode_file.name
|
||||
extracted_content = extracted_file.read_text(encoding="utf-8")
|
||||
assert extracted_content == content
|
||||
|
||||
def test_special_character_filenames(self, comprehensive_test_files, temp_dir):
|
||||
"""Test files with special characters in names."""
|
||||
special_files = [f for f in comprehensive_test_files if " " in f.name]
|
||||
assert len(special_files) >= 1
|
||||
|
||||
archive_path = temp_dir / "special_chars.tzst"
|
||||
file_paths = [str(f) for f in special_files if f.is_file()]
|
||||
create_archive(archive_path, file_paths)
|
||||
|
||||
contents = list_archive(archive_path)
|
||||
assert any(" " in item["name"] for item in contents)
|
||||
|
||||
def test_deeply_nested_structure(self, comprehensive_test_files, temp_dir):
|
||||
"""Test deeply nested directory structure."""
|
||||
nested_files = [f for f in comprehensive_test_files if "deepest" in f.name]
|
||||
assert len(nested_files) >= 1
|
||||
|
||||
nested_file = nested_files[0]
|
||||
assert "nested" in str(nested_file.parent)
|
||||
|
||||
# Create archive maintaining directory structure
|
||||
archive_path = temp_dir / "nested.tzst"
|
||||
with TzstArchive(archive_path, mode="w") as archive:
|
||||
archive.add(
|
||||
str(nested_file), arcname=str(nested_file.relative_to(temp_dir))
|
||||
)
|
||||
|
||||
contents = list_archive(archive_path)
|
||||
assert any("nested" in item["name"] for item in contents)
|
||||
|
||||
def test_empty_and_whitespace_files(self, comprehensive_test_files, temp_dir):
|
||||
"""Test empty and whitespace-only files."""
|
||||
empty_files = [
|
||||
f
|
||||
for f in comprehensive_test_files
|
||||
if "empty" in f.name or "whitespace" in f.name or "newlines" in f.name
|
||||
]
|
||||
assert len(empty_files) >= 3
|
||||
|
||||
archive_path = temp_dir / "empty_whitespace.tzst"
|
||||
file_paths = [str(f) for f in empty_files if f.is_file()]
|
||||
create_archive(archive_path, file_paths)
|
||||
|
||||
# Extract and verify these special cases are handled
|
||||
extract_dir = temp_dir / "extracted_empty"
|
||||
extract_archive(archive_path, extract_dir)
|
||||
|
||||
for file_path in empty_files:
|
||||
if file_path.is_file():
|
||||
extracted_file = extract_dir / file_path.name
|
||||
assert extracted_file.exists()
|
||||
|
||||
def test_binary_data_handling(self, comprehensive_test_files, temp_dir):
|
||||
"""Test binary files with null bytes and binary data."""
|
||||
binary_files = [f for f in comprehensive_test_files if f.suffix == ".bin"]
|
||||
assert len(binary_files) >= 2
|
||||
|
||||
archive_path = temp_dir / "binary.tzst"
|
||||
file_paths = [str(f) for f in binary_files if f.is_file()]
|
||||
create_archive(archive_path, file_paths)
|
||||
|
||||
# Extract and verify binary content is preserved
|
||||
extract_dir = temp_dir / "extracted_binary"
|
||||
extract_archive(archive_path, extract_dir)
|
||||
|
||||
for file_path in binary_files:
|
||||
if file_path.is_file():
|
||||
extracted_file = extract_dir / file_path.name
|
||||
assert extracted_file.exists()
|
||||
# Verify binary content is identical
|
||||
original_content = file_path.read_bytes()
|
||||
extracted_content = extracted_file.read_bytes()
|
||||
assert original_content == extracted_content
|
||||
|
||||
def test_large_file_handling(self, comprehensive_test_files, temp_dir):
|
||||
"""Test large file handling."""
|
||||
large_files = [f for f in comprehensive_test_files if "large" in f.name]
|
||||
assert len(large_files) >= 1
|
||||
|
||||
large_file = large_files[0]
|
||||
# Verify it's actually large
|
||||
assert large_file.stat().st_size > 100000 # Should be > 100KB
|
||||
|
||||
archive_path = temp_dir / "large.tzst"
|
||||
create_archive(archive_path, [str(large_file)])
|
||||
|
||||
# Test with streaming mode
|
||||
archive_path_streaming = temp_dir / "large_streaming.tzst"
|
||||
with TzstArchive(archive_path_streaming, mode="w", streaming=True) as archive:
|
||||
archive.add(str(large_file), arcname=large_file.name)
|
||||
|
||||
# Both archives should exist
|
||||
assert archive_path.exists()
|
||||
assert archive_path_streaming.exists()
|
||||
+362
-492
@@ -1,335 +1,24 @@
|
||||
"""Tests for tzst core functionality."""
|
||||
"""Remaining core tests for tzst after reorganization.
|
||||
|
||||
import pytest
|
||||
This file contains test classes that weren't moved during the test organization:
|
||||
- TestExtensions - File extension handling
|
||||
- TestNonAtomicOperations - Non-atomic file operations
|
||||
- TestCompressionLevelEdgeCases - Compression level edge cases
|
||||
- TestExtractionFilters - Extraction filter security features
|
||||
- TestSecurityDocumentation - Security documentation
|
||||
- TestSecurityEdgeCases - Security edge cases
|
||||
- TestCurrentDirectoryEdgeCases - Current directory edge cases
|
||||
|
||||
Most other tests have been moved to organized subdirectories under tests/.
|
||||
"""
|
||||
|
||||
import os
|
||||
from unittest.mock import patch
|
||||
|
||||
from tzst import TzstArchive, create_archive, extract_archive, list_archive
|
||||
from tzst import test_archive as tzst_test_archive
|
||||
|
||||
|
||||
class TestTzstArchive:
|
||||
"""Test the TzstArchive class."""
|
||||
|
||||
def test_create_and_list_archive(self, sample_files, sample_archive_path):
|
||||
"""Test creating an archive and listing its contents."""
|
||||
# Create archive with relative paths using arcname
|
||||
with TzstArchive(sample_archive_path, "w") as archive:
|
||||
for file_path in sample_files:
|
||||
if file_path.is_file():
|
||||
relative_path = file_path.relative_to(sample_files[0].parent)
|
||||
archive.add(file_path, arcname=str(relative_path))
|
||||
|
||||
assert sample_archive_path.exists()
|
||||
|
||||
# List contents
|
||||
with TzstArchive(sample_archive_path, "r") as archive:
|
||||
contents = archive.list()
|
||||
|
||||
# Should have all files
|
||||
expected_names = [
|
||||
str(f.relative_to(sample_files[0].parent)).replace("\\", "/")
|
||||
for f in sample_files
|
||||
if f.is_file()
|
||||
]
|
||||
actual_names = [item["name"] for item in contents]
|
||||
|
||||
for expected in expected_names:
|
||||
assert expected in actual_names
|
||||
|
||||
def test_extract_archive(self, sample_files, sample_archive_path, temp_dir):
|
||||
"""Test extracting files from an archive."""
|
||||
# Create archive with relative paths using arcname
|
||||
with TzstArchive(sample_archive_path, "w") as archive:
|
||||
for file_path in sample_files:
|
||||
if file_path.is_file():
|
||||
relative_path = file_path.relative_to(sample_files[0].parent)
|
||||
archive.add(file_path, arcname=str(relative_path))
|
||||
|
||||
# Extract to new directory
|
||||
extract_dir = temp_dir / "extracted"
|
||||
with TzstArchive(sample_archive_path, "r") as archive:
|
||||
archive.extract(path=extract_dir)
|
||||
|
||||
# Verify extracted files
|
||||
for file_path in sample_files:
|
||||
if file_path.is_file():
|
||||
relative_path = file_path.relative_to(sample_files[0].parent)
|
||||
extracted_file = extract_dir / relative_path
|
||||
assert extracted_file.exists()
|
||||
|
||||
# Compare content
|
||||
original_content = file_path.read_bytes()
|
||||
extracted_content = extracted_file.read_bytes()
|
||||
assert original_content == extracted_content
|
||||
|
||||
def test_archive_test(self, sample_files, sample_archive_path):
|
||||
"""Test archive integrity testing."""
|
||||
# Create archive with relative paths using arcname
|
||||
with TzstArchive(sample_archive_path, "w") as archive:
|
||||
for file_path in sample_files:
|
||||
if file_path.is_file():
|
||||
relative_path = file_path.relative_to(sample_files[0].parent)
|
||||
archive.add(file_path, arcname=str(relative_path))
|
||||
|
||||
# Test archive integrity
|
||||
with TzstArchive(sample_archive_path, "r") as archive:
|
||||
assert archive.test() is True
|
||||
|
||||
def test_verbose_listing(self, sample_files, sample_archive_path):
|
||||
"""Test verbose listing of archive contents."""
|
||||
# Create archive with relative paths using arcname
|
||||
with TzstArchive(sample_archive_path, "w") as archive:
|
||||
for file_path in sample_files:
|
||||
if file_path.is_file():
|
||||
relative_path = file_path.relative_to(sample_files[0].parent)
|
||||
archive.add(file_path, arcname=str(relative_path))
|
||||
|
||||
# Get verbose listing
|
||||
with TzstArchive(sample_archive_path, "r") as archive:
|
||||
contents = archive.list(verbose=True)
|
||||
|
||||
# Check that verbose fields are present
|
||||
for item in contents:
|
||||
assert "mode" in item
|
||||
assert "mtime" in item
|
||||
assert "mtime_str" in item
|
||||
assert "uid" in item
|
||||
assert "gid" in item
|
||||
|
||||
def test_streaming_mode_archive(self, sample_files, sample_archive_path):
|
||||
"""Test streaming mode for reading archives."""
|
||||
# Create archive normally
|
||||
file_paths = [f for f in sample_files if f.is_file()]
|
||||
with TzstArchive(sample_archive_path, "w") as archive:
|
||||
for file_path in file_paths:
|
||||
relative_path = file_path.relative_to(sample_files[0].parent)
|
||||
archive.add(file_path, arcname=str(relative_path))
|
||||
|
||||
# Test streaming mode reading
|
||||
with TzstArchive(sample_archive_path, "r", streaming=True) as archive:
|
||||
contents = archive.list()
|
||||
assert len(contents) > 0
|
||||
|
||||
def test_streaming_vs_buffered_mode(self, sample_files, sample_archive_path):
|
||||
"""Test that streaming and buffered modes produce same results."""
|
||||
# Create archive
|
||||
file_paths = [f for f in sample_files if f.is_file()]
|
||||
with TzstArchive(sample_archive_path, "w") as archive:
|
||||
for file_path in file_paths:
|
||||
relative_path = file_path.relative_to(sample_files[0].parent)
|
||||
archive.add(file_path, arcname=str(relative_path))
|
||||
|
||||
# Read with buffered mode
|
||||
with TzstArchive(sample_archive_path, "r", streaming=False) as archive:
|
||||
buffered_contents = archive.list()
|
||||
|
||||
# Read with streaming mode
|
||||
with TzstArchive(sample_archive_path, "r", streaming=True) as archive:
|
||||
streaming_contents = archive.list()
|
||||
|
||||
# Results should be identical
|
||||
assert len(buffered_contents) == len(streaming_contents)
|
||||
for buffered, streaming in zip(
|
||||
buffered_contents, streaming_contents, strict=True
|
||||
):
|
||||
assert buffered["name"] == streaming["name"]
|
||||
assert buffered["size"] == streaming["size"]
|
||||
|
||||
|
||||
class TestConvenienceFunctions:
|
||||
"""Test the convenience functions."""
|
||||
|
||||
def test_create_archive_function(self, sample_files, sample_archive_path):
|
||||
"""Test create_archive function."""
|
||||
file_paths = [f for f in sample_files if f.is_file()]
|
||||
create_archive(sample_archive_path, file_paths)
|
||||
|
||||
assert sample_archive_path.exists()
|
||||
|
||||
# Verify contents
|
||||
contents = list_archive(sample_archive_path)
|
||||
assert len(contents) > 0
|
||||
|
||||
def test_extract_archive_function(
|
||||
self, sample_files, sample_archive_path, temp_dir
|
||||
):
|
||||
"""Test extract_archive function."""
|
||||
# Create archive first
|
||||
file_paths = [f for f in sample_files if f.is_file()]
|
||||
create_archive(sample_archive_path, file_paths)
|
||||
|
||||
# Extract
|
||||
extract_dir = temp_dir / "extracted"
|
||||
extract_archive(sample_archive_path, extract_dir)
|
||||
|
||||
# Verify extraction
|
||||
assert extract_dir.exists()
|
||||
extracted_files = list(extract_dir.rglob("*"))
|
||||
assert len(extracted_files) > 0
|
||||
|
||||
def test_extract_archive_flat(self, sample_files, sample_archive_path, temp_dir):
|
||||
"""Test flat extraction."""
|
||||
# Create archive first
|
||||
file_paths = [f for f in sample_files if f.is_file()]
|
||||
create_archive(sample_archive_path, file_paths)
|
||||
|
||||
# Extract flat
|
||||
extract_dir = temp_dir / "extracted_flat"
|
||||
extract_archive(sample_archive_path, extract_dir, flatten=True)
|
||||
|
||||
# Verify flat extraction (no subdirectories)
|
||||
assert extract_dir.exists()
|
||||
extracted_files = [f for f in extract_dir.iterdir() if f.is_file()]
|
||||
extracted_dirs = [d for d in extract_dir.iterdir() if d.is_dir()]
|
||||
|
||||
assert len(extracted_files) > 0
|
||||
assert len(extracted_dirs) == 0 # Should be flat
|
||||
|
||||
def test_list_archive_function(self, sample_files, sample_archive_path):
|
||||
"""Test list_archive function."""
|
||||
# Create archive first
|
||||
file_paths = [f for f in sample_files if f.is_file()]
|
||||
create_archive(sample_archive_path, file_paths)
|
||||
|
||||
# List contents
|
||||
contents = list_archive(sample_archive_path)
|
||||
assert len(contents) > 0
|
||||
|
||||
# Test verbose listing
|
||||
verbose_contents = list_archive(sample_archive_path, verbose=True)
|
||||
assert len(verbose_contents) == len(contents)
|
||||
assert "mode" in verbose_contents[0]
|
||||
|
||||
def test_test_archive_function(self, sample_files, sample_archive_path):
|
||||
"""Test test_archive function."""
|
||||
# Create archive first
|
||||
file_paths = [f for f in sample_files if f.is_file()]
|
||||
create_archive(sample_archive_path, file_paths)
|
||||
|
||||
# Test archive
|
||||
assert tzst_test_archive(sample_archive_path) is True
|
||||
|
||||
# Test non-existent archive
|
||||
fake_archive = sample_archive_path.parent / "fake.tzst"
|
||||
assert tzst_test_archive(fake_archive) is False
|
||||
|
||||
def test_streaming_convenience_functions(
|
||||
self, sample_files, sample_archive_path, temp_dir
|
||||
):
|
||||
"""Test convenience functions with streaming parameter."""
|
||||
# Create archive first
|
||||
file_paths = [f for f in sample_files if f.is_file()]
|
||||
create_archive(sample_archive_path, file_paths)
|
||||
|
||||
# Test list_archive with streaming
|
||||
contents_normal = list_archive(sample_archive_path, streaming=False)
|
||||
contents_streaming = list_archive(sample_archive_path, streaming=True)
|
||||
assert len(contents_normal) == len(contents_streaming)
|
||||
|
||||
# Test test_archive with streaming
|
||||
assert tzst_test_archive(sample_archive_path, streaming=False) is True
|
||||
assert tzst_test_archive(sample_archive_path, streaming=True) is True
|
||||
|
||||
# Test extract_archive with streaming
|
||||
extract_dir_normal = temp_dir / "extract_normal"
|
||||
extract_dir_streaming = temp_dir / "extract_streaming"
|
||||
|
||||
extract_archive(sample_archive_path, extract_dir_normal, streaming=False)
|
||||
extract_archive(sample_archive_path, extract_dir_streaming, streaming=True)
|
||||
|
||||
assert extract_dir_normal.exists()
|
||||
assert extract_dir_streaming.exists()
|
||||
|
||||
def test_atomic_file_operations(self, sample_files, temp_dir):
|
||||
"""Test atomic file operations."""
|
||||
archive_path = temp_dir / "atomic_test.tzst"
|
||||
file_paths = [f for f in sample_files if f.is_file()]
|
||||
|
||||
# Test with atomic operations enabled
|
||||
create_archive(archive_path, file_paths, use_temp_file=True)
|
||||
assert archive_path.exists()
|
||||
|
||||
# Verify archive is valid
|
||||
assert tzst_test_archive(archive_path) is True
|
||||
|
||||
def test_non_atomic_file_creation(self, sample_files, temp_dir):
|
||||
"""Test that non-atomic creation also works."""
|
||||
archive_path = temp_dir / "non_atomic_test.tzst"
|
||||
file_paths = [f for f in sample_files if f.is_file()]
|
||||
|
||||
# Test with atomic operations disabled
|
||||
create_archive(archive_path, file_paths, use_temp_file=False)
|
||||
assert archive_path.exists()
|
||||
|
||||
# Verify archive is valid
|
||||
assert tzst_test_archive(archive_path) is True
|
||||
|
||||
def test_atomic_cleanup_on_error(self, temp_dir):
|
||||
"""Test that temporary files are cleaned up on errors."""
|
||||
archive_path = temp_dir / "cleanup_test.tzst"
|
||||
|
||||
# Try to create archive with non-existent files
|
||||
with pytest.raises(FileNotFoundError):
|
||||
create_archive(archive_path, ["non_existent_file.txt"], use_temp_file=True)
|
||||
|
||||
# Archive should not exist
|
||||
assert not archive_path.exists()
|
||||
|
||||
# No temporary files should be left behind
|
||||
temp_files = list(temp_dir.glob(".cleanup_test.tzst.*"))
|
||||
assert len(temp_files) == 0
|
||||
|
||||
def test_compression_level_validation(self, sample_files, temp_dir):
|
||||
"""Test compression level validation."""
|
||||
file_paths = [f for f in sample_files if f.is_file()]
|
||||
|
||||
# Test valid compression levels
|
||||
for level in [1, 3, 10, 22]:
|
||||
archive_path = temp_dir / f"level_{level}.tzst"
|
||||
create_archive(archive_path, file_paths, compression_level=level)
|
||||
assert archive_path.exists()
|
||||
assert tzst_test_archive(archive_path) is True
|
||||
|
||||
# Test invalid compression levels
|
||||
for invalid_level in [0, 23, -1, 100]:
|
||||
archive_path = temp_dir / f"invalid_{invalid_level}.tzst"
|
||||
with pytest.raises(ValueError) as exc_info:
|
||||
create_archive(
|
||||
archive_path, file_paths, compression_level=invalid_level
|
||||
)
|
||||
|
||||
assert "compression level" in str(exc_info.value).lower()
|
||||
assert "1" in str(exc_info.value) and "22" in str(exc_info.value)
|
||||
|
||||
|
||||
class TestErrorHandling:
|
||||
"""Test error handling."""
|
||||
|
||||
def test_invalid_archive_mode(self, sample_archive_path):
|
||||
"""Test invalid archive mode."""
|
||||
with pytest.raises(ValueError):
|
||||
TzstArchive(sample_archive_path, "invalid")
|
||||
|
||||
def test_append_mode_not_supported(self, sample_archive_path):
|
||||
"""Test that append mode raises NotImplementedError."""
|
||||
with pytest.raises(NotImplementedError):
|
||||
TzstArchive(sample_archive_path, "a")
|
||||
|
||||
def test_archive_not_open(self, sample_archive_path):
|
||||
"""Test operations on non-open archive."""
|
||||
archive = TzstArchive(sample_archive_path, "r")
|
||||
|
||||
with pytest.raises(RuntimeError):
|
||||
archive.getnames()
|
||||
|
||||
def test_file_not_found(self, temp_dir):
|
||||
"""Test handling of non-existent files."""
|
||||
archive_path = temp_dir / "test.tzst"
|
||||
fake_file = temp_dir / "fake.txt"
|
||||
|
||||
with pytest.raises(FileNotFoundError):
|
||||
create_archive(archive_path, [fake_file])
|
||||
|
||||
|
||||
class TestExtensions:
|
||||
"""Test file extension handling."""
|
||||
|
||||
@@ -358,106 +47,13 @@ class TestExtensions:
|
||||
assert expected_path.exists()
|
||||
|
||||
|
||||
class TestStreamingMode:
|
||||
"""Test streaming mode improvements."""
|
||||
class TestNonAtomicOperations:
|
||||
"""Test non-atomic file operations (critical fix verification)."""
|
||||
|
||||
def test_streaming_archive_creation_and_extraction(self, sample_files, temp_dir):
|
||||
"""Test that streaming mode works for reading archives."""
|
||||
archive_path = temp_dir / "streaming_test.tzst"
|
||||
def test_non_atomic_archive_creation(self, sample_files, temp_dir):
|
||||
"""Test non-atomic archive creation works correctly."""
|
||||
file_paths = [f for f in sample_files if f.is_file()]
|
||||
|
||||
# Create archive normally
|
||||
create_archive(archive_path, file_paths)
|
||||
|
||||
# Test streaming mode reading
|
||||
with TzstArchive(archive_path, "r", streaming=True) as archive:
|
||||
contents = archive.list()
|
||||
assert len(contents) > 0
|
||||
|
||||
# Test extraction in streaming mode
|
||||
extract_dir = temp_dir / "streaming_extracted"
|
||||
try:
|
||||
archive.extract(path=extract_dir)
|
||||
assert extract_dir.exists()
|
||||
except RuntimeError as e:
|
||||
if "streaming mode" in str(e):
|
||||
# This is expected for some archives in streaming mode
|
||||
# Test that we can still read the contents
|
||||
assert len(contents) > 0
|
||||
else:
|
||||
raise
|
||||
|
||||
def test_streaming_vs_buffered_mode(self, sample_files, temp_dir):
|
||||
"""Test that streaming and buffered modes produce same results."""
|
||||
archive_path = temp_dir / "comparison_test.tzst"
|
||||
file_paths = [f for f in sample_files if f.is_file()]
|
||||
|
||||
# Create archive
|
||||
create_archive(archive_path, file_paths)
|
||||
|
||||
# Read with buffered mode
|
||||
with TzstArchive(archive_path, "r", streaming=False) as archive:
|
||||
buffered_contents = archive.list()
|
||||
|
||||
# Read with streaming mode
|
||||
with TzstArchive(archive_path, "r", streaming=True) as archive:
|
||||
streaming_contents = archive.list()
|
||||
|
||||
# Results should be identical
|
||||
assert len(buffered_contents) == len(streaming_contents)
|
||||
for buffered, streaming in zip(
|
||||
buffered_contents, streaming_contents, strict=True
|
||||
):
|
||||
assert buffered["name"] == streaming["name"]
|
||||
assert buffered["size"] == streaming["size"]
|
||||
|
||||
def test_convenience_functions_with_streaming(self, sample_files, temp_dir):
|
||||
"""Test convenience functions with streaming parameter."""
|
||||
archive_path = temp_dir / "convenience_streaming.tzst"
|
||||
file_paths = [f for f in sample_files if f.is_file()]
|
||||
|
||||
# Create archive
|
||||
create_archive(archive_path, file_paths)
|
||||
|
||||
# Test list_archive with streaming
|
||||
contents_normal = list_archive(archive_path, streaming=False)
|
||||
contents_streaming = list_archive(archive_path, streaming=True)
|
||||
assert len(contents_normal) == len(contents_streaming)
|
||||
|
||||
# Test test_archive with streaming
|
||||
assert tzst_test_archive(archive_path, streaming=False) is True
|
||||
assert tzst_test_archive(archive_path, streaming=True) is True
|
||||
|
||||
# Test extract_archive with streaming
|
||||
extract_dir_normal = temp_dir / "extract_normal"
|
||||
extract_dir_streaming = temp_dir / "extract_streaming"
|
||||
|
||||
extract_archive(archive_path, extract_dir_normal, streaming=False)
|
||||
extract_archive(archive_path, extract_dir_streaming, streaming=True)
|
||||
|
||||
assert extract_dir_normal.exists()
|
||||
assert extract_dir_streaming.exists()
|
||||
|
||||
|
||||
class TestAtomicFileOperations:
|
||||
"""Test atomic file operations."""
|
||||
|
||||
def test_atomic_file_creation(self, sample_files, temp_dir):
|
||||
"""Test that atomic file creation works."""
|
||||
archive_path = temp_dir / "atomic_test.tzst"
|
||||
file_paths = [f for f in sample_files if f.is_file()]
|
||||
|
||||
# Test with atomic operations enabled
|
||||
create_archive(archive_path, file_paths, use_temp_file=True)
|
||||
assert archive_path.exists()
|
||||
|
||||
# Verify archive is valid
|
||||
assert tzst_test_archive(archive_path) is True
|
||||
|
||||
def test_non_atomic_file_creation(self, sample_files, temp_dir):
|
||||
"""Test that non-atomic creation also works."""
|
||||
archive_path = temp_dir / "non_atomic_test.tzst"
|
||||
file_paths = [f for f in sample_files if f.is_file()]
|
||||
|
||||
# Test with atomic operations disabled
|
||||
create_archive(archive_path, file_paths, use_temp_file=False)
|
||||
@@ -466,79 +62,353 @@ class TestAtomicFileOperations:
|
||||
# Verify archive is valid
|
||||
assert tzst_test_archive(archive_path) is True
|
||||
|
||||
def test_atomic_cleanup_on_error(self, temp_dir):
|
||||
"""Test that temporary files are cleaned up on errors."""
|
||||
archive_path = temp_dir / "cleanup_test.tzst"
|
||||
# Verify contents
|
||||
contents = list_archive(archive_path)
|
||||
assert len(contents) > 0
|
||||
|
||||
# Try to create archive with non-existent files
|
||||
with pytest.raises(FileNotFoundError):
|
||||
create_archive(archive_path, ["non_existent_file.txt"], use_temp_file=True)
|
||||
|
||||
# Archive should not exist
|
||||
assert not archive_path.exists()
|
||||
|
||||
# No temporary files should be left behind
|
||||
temp_files = list(temp_dir.glob(".cleanup_test.tzst.*"))
|
||||
assert len(temp_files) == 0
|
||||
|
||||
|
||||
class TestAppendModeDocumentation:
|
||||
"""Test that append mode provides helpful error messages."""
|
||||
|
||||
def test_append_mode_error_message(self, temp_dir):
|
||||
"""Test that append mode raises informative error."""
|
||||
archive_path = temp_dir / "append_test.tzst"
|
||||
|
||||
with pytest.raises(NotImplementedError) as exc_info:
|
||||
TzstArchive(archive_path, "a")
|
||||
|
||||
error_msg = str(exc_info.value)
|
||||
assert "append mode" in error_msg.lower()
|
||||
assert "alternatives" in error_msg.lower() or "alternative" in error_msg.lower()
|
||||
assert "decompressing" in error_msg.lower()
|
||||
assert "recompressing" in error_msg.lower()
|
||||
|
||||
def test_append_mode_error_in_open(self, temp_dir):
|
||||
"""Test append mode error when opening existing archive."""
|
||||
archive_path = temp_dir / "append_open_test.tzst"
|
||||
|
||||
# Create an archive first
|
||||
with TzstArchive(archive_path, "w"):
|
||||
pass
|
||||
|
||||
# Try to open in append mode
|
||||
with pytest.raises(NotImplementedError) as exc_info:
|
||||
TzstArchive(archive_path, "a")
|
||||
|
||||
error_msg = str(exc_info.value)
|
||||
assert (
|
||||
"multiple archives" in error_msg.lower() or "recreate" in error_msg.lower()
|
||||
)
|
||||
|
||||
|
||||
class TestCompressionLevelValidation:
|
||||
"""Test compression level validation improvements."""
|
||||
|
||||
def test_valid_compression_levels(self, sample_files, temp_dir):
|
||||
"""Test that valid compression levels work."""
|
||||
def test_non_atomic_vs_atomic_equivalence(self, sample_files, temp_dir):
|
||||
"""Test that atomic and non-atomic modes produce equivalent results."""
|
||||
file_paths = [f for f in sample_files if f.is_file()]
|
||||
|
||||
for level in [1, 3, 10, 22]:
|
||||
archive_path = temp_dir / f"level_{level}.tzst"
|
||||
create_archive(archive_path, file_paths, compression_level=level)
|
||||
assert archive_path.exists()
|
||||
assert tzst_test_archive(archive_path) is True
|
||||
# Create with atomic mode
|
||||
atomic_archive = temp_dir / "atomic.tzst"
|
||||
create_archive(atomic_archive, file_paths, use_temp_file=True)
|
||||
|
||||
def test_invalid_compression_levels(self, sample_files, temp_dir):
|
||||
"""Test that invalid compression levels raise errors."""
|
||||
# Create with non-atomic mode
|
||||
non_atomic_archive = temp_dir / "non_atomic.tzst"
|
||||
create_archive(non_atomic_archive, file_paths, use_temp_file=False)
|
||||
|
||||
# Both should be valid
|
||||
assert tzst_test_archive(atomic_archive) is True
|
||||
assert tzst_test_archive(non_atomic_archive) is True
|
||||
|
||||
# Both should have same file contents
|
||||
atomic_contents = list_archive(atomic_archive)
|
||||
non_atomic_contents = list_archive(non_atomic_archive)
|
||||
assert len(atomic_contents) == len(non_atomic_contents)
|
||||
|
||||
def test_non_atomic_path_resolution(self, temp_dir):
|
||||
"""Test that non-atomic mode handles path resolution correctly."""
|
||||
# Create nested directory structure
|
||||
nested_dir = temp_dir / "level1" / "level2" / "level3"
|
||||
nested_dir.mkdir(parents=True, exist_ok=True)
|
||||
|
||||
test_file = nested_dir / "deep_file.txt"
|
||||
test_file.write_text("Deep file content")
|
||||
|
||||
# Create archive from parent directory using non-atomic mode
|
||||
archive_path = temp_dir / "deep_structure.tzst"
|
||||
create_archive(archive_path, [test_file], use_temp_file=False)
|
||||
|
||||
assert archive_path.exists()
|
||||
assert tzst_test_archive(archive_path) is True
|
||||
|
||||
|
||||
class TestCompressionLevelEdgeCases:
|
||||
"""Test compression level edge cases and validation."""
|
||||
|
||||
def test_compression_effectiveness(self, temp_dir):
|
||||
"""Test that higher compression levels produce smaller files."""
|
||||
# Create a large compressible file
|
||||
large_file = temp_dir / "compressible.txt"
|
||||
content = "This is highly compressible content. " * 10000
|
||||
large_file.write_text(content)
|
||||
|
||||
# Test different compression levels
|
||||
sizes = {}
|
||||
for level in [1, 11, 22]:
|
||||
archive_path = temp_dir / f"compressed_level_{level}.tzst"
|
||||
create_archive(archive_path, [large_file], compression_level=level)
|
||||
sizes[level] = archive_path.stat().st_size
|
||||
|
||||
# Higher compression should generally result in smaller files
|
||||
# (though this isn't guaranteed for all data types)
|
||||
assert sizes[1] > 0
|
||||
assert sizes[22] > 0
|
||||
|
||||
|
||||
class TestExtractionFilters:
|
||||
"""Test extraction filter security features."""
|
||||
|
||||
def test_default_filter_is_data(self, sample_files, temp_dir):
|
||||
"""Test that the default filter is 'data' for security."""
|
||||
archive_path = temp_dir / "test_security.tzst"
|
||||
file_paths = [f for f in sample_files if f.is_file()]
|
||||
|
||||
for invalid_level in [0, 23, -1, 100]:
|
||||
archive_path = temp_dir / f"invalid_{invalid_level}.tzst"
|
||||
with pytest.raises(ValueError) as exc_info:
|
||||
create_archive(
|
||||
archive_path, file_paths, compression_level=invalid_level
|
||||
)
|
||||
# Create archive
|
||||
with TzstArchive(archive_path, "w") as archive:
|
||||
for file_path in file_paths:
|
||||
relative_path = file_path.relative_to(sample_files[0].parent)
|
||||
archive.add(file_path, arcname=str(relative_path))
|
||||
|
||||
assert "compression level" in str(exc_info.value).lower()
|
||||
assert "1" in str(exc_info.value) and "22" in str(exc_info.value)
|
||||
# Test that default filter is 'data'
|
||||
extract_dir = temp_dir / "extracted_default"
|
||||
|
||||
with patch("tarfile.TarFile.extractall") as mock_extractall:
|
||||
with TzstArchive(archive_path, "r") as archive:
|
||||
archive.extract(path=extract_dir)
|
||||
|
||||
# Verify that 'data' filter was used
|
||||
mock_extractall.assert_called_once()
|
||||
call_args = mock_extractall.call_args
|
||||
assert "filter" in call_args[1]
|
||||
assert call_args[1]["filter"] == "data"
|
||||
|
||||
def test_data_filter_explicit(self, sample_files, temp_dir):
|
||||
"""Test explicitly setting 'data' filter."""
|
||||
archive_path = temp_dir / "test_data_filter.tzst"
|
||||
file_paths = [f for f in sample_files if f.is_file()]
|
||||
|
||||
# Create archive
|
||||
with TzstArchive(archive_path, "w") as archive:
|
||||
for file_path in file_paths:
|
||||
relative_path = file_path.relative_to(sample_files[0].parent)
|
||||
archive.add(file_path, arcname=str(relative_path))
|
||||
|
||||
# Test extraction with explicit 'data' filter
|
||||
extract_dir = temp_dir / "extracted_data"
|
||||
|
||||
with patch("tarfile.TarFile.extractall") as mock_extractall:
|
||||
with TzstArchive(archive_path, "r") as archive:
|
||||
archive.extract(path=extract_dir, filter="data")
|
||||
|
||||
mock_extractall.assert_called_once()
|
||||
call_args = mock_extractall.call_args
|
||||
assert call_args[1]["filter"] == "data"
|
||||
|
||||
def test_tar_filter(self, sample_files, temp_dir):
|
||||
"""Test 'tar' filter for Unix-like features."""
|
||||
archive_path = temp_dir / "test_tar_filter.tzst"
|
||||
file_paths = [f for f in sample_files if f.is_file()]
|
||||
|
||||
# Create archive
|
||||
with TzstArchive(archive_path, "w") as archive:
|
||||
for file_path in file_paths:
|
||||
relative_path = file_path.relative_to(sample_files[0].parent)
|
||||
archive.add(file_path, arcname=str(relative_path))
|
||||
|
||||
# Test extraction with 'tar' filter
|
||||
extract_dir = temp_dir / "extracted_tar"
|
||||
|
||||
with patch("tarfile.TarFile.extractall") as mock_extractall:
|
||||
with TzstArchive(archive_path, "r") as archive:
|
||||
archive.extract(path=extract_dir, filter="tar")
|
||||
|
||||
mock_extractall.assert_called_once()
|
||||
call_args = mock_extractall.call_args
|
||||
assert call_args[1]["filter"] == "tar"
|
||||
|
||||
def test_fully_trusted_filter(self, sample_files, temp_dir):
|
||||
"""Test 'fully_trusted' filter (dangerous but complete)."""
|
||||
archive_path = temp_dir / "test_trusted_filter.tzst"
|
||||
file_paths = [f for f in sample_files if f.is_file()]
|
||||
|
||||
# Create archive
|
||||
with TzstArchive(archive_path, "w") as archive:
|
||||
for file_path in file_paths:
|
||||
relative_path = file_path.relative_to(sample_files[0].parent)
|
||||
archive.add(file_path, arcname=str(relative_path))
|
||||
|
||||
# Test extraction with 'fully_trusted' filter
|
||||
extract_dir = temp_dir / "extracted_trusted"
|
||||
|
||||
with patch("tarfile.TarFile.extractall") as mock_extractall:
|
||||
with TzstArchive(archive_path, "r") as archive:
|
||||
archive.extract(path=extract_dir, filter="fully_trusted")
|
||||
|
||||
mock_extractall.assert_called_once()
|
||||
call_args = mock_extractall.call_args
|
||||
assert call_args[1]["filter"] == "fully_trusted"
|
||||
|
||||
def test_convenience_function_filter(self, sample_files, temp_dir):
|
||||
"""Test filter parameter in extract_archive convenience function."""
|
||||
archive_path = temp_dir / "test_convenience_filter.tzst"
|
||||
file_paths = [f for f in sample_files if f.is_file()]
|
||||
|
||||
# Create archive
|
||||
with TzstArchive(archive_path, "w") as archive:
|
||||
for file_path in file_paths:
|
||||
relative_path = file_path.relative_to(sample_files[0].parent)
|
||||
archive.add(file_path, arcname=str(relative_path))
|
||||
|
||||
# Test extract_archive with different filters
|
||||
for filter_type in ["data", "tar", "fully_trusted"]:
|
||||
extract_dir = temp_dir / f"extracted_conv_{filter_type}"
|
||||
|
||||
# This should not raise an exception
|
||||
extract_archive(archive_path, extract_dir, filter=filter_type)
|
||||
|
||||
# Verify files were extracted
|
||||
assert extract_dir.exists()
|
||||
extracted_files = list(extract_dir.rglob("*"))
|
||||
assert len([f for f in extracted_files if f.is_file()]) > 0
|
||||
|
||||
|
||||
class TestSecurityDocumentation:
|
||||
"""Test security documentation and warnings."""
|
||||
|
||||
def test_security_filter_documentation(self, sample_files, temp_dir):
|
||||
"""Test that security filters are properly documented."""
|
||||
# This test verifies that the API provides proper guidance
|
||||
archive_path = temp_dir / "test_docs.tzst"
|
||||
file_paths = [f for f in sample_files if f.is_file()]
|
||||
|
||||
# Create archive
|
||||
with TzstArchive(archive_path, "w") as archive:
|
||||
for file_path in file_paths:
|
||||
relative_path = file_path.relative_to(sample_files[0].parent)
|
||||
archive.add(file_path, arcname=str(relative_path))
|
||||
|
||||
# Test that TzstArchive.extract method accepts filter parameter
|
||||
with TzstArchive(archive_path, "r") as archive:
|
||||
# Should accept various filter types without error
|
||||
extract_dir = temp_dir / "doc_test"
|
||||
archive.extract(path=extract_dir, filter="data")
|
||||
|
||||
|
||||
class TestSecurityEdgeCases:
|
||||
"""Test security edge cases and boundary conditions."""
|
||||
|
||||
def test_filter_with_empty_archive(self, temp_dir):
|
||||
"""Test filter behavior with empty archives."""
|
||||
archive_path = temp_dir / "empty_security.tzst"
|
||||
|
||||
# Create empty archive
|
||||
with TzstArchive(archive_path, "w") as archive:
|
||||
pass # Empty archive
|
||||
|
||||
# Test extraction with filter on empty archive
|
||||
extract_dir = temp_dir / "empty_extracted"
|
||||
with TzstArchive(archive_path, "r") as archive:
|
||||
archive.extract(path=extract_dir, filter="data")
|
||||
|
||||
# Should succeed without error
|
||||
assert extract_dir.exists()
|
||||
|
||||
def test_filter_parameter_validation(self, sample_files, temp_dir):
|
||||
"""Test that invalid filter parameters are handled."""
|
||||
archive_path = temp_dir / "validation_test.tzst"
|
||||
file_paths = [f for f in sample_files if f.is_file()]
|
||||
|
||||
# Create archive
|
||||
with TzstArchive(archive_path, "w") as archive:
|
||||
for file_path in file_paths:
|
||||
relative_path = file_path.relative_to(sample_files[0].parent)
|
||||
archive.add(file_path, arcname=str(relative_path))
|
||||
|
||||
# Test invalid filter string
|
||||
extract_dir = temp_dir / "invalid_filter"
|
||||
with TzstArchive(archive_path, "r") as archive:
|
||||
# Invalid filter should be handled gracefully
|
||||
# (exact behavior depends on implementation)
|
||||
try:
|
||||
archive.extract(path=extract_dir, filter="invalid_filter_name")
|
||||
except (ValueError, TypeError):
|
||||
pass # Expected behavior for invalid filter
|
||||
|
||||
|
||||
class TestCurrentDirectoryEdgeCases:
|
||||
"""Test edge cases for current directory archiving."""
|
||||
|
||||
def test_empty_current_directory(self, temp_dir):
|
||||
"""Test archiving empty current directory."""
|
||||
empty_dir = temp_dir / "empty_work"
|
||||
empty_dir.mkdir()
|
||||
|
||||
original_cwd = os.getcwd()
|
||||
try:
|
||||
os.chdir(empty_dir)
|
||||
archive_path = empty_dir / "empty.tzst"
|
||||
|
||||
create_archive(archive_path, ["."])
|
||||
|
||||
contents = list_archive(archive_path)
|
||||
content_names = [item["name"] for item in contents]
|
||||
|
||||
# Should only contain the archive file exclusion, so be empty
|
||||
# (archive file itself should be excluded)
|
||||
assert len(content_names) == 0
|
||||
|
||||
finally:
|
||||
os.chdir(original_cwd)
|
||||
|
||||
def test_current_directory_with_subdirectories(self, temp_dir):
|
||||
"""Test current directory archiving with nested subdirectories."""
|
||||
work_dir = temp_dir / "nested_work"
|
||||
work_dir.mkdir()
|
||||
|
||||
# Create nested structure
|
||||
level1 = work_dir / "level1"
|
||||
level2 = level1 / "level2"
|
||||
level3 = level2 / "level3"
|
||||
|
||||
level1.mkdir()
|
||||
level2.mkdir()
|
||||
level3.mkdir()
|
||||
|
||||
# Create files at various levels
|
||||
(work_dir / "root.txt").write_text("Root level")
|
||||
(level1 / "l1.txt").write_text("Level 1")
|
||||
(level2 / "l2.txt").write_text("Level 2")
|
||||
(level3 / "l3.txt").write_text("Level 3")
|
||||
|
||||
original_cwd = os.getcwd()
|
||||
try:
|
||||
os.chdir(work_dir)
|
||||
archive_path = work_dir / "nested.tzst"
|
||||
|
||||
create_archive(archive_path, ["."])
|
||||
|
||||
contents = list_archive(archive_path)
|
||||
content_names = [item["name"] for item in contents]
|
||||
|
||||
# Should preserve directory structure without wrapper
|
||||
assert "root.txt" in content_names
|
||||
assert "level1/l1.txt" in content_names
|
||||
assert "level1/level2/l2.txt" in content_names
|
||||
assert "level1/level2/level3/l3.txt" in content_names
|
||||
|
||||
# Extract and verify structure
|
||||
extract_dir = temp_dir / "nested_extracted"
|
||||
extract_archive(archive_path, extract_dir)
|
||||
|
||||
assert (extract_dir / "root.txt").exists()
|
||||
assert (extract_dir / "level1" / "l1.txt").exists()
|
||||
assert (extract_dir / "level1" / "level2" / "l2.txt").exists()
|
||||
assert (extract_dir / "level1" / "level2" / "level3" / "l3.txt").exists()
|
||||
|
||||
finally:
|
||||
os.chdir(original_cwd)
|
||||
|
||||
def test_atomic_vs_non_atomic_current_directory(self, temp_dir):
|
||||
"""Test that atomic and non-atomic modes work the same for current directory."""
|
||||
work_dir = temp_dir / "atomic_test"
|
||||
work_dir.mkdir()
|
||||
|
||||
# Create test files
|
||||
(work_dir / "test1.txt").write_text("Test content 1")
|
||||
(work_dir / "test2.txt").write_text("Test content 2")
|
||||
|
||||
original_cwd = os.getcwd()
|
||||
try:
|
||||
os.chdir(work_dir)
|
||||
|
||||
# Test atomic mode
|
||||
atomic_archive = work_dir / "atomic.tzst"
|
||||
create_archive(atomic_archive, ["."], use_temp_file=True)
|
||||
atomic_contents = list_archive(atomic_archive)
|
||||
atomic_names = [item["name"] for item in atomic_contents]
|
||||
|
||||
# Test non-atomic mode
|
||||
non_atomic_archive = work_dir / "non_atomic.tzst"
|
||||
create_archive(non_atomic_archive, ["."], use_temp_file=False)
|
||||
non_atomic_contents = list_archive(non_atomic_archive)
|
||||
non_atomic_names = [item["name"] for item in non_atomic_contents]
|
||||
|
||||
# Both should have the same contents
|
||||
assert set(atomic_names) == set(non_atomic_names)
|
||||
assert "test1.txt" in atomic_names
|
||||
assert "test2.txt" in atomic_names
|
||||
|
||||
finally:
|
||||
os.chdir(original_cwd)
|
||||
@@ -0,0 +1,395 @@
|
||||
"""Tests to cover missing lines in core.py for improved coverage."""
|
||||
|
||||
import tarfile
|
||||
from unittest.mock import MagicMock, patch
|
||||
|
||||
import pytest
|
||||
|
||||
from tzst.core import TzstArchive
|
||||
from tzst.exceptions import TzstArchiveError, TzstDecompressionError
|
||||
|
||||
|
||||
class TestCoreMissingLines:
|
||||
"""Test specific missing lines in core.py."""
|
||||
|
||||
def test_append_mode_error_handling(self, temp_dir):
|
||||
"""Test append mode error handling (lines 126-137)."""
|
||||
archive_path = temp_dir / "test.tzst"
|
||||
|
||||
# Test append mode raises NotImplementedError
|
||||
with pytest.raises(
|
||||
NotImplementedError, match="Append mode is not currently supported"
|
||||
):
|
||||
TzstArchive(archive_path, mode="a")
|
||||
|
||||
def test_invalid_mode_error_after_open(self, temp_dir):
|
||||
"""Test invalid mode error in __enter__ method (line 137)."""
|
||||
archive_path = temp_dir / "test.tzst"
|
||||
|
||||
# Create archive instance with invalid mode after validation passes
|
||||
archive = TzstArchive.__new__(TzstArchive)
|
||||
archive.filename = archive_path
|
||||
archive.mode = "invalid" # Set invalid mode after construction
|
||||
archive.compression_level = 3
|
||||
archive.streaming = False
|
||||
archive._tarfile = None
|
||||
archive._fileobj = None
|
||||
archive._compressed_stream = None
|
||||
|
||||
with pytest.raises(TzstArchiveError, match="Failed to open archive"):
|
||||
archive.__enter__()
|
||||
|
||||
def test_zstd_error_handling_in_open(self, temp_dir):
|
||||
"""Test zstd error handling during archive opening (lines 133-137)."""
|
||||
archive_path = temp_dir / "test.tzst"
|
||||
|
||||
# Create a file that will cause zstd decompression error
|
||||
archive_path.write_bytes(b"invalid zstd data")
|
||||
|
||||
# Try to open as read mode - should raise TzstDecompressionError
|
||||
with pytest.raises(TzstDecompressionError, match="Failed to open archive"):
|
||||
with TzstArchive(archive_path, mode="r"):
|
||||
pass
|
||||
|
||||
def test_generic_error_handling_in_open(self, temp_dir):
|
||||
"""Test generic error handling during archive opening."""
|
||||
archive_path = temp_dir / "test.tzst"
|
||||
|
||||
# Mock to raise a generic exception (not zstd-related)
|
||||
with patch("builtins.open", side_effect=PermissionError("Permission denied")):
|
||||
with pytest.raises(TzstArchiveError, match="Failed to open archive"):
|
||||
with TzstArchive(archive_path, mode="r"):
|
||||
pass
|
||||
|
||||
def test_close_error_handling(self, temp_dir):
|
||||
"""Test error handling in close method (lines 146-157)."""
|
||||
archive_path = temp_dir / "test.tzst"
|
||||
|
||||
# Create archive and manually set objects that will raise on close
|
||||
with TzstArchive(archive_path, mode="w") as archive:
|
||||
pass
|
||||
|
||||
# Now manually create problematic objects
|
||||
archive = TzstArchive.__new__(TzstArchive)
|
||||
archive._tarfile = MagicMock()
|
||||
archive._tarfile.close.side_effect = Exception("Close error")
|
||||
archive._compressed_stream = MagicMock()
|
||||
archive._compressed_stream.close.side_effect = Exception("Close error")
|
||||
archive._fileobj = MagicMock()
|
||||
archive._fileobj.close.side_effect = Exception("Close error")
|
||||
|
||||
# close() should handle exceptions gracefully
|
||||
archive.close() # Should not raise
|
||||
|
||||
assert archive._tarfile is None
|
||||
assert archive._compressed_stream is None
|
||||
assert archive._fileobj is None
|
||||
|
||||
def test_archive_not_open_for_reading_errors(self, temp_dir):
|
||||
"""Test RuntimeError for operations on archives not open for reading (lines 186, 188, 192)."""
|
||||
archive_path = temp_dir / "test.tzst"
|
||||
|
||||
# Create archive in write mode
|
||||
with TzstArchive(archive_path, mode="w") as archive:
|
||||
# Test getmembers() on write mode
|
||||
with pytest.raises(RuntimeError, match="Archive not open for reading"):
|
||||
archive.getmembers()
|
||||
|
||||
# Test getnames() on write mode
|
||||
with pytest.raises(RuntimeError, match="Archive not open for reading"):
|
||||
archive.getnames()
|
||||
|
||||
# Test extractfile() on write mode
|
||||
with pytest.raises(RuntimeError, match="Archive not open for reading"):
|
||||
archive.extractfile("test")
|
||||
|
||||
def test_streaming_member_extraction_error(self, temp_dir):
|
||||
"""Test streaming mode member extraction error (lines 242, 249-254)."""
|
||||
# Create a test archive first
|
||||
test_file = temp_dir / "test.txt"
|
||||
test_file.write_text("test content")
|
||||
|
||||
archive_path = temp_dir / "test.tzst"
|
||||
with TzstArchive(archive_path, mode="w") as archive:
|
||||
archive.add(str(test_file), arcname="test.txt")
|
||||
|
||||
# Try to extract specific member in streaming mode
|
||||
with TzstArchive(archive_path, mode="r", streaming=True) as archive:
|
||||
members = archive.getmembers()
|
||||
member = members[0]
|
||||
|
||||
extract_dir = temp_dir / "extract"
|
||||
extract_dir.mkdir() # Should raise RuntimeError for specific member extraction in streaming mode
|
||||
with pytest.raises(
|
||||
RuntimeError,
|
||||
match="Extracting specific members is not supported in streaming mode",
|
||||
):
|
||||
archive.extract(member=member.name, path=extract_dir)
|
||||
|
||||
def test_streaming_extraction_failure_handling(self, temp_dir):
|
||||
"""Test streaming extraction failure handling (lines 263-272)."""
|
||||
# Create archive first
|
||||
test_file = temp_dir / "test.txt"
|
||||
test_file.write_text("test content")
|
||||
|
||||
archive_path = temp_dir / "test.tzst"
|
||||
with TzstArchive(archive_path, mode="w") as archive:
|
||||
archive.add(
|
||||
str(test_file), arcname="test.txt"
|
||||
) # Mock tarfile to raise StreamError
|
||||
with TzstArchive(archive_path, mode="r", streaming=True) as archive:
|
||||
extract_dir = temp_dir / "extract"
|
||||
extract_dir.mkdir()
|
||||
|
||||
# Mock extractall to raise StreamError with streaming-related message
|
||||
with patch.object(
|
||||
archive._tarfile,
|
||||
"extractall",
|
||||
side_effect=tarfile.StreamError("seeking not supported"),
|
||||
):
|
||||
with pytest.raises(
|
||||
RuntimeError, match="Extraction failed in streaming mode"
|
||||
):
|
||||
archive.extract(path=extract_dir)
|
||||
|
||||
def test_extractfile_not_open_error(self, temp_dir):
|
||||
"""Test extractfile when archive is not open (line 307)."""
|
||||
archive_path = temp_dir / "test.tzst"
|
||||
|
||||
# Create closed archive
|
||||
archive = TzstArchive(archive_path, mode="r")
|
||||
# Don't open it
|
||||
|
||||
with pytest.raises(RuntimeError, match="Archive not open"):
|
||||
archive.extractfile("test")
|
||||
|
||||
def test_extractfile_write_mode_error(self, temp_dir):
|
||||
"""Test extractfile in write mode (already covered but ensuring line coverage)."""
|
||||
archive_path = temp_dir / "test.tzst"
|
||||
|
||||
with TzstArchive(archive_path, mode="w") as archive:
|
||||
with pytest.raises(RuntimeError, match="Archive not open for reading"):
|
||||
archive.extractfile("test")
|
||||
|
||||
def test_add_method_not_open_error(self, temp_dir):
|
||||
"""Test add method when archive is not open (line 325, 327)."""
|
||||
archive_path = temp_dir / "test.tzst"
|
||||
test_file = temp_dir / "test.txt"
|
||||
test_file.write_text("test content")
|
||||
|
||||
# Create archive but don't open it
|
||||
archive = TzstArchive(archive_path, mode="w")
|
||||
|
||||
with pytest.raises(RuntimeError, match="Archive not open"):
|
||||
archive.add(str(test_file))
|
||||
|
||||
def test_add_method_read_mode_error(self, temp_dir):
|
||||
"""Test add method in read mode (line 327)."""
|
||||
# Create archive first
|
||||
test_file = temp_dir / "test.txt"
|
||||
test_file.write_text("test content")
|
||||
|
||||
archive_path = temp_dir / "test.tzst"
|
||||
with TzstArchive(archive_path, mode="w") as archive:
|
||||
archive.add(str(test_file), arcname="test.txt")
|
||||
|
||||
# Try to add to archive in read mode
|
||||
with TzstArchive(archive_path, mode="r") as archive:
|
||||
with pytest.raises(RuntimeError, match="Archive not open for writing"):
|
||||
archive.add(str(test_file))
|
||||
|
||||
def test_file_not_found_in_add(self, temp_dir):
|
||||
"""Test file not found error in add method (line 373)."""
|
||||
archive_path = temp_dir / "test.tzst"
|
||||
missing_file = temp_dir / "missing.txt"
|
||||
|
||||
with TzstArchive(archive_path, mode="w") as archive:
|
||||
with pytest.raises(FileNotFoundError):
|
||||
archive.add(str(missing_file))
|
||||
|
||||
def test_add_method_generic_error_handling(self, temp_dir):
|
||||
"""Test generic error handling in add method (line 375)."""
|
||||
archive_path = temp_dir / "test.tzst"
|
||||
test_file = temp_dir / "test.txt"
|
||||
test_file.write_text("test content")
|
||||
|
||||
with TzstArchive(archive_path, mode="w") as archive:
|
||||
# Mock add to raise generic exception
|
||||
with patch.object(
|
||||
archive._tarfile,
|
||||
"add",
|
||||
side_effect=PermissionError("Permission denied"),
|
||||
):
|
||||
with pytest.raises(TzstArchiveError, match="Failed to add"):
|
||||
archive.add(str(test_file))
|
||||
|
||||
def test_test_method_not_open_error(self, temp_dir):
|
||||
"""Test test method when archive is not open (line 390-391)."""
|
||||
archive_path = temp_dir / "test.tzst"
|
||||
|
||||
# Create archive but don't open it
|
||||
archive = TzstArchive(archive_path, mode="r")
|
||||
|
||||
with pytest.raises(RuntimeError, match="Archive not open"):
|
||||
archive.test()
|
||||
|
||||
def test_test_method_write_mode_error(self, temp_dir):
|
||||
"""Test test method in write mode (line 391)."""
|
||||
archive_path = temp_dir / "test.tzst"
|
||||
|
||||
with TzstArchive(archive_path, mode="w") as archive:
|
||||
with pytest.raises(RuntimeError, match="Archive not open for reading"):
|
||||
archive.test()
|
||||
|
||||
def test_test_method_streaming_mode_info(self, temp_dir):
|
||||
"""Test test method streaming mode information (line 427)."""
|
||||
# Create archive first
|
||||
test_file = temp_dir / "test.txt"
|
||||
test_file.write_text("test content")
|
||||
|
||||
archive_path = temp_dir / "test.tzst"
|
||||
with TzstArchive(archive_path, mode="w") as archive:
|
||||
archive.add(str(test_file), arcname="test.txt")
|
||||
|
||||
# Test in streaming mode - should provide different behavior info
|
||||
with TzstArchive(archive_path, mode="r", streaming=True) as archive:
|
||||
# This should work but may have streaming-specific behavior
|
||||
result = archive.test()
|
||||
assert isinstance(result, bool)
|
||||
|
||||
def test_list_method_not_open_error(self, temp_dir):
|
||||
"""Test list method when archive is not open (line 454-455)."""
|
||||
archive_path = temp_dir / "test.tzst"
|
||||
|
||||
# Create archive but don't open it
|
||||
archive = TzstArchive(archive_path, mode="r")
|
||||
|
||||
with pytest.raises(RuntimeError, match="Archive not open"):
|
||||
list(archive.list())
|
||||
|
||||
def test_list_method_write_mode_error(self, temp_dir):
|
||||
"""Test list method in write mode (line 455)."""
|
||||
archive_path = temp_dir / "test.tzst"
|
||||
|
||||
with TzstArchive(archive_path, mode="w") as archive:
|
||||
with pytest.raises(RuntimeError, match="Archive not open for reading"):
|
||||
list(archive.list())
|
||||
|
||||
def test_context_manager_exception_handling(self, temp_dir):
|
||||
"""Test context manager exception handling (lines 502-504)."""
|
||||
archive_path = temp_dir / "test.tzst"
|
||||
|
||||
# Store references for cleanup
|
||||
fileobj = None
|
||||
compressed_stream = None
|
||||
tarfile_obj = None
|
||||
|
||||
# Test that close exceptions are suppressed during context manager exit
|
||||
with patch("tzst.core.TzstArchive.close", side_effect=Exception("Close error")):
|
||||
try:
|
||||
with TzstArchive(archive_path, mode="w") as archive:
|
||||
# Store references to underlying objects for manual cleanup
|
||||
fileobj = archive._fileobj
|
||||
compressed_stream = archive._compressed_stream
|
||||
tarfile_obj = archive._tarfile
|
||||
raise ValueError("Test exception")
|
||||
except ValueError:
|
||||
pass # Expected - the original exception should not be masked
|
||||
finally:
|
||||
# Manually clean up since mocked close() failed
|
||||
try:
|
||||
if tarfile_obj:
|
||||
tarfile_obj.close()
|
||||
except Exception:
|
||||
pass
|
||||
try:
|
||||
if compressed_stream:
|
||||
compressed_stream.close()
|
||||
except Exception:
|
||||
pass
|
||||
try:
|
||||
if fileobj:
|
||||
fileobj.close()
|
||||
except Exception:
|
||||
pass
|
||||
# Ensure the file is removed to prevent permission errors
|
||||
try:
|
||||
if archive_path.exists():
|
||||
archive_path.unlink()
|
||||
except (PermissionError, OSError):
|
||||
pass
|
||||
|
||||
# The close exception should be suppressed by __exit__
|
||||
|
||||
def test_streaming_mode_directory_creation_error(self, temp_dir):
|
||||
"""Test directory creation error in streaming mode (line 521)."""
|
||||
# Create archive first
|
||||
test_file = temp_dir / "test.txt"
|
||||
test_file.write_text("test content")
|
||||
|
||||
archive_path = temp_dir / "test.tzst"
|
||||
with TzstArchive(archive_path, mode="w") as archive:
|
||||
archive.add(str(test_file), arcname="test.txt")
|
||||
|
||||
with TzstArchive(archive_path, mode="r", streaming=True) as archive:
|
||||
# Mock path creation to fail
|
||||
extract_dir = temp_dir / "extract"
|
||||
|
||||
with patch("pathlib.Path.mkdir", side_effect=OSError("Permission denied")):
|
||||
with pytest.raises(OSError):
|
||||
archive.extractall(path=extract_dir)
|
||||
|
||||
def test_list_verbose_mode_edge_cases(self, temp_dir):
|
||||
"""Test list method verbose mode edge cases (lines 573, 588-589)."""
|
||||
# Create archive with special files
|
||||
test_file = temp_dir / "test.txt"
|
||||
test_file.write_text("test content")
|
||||
|
||||
# Create a directory
|
||||
test_dir = temp_dir / "test_dir"
|
||||
test_dir.mkdir()
|
||||
|
||||
archive_path = temp_dir / "test.tzst"
|
||||
with TzstArchive(archive_path, mode="w") as archive:
|
||||
archive.add(str(test_file), arcname="test.txt")
|
||||
archive.add(str(test_dir), arcname="test_dir")
|
||||
|
||||
with TzstArchive(archive_path, mode="r") as archive:
|
||||
# Test verbose listing
|
||||
items = list(archive.list(verbose=True))
|
||||
assert len(items) >= 2
|
||||
|
||||
# Should have both file and directory entries
|
||||
file_items = [item for item in items if item.get("is_file", False)]
|
||||
dir_items = [item for item in items if item.get("is_dir", False)]
|
||||
|
||||
assert len(file_items) >= 1
|
||||
assert len(dir_items) >= 1
|
||||
|
||||
def test_extractall_with_members_parameter(self, temp_dir):
|
||||
"""Test extractall with members parameter for selective extraction."""
|
||||
# Create archive with multiple files
|
||||
test_file1 = temp_dir / "test1.txt"
|
||||
test_file1.write_text("content1")
|
||||
test_file2 = temp_dir / "test2.txt"
|
||||
test_file2.write_text("content2")
|
||||
|
||||
archive_path = temp_dir / "test.tzst"
|
||||
with TzstArchive(archive_path, mode="w") as archive:
|
||||
archive.add(str(test_file1), arcname="test1.txt")
|
||||
archive.add(str(test_file2), arcname="test2.txt")
|
||||
|
||||
# Extract only specific members
|
||||
with TzstArchive(archive_path, mode="r") as archive:
|
||||
members = archive.getmembers()
|
||||
first_member = members[0]
|
||||
|
||||
extract_dir = temp_dir / "extract"
|
||||
extract_dir.mkdir()
|
||||
|
||||
# Extract only first member
|
||||
archive.extractall(path=extract_dir, members=[first_member])
|
||||
|
||||
# Verify only one file was extracted
|
||||
extracted_files = list(extract_dir.glob("*.txt"))
|
||||
assert len(extracted_files) == 1
|
||||
@@ -0,0 +1,374 @@
|
||||
"""Tests to cover missing lines in core.py for improved coverage."""
|
||||
|
||||
import tarfile
|
||||
from unittest.mock import MagicMock, patch
|
||||
|
||||
import pytest
|
||||
|
||||
from tzst.core import TzstArchive
|
||||
from tzst.exceptions import TzstArchiveError, TzstDecompressionError
|
||||
|
||||
|
||||
class TestCoreMissingLines:
|
||||
"""Test specific missing lines in core.py."""
|
||||
|
||||
def test_append_mode_error_handling(self, temp_dir):
|
||||
"""Test append mode error handling (lines 126-137)."""
|
||||
archive_path = temp_dir / "test.tzst"
|
||||
|
||||
# Test append mode raises NotImplementedError
|
||||
with pytest.raises(
|
||||
NotImplementedError, match="Append mode is not currently supported"
|
||||
):
|
||||
TzstArchive(archive_path, mode="a")
|
||||
|
||||
def test_invalid_mode_error_after_open(self, temp_dir):
|
||||
"""Test invalid mode error in __enter__ method (line 137)."""
|
||||
archive_path = temp_dir / "test.tzst"
|
||||
|
||||
# Create archive instance with invalid mode after validation passes
|
||||
archive = TzstArchive.__new__(TzstArchive)
|
||||
archive.filename = archive_path
|
||||
archive.mode = "invalid" # Set invalid mode after construction
|
||||
archive.compression_level = 3
|
||||
archive.streaming = False
|
||||
archive._tarfile = None
|
||||
archive._fileobj = None
|
||||
archive._compressed_stream = None
|
||||
|
||||
with pytest.raises(TzstArchiveError, match="Failed to open archive"):
|
||||
archive.__enter__()
|
||||
|
||||
def test_zstd_error_handling_in_open(self, temp_dir):
|
||||
"""Test zstd error handling during archive opening (lines 133-137)."""
|
||||
archive_path = temp_dir / "test.tzst"
|
||||
|
||||
# Create a file that will cause zstd decompression error
|
||||
archive_path.write_bytes(b"invalid zstd data")
|
||||
|
||||
# Try to open as read mode - should raise TzstDecompressionError
|
||||
with pytest.raises(TzstDecompressionError, match="Failed to open archive"):
|
||||
with TzstArchive(archive_path, mode="r"):
|
||||
pass
|
||||
|
||||
def test_generic_error_handling_in_open(self, temp_dir):
|
||||
"""Test generic error handling during archive opening."""
|
||||
archive_path = temp_dir / "test.tzst"
|
||||
|
||||
# Mock to raise a generic exception (not zstd-related)
|
||||
with patch("builtins.open", side_effect=PermissionError("Permission denied")):
|
||||
with pytest.raises(TzstArchiveError, match="Failed to open archive"):
|
||||
with TzstArchive(archive_path, mode="r"):
|
||||
pass
|
||||
|
||||
def test_close_error_handling(self, temp_dir):
|
||||
"""Test error handling in close method (lines 146-157)."""
|
||||
archive_path = temp_dir / "test.tzst"
|
||||
|
||||
# Create archive and manually set objects that will raise on close
|
||||
with TzstArchive(archive_path, mode="w") as archive:
|
||||
pass
|
||||
|
||||
# Now manually create problematic objects
|
||||
archive = TzstArchive.__new__(TzstArchive)
|
||||
archive._tarfile = MagicMock()
|
||||
archive._tarfile.close.side_effect = Exception("Close error")
|
||||
archive._compressed_stream = MagicMock()
|
||||
archive._compressed_stream.close.side_effect = Exception("Close error")
|
||||
archive._fileobj = MagicMock()
|
||||
archive._fileobj.close.side_effect = Exception("Close error")
|
||||
|
||||
# close() should handle exceptions gracefully
|
||||
archive.close() # Should not raise
|
||||
|
||||
assert archive._tarfile is None
|
||||
assert archive._compressed_stream is None
|
||||
assert archive._fileobj is None
|
||||
|
||||
def test_archive_not_open_for_reading_errors(self, temp_dir):
|
||||
"""Test RuntimeError for operations on archives not open for reading."""
|
||||
archive_path = temp_dir / "test.tzst"
|
||||
|
||||
# Create archive in write mode
|
||||
with TzstArchive(archive_path, mode="w") as archive:
|
||||
# Test getmembers() on write mode
|
||||
with pytest.raises(RuntimeError, match="Archive not open for reading"):
|
||||
archive.getmembers()
|
||||
|
||||
# Test getnames() on write mode
|
||||
with pytest.raises(RuntimeError, match="Archive not open for reading"):
|
||||
archive.getnames()
|
||||
|
||||
# Test extractfile() on write mode
|
||||
with pytest.raises(RuntimeError, match="Archive not open for reading"):
|
||||
archive.extractfile("test")
|
||||
|
||||
def test_streaming_member_extraction_error(self, temp_dir):
|
||||
"""Test streaming mode member extraction error."""
|
||||
# Create a test archive first
|
||||
test_file = temp_dir / "test.txt"
|
||||
test_file.write_text("test content")
|
||||
archive_path = temp_dir / "test.tzst"
|
||||
|
||||
with TzstArchive(archive_path, mode="w") as archive:
|
||||
archive.add(str(test_file), arcname="test.txt")
|
||||
|
||||
# Try to extract specific member in streaming mode
|
||||
with TzstArchive(archive_path, mode="r", streaming=True) as archive:
|
||||
extract_dir = temp_dir / "extract"
|
||||
extract_dir.mkdir()
|
||||
|
||||
# Should raise RuntimeError for specific member extraction in streaming mode
|
||||
with pytest.raises(
|
||||
RuntimeError,
|
||||
match="Extracting specific members is not supported in streaming mode",
|
||||
):
|
||||
archive.extract(member="test.txt", path=extract_dir)
|
||||
|
||||
def test_streaming_extraction_failure_handling(self, temp_dir):
|
||||
"""Test streaming extraction failure handling."""
|
||||
# Create archive first
|
||||
test_file = temp_dir / "test.txt"
|
||||
test_file.write_text("test content")
|
||||
archive_path = temp_dir / "test.tzst"
|
||||
|
||||
with TzstArchive(archive_path, mode="w") as archive:
|
||||
archive.add(str(test_file), arcname="test.txt")
|
||||
|
||||
# Mock tarfile to raise StreamError
|
||||
with TzstArchive(archive_path, mode="r", streaming=True) as archive:
|
||||
extract_dir = temp_dir / "extract"
|
||||
extract_dir.mkdir()
|
||||
|
||||
# Mock extract to raise StreamError with streaming-related message
|
||||
with patch.object(
|
||||
archive._tarfile,
|
||||
"extractall",
|
||||
side_effect=tarfile.StreamError("seeking not supported"),
|
||||
):
|
||||
with pytest.raises(
|
||||
RuntimeError, match="Extraction failed in streaming mode"
|
||||
):
|
||||
archive.extract(path=extract_dir)
|
||||
|
||||
def test_extractfile_not_open_error(self, temp_dir):
|
||||
"""Test extractfile when archive is not open."""
|
||||
archive_path = temp_dir / "test.tzst"
|
||||
|
||||
# Create closed archive
|
||||
archive = TzstArchive(archive_path, mode="r")
|
||||
# Don't open it
|
||||
|
||||
with pytest.raises(RuntimeError, match="Archive not open"):
|
||||
archive.extractfile("test")
|
||||
|
||||
def test_extractfile_write_mode_error(self, temp_dir):
|
||||
"""Test extractfile in write mode."""
|
||||
archive_path = temp_dir / "test.tzst"
|
||||
|
||||
with TzstArchive(archive_path, mode="w") as archive:
|
||||
with pytest.raises(RuntimeError, match="Archive not open for reading"):
|
||||
archive.extractfile("test")
|
||||
|
||||
def test_add_method_not_open_error(self, temp_dir):
|
||||
"""Test add method when archive is not open."""
|
||||
archive_path = temp_dir / "test.tzst"
|
||||
test_file = temp_dir / "test.txt"
|
||||
test_file.write_text("test content")
|
||||
|
||||
# Create archive but don't open it
|
||||
archive = TzstArchive(archive_path, mode="w")
|
||||
|
||||
with pytest.raises(RuntimeError, match="Archive not open"):
|
||||
archive.add(str(test_file))
|
||||
|
||||
def test_add_method_read_mode_error(self, temp_dir):
|
||||
"""Test add method in read mode."""
|
||||
# Create archive first
|
||||
test_file = temp_dir / "test.txt"
|
||||
test_file.write_text("test content")
|
||||
archive_path = temp_dir / "test.tzst"
|
||||
|
||||
with TzstArchive(archive_path, mode="w") as archive:
|
||||
archive.add(str(test_file), arcname="test.txt")
|
||||
|
||||
# Try to add to archive in read mode
|
||||
with TzstArchive(archive_path, mode="r") as archive:
|
||||
with pytest.raises(RuntimeError, match="Archive not open for writing"):
|
||||
archive.add(str(test_file))
|
||||
|
||||
def test_file_not_found_in_add(self, temp_dir):
|
||||
"""Test file not found error in add method."""
|
||||
archive_path = temp_dir / "test.tzst"
|
||||
missing_file = temp_dir / "missing.txt"
|
||||
|
||||
with TzstArchive(archive_path, mode="w") as archive:
|
||||
with pytest.raises(FileNotFoundError):
|
||||
archive.add(str(missing_file))
|
||||
|
||||
def test_add_method_generic_error_handling(self, temp_dir):
|
||||
"""Test generic error handling in add method."""
|
||||
archive_path = temp_dir / "test.tzst"
|
||||
test_file = temp_dir / "test.txt"
|
||||
test_file.write_text("test content")
|
||||
|
||||
with TzstArchive(archive_path, mode="w") as archive:
|
||||
# Mock add to raise generic exception
|
||||
with patch.object(
|
||||
archive._tarfile,
|
||||
"add",
|
||||
side_effect=PermissionError("Permission denied"),
|
||||
):
|
||||
with pytest.raises(TzstArchiveError, match="Failed to add"):
|
||||
archive.add(str(test_file))
|
||||
|
||||
def test_test_method_not_open_error(self, temp_dir):
|
||||
"""Test test method when archive is not open."""
|
||||
archive_path = temp_dir / "test.tzst"
|
||||
|
||||
# Create archive but don't open it
|
||||
archive = TzstArchive(archive_path, mode="r")
|
||||
|
||||
with pytest.raises(RuntimeError, match="Archive not open"):
|
||||
archive.test()
|
||||
|
||||
def test_test_method_write_mode_error(self, temp_dir):
|
||||
"""Test test method in write mode."""
|
||||
archive_path = temp_dir / "test.tzst"
|
||||
|
||||
with TzstArchive(archive_path, mode="w") as archive:
|
||||
with pytest.raises(RuntimeError, match="Archive not open for reading"):
|
||||
archive.test()
|
||||
|
||||
def test_test_method_streaming_mode_info(self, temp_dir):
|
||||
"""Test test method streaming mode information."""
|
||||
# Create archive first
|
||||
test_file = temp_dir / "test.txt"
|
||||
test_file.write_text("test content")
|
||||
archive_path = temp_dir / "test.tzst"
|
||||
|
||||
with TzstArchive(archive_path, mode="w") as archive:
|
||||
archive.add(str(test_file), arcname="test.txt")
|
||||
|
||||
# Test in streaming mode - should provide different behavior info
|
||||
with TzstArchive(archive_path, mode="r", streaming=True) as archive:
|
||||
# This should work but may have streaming-specific behavior
|
||||
result = archive.test()
|
||||
assert isinstance(result, bool)
|
||||
|
||||
def test_list_method_not_open_error(self, temp_dir):
|
||||
"""Test list method when archive is not open."""
|
||||
archive_path = temp_dir / "test.tzst"
|
||||
|
||||
# Create archive but don't open it
|
||||
archive = TzstArchive(archive_path, mode="r")
|
||||
|
||||
with pytest.raises(RuntimeError, match="Archive not open"):
|
||||
list(archive.list())
|
||||
|
||||
def test_list_method_write_mode_error(self, temp_dir):
|
||||
"""Test list method in write mode."""
|
||||
archive_path = temp_dir / "test.tzst"
|
||||
|
||||
with TzstArchive(archive_path, mode="w") as archive:
|
||||
with pytest.raises(RuntimeError, match="Archive not open for reading"):
|
||||
list(archive.list())
|
||||
|
||||
def test_context_manager_exception_handling(self, temp_dir):
|
||||
"""Test context manager exception handling."""
|
||||
archive_path = temp_dir / "test.tzst"
|
||||
|
||||
# Store references for cleanup
|
||||
fileobj = None
|
||||
compressed_stream = None
|
||||
tarfile_obj = None
|
||||
|
||||
# Test that close exceptions are suppressed during context manager exit
|
||||
with patch("tzst.core.TzstArchive.close", side_effect=Exception("Close error")):
|
||||
try:
|
||||
with TzstArchive(archive_path, mode="w") as archive:
|
||||
# Store references to underlying objects for manual cleanup
|
||||
fileobj = archive._fileobj
|
||||
compressed_stream = archive._compressed_stream
|
||||
tarfile_obj = archive._tarfile
|
||||
raise ValueError("Test exception")
|
||||
except ValueError:
|
||||
pass # Expected - the original exception should not be masked
|
||||
finally:
|
||||
# Manually clean up since mocked close() failed
|
||||
try:
|
||||
if tarfile_obj:
|
||||
tarfile_obj.close()
|
||||
except Exception:
|
||||
pass
|
||||
try:
|
||||
if compressed_stream:
|
||||
compressed_stream.close()
|
||||
except Exception:
|
||||
pass
|
||||
try:
|
||||
if fileobj:
|
||||
fileobj.close()
|
||||
except Exception:
|
||||
pass
|
||||
# Ensure the file is removed to prevent permission errors
|
||||
try:
|
||||
if archive_path.exists():
|
||||
archive_path.unlink()
|
||||
except (PermissionError, OSError):
|
||||
pass
|
||||
|
||||
# The close exception should be suppressed by __exit__
|
||||
|
||||
def test_list_verbose_mode_edge_cases(self, temp_dir):
|
||||
"""Test list method verbose mode edge cases."""
|
||||
# Create archive with special files
|
||||
test_file = temp_dir / "test.txt"
|
||||
test_file.write_text("test content")
|
||||
|
||||
# Create a directory
|
||||
test_dir = temp_dir / "test_dir"
|
||||
test_dir.mkdir()
|
||||
|
||||
archive_path = temp_dir / "test.tzst"
|
||||
with TzstArchive(archive_path, mode="w") as archive:
|
||||
archive.add(str(test_file), arcname="test.txt")
|
||||
archive.add(str(test_dir), arcname="test_dir")
|
||||
|
||||
with TzstArchive(archive_path, mode="r") as archive:
|
||||
# Test verbose listing
|
||||
items = list(archive.list(verbose=True))
|
||||
assert len(items) >= 2
|
||||
|
||||
# Should have both file and directory entries
|
||||
file_items = [item for item in items if item.get("is_file", False)]
|
||||
dir_items = [item for item in items if item.get("is_dir", False)]
|
||||
|
||||
assert len(file_items) >= 1
|
||||
assert len(dir_items) >= 1
|
||||
|
||||
def test_extract_with_members_parameter(self, temp_dir):
|
||||
"""Test extract with specific member for selective extraction."""
|
||||
# Create archive with multiple files
|
||||
test_file1 = temp_dir / "test1.txt"
|
||||
test_file1.write_text("content1")
|
||||
test_file2 = temp_dir / "test2.txt"
|
||||
test_file2.write_text("content2")
|
||||
|
||||
archive_path = temp_dir / "test.tzst"
|
||||
with TzstArchive(archive_path, mode="w") as archive:
|
||||
archive.add(str(test_file1), arcname="test1.txt")
|
||||
archive.add(str(test_file2), arcname="test2.txt")
|
||||
|
||||
# Extract only specific member
|
||||
with TzstArchive(archive_path, mode="r") as archive:
|
||||
extract_dir = temp_dir / "extract"
|
||||
extract_dir.mkdir()
|
||||
|
||||
# Extract only first member
|
||||
archive.extract(member="test1.txt", path=extract_dir)
|
||||
|
||||
# Verify only one file was extracted
|
||||
extracted_files = list(extract_dir.glob("*.txt"))
|
||||
assert len(extracted_files) == 1
|
||||
assert extracted_files[0].name == "test1.txt"
|
||||
@@ -0,0 +1,385 @@
|
||||
"""Platform-specific and cross-platform compatibility tests."""
|
||||
|
||||
import sys
|
||||
|
||||
import pytest
|
||||
|
||||
from tzst.cli import main
|
||||
|
||||
|
||||
@pytest.mark.skipif(sys.platform == "win32", reason="Unix-specific features")
|
||||
class TestUnixSpecificFeatures:
|
||||
"""Test Unix/Linux specific features like symlinks and permissions."""
|
||||
|
||||
def test_symbolic_links_preservation(self, temp_dir):
|
||||
"""Test that symbolic links are preserved in archives."""
|
||||
# Create target file and symlink
|
||||
target_file = temp_dir / "target.txt"
|
||||
target_file.write_text("Symlink target content")
|
||||
|
||||
symlink_file = temp_dir / "symlink.txt"
|
||||
try:
|
||||
symlink_file.symlink_to(target_file)
|
||||
except OSError:
|
||||
pytest.skip("Symlinks not supported on this system")
|
||||
|
||||
# Create archive with symlink
|
||||
archive_path = temp_dir / "symlinks.tzst"
|
||||
result = main(["a", str(archive_path), str(target_file), str(symlink_file)])
|
||||
assert result == 0
|
||||
|
||||
# Extract and verify symlink is preserved
|
||||
extract_dir = temp_dir / "symlink_extracted"
|
||||
result = main(["x", str(archive_path), "-o", str(extract_dir)])
|
||||
assert result == 0
|
||||
|
||||
extracted_symlink = extract_dir / "symlink.txt"
|
||||
if extracted_symlink.exists():
|
||||
# Verify it's still a symlink
|
||||
assert extracted_symlink.is_symlink()
|
||||
|
||||
def test_file_permissions_preservation(self, temp_dir):
|
||||
"""Test that file permissions are preserved."""
|
||||
# Create file with specific permissions
|
||||
perm_file = temp_dir / "permissions.txt"
|
||||
perm_file.write_text("Permission test content")
|
||||
|
||||
# Set specific permissions (readable/writable by owner only)
|
||||
try:
|
||||
perm_file.chmod(0o600)
|
||||
except OSError:
|
||||
pytest.skip("Permission setting not supported")
|
||||
|
||||
original_mode = perm_file.stat().st_mode
|
||||
|
||||
# Create archive
|
||||
archive_path = temp_dir / "permissions.tzst"
|
||||
result = main(["a", str(archive_path), str(perm_file)])
|
||||
assert result == 0
|
||||
|
||||
# Extract and verify permissions
|
||||
extract_dir = temp_dir / "perm_extracted"
|
||||
result = main(["x", str(archive_path), "-o", str(extract_dir)])
|
||||
assert result == 0
|
||||
|
||||
extracted_file = extract_dir / "permissions.txt"
|
||||
if extracted_file.exists():
|
||||
# Permissions should be preserved (may vary by system)
|
||||
extracted_mode = extracted_file.stat().st_mode
|
||||
# At minimum, check that it's not world-writable if original wasn't
|
||||
if not (original_mode & 0o002):
|
||||
assert not (extracted_mode & 0o002)
|
||||
|
||||
def test_executable_files(self, temp_dir):
|
||||
"""Test handling of executable files."""
|
||||
# Create executable script
|
||||
script_file = temp_dir / "test_script.sh"
|
||||
script_file.write_text("#!/bin/bash\necho 'Hello World'\n")
|
||||
|
||||
try:
|
||||
script_file.chmod(0o755) # Make executable
|
||||
except OSError:
|
||||
pytest.skip("Permission setting not supported")
|
||||
|
||||
# Create archive
|
||||
archive_path = temp_dir / "executable.tzst"
|
||||
result = main(["a", str(archive_path), str(script_file)])
|
||||
assert result == 0
|
||||
|
||||
# Extract and verify executable bit preserved
|
||||
extract_dir = temp_dir / "exec_extracted"
|
||||
result = main(["x", str(archive_path), "-o", str(extract_dir)])
|
||||
assert result == 0
|
||||
|
||||
extracted_script = extract_dir / "test_script.sh"
|
||||
if extracted_script.exists():
|
||||
# Check if executable bit is preserved
|
||||
mode = extracted_script.stat().st_mode
|
||||
assert mode & 0o100 # Owner execute bit
|
||||
|
||||
|
||||
@pytest.mark.skipif(sys.platform != "win32", reason="Windows-specific tests")
|
||||
class TestWindowsSpecificFeatures:
|
||||
"""Test Windows specific behaviors and limitations."""
|
||||
|
||||
def test_windows_reserved_names(self, temp_dir):
|
||||
"""Test handling of Windows reserved names."""
|
||||
# Windows reserved names that should be handled gracefully
|
||||
reserved_names = ["CON.txt", "PRN.txt", "AUX.txt", "NUL.txt"]
|
||||
|
||||
files_created = []
|
||||
for name in reserved_names:
|
||||
try:
|
||||
reserved_file = temp_dir / name
|
||||
reserved_file.write_text(f"Content for {name}")
|
||||
files_created.append(str(reserved_file))
|
||||
except OSError:
|
||||
# Expected on Windows - these names are reserved
|
||||
continue
|
||||
|
||||
if files_created:
|
||||
archive_path = temp_dir / "reserved_names.tzst"
|
||||
result = main(["a", str(archive_path), *files_created])
|
||||
# Should either succeed or fail gracefully
|
||||
assert result in [0, 1]
|
||||
|
||||
def test_windows_long_path_handling(self, temp_dir):
|
||||
"""Test handling of Windows long paths."""
|
||||
# Create deeply nested path (Windows has 260 char limit traditionally)
|
||||
deep_path = temp_dir
|
||||
for i in range(10):
|
||||
deep_path = deep_path / f"very_long_directory_name_{i}"
|
||||
|
||||
try:
|
||||
deep_path.mkdir(parents=True)
|
||||
deep_file = deep_path / "deep_file.txt"
|
||||
deep_file.write_text("Deep path content")
|
||||
|
||||
archive_path = temp_dir / "long_path.tzst"
|
||||
result = main(["a", str(archive_path), str(deep_file)])
|
||||
# Should handle long paths gracefully
|
||||
assert result in [0, 1]
|
||||
except OSError:
|
||||
# Skip if system doesn't support such deep paths
|
||||
pytest.skip("System doesn't support deep paths")
|
||||
|
||||
|
||||
class TestUnicodeAndInternational:
|
||||
"""Test Unicode and international character handling."""
|
||||
|
||||
def test_unicode_content_various_encodings(self, temp_dir):
|
||||
"""Test files with various Unicode content."""
|
||||
# Test various Unicode characters
|
||||
unicode_tests = [
|
||||
("chinese.txt", "你好世界 Hello World"),
|
||||
("emoji.txt", "🌟⭐✨🎉🚀💖"),
|
||||
("cyrillic.txt", "Привет мир"),
|
||||
("arabic.txt", "مرحبا بالعالم"),
|
||||
("japanese.txt", "こんにちは世界"),
|
||||
]
|
||||
|
||||
files_created = []
|
||||
for filename, content in unicode_tests:
|
||||
try:
|
||||
unicode_file = temp_dir / filename
|
||||
unicode_file.write_text(content, encoding="utf-8")
|
||||
files_created.append(str(unicode_file))
|
||||
except (OSError, UnicodeError):
|
||||
# Skip files that can't be created on this system
|
||||
continue
|
||||
|
||||
if files_created:
|
||||
archive_path = temp_dir / "unicode_content.tzst"
|
||||
result = main(["a", str(archive_path), *files_created])
|
||||
assert result == 0
|
||||
|
||||
# Extract and verify content preservation
|
||||
extract_dir = temp_dir / "unicode_extracted"
|
||||
result = main(["x", str(archive_path), "-o", str(extract_dir)])
|
||||
assert result == 0
|
||||
|
||||
# Verify at least one file's content
|
||||
for filename, expected_content in unicode_tests:
|
||||
extracted_file = extract_dir / filename
|
||||
if extracted_file.exists():
|
||||
actual_content = extracted_file.read_text(encoding="utf-8")
|
||||
assert actual_content == expected_content
|
||||
|
||||
def test_unicode_filenames_platform_dependent(self, temp_dir):
|
||||
"""Test Unicode filenames with platform-aware expectations."""
|
||||
unicode_filenames = [
|
||||
"测试文件.txt",
|
||||
"файл_тест.txt",
|
||||
"αρχείο_δοκιμή.txt",
|
||||
"ファイル_テスト.txt",
|
||||
]
|
||||
|
||||
files_created = []
|
||||
for filename in unicode_filenames:
|
||||
try:
|
||||
unicode_file = temp_dir / filename
|
||||
unicode_file.write_text(f"Content of {filename}", encoding="utf-8")
|
||||
files_created.append(str(unicode_file))
|
||||
except (OSError, UnicodeError):
|
||||
# Skip files that can't be created
|
||||
continue
|
||||
|
||||
if files_created:
|
||||
archive_path = temp_dir / "unicode_filenames.tzst"
|
||||
result = main(["a", str(archive_path), *files_created])
|
||||
|
||||
# On Windows PowerShell, this might fail due to encoding issues
|
||||
if sys.platform == "win32":
|
||||
assert result in [0, 1] # May fail on Windows
|
||||
else:
|
||||
assert result == 0 # Should succeed on Unix
|
||||
|
||||
|
||||
class TestPerformanceAndScalability:
|
||||
"""Test performance-related scenarios and scalability."""
|
||||
|
||||
def test_many_small_files_archive(self, temp_dir):
|
||||
"""Test archiving many small files efficiently.""" # Create 100 small files
|
||||
files = []
|
||||
for i in range(100):
|
||||
small_file = temp_dir / f"small_{i:03d}.txt"
|
||||
small_file.write_text(f"Small file {i} content")
|
||||
files.append(str(small_file))
|
||||
|
||||
archive_path = temp_dir / "many_small.tzst"
|
||||
result = main(["a", str(archive_path), *files])
|
||||
assert result == 0
|
||||
|
||||
# Verify archive integrity
|
||||
result = main(["t", str(archive_path)])
|
||||
assert result == 0
|
||||
|
||||
def test_large_single_file_archive(self, temp_dir):
|
||||
"""Test archiving a single large file."""
|
||||
# Create 5MB file
|
||||
large_file = temp_dir / "large.txt"
|
||||
content = "This is large file content.\n" * 200000 # ~5MB
|
||||
large_file.write_text(content)
|
||||
|
||||
archive_path = temp_dir / "large_single.tzst"
|
||||
result = main(["a", str(archive_path), str(large_file), "-l", "22"])
|
||||
assert result == 0
|
||||
|
||||
# Verify archive can be tested
|
||||
result = main(["t", str(archive_path)])
|
||||
assert result == 0
|
||||
|
||||
def test_compression_efficiency_levels(self, temp_dir):
|
||||
"""Test different compression levels for efficiency."""
|
||||
# Create test file with repetitive content (compresses well)
|
||||
test_file = temp_dir / "compressible.txt"
|
||||
repetitive_content = "ABCD" * 10000 # 40KB of repetitive data
|
||||
test_file.write_text(repetitive_content)
|
||||
|
||||
# Test different compression levels
|
||||
for level in [1, 3, 10, 22]:
|
||||
archive_path = temp_dir / f"compression_level_{level}.tzst"
|
||||
result = main(["a", str(archive_path), str(test_file), "-l", str(level)])
|
||||
assert result == 0
|
||||
|
||||
# Higher compression should generally result in smaller files
|
||||
# (though not guaranteed for all content types)
|
||||
assert archive_path.exists()
|
||||
assert archive_path.stat().st_size > 0
|
||||
|
||||
|
||||
class TestErrorRecoveryAndRobustness:
|
||||
"""Test error recovery and robustness scenarios."""
|
||||
|
||||
def test_disk_space_simulation(self, temp_dir):
|
||||
"""Test behavior when disk space might be limited."""
|
||||
# Create moderately large file
|
||||
large_file = temp_dir / "disk_space_test.txt"
|
||||
content = "Large content for disk space test.\n" * 50000
|
||||
large_file.write_text(content)
|
||||
|
||||
# Try to create archive with maximum compression
|
||||
archive_path = temp_dir / "disk_space.tzst"
|
||||
result = main(["a", str(archive_path), str(large_file), "-l", "22"])
|
||||
# Should either succeed or fail gracefully
|
||||
assert result in [0, 1]
|
||||
|
||||
def test_interrupted_operation_simulation(self, temp_dir):
|
||||
"""Test that partial operations don't leave corrupted files."""
|
||||
# Create test file
|
||||
test_file = temp_dir / "interrupt_test.txt"
|
||||
test_file.write_text("Interrupt test content")
|
||||
|
||||
# Create archive successfully first
|
||||
archive_path = temp_dir / "interrupt_test.tzst"
|
||||
result = main(["a", str(archive_path), str(test_file)])
|
||||
assert result == 0
|
||||
|
||||
# Verify archive integrity
|
||||
result = main(["t", str(archive_path)])
|
||||
assert result == 0
|
||||
|
||||
# If archive exists, it should be valid
|
||||
if archive_path.exists():
|
||||
result = main(["l", str(archive_path)])
|
||||
assert result == 0
|
||||
|
||||
def test_readonly_archive_handling(self, temp_dir):
|
||||
"""Test behavior with read-only archives."""
|
||||
# Create archive first
|
||||
test_file = temp_dir / "readonly_test.txt"
|
||||
test_file.write_text("Read-only test content")
|
||||
|
||||
archive_path = temp_dir / "readonly.tzst"
|
||||
result = main(["a", str(archive_path), str(test_file)])
|
||||
assert result == 0
|
||||
|
||||
# Make archive read-only
|
||||
try:
|
||||
archive_path.chmod(0o444)
|
||||
except OSError:
|
||||
pytest.skip("Cannot set read-only permissions")
|
||||
|
||||
# Should still be able to read the archive
|
||||
result = main(["l", str(archive_path)])
|
||||
assert result == 0
|
||||
|
||||
result = main(["t", str(archive_path)])
|
||||
assert result == 0
|
||||
|
||||
# Extract should work
|
||||
extract_dir = temp_dir / "readonly_extracted"
|
||||
result = main(["x", str(archive_path), "-o", str(extract_dir)])
|
||||
assert result == 0
|
||||
|
||||
|
||||
class TestCommandLineInterfaceEdgeCases:
|
||||
"""Test edge cases in command line interface handling."""
|
||||
|
||||
def test_empty_argument_handling(self, temp_dir):
|
||||
"""Test behavior with empty or unusual arguments."""
|
||||
# Test with empty string as filename (treated as current directory)
|
||||
archive_path = temp_dir / "test.tzst"
|
||||
result = main(["a", str(archive_path), ""])
|
||||
assert result == 0 # Empty string is treated as current directory
|
||||
|
||||
# Test with very long argument
|
||||
long_arg = "a" * 1000
|
||||
result = main(["a", "test.tzst", long_arg])
|
||||
assert result == 1 # Should fail gracefully
|
||||
|
||||
def test_special_characters_in_archive_names(self, temp_dir):
|
||||
"""Test archive names with special characters."""
|
||||
test_file = temp_dir / "special_arch_test.txt"
|
||||
test_file.write_text("Special archive name test")
|
||||
|
||||
# Test various special characters in archive names
|
||||
special_names = [
|
||||
"archive with spaces.tzst",
|
||||
"archive-with-dashes.tzst",
|
||||
"archive_with_underscores.tzst",
|
||||
"archive.with.dots.tzst",
|
||||
]
|
||||
|
||||
for name in special_names:
|
||||
archive_path = temp_dir / name
|
||||
result = main(["a", str(archive_path), str(test_file)])
|
||||
assert result == 0
|
||||
assert archive_path.exists()
|
||||
|
||||
def test_output_directory_creation(self, temp_dir):
|
||||
"""Test that output directories are created when they don't exist."""
|
||||
# Create test archive first
|
||||
test_file = temp_dir / "output_test.txt"
|
||||
test_file.write_text("Output directory test")
|
||||
|
||||
archive_path = temp_dir / "output.tzst"
|
||||
result = main(["a", str(archive_path), str(test_file)])
|
||||
assert result == 0
|
||||
|
||||
# Extract to non-existent directory
|
||||
nonexistent_dir = temp_dir / "does" / "not" / "exist"
|
||||
result = main(["x", str(archive_path), "-o", str(nonexistent_dir)])
|
||||
assert result == 0
|
||||
assert nonexistent_dir.exists()
|
||||
assert (nonexistent_dir / "output_test.txt").exists()
|
||||
@@ -1,245 +0,0 @@
|
||||
"""Tests for security features and extraction filters."""
|
||||
|
||||
from unittest.mock import patch
|
||||
|
||||
from tzst import TzstArchive, extract_archive
|
||||
|
||||
|
||||
class TestExtractionFilters:
|
||||
"""Test extraction filter security features."""
|
||||
|
||||
def test_default_filter_is_data(self, sample_files, temp_dir):
|
||||
"""Test that the default filter is 'data' for security."""
|
||||
archive_path = temp_dir / "test_security.tzst"
|
||||
file_paths = [f for f in sample_files if f.is_file()]
|
||||
|
||||
# Create archive
|
||||
with TzstArchive(archive_path, "w") as archive:
|
||||
for file_path in file_paths:
|
||||
relative_path = file_path.relative_to(sample_files[0].parent)
|
||||
archive.add(file_path, arcname=str(relative_path))
|
||||
|
||||
# Test that default filter is 'data'
|
||||
extract_dir = temp_dir / "extracted_default"
|
||||
|
||||
with patch("tarfile.TarFile.extractall") as mock_extractall:
|
||||
with TzstArchive(archive_path, "r") as archive:
|
||||
archive.extract(path=extract_dir)
|
||||
|
||||
# Verify that 'data' filter was used
|
||||
mock_extractall.assert_called_once()
|
||||
call_args = mock_extractall.call_args
|
||||
assert "filter" in call_args[1]
|
||||
assert call_args[1]["filter"] == "data"
|
||||
|
||||
def test_data_filter_explicit(self, sample_files, temp_dir):
|
||||
"""Test explicitly setting 'data' filter."""
|
||||
archive_path = temp_dir / "test_data_filter.tzst"
|
||||
file_paths = [f for f in sample_files if f.is_file()]
|
||||
|
||||
# Create archive
|
||||
with TzstArchive(archive_path, "w") as archive:
|
||||
for file_path in file_paths:
|
||||
relative_path = file_path.relative_to(sample_files[0].parent)
|
||||
archive.add(file_path, arcname=str(relative_path))
|
||||
|
||||
# Test extraction with explicit 'data' filter
|
||||
extract_dir = temp_dir / "extracted_data"
|
||||
|
||||
with patch("tarfile.TarFile.extractall") as mock_extractall:
|
||||
with TzstArchive(archive_path, "r") as archive:
|
||||
archive.extract(path=extract_dir, filter="data")
|
||||
|
||||
mock_extractall.assert_called_once()
|
||||
call_args = mock_extractall.call_args
|
||||
assert call_args[1]["filter"] == "data"
|
||||
|
||||
def test_tar_filter(self, sample_files, temp_dir):
|
||||
"""Test 'tar' filter for Unix-like features."""
|
||||
archive_path = temp_dir / "test_tar_filter.tzst"
|
||||
file_paths = [f for f in sample_files if f.is_file()]
|
||||
|
||||
# Create archive
|
||||
with TzstArchive(archive_path, "w") as archive:
|
||||
for file_path in file_paths:
|
||||
relative_path = file_path.relative_to(sample_files[0].parent)
|
||||
archive.add(file_path, arcname=str(relative_path))
|
||||
|
||||
# Test extraction with 'tar' filter
|
||||
extract_dir = temp_dir / "extracted_tar"
|
||||
|
||||
with patch("tarfile.TarFile.extractall") as mock_extractall:
|
||||
with TzstArchive(archive_path, "r") as archive:
|
||||
archive.extract(path=extract_dir, filter="tar")
|
||||
|
||||
mock_extractall.assert_called_once()
|
||||
call_args = mock_extractall.call_args
|
||||
assert call_args[1]["filter"] == "tar"
|
||||
|
||||
def test_fully_trusted_filter(self, sample_files, temp_dir):
|
||||
"""Test 'fully_trusted' filter (dangerous but complete)."""
|
||||
archive_path = temp_dir / "test_trusted_filter.tzst"
|
||||
file_paths = [f for f in sample_files if f.is_file()]
|
||||
|
||||
# Create archive
|
||||
with TzstArchive(archive_path, "w") as archive:
|
||||
for file_path in file_paths:
|
||||
relative_path = file_path.relative_to(sample_files[0].parent)
|
||||
archive.add(file_path, arcname=str(relative_path))
|
||||
|
||||
# Test extraction with 'fully_trusted' filter
|
||||
extract_dir = temp_dir / "extracted_trusted"
|
||||
|
||||
with patch("tarfile.TarFile.extractall") as mock_extractall:
|
||||
with TzstArchive(archive_path, "r") as archive:
|
||||
archive.extract(path=extract_dir, filter="fully_trusted")
|
||||
|
||||
mock_extractall.assert_called_once()
|
||||
call_args = mock_extractall.call_args
|
||||
assert call_args[1]["filter"] == "fully_trusted"
|
||||
|
||||
def test_none_filter_with_warning(self, sample_files, temp_dir, capsys):
|
||||
"""Test None filter shows deprecation warning."""
|
||||
archive_path = temp_dir / "test_none_filter.tzst"
|
||||
file_paths = [f for f in sample_files if f.is_file()]
|
||||
|
||||
# Create archive
|
||||
with TzstArchive(archive_path, "w") as archive:
|
||||
for file_path in file_paths:
|
||||
relative_path = file_path.relative_to(sample_files[0].parent)
|
||||
archive.add(file_path, arcname=str(relative_path))
|
||||
|
||||
# Test extraction with None filter
|
||||
extract_dir = temp_dir / "extracted_none"
|
||||
|
||||
with patch("tarfile.TarFile.extractall") as mock_extractall:
|
||||
with TzstArchive(archive_path, "r") as archive:
|
||||
archive.extract(path=extract_dir, filter=None)
|
||||
|
||||
mock_extractall.assert_called_once()
|
||||
call_args = mock_extractall.call_args
|
||||
assert call_args[1]["filter"] is None
|
||||
|
||||
def test_custom_filter_function(self, sample_files, temp_dir):
|
||||
"""Test custom filter function."""
|
||||
archive_path = temp_dir / "test_custom_filter.tzst"
|
||||
file_paths = [f for f in sample_files if f.is_file()]
|
||||
|
||||
# Create archive
|
||||
with TzstArchive(archive_path, "w") as archive:
|
||||
for file_path in file_paths:
|
||||
relative_path = file_path.relative_to(sample_files[0].parent)
|
||||
archive.add(file_path, arcname=str(relative_path))
|
||||
|
||||
# Define custom filter function
|
||||
def custom_filter(member, path):
|
||||
"""Custom filter that only allows regular files."""
|
||||
if member.isfile():
|
||||
return member
|
||||
return None
|
||||
|
||||
# Test extraction with custom filter
|
||||
extract_dir = temp_dir / "extracted_custom"
|
||||
|
||||
with patch("tarfile.TarFile.extractall") as mock_extractall:
|
||||
with TzstArchive(archive_path, "r") as archive:
|
||||
archive.extract(path=extract_dir, filter=custom_filter)
|
||||
|
||||
mock_extractall.assert_called_once()
|
||||
call_args = mock_extractall.call_args
|
||||
assert call_args[1]["filter"] == custom_filter
|
||||
|
||||
def test_convenience_function_filter(self, sample_files, temp_dir):
|
||||
"""Test filter parameter in extract_archive convenience function."""
|
||||
archive_path = temp_dir / "test_convenience_filter.tzst"
|
||||
file_paths = [f for f in sample_files if f.is_file()]
|
||||
|
||||
# Create archive
|
||||
with TzstArchive(archive_path, "w") as archive:
|
||||
for file_path in file_paths:
|
||||
relative_path = file_path.relative_to(sample_files[0].parent)
|
||||
archive.add(file_path, arcname=str(relative_path))
|
||||
|
||||
# Test extract_archive with different filters
|
||||
for filter_type in ["data", "tar", "fully_trusted"]:
|
||||
extract_dir = temp_dir / f"extracted_conv_{filter_type}"
|
||||
|
||||
# This should not raise an exception
|
||||
extract_archive(archive_path, extract_dir, filter=filter_type)
|
||||
|
||||
# Verify files were extracted
|
||||
assert extract_dir.exists()
|
||||
extracted_files = list(extract_dir.rglob("*"))
|
||||
assert len([f for f in extracted_files if f.is_file()]) > 0
|
||||
|
||||
|
||||
class TestSecurityDocumentation:
|
||||
"""Test security-related documentation and warnings."""
|
||||
|
||||
def test_extract_method_has_security_warning(self):
|
||||
"""Test that extract method has proper security warning in docstring."""
|
||||
docstring = TzstArchive.extract.__doc__
|
||||
assert docstring is not None
|
||||
assert "Never extract archives from untrusted sources" in docstring
|
||||
assert "filter" in docstring
|
||||
assert "data" in docstring
|
||||
|
||||
def test_extract_archive_has_security_warning(self):
|
||||
"""Test that extract_archive function has proper security warning."""
|
||||
docstring = extract_archive.__doc__
|
||||
assert docstring is not None
|
||||
assert "Never extract archives from untrusted sources" in docstring
|
||||
assert "path traversal attacks" in docstring
|
||||
assert "data" in docstring
|
||||
|
||||
|
||||
class TestSecurityEdgeCases:
|
||||
"""Test edge cases and error conditions for security features."""
|
||||
|
||||
def test_streaming_mode_filter_compatibility(self, sample_files, temp_dir):
|
||||
"""Test that filters work correctly with streaming mode."""
|
||||
archive_path = temp_dir / "test_streaming_filter.tzst"
|
||||
file_paths = [f for f in sample_files if f.is_file()]
|
||||
|
||||
# Create archive
|
||||
with TzstArchive(archive_path, "w") as archive:
|
||||
for file_path in file_paths:
|
||||
relative_path = file_path.relative_to(sample_files[0].parent)
|
||||
archive.add(file_path, arcname=str(relative_path))
|
||||
|
||||
# Test extraction in streaming mode with filter
|
||||
extract_dir = temp_dir / "extracted_streaming"
|
||||
|
||||
# This should work without errors
|
||||
with TzstArchive(archive_path, "r", streaming=True) as archive:
|
||||
archive.extract(path=extract_dir, filter="data")
|
||||
|
||||
# Verify files were extracted
|
||||
assert extract_dir.exists()
|
||||
extracted_files = list(extract_dir.rglob("*"))
|
||||
assert len([f for f in extracted_files if f.is_file()]) > 0
|
||||
|
||||
def test_filter_with_specific_member_extraction(self, sample_files, temp_dir):
|
||||
"""Test filter when extracting specific members."""
|
||||
archive_path = temp_dir / "test_member_filter.tzst"
|
||||
file_paths = [f for f in sample_files if f.is_file()]
|
||||
|
||||
# Create archive
|
||||
with TzstArchive(archive_path, "w") as archive:
|
||||
for file_path in file_paths:
|
||||
relative_path = file_path.relative_to(sample_files[0].parent)
|
||||
archive.add(file_path, arcname=str(relative_path))
|
||||
|
||||
# Test extracting specific member with filter
|
||||
extract_dir = temp_dir / "extracted_member"
|
||||
|
||||
if file_paths:
|
||||
with TzstArchive(archive_path, "r") as archive:
|
||||
members = archive.getnames()
|
||||
if members:
|
||||
# Extract first member with data filter
|
||||
archive.extract(member=members[0], path=extract_dir, filter="data")
|
||||
# Verify file was extracted
|
||||
assert extract_dir.exists()
|
||||
extracted_file = extract_dir / members[0]
|
||||
assert extracted_file.exists()
|
||||
@@ -0,0 +1 @@
|
||||
"""Unit tests for tzst library components."""
|
||||
@@ -0,0 +1,150 @@
|
||||
"""Tests for TzstArchive class core functionality."""
|
||||
|
||||
from tzst import TzstArchive
|
||||
|
||||
|
||||
class TestBasicImportAndCreation:
|
||||
"""Test basic import and creation functionality."""
|
||||
|
||||
def test_simple_import(self):
|
||||
"""Test that we can import TzstArchive."""
|
||||
assert TzstArchive is not None
|
||||
|
||||
def test_simple_creation(self):
|
||||
"""Test simple TzstArchive creation."""
|
||||
archive = TzstArchive("test.tzst")
|
||||
assert archive.filename.name == "test.tzst"
|
||||
assert archive.mode == "r"
|
||||
|
||||
|
||||
class TestTzstArchiveBasics:
|
||||
"""Test basic TzstArchive class functionality."""
|
||||
|
||||
def test_create_and_list_archive(self, sample_files, sample_archive_path):
|
||||
"""Test creating an archive and listing its contents."""
|
||||
# Create archive with relative paths using arcname
|
||||
with TzstArchive(sample_archive_path, "w") as archive:
|
||||
for file_path in sample_files:
|
||||
if file_path.is_file():
|
||||
relative_path = file_path.relative_to(sample_files[0].parent)
|
||||
archive.add(file_path, arcname=str(relative_path))
|
||||
|
||||
assert sample_archive_path.exists()
|
||||
|
||||
# List contents
|
||||
with TzstArchive(sample_archive_path, "r") as archive:
|
||||
contents = archive.list()
|
||||
|
||||
# Should have all files
|
||||
expected_names = [
|
||||
str(f.relative_to(sample_files[0].parent)).replace("\\", "/")
|
||||
for f in sample_files
|
||||
if f.is_file()
|
||||
]
|
||||
actual_names = [item["name"] for item in contents]
|
||||
|
||||
for expected in expected_names:
|
||||
assert expected in actual_names
|
||||
|
||||
def test_extract_archive(self, sample_files, sample_archive_path, temp_dir):
|
||||
"""Test extracting files from an archive."""
|
||||
# Create archive with relative paths using arcname
|
||||
with TzstArchive(sample_archive_path, "w") as archive:
|
||||
for file_path in sample_files:
|
||||
if file_path.is_file():
|
||||
relative_path = file_path.relative_to(sample_files[0].parent)
|
||||
archive.add(file_path, arcname=str(relative_path))
|
||||
|
||||
# Extract to new directory
|
||||
extract_dir = temp_dir / "extracted"
|
||||
with TzstArchive(sample_archive_path, "r") as archive:
|
||||
archive.extract(path=extract_dir)
|
||||
|
||||
# Verify extracted files
|
||||
for file_path in sample_files:
|
||||
if file_path.is_file():
|
||||
relative_path = file_path.relative_to(sample_files[0].parent)
|
||||
extracted_file = extract_dir / relative_path
|
||||
assert extracted_file.exists()
|
||||
|
||||
# Compare content
|
||||
original_content = file_path.read_bytes()
|
||||
extracted_content = extracted_file.read_bytes()
|
||||
assert original_content == extracted_content
|
||||
|
||||
def test_archive_test(self, sample_files, sample_archive_path):
|
||||
"""Test archive integrity testing."""
|
||||
# Create archive with relative paths using arcname
|
||||
with TzstArchive(sample_archive_path, "w") as archive:
|
||||
for file_path in sample_files:
|
||||
if file_path.is_file():
|
||||
relative_path = file_path.relative_to(sample_files[0].parent)
|
||||
archive.add(file_path, arcname=str(relative_path))
|
||||
|
||||
# Test archive integrity
|
||||
with TzstArchive(sample_archive_path, "r") as archive:
|
||||
assert archive.test() is True
|
||||
|
||||
def test_verbose_listing(self, sample_files, sample_archive_path):
|
||||
"""Test verbose listing of archive contents."""
|
||||
# Create archive with relative paths using arcname
|
||||
with TzstArchive(sample_archive_path, "w") as archive:
|
||||
for file_path in sample_files:
|
||||
if file_path.is_file():
|
||||
relative_path = file_path.relative_to(sample_files[0].parent)
|
||||
archive.add(file_path, arcname=str(relative_path))
|
||||
|
||||
# Get verbose listing
|
||||
with TzstArchive(sample_archive_path, "r") as archive:
|
||||
contents = archive.list(verbose=True)
|
||||
|
||||
# Check that verbose fields are present
|
||||
for item in contents:
|
||||
assert "mode" in item
|
||||
assert "mtime" in item
|
||||
assert "mtime_str" in item
|
||||
assert "uid" in item
|
||||
assert "gid" in item
|
||||
|
||||
|
||||
class TestTzstArchiveStreamingMode:
|
||||
"""Test streaming mode functionality."""
|
||||
|
||||
def test_streaming_mode_archive(self, sample_files, sample_archive_path):
|
||||
"""Test streaming mode for reading archives."""
|
||||
# Create archive normally
|
||||
file_paths = [f for f in sample_files if f.is_file()]
|
||||
with TzstArchive(sample_archive_path, "w") as archive:
|
||||
for file_path in file_paths:
|
||||
relative_path = file_path.relative_to(sample_files[0].parent)
|
||||
archive.add(file_path, arcname=str(relative_path))
|
||||
|
||||
# Test streaming mode reading
|
||||
with TzstArchive(sample_archive_path, "r", streaming=True) as archive:
|
||||
contents = archive.list()
|
||||
assert len(contents) > 0
|
||||
|
||||
def test_streaming_vs_buffered_mode(self, sample_files, sample_archive_path):
|
||||
"""Test that streaming and buffered modes produce same results."""
|
||||
# Create archive
|
||||
file_paths = [f for f in sample_files if f.is_file()]
|
||||
with TzstArchive(sample_archive_path, "w") as archive:
|
||||
for file_path in file_paths:
|
||||
relative_path = file_path.relative_to(sample_files[0].parent)
|
||||
archive.add(file_path, arcname=str(relative_path))
|
||||
|
||||
# Read with buffered mode
|
||||
with TzstArchive(sample_archive_path, "r", streaming=False) as archive:
|
||||
buffered_contents = archive.list()
|
||||
|
||||
# Read with streaming mode
|
||||
with TzstArchive(sample_archive_path, "r", streaming=True) as archive:
|
||||
streaming_contents = archive.list()
|
||||
|
||||
# Results should be identical
|
||||
assert len(buffered_contents) == len(streaming_contents)
|
||||
for buffered, streaming in zip(
|
||||
buffered_contents, streaming_contents, strict=True
|
||||
):
|
||||
assert buffered["name"] == streaming["name"]
|
||||
assert buffered["size"] == streaming["size"]
|
||||
@@ -0,0 +1,257 @@
|
||||
"""Tests for tzst convenience functions."""
|
||||
|
||||
import pytest
|
||||
|
||||
from tzst import create_archive, extract_archive, list_archive
|
||||
from tzst import test_archive as tzst_test_archive
|
||||
|
||||
|
||||
class TestConvenienceFunctions:
|
||||
"""Test the convenience functions."""
|
||||
|
||||
def test_create_archive_function(self, sample_files, sample_archive_path):
|
||||
"""Test create_archive function."""
|
||||
file_paths = [f for f in sample_files if f.is_file()]
|
||||
create_archive(sample_archive_path, file_paths)
|
||||
|
||||
assert sample_archive_path.exists()
|
||||
|
||||
# Verify contents
|
||||
contents = list_archive(sample_archive_path)
|
||||
assert len(contents) > 0
|
||||
|
||||
def test_extract_archive_function(
|
||||
self, sample_files, sample_archive_path, temp_dir
|
||||
):
|
||||
"""Test extract_archive function."""
|
||||
# Create archive first
|
||||
file_paths = [f for f in sample_files if f.is_file()]
|
||||
create_archive(sample_archive_path, file_paths)
|
||||
|
||||
# Extract
|
||||
extract_dir = temp_dir / "extracted"
|
||||
extract_archive(sample_archive_path, extract_dir)
|
||||
|
||||
# Verify extraction
|
||||
assert extract_dir.exists()
|
||||
extracted_files = list(extract_dir.rglob("*"))
|
||||
assert len(extracted_files) > 0
|
||||
|
||||
def test_extract_archive_flat(self, sample_files, sample_archive_path, temp_dir):
|
||||
"""Test flat extraction."""
|
||||
# Create archive first
|
||||
file_paths = [f for f in sample_files if f.is_file()]
|
||||
create_archive(sample_archive_path, file_paths)
|
||||
|
||||
# Extract flat
|
||||
extract_dir = temp_dir / "extracted_flat"
|
||||
extract_archive(sample_archive_path, extract_dir, flatten=True)
|
||||
|
||||
# Verify flat extraction (no subdirectories)
|
||||
assert extract_dir.exists()
|
||||
extracted_files = [f for f in extract_dir.iterdir() if f.is_file()]
|
||||
extracted_dirs = [d for d in extract_dir.iterdir() if d.is_dir()]
|
||||
|
||||
assert len(extracted_files) > 0
|
||||
assert len(extracted_dirs) == 0 # Should be flat
|
||||
|
||||
def test_list_archive_function(self, sample_files, sample_archive_path):
|
||||
"""Test list_archive function."""
|
||||
# Create archive first
|
||||
file_paths = [f for f in sample_files if f.is_file()]
|
||||
create_archive(sample_archive_path, file_paths)
|
||||
|
||||
# List contents
|
||||
contents = list_archive(sample_archive_path)
|
||||
assert len(contents) > 0
|
||||
|
||||
# Test verbose listing
|
||||
verbose_contents = list_archive(sample_archive_path, verbose=True)
|
||||
assert len(verbose_contents) == len(contents)
|
||||
assert "mode" in verbose_contents[0]
|
||||
|
||||
def test_test_archive_function(self, sample_files, sample_archive_path):
|
||||
"""Test test_archive function."""
|
||||
# Create archive first
|
||||
file_paths = [f for f in sample_files if f.is_file()]
|
||||
create_archive(sample_archive_path, file_paths)
|
||||
|
||||
# Test archive
|
||||
assert tzst_test_archive(sample_archive_path) is True
|
||||
|
||||
# Test non-existent archive
|
||||
fake_archive = sample_archive_path.parent / "fake.tzst"
|
||||
assert tzst_test_archive(fake_archive) is False
|
||||
|
||||
def test_streaming_convenience_functions(
|
||||
self, sample_files, sample_archive_path, temp_dir
|
||||
):
|
||||
"""Test convenience functions with streaming parameter."""
|
||||
# Create archive first
|
||||
file_paths = [f for f in sample_files if f.is_file()]
|
||||
create_archive(sample_archive_path, file_paths)
|
||||
|
||||
# Test list_archive with streaming
|
||||
contents_normal = list_archive(sample_archive_path, streaming=False)
|
||||
contents_streaming = list_archive(sample_archive_path, streaming=True)
|
||||
assert len(contents_normal) == len(contents_streaming)
|
||||
|
||||
# Test test_archive with streaming
|
||||
assert tzst_test_archive(sample_archive_path, streaming=False) is True
|
||||
assert tzst_test_archive(sample_archive_path, streaming=True) is True
|
||||
|
||||
# Test extract_archive with streaming
|
||||
extract_dir_normal = temp_dir / "extract_normal"
|
||||
extract_dir_streaming = temp_dir / "extract_streaming"
|
||||
|
||||
extract_archive(sample_archive_path, extract_dir_normal, streaming=False)
|
||||
extract_archive(sample_archive_path, extract_dir_streaming, streaming=True)
|
||||
|
||||
assert extract_dir_normal.exists()
|
||||
assert extract_dir_streaming.exists()
|
||||
|
||||
|
||||
class TestAtomicOperations:
|
||||
"""Test atomic file operations."""
|
||||
|
||||
def test_atomic_file_operations(self, sample_files, temp_dir):
|
||||
"""Test atomic file operations."""
|
||||
archive_path = temp_dir / "atomic_test.tzst"
|
||||
file_paths = [f for f in sample_files if f.is_file()]
|
||||
|
||||
# Test with atomic operations enabled
|
||||
create_archive(archive_path, file_paths, use_temp_file=True)
|
||||
assert archive_path.exists()
|
||||
|
||||
# Verify archive is valid
|
||||
assert tzst_test_archive(archive_path) is True
|
||||
|
||||
def test_non_atomic_file_creation(self, sample_files, temp_dir):
|
||||
"""Test that non-atomic creation also works."""
|
||||
archive_path = temp_dir / "non_atomic_test.tzst"
|
||||
file_paths = [f for f in sample_files if f.is_file()]
|
||||
|
||||
# Test with atomic operations disabled
|
||||
create_archive(archive_path, file_paths, use_temp_file=False)
|
||||
assert archive_path.exists()
|
||||
|
||||
# Verify archive is valid
|
||||
assert tzst_test_archive(archive_path) is True
|
||||
|
||||
def test_atomic_cleanup_on_error(self, temp_dir):
|
||||
"""Test that temporary files are cleaned up on errors."""
|
||||
archive_path = temp_dir / "cleanup_test.tzst"
|
||||
|
||||
# Try to create archive with non-existent files
|
||||
with pytest.raises(FileNotFoundError):
|
||||
create_archive(archive_path, ["non_existent_file.txt"], use_temp_file=True)
|
||||
|
||||
# Archive should not exist
|
||||
assert not archive_path.exists()
|
||||
|
||||
# No temporary files should be left behind
|
||||
temp_files = list(temp_dir.glob(".cleanup_test.tzst.*"))
|
||||
assert len(temp_files) == 0
|
||||
|
||||
|
||||
class TestCompressionLevels:
|
||||
"""Test compression level validation and functionality."""
|
||||
|
||||
def test_compression_level_validation(self, sample_files, temp_dir):
|
||||
"""Test compression level validation."""
|
||||
file_paths = [f for f in sample_files if f.is_file()]
|
||||
|
||||
# Test valid compression levels
|
||||
for level in [1, 3, 10, 22]:
|
||||
archive_path = temp_dir / f"level_{level}.tzst"
|
||||
create_archive(archive_path, file_paths, compression_level=level)
|
||||
assert archive_path.exists()
|
||||
assert tzst_test_archive(archive_path) is True
|
||||
|
||||
# Test invalid compression levels
|
||||
for invalid_level in [0, 23, -1, 100]:
|
||||
archive_path = temp_dir / f"invalid_{invalid_level}.tzst"
|
||||
with pytest.raises(ValueError) as exc_info:
|
||||
create_archive(
|
||||
archive_path, file_paths, compression_level=invalid_level
|
||||
)
|
||||
|
||||
assert "compression level" in str(exc_info.value).lower()
|
||||
assert "1" in str(exc_info.value) and "22" in str(exc_info.value)
|
||||
|
||||
|
||||
class TestEdgeCaseCoverage:
|
||||
"""Test edge cases to improve coverage."""
|
||||
|
||||
def test_empty_files_list(self, temp_dir):
|
||||
"""Test create_archive with empty files list."""
|
||||
archive_path = temp_dir / "empty.tzst"
|
||||
|
||||
# Should create an empty archive
|
||||
create_archive(archive_path, [])
|
||||
|
||||
assert archive_path.exists()
|
||||
contents = list_archive(archive_path)
|
||||
assert len(contents) == 0
|
||||
|
||||
def test_create_archive_with_use_temp_file_false(self, temp_dir):
|
||||
"""Test creating archive with use_temp_file=False."""
|
||||
test_file = temp_dir / "test.txt"
|
||||
test_file.write_text("test content")
|
||||
archive_path = temp_dir / "test.tzst"
|
||||
|
||||
# Test non-atomic mode
|
||||
create_archive(archive_path, [str(test_file)], use_temp_file=False)
|
||||
|
||||
assert archive_path.exists()
|
||||
contents = list_archive(archive_path)
|
||||
assert len(contents) == 1
|
||||
|
||||
def test_extract_archive_to_specific_path(self, temp_dir):
|
||||
"""Test extracting archive to specific path."""
|
||||
# Create test archive
|
||||
test_file = temp_dir / "test.txt"
|
||||
test_file.write_text("test content")
|
||||
archive_path = temp_dir / "test.tzst"
|
||||
create_archive(archive_path, [str(test_file)])
|
||||
|
||||
# Extract to specific directory
|
||||
extract_dir = temp_dir / "extracted"
|
||||
extract_archive(archive_path, extract_dir)
|
||||
|
||||
assert extract_dir.exists()
|
||||
assert (extract_dir / "test.txt").exists()
|
||||
assert (extract_dir / "test.txt").read_text() == "test content"
|
||||
|
||||
def test_list_archive_with_streaming(self, temp_dir):
|
||||
"""Test listing archive with streaming mode."""
|
||||
test_file = temp_dir / "test.txt"
|
||||
test_file.write_text("test content")
|
||||
archive_path = temp_dir / "test.tzst"
|
||||
create_archive(archive_path, [str(test_file)])
|
||||
|
||||
# Test with streaming=True
|
||||
contents = list_archive(archive_path, streaming=True)
|
||||
assert len(contents) == 1
|
||||
assert contents[0]["name"] == "test.txt"
|
||||
|
||||
def test_test_archive_success(self, temp_dir):
|
||||
"""Test testing a valid archive."""
|
||||
test_file = temp_dir / "test.txt"
|
||||
test_file.write_text("test content")
|
||||
archive_path = temp_dir / "test.tzst"
|
||||
create_archive(archive_path, [str(test_file)])
|
||||
|
||||
# Test archive - should return True for valid archive
|
||||
result = tzst_test_archive(archive_path)
|
||||
assert result is True
|
||||
|
||||
def test_test_archive_failure(self, temp_dir):
|
||||
"""Test testing an invalid archive."""
|
||||
# Create a file that's not a valid archive
|
||||
invalid_archive = temp_dir / "invalid.tzst"
|
||||
invalid_archive.write_text("This is not a valid archive")
|
||||
|
||||
# Test archive - should return False for invalid archive
|
||||
result = tzst_test_archive(invalid_archive)
|
||||
assert result is False
|
||||
@@ -0,0 +1,432 @@
|
||||
"""Tests for security features and error handling."""
|
||||
|
||||
from unittest.mock import patch
|
||||
|
||||
import pytest
|
||||
|
||||
from tzst import TzstArchive, create_archive, extract_archive
|
||||
from tzst import test_archive as tzst_test_archive
|
||||
|
||||
|
||||
class TestErrorHandling:
|
||||
"""Test error handling."""
|
||||
|
||||
def test_invalid_archive_mode(self, sample_archive_path):
|
||||
"""Test invalid archive mode."""
|
||||
with pytest.raises(ValueError):
|
||||
TzstArchive(sample_archive_path, "invalid")
|
||||
|
||||
def test_append_mode_not_supported(self, sample_archive_path):
|
||||
"""Test that append mode raises NotImplementedError."""
|
||||
with pytest.raises(NotImplementedError):
|
||||
TzstArchive(sample_archive_path, "a")
|
||||
|
||||
def test_archive_not_open(self, sample_archive_path):
|
||||
"""Test operations on non-open archive."""
|
||||
archive = TzstArchive(sample_archive_path, "r")
|
||||
|
||||
with pytest.raises(RuntimeError):
|
||||
archive.getnames()
|
||||
|
||||
def test_file_not_found(self, temp_dir):
|
||||
"""Test handling of non-existent files."""
|
||||
archive_path = temp_dir / "test.tzst"
|
||||
fake_file = temp_dir / "fake.txt"
|
||||
|
||||
with pytest.raises(FileNotFoundError):
|
||||
create_archive(archive_path, [fake_file])
|
||||
|
||||
|
||||
class TestSecurityFiltering:
|
||||
"""Test security filtering mechanisms."""
|
||||
|
||||
def test_tar_filter_extraction(self, sample_files, temp_dir):
|
||||
"""Test extraction with TAR security filter."""
|
||||
file_paths = [f for f in sample_files if f.is_file()]
|
||||
archive_path = temp_dir / "tar_filter_test.tzst"
|
||||
|
||||
# Create archive
|
||||
create_archive(archive_path, file_paths)
|
||||
|
||||
# Extract with tar filter
|
||||
extract_dir = temp_dir / "tar_filtered"
|
||||
with patch("tzst.core.TzstArchive.extract") as mock_extract:
|
||||
extract_archive(archive_path, extract_dir, filter="tar")
|
||||
|
||||
# Verify filter was passed
|
||||
call_args = mock_extract.call_args
|
||||
assert call_args[1]["filter"] == "tar"
|
||||
|
||||
def test_data_filter_extraction(self, sample_files, temp_dir):
|
||||
"""Test extraction with data security filter."""
|
||||
file_paths = [f for f in sample_files if f.is_file()]
|
||||
archive_path = temp_dir / "data_filter_test.tzst"
|
||||
|
||||
# Create archive
|
||||
create_archive(archive_path, file_paths)
|
||||
|
||||
# Extract with data filter (default for security)
|
||||
extract_dir = temp_dir / "data_filtered"
|
||||
with patch("tzst.core.TzstArchive.extract") as mock_extract:
|
||||
extract_archive(archive_path, extract_dir, filter="data")
|
||||
|
||||
# Verify filter was passed
|
||||
call_args = mock_extract.call_args
|
||||
assert call_args[1]["filter"] == "data"
|
||||
|
||||
def test_invalid_filter_raises_error(self, sample_files, temp_dir):
|
||||
"""Test that invalid filters raise appropriate errors."""
|
||||
file_paths = [f for f in sample_files if f.is_file()]
|
||||
archive_path = temp_dir / "invalid_filter_test.tzst"
|
||||
|
||||
# Create archive
|
||||
create_archive(archive_path, file_paths)
|
||||
|
||||
# Try to extract with invalid filter
|
||||
extract_dir = temp_dir / "invalid_filtered"
|
||||
with pytest.raises(ValueError):
|
||||
extract_archive(archive_path, extract_dir, filter="invalid")
|
||||
|
||||
|
||||
class TestCompressionValidation:
|
||||
"""Test compression level validation."""
|
||||
|
||||
def test_compression_level_boundary_values(self, sample_files, temp_dir):
|
||||
"""Test boundary compression level values."""
|
||||
file_paths = [f for f in sample_files if f.is_file()]
|
||||
|
||||
# Test level 1 (minimum valid)
|
||||
archive_path_min = temp_dir / "min_compression.tzst"
|
||||
create_archive(archive_path_min, file_paths, compression_level=1)
|
||||
assert archive_path_min.exists()
|
||||
assert tzst_test_archive(archive_path_min) is True
|
||||
|
||||
# Test level 22 (maximum valid)
|
||||
archive_path_max = temp_dir / "max_compression.tzst"
|
||||
create_archive(archive_path_max, file_paths, compression_level=22)
|
||||
assert archive_path_max.exists()
|
||||
assert tzst_test_archive(archive_path_max) is True
|
||||
|
||||
def test_invalid_compression_levels_raise_error(self, sample_files, temp_dir):
|
||||
"""Test that invalid compression levels raise appropriate errors."""
|
||||
file_paths = [f for f in sample_files if f.is_file()]
|
||||
archive_path = temp_dir / "invalid_compression.tzst"
|
||||
|
||||
# Test invalid levels that should raise ValueError
|
||||
for invalid_level in [0, 23, -1, 100]:
|
||||
with pytest.raises(ValueError):
|
||||
create_archive(
|
||||
archive_path, file_paths, compression_level=invalid_level
|
||||
)
|
||||
|
||||
def test_specific_compression_level_validation(self, temp_dir):
|
||||
"""Test specific compression level validation to cover missing lines."""
|
||||
import tempfile
|
||||
from pathlib import Path
|
||||
|
||||
with tempfile.NamedTemporaryFile(suffix=".tzst", delete=False) as f:
|
||||
archive_path = Path(f.name)
|
||||
|
||||
try:
|
||||
# Test compression level too low (line 60)
|
||||
with pytest.raises(ValueError, match="Invalid compression level '0'"):
|
||||
TzstArchive(archive_path, mode="w", compression_level=0)
|
||||
|
||||
# Test compression level too high (line 66)
|
||||
with pytest.raises(ValueError, match="Invalid compression level '23'"):
|
||||
TzstArchive(archive_path, mode="w", compression_level=23)
|
||||
|
||||
finally:
|
||||
archive_path.unlink(missing_ok=True)
|
||||
|
||||
def test_invalid_mode_validation(self, temp_dir):
|
||||
"""Test invalid mode validation to cover missing lines."""
|
||||
import tempfile
|
||||
from pathlib import Path
|
||||
|
||||
with tempfile.NamedTemporaryFile(suffix=".tzst", delete=False) as f:
|
||||
archive_path = Path(f.name)
|
||||
|
||||
try:
|
||||
# Test invalid mode (line 54)
|
||||
with pytest.raises(ValueError, match="Invalid mode 'x'"):
|
||||
TzstArchive(archive_path, mode="x")
|
||||
|
||||
# Test invalid mode with additional characters
|
||||
with pytest.raises(ValueError, match="Invalid mode 'rb'"):
|
||||
TzstArchive(archive_path, mode="rb")
|
||||
|
||||
finally:
|
||||
archive_path.unlink(missing_ok=True)
|
||||
|
||||
def test_runtime_errors_for_wrong_mode_operations(self, temp_dir):
|
||||
"""Test RuntimeError for operations on wrong mode archives."""
|
||||
import tempfile
|
||||
from pathlib import Path
|
||||
|
||||
with tempfile.NamedTemporaryFile(suffix=".tzst", delete=False) as f:
|
||||
archive_path = Path(f.name)
|
||||
|
||||
try:
|
||||
# Create empty archive first
|
||||
with TzstArchive(archive_path, mode="w") as archive:
|
||||
pass
|
||||
|
||||
# Test read operations on write mode
|
||||
with TzstArchive(archive_path, mode="w") as archive:
|
||||
with pytest.raises(RuntimeError, match="Archive not open for reading"):
|
||||
archive.getmembers()
|
||||
|
||||
with pytest.raises(RuntimeError, match="Archive not open for reading"):
|
||||
archive.getnames()
|
||||
|
||||
with pytest.raises(RuntimeError, match="Archive not open for reading"):
|
||||
archive.extractfile("test")
|
||||
|
||||
finally:
|
||||
archive_path.unlink(missing_ok=True)
|
||||
|
||||
def test_operations_on_closed_archive(self, temp_dir):
|
||||
"""Test operations on closed archive to cover missing lines."""
|
||||
import tempfile
|
||||
from pathlib import Path
|
||||
|
||||
with tempfile.NamedTemporaryFile(suffix=".tzst", delete=False) as f:
|
||||
archive_path = Path(f.name)
|
||||
|
||||
try:
|
||||
# Create and close archive
|
||||
archive = TzstArchive(archive_path, mode="w")
|
||||
archive.close()
|
||||
|
||||
# Test operations on closed archive
|
||||
with pytest.raises(RuntimeError, match="Archive not open"):
|
||||
archive.getmembers()
|
||||
|
||||
with pytest.raises(RuntimeError, match="Archive not open"):
|
||||
archive.getnames()
|
||||
|
||||
with pytest.raises(RuntimeError, match="Archive not open"):
|
||||
archive.extractfile("test")
|
||||
|
||||
finally:
|
||||
archive_path.unlink(missing_ok=True)
|
||||
|
||||
def test_close_error_handling(self, temp_dir):
|
||||
"""Test error handling in close method."""
|
||||
from pathlib import Path
|
||||
from unittest.mock import MagicMock
|
||||
|
||||
# Create a mock archive that will raise exceptions during close
|
||||
archive = TzstArchive.__new__(TzstArchive)
|
||||
archive.path = Path("test.tzst")
|
||||
archive.mode = "w"
|
||||
archive.compression_level = 3
|
||||
|
||||
# Create mock objects that raise exceptions when closed
|
||||
mock_tarfile = MagicMock()
|
||||
mock_tarfile.close.side_effect = Exception("Mock tarfile close error")
|
||||
|
||||
mock_stream = MagicMock()
|
||||
mock_stream.close.side_effect = Exception("Mock stream close error")
|
||||
|
||||
mock_fileobj = MagicMock()
|
||||
mock_fileobj.close.side_effect = Exception("Mock fileobj close error")
|
||||
|
||||
archive._tarfile = mock_tarfile
|
||||
archive._compressed_stream = mock_stream
|
||||
archive._fileobj = mock_fileobj
|
||||
|
||||
# This should not raise an exception despite the mock exceptions
|
||||
archive.close()
|
||||
|
||||
# Verify all close methods were called
|
||||
mock_tarfile.close.assert_called_once()
|
||||
mock_stream.close.assert_called_once()
|
||||
mock_fileobj.close.assert_called_once()
|
||||
|
||||
|
||||
class TestSpecialFileTypes:
|
||||
"""Test handling of special file types and edge cases."""
|
||||
|
||||
def test_empty_files(self, temp_dir):
|
||||
"""Test archiving and extracting empty files."""
|
||||
empty_file = temp_dir / "empty.txt"
|
||||
empty_file.touch()
|
||||
|
||||
archive_path = temp_dir / "empty_file.tzst"
|
||||
create_archive(archive_path, [empty_file])
|
||||
|
||||
assert archive_path.exists()
|
||||
assert tzst_test_archive(archive_path) is True
|
||||
|
||||
# Test extraction
|
||||
extract_dir = temp_dir / "empty_extracted"
|
||||
extract_archive(archive_path, extract_dir)
|
||||
|
||||
extracted_file = extract_dir / "empty.txt"
|
||||
assert extracted_file.exists()
|
||||
assert extracted_file.stat().st_size == 0
|
||||
|
||||
def test_binary_files(self, temp_dir):
|
||||
"""Test archiving binary files."""
|
||||
binary_file = temp_dir / "binary.bin"
|
||||
binary_content = bytes(range(256)) * 100 # 25.6KB of binary data
|
||||
binary_file.write_bytes(binary_content)
|
||||
|
||||
archive_path = temp_dir / "binary_file.tzst"
|
||||
create_archive(archive_path, [binary_file])
|
||||
|
||||
assert archive_path.exists()
|
||||
assert tzst_test_archive(archive_path) is True
|
||||
|
||||
# Test extraction
|
||||
extract_dir = temp_dir / "binary_extracted"
|
||||
extract_archive(archive_path, extract_dir)
|
||||
|
||||
extracted_file = extract_dir / "binary.bin"
|
||||
assert extracted_file.exists()
|
||||
extracted_content = extracted_file.read_bytes()
|
||||
assert extracted_content == binary_content
|
||||
|
||||
def test_files_with_special_characters(self, temp_dir):
|
||||
"""Test files with special characters in names."""
|
||||
special_chars_file = temp_dir / "file!@#$%^&()_+{}[]-',.txt"
|
||||
special_chars_file.write_text("Special characters content")
|
||||
|
||||
archive_path = temp_dir / "special_chars.tzst"
|
||||
create_archive(archive_path, [special_chars_file])
|
||||
|
||||
assert archive_path.exists()
|
||||
assert tzst_test_archive(archive_path) is True
|
||||
|
||||
|
||||
class TestAppendModeDocumentation:
|
||||
"""Test that append mode provides helpful error messages."""
|
||||
|
||||
def test_append_mode_error_message(self, temp_dir):
|
||||
"""Test that append mode raises informative error."""
|
||||
archive_path = temp_dir / "append_test.tzst"
|
||||
|
||||
with pytest.raises(NotImplementedError) as exc_info:
|
||||
TzstArchive(archive_path, "a")
|
||||
|
||||
error_msg = str(exc_info.value)
|
||||
assert "append mode" in error_msg.lower()
|
||||
assert "alternatives" in error_msg.lower() or "alternative" in error_msg.lower()
|
||||
assert "decompressing" in error_msg.lower()
|
||||
assert "recompressing" in error_msg.lower()
|
||||
|
||||
def test_append_mode_error_in_open(self, temp_dir):
|
||||
"""Test append mode error when opening existing archive."""
|
||||
archive_path = temp_dir / "append_open_test.tzst"
|
||||
|
||||
# Create an archive first
|
||||
with TzstArchive(archive_path, "w"):
|
||||
pass
|
||||
|
||||
# Try to open in append mode
|
||||
with pytest.raises(NotImplementedError) as exc_info:
|
||||
TzstArchive(archive_path, "a")
|
||||
|
||||
error_msg = str(exc_info.value)
|
||||
assert (
|
||||
"multiple archives" in error_msg.lower() or "recreate" in error_msg.lower()
|
||||
)
|
||||
|
||||
|
||||
class TestSpecificMissingLineCoverage:
|
||||
"""Test specific missing lines from coverage report."""
|
||||
|
||||
def test_invalid_mode_validation_specific(self, temp_dir):
|
||||
"""Test specific invalid mode validation to cover missing lines."""
|
||||
import tempfile
|
||||
from pathlib import Path
|
||||
|
||||
with tempfile.NamedTemporaryFile(suffix=".tzst", delete=False) as f:
|
||||
archive_path = Path(f.name)
|
||||
|
||||
try:
|
||||
# Test invalid mode (line 54)
|
||||
with pytest.raises(ValueError, match="Invalid mode 'x'"):
|
||||
TzstArchive(archive_path, mode="x")
|
||||
|
||||
# Test invalid mode with additional characters
|
||||
with pytest.raises(ValueError, match="Invalid mode 'rb'"):
|
||||
TzstArchive(archive_path, mode="rb")
|
||||
|
||||
finally:
|
||||
archive_path.unlink(missing_ok=True)
|
||||
|
||||
def test_compression_level_validation_specific(self, temp_dir):
|
||||
"""Test specific compression level validation to cover missing lines."""
|
||||
import tempfile
|
||||
from pathlib import Path
|
||||
|
||||
with tempfile.NamedTemporaryFile(suffix=".tzst", delete=False) as f:
|
||||
archive_path = Path(f.name)
|
||||
|
||||
try:
|
||||
# Test compression level too low (line 60)
|
||||
with pytest.raises(ValueError, match="Invalid compression level '0'"):
|
||||
TzstArchive(archive_path, mode="w", compression_level=0)
|
||||
|
||||
# Test compression level too high (line 66)
|
||||
with pytest.raises(ValueError, match="Invalid compression level '23'"):
|
||||
TzstArchive(archive_path, mode="w", compression_level=23)
|
||||
|
||||
finally:
|
||||
archive_path.unlink(missing_ok=True)
|
||||
|
||||
def test_runtime_errors_for_wrong_mode_operations(self, temp_dir):
|
||||
"""Test RuntimeError for operations on wrong mode archives."""
|
||||
import tempfile
|
||||
from pathlib import Path
|
||||
|
||||
with tempfile.NamedTemporaryFile(suffix=".tzst", delete=False) as f:
|
||||
archive_path = Path(f.name)
|
||||
|
||||
try:
|
||||
# Create empty archive first
|
||||
with TzstArchive(archive_path, mode="w") as archive:
|
||||
pass
|
||||
|
||||
# Test read operations on write mode
|
||||
with TzstArchive(archive_path, mode="w") as archive:
|
||||
with pytest.raises(RuntimeError, match="Archive not open for reading"):
|
||||
archive.getmembers()
|
||||
|
||||
with pytest.raises(RuntimeError, match="Archive not open for reading"):
|
||||
archive.getnames()
|
||||
|
||||
with pytest.raises(RuntimeError, match="Archive not open for reading"):
|
||||
archive.extractfile("test")
|
||||
|
||||
finally:
|
||||
archive_path.unlink(missing_ok=True)
|
||||
|
||||
def test_operations_on_closed_archive_specific(self, temp_dir):
|
||||
"""Test operations on closed archive to cover missing lines."""
|
||||
import tempfile
|
||||
from pathlib import Path
|
||||
|
||||
with tempfile.NamedTemporaryFile(suffix=".tzst", delete=False) as f:
|
||||
archive_path = Path(f.name)
|
||||
|
||||
try:
|
||||
# Create and close archive
|
||||
archive = TzstArchive(archive_path, mode="w")
|
||||
archive.close()
|
||||
|
||||
# Test operations on closed archive
|
||||
with pytest.raises(RuntimeError, match="Archive not open"):
|
||||
archive.getmembers()
|
||||
|
||||
with pytest.raises(RuntimeError, match="Archive not open"):
|
||||
archive.getnames()
|
||||
|
||||
with pytest.raises(RuntimeError, match="Archive not open"):
|
||||
archive.extractfile("test")
|
||||
|
||||
finally:
|
||||
archive_path.unlink(missing_ok=True)
|
||||
Reference in new issue
Block a user