Initial commit
This commit is contained in:
commit
7bf2b23738
10 files changed
+945
No files matched your search
+78
@@ -0,0 +1,78 @@
|
|||||||
|
# Downloaded ISO files
|
||||||
|
isos/
|
||||||
|
*.iso
|
||||||
|
|
||||||
|
# Python
|
||||||
|
__pycache__/
|
||||||
|
*.py[cod]
|
||||||
|
*$py.class
|
||||||
|
*.so
|
||||||
|
.Python
|
||||||
|
build/
|
||||||
|
develop-eggs/
|
||||||
|
dist/
|
||||||
|
downloads/
|
||||||
|
eggs/
|
||||||
|
.eggs/
|
||||||
|
lib/
|
||||||
|
lib64/
|
||||||
|
parts/
|
||||||
|
sdist/
|
||||||
|
var/
|
||||||
|
wheels/
|
||||||
|
share/python-wheels/
|
||||||
|
*.egg-info/
|
||||||
|
.installed.cfg
|
||||||
|
*.egg
|
||||||
|
MANIFEST
|
||||||
|
|
||||||
|
# PyInstaller
|
||||||
|
*.manifest
|
||||||
|
*.spec
|
||||||
|
|
||||||
|
# Installer logs
|
||||||
|
pip-log.txt
|
||||||
|
pip-delete-this-directory.txt
|
||||||
|
|
||||||
|
# Unit test / coverage reports
|
||||||
|
htmlcov/
|
||||||
|
.tox/
|
||||||
|
.nox/
|
||||||
|
.coverage
|
||||||
|
.coverage.*
|
||||||
|
.cache
|
||||||
|
nosetests.xml
|
||||||
|
coverage.xml
|
||||||
|
*.cover
|
||||||
|
*.py,cover
|
||||||
|
.hypothesis/
|
||||||
|
.pytest_cache/
|
||||||
|
cover/
|
||||||
|
|
||||||
|
# Environments
|
||||||
|
.env
|
||||||
|
.venv
|
||||||
|
env/
|
||||||
|
venv/
|
||||||
|
ENV/
|
||||||
|
env.bak/
|
||||||
|
venv.bak/
|
||||||
|
|
||||||
|
# IDE
|
||||||
|
.vscode/
|
||||||
|
.idea/
|
||||||
|
*.swp
|
||||||
|
*.swo
|
||||||
|
*~
|
||||||
|
|
||||||
|
# OS
|
||||||
|
.DS_Store
|
||||||
|
.DS_Store?
|
||||||
|
._*
|
||||||
|
.Spotlight-V100
|
||||||
|
.Trashes
|
||||||
|
ehthumbs.db
|
||||||
|
Thumbs.db
|
||||||
|
|
||||||
|
# Logs
|
||||||
|
*.log
|
||||||
@@ -0,0 +1,21 @@
|
|||||||
|
MIT License
|
||||||
|
|
||||||
|
Copyright (c) 2025 Xi Xu
|
||||||
|
|
||||||
|
Permission is hereby granted, free of charge, to any person obtaining a copy
|
||||||
|
of this software and associated documentation files (the "Software"), to deal
|
||||||
|
in the Software without restriction, including without limitation the rights
|
||||||
|
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
||||||
|
copies of the Software, and to permit persons to whom the Software is
|
||||||
|
furnished to do so, subject to the following conditions:
|
||||||
|
|
||||||
|
The above copyright notice and this permission notice shall be included in all
|
||||||
|
copies or substantial portions of the Software.
|
||||||
|
|
||||||
|
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
||||||
|
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
||||||
|
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
||||||
|
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
||||||
|
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
||||||
|
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
||||||
|
SOFTWARE.
|
||||||
@@ -0,0 +1,194 @@
|
|||||||
|
# Linux ISO Downloader
|
||||||
|
|
||||||
|
A Python utility for downloading Linux distribution ISO files in parallel with resume capability, progress tracking, and robust error handling.
|
||||||
|
|
||||||
|
## 🚀 Quick Start
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Clone the repository
|
||||||
|
git clone https://github.com/your-username/linux-iso.git
|
||||||
|
cd linux-iso
|
||||||
|
|
||||||
|
# Run the setup script (recommended for first-time users)
|
||||||
|
python setup.py
|
||||||
|
|
||||||
|
# Or install dependencies manually
|
||||||
|
pip install -r requirements.txt
|
||||||
|
|
||||||
|
# Start downloading all Linux ISOs
|
||||||
|
python download_isos.py
|
||||||
|
```
|
||||||
|
|
||||||
|
## ⭐ Features
|
||||||
|
|
||||||
|
- **⚡ Parallel Downloads**: Download multiple ISO files simultaneously with configurable worker threads
|
||||||
|
- **🔄 Resume Support**: Automatically resume interrupted downloads
|
||||||
|
- **📊 Progress Tracking**: Real-time progress bars for each download
|
||||||
|
- **🔁 Retry Logic**: Configurable retry attempts with exponential backoff
|
||||||
|
- **🛡️ Error Handling**: Robust error handling with detailed failure reporting
|
||||||
|
- **⚙️ Flexible Configuration**: Command-line options for all parameters
|
||||||
|
|
||||||
|
## 🐧 Supported Distributions
|
||||||
|
|
||||||
|
The included `urls.txt` contains direct links to the latest stable releases of popular Linux distributions:
|
||||||
|
|
||||||
|
| Distribution | Architecture | Version |
|
||||||
|
|--------------|-------------|---------|
|
||||||
|
| AlmaLinux | x86_64 | 10.0 |
|
||||||
|
| Arch Linux | x86_64 | Latest |
|
||||||
|
| CachyOS | x86_64 | Latest |
|
||||||
|
| CentOS | x86_64 | 7.9/8.5 |
|
||||||
|
| Debian | amd64 | 12.11.0 |
|
||||||
|
| EndeavourOS | x86_64 | Latest |
|
||||||
|
| Fedora | x86_64 | 42 |
|
||||||
|
| Kali Linux | amd64 | 2025.2 |
|
||||||
|
| Linux Mint | x86_64 | 22.1 |
|
||||||
|
| Manjaro | x86_64 | 25.0.4 |
|
||||||
|
| openEuler | x86_64 | 24.03-LTS-SP2 |
|
||||||
|
| Pop!_OS | amd64 | 22.04 |
|
||||||
|
| openSUSE | x86_64 | Tumbleweed |
|
||||||
|
| Oracle Linux | x86_64 | 10.0 |
|
||||||
|
| Rocky Linux | x86_64 | 10.0 |
|
||||||
|
| Ubuntu | amd64 | 24.04.2 |
|
||||||
|
|
||||||
|
## 📦 Installation
|
||||||
|
|
||||||
|
### Prerequisites
|
||||||
|
|
||||||
|
- Python 3.6 or higher
|
||||||
|
- pip package manager
|
||||||
|
|
||||||
|
### Install Dependencies
|
||||||
|
|
||||||
|
```bash
|
||||||
|
pip install -r requirements.txt
|
||||||
|
```
|
||||||
|
|
||||||
|
## Usage
|
||||||
|
|
||||||
|
### Basic Usage
|
||||||
|
|
||||||
|
Download all ISOs from the default `urls.txt` file:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
python download_isos.py
|
||||||
|
```
|
||||||
|
|
||||||
|
### Advanced Usage
|
||||||
|
|
||||||
|
```bash
|
||||||
|
python download_isos.py --urls-file custom_urls.txt --output-dir /path/to/isos --max-workers 10 --retries 3
|
||||||
|
```
|
||||||
|
|
||||||
|
### Command Line Options
|
||||||
|
|
||||||
|
- `--urls-file`: File containing URLs to download (default: `urls.txt`)
|
||||||
|
- `--output-dir`: Directory to save downloaded files (default: `./isos`)
|
||||||
|
- `--max-workers`: Maximum number of parallel downloads (default: 5)
|
||||||
|
- `--retries`: Number of retry attempts per file (default: 5)
|
||||||
|
- `--chunk-size`: Chunk size for downloading in bytes (default: 8192)
|
||||||
|
|
||||||
|
### Help
|
||||||
|
|
||||||
|
```bash
|
||||||
|
python download_isos.py --help
|
||||||
|
```
|
||||||
|
|
||||||
|
## URL File Format
|
||||||
|
|
||||||
|
The URL file should contain one URL per line. Empty lines are ignored. Example:
|
||||||
|
|
||||||
|
```text
|
||||||
|
https://releases.ubuntu.com/24.04/ubuntu-24.04.2-desktop-amd64.iso
|
||||||
|
https://download.fedoraproject.org/pub/fedora/linux/releases/42/Workstation/x86_64/iso/Fedora-Workstation-Live-42-1.1.x86_64.iso
|
||||||
|
```
|
||||||
|
|
||||||
|
## Examples
|
||||||
|
|
||||||
|
### Download with Custom Settings
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Use 8 parallel workers and save to custom directory
|
||||||
|
python download_isos.py --max-workers 8 --output-dir ~/Downloads/linux-isos
|
||||||
|
|
||||||
|
# Use custom URL file with different retry settings
|
||||||
|
python download_isos.py --urls-file my_distros.txt --retries 10
|
||||||
|
```
|
||||||
|
|
||||||
|
### Download Single Distribution
|
||||||
|
|
||||||
|
Create a custom URL file with just one distribution:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
echo "https://releases.ubuntu.com/24.04/ubuntu-24.04.2-desktop-amd64.iso" > ubuntu.txt
|
||||||
|
python download_isos.py --urls-file ubuntu.txt
|
||||||
|
```
|
||||||
|
|
||||||
|
## 📚 Examples
|
||||||
|
|
||||||
|
The `examples/` directory contains sample scripts demonstrating various usage patterns:
|
||||||
|
|
||||||
|
- **`download_specific.py`** - Download only specific distributions (Ubuntu + Fedora)
|
||||||
|
- **`create_custom_categories.py`** - Create and use custom distribution categories
|
||||||
|
|
||||||
|
See the [examples README](examples/README.md) for detailed usage instructions.
|
||||||
|
|
||||||
|
## Output
|
||||||
|
|
||||||
|
The script provides real-time progress information:
|
||||||
|
|
||||||
|
- Individual progress bars for each file
|
||||||
|
- Download speed and ETA
|
||||||
|
- Success/failure summary
|
||||||
|
- Detailed error reporting for failed downloads
|
||||||
|
|
||||||
|
## Error Handling
|
||||||
|
|
||||||
|
- **Network Issues**: Automatic retry with exponential backoff
|
||||||
|
- **Interrupted Downloads**: Resume from where it left off
|
||||||
|
- **Invalid URLs**: Detailed error messages
|
||||||
|
- **Disk Space**: Handles low disk space gracefully
|
||||||
|
- **Permissions**: Clear error messages for permission issues
|
||||||
|
|
||||||
|
## Performance Tips
|
||||||
|
|
||||||
|
1. **Optimal Worker Count**: Start with 5 workers and adjust based on your network and system performance
|
||||||
|
2. **Chunk Size**: Default 8KB works well for most cases; increase for faster networks
|
||||||
|
3. **Retry Attempts**: Increase for unreliable connections
|
||||||
|
4. **Resume Downloads**: The script automatically resumes interrupted downloads
|
||||||
|
|
||||||
|
## Troubleshooting
|
||||||
|
|
||||||
|
### Common Issues
|
||||||
|
|
||||||
|
**Downloads are slow:**
|
||||||
|
|
||||||
|
- Reduce `--max-workers` if you have limited bandwidth
|
||||||
|
- Check your internet connection
|
||||||
|
- Some mirrors may be slower than others
|
||||||
|
|
||||||
|
**Downloads fail repeatedly:**
|
||||||
|
|
||||||
|
- Increase `--retries` for unreliable connections
|
||||||
|
- Check if the URLs are still valid
|
||||||
|
- Verify you have write permissions to the output directory
|
||||||
|
|
||||||
|
**Out of disk space:**
|
||||||
|
|
||||||
|
- Check available disk space before starting large downloads
|
||||||
|
- Use `--output-dir` to specify a location with more space
|
||||||
|
|
||||||
|
## 🏆 Acknowledgments
|
||||||
|
|
||||||
|
- Thanks to all the Linux distribution maintainers for providing direct download links
|
||||||
|
- Built with Python's `requests` library for HTTP handling
|
||||||
|
- Uses `tqdm` for beautiful progress bars
|
||||||
|
- Implements concurrent downloads with `concurrent.futures`
|
||||||
|
|
||||||
|
## 📄 License
|
||||||
|
|
||||||
|
This project is licensed under the MIT License - see the [LICENSE](LICENSE) file for details.
|
||||||
|
|
||||||
|
## ⚠️ Disclaimer
|
||||||
|
|
||||||
|
This tool downloads files from third-party servers. Please respect the bandwidth and terms of service of the mirror providers. Consider using official torrents or mirrors when available.
|
||||||
@@ -0,0 +1,222 @@
|
|||||||
|
#!/usr/bin/env python3
|
||||||
|
|
||||||
|
import argparse
|
||||||
|
import concurrent.futures
|
||||||
|
import os
|
||||||
|
import sys
|
||||||
|
import time
|
||||||
|
from pathlib import Path
|
||||||
|
from urllib.parse import urlparse
|
||||||
|
|
||||||
|
import requests
|
||||||
|
from tqdm import tqdm
|
||||||
|
|
||||||
|
|
||||||
|
class ISODownloader:
|
||||||
|
def __init__(self, output_dir, max_workers=5, retry_attempts=3, chunk_size=8192):
|
||||||
|
self.output_dir = Path(output_dir)
|
||||||
|
self.max_workers = max_workers
|
||||||
|
self.retry_attempts = retry_attempts
|
||||||
|
self.chunk_size = chunk_size
|
||||||
|
self.session = requests.Session()
|
||||||
|
# Configure session with longer timeouts
|
||||||
|
self.session.mount(
|
||||||
|
"https://",
|
||||||
|
requests.adapters.HTTPAdapter(
|
||||||
|
max_retries=3, pool_connections=max_workers, pool_maxsize=max_workers
|
||||||
|
),
|
||||||
|
)
|
||||||
|
self.session.mount(
|
||||||
|
"http://",
|
||||||
|
requests.adapters.HTTPAdapter(
|
||||||
|
max_retries=3, pool_connections=max_workers, pool_maxsize=max_workers
|
||||||
|
),
|
||||||
|
)
|
||||||
|
|
||||||
|
def download_file(self, url):
|
||||||
|
"""Download a single file with retry capability and progress tracking"""
|
||||||
|
file_name = os.path.basename(urlparse(url).path)
|
||||||
|
file_path = self.output_dir / file_name
|
||||||
|
|
||||||
|
# Create output directory if it doesn't exist
|
||||||
|
self.output_dir.mkdir(parents=True, exist_ok=True)
|
||||||
|
|
||||||
|
# Check if file already exists and get its size
|
||||||
|
file_size = file_path.stat().st_size if file_path.exists() else 0
|
||||||
|
|
||||||
|
for attempt in range(1, self.retry_attempts + 1):
|
||||||
|
try:
|
||||||
|
# Setup the request with resume capability if file exists
|
||||||
|
headers = {}
|
||||||
|
if file_size > 0:
|
||||||
|
headers["Range"] = f"bytes={file_size}-"
|
||||||
|
|
||||||
|
# Make the request with a timeout
|
||||||
|
response = self.session.get(
|
||||||
|
url, headers=headers, stream=True, timeout=30, allow_redirects=True
|
||||||
|
)
|
||||||
|
|
||||||
|
if (
|
||||||
|
response.status_code == 416
|
||||||
|
): # Range not satisfiable, file may be complete
|
||||||
|
print(f"File {file_name} appears to be complete, skipping.")
|
||||||
|
return True
|
||||||
|
|
||||||
|
# Handle non-successful status codes
|
||||||
|
response.raise_for_status()
|
||||||
|
|
||||||
|
# Get total file size
|
||||||
|
total_size = int(response.headers.get("content-length", 0))
|
||||||
|
if file_size > 0 and response.status_code == 206: # Partial content
|
||||||
|
total_size += file_size
|
||||||
|
else:
|
||||||
|
# If not a resume, start from scratch
|
||||||
|
file_size = 0
|
||||||
|
|
||||||
|
# Setup progress bar
|
||||||
|
desc = f"[Attempt {attempt}] {file_name}"
|
||||||
|
with tqdm(
|
||||||
|
total=total_size,
|
||||||
|
initial=file_size,
|
||||||
|
unit="B",
|
||||||
|
unit_scale=True,
|
||||||
|
desc=desc,
|
||||||
|
ncols=100,
|
||||||
|
) as pbar:
|
||||||
|
# Write to file
|
||||||
|
mode = "ab" if file_size > 0 else "wb"
|
||||||
|
with open(file_path, mode) as f:
|
||||||
|
for chunk in response.iter_content(chunk_size=self.chunk_size):
|
||||||
|
if chunk: # filter out keep-alive chunks
|
||||||
|
f.write(chunk)
|
||||||
|
pbar.update(len(chunk))
|
||||||
|
|
||||||
|
return True
|
||||||
|
|
||||||
|
except (OSError, requests.exceptions.RequestException) as e:
|
||||||
|
if attempt < self.retry_attempts:
|
||||||
|
wait_time = 2**attempt # Exponential backoff
|
||||||
|
print(f"Error downloading {file_name}: {e}")
|
||||||
|
print(
|
||||||
|
f"Retrying in {wait_time} seconds... (Attempt {attempt}/{self.retry_attempts})"
|
||||||
|
)
|
||||||
|
time.sleep(wait_time)
|
||||||
|
else:
|
||||||
|
print(
|
||||||
|
f"Failed to download {file_name} after {self.retry_attempts} attempts: {e}"
|
||||||
|
)
|
||||||
|
return False
|
||||||
|
|
||||||
|
def download_all(self, urls):
|
||||||
|
"""Download all files in parallel"""
|
||||||
|
results = []
|
||||||
|
successful = 0
|
||||||
|
failed = 0
|
||||||
|
|
||||||
|
print(f"Starting downloads with {self.max_workers} parallel workers")
|
||||||
|
|
||||||
|
with concurrent.futures.ThreadPoolExecutor(
|
||||||
|
max_workers=self.max_workers
|
||||||
|
) as executor:
|
||||||
|
# Submit all download tasks
|
||||||
|
future_to_url = {
|
||||||
|
executor.submit(self.download_file, url): url for url in urls
|
||||||
|
}
|
||||||
|
|
||||||
|
# Process results as they complete
|
||||||
|
for future in concurrent.futures.as_completed(future_to_url):
|
||||||
|
url = future_to_url[future]
|
||||||
|
file_name = os.path.basename(urlparse(url).path)
|
||||||
|
|
||||||
|
try:
|
||||||
|
if future.result():
|
||||||
|
successful += 1
|
||||||
|
else:
|
||||||
|
failed += 1
|
||||||
|
results.append(f"Failed: {file_name}")
|
||||||
|
except Exception as e:
|
||||||
|
failed += 1
|
||||||
|
results.append(f"Error with {file_name}: {str(e)}")
|
||||||
|
|
||||||
|
print("\nDownload Summary:")
|
||||||
|
print(f" Successful: {successful}")
|
||||||
|
print(f" Failed: {failed}")
|
||||||
|
|
||||||
|
if failed > 0:
|
||||||
|
print("\nFailed Downloads:")
|
||||||
|
for result in results:
|
||||||
|
print(f" - {result}")
|
||||||
|
|
||||||
|
return successful, failed, results
|
||||||
|
|
||||||
|
|
||||||
|
def main():
|
||||||
|
parser = argparse.ArgumentParser(description="Download ISO files in parallel.")
|
||||||
|
parser.add_argument(
|
||||||
|
"--urls-file",
|
||||||
|
default="urls.txt",
|
||||||
|
help="File containing URLs to download (default: urls.txt)",
|
||||||
|
)
|
||||||
|
parser.add_argument(
|
||||||
|
"--output-dir",
|
||||||
|
default="./isos",
|
||||||
|
help="Directory to save downloaded files (default: ./isos)",
|
||||||
|
)
|
||||||
|
parser.add_argument(
|
||||||
|
"--max-workers",
|
||||||
|
type=int,
|
||||||
|
default=5,
|
||||||
|
help="Maximum number of parallel downloads (default: 5)",
|
||||||
|
)
|
||||||
|
parser.add_argument(
|
||||||
|
"--retries",
|
||||||
|
type=int,
|
||||||
|
default=5,
|
||||||
|
help="Number of retry attempts per file (default: 5)",
|
||||||
|
)
|
||||||
|
parser.add_argument(
|
||||||
|
"--chunk-size",
|
||||||
|
type=int,
|
||||||
|
default=8192,
|
||||||
|
help="Chunk size for downloading in bytes (default: 8192)",
|
||||||
|
)
|
||||||
|
|
||||||
|
args = parser.parse_args()
|
||||||
|
|
||||||
|
# Read URLs from file
|
||||||
|
try:
|
||||||
|
with open(args.urls_file) as f:
|
||||||
|
urls = [line.strip() for line in f if line.strip()]
|
||||||
|
except OSError as e:
|
||||||
|
print(f"Error reading URLs file: {e}")
|
||||||
|
sys.exit(1)
|
||||||
|
|
||||||
|
if not urls:
|
||||||
|
print("No URLs found in the file.")
|
||||||
|
sys.exit(1)
|
||||||
|
|
||||||
|
print(f"Found {len(urls)} URLs to download.")
|
||||||
|
|
||||||
|
# Initialize downloader
|
||||||
|
downloader = ISODownloader(
|
||||||
|
args.output_dir,
|
||||||
|
max_workers=args.max_workers,
|
||||||
|
retry_attempts=args.retries,
|
||||||
|
chunk_size=args.chunk_size,
|
||||||
|
)
|
||||||
|
|
||||||
|
# Start the download
|
||||||
|
start_time = time.time()
|
||||||
|
successful, failed, results = downloader.download_all(urls)
|
||||||
|
elapsed_time = time.time() - start_time
|
||||||
|
|
||||||
|
# Print summary
|
||||||
|
print(f"\nDownload process completed in {elapsed_time:.2f} seconds.")
|
||||||
|
print(f"Files saved to: {os.path.abspath(args.output_dir)}")
|
||||||
|
|
||||||
|
if failed > 0:
|
||||||
|
sys.exit(1)
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
main()
|
||||||
@@ -0,0 +1,101 @@
|
|||||||
|
# Examples
|
||||||
|
|
||||||
|
This directory contains example scripts demonstrating various ways to use the Linux ISO downloader.
|
||||||
|
|
||||||
|
## Available Examples
|
||||||
|
|
||||||
|
### 1. `download_specific.py`
|
||||||
|
|
||||||
|
Download only specific distributions (Ubuntu and Fedora in this example).
|
||||||
|
|
||||||
|
```bash
|
||||||
|
python examples/download_specific.py
|
||||||
|
```
|
||||||
|
|
||||||
|
**Features:**
|
||||||
|
|
||||||
|
- Downloads only Ubuntu and Fedora ISOs
|
||||||
|
- Uses fewer parallel workers (2) for demonstration
|
||||||
|
- Saves files to `./examples_output/`
|
||||||
|
|
||||||
|
### 2. `create_custom_categories.py`
|
||||||
|
|
||||||
|
Demonstrates how to create custom URL files for different categories of Linux distributions.
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Create category files and download lightweight distributions
|
||||||
|
python examples/create_custom_categories.py
|
||||||
|
|
||||||
|
# Download enterprise distributions
|
||||||
|
python examples/create_custom_categories.py --enterprise
|
||||||
|
|
||||||
|
# Show help
|
||||||
|
python examples/create_custom_categories.py --help
|
||||||
|
```
|
||||||
|
|
||||||
|
**Features:**
|
||||||
|
|
||||||
|
- Creates custom URL files for different distribution categories
|
||||||
|
- Lightweight distributions (Arch-based): Arch Linux, EndeavourOS, Manjaro
|
||||||
|
- Enterprise distributions (RHEL-based): AlmaLinux, Rocky Linux, Oracle Linux
|
||||||
|
- Organizes downloads into category-specific subdirectories
|
||||||
|
|
||||||
|
## Output Structure
|
||||||
|
|
||||||
|
When running examples, files are organized as follows:
|
||||||
|
|
||||||
|
```text
|
||||||
|
examples_output/
|
||||||
|
├── lightweight/ # From create_custom_categories.py
|
||||||
|
│ ├── archlinux-x86_64.iso
|
||||||
|
│ ├── EndeavourOS-Mercury-Neo-2025.03.19.iso
|
||||||
|
│ └── manjaro-gnome-25.0.4-250623-linux612.iso
|
||||||
|
├── enterprise/ # From create_custom_categories.py --enterprise
|
||||||
|
│ ├── AlmaLinux-10.0-x86_64-dvd.iso
|
||||||
|
│ ├── Rocky-10.0-x86_64-dvd1.iso
|
||||||
|
│ └── OracleLinux-R10-U0-x86_64-dvd.iso
|
||||||
|
└── ubuntu-24.04.2-desktop-amd64.iso # From download_specific.py
|
||||||
|
└── Fedora-Workstation-Live-42-1.1.x86_64.iso
|
||||||
|
```
|
||||||
|
|
||||||
|
## Custom URL Files
|
||||||
|
|
||||||
|
The `create_custom_categories.py` script generates these URL files:
|
||||||
|
|
||||||
|
- `examples/lightweight_distros.txt` - Arch-based distributions
|
||||||
|
- `examples/enterprise_distros.txt` - Enterprise Linux distributions
|
||||||
|
|
||||||
|
You can modify these files or create your own custom URL files to download specific sets of distributions.
|
||||||
|
|
||||||
|
## Tips
|
||||||
|
|
||||||
|
1. **Start Small**: Begin with the `download_specific.py` example to understand the basic usage
|
||||||
|
2. **Custom Categories**: Use `create_custom_categories.py` as a template for your own categorization needs
|
||||||
|
3. **Modify URLs**: Edit the generated URL files in the `examples/` directory to customize which distributions to download
|
||||||
|
4. **Output Directories**: Each example uses different output directories to keep downloads organized
|
||||||
|
|
||||||
|
## Creating Your Own Examples
|
||||||
|
|
||||||
|
Use these examples as templates to create your own custom download scripts:
|
||||||
|
|
||||||
|
1. Import the `ISODownloader` class from `download_isos`
|
||||||
|
2. Define your list of URLs
|
||||||
|
3. Create a downloader instance with your preferred settings
|
||||||
|
4. Call `download_all()` with your URLs
|
||||||
|
|
||||||
|
```python
|
||||||
|
from download_isos import ISODownloader
|
||||||
|
|
||||||
|
# Your custom URLs
|
||||||
|
urls = ["https://example.com/distro.iso"]
|
||||||
|
|
||||||
|
# Create downloader
|
||||||
|
downloader = ISODownloader(
|
||||||
|
output_dir="./my_downloads",
|
||||||
|
max_workers=3,
|
||||||
|
retry_attempts=5
|
||||||
|
)
|
||||||
|
|
||||||
|
# Download
|
||||||
|
successful, failed, results = downloader.download_all(urls)
|
||||||
|
```
|
||||||
@@ -0,0 +1,115 @@
|
|||||||
|
#!/usr/bin/env python3
|
||||||
|
"""
|
||||||
|
Example script demonstrating how to create custom URL files and download
|
||||||
|
specific sets of distributions.
|
||||||
|
This example shows how to create different categories of Linux distributions.
|
||||||
|
"""
|
||||||
|
|
||||||
|
import os
|
||||||
|
|
||||||
|
from download_isos import ISODownloader
|
||||||
|
|
||||||
|
|
||||||
|
def create_lightweight_distros():
|
||||||
|
"""Create a URL file for lightweight Linux distributions."""
|
||||||
|
lightweight_urls = [
|
||||||
|
"https://geo.mirror.pkgbuild.com/iso/2025.07.01/archlinux-x86_64.iso",
|
||||||
|
"https://mirrors.gigenet.com/endeavouros/iso/EndeavourOS_Mercury-Neo-2025.03.19.iso",
|
||||||
|
"https://download.manjaro.org/gnome/25.0.4/manjaro-gnome-25.0.4-250623-linux612.iso",
|
||||||
|
]
|
||||||
|
|
||||||
|
with open("examples/lightweight_distros.txt", "w") as f:
|
||||||
|
for url in lightweight_urls:
|
||||||
|
f.write(url + "\n")
|
||||||
|
|
||||||
|
print("✅ Created lightweight_distros.txt")
|
||||||
|
|
||||||
|
|
||||||
|
def create_enterprise_distros():
|
||||||
|
"""Create a URL file for enterprise Linux distributions."""
|
||||||
|
enterprise_urls = [
|
||||||
|
"https://repo.almalinux.org/almalinux/10/isos/x86_64/AlmaLinux-10.0-x86_64-dvd.iso",
|
||||||
|
"https://download.rockylinux.org/pub/rocky/10/isos/x86_64/Rocky-10.0-x86_64-dvd1.iso",
|
||||||
|
"https://yum.oracle.com/ISOS/OracleLinux/OL10/u0/x86_64/OracleLinux-R10-U0-x86_64-dvd.iso",
|
||||||
|
]
|
||||||
|
|
||||||
|
with open("examples/enterprise_distros.txt", "w") as f:
|
||||||
|
for url in enterprise_urls:
|
||||||
|
f.write(url + "\n")
|
||||||
|
|
||||||
|
print("✅ Created enterprise_distros.txt")
|
||||||
|
|
||||||
|
|
||||||
|
def download_category(url_file, category_name):
|
||||||
|
"""Download ISOs from a specific category URL file."""
|
||||||
|
print(f"\n🐧 Downloading {category_name} distributions...")
|
||||||
|
|
||||||
|
# Read URLs from the category file
|
||||||
|
try:
|
||||||
|
with open(url_file) as f:
|
||||||
|
urls = [line.strip() for line in f if line.strip()]
|
||||||
|
except FileNotFoundError:
|
||||||
|
print(f"❌ Error: {url_file} not found!")
|
||||||
|
return
|
||||||
|
|
||||||
|
if not urls:
|
||||||
|
print(f"❌ No URLs found in {url_file}")
|
||||||
|
return
|
||||||
|
|
||||||
|
print(f"📋 Found {len(urls)} URLs to download")
|
||||||
|
|
||||||
|
# Create output directory based on category
|
||||||
|
output_dir = f"./examples_output/{category_name.lower().replace(' ', '_')}"
|
||||||
|
|
||||||
|
# Create downloader instance
|
||||||
|
downloader = ISODownloader(output_dir=output_dir, max_workers=3, retry_attempts=3)
|
||||||
|
|
||||||
|
# Download the files
|
||||||
|
successful, failed, results = downloader.download_all(urls)
|
||||||
|
|
||||||
|
print(f"\n✅ Successfully downloaded: {successful} files")
|
||||||
|
if failed > 0:
|
||||||
|
print(f"❌ Failed downloads: {failed}")
|
||||||
|
for result in results:
|
||||||
|
print(f" - {result}")
|
||||||
|
|
||||||
|
print(f"\n📂 Files saved to: {os.path.abspath(output_dir)}")
|
||||||
|
|
||||||
|
|
||||||
|
def main():
|
||||||
|
"""Main function to demonstrate custom URL file creation and usage."""
|
||||||
|
print("🔧 Creating custom URL files for different distribution categories...")
|
||||||
|
|
||||||
|
# Create the examples directory if it doesn't exist
|
||||||
|
os.makedirs("examples", exist_ok=True)
|
||||||
|
|
||||||
|
# Create category-specific URL files
|
||||||
|
create_lightweight_distros()
|
||||||
|
create_enterprise_distros()
|
||||||
|
|
||||||
|
print("\n📋 Available categories:")
|
||||||
|
print(" 1. Lightweight distributions (Arch-based)")
|
||||||
|
print(" 2. Enterprise distributions (RHEL-based)")
|
||||||
|
|
||||||
|
# Example: Download lightweight distributions
|
||||||
|
print("\n" + "=" * 50)
|
||||||
|
download_category("examples/lightweight_distros.txt", "Lightweight")
|
||||||
|
|
||||||
|
print("\n" + "=" * 50)
|
||||||
|
print("💡 To download enterprise distributions, run:")
|
||||||
|
print(" python examples/create_custom_categories.py --enterprise")
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
import sys
|
||||||
|
|
||||||
|
if "--enterprise" in sys.argv:
|
||||||
|
download_category("examples/enterprise_distros.txt", "Enterprise")
|
||||||
|
elif "--help" in sys.argv or "-h" in sys.argv:
|
||||||
|
print("Usage:")
|
||||||
|
print(" python examples/create_custom_categories.py")
|
||||||
|
print(" # Create files and download lightweight")
|
||||||
|
print(" python examples/create_custom_categories.py --enterprise")
|
||||||
|
print(" # Download enterprise distributions")
|
||||||
|
else:
|
||||||
|
main()
|
||||||
@@ -0,0 +1,42 @@
|
|||||||
|
#!/usr/bin/env python3
|
||||||
|
"""
|
||||||
|
Example script demonstrating how to use the ISO downloader for specific distributions.
|
||||||
|
This script shows how to download only Ubuntu and Fedora ISOs.
|
||||||
|
"""
|
||||||
|
|
||||||
|
import os
|
||||||
|
|
||||||
|
from download_isos import ISODownloader
|
||||||
|
|
||||||
|
|
||||||
|
def main():
|
||||||
|
# Define specific URLs for Ubuntu and Fedora
|
||||||
|
ubuntu_fedora_urls = [
|
||||||
|
"https://releases.ubuntu.com/noble/ubuntu-24.04.2-desktop-amd64.iso",
|
||||||
|
"https://download.fedoraproject.org/pub/fedora/linux/releases/42/Workstation/x86_64/iso/Fedora-Workstation-Live-42-1.1.x86_64.iso",
|
||||||
|
]
|
||||||
|
|
||||||
|
print("🐧 Downloading Ubuntu and Fedora ISOs...")
|
||||||
|
print("📁 Output directory: ./examples_output")
|
||||||
|
|
||||||
|
# Create downloader instance
|
||||||
|
downloader = ISODownloader(
|
||||||
|
output_dir="./examples_output",
|
||||||
|
max_workers=2, # Use fewer workers for this example
|
||||||
|
retry_attempts=3,
|
||||||
|
)
|
||||||
|
|
||||||
|
# Download the files
|
||||||
|
successful, failed, results = downloader.download_all(ubuntu_fedora_urls)
|
||||||
|
|
||||||
|
print(f"\n✅ Successfully downloaded: {successful} files")
|
||||||
|
if failed > 0:
|
||||||
|
print(f"❌ Failed downloads: {failed}")
|
||||||
|
for result in results:
|
||||||
|
print(f" - {result}")
|
||||||
|
|
||||||
|
print(f"\n📂 Files saved to: {os.path.abspath('./examples_output')}")
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
main()
|
||||||
@@ -0,0 +1,2 @@
|
|||||||
|
requests>=2.31.0
|
||||||
|
tqdm>=4.66.0
|
||||||
@@ -0,0 +1,153 @@
|
|||||||
|
#!/usr/bin/env python3
|
||||||
|
"""
|
||||||
|
Setup script to help users get started with the Linux ISO downloader.
|
||||||
|
This script checks dependencies and provides initial setup guidance.
|
||||||
|
"""
|
||||||
|
|
||||||
|
import subprocess
|
||||||
|
import sys
|
||||||
|
from pathlib import Path
|
||||||
|
|
||||||
|
|
||||||
|
def check_python_version():
|
||||||
|
"""Check if Python version is compatible."""
|
||||||
|
version = sys.version_info
|
||||||
|
if version.major < 3 or (version.major == 3 and version.minor < 6):
|
||||||
|
print("❌ Python 3.6 or higher is required.")
|
||||||
|
print(f" Current version: Python {version.major}.{version.minor}")
|
||||||
|
return False
|
||||||
|
|
||||||
|
print(f"✅ Python {version.major}.{version.minor}.{version.micro} - Compatible")
|
||||||
|
return True
|
||||||
|
|
||||||
|
|
||||||
|
def check_dependencies():
|
||||||
|
"""Check if required dependencies are installed."""
|
||||||
|
try:
|
||||||
|
import requests
|
||||||
|
|
||||||
|
print(f"✅ requests {requests.__version__} - Installed")
|
||||||
|
except ImportError:
|
||||||
|
print("❌ requests library not found")
|
||||||
|
return False
|
||||||
|
|
||||||
|
try:
|
||||||
|
import tqdm
|
||||||
|
|
||||||
|
print(f"✅ tqdm {tqdm.__version__} - Installed")
|
||||||
|
except ImportError:
|
||||||
|
print("❌ tqdm library not found")
|
||||||
|
return False
|
||||||
|
|
||||||
|
return True
|
||||||
|
|
||||||
|
|
||||||
|
def install_dependencies():
|
||||||
|
"""Install required dependencies."""
|
||||||
|
print("\n🔧 Installing dependencies...")
|
||||||
|
try:
|
||||||
|
subprocess.check_call(
|
||||||
|
[sys.executable, "-m", "pip", "install", "-r", "requirements.txt"]
|
||||||
|
)
|
||||||
|
print("✅ Dependencies installed successfully!")
|
||||||
|
return True
|
||||||
|
except subprocess.CalledProcessError:
|
||||||
|
print("❌ Failed to install dependencies")
|
||||||
|
return False
|
||||||
|
|
||||||
|
|
||||||
|
def create_output_directory():
|
||||||
|
"""Create the default output directory."""
|
||||||
|
output_dir = Path("./isos")
|
||||||
|
if not output_dir.exists():
|
||||||
|
output_dir.mkdir(parents=True, exist_ok=True)
|
||||||
|
print(f"✅ Created output directory: {output_dir.absolute()}")
|
||||||
|
else:
|
||||||
|
print(f"✅ Output directory already exists: {output_dir.absolute()}")
|
||||||
|
|
||||||
|
|
||||||
|
def check_urls_file():
|
||||||
|
"""Check if the URLs file exists."""
|
||||||
|
urls_file = Path("urls.txt")
|
||||||
|
if urls_file.exists():
|
||||||
|
with open(urls_file) as f:
|
||||||
|
urls = [line.strip() for line in f if line.strip()]
|
||||||
|
print(f"✅ Found {len(urls)} URLs in urls.txt")
|
||||||
|
return True
|
||||||
|
else:
|
||||||
|
print("❌ urls.txt file not found")
|
||||||
|
return False
|
||||||
|
|
||||||
|
|
||||||
|
def print_usage_examples():
|
||||||
|
"""Print usage examples."""
|
||||||
|
print("\n" + "=" * 60)
|
||||||
|
print("🚀 QUICK START EXAMPLES")
|
||||||
|
print("=" * 60)
|
||||||
|
|
||||||
|
print("\n1. Download all Linux ISOs:")
|
||||||
|
print(" python download_isos.py")
|
||||||
|
|
||||||
|
print("\n2. Download with custom settings:")
|
||||||
|
print(" python download_isos.py --max-workers 3 --retries 5")
|
||||||
|
|
||||||
|
print("\n3. Download to custom directory:")
|
||||||
|
print(" python download_isos.py --output-dir /path/to/downloads")
|
||||||
|
|
||||||
|
print("\n4. Run example scripts:")
|
||||||
|
print(" python examples/download_specific.py")
|
||||||
|
print(" python examples/create_custom_categories.py")
|
||||||
|
|
||||||
|
print("\n5. Get help:")
|
||||||
|
print(" python download_isos.py --help")
|
||||||
|
|
||||||
|
print("\n" + "=" * 60)
|
||||||
|
print("📋 WHAT'S INCLUDED")
|
||||||
|
print("=" * 60)
|
||||||
|
print("• 17 Linux distributions in urls.txt")
|
||||||
|
print("• Parallel downloading with progress bars")
|
||||||
|
print("• Automatic resume for interrupted downloads")
|
||||||
|
print("• Example scripts in the examples/ directory")
|
||||||
|
print("• Comprehensive error handling and retry logic")
|
||||||
|
|
||||||
|
|
||||||
|
def main():
|
||||||
|
"""Main setup function."""
|
||||||
|
print("🐧 Linux ISO Downloader Setup")
|
||||||
|
print("=" * 40)
|
||||||
|
|
||||||
|
# Check Python version
|
||||||
|
if not check_python_version():
|
||||||
|
sys.exit(1)
|
||||||
|
|
||||||
|
# Check if dependencies are already installed
|
||||||
|
if not check_dependencies():
|
||||||
|
print("\n📦 Installing missing dependencies...")
|
||||||
|
if not install_dependencies():
|
||||||
|
print("\n❌ Setup failed. Please install dependencies manually:")
|
||||||
|
print(" pip install -r requirements.txt")
|
||||||
|
sys.exit(1)
|
||||||
|
|
||||||
|
# Verify installation
|
||||||
|
print("\n🔍 Verifying installation...")
|
||||||
|
if not check_dependencies():
|
||||||
|
print("❌ Dependency verification failed")
|
||||||
|
sys.exit(1)
|
||||||
|
|
||||||
|
# Create output directory
|
||||||
|
print("\n📁 Setting up directories...")
|
||||||
|
create_output_directory()
|
||||||
|
|
||||||
|
# Check URLs file
|
||||||
|
print("\n🔗 Checking URL configuration...")
|
||||||
|
check_urls_file()
|
||||||
|
|
||||||
|
# Print usage examples
|
||||||
|
print_usage_examples()
|
||||||
|
|
||||||
|
print("\n✅ Setup complete! You're ready to download Linux ISOs.")
|
||||||
|
print("\n💡 Tip: Start with 'python download_isos.py --help' to see all options.")
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
main()
|
||||||
@@ -0,0 +1,17 @@
|
|||||||
|
https://repo.almalinux.org/almalinux/10/isos/x86_64/AlmaLinux-10.0-x86_64-dvd.iso
|
||||||
|
https://geo.mirror.pkgbuild.com/iso/2025.07.01/archlinux-x86_64.iso
|
||||||
|
https://cdn77.cachyos.org/ISO/desktop/250530/cachyos-desktop-linux-250530.iso
|
||||||
|
https://vault.centos.org/7.9.2009/isos/x86_64/CentOS-7-x86_64-Everything-2207-02.iso
|
||||||
|
https://vault.centos.org/8.5.2111/isos/x86_64/CentOS-8.5.2111-x86_64-dvd1.iso
|
||||||
|
https://cdimage.debian.org/debian-cd/current/amd64/iso-dvd/debian-12.11.0-amd64-DVD-1.iso
|
||||||
|
https://mirrors.gigenet.com/endeavouros/iso/EndeavourOS_Mercury-Neo-2025.03.19.iso
|
||||||
|
https://download.fedoraproject.org/pub/fedora/linux/releases/42/Workstation/x86_64/iso/Fedora-Workstation-Live-42-1.1.x86_64.iso
|
||||||
|
https://cdimage.kali.org/kali-2025.2/kali-linux-2025.2-installer-amd64.iso
|
||||||
|
https://mirrors.cicku.me/linuxmint/iso/stable/22.1/linuxmint-22.1-cinnamon-64bit.iso
|
||||||
|
https://download.manjaro.org/gnome/25.0.4/manjaro-gnome-25.0.4-250623-linux612.iso
|
||||||
|
https://mirror.nyist.edu.cn/openeuler/openEuler-24.03-LTS-SP2/DevStation/x86_64/openEuler-24.03-LTS-SP2-DevStation-x86_64-dvd.iso
|
||||||
|
https://iso.pop-os.org/22.04/amd64/nvidia/55/pop-os_22.04_amd64_nvidia_55.iso
|
||||||
|
https://download.opensuse.org/tumbleweed/iso/openSUSE-Tumbleweed-DVD-x86_64-Current.iso
|
||||||
|
https://yum.oracle.com/ISOS/OracleLinux/OL10/u0/x86_64/OracleLinux-R10-U0-x86_64-dvd.iso
|
||||||
|
https://download.rockylinux.org/pub/rocky/10/isos/x86_64/Rocky-10.0-x86_64-dvd1.iso
|
||||||
|
https://releases.ubuntu.com/noble/ubuntu-24.04.2-desktop-amd64.iso
|
||||||
Reference in new issue
Block a user