diff --git a/README.md b/README.md index 48067cd..f13e68c 100644 --- a/README.md +++ b/README.md @@ -256,6 +256,7 @@ Using the pre-deployed instance **`xget.xi-xu.me`** or your own deployed instanc | openSUSE | `opensuse` | `https://download.opensuse.org/...` | `https://xget.xi-xu.me/opensuse/...` | | Arch Linux | `arch` | `https://geo.mirror.pkgbuild.com/...` | `https://xget.xi-xu.me/arch/...` | | arXiv | `arxiv` | `https://arxiv.org/...` | `https://xget.xi-xu.me/arxiv/...` | +| Flathub | `flathub` | `https://dl.flathub.org/...` | `https://xget.xi-xu.me/flathub/...` | | F-Droid | `fdroid` | `https://f-droid.org/...` | `https://xget.xi-xu.me/fdroid/...` | | Jenkins Plugins | `jenkins` | `https://updates.jenkins.io/...` | `https://xget.xi-xu.me/jenkins/...` | | Container Registries | `cr` | See [Container Registries](#container-registries) | See [Container Registries](#container-registries) | @@ -707,6 +708,22 @@ https://f-droid.org/api/v1/packages/org.fdroid.fdroid https://xget.xi-xu.me/fdroid/api/v1/packages/org.fdroid.fdroid ``` +#### Flathub + +```url +# Flathub repository original URL +https://dl.flathub.org/repo/summary + +# Converted (add flathub prefix) +https://xget.xi-xu.me/flathub/repo/summary + +# Flathub app reference original URL +https://dl.flathub.org/repo/appstream/org.gnome.gedit.flatpakref + +# Converted (add flathub prefix) +https://xget.xi-xu.me/flathub/repo/appstream/org.gnome.gedit.flatpakref +``` + #### Jenkins Plugins ```url @@ -1866,6 +1883,55 @@ task checkFDroidAvailability { } ``` +### Flathub Repository Mirror + +#### Configure Flatpak / Flathub to Use Xget Mirror + +```bash +# Add the Flathub remote via the rewritten .flatpakrepo descriptor +flatpak remote-add --if-not-exists flathub \ + https://xget.xi-xu.me/flathub/repo/flathub.flatpakrepo + +# Or add the OSTree repository directly +flatpak remote-add --if-not-exists flathub \ + https://xget.xi-xu.me/flathub/repo/ +``` + +#### Supported Flathub Services + +```url +# OSTree repository metadata +https://xget.xi-xu.me/flathub/repo/summary +https://xget.xi-xu.me/flathub/repo/summary.sig + +# Flatpak remote descriptor +https://xget.xi-xu.me/flathub/repo/flathub.flatpakrepo + +# App reference descriptor +https://xget.xi-xu.me/flathub/repo/appstream/[app-id].flatpakref + +# Repository objects and static deltas +https://xget.xi-xu.me/flathub/repo/objects/... +https://xget.xi-xu.me/flathub/repo/deltas/... +``` + +#### Usage Examples + +```bash +# Inspect remote contents +flatpak remote-ls flathub + +# Install an app from the Xget-backed Flathub mirror +flatpak install flathub org.gnome.gedit + +# Install directly from a rewritten .flatpakref +flatpak install --from \ + https://xget.xi-xu.me/flathub/repo/appstream/org.gnome.gedit.flatpakref + +# Update installed apps and runtimes +flatpak update +``` + ### Jenkins Plugin Download #### Use Xget to Accelerate Jenkins Plugin Download and Update diff --git a/README.zh-Hans.md b/README.zh-Hans.md index a0f0b26..6d581a6 100644 --- a/README.zh-Hans.md +++ b/README.zh-Hans.md @@ -256,6 +256,7 @@ classDiagram | openSUSE | `opensuse` | `https://download.opensuse.org/...` | `https://xget.xi-xu.me/opensuse/...` | | Arch Linux | `arch` | `https://geo.mirror.pkgbuild.com/...` | `https://xget.xi-xu.me/arch/...` | | arXiv | `arxiv` | `https://arxiv.org/...` | `https://xget.xi-xu.me/arxiv/...` | +| Flathub | `flathub` | `https://dl.flathub.org/...` | `https://xget.xi-xu.me/flathub/...` | | F-Droid | `fdroid` | `https://f-droid.org/...` | `https://xget.xi-xu.me/fdroid/...` | | Jenkins 插件 | `jenkins` | `https://updates.jenkins.io/...` | `https://xget.xi-xu.me/jenkins/...` | | 容器注册表 | `cr` | 见[容器注册表](#容器注册表) | 见[容器注册表](#容器注册表) | @@ -707,6 +708,22 @@ https://f-droid.org/api/v1/packages/org.fdroid.fdroid https://xget.xi-xu.me/fdroid/api/v1/packages/org.fdroid.fdroid ``` +#### Flathub + +```url +# Flathub 存储库原始 URL +https://dl.flathub.org/repo/summary + +# 转换后（添加 flathub 前缀） +https://xget.xi-xu.me/flathub/repo/summary + +# Flathub 应用引用原始 URL +https://dl.flathub.org/repo/appstream/org.gnome.gedit.flatpakref + +# 转换后（添加 flathub 前缀） +https://xget.xi-xu.me/flathub/repo/appstream/org.gnome.gedit.flatpakref +``` + #### Jenkins 插件 ```url @@ -1866,6 +1883,55 @@ task checkFDroidAvailability { } ``` +### Flathub 存储库镜像 + +#### 配置 Flatpak / Flathub 使用 Xget 镜像 + +```bash +# 通过重写后的 .flatpakrepo 描述文件添加 Flathub 远程仓库 +flatpak remote-add --if-not-exists flathub \ + https://xget.xi-xu.me/flathub/repo/flathub.flatpakrepo + +# 或者直接添加 OSTree 存储库地址 +flatpak remote-add --if-not-exists flathub \ + https://xget.xi-xu.me/flathub/repo/ +``` + +#### 支持的 Flathub 服务 + +```url +# OSTree 存储库元数据 +https://xget.xi-xu.me/flathub/repo/summary +https://xget.xi-xu.me/flathub/repo/summary.sig + +# Flatpak 远程仓库描述文件 +https://xget.xi-xu.me/flathub/repo/flathub.flatpakrepo + +# 应用引用描述文件 +https://xget.xi-xu.me/flathub/repo/appstream/[应用 ID].flatpakref + +# 存储库对象与静态增量 +https://xget.xi-xu.me/flathub/repo/objects/... +https://xget.xi-xu.me/flathub/repo/deltas/... +``` + +#### 使用示例 + +```bash +# 查看远程仓库内容 +flatpak remote-ls flathub + +# 从 Xget 加速的 Flathub 镜像安装应用 +flatpak install flathub org.gnome.gedit + +# 直接通过重写后的 .flatpakref 安装 +flatpak install --from \ + https://xget.xi-xu.me/flathub/repo/appstream/org.gnome.gedit.flatpakref + +# 更新已安装的应用和运行时 +flatpak update +``` + ### Jenkins 插件下载 #### 使用 Xget 加速 Jenkins 插件下载和更新 diff --git a/README.zh-Hant.md b/README.zh-Hant.md index ed12982..17688ee 100644 --- a/README.zh-Hant.md +++ b/README.zh-Hant.md @@ -256,6 +256,7 @@ classDiagram | openSUSE | `opensuse` | `https://download.opensuse.org/...` | `https://xget.xi-xu.me/opensuse/...` | | Arch Linux | `arch` | `https://geo.mirror.pkgbuild.com/...` | `https://xget.xi-xu.me/arch/...` | | arXiv | `arxiv` | `https://arxiv.org/...` | `https://xget.xi-xu.me/arxiv/...` | +| Flathub | `flathub` | `https://dl.flathub.org/...` | `https://xget.xi-xu.me/flathub/...` | | F-Droid | `fdroid` | `https://f-droid.org/...` | `https://xget.xi-xu.me/fdroid/...` | | Jenkins 外掛程式 | `jenkins` | `https://updates.jenkins.io/...` | `https://xget.xi-xu.me/jenkins/...` | | 容器註冊表 | `cr` | 見[容器註冊表](#容器註冊表) | 見[容器註冊表](#容器註冊表) | @@ -707,6 +708,22 @@ https://f-droid.org/api/v1/packages/org.fdroid.fdroid https://xget.xi-xu.me/fdroid/api/v1/packages/org.fdroid.fdroid ``` +#### Flathub + +```url +# Flathub 儲存庫原始 URL +https://dl.flathub.org/repo/summary + +# 轉換後（新增 flathub 前綴） +https://xget.xi-xu.me/flathub/repo/summary + +# Flathub 應用程式引用原始 URL +https://dl.flathub.org/repo/appstream/org.gnome.gedit.flatpakref + +# 轉換後（新增 flathub 前綴） +https://xget.xi-xu.me/flathub/repo/appstream/org.gnome.gedit.flatpakref +``` + #### Jenkins 外掛程式 ```url @@ -1866,6 +1883,55 @@ task checkFDroidAvailability { } ``` +### Flathub 儲存庫鏡像 + +#### 配置 Flatpak / Flathub 使用 Xget 鏡像 + +```bash +# 透過重寫後的 .flatpakrepo 描述檔新增 Flathub 遠端儲存庫 +flatpak remote-add --if-not-exists flathub \ + https://xget.xi-xu.me/flathub/repo/flathub.flatpakrepo + +# 或者直接新增 OSTree 儲存庫位址 +flatpak remote-add --if-not-exists flathub \ + https://xget.xi-xu.me/flathub/repo/ +``` + +#### 支援的 Flathub 服務 + +```url +# OSTree 儲存庫中繼資料 +https://xget.xi-xu.me/flathub/repo/summary +https://xget.xi-xu.me/flathub/repo/summary.sig + +# Flatpak 遠端儲存庫描述檔 +https://xget.xi-xu.me/flathub/repo/flathub.flatpakrepo + +# 應用程式引用描述檔 +https://xget.xi-xu.me/flathub/repo/appstream/[應用程式 ID].flatpakref + +# 儲存庫物件與靜態增量 +https://xget.xi-xu.me/flathub/repo/objects/... +https://xget.xi-xu.me/flathub/repo/deltas/... +``` + +#### 使用範例 + +```bash +# 檢視遠端儲存庫內容 +flatpak remote-ls flathub + +# 從 Xget 加速的 Flathub 鏡像安裝應用程式 +flatpak install flathub org.gnome.gedit + +# 直接透過重寫後的 .flatpakref 安裝 +flatpak install --from \ + https://xget.xi-xu.me/flathub/repo/appstream/org.gnome.gedit.flatpakref + +# 更新已安裝的應用程式與執行時 +flatpak update +``` + ### Jenkins 外掛程式下載 #### 使用 Xget 加速 Jenkins 外掛程式下載和更新 diff --git a/src/config/platforms.js b/src/config/platforms.js index dab6cd4..c4d2535 100644 --- a/src/config/platforms.js +++ b/src/config/platforms.js @@ -63,6 +63,7 @@ * * **Other Resources:** * - `arxiv` - arXiv papers (arxiv.org) + * - `flathub` - Flathub Flatpak repository (dl.flathub.org) * - `fdroid` - F-Droid Android apps (f-droid.org) * - `jenkins` - Jenkins plugins (updates.jenkins.io) * @@ -171,6 +172,7 @@ export const PLATFORMS = { // Other Resources arxiv: 'https://arxiv.org', + flathub: 'https://dl.flathub.org', fdroid: 'https://f-droid.org', jenkins: 'https://updates.jenkins.io', diff --git a/src/index.js b/src/index.js index 9ad4349..d350521 100644 --- a/src/index.js +++ b/src/index.js @@ -22,6 +22,11 @@ import { } from './protocols/docker.js'; import { configureGitHeaders, isGitLFSRequest, isGitRequest } from './protocols/git.js'; import { PerformanceMonitor, addPerformanceHeaders } from './utils/performance.js'; +import { + isFlatpakReferenceFilePath, + rewriteTextResponse, + shouldRewriteTextResponse +} from './utils/rewrite.js'; import { addCorsHeaders, addSecurityHeaders, createErrorResponse } from './utils/security.js'; import { getAllowedMethods, isDockerRequest, validateRequest } from './utils/validation.js'; @@ -546,26 +551,21 @@ async function handleRequest(request, env, ctx) { let rewrittenContentLength = null; if ( - platform === 'pypi' && - response.headers.get('content-type')?.includes('text/html') + shouldRewriteTextResponse( + platform, + effectivePath, + response.headers.get('content-type') || '' + ) ) { - const originalText = await response.text(); - const rewrittenText = originalText.replace( - /https:\/\/files\.pythonhosted\.org/g, - `${url.origin}/pypi/files` - ); - responseBody = rewrittenText; - rewrittenContentLength = new TextEncoder().encode(rewrittenText).byteLength; - } - - if ( - platform === 'npm' && - response.headers.get('content-type')?.includes('application/json') - ) { - const originalText = await response.text(); - const rewrittenText = originalText.replace( - /https:\/\/registry.npmjs.org\/([^/]+)/g, - `${url.origin}/npm/$1` + const originalText = + platform === 'flathub' && isFlatpakReferenceFilePath(effectivePath) + ? new TextDecoder().decode(await response.arrayBuffer()) + : await response.text(); + const rewrittenText = rewriteTextResponse( + platform, + effectivePath, + originalText, + url.origin ); responseBody = rewrittenText; rewrittenContentLength = new TextEncoder().encode(rewrittenText).byteLength; diff --git a/src/utils/rewrite.js b/src/utils/rewrite.js new file mode 100644 index 0000000..af94cfd --- /dev/null +++ b/src/utils/rewrite.js @@ -0,0 +1,65 @@ +/** + * Xget - Platform-specific upstream response rewriting helpers. + */ + +const FLATHUB_REPO_BASE_URL_PATTERN = /https:\/\/(?:dl\.)?flathub\.org\/repo\//g; +const FLATPAK_REFERENCE_FILE_PATTERN = /\.(flatpakrepo|flatpakref)$/i; + +/** + * Checks whether a successful upstream response should be rewritten before returning it. + * @param {string} platform + * @param {string} requestPath + * @param {string} contentType + * @returns {boolean} True when the upstream response body should be rewritten. + */ +export function shouldRewriteTextResponse(platform, requestPath, contentType = '') { + if (platform === 'pypi') { + return contentType.includes('text/html'); + } + + if (platform === 'npm') { + return contentType.includes('application/json'); + } + + if (platform === 'flathub') { + return FLATPAK_REFERENCE_FILE_PATTERN.test(requestPath); + } + + return false; +} + +/** + * Checks whether a request path points to a Flatpak descriptor file. + * @param {string} requestPath + * @returns {boolean} True when the request targets a `.flatpakrepo` or `.flatpakref` file. + */ +export function isFlatpakReferenceFilePath(requestPath) { + return FLATPAK_REFERENCE_FILE_PATTERN.test(requestPath); +} + +/** + * Rewrites upstream text responses so follow-up requests continue flowing through Xget. + * @param {string} platform + * @param {string} requestPath + * @param {string} originalText + * @param {string} origin + * @returns {string} Rewritten response text. + */ +export function rewriteTextResponse(platform, requestPath, originalText, origin) { + if (platform === 'pypi') { + return originalText.replace(/https:\/\/files\.pythonhosted\.org/g, `${origin}/pypi/files`); + } + + if (platform === 'npm') { + return originalText.replace( + /https:\/\/registry\.npmjs\.org\/([^/]+)/g, + `${origin}/npm/$1` + ); + } + + if (platform === 'flathub' && isFlatpakReferenceFilePath(requestPath)) { + return originalText.replace(FLATHUB_REPO_BASE_URL_PATTERN, `${origin}/flathub/repo/`); + } + + return originalText; +} diff --git a/test/index.test.js b/test/index.test.js index cd2b64c..9d17999 100644 --- a/test/index.test.js +++ b/test/index.test.js @@ -93,6 +93,14 @@ describe('Xget Core Functionality', () => { expect(response.status).not.toBe(400); }); + it('should handle Flathub URLs correctly', async () => { + const testUrl = 'https://example.com/flathub/repo/summary'; + const response = await SELF.fetch(testUrl, { method: 'HEAD' }); + + // Should attempt to proxy to Flathub + expect(response.status).not.toBe(400); + }); + it('should not treat nested /v2/ path segments as container registry requests', async () => { const testUrl = 'https://example.com/gh/microsoft/vscode/releases/download/v2/file.tar.gz'; const response = await SELF.fetch(testUrl, { method: 'HEAD' }); diff --git a/test/platforms/flathub.test.js b/test/platforms/flathub.test.js new file mode 100644 index 0000000..92aa383 --- /dev/null +++ b/test/platforms/flathub.test.js @@ -0,0 +1,56 @@ +import { describe, expect, it } from 'vitest'; +import { PLATFORMS, transformPath } from '../../src/config/platforms.js'; + +describe('Flathub Platform Configuration', () => { + it('should have Flathub platform configured', () => { + expect(PLATFORMS.flathub).toBe('https://dl.flathub.org'); + }); + + it('should transform Flathub repository paths correctly', () => { + const testCases = [ + { + input: '/flathub/repo/summary', + expected: '/repo/summary', + description: 'repository summary' + }, + { + input: '/flathub/repo/summary.sig', + expected: '/repo/summary.sig', + description: 'repository summary signature' + }, + { + input: '/flathub/repo/flathub.flatpakrepo', + expected: '/repo/flathub.flatpakrepo', + description: 'remote descriptor' + }, + { + input: '/flathub/repo/appstream/org.gnome.gedit.flatpakref', + expected: '/repo/appstream/org.gnome.gedit.flatpakref', + description: 'application reference' + }, + { + input: '/flathub/repo/objects/12/34567890abcdef.filez', + expected: '/repo/objects/12/34567890abcdef.filez', + description: 'content-addressed object' + }, + { + input: '/flathub/repo/deltas/ABCD.superblock', + expected: '/repo/deltas/ABCD.superblock', + description: 'static delta' + } + ]; + + testCases.forEach(({ input, expected, description }) => { + const result = transformPath(input, 'flathub'); + expect(result, `Failed for ${description}: ${input}`).toBe(expected); + }); + }); + + it('should handle root path correctly', () => { + expect(transformPath('/flathub/', 'flathub')).toBe('/'); + }); + + it('should preserve already transformed Flathub repository paths', () => { + expect(transformPath('/repo/summary', 'flathub')).toBe('/repo/summary'); + }); +}); diff --git a/test/unit/flathub-rewrite.test.js b/test/unit/flathub-rewrite.test.js new file mode 100644 index 0000000..294c6eb --- /dev/null +++ b/test/unit/flathub-rewrite.test.js @@ -0,0 +1,114 @@ +import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest'; + +import worker from '../../src/index.js'; + +/** @type {ExecutionContext} */ +const executionContext = { + waitUntil() {}, + passThroughOnException() {} +}; + +/** + * Reads a response body as UTF-8 text without relying on the response MIME type. + * @param {Response} response + * @returns {Promise} Decoded UTF-8 response text. + */ +async function readUtf8Text(response) { + return new TextDecoder().decode(await response.arrayBuffer()); +} + +describe('Flathub Response Rewriting', () => { + beforeEach(() => { + vi.stubGlobal('caches', { + default: { + match: vi.fn(async () => null), + put: vi.fn(async () => undefined) + } + }); + }); + + afterEach(() => { + vi.unstubAllGlobals(); + vi.restoreAllMocks(); + }); + + it('rewrites .flatpakrepo URLs to stay on the Xget mirror', async () => { + vi.spyOn(globalThis, 'fetch').mockResolvedValue( + new Response( + [ + '[Flatpak Repo]', + 'Url=https://dl.flathub.org/repo/', + 'Icon=https://dl.flathub.org/repo/logo.svg', + 'Homepage=https://flathub.org/' + ].join('\n'), + { + status: 200, + headers: { 'Content-Type': 'application/octet-stream' } + } + ) + ); + + const response = await worker.fetch( + new Request('https://example.com/flathub/repo/flathub.flatpakrepo'), + {}, + executionContext + ); + + expect(response.status).toBe(200); + + const body = await readUtf8Text(response); + expect(body).toContain('Url=https://example.com/flathub/repo/'); + expect(body).toContain('Icon=https://example.com/flathub/repo/logo.svg'); + expect(body).toContain('Homepage=https://flathub.org/'); + expect(response.headers.get('Content-Length')).toBe(String(new TextEncoder().encode(body).length)); + }); + + it('rewrites .flatpakref URLs to stay on the Xget mirror', async () => { + vi.spyOn(globalThis, 'fetch').mockResolvedValue( + new Response( + [ + '[Flatpak Ref]', + 'Name=org.gnome.gedit', + 'Url=https://dl.flathub.org/repo/', + 'RuntimeRepo=https://dl.flathub.org/repo/flathub.flatpakrepo' + ].join('\n'), + { + status: 200, + headers: { 'Content-Type': 'application/octet-stream' } + } + ) + ); + + const response = await worker.fetch( + new Request('https://example.com/flathub/repo/appstream/org.gnome.gedit.flatpakref'), + {}, + executionContext + ); + + expect(response.status).toBe(200); + + const body = await readUtf8Text(response); + expect(body).toContain('Url=https://example.com/flathub/repo/'); + expect(body).toContain( + 'RuntimeRepo=https://example.com/flathub/repo/flathub.flatpakrepo' + ); + }); + + it('does not rewrite binary repository metadata like summary files', async () => { + vi.spyOn(globalThis, 'fetch').mockResolvedValue( + new Response('summary-binary-payload', { + status: 200, + headers: { 'Content-Type': 'application/octet-stream' } + }) + ); + + const response = await worker.fetch( + new Request('https://example.com/flathub/repo/summary'), + {}, + executionContext + ); + + expect(response.status).toBe(200); + expect(await readUtf8Text(response)).toBe('summary-binary-payload'); + }); +}); diff --git a/test/unit/platforms.test.js b/test/unit/platforms.test.js index c702ffc..4e58bd5 100644 --- a/test/unit/platforms.test.js +++ b/test/unit/platforms.test.js @@ -16,6 +16,7 @@ describe('Platform Configuration', () => { 'npm', 'pypi', 'conda', + 'flathub', 'homebrew' ]; @@ -134,6 +135,13 @@ describe('Platform Configuration', () => { ).toBe('/conda-forge/linux-64/repodata.json'); }); + it('should transform Flathub paths correctly', () => { + expect(transformPath('/flathub/repo/summary', 'flathub')).toBe('/repo/summary'); + expect(transformPath('/flathub/repo/flathub.flatpakrepo', 'flathub')).toBe( + '/repo/flathub.flatpakrepo' + ); + }); + it('should transform container registry paths correctly', () => { expect( transformPath('/cr/ghcr/v2/nginxinc/nginx-unprivileged/manifests/latest', 'cr-ghcr') @@ -191,6 +199,10 @@ describe('Platform Configuration', () => { expect(PLATFORMS['conda-community']).toBe('https://conda.anaconda.org'); }); + it('should have correct Flathub base URL', () => { + expect(PLATFORMS.flathub).toBe('https://dl.flathub.org'); + }); + it('should have correct container registry base URLs', () => { expect(PLATFORMS['cr-ghcr']).toBe('https://ghcr.io'); expect(PLATFORMS['cr-gcr']).toBe('https://gcr.io');