feat(pages): configurable limits, multi-format packages, and dual versioning

Make Pages package size and history retention system-configurable, support
zip/tar.gz/tar.xz/tar.bz2/tar/7z uploads, prune history with clear keep-N
semantics, and rebind agent config to the live active Pages deployment so
main-config rollback never depends on pruned packages.
This commit is contained in:
ryan
2026-07-17 17:16:48 +08:00
parent 368df3f76b
commit a0fcf9f627
32 changed files with 2286 additions and 370 deletions
+13
View File
@@ -21,6 +21,19 @@ sidebar: false
## [unreleased] ## [unreleased]
### 新增
- Pages 现支持上传 zip、tar.gz、tar.xz、tar.bz2、tar、7z 等常用压缩格式的部署包。
- 管理员可在运维设置中配置 Pages 部署包大小上限与每个项目的历史部署保留数量。
### 变更
- Pages 部署包上传体积限制改为系统动态配置,默认仍为 100 MiB,可按环境调整。
- 上传新部署后会按保留策略自动清理超出数量的历史部署包,减少磁盘占用。
- Agent 在同步 Pages 部署时信任控制面已完成的包校验,不再重复限制文件数与展开体积,仅校验下载完整性并安全解压到本地。
- 明确 Pages 历史部署保留语义为每个项目最多 N 条(激活必留、其余按从新到旧填充),裁剪失败会记录日志且不回滚已成功上传。
- 主配置版本与 Pages 部署双轨管理:下发给 Agent 的配置会将 Pages 路由重绑定到项目当前激活部署,主配置回滚不再依赖已裁剪的旧部署包。
## [v3.3.0] - 2026-07-14 ## [v3.3.0] - 2026-07-14
### 新增 ### 新增
+1 -1
View File
@@ -98,7 +98,7 @@ Agent 对数据面 OpenResty 的管控实现了端到端的闭环,包含配置
* `certs/`:证书存放目录(文件命名为 `{cert_id}.crt` 和 `{cert_id}.key`)。 * `certs/`:证书存放目录(文件命名为 `{cert_id}.crt` 和 `{cert_id}.key`)。
* `waf/` 与 `pow/`:WAF 及防 CC 挑战所需的专用 Lua 运行时脚本。 * `waf/` 与 `pow/`:WAF 及防 CC 挑战所需的专用 Lua 运行时脚本。
* `waf_config.json` 与 `waf_ip_groups.json`:WAF 过滤引擎所需的结构化规则配置文件。 * `waf_config.json` 与 `waf_ip_groups.json`:WAF 过滤引擎所需的结构化规则配置文件。
* `pages_dir`:Pages 静态站点部署目录,默认位于 `data_dir/var/lib/openflare/pages`。当激活配置引用 Pages 部署时,Agent 会下载部署 zip、校验 checksum、解压到部署 release 目录,并切换 `deployments/{deployment_id}/current` 供 OpenResty `root`/`try_files` 读取。 * `pages_dir`:Pages 静态站点部署目录,默认位于 `data_dir/var/lib/openflare/pages`。当激活配置引用 Pages 部署时,Agent 会下载部署包(zip / tar.gz / tar.xz / tar.bz2 / tar / 7z)、校验下载 checksum、解压到部署 release 目录,并切换 `deployments/{deployment_id}/current` 供 OpenResty `root`/`try_files` 读取。业务侧体积/文件数校验由控制面完成,Agent 信任控制面结果,解压时仅保留本机路径安全防护。
### 2. 精细化的重载动作 ### 2. 精细化的重载动作
1. **备份当前配置**:在写入新文件之前,Agent 会将现有的配置文件复制到 `.backup` 临时目录下,保留完整的现场快照。 1. **备份当前配置**:在写入新文件之前,Agent 会将现有的配置文件复制到 `.backup` 临时目录下,保留完整的现场快照。
+1 -1
View File
@@ -26,7 +26,7 @@ OpenFlare 适合需要统一管理多台 OpenResty 代理节点的团队,具
| **配置版本控制** | 支持全局单一激活版本的预览、发布、不可变快照历史与秒级一键回滚 | [Agent 与发布模型](./agent-design.md) | | **配置版本控制** | 支持全局单一激活版本的预览、发布、不可变快照历史与秒级一键回滚 | [Agent 与发布模型](./agent-design.md) |
| **WAF 安全防护** | 支持可视化 DAG 编排规则、手动/自动/订阅型 IP 组、GeoIP 匹配与 PoW CC 防护 | [WAF 设计](./waf-design.md) / [WAF 可编排规则设计](./waf-orchestration-design.md) / [WAF 使用指南](../guide/waf-usage.md) | | **WAF 安全防护** | 支持可视化 DAG 编排规则、手动/自动/订阅型 IP 组、GeoIP 匹配与 PoW CC 防护 | [WAF 设计](./waf-design.md) / [WAF 可编排规则设计](./waf-orchestration-design.md) / [WAF 使用指南](../guide/waf-usage.md) |
| **内网穿透** | 通过中继节点(Relay)与内网客户端(OpenFlared),反向穿透暴露内网 Web 服务 | [内网穿透设计](./tunnel-design.md) / [穿透使用指南](../guide/tunnel-usage.md) | | **内网穿透** | 通过中继节点(Relay)与内网客户端(OpenFlared),反向穿透暴露内网 Web 服务 | [内网穿透设计](./tunnel-design.md) / [穿透使用指南](../guide/tunnel-usage.md) |
| **Pages 静态托管** | 直接上传前端 zip 包,由边缘节点拉取并由 OpenResty 本地服务,支持 API 反代与 SPA Fallback | [Pages 静态托管设计](./pages-design.md) | | **Pages 静态托管** | 直接上传前端压缩包(zip / tar.gz / tar.xz / 7z 等),由边缘节点拉取并由 OpenResty 本地服务,支持 API 反代与 SPA Fallback | [Pages 静态托管设计](./pages-design.md) |
| **TLS 证书自动续期** | 将证书显式绑定到 Zone 域名,并通过 ACME 协议向 Let's Encrypt 申请/续期证书 | [Zone 与域名资源设计](./zone-design.md) | | **TLS 证书自动续期** | 将证书显式绑定到 Zone 域名,并通过 ACME 协议向 Let's Encrypt 申请/续期证书 | [Zone 与域名资源设计](./zone-design.md) |
| **多节点监控与观测** | 收集节点资源快照、健康事件,聚合请求指标与访问日志明细 | [系统架构](./architecture.md) | | **多节点监控与观测** | 收集节点资源快照、健康事件,聚合请求指标与访问日志明细 | [系统架构](./architecture.md) |
+22 -13
View File
@@ -19,11 +19,12 @@
## 核心功能 ## 核心功能
Pages 静态托管子系统包含以下核心能力: Pages 静态托管子系统包含以下核心能力:
* **Direct Upload 部署模式**:支持直接上传预构建的 `.zip` 静态资源包,省去复杂的 Git 集成和构建环境依赖。 * **Direct Upload 部署模式**:支持直接上传预构建的静态资源压缩包(`zip`、`tar.gz`、`tar.xz`、`tar.bz2`、`tar`、`7z`),省去复杂的 Git 集成和构建环境依赖。
* **不可变部署快照**:每次上传产生一个带唯一 ID 和 SHA-256 Checksum 的不可变部署记录。历史包永久保留,支持随时激活和回滚。 * **不可变部署快照**:每次上传产生一个带唯一 ID 和 SHA-256 Checksum 的不可变部署记录。支持按系统配置保留最近 N 个历史部署,并可随时激活和回滚。
* **SPA Fallback 支持**:支持对单页应用(SPA)进行 Fallback 路由配置,请求找不到静态文件时自动重定向到入口文件。 * **SPA Fallback 支持**:支持对单页应用(SPA)进行 Fallback 路由配置,请求找不到静态文件时自动重定向到入口文件。
* **内置 API 反代服务**:支持在 Pages 规则内一键启用 API 代理,消除跨域问题,将请求转发给指定的后端服务。 * **内置 API 反代服务**:支持在 Pages 规则内一键启用 API 代理,消除跨域问题,将请求转发给指定的后端服务。
* **安全包校验与解压缩**:内置 Zip-Slip 路径逃逸防御、防软链接劫持、文件大小/数量硬上限控制,保障节点物理安全。 * **安全包校验与解压缩**:内置路径逃逸防御、防软链接劫持、文件大小/数量上限与可配置上传包体积控制,保障节点物理安全。
* **可配置限额**:管理员可在运维设置中调整「部署包大小上限」与「历史部署保留数」。
--- ---
@@ -40,7 +41,7 @@ graph TD
%% 控制流与心跳 %% 控制流与心跳
Server[OpenFlare Server 控制面] <-->|Agent API / Heartbeat| Agent[openflare-agent 进程] Server[OpenFlare Server 控制面] <-->|Agent API / Heartbeat| Agent[openflare-agent 进程]
Server -.->|5. 存储 ZIP 部署包| LocalStore[(Server 本地存储)] Server -.->|5. 存储部署包| LocalStore[(Server 本地存储)]
Agent -->|1. 发现新版本| Server Agent -->|1. 发现新版本| Server
Agent -->|2. 下载部署包| Server Agent -->|2. 下载部署包| Server
@@ -88,21 +89,29 @@ graph TD
} }
``` ```
### 3. 与主配置版本的双轨关系
* **主配置版本**(`config_versions`)与 **Pages 部署** 是两套独立的版本体系。
* 快照里记录的 `pages_deployment` 仅反映**发布当时**的激活部署,用于审计与当时渲染结果存档。
* **运行时**:下发给 Agent 的配置会把各 Pages 路由**重绑定到该项目当前激活部署**(最新包)。主配置回滚/重新激活旧版本时,**不要求**仍能拉到旧 Pages 包,只保证指向当前最新激活部署。
* 因此 Pages 历史裁剪可以安全删除非激活部署,无需为「主配置回滚到旧 Pages 包」预留存储。
--- ---
## Server 端 (控制面) 职责与生命周期 ## Server 端 (控制面) 职责与生命周期
### 1. ZIP 包安全校验与分析 ### 1. 部署包安全校验与分析
为了避免不可信的用户上传恶意压缩包攻击服务器,控制面在 `UploadPagesDeployment` 时执行严格的流式校验: 为了避免不可信的用户上传恶意压缩包攻击服务器,控制面在 `UploadDeployment` 时执行严格校验:
* **大小限制**:ZIP 压缩包不得超过 25 MiB(保守的 V1 默认值),且展开后的解压总体积不得超过 100 MiB。 * **格式支持**:`zip`、`tar.gz` / `tgz`、`tar.xz` / `txz`、`tar.bz2` / `tbz2`、`tar`、`7z`。
* **大小限制**:压缩包体积由系统配置 `pages_max_package_size_mb` 控制(默认 100 MiB,范围 1~2048);展开后总体积上限为「包大小 × 4」且不低于 100 MiB。
* **数量限制**:压缩包中包含的静态文件总数不得超过 1,000 个。 * **数量限制**:压缩包中包含的静态文件总数不得超过 1,000 个。
* **软链接阻断**:遍历 ZIP 文件,一旦检测到任何软链接 (`os.ModeSymlink`),立即抛出错误并拒绝上传,防御软链接劫持攻击。 * **软链接阻断**:遍历归档文件,一旦检测到任何软链接,立即抛出错误并拒绝上传,防御软链接劫持攻击。
* **Zip-Slip 防御**:对每个压缩文件路径进行 `Clean` 并检查是否包含 `..` 或以 `/` 开头,防御目录跨越漏洞,防止写入系统敏感路径。 * **路径逃逸防御**:对每个压缩文件路径进行 `Clean` 并检查是否包含 `..` 或以 `/` 开头,防御目录跨越漏洞,防止写入系统敏感路径。
* **入口文件校验**:项目指定的入口文件(例如 `index.html`,可在 `project.RootDir` 下)必须在 ZIP 压缩包中存在,否则拒绝上传。 * **入口文件校验**:项目指定的入口文件(例如 `index.html`,可在 `project.RootDir` 下)必须在部署包中存在,否则拒绝上传。
* **公共根目录去噪**:许多打包工具(如 GitHub 导出的 zip)会包含一个多余的主文件夹作为公共根前缀。控制面自动探测公共根前缀并将其安全剥离。 * **公共根目录去噪**:许多打包工具会包含一个多余的主文件夹作为公共根前缀。控制面自动探测公共根前缀并将其安全剥离。
* **历史保留**:系统配置 `pages_max_history_count`(默认 20,0 表示不限制)在每次上传成功后执行裁剪。语义为:**每个项目最多保留 N 条部署**;当前激活部署始终保留;其余名额按部署 ID 从新到旧填充;超出的非激活部署连同文件清单与存储对象一并删除。上传已成功时裁剪失败只记日志、不回滚上传;并发上传下可能短暂超过 N,后续上传的裁剪会收敛回 N。主配置版本回滚不依赖旧 Pages 包(见上节双轨关系)。
### 2. 部署包存储规划 ### 2. 部署包存储规划
控制面仅将 zip 文件存储在本地存储目录 `artifacts/{project_slug}/{checksum}.zip`,并在数据库中记录路径和清单。**大体积静态包不写入 config_versions 记录和任何配置推送通道**,以保障控制面数据同步的轻量与高效。 控制面通过统一上传框架(`upload.Ingest`)存储原始部署包,并在数据库中记录 `upload_id` 与文件清单。**大体积静态包不写入 config_versions 记录和任何配置推送通道**,以保障控制面数据同步的轻量与高效。
--- ---
@@ -118,7 +127,7 @@ Agent 运行在各边缘代理节点上,在应用配置版本前,必须先
### 2. 安全解压缩与原子切换 ### 2. 安全解压缩与原子切换
为了保证配置应用过程的“无缝”且能在出错时立即回滚: 为了保证配置应用过程的“无缝”且能在出错时立即回滚:
1. Agent 将下载的部署包数据写入临时目录,并重新计算 SHA-256 Checksum。如果与配置指明的 checksum 不符,立即报错并阻断发布流程。 1. Agent 将下载的部署包数据写入临时目录,并重新计算 SHA-256 Checksum。如果与配置指明的 checksum 不符,立即报错并阻断发布流程。
2. 解压部署包至临时目录 `releases/{checksum}.tmp`。解压时同样执行 Zip-Slip 目录跨越和软链接校验防御。 2. 解压部署包至临时目录 `releases/{checksum}.tmp`。解压支持 zip / tar.* / 7z。**Agent 默认信任控制面**:不再重复校验文件数/体积等业务限额(控制面上传时已完成);仅做本机落盘安全处理(路径逃逸、软链接拒绝)与下载 checksum 完整性校验。
3. 解压成功后,写入标记文件 `.openflare-pages.json`。 3. 解压成功后,写入标记文件 `.openflare-pages.json`。
4. 清理 `releases/{checksum}` 目录,将整个临时目录重命名为 `releases/{checksum}`。 4. 清理 `releases/{checksum}` 目录,将整个临时目录重命名为 `releases/{checksum}`。
5. **原子切换**:建立拷贝当前部署的物理副本到目标位置 `deployments/{deployment_id}/current`。切换前先备份上一版本的 `current`,一旦重载配置失败,Agent 能够快速恢复 `current` 目录并回滚 OpenResty。 5. **原子切换**:建立拷贝当前部署的物理副本到目标位置 `deployments/{deployment_id}/current`。切换前先备份上一版本的 `current`,一旦重载配置失败,Agent 能够快速恢复 `current` 目录并回滚 OpenResty。
@@ -20,6 +20,8 @@ export type OpenFlareOpsFields = {
uptime_kuma_timeout: string; uptime_kuma_timeout: string;
database_auto_cleanup_enabled: boolean; database_auto_cleanup_enabled: boolean;
database_auto_cleanup_retention_days: string; database_auto_cleanup_retention_days: string;
pages_max_package_size_mb: string;
pages_max_history_count: string;
}; };
export const defaultOpenFlareOpsFields: OpenFlareOpsFields = { export const defaultOpenFlareOpsFields: OpenFlareOpsFields = {
@@ -42,6 +44,8 @@ export const defaultOpenFlareOpsFields: OpenFlareOpsFields = {
uptime_kuma_timeout: '48', uptime_kuma_timeout: '48',
database_auto_cleanup_enabled: false, database_auto_cleanup_enabled: false,
database_auto_cleanup_retention_days: '30', database_auto_cleanup_retention_days: '30',
pages_max_package_size_mb: '100',
pages_max_history_count: '20',
}; };
export const INSTALLER_SCRIPT_URL = export const INSTALLER_SCRIPT_URL =
@@ -90,6 +94,8 @@ export function mapOptionsToOpsFields(
), ),
database_auto_cleanup_retention_days: database_auto_cleanup_retention_days:
optionMap.database_auto_cleanup_retention_days ?? '30', optionMap.database_auto_cleanup_retention_days ?? '30',
pages_max_package_size_mb: optionMap.pages_max_package_size_mb ?? '100',
pages_max_history_count: optionMap.pages_max_history_count ?? '20',
}; };
} }
@@ -226,3 +232,30 @@ export function databaseAutoCleanupEntries(
}, },
]; ];
} }
export function validatePagesFields(fields: OpenFlareOpsFields) {
const packageSize = Number.parseInt(fields.pages_max_package_size_mb, 10);
const historyCount = Number.parseInt(fields.pages_max_history_count, 10);
if (Number.isNaN(packageSize) || packageSize < 1 || packageSize > 2048) {
throw new Error('Pages 部署包大小上限必须为 1~2048 MiB。');
}
if (Number.isNaN(historyCount) || historyCount < 0) {
throw new Error(
'Pages 历史保留数必须为大于等于 0 的整数(0 表示不限制)。',
);
}
}
export function pagesOptionEntries(fields: OpenFlareOpsFields): OptionItem[] {
validatePagesFields(fields);
return [
{
key: 'pages_max_package_size_mb',
value: String(Number.parseInt(fields.pages_max_package_size_mb, 10)),
},
{
key: 'pages_max_history_count',
value: String(Number.parseInt(fields.pages_max_history_count, 10)),
},
];
}
@@ -64,6 +64,7 @@ import {
mapOptionsToOpsFields, mapOptionsToOpsFields,
type OpenFlareOpsFields, type OpenFlareOpsFields,
optionsToMap, optionsToMap,
pagesOptionEntries,
uptimeKumaOptionEntries, uptimeKumaOptionEntries,
} from './openflare-ops-utils'; } from './openflare-ops-utils';
import { UptimeKumaSiteSelectModal } from './uptimekuma-site-modal'; import { UptimeKumaSiteSelectModal } from './uptimekuma-site-modal';
@@ -258,6 +259,17 @@ export function OpenFlareOpsSettings() {
} }
}; };
const savePagesSettings = () => {
try {
saveMutation.mutate({
section: 'pages',
entries: pagesOptionEntries(fields),
});
} catch (error) {
toast.error(error instanceof Error ? error.message : '参数校验失败');
}
};
if (optionsQuery.isLoading) { if (optionsQuery.isLoading) {
return ( return (
<LoadingStateWithBorder <LoadingStateWithBorder
@@ -413,6 +425,55 @@ export function OpenFlareOpsSettings() {
</Card> </Card>
</div> </div>
<Card className='border-dashed shadow-none'>
<CardHeader className='flex flex-row items-center justify-between gap-4'>
<div>
<CardTitle className='text-base'>Pages 静态托管</CardTitle>
<CardDescription>
配置部署包上传体积上限与每个项目的历史部署保留数量。
</CardDescription>
</div>
<Button
size='sm'
disabled={savingSection === 'pages'}
onClick={savePagesSettings}
>
{savingSection === 'pages' ? (
<Loader2 className='size-4 animate-spin mr-1' />
) : (
<Save className='size-3.5 mr-1' />
)}
保存
</Button>
</CardHeader>
<CardContent className='space-y-4'>
<div className='grid gap-4 md:grid-cols-2'>
<FieldInput
label='部署包大小上限 (MiB)'
value={fields.pages_max_package_size_mb}
type='number'
onChange={(value) =>
updateField('pages_max_package_size_mb', value)
}
placeholder='100'
/>
<FieldInput
label='历史部署保留数(0 不限制)'
value={fields.pages_max_history_count}
type='number'
onChange={(value) =>
updateField('pages_max_history_count', value)
}
placeholder='20'
/>
</div>
<p className='text-xs text-muted-foreground'>
每个项目最多保留 N 条部署:激活部署始终保留,其余按从新到旧填充;超出的非激活部署会在上传成功后自动清理。支持
zip、tar.gz、tar.xz、tar.bz2、tar、7z 格式。
</p>
</CardContent>
</Card>
<Card className='border-dashed shadow-none'> <Card className='border-dashed shadow-none'>
<CardHeader className='flex flex-row items-center justify-between gap-4'> <CardHeader className='flex flex-row items-center justify-between gap-4'>
<div> <div>
@@ -27,6 +27,27 @@ import {
projectsQueryKey, projectsQueryKey,
} from './pages-utils'; } from './pages-utils';
const PAGES_PACKAGE_ACCEPT =
'.zip,.tar.gz,.tgz,.tar.xz,.txz,.tar.bz2,.tbz2,.tbz,.tar,.7z';
const PAGES_PACKAGE_EXTENSIONS = [
'.zip',
'.tar.gz',
'.tgz',
'.tar.xz',
'.txz',
'.tar.bz2',
'.tbz2',
'.tbz',
'.tar',
'.7z',
] as const;
function isSupportedPagesPackage(fileName: string) {
const lower = fileName.toLowerCase();
return PAGES_PACKAGE_EXTENSIONS.some((ext) => lower.endsWith(ext));
}
interface DeploymentUploadDialogProps { interface DeploymentUploadDialogProps {
open: boolean; open: boolean;
onOpenChange: (open: boolean) => void; onOpenChange: (open: boolean) => void;
@@ -58,7 +79,7 @@ export function DeploymentUploadDialog({
const uploadMutation = useMutation({ const uploadMutation = useMutation({
mutationFn: () => { mutationFn: () => {
if (!file) throw new Error('请选择 zip 部署包'); if (!file) throw new Error('请选择部署包');
return PagesService.uploadDeployment(projectId, { return PagesService.uploadDeployment(projectId, {
file, file,
onProgress: setUploadProgress, onProgress: setUploadProgress,
@@ -83,8 +104,8 @@ export function DeploymentUploadDialog({
const handleFileSelect = (selected: File | null) => { const handleFileSelect = (selected: File | null) => {
if (!selected) return; if (!selected) return;
if (!selected.name.toLowerCase().endsWith('.zip')) { if (!isSupportedPagesPackage(selected.name)) {
toast.error('仅支持 zip 格式的文件'); toast.error('仅支持 zip、tar.gz、tar.xz、tar.bz2、tar、7z 格式的部署包');
return; return;
} }
setFile(selected); setFile(selected);
@@ -96,7 +117,8 @@ export function DeploymentUploadDialog({
<DialogHeader> <DialogHeader>
<DialogTitle>上传部署包</DialogTitle> <DialogTitle>上传部署包</DialogTitle>
<DialogDescription> <DialogDescription>
上传已构建的 zip 静态资源包,部署后可在列表中激活。 上传已构建的静态资源压缩包(zip / tar.gz / tar.xz / tar.bz2 / tar /
7z),部署后可在列表中激活。
</DialogDescription> </DialogDescription>
</DialogHeader> </DialogHeader>
@@ -121,7 +143,10 @@ export function DeploymentUploadDialog({
}} }}
> >
<UploadCloud className='size-8 mx-auto text-muted-foreground' /> <UploadCloud className='size-8 mx-auto text-muted-foreground' />
<p className='mt-3 text-sm'>拖拽 zip 文件到此处,或点击选择文件</p> <p className='mt-3 text-sm'>拖拽部署包到此处,或点击选择文件</p>
<p className='mt-1 text-xs text-muted-foreground'>
支持 zip、tar.gz、tar.xz、tar.bz2、tar、7z
</p>
<Button <Button
type='button' type='button'
variant='outline' variant='outline'
@@ -134,7 +159,7 @@ export function DeploymentUploadDialog({
<input <input
ref={fileInputRef} ref={fileInputRef}
type='file' type='file'
accept='.zip' accept={PAGES_PACKAGE_ACCEPT}
className='hidden' className='hidden'
onChange={(e) => handleFileSelect(e.target.files?.[0] ?? null)} onChange={(e) => handleFileSelect(e.target.files?.[0] ?? null)}
/> />
+2 -1
View File
@@ -30,7 +30,8 @@ export default function PagesPage() {
<div> <div>
<h1 className='text-2xl font-semibold tracking-tight'>Pages</h1> <h1 className='text-2xl font-semibold tracking-tight'>Pages</h1>
<p className='text-sm text-muted-foreground'> <p className='text-sm text-muted-foreground'>
边缘静态站点托管,上传 zip 部署包并在代理规则中选择 Pages 上游。 边缘静态站点托管,上传静态资源部署包并在代理规则中选择 Pages
上游。
</p> </p>
</div> </div>
</div> </div>
+19 -12
View File
@@ -10,6 +10,7 @@ require (
github.com/aws/aws-sdk-go-v2/config v1.32.16 github.com/aws/aws-sdk-go-v2/config v1.32.16
github.com/aws/aws-sdk-go-v2/credentials v1.19.15 github.com/aws/aws-sdk-go-v2/credentials v1.19.15
github.com/aws/aws-sdk-go-v2/service/s3 v1.99.1 github.com/aws/aws-sdk-go-v2/service/s3 v1.99.1
github.com/bodgit/sevenzip v1.6.5
github.com/bwmarrin/snowflake v0.3.0 github.com/bwmarrin/snowflake v0.3.0
github.com/coreos/go-oidc/v3 v3.17.0 github.com/coreos/go-oidc/v3 v3.17.0
github.com/deepteams/webp v1.2.3 github.com/deepteams/webp v1.2.3
@@ -39,7 +40,9 @@ require (
github.com/swaggo/files v1.0.1 github.com/swaggo/files v1.0.1
github.com/swaggo/gin-swagger v1.6.1 github.com/swaggo/gin-swagger v1.6.1
github.com/swaggo/swag v1.16.6 github.com/swaggo/swag v1.16.6
github.com/ulikunitz/xz v0.5.15
github.com/uptrace/opentelemetry-go-extra/otelzap v0.3.2 github.com/uptrace/opentelemetry-go-extra/otelzap v0.3.2
github.com/yuin/gopher-lua v1.1.1
go.opentelemetry.io/contrib/instrumentation/github.com/gin-gonic/gin/otelgin v0.61.0 go.opentelemetry.io/contrib/instrumentation/github.com/gin-gonic/gin/otelgin v0.61.0
go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp v0.68.0 go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp v0.68.0
go.opentelemetry.io/otel v1.43.0 go.opentelemetry.io/otel v1.43.0
@@ -47,14 +50,13 @@ require (
go.opentelemetry.io/otel/sdk v1.43.0 go.opentelemetry.io/otel/sdk v1.43.0
go.opentelemetry.io/otel/trace v1.43.0 go.opentelemetry.io/otel/trace v1.43.0
go.uber.org/zap v1.27.1 go.uber.org/zap v1.27.1
golang.org/x/crypto v0.51.0 golang.org/x/crypto v0.53.0
golang.org/x/image v0.42.0 golang.org/x/image v0.42.0
golang.org/x/mod v0.36.0 golang.org/x/mod v0.37.0
golang.org/x/net v0.54.0 golang.org/x/net v0.56.0
golang.org/x/oauth2 v0.36.0 golang.org/x/oauth2 v0.36.0
golang.org/x/sync v0.21.0 golang.org/x/sync v0.22.0
gopkg.in/natefinch/lumberjack.v2 v2.2.1 gopkg.in/natefinch/lumberjack.v2 v2.2.1
gorm.io/driver/clickhouse v0.7.0
gorm.io/driver/postgres v1.6.0 gorm.io/driver/postgres v1.6.0
gorm.io/driver/sqlite v1.6.0 gorm.io/driver/sqlite v1.6.0
gorm.io/gorm v1.31.1 gorm.io/gorm v1.31.1
@@ -66,7 +68,7 @@ require (
filippo.io/edwards25519 v1.2.0 // indirect filippo.io/edwards25519 v1.2.0 // indirect
github.com/ClickHouse/ch-go v0.71.0 // indirect github.com/ClickHouse/ch-go v0.71.0 // indirect
github.com/KyleBanks/depth v1.2.1 // indirect github.com/KyleBanks/depth v1.2.1 // indirect
github.com/andybalholm/brotli v1.2.1 // indirect github.com/andybalholm/brotli v1.2.2 // indirect
github.com/aws/aws-sdk-go-v2/aws/protocol/eventstream v1.7.9 // indirect github.com/aws/aws-sdk-go-v2/aws/protocol/eventstream v1.7.9 // indirect
github.com/aws/aws-sdk-go-v2/feature/ec2/imds v1.18.22 // indirect github.com/aws/aws-sdk-go-v2/feature/ec2/imds v1.18.22 // indirect
github.com/aws/aws-sdk-go-v2/internal/configsources v1.4.22 // indirect github.com/aws/aws-sdk-go-v2/internal/configsources v1.4.22 // indirect
@@ -81,6 +83,8 @@ require (
github.com/aws/aws-sdk-go-v2/service/ssooidc v1.35.20 // indirect github.com/aws/aws-sdk-go-v2/service/ssooidc v1.35.20 // indirect
github.com/aws/aws-sdk-go-v2/service/sts v1.42.0 // indirect github.com/aws/aws-sdk-go-v2/service/sts v1.42.0 // indirect
github.com/aws/smithy-go v1.25.0 // indirect github.com/aws/smithy-go v1.25.0 // indirect
github.com/bodgit/plumbing v1.3.0 // indirect
github.com/bodgit/windows v1.0.1 // indirect
github.com/boj/redistore v1.4.1 // indirect github.com/boj/redistore v1.4.1 // indirect
github.com/bytedance/gopkg v0.1.3 // indirect github.com/bytedance/gopkg v0.1.3 // indirect
github.com/bytedance/sonic v1.14.2 // indirect github.com/bytedance/sonic v1.14.2 // indirect
@@ -124,6 +128,7 @@ require (
github.com/gorilla/sessions v1.4.0 // indirect github.com/gorilla/sessions v1.4.0 // indirect
github.com/grpc-ecosystem/grpc-gateway/v2 v2.28.0 // indirect github.com/grpc-ecosystem/grpc-gateway/v2 v2.28.0 // indirect
github.com/hashicorp/go-version v1.9.0 // indirect github.com/hashicorp/go-version v1.9.0 // indirect
github.com/hashicorp/golang-lru/v2 v2.0.7 // indirect
github.com/inconshreveable/mousetrap v1.1.0 // indirect github.com/inconshreveable/mousetrap v1.1.0 // indirect
github.com/jackc/pgpassfile v1.0.0 // indirect github.com/jackc/pgpassfile v1.0.0 // indirect
github.com/jackc/pgservicefile v0.0.0-20240606120523-5a60cdf6a761 // indirect github.com/jackc/pgservicefile v0.0.0-20240606120523-5a60cdf6a761 // indirect
@@ -132,7 +137,7 @@ require (
github.com/jinzhu/inflection v1.0.0 // indirect github.com/jinzhu/inflection v1.0.0 // indirect
github.com/jinzhu/now v1.1.5 // indirect github.com/jinzhu/now v1.1.5 // indirect
github.com/json-iterator/go v1.1.13-0.20220915233716-71ac16282d12 // indirect github.com/json-iterator/go v1.1.13-0.20220915233716-71ac16282d12 // indirect
github.com/klauspost/compress v1.18.5 // indirect github.com/klauspost/compress v1.19.0 // indirect
github.com/klauspost/cpuid/v2 v2.3.0 // indirect github.com/klauspost/cpuid/v2 v2.3.0 // indirect
github.com/leodido/go-urn v1.4.0 // indirect github.com/leodido/go-urn v1.4.0 // indirect
github.com/mattn/go-isatty v0.0.21 // indirect github.com/mattn/go-isatty v0.0.21 // indirect
@@ -144,7 +149,7 @@ require (
github.com/ncruces/go-strftime v1.0.0 // indirect github.com/ncruces/go-strftime v1.0.0 // indirect
github.com/paulmach/orb v0.13.0 // indirect github.com/paulmach/orb v0.13.0 // indirect
github.com/pelletier/go-toml/v2 v2.2.4 // indirect github.com/pelletier/go-toml/v2 v2.2.4 // indirect
github.com/pierrec/lz4/v4 v4.1.26 // indirect github.com/pierrec/lz4/v4 v4.1.27 // indirect
github.com/pmezard/go-difflib v1.0.1-0.20181226105442-5d4384ee4fb2 // indirect github.com/pmezard/go-difflib v1.0.1-0.20181226105442-5d4384ee4fb2 // indirect
github.com/quic-go/qpack v0.5.1 // indirect github.com/quic-go/qpack v0.5.1 // indirect
github.com/quic-go/quic-go v0.55.0 // indirect github.com/quic-go/quic-go v0.55.0 // indirect
@@ -156,11 +161,11 @@ require (
github.com/spf13/afero v1.15.0 // indirect github.com/spf13/afero v1.15.0 // indirect
github.com/spf13/cast v1.10.0 // indirect github.com/spf13/cast v1.10.0 // indirect
github.com/spf13/pflag v1.0.10 // indirect github.com/spf13/pflag v1.0.10 // indirect
github.com/stangelandcl/ppmd v0.1.1 // indirect
github.com/subosito/gotenv v1.6.0 // indirect github.com/subosito/gotenv v1.6.0 // indirect
github.com/twitchyliquid64/golang-asm v0.15.1 // indirect github.com/twitchyliquid64/golang-asm v0.15.1 // indirect
github.com/ugorji/go/codec v1.3.1 // indirect github.com/ugorji/go/codec v1.3.1 // indirect
github.com/uptrace/opentelemetry-go-extra/otelutil v0.3.2 // indirect github.com/uptrace/opentelemetry-go-extra/otelutil v0.3.2 // indirect
github.com/yuin/gopher-lua v1.1.1 // indirect
go.opentelemetry.io/auto/sdk v1.2.1 // indirect go.opentelemetry.io/auto/sdk v1.2.1 // indirect
go.opentelemetry.io/otel/exporters/otlp/otlptrace v1.43.0 // indirect go.opentelemetry.io/otel/exporters/otlp/otlptrace v1.43.0 // indirect
go.opentelemetry.io/otel/log v0.12.2 // indirect go.opentelemetry.io/otel/log v0.12.2 // indirect
@@ -169,16 +174,18 @@ require (
go.uber.org/mock v0.6.0 // indirect go.uber.org/mock v0.6.0 // indirect
go.uber.org/multierr v1.11.0 // indirect go.uber.org/multierr v1.11.0 // indirect
go.yaml.in/yaml/v3 v3.0.4 // indirect go.yaml.in/yaml/v3 v3.0.4 // indirect
go4.org v0.0.0-20260112195520-a5071408f32f // indirect
golang.org/x/arch v0.22.0 // indirect golang.org/x/arch v0.22.0 // indirect
golang.org/x/sys v0.44.0 // indirect golang.org/x/sys v0.46.0 // indirect
golang.org/x/text v0.38.0 // indirect golang.org/x/text v0.40.0 // indirect
golang.org/x/time v0.15.0 // indirect golang.org/x/time v0.15.0 // indirect
golang.org/x/tools v0.45.0 // indirect golang.org/x/tools v0.47.0 // indirect
google.golang.org/genproto/googleapis/api v0.0.0-20260401024825-9d38bb4040a9 // indirect google.golang.org/genproto/googleapis/api v0.0.0-20260401024825-9d38bb4040a9 // indirect
google.golang.org/genproto/googleapis/rpc v0.0.0-20260420184626-e10c466a9529 // indirect google.golang.org/genproto/googleapis/rpc v0.0.0-20260420184626-e10c466a9529 // indirect
google.golang.org/grpc v1.80.0 // indirect google.golang.org/grpc v1.80.0 // indirect
google.golang.org/protobuf v1.36.11 // indirect google.golang.org/protobuf v1.36.11 // indirect
gopkg.in/yaml.v3 v3.0.1 // indirect gopkg.in/yaml.v3 v3.0.1 // indirect
gorm.io/driver/clickhouse v0.7.0 // indirect
gorm.io/driver/mysql v1.6.0 // indirect gorm.io/driver/mysql v1.6.0 // indirect
modernc.org/libc v1.72.1 // indirect modernc.org/libc v1.72.1 // indirect
modernc.org/mathutil v1.7.1 // indirect modernc.org/mathutil v1.7.1 // indirect
+35 -20
View File
@@ -10,8 +10,8 @@ github.com/alicebob/miniredis/v2 v2.38.0 h1:nZAzCR+Lj+Vxk4ZXzm2NuKq2O33RXj1XxJ2e
github.com/alicebob/miniredis/v2 v2.38.0/go.mod h1:TcL7YfarKPGDAthEtl5NBeHZfeUQj6OXMm/+iu5cLMM= github.com/alicebob/miniredis/v2 v2.38.0/go.mod h1:TcL7YfarKPGDAthEtl5NBeHZfeUQj6OXMm/+iu5cLMM=
github.com/aliyun/alibabacloud-oss-go-sdk-v2 v1.5.1 h1:vtiFd0hhPAbyYJjztl0wYUq/PqEGkIlDmVuTIy6zw8Y= github.com/aliyun/alibabacloud-oss-go-sdk-v2 v1.5.1 h1:vtiFd0hhPAbyYJjztl0wYUq/PqEGkIlDmVuTIy6zw8Y=
github.com/aliyun/alibabacloud-oss-go-sdk-v2 v1.5.1/go.mod h1:FTzydeQVmR24FI0D6XWUOMKckjXehM/jgMn1xC+DA9M= github.com/aliyun/alibabacloud-oss-go-sdk-v2 v1.5.1/go.mod h1:FTzydeQVmR24FI0D6XWUOMKckjXehM/jgMn1xC+DA9M=
github.com/andybalholm/brotli v1.2.1 h1:R+f5xP285VArJDRgowrfb9DqL18yVK0gKAW/F+eTWro= github.com/andybalholm/brotli v1.2.2 h1:HzTuoo2ErYQqf5qvcJInB8uvqSVxRttzkFexPWtnceM=
github.com/andybalholm/brotli v1.2.1/go.mod h1:rzTDkvFWvIrjDXZHkuS16NPggd91W3kUSvPlQ1pLaKY= github.com/andybalholm/brotli v1.2.2/go.mod h1:rzTDkvFWvIrjDXZHkuS16NPggd91W3kUSvPlQ1pLaKY=
github.com/aws/aws-sdk-go-v2 v1.41.6 h1:1AX0AthnBQzMx1vbmir3Y4WsnJgiydmnJjiLu+LvXOg= github.com/aws/aws-sdk-go-v2 v1.41.6 h1:1AX0AthnBQzMx1vbmir3Y4WsnJgiydmnJjiLu+LvXOg=
github.com/aws/aws-sdk-go-v2 v1.41.6/go.mod h1:dy0UzBIfwSeot4grGvY1AqFWN5zgziMmWGzysDnHFcQ= github.com/aws/aws-sdk-go-v2 v1.41.6/go.mod h1:dy0UzBIfwSeot4grGvY1AqFWN5zgziMmWGzysDnHFcQ=
github.com/aws/aws-sdk-go-v2/aws/protocol/eventstream v1.7.9 h1:adBsCIIpLbLmYnkQU+nAChU5yhVTvu5PerROm+/Kq2A= github.com/aws/aws-sdk-go-v2/aws/protocol/eventstream v1.7.9 h1:adBsCIIpLbLmYnkQU+nAChU5yhVTvu5PerROm+/Kq2A=
@@ -48,6 +48,12 @@ github.com/aws/aws-sdk-go-v2/service/sts v1.42.0 h1:ks8KBcZPh3PYISr5dAiXCM5/Thcu
github.com/aws/aws-sdk-go-v2/service/sts v1.42.0/go.mod h1:pFw33T0WLvXU3rw1WBkpMlkgIn54eCB5FYLhjDc9Foo= github.com/aws/aws-sdk-go-v2/service/sts v1.42.0/go.mod h1:pFw33T0WLvXU3rw1WBkpMlkgIn54eCB5FYLhjDc9Foo=
github.com/aws/smithy-go v1.25.0 h1:Sz/XJ64rwuiKtB6j98nDIPyYrV1nVNJ4YU74gttcl5U= github.com/aws/smithy-go v1.25.0 h1:Sz/XJ64rwuiKtB6j98nDIPyYrV1nVNJ4YU74gttcl5U=
github.com/aws/smithy-go v1.25.0/go.mod h1:YE2RhdIuDbA5E5bTdciG9KrW3+TiEONeUWCqxX9i1Fc= github.com/aws/smithy-go v1.25.0/go.mod h1:YE2RhdIuDbA5E5bTdciG9KrW3+TiEONeUWCqxX9i1Fc=
github.com/bodgit/plumbing v1.3.0 h1:pf9Itz1JOQgn7vEOE7v7nlEfBykYqvUYioC61TwWCFU=
github.com/bodgit/plumbing v1.3.0/go.mod h1:JOTb4XiRu5xfnmdnDJo6GmSbSbtSyufrsyZFByMtKEs=
github.com/bodgit/sevenzip v1.6.5 h1:7H7BxgmeX0j6UX42lH+KXQ92WgMQJ49DoocFdfHbCng=
github.com/bodgit/sevenzip v1.6.5/go.mod h1:GhuB6Lq1xCpP1sps+horjZ8lgiKPJcy2zUX3prla9wc=
github.com/bodgit/windows v1.0.1 h1:tF7K6KOluPYygXa3Z2594zxlkbKPAOvqr97etrGNIz4=
github.com/bodgit/windows v1.0.1/go.mod h1:a6JLwrB4KrTR5hBpp8FI9/9W9jJfeQ2h4XDXU74ZCdM=
github.com/boj/redistore v1.4.1 h1:lP9ZZWqKMq2RIqexlZX1w1ODSnegL+puxGIujkU5tIw= github.com/boj/redistore v1.4.1 h1:lP9ZZWqKMq2RIqexlZX1w1ODSnegL+puxGIujkU5tIw=
github.com/boj/redistore v1.4.1/go.mod h1:c0Tvw6aMjslog4jHIAcNv6EtJM849YoOAhMY7JBbWpI= github.com/boj/redistore v1.4.1/go.mod h1:c0Tvw6aMjslog4jHIAcNv6EtJM849YoOAhMY7JBbWpI=
github.com/bsm/ginkgo/v2 v2.12.0 h1:Ny8MWAHyOepLGlLKYmXG4IEkioBysk6GpaRTLC8zwWs= github.com/bsm/ginkgo/v2 v2.12.0 h1:Ny8MWAHyOepLGlLKYmXG4IEkioBysk6GpaRTLC8zwWs=
@@ -206,8 +212,8 @@ github.com/jinzhu/now v1.1.5 h1:/o9tlHleP7gOFmsnYNz3RGnqzefHA47wQpKrrdTIwXQ=
github.com/jinzhu/now v1.1.5/go.mod h1:d3SSVoowX0Lcu0IBviAWJpolVfI5UJVZZ7cO71lE/z8= github.com/jinzhu/now v1.1.5/go.mod h1:d3SSVoowX0Lcu0IBviAWJpolVfI5UJVZZ7cO71lE/z8=
github.com/json-iterator/go v1.1.13-0.20220915233716-71ac16282d12 h1:9Nu54bhS/H/Kgo2/7xNSUuC5G28VR8ljfrLKU2G4IjU= github.com/json-iterator/go v1.1.13-0.20220915233716-71ac16282d12 h1:9Nu54bhS/H/Kgo2/7xNSUuC5G28VR8ljfrLKU2G4IjU=
github.com/json-iterator/go v1.1.13-0.20220915233716-71ac16282d12/go.mod h1:TBzl5BIHNXfS9+C35ZyJaklL7mLDbgUkcgXzSLa8Tk0= github.com/json-iterator/go v1.1.13-0.20220915233716-71ac16282d12/go.mod h1:TBzl5BIHNXfS9+C35ZyJaklL7mLDbgUkcgXzSLa8Tk0=
github.com/klauspost/compress v1.18.5 h1:/h1gH5Ce+VWNLSWqPzOVn6XBO+vJbCNGvjoaGBFW2IE= github.com/klauspost/compress v1.19.0 h1:sXLILfc9jV2QYWkzFOPWStmcUVH2RHEB1JCdY2oVvCQ=
github.com/klauspost/compress v1.18.5/go.mod h1:cwPg85FWrGar70rWktvGQj8/hthj3wpl0PGDogxkrSQ= github.com/klauspost/compress v1.19.0/go.mod h1:cwPg85FWrGar70rWktvGQj8/hthj3wpl0PGDogxkrSQ=
github.com/klauspost/cpuid/v2 v2.3.0 h1:S4CRMLnYUhGeDFDqkGriYKdfoFlDnMtqTiI/sFzhA9Y= github.com/klauspost/cpuid/v2 v2.3.0 h1:S4CRMLnYUhGeDFDqkGriYKdfoFlDnMtqTiI/sFzhA9Y=
github.com/klauspost/cpuid/v2 v2.3.0/go.mod h1:hqwkgyIinND0mEev00jJYCxPNVRVXFQeu1XKlok6oO0= github.com/klauspost/cpuid/v2 v2.3.0/go.mod h1:hqwkgyIinND0mEev00jJYCxPNVRVXFQeu1XKlok6oO0=
github.com/kr/pretty v0.3.1 h1:flRD4NNwYAUpkphVc1HcthR4KEIFJ65n8Mw5qdRn3LE= github.com/kr/pretty v0.3.1 h1:flRD4NNwYAUpkphVc1HcthR4KEIFJ65n8Mw5qdRn3LE=
@@ -242,8 +248,8 @@ github.com/pelletier/go-toml/v2 v2.2.4 h1:mye9XuhQ6gvn5h28+VilKrrPoQVanw5PMw/TB0
github.com/pelletier/go-toml/v2 v2.2.4/go.mod h1:2gIqNv+qfxSVS7cM2xJQKtLSTLUE9V8t9Stt+h56mCY= github.com/pelletier/go-toml/v2 v2.2.4/go.mod h1:2gIqNv+qfxSVS7cM2xJQKtLSTLUE9V8t9Stt+h56mCY=
github.com/peterbourgon/diskv/v3 v3.0.1 h1:x06SQA46+PKIUftmEujdwSEpIx8kR+M9eLYsUxeYveU= github.com/peterbourgon/diskv/v3 v3.0.1 h1:x06SQA46+PKIUftmEujdwSEpIx8kR+M9eLYsUxeYveU=
github.com/peterbourgon/diskv/v3 v3.0.1/go.mod h1:kJ5Ny7vLdARGU3WUuy6uzO6T0nb/2gWcT1JiBvRmb5o= github.com/peterbourgon/diskv/v3 v3.0.1/go.mod h1:kJ5Ny7vLdARGU3WUuy6uzO6T0nb/2gWcT1JiBvRmb5o=
github.com/pierrec/lz4/v4 v4.1.26 h1:GrpZw1gZttORinvzBdXPUXATeqlJjqUG/D87TKMnhjY= github.com/pierrec/lz4/v4 v4.1.27 h1:+PhzhWDrjRj89TH2sw43nE3+4+W8lSxIuQadEHZyjUk=
github.com/pierrec/lz4/v4 v4.1.26/go.mod h1:EoQMVJgeeEOMsCqCzqFm2O0cJvljX2nGZjcRIPL34O4= github.com/pierrec/lz4/v4 v4.1.27/go.mod h1:EoQMVJgeeEOMsCqCzqFm2O0cJvljX2nGZjcRIPL34O4=
github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4= github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4=
github.com/pmezard/go-difflib v1.0.1-0.20181226105442-5d4384ee4fb2 h1:Jamvg5psRIccs7FGNTlIRMkT8wgtp5eCXdBlqhYGL6U= github.com/pmezard/go-difflib v1.0.1-0.20181226105442-5d4384ee4fb2 h1:Jamvg5psRIccs7FGNTlIRMkT8wgtp5eCXdBlqhYGL6U=
github.com/pmezard/go-difflib v1.0.1-0.20181226105442-5d4384ee4fb2/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4= github.com/pmezard/go-difflib v1.0.1-0.20181226105442-5d4384ee4fb2/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4=
@@ -285,14 +291,19 @@ github.com/spf13/pflag v1.0.10 h1:4EBh2KAYBwaONj6b2Ye1GiHfwjqyROoF4RwYO+vPwFk=
github.com/spf13/pflag v1.0.10/go.mod h1:McXfInJRrz4CZXVZOBLb0bTZqETkiAhM9Iw0y3An2Bg= github.com/spf13/pflag v1.0.10/go.mod h1:McXfInJRrz4CZXVZOBLb0bTZqETkiAhM9Iw0y3An2Bg=
github.com/spf13/viper v1.21.0 h1:x5S+0EU27Lbphp4UKm1C+1oQO+rKx36vfCoaVebLFSU= github.com/spf13/viper v1.21.0 h1:x5S+0EU27Lbphp4UKm1C+1oQO+rKx36vfCoaVebLFSU=
github.com/spf13/viper v1.21.0/go.mod h1:P0lhsswPGWD/1lZJ9ny3fYnVqxiegrlNrEmgLjbTCAY= github.com/spf13/viper v1.21.0/go.mod h1:P0lhsswPGWD/1lZJ9ny3fYnVqxiegrlNrEmgLjbTCAY=
github.com/stangelandcl/ppmd v0.1.1 h1:c25QazhlWUn5nmR1QOzafKhQxBicAr7GGCKER2aJ8H8=
github.com/stangelandcl/ppmd v0.1.1/go.mod h1:Rrv7M+/2P5jYr/GMLhBl7Ug3uJ1bUiVzr5LbbaV6xgY=
github.com/stretchr/objx v0.1.0/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+wExME= github.com/stretchr/objx v0.1.0/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+wExME=
github.com/stretchr/objx v0.4.0/go.mod h1:YvHI0jy2hoMjB+UWwv71VJQ9isScKT/TqJzVSSt89Yw= github.com/stretchr/objx v0.4.0/go.mod h1:YvHI0jy2hoMjB+UWwv71VJQ9isScKT/TqJzVSSt89Yw=
github.com/stretchr/objx v0.5.0/go.mod h1:Yh+to48EsGEfYuaHDzXPcE3xhTkx73EhmCGUpEOglKo= github.com/stretchr/objx v0.5.0/go.mod h1:Yh+to48EsGEfYuaHDzXPcE3xhTkx73EhmCGUpEOglKo=
github.com/stretchr/objx v0.5.2/go.mod h1:FRsXN1f5AsAjCGJKqEizvkpNtU+EGNCLh3NxZ/8L+MA= github.com/stretchr/objx v0.5.2/go.mod h1:FRsXN1f5AsAjCGJKqEizvkpNtU+EGNCLh3NxZ/8L+MA=
github.com/stretchr/objx v0.5.3 h1:jmXUvGomnU1o3W/V5h2VEradbpJDwGrzugQQvL0POH4=
github.com/stretchr/objx v0.5.3/go.mod h1:rDQraq+vQZU7Fde9LOZLr8Tax6zZvy4kuNKF+QYS+U0=
github.com/stretchr/testify v1.3.0/go.mod h1:M5WIy9Dh21IEIfnGCwXGc5bZfKNJtfHm1UVUgZn+9EI= github.com/stretchr/testify v1.3.0/go.mod h1:M5WIy9Dh21IEIfnGCwXGc5bZfKNJtfHm1UVUgZn+9EI=
github.com/stretchr/testify v1.7.0/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/h/Wwjteg= github.com/stretchr/testify v1.7.0/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/h/Wwjteg=
github.com/stretchr/testify v1.7.1/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/h/Wwjteg= github.com/stretchr/testify v1.7.1/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/h/Wwjteg=
github.com/stretchr/testify v1.8.0/go.mod h1:yNjHg4UonilssWZ8iaSj1OCr/vHnekPRkoO+kdMU+MU= github.com/stretchr/testify v1.8.0/go.mod h1:yNjHg4UonilssWZ8iaSj1OCr/vHnekPRkoO+kdMU+MU=
github.com/stretchr/testify v1.8.1/go.mod h1:w2LPCIKwWwSfY2zedu0+kehJoqGctiVI29o6fzry7u4=
github.com/stretchr/testify v1.8.4/go.mod h1:sz/lmYIOXD/1dqDmKjjqLyZ2RngseejIcXlSw2iwfAo= github.com/stretchr/testify v1.8.4/go.mod h1:sz/lmYIOXD/1dqDmKjjqLyZ2RngseejIcXlSw2iwfAo=
github.com/stretchr/testify v1.10.0/go.mod h1:r2ic/lqez/lEtzL7wO/rwa5dbSLXVDPFyf8C91i36aY= github.com/stretchr/testify v1.10.0/go.mod h1:r2ic/lqez/lEtzL7wO/rwa5dbSLXVDPFyf8C91i36aY=
github.com/stretchr/testify v1.11.1 h1:7s2iGBzp5EwR7/aIZr8ao5+dra3wiQyKjjFuvgVKu7U= github.com/stretchr/testify v1.11.1 h1:7s2iGBzp5EwR7/aIZr8ao5+dra3wiQyKjjFuvgVKu7U=
@@ -311,6 +322,8 @@ github.com/twitchyliquid64/golang-asm v0.15.1 h1:SU5vSMR7hnwNxj24w34ZyCi/FmDZTkS
github.com/twitchyliquid64/golang-asm v0.15.1/go.mod h1:a1lVb/DtPvCB8fslRZhAngC2+aY1QWCk3Cedj/Gdt08= github.com/twitchyliquid64/golang-asm v0.15.1/go.mod h1:a1lVb/DtPvCB8fslRZhAngC2+aY1QWCk3Cedj/Gdt08=
github.com/ugorji/go/codec v1.3.1 h1:waO7eEiFDwidsBN6agj1vJQ4AG7lh2yqXyOXqhgQuyY= github.com/ugorji/go/codec v1.3.1 h1:waO7eEiFDwidsBN6agj1vJQ4AG7lh2yqXyOXqhgQuyY=
github.com/ugorji/go/codec v1.3.1/go.mod h1:pRBVtBSKl77K30Bv8R2P+cLSGaTtex6fsA2Wjqmfxj4= github.com/ugorji/go/codec v1.3.1/go.mod h1:pRBVtBSKl77K30Bv8R2P+cLSGaTtex6fsA2Wjqmfxj4=
github.com/ulikunitz/xz v0.5.15 h1:9DNdB5s+SgV3bQ2ApL10xRc35ck0DuIX/isZvIk+ubY=
github.com/ulikunitz/xz v0.5.15/go.mod h1:nbz6k7qbPmH4IRqmfOplQw/tblSgqTqBwxkY0oWt/14=
github.com/uptrace/opentelemetry-go-extra/otelutil v0.3.2 h1:3/aHKUq7qaFMWxyQV0W2ryNgg8x8rVeKVA20KJUkfS0= github.com/uptrace/opentelemetry-go-extra/otelutil v0.3.2 h1:3/aHKUq7qaFMWxyQV0W2ryNgg8x8rVeKVA20KJUkfS0=
github.com/uptrace/opentelemetry-go-extra/otelutil v0.3.2/go.mod h1:Zit4b8AQXaXvA68+nzmbyDzqiyFRISyw1JiD5JqUBjw= github.com/uptrace/opentelemetry-go-extra/otelutil v0.3.2/go.mod h1:Zit4b8AQXaXvA68+nzmbyDzqiyFRISyw1JiD5JqUBjw=
github.com/uptrace/opentelemetry-go-extra/otelzap v0.3.2 h1:cj/Z6FKTTYBnstI0Lni9PA+k2foounKIPUmj1LBwNiQ= github.com/uptrace/opentelemetry-go-extra/otelzap v0.3.2 h1:cj/Z6FKTTYBnstI0Lni9PA+k2foounKIPUmj1LBwNiQ=
@@ -356,37 +369,39 @@ go.uber.org/zap v1.27.1 h1:08RqriUEv8+ArZRYSTXy1LeBScaMpVSTBhCeaZYfMYc=
go.uber.org/zap v1.27.1/go.mod h1:GB2qFLM7cTU87MWRP2mPIjqfIDnGu+VIO4V/SdhGo2E= go.uber.org/zap v1.27.1/go.mod h1:GB2qFLM7cTU87MWRP2mPIjqfIDnGu+VIO4V/SdhGo2E=
go.yaml.in/yaml/v3 v3.0.4 h1:tfq32ie2Jv2UxXFdLJdh3jXuOzWiL1fo0bu/FbuKpbc= go.yaml.in/yaml/v3 v3.0.4 h1:tfq32ie2Jv2UxXFdLJdh3jXuOzWiL1fo0bu/FbuKpbc=
go.yaml.in/yaml/v3 v3.0.4/go.mod h1:DhzuOOF2ATzADvBadXxruRBLzYTpT36CKvDb3+aBEFg= go.yaml.in/yaml/v3 v3.0.4/go.mod h1:DhzuOOF2ATzADvBadXxruRBLzYTpT36CKvDb3+aBEFg=
go4.org v0.0.0-20260112195520-a5071408f32f h1:ziUVAjmTPwQMBmYR1tbdRFJPtTcQUI12fH9QQjfb0Sw=
go4.org v0.0.0-20260112195520-a5071408f32f/go.mod h1:ZRJnO5ZI4zAwMFp+dS1+V6J6MSyAowhRqAE+DPa1Xp0=
golang.org/x/arch v0.22.0 h1:c/Zle32i5ttqRXjdLyyHZESLD/bB90DCU1g9l/0YBDI= golang.org/x/arch v0.22.0 h1:c/Zle32i5ttqRXjdLyyHZESLD/bB90DCU1g9l/0YBDI=
golang.org/x/arch v0.22.0/go.mod h1:dNHoOeKiyja7GTvF9NJS1l3Z2yntpQNzgrjh1cU103A= golang.org/x/arch v0.22.0/go.mod h1:dNHoOeKiyja7GTvF9NJS1l3Z2yntpQNzgrjh1cU103A=
golang.org/x/crypto v0.0.0-20190308221718-c2843e01d9a2/go.mod h1:djNgcEr1/C05ACkg1iLfiJU5Ep61QUkGW8qpdssI0+w= golang.org/x/crypto v0.0.0-20190308221718-c2843e01d9a2/go.mod h1:djNgcEr1/C05ACkg1iLfiJU5Ep61QUkGW8qpdssI0+w=
golang.org/x/crypto v0.0.0-20210921155107-089bfa567519/go.mod h1:GvvjBRRGRdwPK5ydBHafDWAxML/pGHZbMvKqRZ5+Abc= golang.org/x/crypto v0.0.0-20210921155107-089bfa567519/go.mod h1:GvvjBRRGRdwPK5ydBHafDWAxML/pGHZbMvKqRZ5+Abc=
golang.org/x/crypto v0.51.0 h1:IBPXwPfKxY7cWQZ38ZCIRPI50YLeevDLlLnyC5wRGTI= golang.org/x/crypto v0.53.0 h1:QZ4Muo8THX6CizN2vPPd5fBGHyogrdK9fG4wLPFUsto=
golang.org/x/crypto v0.51.0/go.mod h1:8AdwkbraGNABw2kOX6YFPs3WM22XqI4EXEd8g+x7Oc8= golang.org/x/crypto v0.53.0/go.mod h1:DNLU434OwVakk9PzuwV8w62mAJpRJL3vsgcfp4Qnsio=
golang.org/x/image v0.42.0 h1:1gSs6ehNWXLbkHBIPcWztk3D/6aIA/8hauiAYtlodVY= golang.org/x/image v0.42.0 h1:1gSs6ehNWXLbkHBIPcWztk3D/6aIA/8hauiAYtlodVY=
golang.org/x/image v0.42.0/go.mod h1:rrpelvGFt+kLPAjPM4HeWPgrl0FtafueU//e5N0qk/Q= golang.org/x/image v0.42.0/go.mod h1:rrpelvGFt+kLPAjPM4HeWPgrl0FtafueU//e5N0qk/Q=
golang.org/x/mod v0.6.0-dev.0.20220419223038-86c51ed26bb4/go.mod h1:jJ57K6gSWd91VN4djpZkiMVwK6gcyfeH4XE8wZrZaV4= golang.org/x/mod v0.6.0-dev.0.20220419223038-86c51ed26bb4/go.mod h1:jJ57K6gSWd91VN4djpZkiMVwK6gcyfeH4XE8wZrZaV4=
golang.org/x/mod v0.36.0 h1:JJjpVx6myfUsUdAzZuOSTTmRE0PfZeNWzzvKrP7amb4= golang.org/x/mod v0.37.0 h1:vF1DjpVEshcIqoEaauuHebaLk1O1forxjxBaVn884JQ=
golang.org/x/mod v0.36.0/go.mod h1:moc6ELqsWcOw5Ef3xVprK5ul/MvtVvkIXLziUOICjUQ= golang.org/x/mod v0.37.0/go.mod h1:m8S8VeM9r4dzDwjrKO0a1sZP3YjeMamRRlD+fmR2Q/0=
golang.org/x/net v0.0.0-20190620200207-3b0461eec859/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s= golang.org/x/net v0.0.0-20190620200207-3b0461eec859/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s=
golang.org/x/net v0.0.0-20210226172049-e18ecbb05110/go.mod h1:m0MpNAwzfU5UDzcl9v0D8zg8gWTRqZa9RBIspLL5mdg= golang.org/x/net v0.0.0-20210226172049-e18ecbb05110/go.mod h1:m0MpNAwzfU5UDzcl9v0D8zg8gWTRqZa9RBIspLL5mdg=
golang.org/x/net v0.0.0-20220722155237-a158d28d115b/go.mod h1:XRhObCWvk6IyKnWLug+ECip1KBveYUHfp+8e9klMJ9c= golang.org/x/net v0.0.0-20220722155237-a158d28d115b/go.mod h1:XRhObCWvk6IyKnWLug+ECip1KBveYUHfp+8e9klMJ9c=
golang.org/x/net v0.7.0/go.mod h1:2Tu9+aMcznHK/AK1HMvgo6xiTLG5rD5rZLDS+rp2Bjs= golang.org/x/net v0.7.0/go.mod h1:2Tu9+aMcznHK/AK1HMvgo6xiTLG5rD5rZLDS+rp2Bjs=
golang.org/x/net v0.54.0 h1:2zJIZAxAHV/OHCDTCOHAYehQzLfSXuf/5SoL/Dv6w/w= golang.org/x/net v0.56.0 h1:Rw8j/hFzGvJUZwNBXnAtf5sVDVt+65SK2C7IxCxZt5o=
golang.org/x/net v0.54.0/go.mod h1:Sj4oj8jK6XmHpBZU/zWHw3BV3abl4Kvi+Ut7cQcY+cQ= golang.org/x/net v0.56.0/go.mod h1:D3Ku6r+V6JROoZK144D2XfMHFcMq/0zSfLelVTCFKec=
golang.org/x/oauth2 v0.36.0 h1:peZ/1z27fi9hUOFCAZaHyrpWG5lwe0RJEEEeH0ThlIs= golang.org/x/oauth2 v0.36.0 h1:peZ/1z27fi9hUOFCAZaHyrpWG5lwe0RJEEEeH0ThlIs=
golang.org/x/oauth2 v0.36.0/go.mod h1:YDBUJMTkDnJS+A4BP4eZBjCqtokkg1hODuPjwiGPO7Q= golang.org/x/oauth2 v0.36.0/go.mod h1:YDBUJMTkDnJS+A4BP4eZBjCqtokkg1hODuPjwiGPO7Q=
golang.org/x/sync v0.0.0-20190423024810-112230192c58/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM= golang.org/x/sync v0.0.0-20190423024810-112230192c58/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sync v0.0.0-20220722155255-886fb9371eb4/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM= golang.org/x/sync v0.0.0-20220722155255-886fb9371eb4/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sync v0.21.0 h1:HLII4xRRTtCRkxYp4HNFF0Js/Og6q2i++KXbg0gHCwM= golang.org/x/sync v0.22.0 h1:SZjpbeLmrCk4xhRSZFNZW5gFUeCeFgjekvI/+gfScek=
golang.org/x/sync v0.21.0/go.mod h1:9xrNwdLfx4jkKbNva9FpL6vEN7evnE43NNNJQ2LF3+0= golang.org/x/sync v0.22.0/go.mod h1:9xrNwdLfx4jkKbNva9FpL6vEN7evnE43NNNJQ2LF3+0=
golang.org/x/sys v0.0.0-20190215142949-d0b11bdaac8a/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY= golang.org/x/sys v0.0.0-20190215142949-d0b11bdaac8a/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
golang.org/x/sys v0.0.0-20201119102817-f84b799fce68/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= golang.org/x/sys v0.0.0-20201119102817-f84b799fce68/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20210615035016-665e8c7367d1/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= golang.org/x/sys v0.0.0-20210615035016-665e8c7367d1/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.0.0-20220520151302-bc2c85ada10a/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= golang.org/x/sys v0.0.0-20220520151302-bc2c85ada10a/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.0.0-20220722155257-8c9f86f7a55f/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= golang.org/x/sys v0.0.0-20220722155257-8c9f86f7a55f/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.5.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= golang.org/x/sys v0.5.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.44.0 h1:ildZl3J4uzeKP07r2F++Op7E9B29JRUy+a27EibtBTQ= golang.org/x/sys v0.46.0 h1:noSf2Fq6F8DBgS+LysIkx7rIExoNHJsxOAtPp4rthXw=
golang.org/x/sys v0.44.0/go.mod h1:4GL1E5IUh+htKOUEOaiffhrAeqysfVGipDYzABqnCmw= golang.org/x/sys v0.46.0/go.mod h1:4GL1E5IUh+htKOUEOaiffhrAeqysfVGipDYzABqnCmw=
golang.org/x/term v0.0.0-20201126162022-7de9c90e9dd1/go.mod h1:bj7SfCRtBDWHUb9snDiAeCFNEtKQo2Wmx5Cou7ajbmo= golang.org/x/term v0.0.0-20201126162022-7de9c90e9dd1/go.mod h1:bj7SfCRtBDWHUb9snDiAeCFNEtKQo2Wmx5Cou7ajbmo=
golang.org/x/term v0.0.0-20210927222741-03fcf44c2211/go.mod h1:jbD1KX2456YbFQfuXm/mYQcufACuNUgVhRMnK/tPxf8= golang.org/x/term v0.0.0-20210927222741-03fcf44c2211/go.mod h1:jbD1KX2456YbFQfuXm/mYQcufACuNUgVhRMnK/tPxf8=
golang.org/x/term v0.5.0/go.mod h1:jMB1sMXY+tzblOD4FWmEbocvup2/aLOaQEp7JmGp78k= golang.org/x/term v0.5.0/go.mod h1:jMB1sMXY+tzblOD4FWmEbocvup2/aLOaQEp7JmGp78k=
@@ -394,15 +409,15 @@ golang.org/x/text v0.3.0/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ=
golang.org/x/text v0.3.3/go.mod h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ= golang.org/x/text v0.3.3/go.mod h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ=
golang.org/x/text v0.3.7/go.mod h1:u+2+/6zg+i71rQMx5EYifcz6MCKuco9NR6JIITiCfzQ= golang.org/x/text v0.3.7/go.mod h1:u+2+/6zg+i71rQMx5EYifcz6MCKuco9NR6JIITiCfzQ=
golang.org/x/text v0.7.0/go.mod h1:mrYo+phRRbMaCq/xk9113O4dZlRixOauAjOtrjsXDZ8= golang.org/x/text v0.7.0/go.mod h1:mrYo+phRRbMaCq/xk9113O4dZlRixOauAjOtrjsXDZ8=
golang.org/x/text v0.38.0 h1:sXmwo9DwP3OK9EZ7PqAdaooSGozfl/3a6/xJcbzPRhE= golang.org/x/text v0.40.0 h1:Ub2Z6/xjgF1WrYQz2nuITOEegKFtiIy+rieRJ5lHZKs=
golang.org/x/text v0.38.0/go.mod h1:YXZt3QhHUKYT53r2lLKFIVi6Ao1jdzrTR/KQ09qyxF4= golang.org/x/text v0.40.0/go.mod h1:hpnzDAfGV753zIKo+wk3u1bVKCGPbrnF7+7LBF/UHVY=
golang.org/x/time v0.15.0 h1:bbrp8t3bGUeFOx08pvsMYRTCVSMk89u4tKbNOZbp88U= golang.org/x/time v0.15.0 h1:bbrp8t3bGUeFOx08pvsMYRTCVSMk89u4tKbNOZbp88U=
golang.org/x/time v0.15.0/go.mod h1:Y4YMaQmXwGQZoFaVFk4YpCt4FLQMYKZe9oeV/f4MSno= golang.org/x/time v0.15.0/go.mod h1:Y4YMaQmXwGQZoFaVFk4YpCt4FLQMYKZe9oeV/f4MSno=
golang.org/x/tools v0.0.0-20180917221912-90fa682c2a6e/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ= golang.org/x/tools v0.0.0-20180917221912-90fa682c2a6e/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ=
golang.org/x/tools v0.0.0-20191119224855-298f0cb1881e/go.mod h1:b+2E5dAYhXwXZwtnZ6UAqBI28+e2cm9otk0dWdXHAEo= golang.org/x/tools v0.0.0-20191119224855-298f0cb1881e/go.mod h1:b+2E5dAYhXwXZwtnZ6UAqBI28+e2cm9otk0dWdXHAEo=
golang.org/x/tools v0.1.12/go.mod h1:hNGJHUnrk76NpqgfD5Aqm5Crs+Hm0VOH/i9J2+nxYbc= golang.org/x/tools v0.1.12/go.mod h1:hNGJHUnrk76NpqgfD5Aqm5Crs+Hm0VOH/i9J2+nxYbc=
golang.org/x/tools v0.45.0 h1:18qN3FAooORvApf5XjCXgsuayZOEtXf6JK18I3+ONa8= golang.org/x/tools v0.47.0 h1:7Kn5x/d1svx/PzryTsqeoZN4TZwqeH5pGWjefhLi/1Q=
golang.org/x/tools v0.45.0/go.mod h1:LuUGqqaXcXMEFEruIVJVm5mgDD8vww/z/SR1gQ4uE/0= golang.org/x/tools v0.47.0/go.mod h1:dFHnyTvFWY212G+h7ZY4Vsp/K3U4/7W9TyVaAul8uCA=
golang.org/x/xerrors v0.0.0-20190717185122-a985d3407aa7/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0= golang.org/x/xerrors v0.0.0-20190717185122-a985d3407aa7/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
gonum.org/v1/gonum v0.17.0 h1:VbpOemQlsSMrYmn7T2OUvQ4dqxQXU+ouZFQsZOx50z4= gonum.org/v1/gonum v0.17.0 h1:VbpOemQlsSMrYmn7T2OUvQ4dqxQXU+ouZFQsZOx50z4=
gonum.org/v1/gonum v0.17.0/go.mod h1:El3tOrEuMpv2UdMrbNlKEh9vd86bmQ6vqIcDwxEOc1E= gonum.org/v1/gonum v0.17.0/go.mod h1:El3tOrEuMpv2UdMrbNlKEh9vd86bmQ6vqIcDwxEOc1E=
+14 -135
View File
@@ -2,8 +2,6 @@
package sync package sync
import ( import (
"archive/zip"
"bytes"
"context" "context"
"crypto/sha256" "crypto/sha256"
"encoding/hex" "encoding/hex"
@@ -11,21 +9,19 @@ import (
"errors" "errors"
"fmt" "fmt"
"io" "io"
"math"
"os" "os"
"path"
"path/filepath" "path/filepath"
"strings" "strings"
"github.com/Rain-kl/Wavelet/internal/apps/agent/protocol" "github.com/Rain-kl/Wavelet/internal/apps/agent/protocol"
"github.com/Rain-kl/Wavelet/internal/apps/agent/state" "github.com/Rain-kl/Wavelet/internal/apps/agent/state"
"github.com/Rain-kl/Wavelet/pkg/pagesarchive"
) )
const ( const (
pagesMaxExtractedFileBytes = 100 * 1024 * 1024 pagesDirPerm = 0o755
pagesDirPerm = 0o755 pagesFilePerm = 0o644
pagesFilePerm = 0o644 pagesManifestFilePerm = 0o644
pagesManifestFilePerm = 0o644
) )
type pagesSourceDocument struct { type pagesSourceDocument struct {
@@ -223,89 +219,26 @@ func referencedPagesDeployments(config *protocol.ActiveConfigResponse) ([]pagesD
return result, nil return result, nil
} }
func findCommonRootPrefix(files []*zip.File) (string, error) {
var firstFilePath string
hasMultipleFiles := false
for _, item := range files {
relativePath, skip, err := normalizePagesArchivePath(item.Name)
if err != nil {
return "", err
}
if skip {
continue
}
normalizedPath := filepath.ToSlash(relativePath)
if firstFilePath == "" {
firstFilePath = normalizedPath
} else {
hasMultipleFiles = true
}
}
if firstFilePath == "" {
return "", nil
}
parts := strings.Split(firstFilePath, "/")
if len(parts) <= 1 {
return "", nil
}
commonPrefix := parts[0] + "/"
if hasMultipleFiles {
for _, item := range files {
relativePath, skip, err := normalizePagesArchivePath(item.Name)
if err != nil {
return "", err
}
if skip {
continue
}
normalizedPath := filepath.ToSlash(relativePath)
if !strings.HasPrefix(normalizedPath, commonPrefix) {
return "", nil
}
}
}
return commonPrefix, nil
}
func extractPagesPackage(packageBytes []byte, releaseDir string, deployment pagesDeploymentSource) error { func extractPagesPackage(packageBytes []byte, releaseDir string, deployment pagesDeploymentSource) error {
tmpDir := releaseDir + ".tmp" tmpDir := releaseDir + ".tmp"
_ = os.RemoveAll(tmpDir) _ = os.RemoveAll(tmpDir)
if err := os.MkdirAll(tmpDir, pagesDirPerm); err != nil { if err := os.MkdirAll(tmpDir, pagesDirPerm); err != nil {
return err return err
} }
reader, err := zip.NewReader(bytes.NewReader(packageBytes), int64(len(packageBytes))) format, err := pagesarchive.DetectFormat("", packageBytes)
if err != nil { if err != nil {
_ = os.RemoveAll(tmpDir) _ = os.RemoveAll(tmpDir)
return fmt.Errorf("open Pages zip: %w", err) return fmt.Errorf("detect Pages package format: %w", err)
} }
commonPrefix, err := findCommonRootPrefix(reader.File) // Control plane already inspected and accepted this package. Agent only
if err != nil { // verifies download integrity (checksum) and performs local-safe extract
// (path escape / symlink guards). Size and file-count limits are not re-applied.
if err := pagesarchive.ExtractBytes(packageBytes, format, tmpDir, pagesarchive.ExtractOptions{
StripCommonRoot: true,
EnforceLimits: false,
}); err != nil {
_ = os.RemoveAll(tmpDir) _ = os.RemoveAll(tmpDir)
return err return fmt.Errorf("extract Pages package: %w", err)
}
for _, item := range reader.File {
relativePath, skip, err := normalizePagesArchivePath(item.Name)
if err != nil {
_ = os.RemoveAll(tmpDir)
return err
}
if skip {
continue
}
if commonPrefix != "" {
slashPath := filepath.ToSlash(relativePath)
if strings.HasPrefix(slashPath, commonPrefix) {
relativePath = filepath.FromSlash(strings.TrimPrefix(slashPath, commonPrefix))
}
}
if item.FileInfo().Mode()&os.ModeSymlink != 0 {
_ = os.RemoveAll(tmpDir)
return fmt.Errorf("pages package contains unsupported symlink: %s", relativePath)
}
if err := extractPagesFile(item, filepath.Join(tmpDir, relativePath)); err != nil {
_ = os.RemoveAll(tmpDir)
return err
}
} }
if err := writePagesMarker(tmpDir, deployment); err != nil { if err := writePagesMarker(tmpDir, deployment); err != nil {
_ = os.RemoveAll(tmpDir) _ = os.RemoveAll(tmpDir)
@@ -315,42 +248,6 @@ func extractPagesPackage(packageBytes []byte, releaseDir string, deployment page
return os.Rename(tmpDir, releaseDir) return os.Rename(tmpDir, releaseDir)
} }
func copyPagesZipEntryContent(dst io.Writer, src io.Reader, declaredSize uint64) (int64, error) {
if declaredSize > pagesMaxExtractedFileBytes || declaredSize > uint64(math.MaxInt64) {
return 0, errors.New("pages file size out of bounds")
}
if declaredSize > 0 {
return io.CopyN(dst, src, int64(declaredSize)) //nolint:gosec // declaredSize is bounded to math.MaxInt64 above
}
limited := io.LimitReader(src, pagesMaxExtractedFileBytes+1)
written, err := io.Copy(dst, limited)
if written > pagesMaxExtractedFileBytes {
return written, errors.New("pages file size out of bounds")
}
return written, err
}
func extractPagesFile(item *zip.File, targetPath string) error {
if err := os.MkdirAll(filepath.Dir(targetPath), pagesDirPerm); err != nil {
return err
}
source, err := item.Open()
if err != nil {
return err
}
defer func() { _ = source.Close() }()
target, err := os.OpenFile(targetPath, os.O_CREATE|os.O_WRONLY|os.O_TRUNC, pagesFilePerm) //nolint:gosec // targetPath is under managed PagesDir from validated zip entry
if err != nil {
return err
}
defer func() { _ = target.Close() }()
_, err = copyPagesZipEntryContent(target, source, item.UncompressedSize64)
if err != nil {
return fmt.Errorf("%s: %w", item.Name, err)
}
return nil
}
func switchPagesCurrentDir(baseDir string, deploymentID uint, releaseDir string) error { func switchPagesCurrentDir(baseDir string, deploymentID uint, releaseDir string) error {
currentDir := pagesCurrentDir(baseDir, deploymentID) currentDir := pagesCurrentDir(baseDir, deploymentID)
previousDir := currentDir + ".previous" previousDir := currentDir + ".previous"
@@ -440,24 +337,6 @@ func copyPagesDir(sourceDir string, targetDir string) error {
}) })
} }
func normalizePagesArchivePath(raw string) (string, bool, error) {
name := strings.TrimSpace(filepath.ToSlash(raw))
if name == "" || strings.HasSuffix(name, "/") {
return "", true, nil
}
if strings.HasPrefix(name, "/") {
return "", false, fmt.Errorf("pages package contains absolute path: %s", raw)
}
cleaned := path.Clean(name)
if cleaned == "." {
return "", true, nil
}
if cleaned == ".." || strings.HasPrefix(cleaned, "../") || strings.Contains(cleaned, "/../") {
return "", false, fmt.Errorf("pages package path escapes deployment root: %s", raw)
}
return filepath.FromSlash(cleaned), false, nil
}
func markerMatches(dir string, deployment pagesDeploymentSource) bool { func markerMatches(dir string, deployment pagesDeploymentSource) bool {
data, err := os.ReadFile(filepath.Join(dir, ".openflare-pages.json")) //nolint:gosec // dir is managed PagesDir data, err := os.ReadFile(filepath.Join(dir, ".openflare-pages.json")) //nolint:gosec // dir is managed PagesDir
if err != nil { if err != nil {
+1 -1
View File
@@ -525,7 +525,7 @@ func TestSyncOnceRejectsPagesZipSlipBeforeApply(t *testing.T) {
service.SetPagesDir(t.TempDir()) service.SetPagesDir(t.TempDir())
err := service.SyncOnce(context.Background(), &protocol.ActiveConfigMeta{Version: "20260309-102", Checksum: "pages-config-checksum"}) err := service.SyncOnce(context.Background(), &protocol.ActiveConfigMeta{Version: "20260309-102", Checksum: "pages-config-checksum"})
if err == nil || !strings.Contains(err.Error(), "escapes deployment root") { if err == nil || (!strings.Contains(err.Error(), "escapes deployment root") && !strings.Contains(err.Error(), "escapes directory")) {
t.Fatalf("expected zip-slip rejection, got %v", err) t.Fatalf("expected zip-slip rejection, got %v", err)
} }
if len(manager.applyRouteContents) != 0 { if len(manager.applyRouteContents) != 0 {
+12 -1
View File
@@ -9,6 +9,7 @@ import (
"errors" "errors"
"strings" "strings"
"github.com/Rain-kl/Wavelet/internal/apps/openflare/pages"
"github.com/Rain-kl/Wavelet/internal/model" "github.com/Rain-kl/Wavelet/internal/model"
openrestyrender "github.com/Rain-kl/Wavelet/pkg/render/openresty" openrestyrender "github.com/Rain-kl/Wavelet/pkg/render/openresty"
"gorm.io/gorm" "gorm.io/gorm"
@@ -38,10 +39,20 @@ func getActiveConfigForAgent(ctx context.Context) (*ConfigResponse, error) {
} }
} }
// Main config version history is independent of Pages deployment history.
// Agents always receive pages routes bound to each project's current active
// deployment so config rollback never depends on pruned packages.
sourceJSON := version.SnapshotJSON
if rebound, rebindErr := pages.RebindSnapshotPagesToCurrentActive(ctx, version.SnapshotJSON); rebindErr != nil {
return nil, rebindErr
} else if strings.TrimSpace(rebound) != "" {
sourceJSON = rebound
}
return &ConfigResponse{ return &ConfigResponse{
Version: version.Version, Version: version.Version,
Checksum: version.Checksum, Checksum: version.Checksum,
SourceConfigJSON: version.SnapshotJSON, SourceConfigJSON: sourceJSON,
SupportFiles: sourceSupportFiles(supportFiles), SupportFiles: sourceSupportFiles(supportFiles),
CreatedAt: version.CreatedAt, CreatedAt: version.CreatedAt,
}, nil }, nil
@@ -59,6 +59,9 @@ func validateOptionWithState(ctx context.Context, option model.OpenFlareOption,
if err := validateAgentOption(option.Key, option.Value); err != nil { if err := validateAgentOption(option.Key, option.Value); err != nil {
return err return err
} }
if err := validatePagesOption(option.Key, option.Value); err != nil {
return err
}
return validateUptimeKumaOption(ctx, option.Key, option.Value, state) return validateUptimeKumaOption(ctx, option.Key, option.Value, state)
} }
@@ -109,6 +112,23 @@ func validateAgentOption(key, value string) error {
return nil return nil
} }
func validatePagesOption(key, value string) error {
trimmed := strings.TrimSpace(value)
switch key {
case model.ConfigKeyPagesMaxPackageSizeMB:
intValue, err := strconv.Atoi(trimmed)
if err != nil || intValue < 1 || intValue > 2048 {
return fmt.Errorf("%s 必须为 1~2048 的整数(MiB)", key)
}
case model.ConfigKeyPagesMaxHistoryCount:
intValue, err := strconv.Atoi(trimmed)
if err != nil || intValue < 0 {
return fmt.Errorf("%s 必须为大于等于 0 的整数(0 表示不限制)", key)
}
}
return nil
}
func validateUptimeKumaOption(ctx context.Context, key, value string, state map[string]string) error { func validateUptimeKumaOption(ctx context.Context, key, value string, state map[string]string) error {
trimmed := strings.TrimSpace(value) trimmed := strings.TrimSpace(value)
switch key { switch key {
+6 -2
View File
@@ -14,9 +14,13 @@ const (
errPagesDeploymentMismatch = "pages 部署不属于该项目" errPagesDeploymentMismatch = "pages 部署不属于该项目"
errPagesDeleteActiveDeploy = "不能删除当前激活的 Pages 部署" errPagesDeleteActiveDeploy = "不能删除当前激活的 Pages 部署"
errPagesPackageMissing = "缺少 Pages 部署包" errPagesPackageMissing = "缺少 Pages 部署包"
errPagesPackageNotZip = "pages 部署包必须是 .zip 文件" errPagesPackageNotZip = "pages 部署包必须是 .zip 文件" // legacy alias kept for tests
errPagesPackageInvalidZip = "pages 部署包不是有效 zip 文件" errPagesPackageUnsupported = "pages 部署包仅支持 zip、tar.gz、tar.xz、tar.bz2、tar、7z 格式"
errPagesPackageInvalidZip = "pages 部署包不是有效 zip 文件" // legacy alias
errPagesPackageInvalid = "pages 部署包不是有效的压缩文件"
errPagesPackageEmpty = "pages 部署包不能为空" errPagesPackageEmpty = "pages 部署包不能为空"
errPagesPackageExtractedTooLarge = "pages 部署包展开后体积超过限制"
errPagesPackageFileTooLarge = "pages 部署包内文件过大"
errPagesAPIProxyPathRequired = "启用 API 反代时,匹配路径不能为空" errPagesAPIProxyPathRequired = "启用 API 反代时,匹配路径不能为空"
errPagesAPIProxyPathPrefix = "API 反代匹配路径必须以 '/' 开头" errPagesAPIProxyPathPrefix = "API 反代匹配路径必须以 '/' 开头"
errPagesAPIProxyPassRequired = "启用 API 反代时,后端服务地址不能为空" //nolint:gosec // false positive: this is an error message, not hardcoded credentials errPagesAPIProxyPassRequired = "启用 API 反代时,后端服务地址不能为空" //nolint:gosec // false positive: this is an error message, not hardcoded credentials
+164 -167
View File
@@ -4,14 +4,12 @@
package pages package pages
import ( import (
"archive/zip"
"context" "context"
"crypto/sha256" "crypto/sha256"
"encoding/hex" "encoding/hex"
"errors" "errors"
"fmt" "fmt"
"io" "io"
"math"
"mime/multipart" "mime/multipart"
"os" "os"
"path" "path"
@@ -22,17 +20,22 @@ import (
"github.com/Rain-kl/Wavelet/internal/apps/upload" "github.com/Rain-kl/Wavelet/internal/apps/upload"
"github.com/Rain-kl/Wavelet/internal/model" "github.com/Rain-kl/Wavelet/internal/model"
"github.com/Rain-kl/Wavelet/internal/repository" "github.com/Rain-kl/Wavelet/internal/repository"
"github.com/Rain-kl/Wavelet/pkg/logger"
"github.com/Rain-kl/Wavelet/pkg/pagesarchive"
) )
const ( const (
pagesMaxDeploymentFiles = 1000 pagesMaxDeploymentFiles = 1000
pagesMaxDeploymentBytes = 100 * 1024 * 1024 defaultPagesMaxPackageSizeMB = 100
defaultPagesEntryFile = "index.html" maxPagesMaxPackageSizeMB = 2048
defaultPagesFallbackPath = "/index.html" defaultPagesMaxHistoryCount = 20
pagesDeploymentUploadType = "openflare_pages_deployment" defaultPagesEntryFile = "index.html"
mimeTypeApplicationZip = "application/zip" defaultPagesFallbackPath = "/index.html"
pagesMaxPathLength = 512 pagesDeploymentUploadType = "openflare_pages_deployment"
bytesPerKiB = 1024 pagesMaxPathLength = 512
bytesPerMiB = 1024 * 1024
pagesExtractedSizeMultiplier = 4
pagesMinExtractedSizeBytes = 100 * bytesPerMiB
) )
var pagesSlugPattern = regexp.MustCompile(`^[a-z0-9][a-z0-9-]{0,126}[a-z0-9]$|^[a-z0-9]$`) var pagesSlugPattern = regexp.MustCompile(`^[a-z0-9][a-z0-9-]{0,126}[a-z0-9]$|^[a-z0-9]$`)
@@ -42,6 +45,14 @@ type deploymentManifest struct {
FileCount int FileCount int
TotalSize int64 TotalSize int64
EntryFile string EntryFile string
Format pagesarchive.Format
}
type pagesLimits struct {
PackageBytes int64
ExtractedBytes int64
MaxFiles int
HistoryCount int
} }
func isUniqueConstraintError(err error) bool { func isUniqueConstraintError(err error) bool {
@@ -51,6 +62,38 @@ func isUniqueConstraintError(err error) bool {
return strings.Contains(strings.ToLower(err.Error()), "unique") return strings.Contains(strings.ToLower(err.Error()), "unique")
} }
func resolvePagesLimits(ctx context.Context) pagesLimits {
packageMB := defaultPagesMaxPackageSizeMB
if value, err := repository.GetIntByKey(ctx, model.ConfigKeyPagesMaxPackageSizeMB); err == nil && value > 0 {
packageMB = value
}
if packageMB > maxPagesMaxPackageSizeMB {
packageMB = maxPagesMaxPackageSizeMB
}
historyCount := defaultPagesMaxHistoryCount
if value, err := repository.GetIntByKey(ctx, model.ConfigKeyPagesMaxHistoryCount); err == nil {
if value < 0 {
historyCount = 0
} else {
historyCount = value
}
}
packageBytes := int64(packageMB) * bytesPerMiB
extractedBytes := packageBytes * pagesExtractedSizeMultiplier
if extractedBytes < pagesMinExtractedSizeBytes {
extractedBytes = pagesMinExtractedSizeBytes
}
return pagesLimits{
PackageBytes: packageBytes,
ExtractedBytes: extractedBytes,
MaxFiles: pagesMaxDeploymentFiles,
HistoryCount: historyCount,
}
}
func normalizePagesSlug(raw string) string { func normalizePagesSlug(raw string) string {
value := strings.ToLower(strings.TrimSpace(raw)) value := strings.ToLower(strings.TrimSpace(raw))
var builder strings.Builder var builder strings.Builder
@@ -76,7 +119,7 @@ func validateAndNormalizePagesRootDir(raw string) (string, error) {
return "", nil return "", nil
} }
if len(value) > pagesMaxPathLength { if len(value) > pagesMaxPathLength {
return "", errors.New("pages 根目录长度不能超过 512") // error 消息首字母小写 return "", errors.New("pages 根目录长度不能超过 512")
} }
if strings.Contains(value, "\\") || strings.ContainsAny(value, "\"';") { if strings.Contains(value, "\\") || strings.ContainsAny(value, "\"';") {
return "", errors.New("pages 根目录包含不支持的字符") return "", errors.New("pages 根目录包含不支持的字符")
@@ -151,29 +194,50 @@ func normalizePagesEntryFile(raw string) string {
return strings.TrimPrefix(value, "/") return strings.TrimPrefix(value, "/")
} }
func persistPagesUploadTemp(fileHeader *multipart.FileHeader) (string, string, int64, error) { func persistPagesUploadTemp(fileHeader *multipart.FileHeader, maxPackageBytes int64) (string, string, int64, pagesarchive.Format, error) {
format, ok := pagesarchive.DetectFormatFromName(fileHeader.Filename)
if !ok {
return "", "", 0, "", errors.New(errPagesPackageUnsupported)
}
file, err := fileHeader.Open() file, err := fileHeader.Open()
if err != nil { if err != nil {
return "", "", 0, err return "", "", 0, "", err
} }
defer func() { _ = file.Close() }() defer func() { _ = file.Close() }()
temp, err := os.CreateTemp("", "openflare-pages-*.zip") temp, err := os.CreateTemp("", "openflare-pages-*."+safeTempSuffix(format))
if err != nil { if err != nil {
return "", "", 0, err return "", "", 0, "", err
} }
defer func() { _ = temp.Close() }() defer func() { _ = temp.Close() }()
hash := sha256.New() hash := sha256.New()
limited := io.LimitReader(file, pagesMaxDeploymentBytes+1) limited := io.LimitReader(file, maxPackageBytes+1)
written, err := io.Copy(io.MultiWriter(temp, hash), limited) written, err := io.Copy(io.MultiWriter(temp, hash), limited)
if err != nil { if err != nil {
_ = os.Remove(temp.Name()) _ = os.Remove(temp.Name())
return "", "", 0, err return "", "", 0, "", err
} }
if written > pagesMaxDeploymentBytes { if written > maxPackageBytes {
_ = os.Remove(temp.Name()) _ = os.Remove(temp.Name())
return "", "", 0, fmt.Errorf("pages 部署包不能超过 %d MiB", pagesMaxDeploymentBytes/bytesPerKiB/bytesPerKiB) return "", "", 0, "", fmt.Errorf("pages 部署包不能超过 %d MiB", maxPackageBytes/bytesPerMiB)
}
return temp.Name(), hex.EncodeToString(hash.Sum(nil)), written, format, nil
}
func safeTempSuffix(format pagesarchive.Format) string {
switch format {
case pagesarchive.FormatTarGz:
return "tar.gz"
case pagesarchive.FormatTarXz:
return "tar.xz"
case pagesarchive.FormatTarBz2:
return "tar.bz2"
case pagesarchive.FormatSevenZip:
return "7z"
case pagesarchive.FormatTar:
return "tar"
default:
return "zip"
} }
return temp.Name(), hex.EncodeToString(hash.Sum(nil)), written, nil
} }
func pagesLegacyRelativeCandidates(project *model.PagesProject, deployment *model.PagesDeployment) []string { func pagesLegacyRelativeCandidates(project *model.PagesProject, deployment *model.PagesDeployment) []string {
@@ -185,6 +249,7 @@ func pagesLegacyRelativeCandidates(project *model.PagesProject, deployment *mode
if slug == "" || checksum == "" { if slug == "" || checksum == "" {
return nil return nil
} }
// Legacy artifacts were always stored as .zip.
fileName := checksum + ".zip" fileName := checksum + ".zip"
return []string{ return []string{
filepath.Join("artifacts", slug, fileName), filepath.Join("artifacts", slug, fileName),
@@ -199,14 +264,16 @@ func ingestPagesDeploymentPackage(
checksum string, checksum string,
projectSlug string, projectSlug string,
fileName string, fileName string,
format pagesarchive.Format,
) (upload.IngestResult, error) { ) (upload.IngestResult, error) {
systemUser := repository.GetSystemUser(ctx) systemUser := repository.GetSystemUser(ctx)
accessMode := 0 accessMode := 0
extension := pagesarchive.NormalizeNameExtension(fileName, format)
return upload.IngestFromLocalPath(ctx, localPath, upload.IngestRequest{ return upload.IngestFromLocalPath(ctx, localPath, upload.IngestRequest{
UserID: systemUser.ID, UserID: systemUser.ID,
FileName: fileName, FileName: fileName,
MimeType: mimeTypeApplicationZip, MimeType: pagesarchive.MIMEType(format),
Extension: "zip", Extension: extension,
Hash: checksum, Hash: checksum,
Type: pagesDeploymentUploadType, Type: pagesDeploymentUploadType,
AccessMode: &accessMode, AccessMode: &accessMode,
@@ -215,6 +282,7 @@ func ingestPagesDeploymentPackage(
Metadata: model.UploadMetadata{ Metadata: model.UploadMetadata{
Extra: map[string]any{ Extra: map[string]any{
"project_slug": projectSlug, "project_slug": projectSlug,
"format": string(format),
}, },
}, },
}) })
@@ -224,167 +292,96 @@ func removeDeploymentArtifact(ctx context.Context, deployment *model.PagesDeploy
if deployment == nil { if deployment == nil {
return return
} }
if deployment.UploadID > 0 { if deployment.UploadID == 0 {
_, _ = upload.Remove(ctx, deployment.UploadID) return
}
if _, err := upload.Remove(ctx, deployment.UploadID); err != nil {
// Soft-delete / storage cleanup failure must not undo DB prune; log for ops.
logger.WarnF(ctx,
"[Pages] remove deployment artifact failed: deployment_id=%d upload_id=%d error=%v",
deployment.ID, deployment.UploadID, err,
)
} }
} }
func findCommonRootPrefix(files []*zip.File) (string, error) { func inspectPagesPackage(packagePath string, format pagesarchive.Format, rootDir string, entryFile string, limits pagesLimits) (*deploymentManifest, error) {
var firstFilePath string archiveManifest, err := pagesarchive.InspectFile(packagePath, format, pagesarchive.InspectOptions{
hasMultipleFiles := false RootDir: rootDir,
for _, item := range files {
normalizedPath, skip, err := normalizePagesZipPath(item.Name)
if err != nil {
return "", err
}
if skip {
continue
}
if firstFilePath == "" {
firstFilePath = normalizedPath
} else {
hasMultipleFiles = true
}
}
if firstFilePath == "" {
return "", nil
}
parts := strings.Split(firstFilePath, "/")
if len(parts) <= 1 {
return "", nil
}
commonPrefix := parts[0] + "/"
if hasMultipleFiles {
for _, item := range files {
normalizedPath, skip, err := normalizePagesZipPath(item.Name)
if err != nil {
return "", err
}
if skip {
continue
}
if !strings.HasPrefix(normalizedPath, commonPrefix) {
return "", nil
}
}
}
return commonPrefix, nil
}
func inspectPagesZip(zipPath string, rootDir string, entryFile string) (*deploymentManifest, error) {
reader, err := zip.OpenReader(zipPath)
if err != nil {
return nil, errors.New(errPagesPackageInvalidZip)
}
defer func() { _ = reader.Close() }()
commonPrefix, err := findCommonRootPrefix(reader.File)
if err != nil {
return nil, err
}
manifest := &deploymentManifest{
Files: []model.PagesDeploymentFile{},
EntryFile: entryFile, EntryFile: entryFile,
Limits: pagesarchive.Limits{
MaxFiles: limits.MaxFiles,
MaxFileBytes: limits.ExtractedBytes,
MaxTotalBytes: limits.ExtractedBytes,
},
})
if err != nil {
return nil, mapPagesArchiveError(err)
} }
targetEntryPath := entryFile manifest := &deploymentManifest{
if rootDir != "" { Files: make([]model.PagesDeploymentFile, 0, len(archiveManifest.Files)),
targetEntryPath = path.Join(rootDir, entryFile) FileCount: archiveManifest.FileCount,
TotalSize: archiveManifest.TotalSize,
EntryFile: entryFile,
Format: format,
} }
entrySeen := false for _, file := range archiveManifest.Files {
for _, item := range reader.File {
normalizedPath, skip, err := normalizePagesZipPath(item.Name)
if err != nil {
return nil, err
}
if skip {
continue
}
if commonPrefix != "" {
normalizedPath = strings.TrimPrefix(normalizedPath, commonPrefix)
}
if item.FileInfo().Mode()&os.ModeSymlink != 0 {
return nil, fmt.Errorf("pages 部署包不支持符号链接: %s", normalizedPath)
}
if item.UncompressedSize64 > pagesMaxDeploymentBytes {
return nil, fmt.Errorf("pages 文件过大: %s", normalizedPath)
}
manifest.FileCount++
if manifest.FileCount > pagesMaxDeploymentFiles {
return nil, fmt.Errorf("pages 部署文件数不能超过 %d", pagesMaxDeploymentFiles)
}
checksum, fileSize, err := checksumZipFile(item)
if err != nil {
return nil, fmt.Errorf("%s: %w", normalizedPath, err)
}
manifest.TotalSize += fileSize
if manifest.TotalSize > pagesMaxDeploymentBytes {
return nil, fmt.Errorf("pages 部署展开后不能超过 %d MiB", pagesMaxDeploymentBytes/bytesPerKiB/bytesPerKiB)
}
if normalizedPath == targetEntryPath {
entrySeen = true
}
manifest.Files = append(manifest.Files, model.PagesDeploymentFile{ manifest.Files = append(manifest.Files, model.PagesDeploymentFile{
Path: normalizedPath, Path: file.Path,
Size: fileSize, Size: file.Size,
Checksum: checksum, Checksum: file.Checksum,
}) })
} }
if manifest.FileCount == 0 {
return nil, errors.New(errPagesPackageEmpty)
}
if !entrySeen {
return nil, fmt.Errorf("pages 部署包缺少入口文件 %s", targetEntryPath)
}
return manifest, nil return manifest, nil
} }
func normalizePagesZipPath(raw string) (string, bool, error) { func mapPagesArchiveError(err error) error {
name := strings.TrimSpace(filepath.ToSlash(raw)) if err == nil {
if name == "" { return nil
return "", true, nil
} }
if strings.HasSuffix(name, "/") { message := err.Error()
return "", true, nil switch {
case strings.Contains(message, "unsupported pages package format"):
return errors.New(errPagesPackageUnsupported)
case strings.Contains(message, "open zip"), strings.Contains(message, "open gzip"),
strings.Contains(message, "open xz"), strings.Contains(message, "open 7z"),
strings.Contains(message, "read tar"):
return errors.New(errPagesPackageInvalid)
case strings.Contains(message, "empty"):
return errors.New(errPagesPackageEmpty)
case strings.Contains(message, "missing entry file"):
return err
case strings.Contains(message, "file count exceeds"):
return fmt.Errorf("pages 部署文件数不能超过 %d", pagesMaxDeploymentFiles)
case strings.Contains(message, "extracted size exceeds"):
return errors.New(errPagesPackageExtractedTooLarge)
case strings.Contains(message, "file too large"), strings.Contains(message, "size out of bounds"):
return errors.New(errPagesPackageFileTooLarge)
case strings.Contains(message, "symlink"):
return err
case strings.Contains(message, "absolute path"), strings.Contains(message, "escapes directory"):
return err
default:
return err
} }
if strings.HasPrefix(name, "/") || path.IsAbs(name) {
return "", false, fmt.Errorf("pages 部署包不能包含绝对路径: %s", raw)
}
cleaned := path.Clean(name)
if cleaned == "." {
return "", true, nil
}
if cleaned == ".." || strings.HasPrefix(cleaned, "../") || strings.Contains(cleaned, "/../") {
return "", false, fmt.Errorf("pages 部署包路径不能逃逸目录: %s", raw)
}
return cleaned, false, nil
} }
func copyPagesZipEntryContent(dst io.Writer, src io.Reader, declaredSize uint64) (int64, error) { func packageDownloadName(deploymentID uint, fileName string, contentType string) string {
if declaredSize > pagesMaxDeploymentBytes || declaredSize > uint64(math.MaxInt64) { if format, ok := pagesarchive.DetectFormatFromName(fileName); ok {
return 0, errors.New("pages file size out of bounds") return fmt.Sprintf("pages-deployment-%d.%s", deploymentID, pagesarchive.Extension(format))
} }
if declaredSize > 0 { // Fall back by content type.
return io.CopyN(dst, src, int64(declaredSize)) //nolint:gosec // declaredSize is bounded to math.MaxInt64 above switch strings.ToLower(strings.TrimSpace(contentType)) {
case "application/gzip", "application/x-gzip":
return fmt.Sprintf("pages-deployment-%d.tar.gz", deploymentID)
case "application/x-xz":
return fmt.Sprintf("pages-deployment-%d.tar.xz", deploymentID)
case "application/x-bzip2":
return fmt.Sprintf("pages-deployment-%d.tar.bz2", deploymentID)
case "application/x-7z-compressed":
return fmt.Sprintf("pages-deployment-%d.7z", deploymentID)
case "application/x-tar":
return fmt.Sprintf("pages-deployment-%d.tar", deploymentID)
default:
return fmt.Sprintf("pages-deployment-%d.zip", deploymentID)
} }
limited := io.LimitReader(src, pagesMaxDeploymentBytes+1)
written, err := io.Copy(dst, limited)
if written > pagesMaxDeploymentBytes {
return written, errors.New("pages file size out of bounds")
}
return written, err
}
func checksumZipFile(item *zip.File) (string, int64, error) {
file, err := item.Open()
if err != nil {
return "", 0, err
}
defer func() { _ = file.Close() }()
hash := sha256.New()
written, err := copyPagesZipEntryContent(hash, file, item.UncompressedSize64)
if err != nil {
return "", written, err
}
return hex.EncodeToString(hash.Sum(nil)), written, nil
} }
+206 -10
View File
@@ -12,13 +12,14 @@ import (
"mime/multipart" "mime/multipart"
"net/url" "net/url"
"os" "os"
"path/filepath"
"strings" "strings"
"time" "time"
"github.com/Rain-kl/Wavelet/internal/apps/upload" "github.com/Rain-kl/Wavelet/internal/apps/upload"
"github.com/Rain-kl/Wavelet/internal/db" "github.com/Rain-kl/Wavelet/internal/db"
"github.com/Rain-kl/Wavelet/internal/model" "github.com/Rain-kl/Wavelet/internal/model"
"github.com/Rain-kl/Wavelet/pkg/logger"
"github.com/Rain-kl/Wavelet/pkg/pagesarchive"
"gorm.io/gorm" "gorm.io/gorm"
) )
@@ -251,20 +252,18 @@ func UploadDeployment(ctx context.Context, projectID uint, fileHeader *multipart
if fileHeader == nil { if fileHeader == nil {
return nil, errors.New(errPagesPackageMissing) return nil, errors.New(errPagesPackageMissing)
} }
if !strings.EqualFold(filepath.Ext(fileHeader.Filename), ".zip") { limits := resolvePagesLimits(ctx)
return nil, errors.New(errPagesPackageNotZip)
}
rootDir, err := validateAndNormalizePagesRootDir(project.RootDir) rootDir, err := validateAndNormalizePagesRootDir(project.RootDir)
if err != nil { if err != nil {
return nil, err return nil, err
} }
entryFile := normalizePagesEntryFile(project.EntryFile) entryFile := normalizePagesEntryFile(project.EntryFile)
tempPath, checksum, _, err := persistPagesUploadTemp(fileHeader) tempPath, checksum, _, format, err := persistPagesUploadTemp(fileHeader, limits.PackageBytes)
if err != nil { if err != nil {
return nil, err return nil, err
} }
defer func() { _ = os.Remove(tempPath) }() defer func() { _ = os.Remove(tempPath) }()
manifest, err := inspectPagesZip(tempPath, rootDir, entryFile) manifest, err := inspectPagesPackage(tempPath, format, rootDir, entryFile, limits)
if err != nil { if err != nil {
return nil, err return nil, err
} }
@@ -274,6 +273,7 @@ func UploadDeployment(ctx context.Context, projectID uint, fileHeader *multipart
checksum, checksum,
project.Slug, project.Slug,
fileHeader.Filename, fileHeader.Filename,
format,
) )
if err != nil { if err != nil {
return nil, err return nil, err
@@ -320,10 +320,160 @@ func UploadDeployment(ctx context.Context, projectID uint, fileHeader *multipart
return nil, err return nil, err
} }
ingestCommitted = true ingestCommitted = true
// History prune must not fail the already-committed upload. Log and continue;
// a later upload (or a retry pass inside prune) will re-attempt cleanup.
if pruneErr := pruneProjectDeploymentHistory(ctx, project.ID, limits.HistoryCount); pruneErr != nil {
logger.ErrorF(ctx,
"[Pages] prune deployment history failed: project_id=%d keep=%d error=%v",
project.ID, limits.HistoryCount, pruneErr,
)
}
view := buildDeploymentView(deployment) view := buildDeploymentView(deployment)
return &view, nil return &view, nil
} }
// pruneProjectDeploymentHistory enforces the retention policy for one project.
//
// Policy (keepCount > 0):
// - At most keepCount deployment rows remain for the project.
// - The current active deployment is always retained (if any).
// - Remaining slots are filled by newest deployments first (id desc).
// - All other non-kept deployments are deleted with their file lists and artifacts.
//
// keepCount <= 0 means unlimited history.
//
// Concurrency: DB row deletes run in a single transaction after a consistent read
// of project + deployments. Concurrent uploads may briefly exceed keepCount; the
// next successful prune brings the project back within the limit (eventual).
func pruneProjectDeploymentHistory(ctx context.Context, projectID uint, keepCount int) error {
if keepCount <= 0 {
return nil
}
// Two passes: first pass after upload, second pass heals a concurrent race
// that inserted another deployment between our list and delete.
var lastErr error
for pass := 0; pass < 2; pass++ {
deleted, err := pruneProjectDeploymentHistoryOnce(ctx, projectID, keepCount)
if err != nil {
lastErr = err
break
}
if deleted == 0 {
break
}
}
return lastErr
}
// pruneProjectDeploymentHistoryOnce performs one list → select → delete cycle.
// Returns the number of deployments deleted from the database.
func pruneProjectDeploymentHistoryOnce(ctx context.Context, projectID uint, keepCount int) (int, error) {
project, err := model.GetPagesProjectByID(ctx, projectID)
if err != nil {
return 0, fmt.Errorf("load pages project: %w", err)
}
deployments, err := model.ListPagesDeployments(ctx, projectID)
if err != nil {
return 0, fmt.Errorf("list pages deployments: %w", err)
}
if len(deployments) <= keepCount {
return 0, nil
}
var activeID uint
if project.ActiveDeploymentID != nil {
activeID = *project.ActiveDeploymentID
}
toDelete := selectDeploymentsToPrune(deployments, activeID, keepCount)
if len(toDelete) == 0 {
return 0, nil
}
// Delete metadata in one transaction so partial prune does not leave
// orphan file-list rows without a parent deployment.
if err := db.DB(ctx).Transaction(func(tx *gorm.DB) error {
for index := range toDelete {
deployment := toDelete[index]
// Never delete the active deployment even if project pointer raced.
if activeID != 0 && deployment.ID == activeID {
continue
}
if project.ActiveDeploymentID != nil && deployment.ID == *project.ActiveDeploymentID {
continue
}
if err := tx.Where("deployment_id = ?", deployment.ID).Delete(&model.PagesDeploymentFile{}).Error; err != nil {
return fmt.Errorf("delete deployment files id=%d: %w", deployment.ID, err)
}
if err := tx.Where("id = ? AND project_id = ?", deployment.ID, projectID).
Delete(&model.PagesDeployment{}).Error; err != nil {
return fmt.Errorf("delete deployment id=%d: %w", deployment.ID, err)
}
}
return nil
}); err != nil {
return 0, err
}
// Artifacts are best-effort outside the transaction (object storage I/O).
for index := range toDelete {
deployment := toDelete[index]
if activeID != 0 && deployment.ID == activeID {
continue
}
removeDeploymentArtifact(ctx, &deployment)
}
logger.InfoF(ctx,
"[Pages] pruned deployment history: project_id=%d keep=%d deleted=%d",
projectID, keepCount, len(toDelete),
)
return len(toDelete), nil
}
// selectDeploymentsToPrune returns deployments that should be removed under the
// "at most keepCount, always keep active, fill with newest" policy.
// deployments must be ordered newest-first (id desc).
func selectDeploymentsToPrune(deployments []model.PagesDeployment, activeID uint, keepCount int) []model.PagesDeployment {
if keepCount <= 0 || len(deployments) <= keepCount {
return nil
}
keepIDs := make(map[uint]struct{}, keepCount)
// 1) Active is always retained and occupies one slot when present.
if activeID != 0 {
// Only count active if it still exists in the list.
for _, deployment := range deployments {
if deployment.ID == activeID {
keepIDs[activeID] = struct{}{}
break
}
}
}
// 2) Fill remaining slots from newest to oldest.
for _, deployment := range deployments {
if len(keepIDs) >= keepCount {
break
}
keepIDs[deployment.ID] = struct{}{}
}
toDelete := make([]model.PagesDeployment, 0, len(deployments)-len(keepIDs))
for _, deployment := range deployments {
if _, keep := keepIDs[deployment.ID]; keep {
continue
}
// Safety: never mark active for deletion.
if activeID != 0 && deployment.ID == activeID {
continue
}
toDelete = append(toDelete, deployment)
}
return toDelete
}
// ActivateDeployment 激活 Pages 部署。 // ActivateDeployment 激活 Pages 部署。
func ActivateDeployment(ctx context.Context, projectID uint, deploymentID uint) (*View, error) { func ActivateDeployment(ctx context.Context, projectID uint, deploymentID uint) (*View, error) {
project, err := model.GetPagesProjectByID(ctx, projectID) project, err := model.GetPagesProjectByID(ctx, projectID)
@@ -422,10 +572,14 @@ func openDeploymentPackageFromUpload(ctx context.Context, uploadID uint64, deplo
} }
contentType := opened.ContentType contentType := opened.ContentType
if contentType == "" { if contentType == "" {
contentType = mimeTypeApplicationZip contentType = opened.Upload.MimeType
} }
if contentType == "" {
contentType = "application/octet-stream"
}
fileName := packageDownloadName(deploymentID, opened.Upload.FileName, contentType)
return DeploymentPackage{ return DeploymentPackage{
FileName: fmt.Sprintf("pages-deployment-%d.zip", deploymentID), FileName: fileName,
ContentType: contentType, ContentType: contentType,
ContentLength: opened.ContentLength, ContentLength: opened.ContentLength,
Body: opened.Body, Body: opened.Body,
@@ -478,6 +632,7 @@ func hydrateLegacyDeploymentUpload(
deployment.Checksum, deployment.Checksum,
project.Slug, project.Slug,
fmt.Sprintf("pages-deployment-%d.zip", deployment.ID), fmt.Sprintf("pages-deployment-%d.zip", deployment.ID),
pagesarchive.FormatZip,
) )
if err != nil { if err != nil {
return nil, err return nil, err
@@ -493,7 +648,26 @@ func hydrateLegacyDeploymentUpload(
return &ingestResult.Upload, nil return &ingestResult.Upload, nil
} }
// ensureDeploymentInActiveSnapshot allows Agent package download when the
// deployment is the project's current active deployment and that project is
// used by at least one pages route in the active main config.
//
// Main config versions pin historical pages_deployment ids for audit only.
// Runtime download always follows the live active Pages deployment (dual
// version control); rolling back main config must not require old packages.
func ensureDeploymentInActiveSnapshot(ctx context.Context, deploymentID uint) error { func ensureDeploymentInActiveSnapshot(ctx context.Context, deploymentID uint) error {
deployment, err := model.GetPagesDeploymentByID(ctx, deploymentID)
if err != nil {
return err
}
project, err := model.GetPagesProjectByID(ctx, deployment.ProjectID)
if err != nil {
return err
}
if project.ActiveDeploymentID == nil || *project.ActiveDeploymentID != deployment.ID {
return errors.New(errPagesPackageNotInActiveConfig)
}
version, err := model.GetActiveConfigVersion(ctx) version, err := model.GetActiveConfigVersion(ctx)
if err != nil { if err != nil {
if errors.Is(err, gorm.ErrRecordNotFound) { if errors.Is(err, gorm.ErrRecordNotFound) {
@@ -506,10 +680,30 @@ func ensureDeploymentInActiveSnapshot(ctx context.Context, deploymentID uint) er
return err return err
} }
for _, route := range routes { for _, route := range routes {
if route.UpstreamType != "pages" || route.PagesDeployment == nil { if !strings.EqualFold(strings.TrimSpace(route.UpstreamType), "pages") {
continue continue
} }
if route.PagesDeployment.DeploymentID == deploymentID { if route.PagesProjectID != nil && *route.PagesProjectID == project.ID {
return nil
}
if route.PagesDeployment == nil {
continue
}
if route.PagesDeployment.ProjectID == project.ID {
return nil
}
// Frozen snapshot may only carry deployment_id; resolve project via that row.
if route.PagesDeployment.DeploymentID == 0 {
continue
}
if route.PagesDeployment.DeploymentID == deployment.ID {
return nil
}
snapDeployment, snapErr := model.GetPagesDeploymentByID(ctx, route.PagesDeployment.DeploymentID)
if snapErr != nil {
continue
}
if snapDeployment.ProjectID == project.ID {
return nil return nil
} }
} }
@@ -518,10 +712,12 @@ func ensureDeploymentInActiveSnapshot(ctx context.Context, deploymentID uint) er
type snapshotPagesDeployment struct { type snapshotPagesDeployment struct {
DeploymentID uint `json:"deployment_id"` DeploymentID uint `json:"deployment_id"`
ProjectID uint `json:"project_id"`
} }
type snapshotRouteRef struct { type snapshotRouteRef struct {
UpstreamType string `json:"upstream_type"` UpstreamType string `json:"upstream_type"`
PagesProjectID *uint `json:"pages_project_id"`
PagesDeployment *snapshotPagesDeployment `json:"pages_deployment"` PagesDeployment *snapshotPagesDeployment `json:"pages_deployment"`
} }
@@ -4,8 +4,10 @@
package pages package pages
import ( import (
"archive/tar"
"archive/zip" "archive/zip"
"bytes" "bytes"
"compress/gzip"
"context" "context"
"fmt" "fmt"
"io" "io"
@@ -17,6 +19,7 @@ import (
"github.com/Rain-kl/Wavelet/internal/db" "github.com/Rain-kl/Wavelet/internal/db"
"github.com/Rain-kl/Wavelet/internal/model" "github.com/Rain-kl/Wavelet/internal/model"
"github.com/Rain-kl/Wavelet/internal/repository"
"github.com/Rain-kl/Wavelet/internal/storage" "github.com/Rain-kl/Wavelet/internal/storage"
"github.com/glebarez/sqlite" "github.com/glebarez/sqlite"
"github.com/stretchr/testify/assert" "github.com/stretchr/testify/assert"
@@ -40,6 +43,7 @@ func setupPagesTestDB(t *testing.T) func() {
&model.PagesDeployment{}, &model.PagesDeployment{},
&model.PagesDeploymentFile{}, &model.PagesDeploymentFile{},
&model.ConfigVersion{}, &model.ConfigVersion{},
&model.SystemConfig{},
)) ))
require.NoError(t, sqliteDB.Create(&model.User{ require.NoError(t, sqliteDB.Create(&model.User{
ID: 999, ID: 999,
@@ -48,8 +52,25 @@ func setupPagesTestDB(t *testing.T) func() {
Nickname: "系统", Nickname: "系统",
IsActive: true, IsActive: true,
}).Error) }).Error)
require.NoError(t, sqliteDB.Create([]model.SystemConfig{
{
Key: model.ConfigKeyPagesMaxPackageSizeMB,
Value: "100",
Type: "business",
Description: "Pages 部署包上传大小上限(MiB)",
},
{
Key: model.ConfigKeyPagesMaxHistoryCount,
Value: "0", // unlimited for existing tests
Type: "business",
Description: "Pages 每个项目最大历史部署保留数(0 表示不限制)",
},
}).Error)
db.SetDB(sqliteDB) db.SetDB(sqliteDB)
// Clear process-global system config RAM cache so tests do not see stale values.
_ = repository.InvalidateSystemConfigCache(context.Background(), model.ConfigKeyPagesMaxPackageSizeMB)
_ = repository.InvalidateSystemConfigCache(context.Background(), model.ConfigKeyPagesMaxHistoryCount)
return func() { return func() {
db.SetDB(nil) db.SetDB(nil)
} }
@@ -339,6 +360,149 @@ func testPagesZip(t *testing.T, files map[string]string) []byte {
return buffer.Bytes() return buffer.Bytes()
} }
func TestUploadDeploymentAcceptsTarGz(t *testing.T) {
cleanup := setupPagesTestDB(t)
defer cleanup()
_, disableStorage := setupPagesStorageMock(t)
defer disableStorage()
ctx := context.Background()
project, err := CreateProject(ctx, Input{
Name: "TarGz Site",
Slug: "tar-gz-site",
Enabled: true,
})
require.NoError(t, err)
deployment, err := UploadDeployment(ctx, project.ID, testPagesMultipartFile(t, "site.tar.gz", testPagesTarGz(t, map[string]string{
"index.html": "tar-ok",
"app.js": "1",
})), "root")
require.NoError(t, err)
assert.Equal(t, 2, deployment.FileCount)
assert.NotZero(t, deployment.UploadID)
}
func TestSelectDeploymentsToPruneKeepsActiveAndNewest(t *testing.T) {
// ids 4(newest) ... 1(oldest); active is oldest id=1; keep=2 → keep {1,4}, prune {3,2}
deployments := []model.PagesDeployment{
{ID: 4, ProjectID: 1},
{ID: 3, ProjectID: 1},
{ID: 2, ProjectID: 1},
{ID: 1, ProjectID: 1},
}
toDelete := selectDeploymentsToPrune(deployments, 1, 2)
require.Len(t, toDelete, 2)
assert.Equal(t, uint(3), toDelete[0].ID)
assert.Equal(t, uint(2), toDelete[1].ID)
// active is newest; keep=2 → keep {4,3}, prune {2,1}
toDelete = selectDeploymentsToPrune(deployments, 4, 2)
require.Len(t, toDelete, 2)
assert.Equal(t, uint(2), toDelete[0].ID)
assert.Equal(t, uint(1), toDelete[1].ID)
// no active; keep=2 → keep {4,3}
toDelete = selectDeploymentsToPrune(deployments, 0, 2)
require.Len(t, toDelete, 2)
assert.Equal(t, uint(2), toDelete[0].ID)
assert.Equal(t, uint(1), toDelete[1].ID)
// keep=1 with active → only active, prune the rest
toDelete = selectDeploymentsToPrune(deployments, 2, 1)
require.Len(t, toDelete, 3)
for _, item := range toDelete {
assert.NotEqual(t, uint(2), item.ID)
}
// already within limit
assert.Nil(t, selectDeploymentsToPrune(deployments[:2], 4, 2))
// unlimited
assert.Nil(t, selectDeploymentsToPrune(deployments, 1, 0))
}
func TestPruneProjectDeploymentHistory(t *testing.T) {
cleanup := setupPagesTestDB(t)
defer cleanup()
_, disableStorage := setupPagesStorageMock(t)
defer disableStorage()
ctx := context.Background()
require.NoError(t, db.DB(ctx).Model(&model.SystemConfig{}).
Where("key = ?", model.ConfigKeyPagesMaxHistoryCount).
Update("value", "2").Error)
require.NoError(t, repository.InvalidateSystemConfigCache(ctx, model.ConfigKeyPagesMaxHistoryCount))
project, err := CreateProject(ctx, Input{
Name: "History Site",
Slug: "history-site",
Enabled: true,
})
require.NoError(t, err)
var ids []uint
for i := 0; i < 3; i++ {
deployment, uploadErr := UploadDeployment(ctx, project.ID, testPagesMultipartFile(t, "site.zip", testPagesZip(t, map[string]string{
"index.html": fmt.Sprintf("v%d", i),
})), "root")
require.NoError(t, uploadErr)
ids = append(ids, deployment.ID)
}
// After 3 uploads with keep=2 and no active: only 2 newest remain.
deployments, err := model.ListPagesDeployments(ctx, project.ID)
require.NoError(t, err)
require.Len(t, deployments, 2)
assert.Equal(t, ids[2], deployments[0].ID)
assert.Equal(t, ids[1], deployments[1].ID)
// Activate the older of the remaining two, then upload again.
_, err = ActivateDeployment(ctx, project.ID, ids[1])
require.NoError(t, err)
latest, err := UploadDeployment(ctx, project.ID, testPagesMultipartFile(t, "site.zip", testPagesZip(t, map[string]string{
"index.html": "v-latest",
})), "root")
require.NoError(t, err)
deployments, err = model.ListPagesDeployments(ctx, project.ID)
require.NoError(t, err)
require.Len(t, deployments, 2, "must be at most N=2, not active+N newest")
storedProject, err := model.GetPagesProjectByID(ctx, project.ID)
require.NoError(t, err)
require.NotNil(t, storedProject.ActiveDeploymentID)
assert.Equal(t, ids[1], *storedProject.ActiveDeploymentID)
kept := map[uint]struct{}{}
for _, item := range deployments {
kept[item.ID] = struct{}{}
}
_, hasActive := kept[ids[1]]
_, hasLatest := kept[latest.ID]
assert.True(t, hasActive, "active deployment must be retained")
assert.True(t, hasLatest, "newest deployment must fill remaining slot")
}
func testPagesTarGz(t *testing.T, files map[string]string) []byte {
t.Helper()
var buffer bytes.Buffer
gzWriter := gzip.NewWriter(&buffer)
tarWriter := tar.NewWriter(gzWriter)
for name, content := range files {
require.NoError(t, tarWriter.WriteHeader(&tar.Header{
Name: name,
Mode: 0o644,
Size: int64(len(content)),
}))
_, err := tarWriter.Write([]byte(content))
require.NoError(t, err)
}
require.NoError(t, tarWriter.Close())
require.NoError(t, gzWriter.Close())
return buffer.Bytes()
}
func testPagesMultipartFile(t *testing.T, fileName string, content []byte) *multipart.FileHeader { func testPagesMultipartFile(t *testing.T, fileName string, content []byte) *multipart.FileHeader {
t.Helper() t.Helper()
+236
View File
@@ -0,0 +1,236 @@
// Copyright 2026 Arctel.net
// SPDX-License-Identifier: Apache-2.0
package pages
import (
"context"
"encoding/json"
"fmt"
"strings"
"github.com/Rain-kl/Wavelet/internal/model"
openrestyrender "github.com/Rain-kl/Wavelet/pkg/render/openresty"
"gorm.io/gorm"
)
// RebindSnapshotPagesToCurrentActive rewrites pages_deployment fields so every
// pages route points at the project's current active deployment.
//
// Main config versions and Pages deployments are independent. Rolling back a
// main config version must not require old Pages packages; Agents always follow
// the live active deployment for each referenced project.
//
// Returns the original JSON unchanged when there are no pages routes. Does not
// mutate stored config_versions rows. Non-pages route fields are preserved.
func RebindSnapshotPagesToCurrentActive(ctx context.Context, snapshotJSON string) (string, error) {
text := strings.TrimSpace(snapshotJSON)
if text == "" {
return snapshotJSON, nil
}
if strings.HasPrefix(text, "[") {
var routes []map[string]json.RawMessage
if err := json.Unmarshal([]byte(text), &routes); err != nil {
return "", fmt.Errorf("parse pages snapshot routes: %w", err)
}
changed, err := rebindPagesRouteMaps(ctx, routes)
if err != nil {
return "", err
}
if !changed {
return snapshotJSON, nil
}
encoded, err := json.Marshal(routes)
if err != nil {
return "", err
}
return string(encoded), nil
}
var raw map[string]json.RawMessage
if err := json.Unmarshal([]byte(text), &raw); err != nil {
return "", fmt.Errorf("parse pages snapshot document: %w", err)
}
routesRaw, ok := raw["routes"]
if !ok || len(routesRaw) == 0 {
return snapshotJSON, nil
}
var routes []map[string]json.RawMessage
if err := json.Unmarshal(routesRaw, &routes); err != nil {
return "", fmt.Errorf("parse pages snapshot routes: %w", err)
}
changed, err := rebindPagesRouteMaps(ctx, routes)
if err != nil {
return "", err
}
if !changed {
return snapshotJSON, nil
}
encodedRoutes, err := json.Marshal(routes)
if err != nil {
return "", err
}
raw["routes"] = encodedRoutes
encoded, err := json.Marshal(raw)
if err != nil {
return "", err
}
return string(encoded), nil
}
func rebindPagesRouteMaps(ctx context.Context, routes []map[string]json.RawMessage) (bool, error) {
changed := false
for index := range routes {
route := routes[index]
if route == nil {
continue
}
upstreamType, _ := rawJSONString(route["upstream_type"])
if !strings.EqualFold(strings.TrimSpace(upstreamType), "pages") {
continue
}
siteName, _ := rawJSONString(route["site_name"])
projectID, err := resolveProjectIDFromRouteMap(route)
if err != nil {
if siteName == "" {
siteName = "pages"
}
return false, fmt.Errorf("路由 %s %w", siteName, err)
}
project, activeDeployment, err := loadActivePagesProject(ctx, projectID, siteName)
if err != nil {
return false, err
}
deployment := buildLivePagesDeployment(project, activeDeployment)
projectIDCopy := project.ID
originURL := fmt.Sprintf("openflare-pages://project/%d", project.ID)
if err := putJSON(route, "pages_project_id", projectIDCopy); err != nil {
return false, err
}
if err := putJSON(route, "pages_deployment", deployment); err != nil {
return false, err
}
if err := putJSON(route, "origin_url", originURL); err != nil {
return false, err
}
if err := putJSON(route, "upstreams", []string{originURL}); err != nil {
return false, err
}
routes[index] = route
changed = true
}
return changed, nil
}
const jsonNullLiteral = "null"
func isPresentJSON(raw json.RawMessage) bool {
return len(raw) > 0 && string(raw) != jsonNullLiteral
}
func resolveProjectIDFromRouteMap(route map[string]json.RawMessage) (uint, error) {
if raw, ok := route["pages_project_id"]; ok && isPresentJSON(raw) {
var projectID uint
if err := json.Unmarshal(raw, &projectID); err == nil && projectID != 0 {
return projectID, nil
}
}
if raw, ok := route["pages_deployment"]; ok && isPresentJSON(raw) {
var deployment struct {
ProjectID uint `json:"project_id"`
}
if err := json.Unmarshal(raw, &deployment); err == nil && deployment.ProjectID != 0 {
return deployment.ProjectID, nil
}
}
return 0, fmt.Errorf("pages 配置无效: 缺少 pages_project_id")
}
func loadActivePagesProject(ctx context.Context, projectID uint, siteName string) (*model.PagesProject, *model.PagesDeployment, error) {
if siteName == "" {
siteName = "pages"
}
project, err := model.GetPagesProjectByID(ctx, projectID)
if err != nil {
if errorsIsNotFound(err) {
return nil, nil, fmt.Errorf("路由 %s Pages 配置无效: pages 项目不存在", siteName)
}
return nil, nil, err
}
if !project.Enabled {
return nil, nil, fmt.Errorf("路由 %s Pages 配置无效: pages 项目未启用", siteName)
}
if project.ActiveDeploymentID == nil || *project.ActiveDeploymentID == 0 {
return nil, nil, fmt.Errorf("路由 %s Pages 配置无效: pages 项目没有激活部署", siteName)
}
activeDeployment, err := model.GetPagesDeploymentByID(ctx, *project.ActiveDeploymentID)
if err != nil {
if errorsIsNotFound(err) {
return nil, nil, fmt.Errorf("路由 %s Pages 配置无效: pages 激活部署不存在", siteName)
}
return nil, nil, err
}
if activeDeployment.ProjectID != project.ID {
return nil, nil, fmt.Errorf("路由 %s Pages 配置无效: pages 激活部署不匹配", siteName)
}
if strings.TrimSpace(activeDeployment.Checksum) == "" {
return nil, nil, fmt.Errorf("路由 %s Pages 配置无效: pages 部署校验和缺失", siteName)
}
return project, activeDeployment, nil
}
func buildLivePagesDeployment(project *model.PagesProject, active *model.PagesDeployment) *openrestyrender.PagesDeployment {
entryFile := strings.TrimSpace(project.EntryFile)
if entryFile == "" {
entryFile = defaultPagesEntryFile
}
fallbackPath := strings.TrimSpace(project.SPAFallbackPath)
if fallbackPath == "" {
fallbackPath = defaultPagesFallbackPath
}
return &openrestyrender.PagesDeployment{
ProjectID: project.ID,
ProjectSlug: strings.TrimSpace(project.Slug),
DeploymentID: active.ID,
DeploymentNumber: active.DeploymentNumber,
Checksum: strings.TrimSpace(active.Checksum),
EntryFile: entryFile,
SPAFallbackEnabled: project.SPAFallbackEnabled,
SPAFallbackPath: fallbackPath,
APIProxyEnabled: project.APIProxyEnabled,
APIProxyPath: strings.TrimSpace(project.APIProxyPath),
APIProxyPass: strings.TrimSpace(project.APIProxyPass),
APIProxyRewrite: strings.TrimSpace(project.APIProxyRewrite),
LocalRoot: fmt.Sprintf(
"%s/deployments/%d/current",
openrestyrender.PagesDirPlaceholder,
active.ID,
),
}
}
func rawJSONString(raw json.RawMessage) (string, bool) {
if !isPresentJSON(raw) {
return "", false
}
var value string
if err := json.Unmarshal(raw, &value); err != nil {
return "", false
}
return value, true
}
func putJSON(route map[string]json.RawMessage, key string, value any) error {
encoded, err := json.Marshal(value)
if err != nil {
return err
}
route[key] = encoded
return nil
}
func errorsIsNotFound(err error) bool {
return err != nil && (err == gorm.ErrRecordNotFound || strings.Contains(strings.ToLower(err.Error()), "record not found"))
}
@@ -0,0 +1,87 @@
// Copyright 2026 Arctel.net
// SPDX-License-Identifier: Apache-2.0
package pages
import (
"context"
"encoding/json"
"testing"
"github.com/Rain-kl/Wavelet/internal/db"
"github.com/Rain-kl/Wavelet/internal/model"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
)
func TestRebindSnapshotPagesToCurrentActive(t *testing.T) {
cleanup := setupPagesTestDB(t)
defer cleanup()
ctx := context.Background()
project, err := CreateProject(ctx, Input{
Name: "Rebind Site",
Slug: "rebind-site",
Enabled: true,
})
require.NoError(t, err)
old := &model.PagesDeployment{
ProjectID: project.ID,
DeploymentNumber: 1,
Checksum: "old-checksum",
Status: model.PagesDeploymentStatusUploaded,
FileCount: 1,
}
require.NoError(t, db.DB(ctx).Create(old).Error)
active := &model.PagesDeployment{
ProjectID: project.ID,
DeploymentNumber: 2,
Checksum: "new-checksum",
Status: model.PagesDeploymentStatusActive,
FileCount: 1,
}
require.NoError(t, db.DB(ctx).Create(active).Error)
require.NoError(t, db.DB(ctx).Model(&model.PagesProject{}).
Where("id = ?", project.ID).
Update("active_deployment_id", active.ID).Error)
// Frozen snapshot still points at the old deployment (simulates old main config).
frozen := map[string]any{
"routes": []map[string]any{
{
"site_name": "rebind",
"origin_url": "openflare-pages://project/1",
"enabled": true,
"upstream_type": "pages",
"pages_project_id": project.ID,
"pages_deployment": map[string]any{
"project_id": project.ID,
"deployment_id": old.ID,
"checksum": "old-checksum",
"local_root": "__OPENFLARE_PAGES_DIR__/deployments/1/current",
},
"extra_keep_me": "yes",
},
},
"waf": map[string]any{"rule_groups": []any{}},
}
frozenJSON, err := json.Marshal(frozen)
require.NoError(t, err)
reboundJSON, err := RebindSnapshotPagesToCurrentActive(ctx, string(frozenJSON))
require.NoError(t, err)
var rebound map[string]any
require.NoError(t, json.Unmarshal([]byte(reboundJSON), &rebound))
_, hasWAF := rebound["waf"]
assert.True(t, hasWAF)
routes := rebound["routes"].([]any)
require.Len(t, routes, 1)
route := routes[0].(map[string]any)
assert.Equal(t, "yes", route["extra_keep_me"])
deployment := route["pages_deployment"].(map[string]any)
assert.EqualValues(t, active.ID, deployment["deployment_id"])
assert.Equal(t, "new-checksum", deployment["checksum"])
}
@@ -0,0 +1,9 @@
-- +goose Up
INSERT INTO w_system_configs (key, value, type, visibility, description, created_at, updated_at)
VALUES
('pages_max_package_size_mb', '100', 'business', 0, 'Pages 部署包上传大小上限(MiB)', CURRENT_TIMESTAMP, CURRENT_TIMESTAMP),
('pages_max_history_count', '20', 'business', 0, 'Pages 每个项目最大历史部署保留数(0 表示不限制)', CURRENT_TIMESTAMP, CURRENT_TIMESTAMP)
ON CONFLICT (key) DO NOTHING;
-- +goose Down
DELETE FROM w_system_configs WHERE key IN ('pages_max_package_size_mb', 'pages_max_history_count');
@@ -0,0 +1,9 @@
-- +goose Up
INSERT INTO w_system_configs (key, value, type, visibility, description, created_at, updated_at)
VALUES
('pages_max_package_size_mb', '100', 'business', 0, 'Pages 部署包上传大小上限(MiB)', CURRENT_TIMESTAMP, CURRENT_TIMESTAMP),
('pages_max_history_count', '20', 'business', 0, 'Pages 每个项目最大历史部署保留数(0 表示不限制)', CURRENT_TIMESTAMP, CURRENT_TIMESTAMP)
ON CONFLICT (key) DO NOTHING;
-- +goose Down
DELETE FROM w_system_configs WHERE key IN ('pages_max_package_size_mb', 'pages_max_history_count');
+4
View File
@@ -50,6 +50,10 @@ const (
ConfigKeyDatabaseAutoCleanupEnabled = "database_auto_cleanup_enabled" // 数据库自动清理开关 ConfigKeyDatabaseAutoCleanupEnabled = "database_auto_cleanup_enabled" // 数据库自动清理开关
ConfigKeyDatabaseAutoCleanupRetentionDays = "database_auto_cleanup_retention_days" // 数据库保留天数 ConfigKeyDatabaseAutoCleanupRetentionDays = "database_auto_cleanup_retention_days" // 数据库保留天数
// Pages 静态托管配置
ConfigKeyPagesMaxPackageSizeMB = "pages_max_package_size_mb" // Pages 部署包上传大小上限(MiB)
ConfigKeyPagesMaxHistoryCount = "pages_max_history_count" // Pages 每个项目最大历史部署保留数(0 表示不限制)
// UptimeKuma 集成配置 // UptimeKuma 集成配置
ConfigKeyUptimeKumaEnabled = "uptime_kuma_enabled" // UptimeKuma 集成开关 ConfigKeyUptimeKumaEnabled = "uptime_kuma_enabled" // UptimeKuma 集成开关
ConfigKeyUptimeKumaURL = "uptime_kuma_url" // UptimeKuma URL ConfigKeyUptimeKumaURL = "uptime_kuma_url" // UptimeKuma URL
+12
View File
@@ -295,6 +295,18 @@ func getSeedConfigsPart2() []model.SystemConfig {
Type: configTypeBusiness, Type: configTypeBusiness,
Description: "FRPS 内置 Web 界面端口", Description: "FRPS 内置 Web 界面端口",
}, },
{
Key: model.ConfigKeyPagesMaxPackageSizeMB,
Value: "100",
Type: configTypeBusiness,
Description: "Pages 部署包上传大小上限(MiB)",
},
{
Key: model.ConfigKeyPagesMaxHistoryCount,
Value: "20",
Type: configTypeBusiness,
Description: "Pages 每个项目最大历史部署保留数(0 表示不限制)",
},
} }
} }
+101
View File
@@ -0,0 +1,101 @@
// Copyright 2026 Arctel.net
// SPDX-License-Identifier: Apache-2.0
package pagesarchive
import (
"crypto/sha256"
"encoding/hex"
"fmt"
"io"
"math"
"os"
"path/filepath"
)
// Limits bounds archive inspection / extraction work.
type Limits struct {
// MaxFiles is the maximum number of regular files allowed.
MaxFiles int
// MaxFileBytes is the maximum size of a single extracted file.
MaxFileBytes int64
// MaxTotalBytes is the maximum sum of all extracted file sizes.
MaxTotalBytes int64
}
// FileEntry is a regular file discovered inside a deployment package.
type FileEntry struct {
Path string
Size int64
Checksum string
}
// Manifest is the inspected content of a Pages deployment package.
type Manifest struct {
Files []FileEntry
FileCount int
TotalSize int64
}
// Entry describes one archive member for extraction.
type Entry struct {
// Name is the original path inside the archive.
Name string
// IsDir marks directory entries.
IsDir bool
// IsSymlink marks symbolic links (unsupported for Pages).
IsSymlink bool
// Size is the declared uncompressed size when known; 0 means unknown.
Size uint64
// Open returns a reader for the entry body. Caller must Close it.
Open func() (io.ReadCloser, error)
}
// copyLimited copies src to dst.
// When maxBytes <= 0, size limits are not enforced (trusted extract path).
func copyLimited(dst io.Writer, src io.Reader, declaredSize uint64, maxBytes int64) (int64, error) {
if maxBytes <= 0 {
if declaredSize > 0 {
if declaredSize > uint64(math.MaxInt64) {
return 0, fmt.Errorf("pages file size out of bounds")
}
//nolint:gosec // declaredSize is bounded to MaxInt64 above
return io.CopyN(dst, src, int64(declaredSize))
}
return io.Copy(dst, src)
}
if declaredSize > uint64(maxBytes) || declaredSize > uint64(math.MaxInt64) { //nolint:gosec // maxBytes positive
return 0, fmt.Errorf("pages file size out of bounds")
}
if declaredSize > 0 {
//nolint:gosec // declaredSize is bounded to MaxInt64 above
return io.CopyN(dst, src, int64(declaredSize))
}
limited := io.LimitReader(src, maxBytes+1)
written, err := io.Copy(dst, limited)
if written > maxBytes {
return written, fmt.Errorf("pages file size out of bounds")
}
return written, err
}
func checksumReader(src io.Reader, declaredSize uint64, maxBytes int64) (string, int64, error) {
hash := sha256.New()
written, err := copyLimited(hash, src, declaredSize, maxBytes)
if err != nil {
return "", written, err
}
return hex.EncodeToString(hash.Sum(nil)), written, nil
}
func writeEntryFile(targetPath string, src io.Reader, declaredSize uint64, maxBytes int64, perm os.FileMode) (int64, error) {
if err := os.MkdirAll(filepath.Dir(targetPath), dirPerm); err != nil {
return 0, err
}
target, err := os.OpenFile(targetPath, os.O_CREATE|os.O_WRONLY|os.O_TRUNC, perm) //nolint:gosec // caller validates path under release dir
if err != nil {
return 0, err
}
defer func() { _ = target.Close() }()
return copyLimited(target, src, declaredSize, maxBytes)
}
+154
View File
@@ -0,0 +1,154 @@
// Copyright 2026 Arctel.net
// SPDX-License-Identifier: Apache-2.0
package pagesarchive
import (
"fmt"
"os"
"path/filepath"
)
// ExtractOptions controls package extraction.
type ExtractOptions struct {
// Limits bounds files and sizes during extraction when EnforceLimits is true.
Limits Limits
// StripCommonRoot strips a single shared top-level directory when present.
StripCommonRoot bool
// EnforceLimits enables MaxFiles / MaxFileBytes / MaxTotalBytes checks.
// When false, the caller is assumed to have already validated the package
// (e.g. Agent trusts control-plane inspection). Path-escape and symlink
// guards still apply so local extraction cannot leave destDir.
EnforceLimits bool
}
// ExtractBytes extracts a deployment package into destDir.
func ExtractBytes(data []byte, format Format, destDir string, opts ExtractOptions) error {
if format == "" {
var err error
format, err = DetectFormat("", data)
if err != nil {
return err
}
}
entries, err := listEntries(data, format)
if err != nil {
return err
}
return extractEntries(entries, destDir, opts)
}
// ExtractFile reads path and extracts it into destDir.
func ExtractFile(filePath string, format Format, destDir string, opts ExtractOptions) error {
data, err := os.ReadFile(filePath) //nolint:gosec // controlled path
if err != nil {
return err
}
return ExtractBytes(data, format, destDir, opts)
}
func extractEntries(entries []Entry, destDir string, opts ExtractOptions) error {
limits := Limits{}
if opts.EnforceLimits {
limits = normalizeLimits(opts.Limits)
}
commonPrefix := ""
if opts.StripCommonRoot {
commonPrefix = FindCommonRootPrefix(collectFileNames(entries))
}
var totalSize int64
var fileCount int
for _, entry := range entries {
written, counted, err := extractSingleEntry(entry, destDir, commonPrefix, limits, opts.EnforceLimits)
if err != nil {
return err
}
if !counted {
continue
}
fileCount++
if opts.EnforceLimits && fileCount > limits.MaxFiles {
return fmt.Errorf("pages deployment file count exceeds %d", limits.MaxFiles)
}
totalSize += written
if opts.EnforceLimits && totalSize > limits.MaxTotalBytes {
return fmt.Errorf("pages extracted size exceeds limit")
}
}
if fileCount == 0 {
return fmt.Errorf("pages package is empty")
}
return nil
}
func extractSingleEntry(
entry Entry,
destDir, commonPrefix string,
limits Limits,
enforceLimits bool,
) (written int64, counted bool, err error) {
relativePath, skip, err := NormalizeEntryPath(entry.Name)
if err != nil {
return 0, false, err
}
if skip {
return 0, false, nil
}
if commonPrefix != "" {
relativePath = StripPrefix(relativePath, commonPrefix)
if relativePath == "" {
return 0, false, nil
}
}
if entry.IsSymlink {
return 0, false, fmt.Errorf("pages package contains unsupported symlink: %s", relativePath)
}
targetPath := filepath.Join(destDir, filepath.FromSlash(relativePath))
if !isWithinDir(destDir, targetPath) {
return 0, false, fmt.Errorf("pages package path escapes directory: %s", entry.Name)
}
if entry.IsDir {
if err := os.MkdirAll(targetPath, dirPerm); err != nil {
return 0, false, err
}
return 0, false, nil
}
maxFileBytes := int64(0) // unlimited when not enforcing
if enforceLimits {
if exceedsFileByteLimit(entry.Size, limits.MaxFileBytes) {
return 0, false, fmt.Errorf("pages file too large: %s", relativePath)
}
maxFileBytes = limits.MaxFileBytes
}
src, err := entry.Open()
if err != nil {
return 0, false, fmt.Errorf("%s: %w", relativePath, err)
}
written, writeErr := writeEntryFile(targetPath, src, entry.Size, maxFileBytes, filePerm)
_ = src.Close()
if writeErr != nil {
return 0, false, fmt.Errorf("%s: %w", relativePath, writeErr)
}
return written, true, nil
}
func isWithinDir(baseDir, targetPath string) bool {
cleanBase := filepath.Clean(baseDir)
cleanTarget := filepath.Clean(targetPath)
rel, err := filepath.Rel(cleanBase, cleanTarget)
if err != nil {
return false
}
return rel != ".." && !hasParentRel(rel)
}
func hasParentRel(rel string) bool {
if rel == ".." {
return true
}
return len(rel) >= 3 && (rel[:3] == "../" || rel[:3] == "..\\")
}
+191
View File
@@ -0,0 +1,191 @@
// Copyright 2026 Arctel.net
// SPDX-License-Identifier: Apache-2.0
// Package pagesarchive provides multi-format archive detection, inspection and
// extraction helpers for OpenFlare Pages deployment packages.
package pagesarchive
import (
"bytes"
"fmt"
"path/filepath"
"strings"
)
// Format identifies a supported Pages deployment package archive format.
type Format string
const (
// FormatZip is a ZIP archive.
FormatZip Format = "zip"
// FormatTar is an uncompressed tar archive.
FormatTar Format = "tar"
// FormatTarGz is a gzip-compressed tar archive.
FormatTarGz Format = "tar.gz"
// FormatTarXz is an xz-compressed tar archive.
FormatTarXz Format = "tar.xz"
// FormatTarBz2 is a bzip2-compressed tar archive.
FormatTarBz2 Format = "tar.bz2"
// FormatSevenZip is a 7z archive.
FormatSevenZip Format = "7z"
ustarMagicOffset = 257
ustarMagicMinLen = 262
)
// DetectFormatFromName returns the archive format inferred from a file name.
// Returns an empty Format and false when the extension is unsupported.
func DetectFormatFromName(fileName string) (Format, bool) {
name := strings.ToLower(strings.TrimSpace(fileName))
switch {
case strings.HasSuffix(name, ".tar.gz"), strings.HasSuffix(name, ".tgz"):
return FormatTarGz, true
case strings.HasSuffix(name, ".tar.xz"), strings.HasSuffix(name, ".txz"):
return FormatTarXz, true
case strings.HasSuffix(name, ".tar.bz2"), strings.HasSuffix(name, ".tbz2"), strings.HasSuffix(name, ".tbz"):
return FormatTarBz2, true
case strings.HasSuffix(name, ".tar"):
return FormatTar, true
case strings.HasSuffix(name, ".7z"):
return FormatSevenZip, true
case strings.HasSuffix(name, ".zip"):
return FormatZip, true
default:
return "", false
}
}
// DetectFormatFromBytes returns the archive format inferred from magic bytes.
// Prefer DetectFormatFromName when a reliable file name is available.
func DetectFormatFromBytes(data []byte) (Format, bool) {
if isZipMagic(data) {
return FormatZip, true
}
if isSevenZipMagic(data) {
return FormatSevenZip, true
}
if isXZMagic(data) {
return FormatTarXz, true
}
if isGzipMagic(data) {
return FormatTarGz, true
}
if isBzip2Magic(data) {
return FormatTarBz2, true
}
if looksLikeTar(data) {
return FormatTar, true
}
return "", false
}
// DetectFormat prefers the file name when present, otherwise magic bytes.
func DetectFormat(fileName string, data []byte) (Format, error) {
if format, ok := DetectFormatFromName(fileName); ok {
return format, nil
}
if format, ok := DetectFormatFromBytes(data); ok {
return format, nil
}
return "", fmt.Errorf("unsupported pages package format")
}
// Extension returns the canonical file extension for a format (without leading dot).
func Extension(format Format) string {
switch format {
case FormatZip:
return "zip"
case FormatTar:
return "tar"
case FormatTarGz:
return "tar.gz"
case FormatTarXz:
return "tar.xz"
case FormatTarBz2:
return "tar.bz2"
case FormatSevenZip:
return "7z"
default:
return "bin"
}
}
// MIMEType returns a reasonable content type for the archive format.
func MIMEType(format Format) string {
switch format {
case FormatZip:
return "application/zip"
case FormatTar:
return "application/x-tar"
case FormatTarGz:
return "application/gzip"
case FormatTarXz:
return "application/x-xz"
case FormatTarBz2:
return "application/x-bzip2"
case FormatSevenZip:
return "application/x-7z-compressed"
default:
return "application/octet-stream"
}
}
// SupportedExtensions lists human-readable extensions for UI copy and accept attributes.
func SupportedExtensions() []string {
return []string{".zip", ".tar.gz", ".tgz", ".tar.xz", ".txz", ".tar.bz2", ".tbz2", ".tar", ".7z"}
}
// AcceptAttribute returns a comma-separated accept list for file inputs.
func AcceptAttribute() string {
return strings.Join(SupportedExtensions(), ",")
}
// NormalizeNameExtension returns a storage-safe extension for the given format/name.
func NormalizeNameExtension(fileName string, format Format) string {
if format != "" {
return Extension(format)
}
if formatFromName, ok := DetectFormatFromName(fileName); ok {
return Extension(formatFromName)
}
ext := strings.TrimPrefix(filepath.Ext(fileName), ".")
if ext == "" {
return "bin"
}
return strings.ToLower(ext)
}
func isZipMagic(data []byte) bool {
return len(data) >= 4 &&
data[0] == 0x50 && data[1] == 0x4b &&
(data[2] == 0x03 || data[2] == 0x05 || data[2] == 0x07) &&
(data[3] == 0x04 || data[3] == 0x06 || data[3] == 0x08)
}
func isSevenZipMagic(data []byte) bool {
return len(data) >= 6 &&
data[0] == 0x37 && data[1] == 0x7a && data[2] == 0xbc &&
data[3] == 0xaf && data[4] == 0x27 && data[5] == 0x1c
}
func isXZMagic(data []byte) bool {
return len(data) >= 6 &&
data[0] == 0xfd && data[1] == 0x37 && data[2] == 0x7a &&
data[3] == 0x58 && data[4] == 0x5a && data[5] == 0x00
}
func isGzipMagic(data []byte) bool {
return len(data) >= 2 && data[0] == 0x1f && data[1] == 0x8b
}
func isBzip2Magic(data []byte) bool {
return len(data) >= 3 && data[0] == 0x42 && data[1] == 0x5a && data[2] == 0x68
}
func looksLikeTar(data []byte) bool {
// POSIX ustar magic at offset 257 ("ustar\0" or "ustar ").
if len(data) < ustarMagicMinLen {
return false
}
return bytes.Equal(data[ustarMagicOffset:ustarMagicOffset+5], []byte("ustar"))
}
+148
View File
@@ -0,0 +1,148 @@
// Copyright 2026 Arctel.net
// SPDX-License-Identifier: Apache-2.0
package pagesarchive
import (
"fmt"
"os"
"path"
"strings"
)
// InspectOptions controls package inspection.
type InspectOptions struct {
// RootDir is an optional project root subdirectory that must contain EntryFile.
RootDir string
// EntryFile is the required entry file name (e.g. index.html).
EntryFile string
// Limits bounds files and sizes.
Limits Limits
}
// InspectFile opens path and inspects it as a Pages deployment package.
func InspectFile(filePath string, format Format, opts InspectOptions) (*Manifest, error) {
data, err := os.ReadFile(filePath) //nolint:gosec // filePath is a controlled temp upload path
if err != nil {
return nil, err
}
return InspectBytes(data, format, opts)
}
// InspectBytes inspects an in-memory deployment package.
func InspectBytes(data []byte, format Format, opts InspectOptions) (*Manifest, error) {
if format == "" {
var err error
format, err = DetectFormat("", data)
if err != nil {
return nil, err
}
}
entries, err := listEntries(data, format)
if err != nil {
return nil, err
}
return buildManifest(entries, opts)
}
func buildManifest(entries []Entry, opts InspectOptions) (*Manifest, error) {
limits := normalizeLimits(opts.Limits)
commonPrefix := FindCommonRootPrefix(collectFileNames(entries))
targetEntryPath := resolveTargetEntryPath(opts.RootDir, opts.EntryFile)
manifest := &Manifest{Files: make([]FileEntry, 0)}
entrySeen := false
for _, entry := range entries {
normalizedPath, skip, err := prepareEntryPath(entry, commonPrefix)
if err != nil {
return nil, err
}
if skip {
continue
}
if exceedsFileByteLimit(entry.Size, limits.MaxFileBytes) {
return nil, fmt.Errorf("pages file too large: %s", normalizedPath)
}
fileEntry, err := inspectRegularFile(entry, normalizedPath, limits)
if err != nil {
return nil, err
}
manifest.FileCount++
if manifest.FileCount > limits.MaxFiles {
return nil, fmt.Errorf("pages deployment file count exceeds %d", limits.MaxFiles)
}
manifest.TotalSize += fileEntry.Size
if manifest.TotalSize > limits.MaxTotalBytes {
return nil, fmt.Errorf("pages extracted size exceeds limit")
}
if normalizedPath == targetEntryPath {
entrySeen = true
}
manifest.Files = append(manifest.Files, fileEntry)
}
if manifest.FileCount == 0 {
return nil, fmt.Errorf("pages package is empty")
}
if !entrySeen {
return nil, fmt.Errorf("pages package is missing entry file %s", targetEntryPath)
}
return manifest, nil
}
func collectFileNames(entries []Entry) []string {
names := make([]string, 0, len(entries))
for _, entry := range entries {
if entry.IsDir || entry.IsSymlink {
continue
}
names = append(names, entry.Name)
}
return names
}
func resolveTargetEntryPath(rootDir, entryFile string) string {
normalizedEntry := strings.TrimSpace(entryFile)
if normalizedEntry == "" {
normalizedEntry = "index.html"
}
normalizedRoot := strings.Trim(strings.TrimSpace(rootDir), "/")
if normalizedRoot == "" {
return normalizedEntry
}
return path.Join(normalizedRoot, normalizedEntry)
}
func prepareEntryPath(entry Entry, commonPrefix string) (string, bool, error) {
normalizedPath, skip, err := NormalizeEntryPath(entry.Name)
if err != nil {
return "", false, err
}
if skip || entry.IsDir {
return "", true, nil
}
normalizedPath = StripPrefix(normalizedPath, commonPrefix)
if entry.IsSymlink {
return "", false, fmt.Errorf("pages package contains unsupported symlink: %s", normalizedPath)
}
return normalizedPath, false, nil
}
func inspectRegularFile(entry Entry, normalizedPath string, limits Limits) (FileEntry, error) {
src, err := entry.Open()
if err != nil {
return FileEntry{}, fmt.Errorf("%s: %w", normalizedPath, err)
}
checksum, fileSize, checksumErr := checksumReader(src, entry.Size, limits.MaxFileBytes)
_ = src.Close()
if checksumErr != nil {
return FileEntry{}, fmt.Errorf("%s: %w", normalizedPath, checksumErr)
}
return FileEntry{
Path: normalizedPath,
Size: fileSize,
Checksum: checksum,
}, nil
}
+38
View File
@@ -0,0 +1,38 @@
// Copyright 2026 Arctel.net
// SPDX-License-Identifier: Apache-2.0
package pagesarchive
const (
defaultMaxFiles = 1000
defaultMaxFileBytes = 100 * 1024 * 1024
defaultMaxTotalBytes = 100 * 1024 * 1024
dirPerm = 0o750
filePerm = 0o644
)
// normalizeLimits applies defaults for control-plane inspection.
// Callers that already validated the package should use EnforceLimits=false instead.
func normalizeLimits(limits Limits) Limits {
if limits.MaxFiles <= 0 {
limits.MaxFiles = defaultMaxFiles
}
if limits.MaxFileBytes <= 0 {
limits.MaxFileBytes = defaultMaxFileBytes
}
if limits.MaxTotalBytes <= 0 {
limits.MaxTotalBytes = defaultMaxTotalBytes
}
return limits
}
func exceedsFileByteLimit(size uint64, maxBytes int64) bool {
// maxBytes <= 0 means unlimited (trusted extract path).
if maxBytes <= 0 {
return false
}
if size == 0 {
return false
}
return size > uint64(maxBytes) //nolint:gosec // maxBytes is positive
}
+220
View File
@@ -0,0 +1,220 @@
// Copyright 2026 Arctel.net
// SPDX-License-Identifier: Apache-2.0
package pagesarchive
import (
"archive/tar"
"archive/zip"
"bytes"
"compress/bzip2"
"compress/gzip"
"fmt"
"io"
"os"
"github.com/bodgit/sevenzip"
"github.com/ulikunitz/xz"
)
type archiveFile interface {
Name() string
IsDir() bool
IsSymlink() bool
Size() uint64
Open() (io.ReadCloser, error)
}
type zipArchiveFile struct {
file *zip.File
}
func (z zipArchiveFile) Name() string { return z.file.Name }
func (z zipArchiveFile) IsDir() bool { return z.file.FileInfo().IsDir() }
func (z zipArchiveFile) IsSymlink() bool {
return z.file.Mode()&os.ModeSymlink != 0
}
func (z zipArchiveFile) Size() uint64 { return z.file.UncompressedSize64 }
func (z zipArchiveFile) Open() (io.ReadCloser, error) {
return z.file.Open()
}
type sevenZipArchiveFile struct {
file *sevenzip.File
}
func (z sevenZipArchiveFile) Name() string { return z.file.Name }
func (z sevenZipArchiveFile) IsDir() bool { return z.file.FileInfo().IsDir() }
func (z sevenZipArchiveFile) IsSymlink() bool {
return z.file.Mode()&os.ModeSymlink != 0
}
func (z sevenZipArchiveFile) Size() uint64 { return z.file.UncompressedSize }
func (z sevenZipArchiveFile) Open() (io.ReadCloser, error) {
return z.file.Open()
}
func listEntries(data []byte, format Format) ([]Entry, error) {
switch format {
case FormatZip:
return listZipEntries(data)
case FormatTar:
return listTarEntries(bytes.NewReader(data))
case FormatTarGz:
gzReader, err := gzip.NewReader(bytes.NewReader(data))
if err != nil {
return nil, fmt.Errorf("open gzip pages package: %w", err)
}
defer func() { _ = gzReader.Close() }()
return listTarEntries(gzReader)
case FormatTarXz:
xzReader, err := xz.NewReader(bytes.NewReader(data))
if err != nil {
return nil, fmt.Errorf("open xz pages package: %w", err)
}
return listTarEntries(xzReader)
case FormatTarBz2:
return listTarEntries(bzip2.NewReader(bytes.NewReader(data)))
case FormatSevenZip:
return listSevenZipEntries(data)
default:
return nil, fmt.Errorf("unsupported pages package format: %s", format)
}
}
func entriesFromArchiveFiles(files []archiveFile) []Entry {
entries := make([]Entry, 0, len(files))
for _, item := range files {
file := item
entries = append(entries, Entry{
Name: file.Name(),
IsDir: file.IsDir(),
IsSymlink: file.IsSymlink(),
Size: file.Size(),
Open: file.Open,
})
}
return entries
}
func listZipEntries(data []byte) ([]Entry, error) {
reader, err := zip.NewReader(bytes.NewReader(data), int64(len(data)))
if err != nil {
return nil, fmt.Errorf("open zip pages package: %w", err)
}
files := make([]archiveFile, 0, len(reader.File))
for _, item := range reader.File {
files = append(files, zipArchiveFile{file: item})
}
return entriesFromArchiveFiles(files), nil
}
func listSevenZipEntries(data []byte) ([]Entry, error) {
reader, err := sevenzip.NewReader(bytes.NewReader(data), int64(len(data)))
if err != nil {
return nil, fmt.Errorf("open 7z pages package: %w", err)
}
files := make([]archiveFile, 0, len(reader.File))
for _, item := range reader.File {
files = append(files, sevenZipArchiveFile{file: item})
}
return entriesFromArchiveFiles(files), nil
}
func listTarEntries(r io.Reader) ([]Entry, error) {
tarReader := tar.NewReader(r)
// Tar is sequential: materialize regular file bodies so entries can be opened later.
type materialised struct {
header *tar.Header
body []byte
}
items := make([]materialised, 0)
for {
header, err := tarReader.Next()
if err == io.EOF {
break
}
if err != nil {
return nil, fmt.Errorf("read tar pages package: %w", err)
}
item, skip, err := materialiseTarHeader(tarReader, header)
if err != nil {
return nil, err
}
if skip {
continue
}
items = append(items, item)
}
entries := make([]Entry, 0, len(items))
for _, item := range items {
entries = append(entries, tarEntryFromMaterialised(item.header, item.body))
}
return entries, nil
}
func materialiseTarHeader(tarReader *tar.Reader, header *tar.Header) (item struct {
header *tar.Header
body []byte
}, skip bool, err error) {
switch header.Typeflag {
case tar.TypeDir, tar.TypeSymlink, tar.TypeLink:
return struct {
header *tar.Header
body []byte
}{header: header}, false, nil
case tar.TypeReg, tar.TypeRegA: //nolint:staticcheck // TypeRegA still appears in older archives
body, readErr := readTarBody(tarReader, header)
if readErr != nil {
return item, false, readErr
}
return struct {
header *tar.Header
body []byte
}{header: header, body: body}, false, nil
default:
if header.Size > 0 {
if _, copyErr := io.CopyN(io.Discard, tarReader, header.Size); copyErr != nil {
return item, false, fmt.Errorf("skip tar entry %s: %w", header.Name, copyErr)
}
}
return item, true, nil
}
}
func readTarBody(tarReader *tar.Reader, header *tar.Header) ([]byte, error) {
if header.Size > 0 {
body := make([]byte, header.Size)
if _, err := io.ReadFull(tarReader, body); err != nil {
return nil, fmt.Errorf("read tar entry %s: %w", header.Name, err)
}
return body, nil
}
body, err := io.ReadAll(tarReader)
if err != nil {
return nil, fmt.Errorf("read tar entry %s: %w", header.Name, err)
}
return body, nil
}
func tarEntryFromMaterialised(header *tar.Header, body []byte) Entry {
size := header.Size
if int64(len(body)) > size {
size = int64(len(body))
}
entry := Entry{
Name: header.Name,
IsDir: header.Typeflag == tar.TypeDir,
IsSymlink: header.Typeflag == tar.TypeSymlink || header.Typeflag == tar.TypeLink,
Size: uint64(size), //nolint:gosec // non-negative sizes
Open: func() (io.ReadCloser, error) {
return io.NopCloser(bytes.NewReader(body)), nil
},
}
if entry.IsDir || entry.IsSymlink {
entry.Open = func() (io.ReadCloser, error) {
return io.NopCloser(bytes.NewReader(nil)), nil
}
}
return entry
}
+187
View File
@@ -0,0 +1,187 @@
// Copyright 2026 Arctel.net
// SPDX-License-Identifier: Apache-2.0
package pagesarchive
import (
"archive/tar"
"archive/zip"
"bytes"
"compress/gzip"
"os"
"path/filepath"
"strings"
"testing"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
"github.com/ulikunitz/xz"
)
func TestDetectFormatFromName(t *testing.T) {
cases := map[string]Format{
"site.zip": FormatZip,
"site.TAR.GZ": FormatTarGz,
"site.tgz": FormatTarGz,
"site.tar.xz": FormatTarXz,
"site.txz": FormatTarXz,
"site.tar.bz2": FormatTarBz2,
"site.tar": FormatTar,
"site.7z": FormatSevenZip,
}
for name, want := range cases {
got, ok := DetectFormatFromName(name)
assert.True(t, ok, name)
assert.Equal(t, want, got, name)
}
_, ok := DetectFormatFromName("site.rar")
assert.False(t, ok)
}
func TestInspectAndExtractZip(t *testing.T) {
data := testZip(t, map[string]string{
"dist/index.html": "<html>ok</html>",
"dist/app.js": "console.log(1)",
})
manifest, err := InspectBytes(data, FormatZip, InspectOptions{
EntryFile: "index.html",
Limits: Limits{MaxFiles: 100, MaxFileBytes: 1 << 20, MaxTotalBytes: 1 << 20},
})
require.NoError(t, err)
assert.Equal(t, 2, manifest.FileCount)
paths := make(map[string]struct{}, len(manifest.Files))
for _, file := range manifest.Files {
paths[file.Path] = struct{}{}
}
assert.Contains(t, paths, "index.html")
assert.Contains(t, paths, "app.js")
dest := t.TempDir()
require.NoError(t, ExtractBytes(data, FormatZip, dest, ExtractOptions{
StripCommonRoot: true,
EnforceLimits: true,
Limits: Limits{MaxFiles: 100, MaxFileBytes: 1 << 20, MaxTotalBytes: 1 << 20},
}))
body, err := os.ReadFile(filepath.Join(dest, "index.html")) //nolint:gosec
require.NoError(t, err)
assert.Equal(t, "<html>ok</html>", string(body))
}
func TestExtractTrustedSkipsSizeLimits(t *testing.T) {
// Content larger than a tiny limit would fail if limits were enforced.
large := strings.Repeat("x", 64)
data := testZip(t, map[string]string{
"index.html": large,
})
dest := t.TempDir()
require.NoError(t, ExtractBytes(data, FormatZip, dest, ExtractOptions{
// Agent trusts control-plane validation: no size/count re-check.
EnforceLimits: false,
}))
body, err := os.ReadFile(filepath.Join(dest, "index.html")) //nolint:gosec
require.NoError(t, err)
assert.Equal(t, large, string(body))
}
func TestInspectAndExtractTarGz(t *testing.T) {
data := testTarGz(t, map[string]string{
"index.html": "<html>tar</html>",
"style.css": "body{}",
})
format, err := DetectFormat("site.tar.gz", data)
require.NoError(t, err)
assert.Equal(t, FormatTarGz, format)
manifest, err := InspectBytes(data, format, InspectOptions{
EntryFile: "index.html",
Limits: Limits{MaxFiles: 100, MaxFileBytes: 1 << 20, MaxTotalBytes: 1 << 20},
})
require.NoError(t, err)
assert.Equal(t, 2, manifest.FileCount)
dest := t.TempDir()
require.NoError(t, ExtractBytes(data, format, dest, ExtractOptions{
EnforceLimits: true,
Limits: Limits{MaxFiles: 100, MaxFileBytes: 1 << 20, MaxTotalBytes: 1 << 20},
}))
body, err := os.ReadFile(filepath.Join(dest, "index.html")) //nolint:gosec
require.NoError(t, err)
assert.Equal(t, "<html>tar</html>", string(body))
}
func TestInspectTarXz(t *testing.T) {
data := testTarXz(t, map[string]string{
"index.html": "<html>xz</html>",
})
manifest, err := InspectBytes(data, FormatTarXz, InspectOptions{
EntryFile: "index.html",
Limits: Limits{MaxFiles: 10, MaxFileBytes: 1 << 20, MaxTotalBytes: 1 << 20},
})
require.NoError(t, err)
assert.Equal(t, 1, manifest.FileCount)
}
func TestRejectZipSlip(t *testing.T) {
data := testZip(t, map[string]string{
"../evil.txt": "x",
"index.html": "ok",
})
_, err := InspectBytes(data, FormatZip, InspectOptions{
EntryFile: "index.html",
Limits: Limits{MaxFiles: 10, MaxFileBytes: 1 << 20, MaxTotalBytes: 1 << 20},
})
require.Error(t, err)
}
func testZip(t *testing.T, files map[string]string) []byte {
t.Helper()
var buffer bytes.Buffer
writer := zip.NewWriter(&buffer)
for name, content := range files {
file, err := writer.Create(name)
require.NoError(t, err)
_, err = file.Write([]byte(content))
require.NoError(t, err)
}
require.NoError(t, writer.Close())
return buffer.Bytes()
}
func testTarGz(t *testing.T, files map[string]string) []byte {
t.Helper()
var buffer bytes.Buffer
gzWriter := gzip.NewWriter(&buffer)
tarWriter := tar.NewWriter(gzWriter)
for name, content := range files {
require.NoError(t, tarWriter.WriteHeader(&tar.Header{
Name: name,
Mode: 0o644,
Size: int64(len(content)),
}))
_, err := tarWriter.Write([]byte(content))
require.NoError(t, err)
}
require.NoError(t, tarWriter.Close())
require.NoError(t, gzWriter.Close())
return buffer.Bytes()
}
func testTarXz(t *testing.T, files map[string]string) []byte {
t.Helper()
var buffer bytes.Buffer
xzWriter, err := xz.NewWriter(&buffer)
require.NoError(t, err)
tarWriter := tar.NewWriter(xzWriter)
for name, content := range files {
require.NoError(t, tarWriter.WriteHeader(&tar.Header{
Name: name,
Mode: 0o644,
Size: int64(len(content)),
}))
_, err := tarWriter.Write([]byte(content))
require.NoError(t, err)
}
require.NoError(t, tarWriter.Close())
require.NoError(t, xzWriter.Close())
return buffer.Bytes()
}
+85
View File
@@ -0,0 +1,85 @@
// Copyright 2026 Arctel.net
// SPDX-License-Identifier: Apache-2.0
package pagesarchive
import (
"fmt"
"path"
"path/filepath"
"strings"
)
// NormalizeEntryPath cleans an archive entry path and rejects zip-slip / absolute paths.
// skip=true means the entry should be ignored (empty path or directory marker).
func NormalizeEntryPath(raw string) (cleaned string, skip bool, err error) {
name := strings.TrimSpace(filepath.ToSlash(raw))
if name == "" {
return "", true, nil
}
if strings.HasSuffix(name, "/") {
return "", true, nil
}
if strings.HasPrefix(name, "/") || path.IsAbs(name) {
return "", false, fmt.Errorf("pages package contains absolute path: %s", raw)
}
// Reject Windows drive / UNC-style paths that may appear after ToSlash.
if len(name) >= 2 && name[1] == ':' {
return "", false, fmt.Errorf("pages package contains absolute path: %s", raw)
}
cleanedPath := path.Clean(name)
if cleanedPath == "." {
return "", true, nil
}
if cleanedPath == ".." || strings.HasPrefix(cleanedPath, "../") || strings.Contains(cleanedPath, "/../") {
return "", false, fmt.Errorf("pages package path escapes directory: %s", raw)
}
return cleanedPath, false, nil
}
// FindCommonRootPrefix returns a trailing-slash directory prefix shared by all file paths.
// When files do not share a single root folder the result is empty.
func FindCommonRootPrefix(paths []string) string {
var firstFilePath string
hasMultipleFiles := false
for _, item := range paths {
normalizedPath, skip, err := NormalizeEntryPath(item)
if err != nil || skip {
continue
}
if firstFilePath == "" {
firstFilePath = normalizedPath
} else {
hasMultipleFiles = true
}
}
if firstFilePath == "" {
return ""
}
parts := strings.Split(firstFilePath, "/")
if len(parts) <= 1 {
return ""
}
commonPrefix := parts[0] + "/"
if !hasMultipleFiles {
return commonPrefix
}
for _, item := range paths {
normalizedPath, skip, err := NormalizeEntryPath(item)
if err != nil || skip {
continue
}
if !strings.HasPrefix(normalizedPath, commonPrefix) {
return ""
}
}
return commonPrefix
}
// StripPrefix removes a directory prefix from a cleaned path when present.
func StripPrefix(normalizedPath, prefix string) string {
if prefix == "" {
return normalizedPath
}
return strings.TrimPrefix(normalizedPath, prefix)
}