mirror of
https://github.com/Rain-kl/OpenFlare.git
synced 2026-09-28 05:46:36 +08:00
feat(pages): configurable limits, multi-format packages, and dual versioning
Make Pages package size and history retention system-configurable, support zip/tar.gz/tar.xz/tar.bz2/tar/7z uploads, prune history with clear keep-N semantics, and rebind agent config to the live active Pages deployment so main-config rollback never depends on pruned packages.
This commit is contained in:
@@ -21,6 +21,19 @@ sidebar: false
|
|||||||
|
|
||||||
## [unreleased]
|
## [unreleased]
|
||||||
|
|
||||||
|
### 新增
|
||||||
|
|
||||||
|
- Pages 现支持上传 zip、tar.gz、tar.xz、tar.bz2、tar、7z 等常用压缩格式的部署包。
|
||||||
|
- 管理员可在运维设置中配置 Pages 部署包大小上限与每个项目的历史部署保留数量。
|
||||||
|
|
||||||
|
### 变更
|
||||||
|
|
||||||
|
- Pages 部署包上传体积限制改为系统动态配置,默认仍为 100 MiB,可按环境调整。
|
||||||
|
- 上传新部署后会按保留策略自动清理超出数量的历史部署包,减少磁盘占用。
|
||||||
|
- Agent 在同步 Pages 部署时信任控制面已完成的包校验,不再重复限制文件数与展开体积,仅校验下载完整性并安全解压到本地。
|
||||||
|
- 明确 Pages 历史部署保留语义为每个项目最多 N 条(激活必留、其余按从新到旧填充),裁剪失败会记录日志且不回滚已成功上传。
|
||||||
|
- 主配置版本与 Pages 部署双轨管理:下发给 Agent 的配置会将 Pages 路由重绑定到项目当前激活部署,主配置回滚不再依赖已裁剪的旧部署包。
|
||||||
|
|
||||||
## [v3.3.0] - 2026-07-14
|
## [v3.3.0] - 2026-07-14
|
||||||
|
|
||||||
### 新增
|
### 新增
|
||||||
|
|||||||
@@ -98,7 +98,7 @@ Agent 对数据面 OpenResty 的管控实现了端到端的闭环,包含配置
|
|||||||
* `certs/`:证书存放目录(文件命名为 `{cert_id}.crt` 和 `{cert_id}.key`)。
|
* `certs/`:证书存放目录(文件命名为 `{cert_id}.crt` 和 `{cert_id}.key`)。
|
||||||
* `waf/` 与 `pow/`:WAF 及防 CC 挑战所需的专用 Lua 运行时脚本。
|
* `waf/` 与 `pow/`:WAF 及防 CC 挑战所需的专用 Lua 运行时脚本。
|
||||||
* `waf_config.json` 与 `waf_ip_groups.json`:WAF 过滤引擎所需的结构化规则配置文件。
|
* `waf_config.json` 与 `waf_ip_groups.json`:WAF 过滤引擎所需的结构化规则配置文件。
|
||||||
* `pages_dir`:Pages 静态站点部署目录,默认位于 `data_dir/var/lib/openflare/pages`。当激活配置引用 Pages 部署时,Agent 会下载部署 zip、校验 checksum、解压到部署 release 目录,并切换 `deployments/{deployment_id}/current` 供 OpenResty `root`/`try_files` 读取。
|
* `pages_dir`:Pages 静态站点部署目录,默认位于 `data_dir/var/lib/openflare/pages`。当激活配置引用 Pages 部署时,Agent 会下载部署包(zip / tar.gz / tar.xz / tar.bz2 / tar / 7z)、校验下载 checksum、解压到部署 release 目录,并切换 `deployments/{deployment_id}/current` 供 OpenResty `root`/`try_files` 读取。业务侧体积/文件数校验由控制面完成,Agent 信任控制面结果,解压时仅保留本机路径安全防护。
|
||||||
|
|
||||||
### 2. 精细化的重载动作
|
### 2. 精细化的重载动作
|
||||||
1. **备份当前配置**:在写入新文件之前,Agent 会将现有的配置文件复制到 `.backup` 临时目录下,保留完整的现场快照。
|
1. **备份当前配置**:在写入新文件之前,Agent 会将现有的配置文件复制到 `.backup` 临时目录下,保留完整的现场快照。
|
||||||
|
|||||||
@@ -26,7 +26,7 @@ OpenFlare 适合需要统一管理多台 OpenResty 代理节点的团队,具
|
|||||||
| **配置版本控制** | 支持全局单一激活版本的预览、发布、不可变快照历史与秒级一键回滚 | [Agent 与发布模型](./agent-design.md) |
|
| **配置版本控制** | 支持全局单一激活版本的预览、发布、不可变快照历史与秒级一键回滚 | [Agent 与发布模型](./agent-design.md) |
|
||||||
| **WAF 安全防护** | 支持可视化 DAG 编排规则、手动/自动/订阅型 IP 组、GeoIP 匹配与 PoW CC 防护 | [WAF 设计](./waf-design.md) / [WAF 可编排规则设计](./waf-orchestration-design.md) / [WAF 使用指南](../guide/waf-usage.md) |
|
| **WAF 安全防护** | 支持可视化 DAG 编排规则、手动/自动/订阅型 IP 组、GeoIP 匹配与 PoW CC 防护 | [WAF 设计](./waf-design.md) / [WAF 可编排规则设计](./waf-orchestration-design.md) / [WAF 使用指南](../guide/waf-usage.md) |
|
||||||
| **内网穿透** | 通过中继节点(Relay)与内网客户端(OpenFlared),反向穿透暴露内网 Web 服务 | [内网穿透设计](./tunnel-design.md) / [穿透使用指南](../guide/tunnel-usage.md) |
|
| **内网穿透** | 通过中继节点(Relay)与内网客户端(OpenFlared),反向穿透暴露内网 Web 服务 | [内网穿透设计](./tunnel-design.md) / [穿透使用指南](../guide/tunnel-usage.md) |
|
||||||
| **Pages 静态托管** | 直接上传前端 zip 包,由边缘节点拉取并由 OpenResty 本地服务,支持 API 反代与 SPA Fallback | [Pages 静态托管设计](./pages-design.md) |
|
| **Pages 静态托管** | 直接上传前端压缩包(zip / tar.gz / tar.xz / 7z 等),由边缘节点拉取并由 OpenResty 本地服务,支持 API 反代与 SPA Fallback | [Pages 静态托管设计](./pages-design.md) |
|
||||||
| **TLS 证书自动续期** | 将证书显式绑定到 Zone 域名,并通过 ACME 协议向 Let's Encrypt 申请/续期证书 | [Zone 与域名资源设计](./zone-design.md) |
|
| **TLS 证书自动续期** | 将证书显式绑定到 Zone 域名,并通过 ACME 协议向 Let's Encrypt 申请/续期证书 | [Zone 与域名资源设计](./zone-design.md) |
|
||||||
| **多节点监控与观测** | 收集节点资源快照、健康事件,聚合请求指标与访问日志明细 | [系统架构](./architecture.md) |
|
| **多节点监控与观测** | 收集节点资源快照、健康事件,聚合请求指标与访问日志明细 | [系统架构](./architecture.md) |
|
||||||
|
|
||||||
|
|||||||
+22
-13
@@ -19,11 +19,12 @@
|
|||||||
## 核心功能
|
## 核心功能
|
||||||
|
|
||||||
Pages 静态托管子系统包含以下核心能力:
|
Pages 静态托管子系统包含以下核心能力:
|
||||||
* **Direct Upload 部署模式**:支持直接上传预构建的 `.zip` 静态资源包,省去复杂的 Git 集成和构建环境依赖。
|
* **Direct Upload 部署模式**:支持直接上传预构建的静态资源压缩包(`zip`、`tar.gz`、`tar.xz`、`tar.bz2`、`tar`、`7z`),省去复杂的 Git 集成和构建环境依赖。
|
||||||
* **不可变部署快照**:每次上传产生一个带唯一 ID 和 SHA-256 Checksum 的不可变部署记录。历史包永久保留,支持随时激活和回滚。
|
* **不可变部署快照**:每次上传产生一个带唯一 ID 和 SHA-256 Checksum 的不可变部署记录。支持按系统配置保留最近 N 个历史部署,并可随时激活和回滚。
|
||||||
* **SPA Fallback 支持**:支持对单页应用(SPA)进行 Fallback 路由配置,请求找不到静态文件时自动重定向到入口文件。
|
* **SPA Fallback 支持**:支持对单页应用(SPA)进行 Fallback 路由配置,请求找不到静态文件时自动重定向到入口文件。
|
||||||
* **内置 API 反代服务**:支持在 Pages 规则内一键启用 API 代理,消除跨域问题,将请求转发给指定的后端服务。
|
* **内置 API 反代服务**:支持在 Pages 规则内一键启用 API 代理,消除跨域问题,将请求转发给指定的后端服务。
|
||||||
* **安全包校验与解压缩**:内置 Zip-Slip 路径逃逸防御、防软链接劫持、文件大小/数量硬上限控制,保障节点物理安全。
|
* **安全包校验与解压缩**:内置路径逃逸防御、防软链接劫持、文件大小/数量上限与可配置上传包体积控制,保障节点物理安全。
|
||||||
|
* **可配置限额**:管理员可在运维设置中调整「部署包大小上限」与「历史部署保留数」。
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
@@ -40,7 +41,7 @@ graph TD
|
|||||||
|
|
||||||
%% 控制流与心跳
|
%% 控制流与心跳
|
||||||
Server[OpenFlare Server 控制面] <-->|Agent API / Heartbeat| Agent[openflare-agent 进程]
|
Server[OpenFlare Server 控制面] <-->|Agent API / Heartbeat| Agent[openflare-agent 进程]
|
||||||
Server -.->|5. 存储 ZIP 部署包| LocalStore[(Server 本地存储)]
|
Server -.->|5. 存储部署包| LocalStore[(Server 本地存储)]
|
||||||
|
|
||||||
Agent -->|1. 发现新版本| Server
|
Agent -->|1. 发现新版本| Server
|
||||||
Agent -->|2. 下载部署包| Server
|
Agent -->|2. 下载部署包| Server
|
||||||
@@ -88,21 +89,29 @@ graph TD
|
|||||||
}
|
}
|
||||||
```
|
```
|
||||||
|
|
||||||
|
### 3. 与主配置版本的双轨关系
|
||||||
|
* **主配置版本**(`config_versions`)与 **Pages 部署** 是两套独立的版本体系。
|
||||||
|
* 快照里记录的 `pages_deployment` 仅反映**发布当时**的激活部署,用于审计与当时渲染结果存档。
|
||||||
|
* **运行时**:下发给 Agent 的配置会把各 Pages 路由**重绑定到该项目当前激活部署**(最新包)。主配置回滚/重新激活旧版本时,**不要求**仍能拉到旧 Pages 包,只保证指向当前最新激活部署。
|
||||||
|
* 因此 Pages 历史裁剪可以安全删除非激活部署,无需为「主配置回滚到旧 Pages 包」预留存储。
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## Server 端 (控制面) 职责与生命周期
|
## Server 端 (控制面) 职责与生命周期
|
||||||
|
|
||||||
### 1. ZIP 包安全校验与分析
|
### 1. 部署包安全校验与分析
|
||||||
为了避免不可信的用户上传恶意压缩包攻击服务器,控制面在 `UploadPagesDeployment` 时执行严格的流式校验:
|
为了避免不可信的用户上传恶意压缩包攻击服务器,控制面在 `UploadDeployment` 时执行严格校验:
|
||||||
* **大小限制**:ZIP 压缩包不得超过 25 MiB(保守的 V1 默认值),且展开后的解压总体积不得超过 100 MiB。
|
* **格式支持**:`zip`、`tar.gz` / `tgz`、`tar.xz` / `txz`、`tar.bz2` / `tbz2`、`tar`、`7z`。
|
||||||
|
* **大小限制**:压缩包体积由系统配置 `pages_max_package_size_mb` 控制(默认 100 MiB,范围 1~2048);展开后总体积上限为「包大小 × 4」且不低于 100 MiB。
|
||||||
* **数量限制**:压缩包中包含的静态文件总数不得超过 1,000 个。
|
* **数量限制**:压缩包中包含的静态文件总数不得超过 1,000 个。
|
||||||
* **软链接阻断**:遍历 ZIP 文件,一旦检测到任何软链接 (`os.ModeSymlink`),立即抛出错误并拒绝上传,防御软链接劫持攻击。
|
* **软链接阻断**:遍历归档文件,一旦检测到任何软链接,立即抛出错误并拒绝上传,防御软链接劫持攻击。
|
||||||
* **Zip-Slip 防御**:对每个压缩文件路径进行 `Clean` 并检查是否包含 `..` 或以 `/` 开头,防御目录跨越漏洞,防止写入系统敏感路径。
|
* **路径逃逸防御**:对每个压缩文件路径进行 `Clean` 并检查是否包含 `..` 或以 `/` 开头,防御目录跨越漏洞,防止写入系统敏感路径。
|
||||||
* **入口文件校验**:项目指定的入口文件(例如 `index.html`,可在 `project.RootDir` 下)必须在 ZIP 压缩包中存在,否则拒绝上传。
|
* **入口文件校验**:项目指定的入口文件(例如 `index.html`,可在 `project.RootDir` 下)必须在部署包中存在,否则拒绝上传。
|
||||||
* **公共根目录去噪**:许多打包工具(如 GitHub 导出的 zip)会包含一个多余的主文件夹作为公共根前缀。控制面自动探测公共根前缀并将其安全剥离。
|
* **公共根目录去噪**:许多打包工具会包含一个多余的主文件夹作为公共根前缀。控制面自动探测公共根前缀并将其安全剥离。
|
||||||
|
* **历史保留**:系统配置 `pages_max_history_count`(默认 20,0 表示不限制)在每次上传成功后执行裁剪。语义为:**每个项目最多保留 N 条部署**;当前激活部署始终保留;其余名额按部署 ID 从新到旧填充;超出的非激活部署连同文件清单与存储对象一并删除。上传已成功时裁剪失败只记日志、不回滚上传;并发上传下可能短暂超过 N,后续上传的裁剪会收敛回 N。主配置版本回滚不依赖旧 Pages 包(见上节双轨关系)。
|
||||||
|
|
||||||
### 2. 部署包存储规划
|
### 2. 部署包存储规划
|
||||||
控制面仅将 zip 文件存储在本地存储目录 `artifacts/{project_slug}/{checksum}.zip`,并在数据库中记录路径和清单。**大体积静态包不写入 config_versions 记录和任何配置推送通道**,以保障控制面数据同步的轻量与高效。
|
控制面通过统一上传框架(`upload.Ingest`)存储原始部署包,并在数据库中记录 `upload_id` 与文件清单。**大体积静态包不写入 config_versions 记录和任何配置推送通道**,以保障控制面数据同步的轻量与高效。
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
@@ -118,7 +127,7 @@ Agent 运行在各边缘代理节点上,在应用配置版本前,必须先
|
|||||||
### 2. 安全解压缩与原子切换
|
### 2. 安全解压缩与原子切换
|
||||||
为了保证配置应用过程的“无缝”且能在出错时立即回滚:
|
为了保证配置应用过程的“无缝”且能在出错时立即回滚:
|
||||||
1. Agent 将下载的部署包数据写入临时目录,并重新计算 SHA-256 Checksum。如果与配置指明的 checksum 不符,立即报错并阻断发布流程。
|
1. Agent 将下载的部署包数据写入临时目录,并重新计算 SHA-256 Checksum。如果与配置指明的 checksum 不符,立即报错并阻断发布流程。
|
||||||
2. 解压部署包至临时目录 `releases/{checksum}.tmp`。解压时同样执行 Zip-Slip 目录跨越和软链接校验防御。
|
2. 解压部署包至临时目录 `releases/{checksum}.tmp`。解压支持 zip / tar.* / 7z。**Agent 默认信任控制面**:不再重复校验文件数/体积等业务限额(控制面上传时已完成);仅做本机落盘安全处理(路径逃逸、软链接拒绝)与下载 checksum 完整性校验。
|
||||||
3. 解压成功后,写入标记文件 `.openflare-pages.json`。
|
3. 解压成功后,写入标记文件 `.openflare-pages.json`。
|
||||||
4. 清理 `releases/{checksum}` 目录,将整个临时目录重命名为 `releases/{checksum}`。
|
4. 清理 `releases/{checksum}` 目录,将整个临时目录重命名为 `releases/{checksum}`。
|
||||||
5. **原子切换**:建立拷贝当前部署的物理副本到目标位置 `deployments/{deployment_id}/current`。切换前先备份上一版本的 `current`,一旦重载配置失败,Agent 能够快速恢复 `current` 目录并回滚 OpenResty。
|
5. **原子切换**:建立拷贝当前部署的物理副本到目标位置 `deployments/{deployment_id}/current`。切换前先备份上一版本的 `current`,一旦重载配置失败,Agent 能够快速恢复 `current` 目录并回滚 OpenResty。
|
||||||
|
|||||||
@@ -20,6 +20,8 @@ export type OpenFlareOpsFields = {
|
|||||||
uptime_kuma_timeout: string;
|
uptime_kuma_timeout: string;
|
||||||
database_auto_cleanup_enabled: boolean;
|
database_auto_cleanup_enabled: boolean;
|
||||||
database_auto_cleanup_retention_days: string;
|
database_auto_cleanup_retention_days: string;
|
||||||
|
pages_max_package_size_mb: string;
|
||||||
|
pages_max_history_count: string;
|
||||||
};
|
};
|
||||||
|
|
||||||
export const defaultOpenFlareOpsFields: OpenFlareOpsFields = {
|
export const defaultOpenFlareOpsFields: OpenFlareOpsFields = {
|
||||||
@@ -42,6 +44,8 @@ export const defaultOpenFlareOpsFields: OpenFlareOpsFields = {
|
|||||||
uptime_kuma_timeout: '48',
|
uptime_kuma_timeout: '48',
|
||||||
database_auto_cleanup_enabled: false,
|
database_auto_cleanup_enabled: false,
|
||||||
database_auto_cleanup_retention_days: '30',
|
database_auto_cleanup_retention_days: '30',
|
||||||
|
pages_max_package_size_mb: '100',
|
||||||
|
pages_max_history_count: '20',
|
||||||
};
|
};
|
||||||
|
|
||||||
export const INSTALLER_SCRIPT_URL =
|
export const INSTALLER_SCRIPT_URL =
|
||||||
@@ -90,6 +94,8 @@ export function mapOptionsToOpsFields(
|
|||||||
),
|
),
|
||||||
database_auto_cleanup_retention_days:
|
database_auto_cleanup_retention_days:
|
||||||
optionMap.database_auto_cleanup_retention_days ?? '30',
|
optionMap.database_auto_cleanup_retention_days ?? '30',
|
||||||
|
pages_max_package_size_mb: optionMap.pages_max_package_size_mb ?? '100',
|
||||||
|
pages_max_history_count: optionMap.pages_max_history_count ?? '20',
|
||||||
};
|
};
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -226,3 +232,30 @@ export function databaseAutoCleanupEntries(
|
|||||||
},
|
},
|
||||||
];
|
];
|
||||||
}
|
}
|
||||||
|
|
||||||
|
export function validatePagesFields(fields: OpenFlareOpsFields) {
|
||||||
|
const packageSize = Number.parseInt(fields.pages_max_package_size_mb, 10);
|
||||||
|
const historyCount = Number.parseInt(fields.pages_max_history_count, 10);
|
||||||
|
if (Number.isNaN(packageSize) || packageSize < 1 || packageSize > 2048) {
|
||||||
|
throw new Error('Pages 部署包大小上限必须为 1~2048 MiB。');
|
||||||
|
}
|
||||||
|
if (Number.isNaN(historyCount) || historyCount < 0) {
|
||||||
|
throw new Error(
|
||||||
|
'Pages 历史保留数必须为大于等于 0 的整数(0 表示不限制)。',
|
||||||
|
);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
export function pagesOptionEntries(fields: OpenFlareOpsFields): OptionItem[] {
|
||||||
|
validatePagesFields(fields);
|
||||||
|
return [
|
||||||
|
{
|
||||||
|
key: 'pages_max_package_size_mb',
|
||||||
|
value: String(Number.parseInt(fields.pages_max_package_size_mb, 10)),
|
||||||
|
},
|
||||||
|
{
|
||||||
|
key: 'pages_max_history_count',
|
||||||
|
value: String(Number.parseInt(fields.pages_max_history_count, 10)),
|
||||||
|
},
|
||||||
|
];
|
||||||
|
}
|
||||||
|
|||||||
@@ -64,6 +64,7 @@ import {
|
|||||||
mapOptionsToOpsFields,
|
mapOptionsToOpsFields,
|
||||||
type OpenFlareOpsFields,
|
type OpenFlareOpsFields,
|
||||||
optionsToMap,
|
optionsToMap,
|
||||||
|
pagesOptionEntries,
|
||||||
uptimeKumaOptionEntries,
|
uptimeKumaOptionEntries,
|
||||||
} from './openflare-ops-utils';
|
} from './openflare-ops-utils';
|
||||||
import { UptimeKumaSiteSelectModal } from './uptimekuma-site-modal';
|
import { UptimeKumaSiteSelectModal } from './uptimekuma-site-modal';
|
||||||
@@ -258,6 +259,17 @@ export function OpenFlareOpsSettings() {
|
|||||||
}
|
}
|
||||||
};
|
};
|
||||||
|
|
||||||
|
const savePagesSettings = () => {
|
||||||
|
try {
|
||||||
|
saveMutation.mutate({
|
||||||
|
section: 'pages',
|
||||||
|
entries: pagesOptionEntries(fields),
|
||||||
|
});
|
||||||
|
} catch (error) {
|
||||||
|
toast.error(error instanceof Error ? error.message : '参数校验失败');
|
||||||
|
}
|
||||||
|
};
|
||||||
|
|
||||||
if (optionsQuery.isLoading) {
|
if (optionsQuery.isLoading) {
|
||||||
return (
|
return (
|
||||||
<LoadingStateWithBorder
|
<LoadingStateWithBorder
|
||||||
@@ -413,6 +425,55 @@ export function OpenFlareOpsSettings() {
|
|||||||
</Card>
|
</Card>
|
||||||
</div>
|
</div>
|
||||||
|
|
||||||
|
<Card className='border-dashed shadow-none'>
|
||||||
|
<CardHeader className='flex flex-row items-center justify-between gap-4'>
|
||||||
|
<div>
|
||||||
|
<CardTitle className='text-base'>Pages 静态托管</CardTitle>
|
||||||
|
<CardDescription>
|
||||||
|
配置部署包上传体积上限与每个项目的历史部署保留数量。
|
||||||
|
</CardDescription>
|
||||||
|
</div>
|
||||||
|
<Button
|
||||||
|
size='sm'
|
||||||
|
disabled={savingSection === 'pages'}
|
||||||
|
onClick={savePagesSettings}
|
||||||
|
>
|
||||||
|
{savingSection === 'pages' ? (
|
||||||
|
<Loader2 className='size-4 animate-spin mr-1' />
|
||||||
|
) : (
|
||||||
|
<Save className='size-3.5 mr-1' />
|
||||||
|
)}
|
||||||
|
保存
|
||||||
|
</Button>
|
||||||
|
</CardHeader>
|
||||||
|
<CardContent className='space-y-4'>
|
||||||
|
<div className='grid gap-4 md:grid-cols-2'>
|
||||||
|
<FieldInput
|
||||||
|
label='部署包大小上限 (MiB)'
|
||||||
|
value={fields.pages_max_package_size_mb}
|
||||||
|
type='number'
|
||||||
|
onChange={(value) =>
|
||||||
|
updateField('pages_max_package_size_mb', value)
|
||||||
|
}
|
||||||
|
placeholder='100'
|
||||||
|
/>
|
||||||
|
<FieldInput
|
||||||
|
label='历史部署保留数(0 不限制)'
|
||||||
|
value={fields.pages_max_history_count}
|
||||||
|
type='number'
|
||||||
|
onChange={(value) =>
|
||||||
|
updateField('pages_max_history_count', value)
|
||||||
|
}
|
||||||
|
placeholder='20'
|
||||||
|
/>
|
||||||
|
</div>
|
||||||
|
<p className='text-xs text-muted-foreground'>
|
||||||
|
每个项目最多保留 N 条部署:激活部署始终保留,其余按从新到旧填充;超出的非激活部署会在上传成功后自动清理。支持
|
||||||
|
zip、tar.gz、tar.xz、tar.bz2、tar、7z 格式。
|
||||||
|
</p>
|
||||||
|
</CardContent>
|
||||||
|
</Card>
|
||||||
|
|
||||||
<Card className='border-dashed shadow-none'>
|
<Card className='border-dashed shadow-none'>
|
||||||
<CardHeader className='flex flex-row items-center justify-between gap-4'>
|
<CardHeader className='flex flex-row items-center justify-between gap-4'>
|
||||||
<div>
|
<div>
|
||||||
|
|||||||
@@ -27,6 +27,27 @@ import {
|
|||||||
projectsQueryKey,
|
projectsQueryKey,
|
||||||
} from './pages-utils';
|
} from './pages-utils';
|
||||||
|
|
||||||
|
const PAGES_PACKAGE_ACCEPT =
|
||||||
|
'.zip,.tar.gz,.tgz,.tar.xz,.txz,.tar.bz2,.tbz2,.tbz,.tar,.7z';
|
||||||
|
|
||||||
|
const PAGES_PACKAGE_EXTENSIONS = [
|
||||||
|
'.zip',
|
||||||
|
'.tar.gz',
|
||||||
|
'.tgz',
|
||||||
|
'.tar.xz',
|
||||||
|
'.txz',
|
||||||
|
'.tar.bz2',
|
||||||
|
'.tbz2',
|
||||||
|
'.tbz',
|
||||||
|
'.tar',
|
||||||
|
'.7z',
|
||||||
|
] as const;
|
||||||
|
|
||||||
|
function isSupportedPagesPackage(fileName: string) {
|
||||||
|
const lower = fileName.toLowerCase();
|
||||||
|
return PAGES_PACKAGE_EXTENSIONS.some((ext) => lower.endsWith(ext));
|
||||||
|
}
|
||||||
|
|
||||||
interface DeploymentUploadDialogProps {
|
interface DeploymentUploadDialogProps {
|
||||||
open: boolean;
|
open: boolean;
|
||||||
onOpenChange: (open: boolean) => void;
|
onOpenChange: (open: boolean) => void;
|
||||||
@@ -58,7 +79,7 @@ export function DeploymentUploadDialog({
|
|||||||
|
|
||||||
const uploadMutation = useMutation({
|
const uploadMutation = useMutation({
|
||||||
mutationFn: () => {
|
mutationFn: () => {
|
||||||
if (!file) throw new Error('请选择 zip 部署包');
|
if (!file) throw new Error('请选择部署包');
|
||||||
return PagesService.uploadDeployment(projectId, {
|
return PagesService.uploadDeployment(projectId, {
|
||||||
file,
|
file,
|
||||||
onProgress: setUploadProgress,
|
onProgress: setUploadProgress,
|
||||||
@@ -83,8 +104,8 @@ export function DeploymentUploadDialog({
|
|||||||
|
|
||||||
const handleFileSelect = (selected: File | null) => {
|
const handleFileSelect = (selected: File | null) => {
|
||||||
if (!selected) return;
|
if (!selected) return;
|
||||||
if (!selected.name.toLowerCase().endsWith('.zip')) {
|
if (!isSupportedPagesPackage(selected.name)) {
|
||||||
toast.error('仅支持 zip 格式的文件');
|
toast.error('仅支持 zip、tar.gz、tar.xz、tar.bz2、tar、7z 格式的部署包');
|
||||||
return;
|
return;
|
||||||
}
|
}
|
||||||
setFile(selected);
|
setFile(selected);
|
||||||
@@ -96,7 +117,8 @@ export function DeploymentUploadDialog({
|
|||||||
<DialogHeader>
|
<DialogHeader>
|
||||||
<DialogTitle>上传部署包</DialogTitle>
|
<DialogTitle>上传部署包</DialogTitle>
|
||||||
<DialogDescription>
|
<DialogDescription>
|
||||||
上传已构建的 zip 静态资源包,部署后可在列表中激活。
|
上传已构建的静态资源压缩包(zip / tar.gz / tar.xz / tar.bz2 / tar /
|
||||||
|
7z),部署后可在列表中激活。
|
||||||
</DialogDescription>
|
</DialogDescription>
|
||||||
</DialogHeader>
|
</DialogHeader>
|
||||||
|
|
||||||
@@ -121,7 +143,10 @@ export function DeploymentUploadDialog({
|
|||||||
}}
|
}}
|
||||||
>
|
>
|
||||||
<UploadCloud className='size-8 mx-auto text-muted-foreground' />
|
<UploadCloud className='size-8 mx-auto text-muted-foreground' />
|
||||||
<p className='mt-3 text-sm'>拖拽 zip 文件到此处,或点击选择文件</p>
|
<p className='mt-3 text-sm'>拖拽部署包到此处,或点击选择文件</p>
|
||||||
|
<p className='mt-1 text-xs text-muted-foreground'>
|
||||||
|
支持 zip、tar.gz、tar.xz、tar.bz2、tar、7z
|
||||||
|
</p>
|
||||||
<Button
|
<Button
|
||||||
type='button'
|
type='button'
|
||||||
variant='outline'
|
variant='outline'
|
||||||
@@ -134,7 +159,7 @@ export function DeploymentUploadDialog({
|
|||||||
<input
|
<input
|
||||||
ref={fileInputRef}
|
ref={fileInputRef}
|
||||||
type='file'
|
type='file'
|
||||||
accept='.zip'
|
accept={PAGES_PACKAGE_ACCEPT}
|
||||||
className='hidden'
|
className='hidden'
|
||||||
onChange={(e) => handleFileSelect(e.target.files?.[0] ?? null)}
|
onChange={(e) => handleFileSelect(e.target.files?.[0] ?? null)}
|
||||||
/>
|
/>
|
||||||
|
|||||||
@@ -30,7 +30,8 @@ export default function PagesPage() {
|
|||||||
<div>
|
<div>
|
||||||
<h1 className='text-2xl font-semibold tracking-tight'>Pages</h1>
|
<h1 className='text-2xl font-semibold tracking-tight'>Pages</h1>
|
||||||
<p className='text-sm text-muted-foreground'>
|
<p className='text-sm text-muted-foreground'>
|
||||||
边缘静态站点托管,上传 zip 部署包并在代理规则中选择 Pages 上游。
|
边缘静态站点托管,上传静态资源部署包并在代理规则中选择 Pages
|
||||||
|
上游。
|
||||||
</p>
|
</p>
|
||||||
</div>
|
</div>
|
||||||
</div>
|
</div>
|
||||||
|
|||||||
@@ -10,6 +10,7 @@ require (
|
|||||||
github.com/aws/aws-sdk-go-v2/config v1.32.16
|
github.com/aws/aws-sdk-go-v2/config v1.32.16
|
||||||
github.com/aws/aws-sdk-go-v2/credentials v1.19.15
|
github.com/aws/aws-sdk-go-v2/credentials v1.19.15
|
||||||
github.com/aws/aws-sdk-go-v2/service/s3 v1.99.1
|
github.com/aws/aws-sdk-go-v2/service/s3 v1.99.1
|
||||||
|
github.com/bodgit/sevenzip v1.6.5
|
||||||
github.com/bwmarrin/snowflake v0.3.0
|
github.com/bwmarrin/snowflake v0.3.0
|
||||||
github.com/coreos/go-oidc/v3 v3.17.0
|
github.com/coreos/go-oidc/v3 v3.17.0
|
||||||
github.com/deepteams/webp v1.2.3
|
github.com/deepteams/webp v1.2.3
|
||||||
@@ -39,7 +40,9 @@ require (
|
|||||||
github.com/swaggo/files v1.0.1
|
github.com/swaggo/files v1.0.1
|
||||||
github.com/swaggo/gin-swagger v1.6.1
|
github.com/swaggo/gin-swagger v1.6.1
|
||||||
github.com/swaggo/swag v1.16.6
|
github.com/swaggo/swag v1.16.6
|
||||||
|
github.com/ulikunitz/xz v0.5.15
|
||||||
github.com/uptrace/opentelemetry-go-extra/otelzap v0.3.2
|
github.com/uptrace/opentelemetry-go-extra/otelzap v0.3.2
|
||||||
|
github.com/yuin/gopher-lua v1.1.1
|
||||||
go.opentelemetry.io/contrib/instrumentation/github.com/gin-gonic/gin/otelgin v0.61.0
|
go.opentelemetry.io/contrib/instrumentation/github.com/gin-gonic/gin/otelgin v0.61.0
|
||||||
go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp v0.68.0
|
go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp v0.68.0
|
||||||
go.opentelemetry.io/otel v1.43.0
|
go.opentelemetry.io/otel v1.43.0
|
||||||
@@ -47,14 +50,13 @@ require (
|
|||||||
go.opentelemetry.io/otel/sdk v1.43.0
|
go.opentelemetry.io/otel/sdk v1.43.0
|
||||||
go.opentelemetry.io/otel/trace v1.43.0
|
go.opentelemetry.io/otel/trace v1.43.0
|
||||||
go.uber.org/zap v1.27.1
|
go.uber.org/zap v1.27.1
|
||||||
golang.org/x/crypto v0.51.0
|
golang.org/x/crypto v0.53.0
|
||||||
golang.org/x/image v0.42.0
|
golang.org/x/image v0.42.0
|
||||||
golang.org/x/mod v0.36.0
|
golang.org/x/mod v0.37.0
|
||||||
golang.org/x/net v0.54.0
|
golang.org/x/net v0.56.0
|
||||||
golang.org/x/oauth2 v0.36.0
|
golang.org/x/oauth2 v0.36.0
|
||||||
golang.org/x/sync v0.21.0
|
golang.org/x/sync v0.22.0
|
||||||
gopkg.in/natefinch/lumberjack.v2 v2.2.1
|
gopkg.in/natefinch/lumberjack.v2 v2.2.1
|
||||||
gorm.io/driver/clickhouse v0.7.0
|
|
||||||
gorm.io/driver/postgres v1.6.0
|
gorm.io/driver/postgres v1.6.0
|
||||||
gorm.io/driver/sqlite v1.6.0
|
gorm.io/driver/sqlite v1.6.0
|
||||||
gorm.io/gorm v1.31.1
|
gorm.io/gorm v1.31.1
|
||||||
@@ -66,7 +68,7 @@ require (
|
|||||||
filippo.io/edwards25519 v1.2.0 // indirect
|
filippo.io/edwards25519 v1.2.0 // indirect
|
||||||
github.com/ClickHouse/ch-go v0.71.0 // indirect
|
github.com/ClickHouse/ch-go v0.71.0 // indirect
|
||||||
github.com/KyleBanks/depth v1.2.1 // indirect
|
github.com/KyleBanks/depth v1.2.1 // indirect
|
||||||
github.com/andybalholm/brotli v1.2.1 // indirect
|
github.com/andybalholm/brotli v1.2.2 // indirect
|
||||||
github.com/aws/aws-sdk-go-v2/aws/protocol/eventstream v1.7.9 // indirect
|
github.com/aws/aws-sdk-go-v2/aws/protocol/eventstream v1.7.9 // indirect
|
||||||
github.com/aws/aws-sdk-go-v2/feature/ec2/imds v1.18.22 // indirect
|
github.com/aws/aws-sdk-go-v2/feature/ec2/imds v1.18.22 // indirect
|
||||||
github.com/aws/aws-sdk-go-v2/internal/configsources v1.4.22 // indirect
|
github.com/aws/aws-sdk-go-v2/internal/configsources v1.4.22 // indirect
|
||||||
@@ -81,6 +83,8 @@ require (
|
|||||||
github.com/aws/aws-sdk-go-v2/service/ssooidc v1.35.20 // indirect
|
github.com/aws/aws-sdk-go-v2/service/ssooidc v1.35.20 // indirect
|
||||||
github.com/aws/aws-sdk-go-v2/service/sts v1.42.0 // indirect
|
github.com/aws/aws-sdk-go-v2/service/sts v1.42.0 // indirect
|
||||||
github.com/aws/smithy-go v1.25.0 // indirect
|
github.com/aws/smithy-go v1.25.0 // indirect
|
||||||
|
github.com/bodgit/plumbing v1.3.0 // indirect
|
||||||
|
github.com/bodgit/windows v1.0.1 // indirect
|
||||||
github.com/boj/redistore v1.4.1 // indirect
|
github.com/boj/redistore v1.4.1 // indirect
|
||||||
github.com/bytedance/gopkg v0.1.3 // indirect
|
github.com/bytedance/gopkg v0.1.3 // indirect
|
||||||
github.com/bytedance/sonic v1.14.2 // indirect
|
github.com/bytedance/sonic v1.14.2 // indirect
|
||||||
@@ -124,6 +128,7 @@ require (
|
|||||||
github.com/gorilla/sessions v1.4.0 // indirect
|
github.com/gorilla/sessions v1.4.0 // indirect
|
||||||
github.com/grpc-ecosystem/grpc-gateway/v2 v2.28.0 // indirect
|
github.com/grpc-ecosystem/grpc-gateway/v2 v2.28.0 // indirect
|
||||||
github.com/hashicorp/go-version v1.9.0 // indirect
|
github.com/hashicorp/go-version v1.9.0 // indirect
|
||||||
|
github.com/hashicorp/golang-lru/v2 v2.0.7 // indirect
|
||||||
github.com/inconshreveable/mousetrap v1.1.0 // indirect
|
github.com/inconshreveable/mousetrap v1.1.0 // indirect
|
||||||
github.com/jackc/pgpassfile v1.0.0 // indirect
|
github.com/jackc/pgpassfile v1.0.0 // indirect
|
||||||
github.com/jackc/pgservicefile v0.0.0-20240606120523-5a60cdf6a761 // indirect
|
github.com/jackc/pgservicefile v0.0.0-20240606120523-5a60cdf6a761 // indirect
|
||||||
@@ -132,7 +137,7 @@ require (
|
|||||||
github.com/jinzhu/inflection v1.0.0 // indirect
|
github.com/jinzhu/inflection v1.0.0 // indirect
|
||||||
github.com/jinzhu/now v1.1.5 // indirect
|
github.com/jinzhu/now v1.1.5 // indirect
|
||||||
github.com/json-iterator/go v1.1.13-0.20220915233716-71ac16282d12 // indirect
|
github.com/json-iterator/go v1.1.13-0.20220915233716-71ac16282d12 // indirect
|
||||||
github.com/klauspost/compress v1.18.5 // indirect
|
github.com/klauspost/compress v1.19.0 // indirect
|
||||||
github.com/klauspost/cpuid/v2 v2.3.0 // indirect
|
github.com/klauspost/cpuid/v2 v2.3.0 // indirect
|
||||||
github.com/leodido/go-urn v1.4.0 // indirect
|
github.com/leodido/go-urn v1.4.0 // indirect
|
||||||
github.com/mattn/go-isatty v0.0.21 // indirect
|
github.com/mattn/go-isatty v0.0.21 // indirect
|
||||||
@@ -144,7 +149,7 @@ require (
|
|||||||
github.com/ncruces/go-strftime v1.0.0 // indirect
|
github.com/ncruces/go-strftime v1.0.0 // indirect
|
||||||
github.com/paulmach/orb v0.13.0 // indirect
|
github.com/paulmach/orb v0.13.0 // indirect
|
||||||
github.com/pelletier/go-toml/v2 v2.2.4 // indirect
|
github.com/pelletier/go-toml/v2 v2.2.4 // indirect
|
||||||
github.com/pierrec/lz4/v4 v4.1.26 // indirect
|
github.com/pierrec/lz4/v4 v4.1.27 // indirect
|
||||||
github.com/pmezard/go-difflib v1.0.1-0.20181226105442-5d4384ee4fb2 // indirect
|
github.com/pmezard/go-difflib v1.0.1-0.20181226105442-5d4384ee4fb2 // indirect
|
||||||
github.com/quic-go/qpack v0.5.1 // indirect
|
github.com/quic-go/qpack v0.5.1 // indirect
|
||||||
github.com/quic-go/quic-go v0.55.0 // indirect
|
github.com/quic-go/quic-go v0.55.0 // indirect
|
||||||
@@ -156,11 +161,11 @@ require (
|
|||||||
github.com/spf13/afero v1.15.0 // indirect
|
github.com/spf13/afero v1.15.0 // indirect
|
||||||
github.com/spf13/cast v1.10.0 // indirect
|
github.com/spf13/cast v1.10.0 // indirect
|
||||||
github.com/spf13/pflag v1.0.10 // indirect
|
github.com/spf13/pflag v1.0.10 // indirect
|
||||||
|
github.com/stangelandcl/ppmd v0.1.1 // indirect
|
||||||
github.com/subosito/gotenv v1.6.0 // indirect
|
github.com/subosito/gotenv v1.6.0 // indirect
|
||||||
github.com/twitchyliquid64/golang-asm v0.15.1 // indirect
|
github.com/twitchyliquid64/golang-asm v0.15.1 // indirect
|
||||||
github.com/ugorji/go/codec v1.3.1 // indirect
|
github.com/ugorji/go/codec v1.3.1 // indirect
|
||||||
github.com/uptrace/opentelemetry-go-extra/otelutil v0.3.2 // indirect
|
github.com/uptrace/opentelemetry-go-extra/otelutil v0.3.2 // indirect
|
||||||
github.com/yuin/gopher-lua v1.1.1 // indirect
|
|
||||||
go.opentelemetry.io/auto/sdk v1.2.1 // indirect
|
go.opentelemetry.io/auto/sdk v1.2.1 // indirect
|
||||||
go.opentelemetry.io/otel/exporters/otlp/otlptrace v1.43.0 // indirect
|
go.opentelemetry.io/otel/exporters/otlp/otlptrace v1.43.0 // indirect
|
||||||
go.opentelemetry.io/otel/log v0.12.2 // indirect
|
go.opentelemetry.io/otel/log v0.12.2 // indirect
|
||||||
@@ -169,16 +174,18 @@ require (
|
|||||||
go.uber.org/mock v0.6.0 // indirect
|
go.uber.org/mock v0.6.0 // indirect
|
||||||
go.uber.org/multierr v1.11.0 // indirect
|
go.uber.org/multierr v1.11.0 // indirect
|
||||||
go.yaml.in/yaml/v3 v3.0.4 // indirect
|
go.yaml.in/yaml/v3 v3.0.4 // indirect
|
||||||
|
go4.org v0.0.0-20260112195520-a5071408f32f // indirect
|
||||||
golang.org/x/arch v0.22.0 // indirect
|
golang.org/x/arch v0.22.0 // indirect
|
||||||
golang.org/x/sys v0.44.0 // indirect
|
golang.org/x/sys v0.46.0 // indirect
|
||||||
golang.org/x/text v0.38.0 // indirect
|
golang.org/x/text v0.40.0 // indirect
|
||||||
golang.org/x/time v0.15.0 // indirect
|
golang.org/x/time v0.15.0 // indirect
|
||||||
golang.org/x/tools v0.45.0 // indirect
|
golang.org/x/tools v0.47.0 // indirect
|
||||||
google.golang.org/genproto/googleapis/api v0.0.0-20260401024825-9d38bb4040a9 // indirect
|
google.golang.org/genproto/googleapis/api v0.0.0-20260401024825-9d38bb4040a9 // indirect
|
||||||
google.golang.org/genproto/googleapis/rpc v0.0.0-20260420184626-e10c466a9529 // indirect
|
google.golang.org/genproto/googleapis/rpc v0.0.0-20260420184626-e10c466a9529 // indirect
|
||||||
google.golang.org/grpc v1.80.0 // indirect
|
google.golang.org/grpc v1.80.0 // indirect
|
||||||
google.golang.org/protobuf v1.36.11 // indirect
|
google.golang.org/protobuf v1.36.11 // indirect
|
||||||
gopkg.in/yaml.v3 v3.0.1 // indirect
|
gopkg.in/yaml.v3 v3.0.1 // indirect
|
||||||
|
gorm.io/driver/clickhouse v0.7.0 // indirect
|
||||||
gorm.io/driver/mysql v1.6.0 // indirect
|
gorm.io/driver/mysql v1.6.0 // indirect
|
||||||
modernc.org/libc v1.72.1 // indirect
|
modernc.org/libc v1.72.1 // indirect
|
||||||
modernc.org/mathutil v1.7.1 // indirect
|
modernc.org/mathutil v1.7.1 // indirect
|
||||||
|
|||||||
@@ -10,8 +10,8 @@ github.com/alicebob/miniredis/v2 v2.38.0 h1:nZAzCR+Lj+Vxk4ZXzm2NuKq2O33RXj1XxJ2e
|
|||||||
github.com/alicebob/miniredis/v2 v2.38.0/go.mod h1:TcL7YfarKPGDAthEtl5NBeHZfeUQj6OXMm/+iu5cLMM=
|
github.com/alicebob/miniredis/v2 v2.38.0/go.mod h1:TcL7YfarKPGDAthEtl5NBeHZfeUQj6OXMm/+iu5cLMM=
|
||||||
github.com/aliyun/alibabacloud-oss-go-sdk-v2 v1.5.1 h1:vtiFd0hhPAbyYJjztl0wYUq/PqEGkIlDmVuTIy6zw8Y=
|
github.com/aliyun/alibabacloud-oss-go-sdk-v2 v1.5.1 h1:vtiFd0hhPAbyYJjztl0wYUq/PqEGkIlDmVuTIy6zw8Y=
|
||||||
github.com/aliyun/alibabacloud-oss-go-sdk-v2 v1.5.1/go.mod h1:FTzydeQVmR24FI0D6XWUOMKckjXehM/jgMn1xC+DA9M=
|
github.com/aliyun/alibabacloud-oss-go-sdk-v2 v1.5.1/go.mod h1:FTzydeQVmR24FI0D6XWUOMKckjXehM/jgMn1xC+DA9M=
|
||||||
github.com/andybalholm/brotli v1.2.1 h1:R+f5xP285VArJDRgowrfb9DqL18yVK0gKAW/F+eTWro=
|
github.com/andybalholm/brotli v1.2.2 h1:HzTuoo2ErYQqf5qvcJInB8uvqSVxRttzkFexPWtnceM=
|
||||||
github.com/andybalholm/brotli v1.2.1/go.mod h1:rzTDkvFWvIrjDXZHkuS16NPggd91W3kUSvPlQ1pLaKY=
|
github.com/andybalholm/brotli v1.2.2/go.mod h1:rzTDkvFWvIrjDXZHkuS16NPggd91W3kUSvPlQ1pLaKY=
|
||||||
github.com/aws/aws-sdk-go-v2 v1.41.6 h1:1AX0AthnBQzMx1vbmir3Y4WsnJgiydmnJjiLu+LvXOg=
|
github.com/aws/aws-sdk-go-v2 v1.41.6 h1:1AX0AthnBQzMx1vbmir3Y4WsnJgiydmnJjiLu+LvXOg=
|
||||||
github.com/aws/aws-sdk-go-v2 v1.41.6/go.mod h1:dy0UzBIfwSeot4grGvY1AqFWN5zgziMmWGzysDnHFcQ=
|
github.com/aws/aws-sdk-go-v2 v1.41.6/go.mod h1:dy0UzBIfwSeot4grGvY1AqFWN5zgziMmWGzysDnHFcQ=
|
||||||
github.com/aws/aws-sdk-go-v2/aws/protocol/eventstream v1.7.9 h1:adBsCIIpLbLmYnkQU+nAChU5yhVTvu5PerROm+/Kq2A=
|
github.com/aws/aws-sdk-go-v2/aws/protocol/eventstream v1.7.9 h1:adBsCIIpLbLmYnkQU+nAChU5yhVTvu5PerROm+/Kq2A=
|
||||||
@@ -48,6 +48,12 @@ github.com/aws/aws-sdk-go-v2/service/sts v1.42.0 h1:ks8KBcZPh3PYISr5dAiXCM5/Thcu
|
|||||||
github.com/aws/aws-sdk-go-v2/service/sts v1.42.0/go.mod h1:pFw33T0WLvXU3rw1WBkpMlkgIn54eCB5FYLhjDc9Foo=
|
github.com/aws/aws-sdk-go-v2/service/sts v1.42.0/go.mod h1:pFw33T0WLvXU3rw1WBkpMlkgIn54eCB5FYLhjDc9Foo=
|
||||||
github.com/aws/smithy-go v1.25.0 h1:Sz/XJ64rwuiKtB6j98nDIPyYrV1nVNJ4YU74gttcl5U=
|
github.com/aws/smithy-go v1.25.0 h1:Sz/XJ64rwuiKtB6j98nDIPyYrV1nVNJ4YU74gttcl5U=
|
||||||
github.com/aws/smithy-go v1.25.0/go.mod h1:YE2RhdIuDbA5E5bTdciG9KrW3+TiEONeUWCqxX9i1Fc=
|
github.com/aws/smithy-go v1.25.0/go.mod h1:YE2RhdIuDbA5E5bTdciG9KrW3+TiEONeUWCqxX9i1Fc=
|
||||||
|
github.com/bodgit/plumbing v1.3.0 h1:pf9Itz1JOQgn7vEOE7v7nlEfBykYqvUYioC61TwWCFU=
|
||||||
|
github.com/bodgit/plumbing v1.3.0/go.mod h1:JOTb4XiRu5xfnmdnDJo6GmSbSbtSyufrsyZFByMtKEs=
|
||||||
|
github.com/bodgit/sevenzip v1.6.5 h1:7H7BxgmeX0j6UX42lH+KXQ92WgMQJ49DoocFdfHbCng=
|
||||||
|
github.com/bodgit/sevenzip v1.6.5/go.mod h1:GhuB6Lq1xCpP1sps+horjZ8lgiKPJcy2zUX3prla9wc=
|
||||||
|
github.com/bodgit/windows v1.0.1 h1:tF7K6KOluPYygXa3Z2594zxlkbKPAOvqr97etrGNIz4=
|
||||||
|
github.com/bodgit/windows v1.0.1/go.mod h1:a6JLwrB4KrTR5hBpp8FI9/9W9jJfeQ2h4XDXU74ZCdM=
|
||||||
github.com/boj/redistore v1.4.1 h1:lP9ZZWqKMq2RIqexlZX1w1ODSnegL+puxGIujkU5tIw=
|
github.com/boj/redistore v1.4.1 h1:lP9ZZWqKMq2RIqexlZX1w1ODSnegL+puxGIujkU5tIw=
|
||||||
github.com/boj/redistore v1.4.1/go.mod h1:c0Tvw6aMjslog4jHIAcNv6EtJM849YoOAhMY7JBbWpI=
|
github.com/boj/redistore v1.4.1/go.mod h1:c0Tvw6aMjslog4jHIAcNv6EtJM849YoOAhMY7JBbWpI=
|
||||||
github.com/bsm/ginkgo/v2 v2.12.0 h1:Ny8MWAHyOepLGlLKYmXG4IEkioBysk6GpaRTLC8zwWs=
|
github.com/bsm/ginkgo/v2 v2.12.0 h1:Ny8MWAHyOepLGlLKYmXG4IEkioBysk6GpaRTLC8zwWs=
|
||||||
@@ -206,8 +212,8 @@ github.com/jinzhu/now v1.1.5 h1:/o9tlHleP7gOFmsnYNz3RGnqzefHA47wQpKrrdTIwXQ=
|
|||||||
github.com/jinzhu/now v1.1.5/go.mod h1:d3SSVoowX0Lcu0IBviAWJpolVfI5UJVZZ7cO71lE/z8=
|
github.com/jinzhu/now v1.1.5/go.mod h1:d3SSVoowX0Lcu0IBviAWJpolVfI5UJVZZ7cO71lE/z8=
|
||||||
github.com/json-iterator/go v1.1.13-0.20220915233716-71ac16282d12 h1:9Nu54bhS/H/Kgo2/7xNSUuC5G28VR8ljfrLKU2G4IjU=
|
github.com/json-iterator/go v1.1.13-0.20220915233716-71ac16282d12 h1:9Nu54bhS/H/Kgo2/7xNSUuC5G28VR8ljfrLKU2G4IjU=
|
||||||
github.com/json-iterator/go v1.1.13-0.20220915233716-71ac16282d12/go.mod h1:TBzl5BIHNXfS9+C35ZyJaklL7mLDbgUkcgXzSLa8Tk0=
|
github.com/json-iterator/go v1.1.13-0.20220915233716-71ac16282d12/go.mod h1:TBzl5BIHNXfS9+C35ZyJaklL7mLDbgUkcgXzSLa8Tk0=
|
||||||
github.com/klauspost/compress v1.18.5 h1:/h1gH5Ce+VWNLSWqPzOVn6XBO+vJbCNGvjoaGBFW2IE=
|
github.com/klauspost/compress v1.19.0 h1:sXLILfc9jV2QYWkzFOPWStmcUVH2RHEB1JCdY2oVvCQ=
|
||||||
github.com/klauspost/compress v1.18.5/go.mod h1:cwPg85FWrGar70rWktvGQj8/hthj3wpl0PGDogxkrSQ=
|
github.com/klauspost/compress v1.19.0/go.mod h1:cwPg85FWrGar70rWktvGQj8/hthj3wpl0PGDogxkrSQ=
|
||||||
github.com/klauspost/cpuid/v2 v2.3.0 h1:S4CRMLnYUhGeDFDqkGriYKdfoFlDnMtqTiI/sFzhA9Y=
|
github.com/klauspost/cpuid/v2 v2.3.0 h1:S4CRMLnYUhGeDFDqkGriYKdfoFlDnMtqTiI/sFzhA9Y=
|
||||||
github.com/klauspost/cpuid/v2 v2.3.0/go.mod h1:hqwkgyIinND0mEev00jJYCxPNVRVXFQeu1XKlok6oO0=
|
github.com/klauspost/cpuid/v2 v2.3.0/go.mod h1:hqwkgyIinND0mEev00jJYCxPNVRVXFQeu1XKlok6oO0=
|
||||||
github.com/kr/pretty v0.3.1 h1:flRD4NNwYAUpkphVc1HcthR4KEIFJ65n8Mw5qdRn3LE=
|
github.com/kr/pretty v0.3.1 h1:flRD4NNwYAUpkphVc1HcthR4KEIFJ65n8Mw5qdRn3LE=
|
||||||
@@ -242,8 +248,8 @@ github.com/pelletier/go-toml/v2 v2.2.4 h1:mye9XuhQ6gvn5h28+VilKrrPoQVanw5PMw/TB0
|
|||||||
github.com/pelletier/go-toml/v2 v2.2.4/go.mod h1:2gIqNv+qfxSVS7cM2xJQKtLSTLUE9V8t9Stt+h56mCY=
|
github.com/pelletier/go-toml/v2 v2.2.4/go.mod h1:2gIqNv+qfxSVS7cM2xJQKtLSTLUE9V8t9Stt+h56mCY=
|
||||||
github.com/peterbourgon/diskv/v3 v3.0.1 h1:x06SQA46+PKIUftmEujdwSEpIx8kR+M9eLYsUxeYveU=
|
github.com/peterbourgon/diskv/v3 v3.0.1 h1:x06SQA46+PKIUftmEujdwSEpIx8kR+M9eLYsUxeYveU=
|
||||||
github.com/peterbourgon/diskv/v3 v3.0.1/go.mod h1:kJ5Ny7vLdARGU3WUuy6uzO6T0nb/2gWcT1JiBvRmb5o=
|
github.com/peterbourgon/diskv/v3 v3.0.1/go.mod h1:kJ5Ny7vLdARGU3WUuy6uzO6T0nb/2gWcT1JiBvRmb5o=
|
||||||
github.com/pierrec/lz4/v4 v4.1.26 h1:GrpZw1gZttORinvzBdXPUXATeqlJjqUG/D87TKMnhjY=
|
github.com/pierrec/lz4/v4 v4.1.27 h1:+PhzhWDrjRj89TH2sw43nE3+4+W8lSxIuQadEHZyjUk=
|
||||||
github.com/pierrec/lz4/v4 v4.1.26/go.mod h1:EoQMVJgeeEOMsCqCzqFm2O0cJvljX2nGZjcRIPL34O4=
|
github.com/pierrec/lz4/v4 v4.1.27/go.mod h1:EoQMVJgeeEOMsCqCzqFm2O0cJvljX2nGZjcRIPL34O4=
|
||||||
github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4=
|
github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4=
|
||||||
github.com/pmezard/go-difflib v1.0.1-0.20181226105442-5d4384ee4fb2 h1:Jamvg5psRIccs7FGNTlIRMkT8wgtp5eCXdBlqhYGL6U=
|
github.com/pmezard/go-difflib v1.0.1-0.20181226105442-5d4384ee4fb2 h1:Jamvg5psRIccs7FGNTlIRMkT8wgtp5eCXdBlqhYGL6U=
|
||||||
github.com/pmezard/go-difflib v1.0.1-0.20181226105442-5d4384ee4fb2/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4=
|
github.com/pmezard/go-difflib v1.0.1-0.20181226105442-5d4384ee4fb2/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4=
|
||||||
@@ -285,14 +291,19 @@ github.com/spf13/pflag v1.0.10 h1:4EBh2KAYBwaONj6b2Ye1GiHfwjqyROoF4RwYO+vPwFk=
|
|||||||
github.com/spf13/pflag v1.0.10/go.mod h1:McXfInJRrz4CZXVZOBLb0bTZqETkiAhM9Iw0y3An2Bg=
|
github.com/spf13/pflag v1.0.10/go.mod h1:McXfInJRrz4CZXVZOBLb0bTZqETkiAhM9Iw0y3An2Bg=
|
||||||
github.com/spf13/viper v1.21.0 h1:x5S+0EU27Lbphp4UKm1C+1oQO+rKx36vfCoaVebLFSU=
|
github.com/spf13/viper v1.21.0 h1:x5S+0EU27Lbphp4UKm1C+1oQO+rKx36vfCoaVebLFSU=
|
||||||
github.com/spf13/viper v1.21.0/go.mod h1:P0lhsswPGWD/1lZJ9ny3fYnVqxiegrlNrEmgLjbTCAY=
|
github.com/spf13/viper v1.21.0/go.mod h1:P0lhsswPGWD/1lZJ9ny3fYnVqxiegrlNrEmgLjbTCAY=
|
||||||
|
github.com/stangelandcl/ppmd v0.1.1 h1:c25QazhlWUn5nmR1QOzafKhQxBicAr7GGCKER2aJ8H8=
|
||||||
|
github.com/stangelandcl/ppmd v0.1.1/go.mod h1:Rrv7M+/2P5jYr/GMLhBl7Ug3uJ1bUiVzr5LbbaV6xgY=
|
||||||
github.com/stretchr/objx v0.1.0/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+wExME=
|
github.com/stretchr/objx v0.1.0/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+wExME=
|
||||||
github.com/stretchr/objx v0.4.0/go.mod h1:YvHI0jy2hoMjB+UWwv71VJQ9isScKT/TqJzVSSt89Yw=
|
github.com/stretchr/objx v0.4.0/go.mod h1:YvHI0jy2hoMjB+UWwv71VJQ9isScKT/TqJzVSSt89Yw=
|
||||||
github.com/stretchr/objx v0.5.0/go.mod h1:Yh+to48EsGEfYuaHDzXPcE3xhTkx73EhmCGUpEOglKo=
|
github.com/stretchr/objx v0.5.0/go.mod h1:Yh+to48EsGEfYuaHDzXPcE3xhTkx73EhmCGUpEOglKo=
|
||||||
github.com/stretchr/objx v0.5.2/go.mod h1:FRsXN1f5AsAjCGJKqEizvkpNtU+EGNCLh3NxZ/8L+MA=
|
github.com/stretchr/objx v0.5.2/go.mod h1:FRsXN1f5AsAjCGJKqEizvkpNtU+EGNCLh3NxZ/8L+MA=
|
||||||
|
github.com/stretchr/objx v0.5.3 h1:jmXUvGomnU1o3W/V5h2VEradbpJDwGrzugQQvL0POH4=
|
||||||
|
github.com/stretchr/objx v0.5.3/go.mod h1:rDQraq+vQZU7Fde9LOZLr8Tax6zZvy4kuNKF+QYS+U0=
|
||||||
github.com/stretchr/testify v1.3.0/go.mod h1:M5WIy9Dh21IEIfnGCwXGc5bZfKNJtfHm1UVUgZn+9EI=
|
github.com/stretchr/testify v1.3.0/go.mod h1:M5WIy9Dh21IEIfnGCwXGc5bZfKNJtfHm1UVUgZn+9EI=
|
||||||
github.com/stretchr/testify v1.7.0/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/h/Wwjteg=
|
github.com/stretchr/testify v1.7.0/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/h/Wwjteg=
|
||||||
github.com/stretchr/testify v1.7.1/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/h/Wwjteg=
|
github.com/stretchr/testify v1.7.1/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/h/Wwjteg=
|
||||||
github.com/stretchr/testify v1.8.0/go.mod h1:yNjHg4UonilssWZ8iaSj1OCr/vHnekPRkoO+kdMU+MU=
|
github.com/stretchr/testify v1.8.0/go.mod h1:yNjHg4UonilssWZ8iaSj1OCr/vHnekPRkoO+kdMU+MU=
|
||||||
|
github.com/stretchr/testify v1.8.1/go.mod h1:w2LPCIKwWwSfY2zedu0+kehJoqGctiVI29o6fzry7u4=
|
||||||
github.com/stretchr/testify v1.8.4/go.mod h1:sz/lmYIOXD/1dqDmKjjqLyZ2RngseejIcXlSw2iwfAo=
|
github.com/stretchr/testify v1.8.4/go.mod h1:sz/lmYIOXD/1dqDmKjjqLyZ2RngseejIcXlSw2iwfAo=
|
||||||
github.com/stretchr/testify v1.10.0/go.mod h1:r2ic/lqez/lEtzL7wO/rwa5dbSLXVDPFyf8C91i36aY=
|
github.com/stretchr/testify v1.10.0/go.mod h1:r2ic/lqez/lEtzL7wO/rwa5dbSLXVDPFyf8C91i36aY=
|
||||||
github.com/stretchr/testify v1.11.1 h1:7s2iGBzp5EwR7/aIZr8ao5+dra3wiQyKjjFuvgVKu7U=
|
github.com/stretchr/testify v1.11.1 h1:7s2iGBzp5EwR7/aIZr8ao5+dra3wiQyKjjFuvgVKu7U=
|
||||||
@@ -311,6 +322,8 @@ github.com/twitchyliquid64/golang-asm v0.15.1 h1:SU5vSMR7hnwNxj24w34ZyCi/FmDZTkS
|
|||||||
github.com/twitchyliquid64/golang-asm v0.15.1/go.mod h1:a1lVb/DtPvCB8fslRZhAngC2+aY1QWCk3Cedj/Gdt08=
|
github.com/twitchyliquid64/golang-asm v0.15.1/go.mod h1:a1lVb/DtPvCB8fslRZhAngC2+aY1QWCk3Cedj/Gdt08=
|
||||||
github.com/ugorji/go/codec v1.3.1 h1:waO7eEiFDwidsBN6agj1vJQ4AG7lh2yqXyOXqhgQuyY=
|
github.com/ugorji/go/codec v1.3.1 h1:waO7eEiFDwidsBN6agj1vJQ4AG7lh2yqXyOXqhgQuyY=
|
||||||
github.com/ugorji/go/codec v1.3.1/go.mod h1:pRBVtBSKl77K30Bv8R2P+cLSGaTtex6fsA2Wjqmfxj4=
|
github.com/ugorji/go/codec v1.3.1/go.mod h1:pRBVtBSKl77K30Bv8R2P+cLSGaTtex6fsA2Wjqmfxj4=
|
||||||
|
github.com/ulikunitz/xz v0.5.15 h1:9DNdB5s+SgV3bQ2ApL10xRc35ck0DuIX/isZvIk+ubY=
|
||||||
|
github.com/ulikunitz/xz v0.5.15/go.mod h1:nbz6k7qbPmH4IRqmfOplQw/tblSgqTqBwxkY0oWt/14=
|
||||||
github.com/uptrace/opentelemetry-go-extra/otelutil v0.3.2 h1:3/aHKUq7qaFMWxyQV0W2ryNgg8x8rVeKVA20KJUkfS0=
|
github.com/uptrace/opentelemetry-go-extra/otelutil v0.3.2 h1:3/aHKUq7qaFMWxyQV0W2ryNgg8x8rVeKVA20KJUkfS0=
|
||||||
github.com/uptrace/opentelemetry-go-extra/otelutil v0.3.2/go.mod h1:Zit4b8AQXaXvA68+nzmbyDzqiyFRISyw1JiD5JqUBjw=
|
github.com/uptrace/opentelemetry-go-extra/otelutil v0.3.2/go.mod h1:Zit4b8AQXaXvA68+nzmbyDzqiyFRISyw1JiD5JqUBjw=
|
||||||
github.com/uptrace/opentelemetry-go-extra/otelzap v0.3.2 h1:cj/Z6FKTTYBnstI0Lni9PA+k2foounKIPUmj1LBwNiQ=
|
github.com/uptrace/opentelemetry-go-extra/otelzap v0.3.2 h1:cj/Z6FKTTYBnstI0Lni9PA+k2foounKIPUmj1LBwNiQ=
|
||||||
@@ -356,37 +369,39 @@ go.uber.org/zap v1.27.1 h1:08RqriUEv8+ArZRYSTXy1LeBScaMpVSTBhCeaZYfMYc=
|
|||||||
go.uber.org/zap v1.27.1/go.mod h1:GB2qFLM7cTU87MWRP2mPIjqfIDnGu+VIO4V/SdhGo2E=
|
go.uber.org/zap v1.27.1/go.mod h1:GB2qFLM7cTU87MWRP2mPIjqfIDnGu+VIO4V/SdhGo2E=
|
||||||
go.yaml.in/yaml/v3 v3.0.4 h1:tfq32ie2Jv2UxXFdLJdh3jXuOzWiL1fo0bu/FbuKpbc=
|
go.yaml.in/yaml/v3 v3.0.4 h1:tfq32ie2Jv2UxXFdLJdh3jXuOzWiL1fo0bu/FbuKpbc=
|
||||||
go.yaml.in/yaml/v3 v3.0.4/go.mod h1:DhzuOOF2ATzADvBadXxruRBLzYTpT36CKvDb3+aBEFg=
|
go.yaml.in/yaml/v3 v3.0.4/go.mod h1:DhzuOOF2ATzADvBadXxruRBLzYTpT36CKvDb3+aBEFg=
|
||||||
|
go4.org v0.0.0-20260112195520-a5071408f32f h1:ziUVAjmTPwQMBmYR1tbdRFJPtTcQUI12fH9QQjfb0Sw=
|
||||||
|
go4.org v0.0.0-20260112195520-a5071408f32f/go.mod h1:ZRJnO5ZI4zAwMFp+dS1+V6J6MSyAowhRqAE+DPa1Xp0=
|
||||||
golang.org/x/arch v0.22.0 h1:c/Zle32i5ttqRXjdLyyHZESLD/bB90DCU1g9l/0YBDI=
|
golang.org/x/arch v0.22.0 h1:c/Zle32i5ttqRXjdLyyHZESLD/bB90DCU1g9l/0YBDI=
|
||||||
golang.org/x/arch v0.22.0/go.mod h1:dNHoOeKiyja7GTvF9NJS1l3Z2yntpQNzgrjh1cU103A=
|
golang.org/x/arch v0.22.0/go.mod h1:dNHoOeKiyja7GTvF9NJS1l3Z2yntpQNzgrjh1cU103A=
|
||||||
golang.org/x/crypto v0.0.0-20190308221718-c2843e01d9a2/go.mod h1:djNgcEr1/C05ACkg1iLfiJU5Ep61QUkGW8qpdssI0+w=
|
golang.org/x/crypto v0.0.0-20190308221718-c2843e01d9a2/go.mod h1:djNgcEr1/C05ACkg1iLfiJU5Ep61QUkGW8qpdssI0+w=
|
||||||
golang.org/x/crypto v0.0.0-20210921155107-089bfa567519/go.mod h1:GvvjBRRGRdwPK5ydBHafDWAxML/pGHZbMvKqRZ5+Abc=
|
golang.org/x/crypto v0.0.0-20210921155107-089bfa567519/go.mod h1:GvvjBRRGRdwPK5ydBHafDWAxML/pGHZbMvKqRZ5+Abc=
|
||||||
golang.org/x/crypto v0.51.0 h1:IBPXwPfKxY7cWQZ38ZCIRPI50YLeevDLlLnyC5wRGTI=
|
golang.org/x/crypto v0.53.0 h1:QZ4Muo8THX6CizN2vPPd5fBGHyogrdK9fG4wLPFUsto=
|
||||||
golang.org/x/crypto v0.51.0/go.mod h1:8AdwkbraGNABw2kOX6YFPs3WM22XqI4EXEd8g+x7Oc8=
|
golang.org/x/crypto v0.53.0/go.mod h1:DNLU434OwVakk9PzuwV8w62mAJpRJL3vsgcfp4Qnsio=
|
||||||
golang.org/x/image v0.42.0 h1:1gSs6ehNWXLbkHBIPcWztk3D/6aIA/8hauiAYtlodVY=
|
golang.org/x/image v0.42.0 h1:1gSs6ehNWXLbkHBIPcWztk3D/6aIA/8hauiAYtlodVY=
|
||||||
golang.org/x/image v0.42.0/go.mod h1:rrpelvGFt+kLPAjPM4HeWPgrl0FtafueU//e5N0qk/Q=
|
golang.org/x/image v0.42.0/go.mod h1:rrpelvGFt+kLPAjPM4HeWPgrl0FtafueU//e5N0qk/Q=
|
||||||
golang.org/x/mod v0.6.0-dev.0.20220419223038-86c51ed26bb4/go.mod h1:jJ57K6gSWd91VN4djpZkiMVwK6gcyfeH4XE8wZrZaV4=
|
golang.org/x/mod v0.6.0-dev.0.20220419223038-86c51ed26bb4/go.mod h1:jJ57K6gSWd91VN4djpZkiMVwK6gcyfeH4XE8wZrZaV4=
|
||||||
golang.org/x/mod v0.36.0 h1:JJjpVx6myfUsUdAzZuOSTTmRE0PfZeNWzzvKrP7amb4=
|
golang.org/x/mod v0.37.0 h1:vF1DjpVEshcIqoEaauuHebaLk1O1forxjxBaVn884JQ=
|
||||||
golang.org/x/mod v0.36.0/go.mod h1:moc6ELqsWcOw5Ef3xVprK5ul/MvtVvkIXLziUOICjUQ=
|
golang.org/x/mod v0.37.0/go.mod h1:m8S8VeM9r4dzDwjrKO0a1sZP3YjeMamRRlD+fmR2Q/0=
|
||||||
golang.org/x/net v0.0.0-20190620200207-3b0461eec859/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s=
|
golang.org/x/net v0.0.0-20190620200207-3b0461eec859/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s=
|
||||||
golang.org/x/net v0.0.0-20210226172049-e18ecbb05110/go.mod h1:m0MpNAwzfU5UDzcl9v0D8zg8gWTRqZa9RBIspLL5mdg=
|
golang.org/x/net v0.0.0-20210226172049-e18ecbb05110/go.mod h1:m0MpNAwzfU5UDzcl9v0D8zg8gWTRqZa9RBIspLL5mdg=
|
||||||
golang.org/x/net v0.0.0-20220722155237-a158d28d115b/go.mod h1:XRhObCWvk6IyKnWLug+ECip1KBveYUHfp+8e9klMJ9c=
|
golang.org/x/net v0.0.0-20220722155237-a158d28d115b/go.mod h1:XRhObCWvk6IyKnWLug+ECip1KBveYUHfp+8e9klMJ9c=
|
||||||
golang.org/x/net v0.7.0/go.mod h1:2Tu9+aMcznHK/AK1HMvgo6xiTLG5rD5rZLDS+rp2Bjs=
|
golang.org/x/net v0.7.0/go.mod h1:2Tu9+aMcznHK/AK1HMvgo6xiTLG5rD5rZLDS+rp2Bjs=
|
||||||
golang.org/x/net v0.54.0 h1:2zJIZAxAHV/OHCDTCOHAYehQzLfSXuf/5SoL/Dv6w/w=
|
golang.org/x/net v0.56.0 h1:Rw8j/hFzGvJUZwNBXnAtf5sVDVt+65SK2C7IxCxZt5o=
|
||||||
golang.org/x/net v0.54.0/go.mod h1:Sj4oj8jK6XmHpBZU/zWHw3BV3abl4Kvi+Ut7cQcY+cQ=
|
golang.org/x/net v0.56.0/go.mod h1:D3Ku6r+V6JROoZK144D2XfMHFcMq/0zSfLelVTCFKec=
|
||||||
golang.org/x/oauth2 v0.36.0 h1:peZ/1z27fi9hUOFCAZaHyrpWG5lwe0RJEEEeH0ThlIs=
|
golang.org/x/oauth2 v0.36.0 h1:peZ/1z27fi9hUOFCAZaHyrpWG5lwe0RJEEEeH0ThlIs=
|
||||||
golang.org/x/oauth2 v0.36.0/go.mod h1:YDBUJMTkDnJS+A4BP4eZBjCqtokkg1hODuPjwiGPO7Q=
|
golang.org/x/oauth2 v0.36.0/go.mod h1:YDBUJMTkDnJS+A4BP4eZBjCqtokkg1hODuPjwiGPO7Q=
|
||||||
golang.org/x/sync v0.0.0-20190423024810-112230192c58/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
|
golang.org/x/sync v0.0.0-20190423024810-112230192c58/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
|
||||||
golang.org/x/sync v0.0.0-20220722155255-886fb9371eb4/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
|
golang.org/x/sync v0.0.0-20220722155255-886fb9371eb4/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
|
||||||
golang.org/x/sync v0.21.0 h1:HLII4xRRTtCRkxYp4HNFF0Js/Og6q2i++KXbg0gHCwM=
|
golang.org/x/sync v0.22.0 h1:SZjpbeLmrCk4xhRSZFNZW5gFUeCeFgjekvI/+gfScek=
|
||||||
golang.org/x/sync v0.21.0/go.mod h1:9xrNwdLfx4jkKbNva9FpL6vEN7evnE43NNNJQ2LF3+0=
|
golang.org/x/sync v0.22.0/go.mod h1:9xrNwdLfx4jkKbNva9FpL6vEN7evnE43NNNJQ2LF3+0=
|
||||||
golang.org/x/sys v0.0.0-20190215142949-d0b11bdaac8a/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
|
golang.org/x/sys v0.0.0-20190215142949-d0b11bdaac8a/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
|
||||||
golang.org/x/sys v0.0.0-20201119102817-f84b799fce68/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
|
golang.org/x/sys v0.0.0-20201119102817-f84b799fce68/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
|
||||||
golang.org/x/sys v0.0.0-20210615035016-665e8c7367d1/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
|
golang.org/x/sys v0.0.0-20210615035016-665e8c7367d1/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
|
||||||
golang.org/x/sys v0.0.0-20220520151302-bc2c85ada10a/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
|
golang.org/x/sys v0.0.0-20220520151302-bc2c85ada10a/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
|
||||||
golang.org/x/sys v0.0.0-20220722155257-8c9f86f7a55f/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
|
golang.org/x/sys v0.0.0-20220722155257-8c9f86f7a55f/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
|
||||||
golang.org/x/sys v0.5.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
|
golang.org/x/sys v0.5.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
|
||||||
golang.org/x/sys v0.44.0 h1:ildZl3J4uzeKP07r2F++Op7E9B29JRUy+a27EibtBTQ=
|
golang.org/x/sys v0.46.0 h1:noSf2Fq6F8DBgS+LysIkx7rIExoNHJsxOAtPp4rthXw=
|
||||||
golang.org/x/sys v0.44.0/go.mod h1:4GL1E5IUh+htKOUEOaiffhrAeqysfVGipDYzABqnCmw=
|
golang.org/x/sys v0.46.0/go.mod h1:4GL1E5IUh+htKOUEOaiffhrAeqysfVGipDYzABqnCmw=
|
||||||
golang.org/x/term v0.0.0-20201126162022-7de9c90e9dd1/go.mod h1:bj7SfCRtBDWHUb9snDiAeCFNEtKQo2Wmx5Cou7ajbmo=
|
golang.org/x/term v0.0.0-20201126162022-7de9c90e9dd1/go.mod h1:bj7SfCRtBDWHUb9snDiAeCFNEtKQo2Wmx5Cou7ajbmo=
|
||||||
golang.org/x/term v0.0.0-20210927222741-03fcf44c2211/go.mod h1:jbD1KX2456YbFQfuXm/mYQcufACuNUgVhRMnK/tPxf8=
|
golang.org/x/term v0.0.0-20210927222741-03fcf44c2211/go.mod h1:jbD1KX2456YbFQfuXm/mYQcufACuNUgVhRMnK/tPxf8=
|
||||||
golang.org/x/term v0.5.0/go.mod h1:jMB1sMXY+tzblOD4FWmEbocvup2/aLOaQEp7JmGp78k=
|
golang.org/x/term v0.5.0/go.mod h1:jMB1sMXY+tzblOD4FWmEbocvup2/aLOaQEp7JmGp78k=
|
||||||
@@ -394,15 +409,15 @@ golang.org/x/text v0.3.0/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ=
|
|||||||
golang.org/x/text v0.3.3/go.mod h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ=
|
golang.org/x/text v0.3.3/go.mod h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ=
|
||||||
golang.org/x/text v0.3.7/go.mod h1:u+2+/6zg+i71rQMx5EYifcz6MCKuco9NR6JIITiCfzQ=
|
golang.org/x/text v0.3.7/go.mod h1:u+2+/6zg+i71rQMx5EYifcz6MCKuco9NR6JIITiCfzQ=
|
||||||
golang.org/x/text v0.7.0/go.mod h1:mrYo+phRRbMaCq/xk9113O4dZlRixOauAjOtrjsXDZ8=
|
golang.org/x/text v0.7.0/go.mod h1:mrYo+phRRbMaCq/xk9113O4dZlRixOauAjOtrjsXDZ8=
|
||||||
golang.org/x/text v0.38.0 h1:sXmwo9DwP3OK9EZ7PqAdaooSGozfl/3a6/xJcbzPRhE=
|
golang.org/x/text v0.40.0 h1:Ub2Z6/xjgF1WrYQz2nuITOEegKFtiIy+rieRJ5lHZKs=
|
||||||
golang.org/x/text v0.38.0/go.mod h1:YXZt3QhHUKYT53r2lLKFIVi6Ao1jdzrTR/KQ09qyxF4=
|
golang.org/x/text v0.40.0/go.mod h1:hpnzDAfGV753zIKo+wk3u1bVKCGPbrnF7+7LBF/UHVY=
|
||||||
golang.org/x/time v0.15.0 h1:bbrp8t3bGUeFOx08pvsMYRTCVSMk89u4tKbNOZbp88U=
|
golang.org/x/time v0.15.0 h1:bbrp8t3bGUeFOx08pvsMYRTCVSMk89u4tKbNOZbp88U=
|
||||||
golang.org/x/time v0.15.0/go.mod h1:Y4YMaQmXwGQZoFaVFk4YpCt4FLQMYKZe9oeV/f4MSno=
|
golang.org/x/time v0.15.0/go.mod h1:Y4YMaQmXwGQZoFaVFk4YpCt4FLQMYKZe9oeV/f4MSno=
|
||||||
golang.org/x/tools v0.0.0-20180917221912-90fa682c2a6e/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ=
|
golang.org/x/tools v0.0.0-20180917221912-90fa682c2a6e/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ=
|
||||||
golang.org/x/tools v0.0.0-20191119224855-298f0cb1881e/go.mod h1:b+2E5dAYhXwXZwtnZ6UAqBI28+e2cm9otk0dWdXHAEo=
|
golang.org/x/tools v0.0.0-20191119224855-298f0cb1881e/go.mod h1:b+2E5dAYhXwXZwtnZ6UAqBI28+e2cm9otk0dWdXHAEo=
|
||||||
golang.org/x/tools v0.1.12/go.mod h1:hNGJHUnrk76NpqgfD5Aqm5Crs+Hm0VOH/i9J2+nxYbc=
|
golang.org/x/tools v0.1.12/go.mod h1:hNGJHUnrk76NpqgfD5Aqm5Crs+Hm0VOH/i9J2+nxYbc=
|
||||||
golang.org/x/tools v0.45.0 h1:18qN3FAooORvApf5XjCXgsuayZOEtXf6JK18I3+ONa8=
|
golang.org/x/tools v0.47.0 h1:7Kn5x/d1svx/PzryTsqeoZN4TZwqeH5pGWjefhLi/1Q=
|
||||||
golang.org/x/tools v0.45.0/go.mod h1:LuUGqqaXcXMEFEruIVJVm5mgDD8vww/z/SR1gQ4uE/0=
|
golang.org/x/tools v0.47.0/go.mod h1:dFHnyTvFWY212G+h7ZY4Vsp/K3U4/7W9TyVaAul8uCA=
|
||||||
golang.org/x/xerrors v0.0.0-20190717185122-a985d3407aa7/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
|
golang.org/x/xerrors v0.0.0-20190717185122-a985d3407aa7/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
|
||||||
gonum.org/v1/gonum v0.17.0 h1:VbpOemQlsSMrYmn7T2OUvQ4dqxQXU+ouZFQsZOx50z4=
|
gonum.org/v1/gonum v0.17.0 h1:VbpOemQlsSMrYmn7T2OUvQ4dqxQXU+ouZFQsZOx50z4=
|
||||||
gonum.org/v1/gonum v0.17.0/go.mod h1:El3tOrEuMpv2UdMrbNlKEh9vd86bmQ6vqIcDwxEOc1E=
|
gonum.org/v1/gonum v0.17.0/go.mod h1:El3tOrEuMpv2UdMrbNlKEh9vd86bmQ6vqIcDwxEOc1E=
|
||||||
|
|||||||
@@ -2,8 +2,6 @@
|
|||||||
package sync
|
package sync
|
||||||
|
|
||||||
import (
|
import (
|
||||||
"archive/zip"
|
|
||||||
"bytes"
|
|
||||||
"context"
|
"context"
|
||||||
"crypto/sha256"
|
"crypto/sha256"
|
||||||
"encoding/hex"
|
"encoding/hex"
|
||||||
@@ -11,21 +9,19 @@ import (
|
|||||||
"errors"
|
"errors"
|
||||||
"fmt"
|
"fmt"
|
||||||
"io"
|
"io"
|
||||||
"math"
|
|
||||||
"os"
|
"os"
|
||||||
"path"
|
|
||||||
"path/filepath"
|
"path/filepath"
|
||||||
"strings"
|
"strings"
|
||||||
|
|
||||||
"github.com/Rain-kl/Wavelet/internal/apps/agent/protocol"
|
"github.com/Rain-kl/Wavelet/internal/apps/agent/protocol"
|
||||||
"github.com/Rain-kl/Wavelet/internal/apps/agent/state"
|
"github.com/Rain-kl/Wavelet/internal/apps/agent/state"
|
||||||
|
"github.com/Rain-kl/Wavelet/pkg/pagesarchive"
|
||||||
)
|
)
|
||||||
|
|
||||||
const (
|
const (
|
||||||
pagesMaxExtractedFileBytes = 100 * 1024 * 1024
|
pagesDirPerm = 0o755
|
||||||
pagesDirPerm = 0o755
|
pagesFilePerm = 0o644
|
||||||
pagesFilePerm = 0o644
|
pagesManifestFilePerm = 0o644
|
||||||
pagesManifestFilePerm = 0o644
|
|
||||||
)
|
)
|
||||||
|
|
||||||
type pagesSourceDocument struct {
|
type pagesSourceDocument struct {
|
||||||
@@ -223,89 +219,26 @@ func referencedPagesDeployments(config *protocol.ActiveConfigResponse) ([]pagesD
|
|||||||
return result, nil
|
return result, nil
|
||||||
}
|
}
|
||||||
|
|
||||||
func findCommonRootPrefix(files []*zip.File) (string, error) {
|
|
||||||
var firstFilePath string
|
|
||||||
hasMultipleFiles := false
|
|
||||||
for _, item := range files {
|
|
||||||
relativePath, skip, err := normalizePagesArchivePath(item.Name)
|
|
||||||
if err != nil {
|
|
||||||
return "", err
|
|
||||||
}
|
|
||||||
if skip {
|
|
||||||
continue
|
|
||||||
}
|
|
||||||
normalizedPath := filepath.ToSlash(relativePath)
|
|
||||||
if firstFilePath == "" {
|
|
||||||
firstFilePath = normalizedPath
|
|
||||||
} else {
|
|
||||||
hasMultipleFiles = true
|
|
||||||
}
|
|
||||||
}
|
|
||||||
if firstFilePath == "" {
|
|
||||||
return "", nil
|
|
||||||
}
|
|
||||||
parts := strings.Split(firstFilePath, "/")
|
|
||||||
if len(parts) <= 1 {
|
|
||||||
return "", nil
|
|
||||||
}
|
|
||||||
commonPrefix := parts[0] + "/"
|
|
||||||
if hasMultipleFiles {
|
|
||||||
for _, item := range files {
|
|
||||||
relativePath, skip, err := normalizePagesArchivePath(item.Name)
|
|
||||||
if err != nil {
|
|
||||||
return "", err
|
|
||||||
}
|
|
||||||
if skip {
|
|
||||||
continue
|
|
||||||
}
|
|
||||||
normalizedPath := filepath.ToSlash(relativePath)
|
|
||||||
if !strings.HasPrefix(normalizedPath, commonPrefix) {
|
|
||||||
return "", nil
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
return commonPrefix, nil
|
|
||||||
}
|
|
||||||
|
|
||||||
func extractPagesPackage(packageBytes []byte, releaseDir string, deployment pagesDeploymentSource) error {
|
func extractPagesPackage(packageBytes []byte, releaseDir string, deployment pagesDeploymentSource) error {
|
||||||
tmpDir := releaseDir + ".tmp"
|
tmpDir := releaseDir + ".tmp"
|
||||||
_ = os.RemoveAll(tmpDir)
|
_ = os.RemoveAll(tmpDir)
|
||||||
if err := os.MkdirAll(tmpDir, pagesDirPerm); err != nil {
|
if err := os.MkdirAll(tmpDir, pagesDirPerm); err != nil {
|
||||||
return err
|
return err
|
||||||
}
|
}
|
||||||
reader, err := zip.NewReader(bytes.NewReader(packageBytes), int64(len(packageBytes)))
|
format, err := pagesarchive.DetectFormat("", packageBytes)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
_ = os.RemoveAll(tmpDir)
|
_ = os.RemoveAll(tmpDir)
|
||||||
return fmt.Errorf("open Pages zip: %w", err)
|
return fmt.Errorf("detect Pages package format: %w", err)
|
||||||
}
|
}
|
||||||
commonPrefix, err := findCommonRootPrefix(reader.File)
|
// Control plane already inspected and accepted this package. Agent only
|
||||||
if err != nil {
|
// verifies download integrity (checksum) and performs local-safe extract
|
||||||
|
// (path escape / symlink guards). Size and file-count limits are not re-applied.
|
||||||
|
if err := pagesarchive.ExtractBytes(packageBytes, format, tmpDir, pagesarchive.ExtractOptions{
|
||||||
|
StripCommonRoot: true,
|
||||||
|
EnforceLimits: false,
|
||||||
|
}); err != nil {
|
||||||
_ = os.RemoveAll(tmpDir)
|
_ = os.RemoveAll(tmpDir)
|
||||||
return err
|
return fmt.Errorf("extract Pages package: %w", err)
|
||||||
}
|
|
||||||
for _, item := range reader.File {
|
|
||||||
relativePath, skip, err := normalizePagesArchivePath(item.Name)
|
|
||||||
if err != nil {
|
|
||||||
_ = os.RemoveAll(tmpDir)
|
|
||||||
return err
|
|
||||||
}
|
|
||||||
if skip {
|
|
||||||
continue
|
|
||||||
}
|
|
||||||
if commonPrefix != "" {
|
|
||||||
slashPath := filepath.ToSlash(relativePath)
|
|
||||||
if strings.HasPrefix(slashPath, commonPrefix) {
|
|
||||||
relativePath = filepath.FromSlash(strings.TrimPrefix(slashPath, commonPrefix))
|
|
||||||
}
|
|
||||||
}
|
|
||||||
if item.FileInfo().Mode()&os.ModeSymlink != 0 {
|
|
||||||
_ = os.RemoveAll(tmpDir)
|
|
||||||
return fmt.Errorf("pages package contains unsupported symlink: %s", relativePath)
|
|
||||||
}
|
|
||||||
if err := extractPagesFile(item, filepath.Join(tmpDir, relativePath)); err != nil {
|
|
||||||
_ = os.RemoveAll(tmpDir)
|
|
||||||
return err
|
|
||||||
}
|
|
||||||
}
|
}
|
||||||
if err := writePagesMarker(tmpDir, deployment); err != nil {
|
if err := writePagesMarker(tmpDir, deployment); err != nil {
|
||||||
_ = os.RemoveAll(tmpDir)
|
_ = os.RemoveAll(tmpDir)
|
||||||
@@ -315,42 +248,6 @@ func extractPagesPackage(packageBytes []byte, releaseDir string, deployment page
|
|||||||
return os.Rename(tmpDir, releaseDir)
|
return os.Rename(tmpDir, releaseDir)
|
||||||
}
|
}
|
||||||
|
|
||||||
func copyPagesZipEntryContent(dst io.Writer, src io.Reader, declaredSize uint64) (int64, error) {
|
|
||||||
if declaredSize > pagesMaxExtractedFileBytes || declaredSize > uint64(math.MaxInt64) {
|
|
||||||
return 0, errors.New("pages file size out of bounds")
|
|
||||||
}
|
|
||||||
if declaredSize > 0 {
|
|
||||||
return io.CopyN(dst, src, int64(declaredSize)) //nolint:gosec // declaredSize is bounded to math.MaxInt64 above
|
|
||||||
}
|
|
||||||
limited := io.LimitReader(src, pagesMaxExtractedFileBytes+1)
|
|
||||||
written, err := io.Copy(dst, limited)
|
|
||||||
if written > pagesMaxExtractedFileBytes {
|
|
||||||
return written, errors.New("pages file size out of bounds")
|
|
||||||
}
|
|
||||||
return written, err
|
|
||||||
}
|
|
||||||
|
|
||||||
func extractPagesFile(item *zip.File, targetPath string) error {
|
|
||||||
if err := os.MkdirAll(filepath.Dir(targetPath), pagesDirPerm); err != nil {
|
|
||||||
return err
|
|
||||||
}
|
|
||||||
source, err := item.Open()
|
|
||||||
if err != nil {
|
|
||||||
return err
|
|
||||||
}
|
|
||||||
defer func() { _ = source.Close() }()
|
|
||||||
target, err := os.OpenFile(targetPath, os.O_CREATE|os.O_WRONLY|os.O_TRUNC, pagesFilePerm) //nolint:gosec // targetPath is under managed PagesDir from validated zip entry
|
|
||||||
if err != nil {
|
|
||||||
return err
|
|
||||||
}
|
|
||||||
defer func() { _ = target.Close() }()
|
|
||||||
_, err = copyPagesZipEntryContent(target, source, item.UncompressedSize64)
|
|
||||||
if err != nil {
|
|
||||||
return fmt.Errorf("%s: %w", item.Name, err)
|
|
||||||
}
|
|
||||||
return nil
|
|
||||||
}
|
|
||||||
|
|
||||||
func switchPagesCurrentDir(baseDir string, deploymentID uint, releaseDir string) error {
|
func switchPagesCurrentDir(baseDir string, deploymentID uint, releaseDir string) error {
|
||||||
currentDir := pagesCurrentDir(baseDir, deploymentID)
|
currentDir := pagesCurrentDir(baseDir, deploymentID)
|
||||||
previousDir := currentDir + ".previous"
|
previousDir := currentDir + ".previous"
|
||||||
@@ -440,24 +337,6 @@ func copyPagesDir(sourceDir string, targetDir string) error {
|
|||||||
})
|
})
|
||||||
}
|
}
|
||||||
|
|
||||||
func normalizePagesArchivePath(raw string) (string, bool, error) {
|
|
||||||
name := strings.TrimSpace(filepath.ToSlash(raw))
|
|
||||||
if name == "" || strings.HasSuffix(name, "/") {
|
|
||||||
return "", true, nil
|
|
||||||
}
|
|
||||||
if strings.HasPrefix(name, "/") {
|
|
||||||
return "", false, fmt.Errorf("pages package contains absolute path: %s", raw)
|
|
||||||
}
|
|
||||||
cleaned := path.Clean(name)
|
|
||||||
if cleaned == "." {
|
|
||||||
return "", true, nil
|
|
||||||
}
|
|
||||||
if cleaned == ".." || strings.HasPrefix(cleaned, "../") || strings.Contains(cleaned, "/../") {
|
|
||||||
return "", false, fmt.Errorf("pages package path escapes deployment root: %s", raw)
|
|
||||||
}
|
|
||||||
return filepath.FromSlash(cleaned), false, nil
|
|
||||||
}
|
|
||||||
|
|
||||||
func markerMatches(dir string, deployment pagesDeploymentSource) bool {
|
func markerMatches(dir string, deployment pagesDeploymentSource) bool {
|
||||||
data, err := os.ReadFile(filepath.Join(dir, ".openflare-pages.json")) //nolint:gosec // dir is managed PagesDir
|
data, err := os.ReadFile(filepath.Join(dir, ".openflare-pages.json")) //nolint:gosec // dir is managed PagesDir
|
||||||
if err != nil {
|
if err != nil {
|
||||||
|
|||||||
@@ -525,7 +525,7 @@ func TestSyncOnceRejectsPagesZipSlipBeforeApply(t *testing.T) {
|
|||||||
service.SetPagesDir(t.TempDir())
|
service.SetPagesDir(t.TempDir())
|
||||||
|
|
||||||
err := service.SyncOnce(context.Background(), &protocol.ActiveConfigMeta{Version: "20260309-102", Checksum: "pages-config-checksum"})
|
err := service.SyncOnce(context.Background(), &protocol.ActiveConfigMeta{Version: "20260309-102", Checksum: "pages-config-checksum"})
|
||||||
if err == nil || !strings.Contains(err.Error(), "escapes deployment root") {
|
if err == nil || (!strings.Contains(err.Error(), "escapes deployment root") && !strings.Contains(err.Error(), "escapes directory")) {
|
||||||
t.Fatalf("expected zip-slip rejection, got %v", err)
|
t.Fatalf("expected zip-slip rejection, got %v", err)
|
||||||
}
|
}
|
||||||
if len(manager.applyRouteContents) != 0 {
|
if len(manager.applyRouteContents) != 0 {
|
||||||
|
|||||||
@@ -9,6 +9,7 @@ import (
|
|||||||
"errors"
|
"errors"
|
||||||
"strings"
|
"strings"
|
||||||
|
|
||||||
|
"github.com/Rain-kl/Wavelet/internal/apps/openflare/pages"
|
||||||
"github.com/Rain-kl/Wavelet/internal/model"
|
"github.com/Rain-kl/Wavelet/internal/model"
|
||||||
openrestyrender "github.com/Rain-kl/Wavelet/pkg/render/openresty"
|
openrestyrender "github.com/Rain-kl/Wavelet/pkg/render/openresty"
|
||||||
"gorm.io/gorm"
|
"gorm.io/gorm"
|
||||||
@@ -38,10 +39,20 @@ func getActiveConfigForAgent(ctx context.Context) (*ConfigResponse, error) {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// Main config version history is independent of Pages deployment history.
|
||||||
|
// Agents always receive pages routes bound to each project's current active
|
||||||
|
// deployment so config rollback never depends on pruned packages.
|
||||||
|
sourceJSON := version.SnapshotJSON
|
||||||
|
if rebound, rebindErr := pages.RebindSnapshotPagesToCurrentActive(ctx, version.SnapshotJSON); rebindErr != nil {
|
||||||
|
return nil, rebindErr
|
||||||
|
} else if strings.TrimSpace(rebound) != "" {
|
||||||
|
sourceJSON = rebound
|
||||||
|
}
|
||||||
|
|
||||||
return &ConfigResponse{
|
return &ConfigResponse{
|
||||||
Version: version.Version,
|
Version: version.Version,
|
||||||
Checksum: version.Checksum,
|
Checksum: version.Checksum,
|
||||||
SourceConfigJSON: version.SnapshotJSON,
|
SourceConfigJSON: sourceJSON,
|
||||||
SupportFiles: sourceSupportFiles(supportFiles),
|
SupportFiles: sourceSupportFiles(supportFiles),
|
||||||
CreatedAt: version.CreatedAt,
|
CreatedAt: version.CreatedAt,
|
||||||
}, nil
|
}, nil
|
||||||
|
|||||||
@@ -59,6 +59,9 @@ func validateOptionWithState(ctx context.Context, option model.OpenFlareOption,
|
|||||||
if err := validateAgentOption(option.Key, option.Value); err != nil {
|
if err := validateAgentOption(option.Key, option.Value); err != nil {
|
||||||
return err
|
return err
|
||||||
}
|
}
|
||||||
|
if err := validatePagesOption(option.Key, option.Value); err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
return validateUptimeKumaOption(ctx, option.Key, option.Value, state)
|
return validateUptimeKumaOption(ctx, option.Key, option.Value, state)
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -109,6 +112,23 @@ func validateAgentOption(key, value string) error {
|
|||||||
return nil
|
return nil
|
||||||
}
|
}
|
||||||
|
|
||||||
|
func validatePagesOption(key, value string) error {
|
||||||
|
trimmed := strings.TrimSpace(value)
|
||||||
|
switch key {
|
||||||
|
case model.ConfigKeyPagesMaxPackageSizeMB:
|
||||||
|
intValue, err := strconv.Atoi(trimmed)
|
||||||
|
if err != nil || intValue < 1 || intValue > 2048 {
|
||||||
|
return fmt.Errorf("%s 必须为 1~2048 的整数(MiB)", key)
|
||||||
|
}
|
||||||
|
case model.ConfigKeyPagesMaxHistoryCount:
|
||||||
|
intValue, err := strconv.Atoi(trimmed)
|
||||||
|
if err != nil || intValue < 0 {
|
||||||
|
return fmt.Errorf("%s 必须为大于等于 0 的整数(0 表示不限制)", key)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
|
||||||
func validateUptimeKumaOption(ctx context.Context, key, value string, state map[string]string) error {
|
func validateUptimeKumaOption(ctx context.Context, key, value string, state map[string]string) error {
|
||||||
trimmed := strings.TrimSpace(value)
|
trimmed := strings.TrimSpace(value)
|
||||||
switch key {
|
switch key {
|
||||||
|
|||||||
@@ -14,9 +14,13 @@ const (
|
|||||||
errPagesDeploymentMismatch = "pages 部署不属于该项目"
|
errPagesDeploymentMismatch = "pages 部署不属于该项目"
|
||||||
errPagesDeleteActiveDeploy = "不能删除当前激活的 Pages 部署"
|
errPagesDeleteActiveDeploy = "不能删除当前激活的 Pages 部署"
|
||||||
errPagesPackageMissing = "缺少 Pages 部署包"
|
errPagesPackageMissing = "缺少 Pages 部署包"
|
||||||
errPagesPackageNotZip = "pages 部署包必须是 .zip 文件"
|
errPagesPackageNotZip = "pages 部署包必须是 .zip 文件" // legacy alias kept for tests
|
||||||
errPagesPackageInvalidZip = "pages 部署包不是有效 zip 文件"
|
errPagesPackageUnsupported = "pages 部署包仅支持 zip、tar.gz、tar.xz、tar.bz2、tar、7z 格式"
|
||||||
|
errPagesPackageInvalidZip = "pages 部署包不是有效 zip 文件" // legacy alias
|
||||||
|
errPagesPackageInvalid = "pages 部署包不是有效的压缩文件"
|
||||||
errPagesPackageEmpty = "pages 部署包不能为空"
|
errPagesPackageEmpty = "pages 部署包不能为空"
|
||||||
|
errPagesPackageExtractedTooLarge = "pages 部署包展开后体积超过限制"
|
||||||
|
errPagesPackageFileTooLarge = "pages 部署包内文件过大"
|
||||||
errPagesAPIProxyPathRequired = "启用 API 反代时,匹配路径不能为空"
|
errPagesAPIProxyPathRequired = "启用 API 反代时,匹配路径不能为空"
|
||||||
errPagesAPIProxyPathPrefix = "API 反代匹配路径必须以 '/' 开头"
|
errPagesAPIProxyPathPrefix = "API 反代匹配路径必须以 '/' 开头"
|
||||||
errPagesAPIProxyPassRequired = "启用 API 反代时,后端服务地址不能为空" //nolint:gosec // false positive: this is an error message, not hardcoded credentials
|
errPagesAPIProxyPassRequired = "启用 API 反代时,后端服务地址不能为空" //nolint:gosec // false positive: this is an error message, not hardcoded credentials
|
||||||
|
|||||||
@@ -4,14 +4,12 @@
|
|||||||
package pages
|
package pages
|
||||||
|
|
||||||
import (
|
import (
|
||||||
"archive/zip"
|
|
||||||
"context"
|
"context"
|
||||||
"crypto/sha256"
|
"crypto/sha256"
|
||||||
"encoding/hex"
|
"encoding/hex"
|
||||||
"errors"
|
"errors"
|
||||||
"fmt"
|
"fmt"
|
||||||
"io"
|
"io"
|
||||||
"math"
|
|
||||||
"mime/multipart"
|
"mime/multipart"
|
||||||
"os"
|
"os"
|
||||||
"path"
|
"path"
|
||||||
@@ -22,17 +20,22 @@ import (
|
|||||||
"github.com/Rain-kl/Wavelet/internal/apps/upload"
|
"github.com/Rain-kl/Wavelet/internal/apps/upload"
|
||||||
"github.com/Rain-kl/Wavelet/internal/model"
|
"github.com/Rain-kl/Wavelet/internal/model"
|
||||||
"github.com/Rain-kl/Wavelet/internal/repository"
|
"github.com/Rain-kl/Wavelet/internal/repository"
|
||||||
|
"github.com/Rain-kl/Wavelet/pkg/logger"
|
||||||
|
"github.com/Rain-kl/Wavelet/pkg/pagesarchive"
|
||||||
)
|
)
|
||||||
|
|
||||||
const (
|
const (
|
||||||
pagesMaxDeploymentFiles = 1000
|
pagesMaxDeploymentFiles = 1000
|
||||||
pagesMaxDeploymentBytes = 100 * 1024 * 1024
|
defaultPagesMaxPackageSizeMB = 100
|
||||||
defaultPagesEntryFile = "index.html"
|
maxPagesMaxPackageSizeMB = 2048
|
||||||
defaultPagesFallbackPath = "/index.html"
|
defaultPagesMaxHistoryCount = 20
|
||||||
pagesDeploymentUploadType = "openflare_pages_deployment"
|
defaultPagesEntryFile = "index.html"
|
||||||
mimeTypeApplicationZip = "application/zip"
|
defaultPagesFallbackPath = "/index.html"
|
||||||
pagesMaxPathLength = 512
|
pagesDeploymentUploadType = "openflare_pages_deployment"
|
||||||
bytesPerKiB = 1024
|
pagesMaxPathLength = 512
|
||||||
|
bytesPerMiB = 1024 * 1024
|
||||||
|
pagesExtractedSizeMultiplier = 4
|
||||||
|
pagesMinExtractedSizeBytes = 100 * bytesPerMiB
|
||||||
)
|
)
|
||||||
|
|
||||||
var pagesSlugPattern = regexp.MustCompile(`^[a-z0-9][a-z0-9-]{0,126}[a-z0-9]$|^[a-z0-9]$`)
|
var pagesSlugPattern = regexp.MustCompile(`^[a-z0-9][a-z0-9-]{0,126}[a-z0-9]$|^[a-z0-9]$`)
|
||||||
@@ -42,6 +45,14 @@ type deploymentManifest struct {
|
|||||||
FileCount int
|
FileCount int
|
||||||
TotalSize int64
|
TotalSize int64
|
||||||
EntryFile string
|
EntryFile string
|
||||||
|
Format pagesarchive.Format
|
||||||
|
}
|
||||||
|
|
||||||
|
type pagesLimits struct {
|
||||||
|
PackageBytes int64
|
||||||
|
ExtractedBytes int64
|
||||||
|
MaxFiles int
|
||||||
|
HistoryCount int
|
||||||
}
|
}
|
||||||
|
|
||||||
func isUniqueConstraintError(err error) bool {
|
func isUniqueConstraintError(err error) bool {
|
||||||
@@ -51,6 +62,38 @@ func isUniqueConstraintError(err error) bool {
|
|||||||
return strings.Contains(strings.ToLower(err.Error()), "unique")
|
return strings.Contains(strings.ToLower(err.Error()), "unique")
|
||||||
}
|
}
|
||||||
|
|
||||||
|
func resolvePagesLimits(ctx context.Context) pagesLimits {
|
||||||
|
packageMB := defaultPagesMaxPackageSizeMB
|
||||||
|
if value, err := repository.GetIntByKey(ctx, model.ConfigKeyPagesMaxPackageSizeMB); err == nil && value > 0 {
|
||||||
|
packageMB = value
|
||||||
|
}
|
||||||
|
if packageMB > maxPagesMaxPackageSizeMB {
|
||||||
|
packageMB = maxPagesMaxPackageSizeMB
|
||||||
|
}
|
||||||
|
|
||||||
|
historyCount := defaultPagesMaxHistoryCount
|
||||||
|
if value, err := repository.GetIntByKey(ctx, model.ConfigKeyPagesMaxHistoryCount); err == nil {
|
||||||
|
if value < 0 {
|
||||||
|
historyCount = 0
|
||||||
|
} else {
|
||||||
|
historyCount = value
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
packageBytes := int64(packageMB) * bytesPerMiB
|
||||||
|
extractedBytes := packageBytes * pagesExtractedSizeMultiplier
|
||||||
|
if extractedBytes < pagesMinExtractedSizeBytes {
|
||||||
|
extractedBytes = pagesMinExtractedSizeBytes
|
||||||
|
}
|
||||||
|
|
||||||
|
return pagesLimits{
|
||||||
|
PackageBytes: packageBytes,
|
||||||
|
ExtractedBytes: extractedBytes,
|
||||||
|
MaxFiles: pagesMaxDeploymentFiles,
|
||||||
|
HistoryCount: historyCount,
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
func normalizePagesSlug(raw string) string {
|
func normalizePagesSlug(raw string) string {
|
||||||
value := strings.ToLower(strings.TrimSpace(raw))
|
value := strings.ToLower(strings.TrimSpace(raw))
|
||||||
var builder strings.Builder
|
var builder strings.Builder
|
||||||
@@ -76,7 +119,7 @@ func validateAndNormalizePagesRootDir(raw string) (string, error) {
|
|||||||
return "", nil
|
return "", nil
|
||||||
}
|
}
|
||||||
if len(value) > pagesMaxPathLength {
|
if len(value) > pagesMaxPathLength {
|
||||||
return "", errors.New("pages 根目录长度不能超过 512") // error 消息首字母小写
|
return "", errors.New("pages 根目录长度不能超过 512")
|
||||||
}
|
}
|
||||||
if strings.Contains(value, "\\") || strings.ContainsAny(value, "\"';") {
|
if strings.Contains(value, "\\") || strings.ContainsAny(value, "\"';") {
|
||||||
return "", errors.New("pages 根目录包含不支持的字符")
|
return "", errors.New("pages 根目录包含不支持的字符")
|
||||||
@@ -151,29 +194,50 @@ func normalizePagesEntryFile(raw string) string {
|
|||||||
return strings.TrimPrefix(value, "/")
|
return strings.TrimPrefix(value, "/")
|
||||||
}
|
}
|
||||||
|
|
||||||
func persistPagesUploadTemp(fileHeader *multipart.FileHeader) (string, string, int64, error) {
|
func persistPagesUploadTemp(fileHeader *multipart.FileHeader, maxPackageBytes int64) (string, string, int64, pagesarchive.Format, error) {
|
||||||
|
format, ok := pagesarchive.DetectFormatFromName(fileHeader.Filename)
|
||||||
|
if !ok {
|
||||||
|
return "", "", 0, "", errors.New(errPagesPackageUnsupported)
|
||||||
|
}
|
||||||
file, err := fileHeader.Open()
|
file, err := fileHeader.Open()
|
||||||
if err != nil {
|
if err != nil {
|
||||||
return "", "", 0, err
|
return "", "", 0, "", err
|
||||||
}
|
}
|
||||||
defer func() { _ = file.Close() }()
|
defer func() { _ = file.Close() }()
|
||||||
temp, err := os.CreateTemp("", "openflare-pages-*.zip")
|
temp, err := os.CreateTemp("", "openflare-pages-*."+safeTempSuffix(format))
|
||||||
if err != nil {
|
if err != nil {
|
||||||
return "", "", 0, err
|
return "", "", 0, "", err
|
||||||
}
|
}
|
||||||
defer func() { _ = temp.Close() }()
|
defer func() { _ = temp.Close() }()
|
||||||
hash := sha256.New()
|
hash := sha256.New()
|
||||||
limited := io.LimitReader(file, pagesMaxDeploymentBytes+1)
|
limited := io.LimitReader(file, maxPackageBytes+1)
|
||||||
written, err := io.Copy(io.MultiWriter(temp, hash), limited)
|
written, err := io.Copy(io.MultiWriter(temp, hash), limited)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
_ = os.Remove(temp.Name())
|
_ = os.Remove(temp.Name())
|
||||||
return "", "", 0, err
|
return "", "", 0, "", err
|
||||||
}
|
}
|
||||||
if written > pagesMaxDeploymentBytes {
|
if written > maxPackageBytes {
|
||||||
_ = os.Remove(temp.Name())
|
_ = os.Remove(temp.Name())
|
||||||
return "", "", 0, fmt.Errorf("pages 部署包不能超过 %d MiB", pagesMaxDeploymentBytes/bytesPerKiB/bytesPerKiB)
|
return "", "", 0, "", fmt.Errorf("pages 部署包不能超过 %d MiB", maxPackageBytes/bytesPerMiB)
|
||||||
|
}
|
||||||
|
return temp.Name(), hex.EncodeToString(hash.Sum(nil)), written, format, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func safeTempSuffix(format pagesarchive.Format) string {
|
||||||
|
switch format {
|
||||||
|
case pagesarchive.FormatTarGz:
|
||||||
|
return "tar.gz"
|
||||||
|
case pagesarchive.FormatTarXz:
|
||||||
|
return "tar.xz"
|
||||||
|
case pagesarchive.FormatTarBz2:
|
||||||
|
return "tar.bz2"
|
||||||
|
case pagesarchive.FormatSevenZip:
|
||||||
|
return "7z"
|
||||||
|
case pagesarchive.FormatTar:
|
||||||
|
return "tar"
|
||||||
|
default:
|
||||||
|
return "zip"
|
||||||
}
|
}
|
||||||
return temp.Name(), hex.EncodeToString(hash.Sum(nil)), written, nil
|
|
||||||
}
|
}
|
||||||
|
|
||||||
func pagesLegacyRelativeCandidates(project *model.PagesProject, deployment *model.PagesDeployment) []string {
|
func pagesLegacyRelativeCandidates(project *model.PagesProject, deployment *model.PagesDeployment) []string {
|
||||||
@@ -185,6 +249,7 @@ func pagesLegacyRelativeCandidates(project *model.PagesProject, deployment *mode
|
|||||||
if slug == "" || checksum == "" {
|
if slug == "" || checksum == "" {
|
||||||
return nil
|
return nil
|
||||||
}
|
}
|
||||||
|
// Legacy artifacts were always stored as .zip.
|
||||||
fileName := checksum + ".zip"
|
fileName := checksum + ".zip"
|
||||||
return []string{
|
return []string{
|
||||||
filepath.Join("artifacts", slug, fileName),
|
filepath.Join("artifacts", slug, fileName),
|
||||||
@@ -199,14 +264,16 @@ func ingestPagesDeploymentPackage(
|
|||||||
checksum string,
|
checksum string,
|
||||||
projectSlug string,
|
projectSlug string,
|
||||||
fileName string,
|
fileName string,
|
||||||
|
format pagesarchive.Format,
|
||||||
) (upload.IngestResult, error) {
|
) (upload.IngestResult, error) {
|
||||||
systemUser := repository.GetSystemUser(ctx)
|
systemUser := repository.GetSystemUser(ctx)
|
||||||
accessMode := 0
|
accessMode := 0
|
||||||
|
extension := pagesarchive.NormalizeNameExtension(fileName, format)
|
||||||
return upload.IngestFromLocalPath(ctx, localPath, upload.IngestRequest{
|
return upload.IngestFromLocalPath(ctx, localPath, upload.IngestRequest{
|
||||||
UserID: systemUser.ID,
|
UserID: systemUser.ID,
|
||||||
FileName: fileName,
|
FileName: fileName,
|
||||||
MimeType: mimeTypeApplicationZip,
|
MimeType: pagesarchive.MIMEType(format),
|
||||||
Extension: "zip",
|
Extension: extension,
|
||||||
Hash: checksum,
|
Hash: checksum,
|
||||||
Type: pagesDeploymentUploadType,
|
Type: pagesDeploymentUploadType,
|
||||||
AccessMode: &accessMode,
|
AccessMode: &accessMode,
|
||||||
@@ -215,6 +282,7 @@ func ingestPagesDeploymentPackage(
|
|||||||
Metadata: model.UploadMetadata{
|
Metadata: model.UploadMetadata{
|
||||||
Extra: map[string]any{
|
Extra: map[string]any{
|
||||||
"project_slug": projectSlug,
|
"project_slug": projectSlug,
|
||||||
|
"format": string(format),
|
||||||
},
|
},
|
||||||
},
|
},
|
||||||
})
|
})
|
||||||
@@ -224,167 +292,96 @@ func removeDeploymentArtifact(ctx context.Context, deployment *model.PagesDeploy
|
|||||||
if deployment == nil {
|
if deployment == nil {
|
||||||
return
|
return
|
||||||
}
|
}
|
||||||
if deployment.UploadID > 0 {
|
if deployment.UploadID == 0 {
|
||||||
_, _ = upload.Remove(ctx, deployment.UploadID)
|
return
|
||||||
|
}
|
||||||
|
if _, err := upload.Remove(ctx, deployment.UploadID); err != nil {
|
||||||
|
// Soft-delete / storage cleanup failure must not undo DB prune; log for ops.
|
||||||
|
logger.WarnF(ctx,
|
||||||
|
"[Pages] remove deployment artifact failed: deployment_id=%d upload_id=%d error=%v",
|
||||||
|
deployment.ID, deployment.UploadID, err,
|
||||||
|
)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
func findCommonRootPrefix(files []*zip.File) (string, error) {
|
func inspectPagesPackage(packagePath string, format pagesarchive.Format, rootDir string, entryFile string, limits pagesLimits) (*deploymentManifest, error) {
|
||||||
var firstFilePath string
|
archiveManifest, err := pagesarchive.InspectFile(packagePath, format, pagesarchive.InspectOptions{
|
||||||
hasMultipleFiles := false
|
RootDir: rootDir,
|
||||||
for _, item := range files {
|
|
||||||
normalizedPath, skip, err := normalizePagesZipPath(item.Name)
|
|
||||||
if err != nil {
|
|
||||||
return "", err
|
|
||||||
}
|
|
||||||
if skip {
|
|
||||||
continue
|
|
||||||
}
|
|
||||||
if firstFilePath == "" {
|
|
||||||
firstFilePath = normalizedPath
|
|
||||||
} else {
|
|
||||||
hasMultipleFiles = true
|
|
||||||
}
|
|
||||||
}
|
|
||||||
if firstFilePath == "" {
|
|
||||||
return "", nil
|
|
||||||
}
|
|
||||||
parts := strings.Split(firstFilePath, "/")
|
|
||||||
if len(parts) <= 1 {
|
|
||||||
return "", nil
|
|
||||||
}
|
|
||||||
commonPrefix := parts[0] + "/"
|
|
||||||
if hasMultipleFiles {
|
|
||||||
for _, item := range files {
|
|
||||||
normalizedPath, skip, err := normalizePagesZipPath(item.Name)
|
|
||||||
if err != nil {
|
|
||||||
return "", err
|
|
||||||
}
|
|
||||||
if skip {
|
|
||||||
continue
|
|
||||||
}
|
|
||||||
if !strings.HasPrefix(normalizedPath, commonPrefix) {
|
|
||||||
return "", nil
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
return commonPrefix, nil
|
|
||||||
}
|
|
||||||
|
|
||||||
func inspectPagesZip(zipPath string, rootDir string, entryFile string) (*deploymentManifest, error) {
|
|
||||||
reader, err := zip.OpenReader(zipPath)
|
|
||||||
if err != nil {
|
|
||||||
return nil, errors.New(errPagesPackageInvalidZip)
|
|
||||||
}
|
|
||||||
defer func() { _ = reader.Close() }()
|
|
||||||
|
|
||||||
commonPrefix, err := findCommonRootPrefix(reader.File)
|
|
||||||
if err != nil {
|
|
||||||
return nil, err
|
|
||||||
}
|
|
||||||
|
|
||||||
manifest := &deploymentManifest{
|
|
||||||
Files: []model.PagesDeploymentFile{},
|
|
||||||
EntryFile: entryFile,
|
EntryFile: entryFile,
|
||||||
|
Limits: pagesarchive.Limits{
|
||||||
|
MaxFiles: limits.MaxFiles,
|
||||||
|
MaxFileBytes: limits.ExtractedBytes,
|
||||||
|
MaxTotalBytes: limits.ExtractedBytes,
|
||||||
|
},
|
||||||
|
})
|
||||||
|
if err != nil {
|
||||||
|
return nil, mapPagesArchiveError(err)
|
||||||
}
|
}
|
||||||
targetEntryPath := entryFile
|
manifest := &deploymentManifest{
|
||||||
if rootDir != "" {
|
Files: make([]model.PagesDeploymentFile, 0, len(archiveManifest.Files)),
|
||||||
targetEntryPath = path.Join(rootDir, entryFile)
|
FileCount: archiveManifest.FileCount,
|
||||||
|
TotalSize: archiveManifest.TotalSize,
|
||||||
|
EntryFile: entryFile,
|
||||||
|
Format: format,
|
||||||
}
|
}
|
||||||
entrySeen := false
|
for _, file := range archiveManifest.Files {
|
||||||
for _, item := range reader.File {
|
|
||||||
normalizedPath, skip, err := normalizePagesZipPath(item.Name)
|
|
||||||
if err != nil {
|
|
||||||
return nil, err
|
|
||||||
}
|
|
||||||
if skip {
|
|
||||||
continue
|
|
||||||
}
|
|
||||||
if commonPrefix != "" {
|
|
||||||
normalizedPath = strings.TrimPrefix(normalizedPath, commonPrefix)
|
|
||||||
}
|
|
||||||
if item.FileInfo().Mode()&os.ModeSymlink != 0 {
|
|
||||||
return nil, fmt.Errorf("pages 部署包不支持符号链接: %s", normalizedPath)
|
|
||||||
}
|
|
||||||
if item.UncompressedSize64 > pagesMaxDeploymentBytes {
|
|
||||||
return nil, fmt.Errorf("pages 文件过大: %s", normalizedPath)
|
|
||||||
}
|
|
||||||
manifest.FileCount++
|
|
||||||
if manifest.FileCount > pagesMaxDeploymentFiles {
|
|
||||||
return nil, fmt.Errorf("pages 部署文件数不能超过 %d", pagesMaxDeploymentFiles)
|
|
||||||
}
|
|
||||||
checksum, fileSize, err := checksumZipFile(item)
|
|
||||||
if err != nil {
|
|
||||||
return nil, fmt.Errorf("%s: %w", normalizedPath, err)
|
|
||||||
}
|
|
||||||
manifest.TotalSize += fileSize
|
|
||||||
if manifest.TotalSize > pagesMaxDeploymentBytes {
|
|
||||||
return nil, fmt.Errorf("pages 部署展开后不能超过 %d MiB", pagesMaxDeploymentBytes/bytesPerKiB/bytesPerKiB)
|
|
||||||
}
|
|
||||||
if normalizedPath == targetEntryPath {
|
|
||||||
entrySeen = true
|
|
||||||
}
|
|
||||||
manifest.Files = append(manifest.Files, model.PagesDeploymentFile{
|
manifest.Files = append(manifest.Files, model.PagesDeploymentFile{
|
||||||
Path: normalizedPath,
|
Path: file.Path,
|
||||||
Size: fileSize,
|
Size: file.Size,
|
||||||
Checksum: checksum,
|
Checksum: file.Checksum,
|
||||||
})
|
})
|
||||||
}
|
}
|
||||||
if manifest.FileCount == 0 {
|
|
||||||
return nil, errors.New(errPagesPackageEmpty)
|
|
||||||
}
|
|
||||||
if !entrySeen {
|
|
||||||
return nil, fmt.Errorf("pages 部署包缺少入口文件 %s", targetEntryPath)
|
|
||||||
}
|
|
||||||
return manifest, nil
|
return manifest, nil
|
||||||
}
|
}
|
||||||
|
|
||||||
func normalizePagesZipPath(raw string) (string, bool, error) {
|
func mapPagesArchiveError(err error) error {
|
||||||
name := strings.TrimSpace(filepath.ToSlash(raw))
|
if err == nil {
|
||||||
if name == "" {
|
return nil
|
||||||
return "", true, nil
|
|
||||||
}
|
}
|
||||||
if strings.HasSuffix(name, "/") {
|
message := err.Error()
|
||||||
return "", true, nil
|
switch {
|
||||||
|
case strings.Contains(message, "unsupported pages package format"):
|
||||||
|
return errors.New(errPagesPackageUnsupported)
|
||||||
|
case strings.Contains(message, "open zip"), strings.Contains(message, "open gzip"),
|
||||||
|
strings.Contains(message, "open xz"), strings.Contains(message, "open 7z"),
|
||||||
|
strings.Contains(message, "read tar"):
|
||||||
|
return errors.New(errPagesPackageInvalid)
|
||||||
|
case strings.Contains(message, "empty"):
|
||||||
|
return errors.New(errPagesPackageEmpty)
|
||||||
|
case strings.Contains(message, "missing entry file"):
|
||||||
|
return err
|
||||||
|
case strings.Contains(message, "file count exceeds"):
|
||||||
|
return fmt.Errorf("pages 部署文件数不能超过 %d", pagesMaxDeploymentFiles)
|
||||||
|
case strings.Contains(message, "extracted size exceeds"):
|
||||||
|
return errors.New(errPagesPackageExtractedTooLarge)
|
||||||
|
case strings.Contains(message, "file too large"), strings.Contains(message, "size out of bounds"):
|
||||||
|
return errors.New(errPagesPackageFileTooLarge)
|
||||||
|
case strings.Contains(message, "symlink"):
|
||||||
|
return err
|
||||||
|
case strings.Contains(message, "absolute path"), strings.Contains(message, "escapes directory"):
|
||||||
|
return err
|
||||||
|
default:
|
||||||
|
return err
|
||||||
}
|
}
|
||||||
if strings.HasPrefix(name, "/") || path.IsAbs(name) {
|
|
||||||
return "", false, fmt.Errorf("pages 部署包不能包含绝对路径: %s", raw)
|
|
||||||
}
|
|
||||||
cleaned := path.Clean(name)
|
|
||||||
if cleaned == "." {
|
|
||||||
return "", true, nil
|
|
||||||
}
|
|
||||||
if cleaned == ".." || strings.HasPrefix(cleaned, "../") || strings.Contains(cleaned, "/../") {
|
|
||||||
return "", false, fmt.Errorf("pages 部署包路径不能逃逸目录: %s", raw)
|
|
||||||
}
|
|
||||||
return cleaned, false, nil
|
|
||||||
}
|
}
|
||||||
|
|
||||||
func copyPagesZipEntryContent(dst io.Writer, src io.Reader, declaredSize uint64) (int64, error) {
|
func packageDownloadName(deploymentID uint, fileName string, contentType string) string {
|
||||||
if declaredSize > pagesMaxDeploymentBytes || declaredSize > uint64(math.MaxInt64) {
|
if format, ok := pagesarchive.DetectFormatFromName(fileName); ok {
|
||||||
return 0, errors.New("pages file size out of bounds")
|
return fmt.Sprintf("pages-deployment-%d.%s", deploymentID, pagesarchive.Extension(format))
|
||||||
}
|
}
|
||||||
if declaredSize > 0 {
|
// Fall back by content type.
|
||||||
return io.CopyN(dst, src, int64(declaredSize)) //nolint:gosec // declaredSize is bounded to math.MaxInt64 above
|
switch strings.ToLower(strings.TrimSpace(contentType)) {
|
||||||
|
case "application/gzip", "application/x-gzip":
|
||||||
|
return fmt.Sprintf("pages-deployment-%d.tar.gz", deploymentID)
|
||||||
|
case "application/x-xz":
|
||||||
|
return fmt.Sprintf("pages-deployment-%d.tar.xz", deploymentID)
|
||||||
|
case "application/x-bzip2":
|
||||||
|
return fmt.Sprintf("pages-deployment-%d.tar.bz2", deploymentID)
|
||||||
|
case "application/x-7z-compressed":
|
||||||
|
return fmt.Sprintf("pages-deployment-%d.7z", deploymentID)
|
||||||
|
case "application/x-tar":
|
||||||
|
return fmt.Sprintf("pages-deployment-%d.tar", deploymentID)
|
||||||
|
default:
|
||||||
|
return fmt.Sprintf("pages-deployment-%d.zip", deploymentID)
|
||||||
}
|
}
|
||||||
limited := io.LimitReader(src, pagesMaxDeploymentBytes+1)
|
|
||||||
written, err := io.Copy(dst, limited)
|
|
||||||
if written > pagesMaxDeploymentBytes {
|
|
||||||
return written, errors.New("pages file size out of bounds")
|
|
||||||
}
|
|
||||||
return written, err
|
|
||||||
}
|
|
||||||
|
|
||||||
func checksumZipFile(item *zip.File) (string, int64, error) {
|
|
||||||
file, err := item.Open()
|
|
||||||
if err != nil {
|
|
||||||
return "", 0, err
|
|
||||||
}
|
|
||||||
defer func() { _ = file.Close() }()
|
|
||||||
hash := sha256.New()
|
|
||||||
written, err := copyPagesZipEntryContent(hash, file, item.UncompressedSize64)
|
|
||||||
if err != nil {
|
|
||||||
return "", written, err
|
|
||||||
}
|
|
||||||
return hex.EncodeToString(hash.Sum(nil)), written, nil
|
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -12,13 +12,14 @@ import (
|
|||||||
"mime/multipart"
|
"mime/multipart"
|
||||||
"net/url"
|
"net/url"
|
||||||
"os"
|
"os"
|
||||||
"path/filepath"
|
|
||||||
"strings"
|
"strings"
|
||||||
"time"
|
"time"
|
||||||
|
|
||||||
"github.com/Rain-kl/Wavelet/internal/apps/upload"
|
"github.com/Rain-kl/Wavelet/internal/apps/upload"
|
||||||
"github.com/Rain-kl/Wavelet/internal/db"
|
"github.com/Rain-kl/Wavelet/internal/db"
|
||||||
"github.com/Rain-kl/Wavelet/internal/model"
|
"github.com/Rain-kl/Wavelet/internal/model"
|
||||||
|
"github.com/Rain-kl/Wavelet/pkg/logger"
|
||||||
|
"github.com/Rain-kl/Wavelet/pkg/pagesarchive"
|
||||||
"gorm.io/gorm"
|
"gorm.io/gorm"
|
||||||
)
|
)
|
||||||
|
|
||||||
@@ -251,20 +252,18 @@ func UploadDeployment(ctx context.Context, projectID uint, fileHeader *multipart
|
|||||||
if fileHeader == nil {
|
if fileHeader == nil {
|
||||||
return nil, errors.New(errPagesPackageMissing)
|
return nil, errors.New(errPagesPackageMissing)
|
||||||
}
|
}
|
||||||
if !strings.EqualFold(filepath.Ext(fileHeader.Filename), ".zip") {
|
limits := resolvePagesLimits(ctx)
|
||||||
return nil, errors.New(errPagesPackageNotZip)
|
|
||||||
}
|
|
||||||
rootDir, err := validateAndNormalizePagesRootDir(project.RootDir)
|
rootDir, err := validateAndNormalizePagesRootDir(project.RootDir)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
return nil, err
|
return nil, err
|
||||||
}
|
}
|
||||||
entryFile := normalizePagesEntryFile(project.EntryFile)
|
entryFile := normalizePagesEntryFile(project.EntryFile)
|
||||||
tempPath, checksum, _, err := persistPagesUploadTemp(fileHeader)
|
tempPath, checksum, _, format, err := persistPagesUploadTemp(fileHeader, limits.PackageBytes)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
return nil, err
|
return nil, err
|
||||||
}
|
}
|
||||||
defer func() { _ = os.Remove(tempPath) }()
|
defer func() { _ = os.Remove(tempPath) }()
|
||||||
manifest, err := inspectPagesZip(tempPath, rootDir, entryFile)
|
manifest, err := inspectPagesPackage(tempPath, format, rootDir, entryFile, limits)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
return nil, err
|
return nil, err
|
||||||
}
|
}
|
||||||
@@ -274,6 +273,7 @@ func UploadDeployment(ctx context.Context, projectID uint, fileHeader *multipart
|
|||||||
checksum,
|
checksum,
|
||||||
project.Slug,
|
project.Slug,
|
||||||
fileHeader.Filename,
|
fileHeader.Filename,
|
||||||
|
format,
|
||||||
)
|
)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
return nil, err
|
return nil, err
|
||||||
@@ -320,10 +320,160 @@ func UploadDeployment(ctx context.Context, projectID uint, fileHeader *multipart
|
|||||||
return nil, err
|
return nil, err
|
||||||
}
|
}
|
||||||
ingestCommitted = true
|
ingestCommitted = true
|
||||||
|
|
||||||
|
// History prune must not fail the already-committed upload. Log and continue;
|
||||||
|
// a later upload (or a retry pass inside prune) will re-attempt cleanup.
|
||||||
|
if pruneErr := pruneProjectDeploymentHistory(ctx, project.ID, limits.HistoryCount); pruneErr != nil {
|
||||||
|
logger.ErrorF(ctx,
|
||||||
|
"[Pages] prune deployment history failed: project_id=%d keep=%d error=%v",
|
||||||
|
project.ID, limits.HistoryCount, pruneErr,
|
||||||
|
)
|
||||||
|
}
|
||||||
|
|
||||||
view := buildDeploymentView(deployment)
|
view := buildDeploymentView(deployment)
|
||||||
return &view, nil
|
return &view, nil
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// pruneProjectDeploymentHistory enforces the retention policy for one project.
|
||||||
|
//
|
||||||
|
// Policy (keepCount > 0):
|
||||||
|
// - At most keepCount deployment rows remain for the project.
|
||||||
|
// - The current active deployment is always retained (if any).
|
||||||
|
// - Remaining slots are filled by newest deployments first (id desc).
|
||||||
|
// - All other non-kept deployments are deleted with their file lists and artifacts.
|
||||||
|
//
|
||||||
|
// keepCount <= 0 means unlimited history.
|
||||||
|
//
|
||||||
|
// Concurrency: DB row deletes run in a single transaction after a consistent read
|
||||||
|
// of project + deployments. Concurrent uploads may briefly exceed keepCount; the
|
||||||
|
// next successful prune brings the project back within the limit (eventual).
|
||||||
|
func pruneProjectDeploymentHistory(ctx context.Context, projectID uint, keepCount int) error {
|
||||||
|
if keepCount <= 0 {
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// Two passes: first pass after upload, second pass heals a concurrent race
|
||||||
|
// that inserted another deployment between our list and delete.
|
||||||
|
var lastErr error
|
||||||
|
for pass := 0; pass < 2; pass++ {
|
||||||
|
deleted, err := pruneProjectDeploymentHistoryOnce(ctx, projectID, keepCount)
|
||||||
|
if err != nil {
|
||||||
|
lastErr = err
|
||||||
|
break
|
||||||
|
}
|
||||||
|
if deleted == 0 {
|
||||||
|
break
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return lastErr
|
||||||
|
}
|
||||||
|
|
||||||
|
// pruneProjectDeploymentHistoryOnce performs one list → select → delete cycle.
|
||||||
|
// Returns the number of deployments deleted from the database.
|
||||||
|
func pruneProjectDeploymentHistoryOnce(ctx context.Context, projectID uint, keepCount int) (int, error) {
|
||||||
|
project, err := model.GetPagesProjectByID(ctx, projectID)
|
||||||
|
if err != nil {
|
||||||
|
return 0, fmt.Errorf("load pages project: %w", err)
|
||||||
|
}
|
||||||
|
deployments, err := model.ListPagesDeployments(ctx, projectID)
|
||||||
|
if err != nil {
|
||||||
|
return 0, fmt.Errorf("list pages deployments: %w", err)
|
||||||
|
}
|
||||||
|
if len(deployments) <= keepCount {
|
||||||
|
return 0, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
var activeID uint
|
||||||
|
if project.ActiveDeploymentID != nil {
|
||||||
|
activeID = *project.ActiveDeploymentID
|
||||||
|
}
|
||||||
|
toDelete := selectDeploymentsToPrune(deployments, activeID, keepCount)
|
||||||
|
if len(toDelete) == 0 {
|
||||||
|
return 0, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// Delete metadata in one transaction so partial prune does not leave
|
||||||
|
// orphan file-list rows without a parent deployment.
|
||||||
|
if err := db.DB(ctx).Transaction(func(tx *gorm.DB) error {
|
||||||
|
for index := range toDelete {
|
||||||
|
deployment := toDelete[index]
|
||||||
|
// Never delete the active deployment even if project pointer raced.
|
||||||
|
if activeID != 0 && deployment.ID == activeID {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
if project.ActiveDeploymentID != nil && deployment.ID == *project.ActiveDeploymentID {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
if err := tx.Where("deployment_id = ?", deployment.ID).Delete(&model.PagesDeploymentFile{}).Error; err != nil {
|
||||||
|
return fmt.Errorf("delete deployment files id=%d: %w", deployment.ID, err)
|
||||||
|
}
|
||||||
|
if err := tx.Where("id = ? AND project_id = ?", deployment.ID, projectID).
|
||||||
|
Delete(&model.PagesDeployment{}).Error; err != nil {
|
||||||
|
return fmt.Errorf("delete deployment id=%d: %w", deployment.ID, err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return nil
|
||||||
|
}); err != nil {
|
||||||
|
return 0, err
|
||||||
|
}
|
||||||
|
|
||||||
|
// Artifacts are best-effort outside the transaction (object storage I/O).
|
||||||
|
for index := range toDelete {
|
||||||
|
deployment := toDelete[index]
|
||||||
|
if activeID != 0 && deployment.ID == activeID {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
removeDeploymentArtifact(ctx, &deployment)
|
||||||
|
}
|
||||||
|
|
||||||
|
logger.InfoF(ctx,
|
||||||
|
"[Pages] pruned deployment history: project_id=%d keep=%d deleted=%d",
|
||||||
|
projectID, keepCount, len(toDelete),
|
||||||
|
)
|
||||||
|
return len(toDelete), nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// selectDeploymentsToPrune returns deployments that should be removed under the
|
||||||
|
// "at most keepCount, always keep active, fill with newest" policy.
|
||||||
|
// deployments must be ordered newest-first (id desc).
|
||||||
|
func selectDeploymentsToPrune(deployments []model.PagesDeployment, activeID uint, keepCount int) []model.PagesDeployment {
|
||||||
|
if keepCount <= 0 || len(deployments) <= keepCount {
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
|
||||||
|
keepIDs := make(map[uint]struct{}, keepCount)
|
||||||
|
// 1) Active is always retained and occupies one slot when present.
|
||||||
|
if activeID != 0 {
|
||||||
|
// Only count active if it still exists in the list.
|
||||||
|
for _, deployment := range deployments {
|
||||||
|
if deployment.ID == activeID {
|
||||||
|
keepIDs[activeID] = struct{}{}
|
||||||
|
break
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
// 2) Fill remaining slots from newest to oldest.
|
||||||
|
for _, deployment := range deployments {
|
||||||
|
if len(keepIDs) >= keepCount {
|
||||||
|
break
|
||||||
|
}
|
||||||
|
keepIDs[deployment.ID] = struct{}{}
|
||||||
|
}
|
||||||
|
|
||||||
|
toDelete := make([]model.PagesDeployment, 0, len(deployments)-len(keepIDs))
|
||||||
|
for _, deployment := range deployments {
|
||||||
|
if _, keep := keepIDs[deployment.ID]; keep {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
// Safety: never mark active for deletion.
|
||||||
|
if activeID != 0 && deployment.ID == activeID {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
toDelete = append(toDelete, deployment)
|
||||||
|
}
|
||||||
|
return toDelete
|
||||||
|
}
|
||||||
|
|
||||||
// ActivateDeployment 激活 Pages 部署。
|
// ActivateDeployment 激活 Pages 部署。
|
||||||
func ActivateDeployment(ctx context.Context, projectID uint, deploymentID uint) (*View, error) {
|
func ActivateDeployment(ctx context.Context, projectID uint, deploymentID uint) (*View, error) {
|
||||||
project, err := model.GetPagesProjectByID(ctx, projectID)
|
project, err := model.GetPagesProjectByID(ctx, projectID)
|
||||||
@@ -422,10 +572,14 @@ func openDeploymentPackageFromUpload(ctx context.Context, uploadID uint64, deplo
|
|||||||
}
|
}
|
||||||
contentType := opened.ContentType
|
contentType := opened.ContentType
|
||||||
if contentType == "" {
|
if contentType == "" {
|
||||||
contentType = mimeTypeApplicationZip
|
contentType = opened.Upload.MimeType
|
||||||
}
|
}
|
||||||
|
if contentType == "" {
|
||||||
|
contentType = "application/octet-stream"
|
||||||
|
}
|
||||||
|
fileName := packageDownloadName(deploymentID, opened.Upload.FileName, contentType)
|
||||||
return DeploymentPackage{
|
return DeploymentPackage{
|
||||||
FileName: fmt.Sprintf("pages-deployment-%d.zip", deploymentID),
|
FileName: fileName,
|
||||||
ContentType: contentType,
|
ContentType: contentType,
|
||||||
ContentLength: opened.ContentLength,
|
ContentLength: opened.ContentLength,
|
||||||
Body: opened.Body,
|
Body: opened.Body,
|
||||||
@@ -478,6 +632,7 @@ func hydrateLegacyDeploymentUpload(
|
|||||||
deployment.Checksum,
|
deployment.Checksum,
|
||||||
project.Slug,
|
project.Slug,
|
||||||
fmt.Sprintf("pages-deployment-%d.zip", deployment.ID),
|
fmt.Sprintf("pages-deployment-%d.zip", deployment.ID),
|
||||||
|
pagesarchive.FormatZip,
|
||||||
)
|
)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
return nil, err
|
return nil, err
|
||||||
@@ -493,7 +648,26 @@ func hydrateLegacyDeploymentUpload(
|
|||||||
return &ingestResult.Upload, nil
|
return &ingestResult.Upload, nil
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// ensureDeploymentInActiveSnapshot allows Agent package download when the
|
||||||
|
// deployment is the project's current active deployment and that project is
|
||||||
|
// used by at least one pages route in the active main config.
|
||||||
|
//
|
||||||
|
// Main config versions pin historical pages_deployment ids for audit only.
|
||||||
|
// Runtime download always follows the live active Pages deployment (dual
|
||||||
|
// version control); rolling back main config must not require old packages.
|
||||||
func ensureDeploymentInActiveSnapshot(ctx context.Context, deploymentID uint) error {
|
func ensureDeploymentInActiveSnapshot(ctx context.Context, deploymentID uint) error {
|
||||||
|
deployment, err := model.GetPagesDeploymentByID(ctx, deploymentID)
|
||||||
|
if err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
project, err := model.GetPagesProjectByID(ctx, deployment.ProjectID)
|
||||||
|
if err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
if project.ActiveDeploymentID == nil || *project.ActiveDeploymentID != deployment.ID {
|
||||||
|
return errors.New(errPagesPackageNotInActiveConfig)
|
||||||
|
}
|
||||||
|
|
||||||
version, err := model.GetActiveConfigVersion(ctx)
|
version, err := model.GetActiveConfigVersion(ctx)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
if errors.Is(err, gorm.ErrRecordNotFound) {
|
if errors.Is(err, gorm.ErrRecordNotFound) {
|
||||||
@@ -506,10 +680,30 @@ func ensureDeploymentInActiveSnapshot(ctx context.Context, deploymentID uint) er
|
|||||||
return err
|
return err
|
||||||
}
|
}
|
||||||
for _, route := range routes {
|
for _, route := range routes {
|
||||||
if route.UpstreamType != "pages" || route.PagesDeployment == nil {
|
if !strings.EqualFold(strings.TrimSpace(route.UpstreamType), "pages") {
|
||||||
continue
|
continue
|
||||||
}
|
}
|
||||||
if route.PagesDeployment.DeploymentID == deploymentID {
|
if route.PagesProjectID != nil && *route.PagesProjectID == project.ID {
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
if route.PagesDeployment == nil {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
if route.PagesDeployment.ProjectID == project.ID {
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
// Frozen snapshot may only carry deployment_id; resolve project via that row.
|
||||||
|
if route.PagesDeployment.DeploymentID == 0 {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
if route.PagesDeployment.DeploymentID == deployment.ID {
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
snapDeployment, snapErr := model.GetPagesDeploymentByID(ctx, route.PagesDeployment.DeploymentID)
|
||||||
|
if snapErr != nil {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
if snapDeployment.ProjectID == project.ID {
|
||||||
return nil
|
return nil
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
@@ -518,10 +712,12 @@ func ensureDeploymentInActiveSnapshot(ctx context.Context, deploymentID uint) er
|
|||||||
|
|
||||||
type snapshotPagesDeployment struct {
|
type snapshotPagesDeployment struct {
|
||||||
DeploymentID uint `json:"deployment_id"`
|
DeploymentID uint `json:"deployment_id"`
|
||||||
|
ProjectID uint `json:"project_id"`
|
||||||
}
|
}
|
||||||
|
|
||||||
type snapshotRouteRef struct {
|
type snapshotRouteRef struct {
|
||||||
UpstreamType string `json:"upstream_type"`
|
UpstreamType string `json:"upstream_type"`
|
||||||
|
PagesProjectID *uint `json:"pages_project_id"`
|
||||||
PagesDeployment *snapshotPagesDeployment `json:"pages_deployment"`
|
PagesDeployment *snapshotPagesDeployment `json:"pages_deployment"`
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|||||||
@@ -4,8 +4,10 @@
|
|||||||
package pages
|
package pages
|
||||||
|
|
||||||
import (
|
import (
|
||||||
|
"archive/tar"
|
||||||
"archive/zip"
|
"archive/zip"
|
||||||
"bytes"
|
"bytes"
|
||||||
|
"compress/gzip"
|
||||||
"context"
|
"context"
|
||||||
"fmt"
|
"fmt"
|
||||||
"io"
|
"io"
|
||||||
@@ -17,6 +19,7 @@ import (
|
|||||||
|
|
||||||
"github.com/Rain-kl/Wavelet/internal/db"
|
"github.com/Rain-kl/Wavelet/internal/db"
|
||||||
"github.com/Rain-kl/Wavelet/internal/model"
|
"github.com/Rain-kl/Wavelet/internal/model"
|
||||||
|
"github.com/Rain-kl/Wavelet/internal/repository"
|
||||||
"github.com/Rain-kl/Wavelet/internal/storage"
|
"github.com/Rain-kl/Wavelet/internal/storage"
|
||||||
"github.com/glebarez/sqlite"
|
"github.com/glebarez/sqlite"
|
||||||
"github.com/stretchr/testify/assert"
|
"github.com/stretchr/testify/assert"
|
||||||
@@ -40,6 +43,7 @@ func setupPagesTestDB(t *testing.T) func() {
|
|||||||
&model.PagesDeployment{},
|
&model.PagesDeployment{},
|
||||||
&model.PagesDeploymentFile{},
|
&model.PagesDeploymentFile{},
|
||||||
&model.ConfigVersion{},
|
&model.ConfigVersion{},
|
||||||
|
&model.SystemConfig{},
|
||||||
))
|
))
|
||||||
require.NoError(t, sqliteDB.Create(&model.User{
|
require.NoError(t, sqliteDB.Create(&model.User{
|
||||||
ID: 999,
|
ID: 999,
|
||||||
@@ -48,8 +52,25 @@ func setupPagesTestDB(t *testing.T) func() {
|
|||||||
Nickname: "系统",
|
Nickname: "系统",
|
||||||
IsActive: true,
|
IsActive: true,
|
||||||
}).Error)
|
}).Error)
|
||||||
|
require.NoError(t, sqliteDB.Create([]model.SystemConfig{
|
||||||
|
{
|
||||||
|
Key: model.ConfigKeyPagesMaxPackageSizeMB,
|
||||||
|
Value: "100",
|
||||||
|
Type: "business",
|
||||||
|
Description: "Pages 部署包上传大小上限(MiB)",
|
||||||
|
},
|
||||||
|
{
|
||||||
|
Key: model.ConfigKeyPagesMaxHistoryCount,
|
||||||
|
Value: "0", // unlimited for existing tests
|
||||||
|
Type: "business",
|
||||||
|
Description: "Pages 每个项目最大历史部署保留数(0 表示不限制)",
|
||||||
|
},
|
||||||
|
}).Error)
|
||||||
|
|
||||||
db.SetDB(sqliteDB)
|
db.SetDB(sqliteDB)
|
||||||
|
// Clear process-global system config RAM cache so tests do not see stale values.
|
||||||
|
_ = repository.InvalidateSystemConfigCache(context.Background(), model.ConfigKeyPagesMaxPackageSizeMB)
|
||||||
|
_ = repository.InvalidateSystemConfigCache(context.Background(), model.ConfigKeyPagesMaxHistoryCount)
|
||||||
return func() {
|
return func() {
|
||||||
db.SetDB(nil)
|
db.SetDB(nil)
|
||||||
}
|
}
|
||||||
@@ -339,6 +360,149 @@ func testPagesZip(t *testing.T, files map[string]string) []byte {
|
|||||||
return buffer.Bytes()
|
return buffer.Bytes()
|
||||||
}
|
}
|
||||||
|
|
||||||
|
func TestUploadDeploymentAcceptsTarGz(t *testing.T) {
|
||||||
|
cleanup := setupPagesTestDB(t)
|
||||||
|
defer cleanup()
|
||||||
|
_, disableStorage := setupPagesStorageMock(t)
|
||||||
|
defer disableStorage()
|
||||||
|
ctx := context.Background()
|
||||||
|
|
||||||
|
project, err := CreateProject(ctx, Input{
|
||||||
|
Name: "TarGz Site",
|
||||||
|
Slug: "tar-gz-site",
|
||||||
|
Enabled: true,
|
||||||
|
})
|
||||||
|
require.NoError(t, err)
|
||||||
|
|
||||||
|
deployment, err := UploadDeployment(ctx, project.ID, testPagesMultipartFile(t, "site.tar.gz", testPagesTarGz(t, map[string]string{
|
||||||
|
"index.html": "tar-ok",
|
||||||
|
"app.js": "1",
|
||||||
|
})), "root")
|
||||||
|
require.NoError(t, err)
|
||||||
|
assert.Equal(t, 2, deployment.FileCount)
|
||||||
|
assert.NotZero(t, deployment.UploadID)
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestSelectDeploymentsToPruneKeepsActiveAndNewest(t *testing.T) {
|
||||||
|
// ids 4(newest) ... 1(oldest); active is oldest id=1; keep=2 → keep {1,4}, prune {3,2}
|
||||||
|
deployments := []model.PagesDeployment{
|
||||||
|
{ID: 4, ProjectID: 1},
|
||||||
|
{ID: 3, ProjectID: 1},
|
||||||
|
{ID: 2, ProjectID: 1},
|
||||||
|
{ID: 1, ProjectID: 1},
|
||||||
|
}
|
||||||
|
toDelete := selectDeploymentsToPrune(deployments, 1, 2)
|
||||||
|
require.Len(t, toDelete, 2)
|
||||||
|
assert.Equal(t, uint(3), toDelete[0].ID)
|
||||||
|
assert.Equal(t, uint(2), toDelete[1].ID)
|
||||||
|
|
||||||
|
// active is newest; keep=2 → keep {4,3}, prune {2,1}
|
||||||
|
toDelete = selectDeploymentsToPrune(deployments, 4, 2)
|
||||||
|
require.Len(t, toDelete, 2)
|
||||||
|
assert.Equal(t, uint(2), toDelete[0].ID)
|
||||||
|
assert.Equal(t, uint(1), toDelete[1].ID)
|
||||||
|
|
||||||
|
// no active; keep=2 → keep {4,3}
|
||||||
|
toDelete = selectDeploymentsToPrune(deployments, 0, 2)
|
||||||
|
require.Len(t, toDelete, 2)
|
||||||
|
assert.Equal(t, uint(2), toDelete[0].ID)
|
||||||
|
assert.Equal(t, uint(1), toDelete[1].ID)
|
||||||
|
|
||||||
|
// keep=1 with active → only active, prune the rest
|
||||||
|
toDelete = selectDeploymentsToPrune(deployments, 2, 1)
|
||||||
|
require.Len(t, toDelete, 3)
|
||||||
|
for _, item := range toDelete {
|
||||||
|
assert.NotEqual(t, uint(2), item.ID)
|
||||||
|
}
|
||||||
|
|
||||||
|
// already within limit
|
||||||
|
assert.Nil(t, selectDeploymentsToPrune(deployments[:2], 4, 2))
|
||||||
|
// unlimited
|
||||||
|
assert.Nil(t, selectDeploymentsToPrune(deployments, 1, 0))
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestPruneProjectDeploymentHistory(t *testing.T) {
|
||||||
|
cleanup := setupPagesTestDB(t)
|
||||||
|
defer cleanup()
|
||||||
|
_, disableStorage := setupPagesStorageMock(t)
|
||||||
|
defer disableStorage()
|
||||||
|
ctx := context.Background()
|
||||||
|
|
||||||
|
require.NoError(t, db.DB(ctx).Model(&model.SystemConfig{}).
|
||||||
|
Where("key = ?", model.ConfigKeyPagesMaxHistoryCount).
|
||||||
|
Update("value", "2").Error)
|
||||||
|
require.NoError(t, repository.InvalidateSystemConfigCache(ctx, model.ConfigKeyPagesMaxHistoryCount))
|
||||||
|
|
||||||
|
project, err := CreateProject(ctx, Input{
|
||||||
|
Name: "History Site",
|
||||||
|
Slug: "history-site",
|
||||||
|
Enabled: true,
|
||||||
|
})
|
||||||
|
require.NoError(t, err)
|
||||||
|
|
||||||
|
var ids []uint
|
||||||
|
for i := 0; i < 3; i++ {
|
||||||
|
deployment, uploadErr := UploadDeployment(ctx, project.ID, testPagesMultipartFile(t, "site.zip", testPagesZip(t, map[string]string{
|
||||||
|
"index.html": fmt.Sprintf("v%d", i),
|
||||||
|
})), "root")
|
||||||
|
require.NoError(t, uploadErr)
|
||||||
|
ids = append(ids, deployment.ID)
|
||||||
|
}
|
||||||
|
// After 3 uploads with keep=2 and no active: only 2 newest remain.
|
||||||
|
deployments, err := model.ListPagesDeployments(ctx, project.ID)
|
||||||
|
require.NoError(t, err)
|
||||||
|
require.Len(t, deployments, 2)
|
||||||
|
assert.Equal(t, ids[2], deployments[0].ID)
|
||||||
|
assert.Equal(t, ids[1], deployments[1].ID)
|
||||||
|
|
||||||
|
// Activate the older of the remaining two, then upload again.
|
||||||
|
_, err = ActivateDeployment(ctx, project.ID, ids[1])
|
||||||
|
require.NoError(t, err)
|
||||||
|
|
||||||
|
latest, err := UploadDeployment(ctx, project.ID, testPagesMultipartFile(t, "site.zip", testPagesZip(t, map[string]string{
|
||||||
|
"index.html": "v-latest",
|
||||||
|
})), "root")
|
||||||
|
require.NoError(t, err)
|
||||||
|
|
||||||
|
deployments, err = model.ListPagesDeployments(ctx, project.ID)
|
||||||
|
require.NoError(t, err)
|
||||||
|
require.Len(t, deployments, 2, "must be at most N=2, not active+N newest")
|
||||||
|
|
||||||
|
storedProject, err := model.GetPagesProjectByID(ctx, project.ID)
|
||||||
|
require.NoError(t, err)
|
||||||
|
require.NotNil(t, storedProject.ActiveDeploymentID)
|
||||||
|
assert.Equal(t, ids[1], *storedProject.ActiveDeploymentID)
|
||||||
|
|
||||||
|
kept := map[uint]struct{}{}
|
||||||
|
for _, item := range deployments {
|
||||||
|
kept[item.ID] = struct{}{}
|
||||||
|
}
|
||||||
|
_, hasActive := kept[ids[1]]
|
||||||
|
_, hasLatest := kept[latest.ID]
|
||||||
|
assert.True(t, hasActive, "active deployment must be retained")
|
||||||
|
assert.True(t, hasLatest, "newest deployment must fill remaining slot")
|
||||||
|
}
|
||||||
|
|
||||||
|
func testPagesTarGz(t *testing.T, files map[string]string) []byte {
|
||||||
|
t.Helper()
|
||||||
|
|
||||||
|
var buffer bytes.Buffer
|
||||||
|
gzWriter := gzip.NewWriter(&buffer)
|
||||||
|
tarWriter := tar.NewWriter(gzWriter)
|
||||||
|
for name, content := range files {
|
||||||
|
require.NoError(t, tarWriter.WriteHeader(&tar.Header{
|
||||||
|
Name: name,
|
||||||
|
Mode: 0o644,
|
||||||
|
Size: int64(len(content)),
|
||||||
|
}))
|
||||||
|
_, err := tarWriter.Write([]byte(content))
|
||||||
|
require.NoError(t, err)
|
||||||
|
}
|
||||||
|
require.NoError(t, tarWriter.Close())
|
||||||
|
require.NoError(t, gzWriter.Close())
|
||||||
|
return buffer.Bytes()
|
||||||
|
}
|
||||||
|
|
||||||
func testPagesMultipartFile(t *testing.T, fileName string, content []byte) *multipart.FileHeader {
|
func testPagesMultipartFile(t *testing.T, fileName string, content []byte) *multipart.FileHeader {
|
||||||
t.Helper()
|
t.Helper()
|
||||||
|
|
||||||
|
|||||||
@@ -0,0 +1,236 @@
|
|||||||
|
// Copyright 2026 Arctel.net
|
||||||
|
// SPDX-License-Identifier: Apache-2.0
|
||||||
|
|
||||||
|
package pages
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
"encoding/json"
|
||||||
|
"fmt"
|
||||||
|
"strings"
|
||||||
|
|
||||||
|
"github.com/Rain-kl/Wavelet/internal/model"
|
||||||
|
openrestyrender "github.com/Rain-kl/Wavelet/pkg/render/openresty"
|
||||||
|
"gorm.io/gorm"
|
||||||
|
)
|
||||||
|
|
||||||
|
// RebindSnapshotPagesToCurrentActive rewrites pages_deployment fields so every
|
||||||
|
// pages route points at the project's current active deployment.
|
||||||
|
//
|
||||||
|
// Main config versions and Pages deployments are independent. Rolling back a
|
||||||
|
// main config version must not require old Pages packages; Agents always follow
|
||||||
|
// the live active deployment for each referenced project.
|
||||||
|
//
|
||||||
|
// Returns the original JSON unchanged when there are no pages routes. Does not
|
||||||
|
// mutate stored config_versions rows. Non-pages route fields are preserved.
|
||||||
|
func RebindSnapshotPagesToCurrentActive(ctx context.Context, snapshotJSON string) (string, error) {
|
||||||
|
text := strings.TrimSpace(snapshotJSON)
|
||||||
|
if text == "" {
|
||||||
|
return snapshotJSON, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
if strings.HasPrefix(text, "[") {
|
||||||
|
var routes []map[string]json.RawMessage
|
||||||
|
if err := json.Unmarshal([]byte(text), &routes); err != nil {
|
||||||
|
return "", fmt.Errorf("parse pages snapshot routes: %w", err)
|
||||||
|
}
|
||||||
|
changed, err := rebindPagesRouteMaps(ctx, routes)
|
||||||
|
if err != nil {
|
||||||
|
return "", err
|
||||||
|
}
|
||||||
|
if !changed {
|
||||||
|
return snapshotJSON, nil
|
||||||
|
}
|
||||||
|
encoded, err := json.Marshal(routes)
|
||||||
|
if err != nil {
|
||||||
|
return "", err
|
||||||
|
}
|
||||||
|
return string(encoded), nil
|
||||||
|
}
|
||||||
|
|
||||||
|
var raw map[string]json.RawMessage
|
||||||
|
if err := json.Unmarshal([]byte(text), &raw); err != nil {
|
||||||
|
return "", fmt.Errorf("parse pages snapshot document: %w", err)
|
||||||
|
}
|
||||||
|
routesRaw, ok := raw["routes"]
|
||||||
|
if !ok || len(routesRaw) == 0 {
|
||||||
|
return snapshotJSON, nil
|
||||||
|
}
|
||||||
|
var routes []map[string]json.RawMessage
|
||||||
|
if err := json.Unmarshal(routesRaw, &routes); err != nil {
|
||||||
|
return "", fmt.Errorf("parse pages snapshot routes: %w", err)
|
||||||
|
}
|
||||||
|
changed, err := rebindPagesRouteMaps(ctx, routes)
|
||||||
|
if err != nil {
|
||||||
|
return "", err
|
||||||
|
}
|
||||||
|
if !changed {
|
||||||
|
return snapshotJSON, nil
|
||||||
|
}
|
||||||
|
encodedRoutes, err := json.Marshal(routes)
|
||||||
|
if err != nil {
|
||||||
|
return "", err
|
||||||
|
}
|
||||||
|
raw["routes"] = encodedRoutes
|
||||||
|
encoded, err := json.Marshal(raw)
|
||||||
|
if err != nil {
|
||||||
|
return "", err
|
||||||
|
}
|
||||||
|
return string(encoded), nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func rebindPagesRouteMaps(ctx context.Context, routes []map[string]json.RawMessage) (bool, error) {
|
||||||
|
changed := false
|
||||||
|
for index := range routes {
|
||||||
|
route := routes[index]
|
||||||
|
if route == nil {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
upstreamType, _ := rawJSONString(route["upstream_type"])
|
||||||
|
if !strings.EqualFold(strings.TrimSpace(upstreamType), "pages") {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
siteName, _ := rawJSONString(route["site_name"])
|
||||||
|
projectID, err := resolveProjectIDFromRouteMap(route)
|
||||||
|
if err != nil {
|
||||||
|
if siteName == "" {
|
||||||
|
siteName = "pages"
|
||||||
|
}
|
||||||
|
return false, fmt.Errorf("路由 %s %w", siteName, err)
|
||||||
|
}
|
||||||
|
project, activeDeployment, err := loadActivePagesProject(ctx, projectID, siteName)
|
||||||
|
if err != nil {
|
||||||
|
return false, err
|
||||||
|
}
|
||||||
|
deployment := buildLivePagesDeployment(project, activeDeployment)
|
||||||
|
projectIDCopy := project.ID
|
||||||
|
originURL := fmt.Sprintf("openflare-pages://project/%d", project.ID)
|
||||||
|
|
||||||
|
if err := putJSON(route, "pages_project_id", projectIDCopy); err != nil {
|
||||||
|
return false, err
|
||||||
|
}
|
||||||
|
if err := putJSON(route, "pages_deployment", deployment); err != nil {
|
||||||
|
return false, err
|
||||||
|
}
|
||||||
|
if err := putJSON(route, "origin_url", originURL); err != nil {
|
||||||
|
return false, err
|
||||||
|
}
|
||||||
|
if err := putJSON(route, "upstreams", []string{originURL}); err != nil {
|
||||||
|
return false, err
|
||||||
|
}
|
||||||
|
routes[index] = route
|
||||||
|
changed = true
|
||||||
|
}
|
||||||
|
return changed, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
const jsonNullLiteral = "null"
|
||||||
|
|
||||||
|
func isPresentJSON(raw json.RawMessage) bool {
|
||||||
|
return len(raw) > 0 && string(raw) != jsonNullLiteral
|
||||||
|
}
|
||||||
|
|
||||||
|
func resolveProjectIDFromRouteMap(route map[string]json.RawMessage) (uint, error) {
|
||||||
|
if raw, ok := route["pages_project_id"]; ok && isPresentJSON(raw) {
|
||||||
|
var projectID uint
|
||||||
|
if err := json.Unmarshal(raw, &projectID); err == nil && projectID != 0 {
|
||||||
|
return projectID, nil
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if raw, ok := route["pages_deployment"]; ok && isPresentJSON(raw) {
|
||||||
|
var deployment struct {
|
||||||
|
ProjectID uint `json:"project_id"`
|
||||||
|
}
|
||||||
|
if err := json.Unmarshal(raw, &deployment); err == nil && deployment.ProjectID != 0 {
|
||||||
|
return deployment.ProjectID, nil
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return 0, fmt.Errorf("pages 配置无效: 缺少 pages_project_id")
|
||||||
|
}
|
||||||
|
|
||||||
|
func loadActivePagesProject(ctx context.Context, projectID uint, siteName string) (*model.PagesProject, *model.PagesDeployment, error) {
|
||||||
|
if siteName == "" {
|
||||||
|
siteName = "pages"
|
||||||
|
}
|
||||||
|
project, err := model.GetPagesProjectByID(ctx, projectID)
|
||||||
|
if err != nil {
|
||||||
|
if errorsIsNotFound(err) {
|
||||||
|
return nil, nil, fmt.Errorf("路由 %s Pages 配置无效: pages 项目不存在", siteName)
|
||||||
|
}
|
||||||
|
return nil, nil, err
|
||||||
|
}
|
||||||
|
if !project.Enabled {
|
||||||
|
return nil, nil, fmt.Errorf("路由 %s Pages 配置无效: pages 项目未启用", siteName)
|
||||||
|
}
|
||||||
|
if project.ActiveDeploymentID == nil || *project.ActiveDeploymentID == 0 {
|
||||||
|
return nil, nil, fmt.Errorf("路由 %s Pages 配置无效: pages 项目没有激活部署", siteName)
|
||||||
|
}
|
||||||
|
activeDeployment, err := model.GetPagesDeploymentByID(ctx, *project.ActiveDeploymentID)
|
||||||
|
if err != nil {
|
||||||
|
if errorsIsNotFound(err) {
|
||||||
|
return nil, nil, fmt.Errorf("路由 %s Pages 配置无效: pages 激活部署不存在", siteName)
|
||||||
|
}
|
||||||
|
return nil, nil, err
|
||||||
|
}
|
||||||
|
if activeDeployment.ProjectID != project.ID {
|
||||||
|
return nil, nil, fmt.Errorf("路由 %s Pages 配置无效: pages 激活部署不匹配", siteName)
|
||||||
|
}
|
||||||
|
if strings.TrimSpace(activeDeployment.Checksum) == "" {
|
||||||
|
return nil, nil, fmt.Errorf("路由 %s Pages 配置无效: pages 部署校验和缺失", siteName)
|
||||||
|
}
|
||||||
|
return project, activeDeployment, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func buildLivePagesDeployment(project *model.PagesProject, active *model.PagesDeployment) *openrestyrender.PagesDeployment {
|
||||||
|
entryFile := strings.TrimSpace(project.EntryFile)
|
||||||
|
if entryFile == "" {
|
||||||
|
entryFile = defaultPagesEntryFile
|
||||||
|
}
|
||||||
|
fallbackPath := strings.TrimSpace(project.SPAFallbackPath)
|
||||||
|
if fallbackPath == "" {
|
||||||
|
fallbackPath = defaultPagesFallbackPath
|
||||||
|
}
|
||||||
|
return &openrestyrender.PagesDeployment{
|
||||||
|
ProjectID: project.ID,
|
||||||
|
ProjectSlug: strings.TrimSpace(project.Slug),
|
||||||
|
DeploymentID: active.ID,
|
||||||
|
DeploymentNumber: active.DeploymentNumber,
|
||||||
|
Checksum: strings.TrimSpace(active.Checksum),
|
||||||
|
EntryFile: entryFile,
|
||||||
|
SPAFallbackEnabled: project.SPAFallbackEnabled,
|
||||||
|
SPAFallbackPath: fallbackPath,
|
||||||
|
APIProxyEnabled: project.APIProxyEnabled,
|
||||||
|
APIProxyPath: strings.TrimSpace(project.APIProxyPath),
|
||||||
|
APIProxyPass: strings.TrimSpace(project.APIProxyPass),
|
||||||
|
APIProxyRewrite: strings.TrimSpace(project.APIProxyRewrite),
|
||||||
|
LocalRoot: fmt.Sprintf(
|
||||||
|
"%s/deployments/%d/current",
|
||||||
|
openrestyrender.PagesDirPlaceholder,
|
||||||
|
active.ID,
|
||||||
|
),
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func rawJSONString(raw json.RawMessage) (string, bool) {
|
||||||
|
if !isPresentJSON(raw) {
|
||||||
|
return "", false
|
||||||
|
}
|
||||||
|
var value string
|
||||||
|
if err := json.Unmarshal(raw, &value); err != nil {
|
||||||
|
return "", false
|
||||||
|
}
|
||||||
|
return value, true
|
||||||
|
}
|
||||||
|
|
||||||
|
func putJSON(route map[string]json.RawMessage, key string, value any) error {
|
||||||
|
encoded, err := json.Marshal(value)
|
||||||
|
if err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
route[key] = encoded
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func errorsIsNotFound(err error) bool {
|
||||||
|
return err != nil && (err == gorm.ErrRecordNotFound || strings.Contains(strings.ToLower(err.Error()), "record not found"))
|
||||||
|
}
|
||||||
@@ -0,0 +1,87 @@
|
|||||||
|
// Copyright 2026 Arctel.net
|
||||||
|
// SPDX-License-Identifier: Apache-2.0
|
||||||
|
|
||||||
|
package pages
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
"encoding/json"
|
||||||
|
"testing"
|
||||||
|
|
||||||
|
"github.com/Rain-kl/Wavelet/internal/db"
|
||||||
|
"github.com/Rain-kl/Wavelet/internal/model"
|
||||||
|
"github.com/stretchr/testify/assert"
|
||||||
|
"github.com/stretchr/testify/require"
|
||||||
|
)
|
||||||
|
|
||||||
|
func TestRebindSnapshotPagesToCurrentActive(t *testing.T) {
|
||||||
|
cleanup := setupPagesTestDB(t)
|
||||||
|
defer cleanup()
|
||||||
|
ctx := context.Background()
|
||||||
|
|
||||||
|
project, err := CreateProject(ctx, Input{
|
||||||
|
Name: "Rebind Site",
|
||||||
|
Slug: "rebind-site",
|
||||||
|
Enabled: true,
|
||||||
|
})
|
||||||
|
require.NoError(t, err)
|
||||||
|
|
||||||
|
old := &model.PagesDeployment{
|
||||||
|
ProjectID: project.ID,
|
||||||
|
DeploymentNumber: 1,
|
||||||
|
Checksum: "old-checksum",
|
||||||
|
Status: model.PagesDeploymentStatusUploaded,
|
||||||
|
FileCount: 1,
|
||||||
|
}
|
||||||
|
require.NoError(t, db.DB(ctx).Create(old).Error)
|
||||||
|
active := &model.PagesDeployment{
|
||||||
|
ProjectID: project.ID,
|
||||||
|
DeploymentNumber: 2,
|
||||||
|
Checksum: "new-checksum",
|
||||||
|
Status: model.PagesDeploymentStatusActive,
|
||||||
|
FileCount: 1,
|
||||||
|
}
|
||||||
|
require.NoError(t, db.DB(ctx).Create(active).Error)
|
||||||
|
require.NoError(t, db.DB(ctx).Model(&model.PagesProject{}).
|
||||||
|
Where("id = ?", project.ID).
|
||||||
|
Update("active_deployment_id", active.ID).Error)
|
||||||
|
|
||||||
|
// Frozen snapshot still points at the old deployment (simulates old main config).
|
||||||
|
frozen := map[string]any{
|
||||||
|
"routes": []map[string]any{
|
||||||
|
{
|
||||||
|
"site_name": "rebind",
|
||||||
|
"origin_url": "openflare-pages://project/1",
|
||||||
|
"enabled": true,
|
||||||
|
"upstream_type": "pages",
|
||||||
|
"pages_project_id": project.ID,
|
||||||
|
"pages_deployment": map[string]any{
|
||||||
|
"project_id": project.ID,
|
||||||
|
"deployment_id": old.ID,
|
||||||
|
"checksum": "old-checksum",
|
||||||
|
"local_root": "__OPENFLARE_PAGES_DIR__/deployments/1/current",
|
||||||
|
},
|
||||||
|
"extra_keep_me": "yes",
|
||||||
|
},
|
||||||
|
},
|
||||||
|
"waf": map[string]any{"rule_groups": []any{}},
|
||||||
|
}
|
||||||
|
frozenJSON, err := json.Marshal(frozen)
|
||||||
|
require.NoError(t, err)
|
||||||
|
|
||||||
|
reboundJSON, err := RebindSnapshotPagesToCurrentActive(ctx, string(frozenJSON))
|
||||||
|
require.NoError(t, err)
|
||||||
|
|
||||||
|
var rebound map[string]any
|
||||||
|
require.NoError(t, json.Unmarshal([]byte(reboundJSON), &rebound))
|
||||||
|
_, hasWAF := rebound["waf"]
|
||||||
|
assert.True(t, hasWAF)
|
||||||
|
|
||||||
|
routes := rebound["routes"].([]any)
|
||||||
|
require.Len(t, routes, 1)
|
||||||
|
route := routes[0].(map[string]any)
|
||||||
|
assert.Equal(t, "yes", route["extra_keep_me"])
|
||||||
|
deployment := route["pages_deployment"].(map[string]any)
|
||||||
|
assert.EqualValues(t, active.ID, deployment["deployment_id"])
|
||||||
|
assert.Equal(t, "new-checksum", deployment["checksum"])
|
||||||
|
}
|
||||||
@@ -0,0 +1,9 @@
|
|||||||
|
-- +goose Up
|
||||||
|
INSERT INTO w_system_configs (key, value, type, visibility, description, created_at, updated_at)
|
||||||
|
VALUES
|
||||||
|
('pages_max_package_size_mb', '100', 'business', 0, 'Pages 部署包上传大小上限(MiB)', CURRENT_TIMESTAMP, CURRENT_TIMESTAMP),
|
||||||
|
('pages_max_history_count', '20', 'business', 0, 'Pages 每个项目最大历史部署保留数(0 表示不限制)', CURRENT_TIMESTAMP, CURRENT_TIMESTAMP)
|
||||||
|
ON CONFLICT (key) DO NOTHING;
|
||||||
|
|
||||||
|
-- +goose Down
|
||||||
|
DELETE FROM w_system_configs WHERE key IN ('pages_max_package_size_mb', 'pages_max_history_count');
|
||||||
@@ -0,0 +1,9 @@
|
|||||||
|
-- +goose Up
|
||||||
|
INSERT INTO w_system_configs (key, value, type, visibility, description, created_at, updated_at)
|
||||||
|
VALUES
|
||||||
|
('pages_max_package_size_mb', '100', 'business', 0, 'Pages 部署包上传大小上限(MiB)', CURRENT_TIMESTAMP, CURRENT_TIMESTAMP),
|
||||||
|
('pages_max_history_count', '20', 'business', 0, 'Pages 每个项目最大历史部署保留数(0 表示不限制)', CURRENT_TIMESTAMP, CURRENT_TIMESTAMP)
|
||||||
|
ON CONFLICT (key) DO NOTHING;
|
||||||
|
|
||||||
|
-- +goose Down
|
||||||
|
DELETE FROM w_system_configs WHERE key IN ('pages_max_package_size_mb', 'pages_max_history_count');
|
||||||
@@ -50,6 +50,10 @@ const (
|
|||||||
ConfigKeyDatabaseAutoCleanupEnabled = "database_auto_cleanup_enabled" // 数据库自动清理开关
|
ConfigKeyDatabaseAutoCleanupEnabled = "database_auto_cleanup_enabled" // 数据库自动清理开关
|
||||||
ConfigKeyDatabaseAutoCleanupRetentionDays = "database_auto_cleanup_retention_days" // 数据库保留天数
|
ConfigKeyDatabaseAutoCleanupRetentionDays = "database_auto_cleanup_retention_days" // 数据库保留天数
|
||||||
|
|
||||||
|
// Pages 静态托管配置
|
||||||
|
ConfigKeyPagesMaxPackageSizeMB = "pages_max_package_size_mb" // Pages 部署包上传大小上限(MiB)
|
||||||
|
ConfigKeyPagesMaxHistoryCount = "pages_max_history_count" // Pages 每个项目最大历史部署保留数(0 表示不限制)
|
||||||
|
|
||||||
// UptimeKuma 集成配置
|
// UptimeKuma 集成配置
|
||||||
ConfigKeyUptimeKumaEnabled = "uptime_kuma_enabled" // UptimeKuma 集成开关
|
ConfigKeyUptimeKumaEnabled = "uptime_kuma_enabled" // UptimeKuma 集成开关
|
||||||
ConfigKeyUptimeKumaURL = "uptime_kuma_url" // UptimeKuma URL
|
ConfigKeyUptimeKumaURL = "uptime_kuma_url" // UptimeKuma URL
|
||||||
|
|||||||
@@ -295,6 +295,18 @@ func getSeedConfigsPart2() []model.SystemConfig {
|
|||||||
Type: configTypeBusiness,
|
Type: configTypeBusiness,
|
||||||
Description: "FRPS 内置 Web 界面端口",
|
Description: "FRPS 内置 Web 界面端口",
|
||||||
},
|
},
|
||||||
|
{
|
||||||
|
Key: model.ConfigKeyPagesMaxPackageSizeMB,
|
||||||
|
Value: "100",
|
||||||
|
Type: configTypeBusiness,
|
||||||
|
Description: "Pages 部署包上传大小上限(MiB)",
|
||||||
|
},
|
||||||
|
{
|
||||||
|
Key: model.ConfigKeyPagesMaxHistoryCount,
|
||||||
|
Value: "20",
|
||||||
|
Type: configTypeBusiness,
|
||||||
|
Description: "Pages 每个项目最大历史部署保留数(0 表示不限制)",
|
||||||
|
},
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|||||||
@@ -0,0 +1,101 @@
|
|||||||
|
// Copyright 2026 Arctel.net
|
||||||
|
// SPDX-License-Identifier: Apache-2.0
|
||||||
|
|
||||||
|
package pagesarchive
|
||||||
|
|
||||||
|
import (
|
||||||
|
"crypto/sha256"
|
||||||
|
"encoding/hex"
|
||||||
|
"fmt"
|
||||||
|
"io"
|
||||||
|
"math"
|
||||||
|
"os"
|
||||||
|
"path/filepath"
|
||||||
|
)
|
||||||
|
|
||||||
|
// Limits bounds archive inspection / extraction work.
|
||||||
|
type Limits struct {
|
||||||
|
// MaxFiles is the maximum number of regular files allowed.
|
||||||
|
MaxFiles int
|
||||||
|
// MaxFileBytes is the maximum size of a single extracted file.
|
||||||
|
MaxFileBytes int64
|
||||||
|
// MaxTotalBytes is the maximum sum of all extracted file sizes.
|
||||||
|
MaxTotalBytes int64
|
||||||
|
}
|
||||||
|
|
||||||
|
// FileEntry is a regular file discovered inside a deployment package.
|
||||||
|
type FileEntry struct {
|
||||||
|
Path string
|
||||||
|
Size int64
|
||||||
|
Checksum string
|
||||||
|
}
|
||||||
|
|
||||||
|
// Manifest is the inspected content of a Pages deployment package.
|
||||||
|
type Manifest struct {
|
||||||
|
Files []FileEntry
|
||||||
|
FileCount int
|
||||||
|
TotalSize int64
|
||||||
|
}
|
||||||
|
|
||||||
|
// Entry describes one archive member for extraction.
|
||||||
|
type Entry struct {
|
||||||
|
// Name is the original path inside the archive.
|
||||||
|
Name string
|
||||||
|
// IsDir marks directory entries.
|
||||||
|
IsDir bool
|
||||||
|
// IsSymlink marks symbolic links (unsupported for Pages).
|
||||||
|
IsSymlink bool
|
||||||
|
// Size is the declared uncompressed size when known; 0 means unknown.
|
||||||
|
Size uint64
|
||||||
|
// Open returns a reader for the entry body. Caller must Close it.
|
||||||
|
Open func() (io.ReadCloser, error)
|
||||||
|
}
|
||||||
|
|
||||||
|
// copyLimited copies src to dst.
|
||||||
|
// When maxBytes <= 0, size limits are not enforced (trusted extract path).
|
||||||
|
func copyLimited(dst io.Writer, src io.Reader, declaredSize uint64, maxBytes int64) (int64, error) {
|
||||||
|
if maxBytes <= 0 {
|
||||||
|
if declaredSize > 0 {
|
||||||
|
if declaredSize > uint64(math.MaxInt64) {
|
||||||
|
return 0, fmt.Errorf("pages file size out of bounds")
|
||||||
|
}
|
||||||
|
//nolint:gosec // declaredSize is bounded to MaxInt64 above
|
||||||
|
return io.CopyN(dst, src, int64(declaredSize))
|
||||||
|
}
|
||||||
|
return io.Copy(dst, src)
|
||||||
|
}
|
||||||
|
if declaredSize > uint64(maxBytes) || declaredSize > uint64(math.MaxInt64) { //nolint:gosec // maxBytes positive
|
||||||
|
return 0, fmt.Errorf("pages file size out of bounds")
|
||||||
|
}
|
||||||
|
if declaredSize > 0 {
|
||||||
|
//nolint:gosec // declaredSize is bounded to MaxInt64 above
|
||||||
|
return io.CopyN(dst, src, int64(declaredSize))
|
||||||
|
}
|
||||||
|
limited := io.LimitReader(src, maxBytes+1)
|
||||||
|
written, err := io.Copy(dst, limited)
|
||||||
|
if written > maxBytes {
|
||||||
|
return written, fmt.Errorf("pages file size out of bounds")
|
||||||
|
}
|
||||||
|
return written, err
|
||||||
|
}
|
||||||
|
|
||||||
|
func checksumReader(src io.Reader, declaredSize uint64, maxBytes int64) (string, int64, error) {
|
||||||
|
hash := sha256.New()
|
||||||
|
written, err := copyLimited(hash, src, declaredSize, maxBytes)
|
||||||
|
if err != nil {
|
||||||
|
return "", written, err
|
||||||
|
}
|
||||||
|
return hex.EncodeToString(hash.Sum(nil)), written, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func writeEntryFile(targetPath string, src io.Reader, declaredSize uint64, maxBytes int64, perm os.FileMode) (int64, error) {
|
||||||
|
if err := os.MkdirAll(filepath.Dir(targetPath), dirPerm); err != nil {
|
||||||
|
return 0, err
|
||||||
|
}
|
||||||
|
target, err := os.OpenFile(targetPath, os.O_CREATE|os.O_WRONLY|os.O_TRUNC, perm) //nolint:gosec // caller validates path under release dir
|
||||||
|
if err != nil {
|
||||||
|
return 0, err
|
||||||
|
}
|
||||||
|
defer func() { _ = target.Close() }()
|
||||||
|
return copyLimited(target, src, declaredSize, maxBytes)
|
||||||
|
}
|
||||||
@@ -0,0 +1,154 @@
|
|||||||
|
// Copyright 2026 Arctel.net
|
||||||
|
// SPDX-License-Identifier: Apache-2.0
|
||||||
|
|
||||||
|
package pagesarchive
|
||||||
|
|
||||||
|
import (
|
||||||
|
"fmt"
|
||||||
|
"os"
|
||||||
|
"path/filepath"
|
||||||
|
)
|
||||||
|
|
||||||
|
// ExtractOptions controls package extraction.
|
||||||
|
type ExtractOptions struct {
|
||||||
|
// Limits bounds files and sizes during extraction when EnforceLimits is true.
|
||||||
|
Limits Limits
|
||||||
|
// StripCommonRoot strips a single shared top-level directory when present.
|
||||||
|
StripCommonRoot bool
|
||||||
|
// EnforceLimits enables MaxFiles / MaxFileBytes / MaxTotalBytes checks.
|
||||||
|
// When false, the caller is assumed to have already validated the package
|
||||||
|
// (e.g. Agent trusts control-plane inspection). Path-escape and symlink
|
||||||
|
// guards still apply so local extraction cannot leave destDir.
|
||||||
|
EnforceLimits bool
|
||||||
|
}
|
||||||
|
|
||||||
|
// ExtractBytes extracts a deployment package into destDir.
|
||||||
|
func ExtractBytes(data []byte, format Format, destDir string, opts ExtractOptions) error {
|
||||||
|
if format == "" {
|
||||||
|
var err error
|
||||||
|
format, err = DetectFormat("", data)
|
||||||
|
if err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
}
|
||||||
|
entries, err := listEntries(data, format)
|
||||||
|
if err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
return extractEntries(entries, destDir, opts)
|
||||||
|
}
|
||||||
|
|
||||||
|
// ExtractFile reads path and extracts it into destDir.
|
||||||
|
func ExtractFile(filePath string, format Format, destDir string, opts ExtractOptions) error {
|
||||||
|
data, err := os.ReadFile(filePath) //nolint:gosec // controlled path
|
||||||
|
if err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
return ExtractBytes(data, format, destDir, opts)
|
||||||
|
}
|
||||||
|
|
||||||
|
func extractEntries(entries []Entry, destDir string, opts ExtractOptions) error {
|
||||||
|
limits := Limits{}
|
||||||
|
if opts.EnforceLimits {
|
||||||
|
limits = normalizeLimits(opts.Limits)
|
||||||
|
}
|
||||||
|
commonPrefix := ""
|
||||||
|
if opts.StripCommonRoot {
|
||||||
|
commonPrefix = FindCommonRootPrefix(collectFileNames(entries))
|
||||||
|
}
|
||||||
|
|
||||||
|
var totalSize int64
|
||||||
|
var fileCount int
|
||||||
|
for _, entry := range entries {
|
||||||
|
written, counted, err := extractSingleEntry(entry, destDir, commonPrefix, limits, opts.EnforceLimits)
|
||||||
|
if err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
if !counted {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
fileCount++
|
||||||
|
if opts.EnforceLimits && fileCount > limits.MaxFiles {
|
||||||
|
return fmt.Errorf("pages deployment file count exceeds %d", limits.MaxFiles)
|
||||||
|
}
|
||||||
|
totalSize += written
|
||||||
|
if opts.EnforceLimits && totalSize > limits.MaxTotalBytes {
|
||||||
|
return fmt.Errorf("pages extracted size exceeds limit")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if fileCount == 0 {
|
||||||
|
return fmt.Errorf("pages package is empty")
|
||||||
|
}
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func extractSingleEntry(
|
||||||
|
entry Entry,
|
||||||
|
destDir, commonPrefix string,
|
||||||
|
limits Limits,
|
||||||
|
enforceLimits bool,
|
||||||
|
) (written int64, counted bool, err error) {
|
||||||
|
relativePath, skip, err := NormalizeEntryPath(entry.Name)
|
||||||
|
if err != nil {
|
||||||
|
return 0, false, err
|
||||||
|
}
|
||||||
|
if skip {
|
||||||
|
return 0, false, nil
|
||||||
|
}
|
||||||
|
if commonPrefix != "" {
|
||||||
|
relativePath = StripPrefix(relativePath, commonPrefix)
|
||||||
|
if relativePath == "" {
|
||||||
|
return 0, false, nil
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if entry.IsSymlink {
|
||||||
|
return 0, false, fmt.Errorf("pages package contains unsupported symlink: %s", relativePath)
|
||||||
|
}
|
||||||
|
|
||||||
|
targetPath := filepath.Join(destDir, filepath.FromSlash(relativePath))
|
||||||
|
if !isWithinDir(destDir, targetPath) {
|
||||||
|
return 0, false, fmt.Errorf("pages package path escapes directory: %s", entry.Name)
|
||||||
|
}
|
||||||
|
|
||||||
|
if entry.IsDir {
|
||||||
|
if err := os.MkdirAll(targetPath, dirPerm); err != nil {
|
||||||
|
return 0, false, err
|
||||||
|
}
|
||||||
|
return 0, false, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
maxFileBytes := int64(0) // unlimited when not enforcing
|
||||||
|
if enforceLimits {
|
||||||
|
if exceedsFileByteLimit(entry.Size, limits.MaxFileBytes) {
|
||||||
|
return 0, false, fmt.Errorf("pages file too large: %s", relativePath)
|
||||||
|
}
|
||||||
|
maxFileBytes = limits.MaxFileBytes
|
||||||
|
}
|
||||||
|
src, err := entry.Open()
|
||||||
|
if err != nil {
|
||||||
|
return 0, false, fmt.Errorf("%s: %w", relativePath, err)
|
||||||
|
}
|
||||||
|
written, writeErr := writeEntryFile(targetPath, src, entry.Size, maxFileBytes, filePerm)
|
||||||
|
_ = src.Close()
|
||||||
|
if writeErr != nil {
|
||||||
|
return 0, false, fmt.Errorf("%s: %w", relativePath, writeErr)
|
||||||
|
}
|
||||||
|
return written, true, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func isWithinDir(baseDir, targetPath string) bool {
|
||||||
|
cleanBase := filepath.Clean(baseDir)
|
||||||
|
cleanTarget := filepath.Clean(targetPath)
|
||||||
|
rel, err := filepath.Rel(cleanBase, cleanTarget)
|
||||||
|
if err != nil {
|
||||||
|
return false
|
||||||
|
}
|
||||||
|
return rel != ".." && !hasParentRel(rel)
|
||||||
|
}
|
||||||
|
|
||||||
|
func hasParentRel(rel string) bool {
|
||||||
|
if rel == ".." {
|
||||||
|
return true
|
||||||
|
}
|
||||||
|
return len(rel) >= 3 && (rel[:3] == "../" || rel[:3] == "..\\")
|
||||||
|
}
|
||||||
@@ -0,0 +1,191 @@
|
|||||||
|
// Copyright 2026 Arctel.net
|
||||||
|
// SPDX-License-Identifier: Apache-2.0
|
||||||
|
|
||||||
|
// Package pagesarchive provides multi-format archive detection, inspection and
|
||||||
|
// extraction helpers for OpenFlare Pages deployment packages.
|
||||||
|
package pagesarchive
|
||||||
|
|
||||||
|
import (
|
||||||
|
"bytes"
|
||||||
|
"fmt"
|
||||||
|
"path/filepath"
|
||||||
|
"strings"
|
||||||
|
)
|
||||||
|
|
||||||
|
// Format identifies a supported Pages deployment package archive format.
|
||||||
|
type Format string
|
||||||
|
|
||||||
|
const (
|
||||||
|
// FormatZip is a ZIP archive.
|
||||||
|
FormatZip Format = "zip"
|
||||||
|
// FormatTar is an uncompressed tar archive.
|
||||||
|
FormatTar Format = "tar"
|
||||||
|
// FormatTarGz is a gzip-compressed tar archive.
|
||||||
|
FormatTarGz Format = "tar.gz"
|
||||||
|
// FormatTarXz is an xz-compressed tar archive.
|
||||||
|
FormatTarXz Format = "tar.xz"
|
||||||
|
// FormatTarBz2 is a bzip2-compressed tar archive.
|
||||||
|
FormatTarBz2 Format = "tar.bz2"
|
||||||
|
// FormatSevenZip is a 7z archive.
|
||||||
|
FormatSevenZip Format = "7z"
|
||||||
|
|
||||||
|
ustarMagicOffset = 257
|
||||||
|
ustarMagicMinLen = 262
|
||||||
|
)
|
||||||
|
|
||||||
|
// DetectFormatFromName returns the archive format inferred from a file name.
|
||||||
|
// Returns an empty Format and false when the extension is unsupported.
|
||||||
|
func DetectFormatFromName(fileName string) (Format, bool) {
|
||||||
|
name := strings.ToLower(strings.TrimSpace(fileName))
|
||||||
|
switch {
|
||||||
|
case strings.HasSuffix(name, ".tar.gz"), strings.HasSuffix(name, ".tgz"):
|
||||||
|
return FormatTarGz, true
|
||||||
|
case strings.HasSuffix(name, ".tar.xz"), strings.HasSuffix(name, ".txz"):
|
||||||
|
return FormatTarXz, true
|
||||||
|
case strings.HasSuffix(name, ".tar.bz2"), strings.HasSuffix(name, ".tbz2"), strings.HasSuffix(name, ".tbz"):
|
||||||
|
return FormatTarBz2, true
|
||||||
|
case strings.HasSuffix(name, ".tar"):
|
||||||
|
return FormatTar, true
|
||||||
|
case strings.HasSuffix(name, ".7z"):
|
||||||
|
return FormatSevenZip, true
|
||||||
|
case strings.HasSuffix(name, ".zip"):
|
||||||
|
return FormatZip, true
|
||||||
|
default:
|
||||||
|
return "", false
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// DetectFormatFromBytes returns the archive format inferred from magic bytes.
|
||||||
|
// Prefer DetectFormatFromName when a reliable file name is available.
|
||||||
|
func DetectFormatFromBytes(data []byte) (Format, bool) {
|
||||||
|
if isZipMagic(data) {
|
||||||
|
return FormatZip, true
|
||||||
|
}
|
||||||
|
if isSevenZipMagic(data) {
|
||||||
|
return FormatSevenZip, true
|
||||||
|
}
|
||||||
|
if isXZMagic(data) {
|
||||||
|
return FormatTarXz, true
|
||||||
|
}
|
||||||
|
if isGzipMagic(data) {
|
||||||
|
return FormatTarGz, true
|
||||||
|
}
|
||||||
|
if isBzip2Magic(data) {
|
||||||
|
return FormatTarBz2, true
|
||||||
|
}
|
||||||
|
if looksLikeTar(data) {
|
||||||
|
return FormatTar, true
|
||||||
|
}
|
||||||
|
return "", false
|
||||||
|
}
|
||||||
|
|
||||||
|
// DetectFormat prefers the file name when present, otherwise magic bytes.
|
||||||
|
func DetectFormat(fileName string, data []byte) (Format, error) {
|
||||||
|
if format, ok := DetectFormatFromName(fileName); ok {
|
||||||
|
return format, nil
|
||||||
|
}
|
||||||
|
if format, ok := DetectFormatFromBytes(data); ok {
|
||||||
|
return format, nil
|
||||||
|
}
|
||||||
|
return "", fmt.Errorf("unsupported pages package format")
|
||||||
|
}
|
||||||
|
|
||||||
|
// Extension returns the canonical file extension for a format (without leading dot).
|
||||||
|
func Extension(format Format) string {
|
||||||
|
switch format {
|
||||||
|
case FormatZip:
|
||||||
|
return "zip"
|
||||||
|
case FormatTar:
|
||||||
|
return "tar"
|
||||||
|
case FormatTarGz:
|
||||||
|
return "tar.gz"
|
||||||
|
case FormatTarXz:
|
||||||
|
return "tar.xz"
|
||||||
|
case FormatTarBz2:
|
||||||
|
return "tar.bz2"
|
||||||
|
case FormatSevenZip:
|
||||||
|
return "7z"
|
||||||
|
default:
|
||||||
|
return "bin"
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// MIMEType returns a reasonable content type for the archive format.
|
||||||
|
func MIMEType(format Format) string {
|
||||||
|
switch format {
|
||||||
|
case FormatZip:
|
||||||
|
return "application/zip"
|
||||||
|
case FormatTar:
|
||||||
|
return "application/x-tar"
|
||||||
|
case FormatTarGz:
|
||||||
|
return "application/gzip"
|
||||||
|
case FormatTarXz:
|
||||||
|
return "application/x-xz"
|
||||||
|
case FormatTarBz2:
|
||||||
|
return "application/x-bzip2"
|
||||||
|
case FormatSevenZip:
|
||||||
|
return "application/x-7z-compressed"
|
||||||
|
default:
|
||||||
|
return "application/octet-stream"
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// SupportedExtensions lists human-readable extensions for UI copy and accept attributes.
|
||||||
|
func SupportedExtensions() []string {
|
||||||
|
return []string{".zip", ".tar.gz", ".tgz", ".tar.xz", ".txz", ".tar.bz2", ".tbz2", ".tar", ".7z"}
|
||||||
|
}
|
||||||
|
|
||||||
|
// AcceptAttribute returns a comma-separated accept list for file inputs.
|
||||||
|
func AcceptAttribute() string {
|
||||||
|
return strings.Join(SupportedExtensions(), ",")
|
||||||
|
}
|
||||||
|
|
||||||
|
// NormalizeNameExtension returns a storage-safe extension for the given format/name.
|
||||||
|
func NormalizeNameExtension(fileName string, format Format) string {
|
||||||
|
if format != "" {
|
||||||
|
return Extension(format)
|
||||||
|
}
|
||||||
|
if formatFromName, ok := DetectFormatFromName(fileName); ok {
|
||||||
|
return Extension(formatFromName)
|
||||||
|
}
|
||||||
|
ext := strings.TrimPrefix(filepath.Ext(fileName), ".")
|
||||||
|
if ext == "" {
|
||||||
|
return "bin"
|
||||||
|
}
|
||||||
|
return strings.ToLower(ext)
|
||||||
|
}
|
||||||
|
|
||||||
|
func isZipMagic(data []byte) bool {
|
||||||
|
return len(data) >= 4 &&
|
||||||
|
data[0] == 0x50 && data[1] == 0x4b &&
|
||||||
|
(data[2] == 0x03 || data[2] == 0x05 || data[2] == 0x07) &&
|
||||||
|
(data[3] == 0x04 || data[3] == 0x06 || data[3] == 0x08)
|
||||||
|
}
|
||||||
|
|
||||||
|
func isSevenZipMagic(data []byte) bool {
|
||||||
|
return len(data) >= 6 &&
|
||||||
|
data[0] == 0x37 && data[1] == 0x7a && data[2] == 0xbc &&
|
||||||
|
data[3] == 0xaf && data[4] == 0x27 && data[5] == 0x1c
|
||||||
|
}
|
||||||
|
|
||||||
|
func isXZMagic(data []byte) bool {
|
||||||
|
return len(data) >= 6 &&
|
||||||
|
data[0] == 0xfd && data[1] == 0x37 && data[2] == 0x7a &&
|
||||||
|
data[3] == 0x58 && data[4] == 0x5a && data[5] == 0x00
|
||||||
|
}
|
||||||
|
|
||||||
|
func isGzipMagic(data []byte) bool {
|
||||||
|
return len(data) >= 2 && data[0] == 0x1f && data[1] == 0x8b
|
||||||
|
}
|
||||||
|
|
||||||
|
func isBzip2Magic(data []byte) bool {
|
||||||
|
return len(data) >= 3 && data[0] == 0x42 && data[1] == 0x5a && data[2] == 0x68
|
||||||
|
}
|
||||||
|
|
||||||
|
func looksLikeTar(data []byte) bool {
|
||||||
|
// POSIX ustar magic at offset 257 ("ustar\0" or "ustar ").
|
||||||
|
if len(data) < ustarMagicMinLen {
|
||||||
|
return false
|
||||||
|
}
|
||||||
|
return bytes.Equal(data[ustarMagicOffset:ustarMagicOffset+5], []byte("ustar"))
|
||||||
|
}
|
||||||
@@ -0,0 +1,148 @@
|
|||||||
|
// Copyright 2026 Arctel.net
|
||||||
|
// SPDX-License-Identifier: Apache-2.0
|
||||||
|
|
||||||
|
package pagesarchive
|
||||||
|
|
||||||
|
import (
|
||||||
|
"fmt"
|
||||||
|
"os"
|
||||||
|
"path"
|
||||||
|
"strings"
|
||||||
|
)
|
||||||
|
|
||||||
|
// InspectOptions controls package inspection.
|
||||||
|
type InspectOptions struct {
|
||||||
|
// RootDir is an optional project root subdirectory that must contain EntryFile.
|
||||||
|
RootDir string
|
||||||
|
// EntryFile is the required entry file name (e.g. index.html).
|
||||||
|
EntryFile string
|
||||||
|
// Limits bounds files and sizes.
|
||||||
|
Limits Limits
|
||||||
|
}
|
||||||
|
|
||||||
|
// InspectFile opens path and inspects it as a Pages deployment package.
|
||||||
|
func InspectFile(filePath string, format Format, opts InspectOptions) (*Manifest, error) {
|
||||||
|
data, err := os.ReadFile(filePath) //nolint:gosec // filePath is a controlled temp upload path
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
return InspectBytes(data, format, opts)
|
||||||
|
}
|
||||||
|
|
||||||
|
// InspectBytes inspects an in-memory deployment package.
|
||||||
|
func InspectBytes(data []byte, format Format, opts InspectOptions) (*Manifest, error) {
|
||||||
|
if format == "" {
|
||||||
|
var err error
|
||||||
|
format, err = DetectFormat("", data)
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
}
|
||||||
|
entries, err := listEntries(data, format)
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
return buildManifest(entries, opts)
|
||||||
|
}
|
||||||
|
|
||||||
|
func buildManifest(entries []Entry, opts InspectOptions) (*Manifest, error) {
|
||||||
|
limits := normalizeLimits(opts.Limits)
|
||||||
|
commonPrefix := FindCommonRootPrefix(collectFileNames(entries))
|
||||||
|
targetEntryPath := resolveTargetEntryPath(opts.RootDir, opts.EntryFile)
|
||||||
|
|
||||||
|
manifest := &Manifest{Files: make([]FileEntry, 0)}
|
||||||
|
entrySeen := false
|
||||||
|
|
||||||
|
for _, entry := range entries {
|
||||||
|
normalizedPath, skip, err := prepareEntryPath(entry, commonPrefix)
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
if skip {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
if exceedsFileByteLimit(entry.Size, limits.MaxFileBytes) {
|
||||||
|
return nil, fmt.Errorf("pages file too large: %s", normalizedPath)
|
||||||
|
}
|
||||||
|
|
||||||
|
fileEntry, err := inspectRegularFile(entry, normalizedPath, limits)
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
manifest.FileCount++
|
||||||
|
if manifest.FileCount > limits.MaxFiles {
|
||||||
|
return nil, fmt.Errorf("pages deployment file count exceeds %d", limits.MaxFiles)
|
||||||
|
}
|
||||||
|
manifest.TotalSize += fileEntry.Size
|
||||||
|
if manifest.TotalSize > limits.MaxTotalBytes {
|
||||||
|
return nil, fmt.Errorf("pages extracted size exceeds limit")
|
||||||
|
}
|
||||||
|
if normalizedPath == targetEntryPath {
|
||||||
|
entrySeen = true
|
||||||
|
}
|
||||||
|
manifest.Files = append(manifest.Files, fileEntry)
|
||||||
|
}
|
||||||
|
|
||||||
|
if manifest.FileCount == 0 {
|
||||||
|
return nil, fmt.Errorf("pages package is empty")
|
||||||
|
}
|
||||||
|
if !entrySeen {
|
||||||
|
return nil, fmt.Errorf("pages package is missing entry file %s", targetEntryPath)
|
||||||
|
}
|
||||||
|
return manifest, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func collectFileNames(entries []Entry) []string {
|
||||||
|
names := make([]string, 0, len(entries))
|
||||||
|
for _, entry := range entries {
|
||||||
|
if entry.IsDir || entry.IsSymlink {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
names = append(names, entry.Name)
|
||||||
|
}
|
||||||
|
return names
|
||||||
|
}
|
||||||
|
|
||||||
|
func resolveTargetEntryPath(rootDir, entryFile string) string {
|
||||||
|
normalizedEntry := strings.TrimSpace(entryFile)
|
||||||
|
if normalizedEntry == "" {
|
||||||
|
normalizedEntry = "index.html"
|
||||||
|
}
|
||||||
|
normalizedRoot := strings.Trim(strings.TrimSpace(rootDir), "/")
|
||||||
|
if normalizedRoot == "" {
|
||||||
|
return normalizedEntry
|
||||||
|
}
|
||||||
|
return path.Join(normalizedRoot, normalizedEntry)
|
||||||
|
}
|
||||||
|
|
||||||
|
func prepareEntryPath(entry Entry, commonPrefix string) (string, bool, error) {
|
||||||
|
normalizedPath, skip, err := NormalizeEntryPath(entry.Name)
|
||||||
|
if err != nil {
|
||||||
|
return "", false, err
|
||||||
|
}
|
||||||
|
if skip || entry.IsDir {
|
||||||
|
return "", true, nil
|
||||||
|
}
|
||||||
|
normalizedPath = StripPrefix(normalizedPath, commonPrefix)
|
||||||
|
if entry.IsSymlink {
|
||||||
|
return "", false, fmt.Errorf("pages package contains unsupported symlink: %s", normalizedPath)
|
||||||
|
}
|
||||||
|
return normalizedPath, false, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func inspectRegularFile(entry Entry, normalizedPath string, limits Limits) (FileEntry, error) {
|
||||||
|
src, err := entry.Open()
|
||||||
|
if err != nil {
|
||||||
|
return FileEntry{}, fmt.Errorf("%s: %w", normalizedPath, err)
|
||||||
|
}
|
||||||
|
checksum, fileSize, checksumErr := checksumReader(src, entry.Size, limits.MaxFileBytes)
|
||||||
|
_ = src.Close()
|
||||||
|
if checksumErr != nil {
|
||||||
|
return FileEntry{}, fmt.Errorf("%s: %w", normalizedPath, checksumErr)
|
||||||
|
}
|
||||||
|
return FileEntry{
|
||||||
|
Path: normalizedPath,
|
||||||
|
Size: fileSize,
|
||||||
|
Checksum: checksum,
|
||||||
|
}, nil
|
||||||
|
}
|
||||||
@@ -0,0 +1,38 @@
|
|||||||
|
// Copyright 2026 Arctel.net
|
||||||
|
// SPDX-License-Identifier: Apache-2.0
|
||||||
|
|
||||||
|
package pagesarchive
|
||||||
|
|
||||||
|
const (
|
||||||
|
defaultMaxFiles = 1000
|
||||||
|
defaultMaxFileBytes = 100 * 1024 * 1024
|
||||||
|
defaultMaxTotalBytes = 100 * 1024 * 1024
|
||||||
|
dirPerm = 0o750
|
||||||
|
filePerm = 0o644
|
||||||
|
)
|
||||||
|
|
||||||
|
// normalizeLimits applies defaults for control-plane inspection.
|
||||||
|
// Callers that already validated the package should use EnforceLimits=false instead.
|
||||||
|
func normalizeLimits(limits Limits) Limits {
|
||||||
|
if limits.MaxFiles <= 0 {
|
||||||
|
limits.MaxFiles = defaultMaxFiles
|
||||||
|
}
|
||||||
|
if limits.MaxFileBytes <= 0 {
|
||||||
|
limits.MaxFileBytes = defaultMaxFileBytes
|
||||||
|
}
|
||||||
|
if limits.MaxTotalBytes <= 0 {
|
||||||
|
limits.MaxTotalBytes = defaultMaxTotalBytes
|
||||||
|
}
|
||||||
|
return limits
|
||||||
|
}
|
||||||
|
|
||||||
|
func exceedsFileByteLimit(size uint64, maxBytes int64) bool {
|
||||||
|
// maxBytes <= 0 means unlimited (trusted extract path).
|
||||||
|
if maxBytes <= 0 {
|
||||||
|
return false
|
||||||
|
}
|
||||||
|
if size == 0 {
|
||||||
|
return false
|
||||||
|
}
|
||||||
|
return size > uint64(maxBytes) //nolint:gosec // maxBytes is positive
|
||||||
|
}
|
||||||
@@ -0,0 +1,220 @@
|
|||||||
|
// Copyright 2026 Arctel.net
|
||||||
|
// SPDX-License-Identifier: Apache-2.0
|
||||||
|
|
||||||
|
package pagesarchive
|
||||||
|
|
||||||
|
import (
|
||||||
|
"archive/tar"
|
||||||
|
"archive/zip"
|
||||||
|
"bytes"
|
||||||
|
"compress/bzip2"
|
||||||
|
"compress/gzip"
|
||||||
|
"fmt"
|
||||||
|
"io"
|
||||||
|
"os"
|
||||||
|
|
||||||
|
"github.com/bodgit/sevenzip"
|
||||||
|
"github.com/ulikunitz/xz"
|
||||||
|
)
|
||||||
|
|
||||||
|
type archiveFile interface {
|
||||||
|
Name() string
|
||||||
|
IsDir() bool
|
||||||
|
IsSymlink() bool
|
||||||
|
Size() uint64
|
||||||
|
Open() (io.ReadCloser, error)
|
||||||
|
}
|
||||||
|
|
||||||
|
type zipArchiveFile struct {
|
||||||
|
file *zip.File
|
||||||
|
}
|
||||||
|
|
||||||
|
func (z zipArchiveFile) Name() string { return z.file.Name }
|
||||||
|
func (z zipArchiveFile) IsDir() bool { return z.file.FileInfo().IsDir() }
|
||||||
|
func (z zipArchiveFile) IsSymlink() bool {
|
||||||
|
return z.file.Mode()&os.ModeSymlink != 0
|
||||||
|
}
|
||||||
|
func (z zipArchiveFile) Size() uint64 { return z.file.UncompressedSize64 }
|
||||||
|
func (z zipArchiveFile) Open() (io.ReadCloser, error) {
|
||||||
|
return z.file.Open()
|
||||||
|
}
|
||||||
|
|
||||||
|
type sevenZipArchiveFile struct {
|
||||||
|
file *sevenzip.File
|
||||||
|
}
|
||||||
|
|
||||||
|
func (z sevenZipArchiveFile) Name() string { return z.file.Name }
|
||||||
|
func (z sevenZipArchiveFile) IsDir() bool { return z.file.FileInfo().IsDir() }
|
||||||
|
func (z sevenZipArchiveFile) IsSymlink() bool {
|
||||||
|
return z.file.Mode()&os.ModeSymlink != 0
|
||||||
|
}
|
||||||
|
func (z sevenZipArchiveFile) Size() uint64 { return z.file.UncompressedSize }
|
||||||
|
func (z sevenZipArchiveFile) Open() (io.ReadCloser, error) {
|
||||||
|
return z.file.Open()
|
||||||
|
}
|
||||||
|
|
||||||
|
func listEntries(data []byte, format Format) ([]Entry, error) {
|
||||||
|
switch format {
|
||||||
|
case FormatZip:
|
||||||
|
return listZipEntries(data)
|
||||||
|
case FormatTar:
|
||||||
|
return listTarEntries(bytes.NewReader(data))
|
||||||
|
case FormatTarGz:
|
||||||
|
gzReader, err := gzip.NewReader(bytes.NewReader(data))
|
||||||
|
if err != nil {
|
||||||
|
return nil, fmt.Errorf("open gzip pages package: %w", err)
|
||||||
|
}
|
||||||
|
defer func() { _ = gzReader.Close() }()
|
||||||
|
return listTarEntries(gzReader)
|
||||||
|
case FormatTarXz:
|
||||||
|
xzReader, err := xz.NewReader(bytes.NewReader(data))
|
||||||
|
if err != nil {
|
||||||
|
return nil, fmt.Errorf("open xz pages package: %w", err)
|
||||||
|
}
|
||||||
|
return listTarEntries(xzReader)
|
||||||
|
case FormatTarBz2:
|
||||||
|
return listTarEntries(bzip2.NewReader(bytes.NewReader(data)))
|
||||||
|
case FormatSevenZip:
|
||||||
|
return listSevenZipEntries(data)
|
||||||
|
default:
|
||||||
|
return nil, fmt.Errorf("unsupported pages package format: %s", format)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func entriesFromArchiveFiles(files []archiveFile) []Entry {
|
||||||
|
entries := make([]Entry, 0, len(files))
|
||||||
|
for _, item := range files {
|
||||||
|
file := item
|
||||||
|
entries = append(entries, Entry{
|
||||||
|
Name: file.Name(),
|
||||||
|
IsDir: file.IsDir(),
|
||||||
|
IsSymlink: file.IsSymlink(),
|
||||||
|
Size: file.Size(),
|
||||||
|
Open: file.Open,
|
||||||
|
})
|
||||||
|
}
|
||||||
|
return entries
|
||||||
|
}
|
||||||
|
|
||||||
|
func listZipEntries(data []byte) ([]Entry, error) {
|
||||||
|
reader, err := zip.NewReader(bytes.NewReader(data), int64(len(data)))
|
||||||
|
if err != nil {
|
||||||
|
return nil, fmt.Errorf("open zip pages package: %w", err)
|
||||||
|
}
|
||||||
|
files := make([]archiveFile, 0, len(reader.File))
|
||||||
|
for _, item := range reader.File {
|
||||||
|
files = append(files, zipArchiveFile{file: item})
|
||||||
|
}
|
||||||
|
return entriesFromArchiveFiles(files), nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func listSevenZipEntries(data []byte) ([]Entry, error) {
|
||||||
|
reader, err := sevenzip.NewReader(bytes.NewReader(data), int64(len(data)))
|
||||||
|
if err != nil {
|
||||||
|
return nil, fmt.Errorf("open 7z pages package: %w", err)
|
||||||
|
}
|
||||||
|
files := make([]archiveFile, 0, len(reader.File))
|
||||||
|
for _, item := range reader.File {
|
||||||
|
files = append(files, sevenZipArchiveFile{file: item})
|
||||||
|
}
|
||||||
|
return entriesFromArchiveFiles(files), nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func listTarEntries(r io.Reader) ([]Entry, error) {
|
||||||
|
tarReader := tar.NewReader(r)
|
||||||
|
// Tar is sequential: materialize regular file bodies so entries can be opened later.
|
||||||
|
type materialised struct {
|
||||||
|
header *tar.Header
|
||||||
|
body []byte
|
||||||
|
}
|
||||||
|
items := make([]materialised, 0)
|
||||||
|
for {
|
||||||
|
header, err := tarReader.Next()
|
||||||
|
if err == io.EOF {
|
||||||
|
break
|
||||||
|
}
|
||||||
|
if err != nil {
|
||||||
|
return nil, fmt.Errorf("read tar pages package: %w", err)
|
||||||
|
}
|
||||||
|
item, skip, err := materialiseTarHeader(tarReader, header)
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
if skip {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
items = append(items, item)
|
||||||
|
}
|
||||||
|
|
||||||
|
entries := make([]Entry, 0, len(items))
|
||||||
|
for _, item := range items {
|
||||||
|
entries = append(entries, tarEntryFromMaterialised(item.header, item.body))
|
||||||
|
}
|
||||||
|
return entries, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func materialiseTarHeader(tarReader *tar.Reader, header *tar.Header) (item struct {
|
||||||
|
header *tar.Header
|
||||||
|
body []byte
|
||||||
|
}, skip bool, err error) {
|
||||||
|
switch header.Typeflag {
|
||||||
|
case tar.TypeDir, tar.TypeSymlink, tar.TypeLink:
|
||||||
|
return struct {
|
||||||
|
header *tar.Header
|
||||||
|
body []byte
|
||||||
|
}{header: header}, false, nil
|
||||||
|
case tar.TypeReg, tar.TypeRegA: //nolint:staticcheck // TypeRegA still appears in older archives
|
||||||
|
body, readErr := readTarBody(tarReader, header)
|
||||||
|
if readErr != nil {
|
||||||
|
return item, false, readErr
|
||||||
|
}
|
||||||
|
return struct {
|
||||||
|
header *tar.Header
|
||||||
|
body []byte
|
||||||
|
}{header: header, body: body}, false, nil
|
||||||
|
default:
|
||||||
|
if header.Size > 0 {
|
||||||
|
if _, copyErr := io.CopyN(io.Discard, tarReader, header.Size); copyErr != nil {
|
||||||
|
return item, false, fmt.Errorf("skip tar entry %s: %w", header.Name, copyErr)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return item, true, nil
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func readTarBody(tarReader *tar.Reader, header *tar.Header) ([]byte, error) {
|
||||||
|
if header.Size > 0 {
|
||||||
|
body := make([]byte, header.Size)
|
||||||
|
if _, err := io.ReadFull(tarReader, body); err != nil {
|
||||||
|
return nil, fmt.Errorf("read tar entry %s: %w", header.Name, err)
|
||||||
|
}
|
||||||
|
return body, nil
|
||||||
|
}
|
||||||
|
body, err := io.ReadAll(tarReader)
|
||||||
|
if err != nil {
|
||||||
|
return nil, fmt.Errorf("read tar entry %s: %w", header.Name, err)
|
||||||
|
}
|
||||||
|
return body, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func tarEntryFromMaterialised(header *tar.Header, body []byte) Entry {
|
||||||
|
size := header.Size
|
||||||
|
if int64(len(body)) > size {
|
||||||
|
size = int64(len(body))
|
||||||
|
}
|
||||||
|
entry := Entry{
|
||||||
|
Name: header.Name,
|
||||||
|
IsDir: header.Typeflag == tar.TypeDir,
|
||||||
|
IsSymlink: header.Typeflag == tar.TypeSymlink || header.Typeflag == tar.TypeLink,
|
||||||
|
Size: uint64(size), //nolint:gosec // non-negative sizes
|
||||||
|
Open: func() (io.ReadCloser, error) {
|
||||||
|
return io.NopCloser(bytes.NewReader(body)), nil
|
||||||
|
},
|
||||||
|
}
|
||||||
|
if entry.IsDir || entry.IsSymlink {
|
||||||
|
entry.Open = func() (io.ReadCloser, error) {
|
||||||
|
return io.NopCloser(bytes.NewReader(nil)), nil
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return entry
|
||||||
|
}
|
||||||
@@ -0,0 +1,187 @@
|
|||||||
|
// Copyright 2026 Arctel.net
|
||||||
|
// SPDX-License-Identifier: Apache-2.0
|
||||||
|
|
||||||
|
package pagesarchive
|
||||||
|
|
||||||
|
import (
|
||||||
|
"archive/tar"
|
||||||
|
"archive/zip"
|
||||||
|
"bytes"
|
||||||
|
"compress/gzip"
|
||||||
|
"os"
|
||||||
|
"path/filepath"
|
||||||
|
"strings"
|
||||||
|
"testing"
|
||||||
|
|
||||||
|
"github.com/stretchr/testify/assert"
|
||||||
|
"github.com/stretchr/testify/require"
|
||||||
|
"github.com/ulikunitz/xz"
|
||||||
|
)
|
||||||
|
|
||||||
|
func TestDetectFormatFromName(t *testing.T) {
|
||||||
|
cases := map[string]Format{
|
||||||
|
"site.zip": FormatZip,
|
||||||
|
"site.TAR.GZ": FormatTarGz,
|
||||||
|
"site.tgz": FormatTarGz,
|
||||||
|
"site.tar.xz": FormatTarXz,
|
||||||
|
"site.txz": FormatTarXz,
|
||||||
|
"site.tar.bz2": FormatTarBz2,
|
||||||
|
"site.tar": FormatTar,
|
||||||
|
"site.7z": FormatSevenZip,
|
||||||
|
}
|
||||||
|
for name, want := range cases {
|
||||||
|
got, ok := DetectFormatFromName(name)
|
||||||
|
assert.True(t, ok, name)
|
||||||
|
assert.Equal(t, want, got, name)
|
||||||
|
}
|
||||||
|
_, ok := DetectFormatFromName("site.rar")
|
||||||
|
assert.False(t, ok)
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestInspectAndExtractZip(t *testing.T) {
|
||||||
|
data := testZip(t, map[string]string{
|
||||||
|
"dist/index.html": "<html>ok</html>",
|
||||||
|
"dist/app.js": "console.log(1)",
|
||||||
|
})
|
||||||
|
manifest, err := InspectBytes(data, FormatZip, InspectOptions{
|
||||||
|
EntryFile: "index.html",
|
||||||
|
Limits: Limits{MaxFiles: 100, MaxFileBytes: 1 << 20, MaxTotalBytes: 1 << 20},
|
||||||
|
})
|
||||||
|
require.NoError(t, err)
|
||||||
|
assert.Equal(t, 2, manifest.FileCount)
|
||||||
|
paths := make(map[string]struct{}, len(manifest.Files))
|
||||||
|
for _, file := range manifest.Files {
|
||||||
|
paths[file.Path] = struct{}{}
|
||||||
|
}
|
||||||
|
assert.Contains(t, paths, "index.html")
|
||||||
|
assert.Contains(t, paths, "app.js")
|
||||||
|
|
||||||
|
dest := t.TempDir()
|
||||||
|
require.NoError(t, ExtractBytes(data, FormatZip, dest, ExtractOptions{
|
||||||
|
StripCommonRoot: true,
|
||||||
|
EnforceLimits: true,
|
||||||
|
Limits: Limits{MaxFiles: 100, MaxFileBytes: 1 << 20, MaxTotalBytes: 1 << 20},
|
||||||
|
}))
|
||||||
|
body, err := os.ReadFile(filepath.Join(dest, "index.html")) //nolint:gosec
|
||||||
|
require.NoError(t, err)
|
||||||
|
assert.Equal(t, "<html>ok</html>", string(body))
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestExtractTrustedSkipsSizeLimits(t *testing.T) {
|
||||||
|
// Content larger than a tiny limit would fail if limits were enforced.
|
||||||
|
large := strings.Repeat("x", 64)
|
||||||
|
data := testZip(t, map[string]string{
|
||||||
|
"index.html": large,
|
||||||
|
})
|
||||||
|
dest := t.TempDir()
|
||||||
|
require.NoError(t, ExtractBytes(data, FormatZip, dest, ExtractOptions{
|
||||||
|
// Agent trusts control-plane validation: no size/count re-check.
|
||||||
|
EnforceLimits: false,
|
||||||
|
}))
|
||||||
|
body, err := os.ReadFile(filepath.Join(dest, "index.html")) //nolint:gosec
|
||||||
|
require.NoError(t, err)
|
||||||
|
assert.Equal(t, large, string(body))
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestInspectAndExtractTarGz(t *testing.T) {
|
||||||
|
data := testTarGz(t, map[string]string{
|
||||||
|
"index.html": "<html>tar</html>",
|
||||||
|
"style.css": "body{}",
|
||||||
|
})
|
||||||
|
format, err := DetectFormat("site.tar.gz", data)
|
||||||
|
require.NoError(t, err)
|
||||||
|
assert.Equal(t, FormatTarGz, format)
|
||||||
|
|
||||||
|
manifest, err := InspectBytes(data, format, InspectOptions{
|
||||||
|
EntryFile: "index.html",
|
||||||
|
Limits: Limits{MaxFiles: 100, MaxFileBytes: 1 << 20, MaxTotalBytes: 1 << 20},
|
||||||
|
})
|
||||||
|
require.NoError(t, err)
|
||||||
|
assert.Equal(t, 2, manifest.FileCount)
|
||||||
|
|
||||||
|
dest := t.TempDir()
|
||||||
|
require.NoError(t, ExtractBytes(data, format, dest, ExtractOptions{
|
||||||
|
EnforceLimits: true,
|
||||||
|
Limits: Limits{MaxFiles: 100, MaxFileBytes: 1 << 20, MaxTotalBytes: 1 << 20},
|
||||||
|
}))
|
||||||
|
body, err := os.ReadFile(filepath.Join(dest, "index.html")) //nolint:gosec
|
||||||
|
require.NoError(t, err)
|
||||||
|
assert.Equal(t, "<html>tar</html>", string(body))
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestInspectTarXz(t *testing.T) {
|
||||||
|
data := testTarXz(t, map[string]string{
|
||||||
|
"index.html": "<html>xz</html>",
|
||||||
|
})
|
||||||
|
manifest, err := InspectBytes(data, FormatTarXz, InspectOptions{
|
||||||
|
EntryFile: "index.html",
|
||||||
|
Limits: Limits{MaxFiles: 10, MaxFileBytes: 1 << 20, MaxTotalBytes: 1 << 20},
|
||||||
|
})
|
||||||
|
require.NoError(t, err)
|
||||||
|
assert.Equal(t, 1, manifest.FileCount)
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestRejectZipSlip(t *testing.T) {
|
||||||
|
data := testZip(t, map[string]string{
|
||||||
|
"../evil.txt": "x",
|
||||||
|
"index.html": "ok",
|
||||||
|
})
|
||||||
|
_, err := InspectBytes(data, FormatZip, InspectOptions{
|
||||||
|
EntryFile: "index.html",
|
||||||
|
Limits: Limits{MaxFiles: 10, MaxFileBytes: 1 << 20, MaxTotalBytes: 1 << 20},
|
||||||
|
})
|
||||||
|
require.Error(t, err)
|
||||||
|
}
|
||||||
|
|
||||||
|
func testZip(t *testing.T, files map[string]string) []byte {
|
||||||
|
t.Helper()
|
||||||
|
var buffer bytes.Buffer
|
||||||
|
writer := zip.NewWriter(&buffer)
|
||||||
|
for name, content := range files {
|
||||||
|
file, err := writer.Create(name)
|
||||||
|
require.NoError(t, err)
|
||||||
|
_, err = file.Write([]byte(content))
|
||||||
|
require.NoError(t, err)
|
||||||
|
}
|
||||||
|
require.NoError(t, writer.Close())
|
||||||
|
return buffer.Bytes()
|
||||||
|
}
|
||||||
|
|
||||||
|
func testTarGz(t *testing.T, files map[string]string) []byte {
|
||||||
|
t.Helper()
|
||||||
|
var buffer bytes.Buffer
|
||||||
|
gzWriter := gzip.NewWriter(&buffer)
|
||||||
|
tarWriter := tar.NewWriter(gzWriter)
|
||||||
|
for name, content := range files {
|
||||||
|
require.NoError(t, tarWriter.WriteHeader(&tar.Header{
|
||||||
|
Name: name,
|
||||||
|
Mode: 0o644,
|
||||||
|
Size: int64(len(content)),
|
||||||
|
}))
|
||||||
|
_, err := tarWriter.Write([]byte(content))
|
||||||
|
require.NoError(t, err)
|
||||||
|
}
|
||||||
|
require.NoError(t, tarWriter.Close())
|
||||||
|
require.NoError(t, gzWriter.Close())
|
||||||
|
return buffer.Bytes()
|
||||||
|
}
|
||||||
|
|
||||||
|
func testTarXz(t *testing.T, files map[string]string) []byte {
|
||||||
|
t.Helper()
|
||||||
|
var buffer bytes.Buffer
|
||||||
|
xzWriter, err := xz.NewWriter(&buffer)
|
||||||
|
require.NoError(t, err)
|
||||||
|
tarWriter := tar.NewWriter(xzWriter)
|
||||||
|
for name, content := range files {
|
||||||
|
require.NoError(t, tarWriter.WriteHeader(&tar.Header{
|
||||||
|
Name: name,
|
||||||
|
Mode: 0o644,
|
||||||
|
Size: int64(len(content)),
|
||||||
|
}))
|
||||||
|
_, err := tarWriter.Write([]byte(content))
|
||||||
|
require.NoError(t, err)
|
||||||
|
}
|
||||||
|
require.NoError(t, tarWriter.Close())
|
||||||
|
require.NoError(t, xzWriter.Close())
|
||||||
|
return buffer.Bytes()
|
||||||
|
}
|
||||||
@@ -0,0 +1,85 @@
|
|||||||
|
// Copyright 2026 Arctel.net
|
||||||
|
// SPDX-License-Identifier: Apache-2.0
|
||||||
|
|
||||||
|
package pagesarchive
|
||||||
|
|
||||||
|
import (
|
||||||
|
"fmt"
|
||||||
|
"path"
|
||||||
|
"path/filepath"
|
||||||
|
"strings"
|
||||||
|
)
|
||||||
|
|
||||||
|
// NormalizeEntryPath cleans an archive entry path and rejects zip-slip / absolute paths.
|
||||||
|
// skip=true means the entry should be ignored (empty path or directory marker).
|
||||||
|
func NormalizeEntryPath(raw string) (cleaned string, skip bool, err error) {
|
||||||
|
name := strings.TrimSpace(filepath.ToSlash(raw))
|
||||||
|
if name == "" {
|
||||||
|
return "", true, nil
|
||||||
|
}
|
||||||
|
if strings.HasSuffix(name, "/") {
|
||||||
|
return "", true, nil
|
||||||
|
}
|
||||||
|
if strings.HasPrefix(name, "/") || path.IsAbs(name) {
|
||||||
|
return "", false, fmt.Errorf("pages package contains absolute path: %s", raw)
|
||||||
|
}
|
||||||
|
// Reject Windows drive / UNC-style paths that may appear after ToSlash.
|
||||||
|
if len(name) >= 2 && name[1] == ':' {
|
||||||
|
return "", false, fmt.Errorf("pages package contains absolute path: %s", raw)
|
||||||
|
}
|
||||||
|
cleanedPath := path.Clean(name)
|
||||||
|
if cleanedPath == "." {
|
||||||
|
return "", true, nil
|
||||||
|
}
|
||||||
|
if cleanedPath == ".." || strings.HasPrefix(cleanedPath, "../") || strings.Contains(cleanedPath, "/../") {
|
||||||
|
return "", false, fmt.Errorf("pages package path escapes directory: %s", raw)
|
||||||
|
}
|
||||||
|
return cleanedPath, false, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// FindCommonRootPrefix returns a trailing-slash directory prefix shared by all file paths.
|
||||||
|
// When files do not share a single root folder the result is empty.
|
||||||
|
func FindCommonRootPrefix(paths []string) string {
|
||||||
|
var firstFilePath string
|
||||||
|
hasMultipleFiles := false
|
||||||
|
for _, item := range paths {
|
||||||
|
normalizedPath, skip, err := NormalizeEntryPath(item)
|
||||||
|
if err != nil || skip {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
if firstFilePath == "" {
|
||||||
|
firstFilePath = normalizedPath
|
||||||
|
} else {
|
||||||
|
hasMultipleFiles = true
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if firstFilePath == "" {
|
||||||
|
return ""
|
||||||
|
}
|
||||||
|
parts := strings.Split(firstFilePath, "/")
|
||||||
|
if len(parts) <= 1 {
|
||||||
|
return ""
|
||||||
|
}
|
||||||
|
commonPrefix := parts[0] + "/"
|
||||||
|
if !hasMultipleFiles {
|
||||||
|
return commonPrefix
|
||||||
|
}
|
||||||
|
for _, item := range paths {
|
||||||
|
normalizedPath, skip, err := NormalizeEntryPath(item)
|
||||||
|
if err != nil || skip {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
if !strings.HasPrefix(normalizedPath, commonPrefix) {
|
||||||
|
return ""
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return commonPrefix
|
||||||
|
}
|
||||||
|
|
||||||
|
// StripPrefix removes a directory prefix from a cleaned path when present.
|
||||||
|
func StripPrefix(normalizedPath, prefix string) string {
|
||||||
|
if prefix == "" {
|
||||||
|
return normalizedPath
|
||||||
|
}
|
||||||
|
return strings.TrimPrefix(normalizedPath, prefix)
|
||||||
|
}
|
||||||
Reference in New Issue
Block a user