mirror of
https://github.com/Rain-kl/OpenFlare.git
synced 2026-10-11 01:36:37 +08:00
merge: 合并 PR #22 Pages 部署源 V2 到 feat/pages-source-sync-v2
基于最新 main 合并 deqiying/feat/pages-source-sync-v2, 解决 docs/changelog/index.md 与限流相关条目的冲突。
This commit is contained in:
@@ -42,7 +42,7 @@ OpenFlare 是开源 CDN 编排与边缘安全平台。它支持反向代理、
|
|||||||
* **安全内网穿透(Tunnels)**:开源版的 Cloudflare Tunnels。无须公网 IP 或暴露入向端口,通过 Relay 中继节点与 OpenFlared 客户端安全反向穿透内网 Web 服务至公网。
|
* **安全内网穿透(Tunnels)**:开源版的 Cloudflare Tunnels。无须公网 IP 或暴露入向端口,通过 Relay 中继节点与 OpenFlared 客户端安全反向穿透内网 Web 服务至公网。
|
||||||
* **边缘 WAF 安全防护**:提供全局与自定义规则组,支持手动/自动/订阅型 IP 组、MaxMind GeoIP 国家级地域准入、IP 组成员 Checksum 差分同步(无需 Nginx 重载)以及自定义拦截响应。
|
* **边缘 WAF 安全防护**:提供全局与自定义规则组,支持手动/自动/订阅型 IP 组、MaxMind GeoIP 国家级地域准入、IP 组成员 Checksum 差分同步(无需 Nginx 重载)以及自定义拦截响应。
|
||||||
* **防 CC 与人机挑战(PoW)**:内置高性能客户端密码学 Proof of Work 挑战(类似 Turnstile),在网关边缘秒级拦截并阻断僵尸网络与爬虫。
|
* **防 CC 与人机挑战(PoW)**:内置高性能客户端密码学 Proof of Work 挑战(类似 Turnstile),在网关边缘秒级拦截并阻断僵尸网络与爬虫。
|
||||||
* **Pages 静态托管**:直接上传预构建 ZIP 包,由边缘 Agent 拉取并通过 OpenResty 本地提供服务,支持 SPA Fallback 与内置 API 反向代理配置。
|
* **Pages 静态托管**:支持上传或从受限 Remote URL、公开 GitHub Release asset 同步预构建产物;GitHub latest 可定时检查并可选自动发布。所有来源统一生成不可变部署,由边缘 Agent 拉取并通过 OpenResty 本地提供服务,支持回滚、SPA Fallback 与 API 反向代理。
|
||||||
* **TLS 证书自动化**:支持证书动态上传、多域名证书自动匹配绑定,以及通过 ACME 协议向 Let's Encrypt 自动申请与续期证书。
|
* **TLS 证书自动化**:支持证书动态上传、多域名证书自动匹配绑定,以及通过 ACME 协议向 Let's Encrypt 自动申请与续期证书。
|
||||||
* **Uptime Kuma 监控同步**:与 Uptime Kuma 集成,自动差分同步监控站点列表,实时感知节点存活与服务可用状态。
|
* **Uptime Kuma 监控同步**:与 Uptime Kuma 集成,自动差分同步监控站点列表,实时感知节点存活与服务可用状态。
|
||||||
* **SSO 单点登录**:支持 GitHub OAuth 与标准 OIDC 协议,无缝接入企业身份提供商实现统一登录。
|
* **SSO 单点登录**:支持 GitHub OAuth 与标准 OIDC 协议,无缝接入企业身份提供商实现统一登录。
|
||||||
|
|||||||
@@ -25,6 +25,7 @@ sidebar: false
|
|||||||
### 新增
|
### 新增
|
||||||
|
|
||||||
- 安全性新增「限流」设置:可为边缘站点配置默认并发与带宽;站点未设置时继承,填 `-1` 可显式关闭。
|
- 安全性新增「限流」设置:可为边缘站点配置默认并发与带宽;站点未设置时继承,填 `-1` 可显式关闭。
|
||||||
|
- Pages 项目新增持久部署源,可配置 Remote URL 或公开 GitHub Release,并支持手动检查、同步发布、来源状态查看与同一 Release 资源替换确认;GitHub latest 来源可按设定间隔自动检查并发布更新,部署历史会保留安全的来源快照。
|
||||||
|
|
||||||
### 改进
|
### 改进
|
||||||
|
|
||||||
@@ -32,6 +33,10 @@ sidebar: false
|
|||||||
- Agent Docker 部署命令默认挂载命名卷 `openflare-agent-pages` 持久化 Pages 目录,重建容器时无需重新拉取静态站点包。
|
- Agent Docker 部署命令默认挂载命名卷 `openflare-agent-pages` 持久化 Pages 目录,重建容器时无需重新拉取静态站点包。
|
||||||
- 限流页新增「分析」视图:默认展示近 24 小时请求压力(RPS)与独立访客双轴趋势(1 分钟桶),支持域名过滤与 24 小时/3 天预设,并按窗口平均 RPS 排行域名与 IP;原全局默认配置迁入「配置」页签。
|
- 限流页新增「分析」视图:默认展示近 24 小时请求压力(RPS)与独立访客双轴趋势(1 分钟桶),支持域名过滤与 24 小时/3 天预设,并按窗口平均 RPS 排行域名与 IP;原全局默认配置迁入「配置」页签。
|
||||||
|
|
||||||
|
### 修复
|
||||||
|
|
||||||
|
- 修复 Pages 部署包路径校验、归档展开限额、历史版本裁剪、代理路由绑定与 Agent 下载过程中的安全和一致性问题;大包改为流式处理,部署入口、旧版目录切换、保留版本及上传记录在并发场景下更加可靠,异常中断遗留的部署包也会被安全补偿清理。
|
||||||
|
|
||||||
## [v3.4.1] - 2026-07-19
|
## [v3.4.1] - 2026-07-19
|
||||||
|
|
||||||
### 新增
|
### 新增
|
||||||
|
|||||||
@@ -98,7 +98,7 @@ Agent 对数据面 OpenResty 的管控实现了端到端的闭环,包含配置
|
|||||||
* `certs/`:证书存放目录(文件命名为 `{cert_id}.crt` 和 `{cert_id}.key`)。
|
* `certs/`:证书存放目录(文件命名为 `{cert_id}.crt` 和 `{cert_id}.key`)。
|
||||||
* `waf/` 与 `pow/`:WAF 及防 CC 挑战所需的专用 Lua 运行时脚本。
|
* `waf/` 与 `pow/`:WAF 及防 CC 挑战所需的专用 Lua 运行时脚本。
|
||||||
* `waf_config.json` 与 `waf_ip_groups.json`:WAF 过滤引擎所需的结构化规则配置文件。
|
* `waf_config.json` 与 `waf_ip_groups.json`:WAF 过滤引擎所需的结构化规则配置文件。
|
||||||
* `pages_dir`:Pages 静态站点部署目录,默认位于 `data_dir/var/lib/openflare/pages`。当激活配置引用 Pages **项目**时,Agent 按 `project_id` 请求控制面「最新激活包」(hash + package,下载后再校验 hash 防竞态),解压到 `projects/{project_id}/releases/{hash}`,切换 `current` 后**立即删除同项目其它历史 release**(仅保留最新)。项目内切换激活无需重发主配置;多项目对账时单项目失败不阻塞其它项目。
|
* `pages_dir`:Pages 静态站点部署目录,默认位于 `data_dir/var/lib/openflare/pages`。当激活配置引用 Pages **项目**时,Agent 按 `project_id` 请求控制面「最新激活包」(hash + package),以流式方式写入临时文件并执行实际响应上限与 SHA-256 校验,再安全解压到 `projects/{project_id}/releases/{hash}`。解压后会复核文件数与总字节,绝对防御上限为 2 GiB 包、1,000 个文件、单文件及总量 8 GiB;随后原子切换 `current` 并**立即删除同项目其它历史 release**(仅保留最新)。项目内切换激活无需重发主配置;多项目对账时单项目失败不阻塞其它项目。
|
||||||
|
|
||||||
### 2. 精细化的重载动作
|
### 2. 精细化的重载动作
|
||||||
1. **备份当前配置**:在写入新文件之前,Agent 会将现有的配置文件复制到 `.backup` 临时目录下,保留完整的现场快照。
|
1. **备份当前配置**:在写入新文件之前,Agent 会将现有的配置文件复制到 `.backup` 临时目录下,保留完整的现场快照。
|
||||||
@@ -174,3 +174,4 @@ graph TD
|
|||||||
2. **严格的 Token 过滤与前缀验证**:Agent 侧向 Server 请求资源时,接口端点固定以 `/api/v1/agent/` 为前缀,并强制携带 `X-Agent-Token` 进行签名或令牌核验。
|
2. **严格的 Token 过滤与前缀验证**:Agent 侧向 Server 请求资源时,接口端点固定以 `/api/v1/agent/` 为前缀,并强制携带 `X-Agent-Token` 进行签名或令牌核验。
|
||||||
3. **节点自治原则**:Agent 须具备完备的离线工作能力。在与 Server 失去连接期间,本地 OpenResty 必须依靠本地已落地的配置保持反向代理服务的绝对正常运行。
|
3. **节点自治原则**:Agent 须具备完备的离线工作能力。在与 Server 失去连接期间,本地 OpenResty 必须依靠本地已落地的配置保持反向代理服务的绝对正常运行。
|
||||||
4. **观测只上报事实**:访问日志以明细形式上送;主机指标上报计数器/瞬时读数。禁止在 Agent 内计算业务 UV、Top 域名、24h 已提供数据等结论性指标(由 Server 聚合)。详见 [边缘可观测与业务流量统计](./observability-design.md)。
|
4. **观测只上报事实**:访问日志以明细形式上送;主机指标上报计数器/瞬时读数。禁止在 Agent 内计算业务 UV、Top 域名、24h 已提供数据等结论性指标(由 Server 聚合)。详见 [边缘可观测与业务流量统计](./observability-design.md)。
|
||||||
|
5. **Pages 只消费控制面产物**:Remote URL、GitHub Release、自动 scanner,以及未来仓库 checkout/build executor 均属于 Server 职责。Agent 不接收外部 URL、访问令牌、仓库凭据或任意 clone/install/build 命令,只拉取已经激活且带完整性元数据的部署包。
|
||||||
|
|||||||
@@ -80,7 +80,7 @@ OpenResty (Agent, TLS/WAF)
|
|||||||
* 提供管理端 REST API(`/api/v1/d/*`),通过 **Session Cookie** 鉴权,可选 `X-Access-Token` 访问令牌。
|
* 提供管理端 REST API(`/api/v1/d/*`),通过 **Session Cookie** 鉴权,可选 `X-Access-Token` 访问令牌。
|
||||||
* 边缘节点协议走 `/api/v1/agent|relay|tunnel/*`,分别使用 `X-Agent-Token` / `X-Tunnel-Token` 鉴权。
|
* 边缘节点协议走 `/api/v1/agent|relay|tunnel/*`,分别使用 `X-Agent-Token` / `X-Tunnel-Token` 鉴权。
|
||||||
* 包含配置编译器(Compiler),将数据库中的规则、证书与全局参数统一编译为不可变的配置快照及 OpenResty 物理配置文件文本。
|
* 包含配置编译器(Compiler),将数据库中的规则、证书与全局参数统一编译为不可变的配置快照及 OpenResty 物理配置文件文本。
|
||||||
* 存储 Pages 部署 ZIP 包于本地 Artifacts 目录,并向 Agent 提供受控的下载接口。
|
* 统一接收 Pages 本地上传、Remote URL 与公开 GitHub Release 预构建产物,完成来源检查、受限下载、归档校验和不可变 deployment;manual 上传生成待显式激活的 candidate,持久来源 sync 才 create-or-load 并原子激活。Server 向 Agent 提供受控的 latest 下载接口;内部 scanner 负责 GitHub latest 的限量检查、租约恢复、可选自动发布与孤儿上传记录补偿,通用任务管理入口不能修改该排程。未来仓库源码构建由独立 Server build executor 扩展,Agent 不执行第三方拉取或构建命令。
|
||||||
* 后台集成 Uptime Kuma 监控同步服务,自动为可用站点维护 HTTP 探测任务。
|
* 后台集成 Uptime Kuma 监控同步服务,自动为可用站点维护 HTTP 探测任务。
|
||||||
* 启动入口为根目录 `main.go` + `internal/cmd/`(`api` / `worker` / `scheduler` / `all`);OpenFlare 业务在 `internal/apps/openflare/`,边缘协议处理在 `internal/apps/openflare/{agent,relay,flared}/`。
|
* 启动入口为根目录 `main.go` + `internal/cmd/`(`api` / `worker` / `scheduler` / `all`);OpenFlare 业务在 `internal/apps/openflare/`,边缘协议处理在 `internal/apps/openflare/{agent,relay,flared}/`。
|
||||||
* *详细设计请参阅:[Agent 与发布模型设计](./agent-design.md) 以及 [Uptime Kuma 监控同步设计](./kuma-design.md)*
|
* *详细设计请参阅:[Agent 与发布模型设计](./agent-design.md) 以及 [Uptime Kuma 监控同步设计](./kuma-design.md)*
|
||||||
@@ -126,6 +126,7 @@ OpenResty (Agent, TLS/WAF)
|
|||||||
### 2. 静态托管与 API 代理流
|
### 2. 静态托管与 API 代理流
|
||||||
* 静态资源解压落地于 Agent 节点的 `projects/{project_id}/current` 下(按项目 latest 拉取,仅保留最新包),OpenResty 通过 `root`/`index`/`try_files` 在边缘直接提供静态资源服务。
|
* 静态资源解压落地于 Agent 节点的 `projects/{project_id}/current` 下(按项目 latest 拉取,仅保留最新包),OpenResty 通过 `root`/`index`/`try_files` 在边缘直接提供静态资源服务。
|
||||||
* 当启用 API 代理时,OpenResty 自动根据站点配置的 `api_proxy_path`(如 `/api`)将 API 请求重写并转发(`proxy_pass`)给后端动态接口。
|
* 当启用 API 代理时,OpenResty 自动根据站点配置的 `api_proxy_path`(如 `/api`)将 API 请求重写并转发(`proxy_pass`)给后端动态接口。
|
||||||
|
* 管理员操作和内部 scanner 都只生成受约束的 artifact candidate,并复用统一 inspect、`upload.Ingest` 与 deployment pipeline。manual 上传创建新的未激活 candidate;持久来源 sync/scanner 才 create-or-load 并原子激活。未来 repository build executor 也只能向同一 artifact pipeline 输出产物;Agent 始终只是 active deployment 消费者。
|
||||||
* *部署包校验、解压逃逸防御及 Nginx 规则渲染详见:[Pages 静态托管设计文档](./pages-design.md)*
|
* *部署包校验、解压逃逸防御及 Nginx 规则渲染详见:[Pages 静态托管设计文档](./pages-design.md)*
|
||||||
|
|
||||||
### 3. WAF 安全过滤流
|
### 3. WAF 安全过滤流
|
||||||
@@ -160,7 +161,7 @@ OpenResty 健康与连接数 --> 边缘健康(瞬时,不作 24h 业务总量
|
|||||||
当前系统核心实体包括:
|
当前系统核心实体包括:
|
||||||
|
|
||||||
* **反代与配置**:`zones` (根域管理边界), `zone_domains` (明确域名与证书/路由关联), `proxy_routes` (路由策略), `origins` (源站), `config_versions` (配置版本), `tls_certificates` (证书). 详见 [Zone 与域名资源设计](./zone-design.md)。
|
* **反代与配置**:`zones` (根域管理边界), `zone_domains` (明确域名与证书/路由关联), `proxy_routes` (路由策略), `origins` (源站), `config_versions` (配置版本), `tls_certificates` (证书). 详见 [Zone 与域名资源设计](./zone-design.md)。
|
||||||
* **Pages 静态托管**:`pages_projects` (Pages项目), `pages_deployments` (不可变部署), `pages_deployment_files` (部署文件清单).
|
* **Pages 静态托管**:`of_pages_projects` (Pages项目), `of_pages_project_sources` / `of_pages_project_source_runtime` (可变来源配置与运行态), `of_pages_deployments` (不可变部署), `of_pages_deployment_files` (部署文件清单).
|
||||||
* **节点与穿透**:`nodes` (节点), `tunnels` (隧道客户端), `node_system_profiles` (系统概况), `apply_logs` (应用日志).
|
* **节点与穿透**:`nodes` (节点), `tunnels` (隧道客户端), `node_system_profiles` (系统概况), `apply_logs` (应用日志).
|
||||||
* **WAF 与安全**:`waf_rule_groups` (WAF规则组), `waf_ip_groups` (WAF IP组), `waf_rule_group_bindings` (网站WAF绑定).
|
* **WAF 与安全**:`waf_rule_groups` (WAF规则组), `waf_ip_groups` (WAF IP组), `waf_rule_group_bindings` (网站WAF绑定).
|
||||||
* **系统与账号**:`acme_accounts` (ACME账户), `dns_accounts` (DNS账户), `geoip_update_configs` (GeoIP更新配置).
|
* **系统与账号**:`acme_accounts` (ACME账户), `dns_accounts` (DNS账户), `geoip_update_configs` (GeoIP更新配置).
|
||||||
@@ -179,6 +180,7 @@ OpenResty 健康与连接数 --> 边缘健康(瞬时,不作 24h 业务总量
|
|||||||
| 运行时配置与控制库解耦 | WAF 规则发布时编译并随 OpenResty reload 加载;动态 IP 组通过 checksum 驱动的内存快照独立刷新 |
|
| 运行时配置与控制库解耦 | WAF 规则发布时编译并随 OpenResty reload 加载;动态 IP 组通过 checksum 驱动的内存快照独立刷新 |
|
||||||
| 业务流量以访问日志为唯一真相 | Agent 禁止业务预聚合;看板与 Zone 共用 Server 侧聚合,避免 openresty_tx 与 bytes_sent 双轨 |
|
| 业务流量以访问日志为唯一真相 | Agent 禁止业务预聚合;看板与 Zone 共用 Server 侧聚合,避免 openresty_tx 与 bytes_sent 双轨 |
|
||||||
| 业务交付 / 边缘健康 / 主机资源分层 | 已提供数据≠宿主机网卡出站≠OpenResty 连接数,UI 与 API 分名分区 |
|
| 业务交付 / 边缘健康 / 主机资源分层 | 已提供数据≠宿主机网卡出站≠OpenResty 连接数,UI 与 API 分名分区 |
|
||||||
|
| Pages artifact 与仓库构建分离 | 现有来源只导入预构建产物;未来 checkout/build 由 Server 隔离 executor 完成并复用 artifact pipeline,Agent 不执行第三方构建 |
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
|
|||||||
@@ -27,7 +27,7 @@ OpenFlare 适合需要统一管理多台 OpenResty 代理节点的团队,具
|
|||||||
| **配置版本控制** | 支持全局单一激活版本的预览、发布、不可变快照历史与秒级一键回滚 | [Agent 与发布模型](./agent-design.md) |
|
| **配置版本控制** | 支持全局单一激活版本的预览、发布、不可变快照历史与秒级一键回滚 | [Agent 与发布模型](./agent-design.md) |
|
||||||
| **WAF 安全防护** | 支持可视化 DAG 编排规则、手动/自动/订阅型 IP 组、GeoIP 匹配与 PoW CC 防护 | [WAF 设计](./waf-design.md) / [WAF 可编排规则设计](./waf-orchestration-design.md) / [WAF 使用指南](../guide/waf-usage.md) |
|
| **WAF 安全防护** | 支持可视化 DAG 编排规则、手动/自动/订阅型 IP 组、GeoIP 匹配与 PoW CC 防护 | [WAF 设计](./waf-design.md) / [WAF 可编排规则设计](./waf-orchestration-design.md) / [WAF 使用指南](../guide/waf-usage.md) |
|
||||||
| **内网穿透** | 通过中继节点(Relay)与内网客户端(OpenFlared),反向穿透暴露内网 Web 服务 | [内网穿透设计](./tunnel-design.md) / [穿透使用指南](../guide/tunnel-usage.md) |
|
| **内网穿透** | 通过中继节点(Relay)与内网客户端(OpenFlared),反向穿透暴露内网 Web 服务 | [内网穿透设计](./tunnel-design.md) / [穿透使用指南](../guide/tunnel-usage.md) |
|
||||||
| **Pages 静态托管** | 直接上传前端压缩包(zip / tar.gz / tar.xz / 7z 等),由边缘节点拉取并由 OpenResty 本地服务,支持 API 反代与 SPA Fallback | [Pages 静态托管设计](./pages-design.md) |
|
| **Pages 静态托管** | 支持上传或从 Remote URL、公开 GitHub Release 同步预构建产物;GitHub latest 可定时检查并可选自动发布。不可变部署由边缘节点拉取并由 OpenResty 本地服务,支持回滚、API 反代与 SPA Fallback | [Pages 静态托管设计](./pages-design.md) / [Pages 使用指南](../guide/pages-usage.md) |
|
||||||
| **TLS 证书自动续期** | 将证书显式绑定到 Zone 域名,并通过 ACME 协议向 Let's Encrypt 申请/续期证书 | [Zone 与域名资源设计](./zone-design.md) |
|
| **TLS 证书自动续期** | 将证书显式绑定到 Zone 域名,并通过 ACME 协议向 Let's Encrypt 申请/续期证书 | [Zone 与域名资源设计](./zone-design.md) |
|
||||||
| **多节点监控与观测** | 访问日志为业务流量唯一真相;Agent 只上报明细与主机读数,Server 统一聚合;与 Zone/看板对账 | [观测数据传输模型](./observability-transport-model.md) / [边缘可观测与业务流量统计](./observability-design.md) / [上报协议与表结构](./observability-data-model.md) / [系统架构](./architecture.md) |
|
| **多节点监控与观测** | 访问日志为业务流量唯一真相;Agent 只上报明细与主机读数,Server 统一聚合;与 Zone/看板对账 | [观测数据传输模型](./observability-transport-model.md) / [边缘可观测与业务流量统计](./observability-design.md) / [上报协议与表结构](./observability-data-model.md) / [系统架构](./architecture.md) |
|
||||||
|
|
||||||
@@ -53,8 +53,9 @@ OpenFlare 适合需要统一管理多台 OpenResty 代理节点的团队,具
|
|||||||
* **Tunnel 与 Node 体系隔离**:Tunnel 客户端在内网发起出向建连,与控制面托管的边缘 Node(公网节点)是独立的实体,使用专属的 `tunnel_token` 进行鉴权。
|
* **Tunnel 与 Node 体系隔离**:Tunnel 客户端在内网发起出向建连,与控制面托管的边缘 Node(公网节点)是独立的实体,使用专属的 `tunnel_token` 进行鉴权。
|
||||||
|
|
||||||
### 4. Pages 静态托管边界
|
### 4. Pages 静态托管边界
|
||||||
* **Direct Upload 托管模式**:仅支持直接上传预构建的 ZIP 静态资源包。不支持外部 Git 仓库自动构建、边缘 Serverless 函数、动态 SSR 服务或生成的二级预览域名。
|
* **预构建产物来源**:项目可保持手动上传,或配置一个 Remote URL / 公开 GitHub Release asset 来源。Remote 与固定 tag 只支持手动操作;只有 GitHub latest 进入定时检查并可选择自动更新。来源可切换,但不可变 deployment 与当前生产版本不会随 source 编辑或删除而丢失。
|
||||||
* **包体硬上限限制**:为了保障边缘节点安全,ZIP 压缩包体最大 25 MiB,解压文件树不超过 1,000 个且总体积不超过 100 MiB。禁止上传含有任何软链接或目录跨越(Zip-Slip)的安全高危压缩包。
|
* **归档与资源上限**:支持 `zip`、`tar.gz` / `tgz`、`tar.xz` / `txz`、`tar.bz2` / `tbz2`、`tar`、`7z`。压缩包上限由 `pages_max_package_size_mb` 控制(默认 100 MiB,范围 1~2048);展开后的单文件和总量上限为包上限的 4 倍且最低 100 MiB,最多 1,000 个常规文件。Server 与 Agent 都校验实际字节,并拒绝路径逃逸、软/硬链接与特殊文件。
|
||||||
|
* **构建与运行时边界**:当前不从外部 Git 仓库拉取源码或执行构建,也不提供边缘 Serverless、动态 SSR 或二级预览域名。未来仓库集成必须使用独立 `git_repository` Provider 与 Server 侧隔离 build executor,只向统一 artifact 管线输出受限产物;Agent 不接收仓库凭据、外部 URL 或 clone/install/build 命令。
|
||||||
|
|
||||||
### 5. 系统与版本边界
|
### 5. 系统与版本边界
|
||||||
* **全局单一激活版本**:所有节点拉取并消费同一份全局激活配置。不进行按节点分组的差异化配置发布。
|
* **全局单一激活版本**:所有节点拉取并消费同一份全局激活配置。不进行按节点分组的差异化配置发布。
|
||||||
|
|||||||
+74
-51
@@ -12,20 +12,31 @@
|
|||||||
2. **多节点分发困难**:当控制面管理多台边缘节点时,将静态文件同步分发到所有节点,并确保文件一致性,需要维护复杂的同步脚本(如 rsync 等)。
|
2. **多节点分发困难**:当控制面管理多台边缘节点时,将静态文件同步分发到所有节点,并确保文件一致性,需要维护复杂的同步脚本(如 rsync 等)。
|
||||||
3. **回滚缺乏一致性**:一旦新前端包发布失败或存在严重缺陷,不仅要恢复静态文件,还要恢复对应的反代规则,很难做到原子回滚。
|
3. **回滚缺乏一致性**:一旦新前端包发布失败或存在严重缺陷,不仅要恢复静态文件,还要恢复对应的反代规则,很难做到原子回滚。
|
||||||
|
|
||||||
为了解决这些问题,OpenFlare 引入了受 Cloudflare Pages 启发的 **Pages 静态托管** 功能。该功能将“前端部署包上传”与“网站代理规则配置”合二为一,依托 OpenFlare 的 pull-based(拉取式)协同架构,实现静态文件分发与反代配置发布的强一致性、不可变性与一键秒级回滚。
|
为了解决这些问题,OpenFlare 引入了受 Cloudflare Pages 启发的 **Pages 静态托管** 功能。该功能将“预构建产物导入”与“网站代理规则配置”纳入同一控制面,依托 OpenFlare 的 pull-based(拉取式)协同架构,以不可变 deployment、单节点原子切换和周期对账实现多 Agent 最终收敛,并支持快速回滚。
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## 核心功能
|
## 核心功能
|
||||||
|
|
||||||
Pages 静态托管子系统包含以下核心能力:
|
Pages 静态托管子系统包含以下核心能力:
|
||||||
* **Direct Upload 部署模式**:支持直接上传预构建的静态资源压缩包(`zip`、`tar.gz`、`tar.xz`、`tar.bz2`、`tar`、`7z`),或填写 URL 由控制面代为下载导入(允许内网地址与自签证书),省去复杂的 Git 集成和构建环境依赖。
|
* **预构建产物部署**:支持直接上传静态资源压缩包,也可为项目保存一个 Remote URL 或公开 GitHub Release asset 来源。外部来源只由 Server 访问,成功同步后统一创建或复用不可变 deployment 并原子激活。
|
||||||
* **不可变部署快照**:每次上传产生一个带唯一 ID 和 SHA-256 Checksum 的不可变部署记录。支持按系统配置保留最近 N 个历史部署,并可随时激活和回滚。
|
* **不可变部署快照**:本地上传每次创建新的候选 deployment;持久来源同步按 source identity/revision 创建或复用 deployment 并激活。所有部署都有唯一 ID 和整包 SHA-256,支持按系统配置保留最近 N 个历史版本并随时回滚。
|
||||||
|
* **检查与自动更新**:GitHub latest 可按项目间隔定时检查;默认只提示可用更新,管理员显式开启后才按检查到的精确 revision 自动同步并发布。
|
||||||
* **SPA Fallback 支持**:支持对单页应用(SPA)进行 Fallback 路由配置,请求找不到静态文件时自动重定向到入口文件。
|
* **SPA Fallback 支持**:支持对单页应用(SPA)进行 Fallback 路由配置,请求找不到静态文件时自动重定向到入口文件。
|
||||||
* **内置 API 反代服务**:支持在 Pages 规则内一键启用 API 代理,消除跨域问题,将请求转发给指定的后端服务。
|
* **内置 API 反代服务**:支持在 Pages 规则内一键启用 API 代理,消除跨域问题,将请求转发给指定的后端服务。
|
||||||
* **安全包校验与解压缩**:内置路径逃逸防御、防软链接劫持、文件大小/数量上限与可配置上传包体积控制,保障节点物理安全。
|
* **安全包校验与解压缩**:内置路径逃逸防御、防软链接劫持、文件大小/数量上限与可配置上传包体积控制,保障节点物理安全。
|
||||||
* **可配置限额**:管理员可在运维设置中调整「部署包大小上限」与「历史部署保留数」。
|
* **可配置限额**:管理员可在运维设置中调整「部署包大小上限」与「历史部署保留数」。
|
||||||
|
|
||||||
|
### 部署源与未来构建边界
|
||||||
|
|
||||||
|
项目当前支持 manual、Remote URL、GitHub Release 三种来源视图。无 source 记录即 manual;切换或删除 source 不删除历史 deployment,也不改变当前 active deployment。Remote URL 只允许手动“同步并发布”;GitHub Release 支持 latest/tag 手动检查与同步,只有 latest 可选择定时检查和自动更新。
|
||||||
|
|
||||||
|
source 是可变配置,deployment 是不可变事实。source 配置与运行态游标、状态、租约分别存储;deployment 只保存创建时的安全 provenance 快照。所有产物都复用“下载或接收产物 → 真实字节与入口校验 → `upload.Ingest` → deployment”的 artifact pipeline:manual 上传停在 candidate,等待管理员显式激活;持久来源 sync 才在同一业务事务中 create-or-load 并原子激活。Agent 只消费 active deployment,不感知来源类型。
|
||||||
|
|
||||||
|
后续从 Git 仓库拉取源码并自动构建时,将新增独立 `git_repository` provider 与隔离的 build executor。它输出受限的预构建产物后继续复用上述导入管线;不得把 clone、依赖安装或任意构建命令下发给 Agent,也不得把 branch/build/env 字段塞入现有 `github_release` source。当前 V2 不增加这些未来字段或空任务,只稳定 provider 输出、source discriminated view 与 deployment provenance 三个扩展边界。
|
||||||
|
|
||||||
|
管理端信息架构参考 Cloudflare Pages 当前把 [Git integration](https://developers.cloudflare.com/pages/configuration/git-integration/) 与 [Direct Upload](https://developers.cloudflare.com/pages/get-started/direct-upload/) 分离、并统一展示生产状态与历史部署的方式:OpenFlare 项目详情按“当前生产部署 → 部署源 → 部署历史”组织。OpenFlare 仍允许切换来源并保留历史部署,不采用 Cloudflare 项目创建后来源不可切换的限制。
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## Pages 静态托管架构
|
## Pages 静态托管架构
|
||||||
@@ -40,35 +51,41 @@ graph TD
|
|||||||
OpenResty -->|2. 转发 API 代理| BackEnd[4. 后端 API 服务]
|
OpenResty -->|2. 转发 API 代理| BackEnd[4. 后端 API 服务]
|
||||||
|
|
||||||
%% 控制流与心跳
|
%% 控制流与心跳
|
||||||
Server[OpenFlare Server 控制面] <-->|Agent API / Heartbeat| Agent[openflare-agent 进程]
|
Admin[管理员 / CI] -->|上传或配置来源| Server[OpenFlare Server 控制面]
|
||||||
Server -.->|5. 存储部署包| LocalStore[(Server 本地存储)]
|
Providers[Remote / GitHub Provider] -->|受限 artifact candidate| Server
|
||||||
|
Scanner[内部 scanner / action task] -->|检查与自动同步| Server
|
||||||
|
Server <-->|Agent API / Heartbeat| Agent[openflare-agent 进程]
|
||||||
|
Server -.->|统一 upload.Ingest| UploadStore[(平台 upload backend)]
|
||||||
|
|
||||||
Agent -->|1. 发现新版本| Server
|
Agent -->|1. 发现新版本| Server
|
||||||
Agent -->|2. 下载部署包| Server
|
Agent -->|2. 下载部署包| Server
|
||||||
Agent -->|3. 校验并解压缩| StaticFiles
|
Agent -->|3. 校验、解压并原子切换| StaticFiles
|
||||||
Agent -->|4. 应用并 Reload| OpenResty
|
|
||||||
|
|
||||||
style Browser fill:#f9f,stroke:#333,stroke-width:2px
|
style Browser fill:#f9f,stroke:#333,stroke-width:2px
|
||||||
style StaticFiles fill:#9f9,stroke:#333,stroke-width:2px
|
style StaticFiles fill:#9f9,stroke:#333,stroke-width:2px
|
||||||
style Server fill:#f96,stroke:#333,stroke-width:2px
|
style Server fill:#f96,stroke:#333,stroke-width:2px
|
||||||
```
|
```
|
||||||
|
|
||||||
* **控制面(Control Plane)**:Server 接收前端上传的部署包,并将包存储于本地磁盘,元数据写入数据库。配置发布时,编译出带有 `pages_deployment` 详情的不可变全局版本快照。
|
* **控制面(Control Plane)**:Server 接收本地上传,或通过受限 Provider 获取 Remote/GitHub 预构建产物;action task 与内部 scanner 负责检查、同步和自动更新。所有产物经统一 inspect 与 `upload.Ingest` 写入平台存储后端;manual 上传创建新的 candidate,持久来源 sync 则 create-or-load deployment 并原子激活。配置发布时只编译稳定的项目锚点与静态服务元数据。
|
||||||
* **数据面(Data Plane)**:Agent 在心跳同步中发现版本更新并引用了 Pages 部署,通过专属 API 下载对应的部署包并执行校验解压缩。OpenResty 拦截域名请求,在本地提供静态文件服务。
|
* **数据面(Data Plane)**:Agent 在心跳/WS 对账中发现配置引用的 Pages 项目,通过专属 API 拉取该项目当前激活包并执行校验解压缩。OpenResty 在本地提供静态文件服务;Agent 不感知产物来自上传、Remote、GitHub 或未来 build executor。
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## 数据模型与元数据设计
|
## 数据模型与元数据设计
|
||||||
|
|
||||||
### 1. 核心数据库实体
|
### 1. 核心数据库实体
|
||||||
* **Pages 项目 (`pages_projects`)**:
|
* **Pages 项目 (`of_pages_projects`)**:
|
||||||
* 记录项目的业务名称、Slug 标识(URL 友好型)、启用状态、静态服务根目录(RootDir,可为空)、入口文件名(EntryFile,默认 `index.html`)、SPA Fallback 设置,以及 API 反向代理配置(APIProxyPath, APIProxyPass, APIProxyRewrite)。
|
* 记录项目的业务名称、Slug 标识(URL 友好型)、启用状态、静态服务根目录(RootDir,可为空)、入口文件名(EntryFile,默认 `index.html`)、SPA Fallback 设置,以及 API 反向代理配置(APIProxyPath, APIProxyPass, APIProxyRewrite)。
|
||||||
* **Pages 部署 (`pages_deployments`)**:
|
* **部署源配置 (`of_pages_project_sources`)**:
|
||||||
* 记录单次上传生成的不可变快照。包含:部署号 (DeploymentNumber, 递增序列)、SHA-256 Checksum 校验和、部署状态 (uploaded/active)、部署包的本地存储路径、解压后的文件数与总字节数。
|
* 每个项目最多一条可变来源配置,使用 `source_type` 区分 Remote URL 与 GitHub Release。`config_version` 用于 fence 旧任务;Remote 完整 URL 只保存在配置表中,不会进入响应、日志、任务 payload 或 deployment provenance。V2 不承诺数据库列加密。
|
||||||
* **部署文件清单 (`pages_deployment_files`)**:
|
* **部署源运行态 (`of_pages_project_source_runtime`)**:
|
||||||
* 存储每次部署的完整静态文件树路径与文件大小(来自压缩包声明的未压缩大小),供控制台展示与统计。
|
* 与 source 1:1 保存 ETag、seen/applied revision、最近检查/同步、下次检查、错误和 lease。状态固定为 `idle | checking | update_available | syncing | failed | attention`,排队/完成状态由 `TaskExecution` 承担。
|
||||||
* 不再为包内每个文件计算内容哈希;完整性由**整包** SHA-256(`pages_deployments.checksum`)保证,Agent 拉取时只校验整包 hash。
|
* **Pages 部署 (`of_pages_deployments`)**:
|
||||||
* 控制面 inspect 通过文件句柄 / 随机访问读取归档索引(zip/7z 中央目录;tar 流式读 header 并丢弃 body),避免将整包 `ReadFile` 进内存,也避免逐文件解压算 hash。
|
* 记录不可变部署事实:项目内递增部署号、整包 SHA-256、`upload_id`、文件数/总字节、创建者,以及可空的 source identity/revision、来源安全快照与 trigger。`artifact_path` 仅为旧数据兼容字段,不再是新部署的存储真相。
|
||||||
|
* **部署文件清单 (`of_pages_deployment_files`)**:
|
||||||
|
* 存储每次部署的完整常规文件路径与实际字节数,供控制台展示与统计。
|
||||||
|
* 不再为包内每个文件计算内容哈希;完整性由**整包** SHA-256(`of_pages_deployments.checksum`)保证,Agent 拉取时校验整包 hash。
|
||||||
|
* 控制面 inspect 通过文件句柄读取归档,流式消费每个常规文件体并核对声明大小与实际字节,避免将整包 `ReadFile` 进内存,也避免逐文件落盘计算 hash。
|
||||||
|
|
||||||
### 2. 路由关联与快照
|
### 2. 路由关联与快照
|
||||||
`proxy_routes` 路由规则通过 `upstream_type = "pages"` 及 `pages_project_id` 关联 Pages 项目。当路由类型为 `pages` 且该项目存在已激活的部署时,才允许将该路由加入发布流程。
|
`proxy_routes` 路由规则通过 `upstream_type = "pages"` 及 `pages_project_id` 关联 Pages 项目。当路由类型为 `pages` 且该项目存在已激活的部署时,才允许将该路由加入发布流程。
|
||||||
@@ -87,7 +104,7 @@ graph TD
|
|||||||
"api_proxy_path": "/api",
|
"api_proxy_path": "/api",
|
||||||
"api_proxy_pass": "http://api.internal:8000",
|
"api_proxy_pass": "http://api.internal:8000",
|
||||||
"api_proxy_rewrite": "/api/(.*) /$1",
|
"api_proxy_rewrite": "/api/(.*) /$1",
|
||||||
"local_root": "__OPENFLARE_PAGES_DIR__/deployments/12/current"
|
"local_root": "__OPENFLARE_PAGES_DIR__/projects/1/current"
|
||||||
}
|
}
|
||||||
```
|
```
|
||||||
|
|
||||||
@@ -98,7 +115,7 @@ graph TD
|
|||||||
* Agent 按项目请求「最新激活包」(类似 `github/release/latest`):
|
* Agent 按项目请求「最新激活包」(类似 `github/release/latest`):
|
||||||
* `GET /api/v1/agent/pages/projects/:project_id/latest/hash`
|
* `GET /api/v1/agent/pages/projects/:project_id/latest/hash`
|
||||||
* `GET /api/v1/agent/pages/projects/:project_id/latest/package`
|
* `GET /api/v1/agent/pages/projects/:project_id/latest/package`
|
||||||
* 控制面根据该项目**当前激活部署**返回哈希与压缩包;Agent 不关心具体 deployment_id。
|
* 控制面根据该项目**当前激活部署**返回 deployment ID、哈希、包大小与展开清单元数据。Agent 用 deployment ID 与其它 latest 元数据识别下载期间的指针竞态,但主配置和本地目录的稳定锚点仍是 project ID。
|
||||||
* 因此:在项目内切换激活部署后,**不必发布主配置**;Agent 在周期性对账时轮询 latest hash,发现变化即下载并切换 `current`。
|
* 因此:在项目内切换激活部署后,**不必发布主配置**;Agent 在周期性对账时轮询 latest hash,发现变化即下载并切换 `current`。
|
||||||
* 快照中的 `pages_deployment` 字段仍可记录发布时元数据(入口文件、SPA/API 代理等),但不作为 Agent 拉包的版本锁定。
|
* 快照中的 `pages_deployment` 字段仍可记录发布时元数据(入口文件、SPA/API 代理等),但不作为 Agent 拉包的版本锁定。
|
||||||
|
|
||||||
@@ -107,40 +124,49 @@ graph TD
|
|||||||
## Server 端 (控制面) 职责与生命周期
|
## Server 端 (控制面) 职责与生命周期
|
||||||
|
|
||||||
### 1. 部署包安全校验与分析
|
### 1. 部署包安全校验与分析
|
||||||
为了避免不可信的用户上传恶意压缩包攻击服务器,控制面在 `UploadDeployment` 时执行严格校验:
|
为了避免不可信产物攻击服务器,控制面对本地上传和所有外部来源执行同一套严格校验:
|
||||||
* **格式支持**:`zip`、`tar.gz` / `tgz`、`tar.xz` / `txz`、`tar.bz2` / `tbz2`、`tar`、`7z`。
|
* **格式支持**:`zip`、`tar.gz` / `tgz`、`tar.xz` / `txz`、`tar.bz2` / `tbz2`、`tar`、`7z`。
|
||||||
* **大小限制**:压缩包体积由系统配置 `pages_max_package_size_mb` 控制(默认 100 MiB,范围 1~2048);展开后总体积上限为「包大小 × 4」且不低于 100 MiB(按归档**声明**的未压缩大小累计,默认不流式重读每个文件内容)。
|
* **大小限制**:压缩包体积由系统配置 `pages_max_package_size_mb` 控制(默认 100 MiB,范围 1~2048);展开后的单文件与总体积上限为「包大小 × 4」且不低于 100 MiB。inspect 始终流式读取常规文件体,核对声明大小与实际字节并按实际值执行上限。
|
||||||
* **数量限制**:压缩包中包含的静态文件总数不得超过 1,000 个。
|
* **数量限制**:压缩包中包含的静态文件总数不得超过 1,000 个。
|
||||||
* **软链接阻断**:遍历归档文件,一旦检测到任何软链接,立即抛出错误并拒绝上传,防御软链接劫持攻击。
|
* **软链接阻断**:遍历归档文件,一旦检测到任何软链接,立即抛出错误并拒绝上传,防御软链接劫持攻击。
|
||||||
* **路径逃逸防御**:对每个压缩文件路径进行 `Clean` 并检查是否包含 `..` 或以 `/` 开头,防御目录跨越漏洞,防止写入系统敏感路径。
|
* **路径逃逸防御**:对每个压缩文件路径进行 `Clean` 并检查是否包含 `..` 或以 `/` 开头,防御目录跨越漏洞,防止写入系统敏感路径。
|
||||||
* **入口文件校验**:项目指定的入口文件(例如 `index.html`,可在 `project.RootDir` 下)必须在部署包中存在,否则拒绝上传。
|
* **入口文件校验**:项目指定的入口文件(例如 `index.html`,可在 `project.RootDir` 下)必须在部署包中存在,否则拒绝上传。
|
||||||
* **公共根目录去噪**:许多打包工具会包含一个多余的主文件夹作为公共根前缀。控制面自动探测公共根前缀并将其安全剥离。
|
* **公共根目录去噪**:许多打包工具会包含一个多余的主文件夹作为公共根前缀。控制面自动探测公共根前缀并将其安全剥离。
|
||||||
* **整包完整性**:上传/导入时对压缩包字节计算一次 SHA-256,写入部署记录;Agent 拉包后按整包 hash 对账。包内单文件不做内容哈希。
|
* **整包完整性**:上传/导入时对压缩包字节计算一次 SHA-256,写入部署记录;Agent 拉包后按整包 hash 对账。包内单文件不做内容哈希。
|
||||||
* **可选体积实测**:`InspectOptions.VerifySizes` 可对流式统计实际字节并与声明大小比对(仍不算 hash);默认关闭以降低上传 CPU/IO。
|
* **实际体积复核**:`InspectOptions.VerifySizes` 只保留兼容意义;当前 inspect 无论该值为何都会读取常规文件体、核对声明值并累计实际大小,但仍不为单文件计算内容 hash。
|
||||||
* **历史保留**:系统配置 `pages_max_history_count`(默认 20,0 表示不限制)在每次上传成功后执行裁剪。语义为:**每个项目最多保留 N 条部署**;当前激活部署始终保留;其余名额按部署 ID 从新到旧填充;超出的非激活部署连同文件清单与存储对象一并删除。上传已成功时裁剪失败只记日志、不回滚上传;并发上传下可能短暂超过 N,后续上传的裁剪会收敛回 N。主配置版本回滚不依赖旧 Pages 包(见上节双轨关系)。
|
* **历史保留**:系统配置 `pages_max_history_count`(默认 20,0 表示不限制)在部署成功后执行裁剪。通常语义为:**每个项目最多保留 N 条部署**;当前激活部署始终保留,其余名额按部署 ID 从新到旧填充。`history_count=1` 时,manual 上传会临时保留 active 与最新 candidate 两条,下一次上传替换旧 candidate;candidate 激活后恢复严格上限。超出的非激活 deployment 与文件清单会删除,对应 upload record 通过平台原语幂等软删除;Pages 不直接物理删除可能被 dedup 共享的 blob。部署已成功时裁剪失败只记日志、不回滚激活;并发操作下可能短暂超过 N,后续裁剪会收敛回 N。主配置版本回滚不依赖旧 Pages 包(见上节双轨关系)。
|
||||||
|
|
||||||
### 2. 部署包存储规划
|
### 2. 部署包存储规划
|
||||||
控制面通过统一上传框架(`upload.Ingest`)存储原始部署包,并在数据库中记录 `upload_id` 与文件清单。**大体积静态包不写入 config_versions 记录和任何配置推送通道**,以保障控制面数据同步的轻量与高效。
|
控制面通过统一上传框架(`upload.Ingest`)把本地、Remote 和 GitHub 产物存入配置的本地/S3 后端,并在数据库中记录 `upload_id` 与文件清单。**大体积静态包不写入 config_versions 记录和任何配置推送通道**,以保障控制面数据同步的轻量与高效。
|
||||||
|
|
||||||
|
### 3. 来源检查、自动更新与上传补偿
|
||||||
|
|
||||||
|
* `openflare:pages_source_action` 执行管理员 check/sync 或 scanner 派发的精确 revision sync;payload 不携带 URL、Token、ETag 或 lease token。手动 sync 只接受真实用户 actor,自动 sync 只接受系统 actor 与 `scheduled_auto_update` trigger。
|
||||||
|
* `openflare:pages_source_scan` 是固定 `*/5 * * * *` 的 internal-only TaskHandler,只接受 `{}`,不会出现在通用任务类型与排程管理界面。每轮按“恢复过期 lease → 补偿 orphan upload → 扫描到期来源”执行。
|
||||||
|
* scanner 按 `next_check_at, source_id` 稳定排序,每批最多串行检查 20 个 GitHub latest source;ETag/304 仍推进检查时间,403/429 记录状态码和实际退避截止时间,单来源失败不阻塞后续来源。
|
||||||
|
* 发现更新总会先保存 seen cursor。只有 `auto_update_enabled=true` 且状态为普通 `update_available` 时,才携带本次检查得到的精确 revision 派发同步;`attention`、Remote 和固定 tag 不会自动发布。人工激活其它 deployment 会 fence 在途任务并关闭 auto。
|
||||||
|
* orphan 补偿每轮最多检查 100 条至少隔离 2 小时的 upload record,并要求 system owner、Pages 保留 type、V2 marker、无 deployment 引用。候选在 `project → source → runtime → upload` 锁序内复查,只通过上传框架软删除 record 和更新统计,不直接物理删除可能被 dedup 共享的 blob。
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## Agent 端 (数据落地) 职责与自愈
|
## Agent 端 (数据落地) 职责与自愈
|
||||||
|
|
||||||
Agent 运行在各边缘代理节点上,在应用配置版本前,必须先将 Pages 静态资源“原子”地拉取到节点本地。
|
Agent 运行在各边缘代理节点上:首次应用引用 Pages 项目的配置时,以及后续周期性 latest 对账时,都会把当前激活的静态资源“原子”地拉取到节点本地。
|
||||||
|
|
||||||
### 1. 按项目拉取 latest
|
### 1. 按项目拉取 latest
|
||||||
1. Agent 从激活主配置中解析 `UpstreamType == "pages"` 的路由,收集稳定锚点 **`pages_project_id`**。
|
1. Agent 从激活主配置中解析 `UpstreamType == "pages"` 的路由,收集稳定锚点 **`pages_project_id`**。
|
||||||
2. 对每个项目调用 `GET /api/v1/agent/pages/projects/:project_id/latest/hash` 获取控制面当前激活包哈希(类似 latest 指针)。
|
2. 对每个项目调用 `GET /api/v1/agent/pages/projects/:project_id/latest/hash` 获取控制面当前激活包哈希(类似 latest 指针)。
|
||||||
3. 若本地 `projects/{project_id}/releases/{hash}` 尚未就绪,再下载 `.../latest/package`。下载后 **再次请求 hash** 与包内容 SHA-256 对齐,避免激活切换造成的竞态;不一致则有限次重试。
|
3. 若本地 `projects/{project_id}/releases/{hash}` 尚未就绪,再把 `.../latest/package` 流式下载到临时文件,执行真实响应上限与 SHA-256;下载后 **再次请求 hash**,避免激活切换造成的竞态,不一致则有限次重试。
|
||||||
4. 请求头携带节点 `X-Agent-Token`。
|
4. 请求头携带节点 `X-Agent-Token`。
|
||||||
|
|
||||||
### 2. 安全解压缩、原子切换与只保留最新
|
### 2. 安全解压缩、原子切换与只保留最新
|
||||||
1. 下载字节计算 SHA-256,须与「下载后再次查询」的 latest hash 一致。
|
1. 包体绝对上限为 2 GiB;下载内容的 SHA-256 须与「下载后再次查询」的 latest hash 一致,整个包不会进入 `[]byte`。
|
||||||
2. 解压至 `projects/{project_id}/releases/{hash}.tmp`(支持 zip / tar.* / 7z)。Agent 信任控制面业务校验,仅做路径逃逸/软链防护。
|
2. 解压至 `projects/{project_id}/releases/.{hash}-<random>.tmp` 随机 staging 目录(支持 zip / tar.* / 7z),拒绝路径逃逸、链接和特殊文件。Agent 同时服从 Server metadata 上限与本地绝对上限:最多 1,000 个文件,单文件及总量最多 8 GiB。
|
||||||
3. 写入 `.openflare-pages.json` 后 rename 为 `releases/{hash}`。
|
3. 解压完成后遍历实际文件树,精确复核文件数与总字节是否等于 Server metadata;不一致时拒绝切换。
|
||||||
4. **原子切换** `projects/{project_id}/current` 指向新 release(优先 symlink,失败则拷贝)。
|
4. 写入 `.openflare-pages.json` 后 rename 为 `releases/{hash}`。
|
||||||
5. **仅当新包已就绪且 current 切换成功后**,删除该项目下其它 `releases/*`(含 `.tmp`),**不保留历史部署包**。边缘节点每个项目永远只保留一份最新内容。
|
5. **原子切换** `projects/{project_id}/current` 指向新 release(优先 symlink,失败则拷贝)。
|
||||||
6. 多项目对账时 **隔离失败**:单个项目失败记日志并继续其它项目,最后汇总返回错误。
|
6. **仅当新包已就绪且 current 切换成功后**,删除该项目下其它 `releases/*`(含 `.tmp`),**不保留历史部署包**。边缘节点每个项目永远只保留一份最新内容。
|
||||||
|
7. 多项目对账时 **隔离失败**:单个项目失败记日志并继续其它项目,最后汇总返回错误。
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
@@ -211,26 +237,23 @@ server {
|
|||||||
|
|
||||||
## 交互逻辑与同步流程
|
## 交互逻辑与同步流程
|
||||||
|
|
||||||
一次完整的 Pages 上传与全局生效的生命周期如下:
|
一次完整的预构建产物导入与生效生命周期如下。首次绑定项目需要发布主配置;后续 active deployment 变化通过项目 latest 独立收敛:
|
||||||
|
|
||||||
```text
|
```text
|
||||||
[ 前端管理员 ] [ Server (控制面) ] [ Agent (数据落地) ] [ OpenResty ]
|
[管理员 / scanner] [Server 控制面] [Agent] [OpenResty]
|
||||||
| | | |
|
| | | |
|
||||||
|--- 1. 上传 ZIP 包 ----->| | |
|
|-- manual 上传 ------>|-- inspect / Ingest ---->| |
|
||||||
| |--- 2. 安全校验与解压分析 ----| |
|
| |-- 创建 candidate | |
|
||||||
| |--- 3. 归档包与持久化清单 ---| |
|
|-- 显式激活 candidate ->|-- 切换 active | |
|
||||||
| | | |
|
| | | |
|
||||||
|--- 4. 绑定路由并发布 -->| | |
|
|-- source sync ------>|-- inspect / Ingest | |
|
||||||
| |--- 5. 生成新配置版本并广播 ->| |
|
| |-- create/load + 原子激活 | |
|
||||||
| | | |
|
| | | |
|
||||||
| | |--- 6. 下载 ZIP 部署包 -->|
|
|-- 首次绑定项目并发布 ->|-- 广播项目锚点 -------->|-- 写入/重载路由 ---------->|
|
||||||
| | |<-- 7. 返回文件数据 -------|
|
| | | |
|
||||||
| | | |
|
|-- 后续激活/同步/回滚 ->|-- active latest 改变 ---| |
|
||||||
| | |--- 8. 强一致性 Checksum -|
|
| |<-- latest 元数据对账 ----| |
|
||||||
| | |--- 9. 安全解压缩 -------|
|
| |--- 流式返回 package ---->| |
|
||||||
| | |--- 10. 原子切换 current -|
|
| | |-- 校验、解压、复核 --------|
|
||||||
| | |--- 11. 测试与重载配置 ---->|
|
| | |-- 原子切换 current -------->|
|
||||||
| | |<-- 12. 重载成功 ---------|
|
|
||||||
| |<-- 13. 上报 Apply Success | |
|
|
||||||
| | | |
|
|
||||||
```
|
```
|
||||||
|
|||||||
+590
-4
@@ -2595,7 +2595,7 @@ const docTemplate = `{
|
|||||||
"SessionCookie": []
|
"SessionCookie": []
|
||||||
}
|
}
|
||||||
],
|
],
|
||||||
"description": "返回系统所有的定时任务配置列表,包括名称、关联的异步任务类型、Cron 表达式和启用状态,需要管理员权限",
|
"description": "返回管理员可管理的定时任务配置列表,包括名称、关联的异步任务类型、Cron 表达式和启用状态;系统内部排程不会暴露,需要管理员权限",
|
||||||
"produces": [
|
"produces": [
|
||||||
"application/json"
|
"application/json"
|
||||||
],
|
],
|
||||||
@@ -2860,6 +2860,12 @@ const docTemplate = `{
|
|||||||
"$ref": "#/definitions/response.Any"
|
"$ref": "#/definitions/response.Any"
|
||||||
}
|
}
|
||||||
},
|
},
|
||||||
|
"404": {
|
||||||
|
"description": "定时任务不存在",
|
||||||
|
"schema": {
|
||||||
|
"$ref": "#/definitions/response.Any"
|
||||||
|
}
|
||||||
|
},
|
||||||
"500": {
|
"500": {
|
||||||
"description": "删除定时任务失败",
|
"description": "删除定时任务失败",
|
||||||
"schema": {
|
"schema": {
|
||||||
@@ -3738,6 +3744,12 @@ const docTemplate = `{
|
|||||||
"schema": {
|
"schema": {
|
||||||
"$ref": "#/definitions/response.Any"
|
"$ref": "#/definitions/response.Any"
|
||||||
}
|
}
|
||||||
|
},
|
||||||
|
"409": {
|
||||||
|
"description": "系统保留类型或存储只读",
|
||||||
|
"schema": {
|
||||||
|
"$ref": "#/definitions/response.Any"
|
||||||
|
}
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
@@ -8578,7 +8590,7 @@ const docTemplate = `{
|
|||||||
"SessionCookie": []
|
"SessionCookie": []
|
||||||
}
|
}
|
||||||
],
|
],
|
||||||
"description": "从用户提供的 HTTP(S) 链接下载部署包并创建部署记录;服务端使用浏览器伪装请求头拉取,允许内网地址与不安全 TLS 证书,需要管理员权限",
|
"description": "已弃用的一次性 URL 导入;使用 trusted_internal 策略兼容内网与自签名证书,不创建持久部署源",
|
||||||
"consumes": [
|
"consumes": [
|
||||||
"application/json"
|
"application/json"
|
||||||
],
|
],
|
||||||
@@ -8589,6 +8601,7 @@ const docTemplate = `{
|
|||||||
"openflare-pages"
|
"openflare-pages"
|
||||||
],
|
],
|
||||||
"summary": "从 URL 导入 Pages 部署包",
|
"summary": "从 URL 导入 Pages 部署包",
|
||||||
|
"deprecated": true,
|
||||||
"parameters": [
|
"parameters": [
|
||||||
{
|
{
|
||||||
"type": "integer",
|
"type": "integer",
|
||||||
@@ -8795,6 +8808,391 @@ const docTemplate = `{
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
},
|
},
|
||||||
|
"/api/v1/d/pages/{id}/source": {
|
||||||
|
"get": {
|
||||||
|
"security": [
|
||||||
|
{
|
||||||
|
"SessionCookie": []
|
||||||
|
}
|
||||||
|
],
|
||||||
|
"description": "返回脱敏后的项目部署源配置与运行状态,需要管理员权限",
|
||||||
|
"produces": [
|
||||||
|
"application/json"
|
||||||
|
],
|
||||||
|
"tags": [
|
||||||
|
"openflare-pages"
|
||||||
|
],
|
||||||
|
"summary": "获取 Pages 部署源",
|
||||||
|
"parameters": [
|
||||||
|
{
|
||||||
|
"type": "integer",
|
||||||
|
"description": "项目 ID",
|
||||||
|
"name": "id",
|
||||||
|
"in": "path",
|
||||||
|
"required": true
|
||||||
|
}
|
||||||
|
],
|
||||||
|
"responses": {
|
||||||
|
"200": {
|
||||||
|
"description": "部署源",
|
||||||
|
"schema": {
|
||||||
|
"allOf": [
|
||||||
|
{
|
||||||
|
"$ref": "#/definitions/response.Any"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"type": "object",
|
||||||
|
"properties": {
|
||||||
|
"data": {
|
||||||
|
"$ref": "#/definitions/pages.SourceView"
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
]
|
||||||
|
}
|
||||||
|
},
|
||||||
|
"400": {
|
||||||
|
"description": "参数错误",
|
||||||
|
"schema": {
|
||||||
|
"$ref": "#/definitions/response.Any"
|
||||||
|
}
|
||||||
|
},
|
||||||
|
"401": {
|
||||||
|
"description": "未登录",
|
||||||
|
"schema": {
|
||||||
|
"$ref": "#/definitions/response.Any"
|
||||||
|
}
|
||||||
|
},
|
||||||
|
"404": {
|
||||||
|
"description": "项目或部署源不存在",
|
||||||
|
"schema": {
|
||||||
|
"$ref": "#/definitions/response.Any"
|
||||||
|
}
|
||||||
|
},
|
||||||
|
"500": {
|
||||||
|
"description": "内部错误",
|
||||||
|
"schema": {
|
||||||
|
"$ref": "#/definitions/response.Any"
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
},
|
||||||
|
"/api/v1/d/pages/{id}/source/check": {
|
||||||
|
"post": {
|
||||||
|
"security": [
|
||||||
|
{
|
||||||
|
"SessionCookie": []
|
||||||
|
}
|
||||||
|
],
|
||||||
|
"description": "异步检查 GitHub Release 来源;Remote URL 来源不支持检查更新",
|
||||||
|
"produces": [
|
||||||
|
"application/json"
|
||||||
|
],
|
||||||
|
"tags": [
|
||||||
|
"openflare-pages"
|
||||||
|
],
|
||||||
|
"summary": "检查 Pages 部署源",
|
||||||
|
"parameters": [
|
||||||
|
{
|
||||||
|
"type": "integer",
|
||||||
|
"description": "项目 ID",
|
||||||
|
"name": "id",
|
||||||
|
"in": "path",
|
||||||
|
"required": true
|
||||||
|
}
|
||||||
|
],
|
||||||
|
"responses": {
|
||||||
|
"200": {
|
||||||
|
"description": "任务回执",
|
||||||
|
"schema": {
|
||||||
|
"allOf": [
|
||||||
|
{
|
||||||
|
"$ref": "#/definitions/response.Any"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"type": "object",
|
||||||
|
"properties": {
|
||||||
|
"data": {
|
||||||
|
"$ref": "#/definitions/pages.SourceActionReceipt"
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
]
|
||||||
|
}
|
||||||
|
},
|
||||||
|
"400": {
|
||||||
|
"description": "当前来源不支持检查",
|
||||||
|
"schema": {
|
||||||
|
"$ref": "#/definitions/response.Any"
|
||||||
|
}
|
||||||
|
},
|
||||||
|
"401": {
|
||||||
|
"description": "未登录",
|
||||||
|
"schema": {
|
||||||
|
"$ref": "#/definitions/response.Any"
|
||||||
|
}
|
||||||
|
},
|
||||||
|
"404": {
|
||||||
|
"description": "部署源不存在",
|
||||||
|
"schema": {
|
||||||
|
"$ref": "#/definitions/response.Any"
|
||||||
|
}
|
||||||
|
},
|
||||||
|
"409": {
|
||||||
|
"description": "来源任务正在执行",
|
||||||
|
"schema": {
|
||||||
|
"$ref": "#/definitions/response.Any"
|
||||||
|
}
|
||||||
|
},
|
||||||
|
"500": {
|
||||||
|
"description": "内部错误",
|
||||||
|
"schema": {
|
||||||
|
"$ref": "#/definitions/response.Any"
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
},
|
||||||
|
"/api/v1/d/pages/{id}/source/delete": {
|
||||||
|
"post": {
|
||||||
|
"security": [
|
||||||
|
{
|
||||||
|
"SessionCookie": []
|
||||||
|
}
|
||||||
|
],
|
||||||
|
"description": "幂等删除持久部署源;已有部署历史与当前生产部署保持不变",
|
||||||
|
"produces": [
|
||||||
|
"application/json"
|
||||||
|
],
|
||||||
|
"tags": [
|
||||||
|
"openflare-pages"
|
||||||
|
],
|
||||||
|
"summary": "删除 Pages 部署源",
|
||||||
|
"parameters": [
|
||||||
|
{
|
||||||
|
"type": "integer",
|
||||||
|
"description": "项目 ID",
|
||||||
|
"name": "id",
|
||||||
|
"in": "path",
|
||||||
|
"required": true
|
||||||
|
}
|
||||||
|
],
|
||||||
|
"responses": {
|
||||||
|
"200": {
|
||||||
|
"description": "手动来源视图",
|
||||||
|
"schema": {
|
||||||
|
"allOf": [
|
||||||
|
{
|
||||||
|
"$ref": "#/definitions/response.Any"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"type": "object",
|
||||||
|
"properties": {
|
||||||
|
"data": {
|
||||||
|
"$ref": "#/definitions/pages.SourceView"
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
]
|
||||||
|
}
|
||||||
|
},
|
||||||
|
"400": {
|
||||||
|
"description": "参数错误",
|
||||||
|
"schema": {
|
||||||
|
"$ref": "#/definitions/response.Any"
|
||||||
|
}
|
||||||
|
},
|
||||||
|
"401": {
|
||||||
|
"description": "未登录",
|
||||||
|
"schema": {
|
||||||
|
"$ref": "#/definitions/response.Any"
|
||||||
|
}
|
||||||
|
},
|
||||||
|
"404": {
|
||||||
|
"description": "项目不存在",
|
||||||
|
"schema": {
|
||||||
|
"$ref": "#/definitions/response.Any"
|
||||||
|
}
|
||||||
|
},
|
||||||
|
"500": {
|
||||||
|
"description": "内部错误",
|
||||||
|
"schema": {
|
||||||
|
"$ref": "#/definitions/response.Any"
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
},
|
||||||
|
"/api/v1/d/pages/{id}/source/sync": {
|
||||||
|
"post": {
|
||||||
|
"security": [
|
||||||
|
{
|
||||||
|
"SessionCookie": []
|
||||||
|
}
|
||||||
|
],
|
||||||
|
"description": "异步下载、校验并原子激活来源部署包;空请求体与空 JSON 对象均有效",
|
||||||
|
"consumes": [
|
||||||
|
"application/json"
|
||||||
|
],
|
||||||
|
"produces": [
|
||||||
|
"application/json"
|
||||||
|
],
|
||||||
|
"tags": [
|
||||||
|
"openflare-pages"
|
||||||
|
],
|
||||||
|
"summary": "同步并发布 Pages 部署源",
|
||||||
|
"parameters": [
|
||||||
|
{
|
||||||
|
"type": "integer",
|
||||||
|
"description": "项目 ID",
|
||||||
|
"name": "id",
|
||||||
|
"in": "path",
|
||||||
|
"required": true
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"description": "同步参数",
|
||||||
|
"name": "request",
|
||||||
|
"in": "body",
|
||||||
|
"schema": {
|
||||||
|
"$ref": "#/definitions/pages.SourceSyncInput"
|
||||||
|
}
|
||||||
|
}
|
||||||
|
],
|
||||||
|
"responses": {
|
||||||
|
"200": {
|
||||||
|
"description": "任务回执",
|
||||||
|
"schema": {
|
||||||
|
"allOf": [
|
||||||
|
{
|
||||||
|
"$ref": "#/definitions/response.Any"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"type": "object",
|
||||||
|
"properties": {
|
||||||
|
"data": {
|
||||||
|
"$ref": "#/definitions/pages.SourceActionReceipt"
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
]
|
||||||
|
}
|
||||||
|
},
|
||||||
|
"400": {
|
||||||
|
"description": "参数或来源类型无效",
|
||||||
|
"schema": {
|
||||||
|
"$ref": "#/definitions/response.Any"
|
||||||
|
}
|
||||||
|
},
|
||||||
|
"401": {
|
||||||
|
"description": "未登录",
|
||||||
|
"schema": {
|
||||||
|
"$ref": "#/definitions/response.Any"
|
||||||
|
}
|
||||||
|
},
|
||||||
|
"404": {
|
||||||
|
"description": "部署源不存在",
|
||||||
|
"schema": {
|
||||||
|
"$ref": "#/definitions/response.Any"
|
||||||
|
}
|
||||||
|
},
|
||||||
|
"409": {
|
||||||
|
"description": "来源任务正在执行",
|
||||||
|
"schema": {
|
||||||
|
"$ref": "#/definitions/response.Any"
|
||||||
|
}
|
||||||
|
},
|
||||||
|
"500": {
|
||||||
|
"description": "内部错误",
|
||||||
|
"schema": {
|
||||||
|
"$ref": "#/definitions/response.Any"
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
},
|
||||||
|
"/api/v1/d/pages/{id}/source/update": {
|
||||||
|
"post": {
|
||||||
|
"security": [
|
||||||
|
{
|
||||||
|
"SessionCookie": []
|
||||||
|
}
|
||||||
|
],
|
||||||
|
"description": "支持 Remote URL 与公开 GitHub Release 来源;敏感地址仅写入,不会在响应中返回",
|
||||||
|
"consumes": [
|
||||||
|
"application/json"
|
||||||
|
],
|
||||||
|
"produces": [
|
||||||
|
"application/json"
|
||||||
|
],
|
||||||
|
"tags": [
|
||||||
|
"openflare-pages"
|
||||||
|
],
|
||||||
|
"summary": "更新 Pages 部署源",
|
||||||
|
"parameters": [
|
||||||
|
{
|
||||||
|
"type": "integer",
|
||||||
|
"description": "项目 ID",
|
||||||
|
"name": "id",
|
||||||
|
"in": "path",
|
||||||
|
"required": true
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"description": "部署源配置",
|
||||||
|
"name": "request",
|
||||||
|
"in": "body",
|
||||||
|
"required": true,
|
||||||
|
"schema": {
|
||||||
|
"$ref": "#/definitions/pages.SourceUpdateInput"
|
||||||
|
}
|
||||||
|
}
|
||||||
|
],
|
||||||
|
"responses": {
|
||||||
|
"200": {
|
||||||
|
"description": "更新结果",
|
||||||
|
"schema": {
|
||||||
|
"allOf": [
|
||||||
|
{
|
||||||
|
"$ref": "#/definitions/response.Any"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"type": "object",
|
||||||
|
"properties": {
|
||||||
|
"data": {
|
||||||
|
"$ref": "#/definitions/pages.SourceUpdateResult"
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
]
|
||||||
|
}
|
||||||
|
},
|
||||||
|
"400": {
|
||||||
|
"description": "配置无效",
|
||||||
|
"schema": {
|
||||||
|
"$ref": "#/definitions/response.Any"
|
||||||
|
}
|
||||||
|
},
|
||||||
|
"401": {
|
||||||
|
"description": "未登录",
|
||||||
|
"schema": {
|
||||||
|
"$ref": "#/definitions/response.Any"
|
||||||
|
}
|
||||||
|
},
|
||||||
|
"404": {
|
||||||
|
"description": "项目不存在",
|
||||||
|
"schema": {
|
||||||
|
"$ref": "#/definitions/response.Any"
|
||||||
|
}
|
||||||
|
},
|
||||||
|
"500": {
|
||||||
|
"description": "内部错误",
|
||||||
|
"schema": {
|
||||||
|
"$ref": "#/definitions/response.Any"
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
},
|
||||||
"/api/v1/d/pages/{id}/update": {
|
"/api/v1/d/pages/{id}/update": {
|
||||||
"post": {
|
"post": {
|
||||||
"security": [
|
"security": [
|
||||||
@@ -12535,6 +12933,12 @@ const docTemplate = `{
|
|||||||
"$ref": "#/definitions/response.Any"
|
"$ref": "#/definitions/response.Any"
|
||||||
}
|
}
|
||||||
},
|
},
|
||||||
|
"409": {
|
||||||
|
"description": "系统保留类型或存储只读",
|
||||||
|
"schema": {
|
||||||
|
"$ref": "#/definitions/response.Any"
|
||||||
|
}
|
||||||
|
},
|
||||||
"500": {
|
"500": {
|
||||||
"description": "内部错误",
|
"description": "内部错误",
|
||||||
"schema": {
|
"schema": {
|
||||||
@@ -12729,6 +13133,12 @@ const docTemplate = `{
|
|||||||
"schema": {
|
"schema": {
|
||||||
"$ref": "#/definitions/response.Any"
|
"$ref": "#/definitions/response.Any"
|
||||||
}
|
}
|
||||||
|
},
|
||||||
|
"409": {
|
||||||
|
"description": "系统保留类型或存储只读",
|
||||||
|
"schema": {
|
||||||
|
"$ref": "#/definitions/response.Any"
|
||||||
|
}
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
@@ -14574,11 +14984,20 @@ const docTemplate = `{
|
|||||||
"deployment_id": {
|
"deployment_id": {
|
||||||
"type": "integer"
|
"type": "integer"
|
||||||
},
|
},
|
||||||
|
"file_count": {
|
||||||
|
"type": "integer"
|
||||||
|
},
|
||||||
"hash": {
|
"hash": {
|
||||||
"type": "string"
|
"type": "string"
|
||||||
},
|
},
|
||||||
|
"package_size": {
|
||||||
|
"type": "integer"
|
||||||
|
},
|
||||||
"project_id": {
|
"project_id": {
|
||||||
"type": "integer"
|
"type": "integer"
|
||||||
|
},
|
||||||
|
"total_size": {
|
||||||
|
"type": "integer"
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
},
|
},
|
||||||
@@ -16568,9 +16987,15 @@ const docTemplate = `{
|
|||||||
"observability.AccessLogView": {
|
"observability.AccessLogView": {
|
||||||
"type": "object",
|
"type": "object",
|
||||||
"properties": {
|
"properties": {
|
||||||
|
"bytes_sent": {
|
||||||
|
"type": "integer"
|
||||||
|
},
|
||||||
"cache_status": {
|
"cache_status": {
|
||||||
"type": "string"
|
"type": "string"
|
||||||
},
|
},
|
||||||
|
"created_at": {
|
||||||
|
"type": "string"
|
||||||
|
},
|
||||||
"host": {
|
"host": {
|
||||||
"type": "string"
|
"type": "string"
|
||||||
},
|
},
|
||||||
@@ -16595,6 +17020,12 @@ const docTemplate = `{
|
|||||||
"remote_addr": {
|
"remote_addr": {
|
||||||
"type": "string"
|
"type": "string"
|
||||||
},
|
},
|
||||||
|
"request_length": {
|
||||||
|
"type": "integer"
|
||||||
|
},
|
||||||
|
"request_time_ms": {
|
||||||
|
"type": "integer"
|
||||||
|
},
|
||||||
"status_code": {
|
"status_code": {
|
||||||
"type": "integer"
|
"type": "integer"
|
||||||
},
|
},
|
||||||
@@ -17290,12 +17721,21 @@ const docTemplate = `{
|
|||||||
"project_id": {
|
"project_id": {
|
||||||
"type": "integer"
|
"type": "integer"
|
||||||
},
|
},
|
||||||
|
"source_label": {
|
||||||
|
"type": "string"
|
||||||
|
},
|
||||||
|
"source_type": {
|
||||||
|
"type": "string"
|
||||||
|
},
|
||||||
"status": {
|
"status": {
|
||||||
"type": "string"
|
"type": "string"
|
||||||
},
|
},
|
||||||
"total_size": {
|
"total_size": {
|
||||||
"type": "integer"
|
"type": "integer"
|
||||||
},
|
},
|
||||||
|
"trigger_type": {
|
||||||
|
"type": "string"
|
||||||
|
},
|
||||||
"upload_id": {
|
"upload_id": {
|
||||||
"type": "string",
|
"type": "string",
|
||||||
"example": "0"
|
"example": "0"
|
||||||
@@ -17343,6 +17783,150 @@ const docTemplate = `{
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
},
|
},
|
||||||
|
"pages.SourceActionReceipt": {
|
||||||
|
"type": "object",
|
||||||
|
"properties": {
|
||||||
|
"action": {
|
||||||
|
"type": "string"
|
||||||
|
},
|
||||||
|
"execution_id": {
|
||||||
|
"type": "string"
|
||||||
|
},
|
||||||
|
"task_id": {
|
||||||
|
"type": "string"
|
||||||
|
}
|
||||||
|
}
|
||||||
|
},
|
||||||
|
"pages.SourceRevisionView": {
|
||||||
|
"type": "object",
|
||||||
|
"properties": {
|
||||||
|
"asset_name": {
|
||||||
|
"type": "string"
|
||||||
|
},
|
||||||
|
"label": {
|
||||||
|
"type": "string"
|
||||||
|
},
|
||||||
|
"revision": {
|
||||||
|
"type": "string"
|
||||||
|
}
|
||||||
|
}
|
||||||
|
},
|
||||||
|
"pages.SourceSyncInput": {
|
||||||
|
"type": "object",
|
||||||
|
"properties": {
|
||||||
|
"confirmed_revision": {
|
||||||
|
"type": "string"
|
||||||
|
}
|
||||||
|
}
|
||||||
|
},
|
||||||
|
"pages.SourceUpdateInput": {
|
||||||
|
"type": "object",
|
||||||
|
"properties": {
|
||||||
|
"asset_name": {
|
||||||
|
"type": "string"
|
||||||
|
},
|
||||||
|
"auto_update_enabled": {
|
||||||
|
"type": "boolean"
|
||||||
|
},
|
||||||
|
"check_interval_minutes": {
|
||||||
|
"type": "integer"
|
||||||
|
},
|
||||||
|
"release_selector": {
|
||||||
|
"type": "string"
|
||||||
|
},
|
||||||
|
"release_tag": {
|
||||||
|
"type": "string"
|
||||||
|
},
|
||||||
|
"remote_network_policy": {
|
||||||
|
"type": "string"
|
||||||
|
},
|
||||||
|
"remote_url": {
|
||||||
|
"type": "string"
|
||||||
|
},
|
||||||
|
"remote_url_set": {
|
||||||
|
"type": "boolean"
|
||||||
|
},
|
||||||
|
"repository_url": {
|
||||||
|
"type": "string"
|
||||||
|
},
|
||||||
|
"source_type": {
|
||||||
|
"type": "string"
|
||||||
|
}
|
||||||
|
}
|
||||||
|
},
|
||||||
|
"pages.SourceUpdateResult": {
|
||||||
|
"type": "object",
|
||||||
|
"properties": {
|
||||||
|
"check_task": {
|
||||||
|
"$ref": "#/definitions/pages.SourceActionReceipt"
|
||||||
|
},
|
||||||
|
"source": {
|
||||||
|
"$ref": "#/definitions/pages.SourceView"
|
||||||
|
},
|
||||||
|
"warning": {
|
||||||
|
"type": "string"
|
||||||
|
}
|
||||||
|
}
|
||||||
|
},
|
||||||
|
"pages.SourceView": {
|
||||||
|
"type": "object",
|
||||||
|
"properties": {
|
||||||
|
"asset_name": {
|
||||||
|
"type": "string"
|
||||||
|
},
|
||||||
|
"auto_update_enabled": {
|
||||||
|
"type": "boolean"
|
||||||
|
},
|
||||||
|
"check_interval_minutes": {
|
||||||
|
"type": "integer"
|
||||||
|
},
|
||||||
|
"display_url": {
|
||||||
|
"type": "string"
|
||||||
|
},
|
||||||
|
"github_repository": {
|
||||||
|
"type": "string"
|
||||||
|
},
|
||||||
|
"has_remote_url": {
|
||||||
|
"type": "boolean"
|
||||||
|
},
|
||||||
|
"last_applied": {
|
||||||
|
"$ref": "#/definitions/pages.SourceRevisionView"
|
||||||
|
},
|
||||||
|
"last_checked_at": {
|
||||||
|
"type": "string"
|
||||||
|
},
|
||||||
|
"last_error": {
|
||||||
|
"type": "string"
|
||||||
|
},
|
||||||
|
"last_seen": {
|
||||||
|
"$ref": "#/definitions/pages.SourceRevisionView"
|
||||||
|
},
|
||||||
|
"last_synced_at": {
|
||||||
|
"type": "string"
|
||||||
|
},
|
||||||
|
"next_check_at": {
|
||||||
|
"type": "string"
|
||||||
|
},
|
||||||
|
"release_selector": {
|
||||||
|
"type": "string"
|
||||||
|
},
|
||||||
|
"release_tag": {
|
||||||
|
"type": "string"
|
||||||
|
},
|
||||||
|
"remote_network_policy": {
|
||||||
|
"type": "string"
|
||||||
|
},
|
||||||
|
"source_type": {
|
||||||
|
"type": "string"
|
||||||
|
},
|
||||||
|
"sync_status": {
|
||||||
|
"type": "string"
|
||||||
|
},
|
||||||
|
"update_available": {
|
||||||
|
"type": "boolean"
|
||||||
|
}
|
||||||
|
}
|
||||||
|
},
|
||||||
"pages.UploadFromURLInput": {
|
"pages.UploadFromURLInput": {
|
||||||
"type": "object",
|
"type": "object",
|
||||||
"properties": {
|
"properties": {
|
||||||
@@ -19310,7 +19894,8 @@ const docTemplate = `{
|
|||||||
"block",
|
"block",
|
||||||
"ip_match",
|
"ip_match",
|
||||||
"geo_match",
|
"geo_match",
|
||||||
"pow"
|
"pow",
|
||||||
|
"ua_check"
|
||||||
],
|
],
|
||||||
"x-enum-varnames": [
|
"x-enum-varnames": [
|
||||||
"RuleNodeStart",
|
"RuleNodeStart",
|
||||||
@@ -19318,7 +19903,8 @@ const docTemplate = `{
|
|||||||
"RuleNodeBlock",
|
"RuleNodeBlock",
|
||||||
"RuleNodeIPMatch",
|
"RuleNodeIPMatch",
|
||||||
"RuleNodeGeoMatch",
|
"RuleNodeGeoMatch",
|
||||||
"RuleNodePoW"
|
"RuleNodePoW",
|
||||||
|
"RuleNodeUACheck"
|
||||||
]
|
]
|
||||||
},
|
},
|
||||||
"waf.RulePosition": {
|
"waf.RulePosition": {
|
||||||
|
|||||||
+89
-59
@@ -1,86 +1,116 @@
|
|||||||
# Pages 静态托管使用
|
# Pages 静态托管使用
|
||||||
|
|
||||||
你会学到:如何在 OpenFlare 中使用 Pages 静态托管功能部署前端项目(如 React、Vue 等 SPA 或 VitePress、Hugo 等静态站点),配置单页应用 (SPA) Fallback 路由以及接口反向代理 (API Proxy),并理解不可变部署与 Agent 侧原子切换的底层逻辑。
|
你会学到:如何通过本地上传、Remote URL 或公开 GitHub Release asset 部署预构建静态站点,配置 SPA Fallback 与 API 反向代理,并安全地检查更新、自动发布和回滚。
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## 核心机制与工作流
|
## 核心机制与页面结构
|
||||||
|
|
||||||
OpenFlare Pages 提供受 Cloudflare Pages 启发的 **Direct Upload (直接上传)** 静态网站托管服务。它与常规代理站点的不同之处在于,数据面的边缘节点 (Agent) 会将静态文件拉取并解压到节点本地,直接通过本地的 OpenResty 提供高性能的静态文件服务,无需维护额外的 Nginx 宿主机静态目录同步。
|
OpenFlare Pages 受 Cloudflare Pages 的 Direct Upload 与部署历史交互启发,但当前处理的是**预构建产物**,不是仓库源码构建。项目详情按“当前生产部署 → 部署源 → 部署历史”组织:来源配置可以变化,已经创建的 deployment 保持不可变。
|
||||||
|
|
||||||
```text
|
```text
|
||||||
[ 管理员 / CI ] ────── 1. 上传 ZIP 压缩包 ──────► [ OpenFlare Server ]
|
本地上传 ─> 统一校验 / upload.Ingest ─> 新 candidate ─> 管理员显式激活 ─┐
|
||||||
│
|
Remote URL ── Server 受限下载 ────────┐ │
|
||||||
[ 访客浏览器 ] ◄────── 4. 访问页面 / 静态资源 ────────── [ Agent 节点 / OpenResty ]
|
GitHub Release asset ─ Server 解析 ───┴─> create/load deployment ─────┤
|
||||||
▲
|
└─> source sync 原子激活 ────────┘
|
||||||
│
|
|
|
||||||
2. 检查 Checksum 并拉取 ZIP
|
v
|
||||||
3. 解压并原子切换 current 链接
|
Agent 按项目 latest 拉取
|
||||||
|
|
|
||||||
|
v
|
||||||
|
OpenResty 本地静态服务
|
||||||
```
|
```
|
||||||
|
|
||||||
1. **直接上传部署包**:在控制面上传预构建好的网站 `.zip` 压缩包,Server 会生成一条带有唯一 SHA-256 校验和 (Checksum) 的不可变部署记录。
|
外部 URL、GitHub 元数据和自动检查都只由 Server 处理。Agent 只从控制面拉取当前激活的部署包,不接收外部来源凭据,也不执行 `git clone`、依赖安装或构建命令。
|
||||||
2. **发布与推送**:在路由配置中将源站类型 (Upstream Type) 设为 `Pages 静态托管` 并绑定项目。发布配置版本后,Server 会广播给所有 Agent 节点。
|
|
||||||
3. **安全拉取与部署**:Agent 节点识别到新配置引用了新的 Pages 部署,增量下载 ZIP 包,校验 Checksum 保证一致性,并在本地解压、完成原子目录切换,重载 OpenResty 使服务生效。
|
|
||||||
|
|
||||||
---
|
## 第一步:创建项目
|
||||||
|
|
||||||
## 第一步:上传部署包与创建 Pages 项目
|
1. 登录管理端,进入 **「Pages」**,点击 **「创建项目」**。
|
||||||
|
2. 填写项目名称与唯一 Slug。
|
||||||
|
3. 配置内容入口:
|
||||||
|
* **入口文件名**:默认 `index.html`。
|
||||||
|
* **静态资源根路径(RootDir)**:产物位于 `dist/` 等子目录时填写该相对路径;产物就在归档根目录时留空。
|
||||||
|
4. 按需设置 SPA Fallback 与 API 代理。RootDir 和入口文件是项目级配置,会统一应用于所有来源。
|
||||||
|
|
||||||
1. 登录管理端控制面板,进入左侧导航 **「Pages」** 菜单,点击 **「创建项目」**。
|
## 第二步:选择部署源
|
||||||
2. 填写项目基本信息:
|
|
||||||
* **项目名称**:业务名称(如 `我的前端应用`)。
|
|
||||||
* **项目标识 (Slug)**:URL 友好的唯一英文标识(如 `my-react-app`),将作为存储目录的文件夹名。
|
|
||||||
3. 设定站点目录结构与入口:
|
|
||||||
* **入口文件名**:默认为 `index.html`。
|
|
||||||
* **静态资源根路径 (RootDir)**:如果你的打包产物在压缩包的子目录下(例如打包出来的 zip 里包含一个 `dist/` 目录),则需要在这里填入子路径(如 `dist`)。若打包产物直接在 zip 根目录,留空即可。
|
|
||||||
4. **上传 ZIP 压缩包**:
|
|
||||||
* 上传你的项目静态资源打包生成的 `.zip` 文件。
|
|
||||||
|
|
||||||
> [!IMPORTANT]
|
### 1. 手动上传
|
||||||
> **部署包安全限制规范**
|
|
||||||
> 为了保障控制面和边缘节点的系统安全与性能,上传的部署包必须满足以下硬性指标,否则会被系统拒绝:
|
|
||||||
> * **大小限制**:ZIP 压缩包体积不得超过 **25 MiB**,解压后的总文件大小不得超过 **100 MiB**。
|
|
||||||
> * **数量限制**:解压后的文件总数不得超过 **1,000 个**。
|
|
||||||
> * **软链接拦截**:ZIP 包内禁止包含任何软链接 (Symbolic Link),防御软链接劫持攻击。
|
|
||||||
> * **Zip-Slip 防御**:压缩包中所有文件路径会被强制规范化,禁止使用 `..` 或以 `/` 开头,防止解压路径穿越攻击。
|
|
||||||
> * **入口文件检查**:你指定的入口文件(在静态资源根路径下,如 `dist/index.html`)**必须在压缩包中存在**。
|
|
||||||
|
|
||||||
---
|
不配置持久来源时,项目保持手动模式。点击 **「上传部署包」** 选择预构建归档;上传成功会创建一条候选 deployment,再从部署历史中显式激活。重复上传不会修改已有 deployment。
|
||||||
|
|
||||||
## 第二步:配置高级路由规则
|
支持 `zip`、`tar.gz` / `tgz`、`tar.xz` / `txz`、`tar.bz2` / `tbz2`、`tar` 与 `7z`。
|
||||||
|
|
||||||
在项目详情的配置页面中,你可以根据前端项目类型开启以下高级特性:
|
### 2. Remote URL
|
||||||
|
|
||||||
### 1. 单页应用 (SPA) Fallback 路由
|
在部署源卡片中选择 **Remote URL**,填写 HTTP(S) 地址并选择网络策略:
|
||||||
对于使用 React Router、Vue Router 等进行前端路由的单页应用 (SPA),当用户直接刷新类似 `/profile/settings` 的子路径时,边缘节点本地并不存在该物理文件,会导致 404 错误。
|
|
||||||
* **配置方式**:在项目设置中开启 **「SPA Fallback」**,并将路径设为入口文件(如 `/index.html`)。
|
|
||||||
* **生效逻辑**:开启后,如果访客请求的静态资源在物理上不存在,OpenResty 会自动降级重定向渲染入口文件,将路由交由前端 JavaScript 接管,避免 404 报错。
|
|
||||||
|
|
||||||
### 2. 内置 API 反向代理
|
* **public**:默认策略,拒绝 loopback、私网、链路本地地址、DNS rebinding、自签 TLS,以及重定向到非公网目标。
|
||||||
为了避免前端请求后端 API 时遭遇跨域 (CORS) 限制,Pages 托管支持在同一个域名下直通后端 API。
|
* **trusted_internal**:仅用于明确受信的内网或自签服务;保存前需要再次确认风险。
|
||||||
* **配置方式**:
|
|
||||||
* **API 代理路径 (APIProxyPath)**:匹配的 URL 前缀(如 `/api`)。
|
|
||||||
* **后端服务地址 (APIProxyPass)**:后端 API 的源站地址(如 `http://10.0.0.5:8080`)。
|
|
||||||
* **重写规则 (APIProxyRewrite)**:可选。如果需要剥离前缀或重写路径,可使用正则匹配。例如:
|
|
||||||
* 剥离前缀:将请求 `/api/users` 重写为 `/users` 发送给后端,配置为 `^/api/(.*)$ /$1`。
|
|
||||||
* **生效逻辑**:所有以 `/api` 开头的请求会被直接转发至后端服务,而其他请求则继续由静态托管服务处理。
|
|
||||||
|
|
||||||
---
|
保存后地址只以脱敏形式展示。编辑其它配置时无需重新填写;只有选择更换地址时才提交新 URL。Remote 来源只提供 **「同步并发布」**:每次由 Server 下载、校验并原子激活,不支持“检查更新”、定时检查或自动更新。
|
||||||
|
|
||||||
## 第三步:绑定代理路由并发布
|
### 3. GitHub Release
|
||||||
|
|
||||||
Pages 项目配置并上传好部署包后,需要绑定到对外公开的域名上才能被访客访问。
|
GitHub 来源仅支持公开 `github.com` 仓库。填写:
|
||||||
|
|
||||||
1. 导航至左侧菜单 **「规则管理」**,创建或编辑一条代理规则。
|
* `https://github.com/{owner}/{repo}` 格式的仓库地址;
|
||||||
2. 切换到 **「反向代理」** 选项卡:
|
* **最新 Release** 或 **固定 Tag**;
|
||||||
* **源站类型**:选择 **「Pages」**。
|
* 精确、区分大小写的 Release Asset 文件名,默认 `dist.zip`。
|
||||||
* **选择 Pages 项目**:选择你刚才创建的项目,并关联要激活的部署版本(默认会自动关联最新上传成功的部署)。
|
|
||||||
3. 点击右上角 **「配置预览」** -> 确认无误后点击 **「发布并激活」**。
|
|
||||||
|
|
||||||
## 运维与回滚
|
两种选择都可手动 **「检查更新」** 和 **「同步并发布」**。区别如下:
|
||||||
|
|
||||||
* **不可变部署与回滚**:每次在 Pages 项目下上传 `.zip` 文件,系统都会产生一个全新且唯一的部署版本。如果在历史部署列表中将上一版本设为激活并重新发布,可实现边缘节点的秒级回滚。
|
* **latest**:可设置 5~1440 分钟检查间隔,默认 60 分钟;自动更新默认关闭。开启后,scanner 发现新 revision 才会异步同步并发布。
|
||||||
* **原子切换与自愈**:边缘节点(Agent)在拉取静态资源包时,会执行校验与流式解压,并通过原子切换物理目录来保障服务的无缝过渡。同时,Agent 会定时清理不再引用的历史部署包。
|
* **tag**:只支持管理员手动检查和同步,不参与定时 scanner。
|
||||||
|
|
||||||
|
“检查更新”只解析 Release/asset 并更新版本游标,不下载部署包;“同步并发布”才会下载、校验、创建或复用 deployment 并激活。如果同一个 Release 下的 asset 被替换,来源会进入 **「需要确认」**,必须确认页面显示的精确 revision 后才能发布,避免静默覆盖。
|
||||||
|
|
||||||
|
GitHub Release 在这里是预构建产物源,不等同于连接代码仓库自动构建。未来仓库集成会使用独立的 `git_repository` 来源和 Server build executor,再把构建产物送入同一部署管线。
|
||||||
|
|
||||||
|
### 4. 切换或删除来源
|
||||||
|
|
||||||
|
可以在手动、Remote 和 GitHub Release 之间切换。修改或删除来源不会删除当前生产部署和历史 deployment;切回手动模式后可继续上传并显式激活。
|
||||||
|
|
||||||
|
## 部署包安全限制
|
||||||
|
|
||||||
|
部署包必须满足以下约束:
|
||||||
|
|
||||||
|
* 压缩包大小由系统配置 `pages_max_package_size_mb` 控制,默认 100 MiB,可配置 1~2048 MiB。
|
||||||
|
* 展开后的单文件和总量上限为“包大小上限 × 4”,且最低为 100 MiB;最多 1,000 个常规文件。
|
||||||
|
* 控制面会流式读取常规文件体,核对声明大小与实际字节,并校验项目入口文件。
|
||||||
|
* 归档中的绝对路径、`..` 路径逃逸、软链接、硬链接和特殊文件都会被拒绝。
|
||||||
|
|
||||||
|
Agent 下载时还会执行 SHA-256、真实响应字节上限、解压后文件数与总大小复核;失败不会切换现有 `current`。
|
||||||
|
|
||||||
|
## 第三步:配置高级路由规则
|
||||||
|
|
||||||
|
### 1. SPA Fallback
|
||||||
|
|
||||||
|
使用 React Router、Vue Router 等前端路由时,开启 **「SPA Fallback」** 并设置入口路径(通常为 `/index.html`)。访客直接访问不存在的物理路径时,OpenResty 会回退到入口文件交由前端路由处理。
|
||||||
|
|
||||||
|
### 2. API 反向代理
|
||||||
|
|
||||||
|
Pages 可在同一域名下把指定前缀转发到后端 API:
|
||||||
|
|
||||||
|
* **APIProxyPath**:匹配前缀,例如 `/api`。
|
||||||
|
* **APIProxyPass**:后端地址,例如 `http://10.0.0.5:8080`。
|
||||||
|
* **APIProxyRewrite**:可选的路径重写规则。
|
||||||
|
|
||||||
|
匹配 API 前缀的请求走反向代理,其余请求继续由静态站点处理。
|
||||||
|
|
||||||
|
## 第四步:绑定路由并首次发布
|
||||||
|
|
||||||
|
1. 创建或编辑一条代理规则。
|
||||||
|
2. 将源站类型设为 **Pages**,并选择 Pages **项目**。
|
||||||
|
3. 预览配置后发布并激活。
|
||||||
|
|
||||||
|
路由绑定的是稳定的项目 ID,不是某个 deployment。首次发布让 Agent 获得项目锚点;此后本地上传、来源同步、自动更新或人工回滚只会改变项目的 active deployment,Agent 会通过 latest hash 对账收敛,无需重新发布主配置。
|
||||||
|
|
||||||
|
## 运维、状态与回滚
|
||||||
|
|
||||||
|
* 来源卡片展示最近检查/同步、已发现与已应用 revision、下次检查和安全错误。检查或同步任务运行时,页面会轮询任务状态;latest 空闲时只在接近检查时间时低频刷新。
|
||||||
|
* 自动更新失败不会替换旧 active deployment;单个来源失败也不会阻塞 scanner 处理其它项目。
|
||||||
|
* 在部署历史中激活其它 deployment 即完成人工回滚。系统会 fence 在途来源任务,并关闭该来源的自动更新,避免下一轮 latest 又覆盖人工选择;重复激活当前版本是 no-op。
|
||||||
|
* Agent 下载到临时文件并校验 SHA-256,安全解压后原子切换 `current`。任一步失败都保留旧内容,多项目对账时单项目失败不影响其它项目。
|
||||||
|
|
||||||
> [!TIP]
|
> [!TIP]
|
||||||
> 关于不可变部署、目录结构设计、增量拉取和安全防逃逸校验等底层架构与自愈细节,请参阅 [Pages 静态托管设计](../design/pages-design.md)。
|
> 关于来源状态机、自动 scanner、上传补偿、不可变部署和 Agent 原子切换,请参阅 [Pages 静态托管设计](../design/pages-design.md)。
|
||||||
|
|||||||
File diff suppressed because it is too large
Load Diff
@@ -21,6 +21,10 @@
|
|||||||
* [访问日志 IP 明细 Tab](./20260719-access-log-ip-tab.md):第三 Tab 按 IP 聚合列表(时间窗/流量/2xx 比例);IP 情报迁入独立详情;日志详情仅请求字段。
|
* [访问日志 IP 明细 Tab](./20260719-access-log-ip-tab.md):第三 Tab 按 IP 聚合列表(时间窗/流量/2xx 比例);IP 情报迁入独立详情;日志详情仅请求字段。
|
||||||
* [边缘限流全局默认](./20260719-http-default-rate-limit.md):全局默认并发/带宽;站点 0 继承、-1 关闭;RenderRouteConfig 合并。
|
* [边缘限流全局默认](./20260719-http-default-rate-limit.md):全局默认并发/带宽;站点 0 继承、-1 关闭;RenderRouteConfig 合并。
|
||||||
|
|
||||||
|
## 已完成的计划
|
||||||
|
|
||||||
|
* [Pages 项目部署源与 GitHub Releases 自动更新 V2](./20260719-pages-source-sync-v2.md):已完成 Remote URL / GitHub Release 来源、不可变部署、自动检查更新与安全回滚,并预留独立仓库构建 Provider 边界;生产环境验收边界见计划内验证记录。
|
||||||
|
|
||||||
## 使用建议
|
## 使用建议
|
||||||
|
|
||||||
* **命名规范**:正在进行的开发计划建议命名为 `docs/plan/YYYYMMDD-[feature-name].md`,接手计划建议命名为 `docs/plan/handover-[task-name].md`。
|
* **命名规范**:正在进行的开发计划建议命名为 `docs/plan/YYYYMMDD-[feature-name].md`,接手计划建议命名为 `docs/plan/handover-[task-name].md`。
|
||||||
|
|||||||
+590
-4
@@ -2588,7 +2588,7 @@
|
|||||||
"SessionCookie": []
|
"SessionCookie": []
|
||||||
}
|
}
|
||||||
],
|
],
|
||||||
"description": "返回系统所有的定时任务配置列表,包括名称、关联的异步任务类型、Cron 表达式和启用状态,需要管理员权限",
|
"description": "返回管理员可管理的定时任务配置列表,包括名称、关联的异步任务类型、Cron 表达式和启用状态;系统内部排程不会暴露,需要管理员权限",
|
||||||
"produces": [
|
"produces": [
|
||||||
"application/json"
|
"application/json"
|
||||||
],
|
],
|
||||||
@@ -2853,6 +2853,12 @@
|
|||||||
"$ref": "#/definitions/response.Any"
|
"$ref": "#/definitions/response.Any"
|
||||||
}
|
}
|
||||||
},
|
},
|
||||||
|
"404": {
|
||||||
|
"description": "定时任务不存在",
|
||||||
|
"schema": {
|
||||||
|
"$ref": "#/definitions/response.Any"
|
||||||
|
}
|
||||||
|
},
|
||||||
"500": {
|
"500": {
|
||||||
"description": "删除定时任务失败",
|
"description": "删除定时任务失败",
|
||||||
"schema": {
|
"schema": {
|
||||||
@@ -3731,6 +3737,12 @@
|
|||||||
"schema": {
|
"schema": {
|
||||||
"$ref": "#/definitions/response.Any"
|
"$ref": "#/definitions/response.Any"
|
||||||
}
|
}
|
||||||
|
},
|
||||||
|
"409": {
|
||||||
|
"description": "系统保留类型或存储只读",
|
||||||
|
"schema": {
|
||||||
|
"$ref": "#/definitions/response.Any"
|
||||||
|
}
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
@@ -8571,7 +8583,7 @@
|
|||||||
"SessionCookie": []
|
"SessionCookie": []
|
||||||
}
|
}
|
||||||
],
|
],
|
||||||
"description": "从用户提供的 HTTP(S) 链接下载部署包并创建部署记录;服务端使用浏览器伪装请求头拉取,允许内网地址与不安全 TLS 证书,需要管理员权限",
|
"description": "已弃用的一次性 URL 导入;使用 trusted_internal 策略兼容内网与自签名证书,不创建持久部署源",
|
||||||
"consumes": [
|
"consumes": [
|
||||||
"application/json"
|
"application/json"
|
||||||
],
|
],
|
||||||
@@ -8582,6 +8594,7 @@
|
|||||||
"openflare-pages"
|
"openflare-pages"
|
||||||
],
|
],
|
||||||
"summary": "从 URL 导入 Pages 部署包",
|
"summary": "从 URL 导入 Pages 部署包",
|
||||||
|
"deprecated": true,
|
||||||
"parameters": [
|
"parameters": [
|
||||||
{
|
{
|
||||||
"type": "integer",
|
"type": "integer",
|
||||||
@@ -8788,6 +8801,391 @@
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
},
|
},
|
||||||
|
"/api/v1/d/pages/{id}/source": {
|
||||||
|
"get": {
|
||||||
|
"security": [
|
||||||
|
{
|
||||||
|
"SessionCookie": []
|
||||||
|
}
|
||||||
|
],
|
||||||
|
"description": "返回脱敏后的项目部署源配置与运行状态,需要管理员权限",
|
||||||
|
"produces": [
|
||||||
|
"application/json"
|
||||||
|
],
|
||||||
|
"tags": [
|
||||||
|
"openflare-pages"
|
||||||
|
],
|
||||||
|
"summary": "获取 Pages 部署源",
|
||||||
|
"parameters": [
|
||||||
|
{
|
||||||
|
"type": "integer",
|
||||||
|
"description": "项目 ID",
|
||||||
|
"name": "id",
|
||||||
|
"in": "path",
|
||||||
|
"required": true
|
||||||
|
}
|
||||||
|
],
|
||||||
|
"responses": {
|
||||||
|
"200": {
|
||||||
|
"description": "部署源",
|
||||||
|
"schema": {
|
||||||
|
"allOf": [
|
||||||
|
{
|
||||||
|
"$ref": "#/definitions/response.Any"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"type": "object",
|
||||||
|
"properties": {
|
||||||
|
"data": {
|
||||||
|
"$ref": "#/definitions/pages.SourceView"
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
]
|
||||||
|
}
|
||||||
|
},
|
||||||
|
"400": {
|
||||||
|
"description": "参数错误",
|
||||||
|
"schema": {
|
||||||
|
"$ref": "#/definitions/response.Any"
|
||||||
|
}
|
||||||
|
},
|
||||||
|
"401": {
|
||||||
|
"description": "未登录",
|
||||||
|
"schema": {
|
||||||
|
"$ref": "#/definitions/response.Any"
|
||||||
|
}
|
||||||
|
},
|
||||||
|
"404": {
|
||||||
|
"description": "项目或部署源不存在",
|
||||||
|
"schema": {
|
||||||
|
"$ref": "#/definitions/response.Any"
|
||||||
|
}
|
||||||
|
},
|
||||||
|
"500": {
|
||||||
|
"description": "内部错误",
|
||||||
|
"schema": {
|
||||||
|
"$ref": "#/definitions/response.Any"
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
},
|
||||||
|
"/api/v1/d/pages/{id}/source/check": {
|
||||||
|
"post": {
|
||||||
|
"security": [
|
||||||
|
{
|
||||||
|
"SessionCookie": []
|
||||||
|
}
|
||||||
|
],
|
||||||
|
"description": "异步检查 GitHub Release 来源;Remote URL 来源不支持检查更新",
|
||||||
|
"produces": [
|
||||||
|
"application/json"
|
||||||
|
],
|
||||||
|
"tags": [
|
||||||
|
"openflare-pages"
|
||||||
|
],
|
||||||
|
"summary": "检查 Pages 部署源",
|
||||||
|
"parameters": [
|
||||||
|
{
|
||||||
|
"type": "integer",
|
||||||
|
"description": "项目 ID",
|
||||||
|
"name": "id",
|
||||||
|
"in": "path",
|
||||||
|
"required": true
|
||||||
|
}
|
||||||
|
],
|
||||||
|
"responses": {
|
||||||
|
"200": {
|
||||||
|
"description": "任务回执",
|
||||||
|
"schema": {
|
||||||
|
"allOf": [
|
||||||
|
{
|
||||||
|
"$ref": "#/definitions/response.Any"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"type": "object",
|
||||||
|
"properties": {
|
||||||
|
"data": {
|
||||||
|
"$ref": "#/definitions/pages.SourceActionReceipt"
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
]
|
||||||
|
}
|
||||||
|
},
|
||||||
|
"400": {
|
||||||
|
"description": "当前来源不支持检查",
|
||||||
|
"schema": {
|
||||||
|
"$ref": "#/definitions/response.Any"
|
||||||
|
}
|
||||||
|
},
|
||||||
|
"401": {
|
||||||
|
"description": "未登录",
|
||||||
|
"schema": {
|
||||||
|
"$ref": "#/definitions/response.Any"
|
||||||
|
}
|
||||||
|
},
|
||||||
|
"404": {
|
||||||
|
"description": "部署源不存在",
|
||||||
|
"schema": {
|
||||||
|
"$ref": "#/definitions/response.Any"
|
||||||
|
}
|
||||||
|
},
|
||||||
|
"409": {
|
||||||
|
"description": "来源任务正在执行",
|
||||||
|
"schema": {
|
||||||
|
"$ref": "#/definitions/response.Any"
|
||||||
|
}
|
||||||
|
},
|
||||||
|
"500": {
|
||||||
|
"description": "内部错误",
|
||||||
|
"schema": {
|
||||||
|
"$ref": "#/definitions/response.Any"
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
},
|
||||||
|
"/api/v1/d/pages/{id}/source/delete": {
|
||||||
|
"post": {
|
||||||
|
"security": [
|
||||||
|
{
|
||||||
|
"SessionCookie": []
|
||||||
|
}
|
||||||
|
],
|
||||||
|
"description": "幂等删除持久部署源;已有部署历史与当前生产部署保持不变",
|
||||||
|
"produces": [
|
||||||
|
"application/json"
|
||||||
|
],
|
||||||
|
"tags": [
|
||||||
|
"openflare-pages"
|
||||||
|
],
|
||||||
|
"summary": "删除 Pages 部署源",
|
||||||
|
"parameters": [
|
||||||
|
{
|
||||||
|
"type": "integer",
|
||||||
|
"description": "项目 ID",
|
||||||
|
"name": "id",
|
||||||
|
"in": "path",
|
||||||
|
"required": true
|
||||||
|
}
|
||||||
|
],
|
||||||
|
"responses": {
|
||||||
|
"200": {
|
||||||
|
"description": "手动来源视图",
|
||||||
|
"schema": {
|
||||||
|
"allOf": [
|
||||||
|
{
|
||||||
|
"$ref": "#/definitions/response.Any"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"type": "object",
|
||||||
|
"properties": {
|
||||||
|
"data": {
|
||||||
|
"$ref": "#/definitions/pages.SourceView"
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
]
|
||||||
|
}
|
||||||
|
},
|
||||||
|
"400": {
|
||||||
|
"description": "参数错误",
|
||||||
|
"schema": {
|
||||||
|
"$ref": "#/definitions/response.Any"
|
||||||
|
}
|
||||||
|
},
|
||||||
|
"401": {
|
||||||
|
"description": "未登录",
|
||||||
|
"schema": {
|
||||||
|
"$ref": "#/definitions/response.Any"
|
||||||
|
}
|
||||||
|
},
|
||||||
|
"404": {
|
||||||
|
"description": "项目不存在",
|
||||||
|
"schema": {
|
||||||
|
"$ref": "#/definitions/response.Any"
|
||||||
|
}
|
||||||
|
},
|
||||||
|
"500": {
|
||||||
|
"description": "内部错误",
|
||||||
|
"schema": {
|
||||||
|
"$ref": "#/definitions/response.Any"
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
},
|
||||||
|
"/api/v1/d/pages/{id}/source/sync": {
|
||||||
|
"post": {
|
||||||
|
"security": [
|
||||||
|
{
|
||||||
|
"SessionCookie": []
|
||||||
|
}
|
||||||
|
],
|
||||||
|
"description": "异步下载、校验并原子激活来源部署包;空请求体与空 JSON 对象均有效",
|
||||||
|
"consumes": [
|
||||||
|
"application/json"
|
||||||
|
],
|
||||||
|
"produces": [
|
||||||
|
"application/json"
|
||||||
|
],
|
||||||
|
"tags": [
|
||||||
|
"openflare-pages"
|
||||||
|
],
|
||||||
|
"summary": "同步并发布 Pages 部署源",
|
||||||
|
"parameters": [
|
||||||
|
{
|
||||||
|
"type": "integer",
|
||||||
|
"description": "项目 ID",
|
||||||
|
"name": "id",
|
||||||
|
"in": "path",
|
||||||
|
"required": true
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"description": "同步参数",
|
||||||
|
"name": "request",
|
||||||
|
"in": "body",
|
||||||
|
"schema": {
|
||||||
|
"$ref": "#/definitions/pages.SourceSyncInput"
|
||||||
|
}
|
||||||
|
}
|
||||||
|
],
|
||||||
|
"responses": {
|
||||||
|
"200": {
|
||||||
|
"description": "任务回执",
|
||||||
|
"schema": {
|
||||||
|
"allOf": [
|
||||||
|
{
|
||||||
|
"$ref": "#/definitions/response.Any"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"type": "object",
|
||||||
|
"properties": {
|
||||||
|
"data": {
|
||||||
|
"$ref": "#/definitions/pages.SourceActionReceipt"
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
]
|
||||||
|
}
|
||||||
|
},
|
||||||
|
"400": {
|
||||||
|
"description": "参数或来源类型无效",
|
||||||
|
"schema": {
|
||||||
|
"$ref": "#/definitions/response.Any"
|
||||||
|
}
|
||||||
|
},
|
||||||
|
"401": {
|
||||||
|
"description": "未登录",
|
||||||
|
"schema": {
|
||||||
|
"$ref": "#/definitions/response.Any"
|
||||||
|
}
|
||||||
|
},
|
||||||
|
"404": {
|
||||||
|
"description": "部署源不存在",
|
||||||
|
"schema": {
|
||||||
|
"$ref": "#/definitions/response.Any"
|
||||||
|
}
|
||||||
|
},
|
||||||
|
"409": {
|
||||||
|
"description": "来源任务正在执行",
|
||||||
|
"schema": {
|
||||||
|
"$ref": "#/definitions/response.Any"
|
||||||
|
}
|
||||||
|
},
|
||||||
|
"500": {
|
||||||
|
"description": "内部错误",
|
||||||
|
"schema": {
|
||||||
|
"$ref": "#/definitions/response.Any"
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
},
|
||||||
|
"/api/v1/d/pages/{id}/source/update": {
|
||||||
|
"post": {
|
||||||
|
"security": [
|
||||||
|
{
|
||||||
|
"SessionCookie": []
|
||||||
|
}
|
||||||
|
],
|
||||||
|
"description": "支持 Remote URL 与公开 GitHub Release 来源;敏感地址仅写入,不会在响应中返回",
|
||||||
|
"consumes": [
|
||||||
|
"application/json"
|
||||||
|
],
|
||||||
|
"produces": [
|
||||||
|
"application/json"
|
||||||
|
],
|
||||||
|
"tags": [
|
||||||
|
"openflare-pages"
|
||||||
|
],
|
||||||
|
"summary": "更新 Pages 部署源",
|
||||||
|
"parameters": [
|
||||||
|
{
|
||||||
|
"type": "integer",
|
||||||
|
"description": "项目 ID",
|
||||||
|
"name": "id",
|
||||||
|
"in": "path",
|
||||||
|
"required": true
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"description": "部署源配置",
|
||||||
|
"name": "request",
|
||||||
|
"in": "body",
|
||||||
|
"required": true,
|
||||||
|
"schema": {
|
||||||
|
"$ref": "#/definitions/pages.SourceUpdateInput"
|
||||||
|
}
|
||||||
|
}
|
||||||
|
],
|
||||||
|
"responses": {
|
||||||
|
"200": {
|
||||||
|
"description": "更新结果",
|
||||||
|
"schema": {
|
||||||
|
"allOf": [
|
||||||
|
{
|
||||||
|
"$ref": "#/definitions/response.Any"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"type": "object",
|
||||||
|
"properties": {
|
||||||
|
"data": {
|
||||||
|
"$ref": "#/definitions/pages.SourceUpdateResult"
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
]
|
||||||
|
}
|
||||||
|
},
|
||||||
|
"400": {
|
||||||
|
"description": "配置无效",
|
||||||
|
"schema": {
|
||||||
|
"$ref": "#/definitions/response.Any"
|
||||||
|
}
|
||||||
|
},
|
||||||
|
"401": {
|
||||||
|
"description": "未登录",
|
||||||
|
"schema": {
|
||||||
|
"$ref": "#/definitions/response.Any"
|
||||||
|
}
|
||||||
|
},
|
||||||
|
"404": {
|
||||||
|
"description": "项目不存在",
|
||||||
|
"schema": {
|
||||||
|
"$ref": "#/definitions/response.Any"
|
||||||
|
}
|
||||||
|
},
|
||||||
|
"500": {
|
||||||
|
"description": "内部错误",
|
||||||
|
"schema": {
|
||||||
|
"$ref": "#/definitions/response.Any"
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
},
|
||||||
"/api/v1/d/pages/{id}/update": {
|
"/api/v1/d/pages/{id}/update": {
|
||||||
"post": {
|
"post": {
|
||||||
"security": [
|
"security": [
|
||||||
@@ -12528,6 +12926,12 @@
|
|||||||
"$ref": "#/definitions/response.Any"
|
"$ref": "#/definitions/response.Any"
|
||||||
}
|
}
|
||||||
},
|
},
|
||||||
|
"409": {
|
||||||
|
"description": "系统保留类型或存储只读",
|
||||||
|
"schema": {
|
||||||
|
"$ref": "#/definitions/response.Any"
|
||||||
|
}
|
||||||
|
},
|
||||||
"500": {
|
"500": {
|
||||||
"description": "内部错误",
|
"description": "内部错误",
|
||||||
"schema": {
|
"schema": {
|
||||||
@@ -12722,6 +13126,12 @@
|
|||||||
"schema": {
|
"schema": {
|
||||||
"$ref": "#/definitions/response.Any"
|
"$ref": "#/definitions/response.Any"
|
||||||
}
|
}
|
||||||
|
},
|
||||||
|
"409": {
|
||||||
|
"description": "系统保留类型或存储只读",
|
||||||
|
"schema": {
|
||||||
|
"$ref": "#/definitions/response.Any"
|
||||||
|
}
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
@@ -14567,11 +14977,20 @@
|
|||||||
"deployment_id": {
|
"deployment_id": {
|
||||||
"type": "integer"
|
"type": "integer"
|
||||||
},
|
},
|
||||||
|
"file_count": {
|
||||||
|
"type": "integer"
|
||||||
|
},
|
||||||
"hash": {
|
"hash": {
|
||||||
"type": "string"
|
"type": "string"
|
||||||
},
|
},
|
||||||
|
"package_size": {
|
||||||
|
"type": "integer"
|
||||||
|
},
|
||||||
"project_id": {
|
"project_id": {
|
||||||
"type": "integer"
|
"type": "integer"
|
||||||
|
},
|
||||||
|
"total_size": {
|
||||||
|
"type": "integer"
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
},
|
},
|
||||||
@@ -16561,9 +16980,15 @@
|
|||||||
"observability.AccessLogView": {
|
"observability.AccessLogView": {
|
||||||
"type": "object",
|
"type": "object",
|
||||||
"properties": {
|
"properties": {
|
||||||
|
"bytes_sent": {
|
||||||
|
"type": "integer"
|
||||||
|
},
|
||||||
"cache_status": {
|
"cache_status": {
|
||||||
"type": "string"
|
"type": "string"
|
||||||
},
|
},
|
||||||
|
"created_at": {
|
||||||
|
"type": "string"
|
||||||
|
},
|
||||||
"host": {
|
"host": {
|
||||||
"type": "string"
|
"type": "string"
|
||||||
},
|
},
|
||||||
@@ -16588,6 +17013,12 @@
|
|||||||
"remote_addr": {
|
"remote_addr": {
|
||||||
"type": "string"
|
"type": "string"
|
||||||
},
|
},
|
||||||
|
"request_length": {
|
||||||
|
"type": "integer"
|
||||||
|
},
|
||||||
|
"request_time_ms": {
|
||||||
|
"type": "integer"
|
||||||
|
},
|
||||||
"status_code": {
|
"status_code": {
|
||||||
"type": "integer"
|
"type": "integer"
|
||||||
},
|
},
|
||||||
@@ -17283,12 +17714,21 @@
|
|||||||
"project_id": {
|
"project_id": {
|
||||||
"type": "integer"
|
"type": "integer"
|
||||||
},
|
},
|
||||||
|
"source_label": {
|
||||||
|
"type": "string"
|
||||||
|
},
|
||||||
|
"source_type": {
|
||||||
|
"type": "string"
|
||||||
|
},
|
||||||
"status": {
|
"status": {
|
||||||
"type": "string"
|
"type": "string"
|
||||||
},
|
},
|
||||||
"total_size": {
|
"total_size": {
|
||||||
"type": "integer"
|
"type": "integer"
|
||||||
},
|
},
|
||||||
|
"trigger_type": {
|
||||||
|
"type": "string"
|
||||||
|
},
|
||||||
"upload_id": {
|
"upload_id": {
|
||||||
"type": "string",
|
"type": "string",
|
||||||
"example": "0"
|
"example": "0"
|
||||||
@@ -17336,6 +17776,150 @@
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
},
|
},
|
||||||
|
"pages.SourceActionReceipt": {
|
||||||
|
"type": "object",
|
||||||
|
"properties": {
|
||||||
|
"action": {
|
||||||
|
"type": "string"
|
||||||
|
},
|
||||||
|
"execution_id": {
|
||||||
|
"type": "string"
|
||||||
|
},
|
||||||
|
"task_id": {
|
||||||
|
"type": "string"
|
||||||
|
}
|
||||||
|
}
|
||||||
|
},
|
||||||
|
"pages.SourceRevisionView": {
|
||||||
|
"type": "object",
|
||||||
|
"properties": {
|
||||||
|
"asset_name": {
|
||||||
|
"type": "string"
|
||||||
|
},
|
||||||
|
"label": {
|
||||||
|
"type": "string"
|
||||||
|
},
|
||||||
|
"revision": {
|
||||||
|
"type": "string"
|
||||||
|
}
|
||||||
|
}
|
||||||
|
},
|
||||||
|
"pages.SourceSyncInput": {
|
||||||
|
"type": "object",
|
||||||
|
"properties": {
|
||||||
|
"confirmed_revision": {
|
||||||
|
"type": "string"
|
||||||
|
}
|
||||||
|
}
|
||||||
|
},
|
||||||
|
"pages.SourceUpdateInput": {
|
||||||
|
"type": "object",
|
||||||
|
"properties": {
|
||||||
|
"asset_name": {
|
||||||
|
"type": "string"
|
||||||
|
},
|
||||||
|
"auto_update_enabled": {
|
||||||
|
"type": "boolean"
|
||||||
|
},
|
||||||
|
"check_interval_minutes": {
|
||||||
|
"type": "integer"
|
||||||
|
},
|
||||||
|
"release_selector": {
|
||||||
|
"type": "string"
|
||||||
|
},
|
||||||
|
"release_tag": {
|
||||||
|
"type": "string"
|
||||||
|
},
|
||||||
|
"remote_network_policy": {
|
||||||
|
"type": "string"
|
||||||
|
},
|
||||||
|
"remote_url": {
|
||||||
|
"type": "string"
|
||||||
|
},
|
||||||
|
"remote_url_set": {
|
||||||
|
"type": "boolean"
|
||||||
|
},
|
||||||
|
"repository_url": {
|
||||||
|
"type": "string"
|
||||||
|
},
|
||||||
|
"source_type": {
|
||||||
|
"type": "string"
|
||||||
|
}
|
||||||
|
}
|
||||||
|
},
|
||||||
|
"pages.SourceUpdateResult": {
|
||||||
|
"type": "object",
|
||||||
|
"properties": {
|
||||||
|
"check_task": {
|
||||||
|
"$ref": "#/definitions/pages.SourceActionReceipt"
|
||||||
|
},
|
||||||
|
"source": {
|
||||||
|
"$ref": "#/definitions/pages.SourceView"
|
||||||
|
},
|
||||||
|
"warning": {
|
||||||
|
"type": "string"
|
||||||
|
}
|
||||||
|
}
|
||||||
|
},
|
||||||
|
"pages.SourceView": {
|
||||||
|
"type": "object",
|
||||||
|
"properties": {
|
||||||
|
"asset_name": {
|
||||||
|
"type": "string"
|
||||||
|
},
|
||||||
|
"auto_update_enabled": {
|
||||||
|
"type": "boolean"
|
||||||
|
},
|
||||||
|
"check_interval_minutes": {
|
||||||
|
"type": "integer"
|
||||||
|
},
|
||||||
|
"display_url": {
|
||||||
|
"type": "string"
|
||||||
|
},
|
||||||
|
"github_repository": {
|
||||||
|
"type": "string"
|
||||||
|
},
|
||||||
|
"has_remote_url": {
|
||||||
|
"type": "boolean"
|
||||||
|
},
|
||||||
|
"last_applied": {
|
||||||
|
"$ref": "#/definitions/pages.SourceRevisionView"
|
||||||
|
},
|
||||||
|
"last_checked_at": {
|
||||||
|
"type": "string"
|
||||||
|
},
|
||||||
|
"last_error": {
|
||||||
|
"type": "string"
|
||||||
|
},
|
||||||
|
"last_seen": {
|
||||||
|
"$ref": "#/definitions/pages.SourceRevisionView"
|
||||||
|
},
|
||||||
|
"last_synced_at": {
|
||||||
|
"type": "string"
|
||||||
|
},
|
||||||
|
"next_check_at": {
|
||||||
|
"type": "string"
|
||||||
|
},
|
||||||
|
"release_selector": {
|
||||||
|
"type": "string"
|
||||||
|
},
|
||||||
|
"release_tag": {
|
||||||
|
"type": "string"
|
||||||
|
},
|
||||||
|
"remote_network_policy": {
|
||||||
|
"type": "string"
|
||||||
|
},
|
||||||
|
"source_type": {
|
||||||
|
"type": "string"
|
||||||
|
},
|
||||||
|
"sync_status": {
|
||||||
|
"type": "string"
|
||||||
|
},
|
||||||
|
"update_available": {
|
||||||
|
"type": "boolean"
|
||||||
|
}
|
||||||
|
}
|
||||||
|
},
|
||||||
"pages.UploadFromURLInput": {
|
"pages.UploadFromURLInput": {
|
||||||
"type": "object",
|
"type": "object",
|
||||||
"properties": {
|
"properties": {
|
||||||
@@ -19303,7 +19887,8 @@
|
|||||||
"block",
|
"block",
|
||||||
"ip_match",
|
"ip_match",
|
||||||
"geo_match",
|
"geo_match",
|
||||||
"pow"
|
"pow",
|
||||||
|
"ua_check"
|
||||||
],
|
],
|
||||||
"x-enum-varnames": [
|
"x-enum-varnames": [
|
||||||
"RuleNodeStart",
|
"RuleNodeStart",
|
||||||
@@ -19311,7 +19896,8 @@
|
|||||||
"RuleNodeBlock",
|
"RuleNodeBlock",
|
||||||
"RuleNodeIPMatch",
|
"RuleNodeIPMatch",
|
||||||
"RuleNodeGeoMatch",
|
"RuleNodeGeoMatch",
|
||||||
"RuleNodePoW"
|
"RuleNodePoW",
|
||||||
|
"RuleNodeUACheck"
|
||||||
]
|
]
|
||||||
},
|
},
|
||||||
"waf.RulePosition": {
|
"waf.RulePosition": {
|
||||||
|
|||||||
+368
-2
@@ -760,10 +760,16 @@ definitions:
|
|||||||
properties:
|
properties:
|
||||||
deployment_id:
|
deployment_id:
|
||||||
type: integer
|
type: integer
|
||||||
|
file_count:
|
||||||
|
type: integer
|
||||||
hash:
|
hash:
|
||||||
type: string
|
type: string
|
||||||
|
package_size:
|
||||||
|
type: integer
|
||||||
project_id:
|
project_id:
|
||||||
type: integer
|
type: integer
|
||||||
|
total_size:
|
||||||
|
type: integer
|
||||||
type: object
|
type: object
|
||||||
github_com_Rain-kl_Wavelet_pkg_protocol.WAFIPGroup:
|
github_com_Rain-kl_Wavelet_pkg_protocol.WAFIPGroup:
|
||||||
properties:
|
properties:
|
||||||
@@ -2085,8 +2091,12 @@ definitions:
|
|||||||
type: object
|
type: object
|
||||||
observability.AccessLogView:
|
observability.AccessLogView:
|
||||||
properties:
|
properties:
|
||||||
|
bytes_sent:
|
||||||
|
type: integer
|
||||||
cache_status:
|
cache_status:
|
||||||
type: string
|
type: string
|
||||||
|
created_at:
|
||||||
|
type: string
|
||||||
host:
|
host:
|
||||||
type: string
|
type: string
|
||||||
id:
|
id:
|
||||||
@@ -2103,6 +2113,10 @@ definitions:
|
|||||||
type: string
|
type: string
|
||||||
remote_addr:
|
remote_addr:
|
||||||
type: string
|
type: string
|
||||||
|
request_length:
|
||||||
|
type: integer
|
||||||
|
request_time_ms:
|
||||||
|
type: integer
|
||||||
status_code:
|
status_code:
|
||||||
type: integer
|
type: integer
|
||||||
user_agent:
|
user_agent:
|
||||||
@@ -2557,10 +2571,16 @@ definitions:
|
|||||||
type: integer
|
type: integer
|
||||||
project_id:
|
project_id:
|
||||||
type: integer
|
type: integer
|
||||||
|
source_label:
|
||||||
|
type: string
|
||||||
|
source_type:
|
||||||
|
type: string
|
||||||
status:
|
status:
|
||||||
type: string
|
type: string
|
||||||
total_size:
|
total_size:
|
||||||
type: integer
|
type: integer
|
||||||
|
trigger_type:
|
||||||
|
type: string
|
||||||
upload_id:
|
upload_id:
|
||||||
example: "0"
|
example: "0"
|
||||||
type: string
|
type: string
|
||||||
@@ -2592,6 +2612,100 @@ definitions:
|
|||||||
spa_fallback_path:
|
spa_fallback_path:
|
||||||
type: string
|
type: string
|
||||||
type: object
|
type: object
|
||||||
|
pages.SourceActionReceipt:
|
||||||
|
properties:
|
||||||
|
action:
|
||||||
|
type: string
|
||||||
|
execution_id:
|
||||||
|
type: string
|
||||||
|
task_id:
|
||||||
|
type: string
|
||||||
|
type: object
|
||||||
|
pages.SourceRevisionView:
|
||||||
|
properties:
|
||||||
|
asset_name:
|
||||||
|
type: string
|
||||||
|
label:
|
||||||
|
type: string
|
||||||
|
revision:
|
||||||
|
type: string
|
||||||
|
type: object
|
||||||
|
pages.SourceSyncInput:
|
||||||
|
properties:
|
||||||
|
confirmed_revision:
|
||||||
|
type: string
|
||||||
|
type: object
|
||||||
|
pages.SourceUpdateInput:
|
||||||
|
properties:
|
||||||
|
asset_name:
|
||||||
|
type: string
|
||||||
|
auto_update_enabled:
|
||||||
|
type: boolean
|
||||||
|
check_interval_minutes:
|
||||||
|
type: integer
|
||||||
|
release_selector:
|
||||||
|
type: string
|
||||||
|
release_tag:
|
||||||
|
type: string
|
||||||
|
remote_network_policy:
|
||||||
|
type: string
|
||||||
|
remote_url:
|
||||||
|
type: string
|
||||||
|
remote_url_set:
|
||||||
|
type: boolean
|
||||||
|
repository_url:
|
||||||
|
type: string
|
||||||
|
source_type:
|
||||||
|
type: string
|
||||||
|
type: object
|
||||||
|
pages.SourceUpdateResult:
|
||||||
|
properties:
|
||||||
|
check_task:
|
||||||
|
$ref: '#/definitions/pages.SourceActionReceipt'
|
||||||
|
source:
|
||||||
|
$ref: '#/definitions/pages.SourceView'
|
||||||
|
warning:
|
||||||
|
type: string
|
||||||
|
type: object
|
||||||
|
pages.SourceView:
|
||||||
|
properties:
|
||||||
|
asset_name:
|
||||||
|
type: string
|
||||||
|
auto_update_enabled:
|
||||||
|
type: boolean
|
||||||
|
check_interval_minutes:
|
||||||
|
type: integer
|
||||||
|
display_url:
|
||||||
|
type: string
|
||||||
|
github_repository:
|
||||||
|
type: string
|
||||||
|
has_remote_url:
|
||||||
|
type: boolean
|
||||||
|
last_applied:
|
||||||
|
$ref: '#/definitions/pages.SourceRevisionView'
|
||||||
|
last_checked_at:
|
||||||
|
type: string
|
||||||
|
last_error:
|
||||||
|
type: string
|
||||||
|
last_seen:
|
||||||
|
$ref: '#/definitions/pages.SourceRevisionView'
|
||||||
|
last_synced_at:
|
||||||
|
type: string
|
||||||
|
next_check_at:
|
||||||
|
type: string
|
||||||
|
release_selector:
|
||||||
|
type: string
|
||||||
|
release_tag:
|
||||||
|
type: string
|
||||||
|
remote_network_policy:
|
||||||
|
type: string
|
||||||
|
source_type:
|
||||||
|
type: string
|
||||||
|
sync_status:
|
||||||
|
type: string
|
||||||
|
update_available:
|
||||||
|
type: boolean
|
||||||
|
type: object
|
||||||
pages.UploadFromURLInput:
|
pages.UploadFromURLInput:
|
||||||
properties:
|
properties:
|
||||||
url:
|
url:
|
||||||
@@ -3908,6 +4022,7 @@ definitions:
|
|||||||
- ip_match
|
- ip_match
|
||||||
- geo_match
|
- geo_match
|
||||||
- pow
|
- pow
|
||||||
|
- ua_check
|
||||||
type: string
|
type: string
|
||||||
x-enum-varnames:
|
x-enum-varnames:
|
||||||
- RuleNodeStart
|
- RuleNodeStart
|
||||||
@@ -3916,6 +4031,7 @@ definitions:
|
|||||||
- RuleNodeIPMatch
|
- RuleNodeIPMatch
|
||||||
- RuleNodeGeoMatch
|
- RuleNodeGeoMatch
|
||||||
- RuleNodePoW
|
- RuleNodePoW
|
||||||
|
- RuleNodeUACheck
|
||||||
waf.RulePosition:
|
waf.RulePosition:
|
||||||
properties:
|
properties:
|
||||||
x:
|
x:
|
||||||
@@ -5601,7 +5717,7 @@ paths:
|
|||||||
- admin
|
- admin
|
||||||
/api/v1/admin/tasks/schedules:
|
/api/v1/admin/tasks/schedules:
|
||||||
get:
|
get:
|
||||||
description: 返回系统所有的定时任务配置列表,包括名称、关联的异步任务类型、Cron 表达式和启用状态,需要管理员权限
|
description: 返回管理员可管理的定时任务配置列表,包括名称、关联的异步任务类型、Cron 表达式和启用状态;系统内部排程不会暴露,需要管理员权限
|
||||||
produces:
|
produces:
|
||||||
- application/json
|
- application/json
|
||||||
responses:
|
responses:
|
||||||
@@ -5706,6 +5822,10 @@ paths:
|
|||||||
description: 无管理员权限
|
description: 无管理员权限
|
||||||
schema:
|
schema:
|
||||||
$ref: '#/definitions/response.Any'
|
$ref: '#/definitions/response.Any'
|
||||||
|
"404":
|
||||||
|
description: 定时任务不存在
|
||||||
|
schema:
|
||||||
|
$ref: '#/definitions/response.Any'
|
||||||
"500":
|
"500":
|
||||||
description: 删除定时任务失败
|
description: 删除定时任务失败
|
||||||
schema:
|
schema:
|
||||||
@@ -6157,6 +6277,10 @@ paths:
|
|||||||
description: 文件不存在
|
description: 文件不存在
|
||||||
schema:
|
schema:
|
||||||
$ref: '#/definitions/response.Any'
|
$ref: '#/definitions/response.Any'
|
||||||
|
"409":
|
||||||
|
description: 系统保留类型或存储只读
|
||||||
|
schema:
|
||||||
|
$ref: '#/definitions/response.Any'
|
||||||
security:
|
security:
|
||||||
- SessionCookie: []
|
- SessionCookie: []
|
||||||
summary: 删除文件
|
summary: 删除文件
|
||||||
@@ -9257,7 +9381,8 @@ paths:
|
|||||||
post:
|
post:
|
||||||
consumes:
|
consumes:
|
||||||
- application/json
|
- application/json
|
||||||
description: 从用户提供的 HTTP(S) 链接下载部署包并创建部署记录;服务端使用浏览器伪装请求头拉取,允许内网地址与不安全 TLS 证书,需要管理员权限
|
deprecated: true
|
||||||
|
description: 已弃用的一次性 URL 导入;使用 trusted_internal 策略兼容内网与自签名证书,不创建持久部署源
|
||||||
parameters:
|
parameters:
|
||||||
- description: 项目 ID
|
- description: 项目 ID
|
||||||
in: path
|
in: path
|
||||||
@@ -9303,6 +9428,239 @@ paths:
|
|||||||
summary: 从 URL 导入 Pages 部署包
|
summary: 从 URL 导入 Pages 部署包
|
||||||
tags:
|
tags:
|
||||||
- openflare-pages
|
- openflare-pages
|
||||||
|
/api/v1/d/pages/{id}/source:
|
||||||
|
get:
|
||||||
|
description: 返回脱敏后的项目部署源配置与运行状态,需要管理员权限
|
||||||
|
parameters:
|
||||||
|
- description: 项目 ID
|
||||||
|
in: path
|
||||||
|
name: id
|
||||||
|
required: true
|
||||||
|
type: integer
|
||||||
|
produces:
|
||||||
|
- application/json
|
||||||
|
responses:
|
||||||
|
"200":
|
||||||
|
description: 部署源
|
||||||
|
schema:
|
||||||
|
allOf:
|
||||||
|
- $ref: '#/definitions/response.Any'
|
||||||
|
- properties:
|
||||||
|
data:
|
||||||
|
$ref: '#/definitions/pages.SourceView'
|
||||||
|
type: object
|
||||||
|
"400":
|
||||||
|
description: 参数错误
|
||||||
|
schema:
|
||||||
|
$ref: '#/definitions/response.Any'
|
||||||
|
"401":
|
||||||
|
description: 未登录
|
||||||
|
schema:
|
||||||
|
$ref: '#/definitions/response.Any'
|
||||||
|
"404":
|
||||||
|
description: 项目或部署源不存在
|
||||||
|
schema:
|
||||||
|
$ref: '#/definitions/response.Any'
|
||||||
|
"500":
|
||||||
|
description: 内部错误
|
||||||
|
schema:
|
||||||
|
$ref: '#/definitions/response.Any'
|
||||||
|
security:
|
||||||
|
- SessionCookie: []
|
||||||
|
summary: 获取 Pages 部署源
|
||||||
|
tags:
|
||||||
|
- openflare-pages
|
||||||
|
/api/v1/d/pages/{id}/source/check:
|
||||||
|
post:
|
||||||
|
description: 异步检查 GitHub Release 来源;Remote URL 来源不支持检查更新
|
||||||
|
parameters:
|
||||||
|
- description: 项目 ID
|
||||||
|
in: path
|
||||||
|
name: id
|
||||||
|
required: true
|
||||||
|
type: integer
|
||||||
|
produces:
|
||||||
|
- application/json
|
||||||
|
responses:
|
||||||
|
"200":
|
||||||
|
description: 任务回执
|
||||||
|
schema:
|
||||||
|
allOf:
|
||||||
|
- $ref: '#/definitions/response.Any'
|
||||||
|
- properties:
|
||||||
|
data:
|
||||||
|
$ref: '#/definitions/pages.SourceActionReceipt'
|
||||||
|
type: object
|
||||||
|
"400":
|
||||||
|
description: 当前来源不支持检查
|
||||||
|
schema:
|
||||||
|
$ref: '#/definitions/response.Any'
|
||||||
|
"401":
|
||||||
|
description: 未登录
|
||||||
|
schema:
|
||||||
|
$ref: '#/definitions/response.Any'
|
||||||
|
"404":
|
||||||
|
description: 部署源不存在
|
||||||
|
schema:
|
||||||
|
$ref: '#/definitions/response.Any'
|
||||||
|
"409":
|
||||||
|
description: 来源任务正在执行
|
||||||
|
schema:
|
||||||
|
$ref: '#/definitions/response.Any'
|
||||||
|
"500":
|
||||||
|
description: 内部错误
|
||||||
|
schema:
|
||||||
|
$ref: '#/definitions/response.Any'
|
||||||
|
security:
|
||||||
|
- SessionCookie: []
|
||||||
|
summary: 检查 Pages 部署源
|
||||||
|
tags:
|
||||||
|
- openflare-pages
|
||||||
|
/api/v1/d/pages/{id}/source/delete:
|
||||||
|
post:
|
||||||
|
description: 幂等删除持久部署源;已有部署历史与当前生产部署保持不变
|
||||||
|
parameters:
|
||||||
|
- description: 项目 ID
|
||||||
|
in: path
|
||||||
|
name: id
|
||||||
|
required: true
|
||||||
|
type: integer
|
||||||
|
produces:
|
||||||
|
- application/json
|
||||||
|
responses:
|
||||||
|
"200":
|
||||||
|
description: 手动来源视图
|
||||||
|
schema:
|
||||||
|
allOf:
|
||||||
|
- $ref: '#/definitions/response.Any'
|
||||||
|
- properties:
|
||||||
|
data:
|
||||||
|
$ref: '#/definitions/pages.SourceView'
|
||||||
|
type: object
|
||||||
|
"400":
|
||||||
|
description: 参数错误
|
||||||
|
schema:
|
||||||
|
$ref: '#/definitions/response.Any'
|
||||||
|
"401":
|
||||||
|
description: 未登录
|
||||||
|
schema:
|
||||||
|
$ref: '#/definitions/response.Any'
|
||||||
|
"404":
|
||||||
|
description: 项目不存在
|
||||||
|
schema:
|
||||||
|
$ref: '#/definitions/response.Any'
|
||||||
|
"500":
|
||||||
|
description: 内部错误
|
||||||
|
schema:
|
||||||
|
$ref: '#/definitions/response.Any'
|
||||||
|
security:
|
||||||
|
- SessionCookie: []
|
||||||
|
summary: 删除 Pages 部署源
|
||||||
|
tags:
|
||||||
|
- openflare-pages
|
||||||
|
/api/v1/d/pages/{id}/source/sync:
|
||||||
|
post:
|
||||||
|
consumes:
|
||||||
|
- application/json
|
||||||
|
description: 异步下载、校验并原子激活来源部署包;空请求体与空 JSON 对象均有效
|
||||||
|
parameters:
|
||||||
|
- description: 项目 ID
|
||||||
|
in: path
|
||||||
|
name: id
|
||||||
|
required: true
|
||||||
|
type: integer
|
||||||
|
- description: 同步参数
|
||||||
|
in: body
|
||||||
|
name: request
|
||||||
|
schema:
|
||||||
|
$ref: '#/definitions/pages.SourceSyncInput'
|
||||||
|
produces:
|
||||||
|
- application/json
|
||||||
|
responses:
|
||||||
|
"200":
|
||||||
|
description: 任务回执
|
||||||
|
schema:
|
||||||
|
allOf:
|
||||||
|
- $ref: '#/definitions/response.Any'
|
||||||
|
- properties:
|
||||||
|
data:
|
||||||
|
$ref: '#/definitions/pages.SourceActionReceipt'
|
||||||
|
type: object
|
||||||
|
"400":
|
||||||
|
description: 参数或来源类型无效
|
||||||
|
schema:
|
||||||
|
$ref: '#/definitions/response.Any'
|
||||||
|
"401":
|
||||||
|
description: 未登录
|
||||||
|
schema:
|
||||||
|
$ref: '#/definitions/response.Any'
|
||||||
|
"404":
|
||||||
|
description: 部署源不存在
|
||||||
|
schema:
|
||||||
|
$ref: '#/definitions/response.Any'
|
||||||
|
"409":
|
||||||
|
description: 来源任务正在执行
|
||||||
|
schema:
|
||||||
|
$ref: '#/definitions/response.Any'
|
||||||
|
"500":
|
||||||
|
description: 内部错误
|
||||||
|
schema:
|
||||||
|
$ref: '#/definitions/response.Any'
|
||||||
|
security:
|
||||||
|
- SessionCookie: []
|
||||||
|
summary: 同步并发布 Pages 部署源
|
||||||
|
tags:
|
||||||
|
- openflare-pages
|
||||||
|
/api/v1/d/pages/{id}/source/update:
|
||||||
|
post:
|
||||||
|
consumes:
|
||||||
|
- application/json
|
||||||
|
description: 支持 Remote URL 与公开 GitHub Release 来源;敏感地址仅写入,不会在响应中返回
|
||||||
|
parameters:
|
||||||
|
- description: 项目 ID
|
||||||
|
in: path
|
||||||
|
name: id
|
||||||
|
required: true
|
||||||
|
type: integer
|
||||||
|
- description: 部署源配置
|
||||||
|
in: body
|
||||||
|
name: request
|
||||||
|
required: true
|
||||||
|
schema:
|
||||||
|
$ref: '#/definitions/pages.SourceUpdateInput'
|
||||||
|
produces:
|
||||||
|
- application/json
|
||||||
|
responses:
|
||||||
|
"200":
|
||||||
|
description: 更新结果
|
||||||
|
schema:
|
||||||
|
allOf:
|
||||||
|
- $ref: '#/definitions/response.Any'
|
||||||
|
- properties:
|
||||||
|
data:
|
||||||
|
$ref: '#/definitions/pages.SourceUpdateResult'
|
||||||
|
type: object
|
||||||
|
"400":
|
||||||
|
description: 配置无效
|
||||||
|
schema:
|
||||||
|
$ref: '#/definitions/response.Any'
|
||||||
|
"401":
|
||||||
|
description: 未登录
|
||||||
|
schema:
|
||||||
|
$ref: '#/definitions/response.Any'
|
||||||
|
"404":
|
||||||
|
description: 项目不存在
|
||||||
|
schema:
|
||||||
|
$ref: '#/definitions/response.Any'
|
||||||
|
"500":
|
||||||
|
description: 内部错误
|
||||||
|
schema:
|
||||||
|
$ref: '#/definitions/response.Any'
|
||||||
|
security:
|
||||||
|
- SessionCookie: []
|
||||||
|
summary: 更新 Pages 部署源
|
||||||
|
tags:
|
||||||
|
- openflare-pages
|
||||||
/api/v1/d/pages/{id}/update:
|
/api/v1/d/pages/{id}/update:
|
||||||
post:
|
post:
|
||||||
consumes:
|
consumes:
|
||||||
@@ -11589,6 +11947,10 @@ paths:
|
|||||||
description: 未登录
|
description: 未登录
|
||||||
schema:
|
schema:
|
||||||
$ref: '#/definitions/response.Any'
|
$ref: '#/definitions/response.Any'
|
||||||
|
"409":
|
||||||
|
description: 系统保留类型或存储只读
|
||||||
|
schema:
|
||||||
|
$ref: '#/definitions/response.Any'
|
||||||
"500":
|
"500":
|
||||||
description: 内部错误
|
description: 内部错误
|
||||||
schema:
|
schema:
|
||||||
@@ -11622,6 +11984,10 @@ paths:
|
|||||||
description: 文件不存在
|
description: 文件不存在
|
||||||
schema:
|
schema:
|
||||||
$ref: '#/definitions/response.Any'
|
$ref: '#/definitions/response.Any'
|
||||||
|
"409":
|
||||||
|
description: 系统保留类型或存储只读
|
||||||
|
schema:
|
||||||
|
$ref: '#/definitions/response.Any'
|
||||||
security:
|
security:
|
||||||
- SessionCookie: []
|
- SessionCookie: []
|
||||||
summary: 删除我的文件
|
summary: 删除我的文件
|
||||||
|
|||||||
@@ -2,7 +2,7 @@
|
|||||||
|
|
||||||
import { useRef, useState } from 'react';
|
import { useRef, useState } from 'react';
|
||||||
import { useMutation, useQueryClient } from '@tanstack/react-query';
|
import { useMutation, useQueryClient } from '@tanstack/react-query';
|
||||||
import { Link2, Loader2, UploadCloud } from 'lucide-react';
|
import { UploadCloud } from 'lucide-react';
|
||||||
import { toast } from 'sonner';
|
import { toast } from 'sonner';
|
||||||
|
|
||||||
import { Button } from '@/components/ui/button';
|
import { Button } from '@/components/ui/button';
|
||||||
@@ -14,10 +14,14 @@ import {
|
|||||||
DialogHeader,
|
DialogHeader,
|
||||||
DialogTitle,
|
DialogTitle,
|
||||||
} from '@/components/ui/dialog';
|
} from '@/components/ui/dialog';
|
||||||
import { Input } from '@/components/ui/input';
|
import {
|
||||||
import { Label } from '@/components/ui/label';
|
Field,
|
||||||
|
FieldDescription,
|
||||||
|
FieldGroup,
|
||||||
|
FieldLabel,
|
||||||
|
} from '@/components/ui/field';
|
||||||
import { Progress } from '@/components/ui/progress';
|
import { Progress } from '@/components/ui/progress';
|
||||||
import { Tabs, TabsContent, TabsList, TabsTrigger } from '@/components/ui/tabs';
|
import { Spinner } from '@/components/ui/spinner';
|
||||||
import { PagesService } from '@/lib/services/openflare';
|
import { PagesService } from '@/lib/services/openflare';
|
||||||
import { cn } from '@/lib/utils';
|
import { cn } from '@/lib/utils';
|
||||||
|
|
||||||
@@ -46,52 +50,51 @@ const PAGES_PACKAGE_EXTENSIONS = [
|
|||||||
|
|
||||||
function isSupportedPagesPackage(fileName: string) {
|
function isSupportedPagesPackage(fileName: string) {
|
||||||
const lower = fileName.toLowerCase();
|
const lower = fileName.toLowerCase();
|
||||||
return PAGES_PACKAGE_EXTENSIONS.some((ext) => lower.endsWith(ext));
|
return PAGES_PACKAGE_EXTENSIONS.some((extension) =>
|
||||||
|
lower.endsWith(extension),
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
|
export function pagesEntryPath(rootDir: string, entryFile: string) {
|
||||||
|
const root = rootDir.trim().replace(/^\/+|\/+$/g, '');
|
||||||
|
const entry = entryFile.trim().replace(/^\/+/, '');
|
||||||
|
return root ? `${root}/${entry}` : entry;
|
||||||
}
|
}
|
||||||
|
|
||||||
interface DeploymentUploadDialogProps {
|
interface DeploymentUploadDialogProps {
|
||||||
open: boolean;
|
open: boolean;
|
||||||
onOpenChange: (open: boolean) => void;
|
onOpenChange: (open: boolean) => void;
|
||||||
projectId: number;
|
projectId: number;
|
||||||
|
rootDir: string;
|
||||||
|
entryFile: string;
|
||||||
}
|
}
|
||||||
|
|
||||||
export function DeploymentUploadDialog({
|
export function DeploymentUploadDialog({
|
||||||
open,
|
open,
|
||||||
onOpenChange,
|
onOpenChange,
|
||||||
projectId,
|
projectId,
|
||||||
|
rootDir,
|
||||||
|
entryFile,
|
||||||
}: DeploymentUploadDialogProps) {
|
}: DeploymentUploadDialogProps) {
|
||||||
const queryClient = useQueryClient();
|
const queryClient = useQueryClient();
|
||||||
const fileInputRef = useRef<HTMLInputElement>(null);
|
const fileInputRef = useRef<HTMLInputElement>(null);
|
||||||
const [mode, setMode] = useState<'file' | 'url'>('file');
|
|
||||||
const [file, setFile] = useState<File | null>(null);
|
const [file, setFile] = useState<File | null>(null);
|
||||||
const [packageURL, setPackageURL] = useState('');
|
|
||||||
const [isDragActive, setIsDragActive] = useState(false);
|
const [isDragActive, setIsDragActive] = useState(false);
|
||||||
const [uploadProgress, setUploadProgress] = useState<number | null>(null);
|
const [uploadProgress, setUploadProgress] = useState<number | null>(null);
|
||||||
|
|
||||||
const resetForm = () => {
|
const resetForm = () => {
|
||||||
setFile(null);
|
setFile(null);
|
||||||
setPackageURL('');
|
|
||||||
setIsDragActive(false);
|
setIsDragActive(false);
|
||||||
setUploadProgress(null);
|
setUploadProgress(null);
|
||||||
if (fileInputRef.current) fileInputRef.current.value = '';
|
if (fileInputRef.current) fileInputRef.current.value = '';
|
||||||
};
|
};
|
||||||
|
|
||||||
const handleClose = (nextOpen: boolean) => {
|
const handleOpenChange = (nextOpen: boolean) => {
|
||||||
if (!nextOpen) resetForm();
|
if (!nextOpen) resetForm();
|
||||||
onOpenChange(nextOpen);
|
onOpenChange(nextOpen);
|
||||||
};
|
};
|
||||||
|
|
||||||
const invalidate = async () => {
|
const uploadMutation = useMutation({
|
||||||
await Promise.all([
|
|
||||||
queryClient.invalidateQueries({
|
|
||||||
queryKey: deploymentsQueryKey(projectId),
|
|
||||||
}),
|
|
||||||
queryClient.invalidateQueries({ queryKey: projectQueryKey(projectId) }),
|
|
||||||
queryClient.invalidateQueries({ queryKey: projectsQueryKey }),
|
|
||||||
]);
|
|
||||||
};
|
|
||||||
|
|
||||||
const uploadFileMutation = useMutation({
|
|
||||||
mutationFn: () => {
|
mutationFn: () => {
|
||||||
if (!file) throw new Error('请选择部署包');
|
if (!file) throw new Error('请选择部署包');
|
||||||
return PagesService.uploadDeployment(projectId, {
|
return PagesService.uploadDeployment(projectId, {
|
||||||
@@ -101,8 +104,14 @@ export function DeploymentUploadDialog({
|
|||||||
},
|
},
|
||||||
onSuccess: async () => {
|
onSuccess: async () => {
|
||||||
toast.success('部署包上传成功');
|
toast.success('部署包上传成功');
|
||||||
await invalidate();
|
await Promise.all([
|
||||||
handleClose(false);
|
queryClient.invalidateQueries({
|
||||||
|
queryKey: deploymentsQueryKey(projectId),
|
||||||
|
}),
|
||||||
|
queryClient.invalidateQueries({ queryKey: projectQueryKey(projectId) }),
|
||||||
|
queryClient.invalidateQueries({ queryKey: projectsQueryKey }),
|
||||||
|
]);
|
||||||
|
handleOpenChange(false);
|
||||||
},
|
},
|
||||||
onError: (error) => {
|
onError: (error) => {
|
||||||
toast.error(error instanceof Error ? error.message : '上传失败');
|
toast.error(error instanceof Error ? error.message : '上传失败');
|
||||||
@@ -110,27 +119,6 @@ export function DeploymentUploadDialog({
|
|||||||
},
|
},
|
||||||
});
|
});
|
||||||
|
|
||||||
const uploadURLMutation = useMutation({
|
|
||||||
mutationFn: () => {
|
|
||||||
const url = packageURL.trim();
|
|
||||||
if (!url) throw new Error('请填写部署包下载链接');
|
|
||||||
if (!/^https?:\/\//i.test(url)) {
|
|
||||||
throw new Error('链接必须以 http:// 或 https:// 开头');
|
|
||||||
}
|
|
||||||
return PagesService.uploadDeploymentFromURL(projectId, { url });
|
|
||||||
},
|
|
||||||
onSuccess: async () => {
|
|
||||||
toast.success('已从链接下载并创建部署');
|
|
||||||
await invalidate();
|
|
||||||
handleClose(false);
|
|
||||||
},
|
|
||||||
onError: (error) => {
|
|
||||||
toast.error(error instanceof Error ? error.message : '从链接导入失败');
|
|
||||||
},
|
|
||||||
});
|
|
||||||
|
|
||||||
const isPending = uploadFileMutation.isPending || uploadURLMutation.isPending;
|
|
||||||
|
|
||||||
const handleFileSelect = (selected: File | null) => {
|
const handleFileSelect = (selected: File | null) => {
|
||||||
if (!selected) return;
|
if (!selected) return;
|
||||||
if (!isSupportedPagesPackage(selected.name)) {
|
if (!isSupportedPagesPackage(selected.name)) {
|
||||||
@@ -140,150 +128,112 @@ export function DeploymentUploadDialog({
|
|||||||
setFile(selected);
|
setFile(selected);
|
||||||
};
|
};
|
||||||
|
|
||||||
const handleSubmit = () => {
|
|
||||||
if (mode === 'file') {
|
|
||||||
uploadFileMutation.mutate();
|
|
||||||
return;
|
|
||||||
}
|
|
||||||
uploadURLMutation.mutate();
|
|
||||||
};
|
|
||||||
|
|
||||||
const canSubmit =
|
|
||||||
mode === 'file' ? Boolean(file) : packageURL.trim().length > 0;
|
|
||||||
|
|
||||||
return (
|
return (
|
||||||
<Dialog open={open} onOpenChange={handleClose}>
|
<Dialog open={open} onOpenChange={handleOpenChange}>
|
||||||
<DialogContent className='sm:max-w-lg'>
|
<DialogContent className='sm:max-w-lg'>
|
||||||
<DialogHeader>
|
<DialogHeader>
|
||||||
<DialogTitle>上传部署包</DialogTitle>
|
<DialogTitle>上传部署包</DialogTitle>
|
||||||
<DialogDescription>
|
<DialogDescription>
|
||||||
支持本地上传或从 URL 下载静态资源压缩包(zip / tar.gz / tar.xz /
|
上传不可变的静态资源压缩包,完成后可在部署历史中激活。
|
||||||
tar.bz2 / tar / 7z),创建部署后可在列表中激活。
|
|
||||||
</DialogDescription>
|
</DialogDescription>
|
||||||
</DialogHeader>
|
</DialogHeader>
|
||||||
|
|
||||||
<Tabs
|
<FieldGroup>
|
||||||
value={mode}
|
<Field>
|
||||||
onValueChange={(value) => setMode(value as 'file' | 'url')}
|
<FieldLabel htmlFor='pages-package'>本地部署包</FieldLabel>
|
||||||
className='w-full'
|
<button
|
||||||
>
|
type='button'
|
||||||
<TabsList className='grid w-full grid-cols-2'>
|
|
||||||
<TabsTrigger value='file'>本地上传</TabsTrigger>
|
|
||||||
<TabsTrigger value='url'>从 URL 下载</TabsTrigger>
|
|
||||||
</TabsList>
|
|
||||||
|
|
||||||
<TabsContent value='file' className='space-y-4 mt-4'>
|
|
||||||
<div
|
|
||||||
className={cn(
|
className={cn(
|
||||||
'rounded-lg border border-dashed p-8 text-center transition',
|
'flex min-h-52 w-full flex-col items-center justify-center gap-3 rounded-lg border border-dashed p-8 text-center transition-colors',
|
||||||
isDragActive ? 'border-primary bg-primary/5' : 'bg-muted/20',
|
isDragActive ? 'border-primary bg-primary/5' : 'bg-muted/20',
|
||||||
)}
|
)}
|
||||||
onDragEnter={(e) => {
|
onClick={() => fileInputRef.current?.click()}
|
||||||
e.preventDefault();
|
onDragEnter={(event) => {
|
||||||
|
event.preventDefault();
|
||||||
setIsDragActive(true);
|
setIsDragActive(true);
|
||||||
}}
|
}}
|
||||||
onDragOver={(e) => e.preventDefault()}
|
onDragOver={(event) => event.preventDefault()}
|
||||||
onDragLeave={(e) => {
|
onDragLeave={(event) => {
|
||||||
e.preventDefault();
|
event.preventDefault();
|
||||||
setIsDragActive(false);
|
setIsDragActive(false);
|
||||||
}}
|
}}
|
||||||
onDrop={(e) => {
|
onDrop={(event) => {
|
||||||
e.preventDefault();
|
event.preventDefault();
|
||||||
setIsDragActive(false);
|
setIsDragActive(false);
|
||||||
handleFileSelect(e.dataTransfer.files[0] ?? null);
|
handleFileSelect(event.dataTransfer.files[0] ?? null);
|
||||||
}}
|
}}
|
||||||
>
|
>
|
||||||
<UploadCloud className='size-8 mx-auto text-muted-foreground' />
|
<UploadCloud className='size-8 text-muted-foreground' />
|
||||||
<p className='mt-3 text-sm'>拖拽部署包到此处,或点击选择文件</p>
|
<span className='text-sm font-medium'>
|
||||||
<p className='mt-1 text-xs text-muted-foreground'>
|
拖拽部署包到此处,或点击选择文件
|
||||||
支持 zip、tar.gz、tar.xz、tar.bz2、tar、7z
|
</span>
|
||||||
</p>
|
<span className='text-xs text-muted-foreground'>
|
||||||
<Button
|
zip、tar.gz、tar.xz、tar.bz2、tar、7z
|
||||||
type='button'
|
</span>
|
||||||
variant='outline'
|
</button>
|
||||||
size='sm'
|
<input
|
||||||
className='mt-3'
|
ref={fileInputRef}
|
||||||
onClick={() => fileInputRef.current?.click()}
|
id='pages-package'
|
||||||
>
|
type='file'
|
||||||
选择文件
|
accept={PAGES_PACKAGE_ACCEPT}
|
||||||
</Button>
|
className='hidden'
|
||||||
<input
|
onChange={(event) =>
|
||||||
ref={fileInputRef}
|
handleFileSelect(event.target.files?.[0] ?? null)
|
||||||
type='file'
|
}
|
||||||
accept={PAGES_PACKAGE_ACCEPT}
|
/>
|
||||||
className='hidden'
|
|
||||||
onChange={(e) => handleFileSelect(e.target.files?.[0] ?? null)}
|
|
||||||
/>
|
|
||||||
</div>
|
|
||||||
|
|
||||||
{file ? (
|
{file ? (
|
||||||
<div className='rounded-lg border border-dashed px-4 py-3 text-sm'>
|
<FieldDescription>
|
||||||
<p className='font-medium'>{file.name}</p>
|
已选择 {file.name}({formatBytes(file.size)})
|
||||||
<p className='text-xs text-muted-foreground mt-1'>
|
</FieldDescription>
|
||||||
{formatBytes(file.size)}
|
) : (
|
||||||
</p>
|
<FieldDescription>请选择一个受支持的压缩包。</FieldDescription>
|
||||||
</div>
|
)}
|
||||||
) : null}
|
</Field>
|
||||||
|
|
||||||
{uploadProgress !== null ? (
|
<Field>
|
||||||
<div className='space-y-1.5'>
|
<FieldLabel>部署入口</FieldLabel>
|
||||||
<div className='flex justify-between text-xs text-muted-foreground'>
|
<div className='rounded-md border bg-muted/20 px-3 py-2 font-mono text-sm'>
|
||||||
<span>
|
{pagesEntryPath(rootDir, entryFile)}
|
||||||
{uploadProgress >= 100 ? '服务端处理中' : '上传进度'}
|
|
||||||
</span>
|
|
||||||
<span>
|
|
||||||
{uploadProgress >= 100 ? '请稍候' : `${uploadProgress}%`}
|
|
||||||
</span>
|
|
||||||
</div>
|
|
||||||
<Progress
|
|
||||||
value={uploadProgress >= 100 ? 100 : uploadProgress}
|
|
||||||
/>
|
|
||||||
</div>
|
|
||||||
) : null}
|
|
||||||
</TabsContent>
|
|
||||||
|
|
||||||
<TabsContent value='url' className='space-y-4 mt-4'>
|
|
||||||
<div className='space-y-2'>
|
|
||||||
<Label htmlFor='package-url'>部署包下载链接</Label>
|
|
||||||
<div className='relative'>
|
|
||||||
<Link2 className='absolute left-3 top-1/2 size-4 -translate-y-1/2 text-muted-foreground' />
|
|
||||||
<Input
|
|
||||||
id='package-url'
|
|
||||||
className='pl-9'
|
|
||||||
placeholder='https://example.com/dist/site.zip'
|
|
||||||
value={packageURL}
|
|
||||||
onChange={(e) => setPackageURL(e.target.value)}
|
|
||||||
disabled={isPending}
|
|
||||||
/>
|
|
||||||
</div>
|
|
||||||
<p className='text-xs text-muted-foreground'>
|
|
||||||
服务端将使用浏览器环境请求头从该链接下载压缩包,支持内网地址与自签证书
|
|
||||||
HTTPS。
|
|
||||||
</p>
|
|
||||||
</div>
|
</div>
|
||||||
</TabsContent>
|
<FieldDescription>
|
||||||
</Tabs>
|
入口来自项目设置;部署包上传不会覆盖该配置。
|
||||||
|
</FieldDescription>
|
||||||
|
</Field>
|
||||||
|
|
||||||
<div className='space-y-1.5'>
|
{uploadProgress !== null ? (
|
||||||
<Label htmlFor='entryFile'>入口文件</Label>
|
<Field>
|
||||||
<Input id='entryFile' defaultValue='index.html' disabled />
|
<div className='flex items-center justify-between text-xs text-muted-foreground'>
|
||||||
</div>
|
<span>
|
||||||
|
{uploadProgress >= 100 ? '服务端处理中' : '上传进度'}
|
||||||
|
</span>
|
||||||
|
<span>
|
||||||
|
{uploadProgress >= 100 ? '请稍候' : `${uploadProgress}%`}
|
||||||
|
</span>
|
||||||
|
</div>
|
||||||
|
<Progress value={Math.min(uploadProgress, 100)} />
|
||||||
|
</Field>
|
||||||
|
) : null}
|
||||||
|
</FieldGroup>
|
||||||
|
|
||||||
<DialogFooter>
|
<DialogFooter>
|
||||||
<Button variant='outline' onClick={() => handleClose(false)}>
|
<Button
|
||||||
|
type='button'
|
||||||
|
variant='outline'
|
||||||
|
onClick={() => handleOpenChange(false)}
|
||||||
|
>
|
||||||
取消
|
取消
|
||||||
</Button>
|
</Button>
|
||||||
<Button onClick={handleSubmit} disabled={!canSubmit || isPending}>
|
<Button
|
||||||
{isPending ? (
|
type='button'
|
||||||
<>
|
disabled={!file || uploadMutation.isPending}
|
||||||
<Loader2 className='size-4 animate-spin mr-1' />
|
onClick={() => uploadMutation.mutate()}
|
||||||
{mode === 'url' ? '下载中...' : '上传中...'}
|
>
|
||||||
</>
|
{uploadMutation.isPending ? (
|
||||||
) : mode === 'url' ? (
|
<Spinner data-icon='inline-start' />
|
||||||
'下载并创建部署'
|
|
||||||
) : (
|
) : (
|
||||||
'上传并创建部署'
|
<UploadCloud data-icon='inline-start' />
|
||||||
)}
|
)}
|
||||||
|
{uploadMutation.isPending ? '上传中...' : '上传并创建部署'}
|
||||||
</Button>
|
</Button>
|
||||||
</DialogFooter>
|
</DialogFooter>
|
||||||
</DialogContent>
|
</DialogContent>
|
||||||
|
|||||||
@@ -17,6 +17,10 @@ export function deploymentsQueryKey(projectId: number) {
|
|||||||
return ['openflare', 'pages', 'deployments', projectId] as const;
|
return ['openflare', 'pages', 'deployments', projectId] as const;
|
||||||
}
|
}
|
||||||
|
|
||||||
|
export function sourceQueryKey(projectId: number) {
|
||||||
|
return ['openflare', 'pages', 'source', projectId] as const;
|
||||||
|
}
|
||||||
|
|
||||||
export function deploymentFilesQueryKey(
|
export function deploymentFilesQueryKey(
|
||||||
projectId: number,
|
projectId: number,
|
||||||
deploymentId: number,
|
deploymentId: number,
|
||||||
|
|||||||
@@ -0,0 +1,86 @@
|
|||||||
|
'use client';
|
||||||
|
|
||||||
|
import { useQuery } from '@tanstack/react-query';
|
||||||
|
|
||||||
|
import { EmptyInline } from '@/components/layout/empty';
|
||||||
|
import { ErrorInline } from '@/components/layout/error';
|
||||||
|
import { Skeleton } from '@/components/ui/skeleton';
|
||||||
|
import {
|
||||||
|
Table,
|
||||||
|
TableBody,
|
||||||
|
TableCell,
|
||||||
|
TableHead,
|
||||||
|
TableHeader,
|
||||||
|
TableRow,
|
||||||
|
} from '@/components/ui/table';
|
||||||
|
import { PagesService } from '@/lib/services/openflare';
|
||||||
|
|
||||||
|
import {
|
||||||
|
deploymentFilesQueryKey,
|
||||||
|
formatBytes,
|
||||||
|
} from '../../components/pages-utils';
|
||||||
|
|
||||||
|
interface DeploymentFilesPanelProps {
|
||||||
|
projectId: number;
|
||||||
|
deploymentId: number;
|
||||||
|
}
|
||||||
|
|
||||||
|
export function DeploymentFilesPanel({
|
||||||
|
projectId,
|
||||||
|
deploymentId,
|
||||||
|
}: DeploymentFilesPanelProps) {
|
||||||
|
const filesQuery = useQuery({
|
||||||
|
queryKey: deploymentFilesQueryKey(projectId, deploymentId),
|
||||||
|
queryFn: () => PagesService.listDeploymentFiles(deploymentId),
|
||||||
|
});
|
||||||
|
|
||||||
|
if (filesQuery.isLoading) {
|
||||||
|
return (
|
||||||
|
<div className='flex flex-col gap-2 p-4'>
|
||||||
|
<Skeleton className='h-8 w-full' />
|
||||||
|
<Skeleton className='h-8 w-full' />
|
||||||
|
</div>
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
|
if (filesQuery.isError) {
|
||||||
|
return (
|
||||||
|
<div className='p-4'>
|
||||||
|
<ErrorInline
|
||||||
|
message={
|
||||||
|
filesQuery.error instanceof Error
|
||||||
|
? filesQuery.error.message
|
||||||
|
: '文件清单加载失败'
|
||||||
|
}
|
||||||
|
onRetry={() => void filesQuery.refetch()}
|
||||||
|
/>
|
||||||
|
</div>
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
|
const files = filesQuery.data ?? [];
|
||||||
|
if (files.length === 0) {
|
||||||
|
return <EmptyInline message='暂无文件记录' />;
|
||||||
|
}
|
||||||
|
|
||||||
|
return (
|
||||||
|
<Table>
|
||||||
|
<TableHeader>
|
||||||
|
<TableRow>
|
||||||
|
<TableHead>路径</TableHead>
|
||||||
|
<TableHead className='text-right'>大小</TableHead>
|
||||||
|
</TableRow>
|
||||||
|
</TableHeader>
|
||||||
|
<TableBody>
|
||||||
|
{files.map((file) => (
|
||||||
|
<TableRow key={file.id}>
|
||||||
|
<TableCell className='font-mono text-xs'>{file.path}</TableCell>
|
||||||
|
<TableCell className='text-right text-xs text-muted-foreground'>
|
||||||
|
{formatBytes(file.size)}
|
||||||
|
</TableCell>
|
||||||
|
</TableRow>
|
||||||
|
))}
|
||||||
|
</TableBody>
|
||||||
|
</Table>
|
||||||
|
);
|
||||||
|
}
|
||||||
@@ -0,0 +1,300 @@
|
|||||||
|
'use client';
|
||||||
|
|
||||||
|
import { useMemo, useState } from 'react';
|
||||||
|
import { useMutation, useQuery, useQueryClient } from '@tanstack/react-query';
|
||||||
|
import { ChevronDown, ChevronRight } from 'lucide-react';
|
||||||
|
import { toast } from 'sonner';
|
||||||
|
|
||||||
|
import { EmptyStateWithBorder } from '@/components/layout/empty';
|
||||||
|
import { ErrorInline } from '@/components/layout/error';
|
||||||
|
import { LoadingStateWithBorder } from '@/components/layout/loading';
|
||||||
|
import {
|
||||||
|
AlertDialog,
|
||||||
|
AlertDialogAction,
|
||||||
|
AlertDialogCancel,
|
||||||
|
AlertDialogContent,
|
||||||
|
AlertDialogDescription,
|
||||||
|
AlertDialogFooter,
|
||||||
|
AlertDialogHeader,
|
||||||
|
AlertDialogTitle,
|
||||||
|
} from '@/components/ui/alert-dialog';
|
||||||
|
import { Badge } from '@/components/ui/badge';
|
||||||
|
import { Button } from '@/components/ui/button';
|
||||||
|
import {
|
||||||
|
Card,
|
||||||
|
CardContent,
|
||||||
|
CardDescription,
|
||||||
|
CardHeader,
|
||||||
|
CardTitle,
|
||||||
|
} from '@/components/ui/card';
|
||||||
|
import { Spinner } from '@/components/ui/spinner';
|
||||||
|
import { type PagesDeployment, PagesService } from '@/lib/services/openflare';
|
||||||
|
import { formatDateTime } from '@/lib/utils';
|
||||||
|
|
||||||
|
import {
|
||||||
|
deploymentFilesQueryKey,
|
||||||
|
deploymentsQueryKey,
|
||||||
|
formatBytes,
|
||||||
|
projectQueryKey,
|
||||||
|
projectsQueryKey,
|
||||||
|
sourceQueryKey,
|
||||||
|
} from '../../components/pages-utils';
|
||||||
|
import { DeploymentFilesPanel } from './deployment-files-panel';
|
||||||
|
|
||||||
|
const SOURCE_LABELS: Record<PagesDeployment['source_type'], string> = {
|
||||||
|
manual_upload: '本地上传',
|
||||||
|
manual_url: 'URL 导入',
|
||||||
|
remote_url: 'Remote URL',
|
||||||
|
github_release: 'GitHub',
|
||||||
|
};
|
||||||
|
|
||||||
|
const TRIGGER_LABELS: Record<PagesDeployment['trigger_type'], string> = {
|
||||||
|
manual_upload: '手动上传',
|
||||||
|
manual_url: '手动导入',
|
||||||
|
manual_sync: '手动同步',
|
||||||
|
scheduled_auto_update: '定时更新',
|
||||||
|
};
|
||||||
|
|
||||||
|
interface DeploymentHistoryProps {
|
||||||
|
projectId: number;
|
||||||
|
activeDeploymentId?: number | null;
|
||||||
|
}
|
||||||
|
|
||||||
|
type PendingAction = {
|
||||||
|
type: 'activate' | 'delete';
|
||||||
|
deployment: PagesDeployment;
|
||||||
|
};
|
||||||
|
|
||||||
|
function deploymentSnapshot(deployment: PagesDeployment) {
|
||||||
|
return [
|
||||||
|
SOURCE_LABELS[deployment.source_type],
|
||||||
|
deployment.source_label,
|
||||||
|
TRIGGER_LABELS[deployment.trigger_type],
|
||||||
|
]
|
||||||
|
.filter(Boolean)
|
||||||
|
.join(' · ');
|
||||||
|
}
|
||||||
|
|
||||||
|
export function DeploymentHistory({
|
||||||
|
projectId,
|
||||||
|
activeDeploymentId,
|
||||||
|
}: DeploymentHistoryProps) {
|
||||||
|
const queryClient = useQueryClient();
|
||||||
|
const [expandedDeploymentId, setExpandedDeploymentId] = useState<
|
||||||
|
number | null
|
||||||
|
>(null);
|
||||||
|
const [pendingAction, setPendingAction] = useState<PendingAction | null>(
|
||||||
|
null,
|
||||||
|
);
|
||||||
|
|
||||||
|
const deploymentsQuery = useQuery({
|
||||||
|
queryKey: deploymentsQueryKey(projectId),
|
||||||
|
queryFn: () => PagesService.listDeployments(projectId),
|
||||||
|
});
|
||||||
|
|
||||||
|
const deployments = useMemo(() => {
|
||||||
|
const records = [...(deploymentsQuery.data ?? [])];
|
||||||
|
return records.sort((left, right) => {
|
||||||
|
const leftActive =
|
||||||
|
left.id === activeDeploymentId || left.status === 'active';
|
||||||
|
const rightActive =
|
||||||
|
right.id === activeDeploymentId || right.status === 'active';
|
||||||
|
if (leftActive !== rightActive) return leftActive ? -1 : 1;
|
||||||
|
return right.deployment_number - left.deployment_number;
|
||||||
|
});
|
||||||
|
}, [activeDeploymentId, deploymentsQuery.data]);
|
||||||
|
|
||||||
|
const invalidateDeploymentState = async () => {
|
||||||
|
await Promise.all([
|
||||||
|
queryClient.invalidateQueries({
|
||||||
|
queryKey: deploymentsQueryKey(projectId),
|
||||||
|
}),
|
||||||
|
queryClient.invalidateQueries({ queryKey: projectQueryKey(projectId) }),
|
||||||
|
queryClient.invalidateQueries({ queryKey: sourceQueryKey(projectId) }),
|
||||||
|
queryClient.invalidateQueries({ queryKey: projectsQueryKey }),
|
||||||
|
]);
|
||||||
|
};
|
||||||
|
|
||||||
|
const activateMutation = useMutation({
|
||||||
|
mutationFn: (deploymentId: number) =>
|
||||||
|
PagesService.activateDeployment(projectId, deploymentId),
|
||||||
|
onSuccess: async () => {
|
||||||
|
toast.success('历史部署已激活;自动更新(如已开启)已关闭');
|
||||||
|
await invalidateDeploymentState();
|
||||||
|
setPendingAction(null);
|
||||||
|
},
|
||||||
|
onError: (error) => {
|
||||||
|
toast.error(error instanceof Error ? error.message : '激活失败');
|
||||||
|
},
|
||||||
|
});
|
||||||
|
|
||||||
|
const deleteMutation = useMutation({
|
||||||
|
mutationFn: (deploymentId: number) =>
|
||||||
|
PagesService.deleteDeployment(projectId, deploymentId),
|
||||||
|
onSuccess: async (_, deploymentId) => {
|
||||||
|
toast.success('部署已删除');
|
||||||
|
queryClient.removeQueries({
|
||||||
|
queryKey: deploymentFilesQueryKey(projectId, deploymentId),
|
||||||
|
});
|
||||||
|
await invalidateDeploymentState();
|
||||||
|
setPendingAction(null);
|
||||||
|
},
|
||||||
|
onError: (error) => {
|
||||||
|
toast.error(error instanceof Error ? error.message : '删除失败');
|
||||||
|
},
|
||||||
|
});
|
||||||
|
|
||||||
|
const actionPending = activateMutation.isPending || deleteMutation.isPending;
|
||||||
|
|
||||||
|
return (
|
||||||
|
<>
|
||||||
|
<Card>
|
||||||
|
<CardHeader>
|
||||||
|
<CardTitle>部署历史</CardTitle>
|
||||||
|
<CardDescription>
|
||||||
|
部署记录不可变,来源信息是创建部署时的安全快照。
|
||||||
|
</CardDescription>
|
||||||
|
</CardHeader>
|
||||||
|
<CardContent className='flex flex-col gap-3'>
|
||||||
|
{deploymentsQuery.isLoading ? (
|
||||||
|
<LoadingStateWithBorder description='加载部署历史...' />
|
||||||
|
) : deploymentsQuery.isError ? (
|
||||||
|
<div className='rounded-lg border p-4'>
|
||||||
|
<ErrorInline
|
||||||
|
message={
|
||||||
|
deploymentsQuery.error instanceof Error
|
||||||
|
? deploymentsQuery.error.message
|
||||||
|
: '部署历史加载失败'
|
||||||
|
}
|
||||||
|
onRetry={() => void deploymentsQuery.refetch()}
|
||||||
|
/>
|
||||||
|
</div>
|
||||||
|
) : deployments.length === 0 ? (
|
||||||
|
<EmptyStateWithBorder
|
||||||
|
title='暂无部署'
|
||||||
|
description='上传本地部署包,或配置 Remote URL / GitHub Release 来源后同步发布。'
|
||||||
|
/>
|
||||||
|
) : (
|
||||||
|
deployments.map((deployment) => {
|
||||||
|
const active =
|
||||||
|
deployment.id === activeDeploymentId ||
|
||||||
|
deployment.status === 'active';
|
||||||
|
const expanded = expandedDeploymentId === deployment.id;
|
||||||
|
|
||||||
|
return (
|
||||||
|
<div key={deployment.id} className='rounded-lg border'>
|
||||||
|
<div className='flex flex-col gap-4 p-4 md:flex-row md:items-center md:justify-between'>
|
||||||
|
<div className='flex min-w-0 items-start gap-2'>
|
||||||
|
<Button
|
||||||
|
type='button'
|
||||||
|
variant='ghost'
|
||||||
|
size='icon-sm'
|
||||||
|
aria-label={expanded ? '收起文件清单' : '展开文件清单'}
|
||||||
|
onClick={() =>
|
||||||
|
setExpandedDeploymentId(
|
||||||
|
expanded ? null : deployment.id,
|
||||||
|
)
|
||||||
|
}
|
||||||
|
>
|
||||||
|
{expanded ? <ChevronDown /> : <ChevronRight />}
|
||||||
|
</Button>
|
||||||
|
<div className='flex min-w-0 flex-col gap-2'>
|
||||||
|
<div className='flex flex-wrap items-center gap-2'>
|
||||||
|
<span className='text-sm font-medium'>
|
||||||
|
部署 #{deployment.deployment_number}
|
||||||
|
</span>
|
||||||
|
<Badge variant={active ? 'default' : 'outline'}>
|
||||||
|
{active ? '当前生产部署' : '历史部署'}
|
||||||
|
</Badge>
|
||||||
|
<Badge variant='secondary'>
|
||||||
|
{deploymentSnapshot(deployment)}
|
||||||
|
</Badge>
|
||||||
|
</div>
|
||||||
|
<p className='truncate text-xs text-muted-foreground'>
|
||||||
|
{deployment.checksum.slice(0, 16)} ·{' '}
|
||||||
|
{deployment.file_count} 个文件 ·{' '}
|
||||||
|
{formatBytes(deployment.total_size)}
|
||||||
|
</p>
|
||||||
|
<p className='text-xs text-muted-foreground'>
|
||||||
|
创建于 {formatDateTime(deployment.created_at)}
|
||||||
|
</p>
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
|
<div className='flex gap-2 md:ml-10'>
|
||||||
|
<Button
|
||||||
|
type='button'
|
||||||
|
variant='outline'
|
||||||
|
size='sm'
|
||||||
|
disabled={active || actionPending}
|
||||||
|
onClick={() =>
|
||||||
|
setPendingAction({ type: 'activate', deployment })
|
||||||
|
}
|
||||||
|
>
|
||||||
|
激活
|
||||||
|
</Button>
|
||||||
|
<Button
|
||||||
|
type='button'
|
||||||
|
variant='destructive'
|
||||||
|
size='sm'
|
||||||
|
disabled={active || actionPending}
|
||||||
|
onClick={() =>
|
||||||
|
setPendingAction({ type: 'delete', deployment })
|
||||||
|
}
|
||||||
|
>
|
||||||
|
删除
|
||||||
|
</Button>
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
|
{expanded ? (
|
||||||
|
<DeploymentFilesPanel
|
||||||
|
projectId={projectId}
|
||||||
|
deploymentId={deployment.id}
|
||||||
|
/>
|
||||||
|
) : null}
|
||||||
|
</div>
|
||||||
|
);
|
||||||
|
})
|
||||||
|
)}
|
||||||
|
</CardContent>
|
||||||
|
</Card>
|
||||||
|
|
||||||
|
<AlertDialog
|
||||||
|
open={pendingAction !== null}
|
||||||
|
onOpenChange={(open) => {
|
||||||
|
if (!open && !actionPending) setPendingAction(null);
|
||||||
|
}}
|
||||||
|
>
|
||||||
|
<AlertDialogContent>
|
||||||
|
<AlertDialogHeader>
|
||||||
|
<AlertDialogTitle>
|
||||||
|
{pendingAction?.type === 'activate' ? '激活历史部署' : '删除部署'}
|
||||||
|
</AlertDialogTitle>
|
||||||
|
<AlertDialogDescription>
|
||||||
|
{pendingAction?.type === 'activate'
|
||||||
|
? '激活其它历史部署会终止当前来源任务;若已开启自动更新,将同时关闭自动更新。'
|
||||||
|
: `确认删除部署 #${pendingAction?.deployment.deployment_number} 吗?此操作不可恢复。`}
|
||||||
|
</AlertDialogDescription>
|
||||||
|
</AlertDialogHeader>
|
||||||
|
<AlertDialogFooter>
|
||||||
|
<AlertDialogCancel disabled={actionPending}>取消</AlertDialogCancel>
|
||||||
|
<AlertDialogAction
|
||||||
|
disabled={actionPending}
|
||||||
|
onClick={(event) => {
|
||||||
|
event.preventDefault();
|
||||||
|
if (!pendingAction) return;
|
||||||
|
if (pendingAction.type === 'activate') {
|
||||||
|
activateMutation.mutate(pendingAction.deployment.id);
|
||||||
|
} else {
|
||||||
|
deleteMutation.mutate(pendingAction.deployment.id);
|
||||||
|
}
|
||||||
|
}}
|
||||||
|
>
|
||||||
|
{actionPending ? <Spinner data-icon='inline-start' /> : null}
|
||||||
|
确认
|
||||||
|
</AlertDialogAction>
|
||||||
|
</AlertDialogFooter>
|
||||||
|
</AlertDialogContent>
|
||||||
|
</AlertDialog>
|
||||||
|
</>
|
||||||
|
);
|
||||||
|
}
|
||||||
@@ -0,0 +1,613 @@
|
|||||||
|
'use client';
|
||||||
|
|
||||||
|
import { useCallback, useEffect, useRef, useState } from 'react';
|
||||||
|
import { useMutation, useQuery, useQueryClient } from '@tanstack/react-query';
|
||||||
|
import {
|
||||||
|
Download,
|
||||||
|
Github,
|
||||||
|
Pencil,
|
||||||
|
RefreshCw,
|
||||||
|
RotateCcw,
|
||||||
|
Search,
|
||||||
|
} from 'lucide-react';
|
||||||
|
import { toast } from 'sonner';
|
||||||
|
|
||||||
|
import { ErrorInline } from '@/components/layout/error';
|
||||||
|
import {
|
||||||
|
AlertDialog,
|
||||||
|
AlertDialogAction,
|
||||||
|
AlertDialogCancel,
|
||||||
|
AlertDialogContent,
|
||||||
|
AlertDialogDescription,
|
||||||
|
AlertDialogFooter,
|
||||||
|
AlertDialogHeader,
|
||||||
|
AlertDialogTitle,
|
||||||
|
} from '@/components/ui/alert-dialog';
|
||||||
|
import { Badge } from '@/components/ui/badge';
|
||||||
|
import { Button } from '@/components/ui/button';
|
||||||
|
import {
|
||||||
|
Card,
|
||||||
|
CardAction,
|
||||||
|
CardContent,
|
||||||
|
CardDescription,
|
||||||
|
CardFooter,
|
||||||
|
CardHeader,
|
||||||
|
CardTitle,
|
||||||
|
} from '@/components/ui/card';
|
||||||
|
import { Skeleton } from '@/components/ui/skeleton';
|
||||||
|
import { Spinner } from '@/components/ui/spinner';
|
||||||
|
import { AdminTaskService } from '@/lib/services/admin';
|
||||||
|
import {
|
||||||
|
type PagesSource,
|
||||||
|
type PagesSourceActionPayload,
|
||||||
|
type PagesSourceActionReceipt,
|
||||||
|
type PagesSourceStatus,
|
||||||
|
PagesService,
|
||||||
|
} from '@/lib/services/openflare';
|
||||||
|
|
||||||
|
import {
|
||||||
|
deploymentsQueryKey,
|
||||||
|
projectQueryKey,
|
||||||
|
projectsQueryKey,
|
||||||
|
sourceQueryKey,
|
||||||
|
} from '../../components/pages-utils';
|
||||||
|
import { type PagesSourceMode, PagesSourceDialog } from './pages-source-dialog';
|
||||||
|
import {
|
||||||
|
GitHubSourceDetails,
|
||||||
|
RemoteSourceDetails,
|
||||||
|
} from './pages-source-details';
|
||||||
|
|
||||||
|
const ACTION_POLL_INTERVAL = 2_000;
|
||||||
|
const ACTION_MAX_WAIT = 16 * 60 * 1_000;
|
||||||
|
const LATEST_POLL_INTERVAL = 5 * 60 * 1_000;
|
||||||
|
const LATEST_NEAR_DUE_POLL_INTERVAL = 30_000;
|
||||||
|
const LATEST_OVERDUE_MAX_WAIT = 10 * 60 * 1_000;
|
||||||
|
|
||||||
|
const SOURCE_STATUS: Record<
|
||||||
|
PagesSourceStatus,
|
||||||
|
{
|
||||||
|
label: string;
|
||||||
|
variant: 'default' | 'secondary' | 'destructive' | 'outline';
|
||||||
|
}
|
||||||
|
> = {
|
||||||
|
idle: { label: '空闲', variant: 'outline' },
|
||||||
|
checking: { label: '检查中', variant: 'secondary' },
|
||||||
|
update_available: { label: '有可用更新', variant: 'default' },
|
||||||
|
syncing: { label: '同步中', variant: 'secondary' },
|
||||||
|
failed: { label: '最近同步失败', variant: 'destructive' },
|
||||||
|
attention: { label: '需要确认', variant: 'destructive' },
|
||||||
|
};
|
||||||
|
|
||||||
|
interface ActiveAction {
|
||||||
|
receipt: PagesSourceActionReceipt;
|
||||||
|
startedAt: number;
|
||||||
|
}
|
||||||
|
|
||||||
|
export interface LatestSourceOverdueWindow {
|
||||||
|
nextCheckAt: string;
|
||||||
|
startedAt: number;
|
||||||
|
}
|
||||||
|
|
||||||
|
export interface LatestSourcePollingDecision {
|
||||||
|
interval: number | false;
|
||||||
|
overdueWindow: LatestSourceOverdueWindow | null;
|
||||||
|
}
|
||||||
|
|
||||||
|
export function getLatestSourceIdlePollingDecision(
|
||||||
|
source: PagesSource | undefined,
|
||||||
|
now: number,
|
||||||
|
overdueWindow: LatestSourceOverdueWindow | null,
|
||||||
|
): LatestSourcePollingDecision {
|
||||||
|
if (
|
||||||
|
source?.source_type !== 'github_release' ||
|
||||||
|
source.release_selector !== 'latest'
|
||||||
|
) {
|
||||||
|
return { interval: false, overdueWindow: null };
|
||||||
|
}
|
||||||
|
|
||||||
|
const nextCheckAt = source.next_check_at;
|
||||||
|
const nextCheckTime = nextCheckAt ? Date.parse(nextCheckAt) : Number.NaN;
|
||||||
|
if (!nextCheckAt || !Number.isFinite(nextCheckTime)) {
|
||||||
|
return { interval: LATEST_POLL_INTERVAL, overdueWindow: null };
|
||||||
|
}
|
||||||
|
|
||||||
|
const timeUntilCheck = nextCheckTime - now;
|
||||||
|
if (timeUntilCheck > LATEST_POLL_INTERVAL) {
|
||||||
|
return { interval: LATEST_POLL_INTERVAL, overdueWindow: null };
|
||||||
|
}
|
||||||
|
if (timeUntilCheck > 0) {
|
||||||
|
return { interval: LATEST_NEAR_DUE_POLL_INTERVAL, overdueWindow: null };
|
||||||
|
}
|
||||||
|
|
||||||
|
const currentWindow =
|
||||||
|
overdueWindow?.nextCheckAt === nextCheckAt
|
||||||
|
? overdueWindow
|
||||||
|
: { nextCheckAt, startedAt: now };
|
||||||
|
if (now - currentWindow.startedAt >= LATEST_OVERDUE_MAX_WAIT) {
|
||||||
|
return { interval: false, overdueWindow: currentWindow };
|
||||||
|
}
|
||||||
|
return {
|
||||||
|
interval: LATEST_NEAR_DUE_POLL_INTERVAL,
|
||||||
|
overdueWindow: currentWindow,
|
||||||
|
};
|
||||||
|
}
|
||||||
|
|
||||||
|
function sourceDeploymentFingerprint(source: PagesSource) {
|
||||||
|
if (source.source_type === 'manual') return '|';
|
||||||
|
return `${source.last_synced_at ?? ''}|${source.last_applied?.revision ?? ''}`;
|
||||||
|
}
|
||||||
|
|
||||||
|
function sourceActionLabel(action: PagesSourceActionReceipt['action']) {
|
||||||
|
return action === 'check' ? '检查' : '同步并发布';
|
||||||
|
}
|
||||||
|
|
||||||
|
export function PagesSourceCard({ projectId }: { projectId: number }) {
|
||||||
|
const queryClient = useQueryClient();
|
||||||
|
const handledExecutionID = useRef<string | null>(null);
|
||||||
|
const sourcePollingStartedAt = useRef<number | null>(null);
|
||||||
|
const latestOverdueWindow = useRef<LatestSourceOverdueWindow | null>(null);
|
||||||
|
const sourceDeploymentState = useRef<string | undefined>(undefined);
|
||||||
|
const [dialogOpen, setDialogOpen] = useState(false);
|
||||||
|
const [dialogMode, setDialogMode] = useState<PagesSourceMode>('manual');
|
||||||
|
const [activeAction, setActiveAction] = useState<ActiveAction | null>(null);
|
||||||
|
const [actionTimedOut, setActionTimedOut] = useState(false);
|
||||||
|
const [attentionDialogOpen, setAttentionDialogOpen] = useState(false);
|
||||||
|
|
||||||
|
const sourceQuery = useQuery({
|
||||||
|
queryKey: sourceQueryKey(projectId),
|
||||||
|
queryFn: () => PagesService.getSource(projectId),
|
||||||
|
refetchInterval: (query) => {
|
||||||
|
const source = query.state.data;
|
||||||
|
if (
|
||||||
|
source &&
|
||||||
|
source.source_type !== 'manual' &&
|
||||||
|
(source.sync_status === 'checking' || source.sync_status === 'syncing')
|
||||||
|
) {
|
||||||
|
sourcePollingStartedAt.current ??= Date.now();
|
||||||
|
latestOverdueWindow.current = null;
|
||||||
|
return Date.now() - sourcePollingStartedAt.current < ACTION_MAX_WAIT
|
||||||
|
? ACTION_POLL_INTERVAL
|
||||||
|
: false;
|
||||||
|
}
|
||||||
|
sourcePollingStartedAt.current = null;
|
||||||
|
const decision = getLatestSourceIdlePollingDecision(
|
||||||
|
source,
|
||||||
|
Date.now(),
|
||||||
|
latestOverdueWindow.current,
|
||||||
|
);
|
||||||
|
latestOverdueWindow.current = decision.overdueWindow;
|
||||||
|
return decision.interval;
|
||||||
|
},
|
||||||
|
});
|
||||||
|
|
||||||
|
const invalidateSourceState = useCallback(
|
||||||
|
() =>
|
||||||
|
Promise.all([
|
||||||
|
queryClient.invalidateQueries({ queryKey: sourceQueryKey(projectId) }),
|
||||||
|
queryClient.invalidateQueries({
|
||||||
|
queryKey: projectQueryKey(projectId),
|
||||||
|
}),
|
||||||
|
queryClient.invalidateQueries({
|
||||||
|
queryKey: deploymentsQueryKey(projectId),
|
||||||
|
}),
|
||||||
|
queryClient.invalidateQueries({
|
||||||
|
queryKey: ['openflare', 'pages', 'deployment-files', projectId],
|
||||||
|
}),
|
||||||
|
queryClient.invalidateQueries({ queryKey: projectsQueryKey }),
|
||||||
|
]),
|
||||||
|
[projectId, queryClient],
|
||||||
|
);
|
||||||
|
|
||||||
|
const executionQuery = useQuery({
|
||||||
|
queryKey: [
|
||||||
|
'admin',
|
||||||
|
'task-execution',
|
||||||
|
activeAction?.receipt.execution_id ?? '',
|
||||||
|
],
|
||||||
|
queryFn: () => {
|
||||||
|
const executionID = activeAction?.receipt.execution_id;
|
||||||
|
if (!executionID) throw new Error('缺少任务执行 ID');
|
||||||
|
return AdminTaskService.getTaskExecution(executionID);
|
||||||
|
},
|
||||||
|
enabled: Boolean(activeAction) && !actionTimedOut,
|
||||||
|
refetchInterval: (query) => {
|
||||||
|
if (actionTimedOut) return false;
|
||||||
|
const status = query.state.data?.status;
|
||||||
|
return status === 'pending' || status === 'running'
|
||||||
|
? ACTION_POLL_INTERVAL
|
||||||
|
: false;
|
||||||
|
},
|
||||||
|
});
|
||||||
|
|
||||||
|
const beginActionPolling = useCallback(
|
||||||
|
(receipt: PagesSourceActionReceipt) => {
|
||||||
|
handledExecutionID.current = null;
|
||||||
|
setActiveAction({ receipt, startedAt: Date.now() });
|
||||||
|
setActionTimedOut(false);
|
||||||
|
},
|
||||||
|
[],
|
||||||
|
);
|
||||||
|
|
||||||
|
useEffect(() => {
|
||||||
|
if (!activeAction || actionTimedOut) return;
|
||||||
|
const elapsed = Date.now() - activeAction.startedAt;
|
||||||
|
const remaining = Math.max(0, ACTION_MAX_WAIT - elapsed);
|
||||||
|
const timeout = window.setTimeout(() => setActionTimedOut(true), remaining);
|
||||||
|
return () => window.clearTimeout(timeout);
|
||||||
|
}, [actionTimedOut, activeAction]);
|
||||||
|
|
||||||
|
useEffect(() => {
|
||||||
|
const source = sourceQuery.data;
|
||||||
|
if (!source) return;
|
||||||
|
const fingerprint = sourceDeploymentFingerprint(source);
|
||||||
|
const previousFingerprint = sourceDeploymentState.current;
|
||||||
|
sourceDeploymentState.current = fingerprint;
|
||||||
|
if (
|
||||||
|
previousFingerprint === undefined ||
|
||||||
|
previousFingerprint === fingerprint
|
||||||
|
) {
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
void invalidateSourceState();
|
||||||
|
}, [invalidateSourceState, sourceQuery.data]);
|
||||||
|
|
||||||
|
useEffect(() => {
|
||||||
|
const execution = executionQuery.data;
|
||||||
|
if (
|
||||||
|
!activeAction ||
|
||||||
|
!execution ||
|
||||||
|
!['succeeded', 'failed'].includes(execution.status)
|
||||||
|
) {
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
if (handledExecutionID.current === execution.id) return;
|
||||||
|
handledExecutionID.current = execution.id;
|
||||||
|
|
||||||
|
void invalidateSourceState();
|
||||||
|
|
||||||
|
const actionLabel = sourceActionLabel(activeAction.receipt.action);
|
||||||
|
if (execution.status === 'succeeded') {
|
||||||
|
toast.success(`部署源${actionLabel}完成`);
|
||||||
|
} else {
|
||||||
|
toast.error(execution.error_message || `部署源${actionLabel}失败`);
|
||||||
|
}
|
||||||
|
setActiveAction(null);
|
||||||
|
setActionTimedOut(false);
|
||||||
|
}, [activeAction, executionQuery.data, invalidateSourceState]);
|
||||||
|
|
||||||
|
const checkMutation = useMutation({
|
||||||
|
mutationFn: () => PagesService.checkSource(projectId),
|
||||||
|
onSuccess: async (receipt) => {
|
||||||
|
beginActionPolling(receipt);
|
||||||
|
await queryClient.invalidateQueries({
|
||||||
|
queryKey: sourceQueryKey(projectId),
|
||||||
|
});
|
||||||
|
toast.success('检查任务已提交');
|
||||||
|
},
|
||||||
|
onError: (error) => {
|
||||||
|
toast.error(error instanceof Error ? error.message : '检查任务提交失败');
|
||||||
|
},
|
||||||
|
});
|
||||||
|
|
||||||
|
const syncMutation = useMutation({
|
||||||
|
mutationFn: (payload: PagesSourceActionPayload) =>
|
||||||
|
PagesService.syncSource(projectId, payload),
|
||||||
|
onSuccess: async (receipt) => {
|
||||||
|
setAttentionDialogOpen(false);
|
||||||
|
beginActionPolling(receipt);
|
||||||
|
await queryClient.invalidateQueries({
|
||||||
|
queryKey: sourceQueryKey(projectId),
|
||||||
|
});
|
||||||
|
toast.success('同步任务已提交');
|
||||||
|
},
|
||||||
|
onError: (error) => {
|
||||||
|
toast.error(error instanceof Error ? error.message : '同步任务提交失败');
|
||||||
|
},
|
||||||
|
});
|
||||||
|
|
||||||
|
const source = sourceQuery.data;
|
||||||
|
const executionBusy =
|
||||||
|
activeAction !== null &&
|
||||||
|
(executionQuery.data?.status === undefined ||
|
||||||
|
executionQuery.data.status === 'pending' ||
|
||||||
|
executionQuery.data.status === 'running');
|
||||||
|
const sourceBusy =
|
||||||
|
source?.source_type !== 'manual' &&
|
||||||
|
(source?.sync_status === 'checking' || source?.sync_status === 'syncing');
|
||||||
|
const actionsDisabled =
|
||||||
|
checkMutation.isPending ||
|
||||||
|
syncMutation.isPending ||
|
||||||
|
executionBusy ||
|
||||||
|
sourceBusy;
|
||||||
|
const checkBusy =
|
||||||
|
checkMutation.isPending ||
|
||||||
|
(executionBusy && activeAction?.receipt.action === 'check') ||
|
||||||
|
(source?.source_type !== 'manual' && source?.sync_status === 'checking');
|
||||||
|
const syncBusy =
|
||||||
|
syncMutation.isPending ||
|
||||||
|
(executionBusy && activeAction?.receipt.action === 'sync') ||
|
||||||
|
(source?.source_type !== 'manual' && source?.sync_status === 'syncing');
|
||||||
|
const dispatchError = checkMutation.error ?? syncMutation.error;
|
||||||
|
|
||||||
|
const openSourceDialog = (mode: PagesSourceMode) => {
|
||||||
|
setDialogMode(mode);
|
||||||
|
setDialogOpen(true);
|
||||||
|
};
|
||||||
|
|
||||||
|
const dispatchSync = () => {
|
||||||
|
checkMutation.reset();
|
||||||
|
if (
|
||||||
|
source?.source_type === 'github_release' &&
|
||||||
|
source.sync_status === 'attention'
|
||||||
|
) {
|
||||||
|
setAttentionDialogOpen(true);
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
syncMutation.mutate({});
|
||||||
|
};
|
||||||
|
|
||||||
|
if (sourceQuery.isLoading) {
|
||||||
|
return (
|
||||||
|
<Card>
|
||||||
|
<CardHeader>
|
||||||
|
<CardTitle>部署源</CardTitle>
|
||||||
|
<CardDescription>加载来源配置...</CardDescription>
|
||||||
|
</CardHeader>
|
||||||
|
<CardContent className='flex flex-col gap-3'>
|
||||||
|
<Skeleton className='h-10 w-full' />
|
||||||
|
<Skeleton className='h-20 w-full' />
|
||||||
|
</CardContent>
|
||||||
|
</Card>
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
|
if (sourceQuery.isError || !source) {
|
||||||
|
return (
|
||||||
|
<Card>
|
||||||
|
<CardHeader>
|
||||||
|
<CardTitle>部署源</CardTitle>
|
||||||
|
<CardDescription>来源配置与部署历史相互独立。</CardDescription>
|
||||||
|
</CardHeader>
|
||||||
|
<CardContent>
|
||||||
|
<ErrorInline
|
||||||
|
message={
|
||||||
|
sourceQuery.error instanceof Error
|
||||||
|
? sourceQuery.error.message
|
||||||
|
: '部署源加载失败'
|
||||||
|
}
|
||||||
|
onRetry={() => void sourceQuery.refetch()}
|
||||||
|
/>
|
||||||
|
</CardContent>
|
||||||
|
</Card>
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
|
const effectiveSourceStatus = executionBusy
|
||||||
|
? activeAction?.receipt.action === 'check'
|
||||||
|
? 'checking'
|
||||||
|
: 'syncing'
|
||||||
|
: source.source_type === 'manual'
|
||||||
|
? undefined
|
||||||
|
: (source.sync_status ?? 'idle');
|
||||||
|
const status =
|
||||||
|
source.source_type === 'manual'
|
||||||
|
? null
|
||||||
|
: SOURCE_STATUS[effectiveSourceStatus ?? 'idle'];
|
||||||
|
const attentionRevision =
|
||||||
|
source.source_type === 'github_release' &&
|
||||||
|
source.sync_status === 'attention'
|
||||||
|
? source.last_seen
|
||||||
|
: undefined;
|
||||||
|
|
||||||
|
return (
|
||||||
|
<>
|
||||||
|
<Card>
|
||||||
|
<CardHeader>
|
||||||
|
<CardTitle>部署源</CardTitle>
|
||||||
|
<CardDescription>
|
||||||
|
来源配置负责发现内容,发布结果记录在独立的部署历史中。
|
||||||
|
</CardDescription>
|
||||||
|
<CardAction>
|
||||||
|
{status ? (
|
||||||
|
<Badge variant={status.variant}>{status.label}</Badge>
|
||||||
|
) : (
|
||||||
|
<Badge variant='outline'>手动部署</Badge>
|
||||||
|
)}
|
||||||
|
</CardAction>
|
||||||
|
</CardHeader>
|
||||||
|
|
||||||
|
<CardContent className='flex flex-col gap-4'>
|
||||||
|
{source.source_type === 'manual' ? (
|
||||||
|
<div className='rounded-lg border bg-muted/20 p-4'>
|
||||||
|
<p className='text-sm font-medium'>本地部署包</p>
|
||||||
|
<p className='mt-1 text-sm text-muted-foreground'>
|
||||||
|
当前没有持久化远端来源。上传部署包后,再从部署历史显式激活。
|
||||||
|
</p>
|
||||||
|
</div>
|
||||||
|
) : source.source_type === 'remote_url' ? (
|
||||||
|
<RemoteSourceDetails source={source} />
|
||||||
|
) : (
|
||||||
|
<GitHubSourceDetails source={source} />
|
||||||
|
)}
|
||||||
|
|
||||||
|
{dispatchError ? (
|
||||||
|
<ErrorInline
|
||||||
|
message={
|
||||||
|
dispatchError instanceof Error
|
||||||
|
? dispatchError.message
|
||||||
|
: '来源任务提交失败'
|
||||||
|
}
|
||||||
|
/>
|
||||||
|
) : null}
|
||||||
|
{executionQuery.isError ? (
|
||||||
|
<ErrorInline
|
||||||
|
message={
|
||||||
|
executionQuery.error instanceof Error
|
||||||
|
? executionQuery.error.message
|
||||||
|
: '任务状态读取失败'
|
||||||
|
}
|
||||||
|
onRetry={() => void executionQuery.refetch()}
|
||||||
|
/>
|
||||||
|
) : null}
|
||||||
|
{actionTimedOut ? (
|
||||||
|
<div className='flex flex-col gap-2 rounded-lg border p-3 sm:flex-row sm:items-center sm:justify-between'>
|
||||||
|
<span className='text-xs text-muted-foreground'>
|
||||||
|
自动等待已停止,任务可能仍在后台运行。
|
||||||
|
</span>
|
||||||
|
<Button
|
||||||
|
type='button'
|
||||||
|
variant='outline'
|
||||||
|
size='sm'
|
||||||
|
onClick={() => {
|
||||||
|
if (!activeAction) return;
|
||||||
|
setActiveAction({ ...activeAction, startedAt: Date.now() });
|
||||||
|
setActionTimedOut(false);
|
||||||
|
void executionQuery.refetch();
|
||||||
|
void sourceQuery.refetch();
|
||||||
|
}}
|
||||||
|
>
|
||||||
|
<RefreshCw data-icon='inline-start' />
|
||||||
|
刷新任务状态
|
||||||
|
</Button>
|
||||||
|
</div>
|
||||||
|
) : null}
|
||||||
|
</CardContent>
|
||||||
|
|
||||||
|
<CardFooter className='flex flex-wrap gap-2 border-t'>
|
||||||
|
{source.source_type === 'manual' ? (
|
||||||
|
<>
|
||||||
|
<Button
|
||||||
|
type='button'
|
||||||
|
variant='outline'
|
||||||
|
onClick={() => openSourceDialog('remote_url')}
|
||||||
|
>
|
||||||
|
<Download data-icon='inline-start' />
|
||||||
|
配置 Remote URL
|
||||||
|
</Button>
|
||||||
|
<Button
|
||||||
|
type='button'
|
||||||
|
onClick={() => openSourceDialog('github_release')}
|
||||||
|
>
|
||||||
|
<Github data-icon='inline-start' />
|
||||||
|
配置 GitHub Release
|
||||||
|
</Button>
|
||||||
|
</>
|
||||||
|
) : (
|
||||||
|
<>
|
||||||
|
<Button
|
||||||
|
type='button'
|
||||||
|
variant='outline'
|
||||||
|
disabled={actionsDisabled}
|
||||||
|
onClick={() => openSourceDialog(source.source_type)}
|
||||||
|
>
|
||||||
|
<Pencil data-icon='inline-start' />
|
||||||
|
编辑来源
|
||||||
|
</Button>
|
||||||
|
{source.source_type === 'github_release' ? (
|
||||||
|
<Button
|
||||||
|
type='button'
|
||||||
|
variant='outline'
|
||||||
|
disabled={actionsDisabled}
|
||||||
|
onClick={() => {
|
||||||
|
syncMutation.reset();
|
||||||
|
checkMutation.mutate();
|
||||||
|
}}
|
||||||
|
>
|
||||||
|
{checkBusy ? (
|
||||||
|
<Spinner data-icon='inline-start' />
|
||||||
|
) : (
|
||||||
|
<Search data-icon='inline-start' />
|
||||||
|
)}
|
||||||
|
检查更新
|
||||||
|
</Button>
|
||||||
|
) : null}
|
||||||
|
<Button
|
||||||
|
type='button'
|
||||||
|
disabled={
|
||||||
|
actionsDisabled ||
|
||||||
|
(source.sync_status === 'attention' && !attentionRevision)
|
||||||
|
}
|
||||||
|
onClick={dispatchSync}
|
||||||
|
>
|
||||||
|
{syncBusy ? (
|
||||||
|
<Spinner data-icon='inline-start' />
|
||||||
|
) : (
|
||||||
|
<Download data-icon='inline-start' />
|
||||||
|
)}
|
||||||
|
同步并发布
|
||||||
|
</Button>
|
||||||
|
<Button
|
||||||
|
type='button'
|
||||||
|
variant='ghost'
|
||||||
|
disabled={actionsDisabled}
|
||||||
|
onClick={() => openSourceDialog('manual')}
|
||||||
|
>
|
||||||
|
<RotateCcw data-icon='inline-start' />
|
||||||
|
切换回手动
|
||||||
|
</Button>
|
||||||
|
</>
|
||||||
|
)}
|
||||||
|
<Button
|
||||||
|
type='button'
|
||||||
|
variant='ghost'
|
||||||
|
size='sm'
|
||||||
|
className='ml-auto'
|
||||||
|
onClick={() => void sourceQuery.refetch()}
|
||||||
|
>
|
||||||
|
<RefreshCw data-icon='inline-start' />
|
||||||
|
刷新
|
||||||
|
</Button>
|
||||||
|
</CardFooter>
|
||||||
|
</Card>
|
||||||
|
|
||||||
|
<PagesSourceDialog
|
||||||
|
open={dialogOpen}
|
||||||
|
onOpenChange={setDialogOpen}
|
||||||
|
projectId={projectId}
|
||||||
|
source={source}
|
||||||
|
initialMode={dialogMode}
|
||||||
|
onActionDispatched={beginActionPolling}
|
||||||
|
/>
|
||||||
|
|
||||||
|
<AlertDialog
|
||||||
|
open={attentionDialogOpen}
|
||||||
|
onOpenChange={(open) => {
|
||||||
|
if (!syncMutation.isPending) setAttentionDialogOpen(open);
|
||||||
|
}}
|
||||||
|
>
|
||||||
|
<AlertDialogContent>
|
||||||
|
<AlertDialogHeader>
|
||||||
|
<AlertDialogTitle>确认发布当前 GitHub revision</AlertDialogTitle>
|
||||||
|
<AlertDialogDescription>
|
||||||
|
<span>这将发布卡片当前显示的精确 revision:</span>{' '}
|
||||||
|
<span className='break-all font-mono'>
|
||||||
|
{attentionRevision?.revision ??
|
||||||
|
'当前 revision 已不可用,请刷新后重试'}
|
||||||
|
</span>
|
||||||
|
</AlertDialogDescription>
|
||||||
|
</AlertDialogHeader>
|
||||||
|
<AlertDialogFooter>
|
||||||
|
<AlertDialogCancel disabled={syncMutation.isPending}>
|
||||||
|
取消
|
||||||
|
</AlertDialogCancel>
|
||||||
|
<AlertDialogAction
|
||||||
|
disabled={syncMutation.isPending || !attentionRevision}
|
||||||
|
onClick={(event) => {
|
||||||
|
event.preventDefault();
|
||||||
|
if (!attentionRevision) return;
|
||||||
|
syncMutation.mutate({
|
||||||
|
confirmed_revision: attentionRevision.revision,
|
||||||
|
});
|
||||||
|
}}
|
||||||
|
>
|
||||||
|
{syncMutation.isPending ? (
|
||||||
|
<Spinner data-icon='inline-start' />
|
||||||
|
) : null}
|
||||||
|
确认并发布
|
||||||
|
</AlertDialogAction>
|
||||||
|
</AlertDialogFooter>
|
||||||
|
</AlertDialogContent>
|
||||||
|
</AlertDialog>
|
||||||
|
</>
|
||||||
|
);
|
||||||
|
}
|
||||||
@@ -0,0 +1,163 @@
|
|||||||
|
import { TriangleAlert } from 'lucide-react';
|
||||||
|
|
||||||
|
import { ErrorInline } from '@/components/layout/error';
|
||||||
|
import { Alert, AlertDescription, AlertTitle } from '@/components/ui/alert';
|
||||||
|
import {
|
||||||
|
type PagesGitHubReleaseSource,
|
||||||
|
type PagesRemoteURLSource,
|
||||||
|
type PagesSourceRevision,
|
||||||
|
} from '@/lib/services/openflare';
|
||||||
|
import { formatDateTime } from '@/lib/utils';
|
||||||
|
|
||||||
|
function revisionSummary(revision?: PagesSourceRevision) {
|
||||||
|
if (!revision) return '尚无记录';
|
||||||
|
return `${revision.label} · ${revision.revision.slice(0, 12)}`;
|
||||||
|
}
|
||||||
|
|
||||||
|
export function RemoteSourceDetails({
|
||||||
|
source,
|
||||||
|
}: {
|
||||||
|
source: PagesRemoteURLSource;
|
||||||
|
}) {
|
||||||
|
return (
|
||||||
|
<div className='grid gap-4 md:grid-cols-2'>
|
||||||
|
<div className='flex min-w-0 flex-col gap-1 rounded-lg border p-4 md:col-span-2'>
|
||||||
|
<span className='text-xs text-muted-foreground'>脱敏地址</span>
|
||||||
|
<code className='truncate text-sm'>{source.display_url}</code>
|
||||||
|
</div>
|
||||||
|
<div className='flex flex-col gap-1 rounded-lg border p-4'>
|
||||||
|
<span className='text-xs text-muted-foreground'>网络策略</span>
|
||||||
|
<span className='text-sm font-medium'>
|
||||||
|
{source.remote_network_policy === 'trusted_internal'
|
||||||
|
? '受信内网模式'
|
||||||
|
: '公网安全模式'}
|
||||||
|
</span>
|
||||||
|
</div>
|
||||||
|
<div className='flex flex-col gap-1 rounded-lg border p-4'>
|
||||||
|
<span className='text-xs text-muted-foreground'>最近同步</span>
|
||||||
|
<span className='text-sm font-medium'>
|
||||||
|
{source.last_synced_at
|
||||||
|
? formatDateTime(source.last_synced_at)
|
||||||
|
: '尚未同步'}
|
||||||
|
</span>
|
||||||
|
</div>
|
||||||
|
<div className='flex flex-col gap-1 rounded-lg border p-4 md:col-span-2'>
|
||||||
|
<span className='text-xs text-muted-foreground'>已应用 revision</span>
|
||||||
|
<span className='font-mono text-sm'>
|
||||||
|
{revisionSummary(source.last_applied)}
|
||||||
|
</span>
|
||||||
|
</div>
|
||||||
|
{source.last_error ? (
|
||||||
|
<div className='md:col-span-2'>
|
||||||
|
<ErrorInline message={source.last_error} />
|
||||||
|
</div>
|
||||||
|
) : null}
|
||||||
|
</div>
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
|
export function GitHubSourceDetails({
|
||||||
|
source,
|
||||||
|
}: {
|
||||||
|
source: PagesGitHubReleaseSource;
|
||||||
|
}) {
|
||||||
|
const attentionRevision =
|
||||||
|
source.sync_status === 'attention' ? source.last_seen : undefined;
|
||||||
|
|
||||||
|
return (
|
||||||
|
<div className='flex flex-col gap-4'>
|
||||||
|
{attentionRevision ? (
|
||||||
|
<Alert variant='destructive'>
|
||||||
|
<TriangleAlert />
|
||||||
|
<AlertTitle>Release Asset 发生变化,需要显式确认</AlertTitle>
|
||||||
|
<AlertDescription>
|
||||||
|
<p>
|
||||||
|
当前远端 revision
|
||||||
|
与已发布内容不一致。请核对版本和资源后,再确认发布这一精确
|
||||||
|
revision。
|
||||||
|
</p>
|
||||||
|
<code className='break-all'>{attentionRevision.revision}</code>
|
||||||
|
</AlertDescription>
|
||||||
|
</Alert>
|
||||||
|
) : null}
|
||||||
|
|
||||||
|
<div className='grid gap-4 md:grid-cols-2'>
|
||||||
|
<div className='flex min-w-0 flex-col gap-1 rounded-lg border p-4 md:col-span-2'>
|
||||||
|
<span className='text-xs text-muted-foreground'>GitHub 仓库</span>
|
||||||
|
<code className='truncate text-sm'>{source.github_repository}</code>
|
||||||
|
</div>
|
||||||
|
<div className='flex flex-col gap-1 rounded-lg border p-4'>
|
||||||
|
<span className='text-xs text-muted-foreground'>Release 选择</span>
|
||||||
|
<span className='text-sm font-medium'>
|
||||||
|
{source.release_selector === 'latest'
|
||||||
|
? '最新 Release'
|
||||||
|
: `固定 Tag · ${source.release_tag ?? '未提供'}`}
|
||||||
|
</span>
|
||||||
|
</div>
|
||||||
|
<div className='flex min-w-0 flex-col gap-1 rounded-lg border p-4'>
|
||||||
|
<span className='text-xs text-muted-foreground'>Release Asset</span>
|
||||||
|
<code className='truncate text-sm'>{source.asset_name}</code>
|
||||||
|
</div>
|
||||||
|
{source.release_selector === 'latest' ? (
|
||||||
|
<>
|
||||||
|
<div className='flex flex-col gap-1 rounded-lg border p-4'>
|
||||||
|
<span className='text-xs text-muted-foreground'>自动更新</span>
|
||||||
|
<span className='text-sm font-medium'>
|
||||||
|
{source.auto_update_enabled ? '已开启' : '已关闭'}
|
||||||
|
</span>
|
||||||
|
</div>
|
||||||
|
<div className='flex flex-col gap-1 rounded-lg border p-4'>
|
||||||
|
<span className='text-xs text-muted-foreground'>检查间隔</span>
|
||||||
|
<span className='text-sm font-medium'>
|
||||||
|
{source.check_interval_minutes} 分钟
|
||||||
|
</span>
|
||||||
|
</div>
|
||||||
|
<div className='flex flex-col gap-1 rounded-lg border p-4 md:col-span-2'>
|
||||||
|
<span className='text-xs text-muted-foreground'>
|
||||||
|
下次检查时间
|
||||||
|
</span>
|
||||||
|
<span className='text-sm font-medium'>
|
||||||
|
{source.next_check_at
|
||||||
|
? formatDateTime(source.next_check_at)
|
||||||
|
: '等待调度'}
|
||||||
|
</span>
|
||||||
|
</div>
|
||||||
|
</>
|
||||||
|
) : null}
|
||||||
|
<div className='flex flex-col gap-1 rounded-lg border p-4'>
|
||||||
|
<span className='text-xs text-muted-foreground'>远端已发现</span>
|
||||||
|
<span className='font-mono text-sm'>
|
||||||
|
{revisionSummary(source.last_seen)}
|
||||||
|
</span>
|
||||||
|
</div>
|
||||||
|
<div className='flex flex-col gap-1 rounded-lg border p-4'>
|
||||||
|
<span className='text-xs text-muted-foreground'>当前已应用</span>
|
||||||
|
<span className='font-mono text-sm'>
|
||||||
|
{revisionSummary(source.last_applied)}
|
||||||
|
</span>
|
||||||
|
</div>
|
||||||
|
<div className='flex flex-col gap-1 rounded-lg border p-4'>
|
||||||
|
<span className='text-xs text-muted-foreground'>最近检查</span>
|
||||||
|
<span className='text-sm font-medium'>
|
||||||
|
{source.last_checked_at
|
||||||
|
? formatDateTime(source.last_checked_at)
|
||||||
|
: '尚未检查'}
|
||||||
|
</span>
|
||||||
|
</div>
|
||||||
|
<div className='flex flex-col gap-1 rounded-lg border p-4'>
|
||||||
|
<span className='text-xs text-muted-foreground'>最近同步</span>
|
||||||
|
<span className='text-sm font-medium'>
|
||||||
|
{source.last_synced_at
|
||||||
|
? formatDateTime(source.last_synced_at)
|
||||||
|
: '尚未同步'}
|
||||||
|
</span>
|
||||||
|
</div>
|
||||||
|
{source.last_error ? (
|
||||||
|
<div className='md:col-span-2'>
|
||||||
|
<ErrorInline message={source.last_error} />
|
||||||
|
</div>
|
||||||
|
) : null}
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
|
);
|
||||||
|
}
|
||||||
@@ -0,0 +1,591 @@
|
|||||||
|
'use client';
|
||||||
|
|
||||||
|
import { useEffect, useRef, useState } from 'react';
|
||||||
|
import { useMutation, useQueryClient } from '@tanstack/react-query';
|
||||||
|
import { Eye, EyeOff } from 'lucide-react';
|
||||||
|
import { toast } from 'sonner';
|
||||||
|
|
||||||
|
import {
|
||||||
|
AlertDialog,
|
||||||
|
AlertDialogAction,
|
||||||
|
AlertDialogCancel,
|
||||||
|
AlertDialogContent,
|
||||||
|
AlertDialogDescription,
|
||||||
|
AlertDialogFooter,
|
||||||
|
AlertDialogHeader,
|
||||||
|
AlertDialogTitle,
|
||||||
|
} from '@/components/ui/alert-dialog';
|
||||||
|
import { Button } from '@/components/ui/button';
|
||||||
|
import {
|
||||||
|
Dialog,
|
||||||
|
DialogContent,
|
||||||
|
DialogDescription,
|
||||||
|
DialogFooter,
|
||||||
|
DialogHeader,
|
||||||
|
DialogTitle,
|
||||||
|
} from '@/components/ui/dialog';
|
||||||
|
import {
|
||||||
|
Field,
|
||||||
|
FieldDescription,
|
||||||
|
FieldGroup,
|
||||||
|
FieldLabel,
|
||||||
|
FieldTitle,
|
||||||
|
} from '@/components/ui/field';
|
||||||
|
import { Input } from '@/components/ui/input';
|
||||||
|
import { Spinner } from '@/components/ui/spinner';
|
||||||
|
import { ToggleGroup, ToggleGroupItem } from '@/components/ui/toggle-group';
|
||||||
|
import {
|
||||||
|
type PagesRemoteNetworkPolicy,
|
||||||
|
type PagesSource,
|
||||||
|
type PagesSourceActionReceipt,
|
||||||
|
type PagesSourceUpdatePayload,
|
||||||
|
PagesService,
|
||||||
|
} from '@/lib/services/openflare';
|
||||||
|
|
||||||
|
import {
|
||||||
|
deploymentsQueryKey,
|
||||||
|
projectQueryKey,
|
||||||
|
projectsQueryKey,
|
||||||
|
sourceQueryKey,
|
||||||
|
} from '../../components/pages-utils';
|
||||||
|
import {
|
||||||
|
type PagesGitHubSourceFormErrors,
|
||||||
|
type PagesGitHubSourceFormValue,
|
||||||
|
PagesSourceGitHubFields,
|
||||||
|
} from './pages-source-github-fields';
|
||||||
|
import {
|
||||||
|
validGitHubAssetName,
|
||||||
|
validGitHubReleaseTag,
|
||||||
|
validGitHubRepositoryURL,
|
||||||
|
} from './pages-source-validation';
|
||||||
|
|
||||||
|
export type PagesSourceMode = 'manual' | 'remote_url' | 'github_release';
|
||||||
|
|
||||||
|
type Confirmation = 'trusted_internal' | 'manual' | null;
|
||||||
|
|
||||||
|
interface PagesSourceDialogProps {
|
||||||
|
open: boolean;
|
||||||
|
onOpenChange: (open: boolean) => void;
|
||||||
|
projectId: number;
|
||||||
|
source: PagesSource;
|
||||||
|
initialMode?: PagesSourceMode;
|
||||||
|
onActionDispatched?: (receipt: PagesSourceActionReceipt) => void;
|
||||||
|
}
|
||||||
|
|
||||||
|
const DEFAULT_GITHUB_ASSET = 'dist.zip';
|
||||||
|
const DEFAULT_GITHUB_CHECK_INTERVAL = 60;
|
||||||
|
const EMPTY_GITHUB_ERRORS: PagesGitHubSourceFormErrors = {
|
||||||
|
repository: '',
|
||||||
|
releaseTag: '',
|
||||||
|
assetName: '',
|
||||||
|
checkInterval: '',
|
||||||
|
};
|
||||||
|
|
||||||
|
function githubRepositoryURL(repository: string) {
|
||||||
|
const value = repository.trim();
|
||||||
|
return value ? `https://github.com/${value}` : '';
|
||||||
|
}
|
||||||
|
|
||||||
|
export function PagesSourceDialog({
|
||||||
|
open,
|
||||||
|
onOpenChange,
|
||||||
|
projectId,
|
||||||
|
source,
|
||||||
|
initialMode,
|
||||||
|
onActionDispatched,
|
||||||
|
}: PagesSourceDialogProps) {
|
||||||
|
const queryClient = useQueryClient();
|
||||||
|
const [mode, setMode] = useState<PagesSourceMode>('manual');
|
||||||
|
const [networkPolicy, setNetworkPolicy] =
|
||||||
|
useState<PagesRemoteNetworkPolicy>('public');
|
||||||
|
const [replaceURL, setReplaceURL] = useState(false);
|
||||||
|
const [remoteURL, setRemoteURL] = useState('');
|
||||||
|
const [revealRemoteURL, setRevealRemoteURL] = useState(false);
|
||||||
|
const [urlError, setURLError] = useState('');
|
||||||
|
const [githubForm, setGitHubForm] = useState<PagesGitHubSourceFormValue>({
|
||||||
|
repositoryURL: '',
|
||||||
|
releaseSelector: 'latest',
|
||||||
|
releaseTag: '',
|
||||||
|
assetName: DEFAULT_GITHUB_ASSET,
|
||||||
|
autoUpdateEnabled: false,
|
||||||
|
checkIntervalMinutes: String(DEFAULT_GITHUB_CHECK_INTERVAL),
|
||||||
|
});
|
||||||
|
const [githubErrors, setGitHubErrors] =
|
||||||
|
useState<PagesGitHubSourceFormErrors>(EMPTY_GITHUB_ERRORS);
|
||||||
|
const [confirmation, setConfirmation] = useState<Confirmation>(null);
|
||||||
|
const initializedForOpen = useRef(false);
|
||||||
|
|
||||||
|
useEffect(() => {
|
||||||
|
if (!open) {
|
||||||
|
initializedForOpen.current = false;
|
||||||
|
setRemoteURL('');
|
||||||
|
setRevealRemoteURL(false);
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
// Runtime polling may replace the source view while the dialog is open.
|
||||||
|
// Initialize only on the open edge so it cannot overwrite an unsaved draft.
|
||||||
|
if (initializedForOpen.current) return;
|
||||||
|
initializedForOpen.current = true;
|
||||||
|
const nextMode = initialMode ?? source.source_type;
|
||||||
|
setMode(nextMode);
|
||||||
|
setNetworkPolicy(
|
||||||
|
source.source_type === 'remote_url'
|
||||||
|
? source.remote_network_policy
|
||||||
|
: 'public',
|
||||||
|
);
|
||||||
|
setReplaceURL(source.source_type !== 'remote_url');
|
||||||
|
setRemoteURL('');
|
||||||
|
setRevealRemoteURL(false);
|
||||||
|
setURLError('');
|
||||||
|
setGitHubForm({
|
||||||
|
repositoryURL:
|
||||||
|
source.source_type === 'github_release'
|
||||||
|
? githubRepositoryURL(source.github_repository)
|
||||||
|
: '',
|
||||||
|
releaseSelector:
|
||||||
|
source.source_type === 'github_release'
|
||||||
|
? source.release_selector
|
||||||
|
: 'latest',
|
||||||
|
releaseTag:
|
||||||
|
source.source_type === 'github_release'
|
||||||
|
? (source.release_tag ?? '')
|
||||||
|
: '',
|
||||||
|
assetName:
|
||||||
|
source.source_type === 'github_release'
|
||||||
|
? source.asset_name
|
||||||
|
: DEFAULT_GITHUB_ASSET,
|
||||||
|
autoUpdateEnabled:
|
||||||
|
source.source_type === 'github_release' &&
|
||||||
|
source.release_selector === 'latest'
|
||||||
|
? source.auto_update_enabled
|
||||||
|
: false,
|
||||||
|
checkIntervalMinutes:
|
||||||
|
source.source_type === 'github_release' &&
|
||||||
|
source.release_selector === 'latest'
|
||||||
|
? String(
|
||||||
|
source.check_interval_minutes || DEFAULT_GITHUB_CHECK_INTERVAL,
|
||||||
|
)
|
||||||
|
: String(DEFAULT_GITHUB_CHECK_INTERVAL),
|
||||||
|
});
|
||||||
|
setGitHubErrors(EMPTY_GITHUB_ERRORS);
|
||||||
|
setConfirmation(null);
|
||||||
|
}, [initialMode, open, source]);
|
||||||
|
|
||||||
|
const invalidateSourceState = async () => {
|
||||||
|
await Promise.all([
|
||||||
|
queryClient.invalidateQueries({ queryKey: sourceQueryKey(projectId) }),
|
||||||
|
queryClient.invalidateQueries({ queryKey: projectQueryKey(projectId) }),
|
||||||
|
queryClient.invalidateQueries({
|
||||||
|
queryKey: deploymentsQueryKey(projectId),
|
||||||
|
}),
|
||||||
|
queryClient.invalidateQueries({ queryKey: projectsQueryKey }),
|
||||||
|
]);
|
||||||
|
};
|
||||||
|
|
||||||
|
const updateMutation = useMutation({
|
||||||
|
mutationFn: (payload: PagesSourceUpdatePayload) =>
|
||||||
|
PagesService.updateSource(projectId, payload),
|
||||||
|
onSuccess: async (result) => {
|
||||||
|
queryClient.setQueryData(sourceQueryKey(projectId), result.source);
|
||||||
|
if (result.check_task) onActionDispatched?.(result.check_task);
|
||||||
|
await invalidateSourceState();
|
||||||
|
toast.success('部署源已更新');
|
||||||
|
if (result.warning) toast.warning(result.warning);
|
||||||
|
setConfirmation(null);
|
||||||
|
setRemoteURL('');
|
||||||
|
setRevealRemoteURL(false);
|
||||||
|
onOpenChange(false);
|
||||||
|
},
|
||||||
|
onError: (error) => {
|
||||||
|
toast.error(error instanceof Error ? error.message : '部署源更新失败');
|
||||||
|
},
|
||||||
|
});
|
||||||
|
|
||||||
|
const deleteMutation = useMutation({
|
||||||
|
mutationFn: () => PagesService.deleteSource(projectId),
|
||||||
|
onSuccess: async (manualSource) => {
|
||||||
|
queryClient.setQueryData(sourceQueryKey(projectId), manualSource);
|
||||||
|
await invalidateSourceState();
|
||||||
|
toast.success('已切换回手动部署');
|
||||||
|
setConfirmation(null);
|
||||||
|
setRemoteURL('');
|
||||||
|
setRevealRemoteURL(false);
|
||||||
|
onOpenChange(false);
|
||||||
|
},
|
||||||
|
onError: (error) => {
|
||||||
|
toast.error(error instanceof Error ? error.message : '切换失败');
|
||||||
|
},
|
||||||
|
});
|
||||||
|
|
||||||
|
const isPending = updateMutation.isPending || deleteMutation.isPending;
|
||||||
|
|
||||||
|
const remotePayload = (): PagesSourceUpdatePayload => ({
|
||||||
|
source_type: 'remote_url',
|
||||||
|
remote_url_set: replaceURL,
|
||||||
|
remote_url: replaceURL ? remoteURL.trim() : '',
|
||||||
|
remote_network_policy: networkPolicy,
|
||||||
|
});
|
||||||
|
|
||||||
|
const submitRemote = () => {
|
||||||
|
if (replaceURL) {
|
||||||
|
const value = remoteURL.trim();
|
||||||
|
if (!value) {
|
||||||
|
setURLError('请输入 Remote URL');
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
try {
|
||||||
|
const parsed = new URL(value);
|
||||||
|
if (!['http:', 'https:'].includes(parsed.protocol)) throw new Error();
|
||||||
|
} catch {
|
||||||
|
setURLError('请输入有效的 HTTP(S) URL');
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
setURLError('');
|
||||||
|
if (networkPolicy === 'trusted_internal') {
|
||||||
|
setConfirmation('trusted_internal');
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
updateMutation.mutate(remotePayload());
|
||||||
|
};
|
||||||
|
|
||||||
|
const submitGitHub = () => {
|
||||||
|
const normalizedRepositoryURL = githubForm.repositoryURL.trim();
|
||||||
|
const nextRepositoryError = validGitHubRepositoryURL(
|
||||||
|
normalizedRepositoryURL,
|
||||||
|
)
|
||||||
|
? ''
|
||||||
|
: '请输入 https://github.com/{owner}/{repo} 格式的公开仓库地址';
|
||||||
|
const nextReleaseTagError =
|
||||||
|
githubForm.releaseSelector === 'tag' &&
|
||||||
|
!validGitHubReleaseTag(githubForm.releaseTag)
|
||||||
|
? 'Release tag 须为有效 Git ref(1–255 字节,可使用 /、#、&、=)'
|
||||||
|
: '';
|
||||||
|
const nextAssetNameError = validGitHubAssetName(githubForm.assetName)
|
||||||
|
? ''
|
||||||
|
: 'Asset 文件名须为 1–255 字节,且不能是路径或包含控制、换行、双向文本字符';
|
||||||
|
const checkIntervalMinutes = Number(githubForm.checkIntervalMinutes);
|
||||||
|
const nextCheckIntervalError =
|
||||||
|
githubForm.releaseSelector === 'latest' &&
|
||||||
|
(!Number.isInteger(checkIntervalMinutes) ||
|
||||||
|
checkIntervalMinutes < 5 ||
|
||||||
|
checkIntervalMinutes > 1440)
|
||||||
|
? '检查间隔须为 5–1440 分钟的整数'
|
||||||
|
: '';
|
||||||
|
|
||||||
|
setGitHubErrors({
|
||||||
|
repository: nextRepositoryError,
|
||||||
|
releaseTag: nextReleaseTagError,
|
||||||
|
assetName: nextAssetNameError,
|
||||||
|
checkInterval: nextCheckIntervalError,
|
||||||
|
});
|
||||||
|
if (
|
||||||
|
nextRepositoryError ||
|
||||||
|
nextReleaseTagError ||
|
||||||
|
nextAssetNameError ||
|
||||||
|
nextCheckIntervalError
|
||||||
|
) {
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
|
||||||
|
const payload: PagesSourceUpdatePayload =
|
||||||
|
githubForm.releaseSelector === 'latest'
|
||||||
|
? {
|
||||||
|
source_type: 'github_release',
|
||||||
|
repository_url: normalizedRepositoryURL,
|
||||||
|
release_selector: 'latest',
|
||||||
|
release_tag: '',
|
||||||
|
asset_name: githubForm.assetName,
|
||||||
|
auto_update_enabled: githubForm.autoUpdateEnabled,
|
||||||
|
check_interval_minutes: checkIntervalMinutes,
|
||||||
|
}
|
||||||
|
: {
|
||||||
|
source_type: 'github_release',
|
||||||
|
repository_url: normalizedRepositoryURL,
|
||||||
|
release_selector: 'tag',
|
||||||
|
release_tag: githubForm.releaseTag,
|
||||||
|
asset_name: githubForm.assetName,
|
||||||
|
auto_update_enabled: false,
|
||||||
|
check_interval_minutes: 0,
|
||||||
|
};
|
||||||
|
updateMutation.mutate(payload);
|
||||||
|
};
|
||||||
|
|
||||||
|
const handleSubmit = () => {
|
||||||
|
switch (mode) {
|
||||||
|
case 'manual':
|
||||||
|
if (source.source_type === 'manual') {
|
||||||
|
onOpenChange(false);
|
||||||
|
} else {
|
||||||
|
setConfirmation('manual');
|
||||||
|
}
|
||||||
|
return;
|
||||||
|
case 'remote_url':
|
||||||
|
submitRemote();
|
||||||
|
return;
|
||||||
|
case 'github_release':
|
||||||
|
submitGitHub();
|
||||||
|
}
|
||||||
|
};
|
||||||
|
|
||||||
|
const submitLabel =
|
||||||
|
mode === 'manual'
|
||||||
|
? '使用手动部署'
|
||||||
|
: mode === 'remote_url'
|
||||||
|
? '保存 Remote 来源'
|
||||||
|
: '保存 GitHub 来源';
|
||||||
|
|
||||||
|
return (
|
||||||
|
<>
|
||||||
|
<Dialog
|
||||||
|
open={open}
|
||||||
|
onOpenChange={(nextOpen) => {
|
||||||
|
if (!isPending) onOpenChange(nextOpen);
|
||||||
|
}}
|
||||||
|
>
|
||||||
|
<DialogContent className='sm:max-w-xl'>
|
||||||
|
<DialogHeader>
|
||||||
|
<DialogTitle>部署源设置</DialogTitle>
|
||||||
|
<DialogDescription>
|
||||||
|
手动部署、Remote URL 与 GitHub Release 使用独立配置。
|
||||||
|
</DialogDescription>
|
||||||
|
</DialogHeader>
|
||||||
|
|
||||||
|
<FieldGroup>
|
||||||
|
<Field>
|
||||||
|
<FieldTitle id='pages-source-mode'>来源类型</FieldTitle>
|
||||||
|
<ToggleGroup
|
||||||
|
type='single'
|
||||||
|
variant='outline'
|
||||||
|
value={mode}
|
||||||
|
aria-labelledby='pages-source-mode'
|
||||||
|
className='grid w-full grid-cols-1 sm:grid-cols-3'
|
||||||
|
onValueChange={(value) => {
|
||||||
|
if (
|
||||||
|
value === 'manual' ||
|
||||||
|
value === 'remote_url' ||
|
||||||
|
value === 'github_release'
|
||||||
|
) {
|
||||||
|
setMode(value);
|
||||||
|
if (value !== 'remote_url') {
|
||||||
|
setRemoteURL('');
|
||||||
|
setRevealRemoteURL(false);
|
||||||
|
setURLError('');
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}}
|
||||||
|
>
|
||||||
|
<ToggleGroupItem value='manual' className='w-full'>
|
||||||
|
手动部署
|
||||||
|
</ToggleGroupItem>
|
||||||
|
<ToggleGroupItem value='remote_url' className='w-full'>
|
||||||
|
Remote URL
|
||||||
|
</ToggleGroupItem>
|
||||||
|
<ToggleGroupItem value='github_release' className='w-full'>
|
||||||
|
GitHub Release
|
||||||
|
</ToggleGroupItem>
|
||||||
|
</ToggleGroup>
|
||||||
|
<FieldDescription>
|
||||||
|
<span>
|
||||||
|
远端来源只负责发现内容,发布结果始终保留为不可变部署。
|
||||||
|
</span>
|
||||||
|
<span className='block'>
|
||||||
|
仓库源码构建将在后续作为独立来源类型提供。
|
||||||
|
</span>
|
||||||
|
</FieldDescription>
|
||||||
|
</Field>
|
||||||
|
|
||||||
|
{mode === 'manual' ? (
|
||||||
|
<Field>
|
||||||
|
<FieldLabel>手动部署</FieldLabel>
|
||||||
|
<div className='rounded-lg border bg-muted/20 p-4 text-sm text-muted-foreground'>
|
||||||
|
保留现有部署与当前生产版本,后续通过“上传部署包”创建新部署。
|
||||||
|
</div>
|
||||||
|
</Field>
|
||||||
|
) : mode === 'remote_url' ? (
|
||||||
|
<>
|
||||||
|
<Field data-invalid={Boolean(urlError)}>
|
||||||
|
<FieldLabel htmlFor='pages-remote-url'>Remote URL</FieldLabel>
|
||||||
|
{source.source_type === 'remote_url' && !replaceURL ? (
|
||||||
|
<div className='flex flex-col gap-2 rounded-lg border p-3 sm:flex-row sm:items-center sm:justify-between'>
|
||||||
|
<code className='min-w-0 truncate text-xs'>
|
||||||
|
{source.display_url}
|
||||||
|
</code>
|
||||||
|
<Button
|
||||||
|
type='button'
|
||||||
|
variant='outline'
|
||||||
|
size='sm'
|
||||||
|
onClick={() => {
|
||||||
|
setReplaceURL(true);
|
||||||
|
setRemoteURL('');
|
||||||
|
setRevealRemoteURL(false);
|
||||||
|
}}
|
||||||
|
>
|
||||||
|
更换地址
|
||||||
|
</Button>
|
||||||
|
</div>
|
||||||
|
) : (
|
||||||
|
<div className='flex gap-2'>
|
||||||
|
<Input
|
||||||
|
id='pages-remote-url'
|
||||||
|
type={revealRemoteURL ? 'url' : 'password'}
|
||||||
|
placeholder='https://artifacts.example.com/site.zip?token=...'
|
||||||
|
value={remoteURL}
|
||||||
|
aria-invalid={Boolean(urlError)}
|
||||||
|
autoComplete='off'
|
||||||
|
className='min-w-0 flex-1'
|
||||||
|
onChange={(event) => {
|
||||||
|
setRemoteURL(event.target.value);
|
||||||
|
setURLError('');
|
||||||
|
}}
|
||||||
|
/>
|
||||||
|
<Button
|
||||||
|
type='button'
|
||||||
|
variant='outline'
|
||||||
|
size='icon'
|
||||||
|
aria-label={
|
||||||
|
revealRemoteURL
|
||||||
|
? '隐藏 Remote URL'
|
||||||
|
: '显示 Remote URL'
|
||||||
|
}
|
||||||
|
title={
|
||||||
|
revealRemoteURL
|
||||||
|
? '隐藏 Remote URL'
|
||||||
|
: '显示 Remote URL'
|
||||||
|
}
|
||||||
|
onClick={() =>
|
||||||
|
setRevealRemoteURL((visible) => !visible)
|
||||||
|
}
|
||||||
|
>
|
||||||
|
{revealRemoteURL ? <EyeOff /> : <Eye />}
|
||||||
|
</Button>
|
||||||
|
</div>
|
||||||
|
)}
|
||||||
|
<FieldDescription>
|
||||||
|
{urlError ||
|
||||||
|
(replaceURL
|
||||||
|
? '保存后不会回显原始地址或 query token。'
|
||||||
|
: '界面只显示脱敏地址;留空表示保留当前地址。')}
|
||||||
|
</FieldDescription>
|
||||||
|
{source.source_type === 'remote_url' && replaceURL ? (
|
||||||
|
<Button
|
||||||
|
type='button'
|
||||||
|
variant='ghost'
|
||||||
|
size='sm'
|
||||||
|
className='self-start'
|
||||||
|
onClick={() => {
|
||||||
|
setReplaceURL(false);
|
||||||
|
setRemoteURL('');
|
||||||
|
setRevealRemoteURL(false);
|
||||||
|
setURLError('');
|
||||||
|
}}
|
||||||
|
>
|
||||||
|
保留当前地址
|
||||||
|
</Button>
|
||||||
|
) : null}
|
||||||
|
</Field>
|
||||||
|
|
||||||
|
<Field>
|
||||||
|
<FieldTitle id='pages-network-policy'>网络策略</FieldTitle>
|
||||||
|
<ToggleGroup
|
||||||
|
type='single'
|
||||||
|
variant='outline'
|
||||||
|
value={networkPolicy}
|
||||||
|
aria-labelledby='pages-network-policy'
|
||||||
|
className='grid w-full grid-cols-2'
|
||||||
|
onValueChange={(value) => {
|
||||||
|
if (value === 'public' || value === 'trusted_internal') {
|
||||||
|
setNetworkPolicy(value);
|
||||||
|
}
|
||||||
|
}}
|
||||||
|
>
|
||||||
|
<ToggleGroupItem value='public' className='w-full'>
|
||||||
|
公网安全模式
|
||||||
|
</ToggleGroupItem>
|
||||||
|
<ToggleGroupItem
|
||||||
|
value='trusted_internal'
|
||||||
|
className='w-full'
|
||||||
|
>
|
||||||
|
受信内网模式
|
||||||
|
</ToggleGroupItem>
|
||||||
|
</ToggleGroup>
|
||||||
|
<FieldDescription>
|
||||||
|
{networkPolicy === 'public'
|
||||||
|
? '阻止内网地址、代理与不安全 TLS。'
|
||||||
|
: '允许访问内网地址与自签名证书,仅用于可信来源。'}
|
||||||
|
</FieldDescription>
|
||||||
|
</Field>
|
||||||
|
</>
|
||||||
|
) : (
|
||||||
|
<PagesSourceGitHubFields
|
||||||
|
value={githubForm}
|
||||||
|
errors={githubErrors}
|
||||||
|
defaultAssetName={DEFAULT_GITHUB_ASSET}
|
||||||
|
onChange={setGitHubForm}
|
||||||
|
onErrorsChange={setGitHubErrors}
|
||||||
|
/>
|
||||||
|
)}
|
||||||
|
</FieldGroup>
|
||||||
|
|
||||||
|
<DialogFooter>
|
||||||
|
<Button
|
||||||
|
type='button'
|
||||||
|
variant='outline'
|
||||||
|
disabled={isPending}
|
||||||
|
onClick={() => {
|
||||||
|
setRemoteURL('');
|
||||||
|
setRevealRemoteURL(false);
|
||||||
|
onOpenChange(false);
|
||||||
|
}}
|
||||||
|
>
|
||||||
|
取消
|
||||||
|
</Button>
|
||||||
|
<Button type='button' disabled={isPending} onClick={handleSubmit}>
|
||||||
|
{isPending ? <Spinner data-icon='inline-start' /> : null}
|
||||||
|
{submitLabel}
|
||||||
|
</Button>
|
||||||
|
</DialogFooter>
|
||||||
|
</DialogContent>
|
||||||
|
</Dialog>
|
||||||
|
|
||||||
|
<AlertDialog
|
||||||
|
open={confirmation !== null}
|
||||||
|
onOpenChange={(nextOpen) => {
|
||||||
|
if (!nextOpen && !isPending) setConfirmation(null);
|
||||||
|
}}
|
||||||
|
>
|
||||||
|
<AlertDialogContent>
|
||||||
|
<AlertDialogHeader>
|
||||||
|
<AlertDialogTitle>
|
||||||
|
{confirmation === 'manual'
|
||||||
|
? '切换回手动部署'
|
||||||
|
: '启用受信内网模式'}
|
||||||
|
</AlertDialogTitle>
|
||||||
|
<AlertDialogDescription>
|
||||||
|
{confirmation === 'manual'
|
||||||
|
? '当前来源配置将被删除,但已有部署与当前生产版本会保留。'
|
||||||
|
: '该模式允许访问私有网络并接受自签名证书。请确认此地址属于可信内部来源,且不会被非可信用户控制。'}
|
||||||
|
</AlertDialogDescription>
|
||||||
|
</AlertDialogHeader>
|
||||||
|
<AlertDialogFooter>
|
||||||
|
<AlertDialogCancel disabled={isPending}>取消</AlertDialogCancel>
|
||||||
|
<AlertDialogAction
|
||||||
|
disabled={isPending}
|
||||||
|
onClick={(event) => {
|
||||||
|
event.preventDefault();
|
||||||
|
if (confirmation === 'manual') {
|
||||||
|
deleteMutation.mutate();
|
||||||
|
} else if (confirmation === 'trusted_internal') {
|
||||||
|
updateMutation.mutate(remotePayload());
|
||||||
|
}
|
||||||
|
}}
|
||||||
|
>
|
||||||
|
{isPending ? <Spinner data-icon='inline-start' /> : null}
|
||||||
|
确认
|
||||||
|
</AlertDialogAction>
|
||||||
|
</AlertDialogFooter>
|
||||||
|
</AlertDialogContent>
|
||||||
|
</AlertDialog>
|
||||||
|
</>
|
||||||
|
);
|
||||||
|
}
|
||||||
@@ -0,0 +1,211 @@
|
|||||||
|
import {
|
||||||
|
Field,
|
||||||
|
FieldContent,
|
||||||
|
FieldDescription,
|
||||||
|
FieldError,
|
||||||
|
FieldLabel,
|
||||||
|
FieldTitle,
|
||||||
|
} from '@/components/ui/field';
|
||||||
|
import { Input } from '@/components/ui/input';
|
||||||
|
import { Switch } from '@/components/ui/switch';
|
||||||
|
import { ToggleGroup, ToggleGroupItem } from '@/components/ui/toggle-group';
|
||||||
|
import type { PagesGitHubReleaseSelector } from '@/lib/services/openflare';
|
||||||
|
|
||||||
|
export interface PagesGitHubSourceFormValue {
|
||||||
|
repositoryURL: string;
|
||||||
|
releaseSelector: PagesGitHubReleaseSelector;
|
||||||
|
releaseTag: string;
|
||||||
|
assetName: string;
|
||||||
|
autoUpdateEnabled: boolean;
|
||||||
|
checkIntervalMinutes: string;
|
||||||
|
}
|
||||||
|
|
||||||
|
export interface PagesGitHubSourceFormErrors {
|
||||||
|
repository: string;
|
||||||
|
releaseTag: string;
|
||||||
|
assetName: string;
|
||||||
|
checkInterval: string;
|
||||||
|
}
|
||||||
|
|
||||||
|
interface PagesSourceGitHubFieldsProps {
|
||||||
|
value: PagesGitHubSourceFormValue;
|
||||||
|
errors: PagesGitHubSourceFormErrors;
|
||||||
|
defaultAssetName: string;
|
||||||
|
onChange: (value: PagesGitHubSourceFormValue) => void;
|
||||||
|
onErrorsChange: (errors: PagesGitHubSourceFormErrors) => void;
|
||||||
|
}
|
||||||
|
|
||||||
|
export function PagesSourceGitHubFields({
|
||||||
|
value,
|
||||||
|
errors,
|
||||||
|
defaultAssetName,
|
||||||
|
onChange,
|
||||||
|
onErrorsChange,
|
||||||
|
}: PagesSourceGitHubFieldsProps) {
|
||||||
|
return (
|
||||||
|
<>
|
||||||
|
<Field data-invalid={Boolean(errors.repository)}>
|
||||||
|
<FieldLabel htmlFor='pages-github-repository'>
|
||||||
|
GitHub 仓库 URL
|
||||||
|
</FieldLabel>
|
||||||
|
<Input
|
||||||
|
id='pages-github-repository'
|
||||||
|
type='url'
|
||||||
|
placeholder='https://github.com/owner/repo'
|
||||||
|
value={value.repositoryURL}
|
||||||
|
aria-invalid={Boolean(errors.repository)}
|
||||||
|
aria-describedby='pages-github-repository-description pages-github-repository-error'
|
||||||
|
autoComplete='off'
|
||||||
|
onChange={(event) => {
|
||||||
|
onChange({ ...value, repositoryURL: event.target.value });
|
||||||
|
onErrorsChange({ ...errors, repository: '' });
|
||||||
|
}}
|
||||||
|
/>
|
||||||
|
<FieldDescription id='pages-github-repository-description'>
|
||||||
|
仅支持公开 github.com 仓库。
|
||||||
|
</FieldDescription>
|
||||||
|
<FieldError id='pages-github-repository-error'>
|
||||||
|
{errors.repository}
|
||||||
|
</FieldError>
|
||||||
|
</Field>
|
||||||
|
|
||||||
|
<Field>
|
||||||
|
<FieldTitle id='pages-github-selector'>Release 选择</FieldTitle>
|
||||||
|
<ToggleGroup
|
||||||
|
type='single'
|
||||||
|
variant='outline'
|
||||||
|
value={value.releaseSelector}
|
||||||
|
aria-labelledby='pages-github-selector'
|
||||||
|
className='grid w-full grid-cols-2'
|
||||||
|
onValueChange={(selector) => {
|
||||||
|
if (selector === 'latest' || selector === 'tag') {
|
||||||
|
onChange({
|
||||||
|
...value,
|
||||||
|
releaseSelector: selector,
|
||||||
|
autoUpdateEnabled:
|
||||||
|
selector === 'latest' ? value.autoUpdateEnabled : false,
|
||||||
|
});
|
||||||
|
onErrorsChange({
|
||||||
|
...errors,
|
||||||
|
releaseTag: '',
|
||||||
|
checkInterval: '',
|
||||||
|
});
|
||||||
|
}
|
||||||
|
}}
|
||||||
|
>
|
||||||
|
<ToggleGroupItem value='latest' className='w-full'>
|
||||||
|
最新 Release
|
||||||
|
</ToggleGroupItem>
|
||||||
|
<ToggleGroupItem value='tag' className='w-full'>
|
||||||
|
固定 Tag
|
||||||
|
</ToggleGroupItem>
|
||||||
|
</ToggleGroup>
|
||||||
|
<FieldDescription>
|
||||||
|
{value.releaseSelector === 'latest'
|
||||||
|
? '定时检查最新 Release;是否自动发布由下方开关控制。'
|
||||||
|
: '固定 Tag 仅支持管理员手动检查与同步。'}
|
||||||
|
</FieldDescription>
|
||||||
|
</Field>
|
||||||
|
|
||||||
|
{value.releaseSelector === 'tag' ? (
|
||||||
|
<Field data-invalid={Boolean(errors.releaseTag)}>
|
||||||
|
<FieldLabel htmlFor='pages-github-tag'>Release tag</FieldLabel>
|
||||||
|
<Input
|
||||||
|
id='pages-github-tag'
|
||||||
|
placeholder='v1.2.3'
|
||||||
|
value={value.releaseTag}
|
||||||
|
aria-invalid={Boolean(errors.releaseTag)}
|
||||||
|
aria-describedby='pages-github-tag-description pages-github-tag-error'
|
||||||
|
autoComplete='off'
|
||||||
|
onChange={(event) => {
|
||||||
|
onChange({ ...value, releaseTag: event.target.value });
|
||||||
|
onErrorsChange({ ...errors, releaseTag: '' });
|
||||||
|
}}
|
||||||
|
/>
|
||||||
|
<FieldDescription id='pages-github-tag-description'>
|
||||||
|
精确检查并同步指定 tag。
|
||||||
|
</FieldDescription>
|
||||||
|
<FieldError id='pages-github-tag-error'>
|
||||||
|
{errors.releaseTag}
|
||||||
|
</FieldError>
|
||||||
|
</Field>
|
||||||
|
) : null}
|
||||||
|
|
||||||
|
{value.releaseSelector === 'latest' ? (
|
||||||
|
<>
|
||||||
|
<Field orientation='horizontal'>
|
||||||
|
<FieldContent>
|
||||||
|
<FieldLabel htmlFor='pages-github-auto-update'>
|
||||||
|
自动更新
|
||||||
|
</FieldLabel>
|
||||||
|
<FieldDescription>
|
||||||
|
检查到新的 Release 后自动同步并发布。
|
||||||
|
</FieldDescription>
|
||||||
|
</FieldContent>
|
||||||
|
<Switch
|
||||||
|
id='pages-github-auto-update'
|
||||||
|
checked={value.autoUpdateEnabled}
|
||||||
|
onCheckedChange={(checked) =>
|
||||||
|
onChange({ ...value, autoUpdateEnabled: checked })
|
||||||
|
}
|
||||||
|
/>
|
||||||
|
</Field>
|
||||||
|
|
||||||
|
<Field data-invalid={Boolean(errors.checkInterval)}>
|
||||||
|
<FieldLabel htmlFor='pages-github-check-interval'>
|
||||||
|
检查间隔(分钟)
|
||||||
|
</FieldLabel>
|
||||||
|
<Input
|
||||||
|
id='pages-github-check-interval'
|
||||||
|
type='number'
|
||||||
|
min={5}
|
||||||
|
max={1440}
|
||||||
|
step={1}
|
||||||
|
inputMode='numeric'
|
||||||
|
value={value.checkIntervalMinutes}
|
||||||
|
aria-invalid={Boolean(errors.checkInterval)}
|
||||||
|
aria-describedby='pages-github-check-interval-description pages-github-check-interval-error'
|
||||||
|
onChange={(event) => {
|
||||||
|
onChange({
|
||||||
|
...value,
|
||||||
|
checkIntervalMinutes: event.target.value,
|
||||||
|
});
|
||||||
|
onErrorsChange({ ...errors, checkInterval: '' });
|
||||||
|
}}
|
||||||
|
/>
|
||||||
|
<FieldDescription id='pages-github-check-interval-description'>
|
||||||
|
可设置为 5–1440 分钟。
|
||||||
|
</FieldDescription>
|
||||||
|
<FieldError id='pages-github-check-interval-error'>
|
||||||
|
{errors.checkInterval}
|
||||||
|
</FieldError>
|
||||||
|
</Field>
|
||||||
|
</>
|
||||||
|
) : null}
|
||||||
|
|
||||||
|
<Field data-invalid={Boolean(errors.assetName)}>
|
||||||
|
<FieldLabel htmlFor='pages-github-asset'>
|
||||||
|
Release Asset 文件名
|
||||||
|
</FieldLabel>
|
||||||
|
<Input
|
||||||
|
id='pages-github-asset'
|
||||||
|
placeholder={defaultAssetName}
|
||||||
|
value={value.assetName}
|
||||||
|
aria-invalid={Boolean(errors.assetName)}
|
||||||
|
aria-describedby='pages-github-asset-description pages-github-asset-error'
|
||||||
|
autoComplete='off'
|
||||||
|
onChange={(event) => {
|
||||||
|
onChange({ ...value, assetName: event.target.value });
|
||||||
|
onErrorsChange({ ...errors, assetName: '' });
|
||||||
|
}}
|
||||||
|
/>
|
||||||
|
<FieldDescription id='pages-github-asset-description'>
|
||||||
|
按文件名精确匹配已上传的 Release Asset。
|
||||||
|
</FieldDescription>
|
||||||
|
<FieldError id='pages-github-asset-error'>
|
||||||
|
{errors.assetName}
|
||||||
|
</FieldError>
|
||||||
|
</Field>
|
||||||
|
</>
|
||||||
|
);
|
||||||
|
}
|
||||||
@@ -0,0 +1,68 @@
|
|||||||
|
function hasWellFormedUnicode(value: string) {
|
||||||
|
for (let index = 0; index < value.length; index += 1) {
|
||||||
|
const code = value.charCodeAt(index);
|
||||||
|
if (code >= 0xd800 && code <= 0xdbff) {
|
||||||
|
const next = value.charCodeAt(index + 1);
|
||||||
|
if (!(next >= 0xdc00 && next <= 0xdfff)) return false;
|
||||||
|
index += 1;
|
||||||
|
} else if (code >= 0xdc00 && code <= 0xdfff) {
|
||||||
|
return false;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return true;
|
||||||
|
}
|
||||||
|
|
||||||
|
function validGitHubSafeText(value: string) {
|
||||||
|
return (
|
||||||
|
value !== '' &&
|
||||||
|
new TextEncoder().encode(value).byteLength <= 255 &&
|
||||||
|
!/[\u0000-\u001f\u007f-\u009f\u061c\u200e\u200f\u2028-\u202e\u2066-\u2069]/u.test(
|
||||||
|
value,
|
||||||
|
) &&
|
||||||
|
hasWellFormedUnicode(value)
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
|
export function validGitHubRepositoryURL(raw: string) {
|
||||||
|
const match = /^https:\/\/([^/]+)\/([^/]+)\/([^/]+)$/u.exec(raw);
|
||||||
|
if (!match) return false;
|
||||||
|
|
||||||
|
const [, host, owner, rawRepository] = match;
|
||||||
|
const repository = rawRepository.replace(/\.git$/u, '');
|
||||||
|
return (
|
||||||
|
host.toLowerCase() === 'github.com' &&
|
||||||
|
/^[a-z0-9](?:[a-z0-9-]{0,37}[a-z0-9])?$/iu.test(owner) &&
|
||||||
|
/^[a-z0-9._-]+$/iu.test(repository) &&
|
||||||
|
repository.length <= 100 &&
|
||||||
|
!['.', '..'].includes(repository)
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
|
export function validGitHubAssetName(value: string) {
|
||||||
|
return (
|
||||||
|
validGitHubSafeText(value) &&
|
||||||
|
value !== '.' &&
|
||||||
|
value !== '..' &&
|
||||||
|
!value.includes('/') &&
|
||||||
|
!value.includes('\\')
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
|
export function validGitHubReleaseTag(value: string) {
|
||||||
|
const components = value.split('/');
|
||||||
|
return (
|
||||||
|
validGitHubSafeText(value) &&
|
||||||
|
!value.endsWith('.') &&
|
||||||
|
!value.includes('..') &&
|
||||||
|
!value.includes('@{') &&
|
||||||
|
![' ', '~', '^', ':', '?', '*', '[', '\\'].some((character) =>
|
||||||
|
value.includes(character),
|
||||||
|
) &&
|
||||||
|
components.every(
|
||||||
|
(component) =>
|
||||||
|
component !== '' &&
|
||||||
|
!component.startsWith('.') &&
|
||||||
|
!component.endsWith('.lock'),
|
||||||
|
)
|
||||||
|
);
|
||||||
|
}
|
||||||
@@ -1,505 +0,0 @@
|
|||||||
'use client';
|
|
||||||
|
|
||||||
import Link from 'next/link';
|
|
||||||
import { useMemo, useState } from 'react';
|
|
||||||
import { useMutation, useQuery, useQueryClient } from '@tanstack/react-query';
|
|
||||||
import { useSearchParams } from 'next/navigation';
|
|
||||||
import {
|
|
||||||
ArrowLeft,
|
|
||||||
ChevronDown,
|
|
||||||
ChevronRight,
|
|
||||||
FileText,
|
|
||||||
Loader2,
|
|
||||||
Trash2,
|
|
||||||
Upload,
|
|
||||||
} from 'lucide-react';
|
|
||||||
import { toast } from 'sonner';
|
|
||||||
|
|
||||||
import {
|
|
||||||
AlertDialog,
|
|
||||||
AlertDialogAction,
|
|
||||||
AlertDialogCancel,
|
|
||||||
AlertDialogContent,
|
|
||||||
AlertDialogDescription,
|
|
||||||
AlertDialogFooter,
|
|
||||||
AlertDialogHeader,
|
|
||||||
AlertDialogTitle,
|
|
||||||
} from '@/components/ui/alert-dialog';
|
|
||||||
import { EmptyStateWithBorder } from '@/components/layout/empty';
|
|
||||||
import { ErrorInline } from '@/components/layout/error';
|
|
||||||
import { LoadingStateWithBorder } from '@/components/layout/loading';
|
|
||||||
import { Badge } from '@/components/ui/badge';
|
|
||||||
import { Button } from '@/components/ui/button';
|
|
||||||
import {
|
|
||||||
Table,
|
|
||||||
TableBody,
|
|
||||||
TableCell,
|
|
||||||
TableHead,
|
|
||||||
TableHeader,
|
|
||||||
TableRow,
|
|
||||||
} from '@/components/ui/table';
|
|
||||||
import { type PagesDeployment, PagesService } from '@/lib/services/openflare';
|
|
||||||
import { formatDateTime } from '@/lib/utils';
|
|
||||||
|
|
||||||
import { DeploymentUploadDialog } from '../components/deployment-upload-dialog';
|
|
||||||
import { ProjectEditorDialog } from '../components/project-editor-dialog';
|
|
||||||
import {
|
|
||||||
deploymentFilesQueryKey,
|
|
||||||
deploymentsQueryKey,
|
|
||||||
formatBytes,
|
|
||||||
projectQueryKey,
|
|
||||||
projectsQueryKey,
|
|
||||||
} from '../components/pages-utils';
|
|
||||||
|
|
||||||
function DeploymentFilesPanel({
|
|
||||||
projectId,
|
|
||||||
deployment,
|
|
||||||
}: {
|
|
||||||
projectId: number;
|
|
||||||
deployment: PagesDeployment;
|
|
||||||
}) {
|
|
||||||
const filesQuery = useQuery({
|
|
||||||
queryKey: deploymentFilesQueryKey(projectId, deployment.id),
|
|
||||||
queryFn: () => PagesService.listDeploymentFiles(deployment.id),
|
|
||||||
});
|
|
||||||
|
|
||||||
if (filesQuery.isLoading) {
|
|
||||||
return (
|
|
||||||
<p className='px-4 py-3 text-xs text-muted-foreground'>加载文件清单...</p>
|
|
||||||
);
|
|
||||||
}
|
|
||||||
|
|
||||||
if (filesQuery.isError) {
|
|
||||||
return (
|
|
||||||
<p className='px-4 py-3 text-xs text-destructive'>
|
|
||||||
{filesQuery.error instanceof Error
|
|
||||||
? filesQuery.error.message
|
|
||||||
: '加载失败'}
|
|
||||||
</p>
|
|
||||||
);
|
|
||||||
}
|
|
||||||
|
|
||||||
const files = filesQuery.data ?? [];
|
|
||||||
if (files.length === 0) {
|
|
||||||
return (
|
|
||||||
<p className='px-4 py-3 text-xs text-muted-foreground'>暂无文件记录</p>
|
|
||||||
);
|
|
||||||
}
|
|
||||||
|
|
||||||
return (
|
|
||||||
<div className='border-t border-dashed bg-muted/10'>
|
|
||||||
<Table>
|
|
||||||
<TableHeader>
|
|
||||||
<TableRow className='border-dashed hover:bg-transparent'>
|
|
||||||
<TableHead className='text-xs'>路径</TableHead>
|
|
||||||
<TableHead className='text-xs text-right'>大小</TableHead>
|
|
||||||
</TableRow>
|
|
||||||
</TableHeader>
|
|
||||||
<TableBody>
|
|
||||||
{files.map((file) => (
|
|
||||||
<TableRow key={file.id} className='border-dashed'>
|
|
||||||
<TableCell className='text-xs font-mono'>{file.path}</TableCell>
|
|
||||||
<TableCell className='text-xs text-right text-muted-foreground'>
|
|
||||||
{formatBytes(file.size)}
|
|
||||||
</TableCell>
|
|
||||||
</TableRow>
|
|
||||||
))}
|
|
||||||
</TableBody>
|
|
||||||
</Table>
|
|
||||||
</div>
|
|
||||||
);
|
|
||||||
}
|
|
||||||
|
|
||||||
export function PagesDetailPageClient() {
|
|
||||||
const searchParams = useSearchParams();
|
|
||||||
const queryClient = useQueryClient();
|
|
||||||
const projectId = searchParams.get('id')?.trim() ?? '';
|
|
||||||
const parsedProjectId = Number(projectId);
|
|
||||||
|
|
||||||
const [editorOpen, setEditorOpen] = useState(false);
|
|
||||||
const [uploadOpen, setUploadOpen] = useState(false);
|
|
||||||
const [expandedDeploymentId, setExpandedDeploymentId] = useState<
|
|
||||||
number | null
|
|
||||||
>(null);
|
|
||||||
const [deleteProjectOpen, setDeleteProjectOpen] = useState(false);
|
|
||||||
const [pendingDeploymentAction, setPendingDeploymentAction] = useState<{
|
|
||||||
type: 'activate' | 'delete';
|
|
||||||
deployment: PagesDeployment;
|
|
||||||
} | null>(null);
|
|
||||||
|
|
||||||
const enabled = projectId !== '' && Number.isFinite(parsedProjectId);
|
|
||||||
|
|
||||||
const projectQuery = useQuery({
|
|
||||||
queryKey: projectQueryKey(projectId),
|
|
||||||
queryFn: () => PagesService.getProject(parsedProjectId),
|
|
||||||
enabled,
|
|
||||||
});
|
|
||||||
|
|
||||||
const deploymentsQuery = useQuery({
|
|
||||||
queryKey: deploymentsQueryKey(parsedProjectId),
|
|
||||||
queryFn: () => PagesService.listDeployments(parsedProjectId),
|
|
||||||
enabled,
|
|
||||||
});
|
|
||||||
|
|
||||||
const activateMutation = useMutation({
|
|
||||||
mutationFn: (deploymentId: number) =>
|
|
||||||
PagesService.activateDeployment(parsedProjectId, deploymentId),
|
|
||||||
onSuccess: async () => {
|
|
||||||
toast.success('部署已激活');
|
|
||||||
await Promise.all([
|
|
||||||
queryClient.invalidateQueries({
|
|
||||||
queryKey: deploymentsQueryKey(parsedProjectId),
|
|
||||||
}),
|
|
||||||
queryClient.invalidateQueries({ queryKey: projectQueryKey(projectId) }),
|
|
||||||
queryClient.invalidateQueries({ queryKey: projectsQueryKey }),
|
|
||||||
]);
|
|
||||||
setPendingDeploymentAction(null);
|
|
||||||
},
|
|
||||||
onError: (error) => {
|
|
||||||
toast.error(error instanceof Error ? error.message : '激活失败');
|
|
||||||
},
|
|
||||||
});
|
|
||||||
|
|
||||||
const deleteDeploymentMutation = useMutation({
|
|
||||||
mutationFn: (deploymentId: number) =>
|
|
||||||
PagesService.deleteDeployment(parsedProjectId, deploymentId),
|
|
||||||
onSuccess: async () => {
|
|
||||||
toast.success('部署已删除');
|
|
||||||
await Promise.all([
|
|
||||||
queryClient.invalidateQueries({
|
|
||||||
queryKey: deploymentsQueryKey(parsedProjectId),
|
|
||||||
}),
|
|
||||||
queryClient.invalidateQueries({ queryKey: projectQueryKey(projectId) }),
|
|
||||||
queryClient.invalidateQueries({ queryKey: projectsQueryKey }),
|
|
||||||
]);
|
|
||||||
setPendingDeploymentAction(null);
|
|
||||||
},
|
|
||||||
onError: (error) => {
|
|
||||||
toast.error(error instanceof Error ? error.message : '删除失败');
|
|
||||||
},
|
|
||||||
});
|
|
||||||
|
|
||||||
const deleteProjectMutation = useMutation({
|
|
||||||
mutationFn: () => PagesService.deleteProject(parsedProjectId),
|
|
||||||
onSuccess: async () => {
|
|
||||||
toast.success('项目已删除');
|
|
||||||
await queryClient.invalidateQueries({ queryKey: projectsQueryKey });
|
|
||||||
window.location.href = '/pages';
|
|
||||||
},
|
|
||||||
onError: (error) => {
|
|
||||||
toast.error(error instanceof Error ? error.message : '删除失败');
|
|
||||||
},
|
|
||||||
});
|
|
||||||
|
|
||||||
const project = projectQuery.data;
|
|
||||||
const deployments = useMemo(
|
|
||||||
() => deploymentsQuery.data ?? [],
|
|
||||||
[deploymentsQuery.data],
|
|
||||||
);
|
|
||||||
|
|
||||||
if (!enabled) {
|
|
||||||
return (
|
|
||||||
<div className='py-6 px-1'>
|
|
||||||
<EmptyStateWithBorder description='缺少有效的 Pages 项目 ID。' />
|
|
||||||
</div>
|
|
||||||
);
|
|
||||||
}
|
|
||||||
|
|
||||||
if (projectQuery.isLoading) {
|
|
||||||
return (
|
|
||||||
<div className='py-6 px-1'>
|
|
||||||
<LoadingStateWithBorder icon={FileText} description='加载项目详情...' />
|
|
||||||
</div>
|
|
||||||
);
|
|
||||||
}
|
|
||||||
|
|
||||||
if (projectQuery.isError) {
|
|
||||||
return (
|
|
||||||
<div className='py-6 px-1'>
|
|
||||||
<ErrorInline
|
|
||||||
message={
|
|
||||||
projectQuery.error instanceof Error
|
|
||||||
? projectQuery.error.message
|
|
||||||
: '加载失败'
|
|
||||||
}
|
|
||||||
onRetry={() => void projectQuery.refetch()}
|
|
||||||
/>
|
|
||||||
</div>
|
|
||||||
);
|
|
||||||
}
|
|
||||||
|
|
||||||
if (!project) {
|
|
||||||
return (
|
|
||||||
<div className='py-6 px-1 space-y-4'>
|
|
||||||
<Button variant='ghost' size='sm' asChild>
|
|
||||||
<Link href='/pages'>
|
|
||||||
<ArrowLeft className='size-4 mr-1' />
|
|
||||||
返回列表
|
|
||||||
</Link>
|
|
||||||
</Button>
|
|
||||||
<EmptyStateWithBorder description='Pages 项目不存在或已被删除。' />
|
|
||||||
</div>
|
|
||||||
);
|
|
||||||
}
|
|
||||||
|
|
||||||
return (
|
|
||||||
<div className='py-6 px-1 space-y-6'>
|
|
||||||
<div className='flex flex-col gap-4 lg:flex-row lg:items-start lg:justify-between'>
|
|
||||||
<div className='space-y-2'>
|
|
||||||
<Button variant='ghost' size='sm' className='h-8 px-2 -ml-2' asChild>
|
|
||||||
<Link href='/pages'>
|
|
||||||
<ArrowLeft className='size-4 mr-1' />
|
|
||||||
返回列表
|
|
||||||
</Link>
|
|
||||||
</Button>
|
|
||||||
<div className='flex items-center gap-2'>
|
|
||||||
<FileText className='size-5 text-primary' />
|
|
||||||
<h1 className='text-2xl font-semibold tracking-tight'>
|
|
||||||
{project.name}
|
|
||||||
</h1>
|
|
||||||
</div>
|
|
||||||
<p className='text-sm text-muted-foreground'>
|
|
||||||
{project.slug} · {project.deployment_count} 个部署
|
|
||||||
</p>
|
|
||||||
</div>
|
|
||||||
<div className='flex flex-wrap gap-2'>
|
|
||||||
<Button
|
|
||||||
variant='outline'
|
|
||||||
size='sm'
|
|
||||||
onClick={() => setEditorOpen(true)}
|
|
||||||
>
|
|
||||||
编辑项目
|
|
||||||
</Button>
|
|
||||||
<Button size='sm' onClick={() => setUploadOpen(true)}>
|
|
||||||
<Upload className='size-3.5 mr-1' />
|
|
||||||
上传部署包
|
|
||||||
</Button>
|
|
||||||
<Button
|
|
||||||
variant='destructive'
|
|
||||||
size='sm'
|
|
||||||
onClick={() => setDeleteProjectOpen(true)}
|
|
||||||
>
|
|
||||||
<Trash2 className='size-3.5 mr-1' />
|
|
||||||
删除项目
|
|
||||||
</Button>
|
|
||||||
</div>
|
|
||||||
</div>
|
|
||||||
|
|
||||||
<div className='grid gap-3 sm:grid-cols-3'>
|
|
||||||
<div className='rounded-lg border border-dashed px-4 py-3'>
|
|
||||||
<p className='text-[10px] uppercase tracking-wider text-muted-foreground'>
|
|
||||||
状态
|
|
||||||
</p>
|
|
||||||
<Badge variant='outline' className='mt-2 text-[10px]'>
|
|
||||||
{project.enabled ? '已启用' : '已停用'}
|
|
||||||
</Badge>
|
|
||||||
</div>
|
|
||||||
<div className='rounded-lg border border-dashed px-4 py-3'>
|
|
||||||
<p className='text-[10px] uppercase tracking-wider text-muted-foreground'>
|
|
||||||
当前激活
|
|
||||||
</p>
|
|
||||||
<p className='mt-2 text-sm font-semibold'>
|
|
||||||
{project.active_deployment
|
|
||||||
? `#${project.active_deployment.deployment_number}`
|
|
||||||
: '暂无'}
|
|
||||||
</p>
|
|
||||||
</div>
|
|
||||||
<div className='rounded-lg border border-dashed px-4 py-3'>
|
|
||||||
<p className='text-[10px] uppercase tracking-wider text-muted-foreground'>
|
|
||||||
更新时间
|
|
||||||
</p>
|
|
||||||
<p className='mt-2 text-sm'>{formatDateTime(project.updated_at)}</p>
|
|
||||||
</div>
|
|
||||||
</div>
|
|
||||||
|
|
||||||
<div className='border border-dashed rounded-lg overflow-hidden bg-background'>
|
|
||||||
<div className='px-4 py-3 border-b border-dashed'>
|
|
||||||
<h2 className='text-sm font-semibold'>部署历史</h2>
|
|
||||||
<p className='text-xs text-muted-foreground mt-1'>
|
|
||||||
部署不可变;激活后发布配置,Agent 才会拉取并切换静态资源。
|
|
||||||
</p>
|
|
||||||
</div>
|
|
||||||
|
|
||||||
{deploymentsQuery.isLoading ? (
|
|
||||||
<LoadingStateWithBorder />
|
|
||||||
) : deployments.length === 0 ? (
|
|
||||||
<EmptyStateWithBorder
|
|
||||||
title='暂无部署'
|
|
||||||
description='上传 zip 部署包后,可以在这里激活某个部署版本。'
|
|
||||||
/>
|
|
||||||
) : (
|
|
||||||
<div className='divide-y divide-dashed'>
|
|
||||||
{deployments.map((deployment) => {
|
|
||||||
const expanded = expandedDeploymentId === deployment.id;
|
|
||||||
return (
|
|
||||||
<div key={deployment.id}>
|
|
||||||
<div className='flex flex-col gap-3 p-4 md:flex-row md:items-center md:justify-between'>
|
|
||||||
<div className='flex items-start gap-2'>
|
|
||||||
<Button
|
|
||||||
variant='ghost'
|
|
||||||
size='icon'
|
|
||||||
className='size-7 shrink-0'
|
|
||||||
onClick={() =>
|
|
||||||
setExpandedDeploymentId(
|
|
||||||
expanded ? null : deployment.id,
|
|
||||||
)
|
|
||||||
}
|
|
||||||
>
|
|
||||||
{expanded ? (
|
|
||||||
<ChevronDown className='size-4' />
|
|
||||||
) : (
|
|
||||||
<ChevronRight className='size-4' />
|
|
||||||
)}
|
|
||||||
</Button>
|
|
||||||
<div>
|
|
||||||
<div className='flex items-center gap-2'>
|
|
||||||
<p className='text-sm font-medium'>
|
|
||||||
#{deployment.deployment_number}
|
|
||||||
</p>
|
|
||||||
{deployment.status === 'active' ? (
|
|
||||||
<Badge variant='outline' className='text-[10px]'>
|
|
||||||
已激活
|
|
||||||
</Badge>
|
|
||||||
) : null}
|
|
||||||
</div>
|
|
||||||
<p className='mt-1 text-xs text-muted-foreground'>
|
|
||||||
{deployment.checksum.slice(0, 16)} ·{' '}
|
|
||||||
{deployment.file_count} files ·{' '}
|
|
||||||
{formatBytes(deployment.total_size)}
|
|
||||||
</p>
|
|
||||||
<p className='mt-1 text-xs text-muted-foreground'>
|
|
||||||
创建于 {formatDateTime(deployment.created_at)}
|
|
||||||
</p>
|
|
||||||
</div>
|
|
||||||
</div>
|
|
||||||
<div className='flex gap-2 md:ml-9'>
|
|
||||||
<Button
|
|
||||||
variant='outline'
|
|
||||||
size='sm'
|
|
||||||
disabled={
|
|
||||||
deployment.status === 'active' ||
|
|
||||||
activateMutation.isPending
|
|
||||||
}
|
|
||||||
onClick={() =>
|
|
||||||
setPendingDeploymentAction({
|
|
||||||
type: 'activate',
|
|
||||||
deployment,
|
|
||||||
})
|
|
||||||
}
|
|
||||||
>
|
|
||||||
激活
|
|
||||||
</Button>
|
|
||||||
<Button
|
|
||||||
variant='destructive'
|
|
||||||
size='sm'
|
|
||||||
disabled={
|
|
||||||
deployment.status === 'active' ||
|
|
||||||
deleteDeploymentMutation.isPending
|
|
||||||
}
|
|
||||||
onClick={() =>
|
|
||||||
setPendingDeploymentAction({
|
|
||||||
type: 'delete',
|
|
||||||
deployment,
|
|
||||||
})
|
|
||||||
}
|
|
||||||
>
|
|
||||||
删除
|
|
||||||
</Button>
|
|
||||||
</div>
|
|
||||||
</div>
|
|
||||||
{expanded ? (
|
|
||||||
<DeploymentFilesPanel
|
|
||||||
projectId={parsedProjectId}
|
|
||||||
deployment={deployment}
|
|
||||||
/>
|
|
||||||
) : null}
|
|
||||||
</div>
|
|
||||||
);
|
|
||||||
})}
|
|
||||||
</div>
|
|
||||||
)}
|
|
||||||
</div>
|
|
||||||
|
|
||||||
<ProjectEditorDialog
|
|
||||||
open={editorOpen}
|
|
||||||
onOpenChange={setEditorOpen}
|
|
||||||
project={project}
|
|
||||||
/>
|
|
||||||
<DeploymentUploadDialog
|
|
||||||
open={uploadOpen}
|
|
||||||
onOpenChange={setUploadOpen}
|
|
||||||
projectId={parsedProjectId}
|
|
||||||
/>
|
|
||||||
|
|
||||||
<AlertDialog open={deleteProjectOpen} onOpenChange={setDeleteProjectOpen}>
|
|
||||||
<AlertDialogContent>
|
|
||||||
<AlertDialogHeader>
|
|
||||||
<AlertDialogTitle>删除 Pages 项目</AlertDialogTitle>
|
|
||||||
<AlertDialogDescription>
|
|
||||||
确认删除项目 {project.name} 吗?此操作不可恢复。
|
|
||||||
</AlertDialogDescription>
|
|
||||||
</AlertDialogHeader>
|
|
||||||
<AlertDialogFooter>
|
|
||||||
<AlertDialogCancel>取消</AlertDialogCancel>
|
|
||||||
<AlertDialogAction
|
|
||||||
className='bg-destructive text-destructive-foreground hover:bg-destructive/90'
|
|
||||||
onClick={() => deleteProjectMutation.mutate()}
|
|
||||||
>
|
|
||||||
{deleteProjectMutation.isPending ? (
|
|
||||||
<Loader2 className='size-4 animate-spin' />
|
|
||||||
) : (
|
|
||||||
'确认删除'
|
|
||||||
)}
|
|
||||||
</AlertDialogAction>
|
|
||||||
</AlertDialogFooter>
|
|
||||||
</AlertDialogContent>
|
|
||||||
</AlertDialog>
|
|
||||||
|
|
||||||
<AlertDialog
|
|
||||||
open={pendingDeploymentAction !== null}
|
|
||||||
onOpenChange={(open) => {
|
|
||||||
if (!open) setPendingDeploymentAction(null);
|
|
||||||
}}
|
|
||||||
>
|
|
||||||
<AlertDialogContent>
|
|
||||||
<AlertDialogHeader>
|
|
||||||
<AlertDialogTitle>
|
|
||||||
{pendingDeploymentAction?.type === 'activate'
|
|
||||||
? '激活部署'
|
|
||||||
: '删除部署'}
|
|
||||||
</AlertDialogTitle>
|
|
||||||
<AlertDialogDescription>
|
|
||||||
{pendingDeploymentAction?.type === 'activate'
|
|
||||||
? `确认激活部署 #${pendingDeploymentAction.deployment.deployment_number} 吗?`
|
|
||||||
: `确认删除部署 #${pendingDeploymentAction?.deployment.deployment_number} 吗?`}
|
|
||||||
</AlertDialogDescription>
|
|
||||||
</AlertDialogHeader>
|
|
||||||
<AlertDialogFooter>
|
|
||||||
<AlertDialogCancel>取消</AlertDialogCancel>
|
|
||||||
<AlertDialogAction
|
|
||||||
className={
|
|
||||||
pendingDeploymentAction?.type === 'delete'
|
|
||||||
? 'bg-destructive text-destructive-foreground hover:bg-destructive/90'
|
|
||||||
: undefined
|
|
||||||
}
|
|
||||||
onClick={() => {
|
|
||||||
if (!pendingDeploymentAction) return;
|
|
||||||
if (pendingDeploymentAction.type === 'activate') {
|
|
||||||
activateMutation.mutate(
|
|
||||||
pendingDeploymentAction.deployment.id,
|
|
||||||
);
|
|
||||||
} else {
|
|
||||||
deleteDeploymentMutation.mutate(
|
|
||||||
pendingDeploymentAction.deployment.id,
|
|
||||||
);
|
|
||||||
}
|
|
||||||
}}
|
|
||||||
>
|
|
||||||
确认
|
|
||||||
</AlertDialogAction>
|
|
||||||
</AlertDialogFooter>
|
|
||||||
</AlertDialogContent>
|
|
||||||
</AlertDialog>
|
|
||||||
</div>
|
|
||||||
);
|
|
||||||
}
|
|
||||||
@@ -1,23 +1,319 @@
|
|||||||
import { Suspense } from 'react';
|
'use client';
|
||||||
|
|
||||||
|
import Link from 'next/link';
|
||||||
|
import { Suspense, useState } from 'react';
|
||||||
|
import { useMutation, useQuery, useQueryClient } from '@tanstack/react-query';
|
||||||
|
import { useRouter, useSearchParams } from 'next/navigation';
|
||||||
|
import { ArrowLeft, FileText, Pencil, Trash2, Upload } from 'lucide-react';
|
||||||
|
import { toast } from 'sonner';
|
||||||
|
|
||||||
|
import { EmptyStateWithBorder } from '@/components/layout/empty';
|
||||||
|
import { ErrorInline } from '@/components/layout/error';
|
||||||
|
import { LoadingStateWithBorder } from '@/components/layout/loading';
|
||||||
|
import {
|
||||||
|
AlertDialog,
|
||||||
|
AlertDialogAction,
|
||||||
|
AlertDialogCancel,
|
||||||
|
AlertDialogContent,
|
||||||
|
AlertDialogDescription,
|
||||||
|
AlertDialogFooter,
|
||||||
|
AlertDialogHeader,
|
||||||
|
AlertDialogTitle,
|
||||||
|
} from '@/components/ui/alert-dialog';
|
||||||
|
import { Badge } from '@/components/ui/badge';
|
||||||
|
import { Button } from '@/components/ui/button';
|
||||||
|
import {
|
||||||
|
Card,
|
||||||
|
CardAction,
|
||||||
|
CardContent,
|
||||||
|
CardDescription,
|
||||||
|
CardHeader,
|
||||||
|
CardTitle,
|
||||||
|
} from '@/components/ui/card';
|
||||||
import { Skeleton } from '@/components/ui/skeleton';
|
import { Skeleton } from '@/components/ui/skeleton';
|
||||||
|
import { Spinner } from '@/components/ui/spinner';
|
||||||
|
import { PagesService } from '@/lib/services/openflare';
|
||||||
|
import { formatDateTime } from '@/lib/utils';
|
||||||
|
|
||||||
import { PagesDetailPageClient } from './page-client';
|
import {
|
||||||
|
DeploymentUploadDialog,
|
||||||
|
pagesEntryPath,
|
||||||
|
} from '../components/deployment-upload-dialog';
|
||||||
|
import { ProjectEditorDialog } from '../components/project-editor-dialog';
|
||||||
|
import {
|
||||||
|
projectQueryKey,
|
||||||
|
projectsQueryKey,
|
||||||
|
sourceQueryKey,
|
||||||
|
} from '../components/pages-utils';
|
||||||
|
import { DeploymentHistory } from './components/deployment-history';
|
||||||
|
import { PagesSourceCard } from './components/pages-source-card';
|
||||||
|
|
||||||
function PagesDetailPageFallback() {
|
function PagesDetailPageFallback() {
|
||||||
return (
|
return (
|
||||||
<div className='py-6 px-1 space-y-6'>
|
<div className='flex w-full flex-col gap-6 px-1 py-6'>
|
||||||
<Skeleton className='h-8 w-48' />
|
<Skeleton className='h-8 w-32' />
|
||||||
<Skeleton className='h-10 w-full max-w-xl' />
|
<Skeleton className='h-12 w-full max-w-xl' />
|
||||||
|
<div className='grid gap-4 lg:grid-cols-2'>
|
||||||
|
<Skeleton className='h-40 w-full' />
|
||||||
|
<Skeleton className='h-40 w-full' />
|
||||||
|
</div>
|
||||||
<Skeleton className='h-64 w-full' />
|
<Skeleton className='h-64 w-full' />
|
||||||
</div>
|
</div>
|
||||||
);
|
);
|
||||||
}
|
}
|
||||||
|
|
||||||
|
function PagesDetailRoute() {
|
||||||
|
const searchParams = useSearchParams();
|
||||||
|
const router = useRouter();
|
||||||
|
const queryClient = useQueryClient();
|
||||||
|
const [editorOpen, setEditorOpen] = useState(false);
|
||||||
|
const [uploadOpen, setUploadOpen] = useState(false);
|
||||||
|
const [deleteProjectOpen, setDeleteProjectOpen] = useState(false);
|
||||||
|
|
||||||
|
const rawProjectId = searchParams.get('id')?.trim() ?? '';
|
||||||
|
const projectId = Number(rawProjectId);
|
||||||
|
const validProjectId =
|
||||||
|
rawProjectId !== '' && Number.isInteger(projectId) && projectId > 0;
|
||||||
|
|
||||||
|
const projectQuery = useQuery({
|
||||||
|
queryKey: projectQueryKey(projectId),
|
||||||
|
queryFn: () => PagesService.getProject(projectId),
|
||||||
|
enabled: validProjectId,
|
||||||
|
});
|
||||||
|
|
||||||
|
const deleteProjectMutation = useMutation({
|
||||||
|
mutationFn: () => PagesService.deleteProject(projectId),
|
||||||
|
onSuccess: async () => {
|
||||||
|
toast.success('项目已删除');
|
||||||
|
await queryClient.invalidateQueries({ queryKey: projectsQueryKey });
|
||||||
|
router.push('/pages');
|
||||||
|
},
|
||||||
|
onError: (error) => {
|
||||||
|
toast.error(error instanceof Error ? error.message : '删除失败');
|
||||||
|
},
|
||||||
|
});
|
||||||
|
|
||||||
|
if (!validProjectId) {
|
||||||
|
return (
|
||||||
|
<div className='w-full px-1 py-6'>
|
||||||
|
<EmptyStateWithBorder description='缺少有效的 Pages 项目 ID。' />
|
||||||
|
</div>
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
|
if (projectQuery.isLoading) {
|
||||||
|
return (
|
||||||
|
<div className='w-full px-1 py-6'>
|
||||||
|
<LoadingStateWithBorder icon={FileText} description='加载项目详情...' />
|
||||||
|
</div>
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
|
if (projectQuery.isError) {
|
||||||
|
return (
|
||||||
|
<div className='w-full px-1 py-6'>
|
||||||
|
<div className='rounded-lg border p-4'>
|
||||||
|
<ErrorInline
|
||||||
|
message={
|
||||||
|
projectQuery.error instanceof Error
|
||||||
|
? projectQuery.error.message
|
||||||
|
: '项目详情加载失败'
|
||||||
|
}
|
||||||
|
onRetry={() => void projectQuery.refetch()}
|
||||||
|
/>
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
|
const project = projectQuery.data;
|
||||||
|
if (!project) {
|
||||||
|
return (
|
||||||
|
<div className='flex w-full flex-col gap-4 px-1 py-6'>
|
||||||
|
<Button variant='ghost' size='sm' asChild>
|
||||||
|
<Link href='/pages'>
|
||||||
|
<ArrowLeft data-icon='inline-start' />
|
||||||
|
返回列表
|
||||||
|
</Link>
|
||||||
|
</Button>
|
||||||
|
<EmptyStateWithBorder description='Pages 项目不存在或已被删除。' />
|
||||||
|
</div>
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
|
const activeDeployment = project.active_deployment;
|
||||||
|
const entryPath = pagesEntryPath(project.root_dir ?? '', project.entry_file);
|
||||||
|
|
||||||
|
return (
|
||||||
|
<div className='flex w-full flex-col gap-6 px-1 py-6'>
|
||||||
|
<div className='flex flex-col gap-4'>
|
||||||
|
<Button variant='ghost' size='sm' className='self-start' asChild>
|
||||||
|
<Link href='/pages'>
|
||||||
|
<ArrowLeft data-icon='inline-start' />
|
||||||
|
返回列表
|
||||||
|
</Link>
|
||||||
|
</Button>
|
||||||
|
<div className='flex flex-col gap-4 lg:flex-row lg:items-start lg:justify-between'>
|
||||||
|
<div className='flex flex-col gap-2'>
|
||||||
|
<div className='flex items-center gap-2'>
|
||||||
|
<FileText className='size-5 text-primary' />
|
||||||
|
<h1 className='text-2xl font-semibold tracking-tight'>
|
||||||
|
{project.name}
|
||||||
|
</h1>
|
||||||
|
</div>
|
||||||
|
<p className='text-sm text-muted-foreground'>
|
||||||
|
{project.slug} · {project.deployment_count} 个部署
|
||||||
|
</p>
|
||||||
|
</div>
|
||||||
|
<div className='flex flex-wrap gap-2'>
|
||||||
|
<Button
|
||||||
|
type='button'
|
||||||
|
variant='outline'
|
||||||
|
size='sm'
|
||||||
|
onClick={() => setEditorOpen(true)}
|
||||||
|
>
|
||||||
|
<Pencil data-icon='inline-start' />
|
||||||
|
编辑项目
|
||||||
|
</Button>
|
||||||
|
<Button type='button' size='sm' onClick={() => setUploadOpen(true)}>
|
||||||
|
<Upload data-icon='inline-start' />
|
||||||
|
上传部署包
|
||||||
|
</Button>
|
||||||
|
<Button
|
||||||
|
type='button'
|
||||||
|
variant='destructive'
|
||||||
|
size='sm'
|
||||||
|
onClick={() => setDeleteProjectOpen(true)}
|
||||||
|
>
|
||||||
|
<Trash2 data-icon='inline-start' />
|
||||||
|
删除项目
|
||||||
|
</Button>
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
|
|
||||||
|
<div className='grid gap-4 lg:grid-cols-2'>
|
||||||
|
<Card>
|
||||||
|
<CardHeader>
|
||||||
|
<CardTitle>当前生产部署</CardTitle>
|
||||||
|
<CardDescription>
|
||||||
|
Agent 当前应拉取并提供服务的不可变部署。
|
||||||
|
</CardDescription>
|
||||||
|
<CardAction>
|
||||||
|
<Badge variant={activeDeployment ? 'default' : 'outline'}>
|
||||||
|
{activeDeployment ? '生产中' : '未发布'}
|
||||||
|
</Badge>
|
||||||
|
</CardAction>
|
||||||
|
</CardHeader>
|
||||||
|
<CardContent className='flex flex-col gap-2'>
|
||||||
|
{activeDeployment ? (
|
||||||
|
<>
|
||||||
|
<p className='text-lg font-semibold'>
|
||||||
|
部署 #{activeDeployment.deployment_number}
|
||||||
|
</p>
|
||||||
|
<p className='font-mono text-xs text-muted-foreground'>
|
||||||
|
{activeDeployment.checksum.slice(0, 20)}
|
||||||
|
</p>
|
||||||
|
<p className='text-xs text-muted-foreground'>
|
||||||
|
激活于{' '}
|
||||||
|
{activeDeployment.activated_at
|
||||||
|
? formatDateTime(activeDeployment.activated_at)
|
||||||
|
: '未知时间'}
|
||||||
|
</p>
|
||||||
|
</>
|
||||||
|
) : (
|
||||||
|
<p className='text-sm text-muted-foreground'>
|
||||||
|
尚无生产部署。上传或同步来源后,从部署历史激活一个版本。
|
||||||
|
</p>
|
||||||
|
)}
|
||||||
|
</CardContent>
|
||||||
|
</Card>
|
||||||
|
|
||||||
|
<Card>
|
||||||
|
<CardHeader>
|
||||||
|
<CardTitle>站点入口</CardTitle>
|
||||||
|
<CardDescription>
|
||||||
|
解包校验、发布快照与 Agent 切换共同使用此路径。
|
||||||
|
</CardDescription>
|
||||||
|
<CardAction>
|
||||||
|
<Badge variant={project.enabled ? 'secondary' : 'outline'}>
|
||||||
|
{project.enabled ? '项目已启用' : '项目已停用'}
|
||||||
|
</Badge>
|
||||||
|
</CardAction>
|
||||||
|
</CardHeader>
|
||||||
|
<CardContent className='flex flex-col gap-2'>
|
||||||
|
<code className='rounded-md border bg-muted/20 px-3 py-2 text-sm'>
|
||||||
|
{entryPath}
|
||||||
|
</code>
|
||||||
|
<p className='text-xs text-muted-foreground'>
|
||||||
|
项目更新于 {formatDateTime(project.updated_at)}
|
||||||
|
</p>
|
||||||
|
</CardContent>
|
||||||
|
</Card>
|
||||||
|
</div>
|
||||||
|
|
||||||
|
<PagesSourceCard key={`source-${projectId}`} projectId={projectId} />
|
||||||
|
<DeploymentHistory
|
||||||
|
key={`deployments-${projectId}`}
|
||||||
|
projectId={projectId}
|
||||||
|
activeDeploymentId={project.active_deployment_id}
|
||||||
|
/>
|
||||||
|
|
||||||
|
<ProjectEditorDialog
|
||||||
|
open={editorOpen}
|
||||||
|
onOpenChange={(nextOpen) => {
|
||||||
|
setEditorOpen(nextOpen);
|
||||||
|
if (!nextOpen) {
|
||||||
|
void queryClient.invalidateQueries({
|
||||||
|
queryKey: sourceQueryKey(projectId),
|
||||||
|
});
|
||||||
|
}
|
||||||
|
}}
|
||||||
|
project={project}
|
||||||
|
/>
|
||||||
|
<DeploymentUploadDialog
|
||||||
|
open={uploadOpen}
|
||||||
|
onOpenChange={setUploadOpen}
|
||||||
|
projectId={projectId}
|
||||||
|
rootDir={project.root_dir ?? ''}
|
||||||
|
entryFile={project.entry_file}
|
||||||
|
/>
|
||||||
|
|
||||||
|
<AlertDialog open={deleteProjectOpen} onOpenChange={setDeleteProjectOpen}>
|
||||||
|
<AlertDialogContent>
|
||||||
|
<AlertDialogHeader>
|
||||||
|
<AlertDialogTitle>删除 Pages 项目</AlertDialogTitle>
|
||||||
|
<AlertDialogDescription>
|
||||||
|
确认删除项目 {project.name} 吗?此操作不可恢复。
|
||||||
|
</AlertDialogDescription>
|
||||||
|
</AlertDialogHeader>
|
||||||
|
<AlertDialogFooter>
|
||||||
|
<AlertDialogCancel disabled={deleteProjectMutation.isPending}>
|
||||||
|
取消
|
||||||
|
</AlertDialogCancel>
|
||||||
|
<AlertDialogAction
|
||||||
|
disabled={deleteProjectMutation.isPending}
|
||||||
|
onClick={(event) => {
|
||||||
|
event.preventDefault();
|
||||||
|
deleteProjectMutation.mutate();
|
||||||
|
}}
|
||||||
|
>
|
||||||
|
{deleteProjectMutation.isPending ? (
|
||||||
|
<Spinner data-icon='inline-start' />
|
||||||
|
) : null}
|
||||||
|
确认删除
|
||||||
|
</AlertDialogAction>
|
||||||
|
</AlertDialogFooter>
|
||||||
|
</AlertDialogContent>
|
||||||
|
</AlertDialog>
|
||||||
|
</div>
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
export default function PagesDetailPage() {
|
export default function PagesDetailPage() {
|
||||||
return (
|
return (
|
||||||
<Suspense fallback={<PagesDetailPageFallback />}>
|
<Suspense fallback={<PagesDetailPageFallback />}>
|
||||||
<PagesDetailPageClient />
|
<PagesDetailRoute />
|
||||||
</Suspense>
|
</Suspense>
|
||||||
);
|
);
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -0,0 +1,66 @@
|
|||||||
|
import * as React from 'react';
|
||||||
|
import { cva, type VariantProps } from 'class-variance-authority';
|
||||||
|
|
||||||
|
import { cn } from '@/lib/utils';
|
||||||
|
|
||||||
|
const alertVariants = cva(
|
||||||
|
'relative grid w-full grid-cols-[0_1fr] items-start gap-y-0.5 rounded-lg border px-4 py-3 text-sm has-[>svg]:grid-cols-[calc(var(--spacing)*4)_1fr] has-[>svg]:gap-x-3 [&>svg]:size-4 [&>svg]:translate-y-0.5 [&>svg]:text-current',
|
||||||
|
{
|
||||||
|
variants: {
|
||||||
|
variant: {
|
||||||
|
default: 'bg-card text-card-foreground',
|
||||||
|
destructive:
|
||||||
|
'bg-card text-destructive *:data-[slot=alert-description]:text-destructive/90 [&>svg]:text-current',
|
||||||
|
},
|
||||||
|
},
|
||||||
|
defaultVariants: {
|
||||||
|
variant: 'default',
|
||||||
|
},
|
||||||
|
},
|
||||||
|
);
|
||||||
|
|
||||||
|
function Alert({
|
||||||
|
className,
|
||||||
|
variant,
|
||||||
|
...props
|
||||||
|
}: React.ComponentProps<'div'> & VariantProps<typeof alertVariants>) {
|
||||||
|
return (
|
||||||
|
<div
|
||||||
|
data-slot='alert'
|
||||||
|
role='alert'
|
||||||
|
className={cn(alertVariants({ variant }), className)}
|
||||||
|
{...props}
|
||||||
|
/>
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
|
function AlertTitle({ className, ...props }: React.ComponentProps<'div'>) {
|
||||||
|
return (
|
||||||
|
<div
|
||||||
|
data-slot='alert-title'
|
||||||
|
className={cn(
|
||||||
|
'col-start-2 line-clamp-1 min-h-4 font-medium tracking-tight',
|
||||||
|
className,
|
||||||
|
)}
|
||||||
|
{...props}
|
||||||
|
/>
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
|
function AlertDescription({
|
||||||
|
className,
|
||||||
|
...props
|
||||||
|
}: React.ComponentProps<'div'>) {
|
||||||
|
return (
|
||||||
|
<div
|
||||||
|
data-slot='alert-description'
|
||||||
|
className={cn(
|
||||||
|
'col-start-2 grid justify-items-start gap-1 text-sm text-muted-foreground [&_p]:leading-relaxed',
|
||||||
|
className,
|
||||||
|
)}
|
||||||
|
{...props}
|
||||||
|
/>
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
|
export { Alert, AlertDescription, AlertTitle };
|
||||||
@@ -251,6 +251,9 @@ export type {
|
|||||||
DnsAccountItem,
|
DnsAccountItem,
|
||||||
PagesProject,
|
PagesProject,
|
||||||
PagesDeployment,
|
PagesDeployment,
|
||||||
|
PagesSource,
|
||||||
|
PagesSourceActionReceipt,
|
||||||
|
PagesSourceStatus,
|
||||||
OriginItem,
|
OriginItem,
|
||||||
OriginDetail,
|
OriginDetail,
|
||||||
AccessLogList,
|
AccessLogList,
|
||||||
|
|||||||
@@ -109,8 +109,24 @@ export type {
|
|||||||
OriginMutationPayload,
|
OriginMutationPayload,
|
||||||
PagesDeployment,
|
PagesDeployment,
|
||||||
PagesDeploymentFile,
|
PagesDeploymentFile,
|
||||||
|
PagesGitHubLatestSourceUpdatePayload,
|
||||||
|
PagesGitHubReleaseSelector,
|
||||||
|
PagesGitHubReleaseSource,
|
||||||
|
PagesGitHubSourceUpdatePayload,
|
||||||
|
PagesGitHubTagSourceUpdatePayload,
|
||||||
|
PagesManualSource,
|
||||||
PagesProject,
|
PagesProject,
|
||||||
PagesProjectPayload,
|
PagesProjectPayload,
|
||||||
|
PagesRemoteNetworkPolicy,
|
||||||
|
PagesRemoteSourceUpdatePayload,
|
||||||
|
PagesRemoteURLSource,
|
||||||
|
PagesSource,
|
||||||
|
PagesSourceActionPayload,
|
||||||
|
PagesSourceActionReceipt,
|
||||||
|
PagesSourceRevision,
|
||||||
|
PagesSourceStatus,
|
||||||
|
PagesSourceUpdatePayload,
|
||||||
|
PagesSourceUpdateResult,
|
||||||
AcmeAccountItem,
|
AcmeAccountItem,
|
||||||
DnsAccountItem,
|
DnsAccountItem,
|
||||||
DnsAccountMutationPayload,
|
DnsAccountMutationPayload,
|
||||||
|
|||||||
@@ -12,6 +12,11 @@ import type {
|
|||||||
PagesDeploymentUploadPayload,
|
PagesDeploymentUploadPayload,
|
||||||
PagesProject,
|
PagesProject,
|
||||||
PagesProjectPayload,
|
PagesProjectPayload,
|
||||||
|
PagesSource,
|
||||||
|
PagesSourceActionPayload,
|
||||||
|
PagesSourceActionReceipt,
|
||||||
|
PagesSourceUpdatePayload,
|
||||||
|
PagesSourceUpdateResult,
|
||||||
} from './types';
|
} from './types';
|
||||||
|
|
||||||
export class PagesService extends OpenFlareBaseService {
|
export class PagesService extends OpenFlareBaseService {
|
||||||
@@ -40,6 +45,41 @@ export class PagesService extends OpenFlareBaseService {
|
|||||||
return this.post<void>(`/${id}/delete`);
|
return this.post<void>(`/${id}/delete`);
|
||||||
}
|
}
|
||||||
|
|
||||||
|
static getSource(projectId: number): Promise<PagesSource> {
|
||||||
|
return this.get<PagesSource>(`/${projectId}/source`);
|
||||||
|
}
|
||||||
|
|
||||||
|
static updateSource(
|
||||||
|
projectId: number,
|
||||||
|
payload: PagesSourceUpdatePayload,
|
||||||
|
): Promise<PagesSourceUpdateResult> {
|
||||||
|
return this.post<PagesSourceUpdateResult>(
|
||||||
|
`/${projectId}/source/update`,
|
||||||
|
payload,
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
|
static deleteSource(projectId: number): Promise<PagesSource> {
|
||||||
|
return this.post<PagesSource>(`/${projectId}/source/delete`);
|
||||||
|
}
|
||||||
|
|
||||||
|
static checkSource(projectId: number): Promise<PagesSourceActionReceipt> {
|
||||||
|
return this.post<PagesSourceActionReceipt>(
|
||||||
|
`/${projectId}/source/check`,
|
||||||
|
{},
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
|
static syncSource(
|
||||||
|
projectId: number,
|
||||||
|
payload: PagesSourceActionPayload = {},
|
||||||
|
): Promise<PagesSourceActionReceipt> {
|
||||||
|
return this.post<PagesSourceActionReceipt>(
|
||||||
|
`/${projectId}/source/sync`,
|
||||||
|
payload,
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
static listDeployments(projectId: number): Promise<PagesDeployment[]> {
|
static listDeployments(projectId: number): Promise<PagesDeployment[]> {
|
||||||
return this.get<PagesDeployment[]>(`/${projectId}/deployments`);
|
return this.get<PagesDeployment[]>(`/${projectId}/deployments`);
|
||||||
}
|
}
|
||||||
@@ -58,8 +98,6 @@ export class PagesService extends OpenFlareBaseService {
|
|||||||
): Promise<PagesDeployment> {
|
): Promise<PagesDeployment> {
|
||||||
const formData = new FormData();
|
const formData = new FormData();
|
||||||
formData.append('package', payload.file);
|
formData.append('package', payload.file);
|
||||||
formData.append('root_dir', payload.rootDir ?? '');
|
|
||||||
formData.append('entry_file', payload.entryFile ?? 'index.html');
|
|
||||||
|
|
||||||
return this.postFormData<PagesDeployment>(
|
return this.postFormData<PagesDeployment>(
|
||||||
`/${projectId}/deployments/upload`,
|
`/${projectId}/deployments/upload`,
|
||||||
@@ -75,6 +113,7 @@ export class PagesService extends OpenFlareBaseService {
|
|||||||
return this.post<PagesDeployment>(
|
return this.post<PagesDeployment>(
|
||||||
`/${projectId}/deployments/upload-from-url`,
|
`/${projectId}/deployments/upload-from-url`,
|
||||||
payload,
|
payload,
|
||||||
|
{ timeout: apiConfig.uploadTimeout } as InternalAxiosRequestConfig,
|
||||||
);
|
);
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|||||||
@@ -394,9 +394,11 @@ export interface PagesDeployment {
|
|||||||
status: 'uploaded' | 'active';
|
status: 'uploaded' | 'active';
|
||||||
file_count: number;
|
file_count: number;
|
||||||
total_size: number;
|
total_size: number;
|
||||||
root_dir?: string;
|
|
||||||
entry_file: string;
|
|
||||||
created_by: string;
|
created_by: string;
|
||||||
|
source_type: 'manual_upload' | 'manual_url' | 'remote_url' | 'github_release';
|
||||||
|
source_label: string;
|
||||||
|
trigger_type:
|
||||||
|
'manual_upload' | 'manual_url' | 'manual_sync' | 'scheduled_auto_update';
|
||||||
created_at: string;
|
created_at: string;
|
||||||
activated_at?: string | null;
|
activated_at?: string | null;
|
||||||
}
|
}
|
||||||
@@ -448,8 +450,6 @@ export interface PagesProjectPayload {
|
|||||||
|
|
||||||
export interface PagesDeploymentUploadPayload {
|
export interface PagesDeploymentUploadPayload {
|
||||||
file: File;
|
file: File;
|
||||||
rootDir?: string;
|
|
||||||
entryFile?: string;
|
|
||||||
onProgress?: (percent: number) => void;
|
onProgress?: (percent: number) => void;
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -457,6 +457,124 @@ export interface PagesDeploymentUploadFromURLPayload {
|
|||||||
url: string;
|
url: string;
|
||||||
}
|
}
|
||||||
|
|
||||||
|
export type PagesSourceStatus =
|
||||||
|
'idle' | 'checking' | 'update_available' | 'syncing' | 'failed' | 'attention';
|
||||||
|
|
||||||
|
export type PagesRemoteNetworkPolicy = 'public' | 'trusted_internal';
|
||||||
|
|
||||||
|
export type PagesGitHubReleaseSelector = 'latest' | 'tag';
|
||||||
|
|
||||||
|
export interface PagesSourceRevision {
|
||||||
|
revision: string;
|
||||||
|
label: string;
|
||||||
|
asset_name?: string;
|
||||||
|
}
|
||||||
|
|
||||||
|
interface PagesSourceRuntimeView {
|
||||||
|
sync_status?: PagesSourceStatus;
|
||||||
|
update_available?: boolean;
|
||||||
|
last_seen?: PagesSourceRevision;
|
||||||
|
last_applied?: PagesSourceRevision;
|
||||||
|
last_checked_at?: string | null;
|
||||||
|
last_synced_at?: string | null;
|
||||||
|
next_check_at?: string | null;
|
||||||
|
last_error?: string;
|
||||||
|
}
|
||||||
|
|
||||||
|
export interface PagesManualSource {
|
||||||
|
source_type: 'manual';
|
||||||
|
}
|
||||||
|
|
||||||
|
export interface PagesRemoteURLSource extends PagesSourceRuntimeView {
|
||||||
|
source_type: 'remote_url';
|
||||||
|
has_remote_url: boolean;
|
||||||
|
display_url: string;
|
||||||
|
remote_network_policy: PagesRemoteNetworkPolicy;
|
||||||
|
}
|
||||||
|
|
||||||
|
interface PagesGitHubReleaseSourceBase extends PagesSourceRuntimeView {
|
||||||
|
source_type: 'github_release';
|
||||||
|
github_repository: string;
|
||||||
|
asset_name: string;
|
||||||
|
}
|
||||||
|
|
||||||
|
interface PagesGitHubLatestReleaseSource extends PagesGitHubReleaseSourceBase {
|
||||||
|
release_selector: 'latest';
|
||||||
|
release_tag?: '';
|
||||||
|
auto_update_enabled: boolean;
|
||||||
|
check_interval_minutes: number;
|
||||||
|
}
|
||||||
|
|
||||||
|
interface PagesGitHubTagReleaseSource extends PagesGitHubReleaseSourceBase {
|
||||||
|
release_selector: 'tag';
|
||||||
|
release_tag: string;
|
||||||
|
auto_update_enabled: false;
|
||||||
|
check_interval_minutes?: 0;
|
||||||
|
}
|
||||||
|
|
||||||
|
export type PagesGitHubReleaseSource =
|
||||||
|
PagesGitHubLatestReleaseSource | PagesGitHubTagReleaseSource;
|
||||||
|
|
||||||
|
/**
|
||||||
|
* 部署源使用判别联合,后续仓库构建来源只需增加独立 git_repository variant,
|
||||||
|
* 不需要向 Remote 或 GitHub Release 填入构建字段。
|
||||||
|
*/
|
||||||
|
export type PagesSource =
|
||||||
|
PagesManualSource | PagesRemoteURLSource | PagesGitHubReleaseSource;
|
||||||
|
|
||||||
|
export interface PagesRemoteSourceUpdatePayload {
|
||||||
|
source_type: 'remote_url';
|
||||||
|
remote_url_set: boolean;
|
||||||
|
remote_url: string;
|
||||||
|
remote_network_policy: PagesRemoteNetworkPolicy;
|
||||||
|
}
|
||||||
|
|
||||||
|
interface PagesGitHubSourceUpdateBase {
|
||||||
|
source_type: 'github_release';
|
||||||
|
repository_url: string;
|
||||||
|
asset_name: string;
|
||||||
|
}
|
||||||
|
|
||||||
|
export interface PagesGitHubLatestSourceUpdatePayload extends PagesGitHubSourceUpdateBase {
|
||||||
|
release_selector: 'latest';
|
||||||
|
release_tag: '';
|
||||||
|
auto_update_enabled: boolean;
|
||||||
|
check_interval_minutes: number;
|
||||||
|
}
|
||||||
|
|
||||||
|
export interface PagesGitHubTagSourceUpdatePayload extends PagesGitHubSourceUpdateBase {
|
||||||
|
release_selector: 'tag';
|
||||||
|
release_tag: string;
|
||||||
|
auto_update_enabled: false;
|
||||||
|
check_interval_minutes: 0;
|
||||||
|
}
|
||||||
|
|
||||||
|
export type PagesGitHubSourceUpdatePayload =
|
||||||
|
PagesGitHubLatestSourceUpdatePayload | PagesGitHubTagSourceUpdatePayload;
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Source 更新请求保持 Provider 判别联合;未来仓库拉取构建使用独立 git_repository variant,
|
||||||
|
* 不向 Remote URL 或 GitHub Release payload 混入构建字段。
|
||||||
|
*/
|
||||||
|
export type PagesSourceUpdatePayload =
|
||||||
|
PagesRemoteSourceUpdatePayload | PagesGitHubSourceUpdatePayload;
|
||||||
|
|
||||||
|
export interface PagesSourceActionPayload {
|
||||||
|
confirmed_revision?: string;
|
||||||
|
}
|
||||||
|
|
||||||
|
export interface PagesSourceActionReceipt {
|
||||||
|
task_id: string;
|
||||||
|
execution_id: string;
|
||||||
|
action: 'check' | 'sync';
|
||||||
|
}
|
||||||
|
|
||||||
|
export interface PagesSourceUpdateResult {
|
||||||
|
source: PagesSource;
|
||||||
|
check_task: PagesSourceActionReceipt | null;
|
||||||
|
warning: string;
|
||||||
|
}
|
||||||
|
|
||||||
// ==================== Origins ====================
|
// ==================== Origins ====================
|
||||||
|
|
||||||
export interface OriginItem {
|
export interface OriginItem {
|
||||||
|
|||||||
@@ -2,27 +2,34 @@ import { beforeEach, describe, expect, it, vi } from 'vitest';
|
|||||||
import type { AxiosResponse } from 'axios';
|
import type { AxiosResponse } from 'axios';
|
||||||
|
|
||||||
import apiClient from '@/lib/services/core/api-client';
|
import apiClient from '@/lib/services/core/api-client';
|
||||||
|
import { apiConfig } from '@/lib/services/core/config';
|
||||||
import { PagesService } from '@/lib/services/openflare/pages.service';
|
import { PagesService } from '@/lib/services/openflare/pages.service';
|
||||||
|
|
||||||
vi.mock('@/lib/services/core/api-client', () => ({
|
vi.mock('@/lib/services/core/api-client', () => ({
|
||||||
default: {
|
default: {
|
||||||
get: vi.fn(),
|
get: vi.fn(),
|
||||||
|
post: vi.fn(),
|
||||||
},
|
},
|
||||||
}));
|
}));
|
||||||
|
|
||||||
|
function response<T>(data: T) {
|
||||||
|
return {
|
||||||
|
data: { error_msg: '', data },
|
||||||
|
status: 200,
|
||||||
|
statusText: 'OK',
|
||||||
|
headers: {},
|
||||||
|
config: { headers: {} },
|
||||||
|
} as AxiosResponse;
|
||||||
|
}
|
||||||
|
|
||||||
describe('PagesService', () => {
|
describe('PagesService', () => {
|
||||||
beforeEach(() => {
|
beforeEach(() => {
|
||||||
vi.mocked(apiClient.get).mockReset();
|
vi.mocked(apiClient.get).mockReset();
|
||||||
|
vi.mocked(apiClient.post).mockReset();
|
||||||
});
|
});
|
||||||
|
|
||||||
it('requests deployment files using the backend deployment route', async () => {
|
it('requests deployment files using the backend deployment route', async () => {
|
||||||
vi.mocked(apiClient.get).mockResolvedValue({
|
vi.mocked(apiClient.get).mockResolvedValue(response([]));
|
||||||
data: { error_msg: '', data: [] },
|
|
||||||
status: 200,
|
|
||||||
statusText: 'OK',
|
|
||||||
headers: {},
|
|
||||||
config: { headers: {} },
|
|
||||||
} as AxiosResponse);
|
|
||||||
|
|
||||||
await PagesService.listDeploymentFiles(7);
|
await PagesService.listDeploymentFiles(7);
|
||||||
|
|
||||||
@@ -31,4 +38,156 @@ describe('PagesService', () => {
|
|||||||
expect.objectContaining({ params: undefined }),
|
expect.objectContaining({ params: undefined }),
|
||||||
);
|
);
|
||||||
});
|
});
|
||||||
|
|
||||||
|
it('connects all source endpoints with explicit action payloads', async () => {
|
||||||
|
vi.mocked(apiClient.get).mockResolvedValue(
|
||||||
|
response({ source_type: 'manual' }),
|
||||||
|
);
|
||||||
|
vi.mocked(apiClient.post).mockResolvedValue(
|
||||||
|
response({ source_type: 'manual' }),
|
||||||
|
);
|
||||||
|
|
||||||
|
await PagesService.getSource(12);
|
||||||
|
await PagesService.updateSource(12, {
|
||||||
|
source_type: 'remote_url',
|
||||||
|
remote_url_set: true,
|
||||||
|
remote_url: 'https://example.com/site.zip?token=secret',
|
||||||
|
remote_network_policy: 'public',
|
||||||
|
});
|
||||||
|
await PagesService.deleteSource(12);
|
||||||
|
await PagesService.checkSource(12);
|
||||||
|
await PagesService.syncSource(12);
|
||||||
|
|
||||||
|
expect(apiClient.get).toHaveBeenCalledWith(
|
||||||
|
'/api/v1/d/pages/12/source',
|
||||||
|
expect.objectContaining({ params: undefined }),
|
||||||
|
);
|
||||||
|
expect(apiClient.post).toHaveBeenNthCalledWith(
|
||||||
|
1,
|
||||||
|
'/api/v1/d/pages/12/source/update',
|
||||||
|
expect.objectContaining({
|
||||||
|
source_type: 'remote_url',
|
||||||
|
remote_url_set: true,
|
||||||
|
}),
|
||||||
|
undefined,
|
||||||
|
);
|
||||||
|
expect(apiClient.post).toHaveBeenNthCalledWith(
|
||||||
|
2,
|
||||||
|
'/api/v1/d/pages/12/source/delete',
|
||||||
|
undefined,
|
||||||
|
undefined,
|
||||||
|
);
|
||||||
|
expect(apiClient.post).toHaveBeenNthCalledWith(
|
||||||
|
3,
|
||||||
|
'/api/v1/d/pages/12/source/check',
|
||||||
|
{},
|
||||||
|
undefined,
|
||||||
|
);
|
||||||
|
expect(apiClient.post).toHaveBeenNthCalledWith(
|
||||||
|
4,
|
||||||
|
'/api/v1/d/pages/12/source/sync',
|
||||||
|
{},
|
||||||
|
undefined,
|
||||||
|
);
|
||||||
|
});
|
||||||
|
|
||||||
|
it('sends the complete GitHub latest discriminator payload', async () => {
|
||||||
|
vi.mocked(apiClient.post).mockResolvedValue(
|
||||||
|
response({
|
||||||
|
source: { source_type: 'manual' },
|
||||||
|
check_task: null,
|
||||||
|
warning: '',
|
||||||
|
}),
|
||||||
|
);
|
||||||
|
|
||||||
|
await PagesService.updateSource(12, {
|
||||||
|
source_type: 'github_release',
|
||||||
|
repository_url: 'https://github.com/openflare/site',
|
||||||
|
release_selector: 'latest',
|
||||||
|
release_tag: '',
|
||||||
|
asset_name: 'dist.zip',
|
||||||
|
auto_update_enabled: false,
|
||||||
|
check_interval_minutes: 60,
|
||||||
|
});
|
||||||
|
|
||||||
|
expect(apiClient.post).toHaveBeenCalledWith(
|
||||||
|
'/api/v1/d/pages/12/source/update',
|
||||||
|
{
|
||||||
|
source_type: 'github_release',
|
||||||
|
repository_url: 'https://github.com/openflare/site',
|
||||||
|
release_selector: 'latest',
|
||||||
|
release_tag: '',
|
||||||
|
asset_name: 'dist.zip',
|
||||||
|
auto_update_enabled: false,
|
||||||
|
check_interval_minutes: 60,
|
||||||
|
},
|
||||||
|
undefined,
|
||||||
|
);
|
||||||
|
});
|
||||||
|
|
||||||
|
it('sends safe disabled defaults with the GitHub tag discriminator', async () => {
|
||||||
|
vi.mocked(apiClient.post).mockResolvedValue(
|
||||||
|
response({
|
||||||
|
source: { source_type: 'manual' },
|
||||||
|
check_task: null,
|
||||||
|
warning: '',
|
||||||
|
}),
|
||||||
|
);
|
||||||
|
|
||||||
|
await PagesService.updateSource(12, {
|
||||||
|
source_type: 'github_release',
|
||||||
|
repository_url: 'https://github.com/openflare/site',
|
||||||
|
release_selector: 'tag',
|
||||||
|
release_tag: 'v1.2.3',
|
||||||
|
asset_name: 'site.tar.gz',
|
||||||
|
auto_update_enabled: false,
|
||||||
|
check_interval_minutes: 0,
|
||||||
|
});
|
||||||
|
|
||||||
|
expect(apiClient.post).toHaveBeenCalledWith(
|
||||||
|
'/api/v1/d/pages/12/source/update',
|
||||||
|
{
|
||||||
|
source_type: 'github_release',
|
||||||
|
repository_url: 'https://github.com/openflare/site',
|
||||||
|
release_selector: 'tag',
|
||||||
|
release_tag: 'v1.2.3',
|
||||||
|
asset_name: 'site.tar.gz',
|
||||||
|
auto_update_enabled: false,
|
||||||
|
check_interval_minutes: 0,
|
||||||
|
},
|
||||||
|
undefined,
|
||||||
|
);
|
||||||
|
});
|
||||||
|
|
||||||
|
it('uploads only the package multipart field', async () => {
|
||||||
|
vi.mocked(apiClient.post).mockResolvedValue(response({}));
|
||||||
|
const file = new File(['site'], 'site.zip', {
|
||||||
|
type: 'application/zip',
|
||||||
|
});
|
||||||
|
|
||||||
|
await PagesService.uploadDeployment(8, { file });
|
||||||
|
|
||||||
|
const formData = vi.mocked(apiClient.post).mock.calls[0]?.[1];
|
||||||
|
expect(formData).toBeInstanceOf(FormData);
|
||||||
|
expect(Array.from((formData as FormData).keys())).toEqual(['package']);
|
||||||
|
expect(apiClient.post).toHaveBeenCalledWith(
|
||||||
|
'/api/v1/d/pages/8/deployments/upload',
|
||||||
|
formData,
|
||||||
|
expect.objectContaining({ timeout: apiConfig.uploadTimeout }),
|
||||||
|
);
|
||||||
|
});
|
||||||
|
|
||||||
|
it('keeps the compatibility URL import on the long upload timeout', async () => {
|
||||||
|
vi.mocked(apiClient.post).mockResolvedValue(response({}));
|
||||||
|
|
||||||
|
await PagesService.uploadDeploymentFromURL(8, {
|
||||||
|
url: 'https://example.com/site.zip',
|
||||||
|
});
|
||||||
|
|
||||||
|
expect(apiClient.post).toHaveBeenCalledWith(
|
||||||
|
'/api/v1/d/pages/8/deployments/upload-from-url',
|
||||||
|
{ url: 'https://example.com/site.zip' },
|
||||||
|
expect.objectContaining({ timeout: apiConfig.uploadTimeout }),
|
||||||
|
);
|
||||||
|
});
|
||||||
});
|
});
|
||||||
|
|||||||
@@ -0,0 +1,355 @@
|
|||||||
|
import { act } from 'react';
|
||||||
|
import { QueryClient, QueryClientProvider } from '@tanstack/react-query';
|
||||||
|
import { render, screen, waitFor, within } from '@testing-library/react';
|
||||||
|
import userEvent from '@testing-library/user-event';
|
||||||
|
import { beforeEach, describe, expect, it, vi } from 'vitest';
|
||||||
|
|
||||||
|
import { DeploymentHistory } from '@/app/(main)/pages/detail/components/deployment-history';
|
||||||
|
import {
|
||||||
|
getLatestSourceIdlePollingDecision,
|
||||||
|
PagesSourceCard,
|
||||||
|
} from '@/app/(main)/pages/detail/components/pages-source-card';
|
||||||
|
import {
|
||||||
|
deploymentsQueryKey,
|
||||||
|
projectQueryKey,
|
||||||
|
projectsQueryKey,
|
||||||
|
sourceQueryKey,
|
||||||
|
} from '@/app/(main)/pages/components/pages-utils';
|
||||||
|
import {
|
||||||
|
type PagesDeployment,
|
||||||
|
type PagesGitHubReleaseSource,
|
||||||
|
PagesService,
|
||||||
|
} from '@/lib/services/openflare';
|
||||||
|
|
||||||
|
vi.mock('@/lib/services/openflare', async (importOriginal) => {
|
||||||
|
const actual =
|
||||||
|
await importOriginal<typeof import('@/lib/services/openflare')>();
|
||||||
|
return {
|
||||||
|
...actual,
|
||||||
|
PagesService: {
|
||||||
|
getSource: vi.fn(),
|
||||||
|
updateSource: vi.fn(),
|
||||||
|
deleteSource: vi.fn(),
|
||||||
|
checkSource: vi.fn(),
|
||||||
|
syncSource: vi.fn(),
|
||||||
|
listDeployments: vi.fn(),
|
||||||
|
listDeploymentFiles: vi.fn(),
|
||||||
|
activateDeployment: vi.fn(),
|
||||||
|
deleteDeployment: vi.fn(),
|
||||||
|
},
|
||||||
|
};
|
||||||
|
});
|
||||||
|
|
||||||
|
function renderWithQuery(ui: React.ReactNode) {
|
||||||
|
const queryClient = new QueryClient({
|
||||||
|
defaultOptions: {
|
||||||
|
queries: { retry: false, gcTime: 0 },
|
||||||
|
mutations: { retry: false },
|
||||||
|
},
|
||||||
|
});
|
||||||
|
const rendered = render(
|
||||||
|
<QueryClientProvider client={queryClient}>{ui}</QueryClientProvider>,
|
||||||
|
);
|
||||||
|
return {
|
||||||
|
...rendered,
|
||||||
|
queryClient,
|
||||||
|
rerenderWithQuery: (nextUI: React.ReactNode) =>
|
||||||
|
rendered.rerender(
|
||||||
|
<QueryClientProvider client={queryClient}>
|
||||||
|
{nextUI}
|
||||||
|
</QueryClientProvider>,
|
||||||
|
),
|
||||||
|
};
|
||||||
|
}
|
||||||
|
|
||||||
|
const latestSource: PagesGitHubReleaseSource = {
|
||||||
|
source_type: 'github_release',
|
||||||
|
github_repository: 'openflare/site',
|
||||||
|
release_selector: 'latest',
|
||||||
|
asset_name: 'dist.zip',
|
||||||
|
auto_update_enabled: true,
|
||||||
|
check_interval_minutes: 120,
|
||||||
|
sync_status: 'idle',
|
||||||
|
update_available: false,
|
||||||
|
last_seen: {
|
||||||
|
revision: 'b'.repeat(64),
|
||||||
|
label: 'v1.2.3',
|
||||||
|
asset_name: 'dist.zip',
|
||||||
|
},
|
||||||
|
last_applied: {
|
||||||
|
revision: 'a'.repeat(64),
|
||||||
|
label: 'v1.2.2',
|
||||||
|
asset_name: 'dist.zip',
|
||||||
|
},
|
||||||
|
last_checked_at: '2026-07-19T10:00:00Z',
|
||||||
|
last_synced_at: '2026-07-19T09:00:00Z',
|
||||||
|
next_check_at: '2026-07-19T12:00:00Z',
|
||||||
|
last_error: '',
|
||||||
|
};
|
||||||
|
|
||||||
|
describe('Pages latest source automatic updates', () => {
|
||||||
|
beforeEach(() => {
|
||||||
|
vi.mocked(PagesService.getSource).mockReset();
|
||||||
|
vi.mocked(PagesService.updateSource).mockReset();
|
||||||
|
vi.mocked(PagesService.deleteSource).mockReset();
|
||||||
|
vi.mocked(PagesService.checkSource).mockReset();
|
||||||
|
vi.mocked(PagesService.syncSource).mockReset();
|
||||||
|
vi.mocked(PagesService.listDeployments).mockReset();
|
||||||
|
vi.mocked(PagesService.listDeploymentFiles).mockReset();
|
||||||
|
vi.mocked(PagesService.activateDeployment).mockReset();
|
||||||
|
vi.mocked(PagesService.deleteDeployment).mockReset();
|
||||||
|
});
|
||||||
|
|
||||||
|
it('backfills latest settings and hides them after selecting a fixed tag', async () => {
|
||||||
|
const user = userEvent.setup();
|
||||||
|
vi.mocked(PagesService.getSource).mockResolvedValue(latestSource);
|
||||||
|
|
||||||
|
renderWithQuery(<PagesSourceCard projectId={9} />);
|
||||||
|
|
||||||
|
await user.click(await screen.findByRole('button', { name: '编辑来源' }));
|
||||||
|
expect(screen.getByRole('switch', { name: '自动更新' })).toBeChecked();
|
||||||
|
expect(screen.getByLabelText('检查间隔(分钟)')).toHaveValue(120);
|
||||||
|
|
||||||
|
await user.click(screen.getByRole('radio', { name: '固定 Tag' }));
|
||||||
|
expect(
|
||||||
|
screen.queryByRole('switch', { name: '自动更新' }),
|
||||||
|
).not.toBeInTheDocument();
|
||||||
|
expect(screen.queryByLabelText('检查间隔(分钟)')).not.toBeInTheDocument();
|
||||||
|
});
|
||||||
|
|
||||||
|
it('does not show automatic schedule details for a fixed tag source', async () => {
|
||||||
|
vi.mocked(PagesService.getSource).mockResolvedValue({
|
||||||
|
...latestSource,
|
||||||
|
release_selector: 'tag',
|
||||||
|
release_tag: 'v1.2.3',
|
||||||
|
auto_update_enabled: false,
|
||||||
|
check_interval_minutes: 0,
|
||||||
|
next_check_at: null,
|
||||||
|
});
|
||||||
|
|
||||||
|
renderWithQuery(<PagesSourceCard projectId={9} />);
|
||||||
|
|
||||||
|
expect(await screen.findByText('固定 Tag · v1.2.3')).toBeVisible();
|
||||||
|
expect(screen.queryByText('自动更新')).not.toBeInTheDocument();
|
||||||
|
expect(screen.queryByText('检查间隔')).not.toBeInTheDocument();
|
||||||
|
expect(screen.queryByText('下次检查时间')).not.toBeInTheDocument();
|
||||||
|
});
|
||||||
|
|
||||||
|
it('rejects a latest interval outside 5–1440 minutes', async () => {
|
||||||
|
const user = userEvent.setup();
|
||||||
|
vi.mocked(PagesService.getSource).mockResolvedValue(latestSource);
|
||||||
|
|
||||||
|
renderWithQuery(<PagesSourceCard projectId={9} />);
|
||||||
|
|
||||||
|
await user.click(await screen.findByRole('button', { name: '编辑来源' }));
|
||||||
|
const intervalInput = screen.getByLabelText('检查间隔(分钟)');
|
||||||
|
await user.clear(intervalInput);
|
||||||
|
await user.type(intervalInput, '4');
|
||||||
|
await user.click(screen.getByRole('button', { name: '保存 GitHub 来源' }));
|
||||||
|
|
||||||
|
expect(screen.getByRole('alert')).toHaveTextContent(
|
||||||
|
'检查间隔须为 5–1440 分钟的整数',
|
||||||
|
);
|
||||||
|
expect(intervalInput).toHaveAttribute(
|
||||||
|
'aria-describedby',
|
||||||
|
'pages-github-check-interval-description pages-github-check-interval-error',
|
||||||
|
);
|
||||||
|
expect(PagesService.updateSource).not.toHaveBeenCalled();
|
||||||
|
});
|
||||||
|
|
||||||
|
it('keeps an unsaved source draft when idle polling updates runtime fields', async () => {
|
||||||
|
const user = userEvent.setup();
|
||||||
|
vi.mocked(PagesService.getSource).mockResolvedValue(latestSource);
|
||||||
|
const { queryClient } = renderWithQuery(<PagesSourceCard projectId={9} />);
|
||||||
|
|
||||||
|
await user.click(await screen.findByRole('button', { name: '编辑来源' }));
|
||||||
|
const assetInput = screen.getByLabelText('Release Asset 文件名');
|
||||||
|
const intervalInput = screen.getByLabelText('检查间隔(分钟)');
|
||||||
|
await user.clear(assetInput);
|
||||||
|
await user.type(assetInput, 'draft.zip');
|
||||||
|
await user.clear(intervalInput);
|
||||||
|
await user.type(intervalInput, '30');
|
||||||
|
await user.click(screen.getByRole('switch', { name: '自动更新' }));
|
||||||
|
|
||||||
|
act(() => {
|
||||||
|
queryClient.setQueryData(sourceQueryKey(9), {
|
||||||
|
...latestSource,
|
||||||
|
last_checked_at: '2026-07-19T10:30:00Z',
|
||||||
|
next_check_at: '2026-07-19T12:30:00Z',
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
|
await waitFor(() => {
|
||||||
|
expect(assetInput).toHaveValue('draft.zip');
|
||||||
|
expect(intervalInput).toHaveValue(30);
|
||||||
|
expect(
|
||||||
|
screen.getByRole('switch', { name: '自动更新' }),
|
||||||
|
).not.toBeChecked();
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
|
it('remounts project-scoped source state when the route project changes', async () => {
|
||||||
|
const user = userEvent.setup();
|
||||||
|
vi.mocked(PagesService.getSource).mockImplementation(async (projectId) => ({
|
||||||
|
...latestSource,
|
||||||
|
github_repository:
|
||||||
|
projectId === 9 ? 'openflare/project-nine' : 'openflare/project-ten',
|
||||||
|
}));
|
||||||
|
const { rerenderWithQuery } = renderWithQuery(
|
||||||
|
<PagesSourceCard key='source-9' projectId={9} />,
|
||||||
|
);
|
||||||
|
|
||||||
|
await user.click(await screen.findByRole('button', { name: '编辑来源' }));
|
||||||
|
expect(screen.getByRole('dialog')).toBeVisible();
|
||||||
|
|
||||||
|
rerenderWithQuery(<PagesSourceCard key='source-10' projectId={10} />);
|
||||||
|
|
||||||
|
expect(screen.queryByRole('dialog')).not.toBeInTheDocument();
|
||||||
|
expect(await screen.findByText('openflare/project-ten')).toBeVisible();
|
||||||
|
});
|
||||||
|
|
||||||
|
it('uses low-frequency, near-due and bounded overdue polling', () => {
|
||||||
|
const now = Date.parse('2026-07-19T10:00:00Z');
|
||||||
|
const farSource = {
|
||||||
|
...latestSource,
|
||||||
|
next_check_at: new Date(now + 60 * 60 * 1_000).toISOString(),
|
||||||
|
};
|
||||||
|
expect(getLatestSourceIdlePollingDecision(farSource, now, null)).toEqual({
|
||||||
|
interval: 5 * 60 * 1_000,
|
||||||
|
overdueWindow: null,
|
||||||
|
});
|
||||||
|
|
||||||
|
const nearSource = {
|
||||||
|
...latestSource,
|
||||||
|
next_check_at: new Date(now + 60_000).toISOString(),
|
||||||
|
};
|
||||||
|
expect(getLatestSourceIdlePollingDecision(nearSource, now, null)).toEqual({
|
||||||
|
interval: 30_000,
|
||||||
|
overdueWindow: null,
|
||||||
|
});
|
||||||
|
|
||||||
|
const overdueSource = {
|
||||||
|
...latestSource,
|
||||||
|
next_check_at: new Date(now - 1).toISOString(),
|
||||||
|
};
|
||||||
|
const overdue = getLatestSourceIdlePollingDecision(
|
||||||
|
overdueSource,
|
||||||
|
now,
|
||||||
|
null,
|
||||||
|
);
|
||||||
|
expect(overdue.interval).toBe(30_000);
|
||||||
|
expect(
|
||||||
|
getLatestSourceIdlePollingDecision(
|
||||||
|
overdueSource,
|
||||||
|
now + 10 * 60 * 1_000,
|
||||||
|
overdue.overdueWindow,
|
||||||
|
).interval,
|
||||||
|
).toBe(false);
|
||||||
|
|
||||||
|
expect(
|
||||||
|
getLatestSourceIdlePollingDecision(
|
||||||
|
{
|
||||||
|
...latestSource,
|
||||||
|
release_selector: 'tag',
|
||||||
|
release_tag: 'v1.2.3',
|
||||||
|
auto_update_enabled: false,
|
||||||
|
check_interval_minutes: 0,
|
||||||
|
},
|
||||||
|
now,
|
||||||
|
null,
|
||||||
|
).interval,
|
||||||
|
).toBe(false);
|
||||||
|
});
|
||||||
|
|
||||||
|
it('refreshes source consumers after a background deployment is applied', async () => {
|
||||||
|
vi.mocked(PagesService.getSource).mockResolvedValue(latestSource);
|
||||||
|
const { queryClient } = renderWithQuery(<PagesSourceCard projectId={9} />);
|
||||||
|
expect(await screen.findByText('openflare/site')).toBeVisible();
|
||||||
|
|
||||||
|
const invalidateQueries = vi.spyOn(queryClient, 'invalidateQueries');
|
||||||
|
const updatedSource: PagesGitHubReleaseSource = {
|
||||||
|
...latestSource,
|
||||||
|
last_applied: {
|
||||||
|
revision: 'b'.repeat(64),
|
||||||
|
label: 'v1.2.3',
|
||||||
|
asset_name: 'dist.zip',
|
||||||
|
},
|
||||||
|
last_synced_at: '2026-07-19T10:30:00Z',
|
||||||
|
};
|
||||||
|
vi.mocked(PagesService.getSource).mockResolvedValue(updatedSource);
|
||||||
|
|
||||||
|
act(() => {
|
||||||
|
queryClient.setQueryData(sourceQueryKey(9), updatedSource);
|
||||||
|
});
|
||||||
|
|
||||||
|
await waitFor(() => {
|
||||||
|
expect(invalidateQueries).toHaveBeenCalledWith({
|
||||||
|
queryKey: sourceQueryKey(9),
|
||||||
|
});
|
||||||
|
expect(invalidateQueries).toHaveBeenCalledWith({
|
||||||
|
queryKey: projectQueryKey(9),
|
||||||
|
});
|
||||||
|
expect(invalidateQueries).toHaveBeenCalledWith({
|
||||||
|
queryKey: deploymentsQueryKey(9),
|
||||||
|
});
|
||||||
|
expect(invalidateQueries).toHaveBeenCalledWith({
|
||||||
|
queryKey: ['openflare', 'pages', 'deployment-files', 9],
|
||||||
|
});
|
||||||
|
expect(invalidateQueries).toHaveBeenCalledWith({
|
||||||
|
queryKey: projectsQueryKey,
|
||||||
|
});
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
|
it('warns that rollback closes auto update and refreshes four query groups', async () => {
|
||||||
|
const user = userEvent.setup();
|
||||||
|
const deployment: PagesDeployment = {
|
||||||
|
id: 51,
|
||||||
|
project_id: 9,
|
||||||
|
deployment_number: 7,
|
||||||
|
checksum: 'd'.repeat(64),
|
||||||
|
status: 'uploaded',
|
||||||
|
file_count: 12,
|
||||||
|
total_size: 4_096,
|
||||||
|
created_by: 'user:1',
|
||||||
|
source_type: 'github_release',
|
||||||
|
source_label: 'v1.2.2',
|
||||||
|
trigger_type: 'scheduled_auto_update',
|
||||||
|
created_at: '2026-07-19T10:00:00Z',
|
||||||
|
activated_at: null,
|
||||||
|
};
|
||||||
|
vi.mocked(PagesService.listDeployments).mockResolvedValue([deployment]);
|
||||||
|
vi.mocked(PagesService.activateDeployment).mockResolvedValue({} as never);
|
||||||
|
|
||||||
|
const { queryClient } = renderWithQuery(
|
||||||
|
<DeploymentHistory projectId={9} activeDeploymentId={52} />,
|
||||||
|
);
|
||||||
|
await screen.findByText('GitHub · v1.2.2 · 定时更新');
|
||||||
|
const invalidateQueries = vi.spyOn(queryClient, 'invalidateQueries');
|
||||||
|
|
||||||
|
await user.click(screen.getByRole('button', { name: '激活' }));
|
||||||
|
const dialog = screen.getByRole('alertdialog');
|
||||||
|
expect(
|
||||||
|
within(dialog).getByText(
|
||||||
|
'激活其它历史部署会终止当前来源任务;若已开启自动更新,将同时关闭自动更新。',
|
||||||
|
),
|
||||||
|
).toBeVisible();
|
||||||
|
await user.click(within(dialog).getByRole('button', { name: '确认' }));
|
||||||
|
|
||||||
|
await waitFor(() => {
|
||||||
|
expect(PagesService.activateDeployment).toHaveBeenCalledWith(9, 51);
|
||||||
|
expect(invalidateQueries).toHaveBeenCalledWith({
|
||||||
|
queryKey: deploymentsQueryKey(9),
|
||||||
|
});
|
||||||
|
expect(invalidateQueries).toHaveBeenCalledWith({
|
||||||
|
queryKey: projectQueryKey(9),
|
||||||
|
});
|
||||||
|
expect(invalidateQueries).toHaveBeenCalledWith({
|
||||||
|
queryKey: sourceQueryKey(9),
|
||||||
|
});
|
||||||
|
expect(invalidateQueries).toHaveBeenCalledWith({
|
||||||
|
queryKey: projectsQueryKey,
|
||||||
|
});
|
||||||
|
});
|
||||||
|
});
|
||||||
|
});
|
||||||
@@ -0,0 +1,663 @@
|
|||||||
|
import { QueryClient, QueryClientProvider } from '@tanstack/react-query';
|
||||||
|
import { render, screen, waitFor, within } from '@testing-library/react';
|
||||||
|
import userEvent from '@testing-library/user-event';
|
||||||
|
import { beforeEach, describe, expect, it, vi } from 'vitest';
|
||||||
|
|
||||||
|
import { DeploymentUploadDialog } from '@/app/(main)/pages/components/deployment-upload-dialog';
|
||||||
|
import { DeploymentHistory } from '@/app/(main)/pages/detail/components/deployment-history';
|
||||||
|
import { PagesSourceCard } from '@/app/(main)/pages/detail/components/pages-source-card';
|
||||||
|
import {
|
||||||
|
AdminTaskService,
|
||||||
|
type TaskExecution,
|
||||||
|
type TaskExecutionStatus,
|
||||||
|
} from '@/lib/services/admin';
|
||||||
|
import {
|
||||||
|
type PagesDeployment,
|
||||||
|
type PagesGitHubReleaseSource,
|
||||||
|
type PagesRemoteURLSource,
|
||||||
|
PagesService,
|
||||||
|
} from '@/lib/services/openflare';
|
||||||
|
|
||||||
|
vi.mock('@/lib/services/openflare', async (importOriginal) => {
|
||||||
|
const actual =
|
||||||
|
await importOriginal<typeof import('@/lib/services/openflare')>();
|
||||||
|
return {
|
||||||
|
...actual,
|
||||||
|
PagesService: {
|
||||||
|
getSource: vi.fn(),
|
||||||
|
updateSource: vi.fn(),
|
||||||
|
deleteSource: vi.fn(),
|
||||||
|
checkSource: vi.fn(),
|
||||||
|
syncSource: vi.fn(),
|
||||||
|
listDeployments: vi.fn(),
|
||||||
|
listDeploymentFiles: vi.fn(),
|
||||||
|
activateDeployment: vi.fn(),
|
||||||
|
deleteDeployment: vi.fn(),
|
||||||
|
uploadDeployment: vi.fn(),
|
||||||
|
},
|
||||||
|
};
|
||||||
|
});
|
||||||
|
|
||||||
|
vi.mock('@/lib/services/admin', async (importOriginal) => {
|
||||||
|
const actual = await importOriginal<typeof import('@/lib/services/admin')>();
|
||||||
|
return {
|
||||||
|
...actual,
|
||||||
|
AdminTaskService: {
|
||||||
|
getTaskExecution: vi.fn(),
|
||||||
|
},
|
||||||
|
};
|
||||||
|
});
|
||||||
|
|
||||||
|
function renderWithQuery(ui: React.ReactNode) {
|
||||||
|
const queryClient = new QueryClient({
|
||||||
|
defaultOptions: {
|
||||||
|
queries: { retry: false, gcTime: 0 },
|
||||||
|
mutations: { retry: false },
|
||||||
|
},
|
||||||
|
});
|
||||||
|
return render(
|
||||||
|
<QueryClientProvider client={queryClient}>{ui}</QueryClientProvider>,
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
|
const remoteSource: PagesRemoteURLSource = {
|
||||||
|
source_type: 'remote_url',
|
||||||
|
has_remote_url: true,
|
||||||
|
display_url: 'https://artifacts.example.com/site.zip?***',
|
||||||
|
remote_network_policy: 'public',
|
||||||
|
sync_status: 'idle',
|
||||||
|
last_applied: {
|
||||||
|
revision: 'a'.repeat(64),
|
||||||
|
label: 'site.zip',
|
||||||
|
},
|
||||||
|
last_synced_at: '2026-07-19T10:00:00Z',
|
||||||
|
last_error: '',
|
||||||
|
};
|
||||||
|
|
||||||
|
const githubLatestSource: PagesGitHubReleaseSource = {
|
||||||
|
source_type: 'github_release',
|
||||||
|
github_repository: 'openflare/site',
|
||||||
|
release_selector: 'latest',
|
||||||
|
asset_name: 'dist.zip',
|
||||||
|
auto_update_enabled: false,
|
||||||
|
check_interval_minutes: 60,
|
||||||
|
sync_status: 'update_available',
|
||||||
|
update_available: true,
|
||||||
|
last_seen: {
|
||||||
|
revision: 'b'.repeat(64),
|
||||||
|
label: 'v1.2.3',
|
||||||
|
asset_name: 'dist.zip',
|
||||||
|
},
|
||||||
|
last_applied: {
|
||||||
|
revision: 'a'.repeat(64),
|
||||||
|
label: 'v1.2.2',
|
||||||
|
asset_name: 'dist.zip',
|
||||||
|
},
|
||||||
|
last_checked_at: '2026-07-19T10:00:00Z',
|
||||||
|
last_synced_at: '2026-07-18T10:00:00Z',
|
||||||
|
next_check_at: '2026-07-19T11:00:00Z',
|
||||||
|
last_error: '',
|
||||||
|
};
|
||||||
|
|
||||||
|
const githubAttentionRevision = 'c'.repeat(64);
|
||||||
|
const githubAttentionSource: PagesGitHubReleaseSource = {
|
||||||
|
...githubLatestSource,
|
||||||
|
sync_status: 'attention',
|
||||||
|
update_available: true,
|
||||||
|
last_seen: {
|
||||||
|
revision: githubAttentionRevision,
|
||||||
|
label: 'v1.2.3',
|
||||||
|
asset_name: 'dist.zip',
|
||||||
|
},
|
||||||
|
};
|
||||||
|
|
||||||
|
function taskExecution(
|
||||||
|
status: TaskExecutionStatus,
|
||||||
|
errorMessage = '',
|
||||||
|
): TaskExecution {
|
||||||
|
return {
|
||||||
|
id: '42',
|
||||||
|
task_id: 'manual_of_pages_source_action_1',
|
||||||
|
task_type: 'of_pages_source_action',
|
||||||
|
task_name: 'Pages 来源动作',
|
||||||
|
status,
|
||||||
|
retryable: false,
|
||||||
|
max_retry: 0,
|
||||||
|
retry_count: 0,
|
||||||
|
log: '',
|
||||||
|
error_message: errorMessage,
|
||||||
|
result: '',
|
||||||
|
duration: 1,
|
||||||
|
payload: '',
|
||||||
|
triggered_by: 'admin:1',
|
||||||
|
created_at: '2026-07-19T10:00:00Z',
|
||||||
|
updated_at: '2026-07-19T10:00:01Z',
|
||||||
|
};
|
||||||
|
}
|
||||||
|
|
||||||
|
describe('Pages source UI', () => {
|
||||||
|
beforeEach(() => {
|
||||||
|
vi.mocked(PagesService.getSource).mockReset();
|
||||||
|
vi.mocked(PagesService.updateSource).mockReset();
|
||||||
|
vi.mocked(PagesService.deleteSource).mockReset();
|
||||||
|
vi.mocked(PagesService.checkSource).mockReset();
|
||||||
|
vi.mocked(PagesService.syncSource).mockReset();
|
||||||
|
vi.mocked(PagesService.listDeployments).mockReset();
|
||||||
|
vi.mocked(PagesService.listDeploymentFiles).mockReset();
|
||||||
|
vi.mocked(PagesService.activateDeployment).mockReset();
|
||||||
|
vi.mocked(PagesService.deleteDeployment).mockReset();
|
||||||
|
vi.mocked(PagesService.uploadDeployment).mockReset();
|
||||||
|
vi.mocked(AdminTaskService.getTaskExecution).mockReset();
|
||||||
|
});
|
||||||
|
|
||||||
|
it('offers the three source types without future repository build controls', async () => {
|
||||||
|
const user = userEvent.setup();
|
||||||
|
vi.mocked(PagesService.getSource).mockResolvedValue({
|
||||||
|
source_type: 'manual',
|
||||||
|
});
|
||||||
|
|
||||||
|
renderWithQuery(<PagesSourceCard projectId={9} />);
|
||||||
|
|
||||||
|
expect(await screen.findByText('本地部署包')).toBeVisible();
|
||||||
|
expect(
|
||||||
|
screen.getByRole('button', { name: '配置 Remote URL' }),
|
||||||
|
).toBeVisible();
|
||||||
|
expect(
|
||||||
|
screen.getByRole('button', { name: '配置 GitHub Release' }),
|
||||||
|
).toBeVisible();
|
||||||
|
expect(screen.queryByText('检查更新')).not.toBeInTheDocument();
|
||||||
|
expect(screen.queryByText('自动更新')).not.toBeInTheDocument();
|
||||||
|
|
||||||
|
await user.click(
|
||||||
|
screen.getByRole('button', { name: '配置 GitHub Release' }),
|
||||||
|
);
|
||||||
|
expect(screen.getByRole('radio', { name: '手动部署' })).toBeVisible();
|
||||||
|
expect(screen.getByRole('radio', { name: 'Remote URL' })).toBeVisible();
|
||||||
|
expect(screen.getByRole('radio', { name: 'GitHub Release' })).toBeVisible();
|
||||||
|
expect(
|
||||||
|
screen.getByText('仓库源码构建将在后续作为独立来源类型提供。'),
|
||||||
|
).toBeVisible();
|
||||||
|
expect(screen.getByRole('switch', { name: '自动更新' })).not.toBeChecked();
|
||||||
|
expect(screen.getByLabelText('检查间隔(分钟)')).toHaveValue(60);
|
||||||
|
expect(screen.queryByText('构建命令')).not.toBeInTheDocument();
|
||||||
|
expect(screen.queryByText('输出目录')).not.toBeInTheDocument();
|
||||||
|
});
|
||||||
|
|
||||||
|
it('submits the GitHub latest automatic update settings', async () => {
|
||||||
|
const user = userEvent.setup();
|
||||||
|
vi.mocked(PagesService.getSource).mockResolvedValue({
|
||||||
|
source_type: 'manual',
|
||||||
|
});
|
||||||
|
vi.mocked(PagesService.updateSource).mockResolvedValue({
|
||||||
|
source: githubLatestSource,
|
||||||
|
check_task: null,
|
||||||
|
warning: '',
|
||||||
|
});
|
||||||
|
|
||||||
|
renderWithQuery(<PagesSourceCard projectId={9} />);
|
||||||
|
|
||||||
|
await user.click(
|
||||||
|
await screen.findByRole('button', { name: '配置 GitHub Release' }),
|
||||||
|
);
|
||||||
|
await user.type(
|
||||||
|
screen.getByLabelText('GitHub 仓库 URL'),
|
||||||
|
'https://github.com/openflare/site',
|
||||||
|
);
|
||||||
|
expect(screen.getByLabelText('Release Asset 文件名')).toHaveValue(
|
||||||
|
'dist.zip',
|
||||||
|
);
|
||||||
|
await user.click(screen.getByRole('switch', { name: '自动更新' }));
|
||||||
|
const intervalInput = screen.getByLabelText('检查间隔(分钟)');
|
||||||
|
await user.clear(intervalInput);
|
||||||
|
await user.type(intervalInput, '15');
|
||||||
|
await user.click(screen.getByRole('button', { name: '保存 GitHub 来源' }));
|
||||||
|
|
||||||
|
await waitFor(() => {
|
||||||
|
expect(PagesService.updateSource).toHaveBeenCalledWith(9, {
|
||||||
|
source_type: 'github_release',
|
||||||
|
repository_url: 'https://github.com/openflare/site',
|
||||||
|
release_selector: 'latest',
|
||||||
|
release_tag: '',
|
||||||
|
asset_name: 'dist.zip',
|
||||||
|
auto_update_enabled: true,
|
||||||
|
check_interval_minutes: 15,
|
||||||
|
});
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
|
it('rejects non-canonical GitHub repository URL paths', async () => {
|
||||||
|
const user = userEvent.setup();
|
||||||
|
vi.mocked(PagesService.getSource).mockResolvedValue({
|
||||||
|
source_type: 'manual',
|
||||||
|
});
|
||||||
|
|
||||||
|
renderWithQuery(<PagesSourceCard projectId={9} />);
|
||||||
|
|
||||||
|
await user.click(
|
||||||
|
await screen.findByRole('button', { name: '配置 GitHub Release' }),
|
||||||
|
);
|
||||||
|
const repositoryInput = screen.getByLabelText('GitHub 仓库 URL');
|
||||||
|
await user.type(repositoryInput, 'https://github.com//openflare/site');
|
||||||
|
await user.click(screen.getByRole('button', { name: '保存 GitHub 来源' }));
|
||||||
|
expect(
|
||||||
|
screen.getByText(
|
||||||
|
'请输入 https://github.com/{owner}/{repo} 格式的公开仓库地址',
|
||||||
|
),
|
||||||
|
).toBeVisible();
|
||||||
|
|
||||||
|
await user.clear(repositoryInput);
|
||||||
|
await user.type(repositoryInput, 'https://github.com/openflare/site/');
|
||||||
|
await user.click(screen.getByRole('button', { name: '保存 GitHub 来源' }));
|
||||||
|
expect(PagesService.updateSource).not.toHaveBeenCalled();
|
||||||
|
});
|
||||||
|
|
||||||
|
it('submits the GitHub tag discriminator with safe disabled defaults', async () => {
|
||||||
|
const user = userEvent.setup();
|
||||||
|
vi.mocked(PagesService.getSource).mockResolvedValue({
|
||||||
|
source_type: 'manual',
|
||||||
|
});
|
||||||
|
vi.mocked(PagesService.updateSource).mockResolvedValue({
|
||||||
|
source: {
|
||||||
|
...githubLatestSource,
|
||||||
|
release_selector: 'tag',
|
||||||
|
release_tag: 'v1.2.3',
|
||||||
|
auto_update_enabled: false,
|
||||||
|
check_interval_minutes: 0,
|
||||||
|
next_check_at: null,
|
||||||
|
},
|
||||||
|
check_task: null,
|
||||||
|
warning: '',
|
||||||
|
});
|
||||||
|
|
||||||
|
renderWithQuery(<PagesSourceCard projectId={9} />);
|
||||||
|
|
||||||
|
await user.click(
|
||||||
|
await screen.findByRole('button', { name: '配置 GitHub Release' }),
|
||||||
|
);
|
||||||
|
await user.type(
|
||||||
|
screen.getByLabelText('GitHub 仓库 URL'),
|
||||||
|
'https://github.com/openflare/site',
|
||||||
|
);
|
||||||
|
await user.click(screen.getByRole('radio', { name: '固定 Tag' }));
|
||||||
|
expect(
|
||||||
|
screen.queryByRole('switch', { name: '自动更新' }),
|
||||||
|
).not.toBeInTheDocument();
|
||||||
|
expect(screen.queryByLabelText('检查间隔(分钟)')).not.toBeInTheDocument();
|
||||||
|
await user.type(
|
||||||
|
screen.getByLabelText('Release tag'),
|
||||||
|
'release/candidate#1&channel=stable',
|
||||||
|
);
|
||||||
|
await user.clear(screen.getByLabelText('Release Asset 文件名'));
|
||||||
|
await user.type(
|
||||||
|
screen.getByLabelText('Release Asset 文件名'),
|
||||||
|
' site?arch=amd64#stable.zip ',
|
||||||
|
);
|
||||||
|
await user.click(screen.getByRole('button', { name: '保存 GitHub 来源' }));
|
||||||
|
|
||||||
|
await waitFor(() => {
|
||||||
|
expect(PagesService.updateSource).toHaveBeenCalledWith(9, {
|
||||||
|
source_type: 'github_release',
|
||||||
|
repository_url: 'https://github.com/openflare/site',
|
||||||
|
release_selector: 'tag',
|
||||||
|
release_tag: 'release/candidate#1&channel=stable',
|
||||||
|
asset_name: ' site?arch=amd64#stable.zip ',
|
||||||
|
auto_update_enabled: false,
|
||||||
|
check_interval_minutes: 0,
|
||||||
|
});
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
|
it('rejects unsafe GitHub tag and asset values before saving', async () => {
|
||||||
|
const user = userEvent.setup();
|
||||||
|
vi.mocked(PagesService.getSource).mockResolvedValue({
|
||||||
|
source_type: 'manual',
|
||||||
|
});
|
||||||
|
|
||||||
|
renderWithQuery(<PagesSourceCard projectId={9} />);
|
||||||
|
|
||||||
|
await user.click(
|
||||||
|
await screen.findByRole('button', { name: '配置 GitHub Release' }),
|
||||||
|
);
|
||||||
|
await user.type(
|
||||||
|
screen.getByLabelText('GitHub 仓库 URL'),
|
||||||
|
'https://github.com/openflare/site',
|
||||||
|
);
|
||||||
|
await user.click(screen.getByRole('radio', { name: '固定 Tag' }));
|
||||||
|
await user.type(screen.getByLabelText('Release tag'), 'release//candidate');
|
||||||
|
await user.clear(screen.getByLabelText('Release Asset 文件名'));
|
||||||
|
await user.type(
|
||||||
|
screen.getByLabelText('Release Asset 文件名'),
|
||||||
|
'../dist.zip',
|
||||||
|
);
|
||||||
|
await user.click(screen.getByRole('button', { name: '保存 GitHub 来源' }));
|
||||||
|
|
||||||
|
expect(
|
||||||
|
screen.getByText(
|
||||||
|
'Release tag 须为有效 Git ref(1–255 字节,可使用 /、#、&、=)',
|
||||||
|
),
|
||||||
|
).toBeVisible();
|
||||||
|
expect(
|
||||||
|
screen.getByText(
|
||||||
|
'Asset 文件名须为 1–255 字节,且不能是路径或包含控制、换行、双向文本字符',
|
||||||
|
),
|
||||||
|
).toBeVisible();
|
||||||
|
expect(PagesService.updateSource).not.toHaveBeenCalled();
|
||||||
|
});
|
||||||
|
|
||||||
|
it('polls the initial check receipt and represents the queued window locally', async () => {
|
||||||
|
const user = userEvent.setup();
|
||||||
|
const idleGitHubSource: PagesGitHubReleaseSource = {
|
||||||
|
...githubLatestSource,
|
||||||
|
sync_status: 'idle',
|
||||||
|
update_available: false,
|
||||||
|
};
|
||||||
|
vi.mocked(PagesService.getSource)
|
||||||
|
.mockResolvedValueOnce({ source_type: 'manual' })
|
||||||
|
.mockResolvedValue(idleGitHubSource);
|
||||||
|
vi.mocked(PagesService.updateSource).mockResolvedValue({
|
||||||
|
source: idleGitHubSource,
|
||||||
|
check_task: {
|
||||||
|
task_id: 'manual_of_pages_source_action_1',
|
||||||
|
execution_id: '42',
|
||||||
|
action: 'check',
|
||||||
|
},
|
||||||
|
warning: '',
|
||||||
|
});
|
||||||
|
vi.mocked(AdminTaskService.getTaskExecution).mockResolvedValue(
|
||||||
|
taskExecution('pending'),
|
||||||
|
);
|
||||||
|
|
||||||
|
renderWithQuery(<PagesSourceCard projectId={9} />);
|
||||||
|
|
||||||
|
await user.click(
|
||||||
|
await screen.findByRole('button', { name: '配置 GitHub Release' }),
|
||||||
|
);
|
||||||
|
await user.type(
|
||||||
|
screen.getByLabelText('GitHub 仓库 URL'),
|
||||||
|
'https://github.com/openflare/site',
|
||||||
|
);
|
||||||
|
await user.click(screen.getByRole('button', { name: '保存 GitHub 来源' }));
|
||||||
|
|
||||||
|
await waitFor(() => {
|
||||||
|
expect(AdminTaskService.getTaskExecution).toHaveBeenCalledWith('42');
|
||||||
|
expect(screen.getByText('检查中')).toBeVisible();
|
||||||
|
});
|
||||||
|
const checkButton = screen.getByRole('button', { name: /检查更新/ });
|
||||||
|
expect(checkButton).toBeDisabled();
|
||||||
|
expect(within(checkButton).getByRole('status')).toBeVisible();
|
||||||
|
expect(PagesService.checkSource).not.toHaveBeenCalled();
|
||||||
|
expect(PagesService.syncSource).not.toHaveBeenCalled();
|
||||||
|
});
|
||||||
|
|
||||||
|
it('shows GitHub latest automatic update state and schedule', async () => {
|
||||||
|
vi.mocked(PagesService.getSource).mockResolvedValue(githubLatestSource);
|
||||||
|
|
||||||
|
renderWithQuery(<PagesSourceCard projectId={9} />);
|
||||||
|
|
||||||
|
expect(await screen.findByText('openflare/site')).toBeVisible();
|
||||||
|
expect(screen.getByText('v1.2.3 · bbbbbbbbbbbb')).toBeVisible();
|
||||||
|
expect(screen.getByText('v1.2.2 · aaaaaaaaaaaa')).toBeVisible();
|
||||||
|
expect(screen.getByText('有可用更新')).toBeVisible();
|
||||||
|
expect(screen.getByText('下次检查时间')).toBeVisible();
|
||||||
|
expect(screen.getByText('自动更新')).toBeVisible();
|
||||||
|
expect(screen.getByText('已关闭')).toBeVisible();
|
||||||
|
expect(screen.getByText('检查间隔')).toBeVisible();
|
||||||
|
expect(screen.getByText('60 分钟')).toBeVisible();
|
||||||
|
});
|
||||||
|
|
||||||
|
it('dispatches a GitHub check and starts TaskExecution polling', async () => {
|
||||||
|
const user = userEvent.setup();
|
||||||
|
vi.mocked(PagesService.getSource).mockResolvedValue(githubLatestSource);
|
||||||
|
vi.mocked(PagesService.checkSource).mockResolvedValue({
|
||||||
|
task_id: 'manual_of_pages_source_action_1',
|
||||||
|
execution_id: '42',
|
||||||
|
action: 'check',
|
||||||
|
});
|
||||||
|
vi.mocked(AdminTaskService.getTaskExecution).mockResolvedValue(
|
||||||
|
taskExecution('succeeded'),
|
||||||
|
);
|
||||||
|
|
||||||
|
renderWithQuery(<PagesSourceCard projectId={9} />);
|
||||||
|
|
||||||
|
await user.click(await screen.findByRole('button', { name: '检查更新' }));
|
||||||
|
|
||||||
|
await waitFor(() => {
|
||||||
|
expect(PagesService.checkSource).toHaveBeenCalledWith(9);
|
||||||
|
expect(AdminTaskService.getTaskExecution).toHaveBeenCalledWith('42');
|
||||||
|
expect(PagesService.syncSource).not.toHaveBeenCalled();
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
|
it('renders a GitHub check dispatch error', async () => {
|
||||||
|
const user = userEvent.setup();
|
||||||
|
vi.mocked(PagesService.getSource).mockResolvedValue(githubLatestSource);
|
||||||
|
vi.mocked(PagesService.checkSource).mockRejectedValue(
|
||||||
|
new Error('GitHub API 暂不可用'),
|
||||||
|
);
|
||||||
|
|
||||||
|
renderWithQuery(<PagesSourceCard projectId={9} />);
|
||||||
|
|
||||||
|
await user.click(await screen.findByRole('button', { name: '检查更新' }));
|
||||||
|
|
||||||
|
expect(await screen.findByText('GitHub API 暂不可用')).toBeVisible();
|
||||||
|
});
|
||||||
|
|
||||||
|
it('renders a TaskExecution polling error with an explicit retry', async () => {
|
||||||
|
const user = userEvent.setup();
|
||||||
|
vi.mocked(PagesService.getSource).mockResolvedValue(githubLatestSource);
|
||||||
|
vi.mocked(PagesService.checkSource).mockResolvedValue({
|
||||||
|
task_id: 'manual_of_pages_source_action_1',
|
||||||
|
execution_id: '42',
|
||||||
|
action: 'check',
|
||||||
|
});
|
||||||
|
vi.mocked(AdminTaskService.getTaskExecution).mockRejectedValue(
|
||||||
|
new Error('任务状态暂时不可读取'),
|
||||||
|
);
|
||||||
|
|
||||||
|
renderWithQuery(<PagesSourceCard projectId={9} />);
|
||||||
|
|
||||||
|
await user.click(await screen.findByRole('button', { name: '检查更新' }));
|
||||||
|
|
||||||
|
expect(await screen.findByText('任务状态暂时不可读取')).toBeVisible();
|
||||||
|
expect(screen.getByRole('button', { name: '重试' })).toBeVisible();
|
||||||
|
});
|
||||||
|
|
||||||
|
it('requires the exact currently displayed revision for attention sync', async () => {
|
||||||
|
const user = userEvent.setup();
|
||||||
|
vi.mocked(PagesService.getSource).mockResolvedValue(githubAttentionSource);
|
||||||
|
vi.mocked(PagesService.syncSource).mockResolvedValue({
|
||||||
|
task_id: 'manual_of_pages_source_action_1',
|
||||||
|
execution_id: '42',
|
||||||
|
action: 'sync',
|
||||||
|
});
|
||||||
|
vi.mocked(AdminTaskService.getTaskExecution).mockResolvedValue(
|
||||||
|
taskExecution('succeeded'),
|
||||||
|
);
|
||||||
|
|
||||||
|
renderWithQuery(<PagesSourceCard projectId={9} />);
|
||||||
|
|
||||||
|
const alert = await screen.findByRole('alert');
|
||||||
|
expect(
|
||||||
|
within(alert).getByText('Release Asset 发生变化,需要显式确认'),
|
||||||
|
).toBeVisible();
|
||||||
|
expect(within(alert).getByText(githubAttentionRevision)).toBeVisible();
|
||||||
|
|
||||||
|
await user.click(screen.getByRole('button', { name: '同步并发布' }));
|
||||||
|
const dialog = screen.getByRole('alertdialog');
|
||||||
|
expect(within(dialog).getByText(githubAttentionRevision)).toBeVisible();
|
||||||
|
await user.click(
|
||||||
|
within(dialog).getByRole('button', { name: '确认并发布' }),
|
||||||
|
);
|
||||||
|
|
||||||
|
await waitFor(() => {
|
||||||
|
expect(PagesService.syncSource).toHaveBeenCalledWith(9, {
|
||||||
|
confirmed_revision: githubAttentionRevision,
|
||||||
|
});
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
|
it('never reuses the masked URL as an editable value', async () => {
|
||||||
|
const user = userEvent.setup();
|
||||||
|
vi.mocked(PagesService.getSource).mockResolvedValue(remoteSource);
|
||||||
|
vi.mocked(PagesService.updateSource).mockResolvedValue({
|
||||||
|
source: remoteSource,
|
||||||
|
check_task: null,
|
||||||
|
warning: '',
|
||||||
|
});
|
||||||
|
|
||||||
|
renderWithQuery(<PagesSourceCard projectId={9} />);
|
||||||
|
|
||||||
|
expect(
|
||||||
|
await screen.findByText('https://artifacts.example.com/site.zip?***'),
|
||||||
|
).toBeVisible();
|
||||||
|
expect(screen.queryByText(/token=secret/)).not.toBeInTheDocument();
|
||||||
|
|
||||||
|
await user.click(screen.getByRole('button', { name: '编辑来源' }));
|
||||||
|
await user.click(screen.getByRole('button', { name: '更换地址' }));
|
||||||
|
|
||||||
|
const input = screen.getByPlaceholderText(
|
||||||
|
'https://artifacts.example.com/site.zip?token=...',
|
||||||
|
);
|
||||||
|
expect(input).toHaveValue('');
|
||||||
|
expect(input).toHaveAttribute('type', 'password');
|
||||||
|
await user.type(input, 'https://new.example.com/site.zip?token=new');
|
||||||
|
expect(
|
||||||
|
screen.queryByText('https://new.example.com/site.zip?token=new'),
|
||||||
|
).not.toBeInTheDocument();
|
||||||
|
await user.click(screen.getByRole('button', { name: '显示 Remote URL' }));
|
||||||
|
expect(input).toHaveAttribute('type', 'url');
|
||||||
|
await user.click(screen.getByRole('button', { name: '保存 Remote 来源' }));
|
||||||
|
|
||||||
|
await waitFor(() => {
|
||||||
|
expect(PagesService.updateSource).toHaveBeenCalledWith(9, {
|
||||||
|
source_type: 'remote_url',
|
||||||
|
remote_url_set: true,
|
||||||
|
remote_url: 'https://new.example.com/site.zip?token=new',
|
||||||
|
remote_network_policy: 'public',
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
|
await waitFor(() => {
|
||||||
|
expect(screen.queryByRole('dialog')).not.toBeInTheDocument();
|
||||||
|
});
|
||||||
|
await user.click(screen.getByRole('button', { name: '编辑来源' }));
|
||||||
|
await user.click(screen.getByRole('button', { name: '更换地址' }));
|
||||||
|
const reopenedInput = screen.getByPlaceholderText(
|
||||||
|
'https://artifacts.example.com/site.zip?token=...',
|
||||||
|
);
|
||||||
|
expect(reopenedInput).toHaveValue('');
|
||||||
|
expect(reopenedInput).toHaveAttribute('type', 'password');
|
||||||
|
});
|
||||||
|
|
||||||
|
it('requires a second confirmation for trusted internal networking', async () => {
|
||||||
|
const user = userEvent.setup();
|
||||||
|
vi.mocked(PagesService.getSource).mockResolvedValue(remoteSource);
|
||||||
|
vi.mocked(PagesService.updateSource).mockResolvedValue({
|
||||||
|
source: { ...remoteSource, remote_network_policy: 'trusted_internal' },
|
||||||
|
check_task: null,
|
||||||
|
warning: '',
|
||||||
|
});
|
||||||
|
|
||||||
|
renderWithQuery(<PagesSourceCard projectId={9} />);
|
||||||
|
|
||||||
|
await user.click(await screen.findByRole('button', { name: '编辑来源' }));
|
||||||
|
await user.click(screen.getByRole('radio', { name: '受信内网模式' }));
|
||||||
|
await user.click(screen.getByRole('button', { name: '保存 Remote 来源' }));
|
||||||
|
|
||||||
|
expect(await screen.findByText('启用受信内网模式')).toBeVisible();
|
||||||
|
expect(PagesService.updateSource).not.toHaveBeenCalled();
|
||||||
|
|
||||||
|
await user.click(screen.getByRole('button', { name: '确认' }));
|
||||||
|
await waitFor(() => {
|
||||||
|
expect(PagesService.updateSource).toHaveBeenCalledWith(9, {
|
||||||
|
source_type: 'remote_url',
|
||||||
|
remote_url_set: false,
|
||||||
|
remote_url: '',
|
||||||
|
remote_network_policy: 'trusted_internal',
|
||||||
|
});
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
|
it('polls the existing task execution detail after sync dispatch', async () => {
|
||||||
|
const user = userEvent.setup();
|
||||||
|
vi.mocked(PagesService.getSource).mockResolvedValue(remoteSource);
|
||||||
|
vi.mocked(PagesService.syncSource).mockResolvedValue({
|
||||||
|
task_id: 'manual_of_pages_source_action_1',
|
||||||
|
execution_id: '42',
|
||||||
|
action: 'sync',
|
||||||
|
});
|
||||||
|
vi.mocked(AdminTaskService.getTaskExecution)
|
||||||
|
.mockResolvedValueOnce(taskExecution('pending'))
|
||||||
|
.mockResolvedValue(taskExecution('succeeded'));
|
||||||
|
|
||||||
|
renderWithQuery(<PagesSourceCard projectId={9} />);
|
||||||
|
|
||||||
|
await user.click(await screen.findByRole('button', { name: '同步并发布' }));
|
||||||
|
|
||||||
|
await waitFor(() => {
|
||||||
|
expect(PagesService.syncSource).toHaveBeenCalledWith(9, {});
|
||||||
|
expect(AdminTaskService.getTaskExecution).toHaveBeenCalledWith('42');
|
||||||
|
});
|
||||||
|
expect(screen.getByRole('button', { name: /同步并发布/ })).toBeDisabled();
|
||||||
|
|
||||||
|
await waitFor(
|
||||||
|
() => {
|
||||||
|
expect(AdminTaskService.getTaskExecution).toHaveBeenCalledTimes(2);
|
||||||
|
},
|
||||||
|
{ timeout: 3_500 },
|
||||||
|
);
|
||||||
|
});
|
||||||
|
|
||||||
|
it('shows the actual project entry and no one-off URL upload tab', () => {
|
||||||
|
renderWithQuery(
|
||||||
|
<DeploymentUploadDialog
|
||||||
|
open
|
||||||
|
onOpenChange={vi.fn()}
|
||||||
|
projectId={9}
|
||||||
|
rootDir='dist/site'
|
||||||
|
entryFile='home.html'
|
||||||
|
/>,
|
||||||
|
);
|
||||||
|
|
||||||
|
expect(screen.getByText('dist/site/home.html')).toBeVisible();
|
||||||
|
expect(screen.queryByText('从 URL 下载')).not.toBeInTheDocument();
|
||||||
|
expect(screen.queryByText('部署包下载链接')).not.toBeInTheDocument();
|
||||||
|
});
|
||||||
|
|
||||||
|
it('renders a deployment query failure instead of an empty history', async () => {
|
||||||
|
vi.mocked(PagesService.listDeployments).mockRejectedValue(
|
||||||
|
new Error('部署历史暂时不可用'),
|
||||||
|
);
|
||||||
|
|
||||||
|
renderWithQuery(<DeploymentHistory projectId={9} />);
|
||||||
|
|
||||||
|
expect(await screen.findByText('部署历史暂时不可用')).toBeVisible();
|
||||||
|
expect(screen.queryByText('暂无部署')).not.toBeInTheDocument();
|
||||||
|
});
|
||||||
|
|
||||||
|
it('renders the immutable GitHub deployment provenance in Chinese', async () => {
|
||||||
|
const deployment: PagesDeployment = {
|
||||||
|
id: 51,
|
||||||
|
project_id: 9,
|
||||||
|
deployment_number: 7,
|
||||||
|
checksum: 'd'.repeat(64),
|
||||||
|
status: 'active',
|
||||||
|
file_count: 12,
|
||||||
|
total_size: 4_096,
|
||||||
|
created_by: 'user:1',
|
||||||
|
source_type: 'github_release',
|
||||||
|
source_label: 'v1.2.3',
|
||||||
|
trigger_type: 'manual_sync',
|
||||||
|
created_at: '2026-07-19T10:00:00Z',
|
||||||
|
activated_at: '2026-07-19T10:00:01Z',
|
||||||
|
};
|
||||||
|
vi.mocked(PagesService.listDeployments).mockResolvedValue([deployment]);
|
||||||
|
|
||||||
|
renderWithQuery(
|
||||||
|
<DeploymentHistory projectId={9} activeDeploymentId={51} />,
|
||||||
|
);
|
||||||
|
|
||||||
|
expect(await screen.findByText('GitHub · v1.2.3 · 手动同步')).toBeVisible();
|
||||||
|
expect(screen.getByText('当前生产部署')).toBeVisible();
|
||||||
|
});
|
||||||
|
});
|
||||||
@@ -0,0 +1,100 @@
|
|||||||
|
import { describe, expect, it } from 'vitest';
|
||||||
|
|
||||||
|
import {
|
||||||
|
validGitHubAssetName,
|
||||||
|
validGitHubReleaseTag,
|
||||||
|
validGitHubRepositoryURL,
|
||||||
|
} from '@/app/(main)/pages/detail/components/pages-source-validation';
|
||||||
|
|
||||||
|
describe('Pages GitHub source validation', () => {
|
||||||
|
it('accepts only canonical public GitHub repository URLs', () => {
|
||||||
|
expect(validGitHubRepositoryURL('https://github.com/acme/site')).toBe(true);
|
||||||
|
expect(validGitHubRepositoryURL('https://GitHub.com/acme/site.git')).toBe(
|
||||||
|
true,
|
||||||
|
);
|
||||||
|
|
||||||
|
const invalid = [
|
||||||
|
'http://github.com/acme/site',
|
||||||
|
'https://github.com//acme/site',
|
||||||
|
'https://github.com/acme/site/',
|
||||||
|
'https://github.com/acme/site/extra',
|
||||||
|
'https://github.com/acme/./site',
|
||||||
|
'https://github.com/acme/../site',
|
||||||
|
'https://github.com/acme/%73ite',
|
||||||
|
'https://github.com/acme/site?token=secret',
|
||||||
|
'https://github.com:443/acme/site',
|
||||||
|
String.raw`https://github.com/acme\site`,
|
||||||
|
];
|
||||||
|
for (const value of invalid) {
|
||||||
|
expect(validGitHubRepositoryURL(value), value).toBe(false);
|
||||||
|
}
|
||||||
|
});
|
||||||
|
|
||||||
|
it('mirrors Git ref rules while preserving legal release tag characters', () => {
|
||||||
|
const valid = [
|
||||||
|
'@',
|
||||||
|
'release/v1#stable&channel=prod',
|
||||||
|
'foo.LOCK',
|
||||||
|
'中文/发布=稳定',
|
||||||
|
];
|
||||||
|
for (const value of valid) {
|
||||||
|
expect(validGitHubReleaseTag(value), value).toBe(true);
|
||||||
|
}
|
||||||
|
|
||||||
|
const invalid = [
|
||||||
|
'',
|
||||||
|
'release v1',
|
||||||
|
'release~v1',
|
||||||
|
'release^v1',
|
||||||
|
'release:v1',
|
||||||
|
'release?v1',
|
||||||
|
'release*v1',
|
||||||
|
'release[v1',
|
||||||
|
String.raw`release\v1`,
|
||||||
|
'release..v1',
|
||||||
|
'release@{v1',
|
||||||
|
'release//v1',
|
||||||
|
'/release',
|
||||||
|
'release/',
|
||||||
|
'release.',
|
||||||
|
'.release',
|
||||||
|
'release/.candidate',
|
||||||
|
'release/v1.lock',
|
||||||
|
'release\nsecret',
|
||||||
|
'release\u2028secret',
|
||||||
|
'release\u202esecret',
|
||||||
|
'a'.repeat(256),
|
||||||
|
];
|
||||||
|
for (const value of invalid) {
|
||||||
|
expect(validGitHubReleaseTag(value), value).toBe(false);
|
||||||
|
}
|
||||||
|
});
|
||||||
|
|
||||||
|
it('preserves exact legal asset names and rejects path or display controls', () => {
|
||||||
|
const valid = [
|
||||||
|
'dist.zip',
|
||||||
|
'dist?channel=stable&part#1.zip',
|
||||||
|
' dist.zip ',
|
||||||
|
'build=production.zip',
|
||||||
|
];
|
||||||
|
for (const value of valid) {
|
||||||
|
expect(validGitHubAssetName(value), value).toBe(true);
|
||||||
|
}
|
||||||
|
|
||||||
|
const invalid = [
|
||||||
|
'',
|
||||||
|
'.',
|
||||||
|
'..',
|
||||||
|
'../dist.zip',
|
||||||
|
String.raw`dir\dist.zip`,
|
||||||
|
'dist\n.zip',
|
||||||
|
'dist\u2028.zip',
|
||||||
|
'dist\u202e.zip',
|
||||||
|
'a'.repeat(256),
|
||||||
|
'\ud800',
|
||||||
|
];
|
||||||
|
for (const value of invalid) {
|
||||||
|
expect(validGitHubAssetName(value), value).toBe(false);
|
||||||
|
}
|
||||||
|
});
|
||||||
|
});
|
||||||
@@ -66,7 +66,7 @@ func DispatchTask(c *gin.Context) {
|
|||||||
return
|
return
|
||||||
}
|
}
|
||||||
|
|
||||||
meta := task.GetTaskMeta(req.TaskType)
|
meta := getAdminTaskMeta(req.TaskType)
|
||||||
if meta == nil {
|
if meta == nil {
|
||||||
response.AbortBadRequest(c, InvalidTaskType)
|
response.AbortBadRequest(c, InvalidTaskType)
|
||||||
return
|
return
|
||||||
@@ -202,7 +202,7 @@ func RetryTask(c *gin.Context) {
|
|||||||
|
|
||||||
// ListSchedules 获取定时任务列表
|
// ListSchedules 获取定时任务列表
|
||||||
// @Summary 获取定时任务列表
|
// @Summary 获取定时任务列表
|
||||||
// @Description 返回系统所有的定时任务配置列表,包括名称、关联的异步任务类型、Cron 表达式和启用状态,需要管理员权限
|
// @Description 返回管理员可管理的定时任务配置列表,包括名称、关联的异步任务类型、Cron 表达式和启用状态;系统内部排程不会暴露,需要管理员权限
|
||||||
// @Tags admin
|
// @Tags admin
|
||||||
// @Produce json
|
// @Produce json
|
||||||
// @Security SessionCookie
|
// @Security SessionCookie
|
||||||
@@ -216,7 +216,15 @@ func ListSchedules(c *gin.Context) {
|
|||||||
response.AbortInternal(c, err.Error())
|
response.AbortInternal(c, err.Error())
|
||||||
return
|
return
|
||||||
}
|
}
|
||||||
c.JSON(http.StatusOK, response.OK(schedules))
|
visible := make([]model.Schedule, 0, len(schedules))
|
||||||
|
for _, schedule := range schedules {
|
||||||
|
meta := task.GetTaskMeta(schedule.TaskType)
|
||||||
|
if meta != nil && meta.InternalOnly {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
visible = append(visible, schedule)
|
||||||
|
}
|
||||||
|
c.JSON(http.StatusOK, response.OK(visible))
|
||||||
}
|
}
|
||||||
|
|
||||||
// CreateScheduleRequest 创建定时任务请求
|
// CreateScheduleRequest 创建定时任务请求
|
||||||
@@ -256,7 +264,7 @@ func CreateSchedule(c *gin.Context) {
|
|||||||
}
|
}
|
||||||
|
|
||||||
// 校验关联的异步任务类型
|
// 校验关联的异步任务类型
|
||||||
meta := task.GetTaskMeta(req.TaskType)
|
meta := getAdminTaskMeta(req.TaskType)
|
||||||
if meta == nil {
|
if meta == nil {
|
||||||
response.AbortBadRequest(c, InvalidTaskType)
|
response.AbortBadRequest(c, InvalidTaskType)
|
||||||
return
|
return
|
||||||
@@ -338,6 +346,10 @@ func UpdateSchedule(c *gin.Context) {
|
|||||||
response.AbortNotFound(c, ScheduleNotFound)
|
response.AbortNotFound(c, ScheduleNotFound)
|
||||||
return
|
return
|
||||||
}
|
}
|
||||||
|
if existingMeta := task.GetTaskMeta(schedule.TaskType); existingMeta != nil && existingMeta.InternalOnly {
|
||||||
|
response.AbortBadRequest(c, InvalidTaskType)
|
||||||
|
return
|
||||||
|
}
|
||||||
|
|
||||||
// 校验 Cron 表达式
|
// 校验 Cron 表达式
|
||||||
if _, err := cron.ParseStandard(req.Cron); err != nil {
|
if _, err := cron.ParseStandard(req.Cron); err != nil {
|
||||||
@@ -346,7 +358,7 @@ func UpdateSchedule(c *gin.Context) {
|
|||||||
}
|
}
|
||||||
|
|
||||||
// 校验关联的异步任务类型
|
// 校验关联的异步任务类型
|
||||||
meta := task.GetTaskMeta(req.TaskType)
|
meta := getAdminTaskMeta(req.TaskType)
|
||||||
if meta == nil {
|
if meta == nil {
|
||||||
response.AbortBadRequest(c, InvalidTaskType)
|
response.AbortBadRequest(c, InvalidTaskType)
|
||||||
return
|
return
|
||||||
@@ -382,6 +394,14 @@ func UpdateSchedule(c *gin.Context) {
|
|||||||
c.JSON(http.StatusOK, response.OK(schedule))
|
c.JSON(http.StatusOK, response.OK(schedule))
|
||||||
}
|
}
|
||||||
|
|
||||||
|
func getAdminTaskMeta(taskType string) *task.TaskMeta {
|
||||||
|
meta := task.GetTaskMeta(taskType)
|
||||||
|
if meta == nil || meta.InternalOnly {
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
return meta
|
||||||
|
}
|
||||||
|
|
||||||
// DeleteSchedule 删除定时任务
|
// DeleteSchedule 删除定时任务
|
||||||
// @Summary 删除定时任务
|
// @Summary 删除定时任务
|
||||||
// @Description 删除指定的定时任务配置,并触发调度器热加载,需要管理员权限
|
// @Description 删除指定的定时任务配置,并触发调度器热加载,需要管理员权限
|
||||||
@@ -393,6 +413,7 @@ func UpdateSchedule(c *gin.Context) {
|
|||||||
// @Failure 400 {object} response.Any "参数错误"
|
// @Failure 400 {object} response.Any "参数错误"
|
||||||
// @Failure 401 {object} response.Any "未登录"
|
// @Failure 401 {object} response.Any "未登录"
|
||||||
// @Failure 403 {object} response.Any "无管理员权限"
|
// @Failure 403 {object} response.Any "无管理员权限"
|
||||||
|
// @Failure 404 {object} response.Any "定时任务不存在"
|
||||||
// @Failure 500 {object} response.Any "删除定时任务失败"
|
// @Failure 500 {object} response.Any "删除定时任务失败"
|
||||||
// @Router /api/v1/admin/tasks/schedules/{id} [delete]
|
// @Router /api/v1/admin/tasks/schedules/{id} [delete]
|
||||||
func DeleteSchedule(c *gin.Context) {
|
func DeleteSchedule(c *gin.Context) {
|
||||||
@@ -401,6 +422,15 @@ func DeleteSchedule(c *gin.Context) {
|
|||||||
response.AbortBadRequest(c, "无效的定时任务ID")
|
response.AbortBadRequest(c, "无效的定时任务ID")
|
||||||
return
|
return
|
||||||
}
|
}
|
||||||
|
schedule, err := model.GetScheduleByID(c.Request.Context(), id)
|
||||||
|
if err != nil {
|
||||||
|
response.AbortNotFound(c, ScheduleNotFound)
|
||||||
|
return
|
||||||
|
}
|
||||||
|
if meta := task.GetTaskMeta(schedule.TaskType); meta != nil && meta.InternalOnly {
|
||||||
|
response.AbortBadRequest(c, InvalidTaskType)
|
||||||
|
return
|
||||||
|
}
|
||||||
|
|
||||||
if err := model.DeleteSchedule(c.Request.Context(), id); err != nil {
|
if err := model.DeleteSchedule(c.Request.Context(), id); err != nil {
|
||||||
response.AbortInternal(c, fmt.Sprintf("%s: %v", ScheduleDeleteFailed, err))
|
response.AbortInternal(c, fmt.Sprintf("%s: %v", ScheduleDeleteFailed, err))
|
||||||
|
|||||||
@@ -29,6 +29,20 @@ import (
|
|||||||
"github.com/Rain-kl/Wavelet/internal/common/response"
|
"github.com/Rain-kl/Wavelet/internal/common/response"
|
||||||
)
|
)
|
||||||
|
|
||||||
|
const (
|
||||||
|
testInternalOnlyTaskType = "test_internal_only_admin"
|
||||||
|
testInternalOnlyAsynqTask = "test:internal_only_admin"
|
||||||
|
)
|
||||||
|
|
||||||
|
func registerInternalOnlyTaskMeta() {
|
||||||
|
task.RegisterTaskMeta(task.TaskMeta{
|
||||||
|
Type: testInternalOnlyTaskType,
|
||||||
|
AsynqTask: testInternalOnlyAsynqTask,
|
||||||
|
Name: "内部测试任务",
|
||||||
|
InternalOnly: true,
|
||||||
|
})
|
||||||
|
}
|
||||||
|
|
||||||
func setupTaskTestEnvironment(t *testing.T) func() {
|
func setupTaskTestEnvironment(t *testing.T) func() {
|
||||||
_, mr, cleanup := testhelper.SetupTestEnvironment(t)
|
_, mr, cleanup := testhelper.SetupTestEnvironment(t)
|
||||||
bootstrap.RegisterTasks()
|
bootstrap.RegisterTasks()
|
||||||
@@ -61,12 +75,17 @@ func setupTestRouter(authUser *model.User) *gin.Engine {
|
|||||||
adminGroup.GET("/tasks/executions", ListTaskExecutions)
|
adminGroup.GET("/tasks/executions", ListTaskExecutions)
|
||||||
adminGroup.GET("/tasks/executions/:id", GetTaskExecution)
|
adminGroup.GET("/tasks/executions/:id", GetTaskExecution)
|
||||||
adminGroup.POST("/tasks/executions/:id/retry", RetryTask)
|
adminGroup.POST("/tasks/executions/:id/retry", RetryTask)
|
||||||
|
adminGroup.GET("/tasks/schedules", ListSchedules)
|
||||||
|
adminGroup.POST("/tasks/schedules", CreateSchedule)
|
||||||
|
adminGroup.PUT("/tasks/schedules/:id", UpdateSchedule)
|
||||||
|
adminGroup.DELETE("/tasks/schedules/:id", DeleteSchedule)
|
||||||
return r
|
return r
|
||||||
}
|
}
|
||||||
|
|
||||||
func TestListTaskTypes(t *testing.T) {
|
func TestListTaskTypes(t *testing.T) {
|
||||||
cleanup := setupTaskTestEnvironment(t)
|
cleanup := setupTaskTestEnvironment(t)
|
||||||
defer cleanup()
|
defer cleanup()
|
||||||
|
registerInternalOnlyTaskMeta()
|
||||||
|
|
||||||
adminUser := &model.User{ID: 1001, Username: "admin", IsAdmin: true}
|
adminUser := &model.User{ID: 1001, Username: "admin", IsAdmin: true}
|
||||||
router := setupTestRouter(adminUser)
|
router := setupTestRouter(adminUser)
|
||||||
@@ -93,6 +112,9 @@ func TestListTaskTypes(t *testing.T) {
|
|||||||
foundCleanup := false
|
foundCleanup := false
|
||||||
foundWarmImageCache := false
|
foundWarmImageCache := false
|
||||||
for _, m := range taskMetas {
|
for _, m := range taskMetas {
|
||||||
|
if m.Type == testInternalOnlyTaskType {
|
||||||
|
t.Errorf("internal-only task type %s must not be listed", testInternalOnlyTaskType)
|
||||||
|
}
|
||||||
if m.Type == uploadtask.TaskTypeSystemCleanup {
|
if m.Type == uploadtask.TaskTypeSystemCleanup {
|
||||||
foundCleanup = true
|
foundCleanup = true
|
||||||
}
|
}
|
||||||
@@ -108,6 +130,220 @@ func TestListTaskTypes(t *testing.T) {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
func TestInternalOnlyTaskAdminBoundaries(t *testing.T) {
|
||||||
|
cleanup := setupTaskTestEnvironment(t)
|
||||||
|
defer cleanup()
|
||||||
|
registerInternalOnlyTaskMeta()
|
||||||
|
|
||||||
|
adminUser := &model.User{ID: 1001, Username: "admin", IsAdmin: true}
|
||||||
|
router := setupTestRouter(adminUser)
|
||||||
|
ctx := context.Background()
|
||||||
|
|
||||||
|
t.Run("list hides internal-only schedule", func(t *testing.T) {
|
||||||
|
internalSchedule := &model.Schedule{
|
||||||
|
Name: "隐藏的系统内部排程",
|
||||||
|
TaskType: testInternalOnlyTaskType,
|
||||||
|
Cron: "*/5 * * * *",
|
||||||
|
Payload: "{}",
|
||||||
|
IsActive: true,
|
||||||
|
}
|
||||||
|
publicSchedule := &model.Schedule{
|
||||||
|
Name: "可见的公开排程",
|
||||||
|
TaskType: uploadtask.TaskTypeSystemCleanup,
|
||||||
|
Cron: "0 * * * *",
|
||||||
|
Payload: "{}",
|
||||||
|
IsActive: true,
|
||||||
|
}
|
||||||
|
require.NoError(t, model.CreateSchedule(ctx, internalSchedule))
|
||||||
|
require.NoError(t, model.CreateSchedule(ctx, publicSchedule))
|
||||||
|
|
||||||
|
req := httptest.NewRequest(http.MethodGet, "/api/v1/admin/tasks/schedules", nil)
|
||||||
|
w := httptest.NewRecorder()
|
||||||
|
router.ServeHTTP(w, req)
|
||||||
|
|
||||||
|
assert.Equal(t, http.StatusOK, w.Code)
|
||||||
|
var resp response.Any
|
||||||
|
require.NoError(t, json.Unmarshal(w.Body.Bytes(), &resp))
|
||||||
|
data, err := json.Marshal(resp.Data)
|
||||||
|
require.NoError(t, err)
|
||||||
|
var schedules []model.Schedule
|
||||||
|
require.NoError(t, json.Unmarshal(data, &schedules))
|
||||||
|
assert.NotContains(t, scheduleIDs(schedules), internalSchedule.ID)
|
||||||
|
assert.Contains(t, scheduleIDs(schedules), publicSchedule.ID)
|
||||||
|
})
|
||||||
|
|
||||||
|
t.Run("dispatch rejects internal-only task", func(t *testing.T) {
|
||||||
|
body, err := json.Marshal(DispatchTaskRequest{TaskType: testInternalOnlyTaskType})
|
||||||
|
require.NoError(t, err)
|
||||||
|
req := httptest.NewRequest(http.MethodPost, "/api/v1/admin/tasks/dispatch", bytes.NewReader(body))
|
||||||
|
req.Header.Set("Content-Type", "application/json")
|
||||||
|
w := httptest.NewRecorder()
|
||||||
|
|
||||||
|
router.ServeHTTP(w, req)
|
||||||
|
|
||||||
|
assert.Equal(t, http.StatusBadRequest, w.Code)
|
||||||
|
var resp response.Any
|
||||||
|
require.NoError(t, json.Unmarshal(w.Body.Bytes(), &resp))
|
||||||
|
assert.Equal(t, InvalidTaskType, resp.ErrorMsg)
|
||||||
|
})
|
||||||
|
|
||||||
|
t.Run("create schedule rejects internal-only task", func(t *testing.T) {
|
||||||
|
isActive := true
|
||||||
|
body, err := json.Marshal(CreateScheduleRequest{
|
||||||
|
Name: "内部任务排程",
|
||||||
|
TaskType: testInternalOnlyTaskType,
|
||||||
|
Cron: "0 * * * *",
|
||||||
|
IsActive: &isActive,
|
||||||
|
})
|
||||||
|
require.NoError(t, err)
|
||||||
|
req := httptest.NewRequest(http.MethodPost, "/api/v1/admin/tasks/schedules", bytes.NewReader(body))
|
||||||
|
req.Header.Set("Content-Type", "application/json")
|
||||||
|
w := httptest.NewRecorder()
|
||||||
|
|
||||||
|
router.ServeHTTP(w, req)
|
||||||
|
|
||||||
|
assert.Equal(t, http.StatusBadRequest, w.Code)
|
||||||
|
var resp response.Any
|
||||||
|
require.NoError(t, json.Unmarshal(w.Body.Bytes(), &resp))
|
||||||
|
assert.Equal(t, InvalidTaskType, resp.ErrorMsg)
|
||||||
|
})
|
||||||
|
|
||||||
|
t.Run("update cannot convert existing internal schedule to public task", func(t *testing.T) {
|
||||||
|
schedule := &model.Schedule{
|
||||||
|
Name: "系统内部排程",
|
||||||
|
TaskType: testInternalOnlyTaskType,
|
||||||
|
Cron: "0 * * * *",
|
||||||
|
IsActive: true,
|
||||||
|
}
|
||||||
|
require.NoError(t, model.CreateSchedule(ctx, schedule))
|
||||||
|
isActive := false
|
||||||
|
body, err := json.Marshal(UpdateScheduleRequest{
|
||||||
|
Name: "尝试修改内部排程",
|
||||||
|
TaskType: uploadtask.TaskTypeSystemCleanup,
|
||||||
|
Cron: "5 * * * *",
|
||||||
|
IsActive: &isActive,
|
||||||
|
})
|
||||||
|
require.NoError(t, err)
|
||||||
|
req := httptest.NewRequest(
|
||||||
|
http.MethodPut,
|
||||||
|
fmt.Sprintf("/api/v1/admin/tasks/schedules/%d", schedule.ID),
|
||||||
|
bytes.NewReader(body),
|
||||||
|
)
|
||||||
|
req.Header.Set("Content-Type", "application/json")
|
||||||
|
w := httptest.NewRecorder()
|
||||||
|
|
||||||
|
router.ServeHTTP(w, req)
|
||||||
|
|
||||||
|
assert.Equal(t, http.StatusBadRequest, w.Code)
|
||||||
|
unchanged, err := model.GetScheduleByID(ctx, schedule.ID)
|
||||||
|
require.NoError(t, err)
|
||||||
|
assert.Equal(t, "系统内部排程", unchanged.Name)
|
||||||
|
assert.Equal(t, testInternalOnlyTaskType, unchanged.TaskType)
|
||||||
|
assert.True(t, unchanged.IsActive)
|
||||||
|
})
|
||||||
|
|
||||||
|
t.Run("update public schedule rejects internal-only target task", func(t *testing.T) {
|
||||||
|
schedule := &model.Schedule{
|
||||||
|
Name: "公开排程",
|
||||||
|
TaskType: uploadtask.TaskTypeSystemCleanup,
|
||||||
|
Cron: "0 * * * *",
|
||||||
|
IsActive: true,
|
||||||
|
}
|
||||||
|
require.NoError(t, model.CreateSchedule(ctx, schedule))
|
||||||
|
isActive := true
|
||||||
|
body, err := json.Marshal(UpdateScheduleRequest{
|
||||||
|
Name: "尝试切入内部任务",
|
||||||
|
TaskType: testInternalOnlyTaskType,
|
||||||
|
Cron: "10 * * * *",
|
||||||
|
IsActive: &isActive,
|
||||||
|
})
|
||||||
|
require.NoError(t, err)
|
||||||
|
req := httptest.NewRequest(
|
||||||
|
http.MethodPut,
|
||||||
|
fmt.Sprintf("/api/v1/admin/tasks/schedules/%d", schedule.ID),
|
||||||
|
bytes.NewReader(body),
|
||||||
|
)
|
||||||
|
req.Header.Set("Content-Type", "application/json")
|
||||||
|
w := httptest.NewRecorder()
|
||||||
|
|
||||||
|
router.ServeHTTP(w, req)
|
||||||
|
|
||||||
|
assert.Equal(t, http.StatusBadRequest, w.Code)
|
||||||
|
unchanged, err := model.GetScheduleByID(ctx, schedule.ID)
|
||||||
|
require.NoError(t, err)
|
||||||
|
assert.Equal(t, "公开排程", unchanged.Name)
|
||||||
|
assert.Equal(t, uploadtask.TaskTypeSystemCleanup, unchanged.TaskType)
|
||||||
|
})
|
||||||
|
|
||||||
|
t.Run("delete rejects internal-only schedule", func(t *testing.T) {
|
||||||
|
schedule := &model.Schedule{
|
||||||
|
Name: "不可删除的系统内部排程",
|
||||||
|
TaskType: testInternalOnlyTaskType,
|
||||||
|
Cron: "*/5 * * * *",
|
||||||
|
IsActive: true,
|
||||||
|
}
|
||||||
|
require.NoError(t, model.CreateSchedule(ctx, schedule))
|
||||||
|
req := httptest.NewRequest(
|
||||||
|
http.MethodDelete,
|
||||||
|
fmt.Sprintf("/api/v1/admin/tasks/schedules/%d", schedule.ID),
|
||||||
|
nil,
|
||||||
|
)
|
||||||
|
w := httptest.NewRecorder()
|
||||||
|
|
||||||
|
router.ServeHTTP(w, req)
|
||||||
|
|
||||||
|
assert.Equal(t, http.StatusBadRequest, w.Code)
|
||||||
|
var resp response.Any
|
||||||
|
require.NoError(t, json.Unmarshal(w.Body.Bytes(), &resp))
|
||||||
|
assert.Equal(t, InvalidTaskType, resp.ErrorMsg)
|
||||||
|
preserved, err := model.GetScheduleByID(ctx, schedule.ID)
|
||||||
|
require.NoError(t, err)
|
||||||
|
assert.Equal(t, testInternalOnlyTaskType, preserved.TaskType)
|
||||||
|
})
|
||||||
|
|
||||||
|
t.Run("delete missing schedule returns not found", func(t *testing.T) {
|
||||||
|
req := httptest.NewRequest(http.MethodDelete, "/api/v1/admin/tasks/schedules/999999", nil)
|
||||||
|
w := httptest.NewRecorder()
|
||||||
|
|
||||||
|
router.ServeHTTP(w, req)
|
||||||
|
|
||||||
|
assert.Equal(t, http.StatusNotFound, w.Code)
|
||||||
|
var resp response.Any
|
||||||
|
require.NoError(t, json.Unmarshal(w.Body.Bytes(), &resp))
|
||||||
|
assert.Equal(t, ScheduleNotFound, resp.ErrorMsg)
|
||||||
|
})
|
||||||
|
|
||||||
|
t.Run("delete public schedule remains allowed", func(t *testing.T) {
|
||||||
|
schedule := &model.Schedule{
|
||||||
|
Name: "可删除的公开排程",
|
||||||
|
TaskType: uploadtask.TaskTypeSystemCleanup,
|
||||||
|
Cron: "0 * * * *",
|
||||||
|
IsActive: false,
|
||||||
|
}
|
||||||
|
require.NoError(t, model.CreateSchedule(ctx, schedule))
|
||||||
|
req := httptest.NewRequest(
|
||||||
|
http.MethodDelete,
|
||||||
|
fmt.Sprintf("/api/v1/admin/tasks/schedules/%d", schedule.ID),
|
||||||
|
nil,
|
||||||
|
)
|
||||||
|
w := httptest.NewRecorder()
|
||||||
|
|
||||||
|
router.ServeHTTP(w, req)
|
||||||
|
|
||||||
|
assert.Equal(t, http.StatusOK, w.Code)
|
||||||
|
_, err := model.GetScheduleByID(ctx, schedule.ID)
|
||||||
|
assert.Error(t, err)
|
||||||
|
})
|
||||||
|
}
|
||||||
|
|
||||||
|
func scheduleIDs(schedules []model.Schedule) []uint64 {
|
||||||
|
ids := make([]uint64, 0, len(schedules))
|
||||||
|
for _, schedule := range schedules {
|
||||||
|
ids = append(ids, schedule.ID)
|
||||||
|
}
|
||||||
|
return ids
|
||||||
|
}
|
||||||
|
|
||||||
func TestDispatchTask(t *testing.T) {
|
func TestDispatchTask(t *testing.T) {
|
||||||
cleanup := setupTaskTestEnvironment(t)
|
cleanup := setupTaskTestEnvironment(t)
|
||||||
defer cleanup()
|
defer cleanup()
|
||||||
|
|||||||
@@ -1,9 +1,13 @@
|
|||||||
|
// Copyright 2026 Arctel.net
|
||||||
|
// SPDX-License-Identifier: Apache-2.0
|
||||||
|
|
||||||
// Package httpclient provides an authenticated HTTP client for the agent.
|
// Package httpclient provides an authenticated HTTP client for the agent.
|
||||||
package httpclient
|
package httpclient
|
||||||
|
|
||||||
import (
|
import (
|
||||||
"context"
|
"context"
|
||||||
"encoding/json"
|
"encoding/json"
|
||||||
|
"errors"
|
||||||
"fmt"
|
"fmt"
|
||||||
"io"
|
"io"
|
||||||
"net/http"
|
"net/http"
|
||||||
@@ -13,6 +17,8 @@ import (
|
|||||||
edgehttp "github.com/Rain-kl/Wavelet/internal/apps/edge/httpclient"
|
edgehttp "github.com/Rain-kl/Wavelet/internal/apps/edge/httpclient"
|
||||||
)
|
)
|
||||||
|
|
||||||
|
const pagesControlResponseMaxBytes = int64(64 * 1024)
|
||||||
|
|
||||||
// Client is a HTTP client used by the agent to communicate with the control plane server.
|
// Client is a HTTP client used by the agent to communicate with the control plane server.
|
||||||
type Client struct {
|
type Client struct {
|
||||||
base *edgehttp.Client
|
base *edgehttp.Client
|
||||||
@@ -98,23 +104,43 @@ func (c *Client) GetPagesDeploymentHash(ctx context.Context, deploymentID uint)
|
|||||||
return resp.Data.Hash, nil
|
return resp.Data.Hash, nil
|
||||||
}
|
}
|
||||||
|
|
||||||
// DownloadPagesDeploymentPackage downloads the deployment package for the given Pages deployment ID.
|
// DownloadPagesDeploymentPackage streams the deployment package into dst while
|
||||||
func (c *Client) DownloadPagesDeploymentPackage(ctx context.Context, deploymentID uint) ([]byte, error) {
|
// enforcing maxBytes against both advertised and actual response sizes.
|
||||||
res, err := c.base.DoRaw(ctx, http.MethodGet, fmt.Sprintf("/api/v1/agent/pages/deployments/%d/package", deploymentID), nil)
|
func (c *Client) DownloadPagesDeploymentPackage(
|
||||||
if err != nil {
|
ctx context.Context,
|
||||||
return nil, err
|
deploymentID uint,
|
||||||
}
|
dst io.Writer,
|
||||||
defer func() { _ = res.Body.Close() }()
|
maxBytes int64,
|
||||||
if res.StatusCode != http.StatusOK {
|
) (int64, error) {
|
||||||
return nil, edgehttp.ReadHTTPError(res)
|
return c.downloadPagesPackage(
|
||||||
}
|
ctx,
|
||||||
return io.ReadAll(res.Body)
|
fmt.Sprintf("/api/v1/agent/pages/deployments/%d/package", deploymentID),
|
||||||
|
dst,
|
||||||
|
maxBytes,
|
||||||
|
)
|
||||||
}
|
}
|
||||||
|
|
||||||
// GetPagesProjectLatestHash returns the active deployment package hash for a Pages project.
|
// GetPagesProjectLatestHash returns the active deployment package hash for a Pages project.
|
||||||
func (c *Client) GetPagesProjectLatestHash(ctx context.Context, projectID uint) (*protocol.PagesProjectLatestHashResponse, error) {
|
func (c *Client) GetPagesProjectLatestHash(ctx context.Context, projectID uint) (*protocol.PagesProjectLatestHashResponse, error) {
|
||||||
|
res, err := c.base.DoRaw(
|
||||||
|
ctx,
|
||||||
|
http.MethodGet,
|
||||||
|
fmt.Sprintf("/api/v1/agent/pages/projects/%d/latest/hash", projectID),
|
||||||
|
nil,
|
||||||
|
)
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
defer func() { _ = res.Body.Close() }()
|
||||||
|
body, err := readPagesControlResponse(res, pagesControlResponseMaxBytes)
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
if res.StatusCode != http.StatusOK {
|
||||||
|
return nil, edgehttp.ReadBodyError(body, res.Status)
|
||||||
|
}
|
||||||
resp := protocol.APIResponse[protocol.PagesProjectLatestHashResponse]{}
|
resp := protocol.APIResponse[protocol.PagesProjectLatestHashResponse]{}
|
||||||
if err := c.base.GetJSON(ctx, fmt.Sprintf("/api/v1/agent/pages/projects/%d/latest/hash", projectID), &resp); err != nil {
|
if err := json.Unmarshal(body, &resp); err != nil {
|
||||||
return nil, err
|
return nil, err
|
||||||
}
|
}
|
||||||
if err := edgehttp.APIError(resp.ErrorMsg); err != nil {
|
if err := edgehttp.APIError(resp.ErrorMsg); err != nil {
|
||||||
@@ -123,17 +149,86 @@ func (c *Client) GetPagesProjectLatestHash(ctx context.Context, projectID uint)
|
|||||||
return &resp.Data, nil
|
return &resp.Data, nil
|
||||||
}
|
}
|
||||||
|
|
||||||
// DownloadPagesProjectLatestPackage downloads the active deployment package for a Pages project.
|
// DownloadPagesProjectLatestPackage streams the active deployment package into
|
||||||
func (c *Client) DownloadPagesProjectLatestPackage(ctx context.Context, projectID uint) ([]byte, error) {
|
// dst while enforcing maxBytes against both advertised and actual sizes.
|
||||||
res, err := c.base.DoRaw(ctx, http.MethodGet, fmt.Sprintf("/api/v1/agent/pages/projects/%d/latest/package", projectID), nil)
|
func (c *Client) DownloadPagesProjectLatestPackage(
|
||||||
|
ctx context.Context,
|
||||||
|
projectID uint,
|
||||||
|
dst io.Writer,
|
||||||
|
maxBytes int64,
|
||||||
|
) (int64, error) {
|
||||||
|
return c.downloadPagesPackage(
|
||||||
|
ctx,
|
||||||
|
fmt.Sprintf("/api/v1/agent/pages/projects/%d/latest/package", projectID),
|
||||||
|
dst,
|
||||||
|
maxBytes,
|
||||||
|
)
|
||||||
|
}
|
||||||
|
|
||||||
|
func (c *Client) downloadPagesPackage(
|
||||||
|
ctx context.Context,
|
||||||
|
path string,
|
||||||
|
dst io.Writer,
|
||||||
|
maxBytes int64,
|
||||||
|
) (int64, error) {
|
||||||
|
if dst == nil {
|
||||||
|
return 0, errors.New("pages package destination is required")
|
||||||
|
}
|
||||||
|
if maxBytes <= 0 {
|
||||||
|
return 0, errors.New("pages package byte limit must be positive")
|
||||||
|
}
|
||||||
|
res, err := c.base.DoRaw(ctx, http.MethodGet, path, nil)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
return nil, err
|
return 0, err
|
||||||
}
|
}
|
||||||
defer func() { _ = res.Body.Close() }()
|
defer func() { _ = res.Body.Close() }()
|
||||||
if res.StatusCode != http.StatusOK {
|
if res.StatusCode != http.StatusOK {
|
||||||
return nil, edgehttp.ReadHTTPError(res)
|
body, readErr := readPagesControlResponse(res, pagesControlResponseMaxBytes)
|
||||||
|
if readErr != nil {
|
||||||
|
return 0, readErr
|
||||||
|
}
|
||||||
|
return 0, edgehttp.ReadBodyError(body, res.Status)
|
||||||
}
|
}
|
||||||
return io.ReadAll(res.Body)
|
return copyPagesPackageResponse(dst, res, maxBytes)
|
||||||
|
}
|
||||||
|
|
||||||
|
func readPagesControlResponse(res *http.Response, maxBytes int64) ([]byte, error) {
|
||||||
|
if res.ContentLength > maxBytes {
|
||||||
|
return nil, fmt.Errorf(
|
||||||
|
"pages control response Content-Length %d exceeds limit %d",
|
||||||
|
res.ContentLength,
|
||||||
|
maxBytes,
|
||||||
|
)
|
||||||
|
}
|
||||||
|
limited := &io.LimitedReader{R: res.Body, N: maxBytes + 1}
|
||||||
|
body, err := io.ReadAll(limited)
|
||||||
|
if err != nil {
|
||||||
|
return nil, fmt.Errorf("read pages control response: %w", err)
|
||||||
|
}
|
||||||
|
if int64(len(body)) > maxBytes {
|
||||||
|
return nil, fmt.Errorf("pages control response body exceeds limit %d", maxBytes)
|
||||||
|
}
|
||||||
|
return body, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func copyPagesPackageResponse(dst io.Writer, res *http.Response, maxBytes int64) (int64, error) {
|
||||||
|
if res.ContentLength > maxBytes {
|
||||||
|
return 0, fmt.Errorf(
|
||||||
|
"pages package Content-Length %d exceeds limit %d",
|
||||||
|
res.ContentLength,
|
||||||
|
maxBytes,
|
||||||
|
)
|
||||||
|
}
|
||||||
|
|
||||||
|
limited := &io.LimitedReader{R: res.Body, N: maxBytes + 1}
|
||||||
|
written, err := io.Copy(dst, limited)
|
||||||
|
if err != nil {
|
||||||
|
return written, fmt.Errorf("stream pages package: %w", err)
|
||||||
|
}
|
||||||
|
if written > maxBytes {
|
||||||
|
return written, fmt.Errorf("pages package body exceeds limit %d", maxBytes)
|
||||||
|
}
|
||||||
|
return written, nil
|
||||||
}
|
}
|
||||||
|
|
||||||
// SetToken updates the authentication token used for API requests.
|
// SetToken updates the authentication token used for API requests.
|
||||||
|
|||||||
@@ -0,0 +1,109 @@
|
|||||||
|
// Copyright 2026 Arctel.net
|
||||||
|
// SPDX-License-Identifier: Apache-2.0
|
||||||
|
|
||||||
|
package httpclient
|
||||||
|
|
||||||
|
import (
|
||||||
|
"bytes"
|
||||||
|
"context"
|
||||||
|
"io"
|
||||||
|
"net/http"
|
||||||
|
"net/http/httptest"
|
||||||
|
"strings"
|
||||||
|
"testing"
|
||||||
|
"time"
|
||||||
|
)
|
||||||
|
|
||||||
|
func TestDownloadPagesProjectLatestPackageRejectsChunkedBodyOverLimit(t *testing.T) {
|
||||||
|
server := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, _ *http.Request) {
|
||||||
|
w.WriteHeader(http.StatusOK)
|
||||||
|
if flusher, ok := w.(http.Flusher); ok {
|
||||||
|
flusher.Flush()
|
||||||
|
}
|
||||||
|
_, _ = io.WriteString(w, "123456")
|
||||||
|
}))
|
||||||
|
defer server.Close()
|
||||||
|
|
||||||
|
client := New(server.URL, "test-token", time.Second)
|
||||||
|
var dst bytes.Buffer
|
||||||
|
written, err := client.DownloadPagesProjectLatestPackage(
|
||||||
|
context.Background(),
|
||||||
|
7,
|
||||||
|
&dst,
|
||||||
|
4,
|
||||||
|
)
|
||||||
|
if err == nil || !strings.Contains(err.Error(), "body exceeds limit") {
|
||||||
|
t.Fatalf("DownloadPagesProjectLatestPackage(chunked, limit=4) error = %v, want body limit error", err)
|
||||||
|
}
|
||||||
|
if written != 5 {
|
||||||
|
t.Errorf("DownloadPagesProjectLatestPackage(chunked, limit=4) written = %d, want 5", written)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestCopyPagesPackageResponseRejectsAdvertisedContentLengthBeforeWrite(t *testing.T) {
|
||||||
|
response := &http.Response{
|
||||||
|
Body: io.NopCloser(strings.NewReader("123456")),
|
||||||
|
ContentLength: 6,
|
||||||
|
}
|
||||||
|
var dst bytes.Buffer
|
||||||
|
written, err := copyPagesPackageResponse(&dst, response, 4)
|
||||||
|
if err == nil || !strings.Contains(err.Error(), "Content-Length") {
|
||||||
|
t.Fatalf("copyPagesPackageResponse(Content-Length=6, limit=4) error = %v, want Content-Length limit error", err)
|
||||||
|
}
|
||||||
|
if written != 0 || dst.Len() != 0 {
|
||||||
|
t.Errorf("copyPagesPackageResponse(Content-Length=6, limit=4) wrote (%d, %d buffered), want no writes", written, dst.Len())
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestCopyPagesPackageResponseRejectsForgedSmallContentLength(t *testing.T) {
|
||||||
|
response := &http.Response{
|
||||||
|
Body: io.NopCloser(strings.NewReader("123456")),
|
||||||
|
ContentLength: 2,
|
||||||
|
}
|
||||||
|
var dst bytes.Buffer
|
||||||
|
written, err := copyPagesPackageResponse(&dst, response, 4)
|
||||||
|
if err == nil || !strings.Contains(err.Error(), "body exceeds limit") {
|
||||||
|
t.Fatalf("copyPagesPackageResponse(forged Content-Length=2, limit=4) error = %v, want body limit error", err)
|
||||||
|
}
|
||||||
|
if written != 5 {
|
||||||
|
t.Errorf("copyPagesPackageResponse(forged Content-Length=2, limit=4) written = %d, want 5", written)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestDownloadPagesProjectLatestPackageBoundsChunkedErrorResponse(t *testing.T) {
|
||||||
|
server := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, _ *http.Request) {
|
||||||
|
w.WriteHeader(http.StatusBadRequest)
|
||||||
|
if flusher, ok := w.(http.Flusher); ok {
|
||||||
|
flusher.Flush()
|
||||||
|
}
|
||||||
|
_, _ = io.WriteString(w, strings.Repeat("x", int(pagesControlResponseMaxBytes+1)))
|
||||||
|
}))
|
||||||
|
defer server.Close()
|
||||||
|
|
||||||
|
client := New(server.URL, "test-token", time.Second)
|
||||||
|
var dst bytes.Buffer
|
||||||
|
_, err := client.DownloadPagesProjectLatestPackage(context.Background(), 7, &dst, 1024)
|
||||||
|
if err == nil || !strings.Contains(err.Error(), "control response body exceeds limit") {
|
||||||
|
t.Fatalf("DownloadPagesProjectLatestPackage(large chunked 400) error = %v, want bounded response error", err)
|
||||||
|
}
|
||||||
|
if dst.Len() != 0 {
|
||||||
|
t.Errorf("DownloadPagesProjectLatestPackage(large chunked 400) wrote %d package bytes, want 0", dst.Len())
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestGetPagesProjectLatestHashBoundsChunkedMetadataResponse(t *testing.T) {
|
||||||
|
server := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, _ *http.Request) {
|
||||||
|
w.WriteHeader(http.StatusOK)
|
||||||
|
if flusher, ok := w.(http.Flusher); ok {
|
||||||
|
flusher.Flush()
|
||||||
|
}
|
||||||
|
_, _ = io.WriteString(w, strings.Repeat("x", int(pagesControlResponseMaxBytes+1)))
|
||||||
|
}))
|
||||||
|
defer server.Close()
|
||||||
|
|
||||||
|
client := New(server.URL, "test-token", time.Second)
|
||||||
|
_, err := client.GetPagesProjectLatestHash(context.Background(), 7)
|
||||||
|
if err == nil || !strings.Contains(err.Error(), "control response body exceeds limit") {
|
||||||
|
t.Fatalf("GetPagesProjectLatestHash(large chunked metadata) error = %v, want bounded response error", err)
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -1,3 +1,6 @@
|
|||||||
|
// Copyright 2026 Arctel.net
|
||||||
|
// SPDX-License-Identifier: Apache-2.0
|
||||||
|
|
||||||
// Package sync applies control-plane configuration to the local agent runtime.
|
// Package sync applies control-plane configuration to the local agent runtime.
|
||||||
package sync
|
package sync
|
||||||
|
|
||||||
@@ -20,9 +23,13 @@ import (
|
|||||||
)
|
)
|
||||||
|
|
||||||
const (
|
const (
|
||||||
pagesDirPerm = 0o755
|
pagesDirPerm = 0o755
|
||||||
pagesFilePerm = 0o644
|
pagesFilePerm = 0o644
|
||||||
pagesManifestFilePerm = 0o644
|
pagesManifestFilePerm = 0o644
|
||||||
|
agentPagesMaxPackageBytes = int64(2 * 1024 * 1024 * 1024)
|
||||||
|
agentPagesMaxFiles = 1000
|
||||||
|
agentPagesMaxFileBytes = int64(8 * 1024 * 1024 * 1024)
|
||||||
|
agentPagesMaxTotalBytes = int64(8 * 1024 * 1024 * 1024)
|
||||||
// pagesLatestPullAttempts covers a race where the active deployment changes
|
// pagesLatestPullAttempts covers a race where the active deployment changes
|
||||||
// between the hash probe and the package download.
|
// between the hash probe and the package download.
|
||||||
pagesLatestPullAttempts = 2
|
pagesLatestPullAttempts = 2
|
||||||
@@ -45,6 +52,11 @@ type pagesProjectRef struct {
|
|||||||
Checksum string
|
Checksum string
|
||||||
}
|
}
|
||||||
|
|
||||||
|
type pagesPackageLimits struct {
|
||||||
|
PackageBytes int64
|
||||||
|
Extraction pagesarchive.Limits
|
||||||
|
}
|
||||||
|
|
||||||
type pagesDeploymentMarker struct {
|
type pagesDeploymentMarker struct {
|
||||||
ProjectID uint `json:"project_id"`
|
ProjectID uint `json:"project_id"`
|
||||||
DeploymentID uint `json:"deployment_id,omitempty"`
|
DeploymentID uint `json:"deployment_id,omitempty"`
|
||||||
@@ -191,10 +203,11 @@ func (s *Service) ensurePagesProject(ctx context.Context, snapshot *state.Snapsh
|
|||||||
if err != nil {
|
if err != nil {
|
||||||
return fmt.Errorf("fetch Pages project %d latest hash: %w", projectID, err)
|
return fmt.Errorf("fetch Pages project %d latest hash: %w", projectID, err)
|
||||||
}
|
}
|
||||||
hash := strings.TrimSpace(latest.Hash)
|
limits, err := validatePagesPackageMetadata(projectID, latest)
|
||||||
if hash == "" {
|
if err != nil {
|
||||||
return fmt.Errorf("pages project %d latest hash is empty", projectID)
|
return err
|
||||||
}
|
}
|
||||||
|
hash := strings.TrimSpace(latest.Hash)
|
||||||
effective := pagesProjectRef{
|
effective := pagesProjectRef{
|
||||||
ProjectID: projectID,
|
ProjectID: projectID,
|
||||||
DeploymentID: latest.DeploymentID,
|
DeploymentID: latest.DeploymentID,
|
||||||
@@ -211,44 +224,65 @@ func (s *Service) ensurePagesProject(ctx context.Context, snapshot *state.Snapsh
|
|||||||
return nil
|
return nil
|
||||||
}
|
}
|
||||||
|
|
||||||
packageBytes, err := s.client.DownloadPagesProjectLatestPackage(ctx, projectID)
|
packagePath, got, err := s.downloadPagesProjectPackage(ctx, projectID, latest, limits.PackageBytes)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
return fmt.Errorf("download Pages project %d latest package: %w", projectID, err)
|
return fmt.Errorf("download Pages project %d latest package: %w", projectID, err)
|
||||||
}
|
}
|
||||||
got := checksumBytes(packageBytes)
|
|
||||||
|
|
||||||
// Re-probe latest after download to detect activation races.
|
// Re-probe latest after download to detect activation races.
|
||||||
// Accept the package only when its content hash still matches latest.
|
// A deployment-id-only change is still a latest-pointer race even when
|
||||||
|
// deduplication makes both deployments share the same package hash.
|
||||||
verify, err := s.client.GetPagesProjectLatestHash(ctx, projectID)
|
verify, err := s.client.GetPagesProjectLatestHash(ctx, projectID)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
|
_ = os.Remove(packagePath)
|
||||||
return fmt.Errorf("re-fetch Pages project %d latest hash: %w", projectID, err)
|
return fmt.Errorf("re-fetch Pages project %d latest hash: %w", projectID, err)
|
||||||
}
|
}
|
||||||
verifyHash := strings.TrimSpace(verify.Hash)
|
if _, err := validatePagesPackageMetadata(projectID, verify); err != nil {
|
||||||
if verifyHash == "" {
|
_ = os.Remove(packagePath)
|
||||||
return fmt.Errorf("pages project %d latest hash is empty", projectID)
|
return err
|
||||||
}
|
}
|
||||||
if got != verifyHash {
|
if !samePagesPackageMetadata(latest, verify) {
|
||||||
|
_ = os.Remove(packagePath)
|
||||||
lastErr = fmt.Errorf(
|
lastErr = fmt.Errorf(
|
||||||
"pages project %d package/hash race: downloaded %s, latest now %s (attempt %d/%d)",
|
"pages project %d latest metadata changed during download: deployment %d/%s -> %d/%s (attempt %d/%d)",
|
||||||
projectID, got, verifyHash, attempt+1, pagesLatestPullAttempts,
|
projectID,
|
||||||
|
latest.DeploymentID,
|
||||||
|
strings.TrimSpace(latest.Hash),
|
||||||
|
verify.DeploymentID,
|
||||||
|
strings.TrimSpace(verify.Hash),
|
||||||
|
attempt+1,
|
||||||
|
pagesLatestPullAttempts,
|
||||||
)
|
)
|
||||||
slog.Warn("pages latest package race, retrying",
|
slog.Warn("pages latest metadata race, retrying",
|
||||||
|
"project_id", projectID,
|
||||||
|
"before_deployment_id", latest.DeploymentID,
|
||||||
|
"before_hash", strings.TrimSpace(latest.Hash),
|
||||||
|
"after_deployment_id", verify.DeploymentID,
|
||||||
|
"after_hash", strings.TrimSpace(verify.Hash),
|
||||||
|
"attempt", attempt+1,
|
||||||
|
)
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
if got != hash {
|
||||||
|
_ = os.Remove(packagePath)
|
||||||
|
lastErr = fmt.Errorf(
|
||||||
|
"pages project %d package hash mismatch: downloaded %s, expected %s (attempt %d/%d)",
|
||||||
|
projectID, got, hash, attempt+1, pagesLatestPullAttempts,
|
||||||
|
)
|
||||||
|
slog.Warn("pages latest package hash mismatch, retrying",
|
||||||
"project_id", projectID,
|
"project_id", projectID,
|
||||||
"downloaded_hash", got,
|
"downloaded_hash", got,
|
||||||
"latest_hash", verifyHash,
|
"expected_hash", hash,
|
||||||
"attempt", attempt+1,
|
"attempt", attempt+1,
|
||||||
)
|
)
|
||||||
continue
|
continue
|
||||||
}
|
}
|
||||||
|
|
||||||
effective = pagesProjectRef{
|
|
||||||
ProjectID: projectID,
|
|
||||||
DeploymentID: verify.DeploymentID,
|
|
||||||
Checksum: got,
|
|
||||||
}
|
|
||||||
releaseDir = pagesProjectReleaseDir(s.pagesDir, projectID, got)
|
releaseDir = pagesProjectReleaseDir(s.pagesDir, projectID, got)
|
||||||
if err := extractPagesPackage(packageBytes, releaseDir, effective); err != nil {
|
extractErr := extractPagesPackageFile(packagePath, releaseDir, effective, limits.Extraction, latest)
|
||||||
return err
|
_ = os.Remove(packagePath)
|
||||||
|
if extractErr != nil {
|
||||||
|
return extractErr
|
||||||
}
|
}
|
||||||
if err := switchPagesProjectCurrentDir(s.pagesDir, projectID, releaseDir); err != nil {
|
if err := switchPagesProjectCurrentDir(s.pagesDir, projectID, releaseDir); err != nil {
|
||||||
return err
|
return err
|
||||||
@@ -264,6 +298,145 @@ func (s *Service) ensurePagesProject(ctx context.Context, snapshot *state.Snapsh
|
|||||||
return fmt.Errorf("pages project %d latest pull failed", projectID)
|
return fmt.Errorf("pages project %d latest pull failed", projectID)
|
||||||
}
|
}
|
||||||
|
|
||||||
|
func validatePagesPackageMetadata(
|
||||||
|
projectID uint,
|
||||||
|
metadata *protocol.PagesProjectLatestHashResponse,
|
||||||
|
) (pagesPackageLimits, error) {
|
||||||
|
if metadata == nil {
|
||||||
|
return pagesPackageLimits{}, fmt.Errorf("pages project %d latest metadata is missing", projectID)
|
||||||
|
}
|
||||||
|
if metadata.ProjectID != projectID {
|
||||||
|
return pagesPackageLimits{}, fmt.Errorf(
|
||||||
|
"pages project %d latest metadata has project id %d",
|
||||||
|
projectID,
|
||||||
|
metadata.ProjectID,
|
||||||
|
)
|
||||||
|
}
|
||||||
|
if metadata.DeploymentID == 0 {
|
||||||
|
return pagesPackageLimits{}, fmt.Errorf("pages project %d latest deployment id is missing", projectID)
|
||||||
|
}
|
||||||
|
if strings.TrimSpace(metadata.Hash) == "" {
|
||||||
|
return pagesPackageLimits{}, fmt.Errorf("pages project %d latest hash is empty", projectID)
|
||||||
|
}
|
||||||
|
if metadata.PackageSize <= 0 {
|
||||||
|
return pagesPackageLimits{}, fmt.Errorf("pages project %d package size must be positive", projectID)
|
||||||
|
}
|
||||||
|
if metadata.PackageSize > agentPagesMaxPackageBytes {
|
||||||
|
return pagesPackageLimits{}, fmt.Errorf(
|
||||||
|
"pages project %d package size %d exceeds agent limit %d",
|
||||||
|
projectID,
|
||||||
|
metadata.PackageSize,
|
||||||
|
agentPagesMaxPackageBytes,
|
||||||
|
)
|
||||||
|
}
|
||||||
|
if metadata.FileCount <= 0 {
|
||||||
|
return pagesPackageLimits{}, fmt.Errorf("pages project %d file count must be positive", projectID)
|
||||||
|
}
|
||||||
|
if metadata.FileCount > agentPagesMaxFiles {
|
||||||
|
return pagesPackageLimits{}, fmt.Errorf(
|
||||||
|
"pages project %d file count %d exceeds agent limit %d",
|
||||||
|
projectID,
|
||||||
|
metadata.FileCount,
|
||||||
|
agentPagesMaxFiles,
|
||||||
|
)
|
||||||
|
}
|
||||||
|
if metadata.TotalSize < 0 {
|
||||||
|
return pagesPackageLimits{}, fmt.Errorf("pages project %d total size cannot be negative", projectID)
|
||||||
|
}
|
||||||
|
if metadata.TotalSize > agentPagesMaxTotalBytes {
|
||||||
|
return pagesPackageLimits{}, fmt.Errorf(
|
||||||
|
"pages project %d total size %d exceeds agent limit %d",
|
||||||
|
projectID,
|
||||||
|
metadata.TotalSize,
|
||||||
|
agentPagesMaxTotalBytes,
|
||||||
|
)
|
||||||
|
}
|
||||||
|
|
||||||
|
// pagesarchive treats zero limits as defaults. A one-byte extraction guard
|
||||||
|
// plus the exact post-extraction manifest check below preserves the valid
|
||||||
|
// case of one or more zero-byte files while still enforcing total_size=0.
|
||||||
|
extractedBytes := metadata.TotalSize
|
||||||
|
if extractedBytes == 0 {
|
||||||
|
extractedBytes = 1
|
||||||
|
}
|
||||||
|
maxFileBytes := extractedBytes
|
||||||
|
if maxFileBytes > agentPagesMaxFileBytes {
|
||||||
|
maxFileBytes = agentPagesMaxFileBytes
|
||||||
|
}
|
||||||
|
|
||||||
|
return pagesPackageLimits{
|
||||||
|
PackageBytes: metadata.PackageSize,
|
||||||
|
Extraction: pagesarchive.Limits{
|
||||||
|
MaxFiles: metadata.FileCount,
|
||||||
|
MaxFileBytes: maxFileBytes,
|
||||||
|
MaxTotalBytes: extractedBytes,
|
||||||
|
},
|
||||||
|
}, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func samePagesPackageMetadata(
|
||||||
|
before *protocol.PagesProjectLatestHashResponse,
|
||||||
|
after *protocol.PagesProjectLatestHashResponse,
|
||||||
|
) bool {
|
||||||
|
if before == nil || after == nil {
|
||||||
|
return false
|
||||||
|
}
|
||||||
|
return before.ProjectID == after.ProjectID &&
|
||||||
|
before.DeploymentID == after.DeploymentID &&
|
||||||
|
strings.TrimSpace(before.Hash) == strings.TrimSpace(after.Hash) &&
|
||||||
|
before.PackageSize == after.PackageSize &&
|
||||||
|
before.FileCount == after.FileCount &&
|
||||||
|
before.TotalSize == after.TotalSize
|
||||||
|
}
|
||||||
|
|
||||||
|
func (s *Service) downloadPagesProjectPackage(
|
||||||
|
ctx context.Context,
|
||||||
|
projectID uint,
|
||||||
|
metadata *protocol.PagesProjectLatestHashResponse,
|
||||||
|
maxBytes int64,
|
||||||
|
) (packagePath string, hash string, err error) {
|
||||||
|
releasesRoot := filepath.Join(s.pagesDir, "projects", fmt.Sprintf("%d", projectID), "releases")
|
||||||
|
if err := os.MkdirAll(releasesRoot, pagesDirPerm); err != nil {
|
||||||
|
return "", "", err
|
||||||
|
}
|
||||||
|
packageFile, err := os.CreateTemp(releasesRoot, ".package-*.tmp")
|
||||||
|
if err != nil {
|
||||||
|
return "", "", err
|
||||||
|
}
|
||||||
|
packagePath = packageFile.Name()
|
||||||
|
keep := false
|
||||||
|
defer func() {
|
||||||
|
if closeErr := packageFile.Close(); err == nil && closeErr != nil {
|
||||||
|
err = closeErr
|
||||||
|
}
|
||||||
|
if !keep || err != nil {
|
||||||
|
_ = os.Remove(packagePath)
|
||||||
|
packagePath = ""
|
||||||
|
}
|
||||||
|
}()
|
||||||
|
|
||||||
|
hasher := sha256.New()
|
||||||
|
written, err := s.client.DownloadPagesProjectLatestPackage(
|
||||||
|
ctx,
|
||||||
|
projectID,
|
||||||
|
io.MultiWriter(packageFile, hasher),
|
||||||
|
maxBytes,
|
||||||
|
)
|
||||||
|
if err != nil {
|
||||||
|
return "", "", err
|
||||||
|
}
|
||||||
|
if written != metadata.PackageSize {
|
||||||
|
return "", "", fmt.Errorf(
|
||||||
|
"pages project %d package size %d does not match metadata %d",
|
||||||
|
projectID,
|
||||||
|
written,
|
||||||
|
metadata.PackageSize,
|
||||||
|
)
|
||||||
|
}
|
||||||
|
keep = true
|
||||||
|
return packagePath, hex.EncodeToString(hasher.Sum(nil)), nil
|
||||||
|
}
|
||||||
|
|
||||||
// cleanupPagesProjectStaleReleases keeps only keepHash under projects/{id}/releases.
|
// cleanupPagesProjectStaleReleases keeps only keepHash under projects/{id}/releases.
|
||||||
// Must be called only after the keepHash release is ready and current points at it.
|
// Must be called only after the keepHash release is ready and current points at it.
|
||||||
func cleanupPagesProjectStaleReleases(baseDir string, projectID uint, keepHash string) error {
|
func cleanupPagesProjectStaleReleases(baseDir string, projectID uint, keepHash string) error {
|
||||||
@@ -372,40 +545,258 @@ type pagesDeploymentSource struct {
|
|||||||
Checksum string `json:"checksum"`
|
Checksum string `json:"checksum"`
|
||||||
}
|
}
|
||||||
|
|
||||||
func extractPagesPackage(packageBytes []byte, releaseDir string, project pagesProjectRef) error {
|
func extractPagesPackageFile(
|
||||||
tmpDir := releaseDir + ".tmp"
|
packagePath string,
|
||||||
_ = os.RemoveAll(tmpDir)
|
releaseDir string,
|
||||||
if err := os.MkdirAll(tmpDir, pagesDirPerm); err != nil {
|
project pagesProjectRef,
|
||||||
|
limits pagesarchive.Limits,
|
||||||
|
expected *protocol.PagesProjectLatestHashResponse,
|
||||||
|
) error {
|
||||||
|
if err := os.MkdirAll(filepath.Dir(releaseDir), pagesDirPerm); err != nil {
|
||||||
return err
|
return err
|
||||||
}
|
}
|
||||||
format, err := pagesarchive.DetectFormat("", packageBytes)
|
stagingDir, err := os.MkdirTemp(
|
||||||
|
filepath.Dir(releaseDir),
|
||||||
|
"."+filepath.Base(releaseDir)+"-*.tmp",
|
||||||
|
)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
_ = os.RemoveAll(tmpDir)
|
return err
|
||||||
return fmt.Errorf("detect Pages package format: %w", err)
|
|
||||||
}
|
}
|
||||||
// Control plane already inspected and accepted this package.
|
cleanupStaging := true
|
||||||
if err := pagesarchive.ExtractBytes(packageBytes, format, tmpDir, pagesarchive.ExtractOptions{
|
defer func() {
|
||||||
|
if cleanupStaging {
|
||||||
|
removePagesStagingUnlessCurrent(stagingDir, pagesCurrentDirFromRelease(releaseDir))
|
||||||
|
}
|
||||||
|
}()
|
||||||
|
|
||||||
|
if err := pagesarchive.ExtractFile(packagePath, "", stagingDir, pagesarchive.ExtractOptions{
|
||||||
StripCommonRoot: true,
|
StripCommonRoot: true,
|
||||||
EnforceLimits: false,
|
EnforceLimits: true,
|
||||||
|
Limits: limits,
|
||||||
}); err != nil {
|
}); err != nil {
|
||||||
_ = os.RemoveAll(tmpDir)
|
|
||||||
return fmt.Errorf("extract Pages package: %w", err)
|
return fmt.Errorf("extract Pages package: %w", err)
|
||||||
}
|
}
|
||||||
if err := writePagesMarker(tmpDir, project); err != nil {
|
if err := validateExtractedPagesMetadata(stagingDir, expected); err != nil {
|
||||||
_ = os.RemoveAll(tmpDir)
|
|
||||||
return err
|
return err
|
||||||
}
|
}
|
||||||
_ = os.RemoveAll(releaseDir)
|
if err := writePagesMarker(stagingDir, project); err != nil {
|
||||||
return os.Rename(tmpDir, releaseDir)
|
return err
|
||||||
|
}
|
||||||
|
if err := promotePagesRelease(stagingDir, releaseDir, project); err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
cleanupStaging = false
|
||||||
|
return nil
|
||||||
}
|
}
|
||||||
|
|
||||||
func switchPagesProjectCurrentDir(baseDir string, projectID uint, releaseDir string) error {
|
func validateExtractedPagesMetadata(
|
||||||
currentDir := pagesProjectCurrentDir(baseDir, projectID)
|
dir string,
|
||||||
previousDir := currentDir + ".previous"
|
expected *protocol.PagesProjectLatestHashResponse,
|
||||||
_ = os.RemoveAll(previousDir)
|
) error {
|
||||||
|
if expected == nil {
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
fileCount := 0
|
||||||
|
totalSize := int64(0)
|
||||||
|
err := filepath.WalkDir(dir, func(path string, entry os.DirEntry, walkErr error) error {
|
||||||
|
if walkErr != nil {
|
||||||
|
return walkErr
|
||||||
|
}
|
||||||
|
if entry.IsDir() {
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
info, err := entry.Info()
|
||||||
|
if err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
if !info.Mode().IsRegular() {
|
||||||
|
return fmt.Errorf("pages extracted entry is not a regular file: %s", path)
|
||||||
|
}
|
||||||
|
fileCount++
|
||||||
|
if fileCount > agentPagesMaxFiles {
|
||||||
|
return fmt.Errorf("pages extracted file count exceeds agent limit %d", agentPagesMaxFiles)
|
||||||
|
}
|
||||||
|
if info.Size() < 0 || info.Size() > agentPagesMaxTotalBytes-totalSize {
|
||||||
|
return fmt.Errorf("pages extracted size exceeds agent limit %d", agentPagesMaxTotalBytes)
|
||||||
|
}
|
||||||
|
totalSize += info.Size()
|
||||||
|
return nil
|
||||||
|
})
|
||||||
|
if err != nil {
|
||||||
|
return fmt.Errorf("validate extracted Pages package: %w", err)
|
||||||
|
}
|
||||||
|
if fileCount != expected.FileCount || totalSize != expected.TotalSize {
|
||||||
|
return fmt.Errorf(
|
||||||
|
"pages extracted metadata mismatch: got %d files/%d bytes, expected %d files/%d bytes",
|
||||||
|
fileCount,
|
||||||
|
totalSize,
|
||||||
|
expected.FileCount,
|
||||||
|
expected.TotalSize,
|
||||||
|
)
|
||||||
|
}
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func promotePagesRelease(stagingDir string, releaseDir string, project pagesProjectRef) error {
|
||||||
|
return promotePagesReleaseWithCopy(stagingDir, releaseDir, project, copyPagesDir)
|
||||||
|
}
|
||||||
|
|
||||||
|
func promotePagesReleaseWithCopy(
|
||||||
|
stagingDir string,
|
||||||
|
releaseDir string,
|
||||||
|
project pagesProjectRef,
|
||||||
|
copyDir func(string, string) error,
|
||||||
|
) error {
|
||||||
|
currentDir := pagesCurrentDirFromRelease(releaseDir)
|
||||||
|
defer removePagesStagingUnlessCurrent(stagingDir, currentDir)
|
||||||
|
currentUsesRelease, err := pagesCurrentTargetsRelease(currentDir, releaseDir)
|
||||||
|
if err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
if !currentUsesRelease {
|
||||||
|
if err := os.RemoveAll(releaseDir); err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
return os.Rename(stagingDir, releaseDir)
|
||||||
|
}
|
||||||
|
|
||||||
|
// A same-hash repair cannot remove releaseDir while current still resolves
|
||||||
|
// through it. Keep traffic on the fully validated staging tree, rebuild the
|
||||||
|
// canonical release, then atomically point current back to the canonical path.
|
||||||
|
if err := switchPagesCurrentDir(currentDir, stagingDir, os.Rename); err != nil {
|
||||||
|
return fmt.Errorf("switch Pages current to repair staging: %w", err)
|
||||||
|
}
|
||||||
|
backupDir := stagingDir + ".previous"
|
||||||
|
if err := os.Rename(releaseDir, backupDir); err != nil {
|
||||||
|
restoreErr := switchPagesCurrentDir(currentDir, releaseDir, os.Rename)
|
||||||
|
return errors.Join(
|
||||||
|
fmt.Errorf("move previous Pages release aside: %w", err),
|
||||||
|
restoreErr,
|
||||||
|
)
|
||||||
|
}
|
||||||
|
|
||||||
|
rollback := func(cause error) error {
|
||||||
|
var rollbackErrors []error
|
||||||
|
rollbackErrors = append(rollbackErrors, cause)
|
||||||
|
if err := os.RemoveAll(releaseDir); err != nil {
|
||||||
|
rollbackErrors = append(rollbackErrors, fmt.Errorf("remove failed Pages release repair: %w", err))
|
||||||
|
}
|
||||||
|
if err := os.Rename(backupDir, releaseDir); err != nil {
|
||||||
|
rollbackErrors = append(rollbackErrors, fmt.Errorf("restore previous Pages release: %w", err))
|
||||||
|
return errors.Join(rollbackErrors...)
|
||||||
|
}
|
||||||
|
if err := switchPagesCurrentDir(currentDir, releaseDir, os.Rename); err != nil {
|
||||||
|
rollbackErrors = append(rollbackErrors, fmt.Errorf("restore previous Pages current target: %w", err))
|
||||||
|
}
|
||||||
|
return errors.Join(rollbackErrors...)
|
||||||
|
}
|
||||||
|
|
||||||
|
if err := copyDir(stagingDir, releaseDir); err != nil {
|
||||||
|
return rollback(fmt.Errorf("copy repaired Pages release: %w", err))
|
||||||
|
}
|
||||||
|
if !pagesProjectReleaseReady(releaseDir, project) {
|
||||||
|
return rollback(errors.New("repaired Pages release is not ready"))
|
||||||
|
}
|
||||||
|
if err := switchPagesCurrentDir(currentDir, releaseDir, os.Rename); err != nil {
|
||||||
|
return rollback(fmt.Errorf("switch Pages current to repaired release: %w", err))
|
||||||
|
}
|
||||||
|
if err := os.RemoveAll(backupDir); err != nil {
|
||||||
|
slog.Warn("failed to remove previous Pages release", "path", backupDir, "error", err)
|
||||||
|
}
|
||||||
|
if err := os.RemoveAll(stagingDir); err != nil {
|
||||||
|
slog.Warn("failed to remove Pages repair staging", "path", stagingDir, "error", err)
|
||||||
|
}
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func pagesCurrentDirFromRelease(releaseDir string) string {
|
||||||
|
return filepath.Join(filepath.Dir(filepath.Dir(releaseDir)), "current")
|
||||||
|
}
|
||||||
|
|
||||||
|
func pagesCurrentTargetsRelease(currentDir string, releaseDir string) (bool, error) {
|
||||||
|
if _, err := os.Lstat(currentDir); err != nil {
|
||||||
|
if os.IsNotExist(err) {
|
||||||
|
return false, nil
|
||||||
|
}
|
||||||
|
return false, err
|
||||||
|
}
|
||||||
|
currentInfo, err := os.Stat(currentDir)
|
||||||
|
if err != nil {
|
||||||
|
if os.IsNotExist(err) {
|
||||||
|
return false, nil
|
||||||
|
}
|
||||||
|
return false, fmt.Errorf("stat Pages current target: %w", err)
|
||||||
|
}
|
||||||
|
releaseInfo, err := os.Stat(releaseDir)
|
||||||
|
if err != nil {
|
||||||
|
if os.IsNotExist(err) {
|
||||||
|
return false, nil
|
||||||
|
}
|
||||||
|
return false, err
|
||||||
|
}
|
||||||
|
return os.SameFile(currentInfo, releaseInfo), nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func removePagesStagingUnlessCurrent(stagingDir string, currentDir string) {
|
||||||
|
currentUsesStaging, err := pagesCurrentTargetsRelease(currentDir, stagingDir)
|
||||||
|
if err == nil && currentUsesStaging {
|
||||||
|
slog.Error("preserving Pages staging because current still references it", "path", stagingDir)
|
||||||
|
return
|
||||||
|
}
|
||||||
|
if removeErr := os.RemoveAll(stagingDir); removeErr != nil {
|
||||||
|
slog.Warn("failed to remove Pages staging", "path", stagingDir, "error", removeErr)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func verifyPagesCurrentTarget(currentDir string, releaseDir string) error {
|
||||||
|
currentInfo, err := os.Stat(currentDir)
|
||||||
|
if err != nil {
|
||||||
|
return fmt.Errorf("stat Pages current target: %w", err)
|
||||||
|
}
|
||||||
|
releaseInfo, err := os.Stat(releaseDir)
|
||||||
|
if err != nil {
|
||||||
|
return fmt.Errorf("stat Pages release target: %w", err)
|
||||||
|
}
|
||||||
|
if !os.SameFile(currentInfo, releaseInfo) {
|
||||||
|
return fmt.Errorf("pages current target does not resolve to release %s", releaseDir)
|
||||||
|
}
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func switchPagesCurrentDir(
|
||||||
|
currentDir string,
|
||||||
|
releaseDir string,
|
||||||
|
rename func(string, string) error,
|
||||||
|
) error {
|
||||||
|
return switchPagesCurrentDirWithOps(currentDir, releaseDir, rename, os.Symlink)
|
||||||
|
}
|
||||||
|
|
||||||
|
func switchPagesCurrentDirWithOps(
|
||||||
|
currentDir string,
|
||||||
|
releaseDir string,
|
||||||
|
rename func(string, string) error,
|
||||||
|
symlink func(string, string) error,
|
||||||
|
) error {
|
||||||
if err := os.MkdirAll(filepath.Dir(currentDir), pagesDirPerm); err != nil {
|
if err := os.MkdirAll(filepath.Dir(currentDir), pagesDirPerm); err != nil {
|
||||||
return err
|
return err
|
||||||
}
|
}
|
||||||
|
currentInfo, currentErr := os.Lstat(currentDir)
|
||||||
|
if currentErr != nil && !os.IsNotExist(currentErr) {
|
||||||
|
return currentErr
|
||||||
|
}
|
||||||
|
if currentErr == nil && currentInfo.Mode()&os.ModeSymlink == 0 {
|
||||||
|
return fallbackCopyPagesCurrentDir(currentDir, releaseDir, rename)
|
||||||
|
}
|
||||||
|
|
||||||
|
previousTarget := ""
|
||||||
|
hadPrevious := currentErr == nil
|
||||||
|
if hadPrevious {
|
||||||
|
var err error
|
||||||
|
previousTarget, err = os.Readlink(currentDir)
|
||||||
|
if err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
relTarget, err := filepath.Rel(filepath.Dir(currentDir), releaseDir)
|
relTarget, err := filepath.Rel(filepath.Dir(currentDir), releaseDir)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
@@ -413,44 +804,123 @@ func switchPagesProjectCurrentDir(baseDir string, projectID uint, releaseDir str
|
|||||||
}
|
}
|
||||||
|
|
||||||
tmpSymlink := currentDir + ".tmp"
|
tmpSymlink := currentDir + ".tmp"
|
||||||
_ = os.Remove(tmpSymlink)
|
if err := os.Remove(tmpSymlink); err != nil && !os.IsNotExist(err) {
|
||||||
|
|
||||||
symlinkErr := os.Symlink(relTarget, tmpSymlink)
|
|
||||||
if symlinkErr != nil {
|
|
||||||
return fallbackCopyPagesCurrentDir(currentDir, previousDir, releaseDir)
|
|
||||||
}
|
|
||||||
_ = os.Remove(tmpSymlink)
|
|
||||||
|
|
||||||
if _, err := os.Lstat(currentDir); err == nil {
|
|
||||||
if err := os.Rename(currentDir, previousDir); err != nil {
|
|
||||||
return err
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
if err := os.Symlink(relTarget, currentDir); err != nil {
|
|
||||||
if _, restoreErr := os.Lstat(previousDir); restoreErr == nil {
|
|
||||||
_ = os.Rename(previousDir, currentDir)
|
|
||||||
}
|
|
||||||
return err
|
return err
|
||||||
}
|
}
|
||||||
_ = os.RemoveAll(previousDir)
|
if err := symlink(relTarget, tmpSymlink); err != nil {
|
||||||
|
_ = os.Remove(tmpSymlink)
|
||||||
|
return fallbackCopyPagesCurrentDir(currentDir, releaseDir, rename)
|
||||||
|
}
|
||||||
|
defer func() { _ = os.Remove(tmpSymlink) }()
|
||||||
|
if err := verifyPagesCurrentTarget(tmpSymlink, releaseDir); err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
if err := rename(tmpSymlink, currentDir); err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
if err := verifyPagesCurrentTarget(currentDir, releaseDir); err != nil {
|
||||||
|
rollbackErr := rollbackPagesCurrentSymlink(
|
||||||
|
currentDir,
|
||||||
|
previousTarget,
|
||||||
|
hadPrevious,
|
||||||
|
rename,
|
||||||
|
)
|
||||||
|
return errors.Join(err, rollbackErr)
|
||||||
|
}
|
||||||
return nil
|
return nil
|
||||||
}
|
}
|
||||||
|
|
||||||
func fallbackCopyPagesCurrentDir(currentDir, previousDir, releaseDir string) error {
|
func fallbackCopyPagesCurrentDir(
|
||||||
if _, err := os.Lstat(currentDir); err == nil {
|
currentDir string,
|
||||||
if err := os.Rename(currentDir, previousDir); err != nil {
|
releaseDir string,
|
||||||
return err
|
rename func(string, string) error,
|
||||||
}
|
) error {
|
||||||
}
|
stagingDir := currentDir + ".copy.tmp"
|
||||||
if err := copyPagesDir(releaseDir, currentDir); err != nil {
|
previousDir := currentDir + ".previous"
|
||||||
_ = os.RemoveAll(currentDir)
|
if err := os.RemoveAll(stagingDir); err != nil {
|
||||||
if _, restoreErr := os.Lstat(previousDir); restoreErr == nil {
|
|
||||||
_ = os.Rename(previousDir, currentDir)
|
|
||||||
}
|
|
||||||
return err
|
return err
|
||||||
}
|
}
|
||||||
_ = os.RemoveAll(previousDir)
|
if err := copyPagesDir(releaseDir, stagingDir); err != nil {
|
||||||
|
_ = os.RemoveAll(stagingDir)
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
if err := os.RemoveAll(previousDir); err != nil {
|
||||||
|
_ = os.RemoveAll(stagingDir)
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
|
||||||
|
hadPrevious := false
|
||||||
|
if _, err := os.Lstat(currentDir); err == nil {
|
||||||
|
if err := rename(currentDir, previousDir); err != nil {
|
||||||
|
_ = os.RemoveAll(stagingDir)
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
hadPrevious = true
|
||||||
|
} else if !os.IsNotExist(err) {
|
||||||
|
_ = os.RemoveAll(stagingDir)
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
if err := rename(stagingDir, currentDir); err != nil {
|
||||||
|
var restoreErr error
|
||||||
|
if hadPrevious {
|
||||||
|
restoreErr = rename(previousDir, currentDir)
|
||||||
|
}
|
||||||
|
_ = os.RemoveAll(stagingDir)
|
||||||
|
return errors.Join(err, restoreErr)
|
||||||
|
}
|
||||||
|
if err := os.RemoveAll(previousDir); err != nil {
|
||||||
|
slog.Warn("failed to remove previous Pages current directory", "path", previousDir, "error", err)
|
||||||
|
}
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func switchPagesProjectCurrentDir(baseDir string, projectID uint, releaseDir string) error {
|
||||||
|
return switchPagesProjectCurrentDirWithRename(baseDir, projectID, releaseDir, os.Rename)
|
||||||
|
}
|
||||||
|
|
||||||
|
func switchPagesProjectCurrentDirWithRename(
|
||||||
|
baseDir string,
|
||||||
|
projectID uint,
|
||||||
|
releaseDir string,
|
||||||
|
rename func(string, string) error,
|
||||||
|
) error {
|
||||||
|
return switchPagesCurrentDir(pagesProjectCurrentDir(baseDir, projectID), releaseDir, rename)
|
||||||
|
}
|
||||||
|
|
||||||
|
func rollbackPagesCurrentSymlink(
|
||||||
|
currentDir string,
|
||||||
|
previousTarget string,
|
||||||
|
hadPrevious bool,
|
||||||
|
rename func(string, string) error,
|
||||||
|
) error {
|
||||||
|
if !hadPrevious {
|
||||||
|
if err := os.Remove(currentDir); err != nil && !os.IsNotExist(err) {
|
||||||
|
return fmt.Errorf("remove unverified Pages current symlink: %w", err)
|
||||||
|
}
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
rollbackSymlink := currentDir + ".rollback.tmp"
|
||||||
|
if err := os.Remove(rollbackSymlink); err != nil && !os.IsNotExist(err) {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
if err := os.Symlink(previousTarget, rollbackSymlink); err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
defer func() { _ = os.Remove(rollbackSymlink) }()
|
||||||
|
if err := rename(rollbackSymlink, currentDir); err != nil {
|
||||||
|
return fmt.Errorf("restore previous Pages current symlink: %w", err)
|
||||||
|
}
|
||||||
|
gotTarget, err := os.Readlink(currentDir)
|
||||||
|
if err != nil {
|
||||||
|
return fmt.Errorf("verify restored Pages current symlink: %w", err)
|
||||||
|
}
|
||||||
|
if gotTarget != previousTarget {
|
||||||
|
return fmt.Errorf(
|
||||||
|
"restored Pages current symlink target %q does not match %q",
|
||||||
|
gotTarget,
|
||||||
|
previousTarget,
|
||||||
|
)
|
||||||
|
}
|
||||||
return nil
|
return nil
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -467,24 +937,30 @@ func copyPagesDir(sourceDir string, targetDir string) error {
|
|||||||
if entry.IsDir() {
|
if entry.IsDir() {
|
||||||
return os.MkdirAll(targetPath, pagesDirPerm)
|
return os.MkdirAll(targetPath, pagesDirPerm)
|
||||||
}
|
}
|
||||||
input, err := os.Open(sourcePath) //nolint:gosec // sourcePath is under managed PagesDir walk root
|
return copyPagesFile(sourcePath, targetPath)
|
||||||
if err != nil {
|
|
||||||
return err
|
|
||||||
}
|
|
||||||
defer func() { _ = input.Close() }()
|
|
||||||
if err := os.MkdirAll(filepath.Dir(targetPath), pagesDirPerm); err != nil {
|
|
||||||
return err
|
|
||||||
}
|
|
||||||
output, err := os.OpenFile(targetPath, os.O_CREATE|os.O_WRONLY|os.O_TRUNC, pagesFilePerm) //nolint:gosec // targetPath is under managed PagesDir walk root
|
|
||||||
if err != nil {
|
|
||||||
return err
|
|
||||||
}
|
|
||||||
defer func() { _ = output.Close() }()
|
|
||||||
_, err = io.Copy(output, input)
|
|
||||||
return err
|
|
||||||
})
|
})
|
||||||
}
|
}
|
||||||
|
|
||||||
|
func copyPagesFile(sourcePath string, targetPath string) error {
|
||||||
|
input, err := os.Open(sourcePath) //nolint:gosec // sourcePath is under managed PagesDir walk root
|
||||||
|
if err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
if err := os.MkdirAll(filepath.Dir(targetPath), pagesDirPerm); err != nil {
|
||||||
|
_ = input.Close()
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
output, err := os.OpenFile(targetPath, os.O_CREATE|os.O_WRONLY|os.O_TRUNC, pagesFilePerm) //nolint:gosec // targetPath is under managed PagesDir walk root
|
||||||
|
if err != nil {
|
||||||
|
_ = input.Close()
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
_, copyErr := io.Copy(output, input)
|
||||||
|
outputCloseErr := output.Close()
|
||||||
|
inputCloseErr := input.Close()
|
||||||
|
return errors.Join(copyErr, outputCloseErr, inputCloseErr)
|
||||||
|
}
|
||||||
|
|
||||||
func markerMatches(dir string, project pagesProjectRef) bool {
|
func markerMatches(dir string, project pagesProjectRef) bool {
|
||||||
data, err := os.ReadFile(filepath.Join(dir, ".openflare-pages.json")) //nolint:gosec // dir is managed PagesDir
|
data, err := os.ReadFile(filepath.Join(dir, ".openflare-pages.json")) //nolint:gosec // dir is managed PagesDir
|
||||||
if err != nil {
|
if err != nil {
|
||||||
@@ -515,8 +991,3 @@ func pagesProjectCurrentDir(baseDir string, projectID uint) string {
|
|||||||
func pagesProjectReleaseDir(baseDir string, projectID uint, checksum string) string {
|
func pagesProjectReleaseDir(baseDir string, projectID uint, checksum string) string {
|
||||||
return filepath.Join(baseDir, "projects", fmt.Sprintf("%d", projectID), "releases", checksum)
|
return filepath.Join(baseDir, "projects", fmt.Sprintf("%d", projectID), "releases", checksum)
|
||||||
}
|
}
|
||||||
|
|
||||||
func checksumBytes(data []byte) string {
|
|
||||||
sum := sha256.Sum256(data)
|
|
||||||
return hex.EncodeToString(sum[:])
|
|
||||||
}
|
|
||||||
|
|||||||
@@ -0,0 +1,388 @@
|
|||||||
|
// Copyright 2026 Arctel.net
|
||||||
|
// SPDX-License-Identifier: Apache-2.0
|
||||||
|
|
||||||
|
package sync
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
"errors"
|
||||||
|
"os"
|
||||||
|
"path/filepath"
|
||||||
|
"strings"
|
||||||
|
"testing"
|
||||||
|
|
||||||
|
"github.com/Rain-kl/Wavelet/internal/apps/agent/protocol"
|
||||||
|
"github.com/Rain-kl/Wavelet/internal/apps/agent/state"
|
||||||
|
)
|
||||||
|
|
||||||
|
func TestEnsurePagesProjectRejectsMetadataBeyondAgentCapsBeforeDownload(t *testing.T) {
|
||||||
|
packageBytes := testPagesPackage(t, map[string]string{"index.html": "x"})
|
||||||
|
base := protocol.PagesProjectLatestHashResponse{
|
||||||
|
ProjectID: 1,
|
||||||
|
DeploymentID: 1,
|
||||||
|
Hash: testBytesChecksum(packageBytes),
|
||||||
|
PackageSize: int64(len(packageBytes)),
|
||||||
|
FileCount: 1,
|
||||||
|
TotalSize: 1,
|
||||||
|
}
|
||||||
|
tests := []struct {
|
||||||
|
name string
|
||||||
|
mutate func(*protocol.PagesProjectLatestHashResponse)
|
||||||
|
}{
|
||||||
|
{
|
||||||
|
name: "package size",
|
||||||
|
mutate: func(metadata *protocol.PagesProjectLatestHashResponse) {
|
||||||
|
metadata.PackageSize = agentPagesMaxPackageBytes + 1
|
||||||
|
},
|
||||||
|
},
|
||||||
|
{
|
||||||
|
name: "file count",
|
||||||
|
mutate: func(metadata *protocol.PagesProjectLatestHashResponse) {
|
||||||
|
metadata.FileCount = agentPagesMaxFiles + 1
|
||||||
|
},
|
||||||
|
},
|
||||||
|
{
|
||||||
|
name: "total size",
|
||||||
|
mutate: func(metadata *protocol.PagesProjectLatestHashResponse) {
|
||||||
|
metadata.TotalSize = agentPagesMaxTotalBytes + 1
|
||||||
|
},
|
||||||
|
},
|
||||||
|
}
|
||||||
|
for _, test := range tests {
|
||||||
|
t.Run(test.name, func(t *testing.T) {
|
||||||
|
metadata := base
|
||||||
|
test.mutate(&metadata)
|
||||||
|
client := &fakeClient{
|
||||||
|
pagesPackages: map[uint][]byte{1: packageBytes},
|
||||||
|
pagesMetadata: map[uint]protocol.PagesProjectLatestHashResponse{1: metadata},
|
||||||
|
}
|
||||||
|
service := New(client, &fakeManager{}, nil)
|
||||||
|
service.SetPagesDir(t.TempDir())
|
||||||
|
err := service.ensurePagesProject(context.Background(), &state.Snapshot{}, 1)
|
||||||
|
if err == nil || !strings.Contains(err.Error(), "agent limit") {
|
||||||
|
t.Fatalf("ensurePagesProject(%s metadata) error = %v, want agent limit error", test.name, err)
|
||||||
|
}
|
||||||
|
if client.pagesPackageDownloads != 0 {
|
||||||
|
t.Errorf("ensurePagesProject(%s metadata) downloads = %d, want 0", test.name, client.pagesPackageDownloads)
|
||||||
|
}
|
||||||
|
})
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestEnsurePagesProjectRetriesSameHashDifferentDeployment(t *testing.T) {
|
||||||
|
packageBytes := testPagesPackage(t, map[string]string{"index.html": "same"})
|
||||||
|
hash := testBytesChecksum(packageBytes)
|
||||||
|
client := &racingLatestClient{
|
||||||
|
pkgA: packageBytes,
|
||||||
|
pkgB: packageBytes,
|
||||||
|
hashA: hash,
|
||||||
|
hashB: hash,
|
||||||
|
}
|
||||||
|
service := New(client, &fakeManager{}, nil)
|
||||||
|
pagesDir := t.TempDir()
|
||||||
|
service.SetPagesDir(pagesDir)
|
||||||
|
snapshot := &state.Snapshot{PagesDeployments: []state.PagesDeployment{{ProjectID: 42}}}
|
||||||
|
|
||||||
|
if err := service.ensurePagesProject(context.Background(), snapshot, 42); err != nil {
|
||||||
|
t.Fatalf("ensurePagesProject(same hash deployment race) error = %v", err)
|
||||||
|
}
|
||||||
|
if client.downloadCalls != 2 {
|
||||||
|
t.Errorf("ensurePagesProject(same hash deployment race) downloads = %d, want 2", client.downloadCalls)
|
||||||
|
}
|
||||||
|
if snapshot.PagesDeployments[0].DeploymentID != 2 || snapshot.PagesDeployments[0].Hash != hash {
|
||||||
|
t.Errorf("snapshot Pages deployment = %+v, want deployment 2/hash %s", snapshot.PagesDeployments[0], hash)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestEnsurePagesProjectExtractionFailureCleansTempAndPreservesCurrent(t *testing.T) {
|
||||||
|
projectID := uint(9)
|
||||||
|
oldPackage := testPagesPackage(t, map[string]string{"index.html": "old"})
|
||||||
|
oldHash := testBytesChecksum(oldPackage)
|
||||||
|
newPackage := testPagesPackage(t, map[string]string{"index.html": "new"})
|
||||||
|
newHash := testBytesChecksum(newPackage)
|
||||||
|
pagesDir := t.TempDir()
|
||||||
|
oldRelease := pagesProjectReleaseDir(pagesDir, projectID, oldHash)
|
||||||
|
if err := extractTestPagesPackage(t, oldPackage, oldRelease, pagesProjectRef{
|
||||||
|
ProjectID: projectID,
|
||||||
|
DeploymentID: 1,
|
||||||
|
Checksum: oldHash,
|
||||||
|
}); err != nil {
|
||||||
|
t.Fatalf("extractTestPagesPackage(old) error = %v", err)
|
||||||
|
}
|
||||||
|
if err := switchPagesProjectCurrentDir(pagesDir, projectID, oldRelease); err != nil {
|
||||||
|
t.Fatalf("switchPagesProjectCurrentDir(old) error = %v", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
client := &fakeClient{
|
||||||
|
pagesPackages: map[uint][]byte{projectID: newPackage},
|
||||||
|
pagesMetadata: map[uint]protocol.PagesProjectLatestHashResponse{
|
||||||
|
projectID: {
|
||||||
|
ProjectID: projectID,
|
||||||
|
DeploymentID: 2,
|
||||||
|
Hash: newHash,
|
||||||
|
PackageSize: int64(len(newPackage)),
|
||||||
|
FileCount: 1,
|
||||||
|
TotalSize: 2, // Smaller than the actual three-byte file.
|
||||||
|
},
|
||||||
|
},
|
||||||
|
}
|
||||||
|
service := New(client, &fakeManager{}, nil)
|
||||||
|
service.SetPagesDir(pagesDir)
|
||||||
|
err := service.ensurePagesProject(context.Background(), &state.Snapshot{}, projectID)
|
||||||
|
if err == nil {
|
||||||
|
t.Fatal("ensurePagesProject(metadata-tightened extraction) error = nil, want error")
|
||||||
|
}
|
||||||
|
current, readErr := os.ReadFile(pagesProjectCurrentDir(pagesDir, projectID) + "/index.html")
|
||||||
|
if readErr != nil {
|
||||||
|
t.Fatalf("read old current after failed extraction error = %v", readErr)
|
||||||
|
}
|
||||||
|
if string(current) != "old" {
|
||||||
|
t.Errorf("current content after failed extraction = %q, want %q", current, "old")
|
||||||
|
}
|
||||||
|
entries, readErr := os.ReadDir(filepath.Join(pagesDir, "projects", "9", "releases"))
|
||||||
|
if readErr != nil {
|
||||||
|
t.Fatalf("read releases after failed extraction error = %v", readErr)
|
||||||
|
}
|
||||||
|
if len(entries) != 1 || entries[0].Name() != oldHash {
|
||||||
|
t.Errorf("releases after failed extraction = %v, want only %s", entries, oldHash)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestEnsurePagesProjectAcceptsAllZeroByteFiles(t *testing.T) {
|
||||||
|
packageBytes := testPagesPackage(t, map[string]string{
|
||||||
|
"index.html": "",
|
||||||
|
".gitkeep": "",
|
||||||
|
})
|
||||||
|
client := &fakeClient{pagesPackages: map[uint][]byte{5: packageBytes}}
|
||||||
|
service := New(client, &fakeManager{}, nil)
|
||||||
|
pagesDir := t.TempDir()
|
||||||
|
service.SetPagesDir(pagesDir)
|
||||||
|
|
||||||
|
if err := service.ensurePagesProject(context.Background(), &state.Snapshot{}, 5); err != nil {
|
||||||
|
t.Fatalf("ensurePagesProject(all-zero files) error = %v", err)
|
||||||
|
}
|
||||||
|
for _, name := range []string{"index.html", ".gitkeep"} {
|
||||||
|
info, err := os.Stat(filepath.Join(pagesProjectCurrentDir(pagesDir, 5), name))
|
||||||
|
if err != nil {
|
||||||
|
t.Errorf("stat all-zero file %q error = %v", name, err)
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
if info.Size() != 0 {
|
||||||
|
t.Errorf("all-zero file %q size = %d, want 0", name, info.Size())
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestSwitchPagesProjectCurrentDirRenameFailureKeepsPreviousCurrent(t *testing.T) {
|
||||||
|
pagesDir := t.TempDir()
|
||||||
|
projectID := uint(21)
|
||||||
|
oldRelease := pagesProjectReleaseDir(pagesDir, projectID, "old")
|
||||||
|
newRelease := pagesProjectReleaseDir(pagesDir, projectID, "new")
|
||||||
|
for path, content := range map[string]string{
|
||||||
|
oldRelease: "old",
|
||||||
|
newRelease: "new",
|
||||||
|
} {
|
||||||
|
if err := os.MkdirAll(path, pagesDirPerm); err != nil {
|
||||||
|
t.Fatalf("mkdir release %q error = %v", path, err)
|
||||||
|
}
|
||||||
|
if err := os.WriteFile(filepath.Join(path, "index.html"), []byte(content), pagesFilePerm); err != nil {
|
||||||
|
t.Fatalf("write release %q error = %v", path, err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if err := switchPagesProjectCurrentDir(pagesDir, projectID, oldRelease); err != nil {
|
||||||
|
t.Fatalf("seed previous current error = %v", err)
|
||||||
|
}
|
||||||
|
currentDir := pagesProjectCurrentDir(pagesDir, projectID)
|
||||||
|
renameErr := errors.New("injected current rename failure")
|
||||||
|
err := switchPagesProjectCurrentDirWithRename(
|
||||||
|
pagesDir,
|
||||||
|
projectID,
|
||||||
|
newRelease,
|
||||||
|
func(oldPath string, newPath string) error {
|
||||||
|
if oldPath == currentDir+".tmp" && newPath == currentDir {
|
||||||
|
return renameErr
|
||||||
|
}
|
||||||
|
return os.Rename(oldPath, newPath)
|
||||||
|
},
|
||||||
|
)
|
||||||
|
if !errors.Is(err, renameErr) {
|
||||||
|
t.Fatalf("switchPagesProjectCurrentDirWithRename() error = %v, want injected rename error", err)
|
||||||
|
}
|
||||||
|
current, err := os.ReadFile(filepath.Join(currentDir, "index.html"))
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("read previous current after rename failure error = %v", err)
|
||||||
|
}
|
||||||
|
if string(current) != "old" {
|
||||||
|
t.Errorf("current after rename failure = %q, want %q", current, "old")
|
||||||
|
}
|
||||||
|
if _, err := os.Lstat(currentDir + ".tmp"); !os.IsNotExist(err) {
|
||||||
|
t.Errorf("temporary current symlink remains after rename failure: %v", err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestPromoteSameHashReleaseFailureRestoresPreviousCurrent(t *testing.T) {
|
||||||
|
pagesDir := t.TempDir()
|
||||||
|
projectID := uint(22)
|
||||||
|
hash := "same-hash"
|
||||||
|
project := pagesProjectRef{ProjectID: projectID, DeploymentID: 2, Checksum: hash}
|
||||||
|
releaseDir := pagesProjectReleaseDir(pagesDir, projectID, hash)
|
||||||
|
if err := os.MkdirAll(releaseDir, pagesDirPerm); err != nil {
|
||||||
|
t.Fatalf("mkdir previous same-hash release error = %v", err)
|
||||||
|
}
|
||||||
|
if err := os.WriteFile(filepath.Join(releaseDir, "index.html"), []byte("old"), pagesFilePerm); err != nil {
|
||||||
|
t.Fatalf("write previous same-hash release error = %v", err)
|
||||||
|
}
|
||||||
|
if err := writePagesMarker(releaseDir, project); err != nil {
|
||||||
|
t.Fatalf("write previous same-hash marker error = %v", err)
|
||||||
|
}
|
||||||
|
if err := switchPagesProjectCurrentDir(pagesDir, projectID, releaseDir); err != nil {
|
||||||
|
t.Fatalf("seed same-hash current error = %v", err)
|
||||||
|
}
|
||||||
|
stagingDir, err := os.MkdirTemp(filepath.Dir(releaseDir), ".same-hash-*.tmp")
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("create same-hash staging error = %v", err)
|
||||||
|
}
|
||||||
|
if err := os.WriteFile(filepath.Join(stagingDir, "index.html"), []byte("new"), pagesFilePerm); err != nil {
|
||||||
|
t.Fatalf("write repaired same-hash release error = %v", err)
|
||||||
|
}
|
||||||
|
if err := writePagesMarker(stagingDir, project); err != nil {
|
||||||
|
t.Fatalf("write repaired same-hash marker error = %v", err)
|
||||||
|
}
|
||||||
|
copyErr := errors.New("injected same-hash copy failure")
|
||||||
|
err = promotePagesReleaseWithCopy(
|
||||||
|
stagingDir,
|
||||||
|
releaseDir,
|
||||||
|
project,
|
||||||
|
func(_ string, targetDir string) error {
|
||||||
|
if err := os.MkdirAll(targetDir, pagesDirPerm); err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
if err := os.WriteFile(filepath.Join(targetDir, "index.html"), []byte("partial"), pagesFilePerm); err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
return copyErr
|
||||||
|
},
|
||||||
|
)
|
||||||
|
if !errors.Is(err, copyErr) {
|
||||||
|
t.Fatalf("promotePagesReleaseWithCopy() error = %v, want injected copy error", err)
|
||||||
|
}
|
||||||
|
for name, path := range map[string]string{
|
||||||
|
"current": filepath.Join(pagesProjectCurrentDir(pagesDir, projectID), "index.html"),
|
||||||
|
"release": filepath.Join(releaseDir, "index.html"),
|
||||||
|
} {
|
||||||
|
content, readErr := os.ReadFile(path)
|
||||||
|
if readErr != nil {
|
||||||
|
t.Fatalf("read restored %s after same-hash repair failure error = %v", name, readErr)
|
||||||
|
}
|
||||||
|
if string(content) != "old" {
|
||||||
|
t.Errorf("restored %s after same-hash repair failure = %q, want %q", name, content, "old")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if _, err := os.Stat(stagingDir); !os.IsNotExist(err) {
|
||||||
|
t.Errorf("same-hash staging remains after successful rollback: %v", err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestPromotePagesReleaseRepairsDanglingCurrent(t *testing.T) {
|
||||||
|
pagesDir := t.TempDir()
|
||||||
|
projectID := uint(23)
|
||||||
|
releaseDir := pagesProjectReleaseDir(pagesDir, projectID, "new-hash")
|
||||||
|
currentDir := pagesProjectCurrentDir(pagesDir, projectID)
|
||||||
|
requireTestMkdirAll(t, filepath.Dir(currentDir))
|
||||||
|
relTarget, err := filepath.Rel(filepath.Dir(currentDir), releaseDir)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("relative release target error = %v", err)
|
||||||
|
}
|
||||||
|
if err := os.Symlink(relTarget, currentDir); err != nil {
|
||||||
|
t.Skipf("symlink unsupported: %v", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
requireTestMkdirAll(t, filepath.Dir(releaseDir))
|
||||||
|
stagingDir, err := os.MkdirTemp(filepath.Dir(releaseDir), ".dangling-*.tmp")
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("create dangling repair staging error = %v", err)
|
||||||
|
}
|
||||||
|
if err := os.WriteFile(filepath.Join(stagingDir, "index.html"), []byte("repaired"), pagesFilePerm); err != nil {
|
||||||
|
t.Fatalf("write dangling repair staging error = %v", err)
|
||||||
|
}
|
||||||
|
project := pagesProjectRef{ProjectID: projectID, DeploymentID: 1, Checksum: "new-hash"}
|
||||||
|
if err := writePagesMarker(stagingDir, project); err != nil {
|
||||||
|
t.Fatalf("write dangling repair marker error = %v", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
if err := promotePagesRelease(stagingDir, releaseDir, project); err != nil {
|
||||||
|
t.Fatalf("promotePagesRelease(dangling current) error = %v", err)
|
||||||
|
}
|
||||||
|
content, err := os.ReadFile(filepath.Join(currentDir, "index.html"))
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("read repaired dangling current error = %v", err)
|
||||||
|
}
|
||||||
|
if string(content) != "repaired" {
|
||||||
|
t.Errorf("repaired dangling current = %q, want %q", content, "repaired")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestSwitchPagesCurrentDirCopiesOverLegacyDirectory(t *testing.T) {
|
||||||
|
pagesDir := t.TempDir()
|
||||||
|
currentDir := filepath.Join(pagesDir, "current")
|
||||||
|
releaseDir := filepath.Join(pagesDir, "releases", "new")
|
||||||
|
requireTestMkdirAll(t, currentDir)
|
||||||
|
requireTestMkdirAll(t, releaseDir)
|
||||||
|
if err := os.WriteFile(filepath.Join(currentDir, "index.html"), []byte("old"), pagesFilePerm); err != nil {
|
||||||
|
t.Fatalf("write legacy current error = %v", err)
|
||||||
|
}
|
||||||
|
if err := os.WriteFile(filepath.Join(releaseDir, "index.html"), []byte("new"), pagesFilePerm); err != nil {
|
||||||
|
t.Fatalf("write new release error = %v", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
if err := switchPagesCurrentDir(currentDir, releaseDir, os.Rename); err != nil {
|
||||||
|
t.Fatalf("switchPagesCurrentDir(legacy directory) error = %v", err)
|
||||||
|
}
|
||||||
|
content, err := os.ReadFile(filepath.Join(currentDir, "index.html"))
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("read copied legacy current error = %v", err)
|
||||||
|
}
|
||||||
|
if string(content) != "new" {
|
||||||
|
t.Errorf("copied legacy current = %q, want %q", content, "new")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestSwitchPagesCurrentDirFallsBackWhenSymlinkUnavailable(t *testing.T) {
|
||||||
|
pagesDir := t.TempDir()
|
||||||
|
currentDir := filepath.Join(pagesDir, "current")
|
||||||
|
releaseDir := filepath.Join(pagesDir, "releases", "new")
|
||||||
|
requireTestMkdirAll(t, releaseDir)
|
||||||
|
if err := os.WriteFile(filepath.Join(releaseDir, "index.html"), []byte("new"), pagesFilePerm); err != nil {
|
||||||
|
t.Fatalf("write fallback release error = %v", err)
|
||||||
|
}
|
||||||
|
symlinkErr := errors.New("injected symlink unavailable")
|
||||||
|
if err := switchPagesCurrentDirWithOps(
|
||||||
|
currentDir,
|
||||||
|
releaseDir,
|
||||||
|
os.Rename,
|
||||||
|
func(string, string) error { return symlinkErr },
|
||||||
|
); err != nil {
|
||||||
|
t.Fatalf("switchPagesCurrentDirWithOps(symlink unavailable) error = %v", err)
|
||||||
|
}
|
||||||
|
info, err := os.Lstat(currentDir)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("lstat copied current error = %v", err)
|
||||||
|
}
|
||||||
|
if !info.IsDir() {
|
||||||
|
t.Errorf("copied current mode = %v, want directory", info.Mode())
|
||||||
|
}
|
||||||
|
content, err := os.ReadFile(filepath.Join(currentDir, "index.html"))
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("read fallback current error = %v", err)
|
||||||
|
}
|
||||||
|
if string(content) != "new" {
|
||||||
|
t.Errorf("fallback current = %q, want %q", content, "new")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func requireTestMkdirAll(t *testing.T, dir string) {
|
||||||
|
t.Helper()
|
||||||
|
if err := os.MkdirAll(dir, pagesDirPerm); err != nil {
|
||||||
|
t.Fatalf("mkdir %q error = %v", dir, err)
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -1,3 +1,6 @@
|
|||||||
|
// Copyright 2026 Arctel.net
|
||||||
|
// SPDX-License-Identifier: Apache-2.0
|
||||||
|
|
||||||
package sync
|
package sync
|
||||||
|
|
||||||
import (
|
import (
|
||||||
@@ -7,6 +10,7 @@ import (
|
|||||||
"encoding/json"
|
"encoding/json"
|
||||||
"errors"
|
"errors"
|
||||||
"fmt"
|
"fmt"
|
||||||
|
"io"
|
||||||
"log/slog"
|
"log/slog"
|
||||||
"sort"
|
"sort"
|
||||||
"strconv"
|
"strconv"
|
||||||
@@ -31,9 +35,9 @@ const (
|
|||||||
type ConfigClient interface {
|
type ConfigClient interface {
|
||||||
GetActiveConfig(ctx context.Context) (*protocol.ActiveConfigResponse, error)
|
GetActiveConfig(ctx context.Context) (*protocol.ActiveConfigResponse, error)
|
||||||
GetPagesDeploymentHash(ctx context.Context, deploymentID uint) (string, error)
|
GetPagesDeploymentHash(ctx context.Context, deploymentID uint) (string, error)
|
||||||
DownloadPagesDeploymentPackage(ctx context.Context, deploymentID uint) ([]byte, error)
|
DownloadPagesDeploymentPackage(ctx context.Context, deploymentID uint, dst io.Writer, maxBytes int64) (int64, error)
|
||||||
GetPagesProjectLatestHash(ctx context.Context, projectID uint) (*protocol.PagesProjectLatestHashResponse, error)
|
GetPagesProjectLatestHash(ctx context.Context, projectID uint) (*protocol.PagesProjectLatestHashResponse, error)
|
||||||
DownloadPagesProjectLatestPackage(ctx context.Context, projectID uint) ([]byte, error)
|
DownloadPagesProjectLatestPackage(ctx context.Context, projectID uint, dst io.Writer, maxBytes int64) (int64, error)
|
||||||
ReportApplyLog(ctx context.Context, payload protocol.ApplyLogPayload) error
|
ReportApplyLog(ctx context.Context, payload protocol.ApplyLogPayload) error
|
||||||
SyncWAFIPGroups(ctx context.Context, payload protocol.WAFIPGroupSyncRequest) (*protocol.WAFIPGroupSyncResponse, error)
|
SyncWAFIPGroups(ctx context.Context, payload protocol.WAFIPGroupSyncRequest) (*protocol.WAFIPGroupSyncResponse, error)
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -1,3 +1,6 @@
|
|||||||
|
// Copyright 2026 Arctel.net
|
||||||
|
// SPDX-License-Identifier: Apache-2.0
|
||||||
|
|
||||||
package sync
|
package sync
|
||||||
|
|
||||||
import (
|
import (
|
||||||
@@ -7,6 +10,7 @@ import (
|
|||||||
"crypto/sha256"
|
"crypto/sha256"
|
||||||
"encoding/hex"
|
"encoding/hex"
|
||||||
"fmt"
|
"fmt"
|
||||||
|
"io"
|
||||||
"os"
|
"os"
|
||||||
"path/filepath"
|
"path/filepath"
|
||||||
"strings"
|
"strings"
|
||||||
@@ -16,6 +20,7 @@ import (
|
|||||||
"github.com/Rain-kl/Wavelet/internal/apps/agent/nginx"
|
"github.com/Rain-kl/Wavelet/internal/apps/agent/nginx"
|
||||||
"github.com/Rain-kl/Wavelet/internal/apps/agent/protocol"
|
"github.com/Rain-kl/Wavelet/internal/apps/agent/protocol"
|
||||||
"github.com/Rain-kl/Wavelet/internal/apps/agent/state"
|
"github.com/Rain-kl/Wavelet/internal/apps/agent/state"
|
||||||
|
"github.com/Rain-kl/Wavelet/pkg/pagesarchive"
|
||||||
)
|
)
|
||||||
|
|
||||||
type fakeExecutor struct {
|
type fakeExecutor struct {
|
||||||
@@ -28,15 +33,17 @@ func testPagesSourceConfigJSON(projectID, deploymentID uint, checksum string) st
|
|||||||
}
|
}
|
||||||
|
|
||||||
type fakeClient struct {
|
type fakeClient struct {
|
||||||
config protocol.ActiveConfigResponse
|
config protocol.ActiveConfigResponse
|
||||||
reports []protocol.ApplyLogPayload
|
reports []protocol.ApplyLogPayload
|
||||||
wafSyncCalls []protocol.WAFIPGroupSyncRequest
|
wafSyncCalls []protocol.WAFIPGroupSyncRequest
|
||||||
pagesPackages map[uint][]byte // key: project_id (latest package)
|
pagesPackages map[uint][]byte // key: project_id (latest package)
|
||||||
pagesHashes map[uint]string // key: project_id
|
pagesHashes map[uint]string // key: project_id
|
||||||
pagesLatestDeployIDs map[uint]uint // key: project_id → deployment_id
|
pagesLatestDeployIDs map[uint]uint // key: project_id → deployment_id
|
||||||
wafSyncResult protocol.WAFIPGroupSyncResponse
|
pagesMetadata map[uint]protocol.PagesProjectLatestHashResponse
|
||||||
fetchCalls int
|
pagesPackageDownloads int
|
||||||
hashCalls int
|
wafSyncResult protocol.WAFIPGroupSyncResponse
|
||||||
|
fetchCalls int
|
||||||
|
hashCalls int
|
||||||
}
|
}
|
||||||
|
|
||||||
type fakeManager struct {
|
type fakeManager struct {
|
||||||
@@ -98,17 +105,28 @@ func (f *fakeClient) GetPagesDeploymentHash(ctx context.Context, deploymentID ui
|
|||||||
return f.projectHash(deploymentID)
|
return f.projectHash(deploymentID)
|
||||||
}
|
}
|
||||||
|
|
||||||
func (f *fakeClient) DownloadPagesDeploymentPackage(ctx context.Context, deploymentID uint) ([]byte, error) {
|
func (f *fakeClient) DownloadPagesDeploymentPackage(
|
||||||
|
ctx context.Context,
|
||||||
|
deploymentID uint,
|
||||||
|
dst io.Writer,
|
||||||
|
maxBytes int64,
|
||||||
|
) (int64, error) {
|
||||||
for projectID, depID := range f.pagesLatestDeployIDs {
|
for projectID, depID := range f.pagesLatestDeployIDs {
|
||||||
if depID == deploymentID {
|
if depID == deploymentID {
|
||||||
return f.projectPackage(projectID)
|
return f.writeProjectPackage(projectID, dst, maxBytes)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
return f.projectPackage(deploymentID)
|
return f.writeProjectPackage(deploymentID, dst, maxBytes)
|
||||||
}
|
}
|
||||||
|
|
||||||
func (f *fakeClient) GetPagesProjectLatestHash(ctx context.Context, projectID uint) (*protocol.PagesProjectLatestHashResponse, error) {
|
func (f *fakeClient) GetPagesProjectLatestHash(ctx context.Context, projectID uint) (*protocol.PagesProjectLatestHashResponse, error) {
|
||||||
f.hashCalls++
|
f.hashCalls++
|
||||||
|
if f.pagesMetadata != nil {
|
||||||
|
if metadata, ok := f.pagesMetadata[projectID]; ok {
|
||||||
|
result := metadata
|
||||||
|
return &result, nil
|
||||||
|
}
|
||||||
|
}
|
||||||
hash, err := f.projectHash(projectID)
|
hash, err := f.projectHash(projectID)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
return nil, err
|
return nil, err
|
||||||
@@ -119,15 +137,32 @@ func (f *fakeClient) GetPagesProjectLatestHash(ctx context.Context, projectID ui
|
|||||||
deploymentID = id
|
deploymentID = id
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
packageBytes, err := f.projectPackage(projectID)
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
fileCount, totalSize, err := testPagesPackageStats(packageBytes)
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
return &protocol.PagesProjectLatestHashResponse{
|
return &protocol.PagesProjectLatestHashResponse{
|
||||||
ProjectID: projectID,
|
ProjectID: projectID,
|
||||||
DeploymentID: deploymentID,
|
DeploymentID: deploymentID,
|
||||||
Hash: hash,
|
Hash: hash,
|
||||||
|
PackageSize: int64(len(packageBytes)),
|
||||||
|
FileCount: fileCount,
|
||||||
|
TotalSize: totalSize,
|
||||||
}, nil
|
}, nil
|
||||||
}
|
}
|
||||||
|
|
||||||
func (f *fakeClient) DownloadPagesProjectLatestPackage(ctx context.Context, projectID uint) ([]byte, error) {
|
func (f *fakeClient) DownloadPagesProjectLatestPackage(
|
||||||
return f.projectPackage(projectID)
|
ctx context.Context,
|
||||||
|
projectID uint,
|
||||||
|
dst io.Writer,
|
||||||
|
maxBytes int64,
|
||||||
|
) (int64, error) {
|
||||||
|
f.pagesPackageDownloads++
|
||||||
|
return f.writeProjectPackage(projectID, dst, maxBytes)
|
||||||
}
|
}
|
||||||
|
|
||||||
func (f *fakeClient) projectHash(projectID uint) (string, error) {
|
func (f *fakeClient) projectHash(projectID uint) (string, error) {
|
||||||
@@ -155,6 +190,22 @@ func (f *fakeClient) projectPackage(projectID uint) ([]byte, error) {
|
|||||||
return packageBytes, nil
|
return packageBytes, nil
|
||||||
}
|
}
|
||||||
|
|
||||||
|
func (f *fakeClient) writeProjectPackage(projectID uint, dst io.Writer, maxBytes int64) (int64, error) {
|
||||||
|
packageBytes, err := f.projectPackage(projectID)
|
||||||
|
if err != nil {
|
||||||
|
return 0, err
|
||||||
|
}
|
||||||
|
limited := &io.LimitedReader{R: bytes.NewReader(packageBytes), N: maxBytes + 1}
|
||||||
|
written, err := io.Copy(dst, limited)
|
||||||
|
if err != nil {
|
||||||
|
return written, err
|
||||||
|
}
|
||||||
|
if written > maxBytes {
|
||||||
|
return written, fmt.Errorf("test pages package exceeds limit %d", maxBytes)
|
||||||
|
}
|
||||||
|
return written, nil
|
||||||
|
}
|
||||||
|
|
||||||
func (f *fakeClient) ReportApplyLog(ctx context.Context, payload protocol.ApplyLogPayload) error {
|
func (f *fakeClient) ReportApplyLog(ctx context.Context, payload protocol.ApplyLogPayload) error {
|
||||||
f.reports = append(f.reports, payload)
|
f.reports = append(f.reports, payload)
|
||||||
return nil
|
return nil
|
||||||
@@ -572,7 +623,9 @@ func TestSyncOnceRejectsPagesZipSlipBeforeApply(t *testing.T) {
|
|||||||
service.SetPagesDir(t.TempDir())
|
service.SetPagesDir(t.TempDir())
|
||||||
|
|
||||||
err := service.SyncOnce(context.Background(), &protocol.ActiveConfigMeta{Version: "20260309-102", Checksum: "pages-config-checksum"})
|
err := service.SyncOnce(context.Background(), &protocol.ActiveConfigMeta{Version: "20260309-102", Checksum: "pages-config-checksum"})
|
||||||
if err == nil || (!strings.Contains(err.Error(), "escapes deployment root") && !strings.Contains(err.Error(), "escapes directory")) {
|
if err == nil || (!strings.Contains(err.Error(), "escapes deployment root") &&
|
||||||
|
!strings.Contains(err.Error(), "escapes directory") &&
|
||||||
|
!strings.Contains(err.Error(), "dot segment")) {
|
||||||
t.Fatalf("expected zip-slip rejection, got %v", err)
|
t.Fatalf("expected zip-slip rejection, got %v", err)
|
||||||
}
|
}
|
||||||
if len(manager.applyRouteContents) != 0 {
|
if len(manager.applyRouteContents) != 0 {
|
||||||
@@ -1316,7 +1369,7 @@ func TestSyncOnceRedownloadsPagesDeploymentWhenServerHashChanges(t *testing.T) {
|
|||||||
}
|
}
|
||||||
pagesDir := t.TempDir()
|
pagesDir := t.TempDir()
|
||||||
releaseDir := pagesProjectReleaseDir(pagesDir, projectID, initialHash)
|
releaseDir := pagesProjectReleaseDir(pagesDir, projectID, initialHash)
|
||||||
if err = extractPagesPackage(initialPackage, releaseDir, pagesProjectRef{
|
if err = extractTestPagesPackage(t, initialPackage, releaseDir, pagesProjectRef{
|
||||||
ProjectID: projectID,
|
ProjectID: projectID,
|
||||||
Checksum: initialHash,
|
Checksum: initialHash,
|
||||||
}); err != nil {
|
}); err != nil {
|
||||||
@@ -1385,20 +1438,42 @@ func (r *racingLatestClient) GetPagesProjectLatestHash(ctx context.Context, proj
|
|||||||
// 2: verify after downloading B → B (race)
|
// 2: verify after downloading B → B (race)
|
||||||
// 3+: stable on B for retry
|
// 3+: stable on B for retry
|
||||||
hash, dep := r.hashA, uint(1)
|
hash, dep := r.hashA, uint(1)
|
||||||
|
packageBytes := r.pkgA
|
||||||
if r.hashCall >= 2 {
|
if r.hashCall >= 2 {
|
||||||
hash, dep = r.hashB, 2
|
hash, dep = r.hashB, 2
|
||||||
|
packageBytes = r.pkgB
|
||||||
|
}
|
||||||
|
fileCount, totalSize, err := testPagesPackageStats(packageBytes)
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
}
|
}
|
||||||
return &protocol.PagesProjectLatestHashResponse{
|
return &protocol.PagesProjectLatestHashResponse{
|
||||||
ProjectID: projectID,
|
ProjectID: projectID,
|
||||||
DeploymentID: dep,
|
DeploymentID: dep,
|
||||||
Hash: hash,
|
Hash: hash,
|
||||||
|
PackageSize: int64(len(packageBytes)),
|
||||||
|
FileCount: fileCount,
|
||||||
|
TotalSize: totalSize,
|
||||||
}, nil
|
}, nil
|
||||||
}
|
}
|
||||||
|
|
||||||
func (r *racingLatestClient) DownloadPagesProjectLatestPackage(ctx context.Context, projectID uint) ([]byte, error) {
|
func (r *racingLatestClient) DownloadPagesProjectLatestPackage(
|
||||||
|
ctx context.Context,
|
||||||
|
projectID uint,
|
||||||
|
dst io.Writer,
|
||||||
|
maxBytes int64,
|
||||||
|
) (int64, error) {
|
||||||
r.downloadCalls++
|
r.downloadCalls++
|
||||||
// Always return package B (what "latest download" would stream mid-race / after).
|
// Always return package B (what "latest download" would stream mid-race / after).
|
||||||
return r.pkgB, nil
|
limited := &io.LimitedReader{R: bytes.NewReader(r.pkgB), N: maxBytes + 1}
|
||||||
|
written, err := io.Copy(dst, limited)
|
||||||
|
if err != nil {
|
||||||
|
return written, err
|
||||||
|
}
|
||||||
|
if written > maxBytes {
|
||||||
|
return written, fmt.Errorf("test pages package exceeds limit %d", maxBytes)
|
||||||
|
}
|
||||||
|
return written, nil
|
||||||
}
|
}
|
||||||
|
|
||||||
func TestEnsurePagesProjectSurvivesHashPackageRace(t *testing.T) {
|
func TestEnsurePagesProjectSurvivesHashPackageRace(t *testing.T) {
|
||||||
@@ -1641,6 +1716,41 @@ func testPagesPackage(t *testing.T, files map[string]string) []byte {
|
|||||||
return buffer.Bytes()
|
return buffer.Bytes()
|
||||||
}
|
}
|
||||||
|
|
||||||
|
func extractTestPagesPackage(
|
||||||
|
t *testing.T,
|
||||||
|
packageBytes []byte,
|
||||||
|
releaseDir string,
|
||||||
|
project pagesProjectRef,
|
||||||
|
) error {
|
||||||
|
t.Helper()
|
||||||
|
packagePath := filepath.Join(t.TempDir(), "pages-package.zip")
|
||||||
|
if err := os.WriteFile(packagePath, packageBytes, pagesFilePerm); err != nil {
|
||||||
|
t.Fatalf("write test Pages package error = %v", err)
|
||||||
|
}
|
||||||
|
return extractPagesPackageFile(packagePath, releaseDir, project, pagesarchive.Limits{
|
||||||
|
MaxFiles: agentPagesMaxFiles,
|
||||||
|
MaxFileBytes: agentPagesMaxFileBytes,
|
||||||
|
MaxTotalBytes: agentPagesMaxTotalBytes,
|
||||||
|
}, nil)
|
||||||
|
}
|
||||||
|
|
||||||
|
func testPagesPackageStats(packageBytes []byte) (int, int64, error) {
|
||||||
|
reader, err := zip.NewReader(bytes.NewReader(packageBytes), int64(len(packageBytes)))
|
||||||
|
if err != nil {
|
||||||
|
return 0, 0, err
|
||||||
|
}
|
||||||
|
fileCount := 0
|
||||||
|
totalSize := int64(0)
|
||||||
|
for _, file := range reader.File {
|
||||||
|
if file.FileInfo().IsDir() {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
fileCount++
|
||||||
|
totalSize += int64(file.UncompressedSize64) //nolint:gosec // test packages are memory-bounded
|
||||||
|
}
|
||||||
|
return fileCount, totalSize, nil
|
||||||
|
}
|
||||||
|
|
||||||
func testBytesChecksum(data []byte) string {
|
func testBytesChecksum(data []byte) string {
|
||||||
sum := sha256.Sum256(data)
|
sum := sha256.Sum256(data)
|
||||||
return hex.EncodeToString(sum[:])
|
return hex.EncodeToString(sum[:])
|
||||||
|
|||||||
@@ -224,14 +224,17 @@ func GetPagesProjectLatestHashHandler(c *gin.Context) {
|
|||||||
if !ok {
|
if !ok {
|
||||||
return
|
return
|
||||||
}
|
}
|
||||||
deploymentID, hash, err := pages.GetProjectLatestPackageHash(c.Request.Context(), projectID)
|
metadata, err := pages.GetProjectLatestPackageMetadata(c.Request.Context(), projectID)
|
||||||
if apiutil.AbortBadRequestOnError(c, err) {
|
if apiutil.AbortBadRequestOnError(c, err) {
|
||||||
return
|
return
|
||||||
}
|
}
|
||||||
c.JSON(http.StatusOK, response.OK(protocol.PagesProjectLatestHashResponse{
|
c.JSON(http.StatusOK, response.OK(protocol.PagesProjectLatestHashResponse{
|
||||||
ProjectID: projectID,
|
ProjectID: projectID,
|
||||||
DeploymentID: deploymentID,
|
DeploymentID: metadata.DeploymentID,
|
||||||
Hash: hash,
|
Hash: metadata.Hash,
|
||||||
|
PackageSize: metadata.PackageSize,
|
||||||
|
FileCount: metadata.FileCount,
|
||||||
|
TotalSize: metadata.TotalSize,
|
||||||
}))
|
}))
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|||||||
@@ -7,9 +7,11 @@ import (
|
|||||||
"context"
|
"context"
|
||||||
"errors"
|
"errors"
|
||||||
"fmt"
|
"fmt"
|
||||||
|
"path"
|
||||||
"strings"
|
"strings"
|
||||||
|
|
||||||
"github.com/Rain-kl/Wavelet/internal/model"
|
"github.com/Rain-kl/Wavelet/internal/model"
|
||||||
|
"github.com/Rain-kl/Wavelet/pkg/pagesarchive"
|
||||||
openrestyrender "github.com/Rain-kl/Wavelet/pkg/render/openresty"
|
openrestyrender "github.com/Rain-kl/Wavelet/pkg/render/openresty"
|
||||||
"gorm.io/gorm"
|
"gorm.io/gorm"
|
||||||
)
|
)
|
||||||
@@ -58,23 +60,41 @@ func buildPagesRouteSnapshot(
|
|||||||
}
|
}
|
||||||
|
|
||||||
pagesProjectID = route.PagesProjectID
|
pagesProjectID = route.PagesProjectID
|
||||||
deployment = buildSnapshotPagesDeployment(project, activeDeployment)
|
deployment, err = buildSnapshotPagesDeployment(project, activeDeployment)
|
||||||
|
if err != nil {
|
||||||
|
return "", nil, nil, nil, fmt.Errorf("路由 %s Pages 配置无效: %w", route.SiteName, err)
|
||||||
|
}
|
||||||
originURL = fmt.Sprintf("openflare-pages://project/%d", project.ID)
|
originURL = fmt.Sprintf("openflare-pages://project/%d", project.ID)
|
||||||
return originURL, []string{originURL}, pagesProjectID, deployment, nil
|
return originURL, []string{originURL}, pagesProjectID, deployment, nil
|
||||||
}
|
}
|
||||||
|
|
||||||
func buildSnapshotPagesDeployment(project *model.PagesProject, activeDeployment *model.PagesDeployment) *openrestyrender.PagesDeployment {
|
func buildSnapshotPagesDeployment(
|
||||||
|
project *model.PagesProject,
|
||||||
|
activeDeployment *model.PagesDeployment,
|
||||||
|
) (*openrestyrender.PagesDeployment, error) {
|
||||||
if project == nil || activeDeployment == nil {
|
if project == nil || activeDeployment == nil {
|
||||||
return nil
|
return nil, errors.New("pages 项目或部署为空")
|
||||||
|
}
|
||||||
|
rootDir, err := pagesarchive.NormalizeLogicalPath(strings.TrimSpace(project.RootDir), true)
|
||||||
|
if err != nil {
|
||||||
|
return nil, fmt.Errorf("pages 根目录不合法: %w", err)
|
||||||
}
|
}
|
||||||
entryFile := strings.TrimSpace(project.EntryFile)
|
entryFile := strings.TrimSpace(project.EntryFile)
|
||||||
if entryFile == "" {
|
if entryFile == "" {
|
||||||
entryFile = defaultPagesSnapshotEntryFile
|
entryFile = defaultPagesSnapshotEntryFile
|
||||||
}
|
}
|
||||||
|
entryFile, err = pagesarchive.NormalizeLogicalPath(entryFile, false)
|
||||||
|
if err != nil {
|
||||||
|
return nil, fmt.Errorf("pages 入口文件不合法: %w", err)
|
||||||
|
}
|
||||||
fallbackPath := strings.TrimSpace(project.SPAFallbackPath)
|
fallbackPath := strings.TrimSpace(project.SPAFallbackPath)
|
||||||
if fallbackPath == "" {
|
if fallbackPath == "" {
|
||||||
fallbackPath = defaultPagesSnapshotFallbackPath
|
fallbackPath = defaultPagesSnapshotFallbackPath
|
||||||
}
|
}
|
||||||
|
localRoot := openrestyrender.PagesProjectLocalRoot(project.ID)
|
||||||
|
if rootDir != "" {
|
||||||
|
localRoot = path.Join(localRoot, rootDir)
|
||||||
|
}
|
||||||
return &openrestyrender.PagesDeployment{
|
return &openrestyrender.PagesDeployment{
|
||||||
ProjectID: project.ID,
|
ProjectID: project.ID,
|
||||||
ProjectSlug: strings.TrimSpace(project.Slug),
|
ProjectSlug: strings.TrimSpace(project.Slug),
|
||||||
@@ -90,6 +110,6 @@ func buildSnapshotPagesDeployment(project *model.PagesProject, activeDeployment
|
|||||||
APIProxyRewrite: strings.TrimSpace(project.APIProxyRewrite),
|
APIProxyRewrite: strings.TrimSpace(project.APIProxyRewrite),
|
||||||
// Root is project-scoped so Agents can swap active packages without
|
// Root is project-scoped so Agents can swap active packages without
|
||||||
// re-publishing main config (nginx root stays stable).
|
// re-publishing main config (nginx root stays stable).
|
||||||
LocalRoot: openrestyrender.PagesProjectLocalRoot(project.ID),
|
LocalRoot: localRoot,
|
||||||
}
|
}, nil
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -28,6 +28,7 @@ func TestBuildSnapshotRoutesPages(t *testing.T) {
|
|||||||
Enabled: true,
|
Enabled: true,
|
||||||
SPAFallbackEnabled: true,
|
SPAFallbackEnabled: true,
|
||||||
SPAFallbackPath: "/index.html",
|
SPAFallbackPath: "/index.html",
|
||||||
|
RootDir: "public/site",
|
||||||
EntryFile: "index.html",
|
EntryFile: "index.html",
|
||||||
}
|
}
|
||||||
require.NoError(t, conn.Create(project).Error)
|
require.NoError(t, conn.Create(project).Error)
|
||||||
@@ -64,7 +65,7 @@ func TestBuildSnapshotRoutesPages(t *testing.T) {
|
|||||||
require.NotNil(t, snapshotRoute.PagesDeployment)
|
require.NotNil(t, snapshotRoute.PagesDeployment)
|
||||||
assert.Equal(t, deployment.ID, snapshotRoute.PagesDeployment.DeploymentID)
|
assert.Equal(t, deployment.ID, snapshotRoute.PagesDeployment.DeploymentID)
|
||||||
assert.Equal(t, deployment.Checksum, snapshotRoute.PagesDeployment.Checksum)
|
assert.Equal(t, deployment.Checksum, snapshotRoute.PagesDeployment.Checksum)
|
||||||
assert.Equal(t, "__OPENFLARE_PAGES_DIR__/projects/1/current", snapshotRoute.PagesDeployment.LocalRoot)
|
assert.Equal(t, "__OPENFLARE_PAGES_DIR__/projects/1/current/public/site", snapshotRoute.PagesDeployment.LocalRoot)
|
||||||
|
|
||||||
_, err = renderSnapshotConfig(bundle.SnapshotJSON, nil)
|
_, err = renderSnapshotConfig(bundle.SnapshotJSON, nil)
|
||||||
require.NoError(t, err)
|
require.NoError(t, err)
|
||||||
@@ -78,6 +79,17 @@ func TestBuildSnapshotRoutesPages(t *testing.T) {
|
|||||||
require.NotNil(t, decoded.Routes[0].PagesDeployment)
|
require.NotNil(t, decoded.Routes[0].PagesDeployment)
|
||||||
}
|
}
|
||||||
|
|
||||||
|
func TestBuildSnapshotPagesDeploymentRejectsUnsafeStoredPaths(t *testing.T) {
|
||||||
|
deployment := &model.PagesDeployment{ID: 1, ProjectID: 1, Checksum: "checksum"}
|
||||||
|
for _, project := range []*model.PagesProject{
|
||||||
|
{ID: 1, RootDir: "../escape", EntryFile: "index.html"},
|
||||||
|
{ID: 1, RootDir: "public", EntryFile: "/index.html"},
|
||||||
|
} {
|
||||||
|
_, err := buildSnapshotPagesDeployment(project, deployment)
|
||||||
|
require.Error(t, err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
func requireDB(t *testing.T, ctx context.Context) *gorm.DB {
|
func requireDB(t *testing.T, ctx context.Context) *gorm.DB {
|
||||||
t.Helper()
|
t.Helper()
|
||||||
conn := db.DB(ctx)
|
conn := db.DB(ctx)
|
||||||
|
|||||||
@@ -5,211 +5,38 @@ package pages
|
|||||||
|
|
||||||
import (
|
import (
|
||||||
"context"
|
"context"
|
||||||
"crypto/sha256"
|
|
||||||
"crypto/tls"
|
|
||||||
"encoding/hex"
|
|
||||||
"errors"
|
"errors"
|
||||||
"fmt"
|
|
||||||
"io"
|
|
||||||
"mime"
|
|
||||||
"net"
|
|
||||||
"net/http"
|
|
||||||
"net/url"
|
"net/url"
|
||||||
"os"
|
|
||||||
"path"
|
|
||||||
"path/filepath"
|
|
||||||
"strings"
|
"strings"
|
||||||
"time"
|
|
||||||
|
|
||||||
"github.com/Rain-kl/Wavelet/pkg/pagesarchive"
|
"github.com/Rain-kl/Wavelet/pkg/pagesarchive"
|
||||||
)
|
)
|
||||||
|
|
||||||
const (
|
// downloadPagesPackageFromURL is the deprecated one-shot URL adapter. It uses
|
||||||
pagesURLDownloadTimeout = 10 * time.Minute
|
// the same bounded downloader as persisted sources, with the legacy trusted
|
||||||
pagesURLMaxRedirects = 5
|
// network policy that permits operator-managed internal artifact services.
|
||||||
pagesMagicSniffBytes = 16
|
func downloadPagesPackageFromURL(
|
||||||
pagesURLDialTimeout = 30 * time.Second
|
ctx context.Context,
|
||||||
pagesURLTLSHandshake = 15 * time.Second
|
rawURL string,
|
||||||
pagesBrowserUserAgent = "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/131.0.0.0 Safari/537.36"
|
maxPackageBytes int64,
|
||||||
pagesBrowserAccept = "text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7"
|
) (tempPath string, checksum string, size int64, format pagesarchive.Format, fileName string, err error) {
|
||||||
pagesBrowserAcceptLang = "zh-CN,zh;q=0.9,en-US;q=0.8,en;q=0.7"
|
if _, err := parseAndValidatePagesDownloadURL(rawURL); err != nil {
|
||||||
pagesBrowserSecCHUA = `"Google Chrome";v="131", "Chromium";v="131", "Not_A Brand";v="24"`
|
|
||||||
pagesBrowserSecCHUAMobile = "?0"
|
|
||||||
pagesBrowserSecCHUAPlat = `"Windows"`
|
|
||||||
)
|
|
||||||
|
|
||||||
// downloadPagesPackageFromURL fetches a remote archive with browser-like headers
|
|
||||||
// and writes it to a temp file. Allows private/LAN hosts and insecure TLS certs
|
|
||||||
// (self-signed / internal CA) so operators can pull from internal artifact stores.
|
|
||||||
func downloadPagesPackageFromURL(ctx context.Context, rawURL string, maxPackageBytes int64) (tempPath string, checksum string, size int64, format pagesarchive.Format, fileName string, err error) {
|
|
||||||
parsed, err := parseAndValidatePagesDownloadURL(rawURL)
|
|
||||||
if err != nil {
|
|
||||||
return "", "", 0, "", "", err
|
return "", "", 0, "", "", err
|
||||||
}
|
}
|
||||||
|
candidate, err := FetchRemoteSource(ctx, RemoteSourceRequest{
|
||||||
resp, err := doBrowserDownload(ctx, parsed)
|
URL: strings.TrimSpace(rawURL),
|
||||||
|
NetworkPolicy: RemoteNetworkPolicyTrustedInternal,
|
||||||
|
MaxPackageBytes: maxPackageBytes,
|
||||||
|
})
|
||||||
if err != nil {
|
if err != nil {
|
||||||
return "", "", 0, "", "", err
|
if strings.Contains(err.Error(), errPagesSourceRemoteURLInvalid) {
|
||||||
}
|
return "", "", 0, "", "", errors.New(errPagesPackageURLInvalid)
|
||||||
defer func() { _ = resp.Body.Close() }()
|
|
||||||
if resp.StatusCode < 200 || resp.StatusCode >= 300 {
|
|
||||||
return "", "", 0, "", "", fmt.Errorf("%s: HTTP %d", errPagesPackageURLDownloadFailed, resp.StatusCode)
|
|
||||||
}
|
|
||||||
if resp.ContentLength > 0 && resp.ContentLength > maxPackageBytes {
|
|
||||||
return "", "", 0, "", "", errors.New(errPagesPackageURLTooLarge)
|
|
||||||
}
|
|
||||||
|
|
||||||
fileName = fileNameFromDownload(resp, parsed)
|
|
||||||
format, _ = pagesarchive.DetectFormatFromName(fileName)
|
|
||||||
|
|
||||||
tempPath, checksum, size, err = writeLimitedPackageTemp(resp.Body, format, maxPackageBytes)
|
|
||||||
if err != nil {
|
|
||||||
return "", "", 0, "", "", err
|
|
||||||
}
|
|
||||||
format, fileName, err = ensurePackageFormat(tempPath, format, fileName)
|
|
||||||
if err != nil {
|
|
||||||
_ = os.Remove(tempPath)
|
|
||||||
return "", "", 0, "", "", err
|
|
||||||
}
|
|
||||||
return tempPath, checksum, size, format, fileName, nil
|
|
||||||
}
|
|
||||||
|
|
||||||
func newPagesURLDownloadClient() *http.Client {
|
|
||||||
transport := &http.Transport{
|
|
||||||
Proxy: http.ProxyFromEnvironment,
|
|
||||||
DialContext: (&net.Dialer{
|
|
||||||
Timeout: pagesURLDialTimeout,
|
|
||||||
KeepAlive: pagesURLDialTimeout,
|
|
||||||
}).DialContext,
|
|
||||||
ForceAttemptHTTP2: true,
|
|
||||||
MaxIdleConns: 32,
|
|
||||||
IdleConnTimeout: 90 * time.Second,
|
|
||||||
TLSHandshakeTimeout: pagesURLTLSHandshake,
|
|
||||||
ExpectContinueTimeout: time.Second,
|
|
||||||
// Allow self-signed / internal certificates for artifact hosts.
|
|
||||||
TLSClientConfig: &tls.Config{InsecureSkipVerify: true}, //nolint:gosec // intentional for internal/dev artifact URLs
|
|
||||||
}
|
|
||||||
client := &http.Client{
|
|
||||||
Timeout: pagesURLDownloadTimeout,
|
|
||||||
Transport: transport,
|
|
||||||
}
|
|
||||||
client.CheckRedirect = func(req *http.Request, via []*http.Request) error {
|
|
||||||
if len(via) >= pagesURLMaxRedirects {
|
|
||||||
return errors.New(errPagesPackageURLDownloadFailed)
|
|
||||||
}
|
}
|
||||||
if err := validatePagesDownloadURLValue(req.URL); err != nil {
|
return "", "", 0, "", "", err
|
||||||
return err
|
|
||||||
}
|
|
||||||
applyBrowserDownloadHeaders(req, via[0].URL.String())
|
|
||||||
return nil
|
|
||||||
}
|
|
||||||
return client
|
|
||||||
}
|
|
||||||
|
|
||||||
func doBrowserDownload(ctx context.Context, parsed *url.URL) (*http.Response, error) {
|
|
||||||
client := newPagesURLDownloadClient()
|
|
||||||
req, err := http.NewRequestWithContext(ctx, http.MethodGet, parsed.String(), nil)
|
|
||||||
if err != nil {
|
|
||||||
return nil, errors.New(errPagesPackageURLInvalid)
|
|
||||||
}
|
|
||||||
applyBrowserDownloadHeaders(req, "")
|
|
||||||
resp, err := client.Do(req) //nolint:gosec // scheme validated; private hosts and insecure TLS intentionally allowed
|
|
||||||
if err != nil {
|
|
||||||
return nil, fmt.Errorf("%s: %w", errPagesPackageURLDownloadFailed, err)
|
|
||||||
}
|
|
||||||
return resp, nil
|
|
||||||
}
|
|
||||||
|
|
||||||
func writeLimitedPackageTemp(body io.Reader, format pagesarchive.Format, maxPackageBytes int64) (tempPath, checksum string, size int64, err error) {
|
|
||||||
temp, err := os.CreateTemp("", "openflare-pages-url-*."+safeTempSuffixOrBin(format))
|
|
||||||
if err != nil {
|
|
||||||
return "", "", 0, err
|
|
||||||
}
|
|
||||||
tempPath = temp.Name()
|
|
||||||
defer func() {
|
|
||||||
_ = temp.Close()
|
|
||||||
if err != nil {
|
|
||||||
_ = os.Remove(tempPath)
|
|
||||||
}
|
|
||||||
}()
|
|
||||||
|
|
||||||
hash := sha256.New()
|
|
||||||
written, copyErr := io.Copy(io.MultiWriter(temp, hash), io.LimitReader(body, maxPackageBytes+1))
|
|
||||||
if copyErr != nil {
|
|
||||||
err = fmt.Errorf("%s: %w", errPagesPackageURLDownloadFailed, copyErr)
|
|
||||||
return "", "", 0, err
|
|
||||||
}
|
|
||||||
if written > maxPackageBytes {
|
|
||||||
err = errors.New(errPagesPackageURLTooLarge)
|
|
||||||
return "", "", 0, err
|
|
||||||
}
|
|
||||||
if written == 0 {
|
|
||||||
err = errors.New(errPagesPackageEmpty)
|
|
||||||
return "", "", 0, err
|
|
||||||
}
|
|
||||||
return tempPath, hex.EncodeToString(hash.Sum(nil)), written, nil
|
|
||||||
}
|
|
||||||
|
|
||||||
func ensurePackageFormat(tempPath string, format pagesarchive.Format, fileName string) (pagesarchive.Format, string, error) {
|
|
||||||
if format != "" {
|
|
||||||
return format, fileName, nil
|
|
||||||
}
|
|
||||||
detected, ok := sniffPackageFormat(tempPath)
|
|
||||||
if !ok {
|
|
||||||
return "", fileName, errors.New(errPagesPackageUnsupported)
|
|
||||||
}
|
|
||||||
if !strings.Contains(strings.ToLower(fileName), ".") {
|
|
||||||
fileName = fileName + "." + pagesarchive.Extension(detected)
|
|
||||||
}
|
|
||||||
return detected, fileName, nil
|
|
||||||
}
|
|
||||||
|
|
||||||
func sniffPackageFormat(tempPath string) (pagesarchive.Format, bool) {
|
|
||||||
file, err := os.Open(tempPath) //nolint:gosec // temp path created by us
|
|
||||||
if err != nil {
|
|
||||||
return "", false
|
|
||||||
}
|
|
||||||
defer func() { _ = file.Close() }()
|
|
||||||
head := make([]byte, pagesMagicSniffBytes)
|
|
||||||
n, _ := io.ReadFull(file, head)
|
|
||||||
if n <= 0 {
|
|
||||||
return "", false
|
|
||||||
}
|
|
||||||
return pagesarchive.DetectFormatFromBytes(head[:n])
|
|
||||||
}
|
|
||||||
|
|
||||||
func safeTempSuffixOrBin(format pagesarchive.Format) string {
|
|
||||||
if format == "" {
|
|
||||||
return "bin"
|
|
||||||
}
|
|
||||||
return safeTempSuffix(format)
|
|
||||||
}
|
|
||||||
|
|
||||||
func applyBrowserDownloadHeaders(req *http.Request, referer string) {
|
|
||||||
if req == nil {
|
|
||||||
return
|
|
||||||
}
|
|
||||||
req.Header.Set("User-Agent", pagesBrowserUserAgent)
|
|
||||||
req.Header.Set("Accept", pagesBrowserAccept)
|
|
||||||
req.Header.Set("Accept-Language", pagesBrowserAcceptLang)
|
|
||||||
req.Header.Set("Cache-Control", "no-cache")
|
|
||||||
req.Header.Set("Pragma", "no-cache")
|
|
||||||
req.Header.Set("Upgrade-Insecure-Requests", "1")
|
|
||||||
req.Header.Set("Sec-Fetch-Dest", "document")
|
|
||||||
req.Header.Set("Sec-Fetch-Mode", "navigate")
|
|
||||||
req.Header.Set("Sec-Fetch-Site", "none")
|
|
||||||
req.Header.Set("Sec-Fetch-User", "?1")
|
|
||||||
req.Header.Set("Sec-Ch-Ua", pagesBrowserSecCHUA)
|
|
||||||
req.Header.Set("Sec-Ch-Ua-Mobile", pagesBrowserSecCHUAMobile)
|
|
||||||
req.Header.Set("Sec-Ch-Ua-Platform", pagesBrowserSecCHUAPlat)
|
|
||||||
if referer != "" {
|
|
||||||
req.Header.Set("Referer", referer)
|
|
||||||
req.Header.Set("Sec-Fetch-Site", "cross-site")
|
|
||||||
return
|
|
||||||
}
|
|
||||||
if req.URL != nil {
|
|
||||||
req.Header.Set("Referer", req.URL.Scheme+"://"+req.URL.Host+"/")
|
|
||||||
}
|
}
|
||||||
|
// Ownership transfers to the existing one-shot caller, which removes the
|
||||||
|
// temporary file after the candidate deployment has been created.
|
||||||
|
return candidate.TempPath, candidate.Checksum, candidate.PackageSize, candidate.Format, candidate.SafeLabel, nil
|
||||||
}
|
}
|
||||||
|
|
||||||
func parseAndValidatePagesDownloadURL(raw string) (*url.URL, error) {
|
func parseAndValidatePagesDownloadURL(raw string) (*url.URL, error) {
|
||||||
@@ -218,57 +45,12 @@ func parseAndValidatePagesDownloadURL(raw string) (*url.URL, error) {
|
|||||||
return nil, errors.New(errPagesPackageURLRequired)
|
return nil, errors.New(errPagesPackageURLRequired)
|
||||||
}
|
}
|
||||||
parsed, err := url.Parse(value)
|
parsed, err := url.Parse(value)
|
||||||
if err != nil {
|
if err != nil || parsed.User != nil || parsed.Fragment != "" || parsed.Opaque != "" {
|
||||||
return nil, errors.New(errPagesPackageURLInvalid)
|
return nil, errors.New(errPagesPackageURLInvalid)
|
||||||
}
|
}
|
||||||
if err := validatePagesDownloadURLValue(parsed); err != nil {
|
scheme := strings.ToLower(strings.TrimSpace(parsed.Scheme))
|
||||||
return nil, err
|
if (scheme != remoteSourceSchemeHTTP && scheme != remoteSourceSchemeHTTPS) || strings.TrimSpace(parsed.Hostname()) == "" {
|
||||||
|
return nil, errors.New(errPagesPackageURLInvalid)
|
||||||
}
|
}
|
||||||
return parsed, nil
|
return parsed, nil
|
||||||
}
|
}
|
||||||
|
|
||||||
func validatePagesDownloadURLValue(parsed *url.URL) error {
|
|
||||||
if parsed == nil {
|
|
||||||
return errors.New(errPagesPackageURLInvalid)
|
|
||||||
}
|
|
||||||
scheme := strings.ToLower(strings.TrimSpace(parsed.Scheme))
|
|
||||||
if scheme != "http" && scheme != "https" {
|
|
||||||
return errors.New(errPagesPackageURLInvalid)
|
|
||||||
}
|
|
||||||
if strings.TrimSpace(parsed.Hostname()) == "" {
|
|
||||||
return errors.New(errPagesPackageURLInvalid)
|
|
||||||
}
|
|
||||||
return nil
|
|
||||||
}
|
|
||||||
|
|
||||||
func fileNameFromDownload(resp *http.Response, parsed *url.URL) string {
|
|
||||||
if name := fileNameFromContentDisposition(resp); name != "" {
|
|
||||||
return name
|
|
||||||
}
|
|
||||||
if parsed != nil {
|
|
||||||
base := path.Base(parsed.Path)
|
|
||||||
if base != "" && base != "." && base != "/" {
|
|
||||||
return base
|
|
||||||
}
|
|
||||||
}
|
|
||||||
return "package.bin"
|
|
||||||
}
|
|
||||||
|
|
||||||
func fileNameFromContentDisposition(resp *http.Response) string {
|
|
||||||
if resp == nil {
|
|
||||||
return ""
|
|
||||||
}
|
|
||||||
cd := resp.Header.Get("Content-Disposition")
|
|
||||||
if cd == "" {
|
|
||||||
return ""
|
|
||||||
}
|
|
||||||
_, params, err := mime.ParseMediaType(cd)
|
|
||||||
if err != nil {
|
|
||||||
return ""
|
|
||||||
}
|
|
||||||
name := strings.TrimSpace(params["filename"])
|
|
||||||
if name == "" {
|
|
||||||
return ""
|
|
||||||
}
|
|
||||||
return path.Base(filepath.ToSlash(name))
|
|
||||||
}
|
|
||||||
|
|||||||
@@ -10,7 +10,6 @@ import (
|
|||||||
"net/http"
|
"net/http"
|
||||||
"net/http/httptest"
|
"net/http/httptest"
|
||||||
"os"
|
"os"
|
||||||
"strings"
|
|
||||||
"testing"
|
"testing"
|
||||||
|
|
||||||
"github.com/stretchr/testify/assert"
|
"github.com/stretchr/testify/assert"
|
||||||
@@ -48,10 +47,10 @@ func TestDownloadPagesPackageFromURLAllowsPrivateHost(t *testing.T) {
|
|||||||
require.NoError(t, zw.Close())
|
require.NoError(t, zw.Close())
|
||||||
zipBytes := body.Bytes()
|
zipBytes := body.Bytes()
|
||||||
|
|
||||||
var sawBrowserUA bool
|
var sawProviderUA bool
|
||||||
server := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
server := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
||||||
if strings.Contains(r.Header.Get("User-Agent"), "Mozilla") {
|
if r.Header.Get("User-Agent") == remoteSourceUserAgent {
|
||||||
sawBrowserUA = true
|
sawProviderUA = true
|
||||||
}
|
}
|
||||||
w.Header().Set("Content-Disposition", `attachment; filename="remote-site.zip"`)
|
w.Header().Set("Content-Disposition", `attachment; filename="remote-site.zip"`)
|
||||||
w.Header().Set("Content-Type", "application/zip")
|
w.Header().Set("Content-Type", "application/zip")
|
||||||
@@ -59,12 +58,6 @@ func TestDownloadPagesPackageFromURLAllowsPrivateHost(t *testing.T) {
|
|||||||
}))
|
}))
|
||||||
t.Cleanup(server.Close)
|
t.Cleanup(server.Close)
|
||||||
|
|
||||||
req, err := http.NewRequest(http.MethodGet, server.URL+"/pkg.zip", nil)
|
|
||||||
require.NoError(t, err)
|
|
||||||
applyBrowserDownloadHeaders(req, "")
|
|
||||||
assert.Contains(t, req.Header.Get("User-Agent"), "Mozilla")
|
|
||||||
assert.Contains(t, req.Header.Get("Sec-Fetch-Mode"), "navigate")
|
|
||||||
|
|
||||||
tempPath, checksum, size, format, fileName, err := downloadPagesPackageFromURL(
|
tempPath, checksum, size, format, fileName, err := downloadPagesPackageFromURL(
|
||||||
context.Background(),
|
context.Background(),
|
||||||
server.URL+"/pkg.zip",
|
server.URL+"/pkg.zip",
|
||||||
@@ -72,11 +65,11 @@ func TestDownloadPagesPackageFromURLAllowsPrivateHost(t *testing.T) {
|
|||||||
)
|
)
|
||||||
require.NoError(t, err)
|
require.NoError(t, err)
|
||||||
t.Cleanup(func() { _ = os.Remove(tempPath) })
|
t.Cleanup(func() { _ = os.Remove(tempPath) })
|
||||||
assert.True(t, sawBrowserUA)
|
assert.True(t, sawProviderUA)
|
||||||
assert.NotEmpty(t, checksum)
|
assert.NotEmpty(t, checksum)
|
||||||
assert.Positive(t, size)
|
assert.Positive(t, size)
|
||||||
assert.Equal(t, "zip", string(format))
|
assert.Equal(t, "zip", string(format))
|
||||||
assert.Equal(t, "remote-site.zip", fileName)
|
assert.Equal(t, "pkg.zip", fileName)
|
||||||
}
|
}
|
||||||
|
|
||||||
func TestUploadDeploymentFromURLPrivateHost(t *testing.T) {
|
func TestUploadDeploymentFromURLPrivateHost(t *testing.T) {
|
||||||
|
|||||||
@@ -5,33 +5,64 @@
|
|||||||
package pages
|
package pages
|
||||||
|
|
||||||
const (
|
const (
|
||||||
errPagesProjectNotFound = "pages 项目不存在"
|
errPagesProjectNotFound = "pages 项目不存在"
|
||||||
errPagesSlugExists = "pages 项目标识已存在"
|
errPagesSlugExists = "pages 项目标识已存在"
|
||||||
errPagesNameRequired = "pages 项目名称不能为空"
|
errPagesNameRequired = "pages 项目名称不能为空"
|
||||||
errPagesSlugInvalid = "pages 项目标识只能包含小写字母、数字和连字符"
|
errPagesSlugInvalid = "pages 项目标识只能包含小写字母、数字和连字符"
|
||||||
errPagesDeleteReferenced = "pages 项目已被规则引用,不能删除"
|
errPagesDeleteReferenced = "pages 项目已被规则引用,不能删除"
|
||||||
errPagesDeploymentNotFound = "pages 部署不存在"
|
errPagesDeploymentNotFound = "pages 部署不存在"
|
||||||
errPagesDeploymentMismatch = "pages 部署不属于该项目"
|
errPagesDeploymentMismatch = "pages 部署不属于该项目"
|
||||||
errPagesDeleteActiveDeploy = "不能删除当前激活的 Pages 部署"
|
errPagesDeleteActiveDeploy = "不能删除当前激活的 Pages 部署"
|
||||||
errPagesPackageMissing = "缺少 Pages 部署包"
|
errPagesPackageMissing = "缺少 Pages 部署包"
|
||||||
errPagesPackageURLRequired = "请填写部署包下载链接"
|
errPagesPackageURLRequired = "请填写部署包下载链接"
|
||||||
errPagesPackageURLInvalid = "部署包下载链接无效,仅支持 http/https"
|
errPagesPackageURLInvalid = "部署包下载链接无效,仅支持 http/https"
|
||||||
errPagesPackageURLDownloadFailed = "从链接下载部署包失败"
|
errPagesPackageURLDownloadFailed = "从链接下载部署包失败"
|
||||||
errPagesPackageURLTooLarge = "链接指向的部署包超过大小限制"
|
errPagesPackageURLTooLarge = "链接指向的部署包超过大小限制"
|
||||||
errPagesPackageNotZip = "pages 部署包必须是 .zip 文件" // legacy alias kept for tests
|
errPagesPackageNotZip = "pages 部署包必须是 .zip 文件" // legacy alias kept for tests
|
||||||
errPagesPackageUnsupported = "pages 部署包仅支持 zip、tar.gz、tar.xz、tar.bz2、tar、7z 格式"
|
errPagesPackageUnsupported = "pages 部署包仅支持 zip、tar.gz、tar.xz、tar.bz2、tar、7z 格式"
|
||||||
errPagesPackageInvalidZip = "pages 部署包不是有效 zip 文件" // legacy alias
|
errPagesPackageInvalidZip = "pages 部署包不是有效 zip 文件" // legacy alias
|
||||||
errPagesPackageInvalid = "pages 部署包不是有效的压缩文件"
|
errPagesPackageInvalid = "pages 部署包不是有效的压缩文件"
|
||||||
errPagesPackageEmpty = "pages 部署包不能为空"
|
errPagesPackageEmpty = "pages 部署包不能为空"
|
||||||
errPagesPackageExtractedTooLarge = "pages 部署包展开后体积超过限制"
|
errPagesPackageExtractedTooLarge = "pages 部署包展开后体积超过限制"
|
||||||
errPagesPackageFileTooLarge = "pages 部署包内文件过大"
|
errPagesPackageFileTooLarge = "pages 部署包内文件过大"
|
||||||
errPagesAPIProxyPathRequired = "启用 API 反代时,匹配路径不能为空"
|
errPagesAPIProxyPathRequired = "启用 API 反代时,匹配路径不能为空"
|
||||||
errPagesAPIProxyPathPrefix = "API 反代匹配路径必须以 '/' 开头"
|
errPagesAPIProxyPathPrefix = "API 反代匹配路径必须以 '/' 开头"
|
||||||
errPagesAPIProxyPassRequired = "启用 API 反代时,后端服务地址不能为空" //nolint:gosec // false positive: this is an error message, not hardcoded credentials
|
errPagesAPIProxyPassRequired = "启用 API 反代时,后端服务地址不能为空" //nolint:gosec // false positive: this is an error message, not hardcoded credentials
|
||||||
errPagesAPIProxyPassInvalid = "API 反代后端服务地址必须是有效的 HTTP/HTTPS URL" //nolint:gosec // false positive: this is an error message, not hardcoded credentials
|
errPagesAPIProxyPassInvalid = "API 反代后端服务地址必须是有效的 HTTP/HTTPS URL" //nolint:gosec // false positive: this is an error message, not hardcoded credentials
|
||||||
errPagesPackagePathEmpty = "pages 部署包路径为空"
|
errPagesPackagePathEmpty = "pages 部署包路径为空"
|
||||||
errPagesPackageUploadMissing = "pages 部署包上传记录不存在"
|
errPagesPackageUploadMissing = "pages 部署包上传记录不存在"
|
||||||
errPagesPackageNotInActiveConfig = "pages 部署尚未进入激活配置"
|
errPagesPackageNotInActiveConfig = "pages 部署尚未进入激活配置"
|
||||||
errPagesDeploymentHashMissing = "pages 部署包哈希缺失"
|
errPagesDeploymentHashMissing = "pages 部署包哈希缺失"
|
||||||
errPagesInvalidSnapshotFormat = "配置快照格式无效"
|
errPagesInvalidSnapshotFormat = "配置快照格式无效"
|
||||||
|
errPagesActorMissing = "无法识别当前用户"
|
||||||
|
errPagesEntryFileMissing = "当前激活部署中不存在指定入口文件"
|
||||||
|
errPagesSourceNotFound = "pages 部署源不存在"
|
||||||
|
errPagesSourceTypeRequired = "请选择 pages 部署源类型"
|
||||||
|
errPagesSourceTypeUnsupported = "pages 部署源类型不受支持"
|
||||||
|
errPagesSourceRemoteFields = "远程地址来源不能包含 GitHub 或自动更新配置"
|
||||||
|
errPagesSourceRemoteURLRequired = "请提供远程部署包地址"
|
||||||
|
errPagesSourceRemoteURLMode = "remote_url_set 与 remote_url 参数不匹配"
|
||||||
|
errPagesSourceRemoteURLInvalid = "远程部署包地址无效,仅支持不含用户信息和片段的 http/https 地址"
|
||||||
|
errPagesSourceNetworkPolicy = "远程地址网络策略仅支持 public 或 trusted_internal"
|
||||||
|
errPagesSourceGitHubFields = "GitHub Release 来源不能包含远程地址配置"
|
||||||
|
errPagesSourceRepositoryInvalid = "GitHub 仓库地址无效,仅支持 https://github.com/{owner}/{repo}"
|
||||||
|
errPagesSourceSelectorInvalid = "GitHub Release 选择方式无效"
|
||||||
|
errPagesSourceAssetNameInvalid = "GitHub Release 资源名称必须是安全的文件名"
|
||||||
|
errPagesSourceCheckInterval = "GitHub latest 检查间隔必须在 5 到 1440 分钟之间"
|
||||||
|
errPagesSourceAutoNotAvailable = "自动更新将在后续阶段开放,当前必须保持关闭"
|
||||||
|
errPagesSourceReleaseNotFound = "未找到符合配置的 GitHub Release 资源"
|
||||||
|
errPagesSourceDigestInvalid = "GitHub Release 资源摘要格式无效"
|
||||||
|
errPagesSourceDigestMismatch = "GitHub Release 资源摘要校验失败"
|
||||||
|
errPagesSourceConfirmationNeeded = "检测到同一 Release 的资源已被替换,请刷新并确认当前版本"
|
||||||
|
errPagesSourceConfirmationStale = "确认的版本已变化,请刷新后重新确认"
|
||||||
|
errPagesSourceInitialCheckWarning = "部署源已保存,但首次检查任务入队失败,请稍后手动检查"
|
||||||
|
errPagesSourceCheckUnsupported = "远程地址来源不支持检查更新,请使用立即同步"
|
||||||
|
errPagesSourceActionBusy = "pages 部署源任务正在执行"
|
||||||
|
errPagesSourceActionInvalid = "pages 部署源任务参数无效"
|
||||||
|
errPagesSourceActionStale = "pages 部署源配置已变化,本次任务已跳过"
|
||||||
|
errPagesSourceLeaseLost = "pages 部署源任务执行权已失效"
|
||||||
|
errPagesSourceLeaseExpired = "上次 pages 部署源任务租约已过期"
|
||||||
|
errPagesSourceSyncFailed = "pages 部署源同步失败"
|
||||||
|
errPagesSourceTaskDispatchFailed = "pages 部署源任务入队失败"
|
||||||
|
errPagesSourceInternal = "pages 部署源操作失败,请稍后重试"
|
||||||
)
|
)
|
||||||
|
|||||||
@@ -0,0 +1,347 @@
|
|||||||
|
// Copyright 2026 Arctel.net
|
||||||
|
// SPDX-License-Identifier: Apache-2.0
|
||||||
|
|
||||||
|
package pages
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
"crypto/sha256"
|
||||||
|
"encoding/binary"
|
||||||
|
"encoding/hex"
|
||||||
|
"errors"
|
||||||
|
"net/url"
|
||||||
|
"path"
|
||||||
|
"regexp"
|
||||||
|
"strings"
|
||||||
|
"time"
|
||||||
|
"unicode"
|
||||||
|
"unicode/utf8"
|
||||||
|
|
||||||
|
"github.com/Rain-kl/Wavelet/internal/db"
|
||||||
|
"github.com/Rain-kl/Wavelet/internal/model"
|
||||||
|
"github.com/Rain-kl/Wavelet/pkg/logger"
|
||||||
|
"gorm.io/gorm"
|
||||||
|
"gorm.io/gorm/clause"
|
||||||
|
)
|
||||||
|
|
||||||
|
const (
|
||||||
|
githubReleaseSelectorLatest = "latest"
|
||||||
|
githubReleaseSelectorTag = "tag"
|
||||||
|
githubSourceIdentityDomain = "openflare:pages:github-release:v2"
|
||||||
|
initialCheckRetryDelay = 5 * time.Minute
|
||||||
|
githubRepositoryPathParts = 2
|
||||||
|
githubCheckJitterRange = 301
|
||||||
|
githubCheckJitterCenter = 150
|
||||||
|
)
|
||||||
|
|
||||||
|
var (
|
||||||
|
githubOwnerPattern = regexp.MustCompile(`^[A-Za-z0-9](?:[A-Za-z0-9-]{0,37}[A-Za-z0-9])?$`)
|
||||||
|
githubRepoPattern = regexp.MustCompile(`^[A-Za-z0-9._-]+$`)
|
||||||
|
)
|
||||||
|
|
||||||
|
type githubSourceConfig struct {
|
||||||
|
Repository string
|
||||||
|
Selector string
|
||||||
|
Tag string
|
||||||
|
AssetName string
|
||||||
|
AutoUpdate bool
|
||||||
|
CheckInterval int
|
||||||
|
SourceIdentity string
|
||||||
|
}
|
||||||
|
|
||||||
|
func validateGitHubSourceInput(input SourceUpdateInput) error {
|
||||||
|
if strings.TrimSpace(input.SourceType) != PagesSourceTypeGitHubRelease {
|
||||||
|
return errors.New(errPagesSourceTypeUnsupported)
|
||||||
|
}
|
||||||
|
if input.RemoteURLSet || strings.TrimSpace(input.RemoteURL) != "" ||
|
||||||
|
strings.TrimSpace(input.RemoteNetworkPolicy) != "" {
|
||||||
|
return errors.New(errPagesSourceGitHubFields)
|
||||||
|
}
|
||||||
|
if _, err := normalizeGitHubRepositoryURL(input.RepositoryURL); err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
selector := strings.TrimSpace(input.ReleaseSelector)
|
||||||
|
if selector == "" {
|
||||||
|
selector = githubReleaseSelectorLatest
|
||||||
|
}
|
||||||
|
assetName := input.AssetName
|
||||||
|
if assetName == "" {
|
||||||
|
assetName = defaultGitHubAssetName
|
||||||
|
}
|
||||||
|
if !validGitHubAssetName(assetName) {
|
||||||
|
return errors.New(errPagesSourceAssetNameInvalid)
|
||||||
|
}
|
||||||
|
switch selector {
|
||||||
|
case githubReleaseSelectorLatest:
|
||||||
|
if input.ReleaseTag != "" {
|
||||||
|
return errors.New(errPagesSourceSelectorInvalid)
|
||||||
|
}
|
||||||
|
interval := input.CheckIntervalMinutes
|
||||||
|
if interval != 0 && (interval < minimumCheckInterval || interval > maximumCheckInterval) {
|
||||||
|
return errors.New(errPagesSourceCheckInterval)
|
||||||
|
}
|
||||||
|
case githubReleaseSelectorTag:
|
||||||
|
if !validGitHubReleaseTagConfig(input.ReleaseTag) || input.AutoUpdateEnabled || input.CheckIntervalMinutes != 0 {
|
||||||
|
return errors.New(errPagesSourceSelectorInvalid)
|
||||||
|
}
|
||||||
|
default:
|
||||||
|
return errors.New(errPagesSourceSelectorInvalid)
|
||||||
|
}
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func buildGitHubSourceConfig(input SourceUpdateInput) (githubSourceConfig, error) {
|
||||||
|
repository, err := normalizeGitHubRepositoryURL(input.RepositoryURL)
|
||||||
|
if err != nil {
|
||||||
|
return githubSourceConfig{}, err
|
||||||
|
}
|
||||||
|
selector := strings.TrimSpace(input.ReleaseSelector)
|
||||||
|
if selector == "" {
|
||||||
|
selector = githubReleaseSelectorLatest
|
||||||
|
}
|
||||||
|
tag := input.ReleaseTag
|
||||||
|
assetName := input.AssetName
|
||||||
|
if assetName == "" {
|
||||||
|
assetName = defaultGitHubAssetName
|
||||||
|
}
|
||||||
|
interval := input.CheckIntervalMinutes
|
||||||
|
if selector == githubReleaseSelectorLatest && interval == 0 {
|
||||||
|
interval = defaultCheckInterval
|
||||||
|
}
|
||||||
|
autoUpdate := input.AutoUpdateEnabled
|
||||||
|
if selector == githubReleaseSelectorTag {
|
||||||
|
autoUpdate = false
|
||||||
|
interval = 0
|
||||||
|
}
|
||||||
|
return githubSourceConfig{
|
||||||
|
Repository: repository,
|
||||||
|
Selector: selector,
|
||||||
|
Tag: tag,
|
||||||
|
AssetName: assetName,
|
||||||
|
AutoUpdate: autoUpdate,
|
||||||
|
CheckInterval: interval,
|
||||||
|
SourceIdentity: buildGitHubSourceIdentity(repository, selector, tag, assetName),
|
||||||
|
}, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func buildGitHubSourceIdentity(repository, selector, tag, assetName string) string {
|
||||||
|
fields := [...]string{repository, selector, tag, assetName}
|
||||||
|
encoded := make([]byte, 0, len(githubSourceIdentityDomain)+len(fields)*8+
|
||||||
|
len(repository)+len(selector)+len(tag)+len(assetName))
|
||||||
|
encoded = append(encoded, githubSourceIdentityDomain...)
|
||||||
|
var fieldLength [8]byte
|
||||||
|
for _, field := range fields {
|
||||||
|
// Go strings hold the validated UTF-8 bytes used by GitHub. Prefixing each
|
||||||
|
// field with its byte length prevents delimiter characters from creating
|
||||||
|
// ambiguous identities across field boundaries.
|
||||||
|
binary.BigEndian.PutUint64(fieldLength[:], uint64(len(field)))
|
||||||
|
encoded = append(encoded, fieldLength[:]...)
|
||||||
|
encoded = append(encoded, field...)
|
||||||
|
}
|
||||||
|
identityHash := sha256.Sum256(encoded)
|
||||||
|
return hex.EncodeToString(identityHash[:])
|
||||||
|
}
|
||||||
|
|
||||||
|
func normalizeGitHubRepositoryURL(raw string) (string, error) {
|
||||||
|
parsed, err := url.Parse(raw)
|
||||||
|
if err != nil || parsed.Scheme != "https" || !strings.EqualFold(parsed.Host, "github.com") ||
|
||||||
|
parsed.User != nil || parsed.RawQuery != "" || parsed.ForceQuery || parsed.Fragment != "" ||
|
||||||
|
strings.Contains(raw, "#") ||
|
||||||
|
parsed.EscapedPath() != parsed.Path || !strings.HasPrefix(parsed.Path, "/") ||
|
||||||
|
strings.HasPrefix(parsed.Path, "//") || strings.HasSuffix(parsed.Path, "/") {
|
||||||
|
return "", errors.New(errPagesSourceRepositoryInvalid)
|
||||||
|
}
|
||||||
|
parts := strings.Split(strings.TrimPrefix(parsed.Path, "/"), "/")
|
||||||
|
if len(parts) != githubRepositoryPathParts {
|
||||||
|
return "", errors.New(errPagesSourceRepositoryInvalid)
|
||||||
|
}
|
||||||
|
owner := parts[0]
|
||||||
|
repository := parts[1]
|
||||||
|
repository = strings.TrimSuffix(repository, ".git")
|
||||||
|
if !githubOwnerPattern.MatchString(owner) || !githubRepoPattern.MatchString(repository) ||
|
||||||
|
len(repository) > 100 || repository == "." || repository == ".." {
|
||||||
|
return "", errors.New(errPagesSourceRepositoryInvalid)
|
||||||
|
}
|
||||||
|
return owner + "/" + repository, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func validGitHubReleaseTagConfig(value string) bool {
|
||||||
|
if !validGitHubReleaseDisplayTag(value) ||
|
||||||
|
strings.ContainsAny(value, " ~^:?*[\\") || strings.HasPrefix(value, "/") ||
|
||||||
|
strings.HasSuffix(value, "/") || strings.HasSuffix(value, ".") ||
|
||||||
|
strings.Contains(value, "//") || strings.Contains(value, "..") || strings.Contains(value, "@{") {
|
||||||
|
return false
|
||||||
|
}
|
||||||
|
for component := range strings.SplitSeq(value, "/") {
|
||||||
|
if strings.HasPrefix(component, ".") || strings.HasSuffix(component, ".lock") {
|
||||||
|
return false
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return true
|
||||||
|
}
|
||||||
|
|
||||||
|
func validGitHubReleaseDisplayTag(value string) bool {
|
||||||
|
if value == "" || len(value) > 255 || !utf8.ValidString(value) {
|
||||||
|
return false
|
||||||
|
}
|
||||||
|
for _, character := range value {
|
||||||
|
if unsafeGitHubInputRune(character) {
|
||||||
|
return false
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return true
|
||||||
|
}
|
||||||
|
|
||||||
|
func validGitHubAssetName(value string) bool {
|
||||||
|
if value == "" || len(value) > 255 || !utf8.ValidString(value) ||
|
||||||
|
path.Base(value) != value || strings.Contains(value, "\\") ||
|
||||||
|
value == "." || value == ".." {
|
||||||
|
return false
|
||||||
|
}
|
||||||
|
for _, character := range value {
|
||||||
|
if unsafeGitHubInputRune(character) {
|
||||||
|
return false
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return true
|
||||||
|
}
|
||||||
|
|
||||||
|
func unsafeGitHubInputRune(character rune) bool {
|
||||||
|
return unicode.IsControl(character) || character == '\u2028' || character == '\u2029' ||
|
||||||
|
character == '\u061c' || character == '\u200e' || character == '\u200f' ||
|
||||||
|
(character >= '\u202a' && character <= '\u202e') ||
|
||||||
|
(character >= '\u2066' && character <= '\u2069')
|
||||||
|
}
|
||||||
|
|
||||||
|
func updateGitHubSourceTx(tx *gorm.DB, projectID uint, input SourceUpdateInput) (bool, error) {
|
||||||
|
var project model.PagesProject
|
||||||
|
if err := tx.Clauses(clause.Locking{Strength: pagesRowLockStrength}).First(&project, projectID).Error; err != nil {
|
||||||
|
return false, err
|
||||||
|
}
|
||||||
|
existing, hasExisting, err := loadProjectSourceForUpdate(tx, projectID)
|
||||||
|
if err != nil {
|
||||||
|
return false, err
|
||||||
|
}
|
||||||
|
config, err := buildGitHubSourceConfig(input)
|
||||||
|
if err != nil {
|
||||||
|
return false, err
|
||||||
|
}
|
||||||
|
if !hasExisting {
|
||||||
|
return true, createGitHubSourceTx(tx, projectID, config)
|
||||||
|
}
|
||||||
|
if !githubSourceConfigChanged(existing, config) {
|
||||||
|
return false, nil
|
||||||
|
}
|
||||||
|
var runtime model.PagesProjectSourceRuntime
|
||||||
|
if err := tx.Clauses(clause.Locking{Strength: pagesRowLockStrength}).
|
||||||
|
Where("source_id = ?", existing.ID).First(&runtime).Error; err != nil {
|
||||||
|
return false, err
|
||||||
|
}
|
||||||
|
identityChanged := existing.SourceIdentity != config.SourceIdentity
|
||||||
|
if err := tx.Model(existing).Updates(githubSourceUpdates(config, existing.ConfigVersion+1)).Error; err != nil {
|
||||||
|
return false, err
|
||||||
|
}
|
||||||
|
if err := resetRuntimeAfterGitHubUpdate(tx, &runtime, config, identityChanged); err != nil {
|
||||||
|
return false, err
|
||||||
|
}
|
||||||
|
return true, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func createGitHubSourceTx(tx *gorm.DB, projectID uint, config githubSourceConfig) error {
|
||||||
|
source := &model.PagesProjectSource{
|
||||||
|
ProjectID: projectID,
|
||||||
|
SourceType: PagesSourceTypeGitHubRelease,
|
||||||
|
GitHubRepository: config.Repository,
|
||||||
|
ReleaseSelector: config.Selector,
|
||||||
|
ReleaseTag: config.Tag,
|
||||||
|
AssetName: config.AssetName,
|
||||||
|
AutoUpdateEnabled: config.AutoUpdate,
|
||||||
|
CheckIntervalMinutes: config.CheckInterval,
|
||||||
|
ConfigVersion: 1,
|
||||||
|
SourceIdentity: config.SourceIdentity,
|
||||||
|
}
|
||||||
|
if err := tx.Create(source).Error; err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
runtime := &model.PagesProjectSourceRuntime{SourceID: source.ID, SyncStatus: pagesSourceStatusIdle}
|
||||||
|
if config.Selector == githubReleaseSelectorLatest {
|
||||||
|
next := nextGitHubCheckAt(time.Now(), source.ID, config.CheckInterval)
|
||||||
|
runtime.NextCheckAt = &next
|
||||||
|
}
|
||||||
|
return tx.Create(runtime).Error
|
||||||
|
}
|
||||||
|
|
||||||
|
func githubSourceUpdates(config githubSourceConfig, version int) map[string]any {
|
||||||
|
return map[string]any{
|
||||||
|
"source_type": PagesSourceTypeGitHubRelease,
|
||||||
|
"remote_url": "",
|
||||||
|
"remote_network_policy": "",
|
||||||
|
"github_repository": config.Repository,
|
||||||
|
"release_selector": config.Selector,
|
||||||
|
"release_tag": config.Tag,
|
||||||
|
"asset_name": config.AssetName,
|
||||||
|
sourceColumnAutoUpdateEnabled: config.AutoUpdate,
|
||||||
|
"check_interval_minutes": config.CheckInterval,
|
||||||
|
sourceColumnConfigVersion: version,
|
||||||
|
"source_identity": config.SourceIdentity,
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func githubSourceConfigChanged(existing *model.PagesProjectSource, config githubSourceConfig) bool {
|
||||||
|
return existing.SourceType != PagesSourceTypeGitHubRelease || existing.RemoteURL != "" ||
|
||||||
|
existing.RemoteNetworkPolicy != "" || existing.GitHubRepository != config.Repository ||
|
||||||
|
existing.ReleaseSelector != config.Selector || existing.ReleaseTag != config.Tag ||
|
||||||
|
existing.AssetName != config.AssetName || existing.AutoUpdateEnabled != config.AutoUpdate ||
|
||||||
|
existing.CheckIntervalMinutes != config.CheckInterval
|
||||||
|
}
|
||||||
|
|
||||||
|
func resetRuntimeAfterGitHubUpdate(
|
||||||
|
tx *gorm.DB,
|
||||||
|
runtime *model.PagesProjectSourceRuntime,
|
||||||
|
config githubSourceConfig,
|
||||||
|
identityChanged bool,
|
||||||
|
) error {
|
||||||
|
if err := resetRuntimeAfterSourceUpdate(tx, runtime, identityChanged); err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
var nextCheckAt any
|
||||||
|
if config.Selector == githubReleaseSelectorLatest {
|
||||||
|
next := nextGitHubCheckAt(time.Now(), runtime.SourceID, config.CheckInterval)
|
||||||
|
nextCheckAt = &next
|
||||||
|
}
|
||||||
|
return tx.Model(runtime).Update("next_check_at", nextCheckAt).Error
|
||||||
|
}
|
||||||
|
|
||||||
|
func nextGitHubCheckAt(now time.Time, sourceID uint, intervalMinutes int) time.Time {
|
||||||
|
// A stable, bounded offset avoids a thundering herd without persisting
|
||||||
|
// another scheduling field. Scanner Phase 3 reuses this calculation.
|
||||||
|
jitterSeconds := int64(sourceID%githubCheckJitterRange) - githubCheckJitterCenter
|
||||||
|
return now.Add(time.Duration(intervalMinutes)*time.Minute + time.Duration(jitterSeconds)*time.Second)
|
||||||
|
}
|
||||||
|
|
||||||
|
func markInitialCheckDispatchFailed(ctx context.Context, sourceID uint, configVersion int) {
|
||||||
|
updates := map[string]any{
|
||||||
|
sourceRuntimeColumnSyncStatus: pagesSourceStatusFailed,
|
||||||
|
sourceRuntimeColumnLastError: errPagesSourceInitialCheckWarning,
|
||||||
|
}
|
||||||
|
var source model.PagesProjectSource
|
||||||
|
if err := db.DB(ctx).Where("id = ? AND config_version = ?", sourceID, configVersion).First(&source).Error; err != nil {
|
||||||
|
if !errors.Is(err, gorm.ErrRecordNotFound) {
|
||||||
|
logger.ErrorF(ctx, "[PagesSource] load initial check source snapshot failed: source_id=%d error=%v", sourceID, err)
|
||||||
|
}
|
||||||
|
return
|
||||||
|
}
|
||||||
|
if source.ReleaseSelector == githubReleaseSelectorLatest {
|
||||||
|
next := time.Now().Add(initialCheckRetryDelay)
|
||||||
|
updates["next_check_at"] = &next
|
||||||
|
}
|
||||||
|
now := time.Now()
|
||||||
|
result := db.DB(ctx).Model(&model.PagesProjectSourceRuntime{}).
|
||||||
|
Where("source_id = ?", sourceID).
|
||||||
|
Where("lease_expires_at IS NULL OR lease_expires_at <= ?", now).
|
||||||
|
Where("EXISTS (SELECT 1 FROM of_pages_project_sources source WHERE source.id = ? AND source.config_version = ?)", sourceID, configVersion).
|
||||||
|
Updates(updates)
|
||||||
|
if result.Error != nil {
|
||||||
|
logger.ErrorF(ctx, "[PagesSource] mark initial check dispatch failure: source_id=%d error=%v", sourceID, result.Error)
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -0,0 +1,754 @@
|
|||||||
|
// Copyright 2026 Arctel.net
|
||||||
|
// SPDX-License-Identifier: Apache-2.0
|
||||||
|
|
||||||
|
package pages
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
"crypto/sha256"
|
||||||
|
"encoding/hex"
|
||||||
|
"encoding/json"
|
||||||
|
"errors"
|
||||||
|
"regexp"
|
||||||
|
"strings"
|
||||||
|
"time"
|
||||||
|
|
||||||
|
"github.com/Rain-kl/Wavelet/internal/db"
|
||||||
|
"github.com/Rain-kl/Wavelet/internal/integration/githubrelease"
|
||||||
|
"github.com/Rain-kl/Wavelet/internal/model"
|
||||||
|
"github.com/Rain-kl/Wavelet/internal/task"
|
||||||
|
"github.com/Rain-kl/Wavelet/pkg/logger"
|
||||||
|
"github.com/Rain-kl/Wavelet/pkg/pagesarchive"
|
||||||
|
"gorm.io/gorm"
|
||||||
|
"gorm.io/gorm/clause"
|
||||||
|
)
|
||||||
|
|
||||||
|
const githubSourceDetailProvider = "github"
|
||||||
|
|
||||||
|
var githubDigestPattern = regexp.MustCompile(`^sha256:[0-9a-f]{64}$`)
|
||||||
|
|
||||||
|
type githubSourceProviderDomainError struct {
|
||||||
|
message string
|
||||||
|
permanent bool
|
||||||
|
retryAt *time.Time
|
||||||
|
statusCode int
|
||||||
|
}
|
||||||
|
|
||||||
|
func (domainError *githubSourceProviderDomainError) Error() string {
|
||||||
|
return domainError.message
|
||||||
|
}
|
||||||
|
|
||||||
|
type githubReleaseAPI interface {
|
||||||
|
Resolve(context.Context, githubrelease.ResolveRequest) (githubrelease.ResolveResult, error)
|
||||||
|
Download(context.Context, githubrelease.DownloadRequest) (*githubrelease.DownloadResult, error)
|
||||||
|
}
|
||||||
|
|
||||||
|
var newGitHubReleaseClient = func() githubReleaseAPI {
|
||||||
|
return githubrelease.NewClient()
|
||||||
|
}
|
||||||
|
|
||||||
|
type githubSourceTarget struct {
|
||||||
|
Revision string
|
||||||
|
Detail sourceDetail
|
||||||
|
DetailJSON string
|
||||||
|
Release githubrelease.Release
|
||||||
|
Asset githubrelease.Asset
|
||||||
|
RetryAt *time.Time
|
||||||
|
}
|
||||||
|
|
||||||
|
type githubCheckTaskResult struct {
|
||||||
|
Message string
|
||||||
|
Detail string
|
||||||
|
Revision string
|
||||||
|
Status string
|
||||||
|
RetryAt *time.Time
|
||||||
|
Stale bool
|
||||||
|
}
|
||||||
|
|
||||||
|
type preparedGitHubSource struct {
|
||||||
|
target *githubSourceTarget
|
||||||
|
download *githubrelease.DownloadResult
|
||||||
|
format pagesarchive.Format
|
||||||
|
manifest *deploymentManifest
|
||||||
|
ingestState *sourceIngestState
|
||||||
|
limits pagesLimits
|
||||||
|
}
|
||||||
|
|
||||||
|
func checkGitHubSource(
|
||||||
|
ctx context.Context,
|
||||||
|
snapshot *sourceExecutionSnapshot,
|
||||||
|
) (*githubCheckTaskResult, error) {
|
||||||
|
if snapshot == nil || snapshot.SourceType != PagesSourceTypeGitHubRelease {
|
||||||
|
return nil, errors.New(errPagesSourceTypeUnsupported)
|
||||||
|
}
|
||||||
|
task.AppendLog(ctx, "[check] 正在检查 GitHub Release:repo=%s asset=%s", snapshot.GitHubRepository, snapshot.AssetName)
|
||||||
|
client := newGitHubReleaseClient()
|
||||||
|
result, err := client.Resolve(ctx, githubrelease.ResolveRequest{
|
||||||
|
Repository: snapshot.GitHubRepository,
|
||||||
|
Selector: githubrelease.Selector(snapshot.ReleaseSelector),
|
||||||
|
Tag: snapshot.ReleaseTag,
|
||||||
|
AssetName: snapshot.AssetName,
|
||||||
|
ETag: snapshot.ETag,
|
||||||
|
})
|
||||||
|
if err != nil {
|
||||||
|
logger.WarnF(ctx, "[PagesSource] GitHub resolve failed: source_id=%d repo=%s error=%v", snapshot.SourceID, snapshot.GitHubRepository, err)
|
||||||
|
retryAt, _ := githubrelease.RetryAt(err)
|
||||||
|
domainErr := githubSourceDomainError(err)
|
||||||
|
if failErr := failGitHubCheckLease(ctx, snapshot, domainErr.Error(), retryAt); failErr != nil {
|
||||||
|
if errors.Is(failErr, errSourceFinalFence) {
|
||||||
|
return &githubCheckTaskResult{Message: errPagesSourceActionStale, Stale: true}, nil
|
||||||
|
}
|
||||||
|
return nil, failErr
|
||||||
|
}
|
||||||
|
return nil, domainErr
|
||||||
|
}
|
||||||
|
if result.NotModified {
|
||||||
|
revision, status, err := finishGitHubCheckNotModified(ctx, snapshot, result)
|
||||||
|
if err != nil {
|
||||||
|
if errors.Is(err, errSourceFinalFence) {
|
||||||
|
return &githubCheckTaskResult{Message: errPagesSourceActionStale, Stale: true}, nil
|
||||||
|
}
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
detail, _ := json.Marshal(map[string]string{"revision": revision, pagesDeploymentColumnStatus: status})
|
||||||
|
return &githubCheckTaskResult{
|
||||||
|
Message: "GitHub Release 检查完成,内容未变化",
|
||||||
|
Detail: string(detail),
|
||||||
|
Revision: revision,
|
||||||
|
Status: status,
|
||||||
|
RetryAt: result.RetryAt,
|
||||||
|
}, nil
|
||||||
|
}
|
||||||
|
target, err := buildGitHubSourceTarget(result.Release, result.Asset, result.RetryAt)
|
||||||
|
if err != nil {
|
||||||
|
retryAt := time.Time{}
|
||||||
|
if result.RetryAt != nil {
|
||||||
|
retryAt = result.RetryAt.UTC()
|
||||||
|
}
|
||||||
|
if failErr := failGitHubCheckLease(ctx, snapshot, err.Error(), retryAt); failErr != nil {
|
||||||
|
if errors.Is(failErr, errSourceFinalFence) {
|
||||||
|
return &githubCheckTaskResult{Message: errPagesSourceActionStale, Stale: true}, nil
|
||||||
|
}
|
||||||
|
return nil, failErr
|
||||||
|
}
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
status, err := finishGitHubCheckTarget(ctx, snapshot, result, target)
|
||||||
|
if err != nil {
|
||||||
|
if errors.Is(err, errSourceFinalFence) {
|
||||||
|
return &githubCheckTaskResult{Message: errPagesSourceActionStale, Stale: true}, nil
|
||||||
|
}
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
detail, _ := json.Marshal(map[string]string{"revision": target.Revision, pagesDeploymentColumnStatus: status})
|
||||||
|
message := "GitHub Release 检查完成"
|
||||||
|
switch status {
|
||||||
|
case pagesSourceStatusUpdateAvailable:
|
||||||
|
message = "发现新的 GitHub Release 部署包"
|
||||||
|
case pagesSourceStatusAttention:
|
||||||
|
message = "检测到同一 Release 的资源被替换,需要确认"
|
||||||
|
}
|
||||||
|
return &githubCheckTaskResult{
|
||||||
|
Message: message, Detail: string(detail), Revision: target.Revision,
|
||||||
|
Status: status, RetryAt: result.RetryAt,
|
||||||
|
}, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func buildGitHubSourceTarget(
|
||||||
|
release githubrelease.Release,
|
||||||
|
asset githubrelease.Asset,
|
||||||
|
retryAt *time.Time,
|
||||||
|
) (*githubSourceTarget, error) {
|
||||||
|
digest := strings.ToLower(strings.TrimSpace(asset.Digest))
|
||||||
|
if digest != "" && !githubDigestPattern.MatchString(digest) {
|
||||||
|
return nil, errors.New(errPagesSourceDigestInvalid)
|
||||||
|
}
|
||||||
|
if strings.TrimSpace(release.ID) == "" || strings.TrimSpace(asset.ID) == "" ||
|
||||||
|
!validGitHubReleaseDisplayTag(release.Tag) || !validGitHubAssetName(asset.Name) ||
|
||||||
|
asset.State != "uploaded" || asset.UpdatedAt.IsZero() {
|
||||||
|
return nil, errors.New(errPagesSourceReleaseNotFound)
|
||||||
|
}
|
||||||
|
updatedAt := asset.UpdatedAt.UTC().Format(time.RFC3339Nano)
|
||||||
|
rawRevision := "github:" + release.ID + ":" + asset.ID + ":" + updatedAt + ":" + digest
|
||||||
|
sum := sha256.Sum256([]byte(rawRevision))
|
||||||
|
detail := sourceDetail{
|
||||||
|
Provider: githubSourceDetailProvider,
|
||||||
|
Tag: release.Tag,
|
||||||
|
AssetName: asset.Name,
|
||||||
|
ReleaseID: release.ID,
|
||||||
|
AssetID: asset.ID,
|
||||||
|
AssetUpdatedAt: updatedAt,
|
||||||
|
Digest: digest,
|
||||||
|
}
|
||||||
|
detailJSON, err := json.Marshal(detail)
|
||||||
|
if err != nil {
|
||||||
|
return nil, errors.New(errPagesSourceSyncFailed)
|
||||||
|
}
|
||||||
|
return &githubSourceTarget{
|
||||||
|
Revision: hex.EncodeToString(sum[:]),
|
||||||
|
Detail: detail,
|
||||||
|
DetailJSON: string(detailJSON),
|
||||||
|
Release: release,
|
||||||
|
Asset: asset,
|
||||||
|
RetryAt: retryAt,
|
||||||
|
}, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func finishGitHubCheckNotModified(
|
||||||
|
ctx context.Context,
|
||||||
|
snapshot *sourceExecutionSnapshot,
|
||||||
|
result githubrelease.ResolveResult,
|
||||||
|
) (string, string, error) {
|
||||||
|
var revision string
|
||||||
|
var status string
|
||||||
|
err := db.DB(ctx).Transaction(func(tx *gorm.DB) error {
|
||||||
|
runtime, now, err := lockOwnedSourceRuntime(tx, snapshot)
|
||||||
|
if err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
revision = runtime.LastSeenRevision
|
||||||
|
status = normalizedSourceRuntimeStatus(runtime)
|
||||||
|
updates := githubCheckTerminalUpdates(snapshot, now, result.RetryAt)
|
||||||
|
updates["etag"] = result.ETag
|
||||||
|
updates[sourceRuntimeColumnSyncStatus] = status
|
||||||
|
return tx.Model(runtime).Updates(updates).Error
|
||||||
|
})
|
||||||
|
return revision, status, err
|
||||||
|
}
|
||||||
|
|
||||||
|
func finishGitHubCheckTarget(
|
||||||
|
ctx context.Context,
|
||||||
|
snapshot *sourceExecutionSnapshot,
|
||||||
|
result githubrelease.ResolveResult,
|
||||||
|
target *githubSourceTarget,
|
||||||
|
) (string, error) {
|
||||||
|
status := pagesSourceStatusIdle
|
||||||
|
err := db.DB(ctx).Transaction(func(tx *gorm.DB) error {
|
||||||
|
runtime, now, err := lockOwnedSourceRuntime(tx, snapshot)
|
||||||
|
if err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
status = targetRuntimeStatus(target, runtime.LastAppliedRevision, runtime.LastAppliedDetail)
|
||||||
|
updates := githubCheckTerminalUpdates(snapshot, now, result.RetryAt)
|
||||||
|
updates["etag"] = result.ETag
|
||||||
|
updates["last_seen_revision"] = target.Revision
|
||||||
|
updates["last_seen_detail"] = target.DetailJSON
|
||||||
|
updates[sourceRuntimeColumnSyncStatus] = status
|
||||||
|
return tx.Model(runtime).Updates(updates).Error
|
||||||
|
})
|
||||||
|
return status, err
|
||||||
|
}
|
||||||
|
|
||||||
|
func githubCheckTerminalUpdates(
|
||||||
|
snapshot *sourceExecutionSnapshot,
|
||||||
|
now time.Time,
|
||||||
|
retryAt *time.Time,
|
||||||
|
) map[string]any {
|
||||||
|
updates := map[string]any{
|
||||||
|
sourceRuntimeColumnLastError: "",
|
||||||
|
sourceRuntimeColumnLastCheckedAt: &now,
|
||||||
|
sourceRuntimeColumnLeaseToken: "",
|
||||||
|
sourceRuntimeColumnLeaseExpiresAt: nil,
|
||||||
|
}
|
||||||
|
updates[sourceRuntimeColumnNextCheckAt] = nextCheckAfterGitHubResponse(snapshot, now, retryAt)
|
||||||
|
return updates
|
||||||
|
}
|
||||||
|
|
||||||
|
func nextCheckAfterGitHubResponse(
|
||||||
|
snapshot *sourceExecutionSnapshot,
|
||||||
|
now time.Time,
|
||||||
|
retryAt *time.Time,
|
||||||
|
) any {
|
||||||
|
if snapshot.ReleaseSelector != githubReleaseSelectorLatest {
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
next := nextGitHubCheckAt(now, snapshot.SourceID, snapshot.CheckIntervalMinutes)
|
||||||
|
if retryAt != nil && retryAt.After(next) {
|
||||||
|
next = retryAt.In(now.Location())
|
||||||
|
}
|
||||||
|
return &next
|
||||||
|
}
|
||||||
|
|
||||||
|
func lockOwnedSourceRuntime(
|
||||||
|
tx *gorm.DB,
|
||||||
|
snapshot *sourceExecutionSnapshot,
|
||||||
|
) (*model.PagesProjectSourceRuntime, time.Time, error) {
|
||||||
|
var runtime model.PagesProjectSourceRuntime
|
||||||
|
if err := tx.Clauses(clause.Locking{Strength: pagesRowLockStrength}).
|
||||||
|
Where("source_id = ?", snapshot.SourceID).First(&runtime).Error; err != nil {
|
||||||
|
return nil, time.Time{}, err
|
||||||
|
}
|
||||||
|
now := time.Now()
|
||||||
|
if runtime.LeaseToken != snapshot.LeaseToken || runtime.LeaseExpiresAt == nil ||
|
||||||
|
!runtime.LeaseExpiresAt.After(now) {
|
||||||
|
return nil, time.Time{}, errSourceFinalFence
|
||||||
|
}
|
||||||
|
return &runtime, now, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func failGitHubCheckLease(
|
||||||
|
ctx context.Context,
|
||||||
|
snapshot *sourceExecutionSnapshot,
|
||||||
|
message string,
|
||||||
|
retryAt time.Time,
|
||||||
|
) error {
|
||||||
|
now := time.Now()
|
||||||
|
next := now.Add(initialCheckRetryDelay)
|
||||||
|
if retryAt.After(next) {
|
||||||
|
next = retryAt.In(now.Location())
|
||||||
|
}
|
||||||
|
updates := map[string]any{
|
||||||
|
sourceRuntimeColumnSyncStatus: pagesSourceStatusFailed,
|
||||||
|
sourceRuntimeColumnLastError: safeSourceRuntimeError(message),
|
||||||
|
sourceRuntimeColumnLastCheckedAt: &now,
|
||||||
|
sourceRuntimeColumnLeaseToken: "",
|
||||||
|
sourceRuntimeColumnLeaseExpiresAt: nil,
|
||||||
|
}
|
||||||
|
if snapshot.ReleaseSelector == githubReleaseSelectorLatest {
|
||||||
|
updates[sourceRuntimeColumnNextCheckAt] = &next
|
||||||
|
} else {
|
||||||
|
updates[sourceRuntimeColumnNextCheckAt] = nil
|
||||||
|
}
|
||||||
|
result := db.DB(ctx).Model(&model.PagesProjectSourceRuntime{}).
|
||||||
|
Where("source_id = ? AND lease_token = ? AND lease_expires_at > ?", snapshot.SourceID, snapshot.LeaseToken, now).
|
||||||
|
Updates(updates)
|
||||||
|
if result.Error != nil {
|
||||||
|
return result.Error
|
||||||
|
}
|
||||||
|
if result.RowsAffected != 1 {
|
||||||
|
return errSourceFinalFence
|
||||||
|
}
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func targetRuntimeStatus(
|
||||||
|
target *githubSourceTarget,
|
||||||
|
appliedRevision string,
|
||||||
|
appliedDetail string,
|
||||||
|
) string {
|
||||||
|
if target == nil || target.Revision == appliedRevision {
|
||||||
|
return pagesSourceStatusIdle
|
||||||
|
}
|
||||||
|
applied := sourceDetail{}
|
||||||
|
if unmarshalSourceDetail(appliedDetail, &applied) == nil && target.Detail.ReleaseID != "" &&
|
||||||
|
target.Detail.ReleaseID == applied.ReleaseID {
|
||||||
|
return pagesSourceStatusAttention
|
||||||
|
}
|
||||||
|
return pagesSourceStatusUpdateAvailable
|
||||||
|
}
|
||||||
|
|
||||||
|
func preflightGitHubSyncConfirmation(ctx context.Context, sourceID uint, confirmedRevision string) error {
|
||||||
|
var runtime model.PagesProjectSourceRuntime
|
||||||
|
if err := db.DB(ctx).Where("source_id = ?", sourceID).First(&runtime).Error; err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
replacement := sourceHasSameReleaseReplacement(&runtime)
|
||||||
|
if replacement && confirmedRevision == "" {
|
||||||
|
return errors.New(errPagesSourceConfirmationNeeded)
|
||||||
|
}
|
||||||
|
if confirmedRevision != "" && (!replacement || confirmedRevision != runtime.LastSeenRevision) {
|
||||||
|
return errors.New(errPagesSourceConfirmationStale)
|
||||||
|
}
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func syncGitHubSourceWithTrigger(
|
||||||
|
ctx context.Context,
|
||||||
|
snapshot *sourceExecutionSnapshot,
|
||||||
|
actor string,
|
||||||
|
targetRevision string,
|
||||||
|
confirmedRevision string,
|
||||||
|
triggerType string,
|
||||||
|
) (outcome *sourceSyncOutcome, resultErr error) {
|
||||||
|
if snapshot == nil || snapshot.SourceType != PagesSourceTypeGitHubRelease || !validPagesSourceActor(actor) {
|
||||||
|
return nil, errors.New(errPagesSourceActionInvalid)
|
||||||
|
}
|
||||||
|
if !validSourceDeploymentTrigger(triggerType) {
|
||||||
|
return nil, errors.New(errPagesSourceActionInvalid)
|
||||||
|
}
|
||||||
|
defer func() {
|
||||||
|
resultErr = finalizeGitHubSyncFailure(ctx, snapshot, resultErr)
|
||||||
|
}()
|
||||||
|
workCtx, heartbeat, err := startSourceLeaseHeartbeat(
|
||||||
|
ctx, snapshot, pagesSourceSyncLeaseDuration, pagesSourceHeartbeatInterval,
|
||||||
|
)
|
||||||
|
if err != nil {
|
||||||
|
return sourceHeartbeatOutcome(err)
|
||||||
|
}
|
||||||
|
defer func() { _ = heartbeat.stop() }()
|
||||||
|
|
||||||
|
client := newGitHubReleaseClient()
|
||||||
|
target, guardedOutcome, err := resolveAndGuardGitHubSync(
|
||||||
|
workCtx, client, snapshot, targetRevision, confirmedRevision,
|
||||||
|
)
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
if guardedOutcome != nil {
|
||||||
|
return guardedOutcome, nil
|
||||||
|
}
|
||||||
|
prepared, err := prepareGitHubSyncPackage(workCtx, client, snapshot, target)
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
defer func() {
|
||||||
|
if cleanupErr := prepared.download.Cleanup(); cleanupErr != nil {
|
||||||
|
logger.WarnF(ctx, "[PagesSource] cleanup GitHub package failed: source_id=%d error=%v", snapshot.SourceID, cleanupErr)
|
||||||
|
}
|
||||||
|
}()
|
||||||
|
defer compensateSourceIngest(ctx, snapshot, prepared.ingestState)
|
||||||
|
if heartbeatErr := heartbeat.stop(); heartbeatErr != nil {
|
||||||
|
return sourceHeartbeatOutcome(heartbeatErr)
|
||||||
|
}
|
||||||
|
renewed, err := renewSourceLease(ctx, snapshot, pagesSourceSyncLeaseDuration)
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
if !renewed {
|
||||||
|
return &sourceSyncOutcome{Stale: true}, nil
|
||||||
|
}
|
||||||
|
return activatePreparedGitHubSource(ctx, snapshot, actor, triggerType, prepared)
|
||||||
|
}
|
||||||
|
|
||||||
|
func finalizeGitHubSyncFailure(
|
||||||
|
ctx context.Context,
|
||||||
|
snapshot *sourceExecutionSnapshot,
|
||||||
|
resultErr error,
|
||||||
|
) error {
|
||||||
|
if resultErr == nil {
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
cleanupCtx, cancel := sourceCleanupContext(ctx)
|
||||||
|
defer cancel()
|
||||||
|
finalizerErr := persistGitHubSyncFailure(cleanupCtx, snapshot, resultErr)
|
||||||
|
if finalizerErr == nil {
|
||||||
|
return resultErr
|
||||||
|
}
|
||||||
|
logger.WarnF(
|
||||||
|
cleanupCtx,
|
||||||
|
"[PagesSource] finalize GitHub sync failure failed: source_id=%d source_error=%s error=%v",
|
||||||
|
snapshot.SourceID, safeGitHubSourceError(resultErr), finalizerErr,
|
||||||
|
)
|
||||||
|
// final fence 丢失表示已有新任务接管 runtime,不应覆盖;数据库
|
||||||
|
// finalizer 失败则保持可重试,避免继承永久错误或 provider deadline 分类。
|
||||||
|
if errors.Is(finalizerErr, errSourceFinalFence) {
|
||||||
|
return resultErr
|
||||||
|
}
|
||||||
|
return errors.New(errPagesSourceSyncFailed)
|
||||||
|
}
|
||||||
|
|
||||||
|
func persistGitHubSyncFailure(
|
||||||
|
ctx context.Context,
|
||||||
|
snapshot *sourceExecutionSnapshot,
|
||||||
|
resultErr error,
|
||||||
|
) error {
|
||||||
|
var domainError *githubSourceProviderDomainError
|
||||||
|
if errors.As(resultErr, &domainError) && domainError.retryAt != nil {
|
||||||
|
return failGitHubCheckLease(ctx, snapshot, domainError.message, *domainError.retryAt)
|
||||||
|
}
|
||||||
|
return failSourceLease(ctx, snapshot, safeGitHubSourceError(resultErr))
|
||||||
|
}
|
||||||
|
|
||||||
|
func activatePreparedGitHubSource(
|
||||||
|
ctx context.Context,
|
||||||
|
snapshot *sourceExecutionSnapshot,
|
||||||
|
actor string,
|
||||||
|
triggerType string,
|
||||||
|
prepared *preparedGitHubSource,
|
||||||
|
) (*sourceSyncOutcome, error) {
|
||||||
|
task.AppendLog(ctx, "[activate] 正在原子切换 GitHub Release 部署")
|
||||||
|
deployment, reused, referenced, err := commitSourceDeploymentWithTrigger(
|
||||||
|
ctx, snapshot, prepared.target.Revision, prepared.download.SHA256,
|
||||||
|
prepared.target.Detail, prepared.target.DetailJSON, actor, triggerType, prepared.manifest,
|
||||||
|
prepared.ingestState.Result, prepared.ingestState.HasIngest, prepared.target.RetryAt,
|
||||||
|
)
|
||||||
|
prepared.ingestState.Referenced = referenced
|
||||||
|
if errors.Is(err, errSourceFinalFence) {
|
||||||
|
return &sourceSyncOutcome{Stale: true}, nil
|
||||||
|
}
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
prepared.ingestState.Referenced = prepared.ingestState.HasIngest && deployment.UploadID == prepared.ingestState.Result.Upload.ID
|
||||||
|
if pruneErr := pruneProjectDeploymentHistory(ctx, snapshot.ProjectID, prepared.limits.HistoryCount, 0); pruneErr != nil {
|
||||||
|
logger.ErrorF(ctx, "[PagesSource] strict prune failed after GitHub sync: project_id=%d source_id=%d error=%v", snapshot.ProjectID, snapshot.SourceID, pruneErr)
|
||||||
|
}
|
||||||
|
view := buildDeploymentView(deployment)
|
||||||
|
return &sourceSyncOutcome{Deployment: &view, Reused: reused}, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func resolveAndGuardGitHubSync(
|
||||||
|
ctx context.Context,
|
||||||
|
client githubReleaseAPI,
|
||||||
|
snapshot *sourceExecutionSnapshot,
|
||||||
|
targetRevision string,
|
||||||
|
confirmedRevision string,
|
||||||
|
) (*githubSourceTarget, *sourceSyncOutcome, error) {
|
||||||
|
task.AppendLog(ctx, "[resolve] 正在解析 GitHub Release:repo=%s asset=%s", snapshot.GitHubRepository, snapshot.AssetName)
|
||||||
|
resolved, err := client.Resolve(ctx, githubrelease.ResolveRequest{
|
||||||
|
Repository: snapshot.GitHubRepository,
|
||||||
|
Selector: githubrelease.Selector(snapshot.ReleaseSelector),
|
||||||
|
Tag: snapshot.ReleaseTag,
|
||||||
|
AssetName: snapshot.AssetName,
|
||||||
|
})
|
||||||
|
if err != nil {
|
||||||
|
logger.WarnF(ctx, "[PagesSource] GitHub resolve failed: source_id=%d repo=%s error=%v", snapshot.SourceID, snapshot.GitHubRepository, err)
|
||||||
|
return nil, nil, githubSourceDomainError(err)
|
||||||
|
}
|
||||||
|
if resolved.NotModified {
|
||||||
|
return nil, nil, errors.New(errPagesSourceReleaseNotFound)
|
||||||
|
}
|
||||||
|
target, err := buildGitHubSourceTarget(resolved.Release, resolved.Asset, resolved.RetryAt)
|
||||||
|
if err != nil {
|
||||||
|
return nil, nil, &githubSourceProviderDomainError{
|
||||||
|
message: safeGitHubSourceError(err),
|
||||||
|
permanent: isPermanentSourceSyncError(err),
|
||||||
|
retryAt: resolved.RetryAt,
|
||||||
|
}
|
||||||
|
}
|
||||||
|
guardedOutcome, err := guardGitHubSyncTarget(ctx, snapshot, target, targetRevision, confirmedRevision)
|
||||||
|
return target, guardedOutcome, err
|
||||||
|
}
|
||||||
|
|
||||||
|
func guardGitHubSyncTarget(
|
||||||
|
ctx context.Context,
|
||||||
|
snapshot *sourceExecutionSnapshot,
|
||||||
|
target *githubSourceTarget,
|
||||||
|
targetRevision string,
|
||||||
|
confirmedRevision string,
|
||||||
|
) (*sourceSyncOutcome, error) {
|
||||||
|
status := targetRuntimeStatus(target, snapshot.LastAppliedRevision, snapshot.LastAppliedDetail)
|
||||||
|
if targetRevision != "" && targetRevision != target.Revision {
|
||||||
|
return releaseGuardedGitHubTarget(ctx, snapshot, target, status, "", true, true)
|
||||||
|
}
|
||||||
|
if confirmedRevision != "" && (confirmedRevision != snapshot.LastSeenRevision || confirmedRevision != target.Revision) {
|
||||||
|
return releaseGuardedGitHubTarget(ctx, snapshot, target, status, errPagesSourceConfirmationStale, false, false)
|
||||||
|
}
|
||||||
|
if status == pagesSourceStatusAttention && confirmedRevision != target.Revision {
|
||||||
|
return releaseGuardedGitHubTarget(ctx, snapshot, target, status, errPagesSourceConfirmationNeeded, false, false)
|
||||||
|
}
|
||||||
|
if confirmedRevision != "" && status != pagesSourceStatusAttention {
|
||||||
|
return nil, errors.New(errPagesSourceConfirmationStale)
|
||||||
|
}
|
||||||
|
return nil, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func releaseGuardedGitHubTarget(
|
||||||
|
ctx context.Context,
|
||||||
|
snapshot *sourceExecutionSnapshot,
|
||||||
|
target *githubSourceTarget,
|
||||||
|
status string,
|
||||||
|
lastError string,
|
||||||
|
expedite bool,
|
||||||
|
staleSuccess bool,
|
||||||
|
) (*sourceSyncOutcome, error) {
|
||||||
|
err := releaseGitHubSyncWithoutActivation(ctx, snapshot, target, status, lastError, expedite, target.RetryAt)
|
||||||
|
if errors.Is(err, errSourceFinalFence) || (err == nil && staleSuccess) {
|
||||||
|
return &sourceSyncOutcome{Stale: true}, nil
|
||||||
|
}
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
return nil, errors.New(lastError)
|
||||||
|
}
|
||||||
|
|
||||||
|
func prepareGitHubSyncPackage(
|
||||||
|
ctx context.Context,
|
||||||
|
client githubReleaseAPI,
|
||||||
|
snapshot *sourceExecutionSnapshot,
|
||||||
|
target *githubSourceTarget,
|
||||||
|
) (*preparedGitHubSource, error) {
|
||||||
|
limits := resolvePagesLimits(ctx)
|
||||||
|
task.AppendLog(ctx, "[download] 正在下载 GitHub Release asset:repo=%s asset=%s", snapshot.GitHubRepository, snapshot.AssetName)
|
||||||
|
download, err := client.Download(ctx, githubrelease.DownloadRequest{
|
||||||
|
Repository: snapshot.GitHubRepository,
|
||||||
|
Asset: target.Asset,
|
||||||
|
MaxBytes: limits.PackageBytes,
|
||||||
|
})
|
||||||
|
if err != nil {
|
||||||
|
logger.WarnF(ctx, "[PagesSource] GitHub download failed: source_id=%d repo=%s asset=%s error=%v", snapshot.SourceID, snapshot.GitHubRepository, snapshot.AssetName, err)
|
||||||
|
return nil, githubSourceDomainError(err)
|
||||||
|
}
|
||||||
|
prepared, err := inspectAndIngestGitHubPackage(ctx, snapshot, target, download, limits)
|
||||||
|
if err != nil {
|
||||||
|
if cleanupErr := download.Cleanup(); cleanupErr != nil {
|
||||||
|
logger.WarnF(ctx, "[PagesSource] cleanup GitHub package after preparation failure failed: source_id=%d error=%v", snapshot.SourceID, cleanupErr)
|
||||||
|
}
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
return prepared, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func inspectAndIngestGitHubPackage(
|
||||||
|
ctx context.Context,
|
||||||
|
snapshot *sourceExecutionSnapshot,
|
||||||
|
target *githubSourceTarget,
|
||||||
|
download *githubrelease.DownloadResult,
|
||||||
|
limits pagesLimits,
|
||||||
|
) (*preparedGitHubSource, error) {
|
||||||
|
if download.SHA256 == "" || download.Path == "" {
|
||||||
|
return nil, errors.New(errPagesSourceSyncFailed)
|
||||||
|
}
|
||||||
|
if target.Detail.Digest != "" && "sha256:"+download.SHA256 != target.Detail.Digest {
|
||||||
|
return nil, errors.New(errPagesSourceDigestMismatch)
|
||||||
|
}
|
||||||
|
format, ok := pagesarchive.DetectFormatFromName(target.Asset.Name)
|
||||||
|
var err error
|
||||||
|
if !ok {
|
||||||
|
format, _, err = detectRemoteSourceFormat(download.Path, target.Asset.Name, "")
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
}
|
||||||
|
rootDir, err := validateAndNormalizePagesRootDir(snapshot.RootDir)
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
entryFile, err := validateAndNormalizePagesEntryFile(snapshot.EntryFile)
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
task.AppendLog(ctx, "[verify] 正在校验 GitHub Release 归档与入口")
|
||||||
|
manifest, err := inspectPagesPackage(download.Path, format, rootDir, entryFile, limits)
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
ingestState, err := resolveGitHubSourceIngest(ctx, snapshot, target, download, format)
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
return &preparedGitHubSource{
|
||||||
|
target: target, download: download, format: format,
|
||||||
|
manifest: manifest, ingestState: ingestState, limits: limits,
|
||||||
|
}, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func resolveGitHubSourceIngest(
|
||||||
|
ctx context.Context,
|
||||||
|
snapshot *sourceExecutionSnapshot,
|
||||||
|
target *githubSourceTarget,
|
||||||
|
download *githubrelease.DownloadResult,
|
||||||
|
format pagesarchive.Format,
|
||||||
|
) (*sourceIngestState, error) {
|
||||||
|
if _, err := findSourceDeployment(ctx, snapshot.ProjectID, snapshot.SourceIdentity, target.Revision); err == nil {
|
||||||
|
return &sourceIngestState{}, nil
|
||||||
|
} else if !errors.Is(err, gorm.ErrRecordNotFound) {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
task.AppendLog(ctx, "[ingest] 正在保存 GitHub Release 部署包")
|
||||||
|
result, err := ingestPagesDeploymentPackageWithSource(
|
||||||
|
ctx, download.Path, download.SHA256, snapshot.ProjectID, snapshot.SourceID, target.Asset.Name, format,
|
||||||
|
)
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
return &sourceIngestState{Result: result, HasIngest: true}, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func releaseGitHubSyncWithoutActivation(
|
||||||
|
ctx context.Context,
|
||||||
|
snapshot *sourceExecutionSnapshot,
|
||||||
|
target *githubSourceTarget,
|
||||||
|
status string,
|
||||||
|
lastError string,
|
||||||
|
expedite bool,
|
||||||
|
retryAt *time.Time,
|
||||||
|
) error {
|
||||||
|
now := time.Now()
|
||||||
|
nextCheckAt := nextCheckAfterGitHubResponse(snapshot, now, retryAt)
|
||||||
|
if expedite && snapshot.ReleaseSelector == githubReleaseSelectorLatest {
|
||||||
|
next := now.Add(initialCheckRetryDelay)
|
||||||
|
if retryAt != nil && retryAt.After(next) {
|
||||||
|
next = retryAt.In(now.Location())
|
||||||
|
}
|
||||||
|
nextCheckAt = &next
|
||||||
|
}
|
||||||
|
updates := map[string]any{
|
||||||
|
"last_seen_revision": target.Revision,
|
||||||
|
"last_seen_detail": target.DetailJSON,
|
||||||
|
sourceRuntimeColumnSyncStatus: status,
|
||||||
|
sourceRuntimeColumnLastError: lastError,
|
||||||
|
sourceRuntimeColumnLastCheckedAt: &now,
|
||||||
|
sourceRuntimeColumnNextCheckAt: nextCheckAt,
|
||||||
|
sourceRuntimeColumnLeaseToken: "",
|
||||||
|
sourceRuntimeColumnLeaseExpiresAt: nil,
|
||||||
|
}
|
||||||
|
result := db.DB(ctx).Model(&model.PagesProjectSourceRuntime{}).
|
||||||
|
Where("source_id = ? AND lease_token = ? AND lease_expires_at > ?", snapshot.SourceID, snapshot.LeaseToken, now).
|
||||||
|
Updates(updates)
|
||||||
|
if result.Error != nil {
|
||||||
|
return result.Error
|
||||||
|
}
|
||||||
|
if result.RowsAffected != 1 {
|
||||||
|
return errSourceFinalFence
|
||||||
|
}
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func safeGitHubSourceError(err error) string {
|
||||||
|
if err == nil {
|
||||||
|
return errPagesSourceSyncFailed
|
||||||
|
}
|
||||||
|
message := strings.TrimSpace(err.Error())
|
||||||
|
for _, safeMessage := range []string{
|
||||||
|
errPagesSourceSyncFailed,
|
||||||
|
errPagesSourceReleaseNotFound,
|
||||||
|
errPagesSourceDigestInvalid,
|
||||||
|
errPagesSourceDigestMismatch,
|
||||||
|
errPagesSourceConfirmationNeeded,
|
||||||
|
errPagesSourceConfirmationStale,
|
||||||
|
errPagesPackageURLTooLarge,
|
||||||
|
errPagesPackageEmpty,
|
||||||
|
errPagesPackageUnsupported,
|
||||||
|
errPagesPackageInvalid,
|
||||||
|
errPagesPackageExtractedTooLarge,
|
||||||
|
errPagesPackageFileTooLarge,
|
||||||
|
errPagesEntryFileMissing,
|
||||||
|
} {
|
||||||
|
if message == safeMessage {
|
||||||
|
return safeMessage
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return errPagesSourceSyncFailed
|
||||||
|
}
|
||||||
|
|
||||||
|
func githubSourceDomainError(err error) error {
|
||||||
|
message := errPagesSourceSyncFailed
|
||||||
|
statusCode := 0
|
||||||
|
var providerError *githubrelease.Error
|
||||||
|
if errors.As(err, &providerError) {
|
||||||
|
statusCode = providerError.StatusCode
|
||||||
|
}
|
||||||
|
retryAt, hasRetryAt := githubrelease.RetryAt(err)
|
||||||
|
var retryDeadline *time.Time
|
||||||
|
if hasRetryAt {
|
||||||
|
retryDeadline = &retryAt
|
||||||
|
}
|
||||||
|
if err == nil {
|
||||||
|
return &githubSourceProviderDomainError{message: message, permanent: false, statusCode: statusCode}
|
||||||
|
}
|
||||||
|
if githubrelease.IsDigestError(err) {
|
||||||
|
message = errPagesSourceDigestMismatch
|
||||||
|
return &githubSourceProviderDomainError{message: message, permanent: true, statusCode: statusCode}
|
||||||
|
}
|
||||||
|
if githubrelease.IsNotFound(err) {
|
||||||
|
message = errPagesSourceReleaseNotFound
|
||||||
|
return &githubSourceProviderDomainError{message: message, permanent: true, statusCode: statusCode}
|
||||||
|
}
|
||||||
|
if errors.Is(err, githubrelease.ErrAssetTooLarge) {
|
||||||
|
message = errPagesPackageURLTooLarge
|
||||||
|
return &githubSourceProviderDomainError{message: message, permanent: true, statusCode: statusCode}
|
||||||
|
}
|
||||||
|
if errors.Is(err, githubrelease.ErrEmptyAsset) {
|
||||||
|
message = errPagesPackageEmpty
|
||||||
|
return &githubSourceProviderDomainError{message: message, permanent: true, statusCode: statusCode}
|
||||||
|
}
|
||||||
|
return &githubSourceProviderDomainError{
|
||||||
|
message: message, permanent: !githubrelease.IsRetryable(err), retryAt: retryDeadline, statusCode: statusCode,
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func shouldSkipGitHubActionRetry(err error) bool {
|
||||||
|
var domainError *githubSourceProviderDomainError
|
||||||
|
return errors.As(err, &domainError) && (domainError.permanent || domainError.retryAt != nil)
|
||||||
|
}
|
||||||
@@ -0,0 +1,111 @@
|
|||||||
|
// Copyright 2026 Arctel.net
|
||||||
|
// SPDX-License-Identifier: Apache-2.0
|
||||||
|
|
||||||
|
package pages
|
||||||
|
|
||||||
|
import (
|
||||||
|
"strings"
|
||||||
|
"testing"
|
||||||
|
"time"
|
||||||
|
|
||||||
|
"github.com/Rain-kl/Wavelet/internal/db"
|
||||||
|
"github.com/Rain-kl/Wavelet/internal/model"
|
||||||
|
)
|
||||||
|
|
||||||
|
func TestGitHubSourceIdentityLengthPrefixesFieldsAndResetsRuntime(t *testing.T) {
|
||||||
|
firstInput := SourceUpdateInput{
|
||||||
|
SourceType: PagesSourceTypeGitHubRelease,
|
||||||
|
RepositoryURL: "https://github.com/OpenFlare/site",
|
||||||
|
ReleaseSelector: githubReleaseSelectorTag,
|
||||||
|
ReleaseTag: "release|foo",
|
||||||
|
AssetName: "bar.zip",
|
||||||
|
}
|
||||||
|
secondInput := SourceUpdateInput{
|
||||||
|
SourceType: PagesSourceTypeGitHubRelease,
|
||||||
|
RepositoryURL: "https://github.com/OpenFlare/site",
|
||||||
|
ReleaseSelector: githubReleaseSelectorTag,
|
||||||
|
ReleaseTag: "release",
|
||||||
|
AssetName: "foo|bar.zip",
|
||||||
|
}
|
||||||
|
firstConfig, err := buildGitHubSourceConfig(firstInput)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("buildGitHubSourceConfig(first) error = %v, want nil", err)
|
||||||
|
}
|
||||||
|
secondConfig, err := buildGitHubSourceConfig(secondInput)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("buildGitHubSourceConfig(second) error = %v, want nil", err)
|
||||||
|
}
|
||||||
|
legacyIdentityInput := func(config githubSourceConfig) string {
|
||||||
|
return "github|" + config.Repository + "|" + config.Selector + "|" +
|
||||||
|
config.Tag + "|" + config.AssetName
|
||||||
|
}
|
||||||
|
if firstLegacy, secondLegacy := legacyIdentityInput(firstConfig), legacyIdentityInput(secondConfig); firstLegacy != secondLegacy {
|
||||||
|
t.Fatalf("legacy identity inputs differ: %q != %q; collision fixture is invalid", firstLegacy, secondLegacy)
|
||||||
|
}
|
||||||
|
if firstConfig.SourceIdentity == secondConfig.SourceIdentity {
|
||||||
|
t.Fatalf("length-prefixed identities collide: %q", firstConfig.SourceIdentity)
|
||||||
|
}
|
||||||
|
|
||||||
|
ctx := setupPagesSourceTest(t)
|
||||||
|
project := mustCreatePagesSourceProject(t, ctx, "github-identity-collision")
|
||||||
|
firstSource, _ := mustConfigureGitHubSourceWithoutDispatch(t, ctx, project.ID, firstInput)
|
||||||
|
if got, want := firstSource.SourceIdentity, firstConfig.SourceIdentity; got != want {
|
||||||
|
t.Fatalf("first source identity = %q, want %q", got, want)
|
||||||
|
}
|
||||||
|
|
||||||
|
checkedAt := time.Now().Add(-time.Minute)
|
||||||
|
syncedAt := time.Now().Add(-30 * time.Second)
|
||||||
|
nextCheckAt := time.Now().Add(time.Hour)
|
||||||
|
leaseExpiresAt := time.Now().Add(time.Minute)
|
||||||
|
if err := db.DB(ctx).Model(&model.PagesProjectSourceRuntime{}).
|
||||||
|
Where("source_id = ?", firstSource.ID).
|
||||||
|
Updates(map[string]any{
|
||||||
|
"etag": `"old-etag"`,
|
||||||
|
"last_seen_revision": strings.Repeat("a", 64),
|
||||||
|
"last_seen_detail": `{"provider":"github_release","tag":"release|foo"}`,
|
||||||
|
"last_applied_revision": strings.Repeat("b", 64),
|
||||||
|
"last_applied_detail": `{"provider":"github_release","tag":"older"}`,
|
||||||
|
"sync_status": pagesSourceStatusSyncing,
|
||||||
|
"last_error": "old error",
|
||||||
|
"last_checked_at": &checkedAt,
|
||||||
|
"last_synced_at": &syncedAt,
|
||||||
|
"next_check_at": &nextCheckAt,
|
||||||
|
"lease_expires_at": &leaseExpiresAt,
|
||||||
|
"lease_token": "old-lease",
|
||||||
|
}).Error; err != nil {
|
||||||
|
t.Fatalf("seed runtime cursors error = %v, want nil", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
secondSource, runtime := mustConfigureGitHubSourceWithoutDispatch(t, ctx, project.ID, secondInput)
|
||||||
|
if secondSource.ID != firstSource.ID {
|
||||||
|
t.Errorf("updated source ID = %d, want unchanged %d", secondSource.ID, firstSource.ID)
|
||||||
|
}
|
||||||
|
if got, want := secondSource.SourceIdentity, secondConfig.SourceIdentity; got != want {
|
||||||
|
t.Errorf("updated source identity = %q, want %q", got, want)
|
||||||
|
}
|
||||||
|
if got, want := secondSource.ConfigVersion, firstSource.ConfigVersion+1; got != want {
|
||||||
|
t.Errorf("updated source config version = %d, want %d", got, want)
|
||||||
|
}
|
||||||
|
if runtime.ETag != "" || runtime.LastSeenRevision != "" || runtime.LastSeenDetail != "" ||
|
||||||
|
runtime.LastAppliedRevision != "" || runtime.LastAppliedDetail != "" {
|
||||||
|
t.Errorf("identity change retained runtime cursors: %+v", runtime)
|
||||||
|
}
|
||||||
|
if runtime.LastCheckedAt != nil || runtime.LastSyncedAt != nil || runtime.NextCheckAt != nil {
|
||||||
|
t.Errorf(
|
||||||
|
"identity change retained runtime timestamps: checked=%v synced=%v next=%v",
|
||||||
|
runtime.LastCheckedAt,
|
||||||
|
runtime.LastSyncedAt,
|
||||||
|
runtime.NextCheckAt,
|
||||||
|
)
|
||||||
|
}
|
||||||
|
if runtime.SyncStatus != pagesSourceStatusIdle || runtime.LastError != "" ||
|
||||||
|
runtime.LeaseToken != "" || runtime.LeaseExpiresAt != nil {
|
||||||
|
t.Errorf(
|
||||||
|
"identity change retained runtime state: status=%q error=%q lease=(%q, %v)",
|
||||||
|
runtime.SyncStatus,
|
||||||
|
runtime.LastError,
|
||||||
|
runtime.LeaseToken,
|
||||||
|
runtime.LeaseExpiresAt,
|
||||||
|
)
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -0,0 +1,889 @@
|
|||||||
|
// Copyright 2026 Arctel.net
|
||||||
|
// SPDX-License-Identifier: Apache-2.0
|
||||||
|
|
||||||
|
package pages
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
"crypto/sha256"
|
||||||
|
"encoding/hex"
|
||||||
|
"encoding/json"
|
||||||
|
"errors"
|
||||||
|
"net/http"
|
||||||
|
"os"
|
||||||
|
"path/filepath"
|
||||||
|
"strings"
|
||||||
|
"testing"
|
||||||
|
"time"
|
||||||
|
|
||||||
|
"github.com/Rain-kl/Wavelet/internal/db"
|
||||||
|
"github.com/Rain-kl/Wavelet/internal/integration/githubrelease"
|
||||||
|
"github.com/Rain-kl/Wavelet/internal/model"
|
||||||
|
"github.com/hibiken/asynq"
|
||||||
|
"gorm.io/gorm"
|
||||||
|
)
|
||||||
|
|
||||||
|
type fakeGitHubReleaseClient struct {
|
||||||
|
resolve func(context.Context, githubrelease.ResolveRequest) (githubrelease.ResolveResult, error)
|
||||||
|
download func(context.Context, githubrelease.DownloadRequest) (*githubrelease.DownloadResult, error)
|
||||||
|
}
|
||||||
|
|
||||||
|
func (client *fakeGitHubReleaseClient) Resolve(
|
||||||
|
ctx context.Context,
|
||||||
|
request githubrelease.ResolveRequest,
|
||||||
|
) (githubrelease.ResolveResult, error) {
|
||||||
|
return client.resolve(ctx, request)
|
||||||
|
}
|
||||||
|
|
||||||
|
func (client *fakeGitHubReleaseClient) Download(
|
||||||
|
ctx context.Context,
|
||||||
|
request githubrelease.DownloadRequest,
|
||||||
|
) (*githubrelease.DownloadResult, error) {
|
||||||
|
return client.download(ctx, request)
|
||||||
|
}
|
||||||
|
|
||||||
|
func useFakeGitHubReleaseClient(t *testing.T, client githubReleaseAPI) {
|
||||||
|
t.Helper()
|
||||||
|
previous := newGitHubReleaseClient
|
||||||
|
newGitHubReleaseClient = func() githubReleaseAPI { return client }
|
||||||
|
t.Cleanup(func() { newGitHubReleaseClient = previous })
|
||||||
|
}
|
||||||
|
|
||||||
|
func mustConfigureGitHubSourceWithoutDispatch(
|
||||||
|
t *testing.T,
|
||||||
|
ctx context.Context,
|
||||||
|
projectID uint,
|
||||||
|
input SourceUpdateInput,
|
||||||
|
) (*model.PagesProjectSource, *model.PagesProjectSourceRuntime) {
|
||||||
|
t.Helper()
|
||||||
|
if err := validateGitHubSourceInput(input); err != nil {
|
||||||
|
t.Fatalf("validateGitHubSourceInput(%+v) error = %v, want nil", input, err)
|
||||||
|
}
|
||||||
|
if err := db.DB(ctx).Transaction(func(tx *gorm.DB) error {
|
||||||
|
_, err := updateGitHubSourceTx(tx, projectID, input)
|
||||||
|
return err
|
||||||
|
}); err != nil {
|
||||||
|
t.Fatalf("updateGitHubSourceTx(project=%d) error = %v, want nil", projectID, err)
|
||||||
|
}
|
||||||
|
return mustLoadPagesSource(t, ctx, projectID)
|
||||||
|
}
|
||||||
|
|
||||||
|
func mustLoadPagesSource(
|
||||||
|
t *testing.T,
|
||||||
|
ctx context.Context,
|
||||||
|
projectID uint,
|
||||||
|
) (*model.PagesProjectSource, *model.PagesProjectSourceRuntime) {
|
||||||
|
t.Helper()
|
||||||
|
var source model.PagesProjectSource
|
||||||
|
if err := db.DB(ctx).Where("project_id = ?", projectID).First(&source).Error; err != nil {
|
||||||
|
t.Fatalf("load source for project %d error = %v, want nil", projectID, err)
|
||||||
|
}
|
||||||
|
var runtime model.PagesProjectSourceRuntime
|
||||||
|
if err := db.DB(ctx).Where("source_id = ?", source.ID).First(&runtime).Error; err != nil {
|
||||||
|
t.Fatalf("load runtime for source %d error = %v, want nil", source.ID, err)
|
||||||
|
}
|
||||||
|
return &source, &runtime
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestGitHubSourceValidationNormalizationAndProviderSwitch(t *testing.T) {
|
||||||
|
ctx := setupPagesSourceTest(t)
|
||||||
|
setupPagesSourceDispatchTest(t)
|
||||||
|
project := mustCreatePagesSourceProject(t, ctx, "github-config")
|
||||||
|
input := SourceUpdateInput{
|
||||||
|
SourceType: PagesSourceTypeGitHubRelease,
|
||||||
|
RepositoryURL: "https://github.com/OpenFlare/site.git",
|
||||||
|
}
|
||||||
|
result, err := UpdateSourceAs(ctx, project.ID, input, "user:42")
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("UpdateSourceAs(GitHub) error = %v, want nil", err)
|
||||||
|
}
|
||||||
|
if result.CheckTask == nil || result.CheckTask.Action != sourceActionCheck || result.Warning != "" {
|
||||||
|
t.Errorf("UpdateSourceAs(GitHub) result = %+v, want initial check receipt without warning", result)
|
||||||
|
}
|
||||||
|
execution, err := model.GetTaskExecutionByTaskID(ctx, result.CheckTask.TaskID)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("GetTaskExecutionByTaskID(%q) error = %v, want nil", result.CheckTask.TaskID, err)
|
||||||
|
}
|
||||||
|
var actionPayload SourceActionPayload
|
||||||
|
if err := json.Unmarshal([]byte(execution.Payload), &actionPayload); err != nil {
|
||||||
|
t.Fatalf("json.Unmarshal(initial check payload) error = %v, want nil", err)
|
||||||
|
}
|
||||||
|
if actionPayload.Actor != "user:42" || actionPayload.Action != sourceActionCheck ||
|
||||||
|
actionPayload.TargetRevision != "" || actionPayload.ConfirmedRevision != "" {
|
||||||
|
t.Errorf("initial check payload = %+v, want real actor and credential-free check", actionPayload)
|
||||||
|
}
|
||||||
|
source, runtime := mustLoadPagesSource(t, ctx, project.ID)
|
||||||
|
if got, want := source.GitHubRepository, "OpenFlare/site"; got != want {
|
||||||
|
t.Errorf("GitHubRepository = %q, want %q", got, want)
|
||||||
|
}
|
||||||
|
if got, want := source.ReleaseSelector, githubReleaseSelectorLatest; got != want {
|
||||||
|
t.Errorf("ReleaseSelector = %q, want %q", got, want)
|
||||||
|
}
|
||||||
|
if got, want := source.AssetName, defaultGitHubAssetName; got != want {
|
||||||
|
t.Errorf("AssetName = %q, want %q", got, want)
|
||||||
|
}
|
||||||
|
if got, want := source.CheckIntervalMinutes, defaultCheckInterval; got != want {
|
||||||
|
t.Errorf("CheckIntervalMinutes = %d, want %d", got, want)
|
||||||
|
}
|
||||||
|
if got, want := source.SourceIdentity, "dbbd25307aaa3b88bc25353476940a049428655bd8421ac63045fdcb5fb23c9d"; got != want {
|
||||||
|
t.Errorf("SourceIdentity = %q, want %q", got, want)
|
||||||
|
}
|
||||||
|
if runtime.NextCheckAt == nil {
|
||||||
|
t.Error("GitHub latest NextCheckAt = nil, want scheduled value")
|
||||||
|
}
|
||||||
|
|
||||||
|
var taskCount int64
|
||||||
|
if err := db.DB(ctx).Model(&model.TaskExecution{}).Count(&taskCount).Error; err != nil {
|
||||||
|
t.Fatalf("count initial checks error = %v, want nil", err)
|
||||||
|
}
|
||||||
|
if _, err := UpdateSourceAs(ctx, project.ID, input, "user:42"); err != nil {
|
||||||
|
t.Fatalf("UpdateSourceAs(GitHub no-op) error = %v, want nil", err)
|
||||||
|
}
|
||||||
|
var noOpTaskCount int64
|
||||||
|
if err := db.DB(ctx).Model(&model.TaskExecution{}).Count(&noOpTaskCount).Error; err != nil {
|
||||||
|
t.Fatalf("count no-op checks error = %v, want nil", err)
|
||||||
|
}
|
||||||
|
if noOpTaskCount != taskCount {
|
||||||
|
t.Errorf("no-op initial check count = %d, want unchanged %d", noOpTaskCount, taskCount)
|
||||||
|
}
|
||||||
|
|
||||||
|
secret := "provider-switch-secret"
|
||||||
|
if _, err := UpdateSource(ctx, project.ID, SourceUpdateInput{
|
||||||
|
SourceType: PagesSourceTypeRemoteURL,
|
||||||
|
RemoteURLSet: true,
|
||||||
|
RemoteURL: "https://artifacts.example.com/site.zip?token=" + secret,
|
||||||
|
RemoteNetworkPolicy: RemoteNetworkPolicyPublic,
|
||||||
|
}); err != nil {
|
||||||
|
t.Fatalf("UpdateSource(GitHub to Remote) error = %v, want nil", err)
|
||||||
|
}
|
||||||
|
remote, _ := mustLoadPagesSource(t, ctx, project.ID)
|
||||||
|
if remote.GitHubRepository != "" || remote.ReleaseSelector != "" || remote.AssetName != "" ||
|
||||||
|
remote.AutoUpdateEnabled || remote.CheckIntervalMinutes != 0 {
|
||||||
|
t.Errorf("Remote switched source retained GitHub fields: %+v", remote)
|
||||||
|
}
|
||||||
|
if _, err := UpdateSourceAs(ctx, project.ID, input, "user:42"); err != nil {
|
||||||
|
t.Fatalf("UpdateSourceAs(Remote to GitHub) error = %v, want nil", err)
|
||||||
|
}
|
||||||
|
github, _ := mustLoadPagesSource(t, ctx, project.ID)
|
||||||
|
if github.RemoteURL != "" || github.RemoteNetworkPolicy != "" {
|
||||||
|
t.Errorf("GitHub switched source retained Remote fields: URL=%q policy=%q", github.RemoteURL, github.RemoteNetworkPolicy)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestGitHubSourceSaveSurvivesInitialCheckDispatchFailure(t *testing.T) {
|
||||||
|
ctx := setupPagesSourceTest(t)
|
||||||
|
project := mustCreatePagesSourceProject(t, ctx, "github-dispatch-warning")
|
||||||
|
result, err := UpdateSourceAs(ctx, project.ID, SourceUpdateInput{
|
||||||
|
SourceType: PagesSourceTypeGitHubRelease,
|
||||||
|
RepositoryURL: "https://github.com/a/b",
|
||||||
|
}, "user:9")
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("UpdateSourceAs(dispatch failure) error = %v, want saved source with warning", err)
|
||||||
|
}
|
||||||
|
if result.CheckTask != nil || result.Warning != errPagesSourceInitialCheckWarning {
|
||||||
|
t.Errorf("UpdateSourceAs(dispatch failure) result = %+v, want warning and nil check task", result)
|
||||||
|
}
|
||||||
|
source, runtime := mustLoadPagesSource(t, ctx, project.ID)
|
||||||
|
if source.GitHubRepository != "a/b" || runtime.SyncStatus != pagesSourceStatusFailed ||
|
||||||
|
runtime.LastError != errPagesSourceInitialCheckWarning {
|
||||||
|
t.Errorf("saved source/runtime = repo:%q status:%q error:%q", source.GitHubRepository, runtime.SyncStatus, runtime.LastError)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestGitHubSourceRejectsUnsafeOrModeIncompatibleFields(t *testing.T) {
|
||||||
|
tests := []SourceUpdateInput{
|
||||||
|
{SourceType: PagesSourceTypeGitHubRelease, RepositoryURL: "http://github.com/a/b"},
|
||||||
|
{SourceType: PagesSourceTypeGitHubRelease, RepositoryURL: "https://github.com/a%20b/repo"},
|
||||||
|
{SourceType: PagesSourceTypeGitHubRelease, RepositoryURL: "https://github.com/a/b/extra"},
|
||||||
|
{SourceType: PagesSourceTypeGitHubRelease, RepositoryURL: "https://github.com//a/b"},
|
||||||
|
{SourceType: PagesSourceTypeGitHubRelease, RepositoryURL: "https://github.com/a/b/"},
|
||||||
|
{SourceType: PagesSourceTypeGitHubRelease, RepositoryURL: "https://github.com/a/b?"},
|
||||||
|
{SourceType: PagesSourceTypeGitHubRelease, RepositoryURL: "https://github.com/a/b#"},
|
||||||
|
{SourceType: PagesSourceTypeGitHubRelease, RepositoryURL: "https://github.com/a/b", AssetName: "dist\n.zip"},
|
||||||
|
{SourceType: PagesSourceTypeGitHubRelease, RepositoryURL: "https://github.com/a/b", AssetName: "dist\u202e.zip"},
|
||||||
|
{SourceType: PagesSourceTypeGitHubRelease, RepositoryURL: "https://github.com/a/b", AssetName: "dir/dist.zip"},
|
||||||
|
{SourceType: PagesSourceTypeGitHubRelease, RepositoryURL: "https://github.com/a/b", ReleaseSelector: "tag", ReleaseTag: "v1", CheckIntervalMinutes: 60},
|
||||||
|
{SourceType: PagesSourceTypeGitHubRelease, RepositoryURL: "https://github.com/a/b", ReleaseSelector: "tag", ReleaseTag: "v1", AutoUpdateEnabled: true},
|
||||||
|
{SourceType: PagesSourceTypeGitHubRelease, RepositoryURL: "https://github.com/a/b", ReleaseSelector: "tag", ReleaseTag: " v1"},
|
||||||
|
{SourceType: PagesSourceTypeGitHubRelease, RepositoryURL: "https://github.com/a/b", ReleaseSelector: "tag", ReleaseTag: "v1\n"},
|
||||||
|
{SourceType: PagesSourceTypeGitHubRelease, RepositoryURL: "https://github.com/a/b", ReleaseSelector: "tag", ReleaseTag: "v1\u2028draft"},
|
||||||
|
{SourceType: PagesSourceTypeGitHubRelease, RepositoryURL: "https://github.com/a/b", ReleaseSelector: "tag", ReleaseTag: `v1\draft`},
|
||||||
|
{SourceType: PagesSourceTypeGitHubRelease, RepositoryURL: "https://github.com/a/b", ReleaseSelector: "tag", ReleaseTag: "release//v1"},
|
||||||
|
{SourceType: PagesSourceTypeGitHubRelease, RepositoryURL: "https://github.com/a/b", ReleaseSelector: "tag", ReleaseTag: "release/.draft"},
|
||||||
|
{SourceType: PagesSourceTypeGitHubRelease, RepositoryURL: "https://github.com/a/b", ReleaseSelector: "tag", ReleaseTag: "release/v1.lock"},
|
||||||
|
{SourceType: PagesSourceTypeGitHubRelease, RepositoryURL: "https://github.com/a/b", ReleaseSelector: "latest", ReleaseTag: "v1"},
|
||||||
|
}
|
||||||
|
for _, input := range tests {
|
||||||
|
if err := validateGitHubSourceInput(input); err == nil {
|
||||||
|
t.Errorf("validateGitHubSourceInput(%+v) error = nil, want non-nil", input)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestGitHubSourceAcceptsLegalAssetAndTagCharacters(t *testing.T) {
|
||||||
|
tests := []SourceUpdateInput{
|
||||||
|
{
|
||||||
|
SourceType: PagesSourceTypeGitHubRelease,
|
||||||
|
RepositoryURL: "https://github.com/a/b",
|
||||||
|
AssetName: "dist?channel=stable&part#1.zip",
|
||||||
|
},
|
||||||
|
{
|
||||||
|
SourceType: PagesSourceTypeGitHubRelease,
|
||||||
|
RepositoryURL: "https://github.com/a/b",
|
||||||
|
AssetName: " dist.zip ",
|
||||||
|
},
|
||||||
|
{
|
||||||
|
SourceType: PagesSourceTypeGitHubRelease,
|
||||||
|
RepositoryURL: "https://github.com/a/b",
|
||||||
|
ReleaseSelector: "tag",
|
||||||
|
ReleaseTag: "release/v1#stable&build=1",
|
||||||
|
AssetName: "dist.zip",
|
||||||
|
},
|
||||||
|
{
|
||||||
|
SourceType: PagesSourceTypeGitHubRelease,
|
||||||
|
RepositoryURL: "https://github.com/a/b.git",
|
||||||
|
ReleaseSelector: "tag",
|
||||||
|
ReleaseTag: "@",
|
||||||
|
AssetName: "dist.zip",
|
||||||
|
},
|
||||||
|
{
|
||||||
|
SourceType: PagesSourceTypeGitHubRelease,
|
||||||
|
RepositoryURL: "https://github.com/a/b",
|
||||||
|
ReleaseSelector: "tag",
|
||||||
|
ReleaseTag: "release/v1.LOCK",
|
||||||
|
AssetName: "dist.zip",
|
||||||
|
},
|
||||||
|
}
|
||||||
|
for _, input := range tests {
|
||||||
|
if err := validateGitHubSourceInput(input); err != nil {
|
||||||
|
t.Errorf("validateGitHubSourceInput(%+v) error = %v, want nil", input, err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestInitialCheckFailureUsesExactConfigFence(t *testing.T) {
|
||||||
|
ctx := setupPagesSourceTest(t)
|
||||||
|
project := mustCreatePagesSourceProject(t, ctx, "github-initial-fence")
|
||||||
|
source, _ := mustConfigureGitHubSourceWithoutDispatch(t, ctx, project.ID, SourceUpdateInput{
|
||||||
|
SourceType: PagesSourceTypeGitHubRelease,
|
||||||
|
RepositoryURL: "https://github.com/a/b",
|
||||||
|
})
|
||||||
|
staleVersion := source.ConfigVersion
|
||||||
|
if err := db.DB(ctx).Model(source).Update("config_version", staleVersion+1).Error; err != nil {
|
||||||
|
t.Fatalf("increment source config version error = %v, want nil", err)
|
||||||
|
}
|
||||||
|
markInitialCheckDispatchFailed(ctx, source.ID, staleVersion)
|
||||||
|
_, runtime := mustLoadPagesSource(t, ctx, project.ID)
|
||||||
|
if runtime.SyncStatus != pagesSourceStatusIdle || runtime.LastError != "" {
|
||||||
|
t.Errorf("stale initial failure runtime = status:%q error:%q, want unchanged idle", runtime.SyncStatus, runtime.LastError)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestGitHubCheckUsesETagAndDetectsSameReleaseReplacement(t *testing.T) {
|
||||||
|
ctx := setupPagesSourceTest(t)
|
||||||
|
project := mustCreatePagesSourceProject(t, ctx, "github-check")
|
||||||
|
source, _ := mustConfigureGitHubSourceWithoutDispatch(t, ctx, project.ID, SourceUpdateInput{
|
||||||
|
SourceType: PagesSourceTypeGitHubRelease,
|
||||||
|
RepositoryURL: "https://github.com/a/b",
|
||||||
|
})
|
||||||
|
appliedRevision := strings.Repeat("a", 64)
|
||||||
|
appliedDetail := `{"provider":"github","release_id":"100","asset_id":"1","tag":"release/v1","asset_name":"dist.zip"}`
|
||||||
|
if err := db.DB(ctx).Model(&model.PagesProjectSourceRuntime{}).Where("source_id = ?", source.ID).Updates(map[string]any{
|
||||||
|
"etag": `"old-etag"`,
|
||||||
|
"last_applied_revision": appliedRevision,
|
||||||
|
"last_applied_detail": appliedDetail,
|
||||||
|
}).Error; err != nil {
|
||||||
|
t.Fatalf("seed GitHub runtime error = %v, want nil", err)
|
||||||
|
}
|
||||||
|
updatedAt := time.Date(2026, 7, 19, 10, 0, 0, 0, time.UTC)
|
||||||
|
var gotETag string
|
||||||
|
useFakeGitHubReleaseClient(t, &fakeGitHubReleaseClient{
|
||||||
|
resolve: func(_ context.Context, request githubrelease.ResolveRequest) (githubrelease.ResolveResult, error) {
|
||||||
|
gotETag = request.ETag
|
||||||
|
return githubrelease.ResolveResult{
|
||||||
|
ETag: `"new-etag"`,
|
||||||
|
Release: githubrelease.Release{ID: "100", Tag: "release/v1"},
|
||||||
|
Asset: githubrelease.Asset{ID: "2", Name: "dist.zip", State: "uploaded", UpdatedAt: updatedAt},
|
||||||
|
}, nil
|
||||||
|
},
|
||||||
|
download: func(context.Context, githubrelease.DownloadRequest) (*githubrelease.DownloadResult, error) {
|
||||||
|
t.Fatal("Download called during check, want resolve only")
|
||||||
|
return nil, nil
|
||||||
|
},
|
||||||
|
})
|
||||||
|
snapshot, outcome, err := acquireSourceLease(ctx, source.ID, source.ConfigVersion, sourceActionCheck)
|
||||||
|
if err != nil || outcome != sourceLeaseAcquired {
|
||||||
|
t.Fatalf("acquire check lease = (%+v, %q, %v), want acquired", snapshot, outcome, err)
|
||||||
|
}
|
||||||
|
result, err := checkGitHubSource(ctx, snapshot)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("checkGitHubSource() error = %v, want nil", err)
|
||||||
|
}
|
||||||
|
if result.Stale {
|
||||||
|
t.Error("checkGitHubSource() stale = true, want false")
|
||||||
|
}
|
||||||
|
if got, want := gotETag, `"old-etag"`; got != want {
|
||||||
|
t.Errorf("Resolve ETag = %q, want %q", got, want)
|
||||||
|
}
|
||||||
|
_, runtime := mustLoadPagesSource(t, ctx, project.ID)
|
||||||
|
if runtime.SyncStatus != pagesSourceStatusAttention || runtime.LastSeenRevision == "" {
|
||||||
|
t.Errorf("replacement runtime = status:%q seen:%q, want attention with revision", runtime.SyncStatus, runtime.LastSeenRevision)
|
||||||
|
}
|
||||||
|
view, err := GetSource(ctx, project.ID)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("GetSource() error = %v, want nil", err)
|
||||||
|
}
|
||||||
|
if view.LastSeen == nil || view.LastSeen.Label != "release/v1" {
|
||||||
|
t.Errorf("LastSeen = %+v, want full tag with slash", view.LastSeen)
|
||||||
|
}
|
||||||
|
if err := preflightGitHubSyncConfirmation(ctx, source.ID, ""); err == nil || err.Error() != errPagesSourceConfirmationNeeded {
|
||||||
|
t.Errorf("preflight without confirmation error = %v, want %q", err, errPagesSourceConfirmationNeeded)
|
||||||
|
}
|
||||||
|
if err := preflightGitHubSyncConfirmation(ctx, source.ID, runtime.LastSeenRevision); err != nil {
|
||||||
|
t.Errorf("preflight exact confirmation error = %v, want nil", err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestGitHubCheckNotModifiedRefreshesRuntimeWithoutDeployment(t *testing.T) {
|
||||||
|
ctx := setupPagesSourceTest(t)
|
||||||
|
project := mustCreatePagesSourceProject(t, ctx, "github-304")
|
||||||
|
source, _ := mustConfigureGitHubSourceWithoutDispatch(t, ctx, project.ID, SourceUpdateInput{
|
||||||
|
SourceType: PagesSourceTypeGitHubRelease,
|
||||||
|
RepositoryURL: "https://github.com/a/b",
|
||||||
|
})
|
||||||
|
useFakeGitHubReleaseClient(t, &fakeGitHubReleaseClient{
|
||||||
|
resolve: func(_ context.Context, request githubrelease.ResolveRequest) (githubrelease.ResolveResult, error) {
|
||||||
|
return githubrelease.ResolveResult{NotModified: true, ETag: `"same"`}, nil
|
||||||
|
},
|
||||||
|
download: func(context.Context, githubrelease.DownloadRequest) (*githubrelease.DownloadResult, error) {
|
||||||
|
t.Fatal("Download called for 304 check")
|
||||||
|
return nil, nil
|
||||||
|
},
|
||||||
|
})
|
||||||
|
snapshot, _, _ := acquireSourceLease(ctx, source.ID, source.ConfigVersion, sourceActionCheck)
|
||||||
|
if _, err := checkGitHubSource(ctx, snapshot); err != nil {
|
||||||
|
t.Fatalf("checkGitHubSource(304) error = %v, want nil", err)
|
||||||
|
}
|
||||||
|
_, runtime := mustLoadPagesSource(t, ctx, project.ID)
|
||||||
|
if runtime.ETag != `"same"` || runtime.LastCheckedAt == nil || runtime.NextCheckAt == nil || runtime.LeaseToken != "" {
|
||||||
|
t.Errorf("304 runtime = %+v, want refreshed timestamps/etag and released lease", runtime)
|
||||||
|
}
|
||||||
|
var deployments int64
|
||||||
|
if err := db.DB(ctx).Model(&model.PagesDeployment{}).Where("project_id = ?", project.ID).Count(&deployments).Error; err != nil {
|
||||||
|
t.Fatalf("count deployments error = %v, want nil", err)
|
||||||
|
}
|
||||||
|
if deployments != 0 {
|
||||||
|
t.Errorf("deployments after check = %d, want 0", deployments)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestGitHubTargetMismatchPreservesAttentionAndExpeditesRecheck(t *testing.T) {
|
||||||
|
ctx := setupPagesSourceTest(t)
|
||||||
|
project := mustCreatePagesSourceProject(t, ctx, "github-target-mismatch")
|
||||||
|
source, _ := mustConfigureGitHubSourceWithoutDispatch(t, ctx, project.ID, SourceUpdateInput{
|
||||||
|
SourceType: PagesSourceTypeGitHubRelease,
|
||||||
|
RepositoryURL: "https://github.com/a/b",
|
||||||
|
})
|
||||||
|
appliedRevision := strings.Repeat("a", 64)
|
||||||
|
if err := db.DB(ctx).Model(&model.PagesProjectSourceRuntime{}).Where("source_id = ?", source.ID).Updates(map[string]any{
|
||||||
|
"last_applied_revision": appliedRevision,
|
||||||
|
"last_applied_detail": `{"provider":"github","release_id":"100","asset_id":"1","tag":"v1","asset_name":"dist.zip"}`,
|
||||||
|
}).Error; err != nil {
|
||||||
|
t.Fatalf("seed applied runtime error = %v, want nil", err)
|
||||||
|
}
|
||||||
|
retryAt := time.Now().Add(2 * time.Hour).UTC()
|
||||||
|
useFakeGitHubReleaseClient(t, &fakeGitHubReleaseClient{
|
||||||
|
resolve: func(context.Context, githubrelease.ResolveRequest) (githubrelease.ResolveResult, error) {
|
||||||
|
return githubrelease.ResolveResult{
|
||||||
|
Release: githubrelease.Release{ID: "100", Tag: "v1"},
|
||||||
|
Asset: githubrelease.Asset{
|
||||||
|
ID: "2", Name: "dist.zip", State: "uploaded",
|
||||||
|
UpdatedAt: time.Date(2026, 7, 19, 12, 0, 0, 0, time.UTC),
|
||||||
|
},
|
||||||
|
RetryAt: &retryAt,
|
||||||
|
}, nil
|
||||||
|
},
|
||||||
|
download: func(context.Context, githubrelease.DownloadRequest) (*githubrelease.DownloadResult, error) {
|
||||||
|
t.Fatal("Download called after target mismatch")
|
||||||
|
return nil, nil
|
||||||
|
},
|
||||||
|
})
|
||||||
|
snapshot, _, _ := acquireSourceLease(ctx, source.ID, source.ConfigVersion, sourceActionSync)
|
||||||
|
outcome, err := syncGitHubSource(ctx, snapshot, pagesSourceCreatedBySystem, strings.Repeat("b", 64), "")
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("syncGitHubSource(target mismatch) error = %v, want nil stale outcome", err)
|
||||||
|
}
|
||||||
|
if outcome == nil || !outcome.Stale {
|
||||||
|
t.Errorf("syncGitHubSource(target mismatch) = %+v, want stale", outcome)
|
||||||
|
}
|
||||||
|
_, runtime := mustLoadPagesSource(t, ctx, project.ID)
|
||||||
|
if runtime.SyncStatus != pagesSourceStatusAttention {
|
||||||
|
t.Errorf("target mismatch SyncStatus = %q, want %q", runtime.SyncStatus, pagesSourceStatusAttention)
|
||||||
|
}
|
||||||
|
if runtime.NextCheckAt == nil || runtime.NextCheckAt.Before(retryAt) {
|
||||||
|
t.Errorf("target mismatch NextCheckAt = %v, want server deadline >= %v", runtime.NextCheckAt, retryAt)
|
||||||
|
}
|
||||||
|
var deployments int64
|
||||||
|
if err := db.DB(ctx).Model(&model.PagesDeployment{}).Where("project_id = ?", project.ID).Count(&deployments).Error; err != nil {
|
||||||
|
t.Fatalf("count mismatch deployments error = %v, want nil", err)
|
||||||
|
}
|
||||||
|
if deployments != 0 {
|
||||||
|
t.Errorf("target mismatch deployments = %d, want 0", deployments)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestGitHubCheckLostLeaseReturnsStaleWithoutOverwritingRuntime(t *testing.T) {
|
||||||
|
ctx := setupPagesSourceTest(t)
|
||||||
|
project := mustCreatePagesSourceProject(t, ctx, "github-check-fence")
|
||||||
|
source, _ := mustConfigureGitHubSourceWithoutDispatch(t, ctx, project.ID, SourceUpdateInput{
|
||||||
|
SourceType: PagesSourceTypeGitHubRelease,
|
||||||
|
RepositoryURL: "https://github.com/a/b",
|
||||||
|
})
|
||||||
|
useFakeGitHubReleaseClient(t, &fakeGitHubReleaseClient{
|
||||||
|
resolve: func(context.Context, githubrelease.ResolveRequest) (githubrelease.ResolveResult, error) {
|
||||||
|
return githubrelease.ResolveResult{}, errors.New("transient provider failure")
|
||||||
|
},
|
||||||
|
download: func(context.Context, githubrelease.DownloadRequest) (*githubrelease.DownloadResult, error) {
|
||||||
|
return nil, errors.New("unexpected")
|
||||||
|
},
|
||||||
|
})
|
||||||
|
snapshot, _, _ := acquireSourceLease(ctx, source.ID, source.ConfigVersion, sourceActionCheck)
|
||||||
|
if err := db.DB(ctx).Model(&model.PagesProjectSourceRuntime{}).Where("source_id = ?", source.ID).Updates(map[string]any{
|
||||||
|
"lease_token": "new-owner",
|
||||||
|
"lease_expires_at": time.Now().Add(time.Minute),
|
||||||
|
"sync_status": pagesSourceStatusSyncing,
|
||||||
|
"last_error": "new-owner-state",
|
||||||
|
}).Error; err != nil {
|
||||||
|
t.Fatalf("replace lease owner error = %v, want nil", err)
|
||||||
|
}
|
||||||
|
result, err := checkGitHubSource(ctx, snapshot)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("checkGitHubSource(lost lease) error = %v, want stale no-op", err)
|
||||||
|
}
|
||||||
|
if result == nil || !result.Stale {
|
||||||
|
t.Errorf("checkGitHubSource(lost lease) = %+v, want stale", result)
|
||||||
|
}
|
||||||
|
_, runtime := mustLoadPagesSource(t, ctx, project.ID)
|
||||||
|
if runtime.LeaseToken != "new-owner" || runtime.LastError != "new-owner-state" || runtime.SyncStatus != pagesSourceStatusSyncing {
|
||||||
|
t.Errorf("lost lease runtime = token:%q error:%q status:%q, want new owner state", runtime.LeaseToken, runtime.LastError, runtime.SyncStatus)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestGitHubCheckRateLimitUsesServerDeadlineAndSuppressesFastRetry(t *testing.T) {
|
||||||
|
ctx := setupPagesSourceTest(t)
|
||||||
|
project := mustCreatePagesSourceProject(t, ctx, "github-rate-limit")
|
||||||
|
source, _ := mustConfigureGitHubSourceWithoutDispatch(t, ctx, project.ID, SourceUpdateInput{
|
||||||
|
SourceType: PagesSourceTypeGitHubRelease,
|
||||||
|
RepositoryURL: "https://github.com/a/b",
|
||||||
|
})
|
||||||
|
retryAt := time.Now().Add(2 * time.Hour).UTC()
|
||||||
|
useFakeGitHubReleaseClient(t, &fakeGitHubReleaseClient{
|
||||||
|
resolve: func(context.Context, githubrelease.ResolveRequest) (githubrelease.ResolveResult, error) {
|
||||||
|
return githubrelease.ResolveResult{}, &githubrelease.Error{
|
||||||
|
Kind: githubrelease.ErrMetadata, StatusCode: 429, RetryAt: &retryAt,
|
||||||
|
}
|
||||||
|
},
|
||||||
|
download: func(context.Context, githubrelease.DownloadRequest) (*githubrelease.DownloadResult, error) {
|
||||||
|
return nil, errors.New("unexpected")
|
||||||
|
},
|
||||||
|
})
|
||||||
|
snapshot, _, _ := acquireSourceLease(ctx, source.ID, source.ConfigVersion, sourceActionCheck)
|
||||||
|
_, err := checkGitHubSource(ctx, snapshot)
|
||||||
|
if err == nil || err.Error() != errPagesSourceSyncFailed {
|
||||||
|
t.Fatalf("checkGitHubSource(rate limit) error = %v, want safe sync failure", err)
|
||||||
|
}
|
||||||
|
if !shouldSkipGitHubActionRetry(err) {
|
||||||
|
t.Error("shouldSkipGitHubActionRetry(rate limit) = false, want true")
|
||||||
|
}
|
||||||
|
_, runtime := mustLoadPagesSource(t, ctx, project.ID)
|
||||||
|
if runtime.NextCheckAt == nil || runtime.NextCheckAt.Before(retryAt) || runtime.SyncStatus != pagesSourceStatusFailed {
|
||||||
|
t.Errorf("rate limit runtime = next:%v status:%q, want deadline >= %v and failed", runtime.NextCheckAt, runtime.SyncStatus, retryAt)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestGitHubCheckInvalidResolvedTargetUsesServerDeadline(t *testing.T) {
|
||||||
|
ctx := setupPagesSourceTest(t)
|
||||||
|
project := mustCreatePagesSourceProject(t, ctx, "github-invalid-check-target")
|
||||||
|
source, _ := mustConfigureGitHubSourceWithoutDispatch(t, ctx, project.ID, SourceUpdateInput{
|
||||||
|
SourceType: PagesSourceTypeGitHubRelease,
|
||||||
|
RepositoryURL: "https://github.com/a/b",
|
||||||
|
})
|
||||||
|
retryAt := time.Now().Add(2 * time.Hour).UTC()
|
||||||
|
useFakeGitHubReleaseClient(t, &fakeGitHubReleaseClient{
|
||||||
|
resolve: func(context.Context, githubrelease.ResolveRequest) (githubrelease.ResolveResult, error) {
|
||||||
|
return githubrelease.ResolveResult{
|
||||||
|
Release: githubrelease.Release{ID: "1", Tag: "v1"},
|
||||||
|
Asset: githubrelease.Asset{
|
||||||
|
ID: "2", Name: "dist.zip", State: "uploaded",
|
||||||
|
UpdatedAt: time.Now().UTC(), Digest: "sha256:invalid",
|
||||||
|
},
|
||||||
|
RetryAt: &retryAt,
|
||||||
|
}, nil
|
||||||
|
},
|
||||||
|
download: func(context.Context, githubrelease.DownloadRequest) (*githubrelease.DownloadResult, error) {
|
||||||
|
t.Fatal("Download called after invalid check target")
|
||||||
|
return nil, nil
|
||||||
|
},
|
||||||
|
})
|
||||||
|
snapshot, _, _ := acquireSourceLease(ctx, source.ID, source.ConfigVersion, sourceActionCheck)
|
||||||
|
_, err := checkGitHubSource(ctx, snapshot)
|
||||||
|
if err == nil || err.Error() != errPagesSourceDigestInvalid {
|
||||||
|
t.Fatalf("checkGitHubSource(invalid target) error = %v, want %q", err, errPagesSourceDigestInvalid)
|
||||||
|
}
|
||||||
|
if !isPermanentSourceSyncError(err) {
|
||||||
|
t.Error("invalid check target classification = retryable, want permanent")
|
||||||
|
}
|
||||||
|
_, runtime := mustLoadPagesSource(t, ctx, project.ID)
|
||||||
|
if runtime.SyncStatus != pagesSourceStatusFailed || runtime.LastError != errPagesSourceDigestInvalid ||
|
||||||
|
runtime.NextCheckAt == nil || runtime.NextCheckAt.Before(retryAt) || runtime.LeaseToken != "" {
|
||||||
|
t.Errorf(
|
||||||
|
"invalid check target runtime = status:%q error:%q next:%v lease:%q, want failed/%q/deadline >= %v/cleared",
|
||||||
|
runtime.SyncStatus, runtime.LastError, runtime.NextCheckAt, runtime.LeaseToken,
|
||||||
|
errPagesSourceDigestInvalid, retryAt,
|
||||||
|
)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestGitHubSyncInvalidResolvedTargetUsesServerDeadline(t *testing.T) {
|
||||||
|
ctx := setupPagesSourceTest(t)
|
||||||
|
project := mustCreatePagesSourceProject(t, ctx, "github-invalid-sync-target")
|
||||||
|
source, _ := mustConfigureGitHubSourceWithoutDispatch(t, ctx, project.ID, SourceUpdateInput{
|
||||||
|
SourceType: PagesSourceTypeGitHubRelease,
|
||||||
|
RepositoryURL: "https://github.com/a/b",
|
||||||
|
})
|
||||||
|
retryAt := time.Now().Add(2 * time.Hour).UTC()
|
||||||
|
useFakeGitHubReleaseClient(t, &fakeGitHubReleaseClient{
|
||||||
|
resolve: func(context.Context, githubrelease.ResolveRequest) (githubrelease.ResolveResult, error) {
|
||||||
|
return githubrelease.ResolveResult{
|
||||||
|
Release: githubrelease.Release{ID: "1", Tag: "v1"},
|
||||||
|
Asset: githubrelease.Asset{
|
||||||
|
ID: "2", Name: "dist.zip", State: "uploaded",
|
||||||
|
UpdatedAt: time.Now().UTC(), Digest: "sha256:invalid",
|
||||||
|
},
|
||||||
|
RetryAt: &retryAt,
|
||||||
|
}, nil
|
||||||
|
},
|
||||||
|
download: func(context.Context, githubrelease.DownloadRequest) (*githubrelease.DownloadResult, error) {
|
||||||
|
t.Fatal("Download called after invalid sync target")
|
||||||
|
return nil, nil
|
||||||
|
},
|
||||||
|
})
|
||||||
|
snapshot, _, _ := acquireSourceLease(ctx, source.ID, source.ConfigVersion, sourceActionSync)
|
||||||
|
_, err := syncGitHubSource(ctx, snapshot, "user:7", "", "")
|
||||||
|
if err == nil || err.Error() != errPagesSourceDigestInvalid {
|
||||||
|
t.Fatalf("syncGitHubSource(invalid target) error = %v, want %q", err, errPagesSourceDigestInvalid)
|
||||||
|
}
|
||||||
|
if !isPermanentSourceSyncError(err) {
|
||||||
|
t.Error("invalid sync target classification = retryable, want permanent")
|
||||||
|
}
|
||||||
|
_, runtime := mustLoadPagesSource(t, ctx, project.ID)
|
||||||
|
if runtime.SyncStatus != pagesSourceStatusFailed || runtime.LastError != errPagesSourceDigestInvalid ||
|
||||||
|
runtime.NextCheckAt == nil || runtime.NextCheckAt.Before(retryAt) || runtime.LeaseToken != "" {
|
||||||
|
t.Errorf(
|
||||||
|
"invalid sync target runtime = status:%q error:%q next:%v lease:%q, want failed/%q/deadline >= %v/cleared",
|
||||||
|
runtime.SyncStatus, runtime.LastError, runtime.NextCheckAt, runtime.LeaseToken,
|
||||||
|
errPagesSourceDigestInvalid, retryAt,
|
||||||
|
)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestGitHubCheckHandlerSkipsProviderFastRetry(t *testing.T) {
|
||||||
|
tests := []struct {
|
||||||
|
name string
|
||||||
|
status int
|
||||||
|
retryDate bool
|
||||||
|
}{
|
||||||
|
{name: "bad request", status: http.StatusBadRequest},
|
||||||
|
{name: "rate limited forbidden", status: http.StatusForbidden, retryDate: true},
|
||||||
|
{name: "too many requests", status: http.StatusTooManyRequests, retryDate: true},
|
||||||
|
}
|
||||||
|
for _, test := range tests {
|
||||||
|
t.Run(test.name, func(t *testing.T) {
|
||||||
|
ctx := setupPagesSourceTest(t)
|
||||||
|
project := mustCreatePagesSourceProject(t, ctx, "github-handler-"+strings.ReplaceAll(test.name, " ", "-"))
|
||||||
|
source, _ := mustConfigureGitHubSourceWithoutDispatch(t, ctx, project.ID, SourceUpdateInput{
|
||||||
|
SourceType: PagesSourceTypeGitHubRelease,
|
||||||
|
RepositoryURL: "https://github.com/a/b",
|
||||||
|
})
|
||||||
|
var retryAt *time.Time
|
||||||
|
if test.retryDate {
|
||||||
|
deadline := time.Now().Add(2 * time.Hour).UTC()
|
||||||
|
retryAt = &deadline
|
||||||
|
}
|
||||||
|
useFakeGitHubReleaseClient(t, &fakeGitHubReleaseClient{
|
||||||
|
resolve: func(context.Context, githubrelease.ResolveRequest) (githubrelease.ResolveResult, error) {
|
||||||
|
return githubrelease.ResolveResult{}, &githubrelease.Error{
|
||||||
|
Kind: githubrelease.ErrMetadata, StatusCode: test.status, RetryAt: retryAt,
|
||||||
|
}
|
||||||
|
},
|
||||||
|
download: func(context.Context, githubrelease.DownloadRequest) (*githubrelease.DownloadResult, error) {
|
||||||
|
t.Fatal("Download called after provider check failure")
|
||||||
|
return nil, nil
|
||||||
|
},
|
||||||
|
})
|
||||||
|
raw, err := json.Marshal(SourceActionPayload{
|
||||||
|
SourceID: source.ID, ConfigVersion: source.ConfigVersion,
|
||||||
|
Action: sourceActionCheck, Actor: "user:7",
|
||||||
|
})
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("json.Marshal(check payload) error = %v, want nil", err)
|
||||||
|
}
|
||||||
|
result, err := (&SourceActionHandler{}).Execute(ctx, raw)
|
||||||
|
if result != nil || err == nil || !errors.Is(err, asynq.SkipRetry) {
|
||||||
|
t.Fatalf("SourceActionHandler.Execute(status %d) = result:%+v error:%v, want SkipRetry", test.status, result, err)
|
||||||
|
}
|
||||||
|
_, runtime := mustLoadPagesSource(t, ctx, project.ID)
|
||||||
|
if runtime.SyncStatus != pagesSourceStatusFailed || runtime.NextCheckAt == nil || runtime.LeaseToken != "" {
|
||||||
|
t.Errorf("provider failure runtime = status:%q next:%v lease:%q", runtime.SyncStatus, runtime.NextCheckAt, runtime.LeaseToken)
|
||||||
|
}
|
||||||
|
if retryAt != nil && (runtime.NextCheckAt == nil || runtime.NextCheckAt.Before(*retryAt)) {
|
||||||
|
t.Errorf("provider failure NextCheckAt = %v, want deadline >= %v", runtime.NextCheckAt, *retryAt)
|
||||||
|
}
|
||||||
|
})
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestGitHubSyncActivatesWithMetadataRevisionAndPackageChecksum(t *testing.T) {
|
||||||
|
ctx := setupPagesSourceSyncTest(t)
|
||||||
|
project := mustCreatePagesSourceProject(t, ctx, "github-sync")
|
||||||
|
source, _ := mustConfigureGitHubSourceWithoutDispatch(t, ctx, project.ID, SourceUpdateInput{
|
||||||
|
SourceType: PagesSourceTypeGitHubRelease,
|
||||||
|
RepositoryURL: "https://github.com/a/b",
|
||||||
|
AssetName: "dist.zip",
|
||||||
|
})
|
||||||
|
packageBytes := testPagesZip(t, map[string]string{"index.html": "github-v1"})
|
||||||
|
packageHash := sha256.Sum256(packageBytes)
|
||||||
|
updatedAt := time.Date(2026, 7, 19, 11, 0, 0, 0, time.UTC)
|
||||||
|
retryAt := time.Now().Add(2 * time.Hour).UTC()
|
||||||
|
release := githubrelease.Release{ID: "200", Tag: "release/v2"}
|
||||||
|
asset := githubrelease.Asset{ID: "10", Name: "dist.zip", State: "uploaded", UpdatedAt: updatedAt}
|
||||||
|
client := &fakeGitHubReleaseClient{
|
||||||
|
resolve: func(context.Context, githubrelease.ResolveRequest) (githubrelease.ResolveResult, error) {
|
||||||
|
return githubrelease.ResolveResult{Release: release, Asset: asset, RetryAt: &retryAt}, nil
|
||||||
|
},
|
||||||
|
download: func(_ context.Context, request githubrelease.DownloadRequest) (*githubrelease.DownloadResult, error) {
|
||||||
|
path := filepath.Join(t.TempDir(), "download")
|
||||||
|
if err := os.WriteFile(path, packageBytes, 0o600); err != nil {
|
||||||
|
t.Fatalf("os.WriteFile(download) error = %v, want nil", err)
|
||||||
|
}
|
||||||
|
return &githubrelease.DownloadResult{
|
||||||
|
Path: path, Size: int64(len(packageBytes)), SHA256: hex.EncodeToString(packageHash[:]),
|
||||||
|
}, nil
|
||||||
|
},
|
||||||
|
}
|
||||||
|
useFakeGitHubReleaseClient(t, client)
|
||||||
|
snapshot, _, _ := acquireSourceLease(ctx, source.ID, source.ConfigVersion, sourceActionSync)
|
||||||
|
outcome, err := syncGitHubSource(ctx, snapshot, "user:7", "", "")
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("syncGitHubSource() error = %v, want nil", err)
|
||||||
|
}
|
||||||
|
if outcome == nil || outcome.Deployment == nil || outcome.Stale {
|
||||||
|
t.Fatalf("syncGitHubSource() = %+v, want active deployment", outcome)
|
||||||
|
}
|
||||||
|
deployment, err := model.GetPagesDeploymentByID(ctx, outcome.Deployment.ID)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("GetPagesDeploymentByID(%d) error = %v, want nil", outcome.Deployment.ID, err)
|
||||||
|
}
|
||||||
|
if got, want := deployment.Checksum, hex.EncodeToString(packageHash[:]); got != want {
|
||||||
|
t.Errorf("deployment Checksum = %q, want package hash %q", got, want)
|
||||||
|
}
|
||||||
|
if deployment.SourceRevision == nil || *deployment.SourceRevision == deployment.Checksum {
|
||||||
|
t.Errorf("deployment SourceRevision = %v, want metadata revision distinct from package checksum", deployment.SourceRevision)
|
||||||
|
}
|
||||||
|
if got, want := deployment.SourceLabel, "release/v2"; got != want {
|
||||||
|
t.Errorf("deployment SourceLabel = %q, want %q", got, want)
|
||||||
|
}
|
||||||
|
if deployment.SourceType != PagesSourceTypeGitHubRelease || deployment.TriggerType != pagesSourceTriggerManualSync ||
|
||||||
|
deployment.CreatedBy != "user:7" {
|
||||||
|
t.Errorf("deployment provenance = type:%q trigger:%q actor:%q", deployment.SourceType, deployment.TriggerType, deployment.CreatedBy)
|
||||||
|
}
|
||||||
|
if strings.Contains(deployment.SourceMeta, "http") || strings.Contains(deployment.SourceMeta, "token") {
|
||||||
|
t.Errorf("deployment SourceMeta = %q, want no URL or token", deployment.SourceMeta)
|
||||||
|
}
|
||||||
|
if !strings.Contains(deployment.SourceMeta, `"tag":"release/v2"`) || strings.Contains(deployment.SourceMeta, `"label"`) {
|
||||||
|
t.Errorf("deployment SourceMeta = %q, want provider-specific tag field", deployment.SourceMeta)
|
||||||
|
}
|
||||||
|
_, runtime := mustLoadPagesSource(t, ctx, project.ID)
|
||||||
|
if runtime.NextCheckAt == nil || runtime.NextCheckAt.Before(retryAt) {
|
||||||
|
t.Errorf("sync runtime NextCheckAt = %v, want server deadline >= %v", runtime.NextCheckAt, retryAt)
|
||||||
|
}
|
||||||
|
|
||||||
|
secondSnapshot, _, _ := acquireSourceLease(ctx, source.ID, source.ConfigVersion, sourceActionSync)
|
||||||
|
second, err := syncGitHubSource(ctx, secondSnapshot, "user:7", "", "")
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("syncGitHubSource(idempotent) error = %v, want nil", err)
|
||||||
|
}
|
||||||
|
if second == nil || !second.Reused || second.Deployment == nil || second.Deployment.ID != outcome.Deployment.ID {
|
||||||
|
t.Errorf("syncGitHubSource(idempotent) = %+v, want reused deployment %d", second, outcome.Deployment.ID)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestGitHubSyncActivatesExactConfirmedReplacement(t *testing.T) {
|
||||||
|
ctx := setupPagesSourceSyncTest(t)
|
||||||
|
project := mustCreatePagesSourceProject(t, ctx, "github-confirm-replacement")
|
||||||
|
source, _ := mustConfigureGitHubSourceWithoutDispatch(t, ctx, project.ID, SourceUpdateInput{
|
||||||
|
SourceType: PagesSourceTypeGitHubRelease,
|
||||||
|
RepositoryURL: "https://github.com/a/b",
|
||||||
|
AssetName: "dist.zip",
|
||||||
|
})
|
||||||
|
release := githubrelease.Release{ID: "300", Tag: "v3"}
|
||||||
|
asset := githubrelease.Asset{
|
||||||
|
ID: "12", Name: "dist.zip", State: "uploaded",
|
||||||
|
UpdatedAt: time.Date(2026, 7, 19, 13, 0, 0, 0, time.UTC),
|
||||||
|
}
|
||||||
|
target, err := buildGitHubSourceTarget(release, asset, nil)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("buildGitHubSourceTarget() error = %v, want nil", err)
|
||||||
|
}
|
||||||
|
if err := db.DB(ctx).Model(&model.PagesProjectSourceRuntime{}).Where("source_id = ?", source.ID).Updates(map[string]any{
|
||||||
|
"last_seen_revision": target.Revision,
|
||||||
|
"last_seen_detail": target.DetailJSON,
|
||||||
|
"last_applied_revision": strings.Repeat("a", 64),
|
||||||
|
"last_applied_detail": `{"provider":"github","release_id":"300","asset_id":"11","tag":"v3","asset_name":"dist.zip"}`,
|
||||||
|
"sync_status": pagesSourceStatusAttention,
|
||||||
|
}).Error; err != nil {
|
||||||
|
t.Fatalf("seed replacement cursor error = %v, want nil", err)
|
||||||
|
}
|
||||||
|
packageBytes := testPagesZip(t, map[string]string{"index.html": "confirmed-v3"})
|
||||||
|
packageHash := sha256.Sum256(packageBytes)
|
||||||
|
useFakeGitHubReleaseClient(t, &fakeGitHubReleaseClient{
|
||||||
|
resolve: func(context.Context, githubrelease.ResolveRequest) (githubrelease.ResolveResult, error) {
|
||||||
|
return githubrelease.ResolveResult{Release: release, Asset: asset}, nil
|
||||||
|
},
|
||||||
|
download: func(context.Context, githubrelease.DownloadRequest) (*githubrelease.DownloadResult, error) {
|
||||||
|
path := filepath.Join(t.TempDir(), "confirmed.zip")
|
||||||
|
if err := os.WriteFile(path, packageBytes, 0o600); err != nil {
|
||||||
|
t.Fatalf("os.WriteFile(confirmed package) error = %v, want nil", err)
|
||||||
|
}
|
||||||
|
return &githubrelease.DownloadResult{
|
||||||
|
Path: path, Size: int64(len(packageBytes)), SHA256: hex.EncodeToString(packageHash[:]),
|
||||||
|
}, nil
|
||||||
|
},
|
||||||
|
})
|
||||||
|
snapshot, _, _ := acquireSourceLease(ctx, source.ID, source.ConfigVersion, sourceActionSync)
|
||||||
|
outcome, err := syncGitHubSource(ctx, snapshot, "user:9", "", target.Revision)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("syncGitHubSource(confirmed replacement) error = %v, want nil", err)
|
||||||
|
}
|
||||||
|
if outcome == nil || outcome.Deployment == nil || outcome.Stale {
|
||||||
|
t.Fatalf("syncGitHubSource(confirmed replacement) = %+v, want active deployment", outcome)
|
||||||
|
}
|
||||||
|
_, runtime := mustLoadPagesSource(t, ctx, project.ID)
|
||||||
|
if runtime.SyncStatus != pagesSourceStatusIdle || runtime.LastAppliedRevision != target.Revision {
|
||||||
|
t.Errorf("confirmed replacement runtime = status:%q applied:%q, want idle/%q", runtime.SyncStatus, runtime.LastAppliedRevision, target.Revision)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestSourceActionPayloadSeparatesSystemTargetAndUserConfirmation(t *testing.T) {
|
||||||
|
handler := &SourceActionHandler{}
|
||||||
|
revision := strings.Repeat("a", 64)
|
||||||
|
invalid := []SourceActionPayload{
|
||||||
|
{SourceID: 1, ConfigVersion: 1, Action: sourceActionSync, Actor: "user:1", TargetRevision: revision},
|
||||||
|
{SourceID: 1, ConfigVersion: 1, Action: sourceActionSync, Actor: pagesSourceCreatedBySystem, TriggerType: pagesSourceTriggerManualSync},
|
||||||
|
{SourceID: 1, ConfigVersion: 1, Action: sourceActionSync, Actor: pagesSourceCreatedBySystem, ConfirmedRevision: revision},
|
||||||
|
{SourceID: 1, ConfigVersion: 1, Action: sourceActionSync, Actor: pagesSourceCreatedBySystem, TargetRevision: revision, ConfirmedRevision: revision},
|
||||||
|
}
|
||||||
|
for _, payload := range invalid {
|
||||||
|
raw, _ := json.Marshal(payload)
|
||||||
|
if normalized, err := handler.ValidatePayload(raw); err == nil {
|
||||||
|
t.Errorf("ValidatePayload(%+v) = %s, nil; want error", payload, normalized)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
valid := []SourceActionPayload{
|
||||||
|
{SourceID: 1, ConfigVersion: 1, Action: sourceActionSync, Actor: pagesSourceCreatedBySystem, TriggerType: pagesSourceTriggerScheduledAutoUpdate, TargetRevision: revision},
|
||||||
|
{SourceID: 1, ConfigVersion: 1, Action: sourceActionSync, Actor: "user:1", TriggerType: pagesSourceTriggerManualSync, ConfirmedRevision: revision},
|
||||||
|
}
|
||||||
|
for _, payload := range valid {
|
||||||
|
raw, _ := json.Marshal(payload)
|
||||||
|
if _, err := handler.ValidatePayload(raw); err != nil {
|
||||||
|
t.Errorf("ValidatePayload(%+v) error = %v, want nil", payload, err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestGitHubProviderErrorsMapToSafeRetryClassification(t *testing.T) {
|
||||||
|
tests := []struct {
|
||||||
|
name string
|
||||||
|
provider error
|
||||||
|
want string
|
||||||
|
permanent bool
|
||||||
|
skipRetry bool
|
||||||
|
}{
|
||||||
|
{
|
||||||
|
name: "asset missing", provider: &githubrelease.Error{Kind: githubrelease.ErrAssetNotFound, StatusCode: 200},
|
||||||
|
want: errPagesSourceReleaseNotFound, permanent: true, skipRetry: true,
|
||||||
|
},
|
||||||
|
{
|
||||||
|
name: "digest mismatch", provider: &githubrelease.Error{Kind: githubrelease.ErrDigestMismatch, StatusCode: 200},
|
||||||
|
want: errPagesSourceDigestMismatch, permanent: true, skipRetry: true,
|
||||||
|
},
|
||||||
|
{
|
||||||
|
name: "rate limit", provider: &githubrelease.Error{
|
||||||
|
Kind: githubrelease.ErrMetadata, StatusCode: 429,
|
||||||
|
RetryAt: func() *time.Time { value := time.Now().Add(time.Hour); return &value }(),
|
||||||
|
},
|
||||||
|
want: errPagesSourceSyncFailed, permanent: false, skipRetry: true,
|
||||||
|
},
|
||||||
|
{
|
||||||
|
name: "network", provider: &githubrelease.Error{Kind: githubrelease.ErrDownload},
|
||||||
|
want: errPagesSourceSyncFailed, permanent: false, skipRetry: false,
|
||||||
|
},
|
||||||
|
{
|
||||||
|
name: "forbidden without retry", provider: &githubrelease.Error{Kind: githubrelease.ErrMetadata, StatusCode: 403},
|
||||||
|
want: errPagesSourceSyncFailed, permanent: true, skipRetry: true,
|
||||||
|
},
|
||||||
|
}
|
||||||
|
for _, test := range tests {
|
||||||
|
t.Run(test.name, func(t *testing.T) {
|
||||||
|
domainErr := githubSourceDomainError(test.provider)
|
||||||
|
if got := domainErr.Error(); got != test.want {
|
||||||
|
t.Errorf("githubSourceDomainError() = %q, want %q", got, test.want)
|
||||||
|
}
|
||||||
|
var typedDomainError *githubSourceProviderDomainError
|
||||||
|
if !errors.As(domainErr, &typedDomainError) {
|
||||||
|
t.Fatalf("githubSourceDomainError() type = %T, want *githubSourceProviderDomainError", domainErr)
|
||||||
|
}
|
||||||
|
if got := typedDomainError.permanent; got != test.permanent {
|
||||||
|
t.Errorf("githubSourceProviderDomainError.permanent = %t, want %t", got, test.permanent)
|
||||||
|
}
|
||||||
|
if got := shouldSkipGitHubActionRetry(domainErr); got != test.skipRetry {
|
||||||
|
t.Errorf("shouldSkipGitHubActionRetry() = %t, want %t", got, test.skipRetry)
|
||||||
|
}
|
||||||
|
if strings.Contains(domainErr.Error(), "status=") || strings.Contains(domainErr.Error(), "repo=") {
|
||||||
|
t.Errorf("githubSourceDomainError() = %q, want stable Pages message", domainErr)
|
||||||
|
}
|
||||||
|
})
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestGitHubSyncRejectsStaleConfirmationWithoutChangingActive(t *testing.T) {
|
||||||
|
ctx := setupPagesSourceSyncTest(t)
|
||||||
|
project := mustCreatePagesSourceProject(t, ctx, "github-confirm-stale")
|
||||||
|
oldActive := mustCreateActiveManualDeployment(t, ctx, project.ID, "old")
|
||||||
|
source, _ := mustConfigureGitHubSourceWithoutDispatch(t, ctx, project.ID, SourceUpdateInput{
|
||||||
|
SourceType: PagesSourceTypeGitHubRelease,
|
||||||
|
RepositoryURL: "https://github.com/a/b",
|
||||||
|
})
|
||||||
|
asset := githubrelease.Asset{ID: "2", Name: "dist.zip", State: "uploaded", UpdatedAt: time.Now().UTC()}
|
||||||
|
useFakeGitHubReleaseClient(t, &fakeGitHubReleaseClient{
|
||||||
|
resolve: func(context.Context, githubrelease.ResolveRequest) (githubrelease.ResolveResult, error) {
|
||||||
|
return githubrelease.ResolveResult{Release: githubrelease.Release{ID: "1", Tag: "v1"}, Asset: asset}, nil
|
||||||
|
},
|
||||||
|
download: func(context.Context, githubrelease.DownloadRequest) (*githubrelease.DownloadResult, error) {
|
||||||
|
t.Fatal("Download called for stale confirmation")
|
||||||
|
return nil, nil
|
||||||
|
},
|
||||||
|
})
|
||||||
|
snapshot, _, _ := acquireSourceLease(ctx, source.ID, source.ConfigVersion, sourceActionSync)
|
||||||
|
_, err := syncGitHubSource(ctx, snapshot, "user:1", "", strings.Repeat("f", 64))
|
||||||
|
if err == nil || err.Error() != errPagesSourceConfirmationStale {
|
||||||
|
t.Errorf("syncGitHubSource(stale confirmation) error = %v, want %q", err, errPagesSourceConfirmationStale)
|
||||||
|
}
|
||||||
|
storedProject, loadErr := model.GetPagesProjectByID(ctx, project.ID)
|
||||||
|
if loadErr != nil {
|
||||||
|
t.Fatalf("GetPagesProjectByID() error = %v, want nil", loadErr)
|
||||||
|
}
|
||||||
|
if storedProject.ActiveDeploymentID == nil || *storedProject.ActiveDeploymentID != oldActive.ID {
|
||||||
|
t.Errorf("ActiveDeploymentID = %v, want old active %d", storedProject.ActiveDeploymentID, oldActive.ID)
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -15,13 +15,17 @@ import (
|
|||||||
"path"
|
"path"
|
||||||
"path/filepath"
|
"path/filepath"
|
||||||
"regexp"
|
"regexp"
|
||||||
|
"strconv"
|
||||||
"strings"
|
"strings"
|
||||||
|
|
||||||
"github.com/Rain-kl/Wavelet/internal/apps/upload"
|
"github.com/Rain-kl/Wavelet/internal/apps/upload"
|
||||||
|
"github.com/Rain-kl/Wavelet/internal/db"
|
||||||
"github.com/Rain-kl/Wavelet/internal/model"
|
"github.com/Rain-kl/Wavelet/internal/model"
|
||||||
"github.com/Rain-kl/Wavelet/internal/repository"
|
"github.com/Rain-kl/Wavelet/internal/repository"
|
||||||
"github.com/Rain-kl/Wavelet/pkg/logger"
|
"github.com/Rain-kl/Wavelet/pkg/logger"
|
||||||
"github.com/Rain-kl/Wavelet/pkg/pagesarchive"
|
"github.com/Rain-kl/Wavelet/pkg/pagesarchive"
|
||||||
|
"gorm.io/gorm"
|
||||||
|
"gorm.io/gorm/clause"
|
||||||
)
|
)
|
||||||
|
|
||||||
const (
|
const (
|
||||||
@@ -31,11 +35,15 @@ const (
|
|||||||
defaultPagesMaxHistoryCount = 20
|
defaultPagesMaxHistoryCount = 20
|
||||||
defaultPagesEntryFile = "index.html"
|
defaultPagesEntryFile = "index.html"
|
||||||
defaultPagesFallbackPath = "/index.html"
|
defaultPagesFallbackPath = "/index.html"
|
||||||
pagesDeploymentUploadType = "openflare_pages_deployment"
|
pagesIngestMarkerKey = "pages_ingest_marker"
|
||||||
|
pagesIngestMarkerV2 = "pages_deployment_v2"
|
||||||
|
pagesProjectIDMetadataKey = "pages_project_id"
|
||||||
|
pagesSourceIDMetadataKey = "pages_source_id"
|
||||||
pagesMaxPathLength = 512
|
pagesMaxPathLength = 512
|
||||||
bytesPerMiB = 1024 * 1024
|
bytesPerMiB = 1024 * 1024
|
||||||
pagesExtractedSizeMultiplier = 4
|
pagesExtractedSizeMultiplier = 4
|
||||||
pagesMinExtractedSizeBytes = 100 * bytesPerMiB
|
pagesMinExtractedSizeBytes = 100 * bytesPerMiB
|
||||||
|
pagesRowLockStrength = "UPDATE"
|
||||||
)
|
)
|
||||||
|
|
||||||
var pagesSlugPattern = regexp.MustCompile(`^[a-z0-9][a-z0-9-]{0,126}[a-z0-9]$|^[a-z0-9]$`)
|
var pagesSlugPattern = regexp.MustCompile(`^[a-z0-9][a-z0-9-]{0,126}[a-z0-9]$|^[a-z0-9]$`)
|
||||||
@@ -115,30 +123,14 @@ func normalizePagesSlug(raw string) string {
|
|||||||
|
|
||||||
func validateAndNormalizePagesRootDir(raw string) (string, error) {
|
func validateAndNormalizePagesRootDir(raw string) (string, error) {
|
||||||
value := strings.TrimSpace(raw)
|
value := strings.TrimSpace(raw)
|
||||||
if value == "" {
|
|
||||||
return "", nil
|
|
||||||
}
|
|
||||||
if len(value) > pagesMaxPathLength {
|
if len(value) > pagesMaxPathLength {
|
||||||
return "", errors.New("pages 根目录长度不能超过 512")
|
return "", errors.New("pages 根目录长度不能超过 512")
|
||||||
}
|
}
|
||||||
if strings.Contains(value, "\\") || strings.ContainsAny(value, "\"';") {
|
normalized, err := pagesarchive.NormalizeLogicalPath(value, true)
|
||||||
return "", errors.New("pages 根目录包含不支持的字符")
|
if err != nil {
|
||||||
|
return "", fmt.Errorf("pages 根目录不合法: %w", err)
|
||||||
}
|
}
|
||||||
for _, r := range value {
|
return normalized, nil
|
||||||
if r <= 0x20 || r == 0x7f {
|
|
||||||
return "", errors.New("pages 根目录不能包含空白或控制字符")
|
|
||||||
}
|
|
||||||
}
|
|
||||||
cleaned := path.Clean(filepath.ToSlash(value))
|
|
||||||
if cleaned == "." || cleaned == "/" {
|
|
||||||
return "", nil
|
|
||||||
}
|
|
||||||
for _, segment := range strings.Split(cleaned, "/") {
|
|
||||||
if segment == "." || segment == ".." {
|
|
||||||
return "", errors.New("pages 根目录不能包含 . 或 .. 路径段")
|
|
||||||
}
|
|
||||||
}
|
|
||||||
return strings.TrimPrefix(cleaned, "/"), nil
|
|
||||||
}
|
}
|
||||||
|
|
||||||
func normalizePagesFallbackPath(raw string) (string, error) {
|
func normalizePagesFallbackPath(raw string) (string, error) {
|
||||||
@@ -186,12 +178,19 @@ func normalizeStoredPagesFallbackPath(value string) string {
|
|||||||
return normalized
|
return normalized
|
||||||
}
|
}
|
||||||
|
|
||||||
func normalizePagesEntryFile(raw string) string {
|
func validateAndNormalizePagesEntryFile(raw string) (string, error) {
|
||||||
value := path.Clean(strings.TrimSpace(filepath.ToSlash(raw)))
|
value := strings.TrimSpace(raw)
|
||||||
if value == "." || value == "/" {
|
if value == "" {
|
||||||
return defaultPagesEntryFile
|
value = defaultPagesEntryFile
|
||||||
}
|
}
|
||||||
return strings.TrimPrefix(value, "/")
|
if len(value) > pagesMaxPathLength {
|
||||||
|
return "", errors.New("pages 入口文件长度不能超过 512")
|
||||||
|
}
|
||||||
|
normalized, err := pagesarchive.NormalizeLogicalPath(value, false)
|
||||||
|
if err != nil {
|
||||||
|
return "", fmt.Errorf("pages 入口文件不合法: %w", err)
|
||||||
|
}
|
||||||
|
return normalized, nil
|
||||||
}
|
}
|
||||||
|
|
||||||
func persistPagesUploadTemp(fileHeader *multipart.FileHeader, maxPackageBytes int64) (string, string, int64, pagesarchive.Format, error) {
|
func persistPagesUploadTemp(fileHeader *multipart.FileHeader, maxPackageBytes int64) (string, string, int64, pagesarchive.Format, error) {
|
||||||
@@ -262,41 +261,56 @@ func ingestPagesDeploymentPackage(
|
|||||||
ctx context.Context,
|
ctx context.Context,
|
||||||
localPath string,
|
localPath string,
|
||||||
checksum string,
|
checksum string,
|
||||||
projectSlug string,
|
projectID uint,
|
||||||
|
fileName string,
|
||||||
|
format pagesarchive.Format,
|
||||||
|
) (upload.IngestResult, error) {
|
||||||
|
return ingestPagesDeploymentPackageWithSource(ctx, localPath, checksum, projectID, 0, fileName, format)
|
||||||
|
}
|
||||||
|
|
||||||
|
func ingestPagesDeploymentPackageWithSource(
|
||||||
|
ctx context.Context,
|
||||||
|
localPath string,
|
||||||
|
checksum string,
|
||||||
|
projectID uint,
|
||||||
|
sourceID uint,
|
||||||
fileName string,
|
fileName string,
|
||||||
format pagesarchive.Format,
|
format pagesarchive.Format,
|
||||||
) (upload.IngestResult, error) {
|
) (upload.IngestResult, error) {
|
||||||
systemUser := repository.GetSystemUser(ctx)
|
systemUser := repository.GetSystemUser(ctx)
|
||||||
accessMode := 0
|
accessMode := 0
|
||||||
extension := pagesarchive.NormalizeNameExtension(fileName, format)
|
extension := pagesarchive.NormalizeNameExtension(fileName, format)
|
||||||
|
extra := map[string]any{
|
||||||
|
pagesIngestMarkerKey: pagesIngestMarkerV2,
|
||||||
|
pagesProjectIDMetadataKey: strconv.FormatUint(uint64(projectID), 10),
|
||||||
|
}
|
||||||
|
if sourceID != 0 {
|
||||||
|
extra[pagesSourceIDMetadataKey] = strconv.FormatUint(uint64(sourceID), 10)
|
||||||
|
}
|
||||||
return upload.IngestFromLocalPath(ctx, localPath, upload.IngestRequest{
|
return upload.IngestFromLocalPath(ctx, localPath, upload.IngestRequest{
|
||||||
UserID: systemUser.ID,
|
UserID: systemUser.ID,
|
||||||
FileName: fileName,
|
FileName: fileName,
|
||||||
MimeType: pagesarchive.MIMEType(format),
|
MimeType: pagesarchive.MIMEType(format),
|
||||||
Extension: extension,
|
Extension: extension,
|
||||||
Hash: checksum,
|
Hash: checksum,
|
||||||
Type: pagesDeploymentUploadType,
|
Type: upload.ReservedPagesDeploymentType,
|
||||||
AccessMode: &accessMode,
|
AccessMode: &accessMode,
|
||||||
SkipExtensionCheck: true,
|
SkipExtensionCheck: true,
|
||||||
Policy: upload.PolicyDedupNewRecord,
|
Policy: upload.PolicyDedupNewRecord,
|
||||||
Metadata: model.UploadMetadata{
|
Metadata: model.UploadMetadata{
|
||||||
Extra: map[string]any{
|
Extra: extra,
|
||||||
"project_slug": projectSlug,
|
|
||||||
"format": string(format),
|
|
||||||
},
|
|
||||||
},
|
},
|
||||||
})
|
})
|
||||||
}
|
}
|
||||||
|
|
||||||
func removeDeploymentArtifact(ctx context.Context, deployment *model.PagesDeployment) {
|
func removeDeploymentArtifact(ctx context.Context, projectID uint, deployment *model.PagesDeployment) {
|
||||||
if deployment == nil {
|
if deployment == nil {
|
||||||
return
|
return
|
||||||
}
|
}
|
||||||
if deployment.UploadID == 0 {
|
if deployment.UploadID == 0 {
|
||||||
return
|
return
|
||||||
}
|
}
|
||||||
if _, err := upload.Remove(ctx, deployment.UploadID); err != nil {
|
if err := removePagesUploadIfUnreferenced(ctx, projectID, deployment.UploadID); err != nil {
|
||||||
// Soft-delete / storage cleanup failure must not undo DB prune; log for ops.
|
|
||||||
logger.WarnF(ctx,
|
logger.WarnF(ctx,
|
||||||
"[Pages] remove deployment artifact failed: deployment_id=%d upload_id=%d error=%v",
|
"[Pages] remove deployment artifact failed: deployment_id=%d upload_id=%d error=%v",
|
||||||
deployment.ID, deployment.UploadID, err,
|
deployment.ID, deployment.UploadID, err,
|
||||||
@@ -304,10 +318,58 @@ func removeDeploymentArtifact(ctx context.Context, deployment *model.PagesDeploy
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// removePagesUploadIfUnreferenced soft-deletes a reserved Pages upload only
|
||||||
|
// after locking its project (when present), locking the upload, and rechecking
|
||||||
|
// deployment references in the same transaction.
|
||||||
|
func removePagesUploadIfUnreferenced(ctx context.Context, projectID uint, uploadID uint64) error {
|
||||||
|
if uploadID == 0 {
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
err := db.DB(ctx).Transaction(func(tx *gorm.DB) error {
|
||||||
|
if projectID != 0 {
|
||||||
|
var project model.PagesProject
|
||||||
|
projectErr := tx.Clauses(clause.Locking{Strength: pagesRowLockStrength}).First(&project, projectID).Error
|
||||||
|
if projectErr != nil && !errors.Is(projectErr, gorm.ErrRecordNotFound) {
|
||||||
|
return projectErr
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
var uploadRecord model.Upload
|
||||||
|
if err := tx.Clauses(clause.Locking{Strength: pagesRowLockStrength}).
|
||||||
|
Where("id = ?", uploadID).
|
||||||
|
First(&uploadRecord).Error; err != nil {
|
||||||
|
if errors.Is(err, gorm.ErrRecordNotFound) {
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
if uploadRecord.Type != upload.ReservedPagesDeploymentType {
|
||||||
|
return fmt.Errorf("pages 部署包上传类型不匹配: %s", uploadRecord.Type)
|
||||||
|
}
|
||||||
|
|
||||||
|
var references int64
|
||||||
|
if err := tx.Model(&model.PagesDeployment{}).
|
||||||
|
Where("upload_id = ?", uploadID).
|
||||||
|
Count(&references).Error; err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
if references > 0 {
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
_, err := upload.RemoveLockedTx(tx, &uploadRecord)
|
||||||
|
return err
|
||||||
|
})
|
||||||
|
// Always invalidate after transaction completion, including idempotent no-op,
|
||||||
|
// so a prior post-commit cache interruption can heal on retry.
|
||||||
|
upload.InvalidateUploadMetaCache(ctx, uploadID)
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
|
||||||
func inspectPagesPackage(packagePath string, format pagesarchive.Format, rootDir string, entryFile string, limits pagesLimits) (*deploymentManifest, error) {
|
func inspectPagesPackage(packagePath string, format pagesarchive.Format, rootDir string, entryFile string, limits pagesLimits) (*deploymentManifest, error) {
|
||||||
archiveManifest, err := pagesarchive.InspectFile(packagePath, format, pagesarchive.InspectOptions{
|
archiveManifest, err := pagesarchive.InspectFile(packagePath, format, pagesarchive.InspectOptions{
|
||||||
RootDir: rootDir,
|
RootDir: rootDir,
|
||||||
EntryFile: entryFile,
|
EntryFile: entryFile,
|
||||||
|
VerifySizes: true,
|
||||||
Limits: pagesarchive.Limits{
|
Limits: pagesarchive.Limits{
|
||||||
MaxFiles: limits.MaxFiles,
|
MaxFiles: limits.MaxFiles,
|
||||||
MaxFileBytes: limits.ExtractedBytes,
|
MaxFileBytes: limits.ExtractedBytes,
|
||||||
|
|||||||
@@ -12,6 +12,7 @@ import (
|
|||||||
"mime/multipart"
|
"mime/multipart"
|
||||||
"net/url"
|
"net/url"
|
||||||
"os"
|
"os"
|
||||||
|
"path"
|
||||||
"strings"
|
"strings"
|
||||||
"time"
|
"time"
|
||||||
|
|
||||||
@@ -21,6 +22,7 @@ import (
|
|||||||
"github.com/Rain-kl/Wavelet/pkg/logger"
|
"github.com/Rain-kl/Wavelet/pkg/logger"
|
||||||
"github.com/Rain-kl/Wavelet/pkg/pagesarchive"
|
"github.com/Rain-kl/Wavelet/pkg/pagesarchive"
|
||||||
"gorm.io/gorm"
|
"gorm.io/gorm"
|
||||||
|
"gorm.io/gorm/clause"
|
||||||
)
|
)
|
||||||
|
|
||||||
// DeploymentPackage is a streamable Pages deployment artifact for agent download.
|
// DeploymentPackage is a streamable Pages deployment artifact for agent download.
|
||||||
@@ -58,6 +60,9 @@ type DeploymentView struct {
|
|||||||
FileCount int `json:"file_count"`
|
FileCount int `json:"file_count"`
|
||||||
TotalSize int64 `json:"total_size"`
|
TotalSize int64 `json:"total_size"`
|
||||||
CreatedBy string `json:"created_by"`
|
CreatedBy string `json:"created_by"`
|
||||||
|
SourceType string `json:"source_type"`
|
||||||
|
SourceLabel string `json:"source_label"`
|
||||||
|
TriggerType string `json:"trigger_type"`
|
||||||
CreatedAt time.Time `json:"created_at"`
|
CreatedAt time.Time `json:"created_at"`
|
||||||
ActivatedAt *time.Time `json:"activated_at"`
|
ActivatedAt *time.Time `json:"activated_at"`
|
||||||
}
|
}
|
||||||
@@ -137,28 +142,57 @@ func CreateProject(ctx context.Context, input Input) (*View, error) {
|
|||||||
|
|
||||||
// UpdateProject 更新 Pages 项目。
|
// UpdateProject 更新 Pages 项目。
|
||||||
func UpdateProject(ctx context.Context, id uint, input Input) (*View, error) {
|
func UpdateProject(ctx context.Context, id uint, input Input) (*View, error) {
|
||||||
project, err := model.GetPagesProjectByID(ctx, id)
|
var project *model.PagesProject
|
||||||
|
err := db.DB(ctx).Transaction(func(tx *gorm.DB) error {
|
||||||
|
var existing model.PagesProject
|
||||||
|
if err := tx.Clauses(clause.Locking{Strength: pagesRowLockStrength}).First(&existing, id).Error; err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
updated := existing
|
||||||
|
var err error
|
||||||
|
project, err = buildProject(&updated, input)
|
||||||
|
if err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
contentConfigChanged := existing.RootDir != project.RootDir || existing.EntryFile != project.EntryFile
|
||||||
|
if contentConfigChanged &&
|
||||||
|
existing.ActiveDeploymentID != nil && *existing.ActiveDeploymentID != 0 {
|
||||||
|
if err := ensureDeploymentEntry(tx, *existing.ActiveDeploymentID, project.RootDir, project.EntryFile); err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
}
|
||||||
|
updates := map[string]any{
|
||||||
|
"name": project.Name,
|
||||||
|
"slug": project.Slug,
|
||||||
|
"description": project.Description,
|
||||||
|
"enabled": project.Enabled,
|
||||||
|
"spa_fallback_enabled": project.SPAFallbackEnabled,
|
||||||
|
"spa_fallback_path": project.SPAFallbackPath,
|
||||||
|
"api_proxy_enabled": project.APIProxyEnabled,
|
||||||
|
"api_proxy_path": project.APIProxyPath,
|
||||||
|
"api_proxy_pass": project.APIProxyPass,
|
||||||
|
"api_proxy_rewrite": project.APIProxyRewrite,
|
||||||
|
"root_dir": project.RootDir,
|
||||||
|
"entry_file": project.EntryFile,
|
||||||
|
}
|
||||||
|
if contentConfigChanged {
|
||||||
|
updates["content_config_version"] = existing.ContentConfigVersion + 1
|
||||||
|
var source model.PagesProjectSource
|
||||||
|
sourceErr := tx.Clauses(clause.Locking{Strength: pagesRowLockStrength}).
|
||||||
|
Where("project_id = ?", existing.ID).
|
||||||
|
First(&source).Error
|
||||||
|
if sourceErr != nil && !errors.Is(sourceErr, gorm.ErrRecordNotFound) {
|
||||||
|
return sourceErr
|
||||||
|
}
|
||||||
|
if sourceErr == nil {
|
||||||
|
if err := fenceAndNormalizeRuntime(tx, source.ID); err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return tx.Model(&existing).Updates(updates).Error
|
||||||
|
})
|
||||||
if err != nil {
|
if err != nil {
|
||||||
return nil, err
|
|
||||||
}
|
|
||||||
project, err = buildProject(project, input)
|
|
||||||
if err != nil {
|
|
||||||
return nil, err
|
|
||||||
}
|
|
||||||
if err = db.DB(ctx).Model(project).Updates(map[string]any{
|
|
||||||
"name": project.Name,
|
|
||||||
"slug": project.Slug,
|
|
||||||
"description": project.Description,
|
|
||||||
"enabled": project.Enabled,
|
|
||||||
"spa_fallback_enabled": project.SPAFallbackEnabled,
|
|
||||||
"spa_fallback_path": project.SPAFallbackPath,
|
|
||||||
"api_proxy_enabled": project.APIProxyEnabled,
|
|
||||||
"api_proxy_path": project.APIProxyPath,
|
|
||||||
"api_proxy_pass": project.APIProxyPass,
|
|
||||||
"api_proxy_rewrite": project.APIProxyRewrite,
|
|
||||||
"root_dir": project.RootDir,
|
|
||||||
"entry_file": project.EntryFile,
|
|
||||||
}).Error; err != nil {
|
|
||||||
if isUniqueConstraintError(err) {
|
if isUniqueConstraintError(err) {
|
||||||
return nil, errors.New(errPagesSlugExists)
|
return nil, errors.New(errPagesSlugExists)
|
||||||
}
|
}
|
||||||
@@ -167,24 +201,71 @@ func UpdateProject(ctx context.Context, id uint, input Input) (*View, error) {
|
|||||||
return buildProjectView(ctx, project)
|
return buildProjectView(ctx, project)
|
||||||
}
|
}
|
||||||
|
|
||||||
|
func ensureDeploymentEntry(conn *gorm.DB, deploymentID uint, rootDir, entryFile string) error {
|
||||||
|
targetPath := entryFile
|
||||||
|
if rootDir != "" {
|
||||||
|
targetPath = path.Join(rootDir, entryFile)
|
||||||
|
}
|
||||||
|
var count int64
|
||||||
|
if err := conn.Model(&model.PagesDeploymentFile{}).
|
||||||
|
Where("deployment_id = ? AND path = ?", deploymentID, targetPath).
|
||||||
|
Count(&count).Error; err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
if count == 0 {
|
||||||
|
return fmt.Errorf("%s: %s", errPagesEntryFileMissing, targetPath)
|
||||||
|
}
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
|
||||||
// DeleteProject 删除 Pages 项目。
|
// DeleteProject 删除 Pages 项目。
|
||||||
func DeleteProject(ctx context.Context, id uint) error {
|
func DeleteProject(ctx context.Context, id uint) error {
|
||||||
project, err := model.GetPagesProjectByID(ctx, id)
|
project, err := model.GetPagesProjectByID(ctx, id)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
return err
|
return err
|
||||||
}
|
}
|
||||||
routeCount, err := model.CountProxyRoutesByPagesProjectID(ctx, project.ID)
|
var deployments []model.PagesDeployment
|
||||||
if err != nil {
|
err = db.DB(ctx).Transaction(func(tx *gorm.DB) error {
|
||||||
return err
|
var lockedProject model.PagesProject
|
||||||
}
|
if err := tx.Clauses(clause.Locking{Strength: pagesRowLockStrength}).First(&lockedProject, project.ID).Error; err != nil {
|
||||||
if routeCount > 0 {
|
return err
|
||||||
return errors.New(errPagesDeleteReferenced)
|
}
|
||||||
}
|
if tx.Migrator().HasTable(&model.ProxyRoute{}) {
|
||||||
deployments, err := model.ListPagesDeployments(ctx, project.ID)
|
var routeCount int64
|
||||||
if err != nil {
|
if err := tx.Model(&model.ProxyRoute{}).
|
||||||
return err
|
Where("pages_project_id = ?", project.ID).
|
||||||
}
|
Count(&routeCount).Error; err != nil {
|
||||||
return db.DB(ctx).Transaction(func(tx *gorm.DB) error {
|
return err
|
||||||
|
}
|
||||||
|
if routeCount > 0 {
|
||||||
|
return errors.New(errPagesDeleteReferenced)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
var source model.PagesProjectSource
|
||||||
|
sourceErr := tx.Clauses(clause.Locking{Strength: pagesRowLockStrength}).
|
||||||
|
Where("project_id = ?", project.ID).
|
||||||
|
First(&source).Error
|
||||||
|
if sourceErr != nil && !errors.Is(sourceErr, gorm.ErrRecordNotFound) {
|
||||||
|
return sourceErr
|
||||||
|
}
|
||||||
|
if sourceErr == nil {
|
||||||
|
var runtime model.PagesProjectSourceRuntime
|
||||||
|
runtimeErr := tx.Clauses(clause.Locking{Strength: pagesRowLockStrength}).
|
||||||
|
Where("source_id = ?", source.ID).
|
||||||
|
First(&runtime).Error
|
||||||
|
if runtimeErr != nil && !errors.Is(runtimeErr, gorm.ErrRecordNotFound) {
|
||||||
|
return runtimeErr
|
||||||
|
}
|
||||||
|
if err := tx.Where("source_id = ?", source.ID).Delete(&model.PagesProjectSourceRuntime{}).Error; err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
if err := tx.Delete(&source).Error; err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if err := tx.Where("project_id = ?", project.ID).Find(&deployments).Error; err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
if err := tx.Where(
|
if err := tx.Where(
|
||||||
"deployment_id IN (?)",
|
"deployment_id IN (?)",
|
||||||
tx.Model(&model.PagesDeployment{}).Select("id").Where("project_id = ?", project.ID),
|
tx.Model(&model.PagesDeployment{}).Select("id").Where("project_id = ?", project.ID),
|
||||||
@@ -194,14 +275,18 @@ func DeleteProject(ctx context.Context, id uint) error {
|
|||||||
if err := tx.Where("project_id = ?", project.ID).Delete(&model.PagesDeployment{}).Error; err != nil {
|
if err := tx.Where("project_id = ?", project.ID).Delete(&model.PagesDeployment{}).Error; err != nil {
|
||||||
return err
|
return err
|
||||||
}
|
}
|
||||||
if err := tx.Delete(project).Error; err != nil {
|
if err := tx.Delete(&lockedProject).Error; err != nil {
|
||||||
return err
|
return err
|
||||||
}
|
}
|
||||||
for index := range deployments {
|
|
||||||
removeDeploymentArtifact(ctx, &deployments[index])
|
|
||||||
}
|
|
||||||
return nil
|
return nil
|
||||||
})
|
})
|
||||||
|
if err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
for index := range deployments {
|
||||||
|
removeDeploymentArtifact(ctx, project.ID, &deployments[index])
|
||||||
|
}
|
||||||
|
return nil
|
||||||
}
|
}
|
||||||
|
|
||||||
// ListProjectDeployments 列出项目的全部部署。
|
// ListProjectDeployments 列出项目的全部部署。
|
||||||
@@ -258,7 +343,18 @@ func UploadDeployment(ctx context.Context, projectID uint, fileHeader *multipart
|
|||||||
return nil, err
|
return nil, err
|
||||||
}
|
}
|
||||||
defer func() { _ = os.Remove(tempPath) }()
|
defer func() { _ = os.Remove(tempPath) }()
|
||||||
return createDeploymentFromTempPackage(ctx, project, tempPath, checksum, format, fileHeader.Filename, createdBy, limits)
|
return createDeploymentFromTempPackage(
|
||||||
|
ctx,
|
||||||
|
project,
|
||||||
|
tempPath,
|
||||||
|
checksum,
|
||||||
|
format,
|
||||||
|
fileHeader.Filename,
|
||||||
|
createdBy,
|
||||||
|
"manual_upload",
|
||||||
|
"manual_upload",
|
||||||
|
limits,
|
||||||
|
)
|
||||||
}
|
}
|
||||||
|
|
||||||
// UploadFromURLInput is the request body for downloading a deployment package from a remote URL.
|
// UploadFromURLInput is the request body for downloading a deployment package from a remote URL.
|
||||||
@@ -278,7 +374,18 @@ func UploadDeploymentFromURL(ctx context.Context, projectID uint, rawURL string,
|
|||||||
return nil, err
|
return nil, err
|
||||||
}
|
}
|
||||||
defer func() { _ = os.Remove(tempPath) }()
|
defer func() { _ = os.Remove(tempPath) }()
|
||||||
return createDeploymentFromTempPackage(ctx, project, tempPath, checksum, format, fileName, createdBy, limits)
|
return createDeploymentFromTempPackage(
|
||||||
|
ctx,
|
||||||
|
project,
|
||||||
|
tempPath,
|
||||||
|
checksum,
|
||||||
|
format,
|
||||||
|
fileName,
|
||||||
|
createdBy,
|
||||||
|
"manual_url",
|
||||||
|
"manual_url",
|
||||||
|
limits,
|
||||||
|
)
|
||||||
}
|
}
|
||||||
|
|
||||||
func createDeploymentFromTempPackage(
|
func createDeploymentFromTempPackage(
|
||||||
@@ -289,6 +396,8 @@ func createDeploymentFromTempPackage(
|
|||||||
format pagesarchive.Format,
|
format pagesarchive.Format,
|
||||||
fileName string,
|
fileName string,
|
||||||
createdBy string,
|
createdBy string,
|
||||||
|
sourceType string,
|
||||||
|
triggerType string,
|
||||||
limits pagesLimits,
|
limits pagesLimits,
|
||||||
) (*DeploymentView, error) {
|
) (*DeploymentView, error) {
|
||||||
if project == nil {
|
if project == nil {
|
||||||
@@ -298,7 +407,10 @@ func createDeploymentFromTempPackage(
|
|||||||
if err != nil {
|
if err != nil {
|
||||||
return nil, err
|
return nil, err
|
||||||
}
|
}
|
||||||
entryFile := normalizePagesEntryFile(project.EntryFile)
|
entryFile, err := validateAndNormalizePagesEntryFile(project.EntryFile)
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
manifest, err := inspectPagesPackage(tempPath, format, rootDir, entryFile, limits)
|
manifest, err := inspectPagesPackage(tempPath, format, rootDir, entryFile, limits)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
return nil, err
|
return nil, err
|
||||||
@@ -307,7 +419,7 @@ func createDeploymentFromTempPackage(
|
|||||||
ctx,
|
ctx,
|
||||||
tempPath,
|
tempPath,
|
||||||
checksum,
|
checksum,
|
||||||
project.Slug,
|
project.ID,
|
||||||
fileName,
|
fileName,
|
||||||
format,
|
format,
|
||||||
)
|
)
|
||||||
@@ -317,11 +429,29 @@ func createDeploymentFromTempPackage(
|
|||||||
ingestCommitted := false
|
ingestCommitted := false
|
||||||
defer func() {
|
defer func() {
|
||||||
if !ingestCommitted && ingestResult.Created {
|
if !ingestCommitted && ingestResult.Created {
|
||||||
_, _ = upload.Remove(ctx, ingestResult.Upload.ID)
|
if removeErr := removePagesUploadIfUnreferenced(ctx, project.ID, ingestResult.Upload.ID); removeErr != nil {
|
||||||
|
logger.ErrorF(ctx,
|
||||||
|
"[Pages] compensate deployment upload failed: project_id=%d upload_id=%d error=%v",
|
||||||
|
project.ID, ingestResult.Upload.ID, removeErr,
|
||||||
|
)
|
||||||
|
}
|
||||||
}
|
}
|
||||||
}()
|
}()
|
||||||
deployment := &model.PagesDeployment{}
|
deployment := &model.PagesDeployment{}
|
||||||
err = db.DB(ctx).Transaction(func(tx *gorm.DB) error {
|
err = db.DB(ctx).Transaction(func(tx *gorm.DB) error {
|
||||||
|
var lockedProject model.PagesProject
|
||||||
|
if err := tx.Clauses(clause.Locking{Strength: pagesRowLockStrength}).First(&lockedProject, project.ID).Error; err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
var uploadRecord model.Upload
|
||||||
|
if err := tx.Clauses(clause.Locking{Strength: pagesRowLockStrength}).
|
||||||
|
Where("id = ?", ingestResult.Upload.ID).
|
||||||
|
First(&uploadRecord).Error; err != nil {
|
||||||
|
return errors.New(errPagesPackageUploadMissing)
|
||||||
|
}
|
||||||
|
if uploadRecord.Status != model.UploadStatusUsed || uploadRecord.Type != upload.ReservedPagesDeploymentType {
|
||||||
|
return errors.New(errPagesPackageUploadMissing)
|
||||||
|
}
|
||||||
var maxNumber int
|
var maxNumber int
|
||||||
if err := tx.Model(&model.PagesDeployment{}).
|
if err := tx.Model(&model.PagesDeployment{}).
|
||||||
Where("project_id = ?", project.ID).
|
Where("project_id = ?", project.ID).
|
||||||
@@ -338,6 +468,9 @@ func createDeploymentFromTempPackage(
|
|||||||
FileCount: manifest.FileCount,
|
FileCount: manifest.FileCount,
|
||||||
TotalSize: manifest.TotalSize,
|
TotalSize: manifest.TotalSize,
|
||||||
CreatedBy: strings.TrimSpace(createdBy),
|
CreatedBy: strings.TrimSpace(createdBy),
|
||||||
|
SourceType: sourceType,
|
||||||
|
SourceLabel: safeRemoteSourceLabel(fileName),
|
||||||
|
TriggerType: triggerType,
|
||||||
}
|
}
|
||||||
if err := tx.Create(deployment).Error; err != nil {
|
if err := tx.Create(deployment).Error; err != nil {
|
||||||
return err
|
return err
|
||||||
@@ -357,7 +490,7 @@ func createDeploymentFromTempPackage(
|
|||||||
}
|
}
|
||||||
ingestCommitted = true
|
ingestCommitted = true
|
||||||
|
|
||||||
if pruneErr := pruneProjectDeploymentHistory(ctx, project.ID, limits.HistoryCount); pruneErr != nil {
|
if pruneErr := pruneProjectDeploymentHistory(ctx, project.ID, limits.HistoryCount, deployment.ID); pruneErr != nil {
|
||||||
logger.ErrorF(ctx,
|
logger.ErrorF(ctx,
|
||||||
"[Pages] prune deployment history failed: project_id=%d keep=%d error=%v",
|
"[Pages] prune deployment history failed: project_id=%d keep=%d error=%v",
|
||||||
project.ID, limits.HistoryCount, pruneErr,
|
project.ID, limits.HistoryCount, pruneErr,
|
||||||
@@ -381,7 +514,7 @@ func createDeploymentFromTempPackage(
|
|||||||
// Concurrency: DB row deletes run in a single transaction after a consistent read
|
// Concurrency: DB row deletes run in a single transaction after a consistent read
|
||||||
// of project + deployments. Concurrent uploads may briefly exceed keepCount; the
|
// of project + deployments. Concurrent uploads may briefly exceed keepCount; the
|
||||||
// next successful prune brings the project back within the limit (eventual).
|
// next successful prune brings the project back within the limit (eventual).
|
||||||
func pruneProjectDeploymentHistory(ctx context.Context, projectID uint, keepCount int) error {
|
func pruneProjectDeploymentHistory(ctx context.Context, projectID uint, keepCount int, preserveCandidateID uint) error {
|
||||||
if keepCount <= 0 {
|
if keepCount <= 0 {
|
||||||
return nil
|
return nil
|
||||||
}
|
}
|
||||||
@@ -390,7 +523,7 @@ func pruneProjectDeploymentHistory(ctx context.Context, projectID uint, keepCoun
|
|||||||
// that inserted another deployment between our list and delete.
|
// that inserted another deployment between our list and delete.
|
||||||
var lastErr error
|
var lastErr error
|
||||||
for pass := 0; pass < 2; pass++ {
|
for pass := 0; pass < 2; pass++ {
|
||||||
deleted, err := pruneProjectDeploymentHistoryOnce(ctx, projectID, keepCount)
|
deleted, err := pruneProjectDeploymentHistoryOnce(ctx, projectID, keepCount, preserveCandidateID)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
lastErr = err
|
lastErr = err
|
||||||
break
|
break
|
||||||
@@ -404,73 +537,79 @@ func pruneProjectDeploymentHistory(ctx context.Context, projectID uint, keepCoun
|
|||||||
|
|
||||||
// pruneProjectDeploymentHistoryOnce performs one list → select → delete cycle.
|
// pruneProjectDeploymentHistoryOnce performs one list → select → delete cycle.
|
||||||
// Returns the number of deployments deleted from the database.
|
// Returns the number of deployments deleted from the database.
|
||||||
func pruneProjectDeploymentHistoryOnce(ctx context.Context, projectID uint, keepCount int) (int, error) {
|
func pruneProjectDeploymentHistoryOnce(ctx context.Context, projectID uint, keepCount int, preserveCandidateID uint) (int, error) {
|
||||||
project, err := model.GetPagesProjectByID(ctx, projectID)
|
var deletedDeployments []model.PagesDeployment
|
||||||
if err != nil {
|
err := db.DB(ctx).Transaction(func(tx *gorm.DB) error {
|
||||||
return 0, fmt.Errorf("load pages project: %w", err)
|
var project model.PagesProject
|
||||||
}
|
if err := tx.Clauses(clause.Locking{Strength: pagesRowLockStrength}).First(&project, projectID).Error; err != nil {
|
||||||
deployments, err := model.ListPagesDeployments(ctx, projectID)
|
return fmt.Errorf("load pages project: %w", err)
|
||||||
if err != nil {
|
}
|
||||||
return 0, fmt.Errorf("list pages deployments: %w", err)
|
var deployments []model.PagesDeployment
|
||||||
}
|
if err := tx.Where("project_id = ?", projectID).Order("id desc").Find(&deployments).Error; err != nil {
|
||||||
if len(deployments) <= keepCount {
|
return fmt.Errorf("list pages deployments: %w", err)
|
||||||
return 0, nil
|
}
|
||||||
}
|
var activeID uint
|
||||||
|
if project.ActiveDeploymentID != nil {
|
||||||
var activeID uint
|
activeID = *project.ActiveDeploymentID
|
||||||
if project.ActiveDeploymentID != nil {
|
}
|
||||||
activeID = *project.ActiveDeploymentID
|
// Preserve mode is signaled by a non-zero candidate ID, but the lock-time
|
||||||
}
|
// newest non-active deployment wins. This prevents concurrent upload A/B
|
||||||
toDelete := selectDeploymentsToPrune(deployments, activeID, keepCount)
|
// prune passes from deleting each other's newer candidate.
|
||||||
if len(toDelete) == 0 {
|
resolvedCandidateID := resolveLatestCandidateID(deployments, activeID, preserveCandidateID != 0)
|
||||||
return 0, nil
|
toDelete := selectDeploymentsToPrune(deployments, activeID, resolvedCandidateID, keepCount)
|
||||||
}
|
|
||||||
|
|
||||||
// Delete metadata in one transaction so partial prune does not leave
|
|
||||||
// orphan file-list rows without a parent deployment.
|
|
||||||
if err := db.DB(ctx).Transaction(func(tx *gorm.DB) error {
|
|
||||||
for index := range toDelete {
|
for index := range toDelete {
|
||||||
deployment := toDelete[index]
|
deployment := toDelete[index]
|
||||||
// Never delete the active deployment even if project pointer raced.
|
|
||||||
if activeID != 0 && deployment.ID == activeID {
|
|
||||||
continue
|
|
||||||
}
|
|
||||||
if project.ActiveDeploymentID != nil && deployment.ID == *project.ActiveDeploymentID {
|
|
||||||
continue
|
|
||||||
}
|
|
||||||
if err := tx.Where("deployment_id = ?", deployment.ID).Delete(&model.PagesDeploymentFile{}).Error; err != nil {
|
if err := tx.Where("deployment_id = ?", deployment.ID).Delete(&model.PagesDeploymentFile{}).Error; err != nil {
|
||||||
return fmt.Errorf("delete deployment files id=%d: %w", deployment.ID, err)
|
return fmt.Errorf("delete deployment files id=%d: %w", deployment.ID, err)
|
||||||
}
|
}
|
||||||
if err := tx.Where("id = ? AND project_id = ?", deployment.ID, projectID).
|
result := tx.Where("id = ? AND project_id = ?", deployment.ID, projectID).
|
||||||
Delete(&model.PagesDeployment{}).Error; err != nil {
|
Delete(&model.PagesDeployment{})
|
||||||
return fmt.Errorf("delete deployment id=%d: %w", deployment.ID, err)
|
if result.Error != nil {
|
||||||
|
return fmt.Errorf("delete deployment id=%d: %w", deployment.ID, result.Error)
|
||||||
|
}
|
||||||
|
if result.RowsAffected == 1 {
|
||||||
|
deletedDeployments = append(deletedDeployments, deployment)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
return nil
|
return nil
|
||||||
}); err != nil {
|
})
|
||||||
|
if err != nil {
|
||||||
return 0, err
|
return 0, err
|
||||||
}
|
}
|
||||||
|
|
||||||
// Artifacts are best-effort outside the transaction (object storage I/O).
|
for index := range deletedDeployments {
|
||||||
for index := range toDelete {
|
deployment := deletedDeployments[index]
|
||||||
deployment := toDelete[index]
|
removeDeploymentArtifact(ctx, projectID, &deployment)
|
||||||
if activeID != 0 && deployment.ID == activeID {
|
|
||||||
continue
|
|
||||||
}
|
|
||||||
removeDeploymentArtifact(ctx, &deployment)
|
|
||||||
}
|
}
|
||||||
|
|
||||||
logger.InfoF(ctx,
|
logger.InfoF(ctx,
|
||||||
"[Pages] pruned deployment history: project_id=%d keep=%d deleted=%d",
|
"[Pages] pruned deployment history: project_id=%d keep=%d deleted=%d",
|
||||||
projectID, keepCount, len(toDelete),
|
projectID, keepCount, len(deletedDeployments),
|
||||||
)
|
)
|
||||||
return len(toDelete), nil
|
return len(deletedDeployments), nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func resolveLatestCandidateID(deployments []model.PagesDeployment, activeID uint, preserve bool) uint {
|
||||||
|
if !preserve {
|
||||||
|
return 0
|
||||||
|
}
|
||||||
|
for _, deployment := range deployments {
|
||||||
|
if deployment.ID != activeID {
|
||||||
|
return deployment.ID
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return 0
|
||||||
}
|
}
|
||||||
|
|
||||||
// selectDeploymentsToPrune returns deployments that should be removed under the
|
// selectDeploymentsToPrune returns deployments that should be removed under the
|
||||||
// "at most keepCount, always keep active, fill with newest" policy.
|
// "at most keepCount, always keep active, fill with newest" policy.
|
||||||
// deployments must be ordered newest-first (id desc).
|
// deployments must be ordered newest-first (id desc).
|
||||||
func selectDeploymentsToPrune(deployments []model.PagesDeployment, activeID uint, keepCount int) []model.PagesDeployment {
|
func selectDeploymentsToPrune(
|
||||||
|
deployments []model.PagesDeployment,
|
||||||
|
activeID uint,
|
||||||
|
preserveCandidateID uint,
|
||||||
|
keepCount int,
|
||||||
|
) []model.PagesDeployment {
|
||||||
if keepCount <= 0 || len(deployments) <= keepCount {
|
if keepCount <= 0 || len(deployments) <= keepCount {
|
||||||
return nil
|
return nil
|
||||||
}
|
}
|
||||||
@@ -486,6 +625,17 @@ func selectDeploymentsToPrune(deployments []model.PagesDeployment, activeID uint
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
// A freshly uploaded manual candidate is temporarily protected in addition
|
||||||
|
// to the active deployment. This intentionally permits two rows when the
|
||||||
|
// configured history limit is one.
|
||||||
|
if preserveCandidateID != 0 {
|
||||||
|
for _, deployment := range deployments {
|
||||||
|
if deployment.ID == preserveCandidateID {
|
||||||
|
keepIDs[preserveCandidateID] = struct{}{}
|
||||||
|
break
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
// 2) Fill remaining slots from newest to oldest.
|
// 2) Fill remaining slots from newest to oldest.
|
||||||
for _, deployment := range deployments {
|
for _, deployment := range deployments {
|
||||||
if len(keepIDs) >= keepCount {
|
if len(keepIDs) >= keepCount {
|
||||||
@@ -510,37 +660,208 @@ func selectDeploymentsToPrune(deployments []model.PagesDeployment, activeID uint
|
|||||||
|
|
||||||
// ActivateDeployment 激活 Pages 部署。
|
// ActivateDeployment 激活 Pages 部署。
|
||||||
func ActivateDeployment(ctx context.Context, projectID uint, deploymentID uint) (*View, error) {
|
func ActivateDeployment(ctx context.Context, projectID uint, deploymentID uint) (*View, error) {
|
||||||
project, err := model.GetPagesProjectByID(ctx, projectID)
|
return ActivateDeploymentAs(ctx, projectID, deploymentID, "system:pages-manual-activation")
|
||||||
|
}
|
||||||
|
|
||||||
|
// ActivateDeploymentAs activates a historical deployment and fences any
|
||||||
|
// configured source when the active deployment actually changes.
|
||||||
|
func ActivateDeploymentAs(ctx context.Context, projectID uint, deploymentID uint, actor string) (*View, error) {
|
||||||
|
if err := ensureActivationDeploymentUpload(ctx, projectID, deploymentID); err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
audit, err := activateDeploymentTransaction(ctx, projectID, deploymentID, time.Now())
|
||||||
if err != nil {
|
if err != nil {
|
||||||
return nil, err
|
return nil, err
|
||||||
}
|
}
|
||||||
|
if audit.Noop {
|
||||||
|
return GetProject(ctx, projectID)
|
||||||
|
}
|
||||||
|
logger.InfoF(ctx,
|
||||||
|
"[Pages] manual activation: actor=%s project_id=%d old_deployment_id=%d new_deployment_id=%d source_type=%s source_identity=%s auto_disabled=%t",
|
||||||
|
strings.TrimSpace(actor), projectID, audit.OldDeploymentID, deploymentID,
|
||||||
|
audit.SourceType, audit.SourceIdentity, audit.AutoDisabled,
|
||||||
|
)
|
||||||
|
limits := resolvePagesLimits(ctx)
|
||||||
|
if pruneErr := pruneProjectDeploymentHistory(ctx, projectID, limits.HistoryCount, 0); pruneErr != nil {
|
||||||
|
logger.ErrorF(ctx,
|
||||||
|
"[Pages] strict prune after activation failed: project_id=%d keep=%d error=%v",
|
||||||
|
projectID, limits.HistoryCount, pruneErr,
|
||||||
|
)
|
||||||
|
}
|
||||||
|
return GetProject(ctx, projectID)
|
||||||
|
}
|
||||||
|
|
||||||
|
type deploymentActivationAudit struct {
|
||||||
|
OldDeploymentID uint
|
||||||
|
SourceType string
|
||||||
|
SourceIdentity string
|
||||||
|
AutoDisabled bool
|
||||||
|
Noop bool
|
||||||
|
}
|
||||||
|
|
||||||
|
type deploymentActivationSource struct {
|
||||||
|
Source *model.PagesProjectSource
|
||||||
|
Runtime *model.PagesProjectSourceRuntime
|
||||||
|
}
|
||||||
|
|
||||||
|
func ensureActivationDeploymentUpload(ctx context.Context, projectID uint, deploymentID uint) error {
|
||||||
deployment, err := model.GetPagesDeploymentByID(ctx, deploymentID)
|
deployment, err := model.GetPagesDeploymentByID(ctx, deploymentID)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
if deployment.ProjectID != projectID {
|
||||||
|
return errors.New(errPagesDeploymentMismatch)
|
||||||
|
}
|
||||||
|
if deployment.UploadID != 0 {
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
return ensureDeploymentUploadRecord(ctx, deployment)
|
||||||
|
}
|
||||||
|
|
||||||
|
func activateDeploymentTransaction(
|
||||||
|
ctx context.Context,
|
||||||
|
projectID uint,
|
||||||
|
deploymentID uint,
|
||||||
|
now time.Time,
|
||||||
|
) (deploymentActivationAudit, error) {
|
||||||
|
audit := deploymentActivationAudit{}
|
||||||
|
err := db.DB(ctx).Transaction(func(tx *gorm.DB) error {
|
||||||
|
var project model.PagesProject
|
||||||
|
if err := tx.Clauses(clause.Locking{Strength: pagesRowLockStrength}).First(&project, projectID).Error; err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
if project.ActiveDeploymentID != nil {
|
||||||
|
audit.OldDeploymentID = *project.ActiveDeploymentID
|
||||||
|
}
|
||||||
|
if audit.OldDeploymentID == deploymentID {
|
||||||
|
audit.Noop = true
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
|
||||||
|
sourceState, err := lockDeploymentActivationSource(tx, project.ID)
|
||||||
|
if err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
deployment, err := loadDeploymentActivationTarget(tx, &project, deploymentID)
|
||||||
|
if err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
if err := fenceDeploymentActivationSource(tx, sourceState, deployment, &audit); err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
return switchActiveDeploymentTx(tx, &project, deployment, now)
|
||||||
|
})
|
||||||
|
return audit, err
|
||||||
|
}
|
||||||
|
|
||||||
|
func lockDeploymentActivationSource(tx *gorm.DB, projectID uint) (*deploymentActivationSource, error) {
|
||||||
|
var source model.PagesProjectSource
|
||||||
|
err := tx.Clauses(clause.Locking{Strength: pagesRowLockStrength}).
|
||||||
|
Where("project_id = ?", projectID).
|
||||||
|
First(&source).Error
|
||||||
|
if errors.Is(err, gorm.ErrRecordNotFound) {
|
||||||
|
return nil, nil
|
||||||
|
}
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
var runtime model.PagesProjectSourceRuntime
|
||||||
|
if err := tx.Clauses(clause.Locking{Strength: pagesRowLockStrength}).
|
||||||
|
Where("source_id = ?", source.ID).
|
||||||
|
First(&runtime).Error; err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
return &deploymentActivationSource{Source: &source, Runtime: &runtime}, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func loadDeploymentActivationTarget(
|
||||||
|
tx *gorm.DB,
|
||||||
|
project *model.PagesProject,
|
||||||
|
deploymentID uint,
|
||||||
|
) (*model.PagesDeployment, error) {
|
||||||
|
var deployment model.PagesDeployment
|
||||||
|
if err := tx.First(&deployment, deploymentID).Error; err != nil {
|
||||||
return nil, err
|
return nil, err
|
||||||
}
|
}
|
||||||
if deployment.ProjectID != project.ID {
|
if deployment.ProjectID != project.ID {
|
||||||
return nil, errors.New(errPagesDeploymentMismatch)
|
return nil, errors.New(errPagesDeploymentMismatch)
|
||||||
}
|
}
|
||||||
now := time.Now()
|
rootDir, err := validateAndNormalizePagesRootDir(project.RootDir)
|
||||||
if err = db.DB(ctx).Transaction(func(tx *gorm.DB) error {
|
if err != nil {
|
||||||
if err := tx.Model(&model.PagesDeployment{}).
|
|
||||||
Where("project_id = ?", project.ID).
|
|
||||||
Update("status", model.PagesDeploymentStatusUploaded).Error; err != nil {
|
|
||||||
return err
|
|
||||||
}
|
|
||||||
if err := tx.Model(deployment).Updates(map[string]any{
|
|
||||||
"status": model.PagesDeploymentStatusActive,
|
|
||||||
"activated_at": &now,
|
|
||||||
}).Error; err != nil {
|
|
||||||
return err
|
|
||||||
}
|
|
||||||
return tx.Model(project).Updates(map[string]any{
|
|
||||||
"active_deployment_id": deployment.ID,
|
|
||||||
}).Error
|
|
||||||
}); err != nil {
|
|
||||||
return nil, err
|
return nil, err
|
||||||
}
|
}
|
||||||
return GetProject(ctx, project.ID)
|
entryFile, err := validateAndNormalizePagesEntryFile(project.EntryFile)
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
if err := ensureDeploymentEntry(tx, deployment.ID, rootDir, entryFile); err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
var uploadRecord model.Upload
|
||||||
|
if err := tx.Clauses(clause.Locking{Strength: pagesRowLockStrength}).
|
||||||
|
Where("id = ?", deployment.UploadID).
|
||||||
|
First(&uploadRecord).Error; err != nil {
|
||||||
|
return nil, errors.New(errPagesPackageUploadMissing)
|
||||||
|
}
|
||||||
|
if uploadRecord.Status != model.UploadStatusUsed || uploadRecord.Type != upload.ReservedPagesDeploymentType {
|
||||||
|
return nil, errors.New(errPagesPackageUploadMissing)
|
||||||
|
}
|
||||||
|
return &deployment, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func fenceDeploymentActivationSource(
|
||||||
|
tx *gorm.DB,
|
||||||
|
state *deploymentActivationSource,
|
||||||
|
deployment *model.PagesDeployment,
|
||||||
|
audit *deploymentActivationAudit,
|
||||||
|
) error {
|
||||||
|
if state == nil {
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
audit.SourceType = state.Source.SourceType
|
||||||
|
audit.SourceIdentity = state.Source.SourceIdentity
|
||||||
|
audit.AutoDisabled = state.Source.AutoUpdateEnabled
|
||||||
|
if err := tx.Model(state.Source).Updates(map[string]any{
|
||||||
|
sourceColumnConfigVersion: state.Source.ConfigVersion + 1,
|
||||||
|
sourceColumnAutoUpdateEnabled: false,
|
||||||
|
}).Error; err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
if deployment.SourceIdentity != nil && *deployment.SourceIdentity == state.Source.SourceIdentity &&
|
||||||
|
deployment.SourceRevision != nil {
|
||||||
|
state.Runtime.LastAppliedRevision = *deployment.SourceRevision
|
||||||
|
state.Runtime.LastAppliedDetail = deployment.SourceMeta
|
||||||
|
} else {
|
||||||
|
state.Runtime.LastAppliedRevision = ""
|
||||||
|
state.Runtime.LastAppliedDetail = ""
|
||||||
|
}
|
||||||
|
return tx.Model(state.Runtime).Updates(map[string]any{
|
||||||
|
"last_applied_revision": state.Runtime.LastAppliedRevision,
|
||||||
|
"last_applied_detail": state.Runtime.LastAppliedDetail,
|
||||||
|
sourceRuntimeColumnSyncStatus: normalizedSourceRuntimeStatus(state.Runtime),
|
||||||
|
sourceRuntimeColumnLeaseToken: "",
|
||||||
|
sourceRuntimeColumnLeaseExpiresAt: nil,
|
||||||
|
}).Error
|
||||||
|
}
|
||||||
|
|
||||||
|
func switchActiveDeploymentTx(
|
||||||
|
tx *gorm.DB,
|
||||||
|
project *model.PagesProject,
|
||||||
|
deployment *model.PagesDeployment,
|
||||||
|
now time.Time,
|
||||||
|
) error {
|
||||||
|
if err := tx.Model(&model.PagesDeployment{}).
|
||||||
|
Where("project_id = ?", project.ID).
|
||||||
|
Update("status", model.PagesDeploymentStatusUploaded).Error; err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
if err := tx.Model(deployment).Updates(map[string]any{
|
||||||
|
pagesDeploymentColumnStatus: model.PagesDeploymentStatusActive,
|
||||||
|
"activated_at": &now,
|
||||||
|
}).Error; err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
return tx.Model(project).Update("active_deployment_id", deployment.ID).Error
|
||||||
}
|
}
|
||||||
|
|
||||||
// GetDeploymentPackageHash returns the upload SHA-256 hash of the deployment package.
|
// GetDeploymentPackageHash returns the upload SHA-256 hash of the deployment package.
|
||||||
@@ -728,22 +1049,97 @@ func hydrateLegacyDeploymentUpload(
|
|||||||
ctx,
|
ctx,
|
||||||
artifactPath,
|
artifactPath,
|
||||||
deployment.Checksum,
|
deployment.Checksum,
|
||||||
project.Slug,
|
project.ID,
|
||||||
fmt.Sprintf("pages-deployment-%d.zip", deployment.ID),
|
fmt.Sprintf("pages-deployment-%d.zip", deployment.ID),
|
||||||
pagesarchive.FormatZip,
|
pagesarchive.FormatZip,
|
||||||
)
|
)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
return nil, err
|
return nil, err
|
||||||
}
|
}
|
||||||
if err := db.DB(ctx).Model(deployment).Updates(map[string]any{
|
winnerUploadID, err := attachLegacyDeploymentUpload(
|
||||||
"upload_id": ingestResult.Upload.ID,
|
ctx,
|
||||||
"artifact_path": "",
|
project.ID,
|
||||||
}).Error; err != nil {
|
deployment.ID,
|
||||||
|
ingestResult.Upload.ID,
|
||||||
|
)
|
||||||
|
if ingestResult.Created && (err != nil || winnerUploadID != ingestResult.Upload.ID) {
|
||||||
|
if removeErr := removePagesUploadIfUnreferenced(ctx, project.ID, ingestResult.Upload.ID); removeErr != nil {
|
||||||
|
logger.ErrorF(ctx,
|
||||||
|
"[Pages] compensate legacy deployment upload failed: project_id=%d upload_id=%d error=%v",
|
||||||
|
project.ID, ingestResult.Upload.ID, removeErr,
|
||||||
|
)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if err != nil {
|
||||||
return nil, err
|
return nil, err
|
||||||
}
|
}
|
||||||
deployment.UploadID = ingestResult.Upload.ID
|
winner, err := upload.GetActiveUpload(ctx, winnerUploadID)
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
deployment.UploadID = winnerUploadID
|
||||||
deployment.ArtifactPath = ""
|
deployment.ArtifactPath = ""
|
||||||
return &ingestResult.Upload, nil
|
return &winner, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func attachLegacyDeploymentUpload(
|
||||||
|
ctx context.Context,
|
||||||
|
projectID uint,
|
||||||
|
deploymentID uint,
|
||||||
|
uploadID uint64,
|
||||||
|
) (uint64, error) {
|
||||||
|
winnerUploadID := uint64(0)
|
||||||
|
err := db.DB(ctx).Transaction(func(tx *gorm.DB) error {
|
||||||
|
var err error
|
||||||
|
winnerUploadID, err = attachLegacyDeploymentUploadTx(tx, projectID, deploymentID, uploadID)
|
||||||
|
return err
|
||||||
|
})
|
||||||
|
return winnerUploadID, err
|
||||||
|
}
|
||||||
|
|
||||||
|
func attachLegacyDeploymentUploadTx(
|
||||||
|
tx *gorm.DB,
|
||||||
|
projectID uint,
|
||||||
|
deploymentID uint,
|
||||||
|
uploadID uint64,
|
||||||
|
) (uint64, error) {
|
||||||
|
var lockedProject model.PagesProject
|
||||||
|
if err := tx.Clauses(clause.Locking{Strength: pagesRowLockStrength}).First(&lockedProject, projectID).Error; err != nil {
|
||||||
|
return 0, err
|
||||||
|
}
|
||||||
|
var lockedDeployment model.PagesDeployment
|
||||||
|
if err := tx.Clauses(clause.Locking{Strength: pagesRowLockStrength}).First(&lockedDeployment, deploymentID).Error; err != nil {
|
||||||
|
return 0, err
|
||||||
|
}
|
||||||
|
if lockedDeployment.ProjectID != lockedProject.ID {
|
||||||
|
return 0, errors.New(errPagesDeploymentMismatch)
|
||||||
|
}
|
||||||
|
if lockedDeployment.UploadID != 0 {
|
||||||
|
return lockedDeployment.UploadID, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
var uploadRecord model.Upload
|
||||||
|
if err := tx.Clauses(clause.Locking{Strength: pagesRowLockStrength}).
|
||||||
|
Where("id = ?", uploadID).
|
||||||
|
First(&uploadRecord).Error; err != nil {
|
||||||
|
return 0, err
|
||||||
|
}
|
||||||
|
if uploadRecord.Status != model.UploadStatusUsed || uploadRecord.Type != upload.ReservedPagesDeploymentType {
|
||||||
|
return 0, errors.New(errPagesPackageUploadMissing)
|
||||||
|
}
|
||||||
|
result := tx.Model(&model.PagesDeployment{}).
|
||||||
|
Where("id = ? AND project_id = ? AND upload_id = 0", lockedDeployment.ID, lockedProject.ID).
|
||||||
|
Updates(map[string]any{
|
||||||
|
"upload_id": uploadRecord.ID,
|
||||||
|
"artifact_path": "",
|
||||||
|
})
|
||||||
|
if result.Error != nil {
|
||||||
|
return 0, result.Error
|
||||||
|
}
|
||||||
|
if result.RowsAffected != 1 {
|
||||||
|
return 0, errors.New(errPagesPackageUploadMissing)
|
||||||
|
}
|
||||||
|
return uploadRecord.ID, nil
|
||||||
}
|
}
|
||||||
|
|
||||||
// ensureDeploymentInActiveSnapshot allows download of a specific deployment when
|
// ensureDeploymentInActiveSnapshot allows download of a specific deployment when
|
||||||
@@ -842,30 +1238,34 @@ func parseSnapshotRoutes(snapshotJSON string) ([]snapshotRouteRef, error) {
|
|||||||
|
|
||||||
// DeleteDeployment 删除 Pages 部署。
|
// DeleteDeployment 删除 Pages 部署。
|
||||||
func DeleteDeployment(ctx context.Context, projectID uint, deploymentID uint) error {
|
func DeleteDeployment(ctx context.Context, projectID uint, deploymentID uint) error {
|
||||||
project, err := model.GetPagesProjectByID(ctx, projectID)
|
var removed model.PagesDeployment
|
||||||
if err != nil {
|
err := db.DB(ctx).Transaction(func(tx *gorm.DB) error {
|
||||||
return err
|
var project model.PagesProject
|
||||||
}
|
if err := tx.Clauses(clause.Locking{Strength: pagesRowLockStrength}).First(&project, projectID).Error; err != nil {
|
||||||
deployment, err := model.GetPagesDeploymentByID(ctx, deploymentID)
|
|
||||||
if err != nil {
|
|
||||||
return err
|
|
||||||
}
|
|
||||||
if deployment.ProjectID != project.ID {
|
|
||||||
return errors.New(errPagesDeploymentMismatch)
|
|
||||||
}
|
|
||||||
if project.ActiveDeploymentID != nil && *project.ActiveDeploymentID == deployment.ID {
|
|
||||||
return errors.New(errPagesDeleteActiveDeploy)
|
|
||||||
}
|
|
||||||
return db.DB(ctx).Transaction(func(tx *gorm.DB) error {
|
|
||||||
if err := tx.Where("deployment_id = ?", deployment.ID).Delete(&model.PagesDeploymentFile{}).Error; err != nil {
|
|
||||||
return err
|
return err
|
||||||
}
|
}
|
||||||
if err := tx.Delete(deployment).Error; err != nil {
|
if err := tx.First(&removed, deploymentID).Error; err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
if removed.ProjectID != project.ID {
|
||||||
|
return errors.New(errPagesDeploymentMismatch)
|
||||||
|
}
|
||||||
|
if project.ActiveDeploymentID != nil && *project.ActiveDeploymentID == removed.ID {
|
||||||
|
return errors.New(errPagesDeleteActiveDeploy)
|
||||||
|
}
|
||||||
|
if err := tx.Where("deployment_id = ?", removed.ID).Delete(&model.PagesDeploymentFile{}).Error; err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
if err := tx.Delete(&removed).Error; err != nil {
|
||||||
return err
|
return err
|
||||||
}
|
}
|
||||||
removeDeploymentArtifact(ctx, deployment)
|
|
||||||
return nil
|
return nil
|
||||||
})
|
})
|
||||||
|
if err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
removeDeploymentArtifact(ctx, projectID, &removed)
|
||||||
|
return nil
|
||||||
}
|
}
|
||||||
|
|
||||||
func buildProject(existing *model.PagesProject, input Input) (*model.PagesProject, error) {
|
func buildProject(existing *model.PagesProject, input Input) (*model.PagesProject, error) {
|
||||||
@@ -910,7 +1310,7 @@ func buildProject(existing *model.PagesProject, input Input) (*model.PagesProjec
|
|||||||
return nil, errors.New(errPagesAPIProxyPassRequired)
|
return nil, errors.New(errPagesAPIProxyPassRequired)
|
||||||
}
|
}
|
||||||
parsedURL, err := url.Parse(apiProxyPass)
|
parsedURL, err := url.Parse(apiProxyPass)
|
||||||
if err != nil || (parsedURL.Scheme != "http" && parsedURL.Scheme != "https") || parsedURL.Host == "" {
|
if err != nil || (parsedURL.Scheme != remoteSourceSchemeHTTP && parsedURL.Scheme != remoteSourceSchemeHTTPS) || parsedURL.Host == "" {
|
||||||
return nil, errors.New(errPagesAPIProxyPassInvalid)
|
return nil, errors.New(errPagesAPIProxyPassInvalid)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
@@ -923,7 +1323,11 @@ func buildProject(existing *model.PagesProject, input Input) (*model.PagesProjec
|
|||||||
return nil, err
|
return nil, err
|
||||||
}
|
}
|
||||||
existing.RootDir = rootDir
|
existing.RootDir = rootDir
|
||||||
existing.EntryFile = normalizePagesEntryFile(input.EntryFile)
|
entryFile, err := validateAndNormalizePagesEntryFile(input.EntryFile)
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
existing.EntryFile = entryFile
|
||||||
|
|
||||||
return existing, nil
|
return existing, nil
|
||||||
}
|
}
|
||||||
@@ -979,6 +1383,9 @@ func buildDeploymentView(deployment *model.PagesDeployment) DeploymentView {
|
|||||||
FileCount: deployment.FileCount,
|
FileCount: deployment.FileCount,
|
||||||
TotalSize: deployment.TotalSize,
|
TotalSize: deployment.TotalSize,
|
||||||
CreatedBy: deployment.CreatedBy,
|
CreatedBy: deployment.CreatedBy,
|
||||||
|
SourceType: deployment.SourceType,
|
||||||
|
SourceLabel: deployment.SourceLabel,
|
||||||
|
TriggerType: deployment.TriggerType,
|
||||||
CreatedAt: deployment.CreatedAt,
|
CreatedAt: deployment.CreatedAt,
|
||||||
ActivatedAt: deployment.ActivatedAt,
|
ActivatedAt: deployment.ActivatedAt,
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -17,6 +17,7 @@ import (
|
|||||||
"path/filepath"
|
"path/filepath"
|
||||||
"testing"
|
"testing"
|
||||||
|
|
||||||
|
"github.com/Rain-kl/Wavelet/internal/apps/upload"
|
||||||
"github.com/Rain-kl/Wavelet/internal/db"
|
"github.com/Rain-kl/Wavelet/internal/db"
|
||||||
"github.com/Rain-kl/Wavelet/internal/model"
|
"github.com/Rain-kl/Wavelet/internal/model"
|
||||||
"github.com/Rain-kl/Wavelet/internal/repository"
|
"github.com/Rain-kl/Wavelet/internal/repository"
|
||||||
@@ -42,6 +43,8 @@ func setupPagesTestDB(t *testing.T) func() {
|
|||||||
&model.PagesProject{},
|
&model.PagesProject{},
|
||||||
&model.PagesDeployment{},
|
&model.PagesDeployment{},
|
||||||
&model.PagesDeploymentFile{},
|
&model.PagesDeploymentFile{},
|
||||||
|
&model.PagesProjectSource{},
|
||||||
|
&model.PagesProjectSourceRuntime{},
|
||||||
&model.ConfigVersion{},
|
&model.ConfigVersion{},
|
||||||
&model.SystemConfig{},
|
&model.SystemConfig{},
|
||||||
))
|
))
|
||||||
@@ -163,6 +166,89 @@ func TestCreateProjectRejectsUnsafeFallbackPath(t *testing.T) {
|
|||||||
assert.Contains(t, err.Error(), "回退路径")
|
assert.Contains(t, err.Error(), "回退路径")
|
||||||
}
|
}
|
||||||
|
|
||||||
|
func TestCreateProjectRejectsUnsafeContentPaths(t *testing.T) {
|
||||||
|
cleanup := setupPagesTestDB(t)
|
||||||
|
defer cleanup()
|
||||||
|
ctx := context.Background()
|
||||||
|
|
||||||
|
rootDirs := []string{"/public", "public/../dist", "C:/public", `public\\dist`, "./public", "public\x00dist"}
|
||||||
|
for index, rootDir := range rootDirs {
|
||||||
|
_, err := CreateProject(ctx, Input{
|
||||||
|
Name: fmt.Sprintf("Unsafe Root %d", index),
|
||||||
|
Slug: fmt.Sprintf("unsafe-root-%d", index),
|
||||||
|
RootDir: rootDir,
|
||||||
|
EntryFile: "index.html",
|
||||||
|
})
|
||||||
|
require.Error(t, err, rootDir)
|
||||||
|
}
|
||||||
|
|
||||||
|
entryFiles := []string{"/index.html", "../index.html", "C:/index.html", `public\\index.html`, "./index.html", "index.html;bad"}
|
||||||
|
for index, entryFile := range entryFiles {
|
||||||
|
_, err := CreateProject(ctx, Input{
|
||||||
|
Name: fmt.Sprintf("Unsafe Entry %d", index),
|
||||||
|
Slug: fmt.Sprintf("unsafe-entry-%d", index),
|
||||||
|
EntryFile: entryFile,
|
||||||
|
})
|
||||||
|
require.Error(t, err, entryFile)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestUpdateProjectValidatesActiveDeploymentEntry(t *testing.T) {
|
||||||
|
cleanup := setupPagesTestDB(t)
|
||||||
|
defer cleanup()
|
||||||
|
_, disableStorage := setupPagesStorageMock(t)
|
||||||
|
defer disableStorage()
|
||||||
|
ctx := context.Background()
|
||||||
|
|
||||||
|
project, err := CreateProject(ctx, Input{
|
||||||
|
Name: "Content Root",
|
||||||
|
Slug: "content-root",
|
||||||
|
Enabled: true,
|
||||||
|
EntryFile: "index.html",
|
||||||
|
})
|
||||||
|
require.NoError(t, err)
|
||||||
|
deployment, err := UploadDeployment(ctx, project.ID, testPagesMultipartFile(t, "site.zip", testPagesZip(t, map[string]string{
|
||||||
|
"index.html": "root",
|
||||||
|
"dist/index.html": "dist",
|
||||||
|
})), "user:1")
|
||||||
|
require.NoError(t, err)
|
||||||
|
staleCandidate, err := UploadDeployment(ctx, project.ID, testPagesMultipartFile(t, "stale.zip", testPagesZip(t, map[string]string{
|
||||||
|
"index.html": "stale",
|
||||||
|
})), "user:1")
|
||||||
|
require.NoError(t, err)
|
||||||
|
_, err = ActivateDeployment(ctx, project.ID, deployment.ID)
|
||||||
|
require.NoError(t, err)
|
||||||
|
|
||||||
|
updated, err := UpdateProject(ctx, project.ID, Input{
|
||||||
|
Name: project.Name,
|
||||||
|
Slug: project.Slug,
|
||||||
|
Enabled: true,
|
||||||
|
RootDir: "dist",
|
||||||
|
EntryFile: "index.html",
|
||||||
|
})
|
||||||
|
require.NoError(t, err)
|
||||||
|
assert.Equal(t, "dist", updated.RootDir)
|
||||||
|
|
||||||
|
_, err = ActivateDeployment(ctx, project.ID, staleCandidate.ID)
|
||||||
|
require.Error(t, err)
|
||||||
|
assert.Contains(t, err.Error(), errPagesEntryFileMissing)
|
||||||
|
|
||||||
|
_, err = UpdateProject(ctx, project.ID, Input{
|
||||||
|
Name: project.Name,
|
||||||
|
Slug: project.Slug,
|
||||||
|
Enabled: true,
|
||||||
|
RootDir: "missing",
|
||||||
|
EntryFile: "index.html",
|
||||||
|
})
|
||||||
|
require.Error(t, err)
|
||||||
|
assert.Contains(t, err.Error(), errPagesEntryFileMissing)
|
||||||
|
|
||||||
|
stored, err := model.GetPagesProjectByID(ctx, project.ID)
|
||||||
|
require.NoError(t, err)
|
||||||
|
assert.Equal(t, "dist", stored.RootDir)
|
||||||
|
assert.Equal(t, "index.html", stored.EntryFile)
|
||||||
|
}
|
||||||
|
|
||||||
func TestUploadDeploymentAcceptsZeroByteFiles(t *testing.T) {
|
func TestUploadDeploymentAcceptsZeroByteFiles(t *testing.T) {
|
||||||
cleanup := setupPagesTestDB(t)
|
cleanup := setupPagesTestDB(t)
|
||||||
defer cleanup()
|
defer cleanup()
|
||||||
@@ -213,6 +299,13 @@ func TestUploadDeploymentStoresPackageInUploadFramework(t *testing.T) {
|
|||||||
var uploadCount int64
|
var uploadCount int64
|
||||||
require.NoError(t, db.DB(ctx).Model(&model.Upload{}).Count(&uploadCount).Error)
|
require.NoError(t, db.DB(ctx).Model(&model.Upload{}).Count(&uploadCount).Error)
|
||||||
assert.Equal(t, int64(1), uploadCount)
|
assert.Equal(t, int64(1), uploadCount)
|
||||||
|
var uploadRecord model.Upload
|
||||||
|
require.NoError(t, db.DB(ctx).First(&uploadRecord, storedDeployment.UploadID).Error)
|
||||||
|
assert.Equal(t, upload.ReservedPagesDeploymentType, uploadRecord.Type)
|
||||||
|
assert.Equal(t, pagesIngestMarkerV2, uploadRecord.Metadata.Extra[pagesIngestMarkerKey])
|
||||||
|
assert.Equal(t, fmt.Sprint(project.ID), uploadRecord.Metadata.Extra[pagesProjectIDMetadataKey])
|
||||||
|
assert.NotContains(t, uploadRecord.Metadata.Extra, "project_slug")
|
||||||
|
assert.NotContains(t, uploadRecord.Metadata.Extra, "format")
|
||||||
}
|
}
|
||||||
|
|
||||||
func TestOpenDeploymentPackageHydratesLegacyArtifactPath(t *testing.T) {
|
func TestOpenDeploymentPackageHydratesLegacyArtifactPath(t *testing.T) {
|
||||||
@@ -448,34 +541,42 @@ func TestSelectDeploymentsToPruneKeepsActiveAndNewest(t *testing.T) {
|
|||||||
{ID: 2, ProjectID: 1},
|
{ID: 2, ProjectID: 1},
|
||||||
{ID: 1, ProjectID: 1},
|
{ID: 1, ProjectID: 1},
|
||||||
}
|
}
|
||||||
toDelete := selectDeploymentsToPrune(deployments, 1, 2)
|
toDelete := selectDeploymentsToPrune(deployments, 1, 0, 2)
|
||||||
require.Len(t, toDelete, 2)
|
require.Len(t, toDelete, 2)
|
||||||
assert.Equal(t, uint(3), toDelete[0].ID)
|
assert.Equal(t, uint(3), toDelete[0].ID)
|
||||||
assert.Equal(t, uint(2), toDelete[1].ID)
|
assert.Equal(t, uint(2), toDelete[1].ID)
|
||||||
|
|
||||||
// active is newest; keep=2 → keep {4,3}, prune {2,1}
|
// active is newest; keep=2 → keep {4,3}, prune {2,1}
|
||||||
toDelete = selectDeploymentsToPrune(deployments, 4, 2)
|
toDelete = selectDeploymentsToPrune(deployments, 4, 0, 2)
|
||||||
require.Len(t, toDelete, 2)
|
require.Len(t, toDelete, 2)
|
||||||
assert.Equal(t, uint(2), toDelete[0].ID)
|
assert.Equal(t, uint(2), toDelete[0].ID)
|
||||||
assert.Equal(t, uint(1), toDelete[1].ID)
|
assert.Equal(t, uint(1), toDelete[1].ID)
|
||||||
|
|
||||||
// no active; keep=2 → keep {4,3}
|
// no active; keep=2 → keep {4,3}
|
||||||
toDelete = selectDeploymentsToPrune(deployments, 0, 2)
|
toDelete = selectDeploymentsToPrune(deployments, 0, 0, 2)
|
||||||
require.Len(t, toDelete, 2)
|
require.Len(t, toDelete, 2)
|
||||||
assert.Equal(t, uint(2), toDelete[0].ID)
|
assert.Equal(t, uint(2), toDelete[0].ID)
|
||||||
assert.Equal(t, uint(1), toDelete[1].ID)
|
assert.Equal(t, uint(1), toDelete[1].ID)
|
||||||
|
|
||||||
// keep=1 with active → only active, prune the rest
|
// keep=1 with active → only active, prune the rest
|
||||||
toDelete = selectDeploymentsToPrune(deployments, 2, 1)
|
toDelete = selectDeploymentsToPrune(deployments, 2, 0, 1)
|
||||||
require.Len(t, toDelete, 3)
|
require.Len(t, toDelete, 3)
|
||||||
for _, item := range toDelete {
|
for _, item := range toDelete {
|
||||||
assert.NotEqual(t, uint(2), item.ID)
|
assert.NotEqual(t, uint(2), item.ID)
|
||||||
}
|
}
|
||||||
|
|
||||||
// already within limit
|
// already within limit
|
||||||
assert.Nil(t, selectDeploymentsToPrune(deployments[:2], 4, 2))
|
assert.Nil(t, selectDeploymentsToPrune(deployments[:2], 4, 0, 2))
|
||||||
// unlimited
|
// unlimited
|
||||||
assert.Nil(t, selectDeploymentsToPrune(deployments, 1, 0))
|
assert.Nil(t, selectDeploymentsToPrune(deployments, 1, 0, 0))
|
||||||
|
|
||||||
|
// history=1 temporarily preserves active plus the freshly uploaded candidate.
|
||||||
|
toDelete = selectDeploymentsToPrune(deployments, 2, 4, 1)
|
||||||
|
require.Len(t, toDelete, 2)
|
||||||
|
assert.Equal(t, uint(3), toDelete[0].ID)
|
||||||
|
assert.Equal(t, uint(1), toDelete[1].ID)
|
||||||
|
assert.Equal(t, uint(4), resolveLatestCandidateID(deployments, 2, true))
|
||||||
|
assert.Zero(t, resolveLatestCandidateID(deployments, 2, false))
|
||||||
}
|
}
|
||||||
|
|
||||||
func TestPruneProjectDeploymentHistory(t *testing.T) {
|
func TestPruneProjectDeploymentHistory(t *testing.T) {
|
||||||
@@ -540,6 +641,131 @@ func TestPruneProjectDeploymentHistory(t *testing.T) {
|
|||||||
assert.True(t, hasLatest, "newest deployment must fill remaining slot")
|
assert.True(t, hasLatest, "newest deployment must fill remaining slot")
|
||||||
}
|
}
|
||||||
|
|
||||||
|
func TestHistoryCountOnePreservesFreshCandidateUntilActivation(t *testing.T) {
|
||||||
|
cleanup := setupPagesTestDB(t)
|
||||||
|
defer cleanup()
|
||||||
|
_, disableStorage := setupPagesStorageMock(t)
|
||||||
|
defer disableStorage()
|
||||||
|
ctx := context.Background()
|
||||||
|
|
||||||
|
require.NoError(t, db.DB(ctx).Model(&model.SystemConfig{}).
|
||||||
|
Where("key = ?", model.ConfigKeyPagesMaxHistoryCount).
|
||||||
|
Update("value", "1").Error)
|
||||||
|
require.NoError(t, repository.InvalidateSystemConfigCache(ctx, model.ConfigKeyPagesMaxHistoryCount))
|
||||||
|
|
||||||
|
project, err := CreateProject(ctx, Input{Name: "Single History", Slug: "single-history", Enabled: true})
|
||||||
|
require.NoError(t, err)
|
||||||
|
active, err := UploadDeployment(ctx, project.ID, testPagesMultipartFile(t, "v1.zip", testPagesZip(t, map[string]string{
|
||||||
|
"index.html": "v1",
|
||||||
|
})), "user:1")
|
||||||
|
require.NoError(t, err)
|
||||||
|
_, err = ActivateDeployment(ctx, project.ID, active.ID)
|
||||||
|
require.NoError(t, err)
|
||||||
|
|
||||||
|
oldCandidate, err := UploadDeployment(ctx, project.ID, testPagesMultipartFile(t, "v2.zip", testPagesZip(t, map[string]string{
|
||||||
|
"index.html": "v2",
|
||||||
|
})), "user:1")
|
||||||
|
require.NoError(t, err)
|
||||||
|
deployments, err := model.ListPagesDeployments(ctx, project.ID)
|
||||||
|
require.NoError(t, err)
|
||||||
|
require.Len(t, deployments, 2)
|
||||||
|
|
||||||
|
newCandidate, err := UploadDeployment(ctx, project.ID, testPagesMultipartFile(t, "v3.zip", testPagesZip(t, map[string]string{
|
||||||
|
"index.html": "v3",
|
||||||
|
})), "user:1")
|
||||||
|
require.NoError(t, err)
|
||||||
|
deployments, err = model.ListPagesDeployments(ctx, project.ID)
|
||||||
|
require.NoError(t, err)
|
||||||
|
require.Len(t, deployments, 2)
|
||||||
|
kept := map[uint]bool{}
|
||||||
|
for _, deployment := range deployments {
|
||||||
|
kept[deployment.ID] = true
|
||||||
|
}
|
||||||
|
assert.True(t, kept[active.ID])
|
||||||
|
assert.True(t, kept[newCandidate.ID])
|
||||||
|
assert.False(t, kept[oldCandidate.ID])
|
||||||
|
var removedUpload model.Upload
|
||||||
|
require.NoError(t, db.DB(ctx).First(&removedUpload, oldCandidate.UploadID).Error)
|
||||||
|
assert.Equal(t, model.UploadStatusDeleted, removedUpload.Status)
|
||||||
|
|
||||||
|
_, err = ActivateDeployment(ctx, project.ID, newCandidate.ID)
|
||||||
|
require.NoError(t, err)
|
||||||
|
deployments, err = model.ListPagesDeployments(ctx, project.ID)
|
||||||
|
require.NoError(t, err)
|
||||||
|
require.Len(t, deployments, 1)
|
||||||
|
assert.Equal(t, newCandidate.ID, deployments[0].ID)
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestPruneUsesLockTimeNewestCandidateInsteadOfStaleCaller(t *testing.T) {
|
||||||
|
cleanup := setupPagesTestDB(t)
|
||||||
|
defer cleanup()
|
||||||
|
_, disableStorage := setupPagesStorageMock(t)
|
||||||
|
defer disableStorage()
|
||||||
|
ctx := context.Background()
|
||||||
|
|
||||||
|
project, err := CreateProject(ctx, Input{Name: "Concurrent Candidate", Slug: "concurrent-candidate", Enabled: true})
|
||||||
|
require.NoError(t, err)
|
||||||
|
active, err := UploadDeployment(ctx, project.ID, testPagesMultipartFile(t, "v1.zip", testPagesZip(t, map[string]string{
|
||||||
|
"index.html": "v1",
|
||||||
|
})), "user:1")
|
||||||
|
require.NoError(t, err)
|
||||||
|
_, err = ActivateDeployment(ctx, project.ID, active.ID)
|
||||||
|
require.NoError(t, err)
|
||||||
|
staleCandidate, err := UploadDeployment(ctx, project.ID, testPagesMultipartFile(t, "v2.zip", testPagesZip(t, map[string]string{
|
||||||
|
"index.html": "v2",
|
||||||
|
})), "user:1")
|
||||||
|
require.NoError(t, err)
|
||||||
|
newCandidate, err := UploadDeployment(ctx, project.ID, testPagesMultipartFile(t, "v3.zip", testPagesZip(t, map[string]string{
|
||||||
|
"index.html": "v3",
|
||||||
|
})), "user:1")
|
||||||
|
require.NoError(t, err)
|
||||||
|
|
||||||
|
deleted, err := pruneProjectDeploymentHistoryOnce(ctx, project.ID, 1, staleCandidate.ID)
|
||||||
|
require.NoError(t, err)
|
||||||
|
assert.Equal(t, 1, deleted)
|
||||||
|
deployments, err := model.ListPagesDeployments(ctx, project.ID)
|
||||||
|
require.NoError(t, err)
|
||||||
|
require.Len(t, deployments, 2)
|
||||||
|
kept := map[uint]bool{}
|
||||||
|
for _, deployment := range deployments {
|
||||||
|
kept[deployment.ID] = true
|
||||||
|
}
|
||||||
|
assert.True(t, kept[active.ID])
|
||||||
|
assert.True(t, kept[newCandidate.ID])
|
||||||
|
assert.False(t, kept[staleCandidate.ID])
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestDeleteDeploymentAndProjectSoftDeleteUnreferencedArtifacts(t *testing.T) {
|
||||||
|
cleanup := setupPagesTestDB(t)
|
||||||
|
defer cleanup()
|
||||||
|
_, disableStorage := setupPagesStorageMock(t)
|
||||||
|
defer disableStorage()
|
||||||
|
ctx := context.Background()
|
||||||
|
|
||||||
|
project, err := CreateProject(ctx, Input{Name: "Delete Artifacts", Slug: "delete-artifacts", Enabled: true})
|
||||||
|
require.NoError(t, err)
|
||||||
|
first, err := UploadDeployment(ctx, project.ID, testPagesMultipartFile(t, "first.zip", testPagesZip(t, map[string]string{
|
||||||
|
"index.html": "first",
|
||||||
|
})), "user:1")
|
||||||
|
require.NoError(t, err)
|
||||||
|
second, err := UploadDeployment(ctx, project.ID, testPagesMultipartFile(t, "second.zip", testPagesZip(t, map[string]string{
|
||||||
|
"index.html": "second",
|
||||||
|
})), "user:1")
|
||||||
|
require.NoError(t, err)
|
||||||
|
|
||||||
|
require.NoError(t, DeleteDeployment(ctx, project.ID, second.ID))
|
||||||
|
var secondUpload model.Upload
|
||||||
|
require.NoError(t, db.DB(ctx).First(&secondUpload, second.UploadID).Error)
|
||||||
|
assert.Equal(t, model.UploadStatusDeleted, secondUpload.Status)
|
||||||
|
|
||||||
|
require.NoError(t, DeleteProject(ctx, project.ID))
|
||||||
|
var firstUpload model.Upload
|
||||||
|
require.NoError(t, db.DB(ctx).First(&firstUpload, first.UploadID).Error)
|
||||||
|
assert.Equal(t, model.UploadStatusDeleted, firstUpload.Status)
|
||||||
|
_, err = model.GetPagesProjectByID(ctx, project.ID)
|
||||||
|
assert.Error(t, err)
|
||||||
|
}
|
||||||
|
|
||||||
func testPagesTarGz(t *testing.T, files map[string]string) []byte {
|
func testPagesTarGz(t *testing.T, files map[string]string) []byte {
|
||||||
t.Helper()
|
t.Helper()
|
||||||
|
|
||||||
|
|||||||
@@ -0,0 +1,64 @@
|
|||||||
|
// Copyright 2026 Arctel.net
|
||||||
|
// SPDX-License-Identifier: Apache-2.0
|
||||||
|
|
||||||
|
package pages
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
"errors"
|
||||||
|
"fmt"
|
||||||
|
"strings"
|
||||||
|
|
||||||
|
"github.com/Rain-kl/Wavelet/internal/apps/upload"
|
||||||
|
"github.com/Rain-kl/Wavelet/internal/model"
|
||||||
|
)
|
||||||
|
|
||||||
|
// ProjectLatestPackageMetadata describes the active package limits published to Agents.
|
||||||
|
type ProjectLatestPackageMetadata struct {
|
||||||
|
DeploymentID uint
|
||||||
|
Hash string
|
||||||
|
PackageSize int64
|
||||||
|
FileCount int
|
||||||
|
TotalSize int64
|
||||||
|
}
|
||||||
|
|
||||||
|
// GetProjectLatestPackageMetadata returns one coherent metadata snapshot for a
|
||||||
|
// project's currently active deployment.
|
||||||
|
func GetProjectLatestPackageMetadata(ctx context.Context, projectID uint) (*ProjectLatestPackageMetadata, error) {
|
||||||
|
deployment, err := resolveProjectActiveDeploymentForAgent(ctx, projectID)
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
if deployment.UploadID == 0 {
|
||||||
|
if err := ensureDeploymentUploadRecord(ctx, deployment); err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
deployment, err = model.GetPagesDeploymentByID(ctx, deployment.ID)
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if deployment.UploadID == 0 {
|
||||||
|
return nil, errors.New(errPagesDeploymentNotFound)
|
||||||
|
}
|
||||||
|
|
||||||
|
uploadRecord, err := upload.GetActiveUpload(ctx, deployment.UploadID)
|
||||||
|
if err != nil {
|
||||||
|
return nil, fmt.Errorf("pages 部署包不存在: %w", err)
|
||||||
|
}
|
||||||
|
hash := strings.TrimSpace(uploadRecord.Hash)
|
||||||
|
if hash == "" {
|
||||||
|
hash = strings.TrimSpace(deployment.Checksum)
|
||||||
|
}
|
||||||
|
if hash == "" {
|
||||||
|
return nil, errors.New(errPagesDeploymentHashMissing)
|
||||||
|
}
|
||||||
|
|
||||||
|
return &ProjectLatestPackageMetadata{
|
||||||
|
DeploymentID: deployment.ID,
|
||||||
|
Hash: hash,
|
||||||
|
PackageSize: uploadRecord.FileSize,
|
||||||
|
FileCount: deployment.FileCount,
|
||||||
|
TotalSize: deployment.TotalSize,
|
||||||
|
}, nil
|
||||||
|
}
|
||||||
@@ -0,0 +1,81 @@
|
|||||||
|
// Copyright 2026 Arctel.net
|
||||||
|
// SPDX-License-Identifier: Apache-2.0
|
||||||
|
|
||||||
|
package pages
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
"crypto/sha256"
|
||||||
|
"encoding/hex"
|
||||||
|
"fmt"
|
||||||
|
"testing"
|
||||||
|
|
||||||
|
"github.com/Rain-kl/Wavelet/internal/db"
|
||||||
|
"github.com/Rain-kl/Wavelet/internal/model"
|
||||||
|
)
|
||||||
|
|
||||||
|
func TestGetProjectLatestPackageMetadataPreservesZeroTotalSize(t *testing.T) {
|
||||||
|
cleanup := setupPagesTestDB(t)
|
||||||
|
defer cleanup()
|
||||||
|
_, disableStorage := setupPagesStorageMock(t)
|
||||||
|
defer disableStorage()
|
||||||
|
|
||||||
|
ctx := context.Background()
|
||||||
|
project, err := CreateProject(ctx, Input{
|
||||||
|
Name: "Empty Files",
|
||||||
|
Slug: "empty-files",
|
||||||
|
Enabled: true,
|
||||||
|
EntryFile: "index.html",
|
||||||
|
})
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("CreateProject() error = %v", err)
|
||||||
|
}
|
||||||
|
packageBytes := testPagesZip(t, map[string]string{
|
||||||
|
"index.html": "",
|
||||||
|
".gitkeep": "",
|
||||||
|
})
|
||||||
|
deployment, err := UploadDeployment(
|
||||||
|
ctx,
|
||||||
|
project.ID,
|
||||||
|
testPagesMultipartFile(t, "empty-files.zip", packageBytes),
|
||||||
|
"test",
|
||||||
|
)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("UploadDeployment() error = %v", err)
|
||||||
|
}
|
||||||
|
if _, err := ActivateDeployment(ctx, project.ID, deployment.ID); err != nil {
|
||||||
|
t.Fatalf("ActivateDeployment() error = %v", err)
|
||||||
|
}
|
||||||
|
if err := db.DB(ctx).Create(&model.ConfigVersion{
|
||||||
|
Version: "v-package-metadata",
|
||||||
|
SnapshotJSON: fmt.Sprintf(
|
||||||
|
`{"routes":[{"upstream_type":"pages","pages_project_id":%d}]}`,
|
||||||
|
project.ID,
|
||||||
|
),
|
||||||
|
SupportFilesJSON: "[]",
|
||||||
|
Checksum: "package-metadata-config",
|
||||||
|
IsActive: true,
|
||||||
|
CreatedBy: "test",
|
||||||
|
}).Error; err != nil {
|
||||||
|
t.Fatalf("create active ConfigVersion error = %v", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
got, err := GetProjectLatestPackageMetadata(ctx, project.ID)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("GetProjectLatestPackageMetadata(%d) error = %v", project.ID, err)
|
||||||
|
}
|
||||||
|
wantHashBytes := sha256.Sum256(packageBytes)
|
||||||
|
wantHash := hex.EncodeToString(wantHashBytes[:])
|
||||||
|
if got.DeploymentID != deployment.ID || got.Hash != wantHash {
|
||||||
|
t.Errorf("GetProjectLatestPackageMetadata(%d) identity = (%d, %q), want (%d, %q)",
|
||||||
|
project.ID, got.DeploymentID, got.Hash, deployment.ID, wantHash)
|
||||||
|
}
|
||||||
|
if got.PackageSize != int64(len(packageBytes)) {
|
||||||
|
t.Errorf("GetProjectLatestPackageMetadata(%d).PackageSize = %d, want %d",
|
||||||
|
project.ID, got.PackageSize, len(packageBytes))
|
||||||
|
}
|
||||||
|
if got.FileCount != 2 || got.TotalSize != 0 {
|
||||||
|
t.Errorf("GetProjectLatestPackageMetadata(%d) content = (%d files, %d bytes), want (2 files, 0 bytes)",
|
||||||
|
project.ID, got.FileCount, got.TotalSize)
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -7,6 +7,7 @@ import (
|
|||||||
"context"
|
"context"
|
||||||
"encoding/json"
|
"encoding/json"
|
||||||
"fmt"
|
"fmt"
|
||||||
|
"path"
|
||||||
"strings"
|
"strings"
|
||||||
|
|
||||||
"github.com/Rain-kl/Wavelet/internal/model"
|
"github.com/Rain-kl/Wavelet/internal/model"
|
||||||
@@ -102,7 +103,10 @@ func rebindPagesRouteMaps(ctx context.Context, routes []map[string]json.RawMessa
|
|||||||
if err != nil {
|
if err != nil {
|
||||||
return false, err
|
return false, err
|
||||||
}
|
}
|
||||||
deployment := buildLivePagesDeployment(project, activeDeployment)
|
deployment, err := buildLivePagesDeployment(project, activeDeployment)
|
||||||
|
if err != nil {
|
||||||
|
return false, err
|
||||||
|
}
|
||||||
projectIDCopy := project.ID
|
projectIDCopy := project.ID
|
||||||
originURL := fmt.Sprintf("openflare-pages://project/%d", project.ID)
|
originURL := fmt.Sprintf("openflare-pages://project/%d", project.ID)
|
||||||
|
|
||||||
@@ -181,15 +185,26 @@ func loadActivePagesProject(ctx context.Context, projectID uint, siteName string
|
|||||||
return project, activeDeployment, nil
|
return project, activeDeployment, nil
|
||||||
}
|
}
|
||||||
|
|
||||||
func buildLivePagesDeployment(project *model.PagesProject, active *model.PagesDeployment) *openrestyrender.PagesDeployment {
|
func buildLivePagesDeployment(
|
||||||
entryFile := strings.TrimSpace(project.EntryFile)
|
project *model.PagesProject,
|
||||||
if entryFile == "" {
|
active *model.PagesDeployment,
|
||||||
entryFile = defaultPagesEntryFile
|
) (*openrestyrender.PagesDeployment, error) {
|
||||||
|
rootDir, err := validateAndNormalizePagesRootDir(project.RootDir)
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
entryFile, err := validateAndNormalizePagesEntryFile(project.EntryFile)
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
}
|
}
|
||||||
fallbackPath := strings.TrimSpace(project.SPAFallbackPath)
|
fallbackPath := strings.TrimSpace(project.SPAFallbackPath)
|
||||||
if fallbackPath == "" {
|
if fallbackPath == "" {
|
||||||
fallbackPath = defaultPagesFallbackPath
|
fallbackPath = defaultPagesFallbackPath
|
||||||
}
|
}
|
||||||
|
localRoot := openrestyrender.PagesProjectLocalRoot(project.ID)
|
||||||
|
if rootDir != "" {
|
||||||
|
localRoot = path.Join(localRoot, rootDir)
|
||||||
|
}
|
||||||
return &openrestyrender.PagesDeployment{
|
return &openrestyrender.PagesDeployment{
|
||||||
ProjectID: project.ID,
|
ProjectID: project.ID,
|
||||||
ProjectSlug: strings.TrimSpace(project.Slug),
|
ProjectSlug: strings.TrimSpace(project.Slug),
|
||||||
@@ -203,8 +218,8 @@ func buildLivePagesDeployment(project *model.PagesProject, active *model.PagesDe
|
|||||||
APIProxyPath: strings.TrimSpace(project.APIProxyPath),
|
APIProxyPath: strings.TrimSpace(project.APIProxyPath),
|
||||||
APIProxyPass: strings.TrimSpace(project.APIProxyPass),
|
APIProxyPass: strings.TrimSpace(project.APIProxyPass),
|
||||||
APIProxyRewrite: strings.TrimSpace(project.APIProxyRewrite),
|
APIProxyRewrite: strings.TrimSpace(project.APIProxyRewrite),
|
||||||
LocalRoot: openrestyrender.PagesProjectLocalRoot(project.ID),
|
LocalRoot: localRoot,
|
||||||
}
|
}, nil
|
||||||
}
|
}
|
||||||
|
|
||||||
func rawJSONString(raw json.RawMessage) (string, bool) {
|
func rawJSONString(raw json.RawMessage) (string, bool) {
|
||||||
|
|||||||
@@ -20,9 +20,11 @@ func TestRebindSnapshotPagesToCurrentActive(t *testing.T) {
|
|||||||
ctx := context.Background()
|
ctx := context.Background()
|
||||||
|
|
||||||
project, err := CreateProject(ctx, Input{
|
project, err := CreateProject(ctx, Input{
|
||||||
Name: "Rebind Site",
|
Name: "Rebind Site",
|
||||||
Slug: "rebind-site",
|
Slug: "rebind-site",
|
||||||
Enabled: true,
|
Enabled: true,
|
||||||
|
RootDir: "public/site",
|
||||||
|
EntryFile: "index.html",
|
||||||
})
|
})
|
||||||
require.NoError(t, err)
|
require.NoError(t, err)
|
||||||
|
|
||||||
@@ -84,4 +86,5 @@ func TestRebindSnapshotPagesToCurrentActive(t *testing.T) {
|
|||||||
deployment := route["pages_deployment"].(map[string]any)
|
deployment := route["pages_deployment"].(map[string]any)
|
||||||
assert.EqualValues(t, active.ID, deployment["deployment_id"])
|
assert.EqualValues(t, active.ID, deployment["deployment_id"])
|
||||||
assert.Equal(t, "new-checksum", deployment["checksum"])
|
assert.Equal(t, "new-checksum", deployment["checksum"])
|
||||||
|
assert.Equal(t, "__OPENFLARE_PAGES_DIR__/projects/1/current/public/site", deployment["local_root"])
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -4,12 +4,20 @@
|
|||||||
package pages
|
package pages
|
||||||
|
|
||||||
import (
|
import (
|
||||||
|
"encoding/json"
|
||||||
|
"errors"
|
||||||
|
"fmt"
|
||||||
|
"io"
|
||||||
"net/http"
|
"net/http"
|
||||||
"strconv"
|
"strconv"
|
||||||
|
|
||||||
|
"github.com/Rain-kl/Wavelet/internal/apps/oauth"
|
||||||
"github.com/Rain-kl/Wavelet/internal/apps/openflare/apiutil"
|
"github.com/Rain-kl/Wavelet/internal/apps/openflare/apiutil"
|
||||||
"github.com/Rain-kl/Wavelet/internal/common/response"
|
"github.com/Rain-kl/Wavelet/internal/common/response"
|
||||||
|
"github.com/Rain-kl/Wavelet/internal/model"
|
||||||
|
"github.com/Rain-kl/Wavelet/pkg/logger"
|
||||||
"github.com/gin-gonic/gin"
|
"github.com/gin-gonic/gin"
|
||||||
|
"gorm.io/gorm"
|
||||||
)
|
)
|
||||||
|
|
||||||
func handleLogicError(c *gin.Context, err error) bool {
|
func handleLogicError(c *gin.Context, err error) bool {
|
||||||
@@ -19,6 +27,64 @@ func handleLogicError(c *gin.Context, err error) bool {
|
|||||||
return apiutil.AbortNotFoundIfMissing(c, err, errPagesProjectNotFound)
|
return apiutil.AbortNotFoundIfMissing(c, err, errPagesProjectNotFound)
|
||||||
}
|
}
|
||||||
|
|
||||||
|
func handleSourceLogicError(c *gin.Context, err error) bool {
|
||||||
|
if err == nil {
|
||||||
|
return false
|
||||||
|
}
|
||||||
|
if errors.Is(err, gorm.ErrRecordNotFound) || err.Error() == errPagesSourceNotFound {
|
||||||
|
response.AbortNotFound(c, errPagesSourceNotFound)
|
||||||
|
return true
|
||||||
|
}
|
||||||
|
switch err.Error() {
|
||||||
|
case errPagesSourceActionBusy:
|
||||||
|
response.AbortConflict(c, errPagesSourceActionBusy)
|
||||||
|
case errPagesSourceTypeRequired,
|
||||||
|
errPagesSourceTypeUnsupported,
|
||||||
|
errPagesSourceRemoteFields,
|
||||||
|
errPagesSourceRemoteURLRequired,
|
||||||
|
errPagesSourceRemoteURLMode,
|
||||||
|
errPagesSourceRemoteURLInvalid,
|
||||||
|
errPagesSourceNetworkPolicy,
|
||||||
|
errPagesSourceGitHubFields,
|
||||||
|
errPagesSourceRepositoryInvalid,
|
||||||
|
errPagesSourceSelectorInvalid,
|
||||||
|
errPagesSourceAssetNameInvalid,
|
||||||
|
errPagesSourceCheckInterval,
|
||||||
|
errPagesSourceAutoNotAvailable,
|
||||||
|
errPagesSourceReleaseNotFound,
|
||||||
|
errPagesSourceDigestInvalid,
|
||||||
|
errPagesSourceDigestMismatch,
|
||||||
|
errPagesSourceConfirmationNeeded,
|
||||||
|
errPagesSourceConfirmationStale,
|
||||||
|
errPagesSourceCheckUnsupported,
|
||||||
|
errPagesSourceActionInvalid:
|
||||||
|
response.AbortBadRequest(c, err.Error())
|
||||||
|
case errPagesSourceTaskDispatchFailed:
|
||||||
|
response.AbortInternal(c, errPagesSourceInternal)
|
||||||
|
default:
|
||||||
|
logger.ErrorF(c.Request.Context(), "[PagesSource] API operation failed: error=%v", err)
|
||||||
|
response.AbortInternal(c, errPagesSourceInternal)
|
||||||
|
}
|
||||||
|
return true
|
||||||
|
}
|
||||||
|
|
||||||
|
func decodeStrictJSON(c *gin.Context, target any, allowEmpty bool) bool {
|
||||||
|
decoder := json.NewDecoder(c.Request.Body)
|
||||||
|
decoder.DisallowUnknownFields()
|
||||||
|
if err := decoder.Decode(target); err != nil {
|
||||||
|
if allowEmpty && errors.Is(err, io.EOF) {
|
||||||
|
return true
|
||||||
|
}
|
||||||
|
response.AbortBadRequest(c, errPagesSourceActionInvalid)
|
||||||
|
return false
|
||||||
|
}
|
||||||
|
if err := ensureJSONEOF(decoder); err != nil {
|
||||||
|
response.AbortBadRequest(c, errPagesSourceActionInvalid)
|
||||||
|
return false
|
||||||
|
}
|
||||||
|
return true
|
||||||
|
}
|
||||||
|
|
||||||
func deploymentIDParam(c *gin.Context) (uint, bool) {
|
func deploymentIDParam(c *gin.Context) (uint, bool) {
|
||||||
raw := c.Param("deployment_id")
|
raw := c.Param("deployment_id")
|
||||||
if raw == "" {
|
if raw == "" {
|
||||||
@@ -33,6 +99,15 @@ func deploymentIDParam(c *gin.Context) (uint, bool) {
|
|||||||
return uint(id64), true
|
return uint(id64), true
|
||||||
}
|
}
|
||||||
|
|
||||||
|
func currentPagesActor(c *gin.Context) (string, bool) {
|
||||||
|
user, ok := oauth.GetFromContext[*model.User](c, oauth.UserObjKey)
|
||||||
|
if !ok || user == nil || user.ID == 0 {
|
||||||
|
response.AbortUnauthorized(c, errPagesActorMissing)
|
||||||
|
return "", false
|
||||||
|
}
|
||||||
|
return fmt.Sprintf("user:%d", user.ID), true
|
||||||
|
}
|
||||||
|
|
||||||
// ListProjectsHandler 列出全部 Pages 项目。
|
// ListProjectsHandler 列出全部 Pages 项目。
|
||||||
// @Summary 列出 Pages 项目
|
// @Summary 列出 Pages 项目
|
||||||
// @Description 返回全部 OpenFlare Pages 项目,需要管理员权限
|
// @Description 返回全部 OpenFlare Pages 项目,需要管理员权限
|
||||||
@@ -162,6 +237,168 @@ func DeleteProjectHandler(c *gin.Context) {
|
|||||||
c.JSON(http.StatusOK, response.OKNil())
|
c.JSON(http.StatusOK, response.OKNil())
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// GetSourceHandler 获取 Pages 项目的部署源。
|
||||||
|
// @Summary 获取 Pages 部署源
|
||||||
|
// @Description 返回脱敏后的项目部署源配置与运行状态,需要管理员权限
|
||||||
|
// @Tags openflare-pages
|
||||||
|
// @Produce json
|
||||||
|
// @Security SessionCookie
|
||||||
|
// @Param id path int true "项目 ID"
|
||||||
|
// @Success 200 {object} response.Any{data=pages.SourceView} "部署源"
|
||||||
|
// @Failure 400 {object} response.Any "参数错误"
|
||||||
|
// @Failure 401 {object} response.Any "未登录"
|
||||||
|
// @Failure 404 {object} response.Any "项目或部署源不存在"
|
||||||
|
// @Failure 500 {object} response.Any "内部错误"
|
||||||
|
// @Router /api/v1/d/pages/{id}/source [get]
|
||||||
|
func GetSourceHandler(c *gin.Context) {
|
||||||
|
projectID, ok := apiutil.IDParam(c)
|
||||||
|
if !ok {
|
||||||
|
return
|
||||||
|
}
|
||||||
|
source, err := GetSource(c.Request.Context(), projectID)
|
||||||
|
if handleSourceLogicError(c, err) {
|
||||||
|
return
|
||||||
|
}
|
||||||
|
c.JSON(http.StatusOK, response.OK(source))
|
||||||
|
}
|
||||||
|
|
||||||
|
// UpdateSourceHandler 创建或更新 Pages 项目部署源。
|
||||||
|
// @Summary 更新 Pages 部署源
|
||||||
|
// @Description 支持 Remote URL 与公开 GitHub Release 来源;敏感地址仅写入,不会在响应中返回
|
||||||
|
// @Tags openflare-pages
|
||||||
|
// @Accept json
|
||||||
|
// @Produce json
|
||||||
|
// @Security SessionCookie
|
||||||
|
// @Param id path int true "项目 ID"
|
||||||
|
// @Param request body pages.SourceUpdateInput true "部署源配置"
|
||||||
|
// @Success 200 {object} response.Any{data=pages.SourceUpdateResult} "更新结果"
|
||||||
|
// @Failure 400 {object} response.Any "配置无效"
|
||||||
|
// @Failure 401 {object} response.Any "未登录"
|
||||||
|
// @Failure 404 {object} response.Any "项目不存在"
|
||||||
|
// @Failure 500 {object} response.Any "内部错误"
|
||||||
|
// @Router /api/v1/d/pages/{id}/source/update [post]
|
||||||
|
func UpdateSourceHandler(c *gin.Context) {
|
||||||
|
projectID, ok := apiutil.IDParam(c)
|
||||||
|
if !ok {
|
||||||
|
return
|
||||||
|
}
|
||||||
|
var input SourceUpdateInput
|
||||||
|
if !decodeStrictJSON(c, &input, false) {
|
||||||
|
return
|
||||||
|
}
|
||||||
|
actor, ok := currentPagesActor(c)
|
||||||
|
if !ok {
|
||||||
|
return
|
||||||
|
}
|
||||||
|
result, err := UpdateSourceAs(c.Request.Context(), projectID, input, actor)
|
||||||
|
if handleSourceLogicError(c, err) {
|
||||||
|
return
|
||||||
|
}
|
||||||
|
c.JSON(http.StatusOK, response.OK(result))
|
||||||
|
}
|
||||||
|
|
||||||
|
// DeleteSourceHandler 将 Pages 项目切换回手动部署模式。
|
||||||
|
// @Summary 删除 Pages 部署源
|
||||||
|
// @Description 幂等删除持久部署源;已有部署历史与当前生产部署保持不变
|
||||||
|
// @Tags openflare-pages
|
||||||
|
// @Produce json
|
||||||
|
// @Security SessionCookie
|
||||||
|
// @Param id path int true "项目 ID"
|
||||||
|
// @Success 200 {object} response.Any{data=pages.SourceView} "手动来源视图"
|
||||||
|
// @Failure 400 {object} response.Any "参数错误"
|
||||||
|
// @Failure 401 {object} response.Any "未登录"
|
||||||
|
// @Failure 404 {object} response.Any "项目不存在"
|
||||||
|
// @Failure 500 {object} response.Any "内部错误"
|
||||||
|
// @Router /api/v1/d/pages/{id}/source/delete [post]
|
||||||
|
func DeleteSourceHandler(c *gin.Context) {
|
||||||
|
projectID, ok := apiutil.IDParam(c)
|
||||||
|
if !ok {
|
||||||
|
return
|
||||||
|
}
|
||||||
|
source, err := DeleteSource(c.Request.Context(), projectID)
|
||||||
|
if handleSourceLogicError(c, err) {
|
||||||
|
return
|
||||||
|
}
|
||||||
|
c.JSON(http.StatusOK, response.OK(source))
|
||||||
|
}
|
||||||
|
|
||||||
|
// CheckSourceHandler 请求检查 Pages 部署源。
|
||||||
|
// @Summary 检查 Pages 部署源
|
||||||
|
// @Description 异步检查 GitHub Release 来源;Remote URL 来源不支持检查更新
|
||||||
|
// @Tags openflare-pages
|
||||||
|
// @Produce json
|
||||||
|
// @Security SessionCookie
|
||||||
|
// @Param id path int true "项目 ID"
|
||||||
|
// @Success 200 {object} response.Any{data=pages.SourceActionReceipt} "任务回执"
|
||||||
|
// @Failure 400 {object} response.Any "当前来源不支持检查"
|
||||||
|
// @Failure 401 {object} response.Any "未登录"
|
||||||
|
// @Failure 404 {object} response.Any "部署源不存在"
|
||||||
|
// @Failure 409 {object} response.Any "来源任务正在执行"
|
||||||
|
// @Failure 500 {object} response.Any "内部错误"
|
||||||
|
// @Router /api/v1/d/pages/{id}/source/check [post]
|
||||||
|
func CheckSourceHandler(c *gin.Context) {
|
||||||
|
projectID, ok := apiutil.IDParam(c)
|
||||||
|
if !ok {
|
||||||
|
return
|
||||||
|
}
|
||||||
|
actor, ok := currentPagesActor(c)
|
||||||
|
if !ok {
|
||||||
|
return
|
||||||
|
}
|
||||||
|
receipt, err := DispatchSourceAction(c.Request.Context(), projectID, sourceActionCheck, actor, "")
|
||||||
|
if handleSourceLogicError(c, err) {
|
||||||
|
return
|
||||||
|
}
|
||||||
|
c.JSON(http.StatusOK, response.OK(receipt))
|
||||||
|
}
|
||||||
|
|
||||||
|
// SourceSyncInput is the optional source sync action payload.
|
||||||
|
type SourceSyncInput struct {
|
||||||
|
ConfirmedRevision string `json:"confirmed_revision"`
|
||||||
|
}
|
||||||
|
|
||||||
|
// SyncSourceHandler 请求同步并发布 Pages 部署源。
|
||||||
|
// @Summary 同步并发布 Pages 部署源
|
||||||
|
// @Description 异步下载、校验并原子激活来源部署包;空请求体与空 JSON 对象均有效
|
||||||
|
// @Tags openflare-pages
|
||||||
|
// @Accept json
|
||||||
|
// @Produce json
|
||||||
|
// @Security SessionCookie
|
||||||
|
// @Param id path int true "项目 ID"
|
||||||
|
// @Param request body pages.SourceSyncInput false "同步参数"
|
||||||
|
// @Success 200 {object} response.Any{data=pages.SourceActionReceipt} "任务回执"
|
||||||
|
// @Failure 400 {object} response.Any "参数或来源类型无效"
|
||||||
|
// @Failure 401 {object} response.Any "未登录"
|
||||||
|
// @Failure 404 {object} response.Any "部署源不存在"
|
||||||
|
// @Failure 409 {object} response.Any "来源任务正在执行"
|
||||||
|
// @Failure 500 {object} response.Any "内部错误"
|
||||||
|
// @Router /api/v1/d/pages/{id}/source/sync [post]
|
||||||
|
func SyncSourceHandler(c *gin.Context) {
|
||||||
|
projectID, ok := apiutil.IDParam(c)
|
||||||
|
if !ok {
|
||||||
|
return
|
||||||
|
}
|
||||||
|
var input SourceSyncInput
|
||||||
|
if !decodeStrictJSON(c, &input, true) {
|
||||||
|
return
|
||||||
|
}
|
||||||
|
actor, ok := currentPagesActor(c)
|
||||||
|
if !ok {
|
||||||
|
return
|
||||||
|
}
|
||||||
|
receipt, err := DispatchSourceAction(
|
||||||
|
c.Request.Context(),
|
||||||
|
projectID,
|
||||||
|
sourceActionSync,
|
||||||
|
actor,
|
||||||
|
input.ConfirmedRevision,
|
||||||
|
)
|
||||||
|
if handleSourceLogicError(c, err) {
|
||||||
|
return
|
||||||
|
}
|
||||||
|
c.JSON(http.StatusOK, response.OK(receipt))
|
||||||
|
}
|
||||||
|
|
||||||
// ListDeploymentsHandler 列出项目的全部部署。
|
// ListDeploymentsHandler 列出项目的全部部署。
|
||||||
// @Summary 列出 Pages 部署
|
// @Summary 列出 Pages 部署
|
||||||
// @Description 返回指定项目的全部部署记录,需要管理员权限
|
// @Description 返回指定项目的全部部署记录,需要管理员权限
|
||||||
@@ -214,7 +451,11 @@ func UploadDeploymentHandler(c *gin.Context) {
|
|||||||
response.AbortBadRequest(c, errPagesPackageMissing)
|
response.AbortBadRequest(c, errPagesPackageMissing)
|
||||||
return
|
return
|
||||||
}
|
}
|
||||||
deployment, err := UploadDeployment(c.Request.Context(), id, file, "")
|
actor, ok := currentPagesActor(c)
|
||||||
|
if !ok {
|
||||||
|
return
|
||||||
|
}
|
||||||
|
deployment, err := UploadDeployment(c.Request.Context(), id, file, actor)
|
||||||
if handleLogicError(c, err) {
|
if handleLogicError(c, err) {
|
||||||
return
|
return
|
||||||
}
|
}
|
||||||
@@ -223,7 +464,8 @@ func UploadDeploymentHandler(c *gin.Context) {
|
|||||||
|
|
||||||
// UploadDeploymentFromURLHandler 从 URL 下载并创建 Pages 部署。
|
// UploadDeploymentFromURLHandler 从 URL 下载并创建 Pages 部署。
|
||||||
// @Summary 从 URL 导入 Pages 部署包
|
// @Summary 从 URL 导入 Pages 部署包
|
||||||
// @Description 从用户提供的 HTTP(S) 链接下载部署包并创建部署记录;服务端使用浏览器伪装请求头拉取,允许内网地址与不安全 TLS 证书,需要管理员权限
|
// @Description 已弃用的一次性 URL 导入;使用 trusted_internal 策略兼容内网与自签名证书,不创建持久部署源
|
||||||
|
// @Deprecated
|
||||||
// @Tags openflare-pages
|
// @Tags openflare-pages
|
||||||
// @Accept json
|
// @Accept json
|
||||||
// @Produce json
|
// @Produce json
|
||||||
@@ -247,7 +489,11 @@ func UploadDeploymentFromURLHandler(c *gin.Context) {
|
|||||||
response.AbortBadRequest(c, errPagesPackageURLRequired)
|
response.AbortBadRequest(c, errPagesPackageURLRequired)
|
||||||
return
|
return
|
||||||
}
|
}
|
||||||
deployment, err := UploadDeploymentFromURL(c.Request.Context(), id, req.URL, "")
|
actor, ok := currentPagesActor(c)
|
||||||
|
if !ok {
|
||||||
|
return
|
||||||
|
}
|
||||||
|
deployment, err := UploadDeploymentFromURL(c.Request.Context(), id, req.URL, actor)
|
||||||
if handleLogicError(c, err) {
|
if handleLogicError(c, err) {
|
||||||
return
|
return
|
||||||
}
|
}
|
||||||
@@ -278,7 +524,11 @@ func ActivateDeploymentHandler(c *gin.Context) {
|
|||||||
if !ok {
|
if !ok {
|
||||||
return
|
return
|
||||||
}
|
}
|
||||||
project, err := ActivateDeployment(c.Request.Context(), projectID, deploymentID)
|
actor, ok := currentPagesActor(c)
|
||||||
|
if !ok {
|
||||||
|
return
|
||||||
|
}
|
||||||
|
project, err := ActivateDeploymentAs(c.Request.Context(), projectID, deploymentID, actor)
|
||||||
if handleLogicError(c, err) {
|
if handleLogicError(c, err) {
|
||||||
return
|
return
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -0,0 +1,224 @@
|
|||||||
|
// Copyright 2026 Arctel.net
|
||||||
|
// SPDX-License-Identifier: Apache-2.0
|
||||||
|
|
||||||
|
package pages
|
||||||
|
|
||||||
|
import (
|
||||||
|
"bytes"
|
||||||
|
"encoding/json"
|
||||||
|
"fmt"
|
||||||
|
"net/http"
|
||||||
|
"net/http/httptest"
|
||||||
|
"strings"
|
||||||
|
"testing"
|
||||||
|
"time"
|
||||||
|
|
||||||
|
"github.com/Rain-kl/Wavelet/internal/apps/oauth"
|
||||||
|
"github.com/Rain-kl/Wavelet/internal/db"
|
||||||
|
"github.com/Rain-kl/Wavelet/internal/model"
|
||||||
|
"github.com/Rain-kl/Wavelet/internal/task"
|
||||||
|
"github.com/Rain-kl/Wavelet/internal/testhelper"
|
||||||
|
"github.com/alicebob/miniredis/v2"
|
||||||
|
"github.com/gin-gonic/gin"
|
||||||
|
"github.com/hibiken/asynq"
|
||||||
|
"github.com/redis/go-redis/v9"
|
||||||
|
)
|
||||||
|
|
||||||
|
type sourceHandlerEnvelope struct {
|
||||||
|
ErrorMsg string `json:"error_msg"`
|
||||||
|
Data json.RawMessage `json:"data"`
|
||||||
|
}
|
||||||
|
|
||||||
|
func newPagesSourceTestRouter(userID uint64) *gin.Engine {
|
||||||
|
router := testhelper.NewTestGinEngine(func(ctx *gin.Context) {
|
||||||
|
oauth.SetToContext(ctx, oauth.UserObjKey, &model.User{ID: userID})
|
||||||
|
ctx.Next()
|
||||||
|
})
|
||||||
|
router.GET("/api/v1/d/pages/:id/source", GetSourceHandler)
|
||||||
|
router.POST("/api/v1/d/pages/:id/source/update", UpdateSourceHandler)
|
||||||
|
router.POST("/api/v1/d/pages/:id/source/delete", DeleteSourceHandler)
|
||||||
|
router.POST("/api/v1/d/pages/:id/source/check", CheckSourceHandler)
|
||||||
|
router.POST("/api/v1/d/pages/:id/source/sync", SyncSourceHandler)
|
||||||
|
return router
|
||||||
|
}
|
||||||
|
|
||||||
|
func performPagesSourceRequest(
|
||||||
|
t *testing.T,
|
||||||
|
router http.Handler,
|
||||||
|
method string,
|
||||||
|
path string,
|
||||||
|
body []byte,
|
||||||
|
) (int, sourceHandlerEnvelope) {
|
||||||
|
t.Helper()
|
||||||
|
request := httptest.NewRequest(method, path, bytes.NewReader(body))
|
||||||
|
if body != nil {
|
||||||
|
request.Header.Set("Content-Type", "application/json")
|
||||||
|
}
|
||||||
|
recorder := httptest.NewRecorder()
|
||||||
|
router.ServeHTTP(recorder, request)
|
||||||
|
var envelope sourceHandlerEnvelope
|
||||||
|
if err := json.Unmarshal(recorder.Body.Bytes(), &envelope); err != nil {
|
||||||
|
t.Fatalf("json.Unmarshal(%s %s response %q) error = %v, want nil", method, path, recorder.Body.String(), err)
|
||||||
|
}
|
||||||
|
return recorder.Code, envelope
|
||||||
|
}
|
||||||
|
|
||||||
|
func setupPagesSourceDispatchTest(t *testing.T) {
|
||||||
|
t.Helper()
|
||||||
|
miniRedis, err := miniredis.Run()
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("miniredis.Run() error = %v, want nil", err)
|
||||||
|
}
|
||||||
|
redisClient := redis.NewClient(&redis.Options{Addr: miniRedis.Addr()})
|
||||||
|
asynqClient := asynq.NewClient(asynq.RedisClientOpt{Addr: miniRedis.Addr()})
|
||||||
|
previousRedis := db.Redis
|
||||||
|
previousAsynqClient := task.AsynqClient
|
||||||
|
db.Redis = redisClient
|
||||||
|
task.AsynqClient = asynqClient
|
||||||
|
task.RegisterTaskMeta(PagesSourceActionMeta)
|
||||||
|
t.Cleanup(func() {
|
||||||
|
_ = asynqClient.Close()
|
||||||
|
_ = redisClient.Close()
|
||||||
|
miniRedis.Close()
|
||||||
|
task.AsynqClient = previousAsynqClient
|
||||||
|
db.Redis = previousRedis
|
||||||
|
})
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestPagesSourceHandlersReturnStableActionErrors(t *testing.T) {
|
||||||
|
ctx := setupPagesSourceTest(t)
|
||||||
|
router := newPagesSourceTestRouter(42)
|
||||||
|
|
||||||
|
manualProject := mustCreatePagesSourceProject(t, ctx, "handler-no-source")
|
||||||
|
code, envelope := performPagesSourceRequest(
|
||||||
|
t,
|
||||||
|
router,
|
||||||
|
http.MethodPost,
|
||||||
|
fmt.Sprintf("/api/v1/d/pages/%d/source/sync", manualProject.ID),
|
||||||
|
nil,
|
||||||
|
)
|
||||||
|
if got, want := code, http.StatusNotFound; got != want {
|
||||||
|
t.Errorf("POST source/sync without source status = %d, want %d", got, want)
|
||||||
|
}
|
||||||
|
if got, want := envelope.ErrorMsg, errPagesSourceNotFound; got != want {
|
||||||
|
t.Errorf("POST source/sync without source error = %q, want %q", got, want)
|
||||||
|
}
|
||||||
|
|
||||||
|
remoteProject := mustCreatePagesSourceProject(t, ctx, "handler-check")
|
||||||
|
_, _ = mustConfigureRemoteSource(
|
||||||
|
t,
|
||||||
|
ctx,
|
||||||
|
remoteProject.ID,
|
||||||
|
"https://example.com/site.zip?token=handler-secret",
|
||||||
|
RemoteNetworkPolicyPublic,
|
||||||
|
)
|
||||||
|
code, envelope = performPagesSourceRequest(
|
||||||
|
t,
|
||||||
|
router,
|
||||||
|
http.MethodPost,
|
||||||
|
fmt.Sprintf("/api/v1/d/pages/%d/source/check", remoteProject.ID),
|
||||||
|
nil,
|
||||||
|
)
|
||||||
|
if got, want := code, http.StatusBadRequest; got != want {
|
||||||
|
t.Errorf("POST remote source/check status = %d, want %d", got, want)
|
||||||
|
}
|
||||||
|
if got, want := envelope.ErrorMsg, errPagesSourceCheckUnsupported; got != want {
|
||||||
|
t.Errorf("POST remote source/check error = %q, want %q", got, want)
|
||||||
|
}
|
||||||
|
if strings.Contains(string(envelope.Data), "handler-secret") || strings.Contains(envelope.ErrorMsg, "handler-secret") {
|
||||||
|
t.Errorf("POST remote source/check response = %+v, want no URL secret", envelope)
|
||||||
|
}
|
||||||
|
|
||||||
|
busyProject := mustCreatePagesSourceProject(t, ctx, "handler-busy")
|
||||||
|
busySource, _ := mustConfigureRemoteSource(
|
||||||
|
t,
|
||||||
|
ctx,
|
||||||
|
busyProject.ID,
|
||||||
|
"https://example.com/site.zip",
|
||||||
|
RemoteNetworkPolicyPublic,
|
||||||
|
)
|
||||||
|
future := time.Now().Add(time.Minute)
|
||||||
|
if err := db.DB(ctx).Model(&model.PagesProjectSourceRuntime{}).
|
||||||
|
Where("source_id = ?", busySource.ID).
|
||||||
|
Updates(map[string]any{
|
||||||
|
"sync_status": pagesSourceStatusSyncing,
|
||||||
|
"lease_token": "busy-owner",
|
||||||
|
"lease_expires_at": &future,
|
||||||
|
}).Error; err != nil {
|
||||||
|
t.Fatalf("seed busy source runtime error = %v, want nil", err)
|
||||||
|
}
|
||||||
|
code, envelope = performPagesSourceRequest(
|
||||||
|
t,
|
||||||
|
router,
|
||||||
|
http.MethodPost,
|
||||||
|
fmt.Sprintf("/api/v1/d/pages/%d/source/sync", busyProject.ID),
|
||||||
|
[]byte(`{}`),
|
||||||
|
)
|
||||||
|
if got, want := code, http.StatusConflict; got != want {
|
||||||
|
t.Errorf("POST busy source/sync status = %d, want %d", got, want)
|
||||||
|
}
|
||||||
|
if got, want := envelope.ErrorMsg, errPagesSourceActionBusy; got != want {
|
||||||
|
t.Errorf("POST busy source/sync error = %q, want %q", got, want)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestSyncSourceHandlerAcceptsEmptyBodyAndEmptyObject(t *testing.T) {
|
||||||
|
ctx := setupPagesSourceTest(t)
|
||||||
|
setupPagesSourceDispatchTest(t)
|
||||||
|
router := newPagesSourceTestRouter(77)
|
||||||
|
project := mustCreatePagesSourceProject(t, ctx, "handler-empty-sync")
|
||||||
|
source, _ := mustConfigureRemoteSource(
|
||||||
|
t,
|
||||||
|
ctx,
|
||||||
|
project.ID,
|
||||||
|
"https://example.com/site.zip?token=dispatch-secret",
|
||||||
|
RemoteNetworkPolicyPublic,
|
||||||
|
)
|
||||||
|
path := fmt.Sprintf("/api/v1/d/pages/%d/source/sync", project.ID)
|
||||||
|
|
||||||
|
for _, test := range []struct {
|
||||||
|
name string
|
||||||
|
body []byte
|
||||||
|
}{
|
||||||
|
{name: "empty body", body: nil},
|
||||||
|
{name: "empty object", body: []byte(`{}`)},
|
||||||
|
} {
|
||||||
|
t.Run(test.name, func(t *testing.T) {
|
||||||
|
code, envelope := performPagesSourceRequest(t, router, http.MethodPost, path, test.body)
|
||||||
|
if got, want := code, http.StatusOK; got != want {
|
||||||
|
t.Fatalf("POST source/sync (%s) status = %d, want %d; error=%q", test.name, got, want, envelope.ErrorMsg)
|
||||||
|
}
|
||||||
|
if envelope.ErrorMsg != "" {
|
||||||
|
t.Errorf("POST source/sync (%s) error = %q, want empty", test.name, envelope.ErrorMsg)
|
||||||
|
}
|
||||||
|
var receipt SourceActionReceipt
|
||||||
|
if err := json.Unmarshal(envelope.Data, &receipt); err != nil {
|
||||||
|
t.Fatalf("json.Unmarshal(source/sync %s receipt) error = %v, want nil", test.name, err)
|
||||||
|
}
|
||||||
|
if receipt.TaskID == "" || receipt.ExecutionID == "" || receipt.Action != sourceActionSync {
|
||||||
|
t.Errorf("POST source/sync (%s) receipt = %+v, want task/execution IDs and action %q", test.name, receipt, sourceActionSync)
|
||||||
|
}
|
||||||
|
})
|
||||||
|
}
|
||||||
|
|
||||||
|
var executions []model.TaskExecution
|
||||||
|
if err := db.DB(ctx).Where("task_type = ?", PagesSourceActionTask).Order("id asc").Find(&executions).Error; err != nil {
|
||||||
|
t.Fatalf("list Pages source task executions error = %v, want nil", err)
|
||||||
|
}
|
||||||
|
if got, want := len(executions), 2; got != want {
|
||||||
|
t.Fatalf("Pages source task execution count = %d, want %d", got, want)
|
||||||
|
}
|
||||||
|
for _, execution := range executions {
|
||||||
|
if strings.Contains(execution.Payload, "dispatch-secret") || strings.Contains(execution.Payload, "http") {
|
||||||
|
t.Errorf("task execution %q payload = %s, want no Remote URL secret", execution.TaskID, execution.Payload)
|
||||||
|
}
|
||||||
|
var payload SourceActionPayload
|
||||||
|
if err := json.Unmarshal([]byte(execution.Payload), &payload); err != nil {
|
||||||
|
t.Errorf("json.Unmarshal(task execution %q payload) error = %v, want nil", execution.TaskID, err)
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
if payload.SourceID != source.ID || payload.ConfigVersion != source.ConfigVersion || payload.Actor != "user:77" {
|
||||||
|
t.Errorf("task execution %q payload = %+v, want source=%d config=%d actor=user:77", execution.TaskID, payload, source.ID, source.ConfigVersion)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -0,0 +1,100 @@
|
|||||||
|
// Copyright 2026 Arctel.net
|
||||||
|
// SPDX-License-Identifier: Apache-2.0
|
||||||
|
|
||||||
|
package pages
|
||||||
|
|
||||||
|
import (
|
||||||
|
"bytes"
|
||||||
|
"encoding/json"
|
||||||
|
"fmt"
|
||||||
|
"mime/multipart"
|
||||||
|
"net/http"
|
||||||
|
"net/http/httptest"
|
||||||
|
"strconv"
|
||||||
|
"testing"
|
||||||
|
|
||||||
|
"github.com/Rain-kl/Wavelet/internal/apps/oauth"
|
||||||
|
"github.com/Rain-kl/Wavelet/internal/model"
|
||||||
|
"github.com/gin-gonic/gin"
|
||||||
|
"github.com/stretchr/testify/assert"
|
||||||
|
"github.com/stretchr/testify/require"
|
||||||
|
)
|
||||||
|
|
||||||
|
func TestUploadDeploymentHandlerRecordsCurrentUserActor(t *testing.T) {
|
||||||
|
cleanup := setupPagesTestDB(t)
|
||||||
|
defer cleanup()
|
||||||
|
_, disableStorage := setupPagesStorageMock(t)
|
||||||
|
defer disableStorage()
|
||||||
|
|
||||||
|
project, err := CreateProject(t.Context(), Input{Name: "Actor Upload", Slug: "actor-upload", Enabled: true})
|
||||||
|
require.NoError(t, err)
|
||||||
|
packageBytes := testPagesZip(t, map[string]string{"index.html": "ok"})
|
||||||
|
|
||||||
|
var requestBody bytes.Buffer
|
||||||
|
writer := multipart.NewWriter(&requestBody)
|
||||||
|
part, err := writer.CreateFormFile("package", "site.zip")
|
||||||
|
require.NoError(t, err)
|
||||||
|
_, err = part.Write(packageBytes)
|
||||||
|
require.NoError(t, err)
|
||||||
|
require.NoError(t, writer.Close())
|
||||||
|
|
||||||
|
req := httptest.NewRequest(http.MethodPost, "/api/v1/d/pages/1/deployments/upload", &requestBody)
|
||||||
|
req.Header.Set("Content-Type", writer.FormDataContentType())
|
||||||
|
recorder := httptest.NewRecorder()
|
||||||
|
c, _ := gin.CreateTestContext(recorder)
|
||||||
|
c.Request = req
|
||||||
|
c.Params = gin.Params{{Key: "id", Value: strconv.FormatUint(uint64(project.ID), 10)}}
|
||||||
|
oauth.SetToContext(c, oauth.UserObjKey, &model.User{ID: 42})
|
||||||
|
|
||||||
|
UploadDeploymentHandler(c)
|
||||||
|
assert.Equal(t, http.StatusOK, recorder.Code)
|
||||||
|
deployments, err := model.ListPagesDeployments(t.Context(), project.ID)
|
||||||
|
require.NoError(t, err)
|
||||||
|
require.Len(t, deployments, 1)
|
||||||
|
assert.Equal(t, "user:42", deployments[0].CreatedBy)
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestUploadDeploymentFromURLHandlerRecordsCurrentUserActor(t *testing.T) {
|
||||||
|
cleanup := setupPagesTestDB(t)
|
||||||
|
defer cleanup()
|
||||||
|
_, disableStorage := setupPagesStorageMock(t)
|
||||||
|
defer disableStorage()
|
||||||
|
|
||||||
|
project, err := CreateProject(t.Context(), Input{Name: "Actor URL", Slug: "actor-url", Enabled: true})
|
||||||
|
require.NoError(t, err)
|
||||||
|
packageBytes := testPagesZip(t, map[string]string{"index.html": "ok"})
|
||||||
|
server := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, _ *http.Request) {
|
||||||
|
w.Header().Set("Content-Type", "application/zip")
|
||||||
|
w.Header().Set("Content-Disposition", `attachment; filename="site.zip"`)
|
||||||
|
w.WriteHeader(http.StatusOK)
|
||||||
|
_, _ = w.Write(packageBytes)
|
||||||
|
}))
|
||||||
|
defer server.Close()
|
||||||
|
|
||||||
|
body, err := json.Marshal(UploadFromURLInput{URL: server.URL + "/site.zip"})
|
||||||
|
require.NoError(t, err)
|
||||||
|
req := httptest.NewRequest(http.MethodPost, "/api/v1/d/pages/1/deployments/upload-from-url", bytes.NewReader(body))
|
||||||
|
req.Header.Set("Content-Type", "application/json")
|
||||||
|
recorder := httptest.NewRecorder()
|
||||||
|
c, _ := gin.CreateTestContext(recorder)
|
||||||
|
c.Request = req
|
||||||
|
c.Params = gin.Params{{Key: "id", Value: fmt.Sprint(project.ID)}}
|
||||||
|
oauth.SetToContext(c, oauth.UserObjKey, &model.User{ID: 77})
|
||||||
|
|
||||||
|
UploadDeploymentFromURLHandler(c)
|
||||||
|
assert.Equal(t, http.StatusOK, recorder.Code, recorder.Body.String())
|
||||||
|
deployments, err := model.ListPagesDeployments(t.Context(), project.ID)
|
||||||
|
require.NoError(t, err)
|
||||||
|
require.Len(t, deployments, 1)
|
||||||
|
assert.Equal(t, "user:77", deployments[0].CreatedBy)
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestCurrentPagesActorRejectsMissingUser(t *testing.T) {
|
||||||
|
recorder := httptest.NewRecorder()
|
||||||
|
c, _ := gin.CreateTestContext(recorder)
|
||||||
|
|
||||||
|
actor, ok := currentPagesActor(c)
|
||||||
|
assert.False(t, ok)
|
||||||
|
assert.Empty(t, actor)
|
||||||
|
assert.True(t, c.IsAborted())
|
||||||
|
}
|
||||||
@@ -0,0 +1,570 @@
|
|||||||
|
// Copyright 2026 Arctel.net
|
||||||
|
// SPDX-License-Identifier: Apache-2.0
|
||||||
|
|
||||||
|
package pages
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
"crypto/sha256"
|
||||||
|
"encoding/hex"
|
||||||
|
"encoding/json"
|
||||||
|
"errors"
|
||||||
|
"net"
|
||||||
|
"net/url"
|
||||||
|
"path"
|
||||||
|
"strings"
|
||||||
|
"time"
|
||||||
|
|
||||||
|
"github.com/Rain-kl/Wavelet/internal/db"
|
||||||
|
"github.com/Rain-kl/Wavelet/internal/model"
|
||||||
|
"gorm.io/gorm"
|
||||||
|
"gorm.io/gorm/clause"
|
||||||
|
)
|
||||||
|
|
||||||
|
const (
|
||||||
|
// PagesSourceTypeManual represents projects without a persisted source row.
|
||||||
|
PagesSourceTypeManual = "manual"
|
||||||
|
// PagesSourceTypeRemoteURL represents a persisted artifact URL.
|
||||||
|
PagesSourceTypeRemoteURL = "remote_url"
|
||||||
|
// PagesSourceTypeGitHubRelease represents a public GitHub Release asset.
|
||||||
|
PagesSourceTypeGitHubRelease = "github_release"
|
||||||
|
|
||||||
|
pagesSourceStatusIdle = "idle"
|
||||||
|
pagesSourceStatusChecking = "checking"
|
||||||
|
pagesSourceStatusUpdateAvailable = "update_available"
|
||||||
|
pagesSourceStatusSyncing = "syncing"
|
||||||
|
pagesSourceStatusFailed = "failed"
|
||||||
|
pagesSourceStatusAttention = "attention"
|
||||||
|
|
||||||
|
defaultRemoteAssetLabel = "pages-package"
|
||||||
|
defaultGitHubAssetName = "dist.zip"
|
||||||
|
defaultCheckInterval = 60
|
||||||
|
minimumCheckInterval = 5
|
||||||
|
maximumCheckInterval = 1440
|
||||||
|
)
|
||||||
|
|
||||||
|
// SourceUpdateInput is the discriminated source configuration payload.
|
||||||
|
// GitHub fields are accepted by the decoder so mode-incompatible values can be
|
||||||
|
// rejected deterministically; GitHub itself is enabled in Phase 2.
|
||||||
|
type SourceUpdateInput struct {
|
||||||
|
SourceType string `json:"source_type"`
|
||||||
|
RemoteURLSet bool `json:"remote_url_set"`
|
||||||
|
RemoteURL string `json:"remote_url"`
|
||||||
|
RemoteNetworkPolicy string `json:"remote_network_policy"`
|
||||||
|
RepositoryURL string `json:"repository_url"`
|
||||||
|
ReleaseSelector string `json:"release_selector"`
|
||||||
|
ReleaseTag string `json:"release_tag"`
|
||||||
|
AssetName string `json:"asset_name"`
|
||||||
|
AutoUpdateEnabled bool `json:"auto_update_enabled"`
|
||||||
|
CheckIntervalMinutes int `json:"check_interval_minutes"`
|
||||||
|
}
|
||||||
|
|
||||||
|
// SourceRevisionView is a credential-free source cursor.
|
||||||
|
type SourceRevisionView struct {
|
||||||
|
Revision string `json:"revision"`
|
||||||
|
Label string `json:"label"`
|
||||||
|
AssetName string `json:"asset_name,omitempty"`
|
||||||
|
}
|
||||||
|
|
||||||
|
// SourceView is the safe discriminated source view returned to the console.
|
||||||
|
type SourceView struct {
|
||||||
|
SourceType string `json:"source_type"`
|
||||||
|
HasRemoteURL bool `json:"has_remote_url,omitempty"`
|
||||||
|
DisplayURL string `json:"display_url,omitempty"`
|
||||||
|
RemoteNetworkPolicy string `json:"remote_network_policy,omitempty"`
|
||||||
|
GitHubRepository string `json:"github_repository,omitempty"`
|
||||||
|
ReleaseSelector string `json:"release_selector,omitempty"`
|
||||||
|
ReleaseTag string `json:"release_tag,omitempty"`
|
||||||
|
AssetName string `json:"asset_name,omitempty"`
|
||||||
|
AutoUpdateEnabled *bool `json:"auto_update_enabled,omitempty"`
|
||||||
|
CheckIntervalMinutes int `json:"check_interval_minutes,omitempty"`
|
||||||
|
SyncStatus string `json:"sync_status,omitempty"`
|
||||||
|
UpdateAvailable bool `json:"update_available,omitempty"`
|
||||||
|
LastSeen *SourceRevisionView `json:"last_seen,omitempty"`
|
||||||
|
LastApplied *SourceRevisionView `json:"last_applied,omitempty"`
|
||||||
|
LastCheckedAt *time.Time `json:"last_checked_at,omitempty"`
|
||||||
|
LastSyncedAt *time.Time `json:"last_synced_at,omitempty"`
|
||||||
|
NextCheckAt *time.Time `json:"next_check_at,omitempty"`
|
||||||
|
LastError string `json:"last_error,omitempty"`
|
||||||
|
}
|
||||||
|
|
||||||
|
// SourceActionReceipt identifies the internal task execution created by an action API.
|
||||||
|
type SourceActionReceipt struct {
|
||||||
|
TaskID string `json:"task_id"`
|
||||||
|
ExecutionID string `json:"execution_id"`
|
||||||
|
Action string `json:"action"`
|
||||||
|
}
|
||||||
|
|
||||||
|
// SourceUpdateResult is returned after persisting source configuration.
|
||||||
|
type SourceUpdateResult struct {
|
||||||
|
Source *SourceView `json:"source"`
|
||||||
|
CheckTask *SourceActionReceipt `json:"check_task"`
|
||||||
|
Warning string `json:"warning"`
|
||||||
|
}
|
||||||
|
|
||||||
|
type sourceDetail struct {
|
||||||
|
Provider string `json:"provider"`
|
||||||
|
DisplayName string `json:"display_name,omitempty"`
|
||||||
|
Tag string `json:"tag,omitempty"`
|
||||||
|
LegacyLabel string `json:"label,omitempty"`
|
||||||
|
AssetName string `json:"asset_name,omitempty"`
|
||||||
|
ReleaseID string `json:"release_id,omitempty"`
|
||||||
|
AssetID string `json:"asset_id,omitempty"`
|
||||||
|
AssetUpdatedAt string `json:"asset_updated_at,omitempty"`
|
||||||
|
Digest string `json:"digest,omitempty"`
|
||||||
|
}
|
||||||
|
|
||||||
|
type remoteSourceConfig struct {
|
||||||
|
URL string
|
||||||
|
Policy string
|
||||||
|
Identity string
|
||||||
|
}
|
||||||
|
|
||||||
|
// GetSource returns the current persisted source or a manual discriminator.
|
||||||
|
func GetSource(ctx context.Context, projectID uint) (*SourceView, error) {
|
||||||
|
if _, err := model.GetPagesProjectByID(ctx, projectID); err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
source, runtime, err := loadSourceByProject(ctx, projectID)
|
||||||
|
if errors.Is(err, gorm.ErrRecordNotFound) {
|
||||||
|
return &SourceView{SourceType: PagesSourceTypeManual}, nil
|
||||||
|
}
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
return buildSourceView(source, runtime)
|
||||||
|
}
|
||||||
|
|
||||||
|
// UpdateSource creates or updates a source. Direct callers use the system actor;
|
||||||
|
// HTTP handlers should call UpdateSourceAs so the initial check is auditable.
|
||||||
|
func UpdateSource(ctx context.Context, projectID uint, input SourceUpdateInput) (*SourceUpdateResult, error) {
|
||||||
|
return UpdateSourceAs(ctx, projectID, input, pagesSourceCreatedBySystem)
|
||||||
|
}
|
||||||
|
|
||||||
|
// UpdateSourceAs persists source configuration and queues the first GitHub check
|
||||||
|
// after commit when the GitHub configuration was materially changed.
|
||||||
|
func UpdateSourceAs(
|
||||||
|
ctx context.Context,
|
||||||
|
projectID uint,
|
||||||
|
input SourceUpdateInput,
|
||||||
|
actor string,
|
||||||
|
) (*SourceUpdateResult, error) {
|
||||||
|
if !validPagesSourceActor(actor) {
|
||||||
|
return nil, errors.New(errPagesSourceActionInvalid)
|
||||||
|
}
|
||||||
|
if err := validateSourceUpdateInput(input); err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
|
||||||
|
changed := false
|
||||||
|
var persistedSource model.PagesProjectSource
|
||||||
|
err := db.DB(ctx).Transaction(func(tx *gorm.DB) error {
|
||||||
|
var err error
|
||||||
|
switch strings.TrimSpace(input.SourceType) {
|
||||||
|
case PagesSourceTypeRemoteURL:
|
||||||
|
changed, err = updateRemoteSourceTx(tx, projectID, input)
|
||||||
|
case PagesSourceTypeGitHubRelease:
|
||||||
|
changed, err = updateGitHubSourceTx(tx, projectID, input)
|
||||||
|
default:
|
||||||
|
err = errors.New(errPagesSourceTypeUnsupported)
|
||||||
|
}
|
||||||
|
if err != nil || !changed || strings.TrimSpace(input.SourceType) != PagesSourceTypeGitHubRelease {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
return tx.Where("project_id = ?", projectID).First(&persistedSource).Error
|
||||||
|
})
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
|
||||||
|
view, err := GetSource(ctx, projectID)
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
result := &SourceUpdateResult{Source: view, Warning: ""}
|
||||||
|
if changed && strings.TrimSpace(input.SourceType) == PagesSourceTypeGitHubRelease {
|
||||||
|
receipt, dispatchErr := dispatchSourceActionSnapshot(ctx, persistedSource, sourceActionCheck, actor, "", "", "manual")
|
||||||
|
if dispatchErr != nil {
|
||||||
|
result.Warning = errPagesSourceInitialCheckWarning
|
||||||
|
markInitialCheckDispatchFailed(ctx, persistedSource.ID, persistedSource.ConfigVersion)
|
||||||
|
} else {
|
||||||
|
result.CheckTask = receipt
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return result, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func updateRemoteSourceTx(tx *gorm.DB, projectID uint, input SourceUpdateInput) (bool, error) {
|
||||||
|
var project model.PagesProject
|
||||||
|
if err := tx.Clauses(clause.Locking{Strength: pagesRowLockStrength}).First(&project, projectID).Error; err != nil {
|
||||||
|
return false, err
|
||||||
|
}
|
||||||
|
existing, hasExisting, err := loadProjectSourceForUpdate(tx, projectID)
|
||||||
|
if err != nil {
|
||||||
|
return false, err
|
||||||
|
}
|
||||||
|
config, err := buildRemoteSourceConfig(existing, hasExisting, input)
|
||||||
|
if err != nil {
|
||||||
|
return false, err
|
||||||
|
}
|
||||||
|
if !hasExisting {
|
||||||
|
return true, createRemoteSourceTx(tx, projectID, config)
|
||||||
|
}
|
||||||
|
return updateExistingRemoteSourceTx(tx, existing, config)
|
||||||
|
}
|
||||||
|
|
||||||
|
func loadProjectSourceForUpdate(tx *gorm.DB, projectID uint) (*model.PagesProjectSource, bool, error) {
|
||||||
|
var source model.PagesProjectSource
|
||||||
|
err := tx.Clauses(clause.Locking{Strength: pagesRowLockStrength}).
|
||||||
|
Where("project_id = ?", projectID).
|
||||||
|
First(&source).Error
|
||||||
|
if errors.Is(err, gorm.ErrRecordNotFound) {
|
||||||
|
return &source, false, nil
|
||||||
|
}
|
||||||
|
if err != nil {
|
||||||
|
return nil, false, err
|
||||||
|
}
|
||||||
|
return &source, true, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func buildRemoteSourceConfig(
|
||||||
|
existing *model.PagesProjectSource,
|
||||||
|
hasExisting bool,
|
||||||
|
input SourceUpdateInput,
|
||||||
|
) (remoteSourceConfig, error) {
|
||||||
|
remoteURL, err := resolveUpdatedRemoteURL(existing, hasExisting, input)
|
||||||
|
if err != nil {
|
||||||
|
return remoteSourceConfig{}, err
|
||||||
|
}
|
||||||
|
parsedURL, err := parseRemoteSourceURL(remoteURL)
|
||||||
|
if err != nil {
|
||||||
|
return remoteSourceConfig{}, err
|
||||||
|
}
|
||||||
|
policy := strings.TrimSpace(input.RemoteNetworkPolicy)
|
||||||
|
if policy == "" {
|
||||||
|
policy = RemoteNetworkPolicyPublic
|
||||||
|
}
|
||||||
|
return remoteSourceConfig{
|
||||||
|
URL: remoteURL,
|
||||||
|
Policy: policy,
|
||||||
|
Identity: remoteSourceIdentity(parsedURL),
|
||||||
|
}, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func createRemoteSourceTx(tx *gorm.DB, projectID uint, config remoteSourceConfig) error {
|
||||||
|
source := &model.PagesProjectSource{
|
||||||
|
ProjectID: projectID,
|
||||||
|
SourceType: PagesSourceTypeRemoteURL,
|
||||||
|
RemoteURL: config.URL,
|
||||||
|
RemoteNetworkPolicy: config.Policy,
|
||||||
|
AutoUpdateEnabled: false,
|
||||||
|
CheckIntervalMinutes: 0,
|
||||||
|
ConfigVersion: 1,
|
||||||
|
SourceIdentity: config.Identity,
|
||||||
|
}
|
||||||
|
if err := tx.Create(source).Error; err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
return tx.Create(&model.PagesProjectSourceRuntime{
|
||||||
|
SourceID: source.ID,
|
||||||
|
SyncStatus: pagesSourceStatusIdle,
|
||||||
|
}).Error
|
||||||
|
}
|
||||||
|
|
||||||
|
func updateExistingRemoteSourceTx(
|
||||||
|
tx *gorm.DB,
|
||||||
|
existing *model.PagesProjectSource,
|
||||||
|
config remoteSourceConfig,
|
||||||
|
) (bool, error) {
|
||||||
|
if !remoteSourceConfigChanged(existing, config) {
|
||||||
|
return false, nil
|
||||||
|
}
|
||||||
|
var runtime model.PagesProjectSourceRuntime
|
||||||
|
if err := tx.Clauses(clause.Locking{Strength: pagesRowLockStrength}).
|
||||||
|
Where("source_id = ?", existing.ID).
|
||||||
|
First(&runtime).Error; err != nil {
|
||||||
|
return false, err
|
||||||
|
}
|
||||||
|
identityChanged := existing.SourceIdentity != config.Identity
|
||||||
|
if err := tx.Model(existing).Updates(map[string]any{
|
||||||
|
"source_type": PagesSourceTypeRemoteURL,
|
||||||
|
"remote_url": config.URL,
|
||||||
|
"remote_network_policy": config.Policy,
|
||||||
|
"github_repository": "",
|
||||||
|
"release_selector": "",
|
||||||
|
"release_tag": "",
|
||||||
|
"asset_name": "",
|
||||||
|
sourceColumnAutoUpdateEnabled: false,
|
||||||
|
"check_interval_minutes": 0,
|
||||||
|
sourceColumnConfigVersion: existing.ConfigVersion + 1,
|
||||||
|
"source_identity": config.Identity,
|
||||||
|
}).Error; err != nil {
|
||||||
|
return false, err
|
||||||
|
}
|
||||||
|
return true, resetRuntimeAfterSourceUpdate(tx, &runtime, identityChanged)
|
||||||
|
}
|
||||||
|
|
||||||
|
func remoteSourceConfigChanged(existing *model.PagesProjectSource, config remoteSourceConfig) bool {
|
||||||
|
return existing.SourceType != PagesSourceTypeRemoteURL ||
|
||||||
|
existing.RemoteURL != config.URL ||
|
||||||
|
existing.RemoteNetworkPolicy != config.Policy ||
|
||||||
|
existing.GitHubRepository != "" ||
|
||||||
|
existing.ReleaseSelector != "" ||
|
||||||
|
existing.ReleaseTag != "" ||
|
||||||
|
existing.AssetName != "" ||
|
||||||
|
existing.AutoUpdateEnabled ||
|
||||||
|
existing.CheckIntervalMinutes != 0
|
||||||
|
}
|
||||||
|
|
||||||
|
// DeleteSource idempotently switches a project back to manual mode.
|
||||||
|
func DeleteSource(ctx context.Context, projectID uint) (*SourceView, error) {
|
||||||
|
err := db.DB(ctx).Transaction(func(tx *gorm.DB) error {
|
||||||
|
var project model.PagesProject
|
||||||
|
if err := tx.Clauses(clause.Locking{Strength: pagesRowLockStrength}).First(&project, projectID).Error; err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
var source model.PagesProjectSource
|
||||||
|
err := tx.Clauses(clause.Locking{Strength: pagesRowLockStrength}).
|
||||||
|
Where("project_id = ?", projectID).
|
||||||
|
First(&source).Error
|
||||||
|
if errors.Is(err, gorm.ErrRecordNotFound) {
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
if err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
var runtime model.PagesProjectSourceRuntime
|
||||||
|
if err := tx.Clauses(clause.Locking{Strength: pagesRowLockStrength}).
|
||||||
|
Where("source_id = ?", source.ID).
|
||||||
|
First(&runtime).Error; err != nil && !errors.Is(err, gorm.ErrRecordNotFound) {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
if err := tx.Where("source_id = ?", source.ID).Delete(&model.PagesProjectSourceRuntime{}).Error; err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
return tx.Delete(&source).Error
|
||||||
|
})
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
return &SourceView{SourceType: PagesSourceTypeManual}, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func validateRemoteSourceInput(input SourceUpdateInput) error {
|
||||||
|
sourceType := strings.TrimSpace(input.SourceType)
|
||||||
|
if sourceType == "" {
|
||||||
|
return errors.New(errPagesSourceTypeRequired)
|
||||||
|
}
|
||||||
|
if sourceType != PagesSourceTypeRemoteURL {
|
||||||
|
return errors.New(errPagesSourceTypeUnsupported)
|
||||||
|
}
|
||||||
|
if strings.TrimSpace(input.RepositoryURL) != "" || strings.TrimSpace(input.ReleaseSelector) != "" ||
|
||||||
|
strings.TrimSpace(input.ReleaseTag) != "" || strings.TrimSpace(input.AssetName) != "" ||
|
||||||
|
input.AutoUpdateEnabled || input.CheckIntervalMinutes != 0 {
|
||||||
|
return errors.New(errPagesSourceRemoteFields)
|
||||||
|
}
|
||||||
|
policy := strings.TrimSpace(input.RemoteNetworkPolicy)
|
||||||
|
if policy != "" && policy != RemoteNetworkPolicyPublic && policy != RemoteNetworkPolicyTrustedInternal {
|
||||||
|
return errors.New(errPagesSourceNetworkPolicy)
|
||||||
|
}
|
||||||
|
if !input.RemoteURLSet && strings.TrimSpace(input.RemoteURL) != "" {
|
||||||
|
return errors.New(errPagesSourceRemoteURLMode)
|
||||||
|
}
|
||||||
|
if input.RemoteURLSet && strings.TrimSpace(input.RemoteURL) == "" {
|
||||||
|
return errors.New(errPagesSourceRemoteURLRequired)
|
||||||
|
}
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func validateSourceUpdateInput(input SourceUpdateInput) error {
|
||||||
|
switch strings.TrimSpace(input.SourceType) {
|
||||||
|
case PagesSourceTypeRemoteURL:
|
||||||
|
return validateRemoteSourceInput(input)
|
||||||
|
case PagesSourceTypeGitHubRelease:
|
||||||
|
return validateGitHubSourceInput(input)
|
||||||
|
case "":
|
||||||
|
return errors.New(errPagesSourceTypeRequired)
|
||||||
|
default:
|
||||||
|
return errors.New(errPagesSourceTypeUnsupported)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func resolveUpdatedRemoteURL(existing *model.PagesProjectSource, hasExisting bool, input SourceUpdateInput) (string, error) {
|
||||||
|
if input.RemoteURLSet {
|
||||||
|
return strings.TrimSpace(input.RemoteURL), nil
|
||||||
|
}
|
||||||
|
if !hasExisting || existing.SourceType != PagesSourceTypeRemoteURL || strings.TrimSpace(existing.RemoteURL) == "" {
|
||||||
|
return "", errors.New(errPagesSourceRemoteURLRequired)
|
||||||
|
}
|
||||||
|
return existing.RemoteURL, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func parseRemoteSourceURL(raw string) (*url.URL, error) {
|
||||||
|
parsed, err := url.Parse(strings.TrimSpace(raw))
|
||||||
|
if err != nil || parsed.Host == "" || parsed.User != nil || parsed.Fragment != "" {
|
||||||
|
return nil, errors.New(errPagesSourceRemoteURLInvalid)
|
||||||
|
}
|
||||||
|
parsed.Scheme = strings.ToLower(parsed.Scheme)
|
||||||
|
if parsed.Scheme != remoteSourceSchemeHTTP && parsed.Scheme != remoteSourceSchemeHTTPS {
|
||||||
|
return nil, errors.New(errPagesSourceRemoteURLInvalid)
|
||||||
|
}
|
||||||
|
if strings.TrimSpace(parsed.Hostname()) == "" {
|
||||||
|
return nil, errors.New(errPagesSourceRemoteURLInvalid)
|
||||||
|
}
|
||||||
|
return parsed, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func remoteSourceIdentity(parsed *url.URL) string {
|
||||||
|
hostname := strings.ToLower(parsed.Hostname())
|
||||||
|
port := parsed.Port()
|
||||||
|
if (parsed.Scheme == "https" && port == "443") || (parsed.Scheme == "http" && port == "80") {
|
||||||
|
port = ""
|
||||||
|
}
|
||||||
|
host := hostname
|
||||||
|
if port != "" {
|
||||||
|
host = net.JoinHostPort(hostname, port)
|
||||||
|
} else if strings.Contains(hostname, ":") {
|
||||||
|
host = "[" + hostname + "]"
|
||||||
|
}
|
||||||
|
canonicalPath := parsed.EscapedPath()
|
||||||
|
if canonicalPath == "" {
|
||||||
|
canonicalPath = "/"
|
||||||
|
}
|
||||||
|
canonicalPath = path.Clean("/" + strings.TrimPrefix(canonicalPath, "/"))
|
||||||
|
canonical := parsed.Scheme + "://" + host + canonicalPath
|
||||||
|
sum := sha256.Sum256([]byte("remote_url|" + canonical))
|
||||||
|
return hex.EncodeToString(sum[:])
|
||||||
|
}
|
||||||
|
|
||||||
|
func displayRemoteSourceURL(raw string) string {
|
||||||
|
parsed, err := parseRemoteSourceURL(raw)
|
||||||
|
if err != nil {
|
||||||
|
return ""
|
||||||
|
}
|
||||||
|
hadQuery := parsed.RawQuery != ""
|
||||||
|
parsed.RawQuery = ""
|
||||||
|
display := parsed.String()
|
||||||
|
if hadQuery {
|
||||||
|
display += "?***"
|
||||||
|
}
|
||||||
|
return display
|
||||||
|
}
|
||||||
|
|
||||||
|
func loadSourceByProject(ctx context.Context, projectID uint) (*model.PagesProjectSource, *model.PagesProjectSourceRuntime, error) {
|
||||||
|
var source model.PagesProjectSource
|
||||||
|
if err := db.DB(ctx).Where("project_id = ?", projectID).First(&source).Error; err != nil {
|
||||||
|
return nil, nil, err
|
||||||
|
}
|
||||||
|
var runtime model.PagesProjectSourceRuntime
|
||||||
|
if err := db.DB(ctx).Where("source_id = ?", source.ID).First(&runtime).Error; err != nil {
|
||||||
|
return nil, nil, err
|
||||||
|
}
|
||||||
|
return &source, &runtime, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func buildSourceView(source *model.PagesProjectSource, runtime *model.PagesProjectSourceRuntime) (*SourceView, error) {
|
||||||
|
if source == nil || runtime == nil {
|
||||||
|
return nil, errors.New(errPagesSourceNotFound)
|
||||||
|
}
|
||||||
|
view := &SourceView{
|
||||||
|
SourceType: source.SourceType,
|
||||||
|
SyncStatus: runtime.SyncStatus,
|
||||||
|
LastSyncedAt: runtime.LastSyncedAt,
|
||||||
|
LastError: runtime.LastError,
|
||||||
|
}
|
||||||
|
if runtime.LastAppliedRevision != "" {
|
||||||
|
view.LastApplied = revisionView(runtime.LastAppliedRevision, runtime.LastAppliedDetail)
|
||||||
|
}
|
||||||
|
switch source.SourceType {
|
||||||
|
case PagesSourceTypeRemoteURL:
|
||||||
|
view.HasRemoteURL = strings.TrimSpace(source.RemoteURL) != ""
|
||||||
|
view.DisplayURL = displayRemoteSourceURL(source.RemoteURL)
|
||||||
|
view.RemoteNetworkPolicy = source.RemoteNetworkPolicy
|
||||||
|
case PagesSourceTypeGitHubRelease:
|
||||||
|
view.LastCheckedAt = runtime.LastCheckedAt
|
||||||
|
view.NextCheckAt = runtime.NextCheckAt
|
||||||
|
view.UpdateAvailable = runtime.LastSeenRevision != "" && runtime.LastSeenRevision != runtime.LastAppliedRevision
|
||||||
|
if runtime.LastSeenRevision != "" {
|
||||||
|
view.LastSeen = revisionView(runtime.LastSeenRevision, runtime.LastSeenDetail)
|
||||||
|
}
|
||||||
|
view.GitHubRepository = source.GitHubRepository
|
||||||
|
view.ReleaseSelector = source.ReleaseSelector
|
||||||
|
view.ReleaseTag = source.ReleaseTag
|
||||||
|
view.AssetName = source.AssetName
|
||||||
|
autoUpdateEnabled := source.AutoUpdateEnabled
|
||||||
|
view.AutoUpdateEnabled = &autoUpdateEnabled
|
||||||
|
view.CheckIntervalMinutes = source.CheckIntervalMinutes
|
||||||
|
default:
|
||||||
|
return nil, errors.New(errPagesSourceTypeUnsupported)
|
||||||
|
}
|
||||||
|
return view, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func revisionView(revision string, detailJSON string) *SourceRevisionView {
|
||||||
|
detail := sourceDetail{}
|
||||||
|
_ = unmarshalSourceDetail(detailJSON, &detail)
|
||||||
|
label := sourceDetailLabel(detail)
|
||||||
|
if label == "" {
|
||||||
|
label = defaultRemoteAssetLabel
|
||||||
|
}
|
||||||
|
return &SourceRevisionView{
|
||||||
|
Revision: revision,
|
||||||
|
Label: label,
|
||||||
|
AssetName: detail.AssetName,
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func sourceDetailLabel(detail sourceDetail) string {
|
||||||
|
if detail.Provider == githubSourceDetailProvider {
|
||||||
|
if label := strings.TrimSpace(detail.Tag); label != "" {
|
||||||
|
return label
|
||||||
|
}
|
||||||
|
return strings.TrimSpace(detail.LegacyLabel)
|
||||||
|
}
|
||||||
|
if label := strings.TrimSpace(detail.DisplayName); label != "" {
|
||||||
|
return label
|
||||||
|
}
|
||||||
|
return strings.TrimSpace(detail.LegacyLabel)
|
||||||
|
}
|
||||||
|
|
||||||
|
func unmarshalSourceDetail(raw string, detail *sourceDetail) error {
|
||||||
|
if detail == nil || strings.TrimSpace(raw) == "" {
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
return json.Unmarshal([]byte(raw), detail)
|
||||||
|
}
|
||||||
|
|
||||||
|
func resetRuntimeAfterSourceUpdate(tx *gorm.DB, runtime *model.PagesProjectSourceRuntime, identityChanged bool) error {
|
||||||
|
updates := map[string]any{
|
||||||
|
sourceRuntimeColumnLeaseToken: "",
|
||||||
|
sourceRuntimeColumnLeaseExpiresAt: nil,
|
||||||
|
sourceRuntimeColumnLastError: "",
|
||||||
|
}
|
||||||
|
if identityChanged {
|
||||||
|
updates["etag"] = ""
|
||||||
|
updates["last_seen_revision"] = ""
|
||||||
|
updates["last_seen_detail"] = ""
|
||||||
|
updates["last_applied_revision"] = ""
|
||||||
|
updates["last_applied_detail"] = ""
|
||||||
|
updates["last_checked_at"] = nil
|
||||||
|
updates["last_synced_at"] = nil
|
||||||
|
updates["next_check_at"] = nil
|
||||||
|
updates[sourceRuntimeColumnSyncStatus] = pagesSourceStatusIdle
|
||||||
|
} else {
|
||||||
|
updates[sourceRuntimeColumnSyncStatus] = normalizedSourceRuntimeStatus(runtime)
|
||||||
|
}
|
||||||
|
return tx.Model(runtime).Updates(updates).Error
|
||||||
|
}
|
||||||
|
|
||||||
|
func normalizedSourceRuntimeStatus(runtime *model.PagesProjectSourceRuntime) string {
|
||||||
|
if runtime == nil {
|
||||||
|
return pagesSourceStatusIdle
|
||||||
|
}
|
||||||
|
if sourceHasSameReleaseReplacement(runtime) {
|
||||||
|
return pagesSourceStatusAttention
|
||||||
|
}
|
||||||
|
if runtime.LastSeenRevision != "" && runtime.LastSeenRevision != runtime.LastAppliedRevision {
|
||||||
|
return pagesSourceStatusUpdateAvailable
|
||||||
|
}
|
||||||
|
return pagesSourceStatusIdle
|
||||||
|
}
|
||||||
@@ -0,0 +1,51 @@
|
|||||||
|
// Copyright 2026 Arctel.net
|
||||||
|
// SPDX-License-Identifier: Apache-2.0
|
||||||
|
|
||||||
|
package pages
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
"time"
|
||||||
|
|
||||||
|
"github.com/Rain-kl/Wavelet/internal/apps/upload"
|
||||||
|
"github.com/Rain-kl/Wavelet/internal/model"
|
||||||
|
)
|
||||||
|
|
||||||
|
func syncRemoteSource(
|
||||||
|
ctx context.Context,
|
||||||
|
snapshot *sourceExecutionSnapshot,
|
||||||
|
actor string,
|
||||||
|
) (*sourceSyncOutcome, error) {
|
||||||
|
return syncRemoteSourceWithTrigger(ctx, snapshot, actor, pagesSourceTriggerManualSync)
|
||||||
|
}
|
||||||
|
|
||||||
|
func syncGitHubSource(
|
||||||
|
ctx context.Context,
|
||||||
|
snapshot *sourceExecutionSnapshot,
|
||||||
|
actor string,
|
||||||
|
targetRevision string,
|
||||||
|
confirmedRevision string,
|
||||||
|
) (*sourceSyncOutcome, error) {
|
||||||
|
return syncGitHubSourceWithTrigger(
|
||||||
|
ctx, snapshot, actor, targetRevision, confirmedRevision, pagesSourceTriggerManualSync,
|
||||||
|
)
|
||||||
|
}
|
||||||
|
|
||||||
|
func commitSourceDeployment(
|
||||||
|
ctx context.Context,
|
||||||
|
snapshot *sourceExecutionSnapshot,
|
||||||
|
revision string,
|
||||||
|
packageChecksum string,
|
||||||
|
detail sourceDetail,
|
||||||
|
detailJSON string,
|
||||||
|
actor string,
|
||||||
|
manifest *deploymentManifest,
|
||||||
|
ingestResult upload.IngestResult,
|
||||||
|
hasIngest bool,
|
||||||
|
nextCheckNotBefore *time.Time,
|
||||||
|
) (*model.PagesDeployment, bool, bool, error) {
|
||||||
|
return commitSourceDeploymentWithTrigger(
|
||||||
|
ctx, snapshot, revision, packageChecksum, detail, detailJSON, actor,
|
||||||
|
pagesSourceTriggerManualSync, manifest, ingestResult, hasIngest, nextCheckNotBefore,
|
||||||
|
)
|
||||||
|
}
|
||||||
@@ -0,0 +1,328 @@
|
|||||||
|
// Copyright 2026 Arctel.net
|
||||||
|
// SPDX-License-Identifier: Apache-2.0
|
||||||
|
|
||||||
|
package pages
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
"errors"
|
||||||
|
"fmt"
|
||||||
|
"strconv"
|
||||||
|
"time"
|
||||||
|
|
||||||
|
"github.com/Rain-kl/Wavelet/internal/apps/upload"
|
||||||
|
"github.com/Rain-kl/Wavelet/internal/db"
|
||||||
|
"github.com/Rain-kl/Wavelet/internal/model"
|
||||||
|
"github.com/Rain-kl/Wavelet/internal/repository"
|
||||||
|
"github.com/Rain-kl/Wavelet/pkg/logger"
|
||||||
|
"gorm.io/gorm"
|
||||||
|
"gorm.io/gorm/clause"
|
||||||
|
)
|
||||||
|
|
||||||
|
const pagesOrphanUploadIsolation = 2 * time.Hour
|
||||||
|
|
||||||
|
// PagesOrphanCleanupSummary describes one bounded delayed compensation pass.
|
||||||
|
// Every candidate is counted exactly once in one outcome field.
|
||||||
|
//
|
||||||
|
//nolint:revive // Keep the domain-qualified exported name for scanner/task result clarity.
|
||||||
|
type PagesOrphanCleanupSummary struct {
|
||||||
|
Candidates int `json:"candidates"`
|
||||||
|
Reconciled int `json:"reconciled"`
|
||||||
|
Referenced int `json:"referenced"`
|
||||||
|
LeaseBusy int `json:"lease_busy"`
|
||||||
|
InvalidMarker int `json:"invalid_marker"`
|
||||||
|
Skipped int `json:"skipped"`
|
||||||
|
Failed int `json:"failed"`
|
||||||
|
}
|
||||||
|
|
||||||
|
type pagesOrphanMarker struct {
|
||||||
|
ProjectID uint
|
||||||
|
SourceID *uint
|
||||||
|
}
|
||||||
|
|
||||||
|
type pagesOrphanCleanupOutcome uint8
|
||||||
|
|
||||||
|
const (
|
||||||
|
pagesOrphanCleanupSkipped pagesOrphanCleanupOutcome = iota
|
||||||
|
pagesOrphanCleanupReconciled
|
||||||
|
pagesOrphanCleanupReferenced
|
||||||
|
pagesOrphanCleanupLeaseBusy
|
||||||
|
pagesOrphanCleanupInvalidMarker
|
||||||
|
)
|
||||||
|
|
||||||
|
// ReconcilePagesOrphanUploads performs one bounded delayed compensation pass.
|
||||||
|
// Individual candidate failures are counted and logged so they do not prevent
|
||||||
|
// the scanner from continuing with source checks.
|
||||||
|
func ReconcilePagesOrphanUploads(
|
||||||
|
ctx context.Context,
|
||||||
|
now time.Time,
|
||||||
|
) (PagesOrphanCleanupSummary, error) {
|
||||||
|
if now.IsZero() {
|
||||||
|
now = time.Now()
|
||||||
|
}
|
||||||
|
cutoff := now.UTC().Add(-pagesOrphanUploadIsolation)
|
||||||
|
systemUser := repository.GetSystemUser(ctx)
|
||||||
|
candidates, err := model.ListPagesOrphanUploadCandidates(ctx, model.PagesOrphanUploadCandidateQuery{
|
||||||
|
SystemUserID: systemUser.ID,
|
||||||
|
UploadType: upload.ReservedPagesDeploymentType,
|
||||||
|
Marker: pagesIngestMarkerV2,
|
||||||
|
CreatedBefore: cutoff,
|
||||||
|
})
|
||||||
|
if err != nil {
|
||||||
|
return PagesOrphanCleanupSummary{}, err
|
||||||
|
}
|
||||||
|
|
||||||
|
summary := PagesOrphanCleanupSummary{Candidates: len(candidates)}
|
||||||
|
for index := range candidates {
|
||||||
|
if err := ctx.Err(); err != nil {
|
||||||
|
return summary, err
|
||||||
|
}
|
||||||
|
candidate := &candidates[index]
|
||||||
|
marker, err := parsePagesOrphanMarker(candidate.Metadata)
|
||||||
|
if err != nil {
|
||||||
|
summary.InvalidMarker++
|
||||||
|
logger.WarnF(ctx, "[PagesSource] orphan upload marker invalid: upload_id=%d error=%v", candidate.ID, err)
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
|
||||||
|
outcome, err := reconcilePagesOrphanUploadCandidate(
|
||||||
|
ctx,
|
||||||
|
candidate,
|
||||||
|
marker,
|
||||||
|
systemUser.ID,
|
||||||
|
cutoff,
|
||||||
|
)
|
||||||
|
if err != nil {
|
||||||
|
summary.Failed++
|
||||||
|
logger.WarnF(ctx, "[PagesSource] orphan upload reconciliation failed: upload_id=%d error=%v", candidate.ID, err)
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
summary.add(outcome)
|
||||||
|
}
|
||||||
|
return summary, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func (summary *PagesOrphanCleanupSummary) add(outcome pagesOrphanCleanupOutcome) {
|
||||||
|
switch outcome {
|
||||||
|
case pagesOrphanCleanupReconciled:
|
||||||
|
summary.Reconciled++
|
||||||
|
case pagesOrphanCleanupReferenced:
|
||||||
|
summary.Referenced++
|
||||||
|
case pagesOrphanCleanupLeaseBusy:
|
||||||
|
summary.LeaseBusy++
|
||||||
|
case pagesOrphanCleanupInvalidMarker:
|
||||||
|
summary.InvalidMarker++
|
||||||
|
default:
|
||||||
|
summary.Skipped++
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func reconcilePagesOrphanUploadCandidate(
|
||||||
|
ctx context.Context,
|
||||||
|
candidate *model.Upload,
|
||||||
|
marker pagesOrphanMarker,
|
||||||
|
systemUserID uint64,
|
||||||
|
cutoff time.Time,
|
||||||
|
) (pagesOrphanCleanupOutcome, error) {
|
||||||
|
if candidate == nil || candidate.ID == 0 {
|
||||||
|
return pagesOrphanCleanupSkipped, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
outcome := pagesOrphanCleanupSkipped
|
||||||
|
uploadLocked := false
|
||||||
|
err := db.DB(ctx).Transaction(func(tx *gorm.DB) error {
|
||||||
|
scopeOutcome, proceed, err := lockPagesOrphanCleanupScope(ctx, tx, candidate.ID, marker)
|
||||||
|
if err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
if !proceed {
|
||||||
|
outcome = scopeOutcome
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
lockedOutcome, locked, err := reconcileLockedPagesOrphanUpload(
|
||||||
|
ctx,
|
||||||
|
tx,
|
||||||
|
candidate.ID,
|
||||||
|
marker,
|
||||||
|
systemUserID,
|
||||||
|
cutoff,
|
||||||
|
)
|
||||||
|
if err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
outcome = lockedOutcome
|
||||||
|
uploadLocked = locked
|
||||||
|
return nil
|
||||||
|
})
|
||||||
|
if err != nil {
|
||||||
|
return pagesOrphanCleanupSkipped, err
|
||||||
|
}
|
||||||
|
if uploadLocked {
|
||||||
|
// Also heal a prior post-commit cache invalidation interruption when the
|
||||||
|
// status transition was an idempotent no-op.
|
||||||
|
upload.InvalidateUploadMetaCache(ctx, candidate.ID)
|
||||||
|
}
|
||||||
|
return outcome, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func lockPagesOrphanCleanupScope(
|
||||||
|
ctx context.Context,
|
||||||
|
tx *gorm.DB,
|
||||||
|
uploadID uint64,
|
||||||
|
marker pagesOrphanMarker,
|
||||||
|
) (pagesOrphanCleanupOutcome, bool, error) {
|
||||||
|
var project model.PagesProject
|
||||||
|
if _, err := lockOptionalPagesCleanupRecord(tx, &project, "id = ?", marker.ProjectID); err != nil {
|
||||||
|
return pagesOrphanCleanupSkipped, false, err
|
||||||
|
}
|
||||||
|
if marker.SourceID == nil {
|
||||||
|
return pagesOrphanCleanupSkipped, true, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
var source model.PagesProjectSource
|
||||||
|
sourceExists, err := lockOptionalPagesCleanupRecord(tx, &source, "id = ?", *marker.SourceID)
|
||||||
|
if err != nil {
|
||||||
|
return pagesOrphanCleanupSkipped, false, err
|
||||||
|
}
|
||||||
|
if !sourceExists {
|
||||||
|
return pagesOrphanCleanupSkipped, true, nil
|
||||||
|
}
|
||||||
|
if source.ProjectID != marker.ProjectID {
|
||||||
|
logger.WarnF(ctx,
|
||||||
|
"[PagesSource] orphan upload source ownership mismatch: upload_id=%d project_id=%d source_id=%d source_project_id=%d",
|
||||||
|
uploadID,
|
||||||
|
marker.ProjectID,
|
||||||
|
*marker.SourceID,
|
||||||
|
source.ProjectID,
|
||||||
|
)
|
||||||
|
return pagesOrphanCleanupInvalidMarker, false, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
var runtime model.PagesProjectSourceRuntime
|
||||||
|
runtimeExists, err := lockOptionalPagesCleanupRecord(tx, &runtime, "source_id = ?", source.ID)
|
||||||
|
if err != nil {
|
||||||
|
return pagesOrphanCleanupSkipped, false, err
|
||||||
|
}
|
||||||
|
// Read the real clock only after obtaining the runtime row lock. The scanner
|
||||||
|
// snapshot time is only an isolation cutoff and may be stale after lock wait.
|
||||||
|
leaseCheckedAt := time.Now()
|
||||||
|
if runtimeExists && runtime.LeaseExpiresAt != nil && runtime.LeaseExpiresAt.After(leaseCheckedAt) {
|
||||||
|
return pagesOrphanCleanupLeaseBusy, false, nil
|
||||||
|
}
|
||||||
|
return pagesOrphanCleanupSkipped, true, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func reconcileLockedPagesOrphanUpload(
|
||||||
|
ctx context.Context,
|
||||||
|
tx *gorm.DB,
|
||||||
|
uploadID uint64,
|
||||||
|
marker pagesOrphanMarker,
|
||||||
|
systemUserID uint64,
|
||||||
|
cutoff time.Time,
|
||||||
|
) (pagesOrphanCleanupOutcome, bool, error) {
|
||||||
|
var lockedUpload model.Upload
|
||||||
|
found, err := lockOptionalPagesCleanupRecord(tx, &lockedUpload, "id = ?", uploadID)
|
||||||
|
if err != nil || !found {
|
||||||
|
return pagesOrphanCleanupSkipped, false, err
|
||||||
|
}
|
||||||
|
|
||||||
|
lockedMarker, err := parsePagesOrphanMarker(lockedUpload.Metadata)
|
||||||
|
if err != nil {
|
||||||
|
logger.WarnF(ctx, "[PagesSource] orphan upload marker changed or invalid: upload_id=%d error=%v", uploadID, err)
|
||||||
|
return pagesOrphanCleanupInvalidMarker, true, nil
|
||||||
|
}
|
||||||
|
if lockedUpload.Status != model.UploadStatusUsed ||
|
||||||
|
lockedUpload.UserID != systemUserID ||
|
||||||
|
lockedUpload.Type != upload.ReservedPagesDeploymentType ||
|
||||||
|
!lockedUpload.CreatedAt.Before(cutoff) {
|
||||||
|
return pagesOrphanCleanupSkipped, true, nil
|
||||||
|
}
|
||||||
|
if lockedMarker.ProjectID != marker.ProjectID || !sameOptionalPagesSourceID(lockedMarker.SourceID, marker.SourceID) {
|
||||||
|
logger.WarnF(ctx, "[PagesSource] orphan upload marker changed during reconciliation: upload_id=%d", uploadID)
|
||||||
|
return pagesOrphanCleanupInvalidMarker, true, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
var references int64
|
||||||
|
if err := tx.Model(&model.PagesDeployment{}).
|
||||||
|
Where("upload_id = ?", lockedUpload.ID).
|
||||||
|
Count(&references).Error; err != nil {
|
||||||
|
return pagesOrphanCleanupSkipped, true, err
|
||||||
|
}
|
||||||
|
if references > 0 {
|
||||||
|
return pagesOrphanCleanupReferenced, true, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
transitioned, err := upload.RemoveLockedTx(tx, &lockedUpload)
|
||||||
|
if err != nil {
|
||||||
|
return pagesOrphanCleanupSkipped, true, err
|
||||||
|
}
|
||||||
|
if transitioned {
|
||||||
|
return pagesOrphanCleanupReconciled, true, nil
|
||||||
|
}
|
||||||
|
return pagesOrphanCleanupSkipped, true, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func lockOptionalPagesCleanupRecord(
|
||||||
|
tx *gorm.DB,
|
||||||
|
value any,
|
||||||
|
query string,
|
||||||
|
args ...any,
|
||||||
|
) (bool, error) {
|
||||||
|
err := tx.Clauses(clause.Locking{Strength: pagesRowLockStrength}).
|
||||||
|
Where(query, args...).
|
||||||
|
First(value).Error
|
||||||
|
if err == nil {
|
||||||
|
return true, nil
|
||||||
|
}
|
||||||
|
if errors.Is(err, gorm.ErrRecordNotFound) {
|
||||||
|
return false, nil
|
||||||
|
}
|
||||||
|
return false, err
|
||||||
|
}
|
||||||
|
|
||||||
|
func parsePagesOrphanMarker(metadata model.UploadMetadata) (pagesOrphanMarker, error) {
|
||||||
|
if metadata.Extra == nil {
|
||||||
|
return pagesOrphanMarker{}, errors.New("pages marker metadata missing")
|
||||||
|
}
|
||||||
|
marker, ok := metadata.Extra[pagesIngestMarkerKey].(string)
|
||||||
|
if !ok || marker != pagesIngestMarkerV2 {
|
||||||
|
return pagesOrphanMarker{}, errors.New("pages marker version invalid")
|
||||||
|
}
|
||||||
|
projectID, err := parsePagesOrphanMetadataID(metadata.Extra, pagesProjectIDMetadataKey)
|
||||||
|
if err != nil {
|
||||||
|
return pagesOrphanMarker{}, err
|
||||||
|
}
|
||||||
|
result := pagesOrphanMarker{ProjectID: projectID}
|
||||||
|
if _, exists := metadata.Extra[pagesSourceIDMetadataKey]; exists {
|
||||||
|
sourceID, err := parsePagesOrphanMetadataID(metadata.Extra, pagesSourceIDMetadataKey)
|
||||||
|
if err != nil {
|
||||||
|
return pagesOrphanMarker{}, err
|
||||||
|
}
|
||||||
|
result.SourceID = &sourceID
|
||||||
|
}
|
||||||
|
return result, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func parsePagesOrphanMetadataID(extra map[string]any, key string) (uint, error) {
|
||||||
|
raw, exists := extra[key]
|
||||||
|
if !exists {
|
||||||
|
return 0, fmt.Errorf("pages marker %s missing", key)
|
||||||
|
}
|
||||||
|
value, ok := raw.(string)
|
||||||
|
if !ok || value == "" {
|
||||||
|
return 0, fmt.Errorf("pages marker %s must be a decimal string", key)
|
||||||
|
}
|
||||||
|
parsed, err := strconv.ParseUint(value, 10, 64)
|
||||||
|
maxModelID := uint64(^uint(0) >> 1)
|
||||||
|
if err != nil || parsed == 0 || parsed > maxModelID || strconv.FormatUint(parsed, 10) != value {
|
||||||
|
return 0, fmt.Errorf("pages marker %s is not a canonical non-zero decimal ID", key)
|
||||||
|
}
|
||||||
|
return uint(parsed), nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func sameOptionalPagesSourceID(left, right *uint) bool {
|
||||||
|
if left == nil || right == nil {
|
||||||
|
return left == nil && right == nil
|
||||||
|
}
|
||||||
|
return *left == *right
|
||||||
|
}
|
||||||
@@ -0,0 +1,381 @@
|
|||||||
|
// Copyright 2026 Arctel.net
|
||||||
|
// SPDX-License-Identifier: Apache-2.0
|
||||||
|
|
||||||
|
package pages
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
"errors"
|
||||||
|
"strconv"
|
||||||
|
"testing"
|
||||||
|
"time"
|
||||||
|
|
||||||
|
"github.com/Rain-kl/Wavelet/internal/apps/upload"
|
||||||
|
"github.com/Rain-kl/Wavelet/internal/db"
|
||||||
|
"github.com/Rain-kl/Wavelet/internal/model"
|
||||||
|
"gorm.io/gorm"
|
||||||
|
"gorm.io/gorm/clause"
|
||||||
|
)
|
||||||
|
|
||||||
|
func TestParsePagesOrphanMarker(t *testing.T) {
|
||||||
|
tests := []struct {
|
||||||
|
name string
|
||||||
|
extra map[string]any
|
||||||
|
wantProject uint
|
||||||
|
wantSource uint
|
||||||
|
wantSourceOK bool
|
||||||
|
wantErr bool
|
||||||
|
}{
|
||||||
|
{
|
||||||
|
name: "manual source marker",
|
||||||
|
extra: map[string]any{
|
||||||
|
pagesIngestMarkerKey: pagesIngestMarkerV2,
|
||||||
|
pagesProjectIDMetadataKey: "12",
|
||||||
|
},
|
||||||
|
wantProject: 12,
|
||||||
|
},
|
||||||
|
{
|
||||||
|
name: "persistent source marker",
|
||||||
|
extra: map[string]any{
|
||||||
|
pagesIngestMarkerKey: pagesIngestMarkerV2,
|
||||||
|
pagesProjectIDMetadataKey: "12",
|
||||||
|
pagesSourceIDMetadataKey: "34",
|
||||||
|
},
|
||||||
|
wantProject: 12,
|
||||||
|
wantSource: 34,
|
||||||
|
wantSourceOK: true,
|
||||||
|
},
|
||||||
|
{
|
||||||
|
name: "project ID must be canonical decimal",
|
||||||
|
extra: map[string]any{
|
||||||
|
pagesIngestMarkerKey: pagesIngestMarkerV2,
|
||||||
|
pagesProjectIDMetadataKey: "012",
|
||||||
|
},
|
||||||
|
wantErr: true,
|
||||||
|
},
|
||||||
|
{
|
||||||
|
name: "source ID must be a string",
|
||||||
|
extra: map[string]any{
|
||||||
|
pagesIngestMarkerKey: pagesIngestMarkerV2,
|
||||||
|
pagesProjectIDMetadataKey: "12",
|
||||||
|
pagesSourceIDMetadataKey: float64(34),
|
||||||
|
},
|
||||||
|
wantErr: true,
|
||||||
|
},
|
||||||
|
{
|
||||||
|
name: "zero ID rejected",
|
||||||
|
extra: map[string]any{
|
||||||
|
pagesIngestMarkerKey: pagesIngestMarkerV2,
|
||||||
|
pagesProjectIDMetadataKey: "0",
|
||||||
|
},
|
||||||
|
wantErr: true,
|
||||||
|
},
|
||||||
|
{
|
||||||
|
name: "wrong marker version rejected",
|
||||||
|
extra: map[string]any{
|
||||||
|
pagesIngestMarkerKey: "pages_deployment_v1",
|
||||||
|
pagesProjectIDMetadataKey: "12",
|
||||||
|
},
|
||||||
|
wantErr: true,
|
||||||
|
},
|
||||||
|
}
|
||||||
|
|
||||||
|
for _, test := range tests {
|
||||||
|
t.Run(test.name, func(t *testing.T) {
|
||||||
|
got, err := parsePagesOrphanMarker(model.UploadMetadata{Extra: test.extra})
|
||||||
|
if gotErr := err != nil; gotErr != test.wantErr {
|
||||||
|
t.Fatalf("parsePagesOrphanMarker(%v) error = %v, want error presence = %t", test.extra, err, test.wantErr)
|
||||||
|
}
|
||||||
|
if test.wantErr {
|
||||||
|
return
|
||||||
|
}
|
||||||
|
if got.ProjectID != test.wantProject {
|
||||||
|
t.Errorf("parsePagesOrphanMarker(%v).ProjectID = %d, want %d", test.extra, got.ProjectID, test.wantProject)
|
||||||
|
}
|
||||||
|
if gotSourceOK := got.SourceID != nil; gotSourceOK != test.wantSourceOK {
|
||||||
|
t.Fatalf("parsePagesOrphanMarker(%v).SourceID presence = %t, want %t", test.extra, gotSourceOK, test.wantSourceOK)
|
||||||
|
}
|
||||||
|
if got.SourceID != nil && *got.SourceID != test.wantSource {
|
||||||
|
t.Errorf("parsePagesOrphanMarker(%v).SourceID = %d, want %d", test.extra, *got.SourceID, test.wantSource)
|
||||||
|
}
|
||||||
|
})
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestReconcilePagesOrphanUploadsDeletesEligibleUploadOnce(t *testing.T) {
|
||||||
|
cleanup := setupPagesTestDB(t)
|
||||||
|
defer cleanup()
|
||||||
|
ctx := context.Background()
|
||||||
|
now := time.Now().UTC()
|
||||||
|
project := createPagesOrphanProject(t, ctx, "eligible-orphan")
|
||||||
|
candidate := createPagesOrphanUpload(t, ctx, now.Add(-3*time.Hour), project.ID, nil)
|
||||||
|
if err := upload.RebuildUploadStats(ctx); err != nil {
|
||||||
|
t.Fatalf("RebuildUploadStats() error = %v, want nil", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
summary, err := ReconcilePagesOrphanUploads(ctx, now)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("ReconcilePagesOrphanUploads() error = %v, want nil", err)
|
||||||
|
}
|
||||||
|
if summary.Candidates != 1 || summary.Reconciled != 1 || cleanupOutcomeTotal(summary) != 1 {
|
||||||
|
t.Errorf("ReconcilePagesOrphanUploads() summary = %+v, want one reconciled candidate", summary)
|
||||||
|
}
|
||||||
|
assertPagesCleanupUploadStatus(t, ctx, candidate.ID, model.UploadStatusDeleted)
|
||||||
|
assertPagesCleanupTotalStat(t, ctx, 0)
|
||||||
|
|
||||||
|
second, err := ReconcilePagesOrphanUploads(ctx, now.Add(time.Minute))
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("second ReconcilePagesOrphanUploads() error = %v, want nil", err)
|
||||||
|
}
|
||||||
|
if second.Candidates != 0 || cleanupOutcomeTotal(second) != 0 {
|
||||||
|
t.Errorf("second ReconcilePagesOrphanUploads() summary = %+v, want empty", second)
|
||||||
|
}
|
||||||
|
assertPagesCleanupTotalStat(t, ctx, 0)
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestReconcilePagesOrphanUploadsAllowsDeletedProjectAndSource(t *testing.T) {
|
||||||
|
cleanup := setupPagesTestDB(t)
|
||||||
|
defer cleanup()
|
||||||
|
ctx := context.Background()
|
||||||
|
now := time.Now().UTC()
|
||||||
|
missingSourceID := uint(9876)
|
||||||
|
candidate := createPagesOrphanUpload(t, ctx, now.Add(-3*time.Hour), 8765, &missingSourceID)
|
||||||
|
|
||||||
|
summary, err := ReconcilePagesOrphanUploads(ctx, now)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("ReconcilePagesOrphanUploads() error = %v, want nil", err)
|
||||||
|
}
|
||||||
|
if summary.Candidates != 1 || summary.Reconciled != 1 || cleanupOutcomeTotal(summary) != 1 {
|
||||||
|
t.Errorf("ReconcilePagesOrphanUploads() summary = %+v, want deleted project/source treated as one orphan", summary)
|
||||||
|
}
|
||||||
|
assertPagesCleanupUploadStatus(t, ctx, candidate.ID, model.UploadStatusDeleted)
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestReconcilePagesOrphanUploadsSkipsBusyLeaseAndSourceMismatch(t *testing.T) {
|
||||||
|
t.Run("unexpired source lease", func(t *testing.T) {
|
||||||
|
cleanup := setupPagesTestDB(t)
|
||||||
|
defer cleanup()
|
||||||
|
ctx := context.Background()
|
||||||
|
realNow := time.Now().UTC()
|
||||||
|
// A deliberately future scanner snapshot proves lease freshness uses the
|
||||||
|
// real clock after the runtime lock, not this isolation-cutoff input.
|
||||||
|
scannerNow := realNow.Add(24 * time.Hour)
|
||||||
|
project := createPagesOrphanProject(t, ctx, "busy-orphan")
|
||||||
|
source := createPagesOrphanSource(t, ctx, project.ID)
|
||||||
|
future := realNow.Add(time.Hour)
|
||||||
|
if err := db.DB(ctx).Create(&model.PagesProjectSourceRuntime{
|
||||||
|
SourceID: source.ID,
|
||||||
|
LeaseToken: "busy-worker",
|
||||||
|
LeaseExpiresAt: &future,
|
||||||
|
}).Error; err != nil {
|
||||||
|
t.Fatalf("create busy source runtime error = %v, want nil", err)
|
||||||
|
}
|
||||||
|
candidate := createPagesOrphanUpload(t, ctx, realNow.Add(-3*time.Hour), project.ID, &source.ID)
|
||||||
|
|
||||||
|
summary, err := ReconcilePagesOrphanUploads(ctx, scannerNow)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("ReconcilePagesOrphanUploads() error = %v, want nil", err)
|
||||||
|
}
|
||||||
|
if summary.LeaseBusy != 1 || cleanupOutcomeTotal(summary) != 1 {
|
||||||
|
t.Errorf("ReconcilePagesOrphanUploads() summary = %+v, want one lease-busy candidate", summary)
|
||||||
|
}
|
||||||
|
assertPagesCleanupUploadStatus(t, ctx, candidate.ID, model.UploadStatusUsed)
|
||||||
|
})
|
||||||
|
|
||||||
|
t.Run("source belongs to another project", func(t *testing.T) {
|
||||||
|
cleanup := setupPagesTestDB(t)
|
||||||
|
defer cleanup()
|
||||||
|
ctx := context.Background()
|
||||||
|
now := time.Now().UTC()
|
||||||
|
markerProject := createPagesOrphanProject(t, ctx, "marker-project")
|
||||||
|
actualProject := createPagesOrphanProject(t, ctx, "actual-project")
|
||||||
|
source := createPagesOrphanSource(t, ctx, actualProject.ID)
|
||||||
|
candidate := createPagesOrphanUpload(t, ctx, now.Add(-3*time.Hour), markerProject.ID, &source.ID)
|
||||||
|
|
||||||
|
summary, err := ReconcilePagesOrphanUploads(ctx, now)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("ReconcilePagesOrphanUploads() error = %v, want nil", err)
|
||||||
|
}
|
||||||
|
if summary.InvalidMarker != 1 || cleanupOutcomeTotal(summary) != 1 {
|
||||||
|
t.Errorf("ReconcilePagesOrphanUploads() summary = %+v, want one ownership mismatch", summary)
|
||||||
|
}
|
||||||
|
assertPagesCleanupUploadStatus(t, ctx, candidate.ID, model.UploadStatusUsed)
|
||||||
|
})
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestReconcilePagesOrphanUploadsRejectsMalformedMarker(t *testing.T) {
|
||||||
|
cleanup := setupPagesTestDB(t)
|
||||||
|
defer cleanup()
|
||||||
|
ctx := context.Background()
|
||||||
|
now := time.Now().UTC()
|
||||||
|
candidate := createPagesOrphanUpload(t, ctx, now.Add(-3*time.Hour), 1, nil)
|
||||||
|
metadata := candidate.Metadata
|
||||||
|
metadata.Extra[pagesProjectIDMetadataKey] = "01"
|
||||||
|
candidate.Metadata = metadata
|
||||||
|
if err := db.DB(ctx).Save(candidate).Error; err != nil {
|
||||||
|
t.Fatalf("seed malformed candidate marker error = %v, want nil", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
summary, err := ReconcilePagesOrphanUploads(ctx, now)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("ReconcilePagesOrphanUploads() error = %v, want nil", err)
|
||||||
|
}
|
||||||
|
if summary.InvalidMarker != 1 || cleanupOutcomeTotal(summary) != 1 {
|
||||||
|
t.Errorf("ReconcilePagesOrphanUploads() summary = %+v, want one invalid marker", summary)
|
||||||
|
}
|
||||||
|
assertPagesCleanupUploadStatus(t, ctx, candidate.ID, model.UploadStatusUsed)
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestPagesOrphanCleanupAndDeploymentCommitInterleavings(t *testing.T) {
|
||||||
|
t.Run("deployment reference commits first", func(t *testing.T) {
|
||||||
|
cleanup := setupPagesTestDB(t)
|
||||||
|
defer cleanup()
|
||||||
|
ctx := context.Background()
|
||||||
|
now := time.Now().UTC()
|
||||||
|
project := createPagesOrphanProject(t, ctx, "deployment-first")
|
||||||
|
candidate := createPagesOrphanUpload(t, ctx, now.Add(-3*time.Hour), project.ID, nil)
|
||||||
|
marker, err := parsePagesOrphanMarker(candidate.Metadata)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("parsePagesOrphanMarker() error = %v, want nil", err)
|
||||||
|
}
|
||||||
|
if err := db.DB(ctx).Create(&model.PagesDeployment{
|
||||||
|
ProjectID: project.ID,
|
||||||
|
DeploymentNumber: 1,
|
||||||
|
Checksum: "deployment-first",
|
||||||
|
Status: model.PagesDeploymentStatusUploaded,
|
||||||
|
UploadID: candidate.ID,
|
||||||
|
}).Error; err != nil {
|
||||||
|
t.Fatalf("create deployment reference error = %v, want nil", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
outcome, err := reconcilePagesOrphanUploadCandidate(ctx, candidate, marker, 999, now.Add(-2*time.Hour))
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("reconcilePagesOrphanUploadCandidate() error = %v, want nil", err)
|
||||||
|
}
|
||||||
|
if outcome != pagesOrphanCleanupReferenced {
|
||||||
|
t.Errorf("reconcilePagesOrphanUploadCandidate() outcome = %d, want %d", outcome, pagesOrphanCleanupReferenced)
|
||||||
|
}
|
||||||
|
assertPagesCleanupUploadStatus(t, ctx, candidate.ID, model.UploadStatusUsed)
|
||||||
|
})
|
||||||
|
|
||||||
|
t.Run("cleanup commits first", func(t *testing.T) {
|
||||||
|
cleanup := setupPagesTestDB(t)
|
||||||
|
defer cleanup()
|
||||||
|
ctx := context.Background()
|
||||||
|
now := time.Now().UTC()
|
||||||
|
project := createPagesOrphanProject(t, ctx, "cleanup-first")
|
||||||
|
candidate := createPagesOrphanUpload(t, ctx, now.Add(-3*time.Hour), project.ID, nil)
|
||||||
|
|
||||||
|
summary, err := ReconcilePagesOrphanUploads(ctx, now)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("ReconcilePagesOrphanUploads() error = %v, want nil", err)
|
||||||
|
}
|
||||||
|
if summary.Reconciled != 1 {
|
||||||
|
t.Fatalf("ReconcilePagesOrphanUploads() summary = %+v, want one reconciled candidate", summary)
|
||||||
|
}
|
||||||
|
|
||||||
|
target := &model.PagesDeployment{ProjectID: project.ID, UploadID: candidate.ID}
|
||||||
|
err = db.DB(ctx).Transaction(func(tx *gorm.DB) error {
|
||||||
|
var lockedProject model.PagesProject
|
||||||
|
if err := tx.Clauses(clause.Locking{Strength: pagesRowLockStrength}).First(&lockedProject, project.ID).Error; err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
return lockSourceDeploymentUploadsTx(tx, target, upload.IngestResult{}, false)
|
||||||
|
})
|
||||||
|
if !errors.Is(err, errSourceFinalFence) {
|
||||||
|
t.Errorf("final deployment upload lock after cleanup error = %v, want %v", err, errSourceFinalFence)
|
||||||
|
}
|
||||||
|
var references int64
|
||||||
|
if err := db.DB(ctx).Model(&model.PagesDeployment{}).Where("upload_id = ?", candidate.ID).Count(&references).Error; err != nil {
|
||||||
|
t.Fatalf("count deployment references error = %v, want nil", err)
|
||||||
|
}
|
||||||
|
if references != 0 {
|
||||||
|
t.Errorf("deployment references after cleanup-first interleaving = %d, want 0", references)
|
||||||
|
}
|
||||||
|
})
|
||||||
|
}
|
||||||
|
|
||||||
|
func cleanupOutcomeTotal(summary PagesOrphanCleanupSummary) int {
|
||||||
|
return summary.Reconciled + summary.Referenced + summary.LeaseBusy + summary.InvalidMarker + summary.Skipped + summary.Failed
|
||||||
|
}
|
||||||
|
|
||||||
|
func createPagesOrphanProject(t *testing.T, ctx context.Context, slug string) *model.PagesProject {
|
||||||
|
t.Helper()
|
||||||
|
project := &model.PagesProject{Name: slug, Slug: slug, Enabled: true}
|
||||||
|
if err := db.DB(ctx).Create(project).Error; err != nil {
|
||||||
|
t.Fatalf("create Pages orphan project %q error = %v, want nil", slug, err)
|
||||||
|
}
|
||||||
|
return project
|
||||||
|
}
|
||||||
|
|
||||||
|
func createPagesOrphanSource(t *testing.T, ctx context.Context, projectID uint) *model.PagesProjectSource {
|
||||||
|
t.Helper()
|
||||||
|
source := &model.PagesProjectSource{
|
||||||
|
ProjectID: projectID,
|
||||||
|
SourceType: PagesSourceTypeRemoteURL,
|
||||||
|
ConfigVersion: 1,
|
||||||
|
SourceIdentity: "orphan-source-identity",
|
||||||
|
}
|
||||||
|
if err := db.DB(ctx).Create(source).Error; err != nil {
|
||||||
|
t.Fatalf("create Pages orphan source for project %d error = %v, want nil", projectID, err)
|
||||||
|
}
|
||||||
|
return source
|
||||||
|
}
|
||||||
|
|
||||||
|
func createPagesOrphanUpload(
|
||||||
|
t *testing.T,
|
||||||
|
ctx context.Context,
|
||||||
|
createdAt time.Time,
|
||||||
|
projectID uint,
|
||||||
|
sourceID *uint,
|
||||||
|
) *model.Upload {
|
||||||
|
t.Helper()
|
||||||
|
extra := map[string]any{
|
||||||
|
pagesIngestMarkerKey: pagesIngestMarkerV2,
|
||||||
|
pagesProjectIDMetadataKey: strconv.FormatUint(uint64(projectID), 10),
|
||||||
|
}
|
||||||
|
if sourceID != nil {
|
||||||
|
extra[pagesSourceIDMetadataKey] = strconv.FormatUint(uint64(*sourceID), 10)
|
||||||
|
}
|
||||||
|
candidate := &model.Upload{
|
||||||
|
UserID: 999,
|
||||||
|
FileName: "site.zip",
|
||||||
|
FilePath: "pages/orphan-site.zip",
|
||||||
|
FileSize: 64,
|
||||||
|
MimeType: "application/zip",
|
||||||
|
Extension: "zip",
|
||||||
|
Hash: "orphan-checksum",
|
||||||
|
Type: upload.ReservedPagesDeploymentType,
|
||||||
|
Status: model.UploadStatusUsed,
|
||||||
|
AccessMode: 0,
|
||||||
|
Metadata: model.UploadMetadata{Extra: extra},
|
||||||
|
CreatedAt: createdAt,
|
||||||
|
UpdatedAt: createdAt,
|
||||||
|
}
|
||||||
|
if err := db.DB(ctx).Create(candidate).Error; err != nil {
|
||||||
|
t.Fatalf("create Pages orphan upload error = %v, want nil", err)
|
||||||
|
}
|
||||||
|
return candidate
|
||||||
|
}
|
||||||
|
|
||||||
|
func assertPagesCleanupUploadStatus(t *testing.T, ctx context.Context, uploadID uint64, want model.UploadStatus) {
|
||||||
|
t.Helper()
|
||||||
|
var got model.Upload
|
||||||
|
if err := db.DB(ctx).First(&got, uploadID).Error; err != nil {
|
||||||
|
t.Fatalf("load upload %d error = %v, want nil", uploadID, err)
|
||||||
|
}
|
||||||
|
if got.Status != want {
|
||||||
|
t.Errorf("upload %d status = %q, want %q", uploadID, got.Status, want)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func assertPagesCleanupTotalStat(t *testing.T, ctx context.Context, want int64) {
|
||||||
|
t.Helper()
|
||||||
|
var stat model.UploadStat
|
||||||
|
if err := db.DB(ctx).Where("dimension = ? AND stat_key = ?", model.UploadStatDimensionTotal, "").First(&stat).Error; err != nil {
|
||||||
|
t.Fatalf("load total upload stat error = %v, want nil", err)
|
||||||
|
}
|
||||||
|
if stat.FileCount != want {
|
||||||
|
t.Errorf("total upload stat FileCount = %d, want %d", stat.FileCount, want)
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -0,0 +1,549 @@
|
|||||||
|
// Copyright 2026 Arctel.net
|
||||||
|
// SPDX-License-Identifier: Apache-2.0
|
||||||
|
|
||||||
|
package pages
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
"crypto/sha256"
|
||||||
|
"crypto/tls"
|
||||||
|
"encoding/hex"
|
||||||
|
"errors"
|
||||||
|
"fmt"
|
||||||
|
"io"
|
||||||
|
"math"
|
||||||
|
"net"
|
||||||
|
"net/http"
|
||||||
|
"net/netip"
|
||||||
|
"net/url"
|
||||||
|
"os"
|
||||||
|
"path"
|
||||||
|
"strings"
|
||||||
|
"time"
|
||||||
|
|
||||||
|
"github.com/Rain-kl/Wavelet/pkg/httppool"
|
||||||
|
"github.com/Rain-kl/Wavelet/pkg/pagesarchive"
|
||||||
|
)
|
||||||
|
|
||||||
|
const (
|
||||||
|
// RemoteNetworkPolicyPublic only permits publicly routable targets and
|
||||||
|
// performs DNS validation again for every connection.
|
||||||
|
RemoteNetworkPolicyPublic = "public"
|
||||||
|
// RemoteNetworkPolicyTrustedInternal permits private targets and self-signed
|
||||||
|
// TLS certificates. It is an explicit administrator trust boundary.
|
||||||
|
RemoteNetworkPolicyTrustedInternal = "trusted_internal"
|
||||||
|
|
||||||
|
remoteSourceDownloadTimeout = 10 * time.Minute
|
||||||
|
remoteSourceResponseHeaderTimeout = 30 * time.Second
|
||||||
|
remoteSourceDialTimeout = 30 * time.Second
|
||||||
|
remoteSourceDialKeepAlive = 30 * time.Second
|
||||||
|
remoteSourceMaxRedirects = 5
|
||||||
|
remoteSourceMagicSniffBytes = 512
|
||||||
|
remoteSourceMaxSafeLabelBytes = 255
|
||||||
|
remoteSourceFallbackLabel = "package"
|
||||||
|
remoteSourceUserAgent = "OpenFlare Pages Source/2"
|
||||||
|
remoteSourceSchemeHTTP = "http"
|
||||||
|
remoteSourceSchemeHTTPS = "https"
|
||||||
|
)
|
||||||
|
|
||||||
|
type remoteProviderError string
|
||||||
|
|
||||||
|
func (providerError remoteProviderError) Error() string {
|
||||||
|
return string(providerError)
|
||||||
|
}
|
||||||
|
|
||||||
|
const (
|
||||||
|
errRemoteProviderInvalidPolicy remoteProviderError = "远程来源网络策略无效"
|
||||||
|
errRemoteProviderInvalidLimit remoteProviderError = "远程来源部署包大小限制无效"
|
||||||
|
errRemoteProviderBlockedAddress remoteProviderError = "远程来源 public 策略禁止访问非公网地址"
|
||||||
|
errRemoteProviderResolveFailed remoteProviderError = "远程来源地址解析失败"
|
||||||
|
errRemoteProviderRedirectLimit remoteProviderError = "远程来源重定向次数超过限制"
|
||||||
|
errRemoteProviderDownloadFailed remoteProviderError = errPagesPackageURLDownloadFailed
|
||||||
|
errRemoteProviderTooLarge remoteProviderError = errPagesPackageURLTooLarge
|
||||||
|
errRemoteProviderEmpty remoteProviderError = errPagesPackageEmpty
|
||||||
|
errRemoteProviderUnsupported remoteProviderError = errPagesPackageUnsupported
|
||||||
|
errRemoteProviderCleanupFailed remoteProviderError = "清理远程来源临时文件失败"
|
||||||
|
)
|
||||||
|
|
||||||
|
var remoteSourceNonPublicPrefixes = []netip.Prefix{
|
||||||
|
// IPv4 special-use, private, link-local, documentation, multicast and
|
||||||
|
// reserved ranges. A conservative deny list is intentional for SSRF safety.
|
||||||
|
netip.MustParsePrefix("0.0.0.0/8"),
|
||||||
|
netip.MustParsePrefix("10.0.0.0/8"),
|
||||||
|
netip.MustParsePrefix("100.64.0.0/10"),
|
||||||
|
netip.MustParsePrefix("127.0.0.0/8"),
|
||||||
|
netip.MustParsePrefix("169.254.0.0/16"),
|
||||||
|
netip.MustParsePrefix("172.16.0.0/12"),
|
||||||
|
netip.MustParsePrefix("192.0.0.0/24"),
|
||||||
|
netip.MustParsePrefix("192.0.2.0/24"),
|
||||||
|
netip.MustParsePrefix("192.88.99.0/24"),
|
||||||
|
netip.MustParsePrefix("192.168.0.0/16"),
|
||||||
|
netip.MustParsePrefix("198.18.0.0/15"),
|
||||||
|
netip.MustParsePrefix("198.51.100.0/24"),
|
||||||
|
netip.MustParsePrefix("203.0.113.0/24"),
|
||||||
|
netip.MustParsePrefix("224.0.0.0/4"),
|
||||||
|
netip.MustParsePrefix("240.0.0.0/4"),
|
||||||
|
// IPv6 protocol-assignment, documentation and transition ranges that are
|
||||||
|
// not acceptable as direct public artifact origins.
|
||||||
|
netip.MustParsePrefix("2001::/23"),
|
||||||
|
netip.MustParsePrefix("2001:db8::/32"),
|
||||||
|
netip.MustParsePrefix("2002::/16"),
|
||||||
|
netip.MustParsePrefix("3fff::/20"),
|
||||||
|
}
|
||||||
|
|
||||||
|
var remoteSourcePublicIPv6Prefix = netip.MustParsePrefix("2000::/3")
|
||||||
|
|
||||||
|
// RemoteSourceRequest describes one immutable Remote URL package fetch.
|
||||||
|
type RemoteSourceRequest struct {
|
||||||
|
URL string
|
||||||
|
NetworkPolicy string
|
||||||
|
MaxPackageBytes int64
|
||||||
|
}
|
||||||
|
|
||||||
|
// SourceCandidate is a constrained, immutable archive downloaded to a
|
||||||
|
// provider-owned temporary file. The caller owns the file after a successful
|
||||||
|
// fetch and must call Cleanup when processing finishes.
|
||||||
|
type SourceCandidate struct {
|
||||||
|
TempPath string
|
||||||
|
Checksum string
|
||||||
|
PackageSize int64
|
||||||
|
Format pagesarchive.Format
|
||||||
|
SafeLabel string
|
||||||
|
}
|
||||||
|
|
||||||
|
// Cleanup removes the candidate temporary file. It is safe to call repeatedly.
|
||||||
|
func (candidate *SourceCandidate) Cleanup() error {
|
||||||
|
if candidate == nil || candidate.TempPath == "" {
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
tempPath := candidate.TempPath
|
||||||
|
err := os.Remove(tempPath)
|
||||||
|
if err == nil || errors.Is(err, os.ErrNotExist) {
|
||||||
|
candidate.TempPath = ""
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
return errRemoteProviderCleanupFailed
|
||||||
|
}
|
||||||
|
|
||||||
|
type remoteSourceResolver interface {
|
||||||
|
LookupNetIP(context.Context, string, string) ([]netip.Addr, error)
|
||||||
|
}
|
||||||
|
|
||||||
|
type remoteSourceDependencies struct {
|
||||||
|
resolver remoteSourceResolver
|
||||||
|
dialContext func(context.Context, string, string) (net.Conn, error)
|
||||||
|
createTemp func(string, string) (*os.File, error)
|
||||||
|
}
|
||||||
|
|
||||||
|
// FetchRemoteSource downloads a Remote URL package without writing deployment
|
||||||
|
// state. Errors are reduced to safe domain messages and never contain the raw
|
||||||
|
// URL, query, response headers or response body.
|
||||||
|
func FetchRemoteSource(ctx context.Context, request RemoteSourceRequest) (*SourceCandidate, error) {
|
||||||
|
dialer := &net.Dialer{
|
||||||
|
Timeout: remoteSourceDialTimeout,
|
||||||
|
KeepAlive: remoteSourceDialKeepAlive,
|
||||||
|
}
|
||||||
|
dependencies := remoteSourceDependencies{
|
||||||
|
resolver: net.DefaultResolver,
|
||||||
|
dialContext: dialer.DialContext,
|
||||||
|
createTemp: os.CreateTemp,
|
||||||
|
}
|
||||||
|
return fetchRemoteSource(ctx, request, dependencies)
|
||||||
|
}
|
||||||
|
|
||||||
|
func fetchRemoteSource(ctx context.Context, request RemoteSourceRequest, dependencies remoteSourceDependencies) (*SourceCandidate, error) {
|
||||||
|
if request.MaxPackageBytes <= 0 {
|
||||||
|
return nil, errRemoteProviderInvalidLimit
|
||||||
|
}
|
||||||
|
if dependencies.dialContext == nil || dependencies.createTemp == nil {
|
||||||
|
return nil, errRemoteProviderDownloadFailed
|
||||||
|
}
|
||||||
|
policy, err := normalizeRemoteNetworkPolicy(request.NetworkPolicy)
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
parsed, err := parseRemoteSourceURL(request.URL)
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
if err := validateRemoteSourceTarget(ctx, parsed, policy, dependencies.resolver); err != nil {
|
||||||
|
return nil, sanitizeRemoteProviderError(ctx, err)
|
||||||
|
}
|
||||||
|
|
||||||
|
safeLabel, namedFormat := remoteSourceLabel(parsed)
|
||||||
|
client := newRemoteSourceClient(policy, dependencies)
|
||||||
|
defer client.CloseIdleConnections()
|
||||||
|
response, err := requestRemoteSource(ctx, client, parsed)
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
defer func() { _ = response.Body.Close() }()
|
||||||
|
|
||||||
|
if response.StatusCode < http.StatusOK || response.StatusCode >= http.StatusMultipleChoices {
|
||||||
|
return nil, fmt.Errorf("%w: HTTP %d", errRemoteProviderDownloadFailed, response.StatusCode)
|
||||||
|
}
|
||||||
|
if response.ContentLength > request.MaxPackageBytes {
|
||||||
|
return nil, errRemoteProviderTooLarge
|
||||||
|
}
|
||||||
|
|
||||||
|
tempPath, checksum, packageSize, err := streamRemoteSourcePackage(
|
||||||
|
response.Body,
|
||||||
|
request.MaxPackageBytes,
|
||||||
|
dependencies.createTemp,
|
||||||
|
)
|
||||||
|
if err != nil {
|
||||||
|
return nil, sanitizeRemoteProviderError(ctx, err)
|
||||||
|
}
|
||||||
|
format, safeLabel, err := detectRemoteSourceFormat(tempPath, safeLabel, namedFormat)
|
||||||
|
if err != nil {
|
||||||
|
if removeErr := os.Remove(tempPath); removeErr != nil && !errors.Is(removeErr, os.ErrNotExist) {
|
||||||
|
return nil, errRemoteProviderCleanupFailed
|
||||||
|
}
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
|
||||||
|
return &SourceCandidate{
|
||||||
|
TempPath: tempPath,
|
||||||
|
Checksum: checksum,
|
||||||
|
PackageSize: packageSize,
|
||||||
|
Format: format,
|
||||||
|
SafeLabel: safeLabel,
|
||||||
|
}, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func normalizeRemoteNetworkPolicy(policy string) (string, error) {
|
||||||
|
switch strings.TrimSpace(policy) {
|
||||||
|
case "", RemoteNetworkPolicyPublic:
|
||||||
|
return RemoteNetworkPolicyPublic, nil
|
||||||
|
case RemoteNetworkPolicyTrustedInternal:
|
||||||
|
return RemoteNetworkPolicyTrustedInternal, nil
|
||||||
|
default:
|
||||||
|
return "", errRemoteProviderInvalidPolicy
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func newRemoteSourceClient(policy string, dependencies remoteSourceDependencies) *http.Client {
|
||||||
|
tlsConfig := &tls.Config{MinVersion: tls.VersionTLS12}
|
||||||
|
dialContext := dependencies.dialContext
|
||||||
|
if policy == RemoteNetworkPolicyPublic {
|
||||||
|
dialContext = newPublicRemoteSourceDialer(dependencies.resolver, dependencies.dialContext)
|
||||||
|
} else {
|
||||||
|
// trusted_internal is an explicit administrator-selected boundary for
|
||||||
|
// private artifact services using an internal CA or self-signed cert.
|
||||||
|
tlsConfig.InsecureSkipVerify = true //nolint:gosec // required trusted_internal semantics
|
||||||
|
}
|
||||||
|
|
||||||
|
client := &http.Client{
|
||||||
|
Timeout: remoteSourceDownloadTimeout,
|
||||||
|
Transport: httppool.NewTransport(httppool.TransportOptions{
|
||||||
|
Proxy: nil,
|
||||||
|
DialContext: dialContext,
|
||||||
|
TLSClientConfig: tlsConfig,
|
||||||
|
ResponseHeaderTimeout: remoteSourceResponseHeaderTimeout,
|
||||||
|
TraceFilter: remoteSourceTraceFilter,
|
||||||
|
}),
|
||||||
|
}
|
||||||
|
client.CheckRedirect = func(next *http.Request, previous []*http.Request) error {
|
||||||
|
if len(previous) > remoteSourceMaxRedirects {
|
||||||
|
return errRemoteProviderRedirectLimit
|
||||||
|
}
|
||||||
|
stripRemoteSourceRedirectHeaders(next)
|
||||||
|
if err := validateRemoteSourceTarget(next.Context(), next.URL, policy, dependencies.resolver); err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
applyRemoteSourceHeaders(next)
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
return client
|
||||||
|
}
|
||||||
|
|
||||||
|
func requestRemoteSource(ctx context.Context, client *http.Client, parsed *url.URL) (*http.Response, error) {
|
||||||
|
request, err := http.NewRequestWithContext(ctx, http.MethodGet, parsed.String(), nil)
|
||||||
|
if err != nil {
|
||||||
|
return nil, errors.New(errPagesSourceRemoteURLInvalid)
|
||||||
|
}
|
||||||
|
applyRemoteSourceHeaders(request)
|
||||||
|
response, err := client.Do(request) //nolint:gosec // scheme and every dial target are validated above
|
||||||
|
if err == nil {
|
||||||
|
return response, nil
|
||||||
|
}
|
||||||
|
if response != nil && response.Body != nil {
|
||||||
|
_ = response.Body.Close()
|
||||||
|
}
|
||||||
|
return nil, sanitizeRemoteProviderError(ctx, err)
|
||||||
|
}
|
||||||
|
|
||||||
|
func applyRemoteSourceHeaders(request *http.Request) {
|
||||||
|
request.Header.Set("User-Agent", remoteSourceUserAgent)
|
||||||
|
request.Header.Set("Accept", "application/octet-stream,application/zip,application/x-tar,application/gzip,*/*;q=0.1")
|
||||||
|
// Preserve the artifact bytes exactly as stored. Automatic HTTP gzip
|
||||||
|
// decompression would change the checksum, size and archive format.
|
||||||
|
request.Header.Set("Accept-Encoding", "identity")
|
||||||
|
}
|
||||||
|
|
||||||
|
func stripRemoteSourceRedirectHeaders(request *http.Request) {
|
||||||
|
request.Header.Del("Authorization")
|
||||||
|
request.Header.Del("Cookie")
|
||||||
|
request.Header.Del("Proxy-Authorization")
|
||||||
|
request.Header.Del("Referer")
|
||||||
|
}
|
||||||
|
|
||||||
|
func remoteSourceTraceFilter(request *http.Request) bool {
|
||||||
|
// otelhttp records url.full. Signed query strings must never enter traces.
|
||||||
|
return request.URL == nil || request.URL.RawQuery == ""
|
||||||
|
}
|
||||||
|
|
||||||
|
func validateRemoteSourceTarget(ctx context.Context, target *url.URL, policy string, resolver remoteSourceResolver) error {
|
||||||
|
if target == nil || target.User != nil || target.Fragment != "" || target.Opaque != "" {
|
||||||
|
return errors.New(errPagesSourceRemoteURLInvalid)
|
||||||
|
}
|
||||||
|
scheme := strings.ToLower(strings.TrimSpace(target.Scheme))
|
||||||
|
if (scheme != remoteSourceSchemeHTTP && scheme != remoteSourceSchemeHTTPS) || strings.TrimSpace(target.Hostname()) == "" {
|
||||||
|
return errors.New(errPagesSourceRemoteURLInvalid)
|
||||||
|
}
|
||||||
|
if policy != RemoteNetworkPolicyPublic {
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
_, err := resolvePublicRemoteSourceIPs(ctx, resolver, target.Hostname())
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
|
||||||
|
func newPublicRemoteSourceDialer(
|
||||||
|
resolver remoteSourceResolver,
|
||||||
|
directDial func(context.Context, string, string) (net.Conn, error),
|
||||||
|
) func(context.Context, string, string) (net.Conn, error) {
|
||||||
|
return func(ctx context.Context, network string, address string) (net.Conn, error) {
|
||||||
|
host, port, err := net.SplitHostPort(address)
|
||||||
|
if err != nil {
|
||||||
|
return nil, errRemoteProviderDownloadFailed
|
||||||
|
}
|
||||||
|
addresses, err := resolvePublicRemoteSourceIPs(ctx, resolver, host)
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
for _, address := range addresses {
|
||||||
|
if !remoteSourceIPMatchesNetwork(address, network) {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
connection, dialErr := directDial(ctx, network, net.JoinHostPort(address.String(), port))
|
||||||
|
if dialErr == nil {
|
||||||
|
return connection, nil
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return nil, errRemoteProviderDownloadFailed
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func resolvePublicRemoteSourceIPs(ctx context.Context, resolver remoteSourceResolver, host string) ([]netip.Addr, error) {
|
||||||
|
if strings.Contains(host, "%") {
|
||||||
|
return nil, errRemoteProviderBlockedAddress
|
||||||
|
}
|
||||||
|
if literal, parseErr := netip.ParseAddr(host); parseErr == nil {
|
||||||
|
if !isPublicRemoteSourceIP(literal) {
|
||||||
|
return nil, errRemoteProviderBlockedAddress
|
||||||
|
}
|
||||||
|
return []netip.Addr{literal}, nil
|
||||||
|
}
|
||||||
|
if resolver == nil {
|
||||||
|
return nil, errRemoteProviderResolveFailed
|
||||||
|
}
|
||||||
|
addresses, err := resolver.LookupNetIP(ctx, "ip", host)
|
||||||
|
if err != nil || len(addresses) == 0 {
|
||||||
|
return nil, errRemoteProviderResolveFailed
|
||||||
|
}
|
||||||
|
for _, address := range addresses {
|
||||||
|
if !isPublicRemoteSourceIP(address) {
|
||||||
|
return nil, errRemoteProviderBlockedAddress
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return addresses, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func isPublicRemoteSourceIP(address netip.Addr) bool {
|
||||||
|
if !address.IsValid() || address.Zone() != "" {
|
||||||
|
return false
|
||||||
|
}
|
||||||
|
address = address.Unmap()
|
||||||
|
if !address.IsGlobalUnicast() {
|
||||||
|
return false
|
||||||
|
}
|
||||||
|
if address.Is6() && !remoteSourcePublicIPv6Prefix.Contains(address) {
|
||||||
|
return false
|
||||||
|
}
|
||||||
|
for _, prefix := range remoteSourceNonPublicPrefixes {
|
||||||
|
if prefix.Contains(address) {
|
||||||
|
return false
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return true
|
||||||
|
}
|
||||||
|
|
||||||
|
func remoteSourceIPMatchesNetwork(address netip.Addr, network string) bool {
|
||||||
|
switch network {
|
||||||
|
case "tcp4":
|
||||||
|
return address.Unmap().Is4()
|
||||||
|
case "tcp6":
|
||||||
|
return address.Unmap().Is6()
|
||||||
|
default:
|
||||||
|
return true
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func streamRemoteSourcePackage(
|
||||||
|
body io.Reader,
|
||||||
|
maxPackageBytes int64,
|
||||||
|
createTemp func(string, string) (*os.File, error),
|
||||||
|
) (tempPath string, checksum string, packageSize int64, err error) {
|
||||||
|
if createTemp == nil {
|
||||||
|
return "", "", 0, errRemoteProviderDownloadFailed
|
||||||
|
}
|
||||||
|
tempFile, err := createTemp("", "openflare-pages-source-*")
|
||||||
|
if err != nil {
|
||||||
|
return "", "", 0, errRemoteProviderDownloadFailed
|
||||||
|
}
|
||||||
|
createdTempPath := tempFile.Name()
|
||||||
|
tempPath = createdTempPath
|
||||||
|
defer func() {
|
||||||
|
closeErr := tempFile.Close()
|
||||||
|
if err == nil && closeErr != nil {
|
||||||
|
err = errRemoteProviderDownloadFailed
|
||||||
|
}
|
||||||
|
if err != nil {
|
||||||
|
if removeErr := os.Remove(createdTempPath); removeErr != nil && !errors.Is(removeErr, os.ErrNotExist) {
|
||||||
|
err = errRemoteProviderCleanupFailed
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}()
|
||||||
|
|
||||||
|
hasher := sha256.New()
|
||||||
|
readLimit := maxPackageBytes
|
||||||
|
if readLimit < math.MaxInt64 {
|
||||||
|
readLimit++
|
||||||
|
}
|
||||||
|
packageSize, err = io.Copy(io.MultiWriter(tempFile, hasher), io.LimitReader(body, readLimit))
|
||||||
|
if err != nil {
|
||||||
|
return "", "", 0, errRemoteProviderDownloadFailed
|
||||||
|
}
|
||||||
|
if packageSize > maxPackageBytes {
|
||||||
|
return "", "", 0, errRemoteProviderTooLarge
|
||||||
|
}
|
||||||
|
if packageSize == 0 {
|
||||||
|
return "", "", 0, errRemoteProviderEmpty
|
||||||
|
}
|
||||||
|
checksum = hex.EncodeToString(hasher.Sum(nil))
|
||||||
|
return tempPath, checksum, packageSize, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func detectRemoteSourceFormat(
|
||||||
|
tempPath string,
|
||||||
|
safeLabel string,
|
||||||
|
namedFormat pagesarchive.Format,
|
||||||
|
) (pagesarchive.Format, string, error) {
|
||||||
|
if namedFormat != "" {
|
||||||
|
return namedFormat, safeLabel, nil
|
||||||
|
}
|
||||||
|
tempFile, err := os.Open(tempPath) //nolint:gosec // path is a provider-created temporary file
|
||||||
|
if err != nil {
|
||||||
|
return "", safeLabel, errRemoteProviderDownloadFailed
|
||||||
|
}
|
||||||
|
defer func() { _ = tempFile.Close() }()
|
||||||
|
|
||||||
|
head := make([]byte, remoteSourceMagicSniffBytes)
|
||||||
|
readBytes, readErr := io.ReadFull(tempFile, head)
|
||||||
|
if readErr != nil && !errors.Is(readErr, io.EOF) && !errors.Is(readErr, io.ErrUnexpectedEOF) {
|
||||||
|
return "", safeLabel, errRemoteProviderDownloadFailed
|
||||||
|
}
|
||||||
|
format, ok := pagesarchive.DetectFormatFromBytes(head[:readBytes])
|
||||||
|
if !ok {
|
||||||
|
return "", safeLabel, errRemoteProviderUnsupported
|
||||||
|
}
|
||||||
|
return format, appendRemoteSourceLabelExtension(safeLabel, format), nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func remoteSourceLabel(parsed *url.URL) (string, pagesarchive.Format) {
|
||||||
|
baseName := path.Base(parsed.Path)
|
||||||
|
if baseName == "" || baseName == "." || baseName == "/" {
|
||||||
|
baseName = remoteSourceFallbackLabel
|
||||||
|
}
|
||||||
|
safeLabel := sanitizeRemoteSourceLabel(baseName)
|
||||||
|
format, _ := pagesarchive.DetectFormatFromName(safeLabel)
|
||||||
|
return limitRemoteSourceLabel(safeLabel, format), format
|
||||||
|
}
|
||||||
|
|
||||||
|
func sanitizeRemoteSourceLabel(label string) string {
|
||||||
|
var builder strings.Builder
|
||||||
|
lastReplacement := false
|
||||||
|
for _, character := range label {
|
||||||
|
if isRemoteSourceLabelCharacter(character) {
|
||||||
|
builder.WriteRune(character)
|
||||||
|
lastReplacement = false
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
if !lastReplacement {
|
||||||
|
builder.WriteByte('-')
|
||||||
|
lastReplacement = true
|
||||||
|
}
|
||||||
|
}
|
||||||
|
safeLabel := strings.TrimSpace(builder.String())
|
||||||
|
if safeLabel == "" || strings.Trim(safeLabel, "._-") == "" {
|
||||||
|
return remoteSourceFallbackLabel
|
||||||
|
}
|
||||||
|
return safeLabel
|
||||||
|
}
|
||||||
|
|
||||||
|
func isRemoteSourceLabelCharacter(character rune) bool {
|
||||||
|
return character >= 'a' && character <= 'z' ||
|
||||||
|
character >= 'A' && character <= 'Z' ||
|
||||||
|
character >= '0' && character <= '9' ||
|
||||||
|
character == '.' || character == '-' || character == '_'
|
||||||
|
}
|
||||||
|
|
||||||
|
func limitRemoteSourceLabel(label string, format pagesarchive.Format) string {
|
||||||
|
if len(label) <= remoteSourceMaxSafeLabelBytes {
|
||||||
|
return label
|
||||||
|
}
|
||||||
|
if format == "" {
|
||||||
|
return strings.TrimRight(label[:remoteSourceMaxSafeLabelBytes], ".-_")
|
||||||
|
}
|
||||||
|
extension := "." + pagesarchive.Extension(format)
|
||||||
|
prefixLength := remoteSourceMaxSafeLabelBytes - len(extension)
|
||||||
|
prefix := strings.TrimRight(label[:prefixLength], ".-_")
|
||||||
|
if prefix == "" {
|
||||||
|
prefix = remoteSourceFallbackLabel
|
||||||
|
}
|
||||||
|
return prefix + extension
|
||||||
|
}
|
||||||
|
|
||||||
|
func appendRemoteSourceLabelExtension(label string, format pagesarchive.Format) string {
|
||||||
|
extension := "." + pagesarchive.Extension(format)
|
||||||
|
maxPrefixLength := remoteSourceMaxSafeLabelBytes - len(extension)
|
||||||
|
if len(label) > maxPrefixLength {
|
||||||
|
label = strings.TrimRight(label[:maxPrefixLength], ".-_")
|
||||||
|
}
|
||||||
|
if label == "" {
|
||||||
|
label = remoteSourceFallbackLabel
|
||||||
|
}
|
||||||
|
return label + extension
|
||||||
|
}
|
||||||
|
|
||||||
|
func sanitizeRemoteProviderError(ctx context.Context, err error) error {
|
||||||
|
if ctxErr := ctx.Err(); ctxErr != nil {
|
||||||
|
return fmt.Errorf("%w: %w", errRemoteProviderDownloadFailed, ctxErr)
|
||||||
|
}
|
||||||
|
for _, safeError := range []error{
|
||||||
|
errRemoteProviderInvalidPolicy,
|
||||||
|
errRemoteProviderInvalidLimit,
|
||||||
|
errRemoteProviderBlockedAddress,
|
||||||
|
errRemoteProviderResolveFailed,
|
||||||
|
errRemoteProviderRedirectLimit,
|
||||||
|
errRemoteProviderTooLarge,
|
||||||
|
errRemoteProviderEmpty,
|
||||||
|
errRemoteProviderUnsupported,
|
||||||
|
errRemoteProviderCleanupFailed,
|
||||||
|
} {
|
||||||
|
if errors.Is(err, safeError) {
|
||||||
|
return safeError
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return errRemoteProviderDownloadFailed
|
||||||
|
}
|
||||||
@@ -0,0 +1,459 @@
|
|||||||
|
// Copyright 2026 Arctel.net
|
||||||
|
// SPDX-License-Identifier: Apache-2.0
|
||||||
|
|
||||||
|
package pages
|
||||||
|
|
||||||
|
import (
|
||||||
|
"archive/zip"
|
||||||
|
"bytes"
|
||||||
|
"context"
|
||||||
|
"crypto/sha256"
|
||||||
|
"encoding/hex"
|
||||||
|
"errors"
|
||||||
|
"io"
|
||||||
|
"log"
|
||||||
|
"net"
|
||||||
|
"net/http"
|
||||||
|
"net/http/httptest"
|
||||||
|
"net/netip"
|
||||||
|
"os"
|
||||||
|
"strconv"
|
||||||
|
"strings"
|
||||||
|
"sync/atomic"
|
||||||
|
"testing"
|
||||||
|
)
|
||||||
|
|
||||||
|
type remoteSourceResolverFunc func(context.Context, string, string) ([]netip.Addr, error)
|
||||||
|
|
||||||
|
func (function remoteSourceResolverFunc) LookupNetIP(
|
||||||
|
ctx context.Context,
|
||||||
|
network string,
|
||||||
|
host string,
|
||||||
|
) ([]netip.Addr, error) {
|
||||||
|
return function(ctx, network, host)
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestFetchRemoteSourceTrustedInternalSelfSignedAndSafeLabel(t *testing.T) {
|
||||||
|
packageBytes := makeRemoteSourceZIP(t)
|
||||||
|
server := httptest.NewTLSServer(http.HandlerFunc(func(writer http.ResponseWriter, request *http.Request) {
|
||||||
|
if request.URL.Query().Get("token") != "source-secret" {
|
||||||
|
t.Error("signed query did not reach the artifact server")
|
||||||
|
}
|
||||||
|
if request.Header.Get("Accept-Encoding") != "identity" {
|
||||||
|
t.Error("artifact request must disable automatic HTTP decompression")
|
||||||
|
}
|
||||||
|
writer.Header().Set("Content-Disposition", `attachment; filename="redirected.tar.gz"`)
|
||||||
|
_, _ = writer.Write(packageBytes)
|
||||||
|
}))
|
||||||
|
server.Config.ErrorLog = log.New(io.Discard, "", 0)
|
||||||
|
t.Cleanup(server.Close)
|
||||||
|
|
||||||
|
candidate, err := FetchRemoteSource(t.Context(), RemoteSourceRequest{
|
||||||
|
URL: server.URL + "/original/site.zip?token=source-secret",
|
||||||
|
NetworkPolicy: RemoteNetworkPolicyTrustedInternal,
|
||||||
|
MaxPackageBytes: int64(len(packageBytes) + 1),
|
||||||
|
})
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("FetchRemoteSource() error = %v", err)
|
||||||
|
}
|
||||||
|
if candidate.Format != "zip" {
|
||||||
|
t.Fatalf("Format = %q, want zip", candidate.Format)
|
||||||
|
}
|
||||||
|
if candidate.SafeLabel != "site.zip" {
|
||||||
|
t.Fatalf("SafeLabel = %q, want original path basename", candidate.SafeLabel)
|
||||||
|
}
|
||||||
|
if candidate.PackageSize != int64(len(packageBytes)) {
|
||||||
|
t.Fatalf("PackageSize = %d, want %d", candidate.PackageSize, len(packageBytes))
|
||||||
|
}
|
||||||
|
wantChecksum := sha256.Sum256(packageBytes)
|
||||||
|
if candidate.Checksum != hex.EncodeToString(wantChecksum[:]) {
|
||||||
|
t.Fatalf("Checksum = %q, want SHA-256", candidate.Checksum)
|
||||||
|
}
|
||||||
|
downloaded, err := os.ReadFile(candidate.TempPath) //nolint:gosec // provider-owned test temp file
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("ReadFile() error = %v", err)
|
||||||
|
}
|
||||||
|
if !bytes.Equal(downloaded, packageBytes) {
|
||||||
|
t.Fatal("downloaded package differs from response body")
|
||||||
|
}
|
||||||
|
tempPath := candidate.TempPath
|
||||||
|
if err := candidate.Cleanup(); err != nil {
|
||||||
|
t.Fatalf("Cleanup() error = %v", err)
|
||||||
|
}
|
||||||
|
if err := candidate.Cleanup(); err != nil {
|
||||||
|
t.Fatalf("second Cleanup() error = %v", err)
|
||||||
|
}
|
||||||
|
if _, err := os.Stat(tempPath); !errors.Is(err, os.ErrNotExist) {
|
||||||
|
t.Fatalf("temporary file still exists: %v", err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestFetchRemoteSourceKeepsOriginalLabelAcrossRedirect(t *testing.T) {
|
||||||
|
packageBytes := makeRemoteSourceZIP(t)
|
||||||
|
server := httptest.NewServer(http.HandlerFunc(func(writer http.ResponseWriter, request *http.Request) {
|
||||||
|
if request.URL.Path == "/original/site.zip" {
|
||||||
|
writer.Header().Set("Location", "/delivery/final.tar.gz?token=redirect-secret")
|
||||||
|
writer.WriteHeader(http.StatusFound)
|
||||||
|
return
|
||||||
|
}
|
||||||
|
if request.Header.Get("Referer") != "" {
|
||||||
|
t.Error("redirect must not forward a signed source URL as Referer")
|
||||||
|
}
|
||||||
|
writer.Header().Set("Content-Disposition", `attachment; filename="response.7z"`)
|
||||||
|
_, _ = writer.Write(packageBytes)
|
||||||
|
}))
|
||||||
|
t.Cleanup(server.Close)
|
||||||
|
|
||||||
|
candidate, err := FetchRemoteSource(t.Context(), RemoteSourceRequest{
|
||||||
|
URL: server.URL + "/original/site.zip?token=initial-secret",
|
||||||
|
NetworkPolicy: RemoteNetworkPolicyTrustedInternal,
|
||||||
|
MaxPackageBytes: int64(len(packageBytes) + 1),
|
||||||
|
})
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("FetchRemoteSource() error = %v", err)
|
||||||
|
}
|
||||||
|
defer func() { _ = candidate.Cleanup() }()
|
||||||
|
if candidate.SafeLabel != "site.zip" || candidate.Format != "zip" {
|
||||||
|
t.Fatalf("candidate = label %q format %q, want original site.zip", candidate.SafeLabel, candidate.Format)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestFetchRemoteSourcePublicRejectsNonPublicAddresses(t *testing.T) {
|
||||||
|
tests := []string{
|
||||||
|
"http://127.0.0.1/site.zip?token=loopback-secret",
|
||||||
|
"http://[::1]/site.zip?token=ipv6-secret",
|
||||||
|
"http://100.64.0.1/site.zip?token=cgnat-secret",
|
||||||
|
"http://192.0.2.1/site.zip?token=documentation-secret",
|
||||||
|
}
|
||||||
|
for _, rawURL := range tests {
|
||||||
|
t.Run(rawURL, func(t *testing.T) {
|
||||||
|
_, err := FetchRemoteSource(t.Context(), RemoteSourceRequest{
|
||||||
|
URL: rawURL,
|
||||||
|
NetworkPolicy: RemoteNetworkPolicyPublic,
|
||||||
|
MaxPackageBytes: 1024,
|
||||||
|
})
|
||||||
|
if !errors.Is(err, errRemoteProviderBlockedAddress) {
|
||||||
|
t.Fatalf("FetchRemoteSource() error = %v, want blocked address", err)
|
||||||
|
}
|
||||||
|
assertRemoteSourceErrorRedacted(t, err, rawURL, "secret", "token=")
|
||||||
|
})
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestFetchRemoteSourcePublicDialsValidatedIP(t *testing.T) {
|
||||||
|
packageBytes := makeRemoteSourceZIP(t)
|
||||||
|
server := httptest.NewServer(http.HandlerFunc(func(writer http.ResponseWriter, _ *http.Request) {
|
||||||
|
_, _ = writer.Write(packageBytes)
|
||||||
|
}))
|
||||||
|
t.Cleanup(server.Close)
|
||||||
|
|
||||||
|
var dialedAddress string
|
||||||
|
dependencies := mappedRemoteSourceDependencies(server.Listener.Addr().String(), staticPublicRemoteSourceResolver())
|
||||||
|
dependencies.dialContext = func(ctx context.Context, network string, address string) (net.Conn, error) {
|
||||||
|
dialedAddress = address
|
||||||
|
dialer := &net.Dialer{}
|
||||||
|
return dialer.DialContext(ctx, network, server.Listener.Addr().String())
|
||||||
|
}
|
||||||
|
candidate, err := fetchRemoteSource(t.Context(), RemoteSourceRequest{
|
||||||
|
URL: "http://artifact.example/site.zip",
|
||||||
|
NetworkPolicy: RemoteNetworkPolicyPublic,
|
||||||
|
MaxPackageBytes: int64(len(packageBytes) + 1),
|
||||||
|
}, dependencies)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("fetchRemoteSource() error = %v", err)
|
||||||
|
}
|
||||||
|
defer func() { _ = candidate.Cleanup() }()
|
||||||
|
if dialedAddress != "93.184.216.34:80" {
|
||||||
|
t.Fatalf("direct dial address = %q, want validated IP", dialedAddress)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestFetchRemoteSourcePublicRejectsPrivateRedirect(t *testing.T) {
|
||||||
|
var requestCount atomic.Int32
|
||||||
|
server := httptest.NewServer(http.HandlerFunc(func(writer http.ResponseWriter, _ *http.Request) {
|
||||||
|
requestCount.Add(1)
|
||||||
|
writer.Header().Set("Location", "http://127.0.0.1/private.zip?token=redirect-secret")
|
||||||
|
writer.WriteHeader(http.StatusFound)
|
||||||
|
}))
|
||||||
|
t.Cleanup(server.Close)
|
||||||
|
|
||||||
|
dependencies := mappedRemoteSourceDependencies(server.Listener.Addr().String(), staticPublicRemoteSourceResolver())
|
||||||
|
rawURL := "http://artifact.example/start.zip?token=initial-secret"
|
||||||
|
_, err := fetchRemoteSource(t.Context(), RemoteSourceRequest{
|
||||||
|
URL: rawURL,
|
||||||
|
NetworkPolicy: RemoteNetworkPolicyPublic,
|
||||||
|
MaxPackageBytes: 1024,
|
||||||
|
}, dependencies)
|
||||||
|
if !errors.Is(err, errRemoteProviderBlockedAddress) {
|
||||||
|
t.Fatalf("fetchRemoteSource() error = %v, want blocked redirect", err)
|
||||||
|
}
|
||||||
|
if requestCount.Load() != 1 {
|
||||||
|
t.Fatalf("request count = %d, private redirect must not be requested", requestCount.Load())
|
||||||
|
}
|
||||||
|
assertRemoteSourceErrorRedacted(t, err, rawURL, "initial-secret", "redirect-secret", "token=")
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestFetchRemoteSourcePublicRejectsDNSRebinding(t *testing.T) {
|
||||||
|
var lookupCount atomic.Int32
|
||||||
|
resolver := remoteSourceResolverFunc(func(context.Context, string, string) ([]netip.Addr, error) {
|
||||||
|
if lookupCount.Add(1) == 1 {
|
||||||
|
return []netip.Addr{netip.MustParseAddr("93.184.216.34")}, nil
|
||||||
|
}
|
||||||
|
return []netip.Addr{netip.MustParseAddr("127.0.0.1")}, nil
|
||||||
|
})
|
||||||
|
var dialCount atomic.Int32
|
||||||
|
dependencies := remoteSourceDependencies{
|
||||||
|
resolver: resolver,
|
||||||
|
dialContext: func(context.Context, string, string) (net.Conn, error) {
|
||||||
|
dialCount.Add(1)
|
||||||
|
return nil, errors.New("unexpected dial")
|
||||||
|
},
|
||||||
|
createTemp: os.CreateTemp,
|
||||||
|
}
|
||||||
|
rawURL := "http://rebind.example/site.zip?signature=dns-secret"
|
||||||
|
_, err := fetchRemoteSource(t.Context(), RemoteSourceRequest{
|
||||||
|
URL: rawURL,
|
||||||
|
NetworkPolicy: RemoteNetworkPolicyPublic,
|
||||||
|
MaxPackageBytes: 1024,
|
||||||
|
}, dependencies)
|
||||||
|
if !errors.Is(err, errRemoteProviderBlockedAddress) {
|
||||||
|
t.Fatalf("fetchRemoteSource() error = %v, want DNS rebinding rejection", err)
|
||||||
|
}
|
||||||
|
if lookupCount.Load() != 2 {
|
||||||
|
t.Fatalf("DNS lookup count = %d, want preflight plus dial validation", lookupCount.Load())
|
||||||
|
}
|
||||||
|
if dialCount.Load() != 0 {
|
||||||
|
t.Fatalf("direct dial count = %d, rebound address must not be dialed", dialCount.Load())
|
||||||
|
}
|
||||||
|
assertRemoteSourceErrorRedacted(t, err, rawURL, "dns-secret", "signature=")
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestFetchRemoteSourcePublicRejectsSelfSignedTLS(t *testing.T) {
|
||||||
|
packageBytes := makeRemoteSourceZIP(t)
|
||||||
|
server := httptest.NewTLSServer(http.HandlerFunc(func(writer http.ResponseWriter, _ *http.Request) {
|
||||||
|
_, _ = writer.Write(packageBytes)
|
||||||
|
}))
|
||||||
|
server.Config.ErrorLog = log.New(io.Discard, "", 0)
|
||||||
|
t.Cleanup(server.Close)
|
||||||
|
|
||||||
|
dependencies := mappedRemoteSourceDependencies(server.Listener.Addr().String(), staticPublicRemoteSourceResolver())
|
||||||
|
rawURL := "https://artifact.example/site.zip?signature=tls-secret"
|
||||||
|
_, err := fetchRemoteSource(t.Context(), RemoteSourceRequest{
|
||||||
|
URL: rawURL,
|
||||||
|
NetworkPolicy: RemoteNetworkPolicyPublic,
|
||||||
|
MaxPackageBytes: int64(len(packageBytes) + 1),
|
||||||
|
}, dependencies)
|
||||||
|
if !errors.Is(err, errRemoteProviderDownloadFailed) {
|
||||||
|
t.Fatalf("fetchRemoteSource() error = %v, want strict TLS failure", err)
|
||||||
|
}
|
||||||
|
assertRemoteSourceErrorRedacted(t, err, rawURL, "tls-secret", "signature=")
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestFetchRemoteSourceRejectsChunkedBodyOverLimitAndCleansTemp(t *testing.T) {
|
||||||
|
const maxPackageBytes = int64(64)
|
||||||
|
server := httptest.NewServer(http.HandlerFunc(func(writer http.ResponseWriter, _ *http.Request) {
|
||||||
|
_, _ = writer.Write(bytes.Repeat([]byte{'x'}, int(maxPackageBytes)))
|
||||||
|
if flusher, ok := writer.(http.Flusher); ok {
|
||||||
|
flusher.Flush()
|
||||||
|
}
|
||||||
|
_, _ = writer.Write([]byte("overflow"))
|
||||||
|
}))
|
||||||
|
t.Cleanup(server.Close)
|
||||||
|
|
||||||
|
tempDir := t.TempDir()
|
||||||
|
dependencies := defaultRemoteSourceDependenciesForTest()
|
||||||
|
dependencies.createTemp = func(_ string, pattern string) (*os.File, error) {
|
||||||
|
return os.CreateTemp(tempDir, pattern)
|
||||||
|
}
|
||||||
|
rawURL := server.URL + "/site.zip?token=chunk-secret"
|
||||||
|
_, err := fetchRemoteSource(t.Context(), RemoteSourceRequest{
|
||||||
|
URL: rawURL,
|
||||||
|
NetworkPolicy: RemoteNetworkPolicyTrustedInternal,
|
||||||
|
MaxPackageBytes: maxPackageBytes,
|
||||||
|
}, dependencies)
|
||||||
|
if !errors.Is(err, errRemoteProviderTooLarge) {
|
||||||
|
t.Fatalf("fetchRemoteSource() error = %v, want actual stream limit", err)
|
||||||
|
}
|
||||||
|
assertRemoteSourceTempDirEmpty(t, tempDir)
|
||||||
|
assertRemoteSourceErrorRedacted(t, err, rawURL, "chunk-secret", "token=")
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestFetchRemoteSourceRejectsContentLengthBeforeCreatingTemp(t *testing.T) {
|
||||||
|
server := httptest.NewServer(http.HandlerFunc(func(writer http.ResponseWriter, _ *http.Request) {
|
||||||
|
writer.Header().Set("Content-Length", "4096")
|
||||||
|
writer.WriteHeader(http.StatusOK)
|
||||||
|
}))
|
||||||
|
t.Cleanup(server.Close)
|
||||||
|
|
||||||
|
var createCount atomic.Int32
|
||||||
|
dependencies := defaultRemoteSourceDependenciesForTest()
|
||||||
|
dependencies.createTemp = func(directory string, pattern string) (*os.File, error) {
|
||||||
|
createCount.Add(1)
|
||||||
|
return os.CreateTemp(directory, pattern)
|
||||||
|
}
|
||||||
|
_, err := fetchRemoteSource(t.Context(), RemoteSourceRequest{
|
||||||
|
URL: server.URL + "/site.zip",
|
||||||
|
NetworkPolicy: RemoteNetworkPolicyTrustedInternal,
|
||||||
|
MaxPackageBytes: 1024,
|
||||||
|
}, dependencies)
|
||||||
|
if !errors.Is(err, errRemoteProviderTooLarge) {
|
||||||
|
t.Fatalf("fetchRemoteSource() error = %v, want Content-Length rejection", err)
|
||||||
|
}
|
||||||
|
if createCount.Load() != 0 {
|
||||||
|
t.Fatalf("CreateTemp called %d times before Content-Length rejection", createCount.Load())
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestFetchRemoteSourceSniffsAtLeast512BytesForTar(t *testing.T) {
|
||||||
|
packageBytes := make([]byte, remoteSourceMagicSniffBytes)
|
||||||
|
copy(packageBytes[257:], []byte("ustar"))
|
||||||
|
server := httptest.NewServer(http.HandlerFunc(func(writer http.ResponseWriter, _ *http.Request) {
|
||||||
|
_, _ = writer.Write(packageBytes)
|
||||||
|
}))
|
||||||
|
t.Cleanup(server.Close)
|
||||||
|
|
||||||
|
candidate, err := FetchRemoteSource(t.Context(), RemoteSourceRequest{
|
||||||
|
URL: server.URL + "/download",
|
||||||
|
NetworkPolicy: RemoteNetworkPolicyTrustedInternal,
|
||||||
|
MaxPackageBytes: int64(len(packageBytes) + 1),
|
||||||
|
})
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("FetchRemoteSource() error = %v", err)
|
||||||
|
}
|
||||||
|
defer func() { _ = candidate.Cleanup() }()
|
||||||
|
if candidate.Format != "tar" {
|
||||||
|
t.Fatalf("Format = %q, want tar detected at byte 257", candidate.Format)
|
||||||
|
}
|
||||||
|
if candidate.SafeLabel != "download.tar" {
|
||||||
|
t.Fatalf("SafeLabel = %q, want download.tar", candidate.SafeLabel)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestFetchRemoteSourceRedactsURLHeadersAndBodyFromErrors(t *testing.T) {
|
||||||
|
server := httptest.NewServer(http.HandlerFunc(func(writer http.ResponseWriter, _ *http.Request) {
|
||||||
|
writer.Header().Set("X-Artifact-Secret", "header-secret")
|
||||||
|
_, _ = writer.Write([]byte("response-body-secret"))
|
||||||
|
}))
|
||||||
|
t.Cleanup(server.Close)
|
||||||
|
|
||||||
|
tempDir := t.TempDir()
|
||||||
|
dependencies := defaultRemoteSourceDependenciesForTest()
|
||||||
|
dependencies.createTemp = func(_ string, pattern string) (*os.File, error) {
|
||||||
|
return os.CreateTemp(tempDir, pattern)
|
||||||
|
}
|
||||||
|
rawURL := server.URL + "/download?token=query-secret"
|
||||||
|
_, err := fetchRemoteSource(t.Context(), RemoteSourceRequest{
|
||||||
|
URL: rawURL,
|
||||||
|
NetworkPolicy: RemoteNetworkPolicyTrustedInternal,
|
||||||
|
MaxPackageBytes: 1024,
|
||||||
|
}, dependencies)
|
||||||
|
if !errors.Is(err, errRemoteProviderUnsupported) {
|
||||||
|
t.Fatalf("fetchRemoteSource() error = %v, want unsupported archive", err)
|
||||||
|
}
|
||||||
|
assertRemoteSourceTempDirEmpty(t, tempDir)
|
||||||
|
assertRemoteSourceErrorRedacted(
|
||||||
|
t,
|
||||||
|
err,
|
||||||
|
rawURL,
|
||||||
|
"query-secret",
|
||||||
|
"header-secret",
|
||||||
|
"response-body-secret",
|
||||||
|
"token=",
|
||||||
|
)
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestFetchRemoteSourceAllowsFiveRedirectsOnly(t *testing.T) {
|
||||||
|
packageBytes := makeRemoteSourceZIP(t)
|
||||||
|
var requestCount atomic.Int32
|
||||||
|
server := httptest.NewServer(http.HandlerFunc(func(writer http.ResponseWriter, request *http.Request) {
|
||||||
|
requestCount.Add(1)
|
||||||
|
redirectNumber, _ := strconv.Atoi(strings.TrimPrefix(request.URL.Path, "/"))
|
||||||
|
if redirectNumber < remoteSourceMaxRedirects+1 {
|
||||||
|
writer.Header().Set("Location", "/"+strconv.Itoa(redirectNumber+1))
|
||||||
|
writer.WriteHeader(http.StatusFound)
|
||||||
|
return
|
||||||
|
}
|
||||||
|
_, _ = writer.Write(packageBytes)
|
||||||
|
}))
|
||||||
|
t.Cleanup(server.Close)
|
||||||
|
|
||||||
|
dependencies := mappedRemoteSourceDependencies(server.Listener.Addr().String(), staticPublicRemoteSourceResolver())
|
||||||
|
_, err := fetchRemoteSource(t.Context(), RemoteSourceRequest{
|
||||||
|
URL: "http://artifact.example/0",
|
||||||
|
NetworkPolicy: RemoteNetworkPolicyPublic,
|
||||||
|
MaxPackageBytes: int64(len(packageBytes) + 1),
|
||||||
|
}, dependencies)
|
||||||
|
if !errors.Is(err, errRemoteProviderRedirectLimit) {
|
||||||
|
t.Fatalf("fetchRemoteSource() error = %v, want redirect limit", err)
|
||||||
|
}
|
||||||
|
if requestCount.Load() != remoteSourceMaxRedirects+1 {
|
||||||
|
t.Fatalf("request count = %d, want initial plus five redirects", requestCount.Load())
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func staticPublicRemoteSourceResolver() remoteSourceResolver {
|
||||||
|
return remoteSourceResolverFunc(func(context.Context, string, string) ([]netip.Addr, error) {
|
||||||
|
return []netip.Addr{netip.MustParseAddr("93.184.216.34")}, nil
|
||||||
|
})
|
||||||
|
}
|
||||||
|
|
||||||
|
func defaultRemoteSourceDependenciesForTest() remoteSourceDependencies {
|
||||||
|
dialer := &net.Dialer{}
|
||||||
|
return remoteSourceDependencies{
|
||||||
|
resolver: net.DefaultResolver,
|
||||||
|
dialContext: dialer.DialContext,
|
||||||
|
createTemp: os.CreateTemp,
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func mappedRemoteSourceDependencies(targetAddress string, resolver remoteSourceResolver) remoteSourceDependencies {
|
||||||
|
dialer := &net.Dialer{}
|
||||||
|
return remoteSourceDependencies{
|
||||||
|
resolver: resolver,
|
||||||
|
dialContext: func(ctx context.Context, network string, _ string) (net.Conn, error) {
|
||||||
|
return dialer.DialContext(ctx, network, targetAddress)
|
||||||
|
},
|
||||||
|
createTemp: os.CreateTemp,
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func makeRemoteSourceZIP(t *testing.T) []byte {
|
||||||
|
t.Helper()
|
||||||
|
var buffer bytes.Buffer
|
||||||
|
archive := zip.NewWriter(&buffer)
|
||||||
|
file, err := archive.Create("index.html")
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("zip.Create() error = %v", err)
|
||||||
|
}
|
||||||
|
if _, err := file.Write([]byte("<h1>OpenFlare</h1>")); err != nil {
|
||||||
|
t.Fatalf("zip entry Write() error = %v", err)
|
||||||
|
}
|
||||||
|
if err := archive.Close(); err != nil {
|
||||||
|
t.Fatalf("zip.Close() error = %v", err)
|
||||||
|
}
|
||||||
|
return buffer.Bytes()
|
||||||
|
}
|
||||||
|
|
||||||
|
func assertRemoteSourceTempDirEmpty(t *testing.T, directory string) {
|
||||||
|
t.Helper()
|
||||||
|
entries, err := os.ReadDir(directory)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("ReadDir() error = %v", err)
|
||||||
|
}
|
||||||
|
if len(entries) != 0 {
|
||||||
|
t.Fatalf("temporary directory contains %d leaked files", len(entries))
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func assertRemoteSourceErrorRedacted(t *testing.T, err error, sensitiveValues ...string) {
|
||||||
|
t.Helper()
|
||||||
|
if err == nil {
|
||||||
|
t.Fatal("expected an error")
|
||||||
|
}
|
||||||
|
message := err.Error()
|
||||||
|
for _, sensitiveValue := range sensitiveValues {
|
||||||
|
if sensitiveValue != "" && strings.Contains(message, sensitiveValue) {
|
||||||
|
t.Fatalf("error %q contains sensitive value %q", message, sensitiveValue)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -0,0 +1,371 @@
|
|||||||
|
// Copyright 2026 Arctel.net
|
||||||
|
// SPDX-License-Identifier: Apache-2.0
|
||||||
|
|
||||||
|
package pages
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
"crypto/rand"
|
||||||
|
"encoding/hex"
|
||||||
|
"errors"
|
||||||
|
"strings"
|
||||||
|
"time"
|
||||||
|
|
||||||
|
"github.com/Rain-kl/Wavelet/internal/db"
|
||||||
|
"github.com/Rain-kl/Wavelet/internal/model"
|
||||||
|
"gorm.io/gorm"
|
||||||
|
"gorm.io/gorm/clause"
|
||||||
|
)
|
||||||
|
|
||||||
|
const (
|
||||||
|
pagesSourceCheckLeaseDuration = 2 * time.Minute
|
||||||
|
pagesSourceSyncLeaseDuration = 15 * time.Minute
|
||||||
|
sourceLeaseTokenBytes = 32
|
||||||
|
sourceRuntimeErrorMaxBytes = 512
|
||||||
|
sourceRevisionHexLength = 64
|
||||||
|
|
||||||
|
sourceColumnAutoUpdateEnabled = "auto_update_enabled"
|
||||||
|
sourceColumnConfigVersion = "config_version"
|
||||||
|
sourceRuntimeColumnSyncStatus = "sync_status"
|
||||||
|
sourceRuntimeColumnLastError = "last_error"
|
||||||
|
sourceRuntimeColumnLastCheckedAt = "last_checked_at"
|
||||||
|
sourceRuntimeColumnNextCheckAt = "next_check_at"
|
||||||
|
sourceRuntimeColumnLeaseToken = "lease_token"
|
||||||
|
sourceRuntimeColumnLeaseExpiresAt = "lease_expires_at"
|
||||||
|
pagesDeploymentColumnStatus = "status"
|
||||||
|
)
|
||||||
|
|
||||||
|
type sourceLeaseOutcome string
|
||||||
|
|
||||||
|
const (
|
||||||
|
sourceLeaseAcquired sourceLeaseOutcome = "acquired"
|
||||||
|
sourceLeaseBusy sourceLeaseOutcome = "busy"
|
||||||
|
sourceLeaseStale sourceLeaseOutcome = "stale"
|
||||||
|
)
|
||||||
|
|
||||||
|
// sourceExecutionSnapshot captures every mutable value that can affect archive
|
||||||
|
// validation or the atomic activation decision. The queued payload deliberately
|
||||||
|
// does not carry project content configuration.
|
||||||
|
type sourceExecutionSnapshot struct {
|
||||||
|
ProjectID uint
|
||||||
|
SourceID uint
|
||||||
|
SourceConfigVersion int
|
||||||
|
ContentConfigVersion int
|
||||||
|
SourceType string
|
||||||
|
SourceIdentity string
|
||||||
|
RemoteURL string
|
||||||
|
RemoteNetworkPolicy string
|
||||||
|
GitHubRepository string
|
||||||
|
ReleaseSelector string
|
||||||
|
ReleaseTag string
|
||||||
|
AssetName string
|
||||||
|
AutoUpdateEnabled bool
|
||||||
|
CheckIntervalMinutes int
|
||||||
|
ETag string
|
||||||
|
LastSeenRevision string
|
||||||
|
LastSeenDetail string
|
||||||
|
LastAppliedRevision string
|
||||||
|
LastAppliedDetail string
|
||||||
|
RootDir string
|
||||||
|
EntryFile string
|
||||||
|
LeaseToken string
|
||||||
|
LeaseExpiresAt time.Time
|
||||||
|
}
|
||||||
|
|
||||||
|
func acquireSourceLease(
|
||||||
|
ctx context.Context,
|
||||||
|
sourceID uint,
|
||||||
|
expectedConfigVersion int,
|
||||||
|
action string,
|
||||||
|
) (*sourceExecutionSnapshot, sourceLeaseOutcome, error) {
|
||||||
|
leaseDuration, status, err := sourceLeaseParameters(action)
|
||||||
|
if err != nil {
|
||||||
|
return nil, sourceLeaseStale, err
|
||||||
|
}
|
||||||
|
token, err := newSourceLeaseToken()
|
||||||
|
if err != nil {
|
||||||
|
return nil, sourceLeaseStale, err
|
||||||
|
}
|
||||||
|
now := time.Now()
|
||||||
|
expiresAt := now.Add(leaseDuration)
|
||||||
|
|
||||||
|
result := db.DB(ctx).Model(&model.PagesProjectSourceRuntime{}).
|
||||||
|
Where("source_id = ?", sourceID).
|
||||||
|
Where("lease_expires_at IS NULL OR lease_expires_at <= ?", now).
|
||||||
|
Where(
|
||||||
|
"EXISTS (SELECT 1 FROM of_pages_project_sources source WHERE source.id = ? AND source.config_version = ?)",
|
||||||
|
sourceID,
|
||||||
|
expectedConfigVersion,
|
||||||
|
).
|
||||||
|
Updates(map[string]any{
|
||||||
|
sourceRuntimeColumnLeaseToken: token,
|
||||||
|
sourceRuntimeColumnLeaseExpiresAt: expiresAt,
|
||||||
|
sourceRuntimeColumnSyncStatus: status,
|
||||||
|
sourceRuntimeColumnLastError: "",
|
||||||
|
})
|
||||||
|
if result.Error != nil {
|
||||||
|
return nil, sourceLeaseStale, result.Error
|
||||||
|
}
|
||||||
|
if result.RowsAffected == 0 {
|
||||||
|
outcome, inspectErr := inspectSourceLeaseMiss(ctx, sourceID, expectedConfigVersion, now)
|
||||||
|
return nil, outcome, inspectErr
|
||||||
|
}
|
||||||
|
|
||||||
|
snapshot, err := loadSourceExecutionSnapshot(ctx, sourceID, token)
|
||||||
|
if err != nil {
|
||||||
|
if errors.Is(err, errSourceLeaseSnapshotStale) {
|
||||||
|
return nil, sourceLeaseStale, nil
|
||||||
|
}
|
||||||
|
return nil, sourceLeaseStale, err
|
||||||
|
}
|
||||||
|
return snapshot, sourceLeaseAcquired, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
var errSourceLeaseSnapshotStale = errors.New("source lease snapshot stale")
|
||||||
|
|
||||||
|
func loadSourceExecutionSnapshot(
|
||||||
|
ctx context.Context,
|
||||||
|
sourceID uint,
|
||||||
|
token string,
|
||||||
|
) (*sourceExecutionSnapshot, error) {
|
||||||
|
var snapshot sourceExecutionSnapshot
|
||||||
|
err := db.DB(ctx).Transaction(func(tx *gorm.DB) error {
|
||||||
|
var source model.PagesProjectSource
|
||||||
|
if err := tx.Where("id = ?", sourceID).First(&source).Error; err != nil {
|
||||||
|
if errors.Is(err, gorm.ErrRecordNotFound) {
|
||||||
|
return errSourceLeaseSnapshotStale
|
||||||
|
}
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
var project model.PagesProject
|
||||||
|
if err := tx.Clauses(clause.Locking{Strength: pagesRowLockStrength}).
|
||||||
|
First(&project, source.ProjectID).Error; err != nil {
|
||||||
|
if errors.Is(err, gorm.ErrRecordNotFound) {
|
||||||
|
return errSourceLeaseSnapshotStale
|
||||||
|
}
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
if err := tx.Clauses(clause.Locking{Strength: pagesRowLockStrength}).
|
||||||
|
Where("id = ?", sourceID).
|
||||||
|
First(&source).Error; err != nil {
|
||||||
|
if errors.Is(err, gorm.ErrRecordNotFound) {
|
||||||
|
return errSourceLeaseSnapshotStale
|
||||||
|
}
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
var runtime model.PagesProjectSourceRuntime
|
||||||
|
if err := tx.Clauses(clause.Locking{Strength: pagesRowLockStrength}).
|
||||||
|
Where("source_id = ?", source.ID).
|
||||||
|
First(&runtime).Error; err != nil {
|
||||||
|
if errors.Is(err, gorm.ErrRecordNotFound) {
|
||||||
|
return errSourceLeaseSnapshotStale
|
||||||
|
}
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
// 必须在 runtime 行锁拿到后重新取时间,避免锁等待跨过
|
||||||
|
// lease expiry 时仍使用事务开始前的旧时间继续执行。
|
||||||
|
now := time.Now()
|
||||||
|
if runtime.LeaseToken != token || runtime.LeaseExpiresAt == nil || !runtime.LeaseExpiresAt.After(now) {
|
||||||
|
return errSourceLeaseSnapshotStale
|
||||||
|
}
|
||||||
|
snapshot = sourceExecutionSnapshot{
|
||||||
|
ProjectID: project.ID,
|
||||||
|
SourceID: source.ID,
|
||||||
|
SourceConfigVersion: source.ConfigVersion,
|
||||||
|
ContentConfigVersion: project.ContentConfigVersion,
|
||||||
|
SourceType: source.SourceType,
|
||||||
|
SourceIdentity: source.SourceIdentity,
|
||||||
|
RemoteURL: source.RemoteURL,
|
||||||
|
RemoteNetworkPolicy: source.RemoteNetworkPolicy,
|
||||||
|
GitHubRepository: source.GitHubRepository,
|
||||||
|
ReleaseSelector: source.ReleaseSelector,
|
||||||
|
ReleaseTag: source.ReleaseTag,
|
||||||
|
AssetName: source.AssetName,
|
||||||
|
AutoUpdateEnabled: source.AutoUpdateEnabled,
|
||||||
|
CheckIntervalMinutes: source.CheckIntervalMinutes,
|
||||||
|
ETag: runtime.ETag,
|
||||||
|
LastSeenRevision: runtime.LastSeenRevision,
|
||||||
|
LastSeenDetail: runtime.LastSeenDetail,
|
||||||
|
LastAppliedRevision: runtime.LastAppliedRevision,
|
||||||
|
LastAppliedDetail: runtime.LastAppliedDetail,
|
||||||
|
RootDir: project.RootDir,
|
||||||
|
EntryFile: project.EntryFile,
|
||||||
|
LeaseToken: token,
|
||||||
|
LeaseExpiresAt: *runtime.LeaseExpiresAt,
|
||||||
|
}
|
||||||
|
return nil
|
||||||
|
})
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
return &snapshot, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func inspectSourceLeaseMiss(
|
||||||
|
ctx context.Context,
|
||||||
|
sourceID uint,
|
||||||
|
expectedConfigVersion int,
|
||||||
|
now time.Time,
|
||||||
|
) (sourceLeaseOutcome, error) {
|
||||||
|
var source model.PagesProjectSource
|
||||||
|
if err := db.DB(ctx).Where("id = ?", sourceID).First(&source).Error; err != nil {
|
||||||
|
if errors.Is(err, gorm.ErrRecordNotFound) {
|
||||||
|
return sourceLeaseStale, nil
|
||||||
|
}
|
||||||
|
return sourceLeaseStale, err
|
||||||
|
}
|
||||||
|
if source.ConfigVersion != expectedConfigVersion {
|
||||||
|
return sourceLeaseStale, nil
|
||||||
|
}
|
||||||
|
var runtime model.PagesProjectSourceRuntime
|
||||||
|
if err := db.DB(ctx).Where("source_id = ?", sourceID).First(&runtime).Error; err != nil {
|
||||||
|
if errors.Is(err, gorm.ErrRecordNotFound) {
|
||||||
|
return sourceLeaseStale, nil
|
||||||
|
}
|
||||||
|
return sourceLeaseStale, err
|
||||||
|
}
|
||||||
|
if runtime.LeaseExpiresAt != nil && runtime.LeaseExpiresAt.After(now) {
|
||||||
|
return sourceLeaseBusy, nil
|
||||||
|
}
|
||||||
|
return sourceLeaseStale, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func sourceLeaseParameters(action string) (time.Duration, string, error) {
|
||||||
|
switch action {
|
||||||
|
case sourceActionCheck:
|
||||||
|
return pagesSourceCheckLeaseDuration, pagesSourceStatusChecking, nil
|
||||||
|
case sourceActionSync:
|
||||||
|
return pagesSourceSyncLeaseDuration, pagesSourceStatusSyncing, nil
|
||||||
|
default:
|
||||||
|
return 0, "", errors.New(errPagesSourceActionInvalid)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func newSourceLeaseToken() (string, error) {
|
||||||
|
value := make([]byte, sourceLeaseTokenBytes)
|
||||||
|
if _, err := rand.Read(value); err != nil {
|
||||||
|
return "", err
|
||||||
|
}
|
||||||
|
return hex.EncodeToString(value), nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func renewSourceLease(ctx context.Context, snapshot *sourceExecutionSnapshot, duration time.Duration) (bool, error) {
|
||||||
|
if snapshot == nil || snapshot.SourceID == 0 || snapshot.LeaseToken == "" || duration <= 0 {
|
||||||
|
return false, errors.New(errPagesSourceLeaseLost)
|
||||||
|
}
|
||||||
|
now := time.Now()
|
||||||
|
expiresAt := now.Add(duration)
|
||||||
|
result := db.DB(ctx).Model(&model.PagesProjectSourceRuntime{}).
|
||||||
|
Where("source_id = ? AND lease_token = ? AND lease_expires_at > ?", snapshot.SourceID, snapshot.LeaseToken, now).
|
||||||
|
Updates(map[string]any{sourceRuntimeColumnLeaseExpiresAt: expiresAt})
|
||||||
|
if result.Error != nil {
|
||||||
|
return false, result.Error
|
||||||
|
}
|
||||||
|
if result.RowsAffected == 0 {
|
||||||
|
return false, nil
|
||||||
|
}
|
||||||
|
snapshot.LeaseExpiresAt = expiresAt
|
||||||
|
return true, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func failSourceLease(ctx context.Context, snapshot *sourceExecutionSnapshot, message string) error {
|
||||||
|
if snapshot == nil || snapshot.SourceID == 0 || snapshot.LeaseToken == "" {
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
message = safeSourceRuntimeError(message)
|
||||||
|
now := time.Now()
|
||||||
|
return db.DB(ctx).Model(&model.PagesProjectSourceRuntime{}).
|
||||||
|
Where("source_id = ? AND lease_token = ? AND lease_expires_at > ?", snapshot.SourceID, snapshot.LeaseToken, now).
|
||||||
|
Updates(map[string]any{
|
||||||
|
sourceRuntimeColumnSyncStatus: pagesSourceStatusFailed,
|
||||||
|
sourceRuntimeColumnLastError: message,
|
||||||
|
sourceRuntimeColumnLeaseToken: "",
|
||||||
|
sourceRuntimeColumnLeaseExpiresAt: nil,
|
||||||
|
}).Error
|
||||||
|
}
|
||||||
|
|
||||||
|
func safeSourceRuntimeError(message string) string {
|
||||||
|
message = strings.TrimSpace(message)
|
||||||
|
if message == "" {
|
||||||
|
return errPagesSourceSyncFailed
|
||||||
|
}
|
||||||
|
if len(message) > sourceRuntimeErrorMaxBytes {
|
||||||
|
message = message[:sourceRuntimeErrorMaxBytes]
|
||||||
|
}
|
||||||
|
return message
|
||||||
|
}
|
||||||
|
|
||||||
|
func sourceLeaseIsBusy(ctx context.Context, sourceID uint) (bool, error) {
|
||||||
|
var runtime model.PagesProjectSourceRuntime
|
||||||
|
if err := db.DB(ctx).Where("source_id = ?", sourceID).First(&runtime).Error; err != nil {
|
||||||
|
return false, err
|
||||||
|
}
|
||||||
|
return runtime.LeaseExpiresAt != nil && runtime.LeaseExpiresAt.After(time.Now()), nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// recoverExpiredSourceLease clears one exact expired lease owner. Matching the
|
||||||
|
// token, observed expiry and status prevents a scanner from overwriting a
|
||||||
|
// worker that renewed or was replaced after the candidate query.
|
||||||
|
func recoverExpiredSourceLease(
|
||||||
|
ctx context.Context,
|
||||||
|
sourceID uint,
|
||||||
|
token string,
|
||||||
|
expiresAt time.Time,
|
||||||
|
status string,
|
||||||
|
now time.Time,
|
||||||
|
nextCheckAt *time.Time,
|
||||||
|
) (bool, error) {
|
||||||
|
if sourceID == 0 || token == "" ||
|
||||||
|
(status != pagesSourceStatusChecking && status != pagesSourceStatusSyncing) {
|
||||||
|
return false, nil
|
||||||
|
}
|
||||||
|
updates := map[string]any{
|
||||||
|
sourceRuntimeColumnSyncStatus: pagesSourceStatusFailed,
|
||||||
|
sourceRuntimeColumnLastError: errPagesSourceLeaseExpired,
|
||||||
|
sourceRuntimeColumnLeaseToken: "",
|
||||||
|
sourceRuntimeColumnLeaseExpiresAt: nil,
|
||||||
|
sourceRuntimeColumnNextCheckAt: nextCheckAt,
|
||||||
|
}
|
||||||
|
result := db.DB(ctx).Model(&model.PagesProjectSourceRuntime{}).
|
||||||
|
Where("source_id = ?", sourceID).
|
||||||
|
Where("lease_token = ?", token).
|
||||||
|
Where("lease_expires_at = ? AND lease_expires_at <= ?", expiresAt, now).
|
||||||
|
Where("sync_status = ?", status).
|
||||||
|
Updates(updates)
|
||||||
|
if result.Error != nil {
|
||||||
|
return false, result.Error
|
||||||
|
}
|
||||||
|
return result.RowsAffected == 1, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// fenceAndNormalizeRuntime invalidates in-flight work while preserving safe
|
||||||
|
// seen/applied cursors. The caller must already hold the source row lock.
|
||||||
|
func fenceAndNormalizeRuntime(tx *gorm.DB, sourceID uint) error {
|
||||||
|
var runtime model.PagesProjectSourceRuntime
|
||||||
|
if err := tx.Clauses(clause.Locking{Strength: pagesRowLockStrength}).
|
||||||
|
Where("source_id = ?", sourceID).
|
||||||
|
First(&runtime).Error; err != nil {
|
||||||
|
if errors.Is(err, gorm.ErrRecordNotFound) {
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
return tx.Model(&runtime).Updates(map[string]any{
|
||||||
|
sourceRuntimeColumnLeaseToken: "",
|
||||||
|
sourceRuntimeColumnLeaseExpiresAt: nil,
|
||||||
|
sourceRuntimeColumnSyncStatus: normalizedSourceRuntimeStatus(&runtime),
|
||||||
|
}).Error
|
||||||
|
}
|
||||||
|
|
||||||
|
func sourceHasSameReleaseReplacement(runtime *model.PagesProjectSourceRuntime) bool {
|
||||||
|
if runtime == nil || runtime.LastSeenRevision == "" || runtime.LastSeenRevision == runtime.LastAppliedRevision {
|
||||||
|
return false
|
||||||
|
}
|
||||||
|
seen := sourceDetail{}
|
||||||
|
applied := sourceDetail{}
|
||||||
|
if unmarshalSourceDetail(runtime.LastSeenDetail, &seen) != nil ||
|
||||||
|
unmarshalSourceDetail(runtime.LastAppliedDetail, &applied) != nil {
|
||||||
|
return false
|
||||||
|
}
|
||||||
|
return seen.ReleaseID != "" && seen.ReleaseID == applied.ReleaseID
|
||||||
|
}
|
||||||
@@ -0,0 +1,350 @@
|
|||||||
|
// Copyright 2026 Arctel.net
|
||||||
|
// SPDX-License-Identifier: Apache-2.0
|
||||||
|
|
||||||
|
package pages
|
||||||
|
|
||||||
|
import (
|
||||||
|
"errors"
|
||||||
|
"strings"
|
||||||
|
"sync"
|
||||||
|
"testing"
|
||||||
|
"time"
|
||||||
|
|
||||||
|
"github.com/Rain-kl/Wavelet/internal/db"
|
||||||
|
"github.com/Rain-kl/Wavelet/internal/model"
|
||||||
|
)
|
||||||
|
|
||||||
|
func TestSourceLeaseHeartbeatRenewsAndCancelsOnOwnershipLoss(t *testing.T) {
|
||||||
|
ctx := setupPagesSourceTest(t)
|
||||||
|
project := mustCreatePagesSourceProject(t, ctx, "lease-heartbeat")
|
||||||
|
source, _ := mustConfigureRemoteSource(
|
||||||
|
t,
|
||||||
|
ctx,
|
||||||
|
project.ID,
|
||||||
|
"https://example.com/site.zip",
|
||||||
|
RemoteNetworkPolicyPublic,
|
||||||
|
)
|
||||||
|
snapshot, outcome, err := acquireSourceLease(ctx, source.ID, source.ConfigVersion, sourceActionSync)
|
||||||
|
if err != nil || outcome != sourceLeaseAcquired || snapshot == nil {
|
||||||
|
t.Fatalf("acquireSourceLease(heartbeat) = (%+v, %q, %v), want acquired", snapshot, outcome, err)
|
||||||
|
}
|
||||||
|
|
||||||
|
workCtx, heartbeat, err := startSourceLeaseHeartbeat(ctx, snapshot, 500*time.Millisecond, 20*time.Millisecond)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("startSourceLeaseHeartbeat() error = %v, want nil", err)
|
||||||
|
}
|
||||||
|
t.Cleanup(func() { _ = heartbeat.stop() })
|
||||||
|
|
||||||
|
var initial model.PagesProjectSourceRuntime
|
||||||
|
if err := db.DB(ctx).Where("source_id = ?", source.ID).First(&initial).Error; err != nil {
|
||||||
|
t.Fatalf("load initial heartbeat runtime error = %v, want nil", err)
|
||||||
|
}
|
||||||
|
if initial.LeaseExpiresAt == nil {
|
||||||
|
t.Fatal("initial heartbeat expiry = nil, want non-nil")
|
||||||
|
}
|
||||||
|
deadline := time.Now().Add(2 * time.Second)
|
||||||
|
for {
|
||||||
|
var renewedRuntime model.PagesProjectSourceRuntime
|
||||||
|
if err := db.DB(ctx).Where("source_id = ?", source.ID).First(&renewedRuntime).Error; err != nil {
|
||||||
|
t.Fatalf("load renewed heartbeat runtime error = %v, want nil", err)
|
||||||
|
}
|
||||||
|
if renewedRuntime.LeaseExpiresAt != nil && renewedRuntime.LeaseExpiresAt.After(*initial.LeaseExpiresAt) {
|
||||||
|
break
|
||||||
|
}
|
||||||
|
if time.Now().After(deadline) {
|
||||||
|
t.Fatal("heartbeat did not extend lease before deadline")
|
||||||
|
}
|
||||||
|
time.Sleep(10 * time.Millisecond)
|
||||||
|
}
|
||||||
|
|
||||||
|
if err := db.DB(ctx).Model(&model.PagesProjectSourceRuntime{}).
|
||||||
|
Where("source_id = ?", source.ID).
|
||||||
|
Update("lease_token", "replacement-owner").Error; err != nil {
|
||||||
|
t.Fatalf("replace heartbeat lease owner error = %v, want nil", err)
|
||||||
|
}
|
||||||
|
select {
|
||||||
|
case <-workCtx.Done():
|
||||||
|
case <-time.After(2 * time.Second):
|
||||||
|
t.Fatal("heartbeat work context was not canceled after ownership loss")
|
||||||
|
}
|
||||||
|
if err := heartbeat.stop(); !errors.Is(err, errSourceLeaseHeartbeatLost) {
|
||||||
|
t.Fatalf("heartbeat.stop() error = %v, want %v", err, errSourceLeaseHeartbeatLost)
|
||||||
|
}
|
||||||
|
if err := heartbeat.stop(); !errors.Is(err, errSourceLeaseHeartbeatLost) {
|
||||||
|
t.Fatalf("heartbeat.stop() second error = %v, want stable %v", err, errSourceLeaseHeartbeatLost)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestAcquireSourceLeaseConcurrentOnlyOneOwner(t *testing.T) {
|
||||||
|
ctx := setupPagesSourceTest(t)
|
||||||
|
project := mustCreatePagesSourceProject(t, ctx, "lease-concurrent")
|
||||||
|
source, _ := mustConfigureRemoteSource(
|
||||||
|
t,
|
||||||
|
ctx,
|
||||||
|
project.ID,
|
||||||
|
"https://example.com/site.zip",
|
||||||
|
RemoteNetworkPolicyPublic,
|
||||||
|
)
|
||||||
|
type leaseResult struct {
|
||||||
|
snapshot *sourceExecutionSnapshot
|
||||||
|
outcome sourceLeaseOutcome
|
||||||
|
err error
|
||||||
|
}
|
||||||
|
results := make(chan leaseResult, 2)
|
||||||
|
var workers sync.WaitGroup
|
||||||
|
workers.Add(2)
|
||||||
|
for range 2 {
|
||||||
|
go func() {
|
||||||
|
defer workers.Done()
|
||||||
|
snapshot, outcome, err := acquireSourceLease(ctx, source.ID, source.ConfigVersion, sourceActionSync)
|
||||||
|
results <- leaseResult{snapshot: snapshot, outcome: outcome, err: err}
|
||||||
|
}()
|
||||||
|
}
|
||||||
|
workers.Wait()
|
||||||
|
close(results)
|
||||||
|
|
||||||
|
acquired := 0
|
||||||
|
busy := 0
|
||||||
|
for result := range results {
|
||||||
|
if result.err != nil {
|
||||||
|
t.Errorf("acquireSourceLease(concurrent) error = %v, want nil", result.err)
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
switch result.outcome {
|
||||||
|
case sourceLeaseAcquired:
|
||||||
|
acquired++
|
||||||
|
if result.snapshot == nil || result.snapshot.LeaseToken == "" {
|
||||||
|
t.Errorf("acquireSourceLease(concurrent acquired) snapshot = %+v, want token-bearing snapshot", result.snapshot)
|
||||||
|
}
|
||||||
|
case sourceLeaseBusy:
|
||||||
|
busy++
|
||||||
|
if result.snapshot != nil {
|
||||||
|
t.Errorf("acquireSourceLease(concurrent busy) snapshot = %+v, want nil", result.snapshot)
|
||||||
|
}
|
||||||
|
default:
|
||||||
|
t.Errorf("acquireSourceLease(concurrent) outcome = %q, want acquired or busy", result.outcome)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if acquired != 1 || busy != 1 {
|
||||||
|
t.Errorf("concurrent lease outcomes = acquired:%d busy:%d, want 1 and 1", acquired, busy)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestAcquireSourceLeaseMutualExclusionExpiryAndTerminalOwnership(t *testing.T) {
|
||||||
|
ctx := setupPagesSourceTest(t)
|
||||||
|
project := mustCreatePagesSourceProject(t, ctx, "lease-cas")
|
||||||
|
source, _ := mustConfigureRemoteSource(
|
||||||
|
t,
|
||||||
|
ctx,
|
||||||
|
project.ID,
|
||||||
|
"https://example.com/site.zip",
|
||||||
|
RemoteNetworkPolicyPublic,
|
||||||
|
)
|
||||||
|
|
||||||
|
first, outcome, err := acquireSourceLease(ctx, source.ID, source.ConfigVersion, sourceActionSync)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("acquireSourceLease(first) error = %v, want nil", err)
|
||||||
|
}
|
||||||
|
if got, want := outcome, sourceLeaseAcquired; got != want {
|
||||||
|
t.Fatalf("acquireSourceLease(first) outcome = %q, want %q", got, want)
|
||||||
|
}
|
||||||
|
if first == nil || first.LeaseToken == "" {
|
||||||
|
t.Fatalf("acquireSourceLease(first) snapshot = %+v, want token-bearing snapshot", first)
|
||||||
|
}
|
||||||
|
|
||||||
|
second, outcome, err := acquireSourceLease(ctx, source.ID, source.ConfigVersion, sourceActionSync)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("acquireSourceLease(duplicate) error = %v, want nil", err)
|
||||||
|
}
|
||||||
|
if got, want := outcome, sourceLeaseBusy; got != want {
|
||||||
|
t.Errorf("acquireSourceLease(duplicate) outcome = %q, want %q", got, want)
|
||||||
|
}
|
||||||
|
if second != nil {
|
||||||
|
t.Errorf("acquireSourceLease(duplicate) snapshot = %+v, want nil", second)
|
||||||
|
}
|
||||||
|
|
||||||
|
past := time.Now().Add(-time.Second)
|
||||||
|
if err := db.DB(ctx).Model(&model.PagesProjectSourceRuntime{}).
|
||||||
|
Where("source_id = ?", source.ID).
|
||||||
|
Update("lease_expires_at", &past).Error; err != nil {
|
||||||
|
t.Fatalf("expire first lease error = %v, want nil", err)
|
||||||
|
}
|
||||||
|
takeover, outcome, err := acquireSourceLease(ctx, source.ID, source.ConfigVersion, sourceActionSync)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("acquireSourceLease(takeover) error = %v, want nil", err)
|
||||||
|
}
|
||||||
|
if got, want := outcome, sourceLeaseAcquired; got != want {
|
||||||
|
t.Fatalf("acquireSourceLease(takeover) outcome = %q, want %q", got, want)
|
||||||
|
}
|
||||||
|
if takeover == nil {
|
||||||
|
t.Fatal("acquireSourceLease(takeover) snapshot = nil, want non-nil")
|
||||||
|
}
|
||||||
|
if takeover.LeaseToken == "" || takeover.LeaseToken == first.LeaseToken {
|
||||||
|
t.Fatalf("takeover LeaseToken = %q, want non-empty token distinct from %q", takeover.LeaseToken, first.LeaseToken)
|
||||||
|
}
|
||||||
|
|
||||||
|
renewed, err := renewSourceLease(ctx, first, pagesSourceSyncLeaseDuration)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("renewSourceLease(expired owner) error = %v, want nil", err)
|
||||||
|
}
|
||||||
|
if renewed {
|
||||||
|
t.Error("renewSourceLease(expired owner) = true, want false")
|
||||||
|
}
|
||||||
|
if err := failSourceLease(ctx, first, "stale worker must not win"); err != nil {
|
||||||
|
t.Fatalf("failSourceLease(expired owner) error = %v, want nil", err)
|
||||||
|
}
|
||||||
|
var runtime model.PagesProjectSourceRuntime
|
||||||
|
if err := db.DB(ctx).Where("source_id = ?", source.ID).First(&runtime).Error; err != nil {
|
||||||
|
t.Fatalf("load runtime after takeover error = %v, want nil", err)
|
||||||
|
}
|
||||||
|
if got, want := runtime.LeaseToken, takeover.LeaseToken; got != want {
|
||||||
|
t.Errorf("runtime LeaseToken after stale terminal write = %q, want %q", got, want)
|
||||||
|
}
|
||||||
|
if got, want := runtime.SyncStatus, pagesSourceStatusSyncing; got != want {
|
||||||
|
t.Errorf("runtime SyncStatus after stale terminal write = %q, want %q", got, want)
|
||||||
|
}
|
||||||
|
|
||||||
|
renewed, err = renewSourceLease(ctx, takeover, pagesSourceSyncLeaseDuration)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("renewSourceLease(current owner) error = %v, want nil", err)
|
||||||
|
}
|
||||||
|
if !renewed {
|
||||||
|
t.Error("renewSourceLease(current owner) = false, want true")
|
||||||
|
}
|
||||||
|
if err := failSourceLease(ctx, takeover, errPagesSourceSyncFailed); err != nil {
|
||||||
|
t.Fatalf("failSourceLease(current owner) error = %v, want nil", err)
|
||||||
|
}
|
||||||
|
var failedRuntime model.PagesProjectSourceRuntime
|
||||||
|
if err := db.DB(ctx).Where("source_id = ?", source.ID).First(&failedRuntime).Error; err != nil {
|
||||||
|
t.Fatalf("load failed runtime error = %v, want nil", err)
|
||||||
|
}
|
||||||
|
if got, want := failedRuntime.SyncStatus, pagesSourceStatusFailed; got != want {
|
||||||
|
t.Errorf("failed runtime SyncStatus = %q, want %q", got, want)
|
||||||
|
}
|
||||||
|
if failedRuntime.LeaseToken != "" || failedRuntime.LeaseExpiresAt != nil {
|
||||||
|
t.Errorf("failed runtime lease = (%q, %v), want cleared", failedRuntime.LeaseToken, failedRuntime.LeaseExpiresAt)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestSourceConfigAndProjectContentChangesFenceLease(t *testing.T) {
|
||||||
|
ctx := setupPagesSourceTest(t)
|
||||||
|
project := mustCreatePagesSourceProject(t, ctx, "lease-fence")
|
||||||
|
source, _ := mustConfigureRemoteSource(
|
||||||
|
t,
|
||||||
|
ctx,
|
||||||
|
project.ID,
|
||||||
|
"https://example.com/site.zip?token=first",
|
||||||
|
RemoteNetworkPolicyPublic,
|
||||||
|
)
|
||||||
|
|
||||||
|
configSnapshot, outcome, err := acquireSourceLease(ctx, source.ID, source.ConfigVersion, sourceActionSync)
|
||||||
|
if err != nil || outcome != sourceLeaseAcquired {
|
||||||
|
t.Fatalf("acquireSourceLease(config fence) = (%+v, %q, %v), want acquired", configSnapshot, outcome, err)
|
||||||
|
}
|
||||||
|
if _, err := UpdateSource(ctx, project.ID, SourceUpdateInput{
|
||||||
|
SourceType: PagesSourceTypeRemoteURL,
|
||||||
|
RemoteURLSet: true,
|
||||||
|
RemoteURL: "https://example.com/site.zip?token=second",
|
||||||
|
RemoteNetworkPolicy: RemoteNetworkPolicyPublic,
|
||||||
|
}); err != nil {
|
||||||
|
t.Fatalf("UpdateSource(config fence) error = %v, want nil", err)
|
||||||
|
}
|
||||||
|
renewed, err := renewSourceLease(ctx, configSnapshot, pagesSourceSyncLeaseDuration)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("renewSourceLease(after source update) error = %v, want nil", err)
|
||||||
|
}
|
||||||
|
if renewed {
|
||||||
|
t.Error("renewSourceLease(after source update) = true, want false")
|
||||||
|
}
|
||||||
|
var updatedSource model.PagesProjectSource
|
||||||
|
if err := db.DB(ctx).Where("id = ?", source.ID).First(&updatedSource).Error; err != nil {
|
||||||
|
t.Fatalf("load updated source error = %v, want nil", err)
|
||||||
|
}
|
||||||
|
if got, want := updatedSource.ConfigVersion, source.ConfigVersion+1; got != want {
|
||||||
|
t.Errorf("updated source ConfigVersion = %d, want %d", got, want)
|
||||||
|
}
|
||||||
|
if snapshot, staleOutcome, staleErr := acquireSourceLease(ctx, source.ID, source.ConfigVersion, sourceActionSync); staleErr != nil || staleOutcome != sourceLeaseStale || snapshot != nil {
|
||||||
|
t.Errorf("acquireSourceLease(old config) = (%+v, %q, %v), want (nil, %q, nil)", snapshot, staleOutcome, staleErr, sourceLeaseStale)
|
||||||
|
}
|
||||||
|
|
||||||
|
contentSnapshot, outcome, err := acquireSourceLease(ctx, source.ID, updatedSource.ConfigVersion, sourceActionSync)
|
||||||
|
if err != nil || outcome != sourceLeaseAcquired {
|
||||||
|
t.Fatalf("acquireSourceLease(content fence) = (%+v, %q, %v), want acquired", contentSnapshot, outcome, err)
|
||||||
|
}
|
||||||
|
if _, err := UpdateProject(ctx, project.ID, Input{
|
||||||
|
Name: project.Name,
|
||||||
|
Slug: project.Slug,
|
||||||
|
Enabled: true,
|
||||||
|
RootDir: "dist",
|
||||||
|
EntryFile: "index.html",
|
||||||
|
}); err != nil {
|
||||||
|
t.Fatalf("UpdateProject(content fence) error = %v, want nil", err)
|
||||||
|
}
|
||||||
|
renewed, err = renewSourceLease(ctx, contentSnapshot, pagesSourceSyncLeaseDuration)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("renewSourceLease(after content update) error = %v, want nil", err)
|
||||||
|
}
|
||||||
|
if renewed {
|
||||||
|
t.Error("renewSourceLease(after content update) = true, want false")
|
||||||
|
}
|
||||||
|
storedProject, err := model.GetPagesProjectByID(ctx, project.ID)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("GetPagesProjectByID(%d) error = %v, want nil", project.ID, err)
|
||||||
|
}
|
||||||
|
if got, want := storedProject.ContentConfigVersion, project.ContentConfigVersion+1; got != want {
|
||||||
|
t.Errorf("ContentConfigVersion after RootDir update = %d, want %d", got, want)
|
||||||
|
}
|
||||||
|
var runtime model.PagesProjectSourceRuntime
|
||||||
|
if err := db.DB(ctx).Where("source_id = ?", source.ID).First(&runtime).Error; err != nil {
|
||||||
|
t.Fatalf("load fenced runtime error = %v, want nil", err)
|
||||||
|
}
|
||||||
|
if runtime.LeaseToken != "" || runtime.LeaseExpiresAt != nil {
|
||||||
|
t.Errorf("content-fenced runtime lease = (%q, %v), want cleared", runtime.LeaseToken, runtime.LeaseExpiresAt)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestSourceRuntimeUsesOnlySixDocumentedStates(t *testing.T) {
|
||||||
|
states := []string{
|
||||||
|
pagesSourceStatusIdle,
|
||||||
|
pagesSourceStatusChecking,
|
||||||
|
pagesSourceStatusUpdateAvailable,
|
||||||
|
pagesSourceStatusSyncing,
|
||||||
|
pagesSourceStatusFailed,
|
||||||
|
pagesSourceStatusAttention,
|
||||||
|
}
|
||||||
|
seen := make(map[string]struct{}, len(states))
|
||||||
|
for _, state := range states {
|
||||||
|
if strings.TrimSpace(state) == "" {
|
||||||
|
t.Errorf("documented source state = %q, want non-empty", state)
|
||||||
|
}
|
||||||
|
if _, exists := seen[state]; exists {
|
||||||
|
t.Errorf("documented source state %q is duplicated", state)
|
||||||
|
}
|
||||||
|
seen[state] = struct{}{}
|
||||||
|
}
|
||||||
|
if got, want := len(seen), 6; got != want {
|
||||||
|
t.Errorf("unique source states = %d, want %d", got, want)
|
||||||
|
}
|
||||||
|
|
||||||
|
updateRuntime := &model.PagesProjectSourceRuntime{
|
||||||
|
LastSeenRevision: strings.Repeat("a", 64),
|
||||||
|
LastAppliedRevision: strings.Repeat("b", 64),
|
||||||
|
LastSeenDetail: `{"release_id":"new"}`,
|
||||||
|
LastAppliedDetail: `{"release_id":"old"}`,
|
||||||
|
}
|
||||||
|
if got, want := normalizedSourceRuntimeStatus(updateRuntime), pagesSourceStatusUpdateAvailable; got != want {
|
||||||
|
t.Errorf("normalizedSourceRuntimeStatus(update) = %q, want %q", got, want)
|
||||||
|
}
|
||||||
|
attentionRuntime := &model.PagesProjectSourceRuntime{
|
||||||
|
LastSeenRevision: strings.Repeat("a", 64),
|
||||||
|
LastAppliedRevision: strings.Repeat("b", 64),
|
||||||
|
LastSeenDetail: `{"release_id":"same"}`,
|
||||||
|
LastAppliedDetail: `{"release_id":"same"}`,
|
||||||
|
}
|
||||||
|
if got, want := normalizedSourceRuntimeStatus(attentionRuntime), pagesSourceStatusAttention; got != want {
|
||||||
|
t.Errorf("normalizedSourceRuntimeStatus(attention) = %q, want %q", got, want)
|
||||||
|
}
|
||||||
|
if got, want := normalizedSourceRuntimeStatus(&model.PagesProjectSourceRuntime{}), pagesSourceStatusIdle; got != want {
|
||||||
|
t.Errorf("normalizedSourceRuntimeStatus(idle) = %q, want %q", got, want)
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -0,0 +1,478 @@
|
|||||||
|
// Copyright 2026 Arctel.net
|
||||||
|
// SPDX-License-Identifier: Apache-2.0
|
||||||
|
|
||||||
|
package pages
|
||||||
|
|
||||||
|
import (
|
||||||
|
"bytes"
|
||||||
|
"context"
|
||||||
|
"encoding/json"
|
||||||
|
"errors"
|
||||||
|
"fmt"
|
||||||
|
"time"
|
||||||
|
|
||||||
|
"github.com/Rain-kl/Wavelet/internal/db"
|
||||||
|
"github.com/Rain-kl/Wavelet/internal/model"
|
||||||
|
"github.com/Rain-kl/Wavelet/internal/task"
|
||||||
|
"github.com/Rain-kl/Wavelet/pkg/logger"
|
||||||
|
"gorm.io/gorm"
|
||||||
|
)
|
||||||
|
|
||||||
|
const (
|
||||||
|
// PagesSourceScanTask is the private Asynq task type for the periodic scanner.
|
||||||
|
PagesSourceScanTask = "openflare:pages_source_scan"
|
||||||
|
// TaskTypePagesSourceScan is the internal task meta type seeded in w_schedules.
|
||||||
|
TaskTypePagesSourceScan = "of_pages_source_scan"
|
||||||
|
|
||||||
|
pagesSourceScanBatchSize = 20
|
||||||
|
)
|
||||||
|
|
||||||
|
// PagesSourceScanMeta is available to the scheduler registry but hidden from
|
||||||
|
// generic Admin task dispatch and schedule mutation APIs.
|
||||||
|
var PagesSourceScanMeta = task.TaskMeta{
|
||||||
|
Type: TaskTypePagesSourceScan,
|
||||||
|
AsynqTask: PagesSourceScanTask,
|
||||||
|
Name: "OpenFlare Pages 部署源扫描",
|
||||||
|
Description: "补偿孤儿部署包、恢复过期执行权并串行检查到期的 GitHub latest 部署源",
|
||||||
|
SupportsTime: false,
|
||||||
|
MaxRetry: 0,
|
||||||
|
Queue: task.QueueDefault,
|
||||||
|
Retryable: false,
|
||||||
|
InternalOnly: true,
|
||||||
|
}
|
||||||
|
|
||||||
|
type pagesSourceScanPayload struct{}
|
||||||
|
|
||||||
|
type pagesSourceScanSummary struct {
|
||||||
|
ExpiredCandidates int `json:"expired_candidates"`
|
||||||
|
RecoveredLeases int `json:"recovered_leases"`
|
||||||
|
OrphanCleanup PagesOrphanCleanupSummary `json:"orphan_cleanup"`
|
||||||
|
DueSources int `json:"due_sources"`
|
||||||
|
SelectedSources int `json:"selected_sources"`
|
||||||
|
CheckedSources int `json:"checked_sources"`
|
||||||
|
UpdatesFound int `json:"updates_found"`
|
||||||
|
AttentionSources int `json:"attention_sources"`
|
||||||
|
DispatchedSyncs int `json:"dispatched_syncs"`
|
||||||
|
FailedDispatches int `json:"failed_dispatches"`
|
||||||
|
BusySources int `json:"busy_sources"`
|
||||||
|
StaleSources int `json:"stale_sources"`
|
||||||
|
FailedSources int `json:"failed_sources"`
|
||||||
|
Backlog int `json:"backlog"`
|
||||||
|
ProviderBackoffs []pagesSourceProviderBackoff `json:"provider_backoffs,omitempty"`
|
||||||
|
}
|
||||||
|
|
||||||
|
type pagesSourceProviderBackoff struct {
|
||||||
|
SourceID uint `json:"source_id"`
|
||||||
|
StatusCode int `json:"status_code"`
|
||||||
|
RetryAt string `json:"retry_at"`
|
||||||
|
}
|
||||||
|
|
||||||
|
type expiredSourceLeaseCandidate struct {
|
||||||
|
SourceID uint
|
||||||
|
LeaseToken string
|
||||||
|
LeaseExpiresAt time.Time
|
||||||
|
SyncStatus string
|
||||||
|
SourceType string
|
||||||
|
ReleaseSelector string
|
||||||
|
}
|
||||||
|
|
||||||
|
type dueGitHubSourceCandidate struct {
|
||||||
|
SourceID uint
|
||||||
|
ConfigVersion int
|
||||||
|
}
|
||||||
|
|
||||||
|
var (
|
||||||
|
pagesSourceScanNow = time.Now
|
||||||
|
reconcilePagesSourceOrphans = ReconcilePagesOrphanUploads
|
||||||
|
dispatchPagesSourceAutoSync = func(
|
||||||
|
ctx context.Context,
|
||||||
|
source model.PagesProjectSource,
|
||||||
|
targetRevision string,
|
||||||
|
) (*SourceActionReceipt, error) {
|
||||||
|
return dispatchSourceActionSnapshotWithTrigger(
|
||||||
|
ctx,
|
||||||
|
source,
|
||||||
|
sourceActionSync,
|
||||||
|
pagesSourceCreatedBySystem,
|
||||||
|
pagesSourceTriggerScheduledAutoUpdate,
|
||||||
|
targetRevision,
|
||||||
|
"",
|
||||||
|
"system",
|
||||||
|
)
|
||||||
|
}
|
||||||
|
)
|
||||||
|
|
||||||
|
// SourceScanHandler serializes provider checks inside one scheduled task. A
|
||||||
|
// source-level lease still permits overlapping scanner executions safely.
|
||||||
|
type SourceScanHandler struct{}
|
||||||
|
|
||||||
|
// ValidatePayload accepts only an empty object; the scanner has no user input.
|
||||||
|
func (handler *SourceScanHandler) ValidatePayload(payload []byte) ([]byte, error) {
|
||||||
|
if len(bytes.TrimSpace(payload)) == 0 {
|
||||||
|
payload = []byte("{}")
|
||||||
|
}
|
||||||
|
var input pagesSourceScanPayload
|
||||||
|
decoder := json.NewDecoder(bytes.NewReader(payload))
|
||||||
|
decoder.DisallowUnknownFields()
|
||||||
|
if err := decoder.Decode(&input); err != nil {
|
||||||
|
return nil, errors.New(errPagesSourceActionInvalid)
|
||||||
|
}
|
||||||
|
if err := ensureJSONEOF(decoder); err != nil {
|
||||||
|
return nil, errors.New(errPagesSourceActionInvalid)
|
||||||
|
}
|
||||||
|
return []byte("{}"), nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// Execute recovers expired leases and checks at most 20 due latest sources in
|
||||||
|
// stable order. Provider and dispatch failures are isolated per source.
|
||||||
|
func (handler *SourceScanHandler) Execute(ctx context.Context, payload []byte) (*task.TaskResult, error) {
|
||||||
|
if _, err := handler.ValidatePayload(payload); err != nil {
|
||||||
|
return nil, task.PermanentError(errPagesSourceActionInvalid)
|
||||||
|
}
|
||||||
|
|
||||||
|
now := pagesSourceScanNow()
|
||||||
|
summary := pagesSourceScanSummary{}
|
||||||
|
if err := recoverExpiredPagesSourceLeases(ctx, now, &summary); err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
orphanSummary, err := reconcilePagesSourceOrphans(ctx, now)
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
summary.OrphanCleanup = orphanSummary
|
||||||
|
task.AppendLog(
|
||||||
|
ctx,
|
||||||
|
"[cleanup] orphan 候选=%d,已补偿=%d,仍被引用=%d,lease busy=%d,非法 marker=%d,跳过=%d,失败=%d",
|
||||||
|
orphanSummary.Candidates,
|
||||||
|
orphanSummary.Reconciled,
|
||||||
|
orphanSummary.Referenced,
|
||||||
|
orphanSummary.LeaseBusy,
|
||||||
|
orphanSummary.InvalidMarker,
|
||||||
|
orphanSummary.Skipped,
|
||||||
|
orphanSummary.Failed,
|
||||||
|
)
|
||||||
|
if err := scanDueGitHubSources(ctx, now, &summary); err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
|
||||||
|
detail, err := json.Marshal(summary)
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
message := fmt.Sprintf(
|
||||||
|
"Pages 部署源扫描完成:恢复 %d 个租约,补偿 %d 个孤儿记录,检查 %d 个来源,投递 %d 个自动更新,积压 %d 个",
|
||||||
|
summary.RecoveredLeases,
|
||||||
|
summary.OrphanCleanup.Reconciled,
|
||||||
|
summary.CheckedSources,
|
||||||
|
summary.DispatchedSyncs,
|
||||||
|
summary.Backlog,
|
||||||
|
)
|
||||||
|
return &task.TaskResult{Message: message, Detail: string(detail)}, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func recoverExpiredPagesSourceLeases(
|
||||||
|
ctx context.Context,
|
||||||
|
now time.Time,
|
||||||
|
summary *pagesSourceScanSummary,
|
||||||
|
) error {
|
||||||
|
var candidates []expiredSourceLeaseCandidate
|
||||||
|
err := db.DB(ctx).
|
||||||
|
Table("of_pages_project_source_runtime AS runtime").
|
||||||
|
Select(`runtime.source_id, runtime.lease_token, runtime.lease_expires_at,
|
||||||
|
runtime.sync_status, source.source_type, source.release_selector`).
|
||||||
|
Joins("JOIN of_pages_project_sources AS source ON source.id = runtime.source_id").
|
||||||
|
Where("runtime.lease_token <> ''").
|
||||||
|
Where("runtime.lease_expires_at IS NOT NULL AND runtime.lease_expires_at <= ?", now).
|
||||||
|
Where("runtime.sync_status IN ?", []string{pagesSourceStatusChecking, pagesSourceStatusSyncing}).
|
||||||
|
Order("runtime.source_id ASC").
|
||||||
|
Scan(&candidates).Error
|
||||||
|
if err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
summary.ExpiredCandidates = len(candidates)
|
||||||
|
for _, candidate := range candidates {
|
||||||
|
var nextCheckAt *time.Time
|
||||||
|
if candidate.SourceType == PagesSourceTypeGitHubRelease &&
|
||||||
|
candidate.ReleaseSelector == githubReleaseSelectorLatest {
|
||||||
|
next := nextGitHubCheckAt(now, candidate.SourceID, minimumCheckInterval)
|
||||||
|
nextCheckAt = &next
|
||||||
|
}
|
||||||
|
recovered, recoverErr := recoverExpiredSourceLease(
|
||||||
|
ctx,
|
||||||
|
candidate.SourceID,
|
||||||
|
candidate.LeaseToken,
|
||||||
|
candidate.LeaseExpiresAt,
|
||||||
|
candidate.SyncStatus,
|
||||||
|
now,
|
||||||
|
nextCheckAt,
|
||||||
|
)
|
||||||
|
if recoverErr != nil {
|
||||||
|
summary.FailedSources++
|
||||||
|
logger.WarnF(
|
||||||
|
ctx,
|
||||||
|
"[PagesSourceScan] recover expired lease failed: source_id=%d error=%v",
|
||||||
|
candidate.SourceID,
|
||||||
|
recoverErr,
|
||||||
|
)
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
if recovered {
|
||||||
|
summary.RecoveredLeases++
|
||||||
|
task.AppendLog(ctx, "[recover] 已恢复过期来源租约:source_id=%d", candidate.SourceID)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func scanDueGitHubSources(
|
||||||
|
ctx context.Context,
|
||||||
|
now time.Time,
|
||||||
|
summary *pagesSourceScanSummary,
|
||||||
|
) error {
|
||||||
|
dueQuery := func() *gorm.DB {
|
||||||
|
return db.DB(ctx).
|
||||||
|
Table("of_pages_project_source_runtime AS runtime").
|
||||||
|
Joins("JOIN of_pages_project_sources AS source ON source.id = runtime.source_id").
|
||||||
|
Where("source.source_type = ?", PagesSourceTypeGitHubRelease).
|
||||||
|
Where("source.release_selector = ?", githubReleaseSelectorLatest).
|
||||||
|
Where("runtime.next_check_at IS NOT NULL AND runtime.next_check_at <= ?", now)
|
||||||
|
}
|
||||||
|
var dueCount int64
|
||||||
|
if err := dueQuery().Count(&dueCount).Error; err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
summary.DueSources = int(dueCount)
|
||||||
|
|
||||||
|
var candidates []dueGitHubSourceCandidate
|
||||||
|
if err := dueQuery().
|
||||||
|
Select("source.id AS source_id, source.config_version").
|
||||||
|
Order("runtime.next_check_at ASC").
|
||||||
|
Order("source.id ASC").
|
||||||
|
Limit(pagesSourceScanBatchSize).
|
||||||
|
Scan(&candidates).Error; err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
summary.SelectedSources = len(candidates)
|
||||||
|
task.AppendLog(
|
||||||
|
ctx,
|
||||||
|
"[scan] 到期来源=%d,本批=%d",
|
||||||
|
summary.DueSources,
|
||||||
|
summary.SelectedSources,
|
||||||
|
)
|
||||||
|
|
||||||
|
for _, candidate := range candidates {
|
||||||
|
scanOneDueGitHubSource(ctx, candidate, summary)
|
||||||
|
}
|
||||||
|
var remainingDue int64
|
||||||
|
if err := dueQuery().Count(&remainingDue).Error; err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
summary.Backlog = int(remainingDue)
|
||||||
|
task.AppendLog(ctx, "[scan] 本批处理后仍到期来源=%d", summary.Backlog)
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func scanOneDueGitHubSource(
|
||||||
|
ctx context.Context,
|
||||||
|
candidate dueGitHubSourceCandidate,
|
||||||
|
summary *pagesSourceScanSummary,
|
||||||
|
) {
|
||||||
|
snapshot, outcome, err := acquireSourceLease(
|
||||||
|
ctx,
|
||||||
|
candidate.SourceID,
|
||||||
|
candidate.ConfigVersion,
|
||||||
|
sourceActionCheck,
|
||||||
|
)
|
||||||
|
if err != nil {
|
||||||
|
summary.FailedSources++
|
||||||
|
logger.WarnF(ctx, "[PagesSourceScan] acquire check lease failed: source_id=%d error=%v", candidate.SourceID, err)
|
||||||
|
return
|
||||||
|
}
|
||||||
|
switch outcome {
|
||||||
|
case sourceLeaseBusy:
|
||||||
|
summary.BusySources++
|
||||||
|
task.AppendLog(ctx, "[check] 来源正在执行其它任务,跳过:source_id=%d", candidate.SourceID)
|
||||||
|
return
|
||||||
|
case sourceLeaseStale:
|
||||||
|
summary.StaleSources++
|
||||||
|
return
|
||||||
|
}
|
||||||
|
if snapshot == nil || snapshot.SourceType != PagesSourceTypeGitHubRelease ||
|
||||||
|
snapshot.ReleaseSelector != githubReleaseSelectorLatest {
|
||||||
|
summary.StaleSources++
|
||||||
|
if snapshot != nil {
|
||||||
|
if finalizeErr := failSourceLease(ctx, snapshot, errPagesSourceActionStale); finalizeErr != nil {
|
||||||
|
logger.WarnF(
|
||||||
|
ctx,
|
||||||
|
"[PagesSourceScan] finalize stale source failed: source_id=%d error=%v",
|
||||||
|
snapshot.SourceID,
|
||||||
|
finalizeErr,
|
||||||
|
)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return
|
||||||
|
}
|
||||||
|
|
||||||
|
checkResult, checkErr := checkGitHubSource(ctx, snapshot)
|
||||||
|
if checkErr != nil {
|
||||||
|
summary.FailedSources++
|
||||||
|
recordPagesSourceProviderBackoff(ctx, candidate.SourceID, checkErr, summary)
|
||||||
|
logger.WarnF(
|
||||||
|
ctx,
|
||||||
|
"[PagesSourceScan] source check failed: source_id=%d error=%s",
|
||||||
|
candidate.SourceID,
|
||||||
|
safeGitHubSourceError(checkErr),
|
||||||
|
)
|
||||||
|
return
|
||||||
|
}
|
||||||
|
if checkResult == nil || checkResult.Stale {
|
||||||
|
summary.StaleSources++
|
||||||
|
return
|
||||||
|
}
|
||||||
|
handleCheckedGitHubSource(ctx, snapshot, checkResult, summary)
|
||||||
|
}
|
||||||
|
|
||||||
|
func recordPagesSourceProviderBackoff(
|
||||||
|
ctx context.Context,
|
||||||
|
sourceID uint,
|
||||||
|
checkErr error,
|
||||||
|
summary *pagesSourceScanSummary,
|
||||||
|
) {
|
||||||
|
var domainError *githubSourceProviderDomainError
|
||||||
|
if !errors.As(checkErr, &domainError) ||
|
||||||
|
(domainError.statusCode != 403 && domainError.statusCode != 429) {
|
||||||
|
return
|
||||||
|
}
|
||||||
|
|
||||||
|
retryAt := domainError.retryAt
|
||||||
|
var runtime model.PagesProjectSourceRuntime
|
||||||
|
if err := db.DB(ctx).
|
||||||
|
Select("next_check_at").
|
||||||
|
Where("source_id = ?", sourceID).
|
||||||
|
First(&runtime).Error; err != nil {
|
||||||
|
logger.WarnF(ctx, "[PagesSourceScan] load provider backoff deadline failed: source_id=%d error=%v", sourceID, err)
|
||||||
|
} else if runtime.NextCheckAt != nil {
|
||||||
|
retryAt = runtime.NextCheckAt
|
||||||
|
}
|
||||||
|
|
||||||
|
retryAtText := "unknown"
|
||||||
|
if retryAt != nil {
|
||||||
|
retryAtText = retryAt.UTC().Format(time.RFC3339)
|
||||||
|
}
|
||||||
|
summary.ProviderBackoffs = append(summary.ProviderBackoffs, pagesSourceProviderBackoff{
|
||||||
|
SourceID: sourceID, StatusCode: domainError.statusCode, RetryAt: retryAtText,
|
||||||
|
})
|
||||||
|
task.AppendLog(
|
||||||
|
ctx,
|
||||||
|
"[check] GitHub provider 退避:source_id=%d status=%d retry_at=%s",
|
||||||
|
sourceID,
|
||||||
|
domainError.statusCode,
|
||||||
|
retryAtText,
|
||||||
|
)
|
||||||
|
}
|
||||||
|
|
||||||
|
func handleCheckedGitHubSource(
|
||||||
|
ctx context.Context,
|
||||||
|
snapshot *sourceExecutionSnapshot,
|
||||||
|
checkResult *githubCheckTaskResult,
|
||||||
|
summary *pagesSourceScanSummary,
|
||||||
|
) {
|
||||||
|
summary.CheckedSources++
|
||||||
|
switch checkResult.Status {
|
||||||
|
case pagesSourceStatusUpdateAvailable:
|
||||||
|
summary.UpdatesFound++
|
||||||
|
case pagesSourceStatusAttention:
|
||||||
|
summary.AttentionSources++
|
||||||
|
}
|
||||||
|
if !snapshot.AutoUpdateEnabled || checkResult.Status != pagesSourceStatusUpdateAvailable ||
|
||||||
|
!validOptionalSourceRevision(checkResult.Revision) || checkResult.Revision == "" {
|
||||||
|
return
|
||||||
|
}
|
||||||
|
|
||||||
|
source := model.PagesProjectSource{
|
||||||
|
ID: snapshot.SourceID,
|
||||||
|
ProjectID: snapshot.ProjectID,
|
||||||
|
ConfigVersion: snapshot.SourceConfigVersion,
|
||||||
|
}
|
||||||
|
receipt, dispatchErr := dispatchPagesSourceAutoSync(ctx, source, checkResult.Revision)
|
||||||
|
if dispatchErr == nil {
|
||||||
|
summary.DispatchedSyncs++
|
||||||
|
if receipt != nil {
|
||||||
|
task.AppendLog(
|
||||||
|
ctx,
|
||||||
|
"[dispatch] 已投递自动更新:source_id=%d execution_id=%s revision=%s",
|
||||||
|
snapshot.SourceID,
|
||||||
|
receipt.ExecutionID,
|
||||||
|
checkResult.Revision,
|
||||||
|
)
|
||||||
|
}
|
||||||
|
return
|
||||||
|
}
|
||||||
|
|
||||||
|
summary.FailedSources++
|
||||||
|
summary.FailedDispatches++
|
||||||
|
logger.WarnF(
|
||||||
|
ctx,
|
||||||
|
"[PagesSourceScan] dispatch auto sync failed: source_id=%d revision=%s error=%v",
|
||||||
|
snapshot.SourceID,
|
||||||
|
checkResult.Revision,
|
||||||
|
dispatchErr,
|
||||||
|
)
|
||||||
|
updated, recordErr := recordPagesSourceAutoDispatchFailure(
|
||||||
|
ctx,
|
||||||
|
snapshot,
|
||||||
|
checkResult.Revision,
|
||||||
|
checkResult.RetryAt,
|
||||||
|
)
|
||||||
|
if recordErr != nil {
|
||||||
|
logger.WarnF(
|
||||||
|
ctx,
|
||||||
|
"[PagesSourceScan] record auto sync dispatch failure failed: source_id=%d error=%v",
|
||||||
|
snapshot.SourceID,
|
||||||
|
recordErr,
|
||||||
|
)
|
||||||
|
} else if !updated {
|
||||||
|
summary.StaleSources++
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func recordPagesSourceAutoDispatchFailure(
|
||||||
|
ctx context.Context,
|
||||||
|
snapshot *sourceExecutionSnapshot,
|
||||||
|
revision string,
|
||||||
|
retryAt *time.Time,
|
||||||
|
) (bool, error) {
|
||||||
|
if snapshot == nil || revision == "" {
|
||||||
|
return false, nil
|
||||||
|
}
|
||||||
|
now := pagesSourceScanNow()
|
||||||
|
next := nextGitHubCheckAt(now, snapshot.SourceID, minimumCheckInterval)
|
||||||
|
if retryAt != nil && retryAt.After(next) {
|
||||||
|
next = retryAt.In(now.Location())
|
||||||
|
}
|
||||||
|
result := db.DB(ctx).Model(&model.PagesProjectSourceRuntime{}).
|
||||||
|
Where("source_id = ?", snapshot.SourceID).
|
||||||
|
Where("sync_status = ? AND last_seen_revision = ?", pagesSourceStatusUpdateAvailable, revision).
|
||||||
|
Where("lease_expires_at IS NULL OR lease_expires_at <= ?", now).
|
||||||
|
Where(`EXISTS (
|
||||||
|
SELECT 1 FROM of_pages_project_sources AS source
|
||||||
|
WHERE source.id = ? AND source.config_version = ?
|
||||||
|
AND source.source_type = ? AND source.release_selector = ?
|
||||||
|
AND source.auto_update_enabled = ?
|
||||||
|
)`,
|
||||||
|
snapshot.SourceID,
|
||||||
|
snapshot.SourceConfigVersion,
|
||||||
|
PagesSourceTypeGitHubRelease,
|
||||||
|
githubReleaseSelectorLatest,
|
||||||
|
true,
|
||||||
|
).
|
||||||
|
Updates(map[string]any{
|
||||||
|
sourceRuntimeColumnSyncStatus: pagesSourceStatusUpdateAvailable,
|
||||||
|
sourceRuntimeColumnLastError: errPagesSourceTaskDispatchFailed,
|
||||||
|
sourceRuntimeColumnNextCheckAt: &next,
|
||||||
|
})
|
||||||
|
if result.Error != nil {
|
||||||
|
return false, result.Error
|
||||||
|
}
|
||||||
|
return result.RowsAffected == 1, nil
|
||||||
|
}
|
||||||
@@ -0,0 +1,511 @@
|
|||||||
|
// Copyright 2026 Arctel.net
|
||||||
|
// SPDX-License-Identifier: Apache-2.0
|
||||||
|
|
||||||
|
package pages
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
"crypto/sha256"
|
||||||
|
"encoding/hex"
|
||||||
|
"encoding/json"
|
||||||
|
"errors"
|
||||||
|
"fmt"
|
||||||
|
"os"
|
||||||
|
"path/filepath"
|
||||||
|
"strings"
|
||||||
|
"testing"
|
||||||
|
"time"
|
||||||
|
|
||||||
|
"github.com/Rain-kl/Wavelet/internal/db"
|
||||||
|
"github.com/Rain-kl/Wavelet/internal/integration/githubrelease"
|
||||||
|
"github.com/Rain-kl/Wavelet/internal/model"
|
||||||
|
"gorm.io/gorm"
|
||||||
|
)
|
||||||
|
|
||||||
|
type scannerDispatchedSync struct {
|
||||||
|
SourceID uint
|
||||||
|
Revision string
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestGitHubLatestAutoConfigPreservesIdentityAndRuntimeCursor(t *testing.T) {
|
||||||
|
ctx := setupPagesSourceTest(t)
|
||||||
|
project := mustCreatePagesSourceProject(t, ctx, "scanner-auto-config")
|
||||||
|
source, _ := mustConfigureGitHubSourceWithoutDispatch(t, ctx, project.ID, SourceUpdateInput{
|
||||||
|
SourceType: PagesSourceTypeGitHubRelease,
|
||||||
|
RepositoryURL: "https://github.com/scanner/auto-config",
|
||||||
|
AutoUpdateEnabled: false,
|
||||||
|
CheckIntervalMinutes: 60,
|
||||||
|
})
|
||||||
|
identity := source.SourceIdentity
|
||||||
|
seenRevision := strings.Repeat("a", sourceRevisionHexLength)
|
||||||
|
appliedRevision := strings.Repeat("b", sourceRevisionHexLength)
|
||||||
|
future := time.Now().Add(time.Hour)
|
||||||
|
if err := db.DB(ctx).Model(&model.PagesProjectSourceRuntime{}).
|
||||||
|
Where("source_id = ?", source.ID).
|
||||||
|
Updates(map[string]any{
|
||||||
|
"etag": `"cursor-etag"`,
|
||||||
|
"last_seen_revision": seenRevision,
|
||||||
|
"last_seen_detail": `{"provider":"github","release_id":"2","asset_id":"2","tag":"v2","asset_name":"dist.zip"}`,
|
||||||
|
"last_applied_revision": appliedRevision,
|
||||||
|
"last_applied_detail": `{"provider":"github","release_id":"1","asset_id":"1","tag":"v1","asset_name":"dist.zip"}`,
|
||||||
|
"sync_status": pagesSourceStatusSyncing,
|
||||||
|
"last_error": "old error",
|
||||||
|
"lease_token": "in-flight",
|
||||||
|
"lease_expires_at": &future,
|
||||||
|
}).Error; err != nil {
|
||||||
|
t.Fatalf("seed runtime error = %v, want nil", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
input := SourceUpdateInput{
|
||||||
|
SourceType: PagesSourceTypeGitHubRelease,
|
||||||
|
RepositoryURL: "https://github.com/scanner/auto-config",
|
||||||
|
AutoUpdateEnabled: true,
|
||||||
|
CheckIntervalMinutes: 15,
|
||||||
|
}
|
||||||
|
if err := validateGitHubSourceInput(input); err != nil {
|
||||||
|
t.Fatalf("validateGitHubSourceInput(auto latest) error = %v, want nil", err)
|
||||||
|
}
|
||||||
|
if err := db.DB(ctx).Transaction(func(tx *gorm.DB) error {
|
||||||
|
changed, err := updateGitHubSourceTx(tx, project.ID, input)
|
||||||
|
if err == nil && !changed {
|
||||||
|
return errors.New("auto config update was treated as no-op")
|
||||||
|
}
|
||||||
|
return err
|
||||||
|
}); err != nil {
|
||||||
|
t.Fatalf("updateGitHubSourceTx(auto latest) error = %v, want nil", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
updated, runtime := mustLoadPagesSource(t, ctx, project.ID)
|
||||||
|
if updated.SourceIdentity != identity || updated.ConfigVersion != source.ConfigVersion+1 {
|
||||||
|
t.Errorf(
|
||||||
|
"updated source = identity:%q version:%d, want identity:%q version:%d",
|
||||||
|
updated.SourceIdentity,
|
||||||
|
updated.ConfigVersion,
|
||||||
|
identity,
|
||||||
|
source.ConfigVersion+1,
|
||||||
|
)
|
||||||
|
}
|
||||||
|
if !updated.AutoUpdateEnabled || updated.CheckIntervalMinutes != 15 {
|
||||||
|
t.Errorf("updated auto config = enabled:%t interval:%d, want true/15", updated.AutoUpdateEnabled, updated.CheckIntervalMinutes)
|
||||||
|
}
|
||||||
|
if runtime.ETag != `"cursor-etag"` || runtime.LastSeenRevision != seenRevision ||
|
||||||
|
runtime.LastAppliedRevision != appliedRevision {
|
||||||
|
t.Errorf("runtime cursor changed after auto-only update: %+v", runtime)
|
||||||
|
}
|
||||||
|
if runtime.LeaseToken != "" || runtime.LeaseExpiresAt != nil || runtime.LastError != "" ||
|
||||||
|
runtime.SyncStatus != pagesSourceStatusUpdateAvailable || runtime.NextCheckAt == nil {
|
||||||
|
t.Errorf(
|
||||||
|
"runtime fence = token:%q expiry:%v error:%q status:%q next:%v",
|
||||||
|
runtime.LeaseToken,
|
||||||
|
runtime.LeaseExpiresAt,
|
||||||
|
runtime.LastError,
|
||||||
|
runtime.SyncStatus,
|
||||||
|
runtime.NextCheckAt,
|
||||||
|
)
|
||||||
|
}
|
||||||
|
|
||||||
|
tagConfig, err := buildGitHubSourceConfig(SourceUpdateInput{
|
||||||
|
SourceType: PagesSourceTypeGitHubRelease,
|
||||||
|
RepositoryURL: "https://github.com/scanner/auto-config",
|
||||||
|
ReleaseSelector: githubReleaseSelectorTag,
|
||||||
|
ReleaseTag: "v1",
|
||||||
|
AutoUpdateEnabled: true,
|
||||||
|
CheckIntervalMinutes: 60,
|
||||||
|
})
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("buildGitHubSourceConfig(tag) error = %v, want nil", err)
|
||||||
|
}
|
||||||
|
if tagConfig.AutoUpdate || tagConfig.CheckInterval != 0 {
|
||||||
|
t.Errorf("tag config auto/interval = %t/%d, want false/0", tagConfig.AutoUpdate, tagConfig.CheckInterval)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestRecoverExpiredPagesSourceLeaseUsesExactCASAndStableJitter(t *testing.T) {
|
||||||
|
ctx := setupPagesSourceTest(t)
|
||||||
|
project := mustCreatePagesSourceProject(t, ctx, "scanner-expired-lease")
|
||||||
|
source, _ := mustConfigureGitHubSourceWithoutDispatch(t, ctx, project.ID, SourceUpdateInput{
|
||||||
|
SourceType: PagesSourceTypeGitHubRelease,
|
||||||
|
RepositoryURL: "https://github.com/scanner/expired-lease",
|
||||||
|
})
|
||||||
|
now := time.Date(2026, 7, 19, 12, 0, 0, 0, time.UTC)
|
||||||
|
usePagesSourceScannerClock(t, now)
|
||||||
|
expiredAt := now.Add(-time.Minute)
|
||||||
|
if err := db.DB(ctx).Model(&model.PagesProjectSourceRuntime{}).
|
||||||
|
Where("source_id = ?", source.ID).
|
||||||
|
Updates(map[string]any{
|
||||||
|
"sync_status": pagesSourceStatusChecking,
|
||||||
|
"lease_token": "expired-owner",
|
||||||
|
"lease_expires_at": &expiredAt,
|
||||||
|
"next_check_at": &expiredAt,
|
||||||
|
}).Error; err != nil {
|
||||||
|
t.Fatalf("seed expired lease error = %v, want nil", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
summary := pagesSourceScanSummary{}
|
||||||
|
if err := recoverExpiredPagesSourceLeases(ctx, now, &summary); err != nil {
|
||||||
|
t.Fatalf("recoverExpiredPagesSourceLeases() error = %v, want nil", err)
|
||||||
|
}
|
||||||
|
if summary.ExpiredCandidates != 1 || summary.RecoveredLeases != 1 || summary.FailedSources != 0 {
|
||||||
|
t.Errorf("recovery summary = %+v, want one recovered lease", summary)
|
||||||
|
}
|
||||||
|
_, runtime := mustLoadPagesSource(t, ctx, project.ID)
|
||||||
|
wantNext := nextGitHubCheckAt(now, source.ID, minimumCheckInterval)
|
||||||
|
if runtime.SyncStatus != pagesSourceStatusFailed || runtime.LastError != errPagesSourceLeaseExpired ||
|
||||||
|
runtime.LeaseToken != "" || runtime.LeaseExpiresAt != nil || runtime.NextCheckAt == nil ||
|
||||||
|
runtime.NextCheckAt.Sub(wantNext) != 0 {
|
||||||
|
t.Errorf(
|
||||||
|
"recovered runtime = status:%q error:%q token:%q expiry:%v next:%v, want next %v",
|
||||||
|
runtime.SyncStatus,
|
||||||
|
runtime.LastError,
|
||||||
|
runtime.LeaseToken,
|
||||||
|
runtime.LeaseExpiresAt,
|
||||||
|
runtime.NextCheckAt,
|
||||||
|
wantNext,
|
||||||
|
)
|
||||||
|
}
|
||||||
|
|
||||||
|
renewedExpiry := now.Add(time.Minute)
|
||||||
|
if err := db.DB(ctx).Model(&model.PagesProjectSourceRuntime{}).
|
||||||
|
Where("source_id = ?", source.ID).
|
||||||
|
Updates(map[string]any{
|
||||||
|
"sync_status": pagesSourceStatusSyncing,
|
||||||
|
"lease_token": "renewed-owner",
|
||||||
|
"lease_expires_at": &renewedExpiry,
|
||||||
|
}).Error; err != nil {
|
||||||
|
t.Fatalf("seed renewed lease error = %v, want nil", err)
|
||||||
|
}
|
||||||
|
recovered, err := recoverExpiredSourceLease(
|
||||||
|
ctx,
|
||||||
|
source.ID,
|
||||||
|
"renewed-owner",
|
||||||
|
expiredAt,
|
||||||
|
pagesSourceStatusSyncing,
|
||||||
|
now,
|
||||||
|
&wantNext,
|
||||||
|
)
|
||||||
|
if err != nil || recovered {
|
||||||
|
t.Fatalf("recoverExpiredSourceLease(stale expiry) = %t, %v; want false, nil", recovered, err)
|
||||||
|
}
|
||||||
|
_, runtime = mustLoadPagesSource(t, ctx, project.ID)
|
||||||
|
if runtime.LeaseToken != "renewed-owner" || runtime.LeaseExpiresAt == nil ||
|
||||||
|
runtime.LeaseExpiresAt.Sub(renewedExpiry) != 0 || runtime.SyncStatus != pagesSourceStatusSyncing {
|
||||||
|
t.Errorf("stale recovery overwrote renewed lease: %+v", runtime)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestScheduledAutoSyncPersistsExplicitDeploymentTrigger(t *testing.T) {
|
||||||
|
ctx := setupPagesSourceSyncTest(t)
|
||||||
|
project := mustCreatePagesSourceProject(t, ctx, "scanner-scheduled-trigger")
|
||||||
|
source, _ := mustConfigureGitHubSourceWithoutDispatch(t, ctx, project.ID, SourceUpdateInput{
|
||||||
|
SourceType: PagesSourceTypeGitHubRelease,
|
||||||
|
RepositoryURL: "https://github.com/scanner/scheduled-trigger",
|
||||||
|
AutoUpdateEnabled: true,
|
||||||
|
})
|
||||||
|
packageBytes := testPagesZip(t, map[string]string{"index.html": "scheduled-v1"})
|
||||||
|
packageHash := sha256.Sum256(packageBytes)
|
||||||
|
release := githubrelease.Release{ID: "scheduled-release", Tag: "v1"}
|
||||||
|
asset := githubrelease.Asset{
|
||||||
|
ID: "scheduled-asset", Name: defaultGitHubAssetName, State: "uploaded",
|
||||||
|
UpdatedAt: time.Date(2026, 7, 19, 12, 30, 0, 0, time.UTC),
|
||||||
|
}
|
||||||
|
target, err := buildGitHubSourceTarget(release, asset, nil)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("buildGitHubSourceTarget() error = %v, want nil", err)
|
||||||
|
}
|
||||||
|
useFakeGitHubReleaseClient(t, &fakeGitHubReleaseClient{
|
||||||
|
resolve: func(context.Context, githubrelease.ResolveRequest) (githubrelease.ResolveResult, error) {
|
||||||
|
return githubrelease.ResolveResult{Release: release, Asset: asset}, nil
|
||||||
|
},
|
||||||
|
download: func(context.Context, githubrelease.DownloadRequest) (*githubrelease.DownloadResult, error) {
|
||||||
|
path := filepath.Join(t.TempDir(), "scheduled.zip")
|
||||||
|
if err := os.WriteFile(path, packageBytes, 0o600); err != nil {
|
||||||
|
t.Fatalf("os.WriteFile(scheduled package) error = %v", err)
|
||||||
|
}
|
||||||
|
return &githubrelease.DownloadResult{
|
||||||
|
Path: path, Size: int64(len(packageBytes)), SHA256: hex.EncodeToString(packageHash[:]),
|
||||||
|
}, nil
|
||||||
|
},
|
||||||
|
})
|
||||||
|
snapshot, outcome, err := acquireSourceLease(ctx, source.ID, source.ConfigVersion, sourceActionSync)
|
||||||
|
if err != nil || outcome != sourceLeaseAcquired {
|
||||||
|
t.Fatalf("acquireSourceLease() = %+v, %q, %v; want acquired", snapshot, outcome, err)
|
||||||
|
}
|
||||||
|
synced, err := syncGitHubSourceWithTrigger(
|
||||||
|
ctx,
|
||||||
|
snapshot,
|
||||||
|
pagesSourceCreatedBySystem,
|
||||||
|
target.Revision,
|
||||||
|
"",
|
||||||
|
pagesSourceTriggerScheduledAutoUpdate,
|
||||||
|
)
|
||||||
|
if err != nil || synced == nil || synced.Deployment == nil || synced.Stale {
|
||||||
|
t.Fatalf("syncGitHubSourceWithTrigger() = %+v, %v; want active deployment", synced, err)
|
||||||
|
}
|
||||||
|
deployment, err := model.GetPagesDeploymentByID(ctx, synced.Deployment.ID)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("GetPagesDeploymentByID(%d) error = %v", synced.Deployment.ID, err)
|
||||||
|
}
|
||||||
|
if deployment.TriggerType != pagesSourceTriggerScheduledAutoUpdate ||
|
||||||
|
deployment.CreatedBy != pagesSourceCreatedBySystem {
|
||||||
|
t.Errorf(
|
||||||
|
"scheduled provenance = trigger:%q actor:%q, want %q/%q",
|
||||||
|
deployment.TriggerType,
|
||||||
|
deployment.CreatedBy,
|
||||||
|
pagesSourceTriggerScheduledAutoUpdate,
|
||||||
|
pagesSourceCreatedBySystem,
|
||||||
|
)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestPagesSourceScannerSerialBatchIsolation304AndActualBacklog(t *testing.T) {
|
||||||
|
ctx := setupPagesSourceTest(t)
|
||||||
|
now := time.Now().Truncate(time.Second)
|
||||||
|
usePagesSourceScannerClock(t, now)
|
||||||
|
dueAt := now.Add(-time.Hour)
|
||||||
|
|
||||||
|
type fixture struct {
|
||||||
|
source *model.PagesProjectSource
|
||||||
|
runtime *model.PagesProjectSourceRuntime
|
||||||
|
repository string
|
||||||
|
}
|
||||||
|
fixtures := make([]fixture, 0, 22)
|
||||||
|
byRepository := make(map[string]int, 22)
|
||||||
|
for index := 1; index <= 22; index++ {
|
||||||
|
project := mustCreatePagesSourceProject(t, ctx, fmt.Sprintf("scanner-batch-%02d", index))
|
||||||
|
repository := fmt.Sprintf("scanner/source-%02d", index)
|
||||||
|
source, runtime := mustConfigureGitHubSourceWithoutDispatch(t, ctx, project.ID, SourceUpdateInput{
|
||||||
|
SourceType: PagesSourceTypeGitHubRelease,
|
||||||
|
RepositoryURL: "https://github.com/" + repository,
|
||||||
|
AutoUpdateEnabled: index != 4,
|
||||||
|
CheckIntervalMinutes: 60,
|
||||||
|
})
|
||||||
|
if err := db.DB(ctx).Model(&model.PagesProjectSourceRuntime{}).
|
||||||
|
Where("source_id = ?", source.ID).
|
||||||
|
Update("next_check_at", &dueAt).Error; err != nil {
|
||||||
|
t.Fatalf("mark source %d due error = %v, want nil", source.ID, err)
|
||||||
|
}
|
||||||
|
fixtures = append(fixtures, fixture{source: source, runtime: runtime, repository: repository})
|
||||||
|
byRepository[repository] = index
|
||||||
|
}
|
||||||
|
|
||||||
|
busyUntil := now.Add(time.Hour)
|
||||||
|
if err := db.DB(ctx).Model(&model.PagesProjectSourceRuntime{}).
|
||||||
|
Where("source_id = ?", fixtures[0].source.ID).
|
||||||
|
Updates(map[string]any{
|
||||||
|
"sync_status": pagesSourceStatusChecking,
|
||||||
|
"lease_token": "busy-owner",
|
||||||
|
"lease_expires_at": &busyUntil,
|
||||||
|
}).Error; err != nil {
|
||||||
|
t.Fatalf("seed busy source error = %v, want nil", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
stored304Revision := strings.Repeat("3", sourceRevisionHexLength)
|
||||||
|
if err := db.DB(ctx).Model(&model.PagesProjectSourceRuntime{}).
|
||||||
|
Where("source_id = ?", fixtures[2].source.ID).
|
||||||
|
Updates(map[string]any{
|
||||||
|
"etag": `"stored-etag"`,
|
||||||
|
"last_seen_revision": stored304Revision,
|
||||||
|
"last_seen_detail": `{"provider":"github","release_id":"release-3","asset_id":"3","tag":"v3","asset_name":"dist.zip"}`,
|
||||||
|
"sync_status": pagesSourceStatusUpdateAvailable,
|
||||||
|
}).Error; err != nil {
|
||||||
|
t.Fatalf("seed 304 cursor error = %v, want nil", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
if err := db.DB(ctx).Model(&model.PagesProjectSourceRuntime{}).
|
||||||
|
Where("source_id = ?", fixtures[4].source.ID).
|
||||||
|
Updates(map[string]any{
|
||||||
|
"last_applied_revision": strings.Repeat("a", sourceRevisionHexLength),
|
||||||
|
"last_applied_detail": `{"provider":"github","release_id":"shared-release","asset_id":"old","tag":"v5","asset_name":"dist.zip"}`,
|
||||||
|
}).Error; err != nil {
|
||||||
|
t.Fatalf("seed replacement cursor error = %v, want nil", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
retryAt := now.Add(2 * time.Hour)
|
||||||
|
calledRepositories := make([]string, 0, pagesSourceScanBatchSize)
|
||||||
|
useFakeGitHubReleaseClient(t, &fakeGitHubReleaseClient{
|
||||||
|
resolve: func(_ context.Context, request githubrelease.ResolveRequest) (githubrelease.ResolveResult, error) {
|
||||||
|
index := byRepository[request.Repository]
|
||||||
|
calledRepositories = append(calledRepositories, request.Repository)
|
||||||
|
switch index {
|
||||||
|
case 2:
|
||||||
|
return githubrelease.ResolveResult{}, &githubrelease.Error{
|
||||||
|
Kind: githubrelease.ErrMetadata, StatusCode: 429, RetryAt: &retryAt,
|
||||||
|
}
|
||||||
|
case 3:
|
||||||
|
if request.ETag != `"stored-etag"` {
|
||||||
|
t.Errorf("304 source ETag = %q, want stored ETag", request.ETag)
|
||||||
|
}
|
||||||
|
return githubrelease.ResolveResult{NotModified: true, ETag: request.ETag}, nil
|
||||||
|
default:
|
||||||
|
releaseID := fmt.Sprintf("release-%d", index)
|
||||||
|
if index == 5 {
|
||||||
|
releaseID = "shared-release"
|
||||||
|
}
|
||||||
|
result := githubrelease.ResolveResult{
|
||||||
|
ETag: fmt.Sprintf(`"etag-%d"`, index),
|
||||||
|
Release: githubrelease.Release{ID: releaseID, Tag: fmt.Sprintf("v%d", index)},
|
||||||
|
Asset: githubrelease.Asset{
|
||||||
|
ID: fmt.Sprintf("asset-%d", index),
|
||||||
|
Name: defaultGitHubAssetName,
|
||||||
|
State: "uploaded",
|
||||||
|
UpdatedAt: now.Add(time.Duration(index) * time.Minute),
|
||||||
|
},
|
||||||
|
}
|
||||||
|
if index == 6 {
|
||||||
|
result.RetryAt = &retryAt
|
||||||
|
}
|
||||||
|
return result, nil
|
||||||
|
}
|
||||||
|
},
|
||||||
|
download: func(context.Context, githubrelease.DownloadRequest) (*githubrelease.DownloadResult, error) {
|
||||||
|
t.Fatal("scanner downloaded an asset; want check-only behavior")
|
||||||
|
return nil, nil
|
||||||
|
},
|
||||||
|
})
|
||||||
|
|
||||||
|
dispatched := make([]scannerDispatchedSync, 0, pagesSourceScanBatchSize)
|
||||||
|
previousDispatch := dispatchPagesSourceAutoSync
|
||||||
|
dispatchPagesSourceAutoSync = func(
|
||||||
|
_ context.Context,
|
||||||
|
source model.PagesProjectSource,
|
||||||
|
revision string,
|
||||||
|
) (*SourceActionReceipt, error) {
|
||||||
|
if source.ID == fixtures[5].source.ID {
|
||||||
|
return nil, errors.New("injected dispatch failure")
|
||||||
|
}
|
||||||
|
dispatched = append(dispatched, scannerDispatchedSync{SourceID: source.ID, Revision: revision})
|
||||||
|
return &SourceActionReceipt{ExecutionID: fmt.Sprintf("%d", source.ID), Action: sourceActionSync}, nil
|
||||||
|
}
|
||||||
|
t.Cleanup(func() { dispatchPagesSourceAutoSync = previousDispatch })
|
||||||
|
|
||||||
|
result, err := (&SourceScanHandler{}).Execute(ctx, []byte("{}"))
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("SourceScanHandler.Execute() error = %v, want nil", err)
|
||||||
|
}
|
||||||
|
var summary pagesSourceScanSummary
|
||||||
|
if err := json.Unmarshal([]byte(result.Detail), &summary); err != nil {
|
||||||
|
t.Fatalf("json.Unmarshal(scan detail) error = %v, want nil", err)
|
||||||
|
}
|
||||||
|
if summary.DueSources != 22 || summary.SelectedSources != pagesSourceScanBatchSize ||
|
||||||
|
summary.CheckedSources != 18 || summary.UpdatesFound != 17 || summary.AttentionSources != 1 ||
|
||||||
|
summary.DispatchedSyncs != 15 || summary.FailedDispatches != 1 ||
|
||||||
|
summary.BusySources != 1 || summary.FailedSources != 2 ||
|
||||||
|
summary.Backlog != 3 {
|
||||||
|
t.Errorf("scan summary = %+v, want due=22 selected=20 checked=18 updates=17 attention=1 dispatched=15 dispatch_failed=1 busy=1 failed=2 backlog=3", summary)
|
||||||
|
}
|
||||||
|
if len(summary.ProviderBackoffs) != 1 ||
|
||||||
|
summary.ProviderBackoffs[0].SourceID != fixtures[1].source.ID ||
|
||||||
|
summary.ProviderBackoffs[0].StatusCode != 429 ||
|
||||||
|
summary.ProviderBackoffs[0].RetryAt != retryAt.UTC().Format(time.RFC3339) {
|
||||||
|
t.Errorf("scan provider backoffs = %+v, want source=%d status=429 retry_at=%s", summary.ProviderBackoffs, fixtures[1].source.ID, retryAt.UTC().Format(time.RFC3339))
|
||||||
|
}
|
||||||
|
|
||||||
|
if len(calledRepositories) != 19 {
|
||||||
|
t.Fatalf("Resolve calls = %d, want 19 (one busy source in selected batch)", len(calledRepositories))
|
||||||
|
}
|
||||||
|
for index, repository := range calledRepositories {
|
||||||
|
want := fixtures[index+1].repository
|
||||||
|
if repository != want {
|
||||||
|
t.Fatalf("Resolve order[%d] = %q, want %q", index, repository, want)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
if !containsDispatchedSource(dispatched, fixtures[2].source.ID, stored304Revision) {
|
||||||
|
t.Errorf("304 stored revision was not dispatched: %+v", dispatched)
|
||||||
|
}
|
||||||
|
if containsDispatchedSource(dispatched, fixtures[3].source.ID, "") {
|
||||||
|
t.Errorf("auto=false source was dispatched: %+v", dispatched)
|
||||||
|
}
|
||||||
|
if containsDispatchedSource(dispatched, fixtures[4].source.ID, "") {
|
||||||
|
t.Errorf("attention source was dispatched: %+v", dispatched)
|
||||||
|
}
|
||||||
|
|
||||||
|
_, dispatchFailedRuntime := mustLoadPagesSource(t, ctx, fixtures[5].source.ProjectID)
|
||||||
|
if dispatchFailedRuntime.SyncStatus != pagesSourceStatusUpdateAvailable ||
|
||||||
|
dispatchFailedRuntime.LastError != errPagesSourceTaskDispatchFailed ||
|
||||||
|
dispatchFailedRuntime.NextCheckAt == nil || dispatchFailedRuntime.NextCheckAt.Before(retryAt) ||
|
||||||
|
dispatchFailedRuntime.LastSeenRevision == "" {
|
||||||
|
t.Errorf(
|
||||||
|
"dispatch failure runtime = status:%q error:%q next:%v seen:%q, want preserved update and provider deadline >= %v",
|
||||||
|
dispatchFailedRuntime.SyncStatus,
|
||||||
|
dispatchFailedRuntime.LastError,
|
||||||
|
dispatchFailedRuntime.NextCheckAt,
|
||||||
|
dispatchFailedRuntime.LastSeenRevision,
|
||||||
|
retryAt,
|
||||||
|
)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestPagesSourceScannerIncludesOrphanCleanupSummary(t *testing.T) {
|
||||||
|
ctx := setupPagesSourceTest(t)
|
||||||
|
now := time.Now().Truncate(time.Second)
|
||||||
|
usePagesSourceScannerClock(t, now)
|
||||||
|
|
||||||
|
previousReconcile := reconcilePagesSourceOrphans
|
||||||
|
reconcilePagesSourceOrphans = func(
|
||||||
|
_ context.Context,
|
||||||
|
gotNow time.Time,
|
||||||
|
) (PagesOrphanCleanupSummary, error) {
|
||||||
|
if !gotNow.Equal(now) {
|
||||||
|
t.Errorf("orphan cleanup now = %v, want %v", gotNow, now)
|
||||||
|
}
|
||||||
|
return PagesOrphanCleanupSummary{
|
||||||
|
Candidates: 7,
|
||||||
|
Reconciled: 1,
|
||||||
|
Referenced: 2,
|
||||||
|
LeaseBusy: 1,
|
||||||
|
InvalidMarker: 1,
|
||||||
|
Skipped: 1,
|
||||||
|
Failed: 1,
|
||||||
|
}, nil
|
||||||
|
}
|
||||||
|
t.Cleanup(func() { reconcilePagesSourceOrphans = previousReconcile })
|
||||||
|
|
||||||
|
result, err := (&SourceScanHandler{}).Execute(ctx, []byte("{}"))
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("SourceScanHandler.Execute() error = %v, want nil", err)
|
||||||
|
}
|
||||||
|
var summary pagesSourceScanSummary
|
||||||
|
if err := json.Unmarshal([]byte(result.Detail), &summary); err != nil {
|
||||||
|
t.Fatalf("json.Unmarshal(scan detail) error = %v, want nil", err)
|
||||||
|
}
|
||||||
|
if summary.OrphanCleanup.Candidates != 7 || summary.OrphanCleanup.Reconciled != 1 ||
|
||||||
|
summary.OrphanCleanup.Referenced != 2 || summary.OrphanCleanup.LeaseBusy != 1 ||
|
||||||
|
summary.OrphanCleanup.InvalidMarker != 1 || summary.OrphanCleanup.Skipped != 1 ||
|
||||||
|
summary.OrphanCleanup.Failed != 1 {
|
||||||
|
t.Errorf("orphan cleanup summary = %+v, want injected result", summary.OrphanCleanup)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestPagesSourceScanPayloadAndMetaAreInternalOnly(t *testing.T) {
|
||||||
|
handler := &SourceScanHandler{}
|
||||||
|
if normalized, err := handler.ValidatePayload(nil); err != nil || string(normalized) != "{}" {
|
||||||
|
t.Errorf("ValidatePayload(nil) = %s, %v; want {}, nil", normalized, err)
|
||||||
|
}
|
||||||
|
if _, err := handler.ValidatePayload([]byte(`{"unexpected":true}`)); err == nil {
|
||||||
|
t.Error("ValidatePayload(unknown field) error = nil, want non-nil")
|
||||||
|
}
|
||||||
|
if !PagesSourceScanMeta.InternalOnly || PagesSourceScanMeta.Type != TaskTypePagesSourceScan ||
|
||||||
|
PagesSourceScanMeta.AsynqTask != PagesSourceScanTask || PagesSourceScanMeta.MaxRetry != 0 {
|
||||||
|
t.Errorf("PagesSourceScanMeta = %+v, want internal bounded scheduled scanner", PagesSourceScanMeta)
|
||||||
|
}
|
||||||
|
if PagesSourceScanMeta.SupportsTime {
|
||||||
|
t.Error("PagesSourceScanMeta.SupportsTime = true, want empty scanner payload")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func usePagesSourceScannerClock(t *testing.T, now time.Time) {
|
||||||
|
t.Helper()
|
||||||
|
previous := pagesSourceScanNow
|
||||||
|
pagesSourceScanNow = func() time.Time { return now }
|
||||||
|
t.Cleanup(func() { pagesSourceScanNow = previous })
|
||||||
|
}
|
||||||
|
|
||||||
|
func containsDispatchedSource(dispatched []scannerDispatchedSync, sourceID uint, revision string) bool {
|
||||||
|
for _, item := range dispatched {
|
||||||
|
if item.SourceID == sourceID && (revision == "" || item.Revision == revision) {
|
||||||
|
return true
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return false
|
||||||
|
}
|
||||||
@@ -0,0 +1,745 @@
|
|||||||
|
// Copyright 2026 Arctel.net
|
||||||
|
// SPDX-License-Identifier: Apache-2.0
|
||||||
|
|
||||||
|
package pages
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
"encoding/json"
|
||||||
|
"errors"
|
||||||
|
"fmt"
|
||||||
|
"path"
|
||||||
|
"sort"
|
||||||
|
"strings"
|
||||||
|
"sync"
|
||||||
|
"time"
|
||||||
|
"unicode/utf8"
|
||||||
|
|
||||||
|
"github.com/Rain-kl/Wavelet/internal/apps/upload"
|
||||||
|
"github.com/Rain-kl/Wavelet/internal/db"
|
||||||
|
"github.com/Rain-kl/Wavelet/internal/model"
|
||||||
|
"github.com/Rain-kl/Wavelet/internal/task"
|
||||||
|
"github.com/Rain-kl/Wavelet/pkg/logger"
|
||||||
|
"gorm.io/gorm"
|
||||||
|
"gorm.io/gorm/clause"
|
||||||
|
)
|
||||||
|
|
||||||
|
const (
|
||||||
|
pagesSourceTriggerManualSync = "manual_sync"
|
||||||
|
pagesSourceTriggerScheduledAutoUpdate = "scheduled_auto_update"
|
||||||
|
pagesSourceCreatedBySystem = "system:pages-source-sync"
|
||||||
|
pagesSourceHeartbeatInterval = pagesSourceSyncLeaseDuration / 3
|
||||||
|
pagesSourceCleanupTimeout = 15 * time.Second
|
||||||
|
)
|
||||||
|
|
||||||
|
var (
|
||||||
|
errSourceFinalFence = errors.New("pages source final fence rejected")
|
||||||
|
errSourceLeaseHeartbeatLost = errors.New("pages source lease heartbeat lost")
|
||||||
|
sourceCommitNow = time.Now
|
||||||
|
)
|
||||||
|
|
||||||
|
type sourceSyncOutcome struct {
|
||||||
|
Deployment *DeploymentView
|
||||||
|
Reused bool
|
||||||
|
Stale bool
|
||||||
|
}
|
||||||
|
|
||||||
|
type preparedRemoteSource struct {
|
||||||
|
Candidate *SourceCandidate
|
||||||
|
Manifest *deploymentManifest
|
||||||
|
Detail sourceDetail
|
||||||
|
DetailJSON string
|
||||||
|
}
|
||||||
|
|
||||||
|
type sourceIngestState struct {
|
||||||
|
Result upload.IngestResult
|
||||||
|
HasIngest bool
|
||||||
|
Referenced bool
|
||||||
|
}
|
||||||
|
|
||||||
|
type sourceCommitState struct {
|
||||||
|
Project *model.PagesProject
|
||||||
|
Source *model.PagesProjectSource
|
||||||
|
Runtime *model.PagesProjectSourceRuntime
|
||||||
|
Now time.Time
|
||||||
|
}
|
||||||
|
|
||||||
|
type sourceLeaseHeartbeat struct {
|
||||||
|
cancel context.CancelFunc
|
||||||
|
done <-chan error
|
||||||
|
stopOnce sync.Once
|
||||||
|
stopErr error
|
||||||
|
}
|
||||||
|
|
||||||
|
func startSourceLeaseHeartbeat(
|
||||||
|
ctx context.Context,
|
||||||
|
snapshot *sourceExecutionSnapshot,
|
||||||
|
leaseDuration time.Duration,
|
||||||
|
interval time.Duration,
|
||||||
|
) (context.Context, *sourceLeaseHeartbeat, error) {
|
||||||
|
if snapshot == nil || leaseDuration <= 0 || interval <= 0 || interval >= leaseDuration {
|
||||||
|
return nil, nil, errors.New(errPagesSourceLeaseLost)
|
||||||
|
}
|
||||||
|
renewed, err := renewSourceLease(ctx, snapshot, leaseDuration)
|
||||||
|
if err != nil {
|
||||||
|
return nil, nil, err
|
||||||
|
}
|
||||||
|
if !renewed {
|
||||||
|
return nil, nil, errSourceLeaseHeartbeatLost
|
||||||
|
}
|
||||||
|
|
||||||
|
workCtx, cancel := context.WithCancel(ctx)
|
||||||
|
done := make(chan error, 1)
|
||||||
|
heartbeat := &sourceLeaseHeartbeat{cancel: cancel, done: done}
|
||||||
|
go func() {
|
||||||
|
ticker := time.NewTicker(interval)
|
||||||
|
defer ticker.Stop()
|
||||||
|
for {
|
||||||
|
select {
|
||||||
|
case <-workCtx.Done():
|
||||||
|
done <- nil
|
||||||
|
return
|
||||||
|
case <-ticker.C:
|
||||||
|
renewed, renewErr := renewSourceLease(workCtx, snapshot, leaseDuration)
|
||||||
|
if renewErr != nil {
|
||||||
|
if workCtx.Err() != nil {
|
||||||
|
done <- nil
|
||||||
|
return
|
||||||
|
}
|
||||||
|
done <- renewErr
|
||||||
|
cancel()
|
||||||
|
return
|
||||||
|
}
|
||||||
|
if !renewed {
|
||||||
|
done <- errSourceLeaseHeartbeatLost
|
||||||
|
cancel()
|
||||||
|
return
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}()
|
||||||
|
return workCtx, heartbeat, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func (heartbeat *sourceLeaseHeartbeat) stop() error {
|
||||||
|
if heartbeat == nil {
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
heartbeat.stopOnce.Do(func() {
|
||||||
|
heartbeat.cancel()
|
||||||
|
heartbeat.stopErr = <-heartbeat.done
|
||||||
|
})
|
||||||
|
return heartbeat.stopErr
|
||||||
|
}
|
||||||
|
|
||||||
|
func sourceHeartbeatOutcome(err error) (*sourceSyncOutcome, error) {
|
||||||
|
if errors.Is(err, errSourceLeaseHeartbeatLost) {
|
||||||
|
return &sourceSyncOutcome{Stale: true}, nil
|
||||||
|
}
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
|
||||||
|
func recordSourceLeaseFailure(ctx context.Context, snapshot *sourceExecutionSnapshot) {
|
||||||
|
cleanupCtx, cancel := sourceCleanupContext(ctx)
|
||||||
|
defer cancel()
|
||||||
|
if err := failSourceLease(cleanupCtx, snapshot, errPagesSourceSyncFailed); err != nil {
|
||||||
|
var sourceID uint
|
||||||
|
if snapshot != nil {
|
||||||
|
sourceID = snapshot.SourceID
|
||||||
|
}
|
||||||
|
logger.WarnF(cleanupCtx, "[PagesSource] record failed runtime state failed: source_id=%d error=%v", sourceID, err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func sourceCleanupContext(ctx context.Context) (context.Context, context.CancelFunc) {
|
||||||
|
return context.WithTimeout(context.WithoutCancel(ctx), pagesSourceCleanupTimeout)
|
||||||
|
}
|
||||||
|
|
||||||
|
func syncRemoteSourceWithTrigger(
|
||||||
|
ctx context.Context,
|
||||||
|
snapshot *sourceExecutionSnapshot,
|
||||||
|
actor string,
|
||||||
|
triggerType string,
|
||||||
|
) (outcome *sourceSyncOutcome, resultErr error) {
|
||||||
|
if snapshot == nil || snapshot.SourceType != PagesSourceTypeRemoteURL {
|
||||||
|
return nil, errors.New(errPagesSourceTypeUnsupported)
|
||||||
|
}
|
||||||
|
actor = strings.TrimSpace(actor)
|
||||||
|
if actor == "" || !validSourceDeploymentTrigger(triggerType) {
|
||||||
|
return nil, errors.New(errPagesSourceActionInvalid)
|
||||||
|
}
|
||||||
|
defer func() {
|
||||||
|
if resultErr != nil {
|
||||||
|
recordSourceLeaseFailure(ctx, snapshot)
|
||||||
|
}
|
||||||
|
}()
|
||||||
|
|
||||||
|
workCtx, heartbeat, err := startSourceLeaseHeartbeat(
|
||||||
|
ctx,
|
||||||
|
snapshot,
|
||||||
|
pagesSourceSyncLeaseDuration,
|
||||||
|
pagesSourceHeartbeatInterval,
|
||||||
|
)
|
||||||
|
if err != nil {
|
||||||
|
return sourceHeartbeatOutcome(err)
|
||||||
|
}
|
||||||
|
defer func() {
|
||||||
|
_ = heartbeat.stop()
|
||||||
|
}()
|
||||||
|
|
||||||
|
limits := resolvePagesLimits(workCtx)
|
||||||
|
prepared, err := prepareRemoteSource(workCtx, snapshot, limits)
|
||||||
|
if err != nil {
|
||||||
|
if heartbeatErr := heartbeat.stop(); heartbeatErr != nil {
|
||||||
|
return sourceHeartbeatOutcome(heartbeatErr)
|
||||||
|
}
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
defer func() {
|
||||||
|
if cleanupErr := prepared.Candidate.Cleanup(); cleanupErr != nil {
|
||||||
|
logger.WarnF(ctx, "[PagesSource] cleanup temporary package failed: source_id=%d error=%v", snapshot.SourceID, cleanupErr)
|
||||||
|
}
|
||||||
|
}()
|
||||||
|
|
||||||
|
ingestState, err := resolveSourceIngest(workCtx, snapshot, prepared)
|
||||||
|
if err != nil {
|
||||||
|
if heartbeatErr := heartbeat.stop(); heartbeatErr != nil {
|
||||||
|
return sourceHeartbeatOutcome(heartbeatErr)
|
||||||
|
}
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
defer func() {
|
||||||
|
compensateSourceIngest(ctx, snapshot, ingestState)
|
||||||
|
}()
|
||||||
|
|
||||||
|
if heartbeatErr := heartbeat.stop(); heartbeatErr != nil {
|
||||||
|
return sourceHeartbeatOutcome(heartbeatErr)
|
||||||
|
}
|
||||||
|
renewed, err := renewSourceLease(ctx, snapshot, pagesSourceSyncLeaseDuration)
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
if !renewed {
|
||||||
|
return &sourceSyncOutcome{Stale: true}, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
task.AppendLog(ctx, "[activate] 正在原子切换生产部署")
|
||||||
|
deployment, reused, referenced, err := commitSourceDeploymentWithTrigger(
|
||||||
|
ctx,
|
||||||
|
snapshot,
|
||||||
|
prepared.Candidate.Checksum,
|
||||||
|
prepared.Candidate.Checksum,
|
||||||
|
prepared.Detail,
|
||||||
|
prepared.DetailJSON,
|
||||||
|
actor,
|
||||||
|
triggerType,
|
||||||
|
prepared.Manifest,
|
||||||
|
ingestState.Result,
|
||||||
|
ingestState.HasIngest,
|
||||||
|
nil,
|
||||||
|
)
|
||||||
|
ingestState.Referenced = referenced
|
||||||
|
if errors.Is(err, errSourceFinalFence) {
|
||||||
|
return &sourceSyncOutcome{Stale: true}, nil
|
||||||
|
}
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
ingestState.Referenced = ingestState.HasIngest && deployment.UploadID == ingestState.Result.Upload.ID
|
||||||
|
|
||||||
|
if pruneErr := pruneProjectDeploymentHistory(ctx, snapshot.ProjectID, limits.HistoryCount, 0); pruneErr != nil {
|
||||||
|
logger.ErrorF(ctx,
|
||||||
|
"[PagesSource] strict prune failed: project_id=%d source_id=%d keep=%d error=%v",
|
||||||
|
snapshot.ProjectID,
|
||||||
|
snapshot.SourceID,
|
||||||
|
limits.HistoryCount,
|
||||||
|
pruneErr,
|
||||||
|
)
|
||||||
|
}
|
||||||
|
view := buildDeploymentView(deployment)
|
||||||
|
return &sourceSyncOutcome{Deployment: &view, Reused: reused}, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func prepareRemoteSource(
|
||||||
|
ctx context.Context,
|
||||||
|
snapshot *sourceExecutionSnapshot,
|
||||||
|
limits pagesLimits,
|
||||||
|
) (*preparedRemoteSource, error) {
|
||||||
|
task.AppendLog(ctx, "[download] 正在获取远程部署包")
|
||||||
|
candidate, err := FetchRemoteSource(ctx, RemoteSourceRequest{
|
||||||
|
URL: snapshot.RemoteURL,
|
||||||
|
NetworkPolicy: snapshot.RemoteNetworkPolicy,
|
||||||
|
MaxPackageBytes: limits.PackageBytes,
|
||||||
|
})
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
if candidate == nil || candidate.TempPath == "" || candidate.Checksum == "" || candidate.Format == "" {
|
||||||
|
return nil, errors.New(errPagesSourceSyncFailed)
|
||||||
|
}
|
||||||
|
rootDir, err := validateAndNormalizePagesRootDir(snapshot.RootDir)
|
||||||
|
if err != nil {
|
||||||
|
cleanupFailedRemoteCandidate(ctx, snapshot, candidate)
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
entryFile, err := validateAndNormalizePagesEntryFile(snapshot.EntryFile)
|
||||||
|
if err != nil {
|
||||||
|
cleanupFailedRemoteCandidate(ctx, snapshot, candidate)
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
task.AppendLog(ctx, "[verify] 正在校验归档结构与入口文件")
|
||||||
|
manifest, err := inspectPagesPackage(candidate.TempPath, candidate.Format, rootDir, entryFile, limits)
|
||||||
|
if err != nil {
|
||||||
|
cleanupFailedRemoteCandidate(ctx, snapshot, candidate)
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
detail := sourceDetail{Provider: PagesSourceTypeRemoteURL, DisplayName: safeRemoteSourceLabel(candidate.SafeLabel)}
|
||||||
|
detailJSON, err := json.Marshal(detail)
|
||||||
|
if err != nil {
|
||||||
|
cleanupFailedRemoteCandidate(ctx, snapshot, candidate)
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
return &preparedRemoteSource{
|
||||||
|
Candidate: candidate,
|
||||||
|
Manifest: manifest,
|
||||||
|
Detail: detail,
|
||||||
|
DetailJSON: string(detailJSON),
|
||||||
|
}, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func cleanupFailedRemoteCandidate(
|
||||||
|
ctx context.Context,
|
||||||
|
snapshot *sourceExecutionSnapshot,
|
||||||
|
candidate *SourceCandidate,
|
||||||
|
) {
|
||||||
|
if err := candidate.Cleanup(); err != nil {
|
||||||
|
logger.WarnF(ctx, "[PagesSource] cleanup failed preparation package: source_id=%d error=%v", snapshot.SourceID, err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func resolveSourceIngest(
|
||||||
|
ctx context.Context,
|
||||||
|
snapshot *sourceExecutionSnapshot,
|
||||||
|
prepared *preparedRemoteSource,
|
||||||
|
) (*sourceIngestState, error) {
|
||||||
|
_, err := findSourceDeployment(
|
||||||
|
ctx,
|
||||||
|
snapshot.ProjectID,
|
||||||
|
snapshot.SourceIdentity,
|
||||||
|
prepared.Candidate.Checksum,
|
||||||
|
)
|
||||||
|
if err == nil {
|
||||||
|
return &sourceIngestState{}, nil
|
||||||
|
}
|
||||||
|
if !errors.Is(err, gorm.ErrRecordNotFound) {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
task.AppendLog(ctx, "[ingest] 正在保存受管部署包")
|
||||||
|
result, err := ingestPagesDeploymentPackageWithSource(
|
||||||
|
ctx,
|
||||||
|
prepared.Candidate.TempPath,
|
||||||
|
prepared.Candidate.Checksum,
|
||||||
|
snapshot.ProjectID,
|
||||||
|
snapshot.SourceID,
|
||||||
|
sourceDetailLabel(prepared.Detail),
|
||||||
|
prepared.Candidate.Format,
|
||||||
|
)
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
return &sourceIngestState{Result: result, HasIngest: true}, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func compensateSourceIngest(
|
||||||
|
ctx context.Context,
|
||||||
|
snapshot *sourceExecutionSnapshot,
|
||||||
|
state *sourceIngestState,
|
||||||
|
) {
|
||||||
|
if state == nil || !state.HasIngest || !state.Result.Created || state.Referenced {
|
||||||
|
return
|
||||||
|
}
|
||||||
|
cleanupCtx, cancel := sourceCleanupContext(ctx)
|
||||||
|
defer cancel()
|
||||||
|
task.AppendLog(cleanupCtx, "[cleanup] 正在补偿未引用的部署包记录")
|
||||||
|
if err := removePagesUploadIfUnreferenced(cleanupCtx, snapshot.ProjectID, state.Result.Upload.ID); err != nil {
|
||||||
|
logger.ErrorF(cleanupCtx,
|
||||||
|
"[PagesSource] compensate upload failed: project_id=%d source_id=%d upload_id=%d error=%v",
|
||||||
|
snapshot.ProjectID, snapshot.SourceID, state.Result.Upload.ID, err,
|
||||||
|
)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func findSourceDeployment(
|
||||||
|
ctx context.Context,
|
||||||
|
projectID uint,
|
||||||
|
sourceIdentity string,
|
||||||
|
revision string,
|
||||||
|
) (*model.PagesDeployment, error) {
|
||||||
|
var deployment model.PagesDeployment
|
||||||
|
err := db.DB(ctx).
|
||||||
|
Where("project_id = ? AND source_identity = ? AND source_revision = ?", projectID, sourceIdentity, revision).
|
||||||
|
First(&deployment).Error
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
return &deployment, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func commitSourceDeploymentWithTrigger(
|
||||||
|
ctx context.Context,
|
||||||
|
snapshot *sourceExecutionSnapshot,
|
||||||
|
revision string,
|
||||||
|
packageChecksum string,
|
||||||
|
detail sourceDetail,
|
||||||
|
detailJSON string,
|
||||||
|
actor string,
|
||||||
|
triggerType string,
|
||||||
|
manifest *deploymentManifest,
|
||||||
|
ingestResult upload.IngestResult,
|
||||||
|
hasIngest bool,
|
||||||
|
nextCheckNotBefore *time.Time,
|
||||||
|
) (*model.PagesDeployment, bool, bool, error) {
|
||||||
|
if snapshot == nil || manifest == nil {
|
||||||
|
return nil, false, false, errors.New(errPagesSourceSyncFailed)
|
||||||
|
}
|
||||||
|
if !validSourceDeploymentTrigger(triggerType) {
|
||||||
|
return nil, false, false, errors.New(errPagesSourceActionInvalid)
|
||||||
|
}
|
||||||
|
var committed model.PagesDeployment
|
||||||
|
reused := false
|
||||||
|
ingestReferenced := false
|
||||||
|
err := db.DB(ctx).Transaction(func(tx *gorm.DB) error {
|
||||||
|
state, err := lockSourceCommitState(tx, snapshot)
|
||||||
|
if err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
target, targetReused, err := resolveSourceDeploymentTx(
|
||||||
|
tx, state, revision, packageChecksum, detail, detailJSON, actor, triggerType,
|
||||||
|
manifest, ingestResult, hasIngest,
|
||||||
|
)
|
||||||
|
if err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
if err := lockSourceDeploymentUploadsTx(tx, target, ingestResult, hasIngest); err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
if err := ensureDeploymentEntry(tx, target.ID, state.Project.RootDir, state.Project.EntryFile); err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
if err := refreshSourceCommitLease(state, snapshot); err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
if err := activateSourceDeploymentTx(tx, state, target, revision, detailJSON, nextCheckNotBefore); err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
committed = *target
|
||||||
|
reused = targetReused
|
||||||
|
ingestReferenced = hasIngest && target.UploadID == ingestResult.Upload.ID
|
||||||
|
return nil
|
||||||
|
})
|
||||||
|
if err != nil {
|
||||||
|
return nil, false, false, err
|
||||||
|
}
|
||||||
|
return &committed, reused, ingestReferenced, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func lockSourceCommitState(tx *gorm.DB, snapshot *sourceExecutionSnapshot) (*sourceCommitState, error) {
|
||||||
|
state := &sourceCommitState{}
|
||||||
|
var project model.PagesProject
|
||||||
|
if err := tx.Clauses(clause.Locking{Strength: pagesRowLockStrength}).
|
||||||
|
First(&project, snapshot.ProjectID).Error; err != nil {
|
||||||
|
return nil, sourceFenceRecordError(err)
|
||||||
|
}
|
||||||
|
if project.ContentConfigVersion != snapshot.ContentConfigVersion {
|
||||||
|
return nil, errSourceFinalFence
|
||||||
|
}
|
||||||
|
var source model.PagesProjectSource
|
||||||
|
if err := tx.Clauses(clause.Locking{Strength: pagesRowLockStrength}).
|
||||||
|
Where("id = ? AND project_id = ?", snapshot.SourceID, snapshot.ProjectID).
|
||||||
|
First(&source).Error; err != nil {
|
||||||
|
return nil, sourceFenceRecordError(err)
|
||||||
|
}
|
||||||
|
if source.ConfigVersion != snapshot.SourceConfigVersion ||
|
||||||
|
source.SourceIdentity != snapshot.SourceIdentity ||
|
||||||
|
source.SourceType != snapshot.SourceType {
|
||||||
|
return nil, errSourceFinalFence
|
||||||
|
}
|
||||||
|
var runtime model.PagesProjectSourceRuntime
|
||||||
|
if err := tx.Clauses(clause.Locking{Strength: pagesRowLockStrength}).
|
||||||
|
Where("source_id = ?", source.ID).
|
||||||
|
First(&runtime).Error; err != nil {
|
||||||
|
return nil, sourceFenceRecordError(err)
|
||||||
|
}
|
||||||
|
state.Project = &project
|
||||||
|
state.Source = &source
|
||||||
|
state.Runtime = &runtime
|
||||||
|
if err := refreshSourceCommitLease(state, snapshot); err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
return state, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func refreshSourceCommitLease(state *sourceCommitState, snapshot *sourceExecutionSnapshot) error {
|
||||||
|
if state == nil || state.Runtime == nil || snapshot == nil {
|
||||||
|
return errSourceFinalFence
|
||||||
|
}
|
||||||
|
now := sourceCommitNow()
|
||||||
|
if state.Runtime.LeaseToken != snapshot.LeaseToken || state.Runtime.LeaseExpiresAt == nil ||
|
||||||
|
!state.Runtime.LeaseExpiresAt.After(now) {
|
||||||
|
return errSourceFinalFence
|
||||||
|
}
|
||||||
|
state.Now = now
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func sourceFenceRecordError(err error) error {
|
||||||
|
if errors.Is(err, gorm.ErrRecordNotFound) {
|
||||||
|
return errSourceFinalFence
|
||||||
|
}
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
|
||||||
|
func resolveSourceDeploymentTx(
|
||||||
|
tx *gorm.DB,
|
||||||
|
state *sourceCommitState,
|
||||||
|
revision string,
|
||||||
|
packageChecksum string,
|
||||||
|
detail sourceDetail,
|
||||||
|
detailJSON string,
|
||||||
|
actor string,
|
||||||
|
triggerType string,
|
||||||
|
manifest *deploymentManifest,
|
||||||
|
ingestResult upload.IngestResult,
|
||||||
|
hasIngest bool,
|
||||||
|
) (*model.PagesDeployment, bool, error) {
|
||||||
|
var target model.PagesDeployment
|
||||||
|
err := tx.Where(
|
||||||
|
"project_id = ? AND source_identity = ? AND source_revision = ?",
|
||||||
|
state.Project.ID,
|
||||||
|
state.Source.SourceIdentity,
|
||||||
|
revision,
|
||||||
|
).First(&target).Error
|
||||||
|
if err == nil {
|
||||||
|
return &target, true, nil
|
||||||
|
}
|
||||||
|
if !errors.Is(err, gorm.ErrRecordNotFound) {
|
||||||
|
return nil, false, err
|
||||||
|
}
|
||||||
|
if !hasIngest {
|
||||||
|
return nil, false, errSourceFinalFence
|
||||||
|
}
|
||||||
|
return createSourceDeploymentTx(
|
||||||
|
tx, state, revision, packageChecksum, detail, detailJSON, actor, triggerType, manifest, ingestResult,
|
||||||
|
)
|
||||||
|
}
|
||||||
|
|
||||||
|
func createSourceDeploymentTx(
|
||||||
|
tx *gorm.DB,
|
||||||
|
state *sourceCommitState,
|
||||||
|
revision string,
|
||||||
|
packageChecksum string,
|
||||||
|
detail sourceDetail,
|
||||||
|
detailJSON string,
|
||||||
|
actor string,
|
||||||
|
triggerType string,
|
||||||
|
manifest *deploymentManifest,
|
||||||
|
ingestResult upload.IngestResult,
|
||||||
|
) (*model.PagesDeployment, bool, error) {
|
||||||
|
var maxNumber int
|
||||||
|
if err := tx.Model(&model.PagesDeployment{}).
|
||||||
|
Where("project_id = ?", state.Project.ID).
|
||||||
|
Select("COALESCE(MAX(deployment_number), 0)").
|
||||||
|
Scan(&maxNumber).Error; err != nil {
|
||||||
|
return nil, false, err
|
||||||
|
}
|
||||||
|
identity := state.Source.SourceIdentity
|
||||||
|
revisionValue := revision
|
||||||
|
target := &model.PagesDeployment{
|
||||||
|
ProjectID: state.Project.ID,
|
||||||
|
DeploymentNumber: maxNumber + 1,
|
||||||
|
Checksum: packageChecksum,
|
||||||
|
Status: model.PagesDeploymentStatusUploaded,
|
||||||
|
UploadID: ingestResult.Upload.ID,
|
||||||
|
FileCount: manifest.FileCount,
|
||||||
|
TotalSize: manifest.TotalSize,
|
||||||
|
CreatedBy: actor,
|
||||||
|
SourceType: state.Source.SourceType,
|
||||||
|
SourceIdentity: &identity,
|
||||||
|
SourceRevision: &revisionValue,
|
||||||
|
SourceLabel: sourceDetailLabel(detail),
|
||||||
|
SourceMeta: detailJSON,
|
||||||
|
TriggerType: triggerType,
|
||||||
|
}
|
||||||
|
result := tx.Clauses(clause.OnConflict{DoNothing: true}).Create(target)
|
||||||
|
if result.Error != nil {
|
||||||
|
return nil, false, result.Error
|
||||||
|
}
|
||||||
|
if result.RowsAffected == 0 {
|
||||||
|
return reloadSourceDeploymentTx(tx, state.Project.ID, identity, revision)
|
||||||
|
}
|
||||||
|
if err := createSourceDeploymentFilesTx(tx, target.ID, manifest.Files); err != nil {
|
||||||
|
return nil, false, err
|
||||||
|
}
|
||||||
|
return target, false, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func validSourceDeploymentTrigger(triggerType string) bool {
|
||||||
|
return triggerType == pagesSourceTriggerManualSync || triggerType == pagesSourceTriggerScheduledAutoUpdate
|
||||||
|
}
|
||||||
|
|
||||||
|
func reloadSourceDeploymentTx(
|
||||||
|
tx *gorm.DB,
|
||||||
|
projectID uint,
|
||||||
|
identity string,
|
||||||
|
revision string,
|
||||||
|
) (*model.PagesDeployment, bool, error) {
|
||||||
|
var target model.PagesDeployment
|
||||||
|
err := tx.Where(
|
||||||
|
"project_id = ? AND source_identity = ? AND source_revision = ?",
|
||||||
|
projectID,
|
||||||
|
identity,
|
||||||
|
revision,
|
||||||
|
).First(&target).Error
|
||||||
|
return &target, true, err
|
||||||
|
}
|
||||||
|
|
||||||
|
func createSourceDeploymentFilesTx(tx *gorm.DB, deploymentID uint, files []model.PagesDeploymentFile) error {
|
||||||
|
if len(files) == 0 {
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
for index := range files {
|
||||||
|
files[index].DeploymentID = deploymentID
|
||||||
|
}
|
||||||
|
return tx.Create(&files).Error
|
||||||
|
}
|
||||||
|
|
||||||
|
func lockSourceDeploymentUploadsTx(
|
||||||
|
tx *gorm.DB,
|
||||||
|
target *model.PagesDeployment,
|
||||||
|
ingestResult upload.IngestResult,
|
||||||
|
hasIngest bool,
|
||||||
|
) error {
|
||||||
|
uploadIDs := []uint64{target.UploadID}
|
||||||
|
if hasIngest && ingestResult.Upload.ID != 0 && ingestResult.Upload.ID != target.UploadID {
|
||||||
|
uploadIDs = append(uploadIDs, ingestResult.Upload.ID)
|
||||||
|
}
|
||||||
|
sort.Slice(uploadIDs, func(i, j int) bool { return uploadIDs[i] < uploadIDs[j] })
|
||||||
|
var records []model.Upload
|
||||||
|
if err := tx.Clauses(clause.Locking{Strength: pagesRowLockStrength}).
|
||||||
|
Where("id IN ?", uploadIDs).
|
||||||
|
Order("id asc").
|
||||||
|
Find(&records).Error; err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
for index := range records {
|
||||||
|
if records[index].ID != target.UploadID {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
if records[index].Status == model.UploadStatusUsed && records[index].Type == upload.ReservedPagesDeploymentType {
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
break
|
||||||
|
}
|
||||||
|
return errSourceFinalFence
|
||||||
|
}
|
||||||
|
|
||||||
|
func activateSourceDeploymentTx(
|
||||||
|
tx *gorm.DB,
|
||||||
|
state *sourceCommitState,
|
||||||
|
target *model.PagesDeployment,
|
||||||
|
revision string,
|
||||||
|
detailJSON string,
|
||||||
|
nextCheckNotBefore *time.Time,
|
||||||
|
) error {
|
||||||
|
if err := tx.Model(&model.PagesDeployment{}).
|
||||||
|
Where("project_id = ?", state.Project.ID).
|
||||||
|
Update("status", model.PagesDeploymentStatusUploaded).Error; err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
if err := tx.Model(target).Updates(map[string]any{
|
||||||
|
pagesDeploymentColumnStatus: model.PagesDeploymentStatusActive,
|
||||||
|
"activated_at": &state.Now,
|
||||||
|
}).Error; err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
if err := tx.Model(state.Project).Update("active_deployment_id", target.ID).Error; err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
finishedAt := sourceCommitNow()
|
||||||
|
var nextCheckAt any
|
||||||
|
if state.Source.SourceType == PagesSourceTypeGitHubRelease &&
|
||||||
|
state.Source.ReleaseSelector == githubReleaseSelectorLatest {
|
||||||
|
next := nextGitHubCheckAt(finishedAt, state.Source.ID, state.Source.CheckIntervalMinutes)
|
||||||
|
if nextCheckNotBefore != nil && nextCheckNotBefore.After(next) {
|
||||||
|
next = nextCheckNotBefore.In(finishedAt.Location())
|
||||||
|
}
|
||||||
|
nextCheckAt = &next
|
||||||
|
}
|
||||||
|
result := tx.Model(&model.PagesProjectSourceRuntime{}).
|
||||||
|
Where("source_id = ? AND lease_token = ? AND lease_expires_at > ?",
|
||||||
|
state.Runtime.SourceID,
|
||||||
|
state.Runtime.LeaseToken,
|
||||||
|
finishedAt,
|
||||||
|
).
|
||||||
|
Updates(map[string]any{
|
||||||
|
"last_seen_revision": revision,
|
||||||
|
"last_seen_detail": detailJSON,
|
||||||
|
"last_applied_revision": revision,
|
||||||
|
"last_applied_detail": detailJSON,
|
||||||
|
sourceRuntimeColumnSyncStatus: pagesSourceStatusIdle,
|
||||||
|
sourceRuntimeColumnLastError: "",
|
||||||
|
sourceRuntimeColumnLastCheckedAt: &finishedAt,
|
||||||
|
"last_synced_at": &finishedAt,
|
||||||
|
"next_check_at": nextCheckAt,
|
||||||
|
sourceRuntimeColumnLeaseToken: "",
|
||||||
|
sourceRuntimeColumnLeaseExpiresAt: nil,
|
||||||
|
})
|
||||||
|
if result.Error != nil {
|
||||||
|
return result.Error
|
||||||
|
}
|
||||||
|
if result.RowsAffected != 1 {
|
||||||
|
return errSourceFinalFence
|
||||||
|
}
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func safeRemoteSourceLabel(raw string) string {
|
||||||
|
value := strings.ReplaceAll(strings.ToValidUTF8(raw, ""), "\\", "/")
|
||||||
|
value = path.Base(strings.TrimSpace(value))
|
||||||
|
if value == "." || value == "/" {
|
||||||
|
value = ""
|
||||||
|
}
|
||||||
|
var builder strings.Builder
|
||||||
|
for _, character := range value {
|
||||||
|
if character >= 0x20 && character != 0x7f {
|
||||||
|
builder.WriteRune(character)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
value = strings.TrimSpace(builder.String())
|
||||||
|
if value == "" {
|
||||||
|
value = defaultRemoteAssetLabel
|
||||||
|
}
|
||||||
|
if len(value) > remoteSourceMaxSafeLabelBytes {
|
||||||
|
value = value[:remoteSourceMaxSafeLabelBytes]
|
||||||
|
for !utf8.ValidString(value) {
|
||||||
|
_, size := utf8.DecodeLastRuneInString(value)
|
||||||
|
value = value[:len(value)-size]
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return value
|
||||||
|
}
|
||||||
|
|
||||||
|
func sourceSyncResultDetail(outcome *sourceSyncOutcome) string {
|
||||||
|
if outcome == nil || outcome.Deployment == nil {
|
||||||
|
return ""
|
||||||
|
}
|
||||||
|
payload := map[string]any{
|
||||||
|
"deployment_id": outcome.Deployment.ID,
|
||||||
|
"reused": outcome.Reused,
|
||||||
|
}
|
||||||
|
encoded, err := json.Marshal(payload)
|
||||||
|
if err != nil {
|
||||||
|
return fmt.Sprintf(`{"deployment_id":%d}`, outcome.Deployment.ID)
|
||||||
|
}
|
||||||
|
return string(encoded)
|
||||||
|
}
|
||||||
@@ -0,0 +1,608 @@
|
|||||||
|
// Copyright 2026 Arctel.net
|
||||||
|
// SPDX-License-Identifier: Apache-2.0
|
||||||
|
|
||||||
|
package pages
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
"crypto/sha256"
|
||||||
|
"encoding/hex"
|
||||||
|
"errors"
|
||||||
|
"fmt"
|
||||||
|
"net/http"
|
||||||
|
"net/http/httptest"
|
||||||
|
"os"
|
||||||
|
"path/filepath"
|
||||||
|
"strings"
|
||||||
|
"testing"
|
||||||
|
"time"
|
||||||
|
|
||||||
|
"github.com/Rain-kl/Wavelet/internal/apps/upload"
|
||||||
|
"github.com/Rain-kl/Wavelet/internal/db"
|
||||||
|
"github.com/Rain-kl/Wavelet/internal/model"
|
||||||
|
"github.com/Rain-kl/Wavelet/pkg/pagesarchive"
|
||||||
|
"gorm.io/gorm"
|
||||||
|
)
|
||||||
|
|
||||||
|
func setupPagesSourceSyncTest(t *testing.T) context.Context {
|
||||||
|
t.Helper()
|
||||||
|
ctx := setupPagesSourceTest(t)
|
||||||
|
_, disableStorage := setupPagesStorageMock(t)
|
||||||
|
t.Cleanup(disableStorage)
|
||||||
|
return ctx
|
||||||
|
}
|
||||||
|
|
||||||
|
func mustAcquireRemoteSyncLease(
|
||||||
|
t *testing.T,
|
||||||
|
ctx context.Context,
|
||||||
|
source *model.PagesProjectSource,
|
||||||
|
) *sourceExecutionSnapshot {
|
||||||
|
t.Helper()
|
||||||
|
snapshot, outcome, err := acquireSourceLease(ctx, source.ID, source.ConfigVersion, sourceActionSync)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("acquireSourceLease(source=%d) error = %v, want nil", source.ID, err)
|
||||||
|
}
|
||||||
|
if got, want := outcome, sourceLeaseAcquired; got != want {
|
||||||
|
t.Fatalf("acquireSourceLease(source=%d) outcome = %q, want %q", source.ID, got, want)
|
||||||
|
}
|
||||||
|
if snapshot == nil {
|
||||||
|
t.Fatalf("acquireSourceLease(source=%d) snapshot = nil, want non-nil", source.ID)
|
||||||
|
}
|
||||||
|
return snapshot
|
||||||
|
}
|
||||||
|
|
||||||
|
func mustCreateActiveManualDeployment(
|
||||||
|
t *testing.T,
|
||||||
|
ctx context.Context,
|
||||||
|
projectID uint,
|
||||||
|
content string,
|
||||||
|
) *model.PagesDeployment {
|
||||||
|
t.Helper()
|
||||||
|
view, err := UploadDeployment(
|
||||||
|
ctx,
|
||||||
|
projectID,
|
||||||
|
testPagesMultipartFile(t, "manual.zip", testPagesZip(t, map[string]string{"index.html": content})),
|
||||||
|
"user:1",
|
||||||
|
)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("UploadDeployment(project=%d) error = %v, want nil", projectID, err)
|
||||||
|
}
|
||||||
|
if _, err := ActivateDeploymentAs(ctx, projectID, view.ID, "user:1"); err != nil {
|
||||||
|
t.Fatalf("ActivateDeploymentAs(project=%d, deployment=%d) error = %v, want nil", projectID, view.ID, err)
|
||||||
|
}
|
||||||
|
deployment, err := model.GetPagesDeploymentByID(ctx, view.ID)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("GetPagesDeploymentByID(%d) error = %v, want nil", view.ID, err)
|
||||||
|
}
|
||||||
|
return deployment
|
||||||
|
}
|
||||||
|
|
||||||
|
func newPagesArchiveServer(t *testing.T, status int, body []byte, beforeWrite func() error) *httptest.Server {
|
||||||
|
t.Helper()
|
||||||
|
server := httptest.NewServer(http.HandlerFunc(func(writer http.ResponseWriter, _ *http.Request) {
|
||||||
|
if beforeWrite != nil {
|
||||||
|
if err := beforeWrite(); err != nil {
|
||||||
|
writer.WriteHeader(http.StatusInternalServerError)
|
||||||
|
return
|
||||||
|
}
|
||||||
|
}
|
||||||
|
writer.Header().Set("Content-Type", "application/zip")
|
||||||
|
writer.Header().Set("Content-Disposition", `attachment; filename="site.zip"`)
|
||||||
|
writer.WriteHeader(status)
|
||||||
|
_, _ = writer.Write(body)
|
||||||
|
}))
|
||||||
|
t.Cleanup(server.Close)
|
||||||
|
return server
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestSyncRemoteSourceAtomicallyActivatesAndReusesChecksum(t *testing.T) {
|
||||||
|
ctx := setupPagesSourceSyncTest(t)
|
||||||
|
project := mustCreatePagesSourceProject(t, ctx, "sync-success")
|
||||||
|
packageBytes := testPagesZip(t, map[string]string{
|
||||||
|
"index.html": "remote-v1",
|
||||||
|
"assets/app.js": "console.log('ok')",
|
||||||
|
})
|
||||||
|
server := newPagesArchiveServer(t, http.StatusOK, packageBytes, nil)
|
||||||
|
secret := "sync-query-secret"
|
||||||
|
source, _ := mustConfigureRemoteSource(
|
||||||
|
t,
|
||||||
|
ctx,
|
||||||
|
project.ID,
|
||||||
|
server.URL+"/site.zip?token="+secret,
|
||||||
|
RemoteNetworkPolicyTrustedInternal,
|
||||||
|
)
|
||||||
|
|
||||||
|
firstSnapshot := mustAcquireRemoteSyncLease(t, ctx, source)
|
||||||
|
first, err := syncRemoteSource(ctx, firstSnapshot, "user:42")
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("syncRemoteSource(first) error = %v, want nil", err)
|
||||||
|
}
|
||||||
|
if first == nil || first.Stale || first.Reused || first.Deployment == nil {
|
||||||
|
t.Fatalf("syncRemoteSource(first) = %+v, want new active deployment", first)
|
||||||
|
}
|
||||||
|
storedProject, err := model.GetPagesProjectByID(ctx, project.ID)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("GetPagesProjectByID(%d) error = %v, want nil", project.ID, err)
|
||||||
|
}
|
||||||
|
if storedProject.ActiveDeploymentID == nil || *storedProject.ActiveDeploymentID != first.Deployment.ID {
|
||||||
|
t.Fatalf("project ActiveDeploymentID = %v, want %d", storedProject.ActiveDeploymentID, first.Deployment.ID)
|
||||||
|
}
|
||||||
|
deployment, err := model.GetPagesDeploymentByID(ctx, first.Deployment.ID)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("GetPagesDeploymentByID(%d) error = %v, want nil", first.Deployment.ID, err)
|
||||||
|
}
|
||||||
|
expectedHash := sha256.Sum256(packageBytes)
|
||||||
|
if got, want := deployment.Checksum, hex.EncodeToString(expectedHash[:]); got != want {
|
||||||
|
t.Errorf("deployment Checksum = %q, want %q", got, want)
|
||||||
|
}
|
||||||
|
if got, want := deployment.Status, model.PagesDeploymentStatusActive; got != want {
|
||||||
|
t.Errorf("deployment Status = %q, want %q", got, want)
|
||||||
|
}
|
||||||
|
if got, want := deployment.SourceType, PagesSourceTypeRemoteURL; got != want {
|
||||||
|
t.Errorf("deployment SourceType = %q, want %q", got, want)
|
||||||
|
}
|
||||||
|
if deployment.SourceIdentity == nil || *deployment.SourceIdentity != source.SourceIdentity {
|
||||||
|
t.Errorf("deployment SourceIdentity = %v, want %q", deployment.SourceIdentity, source.SourceIdentity)
|
||||||
|
}
|
||||||
|
if deployment.SourceRevision == nil || *deployment.SourceRevision != deployment.Checksum {
|
||||||
|
t.Errorf("deployment SourceRevision = %v, want %q", deployment.SourceRevision, deployment.Checksum)
|
||||||
|
}
|
||||||
|
if got, want := deployment.CreatedBy, "user:42"; got != want {
|
||||||
|
t.Errorf("deployment CreatedBy = %q, want %q", got, want)
|
||||||
|
}
|
||||||
|
if got, want := deployment.TriggerType, pagesSourceTriggerManualSync; got != want {
|
||||||
|
t.Errorf("deployment TriggerType = %q, want %q", got, want)
|
||||||
|
}
|
||||||
|
if strings.Contains(deployment.SourceMeta, secret) || strings.Contains(deployment.SourceLabel, secret) {
|
||||||
|
t.Errorf("deployment provenance = label:%q meta:%q, want no query secret", deployment.SourceLabel, deployment.SourceMeta)
|
||||||
|
}
|
||||||
|
var runtime model.PagesProjectSourceRuntime
|
||||||
|
if err := db.DB(ctx).Where("source_id = ?", source.ID).First(&runtime).Error; err != nil {
|
||||||
|
t.Fatalf("load source runtime error = %v, want nil", err)
|
||||||
|
}
|
||||||
|
if got, want := runtime.SyncStatus, pagesSourceStatusIdle; got != want {
|
||||||
|
t.Errorf("runtime SyncStatus = %q, want %q", got, want)
|
||||||
|
}
|
||||||
|
if got, want := runtime.LastAppliedRevision, deployment.Checksum; got != want {
|
||||||
|
t.Errorf("runtime LastAppliedRevision = %q, want %q", got, want)
|
||||||
|
}
|
||||||
|
if runtime.LeaseToken != "" || runtime.LeaseExpiresAt != nil {
|
||||||
|
t.Errorf("runtime lease = (%q, %v), want cleared", runtime.LeaseToken, runtime.LeaseExpiresAt)
|
||||||
|
}
|
||||||
|
var uploadRecord model.Upload
|
||||||
|
if err := db.DB(ctx).First(&uploadRecord, deployment.UploadID).Error; err != nil {
|
||||||
|
t.Fatalf("load deployment upload %d error = %v, want nil", deployment.UploadID, err)
|
||||||
|
}
|
||||||
|
if got, want := uploadRecord.Status, model.UploadStatusUsed; got != want {
|
||||||
|
t.Errorf("deployment upload Status = %q, want %q", got, want)
|
||||||
|
}
|
||||||
|
if got, want := uploadRecord.Type, upload.ReservedPagesDeploymentType; got != want {
|
||||||
|
t.Errorf("deployment upload Type = %q, want %q", got, want)
|
||||||
|
}
|
||||||
|
if got, want := fmt.Sprint(uploadRecord.Metadata.Extra[pagesSourceIDMetadataKey]), fmt.Sprint(source.ID); got != want {
|
||||||
|
t.Errorf("deployment upload pages_source_id = %q, want %q", got, want)
|
||||||
|
}
|
||||||
|
|
||||||
|
secondSnapshot := mustAcquireRemoteSyncLease(t, ctx, source)
|
||||||
|
second, err := syncRemoteSource(ctx, secondSnapshot, "user:42")
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("syncRemoteSource(second) error = %v, want nil", err)
|
||||||
|
}
|
||||||
|
if second == nil || second.Stale || !second.Reused || second.Deployment == nil {
|
||||||
|
t.Fatalf("syncRemoteSource(second) = %+v, want reused active deployment", second)
|
||||||
|
}
|
||||||
|
if got, want := second.Deployment.ID, first.Deployment.ID; got != want {
|
||||||
|
t.Errorf("reused deployment ID = %d, want %d", got, want)
|
||||||
|
}
|
||||||
|
var deploymentCount, uploadCount int64
|
||||||
|
if err := db.DB(ctx).Model(&model.PagesDeployment{}).Where("project_id = ?", project.ID).Count(&deploymentCount).Error; err != nil {
|
||||||
|
t.Fatalf("count source deployments error = %v, want nil", err)
|
||||||
|
}
|
||||||
|
if err := db.DB(ctx).Model(&model.Upload{}).Count(&uploadCount).Error; err != nil {
|
||||||
|
t.Fatalf("count source uploads error = %v, want nil", err)
|
||||||
|
}
|
||||||
|
if got, want := deploymentCount, int64(1); got != want {
|
||||||
|
t.Errorf("deployment count after identical sync = %d, want %d", got, want)
|
||||||
|
}
|
||||||
|
if got, want := uploadCount, int64(1); got != want {
|
||||||
|
t.Errorf("upload count after identical sync = %d, want %d", got, want)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestSyncRemoteSourceDownloadFailureKeepsOldActive(t *testing.T) {
|
||||||
|
ctx := setupPagesSourceSyncTest(t)
|
||||||
|
project := mustCreatePagesSourceProject(t, ctx, "sync-download-fail")
|
||||||
|
oldActive := mustCreateActiveManualDeployment(t, ctx, project.ID, "old-active")
|
||||||
|
server := newPagesArchiveServer(t, http.StatusBadGateway, []byte("upstream failed"), nil)
|
||||||
|
secret := "download-failure-secret"
|
||||||
|
source, _ := mustConfigureRemoteSource(
|
||||||
|
t,
|
||||||
|
ctx,
|
||||||
|
project.ID,
|
||||||
|
server.URL+"/site.zip?token="+secret,
|
||||||
|
RemoteNetworkPolicyTrustedInternal,
|
||||||
|
)
|
||||||
|
|
||||||
|
_, err := syncRemoteSource(ctx, mustAcquireRemoteSyncLease(t, ctx, source), "user:2")
|
||||||
|
if err == nil {
|
||||||
|
t.Fatal("syncRemoteSource(download failure) error = nil, want non-nil")
|
||||||
|
}
|
||||||
|
if strings.Contains(err.Error(), secret) {
|
||||||
|
t.Errorf("syncRemoteSource(download failure) error = %q, want no query secret", err)
|
||||||
|
}
|
||||||
|
assertPagesSyncFailureState(t, ctx, project.ID, source.ID, oldActive.ID, 1)
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestSyncRemoteSourceArchiveFailureKeepsOldActive(t *testing.T) {
|
||||||
|
ctx := setupPagesSourceSyncTest(t)
|
||||||
|
project := mustCreatePagesSourceProject(t, ctx, "sync-archive-fail")
|
||||||
|
oldActive := mustCreateActiveManualDeployment(t, ctx, project.ID, "old-active")
|
||||||
|
server := newPagesArchiveServer(t, http.StatusOK, []byte("not-a-valid-zip"), nil)
|
||||||
|
secret := "archive-failure-secret"
|
||||||
|
source, _ := mustConfigureRemoteSource(
|
||||||
|
t,
|
||||||
|
ctx,
|
||||||
|
project.ID,
|
||||||
|
server.URL+"/site.zip?token="+secret,
|
||||||
|
RemoteNetworkPolicyTrustedInternal,
|
||||||
|
)
|
||||||
|
|
||||||
|
_, err := syncRemoteSource(ctx, mustAcquireRemoteSyncLease(t, ctx, source), "user:3")
|
||||||
|
if err == nil {
|
||||||
|
t.Fatal("syncRemoteSource(archive failure) error = nil, want non-nil")
|
||||||
|
}
|
||||||
|
if strings.Contains(err.Error(), secret) {
|
||||||
|
t.Errorf("syncRemoteSource(archive failure) error = %q, want no query secret", err)
|
||||||
|
}
|
||||||
|
assertPagesSyncFailureState(t, ctx, project.ID, source.ID, oldActive.ID, 1)
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestSyncRemoteSourceFinalFenceCompensatesIngest(t *testing.T) {
|
||||||
|
ctx := setupPagesSourceSyncTest(t)
|
||||||
|
project := mustCreatePagesSourceProject(t, ctx, "sync-final-fence")
|
||||||
|
oldActive := mustCreateActiveManualDeployment(t, ctx, project.ID, "old-active")
|
||||||
|
packageBytes := testPagesZip(t, map[string]string{"index.html": "never-activate"})
|
||||||
|
mutationResult := make(chan error, 1)
|
||||||
|
server := newPagesArchiveServer(t, http.StatusOK, packageBytes, func() error {
|
||||||
|
err := db.DB(context.Background()).Model(&model.PagesProject{}).
|
||||||
|
Where("id = ?", project.ID).
|
||||||
|
Update("content_config_version", gorm.Expr("content_config_version + 1")).Error
|
||||||
|
mutationResult <- err
|
||||||
|
return err
|
||||||
|
})
|
||||||
|
source, _ := mustConfigureRemoteSource(
|
||||||
|
t,
|
||||||
|
ctx,
|
||||||
|
project.ID,
|
||||||
|
server.URL+"/site.zip?token=final-fence-secret",
|
||||||
|
RemoteNetworkPolicyTrustedInternal,
|
||||||
|
)
|
||||||
|
|
||||||
|
outcome, err := syncRemoteSource(ctx, mustAcquireRemoteSyncLease(t, ctx, source), "user:4")
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("syncRemoteSource(final fence) error = %v, want nil stale outcome", err)
|
||||||
|
}
|
||||||
|
select {
|
||||||
|
case mutationErr := <-mutationResult:
|
||||||
|
if mutationErr != nil {
|
||||||
|
t.Fatalf("content version mutation error = %v, want nil", mutationErr)
|
||||||
|
}
|
||||||
|
case <-time.After(2 * time.Second):
|
||||||
|
t.Fatal("content version mutation was not observed")
|
||||||
|
}
|
||||||
|
if outcome == nil || !outcome.Stale || outcome.Deployment != nil {
|
||||||
|
t.Fatalf("syncRemoteSource(final fence) = %+v, want stale outcome without deployment", outcome)
|
||||||
|
}
|
||||||
|
storedProject, err := model.GetPagesProjectByID(ctx, project.ID)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("GetPagesProjectByID(%d) error = %v, want nil", project.ID, err)
|
||||||
|
}
|
||||||
|
if storedProject.ActiveDeploymentID == nil || *storedProject.ActiveDeploymentID != oldActive.ID {
|
||||||
|
t.Errorf("ActiveDeploymentID after final fence = %v, want %d", storedProject.ActiveDeploymentID, oldActive.ID)
|
||||||
|
}
|
||||||
|
var deployments []model.PagesDeployment
|
||||||
|
if err := db.DB(ctx).Where("project_id = ?", project.ID).Find(&deployments).Error; err != nil {
|
||||||
|
t.Fatalf("list deployments after final fence error = %v, want nil", err)
|
||||||
|
}
|
||||||
|
if got, want := len(deployments), 1; got != want {
|
||||||
|
t.Errorf("deployment count after final fence = %d, want %d", got, want)
|
||||||
|
}
|
||||||
|
var uploads []model.Upload
|
||||||
|
if err := db.DB(ctx).Order("id asc").Find(&uploads).Error; err != nil {
|
||||||
|
t.Fatalf("list uploads after final fence error = %v, want nil", err)
|
||||||
|
}
|
||||||
|
var compensated *model.Upload
|
||||||
|
for index := range uploads {
|
||||||
|
if fmt.Sprint(uploads[index].Metadata.Extra[pagesSourceIDMetadataKey]) == fmt.Sprint(source.ID) {
|
||||||
|
compensated = &uploads[index]
|
||||||
|
break
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if compensated == nil {
|
||||||
|
t.Fatalf("source upload after final fence = nil, want compensated upload record")
|
||||||
|
}
|
||||||
|
if got, want := compensated.Status, model.UploadStatusDeleted; got != want {
|
||||||
|
t.Errorf("compensated upload Status = %q, want %q", got, want)
|
||||||
|
}
|
||||||
|
var danglingCount int64
|
||||||
|
if err := db.DB(ctx).Model(&model.PagesDeployment{}).
|
||||||
|
Where("upload_id = ?", compensated.ID).
|
||||||
|
Count(&danglingCount).Error; err != nil {
|
||||||
|
t.Fatalf("count compensated upload references error = %v, want nil", err)
|
||||||
|
}
|
||||||
|
if got, want := danglingCount, int64(0); got != want {
|
||||||
|
t.Errorf("deployments referencing compensated upload = %d, want %d", got, want)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestCommitSourceDeploymentRechecksLeaseAfterUploadLocks(t *testing.T) {
|
||||||
|
ctx := setupPagesSourceSyncTest(t)
|
||||||
|
project := mustCreatePagesSourceProject(t, ctx, "sync-expiry-recheck")
|
||||||
|
packageBytes := testPagesZip(t, map[string]string{"index.html": "expiry-recheck"})
|
||||||
|
server := newPagesArchiveServer(t, http.StatusOK, packageBytes, nil)
|
||||||
|
source, _ := mustConfigureRemoteSource(
|
||||||
|
t,
|
||||||
|
ctx,
|
||||||
|
project.ID,
|
||||||
|
server.URL+"/site.zip",
|
||||||
|
RemoteNetworkPolicyTrustedInternal,
|
||||||
|
)
|
||||||
|
first, err := syncRemoteSource(ctx, mustAcquireRemoteSyncLease(t, ctx, source), "user:5")
|
||||||
|
if err != nil || first == nil || first.Deployment == nil {
|
||||||
|
t.Fatalf("syncRemoteSource(seed) = (%+v, %v), want deployment", first, err)
|
||||||
|
}
|
||||||
|
deployment, err := model.GetPagesDeploymentByID(ctx, first.Deployment.ID)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("GetPagesDeploymentByID(%d) error = %v, want nil", first.Deployment.ID, err)
|
||||||
|
}
|
||||||
|
if err := db.DB(ctx).Model(&model.PagesProject{}).
|
||||||
|
Where("id = ?", project.ID).
|
||||||
|
Update("active_deployment_id", nil).Error; err != nil {
|
||||||
|
t.Fatalf("clear active deployment error = %v, want nil", err)
|
||||||
|
}
|
||||||
|
if err := db.DB(ctx).Model(&model.PagesDeployment{}).
|
||||||
|
Where("id = ?", deployment.ID).
|
||||||
|
Update("status", model.PagesDeploymentStatusUploaded).Error; err != nil {
|
||||||
|
t.Fatalf("reset deployment status error = %v, want nil", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
snapshot := mustAcquireRemoteSyncLease(t, ctx, source)
|
||||||
|
expiresAt := time.Now().Add(time.Hour)
|
||||||
|
if err := db.DB(ctx).Model(&model.PagesProjectSourceRuntime{}).
|
||||||
|
Where("source_id = ? AND lease_token = ?", source.ID, snapshot.LeaseToken).
|
||||||
|
Update("lease_expires_at", &expiresAt).Error; err != nil {
|
||||||
|
t.Fatalf("set deterministic lease expiry error = %v, want nil", err)
|
||||||
|
}
|
||||||
|
originalNow := sourceCommitNow
|
||||||
|
nowCalls := 0
|
||||||
|
sourceCommitNow = func() time.Time {
|
||||||
|
nowCalls++
|
||||||
|
if nowCalls == 1 {
|
||||||
|
return expiresAt.Add(-time.Second)
|
||||||
|
}
|
||||||
|
return expiresAt.Add(time.Second)
|
||||||
|
}
|
||||||
|
t.Cleanup(func() { sourceCommitNow = originalNow })
|
||||||
|
|
||||||
|
_, _, _, err = commitSourceDeployment(
|
||||||
|
ctx,
|
||||||
|
snapshot,
|
||||||
|
deployment.Checksum,
|
||||||
|
deployment.Checksum,
|
||||||
|
sourceDetail{Provider: PagesSourceTypeRemoteURL, DisplayName: deployment.SourceLabel},
|
||||||
|
deployment.SourceMeta,
|
||||||
|
"user:5",
|
||||||
|
&deploymentManifest{},
|
||||||
|
upload.IngestResult{},
|
||||||
|
false,
|
||||||
|
nil,
|
||||||
|
)
|
||||||
|
if !errors.Is(err, errSourceFinalFence) {
|
||||||
|
t.Fatalf("commitSourceDeployment(expired after upload lock) error = %v, want %v", err, errSourceFinalFence)
|
||||||
|
}
|
||||||
|
if nowCalls != 2 {
|
||||||
|
t.Fatalf("sourceCommitNow calls = %d, want runtime-lock and post-upload checks", nowCalls)
|
||||||
|
}
|
||||||
|
storedProject, err := model.GetPagesProjectByID(ctx, project.ID)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("GetPagesProjectByID(%d) error = %v, want nil", project.ID, err)
|
||||||
|
}
|
||||||
|
if storedProject.ActiveDeploymentID != nil {
|
||||||
|
t.Errorf("ActiveDeploymentID after expiry recheck = %v, want nil", storedProject.ActiveDeploymentID)
|
||||||
|
}
|
||||||
|
storedDeployment, err := model.GetPagesDeploymentByID(ctx, deployment.ID)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("GetPagesDeploymentByID(%d) after expiry error = %v, want nil", deployment.ID, err)
|
||||||
|
}
|
||||||
|
if got, want := storedDeployment.Status, model.PagesDeploymentStatusUploaded; got != want {
|
||||||
|
t.Errorf("deployment status after expiry recheck = %q, want %q", got, want)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestCompensateSourceIngestSurvivesCanceledParentContext(t *testing.T) {
|
||||||
|
ctx := setupPagesSourceSyncTest(t)
|
||||||
|
project := mustCreatePagesSourceProject(t, ctx, "sync-canceled-compensation")
|
||||||
|
source, _ := mustConfigureRemoteSource(
|
||||||
|
t,
|
||||||
|
ctx,
|
||||||
|
project.ID,
|
||||||
|
"https://example.com/site.zip",
|
||||||
|
RemoteNetworkPolicyPublic,
|
||||||
|
)
|
||||||
|
packageBytes := testPagesZip(t, map[string]string{"index.html": "cancel-compensation"})
|
||||||
|
packagePath := filepath.Join(t.TempDir(), "site.zip")
|
||||||
|
if err := os.WriteFile(packagePath, packageBytes, 0o600); err != nil {
|
||||||
|
t.Fatalf("write test package error = %v, want nil", err)
|
||||||
|
}
|
||||||
|
digest := sha256.Sum256(packageBytes)
|
||||||
|
result, err := ingestPagesDeploymentPackageWithSource(
|
||||||
|
ctx,
|
||||||
|
packagePath,
|
||||||
|
hex.EncodeToString(digest[:]),
|
||||||
|
project.ID,
|
||||||
|
source.ID,
|
||||||
|
"site.zip",
|
||||||
|
pagesarchive.FormatZip,
|
||||||
|
)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("ingestPagesDeploymentPackageWithSource() error = %v, want nil", err)
|
||||||
|
}
|
||||||
|
if !result.Created {
|
||||||
|
t.Fatal("ingest result Created = false, want a compensatable record")
|
||||||
|
}
|
||||||
|
|
||||||
|
canceledCtx, cancel := context.WithCancel(ctx)
|
||||||
|
cancel()
|
||||||
|
compensateSourceIngest(canceledCtx, &sourceExecutionSnapshot{
|
||||||
|
ProjectID: project.ID,
|
||||||
|
SourceID: source.ID,
|
||||||
|
}, &sourceIngestState{
|
||||||
|
Result: result,
|
||||||
|
HasIngest: true,
|
||||||
|
})
|
||||||
|
|
||||||
|
var uploadRecord model.Upload
|
||||||
|
if err := db.DB(ctx).Where("id = ?", result.Upload.ID).First(&uploadRecord).Error; err != nil {
|
||||||
|
t.Fatalf("load compensated upload error = %v, want nil", err)
|
||||||
|
}
|
||||||
|
if got, want := uploadRecord.Status, model.UploadStatusDeleted; got != want {
|
||||||
|
t.Errorf("compensated upload status = %q, want %q", got, want)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func assertPagesSyncFailureState(
|
||||||
|
t *testing.T,
|
||||||
|
ctx context.Context,
|
||||||
|
projectID uint,
|
||||||
|
sourceID uint,
|
||||||
|
oldActiveID uint,
|
||||||
|
wantDeploymentCount int64,
|
||||||
|
) {
|
||||||
|
t.Helper()
|
||||||
|
project, err := model.GetPagesProjectByID(ctx, projectID)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("GetPagesProjectByID(%d) error = %v, want nil", projectID, err)
|
||||||
|
}
|
||||||
|
if project.ActiveDeploymentID == nil || *project.ActiveDeploymentID != oldActiveID {
|
||||||
|
t.Errorf("project %d ActiveDeploymentID = %v, want %d", projectID, project.ActiveDeploymentID, oldActiveID)
|
||||||
|
}
|
||||||
|
var deploymentCount int64
|
||||||
|
if err := db.DB(ctx).Model(&model.PagesDeployment{}).
|
||||||
|
Where("project_id = ?", projectID).
|
||||||
|
Count(&deploymentCount).Error; err != nil {
|
||||||
|
t.Fatalf("count project %d deployments error = %v, want nil", projectID, err)
|
||||||
|
}
|
||||||
|
if got, want := deploymentCount, wantDeploymentCount; got != want {
|
||||||
|
t.Errorf("project %d deployment count = %d, want %d", projectID, got, want)
|
||||||
|
}
|
||||||
|
var runtime model.PagesProjectSourceRuntime
|
||||||
|
if err := db.DB(ctx).Where("source_id = ?", sourceID).First(&runtime).Error; err != nil {
|
||||||
|
t.Fatalf("load source %d runtime error = %v, want nil", sourceID, err)
|
||||||
|
}
|
||||||
|
if got, want := runtime.SyncStatus, pagesSourceStatusFailed; got != want {
|
||||||
|
t.Errorf("source %d runtime SyncStatus = %q, want %q", sourceID, got, want)
|
||||||
|
}
|
||||||
|
if runtime.LeaseToken != "" || runtime.LeaseExpiresAt != nil {
|
||||||
|
t.Errorf("source %d runtime lease = (%q, %v), want cleared", sourceID, runtime.LeaseToken, runtime.LeaseExpiresAt)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestCommitSourceDeploymentRejectsDeletedTargetUpload(t *testing.T) {
|
||||||
|
ctx := setupPagesSourceSyncTest(t)
|
||||||
|
project := mustCreatePagesSourceProject(t, ctx, "sync-deleted-upload")
|
||||||
|
packageBytes := testPagesZip(t, map[string]string{"index.html": "content"})
|
||||||
|
server := newPagesArchiveServer(t, http.StatusOK, packageBytes, nil)
|
||||||
|
source, _ := mustConfigureRemoteSource(
|
||||||
|
t,
|
||||||
|
ctx,
|
||||||
|
project.ID,
|
||||||
|
server.URL+"/site.zip",
|
||||||
|
RemoteNetworkPolicyTrustedInternal,
|
||||||
|
)
|
||||||
|
snapshot := mustAcquireRemoteSyncLease(t, ctx, source)
|
||||||
|
|
||||||
|
// A pre-existing source deployment whose upload was removed must never be
|
||||||
|
// reactivated into a dangling active pointer.
|
||||||
|
identity := source.SourceIdentity
|
||||||
|
revision := strings.Repeat("d", 64)
|
||||||
|
uploadRecord := &model.Upload{
|
||||||
|
ID: 987654321,
|
||||||
|
UserID: 999,
|
||||||
|
FileName: "deleted.zip",
|
||||||
|
FilePath: "deleted.zip",
|
||||||
|
FileSize: 1,
|
||||||
|
MimeType: "application/zip",
|
||||||
|
Extension: "zip",
|
||||||
|
Hash: revision,
|
||||||
|
Type: upload.ReservedPagesDeploymentType,
|
||||||
|
Status: model.UploadStatusDeleted,
|
||||||
|
AccessMode: 0,
|
||||||
|
}
|
||||||
|
if err := db.DB(ctx).Create(uploadRecord).Error; err != nil {
|
||||||
|
t.Fatalf("create deleted upload error = %v, want nil", err)
|
||||||
|
}
|
||||||
|
deployment := &model.PagesDeployment{
|
||||||
|
ProjectID: project.ID,
|
||||||
|
DeploymentNumber: 1,
|
||||||
|
Checksum: revision,
|
||||||
|
Status: model.PagesDeploymentStatusUploaded,
|
||||||
|
UploadID: uploadRecord.ID,
|
||||||
|
FileCount: 1,
|
||||||
|
TotalSize: 1,
|
||||||
|
CreatedBy: "user:1",
|
||||||
|
SourceType: PagesSourceTypeRemoteURL,
|
||||||
|
SourceIdentity: &identity,
|
||||||
|
SourceRevision: &revision,
|
||||||
|
SourceLabel: "deleted.zip",
|
||||||
|
SourceMeta: `{"provider":"remote_url","display_name":"deleted.zip"}`,
|
||||||
|
TriggerType: pagesSourceTriggerManualSync,
|
||||||
|
}
|
||||||
|
if err := db.DB(ctx).Create(deployment).Error; err != nil {
|
||||||
|
t.Fatalf("create source deployment error = %v, want nil", err)
|
||||||
|
}
|
||||||
|
if err := db.DB(ctx).Create(&model.PagesDeploymentFile{
|
||||||
|
DeploymentID: deployment.ID,
|
||||||
|
Path: "index.html",
|
||||||
|
Size: 1,
|
||||||
|
Checksum: revision,
|
||||||
|
}).Error; err != nil {
|
||||||
|
t.Fatalf("create source deployment file error = %v, want nil", err)
|
||||||
|
}
|
||||||
|
manifest := &deploymentManifest{
|
||||||
|
FileCount: 1,
|
||||||
|
TotalSize: 1,
|
||||||
|
EntryFile: "index.html",
|
||||||
|
}
|
||||||
|
_, _, _, err := commitSourceDeployment(
|
||||||
|
ctx,
|
||||||
|
snapshot,
|
||||||
|
revision,
|
||||||
|
revision,
|
||||||
|
sourceDetail{Provider: PagesSourceTypeRemoteURL, DisplayName: "deleted.zip"},
|
||||||
|
`{"provider":"remote_url","display_name":"deleted.zip"}`,
|
||||||
|
"user:1",
|
||||||
|
manifest,
|
||||||
|
upload.IngestResult{},
|
||||||
|
false,
|
||||||
|
nil,
|
||||||
|
)
|
||||||
|
if !errors.Is(err, errSourceFinalFence) {
|
||||||
|
t.Errorf("commitSourceDeployment(deleted upload) error = %v, want %v", err, errSourceFinalFence)
|
||||||
|
}
|
||||||
|
storedProject, err := model.GetPagesProjectByID(ctx, project.ID)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("GetPagesProjectByID(%d) error = %v, want nil", project.ID, err)
|
||||||
|
}
|
||||||
|
if storedProject.ActiveDeploymentID != nil {
|
||||||
|
t.Errorf("ActiveDeploymentID after deleted upload rejection = %v, want nil", storedProject.ActiveDeploymentID)
|
||||||
|
}
|
||||||
|
var activeCount int64
|
||||||
|
if err := db.DB(ctx).Model(&model.PagesDeployment{}).
|
||||||
|
Where("project_id = ? AND status = ?", project.ID, model.PagesDeploymentStatusActive).
|
||||||
|
Count(&activeCount).Error; err != nil && !errors.Is(err, gorm.ErrRecordNotFound) {
|
||||||
|
t.Fatalf("count active deployments error = %v, want nil", err)
|
||||||
|
}
|
||||||
|
if got, want := activeCount, int64(0); got != want {
|
||||||
|
t.Errorf("active deployment count after deleted upload rejection = %d, want %d", got, want)
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -0,0 +1,431 @@
|
|||||||
|
// Copyright 2026 Arctel.net
|
||||||
|
// SPDX-License-Identifier: Apache-2.0
|
||||||
|
|
||||||
|
package pages
|
||||||
|
|
||||||
|
import (
|
||||||
|
"bytes"
|
||||||
|
"context"
|
||||||
|
"encoding/hex"
|
||||||
|
"encoding/json"
|
||||||
|
"errors"
|
||||||
|
"io"
|
||||||
|
"strconv"
|
||||||
|
"strings"
|
||||||
|
|
||||||
|
"github.com/Rain-kl/Wavelet/internal/db"
|
||||||
|
"github.com/Rain-kl/Wavelet/internal/model"
|
||||||
|
"github.com/Rain-kl/Wavelet/internal/task"
|
||||||
|
"github.com/Rain-kl/Wavelet/pkg/logger"
|
||||||
|
"gorm.io/gorm"
|
||||||
|
)
|
||||||
|
|
||||||
|
const (
|
||||||
|
// PagesSourceActionTask is the private Asynq task type for source actions.
|
||||||
|
PagesSourceActionTask = "openflare:pages_source_action"
|
||||||
|
// TaskTypePagesSourceAction is the internal task meta type.
|
||||||
|
TaskTypePagesSourceAction = "of_pages_source_action"
|
||||||
|
|
||||||
|
sourceActionCheck = "check"
|
||||||
|
sourceActionSync = "sync"
|
||||||
|
)
|
||||||
|
|
||||||
|
var errUnexpectedJSONTrailingValue = errors.New("unexpected trailing JSON value")
|
||||||
|
|
||||||
|
// PagesSourceActionMeta is intentionally hidden from generic Admin task APIs.
|
||||||
|
var PagesSourceActionMeta = task.TaskMeta{
|
||||||
|
Type: TaskTypePagesSourceAction,
|
||||||
|
AsynqTask: PagesSourceActionTask,
|
||||||
|
Name: "OpenFlare Pages 部署源操作",
|
||||||
|
Description: "检查或同步 Pages 项目部署源",
|
||||||
|
SupportsTime: false,
|
||||||
|
MaxRetry: 2,
|
||||||
|
Queue: task.QueueDefault,
|
||||||
|
Retryable: false,
|
||||||
|
InternalOnly: true,
|
||||||
|
}
|
||||||
|
|
||||||
|
// SourceActionPayload is the credential-free internal queue contract.
|
||||||
|
type SourceActionPayload struct {
|
||||||
|
SourceID uint `json:"source_id"`
|
||||||
|
ConfigVersion int `json:"config_version"`
|
||||||
|
Action string `json:"action"`
|
||||||
|
Actor string `json:"actor"`
|
||||||
|
TriggerType string `json:"trigger_type"`
|
||||||
|
TargetRevision string `json:"target_revision"`
|
||||||
|
ConfirmedRevision string `json:"confirmed_revision"`
|
||||||
|
}
|
||||||
|
|
||||||
|
// SourceActionHandler executes a validated source action.
|
||||||
|
type SourceActionHandler struct{}
|
||||||
|
|
||||||
|
// ValidatePayload rejects unknown keys and normalizes the internal contract.
|
||||||
|
func (h *SourceActionHandler) ValidatePayload(payload []byte) ([]byte, error) {
|
||||||
|
var input SourceActionPayload
|
||||||
|
decoder := json.NewDecoder(bytes.NewReader(payload))
|
||||||
|
decoder.DisallowUnknownFields()
|
||||||
|
if err := decoder.Decode(&input); err != nil {
|
||||||
|
return nil, errors.New(errPagesSourceActionInvalid)
|
||||||
|
}
|
||||||
|
if err := ensureJSONEOF(decoder); err != nil {
|
||||||
|
return nil, errors.New(errPagesSourceActionInvalid)
|
||||||
|
}
|
||||||
|
input.Action = strings.TrimSpace(input.Action)
|
||||||
|
input.Actor = strings.TrimSpace(input.Actor)
|
||||||
|
input.TriggerType = strings.TrimSpace(input.TriggerType)
|
||||||
|
input.TargetRevision = strings.TrimSpace(input.TargetRevision)
|
||||||
|
input.ConfirmedRevision = strings.TrimSpace(input.ConfirmedRevision)
|
||||||
|
if input.Action == sourceActionSync && input.TriggerType == "" {
|
||||||
|
// Keep already queued Phase 2 payloads valid while making every new
|
||||||
|
// dispatch carry an explicit deployment trigger.
|
||||||
|
input.TriggerType = pagesSourceTriggerManualSync
|
||||||
|
}
|
||||||
|
if !validSourceActionPayload(input) {
|
||||||
|
return nil, errors.New(errPagesSourceActionInvalid)
|
||||||
|
}
|
||||||
|
return json.Marshal(input)
|
||||||
|
}
|
||||||
|
|
||||||
|
func validSourceActionPayload(input SourceActionPayload) bool {
|
||||||
|
if input.SourceID == 0 || input.ConfigVersion <= 0 {
|
||||||
|
return false
|
||||||
|
}
|
||||||
|
if input.Action != sourceActionCheck && input.Action != sourceActionSync {
|
||||||
|
return false
|
||||||
|
}
|
||||||
|
if !validPagesSourceActor(input.Actor) {
|
||||||
|
return false
|
||||||
|
}
|
||||||
|
if !validOptionalSourceRevision(input.TargetRevision) ||
|
||||||
|
!validOptionalSourceRevision(input.ConfirmedRevision) {
|
||||||
|
return false
|
||||||
|
}
|
||||||
|
if input.Action == sourceActionCheck {
|
||||||
|
return input.TriggerType == "" && input.TargetRevision == "" && input.ConfirmedRevision == ""
|
||||||
|
}
|
||||||
|
return validSourceSyncPayload(input)
|
||||||
|
}
|
||||||
|
|
||||||
|
func validSourceSyncPayload(input SourceActionPayload) bool {
|
||||||
|
if !validSourceDeploymentTrigger(input.TriggerType) ||
|
||||||
|
(input.TargetRevision != "" && input.ConfirmedRevision != "") {
|
||||||
|
return false
|
||||||
|
}
|
||||||
|
switch input.TriggerType {
|
||||||
|
case pagesSourceTriggerScheduledAutoUpdate:
|
||||||
|
return input.Actor == pagesSourceCreatedBySystem &&
|
||||||
|
input.TargetRevision != "" && input.ConfirmedRevision == ""
|
||||||
|
case pagesSourceTriggerManualSync:
|
||||||
|
if input.TargetRevision != "" || !strings.HasPrefix(input.Actor, "user:") {
|
||||||
|
return false
|
||||||
|
}
|
||||||
|
return true
|
||||||
|
default:
|
||||||
|
return false
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// Execute validates again inside the worker and performs the source action.
|
||||||
|
func (h *SourceActionHandler) Execute(ctx context.Context, payload []byte) (*task.TaskResult, error) {
|
||||||
|
normalized, err := h.ValidatePayload(payload)
|
||||||
|
if err != nil {
|
||||||
|
return nil, task.PermanentError(errPagesSourceActionInvalid)
|
||||||
|
}
|
||||||
|
var input SourceActionPayload
|
||||||
|
if err := json.Unmarshal(normalized, &input); err != nil {
|
||||||
|
return nil, task.PermanentError(errPagesSourceActionInvalid)
|
||||||
|
}
|
||||||
|
|
||||||
|
var source model.PagesProjectSource
|
||||||
|
if err := db.DB(ctx).Where("id = ?", input.SourceID).First(&source).Error; err != nil {
|
||||||
|
if errors.Is(err, gorm.ErrRecordNotFound) {
|
||||||
|
task.AppendLog(ctx, "[resolve] 来源已不存在,本次任务跳过")
|
||||||
|
return &task.TaskResult{Message: errPagesSourceActionStale}, nil
|
||||||
|
}
|
||||||
|
logger.ErrorF(ctx, "[PagesSource] load source failed: source_id=%d error=%v", input.SourceID, err)
|
||||||
|
return nil, errors.New(errPagesSourceSyncFailed)
|
||||||
|
}
|
||||||
|
if source.ConfigVersion != input.ConfigVersion {
|
||||||
|
task.AppendLog(ctx, "[resolve] 来源配置已变化,本次任务跳过")
|
||||||
|
return &task.TaskResult{Message: errPagesSourceActionStale}, nil
|
||||||
|
}
|
||||||
|
if input.Action == sourceActionCheck && source.SourceType == PagesSourceTypeRemoteURL {
|
||||||
|
return nil, task.PermanentError(errPagesSourceCheckUnsupported)
|
||||||
|
}
|
||||||
|
if source.SourceType != PagesSourceTypeRemoteURL && source.SourceType != PagesSourceTypeGitHubRelease {
|
||||||
|
return nil, task.PermanentError(errPagesSourceTypeUnsupported)
|
||||||
|
}
|
||||||
|
if source.SourceType == PagesSourceTypeRemoteURL && (input.TargetRevision != "" || input.ConfirmedRevision != "") {
|
||||||
|
return nil, task.PermanentError(errPagesSourceActionInvalid)
|
||||||
|
}
|
||||||
|
if input.Action == sourceActionCheck && (input.TargetRevision != "" || input.ConfirmedRevision != "") {
|
||||||
|
return nil, task.PermanentError(errPagesSourceActionInvalid)
|
||||||
|
}
|
||||||
|
|
||||||
|
task.AppendLog(ctx, "[resolve] 正在获取来源执行权")
|
||||||
|
snapshot, outcome, err := acquireSourceLease(ctx, input.SourceID, input.ConfigVersion, input.Action)
|
||||||
|
if err != nil {
|
||||||
|
logger.ErrorF(ctx, "[PagesSource] acquire lease failed: source_id=%d error=%v", input.SourceID, err)
|
||||||
|
return nil, errors.New(errPagesSourceSyncFailed)
|
||||||
|
}
|
||||||
|
switch outcome {
|
||||||
|
case sourceLeaseBusy:
|
||||||
|
task.AppendLog(ctx, "[resolve] 已有来源任务正在执行,本次任务跳过")
|
||||||
|
return &task.TaskResult{Message: errPagesSourceActionBusy}, nil
|
||||||
|
case sourceLeaseStale:
|
||||||
|
task.AppendLog(ctx, "[resolve] 来源配置或执行权已变化,本次任务跳过")
|
||||||
|
return &task.TaskResult{Message: errPagesSourceActionStale}, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
if input.Action == sourceActionCheck {
|
||||||
|
return executeGitHubCheckAction(ctx, snapshot)
|
||||||
|
}
|
||||||
|
return executeSourceSyncAction(ctx, &source, snapshot, input)
|
||||||
|
}
|
||||||
|
|
||||||
|
func executeGitHubCheckAction(ctx context.Context, snapshot *sourceExecutionSnapshot) (*task.TaskResult, error) {
|
||||||
|
checkResult, checkErr := checkGitHubSource(ctx, snapshot)
|
||||||
|
if checkErr != nil {
|
||||||
|
logger.ErrorF(ctx, "[PagesSource] check failed: project_id=%d source_id=%d error=%v", snapshot.ProjectID, snapshot.SourceID, checkErr)
|
||||||
|
if isPermanentSourceSyncError(checkErr) || shouldSkipGitHubActionRetry(checkErr) {
|
||||||
|
return nil, task.PermanentError(checkErr.Error())
|
||||||
|
}
|
||||||
|
return nil, errors.New(errPagesSourceSyncFailed)
|
||||||
|
}
|
||||||
|
if checkResult == nil || checkResult.Stale {
|
||||||
|
return &task.TaskResult{Message: errPagesSourceActionStale}, nil
|
||||||
|
}
|
||||||
|
return &task.TaskResult{Message: checkResult.Message, Detail: checkResult.Detail}, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func executeSourceSyncAction(
|
||||||
|
ctx context.Context,
|
||||||
|
source *model.PagesProjectSource,
|
||||||
|
snapshot *sourceExecutionSnapshot,
|
||||||
|
input SourceActionPayload,
|
||||||
|
) (*task.TaskResult, error) {
|
||||||
|
var result *sourceSyncOutcome
|
||||||
|
var err error
|
||||||
|
if source.SourceType == PagesSourceTypeGitHubRelease {
|
||||||
|
result, err = syncGitHubSourceWithTrigger(
|
||||||
|
ctx, snapshot, input.Actor, input.TargetRevision, input.ConfirmedRevision, input.TriggerType,
|
||||||
|
)
|
||||||
|
} else {
|
||||||
|
result, err = syncRemoteSourceWithTrigger(ctx, snapshot, input.Actor, input.TriggerType)
|
||||||
|
}
|
||||||
|
if err != nil {
|
||||||
|
logger.ErrorF(ctx, "[PagesSource] sync failed: project_id=%d source_id=%d error=%v", snapshot.ProjectID, snapshot.SourceID, err)
|
||||||
|
if isPermanentSourceSyncError(err) || shouldSkipGitHubActionRetry(err) {
|
||||||
|
return nil, task.PermanentError(errPagesSourceSyncFailed)
|
||||||
|
}
|
||||||
|
return nil, errors.New(errPagesSourceSyncFailed)
|
||||||
|
}
|
||||||
|
if result == nil || result.Stale {
|
||||||
|
task.AppendLog(ctx, "[activate] 来源配置或执行权已变化,本次任务未切换部署")
|
||||||
|
return &task.TaskResult{Message: errPagesSourceActionStale}, nil
|
||||||
|
}
|
||||||
|
message := "Pages 部署源同步并发布成功"
|
||||||
|
if result.Reused {
|
||||||
|
message = "Pages 部署源内容未变化,已重新激活现有部署"
|
||||||
|
}
|
||||||
|
return &task.TaskResult{Message: message, Detail: sourceSyncResultDetail(result)}, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func ensureJSONEOF(decoder *json.Decoder) error {
|
||||||
|
var trailing any
|
||||||
|
err := decoder.Decode(&trailing)
|
||||||
|
if errors.Is(err, io.EOF) {
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
if err == nil {
|
||||||
|
return errUnexpectedJSONTrailingValue
|
||||||
|
}
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
|
||||||
|
func validPagesSourceActor(actor string) bool {
|
||||||
|
if actor == pagesSourceCreatedBySystem {
|
||||||
|
return true
|
||||||
|
}
|
||||||
|
if !strings.HasPrefix(actor, "user:") {
|
||||||
|
return false
|
||||||
|
}
|
||||||
|
id, err := strconv.ParseUint(strings.TrimPrefix(actor, "user:"), 10, 64)
|
||||||
|
return err == nil && id > 0
|
||||||
|
}
|
||||||
|
|
||||||
|
func validOptionalSourceRevision(value string) bool {
|
||||||
|
if value == "" {
|
||||||
|
return true
|
||||||
|
}
|
||||||
|
if len(value) != sourceRevisionHexLength {
|
||||||
|
return false
|
||||||
|
}
|
||||||
|
decoded, err := hex.DecodeString(value)
|
||||||
|
return err == nil && len(decoded) == 32
|
||||||
|
}
|
||||||
|
|
||||||
|
func isPermanentSourceSyncError(err error) bool {
|
||||||
|
if err == nil {
|
||||||
|
return false
|
||||||
|
}
|
||||||
|
message := err.Error()
|
||||||
|
return strings.Contains(message, errPagesPackageUnsupported) ||
|
||||||
|
strings.Contains(message, errPagesPackageURLTooLarge) ||
|
||||||
|
strings.Contains(message, errPagesPackageInvalid) ||
|
||||||
|
strings.Contains(message, errPagesPackageEmpty) ||
|
||||||
|
strings.Contains(message, errPagesPackageExtractedTooLarge) ||
|
||||||
|
strings.Contains(message, errPagesPackageFileTooLarge) ||
|
||||||
|
strings.Contains(message, errPagesEntryFileMissing) ||
|
||||||
|
strings.Contains(message, errPagesSourceRemoteURLInvalid) ||
|
||||||
|
strings.Contains(message, errPagesSourceNetworkPolicy) ||
|
||||||
|
strings.Contains(message, errPagesSourceReleaseNotFound) ||
|
||||||
|
strings.Contains(message, errPagesSourceDigestInvalid) ||
|
||||||
|
strings.Contains(message, errPagesSourceDigestMismatch) ||
|
||||||
|
strings.Contains(message, errPagesSourceConfirmationNeeded) ||
|
||||||
|
strings.Contains(message, errPagesSourceConfirmationStale)
|
||||||
|
}
|
||||||
|
|
||||||
|
// DispatchSourceAction performs API preflight and enqueues a credential-free action.
|
||||||
|
func DispatchSourceAction(
|
||||||
|
ctx context.Context,
|
||||||
|
projectID uint,
|
||||||
|
action string,
|
||||||
|
actor string,
|
||||||
|
confirmedRevision string,
|
||||||
|
) (*SourceActionReceipt, error) {
|
||||||
|
return dispatchSourceActionByProject(ctx, projectID, action, actor, "", confirmedRevision)
|
||||||
|
}
|
||||||
|
|
||||||
|
func dispatchSourceActionByProject(
|
||||||
|
ctx context.Context,
|
||||||
|
projectID uint,
|
||||||
|
action string,
|
||||||
|
actor string,
|
||||||
|
targetRevision string,
|
||||||
|
confirmedRevision string,
|
||||||
|
) (*SourceActionReceipt, error) {
|
||||||
|
action = strings.TrimSpace(action)
|
||||||
|
targetRevision = strings.TrimSpace(targetRevision)
|
||||||
|
confirmedRevision = strings.TrimSpace(confirmedRevision)
|
||||||
|
if action != sourceActionCheck && action != sourceActionSync {
|
||||||
|
return nil, errors.New(errPagesSourceActionInvalid)
|
||||||
|
}
|
||||||
|
if !validPagesSourceActor(actor) {
|
||||||
|
return nil, errors.New(errPagesSourceActionInvalid)
|
||||||
|
}
|
||||||
|
|
||||||
|
var source model.PagesProjectSource
|
||||||
|
if err := db.DB(ctx).Where("project_id = ?", projectID).First(&source).Error; err != nil {
|
||||||
|
if errors.Is(err, gorm.ErrRecordNotFound) {
|
||||||
|
return nil, errors.New(errPagesSourceNotFound)
|
||||||
|
}
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
if err := validateSourceActionPreflight(ctx, &source, action, targetRevision, confirmedRevision); err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
busy, err := sourceLeaseIsBusy(ctx, source.ID)
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
if busy {
|
||||||
|
return nil, errors.New(errPagesSourceActionBusy)
|
||||||
|
}
|
||||||
|
return dispatchSourceActionSnapshot(ctx, source, action, actor, targetRevision, confirmedRevision, "manual")
|
||||||
|
}
|
||||||
|
|
||||||
|
func validateSourceActionPreflight(
|
||||||
|
ctx context.Context,
|
||||||
|
source *model.PagesProjectSource,
|
||||||
|
action string,
|
||||||
|
targetRevision string,
|
||||||
|
confirmedRevision string,
|
||||||
|
) error {
|
||||||
|
if source == nil {
|
||||||
|
return errors.New(errPagesSourceNotFound)
|
||||||
|
}
|
||||||
|
if source.SourceType != PagesSourceTypeRemoteURL && source.SourceType != PagesSourceTypeGitHubRelease {
|
||||||
|
return errors.New(errPagesSourceTypeUnsupported)
|
||||||
|
}
|
||||||
|
if action == sourceActionCheck && source.SourceType == PagesSourceTypeRemoteURL {
|
||||||
|
return errors.New(errPagesSourceCheckUnsupported)
|
||||||
|
}
|
||||||
|
if source.SourceType == PagesSourceTypeRemoteURL && (targetRevision != "" || confirmedRevision != "") {
|
||||||
|
return errors.New(errPagesSourceActionInvalid)
|
||||||
|
}
|
||||||
|
if action == sourceActionCheck && (targetRevision != "" || confirmedRevision != "") {
|
||||||
|
return errors.New(errPagesSourceActionInvalid)
|
||||||
|
}
|
||||||
|
if source.SourceType == PagesSourceTypeGitHubRelease && action == sourceActionSync {
|
||||||
|
if err := preflightGitHubSyncConfirmation(ctx, source.ID, confirmedRevision); err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func dispatchSourceActionSnapshot(
|
||||||
|
ctx context.Context,
|
||||||
|
source model.PagesProjectSource,
|
||||||
|
action string,
|
||||||
|
actor string,
|
||||||
|
targetRevision string,
|
||||||
|
confirmedRevision string,
|
||||||
|
triggeredBy string,
|
||||||
|
) (*SourceActionReceipt, error) {
|
||||||
|
triggerType := ""
|
||||||
|
if action == sourceActionSync {
|
||||||
|
triggerType = pagesSourceTriggerManualSync
|
||||||
|
}
|
||||||
|
return dispatchSourceActionSnapshotWithTrigger(
|
||||||
|
ctx, source, action, actor, triggerType, targetRevision, confirmedRevision, triggeredBy,
|
||||||
|
)
|
||||||
|
}
|
||||||
|
|
||||||
|
func dispatchSourceActionSnapshotWithTrigger(
|
||||||
|
ctx context.Context,
|
||||||
|
source model.PagesProjectSource,
|
||||||
|
action string,
|
||||||
|
actor string,
|
||||||
|
triggerType string,
|
||||||
|
targetRevision string,
|
||||||
|
confirmedRevision string,
|
||||||
|
triggeredBy string,
|
||||||
|
) (*SourceActionReceipt, error) {
|
||||||
|
if task.AsynqClient == nil {
|
||||||
|
return nil, errors.New(errPagesSourceTaskDispatchFailed)
|
||||||
|
}
|
||||||
|
handler := &SourceActionHandler{}
|
||||||
|
rawPayload, err := json.Marshal(SourceActionPayload{
|
||||||
|
SourceID: source.ID,
|
||||||
|
ConfigVersion: source.ConfigVersion,
|
||||||
|
Action: action,
|
||||||
|
Actor: actor,
|
||||||
|
TriggerType: triggerType,
|
||||||
|
TargetRevision: targetRevision,
|
||||||
|
ConfirmedRevision: confirmedRevision,
|
||||||
|
})
|
||||||
|
if err != nil {
|
||||||
|
return nil, errors.New(errPagesSourceActionInvalid)
|
||||||
|
}
|
||||||
|
payload, err := handler.ValidatePayload(rawPayload)
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
taskID, err := task.DispatchTask(ctx, TaskTypePagesSourceAction, payload, triggeredBy)
|
||||||
|
if err != nil {
|
||||||
|
logger.ErrorF(ctx, "[PagesSource] dispatch action failed: project_id=%d source_id=%d action=%s error=%v", source.ProjectID, source.ID, action, err)
|
||||||
|
return nil, errors.New(errPagesSourceTaskDispatchFailed)
|
||||||
|
}
|
||||||
|
execution, err := model.GetTaskExecutionByTaskID(ctx, taskID)
|
||||||
|
if err != nil {
|
||||||
|
logger.ErrorF(ctx, "[PagesSource] load dispatched execution failed: source_id=%d task_id=%s error=%v", source.ID, taskID, err)
|
||||||
|
return nil, errors.New(errPagesSourceTaskDispatchFailed)
|
||||||
|
}
|
||||||
|
return &SourceActionReceipt{
|
||||||
|
TaskID: taskID,
|
||||||
|
ExecutionID: strconv.FormatUint(execution.ID, 10),
|
||||||
|
Action: action,
|
||||||
|
}, nil
|
||||||
|
}
|
||||||
@@ -0,0 +1,184 @@
|
|||||||
|
// Copyright 2026 Arctel.net
|
||||||
|
// SPDX-License-Identifier: Apache-2.0
|
||||||
|
|
||||||
|
package pages
|
||||||
|
|
||||||
|
import (
|
||||||
|
"encoding/json"
|
||||||
|
"errors"
|
||||||
|
"strings"
|
||||||
|
"testing"
|
||||||
|
|
||||||
|
"github.com/hibiken/asynq"
|
||||||
|
)
|
||||||
|
|
||||||
|
func TestSourceActionPayloadValidationIsStrictAndCredentialFree(t *testing.T) {
|
||||||
|
handler := &SourceActionHandler{}
|
||||||
|
valid := SourceActionPayload{
|
||||||
|
SourceID: 7,
|
||||||
|
ConfigVersion: 3,
|
||||||
|
Action: sourceActionSync,
|
||||||
|
Actor: "user:42",
|
||||||
|
TriggerType: pagesSourceTriggerManualSync,
|
||||||
|
}
|
||||||
|
raw, err := json.Marshal(valid)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("json.Marshal(valid payload) error = %v, want nil", err)
|
||||||
|
}
|
||||||
|
normalized, err := handler.ValidatePayload(raw)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("ValidatePayload(valid) error = %v, want nil", err)
|
||||||
|
}
|
||||||
|
var got SourceActionPayload
|
||||||
|
if err := json.Unmarshal(normalized, &got); err != nil {
|
||||||
|
t.Fatalf("json.Unmarshal(normalized payload) error = %v, want nil", err)
|
||||||
|
}
|
||||||
|
if got != valid {
|
||||||
|
t.Errorf("ValidatePayload(valid) = %+v, want %+v", got, valid)
|
||||||
|
}
|
||||||
|
legacy := valid
|
||||||
|
legacy.TriggerType = ""
|
||||||
|
legacyRaw, err := json.Marshal(legacy)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("json.Marshal(legacy payload) error = %v, want nil", err)
|
||||||
|
}
|
||||||
|
legacyNormalized, err := handler.ValidatePayload(legacyRaw)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("ValidatePayload(legacy payload) error = %v, want nil", err)
|
||||||
|
}
|
||||||
|
var legacyGot SourceActionPayload
|
||||||
|
if err := json.Unmarshal(legacyNormalized, &legacyGot); err != nil {
|
||||||
|
t.Fatalf("json.Unmarshal(legacy normalized payload) error = %v, want nil", err)
|
||||||
|
}
|
||||||
|
if legacyGot.TriggerType != pagesSourceTriggerManualSync {
|
||||||
|
t.Errorf("legacy payload trigger_type = %q, want %q", legacyGot.TriggerType, pagesSourceTriggerManualSync)
|
||||||
|
}
|
||||||
|
for _, forbidden := range []string{"remote_url", "content_config_version", "expected_revision", "lease_token", "etag"} {
|
||||||
|
if strings.Contains(string(normalized), forbidden) {
|
||||||
|
t.Errorf("normalized payload = %s, want no forbidden field %q", normalized, forbidden)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
invalidPayloads := []struct {
|
||||||
|
name string
|
||||||
|
raw string
|
||||||
|
}{
|
||||||
|
{
|
||||||
|
name: "unknown remote URL field",
|
||||||
|
raw: `{"source_id":7,"config_version":3,"action":"sync","actor":"user:42","remote_url":"https://example.com/site.zip?token=secret"}`,
|
||||||
|
},
|
||||||
|
{
|
||||||
|
name: "unknown content version field",
|
||||||
|
raw: `{"source_id":7,"config_version":3,"action":"sync","actor":"user:42","content_config_version":9}`,
|
||||||
|
},
|
||||||
|
{
|
||||||
|
name: "empty actor",
|
||||||
|
raw: `{"source_id":7,"config_version":3,"action":"sync","actor":""}`,
|
||||||
|
},
|
||||||
|
{
|
||||||
|
name: "untrusted system actor",
|
||||||
|
raw: `{"source_id":7,"config_version":3,"action":"sync","actor":"system"}`,
|
||||||
|
},
|
||||||
|
{
|
||||||
|
name: "zero user actor",
|
||||||
|
raw: `{"source_id":7,"config_version":3,"action":"sync","actor":"user:0"}`,
|
||||||
|
},
|
||||||
|
{
|
||||||
|
name: "invalid action",
|
||||||
|
raw: `{"source_id":7,"config_version":3,"action":"activate","actor":"user:42"}`,
|
||||||
|
},
|
||||||
|
{
|
||||||
|
name: "multiple JSON values",
|
||||||
|
raw: `{"source_id":7,"config_version":3,"action":"sync","actor":"user:42"} {}`,
|
||||||
|
},
|
||||||
|
}
|
||||||
|
for _, test := range invalidPayloads {
|
||||||
|
t.Run(test.name, func(t *testing.T) {
|
||||||
|
normalized, err := handler.ValidatePayload([]byte(test.raw))
|
||||||
|
if err == nil {
|
||||||
|
t.Errorf("ValidatePayload(%s) = %s, nil; want non-nil error", test.raw, normalized)
|
||||||
|
}
|
||||||
|
if err != nil && strings.Contains(err.Error(), "secret") {
|
||||||
|
t.Errorf("ValidatePayload(%s) error = %q, want credential-free error", test.name, err)
|
||||||
|
}
|
||||||
|
})
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestSourceActionPayloadAcceptsOnlyRealActors(t *testing.T) {
|
||||||
|
tests := []struct {
|
||||||
|
actor string
|
||||||
|
want bool
|
||||||
|
}{
|
||||||
|
{actor: "user:1", want: true},
|
||||||
|
{actor: "user:18446744073709551615", want: true},
|
||||||
|
{actor: pagesSourceCreatedBySystem, want: true},
|
||||||
|
{actor: "", want: false},
|
||||||
|
{actor: "user:0", want: false},
|
||||||
|
{actor: "user:-1", want: false},
|
||||||
|
{actor: "user:not-a-number", want: false},
|
||||||
|
{actor: "system", want: false},
|
||||||
|
}
|
||||||
|
for _, test := range tests {
|
||||||
|
if got := validPagesSourceActor(test.actor); got != test.want {
|
||||||
|
t.Errorf("validPagesSourceActor(%q) = %t, want %t", test.actor, got, test.want)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestRemoteCheckActionIsPermanentWithoutExposingURL(t *testing.T) {
|
||||||
|
ctx := setupPagesSourceTest(t)
|
||||||
|
project := mustCreatePagesSourceProject(t, ctx, "task-remote-check")
|
||||||
|
secret := "task-query-secret"
|
||||||
|
source, _ := mustConfigureRemoteSource(
|
||||||
|
t,
|
||||||
|
ctx,
|
||||||
|
project.ID,
|
||||||
|
"https://example.com/site.zip?token="+secret,
|
||||||
|
RemoteNetworkPolicyPublic,
|
||||||
|
)
|
||||||
|
raw, err := json.Marshal(SourceActionPayload{
|
||||||
|
SourceID: source.ID,
|
||||||
|
ConfigVersion: source.ConfigVersion,
|
||||||
|
Action: sourceActionCheck,
|
||||||
|
Actor: "user:9",
|
||||||
|
})
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("json.Marshal(check payload) error = %v, want nil", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
result, err := (&SourceActionHandler{}).Execute(ctx, raw)
|
||||||
|
if result != nil {
|
||||||
|
t.Errorf("SourceActionHandler.Execute(remote check) result = %+v, want nil", result)
|
||||||
|
}
|
||||||
|
if err == nil {
|
||||||
|
t.Fatal("SourceActionHandler.Execute(remote check) error = nil, want permanent error")
|
||||||
|
}
|
||||||
|
if !errors.Is(err, asynq.SkipRetry) {
|
||||||
|
t.Errorf("SourceActionHandler.Execute(remote check) error = %v, want errors.Is(asynq.SkipRetry)", err)
|
||||||
|
}
|
||||||
|
if got, want := err.Error(), errPagesSourceCheckUnsupported; got != want {
|
||||||
|
t.Errorf("SourceActionHandler.Execute(remote check) error = %q, want %q", got, want)
|
||||||
|
}
|
||||||
|
if strings.Contains(err.Error(), secret) || strings.Contains(string(raw), secret) {
|
||||||
|
t.Errorf("remote check result error/payload = %q / %s, want no URL secret", err, raw)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestPagesSourceActionMetaIsInternalOnly(t *testing.T) {
|
||||||
|
if !PagesSourceActionMeta.InternalOnly {
|
||||||
|
t.Error("PagesSourceActionMeta.InternalOnly = false, want true")
|
||||||
|
}
|
||||||
|
if PagesSourceActionMeta.Type != TaskTypePagesSourceAction {
|
||||||
|
t.Errorf("PagesSourceActionMeta.Type = %q, want %q", PagesSourceActionMeta.Type, TaskTypePagesSourceAction)
|
||||||
|
}
|
||||||
|
if PagesSourceActionMeta.AsynqTask != PagesSourceActionTask {
|
||||||
|
t.Errorf("PagesSourceActionMeta.AsynqTask = %q, want %q", PagesSourceActionMeta.AsynqTask, PagesSourceActionTask)
|
||||||
|
}
|
||||||
|
if PagesSourceActionMeta.Retryable {
|
||||||
|
t.Error("PagesSourceActionMeta.Retryable = true, want false for manual retry API")
|
||||||
|
}
|
||||||
|
if PagesSourceActionMeta.MaxRetry <= 0 {
|
||||||
|
t.Errorf("PagesSourceActionMeta.MaxRetry = %d, want bounded transient retries", PagesSourceActionMeta.MaxRetry)
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -0,0 +1,413 @@
|
|||||||
|
// Copyright 2026 Arctel.net
|
||||||
|
// SPDX-License-Identifier: Apache-2.0
|
||||||
|
|
||||||
|
package pages
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
"encoding/json"
|
||||||
|
"fmt"
|
||||||
|
"strings"
|
||||||
|
"testing"
|
||||||
|
"time"
|
||||||
|
|
||||||
|
"github.com/Rain-kl/Wavelet/internal/db"
|
||||||
|
"github.com/Rain-kl/Wavelet/internal/model"
|
||||||
|
)
|
||||||
|
|
||||||
|
func setupPagesSourceTest(t *testing.T) context.Context {
|
||||||
|
t.Helper()
|
||||||
|
cleanup := setupPagesTestDB(t)
|
||||||
|
t.Cleanup(cleanup)
|
||||||
|
sqlDB, err := db.DB(t.Context()).DB()
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("db.DB().DB() error = %v, want nil", err)
|
||||||
|
}
|
||||||
|
// SQLite :memory: is scoped to one connection. Keeping one connection also
|
||||||
|
// makes lease tests exercise the production CAS without creating empty
|
||||||
|
// per-connection databases.
|
||||||
|
sqlDB.SetMaxOpenConns(1)
|
||||||
|
return t.Context()
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestRevisionViewReadsLegacySourceDetailLabel(t *testing.T) {
|
||||||
|
tests := []struct {
|
||||||
|
name string
|
||||||
|
detail string
|
||||||
|
want string
|
||||||
|
}{
|
||||||
|
{
|
||||||
|
name: "remote",
|
||||||
|
detail: `{"provider":"remote_url","label":"legacy.zip"}`,
|
||||||
|
want: "legacy.zip",
|
||||||
|
},
|
||||||
|
{
|
||||||
|
name: "github",
|
||||||
|
detail: `{"provider":"github","label":"v1.2.3","asset_name":"dist.zip"}`,
|
||||||
|
want: "v1.2.3",
|
||||||
|
},
|
||||||
|
}
|
||||||
|
for _, test := range tests {
|
||||||
|
t.Run(test.name, func(t *testing.T) {
|
||||||
|
view := revisionView(strings.Repeat("a", 64), test.detail)
|
||||||
|
if view.Label != test.want {
|
||||||
|
t.Errorf("revisionView(%s).Label = %q, want %q", test.name, view.Label, test.want)
|
||||||
|
}
|
||||||
|
})
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func mustCreatePagesSourceProject(t *testing.T, ctx context.Context, slug string) *model.PagesProject {
|
||||||
|
t.Helper()
|
||||||
|
view, err := CreateProject(ctx, Input{
|
||||||
|
Name: "Source " + slug,
|
||||||
|
Slug: slug,
|
||||||
|
Enabled: true,
|
||||||
|
EntryFile: "index.html",
|
||||||
|
})
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("CreateProject(%q) error = %v, want nil", slug, err)
|
||||||
|
}
|
||||||
|
project, err := model.GetPagesProjectByID(ctx, view.ID)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("GetPagesProjectByID(%d) error = %v, want nil", view.ID, err)
|
||||||
|
}
|
||||||
|
return project
|
||||||
|
}
|
||||||
|
|
||||||
|
func mustConfigureRemoteSource(
|
||||||
|
t *testing.T,
|
||||||
|
ctx context.Context,
|
||||||
|
projectID uint,
|
||||||
|
remoteURL string,
|
||||||
|
policy string,
|
||||||
|
) (*model.PagesProjectSource, *model.PagesProjectSourceRuntime) {
|
||||||
|
t.Helper()
|
||||||
|
_, err := UpdateSource(ctx, projectID, SourceUpdateInput{
|
||||||
|
SourceType: PagesSourceTypeRemoteURL,
|
||||||
|
RemoteURLSet: true,
|
||||||
|
RemoteURL: remoteURL,
|
||||||
|
RemoteNetworkPolicy: policy,
|
||||||
|
})
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("UpdateSource(%d, %q) error = %v, want nil", projectID, remoteURL, err)
|
||||||
|
}
|
||||||
|
var source model.PagesProjectSource
|
||||||
|
if err := db.DB(ctx).Where("project_id = ?", projectID).First(&source).Error; err != nil {
|
||||||
|
t.Fatalf("load source for project %d error = %v, want nil", projectID, err)
|
||||||
|
}
|
||||||
|
var runtime model.PagesProjectSourceRuntime
|
||||||
|
if err := db.DB(ctx).Where("source_id = ?", source.ID).First(&runtime).Error; err != nil {
|
||||||
|
t.Fatalf("load runtime for source %d error = %v, want nil", source.ID, err)
|
||||||
|
}
|
||||||
|
return &source, &runtime
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestValidateRemoteSourceInputRejectsModeIncompatibleFields(t *testing.T) {
|
||||||
|
tests := []struct {
|
||||||
|
name string
|
||||||
|
input SourceUpdateInput
|
||||||
|
}{
|
||||||
|
{
|
||||||
|
name: "missing source type",
|
||||||
|
input: SourceUpdateInput{
|
||||||
|
RemoteURLSet: true,
|
||||||
|
RemoteURL: "https://example.com/site.zip",
|
||||||
|
},
|
||||||
|
},
|
||||||
|
{
|
||||||
|
name: "github type reserved for phase two",
|
||||||
|
input: SourceUpdateInput{
|
||||||
|
SourceType: PagesSourceTypeGitHubRelease,
|
||||||
|
RepositoryURL: "https://github.com/example/site",
|
||||||
|
},
|
||||||
|
},
|
||||||
|
{
|
||||||
|
name: "remote rejects repository field",
|
||||||
|
input: SourceUpdateInput{
|
||||||
|
SourceType: PagesSourceTypeRemoteURL,
|
||||||
|
RemoteURLSet: true,
|
||||||
|
RemoteURL: "https://example.com/site.zip",
|
||||||
|
RepositoryURL: "https://github.com/example/site",
|
||||||
|
},
|
||||||
|
},
|
||||||
|
{
|
||||||
|
name: "remote rejects automatic updates",
|
||||||
|
input: SourceUpdateInput{
|
||||||
|
SourceType: PagesSourceTypeRemoteURL,
|
||||||
|
RemoteURLSet: true,
|
||||||
|
RemoteURL: "https://example.com/site.zip",
|
||||||
|
AutoUpdateEnabled: true,
|
||||||
|
},
|
||||||
|
},
|
||||||
|
{
|
||||||
|
name: "url value requires replacement flag",
|
||||||
|
input: SourceUpdateInput{
|
||||||
|
SourceType: PagesSourceTypeRemoteURL,
|
||||||
|
RemoteURL: "https://example.com/site.zip",
|
||||||
|
},
|
||||||
|
},
|
||||||
|
{
|
||||||
|
name: "invalid network policy",
|
||||||
|
input: SourceUpdateInput{
|
||||||
|
SourceType: PagesSourceTypeRemoteURL,
|
||||||
|
RemoteURLSet: true,
|
||||||
|
RemoteURL: "https://example.com/site.zip",
|
||||||
|
RemoteNetworkPolicy: "private",
|
||||||
|
},
|
||||||
|
},
|
||||||
|
}
|
||||||
|
|
||||||
|
for _, test := range tests {
|
||||||
|
t.Run(test.name, func(t *testing.T) {
|
||||||
|
if err := validateRemoteSourceInput(test.input); err == nil {
|
||||||
|
t.Errorf("validateRemoteSourceInput(%+v) error = nil, want non-nil", test.input)
|
||||||
|
}
|
||||||
|
})
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestUpdateSourceNewRemoteRequiresExplicitURL(t *testing.T) {
|
||||||
|
ctx := setupPagesSourceTest(t)
|
||||||
|
project := mustCreatePagesSourceProject(t, ctx, "remote-requires-url")
|
||||||
|
_, err := UpdateSource(ctx, project.ID, SourceUpdateInput{
|
||||||
|
SourceType: PagesSourceTypeRemoteURL,
|
||||||
|
})
|
||||||
|
if err == nil {
|
||||||
|
t.Fatal("UpdateSource(new remote without URL) error = nil, want non-nil")
|
||||||
|
}
|
||||||
|
if got, want := err.Error(), errPagesSourceRemoteURLRequired; got != want {
|
||||||
|
t.Errorf("UpdateSource(new remote without URL) error = %q, want %q", got, want)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestRemoteSourceCRUDPreservesSecretAndResetsRuntimeByIdentity(t *testing.T) {
|
||||||
|
ctx := setupPagesSourceTest(t)
|
||||||
|
project := mustCreatePagesSourceProject(t, ctx, "remote-crud")
|
||||||
|
firstURL := "https://Artifacts.Example.com:443/dist/site.zip?token=first-secret&expires=1"
|
||||||
|
source, runtime := mustConfigureRemoteSource(t, ctx, project.ID, firstURL, RemoteNetworkPolicyPublic)
|
||||||
|
|
||||||
|
if got, want := source.ConfigVersion, 1; got != want {
|
||||||
|
t.Errorf("new source ConfigVersion = %d, want %d", got, want)
|
||||||
|
}
|
||||||
|
if got, want := runtime.SyncStatus, pagesSourceStatusIdle; got != want {
|
||||||
|
t.Errorf("new runtime SyncStatus = %q, want %q", got, want)
|
||||||
|
}
|
||||||
|
view, err := GetSource(ctx, project.ID)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("GetSource(%d) error = %v, want nil", project.ID, err)
|
||||||
|
}
|
||||||
|
if got, want := view.DisplayURL, "https://Artifacts.Example.com:443/dist/site.zip?***"; got != want {
|
||||||
|
t.Errorf("GetSource(%d).DisplayURL = %q, want %q", project.ID, got, want)
|
||||||
|
}
|
||||||
|
encodedView, err := json.Marshal(view)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("json.Marshal(GetSource(%d)) error = %v, want nil", project.ID, err)
|
||||||
|
}
|
||||||
|
if strings.Contains(string(encodedView), "first-secret") || strings.Contains(string(encodedView), "expires=1") {
|
||||||
|
t.Errorf("GetSource(%d) JSON = %s, want credential-free view", project.ID, encodedView)
|
||||||
|
}
|
||||||
|
if _, err := UpdateSource(ctx, project.ID, SourceUpdateInput{
|
||||||
|
SourceType: PagesSourceTypeRemoteURL,
|
||||||
|
RemoteURLSet: true,
|
||||||
|
RemoteURL: firstURL,
|
||||||
|
RemoteNetworkPolicy: RemoteNetworkPolicyPublic,
|
||||||
|
}); err != nil {
|
||||||
|
t.Fatalf("UpdateSource(%d, no-op) error = %v, want nil", project.ID, err)
|
||||||
|
}
|
||||||
|
var unchangedSource model.PagesProjectSource
|
||||||
|
if err := db.DB(ctx).Where("id = ?", source.ID).First(&unchangedSource).Error; err != nil {
|
||||||
|
t.Fatalf("load no-op source error = %v, want nil", err)
|
||||||
|
}
|
||||||
|
if got, want := unchangedSource.ConfigVersion, source.ConfigVersion; got != want {
|
||||||
|
t.Errorf("no-op source ConfigVersion = %d, want unchanged %d", got, want)
|
||||||
|
}
|
||||||
|
|
||||||
|
seenRevision := strings.Repeat("a", 64)
|
||||||
|
appliedRevision := strings.Repeat("b", 64)
|
||||||
|
future := time.Now().Add(time.Hour)
|
||||||
|
if err := db.DB(ctx).Model(&model.PagesProjectSourceRuntime{}).
|
||||||
|
Where("source_id = ?", source.ID).
|
||||||
|
Updates(map[string]any{
|
||||||
|
"last_seen_revision": seenRevision,
|
||||||
|
"last_seen_detail": `{"provider":"remote_url","display_name":"new.zip"}`,
|
||||||
|
"last_applied_revision": appliedRevision,
|
||||||
|
"last_applied_detail": `{"provider":"remote_url","display_name":"old.zip"}`,
|
||||||
|
"sync_status": pagesSourceStatusSyncing,
|
||||||
|
"lease_token": "in-flight",
|
||||||
|
"lease_expires_at": &future,
|
||||||
|
}).Error; err != nil {
|
||||||
|
t.Fatalf("seed source runtime error = %v, want nil", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
// Omit the secret URL while changing policy. The stored URL and cursor must
|
||||||
|
// survive, while the in-flight lease is fenced.
|
||||||
|
if _, err := UpdateSource(ctx, project.ID, SourceUpdateInput{
|
||||||
|
SourceType: PagesSourceTypeRemoteURL,
|
||||||
|
RemoteURLSet: false,
|
||||||
|
RemoteNetworkPolicy: RemoteNetworkPolicyTrustedInternal,
|
||||||
|
}); err != nil {
|
||||||
|
t.Fatalf("UpdateSource(%d, preserve URL) error = %v, want nil", project.ID, err)
|
||||||
|
}
|
||||||
|
preservedSource, preservedRuntime, err := loadSourceByProject(ctx, project.ID)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("loadSourceByProject(%d) error = %v, want nil", project.ID, err)
|
||||||
|
}
|
||||||
|
if got, want := preservedSource.RemoteURL, firstURL; got != want {
|
||||||
|
t.Errorf("preserved RemoteURL = %q, want %q", got, want)
|
||||||
|
}
|
||||||
|
if got, want := preservedSource.ConfigVersion, 2; got != want {
|
||||||
|
t.Errorf("preserved source ConfigVersion = %d, want %d", got, want)
|
||||||
|
}
|
||||||
|
if got, want := preservedSource.SourceIdentity, source.SourceIdentity; got != want {
|
||||||
|
t.Errorf("preserved source identity = %q, want %q", got, want)
|
||||||
|
}
|
||||||
|
if got, want := preservedRuntime.LastSeenRevision, seenRevision; got != want {
|
||||||
|
t.Errorf("preserved LastSeenRevision = %q, want %q", got, want)
|
||||||
|
}
|
||||||
|
if got, want := preservedRuntime.SyncStatus, pagesSourceStatusUpdateAvailable; got != want {
|
||||||
|
t.Errorf("preserved runtime SyncStatus = %q, want %q", got, want)
|
||||||
|
}
|
||||||
|
if preservedRuntime.LeaseToken != "" || preservedRuntime.LeaseExpiresAt != nil {
|
||||||
|
t.Errorf("preserved runtime lease = (%q, %v), want cleared", preservedRuntime.LeaseToken, preservedRuntime.LeaseExpiresAt)
|
||||||
|
}
|
||||||
|
|
||||||
|
// Replacing only the query secret keeps the canonical identity and cursors.
|
||||||
|
queryReplacementURL := "https://artifacts.example.com/dist/site.zip?token=second-secret"
|
||||||
|
if _, err := UpdateSource(ctx, project.ID, SourceUpdateInput{
|
||||||
|
SourceType: PagesSourceTypeRemoteURL,
|
||||||
|
RemoteURLSet: true,
|
||||||
|
RemoteURL: queryReplacementURL,
|
||||||
|
RemoteNetworkPolicy: RemoteNetworkPolicyTrustedInternal,
|
||||||
|
}); err != nil {
|
||||||
|
t.Fatalf("UpdateSource(%d, query replacement) error = %v, want nil", project.ID, err)
|
||||||
|
}
|
||||||
|
querySource, queryRuntime, err := loadSourceByProject(ctx, project.ID)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("loadSourceByProject(%d) after query replacement error = %v, want nil", project.ID, err)
|
||||||
|
}
|
||||||
|
if got, want := querySource.SourceIdentity, source.SourceIdentity; got != want {
|
||||||
|
t.Errorf("query replacement identity = %q, want %q", got, want)
|
||||||
|
}
|
||||||
|
if got, want := queryRuntime.LastSeenRevision, seenRevision; got != want {
|
||||||
|
t.Errorf("query replacement LastSeenRevision = %q, want %q", got, want)
|
||||||
|
}
|
||||||
|
|
||||||
|
// Replacing the path changes identity and clears all remote cursors.
|
||||||
|
pathReplacementURL := "https://artifacts.example.com/dist/other.zip?token=third-secret"
|
||||||
|
if _, err := UpdateSource(ctx, project.ID, SourceUpdateInput{
|
||||||
|
SourceType: PagesSourceTypeRemoteURL,
|
||||||
|
RemoteURLSet: true,
|
||||||
|
RemoteURL: pathReplacementURL,
|
||||||
|
RemoteNetworkPolicy: RemoteNetworkPolicyTrustedInternal,
|
||||||
|
}); err != nil {
|
||||||
|
t.Fatalf("UpdateSource(%d, path replacement) error = %v, want nil", project.ID, err)
|
||||||
|
}
|
||||||
|
pathSource, pathRuntime, err := loadSourceByProject(ctx, project.ID)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("loadSourceByProject(%d) after path replacement error = %v, want nil", project.ID, err)
|
||||||
|
}
|
||||||
|
if pathSource.SourceIdentity == source.SourceIdentity {
|
||||||
|
t.Errorf("path replacement identity = %q, want a new identity", pathSource.SourceIdentity)
|
||||||
|
}
|
||||||
|
if pathRuntime.LastSeenRevision != "" || pathRuntime.LastAppliedRevision != "" {
|
||||||
|
t.Errorf("path replacement cursors = (%q, %q), want empty", pathRuntime.LastSeenRevision, pathRuntime.LastAppliedRevision)
|
||||||
|
}
|
||||||
|
if got, want := pathRuntime.SyncStatus, pagesSourceStatusIdle; got != want {
|
||||||
|
t.Errorf("path replacement SyncStatus = %q, want %q", got, want)
|
||||||
|
}
|
||||||
|
pathView, err := GetSource(ctx, project.ID)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("GetSource(%d) after path replacement error = %v, want nil", project.ID, err)
|
||||||
|
}
|
||||||
|
pathJSON, err := json.Marshal(pathView)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("json.Marshal(path view) error = %v, want nil", err)
|
||||||
|
}
|
||||||
|
for _, secret := range []string{"first-secret", "second-secret", "third-secret"} {
|
||||||
|
if strings.Contains(string(pathJSON), secret) {
|
||||||
|
t.Errorf("path view JSON = %s, want no secret %q", pathJSON, secret)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestRemoteSourceIdentityIgnoresQueryAndNormalizesDefaultPort(t *testing.T) {
|
||||||
|
first, err := parseRemoteSourceURL("HTTPS://Artifacts.Example.com:443/dist/../dist/site.zip?token=one")
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("parseRemoteSourceURL(first) error = %v, want nil", err)
|
||||||
|
}
|
||||||
|
second, err := parseRemoteSourceURL("https://artifacts.example.com/dist/site.zip?token=two")
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("parseRemoteSourceURL(second) error = %v, want nil", err)
|
||||||
|
}
|
||||||
|
if got, want := remoteSourceIdentity(first), remoteSourceIdentity(second); got != want {
|
||||||
|
t.Errorf("remoteSourceIdentity(first) = %q, want %q", got, want)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestDeleteSourceIsIdempotentAndKeepsDeploymentState(t *testing.T) {
|
||||||
|
ctx := setupPagesSourceTest(t)
|
||||||
|
project := mustCreatePagesSourceProject(t, ctx, "source-delete")
|
||||||
|
source, _ := mustConfigureRemoteSource(
|
||||||
|
t,
|
||||||
|
ctx,
|
||||||
|
project.ID,
|
||||||
|
"https://example.com/site.zip?token=delete-secret",
|
||||||
|
RemoteNetworkPolicyPublic,
|
||||||
|
)
|
||||||
|
deployment := &model.PagesDeployment{
|
||||||
|
ProjectID: project.ID,
|
||||||
|
DeploymentNumber: 1,
|
||||||
|
Checksum: strings.Repeat("c", 64),
|
||||||
|
Status: model.PagesDeploymentStatusActive,
|
||||||
|
CreatedBy: "user:1",
|
||||||
|
SourceType: "manual_upload",
|
||||||
|
TriggerType: "manual_upload",
|
||||||
|
}
|
||||||
|
if err := db.DB(ctx).Create(deployment).Error; err != nil {
|
||||||
|
t.Fatalf("create deployment error = %v, want nil", err)
|
||||||
|
}
|
||||||
|
if err := db.DB(ctx).Model(&model.PagesProject{}).
|
||||||
|
Where("id = ?", project.ID).
|
||||||
|
Update("active_deployment_id", deployment.ID).Error; err != nil {
|
||||||
|
t.Fatalf("set active deployment error = %v, want nil", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
for attempt := 1; attempt <= 2; attempt++ {
|
||||||
|
view, err := DeleteSource(ctx, project.ID)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("DeleteSource(%d), attempt %d error = %v, want nil", project.ID, attempt, err)
|
||||||
|
}
|
||||||
|
if got, want := view.SourceType, PagesSourceTypeManual; got != want {
|
||||||
|
t.Errorf("DeleteSource(%d), attempt %d SourceType = %q, want %q", project.ID, attempt, got, want)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
var sourceCount, runtimeCount, deploymentCount int64
|
||||||
|
if err := db.DB(ctx).Model(&model.PagesProjectSource{}).Where("id = ?", source.ID).Count(&sourceCount).Error; err != nil {
|
||||||
|
t.Fatalf("count source error = %v, want nil", err)
|
||||||
|
}
|
||||||
|
if err := db.DB(ctx).Model(&model.PagesProjectSourceRuntime{}).Where("source_id = ?", source.ID).Count(&runtimeCount).Error; err != nil {
|
||||||
|
t.Fatalf("count runtime error = %v, want nil", err)
|
||||||
|
}
|
||||||
|
if err := db.DB(ctx).Model(&model.PagesDeployment{}).Where("id = ?", deployment.ID).Count(&deploymentCount).Error; err != nil {
|
||||||
|
t.Fatalf("count deployment error = %v, want nil", err)
|
||||||
|
}
|
||||||
|
if sourceCount != 0 || runtimeCount != 0 || deploymentCount != 1 {
|
||||||
|
t.Errorf("DeleteSource counts = source:%d runtime:%d deployment:%d, want 0, 0, 1", sourceCount, runtimeCount, deploymentCount)
|
||||||
|
}
|
||||||
|
storedProject, err := model.GetPagesProjectByID(ctx, project.ID)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("GetPagesProjectByID(%d) error = %v, want nil", project.ID, err)
|
||||||
|
}
|
||||||
|
if storedProject.ActiveDeploymentID == nil || *storedProject.ActiveDeploymentID != deployment.ID {
|
||||||
|
t.Errorf("active deployment = %v, want %d", storedProject.ActiveDeploymentID, deployment.ID)
|
||||||
|
}
|
||||||
|
|
||||||
|
manual, err := GetSource(ctx, project.ID)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("GetSource(%d) after delete error = %v, want nil", project.ID, err)
|
||||||
|
}
|
||||||
|
if got, want := fmt.Sprint(manual.SourceType), PagesSourceTypeManual; got != want {
|
||||||
|
t.Errorf("GetSource(%d).SourceType = %q, want %q", project.ID, got, want)
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -6,12 +6,14 @@ package proxy_route
|
|||||||
import (
|
import (
|
||||||
"context"
|
"context"
|
||||||
"errors"
|
"errors"
|
||||||
|
"sort"
|
||||||
"strings"
|
"strings"
|
||||||
"time"
|
"time"
|
||||||
|
|
||||||
"github.com/Rain-kl/Wavelet/internal/db"
|
"github.com/Rain-kl/Wavelet/internal/db"
|
||||||
"github.com/Rain-kl/Wavelet/internal/model"
|
"github.com/Rain-kl/Wavelet/internal/model"
|
||||||
"gorm.io/gorm"
|
"gorm.io/gorm"
|
||||||
|
"gorm.io/gorm/clause"
|
||||||
)
|
)
|
||||||
|
|
||||||
// CustomHeaderInput 自定义响应头。
|
// CustomHeaderInput 自定义响应头。
|
||||||
@@ -122,6 +124,9 @@ func CreateProxyRoute(ctx context.Context, input Input) (*View, error) {
|
|||||||
return nil, err
|
return nil, err
|
||||||
}
|
}
|
||||||
if err = db.DB(ctx).Transaction(func(tx *gorm.DB) error {
|
if err = db.DB(ctx).Transaction(func(tx *gorm.DB) error {
|
||||||
|
if err := lockPagesProjectsForRouteMutation(tx, 0, route); err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
if err := tx.Create(route).Error; err != nil {
|
if err := tx.Create(route).Error; err != nil {
|
||||||
return err
|
return err
|
||||||
}
|
}
|
||||||
@@ -141,11 +146,15 @@ func UpdateProxyRoute(ctx context.Context, id uint, input Input) (*View, error)
|
|||||||
if err != nil {
|
if err != nil {
|
||||||
return nil, err
|
return nil, err
|
||||||
}
|
}
|
||||||
|
previousPagesProjectID := pagesProjectIDForRoute(route)
|
||||||
route, _, err = buildProxyRoute(ctx, route, input)
|
route, _, err = buildProxyRoute(ctx, route, input)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
return nil, err
|
return nil, err
|
||||||
}
|
}
|
||||||
if err = db.DB(ctx).Transaction(func(tx *gorm.DB) error {
|
if err = db.DB(ctx).Transaction(func(tx *gorm.DB) error {
|
||||||
|
if err := lockPagesProjectsForRouteMutation(tx, previousPagesProjectID, route); err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
if err := updateProxyRouteRecord(tx, route); err != nil {
|
if err := updateProxyRouteRecord(tx, route); err != nil {
|
||||||
return err
|
return err
|
||||||
}
|
}
|
||||||
@@ -159,6 +168,58 @@ func UpdateProxyRoute(ctx context.Context, id uint, input Input) (*View, error)
|
|||||||
return buildProxyRouteView(ctx, route)
|
return buildProxyRouteView(ctx, route)
|
||||||
}
|
}
|
||||||
|
|
||||||
|
func pagesProjectIDForRoute(route *model.ProxyRoute) uint {
|
||||||
|
if route == nil || route.UpstreamType != proxyRouteUpstreamTypePages || route.PagesProjectID == nil {
|
||||||
|
return 0
|
||||||
|
}
|
||||||
|
return *route.PagesProjectID
|
||||||
|
}
|
||||||
|
|
||||||
|
func lockPagesProjectsForRouteMutation(tx *gorm.DB, previousProjectID uint, route *model.ProxyRoute) error {
|
||||||
|
nextProjectID := pagesProjectIDForRoute(route)
|
||||||
|
var projectIDs []uint
|
||||||
|
if previousProjectID != 0 {
|
||||||
|
projectIDs = append(projectIDs, previousProjectID)
|
||||||
|
}
|
||||||
|
if nextProjectID != 0 && nextProjectID != previousProjectID {
|
||||||
|
projectIDs = append(projectIDs, nextProjectID)
|
||||||
|
}
|
||||||
|
sort.Slice(projectIDs, func(i int, j int) bool { return projectIDs[i] < projectIDs[j] })
|
||||||
|
|
||||||
|
for _, projectID := range projectIDs {
|
||||||
|
var project model.PagesProject
|
||||||
|
err := tx.Clauses(clause.Locking{Strength: "UPDATE"}).First(&project, projectID).Error
|
||||||
|
if errors.Is(err, gorm.ErrRecordNotFound) && projectID != nextProjectID {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
if errors.Is(err, gorm.ErrRecordNotFound) {
|
||||||
|
return errors.New(errProxyRoutePagesNotFound)
|
||||||
|
}
|
||||||
|
if err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
if projectID == nextProjectID {
|
||||||
|
if err := validateLockedPagesRouteProject(&project); err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func validateLockedPagesRouteProject(project *model.PagesProject) error {
|
||||||
|
if project == nil {
|
||||||
|
return errors.New(errProxyRoutePagesNotFound)
|
||||||
|
}
|
||||||
|
if !project.Enabled {
|
||||||
|
return errors.New(errProxyRoutePagesDisabled)
|
||||||
|
}
|
||||||
|
if project.ActiveDeploymentID == nil || *project.ActiveDeploymentID == 0 {
|
||||||
|
return errors.New(errProxyRoutePagesNoDeploy)
|
||||||
|
}
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
|
||||||
// DeleteProxyRoute 删除代理规则。
|
// DeleteProxyRoute 删除代理规则。
|
||||||
func DeleteProxyRoute(ctx context.Context, id uint) error {
|
func DeleteProxyRoute(ctx context.Context, id uint) error {
|
||||||
if _, err := model.GetProxyRouteByID(ctx, id); err != nil {
|
if _, err := model.GetProxyRouteByID(ctx, id); err != nil {
|
||||||
|
|||||||
@@ -19,7 +19,14 @@ func setupProxyRouteTestDB(t *testing.T) func() {
|
|||||||
t.Helper()
|
t.Helper()
|
||||||
sqliteDB, err := gorm.Open(sqlite.Open(":memory:"), &gorm.Config{DisableForeignKeyConstraintWhenMigrating: true})
|
sqliteDB, err := gorm.Open(sqlite.Open(":memory:"), &gorm.Config{DisableForeignKeyConstraintWhenMigrating: true})
|
||||||
require.NoError(t, err)
|
require.NoError(t, err)
|
||||||
require.NoError(t, sqliteDB.AutoMigrate(&model.ProxyRoute{}, &model.Origin{}, &model.Zone{}, &model.ZoneDomain{}, &model.TLSCertificate{}))
|
require.NoError(t, sqliteDB.AutoMigrate(
|
||||||
|
&model.ProxyRoute{},
|
||||||
|
&model.Origin{},
|
||||||
|
&model.Zone{},
|
||||||
|
&model.ZoneDomain{},
|
||||||
|
&model.TLSCertificate{},
|
||||||
|
&model.PagesProject{},
|
||||||
|
))
|
||||||
db.SetDB(sqliteDB)
|
db.SetDB(sqliteDB)
|
||||||
return func() { db.SetDB(nil) }
|
return func() { db.SetDB(nil) }
|
||||||
}
|
}
|
||||||
@@ -81,6 +88,52 @@ func TestCreateProxyRouteHTTPSRequiresCoveringCertificate(t *testing.T) {
|
|||||||
require.EqualError(t, err, errProxyRouteCertRequired)
|
require.EqualError(t, err, errProxyRouteCertRequired)
|
||||||
}
|
}
|
||||||
|
|
||||||
|
func TestPagesRouteLocksAndRevalidatesTargetProject(t *testing.T) {
|
||||||
|
cleanup := setupProxyRouteTestDB(t)
|
||||||
|
defer cleanup()
|
||||||
|
ctx := context.Background()
|
||||||
|
domain := createZoneDomain(t, ctx, "pages.example.com", nil)
|
||||||
|
activeDeploymentID := uint(99)
|
||||||
|
project := &model.PagesProject{
|
||||||
|
Name: "Pages Site",
|
||||||
|
Slug: "pages-site",
|
||||||
|
Enabled: true,
|
||||||
|
ActiveDeploymentID: &activeDeploymentID,
|
||||||
|
}
|
||||||
|
require.NoError(t, db.DB(ctx).Create(project).Error)
|
||||||
|
|
||||||
|
view, err := CreateProxyRoute(ctx, Input{
|
||||||
|
SiteName: "pages",
|
||||||
|
ZoneDomainIDs: []uint{domain.ID},
|
||||||
|
UpstreamType: proxyRouteUpstreamTypePages,
|
||||||
|
PagesProjectID: &project.ID,
|
||||||
|
Enabled: true,
|
||||||
|
})
|
||||||
|
require.NoError(t, err)
|
||||||
|
require.NotNil(t, view.PagesProjectID)
|
||||||
|
assert.Equal(t, project.ID, *view.PagesProjectID)
|
||||||
|
|
||||||
|
require.NoError(t, db.DB(ctx).Delete(&model.PagesProject{}, project.ID).Error)
|
||||||
|
route := &model.ProxyRoute{UpstreamType: proxyRouteUpstreamTypePages, PagesProjectID: &project.ID}
|
||||||
|
err = db.DB(ctx).Transaction(func(tx *gorm.DB) error {
|
||||||
|
return lockPagesProjectsForRouteMutation(tx, 0, route)
|
||||||
|
})
|
||||||
|
require.EqualError(t, err, errProxyRoutePagesNotFound)
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestRouteCanMoveAwayFromAlreadyMissingPagesProject(t *testing.T) {
|
||||||
|
cleanup := setupProxyRouteTestDB(t)
|
||||||
|
defer cleanup()
|
||||||
|
ctx := context.Background()
|
||||||
|
missingProjectID := uint(404)
|
||||||
|
route := &model.ProxyRoute{UpstreamType: "direct"}
|
||||||
|
|
||||||
|
err := db.DB(ctx).Transaction(func(tx *gorm.DB) error {
|
||||||
|
return lockPagesProjectsForRouteMutation(tx, missingProjectID, route)
|
||||||
|
})
|
||||||
|
require.NoError(t, err)
|
||||||
|
}
|
||||||
|
|
||||||
func TestNormalizeCachePolicyDefaultsAndLegacy(t *testing.T) {
|
func TestNormalizeCachePolicyDefaultsAndLegacy(t *testing.T) {
|
||||||
assert.Equal(t, "", normalizeCachePolicy(false, "static"))
|
assert.Equal(t, "", normalizeCachePolicy(false, "static"))
|
||||||
// Empty/url on write = legacy all (compat); UI sends static explicitly for new default.
|
// Empty/url on write = legacy all (compat); UI sends static explicitly for new default.
|
||||||
|
|||||||
@@ -8,6 +8,7 @@ import (
|
|||||||
"github.com/Rain-kl/Wavelet/internal/apps/upload/filesrv"
|
"github.com/Rain-kl/Wavelet/internal/apps/upload/filesrv"
|
||||||
"github.com/Rain-kl/Wavelet/internal/apps/upload/handler"
|
"github.com/Rain-kl/Wavelet/internal/apps/upload/handler"
|
||||||
"github.com/Rain-kl/Wavelet/internal/apps/upload/ingest"
|
"github.com/Rain-kl/Wavelet/internal/apps/upload/ingest"
|
||||||
|
"github.com/Rain-kl/Wavelet/internal/apps/upload/shared"
|
||||||
uploadstats "github.com/Rain-kl/Wavelet/internal/apps/upload/stats"
|
uploadstats "github.com/Rain-kl/Wavelet/internal/apps/upload/stats"
|
||||||
uploadtask "github.com/Rain-kl/Wavelet/internal/apps/upload/task"
|
uploadtask "github.com/Rain-kl/Wavelet/internal/apps/upload/task"
|
||||||
"github.com/Rain-kl/Wavelet/internal/apps/upload/util"
|
"github.com/Rain-kl/Wavelet/internal/apps/upload/util"
|
||||||
@@ -31,15 +32,17 @@ var (
|
|||||||
|
|
||||||
// Programmatic ingest API
|
// Programmatic ingest API
|
||||||
var (
|
var (
|
||||||
Ingest = ingest.Ingest
|
Ingest = ingest.Ingest
|
||||||
Remove = ingest.Remove
|
Remove = ingest.Remove
|
||||||
RemoveOwned = ingest.RemoveOwned
|
RemoveOwned = ingest.RemoveOwned
|
||||||
FindByHash = ingest.FindByHash
|
RemoveLockedTx = ingest.RemoveLockedTx
|
||||||
GetActiveUpload = ingest.GetActive
|
InvalidateUploadMetaCache = ingest.InvalidateUploadMetaCache
|
||||||
OpenStoredUpload = ingest.OpenActiveObject
|
FindByHash = ingest.FindByHash
|
||||||
ActiveUploadHash = ingest.ActiveHash
|
GetActiveUpload = ingest.GetActive
|
||||||
ResolveLocalFile = ingest.ResolveLocalFile
|
OpenStoredUpload = ingest.OpenActiveObject
|
||||||
IngestFromLocalPath = ingest.FromLocalPath
|
ActiveUploadHash = ingest.ActiveHash
|
||||||
|
ResolveLocalFile = ingest.ResolveLocalFile
|
||||||
|
IngestFromLocalPath = ingest.FromLocalPath
|
||||||
)
|
)
|
||||||
|
|
||||||
type (
|
type (
|
||||||
@@ -54,6 +57,8 @@ const (
|
|||||||
PolicyCreate = ingest.PolicyCreate
|
PolicyCreate = ingest.PolicyCreate
|
||||||
PolicyDedupNewRecord = ingest.PolicyDedupNewRecord
|
PolicyDedupNewRecord = ingest.PolicyDedupNewRecord
|
||||||
PolicyResolveExisting = ingest.PolicyResolveExisting
|
PolicyResolveExisting = ingest.PolicyResolveExisting
|
||||||
|
// ReservedPagesDeploymentType is managed exclusively by the Pages domain.
|
||||||
|
ReservedPagesDeploymentType = shared.ReservedPagesDeploymentType
|
||||||
)
|
)
|
||||||
|
|
||||||
type (
|
type (
|
||||||
@@ -69,6 +74,7 @@ type (
|
|||||||
var (
|
var (
|
||||||
ErrIngestForbidden = ingest.ErrForbidden
|
ErrIngestForbidden = ingest.ErrForbidden
|
||||||
ErrIngestStorageReadOnly = ingest.ErrStorageReadOnly
|
ErrIngestStorageReadOnly = ingest.ErrStorageReadOnly
|
||||||
|
ErrReservedUploadType = ingest.ErrReservedUploadType
|
||||||
)
|
)
|
||||||
|
|
||||||
// Cache management
|
// Cache management
|
||||||
|
|||||||
@@ -4,6 +4,7 @@
|
|||||||
package handler
|
package handler
|
||||||
|
|
||||||
import (
|
import (
|
||||||
|
"errors"
|
||||||
"net/http"
|
"net/http"
|
||||||
"strconv"
|
"strconv"
|
||||||
|
|
||||||
@@ -96,6 +97,7 @@ func ListFiles(c *gin.Context) {
|
|||||||
// @Success 200 {object} response.Any "删除成功"
|
// @Success 200 {object} response.Any "删除成功"
|
||||||
// @Failure 403 {object} response.Any "无权操作"
|
// @Failure 403 {object} response.Any "无权操作"
|
||||||
// @Failure 404 {object} response.Any "文件不存在"
|
// @Failure 404 {object} response.Any "文件不存在"
|
||||||
|
// @Failure 409 {object} response.Any "系统保留类型或存储只读"
|
||||||
// @Router /api/v1/admin/uploads/{id} [delete]
|
// @Router /api/v1/admin/uploads/{id} [delete]
|
||||||
func DeleteFile(c *gin.Context) {
|
func DeleteFile(c *gin.Context) {
|
||||||
ctx := c.Request.Context()
|
ctx := c.Request.Context()
|
||||||
@@ -111,6 +113,10 @@ func DeleteFile(c *gin.Context) {
|
|||||||
}
|
}
|
||||||
|
|
||||||
if _, err := softDeleteUpload(ctx, uploadID); err != nil {
|
if _, err := softDeleteUpload(ctx, uploadID); err != nil {
|
||||||
|
if errors.Is(err, ingest.ErrReservedUploadType) {
|
||||||
|
response.AbortConflict(c, shared.ErrReservedUploadType)
|
||||||
|
return
|
||||||
|
}
|
||||||
if isRecordNotFound(err) {
|
if isRecordNotFound(err) {
|
||||||
response.AbortNotFound(c, "文件记录未找到")
|
response.AbortNotFound(c, "文件记录未找到")
|
||||||
return
|
return
|
||||||
@@ -216,6 +222,7 @@ func ListMyFiles(c *gin.Context) {
|
|||||||
// @Success 200 {object} response.Any "删除成功"
|
// @Success 200 {object} response.Any "删除成功"
|
||||||
// @Failure 403 {object} response.Any "无权操作"
|
// @Failure 403 {object} response.Any "无权操作"
|
||||||
// @Failure 404 {object} response.Any "文件不存在"
|
// @Failure 404 {object} response.Any "文件不存在"
|
||||||
|
// @Failure 409 {object} response.Any "系统保留类型或存储只读"
|
||||||
// @Router /api/v1/upload/{id} [delete]
|
// @Router /api/v1/upload/{id} [delete]
|
||||||
func DeleteMyFile(c *gin.Context) {
|
func DeleteMyFile(c *gin.Context) {
|
||||||
currUser, _ := oauth.GetFromContext[*model.User](c, oauth.UserObjKey)
|
currUser, _ := oauth.GetFromContext[*model.User](c, oauth.UserObjKey)
|
||||||
@@ -232,11 +239,15 @@ func DeleteMyFile(c *gin.Context) {
|
|||||||
}
|
}
|
||||||
|
|
||||||
if _, err := softDeleteOwnedUpload(ctx, currUser.ID, uploadID); err != nil {
|
if _, err := softDeleteOwnedUpload(ctx, currUser.ID, uploadID); err != nil {
|
||||||
|
if errors.Is(err, ingest.ErrReservedUploadType) {
|
||||||
|
response.AbortConflict(c, shared.ErrReservedUploadType)
|
||||||
|
return
|
||||||
|
}
|
||||||
if isRecordNotFound(err) {
|
if isRecordNotFound(err) {
|
||||||
response.AbortNotFound(c, "文件记录未找到")
|
response.AbortNotFound(c, "文件记录未找到")
|
||||||
return
|
return
|
||||||
}
|
}
|
||||||
if err == ingest.ErrForbidden {
|
if errors.Is(err, ingest.ErrForbidden) {
|
||||||
response.AbortForbidden(c, "无权操作")
|
response.AbortForbidden(c, "无权操作")
|
||||||
return
|
return
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -53,6 +53,7 @@ type batchDownloadRequest struct {
|
|||||||
// @Success 200 {object} response.Any{data=model.Upload} "上传成功"
|
// @Success 200 {object} response.Any{data=model.Upload} "上传成功"
|
||||||
// @Failure 400 {object} response.Any "请求参数错误或文件受限"
|
// @Failure 400 {object} response.Any "请求参数错误或文件受限"
|
||||||
// @Failure 401 {object} response.Any "未登录"
|
// @Failure 401 {object} response.Any "未登录"
|
||||||
|
// @Failure 409 {object} response.Any "系统保留类型或存储只读"
|
||||||
// @Failure 500 {object} response.Any "内部错误"
|
// @Failure 500 {object} response.Any "内部错误"
|
||||||
// @Router /api/v1/upload [post]
|
// @Router /api/v1/upload [post]
|
||||||
//
|
//
|
||||||
@@ -107,6 +108,10 @@ func UploadFile(c *gin.Context) {
|
|||||||
}
|
}
|
||||||
|
|
||||||
uploadType := c.DefaultPostForm("type", "generic")
|
uploadType := c.DefaultPostForm("type", "generic")
|
||||||
|
if uploadType == shared.ReservedPagesDeploymentType {
|
||||||
|
response.AbortConflict(c, shared.ErrReservedUploadType)
|
||||||
|
return
|
||||||
|
}
|
||||||
|
|
||||||
accessMode, errMsg := resolveUploadAccessMode(c, uploadType)
|
accessMode, errMsg := resolveUploadAccessMode(c, uploadType)
|
||||||
if errMsg != "" {
|
if errMsg != "" {
|
||||||
|
|||||||
@@ -226,6 +226,42 @@ func TestUploadFile(t *testing.T) {
|
|||||||
}
|
}
|
||||||
})
|
})
|
||||||
|
|
||||||
|
t.Run("upload rejects Pages reserved type", func(t *testing.T) {
|
||||||
|
putCountBefore := putCount
|
||||||
|
imgContent := []byte("\x89PNG\r\n\x1a\n\x00\x00\x00\rIHDR\x00\x00\x00\x01\x00\x00\x00\x01")
|
||||||
|
contentType, body := createMultipartRequest(t, "file", "pages.png", imgContent, map[string]string{
|
||||||
|
"type": shared.ReservedPagesDeploymentType,
|
||||||
|
})
|
||||||
|
req, _ := http.NewRequest("POST", "/api/v1/upload", body)
|
||||||
|
req.Header.Set("Content-Type", contentType)
|
||||||
|
|
||||||
|
w := httptest.NewRecorder()
|
||||||
|
router.ServeHTTP(w, req)
|
||||||
|
|
||||||
|
if w.Code != http.StatusConflict {
|
||||||
|
t.Fatalf("expected status 409, got %d. Body: %s", w.Code, w.Body.String())
|
||||||
|
}
|
||||||
|
var resp testResponse
|
||||||
|
if err := json.Unmarshal(w.Body.Bytes(), &resp); err != nil {
|
||||||
|
t.Fatalf("unmarshal reserved type response: %v", err)
|
||||||
|
}
|
||||||
|
if resp.ErrorMsg != shared.ErrReservedUploadType {
|
||||||
|
t.Fatalf("reserved type error = %q, want %q", resp.ErrorMsg, shared.ErrReservedUploadType)
|
||||||
|
}
|
||||||
|
if putCount != putCountBefore {
|
||||||
|
t.Fatalf("reserved upload wrote storage object: put count %d -> %d", putCountBefore, putCount)
|
||||||
|
}
|
||||||
|
var count int64
|
||||||
|
if err := dbConn.Model(&model.Upload{}).
|
||||||
|
Where("type = ?", shared.ReservedPagesDeploymentType).
|
||||||
|
Count(&count).Error; err != nil {
|
||||||
|
t.Fatalf("count reserved uploads: %v", err)
|
||||||
|
}
|
||||||
|
if count != 0 {
|
||||||
|
t.Fatalf("reserved upload record count = %d, want 0", count)
|
||||||
|
}
|
||||||
|
})
|
||||||
|
|
||||||
t.Run("instant upload deduplication (秒传)", func(t *testing.T) {
|
t.Run("instant upload deduplication (秒传)", func(t *testing.T) {
|
||||||
putCount = 0
|
putCount = 0
|
||||||
imgContent := []byte("\x89PNG\r\n\x1a\n\x00\x00\x00\rIHDR\x00\x00\x00\x01\x00\x00\x00\x01")
|
imgContent := []byte("\x89PNG\r\n\x1a\n\x00\x00\x00\rIHDR\x00\x00\x00\x01\x00\x00\x00\x01")
|
||||||
@@ -993,6 +1029,62 @@ func TestUserUploadManagement(t *testing.T) {
|
|||||||
})
|
})
|
||||||
}
|
}
|
||||||
|
|
||||||
|
func TestDeleteReservedUploadType(t *testing.T) {
|
||||||
|
dbConn, _, cleanup := testhelper.SetupTestEnvironment(t)
|
||||||
|
defer cleanup()
|
||||||
|
|
||||||
|
authUser := &model.User{ID: 1001, Username: "reserved_owner"}
|
||||||
|
router := setupTestRouter(authUser)
|
||||||
|
reserved := model.Upload{
|
||||||
|
ID: 4101,
|
||||||
|
UserID: authUser.ID,
|
||||||
|
FileName: "pages.zip",
|
||||||
|
FilePath: "uploads/pages.zip",
|
||||||
|
FileSize: 128,
|
||||||
|
MimeType: "application/zip",
|
||||||
|
Extension: "zip",
|
||||||
|
Hash: "pages-reserved-hash",
|
||||||
|
Type: shared.ReservedPagesDeploymentType,
|
||||||
|
Status: model.UploadStatusUsed,
|
||||||
|
CreatedAt: time.Now(),
|
||||||
|
}
|
||||||
|
if err := dbConn.Create(&reserved).Error; err != nil {
|
||||||
|
t.Fatalf("seed reserved upload: %v", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
for _, tc := range []struct {
|
||||||
|
name string
|
||||||
|
path string
|
||||||
|
}{
|
||||||
|
{name: "admin delete", path: "/api/v1/admin/uploads/4101"},
|
||||||
|
{name: "owner delete", path: "/api/v1/upload/4101"},
|
||||||
|
} {
|
||||||
|
t.Run(tc.name, func(t *testing.T) {
|
||||||
|
req, _ := http.NewRequest(http.MethodDelete, tc.path, nil)
|
||||||
|
w := httptest.NewRecorder()
|
||||||
|
router.ServeHTTP(w, req)
|
||||||
|
if w.Code != http.StatusConflict {
|
||||||
|
t.Fatalf("expected status 409, got %d. Body: %s", w.Code, w.Body.String())
|
||||||
|
}
|
||||||
|
var resp testResponse
|
||||||
|
if err := json.Unmarshal(w.Body.Bytes(), &resp); err != nil {
|
||||||
|
t.Fatalf("unmarshal delete response: %v", err)
|
||||||
|
}
|
||||||
|
if resp.ErrorMsg != shared.ErrReservedUploadType {
|
||||||
|
t.Fatalf("reserved delete error = %q, want %q", resp.ErrorMsg, shared.ErrReservedUploadType)
|
||||||
|
}
|
||||||
|
})
|
||||||
|
}
|
||||||
|
|
||||||
|
var persisted model.Upload
|
||||||
|
if err := dbConn.First(&persisted, reserved.ID).Error; err != nil {
|
||||||
|
t.Fatalf("reload reserved upload: %v", err)
|
||||||
|
}
|
||||||
|
if persisted.Status != model.UploadStatusUsed {
|
||||||
|
t.Fatalf("reserved upload status = %s, want used", persisted.Status)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
func configureLocalStorageRoot(t *testing.T, dbConn *gorm.DB, tempDir string) {
|
func configureLocalStorageRoot(t *testing.T, dbConn *gorm.DB, tempDir string) {
|
||||||
var sc model.SystemConfig
|
var sc model.SystemConfig
|
||||||
if err := dbConn.Where("key = ?", model.ConfigKeyStorageConfig).First(&sc).Error; err != nil {
|
if err := dbConn.Where("key = ?", model.ConfigKeyStorageConfig).First(&sc).Error; err != nil {
|
||||||
|
|||||||
@@ -12,5 +12,8 @@ import (
|
|||||||
// ErrForbidden indicates the caller is not allowed to mutate the upload record.
|
// ErrForbidden indicates the caller is not allowed to mutate the upload record.
|
||||||
var ErrForbidden = errors.New("upload forbidden")
|
var ErrForbidden = errors.New("upload forbidden")
|
||||||
|
|
||||||
|
// ErrReservedUploadType indicates that a generic mutation targeted a domain-reserved upload type.
|
||||||
|
var ErrReservedUploadType = errors.New(shared.ErrReservedUploadType)
|
||||||
|
|
||||||
// ErrStorageReadOnly indicates the storage backend is in migration read-only mode.
|
// ErrStorageReadOnly indicates the storage backend is in migration read-only mode.
|
||||||
var ErrStorageReadOnly = errors.New(shared.ErrStorageReadOnly)
|
var ErrStorageReadOnly = errors.New(shared.ErrStorageReadOnly)
|
||||||
|
|||||||
@@ -100,13 +100,10 @@ func storeObject(ctx context.Context, objectKey string, reader io.Reader, size i
|
|||||||
return result.Key, nil
|
return result.Key, nil
|
||||||
}
|
}
|
||||||
|
|
||||||
func persistUploadRecord(ctx context.Context, upload *model.Upload, objectKey string) error {
|
func persistUploadRecord(ctx context.Context, upload *model.Upload, objectKey string, storedByRequest bool) error {
|
||||||
if err := createUploadWithStats(ctx, upload); err != nil {
|
if err := createUploadWithStats(ctx, upload); err != nil {
|
||||||
_, backend, backendErr := storage.Active(ctx)
|
if storedByRequest {
|
||||||
if backendErr == nil {
|
cleanupUnpersistedObject(ctx, objectKey)
|
||||||
if deleteErr := backend.Delete(ctx, objectKey); deleteErr != nil {
|
|
||||||
logger.WarnF(ctx, "清理未写入数据库的上传对象失败: %v", deleteErr)
|
|
||||||
}
|
|
||||||
}
|
}
|
||||||
return err
|
return err
|
||||||
}
|
}
|
||||||
@@ -114,6 +111,16 @@ func persistUploadRecord(ctx context.Context, upload *model.Upload, objectKey st
|
|||||||
return nil
|
return nil
|
||||||
}
|
}
|
||||||
|
|
||||||
|
func cleanupUnpersistedObject(ctx context.Context, objectKey string) {
|
||||||
|
_, backend, err := storage.Active(ctx)
|
||||||
|
if err != nil {
|
||||||
|
return
|
||||||
|
}
|
||||||
|
if err := backend.Delete(ctx, objectKey); err != nil {
|
||||||
|
logger.WarnF(ctx, "清理未写入数据库的上传对象失败: %v", err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
func createUploadWithStats(ctx context.Context, upload *model.Upload) error {
|
func createUploadWithStats(ctx context.Context, upload *model.Upload) error {
|
||||||
return db.DB(ctx).Transaction(func(tx *gorm.DB) error {
|
return db.DB(ctx).Transaction(func(tx *gorm.DB) error {
|
||||||
if err := repository.CreateUploadTx(tx, upload); err != nil {
|
if err := repository.CreateUploadTx(tx, upload); err != nil {
|
||||||
@@ -125,6 +132,8 @@ func createUploadWithStats(ctx context.Context, upload *model.Upload) error {
|
|||||||
|
|
||||||
func createDedupRecord(ctx context.Context, existing model.Upload, req Request) (Result, error) {
|
func createDedupRecord(ctx context.Context, existing model.Upload, req Request) (Result, error) {
|
||||||
accessMode := resolveAccessMode(req.Type, req.AccessMode)
|
accessMode := resolveAccessMode(req.Type, req.AccessMode)
|
||||||
|
metadata := req.Metadata
|
||||||
|
metadata.Bucket = existing.Metadata.Bucket
|
||||||
newUpload := model.Upload{
|
newUpload := model.Upload{
|
||||||
ID: idgen.NextUint64ID(),
|
ID: idgen.NextUint64ID(),
|
||||||
UserID: req.UserID,
|
UserID: req.UserID,
|
||||||
@@ -137,9 +146,9 @@ func createDedupRecord(ctx context.Context, existing model.Upload, req Request)
|
|||||||
Type: req.Type,
|
Type: req.Type,
|
||||||
Status: req.Status,
|
Status: req.Status,
|
||||||
AccessMode: accessMode,
|
AccessMode: accessMode,
|
||||||
Metadata: existing.Metadata,
|
Metadata: metadata,
|
||||||
}
|
}
|
||||||
if err := persistUploadRecord(ctx, &newUpload, existing.FilePath); err != nil {
|
if err := persistUploadRecord(ctx, &newUpload, existing.FilePath, false); err != nil {
|
||||||
return Result{}, err
|
return Result{}, err
|
||||||
}
|
}
|
||||||
logger.InfoF(ctx, "文件触发秒传成功! ID: %d, Path: %s", newUpload.ID, existing.FilePath)
|
logger.InfoF(ctx, "文件触发秒传成功! ID: %d, Path: %s", newUpload.ID, existing.FilePath)
|
||||||
@@ -186,7 +195,7 @@ func createNewUpload(ctx context.Context, req Request) (Result, error) {
|
|||||||
AccessMode: accessMode,
|
AccessMode: accessMode,
|
||||||
Metadata: req.Metadata,
|
Metadata: req.Metadata,
|
||||||
}
|
}
|
||||||
if err := persistUploadRecord(ctx, &upload, storedKey); err != nil {
|
if err := persistUploadRecord(ctx, &upload, storedKey, true); err != nil {
|
||||||
return Result{}, err
|
return Result{}, err
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|||||||
@@ -8,15 +8,20 @@ import (
|
|||||||
"context"
|
"context"
|
||||||
"crypto/sha256"
|
"crypto/sha256"
|
||||||
"encoding/hex"
|
"encoding/hex"
|
||||||
|
"errors"
|
||||||
"io"
|
"io"
|
||||||
"os"
|
"os"
|
||||||
|
"sync"
|
||||||
"testing"
|
"testing"
|
||||||
"time"
|
"time"
|
||||||
|
|
||||||
|
uploadcache "github.com/Rain-kl/Wavelet/internal/apps/upload/cache"
|
||||||
|
"github.com/Rain-kl/Wavelet/internal/apps/upload/shared"
|
||||||
"github.com/Rain-kl/Wavelet/internal/db"
|
"github.com/Rain-kl/Wavelet/internal/db"
|
||||||
"github.com/Rain-kl/Wavelet/internal/model"
|
"github.com/Rain-kl/Wavelet/internal/model"
|
||||||
"github.com/Rain-kl/Wavelet/internal/storage"
|
"github.com/Rain-kl/Wavelet/internal/storage"
|
||||||
"github.com/Rain-kl/Wavelet/internal/testhelper"
|
"github.com/Rain-kl/Wavelet/internal/testhelper"
|
||||||
|
"gorm.io/gorm"
|
||||||
)
|
)
|
||||||
|
|
||||||
func TestIngestPolicyCreateIncrementsStats(t *testing.T) {
|
func TestIngestPolicyCreateIncrementsStats(t *testing.T) {
|
||||||
@@ -141,7 +146,11 @@ func TestIngestPolicyDedupNewRecordCreatesSecondRecord(t *testing.T) {
|
|||||||
Extension: "png",
|
Extension: "png",
|
||||||
Hash: hashStr,
|
Hash: hashStr,
|
||||||
Type: "avatar",
|
Type: "avatar",
|
||||||
Policy: PolicyDedupNewRecord,
|
Metadata: model.UploadMetadata{
|
||||||
|
UserAgent: "first-agent",
|
||||||
|
Extra: map[string]any{"record": "first"},
|
||||||
|
},
|
||||||
|
Policy: PolicyDedupNewRecord,
|
||||||
})
|
})
|
||||||
if err != nil {
|
if err != nil {
|
||||||
t.Fatalf("first Ingest returned error: %v", err)
|
t.Fatalf("first Ingest returned error: %v", err)
|
||||||
@@ -149,6 +158,10 @@ func TestIngestPolicyDedupNewRecordCreatesSecondRecord(t *testing.T) {
|
|||||||
if putCount != 1 {
|
if putCount != 1 {
|
||||||
t.Fatalf("putCount after first ingest = %d, want 1", putCount)
|
t.Fatalf("putCount after first ingest = %d, want 1", putCount)
|
||||||
}
|
}
|
||||||
|
first.Upload.Metadata.Bucket = "shared-bucket"
|
||||||
|
if err := dbConn.Save(&first.Upload).Error; err != nil {
|
||||||
|
t.Fatalf("update first upload metadata failed: %v", err)
|
||||||
|
}
|
||||||
|
|
||||||
second, err := Ingest(ctx, Request{
|
second, err := Ingest(ctx, Request{
|
||||||
UserID: 1002,
|
UserID: 1002,
|
||||||
@@ -159,7 +172,12 @@ func TestIngestPolicyDedupNewRecordCreatesSecondRecord(t *testing.T) {
|
|||||||
Extension: "png",
|
Extension: "png",
|
||||||
Hash: hashStr,
|
Hash: hashStr,
|
||||||
Type: "avatar",
|
Type: "avatar",
|
||||||
Policy: PolicyDedupNewRecord,
|
Metadata: model.UploadMetadata{
|
||||||
|
UserAgent: "second-agent",
|
||||||
|
Bucket: "caller-bucket-must-not-survive",
|
||||||
|
Extra: map[string]any{"record": "second"},
|
||||||
|
},
|
||||||
|
Policy: PolicyDedupNewRecord,
|
||||||
})
|
})
|
||||||
if err != nil {
|
if err != nil {
|
||||||
t.Fatalf("second Ingest returned error: %v", err)
|
t.Fatalf("second Ingest returned error: %v", err)
|
||||||
@@ -173,6 +191,15 @@ func TestIngestPolicyDedupNewRecordCreatesSecondRecord(t *testing.T) {
|
|||||||
if first.Upload.ID == second.Upload.ID {
|
if first.Upload.ID == second.Upload.ID {
|
||||||
t.Fatal("dedup records should have unique IDs")
|
t.Fatal("dedup records should have unique IDs")
|
||||||
}
|
}
|
||||||
|
if second.Upload.Metadata.Bucket != "shared-bucket" {
|
||||||
|
t.Fatalf("dedup bucket = %q, want inherited shared-bucket", second.Upload.Metadata.Bucket)
|
||||||
|
}
|
||||||
|
if second.Upload.Metadata.UserAgent != "second-agent" {
|
||||||
|
t.Fatalf("dedup user agent = %q, want caller metadata", second.Upload.Metadata.UserAgent)
|
||||||
|
}
|
||||||
|
if second.Upload.Metadata.Extra["record"] != "second" {
|
||||||
|
t.Fatalf("dedup extra metadata = %#v, want caller metadata", second.Upload.Metadata.Extra)
|
||||||
|
}
|
||||||
|
|
||||||
var count int64
|
var count int64
|
||||||
if err := dbConn.Model(&model.Upload{}).Where("hash = ?", hashStr).Count(&count).Error; err != nil {
|
if err := dbConn.Model(&model.Upload{}).Where("hash = ?", hashStr).Count(&count).Error; err != nil {
|
||||||
@@ -183,6 +210,77 @@ func TestIngestPolicyDedupNewRecordCreatesSecondRecord(t *testing.T) {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
func TestDedupRecordFailureDoesNotDeleteSharedObject(t *testing.T) {
|
||||||
|
dbConn, _, cleanup := testhelper.SetupTestEnvironment(t)
|
||||||
|
defer cleanup()
|
||||||
|
ctx := context.Background()
|
||||||
|
|
||||||
|
content := []byte("\x89PNG\r\n\x1a\nshared-object")
|
||||||
|
hash := sha256.Sum256(content)
|
||||||
|
hashStr := hex.EncodeToString(hash[:])
|
||||||
|
deleteCount := 0
|
||||||
|
restoreStorage, disableStorage := setupMockStorageWithDeleteCount(t, nil, &deleteCount)
|
||||||
|
defer restoreStorage()
|
||||||
|
defer disableStorage()
|
||||||
|
|
||||||
|
first, err := Ingest(ctx, Request{
|
||||||
|
UserID: 1001,
|
||||||
|
Reader: bytes.NewReader(content),
|
||||||
|
Size: int64(len(content)),
|
||||||
|
FileName: "shared.png",
|
||||||
|
MimeType: "image/png",
|
||||||
|
Extension: "png",
|
||||||
|
Hash: hashStr,
|
||||||
|
Type: "avatar",
|
||||||
|
Policy: PolicyDedupNewRecord,
|
||||||
|
})
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("first Ingest returned error: %v", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
const callbackName = "test:reject_dedup_upload_record"
|
||||||
|
if err := dbConn.Callback().Create().Before("gorm:create").Register(callbackName, func(tx *gorm.DB) {
|
||||||
|
upload, ok := tx.Statement.Dest.(*model.Upload)
|
||||||
|
if ok && upload.FileName == "dedup-fail.png" {
|
||||||
|
tx.AddError(errors.New("injected upload create failure"))
|
||||||
|
}
|
||||||
|
}); err != nil {
|
||||||
|
t.Fatalf("register create failure callback: %v", err)
|
||||||
|
}
|
||||||
|
defer func() { _ = dbConn.Callback().Create().Remove(callbackName) }()
|
||||||
|
|
||||||
|
_, err = Ingest(ctx, Request{
|
||||||
|
UserID: 1002,
|
||||||
|
Reader: bytes.NewReader(content),
|
||||||
|
Size: int64(len(content)),
|
||||||
|
FileName: "dedup-fail.png",
|
||||||
|
MimeType: "image/png",
|
||||||
|
Extension: "png",
|
||||||
|
Hash: hashStr,
|
||||||
|
Type: "avatar",
|
||||||
|
Metadata: model.UploadMetadata{
|
||||||
|
Extra: map[string]any{"record": "dedup-failure"},
|
||||||
|
},
|
||||||
|
Policy: PolicyDedupNewRecord,
|
||||||
|
})
|
||||||
|
if err == nil {
|
||||||
|
t.Fatal("dedup Ingest expected injected persistence error")
|
||||||
|
}
|
||||||
|
if deleteCount != 0 {
|
||||||
|
t.Fatalf("shared object delete count = %d, want 0", deleteCount)
|
||||||
|
}
|
||||||
|
|
||||||
|
_, backend, err := storage.Active(ctx)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("load active storage: %v", err)
|
||||||
|
}
|
||||||
|
obj, err := backend.Get(ctx, first.Upload.FilePath)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("shared object became unreadable after dedup failure: %v", err)
|
||||||
|
}
|
||||||
|
_ = obj.Body.Close()
|
||||||
|
}
|
||||||
|
|
||||||
func TestCreateUploadWithStatsRollsBackOnCreateFailure(t *testing.T) {
|
func TestCreateUploadWithStatsRollsBackOnCreateFailure(t *testing.T) {
|
||||||
dbConn, _, cleanup := testhelper.SetupTestEnvironment(t)
|
dbConn, _, cleanup := testhelper.SetupTestEnvironment(t)
|
||||||
defer cleanup()
|
defer cleanup()
|
||||||
@@ -259,6 +357,18 @@ func TestRemoveDecrementsStats(t *testing.T) {
|
|||||||
if _, err := Remove(ctx, result.Upload.ID); err != nil {
|
if _, err := Remove(ctx, result.Upload.ID); err != nil {
|
||||||
t.Fatalf("Remove(%d) returned error: %v", result.Upload.ID, err)
|
t.Fatalf("Remove(%d) returned error: %v", result.Upload.ID, err)
|
||||||
}
|
}
|
||||||
|
stale := result.Upload
|
||||||
|
uploadcache.SetUploadMetaCache(ctx, &stale)
|
||||||
|
removedAgain, err := Remove(ctx, result.Upload.ID)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("second Remove(%d) returned error: %v", result.Upload.ID, err)
|
||||||
|
}
|
||||||
|
if removedAgain.Status != model.UploadStatusDeleted {
|
||||||
|
t.Fatalf("second Remove status = %s, want deleted", removedAgain.Status)
|
||||||
|
}
|
||||||
|
if _, err := uploadcache.GetUploadByID(ctx, result.Upload.ID); !errors.Is(err, gorm.ErrRecordNotFound) {
|
||||||
|
t.Fatalf("cache lookup after idempotent Remove error = %v, want record not found", err)
|
||||||
|
}
|
||||||
|
|
||||||
stats, err := loadTotalStats(ctx)
|
stats, err := loadTotalStats(ctx)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
@@ -269,6 +379,120 @@ func TestRemoveDecrementsStats(t *testing.T) {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
func TestConcurrentRemoveDecrementsStatsOnce(t *testing.T) {
|
||||||
|
_, _, cleanup := testhelper.SetupTestEnvironment(t)
|
||||||
|
defer cleanup()
|
||||||
|
ctx := context.Background()
|
||||||
|
|
||||||
|
content := []byte("\x89PNG\r\n\x1a\nconcurrent-remove")
|
||||||
|
hash := sha256.Sum256(content)
|
||||||
|
restoreStorage, disableStorage := setupMockStorage(t, nil)
|
||||||
|
defer restoreStorage()
|
||||||
|
defer disableStorage()
|
||||||
|
|
||||||
|
result, err := Ingest(ctx, Request{
|
||||||
|
UserID: 1001,
|
||||||
|
Reader: bytes.NewReader(content),
|
||||||
|
Size: int64(len(content)),
|
||||||
|
FileName: "concurrent.png",
|
||||||
|
MimeType: "image/png",
|
||||||
|
Extension: "png",
|
||||||
|
Hash: hex.EncodeToString(hash[:]),
|
||||||
|
Type: "generic",
|
||||||
|
Policy: PolicyCreate,
|
||||||
|
})
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("Ingest returned error: %v", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
const workers = 8
|
||||||
|
start := make(chan struct{})
|
||||||
|
errs := make(chan error, workers)
|
||||||
|
var wg sync.WaitGroup
|
||||||
|
for i := 0; i < workers; i++ {
|
||||||
|
wg.Add(1)
|
||||||
|
go func() {
|
||||||
|
defer wg.Done()
|
||||||
|
<-start
|
||||||
|
_, removeErr := Remove(ctx, result.Upload.ID)
|
||||||
|
errs <- removeErr
|
||||||
|
}()
|
||||||
|
}
|
||||||
|
close(start)
|
||||||
|
wg.Wait()
|
||||||
|
close(errs)
|
||||||
|
for removeErr := range errs {
|
||||||
|
if removeErr != nil {
|
||||||
|
t.Fatalf("concurrent Remove returned error: %v", removeErr)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
stats, err := loadTotalStats(ctx)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("loadTotalStats returned error: %v", err)
|
||||||
|
}
|
||||||
|
if stats.TotalCount != 0 || stats.TotalSize != 0 {
|
||||||
|
t.Fatalf("stats after concurrent remove = count %d size %d, want zero", stats.TotalCount, stats.TotalSize)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestRemoveOwnedAndReservedTypeBoundaries(t *testing.T) {
|
||||||
|
dbConn, _, cleanup := testhelper.SetupTestEnvironment(t)
|
||||||
|
defer cleanup()
|
||||||
|
ctx := context.Background()
|
||||||
|
|
||||||
|
ordinary := model.Upload{
|
||||||
|
ID: 99101,
|
||||||
|
UserID: 1001,
|
||||||
|
FileName: "owned.txt",
|
||||||
|
FilePath: "uploads/owned.txt",
|
||||||
|
FileSize: 16,
|
||||||
|
MimeType: "text/plain",
|
||||||
|
Extension: "txt",
|
||||||
|
Hash: "owned-hash",
|
||||||
|
Type: "generic",
|
||||||
|
Status: model.UploadStatusUsed,
|
||||||
|
CreatedAt: time.Now(),
|
||||||
|
}
|
||||||
|
reserved := model.Upload{
|
||||||
|
ID: 99102,
|
||||||
|
UserID: 1001,
|
||||||
|
FileName: "pages.zip",
|
||||||
|
FilePath: "uploads/pages.zip",
|
||||||
|
FileSize: 32,
|
||||||
|
MimeType: "application/zip",
|
||||||
|
Extension: "zip",
|
||||||
|
Hash: "reserved-hash",
|
||||||
|
Type: shared.ReservedPagesDeploymentType,
|
||||||
|
Status: model.UploadStatusUsed,
|
||||||
|
CreatedAt: time.Now(),
|
||||||
|
}
|
||||||
|
if err := dbConn.Create(&ordinary).Error; err != nil {
|
||||||
|
t.Fatalf("seed ordinary upload: %v", err)
|
||||||
|
}
|
||||||
|
if err := dbConn.Create(&reserved).Error; err != nil {
|
||||||
|
t.Fatalf("seed reserved upload: %v", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
if _, err := RemoveOwned(ctx, 2002, ordinary.ID); !errors.Is(err, ErrForbidden) {
|
||||||
|
t.Fatalf("RemoveOwned non-owner error = %v, want ErrForbidden", err)
|
||||||
|
}
|
||||||
|
if _, err := Remove(ctx, reserved.ID); !errors.Is(err, ErrReservedUploadType) {
|
||||||
|
t.Fatalf("Remove reserved error = %v, want ErrReservedUploadType", err)
|
||||||
|
}
|
||||||
|
if _, err := RemoveOwned(ctx, reserved.UserID, reserved.ID); !errors.Is(err, ErrReservedUploadType) {
|
||||||
|
t.Fatalf("RemoveOwned reserved error = %v, want ErrReservedUploadType", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
var persisted model.Upload
|
||||||
|
if err := dbConn.First(&persisted, reserved.ID).Error; err != nil {
|
||||||
|
t.Fatalf("reload reserved upload: %v", err)
|
||||||
|
}
|
||||||
|
if persisted.Status != model.UploadStatusUsed {
|
||||||
|
t.Fatalf("reserved upload status = %s, want used", persisted.Status)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
type totalStatsSnapshot struct {
|
type totalStatsSnapshot struct {
|
||||||
TotalCount int64
|
TotalCount int64
|
||||||
TotalSize int64
|
TotalSize int64
|
||||||
@@ -289,6 +513,10 @@ func loadTotalStats(ctx context.Context) (totalStatsSnapshot, error) {
|
|||||||
}
|
}
|
||||||
|
|
||||||
func setupMockStorage(t *testing.T, putCount *int) (restore func(), disable func()) {
|
func setupMockStorage(t *testing.T, putCount *int) (restore func(), disable func()) {
|
||||||
|
return setupMockStorageWithDeleteCount(t, putCount, nil)
|
||||||
|
}
|
||||||
|
|
||||||
|
func setupMockStorageWithDeleteCount(t *testing.T, putCount, deleteCount *int) (restore func(), disable func()) {
|
||||||
t.Helper()
|
t.Helper()
|
||||||
mockFiles := make(map[string][]byte)
|
mockFiles := make(map[string][]byte)
|
||||||
restore = storage.MockStorage(
|
restore = storage.MockStorage(
|
||||||
@@ -316,6 +544,9 @@ func setupMockStorage(t *testing.T, putCount *int) (restore func(), disable func
|
|||||||
},
|
},
|
||||||
func(ctx context.Context, key string) error {
|
func(ctx context.Context, key string) error {
|
||||||
delete(mockFiles, key)
|
delete(mockFiles, key)
|
||||||
|
if deleteCount != nil {
|
||||||
|
*deleteCount++
|
||||||
|
}
|
||||||
return nil
|
return nil
|
||||||
},
|
},
|
||||||
)
|
)
|
||||||
|
|||||||
@@ -7,52 +7,76 @@ import (
|
|||||||
"context"
|
"context"
|
||||||
|
|
||||||
uploadcache "github.com/Rain-kl/Wavelet/internal/apps/upload/cache"
|
uploadcache "github.com/Rain-kl/Wavelet/internal/apps/upload/cache"
|
||||||
|
"github.com/Rain-kl/Wavelet/internal/apps/upload/shared"
|
||||||
uploadstats "github.com/Rain-kl/Wavelet/internal/apps/upload/stats"
|
uploadstats "github.com/Rain-kl/Wavelet/internal/apps/upload/stats"
|
||||||
"github.com/Rain-kl/Wavelet/internal/db"
|
"github.com/Rain-kl/Wavelet/internal/db"
|
||||||
"github.com/Rain-kl/Wavelet/internal/model"
|
"github.com/Rain-kl/Wavelet/internal/model"
|
||||||
"github.com/Rain-kl/Wavelet/internal/repository"
|
"github.com/Rain-kl/Wavelet/internal/repository"
|
||||||
"gorm.io/gorm"
|
"gorm.io/gorm"
|
||||||
|
"gorm.io/gorm/clause"
|
||||||
)
|
)
|
||||||
|
|
||||||
// Remove soft-deletes an upload and decrements incremental stats.
|
// Remove soft-deletes an ordinary upload and decrements incremental stats once.
|
||||||
func Remove(ctx context.Context, uploadID uint64) (model.Upload, error) {
|
func Remove(ctx context.Context, uploadID uint64) (model.Upload, error) {
|
||||||
upload, err := repository.GetActiveUploadByID(ctx, uploadID)
|
upload, err := remove(ctx, 0, uploadID, false)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
return model.Upload{}, err
|
return model.Upload{}, err
|
||||||
}
|
}
|
||||||
if err := softDeleteUploadWithStats(ctx, &upload); err != nil {
|
|
||||||
return model.Upload{}, err
|
|
||||||
}
|
|
||||||
upload.Status = model.UploadStatusDeleted
|
|
||||||
return upload, nil
|
return upload, nil
|
||||||
}
|
}
|
||||||
|
|
||||||
// RemoveOwned soft-deletes an upload owned by userID and decrements incremental stats.
|
// RemoveOwned soft-deletes an ordinary upload owned by userID and decrements incremental stats once.
|
||||||
func RemoveOwned(ctx context.Context, userID, uploadID uint64) (model.Upload, error) {
|
func RemoveOwned(ctx context.Context, userID, uploadID uint64) (model.Upload, error) {
|
||||||
upload, err := repository.GetActiveUploadByID(ctx, uploadID)
|
upload, err := remove(ctx, userID, uploadID, true)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
return model.Upload{}, err
|
return model.Upload{}, err
|
||||||
}
|
}
|
||||||
if upload.UserID != userID {
|
|
||||||
return model.Upload{}, ErrForbidden
|
|
||||||
}
|
|
||||||
if err := softDeleteUploadWithStats(ctx, &upload); err != nil {
|
|
||||||
return model.Upload{}, err
|
|
||||||
}
|
|
||||||
upload.Status = model.UploadStatusDeleted
|
|
||||||
return upload, nil
|
return upload, nil
|
||||||
}
|
}
|
||||||
|
|
||||||
func softDeleteUploadWithStats(ctx context.Context, upload *model.Upload) error {
|
func remove(ctx context.Context, userID, uploadID uint64, owned bool) (model.Upload, error) {
|
||||||
statsSnapshot := *upload
|
var upload model.Upload
|
||||||
if err := db.DB(ctx).Transaction(func(tx *gorm.DB) error {
|
if err := db.DB(ctx).Transaction(func(tx *gorm.DB) error {
|
||||||
if err := repository.SoftDeleteUploadTx(tx, upload); err != nil {
|
if err := tx.Clauses(clause.Locking{Strength: "UPDATE"}).
|
||||||
|
Where("id = ?", uploadID).
|
||||||
|
First(&upload).Error; err != nil {
|
||||||
return err
|
return err
|
||||||
}
|
}
|
||||||
return uploadstats.ApplyUploadStatsDeltaTx(tx, &statsSnapshot, -1)
|
if owned && upload.UserID != userID {
|
||||||
}); err != nil {
|
return ErrForbidden
|
||||||
|
}
|
||||||
|
if upload.Type == shared.ReservedPagesDeploymentType {
|
||||||
|
return ErrReservedUploadType
|
||||||
|
}
|
||||||
|
_, err := RemoveLockedTx(tx, &upload)
|
||||||
return err
|
return err
|
||||||
|
}); err != nil {
|
||||||
|
return model.Upload{}, err
|
||||||
}
|
}
|
||||||
uploadcache.InvalidateUploadMetaCache(ctx, upload.ID)
|
|
||||||
return nil
|
InvalidateUploadMetaCache(ctx, uploadID)
|
||||||
|
upload.Status = model.UploadStatusDeleted
|
||||||
|
return upload, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// RemoveLockedTx performs the idempotent active-to-deleted transition for a row
|
||||||
|
// that the caller has already locked in its surrounding transaction.
|
||||||
|
func RemoveLockedTx(tx *gorm.DB, upload *model.Upload) (bool, error) {
|
||||||
|
rowsAffected, err := repository.SoftDeleteUploadTx(tx, upload)
|
||||||
|
if err != nil {
|
||||||
|
return false, err
|
||||||
|
}
|
||||||
|
if rowsAffected == 0 {
|
||||||
|
return false, nil
|
||||||
|
}
|
||||||
|
if err := uploadstats.ApplyUploadStatsDeltaTx(tx, upload, -1); err != nil {
|
||||||
|
return false, err
|
||||||
|
}
|
||||||
|
upload.Status = model.UploadStatusDeleted
|
||||||
|
return true, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// InvalidateUploadMetaCache invalidates upload metadata after the caller commits its transaction.
|
||||||
|
func InvalidateUploadMetaCache(ctx context.Context, uploadID uint64) {
|
||||||
|
uploadcache.InvalidateUploadMetaCache(ctx, uploadID)
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -17,4 +17,6 @@ const (
|
|||||||
FileStatsTrendDays = 7
|
FileStatsTrendDays = 7
|
||||||
MaxS3KeyLength = 1024
|
MaxS3KeyLength = 1024
|
||||||
AccessCacheTTL = 5 // seconds; multiplied by time.Second at use site
|
AccessCacheTTL = 5 // seconds; multiplied by time.Second at use site
|
||||||
|
// ReservedPagesDeploymentType is managed exclusively by the Pages domain.
|
||||||
|
ReservedPagesDeploymentType = "openflare_pages_deployment"
|
||||||
)
|
)
|
||||||
|
|||||||
@@ -27,6 +27,7 @@ const (
|
|||||||
ErrQueryFileCountFailed = "查询文件数量失败"
|
ErrQueryFileCountFailed = "查询文件数量失败"
|
||||||
ErrQueryFileListFailed = "查询文件列表失败"
|
ErrQueryFileListFailed = "查询文件列表失败"
|
||||||
ErrDeleteFileFailed = "删除文件失败"
|
ErrDeleteFileFailed = "删除文件失败"
|
||||||
|
ErrReservedUploadType = "系统保留的文件类型不能通过通用文件接口操作"
|
||||||
ErrStorageReadOnly = "存储迁移维护中,当前仅允许读取文件"
|
ErrStorageReadOnly = "存储迁移维护中,当前仅允许读取文件"
|
||||||
ErrS3KeyRequired = "s3 key must not be empty"
|
ErrS3KeyRequired = "s3 key must not be empty"
|
||||||
ErrS3KeyTooLongFormat = "s3 key exceeds maximum length of %d"
|
ErrS3KeyTooLongFormat = "s3 key exceeds maximum length of %d"
|
||||||
|
|||||||
@@ -10,16 +10,15 @@ import (
|
|||||||
"fmt"
|
"fmt"
|
||||||
"time"
|
"time"
|
||||||
|
|
||||||
uploadcache "github.com/Rain-kl/Wavelet/internal/apps/upload/cache"
|
"github.com/Rain-kl/Wavelet/internal/apps/upload/ingest"
|
||||||
"github.com/Rain-kl/Wavelet/internal/apps/upload/shared"
|
"github.com/Rain-kl/Wavelet/internal/apps/upload/shared"
|
||||||
uploadstats "github.com/Rain-kl/Wavelet/internal/apps/upload/stats"
|
|
||||||
uploadstorage "github.com/Rain-kl/Wavelet/internal/apps/upload/storage"
|
uploadstorage "github.com/Rain-kl/Wavelet/internal/apps/upload/storage"
|
||||||
"github.com/Rain-kl/Wavelet/internal/db"
|
"github.com/Rain-kl/Wavelet/internal/db"
|
||||||
"github.com/Rain-kl/Wavelet/internal/model"
|
"github.com/Rain-kl/Wavelet/internal/model"
|
||||||
"github.com/Rain-kl/Wavelet/internal/storage"
|
|
||||||
"github.com/Rain-kl/Wavelet/internal/task"
|
"github.com/Rain-kl/Wavelet/internal/task"
|
||||||
"github.com/Rain-kl/Wavelet/pkg/logger"
|
"github.com/Rain-kl/Wavelet/pkg/logger"
|
||||||
"gorm.io/gorm"
|
"gorm.io/gorm"
|
||||||
|
"gorm.io/gorm/clause"
|
||||||
)
|
)
|
||||||
|
|
||||||
const (
|
const (
|
||||||
@@ -77,32 +76,31 @@ func (h *SystemCleanupHandler) Execute(ctx context.Context, _ []byte) (*task.Tas
|
|||||||
|
|
||||||
for _, u := range unusedUploads {
|
for _, u := range unusedUploads {
|
||||||
totalProcessed++
|
totalProcessed++
|
||||||
|
transitioned := false
|
||||||
|
|
||||||
if err := db.DB(ctx).Transaction(func(tx *gorm.DB) error {
|
if err := db.DB(ctx).Transaction(func(tx *gorm.DB) error {
|
||||||
if err := tx.Model(&model.Upload{}).
|
var locked model.Upload
|
||||||
Where("id = ? AND status = ?", u.ID, model.UploadStatusPending).
|
if err := tx.Clauses(clause.Locking{Strength: "UPDATE"}).
|
||||||
Update("status", model.UploadStatusDeleted).Error; err != nil {
|
Where("id = ?", u.ID).
|
||||||
|
First(&locked).Error; err != nil {
|
||||||
return err
|
return err
|
||||||
}
|
}
|
||||||
|
if locked.Status != model.UploadStatusPending || !locked.CreatedAt.Before(oneHourAgo) {
|
||||||
_, backend, err := storage.Active(ctx)
|
return nil
|
||||||
if err != nil {
|
|
||||||
return err
|
|
||||||
}
|
}
|
||||||
if err := backend.Delete(ctx, u.FilePath); err != nil {
|
var err error
|
||||||
return err
|
transitioned, err = ingest.RemoveLockedTx(tx, &locked)
|
||||||
}
|
return err
|
||||||
|
|
||||||
return nil
|
|
||||||
}); err != nil {
|
}); err != nil {
|
||||||
task.AppendLog(ctx, "清理上传文件失败 [ID:%d]: %v", u.ID, err)
|
task.AppendLog(ctx, "清理上传文件失败 [ID:%d]: %v", u.ID, err)
|
||||||
lastID = u.ID
|
lastID = u.ID
|
||||||
continue
|
continue
|
||||||
}
|
}
|
||||||
|
|
||||||
uploadstats.RecordUploadStatsRemove(ctx, &u)
|
ingest.InvalidateUploadMetaCache(ctx, u.ID)
|
||||||
uploadcache.InvalidateUploadMetaCache(ctx, u.ID)
|
if transitioned {
|
||||||
totalDeleted++
|
totalDeleted++
|
||||||
|
}
|
||||||
lastID = u.ID
|
lastID = u.ID
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -19,6 +19,7 @@ import (
|
|||||||
|
|
||||||
"github.com/Rain-kl/Wavelet/internal/apps/upload/filesrv"
|
"github.com/Rain-kl/Wavelet/internal/apps/upload/filesrv"
|
||||||
"github.com/Rain-kl/Wavelet/internal/apps/upload/shared"
|
"github.com/Rain-kl/Wavelet/internal/apps/upload/shared"
|
||||||
|
uploadstats "github.com/Rain-kl/Wavelet/internal/apps/upload/stats"
|
||||||
"github.com/Rain-kl/Wavelet/internal/db"
|
"github.com/Rain-kl/Wavelet/internal/db"
|
||||||
"github.com/Rain-kl/Wavelet/internal/diskcache"
|
"github.com/Rain-kl/Wavelet/internal/diskcache"
|
||||||
"github.com/Rain-kl/Wavelet/internal/model"
|
"github.com/Rain-kl/Wavelet/internal/model"
|
||||||
@@ -33,13 +34,17 @@ func TestSystemCleanupHandler_Execute(t *testing.T) {
|
|||||||
_, _, cleanup := testhelper.SetupTestEnvironment(t)
|
_, _, cleanup := testhelper.SetupTestEnvironment(t)
|
||||||
defer cleanup()
|
defer cleanup()
|
||||||
|
|
||||||
// Mock S3 存储(让 DeleteObject 总是成功)
|
deleteCount := 0
|
||||||
|
// Mock S3 存储并记录 Delete,cleanup 不应物理删除共享对象。
|
||||||
storageMock := storage.MockStorage(
|
storageMock := storage.MockStorage(
|
||||||
func(ctx context.Context, key string, body io.Reader, size int64, contentType string) error {
|
func(ctx context.Context, key string, body io.Reader, size int64, contentType string) error {
|
||||||
return nil
|
return nil
|
||||||
},
|
},
|
||||||
func(ctx context.Context, key string) (*storage.Object, error) { return nil, nil },
|
func(ctx context.Context, key string) (*storage.Object, error) { return nil, nil },
|
||||||
func(ctx context.Context, key string) error { return nil },
|
func(ctx context.Context, key string) error {
|
||||||
|
deleteCount++
|
||||||
|
return nil
|
||||||
|
},
|
||||||
)
|
)
|
||||||
defer storageMock()
|
defer storageMock()
|
||||||
storage.IsEnabledFunc = func() bool { return true }
|
storage.IsEnabledFunc = func() bool { return true }
|
||||||
@@ -86,6 +91,7 @@ func TestSystemCleanupHandler_Execute(t *testing.T) {
|
|||||||
for _, r := range records {
|
for _, r := range records {
|
||||||
err := db.DB(ctx).Create(r).Error
|
err := db.DB(ctx).Create(r).Error
|
||||||
require.NoError(t, err)
|
require.NoError(t, err)
|
||||||
|
require.NoError(t, uploadstats.ApplyUploadStatsAdd(ctx, r))
|
||||||
}
|
}
|
||||||
|
|
||||||
// 准备推送历史测试数据:1个旧的(应删除),1个新的(应保留)
|
// 准备推送历史测试数据:1个旧的(应删除),1个新的(应保留)
|
||||||
@@ -147,6 +153,26 @@ func TestSystemCleanupHandler_Execute(t *testing.T) {
|
|||||||
var usedCount int64
|
var usedCount int64
|
||||||
db.DB(ctx).Model(&model.Upload{}).Where("status = ?", model.UploadStatusUsed).Count(&usedCount)
|
db.DB(ctx).Model(&model.Upload{}).Where("status = ?", model.UploadStatusUsed).Count(&usedCount)
|
||||||
assert.Equal(t, int64(1), usedCount, "used 状态的文件不应受影响")
|
assert.Equal(t, int64(1), usedCount, "used 状态的文件不应受影响")
|
||||||
|
assert.Equal(t, 0, deleteCount, "记录级 cleanup 不应调用 storage backend Delete")
|
||||||
|
|
||||||
|
var totalStats model.UploadStat
|
||||||
|
err = db.DB(ctx).
|
||||||
|
Where("dimension = ? AND stat_key = ?", model.UploadStatDimensionTotal, "").
|
||||||
|
First(&totalStats).Error
|
||||||
|
require.NoError(t, err)
|
||||||
|
assert.Equal(t, int64(2), totalStats.FileCount, "cleanup 后统计只应保留 used 与最近 pending 记录")
|
||||||
|
assert.Equal(t, int64(768), totalStats.FileSize, "cleanup 后统计大小应只扣减一次")
|
||||||
|
|
||||||
|
_, err = handler.Execute(ctx, nil)
|
||||||
|
require.NoError(t, err)
|
||||||
|
var statsAfterSecondRun model.UploadStat
|
||||||
|
err = db.DB(ctx).
|
||||||
|
Where("dimension = ? AND stat_key = ?", model.UploadStatDimensionTotal, "").
|
||||||
|
First(&statsAfterSecondRun).Error
|
||||||
|
require.NoError(t, err)
|
||||||
|
assert.Equal(t, totalStats.FileCount, statsAfterSecondRun.FileCount, "重复 cleanup 不应再次扣减统计")
|
||||||
|
assert.Equal(t, totalStats.FileSize, statsAfterSecondRun.FileSize, "重复 cleanup 不应再次扣减统计大小")
|
||||||
|
assert.Equal(t, 0, deleteCount, "重复 cleanup 仍不应调用 storage backend Delete")
|
||||||
|
|
||||||
// 验证推送历史数据状态:10天前的应被删除,今天的应保留
|
// 验证推送历史数据状态:10天前的应被删除,今天的应保留
|
||||||
var pushCount int64
|
var pushCount int64
|
||||||
|
|||||||
@@ -0,0 +1,82 @@
|
|||||||
|
-- +goose Up
|
||||||
|
ALTER TABLE of_pages_projects
|
||||||
|
ADD COLUMN IF NOT EXISTS content_config_version INTEGER NOT NULL DEFAULT 0;
|
||||||
|
|
||||||
|
ALTER TABLE of_pages_deployments
|
||||||
|
ADD COLUMN IF NOT EXISTS source_type VARCHAR(32) NOT NULL DEFAULT '',
|
||||||
|
ADD COLUMN IF NOT EXISTS source_identity CHAR(64),
|
||||||
|
ADD COLUMN IF NOT EXISTS source_revision CHAR(64),
|
||||||
|
ADD COLUMN IF NOT EXISTS source_label VARCHAR(255) NOT NULL DEFAULT '',
|
||||||
|
ADD COLUMN IF NOT EXISTS source_meta TEXT NOT NULL DEFAULT '',
|
||||||
|
ADD COLUMN IF NOT EXISTS trigger_type VARCHAR(32) NOT NULL DEFAULT '';
|
||||||
|
|
||||||
|
UPDATE of_pages_deployments
|
||||||
|
SET source_type = 'manual_upload',
|
||||||
|
trigger_type = 'manual_upload';
|
||||||
|
|
||||||
|
CREATE TABLE IF NOT EXISTS of_pages_project_sources (
|
||||||
|
id BIGSERIAL PRIMARY KEY,
|
||||||
|
project_id BIGINT NOT NULL,
|
||||||
|
source_type VARCHAR(32) NOT NULL DEFAULT '',
|
||||||
|
remote_url TEXT NOT NULL DEFAULT '',
|
||||||
|
remote_network_policy VARCHAR(32) NOT NULL DEFAULT '',
|
||||||
|
github_repository VARCHAR(255) NOT NULL DEFAULT '',
|
||||||
|
release_selector VARCHAR(16) NOT NULL DEFAULT '',
|
||||||
|
release_tag VARCHAR(255) NOT NULL DEFAULT '',
|
||||||
|
asset_name VARCHAR(255) NOT NULL DEFAULT '',
|
||||||
|
auto_update_enabled BOOLEAN NOT NULL DEFAULT FALSE,
|
||||||
|
check_interval_minutes INTEGER NOT NULL DEFAULT 0,
|
||||||
|
config_version INTEGER NOT NULL DEFAULT 0,
|
||||||
|
source_identity CHAR(64) NOT NULL DEFAULT '',
|
||||||
|
created_at TIMESTAMPTZ NOT NULL DEFAULT CURRENT_TIMESTAMP,
|
||||||
|
updated_at TIMESTAMPTZ NOT NULL DEFAULT CURRENT_TIMESTAMP
|
||||||
|
);
|
||||||
|
|
||||||
|
CREATE UNIQUE INDEX IF NOT EXISTS idx_of_pages_project_sources_project_id
|
||||||
|
ON of_pages_project_sources (project_id);
|
||||||
|
|
||||||
|
CREATE TABLE IF NOT EXISTS of_pages_project_source_runtime (
|
||||||
|
source_id BIGINT PRIMARY KEY,
|
||||||
|
etag VARCHAR(512) NOT NULL DEFAULT '',
|
||||||
|
last_seen_revision CHAR(64) NOT NULL DEFAULT '',
|
||||||
|
last_seen_detail TEXT NOT NULL DEFAULT '',
|
||||||
|
last_applied_revision CHAR(64) NOT NULL DEFAULT '',
|
||||||
|
last_applied_detail TEXT NOT NULL DEFAULT '',
|
||||||
|
sync_status VARCHAR(32) NOT NULL DEFAULT '',
|
||||||
|
last_error TEXT NOT NULL DEFAULT '',
|
||||||
|
last_checked_at TIMESTAMPTZ,
|
||||||
|
last_synced_at TIMESTAMPTZ,
|
||||||
|
next_check_at TIMESTAMPTZ,
|
||||||
|
lease_expires_at TIMESTAMPTZ,
|
||||||
|
lease_token VARCHAR(64) NOT NULL DEFAULT '',
|
||||||
|
updated_at TIMESTAMPTZ NOT NULL DEFAULT CURRENT_TIMESTAMP
|
||||||
|
);
|
||||||
|
|
||||||
|
CREATE INDEX IF NOT EXISTS idx_of_pages_project_source_runtime_next_check_at
|
||||||
|
ON of_pages_project_source_runtime (next_check_at);
|
||||||
|
|
||||||
|
CREATE UNIQUE INDEX IF NOT EXISTS idx_of_pages_deployments_project_number
|
||||||
|
ON of_pages_deployments (project_id, deployment_number);
|
||||||
|
|
||||||
|
CREATE UNIQUE INDEX IF NOT EXISTS idx_of_pages_deployments_source_revision
|
||||||
|
ON of_pages_deployments (project_id, source_identity, source_revision)
|
||||||
|
WHERE source_identity IS NOT NULL AND source_revision IS NOT NULL;
|
||||||
|
|
||||||
|
-- +goose Down
|
||||||
|
DROP INDEX IF EXISTS idx_of_pages_deployments_source_revision;
|
||||||
|
DROP INDEX IF EXISTS idx_of_pages_deployments_project_number;
|
||||||
|
DROP INDEX IF EXISTS idx_of_pages_project_source_runtime_next_check_at;
|
||||||
|
DROP TABLE IF EXISTS of_pages_project_source_runtime;
|
||||||
|
DROP INDEX IF EXISTS idx_of_pages_project_sources_project_id;
|
||||||
|
DROP TABLE IF EXISTS of_pages_project_sources;
|
||||||
|
|
||||||
|
ALTER TABLE of_pages_deployments
|
||||||
|
DROP COLUMN IF EXISTS trigger_type,
|
||||||
|
DROP COLUMN IF EXISTS source_meta,
|
||||||
|
DROP COLUMN IF EXISTS source_label,
|
||||||
|
DROP COLUMN IF EXISTS source_revision,
|
||||||
|
DROP COLUMN IF EXISTS source_identity,
|
||||||
|
DROP COLUMN IF EXISTS source_type;
|
||||||
|
|
||||||
|
ALTER TABLE of_pages_projects
|
||||||
|
DROP COLUMN IF EXISTS content_config_version;
|
||||||
@@ -0,0 +1,38 @@
|
|||||||
|
-- +goose Up
|
||||||
|
-- Earlier built-in schedules used explicit IDs, so advance the identity only
|
||||||
|
-- when it trails either existing rows or an already-higher sequence value.
|
||||||
|
SELECT setval(
|
||||||
|
pg_get_serial_sequence('w_schedules', 'id'),
|
||||||
|
GREATEST(
|
||||||
|
1,
|
||||||
|
COALESCE((SELECT MAX(id) FROM w_schedules), 0),
|
||||||
|
COALESCE((
|
||||||
|
SELECT sequences.last_value
|
||||||
|
FROM pg_sequences AS sequences
|
||||||
|
WHERE format('%I.%I', sequences.schemaname, sequences.sequencename)::regclass =
|
||||||
|
pg_get_serial_sequence('w_schedules', 'id')::regclass
|
||||||
|
), 0)
|
||||||
|
),
|
||||||
|
TRUE
|
||||||
|
);
|
||||||
|
|
||||||
|
INSERT INTO w_schedules (name, task_type, cron, payload, is_active, created_at, updated_at)
|
||||||
|
SELECT
|
||||||
|
'OpenFlare Pages 部署源扫描',
|
||||||
|
'of_pages_source_scan',
|
||||||
|
'*/5 * * * *',
|
||||||
|
'{}',
|
||||||
|
TRUE,
|
||||||
|
CURRENT_TIMESTAMP,
|
||||||
|
CURRENT_TIMESTAMP
|
||||||
|
WHERE NOT EXISTS (
|
||||||
|
SELECT 1 FROM w_schedules WHERE task_type = 'of_pages_source_scan'
|
||||||
|
);
|
||||||
|
|
||||||
|
-- +goose Down
|
||||||
|
DELETE FROM w_schedules
|
||||||
|
WHERE task_type = 'of_pages_source_scan'
|
||||||
|
AND name = 'OpenFlare Pages 部署源扫描'
|
||||||
|
AND cron = '*/5 * * * *'
|
||||||
|
AND payload = '{}'
|
||||||
|
AND is_active = TRUE;
|
||||||
@@ -0,0 +1,151 @@
|
|||||||
|
-- +goose Up
|
||||||
|
ALTER TABLE of_pages_projects
|
||||||
|
ADD COLUMN content_config_version INTEGER NOT NULL DEFAULT 0;
|
||||||
|
|
||||||
|
ALTER TABLE of_pages_deployments
|
||||||
|
ADD COLUMN source_type TEXT NOT NULL DEFAULT '';
|
||||||
|
ALTER TABLE of_pages_deployments
|
||||||
|
ADD COLUMN source_identity TEXT;
|
||||||
|
ALTER TABLE of_pages_deployments
|
||||||
|
ADD COLUMN source_revision TEXT;
|
||||||
|
ALTER TABLE of_pages_deployments
|
||||||
|
ADD COLUMN source_label TEXT NOT NULL DEFAULT '';
|
||||||
|
ALTER TABLE of_pages_deployments
|
||||||
|
ADD COLUMN source_meta TEXT NOT NULL DEFAULT '';
|
||||||
|
ALTER TABLE of_pages_deployments
|
||||||
|
ADD COLUMN trigger_type TEXT NOT NULL DEFAULT '';
|
||||||
|
|
||||||
|
UPDATE of_pages_deployments
|
||||||
|
SET source_type = 'manual_upload',
|
||||||
|
trigger_type = 'manual_upload';
|
||||||
|
|
||||||
|
CREATE TABLE IF NOT EXISTS of_pages_project_sources (
|
||||||
|
id INTEGER PRIMARY KEY AUTOINCREMENT,
|
||||||
|
project_id INTEGER NOT NULL,
|
||||||
|
source_type TEXT NOT NULL DEFAULT '',
|
||||||
|
remote_url TEXT NOT NULL DEFAULT '',
|
||||||
|
remote_network_policy TEXT NOT NULL DEFAULT '',
|
||||||
|
github_repository TEXT NOT NULL DEFAULT '',
|
||||||
|
release_selector TEXT NOT NULL DEFAULT '',
|
||||||
|
release_tag TEXT NOT NULL DEFAULT '',
|
||||||
|
asset_name TEXT NOT NULL DEFAULT '',
|
||||||
|
auto_update_enabled INTEGER NOT NULL DEFAULT 0,
|
||||||
|
check_interval_minutes INTEGER NOT NULL DEFAULT 0,
|
||||||
|
config_version INTEGER NOT NULL DEFAULT 0,
|
||||||
|
source_identity TEXT NOT NULL DEFAULT '',
|
||||||
|
created_at DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP,
|
||||||
|
updated_at DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP
|
||||||
|
);
|
||||||
|
|
||||||
|
CREATE UNIQUE INDEX IF NOT EXISTS idx_of_pages_project_sources_project_id
|
||||||
|
ON of_pages_project_sources (project_id);
|
||||||
|
|
||||||
|
CREATE TABLE IF NOT EXISTS of_pages_project_source_runtime (
|
||||||
|
source_id INTEGER PRIMARY KEY,
|
||||||
|
etag TEXT NOT NULL DEFAULT '',
|
||||||
|
last_seen_revision TEXT NOT NULL DEFAULT '',
|
||||||
|
last_seen_detail TEXT NOT NULL DEFAULT '',
|
||||||
|
last_applied_revision TEXT NOT NULL DEFAULT '',
|
||||||
|
last_applied_detail TEXT NOT NULL DEFAULT '',
|
||||||
|
sync_status TEXT NOT NULL DEFAULT '',
|
||||||
|
last_error TEXT NOT NULL DEFAULT '',
|
||||||
|
last_checked_at DATETIME,
|
||||||
|
last_synced_at DATETIME,
|
||||||
|
next_check_at DATETIME,
|
||||||
|
lease_expires_at DATETIME,
|
||||||
|
lease_token TEXT NOT NULL DEFAULT '',
|
||||||
|
updated_at DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP
|
||||||
|
);
|
||||||
|
|
||||||
|
CREATE INDEX IF NOT EXISTS idx_of_pages_project_source_runtime_next_check_at
|
||||||
|
ON of_pages_project_source_runtime (next_check_at);
|
||||||
|
|
||||||
|
CREATE UNIQUE INDEX IF NOT EXISTS idx_of_pages_deployments_project_number
|
||||||
|
ON of_pages_deployments (project_id, deployment_number);
|
||||||
|
|
||||||
|
CREATE UNIQUE INDEX IF NOT EXISTS idx_of_pages_deployments_source_revision
|
||||||
|
ON of_pages_deployments (project_id, source_identity, source_revision)
|
||||||
|
WHERE source_identity IS NOT NULL AND source_revision IS NOT NULL;
|
||||||
|
|
||||||
|
-- +goose Down
|
||||||
|
DROP INDEX IF EXISTS idx_of_pages_deployments_source_revision;
|
||||||
|
DROP INDEX IF EXISTS idx_of_pages_deployments_project_number;
|
||||||
|
DROP INDEX IF EXISTS idx_of_pages_project_source_runtime_next_check_at;
|
||||||
|
DROP TABLE IF EXISTS of_pages_project_source_runtime;
|
||||||
|
DROP INDEX IF EXISTS idx_of_pages_project_sources_project_id;
|
||||||
|
DROP TABLE IF EXISTS of_pages_project_sources;
|
||||||
|
|
||||||
|
-- SQLite 的 Down 必须重建受影响表,完整移除新增列并保留原有数据与索引。
|
||||||
|
CREATE TABLE of_pages_deployments_before_source (
|
||||||
|
id INTEGER PRIMARY KEY AUTOINCREMENT,
|
||||||
|
project_id INTEGER NOT NULL,
|
||||||
|
deployment_number INTEGER NOT NULL,
|
||||||
|
checksum TEXT NOT NULL,
|
||||||
|
status TEXT NOT NULL DEFAULT 'uploaded',
|
||||||
|
upload_id INTEGER NOT NULL DEFAULT 0,
|
||||||
|
artifact_path TEXT NOT NULL,
|
||||||
|
file_count INTEGER NOT NULL DEFAULT 0,
|
||||||
|
total_size INTEGER NOT NULL DEFAULT 0,
|
||||||
|
created_by TEXT NOT NULL DEFAULT '',
|
||||||
|
created_at DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP,
|
||||||
|
activated_at DATETIME
|
||||||
|
);
|
||||||
|
|
||||||
|
INSERT INTO of_pages_deployments_before_source (
|
||||||
|
id, project_id, deployment_number, checksum, status, upload_id, artifact_path,
|
||||||
|
file_count, total_size, created_by, created_at, activated_at
|
||||||
|
)
|
||||||
|
SELECT
|
||||||
|
id, project_id, deployment_number, checksum, status, upload_id, artifact_path,
|
||||||
|
file_count, total_size, created_by, created_at, activated_at
|
||||||
|
FROM of_pages_deployments;
|
||||||
|
|
||||||
|
DROP TABLE of_pages_deployments;
|
||||||
|
ALTER TABLE of_pages_deployments_before_source RENAME TO of_pages_deployments;
|
||||||
|
|
||||||
|
CREATE INDEX IF NOT EXISTS idx_of_pages_deployments_project_id
|
||||||
|
ON of_pages_deployments (project_id);
|
||||||
|
CREATE INDEX IF NOT EXISTS idx_of_pages_deployments_checksum
|
||||||
|
ON of_pages_deployments (checksum);
|
||||||
|
CREATE INDEX IF NOT EXISTS idx_of_pages_deployments_status
|
||||||
|
ON of_pages_deployments (status);
|
||||||
|
CREATE INDEX IF NOT EXISTS idx_of_pages_deployments_upload_id
|
||||||
|
ON of_pages_deployments (upload_id);
|
||||||
|
|
||||||
|
CREATE TABLE of_pages_projects_before_source (
|
||||||
|
id INTEGER PRIMARY KEY AUTOINCREMENT,
|
||||||
|
name TEXT NOT NULL,
|
||||||
|
slug TEXT NOT NULL,
|
||||||
|
description TEXT NOT NULL DEFAULT '',
|
||||||
|
enabled INTEGER NOT NULL DEFAULT 1,
|
||||||
|
spa_fallback_enabled INTEGER NOT NULL DEFAULT 0,
|
||||||
|
spa_fallback_path TEXT NOT NULL DEFAULT '/index.html',
|
||||||
|
api_proxy_enabled INTEGER NOT NULL DEFAULT 0,
|
||||||
|
api_proxy_path TEXT NOT NULL DEFAULT '',
|
||||||
|
api_proxy_pass TEXT NOT NULL DEFAULT '',
|
||||||
|
api_proxy_rewrite TEXT NOT NULL DEFAULT '',
|
||||||
|
active_deployment_id INTEGER,
|
||||||
|
root_dir TEXT NOT NULL DEFAULT '',
|
||||||
|
entry_file TEXT NOT NULL DEFAULT 'index.html',
|
||||||
|
created_at DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP,
|
||||||
|
updated_at DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP
|
||||||
|
);
|
||||||
|
|
||||||
|
INSERT INTO of_pages_projects_before_source (
|
||||||
|
id, name, slug, description, enabled, spa_fallback_enabled, spa_fallback_path,
|
||||||
|
api_proxy_enabled, api_proxy_path, api_proxy_pass, api_proxy_rewrite,
|
||||||
|
active_deployment_id, root_dir, entry_file, created_at, updated_at
|
||||||
|
)
|
||||||
|
SELECT
|
||||||
|
id, name, slug, description, enabled, spa_fallback_enabled, spa_fallback_path,
|
||||||
|
api_proxy_enabled, api_proxy_path, api_proxy_pass, api_proxy_rewrite,
|
||||||
|
active_deployment_id, root_dir, entry_file, created_at, updated_at
|
||||||
|
FROM of_pages_projects;
|
||||||
|
|
||||||
|
DROP TABLE of_pages_projects;
|
||||||
|
ALTER TABLE of_pages_projects_before_source RENAME TO of_pages_projects;
|
||||||
|
|
||||||
|
CREATE UNIQUE INDEX IF NOT EXISTS idx_of_pages_projects_slug
|
||||||
|
ON of_pages_projects (slug);
|
||||||
|
CREATE INDEX IF NOT EXISTS idx_of_pages_projects_active_deployment_id
|
||||||
|
ON of_pages_projects (active_deployment_id);
|
||||||
@@ -0,0 +1,21 @@
|
|||||||
|
-- +goose Up
|
||||||
|
INSERT INTO w_schedules (name, task_type, cron, payload, is_active, created_at, updated_at)
|
||||||
|
SELECT
|
||||||
|
'OpenFlare Pages 部署源扫描',
|
||||||
|
'of_pages_source_scan',
|
||||||
|
'*/5 * * * *',
|
||||||
|
'{}',
|
||||||
|
1,
|
||||||
|
CURRENT_TIMESTAMP,
|
||||||
|
CURRENT_TIMESTAMP
|
||||||
|
WHERE NOT EXISTS (
|
||||||
|
SELECT 1 FROM w_schedules WHERE task_type = 'of_pages_source_scan'
|
||||||
|
);
|
||||||
|
|
||||||
|
-- +goose Down
|
||||||
|
DELETE FROM w_schedules
|
||||||
|
WHERE task_type = 'of_pages_source_scan'
|
||||||
|
AND name = 'OpenFlare Pages 部署源扫描'
|
||||||
|
AND cron = '*/5 * * * *'
|
||||||
|
AND payload = '{}'
|
||||||
|
AND is_active = 1;
|
||||||
@@ -18,9 +18,9 @@ import (
|
|||||||
"gorm.io/gorm"
|
"gorm.io/gorm"
|
||||||
)
|
)
|
||||||
|
|
||||||
// expectedMigratedSystemConfigCount 包含初始 32 项系统配置,以及 202606220004
|
// expectedMigratedSystemConfigCount 包含初始 32 项系统配置、202606220004
|
||||||
// 从 of_options 迁移过来的 48 项业务配置(OpenFlare/UptimeKuma/OpenResty)。
|
// 从 of_options 迁移过来的 48 项业务配置,以及 Pages 的 2 项业务配置。
|
||||||
const expectedMigratedSystemConfigCount = 80
|
const expectedMigratedSystemConfigCount = 82
|
||||||
|
|
||||||
func TestMigrateInitializesSQLiteDatabase(t *testing.T) {
|
func TestMigrateInitializesSQLiteDatabase(t *testing.T) {
|
||||||
sqliteDB, err := gorm.Open(sqlite.Open(":memory:"), &gorm.Config{
|
sqliteDB, err := gorm.Open(sqlite.Open(":memory:"), &gorm.Config{
|
||||||
|
|||||||
@@ -0,0 +1,322 @@
|
|||||||
|
// Copyright 2026 Arctel.net
|
||||||
|
// SPDX-License-Identifier: Apache-2.0
|
||||||
|
|
||||||
|
package migrator
|
||||||
|
|
||||||
|
import (
|
||||||
|
"database/sql"
|
||||||
|
"fmt"
|
||||||
|
"os"
|
||||||
|
"strings"
|
||||||
|
"testing"
|
||||||
|
"time"
|
||||||
|
|
||||||
|
"github.com/Rain-kl/Wavelet/internal/model"
|
||||||
|
"github.com/glebarez/sqlite"
|
||||||
|
"github.com/pressly/goose/v3"
|
||||||
|
"github.com/stretchr/testify/assert"
|
||||||
|
"github.com/stretchr/testify/require"
|
||||||
|
"gorm.io/driver/postgres"
|
||||||
|
"gorm.io/gorm"
|
||||||
|
)
|
||||||
|
|
||||||
|
const (
|
||||||
|
pagesSourcePreviousMigration = int64(202607180001)
|
||||||
|
pagesSourceMigration = int64(202607190001)
|
||||||
|
pagesMigrationProjectID = uint(900001)
|
||||||
|
pagesMigrationDeploymentID = uint(900001)
|
||||||
|
)
|
||||||
|
|
||||||
|
func TestPagesSourceMigrationSQLiteUpDownUp(t *testing.T) {
|
||||||
|
dbPath := t.TempDir() + "/pages-source-migration.db"
|
||||||
|
gormDB, err := gorm.Open(sqlite.Open(dbPath), &gorm.Config{
|
||||||
|
DisableForeignKeyConstraintWhenMigrating: true,
|
||||||
|
})
|
||||||
|
require.NoError(t, err)
|
||||||
|
|
||||||
|
sqlDB, err := gormDB.DB()
|
||||||
|
require.NoError(t, err)
|
||||||
|
sqlDB.SetMaxOpenConns(1)
|
||||||
|
t.Cleanup(func() { require.NoError(t, sqlDB.Close()) })
|
||||||
|
|
||||||
|
runPagesSourceMigrationUpDownUp(t, gormDB, sqlDB, dialectSqlite, "goose/sqlite")
|
||||||
|
|
||||||
|
var indexSQL string
|
||||||
|
require.NoError(t, gormDB.Raw(
|
||||||
|
"SELECT sql FROM sqlite_master WHERE type = 'index' AND name = ?",
|
||||||
|
"idx_of_pages_deployments_source_revision",
|
||||||
|
).Scan(&indexSQL).Error)
|
||||||
|
assert.Contains(t, strings.ToUpper(indexSQL), "WHERE SOURCE_IDENTITY IS NOT NULL AND SOURCE_REVISION IS NOT NULL")
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestPagesSourceMigrationPostgresUpDownUp(t *testing.T) {
|
||||||
|
dsn := strings.TrimSpace(os.Getenv("OPENFLARE_TEST_POSTGRES_DSN"))
|
||||||
|
if dsn == "" {
|
||||||
|
t.Skip("OPENFLARE_TEST_POSTGRES_DSN is not set")
|
||||||
|
}
|
||||||
|
|
||||||
|
gormDB, err := gorm.Open(postgres.Open(dsn), &gorm.Config{
|
||||||
|
DisableForeignKeyConstraintWhenMigrating: true,
|
||||||
|
})
|
||||||
|
require.NoError(t, err)
|
||||||
|
sqlDB, err := gormDB.DB()
|
||||||
|
require.NoError(t, err)
|
||||||
|
sqlDB.SetMaxOpenConns(1)
|
||||||
|
|
||||||
|
schema := fmt.Sprintf("pages_source_migration_%d", time.Now().UnixNano())
|
||||||
|
require.Regexp(t, `^[a-z0-9_]+$`, schema)
|
||||||
|
require.NoError(t, gormDB.Exec(`CREATE SCHEMA "`+schema+`"`).Error)
|
||||||
|
require.NoError(t, gormDB.Exec(`SET search_path TO "`+schema+`"`).Error)
|
||||||
|
t.Cleanup(func() {
|
||||||
|
assert.NoError(t, gormDB.Exec("SET search_path TO public").Error)
|
||||||
|
assert.NoError(t, gormDB.Exec(`DROP SCHEMA IF EXISTS "`+schema+`" CASCADE`).Error)
|
||||||
|
assert.NoError(t, sqlDB.Close())
|
||||||
|
})
|
||||||
|
|
||||||
|
runPagesSourceMigrationUpDownUp(t, gormDB, sqlDB, dialectPostgres, "goose/postgres")
|
||||||
|
}
|
||||||
|
|
||||||
|
func runPagesSourceMigrationUpDownUp(
|
||||||
|
t *testing.T,
|
||||||
|
gormDB *gorm.DB,
|
||||||
|
sqlDB *sql.DB,
|
||||||
|
dialect string,
|
||||||
|
dir string,
|
||||||
|
) {
|
||||||
|
t.Helper()
|
||||||
|
|
||||||
|
goose.SetBaseFS(migrationFS)
|
||||||
|
require.NoError(t, goose.SetDialect(dialect))
|
||||||
|
require.NoError(t, goose.UpTo(sqlDB, dir, pagesSourcePreviousMigration))
|
||||||
|
seedPrePagesSourceMigrationData(t, gormDB)
|
||||||
|
|
||||||
|
require.NoError(t, goose.UpTo(sqlDB, dir, pagesSourceMigration))
|
||||||
|
assertPagesSourceMigrationUp(t, gormDB)
|
||||||
|
|
||||||
|
require.NoError(t, goose.DownTo(sqlDB, dir, pagesSourcePreviousMigration))
|
||||||
|
assertPagesSourceMigrationDown(t, gormDB)
|
||||||
|
|
||||||
|
require.NoError(t, goose.UpTo(sqlDB, dir, pagesSourceMigration))
|
||||||
|
assertPagesSourceMigrationUpAgain(t, gormDB)
|
||||||
|
}
|
||||||
|
|
||||||
|
func seedPrePagesSourceMigrationData(t *testing.T, gormDB *gorm.DB) {
|
||||||
|
t.Helper()
|
||||||
|
|
||||||
|
require.NoError(t, gormDB.Exec(`
|
||||||
|
INSERT INTO of_pages_projects (
|
||||||
|
id, name, slug, description, enabled, active_deployment_id, root_dir, entry_file
|
||||||
|
) VALUES (?, ?, ?, ?, ?, ?, ?, ?)
|
||||||
|
`,
|
||||||
|
pagesMigrationProjectID,
|
||||||
|
"Migration Site",
|
||||||
|
"migration-site",
|
||||||
|
"keep-project-data",
|
||||||
|
true,
|
||||||
|
pagesMigrationDeploymentID,
|
||||||
|
"public",
|
||||||
|
"home.html",
|
||||||
|
).Error)
|
||||||
|
require.NoError(t, gormDB.Exec(`
|
||||||
|
INSERT INTO of_pages_deployments (
|
||||||
|
id, project_id, deployment_number, checksum, status, upload_id, artifact_path,
|
||||||
|
file_count, total_size, created_by
|
||||||
|
) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
|
||||||
|
`,
|
||||||
|
pagesMigrationDeploymentID,
|
||||||
|
pagesMigrationProjectID,
|
||||||
|
1,
|
||||||
|
strings.Repeat("a", 64),
|
||||||
|
model.PagesDeploymentStatusActive,
|
||||||
|
uint64(700001),
|
||||||
|
"legacy/package.zip",
|
||||||
|
2,
|
||||||
|
int64(128),
|
||||||
|
"user:1",
|
||||||
|
).Error)
|
||||||
|
}
|
||||||
|
|
||||||
|
func assertPagesSourceMigrationUp(t *testing.T, gormDB *gorm.DB) {
|
||||||
|
t.Helper()
|
||||||
|
migrator := gormDB.Migrator()
|
||||||
|
assert.True(t, migrator.HasTable(&model.PagesProjectSource{}))
|
||||||
|
assert.True(t, migrator.HasTable(&model.PagesProjectSourceRuntime{}))
|
||||||
|
assert.True(t, migrator.HasColumn(&model.PagesProject{}, "ContentConfigVersion"))
|
||||||
|
assert.True(t, migrator.HasColumn(&model.PagesDeployment{}, "SourceType"))
|
||||||
|
assert.True(t, migrator.HasColumn(&model.PagesDeployment{}, "SourceIdentity"))
|
||||||
|
assert.True(t, migrator.HasColumn(&model.PagesDeployment{}, "SourceRevision"))
|
||||||
|
assert.True(t, migrator.HasIndex(&model.PagesProjectSource{}, "idx_of_pages_project_sources_project_id"))
|
||||||
|
assert.True(t, migrator.HasIndex(&model.PagesProjectSourceRuntime{}, "idx_of_pages_project_source_runtime_next_check_at"))
|
||||||
|
assert.True(t, migrator.HasIndex(&model.PagesDeployment{}, "idx_of_pages_deployments_project_number"))
|
||||||
|
assert.True(t, migrator.HasIndex(&model.PagesDeployment{}, "idx_of_pages_deployments_source_revision"))
|
||||||
|
|
||||||
|
var project model.PagesProject
|
||||||
|
require.NoError(t, gormDB.First(&project, pagesMigrationProjectID).Error)
|
||||||
|
assert.Equal(t, 0, project.ContentConfigVersion)
|
||||||
|
assert.Equal(t, "keep-project-data", project.Description)
|
||||||
|
assert.Equal(t, "public", project.RootDir)
|
||||||
|
assert.Equal(t, "home.html", project.EntryFile)
|
||||||
|
|
||||||
|
var deployment model.PagesDeployment
|
||||||
|
require.NoError(t, gormDB.First(&deployment, pagesMigrationDeploymentID).Error)
|
||||||
|
assert.Equal(t, "manual_upload", deployment.SourceType)
|
||||||
|
assert.Equal(t, "manual_upload", deployment.TriggerType)
|
||||||
|
assert.Nil(t, deployment.SourceIdentity)
|
||||||
|
assert.Nil(t, deployment.SourceRevision)
|
||||||
|
assert.Equal(t, uint64(700001), deployment.UploadID)
|
||||||
|
|
||||||
|
sourceID := createMigrationSourceRuntime(t, gormDB)
|
||||||
|
assertPagesSourceConstraints(t, gormDB, sourceID)
|
||||||
|
}
|
||||||
|
|
||||||
|
func createMigrationSourceRuntime(t *testing.T, gormDB *gorm.DB) uint {
|
||||||
|
t.Helper()
|
||||||
|
source := model.PagesProjectSource{
|
||||||
|
ProjectID: pagesMigrationProjectID,
|
||||||
|
SourceType: "remote_url",
|
||||||
|
RemoteURL: "https://example.com/site.zip?token=secret",
|
||||||
|
RemoteNetworkPolicy: "public",
|
||||||
|
CheckIntervalMinutes: 0,
|
||||||
|
ConfigVersion: 1,
|
||||||
|
SourceIdentity: strings.Repeat("b", 64),
|
||||||
|
}
|
||||||
|
require.NoError(t, gormDB.Create(&source).Error)
|
||||||
|
require.NotZero(t, source.ID)
|
||||||
|
require.NoError(t, gormDB.Create(&model.PagesProjectSourceRuntime{
|
||||||
|
SourceID: source.ID,
|
||||||
|
SyncStatus: "idle",
|
||||||
|
}).Error)
|
||||||
|
return source.ID
|
||||||
|
}
|
||||||
|
|
||||||
|
func assertPagesSourceConstraints(t *testing.T, gormDB *gorm.DB, sourceID uint) {
|
||||||
|
t.Helper()
|
||||||
|
|
||||||
|
duplicateSource := model.PagesProjectSource{
|
||||||
|
ProjectID: pagesMigrationProjectID,
|
||||||
|
SourceType: "remote_url",
|
||||||
|
ConfigVersion: 1,
|
||||||
|
SourceIdentity: strings.Repeat("c", 64),
|
||||||
|
}
|
||||||
|
assert.Error(t, gormDB.Create(&duplicateSource).Error)
|
||||||
|
|
||||||
|
for number := 2; number <= 3; number++ {
|
||||||
|
require.NoError(t, createMigrationDeployment(
|
||||||
|
gormDB,
|
||||||
|
number,
|
||||||
|
strings.Repeat(string(rune('a'+number)), 64),
|
||||||
|
nil,
|
||||||
|
nil,
|
||||||
|
))
|
||||||
|
}
|
||||||
|
|
||||||
|
identity := strings.Repeat("d", 64)
|
||||||
|
revision := strings.Repeat("e", 64)
|
||||||
|
require.NoError(t, createMigrationDeployment(
|
||||||
|
gormDB,
|
||||||
|
4,
|
||||||
|
strings.Repeat("f", 64),
|
||||||
|
&identity,
|
||||||
|
&revision,
|
||||||
|
))
|
||||||
|
assert.Error(t, createMigrationDeployment(
|
||||||
|
gormDB,
|
||||||
|
5,
|
||||||
|
strings.Repeat("0", 64),
|
||||||
|
&identity,
|
||||||
|
&revision,
|
||||||
|
))
|
||||||
|
assert.Error(t, createMigrationDeployment(
|
||||||
|
gormDB,
|
||||||
|
1,
|
||||||
|
strings.Repeat("1", 64),
|
||||||
|
nil,
|
||||||
|
nil,
|
||||||
|
))
|
||||||
|
|
||||||
|
var runtime model.PagesProjectSourceRuntime
|
||||||
|
require.NoError(t, gormDB.First(&runtime, sourceID).Error)
|
||||||
|
assert.Equal(t, "idle", runtime.SyncStatus)
|
||||||
|
}
|
||||||
|
|
||||||
|
func createMigrationDeployment(
|
||||||
|
gormDB *gorm.DB,
|
||||||
|
deploymentNumber int,
|
||||||
|
checksum string,
|
||||||
|
identity *string,
|
||||||
|
revision *string,
|
||||||
|
) error {
|
||||||
|
return gormDB.Create(&model.PagesDeployment{
|
||||||
|
ProjectID: pagesMigrationProjectID,
|
||||||
|
DeploymentNumber: deploymentNumber,
|
||||||
|
Checksum: checksum,
|
||||||
|
Status: model.PagesDeploymentStatusUploaded,
|
||||||
|
UploadID: uint64(710000 + deploymentNumber),
|
||||||
|
ArtifactPath: fmt.Sprintf("legacy/%d.zip", deploymentNumber),
|
||||||
|
SourceType: "manual_upload",
|
||||||
|
SourceIdentity: identity,
|
||||||
|
SourceRevision: revision,
|
||||||
|
TriggerType: "manual_upload",
|
||||||
|
}).Error
|
||||||
|
}
|
||||||
|
|
||||||
|
func assertPagesSourceMigrationDown(t *testing.T, gormDB *gorm.DB) {
|
||||||
|
t.Helper()
|
||||||
|
migrator := gormDB.Migrator()
|
||||||
|
assert.False(t, migrator.HasTable(&model.PagesProjectSource{}))
|
||||||
|
assert.False(t, migrator.HasTable(&model.PagesProjectSourceRuntime{}))
|
||||||
|
assert.False(t, migrator.HasColumn(&model.PagesProject{}, "ContentConfigVersion"))
|
||||||
|
assert.False(t, migrator.HasColumn(&model.PagesDeployment{}, "SourceType"))
|
||||||
|
assert.False(t, migrator.HasColumn(&model.PagesDeployment{}, "SourceIdentity"))
|
||||||
|
assert.False(t, migrator.HasColumn(&model.PagesDeployment{}, "SourceRevision"))
|
||||||
|
assert.False(t, migrator.HasIndex(&model.PagesDeployment{}, "idx_of_pages_deployments_project_number"))
|
||||||
|
assert.False(t, migrator.HasIndex(&model.PagesDeployment{}, "idx_of_pages_deployments_source_revision"))
|
||||||
|
assert.True(t, migrator.HasIndex(&model.PagesDeployment{}, "idx_of_pages_deployments_project_id"))
|
||||||
|
assert.True(t, migrator.HasIndex(&model.PagesDeployment{}, "idx_of_pages_deployments_upload_id"))
|
||||||
|
|
||||||
|
var project struct {
|
||||||
|
Description string
|
||||||
|
RootDir string
|
||||||
|
EntryFile string
|
||||||
|
ActiveDeploymentID *uint
|
||||||
|
}
|
||||||
|
require.NoError(t, gormDB.Table("of_pages_projects").Where("id = ?", pagesMigrationProjectID).Take(&project).Error)
|
||||||
|
assert.Equal(t, "keep-project-data", project.Description)
|
||||||
|
assert.Equal(t, "public", project.RootDir)
|
||||||
|
assert.Equal(t, "home.html", project.EntryFile)
|
||||||
|
require.NotNil(t, project.ActiveDeploymentID)
|
||||||
|
assert.Equal(t, pagesMigrationDeploymentID, *project.ActiveDeploymentID)
|
||||||
|
|
||||||
|
var deployment struct {
|
||||||
|
UploadID uint64
|
||||||
|
ArtifactPath string
|
||||||
|
FileCount int
|
||||||
|
TotalSize int64
|
||||||
|
}
|
||||||
|
require.NoError(t, gormDB.Table("of_pages_deployments").Where("id = ?", pagesMigrationDeploymentID).Take(&deployment).Error)
|
||||||
|
assert.Equal(t, uint64(700001), deployment.UploadID)
|
||||||
|
assert.Equal(t, "legacy/package.zip", deployment.ArtifactPath)
|
||||||
|
assert.Equal(t, 2, deployment.FileCount)
|
||||||
|
assert.Equal(t, int64(128), deployment.TotalSize)
|
||||||
|
|
||||||
|
var count int64
|
||||||
|
require.NoError(t, gormDB.Table("of_pages_deployments").Where("project_id = ?", pagesMigrationProjectID).Count(&count).Error)
|
||||||
|
assert.Equal(t, int64(4), count)
|
||||||
|
}
|
||||||
|
|
||||||
|
func assertPagesSourceMigrationUpAgain(t *testing.T, gormDB *gorm.DB) {
|
||||||
|
t.Helper()
|
||||||
|
assert.True(t, gormDB.Migrator().HasTable(&model.PagesProjectSource{}))
|
||||||
|
assert.True(t, gormDB.Migrator().HasTable(&model.PagesProjectSourceRuntime{}))
|
||||||
|
assert.True(t, gormDB.Migrator().HasColumn(&model.PagesProject{}, "ContentConfigVersion"))
|
||||||
|
assert.True(t, gormDB.Migrator().HasColumn(&model.PagesDeployment{}, "SourceRevision"))
|
||||||
|
|
||||||
|
var count int64
|
||||||
|
require.NoError(t, gormDB.Table("of_pages_deployments").
|
||||||
|
Where("project_id = ? AND source_type = ? AND trigger_type = ?", pagesMigrationProjectID, "manual_upload", "manual_upload").
|
||||||
|
Count(&count).Error)
|
||||||
|
assert.Equal(t, int64(4), count)
|
||||||
|
|
||||||
|
require.NoError(t, gormDB.Table("of_pages_project_sources").Count(&count).Error)
|
||||||
|
assert.Zero(t, count, "source config is intentionally removed by Down and is not reconstructable")
|
||||||
|
}
|
||||||
@@ -0,0 +1,160 @@
|
|||||||
|
// Copyright 2026 Arctel.net
|
||||||
|
// SPDX-License-Identifier: Apache-2.0
|
||||||
|
|
||||||
|
package migrator
|
||||||
|
|
||||||
|
import (
|
||||||
|
"database/sql"
|
||||||
|
"fmt"
|
||||||
|
"os"
|
||||||
|
"strings"
|
||||||
|
"testing"
|
||||||
|
"time"
|
||||||
|
|
||||||
|
"github.com/Rain-kl/Wavelet/internal/model"
|
||||||
|
"github.com/glebarez/sqlite"
|
||||||
|
"github.com/pressly/goose/v3"
|
||||||
|
"github.com/stretchr/testify/assert"
|
||||||
|
"github.com/stretchr/testify/require"
|
||||||
|
"gorm.io/driver/postgres"
|
||||||
|
"gorm.io/gorm"
|
||||||
|
)
|
||||||
|
|
||||||
|
const (
|
||||||
|
pagesSourceScanPreviousMigration = int64(202607190001)
|
||||||
|
pagesSourceScanMigration = int64(202607190002)
|
||||||
|
pagesSourceScanTaskType = "of_pages_source_scan"
|
||||||
|
)
|
||||||
|
|
||||||
|
func TestPagesSourceScanScheduleMigrationSQLite(t *testing.T) {
|
||||||
|
dbPath := t.TempDir() + "/pages-source-scan-migration.db"
|
||||||
|
gormDB, err := gorm.Open(sqlite.Open(dbPath), &gorm.Config{
|
||||||
|
DisableForeignKeyConstraintWhenMigrating: true,
|
||||||
|
})
|
||||||
|
require.NoError(t, err)
|
||||||
|
sqlDB, err := gormDB.DB()
|
||||||
|
require.NoError(t, err)
|
||||||
|
sqlDB.SetMaxOpenConns(1)
|
||||||
|
t.Cleanup(func() { require.NoError(t, sqlDB.Close()) })
|
||||||
|
|
||||||
|
runPagesSourceScanScheduleMigration(t, gormDB, sqlDB, dialectSqlite, "goose/sqlite")
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestPagesSourceScanScheduleMigrationPostgres(t *testing.T) {
|
||||||
|
dsn := strings.TrimSpace(os.Getenv("OPENFLARE_TEST_POSTGRES_DSN"))
|
||||||
|
if dsn == "" {
|
||||||
|
t.Skip("OPENFLARE_TEST_POSTGRES_DSN is not set")
|
||||||
|
}
|
||||||
|
gormDB, err := gorm.Open(postgres.Open(dsn), &gorm.Config{
|
||||||
|
DisableForeignKeyConstraintWhenMigrating: true,
|
||||||
|
})
|
||||||
|
require.NoError(t, err)
|
||||||
|
sqlDB, err := gormDB.DB()
|
||||||
|
require.NoError(t, err)
|
||||||
|
sqlDB.SetMaxOpenConns(1)
|
||||||
|
|
||||||
|
schema := fmt.Sprintf("pages_source_scan_migration_%d", time.Now().UnixNano())
|
||||||
|
require.Regexp(t, `^[a-z0-9_]+$`, schema)
|
||||||
|
require.NoError(t, gormDB.Exec(`CREATE SCHEMA "`+schema+`"`).Error)
|
||||||
|
require.NoError(t, gormDB.Exec(`SET search_path TO "`+schema+`"`).Error)
|
||||||
|
t.Cleanup(func() {
|
||||||
|
assert.NoError(t, gormDB.Exec("SET search_path TO public").Error)
|
||||||
|
assert.NoError(t, gormDB.Exec(`DROP SCHEMA IF EXISTS "`+schema+`" CASCADE`).Error)
|
||||||
|
assert.NoError(t, sqlDB.Close())
|
||||||
|
})
|
||||||
|
|
||||||
|
runPagesSourceScanScheduleMigration(t, gormDB, sqlDB, dialectPostgres, "goose/postgres")
|
||||||
|
}
|
||||||
|
|
||||||
|
func runPagesSourceScanScheduleMigration(
|
||||||
|
t *testing.T,
|
||||||
|
gormDB *gorm.DB,
|
||||||
|
sqlDB *sql.DB,
|
||||||
|
dialect string,
|
||||||
|
dir string,
|
||||||
|
) {
|
||||||
|
t.Helper()
|
||||||
|
goose.SetBaseFS(migrationFS)
|
||||||
|
require.NoError(t, goose.SetDialect(dialect))
|
||||||
|
require.NoError(t, goose.UpTo(sqlDB, dir, pagesSourceScanPreviousMigration))
|
||||||
|
|
||||||
|
var previousMaxID uint64
|
||||||
|
require.NoError(t, gormDB.Table("w_schedules").Select("COALESCE(MAX(id), 0)").Scan(&previousMaxID).Error)
|
||||||
|
require.NoError(t, goose.UpTo(sqlDB, dir, pagesSourceScanMigration))
|
||||||
|
seeded := assertPagesSourceScanSchedule(t, gormDB)
|
||||||
|
assert.NotZero(t, seeded.ID)
|
||||||
|
if dialect == dialectPostgres {
|
||||||
|
assert.Greater(t, seeded.ID, previousMaxID)
|
||||||
|
}
|
||||||
|
|
||||||
|
require.NoError(t, goose.DownTo(sqlDB, dir, pagesSourceScanPreviousMigration))
|
||||||
|
assertPagesSourceScanScheduleMissing(t, gormDB)
|
||||||
|
|
||||||
|
custom := model.Schedule{
|
||||||
|
ID: 900001,
|
||||||
|
Name: "用户保留的 Pages 扫描任务",
|
||||||
|
TaskType: pagesSourceScanTaskType,
|
||||||
|
Cron: "0 * * * *",
|
||||||
|
Payload: `{"custom":true}`,
|
||||||
|
IsActive: false,
|
||||||
|
}
|
||||||
|
require.NoError(t, gormDB.Create(&custom).Error)
|
||||||
|
require.NoError(t, goose.UpTo(sqlDB, dir, pagesSourceScanMigration))
|
||||||
|
var schedules []model.Schedule
|
||||||
|
require.NoError(t, gormDB.Where("task_type = ?", pagesSourceScanTaskType).Find(&schedules).Error)
|
||||||
|
require.Len(t, schedules, 1)
|
||||||
|
assert.Equal(t, custom.ID, schedules[0].ID)
|
||||||
|
assert.Equal(t, custom.Name, schedules[0].Name)
|
||||||
|
|
||||||
|
require.NoError(t, goose.DownTo(sqlDB, dir, pagesSourceScanPreviousMigration))
|
||||||
|
var retained model.Schedule
|
||||||
|
require.NoError(t, gormDB.First(&retained, custom.ID).Error)
|
||||||
|
assert.Equal(t, custom.TaskType, retained.TaskType)
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestPagesSourceScanScheduleMigrationsUseDatabaseGeneratedIDs(t *testing.T) {
|
||||||
|
for _, name := range []string{
|
||||||
|
"goose/postgres/202607190002_seed_pages_source_scan.sql",
|
||||||
|
"goose/sqlite/202607190002_seed_pages_source_scan.sql",
|
||||||
|
} {
|
||||||
|
t.Run(name, func(t *testing.T) {
|
||||||
|
content, err := migrationFS.ReadFile(name)
|
||||||
|
require.NoError(t, err)
|
||||||
|
normalized := strings.ToLower(string(content))
|
||||||
|
assert.NotContains(t, normalized, "insert into w_schedules (id,")
|
||||||
|
assert.NotContains(t, normalized, "coalesce(max(id)")
|
||||||
|
})
|
||||||
|
}
|
||||||
|
|
||||||
|
postgresContent, err := migrationFS.ReadFile("goose/postgres/202607190002_seed_pages_source_scan.sql")
|
||||||
|
require.NoError(t, err)
|
||||||
|
compactPostgres := strings.Join(strings.Fields(strings.ToLower(string(postgresContent))), " ")
|
||||||
|
assert.Contains(
|
||||||
|
t,
|
||||||
|
compactPostgres,
|
||||||
|
"select setval( pg_get_serial_sequence('w_schedules', 'id'), greatest( 1,",
|
||||||
|
"sequence synchronization must retain a valid lower bound for an empty table",
|
||||||
|
)
|
||||||
|
}
|
||||||
|
|
||||||
|
func assertPagesSourceScanSchedule(t *testing.T, gormDB *gorm.DB) model.Schedule {
|
||||||
|
t.Helper()
|
||||||
|
var schedules []model.Schedule
|
||||||
|
require.NoError(t, gormDB.Where("task_type = ?", pagesSourceScanTaskType).Find(&schedules).Error)
|
||||||
|
require.Len(t, schedules, 1)
|
||||||
|
schedule := schedules[0]
|
||||||
|
assert.Equal(t, "OpenFlare Pages 部署源扫描", schedule.Name)
|
||||||
|
assert.Equal(t, "*/5 * * * *", schedule.Cron)
|
||||||
|
assert.Equal(t, "{}", schedule.Payload)
|
||||||
|
assert.True(t, schedule.IsActive)
|
||||||
|
return schedule
|
||||||
|
}
|
||||||
|
|
||||||
|
func assertPagesSourceScanScheduleMissing(t *testing.T, gormDB *gorm.DB) {
|
||||||
|
t.Helper()
|
||||||
|
var count int64
|
||||||
|
require.NoError(t, gormDB.Model(&model.Schedule{}).
|
||||||
|
Where("task_type = ?", pagesSourceScanTaskType).
|
||||||
|
Count(&count).Error)
|
||||||
|
assert.Zero(t, count)
|
||||||
|
}
|
||||||
+21
-16
@@ -57,13 +57,10 @@ func initSQLite() {
|
|||||||
|
|
||||||
// Trace 注入
|
// Trace 注入
|
||||||
if err = db.Use(
|
if err = db.Use(
|
||||||
tracing.NewPlugin(
|
newGORMTracingPlugin([]attribute.KeyValue{
|
||||||
tracing.WithoutMetrics(),
|
attribute.String("db.instance", sqlitePath),
|
||||||
tracing.WithAttributes(
|
attribute.String("db.system", "SQLite"),
|
||||||
attribute.String("db.instance", sqlitePath),
|
}),
|
||||||
attribute.String("db.system", "SQLite"),
|
|
||||||
),
|
|
||||||
),
|
|
||||||
); err != nil {
|
); err != nil {
|
||||||
log.Fatalf("[SQLite] init trace failed: %v\n", err)
|
log.Fatalf("[SQLite] init trace failed: %v\n", err)
|
||||||
}
|
}
|
||||||
@@ -98,15 +95,12 @@ func initPostgres() {
|
|||||||
|
|
||||||
// Trace 注入
|
// Trace 注入
|
||||||
if err = db.Use(
|
if err = db.Use(
|
||||||
tracing.NewPlugin(
|
newGORMTracingPlugin([]attribute.KeyValue{
|
||||||
tracing.WithoutMetrics(),
|
attribute.String("db.instance", dbConfig.Database),
|
||||||
tracing.WithAttributes(
|
attribute.String("db.ip", dbConfig.Host),
|
||||||
attribute.String("db.instance", dbConfig.Database),
|
attribute.String("server.address", net.JoinHostPort(dbConfig.Host, strconv.Itoa(dbConfig.Port))),
|
||||||
attribute.String("db.ip", dbConfig.Host),
|
attribute.String("db.system", "PostgreSQL"),
|
||||||
attribute.String("server.address", net.JoinHostPort(dbConfig.Host, strconv.Itoa(dbConfig.Port))),
|
}),
|
||||||
attribute.String("db.system", "PostgreSQL"),
|
|
||||||
),
|
|
||||||
),
|
|
||||||
); err != nil {
|
); err != nil {
|
||||||
log.Fatalf("[PostgreSQL] init trace failed: %v\n", err)
|
log.Fatalf("[PostgreSQL] init trace failed: %v\n", err)
|
||||||
}
|
}
|
||||||
@@ -160,6 +154,17 @@ func initPostgres() {
|
|||||||
|
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// newGORMTracingPlugin 构造数据库链路追踪插件。查询参数只保留占位符,避免凭据等绑定值进入 Span。
|
||||||
|
func newGORMTracingPlugin(attrs []attribute.KeyValue, extraOptions ...tracing.Option) gorm.Plugin {
|
||||||
|
options := []tracing.Option{
|
||||||
|
tracing.WithoutMetrics(),
|
||||||
|
tracing.WithoutQueryVariables(),
|
||||||
|
tracing.WithAttributes(attrs...),
|
||||||
|
}
|
||||||
|
options = append(options, extraOptions...)
|
||||||
|
return tracing.NewPlugin(options...)
|
||||||
|
}
|
||||||
|
|
||||||
// buildDSN 构建 PostgreSQL DSN
|
// buildDSN 构建 PostgreSQL DSN
|
||||||
func buildDSN(host string, port int, username, password string) string {
|
func buildDSN(host string, port int, username, password string) string {
|
||||||
cfg := config.Config.Database
|
cfg := config.Config.Database
|
||||||
|
|||||||
@@ -49,6 +49,11 @@ func (l *gormZapLogger) Error(ctx context.Context, fmt string, args ...interface
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// ParamsFilter 让 GORM 的 Trace 回调只接收参数化 SQL,避免绑定值被 Dialector.Explain 展开到日志。
|
||||||
|
func (l *gormZapLogger) ParamsFilter(_ context.Context, sql string, _ ...interface{}) (string, []interface{}) {
|
||||||
|
return sql, nil
|
||||||
|
}
|
||||||
|
|
||||||
func (l *gormZapLogger) Trace(ctx context.Context, begin time.Time, fc func() (sql string, rowsAffected int64), err error) {
|
func (l *gormZapLogger) Trace(ctx context.Context, begin time.Time, fc func() (sql string, rowsAffected int64), err error) {
|
||||||
elapsed := time.Since(begin)
|
elapsed := time.Since(begin)
|
||||||
switch {
|
switch {
|
||||||
|
|||||||
@@ -4,11 +4,40 @@
|
|||||||
package db
|
package db
|
||||||
|
|
||||||
import (
|
import (
|
||||||
|
"context"
|
||||||
|
"strings"
|
||||||
"testing"
|
"testing"
|
||||||
|
"time"
|
||||||
|
|
||||||
|
"github.com/glebarez/sqlite"
|
||||||
|
"gorm.io/gorm"
|
||||||
gormLogger "gorm.io/gorm/logger"
|
gormLogger "gorm.io/gorm/logger"
|
||||||
)
|
)
|
||||||
|
|
||||||
|
type paramsFilterCaptureLogger struct {
|
||||||
|
filter *gormZapLogger
|
||||||
|
traces []string
|
||||||
|
}
|
||||||
|
|
||||||
|
func (l *paramsFilterCaptureLogger) LogMode(gormLogger.LogLevel) gormLogger.Interface {
|
||||||
|
return l
|
||||||
|
}
|
||||||
|
|
||||||
|
func (l *paramsFilterCaptureLogger) Info(context.Context, string, ...interface{}) {}
|
||||||
|
|
||||||
|
func (l *paramsFilterCaptureLogger) Warn(context.Context, string, ...interface{}) {}
|
||||||
|
|
||||||
|
func (l *paramsFilterCaptureLogger) Error(context.Context, string, ...interface{}) {}
|
||||||
|
|
||||||
|
func (l *paramsFilterCaptureLogger) ParamsFilter(ctx context.Context, sql string, params ...interface{}) (string, []interface{}) {
|
||||||
|
return l.filter.ParamsFilter(ctx, sql, params...)
|
||||||
|
}
|
||||||
|
|
||||||
|
func (l *paramsFilterCaptureLogger) Trace(_ context.Context, _ time.Time, fc func() (string, int64), _ error) {
|
||||||
|
sql, _ := fc()
|
||||||
|
l.traces = append(l.traces, sql)
|
||||||
|
}
|
||||||
|
|
||||||
func TestParseLogLevel(t *testing.T) {
|
func TestParseLogLevel(t *testing.T) {
|
||||||
t.Parallel()
|
t.Parallel()
|
||||||
|
|
||||||
@@ -37,3 +66,50 @@ func TestParseLogLevel(t *testing.T) {
|
|||||||
})
|
})
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
func TestGormZapLoggerParamsFilterDropsBoundValues(t *testing.T) {
|
||||||
|
t.Parallel()
|
||||||
|
|
||||||
|
const (
|
||||||
|
query = "UPDATE openflare_pages_sources SET remote_url = ? WHERE id = ?"
|
||||||
|
secret = "https://example.test/release.zip?token=super-secret"
|
||||||
|
)
|
||||||
|
|
||||||
|
filteredSQL, filteredParams := (&gormZapLogger{}).ParamsFilter(t.Context(), query, secret, int64(42))
|
||||||
|
if filteredSQL != query {
|
||||||
|
t.Fatalf("ParamsFilter() sql = %q, want %q", filteredSQL, query)
|
||||||
|
}
|
||||||
|
if filteredParams != nil {
|
||||||
|
t.Fatalf("ParamsFilter() params = %#v, want nil", filteredParams)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestGormZapLoggerKeepsParameterizedSQLInTrace(t *testing.T) {
|
||||||
|
t.Parallel()
|
||||||
|
|
||||||
|
const secret = "https://example.test/release.zip?token=trace-secret"
|
||||||
|
capture := ¶msFilterCaptureLogger{filter: &gormZapLogger{}}
|
||||||
|
testDB, err := gorm.Open(sqlite.Open(":memory:"), &gorm.Config{Logger: capture})
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("open sqlite: %v", err)
|
||||||
|
}
|
||||||
|
if err := testDB.Exec("CREATE TABLE source_secrets (remote_url TEXT NOT NULL)").Error; err != nil {
|
||||||
|
t.Fatalf("create table: %v", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
capture.traces = nil
|
||||||
|
if err := testDB.Exec("INSERT INTO source_secrets (remote_url) VALUES (?)", secret).Error; err != nil {
|
||||||
|
t.Fatalf("insert source secret: %v", err)
|
||||||
|
}
|
||||||
|
if len(capture.traces) != 1 {
|
||||||
|
t.Fatalf("trace count = %d, want 1", len(capture.traces))
|
||||||
|
}
|
||||||
|
|
||||||
|
traceSQL := capture.traces[0]
|
||||||
|
if strings.Contains(traceSQL, secret) || strings.Contains(traceSQL, "trace-secret") {
|
||||||
|
t.Fatalf("trace SQL leaked bound value: %q", traceSQL)
|
||||||
|
}
|
||||||
|
if !strings.Contains(traceSQL, "VALUES (?)") {
|
||||||
|
t.Fatalf("trace SQL = %q, want parameter placeholder", traceSQL)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|||||||
@@ -0,0 +1,69 @@
|
|||||||
|
// Copyright 2026 Arctel.net
|
||||||
|
// SPDX-License-Identifier: Apache-2.0
|
||||||
|
|
||||||
|
package db
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
"strings"
|
||||||
|
"testing"
|
||||||
|
|
||||||
|
"github.com/glebarez/sqlite"
|
||||||
|
"go.opentelemetry.io/otel/attribute"
|
||||||
|
sdktrace "go.opentelemetry.io/otel/sdk/trace"
|
||||||
|
"go.opentelemetry.io/otel/sdk/trace/tracetest"
|
||||||
|
semconv "go.opentelemetry.io/otel/semconv/v1.30.0"
|
||||||
|
"gorm.io/gorm"
|
||||||
|
gormLogger "gorm.io/gorm/logger"
|
||||||
|
"gorm.io/plugin/opentelemetry/tracing"
|
||||||
|
)
|
||||||
|
|
||||||
|
func TestGORMTracingPluginDoesNotRecordQueryVariables(t *testing.T) {
|
||||||
|
t.Parallel()
|
||||||
|
|
||||||
|
const secret = "https://example.test/release.zip?token=otel-secret"
|
||||||
|
spanRecorder := tracetest.NewSpanRecorder()
|
||||||
|
tracerProvider := sdktrace.NewTracerProvider(sdktrace.WithSpanProcessor(spanRecorder))
|
||||||
|
t.Cleanup(func() {
|
||||||
|
if err := tracerProvider.Shutdown(context.Background()); err != nil {
|
||||||
|
t.Errorf("shutdown tracer provider: %v", err)
|
||||||
|
}
|
||||||
|
})
|
||||||
|
|
||||||
|
testDB, err := gorm.Open(sqlite.Open(":memory:"), &gorm.Config{
|
||||||
|
Logger: gormLogger.Default.LogMode(gormLogger.Silent),
|
||||||
|
})
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("open sqlite: %v", err)
|
||||||
|
}
|
||||||
|
if err := testDB.Use(newGORMTracingPlugin(
|
||||||
|
[]attribute.KeyValue{attribute.String("db.instance", "trace-test")},
|
||||||
|
tracing.WithTracerProvider(tracerProvider),
|
||||||
|
)); err != nil {
|
||||||
|
t.Fatalf("register tracing plugin: %v", err)
|
||||||
|
}
|
||||||
|
if err := testDB.Exec("CREATE TABLE source_secrets (remote_url TEXT NOT NULL)").Error; err != nil {
|
||||||
|
t.Fatalf("create table: %v", err)
|
||||||
|
}
|
||||||
|
if err := testDB.Exec("INSERT INTO source_secrets (remote_url) VALUES (?)", secret).Error; err != nil {
|
||||||
|
t.Fatalf("insert source secret: %v", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
var queryText string
|
||||||
|
for _, span := range spanRecorder.Ended() {
|
||||||
|
for _, attr := range span.Attributes() {
|
||||||
|
if attr.Key == semconv.DBQueryTextKey && strings.Contains(attr.Value.AsString(), "INSERT INTO source_secrets") {
|
||||||
|
queryText = attr.Value.AsString()
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if queryText == "" {
|
||||||
|
t.Fatal("database query text attribute not found")
|
||||||
|
}
|
||||||
|
if strings.Contains(queryText, secret) || strings.Contains(queryText, "otel-secret") {
|
||||||
|
t.Fatalf("db.query.text leaked bound value: %q", queryText)
|
||||||
|
}
|
||||||
|
if !strings.Contains(queryText, "VALUES (?)") {
|
||||||
|
t.Fatalf("db.query.text = %q, want parameter placeholder", queryText)
|
||||||
|
}
|
||||||
|
}
|
||||||
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user