Compare commits

...

39 Commits

Author SHA1 Message Date
ryan 01ed2c5e36 chore(release): v3.5.2
修复几个遗漏bug
2026-08-09 14:08:04 +08:00
ryan 80696c12fa fix: lint 2026-08-09 13:47:40 +08:00
ryan 3d4d99081e fix(log): PG 日志库批量写入为零 ID 行生成雪花 ID
PostgreSQL 日志表 id 为 NOT NULL 且无默认值,而 GORM 将零值 uint64
主键视为自增并省略 id 列,导致 node access log / 可观测指标等批量
落库持续报 "null value in column id violates not-null constraint"。
在 BatchInsert* 落库前为零 ID 行生成雪花 ID(与 ClickHouse 写入路径
一致),并新增单元回归与 PG 集成回归测试覆盖六张日志表。
2026-08-09 13:47:13 +08:00
ryan 0639855653 fix(openresty): 修复源站错误页「仅针对 GET 请求」未生效
error_page 的 URI 内部重定向会把请求方法改写成 GET,导致内部
Lua 中 ngx.req.get_method() 恒为 GET,get_only 判断永不命中,
POST/PUT 等请求仍返回自定义错误页。

改为命名 location(@__openflare_origin_error)承载错误页:
命名 location 保留原始请求方法与原始错误状态码,非 GET 请求
直接以原状态码退出、不再注入自定义 HTML。附带回归断言,禁止
回退到 URI 内部重定向形式。
2026-08-09 13:42:38 +08:00
ryan 0524ae1da4 chore(release): v3.5.1
### ✨ 新功能
- 日志存储解耦:ClickHouse 变为可选项,不启用时由 PostgreSQL/SQLite 承担全部日志功能;新增「切换日志数据库」任务支持 PostgreSQL/SQLite 与 ClickHouse 间数据迁移(迁移期间冻结日志写入,成功后自动切换主库并保留源数据);`log_database` / `log_db_migration` 设为受保护配置;ClickHouse 改为默认关闭。
- 新增 PostgreSQL/SQLite 日志存储实现:节点访问日志按月分区,统计查询合并为单次扫描、IP 汇总归属地取查询窗口内最新记录、WAF 按 IP 聚合减少扫描次数,并新增 `logged_at` 前导索引与主机名小写表达式索引;过期清理直接删除完全过期的整月分区,启动时兜底预建当月及未来 2 个月分区。
- 性能指标与访问日志的保留时长解耦:新增三库共用的 `metric_retention_days` 配置(默认 3 天),每日垃圾清理按独立短留存清理指标快照。

### 🛠 修复
- 修复 UptimeKuma 同步调试日志泄露凭据:Socket.IO 事件日志不再打印 payload 内容(仅记录长度),避免凭据进入日志。
- 修复日志保留天数配置继承旧键导致的误删风险:`log_retention_days_*` 不再继承 `database_auto_cleanup_retention_days`,统一默认 30 天。

### ⚡️ 优化与改进
- 系统定期垃圾清理由每 2 小时改为每日执行一次(凌晨 3 点,Asia/Shanghai),降低非必要高频扫描。

### 💄 其他/体验
- 服务工作者(SW)注入挑战页改为前台无感知:不再显示「加载中…」文案,页面空白,仅通过浏览器控制台输出 `[sw-challenge]` 调试信息,注入过程不打扰访客。
- 用户访问日志(`w_user_access_logs`)记录禁用:不再采集与写入新的用户访问日志,存量数据与管理端访问日志统计页面保留。
2026-08-09 11:39:28 +08:00
ryan adee4f7b27 docs: update 2026-08-09 11:22:35 +08:00
ryan 1d0f2d6342 fix(log): hard-set log retention days default to 30, drop legacy inheritance
log_retention_days_* 迁移不再继承旧键 database_auto_cleanup_retention_days
的值,统一默认 30 天。此前若旧键残留异常小值(如 2 天)会被静默带入,
导致升级后首次垃圾清理把大部分日志直接删掉。PG/SQLite 双方言同步修改,
文档默认值 90 -> 30。
2026-08-09 10:48:45 +08:00
ryan e3f603f72a fix(security): stop logging UptimeKuma socket payload content 2026-08-09 10:42:21 +08:00
ryan 3b010bb15e feat(log): disable user access log recording
- 移除全局用户访问日志采集中间件与批写入 writer(risk_control 包整包删除),
  不再写入 w_user_access_logs;存量数据与管理端访问日志统计页面保留
- 日志库迁移任务不再排空用户访问日志队列,状态接口不再展示其缓冲队列统计
- 迁移测试的系统配置 seed 计数断言更新为当前实际值(86 → 95),
  注释改为提示新增配置 seed 时同步更新
2026-08-09 10:35:39 +08:00
ryan f530cd4025 perf(log): optimize PG log store queries and expired partition cleanup
- Count/节点访问日志统计改为单次扫描聚合,WAF 按 IP 聚合由三次扫描合并为两次
- IPSummaries 归属地改为取过滤窗口内最新记录(对齐 ClickHouse argMax 口径),
  子查询带窗口条件,可分区裁剪并命中索引
- 新增 goose 迁移:of_node_access_logs (logged_at DESC, id DESC) 前导索引与
  lower(trim(host)) 表达式索引,加速列表排序与主机过滤
- 过期日志清理先按数据校验直接 DROP 完全过期整月分区,再对边界月逐行删除;
  启动时兜底预建当月及未来 2 个月分区,跨月停机重启后首次写入不再报
  "no partition of relation found"
2026-08-09 10:20:58 +08:00
ryan 08d28c2c8e feat(log): drop empty old-month PG partitions during cleanup
系统垃圾清理任务删除过期日志后,顺带清理旧月份空分区表:
PostgreSQL 按月分区的访问日志表(节点/用户)在数据删除后若该月
分区已无数据,则自动删除对应分区表,避免历史分区表无限累积。

- 仅删除「当前月之前」且为空的月份分区,当月/未来月及仍有数据的分区保留
- ClickHouse/SQLite 为 no-op(CH 分区随数据删除自动消失)
- 修复既有集成测试 pg_inherits 查询(inhrelid → inhparent)
- 新增单元测试与 PG 集成测试
2026-08-09 09:33:59 +08:00
ryan 34a0896ff8 chore(task): run system garbage cleanup once daily
系统定期垃圾清理 cron 由每 2 小时(0 */2 * * *)改为每日凌晨 3 点
(0 3 * * *,Asia/Shanghai),降低非必要高频扫描。新增 PG/SQLite
双方言 goose 迁移(含 Down 回滚)与迁移测试。
2026-08-09 09:25:30 +08:00
ryan 0c22e76f4b fix(frontend): optimization 2026-08-09 09:14:54 +08:00
ryan bd2183c8bb feat(log): add independent short retention for performance metrics
性能指标(CPU/内存/磁盘/网络)不再跟随 log_retention_days_*,新增三库共用
的 metric_retention_days 配置(默认 3 天),系统垃圾清理按独立短留存清理
指标快照;访问日志保留时长不变。新增 PG/SQLite 双方言 goose 迁移 seed。
2026-08-09 09:04:33 +08:00
ryan 9df2437e47 fix(config): set ClickHouse to disabled by default and update related documentation 2026-08-09 08:54:26 +08:00
ryan e8c414aa12 fix: ch migrate 2026-08-09 08:47:56 +08:00
ryan 7e8aa5fa0f Merge branch 'codex/log-database-decoupling'
# Conflicts:
#	docs/changelog/index.md
#	frontend/app/(main)/error-pages/page.tsx
#	internal/infra/persistence/migrator/migrator_test.go
2026-08-09 08:37:37 +08:00
ryan 7e518987de chore(release): v3.5.0
### 🛠 修复
- 修复 PoW 挑战页潜在 XSS 风险,状态与错误文案改用纯文本渲染,并限制跳转 URL 仅允许 http/https 协议。
- 修复邮件发送的邮件头注入风险,写入邮件头前自动清除 CR/LF 换行符(CWE-93)。
- 修复 UptimeKuma 同步调试日志泄露凭据问题,输出日志前对密码和 Token 等敏感字段打码。

### ⚡️ 优化与改进
- 新增 Service Worker 离线兜底功能,为启用 HTTPS 的网站自动下发 Service Worker 并缓存离线页,域名不可达时展示离线兜底页面。
- 重构响应页面设置,将源站错误页与 Service Worker 离线页整合至统一的「响应页面」(/responses)标签页,并增加 URL 查询参数 tab 状态同步。

### 💄 其他/体验
- 新增离线页内置预制模板套件(「极简白底」、「线框拓扑」、「包豪斯」),与源站错误页模板风格保持一致,支持编辑界面一键加载与预览。
2026-08-08 23:08:12 +08:00
ryan 61484090f9 fix: 修复源站错误页「仅针对 GET 请求」导致配置发布失败并回滚 2026-08-08 22:37:40 +08:00
ryan 94b74d72f6 fix(frontend): fallback empty offline page html in editor workspace preview 2026-08-08 21:56:44 +08:00
ryan 6487ce666d fix(security): harden PoW XSS, email header injection and UptimeKuma log redaction 2026-08-08 21:52:46 +08:00
ryan c4be34b214 fix(frontend): fallback empty offline page html to default template in preview 2026-08-08 21:52:02 +08:00
ryan fda727cf53 docs: rename contact page references to offline page in changelog
refactor(frontend): rename contact page to offline page and update routes/components

feat(frontend): add preset templates suite for offline contact page
2026-08-08 21:36:58 +08:00
ryan f083da20f2 feat(frontend): add dedicated edit and preview routes for response pages and sync tab state to url 2026-08-08 21:18:53 +08:00
ryan 9797fcdb2f fix(openflare): improve SWOfflineDomains validation and snapshot diff logic 2026-08-08 20:52:07 +08:00
Ryan 93ec3096f3 Potential fix for pull request finding
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
2026-08-08 20:48:54 +08:00
Ryan 0e34301c92 Potential fix for pull request finding
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
2026-08-08 20:48:37 +08:00
Ryan 12b5271f92 Potential fix for pull request finding
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
2026-08-08 20:47:33 +08:00
Ryan 8ee966434d Potential fix for pull request finding
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
2026-08-08 20:46:54 +08:00
ryan 6882481a56 Merge remote-tracking branch 'origin/feature/service-worker' into feature/service-worker 2026-08-08 20:44:27 +08:00
ryan b675038bba refactor(frontend): unify error pages into responses page and remove error-pages route 2026-08-08 20:43:58 +08:00
ryan d17ec9d17d fix(docs): resolve vitepress build error by escaping raw tags and excluding superpowers dir 2026-08-08 20:43:56 +08:00
ryan 8758f9a061 refactor(frontend): unify error pages into responses page and remove error-pages route 2026-08-08 20:40:05 +08:00
ryan 7d93d3d2a1 fix(log): address remaining CodeRabbit suggestions for log database switch and migrations 2026-08-08 20:36:04 +08:00
Ryan 074edf17a1 Potential fix for pull request finding
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
2026-08-08 20:30:36 +08:00
Ryan 6faf525af0 Potential fix for pull request finding
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
2026-08-08 20:30:17 +08:00
ryan a6fc2b7737 fix(log): address CodeRabbit review findings for log database decoupling 2026-08-08 20:15:36 +08:00
ryan ca21ff3a5b feat(option): add sw offline
fix(openresty): scope sw injection per cert partition

fix(lint): satisfy revive and biome format for sw offline feature

docs: sw offline scope changelog

fix(frontend): use scoped query key for sw scope zones

fix(frontend): hide preview link in sw contact page editor

feat(frontend): add sw scope domain picker and contact page fields

refactor(frontend): generalize html editor workspace for reuse

feat(openresty): scope sw offline injection by route domains

feat(openresty): add sw offline domains snapshot field

feat(option): add sw offline domains scope option

docs: fill html editor workspace generalization detail

docs: sw offline scope implementation plan

docs: sw offline scope design

test(openresty): assert single merged access block in sw enabled servers

fix(openresty): restrict sw intercept to https server blocks

fix(openresty): version sw offline cache by html content

fix(agent): escape redir in sw challenge page to prevent xss

fix(agent): return sw.runtime module table and add lua spec

docs: sw offline fallback changelog

fix(frontend): memoize option map to preserve unsaved contact page edits

feat(frontend): add response pages module with contact page tab

feat(agent): ship sw offline lua assets and placeholder substitution

feat(config): wire sw offline options into config snapshot

feat(openresty): render sw offline assets and challenge intercept

feat(openresty): add sw offline ConfigSnapshot fields and placeholder

feat(db): seed sw offline options

feat(option): add sw offline config keys and validation

docs: add service worker offline fallback implementation plan

docs: adopt global-option pattern for SW offline fallback (matches origin error page)

docs: unify offline contact page with error pages as response pages

docs: service worker offline fallback design (issue #23)
2026-08-08 20:14:28 +08:00
ryan 7d71f1e4e1 feat(log): decouple log storage from ClickHouse with switchable logstore
- New internal/repository/logstore abstraction: exported domain interfaces
  (AccessLogStore/ObservabilityStore/UserAccessLogStore/StatusStore),
  config-driven provider (Active/Build/Migrating/SetConfigReader), GORM
  implementation for PostgreSQL/SQLite (incl. hourly rollups computed in
  real time, migration listers, PG partition maintenance), and a ClickHouse
  wrapper preserving the native batch path; repository facade delegates to
  logstore; import-lint test enforces apps never import analyticsrepo.
- ClickHouse is now optional: the log DB is either the main DB (postgres
  when database.enabled, else sqlite) or clickhouse; boot validation +
  first-run seed; log_database / log_db_migration are protected keys.
- New user task 切换日志数据库 (of_log_db_switch): freeze log writes,
  drain batch writers, copy all 6 raw log tables by id (preserving IDs)
  with target-partition pre-creation for PG, flip log_database on success,
  clear the freeze flag on failure.
- Per-store retention (log_retention_days_*) with expiry cleanup folded
  into the daily system_cleanup task; legacy database_auto_cleanup_* and
  of_database_auto_cleanup decommissioned.
- goose migrations: 6 log tables in PG (2 monthly-partitioned) + SQLite,
  retention config seeds, schedule cleanup; GET
  /api/v1/admin/status/log-database endpoint; frontend retention settings,
  switch-task UI and status badge; changelog and docs updated.

docs(plan): log database decoupling implementation plan

docs(design): log database decoupling design (ClickHouse optional)
2026-08-08 19:43:01 +08:00
177 changed files with 17857 additions and 5482 deletions
+12
View File
@@ -38,6 +38,7 @@ description: "Wavelet 项目专用:根据自上一个正式版本 Tag 以来
固定使用以下分类:
```text
### 新增
### 🛠 修复
### ⚡️ 优化与改进
### 💄 其他/体验
@@ -45,15 +46,26 @@ description: "Wavelet 项目专用:根据自上一个正式版本 Tag 以来
分类规则:
- 新功能、新能力、新配置、新任务:放入 ### 新增
- Bug、异常行为、错误逻辑:放入 ### 🛠 修复
- 性能、稳定性、接口、架构、兼容性:放入 ### ⚡️ 优化与改进
- 日志、文案、UI、文档、开发体验:放入 ### 💄 其他/体验
「修复/优化」与「新增」的判定(关键):
- **判定标准是“该功能在上一正式版本中是否已存在”**:
- 已存在 → 本次对其 bug 的修正可计入「🛠 修复」,对其行为/性能的改进可计入「⚡️ 优化与改进」;
- 不存在(本版本新增)→ 该功能的一切内容——包括开发过程中修的 bug、做的性能优化、补的索引——都只属于新功能开发的一部分,不应该在发布说明中提及。
- 禁止把新功能的开发期修复/优化写进「修复」或「优化」:新功能此前版本没有,谈不上“修复/优化了旧行为”。
示例:
```
chore(release): v3.3.0
### ✨ 新功能
- 新增笔记库快照备份功能,支持定时备份与手动一键恢复(仅说明新增的功能, 禁止提及新功能开发时期的优化修复等内容)。
### 🛠 修复
- 修复了通过 MCP 接口操作时笔记库范围限制未正确生效的问题。
- 修复了 MCP 接口返回数据格式不一致的问题。
-19
View File
@@ -1,19 +0,0 @@
root = true
[*]
indent_style = space
indent_size = 4
charset = utf-8
end_of_line = lf
trim_trailing_whitespace = true
insert_final_newline = true
[*.{json,yml,yaml}]
indent_size = 2
[*.md]
insert_final_newline = false
trim_trailing_whitespace = false
[*.{js,ts,css,html,jsx,tsx,vue}]
indent_size = 2
+1 -1
View File
@@ -56,7 +56,7 @@ REDIS_MAINT_NOTIFICATIONS=false
# ─── ClickHouse(必需)────────────────────────────────────────────────────────
# CLICKHOUSE_HOST 设置后会自动启用;测试环境可显式 CLICKHOUSE_ENABLED=true 做 live 联调
CLICKHOUSE_ENABLED=true
CLICKHOUSE_ENABLED=false
# compose 内:clickhouse:9000;本机连映射端口:127.0.0.1:9000
CLICKHOUSE_HOST=clickhouse:9000
CLICKHOUSE_USERNAME=default
+61 -106
View File
@@ -2,9 +2,9 @@
# OpenFlare
**[English](./README.en.md) | [📖 中文](./README.md)**
**[📖 中文](./README.md) | [English](./README.en.md)**
OpenFlare is an open-source CDN orchestration and edge security platform. It supports reverse proxies, centralized configuration synchronization, secure intranet penetration (Tunnels), dynamic WAF protection, and anti-CC challenges.
OpenFlare is an open-source CDN orchestration and edge security platform. It supports reverse proxy, centralized configuration synchronization, in-network tunneling (Tunnels), dynamic WAF protection, and CC defense challenges.
</div>
@@ -21,37 +21,67 @@ OpenFlare is an open-source CDN orchestration and edge security platform. It sup
</p>
> [!WARNING]
> After logging in for the first time with the `root` user, make sure to change the default password `123456`.
>
> The BETA version is a temporary product for the development and testing phase. It may contain unknown issues and should not be used in production environments.
> After the first login with the `admin` user, you must change the default password `12345678`.
>
> The BETA version is a temporary product in the development and testing stage and may have unknown issues. It should not be used in production environments.
## Documentation
**https://open-flare.pages.dev**
Quick links:
Common entry points:
* [Quick Start](https://open-flare.pages.dev/en/guide/quick-start)
* [Deployment Guide](https://open-flare.pages.dev/en/deployment/deployment)
* [Quick Start](https://open-flare.pages.dev/guide/quick-start)
* [Deployment Guide](https://open-flare.pages.dev/deployment/deployment)
* [Configuration Reference](https://open-flare.pages.dev/reference/configuration)
* [System Design](https://open-flare.pages.dev/design/)
## Core Features
## Core Capabilities
* **Reverse Proxy Management**: Website rules as the aggregation boundary, supporting multi-domain binding and multi-upstream load balancing with unified management of all OpenResty node configurations.
* **Immutable Config Version Control**: Full-snapshot publish model based on version numbers (`YYYYMMDD-NNN`), with pre-publish diff preview, a single globally active version, and one-click sub-second rollback.
* **Secure Intranet Penetration (Tunnels)**: An open-source alternative to Cloudflare Tunnels. Securely expose local intranet Web services to the public network via Relay and OpenFlared clients — no public IP or open inbound ports required.
* **Edge WAF Safety Protection**: Provides global and custom rule groups, supporting manual/automatic/subscription IP groups, MaxMind GeoIP country-level access control, Checksum-based differential IP group sync (no Nginx reload), and custom block responses.
* **Anti-CC & Human-Machine Challenge (PoW)**: Built-in high-performance client-side cryptographic Proof of Work challenges (similar to Turnstile) to block and intercept botnets and scrapers at the gateway edge in seconds.
* **Pages Static Hosting**: Upload pre-built ZIP packages directly; edge Agents pull and serve them via local OpenResty, with SPA Fallback and built-in API reverse proxy configuration.
* **Automated TLS Certificate Management**: Supports dynamic certificate upload, automatic multi-domain certificate matching and binding, and ACME-based automatic issuance and renewal via Let's Encrypt.
* **Uptime Kuma Monitoring Sync**: Integrates with Uptime Kuma to automatically sync the monitoring site list using differential updates, providing real-time awareness of node availability and service health.
* **SSO Single Sign-On**: Supports GitHub OAuth and standard OIDC protocol for seamless integration with enterprise identity providers.
* **Unified Observability**: Aggregates node request metrics, real-time access log details, host/Nginx resource snapshots, health events, and a re-upload buffer for network fluctuations.
* **Reverse Proxy Configuration Management**: Uses website rules as the aggregation boundary, supports multi-domain binding and multi-upstream load balancing, and centrally manages reverse proxy configurations for all OpenResty nodes.
* **Secure In-Network Tunneling (Tunnels)**: Open-source version of Cloudflare Tunnels. No public IP or exposed inbound ports are required. Securely reverse-proxy internal web services to the public internet through Relay relay nodes and OpenFlared clients.
* **Edge WAF Security Protection**: Provides global and custom rule groups, supports manual/auto/subscription-type IP groups, MaxMind GeoIP national-level geographic access control, IP group member Checksum differential synchronization (no Nginx reload required), and custom blocking responses.
* **CC Defense and Human-Computer Challenge (PoW)**: Built-in high-performance client-side cryptography Proof of Work challenge (similar to Turnstile). Secures high-speed interception and blocking of zombie networks and crawlers at the gateway edge.
* **Pages Static Hosting**: Supports uploading or synchronizing pre-built artifacts from restricted Remote URLs or public GitHub Release assets. GitHub latest can be checked periodically and optionally auto-published. All sources are unified to generate immutable deployments, pulled by the edge Agent and served locally by OpenResty, supporting rollbacks, SPA Fallback, and API reverse proxy.
* **TLS Certificate Automation**: Supports dynamic certificate uploads, automatic multi-domain certificate matching and binding, and automatic issuance and renewal of certificates from Let's Encrypt via the ACME protocol.
* **Uptime Kuma Monitoring Synchronization**: Integrated with Uptime Kuma to automatically perform differential synchronization of monitoring site lists, real-time awareness of node availability and service status.
* **SSO Single Sign-On**: Supports GitHub OAuth and standard OIDC protocol for seamless integration with enterprise identity providers to achieve unified login.
* **Unified Observability**: Aggregates node request metrics, real-time access log details, host and Nginx resource snapshots, health events, and network fluctuation replenishment buffers.
## Interface Preview
### Dashboard Overview
![OpenFlare dashboard overview](./docs/assets/readme/dashboard-overview.png)
### Access Logs
![OpenFlare version release](./docs/assets/readme/domain_overview.png)
### WAF Protection
![OpenFlare version release](./docs/assets/readme/waf.png)
## Quick Start
### 1. Launch Server
### Hardware Configuration Recommendations
| Component | Minimum Hardware Requirements | Recommended Hardware Requirements | Notes |
|------------------------|-----------------------------------|-----------------------------------|-------|
| **Server Control Plane** | 1 CPU core / 2 GB RAM / 20 GB disk | 2 CPU cores / 4 GB RAM / 50 GB+ disk | Disk usage should be expanded reasonably based on access log retention duration and concurrent traffic |
| **Agent Data Plane** | 1 CPU core / 512 MB RAM / 2 GB disk | 2 CPU cores / 2 GB RAM / 10 GB+ disk | Expanded based on OpenResty concurrent proxy connections and WAF interception processing |
| **Relay Relay Node** | 1 CPU core / 1 GB RAM / 5 GB disk | 2 CPU cores / 2 GB RAM / 20 GB disk | frps transmission relay throughput is mainly limited by bandwidth and CPU throughput |
| **OpenFlared Client** | 1 CPU core / 256 MB RAM / 1 GB disk | 1 CPU core / 512 MB RAM / 5 GB disk | Runs independently on the internal network with extremely low resource consumption; only network throughput needs to be guaranteed |
### 1. Start the Server
Use `docker-compose`:
```bash
# Download environment variable template and create .env file
curl -o .env.example https://raw.githubusercontent.com/Rain-kl/OpenFlare/refs/heads/main/.env.example
cp .env.example .env
```
```yaml
services:
@@ -70,8 +100,6 @@ services:
condition: service_healthy
redis:
condition: service_healthy
clickhouse:
condition: service_healthy
postgres:
image: postgres:17-alpine
@@ -101,118 +129,45 @@ services:
retries: 5
start_period: 5s
clickhouse:
image: clickhouse/clickhouse-server:25.3-alpine
restart: unless-stopped
environment:
CLICKHOUSE_DB: ${CLICKHOUSE_NAME:-openflare}
CLICKHOUSE_USER: ${CLICKHOUSE_USERNAME:-default}
CLICKHOUSE_PASSWORD: ${CLICKHOUSE_PASSWORD:-replace-with-clickhouse-password}
CLICKHOUSE_DEFAULT_ACCESS_MANAGEMENT: 1
TZ: ${TZ:-Asia/Shanghai}
volumes:
- openflare_clickhouse_data:/var/lib/clickhouse
healthcheck:
test: ["CMD", "clickhouse-client", "--user", "${CLICKHOUSE_USERNAME:-default}", "--password", "${CLICKHOUSE_PASSWORD:-replace-with-clickhouse-password}", "--query", "SELECT 1"]
interval: 10s
timeout: 5s
retries: 5
start_period: 15s
volumes:
openflare_uploads:
openflare_postgres_data:
openflare_redis_data:
openflare_clickhouse_data:
```
```bash
docker compose up -d
```
See the [deployment documentation](https://open-flare.pages.dev/deployment/deployment) for details.
Access at: `http://localhost:3000`
Access address: `http://localhost:3000`
Default credentials:
Default account:
* Username: `root`
* Password: `123456`
* Username: `admin`
* Password: `12345678`
### 2. Install Agent
Before installing an Agent, please install OpenResty on the target node first, or use the Agent Docker image with OpenResty built-in.
Before installing the Agent, first install OpenResty on the node or use the built-in OpenResty Agent Docker image.
You can copy the installation command from **Node Management -> Details -> Node Info -> Node Token & Deployment** in the control panel, or directly use the scripts below:
You can copy the installation command from the control panel's **Nodes Management -> Details -> Node Information -> Node ID and Deployment**, or use the script below:
#### Docker Deployment
For Docker deployment, you can directly run the Agent image:
Docker deployment can directly run the Agent image:
```bash
docker pull ghcr.io/rain-kl/openflare-agent:latest
docker rm -f openflare-agent 2>/dev/null || true
docker run -d --name openflare-agent --restart unless-stopped \
-p 80:80 -p 443:443/tcp -p 443:443/udp \
-v openflare-agent-pages:/data/var/lib/openflare/pages \
-e OPENFLARE_SERVER_URL=http://your-server:3000 \
-e OPENFLARE_AGENT_TOKEN=YOUR_AGENT_TOKEN \
ghcr.io/rain-kl/openflare-agent:latest
```
#### Local Installation
## Open Source License
Using `discovery_token` to register:
```bash
curl -fsSL https://raw.githubusercontent.com/Rain-kl/OpenFlare/main/scripts/install-agent.sh | bash -s -- \
--server-url http://your-server:3000 \
--discovery-token YOUR_DISCOVERY_TOKEN
```
Using node-specific `agent_token`:
```bash
curl -fsSL https://raw.githubusercontent.com/Rain-kl/OpenFlare/main/scripts/install-agent.sh | bash -s -- \
--server-url http://your-server:3000 \
--agent-token YOUR_AGENT_TOKEN
```
The installation script defaults to `/opt/openflare-agent`, creates a `openflare-agent.service`, automatically searches for `openresty`, and can be executed repeatedly to reinstall or upgrade the Agent.
### 3. Uninstall Agent
To completely uninstall the Agent and clear local data, run:
```bash
curl -fsSL https://raw.githubusercontent.com/Rain-kl/OpenFlare/main/scripts/uninstall-agent.sh | bash
```
The uninstallation script will stop and remove the `openflare-agent.service`, and delete the entire `/opt/openflare-agent` directory. It will not delete the local OpenResty installation.
### 4. Publish Your First Configuration
1. Log in to the management panel and add a reverse proxy rule.
2. View the preview or change summary before publishing.
3. Activate the new version.
4. Agents will receive the configuration and apply it via WebSocket notification or subsequent heartbeats.
The version number format is fixed as `YYYYMMDD-NNN`. Historical versions are immutable, and rollback is achieved by reactivating an older version.
## UI Preview
### Dashboard Overview
![OpenFlare dashboard overview](./docs/assets/readme/dashboard-overview.png)
### Node Details
![OpenFlare node detail](./docs/assets/readme/node-detail.png)
### Proxy Configuration
![OpenFlare version release](./docs/assets/readme/proxy-route-detail.png)
## License
This project is licensed under [Apache License 2.0](./LICENSE).
This project is licensed under the [Apache License 2.0](./LICENSE).
## Star History
+13 -35
View File
@@ -54,16 +54,25 @@ OpenFlare 是开源 CDN 编排与边缘安全平台。它支持反向代理、
![OpenFlare dashboard overview](./docs/assets/readme/dashboard-overview.png)
### 节点详情
### 访问日志
![OpenFlare node detail](./docs/assets/readme/node-detail.png)
![OpenFlare version release](./docs/assets/readme/domain_overview.png)
### 配置新增
### WAF 防护
![OpenFlare version release](./docs/assets/readme/proxy-route-detail.png)
![OpenFlare version release](./docs/assets/readme/waf.png)
## 快速开始
### 硬件配置推荐
| 组件 | 最低硬件配额 | 推荐硬件配额 | 说明 |
| --- |-------------------------------| --- | --- |
| **Server 控制面** | 1 核 CPU / 2 GB 内存 / 20 GB 磁盘 | 2 核 CPU / 4 GB 内存 / 50 GB+ 磁盘 | 磁盘用量需根据访问日志留存时长与并发流量合理扩容 |
| **Agent 数据面** | 1 核 CPU / 512 MB 内存 / 2 GB 磁盘 | 2 核 CPU / 2 GB 内存 / 10 GB+ 磁盘 | 根据 OpenResty 的并发代理连接量与 WAF 拦截处理扩容 |
| **Relay 中继节点**| 1 核 CPU / 1 GB 内存 / 5 GB 磁盘 | 2 核 CPU / 2 GB 内存 / 20 GB 磁盘 | frps 传输中继吞吐量主要受带宽与 CPU 吞吐能力限制 |
| **OpenFlared 客户端**| 1 核 CPU / 256 MB 内存 / 1 GB 磁盘 | 1 核 CPU / 512 MB 内存 / 5 GB 磁盘 | 独立运行于内网,自身资源占用极小,保障网络吞吐即可 |
### 1. 启动 Server
使用 docker-compose
@@ -72,11 +81,6 @@ OpenFlare 是开源 CDN 编排与边缘安全平台。它支持反向代理、
# 下载环境变量模板并创建 .env 文件
curl -o .env.example https://raw.githubusercontent.com/Rain-kl/OpenFlare/refs/heads/main/.env.example
cp .env.example .env
# ClickHouse 服务端:curl performance.xml 到 ./config/clickhouse,并以单文件方式挂载到 config.d
mkdir -p ./config/clickhouse
curl -fsSL -o ./config/clickhouse/performance.xml \
https://raw.githubusercontent.com/Rain-kl/OpenFlare/refs/heads/main/config/clickhouse/performance.xml
```
```yaml
@@ -96,8 +100,6 @@ services:
condition: service_healthy
redis:
condition: service_healthy
clickhouse:
condition: service_healthy
postgres:
image: postgres:17-alpine
@@ -127,34 +129,10 @@ services:
retries: 5
start_period: 5s
clickhouse:
image: clickhouse/clickhouse-server:25.3-alpine
restart: unless-stopped
environment:
CLICKHOUSE_DB: ${CLICKHOUSE_NAME:-openflare}
CLICKHOUSE_USER: ${CLICKHOUSE_USERNAME:-default}
CLICKHOUSE_PASSWORD: ${CLICKHOUSE_PASSWORD:-replace-with-clickhouse-password}
CLICKHOUSE_DEFAULT_ACCESS_MANAGEMENT: 1
TZ: ${TZ:-Asia/Shanghai}
ulimits:
nofile:
soft: 262144
hard: 262144
volumes:
- openflare_clickhouse_data:/var/lib/clickhouse
- ./config/clickhouse/performance.xml:/etc/clickhouse-server/config.d/performance.xml:ro
healthcheck:
test: ["CMD", "clickhouse-client", "--user", "${CLICKHOUSE_USERNAME:-default}", "--password", "${CLICKHOUSE_PASSWORD:-replace-with-clickhouse-password}", "--query", "SELECT 1"]
interval: 10s
timeout: 5s
retries: 5
start_period: 15s
volumes:
openflare_uploads:
openflare_postgres_data:
openflare_redis_data:
openflare_clickhouse_data:
```
详细部署说明见 [部署文档](https://open-flare.pages.dev/deployment/deployment)。
+4 -2
View File
@@ -99,10 +99,12 @@ otel:
tracer_name: "github.com/Rain-kl/OpenFlare" # Global tracer instrumentation name
# ─── ClickHouse (required) ──────────────────────────────────────────────────────
# ─── ClickHouse (optional) ─────────────────────────────────────────────────────
# Analytics / observability OLAP store. Telemetry writes are best-effort (async batch).
# 默认关闭:缺失本配置块或 enabled: false 时不启用 ClickHouse,日志/指标由主库承担;
# 设置 CLICKHOUSE_HOST 或 CLICKHOUSE_ENABLED=true 可经环境变量启用。
clickhouse:
enabled: true
enabled: false
hosts:
- "127.0.0.1:9000" # compose 内应用可用 clickhouse:9000(经 CLICKHOUSE_HOST)
username: "default"
+1 -1
View File
@@ -110,7 +110,7 @@ services:
- ./data/agent/:/data
environment:
OPENFLARE_SERVER_URL: "http://host.docker.internal:3000"
OPENFLARE_AGENT_TOKEN: "af2fb112f36a0055ec25dd164c908fea"
OPENFLARE_AGENT_TOKEN: "7c7c4c13df0f3a77866bcd8cde492610"
LOG_LEVEL: "debug"
extra_hosts:
- "host.docker.internal:host-gateway"
+2 -1
View File
@@ -14,7 +14,8 @@ export default defineConfig({
'components/**',
'snippets/**',
'plan/**',
'guideline/**'
'guideline/**',
'superpowers/**'
],
markdown: {
Binary file not shown.

After

Width:  |  Height:  |  Size: 58 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 67 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 77 KiB

+43 -8
View File
@@ -11,14 +11,49 @@ sidebar: false
## 重大变更
> [!IMPORTANT]
>
> 3.1.2 版本更新了 CLickHouse 部署配置。
>
> 3.0.0 版本为 Wavelet 平台迁移与架构重构版本,涉及数据库表结构、环境变量以及前后端底层架构的重大变更。请务必在升级前备份数据库,并且更新到 V2.3.4。
> 目前已知的兼容性问题:
>
> - Pages 无法迁移, 升级前请先手动下载并备份 Pages 静态站点的 ZIP 包,升级后重新创建。
> - 性能调优参数重置, 升级后请重新配置
>
>3.5.1 版本解耦了日志存储,ClickHouse 变为可选项,如果想切换数据库, 点击 「任务管理」 -> 「切换日志数据库」任务,按提示迁移数据并切换主库。
>
## [Unreleased]
### 🛠 修复
- 修复源站错误页「仅针对 GET 请求」未生效:`error_page` 内部重定向会把请求方法改写成 GET,导致内部 Lua 无法识别 POST/PUT 等原始方法、仍返回自定义错误页;现改为命名 location(`@__openflare_origin_error`)承载错误页,保留原始请求方法与错误状态码,非 GET 请求不再返回自定义错误页。
- 修复 PostgreSQL 作为日志库时节点访问日志/可观测指标/用户访问日志批量写入失败:GORM 对零值 `uint64` 主键会省略 `id` 列,而 PG 日志表 `id` 无默认值,导致持续报「null value in column id violates not-null constraint」;现于落库前为零 ID 行生成雪花 ID(与 ClickHouse 写入路径一致),并新增回归测试覆盖六张日志表。
## [v3.5.1] - 2026-08-09
### 新增
- 日志存储解耦:ClickHouse 变为可选项,不启用时由 PostgreSQL/SQLite 承担全部日志功能;新增「切换日志数据库」任务支持 PostgreSQL/SQLite 与 ClickHouse 间数据迁移(迁移期间冻结日志写入,成功后自动切换主库并保留源数据);`log_database` / `log_db_migration` 设为受保护配置;ClickHouse 改为默认关闭。
- 新增 PostgreSQL/SQLite 日志存储实现:节点访问日志按月分区,统计查询合并为单次扫描、IP 汇总归属地取查询窗口内最新记录、WAF 按 IP 聚合减少扫描次数,并新增 `logged_at` 前导索引与主机名小写表达式索引;过期清理直接删除完全过期的整月分区,启动时兜底预建当月及未来 2 个月分区。
- 性能指标与访问日志的保留时长解耦:新增三库共用的 `metric_retention_days` 配置(默认 3 天),每日垃圾清理按独立短留存清理指标快照。
### 🛠 修复
- 修复 UptimeKuma 同步调试日志泄露凭据:Socket.IO 事件日志不再打印 payload 内容(仅记录长度),避免凭据进入日志。
- 修复日志保留天数配置继承旧键导致的误删风险:`log_retention_days_*` 不再继承 `database_auto_cleanup_retention_days`,统一默认 30 天。
### ⚡️ 优化与改进
- 系统定期垃圾清理由每 2 小时改为每日执行一次(凌晨 3 点,Asia/Shanghai),降低非必要高频扫描。
### 💄 其他/体验
- 服务工作者(SW)注入挑战页改为前台无感知:不再显示「加载中…」文案,页面空白,仅通过浏览器控制台输出 `[sw-challenge]` 调试信息,注入过程不打扰访客。
- 用户访问日志(`w_user_access_logs`)记录禁用:不再采集与写入新的用户访问日志,存量数据与管理端访问日志统计页面保留。
## [v3.5.0] - 2026-08-08
### 🛠 修复
- 修复 PoW 挑战页潜在 XSS 风险,状态与错误文案改用纯文本渲染,并限制跳转 URL 仅允许 http/https 协议。
- 修复邮件发送的邮件头注入风险,写入邮件头前自动清除 CR/LF 换行符(CWE-93)。
- 修复 UptimeKuma 同步调试日志泄露凭据问题,输出日志前对密码和 Token 等敏感字段打码。
### ⚡️ 优化与改进
- 新增 Service Worker 离线兜底功能,为启用 HTTPS 的网站自动下发 Service Worker 并缓存离线页,域名不可达时展示离线兜底页面。
- 重构响应页面设置,将源站错误页与 Service Worker 离线页整合至统一的「响应页面」(/responses)标签页,并增加 URL 查询参数 tab 状态同步。
### 💄 其他/体验
- 新增离线页内置预制模板套件(「极简白底」、「线框拓扑」、「包豪斯」),与源站错误页模板风格保持一致,支持编辑界面一键加载与预览。
## [v3.4.5] - 2026-08-08
+7 -80
View File
@@ -136,55 +136,6 @@ docker run -d --name openflare-agent --restart unless-stopped \
> **Pages 持久化**
> 默认将 Pages 部署目录挂载到 Docker 命名卷 `openflare-agent-pages`(容器内路径 `/data/var/lib/openflare/pages`)。重建或升级 Agent 容器时无需重新拉取静态站点包。
> [!NOTE]
> **非 Root 安全加固运行**
> Agent 容器内部已完成安全加固,在启动后会统一以低权限非 root 用户 `openflare` 运行。
> 容器已内置了 `cap_net_bind_service` 内核能力,使得低权限进程依然能够正常监听宿主机的 `80` 和 `443` 特权端口。
> 同时,OpenResty 运行时所需的各种临时路径(包括 PID 路径、各类临时缓存目录如 `client_body_temp_path`、`proxy_temp_path` 等)都由 Agent 控制器动态渲染并自动重定向至容器内的 `/data` 目录,彻底避免在非 root 权限运行时写入默认系统路径而导致的权限拒绝错误(Permission Denied)。
> 具体物理缓存写入路径为:
> * 临时缓存目录:`/data/var/cache/nginx`
> * 代理缓存目录:`/data/var/cache/openflare_proxy`
## 启动与验证
systemd 环境:
```bash
systemctl status openflare-agent
journalctl -u openflare-agent -f
```
手动启动:
```bash
/opt/openflare-agent/openflare-agent -config /opt/openflare-agent/agent.json
```
源码运行:
```bash
export LOG_LEVEL='info'
go run ./cmd/agent -config /path/to/agent.json
```
编译后二进制运行:
```bash
go build -o openflare-agent ./cmd/agent
export LOG_LEVEL='info'
./openflare-agent -config /path/to/agent.json
```
在管理端确认:
| 位置 | 期望结果 |
| --- | --- |
| 节点列表 | 节点在线 |
| 节点详情 | 能看到心跳时间、当前版本和基础资源信息 |
| 应用记录 | 发布配置后出现应用结果 |
## 卸载
### 交互式卸载 (推荐)
@@ -195,38 +146,14 @@ export LOG_LEVEL='info'
curl -fsSL https://raw.githubusercontent.com/Rain-kl/OpenFlare/main/scripts/uninstall-agent.sh | bash
```
### 自动化 (非交互式) 卸载
### 卸载
使用命令行传参进行无人值守卸载。
本地卸载(默认):
```bash
curl -fsSL https://raw.githubusercontent.com/Rain-kl/OpenFlare/main/scripts/uninstall-agent.sh | bash -s -- --install-dir /opt/openflare-agent
```
Docker 容器卸载:
```bash
curl -fsSL https://raw.githubusercontent.com/Rain-kl/OpenFlare/main/scripts/uninstall-agent.sh | bash -s -- --docker
```
支持参数:
| 参数 | 说明 |
| --- | --- |
| `--install-dir` | 安装目录,默认 `/opt/openflare-agent`(仅本地卸载生效) |
| `--service-name` | systemd 服务名,默认 `openflare-agent`(仅本地卸载生效) |
| `--docker` | 使用 Docker 容器方式卸载 |
| `--method` | 卸载方式,可选 `local` 或 `docker`(默认 `local`) |
本地卸载只会移除 Agent 服务、进程和安装目录,不会删除本机 OpenResty。Docker 卸载会停止并删除 `openflare-agent` 容器,交互模式下还可以选择是否清理对应的 Docker 镜像。
停止并删除 `openflare-agent` 容器即可
## 常见问题
| 现象 | 处理步骤 |
| --- | --- |
| `agent_token 和 discovery_token 不能同时为空` | 检查 `agent.json` 至少配置了一个 Token |
| 节点一直离线 | 在 Agent 节点执行 `curl -I http://your-server:3000`,确认 Server 地址可达 |
| OpenResty 没有启动 | 查看 `journalctl -u openflare-agent`,确认 `openresty_path` 可执行,80/443 端口未被占用,且运行用户(如 `openflare`)对数据目录具有读写权限 |
| 发布后重复失败 | Agent 会阻断同一 `version + checksum` 的重复应用;需要修正配置后重新发布,或激活旧版本回滚 |
| 现象 | 处理步骤 |
| --- |---------------------------------------------------------------------------------------------------------|
| `agent_token 和 discovery_token 不能同时为空` | 检查 `agent.json` 至少配置了一个 Token |
| 节点一直离线 | 在 Agent 节点执行 `curl -I http://your-server:3000`,确认 Server 地址可达 |
| 发布后重复失败 | Agent 会阻断同一 `version + checksum` 的重复应用;在节点尝试强制同步,或者重新发布版本 |
+4 -62
View File
@@ -47,33 +47,13 @@ Internal Service (192.168.x.x)
## 前置条件
Server:
| 项目 | 要求 |
| --- | --- |
| Go | `1.25+`,仅源码运行需要 |
| Node.js | `18+`,仅源码构建管理端需要 |
| 数据库 | 可写 SQLite 文件目录,或可访问的 PostgreSQL 实例 |
| 端口 | 默认监听 `3000` |
Agent:
| 项目 | 要求 |
| --- | --- |
| 系统 | 安装脚本支持 Linux 和 macOS;systemd 服务仅在 Linux + systemd 环境创建 |
| 架构 | `amd64` 或 `arm64` |
| OpenResty | 本地部署需要可执行 `openresty`,或通过 `--openresty-path` 指定路径 |
| Docker | 仅 Docker 部署 Agent 镜像时需要 |
| 网络 | Agent 节点必须能访问 Server 地址 |
| GeoIP | WAF 地域规则使用 OpenResty 读取本地 MaxMind mmdb;镜像内置文件或首次下载,Agent 负责周期更新 |
### 硬件配置推荐
| 组件 | 最低硬件配额 | 推荐硬件配额 | 说明 |
| --- | --- | --- | --- |
| **Server 控制面** | 1 核 CPU / 1 GB 内存 / 10 GB 磁盘 | 2 核 CPU / 4 GB 内存 / 50 GB+ 磁盘 | 磁盘用量需根据访问日志留存时长与并发流量合理扩容 |
| 组件 | 最低硬件配额 | 推荐硬件配额 | 说明 |
| --- |-------------------------------| --- | --- |
| **Server 控制面** | 1 核 CPU / 2 GB 内存 / 20 GB 磁盘 | 2 核 CPU / 4 GB 内存 / 50 GB+ 磁盘 | 磁盘用量需根据访问日志留存时长与并发流量合理扩容 |
| **Agent 数据面** | 1 核 CPU / 512 MB 内存 / 2 GB 磁盘 | 2 核 CPU / 2 GB 内存 / 10 GB+ 磁盘 | 根据 OpenResty 的并发代理连接量与 WAF 拦截处理扩容 |
| **Relay 中继节点**| 1 核 CPU / 1 GB 内存 / 5 GB 磁盘 | 2 核 CPU / 2 GB 内存 / 20 GB 磁盘 | frps 传输中继吞吐量主要受带宽与 CPU 吞吐能力限制 |
| **Relay 中继节点**| 1 核 CPU / 1 GB 内存 / 5 GB 磁盘 | 2 核 CPU / 2 GB 内存 / 20 GB 磁盘 | frps 传输中继吞吐量主要受带宽与 CPU 吞吐能力限制 |
| **OpenFlared 客户端**| 1 核 CPU / 256 MB 内存 / 1 GB 磁盘 | 1 核 CPU / 512 MB 内存 / 5 GB 磁盘 | 独立运行于内网,自身资源占用极小,保障网络吞吐即可 |
## Docker Compose 部署 Server
@@ -169,41 +149,3 @@ curl -fsSL https://raw.githubusercontent.com/Rain-kl/OpenFlare/main/scripts/inst
systemctl status openflare-agent
journalctl -u openflare-agent -f
```
## 手动运行 Agent
源码运行:
```bash
export LOG_LEVEL='info'
go run ./cmd/agent -config /path/to/agent.json
```
编译后二进制运行:
```bash
go build -o openflare-agent ./cmd/agent
export LOG_LEVEL='info'
./openflare-agent -config /path/to/agent.json
```
最小 `agent.json` 示例:
```json
{
"server_url": "http://127.0.0.1:3000",
"agent_token": "replace-with-node-auth-token",
"data_dir": "./data",
"openresty_path": "openresty",
"heartbeat_interval": 3000,
"request_timeout": 10000
}
```
未配置 `openresty_path` 时,Agent 默认调用 `openresty`。
默认情况下,Agent 在 HTTP 心跳成功后会尝试升级为 WebSocket。升级成功时,Server 发布或激活配置会立即通知 Agent;如果 WebSocket 无法建立或意外断开,Agent 会自动退回 HTTP 心跳同步。
WAF 地域规则依赖 Agent 本地 `GeoLite2-Country.mmdb` / `GeoLite2-City.mmdb`(OpenResty `resty.maxminddb` 读磁盘路径)。Docker 镜像会将 MMDB COPY 到 `data_dir/etc/openflare/`;裸二进制安装时若文件缺失则首次启动按配置 URL 下载。Agent 按配置周期尝试更新;更新失败只记录警告,不影响配置同步与 OpenResty reload。MMDB **不**再嵌入 agent 二进制。Server 控制面可选 MaxMind 提供方仍**仅内嵌 Country**(约 9MB,不含 City)用于离线 seed。
+1 -35
View File
@@ -34,7 +34,7 @@
---
## Docker 运行(推荐)
## Docker 运行
Docker 部署是内网运行最简单也最安全的方式。官方的 `openflared` 镜像已经内置了客户端控制器以及 `frpc v0.69.0` 二进制运行时,无需额外搭建环境。
@@ -51,40 +51,6 @@ docker run -d --name openflared --restart unless-stopped \
---
## 宿主机手动运行
如果您需要直接在内网的 Linux/macOS/Windows 宿主机上独立运行:
### 1. 编译二进制
```bash
go build -o bin/flared ./cmd/flared
```
### 2. 准备 `flared.json`
在程序同级目录下创建 `flared.json` 配置文件:
```json
{
"server_url": "http://your-server-ip:3000",
"tunnel_token": "your-tunnel-auth-token",
"frpc_path": "/usr/local/bin/frpc",
"data_dir": "./data",
"heartbeat_interval": "10s",
"sync_interval": "30s"
}
```
### 3. 运行服务
```bash
export LOG_LEVEL='info'
./flared -config ./flared.json
```
---
## 启动与验证
### 1. 自动同步逻辑
+3 -41
View File
@@ -15,7 +15,7 @@
- 必须确保 `bindPort`(frpc 连接端口,默认 `7000`)可被公网/内网客户端访问。
- 必须确保 `vhostHTTPPort`(HTTP Vhost 端口,默认 `8080`)处于空闲状态,Agent 将在此端口上与 frps 进行流量传递。
3. **软件依赖**(仅限宿主机直接部署):
- 本地需有可执行的 `frps` 二进制文件(建议版本为 `v0.61.0+` 或最新稳定版 `v0.69.0`),或通过参数显式指定路径。
- 本地需有可执行的 `frps` 二进制文件,或通过参数显式指定路径。
---
@@ -40,9 +40,9 @@
---
## Docker 运行(推荐)
## Docker 运行)
Docker 运行是 TunnelRelay 节点最便捷的部署方案。官方镜像内置了 `openflare-relay` 控制器与 `frps v0.69.0` 运行时,开箱即用。
Docker 运行是 TunnelRelay 节点最便捷的部署方案。官方镜像内置了 `openflare-relay` 控制器与 `frps` 运行时,开箱即用。
```bash
docker pull ghcr.io/rain-kl/openflare-relay:latest
@@ -67,39 +67,6 @@ docker run -d --name openflare-relay --restart unless-stopped \
---
## 宿主机手动运行
如果您倾向于在物理机或虚拟机上直接运行:
### 1. 编译二进制
```bash
go build -o bin/openflare-relay ./cmd/relay
```
### 2. 准备 `relay.json`
在程序同级目录下创建 `relay.json` 配置文件:
```json
{
"server_url": "http://127.0.0.1:3000",
"agent_token": "your-relay-node-agent-token",
"frps_path": "/usr/local/bin/frps",
"data_dir": "./data",
"heartbeat_interval": "10s",
"request_timeout": "10s"
}
```
### 3. 运行服务
```bash
export LOG_LEVEL='info'
./openflare-relay -config ./relay.json
```
---
## 启动与验证
@@ -110,11 +77,6 @@ export LOG_LEVEL='info'
docker logs -f openflare-relay
```
如果是在 Linux 上通过 Systemd 托管的,可执行:
```bash
journalctl -u openflare-relay -f
```
### 2. 验证运行状态
启动成功后,Relay 将进行以下工作:
+13 -151
View File
@@ -6,7 +6,8 @@ OpenFlare Server 是 Gin + GORM 单体控制面,负责管理端 UI、管理 AP
> [!IMPORTANT]
> **关于外部依赖**:
> OpenFlare 系统内建了对后台异步任务(Asynq 框架)及海量节点日志分析与度量指标(观测面板)的支持。因此,**无论采用何种部署模式,系统都必须依赖 Redis(或 Valkey)与 ClickHouse 的运行**。各个部署方案的主要差异在于主关系型数据库的选择(SQLite vs PostgreSQL)以及是否启用链路追踪服务(Jaeger)。
> OpenFlare 系统内建了对后台异步任务(Asynq 框架)的支持。因此,**无论采用何种部署模式,系统都必须依赖 Redis(或 Valkey)**。各个部署方案的主要差异在于主关系型数据库的选择(SQLite vs PostgreSQL)以及是否启用链路追踪服务(Jaeger)。
> 若业务流量过大, 建议使用 ClickHouse 存储日志。
> [!TIP]
> **ClickHouse 服务端性能配置(推荐挂载)**
@@ -37,11 +38,11 @@ volumes:
使用 Docker 部署可以免去本地配置 Go 与 Node.js 前端构建环境的麻烦。根据你的服务器硬件配置及业务需求,你可以选择以下三种方案之一:
### 1. 快速启动 (SQLite + Redis + ClickHouse)
### 1. 快速启动 (SQLite + Redis)
> **适用场景**:测试体验、轻量化单机部署。
>
> **特点**:主关系型数据库使用内建的 SQLite 文件
> **特点**:主关系型数据库使用 SQLite
创建 `docker-compose.yaml` 文件:
@@ -70,8 +71,6 @@ services:
depends_on:
redis:
condition: service_healthy
clickhouse:
condition: service_healthy
redis:
image: valkey/valkey:8.0-alpine
@@ -84,47 +83,13 @@ services:
interval: 10s
timeout: 5s
retries: 5
clickhouse:
image: clickhouse/clickhouse-server:25.3-alpine
restart: unless-stopped
environment:
CLICKHOUSE_DB: openflare
CLICKHOUSE_USER: default
CLICKHOUSE_PASSWORD: 123456
CLICKHOUSE_DEFAULT_ACCESS_MANAGEMENT: 1
TZ: Asia/Shanghai
ulimits:
nofile:
soft: 262144
hard: 262144
volumes:
- ./data/clickhouse_data:/var/lib/clickhouse
- ./config/clickhouse/performance.xml:/etc/clickhouse-server/config.d/performance.xml:ro
healthcheck:
test: ["CMD", "clickhouse-client", "--user", "default", "--password", "123456", "--query", "SELECT 1"]
interval: 10s
timeout: 5s
retries: 5
start_period: 15s
```
运行启动命令:
```bash
mkdir -p ./config/clickhouse
curl -fsSL -o ./config/clickhouse/performance.xml \
https://raw.githubusercontent.com/Rain-kl/OpenFlare/refs/heads/main/config/clickhouse/performance.xml
docker compose up -d
```
---
### 2. 生产推荐 (PostgreSQL + Redis + ClickHouse)
### 2. 小流量业务场景 (PostgreSQL + Redis)
> **适用场景**:生产环境、多节点集群管理、高并发高可用要求。
>
> **特点**:完全分层架构。启用专用的 PostgreSQL 服务作为主关系数据库,Redis 负责高并发分布式锁、会话缓存与异步队列,ClickHouse 承载海量日志异步 Flush 与观测指标。
> **适用场景**:生产环境、业务流量中小, PostgreSQL 不会成为日志记录的瓶颈。
创建 `docker-compose.yaml` 文件:
@@ -145,8 +110,6 @@ services:
condition: service_healthy
redis:
condition: service_healthy
clickhouse:
condition: service_healthy
postgres:
image: postgres:17-alpine
@@ -176,45 +139,18 @@ services:
retries: 5
start_period: 5s
clickhouse:
image: clickhouse/clickhouse-server:25.3-alpine
restart: unless-stopped
environment:
CLICKHOUSE_DB: ${CLICKHOUSE_NAME:-openflare}
CLICKHOUSE_USER: ${CLICKHOUSE_USERNAME:-default}
CLICKHOUSE_PASSWORD: ${CLICKHOUSE_PASSWORD:-replace-with-clickhouse-password}
CLICKHOUSE_DEFAULT_ACCESS_MANAGEMENT: 1
TZ: ${TZ:-Asia/Shanghai}
ulimits:
nofile:
soft: 262144
hard: 262144
volumes:
- openflare_clickhouse_data:/var/lib/clickhouse
- ./config/clickhouse/performance.xml:/etc/clickhouse-server/config.d/performance.xml:ro
healthcheck:
test: ["CMD", "clickhouse-client", "--user", "${CLICKHOUSE_USERNAME:-default}", "--password", "${CLICKHOUSE_PASSWORD:-replace-with-clickhouse-password}", "--query", "SELECT 1"]
interval: 10s
timeout: 5s
retries: 5
start_period: 15s
volumes:
openflare_uploads:
openflare_postgres_data:
openflare_redis_data:
openflare_clickhouse_data:
```
创建对应的 `.env` 文件来配置系统环境变量(可复制并修改根目录下的 `.env.example`):
```bash
mkdir -p ./config/clickhouse
curl -fsSL -o ./config/clickhouse/performance.xml \
https://raw.githubusercontent.com/Rain-kl/OpenFlare/refs/heads/main/config/clickhouse/performance.xml
curl -o .env.example https://raw.githubusercontent.com/Rain-kl/OpenFlare/refs/heads/main/.env.example
cp .env.example .env
# 编辑 .env 文件,填入对应的数据库、Redis、ClickHouse 连接地址、密码与 APP_SESSION_SECRET
# 编辑 .env 文件,填入对应的数据库、Redis、密码与 APP_SESSION_SECRET
docker compose up -d
```
@@ -223,9 +159,9 @@ docker compose up -d
### 3. 进阶版 (含 Jaeger 链路追踪的完整编排)
> **适用场景**:开发者调试、系统深度性能诊断、高级可观测性追溯。
> **适用场景**:大流量场景, 需要进行链路性能指标追踪。
>
> **特点**:在“生产推荐”全家桶的基础上,联动拉起 Jaeger 作为 OpenTelemetry (OTel) 链路追踪的后端,收集 Server 运行时各个 API 请求的 Span Trace 信息。
> **特点**:在“生产推荐”全家桶的基础上,使用 ClickHouse 存储日志, 联动 Jaeger 作为 OpenTelemetry (OTel) 链路追踪的后端。
创建 `docker-compose.yaml` 文件:
@@ -340,64 +276,6 @@ docker compose up -d
---
## 方式二:本地部署 (源码/二进制启动)
如果你不希望使用 Docker,也可以直接在本地或虚拟机上从源码构建和运行 Server。由于后台异步任务和可观测指标分析为系统核心防线,**本地部署时依然需要连接外部 Redis 与 ClickHouse 实例**。
### 前置条件
| 项目 | 要求 |
| --- | --- |
| Go | `1.25+` |
| Node.js | `18+` |
| pnpm | 推荐通过 `corepack enable` 使用项目声明的 pnpm |
| 外部服务 | 必须在本地或远端运行 Redis (Valkey) 和 ClickHouse 实例;ClickHouse 建议挂载仓库提供的 `performance.xml`(见上文「ClickHouse 服务端性能配置」) |
### 1. 构建管理端前端
Go Server 运行时需要嵌入前端静态资源。编译 Go 二进制前需要先构建前端静态产物并输出到 Go 服务目录:
```bash
cd frontend
corepack enable
pnpm install
pnpm build:embed
cd ..
```
> **常用前端代码检查命令**:
> * `pnpm lint`
> * `pnpm typecheck`
### 2. 使用 SQLite 启动
关系数据库存储在本地 SQLite 文件,但依然需要提供 Redis 和 ClickHouse 连接配置:
```bash
cp config.example.yaml config.yaml
# 编辑 config.yaml:
# 1. 设置 app.session_secret 为一个随机的长字符串
# 2. 将 database.enabled 设为 false 以启用内置 SQLite
# 3. 将 redis.addrs 与 clickhouse.hosts 修改为你的本地/局域网服务连接信息
# 启动 Server(默认融合模式)
go run main.go all
```
### 3. 使用 PostgreSQL 启动
```bash
cp config.example.yaml config.yaml
# 编辑 config.yaml:
# 1. 设置 app.session_secret
# 2. 将 database.enabled 设为 true,并完整设置 database.*、redis.*、clickhouse.* 字段连接参数
# 启动 Server(默认融合模式)
go run main.go all
```
---
## 首次登录
Server 默认监听 `3000` 端口,启动成功后可以使用浏览器访问:`http://localhost:3000`。
@@ -413,28 +291,12 @@ Server 默认监听 `3000` 端口,启动成功后可以使用浏览器访问
---
## 常用运维指南
### 1. 命令行子服务分进程启动
## 分布式部署
在大型生产部署中,你可以选择将 Server 按职责拆分为多个进程运行:
```bash
go run main.go api # 仅启动管理端与节点通信的 API 服务
go run main.go worker # 仅启动后台任务的 Worker 服务
go run main.go scheduler # 仅启动定时任务的 Scheduler 服务
go run main.go all # 融合模式(在一进程内运行上述所有服务,默认)
```
### 2. 状态验证
```bash
# 验证编译是否通过
go build ./...
# 运行内部单元测试
go test ./internal/apps/openflare/... -count=1
# 检查服务健康状态
curl http://127.0.0.1:3000/api/v1/d/status
go run main.go api # 仅启动管理端与节点通信的 API 服务
go run main.go worker # 仅启动后台任务的 Worker 服务
go run main.go scheduler # 仅启动定时任务的 Scheduler 服务
```
+89 -167
View File
@@ -1024,7 +1024,7 @@ const docTemplate = `{
"SessionCookie": []
}
],
"description": "分页并按照用户、接口路径、时间范围等维度检索 ClickHouse 用户访问日志列表(需要管理员权限,ClickHouse 未启用时报错)",
"description": "分页并按照用户、接口路径、时间范围等维度检索用户访问日志列表(需要管理员权限,日志存储未启用时报错)",
"produces": [
"application/json"
],
@@ -1092,7 +1092,7 @@ const docTemplate = `{
}
},
"400": {
"description": "ClickHouse 未启用或参数错误",
"description": "日志存储未启用或参数错误",
"schema": {
"$ref": "#/definitions/response.Any"
}
@@ -1119,7 +1119,7 @@ const docTemplate = `{
"SessionCookie": []
}
],
"description": "聚合统计最近 7 天的每日访问趋势、浏览器分布以及前 10 名最活跃用户排行(需要管理员权限,ClickHouse 未启用时报错)",
"description": "聚合统计最近 7 天的每日访问趋势、浏览器分布以及前 10 名最活跃用户排行(需要管理员权限,日志存储未启用时报错)",
"produces": [
"application/json"
],
@@ -1147,7 +1147,7 @@ const docTemplate = `{
}
},
"400": {
"description": "ClickHouse 未启用",
"description": "日志存储未启用",
"schema": {
"$ref": "#/definitions/response.Any"
}
@@ -1877,21 +1877,21 @@ const docTemplate = `{
}
}
},
"/api/v1/admin/status/clickhouse": {
"/api/v1/admin/status/log-database": {
"get": {
"security": [
{
"SessionCookie": []
}
],
"description": "返回 ClickHouse parts、mutation、async_insert 队列及进程内 batch writer 指标,需要管理员权限",
"description": "返回当前日志主库、迁移状态、各库保留天数与合法迁移目标,需要管理员权限",
"produces": [
"application/json"
],
"tags": [
"admin"
],
"summary": "获取 ClickHouse 运行指标",
"summary": "获取日志数据库状态",
"responses": {
"200": {
"description": "获取成功",
@@ -1904,19 +1904,13 @@ const docTemplate = `{
"type": "object",
"properties": {
"data": {
"$ref": "#/definitions/analytics.ClickHouseOperationalStats"
"$ref": "#/definitions/status.LogDatabaseStatus"
}
}
}
]
}
},
"400": {
"description": "ClickHouse 未启用",
"schema": {
"$ref": "#/definitions/response.Any"
}
},
"401": {
"description": "未登录",
"schema": {
@@ -2393,6 +2387,18 @@ const docTemplate = `{
"name": "task_type",
"in": "query"
},
{
"type": "string",
"description": "任务类型前缀筛选(与 task_type / task_types 互斥,精确类型优先)",
"name": "task_type_prefix",
"in": "query"
},
{
"type": "string",
"description": "逗号分隔的精确任务类型列表(IN 筛选,优先于前缀)",
"name": "task_types",
"in": "query"
},
{
"type": "integer",
"default": 1,
@@ -8305,80 +8311,6 @@ const docTemplate = `{
}
}
},
"/api/v1/d/option/database/cleanup": {
"post": {
"security": [
{
"SessionCookie": []
}
],
"description": "按目标与保留天数清理可观测性相关数据表,需要管理员权限",
"consumes": [
"application/json"
],
"produces": [
"application/json"
],
"tags": [
"openflare-option"
],
"summary": "清理可观测性数据库",
"parameters": [
{
"description": "清理参数",
"name": "request",
"in": "body",
"schema": {
"$ref": "#/definitions/option.databaseCleanupInput"
}
}
],
"responses": {
"200": {
"description": "清理结果",
"schema": {
"allOf": [
{
"$ref": "#/definitions/response.Any"
},
{
"type": "object",
"properties": {
"data": {
"$ref": "#/definitions/option.databaseCleanupResult"
}
}
}
]
}
},
"400": {
"description": "参数错误",
"schema": {
"$ref": "#/definitions/response.Any"
}
},
"401": {
"description": "未登录",
"schema": {
"$ref": "#/definitions/response.Any"
}
},
"404": {
"description": "无权限或不存在",
"schema": {
"$ref": "#/definitions/response.Any"
}
},
"500": {
"description": "内部错误",
"schema": {
"$ref": "#/definitions/response.Any"
}
}
}
}
},
"/api/v1/d/option/geoip/lookup": {
"post": {
"security": [
@@ -14906,33 +14838,26 @@ const docTemplate = `{
}
}
},
"analytics.ClickHouseOperationalStats": {
"analytics.BatchWriterStats": {
"type": "object",
"properties": {
"active_parts": {
"cap": {
"type": "integer"
},
"async_insert_bytes": {
"depth": {
"type": "integer"
},
"async_insert_queue": {
"drops": {
"type": "integer"
},
"batch_writers": {
"description": "BatchWriters reports in-process queue depth/drops/flush errors for CH writers.",
"type": "array",
"items": {
"$ref": "#/definitions/batchwriter.Stats"
}
"flush_errors": {
"type": "integer"
},
"database": {
"name": {
"type": "string"
},
"pending_mutations": {
"type": "integer"
},
"total_rows": {
"type": "integer"
"running": {
"type": "boolean"
}
}
},
@@ -15024,29 +14949,6 @@ const docTemplate = `{
}
}
},
"batchwriter.Stats": {
"type": "object",
"properties": {
"cap": {
"type": "integer"
},
"depth": {
"type": "integer"
},
"drops": {
"type": "integer"
},
"flush_errors": {
"type": "integer"
},
"name": {
"type": "string"
},
"running": {
"type": "boolean"
}
}
},
"cache.updateCacheConfigRequest": {
"type": "object",
"required": [
@@ -15105,6 +15007,9 @@ const docTemplate = `{
"id": {
"type": "integer"
},
"zone_domain": {
"type": "string"
},
"zone_id": {
"type": "integer"
}
@@ -15826,6 +15731,36 @@ const docTemplate = `{
}
}
},
"github_com_Rain-kl_Wavelet_internal_model_analytics.ClickHouseOperationalStats": {
"type": "object",
"properties": {
"active_parts": {
"type": "integer"
},
"async_insert_bytes": {
"type": "integer"
},
"async_insert_queue": {
"type": "integer"
},
"batch_writers": {
"description": "BatchWriters reports in-process queue depth/drops/flush errors for CH writers.",
"type": "array",
"items": {
"$ref": "#/definitions/analytics.BatchWriterStats"
}
},
"database": {
"type": "string"
},
"pending_mutations": {
"type": "integer"
},
"total_rows": {
"type": "integer"
}
}
},
"github_com_Rain-kl_Wavelet_pkg_protocol.ActiveConfigMeta": {
"type": "object",
"properties": {
@@ -18481,46 +18416,6 @@ const docTemplate = `{
}
}
},
"option.databaseCleanupInput": {
"type": "object",
"properties": {
"retention_days": {
"type": "integer"
},
"target": {
"type": "string"
}
}
},
"option.databaseCleanupResult": {
"type": "object",
"properties": {
"cleanup_mode": {
"type": "string"
},
"delete_all": {
"type": "boolean"
},
"deleted_count": {
"type": "integer"
},
"eligible_count": {
"type": "integer"
},
"retention_days": {
"type": "integer"
},
"table_ttl_days": {
"type": "integer"
},
"target": {
"type": "string"
},
"target_label": {
"type": "string"
}
}
},
"option.geoIPLookupRequest": {
"type": "object",
"properties": {
@@ -19859,6 +19754,33 @@ const docTemplate = `{
}
}
},
"status.LogDatabaseStatus": {
"type": "object",
"properties": {
"active_database": {
"type": "string"
},
"available_targets": {
"type": "array",
"items": {
"type": "string"
}
},
"clickhouse": {
"$ref": "#/definitions/github_com_Rain-kl_Wavelet_internal_model_analytics.ClickHouseOperationalStats"
},
"migration": {
"description": "idle | migrating",
"type": "string"
},
"retention_days": {
"type": "object",
"additionalProperties": {
"type": "integer"
}
}
}
},
"status.SystemStatusResponse": {
"type": "object",
"properties": {
+13 -42
View File
@@ -14,12 +14,12 @@ Agent 统一通过 OpenResty 二进制控制运行时。本地部署需要节点
## 环境要求
| 项目 | 要求 |
| --- | --- |
| Docker / Docker Compose | 用于启动 Server 及其依赖的 PostgreSQL、Redis 和 ClickHouse 容器;如采用 Docker Agent,也用于运行 Agent |
| OpenResty | 本地安装 Agent 时需要可执行 `openresty`,或在安装脚本中指定路径 |
| 可访问端口 | Server 默认监听 `3000`,Agent 节点需要能访问 Server 地址 |
| 浏览器 | 用于访问管理端 |
| 项目 | 要求 |
| --- |------------------------------------------------------------------|
| Docker / Docker Compose | 用于启动 Server 及其依赖的 PostgreSQL、Valkey;如采用 Docker Agent,也用于运行 Agent |
| OpenResty | 本地安装 Agent 时需要可执行 `openresty`,或在安装脚本中指定路径 |
| 可访问端口 | Server 默认监听 `3000`,Agent 节点需要能访问 Server 地址 |
| 浏览器 | 用于访问管理端 |
- **Docker**:`20.10.0+`
- **Docker Compose**:`2.0.0+`
@@ -28,15 +28,7 @@ Agent 统一通过 OpenResty 二进制控制运行时。本地部署需要节点
## 1. 启动 Server
为了保证异步任务队列(Asynq 框架)及可观测流量看板功能完整运行,快速开始推荐采用 **PostgreSQL + Redis + ClickHouse** 经典单机版编排。
先拉取 ClickHouse 服务端性能配置到 `./config/clickhouse`,并以单文件方式挂载:
```bash
mkdir -p ./config/clickhouse
curl -fsSL -o ./config/clickhouse/performance.xml \
https://raw.githubusercontent.com/Rain-kl/OpenFlare/refs/heads/main/config/clickhouse/performance.xml
```
快速开始推荐采用 **PostgreSQL + Redis ** 标准部署方案。
在空目录中创建 `docker-compose.yaml`:
@@ -63,15 +55,11 @@ services:
DB_NAME: "${DB_NAME:-openflare}"
REDIS_ENABLED: "true"
REDIS_ADDR: "redis:6379"
CLICKHOUSE_ENABLED: "true"
CLICKHOUSE_HOST: "clickhouse:9000"
depends_on:
postgres:
condition: service_healthy
redis:
condition: service_healthy
clickhouse:
condition: service_healthy
postgres:
image: postgres:17-alpine
@@ -100,34 +88,11 @@ services:
timeout: 5s
retries: 5
clickhouse:
image: clickhouse/clickhouse-server:25.3-alpine
restart: unless-stopped
environment:
CLICKHOUSE_DB: openflare
CLICKHOUSE_USER: default
CLICKHOUSE_PASSWORD: ${CLICKHOUSE_PASSWORD:-replace-with-clickhouse-password}
CLICKHOUSE_DEFAULT_ACCESS_MANAGEMENT: 1
TZ: Asia/Shanghai
ulimits:
nofile:
soft: 262144
hard: 262144
volumes:
- openflare_clickhouse_data:/var/lib/clickhouse
- ./config/clickhouse/performance.xml:/etc/clickhouse-server/config.d/performance.xml:ro
healthcheck:
test: ["CMD", "clickhouse-client", "--user", "default", "--password", "${CLICKHOUSE_PASSWORD:-replace-with-clickhouse-password}", "--query", "SELECT 1"]
interval: 10s
timeout: 5s
retries: 5
start_period: 15s
volumes:
openflare_uploads:
openflare_postgres_data:
openflare_redis_data:
openflare_clickhouse_data:
```
启动服务:
@@ -158,6 +123,12 @@ http://localhost:3000
> [!WARNING]
> 为了你的系统安全,首次登录后请立即修改默认密码。
如果忘记密码并且没有配置找回密码渠道, 可以使用命令进行重置
```bash
go run main.go reset-paswd # 重置管理员密码
```
---
## 2. 准备 Agent Token
+17 -4
View File
@@ -105,7 +105,7 @@ Server 的所有核心基础配置定义在 `config.yaml` 中,且均支持环
| 配置文件 YAML 路径 | 对应覆盖环境变量 | 作用说明 | 默认值 |
| --- | --- | --- | --- |
| `clickhouse.enabled` | `CLICKHOUSE_ENABLED` | 是否启用 ClickHouse。**系统节点指标与访问日志在此进行海量写入** | `true` |
| `clickhouse.enabled` | `CLICKHOUSE_ENABLED` | 是否启用 ClickHouse。**系统节点指标与访问日志在此进行海量写入**。默认关闭:缺失本配置项或为 `false` 时不启用,日志/指标由主库承担;显式 `true` 或设置 `CLICKHOUSE_HOST` 时启用 | `false` |
| `clickhouse.hosts` | `CLICKHOUSE_HOST` | ClickHouse 集群连接地址数组(环境变量仅设置单地址) | `["127.0.0.1:9000"]` |
| `clickhouse.username` | `CLICKHOUSE_USERNAME` | ClickHouse 账号用户名 | `default` |
| `clickhouse.password` | `CLICKHOUSE_PASSWORD` | ClickHouse 密码 | `replace-with-clickhouse-password` |
@@ -197,8 +197,6 @@ Server 的所有核心基础配置定义在 `config.yaml` 中,且均支持环
| `node_offline_threshold` | `int` | 在管理后台中判定节点失去心跳并标注为离线状态的无响应阈值(毫秒) | `60000` (60s) |
| `agent_update_repo` | `string` | Agent 节点更新下载自身二进制的 Release 仓库源 | `Rain-kl/OpenFlare` |
| `geoip_provider` | `string` | GeoIP 提供商,支持 `maxmind` 等,用于 WAF 防护时地域分析 | `ipinfo` |
| `database_auto_cleanup_enabled` | `bool` | 是否在每天凌晨 3:00 自动清理过期观测历史日志(降低数据库空间) | `true` |
| `database_auto_cleanup_retention_days` | `int` | 自动清理观测数据(访问日志、度量曲线、审计等)的默认保留天数 | `30` |
### 5. Uptime Kuma 监控联动同步
| 配置键 (Key) | 数据类型 | 作用说明 | 默认值 |
@@ -266,12 +264,27 @@ Server 的所有核心基础配置定义在 `config.yaml` 中,且均支持环
| 配置键 (Key) | 数据类型 | 作用说明 | 默认值 |
| --- | --- | --- | --- |
| `origin_error_page_enabled` | `bool` | 是否启用全局源站错误页。开启后,源站或网关返回的匹配状态码由自定义/默认 HTML 替换,**HTTP 状态码保持原值**;关闭后不生成相关指令,恢复透传。修改后需发布配置版本生效 | `true` |
| `origin_error_page_get_only` | `bool` | 是否仅对 **GET** 请求生效。开启后仅 GET 的匹配错误状态码返回自定义错误页;POST/PUT 等其它方法透传源站响应 | `false` |
| `origin_error_page_get_only` | `bool` | 是否仅对 **GET** 请求生效。开启后仅 GET 的匹配错误状态码返回自定义错误页;POST/PUT 等其它方法不返回自定义错误页(保留原始错误状态码) | `false` |
| `origin_error_page_status_codes` | `json` | 触发错误页的状态码标签 JSON 数组。支持单码(如 `522`)与闭区间(如 `500-599`);单码与区间两端均须在 **400–599**,且 `lo ≤ hi`。启用时展开结果不能为空 | `["500-599"]` |
| `origin_error_page_html` | `string` | 错误页自定义 HTML。空字符串表示使用内置 OpenFlare 默认模板(极简白底);支持占位符 `{{status}}`(与 HTTP 状态码一致)、`{{host}}`(请求 Host)。最大 **256 KiB**(按字节)。勿嵌入不可信第三方脚本 | 空 |
---
### 8. 日志存储(Log Database)
日志存储解耦后的运行时配置:日志主库由「切换日志数据库」任务管理(内部/受保护 key,禁止管理员手动修改),访问日志保留天数按存储库分别在业务配置中设置;性能指标(CPU/内存/磁盘/网络)价值衰减快,按三库共用的独立短留存清理。
| 配置键 (Key) | 数据类型 | 作用说明 | 默认值 |
| --- | --- | --- | --- |
| `log_database` | `string` | 当前日志主库(`postgres` / `sqlite` / `clickhouse`)。**内部受保护 key**:仅「切换日志数据库」迁移任务写入,管理员不可手动创建/修改 | 随主库(PostgreSQL 启用时为 `postgres`,否则 `sqlite`;ClickHouse 启用时优先 `clickhouse`) |
| `log_db_migration` | `string` | 日志迁移冻结标记(`migrating` 或空)。**内部受保护 key**:仅迁移任务写入,置位期间日志写入返回 503「日志数据库迁移中,暂不可写」 | 空 |
| `log_retention_days_postgres` | `int` | PostgreSQL 日志库的访问日志过期清理保留天数(过期日志由系统垃圾清理每日任务删除) | `30` |
| `log_retention_days_sqlite` | `int` | SQLite 日志库的访问日志过期清理保留天数 | `30` |
| `log_retention_days_clickhouse` | `int` | ClickHouse 日志库的访问日志过期清理保留天数 | `30` |
| `metric_retention_days` | `int` | 性能指标(CPU/内存/磁盘/网络)保留天数,三库共用独立短留存(不随访问日志保留配置) | `3` |
---
## 前端构建环境变量
| 环境变量 | 作用 | 默认值 |
File diff suppressed because it is too large Load Diff
@@ -0,0 +1,885 @@
# Service Worker 离线兜底 Implementation Plan
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
**Goal:** 给平台所有启用 HTTPS 的网站(反代 + Pages)下发 Service Worker 离线兜底:域名被墙后浏览器从缓存吐出"联系站长"页,避免用户流失。全平台一键批量下发。
**Architecture:** 全局 Option(SystemConfig / OpenRestyConfig snapshot)驱动,与现有 origin error page 完全同模式。渲染层在 HTTPS server 块注入 SW 静态 location + 首页挑战拦截(真实浏览器 UA 且无 cookie 时返回含 `register('/sw.js')` 的挑战页),通过 SupportFile 下发 sw.js / offline.html,Agent 替换占位符落盘。前端「响应页面」模块两个 tab:错误页 / 联系页。
**Tech Stack:** Go 1.25+、Gin、GORM、PostgreSQL/SQLite goose 迁移、OpenResty/Lua、Next.js、TypeScript、shadcn/ui、TanStack Query。
## Global Constraints
- 遵循 AGENTS.md 分层:`apps → repository → model`,禁止 `model → repository`。
- API 错误用 `response.Abort*`;Handler 不直接 `c.JSON`。
- 渲染改动后:`make swagger`(本功能无新 API,跳过);开发完成:`make code-check`;提交前:`make format`。
- 代码/配置变更写入 `docs/changelog/index.md` 的 `[Unreleased]`(中文,用户可读)。
- 配置 key 命名:小写 snake_case。测试临时目录只用 `t.TempDir()`。
- 前端:`variant` + CSS 变量,业务 `className` 不硬编码颜色;根容器 `w-full`,外层 `py-6 px-1`;标题行 `flex items-center gap-2`。
- 配置文件路径占位符统一追加到 `pkg/render/openresty/types.go` 的 const 块。
- 迁移:PostgreSQL 与 SQLite 各一份 goose SQL(`goose/postgres/`、`goose/sqlite/`),见 `database-migration` skill。
---
### Task 1: 后端配置 key 与 Option 校验
**Files:**
- Modify: `internal/model/system_configs.go:114-117`
- Modify: `internal/apps/openflare/option/openresty_validators.go:19-65`
- Modify: `internal/apps/openflare/option/openresty_validators.go:69-79`
**Interfaces:**
- Produces: 常量 `model.ConfigKeySWOfflineEnabled`, `model.ConfigKeySWOfflineHTML`; 校验函数 `validateSWOfflineHTML`。
- [ ] **Step 1: 在 `system_configs.go` 追加 key 常量**
在 `ConfigKeyOriginErrorPageGetOnly`(第 117 行)后追加:
```go
ConfigKeySWOfflineEnabled = "sw_offline_enabled" // 是否启用 Service Worker 离线兜底
ConfigKeySWOfflineHTML = "sw_offline_html" // 离线联系页自定义 HTML(空则内置默认)
```
- [ ] **Step 2: 注册 validator**
在 `openRestyOptionValidators` map(`openresty_validators.go` 第 61-64 行)后追加:
```go
model.ConfigKeySWOfflineEnabled: validateBooleanOption,
model.ConfigKeySWOfflineHTML: validateSWOfflineHTML,
```
- [ ] **Step 3: 在 `validateOpenRestyOption` 增加 HTML 字节数特殊处理**
在第 69-79 行函数内,`if key == model.ConfigKeyOriginErrorPageHTML` 分支改为同时覆盖 SW HTML:
```go
if key == model.ConfigKeyOriginErrorPageHTML || key == model.ConfigKeySWOfflineHTML {
return validateOriginErrorPageHTML(key, value)
}
```
`validateOriginErrorPageHTML` 逻辑(非空、≤256 KiB)对两个 HTML 复用,无需新函数。
- [ ] **Step 4: 运行测试**
Run: `cd /Users/ryan/conductor/workspaces/OpenFlare/islamabad && go build ./... && go test ./internal/apps/openflare/option/...`
Expected: PASS
- [ ] **Step 5: 提交**
```bash
git add internal/model/system_configs.go internal/apps/openflare/option/openresty_validators.go
git commit -m "feat(option): add sw offline config keys and validation"
```
---
### Task 2: goose 迁移(PostgreSQL + SQLite)Seed 全局 Option
**Files:**
- Create: `internal/infra/persistence/migrator/goose/postgres/<YYYYMMDD>NNN_add_sw_offline_options.sql`
- Create: `internal/infra/persistence/migrator/goose/sqlite/<YYYYMMDD>NNN_add_sw_offline_options.sql`
**Interfaces:**
- Consumes: Task 1 key 常量。
- Produces: 数据库 seed 的 `sw_offline_enabled` / `sw_offline_html` 两行 `w_system_configs`。
- [ ] **Step 1: 确认迁移序号**
Run: `ls /Users/ryan/conductor/workspaces/OpenFlare/islamabad/internal/infra/persistence/migrator/goose/postgres/ | tail -3`
取最新序号 +1(如 `202608080001`)。
- [ ] **Step 2: 创建 postgres 迁移**
创建 `goose/postgres/202608080001_add_sw_offline_options.sql`:
```sql
-- +goose Up
INSERT INTO w_system_configs (key, value, type, visibility, description, created_at, updated_at)
VALUES
('sw_offline_enabled', 'false', 'business', 0, '是否启用 Service Worker 离线兜底', CURRENT_TIMESTAMP, CURRENT_TIMESTAMP),
('sw_offline_html', '', 'business', 0, '离线联系页自定义 HTML,空则使用内置默认', CURRENT_TIMESTAMP, CURRENT_TIMESTAMP)
ON CONFLICT (key) DO NOTHING;
-- +goose Down
DELETE FROM w_system_configs WHERE key IN (
'sw_offline_enabled',
'sw_offline_html'
);
```
- [ ] **Step 3: 创建 sqlite 迁移**
创建 `goose/sqlite/202608080001_add_sw_offline_options.sql`(内容与 postgres 相同)。
- [ ] **Step 4: 运行迁移测试**
Run: `go test ./internal/infra/persistence/migrator/...`
Expected: PASS(数据库迁移冒烟通过)
- [ ] **Step 5: 提交**
```bash
git add internal/infra/persistence/migrator/goose/postgres/202608080001_add_sw_offline_options.sql internal/infra/persistence/migrator/goose/sqlite/202608080001_add_sw_offline_options.sql
git commit -m "feat(db): seed sw offline options"
```
---
### Task 3: ConfigSnapshot 渲染类型字段
**Files:**
- Modify: `pkg/render/openresty/types.go:20-26`
- Modify: `pkg/render/openresty/types.go:318-323`(`ConfigSnapshot` 结构体)
**Interfaces:**
- Produces: `ConfigSnapshot.SWOfflineEnabled bool`、`ConfigSnapshot.SWOfflineHTML string`;常量 `SWDirPlaceholder`。
- [ ] **Step 1: 追加占位符常量**
在 `types.go` 占位符 const 块(第 23 行 `ErrorPageTmplPlaceholder` 后)追加:
```go
SWDirPlaceholder = "__OPENFLARE_SW_DIR__"
```
- [ ] **Step 2: 追加 ConfigSnapshot 字段**
在 `ConfigSnapshot` 末尾(`OriginErrorPageGetOnly` 后)追加:
```go
// SWOfflineEnabled enables the Service Worker offline fallback for HTTPS routes.
SWOfflineEnabled bool `json:"sw_offline_enabled,omitempty"`
// SWOfflineHTML is the contact-page HTML served offline; empty uses the built-in default.
SWOfflineHTML string `json:"sw_offline_html,omitempty"`
```
- [ ] **Step 3: 提交**
```bash
git add pkg/render/openresty/types.go
git commit -m "feat(openresty): add sw offline ConfigSnapshot fields and placeholder"
```
---
### Task 4: 渲染层 SW 资源与挑战拦截
**Files:**
- Create: `pkg/render/openresty/service_worker.go`
- Create: `pkg/render/openresty/service_worker_test.go`
- Modify: `pkg/render/openresty/render.go:37-59`(`Render` 追加 support files)
- Modify: `pkg/render/openresty/render.go:90-115`(`RenderRouteConfig` 注入挑战)
**Interfaces:**
- Consumes: `ConfigSnapshot.SWOfflineEnabled` / `.SWOfflineHTML`;`SWDirPlaceholder`。
- Produces: `DefaultSWOfflineHTML string`、`EffectiveSWOfflineHTML(cfg ConfigSnapshot) string`、`ServiceWorkerSupportFiles(cfg ConfigSnapshot) []SupportFile`、`renderServiceWorkerChallenger(cfg ConfigSnapshot) string`。
- [ ] **Step 1: 写失败测试**
创建 `service_worker_test.go`,断言:
1. `EffectiveSWOfflineHTML`:HTML 为空返回内置默认;非空返回自定义。
2. `ServiceWorkerSupportFiles`:仅当 `SWOfflineEnabled` 时返回 `sw/sw.js` 与 `sw/offline.html` 两个文件;未启用返回 nil。
3. `renderServiceWorkerChallenger`:启用且含 sw.js location、offline location、挑战 location;未启用返回空串。
```go
package openresty
import (
"strings"
"testing"
)
func TestEffectiveSWOfflineHTML(t *testing.T) {
if got := EffectiveSWOfflineHTML(ConfigSnapshot{}); got != DefaultSWOfflineHTML {
t.Fatalf("default mismatch")
}
custom := "<html>custom</html>"
if got := EffectiveSWOfflineHTML(ConfigSnapshot{SWOfflineHTML: custom}); got != custom {
t.Fatalf("custom mismatch")
}
}
func TestServiceWorkerSupportFiles(t *testing.T) {
disabled := ServiceWorkerSupportFiles(ConfigSnapshot{})
if disabled != nil {
t.Fatalf("expected nil when disabled, got %v", disabled)
}
enabled := ServiceWorkerSupportFiles(ConfigSnapshot{SWOfflineEnabled: true})
if len(enabled) != 2 {
t.Fatalf("expected 2 support files, got %d", len(enabled))
}
paths := map[string]string{}
for _, f := range enabled {
paths[f.Path] = f.Content
}
if _, ok := paths["sw/sw.js"]; !ok {
t.Fatalf("missing sw/sw.js")
}
if _, ok := paths["sw/offline.html"]; !ok {
t.Fatalf("missing sw/offline.html")
}
}
func TestRenderServiceWorkerChallenger(t *testing.T) {
if got := renderServiceWorkerChallenger(ConfigSnapshot{}); got != "" {
t.Fatalf("expected empty when disabled")
}
got := renderServiceWorkerChallenger(ConfigSnapshot{SWOfflineEnabled: true})
for _, want := range []string{"location = /sw.js", "location = /offline.html", "sw.runtime", "content_by_lua"} {
if !strings.Contains(got, want) {
t.Fatalf("challenger missing %q", want)
}
}
}
```
- [ ] **Step 2: 运行确认失败**
Run: `go test ./pkg/render/openresty/ -run 'TestEffectiveSWOfflineHTML|TestServiceWorkerSupportFiles|TestRenderServiceWorkerChallenger'`
Expected: FAIL(函数未定义)
- [ ] **Step 3: 实现 `service_worker.go`**
```go
package openresty
import (
"strings"
)
const (
SWJSLocation = "location = /sw.js"
SWOfflineLocation = "location = /offline.html"
SWChallengeLua = "sw/challenge.lua"
SWRuntimeLua = "sw/runtime.lua"
swDirPrefix = "sw/"
)
// DefaultSWOfflineHTML is the built-in contact page shown when the domain is blocked.
const DefaultSWOfflineHTML = `<!DOCTYPE html>
<html lang="zh-CN">
<head>
<meta charset="UTF-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<title>网站暂时无法访问 | 联系站长</title>
<style>
* { box-sizing: border-box; margin: 0; padding: 0; }
body { font-family: -apple-system, BlinkMacSystemFont, "Segoe UI", Roboto, Helvetica, Arial, sans-serif; background: #ffffff; color: #333333; height: 100vh; display: flex; flex-direction: column; justify-content: center; align-items: center; text-align: center; padding: 48px 24px; }
h1 { font-size: 28px; font-weight: 700; margin-bottom: 16px; }
p { font-size: 16px; line-height: 1.7; color: #666666; max-width: 520px; }
</style>
</head>
<body>
<h1>网站暂时无法访问</h1>
<p>当前域名暂时无法从网络访问。请通过其他方式联系网站管理员获取最新访问入口。</p>
</body>
</html>
`
// EffectiveSWOfflineHTML returns custom HTML when set, otherwise the built-in default.
func EffectiveSWOfflineHTML(cfg ConfigSnapshot) string {
if strings.TrimSpace(cfg.SWOfflineHTML) == "" {
return DefaultSWOfflineHTML
}
return cfg.SWOfflineHTML
}
// ServiceWorkerSupportFiles returns the sw.js script and offline contact page.
func ServiceWorkerSupportFiles(cfg ConfigSnapshot) []SupportFile {
if !cfg.SWOfflineEnabled {
return nil
}
return []SupportFile{
{Path: swDirPrefix + "sw.js", Content: defaultSWJS()},
{Path: swDirPrefix + "offline.html", Content: EffectiveSWOfflineHTML(cfg)},
}
}
func defaultSWJS() string {
return `var CACHE = "openflare-offline-v1";
var OFFLINE = "/offline.html";
self.addEventListener("install", function (e) {
e.waitUntil(caches.open(CACHE).then(function (c) { return c.addAll([OFFLINE]); }));
self.skipWaiting();
});
self.addEventListener("activate", function (e) {
e.waitUntil(caches.keys().then(function (keys) {
return Promise.all(keys.filter(function (k) { return k !== CACHE; }).map(function (k) { return caches.delete(k); }));
}));
self.clients.claim();
});
self.addEventListener("fetch", function (e) {
if (e.request.method !== "GET") { return; }
e.respondWith(
fetch(e.request).catch(function () {
return caches.match(e.request).then(function (r) { return r || caches.match(OFFLINE); });
})
);
});
`
}
// renderServiceWorkerChallenger emits SW static locations and the homepage
// challenge intercept for HTTPS server blocks.
func renderServiceWorkerChallenger(cfg ConfigSnapshot) string {
if !cfg.SWOfflineEnabled {
return ""
}
var builder strings.Builder
builder.WriteString("\n location = /sw.js {\n")
builder.WriteString(" alias " + SWDirPlaceholder + "/sw.js;\n")
builder.WriteString(" default_type application/javascript;\n")
builder.WriteString(" add_header Service-Worker-Allowed /;\n")
builder.WriteString(" add_header Cache-Control \"no-cache\";\n")
builder.WriteString(" }\n\n")
builder.WriteString(" location = /offline.html {\n")
builder.WriteString(" alias " + SWDirPlaceholder + "/offline.html;\n")
builder.WriteString(" default_type text/html;\n")
builder.WriteString(" add_header Cache-Control \"no-cache\";\n")
builder.WriteString(" }\n\n")
builder.WriteString(" location = /__openflare_sw_challenge {\n")
builder.WriteString(" internal;\n")
builder.WriteString(" content_by_lua_file " + SWDirPlaceholder + "/challenge.lua;\n")
builder.WriteString(" }\n")
return builder.String()
}
```
- [ ] **Step 4: 接入 `Render` 追加 support files**
在 `render.go` `Render` 函数内、`originErrorPageSupportFile` 追加之后追加:
```go
if doc.OpenRestyConfig.SWOfflineEnabled {
files = append(files, ServiceWorkerSupportFiles(doc.OpenRestyConfig)...)
}
```
- [ ] **Step 5: 接入 `RenderRouteConfig` 注入挑战**
在 `RenderRouteConfig` 内 `renderProxyRoute` / `renderPagesRoute` 调用之前,将 SW 拦截接入 server 块。将 `renderAccessBlock(siteName, powEnabled)` 调用处扩展:新建 `renderServerAccess(siteName, powEnabled, cfg)` 封装,并在其中追加 SW 运行时检查。具体为在 `renderAccessBlock` 生成的 access 块内,追加对 `sw.runtime` 的调用。
简化实现:新增 `renderAccessBlockWithSW(siteName string, powEnabled bool, cfg ConfigSnapshot) string`,返回 `renderAccessBlock(siteName, powEnabled)` 与(当 `SWOfflineEnabled` 时)追加:
```
access_by_lua_block {
if not string.find(package.path, "__OPENFLARE_LUA_DIR__/?.lua", 1, true) then
package.path = "__OPENFLARE_LUA_DIR__/?.lua;__OPENFLARE_LUA_DIR__/?/init.lua;" .. package.path
end
require("sw.runtime").check()
}
```
然后将 `renderHTTPProxyServer`、`renderHTTPSServer`、`renderHTTPPagesServer`、`renderHTTPSPagesServer` 中 `renderAccessBlock(...)` 替换为 `renderAccessBlockWithSW(..., cfg)`,并在各自 server 块内追加 `renderServiceWorkerChallenger(cfg)` 输出。
**注意:** `renderAccessBlock` 在既有 powEnabled 分支已含 `access_by_lua_block`。为兼容,`renderAccessBlockWithSW` 在 powEnabled 分支内合并 SW check 到同一块;非 pow 分支额外追加一个块。本步以**仅新增 server 级 SW location + 独立 `access_by_lua_block`** 为最小实现;若 nginx 同 server 存在两个 `access_by_lua_block`,运行时只执行最后一个——**故实现必须合并**。请在实现时确认 `renderAccessBlock` 各分支,将 SW check 合并进唯一 access 块内,避免覆盖 WAF/PoW。
- [ ] **Step 6: 运行测试**
Run: `go test ./pkg/render/openresty/...`
Expected: PASS
- [ ] **Step 7: 提交**
```bash
git add pkg/render/openresty/service_worker.go pkg/render/openresty/service_worker_test.go pkg/render/openresty/render.go
git commit -m "feat(openresty): render sw offline assets and challenge intercept"
```
---
### Task 5: config_version snapshot 接入全局 Option
**Files:**
- Modify: `internal/apps/openflare/config_version/snapshot.go:143-147`(`openRestyConfigSnapshot` 字段)
- Modify: `internal/apps/openflare/config_version/snapshot.go:559-563`(`buildOpenRestyConfigSnapshot` 读取)
- Modify: `internal/apps/openflare/config_version/logics.go:537-541`(diff 追加)
- Modify: `internal/apps/openflare/config_version/logics.go:604-608`(option keys 追加)
**Interfaces:**
- Consumes: `model.ConfigKeySWOfflineEnabled` / `.SWOfflineHTML`。
- Produces: snapshot JSON 内 `sw_offline_enabled` / `sw_offline_html` 字段,触发 checksum 变化。
- [ ] **Step 1: snapshot 结构体追加字段**
在 `openRestyConfigSnapshot`(`snapshot.go:143-147`,`OriginErrorPageGetOnly` 后)追加:
```go
SWOfflineEnabled bool `json:"sw_offline_enabled,omitempty"`
SWOfflineHTML string `json:"sw_offline_html,omitempty"`
```
- [ ] **Step 2: build 读取配置**
在 `buildOpenRestyConfigSnapshot`(`snapshot.go:559-563`,`OriginErrorPageGetOnly` 赋值后)追加:
```go
SWOfflineEnabled: getBoolConfig(model.ConfigKeySWOfflineEnabled, false),
SWOfflineHTML: getStringConfig(model.ConfigKeySWOfflineHTML, ""),
```
- [ ] **Step 3: diff 追加**
在 `diffOpenRestyOptionDetails`(`logics.go:540` 后)追加:
```go
appendIfChanged("SWOfflineEnabled", fmt.Sprintf("%t", left.SWOfflineEnabled), fmt.Sprintf("%t", right.SWOfflineEnabled))
appendIfChanged("SWOfflineHTML", left.SWOfflineHTML, right.SWOfflineHTML)
```
- [ ] **Step 4: option keys 追加**
在 `openRestyOptionKeys()`(`logics.go:607` 后)追加:
```go
"SWOfflineEnabled",
"SWOfflineHTML",
```
- [ ] **Step 5: 运行测试**
Run: `go test ./internal/apps/openflare/config_version/...`
Expected: PASS
- [ ] **Step 6: 提交**
```bash
git add internal/apps/openflare/config_version/snapshot.go internal/apps/openflare/config_version/logics.go
git commit -m "feat(config): wire sw offline options into config snapshot"
```
---
### Task 6: Agent 侧 SW Lua 资源与占位符替换
**Files:**
- Create: `internal/apps/agent/nginx/sw_assets.go`
- Modify: `internal/apps/agent/nginx/manager.go:393-410`(`EnsureLuaAssets` 追加 SW Lua)
- Modify: `internal/apps/agent/nginx/manager.go:526-528`(checksum 归一化 SW 路径)
- Modify: `internal/apps/agent/nginx/manager.go:1381-1383`(renderRouteConfig 替换 SW 占位符)
**Interfaces:**
- Consumes: `openrestyrender.SWDirPlaceholder`、`openrestyrender.SWChallengeLua`、`openrestyrender.SWRuntimeLua`。
- Produces: `ManagedSWLuaFiles() []protocol.SupportFile`(`sw/runtime.lua`、`sw/challenge.lua`)。
- [ ] **Step 1: 创建 `sw_assets.go`**
```go
package nginx
import (
"github.com/Rain-kl/Wavelet/internal/apps/agent/protocol"
)
const openRestySWRuntimeLua = `local source = debug.getinfo(1, "S").source or ""
if string.sub(source, 1, 1) == "@" then
local script_path = string.sub(source, 2)
local base_dir = string.match(script_path, "^(.*)/sw/[^/]+%.lua$")
if base_dir and base_dir ~= "" and not string.find(package.path, base_dir, 1, true) then
package.path = base_dir .. "/?.lua;" .. base_dir .. "/?/init.lua;" .. package.path
end
end
local function is_real_browser(ua)
if not ua or ua == "" then return false end
-- Chrome/Edge/CentOS-style: "Chrome/120"
if string.find(ua, "Chrome/%d", 1, true) then return true end
-- Firefox: "Firefox/120"
if string.find(ua, "Firefox/%d", 1, true) then return true end
-- Safari (non-Chrome, e.g. "Version/17.0 Safari")
if not string.find(ua, "Chrome", 1, true) and string.find(ua, "Safari", 1, true) then return true end
return false
end
local function pass_through()
return true
end
function _M_check()
local ua = ngx.var.http_user_agent or ""
if not is_real_browser(ua) then return pass_through() end
local uri = ngx.var.uri or ""
if uri ~= "/" then return pass_through() end
local cookie = ngx.var["cookie___openflare_sw"]
if cookie and cookie ~= "" then return pass_through() end
-- intercept: internal redirect to challenge page, which registers SW + sets cookie
local redir = ngx.var.scheme .. "://" .. ngx.var.host .. uri .. (ngx.var.args and ("?" .. ngx.var.args) or "")
ngx.req.set_uri_args({ redir = redir })
return ngx.exec("/__openflare_sw_challenge")
end
`
const openRestySWChallengeLua = `local args = ngx.req.get_uri_args()
local redir = args["redir"] or "/"
ngx.header["Set-Cookie"] = "__openflare_sw=1; Path=/; Max-Age=31536000"
ngx.header.content_type = "text/html; charset=utf-8"
ngx.say([[<!DOCTYPE html>
<html lang="zh-CN">
<head>
<meta charset="utf-8">
<meta name="viewport" content="width=device-width, initial-scale=1">
<meta name="robots" content="noindex,nofollow">
<title>加载中...</title>
<script>
if ("serviceWorker" in navigator) {
navigator.serviceWorker.register("/sw.js").then(function () {
location.replace("]] .. redir .. [[");
}).catch(function () {
location.replace("]] .. redir .. [[");
});
} else {
location.replace("]] .. redir .. [[");
}
</script>
</head>
<body>正在加载...</body>
</html>]])
`
// ManagedSWLuaFiles returns embedded Lua assets for the SW offline challenge.
func ManagedSWLuaFiles() []protocol.SupportFile {
return []protocol.SupportFile{
{Path: "sw/runtime.lua", Content: openRestySWRuntimeLua},
{Path: "sw/challenge.lua", Content: openRestySWChallengeLua},
}
}
```
- [ ] **Step 2: `EnsureLuaAssets` 追加 SW Lua**
在 `manager.go:403`(`allSupportFiles` 组装处)追加:
```go
allSupportFiles = append(allSupportFiles, ManagedSWLuaFiles()...)
```
- [ ] **Step 3: checksum 归一化 SW 路径**
在 `manager.go:526-528`(error page 路径归一化后)追加:
```go
swDir := filepath.ToSlash(filepath.Join(m.NginxCertDir, "sw"))
normalizedRoute = strings.ReplaceAll(normalizedRoute, swDir, openrestyrender.SWDirPlaceholder)
```
- [ ] **Step 4: `renderRouteConfig` 替换 SW 占位符**
在 `manager.go:1381-1383`(error page 替换后)追加:
```go
swDir := filepath.ToSlash(filepath.Join(m.NginxCertDir, "sw"))
rendered = strings.ReplaceAll(rendered, openrestyrender.SWDirPlaceholder, swDir)
```
- [ ] **Step 5: 确认 SW 文件落盘**
`renderServiceWorkerChallenger` 中 `alias __OPENFLARE_SW_DIR__/sw.js` 与 `/offline.html` 引用 support files `sw/sw.js`、`sw/offline.html`。这些文件经 Task 4 作为普通 support file 由 `writeManagedCertFiles` 写入 `<CertDir>/sw/`(路径含子目录)。验证 `certFileTargetPath` 支持子目录路径(读 `manager.go` 确认)。若不支持,需在 `writeManagedCertFiles` 中 `os.MkdirAll(filepath.Dir(targetPath))`。**实现时确认并补全目录创建。**
- [ ] **Step 6: 运行测试**
Run: `go build ./... && go test ./internal/apps/agent/nginx/...`
Expected: PASS
- [ ] **Step 7: 提交**
```bash
git add internal/apps/agent/nginx/sw_assets.go internal/apps/agent/nginx/manager.go
git commit -m "feat(agent): ship sw offline lua assets and placeholder substitution"
```
---
### Task 7: 前端「响应页面」模块(两个 tab)
**Files:**
- Create: `frontend/app/(main)/responses/page.tsx`
- Create: `frontend/app/(main)/responses/components/contact-page-tab.tsx`
- Create: `frontend/app/(main)/responses/components/shared.ts`
- Modify: `frontend/lib/navigation/openflare-nav.ts:63-67`
- Modify: `frontend/lib/navigation/openflare-nav.ts:123`
**Interfaces:**
- Consumes: `OptionService.list()` / `OptionService.updateBatch()`(已存在)。
- Produces: 联系页 tab 编辑 `sw_offline_enabled` / `sw_offline_html` 两个 option。
- [ ] **Step 1: 创建共享 helper `shared.ts`**
```ts
export const OPTIONS_QUERY_KEY = ['openflare', 'options'] as const;
export const KEY_SW_ENABLED = 'sw_offline_enabled';
export const KEY_SW_HTML = 'sw_offline_html';
export type ContactPageFields = {
enabled: boolean;
html: string;
};
export const defaultContactPageFields: ContactPageFields = {
enabled: false,
html: '',
};
export function optionsToMap(options: Array<{ key: string; value: string }>) {
return options.reduce<Record<string, string>>((acc, option) => {
acc[option.key] = option.value;
return acc;
}, {});
}
export function mapOptionsToContactFields(
optionMap: Record<string, string>,
): ContactPageFields {
return {
enabled: optionMap[KEY_SW_ENABLED] === 'true',
html: optionMap[KEY_SW_HTML] ?? '',
};
}
export async function invalidateResponseQueries(queryClient: {
invalidateQueries: (opts: {
queryKey: readonly unknown[];
}) => Promise<unknown>;
}) {
await Promise.all([
queryClient.invalidateQueries({ queryKey: OPTIONS_QUERY_KEY }),
queryClient.invalidateQueries({
queryKey: ['openflare', 'config-preview'],
}),
queryClient.invalidateQueries({
queryKey: ['openflare', 'config-versions'],
}),
]);
}
```
- [ ] **Step 2: 创建联系页 tab `contact-page-tab.tsx`**
参考 `error-pages/page.tsx` 交互:一个「启用」开关 + 一个 HTML 文本域 + 保存按钮。保存 `updateBatch([{key: KEY_SW_ENABLED,...},{key: KEY_SW_HTML,...}])`,成功后 `invalidateResponseQueries`。
```tsx
'use client';
import { useEffect, useState } from 'react';
import { useMutation, useQuery, useQueryClient } from '@tanstack/react-query';
import { Loader2, Save } from 'lucide-react';
import { toast } from 'sonner';
import { Button } from '@/components/ui/button';
import {
Card,
CardContent,
CardDescription,
CardHeader,
CardTitle,
} from '@/components/ui/card';
import { Label } from '@/components/ui/label';
import { Switch } from '@/components/ui/switch';
import { Textarea } from '@/components/ui/textarea';
import { OptionService } from '@/lib/services/openflare';
import {
defaultContactPageFields,
invalidateResponseQueries,
KEY_SW_ENABLED,
KEY_SW_HTML,
mapOptionsToContactFields,
optionsToMap,
type ContactPageFields,
} from './shared';
export function ContactPageTab({ optionMap }: { optionMap: Record<string, string> }) {
const queryClient = useQueryClient();
const [fields, setFields] = useState<ContactPageFields>(
defaultContactPageFields,
);
useEffect(() => {
setFields(mapOptionsToContactFields(optionMap));
}, [optionMap]);
const saveMutation = useMutation({
mutationFn: async () => {
await OptionService.updateBatch([
{ key: KEY_SW_ENABLED, value: String(fields.enabled) },
{ key: KEY_SW_HTML, value: fields.html },
]);
},
onSuccess: async () => {
toast.success('联系页已保存,请前往版本发布使配置生效');
await invalidateResponseQueries(queryClient);
},
onError: (error) => {
toast.error(error instanceof Error ? error.message : '保存失败');
},
});
return (
<div className='space-y-6'>
<Card className='border-dashed shadow-none'>
<CardHeader className='flex flex-row items-start justify-between gap-4 space-y-0'>
<div className='space-y-1.5'>
<CardTitle className='text-base'>离线兜底</CardTitle>
<CardDescription>
启用后给启用 HTTPS 的网站下发 Service Worker,域名被墙时浏览器从缓存展示此联系页。
</CardDescription>
</div>
<Button
size='sm'
className='shrink-0'
disabled={saveMutation.isPending}
onClick={() => saveMutation.mutate()}
>
{saveMutation.isPending ? (
<Loader2 className='size-3.5 animate-spin' />
) : (
<Save className='size-3.5' />
)}
保存
</Button>
</CardHeader>
<CardContent className='space-y-4'>
<div className='flex items-start justify-between gap-6'>
<div className='space-y-1'>
<Label className='text-sm font-medium'>启用 Service Worker 离线兜底</Label>
<p className='text-sm text-muted-foreground'>
仅对 HTTPS 网站生效;未启用的站点不受影响。
</p>
</div>
<Switch
checked={fields.enabled}
onCheckedChange={(enabled) =>
setFields((prev) => ({ ...prev, enabled }))
}
aria-label='启用离线兜底'
className='mt-0.5 shrink-0'
/>
</div>
<div className='flex flex-col gap-3'>
<Label htmlFor='sw-offline-html' className='text-sm font-medium'>
离线联系页 HTML
</Label>
<p className='text-sm text-muted-foreground'>
留空则使用内置默认模板。
</p>
<Textarea
id='sw-offline-html'
value={fields.html}
onChange={(e) =>
setFields((prev) => ({ ...prev, html: e.target.value }))
}
rows={12}
className='font-mono'
disabled={!fields.enabled}
/>
</div>
</CardContent>
</Card>
</div>
);
}
```
**注意:** 确认 `frontend/components/ui/` 存在 `textarea.tsx`(shadcn)。若无,用 `make sure` 或 `npx shadcn@latest add textarea` 添加。
- [ ] **Step 3: 创建页面容器 `responses/page.tsx`**
用 Tabs 组件组织「错误页」「联系页」两个 tab。错误页 tab 复用现有 `error-pages` 内容或重定向;联系页 tab 渲染 `ContactPageTab`。加载 `OptionService.list()` 传入 optionMap。
**实现提示:** 为避免重复,错误页 tab 的现有逻辑(`error-pages/page.tsx` 的 policy 卡片 + 预览卡)可先以 `redirect` 到 `/error-pages` 占位,或直接在容器内嵌两 tab。推荐:容器页 `responses/page.tsx` 读取 options,渲染 Tabs(错误页/联系页),错误页 tab 复用 `frontend/app/(main)/error-pages` 现有 UI(通过 import 其组件或在容器内重构)。**保守实现:** 容器页仅放两个 tab,错误页 tab 用 `<Link href='/error-pages'>` 或保留现有 `/error-pages` 路由,联系页 tab 显示新表单;导航入口改为「响应页面」指向 `/responses`。
- [ ] **Step 4: 更新导航**
`openflare-nav.ts` 第 63-67 行将「错误页」项改为「响应页面」:
```ts
{
title: '响应页面',
url: '/responses',
childUrls: ['/error-pages', '/responses/contact'],
},
```
第 123 行 `openflareWebsiteSubNav` 中 `{ title: '错误页', url: '/error-pages' }` 改为 `{ title: '响应页面', url: '/responses' }`。
- [ ] **Step 5: 构建前端**
Run: `cd /Users/ryan/conductor/workspaces/OpenFlare/islamabad/frontend && pnpm type-check`
Expected: PASS
- [ ] **Step 6: 提交**
```bash
git add frontend/app/\(main\)/responses frontend/lib/navigation/openflare-nav.ts
git commit -m "feat(frontend): add response pages module with contact page tab"
```
---
### Task 8: Changelog 与收尾验证
**Files:**
- Modify: `docs/changelog/index.md`
**Interfaces:**
- Produces: `[Unreleased]` 下用户可读中文条目。
- [ ] **Step 1: 追加 changelog**
在 `docs/changelog/index.md` 的 `[Unreleased]` 下追加:
```markdown
### 新增
- 支持 Service Worker 离线兜底:为启用 HTTPS 的网站下发 Service Worker 并缓存离线联系页,域名无法访问时浏览器展示联系站长页面,减少用户流失。配置位于「响应页面」-「联系页」,可在版本发布中批量生效。
```
- [ ] **Step 2: 运行完整校验**
Run: `cd /Users/ryan/conductor/workspaces/OpenFlare/islamabad && make code-check`
Expected: PASS(golangci-lint + 前端类型检查)
- [ ] **Step 3: 运行后端全量测试**
Run: `go test ./...`
Expected: PASS
- [ ] **Step 4: 格式化**
Run: `cd /Users/ryan/conductor/workspaces/OpenFlare/islamabad && make format`
Expected: 无格式变更或已应用
- [ ] **Step 5: 提交**
```bash
git add docs/changelog/index.md
git commit -m "docs: sw offline fallback changelog"
```
---
## Self-Review
**Spec 覆盖检查:**
- 全局 Option(sw_offline_enabled / html)→ Task 1-3、5 ✓
- 渲染层 SW 静态 + 挑战拦截(反代 + Pages,HTTPS-only)→ Task 4 ✓
- SupportFile 下发 sw.js / offline.html,Agent 占位符替换 → Task 4、6 ✓
- UA 白名单(真实浏览器特征)→ Task 6 `is_real_browser` ✓
- Cookie 长过期 + 首次挑战页 → Task 6 ✓
- 前端「响应页面」两 tab → Task 7 ✓
- 迁移 seed → Task 2 ✓
- Changelog → Task 8 ✓
**占位符扫描:** 无 TBD/TODO。Task 4 Step 5 与 Task 7 Step 3 保留实现细节提示(非占位,是给定方向让执行者按实际代码确认),已在文中明确标注"实现时确认"。
**类型一致性:** `ConfigSnapshot.SWOfflineEnabled/HTML` 在 Task 3/4/5 一致;`SWDirPlaceholder` 在 Task 3/4/6 一致;`sw_offline_enabled/sw_offline_html` key 在 Task 1/2/5/7 一致;`renderServiceWorkerChallenger`/`ServiceWorkerSupportFiles`/`EffectiveSWOfflineHTML`/`ManagedSWLuaFiles` 签名跨 Task 一致。
**已知待确认项(执行时需按实际代码落地):**
- Task 4:`renderAccessBlock` 的 access 块合并(避免 WAF/PoW 被覆盖)。
- Task 6:`certFileTargetPath` 是否支持子目录,落盘目录创建。
- Task 7:`textarea` 组件存在性;「响应页面」错误页 tab 与现有 `/error-pages` 路由的复用策略。
File diff suppressed because it is too large Load Diff
@@ -0,0 +1,177 @@
# 日志数据库解耦设计(ClickHouse 可选化)
> 状态:已与用户逐段确认,待用户复核。
> 日期:2026-08-08
## 1. 背景与目标
当前系统日志/分析(访问日志、可观测时序)完全绑定 ClickHouse:`internal/repository/analytics` 直接操作 `db.ChConn`/`db.ChDB`,apps 层(`chwriter`、`risk_control`、`admin/logs`、`admin/status`)依赖 `config.ClickHouse.Enabled` 判断可用性。业务流量小、主机性能低时 ClickHouse 负担大。
目标:
1. **解耦**:ClickHouse 变为可选项;不启用时,主库(PostgreSQL;禁用时 SQLite)完整承接全部日志功能(写入、查询、聚合、清理)。
2. **代码级约束**:上层应用写日志不能直接调用底层库(`analyticsrepo` / `db.ChConn`),用接口 + import-lint 测试保证,而非 AGENTS.md 口头约束。
3. **可迁移**:提供用户触发的「切换日志数据库」任务,支持 PostgreSQL/SQLite ↔ ClickHouse 数据迁移。
4. **表结构**:CH 日志表迁入 PG/SQLite;CH 保持只有日志表的 SQL 脚本;PG/SQLite 包含全部表。
## 2. 现状要点
- 连接:`internal/infra/persistence/clickhouse.go`(`ChConn` 原生批量写 + `ChDB` GORM 查询),`init()` 依据 `clickhouse.enabled`。
- 分析域:`internal/repository/analytics/` 直接读写 CH;apps 通过 `batchwriter` 异步 flush(`chwriter`、`risk_control`)。
- 已有抽象雏形:`internal/repository/openflare_access_log_store.go` / `openflare_observability_store.go` 中的未导出 `accessLogStore` / `observabilityStore` 接口,默认 `clickhouseAccessLogStore{}`,测试可换 memory 实现——默认写死 CH、不可配置切换、接口未导出。
- 迁移:主库 goose(`goose/postgres` + `goose/sqlite` 双方言)与 CH 单方言(`goose/clickhouse`)分离。
- 历史:PG/SQLite 曾有过 `of_node_metric_snapshots`、`of_node_access_logs` 等观测表(`202606190010_create_of_observability_tables.sql`),后由 `202606200005_drop_of_node_observability_timeseries.sql` 删除(迁去 CH)。**旧 DDL 可复活改造**。
- 任务:Asynq + `task.RegisterHandler`/`RegisterTaskMeta`;`system_cleanup`(系统垃圾清理)每日任务已存在;`of_database_auto_cleanup`(可观测清理,schedule id=102)存在。
- 系统配置:`system_configs` 表(key/type/visibility),现有 `database_auto_cleanup_enabled` / `database_auto_cleanup_retention_days`(business)。
## 3. 已确认的核心决策
| # | 决策 |
|---|---|
| 1 | 范围:CH 不启用时,PG(或 SQLite)承担**全部**日志功能;聚合在 PG/SQLite 查询时实时计算,不物理建 MV 同构表。 |
| 2 | 实现:接口定义在 repository 层;PG 用 GORM 全新实现;CH 保留现有原生批量优化(`PrepareBatch`)包进同一接口。 |
| 3 | SQLite 是一等公民:`log_database` ∈ {`postgres`, `sqlite`, `clickhouse`};迁移方向 PG→CH、SQLite→CH、CH→PG、CH→SQLite。 |
| 4 | 日志库只有两种合法状态:**随主库**(`database.enabled` → postgres,否则 sqlite)或 **clickhouse**;不存在主库 PG + 日志 SQLite 的组合。 |
| 5 | 迁移任务「切换日志数据库」:纯复制、**源数据不删除**、可重试;迁移期间**冻结日志写入**(拒绝,不排队积压);全部成功才翻转主库标记。 |
| 6 | 清理统一到 `system_cleanup`(每日一次,日志过期无需实时);保留时间按**存储库**配置(`type=business`)。 |
## 4. 包结构与接口(方案一)
新增 `internal/repository/logstore/`,职责唯一:日志存储抽象。
```
internal/repository/logstore/
├── logstore.go # 导出接口:AccessLogStore / ObservabilityStore / UserAccessLogStore / CleanupStore / StatusStore
├── provider.go # Open(ctx) 按当前日志主库返回实现;ActiveDatabase() 供状态/UI;测试可注入
├── postgres_store.go # GORM 实现(PG 与 SQLite 共用一套,方言差异只在 goose DDL + dialect_* 小文件)
├── dialect_postgres.go # PG 方言 SQL 片段(date_trunc / FILTER / 分区清理)
├── dialect_sqlite.go # SQLite 方言 SQL 片段(strftime / unixepoch)
└── clickhouse_store.go # 把现有 analyticsrepo 原生批量 + GORM 查询包进接口(零性能损耗)
```
- **接口划分**(避免 40+ 方法巨型接口,合成 `logstore.Store` 结构体持有):
- `AccessLogStore`:节点访问日志的 InsertBatch / List / Count / RegionCounts / BucketAggregates / CountBuckets / BucketDimensions / IPAggregates / IPSummaries / CountIPSummaries / WAFIPAggregates / IPTrend / TrafficSummary / ValueCounts / NodeAggregates / DeleteAll / DeleteBefore / DeleteByNodeBefore。
- `ObservabilityStore`:4 表(metric snapshots / edge health / frps / frpc)的 Insert / List / Delete。
- `UserAccessLogStore`:`w_user_access_logs` 的 BatchInsert / Count / List / 统计(DailyTrend / BrowserDistribution / TopActiveUsers 等)。
- `CleanupStore`:按保留天数清理过期数据(PG=分区 DROP + 分批 DELETE;SQLite=分批 DELETE;CH=MODIFY TTL + materialize)。
- `StatusStore`:当前库状态、CH 运行指标(激活时)、GORM 写入器状态。
- **消费面**:`internal/repository` 现有公开函数(`ListOpenFlareAccessLogs`、`InsertOpenFlareAccessLogsBatch`、`InsertOpenFlareMetricSnapshot` 等)**保留签名、改为一行委托 `logstore`**,apps 调用面几乎不动;apps 里现有 `analyticsrepo` 直连(`risk_control`、`chwriter`、`tasks/database_cleanup.go`、`observability/access_log_logics.go`、`admin/logs`、`admin/status`)全部改走 repository/logstore。
- **import-lint 测试**:新增 `go test`,扫描 `internal/apps/**` 的 import,发现 `internal/repository/analytics` 或 `internal/infra/persistence`(`batchwriter` 白名单除外)即失败。这是「代码层面规避」的验收。
- `analyticsrepo` 保留,仅被 `logstore/clickhouse_store.go` 引用(CH 实现细节)。
### 主库标记与启动校验
- `system_configs` 新增内部 key:
- `log_database`(`postgres`/`sqlite`/`clickhouse`):当前日志主库,仅迁移任务写入。
- `log_db_migration`(`"migrating"`/空):迁移冻结标记,仅迁移任务写入。
- **首次 seed**(bootstrap Go 侧,因依赖运行时主库选择):key 缺失时,`clickhouse.enabled` → `clickhouse`(保持现状、不丢现有 CH 数据);否则 → 当前主库(`database.enabled` → `postgres`,否则 `sqlite`)。
- **启动校验**(bootstrap):
- `log_database=clickhouse` 但 `clickhouse.enabled=false` → 启动报错:「当前日志主库为 ClickHouse 但 ClickHouse 未启用。请先重新启用 ClickHouse 配置并启动,在任务管理运行『切换日志数据库』迁移到 PostgreSQL/SQLite 后再禁用 ClickHouse」。
- `log_database=postgres` 但 `database.enabled=false`,或 `log_database=sqlite` 但 `database.enabled=true` → 启动报错(违反「随主库或随 CH」规则)。
- **key 保护**:`log_database`、`log_db_migration` 在配置更新接口(admin system-configs / option 校验)拒绝修改;仅迁移任务可写;启动校验兜底被篡改组合。
- **热切换**:`logstore` 通过系统配置缓存(Redis,更新即失效)读取 `log_database`;翻转后 API 进程自动切到新实现,无需自定义跨进程协议。
## 5. PG/SQLite 表结构与优化
**新建原始日志表(PG + SQLite 双方言 goose,同版本号)**——只建原始表,**不建** CH 物化视图/聚合表(`of_access_log_hourly`、`of_node_metric_capacity_hourly` 等),PG/SQLite 查询时实时聚合:
| 表 | 说明 |
|---|---|
| `w_user_access_logs` | 用户访问日志 |
| `of_node_access_logs` | 节点访问日志(含 user_agent/cache_status/bytes_sent/request_length/request_time_ms 现行列) |
| `of_node_metric_snapshots` | 资源指标 |
| `of_node_edge_health` | 边缘健康 |
| `of_node_obs_frps` | FRPS 观测 |
| `of_node_obs_frpc` | FRPC 观测 |
- **ID**:沿用 snowflake uint64(DDL 用 BIGINT,与 CH UInt64 对齐);不换自增,保证迁移 ID 原样保留、无冲突。
- **时间**:PG `TIMESTAMPTZ`;SQLite `DATETIME`。
- **复合主键**:分区表主键 `(id, 时间列)`(满足 PG 分区键进唯一索引要求)。
### PG 优化
1. **分区**:仅 `of_node_access_logs`、`w_user_access_logs` 两个高频表用 PG 原生 `PARTITION BY RANGE` **按月分区**;可观测 4 表数据量小,普通表 + 索引。SQLite 无原生分区 → 普通表 + 组合索引(方言差异只留在 goose DDL,运行时 GORM 代码共用)。
2. **批量写入**:PG/SQLite 统一 GORM `CreateInBatches`(批次 500–1000);CH 维持原生 `PrepareBatch`。
3. **索引**:
- `of_node_access_logs`:`(logged_at DESC)`、`(node_id, logged_at DESC)`、`(host, logged_at DESC)`;
- `w_user_access_logs`:`(created_at DESC)`、`(user_id, created_at DESC)`;
- 可观测表:`(node_id, captured_at DESC)`。
4. **聚合查询重写**:PG 用 `date_trunc` / `count(DISTINCT)` / `FILTER (WHERE ...)` 等价替换 CH 的 `toStartOfHour` / `uniqExact` / `countIf`;SQLite 用 `strftime` / `unixepoch`。时间分桶等少量方言 SQL 拆到 `dialect_postgres.go` / `dialect_sqlite.go`,store 主体方言中立。
### goose 迁移
- PG/SQLite 各新增一组建表迁移(复活并改造 `202606190010` 旧 DDL,按 database-migration 技能双方言、同版本号规则)。
- CH 目录不动(本来就只有日志表脚本,满足「CH 保持只有日志表 SQL」)。
## 6. 清理(并入 system_cleanup)
- 日志过期清理并入 `system_cleanup`(系统垃圾清理)每日任务;`of_database_auto_cleanup` 专用 schedule(id=102)与任务下线。
- 新增 `type=business` 配置(替换旧 `database_auto_cleanup_enabled` / `database_auto_cleanup_retention_days`):
- `log_retention_days_postgres`(默认 90)
- `log_retention_days_sqlite`(默认 90)
- `log_retention_days_clickhouse`(默认 90)
- `CleanupStore` 按当前生效库读取对应值执行:
- PG:分区 DROP(整月)+ 分批 DELETE(不满月);
- SQLite:分批 DELETE;
- CH:`ALTER TABLE ... MODIFY TTL toDateTime(...) + INTERVAL N DAY` + materialize(保留期由配置驱动,不再依赖 DDL 写死)。
- 旧 key `database_auto_cleanup_*` 由 goose 迁移删除,前端同步清理。
## 7. 迁移任务「切换日志数据库」
**元数据**:Asynq `openflare:log_db_switch`,管理类型 `of_log_db_switch`,名称「切换日志数据库」,参数 `target`(`postgres`/`sqlite`/`clickhouse`),`Retryable: true`。UI 按当前日志主库只展示合法目标(当前=CH → 「主库」;当前=主库 → 「ClickHouse」)。
**执行流程(worker 进程)**:
1. **校验**:`target == 当前主库` → 拒绝;`target=clickhouse` 但 CH 未启用 / `target=postgres` 但 `database.enabled=false` / `target=sqlite` 但 `database.enabled=true` → 拒绝。
2. **写冻结**:写 `log_db_migration = "migrating"`;先让 batchwriter 把在途批次 flush 完;此后 API 进程所有日志写入路径(`risk_control`、`chwriter` 队列、agent 上报落库)检查该 key → 返回明确错误(HTTP 503「日志数据库迁移中,暂不可写」),不排队积压。
3. **复制**:6 张原始日志表逐表、按 id 分批(每批 ~1000)读源 → 写目标(CH→主库用 GORM `CreateInBatches`;主库→CH 用原生 `PrepareBatch`);ID 原样保留;每表/每批 `task.AppendLog` 进度。
- **幂等前提**:开始复制前**清空目标库日志表**(任务参数「覆盖目标库已有日志」默认开启;目标库通常为空,仅「切回去」场景有旧数据)——保证失败重试可重跑不重复。
4. **翻转**:全部成功 → 更新 `log_database = target`、清除迁移标记 → `logstore` 缓存失效自动切到新实现 → 写入恢复(走新库)。
5. **失败**:返回错误触发 Asynq 重试;**失败时清除迁移标记**,写入继续走源库(不丢功能);重试时重新清空目标 + 复制。
**双进程一致性**:迁移标记与主库标记落在 `system_configs`(Redis 缓存,worker 更新后 API 进程自动失效重读)。
## 8. API 与前端
**后端**:
- `GET /api/v1/admin/status/log-database`(改造现有 `/clickhouse` 状态端点):返回当前日志主库、迁移状态(`idle`/`migrating`)、各库保留天数、当前合法迁移目标;CH 为主时附带现有 CH 运行指标,主库为主时附带 GORM 写入器状态。
- 任务「切换日志数据库」走现有任务管理通用派发 API(`RegisterTaskMeta` + Params),无需新派发接口;执行记录/进度复用任务框架。
- 系统配置:新增 3 个 `log_retention_days_*`(business)图形化 + 参数表可见;新增内部 `log_database`、`log_db_migration`(system、visibility=0、受保护);下线 `database_auto_cleanup_*`。
**前端**:
- 任务管理页:出现「切换日志数据库」,参数下拉只显示合法目标;页面展示当前日志主库与迁移状态。
- `/admin/settings` 业务配置:新增「日志保留时间」分组(PG/SQLite/CH 三个数字输入)。
- 状态/仪表盘:日志库状态卡片(当前库 + 迁移中提示)。
## 9. 测试与验证
- **import-lint 测试**:`internal/apps/**` 不得 import `internal/repository/analytics`、`internal/infra/persistence`(`batchwriter` 白名单除外),违规即失败。
- **logstore 单测**:GORM 实现用 SQLite 全量跑;PG 专属(分区 DROP 等)走既有集成测试路径;CH 实现复用现有 analyticsrepo 测试。
- **迁移任务测试**:目标/组合校验、批处理与 ID 保留、清空目标、翻转标记、失败清标记回退、冻结期写入拒绝——用 memory/sqlite 双端模拟,不依赖真实 CH。
- **清理测试**:`system_cleanup` 日志清理步骤(PG 分区 DROP / SQLite 分批 DELETE / CH TTL 修改)与保留配置读取。
- **迁移验证**:goose 空库 Up 全量(PG/SQLite/CH 三套)、`go test ./...`、`make swagger`(API 变更)、`make code-check`、`make format`。
## 10. 非目标(YAGNI)
- 不在 PG/SQLite 物理建聚合/物化视图表(查询实时聚合)。
- 不做 PG ↔ SQLite 日志互迁(非法组合,启动校验拒绝)。
- 迁移成功不自动删除源库数据(保留,后续提供手动清理入口)。
- 不引入 PG COPY 协议(GORM `CreateInBatches` 对低流量足够)。
- 不引入自定义跨进程迁移协议(`system_configs` + Redis 缓存即可)。
## 11. 里程碑建议(供实现计划分解)
1. **M1 抽象与改造**:`logstore` 接口 + PG/SQLite 实现 + `clickhouse_store` 包装 + import-lint 测试 + repository 委托改造 + apps 直连改造 + `log_database`/`log_db_migration` key 与启动校验。
2. **M2 表与清理**:goose 双方言建表迁移 + 保留配置 key + `system_cleanup` 日志清理步骤 + 下线 `of_database_auto_cleanup` 与旧配置。
3. **M3 迁移任务与展示**:迁移任务 Handler + 状态端点 + 任务管理页/业务配置前端 + 日志库状态卡片。
4. **M4 收尾**:全量验证(goose 三套、单测、`make code-check`/`swagger`/`format`)、文档同步(中文)、changelog `[Unreleased]`。
## 12. 实现归档说明(Task 18,2026-08-08)
- 设计稿第 4 节 provider 入口写作 `Open(ctx)`,实现命名为 `Active(ctx)`(按 `log_database` 解析并缓存,配置翻转后重建),另导出 `Build(ctx, database)` / `BuildForMigration(ctx, database)` 供迁移任务构造目标库 store;`ActiveDatabase(ctx)` 供状态端点。
- 设计稿第 4 节列出的 `CleanupStore` 接口未单独落地:清理实现为包级 `CleanupExpired(ctx)`(按当前激活库保留天数删除过期日志并预建 PG 分区),由 `system_cleanup` 每日任务调用。
- 设计稿第 4 节列举的 `tasks/database_cleanup.go` 已随 M2 下线(`of_database_auto_cleanup` 配置与前端 UI 一并移除),日志清理职责并入 `system_cleanup`。
- 迁移复制按 id 升序分页,`copyObservability` 以每批最后一条 id 作为下一批游标(修正计划中 `lastID += n` 的近似写法);失败回退由 `defer setMigrationFlag("")` 保证源库恢复可写,重试前先清空目标库保证幂等。
- 其余实现决策(`SetConfigReader` 注入、`ensureWritable` 统一冻结、解析 helper 迁至 `model/analytics` 等)见计划「自检记录」,与本文档一致。
@@ -0,0 +1,117 @@
# Service Worker 离线兜底设计(issue #23)
- 日期:2026-08-08
- 状态:设计已确认
- 范围:Proxy Route(反代)+ Pages 静态托管 全覆盖
## 1. 背景与目标
当 CDN 域名被墙、浏览器对所有网络请求失败时,用户会直接流失。本功能通过给网站下发 Service Worker,缓存一个"联系站长"离线页;域名被墙后,SW 从缓存吐出该页,保留用户并引导联系站长。
核心约束:
- 平台一键批量下发,避免逐个 Agent 配置。
- 不改源页代码,全部在 OpenResty 边缘层完成。
- 覆盖反代(Proxy Route)与 Pages 静态托管两种网站类型。
## 2. 机制总览
采用「首次挑战页 + Cookie 放行 + UA 白名单」模式,替代 `sub_filter` 响应体重写。
| 环节 | 行为 |
|---|---|
| 真实浏览器 UA(含特征版本,如 `Chrome/120`)首次访问首页 | 返回 SW 挑战页(内嵌 `register('/sw.js')` 与离线页预缓存),设置长过期 Cookie |
| 带 Cookie 的请求 | 直接放行到上游,正常返回真实页面 |
| 未知 UA(爬虫、curl,无真实浏览器特征) | 直接放过,交给 WAF 处理,拿到真实内容 |
### 为什么不用 sub_filter
`sub_filter` 需处理上游 gzip / Content-Type / 大响应扫描 / 流式缓冲等多处坑。本方案不改上游 body,整体替换首次响应,以上问题全部规避;且爬虫(不匹配真实浏览器 UA)天然绕过挑战页,不伤 SEO。
## 3. 分层职责
```
apps/proxy_route ─┐
apps/pages ─┼─ model → repository → 渲染(pkg/render/openresty) → Agent(OpenResty)
前端设置卡 ─┘ ↑ SW 挑战页 + sw.js/offline 落盘
```
### 后端数据(全局 Option,与 origin error page 同模式)
`sw_offline` 相关配置作为**全局 SystemConfig / OpenRestyConfig snapshot 字段**,对所有启用 HTTPS 的路由生效,实现"一键批量下发"。新增字段:
- `sw_offline_enabled`:是否启用 SW 离线兜底
- `sw_offline_html`:联系站长离线页 HTML 内容(默认提供内置模板)
### 渲染层(`pkg/render/openresty`)
新增 `renderServiceWorkerChallenger(cfg ConfigSnapshot)` 工具,为真实提供内容的 HTTPS server 块(`sw_offline_enabled` 且 `EnableHTTPS` 时)输出:
```nginx
# SW 脚本 + 离线页(作为 support file 落盘)
location = /sw.js { alias .../sw.js; add_header Service-Worker-Allowed /; }
location = /offline.html { alias .../offline.html; }
# 仅首页拦截:真实浏览器 UA 且无 cookie → 返回 SW 挑战页
# 否则(带 cookie / 未知 UA)→ 放行到上游
location = / {
if (真实浏览器UA && 无cookie) { content_by_lua 返回 SW 挑战页; }
放行到上游;
}
```
- SW 逻辑:`install` 阶段缓存 `/offline.html`;`fetch` 事件在网络失败时返回 `caches.match('/offline.html')`。
- 仅在 `EnableHTTPS` 时注入(SW 要求 HTTPS 安全上下文)。
- 多域名 server 块:`/sw.js`、`/offline.html`、挑战页在各 `server_name` 下同源可达。
- 仅对首页 `location = /` 触发;js/css/图片/API/子页面请求不拦,零额外开销。
## 4. 数据流
```
用户首次访问首页(真实UA, 无cookie)
→ OpenResty 判断:真实UA && 无cookie
→ 返回 SW 挑战页 (内嵌 register + 预缓存 offline.html)
→ 浏览器执行 → 注册 SW → 设置长过期 cookie
→ 用户再次请求(带cookie)
→ 放行到上游,正常返回真实页面
域名被墙后
→ 所有请求失败 → SW fetch 兜底 → 从缓存返回 /offline.html(联系页)
```
## 5. 边界与风险
| 项 | 处理 |
|---|---|
| 首次即被墙的用户 | SW 未注册,兜底无效(所有 SW 方案共性,接受) |
| HTTP-only 站点 | 跳过注入(SW 需 HTTPS) |
| 反代多域名 | 各域名同源提供 sw.js / offline.html / 挑战页 |
| Cookie 过期 | 设长过期(约 1 年),过期后重新走一次挑战页 |
| 未知 UA | 放过并交给 WAF 处理,不重复拦截 |
| 资源/API 请求 | 不拦,仅首页触发 |
## 6. 测试
- 渲染层单元测试:
- `sw_offline_enabled` 时输出 sw.js / offline.html / 挑战页 location
- 非 HTTPS 或未启用时不输出
- 仅首页触发,子路径/资源不触发
- UA 判定:真实浏览器 / 爬虫 / curl 三种 UA 的放行分支。
- Cookie 有无的放行分支。
- 现有 config snapshot checksum / rebind 测试不回归。
## 7. 前端命名与入口
离线联系页设置与现有 origin error page 设置合并为同一个功能模块,命名为**「响应页面」**(路由 `responses`),内含两个 tab:
- **错误页设置**:源站错误兜底页(现有 origin error page)
- **联系页设置**:SW 离线兜底联系页(本功能)
两者同属「边缘层兜底展示页」语义,统一管理与入口。
## 8. 待实现确认项(写 plan 时细化)
- SW 挑战页与 sw.js 的具体 Lua 实现与落盘路径(对齐现有 support file 机制)。
- `sw_offline_html` 默认内置模板样式(参考 origin error page 内置模板)。
- 「响应页面」前端模块下错误页/联系页两个 tab 的具体位置与交互。
- UA 白名单默认真实浏览器特征集合(Chrome / Firefox / Safari / Edge + 版本号正则)。
- SW 落盘路径:sw.js / offline.html 通过 SupportFile 下发,Agent 替换占位符(类似 ErrorPageTmplPlaceholder 机制)。
@@ -0,0 +1,191 @@
# SW 离线兜底生效范围(域名作用域)设计
- 日期:2026-08-08
- 状态:设计已确认
- 前置:issue #23 Service Worker 离线兜底(`docs/superpowers/specs/2026-08-08-service-worker-offline-design.md`)
- 范围:SW 注入从「全局所有 HTTPS 站点」细化为「总开关 + 域名作用域」
## 1. 背景与目标
issue #23 实现后,`sw_offline_enabled` 为全局布尔开关:开启后对所有启用 HTTPS 的路由注入 Service Worker 离线兜底。本需求将其细化为可选的**生效域名范围**:
- 保留总开关(`sw_offline_enabled`)。
- 新增作用域:管理员选择需要生效的域名,仅作用域内域名注入 SW。
- 域名选择交互参考 `/cloudflare/groups/1` 的「添加域名成员」弹窗(搜索筛选、按 Zone 分组、批量勾选),但**与 Cloudflare 完全解耦**——仅复用交互模式,数据源为平台自身 zones/zone_domains,不涉及 A 记录同步。
核心约束:
- 语义为「总开关 && 域名 ∈ 作用域」交集:总开关关 → 全部不注入;总开关开 + 作用域空 → 不注入;总开关开 + 域名命中 → 注入。
- 与 Cloudflare 指向分组(A 记录)无任何关联。
- 联系页 HTML(`sw_offline_html`)仍为全局单份,不分域名定制。
## 2. 机制总览
```
sw_offline_enabled (bool, 已有) 总开关
sw_offline_html (string, 已有) 联系页 HTML(全局一份)
sw_offline_domains (JSON 字符串数组, 新增) 生效域名作用域
渲染: routeSWEnabled(routeDomains, cfg)
= SWOfflineEnabled && routeDomains ∩ SWOfflineDomains ≠ ∅
命中 → HTTPS server 块注入 access 检查 + SW location
未命中 → 与 feature 前字节一致
```
Support files(`sw/sw.js`、`sw/offline.html`)仅在「总开关开 && 作用域非空」时下发,避免空作用域产生无用资源。
## 3. 数据层
### 3.1 配置 key
`model.ConfigKeySWOfflineDomains = "sw_offline_domains"`(business 类型,visibility 0),值存 JSON 域名字符串数组:
```json
["example.com", "api.example.com"]
```
### 3.2 goose 迁移(postgres + sqlite 各一份)
`INSERT INTO w_system_configs (key, value, type, visibility, description, created_at, updated_at) VALUES ('sw_offline_domains', '[]', 'business', 0, 'SW 离线兜底生效域名列表(JSON 数组,空则仅总开关无效)', CURRENT_TIMESTAMP, CURRENT_TIMESTAMP) ON CONFLICT (key) DO NOTHING;`
Down 删除该 key。migrator 测试计数 92 → 93,并更新注释。
### 3.3 validator
`validateSWOfflineDomains(key, value string) error`,注册进 `openRestyOptionValidators`:
- JSON 解析为 `[]string`,失败报「必须为 JSON 字符串数组」
- 元素去重(重复报错)
- 元素非空、小写规范化校验(复用/对齐 zone `normalizeDomain` 的域名格式约束:无 `*`、无 `://` `/` `?` `#` `@`、`publicsuffix.EffectiveTLDPlusOne` 可解析)
- 数量上限 `maxSWOfflineDomains = 1000`(防滥用)
### 3.4 config_version snapshot
- `openRestyConfigSnapshot`(`snapshot.go`)新增 `SWOfflineDomains []string json:"sw_offline_domains,omitempty"`。
- `buildOpenRestyConfigSnapshot` 新增 `getStringSliceConfig(key string, defaultVal []string) []string`(解析 JSON 数组,失败回退默认),赋值 `SWOfflineDomains: getStringSliceConfig(model.ConfigKeySWOfflineDomains, nil)`。
- `logics.go`:`diffOpenRestyOptionDetails` 追加 `appendIfChanged("SWOfflineDomains", ...)`;`openRestyOptionKeys()` 追加 `"SWOfflineDomains"`。
## 4. 渲染层(pkg/render/openresty)
### 4.1 ConfigSnapshot
`types.go` 的 `ConfigSnapshot` 新增:
```go
// SWOfflineDomains restricts the offline fallback to matching HTTPS routes.
SWOfflineDomains []string `json:"sw_offline_domains,omitempty"`
```
### 4.2 作用域判断
```go
// routeSWEnabled returns true when SW offline fallback applies to this route.
func routeSWEnabled(routeDomains []string, cfg ConfigSnapshot) bool {
if !cfg.SWOfflineEnabled || len(cfg.SWOfflineDomains) == 0 {
return false
}
scope := make(map[string]struct{}, len(cfg.SWOfflineDomains))
for _, d := range cfg.SWOfflineDomains {
scope[d] = struct{}{}
}
for _, d := range routeDomains {
if _, ok := scope[d]; ok {
return true
}
}
return false
}
```
域名精确匹配(存储时已小写规范化)。
### 4.3 server 渲染签名扩展
- `RenderRouteConfig`:每 route 计算 `swEnabled := routeSWEnabled(domains, doc.OpenRestyConfig)`,传入 `renderProxyRoute` / `renderPagesRoute`(新增 `swEnabled bool` 参数)。
- 下传链路:`renderProxyRouteHTTPS` / `renderPagesRouteHTTPS` / `renderHTTPSServer` / `renderHTTPSPagesServer` 均新增 `swEnabled bool` 参数。
- `swEnabled=true` → `renderAccessBlockWithSW(siteName, powEnabled, cfg)` + 追加 `renderServiceWorkerChallenger(cfg)`(现行为,两函数内部不再判断 `SWOfflineEnabled`,条件已上移到 route 层)。
- `swEnabled=false` → 纯 `renderAccessBlock`,无 challenger(与 feature 前字节一致)。
- HTTP(80)server 块保持不注入(issue #23 已定 HTTPS-only)。
- `renderAccessBlockWithSW` / `renderServiceWorkerChallenger` 保留 `cfg` 参数(HTML 内容来自 `cfg.SWOfflineHTML`),仅移除其内部开关判断。
### 4.4 Support files
`Render` 中生成条件从 `if doc.OpenRestyConfig.SWOfflineEnabled` 改为:
```go
if doc.OpenRestyConfig.SWOfflineEnabled && len(doc.OpenRestyConfig.SWOfflineDomains) > 0 {
files = append(files, ServiceWorkerSupportFiles(doc.OpenRestyConfig)...)
}
```
### 4.5 测试
- `routeSWEnabled`:开关关 / 作用域空 / 无交集 / 单域名交集 / 多域名部分交集。
- HTTPS server 渲染:命中 → 含 `require("sw.runtime").check()` + 三个 SW location;未命中 → 与旧输出字节一致。
- `Render`:空作用域不下发 `sw/*` support files。
- 现有 `TestRenderAccessBlockWithSWMergesSingleBlock` 等适配新签名(`cfg` 语义变化:禁用时不再由内部判断,改由上层传 `swEnabled`)。
## 5. 前端(frontend/app/(main)/responses)
### 5.1 联系页 tab 布局
联系页 tab 两张卡片:
**卡片 1:离线兜底(总开关)**
- 标题「离线兜底」+ 描述。
- 右上角「保存」按钮。
- 「启用 Service Worker 离线兜底」Switch(`sw_offline_enabled`)。
- 「生效范围」区块:当前已选域名 badge 列表(可移除)+「添加域名」按钮打开弹窗;开关关闭时整卡禁用/置灰。
- 保存时 `updateBatch` 一次性提交三个 key:
```ts
{ key: KEY_SW_ENABLED, value: String(fields.enabled) },
{ key: KEY_SW_HTML, value: fields.html },
{ key: KEY_SW_DOMAINS, value: JSON.stringify(fields.domains) },
```
- 保存成功后 `invalidateResponseQueries`(toast 提示「请前往版本发布使配置生效」不变)。
**卡片 2:联系页 HTML**
- 复用 `HtmlEditorWorkspace`(见 5.3),无占位符,实时预览原样 HTML。
### 5.2 域名选择弹窗(scope-domain-dialog.tsx)
- 交互复用 `member-add-dialog.tsx`:搜索框(域名/zone 模糊匹配)、按 Zone 分组折叠、组内勾选/取消、全选可见/清空、已选计数。
- 无橙云开关、无 Cloudflare 依赖。
- 数据源:`ZoneService.list()` + 每 zone `ZoneService.getOverview(id)` 并行拉取(`Promise.all`),zone 根域并入对应分组。**不新增后端 API**。
- 弹窗预勾选当前已生效域名;确认后返回选中的域名字符串数组(覆盖式替换本地 fields.domains)。
- 空态:无 zone 时提示「暂无可用域名,请先在 Zone 管理中注册」。
### 5.3 HtmlEditorWorkspace 复用(泛化)
`frontend/app/(main)/error-pages/components/html-editor-workspace.tsx` 泛化并移至 `frontend/components/common/html-editor-workspace.tsx`:
- Props 扩展:
- `maxBytes?: number`(默认 `ORIGIN_ERROR_PAGE_HTML_MAX_BYTES` = 256 KiB,SW 同为 256 KiB 常量可共用)
- `preview?: (html: string) => string`(默认 `previewOriginErrorPageHTML`;SW 传 `(html) => html` 原样预览)
- `footerHint?: React.ReactNode`(预览 footer 提示文案,默认错误页的「`{{status}}`→502 · `{{host}}`→example.com」;SW 传 `null`)
- 错误页 `edit/page.tsx` 改 import 路径,行为不变。
- `frontend/components/common/` 若不存在则创建目录。
### 5.4 shared.ts 与表单
- `KEY_SW_DOMAINS = 'sw_offline_domains'`。
- `ContactPageFields` 增加 `domains: string[]`;`defaultContactPageFields.domains = []`。
- `mapOptionsToContactFields` 解析 `sw_offline_domains` JSON(容错:非法 JSON → `[]`)。
## 6. 验证
- 后端:`go test ./pkg/render/openresty/... ./internal/apps/openflare/option/... ./internal/apps/openflare/config_version/... ./internal/infra/persistence/migrator/...`
- 前端:`pnpm tsc --noEmit` + `eslint`(联系页新字段/弹窗/多 zone 并行拉取)
- 全量:`go test ./...`、`make code-check`、`make format`
- `make swagger`:无新 API(验证无变更即可)
## 7. Changelog
`docs/changelog/index.md` `[Unreleased]` 更新 SW 条目:新增「可指定生效域名范围(仅对选中的 HTTPS 域名生效)」。
## 8. 已知边界
- 作用域存域名字符串数组:域名从 zone/zone_domain 改名后需手动同步作用域(与 `route.Domains` 精确匹配)。
- 联系页 HTML 全局单份,不分域名定制。
- 空作用域 + 总开关开 → 不注入(前端置灰提示先选域名)。
- 匹配为精确匹配,不跨子域通配(选 `example.com` 不自动覆盖 `api.example.com`,需显式加入)。
+89 -167
View File
@@ -1017,7 +1017,7 @@
"SessionCookie": []
}
],
"description": "分页并按照用户、接口路径、时间范围等维度检索 ClickHouse 用户访问日志列表(需要管理员权限,ClickHouse 未启用时报错)",
"description": "分页并按照用户、接口路径、时间范围等维度检索用户访问日志列表(需要管理员权限,日志存储未启用时报错)",
"produces": [
"application/json"
],
@@ -1085,7 +1085,7 @@
}
},
"400": {
"description": "ClickHouse 未启用或参数错误",
"description": "日志存储未启用或参数错误",
"schema": {
"$ref": "#/definitions/response.Any"
}
@@ -1112,7 +1112,7 @@
"SessionCookie": []
}
],
"description": "聚合统计最近 7 天的每日访问趋势、浏览器分布以及前 10 名最活跃用户排行(需要管理员权限,ClickHouse 未启用时报错)",
"description": "聚合统计最近 7 天的每日访问趋势、浏览器分布以及前 10 名最活跃用户排行(需要管理员权限,日志存储未启用时报错)",
"produces": [
"application/json"
],
@@ -1140,7 +1140,7 @@
}
},
"400": {
"description": "ClickHouse 未启用",
"description": "日志存储未启用",
"schema": {
"$ref": "#/definitions/response.Any"
}
@@ -1870,21 +1870,21 @@
}
}
},
"/api/v1/admin/status/clickhouse": {
"/api/v1/admin/status/log-database": {
"get": {
"security": [
{
"SessionCookie": []
}
],
"description": "返回 ClickHouse parts、mutation、async_insert 队列及进程内 batch writer 指标,需要管理员权限",
"description": "返回当前日志主库、迁移状态、各库保留天数与合法迁移目标,需要管理员权限",
"produces": [
"application/json"
],
"tags": [
"admin"
],
"summary": "获取 ClickHouse 运行指标",
"summary": "获取日志数据库状态",
"responses": {
"200": {
"description": "获取成功",
@@ -1897,19 +1897,13 @@
"type": "object",
"properties": {
"data": {
"$ref": "#/definitions/analytics.ClickHouseOperationalStats"
"$ref": "#/definitions/status.LogDatabaseStatus"
}
}
}
]
}
},
"400": {
"description": "ClickHouse 未启用",
"schema": {
"$ref": "#/definitions/response.Any"
}
},
"401": {
"description": "未登录",
"schema": {
@@ -2386,6 +2380,18 @@
"name": "task_type",
"in": "query"
},
{
"type": "string",
"description": "任务类型前缀筛选(与 task_type / task_types 互斥,精确类型优先)",
"name": "task_type_prefix",
"in": "query"
},
{
"type": "string",
"description": "逗号分隔的精确任务类型列表(IN 筛选,优先于前缀)",
"name": "task_types",
"in": "query"
},
{
"type": "integer",
"default": 1,
@@ -8298,80 +8304,6 @@
}
}
},
"/api/v1/d/option/database/cleanup": {
"post": {
"security": [
{
"SessionCookie": []
}
],
"description": "按目标与保留天数清理可观测性相关数据表,需要管理员权限",
"consumes": [
"application/json"
],
"produces": [
"application/json"
],
"tags": [
"openflare-option"
],
"summary": "清理可观测性数据库",
"parameters": [
{
"description": "清理参数",
"name": "request",
"in": "body",
"schema": {
"$ref": "#/definitions/option.databaseCleanupInput"
}
}
],
"responses": {
"200": {
"description": "清理结果",
"schema": {
"allOf": [
{
"$ref": "#/definitions/response.Any"
},
{
"type": "object",
"properties": {
"data": {
"$ref": "#/definitions/option.databaseCleanupResult"
}
}
}
]
}
},
"400": {
"description": "参数错误",
"schema": {
"$ref": "#/definitions/response.Any"
}
},
"401": {
"description": "未登录",
"schema": {
"$ref": "#/definitions/response.Any"
}
},
"404": {
"description": "无权限或不存在",
"schema": {
"$ref": "#/definitions/response.Any"
}
},
"500": {
"description": "内部错误",
"schema": {
"$ref": "#/definitions/response.Any"
}
}
}
}
},
"/api/v1/d/option/geoip/lookup": {
"post": {
"security": [
@@ -14899,33 +14831,26 @@
}
}
},
"analytics.ClickHouseOperationalStats": {
"analytics.BatchWriterStats": {
"type": "object",
"properties": {
"active_parts": {
"cap": {
"type": "integer"
},
"async_insert_bytes": {
"depth": {
"type": "integer"
},
"async_insert_queue": {
"drops": {
"type": "integer"
},
"batch_writers": {
"description": "BatchWriters reports in-process queue depth/drops/flush errors for CH writers.",
"type": "array",
"items": {
"$ref": "#/definitions/batchwriter.Stats"
}
"flush_errors": {
"type": "integer"
},
"database": {
"name": {
"type": "string"
},
"pending_mutations": {
"type": "integer"
},
"total_rows": {
"type": "integer"
"running": {
"type": "boolean"
}
}
},
@@ -15017,29 +14942,6 @@
}
}
},
"batchwriter.Stats": {
"type": "object",
"properties": {
"cap": {
"type": "integer"
},
"depth": {
"type": "integer"
},
"drops": {
"type": "integer"
},
"flush_errors": {
"type": "integer"
},
"name": {
"type": "string"
},
"running": {
"type": "boolean"
}
}
},
"cache.updateCacheConfigRequest": {
"type": "object",
"required": [
@@ -15098,6 +15000,9 @@
"id": {
"type": "integer"
},
"zone_domain": {
"type": "string"
},
"zone_id": {
"type": "integer"
}
@@ -15819,6 +15724,36 @@
}
}
},
"github_com_Rain-kl_Wavelet_internal_model_analytics.ClickHouseOperationalStats": {
"type": "object",
"properties": {
"active_parts": {
"type": "integer"
},
"async_insert_bytes": {
"type": "integer"
},
"async_insert_queue": {
"type": "integer"
},
"batch_writers": {
"description": "BatchWriters reports in-process queue depth/drops/flush errors for CH writers.",
"type": "array",
"items": {
"$ref": "#/definitions/analytics.BatchWriterStats"
}
},
"database": {
"type": "string"
},
"pending_mutations": {
"type": "integer"
},
"total_rows": {
"type": "integer"
}
}
},
"github_com_Rain-kl_Wavelet_pkg_protocol.ActiveConfigMeta": {
"type": "object",
"properties": {
@@ -18474,46 +18409,6 @@
}
}
},
"option.databaseCleanupInput": {
"type": "object",
"properties": {
"retention_days": {
"type": "integer"
},
"target": {
"type": "string"
}
}
},
"option.databaseCleanupResult": {
"type": "object",
"properties": {
"cleanup_mode": {
"type": "string"
},
"delete_all": {
"type": "boolean"
},
"deleted_count": {
"type": "integer"
},
"eligible_count": {
"type": "integer"
},
"retention_days": {
"type": "integer"
},
"table_ttl_days": {
"type": "integer"
},
"target": {
"type": "string"
},
"target_label": {
"type": "string"
}
}
},
"option.geoIPLookupRequest": {
"type": "object",
"properties": {
@@ -19852,6 +19747,33 @@
}
}
},
"status.LogDatabaseStatus": {
"type": "object",
"properties": {
"active_database": {
"type": "string"
},
"available_targets": {
"type": "array",
"items": {
"type": "string"
}
},
"clickhouse": {
"$ref": "#/definitions/github_com_Rain-kl_Wavelet_internal_model_analytics.ClickHouseOperationalStats"
},
"migration": {
"description": "idle | migrating",
"type": "string"
},
"retention_days": {
"type": "object",
"additionalProperties": {
"type": "integer"
}
}
}
},
"status.SystemStatusResponse": {
"type": "object",
"properties": {
+66 -112
View File
@@ -169,26 +169,20 @@ definitions:
$ref: '#/definitions/github_com_Rain-kl_Wavelet_pkg_protocol.WAFIPGroup'
type: array
type: object
analytics.ClickHouseOperationalStats:
analytics.BatchWriterStats:
properties:
active_parts:
cap:
type: integer
async_insert_bytes:
depth:
type: integer
async_insert_queue:
drops:
type: integer
batch_writers:
description: BatchWriters reports in-process queue depth/drops/flush errors
for CH writers.
items:
$ref: '#/definitions/batchwriter.Stats'
type: array
database:
flush_errors:
type: integer
name:
type: string
pending_mutations:
type: integer
total_rows:
type: integer
running:
type: boolean
type: object
apply_log.CleanupInput:
properties:
@@ -247,21 +241,6 @@ definitions:
is_active:
type: boolean
type: object
batchwriter.Stats:
properties:
cap:
type: integer
depth:
type: integer
drops:
type: integer
flush_errors:
type: integer
name:
type: string
running:
type: boolean
type: object
cache.updateCacheConfigRequest:
properties:
lru_enabled:
@@ -301,6 +280,8 @@ definitions:
type: string
id:
type: integer
zone_domain:
type: string
zone_id:
type: integer
type: object
@@ -774,6 +755,27 @@ definitions:
token:
type: string
type: object
github_com_Rain-kl_Wavelet_internal_model_analytics.ClickHouseOperationalStats:
properties:
active_parts:
type: integer
async_insert_bytes:
type: integer
async_insert_queue:
type: integer
batch_writers:
description: BatchWriters reports in-process queue depth/drops/flush errors
for CH writers.
items:
$ref: '#/definitions/analytics.BatchWriterStats'
type: array
database:
type: string
pending_mutations:
type: integer
total_rows:
type: integer
type: object
github_com_Rain-kl_Wavelet_pkg_protocol.ActiveConfigMeta:
properties:
checksum:
@@ -2533,32 +2535,6 @@ definitions:
window_started_at:
type: string
type: object
option.databaseCleanupInput:
properties:
retention_days:
type: integer
target:
type: string
type: object
option.databaseCleanupResult:
properties:
cleanup_mode:
type: string
delete_all:
type: boolean
deleted_count:
type: integer
eligible_count:
type: integer
retention_days:
type: integer
table_ttl_days:
type: integer
target:
type: string
target_label:
type: string
type: object
option.geoIPLookupRequest:
properties:
ip:
@@ -3442,6 +3418,24 @@ definitions:
version:
type: string
type: object
status.LogDatabaseStatus:
properties:
active_database:
type: string
available_targets:
items:
type: string
type: array
clickhouse:
$ref: '#/definitions/github_com_Rain-kl_Wavelet_internal_model_analytics.ClickHouseOperationalStats'
migration:
description: idle | migrating
type: string
retention_days:
additionalProperties:
type: integer
type: object
type: object
status.SystemStatusResponse:
properties:
alloc:
@@ -4935,7 +4929,7 @@ paths:
- admin
/api/v1/admin/logs/access:
get:
description: 分页并按照用户、接口路径、时间范围等维度检索 ClickHouse 用户访问日志列表(需要管理员权限,ClickHouse 未启用时报错)
description: 分页并按照用户、接口路径、时间范围等维度检索用户访问日志列表(需要管理员权限,日志存储未启用时报错)
parameters:
- default: 1
description: 页码
@@ -4976,7 +4970,7 @@ paths:
$ref: '#/definitions/logs.accessLogsResponse'
type: object
"400":
description: ClickHouse 未启用或参数错误
description: 日志存储未启用或参数错误
schema:
$ref: '#/definitions/response.Any'
"401":
@@ -4994,7 +4988,7 @@ paths:
- admin
/api/v1/admin/logs/analytics:
get:
description: 聚合统计最近 7 天的每日访问趋势、浏览器分布以及前 10 名最活跃用户排行(需要管理员权限,ClickHouse 未启用时报错)
description: 聚合统计最近 7 天的每日访问趋势、浏览器分布以及前 10 名最活跃用户排行(需要管理员权限,日志存储未启用时报错)
produces:
- application/json
responses:
@@ -5008,7 +5002,7 @@ paths:
$ref: '#/definitions/logs.logsAnalyticsResponse'
type: object
"400":
description: ClickHouse 未启用
description: 日志存储未启用
schema:
$ref: '#/definitions/response.Any'
"401":
@@ -5434,9 +5428,9 @@ paths:
summary: 获取系统状态信息
tags:
- admin
/api/v1/admin/status/clickhouse:
/api/v1/admin/status/log-database:
get:
description: 返回 ClickHouse parts、mutation、async_insert 队列及进程内 batch writer 指标,需要管理员权限
description: 返回当前日志主库、迁移状态、各库保留天数与合法迁移目标,需要管理员权限
produces:
- application/json
responses:
@@ -5447,12 +5441,8 @@ paths:
- $ref: '#/definitions/response.Any'
- properties:
data:
$ref: '#/definitions/analytics.ClickHouseOperationalStats'
$ref: '#/definitions/status.LogDatabaseStatus'
type: object
"400":
description: ClickHouse 未启用
schema:
$ref: '#/definitions/response.Any'
"401":
description: 未登录
schema:
@@ -5467,7 +5457,7 @@ paths:
$ref: '#/definitions/response.Any'
security:
- SessionCookie: []
summary: 获取 ClickHouse 运行指标
summary: 获取日志数据库状态
tags:
- admin
/api/v1/admin/system-configs:
@@ -5738,6 +5728,14 @@ paths:
in: query
name: task_type
type: string
- description: 任务类型前缀筛选(与 task_type / task_types 互斥,精确类型优先)
in: query
name: task_type_prefix
type: string
- description: 逗号分隔的精确任务类型列表(IN 筛选,优先于前缀)
in: query
name: task_types
type: string
- default: 1
description: 页码
in: query
@@ -9285,50 +9283,6 @@ paths:
summary: 列出 OpenFlare 配置项
tags:
- openflare-option
/api/v1/d/option/database/cleanup:
post:
consumes:
- application/json
description: 按目标与保留天数清理可观测性相关数据表,需要管理员权限
parameters:
- description: 清理参数
in: body
name: request
schema:
$ref: '#/definitions/option.databaseCleanupInput'
produces:
- application/json
responses:
"200":
description: 清理结果
schema:
allOf:
- $ref: '#/definitions/response.Any'
- properties:
data:
$ref: '#/definitions/option.databaseCleanupResult'
type: object
"400":
description: 参数错误
schema:
$ref: '#/definitions/response.Any'
"401":
description: 未登录
schema:
$ref: '#/definitions/response.Any'
"404":
description: 无权限或不存在
schema:
$ref: '#/definitions/response.Any'
"500":
description: 内部错误
schema:
$ref: '#/definitions/response.Any'
security:
- SessionCookie: []
summary: 清理可观测性数据库
tags:
- openflare-option
/api/v1/d/option/geoip/lookup:
post:
consumes:
@@ -18,8 +18,6 @@ export type OpenFlareOpsFields = {
uptime_kuma_retry: string;
uptime_kuma_retry_interval: string;
uptime_kuma_timeout: string;
database_auto_cleanup_enabled: boolean;
database_auto_cleanup_retention_days: string;
pages_max_package_size_mb: string;
pages_max_history_count: string;
};
@@ -42,8 +40,6 @@ export const defaultOpenFlareOpsFields: OpenFlareOpsFields = {
uptime_kuma_retry: '0',
uptime_kuma_retry_interval: '60',
uptime_kuma_timeout: '48',
database_auto_cleanup_enabled: false,
database_auto_cleanup_retention_days: '30',
pages_max_package_size_mb: '100',
pages_max_history_count: '20',
};
@@ -88,12 +84,6 @@ export function mapOptionsToOpsFields(
uptime_kuma_retry: optionMap.uptime_kuma_retry ?? '0',
uptime_kuma_retry_interval: optionMap.uptime_kuma_retry_interval ?? '60',
uptime_kuma_timeout: optionMap.uptime_kuma_timeout ?? '48',
database_auto_cleanup_enabled: toBoolean(
optionMap.database_auto_cleanup_enabled,
false,
),
database_auto_cleanup_retention_days:
optionMap.database_auto_cleanup_retention_days ?? '30',
pages_max_package_size_mb: optionMap.pages_max_package_size_mb ?? '100',
pages_max_history_count: optionMap.pages_max_history_count ?? '20',
};
@@ -165,16 +155,6 @@ export function validateUptimeKumaFields(fields: OpenFlareOpsFields) {
throw new Error('请求超时必须为正整数。');
}
export function validateDatabaseAutoCleanup(fields: OpenFlareOpsFields) {
const retentionDays = Number.parseInt(
fields.database_auto_cleanup_retention_days,
10,
);
if (Number.isNaN(retentionDays) || retentionDays < 1) {
throw new Error('自动清理保留天数至少为 1 天。');
}
}
export function agentOptionEntries(fields: OpenFlareOpsFields): OptionItem[] {
validateAgentFields(fields);
return [
@@ -220,22 +200,6 @@ export function uptimeKumaOptionEntries(
];
}
export function databaseAutoCleanupEntries(
fields: OpenFlareOpsFields,
): OptionItem[] {
validateDatabaseAutoCleanup(fields);
return [
{
key: 'database_auto_cleanup_enabled',
value: String(fields.database_auto_cleanup_enabled),
},
{
key: 'database_auto_cleanup_retention_days',
value: fields.database_auto_cleanup_retention_days,
},
];
}
export function validatePagesFields(fields: OpenFlareOpsFields) {
const packageSize = Number.parseInt(fields.pages_max_package_size_mb, 10);
const historyCount = Number.parseInt(fields.pages_max_history_count, 10);
@@ -11,20 +11,8 @@ import {
RotateCw,
Save,
Server,
Trash2,
} from 'lucide-react';
import { toast } from 'sonner';
import {
AlertDialog,
AlertDialogAction,
AlertDialogCancel,
AlertDialogContent,
AlertDialogDescription,
AlertDialogFooter,
AlertDialogHeader,
AlertDialogTitle,
} from '@/components/ui/alert-dialog';
import { Button } from '@/components/ui/button';
import {
Card,
@@ -46,7 +34,6 @@ import { Switch } from '@/components/ui/switch';
import { Textarea } from '@/components/ui/textarea';
import { ErrorInline } from '@/components/layout/error';
import { LoadingStateWithBorder } from '@/components/layout/loading';
import type { DatabaseCleanupTarget } from '@/lib/services/openflare';
import {
NodeService,
OptionService,
@@ -57,7 +44,6 @@ import {
import {
agentOptionEntries,
buildDiscoveryCommand,
databaseAutoCleanupEntries,
defaultOpenFlareOpsFields,
formatDurationLabel,
getBrowserOrigin,
@@ -72,31 +58,6 @@ import { UptimeKumaSiteSelectModal } from './uptimekuma-site-modal';
const optionsQueryKey = ['openflare', 'options'] as const;
const openflarePublicStatusQueryKey = ['openflare', 'public-status'] as const;
const cleanupTargets: Array<{
target: DatabaseCleanupTarget;
label: string;
description: string;
}> = [
{
target: 'node_access_logs',
label: '访问日志',
description:
'清理 node_access_logs,影响访问明细与 IP 汇总;表 TTL 为 90 天。',
},
{
target: 'node_metric_snapshots',
label: '性能快照',
description:
'清理 node_metric_snapshots,影响节点资源趋势;表 TTL 为 30 天。',
},
{
target: 'node_edge_health',
label: 'OpenResty 健康',
description:
'清理 node_edge_health(OpenResty 连接/健康快照);表 TTL 为 30 天。业务流量请清理访问日志。',
},
];
async function copyText(value: string) {
await navigator.clipboard.writeText(value);
}
@@ -109,11 +70,6 @@ export function OpenFlareOpsSettings() {
const [savingSection, setSavingSection] = useState<string | null>(null);
const [geoIPTestIP, setGeoIPTestIP] = useState('8.8.8.8');
const [uptimeKumaModalOpen, setUptimeKumaModalOpen] = useState(false);
const [cleanupTarget, setCleanupTarget] = useState<{
target: DatabaseCleanupTarget;
label: string;
} | null>(null);
const [cleanupRetentionDays, setCleanupRetentionDays] = useState('');
const optionsQuery = useQuery({
queryKey: optionsQueryKey,
@@ -194,25 +150,6 @@ export function OpenFlareOpsSettings() {
toast.error(error instanceof Error ? error.message : '同步失败'),
});
const cleanupMutation = useMutation({
mutationFn: (payload: {
target: DatabaseCleanupTarget;
retention_days?: number;
}) => OptionService.cleanupDatabase(payload),
onSuccess: (result) => {
setCleanupTarget(null);
setCleanupRetentionDays('');
toast.success(
result.delete_all
? `已清空${result.target_label},共删除 ${result.deleted_count} 条。`
: `已清理${result.target_label},共删除 ${result.deleted_count} 条。`,
);
},
onError: (error) => {
toast.error(error instanceof Error ? error.message : '清理失败');
},
});
const discoveryToken = bootstrapQuery.data?.discovery_token ?? '';
const discoveryCommand = useMemo(() => {
if (!fields.server_address || !discoveryToken) return '';
@@ -248,17 +185,6 @@ export function OpenFlareOpsSettings() {
}
};
const saveDatabaseAutoCleanup = () => {
try {
saveMutation.mutate({
section: 'database-auto',
entries: databaseAutoCleanupEntries(fields),
});
} catch (error) {
toast.error(error instanceof Error ? error.message : '参数校验失败');
}
};
const savePagesSettings = () => {
try {
saveMutation.mutate({
@@ -690,86 +616,6 @@ export function OpenFlareOpsSettings() {
</CardContent>
</Card>
<div className='grid gap-6 xl:grid-cols-2'>
<Card className='border-dashed shadow-none'>
<CardHeader className='flex flex-row items-center justify-between gap-4'>
<div>
<CardTitle className='text-base'>数据库自动清理</CardTitle>
<CardDescription>
每天凌晨 3 点物化 ClickHouse 表 TTL;访问日志至少保留 90
天,其它观测数据至少保留 30 天。
</CardDescription>
</div>
<Button
size='sm'
disabled={savingSection === 'database-auto'}
onClick={saveDatabaseAutoCleanup}
>
保存
</Button>
</CardHeader>
<CardContent className='space-y-4'>
<ToggleRow
label='启用每日自动清理'
checked={fields.database_auto_cleanup_enabled}
onChange={(value) =>
updateField('database_auto_cleanup_enabled', value)
}
/>
<div className='space-y-1.5'>
<FieldInput
label='期望保留天数'
value={fields.database_auto_cleanup_retention_days}
type='number'
onChange={(value) =>
updateField('database_auto_cleanup_retention_days', value)
}
/>
<p className='text-xs text-muted-foreground'>
小于表 TTL 时自动按下限执行:访问日志 90 天,其它观测数据 30
天。
</p>
</div>
</CardContent>
</Card>
<Card className='border-dashed shadow-none'>
<CardHeader>
<CardTitle className='text-base'>手动数据清理</CardTitle>
<CardDescription>
按表 TTL 清理;输入天数不能小于对应表
TTL,留空时将删除该类数据的全部历史记录。
</CardDescription>
</CardHeader>
<CardContent className='space-y-3'>
{cleanupTargets.map((item) => (
<div
key={item.target}
className='flex items-start justify-between gap-3 rounded-lg border border-dashed p-3'
>
<div>
<p className='text-sm font-medium'>{item.label}</p>
<p className='mt-1 text-xs text-muted-foreground'>
{item.description}
</p>
</div>
<Button
type='button'
variant='destructive'
size='sm'
onClick={() =>
setCleanupTarget({ target: item.target, label: item.label })
}
>
<Trash2 className='size-3.5 mr-1' />
清理
</Button>
</div>
))}
</CardContent>
</Card>
</div>
<UptimeKumaSiteSelectModal
open={uptimeKumaModalOpen}
selectedSites={
@@ -782,56 +628,6 @@ export function OpenFlareOpsSettings() {
updateField('uptime_kuma_selected_sites', sites.join(','))
}
/>
<AlertDialog
open={cleanupTarget !== null}
onOpenChange={(open) => !open && setCleanupTarget(null)}
>
<AlertDialogContent>
<AlertDialogHeader>
<AlertDialogTitle>清理{cleanupTarget?.label}</AlertDialogTitle>
<AlertDialogDescription>
输入保留天数后会按该表 TTL 物化过期数据;小于表 TTL
的天数会被拒绝。留空则删除全部历史记录,操作不可恢复。
</AlertDialogDescription>
</AlertDialogHeader>
<FieldInput
label='保留天数'
value={cleanupRetentionDays}
type='number'
onChange={setCleanupRetentionDays}
placeholder='留空则全部删除'
/>
<AlertDialogFooter>
<AlertDialogCancel disabled={cleanupMutation.isPending}>
取消
</AlertDialogCancel>
<AlertDialogAction
disabled={cleanupMutation.isPending}
onClick={(event) => {
event.preventDefault();
if (!cleanupTarget) return;
const trimmed = cleanupRetentionDays.trim();
if (trimmed !== '') {
const retentionDays = Number.parseInt(trimmed, 10);
if (Number.isNaN(retentionDays) || retentionDays < 1) {
toast.error('保留天数至少为 1 天');
return;
}
cleanupMutation.mutate({
target: cleanupTarget.target,
retention_days: retentionDays,
});
return;
}
cleanupMutation.mutate({ target: cleanupTarget.target });
}}
>
{cleanupMutation.isPending ? '清理中...' : '确认清理'}
</AlertDialogAction>
</AlertDialogFooter>
</AlertDialogContent>
</AlertDialog>
</div>
);
}
@@ -1,6 +1,6 @@
'use client';
import { useCallback, useEffect, useState } from 'react';
import { useCallback, useEffect, useMemo, useState } from 'react';
import { toast } from 'sonner';
import { Button } from '@/components/ui/button';
import { Input } from '@/components/ui/input';
@@ -8,6 +8,13 @@ import { Label } from '@/components/ui/label';
import { Textarea } from '@/components/ui/textarea';
import { Switch } from '@/components/ui/switch';
import { Spinner } from '@/components/ui/spinner';
import {
Select,
SelectContent,
SelectItem,
SelectTrigger,
SelectValue,
} from '@/components/ui/select';
import {
Dialog,
DialogContent,
@@ -19,12 +26,17 @@ import {
import {
Calendar as CalendarIcon,
Clock,
Database,
Info,
Layers,
Play,
} from 'lucide-react';
import type { DispatchTaskRequest, TaskMeta } from '@/lib/services/admin';
import type {
DispatchTaskRequest,
LogDatabaseStatus,
TaskMeta,
} from '@/lib/services/admin';
import services from '@/lib/services';
import { buildTaskPayload } from '@/lib/task-param-utils';
import { ErrorInline } from '@/components/layout/error';
@@ -68,6 +80,18 @@ const TASK_CONFIGS: Record<
gradient:
'from-rose-500/10 via-rose-500/5 to-transparent border-rose-200/50 dark:border-rose-800/50 hover:border-rose-400 dark:hover:border-rose-500',
},
of_log_db_switch: {
icon: Database,
color: 'text-teal-600 dark:text-teal-400',
gradient:
'from-teal-500/10 via-teal-500/5 to-transparent border-teal-200/50 dark:border-teal-800/50 hover:border-teal-400 dark:hover:border-teal-500',
},
};
const LOG_DATABASE_LABELS: Record<string, string> = {
postgres: 'PostgreSQL(主库)',
sqlite: 'SQLite(主库)',
clickhouse: 'ClickHouse',
};
const DEFAULT_TASK_CONFIG = {
@@ -192,9 +216,38 @@ export function TaskManager() {
}
}, []);
const [logDbStatus, setLogDbStatus] = useState<LogDatabaseStatus | null>(
null,
);
// 日志库状态用于「切换日志数据库」卡片与 target 下拉;获取失败不阻塞任务列表。
const fetchLogDbStatus = useCallback(async () => {
try {
const data = await services.adminStatus.getLogDatabaseStatus();
setLogDbStatus(data);
} catch {
setLogDbStatus(null);
}
}, []);
useEffect(() => {
fetchTaskTypes();
}, [fetchTaskTypes]);
fetchLogDbStatus();
}, [fetchTaskTypes, fetchLogDbStatus]);
const availableLogDbTargets = useMemo(
() => logDbStatus?.available_targets ?? [],
[logDbStatus],
);
const retentionSummary = useMemo(() => {
const days = logDbStatus?.retention_days ?? {};
const parts: string[] = [];
if (days.postgres != null) parts.push(`PG ${days.postgres}`);
if (days.sqlite != null) parts.push(`SQLite ${days.sqlite}`);
if (days.clickhouse != null) parts.push(`CH ${days.clickhouse}`);
return parts.join(' / ');
}, [logDbStatus]);
useEffect(() => {
if (selectedTaskType) {
@@ -344,6 +397,45 @@ export function TaskManager() {
</div>
</div>
{task.type === 'of_log_db_switch' && logDbStatus && (
<div className='pt-3 mt-3 border-t border-border/50 space-y-1.5'>
<div className='flex items-center justify-between gap-2'>
<span className='text-[10px] text-muted-foreground shrink-0'>
日志主库
</span>
<span className='text-[10px] font-mono text-foreground truncate'>
{LOG_DATABASE_LABELS[logDbStatus.active_database] ||
logDbStatus.active_database}
</span>
</div>
<div className='flex items-center justify-between gap-2'>
<span className='text-[10px] text-muted-foreground shrink-0'>
保留天数
</span>
<span className='text-[10px] font-mono text-muted-foreground truncate'>
{retentionSummary || '-'}
</span>
</div>
<div className='flex items-center justify-between gap-2'>
<span className='text-[10px] text-muted-foreground shrink-0'>
迁移状态
</span>
<Badge
variant={
logDbStatus.migration === 'migrating'
? 'default'
: 'outline'
}
className='text-[10px] h-5 px-1.5'
>
{logDbStatus.migration === 'migrating'
? '迁移中'
: '空闲'}
</Badge>
</div>
</div>
)}
<div className='pt-4 mt-1'>
<Button
className='w-full h-7 text-xs'
@@ -436,70 +528,104 @@ export function TaskManager() {
}
return (
<div className='space-y-4'>
{targetTask.params.map((param) => (
<div key={param.name} className='grid gap-2'>
<Label
htmlFor={`param-${param.name}`}
className='flex items-center gap-1'
>
{param.label}
{param.required && (
<span className='text-destructive font-bold'>*</span>
)}
</Label>
{param.type === 'text' ? (
<Textarea
id={`param-${param.name}`}
placeholder={param.placeholder}
className='text-xs min-h-[80px]'
value={paramValues[param.name] || ''}
onChange={(e) =>
setParamValues((prev) => ({
...prev,
[param.name]: e.target.value,
}))
}
/>
) : param.type === 'boolean' ? (
<div className='flex items-center gap-2 pt-1 h-9'>
<Switch
{targetTask.params.map((param) => {
const isSwitchTarget =
param.name === 'target' &&
getSelectedTaskMeta()?.type === 'of_log_db_switch';
return (
<div key={param.name} className='grid gap-2'>
<Label
htmlFor={`param-${param.name}`}
className='flex items-center gap-1'
>
{param.label}
{param.required && (
<span className='text-destructive font-bold'>
*
</span>
)}
</Label>
{param.type === 'text' ? (
<Textarea
id={`param-${param.name}`}
checked={paramValues[param.name] === 'true'}
onCheckedChange={(checked) =>
placeholder={param.placeholder}
className='text-xs min-h-[80px]'
value={paramValues[param.name] || ''}
onChange={(e) =>
setParamValues((prev) => ({
...prev,
[param.name]: checked ? 'true' : 'false',
[param.name]: e.target.value,
}))
}
/>
<span className='text-xs text-muted-foreground'>
{paramValues[param.name] === 'true'
? '开启'
: '关闭'}
</span>
</div>
) : (
<Input
id={`param-${param.name}`}
type={param.type === 'number' ? 'number' : 'text'}
placeholder={param.placeholder}
className='text-xs'
value={paramValues[param.name] || ''}
onChange={(e) =>
setParamValues((prev) => ({
...prev,
[param.name]: e.target.value,
}))
}
/>
)}
{param.description && (
<p className='text-[10px] text-muted-foreground'>
{param.description}
</p>
)}
</div>
))}
) : param.type === 'boolean' ? (
<div className='flex items-center gap-2 pt-1 h-9'>
<Switch
id={`param-${param.name}`}
checked={paramValues[param.name] === 'true'}
onCheckedChange={(checked) =>
setParamValues((prev) => ({
...prev,
[param.name]: checked ? 'true' : 'false',
}))
}
/>
<span className='text-xs text-muted-foreground'>
{paramValues[param.name] === 'true'
? '开启'
: '关闭'}
</span>
</div>
) : isSwitchTarget &&
availableLogDbTargets.length > 0 ? (
<Select
value={paramValues[param.name] || ''}
onValueChange={(value) =>
setParamValues((prev) => ({
...prev,
[param.name]: value,
}))
}
disabled={dispatching}
>
<SelectTrigger
id={`param-${param.name}`}
className='w-full text-xs'
size='sm'
>
<SelectValue placeholder='选择目标日志库...' />
</SelectTrigger>
<SelectContent>
{availableLogDbTargets.map((target) => (
<SelectItem key={target} value={target}>
{LOG_DATABASE_LABELS[target] || target}
</SelectItem>
))}
</SelectContent>
</Select>
) : (
<Input
id={`param-${param.name}`}
type={param.type === 'number' ? 'number' : 'text'}
placeholder={param.placeholder}
className='text-xs'
value={paramValues[param.name] || ''}
onChange={(e) =>
setParamValues((prev) => ({
...prev,
[param.name]: e.target.value,
}))
}
/>
)}
{param.description && (
<p className='text-[10px] text-muted-foreground'>
{param.description}
</p>
)}
</div>
);
})}
</div>
);
})()}
@@ -1,70 +0,0 @@
'use client';
import type { ReactNode } from 'react';
import { FileWarning } from 'lucide-react';
import type { UseQueryResult } from '@tanstack/react-query';
import { useAuth } from '@/components/providers/auth-provider';
import { EmptyStateWithBorder } from '@/components/layout/empty';
import { ErrorInline } from '@/components/layout/error';
import { LoadingStateWithBorder } from '@/components/layout/loading';
type ErrorPageGateProps = {
optionsQuery: UseQueryResult<Array<{ key: string; value: string }>, Error>;
children: ReactNode;
};
export function ErrorPageGate({ optionsQuery, children }: ErrorPageGateProps) {
const { user, loading: authLoading } = useAuth();
if (authLoading) {
return (
<div className='py-6 px-1'>
<LoadingStateWithBorder
icon={FileWarning}
description='加载权限信息...'
/>
</div>
);
}
if (!user?.is_admin) {
return (
<div className='py-6 px-1'>
<EmptyStateWithBorder
icon={FileWarning}
title='权限不足'
description='只有管理员可以访问源站错误页设置。'
/>
</div>
);
}
if (optionsQuery.isLoading) {
return (
<div className='py-6 px-1'>
<LoadingStateWithBorder
icon={FileWarning}
description='加载错误页配置...'
/>
</div>
);
}
if (optionsQuery.isError) {
return (
<div className='py-6 px-1'>
<ErrorInline
message={
optionsQuery.error instanceof Error
? optionsQuery.error.message
: '加载失败'
}
onRetry={() => void optionsQuery.refetch()}
/>
</div>
);
}
return <>{children}</>;
}
@@ -1,204 +0,0 @@
'use client';
import Link from 'next/link';
import { useEffect, useState } from 'react';
import { useMutation, useQuery, useQueryClient } from '@tanstack/react-query';
import {
ArrowLeft,
FileCode2,
Loader2,
RotateCcw,
Save,
Sparkles,
} from 'lucide-react';
import { toast } from 'sonner';
import { useAuth } from '@/components/providers/auth-provider';
import {
AlertDialog,
AlertDialogAction,
AlertDialogCancel,
AlertDialogContent,
AlertDialogDescription,
AlertDialogFooter,
AlertDialogHeader,
AlertDialogTitle,
} from '@/components/ui/alert-dialog';
import { Button } from '@/components/ui/button';
import {
Select,
SelectContent,
SelectItem,
SelectTrigger,
SelectValue,
} from '@/components/ui/select';
import {
DEFAULT_ORIGIN_ERROR_PAGE_TEMPLATE_ID,
getOriginErrorPageTemplate,
ORIGIN_ERROR_PAGE_TEMPLATES,
} from '@/lib/openflare/origin-error-page-templates';
import { OptionService } from '@/lib/services/openflare';
import { ErrorPageGate } from '../components/error-page-gate';
import { HtmlEditorWorkspace } from '../components/html-editor-workspace';
import {
invalidateErrorPageQueries,
KEY_HTML,
mapOptionsToFields,
OPTIONS_QUERY_KEY,
optionsToMap,
validateErrorPageHTML,
} from '../components/shared';
export default function ErrorPageEditPage() {
const { user } = useAuth();
const queryClient = useQueryClient();
const [html, setHtml] = useState('');
const [restoreOpen, setRestoreOpen] = useState(false);
const [templateId, setTemplateId] = useState(
DEFAULT_ORIGIN_ERROR_PAGE_TEMPLATE_ID,
);
const optionsQuery = useQuery({
queryKey: OPTIONS_QUERY_KEY,
queryFn: () => OptionService.list(),
enabled: !!user?.is_admin,
});
useEffect(() => {
if (!optionsQuery.data) return;
setHtml(mapOptionsToFields(optionsToMap(optionsQuery.data)).html);
}, [optionsQuery.data]);
const saveMutation = useMutation({
mutationFn: async () => {
validateErrorPageHTML(html);
await OptionService.updateBatch([{ key: KEY_HTML, value: html }]);
},
onSuccess: async () => {
toast.success('HTML 模板已保存,请前往版本发布使配置生效');
await invalidateErrorPageQueries(queryClient);
},
onError: (error) => {
toast.error(error instanceof Error ? error.message : '保存失败');
},
});
const loadSelectedTemplate = () => {
const tmpl = getOriginErrorPageTemplate(templateId);
if (!tmpl) {
toast.error('未找到所选模板');
return;
}
setHtml(tmpl.html);
toast.success(`已加载模板「${tmpl.name}」到编辑器`);
};
const restoreDefault = () => {
setHtml('');
setRestoreOpen(false);
toast.success('已清空为使用服务端内置默认模板(需保存后生效)');
};
return (
<ErrorPageGate optionsQuery={optionsQuery}>
<div className='flex flex-col gap-4 py-6 px-1'>
<div className='flex flex-col gap-3 sm:flex-row sm:items-center sm:justify-between'>
<div className='flex items-center gap-3'>
<Button variant='outline' size='icon' className='h-8 w-8' asChild>
<Link href='/error-pages' aria-label='返回错误页'>
<ArrowLeft className='size-4' />
</Link>
</Button>
<div className='flex items-center gap-2'>
<FileCode2 className='size-5 text-primary' />
<div>
<h1 className='text-2xl font-semibold tracking-tight'>编辑</h1>
</div>
</div>
</div>
</div>
<HtmlEditorWorkspace
value={html}
onChange={setHtml}
toolbarRight={
<>
<div className='flex items-center gap-1.5'>
<span className='text-[11px] text-muted-foreground hidden sm:inline'>
内置模板
</span>
<Select value={templateId} onValueChange={setTemplateId}>
<SelectTrigger className='h-7 w-[160px] text-[11px] bg-background'>
<SelectValue placeholder='选择模板' />
</SelectTrigger>
<SelectContent>
{ORIGIN_ERROR_PAGE_TEMPLATES.map((tmpl) => (
<SelectItem
key={tmpl.id}
value={tmpl.id}
className='text-xs'
>
{tmpl.name}
</SelectItem>
))}
</SelectContent>
</Select>
<Button
type='button'
variant='ghost'
size='sm'
className='h-7 px-2 text-[11px]'
onClick={loadSelectedTemplate}
>
<Sparkles className='size-3.5' />
加载模板
</Button>
</div>
<Button
type='button'
variant='ghost'
size='sm'
className='h-7 px-2 text-[11px]'
onClick={() => setRestoreOpen(true)}
>
<RotateCcw className='size-3.5' />
恢复默认
</Button>
<Button
size='sm'
className='h-7 px-3 text-[11px]'
disabled={saveMutation.isPending}
onClick={() => saveMutation.mutate()}
>
{saveMutation.isPending ? (
<Loader2 className='size-3 animate-spin' />
) : (
<Save className='size-3' />
)}
保存
</Button>
</>
}
/>
</div>
<AlertDialog open={restoreOpen} onOpenChange={setRestoreOpen}>
<AlertDialogContent>
<AlertDialogHeader>
<AlertDialogTitle>恢复默认 HTML?</AlertDialogTitle>
<AlertDialogDescription>
将清空编辑器内容,保存后使用服务端内置默认模板。此操作不会自动保存。
</AlertDialogDescription>
</AlertDialogHeader>
<AlertDialogFooter>
<AlertDialogCancel>取消</AlertDialogCancel>
<AlertDialogAction onClick={restoreDefault}>
确认恢复
</AlertDialogAction>
</AlertDialogFooter>
</AlertDialogContent>
</AlertDialog>
</ErrorPageGate>
);
}
-237
View File
@@ -1,237 +0,0 @@
'use client';
import Link from 'next/link';
import { useEffect, useMemo, useState } from 'react';
import { useMutation, useQuery, useQueryClient } from '@tanstack/react-query';
import { Expand, FileWarning, Loader2, Pencil, Save } from 'lucide-react';
import { toast } from 'sonner';
import { useAuth } from '@/components/providers/auth-provider';
import { Button } from '@/components/ui/button';
import {
Card,
CardContent,
CardDescription,
CardHeader,
CardTitle,
} from '@/components/ui/card';
import { Label } from '@/components/ui/label';
import { Switch } from '@/components/ui/switch';
import { TagsInput } from '@/components/ui/tags-input';
import { previewOriginErrorPageHTML } from '@/lib/openflare/default-origin-error-page-html';
import {
validateStatusCodeTagMessage,
validateStatusCodeTags,
} from '@/lib/openflare/status-code-tags';
import { OptionService } from '@/lib/services/openflare';
import { ErrorPageGate } from './components/error-page-gate';
import {
defaultErrorPageFields,
invalidateErrorPageQueries,
KEY_ENABLED,
KEY_GET_ONLY,
KEY_STATUS_CODES,
mapOptionsToFields,
OPTIONS_QUERY_KEY,
optionsToMap,
type ErrorPageFields,
} from './components/shared';
export default function ErrorPagesPage() {
const { user } = useAuth();
const queryClient = useQueryClient();
const [fields, setFields] = useState<ErrorPageFields>(defaultErrorPageFields);
const [tagError, setTagError] = useState<string | null>(null);
const optionsQuery = useQuery({
queryKey: OPTIONS_QUERY_KEY,
queryFn: () => OptionService.list(),
enabled: !!user?.is_admin,
});
useEffect(() => {
if (!optionsQuery.data) return;
setFields(mapOptionsToFields(optionsToMap(optionsQuery.data)));
setTagError(null);
}, [optionsQuery.data]);
const previewSrcDoc = useMemo(
() => previewOriginErrorPageHTML(fields.html),
[fields.html],
);
/** 仅保存策略项(HTML 在编辑页单独保存) */
const savePolicyMutation = useMutation({
mutationFn: async () => {
validateStatusCodeTags(fields.statusCodes);
await OptionService.updateBatch([
{ key: KEY_ENABLED, value: String(fields.enabled) },
{ key: KEY_GET_ONLY, value: String(fields.getOnly) },
{
key: KEY_STATUS_CODES,
value: JSON.stringify(fields.statusCodes),
},
]);
},
onSuccess: async () => {
toast.success('触发策略已保存,请前往版本发布使配置生效');
await invalidateErrorPageQueries(queryClient);
},
onError: (error) => {
toast.error(error instanceof Error ? error.message : '保存失败');
},
});
const handleValidateTag = (tag: string) => {
const message = validateStatusCodeTagMessage(tag);
if (message) {
setTagError(message);
toast.error(message);
return message;
}
setTagError(null);
return null;
};
return (
<ErrorPageGate optionsQuery={optionsQuery}>
<div className='flex flex-col gap-6 py-6 px-1'>
<div className='flex items-center gap-2'>
<FileWarning className='size-5 text-primary' />
<div>
<h1 className='text-2xl font-semibold tracking-tight'>错误页</h1>
<p className='text-sm text-muted-foreground'>
配置源站/网关错误响应时的统一 HTML
页面。策略与模板保存后需发布配置版本后生效。
</p>
</div>
</div>
<Card className='border-dashed shadow-none'>
<CardHeader className='flex flex-row items-start justify-between gap-4 space-y-0'>
<div className='space-y-1.5'>
<CardTitle className='text-base'>触发策略</CardTitle>
<CardDescription>
配置源站错误页的启用条件与触发状态码。修改后需点击保存。
</CardDescription>
</div>
<Button
size='sm'
className='shrink-0'
disabled={savePolicyMutation.isPending}
onClick={() => savePolicyMutation.mutate()}
>
{savePolicyMutation.isPending ? (
<Loader2 className='size-3.5 animate-spin' />
) : (
<Save className='size-3.5' />
)}
保存
</Button>
</CardHeader>
<CardContent className='space-y-0 divide-y'>
<div className='flex items-start justify-between gap-6 pb-5'>
<div className='space-y-1'>
<Label className='text-sm font-medium'>启用源站错误页</Label>
<p className='text-sm text-muted-foreground'>
关闭后会透传源站或 Nginx 默认错误响应。
</p>
</div>
<Switch
checked={fields.enabled}
onCheckedChange={(enabled) =>
setFields((prev) => ({ ...prev, enabled }))
}
aria-label='启用源站错误页'
className='mt-0.5 shrink-0'
/>
</div>
<div className='flex items-start justify-between gap-6 py-5'>
<div className='space-y-1'>
<Label className='text-sm font-medium'>仅针对 GET 请求</Label>
<p className='text-sm text-muted-foreground'>
开启后仅对 GET
请求的匹配错误状态码返回自定义错误页;POST/PUT
等其它方法直接透传源站响应。
</p>
</div>
<Switch
checked={fields.getOnly}
disabled={!fields.enabled}
onCheckedChange={(getOnly) =>
setFields((prev) => ({ ...prev, getOnly }))
}
aria-label='仅针对 GET 请求'
className='mt-0.5 shrink-0'
/>
</div>
<div className='flex flex-col gap-3 pt-5'>
<div className='space-y-1'>
<Label
htmlFor='origin-error-status-codes'
className='text-sm font-medium'
>
触发状态码
</Label>
<p className='text-sm text-muted-foreground'>
支持单码(如 502)或闭区间(如 500-599),范围 400–599。默认
500-599。
</p>
</div>
<TagsInput
id='origin-error-status-codes'
value={fields.statusCodes}
onChange={(statusCodes) => {
setFields((prev) => ({ ...prev, statusCodes }));
setTagError(null);
}}
validateTag={handleValidateTag}
placeholder='例如 502 或 500-599,回车添加'
aria-invalid={!!tagError}
disabled={!fields.enabled}
/>
{tagError ? (
<p className='text-xs text-destructive'>{tagError}</p>
) : null}
</div>
</CardContent>
</Card>
<Card className='border-dashed shadow-none overflow-hidden'>
<CardHeader className='flex flex-row items-start justify-between gap-3 space-y-0'>
<div className='space-y-1.5'>
<CardTitle className='text-base'>页面预览</CardTitle>
</div>
<div className='flex shrink-0 flex-wrap gap-2'>
<Button variant='outline' size='sm' asChild>
<Link href='/error-pages/preview'>
<Expand className='size-3.5' />
预览
</Link>
</Button>
<Button size='sm' asChild>
<Link href='/error-pages/edit'>
<Pencil className='size-3.5' />
编辑
</Link>
</Button>
</div>
</CardHeader>
<CardContent>
<div className='overflow-hidden rounded-md border bg-muted/30'>
<iframe
title='源站错误页预览'
sandbox=''
srcDoc={previewSrcDoc}
className='h-[32rem] w-full bg-background'
/>
</div>
</CardContent>
</Card>
</div>
</ErrorPageGate>
);
}
@@ -1,79 +0,0 @@
'use client';
import Link from 'next/link';
import { useMemo } from 'react';
import { useQuery } from '@tanstack/react-query';
import { ArrowLeft, Pencil, X } from 'lucide-react';
import { useAuth } from '@/components/providers/auth-provider';
import { Button } from '@/components/ui/button';
import { previewOriginErrorPageHTML } from '@/lib/openflare/default-origin-error-page-html';
import { OptionService } from '@/lib/services/openflare';
import { ErrorPageGate } from '../components/error-page-gate';
import {
mapOptionsToFields,
OPTIONS_QUERY_KEY,
optionsToMap,
} from '../components/shared';
/**
* 全屏「真实预览」:覆盖主布局,按访客视角展示替换后的 HTML。
*/
export default function ErrorPagePreviewPage() {
const { user } = useAuth();
const optionsQuery = useQuery({
queryKey: OPTIONS_QUERY_KEY,
queryFn: () => OptionService.list(),
enabled: !!user?.is_admin,
});
const html = useMemo(() => {
if (!optionsQuery.data) return '';
return mapOptionsToFields(optionsToMap(optionsQuery.data)).html;
}, [optionsQuery.data]);
const previewSrcDoc = useMemo(() => previewOriginErrorPageHTML(html), [html]);
return (
<ErrorPageGate optionsQuery={optionsQuery}>
<div className='fixed inset-0 z-50 flex flex-col bg-background'>
<div className='flex items-center justify-between gap-3 border-b bg-background/95 px-4 py-2 backdrop-blur shrink-0'>
<div className='flex items-center gap-2 min-w-0'>
<Button variant='outline' size='icon' className='h-8 w-8' asChild>
<Link href='/error-pages' aria-label='返回错误页'>
<ArrowLeft className='size-4' />
</Link>
</Button>
<div className='min-w-0'>
<p className='text-sm font-semibold truncate'>错误页真实预览</p>
<p className='text-[11px] text-muted-foreground font-mono truncate'>
{'{{status}}'}→502 · {'{{host}}'}→example.com · 全屏展示
</p>
</div>
</div>
<div className='flex items-center gap-2 shrink-0'>
<Button variant='outline' size='sm' asChild>
<Link href='/error-pages/edit'>
<Pencil className='size-3.5' />
编辑
</Link>
</Button>
<Button variant='ghost' size='icon' className='h-8 w-8' asChild>
<Link href='/error-pages' aria-label='关闭预览'>
<X className='size-4' />
</Link>
</Button>
</div>
</div>
<iframe
title='源站错误页真实预览'
sandbox=''
srcDoc={previewSrcDoc}
className='flex-1 w-full border-0 bg-background min-h-0'
/>
</div>
</ErrorPageGate>
);
}
@@ -0,0 +1,214 @@
'use client';
import Link from 'next/link';
import { useEffect, useMemo, useState } from 'react';
import { useMutation, useQueryClient } from '@tanstack/react-query';
import { Expand, Loader2, Pencil, Save } from 'lucide-react';
import { toast } from 'sonner';
import { Button } from '@/components/ui/button';
import {
Card,
CardContent,
CardDescription,
CardHeader,
CardTitle,
} from '@/components/ui/card';
import { Label } from '@/components/ui/label';
import { Switch } from '@/components/ui/switch';
import { TagsInput } from '@/components/ui/tags-input';
import { previewOriginErrorPageHTML } from '@/lib/openflare/default-origin-error-page-html';
import {
validateStatusCodeTagMessage,
validateStatusCodeTags,
} from '@/lib/openflare/status-code-tags';
import { OptionService } from '@/lib/services/openflare';
import {
defaultErrorPageFields,
invalidateResponseQueries,
KEY_ENABLED,
KEY_GET_ONLY,
KEY_STATUS_CODES,
mapOptionsToErrorFields,
type ErrorPageFields,
} from './shared';
export function ErrorPageTab({
optionMap,
}: {
optionMap: Record<string, string>;
}) {
const queryClient = useQueryClient();
const [fields, setFields] = useState<ErrorPageFields>(defaultErrorPageFields);
const [tagError, setTagError] = useState<string | null>(null);
useEffect(() => {
setFields(mapOptionsToErrorFields(optionMap));
setTagError(null);
}, [optionMap]);
const previewSrcDoc = useMemo(
() => previewOriginErrorPageHTML(fields.html),
[fields.html],
);
/** 仅保存策略项 */
const savePolicyMutation = useMutation({
mutationFn: async () => {
validateStatusCodeTags(fields.statusCodes);
await OptionService.updateBatch([
{ key: KEY_ENABLED, value: String(fields.enabled) },
{ key: KEY_GET_ONLY, value: String(fields.getOnly) },
{
key: KEY_STATUS_CODES,
value: JSON.stringify(fields.statusCodes),
},
]);
},
onSuccess: async () => {
toast.success('触发策略已保存,请前往版本发布使配置生效');
await invalidateResponseQueries(queryClient);
},
onError: (error) => {
toast.error(error instanceof Error ? error.message : '保存失败');
},
});
const handleValidateTag = (tag: string) => {
const message = validateStatusCodeTagMessage(tag);
if (message) {
setTagError(message);
toast.error(message);
return message;
}
setTagError(null);
return null;
};
return (
<div className='space-y-6'>
<Card className='border-dashed shadow-none'>
<CardHeader className='flex flex-row items-start justify-between gap-4 space-y-0'>
<div className='space-y-1.5'>
<CardTitle className='text-base'>触发策略</CardTitle>
<CardDescription>
配置源站错误页的启用条件与触发状态码。
</CardDescription>
</div>
<Button
size='sm'
className='shrink-0'
disabled={savePolicyMutation.isPending}
onClick={() => savePolicyMutation.mutate()}
>
{savePolicyMutation.isPending ? (
<Loader2 className='size-3.5 animate-spin' />
) : (
<Save className='size-3.5' />
)}
保存
</Button>
</CardHeader>
<CardContent className='space-y-0 divide-y'>
<div className='flex items-start justify-between gap-6 pb-5'>
<div className='space-y-1'>
<Label className='text-sm font-medium'>启用源站错误页</Label>
<p className='text-sm text-muted-foreground'>
关闭后会透传源站或 Nginx 默认错误响应。
</p>
</div>
<Switch
checked={fields.enabled}
onCheckedChange={(enabled) =>
setFields((prev) => ({ ...prev, enabled }))
}
aria-label='启用源站错误页'
className='mt-0.5 shrink-0'
/>
</div>
<div className='flex items-start justify-between gap-6 py-5'>
<div className='space-y-1'>
<Label className='text-sm font-medium'>仅针对 GET 请求</Label>
<p className='text-sm text-muted-foreground'>
开启后仅对 GET 请求的匹配错误状态码返回自定义错误页。
</p>
</div>
<Switch
checked={fields.getOnly}
disabled={!fields.enabled}
onCheckedChange={(getOnly) =>
setFields((prev) => ({ ...prev, getOnly }))
}
aria-label='仅针对 GET 请求'
className='mt-0.5 shrink-0'
/>
</div>
<div className='flex flex-col gap-3 pt-5'>
<div className='space-y-1'>
<Label
htmlFor='origin-error-status-codes'
className='text-sm font-medium'
>
触发状态码
</Label>
<p className='text-sm text-muted-foreground'>
支持单码(如 502)或闭区间(如 500-599)。
</p>
</div>
<TagsInput
id='origin-error-status-codes'
value={fields.statusCodes}
onChange={(statusCodes) => {
setFields((prev) => ({ ...prev, statusCodes }));
setTagError(null);
}}
validateTag={handleValidateTag}
placeholder='例如 502 或 500-599,回车添加'
aria-invalid={!!tagError}
disabled={!fields.enabled}
/>
{tagError ? (
<p className='text-xs text-destructive'>{tagError}</p>
) : null}
</div>
</CardContent>
</Card>
<Card className='border-dashed shadow-none overflow-hidden'>
<CardHeader className='flex flex-row items-start justify-between gap-3 space-y-0'>
<div className='space-y-1.5'>
<CardTitle className='text-base'>页面预览</CardTitle>
</div>
<div className='flex shrink-0 flex-wrap gap-2'>
<Button variant='outline' size='sm' asChild>
<Link href='/responses/error-page/preview'>
<Expand className='size-3.5' />
预览
</Link>
</Button>
<Button size='sm' asChild>
<Link href='/responses/error-page/edit'>
<Pencil className='size-3.5' />
编辑
</Link>
</Button>
</div>
</CardHeader>
<CardContent>
<div className='overflow-hidden rounded-md border bg-muted/30'>
<iframe
title='源站错误页预览'
sandbox=''
srcDoc={previewSrcDoc}
className='h-[32rem] w-full bg-background'
/>
</div>
</CardContent>
</Card>
</div>
);
}
@@ -0,0 +1,232 @@
'use client';
import Link from 'next/link';
import { useEffect, useMemo, useState } from 'react';
import { useMutation, useQuery, useQueryClient } from '@tanstack/react-query';
import { Expand, Loader2, Pencil, Plus, Save, X } from 'lucide-react';
import { toast } from 'sonner';
import { Badge } from '@/components/ui/badge';
import { Button } from '@/components/ui/button';
import {
Card,
CardContent,
CardDescription,
CardHeader,
CardTitle,
} from '@/components/ui/card';
import { Label } from '@/components/ui/label';
import { Switch } from '@/components/ui/switch';
import { effectiveOfflinePageHTML } from '@/lib/openflare/offline-page-templates';
import {
OptionService,
ZoneService,
zoneQueryKey,
} from '@/lib/services/openflare';
import { ScopeDomainDialog } from './scope-domain-dialog';
import {
defaultOfflinePageFields,
invalidateResponseQueries,
KEY_SW_DOMAINS,
KEY_SW_ENABLED,
KEY_SW_HTML,
mapOptionsToOfflineFields,
type OfflinePageFields,
} from './shared';
export function OfflinePageTab({
optionMap,
}: {
optionMap: Record<string, string>;
}) {
const queryClient = useQueryClient();
const [fields, setFields] = useState<OfflinePageFields>(
defaultOfflinePageFields,
);
const [scopeOpen, setScopeOpen] = useState(false);
useEffect(() => {
setFields(mapOptionsToOfflineFields(optionMap));
}, [optionMap]);
const previewSrcDoc = useMemo(
() => effectiveOfflinePageHTML(fields.html),
[fields.html],
);
const zonesQuery = useQuery({
queryKey: [...zoneQueryKey, 'sw-scope'],
queryFn: async () => {
const zones = await ZoneService.list();
const overviews = await Promise.all(
zones.map((zone) => ZoneService.getOverview(zone.id)),
);
return overviews.map((ov) => ({
zoneDomain: ov.zone.domain,
domains: [ov.zone.domain, ...ov.domains.map((d) => d.domain)],
}));
},
});
const saveMutation = useMutation({
mutationFn: async () => {
await OptionService.updateBatch([
{ key: KEY_SW_ENABLED, value: String(fields.enabled) },
{ key: KEY_SW_HTML, value: fields.html },
{ key: KEY_SW_DOMAINS, value: JSON.stringify(fields.domains) },
]);
},
onSuccess: async () => {
toast.success('离线页已保存,请前往版本发布使配置生效');
await invalidateResponseQueries(queryClient);
},
onError: (error) => {
toast.error(error instanceof Error ? error.message : '保存失败');
},
});
return (
<div className='space-y-6'>
<Card className='border-dashed shadow-none'>
<CardHeader className='flex flex-row items-start justify-between gap-4 space-y-0'>
<div className='space-y-1.5'>
<CardTitle className='text-base'>离线页</CardTitle>
<CardDescription>
启用后给启用 HTTPS 的网站下发 Service
Worker,域名无法访问时浏览器从缓存展示此离线页。
</CardDescription>
</div>
<Button
size='sm'
className='shrink-0'
disabled={saveMutation.isPending}
onClick={() => saveMutation.mutate()}
>
{saveMutation.isPending ? (
<Loader2 className='size-3.5 animate-spin' />
) : (
<Save className='size-3.5' />
)}
保存
</Button>
</CardHeader>
<CardContent className='space-y-4'>
<div className='flex items-start justify-between gap-6'>
<div className='space-y-1'>
<Label className='text-sm font-medium'>
启用 Service Worker 离线页
</Label>
<p className='text-sm text-muted-foreground'>
仅对 HTTPS 网站生效。
</p>
</div>
<Switch
checked={fields.enabled}
onCheckedChange={(enabled) =>
setFields((prev) => ({ ...prev, enabled }))
}
aria-label='启用离线页'
className='mt-0.5 shrink-0'
/>
</div>
<div className='space-y-2'>
<div className='flex items-center justify-between gap-3'>
<div>
<Label className='text-sm font-medium'>生效范围</Label>
<p className='text-sm text-muted-foreground'>
仅对选中的 HTTPS 域名生效;留空则不注入。
</p>
</div>
<Button
type='button'
size='sm'
variant='outline'
disabled={!fields.enabled || saveMutation.isPending}
onClick={() => setScopeOpen(true)}
>
<Plus className='size-3.5' />
添加域名
</Button>
</div>
{fields.domains.length === 0 ? (
<p className='text-sm text-muted-foreground'>
{fields.enabled
? '尚未选择生效域名,保存后不注入任何站点。'
: '启用离线兜底后可选择生效域名。'}
</p>
) : (
<div className='flex flex-wrap gap-2'>
{fields.domains.map((domain) => (
<Badge
key={domain}
variant='secondary'
className='gap-1 font-normal'
>
{domain}
<button
type='button'
className='hover:text-destructive'
disabled={!fields.enabled}
aria-label={`移除 ${domain}`}
onClick={() =>
setFields((prev) => ({
...prev,
domains: prev.domains.filter((d) => d !== domain),
}))
}
>
<X className='size-3' />
</button>
</Badge>
))}
</div>
)}
</div>
</CardContent>
</Card>
<Card className='border-dashed shadow-none overflow-hidden'>
<CardHeader className='flex flex-row items-start justify-between gap-3 space-y-0'>
<div className='space-y-1.5'>
<CardTitle className='text-base'>页面预览</CardTitle>
</div>
<div className='flex shrink-0 flex-wrap gap-2'>
<Button variant='outline' size='sm' asChild>
<Link href='/responses/offline/preview'>
<Expand className='size-3.5' />
预览
</Link>
</Button>
<Button size='sm' asChild>
<Link href='/responses/offline/edit'>
<Pencil className='size-3.5' />
编辑
</Link>
</Button>
</div>
</CardHeader>
<CardContent>
<div className='overflow-hidden rounded-md border bg-muted/30'>
<iframe
title='离线页预览'
sandbox=''
srcDoc={previewSrcDoc}
className='h-[32rem] w-full bg-background'
/>
</div>
</CardContent>
</Card>
<ScopeDomainDialog
open={scopeOpen}
onOpenChange={setScopeOpen}
zones={zonesQuery.data ?? []}
selected={fields.domains}
pending={saveMutation.isPending}
onSubmit={(domains) => {
setFields((prev) => ({ ...prev, domains }));
setScopeOpen(false);
}}
/>
</div>
);
}
@@ -0,0 +1,316 @@
'use client';
import { useEffect, useMemo, useRef, useState } from 'react';
import { Check, ChevronDown, Search } from 'lucide-react';
import { Badge } from '@/components/ui/badge';
import { Button } from '@/components/ui/button';
import { Checkbox } from '@/components/ui/checkbox';
import {
Collapsible,
CollapsibleContent,
CollapsibleTrigger,
} from '@/components/ui/collapsible';
import {
Dialog,
DialogContent,
DialogDescription,
DialogFooter,
DialogHeader,
DialogTitle,
} from '@/components/ui/dialog';
import { Field, FieldGroup, FieldLabel } from '@/components/ui/field';
import { Input } from '@/components/ui/input';
import { cn } from '@/lib/utils';
type ScopeZoneGroup = {
zoneDomain: string;
domains: string[];
};
export function ScopeDomainDialog({
open,
onOpenChange,
zones,
selected,
pending,
onSubmit,
}: {
open: boolean;
onOpenChange: (open: boolean) => void;
zones: ScopeZoneGroup[];
selected: string[];
pending: boolean;
onSubmit: (domains: string[]) => void;
}) {
const [keyword, setKeyword] = useState('');
const [selectedSet, setSelectedSet] = useState<Set<string>>(() => new Set());
const [collapsedZones, setCollapsedZones] = useState<Set<string>>(
() => new Set(),
);
const selectedRef = useRef(selected);
selectedRef.current = selected;
const prevOpenRef = useRef(open);
useEffect(() => {
if (open && !prevOpenRef.current) {
setKeyword('');
setSelectedSet(new Set(selectedRef.current));
setCollapsedZones(new Set());
}
prevOpenRef.current = open;
}, [open]);
const filtered = useMemo(() => {
const normalized = keyword.trim().toLowerCase();
if (!normalized) return zones;
return zones
.map((group) => ({
zoneDomain: group.zoneDomain,
domains: group.domains.filter(
(d) =>
d.toLowerCase().includes(normalized) ||
group.zoneDomain.toLowerCase().includes(normalized),
),
}))
.filter((group) => group.domains.length > 0);
}, [zones, keyword]);
const visibleDomains = useMemo(
() => filtered.flatMap((group) => group.domains),
[filtered],
);
const allVisibleSelected =
visibleDomains.length > 0 &&
visibleDomains.every((d) => selectedSet.has(d));
const toggleOne = (domain: string) => {
setSelectedSet((prev) => {
const next = new Set(prev);
if (next.has(domain)) next.delete(domain);
else next.add(domain);
return next;
});
};
const toggleGroup = (domains: string[]) => {
setSelectedSet((prev) => {
const next = new Set(prev);
const allSelected = domains.every((d) => next.has(d));
for (const d of domains) {
if (allSelected) next.delete(d);
else next.add(d);
}
return next;
});
};
const toggleAllVisible = () => {
setSelectedSet((prev) => {
const next = new Set(prev);
if (allVisibleSelected) {
for (const d of visibleDomains) next.delete(d);
} else {
for (const d of visibleDomains) next.add(d);
}
return next;
});
};
const toggleCollapsed = (zoneDomain: string) => {
setCollapsedZones((prev) => {
const next = new Set(prev);
if (next.has(zoneDomain)) next.delete(zoneDomain);
else next.add(zoneDomain);
return next;
});
};
return (
<Dialog open={open} onOpenChange={onOpenChange}>
<DialogContent className='sm:max-w-lg'>
<DialogHeader>
<DialogTitle>选择生效域名</DialogTitle>
<DialogDescription>
仅对选中的 HTTPS 域名注入离线兜底;搜索筛选与批量勾选。
</DialogDescription>
</DialogHeader>
<FieldGroup>
<Field>
<FieldLabel htmlFor='sw-scope-search'>域名</FieldLabel>
<div className='space-y-2'>
<div className='relative'>
<Search className='pointer-events-none absolute top-1/2 left-2.5 size-4 -translate-y-1/2 text-muted-foreground' />
<Input
id='sw-scope-search'
value={keyword}
onChange={(event) => setKeyword(event.target.value)}
placeholder='搜索域名或顶级域…'
className='pl-8'
disabled={pending}
/>
</div>
<div className='flex flex-wrap items-center justify-between gap-2'>
<div className='flex items-center gap-2 text-xs text-muted-foreground'>
<Badge variant='secondary' className='font-normal'>
已选 {selectedSet.size}
</Badge>
<span>可见 {visibleDomains.length}</span>
</div>
<div className='flex items-center gap-1'>
<Button
type='button'
variant='ghost'
size='sm'
className='h-7 text-xs'
disabled={pending || visibleDomains.length === 0}
onClick={toggleAllVisible}
>
{allVisibleSelected ? '取消全选' : '全选可见'}
</Button>
<Button
type='button'
variant='ghost'
size='sm'
className='h-7 text-xs'
disabled={pending || selectedSet.size === 0}
onClick={() => setSelectedSet(new Set())}
>
清空
</Button>
</div>
</div>
{zones.length === 0 ? (
<div className='rounded-lg border border-dashed px-4 py-8 text-center text-sm text-muted-foreground'>
暂无可用域名,请先在 Zone 管理中注册域名。
</div>
) : filtered.length === 0 ? (
<div className='rounded-lg border border-dashed px-4 py-8 text-center text-sm text-muted-foreground'>
没有匹配的域名
</div>
) : (
<div className='max-h-72 space-y-1 overflow-y-auto rounded-lg border p-2'>
{filtered.map((group) => {
const groupSelected = group.domains.filter((d) =>
selectedSet.has(d),
);
const allSelected =
groupSelected.length === group.domains.length;
const someSelected =
groupSelected.length > 0 && !allSelected;
const open = !collapsedZones.has(group.zoneDomain);
return (
<Collapsible
key={group.zoneDomain}
open={open}
onOpenChange={() => toggleCollapsed(group.zoneDomain)}
>
<div
className={cn(
'rounded-md',
(allSelected || someSelected) && 'bg-muted/30',
)}
>
<div className='flex items-center gap-1 px-1 py-0.5'>
<Checkbox
checked={
allSelected
? true
: someSelected
? 'indeterminate'
: false
}
disabled={pending}
onCheckedChange={() => toggleGroup(group.domains)}
aria-label={`选择顶级域 ${group.zoneDomain}`}
className='ml-1'
/>
<CollapsibleTrigger asChild>
<button
type='button'
className='flex min-w-0 flex-1 items-center gap-2 rounded-md px-1.5 py-1.5 text-left text-sm font-medium hover:bg-muted/60'
>
<ChevronDown
className={cn(
'size-4 shrink-0 text-muted-foreground transition-transform',
!open && '-rotate-90',
)}
/>
<span className='truncate'>
{group.zoneDomain}
</span>
<Badge
variant='outline'
className='ml-auto shrink-0 font-normal text-[10px]'
>
{groupSelected.length}/{group.domains.length}
</Badge>
</button>
</CollapsibleTrigger>
</div>
<CollapsibleContent>
<div className='ml-4 space-y-0.5 border-l border-border/70 py-0.5 pl-2'>
{group.domains.map((domain) => {
const checked = selectedSet.has(domain);
const isApex = domain === group.zoneDomain;
return (
<label
key={domain}
className={cn(
'flex cursor-pointer items-center gap-3 rounded-md px-2 py-1.5 transition-colors hover:bg-muted/60',
checked && 'bg-muted/40',
)}
>
<Checkbox
checked={checked}
disabled={pending}
onCheckedChange={() => toggleOne(domain)}
/>
<span className='min-w-0 flex-1 truncate text-sm'>
{domain}
</span>
{isApex ? (
<Badge
variant='secondary'
className='shrink-0 text-[10px] font-normal'
>
顶级域
</Badge>
) : null}
{checked ? (
<Check className='size-3.5 shrink-0 text-primary' />
) : null}
</label>
);
})}
</div>
</CollapsibleContent>
</div>
</Collapsible>
);
})}
</div>
)}
</div>
</Field>
</FieldGroup>
<DialogFooter>
<Button
variant='outline'
disabled={pending}
onClick={() => onOpenChange(false)}
>
取消
</Button>
<Button
disabled={pending}
onClick={() => onSubmit([...selectedSet].sort())}
>
确定
</Button>
</DialogFooter>
</DialogContent>
</Dialog>
);
}
@@ -1,8 +1,6 @@
import { ORIGIN_ERROR_PAGE_HTML_MAX_BYTES } from '@/lib/openflare/default-origin-error-page-html';
import {
DEFAULT_ORIGIN_ERROR_PAGE_STATUS_TAGS,
parseStatusCodeTagsJSON,
validateStatusCodeTags,
} from '@/lib/openflare/status-code-tags';
export const OPTIONS_QUERY_KEY = ['openflare', 'options'] as const;
@@ -12,6 +10,10 @@ export const KEY_STATUS_CODES = 'origin_error_page_status_codes';
export const KEY_HTML = 'origin_error_page_html';
export const KEY_GET_ONLY = 'origin_error_page_get_only';
export const KEY_SW_ENABLED = 'sw_offline_enabled';
export const KEY_SW_HTML = 'sw_offline_html';
export const KEY_SW_DOMAINS = 'sw_offline_domains';
export type ErrorPageFields = {
enabled: boolean;
getOnly: boolean;
@@ -26,6 +28,18 @@ export const defaultErrorPageFields: ErrorPageFields = {
html: '',
};
export type OfflinePageFields = {
enabled: boolean;
html: string;
domains: string[];
};
export const defaultOfflinePageFields: OfflinePageFields = {
enabled: false,
html: '',
domains: [],
};
export function optionsToMap(options: Array<{ key: string; value: string }>) {
return options.reduce<Record<string, string>>((acc, option) => {
acc[option.key] = option.value;
@@ -33,7 +47,19 @@ export function optionsToMap(options: Array<{ key: string; value: string }>) {
}, {});
}
export function mapOptionsToFields(
function parseDomains(raw: string | undefined): string[] {
if (!raw) return [];
try {
const parsed: unknown = JSON.parse(raw);
return Array.isArray(parsed)
? parsed.filter((item): item is string => typeof item === 'string')
: [];
} catch {
return [];
}
}
export function mapOptionsToErrorFields(
optionMap: Record<string, string>,
): ErrorPageFields {
const enabledRaw = optionMap[KEY_ENABLED];
@@ -46,21 +72,17 @@ export function mapOptionsToFields(
};
}
export function validateErrorPageFields(fields: ErrorPageFields) {
validateStatusCodeTags(fields.statusCodes);
validateErrorPageHTML(fields.html);
export function mapOptionsToOfflineFields(
optionMap: Record<string, string>,
): OfflinePageFields {
return {
enabled: optionMap[KEY_SW_ENABLED] === 'true',
html: optionMap[KEY_SW_HTML] ?? '',
domains: parseDomains(optionMap[KEY_SW_DOMAINS]),
};
}
export function validateErrorPageHTML(html: string) {
const htmlBytes = new TextEncoder().encode(html).length;
if (htmlBytes > ORIGIN_ERROR_PAGE_HTML_MAX_BYTES) {
throw new Error(
`HTML 超过最大长度限制(${ORIGIN_ERROR_PAGE_HTML_MAX_BYTES} 字节)`,
);
}
}
export async function invalidateErrorPageQueries(queryClient: {
export async function invalidateResponseQueries(queryClient: {
invalidateQueries: (opts: {
queryKey: readonly unknown[];
}) => Promise<unknown>;
@@ -0,0 +1,262 @@
'use client';
import Link from 'next/link';
import { useEffect, useState } from 'react';
import { useMutation, useQuery, useQueryClient } from '@tanstack/react-query';
import {
ArrowLeft,
FileCode2,
Loader2,
RotateCcw,
Save,
Sparkles,
} from 'lucide-react';
import { toast } from 'sonner';
import { useAuth } from '@/components/providers/auth-provider';
import {
AlertDialog,
AlertDialogAction,
AlertDialogCancel,
AlertDialogContent,
AlertDialogDescription,
AlertDialogFooter,
AlertDialogHeader,
AlertDialogTitle,
} from '@/components/ui/alert-dialog';
import { Button } from '@/components/ui/button';
import { EmptyStateWithBorder } from '@/components/layout/empty';
import { ErrorInline } from '@/components/layout/error';
import { LoadingStateWithBorder } from '@/components/layout/loading';
import {
Select,
SelectContent,
SelectItem,
SelectTrigger,
SelectValue,
} from '@/components/ui/select';
import { HtmlEditorWorkspace } from '@/components/common/html-editor-workspace';
import {
ORIGIN_ERROR_PAGE_HTML_MAX_BYTES,
previewOriginErrorPageHTML,
} from '@/lib/openflare/default-origin-error-page-html';
import {
DEFAULT_ORIGIN_ERROR_PAGE_TEMPLATE_ID,
getOriginErrorPageTemplate,
ORIGIN_ERROR_PAGE_TEMPLATES,
} from '@/lib/openflare/origin-error-page-templates';
import { OptionService } from '@/lib/services/openflare';
import {
invalidateResponseQueries,
KEY_HTML,
mapOptionsToErrorFields,
OPTIONS_QUERY_KEY,
optionsToMap,
} from '../../components/shared';
export default function ErrorPageEditPage() {
const { user, loading: authLoading } = useAuth();
const queryClient = useQueryClient();
const [html, setHtml] = useState('');
const [restoreOpen, setRestoreOpen] = useState(false);
const [templateId, setTemplateId] = useState(
DEFAULT_ORIGIN_ERROR_PAGE_TEMPLATE_ID,
);
const optionsQuery = useQuery({
queryKey: OPTIONS_QUERY_KEY,
queryFn: () => OptionService.list(),
enabled: !!user?.is_admin,
});
useEffect(() => {
if (!optionsQuery.data) return;
setHtml(mapOptionsToErrorFields(optionsToMap(optionsQuery.data)).html);
}, [optionsQuery.data]);
const saveMutation = useMutation({
mutationFn: async () => {
const bytes = new TextEncoder().encode(html).length;
if (bytes > ORIGIN_ERROR_PAGE_HTML_MAX_BYTES) {
throw new Error('HTML 大小超出限制(最大 256KB)');
}
await OptionService.updateBatch([{ key: KEY_HTML, value: html }]);
},
onSuccess: async () => {
toast.success('源站错误页 HTML 已保存,请前往版本发布使配置生效');
await invalidateResponseQueries(queryClient);
},
onError: (error) => {
toast.error(error instanceof Error ? error.message : '保存失败');
},
});
const loadSelectedTemplate = () => {
const tmpl = getOriginErrorPageTemplate(templateId);
if (!tmpl) {
toast.error('未找到所选模板');
return;
}
setHtml(tmpl.html);
toast.success(`已加载模板「${tmpl.name}」到编辑器`);
};
const restoreDefault = () => {
setHtml('');
setRestoreOpen(false);
toast.success('已清空为使用服务端内置默认模板(需保存后生效)');
};
if (authLoading) {
return (
<div className='w-full py-6 px-1'>
<LoadingStateWithBorder
icon={FileCode2}
description='加载权限信息...'
/>
</div>
);
}
if (!user?.is_admin) {
return (
<div className='w-full py-6 px-1'>
<EmptyStateWithBorder
icon={FileCode2}
title='权限不足'
description='只有管理员可以编辑错误页 HTML。'
/>
</div>
);
}
if (optionsQuery.isLoading) {
return (
<div className='w-full py-6 px-1'>
<LoadingStateWithBorder
icon={FileCode2}
description='加载错误页配置...'
/>
</div>
);
}
if (optionsQuery.isError) {
return (
<div className='w-full py-6 px-1'>
<ErrorInline
message={
optionsQuery.error instanceof Error
? optionsQuery.error.message
: '加载失败'
}
onRetry={() => void optionsQuery.refetch()}
/>
</div>
);
}
return (
<div className='flex flex-col gap-4 py-6 px-1'>
<div className='flex flex-col gap-3 sm:flex-row sm:items-center sm:justify-between'>
<div className='flex items-center gap-3'>
<Button variant='outline' size='icon' className='h-8 w-8' asChild>
<Link href='/responses?tab=error' aria-label='返回错误页'>
<ArrowLeft className='size-4' />
</Link>
</Button>
<div className='flex items-center gap-2'>
<FileCode2 className='size-5 text-primary' />
<div>
<h1 className='text-2xl font-semibold tracking-tight'>
编辑错误页 HTML
</h1>
</div>
</div>
</div>
</div>
<HtmlEditorWorkspace
value={html}
onChange={setHtml}
preview={previewOriginErrorPageHTML}
toolbarRight={
<>
<div className='flex items-center gap-1.5'>
<span className='text-[11px] text-muted-foreground hidden sm:inline'>
内置模板
</span>
<Select value={templateId} onValueChange={setTemplateId}>
<SelectTrigger className='h-7 w-[160px] text-[11px] bg-background'>
<SelectValue placeholder='选择模板' />
</SelectTrigger>
<SelectContent>
{ORIGIN_ERROR_PAGE_TEMPLATES.map((tmpl) => (
<SelectItem
key={tmpl.id}
value={tmpl.id}
className='text-xs'
>
{tmpl.name}
</SelectItem>
))}
</SelectContent>
</Select>
<Button
type='button'
variant='ghost'
size='sm'
className='h-7 px-2 text-[11px]'
onClick={loadSelectedTemplate}
>
<Sparkles className='size-3.5' />
加载模板
</Button>
</div>
<Button
type='button'
variant='ghost'
size='sm'
className='h-7 px-2 text-[11px]'
onClick={() => setRestoreOpen(true)}
>
<RotateCcw className='size-3.5' />
恢复默认
</Button>
<Button
size='sm'
className='h-7 px-3 text-[11px]'
disabled={saveMutation.isPending}
onClick={() => saveMutation.mutate()}
>
{saveMutation.isPending ? (
<Loader2 className='size-3 animate-spin' />
) : (
<Save className='size-3' />
)}
保存
</Button>
</>
}
/>
<AlertDialog open={restoreOpen} onOpenChange={setRestoreOpen}>
<AlertDialogContent>
<AlertDialogHeader>
<AlertDialogTitle>恢复默认 HTML?</AlertDialogTitle>
<AlertDialogDescription>
将清空编辑器内容,保存后使用服务端内置默认模板。此操作不会自动保存。
</AlertDialogDescription>
</AlertDialogHeader>
<AlertDialogFooter>
<AlertDialogCancel>取消</AlertDialogCancel>
<AlertDialogAction onClick={restoreDefault}>
确认恢复
</AlertDialogAction>
</AlertDialogFooter>
</AlertDialogContent>
</AlertDialog>
</div>
);
}
@@ -0,0 +1,118 @@
'use client';
import Link from 'next/link';
import { useMemo } from 'react';
import { useQuery } from '@tanstack/react-query';
import { ArrowLeft, Pencil, X } from 'lucide-react';
import { useAuth } from '@/components/providers/auth-provider';
import { Button } from '@/components/ui/button';
import { EmptyStateWithBorder } from '@/components/layout/empty';
import { ErrorInline } from '@/components/layout/error';
import { LoadingStateWithBorder } from '@/components/layout/loading';
import { previewOriginErrorPageHTML } from '@/lib/openflare/default-origin-error-page-html';
import { OptionService } from '@/lib/services/openflare';
import {
mapOptionsToErrorFields,
OPTIONS_QUERY_KEY,
optionsToMap,
} from '../../components/shared';
export default function ErrorPagePreviewPage() {
const { user, loading: authLoading } = useAuth();
const optionsQuery = useQuery({
queryKey: OPTIONS_QUERY_KEY,
queryFn: () => OptionService.list(),
enabled: !!user?.is_admin,
});
const html = useMemo(() => {
if (!optionsQuery.data) return '';
return mapOptionsToErrorFields(optionsToMap(optionsQuery.data)).html;
}, [optionsQuery.data]);
const previewSrcDoc = useMemo(() => previewOriginErrorPageHTML(html), [html]);
if (authLoading) {
return (
<div className='w-full py-6 px-1'>
<LoadingStateWithBorder description='加载权限信息...' />
</div>
);
}
if (!user?.is_admin) {
return (
<div className='w-full py-6 px-1'>
<EmptyStateWithBorder
title='权限不足'
description='只有管理员可以预览错误页。'
/>
</div>
);
}
if (optionsQuery.isLoading) {
return (
<div className='w-full py-6 px-1'>
<LoadingStateWithBorder description='加载错误页配置...' />
</div>
);
}
if (optionsQuery.isError) {
return (
<div className='w-full py-6 px-1'>
<ErrorInline
message={
optionsQuery.error instanceof Error
? optionsQuery.error.message
: '加载失败'
}
onRetry={() => void optionsQuery.refetch()}
/>
</div>
);
}
return (
<div className='fixed inset-0 z-50 flex flex-col bg-background'>
<div className='flex items-center justify-between gap-3 border-b bg-background/95 px-4 py-2 backdrop-blur shrink-0'>
<div className='flex items-center gap-2 min-w-0'>
<Button variant='outline' size='icon' className='h-8 w-8' asChild>
<Link href='/responses?tab=error' aria-label='返回错误页'>
<ArrowLeft className='size-4' />
</Link>
</Button>
<div className='min-w-0'>
<p className='text-sm font-semibold truncate'>错误页预览</p>
<p className='text-[11px] text-muted-foreground font-mono truncate'>
{'{{status}}'}→502 · {'{{host}}'}→example.com · 全屏展示
</p>
</div>
</div>
<div className='flex items-center gap-2 shrink-0'>
<Button variant='outline' size='sm' asChild>
<Link href='/responses/error-page/edit'>
<Pencil className='size-3.5' />
编辑
</Link>
</Button>
<Button variant='ghost' size='icon' className='h-8 w-8' asChild>
<Link href='/responses?tab=error' aria-label='关闭预览'>
<X className='size-4' />
</Link>
</Button>
</div>
</div>
<iframe
title='源站错误页预览'
sandbox=''
srcDoc={previewSrcDoc}
className='flex-1 w-full border-0 bg-background min-h-0'
/>
</div>
);
}
@@ -0,0 +1,264 @@
'use client';
import Link from 'next/link';
import { useEffect, useState } from 'react';
import { useMutation, useQuery, useQueryClient } from '@tanstack/react-query';
import {
ArrowLeft,
FileCode2,
Loader2,
RotateCcw,
Save,
Sparkles,
} from 'lucide-react';
import { toast } from 'sonner';
import { useAuth } from '@/components/providers/auth-provider';
import {
AlertDialog,
AlertDialogAction,
AlertDialogCancel,
AlertDialogContent,
AlertDialogDescription,
AlertDialogFooter,
AlertDialogHeader,
AlertDialogTitle,
} from '@/components/ui/alert-dialog';
import { Button } from '@/components/ui/button';
import { EmptyStateWithBorder } from '@/components/layout/empty';
import { ErrorInline } from '@/components/layout/error';
import { LoadingStateWithBorder } from '@/components/layout/loading';
import {
Select,
SelectContent,
SelectItem,
SelectTrigger,
SelectValue,
} from '@/components/ui/select';
import { HtmlEditorWorkspace } from '@/components/common/html-editor-workspace';
import {
OFFLINE_PAGE_TEMPLATES,
DEFAULT_OFFLINE_PAGE_TEMPLATE_ID,
effectiveOfflinePageHTML,
getOfflinePageTemplate,
} from '@/lib/openflare/offline-page-templates';
import { OptionService } from '@/lib/services/openflare';
import {
invalidateResponseQueries,
KEY_SW_HTML,
mapOptionsToOfflineFields,
OPTIONS_QUERY_KEY,
optionsToMap,
} from '../../components/shared';
const OFFLINE_PAGE_HTML_MAX_BYTES = 256 * 1024;
export default function OfflinePageEditPage() {
const { user, loading: authLoading } = useAuth();
const queryClient = useQueryClient();
const [html, setHtml] = useState('');
const [restoreOpen, setRestoreOpen] = useState(false);
const [templateId, setTemplateId] = useState(
DEFAULT_OFFLINE_PAGE_TEMPLATE_ID,
);
const optionsQuery = useQuery({
queryKey: OPTIONS_QUERY_KEY,
queryFn: () => OptionService.list(),
enabled: !!user?.is_admin,
});
useEffect(() => {
if (!optionsQuery.data) return;
setHtml(mapOptionsToOfflineFields(optionsToMap(optionsQuery.data)).html);
}, [optionsQuery.data]);
const saveMutation = useMutation({
mutationFn: async () => {
const bytes = new TextEncoder().encode(html).length;
if (bytes > OFFLINE_PAGE_HTML_MAX_BYTES) {
throw new Error('HTML 大小超出限制(最大 256KB)');
}
await OptionService.updateBatch([{ key: KEY_SW_HTML, value: html }]);
},
onSuccess: async () => {
toast.success('离线页 HTML 已保存,请前往版本发布使配置生效');
await invalidateResponseQueries(queryClient);
},
onError: (error) => {
toast.error(error instanceof Error ? error.message : '保存失败');
},
});
const loadSelectedTemplate = () => {
const tmpl = getOfflinePageTemplate(templateId);
if (!tmpl) {
toast.error('未找到所选模板');
return;
}
setHtml(tmpl.html);
toast.success(`已加载模板「${tmpl.name}」到编辑器`);
};
const restoreDefault = () => {
setHtml('');
setRestoreOpen(false);
toast.success('已清空为使用内置默认模板(需保存后生效)');
};
if (authLoading) {
return (
<div className='w-full py-6 px-1'>
<LoadingStateWithBorder
icon={FileCode2}
description='加载权限信息...'
/>
</div>
);
}
if (!user?.is_admin) {
return (
<div className='w-full py-6 px-1'>
<EmptyStateWithBorder
icon={FileCode2}
title='权限不足'
description='只有管理员可以编辑离线页 HTML。'
/>
</div>
);
}
if (optionsQuery.isLoading) {
return (
<div className='w-full py-6 px-1'>
<LoadingStateWithBorder
icon={FileCode2}
description='加载离线页配置...'
/>
</div>
);
}
if (optionsQuery.isError) {
return (
<div className='w-full py-6 px-1'>
<ErrorInline
message={
optionsQuery.error instanceof Error
? optionsQuery.error.message
: '加载失败'
}
onRetry={() => void optionsQuery.refetch()}
/>
</div>
);
}
return (
<div className='flex flex-col gap-4 py-6 px-1'>
<div className='flex flex-col gap-3 sm:flex-row sm:items-center sm:justify-between'>
<div className='flex items-center gap-3'>
<Button variant='outline' size='icon' className='h-8 w-8' asChild>
<Link href='/responses?tab=offline' aria-label='返回离线页'>
<ArrowLeft className='size-4' />
</Link>
</Button>
<div className='flex items-center gap-2'>
<FileCode2 className='size-5 text-primary' />
<div>
<h1 className='text-2xl font-semibold tracking-tight'>
编辑离线页 HTML
</h1>
</div>
</div>
</div>
</div>
<HtmlEditorWorkspace
value={html}
onChange={setHtml}
preview={effectiveOfflinePageHTML}
footerHint={null}
showPreviewLink={false}
previewTitle='离线页实时预览'
toolbarRight={
<>
<div className='flex items-center gap-1.5'>
<span className='text-[11px] text-muted-foreground hidden sm:inline'>
内置模板
</span>
<Select value={templateId} onValueChange={setTemplateId}>
<SelectTrigger className='h-7 w-[160px] text-[11px] bg-background'>
<SelectValue placeholder='选择模板' />
</SelectTrigger>
<SelectContent>
{OFFLINE_PAGE_TEMPLATES.map((tmpl) => (
<SelectItem
key={tmpl.id}
value={tmpl.id}
className='text-xs'
>
{tmpl.name}
</SelectItem>
))}
</SelectContent>
</Select>
<Button
type='button'
variant='ghost'
size='sm'
className='h-7 px-2 text-[11px]'
onClick={loadSelectedTemplate}
>
<Sparkles className='size-3.5' />
加载模板
</Button>
</div>
<Button
type='button'
variant='ghost'
size='sm'
className='h-7 px-2 text-[11px]'
onClick={() => setRestoreOpen(true)}
>
<RotateCcw className='size-3.5' />
恢复默认
</Button>
<Button
size='sm'
className='h-7 px-3 text-[11px]'
disabled={saveMutation.isPending}
onClick={() => saveMutation.mutate()}
>
{saveMutation.isPending ? (
<Loader2 className='size-3 animate-spin' />
) : (
<Save className='size-3' />
)}
保存
</Button>
</>
}
/>
<AlertDialog open={restoreOpen} onOpenChange={setRestoreOpen}>
<AlertDialogContent>
<AlertDialogHeader>
<AlertDialogTitle>恢复默认 HTML?</AlertDialogTitle>
<AlertDialogDescription>
将清空编辑器内容,保存后使用内置默认模板。此操作不会自动保存。
</AlertDialogDescription>
</AlertDialogHeader>
<AlertDialogFooter>
<AlertDialogCancel>取消</AlertDialogCancel>
<AlertDialogAction onClick={restoreDefault}>
确认恢复
</AlertDialogAction>
</AlertDialogFooter>
</AlertDialogContent>
</AlertDialog>
</div>
);
}
@@ -0,0 +1,119 @@
'use client';
import Link from 'next/link';
import { useMemo } from 'react';
import { useQuery } from '@tanstack/react-query';
import { ArrowLeft, Pencil, X } from 'lucide-react';
import { useAuth } from '@/components/providers/auth-provider';
import { Button } from '@/components/ui/button';
import { EmptyStateWithBorder } from '@/components/layout/empty';
import { ErrorInline } from '@/components/layout/error';
import { LoadingStateWithBorder } from '@/components/layout/loading';
import { OptionService } from '@/lib/services/openflare';
import { effectiveOfflinePageHTML } from '@/lib/openflare/offline-page-templates';
import {
mapOptionsToOfflineFields,
OPTIONS_QUERY_KEY,
optionsToMap,
} from '../../components/shared';
export default function OfflinePagePreviewPage() {
const { user, loading: authLoading } = useAuth();
const optionsQuery = useQuery({
queryKey: OPTIONS_QUERY_KEY,
queryFn: () => OptionService.list(),
enabled: !!user?.is_admin,
});
const html = useMemo(() => {
if (!optionsQuery.data) return '';
const rawHtml = mapOptionsToOfflineFields(
optionsToMap(optionsQuery.data),
).html;
return effectiveOfflinePageHTML(rawHtml);
}, [optionsQuery.data]);
if (authLoading) {
return (
<div className='w-full py-6 px-1'>
<LoadingStateWithBorder description='加载权限信息...' />
</div>
);
}
if (!user?.is_admin) {
return (
<div className='w-full py-6 px-1'>
<EmptyStateWithBorder
title='权限不足'
description='只有管理员可以预览离线页。'
/>
</div>
);
}
if (optionsQuery.isLoading) {
return (
<div className='w-full py-6 px-1'>
<LoadingStateWithBorder description='加载离线页配置...' />
</div>
);
}
if (optionsQuery.isError) {
return (
<div className='w-full py-6 px-1'>
<ErrorInline
message={
optionsQuery.error instanceof Error
? optionsQuery.error.message
: '加载失败'
}
onRetry={() => void optionsQuery.refetch()}
/>
</div>
);
}
return (
<div className='fixed inset-0 z-50 flex flex-col bg-background'>
<div className='flex items-center justify-between gap-3 border-b bg-background/95 px-4 py-2 backdrop-blur shrink-0'>
<div className='flex items-center gap-2 min-w-0'>
<Button variant='outline' size='icon' className='h-8 w-8' asChild>
<Link href='/responses?tab=offline' aria-label='返回离线页'>
<ArrowLeft className='size-4' />
</Link>
</Button>
<div className='min-w-0'>
<p className='text-sm font-semibold truncate'>离线页预览</p>
<p className='text-[11px] text-muted-foreground font-mono truncate'>
全屏展示
</p>
</div>
</div>
<div className='flex items-center gap-2 shrink-0'>
<Button variant='outline' size='sm' asChild>
<Link href='/responses/offline/edit'>
<Pencil className='size-3.5' />
编辑
</Link>
</Button>
<Button variant='ghost' size='icon' className='h-8 w-8' asChild>
<Link href='/responses?tab=offline' aria-label='关闭预览'>
<X className='size-4' />
</Link>
</Button>
</div>
</div>
<iframe
title='离线页预览'
sandbox=''
srcDoc={html}
className='flex-1 w-full border-0 bg-background min-h-0'
/>
</div>
);
}
+170
View File
@@ -0,0 +1,170 @@
'use client';
import { Suspense, useMemo } from 'react';
import { useRouter, useSearchParams } from 'next/navigation';
import { useQuery } from '@tanstack/react-query';
import { MessageSquareText } from 'lucide-react';
import { EmptyStateWithBorder } from '@/components/layout/empty';
import { ErrorInline } from '@/components/layout/error';
import { LoadingStateWithBorder } from '@/components/layout/loading';
import { useAuth } from '@/components/providers/auth-provider';
import { Tabs, TabsContent, TabsList, TabsTrigger } from '@/components/ui/tabs';
import { OptionService } from '@/lib/services/openflare';
import { ErrorPageTab } from './components/error-page-tab';
import { OfflinePageTab } from './components/offline-page-tab';
import { OPTIONS_QUERY_KEY, optionsToMap } from './components/shared';
type ResponseTab = 'error' | 'offline';
function resolveTab(tabParam: string | null): ResponseTab {
if (tabParam === 'offline') {
return 'offline';
}
return 'error';
}
function ResponsesPageContent() {
const { user, loading: authLoading } = useAuth();
const router = useRouter();
const searchParams = useSearchParams();
const activeTab = resolveTab(searchParams.get('tab'));
const handleTabChange = (value: string) => {
const nextTab = resolveTab(value);
const params = new URLSearchParams(searchParams.toString());
if (nextTab === 'error') {
params.delete('tab');
} else {
params.set('tab', nextTab);
}
const query = params.toString();
router.replace(query ? `/responses?${query}` : '/responses', {
scroll: false,
});
};
const optionsQuery = useQuery({
queryKey: OPTIONS_QUERY_KEY,
queryFn: () => OptionService.list(),
enabled: !!user?.is_admin,
});
const optionMap = useMemo(
() => optionsToMap(optionsQuery.data ?? []),
[optionsQuery.data],
);
if (authLoading) {
return (
<div className='w-full py-6 px-1'>
<LoadingStateWithBorder
icon={MessageSquareText}
description='加载权限信息...'
/>
</div>
);
}
if (!user?.is_admin) {
return (
<div className='w-full py-6 px-1'>
<EmptyStateWithBorder
icon={MessageSquareText}
title='权限不足'
description='只有管理员可以访问响应页面设置。'
/>
</div>
);
}
if (optionsQuery.isLoading) {
return (
<div className='w-full py-6 px-1'>
<LoadingStateWithBorder
icon={MessageSquareText}
description='加载响应页面配置...'
/>
</div>
);
}
if (optionsQuery.isError) {
return (
<div className='w-full py-6 px-1'>
<ErrorInline
message={
optionsQuery.error instanceof Error
? optionsQuery.error.message
: '加载失败'
}
onRetry={() => void optionsQuery.refetch()}
/>
</div>
);
}
if (!optionsQuery.data) return null;
return (
<div className='flex w-full flex-col gap-6 py-6 px-1'>
<div className='flex items-center gap-2'>
<MessageSquareText className='size-5 text-primary' />
<div>
<h1 className='text-2xl font-semibold tracking-tight'>响应页面</h1>
<p className='text-sm text-muted-foreground'>
配置源站错误页与离线页。
</p>
</div>
</div>
<Tabs
value={activeTab}
onValueChange={handleTabChange}
className='w-full'
>
<TabsList variant='line' className='mb-6 inline-flex w-fit gap-8'>
<TabsTrigger
value='error'
className='px-0 pb-2 text-xs font-semibold'
>
错误页
</TabsTrigger>
<TabsTrigger
value='offline'
className='px-0 pb-2 text-xs font-semibold'
>
离线页
</TabsTrigger>
</TabsList>
<TabsContent value='error' className='focus-visible:outline-none'>
<ErrorPageTab optionMap={optionMap} />
</TabsContent>
<TabsContent value='offline' className='focus-visible:outline-none'>
<OfflinePageTab optionMap={optionMap} />
</TabsContent>
</Tabs>
</div>
);
}
export default function ResponsesPage() {
return (
<Suspense
fallback={
<div className='w-full py-6 px-1'>
<LoadingStateWithBorder
icon={MessageSquareText}
description='加载响应页面...'
/>
</div>
}
>
<ResponsesPageContent />
</Suspense>
);
}
@@ -17,22 +17,35 @@ type HtmlEditorWorkspaceProps = {
value: string;
onChange: (value: string) => void;
toolbarRight?: React.ReactNode;
maxBytes?: number;
preview?: (html: string) => string;
footerHint?: React.ReactNode;
showPreviewLink?: boolean;
previewTitle?: string;
};
/**
* SQL 查询终端同款工作区:整体边框 + 工具栏 + 可拖拽分隔;
* 本页为左代码 / 右实时预览横向布局。
* 通用 HTML 编辑器工作区:左代码 / 右实时预览横向布局,可拖拽分隔。
*/
export function HtmlEditorWorkspace({
value,
onChange,
toolbarRight,
maxBytes = ORIGIN_ERROR_PAGE_HTML_MAX_BYTES,
preview = previewOriginErrorPageHTML,
footerHint = (
<>
{'{{status}}'}→502 · {'{{host}}'}→example.com
</>
),
showPreviewLink = true,
previewTitle = '源站错误页实时预览',
}: HtmlEditorWorkspaceProps) {
const containerRef = useRef<HTMLDivElement>(null);
const [editorWidthPercent, setEditorWidthPercent] = useState(48);
const { resolvedTheme } = useTheme();
const cmTheme = resolvedTheme === 'dark' ? 'dark' : 'light';
const previewSrcDoc = previewOriginErrorPageHTML(value);
const previewSrcDoc = preview(value);
const htmlBytes = new TextEncoder().encode(value).length;
const handleMouseDown = (e: React.MouseEvent) => {
@@ -68,7 +81,7 @@ export function HtmlEditorWorkspace({
<Terminal className='size-4 text-primary shrink-0' />
<span className='text-xs font-semibold'>HTML 编辑器</span>
<span className='text-[11px] text-muted-foreground font-mono truncate'>
{htmlBytes} / {ORIGIN_ERROR_PAGE_HTML_MAX_BYTES} 字节
{htmlBytes} / {maxBytes} 字节
{value.trim() === '' ? ' · 空则使用内置默认' : ''}
</span>
</div>
@@ -112,25 +125,29 @@ export function HtmlEditorWorkspace({
<div className='flex items-center justify-between px-4 py-1.5 border-b bg-muted/20 shrink-0 text-[11px] text-muted-foreground font-mono gap-2'>
<span className='font-semibold'>实时预览</span>
<div className='flex items-center gap-2'>
<span className='hidden sm:inline'>
{'{{status}}'}→502 · {'{{host}}'}→example.com
</span>
<Button
variant='ghost'
size='sm'
className='h-6 px-2 text-[11px]'
asChild
<span
className={footerHint === null ? 'hidden' : 'hidden sm:inline'}
>
<Link href='/error-pages/preview'>
<Expand className='size-3' />
真实预览
</Link>
</Button>
{footerHint ?? null}
</span>
{showPreviewLink ? (
<Button
variant='ghost'
size='sm'
className='h-6 px-2 text-[11px]'
asChild
>
<Link href='/error-pages/preview'>
<Expand className='size-3' />
预览
</Link>
</Button>
) : null}
</div>
</div>
<div className='flex-1 min-h-0 overflow-hidden bg-background'>
<iframe
title='源站错误页实时预览'
title={previewTitle}
sandbox=''
srcDoc={previewSrcDoc}
className='h-full w-full border-0 bg-background'
@@ -1,13 +1,13 @@
'use client';
import { useMemo } from 'react';
import { useEffect, useMemo, useState } from 'react';
import {
useMutation,
useQuery,
useQueryClient,
type UseQueryResult,
} from '@tanstack/react-query';
import { KeyRound, ShieldAlert, X } from 'lucide-react';
import { Database, KeyRound, Save, ShieldAlert, X } from 'lucide-react';
import {
Card,
CardContent,
@@ -16,6 +16,10 @@ import {
CardTitle,
} from '@/components/ui/card';
import { Badge } from '@/components/ui/badge';
import { Button } from '@/components/ui/button';
import { Input } from '@/components/ui/input';
import { Spinner } from '@/components/ui/spinner';
import { Label } from '@/components/ui/label';
import {
Select,
SelectContent,
@@ -28,6 +32,24 @@ import type { SystemConfig } from '@/lib/services/admin';
import { TemplatesManager } from './templates';
import { toast } from 'sonner';
const LOG_RETENTION_FIELDS = [
{
key: 'log_retention_days_postgres',
label: 'PostgreSQL',
description: '访问日志与可观测指标统一保留天数',
},
{
key: 'log_retention_days_sqlite',
label: 'SQLite',
description: 'SQLite 日志保留天数',
},
{
key: 'log_retention_days_clickhouse',
label: 'ClickHouse',
description: 'ClickHouse 日志保留天数',
},
] as const;
interface OperationTabProps {
configs: Record<string, SystemConfig>;
systemConfigsQuery: UseQueryResult<SystemConfig[], Error>;
@@ -44,6 +66,65 @@ export function OperationTab({
queryFn: () => services.adminSystemConfig.listUploadTypes(),
});
const businessConfigsQuery = useQuery({
queryKey: ['admin', 'system-configs', 'business'],
queryFn: () => services.adminSystemConfig.listSystemConfigs('business'),
});
const businessConfigs = useMemo(() => {
return (businessConfigsQuery.data ?? []).reduce<
Record<string, SystemConfig>
>((acc, config) => {
acc[config.key] = config;
return acc;
}, {});
}, [businessConfigsQuery.data]);
const [retentionValues, setRetentionValues] = useState<
Record<string, string>
>({});
useEffect(() => {
if (!businessConfigsQuery.data) return;
setRetentionValues((prev) => {
const next: Record<string, string> = {};
LOG_RETENTION_FIELDS.forEach((field) => {
const config = businessConfigs[field.key];
next[field.key] = config?.value || prev[field.key] || '90';
});
return next;
});
}, [businessConfigsQuery.data, businessConfigs]);
const updateRetentionMutation = useMutation({
mutationFn: async (values: Record<string, string>) => {
for (const field of LOG_RETENTION_FIELDS) {
const raw = (values[field.key] ?? '').trim();
const num = Number(raw);
if (!raw || !Number.isInteger(num) || num < 1) {
throw new Error(`${field.label}必须为大于等于 1 的整数`);
}
const config = businessConfigs[field.key];
if (!config) {
throw new Error(`缺少配置项: ${field.key}`);
}
await services.adminSystemConfig.updateSystemConfig(field.key, {
value: String(num),
description: config.description,
});
}
},
onSuccess: async () => {
await queryClient.invalidateQueries({
queryKey: ['admin', 'system-configs'],
});
toast.success('日志保留时间已更新');
},
onError: (error: Error) => {
toast.error(error.message || '更新日志保留时间失败');
},
});
const updateWhitelistMutation = useMutation({
mutationFn: async (newValue: string) => {
const config = configs['file_access_whitelist'];
@@ -208,6 +289,77 @@ export function OperationTab({
</CardContent>
</Card>
{/* 日志保留时间设置 */}
<Card className='border border-dashed shadow-sm'>
<CardHeader className='border-b border-dashed pb-4'>
<div className='flex items-center gap-2'>
<div className='p-1.5 rounded-lg bg-primary/10 text-primary'>
<Database className='size-4' />
</div>
<div>
<CardTitle className='text-base font-semibold'>
日志保留时间
</CardTitle>
<CardDescription className='text-xs'>
配置各日志数据库的日志保留天数,切换日志数据库后自动按对应配置清理过期日志。
</CardDescription>
</div>
</div>
</CardHeader>
<CardContent className='pt-6'>
<div className='grid gap-4 sm:grid-cols-3'>
{LOG_RETENTION_FIELDS.map((field) => (
<div key={field.key} className='grid gap-2'>
<Label htmlFor={field.key}>{field.label}</Label>
<div className='flex items-center gap-2'>
<Input
id={field.key}
type='number'
min={1}
className='text-xs'
value={retentionValues[field.key] ?? ''}
disabled={
updateRetentionMutation.isPending ||
businessConfigsQuery.isPending
}
onChange={(e) =>
setRetentionValues((prev) => ({
...prev,
[field.key]: e.target.value,
}))
}
/>
<span className='text-xs text-muted-foreground whitespace-nowrap'>
天
</span>
</div>
<p className='text-[10px] text-muted-foreground'>
{field.description}
</p>
</div>
))}
</div>
<div className='mt-4 flex justify-end'>
<Button
type='button'
size='sm'
onClick={() => updateRetentionMutation.mutate(retentionValues)}
disabled={
updateRetentionMutation.isPending ||
businessConfigsQuery.isPending
}
>
{updateRetentionMutation.isPending ? (
<Spinner className='size-3' />
) : (
<Save className='size-3' />
)}
保存
</Button>
</div>
</CardContent>
</Card>
{/* 通知模板管理 */}
<TemplatesManager />
</div>
+1 -2
View File
@@ -55,7 +55,6 @@ import {
LogOut,
Settings,
ShieldCheck,
Terminal,
UserRound,
} from 'lucide-react';
@@ -72,7 +71,7 @@ const data = {
{ title: '存储管理', url: '/admin/files', icon: FolderOpen },
{ title: '数据管理', url: '/admin/database', icon: Database },
{ title: '通知推送', url: '/admin/push', icon: Bell },
{ title: '系统日志', url: '/admin/logs', icon: Terminal },
// { title: '系统日志', url: '/admin/logs', icon: Terminal },
{ title: '系统配置', url: '/admin/system', icon: ShieldCheck },
{ title: '系统设置', url: '/admin/settings', icon: Settings },
],
+9 -4
View File
@@ -61,9 +61,14 @@ export const openflareWebsiteNavGroup: OpenFlareNavGroup = {
},
{ title: '源站地址', url: '/origins', childUrls: ['/origins/detail'] },
{
title: '错误页',
url: '/error-pages',
childUrls: ['/error-pages/edit', '/error-pages/preview'],
title: '响应页面',
url: '/responses',
childUrls: [
'/responses/error-page/edit',
'/responses/error-page/preview',
'/responses/offline/edit',
'/responses/offline/preview',
],
},
],
};
@@ -120,7 +125,7 @@ export const openflareWebsiteSubNav = [
{ title: '证书', url: '/certificates' },
{ title: 'DNS 账号', url: '/dns-accounts' },
{ title: 'Cloudflare', url: '/cloudflare' },
{ title: '错误页', url: '/error-pages' },
{ title: '响应页面', url: '/responses' },
] as const;
const nonConsoleRoutePrefixes = [
@@ -0,0 +1,170 @@
/** 内置离线页模板:baohaus */
const html = `
<!DOCTYPE html>
<html lang="zh-CN">
<head>
<meta charset="UTF-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<title>离线页 | OpenFlare</title>
<style>
*, *::before, *::after {
box-sizing: border-box;
margin: 0;
padding: 0;
}
body {
font-family: 'Inter', -apple-system, BlinkMacSystemFont, "Segoe UI", Roboto, Helvetica, Arial, sans-serif;
background-color: #f4f3ef;
color: #121212;
min-height: 100vh;
display: flex;
flex-direction: column;
justify-content: center;
align-items: center;
padding: 48px 24px;
-webkit-font-smoothing: antialiased;
}
.bauhaus-card {
width: 100%;
max-width: 580px;
background: #ffffff;
border: 3px solid #121212;
box-shadow: 8px 8px 0px #121212;
display: flex;
flex-direction: column;
overflow: hidden;
}
.card-accent-bar {
height: 12px;
display: flex;
}
.accent-red { flex: 2; background-color: #e63946; }
.accent-blue { flex: 3; background-color: #1d3557; }
.accent-yellow { flex: 1; background-color: #f1faee; }
.card-body {
padding: 40px 32px;
display: flex;
flex-direction: column;
gap: 24px;
}
.tag {
align-self: flex-start;
background-color: #121212;
color: #ffffff;
font-size: 11px;
font-weight: 700;
letter-spacing: 0.1em;
text-transform: uppercase;
padding: 4px 10px;
}
.title {
font-size: 28px;
font-weight: 800;
line-height: 1.2;
letter-spacing: -0.02em;
}
.message {
font-size: 15px;
line-height: 1.6;
color: #4a4a4a;
}
.contact-box {
border: 2px solid #121212;
background-color: #f9f8f3;
padding: 20px;
display: flex;
flex-direction: column;
gap: 12px;
}
.contact-line {
display: flex;
align-items: center;
justify-content: space-between;
font-size: 14px;
font-weight: 600;
}
.contact-line a {
color: #e63946;
text-decoration: none;
font-weight: 700;
}
.contact-line a:hover {
text-decoration: underline;
}
.card-footer {
border-top: 2px solid #121212;
padding: 16px 32px;
background-color: #faf9f6;
display: flex;
align-items: center;
justify-content: space-between;
font-size: 12px;
font-weight: 700;
letter-spacing: 0.05em;
}
.brand {
display: flex;
align-items: center;
gap: 6px;
}
.brand-icon {
width: 16px;
height: 16px;
fill: currentColor;
}
</style>
</head>
<body>
<div class="bauhaus-card">
<div class="card-accent-bar">
<div class="accent-red"></div>
<div class="accent-blue"></div>
<div class="accent-yellow"></div>
</div>
<div class="card-body">
<div class="tag">OFFLINE FALLBACK</div>
<h1 class="title">离线联系站长</h1>
<p class="message">
源站网络临时不可达。离线 Service Worker 已拦截请求,您可以使用下方联系方式取得支持。
</p>
<div class="contact-box">
<div class="contact-line">
<span>站长邮箱</span>
<a href="mailto:admin@example.com">admin@example.com</a>
</div>
<div class="contact-line">
<span>服务监控</span>
<a href="https://status.example.com" target="_blank" rel="noopener">status.example.com</a>
</div>
</div>
</div>
<div class="card-footer">
<div class="brand">
<svg class="brand-icon" viewBox="0 0 24 24" xmlns="http://www.w3.org/2000/svg">
<path d="M13 2L3 14H12L11 22L21 10H12L13 2Z" />
</svg>
<span>OpenFlare Service Worker</span>
</div>
<span>OFFLINE MODE</span>
</div>
</div>
</body>
</html>
`;
export default html;
@@ -0,0 +1,49 @@
/**
* 内置离线页 HTML 模板目录。
*/
import baohausHtml from './baohaus';
import lineStyleHtml from './line-style';
import minimalistHtml from './minimalist';
export type OfflinePageTemplate = {
/** 稳定 ID */
id: string;
/** 下拉展示名称 */
name: string;
/** 完整 HTML */
html: string;
};
export const OFFLINE_PAGE_TEMPLATES: OfflinePageTemplate[] = [
{
id: 'minimalist',
name: '极简白底',
html: minimalistHtml,
},
{
id: 'line-style',
name: '线框拓扑',
html: lineStyleHtml,
},
{
id: 'baohaus',
name: '包豪斯',
html: baohausHtml,
},
];
/** 默认预制模板 */
export const DEFAULT_OFFLINE_PAGE_TEMPLATE_ID = 'minimalist';
export const DEFAULT_OFFLINE_PAGE_HTML =
getOfflinePageTemplate(DEFAULT_OFFLINE_PAGE_TEMPLATE_ID)?.html ?? '';
export function effectiveOfflinePageHTML(html: string): string {
return html.trim() === '' ? DEFAULT_OFFLINE_PAGE_HTML : html;
}
export function getOfflinePageTemplate(
id: string,
): OfflinePageTemplate | undefined {
return OFFLINE_PAGE_TEMPLATES.find((item) => item.id === id);
}
@@ -0,0 +1,193 @@
/** 内置离线页模板:line-style */
const html = `
<!DOCTYPE html>
<html lang="zh-CN">
<head>
<meta charset="UTF-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<title>离线页 | OpenFlare</title>
<style>
*, *::before, *::after {
box-sizing: border-box;
margin: 0;
padding: 0;
}
:root {
--color-bg: #fcfcfc;
--color-line: #e0e0e0;
--color-line-hover: #1a1a1a;
--color-text-main: #1a1a1a;
--color-text-sub: #666666;
--color-text-muted: #999999;
--color-accent: #d4d4d4;
--font-geo: 'Inter', -apple-system, BlinkMacSystemFont, 'Segoe UI', Roboto, Helvetica, Arial, sans-serif;
--transition-fast: 150ms cubic-bezier(0.4, 0, 0.2, 1);
}
body {
font-family: var(--font-geo);
background-color: var(--color-bg);
background-image: radial-gradient(circle, var(--color-accent) 0.8px, transparent 0.8px);
background-size: 24px 24px;
color: var(--color-text-main);
min-height: 100vh;
display: flex;
flex-direction: column;
justify-content: center;
align-items: center;
padding: 48px 24px;
-webkit-font-smoothing: antialiased;
letter-spacing: 0.01em;
}
.main-container {
width: 100%;
max-width: 640px;
display: flex;
flex-direction: column;
gap: 32px;
}
.line-box {
border: 1px solid var(--color-line);
background: rgba(255, 255, 255, 0.7);
backdrop-filter: blur(4px);
padding: 32px;
transition: all var(--transition-fast);
position: relative;
}
.line-box:hover {
border-color: var(--color-line-hover);
box-shadow: 0 0 0 1px var(--color-line-hover);
}
.status-badge {
display: inline-flex;
align-items: center;
gap: 8px;
font-size: 11px;
font-weight: 600;
letter-spacing: 0.15em;
text-transform: uppercase;
color: var(--color-text-sub);
margin-bottom: 12px;
}
.status-dot {
width: 6px;
height: 6px;
border-radius: 50%;
background-color: #f59e0b;
}
.title {
font-size: 24px;
font-weight: 700;
letter-spacing: -0.02em;
margin-bottom: 12px;
}
.description {
font-size: 14px;
line-height: 1.6;
color: var(--color-text-sub);
margin-bottom: 24px;
}
.contact-list {
display: flex;
flex-direction: column;
gap: 12px;
border-top: 1px dashed var(--color-line);
padding-top: 20px;
}
.contact-row {
display: flex;
align-items: center;
justify-content: space-between;
font-size: 13px;
}
.contact-label {
color: var(--color-text-muted);
font-size: 12px;
letter-spacing: 0.05em;
text-transform: uppercase;
}
.contact-value a {
color: var(--color-text-main);
font-weight: 600;
text-decoration: none;
border-bottom: 1px solid var(--color-line);
transition: border-color var(--transition-fast);
}
.contact-value a:hover {
border-color: var(--color-text-main);
}
.footer-bar {
display: flex;
align-items: center;
justify-content: space-between;
font-size: 11px;
color: var(--color-text-muted);
letter-spacing: 0.05em;
}
.brand {
display: flex;
align-items: center;
gap: 6px;
font-weight: 600;
color: var(--color-text-main);
}
.brand-icon {
width: 14px;
height: 14px;
fill: currentColor;
}
</style>
</head>
<body>
<div class="main-container">
<div class="line-box">
<div class="status-badge">
<span class="status-dot"></span>
SERVICE WORKER OFFLINE CACHE
</div>
<h1 class="title">网络中断或源站不可达</h1>
<p class="description">
系统已自动启用 Service Worker 兜底保护。如需紧急联系站长,请通过以下渠道:
</p>
<div class="contact-list">
<div class="contact-row">
<span class="contact-label">Email Support</span>
<span class="contact-value"><a href="mailto:admin@example.com">admin@example.com</a></span>
</div>
<div class="contact-row">
<span class="contact-label">Status Page</span>
<span class="contact-value"><a href="https://status.example.com" target="_blank" rel="noopener">status.example.com</a></span>
</div>
</div>
</div>
<div class="footer-bar">
<div class="brand">
<svg class="brand-icon" viewBox="0 0 24 24" xmlns="http://www.w3.org/2000/svg">
<path d="M13 2L3 14H12L11 22L21 10H12L13 2Z" />
</svg>
<span>OpenFlare Edge</span>
</div>
<span>FALLBACK ACTIVE</span>
</div>
</div>
</body>
</html>
`;
export default html;
@@ -0,0 +1,128 @@
/** 内置离线页模板:minimalist(默认) */
const html = `
<!DOCTYPE html>
<html lang="zh-CN">
<head>
<meta charset="UTF-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<title>离线页 | OpenFlare</title>
<style>
* { box-sizing: border-box; margin: 0; padding: 0; }
body {
font-family: -apple-system, BlinkMacSystemFont, "Segoe UI", Roboto, Helvetica, Arial, sans-serif;
background-color: #ffffff;
color: #333333;
min-height: 100vh;
display: flex;
flex-direction: column;
justify-content: center;
align-items: center;
text-align: center;
padding: 48px 24px;
}
.container {
max-width: 560px;
width: 100%;
display: flex;
flex-direction: column;
align-items: center;
gap: 24px;
}
.badge {
display: inline-flex;
align-items: center;
padding: 4px 12px;
border-radius: 9999px;
background-color: #f3f4f6;
color: #4b5563;
font-size: 12px;
font-weight: 500;
letter-spacing: 0.05em;
}
.title {
font-size: 32px;
font-weight: 700;
color: #111827;
line-height: 1.25;
}
.description {
font-size: 15px;
line-height: 1.6;
color: #6b7280;
max-width: 480px;
}
.contact-card {
width: 100%;
background: #f9fafb;
border: 1px solid #e5e7eb;
border-radius: 12px;
padding: 24px;
display: flex;
flex-direction: column;
gap: 16px;
text-align: left;
}
.contact-item {
display: flex;
align-items: center;
gap: 12px;
font-size: 14px;
color: #374151;
}
.contact-item svg {
width: 18px;
height: 18px;
color: #6b7280;
flex-shrink: 0;
}
.contact-item a {
color: #2563eb;
text-decoration: none;
}
.contact-item a:hover {
text-decoration: underline;
}
.footer {
margin-top: 32px;
display: flex;
align-items: center;
gap: 8px;
color: #9ca3af;
font-size: 13px;
}
.brand-icon {
width: 20px;
height: 20px;
fill: currentColor;
}
</style>
</head>
<body>
<div class="container">
<div class="badge">OFFLINE CONTACT</div>
<h1 class="title">站点暂不可用,联系站长</h1>
<p class="description">
当前无法连接至源站。页面已开启 Service Worker 离线保护,您可以通过以下方式与站长取得联系。
</p>
<div class="contact-card">
<div class="contact-item">
<svg fill="none" stroke="currentColor" viewBox="0 0 24 24" xmlns="http://www.w3.org/2000/svg"><path stroke-linecap="round" stroke-linejoin="round" stroke-width="2" d="M3 8l7.89 5.26a2 2 0 002.22 0L21 8M5 19h14a2 2 0 002-2V7a2 2 0 00-2-2H5a2 2 0 00-2 2v10a2 2 0 002 2z"></path></svg>
<span>联系邮箱:<a href="mailto:admin@example.com">admin@example.com</a></span>
</div>
<div class="contact-item">
<svg fill="none" stroke="currentColor" viewBox="0 0 24 24" xmlns="http://www.w3.org/2000/svg"><path stroke-linecap="round" stroke-linejoin="round" stroke-width="2" d="M13 10V3L4 14h7v7l9-11h-7z"></path></svg>
<span>服务状态页:<a href="https://status.example.com" target="_blank" rel="noopener">status.example.com</a></span>
</div>
</div>
<div class="footer">
<svg class="brand-icon" viewBox="0 0 24 24" xmlns="http://www.w3.org/2000/svg">
<path d="M13 2L3 14H12L11 22L21 10H12L13 2Z" />
</svg>
<span>OpenFlare Protection</span>
</div>
</div>
</body>
</html>
`;
export default html;
+1
View File
@@ -29,6 +29,7 @@ export type {
UpdateUserStatusRequest,
UpdateUserRequest,
SystemStatus,
LogDatabaseStatus,
AppUpdateStatus,
Schedule,
CreateScheduleRequest,
@@ -1,5 +1,5 @@
import { BaseService } from '@/lib/services/core';
import type { AppUpdateStatus, SystemStatus } from './types';
import type { AppUpdateStatus, LogDatabaseStatus, SystemStatus } from './types';
export class AdminStatusService extends BaseService {
protected static readonly basePath = '/api/v1/admin';
@@ -8,6 +8,10 @@ export class AdminStatusService extends BaseService {
return this.get<SystemStatus>('/status');
}
static async getLogDatabaseStatus(): Promise<LogDatabaseStatus> {
return this.get<LogDatabaseStatus>('/status/log-database');
}
static async getUpdateStatus(): Promise<AppUpdateStatus> {
return this.get<AppUpdateStatus>('/update');
}
+16
View File
@@ -405,6 +405,22 @@ export interface ToggleAuthSourceRequest {
is_active: boolean;
}
/**
* 日志数据库状态
*/
export interface LogDatabaseStatus {
/** 当前日志主库:postgres | sqlite | clickhouse */
active_database: string;
/** 迁移状态:idle | migrating */
migration: string;
/** 各日志库保留天数 */
retention_days: Record<string, number>;
/** 当前主库的合法迁移目标 */
available_targets: string[];
/** ClickHouse 运行指标(仅主库为 ClickHouse 时) */
clickhouse?: unknown;
}
/**
* 系统状态信息
*/
+1 -1
View File
@@ -158,6 +158,7 @@ export type {
CreateUserRequest,
UpdateUserRequest,
SystemStatus,
LogDatabaseStatus,
AppUpdateStatus,
Schedule,
CreateScheduleRequest,
@@ -260,6 +261,5 @@ export type {
AccessLogOverview,
OptionItem,
GeoIPLookupResult,
DatabaseCleanupResult,
OpenFlarePublicStatus,
} from './openflare';
-3
View File
@@ -104,9 +104,6 @@ export type {
FoldedAccessLogList,
OptionItem,
GeoIPLookupResult,
DatabaseCleanupPayload,
DatabaseCleanupResult,
DatabaseCleanupTarget,
OpenFlarePublicStatus,
OriginDetail,
OriginItem,
@@ -1,7 +1,5 @@
import { OpenFlareBaseService } from './base.service';
import type {
DatabaseCleanupPayload,
DatabaseCleanupResult,
GeoIPLookupResult,
OptionBatchPayload,
OptionItem,
@@ -26,10 +24,4 @@ export class OptionService extends OpenFlareBaseService {
static lookupGeoIP(provider: string, ip: string): Promise<GeoIPLookupResult> {
return this.post<GeoIPLookupResult>('/geoip/lookup', { provider, ip });
}
static cleanupDatabase(
payload: DatabaseCleanupPayload,
): Promise<DatabaseCleanupResult> {
return this.post<DatabaseCleanupResult>('/database/cleanup', payload);
}
}
-21
View File
@@ -866,27 +866,6 @@ export interface GeoIPLookupResult {
longitude?: number | null;
}
export type DatabaseCleanupTarget =
| 'node_access_logs'
| 'node_metric_snapshots'
| 'node_edge_health'
| 'node_obs_frps'
| 'node_obs_frpc';
export interface DatabaseCleanupPayload {
target: DatabaseCleanupTarget;
retention_days?: number;
}
export interface DatabaseCleanupResult {
target: DatabaseCleanupTarget;
target_label: string;
deleted_count: number;
delete_all: boolean;
retention_days?: number;
cutoff?: string;
}
export interface OpenFlarePublicStatus {
version: string;
start_time: number;
+8 -6
View File
@@ -131,20 +131,22 @@ export const searchData: SearchItem[] = [
keywords: ['origin', '源站', '后端', 'backend', '服务器', '负载均衡'],
},
{
id: 'console-error-pages',
title: '错误页',
description: '配置源站错误状态码触发的统一 HTML 错误页',
url: '/error-pages',
id: 'console-responses',
title: '响应页面',
description: '配置源站错误页与离线兜底联系页',
url: '/responses',
category: 'page',
keywords: [
'响应页面',
'错误页',
'联系页',
'源站',
'502',
'503',
'5xx',
'Service Worker',
'离线兜底',
'error page',
'origin error',
'error_page',
],
},
{
+20 -17
View File
@@ -14,10 +14,9 @@ import (
"time"
"github.com/Rain-kl/Wavelet/internal/apps/admin"
"github.com/Rain-kl/Wavelet/internal/infra/config"
db "github.com/Rain-kl/Wavelet/internal/infra/persistence"
analyticsmodel "github.com/Rain-kl/Wavelet/internal/model/analytics"
"github.com/Rain-kl/Wavelet/internal/repository"
analyticsrepo "github.com/Rain-kl/Wavelet/internal/repository/analytics"
"github.com/Rain-kl/Wavelet/internal/repository/logstore"
"github.com/Rain-kl/Wavelet/pkg/logger"
"github.com/gin-gonic/gin"
@@ -158,8 +157,8 @@ type accessLogsResponse struct {
List []accessLogItem `json:"list"`
}
func buildAccessLogFilter(ctx context.Context, c *gin.Context) (analyticsrepo.AccessLogFilter, error) {
filter := analyticsrepo.AccessLogFilter{}
func buildAccessLogFilter(ctx context.Context, c *gin.Context) (analyticsmodel.AccessLogFilter, error) {
filter := analyticsmodel.AccessLogFilter{}
username := c.Query("username")
if username != "" {
@@ -227,7 +226,7 @@ func enrichAccessLogsWithUsers(ctx context.Context, list []accessLogItem) {
// GetAccessLogs 获取 ClickHouse 异步采集的访问日志
// @Summary 获取用户访问日志
// @Description 分页并按照用户、接口路径、时间范围等维度检索 ClickHouse 用户访问日志列表(需要管理员权限,ClickHouse 未启用时报错)
// @Description 分页并按照用户、接口路径、时间范围等维度检索用户访问日志列表(需要管理员权限,日志存储未启用时报错)
// @Tags admin
// @Produce json
// @Security SessionCookie
@@ -238,14 +237,16 @@ func enrichAccessLogsWithUsers(ctx context.Context, list []accessLogItem) {
// @Param start_time query string false "起始时间(RFC3339 或 YYYY-MM-DD HH:MM:SS)"
// @Param end_time query string false "结束时间(RFC3339 或 YYYY-MM-DD HH:MM:SS)"
// @Success 200 {object} response.Any{data=logs.accessLogsResponse} "访问日志列表"
// @Failure 400 {object} response.Any "ClickHouse 未启用或参数错误"
// @Failure 400 {object} response.Any "日志存储未启用或参数错误"
// @Failure 401 {object} response.Any "未登录"
// @Failure 403 {object} response.Any "无管理员权限"
// @Router /api/v1/admin/logs/access [get]
func GetAccessLogs(c *gin.Context) {
ctx := c.Request.Context()
if !config.Config.ClickHouse.Enabled || !db.ChConnReady() {
response.AbortWithError(c, http.StatusBadRequest, "ClickHouse 存储服务未启用,无法检索访问日志")
store, err := logstore.Active(ctx)
if err != nil {
logger.ErrorF(ctx, "获取日志存储实例失败: %v", err)
response.AbortWithError(c, http.StatusBadRequest, "日志存储未启用,无法检索访问日志")
return
}
@@ -271,7 +272,7 @@ func GetAccessLogs(c *gin.Context) {
return
}
logs, total, err := analyticsrepo.ListAccessLogs(ctx, filter, page, pageSize)
logs, total, err := store.UserAccessLogs.List(ctx, filter, page, pageSize)
if err != nil {
response.AbortWithError(c, http.StatusInternalServerError, err.Error())
return
@@ -333,25 +334,27 @@ type logsAnalyticsResponse struct {
// GetLogsAnalytics 获取 ClickHouse 访问日志图表聚合指标
// @Summary 获取访问日志分析数据
// @Description 聚合统计最近 7 天的每日访问趋势、浏览器分布以及前 10 名最活跃用户排行(需要管理员权限,ClickHouse 未启用时报错)
// @Description 聚合统计最近 7 天的每日访问趋势、浏览器分布以及前 10 名最活跃用户排行(需要管理员权限,日志存储未启用时报错)
// @Tags admin
// @Produce json
// @Security SessionCookie
// @Success 200 {object} response.Any{data=logs.logsAnalyticsResponse} "分析统计数据"
// @Failure 400 {object} response.Any "ClickHouse 未启用"
// @Failure 400 {object} response.Any "日志存储未启用"
// @Failure 401 {object} response.Any "未登录"
// @Failure 403 {object} response.Any "无管理员权限"
// @Router /api/v1/admin/logs/analytics [get]
func GetLogsAnalytics(c *gin.Context) {
ctx := c.Request.Context()
if !config.Config.ClickHouse.Enabled || !db.ChConnReady() {
response.AbortWithError(c, http.StatusBadRequest, "ClickHouse 存储服务未启用,无法获取分析数据")
store, err := logstore.Active(ctx)
if err != nil {
logger.ErrorF(ctx, "获取日志存储实例失败: %v", err)
response.AbortWithError(c, http.StatusBadRequest, "日志存储未启用,无法获取分析数据")
return
}
startTime := time.Now().AddDate(0, 0, -(analyticsDays - 1)).Truncate(hoursInDay * time.Hour)
trendPoints, err := analyticsrepo.GetDailyTrend(ctx, analyticsDays)
trendPoints, err := store.UserAccessLogs.GetDailyTrend(ctx, analyticsDays)
if err != nil {
response.AbortWithError(c, http.StatusInternalServerError, "查询访问趋势失败: "+err.Error())
return
@@ -364,7 +367,7 @@ func GetLogsAnalytics(c *gin.Context) {
}
}
browserPoints, err := analyticsrepo.GetBrowserDistribution(ctx, startTime)
browserPoints, err := store.UserAccessLogs.GetBrowserDistribution(ctx, startTime)
if err != nil {
response.AbortWithError(c, http.StatusInternalServerError, "查询浏览器分布失败: "+err.Error())
return
@@ -377,7 +380,7 @@ func GetLogsAnalytics(c *gin.Context) {
}
}
topUserPoints, err := analyticsrepo.GetTopActiveUsers(ctx, startTime, topActiveLimit)
topUserPoints, err := store.UserAccessLogs.GetTopActiveUsers(ctx, startTime, topActiveLimit)
if err != nil {
response.AbortWithError(c, http.StatusInternalServerError, "查询活跃用户失败: "+err.Error())
return
+99 -17
View File
@@ -4,43 +4,126 @@
package status
import (
"context"
"errors"
"net/http"
"github.com/Rain-kl/Wavelet/internal/apps/openflare/chwriter"
"github.com/Rain-kl/Wavelet/internal/apps/risk_control"
"github.com/Rain-kl/Wavelet/internal/infra/config"
db "github.com/Rain-kl/Wavelet/internal/infra/persistence"
"github.com/Rain-kl/Wavelet/internal/infra/persistence/batchwriter"
analyticsrepo "github.com/Rain-kl/Wavelet/internal/repository/analytics"
"github.com/Rain-kl/Wavelet/internal/model"
analyticsmodel "github.com/Rain-kl/Wavelet/internal/model/analytics"
"github.com/Rain-kl/Wavelet/internal/repository"
"github.com/Rain-kl/Wavelet/internal/repository/logstore"
"github.com/Rain-kl/Wavelet/internal/shared/response"
"github.com/Rain-kl/Wavelet/pkg/logger"
"github.com/gin-gonic/gin"
"gorm.io/gorm"
)
// GetClickHouseStatus returns ClickHouse operational metrics for administrators.
// @Summary 获取 ClickHouse 运行指标
// @Description 返回 ClickHouse parts、mutation、async_insert 队列及进程内 batch writer 指标,需要管理员权限
// 日志库名取值(与 model 配置值、logstore provider 分支保持一致)。
const (
logDBNamePostgres = "postgres"
logDBNameSQLite = "sqlite"
logDBNameClickHouse = "clickhouse"
)
// defaultLogRetentionDays 日志保留天数配置缺失时的兜底值(与 seed 默认一致)。
const defaultLogRetentionDays = 90
// LogDatabaseStatus 日志库状态。
type LogDatabaseStatus struct {
ActiveDatabase string `json:"active_database"`
Migration string `json:"migration"` // idle | migrating
RetentionDays map[string]int `json:"retention_days"`
AvailableTargets []string `json:"available_targets"`
ClickHouse *analyticsmodel.ClickHouseOperationalStats `json:"clickhouse,omitempty"`
}
// GetLogDatabaseStatus 返回当前日志库状态。
// @Summary 获取日志数据库状态
// @Description 返回当前日志主库、迁移状态、各库保留天数与合法迁移目标,需要管理员权限
// @Tags admin
// @Produce json
// @Security SessionCookie
// @Success 200 {object} response.Any{data=analyticsrepo.ClickHouseOperationalStats} "获取成功"
// @Failure 400 {object} response.Any "ClickHouse 未启用"
// @Success 200 {object} response.Any{data=status.LogDatabaseStatus} "获取成功"
// @Failure 401 {object} response.Any "未登录"
// @Failure 403 {object} response.Any "无管理员权限"
// @Failure 500 {object} response.Any "内部错误"
// @Router /api/v1/admin/status/clickhouse [get]
func GetClickHouseStatus(c *gin.Context) {
if !config.Config.ClickHouse.Enabled || !db.ChConnReady() {
response.AbortWithError(c, http.StatusBadRequest, "ClickHouse 存储服务未启用")
// @Router /api/v1/admin/status/log-database [get]
func GetLogDatabaseStatus(c *gin.Context) {
ctx := c.Request.Context()
store, err := logstore.Active(ctx)
if err != nil {
logger.ErrorF(ctx, "获取日志存储实例失败: %v", err)
response.AbortInternal(c, "日志存储初始化失败")
return
}
stats, err := analyticsrepo.GetClickHouseOperationalStats(c.Request.Context())
// 分支判定复用同一 store 实例的 ActiveDatabase,避免与 Active 解析之间出现 TOCTOU。
activeDB, err := store.Status.ActiveDatabase(ctx)
if err != nil {
response.AbortInternal(c, "获取 ClickHouse 运行指标失败")
logger.ErrorF(ctx, "获取日志库状态失败: %v", err)
response.AbortInternal(c, "获取日志库状态失败")
return
}
stats.BatchWriters = collectBatchWriterStats()
c.JSON(http.StatusOK, response.OK(stats))
migration := "idle"
if logstore.Migrating(ctx) {
migration = "migrating"
}
out := LogDatabaseStatus{
ActiveDatabase: activeDB,
Migration: migration,
RetentionDays: map[string]int{
logDBNamePostgres: retentionOr(ctx, model.ConfigKeyLogRetentionDaysPostgres),
logDBNameSQLite: retentionOr(ctx, model.ConfigKeyLogRetentionDaysSQLite),
logDBNameClickHouse: retentionOr(ctx, model.ConfigKeyLogRetentionDaysClickHouse),
},
AvailableTargets: availableTargets(activeDB),
}
if activeDB == logDBNameClickHouse {
stats, err := store.Status.ClickHouseOperationalStats(ctx)
if err != nil {
logger.ErrorF(ctx, "获取 ClickHouse 运行指标失败: %v", err)
} else {
stats.BatchWriters = collectBatchWriterStats()
out.ClickHouse = stats
}
}
c.JSON(http.StatusOK, response.OK(out))
}
// retentionOr 读取保留天数配置,缺失或非法时返回默认值。
func retentionOr(ctx context.Context, key string) int {
v, err := repository.GetIntByKey(ctx, key)
if err != nil {
if !errors.Is(err, gorm.ErrRecordNotFound) {
logger.ErrorF(ctx, "读取日志保留天数配置失败 key=%s: %v", key, err)
}
return defaultLogRetentionDays
}
if v < 1 {
return defaultLogRetentionDays
}
return v
}
// availableTargets 返回当前日志主库的合法迁移目标(复用调用方已解析的 active):
// 当前为 clickhouse 时目标为主库(postgres/sqlite 按启动配置);当前为主库时目标为 clickhouse(仅 CH 启用时)。
func availableTargets(active string) []string {
if active == logDBNameClickHouse {
if config.Config.Database.Enabled {
return []string{logDBNamePostgres}
}
return []string{logDBNameSQLite}
}
if config.Config.ClickHouse.Enabled {
return []string{logDBNameClickHouse}
}
return []string{}
}
func collectBatchWriterStats() []batchwriter.Stats {
@@ -48,6 +131,5 @@ func collectBatchWriterStats() []batchwriter.Stats {
if out == nil {
out = make([]batchwriter.Stats, 0, 1)
}
out = append(out, risk_control.LogWriterStats())
return out
}
@@ -0,0 +1,117 @@
// Copyright 2026 Arctel.net
// SPDX-License-Identifier: Apache-2.0
package status
import (
"context"
"encoding/json"
"net/http"
"net/http/httptest"
"reflect"
"testing"
"github.com/Rain-kl/Wavelet/internal/infra/config"
"github.com/Rain-kl/Wavelet/internal/repository/logstore"
"github.com/gin-gonic/gin"
)
// restoreConfig 恢复测试中临时修改的全局配置。
func restoreConfig(t *testing.T) {
t.Helper()
dbEnabled := config.Config.Database.Enabled
chEnabled := config.Config.ClickHouse.Enabled
t.Cleanup(func() {
config.Config.Database.Enabled = dbEnabled
config.Config.ClickHouse.Enabled = chEnabled
})
}
func TestAvailableTargets(t *testing.T) {
restoreConfig(t)
logstore.ResetForTest()
t.Cleanup(logstore.ResetForTest)
// 当前 clickhouse → 主库(postgres/sqlite 按启动配置)。
logstore.SetConfigReader(func(_ context.Context, key string) (string, error) {
if key == "log_database" {
return logDBNameClickHouse, nil
}
return "", nil
})
config.Config.Database.Enabled = true
config.Config.ClickHouse.Enabled = true
if got := availableTargets(logDBNameClickHouse); !reflect.DeepEqual(got, []string{logDBNamePostgres}) {
t.Fatalf("clickhouse active + postgres main: got %v, want [postgres]", got)
}
config.Config.Database.Enabled = false
if got := availableTargets(logDBNameClickHouse); !reflect.DeepEqual(got, []string{logDBNameSQLite}) {
t.Fatalf("clickhouse active + sqlite main: got %v, want [sqlite]", got)
}
// 当前主库 → clickhouse(CH 启用时)。
logstore.SetConfigReader(func(_ context.Context, key string) (string, error) {
if key == "log_database" {
return logDBNamePostgres, nil
}
return "", nil
})
config.Config.ClickHouse.Enabled = true
if got := availableTargets(logDBNamePostgres); !reflect.DeepEqual(got, []string{logDBNameClickHouse}) {
t.Fatalf("postgres active: got %v, want [clickhouse]", got)
}
// CH 禁用时排除 clickhouse。
config.Config.ClickHouse.Enabled = false
if got := availableTargets(logDBNamePostgres); len(got) != 0 {
t.Fatalf("CH disabled: got %v, want empty", got)
}
}
// TestGetLogDatabaseStatusSmoke 覆盖 handler 的 CH 激活分支(无需 DB/CH 连接)。
func TestGetLogDatabaseStatusSmoke(t *testing.T) {
restoreConfig(t)
config.Config.Database.Enabled = false
config.Config.ClickHouse.Enabled = true
logstore.ResetForTest()
t.Cleanup(logstore.ResetForTest)
logstore.SetConfigReader(func(_ context.Context, key string) (string, error) {
if key == "log_database" {
return logDBNameClickHouse, nil
}
return "", nil
})
gin.SetMode(gin.TestMode)
w := httptest.NewRecorder()
c, _ := gin.CreateTestContext(w)
c.Request = httptest.NewRequest(http.MethodGet, "/api/v1/admin/status/log-database", nil)
GetLogDatabaseStatus(c)
if w.Code != http.StatusOK {
t.Fatalf("status code = %d, want 200; body=%s", w.Code, w.Body.String())
}
var resp struct {
Data LogDatabaseStatus `json:"data"`
}
if err := json.Unmarshal(w.Body.Bytes(), &resp); err != nil {
t.Fatalf("unmarshal body: %v", err)
}
if resp.Data.ActiveDatabase != logDBNameClickHouse {
t.Fatalf("active_database = %q, want clickhouse", resp.Data.ActiveDatabase)
}
if resp.Data.Migration != "idle" {
t.Fatalf("migration = %q, want idle", resp.Data.Migration)
}
for _, key := range []string{logDBNamePostgres, logDBNameSQLite, logDBNameClickHouse} {
if got := resp.Data.RetentionDays[key]; got != defaultLogRetentionDays {
t.Fatalf("retention_days[%s] = %d, want default %d", key, got, defaultLogRetentionDays)
}
}
if got := resp.Data.AvailableTargets; !reflect.DeepEqual(got, []string{logDBNameSQLite}) {
t.Fatalf("available_targets = %v, want [sqlite] (test main DB disabled)", got)
}
}
@@ -17,6 +17,11 @@ import (
)
func createSystemConfig(ctx context.Context, req CreateSystemConfigRequest) error {
// 防御:受保护 key(log_database / log_db_migration)仅允许内部写入,Handler 已拦截。
if isProtectedConfigKey(req.Key) {
return errors.New(protectedConfigKeyMessage)
}
exists, err := repository.SystemConfigExists(ctx, req.Key)
if err != nil {
return err
@@ -27,6 +27,17 @@ import (
const maskedConfigValue = "******"
// protectedConfigKeyMessage 命中受保护 key 时返回给管理员的业务错误文案。
const protectedConfigKeyMessage = "该配置项由系统任务管理,禁止手动修改"
// protectedConfigKeys 仅允许内部(迁移任务/bootstrap)写入的 key。
var protectedConfigKeys = map[string]bool{
model.ConfigKeyLogDatabase: true,
model.ConfigKeyLogDBMigration: true,
}
func isProtectedConfigKey(key string) bool { return protectedConfigKeys[key] }
// CreateSystemConfigRequest 创建系统配置请求
type CreateSystemConfigRequest struct {
Key string `json:"key" binding:"required,max=64"`
@@ -64,11 +75,21 @@ func CreateSystemConfig(c *gin.Context) {
return
}
// 与 PUT 路径一致:log_database / log_db_migration 仅允许内部(迁移任务/bootstrap)写入。
if isProtectedConfigKey(req.Key) {
response.AbortBadRequest(c, protectedConfigKeyMessage)
return
}
if err := createSystemConfig(c.Request.Context(), req); err != nil {
if err.Error() == ConfigKeyExists {
response.AbortBadRequest(c, ConfigKeyExists)
return
}
if err.Error() == protectedConfigKeyMessage {
response.AbortBadRequest(c, protectedConfigKeyMessage)
return
}
response.AbortInternal(c, err.Error())
return
}
@@ -155,6 +176,10 @@ func UpdateSystemConfig(c *gin.Context) {
}
key := c.Param("key")
if isProtectedConfigKey(key) {
response.AbortBadRequest(c, protectedConfigKeyMessage)
return
}
if err := updateSystemConfig(c.Request.Context(), key, req); err != nil {
if errors.Is(err, gorm.ErrRecordNotFound) {
response.AbortNotFound(c, SystemConfigNotFound)
@@ -27,7 +27,7 @@ import (
"github.com/Rain-kl/Wavelet/internal/shared/response"
)
const expectedDefaultConfigsCount = 34
const expectedDefaultConfigsCount = 37
func setupTestRouter(authUser *model.User) *gin.Engine {
r := testhelper.NewTestGinEngine()
@@ -168,8 +168,8 @@ func TestListSystemConfigs(t *testing.T) {
var configs []model.SystemConfig
_ = json.Unmarshal(dataBytes, &configs)
if len(configs) != 5 {
t.Errorf("expected 5 business configs, got %d: %v", len(configs), configs)
if len(configs) != 8 {
t.Errorf("expected 8 business configs, got %d: %v", len(configs), configs)
}
})
}
+20
View File
@@ -401,6 +401,7 @@ func (m *Manager) EnsureLuaAssets() error {
return fmt.Errorf("load pow static files: %w", err)
}
allSupportFiles = append(allSupportFiles, powStaticFiles...)
allSupportFiles = append(allSupportFiles, ManagedSWLuaFiles()...)
files := make([]managedFile, 0, len(allSupportFiles))
for _, file := range allSupportFiles {
targetPath, err := luaFileTargetPath(m.LuaDir, file.Path)
@@ -525,6 +526,8 @@ func (m *Manager) CurrentChecksum() (string, error) {
// Longer error-page path must be restored before the cert-dir prefix rewrite.
errorPagePath := filepath.ToSlash(filepath.Join(m.NginxCertDir, openrestyrender.OriginErrorPageSupportPath))
normalizedRoute = strings.ReplaceAll(normalizedRoute, errorPagePath, openrestyrender.ErrorPageTmplPlaceholder)
swDir := filepath.ToSlash(filepath.Join(m.NginxCertDir, "sw"))
normalizedRoute = strings.ReplaceAll(normalizedRoute, swDir, openrestyrender.SWDirPlaceholder)
normalizedRoute = strings.ReplaceAll(normalizedRoute, m.NginxCertDir, openrestyrender.CertDirPlaceholder)
}
if luaDir := m.luaRuntimePath(); luaDir != "" {
@@ -1022,6 +1025,7 @@ func (m *Manager) writeSourceConfig(supportFiles []protocol.SupportFile) error {
}
func (m *Manager) writeManagedCertFiles(certFiles []protocol.SupportFile) error {
certFiles = append(certFiles, ManagedSWLuaFiles()...)
files := make([]managedFile, 0, len(certFiles))
for _, file := range certFiles {
if file.Path == powConfigFileName || file.Path == "waf_config.json" || file.Path == openrestyrender.SourceConfigFileName {
@@ -1115,6 +1119,20 @@ func (m *Manager) readManagedSupportFiles() ([]protocol.SupportFile, error) {
if err != nil {
return nil, err
}
// Agent-shipped SW Lua assets are not part of the rendered bundle, so they
// must stay out of the bundle checksum to keep it aligned with the server.
swManaged := make(map[string]struct{}, len(ManagedSWLuaFiles()))
for _, file := range ManagedSWLuaFiles() {
swManaged[file.Path] = struct{}{}
}
kept := files[:0]
for _, file := range files {
if _, skip := swManaged[file.Path]; skip {
continue
}
kept = append(kept, file)
}
files = kept
powConfig, err := m.readPowConfigFile()
if err != nil {
return nil, err
@@ -1380,6 +1398,8 @@ func (m *Manager) renderRouteConfig(content string) string {
rendered = strings.ReplaceAll(rendered, openrestyrender.CertDirPlaceholder, m.NginxCertDir)
errorPagePath := filepath.ToSlash(filepath.Join(m.NginxCertDir, openrestyrender.OriginErrorPageSupportPath))
rendered = strings.ReplaceAll(rendered, openrestyrender.ErrorPageTmplPlaceholder, errorPagePath)
swDir := filepath.ToSlash(filepath.Join(m.NginxCertDir, "sw"))
rendered = strings.ReplaceAll(rendered, openrestyrender.SWDirPlaceholder, swDir)
}
if luaDir := m.luaRuntimePath(); luaDir != "" {
rendered = strings.ReplaceAll(rendered, openrestyrender.LuaDirPlaceholder, luaDir)
+79
View File
@@ -337,6 +337,85 @@ func TestManagerApplyWritesSupportFilesAndReplacesPlaceholder(t *testing.T) {
}
}
func TestManagerApplyShipsSWAssetsAndReplacesSWDirPlaceholder(t *testing.T) {
tempDir := t.TempDir()
manager := &Manager{
MainConfigPath: filepath.Join(tempDir, "nginx.conf"),
RouteConfigPath: filepath.Join(tempDir, "routes.conf"),
CertDir: filepath.Join(tempDir, "certs"),
NginxCertDir: "/etc/nginx/openflare-certs",
LuaDir: filepath.Join(tempDir, "lua"),
NginxLuaDir: "/etc/nginx/openflare-lua",
Executor: &fakeExecutor{},
}
outcome := manager.Apply(
context.Background(),
"include __OPENFLARE_ROUTE_CONFIG__;",
"alias __OPENFLARE_SW_DIR__/sw.js;\nalias __OPENFLARE_SW_DIR__/offline.html;\ncontent_by_lua_file __OPENFLARE_SW_DIR__/challenge.lua;\nrequire(\"__OPENFLARE_LUA_DIR__\")",
[]protocol.SupportFile{
{Path: "sw/sw.js", Content: "js"},
{Path: "sw/offline.html", Content: "html"},
},
)
if outcome.Status != ApplyStatusSuccess {
t.Fatalf("Apply failed: %#v", outcome)
}
routeData, err := os.ReadFile(manager.RouteConfigPath)
if err != nil {
t.Fatalf("failed to read route config: %v", err)
}
rendered := string(routeData)
for _, want := range []string{
"/etc/nginx/openflare-certs/sw/sw.js",
"/etc/nginx/openflare-certs/sw/offline.html",
"/etc/nginx/openflare-certs/sw/challenge.lua",
} {
if !strings.Contains(rendered, want) {
t.Fatalf("route config missing %q, got %s", want, rendered)
}
}
if strings.Contains(rendered, openrestyrender.SWDirPlaceholder) {
t.Fatalf("route config still contains SW dir placeholder: %s", rendered)
}
for _, path := range []string{
filepath.Join(manager.CertDir, "sw", "sw.js"),
filepath.Join(manager.CertDir, "sw", "offline.html"),
filepath.Join(manager.CertDir, "sw", "challenge.lua"),
filepath.Join(manager.CertDir, "sw", "runtime.lua"),
} {
if _, err := os.Stat(path); err != nil {
t.Fatalf("expected SW asset %s to exist: %v", path, err)
}
}
for _, path := range []string{
filepath.Join(manager.LuaDir, "sw", "challenge.lua"),
filepath.Join(manager.LuaDir, "sw", "runtime.lua"),
} {
if _, err := os.Stat(path); err != nil {
t.Fatalf("expected SW lua asset %s to exist: %v", path, err)
}
}
checksum, err := manager.CurrentChecksum()
if err != nil {
t.Fatalf("CurrentChecksum failed: %v", err)
}
expected := bundleChecksum(
"include __OPENFLARE_ROUTE_CONFIG__;",
"alias __OPENFLARE_SW_DIR__/sw.js;\nalias __OPENFLARE_SW_DIR__/offline.html;\ncontent_by_lua_file __OPENFLARE_SW_DIR__/challenge.lua;\nrequire(\"__OPENFLARE_LUA_DIR__\")",
[]protocol.SupportFile{
{Path: "sw/sw.js", Content: "js"},
{Path: "sw/offline.html", Content: "html"},
},
)
if checksum != expected {
t.Fatalf("unexpected checksum: got %s want %s", checksum, expected)
}
}
func TestManagerRenderMainConfigInitializesWAFRuntimeInWorker(t *testing.T) {
manager := &Manager{NginxLuaDir: "/etc/nginx/openflare-lua"}
rendered := manager.renderMainConfig("events {}\nhttp {\n lua_shared_dict openflare_waf_config 1m;\n server {}\n}\n")
@@ -28,5 +28,5 @@ used under the terms of the Apache 2 license.
@licend The above is the entire license notice
for the JavaScript code in this page.
*/
(()=>{var I=()=>navigator.hardwareConcurrency!==void 0?navigator.hardwareConcurrency:1;function _(e,n,s=5,o=null,i,u=Math.trunc(Math.max(I()/2,1))){console.debug("fast algo");let a="purejs";return window.isSecureContext&&(a="webcrypto"),(navigator.userAgent.includes("Firefox")||navigator.userAgent.includes("Goanna"))&&(console.log("Firefox detected, using pure-JS fallback"),a="purejs"),new Promise((E,x)=>{let M=`${e.basePrefix}/.within.website/x/cmd/anubis/static/js/worker/sha256-${a}.mjs?cacheBuster=${e.version}`,p=[],d=!1,b=()=>{console.log("PoW aborted"),h(),x(new DOMException("Aborted","AbortError"))},h=()=>{d||(d=!0,p.forEach(c=>c.terminate()),o?.removeEventListener("abort",b))};if(o!=null){if(o.aborted)return b();o.addEventListener("abort",b,{once:!0})}for(let c=0;c<u;c++){let g=new Worker(M);g.onmessage=m=>{typeof m.data=="number"?i?.(m.data):(h(),E(m.data))},g.onerror=m=>{h(),x(m)},g.postMessage({data:n,difficulty:s,nonce:c,threads:u}),p.push(g)}})}var j={fast:_,slow:_};var v=(e="",n={})=>{let s=new URL(e,window.location.href);return Object.entries(n).forEach(([o,i])=>s.searchParams.set(o,i)),s.toString()},L=e=>{let n=document.getElementById(e);return n===null?null:JSON.parse(n.textContent)},k=(e,n,s)=>v(`${s}/.within.website/x/cmd/anubis/static/img/${e}.webp`,{cacheBuster:n});var W=async()=>document.documentElement.lang,S=async e=>{let n=L("anubis_base_prefix");if(n!==null)try{return await(await fetch(`${n}/.within.website/x/cmd/anubis/static/locales/${e}.json`)).json()}catch(s){if(console.warn(`Failed to load translations for ${e}, falling back to English`),e!=="en")return await S("en");throw s}},C=()=>{let e=L("anubis_public_url");if(e!==null)return e&&window.location.href.startsWith(e)?new URLSearchParams(window.location.search).get("redir"):window.location.href},$={},D,A=async()=>{D=await W(),$=await S(D)},r=e=>$[`js_${e}`]||$[e]||e;(async()=>{await A();let e=[{name:"Web Workers",msg:r("web_workers_error"),value:window.Worker},{name:"Cookies",msg:r("cookies_error"),value:navigator.cookieEnabled}],n=document.getElementById("status"),s=document.getElementById("image"),o=document.getElementById("title"),i=document.getElementById("progress"),u=L("anubis_version"),a=L("anubis_base_prefix"),E=document.querySelector("details"),x=!1;E&&E.addEventListener("toggle",()=>{E.open&&(x=!0)});let M=({titleMsg:l,statusMsg:f,imageSrc:w})=>{o.innerHTML=l,n.innerHTML=f,s.src=w,i.style.display="none"};n.innerHTML=r("calculating");for(let{value:l,name:f,msg:w}of e)if(!l){M({titleMsg:`${r("missing_feature")} ${f}`,statusMsg:w,imageSrc:k("reject",u,a)});return}let{challenge:p,rules:d}=L("anubis_challenge"),b=j[d.algorithm];if(!b){M({titleMsg:r("challenge_error"),statusMsg:r("challenge_error_msg"),imageSrc:k("reject",u,a)});return}n.innerHTML=`${r("calculating_difficulty")} ${d.difficulty}, `,i.style.display="inline-block";let h=document.createTextNode(`${r("speed")} 0kH/s`);n.appendChild(h);let c=0,g=!1,m=Math.pow(16,-d.difficulty);try{let l=Date.now(),{hash:f,nonce:w}=await b({basePrefix:a,version:u},p.randomData,d.difficulty,null,t=>{let y=Date.now()-l;y-c>1e3&&(c=y,h.data=`${r("speed")} ${(t/y).toFixed(3)}kH/s`);let T=Math.pow(1-m,t),P=(1-Math.pow(T,2))*100;i["aria-valuenow"]=P,i.firstElementChild!==null&&(i.firstElementChild.style.width=`${P}%`),T<.1&&!g&&(n.append(document.createElement("br"),document.createTextNode(r("verification_longer"))),g=!0)}),H=Date.now();if(console.log({hash:f,nonce:w}),x){let y=function(){let T=C();window.location.replace(v(`${a}/.within.website/x/cmd/anubis/api/pass-challenge`,{id:p.id,response:f,nonce:w,redir:T,elapsedTime:H-l}))},t=document.getElementById("progress");t.style.display="flex",t.style.alignItems="center",t.style.justifyContent="center",t.style.height="2rem",t.style.borderRadius="1rem",t.style.cursor="pointer",t.style.background="#b16286",t.style.color="white",t.style.fontWeight="bold",t.style.outline="4px solid #b16286",t.style.outlineOffset="2px",t.style.width="min(20rem, 90%)",t.style.margin="1rem auto 2rem",t.innerHTML=r("finished_reading"),t.onclick=y,setTimeout(y,3e4)}else{let t=C();window.location.replace(v(`${a}/.within.website/x/cmd/anubis/api/pass-challenge`,{id:p.id,response:f,nonce:w,redir:t,elapsedTime:H-l}))}}catch(l){M({titleMsg:r("calculation_error"),statusMsg:`${r("calculation_error_msg")} ${l.message}`,imageSrc:k("reject",u,a)})}})();})();
(()=>{var I=()=>navigator.hardwareConcurrency!==void 0?navigator.hardwareConcurrency:1;function _(e,n,s=5,o=null,i,u=Math.trunc(Math.max(I()/2,1))){console.debug("fast algo");let a="purejs";return window.isSecureContext&&(a="webcrypto"),(navigator.userAgent.includes("Firefox")||navigator.userAgent.includes("Goanna"))&&(console.log("Firefox detected, using pure-JS fallback"),a="purejs"),new Promise((E,x)=>{let M=`${e.basePrefix}/.within.website/x/cmd/anubis/static/js/worker/sha256-${a}.mjs?cacheBuster=${e.version}`,p=[],d=!1,b=()=>{console.log("PoW aborted"),h(),x(new DOMException("Aborted","AbortError"))},h=()=>{d||(d=!0,p.forEach(c=>c.terminate()),o?.removeEventListener("abort",b))};if(o!=null){if(o.aborted)return b();o.addEventListener("abort",b,{once:!0})}for(let c=0;c<u;c++){let g=new Worker(M);g.onmessage=m=>{typeof m.data=="number"?i?.(m.data):(h(),E(m.data))},g.onerror=m=>{h(),x(m)},g.postMessage({data:n,difficulty:s,nonce:c,threads:u}),p.push(g)}})}var j={fast:_,slow:_};var v=(e="",n={})=>{let s=new URL(e,window.location.href);return Object.entries(n).forEach(([o,i])=>s.searchParams.set(o,i)),s.toString()},L=e=>{let n=document.getElementById(e);return n===null?null:JSON.parse(n.textContent)},k=(e,n,s)=>v(`${s}/.within.website/x/cmd/anubis/static/img/${e}.webp`,{cacheBuster:n});var W=async()=>document.documentElement.lang,S=async e=>{let n=L("anubis_base_prefix");if(n!==null)try{return await(await fetch(`${n}/.within.website/x/cmd/anubis/static/locales/${e}.json`)).json()}catch(s){if(console.warn(`Failed to load translations for ${e}, falling back to English`),e!=="en")return await S("en");throw s}},C=()=>{let e=L("anubis_public_url");if(e!==null&&e&&window.location.href.startsWith(e)){let t=new URLSearchParams(window.location.search).get("redir");if(t){try{let u=new URL(t,window.location.href);if(u.protocol==="http:"||u.protocol==="https:")return t}catch(s){}}return window.location.href}return window.location.href},$={},D,A=async()=>{D=await W(),$=await S(D)},r=e=>$[`js_${e}`]||$[e]||e;(async()=>{await A();let e=[{name:"Web Workers",msg:r("web_workers_error"),value:window.Worker},{name:"Cookies",msg:r("cookies_error"),value:navigator.cookieEnabled}],n=document.getElementById("status"),s=document.getElementById("image"),o=document.getElementById("title"),i=document.getElementById("progress"),u=L("anubis_version"),a=L("anubis_base_prefix"),E=document.querySelector("details"),x=!1;E&&E.addEventListener("toggle",()=>{E.open&&(x=!0)});let M=({titleMsg:l,statusMsg:f,imageSrc:w})=>{o.textContent=l,n.textContent=f,s.src=w,i.style.display="none"};n.textContent=r("calculating");for(let{value:l,name:f,msg:w}of e)if(!l){M({titleMsg:`${r("missing_feature")} ${f}`,statusMsg:w,imageSrc:k("reject",u,a)});return}let{challenge:p,rules:d}=L("anubis_challenge"),b=j[d.algorithm];if(!b){M({titleMsg:r("challenge_error"),statusMsg:r("challenge_error_msg"),imageSrc:k("reject",u,a)});return}n.textContent=`${r("calculating_difficulty")} ${d.difficulty}, `,i.style.display="inline-block";let h=document.createTextNode(`${r("speed")} 0kH/s`);n.appendChild(h);let c=0,g=!1,m=Math.pow(16,-d.difficulty);try{let l=Date.now(),{hash:f,nonce:w}=await b({basePrefix:a,version:u},p.randomData,d.difficulty,null,t=>{let y=Date.now()-l;y-c>1e3&&(c=y,h.data=`${r("speed")} ${(t/y).toFixed(3)}kH/s`);let T=Math.pow(1-m,t),P=(1-Math.pow(T,2))*100;i["aria-valuenow"]=P,i.firstElementChild!==null&&(i.firstElementChild.style.width=`${P}%`),T<.1&&!g&&(n.append(document.createElement("br"),document.createTextNode(r("verification_longer"))),g=!0)}),H=Date.now();if(console.log({hash:f,nonce:w}),x){let y=function(){let T=C();window.location.replace(v(`${a}/.within.website/x/cmd/anubis/api/pass-challenge`,{id:p.id,response:f,nonce:w,redir:T,elapsedTime:H-l}))},t=document.getElementById("progress");t.style.display="flex",t.style.alignItems="center",t.style.justifyContent="center",t.style.height="2rem",t.style.borderRadius="1rem",t.style.cursor="pointer",t.style.background="#b16286",t.style.color="white",t.style.fontWeight="bold",t.style.outline="4px solid #b16286",t.style.outlineOffset="2px",t.style.width="min(20rem, 90%)",t.style.margin="1rem auto 2rem",t.textContent=r("finished_reading"),t.onclick=y,setTimeout(y,3e4)}else{let t=C();window.location.replace(v(`${a}/.within.website/x/cmd/anubis/api/pass-challenge`,{id:p.id,response:f,nonce:w,redir:t,elapsedTime:H-l}))}}catch(l){M({titleMsg:r("calculation_error"),statusMsg:`${r("calculation_error_msg")} ${l.message}`,imageSrc:k("reject",u,a)})}})();})();
//# sourceMappingURL=main.mjs.map
+111
View File
@@ -0,0 +1,111 @@
package nginx
import (
"github.com/Rain-kl/Wavelet/internal/apps/agent/protocol"
)
const openRestySWRuntimeLua = `local _M = {}
local source = debug.getinfo(1, "S").source or ""
if string.sub(source, 1, 1) == "@" then
local script_path = string.sub(source, 2)
local base_dir = string.match(script_path, "^(.*)/sw/[^/]+%.lua$")
if base_dir and base_dir ~= "" and not string.find(package.path, base_dir, 1, true) then
package.path = base_dir .. "/?.lua;" .. base_dir .. "/?/init.lua;" .. package.path
end
end
local function is_real_browser(ua)
if not ua or ua == "" then return false end
-- Chrome/Edge/CentOS-style: "Chrome/120" (pattern mode: %d = digit)
if string.find(ua, "Chrome/%d", 1) then return true end
-- Firefox: "Firefox/120"
if string.find(ua, "Firefox/%d", 1) then return true end
-- Safari (non-Chrome, e.g. "Version/17.0 Safari")
if not string.find(ua, "Chrome", 1, true) and string.find(ua, "Safari", 1, true) then return true end
return false
end
local function pass_through()
return true
end
function _M.check()
local ua = ngx.var.http_user_agent or ""
if not is_real_browser(ua) then return pass_through() end
local uri = ngx.var.uri or ""
if uri ~= "/" then return pass_through() end
if ngx.req.get_method and ngx.req.get_method() ~= "GET" then return pass_through() end
local cookie = ngx.var["cookie___openflare_sw"]
if cookie and cookie ~= "" then return pass_through() end
-- intercept: internal redirect to challenge page, which registers SW + sets cookie
local redir = ngx.var.scheme .. "://" .. ngx.var.host .. uri .. (ngx.var.args and ("?" .. ngx.var.args) or "")
ngx.req.set_uri_args({ redir = redir })
return ngx.exec("/__openflare_sw_challenge")
end
return _M
`
const openRestySWChallengeLua = `local args = ngx.req.get_uri_args()
local redir = args["redir"] or "/"
-- Escape redir for embedding inside a JS string literal within an HTML
-- <script> element. Backslashes first so later escapes stay escaped, then
-- double quotes (string-literal break-out), then "<" (prevents a raw
-- "</script" sequence ending the element, which the HTML parser matches
-- case-insensitively), then CR/LF (a raw newline would end the literal).
local function escape_redir(value)
local escaped = string.gsub(value, "\\", "\\\\")
escaped = string.gsub(escaped, '"', '\\"')
escaped = string.gsub(escaped, "<", "\\x3C")
escaped = string.gsub(escaped, string.char(0xE2, 0x80, 0xA8), "\\u2028")
escaped = string.gsub(escaped, string.char(0xE2, 0x80, 0xA9), "\\u2029")
escaped = string.gsub(escaped, "\r", "\\r")
escaped = string.gsub(escaped, "\n", "\\n")
return escaped
end
redir = escape_redir(redir)
ngx.header.content_type = "text/html; charset=utf-8"
ngx.say([[<!DOCTYPE html>
<html lang="zh-CN">
<head>
<meta charset="utf-8">
<meta name="viewport" content="width=device-width, initial-scale=1">
<meta name="robots" content="noindex,nofollow">
<title></title>
<script>
console.debug("[sw-challenge] challenge page loaded, redirect target: ]] .. redir .. [[");
if ("serviceWorker" in navigator) {
console.debug("[sw-challenge] registering service worker /sw.js");
navigator.serviceWorker.register("/sw.js").then(function () {
console.debug("[sw-challenge] service worker registered");
document.cookie = "__openflare_sw=1; Path=/; Max-Age=31536000; Secure; SameSite=Lax";
location.replace("]] .. redir .. [[");
}).catch(function (err) {
console.debug("[sw-challenge] service worker registration failed, redirecting anyway: ", err);
location.replace("]] .. redir .. [[");
});
} else {
console.debug("[sw-challenge] service worker unsupported, redirecting");
document.cookie = "__openflare_sw=1; Path=/; Max-Age=31536000; Secure; SameSite=Lax";
location.replace("]] .. redir .. [[");
}
</script>
</head>
<body></body>
</html>]])
`
// ManagedSWLuaFiles returns embedded Lua assets for the SW offline challenge.
func ManagedSWLuaFiles() []protocol.SupportFile {
return []protocol.SupportFile{
{Path: "sw/runtime.lua", Content: openRestySWRuntimeLua},
{Path: "sw/challenge.lua", Content: openRestySWChallengeLua},
}
}
@@ -0,0 +1,32 @@
package nginx
import (
"os"
"path/filepath"
"testing"
lua "github.com/yuin/gopher-lua"
)
func TestSWRuntimeAndChallenge(t *testing.T) {
state := lua.NewState()
defer state.Close()
runtimePath := filepath.Join(t.TempDir(), "runtime.lua")
if err := os.WriteFile(runtimePath, []byte(openRestySWRuntimeLua), 0o644); err != nil {
t.Fatal(err)
}
challengePath := filepath.Join(t.TempDir(), "challenge.lua")
if err := os.WriteFile(challengePath, []byte(openRestySWChallengeLua), 0o644); err != nil {
t.Fatal(err)
}
specPath, err := filepath.Abs("sw_runtime_spec.lua")
if err != nil {
t.Fatal(err)
}
state.SetGlobal("SW_RUNTIME_PATH", lua.LString(runtimePath))
state.SetGlobal("SW_CHALLENGE_PATH", lua.LString(challengePath))
if err := state.DoFile(specPath); err != nil {
t.Fatalf("SW runtime/challenge specification failed: %v", err)
}
}
@@ -0,0 +1,175 @@
local runtime_path = assert(SW_RUNTIME_PATH, "SW_RUNTIME_PATH is required")
local challenge_path = assert(SW_CHALLENGE_PATH, "SW_CHALLENGE_PATH is required")
local function assert_equal(actual, expected, message)
if actual ~= expected then
error((message or "values differ") .. ": expected " .. tostring(expected) .. ", got " .. tostring(actual), 2)
end
end
-- Stable tables: never rebind `exec_calls` / `redir_args` (closures capture
-- the upvalue slot; rebinding can leave stale values visible under
-- gopher-lua across long test sequences). Clear them in place instead.
local output = {}
local exec_calls = {}
local redir_args = {}
local function clear_state()
for i = 1, #exec_calls do exec_calls[i] = nil end
redir_args.redir = nil
end
ngx = {
var = {},
header = {},
exec = function(uri)
exec_calls[#exec_calls + 1] = uri
return true
end,
say = function(body) output.body = body end,
req = {
get_uri_args = function() return redir_args end,
set_uri_args = function(args) redir_args.redir = args.redir end,
},
}
local function load_runtime()
local chunk = assert(loadfile(runtime_path))
return chunk()
end
local function reset_request(user_agent, uri, cookie, args, method)
clear_state()
ngx.var = {
http_user_agent = user_agent,
uri = uri or "/",
scheme = "https",
host = "example.com",
args = args,
["cookie___openflare_sw"] = cookie,
}
ngx.req.get_method = function() return method or "GET" end
end
local function test_module_contract()
local runtime = load_runtime()
assert_equal(type(runtime), "table", "sw.runtime must return a module table, not true/nil")
assert_equal(type(runtime.check), "function", "sw.runtime must export check()")
end
local function test_non_browser_ua_passes_through()
local runtime = load_runtime()
reset_request("curl/8.0.1")
assert_equal(runtime.check(), true, "non-browser UA passes through")
assert_equal(#exec_calls, 0, "non-browser UA must not intercept")
reset_request("")
assert_equal(runtime.check(), true, "empty UA passes through")
reset_request(nil)
assert_equal(runtime.check(), true, "missing UA passes through")
end
local function test_browser_ua_non_get_passes_through()
local runtime = load_runtime()
reset_request(
"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36",
"/",
nil,
nil,
"POST"
)
assert_equal(runtime.check(), true, "non-GET request passes through")
assert_equal(#exec_calls, 0, "non-GET request must not be intercepted")
end
local function test_browser_ua_with_cookie_passes_through()
local runtime = load_runtime()
reset_request(
"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36",
"/",
"1"
)
assert_equal(runtime.check(), true, "browser UA with cookie passes through")
assert_equal(#exec_calls, 0, "cookie holder must not be intercepted")
end
local function test_browser_ua_root_without_cookie_intercepts()
local runtime = load_runtime()
local chrome = "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36"
reset_request(chrome, "/")
runtime.check()
assert_equal(#exec_calls, 1, "browser without cookie on / must be intercepted once")
assert_equal(exec_calls[1], "/__openflare_sw_challenge", "intercept targets the challenge page")
assert_equal(redir_args.redir, "https://example.com/", "redir arg preserves scheme+host+uri")
reset_request(chrome, "/", nil, "a=1&b=2")
runtime.check()
assert_equal(#exec_calls, 1, "second request also intercepted")
assert_equal(redir_args.redir, "https://example.com/?a=1&b=2", "redir arg keeps the query string")
reset_request("Mozilla/5.0 (X11; Linux x86_64; rv:121.0) Gecko/20100101 Firefox/121.0", "/")
runtime.check()
assert_equal(exec_calls[1], "/__openflare_sw_challenge", "Firefox intercepted")
reset_request(
"Mozilla/5.0 (iPhone; CPU iPhone OS 17_0 like Mac OS X) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/17.0 Mobile/15E148 Safari/604.1",
"/"
)
runtime.check()
assert_equal(exec_calls[1], "/__openflare_sw_challenge", "Safari intercepted")
end
local function test_browser_ua_non_root_passes_through()
local runtime = load_runtime()
reset_request(
"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36",
"/about"
)
assert_equal(runtime.check(), true, "non-root uri passes through")
assert_equal(#exec_calls, 0, "non-root uri must not be intercepted")
end
local function run_challenge(redir_value)
output.body = nil
ngx.header = {}
redir_args.redir = redir_value
local chunk = assert(loadfile(challenge_path))
chunk()
return output.body
end
local function test_challenge_embeds_plain_redir()
local body = run_challenge("https://example.com/page?a=1&b=2")
assert_equal(
string.find(body, 'location.replace("https://example.com/page?a=1&b=2")', 1, true) ~= nil,
true,
"plain redir embedded verbatim"
)
end
local function test_challenge_escapes_script_breakout()
local payload = '"/><script>alert(1)</script>'
local body = run_challenge(payload)
assert_equal(string.find(body, '"><script>', 1, true), nil, "raw breakout sequence must not appear")
assert_equal(string.find(body, '\\x3C/script>', 1, true) ~= nil, true, "less-than must be hex-escaped")
assert_equal(string.find(body, '\\"', 1, true) ~= nil, true, "double quote must be backslash-escaped")
end
local function test_challenge_escapes_backslash_and_newline()
local payload = 'a\\b";' .. string.char(13, 10)
local body = run_challenge(payload)
assert_equal(string.find(body, 'a\\\\b\\";\\r\\n', 1, true) ~= nil, true, "backslash, quote and CRLF escaped")
end
test_module_contract()
test_non_browser_ua_passes_through()
test_browser_ua_non_get_passes_through()
test_browser_ua_with_cookie_passes_through()
test_browser_ua_root_without_cookie_intercepts()
test_browser_ua_non_root_passes_through()
test_challenge_embeds_plain_redir()
test_challenge_escapes_script_breakout()
test_challenge_escapes_backslash_and_newline()
return true
+21 -62
View File
@@ -21,11 +21,6 @@ const (
// TaskTypeSSLRenew is the admin task type for SSL renewal.
TaskTypeSSLRenew = "of_ssl_renew"
// DatabaseAutoCleanupTask prunes observability tables by retention policy.
DatabaseAutoCleanupTask = "openflare:database_auto_cleanup"
// TaskTypeDatabaseAutoCleanup is the admin task type for observability cleanup.
TaskTypeDatabaseAutoCleanup = "of_database_auto_cleanup"
// WAFIPGroupSyncTask syncs due automatic/subscription WAF IP groups.
WAFIPGroupSyncTask = "openflare:waf_ip_group_sync"
// TaskTypeWAFIPGroupSync is the admin task type for WAF IP group sync.
@@ -35,6 +30,11 @@ const (
UptimeKumaSyncTask = "openflare:uptime_kuma_sync"
// TaskTypeUptimeKumaSync is the admin task type for Uptime Kuma sync.
TaskTypeUptimeKumaSync = "of_uptime_kuma_sync"
// LogDBSwitchTask 切换日志数据库任务标识。
LogDBSwitchTask = "openflare:log_db_switch"
// TaskTypeLogDBSwitch is the admin task type for log database switch.
TaskTypeLogDBSwitch = "of_log_db_switch"
)
var (
@@ -54,18 +54,6 @@ var SSLRenewMeta = task.TaskMeta{
Retryable: true,
}
// DatabaseAutoCleanupMeta describes the observability auto-cleanup task.
var DatabaseAutoCleanupMeta = task.TaskMeta{
Type: TaskTypeDatabaseAutoCleanup,
AsynqTask: DatabaseAutoCleanupTask,
Name: "OpenFlare 可观测数据自动清理",
Description: "按保留天数清理访问日志、性能快照与请求聚合数据",
SupportsTime: false,
MaxRetry: task.DefaultMaxRetry,
Queue: task.QueueDefault,
Retryable: true,
}
// WAFIPGroupSyncMeta describes the WAF IP group sync task.
var WAFIPGroupSyncMeta = task.TaskMeta{
Type: TaskTypeWAFIPGroupSync,
@@ -90,6 +78,22 @@ var UptimeKumaSyncMeta = task.TaskMeta{
Retryable: true,
}
// LogDBSwitchMeta 描述切换日志数据库任务。
var LogDBSwitchMeta = task.TaskMeta{
Type: TaskTypeLogDBSwitch,
AsynqTask: LogDBSwitchTask,
Name: "切换日志数据库",
Description: "复制迁移日志数据并在成功后切换日志主库(期间禁止日志写入)",
SupportsTime: false,
MaxRetry: task.DefaultMaxRetry,
Queue: task.QueueDefault,
Retryable: true,
Params: []task.TaskParam{
{Name: "target", Label: "目标日志库", Type: "string", Required: true,
Placeholder: "postgres|sqlite|clickhouse", Description: "迁移目标:postgres(主库为 PG 时)、sqlite(主库为 SQLite 时)或 clickhouse"},
},
}
// SSLRenewHandler renews due TLS certificates.
type SSLRenewHandler struct{}
@@ -105,51 +109,6 @@ func (h *SSLRenewHandler) Execute(ctx context.Context, _ []byte) (*task.TaskResu
return &task.TaskResult{Message: msg}, nil
}
// DatabaseAutoCleanupHandler prunes observability data when auto-cleanup is enabled.
type DatabaseAutoCleanupHandler struct{}
// Execute runs retention-based cleanup for all observability targets.
func (h *DatabaseAutoCleanupHandler) Execute(ctx context.Context, _ []byte) (*task.TaskResult, error) {
// 从 SystemConfig 读取自动清理配置
enabled, _ := repository.GetBoolByKey(ctx, model.ConfigKeyDatabaseAutoCleanupEnabled)
if !enabled {
msg := "自动清理未启用,跳过执行"
task.AppendLog(ctx, "%s", msg)
return &task.TaskResult{Message: msg}, nil
}
retentionDays, _ := repository.GetIntByKey(ctx, model.ConfigKeyDatabaseAutoCleanupRetentionDays)
if retentionDays <= 0 {
retentionDays = 30 // 默认保留 30 天
}
task.AppendLog(ctx, "开始执行可观测数据自动清理,保留天数=%d", retentionDays)
summary, err := tasks.RunDatabaseAutoCleanupOnce(ctx, time.Now())
if err != nil {
task.AppendLog(ctx, "可观测数据自动清理失败: %v", err)
return nil, err
}
if summary == nil {
msg := "自动清理未启用,跳过执行"
task.AppendLog(ctx, "%s", msg)
return &task.TaskResult{Message: msg}, nil
}
var totalDeleted int64
for _, item := range summary.Results {
totalDeleted += item.DeletedCount
task.AppendLog(ctx, "清理 %s:删除 %d 条", item.TargetLabel, item.DeletedCount)
}
msg := fmt.Sprintf(
"可观测数据自动清理完成,保留 %d 天,共删除 %d 条",
summary.RetentionDays,
totalDeleted,
)
task.AppendLog(ctx, "%s", msg)
return &task.TaskResult{Message: msg}, nil
}
// WAFIPGroupSyncHandler syncs due WAF IP groups to agents.
type WAFIPGroupSyncHandler struct{}
@@ -6,7 +6,6 @@ package openflare
import (
"context"
"testing"
"time"
db "github.com/Rain-kl/Wavelet/internal/infra/persistence"
"github.com/Rain-kl/Wavelet/internal/model"
@@ -17,63 +16,6 @@ import (
"gorm.io/gorm"
)
func TestDatabaseAutoCleanupHandlerSkipsWhenDisabled(t *testing.T) {
sqliteDB, err := gorm.Open(sqlite.Open(":memory:"), &gorm.Config{
DisableForeignKeyConstraintWhenMigrating: true,
})
require.NoError(t, err)
require.NoError(t, sqliteDB.AutoMigrate(&model.SystemConfig{}))
db.SetDB(sqliteDB)
t.Cleanup(func() { db.SetDB(nil) })
ctx := context.Background()
require.NoError(t, repository.SaveOrUpdateSystemConfig(ctx, model.ConfigKeyDatabaseAutoCleanupEnabled, "false"))
result, err := (&DatabaseAutoCleanupHandler{}).Execute(ctx, nil)
require.NoError(t, err)
require.NotNil(t, result)
assert.Contains(t, result.Message, "未启用")
}
func TestDatabaseAutoCleanupHandlerDeletesRowsWhenEnabled(t *testing.T) {
sqliteDB, err := gorm.Open(sqlite.Open(":memory:"), &gorm.Config{
DisableForeignKeyConstraintWhenMigrating: true,
})
require.NoError(t, err)
require.NoError(t, sqliteDB.AutoMigrate(&model.SystemConfig{}))
db.SetDB(sqliteDB)
resetAccessLogStore := repository.SetAccessLogStoreForTest(repository.NewMemoryAccessLogStore())
resetObservabilityStore := repository.SetObservabilityStoreForTest(repository.NewMemoryObservabilityStore())
t.Cleanup(func() {
resetObservabilityStore()
resetAccessLogStore()
db.SetDB(nil)
})
ctx := context.Background()
now := time.Now().UTC()
require.NoError(t, repository.InsertOpenFlareAccessLogsBatch(ctx, []*model.OpenFlareAccessLog{{
NodeID: "node-a",
LoggedAt: now.Add(-95 * 24 * time.Hour),
RemoteAddr: "203.0.113.10",
Host: "example.com",
Path: "/access",
StatusCode: 200,
}}))
require.NoError(t, repository.SaveOrUpdateSystemConfig(ctx, model.ConfigKeyDatabaseAutoCleanupEnabled, "true"))
require.NoError(t, repository.SaveOrUpdateSystemConfig(ctx, model.ConfigKeyDatabaseAutoCleanupRetentionDays, "1"))
result, err := (&DatabaseAutoCleanupHandler{}).Execute(ctx, nil)
require.NoError(t, err)
require.NotNil(t, result)
assert.Contains(t, result.Message, "共删除")
rows, err := repository.ListOpenFlareAccessLogs(ctx, model.OpenFlareAccessLogQuery{Page: 0, PageSize: 10})
require.NoError(t, err)
assert.Empty(t, rows)
}
func TestUptimeKumaSyncHandlerSkipsWhenDisabled(t *testing.T) {
sqliteDB, err := gorm.Open(sqlite.Open(":memory:"), &gorm.Config{
DisableForeignKeyConstraintWhenMigrating: true,
+102 -20
View File
@@ -11,13 +11,10 @@ import (
"sync"
"time"
"github.com/Rain-kl/Wavelet/internal/repository"
"github.com/Rain-kl/Wavelet/internal/infra/config"
"github.com/Rain-kl/Wavelet/internal/infra/persistence/batchwriter"
analyticsmodel "github.com/Rain-kl/Wavelet/internal/model/analytics"
"github.com/Rain-kl/Wavelet/internal/platform/lifecycle"
analyticsrepo "github.com/Rain-kl/Wavelet/internal/repository/analytics"
"github.com/Rain-kl/Wavelet/internal/repository/logstore"
"github.com/Rain-kl/Wavelet/pkg/logger"
)
@@ -56,12 +53,10 @@ var (
frpcDedup *dedupSet
)
// Init starts OpenFlare ClickHouse batch writers. Safe to call multiple times.
// Init starts OpenFlare log batch writers. Safe to call multiple times.
// Writers always initialize regardless of ClickHouse.enabled; the active log
// store is resolved via logstore at flush time (PG/SQLite when CH is not active).
func Init(ctx context.Context) {
if !config.Config.ClickHouse.Enabled {
return
}
initOnce.Do(func() {
metricSnapshotDedup = newDedupSet()
edgeHealthDedup = newDedupSet()
@@ -70,25 +65,25 @@ func Init(ctx context.Context) {
metricSnapshotWriter = mustNewObservabilityWriter(
"metric_snapshots",
withFlushRetries(analyticsrepo.BatchInsertNodeMetricSnapshots),
withFlushRetries(flushNodeMetricSnapshots),
metricSnapshotDedup,
metricSnapshotKey,
)
edgeHealthWriter = mustNewObservabilityWriter(
"edge_health",
withFlushRetries(analyticsrepo.BatchInsertNodeEdgeHealth),
withFlushRetries(flushNodeEdgeHealth),
edgeHealthDedup,
edgeHealthKey,
)
frpsWriter = mustNewObservabilityWriter(
"frps_obs",
withFlushRetries(analyticsrepo.BatchInsertNodeObsFrps),
withFlushRetries(flushNodeObsFrps),
frpsDedup,
frpsKey,
)
frpcWriter = mustNewObservabilityWriter(
"frpc_obs",
withFlushRetries(analyticsrepo.BatchInsertNodeObsFrpc),
withFlushRetries(flushNodeObsFrpc),
frpcDedup,
frpcKey,
)
@@ -129,6 +124,51 @@ func Stop(ctx context.Context) error {
return firstErr
}
// Drain 等待所有 OpenFlare 日志 writer 的在途批次落库:轮询队列 Depth 归零后
// 再保持一个最大 flush 周期(observabilityFlushEvery)持续为空才返回;
// 不停止 writer(迁移冻结后由 ensureWritable 拒绝新写入)。未初始化时直接返回 nil。
func Drain(ctx context.Context) error {
return drainWriters(ctx, WriterStats, observabilityFlushEvery)
}
// drainWriters 轮询 stats 直至所有队列 Depth=0 并持续 quietPeriod 无新积压。
func drainWriters(ctx context.Context, stats func() []batchwriter.Stats, quietPeriod time.Duration) error {
if !running() {
return nil
}
ticker := time.NewTicker(drainPollInterval)
defer ticker.Stop()
var quietSince time.Time
for {
if allDepthZero(stats()) {
if quietSince.IsZero() {
quietSince = time.Now()
} else if time.Since(quietSince) >= quietPeriod {
return nil
}
} else {
quietSince = time.Time{}
}
select {
case <-ctx.Done():
return ctx.Err()
case <-ticker.C:
}
}
}
// drainPollInterval 队列轮询间隔。
const drainPollInterval = 50 * time.Millisecond
func allDepthZero(stats []batchwriter.Stats) bool {
for _, s := range stats {
if s.Depth > 0 {
return false
}
}
return true
}
// WriterStats returns queue depth and failure counters for all OpenFlare writers.
func WriterStats() []batchwriter.Stats {
writers := []statsProvider{
@@ -237,7 +277,7 @@ func mustNewNodeAccessLogWriter() *batchwriter.Writer[analyticsmodel.NodeAccessL
}
writer, err := batchwriter.New[analyticsmodel.NodeAccessLog](
cfg,
withFlushRetries(analyticsrepo.BatchInsertNodeAccessLogs),
withFlushRetries(flushNodeAccessLogs),
batchwriter.WithDropHandler[analyticsmodel.NodeAccessLog](func(item analyticsmodel.NodeAccessLog) {
logger.WarnF(context.Background(), "[OpenFlare] node access log queue full, dropping log for node %s path %s", item.NodeID, item.Path)
}),
@@ -280,17 +320,59 @@ func withFlushRetries[T any](flush batchwriter.FlushFunc[T]) batchwriter.FlushFu
}
func wireModelInsertHooks() {
repository.SetObservabilityInsertHooks(repository.ObservabilityInsertHooks{
QueueMetricSnapshot: QueueMetricSnapshot,
QueueEdgeHealth: QueueEdgeHealth,
QueueFrpsObservation: QueueFrpsObservation,
QueueFrpcObservation: QueueFrpcObservation,
logstore.SetObservabilityHooks(logstore.ObservabilityHooks{
QueueMetricSnapshot: QueueMetricSnapshot,
QueueEdgeHealth: QueueEdgeHealth,
QueueNodeObsFrps: QueueFrpsObservation,
QueueNodeObsFrpc: QueueFrpcObservation,
})
repository.SetAccessLogInsertHooks(repository.AccessLogInsertHooks{
logstore.SetAccessLogHooks(logstore.AccessLogHooks{
QueueNodeAccessLogs: QueueNodeAccessLogs,
})
}
// 以下 flush 函数作为 batchwriter 的落库目标:激活库由 logstore 在 flush 时决定。
func flushNodeMetricSnapshots(ctx context.Context, rows []analyticsmodel.NodeMetricSnapshot) error {
s, err := logstore.Active(ctx)
if err != nil {
return err
}
return s.Observability.BatchInsertNodeMetricSnapshots(ctx, rows)
}
func flushNodeEdgeHealth(ctx context.Context, rows []analyticsmodel.NodeEdgeHealth) error {
s, err := logstore.Active(ctx)
if err != nil {
return err
}
return s.Observability.BatchInsertNodeEdgeHealth(ctx, rows)
}
func flushNodeObsFrps(ctx context.Context, rows []analyticsmodel.NodeObsFrps) error {
s, err := logstore.Active(ctx)
if err != nil {
return err
}
return s.Observability.BatchInsertNodeObsFrps(ctx, rows)
}
func flushNodeObsFrpc(ctx context.Context, rows []analyticsmodel.NodeObsFrpc) error {
s, err := logstore.Active(ctx)
if err != nil {
return err
}
return s.Observability.BatchInsertNodeObsFrpc(ctx, rows)
}
func flushNodeAccessLogs(ctx context.Context, rows []analyticsmodel.NodeAccessLog) error {
s, err := logstore.Active(ctx)
if err != nil {
return err
}
return s.AccessLogs.BatchInsertNodeAccessLogs(ctx, rows)
}
func metricSnapshotKey(snapshot analyticsmodel.NodeMetricSnapshot) string {
return fmt.Sprintf("%s|%d", snapshot.NodeID, snapshot.CapturedAt.UTC().UnixNano())
}
@@ -538,6 +538,9 @@ func diffOpenRestyOptionDetails(left openRestyConfigSnapshot, right openRestyCon
appendIfChanged("OriginErrorPageStatusCodes", encodeOriginErrorPageStatusCodes(left.OriginErrorPageStatusCodes), encodeOriginErrorPageStatusCodes(right.OriginErrorPageStatusCodes))
appendIfChanged("OriginErrorPageHTML", left.OriginErrorPageHTML, right.OriginErrorPageHTML)
appendIfChanged("OriginErrorPageGetOnly", fmt.Sprintf("%t", left.OriginErrorPageGetOnly), fmt.Sprintf("%t", right.OriginErrorPageGetOnly))
appendIfChanged("SWOfflineEnabled", fmt.Sprintf("%t", left.SWOfflineEnabled), fmt.Sprintf("%t", right.SWOfflineEnabled))
appendIfChanged("SWOfflineHTML", left.SWOfflineHTML, right.SWOfflineHTML)
appendIfChanged("SWOfflineDomains", encodeSWOfflineDomains(left.SWOfflineDomains), encodeSWOfflineDomains(right.SWOfflineDomains))
return changes
}
@@ -552,6 +555,17 @@ func encodeOriginErrorPageStatusCodes(tags []string) string {
return string(payload)
}
func encodeSWOfflineDomains(domains []string) string {
if len(domains) == 0 {
return ""
}
payload, err := json.Marshal(domains)
if err != nil {
return strings.Join(domains, ",")
}
return string(payload)
}
func extractOptionDiffKeys(details []ConfigOptionDiffItem) []string {
keys := make([]string, 0, len(details))
for _, item := range details {
@@ -605,5 +619,8 @@ func openRestyOptionKeys() []string {
"OriginErrorPageStatusCodes",
"OriginErrorPageHTML",
"OriginErrorPageGetOnly",
"SWOfflineEnabled",
"SWOfflineHTML",
"SWOfflineDomains",
}
}
@@ -101,3 +101,21 @@ func TestDiffOpenRestyOptionDetailsOriginErrorPage(t *testing.T) {
assert.Equal(t, "", keys["OriginErrorPageHTML"].PreviousValue)
assert.Equal(t, "<p>x</p>", keys["OriginErrorPageHTML"].CurrentValue)
}
func TestDiffOpenRestyOptionDetailsSWOfflineDomains(t *testing.T) {
t.Parallel()
left := openRestyConfigSnapshot{
SWOfflineDomains: []string{"a.com,b.com"},
}
right := openRestyConfigSnapshot{
SWOfflineDomains: []string{"a.com", "b.com"},
}
details := diffOpenRestyOptionDetails(left, right)
keys := make(map[string]ConfigOptionDiffItem, len(details))
for _, item := range details {
keys[item.Key] = item
}
assert.Equal(t, `["a.com,b.com"]`, keys["SWOfflineDomains"].PreviousValue)
assert.Equal(t, `["a.com","b.com"]`, keys["SWOfflineDomains"].CurrentValue)
}
@@ -144,6 +144,9 @@ type openRestyConfigSnapshot struct {
OriginErrorPageStatusCodes []string `json:"origin_error_page_status_codes,omitempty"`
OriginErrorPageHTML string `json:"origin_error_page_html,omitempty"`
OriginErrorPageGetOnly bool `json:"origin_error_page_get_only,omitempty"`
SWOfflineEnabled bool `json:"sw_offline_enabled,omitempty"`
SWOfflineHTML string `json:"sw_offline_html,omitempty"`
SWOfflineDomains []string `json:"sw_offline_domains,omitempty"`
}
type snapshotDocument struct {
@@ -514,6 +517,18 @@ func buildOpenRestyConfigSnapshot(ctx context.Context) openRestyConfigSnapshot {
return config.Value
}
getStringSliceConfig := func(key string, defaultVal []string) []string {
config, err := repository.GetSystemConfigByKey(ctx, key)
if err != nil {
return defaultVal
}
var values []string
if err := json.Unmarshal([]byte(config.Value), &values); err != nil {
return defaultVal
}
return values
}
snapshot := openRestyConfigSnapshot{
DefaultServerReturnStatus: getIntConfig(model.ConfigKeyOpenRestyDefaultServerReturnStatus, defaultOpenRestyReturnStatus),
WorkerProcesses: getStringConfig(model.ConfigKeyOpenRestyWorkerProcesses, "auto"),
@@ -560,6 +575,9 @@ func buildOpenRestyConfigSnapshot(ctx context.Context) openRestyConfigSnapshot {
OriginErrorPageStatusCodes: parseOriginErrorPageStatusCodes(getStringConfig(model.ConfigKeyOriginErrorPageStatusCodes, `["500-599"]`)),
OriginErrorPageHTML: getStringConfig(model.ConfigKeyOriginErrorPageHTML, ""),
OriginErrorPageGetOnly: getBoolConfig(model.ConfigKeyOriginErrorPageGetOnly, false),
SWOfflineEnabled: getBoolConfig(model.ConfigKeySWOfflineEnabled, false),
SWOfflineHTML: getStringConfig(model.ConfigKeySWOfflineHTML, ""),
SWOfflineDomains: getStringSliceConfig(model.ConfigKeySWOfflineDomains, nil),
}
if snapshot.DefaultLimitRate == "0" {
snapshot.DefaultLimitRate = ""
@@ -9,6 +9,7 @@ import (
"time"
"github.com/Rain-kl/Wavelet/internal/repository"
"github.com/Rain-kl/Wavelet/internal/testhelper"
db "github.com/Rain-kl/Wavelet/internal/infra/persistence"
"github.com/Rain-kl/Wavelet/internal/model"
@@ -26,11 +27,8 @@ func setupDashboardTestDB(t *testing.T) func() {
require.NoError(t, sqliteDB.AutoMigrate(&model.OpenFlareNode{}))
db.SetDB(sqliteDB)
resetAccessLogStore := repository.SetAccessLogStoreForTest(repository.NewMemoryAccessLogStore())
resetObservabilityStore := repository.SetObservabilityStoreForTest(repository.NewMemoryObservabilityStore())
testhelper.SetupLogStoresForTest(t)
return func() {
resetObservabilityStore()
resetAccessLogStore()
db.SetDB(nil)
}
}
@@ -40,15 +40,12 @@ func setupProtocolTestEnv(t *testing.T) (*gin.Engine, func()) {
db.SetDB(sqliteDB)
agent.ResetAuthCacheForTest()
resetAccessLogStore := repository.SetAccessLogStoreForTest(repository.NewMemoryAccessLogStore())
resetObservabilityStore := repository.SetObservabilityStoreForTest(repository.NewMemoryObservabilityStore())
testhelper.SetupLogStoresForTest(t)
engine := testhelper.NewTestGinEngine()
mountOpenFlareTestRoutes(engine)
cleanup := func() {
resetObservabilityStore()
resetAccessLogStore()
db.SetDB(nil)
agent.ResetAuthCacheForTest()
}
+2 -4
View File
@@ -15,6 +15,7 @@ import (
db "github.com/Rain-kl/Wavelet/internal/infra/persistence"
"github.com/Rain-kl/Wavelet/internal/model"
"github.com/Rain-kl/Wavelet/internal/repository"
"github.com/Rain-kl/Wavelet/internal/testhelper"
"github.com/glebarez/sqlite"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
@@ -41,12 +42,9 @@ func setupNodeTestDB(t *testing.T) func() {
))
db.SetDB(sqliteDB)
resetAccessLogStore := repository.SetAccessLogStoreForTest(repository.NewMemoryAccessLogStore())
resetObservabilityStore := repository.SetObservabilityStoreForTest(repository.NewMemoryObservabilityStore())
testhelper.SetupLogStoresForTest(t)
return func() {
resetObservabilityStore()
resetAccessLogStore()
db.SetDB(nil)
}
}
@@ -11,7 +11,7 @@ import (
"github.com/Rain-kl/Wavelet/internal/repository"
"github.com/Rain-kl/Wavelet/internal/model"
analyticsrepo "github.com/Rain-kl/Wavelet/internal/repository/analytics"
analyticsmodel "github.com/Rain-kl/Wavelet/internal/model/analytics"
"github.com/Rain-kl/Wavelet/pkg/logger"
)
@@ -448,9 +448,9 @@ func buildAccessLogUADistributions(
for _, row := range uaRows {
ua := row.Key
count := row.Value
deviceAcc[analyticsrepo.ParseDeviceType(ua)] += count
browserAcc[analyticsrepo.ParseBrowserName(ua)] += count
osAcc[analyticsrepo.ParseOSName(ua)] += count
deviceAcc[analyticsmodel.ParseDeviceType(ua)] += count
browserAcc[analyticsmodel.ParseBrowserName(ua)] += count
osAcc[analyticsmodel.ParseOSName(ua)] += count
}
topUserAgents = make([]DistributionItem, 0, accessLogOverviewTopLimit)
-43
View File
@@ -10,7 +10,6 @@ import (
"strings"
"github.com/Rain-kl/Wavelet/internal/apps/openflare/geoip"
oftasks "github.com/Rain-kl/Wavelet/internal/apps/openflare/tasks"
"github.com/Rain-kl/Wavelet/internal/apps/openflare/uptimekuma"
"github.com/Rain-kl/Wavelet/internal/buildinfo"
"github.com/Rain-kl/Wavelet/internal/model"
@@ -50,22 +49,6 @@ type geoIPLookupView struct {
Longitude *float64 `json:"longitude,omitempty"`
}
type databaseCleanupInput struct {
Target string `json:"target"`
RetentionDays *int `json:"retention_days"`
}
type databaseCleanupResult struct {
Target string `json:"target"`
TargetLabel string `json:"target_label"`
DeletedCount int64 `json:"deleted_count"`
EligibleCount int64 `json:"eligible_count,omitempty"`
CleanupMode string `json:"cleanup_mode,omitempty"`
TableTTLDays int `json:"table_ttl_days,omitempty"`
DeleteAll bool `json:"delete_all"`
RetentionDays *int `json:"retention_days,omitempty"`
}
type optionBatchPayload struct {
Options []model.OpenFlareOption `json:"options"`
}
@@ -183,32 +166,6 @@ func lookupGeoIP(_ context.Context, provider, rawIP string) (*geoIPLookupView, e
}, nil
}
func cleanupDatabaseObservability(ctx context.Context, input databaseCleanupInput) (*databaseCleanupResult, error) {
target := strings.TrimSpace(input.Target)
if target == "" {
return nil, errors.New(errInvalidParams)
}
result, err := oftasks.CleanupDatabaseObservability(ctx, oftasks.DatabaseCleanupInput{
Target: target,
RetentionDays: input.RetentionDays,
})
if err != nil {
return nil, err
}
return &databaseCleanupResult{
Target: result.Target,
TargetLabel: result.TargetLabel,
DeletedCount: result.DeletedCount,
EligibleCount: result.EligibleCount,
CleanupMode: result.CleanupMode,
TableTTLDays: result.TableTTLDays,
DeleteAll: result.DeleteAll,
RetentionDays: result.RetentionDays,
}, nil
}
func syncUptimeKuma(ctx context.Context) error {
return uptimekuma.SyncToUptimeKuma(ctx)
}
@@ -6,7 +6,6 @@ package option
import (
"context"
"testing"
"time"
db "github.com/Rain-kl/Wavelet/internal/infra/persistence"
"github.com/Rain-kl/Wavelet/internal/model"
@@ -117,55 +116,3 @@ func TestLookupGeoIPDisabledProvider(t *testing.T) {
assert.Equal(t, "disabled", view.Provider)
assert.Equal(t, "8.8.8.8", view.IP)
}
func TestCleanupDatabaseObservabilityDeletesRows(t *testing.T) {
cleanup := setupOptionTestDB(t)
defer cleanup()
ctx := context.Background()
resetAccessLogStore := repository.SetAccessLogStoreForTest(repository.NewMemoryAccessLogStore())
defer resetAccessLogStore()
now := time.Now().UTC()
require.NoError(t, repository.InsertOpenFlareAccessLogsBatch(ctx, []*model.OpenFlareAccessLog{
{
NodeID: "node-a",
LoggedAt: now.Add(-10 * 24 * time.Hour),
RemoteAddr: "203.0.113.1",
Host: "example.com",
Path: "/old",
StatusCode: 200,
},
{
NodeID: "node-a",
LoggedAt: now.Add(-2 * time.Hour),
RemoteAddr: "203.0.113.2",
Host: "example.com",
Path: "/recent",
StatusCode: 200,
},
}))
// Retention shorter than table TTL (90d for access logs) must be rejected.
shortRetention := 7
_, err := cleanupDatabaseObservability(ctx, databaseCleanupInput{
Target: "node_access_logs",
RetentionDays: &shortRetention,
})
require.Error(t, err)
// Full truncate still hard-deletes all rows.
result, err := cleanupDatabaseObservability(ctx, databaseCleanupInput{
Target: "node_access_logs",
})
require.NoError(t, err)
assert.Equal(t, "node_access_logs", result.Target)
assert.Equal(t, "访问日志", result.TargetLabel)
assert.Equal(t, int64(2), result.DeletedCount)
assert.True(t, result.DeleteAll)
assert.Equal(t, "truncate", result.CleanupMode)
rows, err := repository.ListOpenFlareAccessLogs(ctx, model.OpenFlareAccessLogQuery{Page: 0, PageSize: 10})
require.NoError(t, err)
assert.Empty(t, rows)
}
@@ -14,7 +14,10 @@ import (
openrestyrender "github.com/Rain-kl/Wavelet/pkg/render/openresty"
)
const maxOriginErrorPageHTMLBytes = 256 << 10 // 256 KiB
const (
maxOriginErrorPageHTMLBytes = 256 << 10 // 256 KiB
maxSWOfflineDomains = 1000
)
var openRestyOptionValidators = map[string]func(key, value string) error{
model.ConfigKeyOpenRestyDefaultServerReturnStatus: validateOpenRestyDefaultServerReturnStatus,
@@ -62,13 +65,16 @@ var openRestyOptionValidators = map[string]func(key, value string) error{
model.ConfigKeyOriginErrorPageStatusCodes: validateOriginErrorPageStatusCodes,
model.ConfigKeyOriginErrorPageHTML: validateOriginErrorPageHTML,
model.ConfigKeyOriginErrorPageGetOnly: validateBooleanOption,
model.ConfigKeySWOfflineEnabled: validateBooleanOption,
model.ConfigKeySWOfflineHTML: validateSWOfflineHTML,
model.ConfigKeySWOfflineDomains: validateSWOfflineDomains,
}
var openRestyDefaultLimitRatePattern = regexp.MustCompile(`^\d+[kKmM]?$`)
func validateOpenRestyOption(key, value string) error {
// HTML 按原始字节长度校验,避免 TrimSpace 影响上限判断
if key == model.ConfigKeyOriginErrorPageHTML {
if key == model.ConfigKeyOriginErrorPageHTML || key == model.ConfigKeySWOfflineHTML {
return validateOriginErrorPageHTML(key, value)
}
trimmed := strings.TrimSpace(value)
@@ -247,3 +253,32 @@ func validateOriginErrorPageHTML(key, value string) error {
}
return nil
}
func validateSWOfflineHTML(key, value string) error {
return validateOriginErrorPageHTML(key, value)
}
func validateSWOfflineDomains(key, value string) error {
var domains []string
if err := json.Unmarshal([]byte(value), &domains); err != nil || domains == nil {
return fmt.Errorf("%s 必须为 JSON 字符串数组", key)
}
if len(domains) > maxSWOfflineDomains {
return fmt.Errorf("%s 最多支持 %d 个域名", key, maxSWOfflineDomains)
}
seen := make(map[string]struct{}, len(domains))
for _, raw := range domains {
domain := strings.ToLower(strings.TrimSpace(raw))
if domain == "" {
return fmt.Errorf("%s 包含空域名", key)
}
if raw != domain {
return fmt.Errorf("%s 域名必须为小写且不含首尾空格:%s", key, raw)
}
if _, ok := seen[domain]; ok {
return fmt.Errorf("%s 包含重复域名 %s", key, domain)
}
seen[domain] = struct{}{}
}
return nil
}
@@ -4,6 +4,7 @@
package option
import (
"fmt"
"strings"
"testing"
@@ -94,3 +95,32 @@ func TestValidateOriginErrorPageEnabled(t *testing.T) {
require.Error(t, err)
assert.Contains(t, err.Error(), "true 或 false")
}
func TestValidateSWOfflineDomains(t *testing.T) {
cases := []struct {
name string
value string
ok bool
}{
{"empty array", `[]`, true},
{"single", `["example.com"]`, true},
{"multiple", `["example.com","api.example.com"]`, true},
{"invalid json", `not-json`, false},
{"null", "null", false},
{"empty element", `[""]`, false},
{"duplicate", `["example.com","example.com"]`, false},
{"whitespace dedup", `[" Example.com ","example.com"]`, false},
{"over limit", fmt.Sprintf(`[%s]`, strings.Repeat(`"a.com",`, maxSWOfflineDomains)+`"a.com"`), false},
}
for _, tc := range cases {
t.Run(tc.name, func(t *testing.T) {
err := validateSWOfflineDomains("sw_offline_domains", tc.value)
if tc.ok && err != nil {
t.Fatalf("want ok, got %v", err)
}
if !tc.ok && err == nil {
t.Fatal("want error, got nil")
}
})
}
}
-37
View File
@@ -4,9 +4,6 @@
package option
import (
"encoding/json"
"errors"
"io"
"net/http"
"github.com/Rain-kl/Wavelet/internal/apps/openflare/apiutil"
@@ -128,33 +125,6 @@ func LookupGeoIPHandler(c *gin.Context) {
c.JSON(http.StatusOK, response.OK(view))
}
// CleanupDatabaseHandler 清理可观测性数据库数据。
// @Summary 清理可观测性数据库
// @Description 按目标与保留天数清理可观测性相关数据表,需要管理员权限
// @Tags openflare-option
// @Accept json
// @Produce json
// @Security SessionCookie
// @Param request body option.databaseCleanupInput false "清理参数"
// @Success 200 {object} response.Any{data=option.databaseCleanupResult} "清理结果"
// @Failure 400 {object} response.Any "参数错误"
// @Failure 401 {object} response.Any "未登录"
// @Failure 404 {object} response.Any "无权限或不存在"
// @Failure 500 {object} response.Any "内部错误"
// @Router /api/v1/d/option/database/cleanup [post]
func CleanupDatabaseHandler(c *gin.Context) {
var input databaseCleanupInput
if err := bindOptionalJSON(c.Request.Body, &input); err != nil {
response.AbortBadRequest(c, errInvalidParams)
return
}
result, err := cleanupDatabaseObservability(c.Request.Context(), input)
if apiutil.AbortBadRequestOnError(c, err) {
return
}
c.JSON(http.StatusOK, response.OK(result))
}
// SyncUptimeKumaHandler 同步 Uptime Kuma 监控。
// @Summary 同步 Uptime Kuma
// @Description 将 OpenFlare 节点同步到 Uptime Kuma,需要管理员权限
@@ -174,10 +144,3 @@ func SyncUptimeKumaHandler(c *gin.Context) {
}
c.JSON(http.StatusOK, response.OK("同步成功"))
}
func bindOptionalJSON(body io.Reader, target any) error {
if err := json.NewDecoder(body).Decode(target); err != nil && !errors.Is(err, io.EOF) {
return err
}
return nil
}
+17 -5
View File
@@ -27,6 +27,17 @@ var (
const optionValueTrue = "true"
// protectedConfigKeyMessage 命中受保护 key 时返回给管理员的业务错误文案。
const protectedConfigKeyMessage = "该配置项由系统任务管理,禁止手动修改"
// protectedConfigKeys 仅允许内部(迁移任务/bootstrap)写入的 key。
var protectedConfigKeys = map[string]bool{
model.ConfigKeyLogDatabase: true,
model.ConfigKeyLogDBMigration: true,
}
func isProtectedConfigKey(key string) bool { return protectedConfigKeys[key] }
func buildOptionValidationState(ctx context.Context, options []model.OpenFlareOption) map[string]string {
// 从 SystemConfig 读取所有业务配置构建状态
configs, err := repository.ListAdminSystemConfigs(ctx, "business")
@@ -53,7 +64,7 @@ func validateOptionWithState(ctx context.Context, option model.OpenFlareOption,
if err := validateGeoIPOption(option.Key, option.Value); err != nil {
return err
}
if err := validateDatabaseCleanupOption(option.Key, option.Value); err != nil {
if err := validateLogRetentionOption(option.Key, option.Value); err != nil {
return err
}
if err := validateAgentOption(option.Key, option.Value); err != nil {
@@ -100,11 +111,9 @@ func validateGeoIPOption(key, value string) error {
return fmt.Errorf("%s 仅支持 disabled、mmdb、ip-api、geojs、ipinfo", key)
}
func validateDatabaseCleanupOption(key, value string) error {
func validateLogRetentionOption(key, value string) error {
switch key {
case model.ConfigKeyDatabaseAutoCleanupEnabled:
return validateBooleanOption(key, value)
case model.ConfigKeyDatabaseAutoCleanupRetentionDays:
case model.ConfigKeyLogRetentionDaysPostgres, model.ConfigKeyLogRetentionDaysSQLite, model.ConfigKeyLogRetentionDaysClickHouse:
intValue, err := strconv.Atoi(value)
if err != nil || intValue < 1 {
return fmt.Errorf("%s 必须为大于等于 1 的整数天", key)
@@ -221,6 +230,9 @@ func validateOptions(ctx context.Context, options []model.OpenFlareOption) error
if strings.TrimSpace(option.Key) == "" {
return errors.New(errInvalidParams)
}
if isProtectedConfigKey(option.Key) {
return errors.New(protectedConfigKeyMessage)
}
if err := validateOptionWithState(ctx, option, state); err != nil {
return err
}
+2 -2
View File
@@ -10,6 +10,7 @@ import (
"time"
"github.com/Rain-kl/Wavelet/internal/repository"
"github.com/Rain-kl/Wavelet/internal/testhelper"
"github.com/Rain-kl/Wavelet/internal/apps/openflare/agent"
db "github.com/Rain-kl/Wavelet/internal/infra/persistence"
@@ -38,10 +39,9 @@ func setupRelayTestDB(t *testing.T) func() {
db.SetDB(sqliteDB)
agent.ResetAuthCacheForTest()
resetObservabilityStore := repository.SetObservabilityStoreForTest(repository.NewMemoryObservabilityStore())
testhelper.SetupLogStoresForTest(t)
return func() {
resetObservabilityStore()
db.SetDB(nil)
agent.ResetAuthCacheForTest()
}
@@ -1,245 +0,0 @@
// Copyright 2026 Arctel.net
// SPDX-License-Identifier: Apache-2.0
package tasks
import (
"context"
"errors"
"fmt"
"strings"
"time"
"github.com/Rain-kl/Wavelet/internal/model"
"github.com/Rain-kl/Wavelet/internal/repository"
analyticsrepo "github.com/Rain-kl/Wavelet/internal/repository/analytics"
)
const (
// DatabaseCleanupTargetAccessLogs is the API cleanup target for access logs.
DatabaseCleanupTargetAccessLogs = "node_access_logs"
// DatabaseCleanupTargetMetricSnapshots is the API cleanup target for metric snapshots.
DatabaseCleanupTargetMetricSnapshots = "node_metric_snapshots"
// DatabaseCleanupTargetEdgeHealth is the API cleanup target for OpenResty edge health (connections).
DatabaseCleanupTargetEdgeHealth = "node_edge_health"
// DatabaseCleanupTargetObsFrps is the API cleanup target for FRPS observations.
DatabaseCleanupTargetObsFrps = "node_obs_frps"
// DatabaseCleanupTargetObsFrpc is the API cleanup target for FRPC observations.
DatabaseCleanupTargetObsFrpc = "node_obs_frpc"
)
var databaseCleanupTargets = map[string]string{
DatabaseCleanupTargetAccessLogs: "访问日志",
DatabaseCleanupTargetMetricSnapshots: "性能快照",
DatabaseCleanupTargetEdgeHealth: "OpenResty 健康(连接)",
DatabaseCleanupTargetObsFrps: "FRPS 观测",
DatabaseCleanupTargetObsFrpc: "FRPC 观测",
}
// databaseCleanupTableTTLDays maps API targets to ClickHouse DDL TTL days.
var databaseCleanupTableTTLDays = map[string]int{
DatabaseCleanupTargetAccessLogs: analyticsrepo.TableTTLDaysNodeAccessLogs,
DatabaseCleanupTargetMetricSnapshots: analyticsrepo.TableTTLDaysNodeMetricSnapshots,
DatabaseCleanupTargetEdgeHealth: analyticsrepo.TableTTLDaysNodeObs,
DatabaseCleanupTargetObsFrps: analyticsrepo.TableTTLDaysNodeObs,
DatabaseCleanupTargetObsFrpc: analyticsrepo.TableTTLDaysNodeObs,
}
// DatabaseCleanupInput describes a manual observability cleanup request.
type DatabaseCleanupInput struct {
Target string `json:"target"`
RetentionDays *int `json:"retention_days"`
}
// DatabaseCleanupResult summarizes a manual observability cleanup run.
//
// Semantics:
// - delete_all / cleanup_mode=truncate: DeletedCount is hard-deleted rows (TRUNCATE).
// - retention path / cleanup_mode=ttl_materialize: DeletedCount is always 0;
// EligibleCount estimates rows past the table DDL TTL (not an arbitrary younger cutoff).
type DatabaseCleanupResult struct {
Target string `json:"target"`
TargetLabel string `json:"target_label"`
DeletedCount int64 `json:"deleted_count"`
EligibleCount int64 `json:"eligible_count,omitempty"`
CleanupMode string `json:"cleanup_mode,omitempty"`
TableTTLDays int `json:"table_ttl_days,omitempty"`
DeleteAll bool `json:"delete_all"`
RetentionDays *int `json:"retention_days,omitempty"`
Cutoff *time.Time `json:"cutoff,omitempty"`
}
// DatabaseAutoCleanupSummary summarizes a scheduled auto-cleanup run.
type DatabaseAutoCleanupSummary struct {
RetentionDays int `json:"retention_days"`
ExecutedAt time.Time `json:"executed_at"`
Results []DatabaseCleanupResult `json:"results"`
}
// TableTTLDaysForCleanupTarget returns the DDL TTL days for a cleanup target.
func TableTTLDaysForCleanupTarget(target string) (int, bool) {
days, ok := databaseCleanupTableTTLDays[strings.TrimSpace(target)]
return days, ok
}
// CleanupDatabaseObservability deletes observability rows for the given target.
//
// When RetentionDays is nil, rows are hard-deleted via TRUNCATE.
// When RetentionDays is set, ClickHouse only force-materializes the table TTL policy:
// retention_days shorter than the table TTL is rejected (do not fake success).
func CleanupDatabaseObservability(ctx context.Context, input DatabaseCleanupInput) (*DatabaseCleanupResult, error) {
target := strings.TrimSpace(input.Target)
targetLabel, ok := databaseCleanupTargets[target]
if !ok {
return nil, errors.New("unsupported cleanup target")
}
if input.RetentionDays != nil && *input.RetentionDays <= 0 {
return nil, errors.New("retention_days 必须为大于 0 的整数")
}
tableTTLDays := databaseCleanupTableTTLDays[target]
result := &DatabaseCleanupResult{
Target: target,
TargetLabel: targetLabel,
DeleteAll: input.RetentionDays == nil,
TableTTLDays: tableTTLDays,
}
if input.RetentionDays == nil {
deleted, mode, err := deleteAllObservabilityRows(ctx, target)
if err != nil {
return nil, err
}
result.DeletedCount = deleted
result.EligibleCount = deleted
result.CleanupMode = mode
return result, nil
}
retentionDays := *input.RetentionDays
if retentionDays < tableTTLDays {
return nil, fmt.Errorf(
"retention_days 不能小于表 TTL(%d 天);ClickHouse 仅支持按表 TTL 物化过期,更短保留请使用清空全部或调整 DDL",
tableTTLDays,
)
}
// MATERIALIZE TTL only enforces DDL policy; cutoff reported is the table TTL boundary.
tableCutoff := time.Now().UTC().Add(-time.Duration(tableTTLDays) * 24 * time.Hour)
eligible, mode, err := materializeObservabilityTableTTL(ctx, target)
if err != nil {
return nil, err
}
result.DeletedCount = 0
result.EligibleCount = eligible
result.CleanupMode = mode
result.RetentionDays = &retentionDays
result.Cutoff = &tableCutoff
return result, nil
}
// RunDatabaseAutoCleanupOnce runs retention-based cleanup for all observability targets.
//
// Configured retention shorter than a target's table TTL is clamped up to the table TTL
// so the scheduled job can force-materialize each table policy without failing.
func RunDatabaseAutoCleanupOnce(ctx context.Context, now time.Time) (*DatabaseAutoCleanupSummary, error) {
enabled, err := repository.GetBoolByKey(ctx, model.ConfigKeyDatabaseAutoCleanupEnabled)
if err != nil {
return nil, fmt.Errorf("failed to read database_auto_cleanup_enabled: %w", err)
}
if !enabled {
return nil, nil
}
retentionDays, err := repository.GetIntByKey(ctx, model.ConfigKeyDatabaseAutoCleanupRetentionDays)
if err != nil || retentionDays <= 0 {
// Use default value 30 if config read fails or value is invalid
retentionDays = 30
}
results := make([]DatabaseCleanupResult, 0, len(databaseCleanupTargets))
for _, target := range []string{
DatabaseCleanupTargetAccessLogs,
DatabaseCleanupTargetMetricSnapshots,
DatabaseCleanupTargetEdgeHealth,
DatabaseCleanupTargetObsFrps,
DatabaseCleanupTargetObsFrpc,
} {
effectiveDays := retentionDays
if ttl, ok := databaseCleanupTableTTLDays[target]; ok && effectiveDays < ttl {
effectiveDays = ttl
}
result, err := CleanupDatabaseObservability(ctx, DatabaseCleanupInput{
Target: target,
RetentionDays: &effectiveDays,
})
if err != nil {
return nil, err
}
results = append(results, *result)
}
return &DatabaseAutoCleanupSummary{
RetentionDays: retentionDays,
ExecutedAt: now.UTC(),
Results: results,
}, nil
}
func deleteAllObservabilityRows(ctx context.Context, target string) (int64, string, error) {
var (
deleted int64
err error
)
switch target {
case DatabaseCleanupTargetAccessLogs:
deleted, err = repository.DeleteAllOpenFlareAccessLogs(ctx)
case DatabaseCleanupTargetMetricSnapshots:
deleted, err = repository.DeleteAllOpenFlareMetricSnapshots(ctx)
case DatabaseCleanupTargetEdgeHealth:
deleted, err = repository.DeleteAllOpenFlareEdgeHealth(ctx)
case DatabaseCleanupTargetObsFrps:
deleted, err = repository.DeleteAllOpenFlareNodeObservationFrps(ctx)
case DatabaseCleanupTargetObsFrpc:
deleted, err = repository.DeleteAllOpenFlareNodeObservationFrpc(ctx)
default:
return 0, "", errors.New("unsupported cleanup target")
}
if err != nil {
return 0, "", err
}
return deleted, analyticsrepo.CleanupModeTruncate, nil
}
// materializeObservabilityTableTTL triggers table-TTL materialize (or memory-store delete-before
// with the table TTL cutoff for tests) and returns the eligible/estimate row count.
func materializeObservabilityTableTTL(ctx context.Context, target string) (int64, string, error) {
ttlDays, ok := databaseCleanupTableTTLDays[target]
if !ok {
return 0, "", errors.New("unsupported cleanup target")
}
cutoff := time.Now().UTC().Add(-time.Duration(ttlDays) * 24 * time.Hour)
var (
eligible int64
err error
)
switch target {
case DatabaseCleanupTargetAccessLogs:
eligible, err = repository.DeleteOpenFlareAccessLogsBefore(ctx, cutoff)
case DatabaseCleanupTargetMetricSnapshots:
eligible, err = repository.DeleteOpenFlareMetricSnapshotsBefore(ctx, cutoff)
case DatabaseCleanupTargetEdgeHealth:
eligible, err = repository.DeleteOpenFlareEdgeHealthBefore(ctx, cutoff)
case DatabaseCleanupTargetObsFrps:
eligible, err = repository.DeleteOpenFlareNodeObservationFrpsBefore(ctx, cutoff)
case DatabaseCleanupTargetObsFrpc:
eligible, err = repository.DeleteOpenFlareNodeObservationFrpcBefore(ctx, cutoff)
default:
return 0, "", errors.New("unsupported cleanup target")
}
if err != nil {
return 0, "", err
}
return eligible, analyticsrepo.CleanupModeTTLMaterialize, nil
}
@@ -1,208 +0,0 @@
// Copyright 2026 Arctel.net
// SPDX-License-Identifier: Apache-2.0
package tasks
import (
"context"
"testing"
"time"
db "github.com/Rain-kl/Wavelet/internal/infra/persistence"
"github.com/Rain-kl/Wavelet/internal/model"
"github.com/Rain-kl/Wavelet/internal/repository"
analyticsrepo "github.com/Rain-kl/Wavelet/internal/repository/analytics"
"github.com/glebarez/sqlite"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
"gorm.io/gorm"
)
func setupDatabaseCleanupTestDB(t *testing.T) context.Context {
t.Helper()
sqliteDB, err := gorm.Open(sqlite.Open(":memory:"), &gorm.Config{
DisableForeignKeyConstraintWhenMigrating: true,
})
require.NoError(t, err)
require.NoError(t, sqliteDB.AutoMigrate(&model.SystemConfig{}))
db.SetDB(sqliteDB)
resetAccessLogStore := repository.SetAccessLogStoreForTest(repository.NewMemoryAccessLogStore())
resetObservabilityStore := repository.SetObservabilityStoreForTest(repository.NewMemoryObservabilityStore())
t.Cleanup(func() {
resetObservabilityStore()
resetAccessLogStore()
db.SetDB(nil)
})
return context.Background()
}
func TestCleanupDatabaseObservabilityRejectsRetentionShorterThanTableTTL(t *testing.T) {
ctx := setupDatabaseCleanupTestDB(t)
retentionDays := 7 // metric snapshots DDL TTL is 30 days
result, err := CleanupDatabaseObservability(ctx, DatabaseCleanupInput{
Target: DatabaseCleanupTargetMetricSnapshots,
RetentionDays: &retentionDays,
})
require.Error(t, err)
assert.Nil(t, result)
assert.Contains(t, err.Error(), "不能小于表 TTL")
assert.Contains(t, err.Error(), "30")
}
func TestCleanupDatabaseObservabilityRejectsAccessLogRetentionShorterThanTableTTL(t *testing.T) {
ctx := setupDatabaseCleanupTestDB(t)
retentionDays := 30 // access logs DDL TTL is 90 days
result, err := CleanupDatabaseObservability(ctx, DatabaseCleanupInput{
Target: DatabaseCleanupTargetAccessLogs,
RetentionDays: &retentionDays,
})
require.Error(t, err)
assert.Nil(t, result)
assert.Contains(t, err.Error(), "90")
}
func TestCleanupDatabaseObservabilityMaterializeDoesNotClaimHardDelete(t *testing.T) {
ctx := setupDatabaseCleanupTestDB(t)
now := time.Now().UTC()
// One row past metric table TTL (30d), one still inside the window.
require.NoError(t, repository.InsertOpenFlareMetricSnapshot(ctx, &model.OpenFlareMetricSnapshot{
NodeID: "node-a",
CapturedAt: now.Add(-40 * 24 * time.Hour),
CPUUsagePercent: 10,
}))
require.NoError(t, repository.InsertOpenFlareMetricSnapshot(ctx, &model.OpenFlareMetricSnapshot{
NodeID: "node-a",
CapturedAt: now.Add(-12 * time.Hour),
CPUUsagePercent: 20,
}))
retentionDays := analyticsrepo.TableTTLDaysNodeMetricSnapshots
result, err := CleanupDatabaseObservability(ctx, DatabaseCleanupInput{
Target: DatabaseCleanupTargetMetricSnapshots,
RetentionDays: &retentionDays,
})
require.NoError(t, err)
assert.False(t, result.DeleteAll)
assert.Equal(t, analyticsrepo.CleanupModeTTLMaterialize, result.CleanupMode)
assert.Equal(t, analyticsrepo.TableTTLDaysNodeMetricSnapshots, result.TableTTLDays)
// MATERIALIZE is not a counted hard delete.
assert.Equal(t, int64(0), result.DeletedCount)
assert.Equal(t, int64(1), result.EligibleCount)
require.NotNil(t, result.Cutoff)
assert.True(t, result.Cutoff.Before(now.Add(-29*24*time.Hour)))
// Memory store applies the table-TTL cutoff for tests; only the recent row remains.
rows, err := repository.ListOpenFlareMetricSnapshotsSince(ctx, "", time.Time{}, 0)
require.NoError(t, err)
require.Len(t, rows, 1)
assert.Equal(t, float64(20), rows[0].CPUUsagePercent)
}
func TestCleanupDatabaseObservabilityDeletesAllRowsWhenRetentionMissing(t *testing.T) {
ctx := setupDatabaseCleanupTestDB(t)
now := time.Now().UTC()
require.NoError(t, repository.InsertOpenFlareAccessLogsBatch(ctx, []*model.OpenFlareAccessLog{
{
NodeID: "node-a",
LoggedAt: now.Add(-3 * time.Hour),
RemoteAddr: "203.0.113.1",
Host: "example.com",
Path: "/one",
StatusCode: 200,
},
{
NodeID: "node-a",
LoggedAt: now.Add(-2 * time.Hour),
RemoteAddr: "203.0.113.2",
Host: "example.com",
Path: "/two",
StatusCode: 502,
},
}))
result, err := CleanupDatabaseObservability(ctx, DatabaseCleanupInput{
Target: DatabaseCleanupTargetAccessLogs,
})
require.NoError(t, err)
assert.True(t, result.DeleteAll)
assert.Equal(t, analyticsrepo.CleanupModeTruncate, result.CleanupMode)
assert.Equal(t, int64(2), result.DeletedCount)
assert.Equal(t, int64(2), result.EligibleCount)
rows, err := repository.ListOpenFlareAccessLogs(ctx, model.OpenFlareAccessLogQuery{Page: 0, PageSize: 10})
require.NoError(t, err)
assert.Empty(t, rows)
}
func TestRunDatabaseAutoCleanupOnceClampsRetentionToTableTTL(t *testing.T) {
ctx := setupDatabaseCleanupTestDB(t)
now := time.Now().UTC()
// Access logs TTL=90d, metrics TTL=30d. Config retention=1 must clamp, not reject.
require.NoError(t, repository.InsertOpenFlareAccessLogsBatch(ctx, []*model.OpenFlareAccessLog{{
NodeID: "node-a",
LoggedAt: now.Add(-100 * 24 * time.Hour),
RemoteAddr: "203.0.113.10",
Host: "example.com",
Path: "/access",
StatusCode: 200,
}}))
require.NoError(t, repository.InsertOpenFlareMetricSnapshot(ctx, &model.OpenFlareMetricSnapshot{
NodeID: "node-a",
CapturedAt: now.Add(-40 * 24 * time.Hour),
CPUUsagePercent: 10,
}))
require.NoError(t, repository.InsertOpenFlareEdgeHealth(ctx, &model.OpenFlareEdgeHealth{
NodeID: "node-a",
CapturedAt: now.Add(-40 * 24 * time.Hour),
Status: "healthy",
Connections: 2,
}))
require.NoError(t, repository.SaveOrUpdateSystemConfig(ctx, model.ConfigKeyDatabaseAutoCleanupEnabled, "true"))
require.NoError(t, repository.SaveOrUpdateSystemConfig(ctx, model.ConfigKeyDatabaseAutoCleanupRetentionDays, "1"))
summary, err := RunDatabaseAutoCleanupOnce(ctx, now)
require.NoError(t, err)
require.NotNil(t, summary)
require.Len(t, summary.Results, 5)
assert.Equal(t, 1, summary.RetentionDays)
for _, result := range summary.Results {
assert.Equal(t, analyticsrepo.CleanupModeTTLMaterialize, result.CleanupMode)
assert.Equal(t, int64(0), result.DeletedCount, "target %s must not claim hard delete", result.Target)
assert.GreaterOrEqual(t, result.TableTTLDays, 30)
require.NotNil(t, result.RetentionDays)
assert.GreaterOrEqual(t, *result.RetentionDays, result.TableTTLDays)
}
accessLogs, err := repository.ListOpenFlareAccessLogs(ctx, model.OpenFlareAccessLogQuery{Page: 0, PageSize: 10})
require.NoError(t, err)
assert.Empty(t, accessLogs)
metricSnapshots, err := repository.ListOpenFlareMetricSnapshotsSince(ctx, "", time.Time{}, 0)
require.NoError(t, err)
assert.Empty(t, metricSnapshots)
edgeHealth, err := repository.ListOpenFlareEdgeHealth(ctx, "", time.Time{}, 0)
require.NoError(t, err)
assert.Empty(t, edgeHealth)
}
func TestTableTTLDaysForCleanupTarget(t *testing.T) {
days, ok := TableTTLDaysForCleanupTarget(DatabaseCleanupTargetAccessLogs)
require.True(t, ok)
assert.Equal(t, 90, days)
days, ok = TableTTLDaysForCleanupTarget(DatabaseCleanupTargetMetricSnapshots)
require.True(t, ok)
assert.Equal(t, 30, days)
_, ok = TableTTLDaysForCleanupTarget("unknown")
assert.False(t, ok)
}
@@ -0,0 +1,398 @@
// Copyright 2026 Arctel.net
// SPDX-License-Identifier: Apache-2.0
package tasks
import (
"context"
"encoding/json"
"errors"
"fmt"
"time"
"github.com/Rain-kl/Wavelet/internal/apps/openflare/chwriter"
"github.com/Rain-kl/Wavelet/internal/infra/config"
"github.com/Rain-kl/Wavelet/internal/infra/task"
"github.com/Rain-kl/Wavelet/internal/model"
analyticsmodel "github.com/Rain-kl/Wavelet/internal/model/analytics"
"github.com/Rain-kl/Wavelet/internal/repository"
"github.com/Rain-kl/Wavelet/internal/repository/logstore"
"github.com/Rain-kl/Wavelet/pkg/logger"
)
const copyBatchSize = 1000
// 迁移目标库名常量(normalizeTarget 归一化后的取值)。
const (
targetPostgres = "postgres"
targetSQLite = "sqlite"
targetClickHouse = "clickhouse"
)
type logDBSwitchPayload struct {
Target string `json:"target"`
}
// LogDBSwitchHandler 切换日志数据库任务处理器。
type LogDBSwitchHandler struct{}
// ValidatePayload 校验并规范化参数。
func (h *LogDBSwitchHandler) ValidatePayload(payload []byte) ([]byte, error) {
var p logDBSwitchPayload
if err := json.Unmarshal(payload, &p); err != nil {
return nil, fmt.Errorf("参数解析失败: %w", err)
}
p.Target = normalizeTarget(p.Target)
if !validTarget(p.Target) {
return nil, fmt.Errorf("目标日志库不合法: %s", p.Target)
}
out, err := json.Marshal(p)
if err != nil {
return nil, err
}
return out, nil
}
func normalizeTarget(v string) string {
switch v {
case targetPostgres, "postgresql":
return targetPostgres
case targetSQLite, "sqlite3":
return targetSQLite
case targetClickHouse, "ch":
return targetClickHouse
}
return v
}
func validTarget(v string) bool {
return v == targetPostgres || v == targetSQLite || v == targetClickHouse
}
// Execute 执行迁移。
func (h *LogDBSwitchHandler) Execute(ctx context.Context, payload []byte) (*task.TaskResult, error) {
var p logDBSwitchPayload
if err := json.Unmarshal(payload, &p); err != nil {
return nil, fmt.Errorf("参数解析失败: %w", err)
}
p.Target = normalizeTarget(p.Target)
if err := validateSwitch(ctx, p.Target); err != nil {
return nil, err
}
source, err := currentLogDatabase(ctx)
if err != nil {
task.AppendLog(ctx, "读取日志主库失败: %v", err)
return nil, err
}
task.AppendLog(ctx, "开始切换日志数据库:%s -> %s", source, p.Target)
// 设置迁移冻结标记(置位后由 ensureWritable 拒绝新写入)。
if err := setMigrationFlag(ctx, "migrating"); err != nil {
return nil, err
}
// 失败也清除(SaveOrUpdateSystemConfig 会失效 RAM 缓存并广播),保持源库可写。
defer func() {
if err := setMigrationFlag(ctx, ""); err != nil {
logger.ErrorF(ctx, "清除日志迁移冻结标记失败: %v", err)
}
}()
// 冻结标记置位后再排空在途批次(chwriter + 用户访问日志 writer),
// 保证排空完成后不再有新批次进入源库。
if err := drainLogWriters(ctx); err != nil {
return nil, fmt.Errorf("排空日志写入队列失败: %w", err)
}
src, err := logstore.Active(ctx)
if err != nil {
return nil, err
}
dst, err := buildTargetStore(ctx, p.Target)
if err != nil {
return nil, err
}
// 清空目标库日志表(幂等重试前提)。
if err := clearTargetLogTables(ctx, dst); err != nil {
return nil, err
}
// PG 目标:按源库时间范围预建分区,避免历史数据复制报 "no partition of relation found"。
if err := ensureTargetPartitions(ctx, src, dst, p.Target); err != nil {
return nil, err
}
// 逐表复制(6 张日志表)。
if err := copyAccessLogs(ctx, src, dst); err != nil {
return nil, err
}
if err := copyUserAccessLogs(ctx, src, dst); err != nil {
return nil, err
}
if err := copyObservability(ctx, src, dst); err != nil {
return nil, err
}
// 翻转主库标记。
if err := flipLogDatabase(ctx, p.Target); err != nil {
return nil, err
}
task.AppendLog(ctx, "日志数据库已切换为 %s,写入恢复", p.Target)
return &task.TaskResult{Message: fmt.Sprintf("日志数据库已从 %s 切换为 %s", source, p.Target)}, nil
}
func validateSwitch(ctx context.Context, target string) error {
source, err := currentLogDatabase(ctx)
if err != nil {
return err
}
if source == target {
return errors.New("目标日志库与当前日志库相同,无需迁移")
}
switch target {
case "clickhouse":
if !config.Config.ClickHouse.Enabled {
return errors.New("ClickHouse 未启用,无法迁移到 ClickHouse")
}
case "postgres":
if !config.Config.Database.Enabled {
return errors.New("PostgreSQL 未启用(当前主库为 SQLite),无法迁移到 PostgreSQL")
}
case "sqlite":
if config.Config.Database.Enabled {
return errors.New("当前主库为 PostgreSQL,日志库不能设置为 SQLite")
}
}
return nil
}
func currentLogDatabase(ctx context.Context) (string, error) {
cfg, err := repository.GetSystemConfigByKey(ctx, model.ConfigKeyLogDatabase)
if err != nil {
return "", fmt.Errorf("读取日志主库失败: %w", err)
}
if cfg.Value == "" {
return "", errors.New("日志主库配置为空")
}
return cfg.Value, nil
}
// drainLogWriters 等待 chwriter(节点访问日志 + 可观测 4 表)的在途批次全部落库。
// 见设计 §7.2:先排空再冻结。用户访问日志(w_user_access_logs)记录已禁用,无在途批次。
func drainLogWriters(ctx context.Context) error {
return chwriter.Drain(ctx)
}
// setMigrationFlag 写入迁移冻结标记。用 SaveOrUpdateSystemConfig:行缺失时 upsert,
// 并失效 RAM 缓存 + 广播其他节点,保证 logstore.Migrating/resolveDatabase 立即生效。
func setMigrationFlag(ctx context.Context, v string) error {
return repository.SaveOrUpdateSystemConfig(ctx, model.ConfigKeyLogDBMigration, v)
}
// flipLogDatabase 翻转日志主库。同上用 SaveOrUpdateSystemConfig,确保各进程缓存失效后指向新库。
func flipLogDatabase(ctx context.Context, target string) error {
return repository.SaveOrUpdateSystemConfig(ctx, model.ConfigKeyLogDatabase, target)
}
// buildTargetStore 构造目标库 Store(不经过 Active 缓存,直接 Build)。
// 迁移期间冻结标记已置位,目标库的清空/复制写入必须放行,故使用 BuildForMigration。
func buildTargetStore(ctx context.Context, database string) (*logstore.Store, error) {
return logstore.BuildForMigration(ctx, database)
}
func clearTargetLogTables(ctx context.Context, dst *logstore.Store) error {
// 依次清空 6 张表:AccessLogs.DeleteAll、UserAccessLogs.DeleteAll、Observability.DeleteAll*
// (SQLite/PG 用 DeleteAll;CH 用 TRUNCATE 语义)。
if _, err := dst.AccessLogs.DeleteAll(ctx); err != nil {
return fmt.Errorf("清空目标访问日志失败: %w", err)
}
if _, err := dst.UserAccessLogs.DeleteAll(ctx); err != nil {
return fmt.Errorf("清空目标用户访问日志失败: %w", err)
}
for _, fn := range []func(context.Context) (int64, error){
dst.Observability.DeleteAllMetricSnapshots,
dst.Observability.DeleteAllEdgeHealth,
dst.Observability.DeleteAllNodeObservationFrps,
dst.Observability.DeleteAllNodeObservationFrpc,
} {
if _, err := fn(ctx); err != nil {
return err
}
}
return nil
}
// ensureTargetPartitions 目标为 PG 时,按源库时间范围(两表合并)预建分区,
// 否则复制历史数据会报 "no partition of relation found";目标非 PG 为 no-op。
func ensureTargetPartitions(ctx context.Context, src, dst *logstore.Store, target string) error {
if target != targetPostgres {
return nil
}
from, to, err := migrationRange(ctx, src)
if err != nil {
return err
}
if from.IsZero() || to.IsZero() {
task.AppendLog(ctx, "源库无日志数据,跳过分区预建")
return nil
}
if err := dst.AccessLogs.EnsurePartitions(ctx, from, to.AddDate(0, 1, 0)); err != nil {
return fmt.Errorf("预建目标 PG 分区失败: %w", err)
}
task.AppendLog(ctx, "已为目标 PG 预建分区 %s ~ %s", from.Format("2006-01"), to.Format("2006-01"))
return nil
}
// migrationRange 合并源库节点访问日志(logged_at)与用户访问日志(created_at)
// 的最小/最大时间;任一表为空时忽略该表。
func migrationRange(ctx context.Context, src *logstore.Store) (time.Time, time.Time, error) {
fromAccess, toAccess, err := src.AccessLogs.MigrationRange(ctx)
if err != nil {
return time.Time{}, time.Time{}, fmt.Errorf("读取源访问日志时间范围失败: %w", err)
}
fromUser, toUser, err := src.UserAccessLogs.MigrationRange(ctx)
if err != nil {
return time.Time{}, time.Time{}, fmt.Errorf("读取源用户访问日志时间范围失败: %w", err)
}
return minTime(fromAccess, fromUser), maxTime(toAccess, toUser), nil
}
func minTime(a, b time.Time) time.Time {
switch {
case a.IsZero():
return b
case b.IsZero():
return a
case a.Before(b):
return a
default:
return b
}
}
func maxTime(a, b time.Time) time.Time {
switch {
case a.IsZero():
return b
case b.IsZero():
return a
case a.After(b):
return a
default:
return b
}
}
// copyAccessLogs 从 src 复制节点访问日志到 dst。
func copyAccessLogs(ctx context.Context, src, dst *logstore.Store) error {
// 注意:迁移期间 src 已冻结,但复制读取不受冻结影响;每批按 id 升序扫描。
var lastID uint64
for {
rows, err := listNodeAccessLogsByID(ctx, src, lastID, copyBatchSize)
if err != nil {
return err
}
if len(rows) == 0 {
break
}
if err := dst.AccessLogs.BatchInsertNodeAccessLogs(ctx, rows); err != nil {
return fmt.Errorf("写入目标访问日志失败(批 %d): %w", lastID, err)
}
task.AppendLog(ctx, "已复制访问日志 %d 条(截至 id=%d)", len(rows), rows[len(rows)-1].ID)
lastID = rows[len(rows)-1].ID
if len(rows) < copyBatchSize {
break
}
}
return nil
}
func listNodeAccessLogsByID(ctx context.Context, src *logstore.Store, afterID uint64, limit int) ([]analyticsmodel.NodeAccessLog, error) {
return src.AccessLogs.ListForMigration(ctx, afterID, limit)
}
// copyUserAccessLogs 从 src 复制用户访问日志到 dst(按 id 升序分批)。
func copyUserAccessLogs(ctx context.Context, src, dst *logstore.Store) error {
var lastID uint64
for {
rows, err := src.UserAccessLogs.ListForMigration(ctx, lastID, copyBatchSize)
if err != nil {
return err
}
if len(rows) == 0 {
return nil
}
if err := dst.UserAccessLogs.BatchInsert(ctx, rows); err != nil {
return fmt.Errorf("写入目标用户访问日志失败(批 %d): %w", lastID, err)
}
lastID = rows[len(rows)-1].ID
task.AppendLog(ctx, "已复制用户访问日志 %d 条(截至 id=%d)", len(rows), lastID)
if len(rows) < copyBatchSize {
return nil
}
}
}
// copyObservability 复制 4 张可观测表,每张表按 id 升序分批复制,
// 以每批最后一条 id 作为下一批游标(不使用 len 近似)。
func copyObservability(ctx context.Context, src, dst *logstore.Store) error {
if err := copyObsTable(ctx, "metric_snapshots",
src.Observability.ListMetricSnapshotsForMigration,
dst.Observability.BatchInsertNodeMetricSnapshots,
lastMetricSnapshotID); err != nil {
return err
}
if err := copyObsTable(ctx, "edge_health",
src.Observability.ListEdgeHealthForMigration,
dst.Observability.BatchInsertNodeEdgeHealth,
lastEdgeHealthID); err != nil {
return err
}
if err := copyObsTable(ctx, "obs_frps",
src.Observability.ListNodeObsFrpsForMigration,
dst.Observability.BatchInsertNodeObsFrps,
lastObsFrpsID); err != nil {
return err
}
if err := copyObsTable(ctx, "obs_frpc",
src.Observability.ListNodeObsFrpcForMigration,
dst.Observability.BatchInsertNodeObsFrpc,
lastObsFrpcID); err != nil {
return err
}
return nil
}
// copyObsTable 按 id 升序分批复制单张可观测表;idOf 返回批内最后一条 id。
func copyObsTable[T any](ctx context.Context, name string,
list func(context.Context, uint64, int) ([]T, error),
insert func(context.Context, []T) error,
idOf func([]T) uint64,
) error {
var lastID uint64
for {
rows, err := list(ctx, lastID, copyBatchSize)
if err != nil {
return fmt.Errorf("复制 %s 失败: %w", name, err)
}
if len(rows) == 0 {
return nil
}
if err := insert(ctx, rows); err != nil {
return fmt.Errorf("复制 %s 失败: %w", name, err)
}
lastID = idOf(rows)
task.AppendLog(ctx, "已复制 %s %d 条(截至 id=%d)", name, len(rows), lastID)
if len(rows) < copyBatchSize {
return nil
}
}
}
func lastMetricSnapshotID(rows []analyticsmodel.NodeMetricSnapshot) uint64 {
return rows[len(rows)-1].ID
}
func lastEdgeHealthID(rows []analyticsmodel.NodeEdgeHealth) uint64 { return rows[len(rows)-1].ID }
func lastObsFrpsID(rows []analyticsmodel.NodeObsFrps) uint64 { return rows[len(rows)-1].ID }
func lastObsFrpcID(rows []analyticsmodel.NodeObsFrpc) uint64 { return rows[len(rows)-1].ID }
@@ -0,0 +1,397 @@
// Copyright 2026 Arctel.net
// SPDX-License-Identifier: Apache-2.0
package tasks
import (
"context"
"encoding/json"
"errors"
"fmt"
"sync/atomic"
"testing"
"time"
"github.com/glebarez/sqlite"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
"gorm.io/gorm"
"gorm.io/gorm/logger"
"github.com/Rain-kl/Wavelet/internal/infra/config"
db "github.com/Rain-kl/Wavelet/internal/infra/persistence"
"github.com/Rain-kl/Wavelet/internal/model"
analyticsmodel "github.com/Rain-kl/Wavelet/internal/model/analytics"
"github.com/Rain-kl/Wavelet/internal/repository"
"github.com/Rain-kl/Wavelet/internal/repository/logstore"
)
var logDBSwitchDBSeq int64
// newLogDBSwitchDB 构造内存 sqlite 库(含日志 5 表 + 系统配置表)。
func newLogDBSwitchDB(t *testing.T) *gorm.DB {
t.Helper()
dsn := fmt.Sprintf("file:log-db-switch-%d?mode=memory&cache=shared", atomic.AddInt64(&logDBSwitchDBSeq, 1))
gdb, err := gorm.Open(sqlite.Open(dsn), &gorm.Config{Logger: logger.Default.LogMode(logger.Silent)})
require.NoError(t, err)
require.NoError(t, gdb.AutoMigrate(
&model.SystemConfig{},
&analyticsmodel.NodeAccessLog{},
&analyticsmodel.NodeMetricSnapshot{},
&analyticsmodel.NodeEdgeHealth{},
&analyticsmodel.NodeObsFrps{},
&analyticsmodel.NodeObsFrpc{},
&analyticsmodel.UserAccessLog{},
))
return gdb
}
// TestCopyAccessLogsPreservesIDs sqlite→sqlite 模拟:源 store 3 条,目标空库,
// copyAccessLogs 后 ID 保留、数量一致。
func TestCopyAccessLogsPreservesIDs(t *testing.T) {
oldDB, oldCH := config.Config.Database.Enabled, config.Config.ClickHouse.Enabled
config.Config.Database.Enabled, config.Config.ClickHouse.Enabled = false, false
t.Cleanup(func() {
config.Config.Database.Enabled, config.Config.ClickHouse.Enabled = oldDB, oldCH
})
logstore.ResetForTest()
defer logstore.ResetForTest()
ctx := context.Background()
srcDB := newLogDBSwitchDB(t)
dstDB := newLogDBSwitchDB(t)
db.SetDB(srcDB)
src, err := logstore.Active(ctx) // 无 reader 时按 seed 规则解析为 sqlite
require.NoError(t, err)
db.SetDB(dstDB)
dst, err := logstore.BuildForMigration(ctx, "sqlite")
require.NoError(t, err)
t.Cleanup(func() { db.SetDB(nil) })
now := time.Now().UTC()
rows := []analyticsmodel.NodeAccessLog{
{ID: 101, NodeID: "n1", LoggedAt: now, RemoteAddr: "1.1.1.1", Host: "a.example.com", Path: "/"},
{ID: 202, NodeID: "n2", LoggedAt: now, RemoteAddr: "2.2.2.2", Host: "b.example.com", Path: "/x"},
{ID: 303, NodeID: "n1", LoggedAt: now, RemoteAddr: "3.3.3.3", Host: "c.example.com", Path: "/y"},
}
require.NoError(t, src.AccessLogs.BatchInsertNodeAccessLogs(ctx, rows))
require.NoError(t, copyAccessLogs(ctx, src, dst))
var got []analyticsmodel.NodeAccessLog
require.NoError(t, dstDB.Order("id ASC").Find(&got).Error)
require.Len(t, got, 3)
for i, wantID := range []uint64{101, 202, 303} {
assert.Equal(t, wantID, got[i].ID, "row %d id preserved", i)
}
assert.Equal(t, "n1", got[0].NodeID)
assert.Equal(t, "n2", got[1].NodeID)
assert.Equal(t, "n1", got[2].NodeID)
assert.Equal(t, "1.1.1.1", got[0].RemoteAddr)
// 源库保持不变。
var srcCount int64
require.NoError(t, srcDB.Model(&analyticsmodel.NodeAccessLog{}).Count(&srcCount).Error)
assert.Equal(t, int64(3), srcCount)
}
// TestCopyUserAccessLogsPreservesIDs sqlite→sqlite 模拟:源库用户访问日志按 id 升序
// 复制到目标库,ID 保留、数量一致,且源库保持不变。
func TestCopyUserAccessLogsPreservesIDs(t *testing.T) {
oldDB, oldCH := config.Config.Database.Enabled, config.Config.ClickHouse.Enabled
config.Config.Database.Enabled, config.Config.ClickHouse.Enabled = false, false
t.Cleanup(func() {
config.Config.Database.Enabled, config.Config.ClickHouse.Enabled = oldDB, oldCH
})
logstore.ResetForTest()
defer logstore.ResetForTest()
ctx := context.Background()
srcDB := newLogDBSwitchDB(t)
dstDB := newLogDBSwitchDB(t)
db.SetDB(srcDB)
src, err := logstore.Active(ctx)
require.NoError(t, err)
db.SetDB(dstDB)
dst, err := logstore.BuildForMigration(ctx, "sqlite")
require.NoError(t, err)
t.Cleanup(func() { db.SetDB(nil) })
now := time.Now().UTC()
rows := []analyticsmodel.UserAccessLog{
{ID: 11, UserID: 1, Path: "/a", CreatedAt: now},
{ID: 22, UserID: 2, Path: "/b", CreatedAt: now.Add(time.Second)},
{ID: 33, UserID: 1, Path: "/c", CreatedAt: now.Add(2 * time.Second)},
}
require.NoError(t, src.UserAccessLogs.BatchInsert(ctx, rows))
require.NoError(t, copyUserAccessLogs(ctx, src, dst))
var got []analyticsmodel.UserAccessLog
require.NoError(t, dstDB.Order("id ASC").Find(&got).Error)
require.Len(t, got, 3)
for i, wantID := range []uint64{11, 22, 33} {
assert.Equal(t, wantID, got[i].ID, "row %d id preserved", i)
}
var srcCount int64
require.NoError(t, srcDB.Model(&analyticsmodel.UserAccessLog{}).Count(&srcCount).Error)
assert.Equal(t, int64(3), srcCount)
}
// TestClearTargetLogTablesClearsUserAccessLogs 验证清空目标包含用户访问日志表
// (6 张日志表之一),迁移「覆盖目标库已有日志」幂等前提成立。
func TestClearTargetLogTablesClearsUserAccessLogs(t *testing.T) {
oldDB, oldCH := config.Config.Database.Enabled, config.Config.ClickHouse.Enabled
config.Config.Database.Enabled, config.Config.ClickHouse.Enabled = false, false
t.Cleanup(func() {
config.Config.Database.Enabled, config.Config.ClickHouse.Enabled = oldDB, oldCH
})
logstore.ResetForTest()
defer logstore.ResetForTest()
ctx := context.Background()
dstDB := newLogDBSwitchDB(t)
db.SetDB(dstDB)
t.Cleanup(func() { db.SetDB(nil) })
dst, err := logstore.BuildForMigration(ctx, "sqlite")
require.NoError(t, err)
now := time.Now().UTC()
require.NoError(t, dst.UserAccessLogs.BatchInsert(ctx, []analyticsmodel.UserAccessLog{
{ID: 1, UserID: 1, Path: "/a", CreatedAt: now},
{ID: 2, UserID: 2, Path: "/b", CreatedAt: now},
}))
require.NoError(t, clearTargetLogTables(ctx, dst))
var count int64
require.NoError(t, dstDB.Model(&analyticsmodel.UserAccessLog{}).Count(&count).Error)
assert.Zero(t, count, "用户访问日志应被清空")
}
// TestClearTargetLogTablesDuringMigration 回归:冻结标记置位后,BuildForMigration 构造的
// 目标 store 必须放行用户访问日志清空/写入。skipFreeze 未传播到 UserAccessLogs store 时
// DeleteAll 会误报 ErrMigrating,导致真实切换任务在清空目标库阶段失败。
func TestClearTargetLogTablesDuringMigration(t *testing.T) {
logstore.ResetForTest()
defer logstore.ResetForTest()
gdb := newLogDBSwitchDB(t)
db.SetDB(gdb)
t.Cleanup(func() { db.SetDB(nil) })
ctx := context.Background()
logstore.SetConfigReader(func(ctx context.Context, key string) (string, error) {
cfg, err := repository.GetSystemConfigByKey(ctx, key)
if err != nil {
return "", err
}
return cfg.Value, nil
})
// 预置目标库已有日志(迁移「覆盖目标库已有日志」幂等前提)。
now := time.Now().UTC()
require.NoError(t, gdb.Create(&analyticsmodel.UserAccessLog{ID: 1, UserID: 1, Path: "/a", CreatedAt: now}).Error)
// 冻结标记置位(与真实任务 Execute 流程一致)。
require.NoError(t, setMigrationFlag(ctx, "migrating"))
t.Cleanup(func() { _ = setMigrationFlag(ctx, "") })
require.True(t, logstore.Migrating(ctx))
dst, err := logstore.BuildForMigration(ctx, "sqlite")
require.NoError(t, err)
require.NoError(t, clearTargetLogTables(ctx, dst), "迁移冻结期间目标库清空必须放行")
var count int64
require.NoError(t, gdb.Model(&analyticsmodel.UserAccessLog{}).Count(&count).Error)
assert.Zero(t, count, "用户访问日志应被清空")
}
// TestValidateSwitch 各非法组合报错。
func TestValidateSwitch(t *testing.T) {
oldDB, oldCH := config.Config.Database.Enabled, config.Config.ClickHouse.Enabled
t.Cleanup(func() {
config.Config.Database.Enabled, config.Config.ClickHouse.Enabled = oldDB, oldCH
})
gdb := newLogDBSwitchDB(t)
db.SetDB(gdb)
t.Cleanup(func() { db.SetDB(nil) })
ctx := context.Background()
setLogDB := func(v string) {
require.NoError(t, repository.SaveOrUpdateSystemConfig(ctx, model.ConfigKeyLogDatabase, v))
}
t.Run("same target rejected", func(t *testing.T) {
setLogDB("sqlite")
config.Config.Database.Enabled, config.Config.ClickHouse.Enabled = false, false
err := validateSwitch(ctx, "sqlite")
require.Error(t, err)
assert.Contains(t, err.Error(), "相同")
})
t.Run("clickhouse disabled rejected", func(t *testing.T) {
setLogDB("sqlite")
config.Config.Database.Enabled, config.Config.ClickHouse.Enabled = false, false
err := validateSwitch(ctx, "clickhouse")
require.Error(t, err)
assert.Contains(t, err.Error(), "ClickHouse 未启用")
})
t.Run("postgres requires main db enabled", func(t *testing.T) {
setLogDB("sqlite")
config.Config.Database.Enabled, config.Config.ClickHouse.Enabled = false, false
err := validateSwitch(ctx, "postgres")
require.Error(t, err)
assert.Contains(t, err.Error(), "PostgreSQL 未启用")
})
t.Run("sqlite rejected when main db is postgres", func(t *testing.T) {
setLogDB("postgres")
config.Config.Database.Enabled, config.Config.ClickHouse.Enabled = true, false
err := validateSwitch(ctx, "sqlite")
require.Error(t, err)
assert.Contains(t, err.Error(), "SQLite")
})
t.Run("valid postgres migration", func(t *testing.T) {
setLogDB("sqlite")
config.Config.Database.Enabled, config.Config.ClickHouse.Enabled = true, false
require.NoError(t, validateSwitch(ctx, "postgres"))
})
}
// TestLogDBSwitchValidatePayload 参数归一化与非法值拒绝。
func TestLogDBSwitchValidatePayload(t *testing.T) {
h := &LogDBSwitchHandler{}
cases := []struct {
name string
in string
want string
ok bool
}{
{name: "postgresql normalized", in: `{"target":"postgresql"}`, want: "postgres", ok: true},
{name: "sqlite3 normalized", in: `{"target":"sqlite3"}`, want: "sqlite", ok: true},
{name: "ch normalized", in: `{"target":"ch"}`, want: "clickhouse", ok: true},
{name: "postgres passthrough", in: `{"target":"postgres"}`, want: "postgres", ok: true},
{name: "invalid target", in: `{"target":"mysql"}`, ok: false},
{name: "malformed json", in: `not-json`, ok: false},
}
for _, c := range cases {
t.Run(c.name, func(t *testing.T) {
out, err := h.ValidatePayload([]byte(c.in))
if !c.ok {
require.Error(t, err)
return
}
require.NoError(t, err)
var p logDBSwitchPayload
require.NoError(t, json.Unmarshal(out, &p))
assert.Equal(t, c.want, p.Target)
})
}
}
// TestExecuteFailureClearsMigrationFlag 迁移失败后 log_db_migration 冻结标记被清除。
// 在 FRESH DB(不预置 log_db_migration 行)上验证:setMigrationFlag 必须 upsert 建行,
// 且失败后经缓存路径(GetSystemConfigByKey)可观察为空。
func TestExecuteFailureClearsMigrationFlag(t *testing.T) {
oldDB := config.Config.Database.Enabled
config.Config.Database.Enabled = true
t.Cleanup(func() { config.Config.Database.Enabled = oldDB })
logstore.ResetForTest()
defer logstore.ResetForTest()
gdb := newLogDBSwitchDB(t)
db.SetDB(gdb)
t.Cleanup(func() { db.SetDB(nil) })
ctx := context.Background()
// FRESH DB:log_db_migration 行不存在(不预置),log_database 预置为 sqlite。
require.NoError(t, repository.SaveOrUpdateSystemConfig(ctx, model.ConfigKeyLogDatabase, "sqlite"))
_, err := repository.GetSystemConfigByKey(ctx, model.ConfigKeyLogDBMigration)
require.ErrorIs(t, err, gorm.ErrRecordNotFound)
// configReader 对 log_database 报错,使 logstore.Active 在冻结标记置位后失败。
logstore.SetConfigReader(func(_ context.Context, key string) (string, error) {
if key == model.ConfigKeyLogDatabase {
return "", errors.New("reader error")
}
return "", nil
})
_, err = (&LogDBSwitchHandler{}).Execute(ctx, []byte(`{"target":"postgres"}`))
require.Error(t, err)
assert.Contains(t, err.Error(), "reader error")
// 冻结标记必须被 upsert 持久化(行存在)并经缓存路径可观察为空,源库恢复可写。
cfg, err := repository.GetSystemConfigByKey(ctx, model.ConfigKeyLogDBMigration)
require.NoError(t, err, "setMigrationFlag 应 upsert 创建 log_db_migration 行")
assert.Empty(t, cfg.Value, "失败后冻结标记必须清除,源库保持可写")
assert.False(t, logstore.Migrating(ctx))
}
// TestSetMigrationFlagObservableThroughCache 在 FRESH DB 上验证 setMigrationFlag 写入
// 经缓存路径(logstore.Migrating → repository 读取)实时反映:置位 true、清除 false。
func TestSetMigrationFlagObservableThroughCache(t *testing.T) {
logstore.ResetForTest()
defer logstore.ResetForTest()
gdb := newLogDBSwitchDB(t)
db.SetDB(gdb)
t.Cleanup(func() { db.SetDB(nil) })
ctx := context.Background()
// 按 bootstrap 同款注入 repository 读取,走 RAM 缓存路径。
logstore.SetConfigReader(func(ctx context.Context, key string) (string, error) {
cfg, err := repository.GetSystemConfigByKey(ctx, key)
if err != nil {
return "", err
}
return cfg.Value, nil
})
// FRESH DB:行缺失 → fail-open false。
assert.False(t, logstore.Migrating(ctx))
require.NoError(t, setMigrationFlag(ctx, "migrating"))
assert.True(t, logstore.Migrating(ctx), "置位后缓存路径必须立即观察到 migrating")
require.NoError(t, setMigrationFlag(ctx, ""))
assert.False(t, logstore.Migrating(ctx), "清除后缓存路径必须立即观察到非 migrating")
}
// TestFlipLogDatabaseRefreshesCachedConfig 验证翻转日志主库后缓存路径立即反映新库
// (logstore.ActiveDatabase / GetSystemConfigByKey),防止各进程继续写旧库(split-brain)。
func TestFlipLogDatabaseRefreshesCachedConfig(t *testing.T) {
logstore.ResetForTest()
defer logstore.ResetForTest()
gdb := newLogDBSwitchDB(t)
db.SetDB(gdb)
t.Cleanup(func() { db.SetDB(nil) })
ctx := context.Background()
logstore.SetConfigReader(func(ctx context.Context, key string) (string, error) {
cfg, err := repository.GetSystemConfigByKey(ctx, key)
if err != nil {
return "", err
}
return cfg.Value, nil
})
require.NoError(t, repository.SaveOrUpdateSystemConfig(ctx, model.ConfigKeyLogDatabase, "sqlite"))
active, err := logstore.ActiveDatabase(ctx)
require.NoError(t, err)
assert.Equal(t, "sqlite", active) // 预热缓存
require.NoError(t, flipLogDatabase(ctx, "postgres"))
active, err = logstore.ActiveDatabase(ctx)
require.NoError(t, err)
assert.Equal(t, "postgres", active, "翻转后缓存路径必须立即反映新库")
cfg, err := repository.GetSystemConfigByKey(ctx, model.ConfigKeyLogDatabase)
require.NoError(t, err)
assert.Equal(t, "postgres", cfg.Value)
}
+1 -1
View File
@@ -292,7 +292,7 @@ func (c *SocketIOClient) Emit(event string, args ...any) (string, error) {
}
body := fmt.Sprintf("42%d%s", id, string(bs))
slog.Debug("Emitting Socket.IO event", "event", event, "ackID", id, "payload", string(bs))
slog.Debug("Emitting Socket.IO event", "event", event, "ackID", id, "payload_len", len(bs))
u := fmt.Sprintf("%s/socket.io/?EIO=4&transport=polling&sid=%s", c.baseURL, c.sid)
req, err := http.NewRequestWithContext(c.ctx, http.MethodPost, u, strings.NewReader(body))
@@ -12,6 +12,7 @@ import (
"time"
"github.com/Rain-kl/Wavelet/internal/repository"
"github.com/Rain-kl/Wavelet/internal/testhelper"
db "github.com/Rain-kl/Wavelet/internal/infra/persistence"
"github.com/Rain-kl/Wavelet/internal/model"
@@ -34,9 +35,8 @@ func setupIPGroupSyncTestDB(t *testing.T) func() {
))
db.SetDB(sqliteDB)
resetAccessLogStore := repository.SetAccessLogStoreForTest(repository.NewMemoryAccessLogStore())
testhelper.SetupLogStoresForTest(t)
return func() {
resetAccessLogStore()
db.SetDB(nil)
}
}
+2 -2
View File
@@ -9,6 +9,7 @@ import (
"time"
"github.com/Rain-kl/Wavelet/internal/repository"
"github.com/Rain-kl/Wavelet/internal/testhelper"
db "github.com/Rain-kl/Wavelet/internal/infra/persistence"
"github.com/Rain-kl/Wavelet/internal/model"
@@ -61,8 +62,7 @@ func TestLegacyImportUsesEffectiveTLDPlusOne(t *testing.T) {
func TestGetStatsAggregatesZoneHosts(t *testing.T) {
ctx := setupZoneDB(t)
reset := repository.SetAccessLogStoreForTest(repository.NewMemoryAccessLogStore())
t.Cleanup(reset)
testhelper.SetupLogStoresForTest(t)
zone, err := Create(ctx, Input{Domain: "example.com"})
require.NoError(t, err)
-129
View File
@@ -1,129 +0,0 @@
// Copyright 2026 Arctel.net
// SPDX-License-Identifier: Apache-2.0
package risk_control
import (
"context"
"sync"
"time"
"github.com/Rain-kl/Wavelet/internal/infra/config"
"github.com/Rain-kl/Wavelet/internal/infra/persistence/batchwriter"
"github.com/Rain-kl/Wavelet/internal/model/analytics"
"github.com/Rain-kl/Wavelet/internal/platform/lifecycle"
analyticsrepo "github.com/Rain-kl/Wavelet/internal/repository/analytics"
"github.com/Rain-kl/Wavelet/pkg/logger"
)
const (
// Bound visibility lag for sparse access-log traffic when MinBatchSize is not met.
accessLogMaxFlushWait = 3 * time.Second
)
var (
logWriterMu sync.RWMutex
logWriter *batchwriter.Writer[*analytics.UserAccessLog]
)
// InitLogWriter initializes the ClickHouse access-log batch writer.
func InitLogWriter(ctx context.Context) {
if !config.Config.ClickHouse.Enabled {
return
}
logWriterMu.Lock()
defer logWriterMu.Unlock()
if logWriter != nil {
return
}
cfg := batchwriter.DefaultConfig()
cfg.Name = "user_access_logs"
cfg.MaxFlushWait = accessLogMaxFlushWait
writer, err := batchwriter.New[*analytics.UserAccessLog](cfg, func(ctx context.Context, items []*analytics.UserAccessLog) error {
rows := make([]analytics.UserAccessLog, 0, len(items))
for _, item := range items {
if item == nil {
continue
}
rows = append(rows, *item)
}
return analyticsrepo.BatchInsert(ctx, rows)
},
batchwriter.WithDropHandler[*analytics.UserAccessLog](func(item *analytics.UserAccessLog) {
path := ""
if item != nil {
path = item.Path
}
logger.WarnF(context.Background(), "[RiskControl] Log queue full, dropping log item for path: %s", path)
}),
batchwriter.WithFlushErrorHandler[*analytics.UserAccessLog](func(ctx context.Context, items []*analytics.UserAccessLog, err error) {
logger.ErrorF(ctx, "[RiskControl] Send ClickHouse batch failed (batch=%d): %v", len(items), err)
}),
)
if err != nil {
logger.ErrorF(ctx, "[RiskControl] init log writer failed: %v", err)
return
}
writer.Start(ctx)
logWriter = writer
lifecycle.OnShutdown("risk_control_log_writer", StopLogWriter)
}
// StopLogWriter stops the ClickHouse access-log batch writer and drains pending logs.
func StopLogWriter(ctx context.Context) error {
writer := currentLogWriter()
if writer == nil {
return nil
}
return writer.Stop(ctx)
}
// IsBufferFull reports whether the access-log queue has no remaining capacity.
func IsBufferFull() bool {
writer := currentLogWriter()
if writer == nil {
return false
}
return writer.IsFull()
}
// LogWriterStats returns queue depth and failure counters for the access-log writer.
// When the writer is not initialized, it returns a zero-value Stats with the expected name.
func LogWriterStats() batchwriter.Stats {
writer := currentLogWriter()
if writer == nil {
return batchwriter.Stats{Name: "user_access_logs"}
}
return writer.Stats()
}
// QueueAccessLog enqueues an access log without blocking.
func QueueAccessLog(logItem *analytics.UserAccessLog) {
writer := currentLogWriter()
if writer == nil || logItem == nil {
return
}
writer.TryEnqueue(logItem)
}
// SetLogWriterForTest swaps the access-log writer for unit tests.
func SetLogWriterForTest(writer *batchwriter.Writer[*analytics.UserAccessLog]) func() {
logWriterMu.Lock()
previous := logWriter
logWriter = writer
logWriterMu.Unlock()
return func() {
logWriterMu.Lock()
logWriter = previous
logWriterMu.Unlock()
}
}
func currentLogWriter() *batchwriter.Writer[*analytics.UserAccessLog] {
logWriterMu.RLock()
defer logWriterMu.RUnlock()
return logWriter
}
-30
View File
@@ -1,30 +0,0 @@
// Copyright 2026 Arctel.net
// SPDX-License-Identifier: Apache-2.0
package risk_control
import (
"testing"
"time"
)
func TestAccessLogMaxFlushWaitInRange(t *testing.T) {
t.Parallel()
if accessLogMaxFlushWait < 2*time.Second || accessLogMaxFlushWait > 5*time.Second {
t.Fatalf("accessLogMaxFlushWait = %v, want in [2s, 5s]", accessLogMaxFlushWait)
}
}
func TestLogWriterStatsWhenNil(t *testing.T) {
t.Parallel()
reset := SetLogWriterForTest(nil)
t.Cleanup(reset)
stats := LogWriterStats()
if stats.Name != "user_access_logs" {
t.Fatalf("LogWriterStats().Name = %q, want user_access_logs", stats.Name)
}
if stats.Running {
t.Fatal("LogWriterStats().Running = true for nil writer, want false")
}
}
-134
View File
@@ -1,134 +0,0 @@
// Copyright 2026 Arctel.net
// SPDX-License-Identifier: Apache-2.0
// Package risk_control 提供风险控制中间件
package risk_control
import (
"crypto/sha256"
"encoding/hex"
"encoding/json"
"net/http"
"strings"
"time"
"github.com/Rain-kl/Wavelet/internal/apps/oauth"
"github.com/Rain-kl/Wavelet/internal/infra/config"
"github.com/Rain-kl/Wavelet/internal/infra/persistence/idgen"
"github.com/Rain-kl/Wavelet/internal/model"
"github.com/Rain-kl/Wavelet/internal/model/analytics"
"github.com/Rain-kl/Wavelet/internal/shared/response"
"github.com/gin-gonic/gin"
)
const maxAuditLogHeadersBytes = 2 * 1024
var auditLogHeaderAllowlist = map[string]struct{}{
"Authorization": {},
"Cookie": {},
"X-Forwarded-For": {},
"X-Real-Ip": {},
"User-Agent": {},
"Content-Type": {},
}
func marshalAuditLogHeaders(headers http.Header) string {
if headers == nil {
return ""
}
filtered := make(http.Header)
for key, values := range headers {
if _, ok := auditLogHeaderAllowlist[key]; !ok {
continue
}
filtered[key] = redactAuditLogHeaderValues(key, values)
}
headersBytes, err := json.Marshal(filtered)
if err != nil {
return ""
}
if len(headersBytes) <= maxAuditLogHeadersBytes {
return string(headersBytes)
}
return string(headersBytes[:maxAuditLogHeadersBytes])
}
func redactAuditLogHeaderValues(key string, values []string) []string {
switch key {
case "Authorization", "Cookie":
redacted := make([]string, len(values))
for i, value := range values {
redacted[i] = hashAuditLogSensitiveValue(value)
}
return redacted
default:
return values
}
}
func hashAuditLogSensitiveValue(value string) string {
if strings.TrimSpace(value) == "" {
return ""
}
sum := sha256.Sum256([]byte(value))
return "sha256:" + hex.EncodeToString(sum[:8])
}
// RiskControlMiddleware 全局日志采集中间件
func RiskControlMiddleware() gin.HandlerFunc {
return func(c *gin.Context) {
// 如果未启用 ClickHouse,直接放行
if !config.Config.ClickHouse.Enabled {
c.Next()
return
}
// 1. 限流背压检测(检测本地缓冲队列是否已满)
if IsBufferFull() {
response.AbortTooManyRequests(c, "系统繁忙,请稍后再试")
return
}
start := time.Now()
// 2. 执行后续请求(穿过业务处理和认证中间件)
c.Next()
// 3. 后置身份检查:仅记录通过认证的请求
userObj, exists := oauth.GetFromContext[*model.User](c, oauth.UserObjKey)
if !exists || userObj == nil {
return
}
// 4. 计算耗时并异步推送到缓冲队列
latency := time.Since(start).Milliseconds()
headersStr := marshalAuditLogHeaders(c.Request.Header)
const maxHTTPStatus = 999
status := c.Writer.Status()
if status < 0 {
status = 0
} else if status > maxHTTPStatus {
status = maxHTTPStatus
}
logItem := &analytics.UserAccessLog{
ID: idgen.NextUint64ID(),
UserID: userObj.ID, // 直接从 Context 获取已登录用户ID,避免数据库查询
Path: c.Request.URL.Path,
Method: c.Request.Method,
IP: c.ClientIP(),
UserAgent: c.Request.UserAgent(),
Headers: headersStr,
Status: int32(status),
Latency: latency,
CreatedAt: time.Now(),
}
// 非阻塞地推入缓存队列
QueueAccessLog(logItem)
}
}
@@ -1,178 +0,0 @@
// Copyright 2026 Arctel.net
// SPDX-License-Identifier: Apache-2.0
package risk_control
import (
"context"
"encoding/json"
"net/http"
"net/http/httptest"
"testing"
"time"
"github.com/Rain-kl/Wavelet/internal/apps/oauth"
"github.com/Rain-kl/Wavelet/internal/infra/config"
"github.com/Rain-kl/Wavelet/internal/infra/persistence/batchwriter"
"github.com/Rain-kl/Wavelet/internal/model"
"github.com/Rain-kl/Wavelet/internal/model/analytics"
"github.com/Rain-kl/Wavelet/internal/testhelper"
"github.com/gin-gonic/gin"
"github.com/stretchr/testify/assert"
)
const testLogQueueSize = 10_000
func newTestLogWriter(t *testing.T, queueSize int) (*batchwriter.Writer[*analytics.UserAccessLog], chan *analytics.UserAccessLog) {
t.Helper()
received := make(chan *analytics.UserAccessLog, queueSize)
cfg := batchwriter.Config{
QueueSize: queueSize,
MaxBatchSize: 1,
FlushInterval: 5 * time.Millisecond,
}
writer, err := batchwriter.New[*analytics.UserAccessLog](cfg, func(_ context.Context, items []*analytics.UserAccessLog) error {
for _, item := range items {
if item != nil {
received <- item
}
}
return nil
})
assert.NoError(t, err)
writer.Start(context.Background())
t.Cleanup(func() {
stopCtx, cancel := context.WithTimeout(context.Background(), time.Second)
defer cancel()
_ = writer.Stop(stopCtx)
})
return writer, received
}
func TestRiskControlMiddleware(t *testing.T) {
gin.SetMode(gin.TestMode)
t.Run("ClickHouse disabled", func(t *testing.T) {
config.Config.ClickHouse.Enabled = false
defer func() { config.Config.ClickHouse.Enabled = false }()
r := testhelper.NewTestGinEngine(RiskControlMiddleware())
r.GET("/test", func(c *gin.Context) {
c.String(http.StatusOK, "ok")
})
w := httptest.NewRecorder()
req, _ := http.NewRequest(http.MethodGet, "/test", nil)
r.ServeHTTP(w, req)
assert.Equal(t, http.StatusOK, w.Code)
assert.Equal(t, "ok", w.Body.String())
})
t.Run("ClickHouse enabled - Normal Authenticated Request", func(t *testing.T) {
config.Config.ClickHouse.Enabled = true
writer, received := newTestLogWriter(t, testLogQueueSize)
resetWriter := SetLogWriterForTest(writer)
defer func() {
config.Config.ClickHouse.Enabled = false
resetWriter()
}()
r := gin.New()
r.Use(func(c *gin.Context) {
user := &model.User{ID: 12345}
oauth.SetToContext(c, oauth.UserObjKey, user)
c.Next()
})
r.Use(RiskControlMiddleware())
r.GET("/test", func(c *gin.Context) {
c.String(http.StatusOK, "ok")
})
w := httptest.NewRecorder()
req, _ := http.NewRequest(http.MethodGet, "/test", nil)
req.Header.Set("X-Test-Header", "hello")
req.Header.Set("Cookie", "session_id=abcdef123456")
req.Header.Set("Authorization", "Bearer secret-token")
req.Header.Set("Content-Type", "application/json")
r.ServeHTTP(w, req)
assert.Equal(t, http.StatusOK, w.Code)
assert.Equal(t, "ok", w.Body.String())
select {
case logItem := <-received:
assert.Equal(t, uint64(12345), logItem.UserID)
assert.Equal(t, "/test", logItem.Path)
assert.Equal(t, http.MethodGet, logItem.Method)
assert.Equal(t, int32(http.StatusOK), logItem.Status)
assert.NotEmpty(t, logItem.Headers)
assert.NotContains(t, logItem.Headers, "X-Test-Header")
assert.Contains(t, logItem.Headers, "Content-Type")
assert.Contains(t, logItem.Headers, "sha256:")
assert.NotContains(t, logItem.Headers, "secret-token")
assert.NotContains(t, logItem.Headers, "session_id=abcdef123456")
case <-time.After(200 * time.Millisecond):
t.Fatal("expected flushed log item, but got none")
}
})
t.Run("ClickHouse enabled - Unauthenticated Request", func(t *testing.T) {
config.Config.ClickHouse.Enabled = true
writer, received := newTestLogWriter(t, testLogQueueSize)
resetWriter := SetLogWriterForTest(writer)
defer func() {
config.Config.ClickHouse.Enabled = false
resetWriter()
}()
r := testhelper.NewTestGinEngine(RiskControlMiddleware())
r.GET("/test", func(c *gin.Context) {
c.String(http.StatusOK, "ok")
})
w := httptest.NewRecorder()
req, _ := http.NewRequest(http.MethodGet, "/test", nil)
r.ServeHTTP(w, req)
assert.Equal(t, http.StatusOK, w.Code)
assert.Equal(t, "ok", w.Body.String())
select {
case <-received:
t.Fatal("expected no log item for unauthenticated request")
case <-time.After(50 * time.Millisecond):
}
})
t.Run("ClickHouse enabled - Buffer Full Rate Limiting", func(t *testing.T) {
config.Config.ClickHouse.Enabled = true
writer, _ := newTestLogWriter(t, 2)
resetWriter := SetLogWriterForTest(writer)
defer func() {
config.Config.ClickHouse.Enabled = false
resetWriter()
}()
for range 2 {
assert.True(t, writer.TryEnqueue(&analytics.UserAccessLog{}))
}
r := testhelper.NewTestGinEngine(RiskControlMiddleware())
r.GET("/test", func(c *gin.Context) {
c.String(http.StatusOK, "ok")
})
w := httptest.NewRecorder()
req, _ := http.NewRequest(http.MethodGet, "/test", nil)
r.ServeHTTP(w, req)
assert.Equal(t, http.StatusTooManyRequests, w.Code)
var resp map[string]interface{}
err := json.Unmarshal(w.Body.Bytes(), &resp)
assert.NoError(t, err)
assert.Contains(t, resp["error_msg"], "系统繁忙")
})
}

Some files were not shown because too many files have changed in this diff Show More