Compare commits

..

45 Commits

Author SHA1 Message Date
ryan 13c5073bf8 chore(release): v3.1.2
### 🛠 修复
- 修复了节点与仪表盘 24 小时容量、网络、磁盘 IO 趋势在限流查询下几乎为空的问题,改为小时级聚合与计数器增量统计,历史时段可正常展示。
- 修复了静置场景下 ClickHouse CPU 偏高的问题,可观测与访问日志写入改为批量凑批并限制小 part 产生。
- 修复了数据清理接口将「物化表 TTL」误报为已删除行数的问题,短于表 TTL 的保留天数会被明确拒绝。
- 修复了可观测去重在入队/刷盘失败后仍占用键、导致故障窗口数据更易丢失的问题,并在 flush 失败时短重试与释放键。
- 修复了 Dashboard「每节点最新指标」被全局 LIMIT 截断导致安静节点缺失的问题,改为按节点取最新快照。
- 修复了 Docker 部署 ClickHouse 25.x 因后台池与 mutation 空闲阈值不兼容而无法启动的问题。
- 修复了小时预聚合仅有迁移后少量数据时 24 小时趋势再次残缺的问题:读路径按小时 merge(窗口完整走 rollup,缺口用 raw 补齐),并增加历史 backfill 迁移。

### ⚡️ 优化与改进
- 为容量与 OpenResty 指标增加小时预聚合表,窗口完整时优先走 rollup 降低查询压力。
- 审计日志写入增加最长等待刷盘,管理端可观测状态接口暴露 batch writer 队列深度、丢弃与 flush 错误指标。
- 小规格场景下调 ClickHouse 客户端连接池默认值,并调整 async_insert 合并超时与流量小时表 TTL。
- 流量独立访客在小时汇总中改为窗口峰值估计,并修正界面文案,避免被误解为全局真实 UV。

### 💄 其他/体验
- 同步环境变量与配置模板中的 ClickHouse 说明;部署文档改为将 performance.xml 下载到 ./config/clickhouse 后挂载,且不挂载 listen 配置。
2026-07-10 11:42:49 +08:00
ryan da1dd92404 fix(observability): merge rollup+raw hourly trends and backfill history
Prefer capacity/openresty rollups only when they cover the 24h window;
otherwise merge per hour so raw fills pre-MV gaps and rollup wins on
overlap. Add a one-time ANTI JOIN backfill migration for the last 30 days.
2026-07-10 11:27:51 +08:00
ryan 4b11279662 fix(observability): fall back to raw hourly when rollup is incomplete
Materialized capacity/openresty hourly tables only hold data after the MV
exists. Preferring any non-empty rollup hid full raw history and left 24h
charts with only recent hours. Use rollup only when its earliest bucket
covers the query window start.
2026-07-10 11:27:51 +08:00
ryan bbadcca294 ### 🛠 修复
- 修复了节点与仪表盘 24 小时容量、网络、磁盘 IO 趋势在限流查询下几乎为空的问题,改为小时级聚合与计数器增量统计,历史时段可正常展示。
- 修复了静置场景下 ClickHouse CPU 偏高的问题,可观测与访问日志写入改为批量凑批并限制小 part 产生。
- 修复了数据清理接口将「物化表 TTL」误报为已删除行数的问题,短于表 TTL 的保留天数会被明确拒绝。
- 修复了可观测去重在入队/刷盘失败后仍占用键、导致故障窗口数据更易丢失的问题,并在 flush 失败时短重试与释放键。
- 修复了 Dashboard「每节点最新指标」被全局 LIMIT 截断导致安静节点缺失的问题,改为按节点取最新快照。
- 修复了 Docker 部署 ClickHouse 25.x 因后台池与 mutation 空闲阈值不兼容而无法启动的问题。

### ⚡️ 优化与改进
- 为容量与 OpenResty 指标增加小时预聚合表,读路径优先 rollup,降低 24 小时趋势查询压力。
- 审计日志写入增加最长等待刷盘,管理端可观测状态接口暴露 batch writer 队列深度、丢弃与 flush 错误指标。
- 小规格场景下调 ClickHouse 客户端连接池默认值,并调整 async_insert 合并超时与流量小时表 TTL。
- 流量独立访客在小时汇总中改为窗口峰值估计,并修正界面文案,避免被误解为全局真实 UV。

### 💄 其他/体验
- 同步环境变量与配置模板中的 ClickHouse 说明;部署文档改为将 performance.xml 下载到 ./config/clickhouse 后挂载,且不挂载 listen 配置。
2026-07-10 11:27:51 +08:00
ryan 44ce6497a1 docs(docker): use ./config/clickhouse for CH performance mount
Move performance.xml to config/clickhouse and mount the directory to
/etc/clickhouse-server/config.d; update docs and compose paths.
2026-07-10 11:10:32 +08:00
ryan 4b83f91b31 docs(docker): use ./config/clickhouse for CH performance mount
Move performance.xml to config/clickhouse and mount the directory to
/etc/clickhouse-server/config.d; update docs and compose paths.
2026-07-10 10:59:21 +08:00
ryan 9d2fac5d4c docs(config): sync .env.example and config.example.yaml for CH defaults
Align ClickHouse pool/password placeholders and docker-compose env docs with
runtime defaults and published ports used in local Docker testing.
2026-07-10 10:48:30 +08:00
ryan b4b93ff4ed fix(docker): make ClickHouse startable on 25.x and reachable from host
Lower merge-tree free-entry thresholds for small background pools, bind
listen_host to 0.0.0.0 for published ports, and allow CLICKHOUSE_ENABLED=true
in tests for live_ch smoke coverage.
2026-07-10 10:44:49 +08:00
ryan 160e63558f fix(clickhouse): harden R/W path P0–P3 (cleanup, durability, rollups)
Honest TTL cleanup semantics; enqueue-safe dedup with flush retry and writer
metrics; model insert hooks; latest-per-node and hourly metric/openresty
rollups; small-host pool/async defaults, traffic hourly TTL, and UV labeling.
2026-07-10 10:34:04 +08:00
ryan 9b3555c569 fix(clickhouse): cut idle CPU from tiny parts and oversized merge pools
Observability writers flushed every few seconds with MinBatchSize unset,
creating constant small parts and merge load. Enable MinBatchSize with
MaxFlushWait, batch access logs more aggressively, and shrink ClickHouse
background pools for 3c hosts.
2026-07-10 10:08:32 +08:00
ryan b928928958 fix(observability): restore 24h capacity/network/disk trends via CH hourly agg
Node and dashboard 24h capacity, network, and disk IO charts only used the
latest limited raw snapshots (120/500 rows), so historical hour buckets stayed
empty. Prefer ClickHouse hourly aggregates with counter deltas, and fall back
to raw snapshots when aggregation is unavailable.
2026-07-10 09:48:45 +08:00
ryan b312460ddf chore(release): v3.1.1
### ⚡️ 优化与改进
- 将 cap_login_enabled 默认值由 true 变更为 false,默认关闭登录界面 PoW 人机验证。
2026-07-06 12:29:53 +08:00
ryan 50f7257d93 docs: readme 2026-07-05 23:53:26 +08:00
ryan 336185f01c release: v3.1.0
### 🛠 修复
- 修复 ClickHouse TTL 迁移中 DateTime64 时间列无法直接设置 TTL 导致 goose 启动失败的问题,改为通过 toDateTime() 转换后再应用 TTL。
- 修复 ClickHouse 迁移尝试缩短 ORDER BY 排序键时与隐式主键前缀冲突导致迁移失败的问题,移除不支持的 MODIFY ORDER BY 操作。
- 修复系统设置页面 URL tab 参数未包含 openflare-ops 选项卡导致无法正确定位的问题,并在无参数时默认选中 OpenFlare 选项卡。
- 修复系统自更新检测上游 GitHub Release 时,因资产包名称前缀 openflare-server 与仓库名不完全一致导致匹配失败并报错「未找到兼容的 Release」的问题。
- 修复全局搜索数据源覆盖不全的问题,补全所有核心业务控制台页面及管理员专有页面的检索支持。

### ⚡️ 优化与改进
- ClickHouse 启用 async_insert 异步写入缓冲,并调高 block_buffer_size 与连接池默认值,降低小 part 生成与连接争用。
- 优化 ClickHouse 写入路径:移除 Agent 心跳中的同步 ALTER DELETE 保留清理,batchwriter 新增 MinBatchSize 抑制过小批次定时 flush,可观测 writer 批次与 flush 间隔调优并补全去重。
- ClickHouse 分析表新增 TTL 自动过期策略,访问日志 180 天、节点访问日志 90 天、其余观测与聚合表 30 天自动清理。
- 访问日志与 WAF IP 组查询改为 ClickHouse 侧聚合与 SQL 分页,默认限制近 7 天查询窗口,浏览器分布查询增加 Top 100 限制。
- Dashboard 与节点可观测 API 消除无 LIMIT 全表扫描,增加短 TTL 内存缓存,前端轮询间隔分别调整为 60s/30s。
- ClickHouse 遗留治理 Phase 2:保留期清理改为 TTL MATERIALIZE TTL,统一 ChConn 读路径,新增 /admin/status/clickhouse 运维指标与 of_node_traffic_hourly 预聚合 MV。
- 审计访问日志写入时仅保留安全相关请求头并以 SHA-256 脱敏,将 headers 载荷上限收紧至 2KB,减小行宽与 merge CPU 开销。
- Docker 部署为 ClickHouse 增加 nofile ulimits 与性能配置挂载,限制 max_concurrent_queries 与后台合并争用。
- 数据库自动清理任务新增 OpenResty、FRPS、FRPC 观测表清理目标。

### 💄 其他/体验
- 隐藏侧边栏文档库中的「规范示例」与「接口文档」,将「使用文档」及其他相关页面链接统一跳转至外部文档站 https://open-flare.pages.dev/。
- 移除系统设置 OpenFlare 标签页下的版本信息卡片及对应升级管理弹窗逻辑。
- 系统设置页面支持通过 URL 持久化当前选中的 Tab 状态。
2026-07-04 09:43:31 +08:00
ryan 44bba0f19a fix(clickhouse): remove unsupported MODIFY ORDER BY from migration
ClickHouse keeps the implicit PRIMARY KEY when shortening ORDER BY,
which fails with "Primary key must be a prefix of the sorting key".
TTL-only changes are safe and unblock goose startup; narrowing ORDER BY
would require table recreation.
2026-07-02 16:45:48 +08:00
ryan f0eca028f9 fix(clickhouse): cast DateTime64 to DateTime in TTL migration 2026-07-02 16:21:30 +08:00
ryan 58624db397 perf(clickhouse): Phase 2 legacy governance — TTL cleanup, unified pool, MV, ops API
- Replace retention ALTER DELETE with MATERIALIZE TTL; use TRUNCATE for delete-all
- Remove GORM ClickHouse pool; migrate user access log reads to ChConn
- Drop query-side trim(remote_addr); enable wait_for_async_insert=1
- Add of_node_traffic_hourly MV and dashboard traffic trend fallback
- Add GET /admin/status/clickhouse operational metrics endpoint
2026-07-02 15:47:38 +08:00
ryan 38946d1af5 fix(clickhouse): resolve lint issues from optimization stack 2026-07-02 15:28:49 +08:00
ryan caf2ffcff4 perf(clickhouse): P1 TTL migrations, ORDER BY tune, remote_addr normalization 2026-07-02 15:25:03 +08:00
ryan 0e86fe3547 perf(clickhouse): P2 docker server tuning and audit log payload reduction 2026-07-02 15:23:17 +08:00
ryan 6525bef15d perf(clickhouse): P0/P1 access log and WAF query aggregation and SQL pagination 2026-07-02 15:23:17 +08:00
ryan 3e910f1961 perf(clickhouse): P0 dashboard/observability query limits, cache, slower polling 2026-07-02 15:23:17 +08:00
ryan 28c14eb054 perf(clickhouse): enable async_insert and tune connection/buffer defaults 2026-07-02 15:23:17 +08:00
ryan ae618905a3 perf(clickhouse): P0 write path — remove heartbeat DELETE, batchwriter MinBatchSize, tune chwriter 2026-07-02 15:23:17 +08:00
ryan 5ad151469c feat(frontend): delete unused version-upgrade-dialog component and use-openflare-server-upgrade hook
- Permanently delete version-upgrade-dialog.tsx and use-openflare-server-upgrade.ts as they are no longer referenced after removing the version info card from openflare-ops settings.
2026-06-30 21:04:52 +08:00
ryan 6467b32d8e fix(frontend): include openflare-ops tab in whitelist and make it default
- Include 'openflare-ops' in the list of validTabs so that specifying ?tab=openflare-ops correctly loads the OpenFlare settings tab.
- Set fallback tab default to 'openflare-ops' when no tab parameter is specified.
- Document changes in changelog.
2026-06-30 20:56:34 +08:00
ryan cf72420815 feat(frontend): persist selected tab on admin settings page 2026-06-30 20:53:33 +08:00
ryan 34225cb88a fix(updater): resolve release asset name matching for openflare-server
- Update expectedAssetNames helper to match lowercase repoName prefix and lowercase repoName with -server suffix (e.g. openflare-server).
- Fixes 'no compatible release found' error when checking GitHub Action releases.
2026-06-30 20:47:32 +08:00
ryan 389f02b6b0 feat(frontend): update navigation links and expand search coverage
- Hide 'Specification Examples' and 'API Docs' from sidebar documents group, pointing 'Use Docs' externally to pages.dev.
- Complete searchData array to cover all console business pages and missing admin-only pages.
- Add changelog records for these adjustments.
2026-06-30 20:36:55 +08:00
ryan 2fcbb945fb docs: update 2026-06-30 16:52:17 +08:00
ryan 23501259b2 docs: update 2026-06-30 16:42:41 +08:00
ryan c561e65cd3 docs: update 2026-06-30 16:38:53 +08:00
ryan 97095e8f12 release: v3.0.2
### 🛠 修复
- 修复 PostgreSQL 自增主键序列在历史数据迁移(INSERT 指定显式 ID)后与实际数据不同步的问题,通过新增全局序列同步脚本一键重置所有相关表的自增计数器。
2026-06-30 16:17:21 +08:00
ryan 23be2f9296 fix(db): 新增 PostgreSQL 数据库自增序列全局同步迁移脚本
为了解决因历史数据以显式 ID 方式迁移导致 PostgreSQL 自增序列计数器不同步,产生主键冲突唯一性约束报错(如 WAF 规则组和 IP 组保存失败)的问题,在 PostgreSQL 迁移中加入了对所有相关表 pg_get_serial_sequence 重置的代码。同时,在 SQLite 中补齐了对应的同名迁移文件。
2026-06-30 16:13:37 +08:00
ryan 113ea25aa4 release: v3.0.1
### ⚡️ 优化与改进
- 新增管理后台用户个人信息编辑与重置密码功能。
- 新增用户列表邮箱列展示以及基于邮箱的搜索过滤。
- 后端新增 `reset-passwd` 命令行工具,支持通过命令行直接重置用户密码。

### 🛠 修复
- 修复添加 DNS 账号时,因直接传递类静态方法作为 React Query 的 mutationFn 导致 JavaScript 运行时丢失 `this` 上下文报错 `this.post is not a function` 的问题。
- 修复侧边栏一级菜单项当前页面字体颜色被硬编码为 `#6366F1` 的问题,改用 CSS 主题变量 `text-sidebar-primary`,以保证多主题色彩一致。
- 修复默认主题(Default)遗漏声明 `destructive-foreground` 变量,导致删除确认按钮在某些状态下渲染为黑底黑字而无法阅读的问题。
- 修复 Cobra 命令行初始化注册逻辑,确保所有应用运行模式(All, API, Worker, Scheduler)都正确注册为 Cobra 子命令。
- 优化系统设置页面的色彩定义,移除硬编码的 Indigo 靛蓝色以适配多主题切换。

### 💄 其他/体验
- 在 `AGENTS.md` 规范中新增关于防止服务类静态方法 callback 上下文丢失的开发规范指南。
2026-06-30 11:58:09 +08:00
ryan c230d5a744 fix(frontend): 解决添加DNS账号时this.post is not a function报错
在 DnsAccountCreateDialog 组件中,mutationFn 错误地直接传递了类静态方法 DnsAccountService.create,导致执行时丢失 class constructor 上下文。现将其修改为使用箭头函数包裹,以保证 this 指向正确。
2026-06-30 11:52:24 +08:00
ryan c02b649b46 fix(cmd): register all app modes as subcommands in cobra
Resolve unknown command error when launching all/api/worker/scheduler modes due to Cobra strict subcommand validation triggered by reset-passwd. Subcommands now run database migrations via dynamic PreRun hooks.
2026-06-28 11:37:50 +08:00
ryan fb54d6da61 style(frontend): remove hardcoded indigo colors in settings components
Replace all manual bg-indigo and text-indigo overrides with standard CSS variables such as bg-primary/10 and text-primary across common settings modules to support theme integration.
2026-06-28 11:31:20 +08:00
ryan 9c7896df50 feat(admin): support user profile editing, password resetting, and email column with search
- Add UpdateUser API and logics supporting nickname, email, admin flag modification, and password reset.
- Relocate user delete button and confirmation Alert into the EditUserModal.
- Optimize admin Switch change to trigger instant API request with rollback support.
- Fix missing email field in edit form initialization by fetching full profile metadata.
- Render email column in users list and support email-based filtering in UserFilterBar.
- Remove hardcoded styles and sizes from Switch components to follow global theme.
2026-06-28 11:22:11 +08:00
ryan 01ebec6dfb feat(cmd): add reset-passwd command to reset user password
- Added ./wavelet reset-passwd subcommand to reset user passwords via CLI
- Supported --user flag; if not specified, prompts for username interactively
- Supported --password flag; if not specified, generates a secure random password
- Handled access token deletion and cache invalidation
- Added comprehensive unit tests
2026-06-28 10:50:44 +08:00
ryan 2816152536 fix(frontend): adjust sidebar active text color and fix destructive button contrast
- Update sidebar active menu button text color to use theme dynamic `sidebar-primary` variable instead of hardcoded hex value.

- Add missing `destructive-foreground` variables to default theme config and styles, resolving the black-on-black text contrast issue on confirmation dialog delete buttons.

- Update changelog to track these fixes.
2026-06-28 10:34:20 +08:00
ryan b029714c7a docs: update 2026-06-28 10:11:51 +08:00
ryan e0f452eaae docs: update 2026-06-28 10:09:43 +08:00
ryan d721a8fd74 docs: update 2026-06-28 10:08:31 +08:00
ryan 1e349e5cde docs: update default credentials to admin and 12345678 in documents 2026-06-28 10:03:45 +08:00
139 changed files with 6533 additions and 1655 deletions
+21 -7
View File
@@ -1,7 +1,8 @@
# ──────────────────────────────────────────────────────────────────────────────
# openflare — 环境变量配置模板
# 复制此文件为 .env 并填入实际值: cp .env.example .env
# 环境变量优先级高于 config.yaml / config.docker.yaml
# 环境变量优先级高于 config.yaml
# docker compose 会读取本文件(env_file: .env)并替换 compose 中的 ${VAR}
# ──────────────────────────────────────────────────────────────────────────────
# ─── 时区 ─────────────────────────────────────────────────────────────────────
@@ -22,15 +23,16 @@ APP_SESSION_HTTP_ONLY=true
# HTTPS 部署时设为 true,HTTP 环境必须为 false
APP_SESSION_SECURE=true
# ─── 数据库 ────────────────────────────────────────────────────────────────────
# ─── 数据库(PostgreSQL)──────────────────────────────────────────────────────
# 设置 DB_HOST 后自动启用 PostgreSQL,也可通过 DB_ENABLED 显式控制
# DB_ENABLED=false 时使用 SQLite 作为后备数据库
DB_ENABLED=true
# SQLITE_PATH=./data/openflare.db
# compose 内应用连服务名;本机直连 Docker 映射端口时用 127.0.0.1
DB_HOST=postgres
DB_PORT=5432
DB_USERNAME=postgres
DB_PASSWORD=postgres
DB_USERNAME=openflare
DB_PASSWORD=replace-with-strong-password
DB_NAME=openflare
DB_SSL_MODE=disable
DB_TIMEZONE=Asia/Shanghai
@@ -38,7 +40,7 @@ DB_TIMEZONE=Asia/Shanghai
# DB_MAX_IDLE_CONN=16
# DB_MAX_OPEN_CONN=128
# ─── Redis ─────────────────────────────────────────────────────────────────────
# ─── Redis / Valkey ────────────────────────────────────────────────────────────
# 设置 REDIS_ADDR 后自动启用,也可通过 REDIS_ENABLED 显式控制
REDIS_ENABLED=true
REDIS_ADDR=redis:6379
@@ -47,13 +49,20 @@ REDIS_ADDR=redis:6379
# REDIS_DB=0
REDIS_KEY_PREFIX=openflare:
# REDIS_POOL_SIZE=100
# compose 宿主机映射端口(仅 docker-compose 使用)
# REDIS_PORT=6379
# ─── ClickHouse(必需)────────────────────────────────────────────────────
# ─── ClickHouse(必需)────────────────────────────────────────────────────────
# CLICKHOUSE_HOST 设置后会自动启用;测试环境可显式 CLICKHOUSE_ENABLED=true 做 live 联调
CLICKHOUSE_ENABLED=true
# compose 内:clickhouse:9000;本机连映射端口:127.0.0.1:9000
CLICKHOUSE_HOST=clickhouse:9000
CLICKHOUSE_USERNAME=default
CLICKHOUSE_PASSWORD=123456
# 须与 compose clickhouse 服务密码一致(首次初始化后改密码需清 data/clickhouse_data)
CLICKHOUSE_PASSWORD=replace-with-clickhouse-password
CLICKHOUSE_NAME=openflare
# ─── 日志 ──────────────────────────────────────────────────────────────────────
LOG_LEVEL=info
LOG_FORMAT=console
@@ -67,6 +76,11 @@ OTEL_EXPORTER_OTLP_INSECURE=true
OTEL_SAMPLING_RATE=0.0
# 全局 Tracer 命名空间,默认为 github.com/Rain-kl/OpenFlare
# OTEL_TRACER_NAME=github.com/Rain-kl/OpenFlare
# compose 可选端口覆盖
# JAEGER_VERSION=2.19.0
# JAEGER_UI_PORT=16686
# JAEGER_OTLP_GRPC_PORT=4317
# JAEGER_OTLP_HTTP_PORT=4318
# ─── Worker ────────────────────────────────────────────────────────────────────
# WORKER_CONCURRENCY=20
+1
View File
@@ -352,5 +352,6 @@ frontend/lib/services/<service-name>/
```
- 服务类继承 `BaseService`,定义 `basePath`,并暴露有类型的静态方法。
- **防止回调 `this` 上下文丢失(核心规范)**:在传递服务类的静态方法作为组件事件回调(如 `onClick`)或 React Query 的 `mutationFn`/`queryFn` 时,**禁止直接传递静态方法引用**(如 `mutationFn: DnsAccountService.create`),必须使用箭头函数包裹以防止 `this` 上下文丢失导致运行时崩溃(如 `mutationFn: (payload) => DnsAccountService.create(payload)`)。
- 在 `frontend/lib/services/index.ts` 中注册新服务。
+19 -35
View File
@@ -56,7 +56,7 @@ Quick links:
```yaml
services:
openflare:
image: ghcr.io/rain-kl/openflare-server:latest
image: ghcr.io/rain-kl/openflare:latest
restart: unless-stopped
env_file: .env
environment:
@@ -64,7 +64,7 @@ services:
ports:
- "3000:3000"
volumes:
- ./uploads:/app/uploads
- openflare_uploads:/app/uploads
depends_on:
postgres:
condition: service_healthy
@@ -77,13 +77,13 @@ services:
image: postgres:17-alpine
restart: unless-stopped
environment:
POSTGRES_DB: openflare
POSTGRES_USER: openflare
POSTGRES_PASSWORD: replace-with-strong-password
POSTGRES_DB: ${DB_NAME:-openflare}
POSTGRES_USER: ${DB_USERNAME:-openflare}
POSTGRES_PASSWORD: ${DB_PASSWORD:-replace-with-strong-password}
volumes:
- ./data/postgres_data:/var/lib/postgresql/data
- openflare_postgres_data:/var/lib/postgresql/data
healthcheck:
test: ["CMD-SHELL", "pg_isready -U openflare -d openflare"]
test: ["CMD-SHELL", "pg_isready -U ${DB_USERNAME:-openflare} -d ${DB_NAME:-openflare}"]
interval: 10s
timeout: 5s
retries: 5
@@ -93,7 +93,7 @@ services:
restart: unless-stopped
command: ["valkey-server", "--appendonly", "yes"]
volumes:
- ./data/valkey:/data
- openflare_redis_data:/data
healthcheck:
test: ["CMD", "valkey-cli", "ping"]
interval: 10s
@@ -105,19 +105,25 @@ services:
image: clickhouse/clickhouse-server:25.3-alpine
restart: unless-stopped
environment:
CLICKHOUSE_DB: openflare
CLICKHOUSE_USER: default
CLICKHOUSE_PASSWORD: 123456
CLICKHOUSE_DB: ${CLICKHOUSE_NAME:-openflare}
CLICKHOUSE_USER: ${CLICKHOUSE_USERNAME:-default}
CLICKHOUSE_PASSWORD: ${CLICKHOUSE_PASSWORD:-replace-with-clickhouse-password}
CLICKHOUSE_DEFAULT_ACCESS_MANAGEMENT: 1
TZ: ${TZ:-Asia/Shanghai}
volumes:
- ./data/clickhouse_data:/var/lib/clickhouse
- openflare_clickhouse_data:/var/lib/clickhouse
healthcheck:
test: ["CMD", "clickhouse-client", "--query", "SELECT 1"]
test: ["CMD", "clickhouse-client", "--user", "${CLICKHOUSE_USERNAME:-default}", "--password", "${CLICKHOUSE_PASSWORD:-replace-with-clickhouse-password}", "--query", "SELECT 1"]
interval: 10s
timeout: 5s
retries: 5
start_period: 15s
volumes:
openflare_uploads:
openflare_postgres_data:
openflare_redis_data:
openflare_clickhouse_data:
```
```bash
@@ -204,28 +210,6 @@ The version number format is fixed as `YYYYMMDD-NNN`. Historical versions are im
![OpenFlare version release](./docs/assets/readme/proxy-route-detail.png)
## Management Panel & API
The management panel includes:
* Reverse Proxy Rules
* Configuration Versions
* Node Management
* Application Records
* TLS Certificates
* Domain Management
* Pages Static Hosting
* WAF Rule Groups
* Intranet Tunnels
* Uptime Kuma Monitoring Sync
* SSO Login Configuration
* User Management
* Settings
* Version Updates
* PoW Rules
After logging in to the dashboard, access Swagger UI at: `/swagger/index.html`
## License
This project is licensed under [Apache License 2.0](./LICENSE).
+52 -52
View File
@@ -21,7 +21,7 @@ OpenFlare 是开源 CDN 编排与边缘安全平台。它支持反向代理、
</p>
> [!WARNING]
> 使用 `root` 用户初次登录系统后,务必修改默认密码 `123456`。
> 使用 `admin` 用户初次登录系统后,务必修改默认密码 `12345678`。
>
> BETA 版本为开发测试阶段的临时产物,可能存在未知问题,请勿在生产环境使用。
@@ -48,16 +48,41 @@ OpenFlare 是开源 CDN 编排与边缘安全平台。它支持反向代理、
* **SSO 单点登录**:支持 GitHub OAuth 与标准 OIDC 协议,无缝接入企业身份提供商实现统一登录。
* **统一观测**:聚合节点请求指标、实时访问日志明细、宿主机与 Nginx 资源快照、健康事件以及网络波动补传缓冲。
## 界面预览
### 仪表盘总览
![OpenFlare dashboard overview](./docs/assets/readme/dashboard-overview.png)
### 节点详情
![OpenFlare node detail](./docs/assets/readme/node-detail.png)
### 配置新增
![OpenFlare version release](./docs/assets/readme/proxy-route-detail.png)
## 快速开始
### 1. 启动 Server
使用 docker-compose
```bash
# 下载环境变量模板并创建 .env 文件
curl -o .env.example https://raw.githubusercontent.com/Rain-kl/OpenFlare/refs/heads/main/.env.example
cp .env.example .env
# ClickHouse 服务端:curl performance.xml 到 ./config/clickhouse,整目录挂载到 config.d(不要放 listen 配置)
mkdir -p ./config/clickhouse
curl -fsSL -o ./config/clickhouse/performance.xml \
https://raw.githubusercontent.com/Rain-kl/OpenFlare/refs/heads/main/config/clickhouse/performance.xml
```
```yaml
services:
openflare:
image: ghcr.io/rain-kl/openflare-server:latest
image: ghcr.io/rain-kl/openflare:latest
restart: unless-stopped
env_file: .env
environment:
@@ -65,7 +90,7 @@ services:
ports:
- "3000:3000"
volumes:
- ./uploads:/app/uploads
- openflare_uploads:/app/uploads
depends_on:
postgres:
condition: service_healthy
@@ -78,13 +103,13 @@ services:
image: postgres:17-alpine
restart: unless-stopped
environment:
POSTGRES_DB: openflare
POSTGRES_USER: openflare
POSTGRES_PASSWORD: replace-with-strong-password
POSTGRES_DB: ${DB_NAME:-openflare}
POSTGRES_USER: ${DB_USERNAME:-openflare}
POSTGRES_PASSWORD: ${DB_PASSWORD:-replace-with-strong-password}
volumes:
- ./data/postgres_data:/var/lib/postgresql/data
- openflare_postgres_data:/var/lib/postgresql/data
healthcheck:
test: ["CMD-SHELL", "pg_isready -U openflare -d openflare"]
test: ["CMD-SHELL", "pg_isready -U ${DB_USERNAME:-openflare} -d ${DB_NAME:-openflare}"]
interval: 10s
timeout: 5s
retries: 5
@@ -94,7 +119,7 @@ services:
restart: unless-stopped
command: ["valkey-server", "--appendonly", "yes"]
volumes:
- ./data/valkey:/data
- openflare_redis_data:/data
healthcheck:
test: ["CMD", "valkey-cli", "ping"]
interval: 10s
@@ -106,19 +131,30 @@ services:
image: clickhouse/clickhouse-server:25.3-alpine
restart: unless-stopped
environment:
CLICKHOUSE_DB: openflare
CLICKHOUSE_USER: default
CLICKHOUSE_PASSWORD: 123456
CLICKHOUSE_DB: ${CLICKHOUSE_NAME:-openflare}
CLICKHOUSE_USER: ${CLICKHOUSE_USERNAME:-default}
CLICKHOUSE_PASSWORD: ${CLICKHOUSE_PASSWORD:-replace-with-clickhouse-password}
CLICKHOUSE_DEFAULT_ACCESS_MANAGEMENT: 1
TZ: ${TZ:-Asia/Shanghai}
ulimits:
nofile:
soft: 262144
hard: 262144
volumes:
- ./data/clickhouse_data:/var/lib/clickhouse
- openflare_clickhouse_data:/var/lib/clickhouse
- ./config/clickhouse:/etc/clickhouse-server/config.d:ro
healthcheck:
test: ["CMD", "clickhouse-client", "--query", "SELECT 1"]
test: ["CMD", "clickhouse-client", "--user", "${CLICKHOUSE_USERNAME:-default}", "--password", "${CLICKHOUSE_PASSWORD:-replace-with-clickhouse-password}", "--query", "SELECT 1"]
interval: 10s
timeout: 5s
retries: 5
start_period: 15s
volumes:
openflare_uploads:
openflare_postgres_data:
openflare_redis_data:
openflare_clickhouse_data:
```
详细部署说明见 [部署文档](https://open-flare.pages.dev/deployment/deployment)。
@@ -127,8 +163,8 @@ services:
默认账号:
* 用户名:`root`
* 密码:`123456`
* 用户名:`admin`
* 密码:`12345678`
### 2. 安装 Agent
@@ -150,42 +186,6 @@ docker run -d --name openflare-agent --restart unless-stopped \
ghcr.io/rain-kl/openflare-agent:latest
```
## 界面预览
### 仪表盘总览
![OpenFlare dashboard overview](./docs/assets/readme/dashboard-overview.png)
### 节点详情
![OpenFlare node detail](./docs/assets/readme/node-detail.png)
### 配置新增
![OpenFlare version release](./docs/assets/readme/proxy-route-detail.png)
## 管理端与接口
管理端当前覆盖:
* 反代规则
* 配置版本
* 节点管理
* 应用记录
* TLS 证书
* 域名管理
* Pages 静态托管
* WAF 规则组
* 内网穿透(Tunnels)
* Uptime Kuma 监控同步
* SSO 登录配置
* 用户管理
* 设置
* 版本更新
* PoW 规则
登录管理端后,可访问 Swagger UI:`/swagger/index.html`
## 开源协议
本项目采用 [Apache License 2.0](./LICENSE) 开源。
+8 -5
View File
@@ -99,15 +99,18 @@ otel:
# ─── ClickHouse (required) ──────────────────────────────────────────────────────
# Analytics / observability OLAP store. Telemetry writes are best-effort (async batch).
clickhouse:
enabled: true
hosts:
- "127.0.0.1:9000"
- "127.0.0.1:9000" # compose 内应用可用 clickhouse:9000(经 CLICKHOUSE_HOST)
username: "default"
password: "123456"
password: "replace-with-clickhouse-password" # 与 .env / compose CLICKHOUSE_PASSWORD 一致
database: "openflare"
max_idle_conn: 10
max_open_conn: 100
max_idle_conn: 8 # keep warm sockets low to save client + server RAM
max_open_conn: 16 # cap concurrent native sessions on modest CH boxes
conn_max_lifetime: 3600
dial_timeout: 5
block_buffer_size: 10
block_buffer_size: 32 # rows buffered per block; 32 is enough for our batch sizes
# Runtime client also enables async_insert (wait_for_async_insert=1, busy_timeout≈2s)
# in internal/db/clickhouse.go — not configured via YAML.
+25
View File
@@ -0,0 +1,25 @@
<?xml version="1.0"?>
<!--
Tuned for small control-plane hosts (e.g. 3c6g).
background_pool_size * background_merges_mutations_concurrency_ratio must stay
greater than merge_tree number_of_free_entries_in_pool_to_execute_mutation
(ClickHouse 25.x refuses to start otherwise). Keep the merge free-entry
thresholds low so a small pool remains valid.
-->
<clickhouse>
<max_concurrent_queries>20</max_concurrent_queries>
<background_pool_size>4</background_pool_size>
<background_merges_mutations_concurrency_ratio>2</background_merges_mutations_concurrency_ratio>
<background_schedule_pool_size>4</background_schedule_pool_size>
<background_common_pool_size>2</background_common_pool_size>
<background_fetches_pool_size>2</background_fetches_pool_size>
<background_move_pool_size>1</background_move_pool_size>
<mark_cache_size>268435456</mark_cache_size>
<uncompressed_cache_size>0</uncompressed_cache_size>
<merge_tree>
<number_of_free_entries_in_pool_to_execute_mutation>2</number_of_free_entries_in_pool_to_execute_mutation>
<number_of_free_entries_in_pool_to_lower_max_size_of_merge>2</number_of_free_entries_in_pool_to_lower_max_size_of_merge>
<number_of_free_entries_in_pool_to_execute_optimize_entire_partition>2</number_of_free_entries_in_pool_to_execute_optimize_entire_partition>
</merge_tree>
</clickhouse>
+16 -11
View File
@@ -5,7 +5,7 @@ services:
dockerfile: docker/Dockerfile
args:
VERSION: v0.9.9
# image: ghcr.io/rain-kl/openflare-server:latest
# image: ghcr.io/rain-kl/openflare:latest
restart: unless-stopped
env_file: .env
environment:
@@ -34,13 +34,13 @@ services:
ports:
- "5432:5432"
environment:
POSTGRES_DB: openflare
POSTGRES_USER: openflare
POSTGRES_PASSWORD: replace-with-strong-password
POSTGRES_DB: ${DB_NAME:-openflare}
POSTGRES_USER: ${DB_USERNAME:-openflare}
POSTGRES_PASSWORD: ${DB_PASSWORD:-replace-with-strong-password}
volumes:
- ./data/postgres_data:/var/lib/postgresql/data
healthcheck:
test: ["CMD-SHELL", "pg_isready -U openflare -d openflare"]
test: ["CMD-SHELL", "pg_isready -U ${DB_USERNAME:-openflare} -d ${DB_NAME:-openflare}"]
interval: 10s
timeout: 5s
retries: 5
@@ -74,18 +74,23 @@ services:
image: clickhouse/clickhouse-server:25.3-alpine
restart: unless-stopped
environment:
CLICKHOUSE_DB: openflare
CLICKHOUSE_USER: default
CLICKHOUSE_PASSWORD: 123456
CLICKHOUSE_DB: ${CLICKHOUSE_NAME:-openflare}
CLICKHOUSE_USER: ${CLICKHOUSE_USERNAME:-default}
CLICKHOUSE_PASSWORD: ${CLICKHOUSE_PASSWORD:-replace-with-clickhouse-password}
CLICKHOUSE_DEFAULT_ACCESS_MANAGEMENT: 1
TZ: ${TZ:-Asia/Shanghai}
ulimits:
nofile:
soft: 262144
hard: 262144
ports:
- "${CLICKHOUSE_HTTP_PORT:-8123}:8123"
- "${CLICKHOUSE_NATIVE_PORT:-9000}:9000"
- "8123:8123"
- "9000:9000"
volumes:
- ./data/clickhouse_data:/var/lib/clickhouse
- ./config/clickhouse:/etc/clickhouse-server/config.d:ro
healthcheck:
test: ["CMD", "clickhouse-client", "--query", "SELECT 1"]
test: ["CMD", "clickhouse-client", "--user", "${CLICKHOUSE_USERNAME:-default}", "--password", "${CLICKHOUSE_PASSWORD:-replace-with-clickhouse-password}", "--query", "SELECT 1"]
interval: 10s
timeout: 5s
retries: 5
Binary file not shown.

Before

Width:  |  Height:  |  Size: 350 KiB

After

Width:  |  Height:  |  Size: 141 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 272 KiB

After

Width:  |  Height:  |  Size: 131 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 147 KiB

After

Width:  |  Height:  |  Size: 64 KiB

+69
View File
@@ -11,6 +11,9 @@ sidebar: false
## 重大变更
> [!IMPORTANT]
>
> 3.1.2 版本更新了 CLickHouse 部署配置。
>
> 3.0.0 版本为 Wavelet 平台迁移与架构重构版本,涉及数据库表结构、环境变量以及前后端底层架构的重大变更。请务必在升级前备份数据库,并且更新到 V2.3.4。
> 目前已知的兼容性问题:
> - Pages 无法迁移, 升级前请先手动下载并备份 Pages 静态站点的 ZIP 包,升级后重新创建。
@@ -18,6 +21,72 @@ sidebar: false
## [unreleased]
## [v3.1.2] - 2026-07-10
### 修复
- 修复节点/仪表盘 24 小时容量、网络、磁盘 IO 趋势在 ClickHouse 限流查询下几乎为空的问题:改为基于小时级聚合与计数器 delta 统计,避免仅依赖最近有限条原始快照导致历史时段全空。
- 降低静置时 ClickHouse CPU:可观测/访问日志 batchwriter 启用 `MinBatchSize` 与 `MaxFlushWait`,减少心跳小 part 写入;Docker `performance.xml` 收紧小规格后台 merge 池。
- ClickHouse 清理语义:按保留天数仅 `MATERIALIZE` 表 DDL TTL,`deleted_count` 不再伪报删除;短于表 TTL 的保留请求被拒绝。
- 可观测 dedup 仅在入队成功后保留,flush 失败释放键并短重试;审计 writer 增加 `MaxFlushWait`;`/admin/status/clickhouse` 暴露 batch writer 队列深度/丢弃/flush 错误。
- model 层通过 hooks 写入 CH,去除对 `chwriter` 的直接依赖。
- Dashboard 每节点最新指标改为 `LIMIT 1 BY node_id`;新增 metric/openresty 小时预聚合表;读路径按小时 merge(rollup 窗口完整时仅走预聚合,不足时用 raw 补洞),并提供历史 backfill 迁移。
- 小规格默认连接池下调;`async_insert_busy_timeout` 调至 2s;`of_node_traffic_hourly` 增加 30 天 TTL,UV 改为峰值窗口估计并修正前端文案。
- Docker ClickHouse:`performance.xml` 下调 merge free-entry 阈值以兼容小 `background_pool`(避免 25.x 启动 Code 36)。
### 文档
- 同步 `.env.example` 与 `config.example.yaml`;ClickHouse 服务端配置改为 curl `performance.xml` 到 `./config/clickhouse` 后整目录挂载至 `config.d`(不要放入 listen 配置)。
## [v3.1.1] - 2026-07-06
### 修改
- 将 `cap_login_enabled` 默认值由 `true` 变更为 `false`,默认关闭登录界面 PoW 人机验证。
## [v3.1.0] - 2026-07-04
### 修改
- 修复 ClickHouse TTL 迁移:`DateTime64` 时间列通过 `toDateTime()` 转换后再设置 TTL,避免 goose 启动报错;移除 `MODIFY ORDER BY`(ClickHouse 不允许将排序键缩短至短于隐式主键前缀)。
- ClickHouse 遗留治理 Phase 2:保留期清理改为 TTL `MATERIALIZE TTL`(全量清理使用 `TRUNCATE`),消除定时 `ALTER DELETE` mutation;移除 GORM 双连接池并统一 `ChConn` 读路径;查询侧去除 `trim(remote_addr)`;`wait_for_async_insert` 调整为 1;新增 `/admin/status/clickhouse` 运维指标与 `of_node_traffic_hourly` 预聚合 MV。
- ClickHouse 写入路径优化:移除 Agent 心跳路径中的同步 `ALTER DELETE` 保留清理;`batchwriter` 新增 `MinBatchSize` 抑制过小批次定时 flush;可观测 writer 批次提升至 500、flush 间隔 5s,并为 OpenResty/FRPS/FRPC 补全去重。
- ClickHouse 客户端启用 `async_insert` 异步写入缓冲,并调高 `block_buffer_size` 与连接池默认值,降低小 part 与连接争用。
- Dashboard 与节点可观测 API 消除无 `LIMIT` 全表扫描、增加短 TTL 内存缓存,前端轮询间隔分别调整为 60s/30s。
- 访问日志与 WAF IP 组同步改为 ClickHouse 侧聚合与 SQL 分页,默认查询窗口限制为近 7 天,浏览器分布查询增加 Top 100 限制。
- ClickHouse 分析表新增 TTL 自动过期策略:`w_user_access_logs` 180 天、`of_node_access_logs` 90 天,其余节点观测与聚合表 30 天。
- 节点访问日志写入 ClickHouse 时对 `remote_addr` 执行 `TrimSpace` 规范化,避免首尾空白影响 IP 汇总统计。
- 数据库自动清理任务新增 OpenResty、FRPS、FRPC 观测表清理目标。
- Docker 部署为 ClickHouse 服务增加 `nofile` ulimits 与 `docker/clickhouse/config.d/performance.xml` 性能配置挂载,限制 `max_concurrent_queries`、`background_pool_size` 与 `background_merges_mutations_concurrency_ratio`,降低高负载下的合并与查询争用。
- 审计访问日志写入 ClickHouse 时仅保留安全相关请求头(Authorization、Cookie、X-Forwarded-For、X-Real-IP、User-Agent、Content-Type),敏感头字段以 SHA-256 摘要脱敏,并将序列化后的 headers 载荷上限收紧至 2KB,减小 `w_user_access_logs` 行宽与 merge CPU 开销。
- 隐藏侧边栏“文档库”分组中的“规范示例”与“接口文档”,并将“使用文档”及其他相关页面的文档链接统一跳转至外部文档 https://open-flare.pages.dev/
- 修复全局搜索数据源覆盖不全的问题,补全了所有核心业务控制台页面(节点、规则、域名、证书、DNS、源站、WAF、IP组、Pages、版本发布、访问日志、应用记录和性能调优)及缺失的管理员专有页面(存储、数据、推送、日志)的搜索检索支持。
- 修复系统自更新(Updater)检测上游 GitHub Action Release 时,因资产包名称前缀(`openflare-server`)与仓库名不完全一致导致匹配失败并报错“未找到兼容的 Release”的问题。
- 修复系统设置页面(`/admin/settings`)基于 URL `tab` 参数的定位逻辑,补全缺失的 `openflare-ops` (OpenFlare) Tab,且在不带参数时默认选中 OpenFlare 选项卡。
- 移除系统设置中 OpenFlare 标签页下的“版本信息”卡片及对应的升级管理弹窗逻辑。
## [v3.0.2] - 2026-06-30
### 修复
- 修复 PostgreSQL 自增主键序列在历史数据迁移(INSERT 指定显式 ID)后与实际数据不同步的问题,通过新增全局序列同步脚本一键重置所有相关表的自增计数器。
## [v3.0.1] - 2026-06-30
### 新增
- 新增管理后台用户个人信息编辑与重置密码功能。
- 新增用户列表邮箱列展示以及基于邮箱的搜索过滤。
- 后端新增 `reset-passwd` 命令行工具,支持通过命令行直接重置用户密码。
### 修复
- 修复添加 DNS 账号时因直接传递 class static 方法作为 React Query 的 mutationFn 导致 JavaScript 丢失 `this` 上下文报错 `this.post is not a function` 的问题。
- 修复侧边栏一级菜单项当前页面字体颜色被硬编码为 `#6366F1` 的问题,改用 CSS 主题变量 `text-sidebar-primary`,以保证在多主题系统下的色彩一致性。
- 修复默认(Default)主题因遗漏声明 `destructive-foreground` 变量,导致删除按钮(如确认删除证书弹窗)在某些状态下渲染为黑底黑字而无法阅读的问题。
- 修复 Cobra 命令行初始化注册逻辑,确保所有应用运行模式(All, API, Worker, Scheduler)都正确注册为 Cobra 子命令。
- 优化系统设置页面的色彩定义,移除硬编码的 Indigo 靛蓝色以适配多主题切换。
## [v3.0.0] - 2026-06-27
### 升级与迁移注意事项
+2 -40
View File
@@ -81,6 +81,7 @@ Agent:
仓库根目录已提供完整 `docker-compose.yaml`(含 PostgreSQL、Redis、ClickHouse、Jaeger)。
```bash
curl -o .env.example https://raw.githubusercontent.com/Rain-kl/OpenFlare/refs/heads/main/.env.example
cp .env.example .env
# 编辑 .env,至少修改 APP_SESSION_SECRET 与数据库密码
docker compose up -d
@@ -88,7 +89,7 @@ docker compose ps
docker compose logs -f openflare
```
首次访问 `http://localhost:3000`,默认账号为 `root` / `123456`。登录后请立即修改默认密码。
首次访问 `http://localhost:3000`,默认账号为 `admin` / `12345678`。登录后请立即修改默认密码。
## 源码启动 Server
@@ -117,22 +118,6 @@ go run main.go all
Docker 部署是 Agent 推荐的部署方式。Docker 部署时直接运行 Agent 镜像,该镜像基于 OpenResty 镜像制作,内置 Agent 控制器与 OpenResty 二进制。未显式配置 `node_ip` 时,Agent 会优先通过第三方 API 获取真实出口 IP,避免把 Docker 网桥地址登记为节点 IP。
> [!NOTE]
> Agent 镜像已完成非 Root 安全加固,统一以普通用户 `openflare` 权限运行,通过内核 capabilities 授权(`cap_net_bind_service`)监听 80/443 特权端口,并自动重定向临时文件和 PID 路径至容器内 `/data` 目录以防止写入冲突。
挂载配置文件:
```bash
docker pull ghcr.io/rain-kl/openflare-agent:latest
docker rm -f openflare-agent 2>/dev/null || true
docker run -d --name openflare-agent --restart unless-stopped \
-p 80:80 -p 443:443/tcp -p 443:443/udp \
-v ./agent.json:/etc/openflare/agent.json:ro \
ghcr.io/rain-kl/openflare-agent:latest
```
使用环境变量:
```bash
docker pull ghcr.io/rain-kl/openflare-agent:latest
docker rm -f openflare-agent 2>/dev/null || true
@@ -219,26 +204,3 @@ export LOG_LEVEL='info'
默认情况下,Agent 在 HTTP 心跳成功后会尝试升级为 WebSocket。升级成功时,Server 发布或激活配置会立即通知 Agent;如果 WebSocket 无法建立或意外断开,Agent 会自动退回 HTTP 心跳同步。
WAF 地域规则依赖 Agent 本地 `GeoLite2-Country.mmdb`。Agent 启动时会在 `data_dir/etc/openflare/GeoLite2-Country.mmdb` 初始化内置数据库,并按配置周期尝试更新;更新失败只记录警告,不影响配置同步与 OpenResty reload。
## 升级与卸载
Server:
* Root 用户可在管理端顶栏检查并升级正式版。
* 如需尝试 preview 版本,可手动检查对应发布。
* 也可通过上传 Server 二进制的方式执行确认升级。
Agent:
* Agent 默认只跟随正式版自动更新。
* Agent 自更新从 GitHub Release 拉取目标二进制,优先使用 Release API 的 `digest` 字段做 SHA-256 校验;仅当 digest 为空(历史 Release)时才回退读取同名 `.sha256` 侧车文件,校验通过后才替换本地可执行文件。
* 安装脚本可重复执行,用于重装或升级 Agent。
* preview 升级需要手动触发。
卸载 Agent:
```bash
curl -fsSL https://raw.githubusercontent.com/Rain-kl/OpenFlare/main/scripts/uninstall-agent.sh | bash
```
卸载脚本会停止 Agent、删除 systemd 服务和安装目录,不会删除本机 OpenResty。
+92 -30
View File
@@ -8,6 +8,30 @@ OpenFlare Server 是 Gin + GORM 单体控制面,负责管理端 UI、管理 AP
> **关于外部依赖**:
> OpenFlare 系统内建了对后台异步任务(Asynq 框架)及海量节点日志分析与度量指标(观测面板)的支持。因此,**无论采用何种部署模式,系统都必须依赖 Redis(或 Valkey)与 ClickHouse 的运行**。各个部署方案的主要差异在于主关系型数据库的选择(SQLite vs PostgreSQL)以及是否启用链路追踪服务(Jaeger)。
> [!TIP]
> **ClickHouse 服务端性能配置(推荐挂载)**
> 控制面常见为小规格主机(如 3c6g)。仓库提供的 `performance.xml` 会收紧后台 merge/mutation 线程池,避免默认配置在小机器上静置 CPU 偏高或 ClickHouse 25.x 启动校验失败。
> 将本地目录 `./config/clickhouse` 挂载到容器 `/etc/clickhouse-server/config.d`。
> **目录内只放 `performance.xml`,不要放入任何 listen 相关配置**(监听地址沿用官方镜像默认即可)。
部署前将配置拉到本地:
```bash
mkdir -p ./config/clickhouse
curl -fsSL -o ./config/clickhouse/performance.xml \
https://raw.githubusercontent.com/Rain-kl/OpenFlare/refs/heads/main/config/clickhouse/performance.xml
```
在 ClickHouse 服务的 `volumes` 中增加(与数据卷并列):
```yaml
volumes:
- ./data/clickhouse_data:/var/lib/clickhouse # 或 named volume
- ./config/clickhouse:/etc/clickhouse-server/config.d:ro
```
修改 `performance.xml` 后需 `docker compose restart clickhouse` 才生效。
---
## 方式一:Docker 部署 (推荐)
@@ -27,7 +51,7 @@ version: '3.8'
services:
openflare:
image: ghcr.io/rain-kl/openflare-server:latest
image: ghcr.io/rain-kl/openflare:latest
container_name: openflare-server
restart: unless-stopped
ports:
@@ -71,10 +95,15 @@ services:
CLICKHOUSE_PASSWORD: 123456
CLICKHOUSE_DEFAULT_ACCESS_MANAGEMENT: 1
TZ: Asia/Shanghai
ulimits:
nofile:
soft: 262144
hard: 262144
volumes:
- ./data/clickhouse_data:/var/lib/clickhouse
- ./config/clickhouse:/etc/clickhouse-server/config.d:ro
healthcheck:
test: ["CMD", "clickhouse-client", "--query", "SELECT 1"]
test: ["CMD", "clickhouse-client", "--user", "default", "--password", "123456", "--query", "SELECT 1"]
interval: 10s
timeout: 5s
retries: 5
@@ -84,6 +113,9 @@ services:
运行启动命令:
```bash
mkdir -p ./config/clickhouse
curl -fsSL -o ./config/clickhouse/performance.xml \
https://raw.githubusercontent.com/Rain-kl/OpenFlare/refs/heads/main/config/clickhouse/performance.xml
docker compose up -d
```
@@ -100,7 +132,7 @@ docker compose up -d
```yaml
services:
openflare:
image: ghcr.io/rain-kl/openflare-server:latest
image: ghcr.io/rain-kl/openflare:latest
restart: unless-stopped
env_file: .env
environment:
@@ -108,7 +140,7 @@ services:
ports:
- "3000:3000"
volumes:
- ./uploads:/app/uploads
- openflare_uploads:/app/uploads
depends_on:
postgres:
condition: service_healthy
@@ -121,13 +153,13 @@ services:
image: postgres:17-alpine
restart: unless-stopped
environment:
POSTGRES_DB: openflare
POSTGRES_USER: openflare
POSTGRES_PASSWORD: replace-with-strong-password
POSTGRES_DB: ${DB_NAME:-openflare}
POSTGRES_USER: ${DB_USERNAME:-openflare}
POSTGRES_PASSWORD: ${DB_PASSWORD:-replace-with-strong-password}
volumes:
- ./data/postgres_data:/var/lib/postgresql/data
- openflare_postgres_data:/var/lib/postgresql/data
healthcheck:
test: ["CMD-SHELL", "pg_isready -U openflare -d openflare"]
test: ["CMD-SHELL", "pg_isready -U ${DB_USERNAME:-openflare} -d ${DB_NAME:-openflare}"]
interval: 10s
timeout: 5s
retries: 5
@@ -137,7 +169,7 @@ services:
restart: unless-stopped
command: ["valkey-server", "--appendonly", "yes"]
volumes:
- ./data/valkey:/data
- openflare_redis_data:/data
healthcheck:
test: ["CMD", "valkey-cli", "ping"]
interval: 10s
@@ -149,24 +181,39 @@ services:
image: clickhouse/clickhouse-server:25.3-alpine
restart: unless-stopped
environment:
CLICKHOUSE_DB: openflare
CLICKHOUSE_USER: default
CLICKHOUSE_PASSWORD: replace-with-clickhouse-password
CLICKHOUSE_DB: ${CLICKHOUSE_NAME:-openflare}
CLICKHOUSE_USER: ${CLICKHOUSE_USERNAME:-default}
CLICKHOUSE_PASSWORD: ${CLICKHOUSE_PASSWORD:-replace-with-clickhouse-password}
CLICKHOUSE_DEFAULT_ACCESS_MANAGEMENT: 1
TZ: ${TZ:-Asia/Shanghai}
ulimits:
nofile:
soft: 262144
hard: 262144
volumes:
- ./data/clickhouse_data:/var/lib/clickhouse
- openflare_clickhouse_data:/var/lib/clickhouse
- ./config/clickhouse:/etc/clickhouse-server/config.d:ro
healthcheck:
test: ["CMD", "clickhouse-client", "--query", "SELECT 1"]
test: ["CMD", "clickhouse-client", "--user", "${CLICKHOUSE_USERNAME:-default}", "--password", "${CLICKHOUSE_PASSWORD:-replace-with-clickhouse-password}", "--query", "SELECT 1"]
interval: 10s
timeout: 5s
retries: 5
start_period: 15s
volumes:
openflare_uploads:
openflare_postgres_data:
openflare_redis_data:
openflare_clickhouse_data:
```
创建对应的 `.env` 文件来配置系统环境变量(可复制并修改根目录下的 `.env.example`):
```bash
mkdir -p ./config/clickhouse
curl -fsSL -o ./config/clickhouse/performance.xml \
https://raw.githubusercontent.com/Rain-kl/OpenFlare/refs/heads/main/config/clickhouse/performance.xml
curl -o .env.example https://raw.githubusercontent.com/Rain-kl/OpenFlare/refs/heads/main/.env.example
cp .env.example .env
# 编辑 .env 文件,填入对应的数据库、Redis、ClickHouse 连接地址、密码与 APP_SESSION_SECRET
@@ -188,7 +235,7 @@ version: '3.8'
services:
openflare:
image: ghcr.io/rain-kl/openflare-server:latest
image: ghcr.io/rain-kl/openflare:latest
restart: unless-stopped
env_file: .env
environment:
@@ -199,7 +246,7 @@ services:
ports:
- "3000:3000"
volumes:
- ./uploads:/app/uploads
- openflare_uploads:/app/uploads
depends_on:
postgres:
condition: service_healthy
@@ -214,13 +261,13 @@ services:
image: postgres:17-alpine
restart: unless-stopped
environment:
POSTGRES_DB: openflare
POSTGRES_USER: openflare
POSTGRES_PASSWORD: replace-with-strong-password
POSTGRES_DB: ${DB_NAME:-openflare}
POSTGRES_USER: ${DB_USERNAME:-openflare}
POSTGRES_PASSWORD: ${DB_PASSWORD:-replace-with-strong-password}
volumes:
- ./data/postgres_data:/var/lib/postgresql/data
- openflare_postgres_data:/var/lib/postgresql/data
healthcheck:
test: ["CMD-SHELL", "pg_isready -U openflare -d openflare"]
test: ["CMD-SHELL", "pg_isready -U ${DB_USERNAME:-openflare} -d ${DB_NAME:-openflare}"]
interval: 10s
timeout: 5s
retries: 5
@@ -230,7 +277,7 @@ services:
restart: unless-stopped
command: ["valkey-server", "--appendonly", "yes"]
volumes:
- ./data/valkey:/data
- openflare_redis_data:/data
healthcheck:
test: ["CMD", "valkey-cli", "ping"]
interval: 10s
@@ -252,24 +299,39 @@ services:
image: clickhouse/clickhouse-server:25.3-alpine
restart: unless-stopped
environment:
CLICKHOUSE_DB: openflare
CLICKHOUSE_USER: default
CLICKHOUSE_PASSWORD: replace-with-clickhouse-password
CLICKHOUSE_DB: ${CLICKHOUSE_NAME:-openflare}
CLICKHOUSE_USER: ${CLICKHOUSE_USERNAME:-default}
CLICKHOUSE_PASSWORD: ${CLICKHOUSE_PASSWORD:-replace-with-clickhouse-password}
CLICKHOUSE_DEFAULT_ACCESS_MANAGEMENT: 1
TZ: ${TZ:-Asia/Shanghai}
ulimits:
nofile:
soft: 262144
hard: 262144
volumes:
- ./data/clickhouse_data:/var/lib/clickhouse
- openflare_clickhouse_data:/var/lib/clickhouse
- ./config/clickhouse:/etc/clickhouse-server/config.d:ro
healthcheck:
test: ["CMD", "clickhouse-client", "--query", "SELECT 1"]
test: ["CMD", "clickhouse-client", "--user", "${CLICKHOUSE_USERNAME:-default}", "--password", "${CLICKHOUSE_PASSWORD:-replace-with-clickhouse-password}", "--query", "SELECT 1"]
interval: 10s
timeout: 5s
retries: 5
start_period: 15s
volumes:
openflare_uploads:
openflare_postgres_data:
openflare_redis_data:
openflare_clickhouse_data:
```
启动并验证:
```bash
mkdir -p ./config/clickhouse
curl -fsSL -o ./config/clickhouse/performance.xml \
https://raw.githubusercontent.com/Rain-kl/OpenFlare/refs/heads/main/config/clickhouse/performance.xml
curl -o .env.example https://raw.githubusercontent.com/Rain-kl/OpenFlare/refs/heads/main/.env.example
cp .env.example .env
# 编辑 .env 文件并确保设置好 APP_SESSION_SECRET 密码
@@ -290,7 +352,7 @@ docker compose up -d
| Go | `1.25+` |
| Node.js | `18+` |
| pnpm | 推荐通过 `corepack enable` 使用项目声明的 pnpm |
| 外部服务 | 必须在本地或远端运行 Redis (Valkey) 和 ClickHouse 实例 |
| 外部服务 | 必须在本地或远端运行 Redis (Valkey) 和 ClickHouse 实例;ClickHouse 建议挂载仓库提供的 `performance.xml`(见上文「ClickHouse 服务端性能配置」) |
### 1. 构建管理端前端
@@ -345,7 +407,7 @@ Server 默认监听 `3000` 端口,启动成功后可以使用浏览器访问
| 用户名 | 密码 |
| --- | --- |
| `root` | `123456` |
| `admin` | `12345678` |
> [!WARNING]
> 为了你的系统安全,首次登录后请立即前往个人设置页面修改默认密码。
+219 -5
View File
@@ -309,6 +309,7 @@ const docTemplate = `{
"parameters": [
{
"type": "integer",
"format": "int64",
"description": "认证源 ID 或名称",
"name": "id",
"in": "path",
@@ -386,6 +387,7 @@ const docTemplate = `{
"parameters": [
{
"type": "integer",
"format": "int64",
"description": "认证源 ID 或名称",
"name": "id",
"in": "path",
@@ -453,6 +455,7 @@ const docTemplate = `{
"parameters": [
{
"type": "integer",
"format": "int64",
"description": "认证源 ID 或名称",
"name": "id",
"in": "path",
@@ -1363,6 +1366,7 @@ const docTemplate = `{
"parameters": [
{
"type": "integer",
"format": "int64",
"description": "通道ID",
"name": "id",
"in": "path",
@@ -1416,6 +1420,7 @@ const docTemplate = `{
"parameters": [
{
"type": "integer",
"format": "int64",
"description": "通道ID",
"name": "id",
"in": "path",
@@ -1872,6 +1877,67 @@ const docTemplate = `{
}
}
},
"/api/v1/admin/status/clickhouse": {
"get": {
"security": [
{
"SessionCookie": []
}
],
"description": "返回 ClickHouse parts、mutation、async_insert 队列等运维指标,需要管理员权限",
"produces": [
"application/json"
],
"tags": [
"admin"
],
"summary": "获取 ClickHouse 运行指标",
"responses": {
"200": {
"description": "获取成功",
"schema": {
"allOf": [
{
"$ref": "#/definitions/response.Any"
},
{
"type": "object",
"properties": {
"data": {
"$ref": "#/definitions/analytics.ClickHouseOperationalStats"
}
}
}
]
}
},
"400": {
"description": "ClickHouse 未启用",
"schema": {
"$ref": "#/definitions/response.Any"
}
},
"401": {
"description": "未登录",
"schema": {
"$ref": "#/definitions/response.Any"
}
},
"403": {
"description": "无管理员权限",
"schema": {
"$ref": "#/definitions/response.Any"
}
},
"500": {
"description": "内部错误",
"schema": {
"$ref": "#/definitions/response.Any"
}
}
}
}
},
"/api/v1/admin/system-configs": {
"get": {
"security": [
@@ -3368,6 +3434,7 @@ const docTemplate = `{
},
{
"type": "integer",
"format": "int64",
"description": "上传用户 ID",
"name": "user_id",
"in": "query"
@@ -3691,6 +3758,11 @@ const docTemplate = `{
],
"summary": "获取用户列表",
"parameters": [
{
"type": "string",
"name": "email",
"in": "query"
},
{
"minimum": 1,
"type": "integer",
@@ -3909,6 +3981,92 @@ const docTemplate = `{
}
}
},
"put": {
"security": [
{
"SessionCookie": []
}
],
"description": "更新指定用户的昵称、邮箱、管理员权限,并可选重置密码,需要管理员权限",
"consumes": [
"application/json"
],
"produces": [
"application/json"
],
"tags": [
"admin"
],
"summary": "更新用户信息",
"parameters": [
{
"type": "integer",
"description": "用户 ID",
"name": "id",
"in": "path",
"required": true
},
{
"description": "更新参数",
"name": "request",
"in": "body",
"required": true,
"schema": {
"$ref": "#/definitions/user.updateUserRequest"
}
}
],
"responses": {
"200": {
"description": "更新成功",
"schema": {
"allOf": [
{
"$ref": "#/definitions/response.Any"
},
{
"type": "object",
"properties": {
"data": {
"type": "string"
}
}
}
]
}
},
"400": {
"description": "参数错误",
"schema": {
"$ref": "#/definitions/response.Any"
}
},
"401": {
"description": "未登录",
"schema": {
"$ref": "#/definitions/response.Any"
}
},
"403": {
"description": "无管理员权限或尝试修改自身权限",
"schema": {
"$ref": "#/definitions/response.Any"
}
},
"404": {
"description": "用户不存在",
"schema": {
"$ref": "#/definitions/response.Any"
}
},
"500": {
"description": "内部错误",
"schema": {
"$ref": "#/definitions/response.Any"
}
}
}
},
"delete": {
"security": [
{
@@ -11144,6 +11302,7 @@ const docTemplate = `{
"parameters": [
{
"type": "integer",
"format": "int64",
"description": "外部帐号绑定记录 ID",
"name": "id",
"in": "path",
@@ -12948,6 +13107,29 @@ const docTemplate = `{
}
}
},
"analytics.ClickHouseOperationalStats": {
"type": "object",
"properties": {
"active_parts": {
"type": "integer"
},
"async_insert_bytes": {
"type": "integer"
},
"async_insert_queue": {
"type": "integer"
},
"database": {
"type": "string"
},
"pending_mutations": {
"type": "integer"
},
"total_rows": {
"type": "integer"
}
}
},
"apply_log.CleanupInput": {
"type": "object",
"properties": {
@@ -13775,19 +13957,22 @@ const docTemplate = `{
"source_countries": {
"type": "object",
"additionalProperties": {
"type": "integer"
"type": "integer",
"format": "int64"
}
},
"status_codes": {
"type": "object",
"additionalProperties": {
"type": "integer"
"type": "integer",
"format": "int64"
}
},
"top_domains": {
"type": "object",
"additionalProperties": {
"type": "integer"
"type": "integer",
"format": "int64"
}
},
"unique_visitor_count": {
@@ -14217,7 +14402,7 @@ const docTemplate = `{
"type": "string"
},
"id": {
"type": "integer"
"type": "string"
},
"is_active": {
"type": "boolean"
@@ -14252,7 +14437,7 @@ const docTemplate = `{
"type": "string"
},
"id": {
"type": "integer"
"type": "string"
},
"is_active": {
"type": "boolean"
@@ -15092,6 +15277,11 @@ const docTemplate = `{
"UploadStatusPending": "待使用",
"UploadStatusUsed": "已使用"
},
"x-enum-descriptions": [
"待使用",
"已使用",
"已删除"
],
"x-enum-varnames": [
"UploadStatusPending",
"UploadStatusUsed",
@@ -18085,6 +18275,30 @@ const docTemplate = `{
}
}
},
"user.updateUserRequest": {
"type": "object",
"required": [
"email"
],
"properties": {
"email": {
"type": "string",
"maxLength": 255
},
"is_admin": {
"type": "boolean"
},
"nickname": {
"type": "string",
"maxLength": 64
},
"password": {
"type": "string",
"maxLength": 64,
"minLength": 8
}
}
},
"user.updateUserStatusRequest": {
"type": "object",
"properties": {
+36 -17
View File
@@ -30,6 +30,14 @@ Agent 统一通过 OpenResty 二进制控制运行时。本地部署需要节点
为了保证异步任务队列(Asynq 框架)及可观测流量看板功能完整运行,快速开始推荐采用 **PostgreSQL + Redis + ClickHouse** 经典单机版编排。
先拉取 ClickHouse 服务端性能配置到 `./config/clickhouse`(目录内**仅**放 `performance.xml`,不要放 listen 配置):
```bash
mkdir -p ./config/clickhouse
curl -fsSL -o ./config/clickhouse/performance.xml \
https://raw.githubusercontent.com/Rain-kl/OpenFlare/refs/heads/main/config/clickhouse/performance.xml
```
在空目录中创建 `docker-compose.yaml`:
```yaml
@@ -37,26 +45,26 @@ version: '3.8'
services:
openflare:
image: ghcr.io/rain-kl/openflare-server:latest
image: ghcr.io/rain-kl/openflare:latest
container_name: openflare-server
restart: unless-stopped
ports:
- "3000:3000"
volumes:
- ./uploads:/app/uploads
- openflare_uploads:/app/uploads
environment:
TZ: Asia/Shanghai
APP_SESSION_SECRET: 'replace-with-a-long-random-string' # 生产环境请替换为长随机字符串
DB_ENABLED: "true"
DB_HOST: "postgres"
DB_PORT: "5432"
DB_USERNAME: "openflare"
DB_PASSWORD: "replace-with-strong-password"
DB_NAME: "openflare"
DB_USERNAME: "${DB_USERNAME:-openflare}"
DB_PASSWORD: "${DB_PASSWORD:-replace-with-strong-password}"
DB_NAME: "${DB_NAME:-openflare}"
REDIS_ENABLED: "true"
REDIS_ADDRS: "redis:6379"
REDIS_ADDR: "redis:6379"
CLICKHOUSE_ENABLED: "true"
CLICKHOUSE_HOSTS: "clickhouse:9000"
CLICKHOUSE_HOST: "clickhouse:9000"
depends_on:
postgres:
condition: service_healthy
@@ -69,13 +77,13 @@ services:
image: postgres:17-alpine
restart: unless-stopped
environment:
POSTGRES_DB: openflare
POSTGRES_USER: openflare
POSTGRES_PASSWORD: replace-with-strong-password
POSTGRES_DB: ${DB_NAME:-openflare}
POSTGRES_USER: ${DB_USERNAME:-openflare}
POSTGRES_PASSWORD: ${DB_PASSWORD:-replace-with-strong-password}
volumes:
- ./data/postgres_data:/var/lib/postgresql/data
- openflare_postgres_data:/var/lib/postgresql/data
healthcheck:
test: ["CMD-SHELL", "pg_isready -U openflare -d openflare"]
test: ["CMD-SHELL", "pg_isready -U ${DB_USERNAME:-openflare} -d ${DB_NAME:-openflare}"]
interval: 10s
timeout: 5s
retries: 5
@@ -85,7 +93,7 @@ services:
restart: unless-stopped
command: ["valkey-server", "--appendonly", "yes"]
volumes:
- ./data/valkey:/data
- openflare_redis_data:/data
healthcheck:
test: ["CMD", "valkey-cli", "ping"]
interval: 10s
@@ -98,17 +106,28 @@ services:
environment:
CLICKHOUSE_DB: openflare
CLICKHOUSE_USER: default
CLICKHOUSE_PASSWORD: 123456
CLICKHOUSE_PASSWORD: ${CLICKHOUSE_PASSWORD:-replace-with-clickhouse-password}
CLICKHOUSE_DEFAULT_ACCESS_MANAGEMENT: 1
TZ: Asia/Shanghai
ulimits:
nofile:
soft: 262144
hard: 262144
volumes:
- ./data/clickhouse_data:/var/lib/clickhouse
- openflare_clickhouse_data:/var/lib/clickhouse
- ./config/clickhouse:/etc/clickhouse-server/config.d:ro
healthcheck:
test: ["CMD", "clickhouse-client", "--query", "SELECT 1"]
test: ["CMD", "clickhouse-client", "--user", "default", "--password", "${CLICKHOUSE_PASSWORD:-replace-with-clickhouse-password}", "--query", "SELECT 1"]
interval: 10s
timeout: 5s
retries: 5
start_period: 15s
volumes:
openflare_uploads:
openflare_postgres_data:
openflare_redis_data:
openflare_clickhouse_data:
```
启动服务:
@@ -134,7 +153,7 @@ http://localhost:3000
| 用户名 | 密码 |
| --- | --- |
| `root` | `123456` |
| `admin` | `12345678` |
> [!WARNING]
> 为了你的系统安全,首次登录后请立即修改默认密码。
+5 -5
View File
@@ -79,7 +79,7 @@ NEXT_DEV_BACKEND_URL=http://127.0.0.1:3000 pnpm dev
## 默认账号无法登录
默认账号是 `root` / `123456`。首次登录后如果已经修改密码,应使用修改后的密码。
默认账号是 `admin` / `12345678`。首次登录后如果已经修改密码,应使用修改后的密码。
排查步骤:
@@ -90,7 +90,7 @@ NEXT_DEV_BACKEND_URL=http://127.0.0.1:3000 pnpm dev
### 应急重置管理员密码
如果忘记了 `root` 账户的密码,可以通过直接更新数据库中的密码哈希值将其重置为 `123456`(登录后请务必立即修改):
如果忘记了 `admin` 账户的密码,可以通过直接更新数据库中的密码哈希值将其重置为 `12345678`(登录后请务必立即修改):
#### 1. 若使用 SQLite 数据库
停止 Server 运行,使用 sqlite3 客户端打开数据库文件:
@@ -99,16 +99,16 @@ sqlite3 /path/to/openflare.db
```
执行以下 SQL 语句:
```sql
UPDATE users SET password_hash = '$2a$10$wN9aE3zTz83rO7R1uKlhuehJtA3c604pX4Z12B/9.5c0X337t1L4m' WHERE username = 'root';
UPDATE users SET password = '$2a$10$eXpE9i/6S3gPT94/G0mu0.B8ser66ARETFz5NWYSYcrQ4JmtSrMXu' WHERE username = 'admin';
```
输入 `.exit` 退出并重新启动 Server。
#### 2. 若使用 PostgreSQL 数据库
通过您的数据库连接工具(如 psql、pgAdmin 或 DBeaver)连接到 PostgreSQL 实例,选择对应的 `openflare` 数据库,执行以下 SQL 语句:
```sql
UPDATE users SET password_hash = '$2a$10$wN9aE3zTz83rO7R1uKlhuehJtA3c604pX4Z12B/9.5c0X337t1L4m' WHERE username = 'root';
UPDATE users SET password = '$2a$10$eXpE9i/6S3gPT94/G0mu0.B8ser66ARETFz5NWYSYcrQ4JmtSrMXu' WHERE username = 'admin';
```
执行成功后即可使用默认密码 `123456` 重新登录管理后台。
执行成功后即可使用默认密码 `12345678` 重新登录管理后台。
## Agent 无法注册或一直离线
@@ -0,0 +1,22 @@
# ClickHouse P0–P3 修复计划
> 状态: 已完成(已合并主工作区,`make code-check` 通过)
> 策略: 4 个互不干扰 worktree 并行,最后由主代理合并
## 任务拆分
| ID | Worktree 主题 | 范围 | 禁止改动 |
|----|---------------|------|----------|
| WT1 | P0 清理语义 C1 | cleanup maintenance / delete / tasks | chwriter、dashboard、DDL 新 MV |
| WT2 | 写路径 C2+H1+H2+H3 | chwriter、batchwriter、risk_control、model store 分层、status 指标 | goose 迁移、dashboard 读逻辑 |
| WT3 | 读路径 H4+H5 | 最新快照查询、metric/openresty 小时 MV + 读路径 | chwriter、cleanup |
| WT4 | P3 打磨 | 连接池/async_insert、traffic hourly TTL、UV 语义 | model store 分层、cleanup |
## 合并顺序
1. WT1 → 2. WT2 → 3. WT3 → 4. WT4
(迁移文件时间戳已错开,changelog 由主代理统一写)
## 验收
各 worktree: 相关 `go test` + 可运行部分;合并后 `make code-check`。
+47
View File
@@ -0,0 +1,47 @@
# ClickHouse CPU 性能优化计划
> PLAN_ID: `63ba981b`
> 状态: 已完成(含 Phase 2 遗留治理)
> 目标: 完成 P0–P2 优化,降低 ClickHouse CPU 占用
## 背景
ClickHouse CPU 偏高由写入侧(小 part 频繁 flush、心跳同步 DELETE mutation)与查询侧(无 LIMIT 全表扫、高频轮询、WAF 全量拉日志)叠加导致。
## PR Plan
### PR 1: 写入路径 P0 优化
- **Description:** 移除心跳路径同步 `ALTER DELETE`;为 `batchwriter` 增加 `MinBatchSize`;调大可观测 writer 批次与 flush 间隔;为 openresty/frps/frpc 补全去重。
- **Files/components affected:** `internal/apps/openflare/agent/observability.go`, `internal/db/batchwriter/`, `internal/apps/openflare/chwriter/`, `internal/db/batchwriter/*_test.go`
- **Dependencies:** None
### PR 2: ClickHouse 客户端与配置 P1
- **Description:** 启用 `async_insert` 等写入优化 settings;提高 `block_buffer_size` 默认值;更新 `config.example.yaml` 与配置模型注释。
- **Files/components affected:** `internal/db/clickhouse.go`, `internal/config/model.go`, `internal/config/config.go`, `config.example.yaml`
- **Dependencies:** None
### PR 3: Dashboard 与可观测查询 P0
- **Description:** 消除 `limit=0` 无界查询;复用已有限制数据构建趋势;增加服务端短 TTL 缓存;降低前端轮询频率。
- **Files/components affected:** `internal/apps/openflare/dashboard/logics.go`, `internal/apps/openflare/observability/node_logics.go`, `frontend/app/(main)/page.tsx`, `frontend/app/(main)/nodes/components/node-observability.tsx`
- **Dependencies:** None
### PR 4: 访问日志与 WAF 查询 P0/P1
- **Description:** WAF IP 同步改为 ClickHouse 侧聚合;IP 汇总与折叠日志 SQL 分页;消除 count 重复全量扫描;列表 API 强制默认时间窗口。
- **Files/components affected:** `internal/apps/openflare/waf/ip_group_sync.go`, `internal/repository/analytics/node_access_log_stats.go`, `internal/model/openflare_access_log.go`, `internal/apps/openflare/observability/access_log_logics.go`, `internal/repository/analytics/access_log_stats.go`
- **Dependencies:** None
### PR 5: ClickHouse DDL 与数据规范化 P1
- **Description:** 为 7 张分析表添加 TTL;收窄 `of_node_access_logs` ORDER BY;插入时规范化 `remote_addr`(去 trim 查询);将可观测 obs 三表纳入自动清理。
- **Files/components affected:** `internal/db/migrator/goose/clickhouse/`, `internal/repository/analytics/node_access_log_writer.go`, `internal/apps/openflare/tasks/database_cleanup.go`, `internal/model/analytics/`
- **Dependencies:** PR 1
### PR 6: 基础设施与审计减负 P2
- **Description:** Docker ClickHouse 服务端基础调优;审计日志 headers 截断/精简;更新 changelog。
- **Files/components affected:** `docker-compose.yaml`, `docker/clickhouse/` (if needed), `internal/apps/risk_control/middleware.go`, `docs/changelog/index.md`
- **Dependencies:** None
+10 -2
View File
@@ -99,13 +99,21 @@ Server 的所有核心基础配置定义在 `config.yaml` 中,且均支持环
| `redis.pool_size` | `REDIS_POOL_SIZE` | Redis 连接池大小 | `100` |
### 4. ClickHouse 配置 (`clickhouse:`)
> **说明**:下列为 OpenFlare **客户端**连接参数。ClickHouse **服务端**小规格调优:将 `performance.xml` curl 到 `./config/clickhouse/`,compose 挂载 `./config/clickhouse:/etc/clickhouse-server/config.d:ro`(目录内不要放 listen 配置),详见 [启动 Server](../deployment/server.md)。
| 配置文件 YAML 路径 | 对应覆盖环境变量 | 作用说明 | 默认值 |
| --- | --- | --- | --- |
| `clickhouse.enabled` | `CLICKHOUSE_ENABLED` | 是否启用 ClickHouse。**系统节点指标与访问日志在此进行海量写入** | `true` |
| `clickhouse.hosts` | `CLICKHOUSE_HOST` | ClickHouse 集群连接地址数组(环境变量仅设置单地址) | `["127.0.0.1:9000"]` |
| `clickhouse.username` | `CLICKHOUSE_USERNAME` | ClickHouse 账号用户名 | `default` |
| `clickhouse.password` | `CLICKHOUSE_PASSWORD` | ClickHouse 密码 | `123456` |
| `clickhouse.password` | `CLICKHOUSE_PASSWORD` | ClickHouse 密码 | `replace-with-clickhouse-password` |
| `clickhouse.database` | `CLICKHOUSE_NAME` | ClickHouse 存储的数据库名称 | `openflare` |
| `clickhouse.max_idle_conn` | - | 客户端空闲连接数(小规格默认偏低) | `8` |
| `clickhouse.max_open_conn` | - | 客户端最大打开连接数 | `16` |
| `clickhouse.conn_max_lifetime` | - | 连接最大存活时间(秒) | `3600` |
| `clickhouse.dial_timeout` | - | 建连超时(秒) | `5` |
| `clickhouse.block_buffer_size` | - | 原生协议 block 缓冲行数 | `32` |
### 5. 系统日志配置 (`log:`)
| 配置文件 YAML 路径 | 对应覆盖环境变量 | 作用说明 | 默认值 |
@@ -161,7 +169,7 @@ Server 的所有核心基础配置定义在 `config.yaml` 中,且均支持环
### 2. 人机安全校验 (PoW Captcha)
| 配置键 (Key) | 数据类型 | 作用说明 | 默认值 |
| --- | --- | --- | --- |
| `cap_login_enabled` | `bool` | 是否在登录界面强制要求进行本地 PoW 算力防爆破人机验证 | `true` |
| `cap_login_enabled` | `bool` | 是否在登录界面强制要求进行本地 PoW 算力防爆破人机验证 | `false` |
| `cap_auto_solve` | `bool` | 打开页面后是否由浏览器自动开始后台背景计算算力(无需用户手动点击)| `true` |
| `cap_challenge_count` | `int` | 人机验证所需的计算难题数。数量越大,计算要求时间越长(推荐 1~5) | `1` |
| `cap_challenge_difficulty`| `int`| 每次计算所需的 PoW 哈希前缀匹配难度。推荐数值在 3-5 之间 | `4` |
+219 -5
View File
@@ -302,6 +302,7 @@
"parameters": [
{
"type": "integer",
"format": "int64",
"description": "认证源 ID 或名称",
"name": "id",
"in": "path",
@@ -379,6 +380,7 @@
"parameters": [
{
"type": "integer",
"format": "int64",
"description": "认证源 ID 或名称",
"name": "id",
"in": "path",
@@ -446,6 +448,7 @@
"parameters": [
{
"type": "integer",
"format": "int64",
"description": "认证源 ID 或名称",
"name": "id",
"in": "path",
@@ -1356,6 +1359,7 @@
"parameters": [
{
"type": "integer",
"format": "int64",
"description": "通道ID",
"name": "id",
"in": "path",
@@ -1409,6 +1413,7 @@
"parameters": [
{
"type": "integer",
"format": "int64",
"description": "通道ID",
"name": "id",
"in": "path",
@@ -1865,6 +1870,67 @@
}
}
},
"/api/v1/admin/status/clickhouse": {
"get": {
"security": [
{
"SessionCookie": []
}
],
"description": "返回 ClickHouse parts、mutation、async_insert 队列等运维指标,需要管理员权限",
"produces": [
"application/json"
],
"tags": [
"admin"
],
"summary": "获取 ClickHouse 运行指标",
"responses": {
"200": {
"description": "获取成功",
"schema": {
"allOf": [
{
"$ref": "#/definitions/response.Any"
},
{
"type": "object",
"properties": {
"data": {
"$ref": "#/definitions/analytics.ClickHouseOperationalStats"
}
}
}
]
}
},
"400": {
"description": "ClickHouse 未启用",
"schema": {
"$ref": "#/definitions/response.Any"
}
},
"401": {
"description": "未登录",
"schema": {
"$ref": "#/definitions/response.Any"
}
},
"403": {
"description": "无管理员权限",
"schema": {
"$ref": "#/definitions/response.Any"
}
},
"500": {
"description": "内部错误",
"schema": {
"$ref": "#/definitions/response.Any"
}
}
}
}
},
"/api/v1/admin/system-configs": {
"get": {
"security": [
@@ -3361,6 +3427,7 @@
},
{
"type": "integer",
"format": "int64",
"description": "上传用户 ID",
"name": "user_id",
"in": "query"
@@ -3684,6 +3751,11 @@
],
"summary": "获取用户列表",
"parameters": [
{
"type": "string",
"name": "email",
"in": "query"
},
{
"minimum": 1,
"type": "integer",
@@ -3902,6 +3974,92 @@
}
}
},
"put": {
"security": [
{
"SessionCookie": []
}
],
"description": "更新指定用户的昵称、邮箱、管理员权限,并可选重置密码,需要管理员权限",
"consumes": [
"application/json"
],
"produces": [
"application/json"
],
"tags": [
"admin"
],
"summary": "更新用户信息",
"parameters": [
{
"type": "integer",
"description": "用户 ID",
"name": "id",
"in": "path",
"required": true
},
{
"description": "更新参数",
"name": "request",
"in": "body",
"required": true,
"schema": {
"$ref": "#/definitions/user.updateUserRequest"
}
}
],
"responses": {
"200": {
"description": "更新成功",
"schema": {
"allOf": [
{
"$ref": "#/definitions/response.Any"
},
{
"type": "object",
"properties": {
"data": {
"type": "string"
}
}
}
]
}
},
"400": {
"description": "参数错误",
"schema": {
"$ref": "#/definitions/response.Any"
}
},
"401": {
"description": "未登录",
"schema": {
"$ref": "#/definitions/response.Any"
}
},
"403": {
"description": "无管理员权限或尝试修改自身权限",
"schema": {
"$ref": "#/definitions/response.Any"
}
},
"404": {
"description": "用户不存在",
"schema": {
"$ref": "#/definitions/response.Any"
}
},
"500": {
"description": "内部错误",
"schema": {
"$ref": "#/definitions/response.Any"
}
}
}
},
"delete": {
"security": [
{
@@ -11137,6 +11295,7 @@
"parameters": [
{
"type": "integer",
"format": "int64",
"description": "外部帐号绑定记录 ID",
"name": "id",
"in": "path",
@@ -12941,6 +13100,29 @@
}
}
},
"analytics.ClickHouseOperationalStats": {
"type": "object",
"properties": {
"active_parts": {
"type": "integer"
},
"async_insert_bytes": {
"type": "integer"
},
"async_insert_queue": {
"type": "integer"
},
"database": {
"type": "string"
},
"pending_mutations": {
"type": "integer"
},
"total_rows": {
"type": "integer"
}
}
},
"apply_log.CleanupInput": {
"type": "object",
"properties": {
@@ -13768,19 +13950,22 @@
"source_countries": {
"type": "object",
"additionalProperties": {
"type": "integer"
"type": "integer",
"format": "int64"
}
},
"status_codes": {
"type": "object",
"additionalProperties": {
"type": "integer"
"type": "integer",
"format": "int64"
}
},
"top_domains": {
"type": "object",
"additionalProperties": {
"type": "integer"
"type": "integer",
"format": "int64"
}
},
"unique_visitor_count": {
@@ -14210,7 +14395,7 @@
"type": "string"
},
"id": {
"type": "integer"
"type": "string"
},
"is_active": {
"type": "boolean"
@@ -14245,7 +14430,7 @@
"type": "string"
},
"id": {
"type": "integer"
"type": "string"
},
"is_active": {
"type": "boolean"
@@ -15085,6 +15270,11 @@
"UploadStatusPending": "待使用",
"UploadStatusUsed": "已使用"
},
"x-enum-descriptions": [
"待使用",
"已使用",
"已删除"
],
"x-enum-varnames": [
"UploadStatusPending",
"UploadStatusUsed",
@@ -18078,6 +18268,30 @@
}
}
},
"user.updateUserRequest": {
"type": "object",
"required": [
"email"
],
"properties": {
"email": {
"type": "string",
"maxLength": 255
},
"is_admin": {
"type": "boolean"
},
"nickname": {
"type": "string",
"maxLength": 64
},
"password": {
"type": "string",
"maxLength": 64,
"minLength": 8
}
}
},
"user.updateUserStatusRequest": {
"type": "object",
"properties": {
+140 -2
View File
@@ -169,6 +169,21 @@ definitions:
$ref: '#/definitions/github_com_Rain-kl_Wavelet_pkg_protocol.WAFIPGroup'
type: array
type: object
analytics.ClickHouseOperationalStats:
properties:
active_parts:
type: integer
async_insert_bytes:
type: integer
async_insert_queue:
type: integer
database:
type: string
pending_mutations:
type: integer
total_rows:
type: integer
type: object
apply_log.CleanupInput:
properties:
delete_all:
@@ -713,14 +728,17 @@ definitions:
type: integer
source_countries:
additionalProperties:
format: int64
type: integer
type: object
status_codes:
additionalProperties:
format: int64
type: integer
type: object
top_domains:
additionalProperties:
format: int64
type: integer
type: object
unique_visitor_count:
@@ -1004,7 +1022,7 @@ definitions:
created_by:
type: string
id:
type: integer
type: string
is_active:
type: boolean
main_config:
@@ -1027,7 +1045,7 @@ definitions:
created_by:
type: string
id:
type: integer
type: string
is_active:
type: boolean
version:
@@ -1593,6 +1611,10 @@ definitions:
UploadStatusDeleted: 已删除
UploadStatusPending: 待使用
UploadStatusUsed: 已使用
x-enum-descriptions:
- 待使用
- 已使用
- 已删除
x-enum-varnames:
- UploadStatusPending
- UploadStatusUsed
@@ -3577,6 +3599,23 @@ definitions:
website:
type: string
type: object
user.updateUserRequest:
properties:
email:
maxLength: 255
type: string
is_admin:
type: boolean
nickname:
maxLength: 64
type: string
password:
maxLength: 64
minLength: 8
type: string
required:
- email
type: object
user.updateUserStatusRequest:
properties:
is_active:
@@ -4085,6 +4124,7 @@ paths:
description: 删除指定认证源及其关联的所有外部帐号绑定记录,警告:删除后相关用户将无法通过该源登录,需要管理员权限
parameters:
- description: 认证源 ID 或名称
format: int64
in: path
name: id
required: true
@@ -4124,6 +4164,7 @@ paths:
description: 更新指定 ID 的认证源配置。若 client_secret 字段为空,则保留原有密钥不变,需要管理员权限
parameters:
- description: 认证源 ID 或名称
format: int64
in: path
name: id
required: true
@@ -4174,6 +4215,7 @@ paths:
description: 启用或禁用指定认证源。尝试启用时将验证 Client ID 和 Client Secret 是否已配置,需要管理员权限
parameters:
- description: 认证源 ID 或名称
format: int64
in: path
name: id
required: true
@@ -4670,6 +4712,7 @@ paths:
description: 根据ID删除消息通道,需要管理员权限
parameters:
- description: 通道ID
format: int64
in: path
name: id
required: true
@@ -4692,6 +4735,7 @@ paths:
description: 修改消息通道配置,需要管理员权限
parameters:
- description: 通道ID
format: int64
in: path
name: id
required: true
@@ -5016,6 +5060,42 @@ paths:
summary: 获取系统状态信息
tags:
- admin
/api/v1/admin/status/clickhouse:
get:
description: 返回 ClickHouse parts、mutation、async_insert 队列等运维指标,需要管理员权限
produces:
- application/json
responses:
"200":
description: 获取成功
schema:
allOf:
- $ref: '#/definitions/response.Any'
- properties:
data:
$ref: '#/definitions/analytics.ClickHouseOperationalStats'
type: object
"400":
description: ClickHouse 未启用
schema:
$ref: '#/definitions/response.Any'
"401":
description: 未登录
schema:
$ref: '#/definitions/response.Any'
"403":
description: 无管理员权限
schema:
$ref: '#/definitions/response.Any'
"500":
description: 内部错误
schema:
$ref: '#/definitions/response.Any'
security:
- SessionCookie: []
summary: 获取 ClickHouse 运行指标
tags:
- admin
/api/v1/admin/system-configs:
get:
description: 返回所有系统配置列表,支持按配置类型(system/business)过滤,需要管理员权限
@@ -5912,6 +5992,7 @@ paths:
name: extension
type: string
- description: 上传用户 ID
format: int64
in: query
name: user_id
type: integer
@@ -6108,6 +6189,9 @@ paths:
get:
description: 分页返回用户列表,支持按用户 ID 和用户名筛选,需要管理员权限
parameters:
- in: query
name: email
type: string
- in: query
minimum: 1
name: page
@@ -6291,6 +6375,59 @@ paths:
summary: 获取用户详情
tags:
- admin
put:
consumes:
- application/json
description: 更新指定用户的昵称、邮箱、管理员权限,并可选重置密码,需要管理员权限
parameters:
- description: 用户 ID
in: path
name: id
required: true
type: integer
- description: 更新参数
in: body
name: request
required: true
schema:
$ref: '#/definitions/user.updateUserRequest'
produces:
- application/json
responses:
"200":
description: 更新成功
schema:
allOf:
- $ref: '#/definitions/response.Any'
- properties:
data:
type: string
type: object
"400":
description: 参数错误
schema:
$ref: '#/definitions/response.Any'
"401":
description: 未登录
schema:
$ref: '#/definitions/response.Any'
"403":
description: 无管理员权限或尝试修改自身权限
schema:
$ref: '#/definitions/response.Any'
"404":
description: 用户不存在
schema:
$ref: '#/definitions/response.Any'
"500":
description: 内部错误
schema:
$ref: '#/definitions/response.Any'
security:
- SessionCookie: []
summary: 更新用户信息
tags:
- admin
/api/v1/admin/users/{id}/status:
put:
consumes:
@@ -10642,6 +10779,7 @@ paths:
description: 解除当前登录用户与指定外部帐号的绑定关系,需要登录
parameters:
- description: 外部帐号绑定记录 ID
format: int64
in: path
name: id
required: true
@@ -327,7 +327,7 @@ export function EventsTab() {
<Switch
checked={event.enabled}
onCheckedChange={() => toggleEventMutation.mutate(event.id)}
className="scale-75 data-[state=checked]:bg-green-600 h-4 w-7"
className="scale-75"
/>
</TableCell>
<TableCell className="sticky right-0 text-center bg-background z-10 py-1" onClick={(e) => e.stopPropagation()}>
@@ -324,7 +324,7 @@ export function SettingsTab() {
}
})
}}
className="scale-75 data-[state=checked]:bg-green-600 h-4 w-7"
className="scale-75"
/>
</TableCell>
<TableCell className="sticky right-0 text-center bg-background z-10 py-1" onClick={(e) => e.stopPropagation()}>
@@ -16,7 +16,6 @@ import {
AlertDialogHeader,
AlertDialogTitle,
} from "@/components/ui/alert-dialog"
import {Badge} from "@/components/ui/badge"
import {Button} from "@/components/ui/button"
import {Card, CardContent, CardDescription, CardHeader, CardTitle} from "@/components/ui/card"
import {Input} from "@/components/ui/input"
@@ -28,9 +27,6 @@ import {ErrorInline} from "@/components/layout/error"
import {LoadingStateWithBorder} from "@/components/layout/loading"
import type {DatabaseCleanupTarget} from "@/lib/services/openflare"
import {NodeService, OptionService, StatusService, UptimeKumaService,} from "@/lib/services/openflare"
import {AdminStatusService} from "@/lib/services/admin"
import {VersionUpgradeDialog} from "@/app/(main)/components/version-upgrade-dialog"
import {adminUpdateStatusQueryKey, openflarePublicStatusQueryKey,} from "@/lib/hooks/use-openflare-server-upgrade"
import {
agentOptionEntries,
@@ -47,6 +43,7 @@ import {
import {UptimeKumaSiteSelectModal} from "./uptimekuma-site-modal"
const optionsQueryKey = ["openflare", "options"] as const
const openflarePublicStatusQueryKey = ["openflare", "public-status"] as const
const cleanupTargets: Array<{
target: DatabaseCleanupTarget
@@ -85,7 +82,6 @@ export function OpenFlareOpsSettings() {
label: string
} | null>(null)
const [cleanupRetentionDays, setCleanupRetentionDays] = useState("")
const [versionDialogOpen, setVersionDialogOpen] = useState(false)
const optionsQuery = useQuery({
queryKey: optionsQueryKey,
@@ -102,10 +98,6 @@ export function OpenFlareOpsSettings() {
queryFn: () => NodeService.getBootstrapToken(),
})
const releaseQuery = useQuery({
queryKey: adminUpdateStatusQueryKey,
queryFn: () => AdminStatusService.getUpdateStatus(),
})
useEffect(() => {
if (!optionsQuery.data) return
@@ -605,50 +597,6 @@ export function OpenFlareOpsSettings() {
</Card>
</div>
<Card className="border-dashed shadow-none">
<CardHeader className="flex flex-row items-center justify-between gap-4">
<div>
<CardTitle className="text-base">版本信息</CardTitle>
<CardDescription>
检查上游 GitHub Release 并升级当前服务。
</CardDescription>
</div>
<Button type="button" size="sm" onClick={() => setVersionDialogOpen(true)}>
管理升级
</Button>
</CardHeader>
<CardContent className="grid gap-3 sm:grid-cols-2 lg:grid-cols-4">
<InfoCell label="当前版本" value={statusQuery.data?.version ?? releaseQuery.data?.current_version ?? "—"} />
<InfoCell
label="最新 Release"
value={releaseQuery.data?.latest_version ?? "—"}
/>
<div className="rounded-lg border border-dashed px-3 py-2">
<p className="text-[10px] uppercase tracking-wider text-muted-foreground">更新状态</p>
<div className="mt-2">
{releaseQuery.data?.update_available ? (
<Badge variant="secondary">有新版本</Badge>
) : (
<Badge variant="outline">已是最新</Badge>
)}
</div>
</div>
<InfoCell
label="启动时间"
value={
statusQuery.data?.start_time
? new Date(statusQuery.data.start_time * 1000).toLocaleString()
: "—"
}
/>
</CardContent>
</Card>
<VersionUpgradeDialog
open={versionDialogOpen}
onOpenChange={setVersionDialogOpen}
canUpgrade
/>
<UptimeKumaSiteSelectModal
open={uptimeKumaModalOpen}
@@ -7,7 +7,7 @@ import dynamic from "next/dynamic"
import {useEffect, useMemo} from "react"
import {useQuery} from "@tanstack/react-query"
import {Loader2, Settings} from "lucide-react"
import {useRouter} from "next/navigation"
import {useRouter, useSearchParams} from "next/navigation"
import {motion} from "motion/react"
import {Tabs, TabsContent, TabsList, TabsTrigger} from "@/components/ui/tabs"
@@ -64,6 +64,17 @@ function systemConfigMap(configs: SystemConfig[]) {
export function AdminSettingsPageClient() {
const { user, loading } = useAuth()
const router = useRouter()
const searchParams = useSearchParams()
const activeTab = useMemo(() => {
const rawTab = searchParams.get("tab")
const validTabs = ["openflare-ops", "security", "operation", "system", "other", "status", "info"]
return rawTab && validTabs.includes(rawTab) ? rawTab : "openflare-ops"
}, [searchParams])
const handleTabChange = (value: string) => {
router.push(`/admin/settings?tab=${value}`)
}
const systemConfigsQuery = useQuery({
queryKey: ["admin", "system-configs"],
@@ -85,7 +96,7 @@ export function AdminSettingsPageClient() {
if (loading || !user || !user.is_admin) {
return (
<div className="flex items-center justify-center min-h-[400px]">
<Loader2 className="size-6 animate-spin text-indigo-500" />
<Loader2 className="size-6 animate-spin text-primary" />
</div>
)
}
@@ -103,7 +114,7 @@ export function AdminSettingsPageClient() {
<h1 className="text-2xl font-semibold tracking-tight">系统设置</h1>
</div>
</div>
<Tabs defaultValue="security" className="w-full">
<Tabs value={activeTab} onValueChange={handleTabChange} className="w-full">
<TabsList variant="line" className="w-fit inline-flex gap-8 mb-6">
<TabsTrigger value="openflare-ops" className="px-0 pb-2 text-xs font-semibold">
OpenFlare
+14 -2
View File
@@ -1,8 +1,20 @@
// Copyright 2026 Arctel.net
// SPDX-License-Identifier: Apache-2.0
import {Suspense} from "react"
import {Loader2} from "lucide-react"
import {AdminSettingsPageClient} from "./page-client"
export default function AdminSettingsPage() {
return <AdminSettingsPageClient />
}
return (
<Suspense
fallback={
<div className="flex items-center justify-center min-h-[400px]">
<Loader2 className="size-6 animate-spin text-primary" />
</div>
}
>
<AdminSettingsPageClient />
</Suspense>
)
}
@@ -0,0 +1,280 @@
"use client"
import {useEffect, useState} from "react"
import {Button} from "@/components/ui/button"
import {Dialog, DialogContent, DialogDescription, DialogHeader, DialogTitle} from "@/components/ui/dialog"
import {Input} from "@/components/ui/input"
import {Label} from "@/components/ui/label"
import {Switch} from "@/components/ui/switch"
import {useAdminUsers} from "@/contexts/admin-users-context"
import {useAuth} from "@/components/providers/auth-provider"
import type {AdminUser} from "@/lib/services/admin"
import {
AlertDialog,
AlertDialogAction,
AlertDialogCancel,
AlertDialogContent,
AlertDialogDescription,
AlertDialogFooter,
AlertDialogHeader,
AlertDialogTitle
} from "@/components/ui/alert-dialog"
import {Loader2} from "lucide-react"
interface EditUserForm {
nickname: string
email: string
is_admin: boolean
password?: string
}
export function EditUserModal({
isOpen,
onClose,
user,
}: {
isOpen: boolean
onClose: () => void
user: AdminUser | null
}) {
const { updateUser, deleteUser, getUserDetail } = useAdminUsers()
const { user: currentUser } = useAuth()
const isSelf = currentUser && user && currentUser.id.toString() === user.id.toString()
const [form, setForm] = useState<EditUserForm>({
nickname: "",
email: "",
is_admin: false,
password: "",
})
const [saving, setSaving] = useState(false)
const [errors, setErrors] = useState<Record<string, string>>({})
const [showDeleteConfirm, setShowDeleteConfirm] = useState(false)
useEffect(() => {
let active = true
if (isOpen && user) {
setForm({
nickname: user.nickname || "",
email: user.email || "",
is_admin: user.is_admin || false,
password: "",
})
setErrors({})
getUserDetail(user.id)
.then((detail) => {
if (active && detail) {
setForm({
nickname: detail.nickname || "",
email: detail.email || "",
is_admin: detail.is_admin || false,
password: "",
})
}
})
.catch(() => {
// ignore fetching error
})
} else {
setSaving(false)
}
return () => {
active = false
}
}, [isOpen, user, getUserDetail])
const validate = () => {
const newErrors: Record<string, string> = {}
if (!form.email.trim()) {
newErrors.email = "邮箱不能为空"
} else if (!/^[^\s@]+@[^\s@]+\.[^\s@]+$/.test(form.email.trim())) {
newErrors.email = "邮箱格式不正确"
}
if (form.password && form.password.length < 8) {
newErrors.password = "密码长度不能少于 8 位"
}
setErrors(newErrors)
return Object.keys(newErrors).length === 0
}
const handleSave = async (e: React.FormEvent) => {
e.preventDefault()
if (!user || !validate()) return
setSaving(true)
try {
await updateUser(user.id, {
nickname: form.nickname.trim() || undefined,
email: form.email.trim(),
is_admin: form.is_admin,
password: form.password?.trim() || undefined,
})
onClose()
} catch {
// Errors are handled by context toast notifications
} finally {
setSaving(false)
}
}
const handleDelete = async () => {
if (!user) return
setSaving(true)
try {
await deleteUser(user)
onClose()
} catch {
// Errors are handled by context toast notifications
} finally {
setSaving(false)
setShowDeleteConfirm(false)
}
}
if (!user) return null
return (
<>
<Dialog open={isOpen} onOpenChange={(open) => !open && onClose()}>
<DialogContent className="max-w-md">
<DialogHeader>
<DialogTitle>编辑用户</DialogTitle>
<DialogDescription>
修改用户档案、授予/撤销管理权限或重置其密码。
</DialogDescription>
</DialogHeader>
<form onSubmit={handleSave} className="space-y-4 pt-2">
<div className="space-y-1.5">
<Label htmlFor="username" className="text-xs text-muted-foreground">用户名 (不可修改)</Label>
<Input
id="username"
value={user.username}
disabled
className="bg-muted/50 cursor-not-allowed select-none font-mono"
/>
</div>
<div className="space-y-1.5">
<Label htmlFor="nickname">昵称 (选填)</Label>
<Input
id="nickname"
value={form.nickname}
onChange={(e) => setForm((prev) => ({ ...prev, nickname: e.target.value }))}
placeholder="请输入昵称"
/>
</div>
<div className="space-y-1.5">
<Label htmlFor="email">邮箱</Label>
<Input
id="email"
type="email"
value={form.email}
onChange={(e) => setForm((prev) => ({ ...prev, email: e.target.value }))}
placeholder="请输入邮箱地址"
/>
{errors.email && (
<p className="text-xs text-destructive">{errors.email}</p>
)}
</div>
<div className="space-y-1.5">
<Label htmlFor="password">重置密码 (选填)</Label>
<Input
id="password"
type="password"
value={form.password}
onChange={(e) => setForm((prev) => ({ ...prev, password: e.target.value }))}
placeholder="留空表示保持当前密码,输入则重置 (至少 8 位)"
/>
{errors.password && (
<p className="text-xs text-destructive">{errors.password}</p>
)}
</div>
<div className="flex items-center justify-between rounded-lg border border-dashed p-3 bg-muted/10">
<div>
<div className="font-medium text-sm">管理员权限</div>
<div className="text-xs text-muted-foreground">
{isSelf ? "不能撤销当前登录用户的管理员权限。" : "开启后此账号将拥有后台管理权限。"}
</div>
</div>
<Switch
checked={form.is_admin}
disabled={!!isSelf || saving}
onCheckedChange={async (checked) => {
setForm((prev) => ({ ...prev, is_admin: checked }))
setSaving(true)
try {
await updateUser(user.id, {
nickname: form.nickname.trim() || undefined,
email: form.email.trim(),
is_admin: checked,
})
} catch {
setForm((prev) => ({ ...prev, is_admin: !checked }))
} finally {
setSaving(false)
}
}}
/>
</div>
<div className="flex justify-between items-center pt-2 border-t mt-2">
<div>
{!user.is_admin && (
<Button
type="button"
variant="destructive"
onClick={() => setShowDeleteConfirm(true)}
disabled={saving}
>
删除用户
</Button>
)}
</div>
<div className="flex gap-2">
<Button variant="outline" type="button" onClick={onClose} disabled={saving}>
取消
</Button>
<Button type="submit" disabled={saving} variant="secondary">
{saving ? "保存中..." : "保存"}
</Button>
</div>
</div>
</form>
</DialogContent>
</Dialog>
<AlertDialog open={showDeleteConfirm} onOpenChange={setShowDeleteConfirm}>
<AlertDialogContent>
<AlertDialogHeader>
<AlertDialogTitle>确认删除用户</AlertDialogTitle>
<AlertDialogDescription>
确定要删除用户 {user.nickname || user.username} 吗?该操作会移除用户账号,删除后无法撤销。
</AlertDialogDescription>
</AlertDialogHeader>
<AlertDialogFooter>
<AlertDialogCancel disabled={saving}>取消</AlertDialogCancel>
<AlertDialogAction
onClick={(e) => {
e.preventDefault()
handleDelete()
}}
disabled={saving}
>
{saving && <Loader2 className="size-3 animate-spin mr-1" />}
确认删除
</AlertDialogAction>
</AlertDialogFooter>
</AlertDialogContent>
</AlertDialog>
</>
)
}
@@ -1,14 +1,14 @@
"use client"
import * as React from "react"
import {Globe, Loader2, Mail, MapPin, ShieldCheck, Smartphone, Trash2, UserCheck,} from "lucide-react"
import {Globe, Loader2, Mail, MapPin, ShieldCheck, Smartphone, UserCheck,} from "lucide-react"
import {Button} from "@/components/ui/button"
import {Badge} from "@/components/ui/badge"
import {Avatar, AvatarFallback, AvatarImage} from "@/components/ui/avatar"
import {Sheet, SheetContent, SheetTitle} from "@/components/ui/sheet"
import type {AdminUser} from "@/lib/services/admin"
import {cn, formatDateTime} from "@/lib/utils"
import {formatDateTime} from "@/lib/utils"
interface UserDetailSheetProps {
selectedUser: AdminUser | null
@@ -16,7 +16,6 @@ interface UserDetailSheetProps {
onOpenChange: (open: boolean) => void
detailLoading: boolean
onStatusToggle: (user: AdminUser) => Promise<void>
onDeleteTarget: (user: AdminUser) => void
}
export function UserDetailSheet({
@@ -25,7 +24,6 @@ export function UserDetailSheet({
onOpenChange,
detailLoading,
onStatusToggle,
onDeleteTarget,
}: UserDetailSheetProps) {
const displayValue = (value?: string) => value && value.trim() ? value : "-"
@@ -56,7 +54,7 @@ export function UserDetailSheet({
UID: {selectedUser.id}
</Badge>
{selectedUser.is_admin && (
<Badge className="h-4.5 px-1.5 text-[9px] uppercase font-medium bg-primary text-primary-foreground">
<Badge className="h-4.5 px-1.5 text-[9px] uppercase font-medium">
Admin
</Badge>
)}
@@ -156,15 +154,10 @@ export function UserDetailSheet({
</div>
{!selectedUser.is_admin && (
<div className="p-4 border-t bg-background/80 backdrop-blur-md shrink-0 flex flex-col gap-2">
<div className="p-4 border-t bg-background/80 backdrop-blur-md shrink-0">
<Button
variant={selectedUser.is_active ? "destructive" : "default"}
className={cn(
"w-full h-9 text-xs font-medium transition-all active:scale-[0.98]",
selectedUser.is_active
? "bg-red-500 hover:bg-red-600 text-white"
: "bg-primary text-primary-foreground hover:bg-primary/90"
)}
className="w-full h-9 text-xs font-medium transition-all active:scale-[0.98]"
onClick={() => onStatusToggle(selectedUser)}
>
{selectedUser.is_active ? (
@@ -179,14 +172,6 @@ export function UserDetailSheet({
</>
)}
</Button>
<Button
variant="outline"
className="w-full h-9 text-xs font-medium"
onClick={() => onDeleteTarget(selectedUser)}
>
<Trash2 className="size-3 mr-1" />
删除用户
</Button>
</div>
)}
</div>
@@ -18,17 +18,19 @@ export function UserFilterBar() {
pageSize,
searchUserId,
searchUsername,
searchEmail,
statusFilter,
setPage,
setPageSize,
setSearchUserId,
setSearchUsername,
setSearchEmail,
setStatusFilter,
fetchUsers
} = useAdminUsers()
const totalPages = Math.ceil(total / pageSize)
const hasSearchFilter = Boolean(searchUserId || searchUsername)
const hasSearchFilter = Boolean(searchUserId || searchUsername || searchEmail)
return (
<div className="flex flex-col lg:flex-row lg:items-center lg:justify-between gap-3">
@@ -49,8 +51,8 @@ export function UserFilterBar() {
<>
<Separator orientation="vertical" className="mx-1" />
<Badge
variant="secondary"
className="text-[10px] h-3 px-1 rounded-full bg-primary text-primary-foreground"
variant="default"
className="text-[10px] h-3 px-1 rounded-full"
>
!
</Badge>
@@ -72,6 +74,12 @@ export function UserFilterBar() {
value={searchUsername}
onChange={(e) => setSearchUsername(e.target.value)}
/>
<input
className="w-full h-7 px-2 text-xs border border-dashed rounded-md outline-none focus:border-primary bg-background"
placeholder="输入邮箱..."
value={searchEmail}
onChange={(e) => setSearchEmail(e.target.value)}
/>
{hasSearchFilter && (
<Button
variant="ghost"
@@ -80,6 +88,7 @@ export function UserFilterBar() {
onClick={() => {
setSearchUserId("")
setSearchUsername("")
setSearchEmail("")
}}
>
清除
@@ -105,8 +114,8 @@ export function UserFilterBar() {
<>
<Separator orientation="vertical" className="mx-1" />
<Badge
variant="secondary"
className="text-[10px] h-3 px-1 rounded-full bg-primary text-primary-foreground"
variant="default"
className="text-[10px] h-3 px-1 rounded-full"
>
1
</Badge>
@@ -160,6 +169,7 @@ export function UserFilterBar() {
onClick={() => {
setSearchUserId("")
setSearchUsername("")
setSearchEmail("")
setStatusFilter('all')
}}
className="h-5 px-2 lg:px-3 text-[11px] font-medium text-muted-foreground hover:text-foreground"
+31 -67
View File
@@ -6,18 +6,8 @@ import {Switch} from "@/components/ui/switch"
import {Table, TableBody, TableCell, TableHead, TableHeader, TableRow} from "@/components/ui/table"
import {Badge} from "@/components/ui/badge"
import {Avatar, AvatarFallback, AvatarImage} from "@/components/ui/avatar"
import {Eye, Layers, Loader2, Plus, Trash2, UserRound, UserX,} from "lucide-react"
import {Eye, Layers, Pencil, Plus, UserRound, UserX,} from "lucide-react"
import {Tooltip, TooltipContent, TooltipProvider, TooltipTrigger} from "@/components/ui/tooltip"
import {
AlertDialog,
AlertDialogAction,
AlertDialogCancel,
AlertDialogContent,
AlertDialogDescription,
AlertDialogFooter,
AlertDialogHeader,
AlertDialogTitle
} from "@/components/ui/alert-dialog"
import type {AdminUser} from "@/lib/services/admin"
import {formatDateTime} from "@/lib/utils"
@@ -28,6 +18,7 @@ import {useAdminUsers} from "@/contexts/admin-users-context"
import {CreateUserModal} from "./components/create-user-modal"
import {UserFilterBar} from "./components/user-filter-bar"
import {UserDetailSheet} from "./components/user-detail-sheet"
import {EditUserModal} from "./components/edit-user-modal"
export default function UsersPage() {
const {
@@ -36,16 +27,14 @@ export default function UsersPage() {
error,
fetchUsers,
getUserDetail,
updateUserStatus,
deleteUser
updateUserStatus
} = useAdminUsers()
const [selectedUser, setSelectedUser] = useState<AdminUser | null>(null)
const [detailOpen, setDetailOpen] = useState(false)
const [detailLoading, setDetailLoading] = useState(false)
const [deleteTarget, setDeleteTarget] = useState<AdminUser | null>(null)
const [deleteLoading, setDeleteLoading] = useState(false)
const [createModalOpen, setCreateModalOpen] = useState(false)
const [editTarget, setEditTarget] = useState<AdminUser | null>(null)
useEffect(() => {
fetchUsers()
@@ -74,21 +63,7 @@ export default function UsersPage() {
}
}
const handleDeleteUser = async () => {
if (!deleteTarget) return
setDeleteLoading(true)
try {
await deleteUser(deleteTarget)
if (selectedUser?.id === deleteTarget.id) {
setDetailOpen(false)
setSelectedUser(null)
}
setDeleteTarget(null)
} finally {
setDeleteLoading(false)
}
}
return (
<div className="py-6 space-y-4">
@@ -125,7 +100,8 @@ export default function UsersPage() {
<TableRow className="border-b border-dashed hover:bg-transparent">
<TableHead className="w-[90px] whitespace-nowrap py-2 h-8">ID</TableHead>
<TableHead className="w-[120px] whitespace-nowrap py-2 h-8">用户</TableHead>
<TableHead className="whitespace-nowrap min-w-[140px] py-2 h-8 pl-4">上次登陆</TableHead>
<TableHead className="whitespace-nowrap min-w-[160px] py-2 h-8 pl-4">邮箱</TableHead>
<TableHead className="whitespace-nowrap min-w-[140px] py-2 h-8">上次登陆</TableHead>
<TableHead className="whitespace-nowrap min-w-[140px] py-2 h-8">注册时间</TableHead>
<TableHead className="whitespace-nowrap min-w-[140px] py-2 h-8">上次更新</TableHead>
<TableHead className="sticky right-0 text-center bg-background z-10 w-[110px] py-2 h-8">操作</TableHead>
@@ -163,6 +139,10 @@ export default function UsersPage() {
</div>
</TableCell>
<TableCell className="text-[10px] text-muted-foreground font-mono whitespace-nowrap py-1 pl-4">
{user.email || "-"}
</TableCell>
<TableCell className="text-[10px] text-muted-foreground font-mono whitespace-nowrap py-1 pl-4">
{formatDateTime(user.last_login_at)}
</TableCell>
@@ -181,7 +161,7 @@ export default function UsersPage() {
checked={user.is_active}
onCheckedChange={() => handleStatusToggle(user)}
disabled={user.is_admin}
className="scale-75 data-[state=checked]:bg-green-600 h-4 w-7"
className="scale-75"
/>
</div>
</TooltipTrigger>
@@ -201,23 +181,18 @@ export default function UsersPage() {
</TooltipContent>
</Tooltip>
{!user.is_admin && (
<Tooltip>
<TooltipTrigger asChild>
<Button
variant="ghost"
size="icon"
className="h-6 w-6 text-muted-foreground hover:text-destructive"
onClick={() => setDeleteTarget(user)}
>
<Trash2 className="size-3" />
</Button>
</TooltipTrigger>
<TooltipContent side="top" className="text-xs">
删除用户
</TooltipContent>
</Tooltip>
)}
<Tooltip>
<TooltipTrigger asChild>
<Button variant="ghost" size="icon" className="h-6 w-6 text-muted-foreground hover:text-foreground" onClick={() => setEditTarget(user)}>
<Pencil className="size-3" />
</Button>
</TooltipTrigger>
<TooltipContent side="top" className="text-xs">
编辑用户
</TooltipContent>
</Tooltip>
</div>
</TableCell>
</TableRow>
@@ -235,27 +210,16 @@ export default function UsersPage() {
onOpenChange={setDetailOpen}
detailLoading={detailLoading}
onStatusToggle={handleStatusToggle}
onDeleteTarget={setDeleteTarget}
/>
{/* 删除确认警告弹窗 */}
<AlertDialog open={!!deleteTarget} onOpenChange={(open) => !open && !deleteLoading && setDeleteTarget(null)}>
<AlertDialogContent>
<AlertDialogHeader>
<AlertDialogTitle>确认删除用户</AlertDialogTitle>
<AlertDialogDescription>
确定要删除用户 {deleteTarget?.nickname || deleteTarget?.username} 吗?该操作会移除用户账号,删除后无法撤销。
</AlertDialogDescription>
</AlertDialogHeader>
<AlertDialogFooter>
<AlertDialogCancel disabled={deleteLoading}>取消</AlertDialogCancel>
<AlertDialogAction onClick={handleDeleteUser} disabled={deleteLoading}>
{deleteLoading && <Loader2 className="size-3 animate-spin" />}
确认删除
</AlertDialogAction>
</AlertDialogFooter>
</AlertDialogContent>
</AlertDialog>
{/* 编辑用户弹窗 */}
<EditUserModal
user={editTarget}
isOpen={!!editTarget}
onClose={() => setEditTarget(null)}
/>
{/* 新建用户模态弹窗 */}
<CreateUserModal isOpen={createModalOpen} onClose={() => setCreateModalOpen(false)} />
@@ -34,7 +34,7 @@ export function DashboardStatCards({
{formatCompactNumber(traffic.request_count)}
</div>
<p className="text-[10px] text-muted-foreground">
独立访客 {formatCompactNumber(traffic.unique_visitors)} · 错误{' '}
窗口UV(估) {formatCompactNumber(traffic.unique_visitors)} · 错误{' '}
{formatCompactNumber(traffic.error_count)} · 估算 QPS{' '}
{traffic.estimated_qps.toFixed(2)}
</p>
@@ -1,223 +0,0 @@
'use client';
import {useEffect} from 'react';
import {ExternalLink, Loader2} from 'lucide-react';
import ReactMarkdown from 'react-markdown';
import remarkGfm from 'remark-gfm';
import {Badge} from '@/components/ui/badge';
import {Button} from '@/components/ui/button';
import {Card, CardContent, CardDescription, CardHeader, CardTitle} from '@/components/ui/card';
import {
AlertDialog,
AlertDialogAction,
AlertDialogCancel,
AlertDialogContent,
AlertDialogDescription,
AlertDialogFooter,
AlertDialogHeader,
AlertDialogTitle,
AlertDialogTrigger,
} from '@/components/ui/alert-dialog';
import {Dialog, DialogContent, DialogDescription, DialogHeader, DialogTitle} from '@/components/ui/dialog';
import {useOpenFlareServerUpgrade} from '@/lib/hooks/use-openflare-server-upgrade';
import type {AppUpdateStatus} from '@/lib/services/admin/types';
import {formatDateTime} from '@/lib/utils';
import {formatRelativeTime} from '@/app/(main)/nodes/components/node-utils';
function getUpgradeBadge(update: AppUpdateStatus | null | undefined) {
if (!update) {
return { label: '未检查', variant: 'outline' as const };
}
if (update.update_available) {
return { label: '可升级', variant: 'secondary' as const };
}
return { label: '最新', variant: 'default' as const };
}
export function VersionUpgradeDialog({
open,
onOpenChange,
canUpgrade = true,
}: {
open: boolean;
onOpenChange: (open: boolean) => void;
canUpgrade?: boolean;
}) {
const {
currentVersion,
update,
releaseErrorMessage,
isInitialLoading,
isChecking,
isUpgrading,
handleOpen,
handleCheckRelease,
handleUpgrade,
} = useOpenFlareServerUpgrade({ open, canUpgrade });
useEffect(() => {
if (open) {
handleOpen();
}
}, [open, handleOpen]);
const upgradeBadge = getUpgradeBadge(update);
const isBusy = isChecking || isUpgrading;
return (
<Dialog open={open} onOpenChange={onOpenChange}>
<DialogContent className="sm:max-w-3xl max-h-[90vh] overflow-y-auto">
<DialogHeader>
<DialogTitle>服务端版本</DialogTitle>
<DialogDescription>
检查上游 GitHub Release 并升级当前服务。升级开始后服务会短暂重启。
</DialogDescription>
</DialogHeader>
<div className="space-y-4">
<div className="grid gap-4 md:grid-cols-2">
<Card className="border-dashed shadow-none py-4 gap-3">
<CardHeader className="px-4 pb-0">
<CardTitle className="text-sm">当前版本</CardTitle>
</CardHeader>
<CardContent className="px-4">
<div className="flex flex-wrap items-center gap-2">
<p className="text-sm font-medium">{currentVersion}</p>
<Badge variant={upgradeBadge.variant}>{upgradeBadge.label}</Badge>
</div>
</CardContent>
</Card>
<Card className="border-dashed shadow-none py-4 gap-3">
<CardHeader className="px-4 pb-0">
<CardTitle className="text-sm">最新版本</CardTitle>
</CardHeader>
<CardContent className="px-4 space-y-3">
<p className="text-sm font-medium">{update?.latest_version || '未检查'}</p>
{canUpgrade ? (
<Button
type="button"
variant="outline"
size="sm"
disabled={isBusy}
onClick={handleCheckRelease}
>
{isChecking ? '检查中...' : '检查更新'}
</Button>
) : null}
</CardContent>
</Card>
</div>
{isInitialLoading ? (
<div className="flex items-center justify-center py-8 text-sm text-muted-foreground">
<Loader2 className="size-4 mr-2 animate-spin" />
加载版本信息...
</div>
) : null}
{!isInitialLoading && releaseErrorMessage ? (
<div className="rounded-lg border border-destructive/30 bg-destructive/5 px-4 py-3 text-sm text-destructive">
{releaseErrorMessage}
</div>
) : null}
{!isInitialLoading && !releaseErrorMessage && !update ? (
<div className="rounded-lg border border-dashed px-4 py-8 text-center text-sm text-muted-foreground">
尚未检查更新,点击「检查更新」后展示 GitHub Release 信息。
</div>
) : null}
{update ? (
<Card className="border-dashed shadow-none py-4 gap-3">
<CardHeader className="px-4 pb-0">
<CardTitle className="text-sm">GitHub Release · {update.latest_version}</CardTitle>
<CardDescription>
{update.published_at
? `发布时间:${formatRelativeTime(update.published_at)} · ${formatDateTime(update.published_at)}`
: '未提供发布时间'}
</CardDescription>
</CardHeader>
<CardContent className="px-4 space-y-4">
<div className="flex flex-wrap items-center gap-2">
<Badge variant={update.update_available ? 'secondary' : 'default'}>
{update.update_available ? '发现新版本' : '已经是最新版本'}
</Badge>
{update.prerelease ? (
<Badge variant="secondary">Preview 发布</Badge>
) : (
<Badge variant="outline">正式发布</Badge>
)}
{!update.can_upgrade ? (
<Badge variant="destructive">当前平台不支持自动升级</Badge>
) : null}
</div>
<div className="prose prose-sm dark:prose-invert max-w-none text-sm">
<ReactMarkdown remarkPlugins={[remarkGfm]}>
{update.release_notes || '暂无更新说明'}
</ReactMarkdown>
</div>
{update.release_url ? (
<a
href={update.release_url}
target="_blank"
rel="noreferrer"
className="inline-flex items-center text-sm text-primary hover:underline"
>
查看发布详情
<ExternalLink className="size-3 ml-1" />
</a>
) : null}
{canUpgrade ? (
<div className="flex justify-end">
<AlertDialog>
<AlertDialogTrigger asChild>
<Button
type="button"
disabled={
!update.update_available ||
isUpgrading ||
!update.can_upgrade ||
isBusy
}
>
{isUpgrading ? '升级中...' : '立即升级'}
</Button>
</AlertDialogTrigger>
<AlertDialogContent>
<AlertDialogHeader>
<AlertDialogTitle>升级到 {update.latest_version}?</AlertDialogTitle>
<AlertDialogDescription>
服务将下载并校验 {update.asset_name},随后替换当前二进制并重启。请确保安装目录可写,且服务允许原地重启。
</AlertDialogDescription>
</AlertDialogHeader>
<AlertDialogFooter>
<AlertDialogCancel>取消</AlertDialogCancel>
<AlertDialogAction onClick={handleUpgrade}>确认升级</AlertDialogAction>
</AlertDialogFooter>
</AlertDialogContent>
</AlertDialog>
</div>
) : null}
{!update.can_upgrade ? (
<p className="text-sm text-muted-foreground">
{update.current_version === 'dev'
? '开发构建没有可比较的 Release 版本,不能执行自动升级。'
: update.update_available
? '当前平台暂不支持自动替换二进制,请从 Release 页面手动升级。'
: '当前版本无需升级。'}
</p>
) : null}
</CardContent>
</Card>
) : null}
</div>
</DialogContent>
</Dialog>
);
}
@@ -289,7 +289,7 @@ export function NodeObservability({
const observabilityQuery = useQuery({
queryKey: ['openflare', 'node-observability', nodeId],
queryFn: () => NodeService.getObservability(nodeId, { hours: 24, limit: 48 }),
refetchInterval: 10000,
refetchInterval: 30000,
});
const cleanupMutation = useMutation({
@@ -661,7 +661,7 @@ export function NodeObservability({
</p>
<p className="mt-2 text-sm text-muted-foreground">
{trafficSummary
? `近 60 秒 · UV ${formatMetricCount(trafficSummary.unique_visitor_count)}`
? `近 60 秒 · 窗口UV ${formatMetricCount(trafficSummary.unique_visitor_count)}`
: '暂无窗口流量摘要'}
</p>
</div>
+1 -1
View File
@@ -28,7 +28,7 @@ export default function OpenFlareDashboardPage() {
const overviewQuery = useQuery({
queryKey: dashboardQueryKey,
queryFn: () => DashboardService.getOverview(),
refetchInterval: 30_000,
refetchInterval: 60_000,
});
const overview = overviewQuery.data;
@@ -20,6 +20,7 @@ import {Input} from '@/components/ui/input';
import {Label} from '@/components/ui/label';
import {Select, SelectContent, SelectItem, SelectTrigger, SelectValue,} from '@/components/ui/select';
import {DnsAccountService} from '@/lib/services/openflare';
import type {DnsAccountMutationPayload} from '@/lib/services/openflare';
import {getErrorMessage} from './website-utils';
@@ -52,7 +53,7 @@ export function DnsAccountCreateDialog({
});
const createMutation = useMutation({
mutationFn: DnsAccountService.create,
mutationFn: (payload: DnsAccountMutationPayload) => DnsAccountService.create(payload),
onSuccess: async () => {
await queryClient.invalidateQueries({queryKey: dnsAccountsQueryKey});
form.reset();
+6 -3
View File
@@ -7,8 +7,8 @@
@theme inline {
--color-background: var(--background);
--color-foreground: var(--foreground);
--font-sans: var(--font-geist-sans), system-ui, -apple-system, BlinkMacSystemFont, var(--font-noto-sans-sc), sans-serif;
--font-mono: var(--font-geist-mono), 'SF Mono', 'Monaco', 'Inconsolata', 'Roboto Mono', var(--font-noto-sans-sc), monospace;
--font-sans: var(--font-inter), 'PingFang SC', 'Microsoft YaHei', sans-serif;
--font-mono: var(--font-geist-mono), 'SF Mono', 'Monaco', 'Inconsolata', 'Roboto Mono', 'PingFang SC', 'Microsoft YaHei', monospace;
--color-sidebar-ring: var(--sidebar-ring);
--color-sidebar-border: var(--sidebar-border);
--color-sidebar-accent-foreground: var(--sidebar-accent-foreground);
@@ -25,6 +25,7 @@
--color-ring: var(--ring);
--color-input: var(--input);
--color-border: var(--border);
--color-destructive-foreground: var(--destructive-foreground);
--color-destructive: var(--destructive);
--color-accent-foreground: var(--accent-foreground);
--color-accent: var(--accent);
@@ -61,6 +62,7 @@
--accent: oklch(0.967 0.001 286.375);
--accent-foreground: oklch(0.21 0.006 285.885);
--destructive: oklch(0.577 0.245 27.325);
--destructive-foreground: oklch(98.5% 0% 0);
--border: oklch(0.92 0.004 286.32);
--input: oklch(0.92 0.004 286.32);
--ring: oklch(0.705 0.015 286.067);
@@ -95,6 +97,7 @@
--accent: oklch(0.274 0.006 286.033);
--accent-foreground: oklch(0.985 0 0);
--destructive: oklch(0.704 0.191 22.216);
--destructive-foreground: oklch(98.5% 0% 0);
--border: oklch(1 0 0 / 10%);
--input: oklch(1 0 0 / 15%);
--ring: oklch(0.552 0.016 285.938);
@@ -193,4 +196,4 @@
80% { transform: rotate(-1deg); }
100% { transform: rotate(0deg); }
}
}
}
+8 -1
View File
@@ -1,4 +1,5 @@
import type {Metadata} from "next";
import {Inter} from "next/font/google";
import {Toaster} from "@/components/ui/sonner";
import {ThemeProvider} from "@/components/layout/theme-provider";
import {CustomThemeProvider} from "@/lib/theme";
@@ -10,6 +11,12 @@ import {SiteTitleUpdater} from "@/components/providers/title-updater";
import {RobotsMeta} from "@/components/layout/robots-meta";
import "./globals.css";
const inter = Inter({
subsets: ["latin"],
variable: "--font-inter",
display: "swap",
});
export const metadata: Metadata = {
title: "OpenFlare",
description: "OpenFlare 边缘节点与反向代理管理平台",
@@ -23,7 +30,7 @@ export default function RootLayout({
return (
<html
lang="zh-CN"
className="hide-scrollbar font-sans"
className={`hide-scrollbar font-sans ${inter.variable}`}
suppressHydrationWarning
>
<body
+1 -1
View File
@@ -73,7 +73,7 @@ export function LoginForm({ onOTPStateChange }: { onOTPStateChange?: (show: bool
queryFn: () => AuthService.getAuthSources(),
})
const capEnabled = configBool(publicConfigQuery.data?.cap_login_enabled, true)
const capEnabled = configBool(publicConfigQuery.data?.cap_login_enabled, false)
const capAutoSolve = configBool(publicConfigQuery.data?.cap_auto_solve, true)
const loginMutation = useMutation({
+1 -1
View File
@@ -68,7 +68,7 @@ export function RegisterForm() {
const emailRegisterEnabled = configBool(publicConfigQuery.data?.email_register_verification_enabled, false)
const capEnabled = configBool(publicConfigQuery.data?.cap_login_enabled, true)
const capEnabled = configBool(publicConfigQuery.data?.cap_login_enabled, false)
const capAutoSolve = configBool(publicConfigQuery.data?.cap_auto_solve, true)
const [capScope, setCapScope] = useState<'send_email_code' | 'register'>('send_email_code')
+1 -1
View File
@@ -245,7 +245,7 @@ export const apiSections: PolicySection[] = [
"registration_enabled": "false",
"password_login_enabled": "true",
"password_register_enabled": "false",
"cap_login_enabled": "true",
"cap_login_enabled": "false",
"oidc_login_enabled": "true"
}
}`}
+2 -12
View File
@@ -5,7 +5,7 @@ import {AnimatePresence, motion} from "motion/react"
import {useUser} from "@/contexts/user-context"
import {Card, CardHeader, CardTitle} from "@/components/ui/card"
import {Button} from "@/components/ui/button"
import {ArrowRight, ExternalLink, FileText, HelpCircle, Layers, Shield, ShieldCheck, Terminal, User} from "lucide-react"
import {ArrowRight, ExternalLink, HelpCircle, Layers, Shield, ShieldCheck, Terminal, User} from "lucide-react"
import Link from "next/link"
export function HomeMain() {
@@ -22,21 +22,11 @@ export function HomeMain() {
bgColor: "bg-blue-500/10",
borderColor: "hover:border-blue-500/30",
},
{
title: "开发接口文档",
description: "查看开放平台的 RESTful 接口规格说明",
icon: FileText,
url: "/docs/api",
color: "text-emerald-500",
bgColor: "bg-emerald-500/10",
borderColor: "hover:border-emerald-500/30",
external: true,
},
{
title: "使用文档",
description: "学习如何集成 API 及日常操作帮助指南",
icon: HelpCircle,
url: "/docs/how-to-use",
url: "https://open-flare.pages.dev/",
color: "text-purple-500",
bgColor: "bg-purple-500/10",
borderColor: "hover:border-purple-500/30",
@@ -194,7 +194,7 @@ export function AccessTokenMain() {
<CardContent className="pt-6 space-y-4">
{accessTokensQuery.isPending ? (
<div className="flex items-center justify-center py-8">
<Loader2 className="size-6 animate-spin text-indigo-500" />
<Loader2 className="size-6 animate-spin text-primary" />
</div>
) : (accessTokensQuery.data ?? []).length > 0 ? (
<div className="space-y-3">
@@ -231,7 +231,7 @@ export function AccessTokenMain() {
type="button"
variant="outline"
size="sm"
className="text-xs border-dashed text-muted-foreground hover:text-indigo-500 hover:bg-indigo-500/5 rounded-lg h-8 px-2.5"
className="text-xs border-dashed text-muted-foreground hover:text-primary hover:bg-primary/5 rounded-lg h-8 px-2.5"
onClick={() => handleCopyText(token.masked_token, token.id)}
>
{copiedId === token.id ? (
@@ -245,7 +245,7 @@ export function AccessTokenMain() {
type="button"
variant="outline"
size="sm"
className="text-xs border-dashed text-muted-foreground hover:text-indigo-500 hover:bg-indigo-500/5 rounded-lg h-8 px-2.5"
className="text-xs border-dashed text-muted-foreground hover:text-primary hover:bg-primary/5 rounded-lg h-8 px-2.5"
onClick={() => handleRotateToken(token.id, token.name)}
disabled={rotateTokenMutation.isPending}
>
@@ -305,7 +305,7 @@ export function AccessTokenMain() {
value={tokenName}
onChange={(e) => setTokenName(e.target.value)}
disabled={createTokenMutation.isPending}
className="rounded-xl border border-dashed focus:border-indigo-500 focus:ring-0 focus-visible:ring-0"
className="rounded-xl border border-dashed focus:border-primary focus:ring-0 focus-visible:ring-0"
/>
</div>
{user?.is_admin && (
@@ -375,15 +375,15 @@ export function AccessTokenMain() {
{newCreatedToken && (
<div className="space-y-4 py-2">
{/* 明文 Token 文本框 */}
<div className="flex items-center gap-2 rounded-xl bg-indigo-500/5 border border-dashed border-indigo-500/30 p-3">
<span className="font-mono text-xs select-all break-all flex-1 text-indigo-600 font-semibold leading-relaxed">
<div className="flex items-center gap-2 rounded-xl bg-primary/5 border border-dashed border-primary/30 p-3">
<span className="font-mono text-xs select-all break-all flex-1 text-primary font-semibold leading-relaxed">
{newCreatedToken.token}
</span>
<Button
type="button"
size="icon"
variant="outline"
className="size-8 rounded-lg shrink-0 border-dashed text-indigo-500 hover:bg-indigo-500/10 hover:border-indigo-500/30 transition-colors"
className="size-8 rounded-lg shrink-0 border-dashed text-primary hover:bg-primary/10 hover:border-primary/30 transition-colors"
onClick={() => handleCopyText(newCreatedToken.token, 9999)}
>
{copiedId === 9999 ? (
@@ -88,7 +88,7 @@ export function OperationTab({ configs, systemConfigsQuery }: OperationTabProps)
<Card className="border border-dashed shadow-sm">
<CardHeader className="border-b border-dashed pb-4">
<div className="flex items-center gap-2">
<div className="p-1.5 rounded-lg bg-indigo-500/10 text-indigo-500">
<div className="p-1.5 rounded-lg bg-primary/10 text-primary">
<KeyRound className="size-4" />
</div>
<div>
@@ -125,9 +125,9 @@ export function OperationTab({ configs, systemConfigsQuery }: OperationTabProps)
</div>
{/* 当前白名单列表 */}
<div className="rounded-xl border border-dashed p-4 bg-card hover:bg-muted/10 hover:border-indigo-500/30 transition-all duration-300 shadow-sm space-y-3">
<div className="rounded-xl border border-dashed p-4 bg-card hover:bg-muted/10 hover:border-primary/30 transition-all duration-300 shadow-sm space-y-3">
<div className="flex items-center gap-2">
<ShieldAlert className="size-4 text-indigo-500" />
<ShieldAlert className="size-4 text-primary" />
<span className="font-medium text-sm text-foreground">当前免鉴权列表</span>
</div>
@@ -137,14 +137,14 @@ export function OperationTab({ configs, systemConfigsQuery }: OperationTabProps)
<Badge
key={type}
variant="secondary"
className="px-2.5 py-1 text-xs gap-1.5 flex items-center bg-indigo-500/10 text-indigo-700 dark:text-indigo-300 dark:bg-indigo-500/20 border border-indigo-500/20"
className="px-2.5 py-1 text-xs gap-1.5 flex items-center bg-primary/10 text-primary dark:bg-primary/20 border border-primary/20"
>
{availableTypes.find(t => t.value === type)?.label || type}
<button
type="button"
onClick={() => handleRemoveType(type)}
disabled={updateWhitelistMutation.isPending || systemConfigsQuery.isPending}
className="rounded-full outline-hidden hover:bg-indigo-500/20 p-0.5 text-indigo-600 dark:text-indigo-400 cursor-pointer disabled:cursor-not-allowed"
className="rounded-full outline-hidden hover:bg-primary/20 p-0.5 text-primary cursor-pointer disabled:cursor-not-allowed"
>
<X className="size-3" />
</button>
@@ -4,8 +4,6 @@ import {ComponentType, useMemo} from "react"
import {useMutation, useQueryClient} from "@tanstack/react-query"
import {
Bell,
Code,
CreditCard,
Database,
FileText,
FolderOpen,
@@ -61,9 +59,7 @@ const MENU_GROUPS: MenuGroup[] = [
{
name: "文档菜单",
items: [
{ path: "/admin/demo", label: "规范示例", description: "内置 UI 组件与设计规范的展示、调试与参考", icon: Code },
{ path: "/docs/api", label: "接口文档", description: "系统 Swagger 交互式 API 接口文档", icon: CreditCard },
{ path: "/docs/how-to-use", label: "使用文档", description: "面向开发与运营的部署使用指南", icon: FileText },
{ path: "https://open-flare.pages.dev/", label: "使用文档", description: "面向开发与运营的部署使用指南", icon: FileText },
]
}
]
@@ -112,7 +108,7 @@ export function OtherTab({ configs }: OtherTabProps) {
<Card className="border border-dashed shadow-sm">
<CardHeader className="border-b border-dashed pb-4">
<div className="flex items-center gap-2">
<div className="p-1.5 rounded-lg bg-indigo-500/10 text-indigo-500">
<div className="p-1.5 rounded-lg bg-primary/10 text-primary">
<LayoutList className="size-4" />
</div>
<div>
@@ -141,11 +137,11 @@ export function OtherTab({ configs }: OtherTabProps) {
return (
<div
key={item.path}
className="flex items-center justify-between gap-4 rounded-xl border border-dashed p-4 bg-card hover:bg-muted/10 hover:border-indigo-500/30 transition-all duration-300 shadow-sm"
className="flex items-center justify-between gap-4 rounded-xl border border-dashed p-4 bg-card hover:bg-muted/10 hover:border-primary/30 transition-all duration-300 shadow-sm"
>
<div className="space-y-1.5 flex-1 min-w-0 pr-2">
<div className="flex items-center gap-2">
{Icon && <Icon className="size-4 text-indigo-500 shrink-0" />}
{Icon && <Icon className="size-4 text-primary shrink-0" />}
<span className="font-medium text-sm text-foreground truncate">{item.label}</span>
{isReadOnly && (
<span className="text-[9px] px-1.5 py-0.5 rounded bg-muted text-muted-foreground border shrink-0">
@@ -171,8 +167,8 @@ export function OtherTab({ configs }: OtherTabProps) {
</div>
))}
<div className="p-3.5 rounded-lg border border-dashed border-indigo-500/20 bg-indigo-500/5 flex items-start gap-2.5">
<Info className="size-4 text-indigo-500 shrink-0 mt-0.5" />
<div className="p-3.5 rounded-lg border border-dashed border-primary/20 bg-primary/5 flex items-start gap-2.5">
<Info className="size-4 text-primary shrink-0 mt-0.5" />
<div className="text-xs text-muted-foreground leading-relaxed">
<span className="font-semibold text-foreground">安全提示:</span>
为了防止管理员在关闭“系统设置”后导致无法重新访问此配置页,系统限制了“系统设置”的关闭权限。其它所有菜单均可自由开关,隐藏后对应的分组标题在为空时也会自动隐藏。
@@ -275,7 +275,7 @@ export function ProfileMain() {
>
<Avatar className="size-20 md:size-24 border-2 border-primary/10 shadow-md">
<AvatarImage src={user.avatar_url} alt={user.nickname || user.username} />
<AvatarFallback className="text-2xl bg-indigo-600 text-white font-bold">
<AvatarFallback className="text-2xl bg-primary text-primary-foreground font-bold">
{(user.nickname || user.username).slice(0, 2).toUpperCase()}
</AvatarFallback>
</Avatar>
@@ -342,7 +342,7 @@ export function ProfileMain() {
href={user.website.startsWith("http") ? user.website : `http://${user.website}`}
target="_blank"
rel="noopener noreferrer"
className="text-indigo-600 hover:underline"
className="text-primary hover:underline"
>
{user.website}
</a>
@@ -473,7 +473,7 @@ export function ProfileMain() {
<div className="space-y-6 bg-card border border-dashed rounded-lg p-6 flex flex-col justify-between">
<div>
<div className="border-b pb-4 flex items-center gap-2">
<div className="p-1.5 rounded-lg bg-indigo-500/10 text-indigo-500">
<div className="p-1.5 rounded-lg bg-primary/10 text-primary">
<Link2 className="size-4" />
</div>
<div>
@@ -487,7 +487,7 @@ export function ProfileMain() {
<h3 className="text-[11px] font-semibold text-muted-foreground uppercase tracking-wider">已绑定账号</h3>
{externalAccountBindingsQuery.isPending ? (
<div className="flex items-center justify-center py-4">
<Loader2 className="size-4 animate-spin text-indigo-500" />
<Loader2 className="size-4 animate-spin text-primary" />
</div>
) : (externalAccountBindingsQuery.data ?? []).length > 0 ? (
<div className="space-y-2">
@@ -534,7 +534,7 @@ export function ProfileMain() {
<h3 className="text-[11px] font-semibold text-muted-foreground uppercase tracking-wider">绑定新账号</h3>
{publicAuthSourcesQuery.isPending ? (
<div className="flex items-center justify-center py-4">
<Loader2 className="size-4 animate-spin text-indigo-500" />
<Loader2 className="size-4 animate-spin text-primary" />
</div>
) : (publicAuthSourcesQuery.data ?? []).length > 0 ? (
<div className="grid grid-cols-1 gap-2">
@@ -543,16 +543,16 @@ export function ProfileMain() {
key={source.id}
type="button"
variant="outline"
className="flex items-center justify-between w-full border border-dashed rounded-xl px-3 py-2 text-left font-normal text-xs hover:bg-indigo-500/5 hover:text-indigo-500 hover:border-indigo-500/30 transition-all duration-300 group h-8"
className="flex items-center justify-between w-full border border-dashed rounded-xl px-3 py-2 text-left font-normal text-xs hover:bg-primary/5 hover:text-primary hover:border-primary/30 transition-all duration-300 group h-8"
onClick={() => {
void bindSourceMutation.mutateAsync(source.name)
}}
>
<div className="flex items-center gap-1.5">
<Link2 className="size-3 text-muted-foreground group-hover:text-indigo-500" />
<Link2 className="size-3 text-muted-foreground group-hover:text-primary" />
<span>绑定 {source.display_name || source.name}</span>
</div>
<ArrowRight className="size-3 opacity-0 -translate-x-1 group-hover:opacity-100 group-hover:translate-x-0 transition-all text-indigo-500" />
<ArrowRight className="size-3 opacity-0 -translate-x-1 group-hover:opacity-100 group-hover:translate-x-0 transition-all text-primary" />
</Button>
))}
</div>
@@ -583,7 +583,7 @@ export function ProfileMain() {
>
<Avatar className="size-20 border-2 border-primary/5 shadow-md">
<AvatarImage src={avatarUrl} alt={nickname} />
<AvatarFallback className="text-xl bg-indigo-600 text-white font-bold">
<AvatarFallback className="text-xl bg-primary text-primary-foreground font-bold">
{(nickname || "U").slice(0, 2).toUpperCase()}
</AvatarFallback>
</Avatar>
@@ -245,7 +245,7 @@ export function SecurityTab({ configs, systemConfigsQuery }: SecurityTabProps) {
<Card className="border border-dashed shadow-sm">
<CardHeader className="border-b border-dashed pb-4">
<div className="flex items-center gap-2">
<div className="p-1.5 rounded-lg bg-indigo-500/10 text-indigo-500">
<div className="p-1.5 rounded-lg bg-primary/10 text-primary">
<Settings className="size-4" />
</div>
<div>
@@ -263,11 +263,11 @@ export function SecurityTab({ configs, systemConfigsQuery }: SecurityTabProps) {
return (
<div
key={item.key}
className="flex items-center justify-between gap-4 rounded-xl border border-dashed p-4 bg-card hover:bg-muted/10 hover:border-indigo-500/30 transition-all duration-300 shadow-sm"
className="flex items-center justify-between gap-4 rounded-xl border border-dashed p-4 bg-card hover:bg-muted/10 hover:border-primary/30 transition-all duration-300 shadow-sm"
>
<div className="space-y-1">
<div className="flex items-center gap-2">
{Icon && <Icon className="size-4 text-indigo-500" />}
{Icon && <Icon className="size-4 text-primary" />}
<span className="font-medium text-sm text-foreground">{item.title}</span>
</div>
<p className="text-xs text-muted-foreground leading-relaxed pr-2">{item.description}</p>
@@ -283,11 +283,11 @@ export function SecurityTab({ configs, systemConfigsQuery }: SecurityTabProps) {
{/* 登录状态保持时间 (选择后立即更改) */}
<div
className="flex items-center justify-between gap-4 rounded-xl border border-dashed p-4 bg-card hover:bg-muted/10 hover:border-indigo-500/30 transition-all duration-300 shadow-sm md:col-span-2"
className="flex items-center justify-between gap-4 rounded-xl border border-dashed p-4 bg-card hover:bg-muted/10 hover:border-primary/30 transition-all duration-300 shadow-sm md:col-span-2"
>
<div className="space-y-1 pr-4">
<div className="flex items-center gap-2">
<Clock className="size-4 text-indigo-500" />
<Clock className="size-4 text-primary" />
<span className="font-medium text-sm text-foreground">登录状态保持时间</span>
</div>
<p className="text-xs text-muted-foreground leading-relaxed pr-2">
@@ -340,7 +340,7 @@ export function SecurityTab({ configs, systemConfigsQuery }: SecurityTabProps) {
<Card className="border border-dashed shadow-sm">
<CardHeader className="border-b border-dashed pb-4 flex flex-row items-center justify-between gap-4">
<div className="flex items-center gap-2">
<div className="p-1.5 rounded-lg bg-indigo-500/10 text-indigo-500">
<div className="p-1.5 rounded-lg bg-primary/10 text-primary">
<Globe className="size-4" />
</div>
<div>
@@ -390,7 +390,7 @@ export function SecurityTab({ configs, systemConfigsQuery }: SecurityTabProps) {
<div className="flex items-center gap-4">
<span className={`text-xs px-2.5 py-1 rounded-lg border font-medium hidden sm:inline-block ${
source.client_secret_configured
? "bg-indigo-500/5 text-indigo-500 border-indigo-500/10"
? "bg-primary/5 text-primary border-primary/10"
: "bg-rose-500/5 text-rose-500 border-rose-500/10"
}`}>
{source.client_secret_configured ? "Secret 已配置" : "Secret 未配置"}
@@ -407,7 +407,7 @@ export function SecurityTab({ configs, systemConfigsQuery }: SecurityTabProps) {
type="button"
variant="ghost"
size="icon"
className="size-8 text-muted-foreground hover:text-indigo-500 hover:bg-indigo-500/10 rounded-lg transition-colors"
className="size-8 text-muted-foreground hover:text-primary hover:bg-primary/10 rounded-lg transition-colors"
onClick={() => {
setSelectedSource(source)
setAuthSourceModalOpen(true)
@@ -458,7 +458,7 @@ export function SecurityTab({ configs, systemConfigsQuery }: SecurityTabProps) {
<Card className="border border-dashed shadow-sm">
<CardHeader className="border-b border-dashed pb-4 flex flex-row items-center justify-between gap-4">
<div className="flex items-center gap-2">
<div className="p-1.5 rounded-lg bg-indigo-500/10 text-indigo-500">
<div className="p-1.5 rounded-lg bg-primary/10 text-primary">
<Shield className="size-4" />
</div>
<div>
@@ -193,7 +193,7 @@ export function SystemTab({ configs, systemConfigsQuery }: SystemTabProps) {
<Card className="border border-dashed shadow-sm">
<CardHeader className="border-b border-dashed pb-4">
<div className="flex items-center gap-2">
<div className="rounded-lg bg-indigo-500/10 p-1.5 text-indigo-500">
<div className="rounded-lg bg-primary/10 p-1.5 text-primary">
<Server className="size-4" />
</div>
<div>
@@ -310,7 +310,7 @@ export function SystemTab({ configs, systemConfigsQuery }: SystemTabProps) {
<Card className="border border-dashed shadow-sm">
<CardHeader className="border-b border-dashed pb-4">
<div className="flex items-center gap-2">
<div className="p-1.5 rounded-lg bg-indigo-500/10 text-indigo-500">
<div className="p-1.5 rounded-lg bg-primary/10 text-primary">
<Mail className="size-4" />
</div>
<div>
@@ -124,7 +124,7 @@ export function TemplatesManager() {
<Card className="border border-dashed shadow-sm">
<CardHeader className="border-b border-dashed pb-4 flex flex-row items-center justify-between gap-4">
<div className="flex items-center gap-2">
<div className="p-1.5 rounded-lg bg-indigo-500/10 text-indigo-500">
<div className="p-1.5 rounded-lg bg-primary/10 text-primary">
<FileText className="size-4" />
</div>
<div>
@@ -154,14 +154,14 @@ export function TemplatesManager() {
{(templatesQuery.data ?? []).map((tmpl) => (
<div
key={tmpl.id}
className="flex flex-col sm:flex-row sm:items-center justify-between gap-4 rounded-xl border border-dashed p-4 bg-card hover:bg-muted/10 hover:border-indigo-500/30 transition-all duration-300 shadow-sm"
className="flex flex-col sm:flex-row sm:items-center justify-between gap-4 rounded-xl border border-dashed p-4 bg-card hover:bg-muted/10 hover:border-primary/30 transition-all duration-300 shadow-sm"
>
<div className="space-y-1.5">
<div className="flex items-center gap-2 flex-wrap">
<span className="font-semibold text-sm text-foreground">{tmpl.name}</span>
<span className={`text-[10px] px-2 py-0.5 rounded-full border font-medium ${
tmpl.is_system
? "bg-indigo-500/10 text-indigo-500 border-indigo-500/20"
? "bg-primary/10 text-primary border-primary/20"
: "bg-amber-500/10 text-amber-500 border-amber-500/20"
}`}>
{tmpl.is_system ? "系统内置" : "自定义"}
@@ -171,7 +171,7 @@ export function TemplatesManager() {
</span>
</div>
<div className="text-xs text-muted-foreground">
标识符: <span className="font-mono text-indigo-500 bg-indigo-500/5 px-1.5 py-0.5 rounded">{tmpl.key}</span>
标识符: <span className="font-mono text-primary bg-primary/5 px-1.5 py-0.5 rounded">{tmpl.key}</span>
{tmpl.subject && ` · 主题: ${tmpl.subject}`}
</div>
{tmpl.description && (
@@ -185,7 +185,7 @@ export function TemplatesManager() {
type="button"
variant="ghost"
size="icon"
className="size-8 text-muted-foreground hover:text-indigo-500 hover:bg-indigo-500/10 rounded-lg transition-colors"
className="size-8 text-muted-foreground hover:text-primary hover:bg-primary/10 rounded-lg transition-colors"
onClick={() => handleOpenEdit(tmpl)}
>
<Pencil className="size-4" />
@@ -126,10 +126,10 @@ curl -X POST https://api.example.com/api/v1/auth/register \\
</ul>
<div className="flex flex-wrap gap-4">
<Link href="/docs/api">
<Link href="https://open-flare.pages.dev/" target="_blank" rel="noopener noreferrer">
<Button variant="secondary" className="rounded-full text-xs hover:bg-muted-foreground/10">
<Book className="w-3 h-3" />
API 文档
使用文档
</Button>
</Link>
</div>
+1 -2
View File
@@ -43,8 +43,7 @@ export const FooterSection = React.memo(function FooterSection({ className }: Fo
<div className="lg:col-span-1">
<h3 className="font-semibold text-foreground mb-6">开发</h3>
<ul className="space-y-4 text-sm text-muted-foreground">
<li><FooterLink href="/docs/how-to-use">快速开始</FooterLink></li>
<li><FooterLink href="/docs/api">API 文档</FooterLink></li>
<li><FooterLink href="https://open-flare.pages.dev/">使用文档</FooterLink></li>
<li><FooterLink href="https://github.com/Rain-kl/OpenFlare">源代码</FooterLink></li>
</ul>
</div>
+1 -1
View File
@@ -72,7 +72,7 @@ export const HeroSection = React.memo(function HeroSection({ className }: HeroSe
</Button>
</Link>
<Link href="/docs/how-to-use" className="w-full sm:w-auto">
<Link href="https://open-flare.pages.dev/" target="_blank" rel="noopener noreferrer" className="w-full sm:w-auto">
<Button
variant="secondary"
size="lg"
+2 -6
View File
@@ -47,8 +47,6 @@ import {
ArrowUpRight,
Bell,
ChevronDown,
Code,
CreditCard,
Database,
FileQuestionMark,
FileText,
@@ -79,9 +77,7 @@ const data = {
{ title: "系统设置", url: "/admin/settings", icon: Settings },
],
document: [
{ title: "规范示例", url: "/admin/demo", icon: Code },
{ title: "接口文档", url: "/docs/api", icon: CreditCard, external: true },
{ title: "使用文档", url: "/docs/how-to-use", icon: FileText, external: true },
{ title: "使用文档", url: "https://open-flare.pages.dev/", icon: FileText, external: true },
],
}
@@ -281,7 +277,7 @@ export function AppSidebar({ ...props }: React.ComponentProps<typeof Sidebar>) {
</DropdownMenuItem>
<DropdownMenuSeparator className="my-2" />
<DropdownMenuItem onClick={() => {
router.push("/docs/how-to-use")
window.open("https://open-flare.pages.dev/", "_blank", "noopener,noreferrer")
handleCloseSidebar()
}}>
<FileQuestionMark className="mr-2 size-4" />
+1 -1
View File
@@ -473,7 +473,7 @@ function SidebarMenuItem({ className, ...props }: React.ComponentProps<"li">) {
}
const sidebarMenuButtonVariants = cva(
"peer/menu-button flex w-full items-center gap-2 overflow-hidden rounded-md p-2 text-left text-sm outline-hidden ring-sidebar-ring transition-[width,height,padding] hover:bg-sidebar-accent hover:text-sidebar-accent-foreground focus-visible:ring-2 active:bg-sidebar-accent active:text-sidebar-accent-foreground disabled:pointer-events-none disabled:opacity-50 group-has-data-[sidebar=menu-action]/menu-item:pr-8 aria-disabled:pointer-events-none aria-disabled:opacity-50 data-[active=true]:font-bold data-[active=true]:text-[#6366F1] data-[state=open]:hover:bg-sidebar-accent data-[state=open]:hover:text-sidebar-accent-foreground group-data-[collapsible=icon]:size-8! group-data-[collapsible=icon]:p-2! [&>span:last-child]:truncate [&>svg]:size-4 [&>svg]:shrink-0",
"peer/menu-button flex w-full items-center gap-2 overflow-hidden rounded-md p-2 text-left text-sm outline-hidden ring-sidebar-ring transition-[width,height,padding] hover:bg-sidebar-accent hover:text-sidebar-accent-foreground focus-visible:ring-2 active:bg-sidebar-accent active:text-sidebar-accent-foreground disabled:pointer-events-none disabled:opacity-50 group-has-data-[sidebar=menu-action]/menu-item:pr-8 aria-disabled:pointer-events-none aria-disabled:opacity-50 data-[active=true]:font-bold data-[active=true]:text-sidebar-primary data-[state=open]:hover:bg-sidebar-accent data-[state=open]:hover:text-sidebar-accent-foreground group-data-[collapsible=icon]:size-8! group-data-[collapsible=icon]:p-2! [&>span:last-child]:truncate [&>svg]:size-4 [&>svg]:shrink-0",
{
variants: {
variant: {
+35 -5
View File
@@ -3,7 +3,7 @@
import * as React from "react"
import {createContext, useCallback, useContext, useEffect, useRef, useState} from "react"
import {toast} from "sonner"
import type {AdminUser, CreateUserRequest, ListUsersRequest} from "@/lib/services/admin"
import type {AdminUser, CreateUserRequest, ListUsersRequest, UpdateUserRequest} from "@/lib/services/admin"
import services from "@/lib/services"
/** 用户列表查询参数 */
@@ -12,6 +12,7 @@ export interface UserQueryParams {
page_size: number
user_id?: string
username?: string
email?: string
status?: 'all' | 'active' | 'inactive'
}
@@ -27,6 +28,7 @@ interface AdminUsersContextState {
pageSize: number
searchUserId: string
searchUsername: string
searchEmail: string
statusFilter: 'all' | 'active' | 'inactive'
// Actions
@@ -34,6 +36,7 @@ interface AdminUsersContextState {
setPageSize: (size: number) => void
setSearchUserId: (userId: string) => void
setSearchUsername: (username: string) => void
setSearchEmail: (email: string) => void
setStatusFilter: (status: 'all' | 'active' | 'inactive') => void
fetchUsers: (force?: boolean) => Promise<void>
@@ -41,6 +44,7 @@ interface AdminUsersContextState {
getUserDetail: (id: string) => Promise<AdminUser>
updateUserStatus: (user: AdminUser) => Promise<void>
createUser: (req: CreateUserRequest) => Promise<AdminUser>
updateUser: (id: string, req: UpdateUserRequest) => Promise<void>
deleteUser: (user: AdminUser) => Promise<void>
}
@@ -60,9 +64,11 @@ export function AdminUsersProvider({ children }: { children: React.ReactNode })
const [pageSize, setPageSize] = useState(20)
const [searchUserId, setSearchUserId] = useState("")
const [searchUsername, setSearchUsername] = useState("")
const [searchEmail, setSearchEmail] = useState("")
const [statusFilter, setStatusFilter] = useState<'all' | 'active' | 'inactive'>('all')
const [debouncedSearchUserId, setDebouncedSearchUserId] = useState("")
const [debouncedSearchUsername, setDebouncedSearchUsername] = useState("")
const [debouncedSearchEmail, setDebouncedSearchEmail] = useState("")
// Cache
const cacheRef = useRef<Record<string, { data: AdminUser[], total: number, timestamp: number }>>({})
@@ -73,12 +79,17 @@ export function AdminUsersProvider({ children }: { children: React.ReactNode })
const timer = setTimeout(() => {
setDebouncedSearchUserId(searchUserId)
setDebouncedSearchUsername(searchUsername)
if (searchUserId !== debouncedSearchUserId || searchUsername !== debouncedSearchUsername) {
setDebouncedSearchEmail(searchEmail)
if (
searchUserId !== debouncedSearchUserId ||
searchUsername !== debouncedSearchUsername ||
searchEmail !== debouncedSearchEmail
) {
setPage(1) // Reset to page 1 on search change
}
}, 500)
return () => clearTimeout(timer)
}, [searchUserId, searchUsername, debouncedSearchUserId, debouncedSearchUsername])
}, [searchUserId, searchUsername, searchEmail, debouncedSearchUserId, debouncedSearchUsername, debouncedSearchEmail])
const generateCacheKey = (params: UserQueryParams) => {
return JSON.stringify(params)
@@ -90,6 +101,7 @@ export function AdminUsersProvider({ children }: { children: React.ReactNode })
page_size: pageSize,
user_id: debouncedSearchUserId || undefined,
username: debouncedSearchUsername || undefined,
email: debouncedSearchEmail || undefined,
status: statusFilter
}
@@ -127,7 +139,8 @@ export function AdminUsersProvider({ children }: { children: React.ReactNode })
page,
page_size: pageSize,
user_id: debouncedSearchUserId || undefined,
username: debouncedSearchUsername || undefined
username: debouncedSearchUsername || undefined,
email: debouncedSearchEmail || undefined
}
const response = await services.adminUser.listUsers(requestParams)
@@ -160,7 +173,7 @@ export function AdminUsersProvider({ children }: { children: React.ReactNode })
setLoading(false)
}
}
}, [page, pageSize, debouncedSearchUserId, debouncedSearchUsername, statusFilter])
}, [page, pageSize, debouncedSearchUserId, debouncedSearchUsername, debouncedSearchEmail, statusFilter])
// Removed auto-fetch useEffect. Consumer (UsersManager) should trigger fetch.
@@ -224,6 +237,20 @@ export function AdminUsersProvider({ children }: { children: React.ReactNode })
}
}
const updateUser = async (id: string, req: UpdateUserRequest) => {
try {
await services.adminUser.updateUser(id, req)
setUsers(prev => prev.map(u =>
u.id === id ? { ...u, ...req } : u
))
cacheRef.current = {}
toast.success('更新用户信息成功')
} catch (err) {
toast.error(err instanceof Error ? err.message : '更新用户信息失败')
throw err
}
}
const value = {
users,
total,
@@ -233,17 +260,20 @@ export function AdminUsersProvider({ children }: { children: React.ReactNode })
pageSize,
searchUserId,
searchUsername,
searchEmail,
statusFilter,
setPage,
setPageSize,
setSearchUserId,
setSearchUsername,
setSearchEmail,
setStatusFilter,
fetchUsers,
refresh,
getUserDetail,
updateUserStatus,
createUser,
updateUser,
deleteUser
}
@@ -1,127 +0,0 @@
'use client';
import {useMutation, useQuery} from '@tanstack/react-query';
import {useCallback, useEffect, useRef, useState} from 'react';
import {AdminStatusService} from '@/lib/services/admin';
import type {AppUpdateStatus} from '@/lib/services/admin/types';
import {StatusService} from '@/lib/services/openflare';
export const openflarePublicStatusQueryKey = ['openflare', 'public-status'] as const;
export const adminUpdateStatusQueryKey = ['admin', 'update'] as const;
export function useOpenFlareServerUpgrade({
open,
canUpgrade,
}: {
open: boolean;
canUpgrade: boolean;
}) {
const [feedback, setFeedback] = useState<string | null>(null);
const upgradeReloadStartedRef = useRef(false);
const upgradeReloadTimerRef = useRef<number | null>(null);
const statusQuery = useQuery({
queryKey: openflarePublicStatusQueryKey,
queryFn: () => StatusService.getPublicStatus(),
enabled: open,
});
const updateQuery = useQuery({
queryKey: adminUpdateStatusQueryKey,
queryFn: () => AdminStatusService.getUpdateStatus(),
enabled: open && canUpgrade,
staleTime: 5 * 60 * 1000,
});
const scheduleUpgradePageReload = useCallback(() => {
if (upgradeReloadStartedRef.current) {
return;
}
upgradeReloadStartedRef.current = true;
setFeedback('服务升级已进入重启阶段,页面将在服务恢复后自动刷新。');
const reloadWhenServerReady = async () => {
try {
await StatusService.getPublicStatus();
window.location.reload();
} catch {
upgradeReloadTimerRef.current = window.setTimeout(reloadWhenServerReady, 1500);
}
};
upgradeReloadTimerRef.current = window.setTimeout(reloadWhenServerReady, 1200);
}, []);
useEffect(() => {
return () => {
if (upgradeReloadTimerRef.current !== null) {
window.clearTimeout(upgradeReloadTimerRef.current);
}
};
}, []);
const upgradeMutation = useMutation({
mutationFn: () => AdminStatusService.applyUpdate(),
onSuccess: () => {
scheduleUpgradePageReload();
setFeedback('升级包已校验完成,服务正在重启。');
},
onError: (error) => {
setFeedback(error instanceof Error ? error.message : '升级失败,请稍后重试。');
},
});
const resetTransientState = useCallback(() => {
setFeedback(null);
upgradeReloadStartedRef.current = false;
}, []);
const handleOpen = useCallback(() => {
resetTransientState();
if (canUpgrade) {
void updateQuery.refetch();
}
}, [canUpgrade, resetTransientState, updateQuery]);
const handleCheckRelease = useCallback(() => {
setFeedback(null);
if (!canUpgrade) {
return;
}
void updateQuery.refetch();
}, [canUpgrade, updateQuery]);
const handleUpgrade = useCallback(() => {
setFeedback(null);
upgradeMutation.mutate();
}, [upgradeMutation]);
const update = updateQuery.data;
const releaseErrorMessage =
feedback ||
(updateQuery.isError
? updateQuery.error instanceof Error
? updateQuery.error.message
: '版本检查失败,请稍后重试。'
: undefined);
const currentVersion = statusQuery.data?.version || update?.current_version || 'unknown';
return {
currentVersion,
update,
releaseErrorMessage,
isInitialLoading: updateQuery.isLoading && !updateQuery.data && canUpgrade,
isChecking: updateQuery.isFetching,
isUpgrading: upgradeMutation.isPending,
handleOpen,
handleCheckRelease,
handleUpgrade,
};
}
export type {AppUpdateStatus};
+2 -1
View File
@@ -27,6 +27,7 @@ export type {
ListUsersRequest,
ListUsersResponse,
UpdateUserStatusRequest,
UpdateUserRequest,
SystemStatus,
AppUpdateStatus,
Schedule,
@@ -40,4 +41,4 @@ export type {
StorageDriver,
StorageConfig,
ObjectStorageConfig,
} from './types';
} from './types';
+16
View File
@@ -310,6 +310,8 @@ export interface ListUsersRequest {
user_id?: string;
/** 用户名前缀过滤(可选) */
username?: string;
/** 邮箱前缀过滤(可选) */
email?: string;
}
/**
@@ -348,6 +350,20 @@ export interface CreateUserRequest {
is_admin?: boolean;
}
/**
* 更新用户请求参数
*/
export interface UpdateUserRequest {
/** 昵称 */
nickname?: string;
/** 邮箱 */
email: string;
/** 是否管理员 */
is_admin?: boolean;
/** 密码(可选重置) */
password?: string;
}
/**
* 认证源信息
*/
+6 -1
View File
@@ -4,6 +4,7 @@ import type {
CreateUserRequest,
ListUsersRequest,
ListUsersResponse,
UpdateUserRequest,
UpdateUserStatusRequest,
} from './types';
@@ -29,4 +30,8 @@ export class AdminUserService extends BaseService {
static async deleteUser(id: string): Promise<void> {
return this.delete<void>(`/users/${id}`);
}
}
static async updateUser(id: string, request: UpdateUserRequest): Promise<void> {
return this.put<void>(`/users/${id}`, request);
}
}
+1
View File
@@ -146,6 +146,7 @@ export type {
ListUsersResponse,
UpdateUserStatusRequest,
CreateUserRequest,
UpdateUserRequest,
SystemStatus,
AppUpdateStatus,
Schedule,
+2
View File
@@ -20,6 +20,7 @@
"accent": "oklch(0.967 0.001 286.375)",
"accent-foreground": "oklch(0.21 0.006 285.885)",
"destructive": "oklch(0.577 0.245 27.325)",
"destructive-foreground": "oklch(98.5% 0% 0)",
"border": "oklch(0.92 0.004 286.32)",
"input": "oklch(0.92 0.004 286.32)",
"ring": "oklch(0.705 0.015 286.067)",
@@ -53,6 +54,7 @@
"accent": "oklch(0.274 0.006 286.033)",
"accent-foreground": "oklch(0.985 0 0)",
"destructive": "oklch(0.704 0.191 22.216)",
"destructive-foreground": "oklch(98.5% 0% 0)",
"border": "oklch(1 0 0 / 10%)",
"input": "oklch(1 0 0 / 15%)",
"ring": "oklch(0.552 0.016 285.938)",
+139 -10
View File
@@ -56,23 +56,121 @@ export const searchData: SearchItem[] = [
},
// ==================== 文档库 ====================
{
id: 'docs-api',
title: '开发接口文档',
description: '查看 RESTful API 接口规格定义',
url: '/docs/api',
category: 'page',
keywords: ['api', 'docs', '文档', '接口', 'specification'],
},
{
id: 'docs-how-to-use',
title: '使用帮助文档',
description: '查看新手教程和集成示例',
url: '/docs/how-to-use',
url: 'https://open-flare.pages.dev/',
category: 'page',
keywords: ['docs', '文档', '使用', 'how to', 'tutorial', '教程', 'help'],
},
// ==================== 业务控制台 ====================
{
id: 'console-nodes',
title: '节点管理',
description: '管理边缘节点、中继节点与内网穿透通道',
url: '/nodes',
category: 'page',
keywords: ['node', '节点', '边缘节点', '中继', '内网穿透', 'tunnel', '服务器'],
},
{
id: 'console-proxy-routes',
title: '规则管理',
description: '配置反向代理、路由匹配规则、WAF 策略与缓存设置',
url: '/proxy-routes',
category: 'page',
keywords: ['route', '规则', '路由', '代理', '反向代理', 'proxy'],
},
{
id: 'console-websites',
title: '域名列表',
description: '管理托管域名及证书绑定与监听配置',
url: '/websites',
category: 'page',
keywords: ['website', 'domain', '网站', '域名', '站点'],
},
{
id: 'console-certificates',
title: 'TLS 证书',
description: '申请与管理 SSL/TLS 证书,支持自动续期',
url: '/certificates',
category: 'page',
keywords: ['certificate', 'ssl', 'tls', '证书', 'https', '加密'],
},
{
id: 'console-dns-accounts',
title: 'DNS 账号',
description: '配置 DNS 服务商 API 凭证以自动申请证书及管理解析',
url: '/dns-accounts',
category: 'page',
keywords: ['dns', 'dns account', '账号', '域名解析', 'cloudflare', 'aliyun', 'tencent'],
},
{
id: 'console-origins',
title: '源站地址',
description: '管理反向代理的目标后端服务器与负载均衡组',
url: '/origins',
category: 'page',
keywords: ['origin', '源站', '后端', 'backend', '服务器', '负载均衡'],
},
{
id: 'console-waf',
title: 'WAF 防火墙',
description: '配置 Web 应用防火墙规则,阻断恶意请求',
url: '/waf',
category: 'page',
keywords: ['waf', '防火墙', '安全', 'security', '拦截', '规则'],
},
{
id: 'console-ip-groups',
title: 'IP 组',
description: '定义 IP 地址列表以在 WAF 或路由中实现黑白名单控制',
url: '/ip-groups',
category: 'page',
keywords: ['ip', 'ip group', 'ip组', '黑名单', '白名单', '访问控制'],
},
{
id: 'console-pages',
title: 'Pages 静态托管',
description: '上传或部署静态网页,提供全球 CDN 加速托管',
url: '/pages',
category: 'page',
keywords: ['pages', '静态托管', 'cdn', '网站', '部署', 'static'],
},
{
id: 'console-config-versions',
title: '版本发布',
description: '查看、对比、发布与回滚系统配置版本',
url: '/config-versions',
category: 'page',
keywords: ['version', 'config', '版本', '发布', '回滚', '对比', '部署'],
},
{
id: 'console-access-logs',
title: '访问日志',
description: '查看并检索全量网站访问请求日志与网络分析数据',
url: '/access-logs',
category: 'page',
keywords: ['log', 'logs', '访问日志', '分析', '流量', '请求'],
},
{
id: 'console-apply-logs',
title: '应用记录',
description: '查看节点配置下发、同步与生效的历史记录',
url: '/apply-logs',
category: 'page',
keywords: ['apply', 'log', 'logs', '应用记录', '配置下发', '同步', '部署历史'],
},
{
id: 'console-performance',
title: '性能调优',
description: '调优网络连接、代理超时与核心系统性能参数',
url: '/performance',
category: 'page',
keywords: ['performance', '性能', '调优', '优化', '参数', '连接', '超时'],
},
// ==================== 个人设置 ====================
{
id: 'settings',
@@ -98,7 +196,6 @@ export const searchData: SearchItem[] = [
category: 'setting',
keywords: ['appearance', '外观', '主题', 'theme', 'dark', 'light'],
},
// ==================== 管理员 ====================
{
id: 'admin-settings',
title: '系统设置',
@@ -131,6 +228,38 @@ export const searchData: SearchItem[] = [
category: 'admin',
keywords: ['admin', '管理员', '任务', '异步', 'tasks', 'scheduler', 'worker'],
},
{
id: 'admin-files',
title: '存储管理',
description: '查看、检索与清理上传到对象存储中的文件 (管理员专属)',
url: '/admin/files',
category: 'admin',
keywords: ['admin', '管理员', '存储', '文件', 'files', 'upload', 's3'],
},
{
id: 'admin-database',
title: '数据管理',
description: '监控数据库表大小、分页浏览物理表内容并支持交互式 SQL (管理员专属)',
url: '/admin/database',
category: 'admin',
keywords: ['admin', '管理员', '数据库', 'database', 'sql', 'query', 'gorm'],
},
{
id: 'admin-push',
title: '通知推送',
description: '配置与下发邮件、Lark 和 Telegram 渠道通知推送 (管理员专属)',
url: '/admin/push',
category: 'admin',
keywords: ['admin', '管理员', '推送', '通知', 'push', 'mail', 'telegram', 'lark'],
},
{
id: 'admin-logs',
title: '系统日志',
description: '查看系统日志与后台异步任务执行日志 (管理员专属)',
url: '/admin/logs',
category: 'admin',
keywords: ['admin', '管理员', '日志', 'logs', 'system log', 'terminal'],
},
]
/**
+3
View File
@@ -25,6 +25,7 @@
--color-ring: var(--ring);
--color-input: var(--input);
--color-border: var(--border);
--color-destructive-foreground: var(--destructive-foreground);
--color-destructive: var(--destructive);
--color-accent-foreground: var(--accent-foreground);
--color-accent: var(--accent);
@@ -61,6 +62,7 @@
--accent: oklch(0.967 0.001 286.375);
--accent-foreground: oklch(0.21 0.006 285.885);
--destructive: oklch(0.577 0.245 27.325);
--destructive-foreground: oklch(98.5% 0% 0);
--border: oklch(0.92 0.004 286.32);
--input: oklch(0.92 0.004 286.32);
--ring: oklch(0.705 0.015 286.067);
@@ -95,6 +97,7 @@
--accent: oklch(0.274 0.006 286.033);
--accent-foreground: oklch(0.985 0 0);
--destructive: oklch(0.704 0.191 22.216);
--destructive-foreground: oklch(98.5% 0% 0);
--border: oklch(1 0 0 / 10%);
--input: oklch(1 0 0 / 15%);
--ring: oklch(0.552 0.016 285.938);
+2 -2
View File
@@ -248,7 +248,7 @@ func enrichAccessLogsWithUsers(ctx context.Context, list []accessLogItem) {
// @Router /api/v1/admin/logs/access [get]
func GetAccessLogs(c *gin.Context) {
ctx := c.Request.Context()
if !config.Config.ClickHouse.Enabled || db.ChDB(ctx) == nil {
if !config.Config.ClickHouse.Enabled || !db.ChConnReady() {
response.AbortWithError(c, http.StatusBadRequest, "ClickHouse 存储服务未启用,无法检索访问日志")
return
}
@@ -348,7 +348,7 @@ type logsAnalyticsResponse struct {
// @Router /api/v1/admin/logs/analytics [get]
func GetLogsAnalytics(c *gin.Context) {
ctx := c.Request.Context()
if !config.Config.ClickHouse.Enabled || db.ChDB(ctx) == nil {
if !config.Config.ClickHouse.Enabled || !db.ChConnReady() {
response.AbortWithError(c, http.StatusBadRequest, "ClickHouse 存储服务未启用,无法获取分析数据")
return
}
+53
View File
@@ -0,0 +1,53 @@
// Copyright 2026 Arctel.net
// SPDX-License-Identifier: Apache-2.0
package status
import (
"net/http"
"github.com/Rain-kl/Wavelet/internal/apps/openflare/chwriter"
"github.com/Rain-kl/Wavelet/internal/apps/risk_control"
"github.com/Rain-kl/Wavelet/internal/common/response"
"github.com/Rain-kl/Wavelet/internal/config"
"github.com/Rain-kl/Wavelet/internal/db"
"github.com/Rain-kl/Wavelet/internal/db/batchwriter"
analyticsrepo "github.com/Rain-kl/Wavelet/internal/repository/analytics"
"github.com/gin-gonic/gin"
)
// GetClickHouseStatus returns ClickHouse operational metrics for administrators.
// @Summary 获取 ClickHouse 运行指标
// @Description 返回 ClickHouse parts、mutation、async_insert 队列及进程内 batch writer 指标,需要管理员权限
// @Tags admin
// @Produce json
// @Security SessionCookie
// @Success 200 {object} response.Any{data=analyticsrepo.ClickHouseOperationalStats} "获取成功"
// @Failure 400 {object} response.Any "ClickHouse 未启用"
// @Failure 401 {object} response.Any "未登录"
// @Failure 403 {object} response.Any "无管理员权限"
// @Failure 500 {object} response.Any "内部错误"
// @Router /api/v1/admin/status/clickhouse [get]
func GetClickHouseStatus(c *gin.Context) {
if !config.Config.ClickHouse.Enabled || !db.ChConnReady() {
response.AbortWithError(c, http.StatusBadRequest, "ClickHouse 存储服务未启用")
return
}
stats, err := analyticsrepo.GetClickHouseOperationalStats(c.Request.Context())
if err != nil {
response.AbortInternal(c, "获取 ClickHouse 运行指标失败")
return
}
stats.BatchWriters = collectBatchWriterStats()
c.JSON(http.StatusOK, response.OK(stats))
}
func collectBatchWriterStats() []batchwriter.Stats {
out := chwriter.WriterStats()
if out == nil {
out = make([]batchwriter.Stats, 0, 1)
}
out = append(out, risk_control.LogWriterStats())
return out
}
+2
View File
@@ -144,6 +144,8 @@ func expectedAssetNames(repository, tag string) []string {
extension = "zip"
}
names = append(names, fmt.Sprintf("%s_%s_%s_%s.%s", repoName, tag, runtime.GOOS, runtime.GOARCH, extension))
names = append(names, fmt.Sprintf("%s_%s_%s_%s.%s", strings.ToLower(repoName), tag, runtime.GOOS, runtime.GOARCH, extension))
names = append(names, fmt.Sprintf("%s-server_%s_%s_%s.%s", strings.ToLower(repoName), tag, runtime.GOOS, runtime.GOARCH, extension))
}
}
return names
+14 -12
View File
@@ -6,16 +6,18 @@
package user
const (
userNotFound = "用户不存在"
cannotDisable = "不能禁用管理员用户"
cannotDelete = "不能删除管理员用户"
cannotDeleteSelf = "不能删除当前登录用户"
updateUserFailed = "更新用户状态失败"
deleteUserFailed = "删除用户失败"
usernameExists = "用户名已存在"
usernameRequired = "用户名不能为空"
passwordTooShort = "密码长度不能少于 8 位" //nolint:gosec // false positive: this is an error message, not hardcoded credentials
createUserFailed = "创建用户失败"
emailRequired = "邮箱不能为空"
emailExists = "邮箱已被注册"
userNotFound = "用户不存在"
cannotDisable = "不能禁用管理员用户"
cannotDelete = "不能删除管理员用户"
cannotDeleteSelf = "不能删除当前登录用户"
updateUserFailed = "更新用户状态失败"
deleteUserFailed = "删除用户失败"
usernameExists = "用户名已存在"
usernameRequired = "用户名不能为空"
passwordTooShort = "密码长度不能少于 8 位" //nolint:gosec // false positive: this is an error message, not hardcoded credentials
createUserFailed = "创建用户失败"
emailRequired = "邮箱不能为空"
emailExists = "邮箱已被注册"
cannotRevokeSelfAdmin = "不能撤销当前登录用户的管理员权限"
updateUserInfoFailed = "更新用户信息失败"
)
+78
View File
@@ -20,6 +20,7 @@ func listUsers(ctx context.Context, req listUsersRequest) (int64, []model.User,
return repository.ListAdminUsers(ctx, repository.AdminUserListFilter{
UserID: req.UserID,
Username: strings.TrimSpace(req.Username),
Email: strings.TrimSpace(req.Email),
Page: req.Page,
PageSize: req.PageSize,
})
@@ -132,3 +133,80 @@ func createUser(ctx context.Context, req createUserRequest) (model.User, error)
}
return newUser, nil
}
type updateUserParam struct {
ID uint64
Nickname string
Email string
IsAdmin bool
Password string
}
func updateUser(ctx context.Context, currentUserID uint64, param updateUserParam) error {
param.Nickname = strings.TrimSpace(param.Nickname)
param.Email = strings.TrimSpace(param.Email)
param.Password = strings.TrimSpace(param.Password)
if param.Email == "" {
return errors.New(emailRequired)
}
targetUser, err := repository.GetAdminUserDetail(ctx, param.ID)
if err != nil {
return err
}
// 不能撤销当前登录用户的管理员权限
if currentUserID == param.ID && !param.IsAdmin && targetUser.IsAdmin {
return errors.New(cannotRevokeSelfAdmin)
}
// 如果修改了邮箱,检查邮箱是否被其他用户占用
if targetUser.Email != param.Email {
count, err := repository.CountUsersByEmail(ctx, param.Email)
if err != nil {
return err
}
if count > 0 {
return errors.New(emailExists)
}
}
// 密码强度校验(如果输入了新密码)
if param.Password != "" && len(param.Password) < minPasswordLength {
return errors.New(passwordTooShort)
}
// 是否需要撤销 Token (重置密码或取消管理员)
needRevokeTokens := (param.Password != "") || (targetUser.IsAdmin && !param.IsAdmin)
var tokens []model.AccessToken
if needRevokeTokens {
_ = db.DB(ctx).Where("user_id = ?", param.ID).Find(&tokens).Error
}
// 更新字段
targetUser.Nickname = param.Nickname
if targetUser.Nickname == "" {
targetUser.Nickname = targetUser.Username
}
targetUser.Email = param.Email
targetUser.IsAdmin = param.IsAdmin
if param.Password != "" {
if err := targetUser.SetEncryptedPassword(param.Password); err != nil {
return err
}
}
// 执行更新
err = repository.UpdateUser(ctx, &targetUser)
if err == nil {
oauth.InvalidateCachedUser(ctx, param.ID)
if needRevokeTokens {
for _, token := range tokens {
oauth.InvalidateCachedToken(ctx, token.TokenHash)
}
}
}
return err
}
+57
View File
@@ -28,6 +28,7 @@ type listUsersRequest struct {
PageSize int `form:"page_size" binding:"min=1,max=100"`
UserID *uint64 `form:"user_id" binding:"omitempty,gt=0"`
Username string `form:"username"`
Email string `form:"email"`
}
type user struct {
@@ -289,3 +290,59 @@ func CreateUser(c *gin.Context) {
c.JSON(http.StatusOK, response.OK(toUser(newUser)))
}
// updateUserRequest 更新用户信息请求
type updateUserRequest struct {
Nickname string `json:"nickname" binding:"max=64"`
Email string `json:"email" binding:"required,email,max=255"`
IsAdmin bool `json:"is_admin"`
Password string `json:"password" binding:"omitempty,min=8,max=64"`
}
// UpdateUser 更新用户信息
// @Summary 更新用户信息
// @Description 更新指定用户的昵称、邮箱、管理员权限,并可选重置密码,需要管理员权限
// @Tags admin
// @Accept json
// @Produce json
// @Security SessionCookie
// @Param id path int true "用户 ID"
// @Param request body user.updateUserRequest true "更新参数"
// @Success 200 {object} response.Any{data=string} "更新成功"
// @Failure 400 {object} response.Any "参数错误"
// @Failure 401 {object} response.Any "未登录"
// @Failure 403 {object} response.Any "无管理员权限或尝试修改自身权限"
// @Failure 404 {object} response.Any "用户不存在"
// @Failure 500 {object} response.Any "内部错误"
// @Router /api/v1/admin/users/{id} [put]
func UpdateUser(c *gin.Context) {
var req updateUserRequest
if err := c.ShouldBindJSON(&req); err != nil {
response.AbortBadRequest(c, err.Error())
return
}
id, ok := parseUserID(c)
if !ok {
return
}
currUser, _ := oauth.GetFromContext[*model.User](c, oauth.UserObjKey)
err := updateUser(c.Request.Context(), currUser.ID, updateUserParam{
ID: id,
Nickname: req.Nickname,
Email: req.Email,
IsAdmin: req.IsAdmin,
Password: req.Password,
})
if err != nil {
if abortUserLogicError(c, err, userNotFound, []string{cannotRevokeSelfAdmin}, []string{emailRequired, emailExists, passwordTooShort}) {
return
}
response.AbortInternal(c, updateUserInfoFailed)
return
}
c.JSON(http.StatusOK, response.OKNil())
}
+5 -5
View File
@@ -56,13 +56,13 @@ func TestProtectionEnabledReflectsLoginSwitch(t *testing.T) {
ResetRuntimeSettingsForTest()
if !ProtectionEnabled(ctx) {
t.Fatal("ProtectionEnabled() = false, want true from seed defaults")
if ProtectionEnabled(ctx) {
t.Fatal("ProtectionEnabled() = true, want false from seed defaults")
}
if err := db.DB(ctx).Model(&model.SystemConfig{}).
Where("key = ?", model.ConfigKeyCapLoginEnabled).
Update("value", "false").Error; err != nil {
Update("value", "true").Error; err != nil {
t.Fatalf("Update(cap_login_enabled) error = %v", err)
}
if err := repository.InvalidateSystemConfigCache(ctx, model.ConfigKeyCapLoginEnabled); err != nil {
@@ -70,8 +70,8 @@ func TestProtectionEnabledReflectsLoginSwitch(t *testing.T) {
}
InvalidateRuntimeSettings()
if ProtectionEnabled(ctx) {
t.Fatal("ProtectionEnabled() = true, want false after config update")
if !ProtectionEnabled(ctx) {
t.Fatal("ProtectionEnabled() = false, want true after config update")
}
}
@@ -24,8 +24,6 @@ const (
healthSeverityInfo = "info"
healthSeverityWarning = "warning"
healthSeverityCritical = "critical"
nodeAccessLogRetentionDays = 90
nodeAccessLogRetentionWindow = nodeAccessLogRetentionDays * 24 * time.Hour
accessLogPathMaxLength = 100
healthEventMessageMaxLength = 4096
)
@@ -237,15 +235,11 @@ func buildNodeAccessLogRecords(nodeID string, direct []NodeAccessLog, buffered [
return records, nil
}
func persistNodeAccessLogs(ctx context.Context, nodeID string, records []*model.OpenFlareAccessLog, reportedAt time.Time) error {
func persistNodeAccessLogs(ctx context.Context, _ string, records []*model.OpenFlareAccessLog, _ time.Time) error {
if len(records) == 0 {
return nil
}
if err := model.InsertOpenFlareAccessLogsBatch(ctx, records); err != nil {
return err
}
_, err := model.DeleteOpenFlareAccessLogsByNodeBefore(ctx, nodeID, reportedAt.Add(-nodeAccessLogRetentionWindow))
return err
return model.InsertOpenFlareAccessLogsBatch(ctx, records)
}
func reconcileNodeHealthEvents(tx *gorm.DB, nodeID string, events []NodeHealthEvent, reportedAt time.Time) error {
+25 -11
View File
@@ -25,7 +25,7 @@ func newDedupSet() *dedupSet {
// markIfNew records key when it has not been seen within dedupTTL.
func (s *dedupSet) markIfNew(key string) bool {
if key == "" {
if s == nil || key == "" {
return false
}
@@ -33,19 +33,33 @@ func (s *dedupSet) markIfNew(key string) bool {
s.mu.Lock()
defer s.mu.Unlock()
// Periodically clean up all expired keys (e.g., every 30 seconds)
if now.Sub(s.lastCleanup) >= 30*time.Second {
for existing, expiresAt := range s.keys {
if now.After(expiresAt) {
delete(s.keys, existing)
}
}
s.lastCleanup = now
}
s.cleanupExpiredLocked(now)
if expiresAt, exists := s.keys[key]; exists && now.Before(expiresAt) {
return false
}
s.keys[key] = now.Add(dedupTTL)
return true
}
}
// unmark removes a key so a later enqueue or flush retry may accept it again.
func (s *dedupSet) unmark(key string) {
if s == nil || key == "" {
return
}
s.mu.Lock()
defer s.mu.Unlock()
delete(s.keys, key)
}
func (s *dedupSet) cleanupExpiredLocked(now time.Time) {
if now.Sub(s.lastCleanup) < 30*time.Second {
return
}
for existing, expiresAt := range s.keys {
if now.After(expiresAt) {
delete(s.keys, existing)
}
}
s.lastCleanup = now
}
+164 -2
View File
@@ -3,7 +3,16 @@
package chwriter
import "testing"
import (
"context"
"errors"
"sync"
"testing"
"time"
"github.com/Rain-kl/Wavelet/internal/db/batchwriter"
analyticsmodel "github.com/Rain-kl/Wavelet/internal/model/analytics"
)
func TestDedupSetMarkIfNew(t *testing.T) {
t.Parallel()
@@ -21,4 +30,157 @@ func TestDedupSetMarkIfNew(t *testing.T) {
if set.markIfNew("") {
t.Fatal("markIfNew() = true, want false on empty key")
}
}
}
func TestDedupSetUnmarkAllowsRetry(t *testing.T) {
t.Parallel()
set := newDedupSet()
if !set.markIfNew("k") {
t.Fatal("markIfNew() = false, want true")
}
set.unmark("k")
if !set.markIfNew("k") {
t.Fatal("markIfNew() after unmark = false, want true")
}
}
func TestQueueWithDedupDoesNotMarkWhenEnqueueFails(t *testing.T) {
t.Parallel()
cfg := batchwriter.DefaultConfig()
cfg.QueueSize = 1
cfg.MaxBatchSize = 10
cfg.FlushInterval = time.Hour
// Block the worker so the queue stays full after one enqueue.
block := make(chan struct{})
writer, err := batchwriter.New[int](cfg, func(context.Context, []int) error {
<-block
return nil
})
if err != nil {
t.Fatalf("New() error = %v", err)
}
writer.Start(context.Background())
t.Cleanup(func() {
close(block)
stopCtx, cancel := context.WithTimeout(context.Background(), time.Second)
defer cancel()
_ = writer.Stop(stopCtx)
})
// Fill the channel buffer (and the worker's current receive slot may empty one).
// Keep enqueueing until full so subsequent queueWithDedup fails.
for i := 0; i < cfg.QueueSize+2; i++ {
_ = writer.TryEnqueue(i)
if writer.IsFull() {
break
}
}
if !writer.IsFull() {
t.Fatal("writer not full after filling; cannot test enqueue failure path")
}
dedup := newDedupSet()
queueWithDedup(writer, dedup, "dedup-key", 99)
// Key must not remain marked after failed enqueue.
if !dedup.markIfNew("dedup-key") {
t.Fatal("dedup key still marked after failed enqueue; want unmark")
}
}
func TestQueueWithDedupMarksOnlyOnSuccess(t *testing.T) {
t.Parallel()
cfg := batchwriter.DefaultConfig()
cfg.MaxBatchSize = 100
cfg.FlushInterval = time.Hour
writer, err := batchwriter.New[int](cfg, func(context.Context, []int) error { return nil })
if err != nil {
t.Fatalf("New() error = %v", err)
}
writer.Start(context.Background())
t.Cleanup(func() {
stopCtx, cancel := context.WithTimeout(context.Background(), time.Second)
defer cancel()
_ = writer.Stop(stopCtx)
})
dedup := newDedupSet()
queueWithDedup(writer, dedup, "ok-key", 1)
if dedup.markIfNew("ok-key") {
t.Fatal("markIfNew() = true after successful enqueue, want false (key marked)")
}
}
func TestFlushErrorHandlerUnmarksKeys(t *testing.T) {
t.Parallel()
dedup := newDedupSet()
flushErr := errors.New("ch down")
var (
mu sync.Mutex
errCount int
)
cfg := batchwriter.Config{
Name: "test_obs",
QueueSize: 10,
MaxBatchSize: 1,
FlushInterval: time.Hour,
}
keyFn := func(s analyticsmodel.NodeMetricSnapshot) string {
return metricSnapshotKey(s)
}
writer, err := batchwriter.New(
cfg,
func(context.Context, []analyticsmodel.NodeMetricSnapshot) error { return flushErr },
batchwriter.WithFlushErrorHandler[analyticsmodel.NodeMetricSnapshot](func(_ context.Context, items []analyticsmodel.NodeMetricSnapshot, err error) {
mu.Lock()
errCount++
mu.Unlock()
for _, item := range items {
dedup.unmark(keyFn(item))
}
}),
)
if err != nil {
t.Fatalf("New() error = %v", err)
}
writer.Start(context.Background())
t.Cleanup(func() {
stopCtx, cancel := context.WithTimeout(context.Background(), time.Second)
defer cancel()
_ = writer.Stop(stopCtx)
})
item := analyticsmodel.NodeMetricSnapshot{
NodeID: "n1",
CapturedAt: time.Unix(1, 0).UTC(),
}
key := keyFn(item)
if !dedup.markIfNew(key) {
t.Fatal("markIfNew failed")
}
if !writer.TryEnqueue(item) {
t.Fatal("TryEnqueue failed")
}
deadline := time.Now().Add(time.Second)
for {
mu.Lock()
ready := errCount >= 1
mu.Unlock()
if ready || time.Now().After(deadline) {
break
}
time.Sleep(5 * time.Millisecond)
}
if !dedup.markIfNew(key) {
t.Fatal("key still marked after flush error unmark; want available for retry")
}
}
@@ -0,0 +1,86 @@
//go:build live_ch
// Copyright 2026 Arctel.net
// SPDX-License-Identifier: Apache-2.0
package chwriter_test
import (
"context"
"testing"
"time"
"github.com/Rain-kl/Wavelet/internal/apps/openflare/chwriter"
"github.com/Rain-kl/Wavelet/internal/db"
"github.com/Rain-kl/Wavelet/internal/model"
)
// Run with Docker ClickHouse + config.yaml:
//
// go test -tags live_ch ./internal/apps/openflare/chwriter -run TestLiveAppWritePath -count=1 -timeout 2m
func TestLiveAppWritePath(t *testing.T) {
if !db.ChConnReady() {
t.Skip("ClickHouse connection not ready")
}
ctx := context.Background()
chwriter.Init(ctx)
now := time.Now().UTC()
nodeID := "e2e-app-write-" + now.Format("150405")
if err := model.InsertOpenFlareMetricSnapshot(ctx, &model.OpenFlareMetricSnapshot{
NodeID: nodeID,
CapturedAt: now,
CPUUsagePercent: 33.3,
MemoryUsedBytes: 111,
MemoryTotalBytes: 1000,
StorageUsedBytes: 222,
StorageTotalBytes: 2000,
DiskReadBytes: 10,
DiskWriteBytes: 20,
NetworkRxBytes: 30,
NetworkTxBytes: 40,
}); err != nil {
t.Fatalf("InsertOpenFlareMetricSnapshot: %v", err)
}
deadline := time.Now().Add(45 * time.Second)
var found bool
for time.Now().Before(deadline) {
rows, err := model.ListOpenFlareMetricSnapshotsSince(ctx, nodeID, now.Add(-time.Minute), 10)
if err != nil {
t.Fatalf("ListOpenFlareMetricSnapshotsSince: %v", err)
}
if len(rows) > 0 {
found = true
t.Logf("found snapshot id=%d cpu=%.1f after flush", rows[0].ID, rows[0].CPUUsagePercent)
break
}
time.Sleep(2 * time.Second)
}
if !found {
t.Fatal("metric snapshot not visible in ClickHouse after flush wait")
}
latest, err := model.ListOpenFlareLatestMetricSnapshotsSince(ctx, "", now.Add(-time.Hour))
if err != nil {
t.Fatalf("ListOpenFlareLatestMetricSnapshotsSince: %v", err)
}
var latestOK bool
for _, row := range latest {
if row != nil && row.NodeID == nodeID {
latestOK = true
break
}
}
if !latestOK {
t.Fatalf("latest-per-node query missing node %s (rows=%d)", nodeID, len(latest))
}
stats := chwriter.WriterStats()
if len(stats) == 0 {
t.Fatal("WriterStats empty after Init")
}
for _, s := range stats {
t.Logf("writer %s running=%v depth=%d drops=%d flush_err=%d", s.Name, s.Running, s.Depth, s.Drops, s.FlushErrors)
}
}
+178 -48
View File
@@ -14,19 +14,30 @@ import (
"github.com/Rain-kl/Wavelet/internal/config"
"github.com/Rain-kl/Wavelet/internal/db/batchwriter"
"github.com/Rain-kl/Wavelet/internal/lifecycle"
"github.com/Rain-kl/Wavelet/internal/model"
analyticsmodel "github.com/Rain-kl/Wavelet/internal/model/analytics"
analyticsrepo "github.com/Rain-kl/Wavelet/internal/repository/analytics"
"github.com/Rain-kl/Wavelet/pkg/logger"
)
const (
// Observability traffic is sparse (heartbeat ~10s/node). Prefer larger batches to
// cut ClickHouse parts/merges; MaxFlushWait bounds visibility lag for single-node labs.
observabilityQueueSize = 5_000
observabilityMaxBatchSize = 200
observabilityFlushEvery = 2 * time.Second
observabilityMaxBatchSize = 500
observabilityMinBatchSize = 20
observabilityFlushEvery = 10 * time.Second
observabilityMaxFlushWait = 30 * time.Second
nodeAccessLogQueueSize = 10_000
nodeAccessLogMaxBatchSize = 1_000
nodeAccessLogFlushEvery = time.Second
nodeAccessLogMinBatchSize = 50
nodeAccessLogFlushEvery = 2 * time.Second
nodeAccessLogMaxFlushWait = 5 * time.Second
// flushAttempts is total tries (1 initial + short retries) before giving up a batch.
flushAttempts = 2
flushRetryBackoff = 50 * time.Millisecond
)
var (
@@ -41,6 +52,9 @@ var (
metricSnapshotDedup *dedupSet
requestReportDedup *dedupSet
openrestyDedup *dedupSet
frpsDedup *dedupSet
frpcDedup *dedupSet
)
// Init starts OpenFlare ClickHouse batch writers. Safe to call multiple times.
@@ -52,12 +66,40 @@ func Init(ctx context.Context) {
initOnce.Do(func() {
metricSnapshotDedup = newDedupSet()
requestReportDedup = newDedupSet()
openrestyDedup = newDedupSet()
frpsDedup = newDedupSet()
frpcDedup = newDedupSet()
metricSnapshotWriter = mustNewObservabilityWriter("metric_snapshots", analyticsrepo.BatchInsertNodeMetricSnapshots)
requestReportWriter = mustNewObservabilityWriter("request_reports", analyticsrepo.BatchInsertNodeRequestReports)
openrestyWriter = mustNewObservabilityWriter("openresty_obs", analyticsrepo.BatchInsertNodeObsOpenresty)
frpsWriter = mustNewObservabilityWriter("frps_obs", analyticsrepo.BatchInsertNodeObsFrps)
frpcWriter = mustNewObservabilityWriter("frpc_obs", analyticsrepo.BatchInsertNodeObsFrpc)
metricSnapshotWriter = mustNewObservabilityWriter(
"metric_snapshots",
withFlushRetries(analyticsrepo.BatchInsertNodeMetricSnapshots),
metricSnapshotDedup,
metricSnapshotKey,
)
requestReportWriter = mustNewObservabilityWriter(
"request_reports",
withFlushRetries(analyticsrepo.BatchInsertNodeRequestReports),
requestReportDedup,
requestReportKey,
)
openrestyWriter = mustNewObservabilityWriter(
"openresty_obs",
withFlushRetries(analyticsrepo.BatchInsertNodeObsOpenresty),
openrestyDedup,
openrestyKey,
)
frpsWriter = mustNewObservabilityWriter(
"frps_obs",
withFlushRetries(analyticsrepo.BatchInsertNodeObsFrps),
frpsDedup,
frpsKey,
)
frpcWriter = mustNewObservabilityWriter(
"frpc_obs",
withFlushRetries(analyticsrepo.BatchInsertNodeObsFrpc),
frpcDedup,
frpcKey,
)
nodeAccessLogWriter = mustNewNodeAccessLogWriter()
metricSnapshotWriter.Start(ctx)
@@ -67,6 +109,7 @@ func Init(ctx context.Context) {
frpcWriter.Start(ctx)
nodeAccessLogWriter.Start(ctx)
wireModelInsertHooks()
lifecycle.OnShutdown("openflare_chwriter", Stop)
})
}
@@ -96,57 +139,49 @@ func Stop(ctx context.Context) error {
return firstErr
}
// WriterStats returns queue depth and failure counters for all OpenFlare writers.
func WriterStats() []batchwriter.Stats {
writers := []statsProvider{
metricSnapshotWriter,
requestReportWriter,
openrestyWriter,
frpsWriter,
frpcWriter,
nodeAccessLogWriter,
}
out := make([]batchwriter.Stats, 0, len(writers))
for _, w := range writers {
if w == nil {
continue
}
out = append(out, w.Stats())
}
return out
}
// QueueMetricSnapshot enqueues a metric snapshot for asynchronous flush.
func QueueMetricSnapshot(snapshot analyticsmodel.NodeMetricSnapshot) {
if metricSnapshotWriter == nil {
return
}
key := fmt.Sprintf("%s|%d", snapshot.NodeID, snapshot.CapturedAt.UTC().UnixNano())
if !metricSnapshotDedup.markIfNew(key) {
return
}
metricSnapshotWriter.TryEnqueue(snapshot)
queueWithDedup(metricSnapshotWriter, metricSnapshotDedup, metricSnapshotKey(snapshot), snapshot)
}
// QueueRequestReport enqueues a request report for asynchronous flush.
func QueueRequestReport(report analyticsmodel.NodeRequestReport) {
if requestReportWriter == nil {
return
}
key := fmt.Sprintf(
"%s|%d|%d",
report.NodeID,
report.WindowStartedAt.UTC().UnixNano(),
report.WindowEndedAt.UTC().UnixNano(),
)
if !requestReportDedup.markIfNew(key) {
return
}
requestReportWriter.TryEnqueue(report)
queueWithDedup(requestReportWriter, requestReportDedup, requestReportKey(report), report)
}
// QueueOpenrestyObservation enqueues an OpenResty observation for asynchronous flush.
func QueueOpenrestyObservation(observation analyticsmodel.NodeObsOpenresty) {
if openrestyWriter == nil {
return
}
openrestyWriter.TryEnqueue(observation)
queueWithDedup(openrestyWriter, openrestyDedup, openrestyKey(observation), observation)
}
// QueueFrpsObservation enqueues an FRPS observation for asynchronous flush.
func QueueFrpsObservation(observation analyticsmodel.NodeObsFrps) {
if frpsWriter == nil {
return
}
frpsWriter.TryEnqueue(observation)
queueWithDedup(frpsWriter, frpsDedup, frpsKey(observation), observation)
}
// QueueFrpcObservation enqueues an FRPC observation for asynchronous flush.
func QueueFrpcObservation(observation analyticsmodel.NodeObsFrpc) {
if frpcWriter == nil {
return
}
frpcWriter.TryEnqueue(observation)
queueWithDedup(frpcWriter, frpcDedup, frpcKey(observation), observation)
}
// QueueNodeAccessLogs enqueues node access logs for asynchronous flush.
@@ -159,19 +194,46 @@ func QueueNodeAccessLogs(logs []analyticsmodel.NodeAccessLog) {
}
}
func mustNewObservabilityWriter[T any](name string, flush batchwriter.FlushFunc[T]) *batchwriter.Writer[T] {
func queueWithDedup[T any](writer *batchwriter.Writer[T], dedup *dedupSet, key string, item T) {
if writer == nil {
return
}
// Mark first so concurrent duplicates still collapse; release on enqueue failure
// so a full queue does not permanently suppress the item.
if !dedup.markIfNew(key) {
return
}
if !writer.TryEnqueue(item) {
dedup.unmark(key)
}
}
func mustNewObservabilityWriter[T any](
name string,
flush batchwriter.FlushFunc[T],
dedup *dedupSet,
keyFn func(T) string,
) *batchwriter.Writer[T] {
cfg := batchwriter.Config{
Name: name,
QueueSize: observabilityQueueSize,
MaxBatchSize: observabilityMaxBatchSize,
MinBatchSize: observabilityMinBatchSize,
FlushInterval: observabilityFlushEvery,
MaxFlushWait: observabilityMaxFlushWait,
}
writer, err := batchwriter.New(
cfg,
flush,
withObservabilityDropHandler[T](name),
batchwriter.WithFlushErrorHandler[T](func(ctx context.Context, batchSize int, err error) {
logger.ErrorF(ctx, "[OpenFlare] flush %s failed (batch=%d): %v", name, batchSize, err)
batchwriter.WithFlushErrorHandler[T](func(ctx context.Context, items []T, err error) {
logger.ErrorF(ctx, "[OpenFlare] flush %s failed (batch=%d): %v", name, len(items), err)
if dedup == nil || keyFn == nil {
return
}
for _, item := range items {
dedup.unmark(keyFn(item))
}
}),
)
if err != nil {
@@ -185,14 +247,18 @@ func mustNewNodeAccessLogWriter() *batchwriter.Writer[analyticsmodel.NodeAccessL
Name: "node_access_logs",
QueueSize: nodeAccessLogQueueSize,
MaxBatchSize: nodeAccessLogMaxBatchSize,
MinBatchSize: nodeAccessLogMinBatchSize,
FlushInterval: nodeAccessLogFlushEvery,
MaxFlushWait: nodeAccessLogMaxFlushWait,
}
writer, err := batchwriter.New[analyticsmodel.NodeAccessLog](cfg, analyticsrepo.BatchInsertNodeAccessLogs,
writer, err := batchwriter.New[analyticsmodel.NodeAccessLog](
cfg,
withFlushRetries(analyticsrepo.BatchInsertNodeAccessLogs),
batchwriter.WithDropHandler[analyticsmodel.NodeAccessLog](func(item analyticsmodel.NodeAccessLog) {
logger.WarnF(context.Background(), "[OpenFlare] node access log queue full, dropping log for node %s path %s", item.NodeID, item.Path)
}),
batchwriter.WithFlushErrorHandler[analyticsmodel.NodeAccessLog](func(ctx context.Context, batchSize int, err error) {
logger.ErrorF(ctx, "[OpenFlare] flush node access logs failed (batch=%d): %v", batchSize, err)
batchwriter.WithFlushErrorHandler[analyticsmodel.NodeAccessLog](func(ctx context.Context, items []analyticsmodel.NodeAccessLog, err error) {
logger.ErrorF(ctx, "[OpenFlare] flush node access logs failed (batch=%d): %v", len(items), err)
}),
)
if err != nil {
@@ -207,10 +273,74 @@ func withObservabilityDropHandler[T any](name string) batchwriter.Option[T] {
})
}
// withFlushRetries wraps a flush function with a short retry to ride out brief CH blips.
func withFlushRetries[T any](flush batchwriter.FlushFunc[T]) batchwriter.FlushFunc[T] {
return func(ctx context.Context, items []T) error {
var err error
for attempt := 1; attempt <= flushAttempts; attempt++ {
err = flush(ctx, items)
if err == nil {
return nil
}
if attempt == flushAttempts {
break
}
select {
case <-ctx.Done():
return ctx.Err()
case <-time.After(flushRetryBackoff * time.Duration(attempt)):
}
}
return err
}
}
func wireModelInsertHooks() {
model.SetObservabilityInsertHooks(model.ObservabilityInsertHooks{
QueueMetricSnapshot: QueueMetricSnapshot,
QueueRequestReport: QueueRequestReport,
QueueOpenrestyObservation: QueueOpenrestyObservation,
QueueFrpsObservation: QueueFrpsObservation,
QueueFrpcObservation: QueueFrpcObservation,
})
model.SetAccessLogInsertHooks(model.AccessLogInsertHooks{
QueueNodeAccessLogs: QueueNodeAccessLogs,
})
}
func metricSnapshotKey(snapshot analyticsmodel.NodeMetricSnapshot) string {
return fmt.Sprintf("%s|%d", snapshot.NodeID, snapshot.CapturedAt.UTC().UnixNano())
}
func requestReportKey(report analyticsmodel.NodeRequestReport) string {
return fmt.Sprintf(
"%s|%d|%d",
report.NodeID,
report.WindowStartedAt.UTC().UnixNano(),
report.WindowEndedAt.UTC().UnixNano(),
)
}
func openrestyKey(observation analyticsmodel.NodeObsOpenresty) string {
return fmt.Sprintf("%s|%d", observation.NodeID, observation.CapturedAt.UTC().UnixNano())
}
func frpsKey(observation analyticsmodel.NodeObsFrps) string {
return fmt.Sprintf("%s|%d", observation.NodeID, observation.CapturedAt.UTC().UnixNano())
}
func frpcKey(observation analyticsmodel.NodeObsFrpc) string {
return fmt.Sprintf("%s|%d", observation.NodeID, observation.CapturedAt.UTC().UnixNano())
}
type batchStopper interface {
Stop(ctx context.Context) error
}
type statsProvider interface {
Stats() batchwriter.Stats
}
func running() bool {
return metricSnapshotWriter != nil && metricSnapshotWriter.Running()
}
}
@@ -0,0 +1,33 @@
// Copyright 2026 Arctel.net
// SPDX-License-Identifier: Apache-2.0
package dashboard
import (
"sync"
"time"
)
const overviewCacheTTL = 30 * time.Second
var overviewCache struct {
mu sync.Mutex
payload *OverviewPayload
expiresAt time.Time
}
func getCachedOverview() (*OverviewPayload, bool) {
overviewCache.mu.Lock()
defer overviewCache.mu.Unlock()
if overviewCache.payload == nil || time.Now().After(overviewCache.expiresAt) {
return nil, false
}
return overviewCache.payload, true
}
func setCachedOverview(payload *OverviewPayload) {
overviewCache.mu.Lock()
defer overviewCache.mu.Unlock()
overviewCache.payload = payload
overviewCache.expiresAt = time.Now().Add(overviewCacheTTL)
}
@@ -18,6 +18,7 @@ const (
nodeStatusPending = "pending"
dashboardDistributionLimit = 8
dashboardOverviewSnapshotLimit = 500
highCPUUsagePercentThreshold = 80
highMemoryUsagePercentThreshold = 85
highStorageUsagePercentThreshold = 85
+22 -13
View File
@@ -97,11 +97,16 @@ type trendsPayload struct {
// GetOverview aggregates dashboard overview data from nodes and observability tables.
func GetOverview(ctx context.Context) (*OverviewPayload, error) {
if payload, ok := getCachedOverview(); ok {
return payload, nil
}
view, err := buildOverviewView(ctx)
if err != nil {
return nil, err
}
return compressOverview(view), nil
payload := compressOverview(view)
setCachedOverview(payload)
return payload, nil
}
func buildOverviewView(ctx context.Context) (*OverviewView, error) {
@@ -112,11 +117,21 @@ func buildOverviewView(ctx context.Context) (*OverviewView, error) {
if err != nil {
return nil, err
}
snapshots, err := model.ListOpenFlareMetricSnapshotsSince(ctx, "", since, 0)
// Latest-per-node health: dedicated LIMIT 1 BY queries (not a global raw LIMIT).
latestSnapshotRows, err := model.ListOpenFlareLatestMetricSnapshotsSince(ctx, "", since)
if err != nil {
return nil, err
}
reports, err := model.ListOpenFlareRequestReportsSince(ctx, "", since, 0)
latestTrafficRows, err := model.ListOpenFlareLatestRequestReportsSince(ctx, "", since)
if err != nil {
return nil, err
}
// Bounded raw windows remain for distributions and trend fallbacks; trends prefer hourly rollups.
snapshots, err := model.ListOpenFlareMetricSnapshotsSince(ctx, "", since, dashboardOverviewSnapshotLimit)
if err != nil {
return nil, err
}
reports, err := model.ListOpenFlareRequestReportsSince(ctx, "", since, dashboardOverviewSnapshotLimit)
if err != nil {
return nil, err
}
@@ -128,27 +143,21 @@ func buildOverviewView(ctx context.Context) (*OverviewView, error) {
if err != nil {
return nil, err
}
openrestySnapshots, err := model.ListOpenFlareNodeObservationOpenresty(ctx, "", since, 0)
openrestySnapshots, err := model.ListOpenFlareNodeObservationOpenresty(ctx, "", since, dashboardOverviewSnapshotLimit)
if err != nil {
return nil, err
}
view := &OverviewView{
GeneratedAt: now,
Nodes: make([]NodeHealth, 0, len(nodes)),
Distributions: observability.BuildTrafficDistributions(reports, accessLogRegions, dashboardDistributionLimit),
Trends: observability.NodeTrends{
Traffic24h: observability.BuildTrafficTrendPoints(now, reports),
Capacity24h: observability.BuildCapacityTrendPoints(now, snapshots),
Network24h: observability.BuildNetworkTrendPoints(now, snapshots, openrestySnapshots),
DiskIO24h: observability.BuildDiskIOTrendPoints(now, snapshots),
},
Trends: observability.BuildNodeTrends(ctx, now, "", snapshots, openrestySnapshots, reports),
}
var cpuNodeCount int
var memoryNodeCount int
latestSnapshots := observability.LatestMetricSnapshotsByNode(snapshots)
latestTrafficReports := observability.LatestTrafficReportsByNode(reports)
latestSnapshots := observability.LatestMetricSnapshotsByNode(latestSnapshotRows)
latestTrafficReports := observability.LatestTrafficReportsByNode(latestTrafficRows)
activeEventsByNode := observability.ActiveHealthEventsByNode(activeEvents)
for _, node := range nodes {
@@ -58,6 +58,32 @@ func TestGetOverviewStructure(t *testing.T) {
OpenrestyStatus: "unknown",
}).Error)
// Seed older + newer snapshots per node; health must use latest-per-node, not a global raw limit.
require.NoError(t, model.InsertOpenFlareMetricSnapshot(ctx, &model.OpenFlareMetricSnapshot{
NodeID: "node-dashboard-1",
CapturedAt: now.Add(-2 * time.Hour),
CPUUsagePercent: 10,
MemoryUsedBytes: 1,
MemoryTotalBytes: 10,
}))
require.NoError(t, model.InsertOpenFlareMetricSnapshot(ctx, &model.OpenFlareMetricSnapshot{
NodeID: "node-dashboard-1",
CapturedAt: now.Add(-time.Minute),
CPUUsagePercent: 55,
MemoryUsedBytes: 5,
MemoryTotalBytes: 10,
StorageUsedBytes: 2,
StorageTotalBytes: 10,
}))
require.NoError(t, model.InsertOpenFlareRequestReport(ctx, &model.OpenFlareRequestReport{
NodeID: "node-dashboard-1",
WindowStartedAt: now.Add(-2 * time.Minute),
WindowEndedAt: now.Add(-time.Minute),
RequestCount: 12,
ErrorCount: 1,
UniqueVisitorCount: 4,
}))
overview, err := GetOverview(ctx)
require.NoError(t, err)
require.NotNil(t, overview)
@@ -69,14 +95,14 @@ func TestGetOverviewStructure(t *testing.T) {
assert.Equal(t, 0, overview.Summary.OfflineNodes)
assert.Equal(t, 0, overview.Summary.UnhealthyNodes)
assert.Equal(t, int64(0), overview.Traffic.RequestCount)
assert.Equal(t, int64(0), overview.Traffic.UniqueVisitors)
assert.Equal(t, int64(0), overview.Traffic.ErrorCount)
assert.Equal(t, float64(0), overview.Traffic.EstimatedQPS)
assert.Equal(t, 0, overview.Traffic.ReportedNodes)
assert.Equal(t, int64(12), overview.Traffic.RequestCount)
assert.Equal(t, int64(4), overview.Traffic.UniqueVisitors)
assert.Equal(t, int64(1), overview.Traffic.ErrorCount)
assert.InDelta(t, 0.2, overview.Traffic.EstimatedQPS, 0.0001)
assert.Equal(t, 1, overview.Traffic.ReportedNodes)
assert.Equal(t, float64(0), overview.Capacity.AverageCPUUsagePercent)
assert.Equal(t, float64(0), overview.Capacity.AverageMemoryUsagePercent)
assert.Equal(t, 55.0, overview.Capacity.AverageCPUUsagePercent)
assert.Equal(t, 50.0, overview.Capacity.AverageMemoryUsagePercent)
assert.Equal(t, 0, overview.Capacity.HighCPUNodes)
assert.Equal(t, 0, overview.Capacity.HighMemoryNodes)
assert.Equal(t, 0, overview.Capacity.HighStorageNodes)
@@ -120,10 +146,20 @@ func TestGetOverviewStructure(t *testing.T) {
assert.Equal(t, "Edge 1", onlineNode[2])
assert.Equal(t, "online", onlineNode[6])
assert.Equal(t, "healthy", onlineNode[7])
// Latest-per-node health fields (indexes match compressDashboardNodes).
assert.Equal(t, 55.0, onlineNode[11]) // cpu_usage_percent from latest snapshot
assert.Equal(t, 50.0, onlineNode[12]) // memory_usage_percent
assert.Equal(t, int64(12), onlineNode[14])
assert.Equal(t, int64(1), onlineNode[15])
assert.Equal(t, int64(4), onlineNode[16])
pendingNode := nodeByID["node-dashboard-2"]
require.NotNil(t, pendingNode)
assert.Equal(t, "Edge 2", pendingNode[2])
assert.Equal(t, "pending", pendingNode[6])
assert.Equal(t, "unknown", pendingNode[7])
assert.Equal(t, 55.0, overview.Capacity.AverageCPUUsagePercent)
assert.Equal(t, 1, overview.Traffic.ReportedNodes)
assert.Equal(t, int64(4), overview.Traffic.UniqueVisitors)
}
@@ -21,11 +21,12 @@ const (
defaultIPTrendBucketMinute = 30
maxIPTrendHours = 168
nodeAccessLogRetentionDays = 90
defaultAccessLogQueryDays = 7
accessLogFieldRemoteAddr = "remote_addr"
accessLogFieldRequestCount = "request_count"
)
var nodeAccessLogRetentionWindow = nodeAccessLogRetentionDays * 24 * time.Hour
var defaultAccessLogQueryWindow = defaultAccessLogQueryDays * 24 * time.Hour
// AccessLogQuery filters access log list queries.
type AccessLogQuery struct {
@@ -346,7 +347,7 @@ func ListFoldedAccessLogIPs(ctx context.Context, input FoldedAccessLogIPQuery) (
// ListAccessLogIPSummaries returns paginated IP summaries.
func ListAccessLogIPSummaries(ctx context.Context, input AccessLogIPSummaryQuery) (*AccessLogIPSummaryList, error) {
normalized := normalizeAccessLogIPSummaryQuery(input)
since := time.Now().UTC().Add(-nodeAccessLogRetentionWindow)
since := defaultAccessLogSince()
recentSince := time.Now().UTC().Add(-3 * time.Hour)
query := model.OpenFlareAccessLogIPSummaryQuery{
NodeID: strings.TrimSpace(normalized.NodeID),
@@ -453,7 +454,7 @@ func buildModelAccessLogQuery(input AccessLogQuery) model.OpenFlareAccessLogQuer
RemoteAddr: strings.TrimSpace(input.RemoteAddr),
Host: strings.TrimSpace(input.Host),
Path: strings.TrimSpace(input.Path),
Since: time.Now().UTC().Add(-nodeAccessLogRetentionWindow),
Since: defaultAccessLogSince(),
Page: input.Page,
PageSize: input.PageSize,
SortBy: input.SortBy,
@@ -461,6 +462,10 @@ func buildModelAccessLogQuery(input AccessLogQuery) model.OpenFlareAccessLogQuer
}
}
func defaultAccessLogSince() time.Time {
return time.Now().UTC().Add(-defaultAccessLogQueryWindow)
}
func listNodeNameMap(ctx context.Context, logs []*model.OpenFlareAccessLog) (map[string]string, error) {
nodeIDs := make([]string, 0, len(logs))
seen := make(map[string]struct{}, len(logs))
@@ -4,6 +4,7 @@
package observability
import (
"context"
"encoding/json"
"sort"
"strings"
@@ -13,6 +14,7 @@ import (
)
const observabilityTrendBuckets = 24
const unknownTrendNodeKey = "__unknown__"
const (
healthEventStatusActive = "active"
@@ -133,6 +135,12 @@ type diskCounterState struct {
seen bool
}
type networkCounterState struct {
rx int64
tx int64
seen bool
}
func buildTrafficWindowSummary(report *model.OpenFlareRequestReport) *TrafficWindowSummary {
if report == nil {
return nil
@@ -257,6 +265,66 @@ func buildHealthSummary(
return summary
}
// BuildNodeTrends builds 24h trend series, preferring ClickHouse hourly aggregates
// over limited raw snapshot windows so capacity/network/disk charts stay complete.
func BuildNodeTrends(
ctx context.Context,
now time.Time,
nodeID string,
snapshots []*model.OpenFlareMetricSnapshot,
openrestyObs []*model.OpenFlareNodeObservationOpenresty,
reports []*model.OpenFlareRequestReport,
) NodeTrends {
trendSince := now.Add(-24 * time.Hour)
trafficTrend := BuildTrafficTrendPoints(now, reports)
if trafficHourly, err := model.ListOpenFlareTrafficHourlySince(ctx, nodeID, trendSince); err == nil && len(trafficHourly) > 0 {
trafficTrend = BuildTrafficTrendPointsFromHourly(now, trafficHourly)
}
capacityTrend := BuildCapacityTrendPoints(now, snapshots)
networkTrend := BuildNetworkTrendPoints(now, snapshots, openrestyObs)
diskIOTrend := BuildDiskIOTrendPoints(now, snapshots)
metricHourly, metricErr := model.ListOpenFlareMetricHourlySince(ctx, nodeID, trendSince)
if metricErr == nil && len(metricHourly) > 0 {
capacityTrend = BuildCapacityTrendPointsFromHourly(now, metricHourly)
diskIOTrend = BuildDiskIOTrendPointsFromHourly(now, metricHourly)
}
openrestyHourly, openrestyErr := model.ListOpenFlareOpenrestyHourlySince(ctx, nodeID, trendSince)
if metricErr == nil && openrestyErr == nil && (len(metricHourly) > 0 || len(openrestyHourly) > 0) {
networkTrend = BuildNetworkTrendPointsFromHourly(now, metricHourly, openrestyHourly)
}
return NodeTrends{
Traffic24h: trafficTrend,
Capacity24h: capacityTrend,
Network24h: networkTrend,
DiskIO24h: diskIOTrend,
}
}
// BuildTrafficTrendPointsFromHourly builds 24h traffic trend buckets from hourly rollups.
func BuildTrafficTrendPointsFromHourly(now time.Time, hourly []*model.OpenFlareTrafficHourly) []TrafficTrendPoint {
start := trendWindowStart(now)
points := make([]TrafficTrendPoint, observabilityTrendBuckets)
for index := range points {
points[index].BucketStartedAt = start.Add(time.Duration(index) * time.Hour)
}
for _, row := range hourly {
if row == nil {
continue
}
index, ok := trendBucketIndex(row.Hour, start)
if !ok {
continue
}
points[index].RequestCount += row.RequestCount
points[index].ErrorCount += row.ErrorCount
points[index].UniqueVisitorCount += row.UniqueVisitorCount
}
return points
}
// BuildTrafficTrendPoints builds 24h traffic trend buckets.
func BuildTrafficTrendPoints(now time.Time, reports []*model.OpenFlareRequestReport) []TrafficTrendPoint {
start := trendWindowStart(now)
@@ -314,7 +382,30 @@ func BuildCapacityTrendPoints(now time.Time, snapshots []*model.OpenFlareMetricS
return points
}
// BuildCapacityTrendPointsFromHourly builds 24h capacity trend buckets from hourly aggregates.
func BuildCapacityTrendPointsFromHourly(now time.Time, hourly []*model.OpenFlareMetricHourly) []CapacityTrendPoint {
start := trendWindowStart(now)
points := make([]CapacityTrendPoint, observabilityTrendBuckets)
for index := range points {
points[index].BucketStartedAt = start.Add(time.Duration(index) * time.Hour)
}
for _, row := range hourly {
if row == nil {
continue
}
index, ok := trendBucketIndex(row.Hour, start)
if !ok {
continue
}
points[index].AverageCPUUsagePercent = row.AverageCPUUsagePercent
points[index].AverageMemoryUsagePercent = row.AverageMemoryUsagePercent
points[index].ReportedNodes = row.ReportedNodes
}
return points
}
// BuildNetworkTrendPoints builds 24h network trend buckets.
// Host and OpenResty counters are cumulative; values are consecutive deltas.
func BuildNetworkTrendPoints(
now time.Time,
snapshots []*model.OpenFlareMetricSnapshot,
@@ -327,24 +418,70 @@ func BuildNetworkTrendPoints(
points[index].BucketStartedAt = start.Add(time.Duration(index) * time.Hour)
accumulators[index].nodes = make(map[string]struct{})
}
sort.Slice(snapshots, func(i int, j int) bool {
if snapshots[i].CapturedAt.Equal(snapshots[j].CapturedAt) {
return snapshots[i].NodeID < snapshots[j].NodeID
}
return snapshots[i].CapturedAt.Before(snapshots[j].CapturedAt)
})
previousHostByNode := make(map[string]networkCounterState, len(snapshots))
for _, snapshot := range snapshots {
if snapshot == nil {
continue
}
nodeKey := snapshot.NodeID
if nodeKey == "" {
nodeKey = unknownTrendNodeKey
}
previous := previousHostByNode[nodeKey]
previousHostByNode[nodeKey] = networkCounterState{
rx: snapshot.NetworkRxBytes,
tx: snapshot.NetworkTxBytes,
seen: true,
}
if !previous.seen {
continue
}
index, ok := trendBucketIndex(snapshot.CapturedAt, start)
if !ok {
continue
}
points[index].NetworkRxBytes += snapshot.NetworkRxBytes
points[index].NetworkTxBytes += snapshot.NetworkTxBytes
points[index].NetworkRxBytes += nonNegativeDelta(snapshot.NetworkRxBytes, previous.rx)
points[index].NetworkTxBytes += nonNegativeDelta(snapshot.NetworkTxBytes, previous.tx)
if snapshot.NodeID != "" {
accumulators[index].nodes[snapshot.NodeID] = struct{}{}
}
}
sort.Slice(openrestyObs, func(i int, j int) bool {
if openrestyObs[i].CapturedAt.Equal(openrestyObs[j].CapturedAt) {
return openrestyObs[i].NodeID < openrestyObs[j].NodeID
}
return openrestyObs[i].CapturedAt.Before(openrestyObs[j].CapturedAt)
})
previousOpenrestyByNode := make(map[string]networkCounterState, len(openrestyObs))
for _, obs := range openrestyObs {
if obs == nil {
continue
}
nodeKey := obs.NodeID
if nodeKey == "" {
nodeKey = unknownTrendNodeKey
}
previous := previousOpenrestyByNode[nodeKey]
previousOpenrestyByNode[nodeKey] = networkCounterState{
rx: obs.OpenrestyRxBytes,
tx: obs.OpenrestyTxBytes,
seen: true,
}
if !previous.seen {
continue
}
index, ok := trendBucketIndex(obs.CapturedAt, start)
if !ok {
continue
}
points[index].OpenrestyRxBytes += obs.OpenrestyRxBytes
points[index].OpenrestyTxBytes += obs.OpenrestyTxBytes
points[index].OpenrestyRxBytes += nonNegativeDelta(obs.OpenrestyRxBytes, previous.rx)
points[index].OpenrestyTxBytes += nonNegativeDelta(obs.OpenrestyTxBytes, previous.tx)
if obs.NodeID != "" {
accumulators[index].nodes[obs.NodeID] = struct{}{}
}
@@ -355,6 +492,48 @@ func BuildNetworkTrendPoints(
return points
}
// BuildNetworkTrendPointsFromHourly builds 24h network trend buckets from hourly aggregates.
func BuildNetworkTrendPointsFromHourly(
now time.Time,
metricHourly []*model.OpenFlareMetricHourly,
openrestyHourly []*model.OpenFlareOpenrestyHourly,
) []NetworkTrendPoint {
start := trendWindowStart(now)
points := make([]NetworkTrendPoint, observabilityTrendBuckets)
for index := range points {
points[index].BucketStartedAt = start.Add(time.Duration(index) * time.Hour)
}
for _, row := range metricHourly {
if row == nil {
continue
}
index, ok := trendBucketIndex(row.Hour, start)
if !ok {
continue
}
points[index].NetworkRxBytes += row.NetworkRxBytes
points[index].NetworkTxBytes += row.NetworkTxBytes
if row.ReportedNodes > points[index].ReportedNodes {
points[index].ReportedNodes = row.ReportedNodes
}
}
for _, row := range openrestyHourly {
if row == nil {
continue
}
index, ok := trendBucketIndex(row.Hour, start)
if !ok {
continue
}
points[index].OpenrestyRxBytes += row.OpenrestyRxBytes
points[index].OpenrestyTxBytes += row.OpenrestyTxBytes
if row.ReportedNodes > points[index].ReportedNodes {
points[index].ReportedNodes = row.ReportedNodes
}
}
return points
}
// BuildDiskIOTrendPoints builds 24h disk IO trend buckets.
func BuildDiskIOTrendPoints(now time.Time, snapshots []*model.OpenFlareMetricSnapshot) []DiskIOTrendPoint {
start := trendWindowStart(now)
@@ -374,7 +553,7 @@ func BuildDiskIOTrendPoints(now time.Time, snapshots []*model.OpenFlareMetricSna
for _, snapshot := range snapshots {
nodeKey := snapshot.NodeID
if nodeKey == "" {
nodeKey = "__unknown__"
nodeKey = unknownTrendNodeKey
}
previous := previousByNode[nodeKey]
previousByNode[nodeKey] = diskCounterState{
@@ -389,16 +568,8 @@ func BuildDiskIOTrendPoints(now time.Time, snapshots []*model.OpenFlareMetricSna
if !ok {
continue
}
readDelta := snapshot.DiskReadBytes - previous.read
writeDelta := snapshot.DiskWriteBytes - previous.write
if readDelta < 0 {
readDelta = 0
}
if writeDelta < 0 {
writeDelta = 0
}
points[index].DiskReadBytes += readDelta
points[index].DiskWriteBytes += writeDelta
points[index].DiskReadBytes += nonNegativeDelta(snapshot.DiskReadBytes, previous.read)
points[index].DiskWriteBytes += nonNegativeDelta(snapshot.DiskWriteBytes, previous.write)
if snapshot.NodeID != "" {
accumulators[index].nodes[snapshot.NodeID] = struct{}{}
}
@@ -409,6 +580,36 @@ func BuildDiskIOTrendPoints(now time.Time, snapshots []*model.OpenFlareMetricSna
return points
}
// BuildDiskIOTrendPointsFromHourly builds 24h disk IO trend buckets from hourly aggregates.
func BuildDiskIOTrendPointsFromHourly(now time.Time, hourly []*model.OpenFlareMetricHourly) []DiskIOTrendPoint {
start := trendWindowStart(now)
points := make([]DiskIOTrendPoint, observabilityTrendBuckets)
for index := range points {
points[index].BucketStartedAt = start.Add(time.Duration(index) * time.Hour)
}
for _, row := range hourly {
if row == nil {
continue
}
index, ok := trendBucketIndex(row.Hour, start)
if !ok {
continue
}
points[index].DiskReadBytes += row.DiskReadBytes
points[index].DiskWriteBytes += row.DiskWriteBytes
points[index].ReportedNodes = row.ReportedNodes
}
return points
}
func nonNegativeDelta(current int64, previous int64) int64 {
delta := current - previous
if delta < 0 {
return 0
}
return delta
}
func latestMetricSnapshot(snapshots []*model.OpenFlareMetricSnapshot) *model.OpenFlareMetricSnapshot {
var latest *model.OpenFlareMetricSnapshot
for _, snapshot := range snapshots {
@@ -10,6 +10,23 @@ import (
"github.com/Rain-kl/Wavelet/internal/model"
)
func TestBuildTrafficTrendPointsFromHourlyBucketsByHour(t *testing.T) {
now := time.Date(2026, 7, 2, 15, 30, 0, 0, time.UTC)
hourly := []*model.OpenFlareTrafficHourly{
{
NodeID: "node-a",
Hour: now.Add(-2 * time.Hour).Truncate(time.Hour),
RequestCount: 12,
ErrorCount: 1,
UniqueVisitorCount: 4,
},
}
points := BuildTrafficTrendPointsFromHourly(now, hourly)
if len(points) != observabilityTrendBuckets {
t.Fatalf("BuildTrafficTrendPointsFromHourly() len = %d, want %d", len(points), observabilityTrendBuckets)
}
}
func TestBuildTrafficTrendPointsBucketsByHour(t *testing.T) {
t.Parallel()
@@ -115,3 +132,84 @@ func TestBuildTrafficWindowSummaryNilWithoutReport(t *testing.T) {
t.Fatalf("buildTrafficWindowSummary(nil) = %#v, want nil", summary)
}
}
func TestBuildCapacityTrendPointsFromHourlyFillsBuckets(t *testing.T) {
t.Parallel()
now := time.Date(2026, 7, 10, 9, 30, 0, 0, time.UTC)
hourly := []*model.OpenFlareMetricHourly{
{
Hour: now.Add(-3 * time.Hour).Truncate(time.Hour),
AverageCPUUsagePercent: 42.5,
AverageMemoryUsagePercent: 61.2,
ReportedNodes: 1,
},
{
Hour: now.Truncate(time.Hour),
AverageCPUUsagePercent: 12.0,
AverageMemoryUsagePercent: 50.0,
ReportedNodes: 2,
},
}
points := BuildCapacityTrendPointsFromHourly(now, hourly)
if len(points) != observabilityTrendBuckets {
t.Fatalf("len = %d, want %d", len(points), observabilityTrendBuckets)
}
if points[len(points)-4].AverageCPUUsagePercent != 42.5 {
t.Fatalf("hour-3 cpu = %v, want 42.5", points[len(points)-4].AverageCPUUsagePercent)
}
if points[len(points)-1].ReportedNodes != 2 {
t.Fatalf("current hour reported_nodes = %d, want 2", points[len(points)-1].ReportedNodes)
}
}
func TestBuildNetworkTrendPointsUsesCounterDeltas(t *testing.T) {
t.Parallel()
now := time.Date(2026, 7, 10, 9, 30, 0, 0, time.UTC)
base := now.Truncate(time.Hour)
snapshots := []*model.OpenFlareMetricSnapshot{
{NodeID: "n1", CapturedAt: base.Add(10 * time.Minute), NetworkRxBytes: 1000, NetworkTxBytes: 2000},
{NodeID: "n1", CapturedAt: base.Add(20 * time.Minute), NetworkRxBytes: 1500, NetworkTxBytes: 2600},
}
openrestyObs := []*model.OpenFlareNodeObservationOpenresty{
{NodeID: "n1", CapturedAt: base.Add(10 * time.Minute), OpenrestyRxBytes: 100, OpenrestyTxBytes: 200},
{NodeID: "n1", CapturedAt: base.Add(20 * time.Minute), OpenrestyRxBytes: 180, OpenrestyTxBytes: 250},
}
points := BuildNetworkTrendPoints(now, snapshots, openrestyObs)
current := points[len(points)-1]
if current.NetworkRxBytes != 500 {
t.Fatalf("network_rx_bytes = %d, want 500", current.NetworkRxBytes)
}
if current.NetworkTxBytes != 600 {
t.Fatalf("network_tx_bytes = %d, want 600", current.NetworkTxBytes)
}
if current.OpenrestyRxBytes != 80 {
t.Fatalf("openresty_rx_bytes = %d, want 80", current.OpenrestyRxBytes)
}
if current.OpenrestyTxBytes != 50 {
t.Fatalf("openresty_tx_bytes = %d, want 50", current.OpenrestyTxBytes)
}
}
func TestBuildDiskIOTrendPointsFromHourlyFillsBuckets(t *testing.T) {
t.Parallel()
now := time.Date(2026, 7, 10, 9, 30, 0, 0, time.UTC)
hourly := []*model.OpenFlareMetricHourly{
{
Hour: now.Add(-1 * time.Hour).Truncate(time.Hour),
DiskReadBytes: 1024,
DiskWriteBytes: 2048,
ReportedNodes: 1,
},
}
points := BuildDiskIOTrendPointsFromHourly(now, hourly)
prev := points[len(points)-2]
if prev.DiskReadBytes != 1024 || prev.DiskWriteBytes != 2048 {
t.Fatalf("previous hour disk io = %#v, want read=1024 write=2048", prev)
}
}
@@ -7,6 +7,7 @@ import (
"context"
"encoding/json"
"errors"
"sync"
"time"
"github.com/Rain-kl/Wavelet/internal/model"
@@ -18,8 +19,19 @@ const (
defaultObservabilityLimit = 120
maxObservabilityLimit = 500
defaultTrafficDistributionLimit = 8
nodeObservabilityCacheTTL = 15 * time.Second
)
var nodeObservabilityCache struct {
mu sync.Mutex
views map[string]cachedNodeObservability
}
type cachedNodeObservability struct {
view *NodeView
expiresAt time.Time
}
// NodeQuery filters node observability data.
type NodeQuery struct {
Hours int `json:"hours"`
@@ -87,6 +99,9 @@ func GetNodeObservability(ctx context.Context, id uint, query NodeQuery) (*NodeV
if err != nil {
return nil, err
}
if view, ok := getCachedNodeObservability(node.NodeID); ok {
return view, nil
}
limit := normalizeObservabilityLimit(query.Limit)
since := now.Add(-normalizeObservabilityWindow(query.Hours))
@@ -115,23 +130,10 @@ func GetNodeObservability(ctx context.Context, id uint, query NodeQuery) (*NodeV
if err != nil {
return nil, err
}
trendSnapshots, err := model.ListOpenFlareMetricSnapshotsSince(ctx, node.NodeID, now.Add(-24*time.Hour), 0)
if err != nil {
return nil, err
}
trendOpenresty, err := model.ListOpenFlareNodeObservationOpenresty(ctx, node.NodeID, now.Add(-24*time.Hour), 0)
if err != nil {
return nil, err
}
trendReports, err := model.ListOpenFlareRequestReportsSince(ctx, node.NodeID, now.Add(-24*time.Hour), 0)
if err != nil {
return nil, err
}
events, err := model.ListOpenFlareHealthEvents(ctx, node.NodeID, false, limit)
if err != nil {
return nil, err
}
view := &NodeView{
NodeID: node.NodeID,
Profile: profile,
@@ -143,12 +145,7 @@ func GetNodeObservability(ctx context.Context, id uint, query NodeQuery) (*NodeV
Distributions: BuildTrafficDistributions(reports, accessLogRegions, defaultTrafficDistributionLimit),
Health: buildHealthSummary(latestMetricSnapshot(snapshots), latestTrafficReport(reports), events),
},
Trends: NodeTrends{
Traffic24h: BuildTrafficTrendPoints(now, trendReports),
Capacity24h: BuildCapacityTrendPoints(now, trendSnapshots),
Network24h: BuildNetworkTrendPoints(now, trendSnapshots, trendOpenresty),
DiskIO24h: BuildDiskIOTrendPoints(now, trendSnapshots),
},
Trends: BuildNodeTrends(ctx, now, node.NodeID, snapshots, openrestyObs, reports),
}
if node.NodeType == "tunnel_relay" {
frpsObs, frpsErr := model.ListOpenFlareNodeObservationFrps(ctx, node.NodeID, time.Time{}, 1)
@@ -161,9 +158,35 @@ func GetNodeObservability(ctx context.Context, id uint, query NodeQuery) (*NodeV
}
view.RelayDashboard = buildRelayDashboardSnapshot(node, latestFrps)
}
setCachedNodeObservability(node.NodeID, view)
return view, nil
}
func getCachedNodeObservability(nodeID string) (*NodeView, bool) {
nodeObservabilityCache.mu.Lock()
defer nodeObservabilityCache.mu.Unlock()
if nodeObservabilityCache.views == nil {
return nil, false
}
entry, ok := nodeObservabilityCache.views[nodeID]
if !ok || time.Now().After(entry.expiresAt) {
return nil, false
}
return entry.view, true
}
func setCachedNodeObservability(nodeID string, view *NodeView) {
nodeObservabilityCache.mu.Lock()
defer nodeObservabilityCache.mu.Unlock()
if nodeObservabilityCache.views == nil {
nodeObservabilityCache.views = make(map[string]cachedNodeObservability)
}
nodeObservabilityCache.views[nodeID] = cachedNodeObservability{
view: view,
expiresAt: time.Now().Add(nodeObservabilityCacheTTL),
}
}
// CleanupHealthEvents removes all health events for a node.
func CleanupHealthEvents(ctx context.Context, id uint) (*HealthEventCleanupResult, error) {
node, err := model.GetOpenFlareNodeByID(ctx, id)
+6
View File
@@ -59,6 +59,9 @@ type databaseCleanupResult struct {
Target string `json:"target"`
TargetLabel string `json:"target_label"`
DeletedCount int64 `json:"deleted_count"`
EligibleCount int64 `json:"eligible_count,omitempty"`
CleanupMode string `json:"cleanup_mode,omitempty"`
TableTTLDays int `json:"table_ttl_days,omitempty"`
DeleteAll bool `json:"delete_all"`
RetentionDays *int `json:"retention_days,omitempty"`
}
@@ -198,6 +201,9 @@ func cleanupDatabaseObservability(ctx context.Context, input databaseCleanupInpu
Target: result.Target,
TargetLabel: result.TargetLabel,
DeletedCount: result.DeletedCount,
EligibleCount: result.EligibleCount,
CleanupMode: result.CleanupMode,
TableTTLDays: result.TableTTLDays,
DeleteAll: result.DeleteAll,
RetentionDays: result.RetentionDays,
}, nil
+14 -9
View File
@@ -146,21 +146,26 @@ func TestCleanupDatabaseObservabilityDeletesRows(t *testing.T) {
},
}))
retention := 7
result, err := cleanupDatabaseObservability(ctx, databaseCleanupInput{
// Retention shorter than table TTL (90d for access logs) must be rejected.
shortRetention := 7
_, err := cleanupDatabaseObservability(ctx, databaseCleanupInput{
Target: "node_access_logs",
RetentionDays: &retention,
RetentionDays: &shortRetention,
})
require.Error(t, err)
// Full truncate still hard-deletes all rows.
result, err := cleanupDatabaseObservability(ctx, databaseCleanupInput{
Target: "node_access_logs",
})
require.NoError(t, err)
assert.Equal(t, "node_access_logs", result.Target)
assert.Equal(t, "访问日志", result.TargetLabel)
assert.Equal(t, int64(1), result.DeletedCount)
assert.False(t, result.DeleteAll)
require.NotNil(t, result.RetentionDays)
assert.Equal(t, 7, *result.RetentionDays)
assert.Equal(t, int64(2), result.DeletedCount)
assert.True(t, result.DeleteAll)
assert.Equal(t, "truncate", result.CleanupMode)
rows, err := model.ListOpenFlareAccessLogs(ctx, model.OpenFlareAccessLogQuery{Page: 0, PageSize: 10})
require.NoError(t, err)
require.Len(t, rows, 1)
assert.Equal(t, "/recent", rows[0].Path)
assert.Empty(t, rows)
}
+123 -25
View File
@@ -12,6 +12,7 @@ import (
"github.com/Rain-kl/Wavelet/internal/model"
"github.com/Rain-kl/Wavelet/internal/repository"
analyticsrepo "github.com/Rain-kl/Wavelet/internal/repository/analytics"
)
const (
@@ -21,12 +22,31 @@ const (
DatabaseCleanupTargetMetricSnapshots = "node_metric_snapshots"
// DatabaseCleanupTargetRequestReports is the API cleanup target for request reports.
DatabaseCleanupTargetRequestReports = "node_request_reports"
// DatabaseCleanupTargetObsOpenresty is the API cleanup target for OpenResty observations.
DatabaseCleanupTargetObsOpenresty = "node_obs_openresty"
// DatabaseCleanupTargetObsFrps is the API cleanup target for FRPS observations.
DatabaseCleanupTargetObsFrps = "node_obs_frps"
// DatabaseCleanupTargetObsFrpc is the API cleanup target for FRPC observations.
DatabaseCleanupTargetObsFrpc = "node_obs_frpc"
)
var databaseCleanupTargets = map[string]string{
DatabaseCleanupTargetAccessLogs: "访问日志",
DatabaseCleanupTargetMetricSnapshots: "性能快照",
DatabaseCleanupTargetRequestReports: "请求聚合",
DatabaseCleanupTargetObsOpenresty: "OpenResty 观测",
DatabaseCleanupTargetObsFrps: "FRPS 观测",
DatabaseCleanupTargetObsFrpc: "FRPC 观测",
}
// databaseCleanupTableTTLDays maps API targets to ClickHouse DDL TTL days.
var databaseCleanupTableTTLDays = map[string]int{
DatabaseCleanupTargetAccessLogs: analyticsrepo.TableTTLDaysNodeAccessLogs,
DatabaseCleanupTargetMetricSnapshots: analyticsrepo.TableTTLDaysNodeMetricSnapshots,
DatabaseCleanupTargetRequestReports: analyticsrepo.TableTTLDaysNodeRequestReports,
DatabaseCleanupTargetObsOpenresty: analyticsrepo.TableTTLDaysNodeObs,
DatabaseCleanupTargetObsFrps: analyticsrepo.TableTTLDaysNodeObs,
DatabaseCleanupTargetObsFrpc: analyticsrepo.TableTTLDaysNodeObs,
}
// DatabaseCleanupInput describes a manual observability cleanup request.
@@ -36,13 +56,21 @@ type DatabaseCleanupInput struct {
}
// DatabaseCleanupResult summarizes a manual observability cleanup run.
//
// Semantics:
// - delete_all / cleanup_mode=truncate: DeletedCount is hard-deleted rows (TRUNCATE).
// - retention path / cleanup_mode=ttl_materialize: DeletedCount is always 0;
// EligibleCount estimates rows past the table DDL TTL (not an arbitrary younger cutoff).
type DatabaseCleanupResult struct {
Target string `json:"target"`
TargetLabel string `json:"target_label"`
DeletedCount int64 `json:"deleted_count"`
DeleteAll bool `json:"delete_all"`
RetentionDays *int `json:"retention_days,omitempty"`
Cutoff *time.Time `json:"cutoff,omitempty"`
Target string `json:"target"`
TargetLabel string `json:"target_label"`
DeletedCount int64 `json:"deleted_count"`
EligibleCount int64 `json:"eligible_count,omitempty"`
CleanupMode string `json:"cleanup_mode,omitempty"`
TableTTLDays int `json:"table_ttl_days,omitempty"`
DeleteAll bool `json:"delete_all"`
RetentionDays *int `json:"retention_days,omitempty"`
Cutoff *time.Time `json:"cutoff,omitempty"`
}
// DatabaseAutoCleanupSummary summarizes a scheduled auto-cleanup run.
@@ -52,7 +80,17 @@ type DatabaseAutoCleanupSummary struct {
Results []DatabaseCleanupResult `json:"results"`
}
// TableTTLDaysForCleanupTarget returns the DDL TTL days for a cleanup target.
func TableTTLDaysForCleanupTarget(target string) (int, bool) {
days, ok := databaseCleanupTableTTLDays[strings.TrimSpace(target)]
return days, ok
}
// CleanupDatabaseObservability deletes observability rows for the given target.
//
// When RetentionDays is nil, rows are hard-deleted via TRUNCATE.
// When RetentionDays is set, ClickHouse only force-materializes the table TTL policy:
// retention_days shorter than the table TTL is rejected (do not fake success).
func CleanupDatabaseObservability(ctx context.Context, input DatabaseCleanupInput) (*DatabaseCleanupResult, error) {
target := strings.TrimSpace(input.Target)
targetLabel, ok := databaseCleanupTargets[target]
@@ -63,34 +101,51 @@ func CleanupDatabaseObservability(ctx context.Context, input DatabaseCleanupInpu
return nil, errors.New("retention_days 必须为大于 0 的整数")
}
tableTTLDays := databaseCleanupTableTTLDays[target]
result := &DatabaseCleanupResult{
Target: target,
TargetLabel: targetLabel,
DeleteAll: input.RetentionDays == nil,
Target: target,
TargetLabel: targetLabel,
DeleteAll: input.RetentionDays == nil,
TableTTLDays: tableTTLDays,
}
if input.RetentionDays == nil {
deleted, err := deleteAllObservabilityRows(ctx, target)
deleted, mode, err := deleteAllObservabilityRows(ctx, target)
if err != nil {
return nil, err
}
result.DeletedCount = deleted
result.EligibleCount = deleted
result.CleanupMode = mode
return result, nil
}
retentionDays := *input.RetentionDays
cutoff := time.Now().UTC().Add(-time.Duration(retentionDays) * 24 * time.Hour)
deleted, err := deleteObservabilityRowsBefore(ctx, target, cutoff)
if retentionDays < tableTTLDays {
return nil, fmt.Errorf(
"retention_days 不能小于表 TTL(%d 天);ClickHouse 仅支持按表 TTL 物化过期,更短保留请使用清空全部或调整 DDL",
tableTTLDays,
)
}
// MATERIALIZE TTL only enforces DDL policy; cutoff reported is the table TTL boundary.
tableCutoff := time.Now().UTC().Add(-time.Duration(tableTTLDays) * 24 * time.Hour)
eligible, mode, err := materializeObservabilityTableTTL(ctx, target)
if err != nil {
return nil, err
}
result.DeletedCount = deleted
result.DeletedCount = 0
result.EligibleCount = eligible
result.CleanupMode = mode
result.RetentionDays = &retentionDays
result.Cutoff = &cutoff
result.Cutoff = &tableCutoff
return result, nil
}
// RunDatabaseAutoCleanupOnce runs retention-based cleanup for all observability targets.
//
// Configured retention shorter than a target's table TTL is clamped up to the table TTL
// so the scheduled job can force-materialize each table policy without failing.
func RunDatabaseAutoCleanupOnce(ctx context.Context, now time.Time) (*DatabaseAutoCleanupSummary, error) {
enabled, err := repository.GetBoolByKey(ctx, model.ConfigKeyDatabaseAutoCleanupEnabled)
if err != nil {
@@ -111,10 +166,17 @@ func RunDatabaseAutoCleanupOnce(ctx context.Context, now time.Time) (*DatabaseAu
DatabaseCleanupTargetAccessLogs,
DatabaseCleanupTargetMetricSnapshots,
DatabaseCleanupTargetRequestReports,
DatabaseCleanupTargetObsOpenresty,
DatabaseCleanupTargetObsFrps,
DatabaseCleanupTargetObsFrpc,
} {
effectiveDays := retentionDays
if ttl, ok := databaseCleanupTableTTLDays[target]; ok && effectiveDays < ttl {
effectiveDays = ttl
}
result, err := CleanupDatabaseObservability(ctx, DatabaseCleanupInput{
Target: target,
RetentionDays: &retentionDays,
RetentionDays: &effectiveDays,
})
if err != nil {
return nil, err
@@ -129,28 +191,64 @@ func RunDatabaseAutoCleanupOnce(ctx context.Context, now time.Time) (*DatabaseAu
}, nil
}
func deleteAllObservabilityRows(ctx context.Context, target string) (int64, error) {
func deleteAllObservabilityRows(ctx context.Context, target string) (int64, string, error) {
var (
deleted int64
err error
)
switch target {
case DatabaseCleanupTargetAccessLogs:
return model.DeleteAllOpenFlareAccessLogs(ctx)
deleted, err = model.DeleteAllOpenFlareAccessLogs(ctx)
case DatabaseCleanupTargetMetricSnapshots:
return model.DeleteAllOpenFlareMetricSnapshots(ctx)
deleted, err = model.DeleteAllOpenFlareMetricSnapshots(ctx)
case DatabaseCleanupTargetRequestReports:
return model.DeleteAllOpenFlareRequestReports(ctx)
deleted, err = model.DeleteAllOpenFlareRequestReports(ctx)
case DatabaseCleanupTargetObsOpenresty:
deleted, err = model.DeleteAllOpenFlareNodeObservationOpenresty(ctx)
case DatabaseCleanupTargetObsFrps:
deleted, err = model.DeleteAllOpenFlareNodeObservationFrps(ctx)
case DatabaseCleanupTargetObsFrpc:
deleted, err = model.DeleteAllOpenFlareNodeObservationFrpc(ctx)
default:
return 0, errors.New("unsupported cleanup target")
return 0, "", errors.New("unsupported cleanup target")
}
if err != nil {
return 0, "", err
}
return deleted, analyticsrepo.CleanupModeTruncate, nil
}
func deleteObservabilityRowsBefore(ctx context.Context, target string, cutoff time.Time) (int64, error) {
// materializeObservabilityTableTTL triggers table-TTL materialize (or memory-store delete-before
// with the table TTL cutoff for tests) and returns the eligible/estimate row count.
func materializeObservabilityTableTTL(ctx context.Context, target string) (int64, string, error) {
ttlDays, ok := databaseCleanupTableTTLDays[target]
if !ok {
return 0, "", errors.New("unsupported cleanup target")
}
cutoff := time.Now().UTC().Add(-time.Duration(ttlDays) * 24 * time.Hour)
var (
eligible int64
err error
)
switch target {
case DatabaseCleanupTargetAccessLogs:
return model.DeleteOpenFlareAccessLogsBefore(ctx, cutoff)
eligible, err = model.DeleteOpenFlareAccessLogsBefore(ctx, cutoff)
case DatabaseCleanupTargetMetricSnapshots:
return model.DeleteOpenFlareMetricSnapshotsBefore(ctx, cutoff)
eligible, err = model.DeleteOpenFlareMetricSnapshotsBefore(ctx, cutoff)
case DatabaseCleanupTargetRequestReports:
return model.DeleteOpenFlareRequestReportsBefore(ctx, cutoff)
eligible, err = model.DeleteOpenFlareRequestReportsBefore(ctx, cutoff)
case DatabaseCleanupTargetObsOpenresty:
eligible, err = model.DeleteOpenFlareNodeObservationOpenrestyBefore(ctx, cutoff)
case DatabaseCleanupTargetObsFrps:
eligible, err = model.DeleteOpenFlareNodeObservationFrpsBefore(ctx, cutoff)
case DatabaseCleanupTargetObsFrpc:
eligible, err = model.DeleteOpenFlareNodeObservationFrpcBefore(ctx, cutoff)
default:
return 0, errors.New("unsupported cleanup target")
return 0, "", errors.New("unsupported cleanup target")
}
if err != nil {
return 0, "", err
}
return eligible, analyticsrepo.CleanupModeTTLMaterialize, nil
}
@@ -11,6 +11,7 @@ import (
"github.com/Rain-kl/Wavelet/internal/db"
"github.com/Rain-kl/Wavelet/internal/model"
"github.com/Rain-kl/Wavelet/internal/repository"
analyticsrepo "github.com/Rain-kl/Wavelet/internal/repository/analytics"
"github.com/glebarez/sqlite"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
@@ -36,13 +37,41 @@ func setupDatabaseCleanupTestDB(t *testing.T) context.Context {
return context.Background()
}
func TestCleanupDatabaseObservabilityDeletesTargetedRows(t *testing.T) {
func TestCleanupDatabaseObservabilityRejectsRetentionShorterThanTableTTL(t *testing.T) {
ctx := setupDatabaseCleanupTestDB(t)
retentionDays := 7 // metric snapshots DDL TTL is 30 days
result, err := CleanupDatabaseObservability(ctx, DatabaseCleanupInput{
Target: DatabaseCleanupTargetMetricSnapshots,
RetentionDays: &retentionDays,
})
require.Error(t, err)
assert.Nil(t, result)
assert.Contains(t, err.Error(), "不能小于表 TTL")
assert.Contains(t, err.Error(), "30")
}
func TestCleanupDatabaseObservabilityRejectsAccessLogRetentionShorterThanTableTTL(t *testing.T) {
ctx := setupDatabaseCleanupTestDB(t)
retentionDays := 30 // access logs DDL TTL is 90 days
result, err := CleanupDatabaseObservability(ctx, DatabaseCleanupInput{
Target: DatabaseCleanupTargetAccessLogs,
RetentionDays: &retentionDays,
})
require.Error(t, err)
assert.Nil(t, result)
assert.Contains(t, err.Error(), "90")
}
func TestCleanupDatabaseObservabilityMaterializeDoesNotClaimHardDelete(t *testing.T) {
ctx := setupDatabaseCleanupTestDB(t)
now := time.Now().UTC()
// One row past metric table TTL (30d), one still inside the window.
require.NoError(t, model.InsertOpenFlareMetricSnapshot(ctx, &model.OpenFlareMetricSnapshot{
NodeID: "node-a",
CapturedAt: now.Add(-10 * 24 * time.Hour),
CapturedAt: now.Add(-40 * 24 * time.Hour),
CPUUsagePercent: 10,
}))
require.NoError(t, model.InsertOpenFlareMetricSnapshot(ctx, &model.OpenFlareMetricSnapshot{
@@ -51,15 +80,22 @@ func TestCleanupDatabaseObservabilityDeletesTargetedRows(t *testing.T) {
CPUUsagePercent: 20,
}))
retentionDays := 7
retentionDays := analyticsrepo.TableTTLDaysNodeMetricSnapshots
result, err := CleanupDatabaseObservability(ctx, DatabaseCleanupInput{
Target: DatabaseCleanupTargetMetricSnapshots,
RetentionDays: &retentionDays,
})
require.NoError(t, err)
assert.False(t, result.DeleteAll)
assert.Equal(t, int64(1), result.DeletedCount)
assert.Equal(t, analyticsrepo.CleanupModeTTLMaterialize, result.CleanupMode)
assert.Equal(t, analyticsrepo.TableTTLDaysNodeMetricSnapshots, result.TableTTLDays)
// MATERIALIZE is not a counted hard delete.
assert.Equal(t, int64(0), result.DeletedCount)
assert.Equal(t, int64(1), result.EligibleCount)
require.NotNil(t, result.Cutoff)
assert.True(t, result.Cutoff.Before(now.Add(-29*24*time.Hour)))
// Memory store applies the table-TTL cutoff for tests; only the recent row remains.
rows, err := model.ListOpenFlareMetricSnapshotsSince(ctx, "", time.Time{}, 0)
require.NoError(t, err)
require.Len(t, rows, 1)
@@ -94,20 +130,23 @@ func TestCleanupDatabaseObservabilityDeletesAllRowsWhenRetentionMissing(t *testi
})
require.NoError(t, err)
assert.True(t, result.DeleteAll)
assert.Equal(t, analyticsrepo.CleanupModeTruncate, result.CleanupMode)
assert.Equal(t, int64(2), result.DeletedCount)
assert.Equal(t, int64(2), result.EligibleCount)
rows, err := model.ListOpenFlareAccessLogs(ctx, model.OpenFlareAccessLogQuery{Page: 0, PageSize: 10})
require.NoError(t, err)
assert.Empty(t, rows)
}
func TestRunDatabaseAutoCleanupOnceDeletesAllObservabilityTargets(t *testing.T) {
func TestRunDatabaseAutoCleanupOnceClampsRetentionToTableTTL(t *testing.T) {
ctx := setupDatabaseCleanupTestDB(t)
now := time.Now().UTC()
// Access logs TTL=90d, metrics TTL=30d. Config retention=1 must clamp, not reject.
require.NoError(t, model.InsertOpenFlareAccessLogsBatch(ctx, []*model.OpenFlareAccessLog{{
NodeID: "node-a",
LoggedAt: now.Add(-48 * time.Hour),
LoggedAt: now.Add(-100 * 24 * time.Hour),
RemoteAddr: "203.0.113.10",
Host: "example.com",
Path: "/access",
@@ -115,13 +154,13 @@ func TestRunDatabaseAutoCleanupOnceDeletesAllObservabilityTargets(t *testing.T)
}}))
require.NoError(t, model.InsertOpenFlareMetricSnapshot(ctx, &model.OpenFlareMetricSnapshot{
NodeID: "node-a",
CapturedAt: now.Add(-48 * time.Hour),
CapturedAt: now.Add(-40 * 24 * time.Hour),
CPUUsagePercent: 10,
}))
require.NoError(t, model.InsertOpenFlareRequestReport(ctx, &model.OpenFlareRequestReport{
NodeID: "node-a",
WindowStartedAt: now.Add(-49 * time.Hour),
WindowEndedAt: now.Add(-48 * time.Hour),
WindowStartedAt: now.Add(-41 * 24 * time.Hour),
WindowEndedAt: now.Add(-40 * 24 * time.Hour),
RequestCount: 15,
}))
@@ -131,7 +170,16 @@ func TestRunDatabaseAutoCleanupOnceDeletesAllObservabilityTargets(t *testing.T)
summary, err := RunDatabaseAutoCleanupOnce(ctx, now)
require.NoError(t, err)
require.NotNil(t, summary)
require.Len(t, summary.Results, 3)
require.Len(t, summary.Results, 6)
assert.Equal(t, 1, summary.RetentionDays)
for _, result := range summary.Results {
assert.Equal(t, analyticsrepo.CleanupModeTTLMaterialize, result.CleanupMode)
assert.Equal(t, int64(0), result.DeletedCount, "target %s must not claim hard delete", result.Target)
assert.GreaterOrEqual(t, result.TableTTLDays, 30)
require.NotNil(t, result.RetentionDays)
assert.GreaterOrEqual(t, *result.RetentionDays, result.TableTTLDays)
}
accessLogs, err := model.ListOpenFlareAccessLogs(ctx, model.OpenFlareAccessLogQuery{Page: 0, PageSize: 10})
require.NoError(t, err)
@@ -145,3 +193,16 @@ func TestRunDatabaseAutoCleanupOnceDeletesAllObservabilityTargets(t *testing.T)
require.NoError(t, err)
assert.Empty(t, requestReports)
}
func TestTableTTLDaysForCleanupTarget(t *testing.T) {
days, ok := TableTTLDaysForCleanupTarget(DatabaseCleanupTargetAccessLogs)
require.True(t, ok)
assert.Equal(t, 90, days)
days, ok = TableTTLDaysForCleanupTarget(DatabaseCleanupTargetMetricSnapshots)
require.True(t, ok)
assert.Equal(t, 30, days)
_, ok = TableTTLDaysForCleanupTarget("unknown")
assert.False(t, ok)
}
+18 -30
View File
@@ -234,15 +234,15 @@ func evaluateParsedIPGroupAutoConfig(ctx context.Context, config ipGroupAutoConf
}
programs = append(programs, program)
}
logs, err := model.ListOpenFlareAccessLogsForWAFIPGroup(ctx, model.OpenFlareAccessLogQuery{
aggregates, err := model.ListOpenFlareAccessLogWAFIPAggregates(ctx, model.OpenFlareAccessLogQuery{
Since: now.Add(-time.Duration(config.LookbackMinutes) * time.Minute),
Until: now,
})
if err != nil {
return nil, err
}
accumulators := make(map[string]*ipGroupAutoAccumulator)
for _, item := range logs {
accumulators := make(map[string]*ipGroupAutoAccumulator, len(aggregates))
for _, item := range aggregates {
if item == nil {
continue
}
@@ -250,30 +250,23 @@ func evaluateParsedIPGroupAutoConfig(ctx context.Context, config ipGroupAutoConf
if !ok {
continue
}
acc := accumulators[ip]
if acc == nil {
acc = &ipGroupAutoAccumulator{
ip: ip,
statusCounts: make(map[int]int),
}
accumulators[ip] = acc
lastSeen := time.Time{}
if item.LastSeenEpoch > 0 {
lastSeen = time.Unix(item.LastSeenEpoch, 0).UTC()
}
acc.requestCount++
acc.statusCounts[item.StatusCode]++
if item.StatusCode == http.StatusNotFound {
acc.status404Count++
statusCounts := make(map[int]int, len(item.StatusCounts))
for code, count := range item.StatusCounts {
statusCounts[code] = count
}
if item.StatusCode >= 400 && item.StatusCode < 500 {
acc.clientErrorCount++
}
if item.StatusCode >= http.StatusInternalServerError {
acc.serverErrorCount++
}
if hostIsIPLiteral(item.Host) {
acc.ipHostCount++
}
if item.LoggedAt.After(acc.lastSeen) {
acc.lastSeen = item.LoggedAt
accumulators[ip] = &ipGroupAutoAccumulator{
ip: ip,
requestCount: item.RequestCount,
status404Count: item.Status404Count,
ipHostCount: item.IPHostCount,
clientErrorCount: item.ClientErrorCount,
serverErrorCount: item.ServerErrorCount,
lastSeen: lastSeen,
statusCounts: statusCounts,
}
}
matched := make([]string, 0)
@@ -336,11 +329,6 @@ func normalizeIPLiteral(value string) (string, bool) {
return addr.String(), true
}
func hostIsIPLiteral(value string) bool {
_, ok := normalizeIPLiteral(value)
return ok
}
func downloadIPGroupSubscription(ctx context.Context, rawURL string) ([]byte, error) {
if err := validateSubscriptionURL(rawURL); err != nil {
return nil, err
+21 -3
View File
@@ -6,6 +6,7 @@ package risk_control
import (
"context"
"sync"
"time"
"github.com/Rain-kl/Wavelet/internal/config"
"github.com/Rain-kl/Wavelet/internal/db/batchwriter"
@@ -15,6 +16,11 @@ import (
"github.com/Rain-kl/Wavelet/pkg/logger"
)
const (
// Bound visibility lag for sparse access-log traffic when MinBatchSize is not met.
accessLogMaxFlushWait = 3 * time.Second
)
var (
logWriterMu sync.RWMutex
logWriter *batchwriter.Writer[*analytics.UserAccessLog]
@@ -33,6 +39,8 @@ func InitLogWriter(ctx context.Context) {
}
cfg := batchwriter.DefaultConfig()
cfg.Name = "user_access_logs"
cfg.MaxFlushWait = accessLogMaxFlushWait
writer, err := batchwriter.New[*analytics.UserAccessLog](cfg, func(ctx context.Context, items []*analytics.UserAccessLog) error {
rows := make([]analytics.UserAccessLog, 0, len(items))
for _, item := range items {
@@ -50,8 +58,8 @@ func InitLogWriter(ctx context.Context) {
}
logger.WarnF(context.Background(), "[RiskControl] Log queue full, dropping log item for path: %s", path)
}),
batchwriter.WithFlushErrorHandler[*analytics.UserAccessLog](func(ctx context.Context, batchSize int, err error) {
logger.ErrorF(ctx, "[RiskControl] Send ClickHouse batch failed (batch=%d): %v", batchSize, err)
batchwriter.WithFlushErrorHandler[*analytics.UserAccessLog](func(ctx context.Context, items []*analytics.UserAccessLog, err error) {
logger.ErrorF(ctx, "[RiskControl] Send ClickHouse batch failed (batch=%d): %v", len(items), err)
}),
)
if err != nil {
@@ -82,6 +90,16 @@ func IsBufferFull() bool {
return writer.IsFull()
}
// LogWriterStats returns queue depth and failure counters for the access-log writer.
// When the writer is not initialized, it returns a zero-value Stats with the expected name.
func LogWriterStats() batchwriter.Stats {
writer := currentLogWriter()
if writer == nil {
return batchwriter.Stats{Name: "user_access_logs"}
}
return writer.Stats()
}
// QueueAccessLog enqueues an access log without blocking.
func QueueAccessLog(logItem *analytics.UserAccessLog) {
writer := currentLogWriter()
@@ -108,4 +126,4 @@ func currentLogWriter() *batchwriter.Writer[*analytics.UserAccessLog] {
logWriterMu.RLock()
defer logWriterMu.RUnlock()
return logWriter
}
}
+30
View File
@@ -0,0 +1,30 @@
// Copyright 2026 Arctel.net
// SPDX-License-Identifier: Apache-2.0
package risk_control
import (
"testing"
"time"
)
func TestAccessLogMaxFlushWaitInRange(t *testing.T) {
t.Parallel()
if accessLogMaxFlushWait < 2*time.Second || accessLogMaxFlushWait > 5*time.Second {
t.Fatalf("accessLogMaxFlushWait = %v, want in [2s, 5s]", accessLogMaxFlushWait)
}
}
func TestLogWriterStatsWhenNil(t *testing.T) {
t.Parallel()
reset := SetLogWriterForTest(nil)
t.Cleanup(reset)
stats := LogWriterStats()
if stats.Name != "user_access_logs" {
t.Fatalf("LogWriterStats().Name = %q, want user_access_logs", stats.Name)
}
if stats.Running {
t.Fatal("LogWriterStats().Running = true for nil writer, want false")
}
}
+59 -13
View File
@@ -5,8 +5,11 @@
package risk_control
import (
"crypto/sha256"
"encoding/hex"
"encoding/json"
"net/http"
"strings"
"time"
"github.com/Rain-kl/Wavelet/internal/apps/oauth"
@@ -18,6 +21,61 @@ import (
"github.com/gin-gonic/gin"
)
const maxAuditLogHeadersBytes = 2 * 1024
var auditLogHeaderAllowlist = map[string]struct{}{
"Authorization": {},
"Cookie": {},
"X-Forwarded-For": {},
"X-Real-Ip": {},
"User-Agent": {},
"Content-Type": {},
}
func marshalAuditLogHeaders(headers http.Header) string {
if headers == nil {
return ""
}
filtered := make(http.Header)
for key, values := range headers {
if _, ok := auditLogHeaderAllowlist[key]; !ok {
continue
}
filtered[key] = redactAuditLogHeaderValues(key, values)
}
headersBytes, err := json.Marshal(filtered)
if err != nil {
return ""
}
if len(headersBytes) <= maxAuditLogHeadersBytes {
return string(headersBytes)
}
return string(headersBytes[:maxAuditLogHeadersBytes])
}
func redactAuditLogHeaderValues(key string, values []string) []string {
switch key {
case "Authorization", "Cookie":
redacted := make([]string, len(values))
for i, value := range values {
redacted[i] = hashAuditLogSensitiveValue(value)
}
return redacted
default:
return values
}
}
func hashAuditLogSensitiveValue(value string) string {
if strings.TrimSpace(value) == "" {
return ""
}
sum := sha256.Sum256([]byte(value))
return "sha256:" + hex.EncodeToString(sum[:8])
}
// RiskControlMiddleware 全局日志采集中间件
func RiskControlMiddleware() gin.HandlerFunc {
return func(c *gin.Context) {
@@ -47,19 +105,7 @@ func RiskControlMiddleware() gin.HandlerFunc {
// 4. 计算耗时并异步推送到缓冲队列
latency := time.Since(start).Milliseconds()
var headersStr string
if c.Request.Header != nil {
// 克隆 Header,避免污染原 HTTP 请求的 Header 对象
clonedHeaders := make(http.Header)
for k, v := range c.Request.Header {
clonedHeaders[k] = v
}
clonedHeaders.Del("Cookie")
if headersBytes, err := json.Marshal(clonedHeaders); err == nil {
headersStr = string(headersBytes)
}
}
headersStr := marshalAuditLogHeaders(c.Request.Header)
const maxHTTPStatus = 999
status := c.Writer.Status()
@@ -94,6 +94,8 @@ func TestRiskControlMiddleware(t *testing.T) {
req, _ := http.NewRequest(http.MethodGet, "/test", nil)
req.Header.Set("X-Test-Header", "hello")
req.Header.Set("Cookie", "session_id=abcdef123456")
req.Header.Set("Authorization", "Bearer secret-token")
req.Header.Set("Content-Type", "application/json")
r.ServeHTTP(w, req)
assert.Equal(t, http.StatusOK, w.Code)
@@ -106,8 +108,11 @@ func TestRiskControlMiddleware(t *testing.T) {
assert.Equal(t, http.MethodGet, logItem.Method)
assert.Equal(t, int32(http.StatusOK), logItem.Status)
assert.NotEmpty(t, logItem.Headers)
assert.Contains(t, logItem.Headers, "X-Test-Header")
assert.NotContains(t, logItem.Headers, "Cookie")
assert.NotContains(t, logItem.Headers, "X-Test-Header")
assert.Contains(t, logItem.Headers, "Content-Type")
assert.Contains(t, logItem.Headers, "sha256:")
assert.NotContains(t, logItem.Headers, "secret-token")
assert.NotContains(t, logItem.Headers, "session_id=abcdef123456")
case <-time.After(200 * time.Millisecond):
t.Fatal("expected flushed log item, but got none")
}
+126
View File
@@ -0,0 +1,126 @@
// Copyright 2026 Arctel.net
// SPDX-License-Identifier: Apache-2.0
package cmd
import (
"bufio"
"context"
"crypto/rand"
"errors"
"fmt"
"log"
"os"
"strings"
"github.com/Rain-kl/Wavelet/internal/apps/oauth"
"github.com/Rain-kl/Wavelet/internal/bootstrap"
"github.com/Rain-kl/Wavelet/internal/db"
"github.com/Rain-kl/Wavelet/internal/db/migrator"
"github.com/Rain-kl/Wavelet/internal/model"
"github.com/Rain-kl/Wavelet/internal/repository"
"github.com/spf13/cobra"
"gorm.io/gorm"
)
var (
usernameFlag string
passwordFlag string
)
const (
passwdCharset = "abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789!@#$%^&*"
defaultPasswordLength = 16
)
func generateRandomPassword(length int) (string, error) {
bytes := make([]byte, length)
if _, err := rand.Read(bytes); err != nil {
return "", err
}
for i, b := range bytes {
bytes[i] = passwdCharset[int(b)%len(passwdCharset)]
}
return string(bytes), nil
}
var resetPasswdCmd = &cobra.Command{
Use: "reset-passwd",
Short: "重置指定账号密码",
PreRun: func(_ *cobra.Command, _ []string) {
migrator.Migrate()
},
Run: func(_ *cobra.Command, _ []string) {
ctx := context.Background()
runBootstrap(bootstrap.Options{})
var username string
if usernameFlag != "" {
username = usernameFlag
} else {
fmt.Print("请输入用户名: ")
reader := bufio.NewReader(os.Stdin)
input, err := reader.ReadString('\n')
if err != nil {
log.Fatalf("读取用户名失败: %v\n", err)
}
username = strings.TrimSpace(input)
if username == "" {
log.Fatal("用户名不能为空\n")
}
}
user, err := repository.GetUserByUsername(ctx, username)
if err != nil {
if errors.Is(err, gorm.ErrRecordNotFound) {
log.Fatalf("错误: 用户 '%s' 不存在\n", username)
}
log.Fatalf("查询用户失败: %v\n", err)
}
var password string
if passwordFlag != "" {
password = passwordFlag
} else {
password, err = generateRandomPassword(defaultPasswordLength)
if err != nil {
log.Fatalf("生成随机密码失败: %v\n", err)
}
}
if err := user.SetEncryptedPassword(password); err != nil {
log.Fatalf("加密密码失败: %v\n", err)
}
err = db.DB(ctx).Transaction(func(tx *gorm.DB) error {
if err := tx.Model(&user).Update("password", user.Password).Error; err != nil {
return err
}
// Invalidate existing tokens
var tokens []model.AccessToken
if err := tx.Where("user_id = ?", user.ID).Find(&tokens).Error; err == nil {
for _, token := range tokens {
oauth.InvalidateCachedToken(ctx, token.TokenHash)
}
}
return tx.Where("user_id = ?", user.ID).Delete(&model.AccessToken{}).Error
})
if err != nil {
log.Fatalf("重置密码失败: %v\n", err)
}
oauth.InvalidateCachedUser(ctx, user.ID)
fmt.Println("成功重置密码!")
fmt.Printf("用户名: %s\n", user.Username)
fmt.Printf("新密码: %s\n", password)
},
}
func init() {
resetPasswdCmd.Flags().StringVar(&usernameFlag, "user", "", "重置密码的目标用户名")
resetPasswdCmd.Flags().StringVar(&passwordFlag, "password", "", "新密码(若不指定,则自动生成随机密码)")
rootCmd.AddCommand(resetPasswdCmd)
}
+227
View File
@@ -0,0 +1,227 @@
// Copyright 2026 Arctel.net
// SPDX-License-Identifier: Apache-2.0
package cmd
import (
"bytes"
"io"
"os"
"testing"
"time"
"github.com/Rain-kl/Wavelet/internal/model"
"github.com/Rain-kl/Wavelet/internal/testhelper"
)
func TestResetPasswdCmd_WithUserAndPassword(t *testing.T) {
dbConn, _, cleanup := testhelper.SetupTestEnvironment(t)
defer cleanup()
// Seed test user
user := model.User{
ID: 1001,
Username: "testuser1",
Nickname: "Test User 1",
Email: "test1@example.com",
IsActive: true,
LastLoginAt: time.Now(),
}
_ = user.SetEncryptedPassword("oldpassword")
if err := dbConn.Create(&user).Error; err != nil {
t.Fatalf("failed to create test user: %v", err)
}
// Create access token to test invalidation/deletion
token := model.AccessToken{
ID: 1,
UserID: user.ID,
Name: "testtoken",
TokenHash: "somehash",
MaskedToken: "some...",
}
if err := dbConn.Create(&token).Error; err != nil {
t.Fatalf("failed to create access token: %v", err)
}
// Override PreRun to bypass goose migrations in unit tests
oldPreRun := resetPasswdCmd.PreRun
resetPasswdCmd.PreRun = nil
defer func() { resetPasswdCmd.PreRun = oldPreRun }()
// Execute command with args
rootCmd.SetArgs([]string{"reset-passwd", "--user", "testuser1", "--password", "newpassword123"})
// Capture output
oldStdout := os.Stdout
r, w, _ := os.Pipe()
os.Stdout = w
err := rootCmd.Execute()
w.Close()
os.Stdout = oldStdout
if err != nil {
t.Fatalf("command execute failed: %v", err)
}
var buf bytes.Buffer
_, _ = io.Copy(&buf, r)
output := buf.String()
if !bytes.Contains([]byte(output), []byte("成功重置密码!")) {
t.Errorf("expected output to contain success message, got: %s", output)
}
// Verify password in DB
var dbUser model.User
if err := dbConn.Where("id = ?", user.ID).First(&dbUser).Error; err != nil {
t.Fatalf("failed to query user from DB: %v", err)
}
if !dbUser.CheckPassword("newpassword123") {
t.Errorf("password was not updated correctly in DB")
}
// Verify token deleted
var count int64
dbConn.Model(&model.AccessToken{}).Where("user_id = ?", user.ID).Count(&count)
if count != 0 {
t.Errorf("expected access tokens to be deleted, got %d", count)
}
}
func TestResetPasswdCmd_WithUserAndRandomPassword(t *testing.T) {
dbConn, _, cleanup := testhelper.SetupTestEnvironment(t)
defer cleanup()
// Seed test user
user := model.User{
ID: 1002,
Username: "testuser2",
Nickname: "Test User 2",
Email: "test2@example.com",
IsActive: true,
LastLoginAt: time.Now(),
}
_ = user.SetEncryptedPassword("oldpassword")
if err := dbConn.Create(&user).Error; err != nil {
t.Fatalf("failed to create test user: %v", err)
}
// Override PreRun
oldPreRun := resetPasswdCmd.PreRun
resetPasswdCmd.PreRun = nil
defer func() { resetPasswdCmd.PreRun = oldPreRun }()
// Reset flags
usernameFlag = ""
passwordFlag = ""
// Execute command with args (no --password)
rootCmd.SetArgs([]string{"reset-passwd", "--user", "testuser2"})
// Capture output
oldStdout := os.Stdout
r, w, _ := os.Pipe()
os.Stdout = w
err := rootCmd.Execute()
w.Close()
os.Stdout = oldStdout
if err != nil {
t.Fatalf("command execute failed: %v", err)
}
var buf bytes.Buffer
_, _ = io.Copy(&buf, r)
output := buf.String()
if !bytes.Contains([]byte(output), []byte("成功重置密码!")) {
t.Errorf("expected output to contain success message, got: %s", output)
}
// Verify password in DB (should be updated and not equal to old one)
var dbUser model.User
if err := dbConn.Where("id = ?", user.ID).First(&dbUser).Error; err != nil {
t.Fatalf("failed to query user from DB: %v", err)
}
if dbUser.CheckPassword("oldpassword") {
t.Errorf("expected password to change, but it matches the old one")
}
}
func TestResetPasswdCmd_InteractiveMode(t *testing.T) {
dbConn, _, cleanup := testhelper.SetupTestEnvironment(t)
defer cleanup()
// Seed test user
user := model.User{
ID: 1003,
Username: "testuser3",
Nickname: "Test User 3",
Email: "test3@example.com",
IsActive: true,
LastLoginAt: time.Now(),
}
_ = user.SetEncryptedPassword("oldpassword")
if err := dbConn.Create(&user).Error; err != nil {
t.Fatalf("failed to create test user: %v", err)
}
// Override PreRun
oldPreRun := resetPasswdCmd.PreRun
resetPasswdCmd.PreRun = nil
defer func() { resetPasswdCmd.PreRun = oldPreRun }()
// Mock stdin
inR, inW, err := os.Pipe()
if err != nil {
t.Fatal(err)
}
defer inR.Close()
defer inW.Close()
oldStdin := os.Stdin
os.Stdin = inR
defer func() { os.Stdin = oldStdin }()
// Write username to stdin
_, _ = inW.WriteString("testuser3\n")
inW.Close()
// Reset flags
usernameFlag = ""
passwordFlag = ""
rootCmd.SetArgs([]string{"reset-passwd"})
// Capture output
oldStdout := os.Stdout
r, w, _ := os.Pipe()
os.Stdout = w
err = rootCmd.Execute()
w.Close()
os.Stdout = oldStdout
if err != nil {
t.Fatalf("command execute failed: %v", err)
}
var buf bytes.Buffer
_, _ = io.Copy(&buf, r)
output := buf.String()
if !bytes.Contains([]byte(output), []byte("成功重置密码!")) {
t.Errorf("expected output to contain success message, got: %s", output)
}
// Verify user password changed in DB
var dbUser model.User
if err := dbConn.Where("id = ?", user.ID).First(&dbUser).Error; err != nil {
t.Fatalf("failed to query user from DB: %v", err)
}
if dbUser.CheckPassword("oldpassword") {
t.Errorf("expected password to change, but it matches the old one")
}
}
+14 -17
View File
@@ -47,23 +47,7 @@ var rootCmd = &cobra.Command{
},
Run: func(_ *cobra.Command, args []string) {
// 无参数时默认以融合模式启动所有服务
if len(args) == 0 {
allCmd.Run(allCmd, args)
return
}
appMode := args[0]
switch appMode {
case "api":
apiCmd.Run(apiCmd, args)
case "scheduler":
schedulerCmd.Run(schedulerCmd, args)
case "worker":
workerCmd.Run(workerCmd, args)
case "all":
allCmd.Run(allCmd, args)
default:
log.Fatal("[CMD] unknown app mode\n")
}
allCmd.Run(allCmd, args)
},
}
@@ -76,6 +60,19 @@ func shutdownTraceProvider() {
func init() {
rootCmd.Version = buildinfo.Version
rootCmd.CompletionOptions.DisableDefaultCmd = true
// 1. 为需要迁移的子命令动态绑定原先 rootCmd.PreRun 拥有的数据库迁移行为
migratePreRun := func(_ *cobra.Command, _ []string) {
migrator.Migrate()
migrator.MigrateClickHouse()
}
allCmd.PreRun = migratePreRun
apiCmd.PreRun = migratePreRun
workerCmd.PreRun = migratePreRun
schedulerCmd.PreRun = migratePreRun
// 2. 集中将这些命令注册为真正的子命令,以解决 Cobra 的 unknown command 校验限制
rootCmd.AddCommand(allCmd, apiCmd, workerCmd, schedulerCmd)
}
// Execute 执行根命令
+16 -5
View File
@@ -118,9 +118,17 @@ func applyDefaults(c *configModel) {
}
func applyClickHouseDefaults(c *configModel) {
// Tests disable ClickHouse by default to avoid accidental connections.
// Opt in with CLICKHOUSE_ENABLED=true for live integration tests (e.g. -tags live_ch).
if isTest() {
c.ClickHouse.Enabled = false
return
if v, ok := os.LookupEnv("CLICKHOUSE_ENABLED"); !ok {
c.ClickHouse.Enabled = false
return
} else if b, err := strconv.ParseBool(v); err != nil || !b {
c.ClickHouse.Enabled = false
return
}
// Keep Enabled=true from env and continue applying host/pool defaults.
}
if !c.ClickHouse.Enabled {
c.ClickHouse.Enabled = true
@@ -134,11 +142,14 @@ func applyClickHouseDefaults(c *configModel) {
if c.ClickHouse.Username == "" {
c.ClickHouse.Username = "default"
}
// Pool / buffer defaults target small control-plane hosts (e.g. 3c6g):
// oversized open/idle pools waste RAM and amplify concurrent CH pressure;
// large block buffers add client memory without helping our small batch inserts.
if c.ClickHouse.MaxIdleConn <= 0 {
c.ClickHouse.MaxIdleConn = 10
c.ClickHouse.MaxIdleConn = 8
}
if c.ClickHouse.MaxOpenConn <= 0 {
c.ClickHouse.MaxOpenConn = 100
c.ClickHouse.MaxOpenConn = 16
}
if c.ClickHouse.ConnMaxLifetime <= 0 {
c.ClickHouse.ConnMaxLifetime = 3600
@@ -147,7 +158,7 @@ func applyClickHouseDefaults(c *configModel) {
c.ClickHouse.DialTimeout = 5
}
if c.ClickHouse.BlockBufferSize == 0 {
c.ClickHouse.BlockBufferSize = 10
c.ClickHouse.BlockBufferSize = 32
}
}
+7 -6
View File
@@ -71,18 +71,19 @@ type databaseReplicaConfig struct {
Password string `mapstructure:"password"`
}
// clickhouse 配置
// clickHouseConfig ClickHouse 原生客户端配置。
// 连接池 / block_buffer 默认值按小型控制面主机(如 3c6g)收敛,见 applyClickHouseDefaults。
type clickHouseConfig struct {
Enabled bool `mapstructure:"enabled"`
Hosts []string `mapstructure:"hosts"`
Username string `mapstructure:"username"`
Password string `mapstructure:"password"`
Database string `mapstructure:"database"`
MaxIdleConn int `mapstructure:"max_idle_conn"`
MaxOpenConn int `mapstructure:"max_open_conn"`
ConnMaxLifetime int `mapstructure:"conn_max_lifetime"`
DialTimeout int `mapstructure:"dial_timeout"`
BlockBufferSize uint8 `mapstructure:"block_buffer_size"`
MaxIdleConn int `mapstructure:"max_idle_conn"` // 默认 8
MaxOpenConn int `mapstructure:"max_open_conn"` // 默认 16
ConnMaxLifetime int `mapstructure:"conn_max_lifetime"` // 秒
DialTimeout int `mapstructure:"dial_timeout"` // 秒
BlockBufferSize uint8 `mapstructure:"block_buffer_size"` // 默认 32
}
// redisConfig Redis配置
+21 -4
View File
@@ -9,9 +9,10 @@ import (
)
const (
defaultQueueSize = 10_000
defaultMaxBatchSize = 1_000
defaultFlushEvery = time.Second
defaultQueueSize = 10_000
defaultMaxBatchSize = 1_000
defaultMinBatchSize = 50
defaultFlushEvery = time.Second
)
// Config controls queue capacity and flush thresholds for a Writer instance.
@@ -25,8 +26,17 @@ type Config struct {
// MaxBatchSize triggers a flush when the in-memory batch reaches this count.
MaxBatchSize int
// FlushInterval triggers a time-based flush even when the batch is smaller.
// MinBatchSize is the minimum in-memory batch size for time-based flushes.
// Zero disables the threshold and preserves legacy interval flush behavior.
// When set, interval flushes below this size are skipped unless MaxFlushWait elapses.
MinBatchSize int
// FlushInterval is how often the worker checks whether a time-based flush should run.
FlushInterval time.Duration
// MaxFlushWait forces a flush of any non-empty batch once the oldest item has waited
// this long, even if MinBatchSize has not been reached. Zero disables the force path.
MaxFlushWait time.Duration
}
// DefaultConfig returns production-friendly defaults aligned with audit log batching.
@@ -34,6 +44,7 @@ func DefaultConfig() Config {
return Config{
QueueSize: defaultQueueSize,
MaxBatchSize: defaultMaxBatchSize,
MinBatchSize: defaultMinBatchSize,
FlushInterval: defaultFlushEvery,
}
}
@@ -45,8 +56,14 @@ func (c Config) validate() error {
if c.MaxBatchSize <= 0 {
return fmt.Errorf("batchwriter: max batch size must be positive")
}
if c.MinBatchSize < 0 {
return fmt.Errorf("batchwriter: min batch size must be non-negative")
}
if c.FlushInterval <= 0 {
return fmt.Errorf("batchwriter: flush interval must be positive")
}
if c.MaxFlushWait < 0 {
return fmt.Errorf("batchwriter: max flush wait must be non-negative")
}
return nil
}
+57 -8
View File
@@ -9,22 +9,34 @@ package batchwriter
import (
"context"
"sync"
"sync/atomic"
"time"
)
// FlushFunc persists a batch of queued items. It is invoked from the worker goroutine.
type FlushFunc[T any] func(ctx context.Context, items []T) error
// FlushErrorHandler is called when FlushFunc returns an error. The batch is discarded
// after the handler returns; the worker continues processing.
type FlushErrorHandler func(ctx context.Context, batchSize int, err error)
// FlushErrorHandler is called when FlushFunc returns an error after optional retries.
// The batch is discarded after the handler returns; the worker continues processing.
// Handlers receive the failed items so callers can release dedup keys or re-queue.
type FlushErrorHandler[T any] func(ctx context.Context, items []T, err error)
// Stats is a point-in-time snapshot of Writer queue and failure counters.
type Stats struct {
Name string `json:"name"`
Depth int `json:"depth"`
Cap int `json:"cap"`
Drops int64 `json:"drops"`
FlushErrors int64 `json:"flush_errors"`
Running bool `json:"running"`
}
// Writer buffers items and flushes them by size or interval.
type Writer[T any] struct {
cfg Config
flush FlushFunc[T]
onFlushError FlushErrorHandler
onFlushError FlushErrorHandler[T]
onDrop func(T)
startOnce sync.Once
@@ -34,13 +46,16 @@ type Writer[T any] struct {
ch chan T
workerCtx context.Context
done chan struct{}
drops atomic.Int64
flushErrors atomic.Int64
}
// Option configures optional Writer callbacks.
type Option[T any] func(*Writer[T])
// WithFlushErrorHandler registers a callback for flush failures.
func WithFlushErrorHandler[T any](handler FlushErrorHandler) Option[T] {
func WithFlushErrorHandler[T any](handler FlushErrorHandler[T]) Option[T] {
return func(w *Writer[T]) {
w.onFlushError = handler
}
@@ -168,22 +183,37 @@ func (w *Writer[T]) Cap() int {
return w.cfg.QueueSize
}
// Stats returns a point-in-time snapshot of queue depth and failure counters.
func (w *Writer[T]) Stats() Stats {
return Stats{
Name: w.cfg.Name,
Depth: w.Len(),
Cap: w.Cap(),
Drops: w.drops.Load(),
FlushErrors: w.flushErrors.Load(),
Running: w.Running(),
}
}
func (w *Writer[T]) run() {
ticker := time.NewTicker(w.cfg.FlushInterval)
defer ticker.Stop()
batch := make([]T, 0, w.cfg.MaxBatchSize)
var batchStartedAt time.Time
flush := func() {
if len(batch) == 0 {
return
}
items := append([]T(nil), batch...)
if err := w.flush(w.workerCtx, items); err != nil {
w.flushErrors.Add(1)
if w.onFlushError != nil {
w.onFlushError(w.workerCtx, len(items), err)
w.onFlushError(w.workerCtx, items, err)
}
}
batch = batch[:0]
batchStartedAt = time.Time{}
}
defer func() {
@@ -197,19 +227,38 @@ func (w *Writer[T]) run() {
if !ok {
return
}
if len(batch) == 0 {
batchStartedAt = time.Now()
}
batch = append(batch, item)
if len(batch) >= w.cfg.MaxBatchSize {
flush()
}
case <-ticker.C:
flush()
if w.shouldFlushOnInterval(len(batch), batchStartedAt, time.Now()) {
flush()
}
}
}
}
func (w *Writer[T]) shouldFlushOnInterval(batchLen int, batchStartedAt time.Time, now time.Time) bool {
if batchLen == 0 {
return false
}
if w.cfg.MinBatchSize == 0 || batchLen >= w.cfg.MinBatchSize {
return true
}
if w.cfg.MaxFlushWait <= 0 || batchStartedAt.IsZero() {
return false
}
return !now.Before(batchStartedAt.Add(w.cfg.MaxFlushWait))
}
func (w *Writer[T]) notifyDrop(item T) {
w.drops.Add(1)
if w.onDrop == nil {
return
}
w.onDrop(item)
}
}
+221 -10
View File
@@ -36,7 +36,7 @@ func TestWriterFlushesOnMaxBatchSize(t *testing.T) {
t.Parallel()
var (
mu sync.Mutex
mu sync.Mutex
batches [][]int
)
cfg := DefaultConfig()
@@ -98,6 +98,7 @@ func TestWriterFlushesOnInterval(t *testing.T) {
)
cfg := DefaultConfig()
cfg.MaxBatchSize = 100
cfg.MinBatchSize = 0
cfg.FlushInterval = 20 * time.Millisecond
writer, err := New[int](cfg, func(_ context.Context, items []int) error {
@@ -219,27 +220,189 @@ func TestWriterStopDrainsQueuedItems(t *testing.T) {
}
}
func TestWriterSkipsIntervalFlushBelowMinBatchSize(t *testing.T) {
t.Parallel()
var (
mu sync.Mutex
batch []int
)
cfg := DefaultConfig()
cfg.MaxBatchSize = 100
cfg.MinBatchSize = 5
cfg.FlushInterval = 20 * time.Millisecond
writer, err := New[int](cfg, func(_ context.Context, items []int) error {
mu.Lock()
defer mu.Unlock()
batch = append([]int(nil), items...)
return nil
})
if err != nil {
t.Fatalf("New() error = %v", err)
}
writer.Start(context.Background())
t.Cleanup(func() {
stopCtx, cancel := context.WithTimeout(context.Background(), time.Second)
defer cancel()
if err := writer.Stop(stopCtx); err != nil {
t.Fatalf("Stop() error = %v", err)
}
})
for i := range 3 {
if !writer.TryEnqueue(i + 1) {
t.Fatalf("TryEnqueue(%d) = false, want true", i+1)
}
}
time.Sleep(100 * time.Millisecond)
mu.Lock()
got := batch
mu.Unlock()
if len(got) != 0 {
t.Fatalf("interval flush with below-min batch = %v, want no flush", got)
}
}
func TestWriterFlushesOnIntervalWhenMinBatchSizeReached(t *testing.T) {
t.Parallel()
var (
mu sync.Mutex
batch []int
)
cfg := DefaultConfig()
cfg.MaxBatchSize = 100
cfg.MinBatchSize = 3
cfg.FlushInterval = 20 * time.Millisecond
writer, err := New[int](cfg, func(_ context.Context, items []int) error {
mu.Lock()
defer mu.Unlock()
batch = append([]int(nil), items...)
return nil
})
if err != nil {
t.Fatalf("New() error = %v", err)
}
writer.Start(context.Background())
t.Cleanup(func() {
stopCtx, cancel := context.WithTimeout(context.Background(), time.Second)
defer cancel()
if err := writer.Stop(stopCtx); err != nil {
t.Fatalf("Stop() error = %v", err)
}
})
for i := range 3 {
if !writer.TryEnqueue(i + 1) {
t.Fatalf("TryEnqueue(%d) = false, want true", i+1)
}
}
deadline := time.Now().Add(time.Second)
for {
mu.Lock()
ready := len(batch) == 3
mu.Unlock()
if ready || time.Now().After(deadline) {
break
}
time.Sleep(5 * time.Millisecond)
}
mu.Lock()
got := batch
mu.Unlock()
want := []int{1, 2, 3}
if diff := cmp.Diff(want, got); diff != "" {
t.Fatalf("interval flush at min batch size mismatch (-want +got):\n%s", diff)
}
}
func TestWriterForcesFlushAfterMaxFlushWait(t *testing.T) {
t.Parallel()
var (
mu sync.Mutex
batch []int
)
cfg := DefaultConfig()
cfg.MaxBatchSize = 100
cfg.MinBatchSize = 50
cfg.FlushInterval = 20 * time.Millisecond
cfg.MaxFlushWait = 80 * time.Millisecond
writer, err := New[int](cfg, func(_ context.Context, items []int) error {
mu.Lock()
defer mu.Unlock()
batch = append([]int(nil), items...)
return nil
})
if err != nil {
t.Fatalf("New() error = %v", err)
}
writer.Start(context.Background())
t.Cleanup(func() {
stopCtx, cancel := context.WithTimeout(context.Background(), time.Second)
defer cancel()
if err := writer.Stop(stopCtx); err != nil {
t.Fatalf("Stop() error = %v", err)
}
})
if !writer.TryEnqueue(1) {
t.Fatal("TryEnqueue(1) = false, want true")
}
deadline := time.Now().Add(time.Second)
for {
mu.Lock()
ready := len(batch) == 1
mu.Unlock()
if ready || time.Now().After(deadline) {
break
}
time.Sleep(5 * time.Millisecond)
}
mu.Lock()
got := batch
mu.Unlock()
if diff := cmp.Diff([]int{1}, got); diff != "" {
t.Fatalf("max flush wait mismatch (-want +got):\n%s", diff)
}
}
func TestWriterInvokesFlushErrorHandler(t *testing.T) {
t.Parallel()
cfg := DefaultConfig()
cfg.Name = "test-flush-err"
cfg.MaxBatchSize = 1
cfg.FlushInterval = time.Hour
flushErr := errors.New("flush failed")
var (
mu sync.Mutex
errCount int
batchSize int
mu sync.Mutex
errCount int
gotItems []int
)
writer, err := New[int](cfg, func(context.Context, []int) error {
return flushErr
}, WithFlushErrorHandler[int](func(_ context.Context, size int, err error) {
}, WithFlushErrorHandler[int](func(_ context.Context, items []int, err error) {
mu.Lock()
defer mu.Unlock()
errCount++
batchSize = size
gotItems = append([]int(nil), items...)
if !errors.Is(err, flushErr) {
t.Errorf("flush error = %v, want %v", err, flushErr)
}
@@ -272,13 +435,61 @@ func TestWriterInvokesFlushErrorHandler(t *testing.T) {
mu.Lock()
gotCount := errCount
gotSize := batchSize
items := gotItems
mu.Unlock()
if gotCount != 1 {
t.Fatalf("flush error handler count = %d, want 1", gotCount)
}
if gotSize != 1 {
t.Fatalf("flush error handler batch size = %d, want 1", gotSize)
if diff := cmp.Diff([]int{7}, items); diff != "" {
t.Fatalf("flush error handler items mismatch (-want +got):\n%s", diff)
}
}
stats := writer.Stats()
if stats.FlushErrors != 1 {
t.Fatalf("Stats().FlushErrors = %d, want 1", stats.FlushErrors)
}
if stats.Name != "test-flush-err" {
t.Fatalf("Stats().Name = %q, want test-flush-err", stats.Name)
}
}
func TestWriterStatsTracksDrops(t *testing.T) {
t.Parallel()
cfg := DefaultConfig()
cfg.Name = "test-drops"
cfg.QueueSize = 1
cfg.MaxBatchSize = 10
cfg.FlushInterval = time.Hour
writer, err := New[int](cfg, func(context.Context, []int) error { return nil })
if err != nil {
t.Fatalf("New() error = %v", err)
}
writer.Start(context.Background())
t.Cleanup(func() {
stopCtx, cancel := context.WithTimeout(context.Background(), time.Second)
defer cancel()
_ = writer.Stop(stopCtx)
})
if !writer.TryEnqueue(1) {
t.Fatal("TryEnqueue(1) = false, want true")
}
if writer.TryEnqueue(2) {
t.Fatal("TryEnqueue(2) = true, want false")
}
stats := writer.Stats()
if stats.Drops != 1 {
t.Fatalf("Stats().Drops = %d, want 1", stats.Drops)
}
if stats.Cap != 1 {
t.Fatalf("Stats().Cap = %d, want 1", stats.Cap)
}
if !stats.Running {
t.Fatal("Stats().Running = false, want true")
}
}
+21 -74
View File
@@ -7,32 +7,32 @@ package db
import (
"context"
"fmt"
"log"
"net/url"
"strconv"
"strings"
"time"
"github.com/ClickHouse/clickhouse-go/v2"
"github.com/ClickHouse/clickhouse-go/v2/lib/driver"
"github.com/Rain-kl/Wavelet/internal/config"
"go.opentelemetry.io/otel/attribute"
clickhouseDriver "gorm.io/driver/clickhouse"
"gorm.io/gorm"
"gorm.io/plugin/opentelemetry/tracing"
)
const (
clickhouseMaxExecTime = 60 // ClickHouse 最大执行时间(秒)
clickhouseReadTimeoutFactor = 2 // ReadTimeout 为 DialTimeout 的倍数
// async_insert 仅挂在运行时 ChConn(写路径)上,不进入 migrator OpenDB:
// 迁移/DDL 需要同步可见结果,且不应走异步 insert 缓冲。
//
// 为何启用:batchwriter 仍可能在短间隔内写出相对小的块;服务端 async_insert
// 把多次 INSERT 合并成更大 part,减轻 3c6g 上 background merge 的 CPU 压力。
// wait_for_async_insert=1:调用方在 flush 返回前等待落盘,避免进程崩溃丢批。
// max_data_size / busy_timeout:约 10MB 或 ~2s 触发刷出,在延迟与 part 数之间折中。
clickhouseAsyncInsertMaxDataSize = 10_000_000
clickhouseAsyncInsertBusyTimeoutMs = 2000
)
var (
// ChConn ClickHouse 原生连接实例,用于批量写入
// ChConn ClickHouse 原生连接实例,用于批量写入与查询
ChConn driver.Conn
chDB *gorm.DB
)
func init() {
@@ -57,39 +57,11 @@ func init() {
log.Fatalf("[ClickHouse] ping failed: %v\n", err)
}
chDB, err = gorm.Open(clickhouseDriver.New(clickhouseDriver.Config{
DSN: buildClickHouseDSN(),
}), &gorm.Config{
SkipDefaultTransaction: true,
})
if err != nil {
log.Fatalf("[ClickHouse] init gorm connection failed: %v\n", err)
}
if err = chDB.Use(
tracing.NewPlugin(
tracing.WithoutMetrics(),
tracing.WithAttributes(
attribute.String("db.instance", cfg.Database),
attribute.String("db.system", "ClickHouse"),
),
),
); err != nil {
log.Fatalf("[ClickHouse] init trace failed: %v\n", err)
}
sqlDB, err := chDB.DB()
if err != nil {
log.Fatalf("[ClickHouse] load sql db failed: %v\n", err)
}
sqlDB.SetMaxIdleConns(cfg.MaxIdleConn)
sqlDB.SetMaxOpenConns(cfg.MaxOpenConn)
sqlDB.SetConnMaxLifetime(time.Duration(cfg.ConnMaxLifetime) * time.Second)
log.Println("[ClickHouse] connection established successfully")
}
// buildClickHouseOptions builds the runtime native client options (queries + batch inserts).
// Migrator uses a separate clickhouse.OpenDB path without async_insert settings.
func buildClickHouseOptions() *clickhouse.Options {
cfg := config.Config.ClickHouse
@@ -101,7 +73,11 @@ func buildClickHouseOptions() *clickhouse.Options {
Password: cfg.Password,
},
Settings: clickhouse.Settings{
"max_execution_time": clickhouseMaxExecTime,
"max_execution_time": clickhouseMaxExecTime,
"async_insert": 1,
"wait_for_async_insert": 1,
"async_insert_max_data_size": clickhouseAsyncInsertMaxDataSize,
"async_insert_busy_timeout_ms": clickhouseAsyncInsertBusyTimeoutMs,
},
Compression: &clickhouse.Compression{
Method: clickhouse.CompressionLZ4,
@@ -115,38 +91,9 @@ func buildClickHouseOptions() *clickhouse.Options {
}
}
func buildClickHouseDSN() string {
cfg := config.Config.ClickHouse
chURL := &url.URL{
Scheme: "clickhouse",
Host: strings.Join(cfg.Hosts, ","),
Path: "/" + cfg.Database,
}
if cfg.Username != "" || cfg.Password != "" {
chURL.User = url.UserPassword(cfg.Username, cfg.Password)
}
query := chURL.Query()
query.Set("dial_timeout", fmt.Sprintf("%ds", cfg.DialTimeout))
query.Set("read_timeout", fmt.Sprintf("%ds", cfg.DialTimeout*clickhouseReadTimeoutFactor))
query.Set("max_execution_time", strconv.Itoa(clickhouseMaxExecTime))
chURL.RawQuery = query.Encode()
return chURL.String()
}
// ChDB returns a context-aware GORM ClickHouse instance.
func ChDB(ctx context.Context) *gorm.DB {
if chDB == nil {
return nil
}
return chDB.WithContext(ctx)
}
// SetChDBForTest sets the package-level ClickHouse GORM instance for testing.
func SetChDBForTest(d *gorm.DB) {
chDB = d
// ChConnReady reports whether the native ClickHouse connection is initialized.
func ChConnReady() bool {
return ChConn != nil
}
// SetChConnForTest sets the package-level native ClickHouse connection for testing.

Some files were not shown because too many files have changed in this diff Show More