Commit Graph

333 Commits

Author SHA1 Message Date
ryan e2ac531abb docs(zone): define zone domain management model 2026-07-12 14:05:04 +08:00
ryan c8b1289043 fix(docker): preserve clickhouse listener config 2026-07-12 12:28:34 +08:00
ryan 13c5073bf8 chore(release): v3.1.2
### 🛠 修复
- 修复了节点与仪表盘 24 小时容量、网络、磁盘 IO 趋势在限流查询下几乎为空的问题,改为小时级聚合与计数器增量统计,历史时段可正常展示。
- 修复了静置场景下 ClickHouse CPU 偏高的问题,可观测与访问日志写入改为批量凑批并限制小 part 产生。
- 修复了数据清理接口将「物化表 TTL」误报为已删除行数的问题,短于表 TTL 的保留天数会被明确拒绝。
- 修复了可观测去重在入队/刷盘失败后仍占用键、导致故障窗口数据更易丢失的问题,并在 flush 失败时短重试与释放键。
- 修复了 Dashboard「每节点最新指标」被全局 LIMIT 截断导致安静节点缺失的问题,改为按节点取最新快照。
- 修复了 Docker 部署 ClickHouse 25.x 因后台池与 mutation 空闲阈值不兼容而无法启动的问题。
- 修复了小时预聚合仅有迁移后少量数据时 24 小时趋势再次残缺的问题:读路径按小时 merge(窗口完整走 rollup,缺口用 raw 补齐),并增加历史 backfill 迁移。

### ⚡️ 优化与改进
- 为容量与 OpenResty 指标增加小时预聚合表,窗口完整时优先走 rollup 降低查询压力。
- 审计日志写入增加最长等待刷盘,管理端可观测状态接口暴露 batch writer 队列深度、丢弃与 flush 错误指标。
- 小规格场景下调 ClickHouse 客户端连接池默认值,并调整 async_insert 合并超时与流量小时表 TTL。
- 流量独立访客在小时汇总中改为窗口峰值估计,并修正界面文案,避免被误解为全局真实 UV。

### 💄 其他/体验
- 同步环境变量与配置模板中的 ClickHouse 说明;部署文档改为将 performance.xml 下载到 ./config/clickhouse 后挂载,且不挂载 listen 配置。
2026-07-10 11:42:49 +08:00
ryan da1dd92404 fix(observability): merge rollup+raw hourly trends and backfill history
Prefer capacity/openresty rollups only when they cover the 24h window;
otherwise merge per hour so raw fills pre-MV gaps and rollup wins on
overlap. Add a one-time ANTI JOIN backfill migration for the last 30 days.
2026-07-10 11:27:51 +08:00
ryan 4b11279662 fix(observability): fall back to raw hourly when rollup is incomplete
Materialized capacity/openresty hourly tables only hold data after the MV
exists. Preferring any non-empty rollup hid full raw history and left 24h
charts with only recent hours. Use rollup only when its earliest bucket
covers the query window start.
2026-07-10 11:27:51 +08:00
ryan bbadcca294 ### 🛠 修复
- 修复了节点与仪表盘 24 小时容量、网络、磁盘 IO 趋势在限流查询下几乎为空的问题,改为小时级聚合与计数器增量统计,历史时段可正常展示。
- 修复了静置场景下 ClickHouse CPU 偏高的问题,可观测与访问日志写入改为批量凑批并限制小 part 产生。
- 修复了数据清理接口将「物化表 TTL」误报为已删除行数的问题,短于表 TTL 的保留天数会被明确拒绝。
- 修复了可观测去重在入队/刷盘失败后仍占用键、导致故障窗口数据更易丢失的问题,并在 flush 失败时短重试与释放键。
- 修复了 Dashboard「每节点最新指标」被全局 LIMIT 截断导致安静节点缺失的问题,改为按节点取最新快照。
- 修复了 Docker 部署 ClickHouse 25.x 因后台池与 mutation 空闲阈值不兼容而无法启动的问题。

### ⚡️ 优化与改进
- 为容量与 OpenResty 指标增加小时预聚合表,读路径优先 rollup,降低 24 小时趋势查询压力。
- 审计日志写入增加最长等待刷盘,管理端可观测状态接口暴露 batch writer 队列深度、丢弃与 flush 错误指标。
- 小规格场景下调 ClickHouse 客户端连接池默认值,并调整 async_insert 合并超时与流量小时表 TTL。
- 流量独立访客在小时汇总中改为窗口峰值估计,并修正界面文案,避免被误解为全局真实 UV。

### 💄 其他/体验
- 同步环境变量与配置模板中的 ClickHouse 说明;部署文档改为将 performance.xml 下载到 ./config/clickhouse 后挂载,且不挂载 listen 配置。
2026-07-10 11:27:51 +08:00
ryan 44ce6497a1 docs(docker): use ./config/clickhouse for CH performance mount
Move performance.xml to config/clickhouse and mount the directory to
/etc/clickhouse-server/config.d; update docs and compose paths.
2026-07-10 11:10:32 +08:00
ryan 4b83f91b31 docs(docker): use ./config/clickhouse for CH performance mount
Move performance.xml to config/clickhouse and mount the directory to
/etc/clickhouse-server/config.d; update docs and compose paths.
2026-07-10 10:59:21 +08:00
ryan 9d2fac5d4c docs(config): sync .env.example and config.example.yaml for CH defaults
Align ClickHouse pool/password placeholders and docker-compose env docs with
runtime defaults and published ports used in local Docker testing.
2026-07-10 10:48:30 +08:00
ryan b4b93ff4ed fix(docker): make ClickHouse startable on 25.x and reachable from host
Lower merge-tree free-entry thresholds for small background pools, bind
listen_host to 0.0.0.0 for published ports, and allow CLICKHOUSE_ENABLED=true
in tests for live_ch smoke coverage.
2026-07-10 10:44:49 +08:00
ryan 160e63558f fix(clickhouse): harden R/W path P0–P3 (cleanup, durability, rollups)
Honest TTL cleanup semantics; enqueue-safe dedup with flush retry and writer
metrics; model insert hooks; latest-per-node and hourly metric/openresty
rollups; small-host pool/async defaults, traffic hourly TTL, and UV labeling.
2026-07-10 10:34:04 +08:00
ryan 9b3555c569 fix(clickhouse): cut idle CPU from tiny parts and oversized merge pools
Observability writers flushed every few seconds with MinBatchSize unset,
creating constant small parts and merge load. Enable MinBatchSize with
MaxFlushWait, batch access logs more aggressively, and shrink ClickHouse
background pools for 3c hosts.
2026-07-10 10:08:32 +08:00
ryan b928928958 fix(observability): restore 24h capacity/network/disk trends via CH hourly agg
Node and dashboard 24h capacity, network, and disk IO charts only used the
latest limited raw snapshots (120/500 rows), so historical hour buckets stayed
empty. Prefer ClickHouse hourly aggregates with counter deltas, and fall back
to raw snapshots when aggregation is unavailable.
2026-07-10 09:48:45 +08:00
ryan b312460ddf chore(release): v3.1.1
### ⚡️ 优化与改进
- 将 cap_login_enabled 默认值由 true 变更为 false,默认关闭登录界面 PoW 人机验证。
2026-07-06 12:29:53 +08:00
ryan 336185f01c release: v3.1.0
### 🛠 修复
- 修复 ClickHouse TTL 迁移中 DateTime64 时间列无法直接设置 TTL 导致 goose 启动失败的问题,改为通过 toDateTime() 转换后再应用 TTL。
- 修复 ClickHouse 迁移尝试缩短 ORDER BY 排序键时与隐式主键前缀冲突导致迁移失败的问题,移除不支持的 MODIFY ORDER BY 操作。
- 修复系统设置页面 URL tab 参数未包含 openflare-ops 选项卡导致无法正确定位的问题,并在无参数时默认选中 OpenFlare 选项卡。
- 修复系统自更新检测上游 GitHub Release 时,因资产包名称前缀 openflare-server 与仓库名不完全一致导致匹配失败并报错「未找到兼容的 Release」的问题。
- 修复全局搜索数据源覆盖不全的问题,补全所有核心业务控制台页面及管理员专有页面的检索支持。

### ⚡️ 优化与改进
- ClickHouse 启用 async_insert 异步写入缓冲,并调高 block_buffer_size 与连接池默认值,降低小 part 生成与连接争用。
- 优化 ClickHouse 写入路径:移除 Agent 心跳中的同步 ALTER DELETE 保留清理,batchwriter 新增 MinBatchSize 抑制过小批次定时 flush,可观测 writer 批次与 flush 间隔调优并补全去重。
- ClickHouse 分析表新增 TTL 自动过期策略,访问日志 180 天、节点访问日志 90 天、其余观测与聚合表 30 天自动清理。
- 访问日志与 WAF IP 组查询改为 ClickHouse 侧聚合与 SQL 分页,默认限制近 7 天查询窗口,浏览器分布查询增加 Top 100 限制。
- Dashboard 与节点可观测 API 消除无 LIMIT 全表扫描,增加短 TTL 内存缓存,前端轮询间隔分别调整为 60s/30s。
- ClickHouse 遗留治理 Phase 2:保留期清理改为 TTL MATERIALIZE TTL,统一 ChConn 读路径,新增 /admin/status/clickhouse 运维指标与 of_node_traffic_hourly 预聚合 MV。
- 审计访问日志写入时仅保留安全相关请求头并以 SHA-256 脱敏,将 headers 载荷上限收紧至 2KB,减小行宽与 merge CPU 开销。
- Docker 部署为 ClickHouse 增加 nofile ulimits 与性能配置挂载,限制 max_concurrent_queries 与后台合并争用。
- 数据库自动清理任务新增 OpenResty、FRPS、FRPC 观测表清理目标。

### 💄 其他/体验
- 隐藏侧边栏文档库中的「规范示例」与「接口文档」,将「使用文档」及其他相关页面链接统一跳转至外部文档站 https://open-flare.pages.dev/。
- 移除系统设置 OpenFlare 标签页下的版本信息卡片及对应升级管理弹窗逻辑。
- 系统设置页面支持通过 URL 持久化当前选中的 Tab 状态。
2026-07-04 09:43:31 +08:00
ryan 44bba0f19a fix(clickhouse): remove unsupported MODIFY ORDER BY from migration
ClickHouse keeps the implicit PRIMARY KEY when shortening ORDER BY,
which fails with "Primary key must be a prefix of the sorting key".
TTL-only changes are safe and unblock goose startup; narrowing ORDER BY
would require table recreation.
2026-07-02 16:45:48 +08:00
ryan f0eca028f9 fix(clickhouse): cast DateTime64 to DateTime in TTL migration 2026-07-02 16:21:30 +08:00
ryan 58624db397 perf(clickhouse): Phase 2 legacy governance — TTL cleanup, unified pool, MV, ops API
- Replace retention ALTER DELETE with MATERIALIZE TTL; use TRUNCATE for delete-all
- Remove GORM ClickHouse pool; migrate user access log reads to ChConn
- Drop query-side trim(remote_addr); enable wait_for_async_insert=1
- Add of_node_traffic_hourly MV and dashboard traffic trend fallback
- Add GET /admin/status/clickhouse operational metrics endpoint
2026-07-02 15:47:38 +08:00
ryan 38946d1af5 fix(clickhouse): resolve lint issues from optimization stack 2026-07-02 15:28:49 +08:00
ryan caf2ffcff4 perf(clickhouse): P1 TTL migrations, ORDER BY tune, remote_addr normalization 2026-07-02 15:25:03 +08:00
ryan 0e86fe3547 perf(clickhouse): P2 docker server tuning and audit log payload reduction 2026-07-02 15:23:17 +08:00
ryan 5ad151469c feat(frontend): delete unused version-upgrade-dialog component and use-openflare-server-upgrade hook
- Permanently delete version-upgrade-dialog.tsx and use-openflare-server-upgrade.ts as they are no longer referenced after removing the version info card from openflare-ops settings.
2026-06-30 21:04:52 +08:00
ryan 6467b32d8e fix(frontend): include openflare-ops tab in whitelist and make it default
- Include 'openflare-ops' in the list of validTabs so that specifying ?tab=openflare-ops correctly loads the OpenFlare settings tab.
- Set fallback tab default to 'openflare-ops' when no tab parameter is specified.
- Document changes in changelog.
2026-06-30 20:56:34 +08:00
ryan 34225cb88a fix(updater): resolve release asset name matching for openflare-server
- Update expectedAssetNames helper to match lowercase repoName prefix and lowercase repoName with -server suffix (e.g. openflare-server).
- Fixes 'no compatible release found' error when checking GitHub Action releases.
2026-06-30 20:47:32 +08:00
ryan 389f02b6b0 feat(frontend): update navigation links and expand search coverage
- Hide 'Specification Examples' and 'API Docs' from sidebar documents group, pointing 'Use Docs' externally to pages.dev.
- Complete searchData array to cover all console business pages and missing admin-only pages.
- Add changelog records for these adjustments.
2026-06-30 20:36:55 +08:00
ryan 2fcbb945fb docs: update 2026-06-30 16:52:17 +08:00
ryan 23501259b2 docs: update 2026-06-30 16:42:41 +08:00
ryan c561e65cd3 docs: update 2026-06-30 16:38:53 +08:00
ryan 97095e8f12 release: v3.0.2
### 🛠 修复
- 修复 PostgreSQL 自增主键序列在历史数据迁移(INSERT 指定显式 ID)后与实际数据不同步的问题,通过新增全局序列同步脚本一键重置所有相关表的自增计数器。
2026-06-30 16:17:21 +08:00
ryan 23be2f9296 fix(db): 新增 PostgreSQL 数据库自增序列全局同步迁移脚本
为了解决因历史数据以显式 ID 方式迁移导致 PostgreSQL 自增序列计数器不同步,产生主键冲突唯一性约束报错(如 WAF 规则组和 IP 组保存失败)的问题,在 PostgreSQL 迁移中加入了对所有相关表 pg_get_serial_sequence 重置的代码。同时,在 SQLite 中补齐了对应的同名迁移文件。
2026-06-30 16:13:37 +08:00
ryan 113ea25aa4 release: v3.0.1
### ⚡️ 优化与改进
- 新增管理后台用户个人信息编辑与重置密码功能。
- 新增用户列表邮箱列展示以及基于邮箱的搜索过滤。
- 后端新增 `reset-passwd` 命令行工具,支持通过命令行直接重置用户密码。

### 🛠 修复
- 修复添加 DNS 账号时,因直接传递类静态方法作为 React Query 的 mutationFn 导致 JavaScript 运行时丢失 `this` 上下文报错 `this.post is not a function` 的问题。
- 修复侧边栏一级菜单项当前页面字体颜色被硬编码为 `#6366F1` 的问题,改用 CSS 主题变量 `text-sidebar-primary`,以保证多主题色彩一致。
- 修复默认主题(Default)遗漏声明 `destructive-foreground` 变量,导致删除确认按钮在某些状态下渲染为黑底黑字而无法阅读的问题。
- 修复 Cobra 命令行初始化注册逻辑,确保所有应用运行模式(All, API, Worker, Scheduler)都正确注册为 Cobra 子命令。
- 优化系统设置页面的色彩定义,移除硬编码的 Indigo 靛蓝色以适配多主题切换。

### 💄 其他/体验
- 在 `AGENTS.md` 规范中新增关于防止服务类静态方法 callback 上下文丢失的开发规范指南。
2026-06-30 11:58:09 +08:00
ryan c230d5a744 fix(frontend): 解决添加DNS账号时this.post is not a function报错
在 DnsAccountCreateDialog 组件中,mutationFn 错误地直接传递了类静态方法 DnsAccountService.create,导致执行时丢失 class constructor 上下文。现将其修改为使用箭头函数包裹,以保证 this 指向正确。
2026-06-30 11:52:24 +08:00
ryan 2816152536 fix(frontend): adjust sidebar active text color and fix destructive button contrast
- Update sidebar active menu button text color to use theme dynamic `sidebar-primary` variable instead of hardcoded hex value.

- Add missing `destructive-foreground` variables to default theme config and styles, resolving the black-on-black text contrast issue on confirmation dialog delete buttons.

- Update changelog to track these fixes.
2026-06-28 10:34:20 +08:00
ryan b029714c7a docs: update 2026-06-28 10:11:51 +08:00
ryan d721a8fd74 docs: update 2026-06-28 10:08:31 +08:00
ryan 1e349e5cde docs: update default credentials to admin and 12345678 in documents 2026-06-28 10:03:45 +08:00
ryan f03c88ffad docs: update 2026-06-27 16:29:58 +08:00
ryan 3038304382 docs(design): update product boundaries and configuration references
- Update docs/design/index.md to document system constraints, including mandatory Redis/ClickHouse dependencies and dynamic Relay Web UI settings.
- Update docs/reference/configuration.md to reflect w_system_configs table keys, option groups, and new environment variable mappings.
- Clean up old plan files.
2026-06-27 16:03:16 +08:00
ryan 196bdabc80 docs(docs): update server start and quick-start docs to require PG/SQLite, Redis, and ClickHouse
- Restructure docs/deployment/server.md into Docker (Quick Start, Production Recommended, Advanced with Jaeger) and Local deployment.
- Update docs/guide/quick-start.md default docker-compose to use PostgreSQL, Redis, and ClickHouse as default.
2026-06-27 15:57:55 +08:00
ryan 77931c3c1e docs(docs): add google analytics tracking tag to head config 2026-06-27 15:41:08 +08:00
ryan a97d87acf5 fix(frontend): fix hasConfigDiff function missing WAF and site changes
Add check for waf_config_changed, added_sites, removed_sites, and modified_sites in hasConfigDiff helper to prevent disabling the publish button on WAF/site edits.
2026-06-27 15:40:16 +08:00
ryan 8d8814b416 refactor(oauth): replace legacy oauth cache with standard ram cache and add pubsub synchronization
- Replaced custom map-based cache in apps/oauth/cache.go with standard pkg/cache/ram framework.
- Implemented Redis Pub/Sub invalidation channels for distributed token and user cache synchronization.
- Created apps/oauth/cache_test.go to verify local cache operations and pub/sub broadcasts.

refactor(cache): generic RAM cache with CoW and unified preheating

Replaced L2 Redis cache and old cache package with process-local generic pkg/cache/ram. Implemented Copy-on-Write for reads, fine-grained locks per type for writes, and unified preheating in bootstrap. Changed cache invalidation to lazy-loading to resolve SQLite deadlocks during transactions.
2026-06-27 14:32:27 +08:00
ryan 02ebb81929 refactor(oauth): replace legacy oauth cache with standard ram cache and add pubsub synchronization
- Replaced custom map-based cache in apps/oauth/cache.go with standard pkg/cache/ram framework.
- Implemented Redis Pub/Sub invalidation channels for distributed token and user cache synchronization.
- Created apps/oauth/cache_test.go to verify local cache operations and pub/sub broadcasts.

refactor(cache): generic RAM cache with CoW and unified preheating

Replaced L2 Redis cache and old cache package with process-local generic pkg/cache/ram. Implemented Copy-on-Write for reads, fine-grained locks per type for writes, and unified preheating in bootstrap. Changed cache invalidation to lazy-loading to resolve SQLite deadlocks during transactions.
2026-06-27 14:26:13 +08:00
ryan ac7b776378 refactor(db): use version string as primary key for ConfigVersion
-transition `of_config_versions` primary key from `id` to `version` string.
-add database migration files `202606270001_make_version_primary_key.sql` for PostgreSQL and SQLite.
-introduce AfterFind/AfterCreate GORM hooks to preserve JSON backward compatibility.
-refactor API controllers, logics, and front-end typescript definitions to receive `string` parameter.
2026-06-27 13:41:50 +08:00
ryan 13d6966cb5 refactor(db): use version string as primary key for ConfigVersion
-transition `of_config_versions` primary key from `id` to `version` string.
-add database migration files `202606270001_make_version_primary_key.sql` for PostgreSQL and SQLite.
-introduce AfterFind/AfterCreate GORM hooks to preserve JSON backward compatibility.
-refactor API controllers, logics, and front-end typescript definitions to receive `string` parameter.
2026-06-27 13:23:30 +08:00
ryan b48414e1fe docs: update 2026-06-26 21:25:18 +08:00
ryan b89dc9ec7e fix(waf): correct whitelist logic to bypass and add config/IP-group edit broadcasts
- Transition WAF whitelist filter from strict block-on-miss to bypass-on-hit logic

- Hook up broadcastIPGroupToAgents to CreateIPGroup and UpdateIPGroup WAF logics

- Hook up BroadcastActiveConfig to PublishConfigVersion and ActivateConfigVersion version logics

- Update WAF Lua tests in manager_test.go
2026-06-26 20:47:30 +08:00
ryan 49eae80c78 修复目录权限问题 2026-06-22 23:21:04 +08:00
ryan 8ed91dbf97 修复证书问题 2026-06-22 22:56:10 +08:00
ryan 92ceecc6ce 迁移配置表 2026-06-22 22:23:49 +08:00