Commit Graph

163 Commits

Author SHA1 Message Date
ryan 4b83f91b31 docs(docker): use ./config/clickhouse for CH performance mount
Move performance.xml to config/clickhouse and mount the directory to
/etc/clickhouse-server/config.d; update docs and compose paths.
2026-07-10 10:59:21 +08:00
ryan 9d2fac5d4c docs(config): sync .env.example and config.example.yaml for CH defaults
Align ClickHouse pool/password placeholders and docker-compose env docs with
runtime defaults and published ports used in local Docker testing.
2026-07-10 10:48:30 +08:00
ryan b4b93ff4ed fix(docker): make ClickHouse startable on 25.x and reachable from host
Lower merge-tree free-entry thresholds for small background pools, bind
listen_host to 0.0.0.0 for published ports, and allow CLICKHOUSE_ENABLED=true
in tests for live_ch smoke coverage.
2026-07-10 10:44:49 +08:00
ryan 160e63558f fix(clickhouse): harden R/W path P0–P3 (cleanup, durability, rollups)
Honest TTL cleanup semantics; enqueue-safe dedup with flush retry and writer
metrics; model insert hooks; latest-per-node and hourly metric/openresty
rollups; small-host pool/async defaults, traffic hourly TTL, and UV labeling.
2026-07-10 10:34:04 +08:00
ryan 9b3555c569 fix(clickhouse): cut idle CPU from tiny parts and oversized merge pools
Observability writers flushed every few seconds with MinBatchSize unset,
creating constant small parts and merge load. Enable MinBatchSize with
MaxFlushWait, batch access logs more aggressively, and shrink ClickHouse
background pools for 3c hosts.
2026-07-10 10:08:32 +08:00
ryan b928928958 fix(observability): restore 24h capacity/network/disk trends via CH hourly agg
Node and dashboard 24h capacity, network, and disk IO charts only used the
latest limited raw snapshots (120/500 rows), so historical hour buckets stayed
empty. Prefer ClickHouse hourly aggregates with counter deltas, and fall back
to raw snapshots when aggregation is unavailable.
2026-07-10 09:48:45 +08:00
ryan b312460ddf chore(release): v3.1.1
### ⚡️ 优化与改进
- 将 cap_login_enabled 默认值由 true 变更为 false,默认关闭登录界面 PoW 人机验证。
2026-07-06 12:29:53 +08:00
ryan 336185f01c release: v3.1.0
### 🛠 修复
- 修复 ClickHouse TTL 迁移中 DateTime64 时间列无法直接设置 TTL 导致 goose 启动失败的问题,改为通过 toDateTime() 转换后再应用 TTL。
- 修复 ClickHouse 迁移尝试缩短 ORDER BY 排序键时与隐式主键前缀冲突导致迁移失败的问题,移除不支持的 MODIFY ORDER BY 操作。
- 修复系统设置页面 URL tab 参数未包含 openflare-ops 选项卡导致无法正确定位的问题,并在无参数时默认选中 OpenFlare 选项卡。
- 修复系统自更新检测上游 GitHub Release 时,因资产包名称前缀 openflare-server 与仓库名不完全一致导致匹配失败并报错「未找到兼容的 Release」的问题。
- 修复全局搜索数据源覆盖不全的问题,补全所有核心业务控制台页面及管理员专有页面的检索支持。

### ⚡️ 优化与改进
- ClickHouse 启用 async_insert 异步写入缓冲,并调高 block_buffer_size 与连接池默认值,降低小 part 生成与连接争用。
- 优化 ClickHouse 写入路径:移除 Agent 心跳中的同步 ALTER DELETE 保留清理,batchwriter 新增 MinBatchSize 抑制过小批次定时 flush,可观测 writer 批次与 flush 间隔调优并补全去重。
- ClickHouse 分析表新增 TTL 自动过期策略,访问日志 180 天、节点访问日志 90 天、其余观测与聚合表 30 天自动清理。
- 访问日志与 WAF IP 组查询改为 ClickHouse 侧聚合与 SQL 分页,默认限制近 7 天查询窗口,浏览器分布查询增加 Top 100 限制。
- Dashboard 与节点可观测 API 消除无 LIMIT 全表扫描,增加短 TTL 内存缓存,前端轮询间隔分别调整为 60s/30s。
- ClickHouse 遗留治理 Phase 2:保留期清理改为 TTL MATERIALIZE TTL,统一 ChConn 读路径,新增 /admin/status/clickhouse 运维指标与 of_node_traffic_hourly 预聚合 MV。
- 审计访问日志写入时仅保留安全相关请求头并以 SHA-256 脱敏,将 headers 载荷上限收紧至 2KB,减小行宽与 merge CPU 开销。
- Docker 部署为 ClickHouse 增加 nofile ulimits 与性能配置挂载,限制 max_concurrent_queries 与后台合并争用。
- 数据库自动清理任务新增 OpenResty、FRPS、FRPC 观测表清理目标。

### 💄 其他/体验
- 隐藏侧边栏文档库中的「规范示例」与「接口文档」,将「使用文档」及其他相关页面链接统一跳转至外部文档站 https://open-flare.pages.dev/。
- 移除系统设置 OpenFlare 标签页下的版本信息卡片及对应升级管理弹窗逻辑。
- 系统设置页面支持通过 URL 持久化当前选中的 Tab 状态。
2026-07-04 09:43:31 +08:00
ryan 44bba0f19a fix(clickhouse): remove unsupported MODIFY ORDER BY from migration
ClickHouse keeps the implicit PRIMARY KEY when shortening ORDER BY,
which fails with "Primary key must be a prefix of the sorting key".
TTL-only changes are safe and unblock goose startup; narrowing ORDER BY
would require table recreation.
2026-07-02 16:45:48 +08:00
ryan f0eca028f9 fix(clickhouse): cast DateTime64 to DateTime in TTL migration 2026-07-02 16:21:30 +08:00
ryan 58624db397 perf(clickhouse): Phase 2 legacy governance — TTL cleanup, unified pool, MV, ops API
- Replace retention ALTER DELETE with MATERIALIZE TTL; use TRUNCATE for delete-all
- Remove GORM ClickHouse pool; migrate user access log reads to ChConn
- Drop query-side trim(remote_addr); enable wait_for_async_insert=1
- Add of_node_traffic_hourly MV and dashboard traffic trend fallback
- Add GET /admin/status/clickhouse operational metrics endpoint
2026-07-02 15:47:38 +08:00
ryan caf2ffcff4 perf(clickhouse): P1 TTL migrations, ORDER BY tune, remote_addr normalization 2026-07-02 15:25:03 +08:00
ryan 0e86fe3547 perf(clickhouse): P2 docker server tuning and audit log payload reduction 2026-07-02 15:23:17 +08:00
ryan 5ad151469c feat(frontend): delete unused version-upgrade-dialog component and use-openflare-server-upgrade hook
- Permanently delete version-upgrade-dialog.tsx and use-openflare-server-upgrade.ts as they are no longer referenced after removing the version info card from openflare-ops settings.
2026-06-30 21:04:52 +08:00
ryan 6467b32d8e fix(frontend): include openflare-ops tab in whitelist and make it default
- Include 'openflare-ops' in the list of validTabs so that specifying ?tab=openflare-ops correctly loads the OpenFlare settings tab.
- Set fallback tab default to 'openflare-ops' when no tab parameter is specified.
- Document changes in changelog.
2026-06-30 20:56:34 +08:00
ryan 34225cb88a fix(updater): resolve release asset name matching for openflare-server
- Update expectedAssetNames helper to match lowercase repoName prefix and lowercase repoName with -server suffix (e.g. openflare-server).
- Fixes 'no compatible release found' error when checking GitHub Action releases.
2026-06-30 20:47:32 +08:00
ryan 389f02b6b0 feat(frontend): update navigation links and expand search coverage
- Hide 'Specification Examples' and 'API Docs' from sidebar documents group, pointing 'Use Docs' externally to pages.dev.
- Complete searchData array to cover all console business pages and missing admin-only pages.
- Add changelog records for these adjustments.
2026-06-30 20:36:55 +08:00
ryan 97095e8f12 release: v3.0.2
### 🛠 修复
- 修复 PostgreSQL 自增主键序列在历史数据迁移(INSERT 指定显式 ID)后与实际数据不同步的问题,通过新增全局序列同步脚本一键重置所有相关表的自增计数器。
2026-06-30 16:17:21 +08:00
ryan 23be2f9296 fix(db): 新增 PostgreSQL 数据库自增序列全局同步迁移脚本
为了解决因历史数据以显式 ID 方式迁移导致 PostgreSQL 自增序列计数器不同步,产生主键冲突唯一性约束报错(如 WAF 规则组和 IP 组保存失败)的问题,在 PostgreSQL 迁移中加入了对所有相关表 pg_get_serial_sequence 重置的代码。同时,在 SQLite 中补齐了对应的同名迁移文件。
2026-06-30 16:13:37 +08:00
ryan 113ea25aa4 release: v3.0.1
### ⚡️ 优化与改进
- 新增管理后台用户个人信息编辑与重置密码功能。
- 新增用户列表邮箱列展示以及基于邮箱的搜索过滤。
- 后端新增 `reset-passwd` 命令行工具,支持通过命令行直接重置用户密码。

### 🛠 修复
- 修复添加 DNS 账号时,因直接传递类静态方法作为 React Query 的 mutationFn 导致 JavaScript 运行时丢失 `this` 上下文报错 `this.post is not a function` 的问题。
- 修复侧边栏一级菜单项当前页面字体颜色被硬编码为 `#6366F1` 的问题,改用 CSS 主题变量 `text-sidebar-primary`,以保证多主题色彩一致。
- 修复默认主题(Default)遗漏声明 `destructive-foreground` 变量,导致删除确认按钮在某些状态下渲染为黑底黑字而无法阅读的问题。
- 修复 Cobra 命令行初始化注册逻辑,确保所有应用运行模式(All, API, Worker, Scheduler)都正确注册为 Cobra 子命令。
- 优化系统设置页面的色彩定义,移除硬编码的 Indigo 靛蓝色以适配多主题切换。

### 💄 其他/体验
- 在 `AGENTS.md` 规范中新增关于防止服务类静态方法 callback 上下文丢失的开发规范指南。
2026-06-30 11:58:09 +08:00
ryan c230d5a744 fix(frontend): 解决添加DNS账号时this.post is not a function报错
在 DnsAccountCreateDialog 组件中,mutationFn 错误地直接传递了类静态方法 DnsAccountService.create,导致执行时丢失 class constructor 上下文。现将其修改为使用箭头函数包裹,以保证 this 指向正确。
2026-06-30 11:52:24 +08:00
ryan 2816152536 fix(frontend): adjust sidebar active text color and fix destructive button contrast
- Update sidebar active menu button text color to use theme dynamic `sidebar-primary` variable instead of hardcoded hex value.

- Add missing `destructive-foreground` variables to default theme config and styles, resolving the black-on-black text contrast issue on confirmation dialog delete buttons.

- Update changelog to track these fixes.
2026-06-28 10:34:20 +08:00
ryan f03c88ffad docs: update 2026-06-27 16:29:58 +08:00
ryan a97d87acf5 fix(frontend): fix hasConfigDiff function missing WAF and site changes
Add check for waf_config_changed, added_sites, removed_sites, and modified_sites in hasConfigDiff helper to prevent disabling the publish button on WAF/site edits.
2026-06-27 15:40:16 +08:00
ryan 8d8814b416 refactor(oauth): replace legacy oauth cache with standard ram cache and add pubsub synchronization
- Replaced custom map-based cache in apps/oauth/cache.go with standard pkg/cache/ram framework.
- Implemented Redis Pub/Sub invalidation channels for distributed token and user cache synchronization.
- Created apps/oauth/cache_test.go to verify local cache operations and pub/sub broadcasts.

refactor(cache): generic RAM cache with CoW and unified preheating

Replaced L2 Redis cache and old cache package with process-local generic pkg/cache/ram. Implemented Copy-on-Write for reads, fine-grained locks per type for writes, and unified preheating in bootstrap. Changed cache invalidation to lazy-loading to resolve SQLite deadlocks during transactions.
2026-06-27 14:32:27 +08:00
ryan ac7b776378 refactor(db): use version string as primary key for ConfigVersion
-transition `of_config_versions` primary key from `id` to `version` string.
-add database migration files `202606270001_make_version_primary_key.sql` for PostgreSQL and SQLite.
-introduce AfterFind/AfterCreate GORM hooks to preserve JSON backward compatibility.
-refactor API controllers, logics, and front-end typescript definitions to receive `string` parameter.
2026-06-27 13:41:50 +08:00
ryan 13d6966cb5 refactor(db): use version string as primary key for ConfigVersion
-transition `of_config_versions` primary key from `id` to `version` string.
-add database migration files `202606270001_make_version_primary_key.sql` for PostgreSQL and SQLite.
-introduce AfterFind/AfterCreate GORM hooks to preserve JSON backward compatibility.
-refactor API controllers, logics, and front-end typescript definitions to receive `string` parameter.
2026-06-27 13:23:30 +08:00
ryan b89dc9ec7e fix(waf): correct whitelist logic to bypass and add config/IP-group edit broadcasts
- Transition WAF whitelist filter from strict block-on-miss to bypass-on-hit logic

- Hook up broadcastIPGroupToAgents to CreateIPGroup and UpdateIPGroup WAF logics

- Hook up BroadcastActiveConfig to PublishConfigVersion and ActivateConfigVersion version logics

- Update WAF Lua tests in manager_test.go
2026-06-26 20:47:30 +08:00
ryan 49eae80c78 修复目录权限问题 2026-06-22 23:21:04 +08:00
ryan 8ed91dbf97 修复证书问题 2026-06-22 22:56:10 +08:00
ryan 92ceecc6ce 迁移配置表 2026-06-22 22:23:49 +08:00
ryan d9b8dc81ee fix(relay): prioritize IPv4 in dual-stack outbound IP detection
修复 Relay 节点在双栈网络环境下可能上报 IPv6 地址,导致 Tunnel frpc 客户端无法连接 frps 的问题。

根因:
- HTTPOutboundIPStrategy 首次尝试使用 tcp4 强制 IPv4 连接
- 失败时回退到双栈 tcp 客户端,此时可能通过 IPv6 连接并返回 IPv6 地址
- Relay 心跳将 IPv6 地址上报给 Server
- Tunnel frpc 尝试连接该 IPv6 地址失败

修复:
- 在回退到双栈客户端后,仍优先返回 IPv4 地址(通过 ip.To4() 转换)
- 仅在完全无 IPv4 路由时才返回 IPv6
- 确保 Relay 心跳上报的 IP 与 frpc 连接兼容

影响范围:
- pkg/geoip.HTTPOutboundIPStrategy.GetOutboundIP
- internal/apps/relay/heartbeat.Service(通过 nodeip.DetectWithContext 调用)

测试:
- 新增 TestHTTPOutboundIPStrategyPrioritizesIPv4 验证 IPv4 优先逻辑
- 所有现有测试保持通过
2026-06-22 15:49:08 +08:00
ryan deb232d840 refactor(edge): unify dynamic IP detection and prioritize IPv4 reporting
- Align agent, relay, and flared to dynamically resolve IP during heartbeat using the nodeip package (when not manually configured).
- Update GeoIP outbound IP strategy to prefer IPv4 HTTP client lookup using tcp4 dialer and fall back to dual-stack tcp.
- Optimize agent profile fingerprinting to exclude dynamic UptimeSeconds and ReportedAtUnix fields, preventing redundant updates.
- Refactor unit tests to prevent outbound network queries during tests.
2026-06-22 15:06:39 +08:00
ryan 346979344f docs(changelog): 补充 WS EOF 修复与 frpc stderr 日志条目 2026-06-22 14:22:52 +08:00
ryan 346024f346 feat(relay): support configurable frps webui port and fix node detail integration
- Implement configurable FRPS WebUI switch and custom port setting (relay_frps_web_ui_port) in system configs.
- Integrate settings into Relay Node detail manage page instead of global settings.
- Dynamically query server version to select matching Docker image tag for Relay installation.
- Clean up legacy code and fix backend linter/test warnings.
2026-06-22 12:48:54 +08:00
ryan 0a3cd250d1 fix relay 2026-06-22 11:56:31 +08:00
ryan 53e6efa33b fix agent 2026-06-22 11:49:36 +08:00
ryan 3d15c65afd perf 2026-06-22 11:32:24 +08:00
ryan 16b02fd3f1 修复 Agent 升级版本比对逻辑 2026-06-22 11:26:55 +08:00
ryan 7b14e15afb refactor(frontend): simplify routing paths for certificates, dns-accounts and ip-groups
- Remove /websites prefix from TLS certificates and DNS accounts routes.
- Remove /waf prefix from IP groups route.
- Correct relative import paths for shared components.
2026-06-22 10:56:40 +08:00
ryan 3a2878d070 feat(api): integrate TLS certificate renewal into async task framework
Replace native goroutines in RenewCertificate logic with Asynq task dispatching to support queue execution, retry capability, and detailed task execution logs.
2026-06-22 10:54:15 +08:00
ryan 895dec208f fix(agent): write nginx pid and temp dirs under data_dir for non-root runtime
OpenResty running as openflare can no longer write pid or client/proxy temp
paths under the OpenResty install prefix. Templates and apply-time rendering
now use __OPENFLARE_PID_PATH__ and __OPENFLARE_NGINX_CACHE_DIR__ under
data_dir/var/run and data_dir/var/cache/nginx, with legacy pid path patched
at apply. Consolidate runtimeuser path helpers into the main package file so
IDEs resolve references across build tags.
2026-06-21 14:40:10 +08:00
ryan 40291136b7 fix(openresty): disable server version disclosure in main config template
Add server_tokens off to the default OpenResty main config template, seeded
option template, and agent safe fallback config so responses no longer
expose nginx/OpenResty version numbers in Server headers or error pages.
2026-06-21 14:25:32 +08:00
ryan d3777eac2d fix(agent): unify agent and openresty runtime user as openflare
Introduce the shared openflare service account for the agent process and
OpenResty workers, normalize data_dir ownership on startup, and ensure
managed paths are chowned with 0755/0644 during sync and apply. Docker
entrypoint fixes volume ownership before dropping privileges; local systemd
install runs the service as openflare with CAP_NET_BIND_SERVICE.
2026-06-21 14:25:20 +08:00
ryan ee047cb351 修复 Pages 站点在未启用 SPA Fallback 时访问根路径 / 返回 404:OpenResty 渲染增加 location = / 精确匹配,通过 try_files 提供入口文件(index 指令在 try_files ... =404 场景下不会作用于根路径)。 2026-06-21 12:18:20 +08:00
ryan 6ed3c0c81f 收敛 Pages 部署包读取路径 2026-06-21 12:04:54 +08:00
ryan 13a375e042 修复 Agent 部署 Pages 问题 2026-06-21 11:53:54 +08:00
ryan 0f904b4b6d 修复代理路由详情认证配置 Tab:移除 PoW 配置(PoW 仅在 WAF 规则组中设置);保留 Basic Auth 保存能力;移除页头重复的「保存当前分区」按钮。 2026-06-21 11:26:37 +08:00
ryan e479ae75e6 修复 Pages 路由发布失败并报 pages module is not available:配置快照发布流程补齐 Pages 项目激活部署解析与 pages_deployment 写入。 2026-06-21 11:12:37 +08:00
ryan 6f267bbf21 修复仪表盘与节点详情「24 小时网络趋势」误按速率展示:改为 OpenResty 入/出站小时流量与近 24 小时总量摘要,Y 轴与 tooltip 自动换算 B/KB/MB/GB。 2026-06-21 11:08:25 +08:00