60 Commits

Author SHA1 Message Date
ryan 73d8173018 recvcheck 7→1:6 个 GORM 模型 TableName 改为指针接收者(GORM 源码确认 reflect.New 判定 Tabler,兼容;模型单测通过)。MillisecondDuration 刻意保留(encoding/json 要求 Marshal 值/Unmarshal 指针的混合)。
Result: {"status":"keep","total_issues":9,"golint_canonicalheader":0,"golint_errname":0,"golint_errorlint":1,"golint_forcetypeassert":0,"golint_gosec":0,"golint_intrange":0,"golint_modernize":3,"golint_nilnil":3,"golint_perfsprint":0,"golint_prealloc":0,"golint_recvcheck":1,"golint_usestdlibvars":0,"golint_wastedassign":0,"golint_total":8,"eslint_problems":1,"eslint_errors":0,"eslint_warnings":1,"tsc_errors":0,"measure_s":45}
2026-08-16 18:00:45 +08:00
ryan 699e95f12c perfsprint 18→0:strconv.Itoa/FormatInt/FormatUint/FormatBool 替代 fmt.Sprintf、无动词 fmt.Errorf→errors.New、纯字符串拼接。全部语义等价(已核对 diff)。修正 fixer 遗留的 import 问题(引入 goimports 统一整理)。
Result: {"status":"keep","total_issues":56,"golint_canonicalheader":8,"golint_errname":1,"golint_errorlint":12,"golint_forcetypeassert":3,"golint_gosec":0,"golint_intrange":3,"golint_modernize":5,"golint_nilnil":3,"golint_perfsprint":0,"golint_prealloc":3,"golint_recvcheck":7,"golint_usestdlibvars":3,"golint_wastedassign":7,"golint_total":55,"eslint_problems":1,"eslint_errors":0,"eslint_warnings":1,"tsc_errors":0,"measure_s":47}
2026-08-16 17:34:44 +08:00
ryan a4dd5ca9e1 feat(dashboard): 首页请求趋势拆分状态码并合并容量到业务流量
- 24 小时请求趋势拆分展示请求总量与 200/400/500 状态码请求量,独占一行;
  时间桶聚合新增 status_200/400/500_count(CH countIf、PG FILTER),
  请求趋势改为基于原始桶聚合(小时 rollup 无状态码口径)
- 首页移除宿主机磁盘指标,容量趋势(CPU/内存)并入业务流量卡片展示
- 压缩协议 traffic_24h 扩展为 7 元组,前端归一化同步更新
2026-08-13 11:10:37 +08:00
ryan 75d1fcf345 feat(access-logs): 日志明细支持按状态码筛选并折叠次要搜索项,修复首页来源分布无数据
- 修复 PostgreSQL/SQLite 日志库下首页「来源分布」卡片无数据:RegionCounts 对空
  节点 ID 误拼 node_id = '' 恒空条件,改为空节点 ID 表示全节点聚合(对齐 CH 语义),
  并过滤空白归属地
- /access-logs?tab=list 新增状态码筛选:状态码下拉含常用 2xx/3xx/4xx/5xx 选项,
  校验 100-599,非法值返回 400;ClickHouse 与 PostgreSQL/SQLite 日志库均支持
- 搜索框折叠:默认仅显示来源 IP 与状态码,节点 ID/访问域名/请求路径折叠进
  「更多筛选」
2026-08-13 09:59:32 +08:00
ryan bd2183c8bb feat(log): add independent short retention for performance metrics
性能指标(CPU/内存/磁盘/网络)不再跟随 log_retention_days_*,新增三库共用
的 metric_retention_days 配置(默认 3 天),系统垃圾清理按独立短留存清理
指标快照;访问日志保留时长不变。新增 PG/SQLite 双方言 goose 迁移 seed。
2026-08-09 09:04:33 +08:00
ryan 7e8aa5fa0f Merge branch 'codex/log-database-decoupling'
# Conflicts:
#	docs/changelog/index.md
#	frontend/app/(main)/error-pages/page.tsx
#	internal/infra/persistence/migrator/migrator_test.go
2026-08-09 08:37:37 +08:00
ryan ca21ff3a5b feat(option): add sw offline
fix(openresty): scope sw injection per cert partition

fix(lint): satisfy revive and biome format for sw offline feature

docs: sw offline scope changelog

fix(frontend): use scoped query key for sw scope zones

fix(frontend): hide preview link in sw contact page editor

feat(frontend): add sw scope domain picker and contact page fields

refactor(frontend): generalize html editor workspace for reuse

feat(openresty): scope sw offline injection by route domains

feat(openresty): add sw offline domains snapshot field

feat(option): add sw offline domains scope option

docs: fill html editor workspace generalization detail

docs: sw offline scope implementation plan

docs: sw offline scope design

test(openresty): assert single merged access block in sw enabled servers

fix(openresty): restrict sw intercept to https server blocks

fix(openresty): version sw offline cache by html content

fix(agent): escape redir in sw challenge page to prevent xss

fix(agent): return sw.runtime module table and add lua spec

docs: sw offline fallback changelog

fix(frontend): memoize option map to preserve unsaved contact page edits

feat(frontend): add response pages module with contact page tab

feat(agent): ship sw offline lua assets and placeholder substitution

feat(config): wire sw offline options into config snapshot

feat(openresty): render sw offline assets and challenge intercept

feat(openresty): add sw offline ConfigSnapshot fields and placeholder

feat(db): seed sw offline options

feat(option): add sw offline config keys and validation

docs: add service worker offline fallback implementation plan

docs: adopt global-option pattern for SW offline fallback (matches origin error page)

docs: unify offline contact page with error pages as response pages

docs: service worker offline fallback design (issue #23)
2026-08-08 20:14:28 +08:00
ryan 7d71f1e4e1 feat(log): decouple log storage from ClickHouse with switchable logstore
- New internal/repository/logstore abstraction: exported domain interfaces
  (AccessLogStore/ObservabilityStore/UserAccessLogStore/StatusStore),
  config-driven provider (Active/Build/Migrating/SetConfigReader), GORM
  implementation for PostgreSQL/SQLite (incl. hourly rollups computed in
  real time, migration listers, PG partition maintenance), and a ClickHouse
  wrapper preserving the native batch path; repository facade delegates to
  logstore; import-lint test enforces apps never import analyticsrepo.
- ClickHouse is now optional: the log DB is either the main DB (postgres
  when database.enabled, else sqlite) or clickhouse; boot validation +
  first-run seed; log_database / log_db_migration are protected keys.
- New user task 切换日志数据库 (of_log_db_switch): freeze log writes,
  drain batch writers, copy all 6 raw log tables by id (preserving IDs)
  with target-partition pre-creation for PG, flip log_database on success,
  clear the freeze flag on failure.
- Per-store retention (log_retention_days_*) with expiry cleanup folded
  into the daily system_cleanup task; legacy database_auto_cleanup_* and
  of_database_auto_cleanup decommissioned.
- goose migrations: 6 log tables in PG (2 monthly-partitioned) + SQLite,
  retention config seeds, schedule cleanup; GET
  /api/v1/admin/status/log-database endpoint; frontend retention settings,
  switch-task UI and status badge; changelog and docs updated.

docs(plan): log database decoupling implementation plan

docs(design): log database decoupling design (ClickHouse optional)
2026-08-08 19:43:01 +08:00
ryan 6738abdec1 feat(openresty): add origin error page GET-only option
Allow restricting custom origin error HTML to GET requests so other
methods pass through origin responses. Adds option seed, snapshot field,
edge limit_except/Lua handling, and admin UI switch.
2026-08-06 20:22:44 +08:00
ryan 7d47db1f34 feat(option): seed and validate origin error page options 2026-08-06 13:59:54 +08:00
ryan 4eced2b721 feat(cloudflare): add DNS pointing integration 2026-08-04 13:30:40 +08:00
ryan 3edcdb9e9f feat(cloudflare): add DNS pointing integration
Implement Cloudflare connection management, pointing groups and members, asynchronous A-record reconciliation, node IP triggers, admin APIs, management pages, migrations, tests, and documentation.
2026-08-04 12:32:37 +08:00
ryan 943818f7d4 refactor(repository): 收敛 model/repository 分层为唯一持久化入口
将 OpenFlare 与平台业务的数据访问从 model 与 apps 直连迁入 repository,
model 仅保留实体与无 IO 规则;补充 code-check 架构守卫与开发规范。
2026-07-24 17:00:17 +08:00
ryan 33a1c32cf8 refactor(structure): group platform, infra, and shared packages
Reorganize internal packages into platform/infra/shared layers and update
imports, docs, and seed-count tests to match current system configs.
2026-07-24 15:41:59 +08:00
ryan ae5345c03e feat(rate-limit): add default request rate limit configuration
- Support openresty_default_limit_req_per_ip in system_configs.
- Add limit_req and limit_req_status 429 directive generation in openresty renderer.
- Implement route-level limit_req_per_ip override and explicit disable.
- Add frontend UI inputs and validation in rate limits tab config.
- Update swagger API docs and changelog for v3.4.3-beta.3.
2026-07-20 10:58:13 +08:00
ryan abe8fb8268 refactor(pages): 精简部署源模型并重构详情页交互
将 Remote 网络策略收敛为 allow_insecure,去掉脱敏与无用字段;
Pages 详情拆为部署/设置 Tab,统一卡片样式与来源信息展示。
2026-07-19 20:23:31 +08:00
ryan c92f986978 merge: 合并 PR #22 Pages 部署源 V2 到 feat/pages-source-sync-v2
基于最新 main 合并 deqiying/feat/pages-source-sync-v2,
解决 docs/changelog/index.md 与限流相关条目的冲突。
2026-07-19 19:36:48 +08:00
deqiying 848884d8cd fix(pages): 增加部署包孤儿补偿
按项目、来源、运行时与上传记录锁序补偿异常中断遗留的部署包。\n同时隐藏并保护系统内部排程,避免通用任务管理入口修改 scanner。
2026-07-19 19:08:43 +08:00
ryan 2bc2d82ad0 feat(config): add openresty default rate limit system options 2026-07-19 18:12:51 +08:00
deqiying 38b0516937 feat(pages): 支持 Remote 部署源同步
新增部署源配置与运行态模型、安全下载、租约续期、原子激活和失败补偿。

接入内部任务与脱敏前端交互,并阻止数据库 Trace 和日志展开敏感查询参数。
2026-07-19 17:36:51 +08:00
ryan 920a530aa7 feat(access-logs): 新增 IP 明细 Tab 并完善日志详情字段
按时间窗聚合 IP 请求数/2xx 比例/入出站流量,支持排序与详情分析;
日志明细详情仅展示请求业务字段,IP 情报迁至独立详情弹窗。
2026-07-19 00:42:59 +08:00
ryan fb5a4e5b59 feat(access-logs): 上报并展示边缘缓存状态 cache_status
OpenResty 日志输出 $upstream_cache_status;Agent/协议/ClickHouse 贯通入库。
明细列表与详情按 HIT/MISS 等推导命中、回源、未缓存三态标签。
2026-07-18 22:50:46 +08:00
ryan e49078ac3b feat(access-logs): 接入 User-Agent 与设备/浏览器/状态码分布
- Agent 访问日志上报 user_agent,OpenResty log_format 输出 http_user_agent
- of_node_access_logs 新增 user_agent 列并写入 ClickHouse
- 访问日志概览新增:设备类型饼图、状态码饼图、浏览器/OS/User-Agent 排行
- 日志明细列表增加 User-Agent 列
- 扩展 UA 解析工具(browser/os/device)并支持 CLI 识别
2026-07-18 21:18:59 +08:00
ryan 177578ef4e feat(access-logs): 重构访问日志为概览与明细双 Tab
新增访问日志概览 API 与前端页面:汇总请求量/访问量/带宽趋势与 Top 排行,
明细列表保留检索;排行榜改为紧凑列表样式。
2026-07-18 20:58:32 +08:00
ryan f0e234df1f feat(obs): 访问日志 SSOT 与 edge_health,去掉协议兼容层
Agent 仅上报 host_metrics/edge_health/access_logs;业务流量与 UV 由
Server 侧访问日志聚合。新增 of_node_edge_health 与 of_access_log_hourly,
删除 request_reports/openresty 吞吐路径;API 不再暴露 traffic_reports
与 openresty_rx|tx。心跳/离线默认阈值与回填迁移一并入库。
2026-07-18 11:53:11 +08:00
ryan a0fcf9f627 feat(pages): configurable limits, multi-format packages, and dual versioning
Make Pages package size and history retention system-configurable, support
zip/tar.gz/tar.xz/tar.bz2/tar/7z uploads, prune history with clear keep-N
semantics, and rebind agent config to the live active Pages deployment so
main-config rollback never depends on pruned packages.
2026-07-17 17:16:48 +08:00
ryan 1eff7878a1 prettier 2026-07-13 15:10:28 +08:00
ryan a1a997bcda feat(waf): complete composable rule orchestration
Add the React Flow rule editor, ordered graph APIs and runtime DAG execution.\n\nPublish rules only on OpenResty reload and reconcile checksum-driven IP group snapshots in bounded shared memory.
2026-07-13 14:17:15 +08:00
ryan af20e2e838 feat(waf): add composable rule graph core 2026-07-13 11:57:36 +08:00
ryan 41cd23a64d feat(api): support traffic bytes tracking in edge access logs and refactor analytics models
- Add `bytes_sent` to `NodeAccessLog` on both Agent and Master Server.
- Create ClickHouse migration `202607120001_add_bytes_sent_to_node_access_logs.sql`.
- Refactor duplicate stats structs by centralizing them into `analyticsmodel` package with type aliases.
- Simplify access log store delegations and remove redundant mapping loops.
- Regenerate Swagger documentation.
- Update changelog index.md.
2026-07-12 17:27:57 +08:00
ryan 31b4886a14 feat(zone): add Cloudflare-style traffic overview charts
Expose Zone stats API with multi-host access-log aggregates and time
series, and render unique visitors, requests and data served on the
Zone overview with 24h/7d/30d range controls.
2026-07-12 16:38:35 +08:00
ryan d615d85a26 refactor: remove unused remarks and add quick domain create
Drop remark fields from Zone, Zone domains, proxy routes, WAF rule
groups and IP groups across models, APIs, UI and DB columns (keep
certificate/origin remarks). Add quick-create domain input for short
labels, @ apex and full FQDNs when binding domains.
2026-07-12 16:16:56 +08:00
ryan 5b52acdd6c refactor(zone): remove legacy route domain storage
第二阶段清理:删除 of_managed_domains 与 of_proxy_routes 冗余域名/证书列,
移除 ManagedDomain 模型与 API、路由侧 legacy 字段维护,以及前端 WebsiteService。
ImportLegacy 在旧列/旧表缺失时跳过对应源,保持幂等。
2026-07-12 15:31:01 +08:00
ryan d0536fcdd5 refactor(proxy): bind routes through zone domains 2026-07-12 14:52:19 +08:00
ryan 50678756d4 feat(zone): add normalized zone domain schema 2026-07-12 14:23:25 +08:00
ryan 160e63558f fix(clickhouse): harden R/W path P0–P3 (cleanup, durability, rollups)
Honest TTL cleanup semantics; enqueue-safe dedup with flush retry and writer
metrics; model insert hooks; latest-per-node and hourly metric/openresty
rollups; small-host pool/async defaults, traffic hourly TTL, and UV labeling.
2026-07-10 10:34:04 +08:00
ryan b928928958 fix(observability): restore 24h capacity/network/disk trends via CH hourly agg
Node and dashboard 24h capacity, network, and disk IO charts only used the
latest limited raw snapshots (120/500 rows), so historical hour buckets stayed
empty. Prefer ClickHouse hourly aggregates with counter deltas, and fall back
to raw snapshots when aggregation is unavailable.
2026-07-10 09:48:45 +08:00
ryan 58624db397 perf(clickhouse): Phase 2 legacy governance — TTL cleanup, unified pool, MV, ops API
- Replace retention ALTER DELETE with MATERIALIZE TTL; use TRUNCATE for delete-all
- Remove GORM ClickHouse pool; migrate user access log reads to ChConn
- Drop query-side trim(remote_addr); enable wait_for_async_insert=1
- Add of_node_traffic_hourly MV and dashboard traffic trend fallback
- Add GET /admin/status/clickhouse operational metrics endpoint
2026-07-02 15:47:38 +08:00
ryan 38946d1af5 fix(clickhouse): resolve lint issues from optimization stack 2026-07-02 15:28:49 +08:00
ryan caf2ffcff4 perf(clickhouse): P1 TTL migrations, ORDER BY tune, remote_addr normalization 2026-07-02 15:25:03 +08:00
ryan 6525bef15d perf(clickhouse): P0/P1 access log and WAF query aggregation and SQL pagination 2026-07-02 15:23:17 +08:00
ryan 13d6966cb5 refactor(db): use version string as primary key for ConfigVersion
-transition `of_config_versions` primary key from `id` to `version` string.
-add database migration files `202606270001_make_version_primary_key.sql` for PostgreSQL and SQLite.
-introduce AfterFind/AfterCreate GORM hooks to preserve JSON backward compatibility.
-refactor API controllers, logics, and front-end typescript definitions to receive `string` parameter.
2026-06-27 13:23:30 +08:00
ryan 92ceecc6ce 迁移配置表 2026-06-22 22:23:49 +08:00
ryan 346024f346 feat(relay): support configurable frps webui port and fix node detail integration
- Implement configurable FRPS WebUI switch and custom port setting (relay_frps_web_ui_port) in system configs.
- Integrate settings into Relay Node detail manage page instead of global settings.
- Dynamically query server version to select matching Docker image tag for Relay installation.
- Clean up legacy code and fix backend linter/test warnings.
2026-06-22 12:48:54 +08:00
ryan ffe98f6307 移除Notice 2026-06-22 12:00:43 +08:00
ryan 0a3cd250d1 fix relay 2026-06-22 11:56:31 +08:00
ryan 895dec208f fix(agent): write nginx pid and temp dirs under data_dir for non-root runtime
OpenResty running as openflare can no longer write pid or client/proxy temp
paths under the OpenResty install prefix. Templates and apply-time rendering
now use __OPENFLARE_PID_PATH__ and __OPENFLARE_NGINX_CACHE_DIR__ under
data_dir/var/run and data_dir/var/cache/nginx, with legacy pid path patched
at apply. Consolidate runtimeuser path helpers into the main package file so
IDEs resolve references across build tags.
2026-06-21 14:40:10 +08:00
ryan 40291136b7 fix(openresty): disable server version disclosure in main config template
Add server_tokens off to the default OpenResty main config template, seeded
option template, and agent safe fallback config so responses no longer
expose nginx/OpenResty version numbers in Server headers or error pages.
2026-06-21 14:25:32 +08:00
ryan d3777eac2d fix(agent): unify agent and openresty runtime user as openflare
Introduce the shared openflare service account for the agent process and
OpenResty workers, normalize data_dir ownership on startup, and ensure
managed paths are chowned with 0755/0644 during sync and apply. Docker
entrypoint fixes volume ownership before dropping privileges; local systemd
install runs the service as openflare with CAP_NET_BIND_SERVICE.
2026-06-21 14:25:20 +08:00
ryan 36f11c6ecb 修复代理路由详情认证配置 Tab:移除 PoW 配置(PoW 仅在 WAF 规则组中设置);保留 Basic Auth 保存能力;移除页头重复的「保存当前分区」按钮。 2026-06-21 11:28:50 +08:00