Commit Graph

55 Commits

Author SHA1 Message Date
ryan 7e8aa5fa0f Merge branch 'codex/log-database-decoupling'
# Conflicts:
#	docs/changelog/index.md
#	frontend/app/(main)/error-pages/page.tsx
#	internal/infra/persistence/migrator/migrator_test.go
2026-08-09 08:37:37 +08:00
ryan ca21ff3a5b feat(option): add sw offline
fix(openresty): scope sw injection per cert partition

fix(lint): satisfy revive and biome format for sw offline feature

docs: sw offline scope changelog

fix(frontend): use scoped query key for sw scope zones

fix(frontend): hide preview link in sw contact page editor

feat(frontend): add sw scope domain picker and contact page fields

refactor(frontend): generalize html editor workspace for reuse

feat(openresty): scope sw offline injection by route domains

feat(openresty): add sw offline domains snapshot field

feat(option): add sw offline domains scope option

docs: fill html editor workspace generalization detail

docs: sw offline scope implementation plan

docs: sw offline scope design

test(openresty): assert single merged access block in sw enabled servers

fix(openresty): restrict sw intercept to https server blocks

fix(openresty): version sw offline cache by html content

fix(agent): escape redir in sw challenge page to prevent xss

fix(agent): return sw.runtime module table and add lua spec

docs: sw offline fallback changelog

fix(frontend): memoize option map to preserve unsaved contact page edits

feat(frontend): add response pages module with contact page tab

feat(agent): ship sw offline lua assets and placeholder substitution

feat(config): wire sw offline options into config snapshot

feat(openresty): render sw offline assets and challenge intercept

feat(openresty): add sw offline ConfigSnapshot fields and placeholder

feat(db): seed sw offline options

feat(option): add sw offline config keys and validation

docs: add service worker offline fallback implementation plan

docs: adopt global-option pattern for SW offline fallback (matches origin error page)

docs: unify offline contact page with error pages as response pages

docs: service worker offline fallback design (issue #23)
2026-08-08 20:14:28 +08:00
ryan 7d71f1e4e1 feat(log): decouple log storage from ClickHouse with switchable logstore
- New internal/repository/logstore abstraction: exported domain interfaces
  (AccessLogStore/ObservabilityStore/UserAccessLogStore/StatusStore),
  config-driven provider (Active/Build/Migrating/SetConfigReader), GORM
  implementation for PostgreSQL/SQLite (incl. hourly rollups computed in
  real time, migration listers, PG partition maintenance), and a ClickHouse
  wrapper preserving the native batch path; repository facade delegates to
  logstore; import-lint test enforces apps never import analyticsrepo.
- ClickHouse is now optional: the log DB is either the main DB (postgres
  when database.enabled, else sqlite) or clickhouse; boot validation +
  first-run seed; log_database / log_db_migration are protected keys.
- New user task 切换日志数据库 (of_log_db_switch): freeze log writes,
  drain batch writers, copy all 6 raw log tables by id (preserving IDs)
  with target-partition pre-creation for PG, flip log_database on success,
  clear the freeze flag on failure.
- Per-store retention (log_retention_days_*) with expiry cleanup folded
  into the daily system_cleanup task; legacy database_auto_cleanup_* and
  of_database_auto_cleanup decommissioned.
- goose migrations: 6 log tables in PG (2 monthly-partitioned) + SQLite,
  retention config seeds, schedule cleanup; GET
  /api/v1/admin/status/log-database endpoint; frontend retention settings,
  switch-task UI and status badge; changelog and docs updated.

docs(plan): log database decoupling implementation plan

docs(design): log database decoupling design (ClickHouse optional)
2026-08-08 19:43:01 +08:00
ryan 6738abdec1 feat(openresty): add origin error page GET-only option
Allow restricting custom origin error HTML to GET requests so other
methods pass through origin responses. Adds option seed, snapshot field,
edge limit_except/Lua handling, and admin UI switch.
2026-08-06 20:22:44 +08:00
ryan 7d47db1f34 feat(option): seed and validate origin error page options 2026-08-06 13:59:54 +08:00
ryan 4eced2b721 feat(cloudflare): add DNS pointing integration 2026-08-04 13:30:40 +08:00
ryan 3edcdb9e9f feat(cloudflare): add DNS pointing integration
Implement Cloudflare connection management, pointing groups and members, asynchronous A-record reconciliation, node IP triggers, admin APIs, management pages, migrations, tests, and documentation.
2026-08-04 12:32:37 +08:00
ryan 943818f7d4 refactor(repository): 收敛 model/repository 分层为唯一持久化入口
将 OpenFlare 与平台业务的数据访问从 model 与 apps 直连迁入 repository,
model 仅保留实体与无 IO 规则;补充 code-check 架构守卫与开发规范。
2026-07-24 17:00:17 +08:00
ryan 33a1c32cf8 refactor(structure): group platform, infra, and shared packages
Reorganize internal packages into platform/infra/shared layers and update
imports, docs, and seed-count tests to match current system configs.
2026-07-24 15:41:59 +08:00
ryan ae5345c03e feat(rate-limit): add default request rate limit configuration
- Support openresty_default_limit_req_per_ip in system_configs.
- Add limit_req and limit_req_status 429 directive generation in openresty renderer.
- Implement route-level limit_req_per_ip override and explicit disable.
- Add frontend UI inputs and validation in rate limits tab config.
- Update swagger API docs and changelog for v3.4.3-beta.3.
2026-07-20 10:58:13 +08:00
ryan abe8fb8268 refactor(pages): 精简部署源模型并重构详情页交互
将 Remote 网络策略收敛为 allow_insecure,去掉脱敏与无用字段;
Pages 详情拆为部署/设置 Tab,统一卡片样式与来源信息展示。
2026-07-19 20:23:31 +08:00
ryan c92f986978 merge: 合并 PR #22 Pages 部署源 V2 到 feat/pages-source-sync-v2
基于最新 main 合并 deqiying/feat/pages-source-sync-v2,
解决 docs/changelog/index.md 与限流相关条目的冲突。
2026-07-19 19:36:48 +08:00
deqiying 848884d8cd fix(pages): 增加部署包孤儿补偿
按项目、来源、运行时与上传记录锁序补偿异常中断遗留的部署包。\n同时隐藏并保护系统内部排程,避免通用任务管理入口修改 scanner。
2026-07-19 19:08:43 +08:00
ryan 2bc2d82ad0 feat(config): add openresty default rate limit system options 2026-07-19 18:12:51 +08:00
deqiying 38b0516937 feat(pages): 支持 Remote 部署源同步
新增部署源配置与运行态模型、安全下载、租约续期、原子激活和失败补偿。

接入内部任务与脱敏前端交互,并阻止数据库 Trace 和日志展开敏感查询参数。
2026-07-19 17:36:51 +08:00
ryan 920a530aa7 feat(access-logs): 新增 IP 明细 Tab 并完善日志详情字段
按时间窗聚合 IP 请求数/2xx 比例/入出站流量,支持排序与详情分析;
日志明细详情仅展示请求业务字段,IP 情报迁至独立详情弹窗。
2026-07-19 00:42:59 +08:00
ryan fb5a4e5b59 feat(access-logs): 上报并展示边缘缓存状态 cache_status
OpenResty 日志输出 $upstream_cache_status;Agent/协议/ClickHouse 贯通入库。
明细列表与详情按 HIT/MISS 等推导命中、回源、未缓存三态标签。
2026-07-18 22:50:46 +08:00
ryan e49078ac3b feat(access-logs): 接入 User-Agent 与设备/浏览器/状态码分布
- Agent 访问日志上报 user_agent,OpenResty log_format 输出 http_user_agent
- of_node_access_logs 新增 user_agent 列并写入 ClickHouse
- 访问日志概览新增:设备类型饼图、状态码饼图、浏览器/OS/User-Agent 排行
- 日志明细列表增加 User-Agent 列
- 扩展 UA 解析工具(browser/os/device)并支持 CLI 识别
2026-07-18 21:18:59 +08:00
ryan 177578ef4e feat(access-logs): 重构访问日志为概览与明细双 Tab
新增访问日志概览 API 与前端页面:汇总请求量/访问量/带宽趋势与 Top 排行,
明细列表保留检索;排行榜改为紧凑列表样式。
2026-07-18 20:58:32 +08:00
ryan f0e234df1f feat(obs): 访问日志 SSOT 与 edge_health,去掉协议兼容层
Agent 仅上报 host_metrics/edge_health/access_logs;业务流量与 UV 由
Server 侧访问日志聚合。新增 of_node_edge_health 与 of_access_log_hourly,
删除 request_reports/openresty 吞吐路径;API 不再暴露 traffic_reports
与 openresty_rx|tx。心跳/离线默认阈值与回填迁移一并入库。
2026-07-18 11:53:11 +08:00
ryan a0fcf9f627 feat(pages): configurable limits, multi-format packages, and dual versioning
Make Pages package size and history retention system-configurable, support
zip/tar.gz/tar.xz/tar.bz2/tar/7z uploads, prune history with clear keep-N
semantics, and rebind agent config to the live active Pages deployment so
main-config rollback never depends on pruned packages.
2026-07-17 17:16:48 +08:00
ryan 1eff7878a1 prettier 2026-07-13 15:10:28 +08:00
ryan a1a997bcda feat(waf): complete composable rule orchestration
Add the React Flow rule editor, ordered graph APIs and runtime DAG execution.\n\nPublish rules only on OpenResty reload and reconcile checksum-driven IP group snapshots in bounded shared memory.
2026-07-13 14:17:15 +08:00
ryan af20e2e838 feat(waf): add composable rule graph core 2026-07-13 11:57:36 +08:00
ryan 41cd23a64d feat(api): support traffic bytes tracking in edge access logs and refactor analytics models
- Add `bytes_sent` to `NodeAccessLog` on both Agent and Master Server.
- Create ClickHouse migration `202607120001_add_bytes_sent_to_node_access_logs.sql`.
- Refactor duplicate stats structs by centralizing them into `analyticsmodel` package with type aliases.
- Simplify access log store delegations and remove redundant mapping loops.
- Regenerate Swagger documentation.
- Update changelog index.md.
2026-07-12 17:27:57 +08:00
ryan 31b4886a14 feat(zone): add Cloudflare-style traffic overview charts
Expose Zone stats API with multi-host access-log aggregates and time
series, and render unique visitors, requests and data served on the
Zone overview with 24h/7d/30d range controls.
2026-07-12 16:38:35 +08:00
ryan d615d85a26 refactor: remove unused remarks and add quick domain create
Drop remark fields from Zone, Zone domains, proxy routes, WAF rule
groups and IP groups across models, APIs, UI and DB columns (keep
certificate/origin remarks). Add quick-create domain input for short
labels, @ apex and full FQDNs when binding domains.
2026-07-12 16:16:56 +08:00
ryan 5b52acdd6c refactor(zone): remove legacy route domain storage
第二阶段清理:删除 of_managed_domains 与 of_proxy_routes 冗余域名/证书列,
移除 ManagedDomain 模型与 API、路由侧 legacy 字段维护,以及前端 WebsiteService。
ImportLegacy 在旧列/旧表缺失时跳过对应源,保持幂等。
2026-07-12 15:31:01 +08:00
ryan d0536fcdd5 refactor(proxy): bind routes through zone domains 2026-07-12 14:52:19 +08:00
ryan 50678756d4 feat(zone): add normalized zone domain schema 2026-07-12 14:23:25 +08:00
ryan 160e63558f fix(clickhouse): harden R/W path P0–P3 (cleanup, durability, rollups)
Honest TTL cleanup semantics; enqueue-safe dedup with flush retry and writer
metrics; model insert hooks; latest-per-node and hourly metric/openresty
rollups; small-host pool/async defaults, traffic hourly TTL, and UV labeling.
2026-07-10 10:34:04 +08:00
ryan b928928958 fix(observability): restore 24h capacity/network/disk trends via CH hourly agg
Node and dashboard 24h capacity, network, and disk IO charts only used the
latest limited raw snapshots (120/500 rows), so historical hour buckets stayed
empty. Prefer ClickHouse hourly aggregates with counter deltas, and fall back
to raw snapshots when aggregation is unavailable.
2026-07-10 09:48:45 +08:00
ryan 58624db397 perf(clickhouse): Phase 2 legacy governance — TTL cleanup, unified pool, MV, ops API
- Replace retention ALTER DELETE with MATERIALIZE TTL; use TRUNCATE for delete-all
- Remove GORM ClickHouse pool; migrate user access log reads to ChConn
- Drop query-side trim(remote_addr); enable wait_for_async_insert=1
- Add of_node_traffic_hourly MV and dashboard traffic trend fallback
- Add GET /admin/status/clickhouse operational metrics endpoint
2026-07-02 15:47:38 +08:00
ryan 38946d1af5 fix(clickhouse): resolve lint issues from optimization stack 2026-07-02 15:28:49 +08:00
ryan caf2ffcff4 perf(clickhouse): P1 TTL migrations, ORDER BY tune, remote_addr normalization 2026-07-02 15:25:03 +08:00
ryan 6525bef15d perf(clickhouse): P0/P1 access log and WAF query aggregation and SQL pagination 2026-07-02 15:23:17 +08:00
ryan 13d6966cb5 refactor(db): use version string as primary key for ConfigVersion
-transition `of_config_versions` primary key from `id` to `version` string.
-add database migration files `202606270001_make_version_primary_key.sql` for PostgreSQL and SQLite.
-introduce AfterFind/AfterCreate GORM hooks to preserve JSON backward compatibility.
-refactor API controllers, logics, and front-end typescript definitions to receive `string` parameter.
2026-06-27 13:23:30 +08:00
ryan 92ceecc6ce 迁移配置表 2026-06-22 22:23:49 +08:00
ryan 346024f346 feat(relay): support configurable frps webui port and fix node detail integration
- Implement configurable FRPS WebUI switch and custom port setting (relay_frps_web_ui_port) in system configs.
- Integrate settings into Relay Node detail manage page instead of global settings.
- Dynamically query server version to select matching Docker image tag for Relay installation.
- Clean up legacy code and fix backend linter/test warnings.
2026-06-22 12:48:54 +08:00
ryan ffe98f6307 移除Notice 2026-06-22 12:00:43 +08:00
ryan 0a3cd250d1 fix relay 2026-06-22 11:56:31 +08:00
ryan 895dec208f fix(agent): write nginx pid and temp dirs under data_dir for non-root runtime
OpenResty running as openflare can no longer write pid or client/proxy temp
paths under the OpenResty install prefix. Templates and apply-time rendering
now use __OPENFLARE_PID_PATH__ and __OPENFLARE_NGINX_CACHE_DIR__ under
data_dir/var/run and data_dir/var/cache/nginx, with legacy pid path patched
at apply. Consolidate runtimeuser path helpers into the main package file so
IDEs resolve references across build tags.
2026-06-21 14:40:10 +08:00
ryan 40291136b7 fix(openresty): disable server version disclosure in main config template
Add server_tokens off to the default OpenResty main config template, seeded
option template, and agent safe fallback config so responses no longer
expose nginx/OpenResty version numbers in Server headers or error pages.
2026-06-21 14:25:32 +08:00
ryan d3777eac2d fix(agent): unify agent and openresty runtime user as openflare
Introduce the shared openflare service account for the agent process and
OpenResty workers, normalize data_dir ownership on startup, and ensure
managed paths are chowned with 0755/0644 during sync and apply. Docker
entrypoint fixes volume ownership before dropping privileges; local systemd
install runs the service as openflare with CAP_NET_BIND_SERVICE.
2026-06-21 14:25:20 +08:00
ryan 36f11c6ecb 修复代理路由详情认证配置 Tab:移除 PoW 配置(PoW 仅在 WAF 规则组中设置);保留 Basic Auth 保存能力;移除页头重复的「保存当前分区」按钮。 2026-06-21 11:28:50 +08:00
ryan e479ae75e6 修复 Pages 路由发布失败并报 pages module is not available:配置快照发布流程补齐 Pages 项目激活部署解析与 pages_deployment 写入。 2026-06-21 11:12:37 +08:00
ryan b12a9b0185 fix: 修复应用日志异常膨胀 2026-06-21 10:21:18 +08:00
ryan 117d473c27 fix: 修复 WAF 规则组保存/绑定网站时报 of_waf_rule_group_bindings_pkey 冲突 2026-06-20 22:05:30 +08:00
ryan 99f6f3231a fix: 修复 WAF 规则组保存/绑定网站时报 of_waf_rule_group_bindings_pkey 冲突 2026-06-20 21:33:46 +08:00
ryan b04a358e5e fix: 配置版本列表按 created_at 倒序展示 2026-06-20 21:32:18 +08:00