Commit Graph

102 Commits

Author SHA1 Message Date
ryan 15e614b304 fix(waf): pow 2026-07-13 17:07:49 +08:00
ryan 46941f65d5 fix(waf): handle empty rule bindings
Encode empty site bindings as arrays and normalize legacy JSON null values in the OpenResty runtime to prevent request-time Lua failures.
2026-07-13 16:49:55 +08:00
ryan a61d55bb1b chore: improve log print 2026-07-13 16:12:30 +08:00
ryan 6b6c786cfe feat(startup): print service banner after listener ready 2026-07-13 15:49:19 +08:00
ryan 0548a8a5d4 fix(redis): add maintenance notification startup switch
Default Redis maintenance notification negotiation to disabled and apply the startup-only setting to both platform and Asynq clients.
2026-07-13 15:43:15 +08:00
ryan 60bc03f519 fix(db): log SQL statements at debug level
Move routine GORM SQL output to debug and omit SQL text from slow-query and query-error logs to prevent sensitive parameters from being emitted at production log levels.
2026-07-13 15:39:48 +08:00
ryan 1eff7878a1 prettier 2026-07-13 15:10:28 +08:00
ryan 74ddf97b36 feat(agent): embed GeoLite2 City database
Initialize missing Country and City databases from embedded assets and use FyraLabs releases for periodic updates.
2026-07-13 14:38:36 +08:00
ryan a1a997bcda feat(waf): complete composable rule orchestration
Add the React Flow rule editor, ordered graph APIs and runtime DAG execution.\n\nPublish rules only on OpenResty reload and reconcile checksum-driven IP group snapshots in bounded shared memory.
2026-07-13 14:17:15 +08:00
ryan af20e2e838 feat(waf): add composable rule graph core 2026-07-13 11:57:36 +08:00
ryan 69467914fc fix(server): 修复 Agent 上报访问日志的 bytes_sent 在 Server 入库链路丢失,导致 Zone 概览“已提供的数据总计”长期为 0 的问题。 2026-07-12 19:02:52 +08:00
ryan c624512da6 fix(frontend): serve zone detail static export fallback 2026-07-12 18:51:01 +08:00
ryan 41cd23a64d feat(api): support traffic bytes tracking in edge access logs and refactor analytics models
- Add `bytes_sent` to `NodeAccessLog` on both Agent and Master Server.
- Create ClickHouse migration `202607120001_add_bytes_sent_to_node_access_logs.sql`.
- Refactor duplicate stats structs by centralizing them into `analyticsmodel` package with type aliases.
- Simplify access log store delegations and remove redundant mapping loops.
- Regenerate Swagger documentation.
- Update changelog index.md.
2026-07-12 17:27:57 +08:00
ryan f02fc9676a fix: resolve all build-test and code-check failures
- Fix Go backend mnd (magic number) and revive lint issues in zone stats.
- Remove unused React/Lucide imports and variables in zone overview.
- Add generateStaticParams and Suspense wrapper for websites/[zoneId] page to support Next.js static HTML export.
- Remove next/font/google dependency to allow fully offline frontend compilation.
- Fix hardcoded time dependency in ssl_renew_test.go.
- Fix async_tasks_test.go to respect minimum 90-day retention clamping in database auto-cleanup.
- Add Zone and ZoneDomain models to test database AutoMigrate schemas.
- Update integration tests to use the new zone_domain_ids route binding scheme.
2026-07-12 17:05:52 +08:00
ryan 31b4886a14 feat(zone): add Cloudflare-style traffic overview charts
Expose Zone stats API with multi-host access-log aggregates and time
series, and render unique visitors, requests and data served on the
Zone overview with 24h/7d/30d range controls.
2026-07-12 16:38:35 +08:00
ryan d615d85a26 refactor: remove unused remarks and add quick domain create
Drop remark fields from Zone, Zone domains, proxy routes, WAF rule
groups and IP groups across models, APIs, UI and DB columns (keep
certificate/origin remarks). Add quick-create domain input for short
labels, @ apex and full FQDNs when binding domains.
2026-07-12 16:16:56 +08:00
ryan 8afd103751 fix(zone): register domain delete/update APIs and drop edit UI
补全 Zone 与 Zone 域名的 update/delete 路由与业务逻辑,修复删除域名
404;前端域名列表移除编辑入口,仅保留添加与删除。
2026-07-12 15:56:45 +08:00
ryan 96b8ddc077 fix(migrator): only import zone domains during upgrade window
Zone 历史导入仅在 goose 版本位于 [202607120002, 202607130001) 时执行,
phase-2 删列完成后日常启动不再进入导入逻辑。
2026-07-12 15:38:28 +08:00
ryan 03b81e5f74 refactor(migrator): use goose SQL only and auto-import zones on upgrade
移除 Go goose 迁移(bridge/legacy data/zone import),改为 goose SQL 占位
与结构迁移脚本;启动时 Migrate 在删旧列前自动导入历史域名,去掉
migrate-zones 手动命令及文档中的人工导入步骤。
2026-07-12 15:36:00 +08:00
ryan 5b52acdd6c refactor(zone): remove legacy route domain storage
第二阶段清理:删除 of_managed_domains 与 of_proxy_routes 冗余域名/证书列,
移除 ManagedDomain 模型与 API、路由侧 legacy 字段维护,以及前端 WebsiteService。
ImportLegacy 在旧列/旧表缺失时跳过对应源,保持幂等。
2026-07-12 15:31:01 +08:00
ryan 350b433cc1 feat(web): add zone-based website management
以稳定 Zone ID 替换旧托管域名详情页:列表展示根域与域名计数,详情提供
概览/域名/路由/证书/设置 Tabs,并补齐 ZoneService 与 vitest 行为测试。
Zone 列表 API 返回 domain_count;同步 Swagger 与重构计划进度。
2026-07-12 15:23:43 +08:00
ryan d4d9bad74d refactor(config): render routes from zone domains 2026-07-12 15:03:24 +08:00
ryan d0536fcdd5 refactor(proxy): bind routes through zone domains 2026-07-12 14:52:19 +08:00
ryan 53c868e99b feat(zone): add zone management api and legacy importer 2026-07-12 14:35:53 +08:00
ryan 50678756d4 feat(zone): add normalized zone domain schema 2026-07-12 14:23:25 +08:00
ryan da1dd92404 fix(observability): merge rollup+raw hourly trends and backfill history
Prefer capacity/openresty rollups only when they cover the 24h window;
otherwise merge per hour so raw fills pre-MV gaps and rollup wins on
overlap. Add a one-time ANTI JOIN backfill migration for the last 30 days.
2026-07-10 11:27:51 +08:00
ryan 4b11279662 fix(observability): fall back to raw hourly when rollup is incomplete
Materialized capacity/openresty hourly tables only hold data after the MV
exists. Preferring any non-empty rollup hid full raw history and left 24h
charts with only recent hours. Use rollup only when its earliest bucket
covers the query window start.
2026-07-10 11:27:51 +08:00
ryan b4b93ff4ed fix(docker): make ClickHouse startable on 25.x and reachable from host
Lower merge-tree free-entry thresholds for small background pools, bind
listen_host to 0.0.0.0 for published ports, and allow CLICKHOUSE_ENABLED=true
in tests for live_ch smoke coverage.
2026-07-10 10:44:49 +08:00
ryan 160e63558f fix(clickhouse): harden R/W path P0–P3 (cleanup, durability, rollups)
Honest TTL cleanup semantics; enqueue-safe dedup with flush retry and writer
metrics; model insert hooks; latest-per-node and hourly metric/openresty
rollups; small-host pool/async defaults, traffic hourly TTL, and UV labeling.
2026-07-10 10:34:04 +08:00
ryan 9b3555c569 fix(clickhouse): cut idle CPU from tiny parts and oversized merge pools
Observability writers flushed every few seconds with MinBatchSize unset,
creating constant small parts and merge load. Enable MinBatchSize with
MaxFlushWait, batch access logs more aggressively, and shrink ClickHouse
background pools for 3c hosts.
2026-07-10 10:08:32 +08:00
ryan b928928958 fix(observability): restore 24h capacity/network/disk trends via CH hourly agg
Node and dashboard 24h capacity, network, and disk IO charts only used the
latest limited raw snapshots (120/500 rows), so historical hour buckets stayed
empty. Prefer ClickHouse hourly aggregates with counter deltas, and fall back
to raw snapshots when aggregation is unavailable.
2026-07-10 09:48:45 +08:00
ryan b312460ddf chore(release): v3.1.1
### ⚡️ 优化与改进
- 将 cap_login_enabled 默认值由 true 变更为 false,默认关闭登录界面 PoW 人机验证。
2026-07-06 12:29:53 +08:00
ryan 44bba0f19a fix(clickhouse): remove unsupported MODIFY ORDER BY from migration
ClickHouse keeps the implicit PRIMARY KEY when shortening ORDER BY,
which fails with "Primary key must be a prefix of the sorting key".
TTL-only changes are safe and unblock goose startup; narrowing ORDER BY
would require table recreation.
2026-07-02 16:45:48 +08:00
ryan f0eca028f9 fix(clickhouse): cast DateTime64 to DateTime in TTL migration 2026-07-02 16:21:30 +08:00
ryan 58624db397 perf(clickhouse): Phase 2 legacy governance — TTL cleanup, unified pool, MV, ops API
- Replace retention ALTER DELETE with MATERIALIZE TTL; use TRUNCATE for delete-all
- Remove GORM ClickHouse pool; migrate user access log reads to ChConn
- Drop query-side trim(remote_addr); enable wait_for_async_insert=1
- Add of_node_traffic_hourly MV and dashboard traffic trend fallback
- Add GET /admin/status/clickhouse operational metrics endpoint
2026-07-02 15:47:38 +08:00
ryan 38946d1af5 fix(clickhouse): resolve lint issues from optimization stack 2026-07-02 15:28:49 +08:00
ryan caf2ffcff4 perf(clickhouse): P1 TTL migrations, ORDER BY tune, remote_addr normalization 2026-07-02 15:25:03 +08:00
ryan 0e86fe3547 perf(clickhouse): P2 docker server tuning and audit log payload reduction 2026-07-02 15:23:17 +08:00
ryan 6525bef15d perf(clickhouse): P0/P1 access log and WAF query aggregation and SQL pagination 2026-07-02 15:23:17 +08:00
ryan 3e910f1961 perf(clickhouse): P0 dashboard/observability query limits, cache, slower polling 2026-07-02 15:23:17 +08:00
ryan 28c14eb054 perf(clickhouse): enable async_insert and tune connection/buffer defaults 2026-07-02 15:23:17 +08:00
ryan ae618905a3 perf(clickhouse): P0 write path — remove heartbeat DELETE, batchwriter MinBatchSize, tune chwriter 2026-07-02 15:23:17 +08:00
ryan 34225cb88a fix(updater): resolve release asset name matching for openflare-server
- Update expectedAssetNames helper to match lowercase repoName prefix and lowercase repoName with -server suffix (e.g. openflare-server).
- Fixes 'no compatible release found' error when checking GitHub Action releases.
2026-06-30 20:47:32 +08:00
ryan 23be2f9296 fix(db): 新增 PostgreSQL 数据库自增序列全局同步迁移脚本
为了解决因历史数据以显式 ID 方式迁移导致 PostgreSQL 自增序列计数器不同步,产生主键冲突唯一性约束报错(如 WAF 规则组和 IP 组保存失败)的问题,在 PostgreSQL 迁移中加入了对所有相关表 pg_get_serial_sequence 重置的代码。同时,在 SQLite 中补齐了对应的同名迁移文件。
2026-06-30 16:13:37 +08:00
ryan c02b649b46 fix(cmd): register all app modes as subcommands in cobra
Resolve unknown command error when launching all/api/worker/scheduler modes due to Cobra strict subcommand validation triggered by reset-passwd. Subcommands now run database migrations via dynamic PreRun hooks.
2026-06-28 11:37:50 +08:00
ryan 9c7896df50 feat(admin): support user profile editing, password resetting, and email column with search
- Add UpdateUser API and logics supporting nickname, email, admin flag modification, and password reset.
- Relocate user delete button and confirmation Alert into the EditUserModal.
- Optimize admin Switch change to trigger instant API request with rollback support.
- Fix missing email field in edit form initialization by fetching full profile metadata.
- Render email column in users list and support email-based filtering in UserFilterBar.
- Remove hardcoded styles and sizes from Switch components to follow global theme.
2026-06-28 11:22:11 +08:00
ryan 01ebec6dfb feat(cmd): add reset-passwd command to reset user password
- Added ./wavelet reset-passwd subcommand to reset user passwords via CLI
- Supported --user flag; if not specified, prompts for username interactively
- Supported --password flag; if not specified, generates a secure random password
- Handled access token deletion and cache invalidation
- Added comprehensive unit tests
2026-06-28 10:50:44 +08:00
ryan 8d8814b416 refactor(oauth): replace legacy oauth cache with standard ram cache and add pubsub synchronization
- Replaced custom map-based cache in apps/oauth/cache.go with standard pkg/cache/ram framework.
- Implemented Redis Pub/Sub invalidation channels for distributed token and user cache synchronization.
- Created apps/oauth/cache_test.go to verify local cache operations and pub/sub broadcasts.

refactor(cache): generic RAM cache with CoW and unified preheating

Replaced L2 Redis cache and old cache package with process-local generic pkg/cache/ram. Implemented Copy-on-Write for reads, fine-grained locks per type for writes, and unified preheating in bootstrap. Changed cache invalidation to lazy-loading to resolve SQLite deadlocks during transactions.
2026-06-27 14:32:27 +08:00
ryan 02ebb81929 refactor(oauth): replace legacy oauth cache with standard ram cache and add pubsub synchronization
- Replaced custom map-based cache in apps/oauth/cache.go with standard pkg/cache/ram framework.
- Implemented Redis Pub/Sub invalidation channels for distributed token and user cache synchronization.
- Created apps/oauth/cache_test.go to verify local cache operations and pub/sub broadcasts.

refactor(cache): generic RAM cache with CoW and unified preheating

Replaced L2 Redis cache and old cache package with process-local generic pkg/cache/ram. Implemented Copy-on-Write for reads, fine-grained locks per type for writes, and unified preheating in bootstrap. Changed cache invalidation to lazy-loading to resolve SQLite deadlocks during transactions.
2026-06-27 14:26:13 +08:00
ryan 60222acf7e refactor(db): use version as primary key for ConfigVersion and reuse model layer
- Transition `of_config_versions` primary key from `id` to `version` string.
- Add database migration files for PostgreSQL and SQLite.
- Introduce GORM hooks to preserve JSON backward compatibility.
- Remove all localized private structures (`configVersionRecord`, `configVersionRow`) across `agent` and `flared` modules.
- Remove local database Row structures (`tlsCertificateRow`, `tunnelNodeRow`, `pagesProjectRow`) in `proxy_route` module.
- Reuse `model` query methods directly to fetch active config, tunnel nodes, and pages projects.
- Cache IP detection results in memory with a 10-minute TTL to prevent frequent HTTP egress queries to realip.cc.
- Integrate multiple fallback IP lookup providers (ifconfig.me, ip.sb, icanhazip.com) to guarantee IP detection reliability.
2026-06-27 14:00:55 +08:00