Commit Graph

11 Commits

Author SHA1 Message Date
ryan 160e63558f fix(clickhouse): harden R/W path P0–P3 (cleanup, durability, rollups)
Honest TTL cleanup semantics; enqueue-safe dedup with flush retry and writer
metrics; model insert hooks; latest-per-node and hourly metric/openresty
rollups; small-host pool/async defaults, traffic hourly TTL, and UV labeling.
2026-07-10 10:34:04 +08:00
ryan b928928958 fix(observability): restore 24h capacity/network/disk trends via CH hourly agg
Node and dashboard 24h capacity, network, and disk IO charts only used the
latest limited raw snapshots (120/500 rows), so historical hour buckets stayed
empty. Prefer ClickHouse hourly aggregates with counter deltas, and fall back
to raw snapshots when aggregation is unavailable.
2026-07-10 09:48:45 +08:00
ryan 58624db397 perf(clickhouse): Phase 2 legacy governance — TTL cleanup, unified pool, MV, ops API
- Replace retention ALTER DELETE with MATERIALIZE TTL; use TRUNCATE for delete-all
- Remove GORM ClickHouse pool; migrate user access log reads to ChConn
- Drop query-side trim(remote_addr); enable wait_for_async_insert=1
- Add of_node_traffic_hourly MV and dashboard traffic trend fallback
- Add GET /admin/status/clickhouse operational metrics endpoint
2026-07-02 15:47:38 +08:00
ryan 38946d1af5 fix(clickhouse): resolve lint issues from optimization stack 2026-07-02 15:28:49 +08:00
ryan caf2ffcff4 perf(clickhouse): P1 TTL migrations, ORDER BY tune, remote_addr normalization 2026-07-02 15:25:03 +08:00
ryan 6525bef15d perf(clickhouse): P0/P1 access log and WAF query aggregation and SQL pagination 2026-07-02 15:23:17 +08:00
ryan 3187934f72 fix(openflare): ClickHouse count scan and dashboard traffic metrics
- Scan ClickHouse count()/countIf() aggregates as uint64 before int64 conversion
- Fix access log list count, region stats, and delete pre-count queries
- Aggregate dashboard 24h traffic totals from hourly trend buckets
- Show current hour value in traffic trend chart summary
- Rename docker-compose service to openflare
- Remove redundant config preview section from performance page
2026-06-19 17:34:19 +08:00
ryan 9491b2a744 feat(clickhouse): wire batchwriter into business ingestion paths
Migrate risk_control audit logs to internal/db/batchwriter and add
openflare/chwriter with per-table async flush for observability
timeseries and node access logs.

Replace single-row ClickHouse inserts and pre-insert SELECT count()
dedup with repository BatchInsert* APIs, in-process TTL dedup for
metric/report snapshots, and bootstrap initialization on API startup.
2026-06-19 17:20:39 +08:00
ryan 578110f615 observation clickhouse 2026-06-19 17:20:39 +08:00
ryan 32861c5db9 fix lint 2026-06-19 17:20:39 +08:00
ryan 63cd906cfc refactor(repo): consolidate openflare-server to root and move subprojects to internal/apps
- Merge all files inside openflare-server to the repository root directory.
- Relocate agent, relay, and flared subprojects from internal/ to internal/apps/.
- Combine docker-compose files and update build context paths to root.
- Update GitHub workflows and Dockerfiles to refer to new directories and package names.
- Rewrite Go package imports across all files.
- Resolve database renew test race condition and clean up docs.
2026-06-19 14:23:29 +08:00