Commit Graph

5 Commits

Author SHA1 Message Date
ryan 160e63558f fix(clickhouse): harden R/W path P0–P3 (cleanup, durability, rollups)
Honest TTL cleanup semantics; enqueue-safe dedup with flush retry and writer
metrics; model insert hooks; latest-per-node and hourly metric/openresty
rollups; small-host pool/async defaults, traffic hourly TTL, and UV labeling.
2026-07-10 10:34:04 +08:00
ryan 9b3555c569 fix(clickhouse): cut idle CPU from tiny parts and oversized merge pools
Observability writers flushed every few seconds with MinBatchSize unset,
creating constant small parts and merge load. Enable MinBatchSize with
MaxFlushWait, batch access logs more aggressively, and shrink ClickHouse
background pools for 3c hosts.
2026-07-10 10:08:32 +08:00
ryan ae618905a3 perf(clickhouse): P0 write path — remove heartbeat DELETE, batchwriter MinBatchSize, tune chwriter 2026-07-02 15:23:17 +08:00
ryan 8c872f9b32 refactor(bootstrap): 引入解耦的全局生命周期管理器以隔离业务停机钩子
- 新建 internal/lifecycle 包,提供全局线程安全的 Shutdown 钩子注册与调度能力。
- 在 chwriter 与 risk_control 初始化阶段通过 OnShutdown 将其 Stop 函数注册到管理器中。
- bootstrap.Stop() 函数仅委托调用 lifecycle.Stop(),不再硬编码引入业务包,防止后续框架同步产生合并冲突。
2026-06-20 09:58:59 +08:00
ryan 9491b2a744 feat(clickhouse): wire batchwriter into business ingestion paths
Migrate risk_control audit logs to internal/db/batchwriter and add
openflare/chwriter with per-table async flush for observability
timeseries and node access logs.

Replace single-row ClickHouse inserts and pre-insert SELECT count()
dedup with repository BatchInsert* APIs, in-process TTL dedup for
metric/report snapshots, and bootstrap initialization on API startup.
2026-06-19 17:20:39 +08:00