mirror of
https://github.com/Rain-kl/OpenFlare.git
synced 2026-10-05 23:26:38 +08:00
feat(log): decouple log storage from ClickHouse with switchable logstore
- New internal/repository/logstore abstraction: exported domain interfaces (AccessLogStore/ObservabilityStore/UserAccessLogStore/StatusStore), config-driven provider (Active/Build/Migrating/SetConfigReader), GORM implementation for PostgreSQL/SQLite (incl. hourly rollups computed in real time, migration listers, PG partition maintenance), and a ClickHouse wrapper preserving the native batch path; repository facade delegates to logstore; import-lint test enforces apps never import analyticsrepo. - ClickHouse is now optional: the log DB is either the main DB (postgres when database.enabled, else sqlite) or clickhouse; boot validation + first-run seed; log_database / log_db_migration are protected keys. - New user task 切换日志数据库 (of_log_db_switch): freeze log writes, drain batch writers, copy all 6 raw log tables by id (preserving IDs) with target-partition pre-creation for PG, flip log_database on success, clear the freeze flag on failure. - Per-store retention (log_retention_days_*) with expiry cleanup folded into the daily system_cleanup task; legacy database_auto_cleanup_* and of_database_auto_cleanup decommissioned. - goose migrations: 6 log tables in PG (2 monthly-partitioned) + SQLite, retention config seeds, schedule cleanup; GET /api/v1/admin/status/log-database endpoint; frontend retention settings, switch-task UI and status badge; changelog and docs updated. docs(plan): log database decoupling implementation plan docs(design): log database decoupling design (ClickHouse optional)
This commit is contained in:
@@ -20,6 +20,18 @@ sidebar: false
|
||||
> - Pages 无法迁移, 升级前请先手动下载并备份 Pages 静态站点的 ZIP 包,升级后重新创建。
|
||||
> - 性能调优参数重置, 升级后请重新配置
|
||||
|
||||
## [Unreleased]
|
||||
|
||||
### 新增
|
||||
|
||||
- 日志存储解耦:新增日志存储抽象(`internal/repository/logstore`),ClickHouse 变为可选项,不启用时由 PostgreSQL/SQLite 承担全部日志功能;新增「切换日志数据库」任务支持 PostgreSQL/SQLite 与 ClickHouse 间数据迁移(迁移期间冻结日志写入,成功后自动切换主库并保留源数据);日志保留时间改为按存储库在业务配置中设置(`log_retention_days_*`),过期清理并入系统垃圾清理每日任务。
|
||||
|
||||
### 修复
|
||||
|
||||
- 「切换日志数据库」迁移任务补齐第 6 张表(用户访问日志)的复制与清空;目标为 PostgreSQL 时按源库时间范围预建分区,历史日志可正常迁移;冻结前先排空批写入队列,避免在途日志丢失。
|
||||
- 节点监控最新指标读取仅在 ClickHouse 为当前日志库时走 ClickHouse 快速路径,日志库切换后仪表盘不再读到旧库数据。
|
||||
- `log_database` / `log_db_migration` 为受保护配置,管理端创建/修改接口均拒绝手动写入。
|
||||
|
||||
## [v3.4.5] - 2026-08-08
|
||||
|
||||
### 改进
|
||||
|
||||
Reference in New Issue
Block a user