tcjlj/OpenFlare
1
0
Fork 0
mirror of https://github.com/Rain-kl/OpenFlare.git synced 2026-10-04 07:06:36 +08:00
Code Issues Packages Projects Releases Wiki Activity

feat(log): decouple log storage from ClickHouse with switchable logstore

Browse Source 
- New internal/repository/logstore abstraction: exported domain interfaces
  (AccessLogStore/ObservabilityStore/UserAccessLogStore/StatusStore),
  config-driven provider (Active/Build/Migrating/SetConfigReader), GORM
  implementation for PostgreSQL/SQLite (incl. hourly rollups computed in
  real time, migration listers, PG partition maintenance), and a ClickHouse
  wrapper preserving the native batch path; repository facade delegates to
  logstore; import-lint test enforces apps never import analyticsrepo.
- ClickHouse is now optional: the log DB is either the main DB (postgres
  when database.enabled, else sqlite) or clickhouse; boot validation +
  first-run seed; log_database / log_db_migration are protected keys.
- New user task 切换日志数据库 (of_log_db_switch): freeze log writes,
  drain batch writers, copy all 6 raw log tables by id (preserving IDs)
  with target-partition pre-creation for PG, flip log_database on success,
  clear the freeze flag on failure.
- Per-store retention (log_retention_days_*) with expiry cleanup folded
  into the daily system_cleanup task; legacy database_auto_cleanup_* and
  of_database_auto_cleanup decommissioned.
- goose migrations: 6 log tables in PG (2 monthly-partitioned) + SQLite,
  retention config seeds, schedule cleanup; GET
  /api/v1/admin/status/log-database endpoint; frontend retention settings,
  switch-task UI and status badge; changelog and docs updated.

docs(plan): log database decoupling implementation plan

docs(design): log database decoupling design (ClickHouse optional)
This commit is contained in:
ryan
2026-08-08 11:36:56 +08:00
parent 734fe45baa
commit 7d71f1e4e1
95 changed files with 10569 additions and 3728 deletions
Download Patch File Download Diff File Expand all files Collapse all files
internal/infra/task/handlers/register.go
+4 -3
View File
@@ -10,6 +10,7 @@ import (
"github.com/Rain-kl/Wavelet/internal/apps/openflare"
cf "github.com/Rain-kl/Wavelet/internal/apps/openflare/cloudflare"
"github.com/Rain-kl/Wavelet/internal/apps/openflare/pages"
"github.com/Rain-kl/Wavelet/internal/apps/openflare/tasks"
"github.com/Rain-kl/Wavelet/internal/apps/openflare/tls"
"github.com/Rain-kl/Wavelet/internal/apps/upload"
"github.com/Rain-kl/Wavelet/internal/apps/user"
@@ -44,15 +45,15 @@ func Register() {
task.RegisterHandler(openflare.SSLRenewTask, &openflare.SSLRenewHandler{})
task.RegisterTaskMeta(openflare.SSLRenewMeta)
task.RegisterHandler(openflare.DatabaseAutoCleanupTask, &openflare.DatabaseAutoCleanupHandler{})
task.RegisterTaskMeta(openflare.DatabaseAutoCleanupMeta)
task.RegisterHandler(openflare.WAFIPGroupSyncTask, &openflare.WAFIPGroupSyncHandler{})
task.RegisterTaskMeta(openflare.WAFIPGroupSyncMeta)
task.RegisterHandler(openflare.UptimeKumaSyncTask, &openflare.UptimeKumaSyncHandler{})
task.RegisterTaskMeta(openflare.UptimeKumaSyncMeta)
task.RegisterHandler(openflare.LogDBSwitchTask, &tasks.LogDBSwitchHandler{})
task.RegisterTaskMeta(openflare.LogDBSwitchMeta)
task.RegisterHandler(cf.SyncMemberTask, &cf.SyncMemberTaskHandler{})
task.RegisterTaskMeta(cf.SyncMemberMeta)
task.RegisterHandler(cf.SyncGroupTask, &cf.SyncGroupTaskHandler{})