flvx/docs/superpowers/plans/2026-04-28-monitoring-retention-storage.md

Monitoring Retention And Storage Display Implementation Plan

For agentic workers: REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (- [ ]) syntax for tracking.

Goal: Add configurable monitoring data retention and show database storage usage on the config page.

Architecture: Store retention in vite_config as monitor_retention_days, parse it through a focused monitoring helper, and reuse it from existing cleanup loops. Add a repository storage-summary helper, expose it via an admin-only API, and render it in the existing React config page.

Tech Stack: Go net/http, GORM, SQLite/PostgreSQL, Vite/React/TypeScript, existing shadcn bridge components.

---

File Structure

• Create: go-backend/internal/monitoring/retention.go for retention constants, parsing, and validation.
• Test: go-backend/internal/monitoring/retention_test.go.
• Modify: go-backend/internal/metrics/ingestion.go and go-backend/internal/metrics/ingestion_test.go for config-driven cleanup.
• Modify: go-backend/internal/http/handler/tunnel_quality_prober.go so tunnel_quality uses the same retention and still prunes when probing is disabled.
• Create: go-backend/internal/store/repo/repository_storage.go and go-backend/internal/store/repo/repository_storage_test.go for database size summaries.
• Modify: go-backend/internal/store/repo/repository.go to keep the SQLite DB path on Repository.
• Create: go-backend/internal/http/handler/storage.go for the storage endpoint.
• Modify: go-backend/internal/http/handler/handler.go to register /api/v1/system/storage and validate monitor_retention_days.
• Modify: go-backend/internal/http/middleware/auth.go so /api/v1/system/* is admin-only.
• Create: go-backend/tests/contract/storage_contract_test.go for endpoint auth/shape coverage.
• Modify: vite-frontend/src/api/types.ts, vite-frontend/src/api/index.ts, and vite-frontend/src/pages/config.tsx for UI display.

Implementation should not create git commits unless the user explicitly requests them.

---

Task 1: Add Retention Config Helper

Files:

• Create: go-backend/internal/monitoring/retention.go

• Create: go-backend/internal/monitoring/retention_test.go

• Modify: go-backend/internal/http/handler/handler.go

• Step 1: Write the failing tests

Create go-backend/internal/monitoring/retention_test.go:

package monitoring

import "testing"

func TestMonitoringRetentionDaysFromConfigMap(t *testing.T) {
	tests := []struct {
		name string
		cfg  map[string]string
		want int
	}{
		{"missing uses default", nil, 7},
		{"valid custom", map[string]string{ConfigMonitorRetentionDays: "3"}, 3},
		{"trimmed custom", map[string]string{ConfigMonitorRetentionDays: " 30 "}, 30},
		{"invalid uses default", map[string]string{ConfigMonitorRetentionDays: "abc"}, 7},
		{"too small uses default", map[string]string{ConfigMonitorRetentionDays: "0"}, 7},
		{"too large uses default", map[string]string{ConfigMonitorRetentionDays: "3651"}, 7},
	}
	for _, tc := range tests {
		t.Run(tc.name, func(t *testing.T) {
			if got := MonitoringRetentionDaysFromConfigMap(tc.cfg); got != tc.want {
				t.Fatalf("expected %d, got %d", tc.want, got)
			}
		})
	}
}

func TestNormalizeMonitoringRetentionDays(t *testing.T) {
	for _, value := range []string{"1", "7", "3650", " 30 "} {
		if got, err := NormalizeMonitoringRetentionDays(value); err != nil || got == "" {
			t.Fatalf("expected %q valid, got value=%q err=%v", value, got, err)
		}
	}
	for _, value := range []string{"", "0", "-1", "3651", "abc", "1.5"} {
		if got, err := NormalizeMonitoringRetentionDays(value); err == nil {
			t.Fatalf("expected %q invalid, got value=%q", value, got)
		}
	}
}

• Step 2: Run tests to verify failure

Run: go test ./internal/monitoring -run 'TestMonitoringRetentionDaysFromConfigMap|TestNormalizeMonitoringRetentionDays' -count=1

Expected: FAIL with undefined ConfigMonitorRetentionDays, MonitoringRetentionDaysFromConfigMap, and NormalizeMonitoringRetentionDays.

• Step 3: Implement helper

Create go-backend/internal/monitoring/retention.go:

package monitoring

import (
	"fmt"
	"strconv"
	"strings"
)

const (
	ConfigMonitorRetentionDays = "monitor_retention_days"
	DefaultMonitorRetentionDays = 7
	MinMonitorRetentionDays     = 1
	MaxMonitorRetentionDays     = 3650
)

func MonitoringRetentionDaysFromConfigMap(cfg map[string]string) int {
	if cfg == nil {
		return DefaultMonitorRetentionDays
	}
	days, err := parseMonitoringRetentionDays(cfg[ConfigMonitorRetentionDays])
	if err != nil {
		return DefaultMonitorRetentionDays
	}
	return days
}

func NormalizeMonitoringRetentionDays(value string) (string, error) {
	days, err := parseMonitoringRetentionDays(value)
	if err != nil {
		return "", err
	}
	return strconv.Itoa(days), nil
}

func parseMonitoringRetentionDays(value string) (int, error) {
	trimmed := strings.TrimSpace(value)
	if trimmed == "" {
		return 0, fmt.Errorf("监控数据保留天数不能为空")
	}
	days, err := strconv.Atoi(trimmed)
	if err != nil {
		return 0, fmt.Errorf("监控数据保留天数必须是整数")
	}
	if days < MinMonitorRetentionDays || days > MaxMonitorRetentionDays {
		return 0, fmt.Errorf("监控数据保留天数必须在 %d 到 %d 之间", MinMonitorRetentionDays, MaxMonitorRetentionDays)
	}
	return days, nil
}

• Step 4: Validate config updates

In go-backend/internal/http/handler/handler.go, add this case to normalizeAndValidateConfigValue:

	case monitoring.ConfigMonitorRetentionDays:
		return monitoring.NormalizeMonitoringRetentionDays(value)

• Step 5: Run tests

Run: go test ./internal/monitoring ./internal/http/handler -run 'TestMonitoringRetention|TestNormalize|Test' -count=1

Expected: PASS or only unrelated pre-existing failures, which must be investigated before continuing.

---

Task 2: Use Retention Config In Cleanup

Files:

• Modify: go-backend/internal/metrics/ingestion.go

• Modify: go-backend/internal/metrics/ingestion_test.go

• Modify: go-backend/internal/http/handler/tunnel_quality_prober.go

• Step 1: Write failing cleanup test

Append to go-backend/internal/metrics/ingestion_test.go, adding go-backend/internal/store/model to imports:

func TestPruneMetricsUsesConfiguredRetentionDays(t *testing.T) {
	r, err := repo.Open(":memory:")
	if err != nil {
		t.Fatalf("open repo: %v", err)
	}
	defer r.Close()

	now := time.Now().UnixMilli()
	if err := r.UpsertConfig("monitor_retention_days", "2", now); err != nil {
		t.Fatalf("upsert retention config: %v", err)
	}

	oldMetric := &model.NodeMetric{NodeID: 1, Timestamp: now - int64(3*24*time.Hour/time.Millisecond), CPUUsage: 10}
	newMetric := &model.NodeMetric{NodeID: 1, Timestamp: now - int64(1*24*time.Hour/time.Millisecond), CPUUsage: 20}
	if err := r.InsertNodeMetric(oldMetric); err != nil {
		t.Fatalf("insert old metric: %v", err)
	}
	if err := r.InsertNodeMetric(newMetric); err != nil {
		t.Fatalf("insert new metric: %v", err)
	}

	svc := NewIngestionService(r)
	svc.pruneMetricsAt(time.UnixMilli(now))

	metrics, err := r.GetNodeMetrics(1, now-int64(4*24*time.Hour/time.Millisecond), now+1000)
	if err != nil {
		t.Fatalf("get node metrics: %v", err)
	}
	if len(metrics) != 1 || metrics[0].CPUUsage != 20 {
		t.Fatalf("expected only newer metric to remain, got %#v", metrics)
	}
}

• Step 2: Run test to verify failure

Run: go test ./internal/metrics -run TestPruneMetricsUsesConfiguredRetentionDays -count=1

Expected: FAIL with undefined pruneMetricsAt.

• Step 3: Implement config-driven prune

In go-backend/internal/metrics/ingestion.go, import go-backend/internal/monitoring and replace pruneMetrics with:

func (s *IngestionService) retentionDaysFromConfig() int {
	if s == nil || s.repo == nil {
		return monitoring.DefaultMonitorRetentionDays
	}
	cfg, err := s.repo.GetConfigsByNames([]string{monitoring.ConfigMonitorRetentionDays})
	if err != nil {
		return monitoring.DefaultMonitorRetentionDays
	}
	return monitoring.MonitoringRetentionDaysFromConfigMap(cfg)
}

func (s *IngestionService) pruneMetrics() {
	s.pruneMetricsAt(time.Now())
}

func (s *IngestionService) pruneMetricsAt(now time.Time) {
	cutoff := now.Add(-time.Duration(s.retentionDaysFromConfig()) * 24 * time.Hour).UnixMilli()
	if s.repo == nil {
		return
	}
	if err := s.repo.PruneNodeMetrics(cutoff); err != nil {
		log.Printf("monitoring prune failed op=node_metric cutoff=%d err=%v", cutoff, err)
	}
	if err := s.repo.PruneTunnelMetrics(cutoff); err != nil {
		log.Printf("monitoring prune failed op=tunnel_metric cutoff=%d err=%v", cutoff, err)
	}
	if err := s.repo.PruneServiceMonitorResults(cutoff); err != nil {
		log.Printf("monitoring prune failed op=service_monitor_result cutoff=%d err=%v", cutoff, err)
	}
}

Remove the unused retentionDays field from IngestionService and remove svc.retentionDays = 1 from existing tests.

• Step 4: Update tunnel quality pruning

In go-backend/internal/http/handler/tunnel_quality_prober.go, import go-backend/internal/monitoring, remove tunnelQualityRetention, remove the if !p.isEnabled() { return } guard from maybePrune, and calculate cutoff with:

func (p *tunnelQualityProber) retentionDays() int {
	if p == nil || p.handler == nil || p.handler.repo == nil {
		return monitoring.DefaultMonitorRetentionDays
	}
	cfg, err := p.handler.repo.GetConfigsByNames([]string{monitoring.ConfigMonitorRetentionDays})
	if err != nil {
		return monitoring.DefaultMonitorRetentionDays
	}
	return monitoring.MonitoringRetentionDaysFromConfigMap(cfg)
}

Then use:

	cutoff := now - int64(time.Duration(p.retentionDays())*24*time.Hour/time.Millisecond)

• Step 5: Run cleanup tests

Run: go test ./internal/metrics ./internal/http/handler -run 'TestPruneMetrics|TestPruneMetricsUsesConfiguredRetentionDays|TunnelQuality' -count=1

Expected: PASS.

---

Task 3: Add Storage Summary Backend API

Files:

• Modify: go-backend/internal/store/repo/repository.go

• Create: go-backend/internal/store/repo/repository_storage.go

• Create: go-backend/internal/store/repo/repository_storage_test.go

• Create: go-backend/internal/http/handler/storage.go

• Modify: go-backend/internal/http/handler/handler.go

• Modify: go-backend/internal/http/middleware/auth.go

• Create: go-backend/tests/contract/storage_contract_test.go

• Step 1: Write failing repository tests

Create go-backend/internal/store/repo/repository_storage_test.go:

package repo

import (
	"path/filepath"
	"testing"

	"go-backend/internal/store/model"
)

func TestDatabaseStorageSummarySQLiteIncludesSize(t *testing.T) {
	dbPath := filepath.Join(t.TempDir(), "storage.db")
	r, err := Open(dbPath)
	if err != nil {
		t.Fatalf("open repo: %v", err)
	}
	defer r.Close()

	if err := r.InsertNodeMetric(&model.NodeMetric{NodeID: 1, Timestamp: 123, CPUUsage: 1}); err != nil {
		t.Fatalf("insert metric: %v", err)
	}
	summary, err := r.DatabaseStorageSummary()
	if err != nil {
		t.Fatalf("storage summary: %v", err)
	}
	if summary.DBType != "sqlite" || summary.DatabaseSizeBytes <= 0 || summary.DatabaseSizeText == "" {
		t.Fatalf("unexpected summary: %#v", summary)
	}
}

func TestFormatDatabaseSize(t *testing.T) {
	for _, tc := range []struct{ bytes int64; want string }{{0, "0 B"}, {512, "512 B"}, {1024, "1.0 KB"}, {1024 * 1024, "1.0 MB"}} {
		if got := formatDatabaseSize(tc.bytes); got != tc.want {
			t.Fatalf("formatDatabaseSize(%d)=%q want %q", tc.bytes, got, tc.want)
		}
	}
}

• Step 2: Run test to verify failure

Run: go test ./internal/store/repo -run 'TestDatabaseStorageSummarySQLiteIncludesSize|TestFormatDatabaseSize' -count=1

Expected: FAIL with undefined DatabaseStorageSummary and formatDatabaseSize.

• Step 3: Implement repository helper

Modify Repository in repository.go:

type Repository struct {
	db     *gorm.DB
	dbPath string
}

Return &Repository{db: db, dbPath: path} from Open and &Repository{db: db} from OpenPostgres.

Create go-backend/internal/store/repo/repository_storage.go:

package repo

import (
	"errors"
	"fmt"
	"os"
)

type DatabaseStorageSummary struct {
	DBType            string `json:"dbType"`
	DatabaseSizeBytes int64  `json:"databaseSizeBytes"`
	DatabaseSizeText  string `json:"databaseSizeText"`
}

func (r *Repository) DatabaseStorageSummary() (DatabaseStorageSummary, error) {
	if r == nil || r.db == nil {
		return DatabaseStorageSummary{}, errors.New("repository not initialized")
	}
	switch r.db.Dialector.Name() {
	case "sqlite":
		size, err := sqliteDatabaseFileSize(r.dbPath)
		if err != nil { return DatabaseStorageSummary{}, err }
		return DatabaseStorageSummary{"sqlite", size, formatDatabaseSize(size)}, nil
	case "postgres":
		var size int64
		if err := r.db.Raw("SELECT pg_database_size(current_database())").Scan(&size).Error; err != nil { return DatabaseStorageSummary{}, err }
		return DatabaseStorageSummary{"postgres", size, formatDatabaseSize(size)}, nil
	default:
		return DatabaseStorageSummary{}, fmt.Errorf("unsupported database dialect %q", r.db.Dialector.Name())
	}
}

func sqliteDatabaseFileSize(path string) (int64, error) {
	if path == "" || path == ":memory:" { return 0, nil }
	var total int64
	for _, candidate := range []string{path, path + "-wal", path + "-shm"} {
		info, err := os.Stat(candidate)
		if err != nil {
			if os.IsNotExist(err) { continue }
			return 0, err
		}
		if !info.IsDir() { total += info.Size() }
	}
	return total, nil
}

func formatDatabaseSize(bytes int64) string {
	if bytes < 1024 { return fmt.Sprintf("%d B", bytes) }
	units := []string{"KB", "MB", "GB", "TB"}
	value := float64(bytes) / 1024
	for _, unit := range units {
		if value < 1024 || unit == "TB" { return fmt.Sprintf("%.1f %s", value, unit) }
		value /= 1024
	}
	return fmt.Sprintf("%d B", bytes)
}

• Step 4: Add API handler and route

Create go-backend/internal/http/handler/storage.go:

package handler

import (
	"net/http"

	"go-backend/internal/http/response"
)

func (h *Handler) storageSummary(w http.ResponseWriter, r *http.Request) {
	if r.Method != http.MethodGet && r.Method != http.MethodPost {
		response.WriteJSON(w, response.ErrDefault("请求失败"))
		return
	}
	if h == nil || h.repo == nil {
		response.WriteJSON(w, response.Err(-2, "repository not initialized"))
		return
	}
	summary, err := h.repo.DatabaseStorageSummary()
	if err != nil {
		response.WriteJSON(w, response.Err(-2, err.Error()))
		return
	}
	response.WriteJSON(w, response.OK(summary))
}

Register in Handler.Register: mux.HandleFunc("/api/v1/system/storage", h.storageSummary).

In requiresAdmin, add:

	if strings.HasPrefix(path, "/api/v1/system/") {
		return true
	}

• Step 5: Write contract test for auth and shape

Create go-backend/tests/contract/storage_contract_test.go with a test that sends GET /api/v1/system/storage as non-admin and expects 403, then as admin and expects code == 0, dbType, numeric databaseSizeBytes, and databaseSizeText.

• Step 6: Run storage tests

Run: go test ./internal/store/repo ./tests/contract -run 'TestDatabaseStorageSummarySQLiteIncludesSize|TestFormatDatabaseSize|TestStorageSummaryRequiresAdminAndReturnsSize' -count=1

Expected: PASS.

---

Task 4: Add Frontend Config UI

Files:

• Modify: vite-frontend/src/api/types.ts

• Modify: vite-frontend/src/api/index.ts

• Modify: vite-frontend/src/pages/config.tsx

• Step 1: Add API type and function

In types.ts add:

export interface StorageSummaryApiData {
  dbType: string;
  databaseSizeBytes: number;
  databaseSizeText: string;
}

In index.ts, import StorageSummaryApiData and add:

export const getStorageSummary = () =>
  Network.get<StorageSummaryApiData>("/system/storage");

• Step 2: Add retention config item

In config.tsx, add to CONFIG_ITEMS near monitoring:

  {
    key: "monitor_retention_days",
    label: "监控数据保留天数",
    placeholder: "7",
    description:
      "统一清理节点指标、隧道流量、服务监控结果和隧道质量历史；默认 7 天。",
    type: "input",
  },

Add "monitor_retention_days" to getInitialConfigs() keys.

• Step 3: Fetch and display database size

In config.tsx, add state:

const [storageSummary, setStorageSummary] = useState<string>("加载中...");

Add a load effect:

useEffect(() => {
  let mounted = true;
  getStorageSummary()
    .then((response) => {
      if (!mounted) return;
      if (response.code === 0 && response.data?.databaseSizeText) {
        setStorageSummary(response.data.databaseSizeText);
      } else {
        setStorageSummary("获取失败");
      }
    })
    .catch(() => {
      if (mounted) setStorageSummary("获取失败");
    });
  return () => {
    mounted = false;
  };
}, []);

Render inside the basic settings card before the save button:

<Divider className="my-2" />
<div className="space-y-1">
  <p className="text-sm font-medium text-gray-700 dark:text-gray-300">
    数据库占用
  </p>
  <p className="text-xs text-gray-500 dark:text-gray-400">
    当前后端数据库文件/实例占用空间，仅用于容量参考。
  </p>
  <div className="rounded-lg border border-divider bg-default-50/60 dark:bg-default-100/10 px-4 py-3 text-sm font-semibold text-default-800 dark:text-default-200">
    {storageSummary}
  </div>
</div>

• Step 4: Build frontend

Run: pnpm run build from vite-frontend.

Expected: TypeScript and Vite build pass.

---

Task 5: Final Verification

Files:

• All files changed by previous tasks.

• Step 1: Run backend tests

Run: go test ./... from go-backend.

Expected: PASS.

• Step 2: Run frontend build

Run: pnpm run build from vite-frontend.

Expected: PASS.

• Step 3: Review diff

Run: git diff --stat and git diff -- docs/superpowers/specs/2026-04-28-monitoring-retention-storage-design.md docs/superpowers/plans/2026-04-28-monitoring-retention-storage.md go-backend vite-frontend.

Expected: Diff is limited to retention config, storage summary, tests, and config UI.

---

Self-Review

• Spec coverage: retention config, uniform cleanup, storage summary API, frontend display, validation, and verification are covered.
• Placeholder scan: no TBD/TODO placeholders; the one contract-test step describes exact assertions even though the surrounding helper functions already exist in contract tests.
• Type consistency: backend JSON fields match frontend StorageSummaryApiData exactly: dbType, databaseSizeBytes, databaseSizeText.