feat(updater): throttle auto-update checks with exponential backoff

Check GitHub releases at most every 6 hours and retry transient failures
(release query, checksum or binary download) with exponential backoff
from 1 minute up to 3 hours. Forced updates bypass the throttle.
This commit is contained in:
ryan
2026-11-07 21:49:46 +08:00
parent 8867c822bb
commit b60546a94d
6 changed files with 627 additions and 31 deletions
+1
View File
@@ -24,6 +24,7 @@ sidebar: false
### 优化 ### 优化
- 节点管理页新增服务端分页及名称、节点标识、IP、状态和类型筛选,仅加载当前页的节点与最新下发记录;详情页改为单节点查询,减少大规模节点列表的轮询和渲染开销,并保留旧列表接口兼容现有集成;分页控件支持中英文及俄语,新增注册和列表错误按界面语言展示。搜索保留 Unicode 原始字符匹配,切页等待时保持已加载数据并禁用旧节点操作,避免页面跳动及错误操作。搜索框按接口的 UTF-8 字节上限校验输入,超限时显示可翻译提示并阻止发送无效查询。 - 节点管理页新增服务端分页及名称、节点标识、IP、状态和类型筛选,仅加载当前页的节点与最新下发记录;详情页改为单节点查询,减少大规模节点列表的轮询和渲染开销,并保留旧列表接口兼容现有集成;分页控件支持中英文及俄语,新增注册和列表错误按界面语言展示。搜索保留 Unicode 原始字符匹配,切页等待时保持已加载数据并禁用旧节点操作,避免页面跳动及错误操作。搜索框按接口的 UTF-8 字节上限校验输入,超限时显示可翻译提示并阻止发送无效查询。
- Agent、Relay 与 Flared 的自动更新检查改为每 6 小时一次,查询 GitHub 或下载更新包失败时按 1 分钟起翻倍退避重试(最长 3 小时),避免频繁请求触发 GitHub API 限流;手动触发的立即更新不受限制。
### 🛠 修复 ### 🛠 修复
@@ -0,0 +1,161 @@
# AutoUpdate 6-Hour Interval and Exponential Backoff Implementation Plan
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
**Goal:** Throttle agent/edge auto-update checks to once every 6 hours, and apply exponential backoff on failure (doubling up to a maximum of 3 hours) to prevent exhausting GitHub API rate limits.
**Architecture:** Add check interval and exponential backoff state tracking to `edgeupdater.Service` protected by a mutex, with clock injection (`NowFunc`) for deterministic testing. Expose `CanCheck(options UpdateOptions) bool` and integrate it into both `CheckAndUpdate` and `edgeheartbeat.TryAutoUpdate` so unthrottled checks and noisy logs are eliminated during periodic heartbeat loops.
**Tech Stack:** Go (1.25), standard library (`sync`, `time`, `net/http`, `log/slog`).
## Global Constraints
- Normal update check interval: 6 hours (`6 * time.Hour`).
- Initial backoff on failure: 1 minute (`1 * time.Minute`), doubling on each consecutive failure.
- Maximum retry backoff: 3 hours (`3 * time.Hour`).
- Success resets failure count and backoff, and sets next check to `now + 6h`.
- Force check (`UpdateOptions.Force = true`) must bypass backoff and interval immediately.
- TDD required: write failing tests first, watch them fail, implement minimal code, verify pass.
---
### Task 1: Add Interval and Exponential Backoff to `edgeupdater.Service`
**Files:**
- Modify: `internal/apps/edge/updater/service.go`
- Test: `internal/apps/edge/updater/service_test.go`
**Interfaces:**
- Consumes: `UpdateOptions{Channel, TagName, Force}`
- Produces: `(s *Service) CanCheck(options UpdateOptions) bool`, `(s *Service) CheckAndUpdate(...) error` with throttling and exponential backoff
- [x] **Step 1: Write the failing tests in `service_test.go`**
Add tests covering:
- `TestCheckAndUpdateIntervalAndBackoff`:
1. Initial state allows check.
2. Successful check sets next check time 6 hours later. Calls before 6 hours return false on `CanCheck` and are skipped in `CheckAndUpdate`.
3. After 6 hours, check is allowed again.
4. Failed check (e.g. mock server returns 403) schedules next check 1 minute later.
5. Second failed check schedules next check 2 minutes later.
6. Subsequent failures double up to 3 hours (capped at 3 hours).
7. Force check (`Force: true`) always succeeds in running regardless of cooldown/backoff.
8. Success after failure resets backoff to initial and schedules next check in 6 hours.
- [x] **Step 2: Run test to verify it fails**
Run: `go test -v ./internal/apps/edge/updater -run TestCheckAndUpdateIntervalAndBackoff`
Expected: FAIL (fields or methods missing or behavior not implemented).
- [x] **Step 3: Write minimal implementation in `service.go`**
1. Define constants:
```go
const (
DefaultCheckInterval = 6 * time.Hour
DefaultInitialBackoff = 1 * time.Minute
DefaultMaxBackoff = 3 * time.Hour
)
```
2. Extend `Config`:
```go
type Config struct {
LocalVersion string
AssetPrefix string
LogLabel string
CheckInterval time.Duration
InitialBackoff time.Duration
MaxBackoff time.Duration
NowFunc func() time.Time
}
```
3. Extend `Service` struct with:
```go
type Service struct {
httpClient *http.Client
lastCheckKey string
localVersion string
assetPrefix string
logLabel string
checkInterval time.Duration
initialBackoff time.Duration
maxBackoff time.Duration
nowFunc func() time.Time
mu sync.Mutex
nextCheckTime time.Time
currentBackoff time.Duration
consecutiveFailures int
}
```
4. Implement `CanCheck(options UpdateOptions) bool`:
```go
func (s *Service) CanCheck(options UpdateOptions) bool {
if options.Force {
return true
}
s.mu.Lock()
defer s.mu.Unlock()
if s.nextCheckTime.IsZero() {
return true
}
now := s.now()
return !now.Before(s.nextCheckTime)
}
```
5. In `CheckAndUpdate`:
- Check `CanCheck(options)`. If false, return `nil`.
- On `getRelease` error: calculate next backoff (initial backoff on 1st failure, double previous backoff capped at maxBackoff on subsequent), increment `consecutiveFailures`, set `nextCheckTime = now.Add(currentBackoff)`, return error.
- On `getRelease` success: reset `consecutiveFailures = 0`, reset `currentBackoff = initialBackoff`, set `nextCheckTime = now.Add(checkInterval)`.
- [x] **Step 4: Run test to verify it passes**
Run: `go test -v ./internal/apps/edge/updater/...`
Expected: PASS.
---
### Task 2: Integrate `CanCheck` in `edgeheartbeat.TryAutoUpdate`
**Files:**
- Modify: `internal/apps/edge/heartbeat/autoupdate.go`
- Test: `internal/apps/edge/heartbeat/autoupdate_test.go`
**Interfaces:**
- Consumes: `(s *Service) CanCheck(options UpdateOptions) bool`
- Produces: `TryAutoUpdate(ctx context.Context, updater *edgeupdater.Service, settings *AutoUpdateSettings, logLabel string)`
- [x] **Step 1: Write the failing test in `autoupdate_test.go`**
Test that `TryAutoUpdate` checks `updater.CanCheck(opts)` and skips both log output and `CheckAndUpdate` call when throttled.
- [x] **Step 2: Run test to verify it fails**
Run: `go test -v ./internal/apps/edge/heartbeat/...`
Expected: FAIL.
- [x] **Step 3: Update `autoupdate.go`**
Update `TryAutoUpdate` to evaluate `updater.CanCheck(opts)` before logging `checking for updates` and executing `CheckAndUpdate`.
- [x] **Step 4: Run test to verify it passes**
Run: `go test -v ./internal/apps/edge/heartbeat/...`
Expected: PASS.
---
### Task 3: Full Verification of Edge and Agent Packages
**Files:**
- Test: All tests in `internal/apps/edge/...` and `internal/apps/agent/...`
- [x] **Step 1: Run comprehensive tests**
Run:
```bash
go test -v ./internal/apps/edge/... ./internal/apps/agent/...
```
Expected: PASS with zero failures or race conditions.
+7 -3
View File
@@ -35,12 +35,16 @@ func TryAutoUpdate(ctx context.Context, updater *edgeupdater.Service, settings *
if force && strings.TrimSpace(settings.UpdateChannel) != "" { if force && strings.TrimSpace(settings.UpdateChannel) != "" {
channel = settings.UpdateChannel channel = settings.UpdateChannel
} }
slog.Info("checking for "+logLabel+" updates", "repo", settings.UpdateRepo, "channel", channel, "force", force) opts := edgeupdater.UpdateOptions{
err := updater.CheckAndUpdate(ctx, settings.UpdateRepo, edgeupdater.UpdateOptions{
Channel: channel, Channel: channel,
TagName: settings.UpdateTag, TagName: settings.UpdateTag,
Force: force, Force: force,
}) }
if !updater.CanCheck(opts) {
return
}
slog.Info("checking for "+logLabel+" updates", "repo", settings.UpdateRepo, "channel", channel, "force", force)
err := updater.CheckAndUpdate(ctx, settings.UpdateRepo, opts)
if err != nil { if err != nil {
slog.Error(logLabel+" update check failed", "error", err) slog.Error(logLabel+" update check failed", "error", err)
} }
@@ -0,0 +1,102 @@
// Copyright 2026 Arctel.net
// SPDX-License-Identifier: Apache-2.0
package heartbeat
import (
"context"
"io"
"log/slog"
"net/http"
"strings"
"testing"
"time"
edgeupdater "github.com/Rain-kl/Wavelet/internal/apps/edge/updater"
)
type roundTripFunc func(req *http.Request) (*http.Response, error)
func (f roundTripFunc) RoundTrip(req *http.Request) (*http.Response, error) {
return f(req)
}
func TestTryAutoUpdateThrottling(t *testing.T) {
currentTime := time.Date(2026, 10, 17, 12, 0, 0, 0, time.UTC)
requestCount := 0
httpClient := &http.Client{
Transport: roundTripFunc(func(req *http.Request) (*http.Response, error) {
requestCount++
return &http.Response{
StatusCode: http.StatusOK,
Status: "200 OK",
Header: make(http.Header),
Body: io.NopCloser(strings.NewReader(`{"tag_name":"v1.0.0"}`)),
}, nil
}),
}
updater := edgeupdater.New(edgeupdater.Config{
LocalVersion: "v1.0.0",
AssetPrefix: "openflare-agent",
LogLabel: "agent",
CheckInterval: 6 * time.Hour,
InitialBackoff: 1 * time.Minute,
MaxBackoff: 3 * time.Hour,
NowFunc: func() time.Time { return currentTime },
HTTPClient: httpClient,
})
settings := &AutoUpdateSettings{
AutoUpdate: true,
UpdateRepo: "Rain-kl/OpenFlare",
}
// 1. Initial call should be permitted
opts := edgeupdater.UpdateOptions{Channel: "stable", Force: false}
if !updater.CanCheck(opts) {
t.Fatalf("expected CanCheck to be true initially")
}
// Run first check manually or via TryAutoUpdate
TryAutoUpdate(context.Background(), updater, settings, "agent")
if requestCount != 1 {
t.Fatalf("expected 1 request after first check, got %d", requestCount)
}
// 2. Immediately after, CanCheck is false (throttled for 6 hours)
if updater.CanCheck(opts) {
t.Fatalf("expected CanCheck to be false after check")
}
// Capture slog output to verify that throttled calls do NOT emit "checking for agent updates"
var logBuf strings.Builder
prevLogger := slog.Default()
slog.SetDefault(slog.New(slog.NewTextHandler(&logBuf, nil)))
defer slog.SetDefault(prevLogger)
// Calling TryAutoUpdate again while throttled should NOT perform any checks or log
TryAutoUpdate(context.Background(), updater, settings, "agent")
if strings.Contains(logBuf.String(), "checking for agent updates") {
t.Fatalf("expected no log message when throttled, got: %s", logBuf.String())
}
if requestCount != 1 {
t.Fatalf("expected no request when throttled, got %d requests", requestCount)
}
// 3. Force check should still log and check even when regular check is throttled
forceSettings := &AutoUpdateSettings{
AutoUpdate: true,
UpdateNow: true,
UpdateRepo: "Rain-kl/OpenFlare",
}
logBuf.Reset()
TryAutoUpdate(context.Background(), updater, forceSettings, "agent")
if !strings.Contains(logBuf.String(), "checking for agent updates") {
t.Fatalf("expected log message for force check, got: %s", logBuf.String())
}
if requestCount != 2 {
t.Fatalf("expected force check to send a request, got %d requests", requestCount)
}
}
+114 -22
View File
@@ -17,6 +17,7 @@ import (
"os" "os"
"runtime" "runtime"
"strings" "strings"
"sync"
"time" "time"
"github.com/Rain-kl/Wavelet/pkg/util" "github.com/Rain-kl/Wavelet/pkg/util"
@@ -27,34 +28,107 @@ const (
goosWindows = "windows" goosWindows = "windows"
updateTmpFilePerm = 0o600 updateTmpFilePerm = 0o600
updateBinaryFilePerm = 0o755 updateBinaryFilePerm = 0o755
// DefaultCheckInterval is the default interval between regular update checks.
DefaultCheckInterval = 6 * time.Hour
// DefaultInitialBackoff is the default delay before retrying after the first failed check.
DefaultInitialBackoff = 1 * time.Minute
// DefaultMaxBackoff caps the exponential retry delay after consecutive failures.
DefaultMaxBackoff = 3 * time.Hour
) )
var replaceAndRestartFunc = replaceAndRestart var replaceAndRestartFunc = replaceAndRestart
// Config defines the configuration for the update service. // Config defines the configuration for the update service.
type Config struct { type Config struct {
LocalVersion string LocalVersion string
AssetPrefix string AssetPrefix string
LogLabel string LogLabel string
CheckInterval time.Duration
InitialBackoff time.Duration
MaxBackoff time.Duration
NowFunc func() time.Time
HTTPClient *http.Client
} }
// Service handles checking and applying application binary updates. // Service handles checking and applying application binary updates.
type Service struct { type Service struct {
httpClient *http.Client httpClient *http.Client
lastCheckKey string lastCheckKey string
localVersion string localVersion string
assetPrefix string assetPrefix string
logLabel string logLabel string
checkInterval time.Duration
initialBackoff time.Duration
maxBackoff time.Duration
nowFunc func() time.Time
mu sync.Mutex
nextCheckTime time.Time
consecutiveFailures int
} }
// New creates a new updater Service with the provided configuration. // New creates a new updater Service with the provided configuration.
func New(cfg Config) *Service { func New(cfg Config) *Service {
return &Service{ checkInterval := cfg.CheckInterval
httpClient: &http.Client{Timeout: 30 * time.Second}, if checkInterval <= 0 {
localVersion: cfg.LocalVersion, checkInterval = DefaultCheckInterval
assetPrefix: cfg.AssetPrefix,
logLabel: cfg.LogLabel,
} }
initialBackoff := cfg.InitialBackoff
if initialBackoff <= 0 {
initialBackoff = DefaultInitialBackoff
}
maxBackoff := cfg.MaxBackoff
if maxBackoff <= 0 {
maxBackoff = DefaultMaxBackoff
}
nowFunc := cfg.NowFunc
if nowFunc == nil {
nowFunc = time.Now
}
httpClient := cfg.HTTPClient
if httpClient == nil {
httpClient = &http.Client{Timeout: 30 * time.Second}
}
return &Service{
httpClient: httpClient,
localVersion: cfg.LocalVersion,
assetPrefix: cfg.AssetPrefix,
logLabel: cfg.LogLabel,
checkInterval: checkInterval,
initialBackoff: initialBackoff,
maxBackoff: maxBackoff,
nowFunc: nowFunc,
}
}
// CanCheck returns whether an update check is allowed to run under the check interval and failure backoff.
func (s *Service) CanCheck(options UpdateOptions) bool {
if options.Force {
return true
}
s.mu.Lock()
defer s.mu.Unlock()
return !s.nowFunc().Before(s.nextCheckTime)
}
// recordFailure schedules the next check using exponential backoff capped at maxBackoff.
func (s *Service) recordFailure() {
s.mu.Lock()
defer s.mu.Unlock()
backoff := s.initialBackoff
for i := 0; i < s.consecutiveFailures && backoff < s.maxBackoff; i++ {
backoff *= 2
}
s.consecutiveFailures++
s.nextCheckTime = s.nowFunc().Add(min(backoff, s.maxBackoff))
}
// recordSuccess resets the failure backoff and schedules the next regular check.
func (s *Service) recordSuccess() {
s.mu.Lock()
defer s.mu.Unlock()
s.consecutiveFailures = 0
s.nextCheckTime = s.nowFunc().Add(s.checkInterval)
} }
// UpdateOptions specifies parameters for checking and applying updates. // UpdateOptions specifies parameters for checking and applying updates.
@@ -78,13 +152,29 @@ type githubAsset struct {
} }
// CheckAndUpdate checks for a newer release on GitHub and performs an update if available. // CheckAndUpdate checks for a newer release on GitHub and performs an update if available.
// Transient failures are retried with exponential backoff; otherwise the next check waits for the check interval.
// Callers are expected to invoke it serially.
func (s *Service) CheckAndUpdate(ctx context.Context, repo string, options UpdateOptions) error { func (s *Service) CheckAndUpdate(ctx context.Context, repo string, options UpdateOptions) error {
if !s.CanCheck(options) {
return nil
}
retry, err := s.checkAndUpdate(ctx, repo, options)
if retry {
s.recordFailure()
} else {
s.recordSuccess()
}
return err
}
// checkAndUpdate performs one update check and reports whether a returned error is transient and worth retrying.
func (s *Service) checkAndUpdate(ctx context.Context, repo string, options UpdateOptions) (retry bool, err error) {
release, err := s.getRelease(ctx, repo, options) release, err := s.getRelease(ctx, repo, options)
if err != nil { if err != nil {
return fmt.Errorf("check latest release: %w", err) return true, fmt.Errorf("check latest release: %w", err)
} }
if release == nil || release.TagName == "" { if release == nil || release.TagName == "" {
return nil return false, nil
} }
remoteVersion := normalizeVersion(release.TagName) remoteVersion := normalizeVersion(release.TagName)
@@ -92,14 +182,14 @@ func (s *Service) CheckAndUpdate(ctx context.Context, repo string, options Updat
checkKey := buildReleaseCheckKey(options, remoteVersion) checkKey := buildReleaseCheckKey(options, remoteVersion)
if remoteVersion == localVersion { if remoteVersion == localVersion {
return nil return false, nil
} }
if !options.Force && checkKey != "" && checkKey == s.lastCheckKey { if !options.Force && checkKey != "" && checkKey == s.lastCheckKey {
return nil return false, nil
} }
if !isNewer(localVersion, remoteVersion) { if !isNewer(localVersion, remoteVersion) {
s.lastCheckKey = checkKey s.lastCheckKey = checkKey
return nil return false, nil
} }
slog.Info(s.logLabel+" update available", "from", localVersion, "to", remoteVersion) slog.Info(s.logLabel+" update available", "from", localVersion, "to", remoteVersion)
@@ -108,20 +198,22 @@ func (s *Service) CheckAndUpdate(ctx context.Context, repo string, options Updat
downloadURL, expectedChecksum, err := s.resolveReleaseAsset(ctx, release, assetName) downloadURL, expectedChecksum, err := s.resolveReleaseAsset(ctx, release, assetName)
if err != nil { if err != nil {
if downloadURL == "" { if downloadURL == "" {
// The release has no asset for this platform; wait for the next regular check.
s.lastCheckKey = checkKey s.lastCheckKey = checkKey
return false, err
} }
return err return true, err
} }
execPath, err := os.Executable() execPath, err := os.Executable()
if err != nil { if err != nil {
return fmt.Errorf("get executable path: %w", err) return false, fmt.Errorf("get executable path: %w", err)
} }
if err = s.downloadAndRestart(ctx, downloadURL, expectedChecksum, execPath); err != nil { if err = s.downloadAndRestart(ctx, downloadURL, expectedChecksum, execPath); err != nil {
return fmt.Errorf("download and restart: %w", err) return true, fmt.Errorf("download and restart: %w", err)
} }
s.lastCheckKey = checkKey s.lastCheckKey = checkKey
return nil return false, nil
} }
func (s *Service) getRelease(ctx context.Context, repo string, options UpdateOptions) (*githubRelease, error) { func (s *Service) getRelease(ctx context.Context, repo string, options UpdateOptions) (*githubRelease, error) {
+242 -6
View File
@@ -14,6 +14,7 @@ import (
"runtime" "runtime"
"strings" "strings"
"testing" "testing"
"time"
) )
type roundTripFunc func(req *http.Request) (*http.Response, error) type roundTripFunc func(req *http.Request) (*http.Response, error)
@@ -23,12 +24,12 @@ func (f roundTripFunc) RoundTrip(req *http.Request) (*http.Response, error) {
} }
func testService(httpClient *http.Client) *Service { func testService(httpClient *http.Client) *Service {
return &Service{ return New(Config{
httpClient: httpClient, LocalVersion: "v1.0.0",
localVersion: "v1.0.0", AssetPrefix: "openflare-agent",
assetPrefix: "openflare-agent", LogLabel: "agent",
logLabel: "agent", HTTPClient: httpClient,
} })
} }
func TestGetLatestPreviewRelease(t *testing.T) { func TestGetLatestPreviewRelease(t *testing.T) {
@@ -106,6 +107,56 @@ func TestCheckAndUpdateRequiresChecksumSource(t *testing.T) {
} }
} }
func TestCheckAndUpdateSchedulesNextCheckByFailureKind(t *testing.T) {
assetName := testService(nil).assetNameForGOOSGOARCH(runtime.GOOS, runtime.GOARCH)
digest := "sha256:" + strings.Repeat("a", sha256.Size*2)
testCases := []struct {
name string
releaseAssetName string
wantRetry bool
}{
{name: "asset download failure retries with backoff", releaseAssetName: assetName, wantRetry: true},
{name: "missing platform asset waits for regular interval", releaseAssetName: "openflare-agent-other", wantRetry: false},
}
for _, tc := range testCases {
t.Run(tc.name, func(t *testing.T) {
currentTime := time.Date(2026, 10, 17, 12, 0, 0, 0, time.UTC)
service := New(Config{
LocalVersion: "v1.0.0",
AssetPrefix: "openflare-agent",
LogLabel: "agent",
NowFunc: func() time.Time { return currentTime },
HTTPClient: &http.Client{
Transport: roundTripFunc(func(req *http.Request) (*http.Response, error) {
if req.URL.Host != "api.github.com" {
return &http.Response{
StatusCode: http.StatusInternalServerError,
Status: "500 Internal Server Error",
Header: make(http.Header),
Body: io.NopCloser(strings.NewReader("error")),
}, nil
}
return &http.Response{
StatusCode: http.StatusOK,
Header: make(http.Header),
Body: io.NopCloser(strings.NewReader(`{"tag_name":"v1.0.1","assets":[` +
`{"name":"` + tc.releaseAssetName + `","browser_download_url":"https://example.test/agent","digest":"` + digest + `"}]}`)),
}, nil
}),
},
})
if err := service.CheckAndUpdate(context.Background(), "Rain-kl/OpenFlare", UpdateOptions{}); err == nil {
t.Fatalf("expected update error")
}
currentTime = currentTime.Add(DefaultInitialBackoff)
if got := service.CanCheck(UpdateOptions{}); got != tc.wantRetry {
t.Fatalf("CanCheck after initial backoff = %v, want %v", got, tc.wantRetry)
}
})
}
}
func TestNormalizeGitHubDigest(t *testing.T) { func TestNormalizeGitHubDigest(t *testing.T) {
checksum := strings.Repeat("a", sha256.Size*2) checksum := strings.Repeat("a", sha256.Size*2)
testCases := []struct { testCases := []struct {
@@ -325,3 +376,188 @@ func TestIsNewerSupportsPrerelease(t *testing.T) {
}) })
} }
} }
func TestCheckAndUpdateIntervalAndBackoff(t *testing.T) {
currentTime := time.Date(2026, 10, 17, 12, 0, 0, 0, time.UTC)
nowFunc := func() time.Time { return currentTime }
var (
statusCode = http.StatusOK
responseBody = `{"tag_name":"v1.0.0"}`
requestCount = 0
)
httpClient := &http.Client{
Transport: roundTripFunc(func(req *http.Request) (*http.Response, error) {
requestCount++
if statusCode != http.StatusOK {
return &http.Response{
StatusCode: statusCode,
Status: http.StatusText(statusCode),
Header: make(http.Header),
Body: io.NopCloser(strings.NewReader("error")),
}, nil
}
return &http.Response{
StatusCode: http.StatusOK,
Status: "200 OK",
Header: make(http.Header),
Body: io.NopCloser(strings.NewReader(responseBody)),
}, nil
}),
}
service := New(Config{
LocalVersion: "v1.0.0",
AssetPrefix: "openflare-agent",
LogLabel: "agent",
CheckInterval: 6 * time.Hour,
InitialBackoff: 1 * time.Minute,
MaxBackoff: 3 * time.Hour,
NowFunc: nowFunc,
HTTPClient: httpClient,
})
// 1. Initial state: CanCheck should be true
if !service.CanCheck(UpdateOptions{}) {
t.Fatalf("expected CanCheck to be true initially")
}
// 2. First check succeeds (version v1.0.0 is same as local, no update needed)
err := service.CheckAndUpdate(context.Background(), "Rain-kl/OpenFlare", UpdateOptions{})
if err != nil {
t.Fatalf("unexpected error on first check: %v", err)
}
if requestCount != 1 {
t.Fatalf("expected 1 request, got %d", requestCount)
}
// Immediately after success: CanCheck should be false, and CheckAndUpdate should be throttled (no HTTP call)
if service.CanCheck(UpdateOptions{}) {
t.Fatalf("expected CanCheck to be false immediately after success")
}
err = service.CheckAndUpdate(context.Background(), "Rain-kl/OpenFlare", UpdateOptions{})
if err != nil {
t.Fatalf("unexpected error on throttled check: %v", err)
}
if requestCount != 1 {
t.Fatalf("expected request count to remain 1, got %d", requestCount)
}
// 3. Fast-forward 5 hours: still throttled (< 6h)
currentTime = currentTime.Add(5 * time.Hour)
if service.CanCheck(UpdateOptions{}) {
t.Fatalf("expected CanCheck to be false after 5 hours")
}
// Force check should bypass throttle even within 6 hours
if !service.CanCheck(UpdateOptions{Force: true}) {
t.Fatalf("expected CanCheck(Force: true) to be true within 6 hours")
}
err = service.CheckAndUpdate(context.Background(), "Rain-kl/OpenFlare", UpdateOptions{Force: true})
if err != nil {
t.Fatalf("unexpected error on force check: %v", err)
}
if requestCount != 2 {
t.Fatalf("expected request count 2 after force check, got %d", requestCount)
}
// 4. Fast-forward to 6 hours after the force check: should be allowed again
currentTime = currentTime.Add(6 * time.Hour)
if !service.CanCheck(UpdateOptions{}) {
t.Fatalf("expected CanCheck to be true after 6 hours")
}
// 5. Simulate Failure: GitHub returns 403 Forbidden
statusCode = http.StatusForbidden
err = service.CheckAndUpdate(context.Background(), "Rain-kl/OpenFlare", UpdateOptions{})
if err == nil {
t.Fatalf("expected error on 403 Forbidden check")
}
if requestCount != 3 {
t.Fatalf("expected request count 3, got %d", requestCount)
}
// Immediately after 1st failure: CanCheck should be false (1 min initial backoff)
if service.CanCheck(UpdateOptions{}) {
t.Fatalf("expected CanCheck to be false after 1st failure")
}
// At +30s: still false
currentTime = currentTime.Add(30 * time.Second)
if service.CanCheck(UpdateOptions{}) {
t.Fatalf("expected CanCheck to be false at +30s after 1st failure")
}
// At +1m (total +60s): should be true
currentTime = currentTime.Add(30 * time.Second)
if !service.CanCheck(UpdateOptions{}) {
t.Fatalf("expected CanCheck to be true at +1m after 1st failure")
}
// 6. Second failure: backoff doubles to 2 minutes
err = service.CheckAndUpdate(context.Background(), "Rain-kl/OpenFlare", UpdateOptions{})
if err == nil {
t.Fatalf("expected error on 2nd failure")
}
if requestCount != 4 {
t.Fatalf("expected request count 4, got %d", requestCount)
}
// At +1m: should still be false
currentTime = currentTime.Add(1 * time.Minute)
if service.CanCheck(UpdateOptions{}) {
t.Fatalf("expected CanCheck to be false at +1m after 2nd failure (backoff is 2m)")
}
// At +2m: should be true
currentTime = currentTime.Add(1 * time.Minute)
if !service.CanCheck(UpdateOptions{}) {
t.Fatalf("expected CanCheck to be true at +2m after 2nd failure")
}
// 7. Simulate multiple consecutive failures to verify exponential doubling and capping at 3 hours
expectedBackoffs := []time.Duration{
4 * time.Minute,
8 * time.Minute,
16 * time.Minute,
32 * time.Minute,
64 * time.Minute,
128 * time.Minute,
3 * time.Hour, // 256m capped at 180m (3h)
3 * time.Hour, // capped at 3h
}
for i, expectedBackoff := range expectedBackoffs {
err = service.CheckAndUpdate(context.Background(), "Rain-kl/OpenFlare", UpdateOptions{})
if err == nil {
t.Fatalf("expected error on consecutive failure iteration %d", i)
}
// Halfway through backoff: should be false
currentTime = currentTime.Add(expectedBackoff / 2)
if service.CanCheck(UpdateOptions{}) {
t.Fatalf("expected CanCheck to be false halfway through backoff %v (iter %d)", expectedBackoff, i)
}
// At full backoff: should be true
currentTime = currentTime.Add(expectedBackoff / 2)
if !service.CanCheck(UpdateOptions{}) {
t.Fatalf("expected CanCheck to be true at full backoff %v (iter %d)", expectedBackoff, i)
}
}
// 8. Recovery: next check succeeds, backoff should be reset and next check scheduled in 6 hours
statusCode = http.StatusOK
err = service.CheckAndUpdate(context.Background(), "Rain-kl/OpenFlare", UpdateOptions{})
if err != nil {
t.Fatalf("expected success on recovery, got: %v", err)
}
// Immediately: CanCheck is false
if service.CanCheck(UpdateOptions{}) {
t.Fatalf("expected CanCheck to be false immediately after recovery")
}
// At +3 hours: still false (because next check is 6 hours, not 3 hours)
currentTime = currentTime.Add(3 * time.Hour)
if service.CanCheck(UpdateOptions{}) {
t.Fatalf("expected CanCheck to be false at +3 hours after recovery")
}
// At +6 hours: should be true
currentTime = currentTime.Add(3 * time.Hour)
if !service.CanCheck(UpdateOptions{}) {
t.Fatalf("expected CanCheck to be true at +6 hours after recovery")
}
}