feat(cordis): add OpenFlare Cordis 架构改造设计

docs(changelog): 修正表述笔误

refactor(cordis): 磁盘缓存改用上上游能力并清理本地副本

按上游/下游归属规约:类型断言守卫已回流 Wavelet(f3d85d5,附回归用例),
本仓库删除 OpenFlare/plugins/server/pkg/cache 整包并改 import 到
Wavelet/pkg/cache/disk,同步后与上游零漂移。

验证:go build 通过;go test ./... exit 0(137 包 ok);256 条路由对拍与
232 条 swagger 操作均零差异;make build-all 四进制;前端零改动。

docs(cordis): 记录 T1 清理结果与五个复用阻塞点

refactor(cordis): server 复用上游 pkg 能力并删除等价本地副本

按上游/下游归属规约清理重复实现,删除 7 个与上游等价的本地包并改 import:
shared/response→pkg/response、pkg/{logger,mail,trace,httppool,cache/ram}→
上游同名包、infra/persistence/batchwriter→pkg/batchwriter。逐项核过差异:
httppool 逐字节相同;logger 的 Config 字段完全一致;response 的 7 个 Abort*
一致;cache/ram 换过去顺带把裸 go 变回带 panic 恢复的 util.Go。

两处非等价差异按语义处理:
- batchwriter.Stats 与 status DTO 原为类型别名,改为消费侧逐字段转换,
  避免 model 反向依赖基础设施类型;
- 上游 pkg/idgen 要求显式 Init(本地副本为懒加载自动初始化),本次保留本地
  副本,待与 infra 初始化一并迁移(已登记在清理计划)。

验证:go build 通过;go test ./... exit 0(138 包 ok);256 条路由对拍零差异;
make swagger 232 条操作零增减,且归一化后与旧文档深度相等——差异仅为
response.Any / logger.LogEntry 两个定义名随包路径改名,接口形状未变。

chore(cordis): 回流内核与 pkg/util 通用能力并清理 vendoring 污染

按新增的上游/下游归属规约:HandleRaw/BasePath 与版本比较、网络、格式化助手
属通用能力,已提交到 Wavelet 分支 feat/cordis-router-raw-routes,本仓库改为
纯同步获取(pkg/util 已零漂移),补丁登记保留至上游合并。

同时修掉我此前 git add -A 造成的污染:首次 vendoring 把上游工作区里被
gitignore 的运行期产物一起提交进来(upload 的 diskcache 缓存块 650 个与
driver_http/dist 前端构建物 380 个,共 12872 行/1030 文件)。sync-upstream.sh
现显式排除 uploads/dist/data/*.db,.gitignore 补上对应兜底规则。

AGENTS.md 增加上游/下游改动归属规约,并把仍指向前 Cordis 布局的硬性约束
(internal/router + Serve、internal/repository/logstore、internal/platform/bootstrap、
internal/cmd)改到当前插件路径。

验证:go build 通过;go test ./... exit 0(144 包 ok);make swagger 232 条
操作与基线逐条一致;make build-all 四进制;gofmt 干净。

feat(cordis): server 插件化并改由内核挂载控制面路由

新增 plugins/server/plugin.go:Apply 以 ctx.Router().Group(app.api_prefix)
声明根级与 /v1 全部路由;33 个注册函数由 *gin.RouterGroup 改为
core.RouterExtension,RegisterCollection 改用内核新增的 HandleRaw 保留
尾部斜杠变体,AdminMiddlewares 返回 []any(Go 不允许把 []T 展开为 ...any)。
删除 router.Serve 与 registerRoutes,装配根改为 core.App +
driver_http.New(WithEngine(router.BuildEngine())),监听、信号与优雅退出归内核;
前端 SPA 的 NoRoute 兜底因内核暂无贡献点而保留在引擎层。

路由保真证据:plugin_parity_test 对拍 baseline/routes-engine.txt 的 256 条
(方法 路径) 零差异;go test ./... exit 0(144 包 ok,含真实 handler 的
openflare/integration 用例走同一条挂载路径);make swagger 232 条操作与基线
逐条一致;golangci-lint 0 issues;make build-all 四进制;embed_frontend
标签编译通过;前端零改动。

已知待补:带 Redis 的实机 HTTP 冒烟(本机 6379 未启动,session store 与
改造前一样在建店阶段即 fatal),以及 bootstrap 的任务/设置/迁移注册迁入 Apply。

feat(core): RouterExtension 增加 HandleRaw 与 BasePath 以保真尾部斜杠路由

server 插件化的前置:Handle 经 cleanPath 会剥掉尾部斜杠,无法表达
/resource 与 /resource/ 两条不同路由,而 OpenFlare 有 20 个历史 list
端点两者都注册且部署关闭了 RedirectTrailingSlash,缺失即 404。新增
HandleRaw 与 BasePath(作用域包装器同样登记反注册),补 extpoints 用例;
并把 router.Serve 拆出 BuildEngine 以便交给 driver_http.WithEngine 复用,
新增路由表导出 harness,固化 256 条 (方法 路径) 基线供插件化对拍。
上游补丁登记于 backend/OpenFlare/upstream-patches.md,同步脚本改为按目录
前缀输出差异并在同步后提醒确认补丁是否仍在。

验证:go build 通过;go test ./... exit 0(143 包 ok);gofmt 干净。

docs(cordis): 记录 server 插件接入内核的可行路径与内核能力缺口

feat(cordis): agent/relay/flared 落地为内核驱动插件

三个边缘守护进程各新增 plugin.go,实现 core.Plugin + core.Driver
(自定义 DriverType 与同名 profile),装配与生命周期从 main 迁入
Apply/Start/Stop:Apply 负责 JSON 配置加载、运行环境与用户确保、
openresty/frps/frpc 管理器与各服务装配;Start 以 util.Go 拉起阻塞式
runner 与 GeoIP 周期更新;Stop 收敛主循环结果并在超时时报错而非静默。

入口改为 core.NewApp(core.WithProfile(...)) + Prepare/Run,保持
-config 旗标、默认路径、退出码与启动/停止日志不变。

验证:go build 通过;go test ./... exit 0(143 包 ok,含 3 个插件身份
与配置失败路径测试);make build-all 四进制产出;三进制实跑缺失配置
均 exit 1 且错误链保留 load {agent,relay,flared} config 原因;gofmt 干净。

refactor(cordis): 按功能职责拆分为 4 个插件与 share 共享层

backend/OpenFlare 不再平铺遗留分层,改为 plugins/{server,agent,relay,flared}
加 share/:控制面业务(openflare/admin/oauth/user/upload/cap/config/health 与
repository/model/infra/router 等支撑层)归 server;三个边缘守护进程各自成插件;
被两个以上插件消费的 protocol/geoip/wsclient/render/pagesarchive/edge 归 share。
同时把 pkg/util 与 buildinfo 合并回上游 pkg(上游已覆盖全部符号,仅 8 个函数与
2 个类型为 OpenFlare 独有,已一并迁入),装配根统一到 backend/cmd(含三个 daemon
入口),Dockerfile 与 release 工作流的构建路径和 -X 注入路径同步更新。

验证:go build 通过;go test ./... exit 0(141 包 ok);make swagger exit 0 且
232 条 API 操作与基线逐条一致;make build-all 产出 4 进制;-X 注入经二进制
strings 实测生效;日志后端直连门禁改写为按 server 插件业务域扫描并在扫描数为 0
时报错(防门禁静默失效);前端零改动。

feat(cordis): 落地 backend/share 共享层与上游同步脚本

跨插件共享资源(控制消息协议、GeoIP+iputil、边缘守护进程日志)从下游包
移入 backend/share,并声明其只能依赖 core/pkg 与标准/第三方库,禁止反向
引用下游业务与具体插件实现;新增 scripts/sync-upstream.sh 只覆盖
backend/{core,pkg,plugins},同步后 --check 报告零差异,证明与上游逐字一致。

go build 通过,go test ./... exit 0(142 包 ok),前端零改动。

refactor(cordis): 采用与 Wavelet 同构的单模块布局并引入上游内核

按上游结构落位:backend/{core,pkg,plugins} 为 Wavelet 上游拷贝,OpenFlare
全部业务收拢到上游 downstream 所对应的位置 backend/OpenFlare/,模块名保持
Wavelet 以保证上游 import 路径逐字一致、同步零改写;三个 daemon 入口移至
backend/OpenFlare/cmd,backend/cmd 与 main.go 作为控制面装配根。

行为不变:go build 通过,142 个测试包全绿(含上游插件测试),232 条 API
操作与改造前逐条一致,四进制产物正常,前端零改动。swagger 暂只扫描下游代码,
待 P4 挂载上游路由后再纳入 plugins/。

style: 修正模块路径改写导致的 import 分组排序漂移

refactor(layout): Go 代码迁入 backend/ 并将模块名简化为 OpenFlare

对齐上游 Wavelet 的仓库布局,为以第二 module 形态 vendoring Cordis 内核与
平台插件做准备:模块路径整体改写为 OpenFlare,Go 目标加 cd backend,
swaggo 产物移至 backend/docs 并把 json/yaml 复制回 docs/ 供站点消费,
Dockerfile 与 release 工作流的构建目录、ldflags 模块路径同步更新。

行为保持不变:232 条路由与改造前逐条一致,95 个测试包全绿,
四进制产物正常,前端零改动。

chore(cordis): 落地改造计划与 schema/路由基线

新增 legacy_dump_test 迁移快照 harness:在临时 sqlite 库上按生产顺序
(goose.UpTo → zone 导入 → goose.Up)跑完 76 个历史迁移并导出 schema 与
版本序列,作为改造前后一致性门禁的唯一事实来源。同时记录 232 条路由清单
与 foundation 实施计划。

docs(cordis): add OpenFlare Cordis 架构改造设计

明确上游以第二 module 形态 vendoring 进 backend/Wavelet、4 个插件
(server/agent/relay/flared) 全部装载内核,并规定保留 76 个历史 goose
迁移 + 一次性版本 stamp 桥接的迁移方案,配套三方 schema 一致性门禁,
确保已部署库不重跑历史、不丢数据。
This commit is contained in:
ryan
2026-08-29 19:28:39 +08:00
parent 9f79fb9969
commit dbaa3bf140
1327 changed files with 91634 additions and 4157 deletions
+25
View File
@@ -0,0 +1,25 @@
// Copyright 2026 Arctel.net
// SPDX-License-Identifier: Apache-2.0
package cache
// RedisConfig declares the configuration read by the Redis cache provider.
type RedisConfig struct {
Enabled bool `config:"enabled" env:"REDIS_ENABLED" default:"false" autoEnable:"REDIS_ADDR"`
Addrs []string `config:"addrs" env:"REDIS_ADDR"`
Username string `config:"username" env:"REDIS_USERNAME"`
Password string `config:"password" env:"REDIS_PASSWORD" secret:"true"`
DB int `config:"db" env:"REDIS_DB"`
ClusterMode bool `config:"cluster_mode" env:"REDIS_CLUSTER_MODE"`
MasterName string `config:"master_name" env:"REDIS_MASTER_NAME"`
KeyPrefix string `config:"key_prefix" env:"REDIS_KEY_PREFIX"`
PoolSize int `config:"pool_size" env:"REDIS_POOL_SIZE"`
MinIdleConn int `config:"min_idle_conn" env:"REDIS_MIN_IDLE_CONN"`
DialTimeout int `config:"dial_timeout" env:"REDIS_DIAL_TIMEOUT"`
ReadTimeout int `config:"read_timeout" env:"REDIS_READ_TIMEOUT"`
WriteTimeout int `config:"write_timeout" env:"REDIS_WRITE_TIMEOUT"`
MaxRetries int `config:"max_retries" env:"REDIS_MAX_RETRIES"`
PoolTimeout int `config:"pool_timeout" env:"REDIS_POOL_TIMEOUT"`
ConnMaxIdleTime int `config:"conn_max_idle_time" env:"REDIS_CONN_MAX_IDLE_TIME"`
MaintNotifications bool `config:"maint_notifications" env:"REDIS_MAINT_NOTIFICATIONS" default:"false"`
}
+12
View File
@@ -0,0 +1,12 @@
// Copyright 2026 Arctel.net
// SPDX-License-Identifier: Apache-2.0
package cache
const (
errRedisHashSetFailed = "failed to set redis hash: %w"
errRedisHashDeleteFailed = "failed to delete redis hash field: %w"
errUnmarshalDataFailed = "failed to unmarshal data: %w"
errMarshalDataFailed = "failed to marshal data: %w"
errRedisKeySetFailed = "failed to set redis key: %w"
)
+298
View File
@@ -0,0 +1,298 @@
// Copyright 2026 Arctel.net
// SPDX-License-Identifier: Apache-2.0
// Package cache provides the multi-tier caching infrastructure plugin for Cordis.
package cache
import (
"Wavelet/core"
"Wavelet/core/contracts"
"Wavelet/pkg/cache/ram"
"Wavelet/pkg/util"
"context"
"encoding/json"
"errors"
"sync"
"time"
"github.com/redis/go-redis/v9"
)
const (
defaultRAMCapacity = 10000
defaultPubSubChannel = "wavelet:cache:invalidation"
)
type ramEntry struct {
data []byte
expireAt time.Time
}
// Option configures the cache plugin.
type Option func(*Plugin)
// WithRedis sets an explicit Redis client instance.
func WithRedis(client redis.UniversalClient) Option {
return func(p *Plugin) {
p.redisClient = client
}
}
// WithKeyPrefix sets a custom Redis key prefix.
func WithKeyPrefix(prefix string) Option {
return func(p *Plugin) {
p.keyPrefix = prefix
}
}
// WithRAMCapacity sets the maximum capacity for the L1 RAM cache.
func WithRAMCapacity(capacity int) Option {
return func(p *Plugin) {
p.ramCapacity = capacity
}
}
// Plugin implements core.Plugin to provide contracts.CacheService.
type Plugin struct {
redisClient redis.UniversalClient
keyPrefix string
ramCapacity int
}
// New creates a new cache infrastructure plugin.
func New(opts ...Option) *Plugin {
p := &Plugin{
ramCapacity: defaultRAMCapacity,
}
for _, opt := range opts {
if opt != nil {
opt(p)
}
}
return p
}
// Name returns the unique identifier of the cache plugin.
func (p *Plugin) Name() string {
return "cache"
}
// DeclareConfig declares the configuration bindings consumed by this plugin.
func (p *Plugin) DeclareConfig() []core.ConfigBinding {
return []core.ConfigBinding{
{Prefix: "redis", Target: &RedisConfig{}},
}
}
// ConfigEnabled gates plugin activation based on whether Redis is enabled.
func (p *Plugin) ConfigEnabled(view core.ConfigView) bool {
return view.Bool("redis.enabled", false)
}
// Apply mounts the multi-layer cache service into the Context.
func (p *Plugin) Apply(ctx *core.Context) error {
var cfg RedisConfig
if err := ctx.Config().Bind("redis", &cfg); err != nil {
return err
}
redisClient := p.redisClient
if redisClient == nil {
if Redis == nil {
var err error
redisClient, err = InitRedisWithConfig(cfg)
if err != nil {
return err
}
} else {
redisClient = Redis
}
}
ramCache, err := ram.New[string, ramEntry](ram.Options{
MaximumSize: p.ramCapacity,
})
if err != nil {
return err
}
svc := &cacheServiceImpl{
ramCache: ramCache,
redisClient: redisClient,
keyPrefix: p.keyPrefix,
pubSubChannel: defaultPubSubChannel,
stopCh: make(chan struct{}),
}
if redisClient != nil {
svc.startPubSubListener()
ctx.OnDispose(func() error {
svc.stopPubSubListener()
if p.redisClient == nil {
Redis = nil
if closeErr := redisClient.Close(); closeErr != nil && !errors.Is(closeErr, redis.ErrClosed) {
return closeErr
}
}
return nil
})
}
core.Provide[contracts.CacheService](ctx, svc)
return nil
}
type cacheServiceImpl struct {
ramCache *ram.Cache[string, ramEntry]
redisClient redis.UniversalClient
keyPrefix string
pubSubChannel string
subOnce sync.Once
stopOnce sync.Once
stopCh chan struct{}
pubsub *redis.PubSub
}
func (s *cacheServiceImpl) prefixedKey(key string) string {
if s.keyPrefix != "" {
return s.keyPrefix + key
}
return PrefixedKey(key)
}
func (s *cacheServiceImpl) startPubSubListener() {
if s.redisClient == nil {
return
}
s.subOnce.Do(func() {
pubsub := s.redisClient.Subscribe(context.Background(), s.pubSubChannel)
s.pubsub = pubsub
util.Go(func() {
ch := pubsub.Channel()
for {
select {
case <-s.stopCh:
return
case msg, ok := <-ch:
if !ok {
return
}
if msg != nil && msg.Payload != "" {
s.ramCache.Invalidate(msg.Payload)
}
}
}
})
})
}
func (s *cacheServiceImpl) stopPubSubListener() {
s.stopOnce.Do(func() {
close(s.stopCh)
if s.pubsub != nil {
_ = s.pubsub.Close()
}
})
}
func (s *cacheServiceImpl) Get(ctx context.Context, key string, target any) error {
// 1. Check L1 RAM cache
if entry, ok := s.ramCache.GetIfPresent(key); ok {
if entry.expireAt.IsZero() || time.Now().Before(entry.expireAt) {
return json.Unmarshal(entry.data, target)
}
// Expired in L1 RAM
s.ramCache.Invalidate(key)
}
// 2. Check L2 Redis cache
if s.redisClient != nil {
data, err := s.redisClient.Get(ctx, s.prefixedKey(key)).Bytes()
if err == nil {
// Backfill L1 RAM cache
s.ramCache.Set(key, ramEntry{
data: data,
})
return json.Unmarshal(data, target)
} else if !errors.Is(err, redis.Nil) {
return err
}
}
return contracts.ErrCacheMiss
}
func (s *cacheServiceImpl) Set(ctx context.Context, key string, value any, ttl time.Duration) error {
data, err := json.Marshal(value)
if err != nil {
return err
}
var expireAt time.Time
if ttl > 0 {
expireAt = time.Now().Add(ttl)
}
// 1. Write L1 RAM cache
s.ramCache.Set(key, ramEntry{
data: data,
expireAt: expireAt,
})
// 2. Write L2 Redis cache
if s.redisClient != nil {
if err := s.redisClient.Set(ctx, s.prefixedKey(key), data, ttl).Err(); err != nil {
return err
}
}
return nil
}
func (s *cacheServiceImpl) Delete(ctx context.Context, key string) error {
// 1. Evict L1 RAM
s.ramCache.Invalidate(key)
// 2. Evict L2 Redis and broadcast invalidation to cluster nodes
if s.redisClient != nil {
if err := s.redisClient.Del(ctx, s.prefixedKey(key)).Err(); err != nil && !errors.Is(err, redis.Nil) {
return err
}
_ = s.redisClient.Publish(ctx, s.pubSubChannel, key).Err()
}
return nil
}
func (s *cacheServiceImpl) Invalidate(ctx context.Context, key string) error {
return s.Delete(ctx, key)
}
func (s *cacheServiceImpl) GetOrSet(ctx context.Context, key string, target any, ttl time.Duration, loader func() (any, error)) error {
err := s.Get(ctx, key, target)
if err == nil {
return nil
}
if !errors.Is(err, contracts.ErrCacheMiss) {
return err
}
val, err := loader()
if err != nil {
return err
}
if err := s.Set(ctx, key, val, ttl); err != nil {
return err
}
data, err := json.Marshal(val)
if err != nil {
return err
}
return json.Unmarshal(data, target)
}
+83
View File
@@ -0,0 +1,83 @@
// Copyright 2026 Arctel.net
// SPDX-License-Identifier: Apache-2.0
package cache_test
import (
"Wavelet/core"
"Wavelet/core/contracts"
"Wavelet/plugins/infra/cache"
"context"
"testing"
"time"
"github.com/alicebob/miniredis/v2"
"github.com/redis/go-redis/v9"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
)
func TestCachePluginOperations(t *testing.T) {
mr, err := miniredis.Run()
require.NoError(t, err)
defer mr.Close()
rdb := redis.NewClient(&redis.Options{
Addr: mr.Addr(),
})
defer func() { _ = rdb.Close() }()
p := cache.New(
cache.WithRedis(rdb),
cache.WithKeyPrefix("app:"),
cache.WithRAMCapacity(500),
)
ctx := core.NewContext(context.Background())
ctx.Config().SetSource(core.NewMapSource(map[string]any{
"redis.enabled": true,
}))
require.NoError(t, ctx.Config().Resolve())
require.NoError(t, p.Apply(ctx))
svc, err := core.Inject[contracts.CacheService](ctx)
require.NoError(t, err)
type Data struct {
Value string `json:"value"`
}
testCtx := context.Background()
// 1. ErrCacheMiss
var out Data
err = svc.Get(testCtx, "missing", &out)
assert.ErrorIs(t, err, contracts.ErrCacheMiss)
// 2. Set & Get
in := Data{Value: "hello"}
require.NoError(t, svc.Set(testCtx, "key1", in, 5*time.Minute))
require.NoError(t, svc.Get(testCtx, "key1", &out))
assert.Equal(t, "hello", out.Value)
// 3. GetOrSet
var target Data
err = svc.GetOrSet(testCtx, "key1", &target, time.Minute, func() (any, error) {
return Data{Value: "from_loader"}, nil
})
require.NoError(t, err)
assert.Equal(t, "hello", target.Value)
err = svc.GetOrSet(testCtx, "key2", &target, time.Minute, func() (any, error) {
return Data{Value: "from_loader"}, nil
})
require.NoError(t, err)
assert.Equal(t, "from_loader", target.Value)
// 4. Invalidate / Delete
require.NoError(t, svc.Invalidate(testCtx, "key1"))
err = svc.Get(testCtx, "key1", &out)
assert.ErrorIs(t, err, contracts.ErrCacheMiss)
require.NoError(t, ctx.Dispose())
}
+223
View File
@@ -0,0 +1,223 @@
// Copyright 2026 Arctel.net
// SPDX-License-Identifier: Apache-2.0
package cache
import (
"context"
"encoding/json"
"fmt"
"log"
"strings"
"sync"
"time"
"github.com/redis/go-redis/extra/redisotel/v9"
"github.com/redis/go-redis/v9"
"github.com/redis/go-redis/v9/maintnotifications"
"go.opentelemetry.io/otel/attribute"
)
var (
// Redis 全局 Redis 客户端实例
Redis redis.UniversalClient
keyPrefixMu sync.RWMutex
keyPrefix string
)
// SetKeyPrefix sets the package-level key prefix.
func SetKeyPrefix(prefix string) {
keyPrefixMu.Lock()
defer keyPrefixMu.Unlock()
keyPrefix = prefix
}
// GetKeyPrefix returns the package-level key prefix.
func GetKeyPrefix() string {
keyPrefixMu.RLock()
defer keyPrefixMu.RUnlock()
return keyPrefix
}
// InitRedisWithConfig initializes the Redis client using the provided RedisConfig.
func InitRedisWithConfig(cfg RedisConfig) (redis.UniversalClient, error) {
if !cfg.Enabled {
log.Println("[Redis] is disabled, skipping Redis initialization")
return nil, nil
}
SetKeyPrefix(cfg.KeyPrefix)
var client redis.UniversalClient
if cfg.ClusterMode {
// Cluster 模式
client = redis.NewClusterClient(&redis.ClusterOptions{
Addrs: cfg.Addrs,
Username: cfg.Username,
Password: cfg.Password,
PoolSize: cfg.PoolSize,
MinIdleConns: cfg.MinIdleConn,
DialTimeout: time.Duration(cfg.DialTimeout) * time.Second,
ReadTimeout: time.Duration(cfg.ReadTimeout) * time.Second,
WriteTimeout: time.Duration(cfg.WriteTimeout) * time.Second,
MaxRetries: cfg.MaxRetries,
PoolTimeout: time.Duration(cfg.PoolTimeout) * time.Second,
ConnMaxIdleTime: time.Duration(cfg.ConnMaxIdleTime) * time.Second,
MaintNotificationsConfig: redisMaintNotificationsConfig(cfg.MaintNotifications),
})
log.Println("[Redis] initialized in Cluster mode")
} else {
// Standalone 或 Sentinel 模式
options := &redis.UniversalOptions{
Addrs: cfg.Addrs,
MasterName: cfg.MasterName, // 非空时启用 Sentinel
Username: cfg.Username,
Password: cfg.Password,
DB: cfg.DB,
PoolSize: cfg.PoolSize,
MinIdleConns: cfg.MinIdleConn,
DialTimeout: time.Duration(cfg.DialTimeout) * time.Second,
ReadTimeout: time.Duration(cfg.ReadTimeout) * time.Second,
WriteTimeout: time.Duration(cfg.WriteTimeout) * time.Second,
MaxRetries: cfg.MaxRetries,
PoolTimeout: time.Duration(cfg.PoolTimeout) * time.Second,
ConnMaxIdleTime: time.Duration(cfg.ConnMaxIdleTime) * time.Second,
MaintNotificationsConfig: redisMaintNotificationsConfig(cfg.MaintNotifications),
}
if cfg.MasterName != "" {
failoverClient := redis.NewFailoverClient(options.Failover())
failoverClient.Options().MaintNotificationsConfig = redisMaintNotificationsConfig(cfg.MaintNotifications)
client = failoverClient
log.Println("[Redis] initialized in Sentinel mode")
} else {
client = redis.NewUniversalClient(options)
log.Println("[Redis] initialized in Standalone mode")
}
}
// OpenTelemetry 追踪(UniversalClient 兼容)
if err := redisotel.InstrumentTracing(
client,
redisotel.WithAttributes(
attribute.String("db.instance", fmt.Sprintf("%v", cfg.DB)),
attribute.String("db.ip", strings.Join(cfg.Addrs, ",")),
attribute.String("db.system", "Redis"),
),
); err != nil {
return nil, fmt.Errorf("redis: init trace: %w", err)
}
// 测试连接
if err := client.Ping(context.Background()).Err(); err != nil {
return nil, fmt.Errorf("redis: ping: %w", err)
}
Redis = client
return client, nil
}
// SetRedisClient 设置包级 Redis 客户端(主要用于测试)
func SetRedisClient(client redis.UniversalClient) {
Redis = client
}
func redisMaintNotificationsConfig(enabled bool) *maintnotifications.Config {
mode := maintnotifications.ModeDisabled
if enabled {
mode = maintnotifications.ModeAuto
}
return &maintnotifications.Config{Mode: mode}
}
// PrefixedKey 返回带前缀的 Key
func PrefixedKey(key string) string {
prefix := GetKeyPrefix()
if prefix == "" {
return key
}
return prefix + key
}
// HSetJSON 将泛型数据序列化为 JSON 并设置到 Redis Hash
// ctx: 上下文
// hashKey: Redis Hash key
// fieldKey: Hash field key
// data: 要存储的数据(泛型)
func HSetJSON[T any](ctx context.Context, hashKey, fieldKey string, data T) error {
jsonData, err := json.Marshal(data)
if err != nil {
return err
}
if err := Redis.HSet(ctx, PrefixedKey(hashKey), fieldKey, jsonData).Err(); err != nil {
return fmt.Errorf(errRedisHashSetFailed, err)
}
return nil
}
// HDel removes one or more fields from a Redis Hash.
func HDel(ctx context.Context, hashKey string, fieldKeys ...string) error {
if Redis == nil || len(fieldKeys) == 0 {
return nil
}
if err := Redis.HDel(ctx, PrefixedKey(hashKey), fieldKeys...).Err(); err != nil {
return fmt.Errorf(errRedisHashDeleteFailed, err)
}
return nil
}
// HGetJSON 从 Redis Hash 获取数据并反序列化为泛型类型
// ctx: 上下文
// hashKey: Redis Hash key
// fieldKey: Hash field key
// data: 用于接收数据的指针(泛型)
func HGetJSON[T any](ctx context.Context, hashKey, fieldKey string, data *T) error {
val, err := Redis.HGet(ctx, PrefixedKey(hashKey), fieldKey).Result()
if err != nil {
return err
}
if err := json.Unmarshal([]byte(val), data); err != nil {
return fmt.Errorf(errUnmarshalDataFailed, err)
}
return nil
}
// GetJSON 从Redis获取数据并反序列化为泛型类型
// ctx: 上下文
// key: Redis key
// data: 用于接收数据的指针(泛型)
func GetJSON[T any](ctx context.Context, key string, data *T) error {
val, err := Redis.Get(ctx, PrefixedKey(key)).Bytes()
if err != nil {
return err
}
if err := json.Unmarshal(val, data); err != nil {
return fmt.Errorf(errUnmarshalDataFailed, err)
}
return nil
}
// SetJSON 将泛型数据序列化为JSON并设置到Redis
// ctx: 上下文
// key: Redis key
// data: 要存储的数据(泛型)
// expiration: 过期时间
func SetJSON[T any](ctx context.Context, key string, data T, expiration time.Duration) error {
jsonData, err := json.Marshal(data)
if err != nil {
return fmt.Errorf(errMarshalDataFailed, err)
}
if err := Redis.Set(ctx, PrefixedKey(key), jsonData, expiration).Err(); err != nil {
return fmt.Errorf(errRedisKeySetFailed, err)
}
return nil
}
+28
View File
@@ -0,0 +1,28 @@
// Copyright 2026 Arctel.net
// SPDX-License-Identifier: Apache-2.0
package cache
import (
"testing"
"github.com/redis/go-redis/v9/maintnotifications"
)
func TestRedisMaintNotificationsConfig(t *testing.T) {
for _, test := range []struct {
name string
enabled bool
want maintnotifications.Mode
}{
{name: "disabled by default", enabled: false, want: maintnotifications.ModeDisabled},
{name: "auto when enabled", enabled: true, want: maintnotifications.ModeAuto},
} {
t.Run(test.name, func(t *testing.T) {
cfg := redisMaintNotificationsConfig(test.enabled)
if cfg.Mode != test.want {
t.Fatalf("maintenance notifications mode = %v, want %v", cfg.Mode, test.want)
}
})
}
}
+100
View File
@@ -0,0 +1,100 @@
// Copyright 2026 Arctel.net
// SPDX-License-Identifier: Apache-2.0
package cache_memory
import (
"Wavelet/core"
"Wavelet/core/contracts"
"Wavelet/pkg/cache/ram"
"context"
"encoding/json"
"time"
)
type ramEntry struct {
data []byte
expireAt time.Time
}
type memoryCacheService struct {
ramCache *ram.Cache[string, ramEntry]
events *core.EventBus
}
func newMemoryCacheService(capacity int, events *core.EventBus) (*memoryCacheService, error) {
ramCache, err := ram.New[string, ramEntry](ram.Options{
MaximumSize: capacity,
})
if err != nil {
return nil, err
}
return &memoryCacheService{
ramCache: ramCache,
events: events,
}, nil
}
func (s *memoryCacheService) Get(_ context.Context, key string, target any) error {
if entry, ok := s.ramCache.GetIfPresent(key); ok {
if entry.expireAt.IsZero() || time.Now().Before(entry.expireAt) {
return json.Unmarshal(entry.data, target)
}
// Expired
s.ramCache.Invalidate(key)
}
return contracts.ErrCacheMiss
}
func (s *memoryCacheService) Set(_ context.Context, key string, value any, ttl time.Duration) error {
data, err := json.Marshal(value)
if err != nil {
return err
}
var expireAt time.Time
if ttl > 0 {
expireAt = time.Now().Add(ttl)
}
s.ramCache.Set(key, ramEntry{
data: data,
expireAt: expireAt,
})
return nil
}
func (s *memoryCacheService) Delete(ctx context.Context, key string) error {
s.ramCache.Invalidate(key)
if s.events != nil {
_ = s.events.Emit(ctx, "cache:invalidate", key)
}
return nil
}
func (s *memoryCacheService) Invalidate(ctx context.Context, key string) error {
return s.Delete(ctx, key)
}
func (s *memoryCacheService) GetOrSet(ctx context.Context, key string, target any, ttl time.Duration, loader func() (any, error)) error {
err := s.Get(ctx, key, target)
if err == nil {
return nil
}
val, err := loader()
if err != nil {
return err
}
if setErr := s.Set(ctx, key, val, ttl); setErr != nil {
return setErr
}
data, err := json.Marshal(val)
if err != nil {
return err
}
return json.Unmarshal(data, target)
}
@@ -0,0 +1,83 @@
// Copyright 2026 Arctel.net
// SPDX-License-Identifier: Apache-2.0
// Package cache_memory provides the in-memory cache infrastructure plugin for Cordis.
package cache_memory
import (
"Wavelet/core"
"Wavelet/core/contracts"
)
const defaultRAMCapacity = 10000
// Option configures the cache_memory plugin.
type Option func(*Plugin)
// WithCapacity sets the maximum capacity for the in-memory cache.
func WithCapacity(capacity int) Option {
return func(p *Plugin) {
p.capacity = capacity
}
}
// Plugin implements core.Plugin to provide contracts.CacheService using in-memory storage.
type Plugin struct {
capacity int
}
// New creates a new in-memory cache infrastructure plugin.
func New(opts ...Option) *Plugin {
p := &Plugin{
capacity: defaultRAMCapacity,
}
for _, opt := range opts {
if opt != nil {
opt(p)
}
}
return p
}
// Name returns the unique identifier of the cache_memory plugin.
func (p *Plugin) Name() string {
return "cache_memory"
}
// Manifest returns the plugin metadata.
func (p *Plugin) Manifest() core.Manifest {
return core.Manifest{
Name: "cache_memory",
Version: "1.0.0",
Description: "Zero-dependency pure in-memory cache infrastructure plugin",
Author: "Wavelet Team",
}
}
// redisGateConfig declares the Redis gate configuration for cache_memory.
type redisGateConfig struct {
Enabled bool `config:"enabled" env:"REDIS_ENABLED" default:"false" autoEnable:"REDIS_ADDR"`
}
// DeclareConfig declares the configuration bindings consumed by this plugin.
func (p *Plugin) DeclareConfig() []core.ConfigBinding {
return []core.ConfigBinding{
{Prefix: "redis", Target: &redisGateConfig{}},
}
}
// ConfigEnabled gates plugin activation when Redis is disabled.
func (p *Plugin) ConfigEnabled(view core.ConfigView) bool {
return !view.Bool("redis.enabled", false)
}
// Apply mounts the in-memory cache service into the Context.
func (p *Plugin) Apply(ctx *core.Context) error {
svc, err := newMemoryCacheService(p.capacity, ctx.Events())
if err != nil {
return err
}
core.Provide[contracts.CacheService](ctx, svc)
return nil
}
@@ -0,0 +1,81 @@
// Copyright 2026 Arctel.net
// SPDX-License-Identifier: Apache-2.0
package cache_memory_test
import (
"Wavelet/core"
"Wavelet/core/contracts"
"Wavelet/plugins/infra/cache_memory"
"context"
"testing"
"time"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
)
func TestCacheMemoryPlugin(t *testing.T) {
ctx := core.NewContext(context.Background())
p := cache_memory.New(cache_memory.WithCapacity(100))
assert.Equal(t, "cache_memory", p.Name())
assert.Equal(t, "1.0.0", p.Manifest().Version)
require.NoError(t, p.Apply(ctx))
cacheSvc, err := core.Inject[contracts.CacheService](ctx)
require.NoError(t, err)
require.NotNil(t, cacheSvc)
reqCtx := context.Background()
// 1. Get non-existent
var target string
err = cacheSvc.Get(reqCtx, "non_existent", &target)
assert.ErrorIs(t, err, contracts.ErrCacheMiss)
// 2. Set and Get
err = cacheSvc.Set(reqCtx, "user:100", "Alice", 10*time.Minute)
require.NoError(t, err)
var name string
err = cacheSvc.Get(reqCtx, "user:100", &name)
require.NoError(t, err)
assert.Equal(t, "Alice", name)
// 3. GetOrSet
var count int
var loaded int
loader := func() (any, error) {
loaded++
return 42, nil
}
err = cacheSvc.GetOrSet(reqCtx, "counter", &count, 10*time.Minute, loader)
require.NoError(t, err)
assert.Equal(t, 42, count)
assert.Equal(t, 1, loaded)
// Second GetOrSet hits cache
var count2 int
err = cacheSvc.GetOrSet(reqCtx, "counter", &count2, 10*time.Minute, loader)
require.NoError(t, err)
assert.Equal(t, 42, count2)
assert.Equal(t, 1, loaded)
// 4. Delete & Invalidate
require.NoError(t, cacheSvc.Delete(reqCtx, "user:100"))
err = cacheSvc.Get(reqCtx, "user:100", &name)
assert.ErrorIs(t, err, contracts.ErrCacheMiss)
require.NoError(t, cacheSvc.Invalidate(reqCtx, "counter"))
err = cacheSvc.Get(reqCtx, "counter", &count)
assert.ErrorIs(t, err, contracts.ErrCacheMiss)
// 5. Expiration
err = cacheSvc.Set(reqCtx, "temp_key", "val", 50*time.Millisecond)
require.NoError(t, err)
time.Sleep(70 * time.Millisecond)
var tempVal string
err = cacheSvc.Get(reqCtx, "temp_key", &tempVal)
assert.ErrorIs(t, err, contracts.ErrCacheMiss)
}
+126
View File
@@ -0,0 +1,126 @@
// Copyright 2026 Arctel.net
// SPDX-License-Identifier: Apache-2.0
// Package config adapts viper to the kernel configuration source contract. It is a
// runtime adapter rather than a core.Plugin: it owns no routes, services or tasks and
// therefore never appears in app.Use. Keeping viper here preserves the micro-kernel
// rule against importing concrete runtime dependencies.
package config
import (
"errors"
"fmt"
"io/fs"
"os"
"github.com/spf13/viper"
)
// DefaultFileName is the configuration file looked up when CONFIG_PATH is unset.
const DefaultFileName = "config.yaml"
// EnvOnlyOrigin is reported by Describe when no configuration file was loaded.
const EnvOnlyOrigin = "<env only>"
// maxSearchDepth bounds the upward directory walk so a misconfigured working directory
// cannot make the loader scan the whole filesystem.
const maxSearchDepth = 5
// Option configures a Source.
type Option func(*Source)
// WithPath pins the configuration file, bypassing CONFIG_PATH and the upward search.
func WithPath(path string) Option {
return func(s *Source) {
s.path = path
}
}
// Source implements core.ConfigSource over a configuration file plus the process environment.
type Source struct {
v *viper.Viper
path string
found bool
}
// NewSource loads the configuration file. A missing file is not an error: the source
// then serves environment values only, matching the behaviour the previous pkg/config
// loader had for deployments that configure everything through the environment.
func NewSource(opts ...Option) (*Source, error) {
s := &Source{}
for _, opt := range opts {
opt(s)
}
if s.path == "" {
s.path = os.Getenv("CONFIG_PATH")
}
if s.path == "" {
s.path = findConfigPath(DefaultFileName)
}
v := viper.New()
v.SetConfigFile(s.path)
err := v.ReadInConfig()
switch {
case err == nil:
s.found = true
case isNotFound(err):
// No file: fall through to environment-only lookups.
default:
if _, statErr := os.Stat(s.path); statErr == nil { //nolint:gosec // s.path comes from CONFIG_PATH or a bounded upward search
return nil, fmt.Errorf("infra/config: read %s: %w", s.path, err)
}
}
s.v = v
return s, nil
}
// isNotFound reports whether the loader failed only because the file is absent.
func isNotFound(err error) bool {
var notFound viper.ConfigFileNotFoundError
return errors.As(err, &notFound) || errors.Is(err, fs.ErrNotExist)
}
// Lookup returns the raw value stored at a dotted path, or false when the file was not
// loaded or the path is absent. Declared defaults therefore stay distinguishable from
// values explicitly set to a zero.
func (s *Source) Lookup(path string) (any, bool) {
if !s.found || !s.v.IsSet(path) {
return nil, false
}
return s.v.Get(path), true
}
// LookupEnv reads a process environment variable.
func (s *Source) LookupEnv(name string) (string, bool) {
return os.LookupEnv(name)
}
// Describe returns the loaded file path, or EnvOnlyOrigin when running on environment values.
func (s *Source) Describe() string {
if !s.found {
return EnvOnlyOrigin
}
return s.path
}
// findConfigPath searches upward from the working directory so tests and binaries run
// from backend/ still find the repository-root configuration file.
func findConfigPath(configPath string) string {
if _, err := os.Stat(configPath); err == nil {
return configPath
}
dir := "."
for range maxSearchDepth {
dir += "/.."
path := dir + "/" + configPath
if _, err := os.Stat(path); err == nil {
return path
}
}
return configPath
}
@@ -0,0 +1,93 @@
// Copyright 2026 Arctel.net
// SPDX-License-Identifier: Apache-2.0
package config_test
import (
"os"
"path/filepath"
"testing"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
"Wavelet/plugins/infra/config"
)
const sampleYAML = "" +
"app:\n addr: \":8000\"\n node_id: 1\n" +
"database:\n enabled: false\n port: 5432\n slow_threshold: 200ms\n" +
"redis:\n addrs:\n - \"127.0.0.1:6379\"\n"
func writeConfig(t *testing.T, body string) string {
t.Helper()
path := filepath.Join(t.TempDir(), "config.yaml")
require.NoError(t, os.WriteFile(path, []byte(body), 0o600))
return path
}
func TestSourceLooksUpNestedPaths(t *testing.T) {
src, err := config.NewSource(config.WithPath(writeConfig(t, sampleYAML)))
require.NoError(t, err)
value, ok := src.Lookup("database.port")
require.True(t, ok)
assert.Equal(t, 5432, value)
_, ok = src.Lookup("database.missing")
assert.False(t, ok)
}
func TestSourceKeepsZeroValuedKeysDistinctFromMissing(t *testing.T) {
src, err := config.NewSource(config.WithPath(writeConfig(t, sampleYAML)))
require.NoError(t, err)
value, ok := src.Lookup("database.enabled")
require.True(t, ok, "an explicitly set false must not look like a missing key")
assert.Equal(t, false, value)
}
func TestSourceTreatsUnsetFileAsEnvOnly(t *testing.T) {
missing := filepath.Join(t.TempDir(), "absent.yaml")
src, err := config.NewSource(config.WithPath(missing))
require.NoError(t, err, "a missing configuration file must fall back to environment values")
_, ok := src.Lookup("app.addr")
assert.False(t, ok)
assert.Equal(t, config.EnvOnlyOrigin, src.Describe())
}
func TestSourceRejectsMalformedFile(t *testing.T) {
src, err := config.NewSource(config.WithPath(writeConfig(t, "app: [unclosed\n")))
assert.Nil(t, src)
require.Error(t, err)
assert.Contains(t, err.Error(), "infra/config")
}
func TestSourceLookupEnvReadsProcessEnvironment(t *testing.T) {
t.Setenv("WAVELET_SOURCE_PROBE", "present")
src, err := config.NewSource(config.WithPath(writeConfig(t, sampleYAML)))
require.NoError(t, err)
value, ok := src.LookupEnv("WAVELET_SOURCE_PROBE")
require.True(t, ok)
assert.Equal(t, "present", value)
_, ok = src.LookupEnv("WAVELET_SOURCE_ABSENT")
assert.False(t, ok)
}
func TestSourcePrefersConfigPathEnvironmentVariable(t *testing.T) {
t.Setenv("CONFIG_PATH", writeConfig(t, "app:\n addr: \":9100\"\n"))
src, err := config.NewSource()
require.NoError(t, err)
value, ok := src.Lookup("app.addr")
require.True(t, ok)
assert.Equal(t, ":9100", value)
}
@@ -0,0 +1,150 @@
// Copyright 2026 Arctel.net
// SPDX-License-Identifier: Apache-2.0
// Package database 提供数据库连接与基础设施
package database
import (
"context"
"fmt"
"log"
"net/url"
"strconv"
"strings"
"time"
"github.com/ClickHouse/clickhouse-go/v2"
"github.com/ClickHouse/clickhouse-go/v2/lib/driver"
"go.opentelemetry.io/otel/attribute"
clickhouseDriver "gorm.io/driver/clickhouse"
"gorm.io/gorm"
"gorm.io/plugin/opentelemetry/tracing"
)
const (
clickhouseMaxExecTime = 60 // ClickHouse 最大执行时间(秒)
clickhouseReadTimeoutFactor = 2 // ReadTimeout 为 DialTimeout 的倍数
)
var (
// ChConn ClickHouse 原生连接实例,用于批量写入
ChConn driver.Conn
chDB *gorm.DB
)
// InitClickHouseWithConfig initializes the ClickHouse connection using the provided configuration.
func InitClickHouseWithConfig(cfg ClickHouseConfig) error {
if !cfg.Enabled {
return nil
}
if cfg.Database == "" {
return fmt.Errorf("[ClickHouse] database name is required (expected: wavelet)")
}
opts := buildClickHouseOptions(cfg)
var err error
ChConn, err = clickhouse.Open(opts)
if err != nil {
return fmt.Errorf("[ClickHouse] init connection failed: %w", err)
}
if err = ChConn.Ping(context.Background()); err != nil {
return fmt.Errorf("[ClickHouse] ping failed: %w", err)
}
chDB, err = gorm.Open(clickhouseDriver.New(clickhouseDriver.Config{
DSN: buildClickHouseDSN(cfg),
}), &gorm.Config{
SkipDefaultTransaction: true,
})
if err != nil {
return fmt.Errorf("[ClickHouse] init gorm connection failed: %w", err)
}
if err = chDB.Use(
tracing.NewPlugin(
tracing.WithoutMetrics(),
tracing.WithAttributes(
attribute.String("db.instance", cfg.Database),
attribute.String("db.system", "ClickHouse"),
),
),
); err != nil {
return fmt.Errorf("[ClickHouse] init trace failed: %w", err)
}
sqlDB, err := chDB.DB()
if err != nil {
return fmt.Errorf("[ClickHouse] load sql db failed: %w", err)
}
sqlDB.SetMaxIdleConns(cfg.MaxIdleConn)
sqlDB.SetMaxOpenConns(cfg.MaxOpenConn)
sqlDB.SetConnMaxLifetime(time.Duration(cfg.ConnMaxLifetime) * time.Second)
log.Println("[ClickHouse] connection established successfully")
return nil
}
func buildClickHouseOptions(cfg ClickHouseConfig) *clickhouse.Options {
return &clickhouse.Options{
Addr: cfg.Hosts,
Auth: clickhouse.Auth{
Database: cfg.Database,
Username: cfg.Username,
Password: cfg.Password,
},
Settings: clickhouse.Settings{
"max_execution_time": clickhouseMaxExecTime,
},
Compression: &clickhouse.Compression{
Method: clickhouse.CompressionLZ4,
},
DialTimeout: time.Duration(cfg.DialTimeout) * time.Second,
MaxOpenConns: cfg.MaxOpenConn,
MaxIdleConns: cfg.MaxIdleConn,
ConnMaxLifetime: time.Duration(cfg.ConnMaxLifetime) * time.Second,
ReadTimeout: time.Duration(cfg.DialTimeout*clickhouseReadTimeoutFactor) * time.Second,
BlockBufferSize: cfg.BlockBufferSize,
}
}
func buildClickHouseDSN(cfg ClickHouseConfig) string {
chURL := &url.URL{
Scheme: "clickhouse",
Host: strings.Join(cfg.Hosts, ","),
Path: "/" + cfg.Database,
}
if cfg.Username != "" || cfg.Password != "" {
chURL.User = url.UserPassword(cfg.Username, cfg.Password)
}
query := chURL.Query()
query.Set("dial_timeout", fmt.Sprintf("%ds", cfg.DialTimeout))
query.Set("read_timeout", fmt.Sprintf("%ds", cfg.DialTimeout*clickhouseReadTimeoutFactor))
query.Set("max_execution_time", strconv.Itoa(clickhouseMaxExecTime))
chURL.RawQuery = query.Encode()
return chURL.String()
}
// ChDB returns a context-aware GORM ClickHouse instance.
func ChDB(ctx context.Context) *gorm.DB {
if chDB == nil {
return nil
}
return chDB.WithContext(ctx)
}
// SetChDBForTest sets the package-level ClickHouse GORM instance for testing.
func SetChDBForTest(d *gorm.DB) {
chDB = d
}
// SetChConnForTest sets the package-level native ClickHouse connection for testing.
func SetChConnForTest(c driver.Conn) {
ChConn = c
}
+57
View File
@@ -0,0 +1,57 @@
// Copyright 2026 Arctel.net
// SPDX-License-Identifier: Apache-2.0
package database
import "time"
// ReplicaConfig declares a single read-replica configuration for PostgreSQL.
type ReplicaConfig struct {
Host string `config:"host"`
Port int `config:"port"`
Username string `config:"username"`
Password string `config:"password" secret:"true"`
}
// Config declares PostgreSQL and SQLite database configuration.
type Config struct {
Enabled bool `config:"enabled" env:"DB_ENABLED" default:"false" autoEnable:"DB_HOST"`
SQLitePath string `config:"sqlite_path" env:"SQLITE_PATH"`
Host string `config:"host" env:"DB_HOST"`
Port int `config:"port" env:"DB_PORT" default:"5432"`
Username string `config:"username" env:"DB_USERNAME"`
Password string `config:"password" env:"DB_PASSWORD" secret:"true"`
Database string `config:"database" env:"DB_NAME" default:"wavelet"`
MaxIdleConn int `config:"max_idle_conn" env:"DB_MAX_IDLE_CONN" default:"10"`
MaxOpenConn int `config:"max_open_conn" env:"DB_MAX_OPEN_CONN" default:"100"`
ConnMaxLifetime int `config:"conn_max_lifetime" env:"DB_CONN_MAX_LIFETIME" default:"3600"`
ConnMaxIdleTime int `config:"conn_max_idle_time" env:"DB_CONN_MAX_IDLE_TIME" default:"600"`
LogLevel string `config:"log_level" env:"DB_LOG_LEVEL" default:"warn"`
SSLMode string `config:"ssl_mode" env:"DB_SSL_MODE" default:"disable"`
TimeZone string `config:"time_zone" env:"DB_TIMEZONE" default:"UTC"`
ApplicationName string `config:"application_name" env:"DB_APPLICATION_NAME" default:"wavelet"`
SearchPath string `config:"search_path" env:"DB_SEARCH_PATH" default:"public"`
PreferSimpleProtocol bool `config:"prefer_simple_protocol" env:"DB_PREFER_SIMPLE_PROTOCOL"`
StatementCacheCapacity int `config:"statement_cache_capacity" env:"DB_STATEMENT_CACHE_CAPACITY"`
DefaultQueryExecMode string `config:"default_query_exec_mode" env:"DB_DEFAULT_QUERY_EXEC_MODE"`
Replicas []ReplicaConfig `config:"replicas"`
SlowThreshold time.Duration `config:"slow_threshold" env:"DB_SLOW_THRESHOLD" default:"200ms"`
}
// ClickHouseConfig declares the configuration for ClickHouse analytical storage.
type ClickHouseConfig struct {
Enabled bool `config:"enabled" env:"CLICKHOUSE_ENABLED" default:"false" autoEnable:"CLICKHOUSE_HOST"`
Hosts []string `config:"hosts" env:"CLICKHOUSE_HOST"`
Username string `config:"username" env:"CLICKHOUSE_USERNAME"`
Password string `config:"password" env:"CLICKHOUSE_PASSWORD" secret:"true"`
Database string `config:"database" env:"CLICKHOUSE_NAME" default:"wavelet"`
MaxIdleConn int `config:"max_idle_conn" env:"CLICKHOUSE_MAX_IDLE_CONN" default:"10"`
MaxOpenConn int `config:"max_open_conn" env:"CLICKHOUSE_MAX_OPEN_CONN" default:"50"`
ConnMaxLifetime int `config:"conn_max_lifetime" env:"CLICKHOUSE_CONN_MAX_LIFETIME" default:"3600"`
DialTimeout int `config:"dial_timeout" env:"CLICKHOUSE_DIAL_TIMEOUT" default:"10"`
BlockBufferSize uint8 `config:"block_buffer_size" env:"CLICKHOUSE_BLOCK_BUFFER_SIZE" default:"10"`
}
type appEnvConfig struct {
Env string `config:"env" env:"APP_ENV" default:"development"`
}
+144
View File
@@ -0,0 +1,144 @@
// Copyright 2026 Arctel.net
// SPDX-License-Identifier: Apache-2.0
// Package database provides the relational database infrastructure plugin for Cordis.
package database
import (
"Wavelet/core"
"Wavelet/core/contracts"
"context"
"gorm.io/gorm"
)
// Option configures the database plugin.
type Option func(*Plugin)
// WithDB configures an explicit *gorm.DB instance for the plugin.
func WithDB(d *gorm.DB) Option {
return func(p *Plugin) {
p.db = d
}
}
// WithNamedDB registers a named secondary database connection.
func WithNamedDB(name string, d *gorm.DB) Option {
return func(p *Plugin) {
if p.namedDBs == nil {
p.namedDBs = make(map[string]*gorm.DB)
}
p.namedDBs[name] = d
}
}
// Plugin implements core.Plugin to provide contracts.DBService into the Cordis micro-kernel.
type Plugin struct {
db *gorm.DB
namedDBs map[string]*gorm.DB
}
// New creates a new database infrastructure plugin.
func New(opts ...Option) *Plugin {
p := &Plugin{
namedDBs: make(map[string]*gorm.DB),
}
for _, opt := range opts {
if opt != nil {
opt(p)
}
}
return p
}
// Name returns the unique identifier of the database plugin.
func (p *Plugin) Name() string {
return "database"
}
// DeclareConfig declares database configuration keys.
func (p *Plugin) DeclareConfig() []core.ConfigBinding {
return []core.ConfigBinding{
{Prefix: "database", Target: &Config{}},
{Prefix: "clickhouse", Target: &ClickHouseConfig{}},
{Prefix: "app", Target: &appEnvConfig{}},
}
}
// Apply mounts the database service into the Context.
func (p *Plugin) Apply(ctx *core.Context) error {
var dbCfg Config
if err := ctx.Config().Bind("database", &dbCfg); err != nil {
return err
}
var chCfg ClickHouseConfig
if err := ctx.Config().Bind("clickhouse", &chCfg); err != nil {
return err
}
var appCfg appEnvConfig
_ = ctx.Config().Bind("app", &appCfg)
targetDB := p.db
if targetDB == nil {
var err error
targetDB, err = InitDBWithConfig(dbCfg, appCfg.Env == "production" || appCfg.Env == "prod")
if err != nil {
return err
}
}
if chCfg.Enabled {
if err := InitClickHouseWithConfig(chCfg); err != nil {
return err
}
}
svc := &dbServiceImpl{
primary: targetDB,
namedDBs: p.namedDBs,
}
if sqlDB, err := targetDB.DB(); err == nil && sqlDB != nil {
ctx.OnDispose(func() error {
return sqlDB.Close()
})
}
core.Provide[contracts.DBService](ctx, svc)
return nil
}
// NewService wraps a GORM DB instance into a contracts.DBService.
func NewService(primary *gorm.DB) contracts.DBService {
return &dbServiceImpl{primary: primary}
}
type dbServiceImpl struct {
primary *gorm.DB
namedDBs map[string]*gorm.DB
}
func (s *dbServiceImpl) GORM() *gorm.DB {
if s.primary != nil {
return s.primary
}
return DB(context.Background())
}
func (s *dbServiceImpl) DB(ctx context.Context) *gorm.DB {
if s.primary != nil {
return s.primary.WithContext(ctx)
}
return DB(ctx)
}
func (s *dbServiceImpl) Named(name string) *gorm.DB {
if s.namedDBs != nil {
if d, ok := s.namedDBs[name]; ok && d != nil {
return d
}
}
return s.GORM()
}
@@ -0,0 +1,63 @@
// Copyright 2026 Arctel.net
// SPDX-License-Identifier: Apache-2.0
package database_test
import (
"Wavelet/core"
"Wavelet/core/contracts"
"Wavelet/plugins/infra/database"
"context"
"path/filepath"
"testing"
"github.com/glebarez/sqlite"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
"gorm.io/gorm"
)
type Item struct {
ID uint64 `gorm:"primaryKey"`
Name string
}
func TestDatabasePlugin(t *testing.T) {
dbPath := filepath.Join(t.TempDir(), "db_test.db")
gdb, err := gorm.Open(sqlite.Open(dbPath), &gorm.Config{})
require.NoError(t, err)
require.NoError(t, gdb.AutoMigrate(&Item{}))
namedPath := filepath.Join(t.TempDir(), "named_test.db")
namedDB, err := gorm.Open(sqlite.Open(namedPath), &gorm.Config{})
require.NoError(t, err)
require.NoError(t, namedDB.AutoMigrate(&Item{}))
p := database.New(
database.WithDB(gdb),
database.WithNamedDB("analytics", namedDB),
)
ctx := core.NewContext(context.Background())
ctx.Config().SetSource(core.NewMapSource(map[string]any{
"database.enabled": false,
}))
require.NoError(t, ctx.Config().Resolve())
require.NoError(t, p.Apply(ctx))
svc, err := core.Inject[contracts.DBService](ctx)
require.NoError(t, err)
require.NotNil(t, svc)
assert.Equal(t, gdb, svc.GORM())
assert.NotNil(t, svc.DB(context.Background()))
assert.Equal(t, namedDB, svc.Named("analytics"))
assert.Equal(t, gdb, svc.Named("non_existent"))
// Verify DB write
item := Item{ID: 1, Name: "TestItem"}
require.NoError(t, svc.DB(context.Background()).Create(&item).Error)
var retrieved Item
require.NoError(t, svc.GORM().First(&retrieved, 1).Error)
assert.Equal(t, "TestItem", retrieved.Name)
}
+227
View File
@@ -0,0 +1,227 @@
// Copyright 2026 Arctel.net
// SPDX-License-Identifier: Apache-2.0
package database
import (
"context"
"fmt"
"log"
"net"
"net/url"
"os"
"path/filepath"
"strconv"
"strings"
"time"
"github.com/glebarez/sqlite"
"go.opentelemetry.io/otel/attribute"
"gorm.io/driver/postgres"
"gorm.io/gorm"
"gorm.io/plugin/dbresolver"
"gorm.io/plugin/opentelemetry/tracing"
)
var db *gorm.DB
const sqliteDirMode = 0o750
// InitDB 初始化主数据库实例(支持 PostgreSQL / SQLite)
func InitDB() (*gorm.DB, error) {
return InitDBWithConfig(Config{}, false)
}
// InitDBWithConfig initializes the main database with the provided config.
func InitDBWithConfig(cfg Config, isProd bool) (*gorm.DB, error) {
if !cfg.Enabled {
return initSQLiteWithConfig(cfg, isProd)
}
return initPostgresWithConfig(cfg, isProd)
}
func initSQLiteWithConfig(cfg Config, isProd bool) (*gorm.DB, error) {
sqlitePath := cfg.SQLitePath
if sqlitePath == "" {
sqlitePath = "./data/wavelet.db"
}
if sqlitePath != ":memory:" && !strings.HasPrefix(sqlitePath, "file:") {
if dir := filepath.Dir(sqlitePath); dir != "" && dir != "." {
if err := os.MkdirAll(dir, sqliteDirMode); err != nil {
return nil, fmt.Errorf("create sqlite directory %q failed: %w", dir, err)
}
}
}
targetDB, err := gorm.Open(sqlite.Open(sqlitePath), &gorm.Config{
DisableForeignKeyConstraintWhenMigrating: true,
Logger: &gormZapLogger{
logLevel: parseLogLevel(cfg.LogLevel),
slowThreshold: cfg.SlowThreshold,
ignoreRecordNotFoundError: isProd,
},
})
if err != nil {
return nil, err
}
// Trace 注入
if err = targetDB.Use(
tracing.NewPlugin(
tracing.WithoutMetrics(),
tracing.WithAttributes(
attribute.String("db.instance", sqlitePath),
attribute.String("db.system", "SQLite"),
),
),
); err != nil {
return nil, err
}
db = targetDB
log.Printf("[SQLite] initialized (path: %s)\n", sqlitePath)
return targetDB, nil
}
func initPostgresWithConfig(cfg Config, isProd bool) (*gorm.DB, error) {
// 构建主库 DSN 并连接
primaryDSN := buildDSN(cfg, cfg.Host, cfg.Port, cfg.Username, cfg.Password)
pgConfig := postgres.Config{
DSN: primaryDSN,
PreferSimpleProtocol: cfg.PreferSimpleProtocol,
}
targetDB, err := gorm.Open(postgres.New(pgConfig), &gorm.Config{
DisableForeignKeyConstraintWhenMigrating: true,
Logger: &gormZapLogger{
logLevel: parseLogLevel(cfg.LogLevel),
slowThreshold: cfg.SlowThreshold,
ignoreRecordNotFoundError: isProd,
},
})
if err != nil {
return nil, err
}
// Trace 注入
if err = targetDB.Use(
tracing.NewPlugin(
tracing.WithoutMetrics(),
tracing.WithAttributes(
attribute.String("db.instance", cfg.Database),
attribute.String("db.ip", cfg.Host),
attribute.String("server.address", net.JoinHostPort(cfg.Host, strconv.Itoa(cfg.Port))),
attribute.String("db.system", "PostgreSQL"),
),
),
); err != nil {
return nil, err
}
if len(cfg.Replicas) > 0 {
var replicaDialectors []gorm.Dialector
for _, replica := range cfg.Replicas {
username := replica.Username
if username == "" {
username = cfg.Username
}
password := replica.Password
if password == "" {
password = cfg.Password
}
replicaDSN := buildDSN(cfg, replica.Host, replica.Port, username, password)
replicaDialectors = append(replicaDialectors, postgres.New(postgres.Config{
DSN: replicaDSN,
PreferSimpleProtocol: cfg.PreferSimpleProtocol,
}))
}
resolver := dbresolver.Register(dbresolver.Config{
Replicas: replicaDialectors,
Policy: dbresolver.RandomPolicy{},
})
resolver.SetMaxIdleConns(cfg.MaxIdleConn).
SetMaxOpenConns(cfg.MaxOpenConn).
SetConnMaxLifetime(time.Duration(cfg.ConnMaxLifetime) * time.Second).
SetConnMaxIdleTime(time.Duration(cfg.ConnMaxIdleTime) * time.Second)
if err = targetDB.Use(resolver); err != nil {
return nil, err
}
log.Printf("[PostgreSQL] initialized in Primary-Replica mode (%d replicas)\n", len(cfg.Replicas))
} else {
log.Println("[PostgreSQL] initialized in Standalone mode")
}
// 获取通用数据库对象设置连接池
sqlDB, err := targetDB.DB()
if err != nil {
return nil, err
}
sqlDB.SetMaxIdleConns(cfg.MaxIdleConn)
sqlDB.SetMaxOpenConns(cfg.MaxOpenConn)
sqlDB.SetConnMaxLifetime(time.Duration(cfg.ConnMaxLifetime) * time.Second)
sqlDB.SetConnMaxIdleTime(time.Duration(cfg.ConnMaxIdleTime) * time.Second)
db = targetDB
return targetDB, nil
}
// buildDSN 构建 PostgreSQL DSN
func buildDSN(cfg Config, host string, port int, username, password string) string {
pqURL := &url.URL{
Scheme: "postgres",
Host: net.JoinHostPort(host, strconv.Itoa(port)),
Path: cfg.Database,
}
if username != "" {
pqURL.User = url.UserPassword(username, password)
}
query := pqURL.Query()
sslMode := cfg.SSLMode
if sslMode == "" {
sslMode = "disable"
}
query.Set("sslmode", sslMode)
if cfg.ApplicationName != "" {
query.Set("application_name", cfg.ApplicationName)
}
if cfg.SearchPath != "" {
query.Set("search_path", cfg.SearchPath)
}
if cfg.DefaultQueryExecMode != "" {
query.Set("default_query_exec_mode", cfg.DefaultQueryExecMode)
}
if cfg.StatementCacheCapacity > 0 {
query.Set("statement_cache_capacity", strconv.Itoa(cfg.StatementCacheCapacity))
}
rawQuery := query.Encode()
if cfg.TimeZone != "" {
if rawQuery != "" {
rawQuery += "&"
}
rawQuery += "TimeZone=" + cfg.TimeZone
}
pqURL.RawQuery = rawQuery
return pqURL.String()
}
// DB 返回带上下文追踪的 GORM 数据库实例
func DB(ctx context.Context) *gorm.DB {
if db == nil {
return nil
}
return db.WithContext(ctx)
}
// SetDB sets the package-level database instance for testing.
func SetDB(d *gorm.DB) {
db = d
}
@@ -0,0 +1,90 @@
// Copyright 2026 Arctel.net
// SPDX-License-Identifier: Apache-2.0
package database
import (
"Wavelet/pkg/logger"
"context"
"errors"
"fmt"
"strings"
"time"
"gorm.io/gorm"
gormLogger "gorm.io/gorm/logger"
)
// nanoToMilli 纳秒转毫秒的除数
const nanoToMilli = 1e6
type gormZapLogger struct {
logLevel gormLogger.LogLevel
ignoreRecordNotFoundError bool
slowThreshold time.Duration
}
func (l *gormZapLogger) LogMode(level gormLogger.LogLevel) gormLogger.Interface {
clone := *l
clone.logLevel = level
return &clone
}
func (l *gormZapLogger) Info(ctx context.Context, fmt string, args ...interface{}) {
if l.logLevel >= gormLogger.Info {
logger.InfoF(ctx, fmt, args...)
}
}
func (l *gormZapLogger) Warn(ctx context.Context, fmt string, args ...interface{}) {
if l.logLevel >= gormLogger.Warn {
logger.WarnF(ctx, fmt, args...)
}
}
func (l *gormZapLogger) Error(ctx context.Context, fmt string, args ...interface{}) {
if l.logLevel >= gormLogger.Error {
logger.ErrorF(ctx, fmt, args...)
}
}
func (l *gormZapLogger) Trace(ctx context.Context, begin time.Time, fc func() (sql string, rowsAffected int64), err error) {
elapsed := time.Since(begin)
switch {
case err != nil && l.logLevel >= gormLogger.Error && (!errors.Is(err, gorm.ErrRecordNotFound) || !l.ignoreRecordNotFoundError):
_, rows := fc()
logger.ErrorF(ctx, "database query failed: %s [%.3fms] [rows:%v]", err, float64(elapsed.Nanoseconds())/nanoToMilli, formatRows(rows))
case elapsed > l.slowThreshold && l.slowThreshold != 0 && l.logLevel >= gormLogger.Warn:
_, rows := fc()
slowLog := fmt.Sprintf("SLOW SQL >= %v", l.slowThreshold)
logger.WarnF(ctx, "%s [%.3fms] [rows:%v]", slowLog, float64(elapsed.Nanoseconds())/nanoToMilli, formatRows(rows))
case l.logLevel == gormLogger.Info:
sql, rows := fc()
logger.DebugF(ctx, "[%.3fms] [rows:%v] %s", float64(elapsed.Nanoseconds())/nanoToMilli, formatRows(rows), sql)
}
}
func formatRows(rows int64) interface{} {
if rows == -1 {
return "-"
}
return rows
}
func parseLogLevel(level string) gormLogger.LogLevel {
level = strings.ToLower(level)
switch level {
case "silent":
return gormLogger.Silent
case "error":
return gormLogger.Error
case "warn":
return gormLogger.Warn
case "info":
return gormLogger.Info
case "debug":
return gormLogger.Info
default:
return gormLogger.Info
}
}
@@ -0,0 +1,39 @@
// Copyright 2026 Arctel.net
// SPDX-License-Identifier: Apache-2.0
package database
import (
"testing"
gormLogger "gorm.io/gorm/logger"
)
func TestParseLogLevel(t *testing.T) {
t.Parallel()
tests := []struct {
name string
configuredLevel string
want gormLogger.LogLevel
}{
{
name: "debug enables SQL trace processing",
configuredLevel: "debug",
want: gormLogger.Info,
},
{
name: "development preserves configured level",
configuredLevel: "warn",
want: gormLogger.Warn,
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
if got := parseLogLevel(tt.configuredLevel); got != tt.want {
t.Fatalf("parseLogLevel() = %v, want %v", got, tt.want)
}
})
}
}
+343
View File
@@ -0,0 +1,343 @@
// Copyright 2026 Arctel.net
// SPDX-License-Identifier: Apache-2.0
package infra_test
import (
"Wavelet/core"
"Wavelet/core/contracts"
"Wavelet/plugins/infra/cache"
"Wavelet/plugins/infra/database"
"Wavelet/plugins/infra/logger"
"Wavelet/plugins/infra/storage"
"Wavelet/plugins/infra/storage/objectstore"
"bytes"
"context"
"io"
"path/filepath"
"sync"
"testing"
"time"
"github.com/alicebob/miniredis/v2"
"github.com/glebarez/sqlite"
"github.com/redis/go-redis/v9"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
"gorm.io/gorm"
)
type TestUser struct {
ID uint64 `gorm:"primaryKey"`
Name string
}
func setupTestDB(t *testing.T) *gorm.DB {
t.Helper()
dbPath := filepath.Join(t.TempDir(), "test.db")
db, err := gorm.Open(sqlite.Open(dbPath), &gorm.Config{})
require.NoError(t, err)
require.NoError(t, db.AutoMigrate(&TestUser{}))
return db
}
func prepareTestContext(values map[string]any, declarers ...core.Plugin) *core.Context {
ctx := core.NewContext(context.Background())
ctx.Config().SetSource(core.NewMapSource(values))
for _, p := range declarers {
if d, ok := p.(interface{ DeclareConfig() []core.ConfigBinding }); ok {
for _, b := range d.DeclareConfig() {
_ = ctx.Config().Declare(p.Name(), b)
}
}
}
_ = ctx.Config().Resolve()
return ctx
}
func TestDatabasePlugin(t *testing.T) {
testDB := setupTestDB(t)
p := database.New(database.WithDB(testDB))
ctx := prepareTestContext(nil, p)
require.Equal(t, "database", p.Name())
require.NoError(t, p.Apply(ctx))
dbSvc, err := core.Inject[contracts.DBService](ctx)
require.NoError(t, err)
require.NotNil(t, dbSvc)
// Verify GORM and DB methods
assert.NotNil(t, dbSvc.GORM())
assert.NotNil(t, dbSvc.DB(context.Background()))
// Test CRUD via service
user := TestUser{ID: 1, Name: "Alice"}
require.NoError(t, dbSvc.DB(context.Background()).Create(&user).Error)
var fetched TestUser
require.NoError(t, dbSvc.GORM().First(&fetched, 1).Error)
assert.Equal(t, "Alice", fetched.Name)
// Test NamedDB fallback
assert.NotNil(t, dbSvc.Named("replica"))
}
func TestCachePluginRAMOnly(t *testing.T) {
p := cache.New()
ctx := prepareTestContext(nil, p)
require.Equal(t, "cache", p.Name())
require.NoError(t, p.Apply(ctx))
cacheSvc, err := core.Inject[contracts.CacheService](ctx)
require.NoError(t, err)
require.NotNil(t, cacheSvc)
type CacheItem struct {
Name string `json:"name"`
Count int `json:"count"`
}
testCtx := context.Background()
// 1. Get non-existing key
var notFound CacheItem
err = cacheSvc.Get(testCtx, "missing:key", &notFound)
assert.ErrorIs(t, err, contracts.ErrCacheMiss)
// 2. Set and Get
item := CacheItem{Name: "item1", Count: 42}
require.NoError(t, cacheSvc.Set(testCtx, "test:item1", item, time.Minute))
var retrieved CacheItem
require.NoError(t, cacheSvc.Get(testCtx, "test:item1", &retrieved))
assert.Equal(t, item, retrieved)
// 3. GetOrSet
var getOrSetTarget CacheItem
var loaderCalled bool
err = cacheSvc.GetOrSet(testCtx, "test:item1", &getOrSetTarget, time.Minute, func() (any, error) {
loaderCalled = true
return CacheItem{Name: "never_called", Count: 0}, nil
})
require.NoError(t, err)
assert.False(t, loaderCalled)
assert.Equal(t, item, getOrSetTarget)
// GetOrSet with cache miss
var newItem CacheItem
err = cacheSvc.GetOrSet(testCtx, "test:item2", &newItem, time.Minute, func() (any, error) {
loaderCalled = true
return CacheItem{Name: "loaded", Count: 99}, nil
})
require.NoError(t, err)
assert.True(t, loaderCalled)
assert.Equal(t, "loaded", newItem.Name)
assert.Equal(t, 99, newItem.Count)
// 4. Delete
require.NoError(t, cacheSvc.Delete(testCtx, "test:item1"))
err = cacheSvc.Get(testCtx, "test:item1", &retrieved)
assert.ErrorIs(t, err, contracts.ErrCacheMiss)
// 5. Invalidate alias
require.NoError(t, cacheSvc.Invalidate(testCtx, "test:item2"))
err = cacheSvc.Get(testCtx, "test:item2", &retrieved)
assert.ErrorIs(t, err, contracts.ErrCacheMiss)
}
func TestCachePluginWithRedisAndPubSub(t *testing.T) {
mr, err := miniredis.Run()
require.NoError(t, err)
defer mr.Close()
rdb := redis.NewClient(&redis.Options{
Addr: mr.Addr(),
})
defer func() { _ = rdb.Close() }()
p1 := cache.New(cache.WithRedis(rdb), cache.WithKeyPrefix("test:"))
p2 := cache.New(cache.WithRedis(rdb), cache.WithKeyPrefix("test:"))
ctx1 := prepareTestContext(map[string]any{"redis.enabled": true}, p1)
ctx2 := prepareTestContext(map[string]any{"redis.enabled": true}, p2)
require.NoError(t, p1.Apply(ctx1))
require.NoError(t, p2.Apply(ctx2))
cache1, err := core.Inject[contracts.CacheService](ctx1)
require.NoError(t, err)
cache2, err := core.Inject[contracts.CacheService](ctx2)
require.NoError(t, err)
testCtx := context.Background()
// Node 1 writes to cache
type UserCache struct {
Name string `json:"name"`
}
require.NoError(t, cache1.Set(testCtx, "user:100", UserCache{Name: "Bob"}, 10*time.Minute))
// Node 2 reads from cache (misses Node 2's RAM, hits Redis, backfills Node 2's RAM)
var u2 UserCache
require.NoError(t, cache2.Get(testCtx, "user:100", &u2))
assert.Equal(t, "Bob", u2.Name)
// Node 1 deletes cache (evicts Node 1 RAM, Redis, and broadcasts to Node 2)
require.NoError(t, cache1.Delete(testCtx, "user:100"))
// Verify Redis is deleted
var uRedis UserCache
err = cache1.Get(testCtx, "user:100", &uRedis)
assert.ErrorIs(t, err, contracts.ErrCacheMiss)
// Clean up contexts
require.NoError(t, ctx1.Dispose())
require.NoError(t, ctx2.Dispose())
}
func TestLoggerPlugin(t *testing.T) {
ctx := core.NewContext(context.Background())
p := logger.New()
require.Equal(t, "logger", p.Name())
require.NoError(t, p.Apply(ctx))
logSvc, err := core.Inject[contracts.LoggerService](ctx)
require.NoError(t, err)
require.NotNil(t, logSvc)
testCtx := context.Background()
// Should not panic on any log call
logSvc.Debug(testCtx, "debug message", "key1", "val1")
logSvc.Info(testCtx, "info message", "userID", 123)
logSvc.Warn(testCtx, "warn message", "warning", true)
logSvc.Error(testCtx, "error message", "err", "something broke")
logSvc.Debugf(testCtx, "formatted debug %d", 1)
logSvc.Infof(testCtx, "formatted info %s", "test")
logSvc.Warnf(testCtx, "formatted warn %v", map[string]int{"a": 1})
logSvc.Errorf(testCtx, "formatted error %s", "fatal")
childLog := logSvc.With("module", "test_module")
require.NotNil(t, childLog)
childLog.Info(testCtx, "child log message", "action", "run")
}
type memoryBackend struct {
mu sync.RWMutex
storage map[string][]byte
}
func newMemoryBackend() *memoryBackend {
return &memoryBackend{
storage: make(map[string][]byte),
}
}
func (m *memoryBackend) Put(ctx context.Context, key string, body io.Reader, size int64, contentType string) (objectstore.PutResult, error) {
m.mu.Lock()
defer m.mu.Unlock()
data, err := io.ReadAll(body)
if err != nil {
return objectstore.PutResult{}, err
}
m.storage[key] = data
return objectstore.PutResult{Key: key, Bucket: "test-bucket"}, nil
}
func (m *memoryBackend) Get(ctx context.Context, key string) (*objectstore.Object, error) {
m.mu.RLock()
defer m.mu.RUnlock()
data, ok := m.storage[key]
if !ok {
return nil, gorm.ErrRecordNotFound
}
return &objectstore.Object{
Body: io.NopCloser(bytes.NewReader(data)),
ContentLength: int64(len(data)),
ContentType: "application/octet-stream",
}, nil
}
func (m *memoryBackend) Delete(ctx context.Context, key string) error {
m.mu.Lock()
defer m.mu.Unlock()
delete(m.storage, key)
return nil
}
func (m *memoryBackend) Test(ctx context.Context) error {
return nil
}
func TestStoragePlugin(t *testing.T) {
ctx := core.NewContext(context.Background())
backend := newMemoryBackend()
p := storage.New(storage.WithBackend(backend))
require.Equal(t, "storage", p.Name())
require.NoError(t, p.Apply(ctx))
storageSvc, err := core.Inject[contracts.StorageService](ctx)
require.NoError(t, err)
require.NotNil(t, storageSvc)
testCtx := context.Background()
// 1. Put
content := []byte("Hello, Wavelet Storage Plugin!")
putRes, err := storageSvc.Put(testCtx, "uploads/hello.txt", bytes.NewReader(content), int64(len(content)), "text/plain")
require.NoError(t, err)
assert.Equal(t, "uploads/hello.txt", putRes.Key)
assert.Equal(t, "test-bucket", putRes.Bucket)
// 2. Get
obj, err := storageSvc.Get(testCtx, "uploads/hello.txt")
require.NoError(t, err)
require.NotNil(t, obj)
data, err := io.ReadAll(obj.Body)
require.NoError(t, err)
assert.Equal(t, content, data)
assert.Equal(t, int64(len(content)), obj.ContentLength)
// 3. Delete
require.NoError(t, storageSvc.Delete(testCtx, "uploads/hello.txt"))
_, err = storageSvc.Get(testCtx, "uploads/hello.txt")
assert.Error(t, err)
}
func TestAllInfraPluginsCombined(t *testing.T) {
testDB := setupTestDB(t)
memBackend := newMemoryBackend()
dbP := database.New(database.WithDB(testDB))
cacheP := cache.New()
logP := logger.New()
storageP := storage.New(storage.WithBackend(memBackend))
ctx := prepareTestContext(nil, dbP, cacheP, logP, storageP)
require.NoError(t, dbP.Apply(ctx))
require.NoError(t, cacheP.Apply(ctx))
require.NoError(t, logP.Apply(ctx))
require.NoError(t, storageP.Apply(ctx))
// Using3 to resolve dependencies concurrently
var resolved bool
err := core.Using3(ctx, func(db contracts.DBService, c contracts.CacheService, l contracts.LoggerService) {
resolved = true
assert.NotNil(t, db)
assert.NotNil(t, c)
assert.NotNil(t, l)
})
require.NoError(t, err)
assert.True(t, resolved)
// Using storage
err = core.Using(ctx, func(s contracts.StorageService) {
assert.NotNil(t, s)
})
require.NoError(t, err)
}
+109
View File
@@ -0,0 +1,109 @@
// Copyright 2026 Arctel.net
// SPDX-License-Identifier: Apache-2.0
// Package logger provides the structured logging infrastructure plugin for Cordis.
package logger
import (
"Wavelet/core"
"Wavelet/core/contracts"
"Wavelet/pkg/logger"
"context"
"fmt"
"strings"
)
// Plugin implements core.Plugin to provide contracts.LoggerService.
type Plugin struct{}
// New creates a new logger infrastructure plugin.
func New() *Plugin {
return &Plugin{}
}
// Name returns the unique identifier of the logger plugin.
func (p *Plugin) Name() string {
return "logger"
}
// Apply mounts the logger service into the Context.
func (p *Plugin) Apply(ctx *core.Context) error {
svc := &loggerServiceImpl{}
core.Provide[contracts.LoggerService](ctx, svc)
return nil
}
type loggerServiceImpl struct {
extraFields []any
}
func (s *loggerServiceImpl) formatMsg(msg string, keysAndValues ...any) string {
allFields := make([]any, 0, len(s.extraFields)+len(keysAndValues))
allFields = append(allFields, s.extraFields...)
allFields = append(allFields, keysAndValues...)
if len(allFields) == 0 {
return msg
}
var sb strings.Builder
sb.WriteString(msg)
sb.WriteString(" [")
for i := 0; i < len(allFields); i += 2 {
if i > 0 {
sb.WriteString(" ")
}
if i+1 < len(allFields) {
fmt.Fprintf(&sb, "%v=%v", allFields[i], allFields[i+1])
} else {
fmt.Fprintf(&sb, "%v", allFields[i])
}
}
sb.WriteString("]")
return sb.String()
}
func (s *loggerServiceImpl) Debug(ctx context.Context, msg string, keysAndValues ...any) {
logger.DebugF(ctx, "%s", s.formatMsg(msg, keysAndValues...))
}
func (s *loggerServiceImpl) Info(ctx context.Context, msg string, keysAndValues ...any) {
logger.InfoF(ctx, "%s", s.formatMsg(msg, keysAndValues...))
}
func (s *loggerServiceImpl) Warn(ctx context.Context, msg string, keysAndValues ...any) {
logger.WarnF(ctx, "%s", s.formatMsg(msg, keysAndValues...))
}
func (s *loggerServiceImpl) Error(ctx context.Context, msg string, keysAndValues ...any) {
logger.ErrorF(ctx, "%s", s.formatMsg(msg, keysAndValues...))
}
func (s *loggerServiceImpl) Debugf(ctx context.Context, format string, args ...any) {
msg := fmt.Sprintf(format, args...)
logger.DebugF(ctx, "%s", s.formatMsg(msg))
}
func (s *loggerServiceImpl) Infof(ctx context.Context, format string, args ...any) {
msg := fmt.Sprintf(format, args...)
logger.InfoF(ctx, "%s", s.formatMsg(msg))
}
func (s *loggerServiceImpl) Warnf(ctx context.Context, format string, args ...any) {
msg := fmt.Sprintf(format, args...)
logger.WarnF(ctx, "%s", s.formatMsg(msg))
}
func (s *loggerServiceImpl) Errorf(ctx context.Context, format string, args ...any) {
msg := fmt.Sprintf(format, args...)
logger.ErrorF(ctx, "%s", s.formatMsg(msg))
}
func (s *loggerServiceImpl) With(keysAndValues ...any) contracts.LoggerService {
merged := make([]any, 0, len(s.extraFields)+len(keysAndValues))
merged = append(merged, s.extraFields...)
merged = append(merged, keysAndValues...)
return &loggerServiceImpl{
extraFields: merged,
}
}
@@ -0,0 +1,43 @@
// Copyright 2026 Arctel.net
// SPDX-License-Identifier: Apache-2.0
package logger_test
import (
"Wavelet/core"
"Wavelet/core/contracts"
"Wavelet/plugins/infra/logger"
"context"
"testing"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
)
func TestLoggerPluginOperations(t *testing.T) {
p := logger.New()
assert.Equal(t, "logger", p.Name())
ctx := core.NewContext(context.Background())
require.NoError(t, p.Apply(ctx))
svc, err := core.Inject[contracts.LoggerService](ctx)
require.NoError(t, err)
require.NotNil(t, svc)
testCtx := context.Background()
svc.Debug(testCtx, "debug msg", "key", "val")
svc.Info(testCtx, "info msg", "user", 1)
svc.Warn(testCtx, "warn msg", "odd_field")
svc.Error(testCtx, "error msg")
svc.Debugf(testCtx, "debug %s", "formatted")
svc.Infof(testCtx, "info %d", 100)
svc.Warnf(testCtx, "warn %v", true)
svc.Errorf(testCtx, "error %s", "fail")
withLogger := svc.With("trace", "t-123", "span", "s-456")
require.NotNil(t, withLogger)
withLogger.Info(testCtx, "enriched message")
}
@@ -0,0 +1,98 @@
// Copyright 2026 Arctel.net
// SPDX-License-Identifier: Apache-2.0
// Package diskcache wraps the generic pkg/cache/disk to provide database configuration integration.
package diskcache
import (
"context"
"strconv"
"sync"
"time"
pkgcache "Wavelet/pkg/cache/disk"
"Wavelet/pkg/util"
)
// Status represents the runtime cache statistics.
type Status = pkgcache.Status
const (
defaultCacheDir = "uploads/diskcache"
defaultMaxSizeMB = 100
defaultTTLMinutes = 60
defaultCleanupInterval = 10
// DefaultExpiration applies the cache-wide default TTL.
DefaultExpiration = pkgcache.DefaultExpiration
// NoExpiration stores the item without a TTL. Size limits and LRU eviction still apply.
NoExpiration = pkgcache.NoExpiration
)
// ErrCacheMiss represents a cache miss.
var ErrCacheMiss = pkgcache.ErrCacheMiss
// DiskCache is a wrapper around the generic pkg/diskcache that integrates with the DB for configs.
type DiskCache struct {
*pkgcache.Cache
}
var (
globalCache *DiskCache
globalCacheOnce sync.Once
)
// GetGlobalCache returns the global singleton DiskCache instance.
func GetGlobalCache() *DiskCache {
globalCacheOnce.Do(func() {
pureCache := pkgcache.New(defaultCacheDir)
globalCache = &DiskCache{pureCache}
// Load initial configs from database
globalCache.ReloadConfig(context.Background())
// Start background routine to clean expired items every 10 minutes
util.Go(func() { globalCache.StartCleanupWorker(defaultCleanupInterval * time.Minute) })
})
return globalCache
}
// New creates a new DiskCache wrapper.
func New(basePath string) *DiskCache {
return &DiskCache{pkgcache.New(basePath)}
}
// ReloadConfig reloads policies from database configs dynamically.
func (c *DiskCache) ReloadConfig(ctx context.Context) {
// Ensure DB is initialized before querying
if getDB(ctx) == nil {
return
}
// 1. Max Size
maxSizeMB := int64(defaultMaxSizeMB)
var maxVal string
if err := getDB(ctx).Table("w_system_configs").Where("key = ?", "disk_cache_max_size_mb").Pluck("value", &maxVal).Error; err == nil && maxVal != "" {
if val, err := strconv.ParseInt(maxVal, 10, 64); err == nil && val > 0 {
maxSizeMB = val
}
}
// 2. Default TTL
ttlMinutes := int64(defaultTTLMinutes)
var ttlVal string
if err := getDB(ctx).Table("w_system_configs").Where("key = ?", "disk_cache_ttl_minutes").Pluck("value", &ttlVal).Error; err == nil && ttlVal != "" {
if val, err := strconv.ParseInt(ttlVal, 10, 64); err == nil && val >= 0 {
ttlMinutes = val
}
}
// 3. LRU Enabled
lruEnabled := true
var lruVal string
if err := getDB(ctx).Table("w_system_configs").Where("key = ?", "disk_cache_lru_enabled").Pluck("value", &lruVal).Error; err == nil && lruVal != "" {
if val, err := strconv.ParseBool(lruVal); err == nil {
lruEnabled = val
}
}
c.UpdatePolicy(maxSizeMB, ttlMinutes, lruEnabled)
}
@@ -0,0 +1,61 @@
// Copyright 2026 Arctel.net
// SPDX-License-Identifier: Apache-2.0
package diskcache
import (
"Wavelet/pkg/testhelper"
"context"
"testing"
"gorm.io/gorm"
)
type mockDBService struct {
db *gorm.DB
}
func (m *mockDBService) GORM() *gorm.DB {
return m.db
}
func (m *mockDBService) DB(ctx context.Context) *gorm.DB {
return m.db.WithContext(ctx)
}
func (m *mockDBService) Named(_ string) *gorm.DB {
return m.db
}
func TestDiskCacheReloadConfig(t *testing.T) {
dbConn, _, cleanup := testhelper.SetupTestEnvironment(t)
SetDBService(&mockDBService{db: dbConn})
defer func() {
SetDBService(nil)
cleanup()
}()
testDir := t.TempDir()
c := New(testDir)
defer func() { _ = c.Clear() }()
// Update DB config values
dbConn.Table("w_system_configs").Where("key = ?", "disk_cache_max_size_mb").Update("value", "250")
dbConn.Table("w_system_configs").Where("key = ?", "disk_cache_ttl_minutes").Update("value", "120")
dbConn.Table("w_system_configs").Where("key = ?", "disk_cache_lru_enabled").Update("value", "false")
// Reload config
c.ReloadConfig(context.Background())
status := c.Status()
if status.MaxSizeMB != 250 {
t.Errorf("expected MaxSizeMB to be 250, got %d", status.MaxSizeMB)
}
if status.TTLMinutes != 120 {
t.Errorf("expected TTLMinutes to be 120, got %d", status.TTLMinutes)
}
if status.LRUEnabled != false {
t.Errorf("expected LRUEnabled to be false, got %v", status.LRUEnabled)
}
}
@@ -0,0 +1,40 @@
// Copyright 2026 Arctel.net
// SPDX-License-Identifier: Apache-2.0
package diskcache
import (
"Wavelet/core"
"Wavelet/core/contracts"
"context"
"sync"
"gorm.io/gorm"
)
var (
dbMu sync.RWMutex
dbSvc contracts.DBService
)
// SetDBService sets the DBService instance for diskcache.
func SetDBService(s contracts.DBService) {
dbMu.Lock()
defer dbMu.Unlock()
dbSvc = s
}
func getDB(ctx context.Context) *gorm.DB {
if c, ok := ctx.(*core.Context); ok && c != nil {
if s, err := core.Inject[contracts.DBService](c); err == nil && s != nil {
return s.DB(ctx)
}
}
dbMu.RLock()
s := dbSvc
dbMu.RUnlock()
if s != nil {
return s.DB(ctx)
}
return nil
}
@@ -0,0 +1,258 @@
// Copyright 2026 Arctel.net
// SPDX-License-Identifier: Apache-2.0
// Package objectstore provides dynamically configured file storage backends.
package objectstore
import (
"context"
"encoding/json"
"errors"
"fmt"
"strings"
"time"
"gorm.io/gorm"
)
// Driver identifies a supported storage backend.
type Driver string
const (
// DriverLocal stores files on the local filesystem.
DriverLocal Driver = "local"
// DriverS3 stores files in an S3-compatible object store.
DriverS3 Driver = "s3"
// DriverR2 stores files in Cloudflare R2.
DriverR2 Driver = "r2"
// DriverMinIO stores files in MinIO.
DriverMinIO Driver = "minio"
// DriverOSS stores files in Aliyun OSS.
DriverOSS Driver = "oss"
// DriverWebDAV stores files through WebDAV.
DriverWebDAV Driver = "webdav"
// ConfigMask replaces secrets returned to the frontend.
ConfigMask = "******"
)
// LocalConfig configures local filesystem storage.
type LocalConfig struct {
Root string `json:"root"`
}
// ObjectConfig configures S3-compatible or OSS object storage.
type ObjectConfig struct {
Endpoint string `json:"endpoint"`
Region string `json:"region"`
Bucket string `json:"bucket"`
AccessKeyID string `json:"access_key_id"`
SecretAccessKey string `json:"secret_access_key"`
AccountID string `json:"account_id,omitempty"`
PathStyle bool `json:"path_style"`
KeyPrefix string `json:"key_prefix"`
CDNURL string `json:"cdn_url"`
}
// WebDAVConfig configures WebDAV storage.
type WebDAVConfig struct {
Endpoint string `json:"endpoint"`
Username string `json:"username"`
Password string `json:"password"`
BasePath string `json:"base_path"`
}
// Config contains all storage backends and the currently active driver.
type Config struct {
Driver Driver `json:"driver"`
Local LocalConfig `json:"local"`
S3 ObjectConfig `json:"s3"`
R2 ObjectConfig `json:"r2"`
MinIO ObjectConfig `json:"minio"`
OSS ObjectConfig `json:"oss"`
WebDAV WebDAVConfig `json:"webdav"`
}
// DefaultConfig returns the local-storage default configuration.
func DefaultConfig() Config {
return Config{
Driver: DriverLocal,
Local: LocalConfig{Root: "."},
S3: ObjectConfig{Region: "us-east-1"},
R2: ObjectConfig{Region: "auto"},
MinIO: ObjectConfig{Region: "us-east-1", PathStyle: true},
}
}
// LoadConfig loads the active storage configuration.
func LoadConfig(ctx context.Context) (Config, error) {
pubSubOnce.Do(startPubSubListener)
cacheMutex.RLock()
isCacheValid := time.Since(lastChecked) < 5*time.Second && activeConfigJSON != ""
configJSON := activeConfigJSON
cacheMutex.RUnlock()
if isCacheValid {
cfg := DefaultConfig()
if strings.TrimSpace(configJSON) != "" {
if err := json.Unmarshal([]byte(configJSON), &cfg); err != nil {
return Config{}, fmt.Errorf("parse storage config from cache: %w", err)
}
}
return cfg, nil
}
return loadConfigByKey(ctx, "storage_config", DefaultConfig())
}
func loadConfigByKey(ctx context.Context, key string, fallback Config) (Config, error) {
var val string
db := getDB(ctx)
if db != nil {
err := db.Table("w_system_configs").Where("key = ?", key).Pluck("value", &val).Error
if err != nil {
if errors.Is(err, gorm.ErrRecordNotFound) {
return fallback, nil
}
return Config{}, err
}
}
if strings.TrimSpace(val) == "" {
return fallback, nil
}
if err := json.Unmarshal([]byte(val), &fallback); err != nil {
return Config{}, fmt.Errorf("parse %s: %w", key, err)
}
return fallback, nil
}
// ValidateConfig validates the selected backend configuration.
func ValidateConfig(cfg Config) error {
switch cfg.Driver {
case DriverLocal:
if strings.TrimSpace(cfg.Local.Root) == "" {
return errors.New("local root is required")
}
case DriverS3:
return validateObjectConfig(cfg.S3, false)
case DriverR2:
if strings.TrimSpace(cfg.R2.AccountID) == "" && strings.TrimSpace(cfg.R2.Endpoint) == "" {
return errors.New("R2 account ID or endpoint is required")
}
return validateObjectConfig(cfg.R2, false)
case DriverMinIO:
if strings.TrimSpace(cfg.MinIO.Endpoint) == "" {
return errors.New("MinIO endpoint is required")
}
return validateObjectConfig(cfg.MinIO, true)
case DriverOSS:
if strings.TrimSpace(cfg.OSS.Endpoint) == "" {
return errors.New("OSS endpoint is required")
}
return validateObjectConfig(cfg.OSS, true)
case DriverWebDAV:
if strings.TrimSpace(cfg.WebDAV.Endpoint) == "" {
return errors.New("WebDAV endpoint is required")
}
default:
return fmt.Errorf("unsupported storage driver %q", cfg.Driver)
}
return nil
}
func validateObjectConfig(cfg ObjectConfig, endpointRequired bool) error {
if endpointRequired && strings.TrimSpace(cfg.Endpoint) == "" {
return errors.New("endpoint is required")
}
if strings.TrimSpace(cfg.Region) == "" {
return errors.New("region is required")
}
if strings.TrimSpace(cfg.Bucket) == "" {
return errors.New("bucket is required")
}
if strings.TrimSpace(cfg.AccessKeyID) == "" || strings.TrimSpace(cfg.SecretAccessKey) == "" {
return errors.New("access key ID and secret access key are required")
}
return nil
}
// SaveActiveConfig persists the active storage configuration.
func SaveActiveConfig(ctx context.Context, cfg Config) error {
return saveSystemConfig(ctx, "storage_config", cfg, "文件存储驱动及连接配置(JSON)")
}
func saveSystemConfig(ctx context.Context, key string, value any, description string) error {
err := getDB(ctx).Transaction(func(tx *gorm.DB) error {
return upsertSystemConfig(ctx, tx, key, value, description)
})
if err == nil && key == "storage_config" {
ResetCache()
PublishCacheInvalidation(ctx)
}
return err
}
func upsertSystemConfig(_ context.Context, tx *gorm.DB, key string, value any, description string) error {
data, err := json.Marshal(value)
if err != nil {
return fmt.Errorf("marshal %s: %w", key, err)
}
type sysConfig struct {
Key string `gorm:"primaryKey"`
Value string
Type string
Visibility string
Description string
}
sc := sysConfig{
Key: key,
Value: string(data),
Type: "system",
Visibility: "hidden",
Description: description,
}
return tx.Table("w_system_configs").Where("key = ?", key).
Assign(map[string]any{"value": sc.Value, "description": description, "visibility": "hidden"}).
FirstOrCreate(&sc).Error
}
// MergeMaskedSecrets restores unchanged secrets from the current configuration.
func MergeMaskedSecrets(next, current Config) Config {
mergeObjectSecret := func(dst *ObjectConfig, src ObjectConfig) {
if dst.AccessKeyID == ConfigMask {
dst.AccessKeyID = src.AccessKeyID
}
if dst.SecretAccessKey == ConfigMask {
dst.SecretAccessKey = src.SecretAccessKey
}
}
mergeObjectSecret(&next.S3, current.S3)
mergeObjectSecret(&next.R2, current.R2)
mergeObjectSecret(&next.MinIO, current.MinIO)
mergeObjectSecret(&next.OSS, current.OSS)
if next.WebDAV.Password == ConfigMask {
next.WebDAV.Password = current.WebDAV.Password
}
return next
}
// MaskSecrets replaces stored credentials with placeholders for API responses.
func MaskSecrets(cfg Config) Config {
maskObject := func(value *ObjectConfig) {
if value.AccessKeyID != "" {
value.AccessKeyID = ConfigMask
}
if value.SecretAccessKey != "" {
value.SecretAccessKey = ConfigMask
}
}
maskObject(&cfg.S3)
maskObject(&cfg.R2)
maskObject(&cfg.MinIO)
maskObject(&cfg.OSS)
if cfg.WebDAV.Password != "" {
cfg.WebDAV.Password = ConfigMask
}
return cfg
}
@@ -0,0 +1,61 @@
// Copyright 2026 Arctel.net
// SPDX-License-Identifier: Apache-2.0
package objectstore
import (
"Wavelet/core"
"Wavelet/core/contracts"
"context"
"sync"
"gorm.io/gorm"
)
var (
dbMu sync.RWMutex
dbSvc contracts.DBService
cacheMu sync.RWMutex
cacheSvc contracts.CacheService
)
// SetDBService sets the DBService instance for objectstore.
func SetDBService(s contracts.DBService) {
dbMu.Lock()
defer dbMu.Unlock()
dbSvc = s
}
// SetCacheService sets the CacheService instance for objectstore.
func SetCacheService(s contracts.CacheService) {
cacheMu.Lock()
defer cacheMu.Unlock()
cacheSvc = s
}
func getDB(ctx context.Context) *gorm.DB {
if c, ok := ctx.(*core.Context); ok && c != nil {
if s, err := core.Inject[contracts.DBService](c); err == nil && s != nil {
return s.DB(ctx)
}
}
dbMu.RLock()
s := dbSvc
dbMu.RUnlock()
if s != nil {
return s.DB(ctx)
}
return nil
}
func getCache(ctx context.Context) contracts.CacheService {
if c, ok := ctx.(*core.Context); ok && c != nil {
if s, err := core.Inject[contracts.CacheService](c); err == nil && s != nil {
return s
}
}
cacheMu.RLock()
s := cacheSvc
cacheMu.RUnlock()
return s
}
@@ -0,0 +1,43 @@
// Copyright 2026 Arctel.net
// SPDX-License-Identifier: Apache-2.0
package objectstore
import (
"Wavelet/pkg/httppool"
"context"
"fmt"
"net/http"
"net/url"
"time"
)
func getHTTPObject(ctx context.Context, baseURL, key string) (*Object, error) {
objectURL, err := url.JoinPath(baseURL, key)
if err != nil {
return nil, fmt.Errorf("build CDN object URL: %w", err)
}
request, err := http.NewRequestWithContext(ctx, http.MethodGet, objectURL, nil)
if err != nil {
return nil, fmt.Errorf("create CDN request: %w", err)
}
const cdnRequestTimeout = 30 * time.Second
client := httppool.NewClient(cdnRequestTimeout)
response, err := client.Do(request)
if err != nil {
return nil, fmt.Errorf("get CDN object: %w", err)
}
if response.StatusCode != http.StatusOK {
_ = response.Body.Close()
return nil, fmt.Errorf("get CDN object: unexpected status %d", response.StatusCode)
}
contentType := response.Header.Get("Content-Type")
if contentType == "" {
contentType = defaultContentType
}
return &Object{
Body: response.Body,
ContentLength: response.ContentLength,
ContentType: contentType,
}, nil
}
@@ -0,0 +1,120 @@
// Copyright 2026 Arctel.net
// SPDX-License-Identifier: Apache-2.0
package objectstore
import (
"context"
"fmt"
"io"
"mime"
"os"
"path/filepath"
"strings"
)
type localBackend struct {
root string
}
func newLocalBackend(cfg LocalConfig) (*localBackend, error) {
root := filepath.Clean(cfg.Root)
if root == "" {
return nil, fmt.Errorf("local root is required")
}
return &localBackend{root: root}, nil
}
func (b *localBackend) Put(_ context.Context, key string, body io.Reader, _ int64, _ string) (PutResult, error) {
path, err := b.path(key)
if err != nil {
return PutResult{}, err
}
if err := os.MkdirAll(filepath.Dir(path), storageDirPerm); err != nil {
return PutResult{}, err
}
file, err := os.OpenFile( //nolint:gosec // path is constrained to the configured storage root.
path,
os.O_CREATE|os.O_TRUNC|os.O_WRONLY,
storageFilePerm,
)
if err != nil {
return PutResult{}, err
}
if _, err := io.Copy(file, body); err != nil {
_ = file.Close()
_ = os.Remove(path)
return PutResult{}, err
}
if err := file.Close(); err != nil {
_ = os.Remove(path)
return PutResult{}, err
}
return PutResult{Key: filepath.ToSlash(key)}, nil
}
func (b *localBackend) Get(_ context.Context, key string) (*Object, error) {
path, err := b.path(key)
if err != nil {
return nil, err
}
file, err := os.Open(path) //nolint:gosec // path is constrained to the configured storage root.
if err != nil {
return nil, err
}
info, err := file.Stat()
if err != nil {
_ = file.Close()
return nil, err
}
contentType := mime.TypeByExtension(filepath.Ext(path))
if contentType == "" {
contentType = defaultContentType
}
return &Object{Body: file, ContentLength: info.Size(), ContentType: contentType}, nil
}
func (b *localBackend) Delete(_ context.Context, key string) error {
path, err := b.path(key)
if err != nil {
return err
}
err = os.Remove(path)
if os.IsNotExist(err) {
return nil
}
return err
}
func (b *localBackend) Test(_ context.Context) error {
return os.MkdirAll(b.root, storageDirPerm)
}
func (b *localBackend) path(key string) (string, error) {
if filepath.IsAbs(key) {
cleanPath := filepath.Clean(key)
absRoot, err := filepath.Abs(b.root)
if err != nil {
return "", err
}
absPath, err := filepath.Abs(cleanPath)
if err != nil {
return "", err
}
rel, err := filepath.Rel(absRoot, absPath)
if err != nil || strings.HasPrefix(rel, "..") {
return "", fmt.Errorf("storage key escapes local root")
}
return cleanPath, nil
}
cleanKey := filepath.Clean(filepath.FromSlash(strings.TrimPrefix(key, "/")))
if cleanKey == "." || cleanKey == "" || strings.HasPrefix(cleanKey, "..") {
return "", fmt.Errorf("invalid local storage key %q", key)
}
path := filepath.Join(b.root, cleanKey)
rel, err := filepath.Rel(b.root, path)
if err != nil || strings.HasPrefix(rel, "..") {
return "", fmt.Errorf("storage key escapes local root")
}
return path, nil
}
@@ -0,0 +1,51 @@
// Copyright 2026 Arctel.net
// SPDX-License-Identifier: Apache-2.0
package objectstore
import (
"bytes"
"context"
"io"
"testing"
)
func TestLocalBackendRoundTrip(t *testing.T) {
backend, err := newLocalBackend(LocalConfig{Root: t.TempDir()})
if err != nil {
t.Fatalf("newLocalBackend() returned error: %v", err)
}
ctx := context.Background()
const key = "uploads/2026/06/13/test.txt"
const content = "wavelet storage"
storedResult, err := backend.Put(ctx, key, bytes.NewBufferString(content), int64(len(content)), "text/plain")
if err != nil {
t.Fatalf("Put(%q) returned error: %v", key, err)
}
if storedResult.Key != key {
t.Errorf("Put(%q) key = %q, want %q", key, storedResult.Key, key)
}
object, err := backend.Get(ctx, key)
if err != nil {
t.Fatalf("Get(%q) returned error: %v", key, err)
}
got, err := io.ReadAll(object.Body)
if err != nil {
t.Fatalf("ReadAll(Get(%q)) returned error: %v", key, err)
}
if err := object.Body.Close(); err != nil {
t.Fatalf("Close(Get(%q)) returned error: %v", key, err)
}
if string(got) != content {
t.Errorf("Get(%q) content = %q, want %q", key, got, content)
}
if err := backend.Delete(ctx, key); err != nil {
t.Fatalf("Delete(%q) returned error: %v", key, err)
}
if _, err := backend.Get(ctx, key); err == nil {
t.Errorf("Get(%q) after Delete() returned nil error", key)
}
}
@@ -0,0 +1,98 @@
// Copyright 2026 Arctel.net
// SPDX-License-Identifier: Apache-2.0
package objectstore
import (
"context"
"fmt"
"io"
"strings"
"github.com/aliyun/alibabacloud-oss-go-sdk-v2/oss"
"github.com/aliyun/alibabacloud-oss-go-sdk-v2/oss/credentials"
)
type ossBackend struct {
client *oss.Client
bucket string
keyPrefix string
cdnURL string
}
func newOSSBackend(cfg ObjectConfig) (*ossBackend, error) {
options := oss.LoadDefaultConfig().
WithCredentialsProvider(credentials.NewStaticCredentialsProvider(cfg.AccessKeyID, cfg.SecretAccessKey)).
WithRegion(cfg.Region)
if cfg.Endpoint != "" {
options.WithEndpoint(cfg.Endpoint)
}
return &ossBackend{
client: oss.NewClient(options),
bucket: cfg.Bucket,
keyPrefix: strings.Trim(cfg.KeyPrefix, "/"),
cdnURL: strings.TrimRight(cfg.CDNURL, "/"),
}, nil
}
func (b *ossBackend) Put(ctx context.Context, key string, body io.Reader, _ int64, _ string) (PutResult, error) {
key = b.key(key)
_, err := b.client.PutObject(ctx, &oss.PutObjectRequest{
Bucket: oss.Ptr(b.bucket),
Key: oss.Ptr(key),
Body: body,
})
if err != nil {
return PutResult{}, fmt.Errorf("put OSS object: %w", err)
}
return PutResult{Key: key, Bucket: b.bucket}, nil
}
func (b *ossBackend) Get(ctx context.Context, key string) (*Object, error) {
key = b.key(key)
if b.cdnURL != "" {
return getHTTPObject(ctx, b.cdnURL, key)
}
output, err := b.client.GetObject(ctx, &oss.GetObjectRequest{
Bucket: oss.Ptr(b.bucket),
Key: oss.Ptr(key),
})
if err != nil {
return nil, fmt.Errorf("get OSS object: %w", err)
}
contentType := defaultContentType
if output.ContentType != nil {
contentType = *output.ContentType
}
return &Object{Body: output.Body, ContentLength: output.ContentLength, ContentType: contentType}, nil
}
func (b *ossBackend) Delete(ctx context.Context, key string) error {
_, err := b.client.DeleteObject(ctx, &oss.DeleteObjectRequest{
Bucket: oss.Ptr(b.bucket),
Key: oss.Ptr(b.key(key)),
})
if err != nil {
return fmt.Errorf("delete OSS object: %w", err)
}
return nil
}
func (b *ossBackend) Test(ctx context.Context) error {
ok, err := b.client.IsBucketExist(ctx, b.bucket)
if err != nil {
return fmt.Errorf("access OSS bucket: %w", err)
}
if !ok {
return fmt.Errorf("OSS bucket %q does not exist", b.bucket)
}
return nil
}
func (b *ossBackend) key(key string) string {
key = strings.TrimLeft(key, "/")
if b.keyPrefix == "" || strings.HasPrefix(key, b.keyPrefix+"/") {
return key
}
return b.keyPrefix + "/" + key
}
@@ -0,0 +1,122 @@
// Copyright 2026 Arctel.net
// SPDX-License-Identifier: Apache-2.0
package objectstore
import (
"context"
"fmt"
"io"
"strings"
"github.com/aws/aws-sdk-go-v2/aws"
awsconfig "github.com/aws/aws-sdk-go-v2/config"
"github.com/aws/aws-sdk-go-v2/credentials"
"github.com/aws/aws-sdk-go-v2/service/s3"
)
type s3Backend struct {
client *s3.Client
bucket string
keyPrefix string
cdnURL string
}
func newS3Backend(ctx context.Context, cfg ObjectConfig) (*s3Backend, error) {
awsCfg, err := awsconfig.LoadDefaultConfig(ctx,
awsconfig.WithRegion(cfg.Region),
awsconfig.WithCredentialsProvider(credentials.NewStaticCredentialsProvider(
cfg.AccessKeyID,
cfg.SecretAccessKey,
"",
)),
)
if err != nil {
return nil, fmt.Errorf("load S3 config: %w", err)
}
client := s3.NewFromConfig(awsCfg, func(options *s3.Options) {
if cfg.Endpoint != "" {
options.BaseEndpoint = aws.String(strings.TrimRight(cfg.Endpoint, "/"))
}
options.UsePathStyle = cfg.PathStyle
})
return &s3Backend{
client: client,
bucket: cfg.Bucket,
keyPrefix: strings.Trim(cfg.KeyPrefix, "/"),
cdnURL: strings.TrimRight(cfg.CDNURL, "/"),
}, nil
}
func newR2Backend(ctx context.Context, cfg ObjectConfig) (*s3Backend, error) {
if cfg.Endpoint == "" {
cfg.Endpoint = fmt.Sprintf("https://%s.r2.cloudflarestorage.com", cfg.AccountID)
}
cfg.Region = "auto"
return newS3Backend(ctx, cfg)
}
func (b *s3Backend) Put(ctx context.Context, key string, body io.Reader, size int64, contentType string) (PutResult, error) {
key = b.key(key)
_, err := b.client.PutObject(ctx, &s3.PutObjectInput{
Bucket: aws.String(b.bucket),
Key: aws.String(key),
Body: body,
ContentLength: aws.Int64(size),
ContentType: aws.String(contentType),
})
if err != nil {
return PutResult{}, fmt.Errorf("put S3 object: %w", err)
}
return PutResult{Key: key, Bucket: b.bucket}, nil
}
func (b *s3Backend) Get(ctx context.Context, key string) (*Object, error) {
key = b.key(key)
if b.cdnURL != "" {
return getHTTPObject(ctx, b.cdnURL, key)
}
output, err := b.client.GetObject(ctx, &s3.GetObjectInput{
Bucket: aws.String(b.bucket),
Key: aws.String(key),
})
if err != nil {
return nil, fmt.Errorf("get S3 object: %w", err)
}
contentType := defaultContentType
if output.ContentType != nil {
contentType = *output.ContentType
}
var size int64
if output.ContentLength != nil {
size = *output.ContentLength
}
return &Object{Body: output.Body, ContentLength: size, ContentType: contentType}, nil
}
func (b *s3Backend) Delete(ctx context.Context, key string) error {
_, err := b.client.DeleteObject(ctx, &s3.DeleteObjectInput{
Bucket: aws.String(b.bucket),
Key: aws.String(b.key(key)),
})
if err != nil {
return fmt.Errorf("delete S3 object: %w", err)
}
return nil
}
func (b *s3Backend) Test(ctx context.Context) error {
_, err := b.client.HeadBucket(ctx, &s3.HeadBucketInput{Bucket: aws.String(b.bucket)})
if err != nil {
return fmt.Errorf("access S3 bucket: %w", err)
}
return nil
}
func (b *s3Backend) key(key string) string {
key = strings.TrimLeft(key, "/")
if b.keyPrefix == "" || strings.HasPrefix(key, b.keyPrefix+"/") {
return key
}
return b.keyPrefix + "/" + key
}
@@ -0,0 +1,212 @@
// Copyright 2026 Arctel.net
// SPDX-License-Identifier: Apache-2.0
package objectstore
import (
"context"
"encoding/json"
"errors"
"fmt"
"io"
"strings"
"sync"
"time"
"gorm.io/gorm"
)
const (
defaultContentType = "application/octet-stream"
storageDirPerm = 0o750
storageFilePerm = 0o600
)
// Object describes a readable stored object.
type Object struct {
Key string
CachePath string
Body io.ReadCloser
ContentLength int64
ContentType string
}
// PutResult describes the result of a successful Put operation.
type PutResult struct {
Key string
Bucket string
}
// Backend defines storage operations used by the upload domain.
type Backend interface {
Put(ctx context.Context, key string, body io.Reader, size int64, contentType string) (PutResult, error)
Get(ctx context.Context, key string) (*Object, error)
Delete(ctx context.Context, key string) error
Test(ctx context.Context) error
}
var (
cacheMutex sync.RWMutex
activeDriver Driver
activeBackend Backend
activeConfigJSON string
lastChecked time.Time
pubSubOnce sync.Once
mockBackend Backend
// IsEnabledFunc controls whether mock/in-memory backend is activated in tests.
IsEnabledFunc = func() bool { return false }
)
// ConfigInvalidationChannel is the Redis pub/sub channel used to evict storage caches cluster-wide.
const ConfigInvalidationChannel = "storage:config_invalidation"
// SetMockBackend forces an in-memory/mock backend for testing.
func SetMockBackend(b Backend) {
mockBackend = b
}
// ResetCache clears cached driver and backend instances.
func ResetCache() {
cacheMutex.Lock()
defer cacheMutex.Unlock()
activeBackend = nil
activeDriver = ""
activeConfigJSON = ""
lastChecked = time.Time{}
}
// PublishCacheInvalidation broadcasts cache eviction to all nodes in the cluster via Redis.
func PublishCacheInvalidation(ctx context.Context) {
if cache := getCache(ctx); cache != nil {
_ = cache.Invalidate(ctx, ConfigInvalidationChannel)
}
ResetCache()
}
// startPubSubListener starts the background subscriber for cache invalidations.
func startPubSubListener() {
}
// Active returns the configured active driver and backend, using an in-memory cache with 5s TTL.
func Active(ctx context.Context) (Driver, Backend, error) {
if IsEnabledFunc() && mockBackend != nil {
return DriverS3, mockBackend, nil
}
pubSubOnce.Do(startPubSubListener)
cacheMutex.RLock()
isCacheValid := time.Since(lastChecked) < 5*time.Second && activeBackend != nil
if isCacheValid {
d, b := activeDriver, activeBackend
cacheMutex.RUnlock()
return d, b, nil
}
cacheMutex.RUnlock()
cacheMutex.Lock()
defer cacheMutex.Unlock()
// Double-check under write lock
if time.Since(lastChecked) < 5*time.Second && activeBackend != nil {
return activeDriver, activeBackend, nil
}
var val string
db := getDB(ctx)
if db != nil {
err := db.Table("w_system_configs").Where("key = ?", "storage_config").Pluck("value", &val).Error
if err != nil && !errors.Is(err, gorm.ErrRecordNotFound) {
return "", nil, err
}
}
sc := struct{ Value string }{Value: val}
lastChecked = time.Now()
// Reuse existing backend client singleton if configuration JSON matches
if sc.Value == activeConfigJSON && activeBackend != nil {
return activeDriver, activeBackend, nil
}
cfg := DefaultConfig()
if strings.TrimSpace(sc.Value) != "" {
if err := json.Unmarshal([]byte(sc.Value), &cfg); err != nil {
return "", nil, fmt.Errorf("parse storage config: %w", err)
}
}
backend, err := NewBackend(ctx, cfg, cfg.Driver)
if err != nil {
return "", nil, err
}
activeDriver = cfg.Driver
activeBackend = backend
activeConfigJSON = sc.Value
return activeDriver, activeBackend, nil
}
type functionBackend struct {
put func(context.Context, string, io.Reader, int64, string) error
get func(context.Context, string) (*Object, error)
delete func(context.Context, string) error
}
func (b *functionBackend) Put(ctx context.Context, key string, body io.Reader, size int64, contentType string) (PutResult, error) {
if err := b.put(ctx, key, body, size, contentType); err != nil {
return PutResult{}, err
}
return PutResult{Key: key}, nil
}
func (b *functionBackend) Get(ctx context.Context, key string) (*Object, error) {
return b.get(ctx, key)
}
func (b *functionBackend) Delete(ctx context.Context, key string) error {
return b.delete(ctx, key)
}
func (b *functionBackend) Test(context.Context) error {
return nil
}
// MockStorage replaces object operations for package tests and returns a restore function.
func MockStorage(
put func(context.Context, string, io.Reader, int64, string) error,
get func(context.Context, string) (*Object, error),
deleteObject func(context.Context, string) error,
) func() {
previous := mockBackend
mockBackend = &functionBackend{put: put, get: get, delete: deleteObject}
return func() {
mockBackend = previous
}
}
// NewBackend constructs a concrete backend from configuration.
func NewBackend(ctx context.Context, cfg Config, driver Driver) (Backend, error) {
if driver == DriverS3 && mockBackend != nil {
return mockBackend, nil
}
switch driver {
case DriverLocal:
return newLocalBackend(cfg.Local)
case DriverS3:
return newS3Backend(ctx, cfg.S3)
case DriverR2:
return newR2Backend(ctx, cfg.R2)
case DriverMinIO:
return newS3Backend(ctx, cfg.MinIO)
case DriverOSS:
return newOSSBackend(cfg.OSS)
case DriverWebDAV:
return newWebDAVBackend(cfg.WebDAV)
default:
return nil, fmt.Errorf("unsupported storage driver %q", driver)
}
}
@@ -0,0 +1,144 @@
// Copyright 2026 Arctel.net
// SPDX-License-Identifier: Apache-2.0
package objectstore
import (
"context"
"encoding/json"
"io"
"sync"
"testing"
"time"
"github.com/alicebob/miniredis/v2"
"github.com/redis/go-redis/v9"
cache "Wavelet/plugins/infra/cache"
)
func TestStorageCache(t *testing.T) {
// 1. Reset cache
ResetCache()
if activeConfigJSON != "" || activeDriver != "" || activeBackend != nil || !lastChecked.IsZero() {
t.Fatal("ResetCache did not clear cache variables")
}
// 2. Set up cache manually
expectedConfig := Config{
Driver: DriverLocal,
Local: LocalConfig{Root: "/tmp/wavelet-test"},
}
cfgJSON, err := json.Marshal(expectedConfig)
if err != nil {
t.Fatalf("Marshal config failed: %v", err)
}
cacheMutex.Lock()
activeConfigJSON = string(cfgJSON)
lastChecked = time.Now()
cacheMutex.Unlock()
// 3. Call LoadConfig and verify it loads from cache (doesn't hit database, which would fail/panic because DB is not initialized)
ctx := context.Background()
loadedCfg, err := LoadConfig(ctx)
if err != nil {
t.Fatalf("LoadConfig failed: %v", err)
}
if loadedCfg.Driver != expectedConfig.Driver || loadedCfg.Local.Root != expectedConfig.Local.Root {
t.Errorf("Loaded config %+v, expected %+v", loadedCfg, expectedConfig)
}
// 4. Test Active() returns cached driver and backend
mockBnd := &functionBackend{
put: func(context.Context, string, io.Reader, int64, string) error { return nil },
get: func(context.Context, string) (*Object, error) { return nil, nil },
delete: func(context.Context, string) error { return nil },
}
cacheMutex.Lock()
activeBackend = mockBnd
activeDriver = DriverLocal
cacheMutex.Unlock()
drv, bnd, err := Active(ctx)
if err != nil {
t.Fatalf("Active failed: %v", err)
}
if drv != DriverLocal || bnd != mockBnd {
t.Errorf("Active returned driver %v, backend %v; expected %v, %v", drv, bnd, DriverLocal, mockBnd)
}
// 5. Test ResetCache again
ResetCache()
if activeConfigJSON != "" || activeDriver != "" || activeBackend != nil || !lastChecked.IsZero() {
t.Fatal("ResetCache did not clear cache variables after setting them")
}
}
func TestStorageCachePubSub(t *testing.T) {
// 1. Start miniredis
mr, err := miniredis.Run()
if err != nil {
t.Fatalf("Failed to run miniredis: %v", err)
}
defer mr.Close()
// 2. Initialize Redis client
rdb := redis.NewClient(&redis.Options{
Addr: mr.Addr(),
})
defer rdb.Close()
// 3. Set cache.Redis to our client
oldRedis := cache.Redis
cache.Redis = rdb
defer func() {
cache.Redis = oldRedis
}()
// Reset cache and set some cached config
ResetCache()
cacheMutex.Lock()
activeConfigJSON = "some_config"
lastChecked = time.Now()
cacheMutex.Unlock()
// 4. Force trigger lazy initialization of subscription
// Reset the once guard so it runs the listener
pubSubOnce = sync.Once{}
ctx := context.Background()
// Create mock backend for Active call
mockBnd := &functionBackend{
put: func(context.Context, string, io.Reader, int64, string) error { return nil },
get: func(context.Context, string) (*Object, error) { return nil, nil },
delete: func(context.Context, string) error { return nil },
}
cacheMutex.Lock()
activeBackend = mockBnd
activeDriver = DriverLocal
cacheMutex.Unlock()
_, _, _ = Active(ctx) // This calls startPubSubListener()
// Allow some time for subscriber connection
time.Sleep(100 * time.Millisecond)
// 5. Publish cache invalidation
PublishCacheInvalidation(ctx)
// Allow message propagation
time.Sleep(100 * time.Millisecond)
// 6. Verify cache was cleared
cacheMutex.RLock()
configJSON := activeConfigJSON
cacheMutex.RUnlock()
if configJSON != "" {
t.Error("Memory cache was not cleared after Redis Pub/Sub broadcast")
}
}
@@ -0,0 +1,102 @@
// Copyright 2026 Arctel.net
// SPDX-License-Identifier: Apache-2.0
package objectstore
import (
"Wavelet/pkg/httppool"
"context"
"fmt"
"io"
"net/http"
"path"
"strings"
"github.com/studio-b12/gowebdav"
)
type contextTransport struct {
ctx context.Context
parent http.RoundTripper
}
func (t *contextTransport) RoundTrip(req *http.Request) (*http.Response, error) {
return t.parent.RoundTrip(req.WithContext(t.ctx))
}
type webDAVBackend struct {
endpoint string
username string
password string
basePath string
}
func newWebDAVBackend(cfg WebDAVConfig) (*webDAVBackend, error) {
return &webDAVBackend{
endpoint: strings.TrimRight(cfg.Endpoint, "/"),
username: cfg.Username,
password: cfg.Password,
basePath: strings.Trim(cfg.BasePath, "/"),
}, nil
}
func (b *webDAVBackend) newClient(ctx context.Context) *gowebdav.Client {
client := gowebdav.NewClient(b.endpoint, b.username, b.password)
client.SetTransport(&contextTransport{
ctx: ctx,
parent: httppool.DefaultTransport(),
})
return client
}
func (b *webDAVBackend) Put(ctx context.Context, key string, body io.Reader, size int64, _ string) (PutResult, error) {
key = b.key(key)
client := b.newClient(ctx)
if dir := path.Dir(key); dir != "." && dir != "/" {
if err := client.MkdirAll(dir, storageDirPerm); err != nil {
return PutResult{}, fmt.Errorf("create WebDAV directory: %w", err)
}
}
if err := client.WriteStreamWithLength(key, body, size, storageFilePerm); err != nil {
return PutResult{}, fmt.Errorf("put WebDAV object: %w", err)
}
return PutResult{Key: key}, nil
}
func (b *webDAVBackend) Get(ctx context.Context, key string) (*Object, error) {
key = b.key(key)
client := b.newClient(ctx)
info, err := client.Stat(key)
if err != nil {
return nil, fmt.Errorf("stat WebDAV object: %w", err)
}
body, err := client.ReadStream(key)
if err != nil {
return nil, fmt.Errorf("get WebDAV object: %w", err)
}
contentType := defaultContentType
if typed, ok := info.(interface{ ContentType() string }); ok && typed.ContentType() != "" {
contentType = typed.ContentType()
}
return &Object{Body: body, ContentLength: info.Size(), ContentType: contentType}, nil
}
func (b *webDAVBackend) Delete(ctx context.Context, key string) error {
client := b.newClient(ctx)
if err := client.Remove(b.key(key)); err != nil {
return fmt.Errorf("delete WebDAV object: %w", err)
}
return nil
}
func (b *webDAVBackend) Test(ctx context.Context) error {
client := b.newClient(ctx)
if err := client.Connect(); err != nil {
return fmt.Errorf("connect WebDAV: %w", err)
}
return nil
}
func (b *webDAVBackend) key(key string) string {
return "/" + path.Join(b.basePath, strings.TrimLeft(key, "/"))
}
+147
View File
@@ -0,0 +1,147 @@
// Copyright 2026 Arctel.net
// SPDX-License-Identifier: Apache-2.0
// Package storage provides the object storage and ingestion infrastructure plugin for Cordis.
package storage
import (
"Wavelet/core"
"Wavelet/core/contracts"
"Wavelet/plugins/infra/storage/diskcache"
"Wavelet/plugins/infra/storage/objectstore"
"context"
"errors"
"fmt"
"io"
)
// Option configures the storage plugin.
type Option func(*Plugin)
// WithBackend sets an explicit storage backend instance (useful for testing or custom engines).
func WithBackend(b objectstore.Backend) Option {
return func(p *Plugin) {
p.backend = b
}
}
// Plugin implements core.Plugin to provide contracts.StorageService.
type Plugin struct {
backend objectstore.Backend
}
// New creates a new storage infrastructure plugin.
func New(opts ...Option) *Plugin {
p := &Plugin{}
for _, opt := range opts {
if opt != nil {
opt(p)
}
}
return p
}
// Name returns the unique identifier of the storage plugin.
func (p *Plugin) Name() string {
return "storage"
}
// Apply mounts the storage service into the Context.
func (p *Plugin) Apply(ctx *core.Context) error {
// Bind DBService
if db, err := core.Inject[contracts.DBService](ctx); err == nil && db != nil {
objectstore.SetDBService(db)
diskcache.SetDBService(db)
} else {
core.When[contracts.DBService](ctx, func(db contracts.DBService) {
objectstore.SetDBService(db)
diskcache.SetDBService(db)
})
}
// Bind CacheService
if cache, err := core.Inject[contracts.CacheService](ctx); err == nil && cache != nil {
objectstore.SetCacheService(cache)
} else {
core.When[contracts.CacheService](ctx, func(cache contracts.CacheService) {
objectstore.SetCacheService(cache)
})
}
ctx.OnDispose(func() error {
objectstore.SetDBService(nil)
diskcache.SetDBService(nil)
objectstore.SetCacheService(nil)
return nil
})
svc := &storageServiceImpl{
backend: p.backend,
}
core.Provide[contracts.StorageService](ctx, svc)
return nil
}
type storageServiceImpl struct {
backend objectstore.Backend
}
func (s *storageServiceImpl) getBackend(ctx context.Context) (objectstore.Backend, error) {
if s.backend != nil {
return s.backend, nil
}
_, b, err := objectstore.Active(ctx)
if err != nil {
return nil, fmt.Errorf("storage: get active backend failed: %w", err)
}
return b, nil
}
func (s *storageServiceImpl) Put(ctx context.Context, key string, body io.Reader, size int64, contentType string) (contracts.StoragePutResult, error) {
b, err := s.getBackend(ctx)
if err != nil {
return contracts.StoragePutResult{}, err
}
res, err := b.Put(ctx, key, body, size, contentType)
if err != nil {
return contracts.StoragePutResult{}, err
}
return contracts.StoragePutResult{
Key: res.Key,
Bucket: res.Bucket,
}, nil
}
func (s *storageServiceImpl) Get(ctx context.Context, key string) (*contracts.StorageObject, error) {
b, err := s.getBackend(ctx)
if err != nil {
return nil, err
}
obj, err := b.Get(ctx, key)
if err != nil {
return nil, err
}
return &contracts.StorageObject{
Key: key,
CachePath: obj.CachePath,
Body: obj.Body,
ContentLength: obj.ContentLength,
ContentType: obj.ContentType,
}, nil
}
func (s *storageServiceImpl) Delete(ctx context.Context, key string) error {
b, err := s.getBackend(ctx)
if err != nil {
return err
}
return b.Delete(ctx, key)
}
func (s *storageServiceImpl) Ingest(_ context.Context, _ io.Reader, _ contracts.IngestOptions) (*contracts.IngestResult, error) {
return nil, errors.New("storage: programmatic ingest is managed by domain/upload plugin")
}
@@ -0,0 +1,99 @@
// Copyright 2026 Arctel.net
// SPDX-License-Identifier: Apache-2.0
package storage_test
import (
"Wavelet/core"
"Wavelet/core/contracts"
"Wavelet/plugins/infra/storage"
"Wavelet/plugins/infra/storage/objectstore"
"bytes"
"context"
"errors"
"io"
"sync"
"testing"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
)
type mockStorageBackend struct {
mu sync.RWMutex
files map[string][]byte
}
func newMockStorageBackend() *mockStorageBackend {
return &mockStorageBackend{files: make(map[string][]byte)}
}
func (m *mockStorageBackend) Put(ctx context.Context, key string, body io.Reader, size int64, contentType string) (objectstore.PutResult, error) {
m.mu.Lock()
defer m.mu.Unlock()
data, err := io.ReadAll(body)
if err != nil {
return objectstore.PutResult{}, err
}
m.files[key] = data
return objectstore.PutResult{Key: key, Bucket: "mock-bucket"}, nil
}
func (m *mockStorageBackend) Get(ctx context.Context, key string) (*objectstore.Object, error) {
m.mu.RLock()
defer m.mu.RUnlock()
data, ok := m.files[key]
if !ok {
return nil, errors.New("file not found")
}
return &objectstore.Object{
Body: io.NopCloser(bytes.NewReader(data)),
ContentLength: int64(len(data)),
ContentType: "application/octet-stream",
}, nil
}
func (m *mockStorageBackend) Delete(ctx context.Context, key string) error {
m.mu.Lock()
defer m.mu.Unlock()
delete(m.files, key)
return nil
}
func (m *mockStorageBackend) Test(ctx context.Context) error {
return nil
}
func TestStoragePluginOperations(t *testing.T) {
backend := newMockStorageBackend()
p := storage.New(storage.WithBackend(backend))
assert.Equal(t, "storage", p.Name())
ctx := core.NewContext(context.Background())
require.NoError(t, p.Apply(ctx))
svc, err := core.Inject[contracts.StorageService](ctx)
require.NoError(t, err)
require.NotNil(t, svc)
testCtx := context.Background()
// Put
content := []byte("wavelet storage content")
putRes, err := svc.Put(testCtx, "avatar/user1.png", bytes.NewReader(content), int64(len(content)), "image/png")
require.NoError(t, err)
assert.Equal(t, "avatar/user1.png", putRes.Key)
assert.Equal(t, "mock-bucket", putRes.Bucket)
// Get
obj, err := svc.Get(testCtx, "avatar/user1.png")
require.NoError(t, err)
data, err := io.ReadAll(obj.Body)
require.NoError(t, err)
assert.Equal(t, content, data)
// Delete
require.NoError(t, svc.Delete(testCtx, "avatar/user1.png"))
_, err = svc.Get(testCtx, "avatar/user1.png")
assert.Error(t, err)
}