Commit Graph

188 Commits

Author SHA1 Message Date
ryan 03fa5d948d refactor(bootstrap): move runtime init from router to cmd layer
Extract SyncEvents and InitLogWriter from router.Serve into bootstrap.Init
called from cmd entry points with trace-aware context. Preserve existing
Register* wiring for task and push domain integrations.
2026-06-18 10:38:08 +08:00
ryan 50c45db561 refactor(user): separate Handler and Logic layer boundaries
Move HTTP handlers out of logics.go and replace gin.Context-coupled
login email verification with context-only processLoginEmailVerification.
Add logics_test.go for pure business logic unit tests.
2026-06-18 10:31:09 +08:00
ryan f826807cf1 feat(task): clean task log 2026-06-17 14:29:38 +08:00
ryan 333e45572a fix(frontend): optimize 2026-06-17 13:58:32 +08:00
ryan 3a05e7f09c refactor(idgen): centralize negative ID handling via panic
Restore NextUint64ID() to uint64-only API so callers need no error
checks. Retry logic stays in idgen; after 3 negative values it panics
instead of returning 0, preventing silent NULL primary keys.
2026-06-17 13:14:17 +08:00
ryan 92dc6a59c0 fix(idgen): retry snowflake generation and error on negative ID
NextUint64ID now retries up to 3 times when Int64() is negative, then
returns an error instead of 0 or Fatalf. All call sites propagate the
error to avoid GORM omitting zero-value primary keys.
2026-06-17 13:11:16 +08:00
ryan a27113feb2 fix(frontend): break service layer circular imports for build
Point service implementations at @/lib/services/core for BaseService
instead of the barrel index, fixing static export prerender failure on
ConfigService initialization.
2026-06-17 12:17:05 +08:00
ryan a44043262e perf(frontend): split admin bundles and tighten data fetching
Add next/dynamic lazy loading for database, logs, and settings heavy
modules; migrate access-logs and task-executions to React Query; parallelize
login auth-sources with public config; consolidate users table tooltips;
enable lazy image decoding; and replace barrel @/lib/services imports with
direct module paths.
2026-06-17 12:12:47 +08:00
ryan 9550fa6ff3 perf(cap): add runtime settings snapshot for dynamic config
Replace per-request GetByKey calls with a singleflight-backed
RuntimeSettings snapshot loaded via ListSystemConfigsByKeys.
Invalidate snapshot on admin cap_* writes and via Redis pub/sub.
Simplify VerifyMiddleware and update PERFORMANCE.md status.
2026-06-17 12:07:02 +08:00
ryan d3ba767087 perf(config): add Otter RAM cache layer for system configs
Introduce pkg/cache/ram on top of existing Redis/DB config reads.
GetByKey now checks local RAM before Redis. Unified invalidation clears
RAM and Redis hash fields on admin writes, with pub/sub for multi-node
RAM eviction. Add tests and wire create/update/migrator write paths.
2026-06-17 11:57:04 +08:00
ryan ca21e0eca7 perf(frontend,config): parallelize auth, virtualize logs, cache public configs
Frontend layout renders immediately; pages gate via RequireAuth and
useAuthRedirect. Login/register skip getUserInfo. Admin log panel uses
useVirtualizer to avoid 2000-row DOM. ListVisibleSystemConfigs caches
visibility=1 configs in Redis with invalidation on create/update.
PERFORMANCE.md marks completed optimizations.
2026-06-17 11:41:24 +08:00
ryan 6a8f9a7aea perf(upload): optimize file hot paths and incremental admin stats
- Use RWMutex for disk cache reads and singleflight for WebP cache misses
- Cache migration read-only state and file access whitelist with pub/sub invalidation
- Add w_upload_stats incremental counters updated on upload/delete
- Add w_uploads composite indexes and goose backfill migrations
- Document performance analysis in docs/PERFORMANCE.md
- Fix RegisterCustomRoutes to accept apiV1Router parameter
2026-06-17 11:33:48 +08:00
ryan e057cd5de5 fix(update): extracted filename mismatch 2026-06-16 10:51:58 +08:00
ryan 82155b8599 chore(release): bump version to v1.2.0
### 🛠 修复
- 修复了异步任务列表在成功状态下允许重试、失败状态下无重试按钮的问题。
- 修复了启用推送事件时,在无可用推送渠道配置下仍尝试触发并阻塞的问题。
- 修复了 WebDAV 存储驱动在 Put/Get/Delete 等操作中由于丢弃 context.Context 导致 HTTP 链路追踪断裂(生成无源 Root Span)的问题。
- 修复了结构化日志在没有 active span 时仍强制打印全零 traceID/spanID 产生的日志冗余噪音。

### ⚡️ 优化与改进
- 实现了全新的系统通知推送机制,支持 Telegram Bot、Lark 机器人及自定义 Webhook 等多种推送渠道。
- 优化了路由结构设计,按照 V1 分类与业务模块实现扁平化的路由解耦。
- 引入了全局 OpenTelemetry 链路追踪(Tracer)框架,集成 Gin, GORM 与 Redis 自动化耗时度量,并补充了统一的全局错误处理中间件。
- 优化了采样器命名,将 ParentBasedErrorAwareSampler 重命名为更契合其真实机制的 ParentBasedRatioSampler。
- 重构并统一了项目架构为基于 Feature 的功能模块化结构,将 internal/util/ 拆分得更加纯净,优化了验证码等公共库提取(pkg/cap)。
- 实现了推送事件与自定义通道的 Redis 缓存机制,极大降低了推送触发时的高频 DB 查询压力。

### 💄 其他/体验
- 优化了前端界面布局,同步系统菜单与侧边栏配置显示。
- 优化了前端自定义通道表格的布局与样式,使其与事件管理 Tab 页面保持一致。
- 更新了项目开发技能手册(Skills),包括新增接口路由规范(new-api)和异步任务开发指南(new-async-task)。
2026-06-16 10:39:22 +08:00
ryan f129a9cfca feat(api): implement global error handler middleware and trace integration
- Introduce APIError type and AbortWithError helper in response package
- Implement errorHandlerMiddleware to record Go errors to Otel Spans and format JSON response
- Register errorHandlerMiddleware globally in router
- Refactor logs analytics handler to use the new unified error pattern
2026-06-16 10:24:14 +08:00
ryan 9fa38dcfdc feat(push): implement Redis caching for push events and custom channels
- Implement cached queries in GetActivePushEventByKey and GetActivePushChannelByName.
- Set TTL for cached items to 24 hours via activePushEventCacheTTL and activePushChannelCacheTTL constants.
- Implement GORM hooks (AfterSave and AfterDelete) on PushEvent and PushChannel to auto-evict Redis caches, guaranteeing cache consistency.
- Evict Redis caches manually inside API handlers for Create/Update/Delete/Toggle event/channel endpoints.
- Update events.go to query models through the new caching methods.
2026-06-16 10:19:06 +08:00
ryan 9515f0811b chore: fix release ci 2026-06-16 10:08:21 +08:00
ryan c6df73e522 tracer 2026-06-16 09:58:15 +08:00
ryan bfa00ae6ac feat(push): validate push channels presence before enabling push event
- Reject toggling an event to enabled in ToggleEvent handler if channels list is empty.
- Add model-level validation in PushEvent.Validate() to prevent enabling events without channels during creation/updates.
- Update TestPushRouters/toggle_event_status unit test to cover this verification.
2026-06-16 08:59:03 +08:00
ryan 6a913701b9 refactor(push): remove legacy push_global_token setting
- Clean up redundant push_global_token INSERT and DELETE statements from 202606140004 migration.
- Add DELETE for push_global_token in 202606160001 migration to wipe out legacy settings on update.
- Adjust expected w_system_configs count in migrator_test.go to 30.
2026-06-16 08:57:35 +08:00
ryan ff60f1c699 refactor(push): clean up legacy push_config and w_push_events hardcoded insertions
- Remove ConfigKeyPushConfig and delete legacy push_config query logic from EventTrigger.Trigger.
- Simplify the push event notification dispatching engine to rely purely on database custom channels.
- Remove w_push_events hardcoded INSERT statement from migration files, letting SyncEvents handle default event registration.
- Rewrite unit tests to use model.PushChannel instead of push_config.
2026-06-16 08:54:32 +08:00
ryan e50e600bc8 docs(skill): update router packaging and categorization guidelines in documentation
- Updated `new-api` skill `SKILL.md` to document the centralized v1.go route registration and domain-driven sub-routing patterns
- Updated `AGENTS.md` instructions with the new router subpackages (`root`, `v1`) and route file layout guidelines
2026-06-16 00:00:53 +08:00
ryan 0adc5e8189 refactor(router): organize root routes and centralize v1 route registration
- Created `internal/router/root` package to register root-level paths
- Moved files serving, robots.txt, and Swagger docs to `root/default.go`
- Registered custom root routes under `root/custom.go`
- Centralized all v1 routes registration in `internal/router/v1/v1.go`
- Simplified `internal/router/router.go` by delegating route registration
- Updated `new-api` skill `SKILL.md` to document the new router structure
2026-06-15 23:42:48 +08:00
ryan dfcbdfe47c docs(skill): update new-api guidelines to reflect router v1 structure
- Updated package structure diagram and routing instructions to point to the new `internal/router/v1` package
- Changed custom routes registration guide to `internal/router/v1/custom.go`
2026-06-15 23:30:45 +08:00
ryan 5016103774 refactor(router): split route registration into v1 package categorizations
- Created `internal/router/v1` subpackage
- Split routes into `admin.go`, `user.go`, `public.go`, and `custom.go`
- Cleaned up imports and helper registration functions from `router.go`
- Removed obsolete `internal/router/custom.go`
2026-06-15 23:30:16 +08:00
ryan 363d2cb4cc refactor(router): merge frontend.go and frontend_embedded.go into one file
- Declared package-level `registerFrontend` variable in `router.go` as a no-op fallback
- Configured `frontend.go` under `embed_frontend` build tag to override `registerFrontend` on package initialization
- Removed separate `frontend_embedded.go`
- Resolved related revive linter warnings
2026-06-15 23:24:09 +08:00
ryan 34de1639f5 feat(frontend): sync menu display config items with sidebar layout
- Added missing routes `/files` and `/admin/push` to menu display management
- Standardized icons, labels and descriptions to match the sidebar layout
- Declared MenuItem/MenuGroup interfaces for type safety
2026-06-15 23:18:04 +08:00
ryan 7f00ed6a65 fix(task): fix manual retry button visibility and validation
- Only show the manual retry button for failed tasks (status === "failed") on the frontend.
- Remove the maximum retry limit validation for manual retries in the task executor.
- Remove obsolete test cases verifying maximum retry limit for manual retries.
2026-06-15 23:10:36 +08:00
ryan 3e8e879661 docs(architecture): update architectural guidelines to reflect feature-based service design
- Updated AGENTS.md, new-api SKILL.md, and new-async-task SKILL.md to remove references to the deleted global internal/service/ package.
- Documented feature-based local logics/services design within internal/apps/<module>/.
- Updated package comments in internal/diskcache to point to pkg/cache/disk.
2026-06-15 17:24:44 +08:00
ryan 239711cea7 refactor(service): adopt feature-based architecture and rename pkg/diskcache
- Moved GORM/Redis CAPTCHA manager from internal/service/cap directly into the cohesive CAPTCHA app folder at internal/apps/cap/.
- Moved background system cleanup handler from internal/service/cleanup.go into internal/apps/upload/cleanup.go.
- Completely removed the global internal/service directory to keep module logic self-contained.
- Renamed the core utility engine pkg/diskcache to pkg/cache/disk to separate underlying utility code from db/config integrations.
- Renamed DiskCache struct in pkg/cache/disk to Cache to resolve revive package-name stuttering warning.
- Regenerated Swagger API documentation and confirmed all tests compile and pass with 0 linter issues.
2026-06-15 16:45:26 +08:00
ryan 953af7d8db refactor(util): move response helper to common/response and session logic to oauth
- Relocated generic HTTP response helpers (Response, OK, Err, etc.) from internal/util/ to a dedicated internal/common/response/ package.
- Renamed ResponseAny to Any to resolve revive stuttering warnings.
- Moved session building options and cookie headers logic from internal/util/ to internal/apps/oauth/.
- Removed all direct imports of Gin/Sessions/HTTP frameworks from internal/util/ to keep general utilities 100% pure.
- Regenerated Swagger API documentation via make swagger.
- All tests and make code-check compile and pass with 0 issues.
2026-06-15 16:39:55 +08:00
ryan b3ed94342c refactor(backend): extract to pkg/cap 2026-06-15 16:39:55 +08:00
ryan 84ae4ec27e refactor(frontend): optimization 2026-06-15 16:39:55 +08:00
ryan 1425fd1dbb feat(frontend): align custom channels table layout and styling with events tab 2026-06-15 16:39:55 +08:00
ryan 2a3b9f8a5f feat(push-task): connect notification module with task module
- Add task_type to w_push_events table and GORM models.
- Implement OnTaskCompleted callback hook in task executor to avoid circular dependencies.
- Implement task listener in push package to trigger notifications on task completion.
- Automatically resolve User objects from payload and results.
- Enhance UI to select task completed events and preview default templates.
- Update Swagger documentation.
2026-06-15 16:39:55 +08:00
ryan de58b118b4 feat(push): add telegram bot push notification channel
Implement TelegramPusher in pkg/push, register config schema in channels_definition.go, add validation in model/push_channel.go, update task routing, and update settings-tab.tsx UI validation.
2026-06-15 15:00:49 +08:00
ryan cb018b3b60 feat(push): implement system notification and push framework 2026-06-14 23:07:08 +08:00
ryan aee457093d chore(release): v1.1.0
### 🛠 修复
- 修复了 CAPTCHA 验证的逻辑和安全性。

### ⚡️ 优化与改进
- 新增了动态存储配置与多后端迁移机制。
- 新增了手动触发存储迁移的 Web GUI 操作界面。
- 实现了存储迁移的并发处理(使用 Group)与上传后 SHA-256 完整性自动校验机制。
- 实现了基于 Redis 的分布式锁与集群多节点缓存失效广播机制,防止迁移任务冲突。
- 实现了 Redis 锁续期守护协程 (watchdog) 防止超长迁移任务锁过期。
- 优化了更新存储配置时的连通性自动校验机制,防止配置错误。
- 优化了存储配置的内存缓存机制,并引入了全局共享连接池以提高 TCP 复用率。
- 优化了上传前已有同名文件的比对逻辑,跳过下载阶段,仅比对 Content-Length 以实现零网络流量跳过。
- 新增了普通用户的独立文件管理 Dashboard 与控制接口。
- 优化了文件管理接口命名空间,将全局管理迁移至管理员级 API Namespace 隔离控制。
- 优化了管理员新建用户接口,增加了邮箱(Email)字段的必填要求。

### 💄 其他/体验
- 重构并优化了文件管理页面,引入了多标签页多维度统计数据大屏。
- 基于 shadcn 原生 UI 组件重构并优化了文件详情和仪表盘组件。
- 抽取后端核心路由注册流程,降低主路由文件圈复杂度,使路由配置更易维护。
2026-06-13 17:13:36 +08:00
ryan 5698169f20 refactor(router): extract route registration into helper methods 2026-06-13 16:29:27 +08:00
ryan 922f764241 feat(user): require email when admin creates a user
- Add `email` as a required field in `createUserRequest`
- Enforce email format verification and database uniqueness checks in the admin user creation handler
- Update the admin user creation frontend modal with validation and form field
- Update the corresponding backend unit tests and regenerate Swagger docs
2026-06-13 16:23:07 +08:00
ryan 5b13d5b464 feat(storage): implement user-group level file management dashboard and APIs
- Expose user-scoped CRUD APIs under `/api/v1/upload` (my files query, stats, rename, delete)
- Update backend handlers and routers with ownership validation checks
- Create a dedicated frontend personal file manager card-list and upload button under `/files`
- Add comprehensive backend test coverage and update API docs
2026-06-13 16:16:43 +08:00
ryan 8b19ffed90 refactor(storage): move file management routes from user to admin namespace
- Remove file list, stats, download, and deletion routes from '/api/v1/upload'
- Move these endpoints under '/api/v1/admin/uploads'
- Remove user-specific filtering from files query and statistics to aggregate system-wide uploads by default
- Allow admins to bypass ownership check when downloading private files
- Update backend unit tests, Swagger documentation, and frontend service client and components
2026-06-13 16:04:21 +08:00
ryan dbabe8b8d7 feat(system_config): validate storage configuration connectivity on update
- Add a live test connectivity check in UpdateSystemConfig before saving the storage configuration.
- Merge masked placeholder secrets from current configuration prior to testing and database storage.
- Extract validation and test logic to validateAndMergeStorageConfig helper to satisfy cyclomatic complexity.
- Add TestUpdateStorageConfigValidation covering successful updates and failed checks.
2026-06-13 15:57:41 +08:00
ryan 1d37242a8b refactor(storage): update Backend.Put to return PutResult and encapsulate bucket mapping
- Update Backend.Put method signature in storage.go to return (PutResult, error).
- Adjust all backend implementations (local, oss, s3, webdav) to return a PutResult enclosing Key and Bucket.
- Refactor storeUploadFile in upload routers.go to extract key/bucket from PutResult, eliminating manual config bucket lookups.
- Remove the unused cfgBucket helper from storage_ops.go.
- Adjust storage_migration_task.go and tests to accommodate the updated method signature.
2026-06-13 15:48:41 +08:00
ryan ce9423f877 feat(upload): implement Redis lock renewal watchdog for storage migration
- Add a watchdog goroutine that periodically extends the Redis lock TTL every 10 minutes to prevent premature lease expiration for long-running migrations.
- Use context.Background with timeout contexts for lock renewal and final deletion to prevent parent context cancellation from aborting lock cleanup.
- Add nolint directives for gosec and contextcheck and define constants for durations to satisfy strict code quality gates.
2026-06-13 15:31:39 +08:00
ryan 66e0a69666 feat(frontend): support manual storage migration triggering
- Modify storage-config-tab.tsx to separate storage configuration saving from task dispatching.
- Save operations now always preserve the currently active storage driver to prevent premature storage engine switching.
- Add a manual '开始迁移' button to trigger storage migration explicitly.
- Display a warning notice to the operator when the selected storage type differs from the active one.
2026-06-13 15:31:39 +08:00
ryan 26a1e71a27 feat(storage): implement Redis distributed lock and cache invalidation broadcasting
- Add Redis distributed lock in Execute of MigrationHandler to prevent concurrent storage migrations.
- Define storage:config_invalidation Redis pub/sub channel to broadcast cache invalidation events.
- Implement background pub/sub listener on all nodes to evict config memory cache concurrently.
- Add integration tests for distributed locking and invalidation propagation using miniredis.
2026-06-13 15:31:39 +08:00
ryan acd430836f feat(upload): implement parallel storage migration and integrity check
- Parallelize storage migration using `errgroup` with a concurrency limit of 10.
- Perform post-copy SHA-256 data integrity validation to prevent silent data corruption.
- Add test case verifying migration with both incorrect and correct hashes.
2026-06-13 15:31:39 +08:00
ryan bc18800b58 perf(storage): add config caching and shared HTTP connection pool
- Implement thread-safe local config caching with a 5s TTL check.
- Reuse backend client singletons in storage.Active and storage.ForDriver.
- Reset in-memory config cache on configuration saves and updates.
- Create internal/httppool package to manage shared HTTP transports.
- Configure WebDAV client and CDN retrieval to reuse the shared pool.
- Add unit tests for httppool and storage caching behaviors.
2026-06-13 15:31:39 +08:00
ryan 9a18bea324 feat(storage): add dynamic storage config and migration
Move storage backend configuration from startup YAML to system_config-backed runtime configuration. Add local, S3-compatible, R2, MinIO, OSS, and WebDAV backend support.

Add a storage migration async task using the existing task dispatch framework. Migration target config is carried in task payload, and maintenance mode is derived from task execution state.

Split upload file management and storage operations, add the admin storage configuration tab, and update migrations and Swagger docs.
2026-06-13 15:31:38 +08:00