Compare commits

..

8 Commits

Author SHA1 Message Date
truewhile 54b0e33968 优化 2026-10-22 00:44:37 +08:00
truewhile 74487359bf 优 2026-10-21 13:21:13 +08:00
truewhile 55854a0adb 优化播放逻辑 2026-10-21 12:02:11 +08:00
truewhile 121a0a3cc5 优化 2026-10-21 10:11:47 +08:00
truewhile 3057664a48 优化 2026-10-21 09:19:14 +08:00
truewhile 7793d50906 优化 2026-10-21 00:06:16 +08:00
truewhile 5ad71b9421 优化查询 2026-10-20 23:29:51 +08:00
truewhile 5615806c2c Merge pull request #49 from truewhile/feat/media-maintenance-and-diagnostics
完善媒体维护、备份恢复与播放诊断
2026-10-20 22:04:10 +08:00
73 changed files with 4346 additions and 364 deletions
BIN
View File
Binary file not shown.
@@ -0,0 +1,46 @@
# Playback Correctness and Isolation Implementation Plan
**Goal:** Fix the five playback defects from the 2026-10-21 review and make playback capability queries side-effect-free, without replacing the player or adding dependencies.
**Scope:** The user approved implementing the reviewed fixes. Work in the current clean checkout; do not commit, push, deploy, or alter unrelated files.
## Design and invariants
- Keep existing media IDs, stream routes, direct playback and external-player URLs compatible.
- Add optional `session_id` to local HLS requests. Authenticated handlers supply the actual user identity; a client parameter never supplies the owner. New web playback sessions use random IDs. Legacy clients are isolated by user and, where available, Emby play-session/device identity.
- Keep transcoder media lookup/logging separate from its internal task key. A task key incorporates user, media and playback session. Each actual ffmpeg run writes to a fresh generation directory, so delayed segment requests never read a replacement run's output.
- Stop only the authenticated caller's playback session. Check media visibility and scoped playback tokens on stop, just as on playlist/segment requests. Segment and playlist queries must match the running generation.
- Cloud child requests resolve their real session media first, compare query media, enforce token scope and visibility against the real media, and reject mismatches before upstream fetch.
- Treat player business positions as absolute source seconds. Only local HLS converts to/from a relative ffmpeg timeline.
- Invalidate outstanding cloud-transcode requests on media/unmount, explicit source/quality changes, and newer requests. Guard both fulfilled and rejected responses.
- Select local fallbacks only from available local qualities. Keep the existing 115-first preference when a cloud quality is available.
- `PlaybackInfo` only reads capabilities/status. Explicit `StartTranscode` requests enqueue cloud work. Retain pending/ready response contracts, polling, and retries.
## Tasks
1. Add regression tests for local-scope isolation, stale generations, stop authorization, cloud media binding, pure capability reads, timeline conversion, available-local selection and stale async results. Run tests before implementation and verify the intended failures.
2. Add tested playback helpers; route skip/undo/progress/quality/subtitle calculations through the common absolute timeline; guard cloud transcode results and fix local fallback choice.
3. Extend existing transcoder/stream methods with an optional scope, preserve legacy internal calls, isolate task keys/output directories, and pass caller scope from web and Emby handlers. Bind web local HLS URLs and stop requests to captured session IDs.
4. Enforce real-media authorization on cloud child requests and authenticated ownership on local stop.
5. Split read-only cloud capability lookup from explicit cloud enqueue. Update existing cloud tests to use the explicit start operation and test no push during reads.
6. Run focused and full Go tests, frontend tests, TypeScript/Vite build, lint diagnostics and review the diff for security, lifecycle and API compatibility. Fix regressions before reporting completion.
## Verification commands
- `node --experimental-strip-types --test src/pages/hlsPlayback.test.ts src/pages/directPlaySeek.test.ts src/pages/playerPageModel.test.ts src/pages/playbackQuality.test.ts src/pages/playbackRequestGuard.test.ts src/utils/skipSegments.test.ts`
- `go test ./internal/service ./internal/handler ./internal/middleware -count=1`
- `go test ./...`
- `npm run build` and targeted ESLint for changed frontend files.
## Out of scope
New remux/audio-only ffmpeg pipelines, speculative browser codec policy changes, player replacement, broad UI refactors, performance improvement claims without measurements, and production deployment.
## Verification outcome
- Six frontend playback regression scripts pass; targeted ESLint and the TypeScript/Vite production build pass. The existing bundle-size warning remains.
- Playback/stream/transcode/progress tests in service, handler and middleware pass, including scoped output reads, stale generation rejection, scoped stops, device-to-segment propagation, read-only capability queries and concurrent cloud start deduplication.
- Whole-repository tests pass with only `TestQueueLibraryRootScanImportsNewLibraryMedia` and `TestSQLiteRestoreStagesWithoutChangingLiveDatabase` excluded. Both were run independently and reproduce Windows file-lock / SQLite errors in unmodified scan/database code. They are not silently disabled in source.
- FFmpeg is not installed in this environment, so the existing real-transcode lifecycle test skips. Actual video, upstream 115 and browser end-to-end playback remain unverified.
- Clients with no playback-session or device identifier retain per-user legacy isolation; new web requests always carry an independent session ID.
- No commit, push or deployment was performed.
+1 -2
View File
@@ -47,12 +47,11 @@ func setDefaults(v *viper.Viper) {
v.SetDefault("cache.cache_dir", "./cache")
v.SetDefault("cache.images_max_size_mb", 2000)
v.SetDefault("cache.images_originals_max_size_mb", 256)
v.SetDefault("cache.images_originals_ttl_hours", 168)
v.SetDefault("cache.memory_max_size_mb", DefaultCacheMemoryMaxSizeMB)
v.SetDefault("cache.redis_url", "")
v.SetDefault("cache.redis_prefix", "mebox")
v.SetDefault("cache.media_ttl_seconds", 90)
v.SetDefault("cache.emby_latest_ttl_seconds", 300)
v.SetDefault("cache.emby_latest_ttl_seconds", 900)
v.SetDefault("cache.reader_content_max_size_mb", 1024)
v.SetDefault("cache.reader_content_ttl_hours", 168)
v.SetDefault("cache.reader_files_ttl_hours", 720)
-3
View File
@@ -50,9 +50,6 @@ func (c *Config) normalize() error {
if c.Cache.ImagesOriginalsMaxSizeMB < 0 {
c.Cache.ImagesOriginalsMaxSizeMB = 0
}
if c.Cache.ImagesOriginalsTTLHours < 0 {
c.Cache.ImagesOriginalsTTLHours = 0
}
if c.Cache.MemoryMaxSizeMB <= 0 {
c.Cache.MemoryMaxSizeMB = DefaultCacheMemoryMaxSizeMB
}
+2 -4
View File
@@ -125,9 +125,6 @@ type CacheConfig struct {
// ImagesOriginalsMaxSizeMB 限制 cache/images 下原图子配额(MB)。原图只是
// 生成缩略图的原料,丢了可以重新回源;0 表示不单独限制(只受总量约束)。
ImagesOriginalsMaxSizeMB int `mapstructure:"images_originals_max_size_mb"`
// ImagesOriginalsTTLHours 是原图的保留时长(小时),超过即删除;0 表示
// 不按时间淘汰。缩短它能让原图缓存保持在一个滚动窗口内。
ImagesOriginalsTTLHours int `mapstructure:"images_originals_ttl_hours"`
// MemoryMaxSizeMB 限制进程内 L1 缓存总字节数,JSON/对象缓存共用该预算。
MemoryMaxSizeMB int `mapstructure:"memory_max_size_mb"`
RedisURL string `mapstructure:"redis_url"`
@@ -136,7 +133,8 @@ type CacheConfig struct {
// EmbyLatestTTLSeconds 是 Emby「最新添加」(Items/Latest) 的缓存时长。
// 客户端刷新首页时会并发请求全部媒体库的 Latest(生产环境观察到 73 个
// 并发),缓存过短会让这批请求同时穿透并各自重建 payload,在低配主机
// 上造成秒级延迟。默认 300 秒,新入库内容最迟 5 分钟后出现在最新列表。
// 上造成秒级延迟。默认 900 秒;过期后仍会先返回陈旧副本并在后台重建,
// 新入库内容最迟 15 分钟后出现在最新列表(扫描完成会主动失效缓存)。
EmbyLatestTTLSeconds int `mapstructure:"emby_latest_ttl_seconds"`
// ReaderContentMaxSizeMB 阅读正文持久缓存的总容量(MB),0 表示不限制。
ReaderContentMaxSizeMB int `mapstructure:"reader_content_max_size_mb"`
+5
View File
@@ -376,6 +376,11 @@ func ensurePerformanceIndexes(db *gorm.DB) error {
`DROP INDEX IF EXISTS idx_media_library_release_active`,
`CREATE INDEX IF NOT EXISTS idx_media_library_created_active ON media(library_id, created_at DESC) WHERE deleted_at IS NULL`,
`CREATE INDEX IF NOT EXISTS idx_media_library_episode_active ON media(library_id, season_num, episode_num, created_at DESC) WHERE deleted_at IS NULL`,
// 虚拟剧集分组索引:series_id 为空的剧集行要按
// (season_num, episode_num, episode_fraction, created_at) 排序后做内存
// 分组。没有它 SQLite 只能全表扫描再建临时 B-tree,而 Emby 的
// 卡片/海报查询每次只关心其中一部剧(生产环境实测单次 1.3s)。
`CREATE INDEX IF NOT EXISTS idx_media_virtual_series_group ON media(season_num, episode_num, episode_fraction, created_at) WHERE deleted_at IS NULL AND (series_id IS NULL OR series_id = '')`,
`CREATE INDEX IF NOT EXISTS idx_media_library_root_active ON media(library_id, library_root_id) WHERE deleted_at IS NULL`,
`CREATE INDEX IF NOT EXISTS idx_media_series_active ON media(series_id, season_num, episode_num) WHERE deleted_at IS NULL`,
`CREATE INDEX IF NOT EXISTS idx_media_library_series_active ON media(library_id, series_id, season_num, episode_num) WHERE deleted_at IS NULL`,
+1 -1
View File
@@ -87,7 +87,7 @@ func updateSettingHandler(svc *service.Container) gin.HandlerFunc {
if req.Key == "transcode.hw_enabled" || req.Key == "transcode.hw_accel" || req.Key == "transcoder.hardware_accel" || req.Key == "transcoder.encoder" {
svc.Transcoder.StopAll()
}
if req.Key == "cache.images_max_size_mb" || req.Key == "cache.images_originals_max_size_mb" || req.Key == "cache.images_originals_ttl_hours" {
if req.Key == "cache.images_max_size_mb" || req.Key == "cache.images_originals_max_size_mb" {
if svc.Scheduler != nil {
_ = svc.Scheduler.RunNowAsync(c.Request.Context(), "image_cache_cleanup")
}
+24 -2
View File
@@ -124,10 +124,32 @@ func cloud115HLSSessionHandler(svc *service.Container) gin.HandlerFunc {
c.JSON(http.StatusBadRequest, gin.H{"error": "missing media_id"})
return
}
if !enforceScopedPlaybackToken(c, mediaID) {
if c.Request.Context().Err() != nil {
c.AbortWithStatus(499)
return
}
err := svc.Cloud115.HLSProxy().ServeChild(
actualMediaID, err := svc.Cloud115.HLSProxy().SessionMediaID(c.Param("session"), c.Param("key"))
if err != nil {
c.JSON(http.StatusNotFound, gin.H{"code": "hls_session_not_found", "error": "hls session not found"})
return
}
if mediaID != actualMediaID {
c.JSON(http.StatusForbidden, gin.H{"error": "playback session media mismatch"})
return
}
if !enforceScopedPlaybackToken(c, actualMediaID) {
return
}
if svc.Repo == nil || svc.Repo.Media == nil {
c.JSON(http.StatusNotFound, gin.H{"error": "media not found"})
return
}
m, findErr := svc.Repo.Media.FindByID(c.Request.Context(), actualMediaID)
if findErr != nil || m == nil || !mediaVisibleForRequest(c, svc, m) {
c.JSON(http.StatusNotFound, gin.H{"error": "media not found"})
return
}
err = svc.Cloud115.HLSProxy().ServeChild(
c.Request.Context(),
c.Writer,
c.Request,
+19
View File
@@ -50,6 +50,25 @@ func embyServeImage(c *gin.Context, svc *service.Container, id, imageType, tag s
embyServePlaceholderImage(c)
return
}
// 媒体库主图:按 Emby 的做法,用库内几张成员海报拼一张封面(folder
// image)。没有自定义封面时才生成;自定义封面继续走下面的代理路径。
if svc.ImageProxy != nil && strings.EqualFold(strings.TrimSpace(imageType), "Primary") {
if sources, lib := service.LibraryCoverSourcesForLibrary(ctx, svc.Repo, id); lib != nil &&
strings.TrimSpace(lib.CoverURL) == "" && len(sources) > 0 {
width, height := libraryCoverRequestSize(c)
if data, err := svc.ImageProxy.LibraryCoverImage(ctx, id, sources, width, height); err == nil && len(data) > 0 {
c.Header("Cache-Control", "public, max-age=300, stale-while-revalidate=86400")
c.Header("Content-Type", "image/jpeg")
c.Header("Content-Length", strconv.Itoa(len(data)))
if c.Request.Method == http.MethodHead {
c.Status(http.StatusOK)
return
}
c.Data(http.StatusOK, "image/jpeg", data)
return
}
}
}
// PersonImageURL handles TMDb/remote people first and falls back to regular
// media/library artwork, so the official /Items/{personId}/Images route
// works for synthetic person IDs as well as normal item IDs.
+10 -2
View File
@@ -418,7 +418,11 @@ func embyVideoHLSPlaylistHandler(svc *service.Container) gin.HandlerFunc {
c.Status(http.StatusNotFound)
return
}
err = svc.Stream.ServeHLSPlaylist(c.Writer, c.Request, c.Param("id"))
scope, ok := localHLSPlaybackScope(c)
if !ok {
return
}
err = svc.Stream.ServeHLSPlaylist(c.Writer, c.Request, c.Param("id"), scope)
if errors.Is(err, service.ErrTranscodeDisabled) {
c.JSON(http.StatusConflict, gin.H{"error": "transcode disabled"})
return
@@ -444,7 +448,11 @@ func embyVideoHLSSegmentHandler(svc *service.Container) gin.HandlerFunc {
c.Status(http.StatusNotFound)
return
}
if err := svc.Stream.ServeHLSSegment(c.Writer, c.Request, c.Param("id"), c.Param("seg")); err != nil {
scope, ok := localHLSPlaybackScope(c)
if !ok {
return
}
if err := svc.Stream.ServeHLSSegment(c.Writer, c.Request, c.Param("id"), c.Param("seg"), scope); err != nil {
c.Status(http.StatusNotFound)
}
}
+37
View File
@@ -0,0 +1,37 @@
package handler
import (
"net/http"
"net/http/httptest"
"testing"
"go.uber.org/zap"
"github.com/truewhile/MeBox/internal/middleware"
"github.com/truewhile/MeBox/internal/model"
"github.com/truewhile/MeBox/internal/service"
)
func TestStopHLSChecksMediaVisibility(t *testing.T) {
router, svc, secret := newPlaybackScopeTestRouter(t)
svc.Transcoder = service.NewTranscoderService(svc.Cfg, zap.NewNop(), svc.Repo, nil)
router.DELETE("/api/hls/:id", middleware.AuthRequired(secret), stopTranscodeHandler(svc))
req := httptest.NewRequest(http.MethodDelete, "/api/hls/nonexistent?session_id=player-a", nil)
req.Header.Set("Authorization", "Bearer "+signedTestToken(t, secret))
rec := httptest.NewRecorder()
router.ServeHTTP(rec, req)
if rec.Code != http.StatusNotFound {
t.Fatalf("stop of a nonexistent/invisible media: status=%d body=%s, want 404", rec.Code, rec.Body.String())
}
profile := &model.PlayProfile{UserID: "user-1", Name: "restricted", IsDefault: true, AllowedLibraryIDs: `["other-library"]`}
if err := svc.Repo.DB.Create(profile).Error; err != nil {
t.Fatal(err)
}
req = httptest.NewRequest(http.MethodDelete, "/api/hls/media-1?session_id=player-a", nil)
req.Header.Set("Authorization", "Bearer "+signedTestToken(t, secret))
rec = httptest.NewRecorder()
router.ServeHTTP(rec, req)
if rec.Code != http.StatusNotFound {
t.Fatalf("invisible existing media can be stopped: %d %s", rec.Code, rec.Body.String())
}
}
+100
View File
@@ -0,0 +1,100 @@
package handler
import (
"context"
"crypto/sha256"
"encoding/hex"
"net/http"
"strconv"
"strings"
"time"
"github.com/gin-gonic/gin"
"github.com/truewhile/MeBox/internal/service"
)
// libraryCoverHandler 输出媒体库封面图(Emby 风格的成员海报拼图)。
//
// 之前 Web 端是拿两张成员海报在前端用 CSS 拼格子,Emby 客户端则直接拿一张
// 评分最高的海报,结果同一个库在不同客户端长得完全不一样。现在服务端生成
// 一张 16:9 的拼图(Emby 的 folder image 做法),Emby 客户端、Web 和 App
// 都指向这一份,外观统一。
func libraryCoverHandler(svc *service.Container) gin.HandlerFunc {
return func(c *gin.Context) {
id := strings.TrimSpace(c.Param("id"))
if id == "" || svc == nil || svc.ImageProxy == nil {
c.JSON(http.StatusNotFound, gin.H{"error": "cover unavailable"})
return
}
ctx, cancel := context.WithTimeout(c.Request.Context(), 25*time.Second)
defer cancel()
sources, lib := service.LibraryCoverSourcesForLibrary(ctx, svc.Repo, id)
if lib == nil {
c.JSON(http.StatusNotFound, gin.H{"error": "library not found"})
return
}
if strings.TrimSpace(lib.CoverURL) != "" {
// 管理员自定义封面优先,保持既有行为(代理该地址)。
if err := svc.ImageProxy.Serve(ctx, c.Writer, c.Request, lib.CoverURL); err != nil {
c.JSON(http.StatusNotFound, gin.H{"error": "custom cover unavailable"})
}
return
}
if len(sources) == 0 {
c.JSON(http.StatusNotFound, gin.H{"error": "no library items with artwork"})
return
}
width, height := libraryCoverRequestSize(c)
data, err := svc.ImageProxy.LibraryCoverImage(ctx, id, sources, width, height)
if err != nil || len(data) == 0 {
c.JSON(http.StatusNotFound, gin.H{"error": "cover generation failed"})
return
}
writeLibraryCoverImage(c, data, http.StatusOK)
}
}
// libraryCoverRequestSize 解析请求的封面尺寸:Web 用 w/h,Emby 用
// maxWidth/maxHeight,两种都兼容。服务端统一输出 16:9 画布(与该库在 Emby
// 里上报的 PrimaryImageAspectRatio 一致),客户端传的尺寸只当作「不超过」的
// 边界使用。
func libraryCoverRequestSize(c *gin.Context) (int, int) {
return firstPositiveQueryIntParam(c, "maxWidth", "maxwidth", "width", "w"),
firstPositiveQueryIntParam(c, "maxHeight", "maxheight", "height", "h")
}
func firstPositiveQueryIntParam(c *gin.Context, names ...string) int {
for _, name := range names {
raw := strings.TrimSpace(c.Query(name))
if raw == "" {
continue
}
if n, err := strconv.Atoi(raw); err == nil && n > 0 {
return n
}
if f, err := strconv.ParseFloat(raw, 64); err == nil && f > 0 {
return int(f)
}
}
return 0
}
func writeLibraryCoverImage(c *gin.Context, data []byte, status int) {
sum := sha256.Sum256(data)
etag := `"` + hex.EncodeToString(sum[:8]) + `"`
c.Header("ETag", etag)
c.Header("Cache-Control", "public, max-age=300, stale-while-revalidate=86400")
if match := strings.TrimSpace(c.GetHeader("If-None-Match")); match != "" && match == etag {
c.Status(http.StatusNotModified)
return
}
c.Header("Content-Type", "image/jpeg")
c.Header("Content-Length", strconv.Itoa(len(data)))
if c.Request.Method == http.MethodHead {
c.Status(status)
return
}
c.Data(status, "image/jpeg", data)
}
@@ -0,0 +1,118 @@
package handler
import (
"bytes"
"image/color"
"image/jpeg"
"net/http"
"net/http/httptest"
"net/url"
"path/filepath"
"testing"
"time"
"github.com/gin-gonic/gin"
"github.com/golang-jwt/jwt/v5"
"github.com/glebarez/sqlite"
"go.uber.org/zap"
"gorm.io/gorm"
"github.com/truewhile/MeBox/internal/config"
"github.com/truewhile/MeBox/internal/middleware"
"github.com/truewhile/MeBox/internal/model"
"github.com/truewhile/MeBox/internal/repository"
"github.com/truewhile/MeBox/internal/service"
)
func coverTestToken(t *testing.T, secret, userID string) string {
t.Helper()
token := jwt.NewWithClaims(jwt.SigningMethodHS256, middleware.Claims{
UserID: userID,
Role: "admin",
RegisteredClaims: jwt.RegisteredClaims{
ExpiresAt: jwt.NewNumericDate(time.Now().Add(time.Hour)),
IssuedAt: jwt.NewNumericDate(time.Now()),
Subject: userID,
},
})
signed, err := token.SignedString([]byte(secret))
if err != nil {
t.Fatalf("sign token: %v", err)
}
return signed
}
// TestLibraryCoverRouteServesImageThroughFullRouter 走完整的 handler.Register
// 路由表请求封面:既验证路由挂载正确,也验证鉴权链(AuthRequired +
// activeUserRequired)不会把它挡成 404。
func TestLibraryCoverRouteServesImageThroughFullRouter(t *testing.T) {
gin.SetMode(gin.TestMode)
const secret = "test-secret"
poster := coverTestJPEG(t, color.RGBA{R: 200, G: 120, B: 60, A: 255})
upstream := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
w.Header().Set("Content-Type", "image/jpeg")
_, _ = w.Write(poster)
}))
defer upstream.Close()
parsed, _ := url.Parse(upstream.URL)
db, err := gorm.Open(sqlite.Open(":memory:"), &gorm.Config{})
if err != nil {
t.Fatalf("open db: %v", err)
}
if err := db.AutoMigrate(&model.Library{}, &model.Media{}, &model.User{}); err != nil {
t.Fatalf("migrate: %v", err)
}
repos := repository.New(db)
user := model.User{Base: model.Base{ID: "user-1"}, Username: "viewer", IsActive: true}
if err := db.Create(&user).Error; err != nil {
t.Fatalf("create user: %v", err)
}
lib := model.Library{Name: "电影", Path: "/media/movies", Type: "movie", Enabled: true}
if err := repos.Library.Create(t.Context(), &lib); err != nil {
t.Fatalf("create library: %v", err)
}
for i := 0; i < 4; i++ {
row := model.Media{
Base: model.Base{ID: "m" + string(rune('a'+i))},
LibraryID: lib.ID,
Title: "Movie " + string(rune('A'+i)),
Path: "/media/movies/m" + string(rune('a'+i)) + ".mkv",
PosterURL: upstream.URL + "/p" + string(rune('1'+i)) + ".jpg",
}
if err := db.Create(&row).Error; err != nil {
t.Fatalf("create media: %v", err)
}
}
cfg := &config.Config{
Cache: config.CacheConfig{CacheDir: filepath.Join(t.TempDir(), "cache")},
Secrets: config.SecretsConfig{JWTSecret: secret},
}
proxy := service.NewImageProxy(cfg, zap.NewNop())
proxy.SetAllowedRemoteHostsProvider(func() []string { return []string{parsed.Host} })
svc := &service.Container{Repo: repos, ImageProxy: proxy}
engine := gin.New()
Register(engine, cfg, zap.NewNop(), svc)
token := coverTestToken(t, secret, user.ID)
req := httptest.NewRequest(http.MethodGet, "/api/libraries/"+lib.ID+"/cover?w=480&h=270&api_key="+token, nil)
w := httptest.NewRecorder()
engine.ServeHTTP(w, req)
if w.Code != http.StatusOK {
t.Fatalf("status = %d body=%s", w.Code, w.Body.String())
}
if ct := w.Header().Get("Content-Type"); ct != "image/jpeg" {
t.Fatalf("content type = %q, want image/jpeg", ct)
}
img, err := jpeg.Decode(bytes.NewReader(w.Body.Bytes()))
if err != nil {
t.Fatalf("decode cover: %v", err)
}
if img.Bounds().Dx() != 480 || img.Bounds().Dy() != 270 {
t.Fatalf("cover size = %v, want 480x270", img.Bounds())
}
}
+228
View File
@@ -0,0 +1,228 @@
package handler
import (
"bytes"
"image"
"image/color"
"image/jpeg"
"net/http"
"net/http/httptest"
"net/url"
"path/filepath"
"testing"
"github.com/gin-gonic/gin"
"github.com/glebarez/sqlite"
"go.uber.org/zap"
"gorm.io/gorm"
"github.com/truewhile/MeBox/internal/config"
"github.com/truewhile/MeBox/internal/model"
"github.com/truewhile/MeBox/internal/repository"
"github.com/truewhile/MeBox/internal/service"
)
func coverTestJPEG(t *testing.T, c color.RGBA) []byte {
t.Helper()
img := image.NewRGBA(image.Rect(0, 0, 160, 240))
for y := 0; y < 240; y++ {
for x := 0; x < 160; x++ {
img.Set(x, y, c)
}
}
var buf bytes.Buffer
if err := jpeg.Encode(&buf, img, &jpeg.Options{Quality: 90}); err != nil {
t.Fatalf("encode jpeg: %v", err)
}
return buf.Bytes()
}
// newLibraryCoverTestRouter 构造一个带真实图片代理的路由:成员海报来自
// 本地 httptest 服务,走的是与生产一致的 ImageProxy 抓取路径。
func newLibraryCoverTestRouter(t *testing.T) (*gin.Engine, *repository.Container, *httptest.Server) {
t.Helper()
gin.SetMode(gin.TestMode)
poster := coverTestJPEG(t, color.RGBA{R: 200, G: 120, B: 60, A: 255})
upstream := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
w.Header().Set("Content-Type", "image/jpeg")
_, _ = w.Write(poster)
}))
t.Cleanup(upstream.Close)
db, err := gorm.Open(sqlite.Open(":memory:"), &gorm.Config{})
if err != nil {
t.Fatalf("open db: %v", err)
}
if err := db.AutoMigrate(&model.Library{}, &model.Media{}); err != nil {
t.Fatalf("migrate: %v", err)
}
repos := repository.New(db)
parsed, err := url.Parse(upstream.URL)
if err != nil {
t.Fatalf("parse upstream: %v", err)
}
cfg := &config.Config{Cache: config.CacheConfig{CacheDir: filepath.Join(t.TempDir(), "cache")}}
proxy := service.NewImageProxy(cfg, zap.NewNop())
proxy.SetAllowedRemoteHostsProvider(func() []string { return []string{parsed.Host} })
svc := &service.Container{Repo: repos, ImageProxy: proxy}
router := gin.New()
router.GET("/libraries/:id/cover", libraryCoverHandler(svc))
router.HEAD("/libraries/:id/cover", libraryCoverHandler(svc))
return router, repos, upstream
}
func seedCoverLibrary(t *testing.T, repos *repository.Container, posterBase string, coverURL string) model.Library {
t.Helper()
lib := model.Library{Name: "电影", Path: "/media/movies", Type: "movie", Enabled: true, CoverURL: coverURL}
if err := repos.Library.Create(t.Context(), &lib); err != nil {
t.Fatalf("create library: %v", err)
}
for i := 0; i < 4; i++ {
row := model.Media{
Base: model.Base{ID: "m" + string(rune('a'+i))},
LibraryID: lib.ID,
Title: "Movie " + string(rune('A'+i)),
Path: "/media/movies/m" + string(rune('a'+i)) + ".mkv",
PosterURL: posterBase + "/p" + string(rune('1'+i)) + ".jpg",
}
if err := repos.DB.Create(&row).Error; err != nil {
t.Fatalf("create media: %v", err)
}
}
return lib
}
// TestLibraryCoverHandlerServesGeneratedCollage 覆盖 Web/App 的封面接口:
// 返回的是服务端拼好的 16:9 图片,而不是单张成员海报。
func TestLibraryCoverHandlerServesGeneratedCollage(t *testing.T) {
router, repos, upstream := newLibraryCoverTestRouter(t)
lib := seedCoverLibrary(t, repos, upstream.URL, "")
req := httptest.NewRequest(http.MethodGet, "/libraries/"+lib.ID+"/cover?w=480&h=270", nil)
w := httptest.NewRecorder()
router.ServeHTTP(w, req)
if w.Code != http.StatusOK {
t.Fatalf("status = %d body=%s", w.Code, w.Body.String())
}
if ct := w.Header().Get("Content-Type"); ct != "image/jpeg" {
t.Fatalf("content type = %q, want image/jpeg", ct)
}
img, err := jpeg.Decode(bytes.NewReader(w.Body.Bytes()))
if err != nil {
t.Fatalf("decode cover: %v", err)
}
if img.Bounds().Dx() != 480 || img.Bounds().Dy() != 270 {
t.Fatalf("cover size = %v, want 480x270", img.Bounds())
}
if etag := w.Header().Get("ETag"); etag == "" {
t.Fatal("expected an ETag so clients can revalidate")
}
// 条件请求应拿到 304。
req = httptest.NewRequest(http.MethodGet, "/libraries/"+lib.ID+"/cover?w=480&h=270", nil)
req.Header.Set("If-None-Match", w.Header().Get("ETag"))
rec := httptest.NewRecorder()
router.ServeHTTP(rec, req)
if rec.Code != http.StatusNotModified {
t.Fatalf("conditional request status = %d, want 304", rec.Code)
}
}
func TestLibraryCoverHandlerUnknownLibraryReturns404(t *testing.T) {
router, _, _ := newLibraryCoverTestRouter(t)
req := httptest.NewRequest(http.MethodGet, "/libraries/missing/cover", nil)
w := httptest.NewRecorder()
router.ServeHTTP(w, req)
if w.Code != http.StatusNotFound {
t.Fatalf("status = %d, want 404", w.Code)
}
}
// TestLibraryCoverHandlerServesCustomCover 管理员填了 cover_url 时必须继续
// 用自定义封面,不能在不知不觉中换成拼图。
func TestLibraryCoverHandlerServesCustomCover(t *testing.T) {
router, repos, upstream := newLibraryCoverTestRouter(t)
lib := seedCoverLibrary(t, repos, upstream.URL, upstream.URL+"/custom.jpg")
req := httptest.NewRequest(http.MethodGet, "/libraries/"+lib.ID+"/cover", nil)
w := httptest.NewRecorder()
router.ServeHTTP(w, req)
if w.Code != http.StatusOK {
t.Fatalf("status = %d body=%s", w.Code, w.Body.String())
}
if _, err := jpeg.Decode(bytes.NewReader(w.Body.Bytes())); err != nil {
t.Fatalf("custom cover should be an image: %v", err)
}
}
// TestEmbyLibraryPrimaryUsesGeneratedCollage 保证 Emby 客户端请求库主图时
// 拿到的也是拼图(与 Web/App 同一份生成逻辑)。
func TestEmbyLibraryPrimaryUsesGeneratedCollage(t *testing.T) {
gin.SetMode(gin.TestMode)
db, err := gorm.Open(sqlite.Open(":memory:"), &gorm.Config{})
if err != nil {
t.Fatalf("open db: %v", err)
}
if err := db.AutoMigrate(&model.Library{}, &model.Media{}); err != nil {
t.Fatalf("migrate: %v", err)
}
poster := coverTestJPEG(t, color.RGBA{G: 180, R: 90, B: 60, A: 255})
upstream := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
w.Header().Set("Content-Type", "image/jpeg")
_, _ = w.Write(poster)
}))
defer upstream.Close()
parsed, _ := url.Parse(upstream.URL)
repos := repository.New(db)
lib := model.Library{Name: "剧集", Path: "/media/tv", Type: "tv", Enabled: true}
if err := repos.Library.Create(t.Context(), &lib); err != nil {
t.Fatalf("create library: %v", err)
}
for i := 0; i < 4; i++ {
row := model.Media{
Base: model.Base{ID: "e" + string(rune('a'+i))},
LibraryID: lib.ID,
Title: "Show " + string(rune('A'+i)),
Path: "/media/tv/show" + string(rune('a'+i)) + "/S01E01.mkv",
SeasonNum: 1, EpisodeNum: 1,
PosterURL: upstream.URL + "/p" + string(rune('1'+i)) + ".jpg",
}
if err := repos.DB.Create(&row).Error; err != nil {
t.Fatalf("create media: %v", err)
}
}
cfg := &config.Config{Cache: config.CacheConfig{CacheDir: filepath.Join(t.TempDir(), "cache")}}
proxy := service.NewImageProxy(cfg, zap.NewNop())
proxy.SetAllowedRemoteHostsProvider(func() []string { return []string{parsed.Host} })
svc := &service.Container{
Repo: repos,
Emby: service.NewEmbyService(cfg, zap.NewNop(), repos),
ImageProxy: proxy,
}
router := gin.New()
registerEmbyRoutes(router, "test-secret", svc)
req := httptest.NewRequest(http.MethodGet, "/Items/"+lib.ID+"/Images/Primary?maxWidth=400&maxHeight=300", nil)
w := httptest.NewRecorder()
router.ServeHTTP(w, req)
if w.Code != http.StatusOK {
t.Fatalf("status = %d body=%s", w.Code, w.Body.String())
}
img, err := jpeg.Decode(bytes.NewReader(w.Body.Bytes()))
if err != nil {
t.Fatalf("decode emby library cover: %v", err)
}
// 请求的是 400×300 的盒子,拼图按 16:9 装进去 → 400×225。
if img.Bounds().Dx() != 400 || img.Bounds().Dy() != 225 {
t.Fatalf("cover size = %v, want 400x225 (16:9 inside the requested box)", img.Bounds())
}
}
+142 -16
View File
@@ -4,8 +4,10 @@ package handler
import (
"context"
"errors"
"fmt"
"math"
"net/http"
"net/url"
"sort"
"strconv"
"strings"
@@ -39,8 +41,32 @@ type webLibraryPayload struct {
RemoteSource string `json:"remote_source,omitempty"`
Total int64 `json:"total,omitempty"`
Cards []service.SeriesCardView `json:"cards,omitempty"`
// CoverImageURL 是客户端可直接使用的「有效封面」:管理员自定义的
// cover_url 优先,否则指向服务端生成的成员海报拼图接口。Web / App 都
// 读这一个字段,避免各自用卡片在前端拼格子、长得还不一样。
CoverImageURL string `json:"cover_image_url,omitempty"`
}
// effectiveLibraryCoverURL 计算库封面的有效地址。远程挂载库的封面由上游
// Emby 决定,这里返回空串,调用方继续用卡片拼图兜底。
func effectiveLibraryCoverURL(lib model.Library, isRemote bool) string {
if isRemote {
return ""
}
if raw := strings.TrimSpace(lib.CoverURL); raw != "" {
return raw
}
if strings.TrimSpace(lib.ID) == "" {
return ""
}
return "/api/libraries/" + url.PathEscape(lib.ID) + "/cover"
}
// remoteLibraryPreviewBudget 限制 /api/libraries?with_preview=1 里远程 Emby
// 库预览(总条数 + 最新卡片)的总耗时。这些预览只用于首页卡片,值得为它
// 排队等待的量很小;此前它跟随请求上下文,上游超时会让整页等 15 秒以上。
var remoteLibraryPreviewBudget = 3 * time.Second
// remoteLibraryItemTypes 远程库内容拉取时按 CollectionType 过滤直属条目,
// 避免电影库里的合集文件夹(Folder) 漏出为电影卡片。
func remoteLibraryItemTypes(collectionType string) string {
@@ -118,9 +144,10 @@ func listLibrariesHandler(svc *service.Container) gin.HandlerFunc {
}
for _, p := range previews {
out = append(out, webLibraryPayload{
Library: p.Library,
Total: p.Total,
Cards: service.NewSeriesCardViews(p.Cards),
Library: p.Library,
Total: p.Total,
Cards: service.NewSeriesCardViews(p.Cards),
CoverImageURL: effectiveLibraryCoverURL(p.Library, false),
})
}
} else {
@@ -139,7 +166,7 @@ func listLibrariesHandler(svc *service.Container) gin.HandlerFunc {
if counts != nil {
total = counts[l.ID]
}
out = append(out, webLibraryPayload{Library: l, Total: total})
out = append(out, webLibraryPayload{Library: l, Total: total, CoverImageURL: effectiveLibraryCoverURL(l, false)})
}
}
// 精确指定目标库时,如果目标全是本地库,就不必枚举远程挂载。
@@ -177,6 +204,12 @@ func listLibrariesHandler(svc *service.Container) gin.HandlerFunc {
remotePayloads[i] = webLibraryPayload{Library: v.Library, IsRemoteEmby: true, RemoteSource: v.AccountName}
}
if withPreview && len(allowedViews) > 0 {
// 远程库预览只是首页卡片上的「最新几条」,绝不能拖住整个
// 库列表:上游 Emby 一旦超时,每个远程库都要等满
// embyRemoteHTTPTimeout(15s),串起来就是 17–30 秒。
// 给整段预览一个总预算,超时就只返回库名,卡片留给下一次刷新。
previewCtx, cancelPreview := context.WithTimeout(ctx, remoteLibraryPreviewBudget)
defer cancelPreview()
const maxRemotePreviewWorkers = 6
sem := make(chan struct{}, maxRemotePreviewWorkers)
var wg sync.WaitGroup
@@ -188,11 +221,11 @@ func listLibrariesHandler(svc *service.Container) gin.HandlerFunc {
select {
case sem <- struct{}{}:
defer func() { <-sem }()
case <-ctx.Done():
case <-previewCtx.Done():
return
}
helper.Run(svc.Log, "media.remotePreview", func() {
acct := svc.EmbyRemote.AccountByID(ctx, v.AccountID)
acct := svc.EmbyRemote.AccountByID(previewCtx, v.AccountID)
if acct == nil {
return
}
@@ -204,10 +237,10 @@ func listLibrariesHandler(svc *service.Container) gin.HandlerFunc {
Name: v.Library.Name,
}
itemTypes := remoteLibraryItemTypes(v.CollectionType)
if _, total, err := svc.EmbyRemote.RemoteLibraryMedia(ctx, tmpMount, acct, v.RemoteID, itemTypes, 0, 1); err == nil {
if _, total, err := svc.EmbyRemote.RemoteLibraryMedia(previewCtx, tmpMount, acct, v.RemoteID, itemTypes, 0, 1); err == nil {
remotePayloads[i].Total = total
}
if cards, err := svc.EmbyRemote.RemoteLatestCards(ctx, tmpMount, acct, v.RemoteID, limit); err == nil {
if cards, err := svc.EmbyRemote.RemoteLatestCards(previewCtx, tmpMount, acct, v.RemoteID, limit); err == nil {
remotePayloads[i].Cards = service.NewSeriesCardViews(cards)
}
})
@@ -260,9 +293,9 @@ func getLibraryHandler(svc *service.Container) gin.HandlerFunc {
c.JSON(http.StatusNotFound, gin.H{"error": "not found"})
return
}
c.JSON(http.StatusOK, webLibraryPayload{Library: libs[0]})
c.JSON(http.StatusOK, webLibraryPayload{Library: libs[0], CoverImageURL: effectiveLibraryCoverURL(libs[0], false)})
} else {
c.JSON(http.StatusOK, webLibraryPayload{Library: *lib})
c.JSON(http.StatusOK, webLibraryPayload{Library: *lib, CoverImageURL: effectiveLibraryCoverURL(*lib, false)})
}
}
}
@@ -705,6 +738,8 @@ func searchMediaHandler(svc *service.Container) gin.HandlerFunc {
}
paginated := c.Query("page") != "" || c.Query("page_size") != ""
sortBy := normalizeMediaSearchSort(c.Query("sort"))
// 远程搜索与本地查询并行发起:本地结果不再排在远程超时之后才返回。
remoteCh := startRemoteMediaSearch(ctx, svc, query, visibility, filter)
// Sorting and grouping must operate on the same candidate set before
// either a page size or the caller's output limit is applied.
items, err := svc.Media.SearchMediaVisibleFiltered(ctx, query, 50000, visibility, filter)
@@ -712,12 +747,9 @@ func searchMediaHandler(svc *service.Container) gin.HandlerFunc {
c.JSON(http.StatusInternalServerError, gin.H{"error": err.Error()})
return
}
if svc.EmbyRemote != nil && query != "" && (filter.LibraryID == "" || service.IsEmbyRemoteID(filter.LibraryID)) {
remote, _ := svc.EmbyRemote.RemoteSearchMedia(ctx, query, 50000, visibility)
for _, item := range remote {
if remoteSearchMatchesFilter(item, filter) {
items = append(items, item)
}
if remoteCh != nil {
for _, item := range <-remoteCh {
items = append(items, item)
}
}
if c.DefaultQuery("group_series", "0") != "0" {
@@ -741,6 +773,100 @@ func searchMediaHandler(svc *service.Container) gin.HandlerFunc {
}
}
const (
// remoteSearchItemLimit 是每个远程 Emby 挂载点返回的条目上限。此前这里传
// 50000(等于让上游做一次全库查询),上游稍慢就会一路跑到 5 秒超时,把每次
// 搜索都拖成秒级;搜索场景只需要少量候选,多出来的结果也会被分页丢掉。
remoteSearchItemLimit = 100
// remoteSearchBudget 限制远程结果最多给一次请求增加多少延迟。
remoteSearchBudget = 2 * time.Second
// remoteSearchCacheTTL 让同一关键词的翻页与重复搜索复用远程结果,避免每翻
// 一页都重新打一遍远程挂载点。
remoteSearchCacheTTL = 30 * time.Second
// remoteSearchFailureCacheTTL 用于上游超时/失败时的短缓存:部分结果只保留
// 几秒,避免反复吃满 remoteSearchBudget,又不至于长期压住后续重试。
remoteSearchFailureCacheTTL = 5 * time.Second
)
// startRemoteMediaSearch 并发发起远程搜索,返回一个只投递一次结果的通道。
// 不满足远程搜索条件时返回 nil。通道带缓冲,调用方即使提前返回也不会泄漏
// goroutine;请求结束时请求上下文被取消,远程调用会随之退出。
func startRemoteMediaSearch(
ctx context.Context,
svc *service.Container,
query string,
visibility service.MediaVisibility,
filter service.MediaSearchFilter,
) <-chan []model.Media {
if svc == nil || svc.EmbyRemote == nil || query == "" {
return nil
}
if filter.LibraryID != "" && !service.IsEmbyRemoteID(filter.LibraryID) {
return nil
}
results := make(chan []model.Media, 1)
go func() {
results <- remoteMediaSearchItems(ctx, svc, query, visibility, filter)
}()
return results
}
func remoteMediaSearchItems(
ctx context.Context,
svc *service.Container,
query string,
visibility service.MediaVisibility,
filter service.MediaSearchFilter,
) []model.Media {
cacheKey := remoteSearchCacheKey(query, visibility)
if svc.Cache != nil {
if cached, ok := svc.Cache.GetObject(cacheKey); ok {
if rows, ok := cached.([]model.Media); ok {
return filterRemoteSearchItems(rows, filter)
}
}
}
searchCtx, cancel := context.WithTimeout(ctx, remoteSearchBudget)
defer cancel()
remote, _ := svc.EmbyRemote.RemoteSearchMedia(searchCtx, query, remoteSearchItemLimit, visibility)
if svc.Cache != nil {
// 空结果也要缓存:否则查不到片名时每次都会重新等到 remoteSearchBudget。
if remote == nil {
remote = []model.Media{}
}
ttl := remoteSearchCacheTTL
if searchCtx.Err() != nil {
ttl = remoteSearchFailureCacheTTL
}
svc.Cache.SetObject(cacheKey, remote, ttl)
}
return filterRemoteSearchItems(remote, filter)
}
func filterRemoteSearchItems(remote []model.Media, filter service.MediaSearchFilter) []model.Media {
if len(remote) == 0 {
return nil
}
out := make([]model.Media, 0, len(remote))
for _, item := range remote {
if remoteSearchMatchesFilter(item, filter) {
out = append(out, item)
}
}
return out
}
// remoteSearchCacheKey 按可见范围隔离远程搜索结果,避免把某个用户可见挂载
// 点的结果复用给可见范围不同的用户。
func remoteSearchCacheKey(query string, visibility service.MediaVisibility) string {
allowed := append([]string(nil), visibility.AllowedLibraryIDs...)
hidden := append([]string(nil), visibility.HiddenLibraryIDs...)
sort.Strings(allowed)
sort.Strings(hidden)
return fmt.Sprintf("media:obj:remote-search:%t|%s|%s|%s",
visibility.IncludeNSFW, strings.Join(allowed, ","), strings.Join(hidden, ","), query)
}
func writeMediaSearchResults[T any](c *gin.Context, items []T, page, size, limit int, paginated bool) {
if items == nil {
items = []T{}
@@ -0,0 +1,136 @@
package handler
import (
"encoding/json"
"fmt"
"net/http"
"net/http/httptest"
"strings"
"testing"
"time"
"github.com/gin-gonic/gin"
"github.com/glebarez/sqlite"
"go.uber.org/zap"
"gorm.io/gorm"
"github.com/truewhile/MeBox/internal/config"
"github.com/truewhile/MeBox/internal/middleware"
"github.com/truewhile/MeBox/internal/model"
"github.com/truewhile/MeBox/internal/repository"
"github.com/truewhile/MeBox/internal/service"
)
// TestListLibrariesRemotePreviewIsBounded 锁定远程库预览的总预算:
// 上游 Emby 卡住时,/api/libraries?with_preview=1 必须按时返回远程库条目
// (只是没有卡片),而不是跟着上游一起等十几秒。线上实测这个接口因此出现过
// 17.4 秒、30 秒(前端放弃)的响应,设置页也跟着整页空转。
func TestListLibrariesRemotePreviewIsBounded(t *testing.T) {
gin.SetMode(gin.TestMode)
oldBudget := remoteLibraryPreviewBudget
remoteLibraryPreviewBudget = 200 * time.Millisecond
t.Cleanup(func() { remoteLibraryPreviewBudget = oldBudget })
// Views / 用户列表正常返回,Preview(Items 查询)永远不返回:模拟上游半挂。
release := make(chan struct{})
upstream := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
w.Header().Set("Content-Type", "application/json")
path := r.URL.Path
switch {
case strings.HasSuffix(path, "/Views"):
_ = json.NewEncoder(w).Encode(map[string]any{
"Items": []map[string]any{{
"Id": "view-1", "Name": "远程动画", "CollectionType": "tvshows",
}},
})
return
case strings.HasSuffix(path, "/Users"):
// api_key 账号要先解析出远程用户 Id,否则请求路径会落到兜底 "0"。
_ = json.NewEncoder(w).Encode([]map[string]any{{"Id": "uid-1", "Name": "admin"}})
return
case r.Method == http.MethodPost:
_ = json.NewEncoder(w).Encode(map[string]any{
"AccessToken": "fake-token",
"User": map[string]any{"Id": "uid-1"},
})
return
}
select {
case <-release:
case <-r.Context().Done():
case <-time.After(30 * time.Second):
}
w.WriteHeader(http.StatusGatewayTimeout)
}))
defer upstream.Close()
defer close(release)
// 唯一库名 + cache=shared:预览协程会并发读库(账号解析)。
dsn := fmt.Sprintf("file:%s?mode=memory&cache=shared", strings.ReplaceAll(t.Name(), "/", "_"))
db, err := gorm.Open(sqlite.Open(dsn), &gorm.Config{})
if err != nil {
t.Fatal(err)
}
if err := db.AutoMigrate(&model.Library{}, &model.Media{}, &model.StrmAccount{}, &model.EmbyMount{}, &model.Setting{}, &model.User{}, &model.PlayProfile{}); err != nil {
t.Fatal(err)
}
repos := repository.New(db)
if err := db.Create(&model.User{Base: model.Base{ID: "admin-1"}, Username: "admin", Role: "admin"}).Error; err != nil {
t.Fatal(err)
}
rawConfig, _ := json.Marshal(map[string]string{"url": upstream.URL, "token": "fake-token"})
acct := model.StrmAccount{
Base: model.Base{ID: "acct-1"}, Name: "远程Emby", Provider: model.StrmProviderEmbyRemote,
Config: string(rawConfig), Enabled: true,
}
if err := repos.StrmAccount.Create(t.Context(), &acct); err != nil {
t.Fatal(err)
}
mount := model.EmbyMount{
Base: model.Base{ID: "mount-1"}, AccountID: acct.ID, RemoteViewID: "view-1",
RemoteViewName: "远程动画", CollectionType: "tvshows", Enabled: true,
}
if err := repos.EmbyMount.Create(t.Context(), &mount); err != nil {
t.Fatal(err)
}
svc := &service.Container{
Repo: repos,
Media: service.NewMediaService(&config.Config{}, zap.NewNop(), repos),
EmbyRemote: service.NewEmbyRemoteService(&config.Config{}, zap.NewNop(), repos, service.NewCryptoService("", zap.NewNop())),
Cache: service.NewRuntimeCacheService(&config.Config{}, zap.NewNop()),
}
w := httptest.NewRecorder()
c, _ := gin.CreateTestContext(w)
c.Set(middleware.CtxUserID, "admin-1")
c.Set(middleware.CtxUserRole, "admin")
c.Request = httptest.NewRequest(http.MethodGet, "/api/libraries?with_preview=1&include_hidden=1", nil)
start := time.Now()
listLibrariesHandler(svc)(c)
elapsed := time.Since(start)
if w.Code != http.StatusOK {
t.Fatalf("status=%d body=%s", w.Code, w.Body.String())
}
if elapsed > 2*time.Second {
t.Fatalf("request took %v with a hung upstream; the preview budget must bound it", elapsed)
}
var payload []webLibraryPayload
if err := json.Unmarshal(w.Body.Bytes(), &payload); err != nil {
t.Fatalf("decode response: %v (%s)", err, w.Body.String())
}
var remote *webLibraryPayload
for i := range payload {
if payload[i].IsRemoteEmby {
remote = &payload[i]
break
}
}
if remote == nil {
t.Fatalf("remote library missing from the bounded response: %#v", payload)
}
if len(remote.Cards) != 0 || remote.Total != 0 {
t.Fatalf("expected the preview to be skipped on timeout, got total=%d cards=%d", remote.Total, len(remote.Cards))
}
}
@@ -0,0 +1,153 @@
package handler
import (
"encoding/json"
"fmt"
"net/http"
"net/http/httptest"
"strings"
"sync"
"testing"
"github.com/gin-gonic/gin"
"github.com/glebarez/sqlite"
"go.uber.org/zap"
"gorm.io/gorm"
"github.com/truewhile/MeBox/internal/config"
"github.com/truewhile/MeBox/internal/middleware"
"github.com/truewhile/MeBox/internal/model"
"github.com/truewhile/MeBox/internal/repository"
"github.com/truewhile/MeBox/internal/service"
)
// TestSearchMediaRemoteSearchBoundedAndCached 防止远程搜索再次退化:
// 1. 每个挂载点的 Limit 必须是 remoteSearchItemLimit,而不是历史全库查询的 50000;
// 2. 同一关键词的重复搜索应命中缓存,不再打上游;
// 3. 指定本地媒体库时不应枚举远程挂载点。
func TestSearchMediaRemoteSearchBoundedAndCached(t *testing.T) {
gin.SetMode(gin.TestMode)
var mu sync.Mutex
searchCalls := 0
searchLimits := make([]string, 0, 4)
upstream := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
if term := r.URL.Query().Get("SearchTerm"); term != "" {
mu.Lock()
searchCalls++
searchLimits = append(searchLimits, r.URL.Query().Get("Limit"))
mu.Unlock()
}
w.Header().Set("Content-Type", "application/json")
_ = json.NewEncoder(w).Encode(map[string]any{
"TotalRecordCount": 1,
"Items": []map[string]any{{
"Id": "remote-1",
"Name": "远方剧集",
"Type": "Series",
"ProductionYear": 2024,
}},
})
}))
defer upstream.Close()
// 唯一库名 + cache=shared:远程搜索与本地查询会并发读库,
// 而 :memory: 下每个连接各自一个空库,并发连接会读到「no such table」。
dsn := fmt.Sprintf("file:%s?mode=memory&cache=shared", strings.ReplaceAll(t.Name(), "/", "_"))
db, err := gorm.Open(sqlite.Open(dsn), &gorm.Config{})
if err != nil {
t.Fatal(err)
}
if err := db.AutoMigrate(&model.Library{}, &model.Media{}, &model.StrmAccount{}, &model.EmbyMount{}, &model.Setting{}, &model.User{}, &model.PlayProfile{}); err != nil {
t.Fatal(err)
}
repos := repository.New(db)
if err := repos.DB.Create(&model.User{Base: model.Base{ID: "user-1"}, Username: "admin", Role: "admin"}).Error; err != nil {
t.Fatal(err)
}
lib := model.Library{Name: "本地电影", Path: "/media/movies", Type: "movie", Enabled: true}
if err := repos.Library.Create(t.Context(), &lib); err != nil {
t.Fatal(err)
}
if err := db.Create(&model.Media{
Base: model.Base{ID: "local-1"}, LibraryID: lib.ID, Title: "流浪地球", Path: "/media/movies/流浪地球.mkv",
}).Error; err != nil {
t.Fatal(err)
}
rawConfig, _ := json.Marshal(map[string]string{"url": upstream.URL, "token": "fake-token"})
acct := model.StrmAccount{
Base: model.Base{ID: "acct"}, Name: "远程Emby", Provider: model.StrmProviderEmbyRemote,
Config: string(rawConfig), Enabled: true,
}
if err := repos.StrmAccount.Create(t.Context(), &acct); err != nil {
t.Fatal(err)
}
mount := model.EmbyMount{
Base: model.Base{ID: "mount-1"}, AccountID: acct.ID, RemoteViewID: "view-1",
RemoteViewName: "远程剧集", CollectionType: "tvshows", Enabled: true,
}
if err := repos.EmbyMount.Create(t.Context(), &mount); err != nil {
t.Fatal(err)
}
svc := &service.Container{
Repo: repos,
Media: service.NewMediaService(&config.Config{}, zap.NewNop(), repos),
EmbyRemote: service.NewEmbyRemoteService(&config.Config{}, zap.NewNop(), repos, service.NewCryptoService("", zap.NewNop())),
Cache: service.NewRuntimeCacheService(&config.Config{}, zap.NewNop()),
}
search := func(params string) []service.MediaItem {
t.Helper()
w := httptest.NewRecorder()
c, _ := gin.CreateTestContext(w)
c.Set(middleware.CtxUserID, "user-1")
c.Set(middleware.CtxUserRole, "admin")
c.Request = httptest.NewRequest(http.MethodGet, "/api/media?"+params, nil)
searchMediaHandler(svc)(c)
if w.Code != http.StatusOK {
t.Fatalf("search status=%d, body=%s", w.Code, w.Body.String())
}
var response struct {
Items []service.MediaItem `json:"items"`
}
if err := json.Unmarshal(w.Body.Bytes(), &response); err != nil {
t.Fatal(err)
}
return response.Items
}
callCount := func() int {
mu.Lock()
defer mu.Unlock()
return searchCalls
}
if items := search("q=远方&page=1&page_size=48"); len(items) != 1 || items[0].Title != "远方剧集" {
t.Fatalf("expected remote result, got %#v", items)
}
if calls := callCount(); calls != 1 {
t.Fatalf("expected exactly one upstream search, got %d", calls)
}
mu.Lock()
limit := searchLimits[0]
mu.Unlock()
if limit != "100" {
t.Fatalf("remote search Limit=%q, want bounded 100 (50000 means a full-library query)", limit)
}
// 同一关键词再来一次(例如刷新/重复搜索)应复用缓存,不再请求上游。
if items := search("q=远方&page=1&page_size=48"); len(items) != 1 || items[0].Title != "远方剧集" {
t.Fatalf("cached search lost the remote result: %#v", items)
}
if calls := callCount(); calls != 1 {
t.Fatalf("expected cached remote search to skip upstream, got %d calls", calls)
}
// 指定本地媒体库时不应枚举远程挂载点。
if items := search("q=流浪&page=1&page_size=48&library_id=" + lib.ID); len(items) != 1 || items[0].ID != "local-1" {
t.Fatalf("expected local-only result, got %#v", items)
}
if calls := callCount(); calls != 1 {
t.Fatalf("local library filter must not hit upstream, got %d calls", calls)
}
}
+5 -1
View File
@@ -6,6 +6,7 @@ import (
"net/http"
"net/http/httptest"
"net/url"
"strings"
"testing"
"github.com/gin-gonic/gin"
@@ -35,7 +36,10 @@ func TestSearchMediaSeriesKeepsRemoteLibrariesSeparate(t *testing.T) {
}))
defer upstream.Close()
db, err := gorm.Open(sqlite.Open(":memory:"), &gorm.Config{})
// 唯一库名 + cache=shared:远程搜索与本地查询会并发读库,
// 而 :memory: 下每个连接各自一个空库,并发连接会读到「no such table」。
dsn := fmt.Sprintf("file:%s?mode=memory&cache=shared", strings.ReplaceAll(t.Name(), "/", "_"))
db, err := gorm.Open(sqlite.Open(dsn), &gorm.Config{})
if err != nil {
t.Fatal(err)
}
+4 -1
View File
@@ -559,7 +559,10 @@ func TestSearchMediaHandlerIncludesEmbyRemote(t *testing.T) {
}))
defer server.Close()
db, err := gorm.Open(sqlite.Open(":memory:"), &gorm.Config{})
// 唯一库名 + cache=shared:远程搜索与本地查询会并发读库,
// 而 :memory: 下每个连接各自一个空库,并发连接会读到「no such table」。
dsn := fmt.Sprintf("file:%s?mode=memory&cache=shared", strings.ReplaceAll(t.Name(), "/", "_"))
db, err := gorm.Open(sqlite.Open(dsn), &gorm.Config{})
if err != nil {
t.Fatal(err)
}
@@ -36,6 +36,9 @@ func registerAuthedLibraryRoutes(authed *gin.RouterGroup, svc *service.Container
authed.GET("/libraries", listLibrariesHandler(svc))
authed.POST("/libraries", middleware.AdminRequired(), createLibraryHandler(svc))
authed.GET("/libraries/:id", getLibraryHandler(svc))
// 服务端生成的媒体库封面(成员海报拼图),Web / App / Emby 共用同一份。
authed.GET("/libraries/:id/cover", libraryCoverHandler(svc))
authed.HEAD("/libraries/:id/cover", libraryCoverHandler(svc))
authed.PATCH("/libraries/:id", middleware.AdminRequired(), updateLibraryHandler(svc))
authed.PUT("/libraries/reorder", middleware.AdminRequired(), reorderLibrariesHandler(svc))
authed.DELETE("/libraries/:id", middleware.AdminRequired(), deleteLibraryHandler(svc))
+43 -3
View File
@@ -27,7 +27,15 @@ func hlsPlaylistHandler(svc *service.Container) gin.HandlerFunc {
if !enforceScopedPlaybackToken(c, m.ID) {
return
}
err = svc.Stream.ServeHLSPlaylist(c.Writer, c.Request, c.Param("id"))
scope, ok := localHLSPlaybackScope(c)
if !ok {
return
}
err = svc.Stream.ServeHLSPlaylist(c.Writer, c.Request, c.Param("id"), scope)
if errors.Is(err, service.ErrHLSGenerationSuperseded) {
c.JSON(http.StatusConflict, gin.H{"code": "hls_generation_superseded", "error": err.Error()})
return
}
if errors.Is(err, service.ErrMediaNotFound) {
c.JSON(http.StatusNotFound, gin.H{"error": "not found"})
return
@@ -62,7 +70,11 @@ func hlsSegmentHandler(svc *service.Container) gin.HandlerFunc {
if !enforceScopedPlaybackToken(c, m.ID) {
return
}
err = svc.Stream.ServeHLSSegment(c.Writer, c.Request, c.Param("id"), c.Param("seg"))
scope, ok := localHLSPlaybackScope(c)
if !ok {
return
}
err = svc.Stream.ServeHLSSegment(c.Writer, c.Request, c.Param("id"), c.Param("seg"), scope)
if err != nil {
c.JSON(http.StatusNotFound, gin.H{"error": err.Error()})
return
@@ -72,11 +84,39 @@ func hlsSegmentHandler(svc *service.Container) gin.HandlerFunc {
func stopTranscodeHandler(svc *service.Container) gin.HandlerFunc {
return func(c *gin.Context) {
svc.Transcoder.StopJob(c.Param("id"))
m, err := findMediaForPlaybackEndpoint(c, svc, c.Param("id"))
if err != nil || m == nil || !mediaVisibleForRequest(c, svc, m) {
c.JSON(http.StatusNotFound, gin.H{"error": "media not found"})
return
}
if !enforceScopedPlaybackToken(c, m.ID) {
return
}
scope, ok := localHLSPlaybackScope(c)
if !ok {
return
}
if svc.Transcoder != nil {
svc.Transcoder.StopPlayback(m.ID, scope)
}
c.Status(http.StatusNoContent)
}
}
func localHLSPlaybackScope(c *gin.Context) (service.HLSPlaybackScope, bool) {
userID := currentUserID(c)
if userID == "" {
c.JSON(http.StatusUnauthorized, gin.H{"error": "missing playback owner"})
return service.HLSPlaybackScope{}, false
}
scope, err := service.HLSPlaybackScopeForRequest(c.Request, userID)
if err != nil {
c.JSON(http.StatusBadRequest, gin.H{"error": err.Error()})
return service.HLSPlaybackScope{}, false
}
return scope, true
}
func imageProxyHandler(svc *service.Container) gin.HandlerFunc {
return func(c *gin.Context) {
raw := c.Query("url")
@@ -24,6 +24,14 @@ func (s *ApiConfigService) TestConnection(ctx context.Context, provider string)
switch provider {
case "tmdb":
return s.testTMDb(cfg)
case "bangumi":
return s.testBangumi(ctx, cfg)
case "thetvdb":
return s.testTheTVDB(ctx, cfg)
case "fanart":
return s.testFanart(ctx, cfg)
case "douban":
return s.testDouban(ctx, cfg)
case "openai":
return s.testOpenAI(cfg)
case "deepseek":
@@ -0,0 +1,191 @@
package service
import (
"context"
"encoding/json"
"errors"
"fmt"
"io"
"net/http"
"net/url"
"strings"
"time"
"github.com/truewhile/MeBox/internal/model"
)
// Bangumi's public metadata endpoints work without a token. If one was saved,
// also check /v0/me: public endpoints alone may accept an expired token.
func (s *ApiConfigService) testBangumi(ctx context.Context, cfg *model.APIConfig) (string, error) {
req, err := newMetadataProbeRequest(ctx, cfg, "https://api.bgm.tv", "/v0/subjects/253", http.MethodGet, nil)
if err != nil {
return "error", err
}
if token := strings.TrimSpace(cfg.APIKey); token != "" {
req.Header.Set("Authorization", "Bearer "+token)
identityReq := req.Clone(ctx)
identityURL := *req.URL
identityURL.Path = strings.TrimSuffix(identityURL.Path, "/subjects/253") + "/me"
identityReq.URL = &identityURL
var identity struct {
ID int `json:"id"`
}
if result, err := testMetadataProbeJSON(identityReq, "Bangumi", &identity); err != nil {
return result, err
}
if identity.ID <= 0 {
return "error", errors.New("Bangumi returned an invalid user response")
}
}
var subject struct {
ID int `json:"id"`
Name string `json:"name"`
}
if result, err := testMetadataProbeJSON(req, "Bangumi", &subject); err != nil {
return result, err
}
if subject.ID != 253 || strings.TrimSpace(subject.Name) == "" {
return "error", errors.New("Bangumi returned an invalid subject response")
}
return "success", nil
}
func (s *ApiConfigService) testTheTVDB(ctx context.Context, cfg *model.APIConfig) (string, error) {
key := strings.TrimSpace(cfg.APIKey)
if key == "" {
return "error", errors.New("TheTVDB API key is required")
}
body, err := json.Marshal(map[string]string{"apikey": key})
if err != nil {
return "error", err
}
req, err := newMetadataProbeRequest(ctx, cfg, "https://api4.thetvdb.com/v4", "/login", http.MethodPost, strings.NewReader(string(body)))
if err != nil {
return "error", err
}
req.Header.Set("Content-Type", "application/json")
var response struct {
Status string `json:"status"`
Data struct {
Token string `json:"token"`
} `json:"data"`
}
if result, err := testMetadataProbeJSON(req, "TheTVDB", &response); err != nil {
return result, err
}
if response.Status != "success" || strings.TrimSpace(response.Data.Token) == "" {
return "error", errors.New("TheTVDB login did not return a valid token")
}
return "success", nil
}
func (s *ApiConfigService) testFanart(ctx context.Context, cfg *model.APIConfig) (string, error) {
key := strings.TrimSpace(cfg.APIKey)
if key == "" {
return "error", errors.New("Fanart API key is required")
}
req, err := newMetadataProbeRequest(ctx, cfg, "https://webservice.fanart.tv/v3", "/movies/550", http.MethodGet, nil)
if err != nil {
return "error", err
}
query := req.URL.Query()
query.Set("api_key", key)
req.URL.RawQuery = query.Encode()
var response struct {
TMDbID json.Number `json:"tmdb_id"`
Name string `json:"name"`
}
if result, err := testMetadataProbeJSON(req, "Fanart", &response); err != nil {
return result, err
}
if response.TMDbID.String() != "550" || strings.TrimSpace(response.Name) == "" {
return "error", errors.New("Fanart returned an invalid movie response")
}
return "success", nil
}
func (s *ApiConfigService) testDouban(ctx context.Context, cfg *model.APIConfig) (string, error) {
req, err := newMetadataProbeRequest(ctx, cfg, "https://movie.douban.com", "/j/subject_suggest", http.MethodGet, nil)
if err != nil {
return "error", err
}
query := req.URL.Query()
query.Set("q", "Inception")
req.URL.RawQuery = query.Encode()
req.Header.Set("User-Agent", userAgents[0])
req.Header.Set("Referer", "https://movie.douban.com/")
req.Header.Set("Accept-Language", "zh-CN,zh;q=0.9,en;q=0.8")
if cookie := strings.TrimSpace(cfg.APIKey); cookie != "" {
req.Header.Set("Cookie", cookie)
}
var suggestions []struct {
ID string `json:"id"`
Title string `json:"title"`
}
if result, err := testMetadataProbeJSON(req, "Douban", &suggestions); err != nil {
return result, err
}
// An empty array is a valid search result; null or an error object is not.
if suggestions == nil {
return "error", errors.New("Douban returned an invalid search response (login or verification may be required)")
}
for _, suggestion := range suggestions {
if suggestion.ID == "" || strings.TrimSpace(suggestion.Title) == "" {
return "error", errors.New("Douban returned an invalid search suggestion")
}
}
return "success", nil
}
func newMetadataProbeRequest(ctx context.Context, cfg *model.APIConfig, defaultBase, path, method string, body io.Reader) (*http.Request, error) {
base := strings.TrimRight(strings.TrimSpace(cfg.BaseURL), "/")
if base == "" {
base = defaultBase
}
req, err := http.NewRequestWithContext(ctx, method, base+path, body)
if err != nil {
// The saved base URL may contain credentials. Never echo it to the UI.
return nil, fmt.Errorf("%s: invalid base URL", cfg.Provider)
}
req.Header.Set("User-Agent", "MeBox/0.1 (https://github.com/truewhile/MeBox)")
req.Header.Set("Accept", "application/json")
return req, nil
}
// Decode a bounded, complete JSON document rather than accepting any HTTP 200
// (reverse proxies and anti-bot/login pages often return 200 as well).
func testMetadataProbeJSON(req *http.Request, provider string, out any) (string, error) {
client := NewExternalHTTPClient(10 * time.Second)
defer client.CloseIdleConnections()
resp, err := client.Do(req)
if err != nil {
// url.Error includes the full request URL, including Fanart's API key.
var urlErr *url.Error
if errors.As(err, &urlErr) {
err = urlErr.Err
}
return "error", fmt.Errorf("%s connection failed: %w", provider, err)
}
defer resp.Body.Close()
if resp.StatusCode == http.StatusUnauthorized {
return "invalid", fmt.Errorf("%s authentication failed (HTTP 401): check configured credentials", provider)
}
if resp.StatusCode == http.StatusForbidden {
return "error", fmt.Errorf("%s access denied (HTTP 403): check credentials, permissions or anti-bot verification", provider)
}
if resp.StatusCode != http.StatusOK {
return "error", fmt.Errorf("%s API returned status %d", provider, resp.StatusCode)
}
body, err := io.ReadAll(io.LimitReader(resp.Body, (1<<20)+1))
if err != nil {
return "error", fmt.Errorf("%s response read failed: %w", provider, err)
}
if len(body) > 1<<20 {
return "error", fmt.Errorf("%s response exceeds 1 MiB", provider)
}
if err := json.Unmarshal(body, out); err != nil {
// Do not include the response body: it may echo credentials.
return "error", fmt.Errorf("%s returned invalid JSON (login or verification page may have intercepted the request)", provider)
}
return "success", nil
}
@@ -0,0 +1,252 @@
package service
import (
"context"
"encoding/json"
"errors"
"io"
"net/http"
"net/http/httptest"
"strings"
"testing"
"time"
"go.uber.org/zap"
"github.com/truewhile/MeBox/internal/config"
"github.com/truewhile/MeBox/internal/repository"
)
type metadataProbeFixture struct {
provider string
path string
key string
body string
}
var metadataProbeFixtures = []metadataProbeFixture{
{"bangumi", "/v0/subjects/253", "", `{"id":253,"name":"カウボーイビバップ","type":2}`},
{"thetvdb", "/login", "tvdb-test-secret", `{"status":"success","data":{"token":"test-jwt"}}`},
{"fanart", "/movies/550", "fanart+test&secret", `{"name":"Fight Club","tmdb_id":"550","imdb_id":"tt0137523","movieposter":[]}`},
{"douban", "/j/subject_suggest", "bid=test-cookie", `[{"id":"3541415","title":"盗梦空间","year":"2010","type":"movie"}]`},
}
func newMetadataConnectionTestService(t *testing.T) *ApiConfigService {
t.Helper()
log := zap.NewNop()
return NewApiConfigService(&config.Config{}, log, repository.New(newServiceTestDB(t)), NewCryptoService("metadata-test-secret", log))
}
func saveMetadataProbeConfig(t *testing.T, svc *ApiConfigService, provider, baseURL, key string) {
t.Helper()
if _, err := svc.Upsert(t.Context(), provider, key, baseURL, "", true); err != nil {
t.Fatal(err)
}
}
// Removing a dispatch branch, ignoring the saved URL/key, or using the wrong
// endpoint must fail this integration test (real database and HTTP server).
func TestMetadataConnectionUsesSavedConfig(t *testing.T) {
svc := newMetadataConnectionTestService(t)
for _, fixture := range metadataProbeFixtures {
t.Run(fixture.provider, func(t *testing.T) {
calls := 0
server := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
calls++
if r.URL.Path != "/proxy"+fixture.path {
t.Errorf("request path = %q", r.URL.Path)
}
if r.Header.Get("User-Agent") == "" {
t.Error("missing User-Agent")
}
if fixture.provider == "thetvdb" {
var body map[string]string
if err := json.NewDecoder(r.Body).Decode(&body); err != nil {
t.Error(err)
}
if r.Method != http.MethodPost || body["apikey"] != fixture.key || r.Header.Get("Content-Type") != "application/json" {
t.Error("TheTVDB login must POST the saved, decrypted API key as JSON")
}
} else if r.Method != http.MethodGet {
t.Errorf("method = %s, want GET", r.Method)
}
if fixture.provider == "fanart" && r.URL.Query().Get("api_key") != fixture.key {
t.Error("Fanart API key was not escaped/passed correctly")
}
if fixture.provider == "douban" {
if r.Header.Get("Cookie") != fixture.key || r.Header.Get("Referer") != "https://movie.douban.com/" || r.URL.Query().Get("q") == "" {
t.Error("Douban search must carry the configured cookie, referer and query")
}
}
w.Header().Set("Content-Type", "application/json")
_, _ = w.Write([]byte(fixture.body))
}))
defer server.Close()
saveMetadataProbeConfig(t, svc, fixture.provider, " "+server.URL+"/proxy/ ", fixture.key)
result, err := svc.TestConnection(t.Context(), fixture.provider)
if err != nil || result != "success" {
t.Fatalf("TestConnection = %q, %v, want success", result, err)
}
if calls != 1 {
t.Errorf("HTTP calls = %d, want 1", calls)
}
})
}
}
func TestMetadataConnectionRejectsBadResponses(t *testing.T) {
svc := newMetadataConnectionTestService(t)
for _, fixture := range metadataProbeFixtures {
for _, response := range []struct {
name string
status int
body string
result string
}{
{"unauthorized", 401, `{"error":"denied"}`, "invalid"},
{"forbidden", 403, `<html>Access denied</html>`, "error"},
{"rate limited", 429, `{}`, "error"},
{"upstream error", 503, `{}`, "error"},
{"challenge page", 200, `<html>Please log in or complete verification</html>`, "error"},
{"empty body", 200, ``, "error"},
{"wrong schema", 200, `{"error":"intercepted"}`, "error"},
{"null", 200, `null`, "error"},
{"trailing garbage", 200, fixture.body + `<html>blocked</html>`, "error"},
} {
t.Run(fixture.provider+"/"+response.name, func(t *testing.T) {
server := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
w.WriteHeader(response.status)
_, _ = w.Write([]byte(response.body))
}))
defer server.Close()
saveMetadataProbeConfig(t, svc, fixture.provider, server.URL, fixture.key)
result, err := svc.TestConnection(t.Context(), fixture.provider)
if err == nil || result != response.result {
t.Fatalf("TestConnection = %q, %v, want %q and error", result, err, response.result)
}
if fixture.key != "" && strings.Contains(err.Error(), fixture.key) {
t.Error("error exposes configured credentials")
}
})
}
}
}
func TestMetadataConnectionRequiresAPIKeys(t *testing.T) {
svc := newMetadataConnectionTestService(t)
for _, provider := range []string{"thetvdb", "fanart"} {
t.Run(provider, func(t *testing.T) {
server := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
t.Error("must reject missing key before making a request")
}))
defer server.Close()
saveMetadataProbeConfig(t, svc, provider, server.URL, " ")
result, err := svc.TestConnection(t.Context(), provider)
if result != "error" || err == nil || !strings.Contains(err.Error(), "API key") {
t.Fatalf("TestConnection = %q, %v, want missing API key error", result, err)
}
})
}
}
func TestMetadataConnectionBangumiValidatesConfiguredToken(t *testing.T) {
svc := newMetadataConnectionTestService(t)
for _, valid := range []bool{true, false} {
name := "valid"
if !valid {
name = "invalid"
}
t.Run(name, func(t *testing.T) {
meCalls, subjectCalls := 0, 0
server := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
if r.Header.Get("Authorization") != "Bearer bangumi-test-token" {
t.Error("missing saved Bangumi bearer token")
}
switch r.URL.Path {
case "/v0/me":
meCalls++
if !valid {
w.WriteHeader(http.StatusUnauthorized)
return
}
_, _ = w.Write([]byte(`{"id":42,"username":"test-user"}`))
case "/v0/subjects/253":
subjectCalls++
_, _ = w.Write([]byte(`{"id":253,"name":"カウボーイビバップ"}`))
default:
t.Errorf("unexpected path: %s", r.URL.Path)
w.WriteHeader(http.StatusNotFound)
}
}))
defer server.Close()
saveMetadataProbeConfig(t, svc, "bangumi", server.URL, "bangumi-test-token")
result, err := svc.TestConnection(t.Context(), "bangumi")
if valid && (result != "success" || err != nil || subjectCalls != 1) {
t.Fatalf("valid token: result=%q err=%v subjectCalls=%d", result, err, subjectCalls)
}
if !valid && (result != "invalid" || err == nil || subjectCalls != 0) {
t.Fatalf("invalid token: result=%q err=%v subjectCalls=%d", result, err, subjectCalls)
}
if meCalls != 1 {
t.Errorf("identity requests = %d, want 1", meCalls)
}
})
}
}
func TestMetadataConnectionDoubanDoesNotRequireCookie(t *testing.T) {
svc := newMetadataConnectionTestService(t)
server := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
if r.Header.Get("Cookie") != "" {
t.Error("unexpected Cookie for public search")
}
_, _ = w.Write([]byte(`[]`))
}))
defer server.Close()
saveMetadataProbeConfig(t, svc, "douban", server.URL, "")
result, err := svc.TestConnection(t.Context(), "douban")
if result != "success" || err != nil {
t.Fatalf("public search with empty results = %q, %v", result, err)
}
}
func TestMetadataConnectionHonorsContext(t *testing.T) {
svc := newMetadataConnectionTestService(t)
for _, fixture := range metadataProbeFixtures {
t.Run(fixture.provider, func(t *testing.T) {
started := make(chan struct{})
server := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
_, _ = io.Copy(io.Discard, r.Body)
close(started)
<-r.Context().Done()
}))
defer server.Close()
saveMetadataProbeConfig(t, svc, fixture.provider, server.URL, fixture.key)
ctx, cancel := context.WithTimeout(t.Context(), 2*time.Second)
defer cancel()
go func() {
select {
case <-started:
cancel()
case <-ctx.Done():
}
}()
result, err := svc.TestConnection(ctx, fixture.provider)
if result != "error" || !errors.Is(err, context.Canceled) {
t.Fatalf("cancelled request = %q, %v, want context.Canceled", result, err)
}
})
}
}
func TestMetadataConnectionTransportErrorsHideKeys(t *testing.T) {
svc := newMetadataConnectionTestService(t)
server := httptest.NewServer(http.NotFoundHandler())
server.Close()
key := "sensitive-fanart-key"
saveMetadataProbeConfig(t, svc, "fanart", server.URL, key)
result, err := svc.TestConnection(t.Context(), "fanart")
if result != "error" || err == nil || strings.Contains(err.Error(), key) {
t.Fatalf("transport failure = %q, %v, want error without credentials", result, err)
}
}
+22 -34
View File
@@ -74,22 +74,20 @@ type ImageCacheScope struct {
DropUnknownResizeVariants bool
}
// ImageCachePool 描述一个图片缓存池:范围 + 自身配额 + 保留时长。
// ImageCachePool 描述一个图片缓存池:范围 + 自身配额。
//
// 分池的意义在于两类缓存的“可再生成本”完全不同:原图只是生成缩略图的原料,
// 丢了可以重新回源;派生缩略图是客户端热路径真正读取的成品,重新生成代价高。
// 因此原图配小配额、短保留,总量超限时也优先淘汰原图。
// 因此原图配小配额,总量超限时也优先淘汰原图。
//
// 淘汰只看占用(配额头寸),不按文件年龄删除:缩略图生成成本高,只要磁盘还
// 放得下就该留着,让再次浏览直接命中缓存。
type ImageCachePool struct {
Name string
Scopes []ImageCacheScope
MaxBytes int64 // 0 = 不单独限制
MaxAge time.Duration // 0 = 不按时间淘汰
MaxBytes int64 // 0 = 不单独限制
}
// imageFailMarkerMaxAge 是失败标记文件的保留时长:标记只用于观测/重试,
// 过期即视为噪音清掉(不会因为标记残留而阻止重新抓取)。
const imageFailMarkerMaxAge = 24 * time.Hour
// tempImageCacheFile 报告文件名是否是下载/缩放过程中的临时文件。它们正在被
// 并发写入,清理时必须跳过。
func tempImageCacheFile(name string) bool {
@@ -182,12 +180,14 @@ func removeImageCacheEntry(entry imageCacheFileEntry, result *PruneImageCacheRes
_ = os.Remove(entry.path + ".fail")
}
// PruneImageCachePools 按池清理图片缓存:
// 1. 删除超过池 MaxAge 的文件(原图按保留时长淘汰);
// 2. 池自身超过 MaxBytes 时按 mtime 淘汰到 80%(留水位,避免连续写入即触发);
// 3. 全部池合计超过 totalBytes 时,仍按“先原图、后派生”的顺序淘汰最旧文件,
// PruneImageCachePools 按池清理图片缓存,只做占用淘汰:
// 1. 池自身超过 MaxBytes 时按 mtime 淘汰到 80%(留水位,避免连续写入即触发);
// 2. 全部池合计超过 totalBytes 时,仍按“先原图、后派生”的顺序淘汰最旧文件,
// 使总量上限始终是硬保证,同时让客户端热路径的缩略图活得更久。
//
// 不按文件年龄删除:过期即删会让一张几天前看过的海报重新解码(2 核机器上
// 每张约 0.2 秒),代价远高于占着那几十 KB 磁盘。
//
// 传入的池顺序即总量超限时的淘汰优先级(排在前面的先被淘汰)。
func PruneImageCachePools(pools []ImageCachePool, totalBytes int64) (PruneImageCacheResult, error) {
var result PruneImageCacheResult
@@ -207,16 +207,18 @@ func PruneImageCachePools(pools []ImageCachePool, totalBytes int64) (PruneImageC
}
result.RemainingBytes = result.TotalBytesBefore
// 过期的失败标记直接清掉:它们不计容量,只用于观测与重试。
cutoffMarkers := time.Now().Add(-imageFailMarkerMaxAge)
// 失败标记不计入容量:只清掉对应缓存文件已经不存在的孤儿标记(它们只是
// 观测/重试用的噪音,留着没有任何作用)。
for _, marker := range failMarkers {
if marker.modTime.Before(cutoffMarkers) {
_ = os.Remove(marker.path)
base := strings.TrimSuffix(marker.path, ".fail")
if _, err := os.Stat(base); err == nil {
continue
}
_ = os.Remove(marker.path)
}
now := time.Now()
deleted := make(map[string]bool, len(entries))
deleteEntry := func(entry imageCacheFileEntry) {
if deleted[entry.path] {
return
@@ -225,20 +227,7 @@ func PruneImageCachePools(pools []ImageCachePool, totalBytes int64) (PruneImageC
removeImageCacheEntry(entry, &result)
}
// 1. 按池保留时长淘汰(原图池)。
for i := range pools {
if pools[i].MaxAge <= 0 {
continue
}
cutoff := now.Add(-pools[i].MaxAge)
for _, entry := range entries {
if entry.pool == i && entry.modTime.Before(cutoff) {
deleteEntry(entry)
}
}
}
// 2. 池自身配额。
// 1. 池自身配额。
for i := range pools {
maxBytes := pools[i].MaxBytes
if maxBytes <= 0 {
@@ -263,7 +252,7 @@ func PruneImageCachePools(pools []ImageCachePool, totalBytes int64) (PruneImageC
}
}
// 3. 总量硬上限:跨池按淘汰优先级(池顺序)再按 mtime 淘汰。
// 2. 总量硬上限:跨池按淘汰优先级(池顺序)再按 mtime 淘汰。
if totalBytes > 0 && result.RemainingBytes > totalBytes {
target := totalBytes * 80 / 100
for _, entry := range evictionOrder(entries, deleted) {
@@ -357,11 +346,10 @@ func PruneImageCache(imagesDir string, maxSizeBytes int64) (PruneImageCacheResul
// 第一个池是原图:历史版本平铺在 images/ 下的旧缓存也归入此池,升级后会被
// 逐步淘汰;第二个池是派生成品(本地缩放结果 + 挂载 Emby 按尺寸返回的成品)。
// 池顺序即总量超限时的淘汰优先级。
func ImageCachePools(imagesDir string, originalsMaxBytes int64, originalsMaxAge time.Duration) []ImageCachePool {
func ImageCachePools(imagesDir string, originalsMaxBytes int64) []ImageCachePool {
originals := ImageCachePool{
Name: "originals",
MaxBytes: originalsMaxBytes,
MaxAge: originalsMaxAge,
Scopes: []ImageCacheScope{
{Root: imagesDir, Recursive: false}, // 旧版平铺布局
{Root: filepath.Join(imagesDir, imageOriginalCacheSubdir), Recursive: true},
+44 -44
View File
@@ -9,7 +9,7 @@ import (
"time"
)
// writePoolFile ?????????????????????????
// writePoolFile ??????????????? mtime ???????
func writePoolFile(t *testing.T, path string, size int, modTime time.Time) {
t.Helper()
if err := os.MkdirAll(filepath.Dir(path), 0o750); err != nil {
@@ -39,42 +39,38 @@ func mustExist(t *testing.T, path string) {
}
}
// currentVariantPath ????????????????.<??x<??q<??>.img??
// ??????????????????????????
// currentVariantPath ???????????????????
// <64 ????>.<?>x<?>q<??>.img
func currentVariantPath(dir, keyChar string, width, height, quality int) string {
key := strings.Repeat(keyChar, sha256HexLength)
return filepath.Join(dir, fmt.Sprintf("%s.%dx%dq%d.img", key, width, height, quality))
}
// TestPruneImageCachePoolsOriginalTTL ????????????????????
// ?? TTL ??????????????????????
func TestPruneImageCachePoolsOriginalTTL(t *testing.T) {
// TestPruneImageCachePoolsKeepsOldFilesUnderQuota ????????????
// ???????????????????????? / TTL ????
func TestPruneImageCachePoolsKeepsOldFilesUnderQuota(t *testing.T) {
imagesDir := t.TempDir()
now := time.Now()
stale := filepath.Join(imagesDir, imageOriginalCacheSubdir, "stale-original")
fresh := filepath.Join(imagesDir, imageOriginalCacheSubdir, "fresh-original")
thumb := currentVariantPath(filepath.Join(imagesDir, imageResizeCacheSubdir), "b", 480, 600, 80)
writePoolFile(t, stale, 400, now.Add(-8*24*time.Hour))
writePoolFile(t, fresh, 400, now.Add(-1*time.Hour))
// ??????????TTL ????
writePoolFile(t, thumb, 400, now.Add(-8*24*time.Hour))
oldOriginal := filepath.Join(imagesDir, imageOriginalCacheSubdir, "old-original")
oldThumb := currentVariantPath(filepath.Join(imagesDir, imageResizeCacheSubdir), "b", 480, 600, 80)
writePoolFile(t, oldOriginal, 400, now.Add(-90*24*time.Hour))
writePoolFile(t, oldThumb, 400, now.Add(-90*24*time.Hour))
pools := ImageCachePools(imagesDir, 0, 7*24*time.Hour)
pools := ImageCachePools(imagesDir, 0)
res, err := PruneImageCachePools(pools, 100*1024*1024)
if err != nil {
t.Fatalf("prune failed: %v", err)
}
if res.DeletedFiles != 1 {
t.Fatalf("deleted files = %d, want 1", res.DeletedFiles)
if res.DeletedFiles != 0 {
t.Fatalf("deleted files = %d, want 0 (age must not trigger deletion)", res.DeletedFiles)
}
mustNotExist(t, stale)
mustExist(t, fresh)
mustExist(t, thumb)
mustExist(t, oldOriginal)
mustExist(t, oldThumb)
}
// TestPruneImageCachePoolsEvictsOriginalsBeforeDerived ????????????
// ????????????????????
// ????????????????
func TestPruneImageCachePoolsEvictsOriginalsBeforeDerived(t *testing.T) {
imagesDir := t.TempDir()
now := time.Now()
@@ -82,13 +78,13 @@ func TestPruneImageCachePoolsEvictsOriginalsBeforeDerived(t *testing.T) {
original := filepath.Join(imagesDir, imageOriginalCacheSubdir, "original")
rendition := filepath.Join(imagesDir, imageRenditionCacheSubdir, "rendition")
thumb := currentVariantPath(filepath.Join(imagesDir, imageResizeCacheSubdir), "c", 480, 600, 80)
// ?????????????????
// ?????????????????????????
writePoolFile(t, original, 1024*1024, now.Add(-72*time.Hour))
writePoolFile(t, rendition, 1024*1024, now.Add(-2*time.Hour))
writePoolFile(t, thumb, 1024*1024, now.Add(-1*time.Hour))
pools := ImageCachePools(imagesDir, 0, 0)
// ?? 3MB????2.4MB -> ????? 1.92MB??
pools := ImageCachePools(imagesDir, 0)
// ?? 3MB??? 2.4MB -> ??? 1.92MB ???
res, err := PruneImageCachePools(pools, 2*1024*1024+400*1024)
if err != nil {
t.Fatalf("prune failed: %v", err)
@@ -104,8 +100,8 @@ func TestPruneImageCachePoolsEvictsOriginalsBeforeDerived(t *testing.T) {
}
}
// TestPruneImageCachePoolsOriginalQuotaKeepsDerived ????????????
// ??????????????
// TestPruneImageCachePoolsOriginalQuotaKeepsDerived ???????????
// ????????????
func TestPruneImageCachePoolsOriginalQuotaKeepsDerived(t *testing.T) {
imagesDir := t.TempDir()
now := time.Now()
@@ -117,8 +113,8 @@ func TestPruneImageCachePoolsOriginalQuotaKeepsDerived(t *testing.T) {
writePoolFile(t, newOriginal, 800, now.Add(-1*time.Hour))
writePoolFile(t, thumb, 800, now.Add(-2*time.Hour))
// ???? 1000 ??????????1600 -> ????800????? 1 ????
pools := ImageCachePools(imagesDir, 1000, 0)
// ???? 1000??? 1600 ?? -> ??? 800??? 1 ???????
pools := ImageCachePools(imagesDir, 1000)
res, err := PruneImageCachePools(pools, 0)
if err != nil {
t.Fatalf("prune failed: %v", err)
@@ -131,24 +127,26 @@ func TestPruneImageCachePoolsOriginalQuotaKeepsDerived(t *testing.T) {
mustExist(t, thumb)
}
// TestPruneImageCachePoolsCleansFailMarkers ????????????????
// ????????????????
// TestPruneImageCachePoolsCleansFailMarkers ???????????????
// ???????????????????????????????????
func TestPruneImageCachePoolsCleansFailMarkers(t *testing.T) {
imagesDir := t.TempDir()
now := time.Now()
entry := filepath.Join(imagesDir, imageOriginalCacheSubdir, "entry")
entryMarker := entry + ".fail"
kept := filepath.Join(imagesDir, imageOriginalCacheSubdir, "kept")
keptMarker := kept + ".fail"
orphanMarker := filepath.Join(imagesDir, imageOriginalCacheSubdir, "orphan.fail")
freshMarker := filepath.Join(imagesDir, imageOriginalCacheSubdir, "fresh.fail")
writePoolFile(t, entry, 900, now.Add(-10*time.Hour))
writePoolFile(t, entryMarker, 35, now.Add(-10*time.Hour))
writePoolFile(t, entry, 900, now.Add(-2*time.Hour))
writePoolFile(t, entryMarker, 35, now.Add(-2*time.Hour))
writePoolFile(t, kept, 300, now.Add(-1*time.Hour))
writePoolFile(t, keptMarker, 35, now.Add(-1*time.Hour))
writePoolFile(t, orphanMarker, 35, now.Add(-48*time.Hour))
writePoolFile(t, freshMarker, 35, now.Add(-1*time.Minute))
// ???? 500????900 ?????????????????
pools := ImageCachePools(imagesDir, 500, 0)
// ???? 500??? 900+300=1200 ???? mtime ????? entry ? 400?
pools := ImageCachePools(imagesDir, 500)
res, err := PruneImageCachePools(pools, 0)
if err != nil {
t.Fatalf("prune failed: %v", err)
@@ -158,26 +156,27 @@ func TestPruneImageCachePoolsCleansFailMarkers(t *testing.T) {
}
mustNotExist(t, entry)
mustNotExist(t, entryMarker)
mustExist(t, kept)
mustExist(t, keptMarker)
mustNotExist(t, orphanMarker)
mustExist(t, freshMarker)
}
// TestPruneImageCachePoolsSkipsTemporaryFiles ????????????????
// TestPruneImageCachePoolsSkipsTemporaryFiles ??????????????
// ?????????????????
func TestPruneImageCachePoolsSkipsTemporaryFiles(t *testing.T) {
imagesDir := t.TempDir()
tmp := filepath.Join(imagesDir, imageOriginalCacheSubdir, "img-1234.tmp")
writePoolFile(t, tmp, 4096, time.Now().Add(-48*time.Hour))
pools := ImageCachePools(imagesDir, 1, 0)
pools := ImageCachePools(imagesDir, 1)
if _, err := PruneImageCachePools(pools, 1); err != nil {
t.Fatalf("prune failed: %v", err)
}
mustExist(t, tmp)
}
// TestPruneImageCachePoolsDropsLegacyResizeVariants ????????.img????
// ?????????????????????????????????????
// TestPruneImageCachePoolsDropsLegacyResizeVariants ?????? .img ??
// ???????????????????????????
func TestPruneImageCachePoolsDropsLegacyResizeVariants(t *testing.T) {
imagesDir := t.TempDir()
resized := filepath.Join(imagesDir, imageResizeCacheSubdir)
@@ -188,7 +187,7 @@ func TestPruneImageCachePoolsDropsLegacyResizeVariants(t *testing.T) {
writePoolFile(t, current, 400, time.Now())
writePoolFile(t, other, 400, time.Now())
pools := ImageCachePools(imagesDir, 0, 0)
pools := ImageCachePools(imagesDir, 0)
res, err := PruneImageCachePools(pools, 0)
if err != nil {
t.Fatalf("prune failed: %v", err)
@@ -201,8 +200,8 @@ func TestPruneImageCachePoolsDropsLegacyResizeVariants(t *testing.T) {
}
}
// TestPruneImageCachePoolsCountsLegacyFlatLayout ??????images/ ????
// ?????????????????
// TestPruneImageCachePoolsCountsLegacyFlatLayout ???????images/ ???
// ???????????????????????
func TestPruneImageCachePoolsCountsLegacyFlatLayout(t *testing.T) {
imagesDir := t.TempDir()
legacy := filepath.Join(imagesDir, "legacy-flat-cache")
@@ -210,7 +209,8 @@ func TestPruneImageCachePoolsCountsLegacyFlatLayout(t *testing.T) {
writePoolFile(t, legacy, 900, time.Now().Add(-30*24*time.Hour))
writePoolFile(t, thumb, 900, time.Now().Add(-30*24*time.Hour))
pools := ImageCachePools(imagesDir, 0, 24*time.Hour)
// ???? 500??????????????????????????
pools := ImageCachePools(imagesDir, 500)
res, err := PruneImageCachePools(pools, 0)
if err != nil {
t.Fatalf("prune failed: %v", err)
@@ -0,0 +1,33 @@
package service
import (
"context"
"io"
"net/http"
"net/http/httptest"
"strings"
"testing"
"time"
)
func TestCloud115HLSChildRejectsMismatchedSessionMediaBeforeFetch(t *testing.T) {
fetched := false
proxy := &Cloud115HLSProxy{sessions: map[string]*cloud115HLSSession{}}
proxy.client = &http.Client{Transport: roundTripFunc(func(req *http.Request) (*http.Response, error) {
fetched = true
return &http.Response{StatusCode: http.StatusOK, Header: http.Header{"Content-Type": []string{"video/mp2t"}}, Body: io.NopCloser(strings.NewReader("private-segment")), Request: req}, nil
})}
proxy.sessions["session-b"] = &cloud115HLSSession{
ID: "session-b", MediaID: "media-b", ExpiresAt: time.Now().Add(time.Hour),
entries: map[string]string{"segment": "https://cpats01.115.com/seg.ts"},
}
for _, query := range []string{"?media_id=media-a", ""} {
fetched = false
req := httptest.NewRequest(http.MethodGet, "/api/cloud115/hls/session-b/segment"+query, nil)
rec := httptest.NewRecorder()
err := proxy.ServeChild(context.Background(), rec, req, "session-b", "segment")
if err == nil || fetched {
t.Errorf("query %q must reject the wrong/missing media before fetch; err=%v fetched=%v", query, err, fetched)
}
}
}
+15 -2
View File
@@ -20,8 +20,9 @@ import (
)
var (
ErrCloud115HLSSessionNotFound = errors.New("115 hls session not found")
ErrCloud115HLSUpstreamExpired = errors.New("115 hls upstream expired")
ErrCloud115HLSSessionNotFound = errors.New("115 hls session not found")
ErrCloud115HLSUpstreamExpired = errors.New("115 hls upstream expired")
ErrCloud115HLSSessionMediaMismatch = errors.New("115 hls session media mismatch")
)
const (
@@ -131,6 +132,9 @@ func (p *Cloud115HLSProxy) ServeChild(ctx context.Context, w http.ResponseWriter
if !ok {
return ErrCloud115HLSSessionNotFound
}
if r == nil || strings.TrimSpace(r.URL.Query().Get("media_id")) != session.MediaID {
return ErrCloud115HLSSessionMediaMismatch
}
resp, err := p.fetchUpstream(ctx, upstream, r.Header.Get("Range"))
if err != nil {
return err
@@ -186,6 +190,15 @@ func (p *Cloud115HLSProxy) ServeChild(ctx context.Context, w http.ResponseWriter
return err
}
// SessionMediaID resolves the server-owned media identity before handler authorization.
func (p *Cloud115HLSProxy) SessionMediaID(sessionID, key string) (string, error) {
session, _, ok := p.lookup(sessionID, key)
if !ok {
return "", ErrCloud115HLSSessionNotFound
}
return session.MediaID, nil
}
func (p *Cloud115HLSProxy) newSession(mediaID string, definition int) *cloud115HLSSession {
now := time.Now()
return &cloud115HLSSession{
+24 -6
View File
@@ -104,11 +104,13 @@ func (s *Cloud115PlaybackService) HLSProxy() *Cloud115HLSProxy {
return s.proxy
}
// PlaybackInfo 返回媒体在 direct/115-cloud/local 三种模式下的能力描述。
//
// requestedDefinition 为空或 0 时默认选择 1080P;该档位不可用时返回
// transcoding 状态(并只触发一次加速转码,避免轮询风暴)。
// PlaybackInfo only reads direct/115-cloud/local playback capabilities.
// Cloud work is queued exclusively by StartTranscode, never by polling.
func (s *Cloud115PlaybackService) PlaybackInfo(ctx context.Context, mediaID string, requestedDefinition int) (*PlaybackInfo, error) {
return s.playbackInfo(ctx, mediaID, requestedDefinition, false)
}
func (s *Cloud115PlaybackService) playbackInfo(ctx context.Context, mediaID string, requestedDefinition int, startTranscode bool) (*PlaybackInfo, error) {
if s == nil || s.repo == nil || s.repo.Media == nil {
return nil, ErrMediaNotFound
}
@@ -179,13 +181,26 @@ func (s *Cloud115PlaybackService) PlaybackInfo(ctx context.Context, mediaID stri
return info, nil
}
info.Transcode = s.ensureCloudTranscode(ctx, account.ID, pickCode, selected, client)
if startTranscode {
info.Transcode = s.ensureCloudTranscode(ctx, account.ID, pickCode, selected, client)
} else {
info.Transcode = Cloud115TranscodeState{
State: "idle", Definition: selected.ID, Message: "该清晰度尚未就绪,可请求 115 云端转码", RetryAfterSec: 5,
}
s.mu.Lock()
if last, ok := s.pushState[account.ID+":"+pickCode]; ok {
info.Transcode.State = "transcoding"
info.Transcode.Message = last.message
info.Transcode.StartedAt = last.at.Unix()
}
s.mu.Unlock()
}
return info, nil
}
// StartTranscode 显式触发某个清晰度的云端转码,返回最新状态。
func (s *Cloud115PlaybackService) StartTranscode(ctx context.Context, mediaID string, definition int) (*PlaybackInfo, error) {
return s.PlaybackInfo(ctx, mediaID, definition)
return s.playbackInfo(ctx, mediaID, definition, true)
}
// ResolveCloud115URL 返回指定清晰度的 115 云端 m3u8 地址。
@@ -258,6 +273,9 @@ func (s *Cloud115PlaybackService) ensureCloudTranscode(
}
}
}
// Reserve the in-flight attempt before releasing the lock. Two simultaneous
// explicit starts must not both submit a push while the first awaits 115.
s.pushState[key] = cloud115PushAttempt{at: now, message: "正在提交 115 云端转码请求…"}
s.mu.Unlock()
err := client.SubmitVideoPush(ctx, pickCode, "vip_push")
+10 -3
View File
@@ -16,7 +16,7 @@ import (
"github.com/truewhile/MeBox/internal/service/cloud115"
)
func TestCloud115PlaybackInfoTriggersTranscodeOnce(t *testing.T) {
func TestCloud115PlaybackInfoIsReadOnlyAndStartTriggersTranscodeOnce(t *testing.T) {
var playCalls int32
var pushCalls int32
var ready int32
@@ -93,9 +93,16 @@ func TestCloud115PlaybackInfoTriggersTranscodeOnce(t *testing.T) {
}
svc := NewCloud115PlaybackService(nil, zap.NewNop(), repos, strm)
info, err := svc.PlaybackInfo(ctx, media.ID, 0)
if _, err := svc.PlaybackInfo(ctx, media.ID, 0); err != nil {
t.Fatalf("read playback info: %v", err)
}
if got := atomic.LoadInt32(&pushCalls); got != 0 {
t.Fatalf("read-only playback info triggered %d cloud transcode pushes", got)
}
info, err := svc.StartTranscode(ctx, media.ID, 0)
if err != nil {
t.Fatalf("first playback info: %v", err)
t.Fatalf("start transcode: %v", err)
}
if info.Provider != model.StrmProvider115 || info.DefaultQuality != "4" {
t.Fatalf("unexpected playback info: %#v", info)
@@ -0,0 +1,77 @@
package service
import (
"context"
"encoding/json"
"net/http"
"net/http/httptest"
"sync"
"sync/atomic"
"testing"
"time"
"go.uber.org/zap"
"github.com/truewhile/MeBox/internal/service/cloud115"
)
func TestCloud115ConcurrentStartsDoNotDuplicateInFlightPush(t *testing.T) {
started := make(chan struct{})
release := make(chan struct{})
var releaseOnce sync.Once
var pushCalls atomic.Int32
server := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
if pushCalls.Add(1) == 1 {
close(started)
}
<-release
w.Header().Set("Content-Type", "application/json")
_ = json.NewEncoder(w).Encode(map[string]any{"state": true, "code": 0, "data": []any{}})
}))
defer server.Close()
defer releaseOnce.Do(func() { close(release) })
oldBase := cloud115.ProAPIBase
cloud115.ProAPIBase = server.URL
defer func() { cloud115.ProAPIBase = oldBase }()
client := cloud115.NewOpenClient("review-app", "access", "refresh")
svc := NewCloud115PlaybackService(nil, zap.NewNop(), nil, nil)
quality := PlaybackQuality{ID: "4", Label: "1080P", Source: "cloud"}
ctx, cancel := context.WithTimeout(t.Context(), 5*time.Second)
defer cancel()
first := make(chan struct{})
go func() { defer close(first); svc.ensureCloudTranscode(ctx, "account", "pick", quality, client) }()
select {
case <-started:
case <-ctx.Done():
t.Fatal("first push did not reach the local server")
}
second := make(chan Cloud115TranscodeState, 1)
go func() { second <- svc.ensureCloudTranscode(ctx, "account", "pick", quality, client) }()
secondReceived := false
select {
case state := <-second:
secondReceived = true
if state.State != "transcoding" {
t.Errorf("in-flight request should be reported as transcoding: %#v", state)
}
case <-time.After(250 * time.Millisecond):
t.Error("second start issued another request instead of reusing the in-flight push")
}
releaseOnce.Do(func() { close(release) })
select {
case <-first:
case <-ctx.Done():
t.Fatal("first push did not finish")
}
// The duplicate path is awaited before restoring the package-wide fake API.
if !secondReceived {
select {
case <-second:
case <-ctx.Done():
t.Fatal("second start did not finish")
}
}
if got := pushCalls.Load(); got != 1 {
t.Fatalf("pushes=%d, want 1", got)
}
}
@@ -0,0 +1,254 @@
package service
import (
"context"
"fmt"
"strconv"
"strings"
"sync"
"testing"
"time"
"github.com/glebarez/sqlite"
"go.uber.org/zap"
"gorm.io/gorm"
"gorm.io/gorm/logger"
"github.com/truewhile/MeBox/internal/config"
"github.com/truewhile/MeBox/internal/model"
"github.com/truewhile/MeBox/internal/repository"
)
// resetEmbyInvalThrottle 清掉全局失效节流,让测试能断言「这一次调用到底有
// 没有真的清缓存」。测试结束后恢复原值,避免影响同包其他用例。
func resetEmbyInvalThrottle(t *testing.T) {
t.Helper()
embyInvalMu.Lock()
previous := embyInvalLast
embyInvalLast = time.Time{}
embyInvalMu.Unlock()
t.Cleanup(func() {
embyInvalMu.Lock()
embyInvalLast = previous
embyInvalMu.Unlock()
})
}
func newCachedTestEmbyService(t *testing.T) *EmbyService {
t.Helper()
svc := newTestEmbyService(t)
cache := NewRuntimeCacheService(&config.Config{}, zap.NewNop())
t.Cleanup(func() { _ = cache.Close() })
return svc.SetRuntimeCache(cache)
}
// TestRecordProgressKeepsEmbyPayloadCache 覆盖首页加载慢的核心回归点:
// 播放期间客户端每 5-10 秒上报一次进度,如果每次都清空 media:emby:*,
// 客户端刷新首页时 73 个媒体库的 Latest 会全部回源重建(生产环境实测
// 首批 8-25s)。普通进度上报必须保留载荷缓存;只有「已看完」翻转才失效。
func TestRecordProgressKeepsEmbyPayloadCache(t *testing.T) {
svc := newCachedTestEmbyService(t)
const mediaID = "ep-1"
media := model.Media{
Base: model.Base{ID: mediaID},
Title: "第 1 集",
Path: `/media/tv/Show/Season 01/S01E01.mkv`,
DurationSec: 100,
}
if err := svc.repo.DB.Create(&media).Error; err != nil {
t.Fatalf("create media: %v", err)
}
key := svc.embyLatestCacheKey("user-1", "", 16)
svc.cache.SetJSONWithStale(t.Context(), key, embyLatestCacheValue{
Items: []map[string]any{{"Id": "cached-item"}},
}, time.Minute, time.Hour)
resetEmbyInvalThrottle(t)
// 60s / 100s:播放中途
if err := svc.RecordProgressWithSession(t.Context(), "user-1", mediaID, 600_000_000, 1_000_000_000, "session-1"); err != nil {
t.Fatalf("record progress: %v", err)
}
if _, stale, ok := svc.cachedLatestItemsWithStale(t.Context(), key); !ok || stale {
t.Fatal("普通进度上报不应让 Emby 列表载荷缓存失效")
}
resetEmbyInvalThrottle(t)
// 98s / 100s:进入片尾 30s 内的「已看完」区间
if err := svc.RecordProgressWithSession(t.Context(), "user-1", mediaID, 980_000_000, 1_000_000_000, "session-2"); err != nil {
t.Fatalf("record completed progress: %v", err)
}
if _, _, ok := svc.cachedLatestItemsWithStale(t.Context(), key); ok {
t.Fatal("看完状态翻转后应让 Emby 列表载荷缓存失效")
}
}
// TestFindSeriesGroupCachesSiblingVirtualSeries 覆盖 N+1 回归点:一次全表
// 扫描算出了整库的剧集分组,若只缓存命中的那一部,同一页的 16 张卡片和
// 16 张海报会各自重扫一遍 media 表。批量记忆后同批分组都应可命中。
func TestFindSeriesGroupCachesSiblingVirtualSeries(t *testing.T) {
svc := newTestEmbyService(t)
lib := model.Library{Name: "剧集", Path: `/media/tv`, Type: "tv", Enabled: true}
if err := svc.repo.Library.Create(t.Context(), &lib); err != nil {
t.Fatalf("create library: %v", err)
}
shows := []string{"Alpha", "Bravo", "Charlie"}
rows := make([]model.Media, 0, len(shows)*2)
for _, show := range shows {
for ep := 1; ep <= 2; ep++ {
rows = append(rows, model.Media{
Base: model.Base{ID: show + "-" + strconv.Itoa(ep)},
LibraryID: lib.ID,
Title: show,
Path: fmt.Sprintf("/media/tv/%s/Season 01/S01E%02d.mkv", show, ep),
SeasonNum: 1,
EpisodeNum: ep,
})
}
}
for i := range rows {
if err := svc.repo.DB.Create(&rows[i]).Error; err != nil {
t.Fatalf("create media: %v", err)
}
}
seriesIDs := map[string]string{}
for i := range rows {
if rows[i].EpisodeNum == 1 {
seriesIDs[rows[i].Title] = svc.seriesIDForMedia(t.Context(), &rows[i])
}
}
target, ok := seriesIDs["Alpha"]
if !ok {
t.Fatal("missing fixture series")
}
if _, found, err := svc.findSeriesGroup(t.Context(), target, ""); err != nil || !found {
t.Fatalf("findSeriesGroup(%s) = (%v, %v), want found", target, found, err)
}
for show, id := range seriesIDs {
if show == "Alpha" {
continue
}
if _, cached := svc.cachedSeriesGroup(id); !cached {
t.Fatalf("同一次扫描算出的 %s 应一并进入缓存,否则每张卡片/海报都会重扫整库", show)
}
}
}
// TestLatestPayloadKeepsStaleCopyForIdleClients 保证「最新添加」载荷在新鲜期
// 之后仍有可用的陈旧副本:客户端空闲很久后再打开首页时,先拿陈旧结果并由
// 后台重建,而不是让全部媒体库同时回源。
func TestLatestPayloadKeepsStaleCopyForIdleClients(t *testing.T) {
svc := newCachedTestEmbyService(t)
key := svc.embyLatestCacheKey("user-1", "", 16)
svc.cache.SetJSONWithStale(t.Context(), key, embyLatestCacheValue{
Items: []map[string]any{{"Id": "stale-item"}},
}, time.Nanosecond, embyLatestStaleWindow)
time.Sleep(2 * time.Millisecond)
items, stale, ok := svc.cachedLatestItemsWithStale(t.Context(), key)
if !ok || !stale {
t.Fatalf("expected a stale hit, got ok=%v stale=%v", ok, stale)
}
if len(items) != 1 || items[0]["Id"] != "stale-item" {
t.Fatalf("unexpected stale payload: %#v", items)
}
}
// mediaScanCounter 统计真正落到 media 表上的查询次数,用来断言「整库数据
// 只扫一次」:GORM 的 Trace 回调里能拿到最终 SQL。
type mediaScanCounter struct {
logger.Interface
mu sync.Mutex
scans int
}
func (c *mediaScanCounter) Trace(ctx context.Context, begin time.Time, fc func() (string, int64), err error) {
sql, _ := fc()
if strings.Contains(sql, "FROM `media`") {
c.mu.Lock()
c.scans++
c.mu.Unlock()
}
}
func (c *mediaScanCounter) Scans() int {
c.mu.Lock()
defer c.mu.Unlock()
return c.scans
}
// TestVirtualSeriesLookupsShareOneScan 覆盖冷启动风暴的回归点:客户端刷新
// 首页时 73 个库的卡片和海报各自要一个虚拟 series 分组。修好之后,第一次
// 扫描把整库分组都写进缓存,同一页剩下的查询都不该再落 SQL;随后一个不存在
// 的虚拟 ID 也不该反复触发整库重扫。
func TestVirtualSeriesLookupsShareOneScan(t *testing.T) {
counter := &mediaScanCounter{Interface: logger.Discard}
db, err := gorm.Open(sqlite.Open(":memory:"), &gorm.Config{Logger: counter})
if err != nil {
t.Fatalf("open db: %v", err)
}
if sqlDB, err := db.DB(); err == nil {
sqlDB.SetMaxOpenConns(1)
t.Cleanup(func() { _ = sqlDB.Close() })
}
if err := db.AutoMigrate(model.AllModels()...); err != nil {
t.Fatalf("migrate: %v", err)
}
svc := NewEmbyService(&config.Config{}, zap.NewNop(), repository.New(db))
lib := model.Library{Name: "剧集", Path: `/media/tv`, Type: "tv", Enabled: true}
if err := svc.repo.Library.Create(t.Context(), &lib); err != nil {
t.Fatalf("create library: %v", err)
}
shows := []string{"Alpha", "Bravo", "Charlie"}
rows := make([]model.Media, 0, len(shows)*2)
for _, show := range shows {
for ep := 1; ep <= 2; ep++ {
rows = append(rows, model.Media{
Base: model.Base{ID: show + "-" + strconv.Itoa(ep)},
LibraryID: lib.ID,
Title: show,
Path: fmt.Sprintf("/media/tv/%s/Season 01/S01E%02d.mkv", show, ep),
SeasonNum: 1,
EpisodeNum: ep,
})
}
}
for i := range rows {
if err := svc.repo.DB.Create(&rows[i]).Error; err != nil {
t.Fatalf("create media: %v", err)
}
}
seriesIDs := map[string]string{}
for i := range rows {
if rows[i].EpisodeNum == 1 {
seriesIDs[rows[i].Title] = svc.seriesIDForMedia(t.Context(), &rows[i])
}
}
if _, found, err := svc.findSeriesGroup(t.Context(), seriesIDs["Alpha"], ""); err != nil || !found {
t.Fatalf("findSeriesGroup(Alpha) = (%v, %v), want found", found, err)
}
if got := counter.Scans(); got != 1 {
t.Fatalf("首次查询应恰好扫描 media 一次,实际 %d 次", got)
}
for _, show := range []string{"Bravo", "Charlie"} {
if _, found, err := svc.findSeriesGroup(t.Context(), seriesIDs[show], ""); err != nil || !found {
t.Fatalf("findSeriesGroup(%s) = (%v, %v), want found", show, found, err)
}
}
if got := counter.Scans(); got != 1 {
t.Fatalf("同批分组应命中缓存,不应再扫 media,实际累计 %d 次", got)
}
if _, found, err := svc.findSeriesGroup(t.Context(), embyVirtualSeriesPrefix+"missing", ""); err != nil || found {
t.Fatalf("未知虚拟 ID 应为 not found,实际 found=%v err=%v", found, err)
}
if got := counter.Scans(); got != 1 {
t.Fatalf("刚扫过整库后,未知 ID 不应重扫 media,实际累计 %d 次", got)
}
}
+20
View File
@@ -56,6 +56,17 @@ type EmbyService struct {
virtualSeries map[string]embySeriesCacheEntry
virtualSeasons map[string]embySeasonCacheEntry
virtualArtwork map[string]embyArtworkCacheEntry
// virtualGroups 把「整库虚拟剧集分组」的扫描折叠成一次。
//
// 客户端刷新首页时会同时请求 73 个库的 Latest,每个库的卡片和海报都要
// 各自的虚拟 series 分组。不同剧的分组缓存键不同,singleflight 键也不同,
// 于是第一波请求会并发重扫整张 media 表(生产环境实测冷启动 14-17s)。
// 按用户折叠成一次扫描后,同一波请求共享结果并整批进缓存。
virtualGroups singleflight.Group
// virtualScanAt 记录每个用户最近一次整库扫描的时间,用于避免为同一个
// 不存在的 id 反复重扫(必须按用户区分:可见性不同,别人的快照不能当
// 自己的答案)。由 virtualMu 保护。
virtualScanAt map[string]time.Time
visibilityMu sync.RWMutex
visibilityCache map[string]embyVisibilityCacheEntry
@@ -186,6 +197,15 @@ const (
embyVirtualSeriesCap = 8000
embyVirtualSeasonCap = 16000
embyVirtualArtworkCap = 24000
// embySeriesGroupBulkCacheEpisodes 限制一次批量记忆的剧集行数。
//
// 批量记忆把「一次扫描服务整页卡片 + 整页海报」的收益拿满,但同时让整库
// 剧集常驻内存。超过该量级时退回只缓存命中的那一部,避免大库吃光内存。
embySeriesGroupBulkCacheEpisodes = 8000
// embyVirtualScanGrace 是「刚做过整库虚拟剧集扫描」的保护窗口。窗口内再
// 请求一个不在缓存里的虚拟 series ID,直接按不存在处理,避免为同一个
// 未知 id 反复重扫整库。
embyVirtualScanGrace = 10 * time.Second
// Clients that already cached the 1x1 placeholder treat a stable tag as
// immutable. Virtual ids are the ones that served that placeholder, so
// only those tags get a suffix that forces a refetch.
+10 -1
View File
@@ -6,6 +6,7 @@ import (
"sort"
"strconv"
"strings"
"time"
)
type embyItemsCacheValue struct {
@@ -77,7 +78,15 @@ func (e *EmbyService) embySimilarCacheKey(mediaID, userID string, limit int) str
}
// defaultEmbyLatestCacheTTLSeconds 是 Emby「最新添加」缓存的兜底时长。
const defaultEmbyLatestCacheTTLSeconds = 300
const defaultEmbyLatestCacheTTLSeconds = 900
// embyLatestStaleWindow 是「最新添加」载荷在进程内保留的陈旧副本时长。
//
// 客户端空闲很久之后再打开首页时,新鲜期早已过去:生产环境观察到这时
// 73 个媒体库的 Latest 同时回源,首批请求要 8-25s 才回来,客户端等不及
// 还会重试,进一步放大压力。保留陈旧副本后,这类请求立刻拿到上次的结果,
// 由后台异步重建(见 refreshLatestItemsAsync),用户不再等 SQL。
const embyLatestStaleWindow = 24 * time.Hour
// embyLatestCacheTTLSeconds 返回「最新添加」列表的缓存时长。它刻意比通用
// 媒体缓存更长:客户端刷新首页时会同时请求全部媒体库的 Latest(生产环境
+1 -1
View File
@@ -153,7 +153,7 @@ func (e *EmbyService) loadLatestItemsCached(ctx context.Context, userID, parentI
}
if e.cache != nil {
freshTTL := time.Duration(e.embyLatestCacheTTLSeconds()) * time.Second
e.cache.SetJSONWithStale(loadCtx, cacheKey, value, freshTTL, freshTTL+30*time.Minute)
e.cache.SetJSONWithStale(loadCtx, cacheKey, value, freshTTL, freshTTL+embyLatestStaleWindow)
}
e.rememberArtworkRefs(value.Artwork)
return value, nil
+130 -40
View File
@@ -41,47 +41,65 @@ func (e *EmbyService) findSeriesGroup(ctx context.Context, id, userID string) (e
return embySeriesGroup{}, false, nil
}
if strings.HasPrefix(id, embyVirtualSeriesPrefix) {
if group, ok := e.cachedSeriesGroup(id); ok {
return group, true, nil
}
return e.findVirtualSeriesGroup(ctx, id, userID)
}
var rows []model.Media
q := e.repo.DB.WithContext(ctx).Model(&model.Media{}).Where("season_num > 0 OR episode_num > 0")
q = e.applyUserMediaVisibility(ctx, q, userID)
if !strings.HasPrefix(id, embyVirtualSeriesPrefix) {
q = q.Where("series_id = ?", id)
} else {
// 虚拟 series ID 只可能来自 series_id 为空的媒体:
// 有 series_id 时分组 key 就是 series_id 本身(UUID,不带虚拟前缀)。
q = q.Where("series_id IS NULL OR series_id = ''")
}
q = q.Where("series_id = ?", id)
if err := q.Order("media.season_num asc, media.episode_num asc, media.episode_fraction asc, media.created_at asc").Limit(embySeriesGroupingLimit).Find(&rows).Error; err != nil {
return embySeriesGroup{}, false, err
}
for _, group := range e.seriesGroupsFromMedia(ctx, rows) {
groups := e.seriesGroupsFromMedia(ctx, rows)
bulkCached := e.rememberScannedSeriesGroups(groups)
for _, group := range groups {
if group.ID == id {
e.rememberSeriesGroup(group)
if !bulkCached {
e.rememberSeriesGroup(group)
}
return group, true, nil
}
}
if !strings.HasPrefix(id, embyVirtualSeriesPrefix) {
if series, err := e.repo.Series.FindByID(ctx, id); err != nil {
return embySeriesGroup{}, false, err
} else if series != nil {
return embySeriesGroup{
ID: series.ID,
LibraryID: series.LibraryID,
Name: series.Title,
PosterURL: series.PosterURL,
BackdropURL: series.BackdropURL,
Overview: series.Overview,
Rating: series.Rating,
Year: series.Year,
TMDbID: series.TMDbID,
BangumiID: series.BangumiID,
CreatedAt: series.CreatedAt,
DateLastMediaAdded: series.CreatedAt,
}, true, nil
if series, err := e.repo.Series.FindByID(ctx, id); err != nil {
return embySeriesGroup{}, false, err
} else if series != nil {
return embySeriesGroup{
ID: series.ID,
LibraryID: series.LibraryID,
Name: series.Title,
PosterURL: series.PosterURL,
BackdropURL: series.BackdropURL,
Overview: series.Overview,
Rating: series.Rating,
Year: series.Year,
TMDbID: series.TMDbID,
BangumiID: series.BangumiID,
CreatedAt: series.CreatedAt,
DateLastMediaAdded: series.CreatedAt,
}, true, nil
}
return embySeriesGroup{}, false, nil
}
// findVirtualSeriesGroup 处理 series_id 为空的剧集分组(虚拟 series ID)。
//
// 虚拟 ID 是 hash(库名 + 剧名),无法反解,只能整库算出分组后按 ID 取用。
// 这里走 loadVirtualSeriesGroups,把同一用户的并发请求折叠成一次扫描:
// 首页一次刷新会同时要 73 个库的卡片和海报,各扫一遍在 2 核主机上要 14-17s。
func (e *EmbyService) findVirtualSeriesGroup(ctx context.Context, id, userID string) (embySeriesGroup, bool, error) {
if group, ok := e.cachedSeriesGroup(id); ok {
return group, true, nil
}
if e.virtualScanRecently(userID) {
return embySeriesGroup{}, false, nil
}
groups, err := e.loadVirtualSeriesGroups(ctx, userID)
if err != nil {
return embySeriesGroup{}, false, err
}
for _, group := range groups {
if group.ID == id {
return group, true, nil
}
}
return embySeriesGroup{}, false, nil
@@ -103,20 +121,14 @@ func (e *EmbyService) findSeasonGroup(ctx context.Context, id, userID string) (e
return season, true, nil
}
// 回退:未刮削(series_id 为空,虚拟 key 由库名+名称派生)的媒体只能全量分组。
var rows []model.Media
q := e.repo.DB.WithContext(ctx).Model(&model.Media{}).
Where("(series_id IS NULL OR series_id = '') AND (season_num > 0 OR episode_num > 0)")
q = e.applyUserMediaVisibility(ctx, q, userID)
if err := q.
Order("media.season_num asc, media.episode_num asc, media.episode_fraction asc, media.created_at asc").
Limit(embySeriesGroupingLimit).
Find(&rows).Error; err != nil {
// 复用共享扫描:首页并发刷新时整库只扫一次。
groups, err := e.loadVirtualSeriesGroups(ctx, userID)
if err != nil {
return embySeasonGroup{}, false, err
}
for _, series := range e.seriesGroupsFromMedia(ctx, rows) {
for _, series := range groups {
for _, season := range e.seasonsForSeries(series) {
if season.ID == id {
e.rememberSeriesGroup(series)
return season, true, nil
}
}
@@ -186,6 +198,84 @@ func (e *EmbyService) seasonGroupForSeries(ctx context.Context, id, seriesID, us
return embySeasonGroup{}, false, nil
}
// rememberScannedSeriesGroups 批量记住一次扫描得到的全部分组,返回是否真的
// 批量写入(超过剧集预算时不做,调用方需自行缓存命中的那一部)。
//
// 只缓存命中的一部时,同一部剧的多张卡片/海报仍要各扫一次;整批写入后,
// 同一次扫描算出来的其他剧也直接命中,于是整页 16 张卡片 + 16 张海报共用
// 一次全表扫描。
func (e *EmbyService) rememberScannedSeriesGroups(groups []embySeriesGroup) bool {
if e == nil || len(groups) == 0 {
return false
}
episodes := 0
for i := range groups {
episodes += len(groups[i].Episodes)
if episodes > embySeriesGroupBulkCacheEpisodes {
return false
}
}
e.rememberSeriesGroups(groups)
return true
}
// loadVirtualSeriesGroups 返回 series_id 为空的整库剧集分组。
//
// 同一用户的并发调用共享同一次扫描:客户端刷新首页时 73 个库的 Latest 同时
// 进来,每个库的卡片和海报又各自要自己的虚拟 series 分组。分组缓存键各不
// 相同,singleflight 也就不会合并,于是第一波请求把整张 media 表并发重算
// 了 73 次(生产环境实测 14-17s)。这里按用户折叠成一次,结果整批进缓存,
// 调用方拿到切片后按 ID 取用即可。
func (e *EmbyService) loadVirtualSeriesGroups(ctx context.Context, userID string) ([]embySeriesGroup, error) {
if e == nil || e.repo == nil {
return nil, nil
}
v, err, _ := e.virtualGroups.Do("virtual:"+userID, func() (any, error) {
// 与 loadLatestItemsCached 一致:一个客户端断开不应取消正在为其他
// 客户端填充的共享扫描。
loadCtx, cancel := context.WithTimeout(context.WithoutCancel(ctx), 30*time.Second)
defer cancel()
var rows []model.Media
q := e.repo.DB.WithContext(loadCtx).Model(&model.Media{}).
Where("(series_id IS NULL OR series_id = '') AND (season_num > 0 OR episode_num > 0)")
q = e.applyUserMediaVisibility(loadCtx, q, userID)
if err := q.
Order("media.season_num asc, media.episode_num asc, media.episode_fraction asc, media.created_at asc").
Limit(embySeriesGroupingLimit).
Find(&rows).Error; err != nil {
return nil, err
}
groups := e.seriesGroupsFromMedia(loadCtx, rows)
e.rememberSeriesGroups(groups)
e.markVirtualScan(userID)
return groups, nil
})
if err != nil {
return nil, err
}
groups, _ := v.([]embySeriesGroup)
return groups, nil
}
// markVirtualScan 记录该用户最近一次整库扫描的时间。
func (e *EmbyService) markVirtualScan(userID string) {
e.virtualMu.Lock()
if e.virtualScanAt == nil {
e.virtualScanAt = make(map[string]time.Time, 4)
}
e.virtualScanAt[userID] = time.Now()
e.virtualMu.Unlock()
}
// virtualScanRecently 报告该用户是否刚做过一次整库扫描。窗口内仍不在缓存里
// 的目标 ID 按不存在处理,避免为同一个未知 ID 反复重扫整库。
func (e *EmbyService) virtualScanRecently(userID string) bool {
e.virtualMu.RLock()
scannedAt := e.virtualScanAt[userID]
e.virtualMu.RUnlock()
return !scannedAt.IsZero() && time.Since(scannedAt) < embyVirtualScanGrace
}
func (e *EmbyService) seriesGroupsFromMedia(ctx context.Context, rows []model.Media) []embySeriesGroup {
byID := map[string]*embySeriesGroup{}
order := []string{}
+29 -3
View File
@@ -26,19 +26,45 @@ func (e *EmbyService) rememberSeriesGroup(group embySeriesGroup) {
if e == nil || strings.TrimSpace(group.ID) == "" {
return
}
e.rememberSeriesGroups([]embySeriesGroup{group})
}
// rememberSeriesGroups 批量记住一组剧集分组(含派生季与封面)。
//
// 一次全表扫描会同时算出整库的剧集分组,而调用方往往只关心其中一部。若只
// 逐个 rememberSeriesGroup,同一次首页刷新里 16 张卡片 + 16 张海报都要各自
// 命中自己的虚拟 series ID,未命中就重扫整张 media 表(生产环境实测单次
// 1.3s,73 个库并发时把连接池排满,首页要 8-25s)。批量写入后冷缓存只付
// 一次扫描,其余全部是 O(1) 命中。
func (e *EmbyService) rememberSeriesGroups(groups []embySeriesGroup) {
if e == nil || len(groups) == 0 {
return
}
expiresAt := time.Now().Add(embyVirtualCacheTTL)
e.virtualMu.Lock()
defer e.virtualMu.Unlock()
if e.virtualSeries == nil {
e.virtualSeries = make(map[string]embySeriesCacheEntry)
e.virtualSeries = make(map[string]embySeriesCacheEntry, len(groups))
}
if e.virtualSeasons == nil {
e.virtualSeasons = make(map[string]embySeasonCacheEntry)
e.virtualSeasons = make(map[string]embySeasonCacheEntry, len(groups))
}
if e.virtualArtwork == nil {
e.virtualArtwork = make(map[string]embyArtworkCacheEntry)
e.virtualArtwork = make(map[string]embyArtworkCacheEntry, len(groups)*2)
}
for _, group := range groups {
e.rememberSeriesGroupLocked(group, expiresAt)
}
// 写入后再裁剪:逐条裁剪会让整批写入退化成 O(n²),且中途裁剪可能把
// 同一批里刚写进去的分组又淘汰掉。
e.trimVirtualCachesLocked(time.Now())
}
// rememberSeriesGroupLocked 写入单个分组,调用方必须已持有 virtualMu。
func (e *EmbyService) rememberSeriesGroupLocked(group embySeriesGroup, expiresAt time.Time) {
if strings.TrimSpace(group.ID) == "" {
return
}
e.virtualSeries[group.ID] = embySeriesCacheEntry{group: group, expiresAt: expiresAt}
e.virtualArtwork[group.ID] = embyArtworkCacheEntry{primary: group.PosterURL, backdrop: group.BackdropURL, expiresAt: expiresAt}
e.virtualArtwork[group.ID+"-bd"] = embyArtworkCacheEntry{primary: group.PosterURL, backdrop: group.BackdropURL, expiresAt: expiresAt}
+27 -2
View File
@@ -2,6 +2,9 @@ package service
import (
"context"
"crypto/sha256"
"encoding/hex"
"io"
"sort"
"strings"
"time"
@@ -278,6 +281,20 @@ func isSupportedEmbyCollectionType(t string) bool {
return false
}
// libraryCoverTag 生成媒体库拼图封面的图片版本号。
//
// 第三方客户端把库主图按 (URL, ImageTag) 缓存,tag 不变就不会重新请求。
// 用成员海报列表的指纹做 tag,库内容变化或生成规则调整时客户端自然会刷新。
func libraryCoverTag(libraryID string, sources []string) string {
h := sha256.New()
_, _ = io.WriteString(h, libraryID)
for _, src := range sources {
_, _ = io.WriteString(h, src)
_, _ = h.Write([]byte{0})
}
return libraryID + "-" + hex.EncodeToString(h.Sum(nil))[:8]
}
func (e *EmbyService) libraryAsView(ctx context.Context, l *model.Library) map[string]any {
collectionType := "movies"
switch l.Type {
@@ -292,9 +309,17 @@ func (e *EmbyService) libraryAsView(ctx context.Context, l *model.Library) map[s
}
// 库封面:与剧集/电影一样,只有在能解析出封面时才广告 ImageTags.Primary,
// 否则客户端会认为该库无主图而不去请求 /Images/Primary。
//
// 拼图封面的 tag 带上「成员海报」的指纹:库内容变化或生成规则调整(例如
// 从 2×2 改成竖切)时 tag 会变,客户端才会丢缓存重新拉图;固定用库 ID
// 会让第三方客户端永远显示第一次缓存下来的封面。
imageTags := map[string]string{}
if e.LibraryHasCover(ctx, l.ID) {
imageTags["Primary"] = l.ID
if sources, lib := LibraryCoverSourcesForLibrary(ctx, e.repo, l.ID); lib != nil {
if len(sources) > 0 {
imageTags["Primary"] = libraryCoverTag(l.ID, sources)
} else if strings.TrimSpace(lib.CoverURL) != "" {
imageTags["Primary"] = l.ID
}
}
return map[string]any{
"Id": l.ID,
+11 -5
View File
@@ -136,7 +136,12 @@ func (e *EmbyService) RecordProgressWithSession(ctx context.Context, userID, med
SessionID: playSessionID,
SessionStartedAtMs: embyPlaySessionStartedAtMs(playSessionID),
})
if err == nil {
// 普通进度上报不清列表载荷缓存。播放期间客户端每 5-10 秒上报一次,
// 每次清空 media:emby:* 会让「最新添加」整批 payload(客户端刷新首页时
// 一次并发 73 个库)反复回源 SQL,生产环境实测因此出现 8-25s 的加载。
// 续播位置由 /Items/Resume 与条目详情实时查库,列表里的进度条允许滞后;
// 只有「已看完」这类状态翻转才让缓存失效(position 到达片尾 30s 内)。
if err == nil && playbackProgressCompleted(pos, dur) {
e.invalidateEmbyItemsCache(ctx)
}
return err
@@ -287,10 +292,11 @@ func applyMeBoxUserData(raw any, history *model.PlaybackHistory, favorite bool)
return userData
}
// embyInvalMu 节流全量缓存失效:播放期间客户端每 5-10s 上报一次进度,
// 每次都 SCAN+DEL 全部 media:emby:* 缓存会把缓存命中率持续打穿(其他
// 客户端每次翻页都回源 SQL)。条目载荷的 UserData 在请求时动态合并,
// 进度类变更做 30s 节流即可,不影响正确性观感。
// embyInvalMu 节流全量缓存失效:收藏/标记已看这类显式操作会改变列表里的
// UserData,必须让 payload 失效。客户端连续点选时会短时间触发多次,节流到
// 30s 一次即可,避免每次都 SCAN+DEL 全部 media:emby:* 缓存。
// 播放进度上报不走这里(见 RecordProgressWithSession):续播位置实时查库,
// 列表里的进度条允许滞后,不值得为它把整批首页载荷打回源 SQL。
var (
embyInvalMu sync.Mutex
embyInvalLast time.Time
+109
View File
@@ -0,0 +1,109 @@
package service
import (
"crypto/sha256"
"encoding/hex"
"errors"
"net/http"
"os"
"path/filepath"
"strings"
"time"
"unicode"
)
var ErrInvalidHLSPlaybackScope = errors.New("invalid HLS playback scope")
var ErrHLSGenerationSuperseded = errors.New("HLS playback generation superseded")
// HLSPlaybackScope separates a media's independent users and player instances.
// UserID must come from server authentication, never from a request parameter.
type HLSPlaybackScope struct {
UserID string
SessionID string
}
func HLSPlaybackScopeForRequest(r *http.Request, userID string) (HLSPlaybackScope, error) {
scope := HLSPlaybackScope{UserID: strings.TrimSpace(userID)}
if scope.UserID == "" || r == nil {
return scope, ErrInvalidHLSPlaybackScope
}
for _, key := range []string{"session_id", "PlaySessionId", "playSessionId", "DeviceId", "deviceId"} {
if value := strings.TrimSpace(r.URL.Query().Get(key)); value != "" {
scope.SessionID = value
break
}
}
if scope.SessionID == "" {
if device := strings.TrimSpace(r.Header.Get("X-Emby-Device-Id")); device != "" {
scope.SessionID = device
}
}
if len(scope.SessionID) > 256 || strings.IndexFunc(scope.SessionID, unicode.IsControl) >= 0 {
return scope, ErrInvalidHLSPlaybackScope
}
// Clients without a session/device identifier remain compatible but only
// share a legacy task with the same authenticated user, never other users.
return scope, nil
}
func (scope HLSPlaybackScope) jobKey(mediaID string) string {
if scope.UserID == "" && scope.SessionID == "" {
return mediaID // Existing internal callers/tests retain their legacy API.
}
sum := sha256.Sum256([]byte(scope.UserID + "\x00" + mediaID + "\x00" + scope.SessionID))
return "player-" + hex.EncodeToString(sum[:])
}
func hlsPlaybackJobKey(mediaID string, scopes []HLSPlaybackScope) string {
if len(scopes) == 0 {
return mediaID
}
return scopes[0].jobKey(mediaID)
}
func (job *hlsJob) key() string {
if job.jobKey != "" {
return job.jobKey
}
return job.mediaID
}
func (t *TranscoderService) jobPlaylistPath(job *hlsJob) string {
dir := job.outputDir
if dir == "" {
dir = t.HLSDir(job.key())
}
return filepath.Join(dir, "index.m3u8")
}
// hlsOutputDir captures one immutable run directory. Even if a new run starts
// after this check, the request cannot accidentally open its replacement files.
func (t *TranscoderService) hlsOutputDir(key string, generation int64) (string, error) {
t.mu.Lock()
defer t.mu.Unlock()
job := t.jobs[key]
if job == nil {
return "", ErrMediaNotFound
}
if job.seekGen != generation {
return "", ErrHLSGenerationSuperseded
}
t.touchJobLocked(key)
if job.outputDir != "" {
return job.outputDir, nil
}
return t.HLSDir(key), nil
}
func (t *TranscoderService) StopPlayback(mediaID string, scope HLSPlaybackScope) {
key := scope.jobKey(mediaID)
unlock := t.lockJobStart(key)
defer unlock()
t.mu.Lock()
job := t.detachJobLocked(key)
t.mu.Unlock()
waitJobExit(job, 12*time.Second)
// Keep cleanup under the same gate as start: a delayed stop must not wipe
// an output directory created by a concurrent replacement request.
_ = os.RemoveAll(t.HLSDir(key))
}
@@ -0,0 +1,133 @@
package service
import (
"net/http"
"net/http/httptest"
"net/url"
"os"
"path/filepath"
"strings"
"testing"
"time"
"go.uber.org/zap"
"github.com/truewhile/MeBox/internal/config"
"github.com/truewhile/MeBox/internal/model"
"github.com/truewhile/MeBox/internal/repository"
)
func cachedScopedHLS(t *testing.T, scopes ...HLSPlaybackScope) (*StreamService, *TranscoderService, map[string]string) {
t.Helper()
cfg := &config.Config{}
cfg.Cache.CacheDir = t.TempDir()
cfg.Transcoder.Enabled = true
db := newServiceTestDB(t)
if err := db.Create(&model.Media{Base: model.Base{ID: "movie"}, Path: "unused-cached-source", DurationSec: 60}).Error; err != nil {
t.Fatal(err)
}
transcoder := NewTranscoderService(cfg, zap.NewNop(), repository.New(db), nil)
paths := map[string]string{}
for _, scope := range scopes {
key := scope.jobKey("movie")
dir := filepath.Join(transcoder.HLSDir(key), "run-cached")
if err := os.MkdirAll(dir, 0o750); err != nil {
t.Fatal(err)
}
playlist := filepath.Join(dir, "index.m3u8")
if err := os.WriteFile(playlist, []byte("#EXTM3U\n#EXTINF:4,\nseg_00000.ts\n"), 0o600); err != nil {
t.Fatal(err)
}
if err := os.WriteFile(filepath.Join(dir, "seg_00000.ts"), []byte(scope.UserID+"/"+scope.SessionID), 0o600); err != nil {
t.Fatal(err)
}
transcoder.jobs[key] = &hlsJob{mediaID: "movie", jobKey: key, outputDir: dir, subtitleStream: -1, startedAt: time.Now(), cancel: func() {}}
paths[key] = playlist
}
return NewStreamService(cfg, zap.NewNop(), repository.New(db), transcoder), transcoder, paths
}
func TestHLSPlaybackEnsureReusesOnlyMatchingOwnerAndSession(t *testing.T) {
a := HLSPlaybackScope{UserID: "viewer", SessionID: "tab-a"}
b := HLSPlaybackScope{UserID: "viewer", SessionID: "tab-b"}
_, transcoder, paths := cachedScopedHLS(t, a, b)
for _, scope := range []HLSPlaybackScope{a, b} {
playlist, err := transcoder.EnsureJobFromSubtitleQuality(t.Context(), "movie", 0, 0, -1, "", scope)
if err != nil {
t.Fatal(err)
}
if playlist != paths[scope.jobKey("movie")] {
t.Fatalf("scope %#v reused another session's playlist %q", scope, playlist)
}
}
}
func TestHLSPlaybackStopRemovesOnlyItsSessionOutput(t *testing.T) {
a := HLSPlaybackScope{UserID: "viewer", SessionID: "tab-a"}
b := HLSPlaybackScope{UserID: "viewer", SessionID: "tab-b"}
_, transcoder, paths := cachedScopedHLS(t, a, b)
transcoder.StopPlayback("movie", a)
if _, err := os.Stat(paths[a.jobKey("movie")]); !os.IsNotExist(err) {
t.Fatalf("stopped session output must be cleaned up, err=%v", err)
}
if _, err := os.Stat(paths[b.jobKey("movie")]); err != nil {
t.Fatalf("another player's output was removed: %v", err)
}
}
func TestHLSPlaybackIdleCleanupRemovesExpiredSessionOutput(t *testing.T) {
a := HLSPlaybackScope{UserID: "viewer", SessionID: "tab-a"}
b := HLSPlaybackScope{UserID: "viewer", SessionID: "tab-b"}
_, transcoder, paths := cachedScopedHLS(t, a, b)
job := transcoder.jobs[a.jobKey("movie")]
job.lastAccess = time.Now().Add(-2 * time.Minute)
if !transcoder.stopIdleJob(job, time.Minute) {
t.Fatal("expired session was not stopped")
}
if _, err := os.Stat(paths[a.jobKey("movie")]); !os.IsNotExist(err) {
t.Fatalf("expired output was retained: %v", err)
}
if _, err := os.Stat(paths[b.jobKey("movie")]); err != nil {
t.Fatalf("active session output was removed: %v", err)
}
}
func TestHLSPlaybackPropagatesHeaderDeviceIdentityToSegments(t *testing.T) {
req := httptest.NewRequest(http.MethodGet, "/api/hls/movie/index.m3u8?token=test", nil)
req.Header.Set("X-Emby-Device-Id", "legacy-device")
scope, err := HLSPlaybackScopeForRequest(req, "viewer")
if err != nil {
t.Fatal(err)
}
svc, _, _ := cachedScopedHLS(t, scope)
rec := httptest.NewRecorder()
if err := svc.ServeHLSPlaylist(rec, req, "movie", scope); err != nil {
t.Fatal(err)
}
var segment string
for _, line := range strings.Split(rec.Body.String(), "\n") {
if strings.HasPrefix(line, "seg_") {
segment = line
break
}
}
u, err := url.Parse(segment)
if err != nil || u.Query().Get("session_id") == "" {
t.Fatalf("playlist must carry the header-only device scope to child URLs: %q err=%v", segment, err)
}
child := httptest.NewRequest(http.MethodGet, "/api/hls/movie/"+segment, nil)
childScope, err := HLSPlaybackScopeForRequest(child, "viewer")
if err != nil {
t.Fatal(err)
}
if childScope.jobKey("movie") != scope.jobKey("movie") {
t.Fatal("child request changed its owner's device scope")
}
childRec := httptest.NewRecorder()
if err := svc.ServeHLSSegment(childRec, child, "movie", "seg_00000.ts", childScope); err != nil {
t.Fatal(err)
}
if childRec.Body.String() != "viewer/"+scope.SessionID {
t.Fatalf("wrong scoped segment: %s", childRec.Body.String())
}
}
+97
View File
@@ -0,0 +1,97 @@
package service
import (
"net/http"
"net/http/httptest"
"os"
"path/filepath"
"testing"
"go.uber.org/zap"
"github.com/truewhile/MeBox/internal/config"
)
func TestHLSPlaybackScopesIsolateSegmentReadsAndStop(t *testing.T) {
cfg := &config.Config{}
cfg.Cache.CacheDir = t.TempDir()
transcoder := NewTranscoderService(cfg, zap.NewNop(), nil, nil)
svc := NewStreamService(cfg, zap.NewNop(), nil, transcoder)
a := HLSPlaybackScope{UserID: "user-a", SessionID: "tab-a"}
b := HLSPlaybackScope{UserID: "user-b", SessionID: "tab-a"}
c := HLSPlaybackScope{UserID: "user-a", SessionID: "tab-c"}
for _, scope := range []HLSPlaybackScope{a, b, c} {
key := scope.jobKey("movie")
dir := filepath.Join(cfg.Cache.CacheDir, key, "run-current")
if err := os.MkdirAll(dir, 0o750); err != nil {
t.Fatal(err)
}
if err := os.WriteFile(filepath.Join(dir, "seg_00000.ts"), []byte(scope.UserID+"/"+scope.SessionID), 0o600); err != nil {
t.Fatal(err)
}
transcoder.jobs[key] = &hlsJob{mediaID: "movie", jobKey: key, outputDir: dir, seekGen: 100, cancel: func() {}}
}
for _, scope := range []HLSPlaybackScope{a, b, c} {
req := httptest.NewRequest(http.MethodGet, "/api/hls/movie/seg_00000.ts?_seek=100", nil)
rec := httptest.NewRecorder()
if err := svc.ServeHLSSegment(rec, req, "movie", "seg_00000.ts", scope); err != nil {
t.Fatal(err)
}
if got, want := rec.Body.String(), scope.UserID+"/"+scope.SessionID; got != want {
t.Fatalf("segment=%q, want %q", got, want)
}
}
transcoder.StopPlayback("movie", a)
if transcoder.jobs[a.jobKey("movie")] != nil {
t.Fatal("stopped scope is still active")
}
if transcoder.jobs[b.jobKey("movie")] == nil || transcoder.jobs[c.jobKey("movie")] == nil {
t.Fatal("stopping one player affected another user/tab")
}
// A delayed stop from a previous local-playback session must not affect a replacement.
transcoder.StopPlayback("movie", a)
if transcoder.jobs[c.jobKey("movie")] == nil {
t.Fatal("old stop cancelled a new session")
}
}
func TestHLSPlaybackRejectsStaleSegmentGeneration(t *testing.T) {
cfg := &config.Config{}
cfg.Cache.CacheDir = t.TempDir()
transcoder := NewTranscoderService(cfg, zap.NewNop(), nil, nil)
svc := NewStreamService(cfg, zap.NewNop(), nil, transcoder)
scope := HLSPlaybackScope{UserID: "viewer", SessionID: "tab"}
key := scope.jobKey("movie")
dir := t.TempDir()
if err := os.WriteFile(filepath.Join(dir, "seg_00000.ts"), []byte("new-generation"), 0o600); err != nil {
t.Fatal(err)
}
transcoder.jobs[key] = &hlsJob{mediaID: "movie", jobKey: key, outputDir: dir, seekGen: 200, cancel: func() {}}
req := httptest.NewRequest(http.MethodGet, "/api/hls/movie/seg_00000.ts?_seek=100", nil)
rec := httptest.NewRecorder()
if err := svc.ServeHLSSegment(rec, req, "movie", "seg_00000.ts", scope); err == nil {
t.Fatal("stale segment request read a replacement generation")
}
if rec.Body.Len() != 0 {
t.Fatal("stale request exposed replacement bytes")
}
}
func TestHLSPlaybackScopeRequiresServerOwner(t *testing.T) {
req := httptest.NewRequest(http.MethodGet, "/?session_id=tab&user_id=attacker-controlled", nil)
if _, err := HLSPlaybackScopeForRequest(req, ""); err == nil {
t.Fatal("missing authenticated owner was accepted")
}
a, err := HLSPlaybackScopeForRequest(req, "real-owner")
if err != nil {
t.Fatal(err)
}
if a.UserID != "real-owner" {
t.Fatal("client parameter supplied the owner")
}
req = httptest.NewRequest(http.MethodGet, "/?PlaySessionId=emby-session", nil)
b, err := HLSPlaybackScopeForRequest(req, "real-owner")
if err != nil || b.SessionID == "" {
t.Fatalf("Emby session must stay scoped: %#v %v", b, err)
}
}
+12 -6
View File
@@ -49,6 +49,10 @@ type ImageProxy struct {
resizeSemMu sync.Mutex
resizeSem chan struct{}
// resizeQueueWait 是一次请求最多为一格解码槽排队的时长。为 0 时用
// defaultResizeQueueWait;测试会把它调小以免真的等待两秒。
resizeQueueWait time.Duration
// libraryRootsFn returns the configured media library roots so that
// sidecar poster/artwork files stored alongside media (under arbitrary
// per-library paths) are allowed by isAllowedLocalPath. It is provided
@@ -86,6 +90,12 @@ const (
// imageRenditionCacheSubdir 存放挂载 Emby 直接按尺寸产出的成品图:它们
// 已经是客户端要的最终尺寸,和派生缩略图一样属于长期保留的热路径缓存。
imageRenditionCacheSubdir = "renditions"
// defaultResizeQueueWait 是请求为一格解码槽排队的最长时间。一张 3MP
// 海报的解码+缩放约 0.15s,2 核机器两格槽位足以在几秒内消化一整屏
// 海报;等待远好于「槽位忙就回退多兆字节原图」(那样每张图在外部网络
// 下要十几秒)。超时后仍回退原图,保证请求不会被无限期挂住。
defaultResizeQueueWait = 3 * time.Second
)
// NewImageProxy is the constructor.
@@ -190,7 +200,7 @@ func imageResizeConcurrency() int {
}
// Prune removes oldest cached images until disk usage is within the configured limit.
// 原图池按独立配额与保留时长优先淘汰,客户端热路径读取的成品图最后淘汰。
// 原图池按独立配额优先淘汰,客户端热路径读取的成品图最后淘汰。
func (p *ImageProxy) Prune() (PruneImageCacheResult, error) {
if p.cfg == nil || p.cfg.Cache.ImagesMaxSizeMB <= 0 {
return PruneImageCacheResult{}, nil
@@ -199,11 +209,7 @@ func (p *ImageProxy) Prune() (PruneImageCacheResult, error) {
if p.cfg.Cache.ImagesOriginalsMaxSizeMB > 0 {
originalsMaxBytes = int64(p.cfg.Cache.ImagesOriginalsMaxSizeMB) * 1024 * 1024
}
var originalsMaxAge time.Duration
if p.cfg.Cache.ImagesOriginalsTTLHours > 0 {
originalsMaxAge = time.Duration(p.cfg.Cache.ImagesOriginalsTTLHours) * time.Hour
}
totalBytes := int64(p.cfg.Cache.ImagesMaxSizeMB) * 1024 * 1024
pools := ImageCachePools(p.cacheDir, originalsMaxBytes, originalsMaxAge)
pools := ImageCachePools(p.cacheDir, originalsMaxBytes)
return PruneImageCachePools(pools, totalBytes)
}
+79 -14
View File
@@ -17,6 +17,7 @@ import (
"path/filepath"
"strconv"
"strings"
"time"
"go.uber.org/zap"
@@ -138,8 +139,8 @@ func fitSize(srcW, srcH, maxW, maxH int) (int, int) {
var errImageResizeTooLarge = errors.New("image source exceeds resize pixel budget")
// resizeImageData 按选项缩放图片并重新编码。返回 unchanged=true 表示原图
// 本身已满足目标尺寸,调用方应直接输出原始字节(不重复编码、不损失质量)。
// resizeImageData 按选项缩放图片并重新编码。返回 unchanged=true 表示源图
// 已经是可直接下发的紧凑字节,调用方应直接输出原始字节(不重复编码)。
func resizeImageData(data []byte, o imageResizeOptions) (out []byte, ctype string, unchanged bool, err error) {
if len(data) == 0 || !o.active() {
return data, detectContentType(data), true, nil
@@ -151,12 +152,21 @@ func resizeImageData(data []byte, o imageResizeOptions) (out []byte, ctype strin
if cfg.Width <= 0 || cfg.Height <= 0 {
return nil, "", false, errors.New("invalid image dimensions")
}
// 先用 DecodeConfig 判断是否需要解码整图:图已够小就零成本返回原字节。
// 先用 DecodeConfig 判断是否需要缩放。尺寸已在目标框内时仍要重编码:
// 刮削来源的海报常是 1000×1500、1MB 以上的 PNG,直接原样下发会让一处
// 海报墙就从「几十 KB 缩略图」变成「几十 MB 原图」——外部网络下每张
// 都要十几秒。只有本来就是紧凑 Web 格式(JPEG/WebP)且体积很小的源图
// 才值得零成本直出。
dstW, dstH := fitSize(cfg.Width, cfg.Height, o.MaxWidth, o.MaxHeight)
if dstW == cfg.Width && dstH == cfg.Height {
unchangedSize := dstW == cfg.Width && dstH == cfg.Height
if unchangedSize && isCompactWebFormat(format) && len(data) <= compactImageSkipBytes {
return data, detectContentType(data), true, nil
}
if int64(cfg.Width)*int64(cfg.Height) > imageResizeMaxSourcePixels {
if unchangedSize {
// 尺寸已满足但像素量超出解码预算:宁可直出原图,也不冒 OOM 风险。
return data, detectContentType(data), true, nil
}
return nil, "", false, errImageResizeTooLarge
}
@@ -165,7 +175,12 @@ func resizeImageData(data []byte, o imageResizeOptions) (out []byte, ctype strin
return nil, "", false, err
}
dst := image.NewRGBA(image.Rect(0, 0, dstW, dstH))
resizeScaled(dst, src, cfg.Width, cfg.Height)
if unchangedSize {
// 不缩放时省掉一次全图重采样,只做格式转换。
draw.Draw(dst, dst.Bounds(), src, src.Bounds().Min, draw.Src)
} else {
resizeScaled(dst, src, cfg.Width, cfg.Height)
}
// 只有可能带透明的源格式才需要逐像素确认,避免 JPEG 的无谓遍历。
// 写实海报的 PNG 通常比等价 JPEG 大一个数量级,因此在确认不含透明
@@ -231,9 +246,12 @@ func isOpaqueImage(img *image.RGBA) bool {
// resizeSourceKey 生成源图身份键:只覆盖源文件身份(路径 + 大小 + mtime),不含
// 目标尺寸。同一源图的所有尺寸档位共用这个前缀,才能互相列举与派生(小图直接
// 从已缓存的大图缩小,而不是重新解码原图)。
//
// 前缀里的 v3 是缓存格式版本:v2 时期「尺寸已在目标框内」的 PNG 会原样直出、
// 不写缓存,而现在同样条件会重编码成 JPEG,旧缓存与新结果语义不同,必须换键。
func (o imageResizeOptions) resizeSourceKey(sourceID string, stat os.FileInfo) string {
h := sha256.New()
_, _ = fmt.Fprintf(h, "v2|%s", sourceID)
_, _ = fmt.Fprintf(h, "v3|%s", sourceID)
if stat != nil {
_, _ = fmt.Fprintf(h, "|%d|%d", stat.Size(), stat.ModTime().UnixNano())
}
@@ -356,13 +374,53 @@ func (p *ImageProxy) tryAcquireResizeSlot() (func(), bool) {
}
}
// acquireResizeSlotFor 对中等体积的图只尝试一次槽位。忙则放弃缩放,
// 避免背景图墙在 2 核机器上排成数秒。超大原图仍等待,以免直出多兆字节。
func (p *ImageProxy) resizeQueueTimeout() time.Duration {
if p != nil && p.resizeQueueWait > 0 {
return p.resizeQueueWait
}
return defaultResizeQueueWait
}
// acquireResizeSlotFor 对紧凑小图只尝试一次槽位,其余情况最多排队
// resizeQueueTimeout。
//
// 从前这里对 1.5MB 以下的图一律「槽位忙就跳过缩放」,结果是并发加载海报墙
// 时几乎每张图都回退成 1MB 上下的 PNG 原图:一页要传几十 MB,外部网络下单张
// 图就要二十多秒。缩略图本身只有几十 KB,排队几百毫秒明显更划算;只有真正
// 大到会拖慢页面的原图才值得在超时后直出。
func (p *ImageProxy) acquireResizeSlotFor(ctx context.Context, size int64) (func(), bool) {
if size > 0 && size <= resizeQueueBypassBytes {
if size > 0 && size <= compactImageSkipBytes {
return p.tryAcquireResizeSlot()
}
return p.acquireResizeSlot(ctx)
return p.acquireResizeSlotWithin(ctx, p.resizeQueueTimeout())
}
// acquireResizeSlotWithin 在 timeout 内等待一格解码槽;超时或请求取消返回
// false,调用方回退原图。
func (p *ImageProxy) acquireResizeSlotWithin(ctx context.Context, timeout time.Duration) (func(), bool) {
if p == nil {
return func() {}, true
}
if timeout <= 0 {
return p.tryAcquireResizeSlot()
}
p.resizeSemMu.Lock()
if p.resizeSem == nil {
p.resizeSem = make(chan struct{}, imageResizeConcurrency())
}
sem := p.resizeSem
p.resizeSemMu.Unlock()
timer := time.NewTimer(timeout)
defer timer.Stop()
select {
case sem <- struct{}{}:
return func() { <-sem }, true
case <-timer.C:
return nil, false
case <-ctx.Done():
return nil, false
}
}
// serveResizedFromFile 从 srcPath 读取图片,按选项缩放后写出,并把结果缓存
@@ -472,10 +530,6 @@ func (p *ImageProxy) ensureResizeCache(ctx context.Context, srcPath string, o im
// ponytail: 200KB 覆盖这台机器上的典型海报(平均约 100KB);更大的图仍尝试缩放。
const compactImageSkipBytes = 200 * 1024
// resizeQueueBypassBytes 是“槽位忙就放弃缩放”的上限。超过它的原图继续排队,
// 以免把未压缩的剧照直接发给客户端。
const resizeQueueBypassBytes = 1536 * 1024
// serveCompactOriginal 在源文件已经很小且是浏览器可直接显示的 JPEG/WebP 时
// 跳过解码。返回 false 表示仍应走缩放路径。
func serveCompactOriginal(w http.ResponseWriter, r *http.Request, srcPath string, stat os.FileInfo) bool {
@@ -504,6 +558,17 @@ func isCompactWebImage(header []byte) bool {
string(header[8:12]) == "WEBP"
}
// isCompactWebFormat 判断 image.DecodeConfig 报告的格式是否已经适合直接下发
// 给浏览器。PNG/BMP/GIF 即使尺寸很小也可能有几百 KB,需要重编码成 JPEG。
func isCompactWebFormat(format string) bool {
switch strings.ToLower(format) {
case "jpeg", "jpg", "webp":
return true
default:
return false
}
}
// writeResizeCache 原子写入缩放结果;失败只记日志,不影响本次响应。
// 与 writeImageCache 一样不再持有全局锁:临时文件名唯一、rename 原子,
// 持锁只会把缩略图的写盘和原图的写盘串成一条队。
+120 -10
View File
@@ -123,17 +123,58 @@ func TestResizeImageDataScalesDownAndReencodesAsJPEG(t *testing.T) {
}
}
func TestResizeImageDataLeavesImagesWithinBoundsUntouched(t *testing.T) {
// 尺寸已经在目标框内的 PNG 也必须转成 JPEG。刮削来的海报常是 1000×1500、
// 1MB 级的 PNG;此前这种情况原样直出,一处海报墙就要下载几十 MB,外部网络下
// 每张图要十几秒(线上实测单张 1.3MB / 20 多秒)。
func TestResizeImageDataReencodesPNGWithinBounds(t *testing.T) {
data := encodeTestPNG(t, 200, 300, 255)
out, _, unchanged, err := resizeImageData(data, imageResizeOptions{MaxWidth: 400, MaxHeight: 600})
out, ctype, unchanged, err := resizeImageData(data, imageResizeOptions{MaxWidth: 400, MaxHeight: 600, Quality: 80})
if err != nil {
t.Fatalf("resize: %v", err)
}
if unchanged {
t.Fatal("PNG within the target box must still be re-encoded as a compact web format")
}
if ctype != "image/jpeg" {
t.Fatalf("content type = %q, want image/jpeg", ctype)
}
cfg, format, err := image.DecodeConfig(bytes.NewReader(out))
if err != nil {
t.Fatalf("decode re-encoded image: %v", err)
}
if format != "jpeg" {
t.Fatalf("format = %q, want jpeg", format)
}
if cfg.Width != 200 || cfg.Height != 300 {
t.Fatalf("re-encoded to %dx%d, want the original 200x300 (no upscaling)", cfg.Width, cfg.Height)
}
}
// 已经是紧凑 Web 格式且体积很小的源图仍然零成本直出。
func TestResizeImageDataLeavesSmallCompactSourceUntouched(t *testing.T) {
var buf bytes.Buffer
img := image.NewRGBA(image.Rect(0, 0, 200, 300))
for y := 0; y < 300; y++ {
for x := 0; x < 200; x++ {
img.Set(x, y, color.RGBA{R: uint8(x % 16 * 16), G: uint8(y % 16 * 16), B: 200, A: 255})
}
}
if err := jpeg.Encode(&buf, img, &jpeg.Options{Quality: 80}); err != nil {
t.Fatalf("encode test jpeg: %v", err)
}
data := buf.Bytes()
if len(data) > compactImageSkipBytes {
t.Fatalf("test jpeg is %d bytes, expected a compact source", len(data))
}
out, ctype, unchanged, err := resizeImageData(data, imageResizeOptions{MaxWidth: 400, MaxHeight: 600})
if err != nil {
t.Fatalf("resize: %v", err)
}
if !unchanged {
t.Fatal("expected an image already within bounds to be returned unchanged")
t.Fatal("a compact JPEG already within bounds should be served unchanged")
}
if !bytes.Equal(out, data) {
t.Fatal("expected the original bytes to be returned verbatim")
if !bytes.Equal(out, data) || ctype != "image/jpeg" {
t.Fatalf("expected the original jpeg bytes back, got %d bytes of %q", len(out), ctype)
}
}
@@ -302,7 +343,9 @@ func TestServeResizedFromFileSkipsDecodeForCompactJPEG(t *testing.T) {
}
}
func TestServeResizedFromFileDoesNotQueueWhenResizeBusy(t *testing.T) {
// 排队上限:槽位一直不空时,请求最多等 resizeQueueWait,然后回退原图,
// 不会被无限期挂住。这条约束保护的是请求本身,而不是「不等就直出原图」。
func TestServeResizedFromFileFallsBackWhenResizeQueueStaysBusy(t *testing.T) {
dir := t.TempDir()
src := dir + string(os.PathSeparator) + "backdrop.jpg"
data := make([]byte, compactImageSkipBytes+1024)
@@ -310,7 +353,7 @@ func TestServeResizedFromFileDoesNotQueueWhenResizeBusy(t *testing.T) {
if err := os.WriteFile(src, data, 0o644); err != nil {
t.Fatalf("write source: %v", err)
}
proxy := &ImageProxy{cacheDir: dir + string(os.PathSeparator) + "cache"}
proxy := &ImageProxy{cacheDir: dir + string(os.PathSeparator) + "cache", resizeQueueWait: 50 * time.Millisecond}
var releases []func()
for i := 0; i < imageResizeConcurrency(); i++ {
release, ok := proxy.acquireResizeSlot(context.Background())
@@ -327,10 +370,77 @@ func TestServeResizedFromFileDoesNotQueueWhenResizeBusy(t *testing.T) {
start := time.Now()
handled := proxy.serveResizedFromFile(httptest.NewRecorder(), httptest.NewRequest("GET", "/x?maxWidth=480", nil), src, imageResizeOptions{MaxWidth: 480})
if time.Since(start) > 200*time.Millisecond {
t.Fatal("resize slot was busy but the request still waited in the queue")
if elapsed := time.Since(start); elapsed > 500*time.Millisecond {
t.Fatalf("request waited %v with every slot held; the queue wait must be bounded", elapsed)
}
if handled {
t.Fatal("expected a busy resize slot to fall back so the caller can serve the original")
t.Fatal("expected a permanently busy queue to fall back so the caller can serve the original")
}
}
// 槽位只是短暂忙碌时必须排队等待并产出缩略图,而不是直出多兆字节原图。
// 这是线上「海报墙每张图 20 多秒」的根因:并发加载时槽位全忙,旧逻辑把
// 1MB 级 PNG 原样下发。
// encodeNoisyPNG 生成难以压缩的 PNG:规则色块会被压到 1KB 级别,测不出
// 「大体积 PNG 被原样下发」这条回归。
func encodeNoisyPNG(t *testing.T, w, h int) []byte {
t.Helper()
img := image.NewRGBA(image.Rect(0, 0, w, h))
seed := uint32(0x9e3779b9)
for y := 0; y < h; y++ {
for x := 0; x < w; x++ {
seed = seed*1664525 + 1013904223
img.Set(x, y, color.RGBA{R: uint8(seed >> 24), G: uint8(seed >> 16), B: uint8(seed >> 8), A: 255})
}
}
var buf bytes.Buffer
if err := png.Encode(&buf, img); err != nil {
t.Fatalf("encode noisy png: %v", err)
}
return buf.Bytes()
}
func TestServeResizedFromFileWaitsForSlotInsteadOfServingOriginal(t *testing.T) {
dir := t.TempDir()
src := dir + string(os.PathSeparator) + "poster.png"
original := encodeNoisyPNG(t, 378, 540)
if len(original) <= compactImageSkipBytes {
t.Fatalf("test poster is %d bytes, want a source above the compact threshold", len(original))
}
if err := os.WriteFile(src, original, 0o644); err != nil {
t.Fatalf("write source: %v", err)
}
proxy := &ImageProxy{cacheDir: dir + string(os.PathSeparator) + "cache"}
var releases []func()
for i := 0; i < imageResizeConcurrency(); i++ {
release, ok := proxy.acquireResizeSlot(context.Background())
if !ok {
t.Fatal("expected a resize slot")
}
releases = append(releases, release)
}
go func() {
time.Sleep(40 * time.Millisecond)
releases[0]()
}()
rec := httptest.NewRecorder()
if !proxy.serveResizedFromFile(rec, httptest.NewRequest("GET", "/x?maxWidth=480&maxHeight=600", nil), src, imageResizeOptions{MaxWidth: 480, MaxHeight: 600}) {
t.Fatal("expected the request to wait for the slot and serve a thumbnail")
}
body := rec.Body.Bytes()
if bytes.Equal(body, original) {
t.Fatal("served the multi-hundred-KB PNG original instead of a re-encoded thumbnail")
}
if rec.Header().Get("Content-Type") != "image/jpeg" {
t.Fatalf("content type = %q, want image/jpeg", rec.Header().Get("Content-Type"))
}
// 同一请求第二次必须直接命中缩放缓存。
rec2 := httptest.NewRecorder()
if !proxy.serveResizedFromFile(rec2, httptest.NewRequest("GET", "/x?maxWidth=480&maxHeight=600", nil), src, imageResizeOptions{MaxWidth: 480, MaxHeight: 600}) {
t.Fatal("expected the cached thumbnail to be reused")
}
if !bytes.Equal(rec2.Body.Bytes(), body) {
t.Fatal("expected the cached thumbnail bytes to be identical")
}
}
+564
View File
@@ -0,0 +1,564 @@
package service
import (
"bytes"
"context"
"crypto/sha256"
"encoding/hex"
"errors"
"fmt"
"image"
"image/jpeg"
"io"
"os"
"path/filepath"
"sort"
"strings"
"sync"
"time"
"golang.org/x/image/draw"
"github.com/truewhile/MeBox/internal/model"
"github.com/truewhile/MeBox/internal/repository"
)
// 媒体库封面拼图(Emby 风格的「封面集」)。
//
// Emby / Jellyfin 在没有管理员自定义封面时,会用库内几张海报拼成一张
// 2×2 的合成图作为库主图(folder image)。MeBox 早先只是挑评分最高的
// 那一张单图,于是每个库的封面看起来都像「随便选了一张」。这里按同样的
// 思路生成拼图,供 Emby 客户端、Web 和 App 共用同一份图片。
const (
// libraryCoverDefaultWidth / Height 是拼图输出的默认画布。
//
// 库条目的 PrimaryImageAspectRatio 报的是 16:9(见 emby_system.go),
// 客户端按这个比例摆卡片;生成同样比例的图可以避免客户端再裁一刀。
libraryCoverDefaultWidth = 480
libraryCoverDefaultHeight = 270
libraryCoverMinWidth = 96
libraryCoverMaxWidth = 1920
libraryCoverAspect = 16.0 / 9.0
// libraryCoverSourceLimit 是拼图实际取用的成员海报数上限。
//
// 封面是竖切分栏:三栏在 16:9 画布上每栏接近 3:5,既保留海报主体,又能
// 让一个库有多部作品可辨识;再多就会把海报裁成很窄的竖条。
libraryCoverSourceLimit = 3
// libraryCoverMaxPanels 是合成函数能接受的最多分栏数(更长的来源列表
// 只取前几张)。留 4 是给「按库类型调栏数」留的余量。
libraryCoverMaxPanels = 4
// libraryCoverCandidatePool 是挑选海报时的候选池大小。只看最近的一批
// 成员,避免为一个库封面扫全库;对「最新入库」的库来说这批就是它最好
// 的那几张。
libraryCoverCandidatePool = 200
libraryCoverQuality = 85
libraryCoverSubdir = "library-covers"
)
// libraryCoverSourceCache 缓存「库 → 成员海报来源」。Emby 客户端进入首页
// 时会对每个库各请求一次主图,把这个查询压到每库 10 分钟一次,避免几十个
// 200 行的列表查询同时打进来。
var libraryCoverSourceCache = struct {
mu sync.Mutex
items map[string]libraryCoverSourceEntry
}{items: make(map[string]libraryCoverSourceEntry)}
type libraryCoverSourceEntry struct {
sources []string
expiresAt time.Time
}
const libraryCoverSourceTTL = 10 * time.Minute
func cachedLibraryCoverSources(libraryID string) ([]string, bool) {
libraryCoverSourceCache.mu.Lock()
defer libraryCoverSourceCache.mu.Unlock()
entry, ok := libraryCoverSourceCache.items[libraryID]
if !ok || time.Now().After(entry.expiresAt) {
if ok {
delete(libraryCoverSourceCache.items, libraryID)
}
return nil, false
}
return entry.sources, true
}
func rememberLibraryCoverSources(libraryID string, sources []string) {
libraryCoverSourceCache.mu.Lock()
defer libraryCoverSourceCache.mu.Unlock()
if len(libraryCoverSourceCache.items) > 4000 {
libraryCoverSourceCache.items = make(map[string]libraryCoverSourceEntry)
}
libraryCoverSourceCache.items[libraryID] = libraryCoverSourceEntry{
sources: sources,
expiresAt: time.Now().Add(libraryCoverSourceTTL),
}
}
// LibraryCoverSources 返回媒体库封面拼图要用的海报来源,最多 limit 张。
//
// 选法与网页旧逻辑一致(库内最近一批成员,按 seriesArtworkScore 取质量
// 最高者),额外按剧集去重,避免四张海报全是同一部剧的不同集。
func LibraryCoverSources(ctx context.Context, repos *repository.Container, libraryID string, limit int) []string {
if repos == nil || repos.Media == nil || strings.TrimSpace(libraryID) == "" {
return nil
}
if limit <= 0 || limit > libraryCoverSourceLimit {
limit = libraryCoverSourceLimit
}
if cached, ok := cachedLibraryCoverSources(libraryID); ok {
if len(cached) > limit {
return cached[:limit]
}
return cached
}
rows, err := repos.Media.ListByLibrariesFilteredNoCount(ctx, []string{libraryID}, 0, libraryCoverCandidatePool,
repository.MediaQueryFilter{IncludeNSFW: true})
if err != nil || len(rows) == 0 {
return nil
}
type candidate struct {
url string
key string
score int
}
candidates := make([]candidate, 0, len(rows))
for i := range rows {
row := rows[i]
raw := strings.TrimSpace(row.PosterURL)
if raw == "" {
raw = strings.TrimSpace(row.BackdropURL)
}
if raw == "" {
continue
}
key := mediaSeriesKey(row)
if key == "" {
key = "id:" + row.ID
}
candidates = append(candidates, candidate{url: raw, key: key, score: seriesArtworkScore(row)})
}
// 评分高的优先;同分保持数据库顺序(最近的在前),封面更贴近「最新入库」。
sort.SliceStable(candidates, func(i, j int) bool { return candidates[i].score > candidates[j].score })
out := make([]string, 0, limit)
seenKey := make(map[string]struct{}, limit)
seenURL := make(map[string]struct{}, limit)
for _, item := range candidates {
if len(out) >= limit {
break
}
if _, ok := seenKey[item.key]; ok {
continue
}
if _, ok := seenURL[item.url]; ok {
continue
}
seenKey[item.key] = struct{}{}
seenURL[item.url] = struct{}{}
out = append(out, item.url)
}
if len(out) == 0 {
rememberLibraryCoverSources(libraryID, nil)
return nil
}
rememberLibraryCoverSources(libraryID, out)
return out
}
// LibraryCoverCacheDir 返回拼图缓存目录(与图片代理的 images/ 分开存放,
// 不会被图片缓存的配额清理误删)。
func LibraryCoverCacheDir(cfgCacheDir string) string {
base := strings.TrimSpace(cfgCacheDir)
if base == "" {
base = "./cache"
}
return filepath.Join(base, libraryCoverSubdir)
}
// LibraryCoverImage 生成(或命中缓存)媒体库封面拼图。
//
// width/height 是期望的输出画布;服务端会按 16:9 收敛,返回的图片不会
// 超过该边界。sources 由调用方提供(见 LibraryCoverSources),这样 Emby
// 路由和 Web API 可以共用同一套生成与缓存逻辑。
func (p *ImageProxy) LibraryCoverImage(ctx context.Context, libraryID string, sources []string, width, height int) ([]byte, error) {
if p == nil {
return nil, errors.New("image proxy unavailable")
}
libraryID = strings.TrimSpace(libraryID)
sources = normalizeCoverSources(sources)
if libraryID == "" || len(sources) == 0 {
return nil, errors.New("no library cover sources")
}
outW, outH := libraryCoverCanvas(width, height)
cachePath, sourceKey := p.libraryCoverCachePath(libraryID, sources, outW, outH)
if data, err := os.ReadFile(cachePath); err == nil && len(data) > 0 { // #nosec G304 -- cachePath is derived from a content hash.
return data, nil
}
key := fmt.Sprintf("library-cover:%s-%dx%d-%s", libraryID, outW, outH, sourceKey)
v, err, _ := p.fetchGroup.Do(key, func() (any, error) {
if data, err := os.ReadFile(cachePath); err == nil && len(data) > 0 { // #nosec G304 -- cachePath is derived from a content hash.
return data, nil
}
data, buildErr := p.buildLibraryCover(ctx, libraryID, sources, outW, outH, cachePath, sourceKey)
if buildErr != nil {
return nil, buildErr
}
return data, nil
})
if err != nil {
return nil, err
}
data, _ := v.([]byte)
if len(data) == 0 {
return nil, errors.New("empty library cover")
}
return data, nil
}
func (p *ImageProxy) buildLibraryCover(ctx context.Context, libraryID string, sources []string, width, height int, cachePath, sourceKey string) ([]byte, error) {
images := p.fetchCoverSources(ctx, sources)
if len(images) == 0 {
return nil, errors.New("no library cover image fetched")
}
// 解码与合成是 CPU 密集操作:和缩略图共用同一组解码槽,避免一次首页
// 刷新把 2 核主机打满。
release, ok := p.acquireResizeSlotWithin(ctx, p.resizeQueueTimeout())
if ok {
defer release()
}
data, err := composeLibraryCover(images, width, height)
if err != nil {
return nil, err
}
if err := os.MkdirAll(filepath.Dir(cachePath), 0o750); err != nil {
return data, nil
}
if err := writeFileAtomic(cachePath, data); err != nil {
p.warn("write library cover cache failed", err)
return data, nil
}
p.pruneLibraryCoverVariants(filepath.Dir(cachePath), libraryID, sourceKey)
return data, nil
}
// fetchCoverSources 并发拉取成员海报(受图片代理自身的缓存/鉴权/SSRF 规则
// 保护)。单张失败只丢弃该张,其余仍然可以拼图。
func (p *ImageProxy) fetchCoverSources(ctx context.Context, sources []string) [][]byte {
out := make([][]byte, len(sources))
var wg sync.WaitGroup
sem := make(chan struct{}, libraryCoverMaxPanels)
fetchCtx, cancel := context.WithTimeout(ctx, 15*time.Second)
defer cancel()
for i, raw := range sources {
i, raw := i, raw
wg.Add(1)
go func() {
defer wg.Done()
sem <- struct{}{}
defer func() { <-sem }()
data, err := p.coverSourceBytes(fetchCtx, raw)
if err != nil || len(data) == 0 {
return
}
out[i] = data
}()
}
wg.Wait()
compacted := make([][]byte, 0, len(out))
for _, data := range out {
if len(data) > 0 {
compacted = append(compacted, data)
}
}
return compacted
}
// coverSourceBytes 读取一张海报来源。
//
// 刮削会把 sidecar 海报存成 /media/... 这类绝对路径,而 fetch 路径只接受
// http(s)(validateURL 要求有 host),直接丢给 Fetch 会全部失败。本地路径
// 按图片代理同一套 isAllowedLocalPath 白名单读盘,远程地址才走 Fetch,
// 这样鉴权、SSRF 校验与磁盘缓存的行为与 /api/img 完全一致。
func (p *ImageProxy) coverSourceBytes(ctx context.Context, raw string) ([]byte, error) {
raw = strings.TrimSpace(raw)
if raw == "" {
return nil, errors.New("empty cover source")
}
if isLocalImagePath(raw) {
abs, err := filepath.Abs(filepath.Clean(raw))
if err != nil {
return nil, err
}
if !p.isAllowedLocalPath(abs) {
return nil, errors.New("local cover path not allowed")
}
return os.ReadFile(abs) // #nosec G304 -- path validated against the library roots above.
}
data, _, err := p.Fetch(ctx, raw)
if err != nil {
return nil, err
}
return data, nil
}
// libraryCoverCachePath 生成缓存路径。
//
// 文件名里的 sourceKey 只由成员海报列表决定(不含尺寸):成员变化时换一批
// 文件,同一批成员的多个尺寸变体可以共存,不会被互相删掉。size 单独进文件名,
// 让 Emby(按客户端 maxWidth/maxHeight)与 Web/App(固定档位)各用自己的成品。
func (p *ImageProxy) libraryCoverCachePath(libraryID string, sources []string, width, height int) (string, string) {
h := sha256.New()
for _, src := range sources {
_, _ = io.WriteString(h, src)
_, _ = h.Write([]byte{0})
}
sourceKey := hex.EncodeToString(h.Sum(nil))[:16]
name := fmt.Sprintf("%s-%dx%d-%s.jpg", safeCoverFileID(libraryID), width, height, sourceKey)
return filepath.Join(LibraryCoverCacheDir(p.cacheDirBase()), name), sourceKey
}
func (p *ImageProxy) cacheDirBase() string {
if p != nil && p.cfg != nil {
return p.cfg.Cache.CacheDir
}
return ""
}
// pruneLibraryCoverVariants 清掉同一个库、旧成员列表留下的封面文件(含各
// 尺寸变体),当前成员列表对应的成品全部保留。
func (p *ImageProxy) pruneLibraryCoverVariants(dir, libraryID, keepSourceKey string) {
entries, err := os.ReadDir(dir)
if err != nil {
return
}
prefix := safeCoverFileID(libraryID) + "-"
for _, entry := range entries {
name := entry.Name()
if entry.IsDir() || !strings.HasPrefix(name, prefix) || !strings.HasSuffix(name, ".jpg") {
continue
}
if strings.Contains(name, "-"+keepSourceKey+".jpg") {
continue
}
_ = os.Remove(filepath.Join(dir, name)) // #nosec G304 -- name comes from ReadDir of our own cache dir.
}
}
// safeCoverFileID 让文件名只包含安全字符(库 ID 来自数据库,但仍做一次
// 白名单过滤,避免任何路径分隔符进入文件名)。
func safeCoverFileID(libraryID string) string {
var b strings.Builder
for _, r := range libraryID {
switch {
case r >= 'a' && r <= 'z', r >= 'A' && r <= 'Z', r >= '0' && r <= '9', r == '-', r == '_':
b.WriteRune(r)
}
}
if b.Len() == 0 {
return "library"
}
return b.String()
}
func normalizeCoverSources(sources []string) []string {
seen := make(map[string]struct{}, len(sources))
out := make([]string, 0, len(sources))
for _, raw := range sources {
raw = strings.TrimSpace(raw)
if raw == "" {
continue
}
if _, ok := seen[raw]; ok {
continue
}
seen[raw] = struct{}{}
out = append(out, raw)
if len(out) >= libraryCoverMaxPanels {
break
}
}
return out
}
// libraryCoverCanvas 把请求的尺寸收敛到 16:9 画布,并夹在合理区间内。
func libraryCoverCanvas(width, height int) (int, int) {
if width <= 0 && height <= 0 {
return libraryCoverDefaultWidth, libraryCoverDefaultHeight
}
if width <= 0 {
width = int(float64(height)*libraryCoverAspect + 0.5)
}
if height <= 0 {
height = int(float64(width)/libraryCoverAspect + 0.5)
}
// 客户端给的是一个「不超过」的盒子:按 16:9 装进去。
if float64(width)/float64(height) > libraryCoverAspect {
width = int(float64(height)*libraryCoverAspect + 0.5)
} else {
height = int(float64(width)/libraryCoverAspect + 0.5)
}
if width < libraryCoverMinWidth {
width = libraryCoverMinWidth
height = int(float64(width)/libraryCoverAspect + 0.5)
}
if width > libraryCoverMaxWidth {
width = libraryCoverMaxWidth
height = int(float64(width)/libraryCoverAspect + 0.5)
}
return width, height
}
// composeLibraryCover 把若干张海报拼成库封面:竖切成若干等宽竖条。
//
// - 1 张:铺满整块画布;
// - 2 张:左右各一半(默认形态,对应网页/客户端卡片的分屏观感);
// - 3/4 张:三等分 / 四等分竖条。
//
// 每格按中心裁切铺满(crop-to-fill),这样不同比例的海报都能对齐分栏。
func composeLibraryCover(images [][]byte, width, height int) ([]byte, error) {
if len(images) == 0 {
return nil, errors.New("no images to compose")
}
if width <= 0 || height <= 0 {
return nil, errors.New("invalid cover size")
}
decoded := make([]image.Image, 0, len(images))
for _, data := range images {
img, ok := decodeCoverImage(data)
if ok {
decoded = append(decoded, img)
}
if len(decoded) >= libraryCoverMaxPanels {
break
}
}
if len(decoded) == 0 {
return nil, errors.New("no decodable cover images")
}
canvas := image.NewRGBA(image.Rect(0, 0, width, height))
cells := make([]image.Rectangle, 0, len(decoded))
if len(decoded) == 1 {
cells = append(cells, canvas.Bounds())
} else {
for i := range decoded {
x0 := width * i / len(decoded)
x1 := width * (i + 1) / len(decoded)
cells = append(cells, image.Rect(x0, 0, x1, height))
}
}
for i, cell := range cells {
drawCoverCell(canvas, cell, decoded[i])
}
var buf bytes.Buffer
if err := jpeg.Encode(&buf, canvas, &jpeg.Options{Quality: libraryCoverQuality}); err != nil {
return nil, err
}
return buf.Bytes(), nil
}
// decodeCoverImage 解码一张海报,并对异常大的图做像素上限保护:一次拼图
// 会同时解码 4 张,2 核 2G 的机器上不能任由超大原图把内存吃光。
func decodeCoverImage(data []byte) (image.Image, bool) {
if len(data) == 0 {
return nil, false
}
cfg, _, err := image.DecodeConfig(bytes.NewReader(data))
if err != nil || cfg.Width <= 0 || cfg.Height <= 0 {
return nil, false
}
if cfg.Width*cfg.Height > imageResizeMaxSourcePixels {
return nil, false
}
img, _, err := image.Decode(bytes.NewReader(data))
if err != nil {
return nil, false
}
return img, true
}
// drawCoverCell 把一张图按中心裁切铺满目标格子。
func drawCoverCell(dst *image.RGBA, cell image.Rectangle, src image.Image) {
if src == nil || cell.Dx() <= 0 || cell.Dy() <= 0 {
return
}
bounds := src.Bounds()
srcW, srcH := bounds.Dx(), bounds.Dy()
if srcW <= 0 || srcH <= 0 {
return
}
targetAspect := float64(cell.Dx()) / float64(cell.Dy())
srcAspect := float64(srcW) / float64(srcH)
crop := bounds
if srcAspect > targetAspect {
cropW := int(float64(srcH)*targetAspect + 0.5)
if cropW < 1 {
cropW = 1
}
x0 := bounds.Min.X + (srcW-cropW)/2
crop = image.Rect(x0, bounds.Min.Y, x0+cropW, bounds.Max.Y)
} else if srcAspect < targetAspect {
cropH := int(float64(srcW)/targetAspect + 0.5)
if cropH < 1 {
cropH = 1
}
y0 := bounds.Min.Y + (srcH-cropH)/2
crop = image.Rect(bounds.Min.X, y0, bounds.Max.X, y0+cropH)
}
scale := float64(cell.Dx()) / float64(crop.Dx())
var scaler draw.Scaler = draw.CatmullRom
if scale < imageResizeCheapScaleRatio {
scaler = draw.ApproxBiLinear
}
scaler.Scale(dst, cell, src, crop, draw.Src, nil)
}
// writeFileAtomic 先写临时文件再 rename,避免并发的读取方读到半个文件。
func writeFileAtomic(path string, data []byte) error {
dir := filepath.Dir(path)
if err := os.MkdirAll(dir, 0o750); err != nil {
return err
}
tmp, err := os.CreateTemp(dir, ".cover-*")
if err != nil {
return err
}
if _, err := tmp.Write(data); err != nil {
_ = tmp.Close()
_ = os.Remove(tmp.Name())
return err
}
if err := tmp.Close(); err != nil {
_ = os.Remove(tmp.Name())
return err
}
if err := os.Rename(tmp.Name(), path); err != nil {
_ = os.Remove(tmp.Name())
return err
}
return nil
}
// LibraryCoverSourcesForLibrary 是给 handler 的便捷封装:库存在且没有自定
// 义封面时返回成员海报来源;远程挂载库由各自的上游负责封面,不在此生成。
func LibraryCoverSourcesForLibrary(ctx context.Context, repos *repository.Container, libraryID string) ([]string, *model.Library) {
if repos == nil || repos.Library == nil {
return nil, nil
}
libraryID = strings.TrimSpace(libraryID)
if libraryID == "" {
return nil, nil
}
lib, err := repos.Library.FindByID(ctx, libraryID)
if err != nil || lib == nil {
return nil, nil
}
if strings.TrimSpace(lib.CoverURL) != "" {
return nil, lib
}
return LibraryCoverSources(ctx, repos, libraryID, libraryCoverSourceLimit), lib
}
+335
View File
@@ -0,0 +1,335 @@
package service
import (
"bytes"
"image"
"image/color"
"image/jpeg"
"net/http"
"net/http/httptest"
"net/url"
"os"
"path/filepath"
"sync/atomic"
"testing"
"go.uber.org/zap"
"github.com/truewhile/MeBox/internal/config"
"github.com/truewhile/MeBox/internal/model"
)
func solidJPEG(t *testing.T, c color.Color, w, h int) []byte {
t.Helper()
img := image.NewRGBA(image.Rect(0, 0, w, h))
for y := 0; y < h; y++ {
for x := 0; x < w; x++ {
img.Set(x, y, c)
}
}
var buf bytes.Buffer
if err := jpeg.Encode(&buf, img, &jpeg.Options{Quality: 95}); err != nil {
t.Fatalf("encode jpeg: %v", err)
}
return buf.Bytes()
}
func decodeJPEG(t *testing.T, data []byte) image.Image {
t.Helper()
img, err := jpeg.Decode(bytes.NewReader(data))
if err != nil {
t.Fatalf("decode composed cover: %v", err)
}
return img
}
// TestComposeLibraryCoverSplitsIntoSideBySideColumns 保证多张海报是竖切成
// 左右分栏(左→右顺序),而不是 2×2 田字格。
func TestComposeLibraryCoverSplitsIntoSideBySideColumns(t *testing.T) {
images := [][]byte{
solidJPEG(t, color.RGBA{R: 220, A: 255}, 200, 300),
solidJPEG(t, color.RGBA{G: 220, A: 255}, 200, 300),
solidJPEG(t, color.RGBA{B: 220, A: 255}, 200, 300),
solidJPEG(t, color.RGBA{R: 220, G: 220, A: 255}, 200, 300),
}
out, err := composeLibraryCover(images, 480, 270)
if err != nil {
t.Fatalf("compose: %v", err)
}
img := decodeJPEG(t, out)
if img.Bounds().Dx() != 480 || img.Bounds().Dy() != 270 {
t.Fatalf("size = %v, want 480x270", img.Bounds())
}
// 四张图 → 四条竖栏;同一栏从顶部到底部颜色一致(竖切,不是上下分割)。
y := img.Bounds().Dy() / 2
checks := []struct {
name string
x int
dominant func(color.RGBA) bool
}{
{"column 1 red", 60, func(c color.RGBA) bool { return c.R > c.G+40 && c.R > c.B+40 }},
{"column 2 green", 180, func(c color.RGBA) bool { return c.G > c.R+40 && c.G > c.B+40 }},
{"column 3 blue", 300, func(c color.RGBA) bool { return c.B > c.R+40 && c.B > c.G+40 }},
{"column 4 yellow", 420, func(c color.RGBA) bool { return c.R > c.B+40 && c.G > c.B+40 }},
}
for _, check := range checks {
for _, yy := range []int{2, y, img.Bounds().Dy() - 3} {
r, g, b, _ := img.At(check.x, yy).RGBA()
got := color.RGBA{R: uint8(r >> 8), G: uint8(g >> 8), B: uint8(b >> 8), A: 255}
if !check.dominant(got) {
t.Fatalf("%s at y=%d: pixel = %v", check.name, yy, got)
}
}
}
}
// TestComposeLibraryCoverHandlesFewerPosters 覆盖 1/2/3 张成员的布局。
func TestComposeLibraryCoverHandlesFewerPosters(t *testing.T) {
cases := []struct {
name string
count int
}{
{"single", 1},
{"pair", 2},
{"triple", 3},
}
for _, tc := range cases {
t.Run(tc.name, func(t *testing.T) {
images := make([][]byte, 0, tc.count)
for i := 0; i < tc.count; i++ {
images = append(images, solidJPEG(t, color.RGBA{R: 200, G: 120, A: 255}, 300, 450))
}
out, err := composeLibraryCover(images, 320, 180)
if err != nil {
t.Fatalf("compose: %v", err)
}
img := decodeJPEG(t, out)
if img.Bounds().Dx() != 320 || img.Bounds().Dy() != 180 {
t.Fatalf("size = %v, want 320x180", img.Bounds())
}
})
}
}
// TestComposeLibraryCoverSkipsUndecodableSources 保证一张坏图不会让整张
// 封面失败:其余可解码的成员仍然会拼出来。
func TestComposeLibraryCoverSkipsUndecodableSources(t *testing.T) {
images := [][]byte{
[]byte("not an image"),
solidJPEG(t, color.RGBA{G: 200, A: 255}, 200, 300),
}
out, err := composeLibraryCover(images, 240, 135)
if err != nil {
t.Fatalf("compose: %v", err)
}
if len(out) == 0 {
t.Fatal("expected composed cover bytes")
}
if _, err := composeLibraryCover([][]byte{[]byte("nope")}, 240, 135); err == nil {
t.Fatal("expected error when no source is decodable")
}
}
func TestLibraryCoverCanvasKeepsSixteenByNine(t *testing.T) {
cases := []struct {
inW, inH int
wantW int
wantH int
checkRatio bool
}{
{0, 0, 480, 270, true},
{640, 480, 640, 360, true}, // 4:3 盒子 → 装进 16:9
{2000, 0, 1920, 1080, true}, // 超上限 → 夹紧
{40, 0, 96, 54, true}, // 低于下限 → 抬到最小宽度
{0, 540, 960, 540, false}, // 只给高度 → 按 16:9 推宽度
}
for _, tc := range cases {
w, h := libraryCoverCanvas(tc.inW, tc.inH)
if w != tc.wantW || h != tc.wantH {
t.Fatalf("libraryCoverCanvas(%d,%d) = %dx%d, want %dx%d", tc.inW, tc.inH, w, h, tc.wantW, tc.wantH)
}
if tc.checkRatio && w != 0 && h != 0 {
ratio := float64(w) / float64(h)
if ratio < 1.7 || ratio > 1.85 {
t.Fatalf("ratio for %dx%d = %.3f, want ~1.78", tc.inW, tc.inH, ratio)
}
}
}
}
// TestLibraryCoverSourcesDedupesSeries 保证封面不会两张都是同一部剧的不同集,
// 并且优先取海报、没有海报时退回剧照、完全没有图的成员被跳过。
func TestLibraryCoverSourcesDedupesSeries(t *testing.T) {
svc := newTestEmbyService(t)
lib := model.Library{Name: "剧集", Path: "/media/tv", Type: "tv", Enabled: true}
if err := svc.repo.Library.Create(t.Context(), &lib); err != nil {
t.Fatalf("create library: %v", err)
}
rows := []model.Media{
{Base: model.Base{ID: "a1"}, LibraryID: lib.ID, Title: "A", Path: "/media/tv/A/S01E01.mkv", SeriesID: "series-a", PosterURL: "https://img/a1.jpg"},
{Base: model.Base{ID: "a2"}, LibraryID: lib.ID, Title: "A", Path: "/media/tv/A/S01E02.mkv", SeriesID: "series-a", PosterURL: "https://img/a2.jpg"},
{Base: model.Base{ID: "b1"}, LibraryID: lib.ID, Title: "B", Path: "/media/tv/B/S01E01.mkv", SeriesID: "series-b", PosterURL: "https://img/b1.jpg"},
{Base: model.Base{ID: "c1"}, LibraryID: lib.ID, Title: "C", Path: "/media/tv/C/S01E01.mkv", SeriesID: "series-c", BackdropURL: "https://img/c1-backdrop.jpg"},
{Base: model.Base{ID: "d1"}, LibraryID: lib.ID, Title: "D", Path: "/media/tv/D/S01E01.mkv", SeriesID: "series-d"},
}
for i := range rows {
if err := svc.repo.DB.Create(&rows[i]).Error; err != nil {
t.Fatalf("create media: %v", err)
}
}
// 竖切三栏:一部剧只出一张,无图成员被跳过 → 正好 3 张。
sources := LibraryCoverSources(t.Context(), svc.repo, lib.ID, libraryCoverSourceLimit)
if len(sources) != 3 {
t.Fatalf("sources = %v, want 3 (一部剧一张、跳过无图成员)", sources)
}
seen := map[string]bool{}
for _, src := range sources {
if seen[src] {
t.Fatalf("duplicate source in %v", sources)
}
seen[src] = true
}
// 同一部剧只会出现一次:具体保留哪一集由查询顺序决定,但绝不能两张。
if seen["https://img/a1.jpg"] && seen["https://img/a2.jpg"] {
t.Fatalf("same series contributed two posters: %v", sources)
}
if !seen["https://img/a1.jpg"] && !seen["https://img/a2.jpg"] {
t.Fatalf("series A is missing from sources: %v", sources)
}
if !seen["https://img/c1-backdrop.jpg"] {
t.Fatalf("member without poster should fall back to backdrop: %v", sources)
}
}
// TestLibraryCoverImageCachesComposedBytes 验证生成结果会落盘复用:同一份
// 内容第二次请求不再回源。
func TestLibraryCoverImageCachesComposedBytes(t *testing.T) {
var hits int32
upstream := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
atomic.AddInt32(&hits, 1)
w.Header().Set("Content-Type", "image/jpeg")
_, _ = w.Write(solidJPEG(t, color.RGBA{R: 180, G: 140, B: 90, A: 255}, 240, 360))
}))
defer upstream.Close()
parsed, err := url.Parse(upstream.URL)
if err != nil {
t.Fatalf("parse upstream: %v", err)
}
cacheCfg := &config.Config{Cache: config.CacheConfig{CacheDir: filepath.Join(t.TempDir(), "cache")}}
proxy := NewImageProxy(cacheCfg, zap.NewNop())
proxy.SetAllowedRemoteHostsProvider(func() []string { return []string{parsed.Host} })
sources := []string{upstream.URL + "/1.jpg", upstream.URL + "/2.jpg", upstream.URL + "/3.jpg", upstream.URL + "/4.jpg"}
first, err := proxy.LibraryCoverImage(t.Context(), "lib-1", sources, 480, 270)
if err != nil {
t.Fatalf("first cover: %v", err)
}
if img := decodeJPEG(t, first); img.Bounds().Dx() != 480 || img.Bounds().Dy() != 270 {
t.Fatalf("size = %v, want 480x270", img.Bounds())
}
hitsAfterFirst := atomic.LoadInt32(&hits)
if hitsAfterFirst != 4 {
t.Fatalf("upstream hits = %d, want one per source", hitsAfterFirst)
}
second, err := proxy.LibraryCoverImage(t.Context(), "lib-1", sources, 480, 270)
if err != nil {
t.Fatalf("second cover: %v", err)
}
if !bytes.Equal(first, second) {
t.Fatal("cached cover bytes differ from the first build")
}
if got := atomic.LoadInt32(&hits); got != hitsAfterFirst {
t.Fatalf("cache miss: upstream hits %d -> %d", hitsAfterFirst, got)
}
// 内容变了要重新生成(成员海报换了新文件)。已缓存的三张海报不会重下,
// 只有新增的那张会回源。
changed := append([]string{}, sources[:3]...)
changed = append(changed, upstream.URL+"/5.jpg")
if _, err := proxy.LibraryCoverImage(t.Context(), "lib-1", changed, 480, 270); err != nil {
t.Fatalf("cover after content change: %v", err)
}
if got := atomic.LoadInt32(&hits); got != hitsAfterFirst+1 {
t.Fatalf("content change should rebuild with one new fetch, hits = %d", got)
}
// 同一批成员的另一个尺寸档位要能共存:Web/App 与 Emby 请求的尺寸不同,
// 不能互相把对方的成品当「旧内容」删掉。
if _, err := proxy.LibraryCoverImage(t.Context(), "lib-1", changed, 400, 225); err != nil {
t.Fatalf("other size cover: %v", err)
}
entries, err := os.ReadDir(LibraryCoverCacheDir(cacheCfg.Cache.CacheDir))
if err != nil {
t.Fatalf("read cover cache dir: %v", err)
}
files := make([]string, 0, len(entries))
for _, entry := range entries {
if !entry.IsDir() {
files = append(files, entry.Name())
}
}
if len(files) != 2 {
t.Fatalf("expected 480x270 and 400x225 variants to coexist, got %v", files)
}
}
// TestLibraryCoverSourceBytesReadsLocalSidecarPoster 覆盖刮削把海报存成
// /media/... 本地绝对路径的情况:这类来源没有 http(s) host,不能走 Fetch,
// 必须按库根目录白名单直接读盘,否则整张拼图会因为「没有可解码的海报」而
// 生成失败(生产环境所有本地库都是这种来源)。
func TestLibraryCoverSourceBytesReadsLocalSidecarPoster(t *testing.T) {
libDir := t.TempDir()
posterPath := filepath.Join(libDir, "poster.jpg")
if err := os.WriteFile(posterPath, solidJPEG(t, color.RGBA{R: 180, G: 90, A: 255}, 200, 300), 0o644); err != nil {
t.Fatalf("write poster: %v", err)
}
proxy := NewImageProxy(&config.Config{Cache: config.CacheConfig{CacheDir: filepath.Join(t.TempDir(), "cache")}}, zap.NewNop())
// 未登记为库根目录时必须拒绝读取。
if _, err := proxy.coverSourceBytes(t.Context(), posterPath); err == nil {
t.Fatal("expected local path outside the library roots to be rejected")
}
proxy.SetLibraryRootsProvider(func() []string { return []string{libDir} })
data, err := proxy.coverSourceBytes(t.Context(), posterPath)
if err != nil {
t.Fatalf("read local poster: %v", err)
}
if len(data) == 0 {
t.Fatal("expected poster bytes")
}
if _, ok := decodeCoverImage(data); !ok {
t.Fatal("local poster should decode")
}
}
// TestLibraryCoverSourcesForLibrarySkipsCustomCover 保证管理员填了自定义
// 封面时不会再生成拼图(沿用自定义图)。
func TestLibraryCoverSourcesForLibrarySkipsCustomCover(t *testing.T) {
svc := newTestEmbyService(t)
lib := model.Library{Name: "电影", Path: "/media/movies", Type: "movie", CoverURL: "https://example.com/custom.jpg"}
if err := svc.repo.Library.Create(t.Context(), &lib); err != nil {
t.Fatalf("create library: %v", err)
}
if err := svc.repo.DB.Create(&model.Media{
Base: model.Base{ID: "m1"},
LibraryID: lib.ID,
Title: "M",
Path: "/media/movies/m1.mkv",
PosterURL: "https://img/m1.jpg",
}).Error; err != nil {
t.Fatalf("create media: %v", err)
}
sources, found := LibraryCoverSourcesForLibrary(t.Context(), svc.repo, lib.ID)
if found == nil || found.CoverURL != lib.CoverURL {
t.Fatalf("library = %#v, want the existing library", found)
}
if len(sources) != 0 {
t.Fatalf("expected no generated sources with a custom cover, got %v", sources)
}
if _, blank := LibraryCoverSourcesForLibrary(t.Context(), svc.repo, "missing"); blank != nil {
t.Fatalf("unknown library should return nil, got %#v", blank)
}
}
-7
View File
@@ -126,13 +126,6 @@ func ApplyRuntimeSetting(cfg *config.Config, key, value string) {
}
cfg.Cache.ImagesOriginalsMaxSizeMB = n
}
case "cache.images_originals_ttl_hours":
if n, err := strconv.Atoi(value); err == nil {
if n < 0 {
n = 0
}
cfg.Cache.ImagesOriginalsTTLHours = n
}
case "cache.memory_max_size_mb":
n := config.DefaultCacheMemoryMaxSizeMB
if value != "" {
+1 -1
View File
@@ -80,10 +80,10 @@ func (s *SchedulerService) SetExpiryWatcher(watcher *TelegramExpiryWatcher) {
}
// ImageCachePolicy 是一次图片缓存清理要用的策略(全部为 0 表示不做任何清理)。
// 只有容量配额,没有保留时长:淘汰只看占用,不按文件年龄删除。
type ImageCachePolicy struct {
TotalBytes int64
OriginalsBytes int64
OriginalsAge time.Duration
}
func (s *SchedulerService) SetImageCachePolicyProvider(fn func() ImageCachePolicy) {
+4 -4
View File
@@ -299,18 +299,18 @@ func (s *SchedulerService) jobCleanTranscodeCache(ctx context.Context) error {
return walkAndPrune(s.cacheDir+"/hls", cutoff)
}
// jobCleanImageCache prunes the image proxy cache: 原图按保留时长与独立配额
// 优先淘汰,总量超限时再淘汰派生成品。
// jobCleanImageCache prunes the image proxy cache: 原图按独立配额优先淘汰,
// 总量超限时再淘汰派生成品。全程只看占用,不按文件年龄删除。
func (s *SchedulerService) jobCleanImageCache(ctx context.Context) error {
if s.cacheDir == "" {
return nil
}
policy := s.imageCachePolicy()
if policy.TotalBytes <= 0 && policy.OriginalsBytes <= 0 && policy.OriginalsAge <= 0 {
if policy.TotalBytes <= 0 && policy.OriginalsBytes <= 0 {
return nil
}
imagesDir := filepath.Join(s.cacheDir, "images")
pools := ImageCachePools(imagesDir, policy.OriginalsBytes, policy.OriginalsAge)
pools := ImageCachePools(imagesDir, policy.OriginalsBytes)
res, err := PruneImageCachePools(pools, policy.TotalBytes)
if err != nil {
if s.log != nil {
-4
View File
@@ -4,7 +4,6 @@ import (
"context"
"errors"
"strings"
"time"
"go.uber.org/zap"
@@ -222,9 +221,6 @@ func (b *serviceContainerBuilder) initAccessAndStorageServices() {
if b.cfg.Cache.ImagesOriginalsMaxSizeMB > 0 {
policy.OriginalsBytes = int64(b.cfg.Cache.ImagesOriginalsMaxSizeMB) * 1024 * 1024
}
if b.cfg.Cache.ImagesOriginalsTTLHours > 0 {
policy.OriginalsAge = time.Duration(b.cfg.Cache.ImagesOriginalsTTLHours) * time.Hour
}
return policy
})
// 阅读正文缓存清理:由阅读模块自己按 TTL + 容量 LRU 淘汰(见 reader_content_cache.go)。
+29 -11
View File
@@ -17,7 +17,8 @@ import (
// Optional query `start` (seconds) restarts ffmpeg from that source offset so
// the web player can scrub the full timeline without waiting for a full
// head-to-tail transcode.
func (s *StreamService) ServeHLSPlaylist(w http.ResponseWriter, r *http.Request, mediaID string) error {
func (s *StreamService) ServeHLSPlaylist(w http.ResponseWriter, r *http.Request, mediaID string, scopes ...HLSPlaybackScope) error {
jobKey := hlsPlaybackJobKey(mediaID, scopes)
// 「客户端直连解码」模式下宿主机不提供转码,HLS 一律拒绝,
// 迫使播放器走 direct play 本地解码。
if s.directPlayOnly(r.Context()) {
@@ -27,19 +28,23 @@ func (s *StreamService) ServeHLSPlaylist(w http.ResponseWriter, r *http.Request,
seekGen := parseHLSSeekGen(r)
subtitleStream := parseHLSSubtitleStream(r)
quality := parseHLSQuality(r)
if _, err := s.transcoder.EnsureJobFromSubtitleQuality(r.Context(), mediaID, startSec, seekGen, subtitleStream, quality); err != nil {
if _, err := s.transcoder.EnsureJobFromSubtitleQuality(r.Context(), mediaID, startSec, seekGen, subtitleStream, quality, scopes...); err != nil {
return err
}
s.transcoder.TouchJob(mediaID)
s.transcoder.TouchJob(jobKey)
readyTimeout := 45 * time.Second
if startSec > 0.05 {
// Mid-file HTTP seeks (esp. WMV) need longer before the first segment appears.
readyTimeout = 120 * time.Second
}
if !s.transcoder.WaitReady(r.Context(), mediaID, readyTimeout) {
if !s.transcoder.WaitReady(r.Context(), jobKey, readyTimeout) {
return errors.New("hls playlist not ready")
}
playlist := s.transcoder.PlaylistPath(mediaID)
outputDir, err := s.transcoder.hlsOutputDir(jobKey, seekGen)
if err != nil {
return err
}
playlist := filepath.Join(outputDir, "index.m3u8")
f, err := os.Open(playlist) // #nosec G304 -- playlist path is generated under the transcoder cache directory for this media ID.
if err != nil {
return err
@@ -49,12 +54,18 @@ func (s *StreamService) ServeHLSPlaylist(w http.ResponseWriter, r *http.Request,
w.Header().Set("Content-Type", "application/vnd.apple.mpegurl")
w.Header().Set("Cache-Control", "no-cache")
w.Header().Set("Content-Disposition", "inline")
if r.URL.RawQuery != "" {
rawQuery := r.URL.RawQuery
if len(scopes) > 0 && scopes[0].SessionID != "" {
query := r.URL.Query()
query.Set("session_id", scopes[0].SessionID)
rawQuery = query.Encode()
}
if rawQuery != "" {
data, err := io.ReadAll(f)
if err != nil {
return err
}
playlist := appendQueryToHLSSegments(string(data), r.URL.RawQuery)
playlist := appendQueryToHLSSegments(string(data), rawQuery)
_, err = io.WriteString(w, playlist)
return err
}
@@ -171,19 +182,26 @@ func filterHLSSegmentQuery(rawQuery string) string {
}
// ServeHLSSegment writes a single .ts segment from the on-disk cache.
func (s *StreamService) ServeHLSSegment(w http.ResponseWriter, r *http.Request, mediaID, segment string) error {
s.transcoder.TouchJob(mediaID)
func (s *StreamService) ServeHLSSegment(w http.ResponseWriter, r *http.Request, mediaID, segment string, scopes ...HLSPlaybackScope) error {
jobKey := hlsPlaybackJobKey(mediaID, scopes)
outputDir, err := s.transcoder.hlsOutputDir(jobKey, parseHLSSeekGen(r))
if err != nil {
if len(scopes) > 0 || parseHLSSeekGen(r) > 0 {
return err
}
outputDir = s.transcoder.HLSDir(jobKey) // Legacy internal static-file callers.
}
// Only allow segments that look like seg_NNNNN.ts so we cannot be tricked
// into reading arbitrary files via path traversal.
if !strings.HasPrefix(segment, "seg_") || !strings.HasSuffix(segment, ".ts") {
return errors.New("bad segment")
}
full := filepath.Join(s.transcoder.HLSDir(mediaID), segment)
full := filepath.Join(outputDir, segment)
abs, err := filepath.Abs(full)
if err != nil {
return err
}
dir, _ := filepath.Abs(s.transcoder.HLSDir(mediaID))
dir, _ := filepath.Abs(outputDir)
if !pathWithin(abs, dir) {
return errors.New("path escape")
}
+89 -32
View File
@@ -1,7 +1,8 @@
// Package service — HLS on-demand transcoder.
//
// TranscoderService spawns ffmpeg processes that segment a source media file
// into HLS (.m3u8 + .ts). The output lives under cache.cache_dir/hls/<id>.
// into HLS (.m3u8 + .ts). Each playback scope owns a cache directory and
// every ffmpeg run writes to its own run-* subdirectory.
// The HTTP layer serves these files directly with a normal http.FileServer.
//
// Encoder selection (read once at startup from the config):
@@ -15,9 +16,11 @@
// plus the kernel module loaded)
//
// Concurrency model:
// - Each Media has at most one active ffmpeg job.
// - jobs[mediaID] tracks the running goroutine + cancel func.
// - Calling Start while a job already exists is a no-op.
// - Each authenticated user/player scope has at most one active ffmpeg job.
// - jobs[jobKey] tracks the running goroutine + cancel func; mediaID remains
// the actual media identity for database access and activity reporting.
// - Legacy internal callers keep media-keyed behaviour.
// - Equivalent requests within one scope reuse the current output.
// - When the playlist file appears on disk we consider the job "ready"
// and unblock the HTTP handler that was waiting on it.
package service
@@ -49,9 +52,10 @@ type TranscoderService struct {
mu sync.Mutex
jobs map[string]*hlsJob
// startGates serializes EnsureJobFrom / StopJob per media so concurrent
// playlist hits cannot spawn multiple ffmpeg writers into one HLS dir.
startGates sync.Map // mediaID -> *sync.Mutex
// Start gates are reference-counted so random playback session keys do not
// accumulate after their last starter/stopper has left.
startGateMu sync.Mutex
startGates map[string]*hlsStartGate
strmResolve func(ctx context.Context, raw string) (*StrmPlayResult, error)
probe *FFprobeService
@@ -66,6 +70,7 @@ type TranscoderService struct {
// hlsJob holds the live state of one ffmpeg run.
type hlsJob struct {
jobKey string
mediaID string
outputDir string
cancel context.CancelFunc
@@ -156,14 +161,19 @@ func (t *TranscoderService) transcodeConfig() *config.Config {
}
}
// HLSDir is the per-media directory that holds index.m3u8 + segment files.
func (t *TranscoderService) HLSDir(mediaID string) string {
return filepath.Join(t.cfg.Cache.CacheDir, "hls", mediaID)
// HLSDir is the cache root for one internal playback job key.
func (t *TranscoderService) HLSDir(jobKey string) string {
return filepath.Join(t.cfg.Cache.CacheDir, "hls", jobKey)
}
// PlaylistPath returns the absolute path of the m3u8 playlist for a media.
func (t *TranscoderService) PlaylistPath(mediaID string) string {
return filepath.Join(t.HLSDir(mediaID), "index.m3u8")
func (t *TranscoderService) PlaylistPath(jobKey string) string {
t.mu.Lock()
defer t.mu.Unlock()
if job := t.jobs[jobKey]; job != nil {
return t.jobPlaylistPath(job)
}
return filepath.Join(t.HLSDir(jobKey), "index.m3u8")
}
// EnsureJob makes sure a transcode is running for mediaID from the start of
@@ -194,7 +204,8 @@ func (t *TranscoderService) EnsureJobFromSubtitle(ctx context.Context, mediaID s
// EnsureJobFromSubtitleQuality is EnsureJobFromSubtitle with an explicit local
// HLS quality profile. Empty qualityID keeps the global transcoder config.
func (t *TranscoderService) EnsureJobFromSubtitleQuality(ctx context.Context, mediaID string, startSec float64, seekGen int64, subtitleStream int, qualityID string) (string, error) {
func (t *TranscoderService) EnsureJobFromSubtitleQuality(ctx context.Context, mediaID string, startSec float64, seekGen int64, subtitleStream int, qualityID string, scopes ...HLSPlaybackScope) (string, error) {
jobKey := hlsPlaybackJobKey(mediaID, scopes)
cfg := t.transcodeConfig()
if !cfg.Transcoder.Enabled {
return "", ErrTranscodeDisabled
@@ -219,28 +230,31 @@ func (t *TranscoderService) EnsureJobFromSubtitleQuality(ctx context.Context, me
return "", ErrMediaNotFound
}
gate := t.mediaStartGate(mediaID)
gate.Lock()
defer gate.Unlock()
unlock := t.lockJobStart(jobKey)
defer unlock()
t.mu.Lock()
if existing, ok := t.jobs[mediaID]; ok {
if existing, ok := t.jobs[jobKey]; ok {
reusable := true
playlist := t.jobPlaylistPath(existing)
if existing.completed {
info, err := os.Stat(t.PlaylistPath(mediaID))
info, err := os.Stat(playlist)
reusable = err == nil && info.Size() > 0
}
if reusable && sameHLSConfiguration(existing, startSec, subtitleStream, qualityID) {
t.touchJobLocked(mediaID)
if seekGen > existing.seekGen {
existing.seekGen = seekGen
}
t.touchJobLocked(jobKey)
t.mu.Unlock()
return t.PlaylistPath(mediaID), nil
return playlist, nil
}
if reusable && !shouldReplaceHLSJobConfigurationQuality(existing, startSec, seekGen, subtitleStream, qualityID) {
t.touchJobLocked(mediaID)
t.touchJobLocked(jobKey)
t.mu.Unlock()
return t.PlaylistPath(mediaID), nil
return playlist, nil
}
prev := t.detachJobLocked(mediaID)
prev := t.detachJobLocked(jobKey)
t.mu.Unlock()
waitJobExit(prev, 12*time.Second)
} else {
@@ -258,6 +272,9 @@ func (t *TranscoderService) EnsureJobFromSubtitleQuality(ctx context.Context, me
input.Quality = quality
t.maybeFillDuration(ctx, m, input)
settings := ffmpegArgSettingsFromConfig(cfg, quality)
// 「原画」档位遇到浏览器可解的 H.264 源时只换封装,避免慢机器上重编码
// 追不上播放(两核软件编码实测 0.1~0.2 倍速)。其它档位要改分辨率,仍重编码。
input.CopyVideo = t.remuxEligible(ctx, m, quality, subtitleStream)
jobEncoder := settings.encoder
if settings.noScale || subtitleStream >= 0 {
jobEncoder = ""
@@ -267,12 +284,23 @@ func (t *TranscoderService) EnsureJobFromSubtitleQuality(ctx context.Context, me
return "", err
}
outDir := t.HLSDir(mediaID)
// Wipe prior segments so a mid-file restart cannot serve stale early chunks.
// Only safe after the previous ffmpeg has exited (waited above / via gate).
if err := resetHLSDir(outDir); err != nil {
rootDir := t.HLSDir(jobKey)
// Runs use distinct directories even when Windows keeps an old segment
// open. A delayed request can never read a newer run's seg_00000.ts.
if err := resetHLSDir(rootDir); err != nil {
return "", err
}
outDir, err := os.MkdirTemp(rootDir, "run-")
if err != nil {
return "", err
}
registered := false
defer func() {
if !registered {
_ = os.RemoveAll(outDir)
_ = os.Remove(rootDir)
}
}()
t.mu.Lock()
if max := t.maxConcurrent(); max > 0 && t.runningJobCountLocked() >= max {
@@ -287,7 +315,11 @@ func (t *TranscoderService) EnsureJobFromSubtitleQuality(ctx context.Context, me
if subtitleStream >= 0 {
jobEncoder = "software (subtitle burn-in)"
}
if input.CopyVideo {
jobEncoder = remuxEncoderLabel
}
job := &hlsJob{
jobKey: jobKey,
mediaID: mediaID,
outputDir: outDir,
cancel: cancel,
@@ -302,7 +334,8 @@ func (t *TranscoderService) EnsureJobFromSubtitleQuality(ctx context.Context, me
ffmpegPath: bin,
config: cfg,
}
t.jobs[mediaID] = job
t.jobs[jobKey] = job
registered = true
t.mu.Unlock()
helper.Go(t.log, "transcoder.monitorIdle", func() { t.monitorIdle(jobCtx, job) })
@@ -310,7 +343,7 @@ func (t *TranscoderService) EnsureJobFromSubtitleQuality(ctx context.Context, me
defer close(job.done)
t.runFFmpeg(jobCtx, job, input)
})
return t.PlaylistPath(mediaID), nil
return filepath.Join(outDir, "index.m3u8"), nil
}
// shouldReplaceHLSJob reports whether an incoming playlist request may cancel
@@ -357,9 +390,33 @@ func sameHLSConfiguration(existing *hlsJob, startSec float64, subtitleStream int
sameHLSStart(existing.startSec, startSec)
}
func (t *TranscoderService) mediaStartGate(mediaID string) *sync.Mutex {
v, _ := t.startGates.LoadOrStore(mediaID, &sync.Mutex{})
return v.(*sync.Mutex)
type hlsStartGate struct {
mu sync.Mutex
users int
}
func (t *TranscoderService) lockJobStart(key string) func() {
t.startGateMu.Lock()
if t.startGates == nil {
t.startGates = make(map[string]*hlsStartGate)
}
gate := t.startGates[key]
if gate == nil {
gate = &hlsStartGate{}
t.startGates[key] = gate
}
gate.users++
t.startGateMu.Unlock()
gate.mu.Lock()
return func() {
gate.mu.Unlock()
t.startGateMu.Lock()
gate.users--
if gate.users == 0 {
delete(t.startGates, key)
}
t.startGateMu.Unlock()
}
}
// detachJobLocked cancels and removes a job from the map without waiting.
+33 -4
View File
@@ -16,6 +16,9 @@ type transcodeInput struct {
StartSec float64
SubtitleStream *int
Quality *LocalHLSQuality
// CopyVideo 为 true 时视频流只换封装(-c:v copy),不重编码。
// 仅当源视频是浏览器可解的老三样(H.264/8bit/4:2:0)且不需要缩放时启用。
CopyVideo bool
}
type ffmpegArgSettings struct {
@@ -52,6 +55,7 @@ func buildFFmpegArgsForInput(cfg *config.Config, input transcodeInput, playlist,
// 原画档位不做硬件缩放的兼容处理,直接走软件编码,保持源分辨率。
settings.encoder = ""
}
copyVideo := input.CopyVideo && input.SubtitleStream == nil
video := ffmpegVideoPlanForSettings(settings)
if input.SubtitleStream != nil {
// Bitmap subtitles must be composited in software. Keeping CUDA/QSV/
@@ -60,13 +64,17 @@ func buildFFmpegArgsForInput(cfg *config.Config, input transcodeInput, playlist,
settings.encoder = ""
video = ffmpegVideoPlanForSettings(settings)
}
if copyVideo {
// 直通:不做缩放、不挂滤镜、不选编码器,整条视频流原样塞进 TS。
video = ffmpegVideoPlan{codec: "copy"}
}
// Mid-file restarts must not use -re: output -ss would otherwise crawl to the
// seek point at 1x wall-clock before emitting the first HLS segment.
realtime := settings.realtime && input.StartSec <= 0.05
args := baseFFmpegArgs(video.preInput, realtime)
args = appendInputAndVideoArgs(args, input, settings, video)
args = appendOutputHLSArgs(args, settings, segments, playlist)
args = appendInputAndVideoArgs(args, input, settings, video, copyVideo)
args = appendOutputHLSArgs(args, settings, segments, playlist, copyVideo)
return args
}
@@ -161,7 +169,7 @@ func baseFFmpegArgs(preInput string, realtime bool) []string {
return args
}
func appendInputAndVideoArgs(args []string, input transcodeInput, settings ffmpegArgSettings, video ffmpegVideoPlan) []string {
func appendInputAndVideoArgs(args []string, input transcodeInput, settings ffmpegArgSettings, video ffmpegVideoPlan, copyVideo bool) []string {
args = append(args, ffmpegHTTPInputArgs(input)...)
ss := ""
if input.StartSec > 0.05 {
@@ -172,6 +180,11 @@ func appendInputAndVideoArgs(args []string, input transcodeInput, settings ffmpe
// scrubbing; CDNs with Range support jump via demuxer seek instead.
if ss != "" {
args = append(args, "-ss", ss)
if copyVideo {
// 复制视频流时 ffmpeg 不能解码,精确 seek 会把关键帧之前的分组丢掉,
// 输出从 P 帧开始(花屏)。退到前一个关键帧,最多提前一两秒。
args = append(args, "-noaccurate_seek")
}
}
args = append(args, "-i", input.Source)
if input.SubtitleStream != nil {
@@ -197,7 +210,23 @@ func appendInputAndVideoArgs(args []string, input transcodeInput, settings ffmpe
return args
}
func appendOutputHLSArgs(args []string, settings ffmpegArgSettings, segments, playlist string) []string {
func appendOutputHLSArgs(args []string, settings ffmpegArgSettings, segments, playlist string, copyVideo bool) []string {
if copyVideo {
// 视频是复制进来的,编码参数一律不适用;只保留音频转 AAC 与切片参数。
// -avoid_negative_ts 让带 B 帧的源在 -ss 之后不会出现负时间戳。
return append(args,
"-avoid_negative_ts", "make_zero",
"-c:a", "aac",
"-ar", "48000",
"-b:a", "128k",
"-ac", "2",
"-f", "hls",
"-hls_time", fmt.Sprintf("%d", settings.segmentSeconds),
"-hls_list_size", "0",
"-hls_segment_filename", segments,
playlist,
)
}
return append(args,
"-pix_fmt", "yuv420p",
"-b:v", settings.bitrate,
+4 -2
View File
@@ -49,16 +49,18 @@ func (t *TranscoderService) runFFmpeg(ctx context.Context, job *hlsJob, input tr
t.mu.Lock()
// Only update the map entry if we are still the registered generation.
if cur, ok := t.jobs[job.mediaID]; ok && cur == job {
if cur, ok := t.jobs[job.key()]; ok && cur == job {
if err == nil && ctx.Err() == nil {
job.completed = true
} else {
delete(t.jobs, job.mediaID)
delete(t.jobs, job.key())
}
}
t.mu.Unlock()
if err != nil || ctx.Err() != nil {
job.cancel()
_ = os.RemoveAll(job.outputDir)
_ = os.Remove(filepath.Dir(job.outputDir)) // Remove the scope root only if empty.
}
t.hub.Publish("transcode", map[string]any{
+13 -10
View File
@@ -26,16 +26,18 @@ func (t *TranscoderService) WaitReady(ctx context.Context, mediaID string, timeo
job, ok := t.jobs[mediaID]
var started time.Time
var completed bool
var playlist string
if ok {
started = job.startedAt
completed = job.completed
playlist = t.jobPlaylistPath(job)
}
t.mu.Unlock()
if !ok {
return false // A failed/exited job cannot become ready later.
}
if ok {
if info, err := os.Stat(t.PlaylistPath(mediaID)); err == nil {
if info, err := os.Stat(playlist); err == nil {
// Allow a small clock skew; reject anything older than this job.
if !info.ModTime().Before(started.Add(-2 * time.Second)) {
t.mu.Lock()
@@ -66,9 +68,8 @@ func (t *TranscoderService) WaitReady(ctx context.Context, mediaID string, timeo
// briefly for it to exit so a subsequent EnsureJobFrom cannot race-write the
// same HLS directory.
func (t *TranscoderService) StopJob(mediaID string) {
gate := t.mediaStartGate(mediaID)
gate.Lock()
defer gate.Unlock()
unlock := t.lockJobStart(mediaID)
defer unlock()
t.mu.Lock()
prev := t.detachJobLocked(mediaID)
@@ -170,7 +171,7 @@ func (t *TranscoderService) monitorIdle(ctx context.Context, job *hlsJob) {
return
case <-ticker.C:
t.mu.Lock()
current, ok := t.jobs[job.mediaID]
current, ok := t.jobs[job.key()]
if !ok || current != job {
t.mu.Unlock()
return
@@ -185,11 +186,10 @@ func (t *TranscoderService) monitorIdle(ctx context.Context, job *hlsJob) {
}
func (t *TranscoderService) stopIdleJob(job *hlsJob, timeout time.Duration) bool {
gate := t.mediaStartGate(job.mediaID)
gate.Lock()
defer gate.Unlock()
unlock := t.lockJobStart(job.key())
defer unlock()
t.mu.Lock()
current := t.jobs[job.mediaID]
current := t.jobs[job.key()]
if current != job {
t.mu.Unlock()
return true
@@ -199,9 +199,12 @@ func (t *TranscoderService) stopIdleJob(job *hlsJob, timeout time.Duration) bool
t.mu.Unlock()
return false
}
t.detachJobLocked(job.mediaID)
t.detachJobLocked(job.key())
t.mu.Unlock()
waitJobExit(job, 12*time.Second)
if job.jobKey != "" {
_ = os.RemoveAll(t.HLSDir(job.key()))
}
t.log.Info("transcode idle timeout", zap.String("media_id", job.mediaID),
zap.Duration("idle_for", idleFor), zap.Duration("timeout", timeout))
return true
+102
View File
@@ -0,0 +1,102 @@
package service
import (
"context"
"encoding/json"
"strings"
"github.com/truewhile/MeBox/internal/model"
)
// 蓝光原盘(m2ts,40 Mbps 级别)在两核小机器上重编码只有 0.1~0.2 倍速,HLS
// 永远追不上播放,播放器等不到切片就报「转码失败」。当源视频本身就是浏览器
// 能解的 H.264 / 8bit / 4:2:0,且用户选的是「原画」档位(不改分辨率)时,
// 这里改走只换封装(-c:v copy)的直通路径:视频不重编码,音频转 AAC,
// CPU 占用接近于零,起播时间从几十秒降到几秒。
//
// 直通只对「原画」生效。1080P/720P/480P 档位意味着要改分辨率,必须重新编码;
// 那类请求仍然按原来的 libx264 路径处理(慢,但码率可控)。
// remuxEncoderLabel 是直通任务在日志与任务面板里的标签。
const remuxEncoderLabel = "remux"
// remuxEligible 判断这次请求能否复制视频流而不是重编码。
func (t *TranscoderService) remuxEligible(ctx context.Context, m *model.Media, quality *LocalHLSQuality, subtitleStream int) bool {
if t == nil || t.repo == nil || m == nil {
return false
}
// 烧录图形字幕要把字幕叠加进画面,必须解码+重编码。
if subtitleStream >= 0 {
return false
}
if !remuxAppliesToQuality(quality) {
return false
}
probe, err := t.repo.MediaProbe.Get(ctx, m.ID)
if err != nil || probe == nil {
// 没有探测缓存就无法确认档次/像素格式,保守走重编码。
return false
}
return remuxCopyableHeight(probe) > 0
}
// remuxAppliesToQuality 只放行「原画」档位(Height 为 0,即不缩放)。
// quality 为 nil 表示调用方没有指定档位(走全局 max_height),同样不直通,
// 否则会绕过配置的分辨率上限给出一个远超预期的码流。
func remuxAppliesToQuality(quality *LocalHLSQuality) bool {
return quality != nil && quality.Height <= 0
}
// remuxCopyableHeight 在源视频可直接复制进 HLS 时返回它的高度,否则返回 0。
func remuxCopyableHeight(probe *model.MediaProbe) int {
if probe == nil || probe.Height <= 0 {
return 0
}
if !strings.EqualFold(strings.TrimSpace(probe.VideoCodec), "h264") {
return 0
}
if !browserSafeH264Payload(probe.Payload) {
return 0
}
return probe.Height
}
// probePayloadStreams 只解析落库 payload 里与直通判断有关的字段。
type probePayloadStreams struct {
Streams []struct {
Type string `json:"type"`
Profile string `json:"profile"`
PixFmt string `json:"pix_fmt"`
} `json:"streams"`
}
// browserSafeH264Payload 只放行 8bit 4:2:0 的 Baseline/Main/High。
// Hi10P(High 10)和 4:2:2 母带浏览器解不出来,复制过去只会黑屏或无声。
func browserSafeH264Payload(payload string) bool {
payload = strings.TrimSpace(payload)
if payload == "" {
// 老数据没有 payload:已经有 codec + 分辨率,按 8bit 常规情况处理。
return true
}
var parsed probePayloadStreams
if err := json.Unmarshal([]byte(payload), &parsed); err != nil {
return false
}
for _, stream := range parsed.Streams {
if !strings.EqualFold(strings.TrimSpace(stream.Type), "video") {
continue
}
switch strings.ToLower(strings.TrimSpace(stream.Profile)) {
case "", "constrained baseline", "baseline", "main", "high":
default:
return false
}
switch strings.ToLower(strings.TrimSpace(stream.PixFmt)) {
case "", "yuv420p", "yuvj420p":
default:
return false
}
return true
}
return true
}
+170
View File
@@ -0,0 +1,170 @@
package service
import (
"strings"
"testing"
"github.com/truewhile/MeBox/internal/config"
"github.com/truewhile/MeBox/internal/model"
)
// 直通档位必须真的把视频流复制过去:一旦又带上 libx264 / -vf / 码率参数,
// 慢机器上的「原画」就还是几十秒起播。
func TestBuildFFmpegArgsRemuxCopiesVideoStream(t *testing.T) {
cfg := &config.Config{}
cfg.Transcoder.SegmentSeconds = 4
cfg.Transcoder.Realtime = true
cfg.Transcoder.Threads = 2
cfg.Transcoder.MaxHeight = 720
quality, ok := LocalHLSQualityByID("source")
if !ok {
t.Fatal("source quality missing")
}
args := buildFFmpegArgsForInput(cfg, transcodeInput{
Source: "https://cdn.example/x.m2ts",
Headers: map[string]string{"User-Agent": "MeBox-115-GoClient/1.0"},
StartSec: 52.4,
Quality: &quality,
CopyVideo: true,
}, "/cache/hls/k/run-1/index.m3u8", "/cache/hls/k/run-1/seg_%05d.ts")
joined := strings.Join(args, " ")
for _, want := range []string{
"-c:v copy",
"-c:a aac",
"-ar 48000",
"-b:a 128k",
"-ac 2",
"-avoid_negative_ts make_zero",
"-ss 52.400",
"-noaccurate_seek",
"-hls_time 4",
"-hls_segment_filename /cache/hls/k/run-1/seg_%05d.ts",
} {
if !strings.Contains(joined, want) {
t.Errorf("remux args missing %q: %s", want, joined)
}
}
for _, unwanted := range []string{
"libx264",
"-vf ",
"-b:v",
"-maxrate",
"-bufsize",
"-force_key_frames",
"-preset",
"-threads",
"-pix_fmt",
} {
if strings.Contains(joined, unwanted) {
t.Errorf("remux args must not contain %q: %s", unwanted, joined)
}
}
// 中途起播不能加 -re,否则会先按 1 倍速爬到目标位置才出第一个切片。
if strings.Contains(joined, " -re ") {
t.Errorf("mid-file remux must not use -re: %s", joined)
}
}
// 烧录图形字幕要把字幕叠进画面,必须解码重编码,不能被直通标志绕过。
func TestBuildFFmpegArgsRemuxIgnoredForSubtitleBurnIn(t *testing.T) {
cfg := &config.Config{}
cfg.Transcoder.MaxHeight = 720
cfg.Transcoder.SegmentSeconds = 4
quality, _ := LocalHLSQualityByID("source")
subtitle := 4
args := buildFFmpegArgsForInput(cfg, transcodeInput{
Source: "/media/ep.mkv",
Quality: &quality,
CopyVideo: true,
SubtitleStream: &subtitle,
}, "/o/x.m3u8", "/o/seg_%05d.ts")
joined := strings.Join(args, " ")
if strings.Contains(joined, "-c:v copy") {
t.Fatalf("subtitle burn-in must not copy the video stream: %s", joined)
}
if !strings.Contains(joined, "libx264") || !strings.Contains(joined, "-filter_complex") {
t.Fatalf("subtitle burn-in should re-encode: %s", joined)
}
if strings.Contains(joined, "-noaccurate_seek") {
t.Fatalf("encoded path should keep accurate seek: %s", joined)
}
}
// 只有「原画」档位(不缩放)可以直通;其它档位要改分辨率。
func TestRemuxAppliesToQuality(t *testing.T) {
if remuxAppliesToQuality(nil) {
t.Fatal("nil quality must not remux: it falls back to the configured max height")
}
source, _ := LocalHLSQualityByID("source")
if !remuxAppliesToQuality(&source) {
t.Fatal("source quality should remux")
}
for _, id := range []string{"1080", "720", "480"} {
quality, ok := LocalHLSQualityByID(id)
if !ok {
t.Fatalf("quality %s missing", id)
}
if remuxAppliesToQuality(&quality) {
t.Fatalf("quality %s must re-encode", id)
}
}
}
// 只有 8bit 4:2:0 的 H.264 能安全复制;Hi10P / 4:2:2 / HEVC 复制过去浏览器黑屏。
func TestRemuxCopyableHeight(t *testing.T) {
const blurayPayload = `{"container":"mpegts","duration_sec":1451,"bit_rate":42609934,"streams":[{"index":0,"type":"video","codec":"h264","profile":"High","width":1920,"height":1080,"pix_fmt":"yuv420p","frame_rate":"23.976"},{"index":1,"type":"audio","codec":"pcm_bluray","channels":2}]}`
cases := []struct {
name string
probe *model.MediaProbe
want int
}{
{
name: "bluray h264 8bit 420",
probe: &model.MediaProbe{VideoCodec: "h264", Height: 1080, Payload: blurayPayload},
want: 1080,
},
{
name: "no payload falls back to codec and height",
probe: &model.MediaProbe{VideoCodec: "h264", Height: 720},
want: 720,
},
{
name: "hevc cannot be copied into browser hls",
probe: &model.MediaProbe{VideoCodec: "hevc", Height: 1080, Payload: `{"streams":[{"type":"video","profile":"Main 10","pix_fmt":"yuv420p10le"}]}`},
want: 0,
},
{
name: "hi10p h264 is unplayable in browsers",
probe: &model.MediaProbe{VideoCodec: "h264", Height: 1080, Payload: `{"streams":[{"type":"video","profile":"High 10","pix_fmt":"yuv420p10le"}]}`},
want: 0,
},
{
name: "4:2:2 h264 is unplayable in browsers",
probe: &model.MediaProbe{VideoCodec: "h264", Height: 1080, Payload: `{"streams":[{"type":"video","profile":"High 4:2:2","pix_fmt":"yuv422p"}]}`},
want: 0,
},
{
name: "missing probe",
probe: nil,
want: 0,
},
{
name: "unknown height",
probe: &model.MediaProbe{VideoCodec: "h264", Height: 0, Payload: blurayPayload},
want: 0,
},
}
for _, tc := range cases {
t.Run(tc.name, func(t *testing.T) {
if got := remuxCopyableHeight(tc.probe); got != tc.want {
t.Fatalf("remuxCopyableHeight() = %d, want %d", got, tc.want)
}
})
}
}
+6 -4
View File
@@ -175,7 +175,7 @@ export function streamURL(mediaId: string, options: { proxy?: boolean } = {}): s
// hlsURL returns the m3u8 playlist URL fed into hls.js.
// startSec > 0 asks the server to (re)start ffmpeg from that source offset.
export function hlsURL(mediaId: string, startSec = 0, subtitleStream?: number, quality?: string): string {
export function hlsURL(mediaId: string, startSec = 0, subtitleStream?: number, quality?: string, sessionID?: string): string {
const safeStart = Math.max(0, Math.round(startSec * 1000) / 1000)
// Always send start= (including 0) so the server can tell an intentional
// restart-from-head apart from a missing query on a stale refresh.
@@ -188,7 +188,8 @@ export function hlsURL(mediaId: string, startSec = 0, subtitleStream?: number, q
? `&subtitle=${encodeURIComponent(String(subtitleStream))}`
: ''
const qualityQuery = quality ? `&quality=${encodeURIComponent(quality)}` : ''
return `/api/hls/${encodeURIComponent(mediaId)}/index.m3u8?${tokenQuery()}${profileQuery()}${start}${subtitle}${qualityQuery}${bust}`
const session = sessionID ? `&session_id=${encodeURIComponent(sessionID)}` : ''
return `/api/hls/${encodeURIComponent(mediaId)}/index.m3u8?${tokenQuery()}${profileQuery()}${start}${subtitle}${qualityQuery}${bust}${session}`
}
// cloudHlsURL returns the MeBox-proxied 115 cloud HLS master playlist URL.
@@ -202,8 +203,9 @@ export function cloudHlsURL(mediaId: string, definition: string): string {
// Stop an on-demand HLS job. keepalive makes the request survive page
// navigation/tab close, where an axios promise can be discarded by browsers.
export function stopHLSJob(mediaId: string): void {
const url = `/api/hls/${encodeURIComponent(mediaId)}?${tokenQuery()}${profileQuery()}`
export function stopHLSJob(mediaId: string, sessionID?: string): void {
const session = sessionID ? `&session_id=${encodeURIComponent(sessionID)}` : ''
const url = `/api/hls/${encodeURIComponent(mediaId)}?${tokenQuery()}${profileQuery()}${session}`
void fetch(url, {
method: 'DELETE',
credentials: 'same-origin',
+3 -1
View File
@@ -464,7 +464,9 @@ export function HomeLibrariesSection({
pinned={pinnedIds.includes(lib.id)}
onTogglePin={onTogglePin ? () => onTogglePin(lib.id) : undefined}
onVisible={() => {
if (!lib.cover_url) {
// 服务端会生成库封面时不必再为首屏卡片拉成员预览;
// 远程挂载库的封面仍由卡片拼图兜底。
if (!lib.cover_url && !lib.cover_image_url) {
queuePreview(lib.id)
}
}}
+70 -64
View File
@@ -1,4 +1,4 @@
import { useCallback, useEffect, useMemo, useRef, useState } from 'react'
import { useCallback, useEffect, useLayoutEffect, useMemo, useRef, useState } from 'react'
import { useLocation, useNavigate, useParams } from 'react-router-dom'
import type Hls from 'hls.js'
import toast from 'react-hot-toast'
@@ -37,11 +37,13 @@ import {
type SubtitlePosition,
type SubtitleStylePreset,
} from '../utils/subtitleDisplay'
import { pickPlayerMode, needsTranscodeForBrowser, isDirectStreamMedia, isStrmMedia, type PlayerMode } from './playerPageModel'
import { pickPlayerMode, needsTranscodeForBrowser, isDirectStreamMedia, isStrmMedia, shouldShowPlayerDiagnostics, type PlayerMode } from './playerPageModel'
import { classifyDirectPlayError, directPlayFailureCopy, directStallTimeoutMs, DIRECT_SEEK_GRACE_MS } from './directPlayError'
import type { DirectPlayFailureReason } from './directPlayError'
import { canSeekDirectNow, DIRECT_SEEK_WAIT_MS } from './directPlaySeek'
import { applyPendingHlsSeek, attachNativeHls, bindPendingHlsSeek, hlsManifestTimeoutMs, playbackPositionSec } from './hlsPlayback'
import { applyPendingHlsSeek, attachNativeHls, bindPendingHlsSeek, hlsManifestTimeoutMs, playbackPositionSec, playbackTimeSec } from './hlsPlayback'
import { availableLocalQuality, defaultLocalQualityId, findPlaybackQualityById } from './playbackQuality'
import { createPlaybackRequestGuard } from './playbackRequestGuard'
import { apiErrorMessage } from './StrmManagePage'
import { PlayerTopBar } from './PlayerTopBar'
import { PlayerVideoStage } from './PlayerVideoStage'
@@ -118,6 +120,14 @@ export function PlayerPage() {
const hlsRef = useRef<Hls | null>(null)
const lastSentRef = useRef(0)
const progressSessionRef = useRef<PlaybackProgressSession | null>(null)
const cloudRequestGuardRef = useRef(createPlaybackRequestGuard())
const currentMediaIdRef = useRef(id)
useLayoutEffect(() => {
const guard = cloudRequestGuardRef.current
currentMediaIdRef.current = id
guard.invalidate()
return () => guard.invalidate()
}, [id])
const directRetryRef = useRef(false)
const retryingDirectRef = useRef(false)
// 程序化续播 seek 后的宽限截止时间;期内直连误报不升到 HLS。
@@ -181,6 +191,7 @@ export function PlayerPage() {
const [playerError, setPlayerError] = useState('')
// 媒体元数据加载失败(404 / 无权限等):舞台区直接展示错误而不是永远「加载中」
const [loadError, setLoadError] = useState('')
const diagnosticsVisible = shouldShowPlayerDiagnostics(playerError, loadError)
// 「客户端直连解码」模式:宿主机不转码,播放器强制 direct play、隐藏 HLS 切换。
const [directOnly, setDirectOnly] = useState(false)
const [directOnlyKnown, setDirectOnlyKnown] = useState(false)
@@ -562,6 +573,7 @@ export function PlayerPage() {
const setPlaybackMode = useCallback((next: PlayerMode) => {
// 一旦显式决定过播放方式,URL 上的 ?mode= 就不再是权威(避免它把状态打回去)。
requestedModeRef.current = null
if (next === 'direct') cloudRequestGuardRef.current.invalidate()
setMode(next)
}, [])
@@ -766,6 +778,11 @@ export function PlayerPage() {
*/
const handleDirectSeekStalled = useCallback(
(target: number) => {
cloudRequestGuardRef.current.invalidate()
if (directOnly || isDirectStreamMedia(mediaRef.current)) {
toast.error('当前播放方式不提供 HLS 转码,请使用外部播放器定位')
return
}
const cloudQuality =
findPlaybackQualityById(playbackInfo, selectedQuality) ??
findPlaybackQualityById(playbackInfo, playbackInfo?.default_quality ?? '')
@@ -779,9 +796,7 @@ export function PlayerPage() {
toast('直连定位失败,已切到 115 云 HLS 跳转')
return
}
const localQuality =
findPlaybackQualityById(playbackInfo, selectedQuality) ??
findPlaybackQualityById(playbackInfo, defaultLocalQualityId(playbackInfo))
const localQuality = availableLocalQuality(playbackInfo, selectedQuality)
if (localQuality?.available) {
setSelectedQuality(localQuality.id)
setHlsSource('local')
@@ -794,7 +809,7 @@ export function PlayerPage() {
}
toast.error('直连定位失败,且当前没有可用的 HLS 转码档位')
},
[playbackInfo, selectedQuality, setPlaybackMode],
[directOnly, playbackInfo, selectedQuality, setPlaybackMode],
)
// Load metadata and pick a default mode.
@@ -1031,17 +1046,22 @@ export function PlayerPage() {
const switchToLocalHLS = useCallback(
(position?: number) => {
cloudRequestGuardRef.current.invalidate()
pendingSeekRef.current = null
setHlsSource('local')
setCloudWaiting(false)
setSelectedQuality((prev) => {
if (playbackInfo?.local_qualities?.some((quality) => quality.id === prev)) return prev
return defaultLocalQualityId(playbackInfo)
})
const quality = availableLocalQuality(playbackInfo, selectedQuality)
if (playbackInfo && !quality) {
setHlsUnavailable(true)
showPlayFailure('本地 HLS 转码不可用,请配置 ffmpeg 或使用外部播放器。')
setPlaybackMode('direct')
return
}
setSelectedQuality(quality?.id ?? defaultLocalQualityId(playbackInfo))
if (position !== undefined) setHlsStartSec(Math.max(0, position))
setPlaybackMode('hls')
},
[playbackInfo, setPlaybackMode],
[playbackInfo, selectedQuality, setPlaybackMode, showPlayFailure],
)
/**
@@ -1079,8 +1099,11 @@ export function PlayerPage() {
const startCloudTranscode = useCallback(
async (definition: number) => {
if (!mediaId) return
const isCurrentRequest = cloudRequestGuardRef.current.begin()
const isCurrent = () => isCurrentRequest() && currentMediaIdRef.current === mediaId && !directOnlyRef.current
try {
const info = await mediaAPI.startCloudTranscode(mediaId, definition)
if (!isCurrent() || info.media_id !== mediaId) return
setPlaybackInfo(info)
if (info.transcode.state === 'ready') {
setCloudWaiting(false)
@@ -1093,6 +1116,7 @@ export function PlayerPage() {
setCloudWaitMessage(info.transcode.message || '115 正在转码…')
setCloudWaitStartedAt(Date.now())
} catch {
if (!isCurrent()) return
switchToLocalHLS()
toast.error('115 云端转码触发失败,已切换本地转码')
}
@@ -1104,6 +1128,7 @@ export function PlayerPage() {
// 的自愈逻辑共用,避免两处各写一遍云端/本地选择。
const enterHlsPlayback = useCallback(
(startSec: number, infoOverride?: PlaybackInfo | null) => {
cloudRequestGuardRef.current.invalidate()
const position = Math.max(0, startSec)
// 允许传入刚拿到的能力信息:从 URL 强制 HLS 时不想等到 state 更新后才选线路。
const info = infoOverride ?? playbackInfo
@@ -1126,6 +1151,7 @@ export function PlayerPage() {
}
} else {
switchToLocalHLS(position)
return
}
setPlaybackMode('hls')
},
@@ -1164,7 +1190,7 @@ export function PlayerPage() {
setCloudWaitMessage(info.transcode.message || '115 正在转码…')
cloudPollTimerRef.current = setTimeout(poll, cloudRetryRef.current * 1000)
} catch {
cloudPollTimerRef.current = setTimeout(poll, 5000)
if (!cancelled) cloudPollTimerRef.current = setTimeout(poll, 5000)
}
}
cloudPollTimerRef.current = setTimeout(poll, cloudRetryRef.current * 1000)
@@ -1337,6 +1363,7 @@ export function PlayerPage() {
if (!currentMedia) return
let cancelled = false
let cleanupHlsResume: (() => void) | undefined
let stopLocalPlayback: (() => void) | undefined
let onDirectMetadata: (() => void) | undefined
teardownHls()
@@ -1347,10 +1374,17 @@ export function PlayerPage() {
teardownHls()
return
}
// Each wired local source owns its job, including seek/quality changes.
// Capture the ID in cleanup so a delayed DELETE cannot stop a newer run.
const localSessionID = hlsSource === 'local' ? newPlaybackProgressSession(mediaId).id : undefined
if (localSessionID) {
stopLocalPlayback = () => stopHLSJob(mediaId, localSessionID)
window.addEventListener('pagehide', stopLocalPlayback)
}
const url =
hlsSource === 'cloud'
? cloudHlsURL(mediaId, selectedQuality)
: hlsURL(mediaId, hlsStartSec, activeBurnedSubtitleStream, selectedQuality)
: hlsURL(mediaId, hlsStartSec, activeBurnedSubtitleStream, selectedQuality, localSessionID)
void import('hls.js').then(({ default: HlsCtor }) => {
if (cancelled || !ref.current) return
if (HlsCtor.isSupported()) {
@@ -1493,6 +1527,10 @@ export function PlayerPage() {
if (onDirectMetadata) video.removeEventListener('loadedmetadata', onDirectMetadata)
cleanupHlsResume?.()
teardownHls()
if (stopLocalPlayback) {
window.removeEventListener('pagehide', stopLocalPlayback)
stopLocalPlayback()
}
}
}, [
activeBurnedSubtitleStream,
@@ -1517,24 +1555,6 @@ export function PlayerPage() {
vr360DirectProxy,
])
// Stop host ffmpeg when leaving this HLS player. The keepalive request also
// survives route navigation while the component is being torn down.
useEffect(() => {
if (!mediaId || mode !== 'hls' || hlsSource !== 'local') return
return () => {
stopHLSJob(mediaId)
}
}, [hlsSource, mediaId, mode])
// React cleanup is not guaranteed when a tab/window closes. pagehide fires
// while the document can still dispatch a keepalive request.
useEffect(() => {
if (!mediaId || mode !== 'hls' || hlsSource !== 'local') return
const stopOnPageExit = () => stopHLSJob(mediaId)
window.addEventListener('pagehide', stopOnPageExit)
return () => window.removeEventListener('pagehide', stopOnPageExit)
}, [hlsSource, mediaId, mode])
// 自动拉取已有的播放进度并恢复播放位置
useEffect(() => {
if (!id) return
@@ -1827,8 +1847,7 @@ export function PlayerPage() {
const sync = () => {
// HLS 转码时 <video>.currentTime 是相对转码起点的,必须把起点加回来。这里与
// 进度上报用的算法保持一致,否则转码场景下整段判断都会错位。
const offset = modeRef.current === 'hls' ? hlsStartSecRef.current : 0
const absolute = offset + (video.currentTime || 0)
const absolute = playbackPositionSec(modeRef.current, hlsSourceRef.current, hlsStartSecRef.current, video.currentTime || 0)
const next = resolveActiveSkip(absolute, skipSegments, {
hasNextEpisode: Boolean(nextEpisode),
excludedKinds: dismissedSkipKinds,
@@ -1853,8 +1872,7 @@ export function PlayerPage() {
selfSkipSeekRef.current = false
return
}
const offset = modeRef.current === 'hls' ? hlsStartSecRef.current : 0
const absolute = offset + (video.currentTime || 0)
const absolute = playbackPositionSec(modeRef.current, hlsSourceRef.current, hlsStartSecRef.current, video.currentTime || 0)
const hit = skipSegments.find(
(segment) =>
(segment.kind === 'intro' || segment.kind === 'recap') &&
@@ -2168,7 +2186,7 @@ export function PlayerPage() {
return
}
selfSkipSeekRef.current = true
video.currentTime = Math.max(0, target - hlsStartSec)
video.currentTime = playbackTimeSec(mode, hlsSource, hlsStartSec, target)
return
}
if (isStrmMedia(mediaRef.current)) {
@@ -2276,6 +2294,7 @@ export function PlayerPage() {
const selectPlaybackQuality = useCallback(
(quality: PlaybackQuality) => {
cloudRequestGuardRef.current.invalidate()
const video = ref.current
const localTime = video?.currentTime || 0
// 云 HLS 的 currentTime 本来就在绝对时间轴上;本地 HLS 的 t=0 对应 hlsStartSec。
@@ -2608,17 +2627,19 @@ export function PlayerPage() {
}
/>
) : null}
<button
type="button"
onClick={() => setDiagnosticsOpen((open) => !open)}
className="fixed bottom-5 right-4 z-[80] inline-flex items-center gap-2 rounded-full border border-white/15 bg-black/75 px-3 py-2 text-xs font-semibold text-white shadow-lg backdrop-blur hover:bg-black/90"
title="播放诊断"
aria-label="播放诊断"
>
<Activity size={15} />诊断
</button>
{diagnosticsOpen && (
<aside className="fixed bottom-16 right-4 z-[80] max-h-[70vh] w-[min(92vw,420px)] overflow-hidden rounded-2xl border border-white/15 bg-[#111827]/95 text-white shadow-2xl backdrop-blur">
{diagnosticsVisible && (
<button
type="button"
onClick={() => setDiagnosticsOpen((open) => !open)}
className="absolute right-2.5 top-14 z-[80] inline-flex items-center gap-2 rounded-full border border-white/15 bg-black/75 px-3 py-2 text-xs font-semibold text-white shadow-lg backdrop-blur hover:bg-black/90 sm:right-3.5 sm:top-16"
title="播放诊断"
aria-label="播放诊断"
>
<Activity size={15} />诊断
</button>
)}
{diagnosticsVisible && diagnosticsOpen && (
<aside className="absolute right-2.5 top-[6.75rem] z-[80] flex max-h-[calc(100%-8.5rem)] w-[min(92vw,420px)] flex-col overflow-hidden rounded-2xl border border-white/15 bg-[#111827]/95 text-white shadow-2xl backdrop-blur sm:right-3.5 sm:top-[7.25rem]">
<div className="flex items-center justify-between border-b border-white/10 px-4 py-3">
<div><h2 className="text-sm font-bold">播放诊断</h2><p className="mt-1 text-[11px] text-white/60">记录最近 40 条本地播放事件</p></div>
<div className="flex gap-2">
@@ -2626,7 +2647,7 @@ export function PlayerPage() {
<button type="button" onClick={() => setDiagnosticsOpen(false)} className="rounded-lg p-1.5 hover:bg-white/10" aria-label="关闭"><X size={16} /></button>
</div>
</div>
<div className="max-h-[55vh] space-y-2 overflow-y-auto p-3">
<div className="min-h-0 flex-1 space-y-2 overflow-y-auto p-3">
<div className="rounded-lg bg-white/5 px-3 py-2 text-xs text-white/70">模式:{mode === 'hls' ? `HLS / ${hlsSource}` : '直连'} · 来源:{isRemoteEmbyID(media?.id) ? '远程 Emby' : isStrmMedia(media) ? 'STRM' : '本地文件'}</div>
{diagnosticEvents.length === 0 ? <p className="py-6 text-center text-xs text-white/50">等待播放事件…</p> : [...diagnosticEvents].reverse().map((event, index) => (
<div key={`${event.at}-${index}`} className="rounded-lg bg-white/5 px-3 py-2">
@@ -2656,21 +2677,6 @@ function formatEpisodeDisplay(ep: Media, siblings: Media[]): string {
return episodeLabel || mediaTitle || title || '未命名'
}
function findPlaybackQualityById(info: PlaybackInfo | null, id: string): PlaybackQuality | undefined {
if (!info || !id) return undefined
return [...(info.cloud_qualities ?? []), ...(info.local_qualities ?? [])].find(
(quality) => quality.id === id,
)
}
function defaultLocalQualityId(info: PlaybackInfo | null): string {
const qualities = info?.local_qualities ?? []
for (const id of ['1080', '720', '480', 'source']) {
if (qualities.some((quality) => quality.id === id)) return id
}
return qualities[0]?.id ?? '720'
}
function looksLikeSeriesTitle(ep: Media, title: string, siblings: Media[]): boolean {
const normalized = normalizeEpisodeTitle(title)
if (!normalized) return true
+12 -5
View File
@@ -36,27 +36,34 @@ export function SettingsPage() {
}
}, [searchParams])
// refresh 只等真正需要的设置项返回。媒体库列表仅供「成人库」等下拉选项
// 使用,晚到不影响设置项渲染;此前用 Promise.all 一并等待,远程 Emby
// 挂载枚举偶尔要 10–17 秒,整页就一直空转。
const refresh = async () => {
setLoading(true)
try {
const [all, libs] = await Promise.all([
adminAPI.listSettings(),
libraryAPI.list({ includeHidden: true }).catch(() => [] as Library[]),
])
const all = await adminAPI.listSettings()
const idx: Record<string, string> = {}
for (const s of all as Setting[]) {
if (ALL_KEYS.has(s.key)) idx[s.key] = s.value
}
setValues(idx)
setLibraries(libs as Library[])
setDirty(new Set())
} finally {
setLoading(false)
}
}
const refreshLibraries = () => {
libraryAPI
.list({ includeHidden: true })
.then((libs) => setLibraries(libs as Library[]))
.catch(() => undefined)
}
useEffect(() => {
refresh().catch(() => undefined)
refreshLibraries()
}, [])
const onChange = (key: string, value: string) => {
+5 -1
View File
@@ -1,5 +1,5 @@
import { strict as assert } from 'node:assert'
import { applyPendingHlsSeek, attachNativeHls, bindPendingHlsSeek, hlsManifestTimeoutMs, playbackPositionSec } from './hlsPlayback.ts'
import { applyPendingHlsSeek, attachNativeHls, bindPendingHlsSeek, hlsManifestTimeoutMs, playbackPositionSec, playbackTimeSec } from './hlsPlayback.ts'
class FakeVideo extends EventTarget {
src = ''
@@ -55,4 +55,8 @@ assert.equal(hlsManifestTimeoutMs('cloud', 1800), 60_000)
assert.equal(playbackPositionSec('hls', 'cloud', 420, 600), 600, 'cloud subtitle/quality switches must not add an old local offset')
assert.equal(playbackPositionSec('hls', 'local', 420, 60), 480, 'local subtitle switches preserve the source position')
assert.equal(playbackPositionSec('direct', 'local', 420, 60), 60)
assert.equal(playbackTimeSec('hls', 'cloud', 600, 640), 640, 'cloud skip/undo uses the absolute target, not target minus a former local offset')
assert.equal(playbackTimeSec('hls', 'local', 600, 640), 40)
assert.equal(playbackTimeSec('direct', 'local', 600, 640), 640)
assert.equal(playbackTimeSec('hls', 'local', 600, 100), 0, 'targets before a local start clamp to zero')
console.log('hlsPlayback.test.ts ok')
+5
View File
@@ -41,3 +41,8 @@ export function attachNativeHls(video: HlsVideo, url: string, pending: PendingSe
export function playbackPositionSec(mode: 'direct' | 'hls', source: 'cloud' | 'local', startSec: number, currentTime: number): number {
return mode === 'hls' && source === 'local' ? startSec + currentTime : currentTime
}
/** Convert an absolute source target to the current media element's timeline. */
export function playbackTimeSec(mode: 'direct' | 'hls', source: 'cloud' | 'local', startSec: number, absoluteSec: number): number {
return Math.max(0, mode === 'hls' && source === 'local' ? absoluteSec - startSec : absoluteSec)
}
+7
View File
@@ -43,5 +43,12 @@ export function getLibraryArtworks(
if (library.cover_url) {
return [{ src: library.cover_url, version: library.updated_at }]
}
// 服务端生成的库封面(成员海报竖切拼图):所有客户端共用一张图,
// 不再由前端拿两张卡片自己拼格子。
// version 里带一个版次后缀:接口地址是固定的,靠它让浏览器在拼图规则
// 调整(如田字格改成竖切)后丢掉旧缓存重新拉图。
if (library.cover_image_url) {
return [{ src: library.cover_image_url, version: `${library.updated_at}-c3` }]
}
return libraryArtworkItems(cards)
}
+19
View File
@@ -0,0 +1,19 @@
import { strict as assert } from 'node:assert'
import { availableLocalQuality, findPlaybackQualityById } from './playbackQuality.ts'
import type { PlaybackInfo } from '../types/playback.ts'
const info: PlaybackInfo = {
media_id: 'm', provider: 'cloud115', default_quality: '4', fallback: [],
transcode: { state: 'idle' },
cloud_qualities: [{ id: '4', label: '1080P', source: 'cloud', available: false }],
local_qualities: [
{ id: 'source', label: '原画', source: 'local', available: false },
{ id: '1080', label: '1080P', source: 'local', available: true },
{ id: '720', label: '720P', source: 'local', available: true },
],
}
assert.equal(availableLocalQuality(info, '4')?.id, '1080', 'pending cloud quality must not mask the available local fallback')
assert.equal(availableLocalQuality(info, '720')?.id, '720', 'preserve an available local preference')
assert.equal(availableLocalQuality({ ...info, local_qualities: info.local_qualities.map(q => ({ ...q, available: false })) }, '4'), undefined)
assert.equal(findPlaybackQualityById(info, '4')?.source, 'cloud')
console.log('playbackQuality.test.ts ok')
+24
View File
@@ -0,0 +1,24 @@
import type { PlaybackInfo, PlaybackQuality } from '../types/playback'
export function findPlaybackQualityById(info: PlaybackInfo | null, id: string): PlaybackQuality | undefined {
if (!info || !id) return undefined
return [...(info.cloud_qualities ?? []), ...(info.local_qualities ?? [])].find(
(quality) => quality.id === id,
)
}
export function defaultLocalQualityId(info: PlaybackInfo | null): string {
const qualities = info?.local_qualities ?? []
for (const id of ['1080', '720', '480', 'source']) {
if (qualities.some((quality) => quality.id === id)) return id
}
return qualities[0]?.id ?? '720'
}
/** A cloud option may exist without being ready; it is never a local fallback. */
export function availableLocalQuality(info: PlaybackInfo | null, preferred = ''): PlaybackQuality | undefined {
const available = info?.local_qualities.filter((quality) => quality.source === 'local' && quality.available) ?? []
return available.find((quality) => quality.id === preferred) ??
['1080', '720', '480', 'source'].map((id) => available.find((quality) => quality.id === id)).find(Boolean) ??
available[0]
}
@@ -0,0 +1,30 @@
import { strict as assert } from 'node:assert'
import { createPlaybackRequestGuard } from './playbackRequestGuard.ts'
const guard = createPlaybackRequestGuard()
const first = guard.begin()
assert.equal(first(), true)
const second = guard.begin()
assert.equal(first(), false, 'a newer quality request supersedes the earlier request')
assert.equal(second(), true)
guard.invalidate()
assert.equal(second(), false, 'media changes, unmount and direct/local switches cancel a pending cloud result')
let resolveRequest: (value: string) => void = () => {}
const request = new Promise<string>(resolve => { resolveRequest = resolve })
const current = guard.begin()
let applied = 'episode-B'
const settled = request.then(value => { if (current()) applied = value })
guard.invalidate()
resolveRequest('episode-A')
await settled
assert.equal(applied, 'episode-B', 'a delayed response cannot overwrite the next media')
let rejected: (error: Error) => void = () => {}
const failed = new Promise<never>((_, reject) => { rejected = reject })
const activeFailure = guard.begin()
let fallbackCount = 0
const caught = failed.catch(() => { if (activeFailure()) fallbackCount += 1 })
guard.invalidate()
rejected(new Error('old request failed'))
await caught
assert.equal(fallbackCount, 0, 'a stale rejection cannot force a local fallback')
console.log('playbackRequestGuard.test.ts ok')
+13
View File
@@ -0,0 +1,13 @@
/** Invalidates both successful and failed responses after a newer playback intent. */
export function createPlaybackRequestGuard() {
let generation = 0
return {
begin(): () => boolean {
const requested = ++generation
return () => requested === generation
},
invalidate(): void {
generation += 1
},
}
}
+13
View File
@@ -0,0 +1,13 @@
import { shouldShowPlayerDiagnostics } from './playerPageModel.ts'
function check(name: string, actual: boolean, expected: boolean) {
if (actual !== expected) throw new Error(`playerDiagnostics: ${name}`)
}
check('hidden by default and during normal playback', shouldShowPlayerDiagnostics('', ''), false)
check('visible when playback fails', shouldShowPlayerDiagnostics('视频播放失败', ''), true)
check('visible when media cannot be loaded', shouldShowPlayerDiagnostics('', '无法加载该媒体'), true)
check('visible when both loading and playback fail', shouldShowPlayerDiagnostics('视频播放失败', '无法加载该媒体'), true)
check('hidden again after errors are cleared', shouldShowPlayerDiagnostics('', ''), false)
console.log('playerDiagnostics.test.ts ok')
+4
View File
@@ -3,6 +3,10 @@ import { isRemoteEmbyID } from '../utils/remoteEmby'
export type PlayerMode = 'direct' | 'hls'
export function shouldShowPlayerDiagnostics(playerError: string, loadError: string): boolean {
return Boolean(playerError || loadError)
}
const directContainers = ['mp4', 'webm', 'm4v']
const directVideoCodecs = ['h264', 'avc', 'avc1']
const directAudioCodecs = ['aac', 'mp3', 'opus']
-8
View File
@@ -130,14 +130,6 @@ export const generalSettingsGroup: SettingGroup = {
defaultValue: '256',
placeholder: '256',
},
{
key: 'cache.images_originals_ttl_hours',
label: '原图保留时长 (小时)',
type: 'number',
hint: '原图缓存的最长保留时间,超过即删除(默认 168 小时 = 7 天)。缩略图都已生成时原图就不再需要,缩短它可以让原图缓存保持在一个滚动窗口内。设为 0 表示不按时间淘汰',
defaultValue: '168',
placeholder: '168',
},
{
key: 'https.enabled',
label: '启用 HTTPS',
+6
View File
@@ -15,6 +15,12 @@ export interface Library {
path: string
type: string
cover_url?: string
/**
* 服务端生成的封面地址(成员海报拼图)。管理员自定义 cover_url 时等于
* 该地址,否则指向 `/api/libraries/:id/cover`。Web/App 统一读这个字段,
* 不用再自己用卡片拼格子。
*/
cover_image_url?: string
enabled: boolean
sort_order?: number
carousel_enabled?: boolean