Compare commits

...

63 Commits

Author SHA1 Message Date
github-actions[bot] 5c9e7fcaa6 chore: bump version to 0.0.60 [skip ci] 2026-09-01 08:26:41 +00:00
truewhile af67f4cd6e 优化 2026-09-01 16:26:25 +08:00
github-actions[bot] 73de139d1f chore: bump version to 0.0.59 [skip ci] 2026-09-01 06:33:57 +00:00
truewhile e0cc481b96 Merge pull request #1 from truewhile/beta
添加emby挂载功能,整合上传,下载,刮削队列到任务队列
2026-09-01 14:33:38 +08:00
truewhile 7ac75ec69a fix: 修复 beta CI 失败的两个平台/网络敏感测试
- TestSanitizePathWithSpecialChars:Windows 盘符断言按 runtime.GOOS 分支
  (Linux 下反斜杠统一为分隔符,期望 D/test/...)
- TestEnrichOneAdultScrapesArtwork:AdultProvider 改用 RoundTripper 全量
  mock,拦截所有外网请求(CI 环境可达 javdb/dmm 时会把真实封面写入
  断言导致失败);断言放宽为封面路径后缀
2026-09-01 14:22:52 +08:00
truewhile 0e73e43cbd ci: 新增 beta 分支自动构建流水线
- 触发:push beta / PR 到 beta / 手动触发
- test-and-build:SPA 构建 + go vet/test/build + 三平台交叉编译
  (linux/amd64、linux/arm64、windows/amd64),版本号为
  {VERSION}-beta.{sha},二进制上传为 artifact
- docker-beta:多架构构建并推送 ghcr.io/{owner}/mmtl:beta
  (PR 事件不推镜像,仅推送 beta / 手动触发时推)
- 与 main 的发布流隔离:不做版本递增、不打 release tag
2026-09-01 14:12:31 +08:00
truewhile bd41ab3fb4 feat: 整合刮削/下载/上传队列为统一的「任务队列」
- 新增 /queue 任务队列页:全部 / 刮削 / 下载 / 上传 类型 Tab(URL 参数
  ?type= 同步),Tab 上实时显示各自任务计数徽章
- 「全部」视图纵向堆叠三类队列,单类视图直接复用原队列组件
  (ScraperQueuePage / StrmQueuePanel,后者已导出)
- 侧边栏导航:删除「刮削队列 / 下载队列 / 上传队列」三个入口,合并为
  「任务队列」单一菜单项;旧路由 /scraper/queue、/strm/downloads、
  /strm/uploads 保留直达
2026-09-01 14:02:25 +08:00
truewhile 73b95a8e38 feat: Emby 挂载独立管理(媒体库级选择挂载 + 每库代理开关)
- 新增 EmbyMount 表与 /admin/emby/mounts CRUD:把远程 Emby 的媒体库按需
  挂载到本项目(同一服务器可选择性挂载/全量挂载),代理开关下沉到每个
  挂载(一个 Emby 服务器可部分库代理、部分库直连)
- 伪装 ID 改为 embyremote~{mountID}~{remoteID}:播放/详情/状态/图片链路
  全部经 ResolveMount 解析挂载与账号
- 新增「Emby 挂载」独立菜单页(/emby-mount):账号管理(添加/测试/编辑/
  删除)+ 挂载列表(代理切换/停用/取消)+ 选择挂载对话框(多选+全量)
- STRM 管理页过滤 emby_remote 账号;账号表单移除 emby 类型
- Boot 幂等 AutoSeedMounts:旧 emby 账号自动全量挂载(沿用旧代理配置),
  升级后媒体库不消失
- 修复:远程流请求注入浏览器 UA(防 Cloudflare 风控)+ 代理强制 Static=true
  (阻止远程 ffmpeg 转码调度,反代纯字节中继)
- 全局搜索改为按挂载逐个搜索(结果归属与伪装 ID 正确)
2026-09-01 13:53:15 +08:00
truewhile 5a5555d28a fix: 远程 Emby 无图条目不再下发无效图片 URL(封面破图修复)
- 远程条目仅当 ImageTags 含 Primary/Backdrop 时才生成图片 URL;
  无图条目 poster_url 为空,前端走 No Poster 优雅占位而非裂图
- 剧集海报回退使用 Emby 的 SeriesPrimaryImageTag(Fields=SeriesPrimaryImage),
  仅当系列真实有图时才回退,避免连环无效请求
- 列表/剧集接口带 SeriesPrimaryImage 字段
2026-09-01 13:26:57 +08:00
truewhile 4dc9cbe2e7 feat: 网页端完整浏览/播放远程 Emby 挂载库
- 后端新增远程库→本地结构映射(emby_remote_web.go):Library/Media/SeriesCard
  与本地完全同构,前端无感知
- /api/libraries(含 with_preview)、/api/libraries/:id、/api/libraries/:id/media、
  /api/libraries/:id/series、/api/libraries/:id/series/episodes、/api/media/:id、
  /api/media/:id/episodes 全部支持远程伪装 ID 透传映射
- /api/stream/:id 对远程条目 302 直连远程 Emby 原地址(播放不经过本机)
- 库内容按 CollectionType 过滤(电影库取 Movie、剧集库取 Series),
  RecursiveItemCount 作为系列集数;修复伪装 ID 双重编码导致的图片/详情 500
- SPA fallback 放行 /library/embyremote~ 前缀(不再被当作 Emby API 路由 404)
- 前端:远程库/条目标记 is_remote_emby,隐藏扫描/刮削/编辑/NFO/回收站/
  HLS 转码等不适用操作;后台管理面板过滤远程库
2026-09-01 11:32:54 +08:00
truewhile 6af0e5fdcf fix: Emby 远程挂载播放地址强制下发 + 凭据状态显示修正
- PlaybackInfo 始终构造 DirectStreamUrl(远程 PlaybackInfo 默认不带该字段,
  此前客户端拿不到播放地址):默认指向远程 Emby 原地址(不代理),
  开启代理时指向 MMTL /Videos/{encoded} 反代端点
- HasStrmAccountCredential 对 emby_remote 放宽为 url+用户名/密码 即视为
  已配置(此前只认 api_key,导致列表误显示「凭据:未配置」)
2026-09-01 11:06:36 +08:00
truewhile f0055b76fe fix: Emby 远程挂载账号编辑保留凭据 + 代理开关回显
- UpdateStrmAccount 对 emby_remote 做合并式更新:只覆盖传入键,未提及的
  地址/用户名/密码/token 密文保留,避免编辑代理开关时清空凭据
- 账号列表/编辑表单回显 proxy_play(服务端解密后下发)
- 前端 StrmAccount 类型与编辑对话框支持 proxy_play 回显
2026-09-01 10:45:52 +08:00
truewhile 0eb3f104f4 feat: Emby 远程挂载(联邦聚合)功能 beta
- 新增 emby_remote 提供方:通过 STRM 账号体系配置远程 Emby 地址/用户名/密码
- EmbyService 聚合远程媒体库 Views/Items/详情/最近添加/全局搜索,远程数据不落库
- 条目 ID 统一伪装为 embyremote~{account}~{id},请求按账号路由回远程
- 播放支持账号级 proxy_play 配置:默认客户端直连远程;开启后由 MMTL 反向代理流/字幕
- 图片/元数据全部实时透传远程,播放状态(已看/收藏)透传回远程 Emby
- cloud 包新增 emby Provider(Ping/List/Resolve),账号测试/目录浏览自动生效
- 前端 STRM 页面支持 Emby 远程挂载账号(地址+凭据+代理开关)
2026-09-01 10:44:01 +08:00
github-actions[bot] c171b38155 chore: bump version to 0.0.58 [skip ci] 2026-08-30 14:32:56 +00:00
truewhile 89d7a6cbb2 优化 2026-08-30 22:32:39 +08:00
github-actions[bot] 85918d1196 chore: bump version to 0.0.57 [skip ci] 2026-08-30 13:41:17 +00:00
truewhile fa9307e2e1 优化 2026-08-30 21:40:58 +08:00
github-actions[bot] 5ab18c2725 chore: bump version to 0.0.56 [skip ci] 2026-08-30 12:05:41 +00:00
truewhile a4a2bde1a4 优化 2026-08-30 20:05:27 +08:00
github-actions[bot] 28113f5fdc chore: bump version to 0.0.55 [skip ci] 2026-08-30 10:50:29 +00:00
truewhile 42b8805e94 优化 2026-08-30 18:50:15 +08:00
github-actions[bot] 51a64f41b9 chore: bump version to 0.0.54 [skip ci] 2026-08-30 09:28:16 +00:00
truewhile a7bd9a942c 优化显示 2026-08-30 17:28:00 +08:00
github-actions[bot] 97491a6175 chore: bump version to 0.0.53 [skip ci] 2026-08-29 14:16:40 +00:00
truewhile ea5cb3a130 优化 2026-08-29 22:16:21 +08:00
github-actions[bot] 921010926b chore: bump version to 0.0.52 [skip ci] 2026-08-28 14:38:24 +00:00
truewhile 7425c3d57b 优化 2026-08-28 22:38:09 +08:00
github-actions[bot] 1b7d4eef46 chore: bump version to 0.0.51 [skip ci] 2026-08-28 14:20:18 +00:00
truewhile c30dab56a3 优化 2026-08-28 22:20:01 +08:00
github-actions[bot] e365250440 chore: bump version to 0.0.50 [skip ci] 2026-08-28 10:28:29 +00:00
truewhile 47d10e1f58 优化 2026-08-28 18:28:09 +08:00
github-actions[bot] e6473300a7 chore: bump version to 0.0.49 [skip ci] 2026-08-28 08:27:32 +00:00
truewhile 994f64f753 优化 2026-08-28 16:27:11 +08:00
github-actions[bot] 90064a5480 chore: bump version to 0.0.48 [skip ci] 2026-08-27 15:40:31 +00:00
truewhile 6e8eac9887 优化 2026-08-27 23:40:12 +08:00
github-actions[bot] d3051eaffe chore: bump version to 0.0.47 [skip ci] 2026-08-27 13:24:18 +00:00
truewhile 496a897782 优化 2026-08-27 21:24:01 +08:00
github-actions[bot] 22b7290ee1 chore: bump version to 0.0.46 [skip ci] 2026-08-27 07:42:16 +00:00
truewhile 14037d5dea 5 2026-08-27 15:42:00 +08:00
github-actions[bot] 152db3fb9f chore: bump version to 0.0.45 [skip ci] 2026-08-27 06:20:25 +00:00
truewhile 0413d123da 4 2026-08-27 14:20:05 +08:00
github-actions[bot] 5f6bd7b5cd chore: bump version to 0.0.44 [skip ci] 2026-08-27 05:41:04 +00:00
truewhile 3c25bb5d61 3 2026-08-27 13:40:46 +08:00
github-actions[bot] 659b91b000 chore: bump version to 0.0.43 [skip ci] 2026-08-27 03:34:05 +00:00
truewhile fe5b3bd56a 2 2026-08-27 11:33:50 +08:00
github-actions[bot] 82bbb116ae chore: bump version to 0.0.42 [skip ci] 2026-08-27 03:07:35 +00:00
truewhile 9f5ff7e6f0 1 2026-08-27 11:07:18 +08:00
github-actions[bot] a00504080a chore: bump version to 0.0.41 [skip ci] 2026-08-27 02:09:16 +00:00
truewhile fc6e2e6f10 优化 strm 同步记录:支持删除记录并展示上传数量统计 2026-08-27 10:08:54 +08:00
github-actions[bot] 65c5f3e4bf chore: bump version to 0.0.40 [skip ci] 2026-08-26 15:35:37 +00:00
truewhile 07e340251b 优化 2026-08-26 23:32:02 +08:00
github-actions[bot] 9b956b928b chore: bump version to 0.0.39 [skip ci] 2026-08-26 13:41:25 +00:00
truewhile c3187f6e3f 优化上传逻辑
优化上传逻辑
2026-08-26 21:41:06 +08:00
github-actions[bot] 60c815a8b3 chore: bump version to 0.0.38 [skip ci] 2026-08-26 09:06:04 +00:00
truewhile 41b155ea31 Merge branch 'main' of https://github.com/truewhile/MMTL 2026-08-26 17:05:45 +08:00
truewhile 2888ae8bf7 8 2026-08-26 17:05:41 +08:00
github-actions[bot] 7363064d89 chore: bump version to 0.0.37 [skip ci] 2026-08-26 08:16:56 +00:00
truewhile 1d53bf2ae1 7 2026-08-26 16:16:39 +08:00
github-actions[bot] 618165ec31 chore: bump version to 0.0.36 [skip ci] 2026-08-26 07:51:14 +00:00
truewhile 87c66a9b8c 6 2026-08-26 15:50:59 +08:00
github-actions[bot] 1ea4724261 chore: bump version to 0.0.35 [skip ci] 2026-08-26 06:50:29 +00:00
truewhile 3d372f039e Merge branch 'main' of https://github.com/truewhile/MMTL 2026-08-26 14:50:10 +08:00
truewhile 0384017e98 6 2026-08-26 14:50:06 +08:00
163 changed files with 14775 additions and 2018 deletions
+117
View File
@@ -27,6 +27,9 @@ permissions:
jobs:
version-and-publish:
runs-on: ubuntu-latest
outputs:
new_version: ${{ steps.bump_version.outputs.new_version }}
tag: ${{ steps.bump_version.outputs.tag }}
steps:
- uses: actions/checkout@v4
with:
@@ -149,3 +152,117 @@ jobs:
VERSION=${{ steps.bump_version.outputs.new_version }}
cache-from: type=gha
cache-to: type=gha,mode=max
# 单文件可执行构建:把前端打包进二进制(go:embed),交叉编译 Windows /
# Linux / macOS 的 amd64 / arm64 产物,作为 GitHub Release 附件发布。
build-frontend:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: actions/setup-node@v4
with:
node-version: '20'
cache: 'npm'
cache-dependency-path: web/package-lock.json
- name: Install
working-directory: web
run: npm ci
- name: Build SPA
working-directory: web
run: npm run build
- name: Upload web/dist
uses: actions/upload-artifact@v4
with:
name: web-dist
path: web/dist
retention-days: 1
# 先创建(幂等)空的 GitHub Release,供后续 build-binaries 并行上传附件,
# 也避免矩阵各 job 并发 upload 时 release 尚不存在而互相竞争。
publish-create-release:
needs: [version-and-publish]
runs-on: ubuntu-latest
permissions:
contents: write
steps:
- uses: actions/checkout@v4
- name: Create release
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
RELEASE_TAG: ${{ needs.version-and-publish.outputs.tag }}
run: |
set -eux
# tag 已由 version-and-publish 推送;若 release 已存在则忽略(--verify-tag 幂等)
gh release create "$RELEASE_TAG" \
--title "MMTL ${{ needs.version-and-publish.outputs.new_version }}" \
--notes "自动化发布 ${{ needs.version-and-publish.outputs.new_version }}" \
--verify-tag --latest || true
build-binaries:
needs: [version-and-publish, build-frontend, publish-create-release]
runs-on: ubuntu-latest
permissions:
contents: write
strategy:
fail-fast: false
matrix:
include:
- goos: linux
goarch: amd64
ext: ""
- goos: linux
goarch: arm64
ext: ""
- goos: windows
goarch: amd64
ext: .exe
- goos: windows
goarch: arm64
ext: .exe
- goos: darwin
goarch: amd64
ext: ""
- goos: darwin
goarch: arm64
ext: ""
steps:
- uses: actions/checkout@v4
- uses: actions/setup-go@v5
with:
go-version: '1.25'
cache: true
- name: Download web/dist
uses: actions/download-artifact@v4
with:
name: web-dist
path: web/dist
- name: Build binary
run: |
CGO_ENABLED=0 GOOS=${{ matrix.goos }} GOARCH=${{ matrix.goarch }} \
go build -trimpath -ldflags="-s -w -X main.version=${{ needs.version-and-publish.outputs.tag }}" \
-o "dist/mmtl-${{ matrix.goos }}-${{ matrix.goarch }}${{ matrix.ext }}" ./cmd/server
- name: Package
run: |
mkdir -p package/mmtl
cp "dist/mmtl-${{ matrix.goos }}-${{ matrix.goarch }}${{ matrix.ext }}" package/mmtl/mmtl${{ matrix.ext }}
cp README.md package/mmtl/ 2>/dev/null || true
if [ "${{ matrix.goos }}" = "windows" ]; then
(cd package && zip -r "../mmtl_${{ matrix.goos }}_${{ matrix.goarch }}.zip" mmtl)
else
tar -czf "mmtl_${{ matrix.goos }}_${{ matrix.goarch }}.tar.gz" -C package mmtl
fi
- name: Upload to GitHub Release
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
RELEASE_TAG: ${{ needs.version-and-publish.outputs.tag }}
run: |
set -eux
PKG="mmtl_${{ matrix.goos }}_${{ matrix.goarch }}.zip"
TAR="mmtl_${{ matrix.goos }}_${{ matrix.goarch }}.tar.gz"
# 并发上传到同一 release 各自文件,--clobber 幂等覆盖
if [ -f "$PKG" ]; then
for i in 1 2 3; do gh release upload "$RELEASE_TAG" "$PKG" --clobber && break || sleep 5; done
fi
if [ -f "$TAR" ]; then
for i in 1 2 3; do gh release upload "$RELEASE_TAG" "$TAR" --clobber && break || sleep 5; done
fi
+133
View File
@@ -0,0 +1,133 @@
# Beta 分支自动构建流水线
#
# 触发:push 到 beta 分支 / PR 到 beta / 手动触发。
# 产出:
# 1. 前端 + 后端编译验证(go vet / go test / go build)
# 2. 多平台可执行二进制 artifact(linux/amd64、linux/arm64、windows/amd64)
# 3. ghcr.io/{owner}/mmtl:beta 多架构 Docker 镜像(linux/amd64 + linux/arm64)
#
# 与 main 分支的发布流(Auto-docker-publish.yml)隔离:beta 不做版本递增、
# 不打 release tag,只构建带 -beta 标识的产物供测试。
name: Beta Build
on:
push:
branches: [beta]
pull_request:
branches: [beta]
workflow_dispatch:
permissions:
contents: read
packages: write
env:
BETA_VERSION_PREFIX: beta
jobs:
# ─────────────────────────────────────────────────────────────────────────────
# 1) 编译验证 + 多平台二进制产物
# ─────────────────────────────────────────────────────────────────────────────
test-and-build:
name: Test & build artifacts
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
with:
fetch-depth: 0
- name: Resolve beta version
id: version
run: |
BASE_VERSION=$(cat VERSION 2>/dev/null || echo "0.0.0")
SHA_SHORT=${GITHUB_SHA:0:7}
echo "full_version=${BASE_VERSION}-beta.${SHA_SHORT}" >> "$GITHUB_OUTPUT"
# The binary embeds the SPA (web/dist) via go:embed, so dist must exist
# before the Go toolchain touches the web package.
- uses: actions/setup-node@v4
with:
node-version: '20'
cache: 'npm'
cache-dependency-path: web/package-lock.json
- name: Build SPA
working-directory: web
run: |
npm ci
npm run build
- uses: actions/setup-go@v5
with:
go-version: '1.25'
cache: true
- name: go vet
run: go vet ./...
- name: go test
run: go test ./...
- name: go build (host)
run: go build ./...
# 多平台可执行文件(嵌入刚构建的 web/dist)
- name: Build linux/amd64
run: CGO_ENABLED=0 GOOS=linux GOARCH=amd64 go build -trimpath -ldflags="-s -w -X main.version=${{ steps.version.outputs.full_version }}" -o dist/mmtl-beta-linux-amd64 ./cmd/server
- name: Build linux/arm64
run: CGO_ENABLED=0 GOOS=linux GOARCH=arm64 go build -trimpath -ldflags="-s -w -X main.version=${{ steps.version.outputs.full_version }}" -o dist/mmtl-beta-linux-arm64 ./cmd/server
- name: Build windows/amd64
run: CGO_ENABLED=0 GOOS=windows GOARCH=amd64 go build -trimpath -ldflags="-s -w -X main.version=${{ steps.version.outputs.full_version }}" -o dist/mmtl-beta-windows-amd64.exe ./cmd/server
- name: Upload artifacts
uses: actions/upload-artifact@v4
with:
name: mmtl-beta-binaries
path: dist/*
if-no-files-found: error
# ─────────────────────────────────────────────────────────────────────────────
# 2) Beta Docker 镜像(ghcr.io/{owner}/mmtl:beta)
# ─────────────────────────────────────────────────────────────────────────────
docker-beta:
name: Build & push beta Docker image
needs: test-and-build
runs-on: ubuntu-latest
# PR 事件不推送镜像,仅 push beta / 手动触发时推送
if: github.event_name != 'pull_request'
steps:
- uses: actions/checkout@v4
- name: Resolve beta version
id: version
run: |
BASE_VERSION=$(cat VERSION 2>/dev/null || echo "0.0.0")
SHA_SHORT=${GITHUB_SHA:0:7}
echo "full_version=${BASE_VERSION}-beta.${SHA_SHORT}" >> "$GITHUB_OUTPUT"
- uses: docker/setup-qemu-action@v3
- uses: docker/setup-buildx-action@v3
- name: Log in to GHCR
uses: docker/login-action@v3
with:
registry: ghcr.io
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}
- name: Build & push
uses: docker/build-push-action@v6
with:
context: .
platforms: linux/amd64,linux/arm64
push: true
provenance: false
sbom: false
tags: ghcr.io/${{ github.repository_owner }}/mmtl:beta
labels: |
org.opencontainers.image.revision=${{ github.sha }}
org.opencontainers.image.source=${{ github.repository }}
build-args: |
VERSION=${{ steps.version.outputs.full_version }}
cache-from: type=gha
cache-to: type=gha,mode=max
+13
View File
@@ -18,6 +18,19 @@ jobs:
go-version: '1.25'
cache: true
# The binary embeds the SPA (web/dist) via go:embed, so the dist must exist
# before the Go toolchain touches the `web` package.
- uses: actions/setup-node@v4
with:
node-version: '20'
cache: 'npm'
cache-dependency-path: web/package-lock.json
- name: Build SPA
working-directory: web
run: |
npm ci
npm run build
- name: go vet
run: go vet ./...
+22 -5
View File
@@ -360,15 +360,17 @@ docker compose -f docker-compose.search.yml up -d --no-deps mmtl
本地开发需要 Go、Node.js 和 npm。
后端会将 `web/dist` 通过 `go:embed` 编进二进制,因此**在编译 / 运行后端之前要先构建前端**,否则 `web` 包会因为缺少嵌入资源而编译失败。
```bash
# 前端依赖与构建(必须先做,产物被 go:embed 打进二进制)
npm --prefix web ci
npm --prefix web run build
# 后端测试
go test ./...
# 前端依赖与构建
npm --prefix web install
npm --prefix web run build
# 本地运行后端
# 本地运行后端(二进制自带前端界面,无需额外 web 目录)
go run ./cmd/server
# 本地运行前端开发服务器
@@ -387,6 +389,21 @@ http://127.0.0.1:3000
http://127.0.0.1:8080/api/health
```
### 交叉编译单文件发布物
CI(`.github/workflows/Auto-docker-publish.yml`)每次发布会自动为 Windows / Linux(含 Debian) / macOS 交叉编译 amd64 + arm64 的单文件可执行程序,并上传到对应的 GitHub Release。你可以在 Releases 页面下载 `.zip`(Windows)或 `.tar.gz`(Linux / macOS)附件,解压后直接运行其中的 `mmtl`(Windows 为 `mmtl.exe`),无需额外携带前端目录。
本地手动交叉编译某个平台:
```bash
# 先构建前端
npm --prefix web ci && npm --prefix web run build
# 例如:构建 Linux amd64 单文件
CGO_ENABLED=0 GOOS=linux GOARCH=amd64 \
go build -trimpath -ldflags="-s -w" -o mmtl-linux-amd64 ./cmd/server
```
## 贡献与反馈
提交 Bug、功能建议或 Pull Request 前,请先阅读 [贡献规范](CONTRIBUTING.md)。
+1 -1
View File
@@ -1 +1 @@
0.0.34
0.0.60
+3 -3
View File
@@ -52,7 +52,7 @@ func TestServeSPANoCachesIndexAndServesRoutes(t *testing.T) {
}
router := gin.New()
serveSPA(router, webDir)
serveSPA(router, os.DirFS(webDir))
for _, path := range []string{"/", "/login", "/library/e1c3507e-2878-40ae-a0e1-6b6e44b7fa7a", "/media/abc"} {
req := httptest.NewRequest(http.MethodGet, path, nil)
@@ -93,7 +93,7 @@ func TestServeSPAServesAssetsImmutableAndBypassesAPIRoutes(t *testing.T) {
}
router := gin.New()
serveSPA(router, webDir)
serveSPA(router, os.DirFS(webDir))
assetReq := httptest.NewRequest(http.MethodGet, "/assets/app.js", nil)
assetResp := httptest.NewRecorder()
@@ -155,7 +155,7 @@ func TestServeSPAServesAssetsImmutableAndBypassesAPIRoutes(t *testing.T) {
func TestServeSPAMissingIndexReportsExplicit404(t *testing.T) {
gin.SetMode(gin.TestMode)
router := gin.New()
serveSPA(router, t.TempDir())
serveSPA(router, os.DirFS(t.TempDir()))
req := httptest.NewRequest(http.MethodGet, "/", nil)
w := httptest.NewRecorder()
+64 -23
View File
@@ -1,6 +1,8 @@
package main
import (
"io/fs"
"mime"
"net/http"
"os"
"path/filepath"
@@ -13,6 +15,8 @@ import (
"github.com/ShukeBta/MMTL/internal/handler"
"github.com/ShukeBta/MMTL/internal/middleware"
"github.com/ShukeBta/MMTL/internal/service"
"github.com/ShukeBta/MMTL/web"
)
func buildRouter(cfg *config.Config, logger *zap.Logger, svc *service.Container) *gin.Engine {
@@ -29,31 +33,41 @@ func buildRouter(cfg *config.Config, logger *zap.Logger, svc *service.Container)
handler.Register(r, cfg, logger, svc)
if cfg.App.WebDir != "" {
serveSPA(r, cfg.App.WebDir)
// Prefer a directory on disk when configured explicitly (e.g. the Docker image
// mounts web/dist from the build stage, or an operator overrides app.web_dir
// with a custom skin). Otherwise fall back to the SPA embedded into the binary,
// which is what makes the cross-platform single-file artifacts work.
uiFS := webui.DistFS()
if dir := cfg.App.WebDir; dir != "" {
disk := os.DirFS(dir)
if index, err := fs.Stat(disk, "index.html"); err == nil && !index.IsDir() {
uiFS = disk
}
}
serveSPA(r, uiFS)
return r
}
// serveSPA serves the React build artifacts and falls back to index.html for
// non-API, non-asset paths so client-side routing keeps working.
func serveSPA(r *gin.Engine, webDir string) {
// non-API, non-asset paths so client-side routing keeps working. The UI tree
// comes from root, which is either the compiled-in SPA or an on-disk web dir.
func serveSPA(r *gin.Engine, root fs.FS) {
assets := r.Group("/assets")
assets.Use(func(c *gin.Context) {
c.Header("Cache-Control", "public, max-age=31536000, immutable")
c.Next()
})
assets.Static("/", filepath.Join(webDir, "assets"))
assets.GET("/*filepath", serveFSDir(root, "assets"))
brand := r.Group("/brand")
brand.Use(func(c *gin.Context) {
setNoCacheHeaders(c)
c.Next()
})
brand.Static("/", filepath.Join(webDir, "brand"))
brand.GET("/*filepath", serveFSDir(root, "brand"))
for _, rootFile := range []string{"/favicon.ico", "/favicon.svg", "/artwork-cache-sw.js"} {
filePath := filepath.Join(webDir, strings.TrimPrefix(rootFile, "/"))
r.GET(rootFile, serveNoCacheFile(filePath))
r.HEAD(rootFile, serveNoCacheFile(filePath))
name := strings.TrimPrefix(rootFile, "/")
r.GET(rootFile, serveFSFile(root, name))
r.HEAD(rootFile, serveFSFile(root, name))
}
r.NoRoute(func(c *gin.Context) {
path := c.Request.URL.Path
@@ -61,28 +75,50 @@ func serveSPA(r *gin.Engine, webDir string) {
c.Status(http.StatusNotFound)
return
}
serveSPAIndex(c, filepath.Join(webDir, "index.html"))
setNoCacheHeaders(c)
data, err := fs.ReadFile(root, "index.html")
if err != nil {
c.String(http.StatusNotFound, "MMTL web UI not found")
return
}
c.Data(http.StatusOK, "text/html; charset=utf-8", data)
})
}
func serveNoCacheFile(filePath string) gin.HandlerFunc {
// serveFSDir serves a static subdirectory of root. A missing asset returns 404.
func serveFSDir(root fs.FS, dir string) gin.HandlerFunc {
sub, err := fs.Sub(root, dir)
if err != nil {
return func(c *gin.Context) { c.Status(http.StatusNotFound) }
}
handler := http.StripPrefix("/"+dir, http.FileServerFS(sub))
return func(c *gin.Context) {
setNoCacheHeaders(c)
if _, err := os.Stat(filePath); err != nil {
c.Status(http.StatusNotFound)
return
}
c.File(filePath)
handler.ServeHTTP(c.Writer, c.Request)
}
}
func serveSPAIndex(c *gin.Context, indexPath string) {
setNoCacheHeaders(c)
if _, err := os.Stat(indexPath); err != nil {
c.String(http.StatusNotFound, "MMTL web UI not found: %s", indexPath)
return
// serveFSFile serves a single root-level file (favicon / service worker) with
// no-cache headers. It reads from root, which may be the embedded SPA or disk.
func serveFSFile(root fs.FS, name string) gin.HandlerFunc {
return func(c *gin.Context) {
setNoCacheHeaders(c)
data, err := fs.ReadFile(root, name)
if err != nil {
c.Status(http.StatusNotFound)
return
}
c.Data(http.StatusOK, mimeTypeByName(name), data)
}
}
// mimeTypeByName returns an HTTP content type guessed from a file extension.
func mimeTypeByName(name string) string {
switch mime.TypeByExtension(filepath.Ext(name)) {
case "":
return "application/octet-stream"
default:
return mime.TypeByExtension(filepath.Ext(name))
}
c.File(indexPath)
}
func setNoCacheHeaders(c *gin.Context) {
@@ -137,6 +173,11 @@ func isFrontendLibraryRoute(path string) bool {
if strings.Contains(id, "/") {
return false
}
// 远程 Emby 挂载库的伪装 ID(embyremote~account~remote)也是前端库路由,
// 需要交给 SPA 而非当作 Emby API 路径 404。
if strings.HasPrefix(id, "embyremote~") {
return true
}
if len(id) != 36 {
return false
}
+3 -1
View File
@@ -3,6 +3,7 @@ module github.com/ShukeBta/MMTL
go 1.25.0
require (
github.com/aliyun/alibabacloud-oss-go-sdk-v2 v1.5.2
github.com/fsnotify/fsnotify v1.7.0
github.com/gin-gonic/gin v1.9.1
github.com/glebarez/sqlite v1.11.0
@@ -16,6 +17,7 @@ require (
go.uber.org/zap v1.27.0
golang.org/x/crypto v0.21.0
golang.org/x/sys v0.20.0
golang.org/x/time v0.15.0
gorm.io/driver/postgres v1.5.7
gorm.io/gorm v1.30.0
)
@@ -66,13 +68,13 @@ require (
github.com/tklauser/numcpus v0.6.1 // indirect
github.com/twitchyliquid64/golang-asm v0.15.1 // indirect
github.com/ugorji/go/codec v1.2.11 // indirect
github.com/ulikunitz/xz v0.5.12 // indirect
github.com/yusufpapurcu/wmi v1.2.4 // indirect
go.uber.org/multierr v1.10.0 // indirect
golang.org/x/arch v0.3.0 // indirect
golang.org/x/exp v0.0.0-20230905200255-921286631fa9 // indirect
golang.org/x/net v0.21.0 // indirect
golang.org/x/text v0.20.0 // indirect
golang.org/x/time v0.15.0 // indirect
google.golang.org/protobuf v1.31.0 // indirect
gopkg.in/ini.v1 v1.67.0 // indirect
gopkg.in/yaml.v3 v3.0.1 // indirect
+4
View File
@@ -1,3 +1,5 @@
github.com/aliyun/alibabacloud-oss-go-sdk-v2 v1.5.2 h1:40yUSXwdkWN851BHCq6uiDhleh7A4+0yIBS+IUAqZVY=
github.com/aliyun/alibabacloud-oss-go-sdk-v2 v1.5.2/go.mod h1:FTzydeQVmR24FI0D6XWUOMKckjXehM/jgMn1xC+DA9M=
github.com/bsm/ginkgo/v2 v2.12.0 h1:Ny8MWAHyOepLGlLKYmXG4IEkioBysk6GpaRTLC8zwWs=
github.com/bsm/ginkgo/v2 v2.12.0/go.mod h1:SwYbGRRDovPVboqFv0tPTcG1sN61LM1Z4ARdbAV9g4c=
github.com/bsm/gomega v1.27.10 h1:yeMWxP2pV2fG3FgAODIY8EiRE3dy0aeFYt4l7wh6yKA=
@@ -150,6 +152,8 @@ github.com/twitchyliquid64/golang-asm v0.15.1 h1:SU5vSMR7hnwNxj24w34ZyCi/FmDZTkS
github.com/twitchyliquid64/golang-asm v0.15.1/go.mod h1:a1lVb/DtPvCB8fslRZhAngC2+aY1QWCk3Cedj/Gdt08=
github.com/ugorji/go/codec v1.2.11 h1:BMaWp1Bb6fHwEtbplGBGJ498wD+LKlNSl25MjdZY4dU=
github.com/ugorji/go/codec v1.2.11/go.mod h1:UNopzCgEMSXjBc6AOMqYvWC1ktqTAfzJZUZgYf6w6lg=
github.com/ulikunitz/xz v0.5.12 h1:37Nm15o69RwBkXM0J6A5OlE67RZTfzUxTj8fB3dfcsc=
github.com/ulikunitz/xz v0.5.12/go.mod h1:nbz6k7qbPmH4IRqmfOplQw/tblSgqTqBwxkY0oWt/14=
github.com/yusufpapurcu/wmi v1.2.4 h1:zFUKzehAFReQwLys1b/iSMl+JQGSCSjtVqQn9bBrPo0=
github.com/yusufpapurcu/wmi v1.2.4/go.mod h1:SBZ9tNy3G9/m5Oi98Zks0QjeHVDvuK0qfxQmPyzfmi0=
go.uber.org/goleak v1.3.0 h1:2K3zAYmnTNqV73imy9J1T3WC+gmCePx2hEGkimedGto=
+2 -2
View File
@@ -3,8 +3,8 @@ package config
import "github.com/spf13/viper"
const (
defaultDatabaseMaxOpenConns = 4
defaultDatabaseMaxIdleConns = 2
defaultDatabaseMaxOpenConns = 16
defaultDatabaseMaxIdleConns = 4
defaultLicenseServerURL = "https://mgosever.3jzs.com"
defaultLicensePublicKey = "MCowBQYDK2VwAyEABRXnXy+urjrbKit6Yu/HiezWgP0NdsZW3tsegJWRrtI="
)
+41
View File
@@ -0,0 +1,41 @@
package config
import (
"fmt"
"os"
"gopkg.in/yaml.v3"
)
// SaveDatabaseConfig updates or creates config.yaml with the specified database configuration.
func SaveDatabaseConfig(dbType, dsn string) error {
configPath := "config.yaml"
data := make(map[string]any)
content, err := os.ReadFile(configPath)
if err == nil {
if err := yaml.Unmarshal(content, &data); err != nil {
data = make(map[string]any)
}
} else if !os.IsNotExist(err) {
return fmt.Errorf("read config.yaml: %w", err)
}
dbSection, ok := data["database"].(map[string]any)
if !ok {
dbSection = make(map[string]any)
}
dbSection["type"] = dbType
dbSection["dsn"] = dsn
data["database"] = dbSection
out, err := yaml.Marshal(data)
if err != nil {
return fmt.Errorf("marshal config.yaml: %w", err)
}
if err := os.WriteFile(configPath, out, 0644); err != nil {
return fmt.Errorf("write config.yaml: %w", err)
}
return nil
}
+36
View File
@@ -0,0 +1,36 @@
package config
import (
"os"
"path/filepath"
"testing"
)
func TestSaveDatabaseConfig(t *testing.T) {
dir := t.TempDir()
wd, _ := os.Getwd()
defer func() { _ = os.Chdir(wd) }()
if err := os.Chdir(dir); err != nil {
t.Fatalf("chdir: %v", err)
}
dsn := "postgres://admin:pass@127.0.0.1:5432/mmtl?sslmode=disable"
if err := SaveDatabaseConfig("postgres", dsn); err != nil {
t.Fatalf("SaveDatabaseConfig error: %v", err)
}
if _, err := os.Stat(filepath.Join(dir, "config.yaml")); err != nil {
t.Fatalf("expected config.yaml to exist: %v", err)
}
loaded, err := Load()
if err != nil {
t.Fatalf("Load error: %v", err)
}
if loaded.Database.Type != "postgres" {
t.Fatalf("expected database.type=postgres, got %s", loaded.Database.Type)
}
if loaded.Database.DSN != dsn {
t.Fatalf("expected dsn=%s, got %s", dsn, loaded.Database.DSN)
}
}
+223
View File
@@ -0,0 +1,223 @@
package database
import (
"context"
"fmt"
"net/url"
"strings"
"time"
"go.uber.org/zap"
"gorm.io/driver/postgres"
"gorm.io/gorm"
"gorm.io/gorm/logger"
"github.com/ShukeBta/MMTL/internal/config"
"github.com/ShukeBta/MMTL/internal/model"
)
// DatabaseStatus describes the currently active database engine and runtime metrics.
type DatabaseStatus struct {
Type string `json:"type"`
DSN string `json:"dsn,omitempty"`
DBPath string `json:"db_path,omitempty"`
OpenConns int `json:"open_conns"`
InUse int `json:"in_use"`
Idle int `json:"idle"`
MaxOpenConns int `json:"max_open_conns"`
TableCounts map[string]int64 `json:"table_counts"`
}
// PostgresTestResult returns latency and version info after testing connection.
type PostgresTestResult struct {
Success bool `json:"success"`
LatencyMS int64 `json:"latency_ms"`
Version string `json:"version,omitempty"`
Message string `json:"message,omitempty"`
Error string `json:"error,omitempty"`
}
// DatabaseMigrationResult returns row counts and execution duration of migration.
type DatabaseMigrationResult struct {
Success bool `json:"success"`
TotalRows int64 `json:"total_rows"`
TableRows map[string]int64 `json:"table_rows"`
DurationMS int64 `json:"duration_ms"`
Message string `json:"message,omitempty"`
Error string `json:"error,omitempty"`
}
// InspectDatabaseStatus queries the currently active database for metrics and table rows.
func InspectDatabaseStatus(db *gorm.DB, cfg *config.Config) *DatabaseStatus {
st := &DatabaseStatus{
Type: "sqlite",
TableCounts: make(map[string]int64),
}
if cfg != nil {
st.DBPath = cfg.Database.DBPath
if cfg.Database.Type == "postgres" || (cfg.Database.Type == "auto" && strings.TrimSpace(cfg.Database.DSN) != "") {
st.Type = "postgres"
st.DSN = MaskDSN(cfg.Database.DSN)
}
}
if isPostgres(db) {
st.Type = "postgres"
}
if db != nil {
if sqlDB, err := db.DB(); err == nil {
stats := sqlDB.Stats()
st.OpenConns = stats.OpenConnections
st.InUse = stats.InUse
st.Idle = stats.Idle
st.MaxOpenConns = stats.MaxOpenConnections
}
// Count rows for major model tables
for _, m := range model.AllModels() {
if tbl, err := modelTableName(db, m); err == nil {
if db.Migrator().HasTable(tbl) {
var count int64
if err := db.Raw("SELECT COUNT(1) FROM " + quoteIdent(tbl)).Scan(&count).Error; err == nil {
st.TableCounts[tbl] = count
}
}
}
}
}
return st
}
// TestPostgres establishes a temporary connection to verify reachability and permissions.
func TestPostgres(dsn string) (*PostgresTestResult, error) {
dsn = strings.TrimSpace(dsn)
if dsn == "" {
return &PostgresTestResult{
Success: false,
Error: "PostgreSQL DSN 不能为空",
}, nil
}
start := time.Now()
testDB, err := gorm.Open(postgres.Open(dsn), &gorm.Config{
Logger: logger.Default.LogMode(logger.Silent),
})
if err != nil {
return &PostgresTestResult{
Success: false,
Error: fmt.Sprintf("连接失败: %v", err),
}, nil
}
sqlDB, err := testDB.DB()
if err != nil {
return &PostgresTestResult{
Success: false,
Error: fmt.Sprintf("获取底层连接失败: %v", err),
}, nil
}
defer sqlDB.Close()
ctx, cancel := context.WithTimeout(context.Background(), 5*time.Second)
defer cancel()
if err := sqlDB.PingContext(ctx); err != nil {
return &PostgresTestResult{
Success: false,
Error: fmt.Sprintf("Ping 超时或失败: %v", err),
}, nil
}
var version string
if err := testDB.WithContext(ctx).Raw("SELECT version()").Scan(&version).Error; err != nil {
version = "PostgreSQL (unknown version)"
}
latency := time.Since(start).Milliseconds()
return &PostgresTestResult{
Success: true,
LatencyMS: latency,
Version: version,
Message: "连接成功",
}, nil
}
// MigrateCurrentToPostgres performs schema initialization and full table data copy into target PostgreSQL.
func MigrateCurrentToPostgres(src *gorm.DB, targetDSN string, batchSize int, log *zap.Logger) (*DatabaseMigrationResult, error) {
targetDSN = strings.TrimSpace(targetDSN)
if targetDSN == "" {
return nil, fmt.Errorf("target PostgreSQL DSN cannot be empty")
}
if src == nil {
return nil, fmt.Errorf("current database is not available")
}
started := time.Now()
targetDB, err := gorm.Open(postgres.Open(targetDSN), &gorm.Config{
Logger: newGormLogger(log),
})
if err != nil {
return nil, fmt.Errorf("open target PostgreSQL: %w", err)
}
targetSQLDB, err := targetDB.DB()
if err == nil {
defer targetSQLDB.Close()
}
// 1. 初始化目标库 Schema、类型与索引
if err := AutoMigrate(targetDB); err != nil {
return nil, fmt.Errorf("auto migrate target PostgreSQL: %w", err)
}
// 2. 安全重置目标数据库的初始默认数据
if err := resetBootstrapTargetBeforeSQLiteMigrationIfSafe(src, targetDB, log); err != nil {
return nil, fmt.Errorf("reset target bootstrap data: %w", err)
}
// 3. 执行数据批量复制
tableRows, totalRows, err := copyModelTables(src, targetDB, batchSize)
if err != nil {
return nil, fmt.Errorf("copy tables: %w", err)
}
// 4. 标记迁移完成
if err := markSQLiteMigrationComplete(targetDB); err != nil {
return nil, fmt.Errorf("mark migration complete: %w", err)
}
duration := time.Since(started).Milliseconds()
return &DatabaseMigrationResult{
Success: true,
TotalRows: totalRows,
TableRows: tableRows,
DurationMS: duration,
Message: fmt.Sprintf("成功迁移 %d 条记录至 PostgreSQL", totalRows),
}, nil
}
// MaskDSN masks the password in a connection string for safe API responses.
func MaskDSN(rawDSN string) string {
rawDSN = strings.TrimSpace(rawDSN)
if rawDSN == "" {
return ""
}
if u, err := url.Parse(rawDSN); err == nil && u.User != nil {
if pass, hasPassword := u.User.Password(); hasPassword && pass != "" {
rawUserPass := u.User.String()
user := u.User.Username()
maskedUserPass := user + ":******"
return strings.Replace(rawDSN, rawUserPass+"@", maskedUserPass+"@", 1)
}
}
// Fallback for keyword-style DSN (e.g. host=... password=...)
if strings.Contains(rawDSN, "password=") {
parts := strings.Fields(rawDSN)
for i, p := range parts {
if strings.HasPrefix(p, "password=") {
parts[i] = "password=******"
}
}
return strings.Join(parts, " ")
}
return rawDSN
}
+71
View File
@@ -0,0 +1,71 @@
package database
import (
"testing"
"github.com/glebarez/sqlite"
"gorm.io/gorm"
"github.com/ShukeBta/MMTL/internal/config"
"github.com/ShukeBta/MMTL/internal/model"
)
func TestMaskDSN(t *testing.T) {
cases := []struct {
in string
want string
}{
{
in: "postgres://admin:secret123@localhost:5432/mmtl?sslmode=disable",
want: "postgres://admin:******@localhost:5432/mmtl?sslmode=disable",
},
{
in: "host=localhost port=5432 user=admin password=secret dbname=mmtl sslmode=disable",
want: "host=localhost port=5432 user=admin password=****** dbname=mmtl sslmode=disable",
},
{
in: "sqlite://data/mmtl.db",
want: "sqlite://data/mmtl.db",
},
{
in: "",
want: "",
},
}
for _, c := range cases {
got := MaskDSN(c.in)
if got != c.want {
t.Errorf("MaskDSN(%q) = %q, want %q", c.in, got, c.want)
}
}
}
func TestInspectDatabaseStatus(t *testing.T) {
db, err := gorm.Open(sqlite.Open(":memory:"), &gorm.Config{})
if err != nil {
t.Fatal(err)
}
if err := db.AutoMigrate(&model.User{}, &model.Media{}); err != nil {
t.Fatal(err)
}
_ = db.Create(&model.User{Username: "testuser", PasswordHash: "h", Role: "user"}).Error
cfg := &config.Config{}
cfg.Database.Type = "sqlite"
cfg.Database.DBPath = "./data/mmtl.db"
st := InspectDatabaseStatus(db, cfg)
if st == nil {
t.Fatal("expected non-nil DatabaseStatus")
}
if st.Type != "sqlite" {
t.Fatalf("expected sqlite, got %s", st.Type)
}
if st.DBPath != "./data/mmtl.db" {
t.Fatalf("expected db_path, got %s", st.DBPath)
}
if st.TableCounts["users"] != 1 {
t.Fatalf("expected 1 user, got %d", st.TableCounts["users"])
}
}
+11 -11
View File
@@ -159,7 +159,7 @@ func TestCopyModelTablesMigratesExistingSQLiteRows(t *testing.T) {
t.Fatal(err)
}
copied, err := copyModelTables(src, dst, 2)
_, copied, err := copyModelTables(src, dst, 2)
if err != nil {
t.Fatal(err)
}
@@ -222,7 +222,7 @@ func TestCopyModelTablesResumesPartialSQLiteMigration(t *testing.T) {
t.Fatal(err)
}
copied, err := copyModelTables(src, dst, 2)
_, copied, err := copyModelTables(src, dst, 2)
if err != nil {
t.Fatal(err)
}
@@ -240,7 +240,7 @@ func TestCopyModelTablesResumesPartialSQLiteMigration(t *testing.T) {
t.Fatalf("genres = %q, want %q", got.Genres, media.Genres)
}
copied, err = copyModelTables(src, dst, 2)
_, copied, err = copyModelTables(src, dst, 2)
if err != nil {
t.Fatal(err)
}
@@ -332,7 +332,7 @@ func TestSQLiteMigrationFallsBackToDataDirDefaultPath(t *testing.T) {
if err := resetBootstrapTargetBeforeSQLiteMigrationIfSafe(src2, dst, nil); err != nil {
t.Fatal(err)
}
copied, err := copyModelTables(src2, dst, 2)
_, copied, err := copyModelTables(src2, dst, 2)
if err != nil {
t.Fatal(err)
}
@@ -408,13 +408,13 @@ func TestOpenSQLiteMigrationSourceUsesFallbackSourcePath(t *testing.T) {
_ = sqlDB2.Close()
}
}()
copied, err := copyModelTables(src2, dst, 2)
if err != nil {
t.Fatal(err)
}
if copied != 2 {
t.Fatalf("copied rows = %d, want 2", copied)
}
_, copied, err := copyModelTables(src2, dst, 2)
if err != nil {
t.Fatal(err)
}
if copied != 2 {
t.Fatalf("copied rows = %d, want 2", copied)
}
var userCount int64
if err := dst.Model(&model.User{}).Where("username = ?", "real-admin").Count(&userCount).Error; err != nil {
t.Fatal(err)
+1 -1
View File
@@ -48,7 +48,7 @@ func MigrateSQLiteToCurrentIfNeeded(cfg *config.Config, target *gorm.DB, log *za
if err := resetBootstrapTargetBeforeSQLiteMigrationIfSafe(src, target, log); err != nil {
return err
}
copied, err := copyModelTables(src, target, 500)
_, copied, err := copyModelTables(src, target, 500)
if err != nil {
return err
}
+16 -13
View File
@@ -13,52 +13,53 @@ import (
"github.com/ShukeBta/MMTL/internal/model"
)
func copyModelTables(src, target *gorm.DB, batchSize int) (int64, error) {
func copyModelTables(src, target *gorm.DB, batchSize int) (map[string]int64, int64, error) {
if batchSize <= 0 {
batchSize = 500
}
var copied int64
tableCounts := make(map[string]int64)
var totalCopied int64
for _, m := range model.AllModels() {
table, err := modelTableName(src, m)
if err != nil {
return copied, err
return tableCounts, totalCopied, err
}
primaryColumns, err := modelPrimaryColumns(src, m)
if err != nil {
return copied, fmt.Errorf("inspect model %T primary keys: %w", m, err)
return tableCounts, totalCopied, fmt.Errorf("inspect model %T primary keys: %w", m, err)
}
exists, err := sqliteTableExists(src, table)
if err != nil {
return copied, err
return tableCounts, totalCopied, err
}
if !exists {
continue
}
var sourceCount int64
if err := src.Raw("SELECT COUNT(1) FROM " + quoteIdent(table)).Scan(&sourceCount).Error; err != nil {
return copied, fmt.Errorf("count sqlite table %s: %w", table, err)
return tableCounts, totalCopied, fmt.Errorf("count sqlite table %s: %w", table, err)
}
if sourceCount == 0 {
continue
}
var targetCount int64
if err := target.Raw("SELECT COUNT(1) FROM " + quoteIdent(table)).Scan(&targetCount).Error; err != nil {
return copied, fmt.Errorf("count target table %s: %w", table, err)
return tableCounts, totalCopied, fmt.Errorf("count target table %s: %w", table, err)
}
modelType := reflect.TypeOf(m)
if modelType.Kind() != reflect.Ptr {
return copied, fmt.Errorf("model %T is not a pointer", m)
return tableCounts, totalCopied, fmt.Errorf("model %T is not a pointer", m)
}
sliceType := reflect.SliceOf(modelType.Elem())
slicePtr := reflect.New(sliceType)
if err := src.Unscoped().Find(slicePtr.Interface()).Error; err != nil {
return copied, fmt.Errorf("read sqlite table %s: %w", table, err)
return tableCounts, totalCopied, fmt.Errorf("read sqlite table %s: %w", table, err)
}
filtered := slicePtr.Elem()
if targetCount > 0 {
primaryKeySet, err := targetPrimaryKeySet(target, table, primaryColumns)
if err != nil {
return copied, err
return tableCounts, totalCopied, err
}
filtered = filterRowsMissingInTarget(target, table, primaryColumns, filtered, primaryKeySet)
}
@@ -68,11 +69,13 @@ func copyModelTables(src, target *gorm.DB, batchSize int) (int64, error) {
filteredPtr := reflect.New(filtered.Type())
filteredPtr.Elem().Set(filtered)
if err := target.Clauses(clause.OnConflict{DoNothing: true}).CreateInBatches(filteredPtr.Interface(), batchSize).Error; err != nil {
return copied, fmt.Errorf("copy sqlite table %s: %w", table, err)
return tableCounts, totalCopied, fmt.Errorf("copy sqlite table %s: %w", table, err)
}
copied += int64(filtered.Len())
copiedForTable := int64(filtered.Len())
tableCounts[table] = copiedForTable
totalCopied += copiedForTable
}
return copied, nil
return tableCounts, totalCopied, nil
}
func modelPrimaryColumns(db *gorm.DB, m any) ([]string, error) {
+25 -2
View File
@@ -4,6 +4,7 @@ import (
"context"
"fmt"
"path/filepath"
"strings"
"gorm.io/gorm"
@@ -32,16 +33,37 @@ func installSQLiteWriteGate(db *gorm.DB) {
gate.Unlock()
}
}
rawLock := func(tx *gorm.DB) {
if tx.Statement != nil && isReadOnlySQL(tx.Statement.SQL.String()) {
return
}
lock(tx)
}
_ = db.Callback().Create().Before("gorm:create").Register("mmtl:sqlite_write_lock", lock)
_ = db.Callback().Create().After("gorm:create").Register("mmtl:sqlite_write_unlock", unlock)
_ = db.Callback().Update().Before("gorm:update").Register("mmtl:sqlite_write_lock", lock)
_ = db.Callback().Update().After("gorm:update").Register("mmtl:sqlite_write_unlock", unlock)
_ = db.Callback().Delete().Before("gorm:delete").Register("mmtl:sqlite_write_lock", lock)
_ = db.Callback().Delete().After("gorm:delete").Register("mmtl:sqlite_write_unlock", unlock)
_ = db.Callback().Raw().Before("gorm:raw").Register("mmtl:sqlite_write_lock", lock)
_ = db.Callback().Raw().Before("gorm:raw").Register("mmtl:sqlite_write_lock", rawLock)
_ = db.Callback().Raw().After("gorm:raw").Register("mmtl:sqlite_write_unlock", unlock)
}
func isReadOnlySQL(sql string) bool {
trimmed := strings.TrimSpace(sql)
if len(trimmed) == 0 {
return false
}
upper := strings.ToUpper(trimmed)
if strings.HasPrefix(upper, "SELECT") || strings.HasPrefix(upper, "EXPLAIN") {
return true
}
if strings.HasPrefix(upper, "WITH") && !strings.Contains(upper, "INSERT") && !strings.Contains(upper, "UPDATE") && !strings.Contains(upper, "DELETE") {
return true
}
return false
}
// sqliteWriteGate serializes in-process SQLite writes while respecting the
// statement context, so request cancellation can break out of a queued write.
type sqliteWriteGate struct {
@@ -84,7 +106,7 @@ func buildSQLiteDSN(cfg *config.Config) string {
}
dsn := dbPath + "?_pragma=foreign_keys(1)"
if cfg.Database.WALMode {
dsn += "&_pragma=journal_mode(WAL)"
dsn += "&_pragma=journal_mode(WAL)&_pragma=synchronous(NORMAL)"
}
if cfg.Database.BusyTimeout > 0 {
dsn += fmt.Sprintf("&_pragma=busy_timeout(%d)", cfg.Database.BusyTimeout)
@@ -92,6 +114,7 @@ func buildSQLiteDSN(cfg *config.Config) string {
if cfg.Database.CacheSize != 0 {
dsn += fmt.Sprintf("&_pragma=cache_size(%d)", cfg.Database.CacheSize)
}
dsn += "&_pragma=temp_store(MEMORY)&_pragma=mmap_size(268435456)"
return dsn
}
+73 -11
View File
@@ -3,6 +3,7 @@ package handler
import (
"context"
"encoding/json"
"errors"
"net/http"
"strings"
@@ -27,6 +28,9 @@ func listUsersHandler(svc *service.Container) gin.HandlerFunc {
if svc.Sessions != nil {
svc.Sessions.ApplyToUsers(c.Request.Context(), users)
}
for i := range users {
users[i].PopulateComputedFields()
}
c.JSON(http.StatusOK, users)
}
}
@@ -188,19 +192,77 @@ func updateUserStatusHandler(svc *service.Container) gin.HandlerFunc {
c.JSON(http.StatusInternalServerError, gin.H{"error": err.Error()})
return
}
if req.IsActive {
_ = svc.Repo.UserDevice.SetKickedByUser(c.Request.Context(), userID, false)
} else {
_ = svc.Repo.UserDevice.SetKickedByUser(c.Request.Context(), userID, true)
if req.IsActive {
_ = svc.Repo.UserDevice.SetKickedByUser(c.Request.Context(), userID, false)
} else {
_ = svc.Repo.UserDevice.SetKickedByUser(c.Request.Context(), userID, true)
}
updated, err := svc.Repo.User.FindByID(c.Request.Context(), userID)
if err != nil {
c.JSON(http.StatusInternalServerError, gin.H{"error": err.Error()})
return
}
updated.PopulateComputedFields()
c.JSON(http.StatusOK, updated)
}
}
type adminUpdateUserLibrariesReq struct {
AllowedLibraryIDs *[]string `json:"allowed_library_ids"`
}
func updateUserLibrariesHandler(svc *service.Container) gin.HandlerFunc {
return func(c *gin.Context) {
var req adminUpdateUserLibrariesReq
if err := c.ShouldBindJSON(&req); err != nil {
c.JSON(http.StatusBadRequest, gin.H{"error": err.Error()})
return
}
userID := c.Param("id")
user, err := svc.Repo.User.FindByID(c.Request.Context(), userID)
if err != nil {
c.JSON(http.StatusInternalServerError, gin.H{"error": err.Error()})
return
}
if user == nil {
c.JSON(http.StatusNotFound, gin.H{"error": "user not found"})
return
}
var rawJSON string
if req.AllowedLibraryIDs != nil && len(*req.AllowedLibraryIDs) > 0 {
var cleanIDs []string
for _, id := range *req.AllowedLibraryIDs {
trimmed := strings.TrimSpace(id)
if trimmed != "" {
cleanIDs = append(cleanIDs, trimmed)
}
}
if len(cleanIDs) > 0 {
data, err := json.Marshal(cleanIDs)
if err != nil {
c.JSON(http.StatusInternalServerError, gin.H{"error": err.Error()})
return
}
rawJSON = string(data)
}
}
updates := map[string]any{"allowed_library_ids": rawJSON}
if err := svc.Repo.User.UpdateFields(c.Request.Context(), userID, updates); err != nil {
c.JSON(http.StatusInternalServerError, gin.H{"error": err.Error()})
return
}
updated, err := svc.Repo.User.FindByID(c.Request.Context(), userID)
if err != nil || updated == nil {
c.JSON(http.StatusInternalServerError, gin.H{"error": "failed to reload user"})
return
}
updated.PopulateComputedFields()
c.JSON(http.StatusOK, updated)
}
updated, err := svc.Repo.User.FindByID(c.Request.Context(), userID)
if err != nil {
c.JSON(http.StatusInternalServerError, gin.H{"error": err.Error()})
return
}
c.JSON(http.StatusOK, updated)
}
}
func annotateProtectedUsers(ctx context.Context, svc *service.Container, users []model.User) error {
firstAdmin, err := svc.Repo.User.FirstAdmin(ctx)
+145
View File
@@ -0,0 +1,145 @@
package handler
import (
"fmt"
"net/http"
"net/url"
"strings"
"github.com/gin-gonic/gin"
"github.com/ShukeBta/MMTL/internal/service"
)
type DatabaseConnectionPayload struct {
Type string `json:"type"`
DSN string `json:"dsn"`
Host string `json:"host"`
Port int `json:"port"`
User string `json:"user"`
Password string `json:"password"`
DBName string `json:"dbname"`
SSLMode string `json:"sslmode"`
}
func (p *DatabaseConnectionPayload) BuildDSN() string {
raw := strings.TrimSpace(p.DSN)
if raw != "" {
return raw
}
host := strings.TrimSpace(p.Host)
if host == "" {
return ""
}
port := p.Port
if port <= 0 {
port = 5432
}
user := strings.TrimSpace(p.User)
dbname := strings.TrimSpace(p.DBName)
if dbname == "" {
dbname = "mmtl"
}
sslmode := strings.TrimSpace(p.SSLMode)
if sslmode == "" {
sslmode = "disable"
}
userInfo := url.User(user)
if p.Password != "" {
userInfo = url.UserPassword(user, p.Password)
}
u := url.URL{
Scheme: "postgres",
User: userInfo,
Host: fmt.Sprintf("%s:%d", host, port),
Path: "/" + dbname,
RawQuery: "sslmode=" + url.QueryEscape(sslmode),
}
return u.String()
}
func getDatabaseStatusHandler(svc *service.Container) gin.HandlerFunc {
return func(c *gin.Context) {
if svc.Database == nil {
c.JSON(http.StatusInternalServerError, gin.H{"error": "database service unavailable"})
return
}
status := svc.Database.GetStatus(c.Request.Context())
c.JSON(http.StatusOK, status)
}
}
func testDatabaseHandler(svc *service.Container) gin.HandlerFunc {
return func(c *gin.Context) {
var req DatabaseConnectionPayload
if err := c.ShouldBindJSON(&req); err != nil {
c.JSON(http.StatusBadRequest, gin.H{"error": "invalid payload: " + err.Error()})
return
}
dsn := req.BuildDSN()
if dsn == "" {
c.JSON(http.StatusBadRequest, gin.H{"error": "请提供有效的 PostgreSQL 连接信息或 DSN"})
return
}
res, err := svc.Database.TestPostgres(c.Request.Context(), dsn)
if err != nil {
c.JSON(http.StatusInternalServerError, gin.H{"error": err.Error()})
return
}
c.JSON(http.StatusOK, res)
}
}
func migrateDatabaseHandler(svc *service.Container) gin.HandlerFunc {
return func(c *gin.Context) {
var req DatabaseConnectionPayload
if err := c.ShouldBindJSON(&req); err != nil {
c.JSON(http.StatusBadRequest, gin.H{"error": "invalid payload: " + err.Error()})
return
}
dsn := req.BuildDSN()
if dsn == "" {
c.JSON(http.StatusBadRequest, gin.H{"error": "请提供目标 PostgreSQL 连接信息或 DSN"})
return
}
res, err := svc.Database.MigrateToPostgres(c.Request.Context(), dsn)
if err != nil {
c.JSON(http.StatusInternalServerError, gin.H{"error": "迁移失败: " + err.Error()})
return
}
c.JSON(http.StatusOK, res)
}
}
func saveDatabaseConfigHandler(svc *service.Container) gin.HandlerFunc {
return func(c *gin.Context) {
var req DatabaseConnectionPayload
if err := c.ShouldBindJSON(&req); err != nil {
c.JSON(http.StatusBadRequest, gin.H{"error": "invalid payload: " + err.Error()})
return
}
dbType := strings.ToLower(strings.TrimSpace(req.Type))
if dbType == "" {
dbType = "postgres"
}
var dsn string
if dbType == "postgres" {
dsn = req.BuildDSN()
if dsn == "" {
c.JSON(http.StatusBadRequest, gin.H{"error": "请提供有效的 PostgreSQL 连接信息或 DSN"})
return
}
}
if err := svc.Database.SaveConfig(c.Request.Context(), dbType, dsn); err != nil {
c.JSON(http.StatusInternalServerError, gin.H{"error": err.Error()})
return
}
c.JSON(http.StatusOK, gin.H{
"message": "数据库配置已成功保存至配置文件,重启服务后将以新数据库运行",
"type": dbType,
})
}
}
+101
View File
@@ -0,0 +1,101 @@
package handler
import (
"bytes"
"encoding/json"
"net/http"
"net/http/httptest"
"testing"
"github.com/gin-gonic/gin"
"github.com/ShukeBta/MMTL/internal/config"
"github.com/ShukeBta/MMTL/internal/service"
)
func TestBuildDSN(t *testing.T) {
cases := []struct {
payload DatabaseConnectionPayload
want string
}{
{
payload: DatabaseConnectionPayload{
DSN: "postgres://myuser:mypass@10.0.0.1:5432/mydb?sslmode=require",
},
want: "postgres://myuser:mypass@10.0.0.1:5432/mydb?sslmode=require",
},
{
payload: DatabaseConnectionPayload{
Host: "127.0.0.1",
Port: 5432,
User: "postgres",
Password: "secretpassword",
DBName: "mmtl_prod",
SSLMode: "disable",
},
want: "postgres://postgres:secretpassword@127.0.0.1:5432/mmtl_prod?sslmode=disable",
},
}
for _, c := range cases {
got := c.payload.BuildDSN()
if got != c.want {
t.Errorf("BuildDSN() = %q, want %q", got, c.want)
}
}
}
func TestGetDatabaseStatusHandler(t *testing.T) {
gin.SetMode(gin.TestMode)
cfg := &config.Config{}
cfg.Database.Type = "sqlite"
cfg.Database.DBPath = "./data/mmtl.db"
svc := &service.Container{
Database: service.NewDatabaseAdminService(cfg, nil, nil, nil),
}
r := gin.New()
r.GET("/api/admin/database/status", getDatabaseStatusHandler(svc))
req := httptest.NewRequest(http.MethodGet, "/api/admin/database/status", nil)
rec := httptest.NewRecorder()
r.ServeHTTP(rec, req)
if rec.Code != http.StatusOK {
t.Fatalf("expected status 200, got %d: %s", rec.Code, rec.Body.String())
}
var resp map[string]any
if err := json.Unmarshal(rec.Body.Bytes(), &resp); err != nil {
t.Fatalf("unmarshal response: %v", err)
}
if resp["type"] != "sqlite" {
t.Fatalf("expected type=sqlite, got %v", resp["type"])
}
}
func TestSaveDatabaseConfigHandler(t *testing.T) {
gin.SetMode(gin.TestMode)
dir := t.TempDir()
cfg := &config.Config{}
cfg.App.DataDir = dir
cfg.Database.Type = "sqlite"
svc := &service.Container{
Database: service.NewDatabaseAdminService(cfg, nil, nil, nil),
}
r := gin.New()
r.POST("/api/admin/database/save-config", saveDatabaseConfigHandler(svc))
body := bytes.NewBufferString(`{"type":"postgres","host":"localhost","port":5432,"user":"admin","password":"pwd","dbname":"mmtl"}`)
req := httptest.NewRequest(http.MethodPost, "/api/admin/database/save-config", body)
req.Header.Set("Content-Type", "application/json")
rec := httptest.NewRecorder()
r.ServeHTTP(rec, req)
if rec.Code != http.StatusOK {
t.Fatalf("expected status 200, got %d: %s", rec.Code, rec.Body.String())
}
}
+88 -2
View File
@@ -3,6 +3,7 @@ package handler
import (
"net/http"
"net/http/httptest"
"strings"
"testing"
"github.com/gin-gonic/gin"
@@ -43,7 +44,92 @@ func TestDeleteUserRefusesRecentRealtimeSession(t *testing.T) {
if w.Code != http.StatusConflict {
t.Fatalf("status = %d body=%s", w.Code, w.Body.String())
}
if found, _ := repos.User.FindByID(t.Context(), viewer.ID); found == nil {
t.Fatal("recent realtime user should not be deleted")
if found, _ := repos.User.FindByID(t.Context(), viewer.ID); found == nil {
t.Fatal("recent realtime user should not be deleted")
}
}
func TestUpdateUserLibraries(t *testing.T) {
gin.SetMode(gin.TestMode)
db, err := gorm.Open(sqlite.Open(":memory:"), &gorm.Config{})
if err != nil {
t.Fatal(err)
}
if err := db.AutoMigrate(model.AllModels()...); err != nil {
t.Fatal(err)
}
repos := repository.New(db)
user := model.User{Base: model.Base{ID: "u1"}, Username: "alice", PasswordHash: "x", Role: "user", IsActive: true}
lib1 := model.Library{Base: model.Base{ID: "lib-1"}, Name: "电影", Type: "movie", Path: "/movie"}
lib2 := model.Library{Base: model.Base{ID: "lib-2"}, Name: "剧集", Type: "tv", Path: "/tv"}
lib3 := model.Library{Base: model.Base{ID: "lib-3"}, Name: "动漫", Type: "anime", Path: "/anime"}
if err := repos.DB.Create(&user).Error; err != nil {
t.Fatal(err)
}
if err := repos.DB.Create(&[]model.Library{lib1, lib2, lib3}).Error; err != nil {
t.Fatal(err)
}
svc := &service.Container{Repo: repos}
router := gin.New()
router.PATCH("/admin/users/:id/libraries", updateUserLibrariesHandler(svc))
// 1. 设置限制为 lib-1 和 lib-2
body := `{"allowed_library_ids":["lib-1","lib-2"]}`
req := httptest.NewRequest(http.MethodPatch, "/admin/users/u1/libraries", strings.NewReader(body))
req.Header.Set("Content-Type", "application/json")
w := httptest.NewRecorder()
router.ServeHTTP(w, req)
if w.Code != http.StatusOK {
t.Fatalf("status = %d body = %s", w.Code, w.Body.String())
}
found, err := repos.User.FindByID(t.Context(), "u1")
if err != nil || found == nil {
t.Fatal("user not found")
}
allowed := found.DecodeAllowedLibraryIDs()
if len(allowed) != 2 || allowed[0] != "lib-1" || allowed[1] != "lib-2" {
t.Fatalf("expected [lib-1, lib-2], got %v", allowed)
}
// 验证可见性
vis := service.UserDefaultMediaVisibility(t.Context(), repos, "u1")
if len(vis.AllowedLibraryIDs) != 2 {
t.Fatalf("expected 2 allowed libraries, got %v", vis.AllowedLibraryIDs)
}
if !service.LibraryVisibleForUser(t.Context(), repos, lib1, vis) {
t.Fatal("lib1 should be visible")
}
if !service.LibraryVisibleForUser(t.Context(), repos, lib2, vis) {
t.Fatal("lib2 should be visible")
}
if service.LibraryVisibleForUser(t.Context(), repos, lib3, vis) {
t.Fatal("lib3 should not be visible")
}
// 2. 清空限制,恢复全部可见
bodyEmpty := `{"allowed_library_ids":[]}`
reqEmpty := httptest.NewRequest(http.MethodPatch, "/admin/users/u1/libraries", strings.NewReader(bodyEmpty))
reqEmpty.Header.Set("Content-Type", "application/json")
wEmpty := httptest.NewRecorder()
router.ServeHTTP(wEmpty, reqEmpty)
if wEmpty.Code != http.StatusOK {
t.Fatalf("status = %d body = %s", wEmpty.Code, wEmpty.Body.String())
}
foundReset, _ := repos.User.FindByID(t.Context(), "u1")
if len(foundReset.DecodeAllowedLibraryIDs()) != 0 {
t.Fatalf("expected nil or empty, got %v", foundReset.DecodeAllowedLibraryIDs())
}
visReset := service.UserDefaultMediaVisibility(t.Context(), repos, "u1")
if len(visReset.AllowedLibraryIDs) != 0 {
t.Fatalf("expected no library restrictions, got %v", visReset.AllowedLibraryIDs)
}
if !service.LibraryVisibleForUser(t.Context(), repos, lib3, visReset) {
t.Fatal("lib3 should now be visible")
}
}
+1
View File
@@ -89,6 +89,7 @@ func meHandler(svc *service.Container) gin.HandlerFunc {
c.JSON(http.StatusNotFound, gin.H{"error": "not found"})
return
}
u.PopulateComputedFields()
c.JSON(http.StatusOK, u)
}
}
+198
View File
@@ -0,0 +1,198 @@
// Emby 挂载管理 HTTP 层:远程 Emby 服务器(账号)下的媒体库挂载 CRUD,
// 以及账号远程媒体库(View)列表预览。
package handler
import (
"net/http"
"strings"
"github.com/gin-gonic/gin"
"github.com/ShukeBta/MMTL/internal/model"
"github.com/ShukeBta/MMTL/internal/service"
)
// embyMountView 挂载的对外 JSON(附带账号信息)。
type embyMountView struct {
model.EmbyMount
AccountName string `json:"account_name"`
}
// embyMountInput 创建挂载的请求体(单个或批量)。
type embyMountInput struct {
AccountID string `json:"account_id" binding:"required"`
Views []embyViewInput `json:"views" binding:"required,min=1"`
}
type embyViewInput struct {
RemoteViewID string `json:"remote_view_id" binding:"required"`
RemoteViewName string `json:"remote_view_name"`
CollectionType string `json:"collection_type"`
Name string `json:"name"`
ProxyPlay bool `json:"proxy_play"`
}
func embyMountViews(mounts []model.EmbyMount, accounts map[string]string) []embyMountView {
out := make([]embyMountView, 0, len(mounts))
for _, m := range mounts {
out = append(out, embyMountView{EmbyMount: m, AccountName: accounts[m.AccountID]})
}
return out
}
// embyAccountViewsHandler 列出账号上的远程媒体库(View),供挂载选择。
func embyAccountViewsHandler(svc *service.Container) gin.HandlerFunc {
return func(c *gin.Context) {
acct := svc.EmbyRemote.AccountByID(c.Request.Context(), c.Param("id"))
if acct == nil {
c.JSON(http.StatusNotFound, gin.H{"error": "账号不存在或已禁用"})
return
}
views, err := svc.EmbyRemote.RemoteViews(c.Request.Context(), acct)
if err != nil {
c.JSON(http.StatusBadGateway, gin.H{"error": err.Error()})
return
}
type viewEntry struct {
RemoteViewID string `json:"remote_view_id"`
RemoteViewName string `json:"remote_view_name"`
CollectionType string `json:"collection_type"`
ChildCount int `json:"child_count"`
AlreadyMounted bool `json:"already_mounted"`
}
mounted := map[string]bool{}
if mounts, err := svc.EmbyRemote.ListMountsByAccount(c.Request.Context(), acct.ID); err == nil {
for _, m := range mounts {
mounted[m.RemoteViewID] = true
}
}
out := make([]viewEntry, 0, len(views))
for _, v := range views {
viewID := service.RemoteItemIDString(v)
if strings.TrimSpace(viewID) == "" {
continue
}
out = append(out, viewEntry{
RemoteViewID: viewID,
RemoteViewName: service.RemoteItemNameString(v),
CollectionType: service.RemoteItemCollectionType(v),
ChildCount: service.RemoteItemChildCount(v),
AlreadyMounted: mounted[viewID],
})
}
c.JSON(http.StatusOK, out)
}
}
// listEmbyMountsHandler 列出全部挂载。
func listEmbyMountsHandler(svc *service.Container) gin.HandlerFunc {
return func(c *gin.Context) {
mounts, err := svc.EmbyRemote.ListMounts(c.Request.Context())
if err != nil {
c.JSON(http.StatusInternalServerError, gin.H{"error": err.Error()})
return
}
names := map[string]string{}
if accounts, err := svc.EmbyRemote.ListAccounts(c.Request.Context()); err == nil {
for _, a := range accounts {
names[a.ID] = a.Name
}
}
out := embyMountViews(mounts, names)
if out == nil {
out = []embyMountView{}
}
c.JSON(http.StatusOK, out)
}
}
// createEmbyMountsHandler 批量创建挂载(同一账号下的多个远程媒体库)。
func createEmbyMountsHandler(svc *service.Container) gin.HandlerFunc {
return func(c *gin.Context) {
var req embyMountInput
if err := c.ShouldBindJSON(&req); err != nil {
c.JSON(http.StatusBadRequest, gin.H{"error": err.Error()})
return
}
mounts := make([]*model.EmbyMount, 0, len(req.Views))
for _, v := range req.Views {
mounts = append(mounts, &model.EmbyMount{
AccountID: req.AccountID,
RemoteViewID: v.RemoteViewID,
RemoteViewName: v.RemoteViewName,
CollectionType: v.CollectionType,
Name: v.Name,
ProxyPlay: v.ProxyPlay,
Enabled: true,
})
}
if _, err := svc.EmbyRemote.CreateMounts(c.Request.Context(), mounts); err != nil {
c.JSON(http.StatusBadRequest, gin.H{"error": err.Error()})
return
}
c.JSON(http.StatusOK, gin.H{"ok": true, "created": len(mounts)})
}
}
// fullMountEmbyAccountHandler 全量挂载:把账号所有远程媒体库一次挂载进来。
func fullMountEmbyAccountHandler(svc *service.Container) gin.HandlerFunc {
return func(c *gin.Context) {
acct := svc.EmbyRemote.AccountByID(c.Request.Context(), c.Param("id"))
if acct == nil {
c.JSON(http.StatusNotFound, gin.H{"error": "账号不存在或已禁用"})
return
}
proxy := c.Query("proxy") == "1" || c.Query("proxy") == "true"
n, err := svc.EmbyRemote.FullMountAccount(c.Request.Context(), acct, proxy)
if err != nil {
c.JSON(http.StatusBadGateway, gin.H{"error": err.Error()})
return
}
c.JSON(http.StatusOK, gin.H{"ok": true, "created": n})
}
}
// updateEmbyMountHandler 更新挂载(显示名 / 代理开关 / 启用)。
func updateEmbyMountHandler(svc *service.Container) gin.HandlerFunc {
return func(c *gin.Context) {
var req struct {
Name *string `json:"name"`
ProxyPlay *bool `json:"proxy_play"`
Enabled *bool `json:"enabled"`
}
if err := c.ShouldBindJSON(&req); err != nil {
c.JSON(http.StatusBadRequest, gin.H{"error": err.Error()})
return
}
mount, err := svc.EmbyRemote.MountByID(c.Request.Context(), c.Param("id"))
if err != nil || mount == nil {
c.JSON(http.StatusNotFound, gin.H{"error": "挂载不存在"})
return
}
if req.Name != nil {
mount.Name = *req.Name
}
if req.ProxyPlay != nil {
mount.ProxyPlay = *req.ProxyPlay
}
if req.Enabled != nil {
mount.Enabled = *req.Enabled
}
if _, err := svc.EmbyRemote.UpdateMount(c.Request.Context(), mount.ID, mount); err != nil {
c.JSON(http.StatusInternalServerError, gin.H{"error": err.Error()})
return
}
c.JSON(http.StatusOK, mount)
}
}
// deleteEmbyMountHandler 删除挂载。
func deleteEmbyMountHandler(svc *service.Container) gin.HandlerFunc {
return func(c *gin.Context) {
if err := svc.EmbyRemote.DeleteMount(c.Request.Context(), c.Param("id")); err != nil {
c.JSON(http.StatusBadRequest, gin.H{"error": err.Error()})
return
}
c.JSON(http.StatusOK, gin.H{"ok": true})
}
}
+37 -5
View File
@@ -34,10 +34,19 @@ func embyPlaybackInfoHandler(svc *service.Container) gin.HandlerFunc {
// embySubtitleStreamHandler serves an external subtitle track advertised in a
// MediaSource's MediaStreams via its Emby index
// (/Videos/:id/Subtitles/:index/Stream). The index maps to a discovered
// sideloaded subtitle file next to the video (SRT/ASS/SSA/VTT, local or
// cloud://), following the same layout appended by mediaStreams.
// sideloaded subtitle track next to the video (SRT/ASS/SSA/VTT, local or
// cloud://), following the same layout appended by mediaStreams. 远程 Emby
// 条目的字幕直接反向代理远程。
func embySubtitleStreamHandler(svc *service.Container) gin.HandlerFunc {
return func(c *gin.Context) {
encodedID := c.Param("id")
if accountID, remoteID, ok := service.DecodeEmbyRemoteID(encodedID); ok {
if err := svc.Emby.ProxyRemoteSubtitle(c.Request.Context(), c.Writer, c.Request, accountID, remoteID, c.Param("index")); err != nil {
embyError(c, http.StatusNotFound, "subtitle not found")
return
}
return
}
uid := c.Param("userId")
if uid == "" {
uid = embyUserID(c)
@@ -213,12 +222,26 @@ func embyAppendAPIKey(raw, token string) string {
return u.String()
}
// embyVideoStreamHandler 是 GET /Videos/{id}/stream 的入口,
// 直接代理到我们的 /api/stream/{id}(同一个 ServeFile)。
// embyVideoStreamHandler 是 GET /Videos/{id}/stream 的入口。
// 远程 Emby 条目(embyremote~ 前缀)走反向代理;本地条目直接代理到
// /api/stream/{id}(同一个 ServeFile)。
func embyVideoStreamHandler(svc *service.Container, cloudMode string) gin.HandlerFunc {
return func(c *gin.Context) {
encodedID := c.Param("id")
if accountID, remoteID, ok := service.DecodeEmbyRemoteID(encodedID); ok {
if err := svc.Emby.ProxyRemoteVideoStream(c.Request.Context(), c.Writer, c.Request, accountID, remoteID); err != nil {
if errors.Is(err, service.ErrEmbyRemoteNotFound) {
c.Status(http.StatusNotFound)
return
}
if !c.Writer.Written() {
c.JSON(http.StatusBadGateway, gin.H{"error": err.Error()})
}
}
return
}
uid := embyUserID(c)
item, err := svc.Emby.Item(c.Request.Context(), c.Param("id"), uid)
item, err := svc.Emby.Item(c.Request.Context(), encodedID, uid)
if err != nil {
c.JSON(http.StatusInternalServerError, gin.H{"error": err.Error()})
return
@@ -296,6 +319,11 @@ func embyShouldRedirectVideoStreamToSTRM(c *gin.Context, svc *service.Container,
func embyVideoHLSPlaylistHandler(svc *service.Container) gin.HandlerFunc {
return func(c *gin.Context) {
// 远程 Emby 条目不做本地转码(播放地址已由 PlaybackInfo 指向远程/代理直连)。
if service.IsEmbyRemoteID(c.Param("id")) {
c.Status(http.StatusNotFound)
return
}
uid := embyUserID(c)
item, err := svc.Emby.Item(c.Request.Context(), c.Param("id"), uid)
if err != nil || item == nil || svc.Stream == nil {
@@ -319,6 +347,10 @@ func embyVideoHLSPlaylistHandler(svc *service.Container) gin.HandlerFunc {
func embyVideoHLSSegmentHandler(svc *service.Container) gin.HandlerFunc {
return func(c *gin.Context) {
if service.IsEmbyRemoteID(c.Param("id")) {
c.Status(http.StatusNotFound)
return
}
uid := embyUserID(c)
item, err := svc.Emby.Item(c.Request.Context(), c.Param("id"), uid)
if err != nil || item == nil || svc.Stream == nil {
+268 -21
View File
@@ -7,6 +7,7 @@ import (
"net/http"
"strconv"
"strings"
"sync"
"github.com/gin-gonic/gin"
@@ -16,17 +17,41 @@ import (
)
type createLibraryReq struct {
Name string `json:"name" binding:"required"`
Path string `json:"path"`
Paths []string `json:"paths"`
Roots []service.LibraryRootInput `json:"roots"`
Type string `json:"type"`
CoverURL string `json:"cover_url"`
Name string `json:"name"`
Path string `json:"path"`
Paths []string `json:"paths"`
Roots []service.LibraryRootInput `json:"roots"`
Type string `json:"type"`
CoverURL string `json:"cover_url"`
CreatePerSubfolder bool `json:"create_per_subfolder"`
}
// webLibraryPayload 是 /api/libraries 返回的库条目:本地库与远程 Emby 挂载库
// 统一结构(远程库附加 is_remote_emby / remote_source 只读标记)。
type webLibraryPayload struct {
model.Library
IsRemoteEmby bool `json:"is_remote_emby,omitempty"`
RemoteSource string `json:"remote_source,omitempty"`
Total int64 `json:"total,omitempty"`
Cards []service.SeriesCard `json:"cards,omitempty"`
}
// remoteLibraryItemTypes 远程库内容拉取时按 CollectionType 过滤直属条目,
// 避免电影库里的合集文件夹(Folder) 漏出为电影卡片。
func remoteLibraryItemTypes(collectionType string) string {
switch collectionType {
case "movies":
return "Movie"
case "tvshows":
return "Series"
}
return ""
}
func listLibrariesHandler(svc *service.Container) gin.HandlerFunc {
return func(c *gin.Context) {
libs, err := svc.Media.ListLibraries(c.Request.Context())
ctx := c.Request.Context()
libs, err := svc.Media.ListLibraries(ctx)
if err != nil {
c.JSON(http.StatusInternalServerError, gin.H{"error": err.Error()})
return
@@ -34,23 +59,102 @@ func listLibrariesHandler(svc *service.Container) gin.HandlerFunc {
role, _ := c.Get(middleware.CtxUserRole)
includeHidden := role == "admin" && (c.Query("include_hidden") == "1" || c.Query("include_hidden") == "true" || c.Query("all") == "1")
if !includeHidden {
libs = service.FilterDisplayCloudLibraries(c.Request.Context(), svc.Repo, libs)
libs = service.FilterDisplayCloudLibraries(ctx, svc.Repo, libs)
visibility := mediaVisibilityForRequest(c, svc)
filtered := libs[:0]
for _, lib := range libs {
if service.LibraryVisibleForUser(c.Request.Context(), svc.Repo, lib, visibility) {
if service.LibraryVisibleForUser(ctx, svc.Repo, lib, visibility) {
filtered = append(filtered, lib)
}
}
libs = filtered
}
c.JSON(http.StatusOK, libs)
withPreview := c.Query("with_preview") == "1" || c.Query("with_preview") == "true"
limit := 10
if withPreview {
limit, _ = strconv.Atoi(c.DefaultQuery("preview_limit", c.DefaultQuery("limit", "10")))
if limit <= 0 {
limit = 10
} else if limit > 100 {
limit = 100
}
}
out := make([]webLibraryPayload, 0, len(libs)+8)
if withPreview {
previews, err := svc.Media.ListLibrariesWithPreview(ctx, libs, mediaVisibilityForRequest(c, svc), limit)
if err != nil {
c.JSON(http.StatusInternalServerError, gin.H{"error": err.Error()})
return
}
for _, p := range previews {
out = append(out, webLibraryPayload{Library: p.Library, Total: p.Total, Cards: p.Cards})
}
} else {
for _, l := range libs {
out = append(out, webLibraryPayload{Library: l})
}
}
// 远程 Emby 挂载库追加在本地库之后。
if svc.EmbyRemote != nil {
if views, err := svc.EmbyRemote.RemoteLibraries(ctx); err == nil {
remotePayloads := make([]webLibraryPayload, len(views))
for i, v := range views {
remotePayloads[i] = webLibraryPayload{Library: v.Library, IsRemoteEmby: true, RemoteSource: v.AccountName}
}
if withPreview && len(views) > 0 {
const maxRemotePreviewWorkers = 6
sem := make(chan struct{}, maxRemotePreviewWorkers)
var wg sync.WaitGroup
for i, v := range views {
i, v := i, v
wg.Add(1)
go func() {
defer wg.Done()
select {
case sem <- struct{}{}:
defer func() { <-sem }()
case <-ctx.Done():
return
}
acct := svc.EmbyRemote.AccountByID(ctx, v.AccountID)
if acct == nil {
return
}
tmpMount := &model.EmbyMount{Base: model.Base{ID: v.MountID}}
itemTypes := remoteLibraryItemTypes(v.CollectionType)
if _, total, err := svc.EmbyRemote.RemoteLibraryMedia(ctx, tmpMount, acct, v.RemoteID, itemTypes, 0, 1); err == nil {
remotePayloads[i].Total = total
}
if cards, err := svc.EmbyRemote.RemoteLatestCards(ctx, tmpMount, acct, v.RemoteID, limit); err == nil {
remotePayloads[i].Cards = cards
}
}()
}
wg.Wait()
}
out = append(out, remotePayloads...)
}
}
c.JSON(http.StatusOK, out)
}
}
func getLibraryHandler(svc *service.Container) gin.HandlerFunc {
return func(c *gin.Context) {
lib, err := svc.Repo.Library.FindByID(c.Request.Context(), c.Param("id"))
ctx := c.Request.Context()
id := c.Param("id")
// 远程 Emby 挂载库详情。
if svc.EmbyRemote != nil && service.IsEmbyRemoteID(id) {
mountID, remoteID, _ := service.DecodeEmbyRemoteID(id)
view, err := svc.EmbyRemote.RemoteLibraryByID(ctx, mountID, remoteID)
if err != nil || view == nil {
c.JSON(http.StatusNotFound, gin.H{"error": "not found"})
return
}
c.JSON(http.StatusOK, webLibraryPayload{Library: view.Library, IsRemoteEmby: true, RemoteSource: view.AccountName})
return
}
lib, err := svc.Repo.Library.FindByID(ctx, id)
if err != nil {
c.JSON(http.StatusInternalServerError, gin.H{"error": err.Error()})
return
@@ -62,14 +166,14 @@ func getLibraryHandler(svc *service.Container) gin.HandlerFunc {
role, _ := c.Get(middleware.CtxUserRole)
includeHidden := role == "admin" && (c.Query("include_hidden") == "1" || c.Query("include_hidden") == "true" || c.Query("all") == "1")
if !includeHidden {
libs := service.FilterDisplayCloudLibraries(c.Request.Context(), svc.Repo, []model.Library{*lib})
if len(libs) == 0 || !service.LibraryVisibleForUser(c.Request.Context(), svc.Repo, libs[0], mediaVisibilityForRequest(c, svc)) {
libs := service.FilterDisplayCloudLibraries(ctx, svc.Repo, []model.Library{*lib})
if len(libs) == 0 || !service.LibraryVisibleForUser(ctx, svc.Repo, libs[0], mediaVisibilityForRequest(c, svc)) {
c.JSON(http.StatusNotFound, gin.H{"error": "not found"})
return
}
c.JSON(http.StatusOK, libs[0])
c.JSON(http.StatusOK, webLibraryPayload{Library: libs[0]})
} else {
c.JSON(http.StatusOK, lib)
c.JSON(http.StatusOK, webLibraryPayload{Library: *lib})
}
}
}
@@ -90,6 +194,35 @@ func createLibraryHandler(svc *service.Container) gin.HandlerFunc {
if len(roots) == 0 && strings.TrimSpace(req.Path) != "" {
roots = append(roots, service.LibraryRootInput{Path: req.Path})
}
var l *model.Library
if req.CreatePerSubfolder {
parent := ""
if len(roots) > 0 {
parent = roots[0].Path
} else if strings.TrimSpace(req.Path) != "" {
parent = req.Path
}
created, err := svc.Media.CreateLibrariesPerSubfolder(c.Request.Context(), parent, req.Type, req.CoverURL)
if err != nil {
c.JSON(http.StatusInternalServerError, gin.H{"error": err.Error()})
return
}
uid, _ := c.Get("ctx_user_id")
for i := range created {
lib := &created[i]
svc.Audit.Record(c.Request.Context(), toString(uid), "library.create", lib.ID, c.ClientIP(), lib.Path)
if svc.Watcher != nil {
go func() { _ = svc.Watcher.Refresh(context.Background()) }()
}
for _, root := range lib.Roots {
if root.Enabled {
queueLibraryRootScan(svc, lib.ID, root.ID)
}
}
}
c.JSON(http.StatusCreated, gin.H{"libraries": created})
return
}
l, err := svc.Media.CreateLibraryWithRootsAndCover(c.Request.Context(), req.Name, req.Type, req.CoverURL, roots)
if err != nil {
c.JSON(http.StatusInternalServerError, gin.H{"error": err.Error()})
@@ -117,7 +250,9 @@ func createLibraryHandler(svc *service.Container) gin.HandlerFunc {
}
type updateLibraryReq struct {
CoverURL string `json:"cover_url"`
CoverURL *string `json:"cover_url"`
SortOrder *int `json:"sort_order"`
CarouselEnabled *bool `json:"carousel_enabled"`
}
func updateLibraryHandler(svc *service.Container) gin.HandlerFunc {
@@ -127,9 +262,17 @@ func updateLibraryHandler(svc *service.Container) gin.HandlerFunc {
c.JSON(http.StatusBadRequest, gin.H{"error": err.Error()})
return
}
if err := svc.Media.UpdateLibraryCover(c.Request.Context(), c.Param("id"), req.CoverURL); err != nil {
c.JSON(http.StatusInternalServerError, gin.H{"error": err.Error()})
return
if req.CoverURL != nil {
if err := svc.Media.UpdateLibraryCover(c.Request.Context(), c.Param("id"), *req.CoverURL); err != nil {
c.JSON(http.StatusInternalServerError, gin.H{"error": err.Error()})
return
}
}
if req.SortOrder != nil || req.CarouselEnabled != nil {
if err := svc.Media.UpdateLibraryFields(c.Request.Context(), c.Param("id"), req.SortOrder, req.CarouselEnabled); err != nil {
c.JSON(http.StatusInternalServerError, gin.H{"error": err.Error()})
return
}
}
lib, err := svc.Repo.Library.FindByID(c.Request.Context(), c.Param("id"))
if err != nil || lib == nil {
@@ -140,6 +283,25 @@ func updateLibraryHandler(svc *service.Container) gin.HandlerFunc {
}
}
type reorderLibrariesReq struct {
IDs []string `json:"ids" binding:"required"`
}
func reorderLibrariesHandler(svc *service.Container) gin.HandlerFunc {
return func(c *gin.Context) {
var req reorderLibrariesReq
if err := c.ShouldBindJSON(&req); err != nil {
c.JSON(http.StatusBadRequest, gin.H{"error": err.Error()})
return
}
if err := svc.Media.ReorderLibraries(c.Request.Context(), req.IDs); err != nil {
c.JSON(http.StatusInternalServerError, gin.H{"error": err.Error()})
return
}
c.JSON(http.StatusOK, gin.H{"updated": len(req.IDs)})
}
}
func deleteLibraryHandler(svc *service.Container) gin.HandlerFunc {
return func(c *gin.Context) {
id := c.Param("id")
@@ -157,8 +319,37 @@ func deleteLibraryHandler(svc *service.Container) gin.HandlerFunc {
func listMediaHandler(svc *service.Container) gin.HandlerFunc {
return func(c *gin.Context) {
id := c.Param("id")
ctx := c.Request.Context()
page, _ := strconv.Atoi(c.DefaultQuery("page", "1"))
size, _ := strconv.Atoi(c.DefaultQuery("page_size", "50"))
// 远程 Emby 库:转发远程直属条目并映射为本地 Media 结构(分页由远程承接)。
if svc.EmbyRemote != nil && service.IsEmbyRemoteID(id) {
mountID, remoteID, _ := service.DecodeEmbyRemoteID(id)
mount, acct, _ := svc.EmbyRemote.ResolveMount(ctx, mountID)
if mount == nil || acct == nil {
c.JSON(http.StatusNotFound, gin.H{"error": "not found"})
return
}
itemTypes := ""
if view, err := svc.EmbyRemote.RemoteLibraryByID(ctx, mountID, remoteID); err == nil && view != nil {
itemTypes = remoteLibraryItemTypes(view.CollectionType)
}
items, total, err := svc.EmbyRemote.RemoteLibraryMedia(ctx, mount, acct, remoteID, itemTypes, (page-1)*size, size)
if err != nil {
c.JSON(http.StatusInternalServerError, gin.H{"error": err.Error()})
return
}
if items == nil {
items = []model.Media{}
}
c.JSON(http.StatusOK, gin.H{
"items": items,
"total": total,
"page": page,
"page_size": size,
})
return
}
groupVersions := c.DefaultQuery("group_versions", "1") != "0"
if !groupVersions {
items, total, err := svc.Media.ListMediaVisible(c.Request.Context(), id, page, size, mediaVisibilityForRequest(c, svc))
@@ -196,7 +387,33 @@ func listMediaHandler(svc *service.Container) gin.HandlerFunc {
func getMediaHandler(svc *service.Container) gin.HandlerFunc {
return func(c *gin.Context) {
m, err := svc.Media.GetMedia(c.Request.Context(), c.Param("id"))
ctx := c.Request.Context()
id := c.Param("id")
// 远程 Emby 条目:拉远程详情并映射为本地 Media 结构。
if svc.EmbyRemote != nil && service.IsEmbyRemoteID(id) {
mountID, remoteID, _ := service.DecodeEmbyRemoteID(id)
mount, acct, _ := svc.EmbyRemote.ResolveMount(ctx, mountID)
if mount == nil || acct == nil {
c.JSON(http.StatusNotFound, gin.H{"error": "not found"})
return
}
m, err := svc.EmbyRemote.RemoteMediaDetail(ctx, mount, acct, remoteID)
if err != nil {
c.JSON(http.StatusInternalServerError, gin.H{"error": err.Error()})
return
}
if m == nil {
c.JSON(http.StatusNotFound, gin.H{"error": "not found"})
return
}
if !mediaVisibleForRequest(c, svc, m) {
c.JSON(http.StatusNotFound, gin.H{"error": "not found"})
return
}
c.JSON(http.StatusOK, m)
return
}
m, err := svc.Media.GetMedia(ctx, id)
if err != nil {
c.JSON(http.StatusInternalServerError, gin.H{"error": err.Error()})
return
@@ -290,7 +507,37 @@ func searchMediaHandler(svc *service.Container) gin.HandlerFunc {
func streamHandler(svc *service.Container) gin.HandlerFunc {
return func(c *gin.Context) {
m, err := svc.Media.GetMedia(c.Request.Context(), c.Param("id"))
ctx := c.Request.Context()
id := c.Param("id")
// 远程 Emby 条目:按挂载代理配置分流——代理走 MMTL 反代,否则 302 直连。
if svc.EmbyRemote != nil && service.IsEmbyRemoteID(id) {
if !enforceScopedPlaybackToken(c, id) {
return
}
mountID, remoteID, _ := service.DecodeEmbyRemoteID(id)
mount, acct, _ := svc.EmbyRemote.ResolveMount(ctx, mountID)
if mount == nil || acct == nil {
c.JSON(http.StatusNotFound, gin.H{"error": "not found"})
return
}
if mount.ProxyPlay {
if err := svc.Emby.ProxyRemoteVideoStream(ctx, c.Writer, c.Request, mountID, remoteID); err != nil {
if !c.Writer.Written() {
c.JSON(http.StatusBadGateway, gin.H{"error": err.Error()})
}
}
return
}
target, err := svc.EmbyRemote.WebStreamURL(ctx, acct, remoteID)
if err != nil {
c.JSON(http.StatusBadGateway, gin.H{"error": err.Error()})
return
}
setRedirectNoStoreHeaders(c)
c.Redirect(http.StatusFound, target)
return
}
m, err := svc.Media.GetMedia(ctx, id)
if err != nil || m == nil || !mediaVisibleForRequest(c, svc, m) {
c.JSON(http.StatusNotFound, gin.H{"error": "not found"})
return
+25 -4
View File
@@ -19,21 +19,40 @@ import (
"github.com/ShukeBta/MMTL/internal/service"
)
func findMediaForPlaybackEndpoint(c *gin.Context, svc *service.Container, id string) (*model.Media, error) {
ctx := c.Request.Context()
if svc.EmbyRemote != nil && service.IsEmbyRemoteID(id) {
mountID, remoteID, _ := service.DecodeEmbyRemoteID(id)
mount, acct, _ := svc.EmbyRemote.ResolveMount(ctx, mountID)
if mount == nil || acct == nil {
return nil, nil
}
return svc.EmbyRemote.RemoteMediaDetail(ctx, mount, acct, remoteID)
}
return svc.Repo.Media.FindByID(ctx, id)
}
// playbackInfoHandler returns the media row + a `stream_url` the React
// player can hit. Mirrors the Python project's surface.
func playbackInfoHandler(svc *service.Container) gin.HandlerFunc {
return func(c *gin.Context) {
m, err := svc.Repo.Media.FindByID(c.Request.Context(), c.Param("id"))
id := c.Param("id")
m, err := findMediaForPlaybackEndpoint(c, svc, id)
if err != nil || m == nil || !mediaVisibleForRequest(c, svc, m) {
c.JSON(http.StatusNotFound, gin.H{"error": "media not found"})
return
}
token := externalPlaybackToken(c, svc, m.ID, m.DurationSec)
profileQuery := externalProfileQuery(c)
hlsURL := "/api/hls/" + m.ID + "/index.m3u8?token=" + url.QueryEscape(token) + profileQuery
if service.IsEmbyRemoteID(m.ID) || service.IsStrmMediaRow(m) {
// Emby 远程挂载与 STRM 媒体一样,默认直连播放,不提供转码地址
hlsURL = ""
}
c.JSON(http.StatusOK, gin.H{
"media": m,
"stream_url": "/api/stream/" + m.ID + "?token=" + url.QueryEscape(token) + profileQuery,
"hls_url": "/api/hls/" + m.ID + "/index.m3u8?token=" + url.QueryEscape(token) + profileQuery,
"hls_url": hlsURL,
})
}
}
@@ -68,7 +87,8 @@ func playbackProgressHandler(svc *service.Container) gin.HandlerFunc {
// produce the per-player launch URL.
func externalPlayersHandler(svc *service.Container) gin.HandlerFunc {
return func(c *gin.Context) {
m, err := svc.Repo.Media.FindByID(c.Request.Context(), c.Param("id"))
id := c.Param("id")
m, err := findMediaForPlaybackEndpoint(c, svc, id)
if err != nil || m == nil || !mediaVisibleForRequest(c, svc, m) {
c.JSON(http.StatusNotFound, gin.H{"error": "media not found"})
return
@@ -93,7 +113,8 @@ func externalPlayersHandler(svc *service.Container) gin.HandlerFunc {
// token query string the external player needs.
func externalURLHandler(svc *service.Container) gin.HandlerFunc {
return func(c *gin.Context) {
m, err := svc.Repo.Media.FindByID(c.Request.Context(), c.Param("id"))
id := c.Param("id")
m, err := findMediaForPlaybackEndpoint(c, svc, id)
if err != nil || m == nil || !mediaVisibleForRequest(c, svc, m) {
c.JSON(http.StatusNotFound, gin.H{"error": "media not found"})
return
+77 -7
View File
@@ -401,11 +401,81 @@ func newPlaybackScopeTestRouter(t *testing.T) (*gin.Engine, *service.Container,
t.Fatal(err)
}
router := gin.New()
api := router.Group("/api")
api.Use(middleware.AuthRequired(cfg.Secrets.JWTSecret))
api.GET("/playback/:id/external-url", externalURLHandler(svc))
api.GET("/playback/:id/external-players", externalPlayersHandler(svc))
api.GET("/stream/:id", streamHandler(svc))
return router, svc, cfg.Secrets.JWTSecret
router := gin.New()
api := router.Group("/api")
api.Use(middleware.AuthRequired(cfg.Secrets.JWTSecret))
api.GET("/playback/:id/info", playbackInfoHandler(svc))
api.GET("/playback/:id/external-url", externalURLHandler(svc))
api.GET("/playback/:id/external-players", externalPlayersHandler(svc))
api.GET("/stream/:id", streamHandler(svc))
api.GET("/hls/:id/index.m3u8", hlsPlaylistHandler(svc))
api.GET("/media/:id/subtitles", listSubtitlesHandler(svc))
return router, svc, cfg.Secrets.JWTSecret
}
func TestPlaybackInfoForSTRMMediaDisablesHLS(t *testing.T) {
router, _, secret := newPlaybackScopeTestRouter(t)
loginToken := signedTestToken(t, secret)
req := httptest.NewRequest(http.MethodGet, "http://nas.local/api/playback/media-1/info", nil)
req.Header.Set("Authorization", "Bearer "+loginToken)
w := httptest.NewRecorder()
router.ServeHTTP(w, req)
if w.Code != http.StatusOK {
t.Fatalf("status = %d body=%s", w.Code, w.Body.String())
}
var payload struct {
StreamURL string `json:"stream_url"`
HlsURL string `json:"hls_url"`
}
if err := json.Unmarshal(w.Body.Bytes(), &payload); err != nil {
t.Fatalf("decode: %v", err)
}
if payload.StreamURL == "" {
t.Fatalf("expected non-empty stream_url")
}
if payload.HlsURL != "" {
t.Fatalf("expected empty hls_url for STRM media, got %q", payload.HlsURL)
}
}
func TestHLSPlaylistForRemoteEmbyMediaDisabled(t *testing.T) {
router, svc, secret := newPlaybackScopeTestRouter(t)
svc.EmbyRemote = &service.EmbyRemoteService{}
loginToken := signedTestToken(t, secret)
req := httptest.NewRequest(http.MethodGet, "http://nas.local/api/hls/embyremote~acct1~item1/index.m3u8", nil)
req.Header.Set("Authorization", "Bearer "+loginToken)
w := httptest.NewRecorder()
router.ServeHTTP(w, req)
if w.Code != http.StatusConflict {
t.Fatalf("status = %d, want %d (409 StatusConflict)", w.Code, http.StatusConflict)
}
}
func TestListSubtitlesForRemoteEmbyMediaReturnsEmptyTracks(t *testing.T) {
router, svc, secret := newPlaybackScopeTestRouter(t)
svc.EmbyRemote = &service.EmbyRemoteService{}
loginToken := signedTestToken(t, secret)
req := httptest.NewRequest(http.MethodGet, "http://nas.local/api/media/embyremote~acct1~item1/subtitles", nil)
req.Header.Set("Authorization", "Bearer "+loginToken)
w := httptest.NewRecorder()
router.ServeHTTP(w, req)
if w.Code != http.StatusOK {
t.Fatalf("status = %d, want 200 OK", w.Code)
}
var payload struct {
Tracks []any `json:"tracks"`
}
if err := json.Unmarshal(w.Body.Bytes(), &payload); err != nil {
t.Fatalf("decode: %v", err)
}
if payload.Tracks == nil || len(payload.Tracks) != 0 {
t.Fatalf("expected empty tracks array, got %v", payload.Tracks)
}
}
+49 -1
View File
@@ -20,9 +20,40 @@ func registerAdminRoutes(api *gin.RouterGroup, cfg *config.Config, svc *service.
registerAdminAPIConfigRoutes(admin, svc)
registerAdminRecognitionWordRoutes(admin, svc)
registerAdminStrmRoutes(admin, svc)
registerAdminScraperRoutes(admin, svc)
registerAdminDatabaseRoutes(admin, svc)
// FFmpeg/FFprobe 工具:状态查询 + 一键下载安装(自动匹配当前平台)。
admin.GET("/tools/ffmpeg/status", ffToolsStatusHandler(svc))
admin.POST("/tools/ffmpeg/install", ffToolsInstallHandler(svc))
}
func registerAdminScraperRoutes(admin *gin.RouterGroup, svc *service.Container) {
admin.GET("/scraper/queue", listScrapeQueueHandler(svc))
admin.POST("/scraper/queue/:id/cancel", cancelScrapeTaskHandler(svc))
admin.POST("/scraper/queue/:id/retry", retryScrapeTaskHandler(svc))
admin.DELETE("/scraper/queue/:id", deleteScrapeTaskHandler(svc))
admin.POST("/scraper/queue/batch", batchActionScrapeTasksHandler(svc))
admin.POST("/scraper/queue/clear-done", clearDoneScrapeTasksHandler(svc))
admin.POST("/scraper/queue/clear-finished", clearFinishedScrapeTasksHandler(svc))
admin.POST("/scraper/queue/clear-canceled", clearCanceledScrapeTasksHandler(svc))
admin.POST("/scraper/queue/retry-failed", retryAllFailedScrapeTasksHandler(svc))
admin.POST("/scraper/queue/cancel-pending", cancelPendingScrapeTasksHandler(svc))
admin.POST("/scraper/queue/enqueue-library/:id", enqueueLibraryScrapeHandler(svc))
admin.POST("/scraper/queue/enqueue-all", enqueueAllScrapeHandler(svc))
admin.POST("/media/repair-rescrape", enqueueAllScrapeHandler(svc))
admin.POST("/libraries/:id/repair-rescrape", enqueueLibraryScrapeHandler(svc))
}
func registerAdminStrmRoutes(admin *gin.RouterGroup, svc *service.Container) {
// Emby 挂载管理:远程 Emby 媒体库挂载(账号复用 strm/accounts)
admin.GET("/emby/accounts/:id/views", embyAccountViewsHandler(svc))
admin.POST("/emby/accounts/:id/full-mount", fullMountEmbyAccountHandler(svc))
admin.GET("/emby/mounts", listEmbyMountsHandler(svc))
admin.POST("/emby/mounts", createEmbyMountsHandler(svc))
admin.PUT("/emby/mounts/:id", updateEmbyMountHandler(svc))
admin.DELETE("/emby/mounts/:id", deleteEmbyMountHandler(svc))
admin.GET("/strm/accounts", listStrmAccountsHandler(svc))
admin.POST("/strm/accounts", createStrmAccountHandler(svc))
admin.PUT("/strm/accounts/:id", updateStrmAccountHandler(svc))
@@ -43,11 +74,15 @@ func registerAdminStrmRoutes(admin *gin.RouterGroup, svc *service.Container) {
admin.POST("/strm/paths/:id/sync", startStrmSyncHandler(svc))
admin.POST("/strm/paths/:id/cancel", cancelStrmSyncHandler(svc))
admin.GET("/strm/records", listStrmSyncRecordsHandler(svc))
admin.DELETE("/strm/records/:id", deleteStrmSyncRecordHandler(svc))
admin.DELETE("/strm/records", clearStrmSyncRecordsHandler(svc))
admin.GET("/strm/local-dirs", listStrmLocalDirsHandler(svc))
admin.GET("/strm/downloads", downloadQueueHandler(svc))
admin.POST("/strm/downloads/:id/cancel", cancelStrmDownloadHandler(svc))
admin.POST("/strm/downloads/:id/retry", retryStrmDownloadHandler(svc))
admin.DELETE("/strm/downloads/:id", deleteStrmDownloadHandler(svc))
admin.POST("/strm/downloads/batch", batchActionDownloadsHandler(svc))
admin.POST("/strm/downloads/clear-done", clearDoneDownloadsHandler(svc))
admin.POST("/strm/downloads/clear-finished", clearFinishedDownloadsHandler(svc))
admin.POST("/strm/downloads/clear-canceled", clearCanceledDownloadsHandler(svc))
@@ -56,8 +91,13 @@ func registerAdminStrmRoutes(admin *gin.RouterGroup, svc *service.Container) {
admin.GET("/strm/uploads", uploadQueueHandler(svc))
admin.POST("/strm/uploads/:id/cancel", cancelStrmUploadHandler(svc))
admin.POST("/strm/uploads/:id/retry", retryStrmUploadHandler(svc))
admin.POST("/strm/uploads/cancel-pending", cancelPendingUploadsHandler(svc))
admin.DELETE("/strm/uploads/:id", deleteStrmUploadHandler(svc))
admin.POST("/strm/uploads/batch", batchActionUploadsHandler(svc))
admin.POST("/strm/uploads/clear-done", clearDoneUploadsHandler(svc))
admin.POST("/strm/uploads/clear-finished", clearFinishedUploadsHandler(svc))
admin.POST("/strm/uploads/clear-canceled", clearCanceledUploadsHandler(svc))
admin.POST("/strm/uploads/retry-failed", retryAllFailedUploadsHandler(svc))
admin.POST("/strm/uploads/cancel-pending", cancelPendingUploadsHandler(svc))
}
func registerAdminUserRoutes(admin *gin.RouterGroup, svc *service.Container) {
@@ -67,6 +107,7 @@ func registerAdminUserRoutes(admin *gin.RouterGroup, svc *service.Container) {
admin.PATCH("/users/:id/password", resetUserPasswordHandler(svc))
admin.PATCH("/users/:id/status", updateUserStatusHandler(svc))
admin.PATCH("/users/:id/role", adminUpdateRoleHandler(svc))
admin.PATCH("/users/:id/libraries", updateUserLibrariesHandler(svc))
admin.DELETE("/users/:id", deleteUserHandler(svc))
admin.GET("/settings", listSettingsHandler(svc))
admin.PUT("/settings", updateSettingHandler(svc))
@@ -114,3 +155,10 @@ func registerAdminRecognitionWordRoutes(admin *gin.RouterGroup, svc *service.Con
admin.POST("/recognition-words/sync", syncRecognitionWordsHandler(svc))
admin.POST("/recognition-words/test", testRecognitionWordsHandler(svc))
}
func registerAdminDatabaseRoutes(admin *gin.RouterGroup, svc *service.Container) {
admin.GET("/database/status", getDatabaseStatusHandler(svc))
admin.POST("/database/test", testDatabaseHandler(svc))
admin.POST("/database/migrate", migrateDatabaseHandler(svc))
admin.POST("/database/save-config", saveDatabaseConfigHandler(svc))
}
@@ -21,6 +21,7 @@ func registerAuthedLibraryRoutes(authed *gin.RouterGroup, svc *service.Container
authed.POST("/libraries", middleware.AdminRequired(), createLibraryHandler(svc))
authed.GET("/libraries/:id", getLibraryHandler(svc))
authed.PATCH("/libraries/:id", middleware.AdminRequired(), updateLibraryHandler(svc))
authed.PUT("/libraries/reorder", middleware.AdminRequired(), reorderLibrariesHandler(svc))
authed.DELETE("/libraries/:id", middleware.AdminRequired(), deleteLibraryHandler(svc))
authed.GET("/libraries/:id/roots", middleware.AdminRequired(), listLibraryRootsHandler(svc))
authed.POST("/libraries/:id/roots", middleware.AdminRequired(), createLibraryRootHandler(svc))
@@ -38,6 +39,7 @@ func registerAuthedLibraryRoutes(authed *gin.RouterGroup, svc *service.Container
func registerAuthedMediaRoutes(authed *gin.RouterGroup, svc *service.Container) {
authed.GET("/media/:id", getMediaHandler(svc))
authed.GET("/media/:id/episodes", listMediaEpisodesHandler(svc))
authed.GET("/media", searchMediaHandler(svc))
authed.PATCH("/media/:id/metadata", middleware.AdminRequired(), updateMediaMetadataHandler(svc))
authed.POST("/media/:id/scrape", middleware.AdminRequired(), scrapeOneHandler(svc))
+167
View File
@@ -0,0 +1,167 @@
package handler
import (
"net/http"
"strconv"
"github.com/gin-gonic/gin"
"github.com/ShukeBta/MMTL/internal/service"
)
func listScrapeQueueHandler(svc *service.Container) gin.HandlerFunc {
return func(c *gin.Context) {
page, _ := strconv.Atoi(c.DefaultQuery("page", "1"))
pageSize, _ := strconv.Atoi(c.DefaultQuery("page_size", "50"))
snap, err := svc.Scraper.ScrapeQueueSnapshot(c.Request.Context(), c.Query("status"), page, pageSize)
if err != nil {
c.JSON(http.StatusInternalServerError, gin.H{"error": err.Error()})
return
}
c.JSON(http.StatusOK, snap)
}
}
func cancelScrapeTaskHandler(svc *service.Container) gin.HandlerFunc {
return func(c *gin.Context) {
if err := svc.Scraper.CancelScrapeTask(c.Request.Context(), c.Param("id")); err != nil {
c.JSON(http.StatusBadRequest, gin.H{"error": err.Error()})
return
}
c.JSON(http.StatusOK, gin.H{"ok": true})
}
}
func retryScrapeTaskHandler(svc *service.Container) gin.HandlerFunc {
return func(c *gin.Context) {
if err := svc.Scraper.RetryScrapeTask(c.Request.Context(), c.Param("id")); err != nil {
c.JSON(http.StatusBadRequest, gin.H{"error": err.Error()})
return
}
c.JSON(http.StatusOK, gin.H{"ok": true})
}
}
func deleteScrapeTaskHandler(svc *service.Container) gin.HandlerFunc {
return func(c *gin.Context) {
if err := svc.Scraper.DeleteScrapeTask(c.Request.Context(), c.Param("id")); err != nil {
c.JSON(http.StatusBadRequest, gin.H{"error": err.Error()})
return
}
c.JSON(http.StatusOK, gin.H{"ok": true})
}
}
func batchActionScrapeTasksHandler(svc *service.Container) gin.HandlerFunc {
return func(c *gin.Context) {
var req queueBatchReq
if err := c.ShouldBindJSON(&req); err != nil {
c.JSON(http.StatusBadRequest, gin.H{"error": err.Error()})
return
}
n, err := svc.Scraper.BatchActionScrapeTasks(c.Request.Context(), req.Action, req.IDs)
if err != nil {
c.JSON(http.StatusInternalServerError, gin.H{"error": err.Error()})
return
}
c.JSON(http.StatusOK, gin.H{"affected": n, "action": req.Action})
}
}
func clearDoneScrapeTasksHandler(svc *service.Container) gin.HandlerFunc {
return func(c *gin.Context) {
n, err := svc.Scraper.ClearDoneScrapeTasks(c.Request.Context())
if err != nil {
c.JSON(http.StatusInternalServerError, gin.H{"error": err.Error()})
return
}
c.JSON(http.StatusOK, gin.H{"deleted": n})
}
}
func clearFinishedScrapeTasksHandler(svc *service.Container) gin.HandlerFunc {
return func(c *gin.Context) {
n, err := svc.Scraper.ClearFinishedScrapeTasks(c.Request.Context())
if err != nil {
c.JSON(http.StatusInternalServerError, gin.H{"error": err.Error()})
return
}
c.JSON(http.StatusOK, gin.H{"deleted": n})
}
}
func clearCanceledScrapeTasksHandler(svc *service.Container) gin.HandlerFunc {
return func(c *gin.Context) {
n, err := svc.Scraper.ClearCanceledScrapeTasks(c.Request.Context())
if err != nil {
c.JSON(http.StatusInternalServerError, gin.H{"error": err.Error()})
return
}
c.JSON(http.StatusOK, gin.H{"deleted": n})
}
}
func retryAllFailedScrapeTasksHandler(svc *service.Container) gin.HandlerFunc {
return func(c *gin.Context) {
n, err := svc.Scraper.RetryAllFailedScrapeTasks(c.Request.Context())
if err != nil {
c.JSON(http.StatusInternalServerError, gin.H{"error": err.Error()})
return
}
c.JSON(http.StatusOK, gin.H{"retried": n})
}
}
func cancelPendingScrapeTasksHandler(svc *service.Container) gin.HandlerFunc {
return func(c *gin.Context) {
n, err := svc.Scraper.CancelPendingScrapeTasks(c.Request.Context())
if err != nil {
c.JSON(http.StatusInternalServerError, gin.H{"error": err.Error()})
return
}
c.JSON(http.StatusOK, gin.H{"canceled": n})
}
}
type enqueueScrapeReq struct {
EpisodeImages bool `json:"episode_images"`
EpisodeArtwork bool `json:"episode_artwork"`
RefreshMatched bool `json:"refresh_matched"`
IncludeMatched bool `json:"include_matched"`
}
func (r enqueueScrapeReq) toOptions() service.ScrapeOptions {
epArtwork := r.EpisodeImages || r.EpisodeArtwork
return service.ScrapeOptions{
EpisodeArtwork: &epArtwork,
IncludeMatched: r.IncludeMatched || r.RefreshMatched,
RetryNoMatch: true,
}
}
func enqueueLibraryScrapeHandler(svc *service.Container) gin.HandlerFunc {
return func(c *gin.Context) {
var req enqueueScrapeReq
_ = c.ShouldBindJSON(&req)
libID := c.Param("id")
n, err := svc.Scraper.EnqueueLibrary(c.Request.Context(), libID, req.toOptions())
if err != nil {
c.JSON(http.StatusInternalServerError, gin.H{"error": err.Error()})
return
}
c.JSON(http.StatusOK, gin.H{"enqueued": n})
}
}
func enqueueAllScrapeHandler(svc *service.Container) gin.HandlerFunc {
return func(c *gin.Context) {
var req enqueueScrapeReq
_ = c.ShouldBindJSON(&req)
n, err := svc.Scraper.EnqueueAll(c.Request.Context(), req.toOptions())
if err != nil {
c.JSON(http.StatusInternalServerError, gin.H{"error": err.Error()})
return
}
c.JSON(http.StatusOK, gin.H{"enqueued": n})
}
}
+100 -2
View File
@@ -65,7 +65,49 @@ func listSeasonsHandler(svc *service.Container) gin.HandlerFunc {
func listLibrarySeriesHandler(svc *service.Container) gin.HandlerFunc {
return func(c *gin.Context) {
libID := c.Param("id")
if lib, err := svc.Repo.Library.FindByID(c.Request.Context(), libID); err == nil && lib != nil {
ctx := c.Request.Context()
// 远程剧集库:远程 Series 映射为系列卡片。
if svc.EmbyRemote != nil && service.IsEmbyRemoteID(libID) {
mountID, remoteID, _ := service.DecodeEmbyRemoteID(libID)
mount, acct, _ := svc.EmbyRemote.ResolveMount(ctx, mountID)
if mount == nil || acct == nil {
c.JSON(http.StatusNotFound, gin.H{"error": "not found"})
return
}
cards, err := svc.EmbyRemote.RemoteSeriesCards(ctx, mount, acct, remoteID)
if err != nil {
writeInternalOrCanceled(c, err)
return
}
page, _ := strconv.Atoi(c.DefaultQuery("page", "1"))
size, _ := strconv.Atoi(c.DefaultQuery("page_size", "500"))
if page < 1 {
page = 1
}
if size <= 0 || size > 1000 {
size = 500
}
start := (page - 1) * size
if start > len(cards) {
start = len(cards)
}
end := start + size
if end > len(cards) {
end = len(cards)
}
pageItems := cards[start:end]
if pageItems == nil {
pageItems = []service.SeriesCard{}
}
c.JSON(http.StatusOK, gin.H{
"items": pageItems,
"total": len(cards),
"page": page,
"page_size": size,
})
return
}
if lib, err := svc.Repo.Library.FindByID(ctx, libID); err == nil && lib != nil {
if !service.LibraryVisibleForUser(c.Request.Context(), svc.Repo, *lib, mediaVisibilityForRequest(c, svc)) {
c.JSON(http.StatusNotFound, gin.H{"error": "not found"})
return
@@ -114,7 +156,27 @@ func listLibrarySeriesEpisodesHandler(svc *service.Container) gin.HandlerFunc {
c.JSON(http.StatusBadRequest, gin.H{"error": "key is required"})
return
}
if lib, err := svc.Repo.Library.FindByID(c.Request.Context(), libID); err == nil && lib != nil {
ctx := c.Request.Context()
// 远程系列 key(伪装系列 ID):转发远程该系列全部剧集。
if svc.EmbyRemote != nil && service.IsEmbyRemoteID(key) {
mountID, remoteSeriesID, _ := service.DecodeEmbyRemoteID(key)
mount, acct, _ := svc.EmbyRemote.ResolveMount(ctx, mountID)
if mount == nil || acct == nil {
c.JSON(http.StatusNotFound, gin.H{"error": "not found"})
return
}
items, err := svc.EmbyRemote.RemoteEpisodes(ctx, mount, acct, remoteSeriesID)
if err != nil {
writeInternalOrCanceled(c, err)
return
}
if items == nil {
items = []model.Media{}
}
c.JSON(http.StatusOK, gin.H{"items": items, "total": len(items)})
return
}
if lib, err := svc.Repo.Library.FindByID(ctx, libID); err == nil && lib != nil {
if !service.LibraryVisibleForUser(c.Request.Context(), svc.Repo, *lib, mediaVisibilityForRequest(c, svc)) {
c.JSON(http.StatusNotFound, gin.H{"error": "not found"})
return
@@ -128,3 +190,39 @@ func listLibrarySeriesEpisodesHandler(svc *service.Container) gin.HandlerFunc {
c.JSON(http.StatusOK, gin.H{"items": items, "total": len(items)})
}
}
func listMediaEpisodesHandler(svc *service.Container) gin.HandlerFunc {
return func(c *gin.Context) {
id := c.Param("id")
if id == "" {
c.JSON(http.StatusBadRequest, gin.H{"error": "id is required"})
return
}
ctx := c.Request.Context()
// 远程条目:单集→同系列集列表;系列/季/文件夹→子集;电影→自身单条。
if svc.EmbyRemote != nil && service.IsEmbyRemoteID(id) {
mountID, remoteID, _ := service.DecodeEmbyRemoteID(id)
mount, acct, _ := svc.EmbyRemote.ResolveMount(ctx, mountID)
if mount == nil || acct == nil {
c.JSON(http.StatusNotFound, gin.H{"error": "not found"})
return
}
items, err := svc.EmbyRemote.RemoteEpisodes(ctx, mount, acct, remoteID)
if err != nil {
writeInternalOrCanceled(c, err)
return
}
if items == nil {
items = []model.Media{}
}
c.JSON(http.StatusOK, gin.H{"items": items, "total": len(items)})
return
}
items, err := svc.Media.ListMediaEpisodes(ctx, id, mediaVisibilityForRequest(c, svc))
if err != nil {
writeInternalOrCanceled(c, err)
return
}
c.JSON(http.StatusOK, gin.H{"items": items, "total": len(items)})
}
}
+21 -55
View File
@@ -2,7 +2,6 @@
package handler
import (
"context"
"errors"
"io"
"net/http"
@@ -14,7 +13,13 @@ import (
func hlsPlaylistHandler(svc *service.Container) gin.HandlerFunc {
return func(c *gin.Context) {
m, err := svc.Media.GetMedia(c.Request.Context(), c.Param("id"))
id := c.Param("id")
// 远程 Emby 挂载媒体与 STRM 一样,默认直连播放,不进行转码。
if svc.EmbyRemote != nil && service.IsEmbyRemoteID(id) {
c.JSON(http.StatusConflict, gin.H{"error": "transcode disabled"})
return
}
m, err := svc.Media.GetMedia(c.Request.Context(), id)
if err != nil || m == nil || !mediaVisibleForRequest(c, svc, m) {
c.JSON(http.StatusNotFound, gin.H{"error": "not found"})
return
@@ -44,7 +49,12 @@ func hlsPlaylistHandler(svc *service.Container) gin.HandlerFunc {
func hlsSegmentHandler(svc *service.Container) gin.HandlerFunc {
return func(c *gin.Context) {
m, err := svc.Media.GetMedia(c.Request.Context(), c.Param("id"))
id := c.Param("id")
if svc.EmbyRemote != nil && service.IsEmbyRemoteID(id) {
c.JSON(http.StatusConflict, gin.H{"error": "transcode disabled"})
return
}
m, err := svc.Media.GetMedia(c.Request.Context(), id)
if err != nil || m == nil || !mediaVisibleForRequest(c, svc, m) {
c.JSON(http.StatusNotFound, gin.H{"error": "not found"})
return
@@ -135,28 +145,12 @@ func scrapeOneHandler(svc *service.Container) gin.HandlerFunc {
return
}
options.IncludeMatched = true
m, err := svc.Repo.Media.FindByID(c.Request.Context(), c.Param("id"))
if err != nil || m == nil {
c.JSON(http.StatusNotFound, gin.H{"error": "not found"})
return
}
task := startScrapeHTTPTask(svc, "手动刮削媒体", m.Title, m.Path)
if err := svc.Scraper.EnrichOneWithOptions(c.Request.Context(), m, options); err != nil {
finishHTTPTask(task, err, "scrape", "手动刮削媒体失败", nil, nil)
task, err := svc.Scraper.EnqueueMedia(c.Request.Context(), c.Param("id"), options)
if err != nil {
c.JSON(http.StatusInternalServerError, gin.H{"error": err.Error()})
return
}
reclassified := reclassifyMediaAfterScrape(c.Request.Context(), svc, m.ID)
refreshed, _ := svc.Repo.Media.FindByID(c.Request.Context(), m.ID)
metrics := map[string]int64{"processed": 1}
if refreshed != nil && refreshed.ScrapeStatus == "matched" {
metrics["matched"] = 1
}
if reclassified > 0 {
metrics["reclassified"] = int64(reclassified)
}
finishHTTPTask(task, nil, "completed", "手动刮削媒体结束", metrics, nil)
c.JSON(http.StatusOK, refreshed)
c.JSON(http.StatusOK, task)
}
}
@@ -170,40 +164,12 @@ func scrapeLibraryHandler(svc *service.Container) gin.HandlerFunc {
return
}
options.IncludeMatched = true
var task *service.TaskHandle
if lib, err := svc.Repo.Library.FindByID(c.Request.Context(), libID); err == nil && lib != nil {
task = startScrapeHTTPTask(svc, "手动刮削媒体库", lib.Name, lib.Path)
} else {
task = startScrapeHTTPTask(svc, "手动刮削媒体库", libID, "")
n, err := svc.Scraper.EnqueueLibrary(c.Request.Context(), libID, options)
if err != nil {
c.JSON(http.StatusInternalServerError, gin.H{"error": err.Error()})
return
}
// Run in the background so HTTP returns instantly; the WS hub
// pushes per-item progress on the "scrape" topic.
go func(libID string, task *service.TaskHandle, options service.ScrapeOptions) {
result, err := svc.Scraper.EnrichLibraryDetailedWithOptions(context.Background(), libID, options)
reclassified := 0
if result.Processed > 0 {
reclassified = reclassifyLibraryAfterScrape(context.Background(), svc, libID)
}
metrics := map[string]int64{
"matched": int64(result.Matched),
"processed": int64(result.Processed),
"candidates": int64(result.Candidates),
}
if reclassified > 0 {
metrics["reclassified"] = int64(reclassified)
}
if result.Failed > 0 {
metrics["errors"] = int64(result.Failed)
}
stage := "completed"
message := "手动刮削媒体库结束"
if err != nil {
stage = "scrape"
message = "手动刮削媒体库失败"
}
finishHTTPTask(task, err, stage, message, metrics, nil)
}(libID, task, options)
c.JSON(http.StatusAccepted, gin.H{"status": "scraping"})
c.JSON(http.StatusOK, gin.H{"status": "queued", "enqueued": n})
}
}
+130 -7
View File
@@ -28,17 +28,25 @@ type strmAccountView struct {
model.StrmAccount
HasCredential bool `json:"has_credential"`
ProviderLabel string `json:"provider_label"`
// ProxyPlay 仅远程 Emby 挂载账号返回:播放流量是否经过 MMTL 代理(编辑回显用)。
ProxyPlay *bool `json:"proxy_play,omitempty"`
}
func strmAccountViews(accounts []model.StrmAccount) []strmAccountView {
func strmAccountViews(svc *service.Container, accounts []model.StrmAccount) []strmAccountView {
out := make([]strmAccountView, 0, len(accounts))
for i := range accounts {
a := accounts[i]
out = append(out, strmAccountView{
view := strmAccountView{
StrmAccount: a,
HasCredential: service.HasStrmAccountCredential(&a),
ProviderLabel: providerLabelOf(a.Provider),
})
}
if a.Provider == model.StrmProviderEmbyRemote && svc != nil && svc.EmbyRemote != nil {
if proxyPlay, err := svc.EmbyRemote.ProxyPlayOf(&a); err == nil {
view.ProxyPlay = &proxyPlay
}
}
out = append(out, view)
}
return out
}
@@ -58,7 +66,7 @@ func listStrmAccountsHandler(svc *service.Container) gin.HandlerFunc {
c.JSON(http.StatusInternalServerError, gin.H{"error": err.Error()})
return
}
c.JSON(http.StatusOK, strmAccountViews(accounts))
c.JSON(http.StatusOK, strmAccountViews(svc, accounts))
}
}
@@ -74,7 +82,7 @@ func createStrmAccountHandler(svc *service.Container) gin.HandlerFunc {
c.JSON(http.StatusBadRequest, gin.H{"error": err.Error()})
return
}
views := strmAccountViews([]model.StrmAccount{*acct})
views := strmAccountViews(svc, []model.StrmAccount{*acct})
c.JSON(http.StatusOK, views[0])
}
}
@@ -92,7 +100,7 @@ func updateStrmAccountHandler(svc *service.Container) gin.HandlerFunc {
c.JSON(http.StatusBadRequest, gin.H{"error": err.Error()})
return
}
views := strmAccountViews([]model.StrmAccount{*acct})
views := strmAccountViews(svc, []model.StrmAccount{*acct})
c.JSON(http.StatusOK, views[0])
}
}
@@ -114,7 +122,7 @@ func testStrmAccountHandler(svc *service.Container) gin.HandlerFunc {
c.JSON(http.StatusNotFound, gin.H{"error": "网盘账号不存在"})
return
}
views := strmAccountViews([]model.StrmAccount{*acct})
views := strmAccountViews(svc, []model.StrmAccount{*acct})
c.JSON(http.StatusOK, views[0])
}
}
@@ -300,6 +308,31 @@ func listStrmSyncRecordsHandler(svc *service.Container) gin.HandlerFunc {
}
}
func deleteStrmSyncRecordHandler(svc *service.Container) gin.HandlerFunc {
return func(c *gin.Context) {
if c.Param("id") == "" {
c.JSON(http.StatusBadRequest, gin.H{"error": "缺少记录 ID"})
return
}
if err := svc.Strm.DeleteSyncRecord(c.Request.Context(), c.Param("id")); err != nil {
c.JSON(http.StatusInternalServerError, gin.H{"error": err.Error()})
return
}
c.JSON(http.StatusOK, gin.H{"ok": true})
}
}
func clearStrmSyncRecordsHandler(svc *service.Container) gin.HandlerFunc {
return func(c *gin.Context) {
deleted, err := svc.Strm.ClearSyncRecords(c.Request.Context(), c.Query("path_id"))
if err != nil {
c.JSON(http.StatusInternalServerError, gin.H{"error": err.Error()})
return
}
c.JSON(http.StatusOK, gin.H{"ok": true, "deleted": deleted})
}
}
// ─── 下载/上传队列 ─────────────────────────────────────────────────────────────
func downloadQueueHandler(svc *service.Container) gin.HandlerFunc {
@@ -368,6 +401,63 @@ func retryStrmUploadHandler(svc *service.Container) gin.HandlerFunc {
}
}
func deleteStrmDownloadHandler(svc *service.Container) gin.HandlerFunc {
return func(c *gin.Context) {
if err := svc.Strm.DeleteDownloadTask(c.Request.Context(), c.Param("id")); err != nil {
c.JSON(http.StatusBadRequest, gin.H{"error": err.Error()})
return
}
c.JSON(http.StatusOK, gin.H{"ok": true})
}
}
func deleteStrmUploadHandler(svc *service.Container) gin.HandlerFunc {
return func(c *gin.Context) {
if err := svc.Strm.DeleteUploadTask(c.Request.Context(), c.Param("id")); err != nil {
c.JSON(http.StatusBadRequest, gin.H{"error": err.Error()})
return
}
c.JSON(http.StatusOK, gin.H{"ok": true})
}
}
type queueBatchReq struct {
Action string `json:"action" binding:"required"`
IDs []string `json:"ids" binding:"required"`
}
func batchActionDownloadsHandler(svc *service.Container) gin.HandlerFunc {
return func(c *gin.Context) {
var req queueBatchReq
if err := c.ShouldBindJSON(&req); err != nil {
c.JSON(http.StatusBadRequest, gin.H{"error": err.Error()})
return
}
n, err := svc.Strm.BatchActionDownloadTasks(c.Request.Context(), req.Action, req.IDs)
if err != nil {
c.JSON(http.StatusInternalServerError, gin.H{"error": err.Error()})
return
}
c.JSON(http.StatusOK, gin.H{"affected": n, "action": req.Action})
}
}
func batchActionUploadsHandler(svc *service.Container) gin.HandlerFunc {
return func(c *gin.Context) {
var req queueBatchReq
if err := c.ShouldBindJSON(&req); err != nil {
c.JSON(http.StatusBadRequest, gin.H{"error": err.Error()})
return
}
n, err := svc.Strm.BatchActionUploadTasks(c.Request.Context(), req.Action, req.IDs)
if err != nil {
c.JSON(http.StatusInternalServerError, gin.H{"error": err.Error()})
return
}
c.JSON(http.StatusOK, gin.H{"affected": n, "action": req.Action})
}
}
// ─── 下载队列批量操作 ─────────────────────────────────────────────────────────
func clearDoneDownloadsHandler(svc *service.Container) gin.HandlerFunc {
@@ -414,6 +504,28 @@ func clearCanceledUploadsHandler(svc *service.Container) gin.HandlerFunc {
}
}
func clearDoneUploadsHandler(svc *service.Container) gin.HandlerFunc {
return func(c *gin.Context) {
n, err := svc.Strm.ClearDoneUploadTasks(c.Request.Context())
if err != nil {
c.JSON(http.StatusInternalServerError, gin.H{"error": err.Error()})
return
}
c.JSON(http.StatusOK, gin.H{"deleted": n})
}
}
func clearFinishedUploadsHandler(svc *service.Container) gin.HandlerFunc {
return func(c *gin.Context) {
n, err := svc.Strm.ClearFinishedUploadTasks(c.Request.Context())
if err != nil {
c.JSON(http.StatusInternalServerError, gin.H{"error": err.Error()})
return
}
c.JSON(http.StatusOK, gin.H{"deleted": n})
}
}
func retryAllFailedDownloadsHandler(svc *service.Container) gin.HandlerFunc {
return func(c *gin.Context) {
n, err := svc.Strm.RetryAllFailedDownloadTasks(c.Request.Context())
@@ -425,6 +537,17 @@ func retryAllFailedDownloadsHandler(svc *service.Container) gin.HandlerFunc {
}
}
func retryAllFailedUploadsHandler(svc *service.Container) gin.HandlerFunc {
return func(c *gin.Context) {
n, err := svc.Strm.RetryAllFailedUploadTasks(c.Request.Context())
if err != nil {
c.JSON(http.StatusInternalServerError, gin.H{"error": err.Error()})
return
}
c.JSON(http.StatusOK, gin.H{"retried": n})
}
}
func cancelPendingDownloadsHandler(svc *service.Container) gin.HandlerFunc {
return func(c *gin.Context) {
n, err := svc.Strm.CancelPendingDownloadTasks(c.Request.Context())
+4 -1
View File
@@ -57,8 +57,11 @@ func TestStrmAdminRoutesAreRegistered(t *testing.T) {
"GET /api/admin/strm/uploads",
"POST /api/admin/strm/uploads/:id/cancel",
"POST /api/admin/strm/uploads/:id/retry",
"POST /api/admin/strm/uploads/cancel-pending",
"POST /api/admin/strm/uploads/clear-done",
"POST /api/admin/strm/uploads/clear-finished",
"POST /api/admin/strm/uploads/clear-canceled",
"POST /api/admin/strm/uploads/retry-failed",
"POST /api/admin/strm/uploads/cancel-pending",
"GET /api/strm/play/:provider/:file",
} {
if !routes[want] {
+7 -2
View File
@@ -11,7 +11,12 @@ import (
func listSubtitlesHandler(svc *service.Container) gin.HandlerFunc {
return func(c *gin.Context) {
tracks, err := svc.Subtitle.Discover(c.Request.Context(), c.Param("id"))
id := c.Param("id")
if svc.EmbyRemote != nil && service.IsEmbyRemoteID(id) {
c.JSON(http.StatusOK, gin.H{"tracks": []service.SubtitleTrack{}})
return
}
tracks, err := svc.Subtitle.Discover(c.Request.Context(), id)
if err != nil {
c.JSON(http.StatusNotFound, gin.H{"error": err.Error()})
return
@@ -31,7 +36,7 @@ func serveSubtitleHandler(svc *service.Container) gin.HandlerFunc {
return
}
c.Header("Content-Type", "text/vtt; charset=utf-8")
c.Header("Cache-Control", "public, max-age=3600")
c.Header("Cache-Control", "no-cache, no-store, must-revalidate")
if err := svc.Subtitle.Serve(c.Request.Context(), c.Param("id"), path, c.Writer); err != nil {
c.JSON(http.StatusNotFound, gin.H{"error": err.Error()})
return
-144
View File
@@ -1,144 +0,0 @@
// Package handler — system tools detection.
package handler
import (
"net/http"
"os/exec"
"strings"
"github.com/gin-gonic/gin"
"go.uber.org/zap"
"github.com/ShukeBta/MMTL/internal/config"
"github.com/ShukeBta/MMTL/internal/service"
)
// SystemHandler handles system-related endpoints.
type SystemHandler struct {
cfg *config.Config
log *zap.Logger
svc *service.Container
}
// NewSystemHandler is the constructor.
func NewSystemHandler(cfg *config.Config, log *zap.Logger, svc *service.Container) *SystemHandler {
return &SystemHandler{cfg: cfg, log: log, svc: svc}
}
// ToolStatus represents the detection status of a system tool.
type ToolStatus struct {
Name string `json:"name"`
DisplayName string `json:"display_name"`
ConfigKey string `json:"config_key"`
Path string `json:"path,omitempty"`
Detected bool `json:"detected"`
Version string `json:"version,omitempty"`
}
// GetToolsStatus returns the status of system tools.
func (h *SystemHandler) GetToolsStatus(c *gin.Context) {
tools := []ToolStatus{
{Name: "ffprobe", DisplayName: "FFprobe", ConfigKey: "app.ffprobe_path"},
{Name: "ffmpeg", DisplayName: "FFmpeg", ConfigKey: "app.ffmpeg_path"},
}
for i := range tools {
// Check configured path first
var configuredPath string
switch tools[i].ConfigKey {
case "app.ffprobe_path":
configuredPath = h.cfg.App.FFprobePath
if configuredPath == "" {
configuredPath = "ffprobe"
}
case "app.ffmpeg_path":
configuredPath = h.cfg.App.FFmpegPath
if configuredPath == "" {
configuredPath = "ffmpeg"
}
}
// Try to find the tool
path, err := exec.LookPath(configuredPath)
if err == nil {
tools[i].Detected = true
tools[i].Path = path
// Try to get version
tools[i].Version = getToolVersion(path)
}
}
c.JSON(http.StatusOK, gin.H{
"tools": tools,
})
}
// getToolVersion attempts to get the version of a tool.
func getToolVersion(path string) string {
out, err := exec.Command(path, "-version").Output()
if err != nil {
return ""
}
// Extract first line as version info
lines := strings.Split(string(out), "\n")
if len(lines) > 0 {
return strings.TrimSpace(lines[0])
}
return ""
}
// InstallTools attempts to auto-install system tools (ffmpeg/ffprobe)
func (h *SystemHandler) InstallTools(c *gin.Context) {
h.log.Info("Received tools auto-install request")
// Call service layer to auto-install
ffprobePath, ffmpegPath := service.AutoInstallFFmpeg(h.log, h.cfg)
result := gin.H{
"installed": ffprobePath != "" || ffmpegPath != "",
}
if ffprobePath != "" {
result["ffprobe_path"] = ffprobePath
result["ffprobe_installed"] = true
}
if ffmpegPath != "" {
result["ffmpeg_path"] = ffmpegPath
result["ffmpeg_installed"] = true
}
// Re-detect tool status
tools := []ToolStatus{
{Name: "ffprobe", DisplayName: "FFprobe", ConfigKey: "app.ffprobe_path"},
{Name: "ffmpeg", DisplayName: "FFmpeg", ConfigKey: "app.ffmpeg_path"},
}
for i := range tools {
var configuredPath string
switch tools[i].ConfigKey {
case "app.ffprobe_path":
configuredPath = h.cfg.App.FFprobePath
if configuredPath == "" {
configuredPath = "ffprobe"
}
case "app.ffmpeg_path":
configuredPath = h.cfg.App.FFmpegPath
if configuredPath == "" {
configuredPath = "ffmpeg"
}
}
path, err := exec.LookPath(configuredPath)
if err == nil {
tools[i].Detected = true
tools[i].Path = path
tools[i].Version = getToolVersion(path)
}
}
result["tools"] = tools
h.log.Info("Tool installation completed", zap.Any("result", result))
c.JSON(http.StatusOK, result)
}
+38
View File
@@ -0,0 +1,38 @@
// Package handler — FFmpeg/FFprobe 工具安装端点。
package handler
import (
"net/http"
"github.com/gin-gonic/gin"
"github.com/ShukeBta/MMTL/internal/service"
)
// ffToolsStatusHandler 返回 ffmpeg/ffprobe 当前安装状态
// (GET /api/admin/tools/ffmpeg/status)。
func ffToolsStatusHandler(svc *service.Container) gin.HandlerFunc {
return func(c *gin.Context) {
if svc == nil || svc.FFTools == nil {
c.JSON(http.StatusServiceUnavailable, gin.H{"error": "FFmpeg 工具服务不可用"})
return
}
c.JSON(http.StatusOK, svc.FFTools.Status(c.Request.Context()))
}
}
// ffToolsInstallHandler 触发后台下载安装(POST /api/admin/tools/ffmpeg/install)。
// 自动匹配当前运行环境(OS+架构),安装到 data/tools/ffmpeg/ 并把路径写入设置。
func ffToolsInstallHandler(svc *service.Container) gin.HandlerFunc {
return func(c *gin.Context) {
if svc == nil || svc.FFTools == nil {
c.JSON(http.StatusServiceUnavailable, gin.H{"error": "FFmpeg 工具服务不可用"})
return
}
if err := svc.FFTools.StartInstall(c.Request.Context()); err != nil {
c.JSON(http.StatusConflict, gin.H{"error": err.Error()})
return
}
c.JSON(http.StatusOK, svc.FFTools.Status(c.Request.Context()))
}
}
+8 -1
View File
@@ -32,7 +32,14 @@ func mediaVisibilityForRequest(c *gin.Context, svc *service.Container) service.M
return visibility
}
visibility.IncludeNSFW = adultEnabled && profile.AllowAdult && !userHidesAdult
visibility.AllowedLibraryIDs = profileAllowedLibraryIDs(*profile)
profileAllowed := profileAllowedLibraryIDs(*profile)
if len(profileAllowed) > 0 {
if len(visibility.AllowedLibraryIDs) > 0 {
visibility.AllowedLibraryIDs = service.IntersectStrings(visibility.AllowedLibraryIDs, profileAllowed)
} else {
visibility.AllowedLibraryIDs = profileAllowed
}
}
if !visibility.IncludeNSFW {
visibility.HiddenLibraryIDs = service.AdultLibraryIDs(c.Request.Context(), svc.Repo)
} else {
+19
View File
@@ -0,0 +1,19 @@
// Emby 媒体库挂载模型。
//
// 远程 Emby 账号(StrmAccount.Provider = emby_remote)只是一个服务器连接;
// 「挂载」才决定把该服务器的哪个媒体库(View)暴露到本项目的媒体库中。
// 这样同一个 Emby 服务器可以按库选择挂载,且每个挂载独立控制是否由 MMTL
// 代理播放流量。
package model
// EmbyMount 是远程 Emby 服务器上一个媒体库(View)的挂载配置。
type EmbyMount struct {
Base
AccountID string `gorm:"size:36;index" json:"account_id"` // StrmAccount.ID(provider=emby_remote)
RemoteViewID string `gorm:"size:128" json:"remote_view_id"` // 远程 Emby 的 View Id
RemoteViewName string `gorm:"size:255" json:"remote_view_name"` // 远程媒体库原名(展示冗余)
CollectionType string `gorm:"size:32" json:"collection_type"` // movies / tvshows / music ...
Name string `gorm:"size:255" json:"name,omitempty"` // 覆盖显示名(可选,默认「账号 · 库名」)
ProxyPlay bool `gorm:"default:false" json:"proxy_play"` // 该挂载播放流量是否经 MMTL 反向代理
Enabled bool `gorm:"default:true" json:"enabled"` // 是否在媒体库中展示
}
+8 -6
View File
@@ -3,12 +3,14 @@ package model
// Library 表示一个逻辑媒体库。Path 保留为兼容字段,指向第一条 LibraryRoot。
type Library struct {
Base
Name string `gorm:"size:128;not null" json:"name"`
Path string `gorm:"size:1024;not null" json:"path"`
Type string `gorm:"size:16;not null;default:movie" json:"type"` // movie / tv / anime / music
CoverURL string `gorm:"size:1024" json:"cover_url,omitempty"`
Enabled bool `gorm:"default:true" json:"enabled"`
Roots []LibraryRoot `gorm:"foreignKey:LibraryID" json:"roots,omitempty"`
Name string `gorm:"size:128;not null" json:"name"`
Path string `gorm:"size:1024;not null" json:"path"`
Type string `gorm:"size:16;not null;default:movie" json:"type"` // movie / tv / anime / music
CoverURL string `gorm:"size:1024" json:"cover_url,omitempty"`
Enabled bool `gorm:"default:true" json:"enabled"`
SortOrder int `gorm:"index;default:0" json:"sort_order"` // 手动拖拽排序用,越小越靠前
CarouselEnabled bool `gorm:"default:false" json:"carousel_enabled"` // 是否参与首页海报轮播(默认不参与)
Roots []LibraryRoot `gorm:"foreignKey:LibraryID" json:"roots,omitempty"`
}
// LibraryRoot 是逻辑媒体库下的一条真实物理/挂载路径。
+2
View File
@@ -57,5 +57,7 @@ func AllModels() []interface{} {
&StrmDownloadTask{},
&StrmUploadTask{},
&StrmDirCache{},
&ScrapeTask{},
&EmbyMount{},
}
}
+34
View File
@@ -0,0 +1,34 @@
package model
import "time"
const (
ScrapeTaskPending = "pending"
ScrapeTaskRunning = "running"
ScrapeTaskDone = "done"
ScrapeTaskFailed = "failed"
ScrapeTaskCanceled = "canceled"
)
// ScrapeTask 表示一条持久化的媒体刮削任务。
type ScrapeTask struct {
Base
MediaID string `gorm:"index;size:36" json:"media_id"`
LibraryID string `gorm:"index;size:36" json:"library_id"`
LibraryName string `gorm:"size:128" json:"library_name"`
MediaTitle string `gorm:"size:255;not null" json:"media_title"`
MediaPath string `gorm:"size:1024;not null" json:"media_path"`
MediaType string `gorm:"size:16" json:"media_type"` // movie / tv / anime / adult
Provider string `gorm:"size:32" json:"provider"` // tmdb / douban / bangumi / thetvdb / metatube
MatchedTitle string `gorm:"size:255" json:"matched_title"`
MatchedYear int `json:"matched_year"`
PosterURL string `gorm:"size:1024" json:"poster_url"`
BackdropURL string `gorm:"size:1024" json:"backdrop_url"`
Status string `gorm:"index;size:16;default:pending" json:"status"` // pending / running / done / failed / canceled
Error string `gorm:"type:text" json:"error"`
RetryCount int `gorm:"default:0" json:"retry_count"`
EpisodeImages bool `gorm:"default:true" json:"episode_images"`
RefreshMatched bool `gorm:"default:false" json:"refresh_matched"`
StartedAt *time.Time `json:"started_at,omitempty"`
FinishedAt *time.Time `json:"finished_at,omitempty"`
}
+1
View File
@@ -13,6 +13,7 @@ const (
StrmProviderCloudDrive = "clouddrive2" // CloudDrive2(WebDAV 桥接)
StrmProviderOpenList = "openlist" // OpenList / AList 兼容桥接
StrmProviderLocal = "local" // 本地目录(无账号)
StrmProviderEmbyRemote = "emby_remote" // 远程 Emby 服务器(API 网关聚合挂载,不走 STRM 同步)
)
// StrmAccount 是一个网盘账号(STRM 同步数据源凭据)。
+36 -1
View File
@@ -1,6 +1,10 @@
package model
import "time"
import (
"encoding/json"
"strings"
"time"
)
// User 是本地账户。第一个注册的管理员(或种子管理员)获得 "admin" 角色;
// 其他所有用户默认为 "user"。
@@ -17,6 +21,10 @@ type User struct {
ForcePasswordReset bool `gorm:"default:false" json:"force_password_reset"`
IsActive bool `gorm:"default:true" json:"is_active"`
LastLoginAt *time.Time `json:"last_login_at,omitempty"`
// AllowedLibraryIDs 存储管理员为该用户指定的受限可访问媒体库 ID 列表(JSON 字符串)。
// 为空时代表不限制(全库可访问)。
AllowedLibraryIDs string `gorm:"type:text" json:"-"`
AllowedLibraryList []string `gorm:"-" json:"allowed_library_ids,omitempty"`
// ExpiredAt is the account expiry time. Nil means the account never
// expires. When set and in the past, the account is treated as expired
// (login blocked) until an admin or a redemption code renews it.
@@ -31,3 +39,30 @@ type User struct {
RealtimeOnline bool `gorm:"-" json:"realtime_online,omitempty"`
RealtimeDeviceCount int `gorm:"-" json:"realtime_device_count,omitempty"`
}
// DecodeAllowedLibraryIDs 解析 AllowedLibraryIDs 字段。
func (u *User) DecodeAllowedLibraryIDs() []string {
if u == nil || strings.TrimSpace(u.AllowedLibraryIDs) == "" {
return nil
}
var ids []string
if err := json.Unmarshal([]byte(u.AllowedLibraryIDs), &ids); err != nil {
return nil
}
var out []string
for _, id := range ids {
trimmed := strings.TrimSpace(id)
if trimmed != "" {
out = append(out, trimmed)
}
}
return out
}
// PopulateComputedFields 填充非 DB 虚拟计算字段(如 AllowedLibraryList)。
func (u *User) PopulateComputedFields() {
if u == nil {
return
}
u.AllowedLibraryList = u.DecodeAllowedLibraryIDs()
}
@@ -0,0 +1,93 @@
package repository
import (
"context"
"errors"
"time"
"gorm.io/gorm"
"github.com/ShukeBta/MMTL/internal/model"
)
// EmbyMountRepository 持久化远程 Emby 媒体库挂载。
type EmbyMountRepository struct{ db *gorm.DB }
func (r *EmbyMountRepository) Create(ctx context.Context, m *model.EmbyMount) error {
return withSQLiteBusyRetry(ctx, func() error {
return r.db.WithContext(ctx).Create(m).Error
})
}
func (r *EmbyMountRepository) CreateInBatches(ctx context.Context, mounts []*model.EmbyMount, batchSize int) error {
if len(mounts) == 0 {
return nil
}
if batchSize <= 0 {
batchSize = 50
}
return withSQLiteBusyRetry(ctx, func() error {
return r.db.WithContext(ctx).CreateInBatches(mounts, batchSize).Error
})
}
func (r *EmbyMountRepository) FindByID(ctx context.Context, id string) (*model.EmbyMount, error) {
var m model.EmbyMount
err := r.db.WithContext(ctx).Where("id = ?", id).First(&m).Error
if errors.Is(err, gorm.ErrRecordNotFound) {
return nil, nil
}
if err != nil {
return nil, err
}
return &m, nil
}
func (r *EmbyMountRepository) List(ctx context.Context) ([]model.EmbyMount, error) {
var rows []model.EmbyMount
err := r.db.WithContext(ctx).Order("created_at desc").Find(&rows).Error
return rows, err
}
func (r *EmbyMountRepository) ListByAccountID(ctx context.Context, accountID string) ([]model.EmbyMount, error) {
var rows []model.EmbyMount
err := r.db.WithContext(ctx).Where("account_id = ?", accountID).Order("created_at asc").Find(&rows).Error
return rows, err
}
func (r *EmbyMountRepository) CountByAccountID(ctx context.Context, accountID string) (int64, error) {
var count int64
err := r.db.WithContext(ctx).Model(&model.EmbyMount{}).Where("account_id = ?", accountID).Count(&count).Error
return count, err
}
func (r *EmbyMountRepository) Update(ctx context.Context, m *model.EmbyMount) error {
return withSQLiteBusyRetry(ctx, func() error {
return r.db.WithContext(ctx).Model(&model.EmbyMount{}).Where("id = ?", m.ID).Updates(map[string]any{
"name": m.Name,
"proxy_play": m.ProxyPlay,
"enabled": m.Enabled,
"remote_view_id": m.RemoteViewID,
"remote_view_name": m.RemoteViewName,
"collection_type": m.CollectionType,
"updated_at": time.Now(),
}).Error
})
}
func (r *EmbyMountRepository) Delete(ctx context.Context, id string) error {
return withSQLiteBusyRetry(ctx, func() error {
return r.db.WithContext(ctx).Where("id = ?", id).Delete(&model.EmbyMount{}).Error
})
}
// DeleteByAccountID 删除账号下全部挂载(删除账号时级联清理)。
func (r *EmbyMountRepository) DeleteByAccountID(ctx context.Context, accountID string) (int64, error) {
var count int64
err := withSQLiteBusyRetry(ctx, func() error {
res := r.db.WithContext(ctx).Where("account_id = ?", accountID).Delete(&model.EmbyMount{})
count = res.RowsAffected
return res.Error
})
return count, err
}
+28 -1
View File
@@ -15,6 +15,11 @@ type LibraryRepository struct{ db *gorm.DB }
// Create persists a new library row.
func (r *LibraryRepository) Create(ctx context.Context, l *model.Library) error {
if l != nil && l.SortOrder == 0 {
var maxSort int
_ = r.db.WithContext(ctx).Model(&model.Library{}).Select("COALESCE(MAX(sort_order), -1)").Scan(&maxSort)
l.SortOrder = maxSort + 1
}
return r.db.WithContext(ctx).Create(l).Error
}
@@ -23,6 +28,11 @@ func (r *LibraryRepository) CreateWithRoots(ctx context.Context, l *model.Librar
return r.Create(ctx, l)
}
return r.db.WithContext(ctx).Transaction(func(tx *gorm.DB) error {
if l != nil && l.SortOrder == 0 {
var maxSort int
_ = tx.Model(&model.Library{}).Select("COALESCE(MAX(sort_order), -1)").Scan(&maxSort)
l.SortOrder = maxSort + 1
}
if err := tx.Create(l).Error; err != nil {
return err
}
@@ -50,7 +60,7 @@ func (r *LibraryRepository) CreateWithRoots(ctx context.Context, l *model.Librar
// List returns all enabled+disabled libraries.
func (r *LibraryRepository) List(ctx context.Context) ([]model.Library, error) {
var ls []model.Library
q := r.db.WithContext(ctx).Order("created_at asc")
q := r.db.WithContext(ctx).Order("sort_order asc, created_at asc")
if r.hasLibraryRootsTable() {
q = q.Preload("Roots", func(db *gorm.DB) *gorm.DB {
return db.Order("sort_order asc, created_at asc")
@@ -60,6 +70,23 @@ func (r *LibraryRepository) List(ctx context.Context) ([]model.Library, error) {
return ls, err
}
// SetSortOrder assigns sort_order to libraries, preserving position order for
// any library not present in the provided map.
func (r *LibraryRepository) SetSortOrder(ctx context.Context, ids []string) error {
if len(ids) == 0 {
return nil
}
return r.db.WithContext(ctx).Transaction(func(tx *gorm.DB) error {
for i, id := range ids {
if err := tx.Model(&model.Library{}).Where("id = ?", id).
Update("sort_order", i).Error; err != nil {
return err
}
}
return nil
})
}
// FindByID returns the library, or (nil, nil) when missing.
func (r *LibraryRepository) FindByID(ctx context.Context, id string) (*model.Library, error) {
var l model.Library
+29
View File
@@ -114,6 +114,35 @@ func (r *MediaRepository) ListByLibrariesFiltered(ctx context.Context, libraryID
return items, total, err
}
type libraryCountRow struct {
LibraryID string `gorm:"column:library_id"`
Total int64 `gorm:"column:total"`
}
// CountByLibraries returns a map of library_id -> total media count for the given library IDs.
func (r *MediaRepository) CountByLibraries(ctx context.Context, libraryIDs []string, filter MediaQueryFilter) (map[string]int64, error) {
out := make(map[string]int64, len(libraryIDs))
if len(libraryIDs) == 0 {
return out, nil
}
var rows []libraryCountRow
q := r.db.WithContext(ctx).Model(&model.Media{}).
Select("library_id, count(*) as total")
if len(libraryIDs) == 1 {
q = q.Where("library_id = ?", libraryIDs[0])
} else {
q = q.Where("library_id IN ?", libraryIDs)
}
q = applyMediaQueryFilter(q, filter)
if err := q.Group("library_id").Scan(&rows).Error; err != nil {
return nil, err
}
for _, row := range rows {
out[row.LibraryID] = row.Total
}
return out, nil
}
// DeleteByLibrary purges all media tied to a library.
func (r *MediaRepository) DeleteByLibrary(ctx context.Context, libraryID string) error {
// FTS 行由 media 表上的触发器同步清理(物理删除触发 FTS 清理)。
+7 -3
View File
@@ -30,9 +30,11 @@ type Container struct {
StrmSyncPath *StrmSyncPathRepository
StrmSyncRecord *StrmSyncRecordRepository
StrmDownload *StrmDownloadTaskRepository
StrmUpload *StrmUploadTaskRepository
StrmDirCache *StrmDirCacheRepository
}
StrmUpload *StrmUploadTaskRepository
StrmDirCache *StrmDirCacheRepository
ScrapeTask *ScrapeTaskRepository
EmbyMount *EmbyMountRepository
}
// New 将每个 repository 连接到单个 *gorm.DB。
func New(db *gorm.DB) *Container {
@@ -60,5 +62,7 @@ func New(db *gorm.DB) *Container {
StrmDownload: &StrmDownloadTaskRepository{db: db},
StrmUpload: &StrmUploadTaskRepository{db: db},
StrmDirCache: &StrmDirCacheRepository{db: db},
ScrapeTask: &ScrapeTaskRepository{db: db},
EmbyMount: &EmbyMountRepository{db: db},
}
}
@@ -0,0 +1,274 @@
package repository
import (
"context"
"errors"
"strings"
"sync"
"time"
"gorm.io/gorm"
"github.com/ShukeBta/MMTL/internal/model"
)
var scrapeClaimMu sync.Mutex
// ScrapeTaskRepository persists model.ScrapeTask.
type ScrapeTaskRepository struct{ db *gorm.DB }
func (r *ScrapeTaskRepository) Create(ctx context.Context, t *model.ScrapeTask) error {
return withSQLiteBusyRetry(ctx, func() error {
return r.db.WithContext(ctx).Create(t).Error
})
}
func (r *ScrapeTaskRepository) CreateBatch(ctx context.Context, tasks []model.ScrapeTask) error {
if len(tasks) == 0 {
return nil
}
return withSQLiteBusyRetry(ctx, func() error {
return r.db.WithContext(ctx).CreateInBatches(tasks, 100).Error
})
}
func (r *ScrapeTaskRepository) FindByID(ctx context.Context, id string) (*model.ScrapeTask, error) {
var t model.ScrapeTask
err := r.db.WithContext(ctx).Where("id = ?", id).First(&t).Error
if errors.Is(err, gorm.ErrRecordNotFound) {
return nil, nil
}
return &t, err
}
func (r *ScrapeTaskRepository) FindActiveByMediaID(ctx context.Context, mediaID string) (*model.ScrapeTask, error) {
var t model.ScrapeTask
err := r.db.WithContext(ctx).
Where("media_id = ? AND status IN ?", mediaID, []string{model.ScrapeTaskPending, model.ScrapeTaskRunning}).
First(&t).Error
if errors.Is(err, gorm.ErrRecordNotFound) {
return nil, nil
}
return &t, err
}
func (r *ScrapeTaskRepository) List(ctx context.Context, status string, page, pageSize int) ([]model.ScrapeTask, int64, error) {
if page < 1 {
page = 1
}
if pageSize < 1 || pageSize > 200 {
pageSize = 50
}
q := r.db.WithContext(ctx).Model(&model.ScrapeTask{})
if strings.TrimSpace(status) != "" && status != "all" {
q = q.Where("status = ?", strings.TrimSpace(status))
}
var total int64
if err := q.Count(&total).Error; err != nil {
return nil, 0, err
}
var rows []model.ScrapeTask
err := q.Order("created_at desc").
Offset((page - 1) * pageSize).
Limit(pageSize).
Find(&rows).Error
return rows, total, err
}
func (r *ScrapeTaskRepository) CountByStatus(ctx context.Context) (map[string]int64, error) {
var rows []struct {
Status string
Count int64
}
err := r.db.WithContext(ctx).Model(&model.ScrapeTask{}).
Select("status, count(*) as count").
Group("status").Scan(&rows).Error
if err != nil {
return nil, err
}
out := map[string]int64{}
for _, row := range rows {
out[row.Status] = row.Count
}
return out, nil
}
// ClaimPending picks pending scrape tasks and marks them running.
func (r *ScrapeTaskRepository) ClaimPending(ctx context.Context, limit int) ([]model.ScrapeTask, error) {
scrapeClaimMu.Lock()
defer scrapeClaimMu.Unlock()
var rows []model.ScrapeTask
err := withSQLiteBusyRetry(ctx, func() error {
return r.db.WithContext(ctx).Transaction(func(tx *gorm.DB) error {
if err := tx.Where("status = ?", model.ScrapeTaskPending).
Order("created_at asc").Limit(limit).Find(&rows).Error; err != nil {
return err
}
if len(rows) == 0 {
return nil
}
ids := make([]string, 0, len(rows))
now := time.Now()
for i := range rows {
ids = append(ids, rows[i].ID)
rows[i].Status = model.ScrapeTaskRunning
rows[i].StartedAt = &now
}
return tx.Model(&model.ScrapeTask{}).Where("id IN ?", ids).
Updates(map[string]any{"status": model.ScrapeTaskRunning, "started_at": now}).Error
})
})
if err != nil {
return nil, err
}
return rows, nil
}
func (r *ScrapeTaskRepository) Update(ctx context.Context, t *model.ScrapeTask) error {
return withSQLiteBusyRetry(ctx, func() error {
return r.db.WithContext(ctx).Model(&model.ScrapeTask{}).Where("id = ?", t.ID).Updates(map[string]any{
"status": t.Status,
"error": t.Error,
"provider": t.Provider,
"matched_title": t.MatchedTitle,
"matched_year": t.MatchedYear,
"poster_url": t.PosterURL,
"backdrop_url": t.BackdropURL,
"retry_count": t.RetryCount,
"started_at": t.StartedAt,
"finished_at": t.FinishedAt,
"updated_at": time.Now(),
}).Error
})
}
func (r *ScrapeTaskRepository) Delete(ctx context.Context, id string) error {
return withSQLiteBusyRetry(ctx, func() error {
return r.db.WithContext(ctx).Unscoped().Where("id = ?", id).Delete(&model.ScrapeTask{}).Error
})
}
func (r *ScrapeTaskRepository) DeleteBatch(ctx context.Context, ids []string) (int64, error) {
if len(ids) == 0 {
return 0, nil
}
var count int64
err := withSQLiteBusyRetry(ctx, func() error {
res := r.db.WithContext(ctx).Unscoped().Where("id IN ?", ids).Delete(&model.ScrapeTask{})
count = res.RowsAffected
return res.Error
})
return count, err
}
func (r *ScrapeTaskRepository) RetryBatch(ctx context.Context, ids []string) (int64, error) {
if len(ids) == 0 {
return 0, nil
}
var count int64
err := withSQLiteBusyRetry(ctx, func() error {
res := r.db.WithContext(ctx).Model(&model.ScrapeTask{}).
Where("id IN ? AND status IN ?", ids, []string{model.ScrapeTaskFailed, model.ScrapeTaskCanceled}).
Updates(map[string]any{
"status": model.ScrapeTaskPending,
"error": "",
"retry_count": 0,
"started_at": nil,
"finished_at": nil,
"updated_at": time.Now(),
})
count = res.RowsAffected
return res.Error
})
return count, err
}
func (r *ScrapeTaskRepository) CancelBatch(ctx context.Context, ids []string) (int64, error) {
if len(ids) == 0 {
return 0, nil
}
now := time.Now()
var count int64
err := withSQLiteBusyRetry(ctx, func() error {
res := r.db.WithContext(ctx).Model(&model.ScrapeTask{}).
Where("id IN ? AND status IN ?", ids, []string{model.ScrapeTaskPending, model.ScrapeTaskRunning}).
Updates(map[string]any{
"status": model.ScrapeTaskCanceled,
"error": "已批量取消",
"finished_at": now,
"updated_at": now,
})
count = res.RowsAffected
return res.Error
})
return count, err
}
func (r *ScrapeTaskRepository) ClearDone(ctx context.Context) (int64, error) {
var count int64
err := withSQLiteBusyRetry(ctx, func() error {
res := r.db.WithContext(ctx).Unscoped().Where("status = ?", model.ScrapeTaskDone).Delete(&model.ScrapeTask{})
count = res.RowsAffected
return res.Error
})
return count, err
}
func (r *ScrapeTaskRepository) ClearFinished(ctx context.Context) (int64, error) {
var count int64
err := withSQLiteBusyRetry(ctx, func() error {
res := r.db.WithContext(ctx).Unscoped().Where("status IN ?", []string{model.ScrapeTaskDone, model.ScrapeTaskFailed, model.ScrapeTaskCanceled}).
Delete(&model.ScrapeTask{})
count = res.RowsAffected
return res.Error
})
return count, err
}
func (r *ScrapeTaskRepository) ClearCanceled(ctx context.Context) (int64, error) {
var count int64
err := withSQLiteBusyRetry(ctx, func() error {
res := r.db.WithContext(ctx).Unscoped().Where("status = ?", model.ScrapeTaskCanceled).Delete(&model.ScrapeTask{})
count = res.RowsAffected
return res.Error
})
return count, err
}
func (r *ScrapeTaskRepository) RetryAllFailed(ctx context.Context) (int64, error) {
var count int64
err := withSQLiteBusyRetry(ctx, func() error {
res := r.db.WithContext(ctx).Model(&model.ScrapeTask{}).
Where("status = ?", model.ScrapeTaskFailed).
Updates(map[string]any{
"status": model.ScrapeTaskPending,
"error": "",
"retry_count": 0,
"started_at": nil,
"finished_at": nil,
"updated_at": time.Now(),
})
count = res.RowsAffected
return res.Error
})
return count, err
}
func (r *ScrapeTaskRepository) CancelPending(ctx context.Context) (int64, error) {
now := time.Now()
var count int64
err := withSQLiteBusyRetry(ctx, func() error {
res := r.db.WithContext(ctx).Model(&model.ScrapeTask{}).
Where("status IN ?", []string{model.ScrapeTaskPending, model.ScrapeTaskRunning}).
Updates(map[string]any{
"status": model.ScrapeTaskCanceled,
"error": "已批量取消",
"finished_at": now,
"updated_at": now,
})
count = res.RowsAffected
return res.Error
})
return count, err
}
+182
View File
@@ -170,6 +170,24 @@ func (r *StrmSyncRecordRepository) List(ctx context.Context, syncPathID string,
return rows, err
}
// Delete 删除单条同步记录(物理删除)。
func (r *StrmSyncRecordRepository) Delete(ctx context.Context, id string) error {
return withSQLiteBusyRetry(ctx, func() error {
return r.db.WithContext(ctx).Unscoped().Where("id = ?", id).Delete(&model.StrmSyncRecord{}).Error
})
}
// DeleteBySyncPathID 删除某同步目录下的全部同步记录(删除同步目录时级联清理)。
func (r *StrmSyncRecordRepository) DeleteBySyncPathID(ctx context.Context, syncPathID string) (int64, error) {
var count int64
err := withSQLiteBusyRetry(ctx, func() error {
res := r.db.WithContext(ctx).Unscoped().Where("sync_path_id = ?", syncPathID).Delete(&model.StrmSyncRecord{})
count = res.RowsAffected
return res.Error
})
return count, err
}
// ─── StrmDownloadTask ──────────────────────────────────────────────────────────
// StrmDownloadTaskRepository persists model.StrmDownloadTask.
@@ -290,6 +308,66 @@ func (r *StrmDownloadTaskRepository) Delete(ctx context.Context, id string) erro
})
}
// DeleteBatch 批量删除指定 ID 的下载任务。
func (r *StrmDownloadTaskRepository) DeleteBatch(ctx context.Context, ids []string) (int64, error) {
if len(ids) == 0 {
return 0, nil
}
var count int64
err := withSQLiteBusyRetry(ctx, func() error {
res := r.db.WithContext(ctx).Unscoped().Where("id IN ?", ids).Delete(&model.StrmDownloadTask{})
count = res.RowsAffected
return res.Error
})
return count, err
}
// RetryBatch 批量重试指定 ID 的失败/已取消下载任务。
func (r *StrmDownloadTaskRepository) RetryBatch(ctx context.Context, ids []string) (int64, error) {
if len(ids) == 0 {
return 0, nil
}
var count int64
err := withSQLiteBusyRetry(ctx, func() error {
res := r.db.WithContext(ctx).Model(&model.StrmDownloadTask{}).
Where("id IN ? AND status IN ?", ids, []string{model.StrmTaskFailed, model.StrmTaskCanceled}).
Updates(map[string]any{
"status": model.StrmTaskPending,
"error": "",
"retry_count": 0,
"next_try_at": nil,
"started_at": nil,
"finished_at": nil,
"updated_at": time.Now(),
})
count = res.RowsAffected
return res.Error
})
return count, err
}
// CancelBatch 批量取消指定 ID 的排队/进行中下载任务。
func (r *StrmDownloadTaskRepository) CancelBatch(ctx context.Context, ids []string) (int64, error) {
if len(ids) == 0 {
return 0, nil
}
now := time.Now()
var count int64
err := withSQLiteBusyRetry(ctx, func() error {
res := r.db.WithContext(ctx).Model(&model.StrmDownloadTask{}).
Where("id IN ? AND status IN ?", ids, []string{model.StrmTaskPending, model.StrmTaskRunning}).
Updates(map[string]any{
"status": model.StrmTaskCanceled,
"error": "已批量取消",
"finished_at": now,
"updated_at": now,
})
count = res.RowsAffected
return res.Error
})
return count, err
}
// ClearDone 清空全部已完成下载任务。
func (r *StrmDownloadTaskRepository) ClearDone(ctx context.Context) (int64, error) {
var count int64
@@ -537,6 +615,89 @@ func (r *StrmUploadTaskRepository) Delete(ctx context.Context, id string) error
})
}
// DeleteBatch 批量删除指定 ID 的上传任务。
func (r *StrmUploadTaskRepository) DeleteBatch(ctx context.Context, ids []string) (int64, error) {
if len(ids) == 0 {
return 0, nil
}
var count int64
err := withSQLiteBusyRetry(ctx, func() error {
res := r.db.WithContext(ctx).Unscoped().Where("id IN ?", ids).Delete(&model.StrmUploadTask{})
count = res.RowsAffected
return res.Error
})
return count, err
}
// RetryBatch 批量重试指定 ID 的失败/已取消上传任务。
func (r *StrmUploadTaskRepository) RetryBatch(ctx context.Context, ids []string) (int64, error) {
if len(ids) == 0 {
return 0, nil
}
var count int64
err := withSQLiteBusyRetry(ctx, func() error {
res := r.db.WithContext(ctx).Model(&model.StrmUploadTask{}).
Where("id IN ? AND status IN ?", ids, []string{model.StrmTaskFailed, model.StrmTaskCanceled}).
Updates(map[string]any{
"status": model.StrmTaskPending,
"error": "",
"retry_count": 0,
"next_try_at": nil,
"started_at": nil,
"finished_at": nil,
"updated_at": time.Now(),
})
count = res.RowsAffected
return res.Error
})
return count, err
}
// CancelBatch 批量取消指定 ID 的排队/进行中上传任务。
func (r *StrmUploadTaskRepository) CancelBatch(ctx context.Context, ids []string) (int64, error) {
if len(ids) == 0 {
return 0, nil
}
now := time.Now()
var count int64
err := withSQLiteBusyRetry(ctx, func() error {
res := r.db.WithContext(ctx).Model(&model.StrmUploadTask{}).
Where("id IN ? AND status IN ?", ids, []string{model.StrmTaskPending, model.StrmTaskRunning}).
Updates(map[string]any{
"status": model.StrmTaskCanceled,
"error": "已批量取消",
"finished_at": now,
"updated_at": now,
})
count = res.RowsAffected
return res.Error
})
return count, err
}
// ClearDone 清空全部已完成上传任务。
func (r *StrmUploadTaskRepository) ClearDone(ctx context.Context) (int64, error) {
var count int64
err := withSQLiteBusyRetry(ctx, func() error {
res := r.db.WithContext(ctx).Unscoped().Where("status = ?", model.StrmTaskDone).Delete(&model.StrmUploadTask{})
count = res.RowsAffected
return res.Error
})
return count, err
}
// ClearFinished 清空全部已完成与失败上传任务(包括已完成、失败及取消)。
func (r *StrmUploadTaskRepository) ClearFinished(ctx context.Context) (int64, error) {
var count int64
err := withSQLiteBusyRetry(ctx, func() error {
res := r.db.WithContext(ctx).Unscoped().Where("status IN ?", []string{model.StrmTaskDone, model.StrmTaskFailed, model.StrmTaskCanceled}).
Delete(&model.StrmUploadTask{})
count = res.RowsAffected
return res.Error
})
return count, err
}
// ClearCanceled 清空全部已取消上传任务。
func (r *StrmUploadTaskRepository) ClearCanceled(ctx context.Context) (int64, error) {
var count int64
@@ -548,6 +709,27 @@ func (r *StrmUploadTaskRepository) ClearCanceled(ctx context.Context) (int64, er
return count, err
}
// RetryAllFailed 把所有失败任务重置回待处理,清空错误与重试计数。
func (r *StrmUploadTaskRepository) RetryAllFailed(ctx context.Context) (int64, error) {
var count int64
err := withSQLiteBusyRetry(ctx, func() error {
res := r.db.WithContext(ctx).Model(&model.StrmUploadTask{}).
Where("status = ?", model.StrmTaskFailed).
Updates(map[string]any{
"status": model.StrmTaskPending,
"error": "",
"retry_count": 0,
"next_try_at": nil,
"started_at": nil,
"finished_at": nil,
"updated_at": time.Now(),
})
count = res.RowsAffected
return res.Error
})
return count, err
}
// CancelPending 批量取消所有排队中和进行中的任务。
func (r *StrmUploadTaskRepository) CancelPending(ctx context.Context) (int64, error) {
now := time.Now()
+4 -1
View File
@@ -30,6 +30,7 @@ const (
Type115 = "cloud115" // 115 网盘
TypeCloudDrive2 = "clouddrive2" // CloudDrive2 桥接网盘
TypeOpenList = "openlist" // OpenList / AList-compatible bridge
TypeEmbyRemote = "emby_remote" // 远程 Emby 服务器(API 网关挂载)
)
// ErrUnsupported is returned for an unknown provider type.
@@ -101,6 +102,8 @@ func New(typ string, cfg map[string]any, client *http.Client) (Provider, error)
return newCloudDrive2(cfg, client), nil
case TypeOpenList:
return newOpenList(cfg, client), nil
case TypeEmbyRemote:
return newEmby(cfg, client), nil
default:
return nil, ErrUnsupported
}
@@ -108,7 +111,7 @@ func New(typ string, cfg map[string]any, client *http.Client) (Provider, error)
// IsCloudType reports whether typ is a cloud-disk provider.
func IsCloudType(typ string) bool {
return typ == Type115 || typ == TypeCloudDrive2 || typ == TypeOpenList
return typ == Type115 || typ == TypeCloudDrive2 || typ == TypeOpenList || typ == TypeEmbyRemote
}
// str coerces a config value to a trimmed string.
+246
View File
@@ -0,0 +1,246 @@
// Emby remote provider: exposes a remote Emby server through the same
// Provider interface used by cloud disks, so account CRUD / connectivity
// test / directory browser work unchanged. This is a thin adapter — the
// federated Emby API aggregation (Views / Items / PlaybackInfo / streaming
// proxy) lives in service.EmbyRemoteService and does not go through the
// cloud-disk sync machinery.
package cloud
import (
"context"
"encoding/json"
"errors"
"fmt"
"io"
"net/http"
"net/url"
"strings"
)
// Emby 远程挂载类型(service 层聚合走 EmbyRemoteService,不走 STRM 同步)。
// embyProvider implements Provider against a remote Emby server using an
// api_key (token) for authentication. DirectLink.Resolve returns the remote
// stream URL; whether MMTL reverse-proxies the bytes is decided by the
// emby.proxy_play account config (defaults to off).
type embyProvider struct {
base string // e.g. http://host:8096(自动补 /emby 前缀)
username string
password string
token string // api_key
userID string // 远程用户 Id
proxyPlay bool
client *http.Client
}
type embyUserPayload struct {
Id string `json:"Id"`
}
type embyLoginResponse struct {
AccessToken string `json:"AccessToken"`
User embyUserPayload `json:"User"`
}
type embyPingResponse struct {
ServerName string `json:"ServerName"`
}
// newEmby builds the provider from the account config map.
func newEmby(cfg map[string]any, client *http.Client) Provider {
p := &embyProvider{
base: strings.TrimRight(str(cfg["url"]), "/"),
username: str(cfg["username"]),
password: str(cfg["password"]),
token: firstNonEmpty(str(cfg["api_key"]), str(cfg["token"])),
userID: str(cfg["remote_user_id"]),
proxyPlay: boolish(cfg["proxy_play"]),
client: client,
}
if p.client == nil {
p.client = &http.Client{Transport: &embyUATransport{base: http.DefaultTransport}}
}
return p
}
// embyUATransport 给远程 Emby 请求注入浏览器 UA(防 Cloudflare 风控拦截)。
type embyUATransport struct {
base http.RoundTripper
}
func (t *embyUATransport) RoundTrip(req *http.Request) (*http.Response, error) {
if strings.TrimSpace(req.Header.Get("User-Agent")) == "" {
req.Header.Set("User-Agent", defaultUA)
}
return t.base.RoundTrip(req)
}
// embyBase normalizes the address so requests go to /emby/... endpoints.
func (p *embyProvider) embyBase() string {
base := strings.TrimRight(p.base, "/")
if !strings.Contains(base, "/emby") {
base += "/emby"
}
return base
}
// externalBase 不追加 /emby(内嵌媒体资源 URL 使用 /emby 会更贴近习惯,此处
// 与 embyBase 保持一致:所有端点统一以 /emby 开头)。
func (p *embyProvider) apiBase() string { return p.embyBase() }
func (p *embyProvider) Type() string { return TypeEmbyRemote }
// Ping 验证地址连通性与凭据(/System/Info)。
func (p *embyProvider) Ping(ctx context.Context) error {
if p.base == "" {
return errors.New("缺少 Emby 地址")
}
token, err := p.ensureToken(ctx)
if err != nil {
return err
}
return p.doJSON(ctx, http.MethodGet, "/System/Info", nil, token, &embyPingResponse{})
}
// doJSON 向远程 Emby 发起带 api_key 的请求并解析 JSON 响应。
func (p *embyProvider) doJSON(ctx context.Context, method, path string, body io.Reader, token string, out any) error {
endpoint := p.apiBase() + path
if token != "" {
sep := "?"
if strings.Contains(endpoint, "?") {
sep = "&"
}
endpoint += sep + "api_key=" + url.QueryEscape(token)
}
req, err := http.NewRequestWithContext(ctx, method, endpoint, body)
if err != nil {
return err
}
req.Header.Set("X-Emby-Token", token)
if body != nil {
req.Header.Set("Content-Type", "application/json")
}
resp, err := p.client.Do(req)
if err != nil {
return err
}
defer resp.Body.Close()
if resp.StatusCode >= 300 {
if resp.StatusCode == http.StatusUnauthorized {
return ErrEmbyUnauthorized
}
data, _ := io.ReadAll(io.LimitReader(resp.Body, 512))
return fmt.Errorf("emby 请求失败(%d): %s", resp.StatusCode, strings.TrimSpace(string(data)))
}
if out == nil {
return nil
}
return json.NewDecoder(resp.Body).Decode(out)
}
// ErrEmbyUnauthorized 表示远程凭据失效(触发重新认证/打回测试)。
var ErrEmbyUnauthorized = errors.New("emby 认证失败或凭据已失效")
// ensureToken 返回可用 api_key:已有则直接用,否则尝试账号密码认证。
func (p *embyProvider) ensureToken(ctx context.Context) (string, error) {
if strings.TrimSpace(p.token) != "" {
return p.token, nil
}
if strings.TrimSpace(p.username) == "" {
return "", errors.New("缺少 Emby 凭据(token 或 用户名/密码)")
}
payload := map[string]string{"Username": p.username, "Pw": p.password}
data, err := json.Marshal(payload)
if err != nil {
return "", err
}
req, err := http.NewRequestWithContext(ctx, http.MethodPost, p.apiBase()+"/Users/AuthenticateByName", strings.NewReader(string(data)))
if err != nil {
return "", err
}
req.Header.Set("Content-Type", "application/json")
req.Header.Set("X-Emby-Authorization", `MediaBrowser Client="MMTL", Device="MMTL-Federated", DeviceId="mmtl-federated", Version="1.0"`)
resp, err := p.client.Do(req)
if err != nil {
return "", err
}
defer resp.Body.Close()
if resp.StatusCode >= 300 {
return "", fmt.Errorf("emby 登录失败(%d)", resp.StatusCode)
}
var login embyLoginResponse
if err := json.NewDecoder(resp.Body).Decode(&login); err != nil {
return "", err
}
if strings.TrimSpace(login.AccessToken) == "" {
return "", errors.New("emby 登录成功但未返回 AccessToken")
}
p.token = login.AccessToken
if login.User.Id != "" {
p.userID = login.User.Id
}
return p.token, nil
}
// embyItemSummary 目录浏览所需的最小 Emby 条目字段。
type embyItemSummary struct {
Id string `json:"Id"`
Name string `json:"Name"`
Type string `json:"Type"`
IsFolder bool `json:"IsFolder"`
ChildCount int `json:"ChildCount"`
RunTimeTicks int64 `json:"RunTimeTicks"`
}
type embyItemListResponse struct {
Items []embyItemSummary `json:"Items"`
}
// List 把远程媒体库(View)展开为目录树:dirID 为空=媒体库列表;否则返回该
// 目录(Movie/Series/Season/Folder)下的条目。用于账号「浏览目录」调试入口。
func (p *embyProvider) List(ctx context.Context, dirID string) ([]FileEntry, error) {
token, err := p.ensureToken(ctx)
if err != nil {
return nil, err
}
userID := p.userID
if userID == "" {
userID = "0" // 某些 Emby 允许用 0 代表管理员
}
path := "/Users/" + url.PathEscape(userID) + "/Items"
if dirID != "" {
path += "?ParentId=" + url.QueryEscape(dirID)
} else {
path += "?IncludeItemTypes=CollectionFolder"
}
var out embyItemListResponse
if err := p.doJSON(ctx, http.MethodGet, path, nil, token, &out); err != nil {
return nil, err
}
entries := make([]FileEntry, 0, len(out.Items))
for _, it := range out.Items {
size := int64(0)
if it.RunTimeTicks > 0 {
size = it.RunTimeTicks / 10_000_000 // 秒
}
entries = append(entries, FileEntry{
ID: it.Id,
Name: it.Name,
IsDir: it.IsFolder || it.Type != "Movie",
Size: size,
})
}
return entries, nil
}
// Resolve 返回远程 Emby 直链。Proxy=true 时由调用方(StrmService.ProxyDirect)
// 反向代理流量;false 时 302 到直链。默认不代理(播放字节不经过 MMTL)。
func (p *embyProvider) Resolve(ctx context.Context, fileRef string) (*DirectLink, error) {
token, err := p.ensureToken(ctx)
if err != nil {
return nil, err
}
u := p.apiBase() + "/Videos/" + url.PathEscape(fileRef) + "/stream"
u += "?api_key=" + url.QueryEscape(token) + "&Static=true&MediaSourceId=" + url.QueryEscape(fileRef)
return &DirectLink{URL: u, Headers: map[string]string{"X-Emby-Token": token}, Proxy: p.proxyPlay}, nil
}
+136
View File
@@ -0,0 +1,136 @@
package cloud
import (
"context"
"net/http"
"net/http/httptest"
"strings"
"testing"
)
// fakeEmbyServer 记录请求,按路径返回远程 Emby 风格响应。
func fakeEmbyServer(t *testing.T) *httptest.Server {
return httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
switch {
case r.Method == http.MethodPost && r.URL.Path == "/emby/Users/AuthenticateByName":
w.Header().Set("Content-Type", "application/json")
_, _ = w.Write([]byte(`{"AccessToken":"remote-token","User":{"Id":"user-9"}}`))
case r.URL.Path == "/emby/System/Info":
w.Header().Set("Content-Type", "application/json")
_, _ = w.Write([]byte(`{"ServerName":"RemoteEmby"}`))
case r.URL.Path == "/emby/Users/user-9/Items" && r.URL.Query().Get("ParentId") == "":
w.Header().Set("Content-Type", "application/json")
_, _ = w.Write([]byte(`{"Items":[{"Id":"view-1","Name":"Movies","Type":"CollectionFolder","IsFolder":true}]}`))
case r.URL.Path == "/emby/Users/user-9/Items":
w.Header().Set("Content-Type", "application/json")
_, _ = w.Write([]byte(`{"Items":[{"Id":"movie-1","Name":"Avatar","Type":"Movie","IsFolder":false}]}`))
case strings.Contains(r.URL.Path, "/emby/Videos/movie-1/stream"):
w.Header().Set("Content-Type", "video/mp4")
_, _ = w.Write([]byte("fake-video-bytes"))
default:
t.Fatalf("unexpected request %s %s", r.Method, r.URL.Path)
}
}))
}
func TestEmbyProviderPingAuthenticatesAndGetsToken(t *testing.T) {
srv := fakeEmbyServer(t)
defer srv.Close()
p, err := New(TypeEmbyRemote, map[string]any{
"url": srv.URL,
"username": "alice",
"password": "secret",
}, srv.Client())
if err != nil {
t.Fatal(err)
}
if err := p.Ping(context.Background()); err != nil {
t.Fatalf("ping: %v", err)
}
// 认证成功后 token 被记住,第二次 Ping 不应再走登录。
if err := p.Ping(context.Background()); err != nil {
t.Fatalf("ping 2: %v", err)
}
}
func TestEmbyProviderListViewsAndChildren(t *testing.T) {
srv := fakeEmbyServer(t)
defer srv.Close()
p, err := New(TypeEmbyRemote, map[string]any{
"url": srv.URL,
"api_key": "fixed-token",
"remote_user_id": "user-9",
}, srv.Client())
if err != nil {
t.Fatal(err)
}
root, err := p.List(context.Background(), "")
if err != nil {
t.Fatalf("list root: %v", err)
}
if len(root) != 1 || root[0].Name != "Movies" || !root[0].IsDir {
t.Fatalf("root listing = %+v", root)
}
children, err := p.List(context.Background(), "view-1")
if err != nil {
t.Fatalf("list children: %v", err)
}
if len(children) != 1 || children[0].Name != "Avatar" || children[0].ID != "movie-1" {
t.Fatalf("children = %+v", children)
}
}
func TestEmbyProviderResolveDirectURLByDefault(t *testing.T) {
srv := fakeEmbyServer(t)
defer srv.Close()
p, err := New(TypeEmbyRemote, map[string]any{
"url": srv.URL,
"api_key": "fixed-token",
"remote_user_id": "user-9",
}, srv.Client())
if err != nil {
t.Fatal(err)
}
link, err := p.Resolve(context.Background(), "movie-1")
if err != nil {
t.Fatalf("resolve: %v", err)
}
if !strings.Contains(link.URL, "/emby/Videos/movie-1/stream") {
t.Fatalf("url = %q", link.URL)
}
if !strings.Contains(link.URL, "api_key=fixed-token") {
t.Fatalf("url missing api_key: %q", link.URL)
}
// 默认不代理播放流量。
if link.Proxy {
t.Fatal("emby remote must not proxy by default")
}
}
func TestEmbyProviderResolveProxyWhenConfigured(t *testing.T) {
srv := fakeEmbyServer(t)
defer srv.Close()
p, err := New(TypeEmbyRemote, map[string]any{
"url": srv.URL,
"api_key": "fixed-token",
"remote_user_id": "user-9",
"proxy_play": "true",
}, srv.Client())
if err != nil {
t.Fatal(err)
}
link, err := p.Resolve(context.Background(), "movie-1")
if err != nil {
t.Fatalf("resolve: %v", err)
}
if !link.Proxy {
t.Fatal("proxy_play=true must mark link as proxied")
}
if link.URL == "" {
t.Fatal("proxy link must still carry the remote URL")
}
}
+46 -10
View File
@@ -17,6 +17,9 @@ package cloud
import (
"context"
"fmt"
"io"
"os"
"path/filepath"
"strings"
"github.com/ShukeBta/MMTL/internal/service/cloud115"
@@ -61,16 +64,16 @@ func (p *openAPI115Provider) List(ctx context.Context, dirID string) ([]FileEntr
if err != nil {
return nil, err
}
for _, f := range files {
out = append(out, FileEntry{
ID: f.FileId,
Name: f.FileName,
IsDir: f.Category == cloud115.TypeDir,
Size: f.FileSize,
MTime: f.Utime,
PickCode: f.PickCode,
})
}
for _, f := range files {
out = append(out, FileEntry{
ID: f.FileId,
Name: f.FileName,
IsDir: f.Category == cloud115.TypeDir,
Size: f.FileSize,
MTime: f.Utime,
PickCode: f.PickCode,
})
}
if len(files) < pageSize {
break
}
@@ -105,6 +108,39 @@ func (p *openAPI115Provider) ResolveWithUA(ctx context.Context, fileRef, ua stri
// OpenClient 暴露底层客户端(token 刷新用)。
func (p *openAPI115Provider) OpenClient() *cloud115.OpenClient { return p.c }
// PutFileNamed 把本地元数据上传到 115 指定父目录(parentCID 为父目录 cid)。
// io.Reader 无法携带文件名,因此走独立的 named 上传接口。将内容落为临时文件后
// 重命名为目标文件名,再交给 115 上传(/open/upload/init 的 file_name 取真实文件名)。
func (p *openAPI115Provider) PutFileNamed(ctx context.Context, parentCID, fileName string, r io.Reader) error {
tmp, err := os.CreateTemp("", "mmtl-upload-*")
if err != nil {
return fmt.Errorf("115: 创建临时文件失败:%w", err)
}
tmpPath := tmp.Name()
defer func() {
_ = tmp.Close()
_ = os.Remove(tmpPath)
}()
if _, err := io.Copy(tmp, r); err != nil {
return fmt.Errorf("115: 写入临时文件失败:%w", err)
}
if err := tmp.Close(); err != nil {
return fmt.Errorf("115: 关闭临时文件失败:%w", err)
}
// 重命名为目标文件名,保证上传到 115 后保留原始文件名
if fileName != "" && fileName != filepath.Base(tmpPath) {
namedPath := filepath.Join(filepath.Dir(tmpPath), fileName)
if err := os.Rename(tmpPath, namedPath); err == nil {
tmpPath = namedPath
}
}
_, err = p.c.Upload(ctx, tmpPath, parentCID, "", "")
if err != nil {
return err
}
return nil
}
// RefreshToken 刷新访问令牌并返回新令牌;refresh_token 失效时返回
// cloud115.IsRefreshTokenDead(err) 为 true 的错误。
func (p *openAPI115Provider) RefreshToken(refreshToken string) (*cloud115.TokenData, error) {
+33 -9
View File
@@ -185,7 +185,7 @@ func (c *OpenClient) doJSON(ctx context.Context, method, rawURL string, form map
// refresh_token 刷新后重试一次。刷新失败或重试后仍失败才返回,
// 避免长时间同步因 token 过期而整体失败。
if isTokenCode(base.Code) {
if access && c.tryRefreshTokenLocked() {
if access && c.tryRefreshTokenLocked(ctx) {
continue
}
if access {
@@ -259,19 +259,43 @@ func (c *OpenClient) doAuthJSONWithUA(ctx context.Context, method, rawURL string
}
// tryRefreshTokenLocked 并发安全地刷新 access_token;成功返回 true(调用方
// 应使用内存中的新 token 重试原请求)。refresh_token 已失效时也会清空内存 token。
func (c *OpenClient) tryRefreshTokenLocked() bool {
// 应使用内存中的新 token 重试原请求)。
//
// 对"refresh_token 本身已失效/被吊销"(IsRefreshTokenDead,如 40140114/116/119/120)
// 这类不可恢复的错误直接放弃并清空内存 token(提示需重新授权)。
// 对其它失败(网络瞬时抖动、刷新接口可重试错误码等)做指数退避重试几次再放弃,
// 避免同步长任务中途 token 到期时恰好撞上一个短暂的刷新失败就整体失败。
func (c *OpenClient) tryRefreshTokenLocked(ctx context.Context) bool {
c.tokenMu.Lock()
defer c.tokenMu.Unlock()
token, err := c.RefreshToken(c.RefreshTokenStr)
if err != nil {
for attempt := 0; attempt < refreshAttempts; attempt++ {
token, err := c.RefreshToken(c.RefreshTokenStr)
if err == nil {
c.SetAuthToken(token.AccessToken, token.RefreshToken)
return true
}
if IsRefreshTokenDead(err) {
c.SetAuthToken("", "")
return false
}
// 可恢复失败:退避后重试。ctx 取消时立即放弃。
if attempt < refreshAttempts-1 {
select {
case <-ctx.Done():
return false
case <-time.After(refreshBackoff(attempt)):
}
}
return false
}
c.SetAuthToken(token.AccessToken, token.RefreshToken)
return true
return false
}
// refreshAttempts 是刷新 access_token 失败时的最大尝试次数(含首次)。
const refreshAttempts = 3
// refreshBackoff 返回第 attempt 次(从 0 计)刷新失败后的退避时长(指数退避)。
func refreshBackoff(attempt int) time.Duration {
return time.Duration(200*(1<<attempt)) * time.Millisecond // 200ms, 400ms
}
// IsThrottleCode 判断是否为限流错误码。
@@ -281,7 +305,7 @@ func IsThrottleCode(code int) bool {
func isTokenCode(code int) bool {
switch code {
case AccessTokenAuthFail, AccessAuthInvalid, AccessTokenExpiryCode, RefreshTokenInvalid:
case AccessTokenAuthFail, AccessAuthInvalid, AccessTokenExpiryCode, AccessTokenFormatInvalid, RefreshTokenInvalid:
return true
}
return false
+97 -55
View File
@@ -371,62 +371,104 @@ func TestThrottleCodeHandling(t *testing.T) {
if err == nil {
t.Fatal("expected throttle error")
}
if !tm.IsThrottled() {
t.Fatal("code 770004 should trigger throttle status")
}
if !tm.IsThrottled() {
t.Fatal("code 770004 should trigger throttle status")
}
}
func TestRemoteFileDetailRelativePath(t *testing.T) {
rootCID := "3238787832374488117" // 影视库
// 场景 1:115 目录 paths 中只有祖先目录链,不包含自身
d1 := &RemoteFileDetail{
FileId: "3251154147730910635",
FileName: "出包王女",
Paths: []struct {
FileId string `json:"file_id"`
Name string `json:"file_name"`
}{
{FileId: "0", Name: "根目录"},
{FileId: "3238787832374488117", Name: "影视库"},
{FileId: "3238787913223892116", Name: "动漫"},
},
}
if got := d1.RelativePath(rootCID); got != "动漫/出包王女" {
t.Errorf("d1.RelativePath = %q, want %q", got, "动漫/出包王女")
}
func TestRemoteFileDetailRelativePath(t *testing.T) {
rootCID := "3238787832374488117" // 影视库
// 场景 1:115 目录 paths 中只有祖先目录链,不包含自身
d1 := &RemoteFileDetail{
FileId: "3251154147730910635",
FileName: "出包王女",
Paths: []struct {
FileId string `json:"file_id"`
Name string `json:"file_name"`
}{
{FileId: "0", Name: "根目录"},
{FileId: "3238787832374488117", Name: "影视库"},
{FileId: "3238787913223892116", Name: "动漫"},
},
}
if got := d1.RelativePath(rootCID); got != "动漫/出包王女" {
t.Errorf("d1.RelativePath = %q, want %q", got, "动漫/出包王女")
}
// 场景 2:祖先中间目录,自身在 paths 末尾
d2 := &RemoteFileDetail{
FileId: "3238787913223892116",
FileName: "动漫",
Paths: []struct {
FileId string `json:"file_id"`
Name string `json:"file_name"`
}{
{FileId: "0", Name: "根目录"},
{FileId: "3238787832374488117", Name: "影视库"},
{FileId: "3238787913223892116", Name: "动漫"},
},
}
if got := d2.RelativePath(rootCID); got != "动漫" {
t.Errorf("d2.RelativePath = %q, want %q", got, "动漫")
}
// 场景 3:根同步目录自身
d3 := &RemoteFileDetail{
FileId: rootCID,
FileName: "影视库",
Paths: []struct {
FileId string `json:"file_id"`
Name string `json:"file_name"`
}{
{FileId: "0", Name: "根目录"},
{FileId: rootCID, Name: "影视库"},
},
}
if got := d3.RelativePath(rootCID); got != "" {
t.Errorf("d3.RelativePath = %q, want %q", got, "")
}
// 场景 2:祖先中间目录,自身在 paths 末尾
d2 := &RemoteFileDetail{
FileId: "3238787913223892116",
FileName: "动漫",
Paths: []struct {
FileId string `json:"file_id"`
Name string `json:"file_name"`
}{
{FileId: "0", Name: "根目录"},
{FileId: "3238787832374488117", Name: "影视库"},
{FileId: "3238787913223892116", Name: "动漫"},
},
}
if got := d2.RelativePath(rootCID); got != "动漫" {
t.Errorf("d2.RelativePath = %q, want %q", got, "动漫")
}
// 场景 3:根同步目录自身
d3 := &RemoteFileDetail{
FileId: rootCID,
FileName: "影视库",
Paths: []struct {
FileId string `json:"file_id"`
Name string `json:"file_name"`
}{
{FileId: "0", Name: "根目录"},
{FileId: rootCID, Name: "影视库"},
},
}
if got := d3.RelativePath(rootCID); got != "" {
t.Errorf("d3.RelativePath = %q, want %q", got, "")
}
}
// TestFsListRefreshContinue 验证 access_token 在请求中途过期(40140126)时:
// 自动用 refresh_token 刷新得到新 token,然后对原请求重试成功(同步得以继续)。
func TestFsListRefreshContinue(t *testing.T) {
var filesCalls int
var refreshCalls int
mockAPI(t, func(w http.ResponseWriter, r *http.Request) {
switch r.URL.Path {
case "/open/refreshToken":
refreshCalls++
w.Write([]byte(`{"state":true,"data":{"access_token":"at2","refresh_token":"rt2","expires_in":7200}}`))
case "/open/ufile/files":
filesCalls++
switch filesCalls {
case 1:
// 第一次用旧 access_token,返回过期错误,应触发刷新
w.Write([]byte(`{"state":false,"code":40140126,"message":"access_token 校验失败"}`))
default:
// 刷新后续请求应使用新 access_token
if got := r.Header.Get("Authorization"); got != "Bearer at2" {
t.Errorf("retried request auth = %q, want Bearer at2", got)
}
w.Write([]byte(`{"state":true,"path":[],"data":[{"fid":"200","fc":"1","fn":"a.mkv","fs":123,"pc":"pickA"}]}`))
}
default:
t.Errorf("unexpected path %s", r.URL.Path)
}
})
c := NewOpenClient("100195125", "at1", "rt1")
files, _, err := c.GetFsList(context.Background(), "0", 0, 100)
if err != nil {
t.Fatalf("expected sync to continue after refresh, got error: %v", err)
}
if filesCalls != 2 {
t.Fatalf("want 2 files calls (original + retried), got %d", filesCalls)
}
if refreshCalls == 0 {
t.Fatal("expected refresh_token to be used once")
}
if len(files) != 1 {
t.Fatalf("want 1 file, got %d", len(files))
}
}
+8 -7
View File
@@ -21,13 +21,14 @@ var (
const (
// 业务错误码
AccessTokenAuthFail = 40140126 // 访问过期,需刷新
AccessTokenExpiryCode = 40140125 // 访问过期,需刷新
AccessAuthInvalid = 40140124 // 访问无效,需刷新
RefreshTokenInvalid = 40140116 // 需重新授权
TokenRefreshFail = 40140121 // 刷新失败,可重试
RequestMaxLimitCode = 770004 // 访问频率过高
RequestRateLimitCode = 406 // 达到访问上限
AccessTokenAuthFail = 40140126 // 访问过期,需刷新
AccessTokenExpiryCode = 40140125 // 访问过期,需刷新
AccessAuthInvalid = 40140124 // 访问无效,需刷新
AccessTokenFormatInvalid = 40140123 // access_token 格式错误,需刷新
RefreshTokenInvalid = 40140116 // 需重新授权
TokenRefreshFail = 40140121 // 刷新失败,可重试
RequestMaxLimitCode = 770004 // 访问频率过高
RequestRateLimitCode = 406 // 达到访问上限
// 刷新 token 的错误码
RefreshTokenFormatInvalid = 40140114
-1
View File
@@ -185,7 +185,6 @@ func (d *RemoteFileDetail) RelativePath(rootCID string) string {
return strings.Join(segments, "/")
}
// ─── 下载直链 ──────────────────────────────────────────────────────────────────
type downloadURLData struct {
+367
View File
@@ -0,0 +1,367 @@
// 阿里云 OSS multipart 分片上传(用于 115 元数据上传直传)。
// 使用 115 下发的临时 STS 凭证,将本地文件分片上传到 OSS,并经 complete 回调
// 通知 115 完成落盘。参考 QMediaSync 的 OSSMultipartUploader 实现。
package cloud115
import (
"context"
"encoding/base64"
"encoding/json"
"errors"
"fmt"
"io"
"os"
"sort"
"github.com/aliyun/alibabacloud-oss-go-sdk-v2/oss"
"github.com/aliyun/alibabacloud-oss-go-sdk-v2/oss/credentials"
)
const (
defaultMultipartPartSize int64 = 32 * 1024 * 1024
multipartPartAlign int64 = 1024 * 1024
maxMultipartParts int64 = 9999
maxMultipartPartSize int64 = 5 * 1024 * 1024 * 1024
)
type ossMultipartClient interface {
InitiateMultipartUpload(context.Context, *oss.InitiateMultipartUploadRequest, ...func(*oss.Options)) (*oss.InitiateMultipartUploadResult, error)
UploadPart(context.Context, *oss.UploadPartRequest, ...func(*oss.Options)) (*oss.UploadPartResult, error)
ListParts(context.Context, *oss.ListPartsRequest, ...func(*oss.Options)) (*oss.ListPartsResult, error)
CompleteMultipartUpload(context.Context, *oss.CompleteMultipartUploadRequest, ...func(*oss.Options)) (*oss.CompleteMultipartUploadResult, error)
AbortMultipartUpload(context.Context, *oss.AbortMultipartUploadRequest, ...func(*oss.Options)) (*oss.AbortMultipartUploadResult, error)
}
// OSSMultipartUploader 封装 OSS multipart 上传。
type OSSMultipartUploader struct {
client ossMultipartClient
}
// OSSMultipartUploadInput 是 multipart 上传输入。
type OSSMultipartUploadInput struct {
Bucket string
Object string
Callback string
CallbackVar string
FilePath string
FileSize int64
UploadId string
PartSize int64
PartRetryMax int
refreshClient func(context.Context) (ossMultipartClient, error)
}
// OSSMultipartUploadResult 是 multipart 上传后的结果。
type OSSMultipartUploadResult struct {
CallbackResult map[string]any
UploadId string
PartSize int64
TotalParts int
UploadedBytes int64
UploadedParts int
}
// CalculateMultipartPartSize 计算 OSS multipart 分片大小与分片数量。
func CalculateMultipartPartSize(fileSize int64) (int64, int, error) {
if fileSize < 0 {
return 0, 0, fmt.Errorf("文件大小不能为负数:%d", fileSize)
}
partSize := defaultMultipartPartSize
minPartSize := ceilDiv(fileSize, maxMultipartParts)
if minPartSize > partSize {
partSize = roundUp(minPartSize, multipartPartAlign)
}
if partSize > maxMultipartPartSize {
return 0, 0, fmt.Errorf("文件过大,所需分片大小 %d 超过 OSS 上限 %d", partSize, maxMultipartPartSize)
}
totalParts := int(ceilDiv(fileSize, partSize))
if totalParts == 0 {
totalParts = 1
}
if int64(totalParts) > maxMultipartParts {
return 0, 0, fmt.Errorf("分片数量 %d 超过上限 %d", totalParts, maxMultipartParts)
}
return partSize, totalParts, nil
}
// NewOSSMultipartUploader 创建 OSS multipart 上传器。
func NewOSSMultipartUploader(endpoint, accessKeyId, accessKeySecret, securityToken string) *OSSMultipartUploader {
return &OSSMultipartUploader{client: newOSSMultipartClient(endpoint, accessKeyId, accessKeySecret, securityToken)}
}
func newOSSMultipartClient(endpoint, accessKeyId, accessKeySecret, securityToken string) ossMultipartClient {
cfg := oss.LoadDefaultConfig().
WithCredentialsProvider(credentials.NewStaticCredentialsProvider(accessKeyId, accessKeySecret, securityToken)).
WithRegion("cn-shenzhen").
WithEndpoint(endpoint)
return oss.NewClient(cfg)
}
// UploadFile 上传文件并完成 OSS multipart,返回 complete callback 结果。
func (u *OSSMultipartUploader) UploadFile(ctx context.Context, input OSSMultipartUploadInput) (map[string]any, error) {
result, err := u.UploadFileWithResult(ctx, input)
if err != nil {
return nil, err
}
return result.CallbackResult, nil
}
// UploadFileWithResult 上传文件并返回 multipart 结果。
func (u *OSSMultipartUploader) UploadFileWithResult(ctx context.Context, input OSSMultipartUploadInput) (OSSMultipartUploadResult, error) {
if input.PartRetryMax <= 0 {
input.PartRetryMax = 3
}
partSize := input.PartSize
totalParts := 0
var err error
if partSize <= 0 {
partSize, totalParts, err = CalculateMultipartPartSize(input.FileSize)
if err != nil {
return OSSMultipartUploadResult{}, err
}
} else {
totalParts = int(ceilDiv(input.FileSize, partSize))
}
uploadId := input.UploadId
if uploadId == "" {
initResult, err := u.client.InitiateMultipartUpload(ctx, &oss.InitiateMultipartUploadRequest{
Bucket: oss.Ptr(input.Bucket),
Key: oss.Ptr(input.Object),
RequestCommon: oss.RequestCommon{
Parameters: map[string]string{"sequential": "1"},
},
})
if err != nil {
return OSSMultipartUploadResult{}, fmt.Errorf("初始化 OSS multipart 失败:%w", err)
}
if initResult.UploadId == nil || *initResult.UploadId == "" {
return OSSMultipartUploadResult{}, fmt.Errorf("初始化 OSS multipart 返回空 upload_id")
}
uploadId = *initResult.UploadId
}
existingPartMap := make(map[int32]int64)
existingParts, err := u.ListUploadedParts(ctx, input.Bucket, input.Object, uploadId)
if err == nil {
for _, part := range existingParts {
existingPartMap[part.PartNumber] = part.Size
}
}
file, err := os.Open(input.FilePath)
if err != nil {
return OSSMultipartUploadResult{}, fmt.Errorf("打开待上传文件失败:%w", err)
}
defer file.Close()
var uploadedBytes int64
uploadedParts := 0
completeParts := make([]oss.UploadPart, 0, totalParts)
for partNumber := 1; partNumber <= totalParts; partNumber++ {
offset := int64(partNumber-1) * partSize
length := minInt64(partSize, input.FileSize-offset)
if length < 0 {
length = 0
}
if existingSize, ok := existingPartMap[int32(partNumber)]; ok && existingSize == length {
uploadedBytes += length
uploadedParts++
}
etag, err := u.uploadPartWithRetry(ctx, input, uploadId, int32(partNumber), file, offset, length)
if err != nil {
return OSSMultipartUploadResult{}, err
}
uploadedBytes += length
uploadedParts++
completeParts = append(completeParts, oss.UploadPart{
PartNumber: int32(partNumber),
ETag: oss.Ptr(etag),
})
}
sort.Slice(completeParts, func(i, j int) bool {
return completeParts[i].PartNumber < completeParts[j].PartNumber
})
// 115 下发的 callback / callback_var 是 JSON 字符串,而 OSS CompleteMultipartUpload
// 要求 callback 参数为 Base64 编码后的 JSON,否则报 "The callback configuration is
// not base64 encoded"。这里把两者转为 Base64 后再提交(参考 QMediaSync 的
// BuildOSSCallbackHeaders)。
cb := input.Callback
cbVar := input.CallbackVar
if cb == "" {
return OSSMultipartUploadResult{}, errors.New("OSS callback 为空")
}
if !json.Valid([]byte(cb)) {
return OSSMultipartUploadResult{}, errors.New("解析 callback 失败:不是合法 JSON")
}
if cbVar == "" {
cbVar = "{}"
}
if !json.Valid([]byte(cbVar)) {
return OSSMultipartUploadResult{}, errors.New("解析 callback_var 失败:不是合法 JSON")
}
completeResult, err := u.client.CompleteMultipartUpload(ctx, &oss.CompleteMultipartUploadRequest{
Bucket: oss.Ptr(input.Bucket),
Key: oss.Ptr(input.Object),
UploadId: oss.Ptr(uploadId),
CompleteMultipartUpload: &oss.CompleteMultipartUpload{
Parts: completeParts,
},
Callback: oss.Ptr(base64.StdEncoding.EncodeToString([]byte(cb))),
CallbackVar: oss.Ptr(base64.StdEncoding.EncodeToString([]byte(cbVar))),
})
if err != nil {
return OSSMultipartUploadResult{}, fmt.Errorf("完成 OSS multipart 失败:%w", err)
}
return OSSMultipartUploadResult{
CallbackResult: completeResult.CallbackResult,
UploadId: uploadId,
PartSize: partSize,
TotalParts: totalParts,
UploadedBytes: uploadedBytes,
UploadedParts: uploadedParts,
}, nil
}
// ListUploadedParts 查询 OSS 已上传分片。
func (u *OSSMultipartUploader) ListUploadedParts(ctx context.Context, bucket, object, uploadId string) ([]struct {
PartNumber int32
Size int64
}, error) {
parts := []struct {
PartNumber int32
Size int64
}{}
result, err := u.client.ListParts(ctx, &oss.ListPartsRequest{
Bucket: oss.Ptr(bucket),
Key: oss.Ptr(object),
UploadId: oss.Ptr(uploadId),
MaxParts: 1000,
})
if err != nil {
return nil, fmt.Errorf("查询 OSS 已上传分片失败:%w", err)
}
for _, part := range result.Parts {
parts = append(parts, struct {
PartNumber int32
Size int64
}{PartNumber: part.PartNumber, Size: part.Size})
}
return parts, nil
}
func (u *OSSMultipartUploader) uploadPartWithRetry(
ctx context.Context,
input OSSMultipartUploadInput,
uploadId string,
partNumber int32,
file *os.File,
offset, length int64,
) (string, error) {
var lastErr error
for attempt := 0; attempt < input.PartRetryMax; attempt++ {
reader := io.NewSectionReader(file, offset, length)
result, err := u.client.UploadPart(ctx, &oss.UploadPartRequest{
Bucket: oss.Ptr(input.Bucket),
Key: oss.Ptr(input.Object),
PartNumber: partNumber,
UploadId: oss.Ptr(uploadId),
Body: reader,
ContentLength: oss.Ptr(length),
})
if err == nil {
if result.ETag == nil || *result.ETag == "" {
return "", fmt.Errorf("OSS part %d 返回空 ETag", partNumber)
}
return *result.ETag, nil
}
lastErr = err
if attempt < input.PartRetryMax-1 && input.refreshClient != nil {
refreshed, refreshErr := input.refreshClient(ctx)
if refreshErr != nil {
lastErr = refreshErr
continue
}
u.client = refreshed
}
}
return "", fmt.Errorf("上传 OSS part %d 失败:%w", partNumber, lastErr)
}
// ParseCompleteCallbackResult 校验并解析 OSS complete 后的 115 callback 结果。
func ParseCompleteCallbackResult(result map[string]any) (UploadCompleteResult, error) {
if result == nil {
return UploadCompleteResult{}, errors.New("OSS complete callback 结果为空")
}
if state, ok := result["state"].(bool); ok && !state {
return UploadCompleteResult{}, fmt.Errorf("115 callback 返回失败:%s", anyToString(result["message"]))
}
if message := anyToString(result["message"]); message != "" {
return UploadCompleteResult{}, fmt.Errorf("115 callback 返回错误:%s", message)
}
data, ok := result["data"].(map[string]any)
if !ok {
return UploadCompleteResult{}, errors.New("115 callback 缺少 data")
}
complete := UploadCompleteResult{
FileId: anyToString(data["file_id"]),
PickCode: anyToString(data["pick_code"]),
ParentId: anyToString(data["parent_id"]),
Sha1: anyToString(data["sha1"]),
Size: anyToInt64(data["size"]),
Mtime: anyToInt64(data["mtime"]),
}
if complete.FileId == "" || complete.PickCode == "" {
return UploadCompleteResult{}, errors.New("115 callback 缺少 file_id/pick_code")
}
return complete, nil
}
func ceilDiv(n, d int64) int64 {
if d <= 0 {
return 0
}
if n <= 0 {
return 0
}
return (n + d - 1) / d
}
func roundUp(n, align int64) int64 {
if align <= 0 {
return n
}
return ceilDiv(n, align) * align
}
func minInt64(a, b int64) int64 {
if a < b {
return a
}
return b
}
func anyToInt64(v any) int64 {
switch t := v.(type) {
case string:
var n int64
fmt.Sscanf(t, "%d", &n)
return n
case float64:
return int64(t)
case int64:
return t
case int:
return int64(t)
default:
return 0
}
}
func anyToString(v any) string {
if s, ok := v.(string); ok {
return s
}
return fmt.Sprintf("%v", v)
}
+7 -2
View File
@@ -26,10 +26,15 @@ var (
executorOnce sync.Once
)
// GetGlobalExecutor 获取全局队列执行器单例(默认 QPS=2, QPM=120, QPH=6000,保障 115 API 调用安全不超频)。
// GetGlobalExecutor 获取全局队列执行器单例(默认 QPS=3, QPM=200, QPH=12000,保障 115 API 调用安全不超频)。
//
// 历史教训:QPS 提到 8 后,下载换直链接口(/open/ufile/downurl,WAF 重点盯防对象)
// 瞬时突发撞上 115 风控,返回阿里云 405 阻断页(HTTP 405),导致全量同步失败。
// 因此回调到 3——这是经过实测的安全上限:宁慢勿触发风控,一旦 405 冷却 180 秒,
// 整体吞吐反而更低。下载实际走 CDN 不受此限速影响,瓶颈仅在换链环节。
func GetGlobalExecutor() *QueueExecutor {
executorOnce.Do(func() {
globalExecutor = NewQueueExecutor(2, 120, 6000)
globalExecutor = NewQueueExecutor(3, 200, 12000)
})
return globalExecutor
}
+49
View File
@@ -0,0 +1,49 @@
package cloud115
import (
"crypto/sha1"
"encoding/hex"
"io"
"os"
)
// FileSHA1 计算文件完整 SHA1(小写 hex)。
func FileSHA1(path string) (string, error) {
f, err := os.Open(path)
if err != nil {
return "", err
}
defer f.Close()
h := sha1.New()
if _, err := io.Copy(h, f); err != nil {
return "", err
}
return hex.EncodeToString(h.Sum(nil)), nil
}
// FileSHA1Partial 计算文件 [start,end](含)字节区间的 SHA1(小写 hex)。
// 用于 115 上传二次签名按 sign_check 指定的区间重算哈希。
func FileSHA1Partial(path string, start, end int64) (string, error) {
if start < 0 {
start = 0
}
if end < start {
end = start
}
f, err := os.Open(path)
if err != nil {
return "", err
}
defer f.Close()
if _, err := f.Seek(start, io.SeekStart); err != nil {
return "", err
}
length := end - start + 1
h := sha1.New()
// io.CopyN 在文件不足 length 字节时会返回 io.EOF,导致小文件(如小于 128 KiB 的
// 元数据图片)无法上传。这里只拷贝实际读到的字节,文件尾对齐到区间终点即可。
if _, err := io.CopyN(h, f, length); err != nil && err != io.EOF {
return "", err
}
return hex.EncodeToString(h.Sum(nil)), nil
}
+355
View File
@@ -0,0 +1,355 @@
// 115 网盘元数据上传能力:115 开放平台调度 + 阿里云 OSS 直传。
// 参考 QMediaSync 的上传流程实现:
//
// POST /open/upload/init 上传初始化/秒传调度(含二次签名)
// GET /open/upload/get_token 获取 OSS 临时上传凭证(STS)
// OSS multipart 分片直传 + callback 完成
//
// 上传目标父目录为 115 目录 ID(cid),而非路径字符串。
package cloud115
import (
"context"
"encoding/json"
"errors"
"fmt"
"os"
"strconv"
"strings"
)
// 115 上传状态码。
const (
UploadInitStatusNeedUpload = 1 // 需要真实上传
UploadInitStatusRapidUploaded = 2 // 秒传成功
UploadInitStatusSignFailed = 6 // 签名验证失败
UploadInitStatusNeedSign = 7 // 需要二次签名
UploadInitStatusSignRejected = 8 // 签名认证失败
)
// UploadInitRequest 是 /open/upload/init 的结构化请求。
type UploadInitRequest struct {
FileName string
FileSize int64
ParentFileId string
FileSha1 string
Preid string
PickCode string
TopUpload string
SignKey string
SignVal string
}
// UploadInitResult 是 /open/upload/init 的调度结果。
type UploadInitResult struct {
PickCode string
Status int
FileId string
Target string
Bucket string
Object string
SignKey string
SignCheck string
Callback UploadResultCallBack
}
type uploadScheduleAPIResult struct {
PickCode string `json:"pick_code"`
Status int `json:"status"`
FileId string `json:"file_id"`
Target string `json:"target"`
Version string `json:"version"`
Bucket string `json:"bucket"`
Object string `json:"object"`
SignKey string `json:"sign_key"`
SignCheck string `json:"sign_check"`
Callback json.RawMessage `json:"callback"`
}
// UploadResultCallBack 是 init 返回给 OSS complete 使用的 callback 内容。
type UploadResultCallBack struct {
Callback string `json:"callback"`
CallbackVar string `json:"callback_var"`
}
// UploadToken 是 /open/upload/get_token 返回的 OSS STS 临时凭证。
type UploadToken struct {
Endpoint string `json:"endpoint"`
AccessKeySecret string `json:"AccessKeySecret"`
AccessKeySecrett string `json:"AccessKeySecrett"`
SecurityToken string `json:"SecurityToken"`
Expiration string `json:"Expiration"`
AccessKeyId string `json:"AccessKeyId"`
}
func (token *UploadToken) normalize() {
if token == nil {
return
}
if token.AccessKeySecret == "" {
token.AccessKeySecret = token.AccessKeySecrett
}
}
// UploadCompleteResult 是 OSS complete callback 成功后的远端文件定位结果。
type UploadCompleteResult struct {
FileId string
PickCode string
ParentId string
Sha1 string
Size int64
Mtime int64
}
// SignCheckRange 是 115 二次认证要求的闭区间 [start,end]。
type SignCheckRange struct {
Start int64
End int64
}
// UploadInit 调用 115 上传初始化/秒传调度接口。
func (c *OpenClient) UploadInit(ctx context.Context, input UploadInitRequest) (*UploadInitResult, error) {
params := buildUploadInitForm(input)
resp, err := c.doAuthJSON(ctx, "POST", ProAPIBase+"/open/upload/init", params, 2)
if err != nil {
return nil, err
}
var raw uploadScheduleAPIResult
if err := json.Unmarshal(resp.Data, &raw); err != nil {
return nil, fmt.Errorf("115: 解析 upload/init 结果失败:%w", err)
}
callback, err := decodeUploadCallback(raw.Callback)
if err != nil {
return nil, err
}
return &UploadInitResult{
PickCode: raw.PickCode,
Status: raw.Status,
FileId: raw.FileId,
Target: raw.Target,
Bucket: raw.Bucket,
Object: raw.Object,
SignKey: raw.SignKey,
SignCheck: raw.SignCheck,
Callback: callback,
}, nil
}
func buildUploadInitForm(input UploadInitRequest) map[string]string {
topUpload := input.TopUpload
if topUpload == "" {
topUpload = "0"
}
params := map[string]string{
"file_name": input.FileName,
"file_size": strconv.FormatInt(input.FileSize, 10),
"target": fmt.Sprintf("U_1_%s", input.ParentFileId),
"fileid": input.FileSha1,
"preid": input.Preid,
"topupload": topUpload,
}
if input.PickCode != "" {
params["pick_code"] = input.PickCode
}
if input.SignKey != "" && input.SignVal != "" {
params["sign_key"] = input.SignKey
params["sign_val"] = input.SignVal
}
return params
}
func decodeUploadCallback(raw json.RawMessage) (UploadResultCallBack, error) {
if len(raw) == 0 || string(raw) == "null" {
return UploadResultCallBack{}, nil
}
if raw[0] == '[' {
var callbacks []UploadResultCallBack
if err := json.Unmarshal(raw, &callbacks); err != nil {
return UploadResultCallBack{}, err
}
if len(callbacks) == 0 {
return UploadResultCallBack{}, nil
}
return callbacks[0], nil
}
var callback UploadResultCallBack
if err := json.Unmarshal(raw, &callback); err != nil {
return UploadResultCallBack{}, err
}
return callback, nil
}
func parseSignCheckRange(value string) (SignCheckRange, error) {
parts := strings.Split(value, "-")
if len(parts) != 2 {
return SignCheckRange{}, fmt.Errorf("sign_check 格式错误:%s", value)
}
start, err := strconv.ParseInt(strings.TrimSpace(parts[0]), 10, 64)
if err != nil {
return SignCheckRange{}, err
}
end, err := strconv.ParseInt(strings.TrimSpace(parts[1]), 10, 64)
if err != nil {
return SignCheckRange{}, err
}
if start < 0 || end < start {
return SignCheckRange{}, fmt.Errorf("sign_check 范围非法:%s", value)
}
return SignCheckRange{Start: start, End: end}, nil
}
// GetUploadToken 获取 115 下发的 OSS 临时上传凭证。
func (c *OpenClient) GetUploadToken(ctx context.Context) (*UploadToken, error) {
resp, err := c.doAuthJSON(ctx, "GET", ProAPIBase+"/open/upload/get_token", nil, 2)
if err != nil {
return nil, err
}
var token UploadToken
if err := json.Unmarshal(resp.Data, &token); err != nil {
return nil, fmt.Errorf("115: 解析 get_token 结果失败:%w", err)
}
token.normalize()
return &token, nil
}
// Upload 上传单个本地文件到 115 指定父目录(cid),返回成功后的远端文件信息。
// filePath 必须是落到磁盘的真实文件路径(调用方负责把 io.Reader 落盘为临时文件)。
func (c *OpenClient) Upload(ctx context.Context, filePath, parentCID, signKey, signVal string) (*UploadCompleteResult, error) {
fileSize := fileSizeOf(filePath)
if fileSize < 0 {
return nil, fmt.Errorf("115: 无法获取文件大小:%s", filePath)
}
fileSha1, err := FileSHA1(filePath)
if err != nil {
return nil, fmt.Errorf("115: 计算文件 SHA1 失败:%w", err)
}
preSha1, err := FileSHA1Partial(filePath, 0, 128*1024-1)
if err != nil {
return nil, fmt.Errorf("115: 计算文件前 128 KiB SHA1 失败:%w", err)
}
request := UploadInitRequest{
FileName: baseNameOf(filePath),
FileSize: fileSize,
ParentFileId: parentCID,
FileSha1: fileSha1,
Preid: preSha1,
TopUpload: "0",
SignKey: signKey,
SignVal: signVal,
}
initResult, err := c.UploadInit(ctx, request)
if err != nil {
return nil, fmt.Errorf("115: 上传初始化失败:%w", err)
}
status := initResult.Status
if status == UploadInitStatusNeedSign {
// 二次签名:按 sign_check 指定区间重算 sha1
rng, err := parseSignCheckRange(initResult.SignCheck)
if err != nil {
return nil, err
}
signValue, err := FileSHA1Partial(filePath, rng.Start, rng.End)
if err != nil {
return nil, err
}
request.SignKey = initResult.SignKey
request.SignVal = signValue
initResult, err = c.UploadInit(ctx, request)
if err != nil {
return nil, fmt.Errorf("115: 上传二次签名失败:%w", err)
}
status = initResult.Status
}
switch status {
case UploadInitStatusRapidUploaded:
// 秒传成功
return &UploadCompleteResult{FileId: initResult.FileId, PickCode: initResult.PickCode}, nil
case UploadInitStatusSignFailed:
return nil, fmt.Errorf("115: 签名验证后失败")
case UploadInitStatusSignRejected:
return nil, fmt.Errorf("115: 签名认证失败")
case UploadInitStatusNeedUpload:
// 真实上传:OSS multipart
default:
return &UploadCompleteResult{FileId: initResult.FileId, PickCode: initResult.PickCode}, nil
}
if initResult.Bucket == "" || initResult.Object == "" {
return nil, fmt.Errorf("115: upload/init 缺少 bucket/object 信息")
}
token, err := c.GetUploadToken(ctx)
if err != nil {
return nil, fmt.Errorf("115: 获取上传凭证失败:%w", err)
}
if token == nil || token.Endpoint == "" || token.AccessKeyId == "" || token.AccessKeySecret == "" {
return nil, fmt.Errorf("115: 上传凭证不完整")
}
uploader := NewOSSMultipartUploader(token.Endpoint, token.AccessKeyId, token.AccessKeySecret, token.SecurityToken)
result, err := uploader.UploadFile(ctx, OSSMultipartUploadInput{
Bucket: initResult.Bucket,
Object: initResult.Object,
Callback: initResult.Callback.Callback,
CallbackVar: initResult.Callback.CallbackVar,
FilePath: filePath,
FileSize: fileSize,
refreshClient: func(ctx context.Context) (ossMultipartClient, error) {
refreshed, rerr := c.GetUploadToken(ctx)
if rerr != nil || refreshed == nil {
return nil, rerr
}
return newOSSMultipartClient(refreshed.Endpoint, refreshed.AccessKeyId, refreshed.AccessKeySecret, refreshed.SecurityToken), nil
},
})
if err != nil {
return nil, fmt.Errorf("115: OSS 上传失败:%w", err)
}
complete, err := ParseCompleteCallbackResult(result)
if err != nil {
return nil, err
}
return &complete, nil
}
// MkDir 在 115 的 parentCid 下创建目录,返回新目录 cid。
func (c *OpenClient) MkDir(ctx context.Context, parentCID, name string) (string, error) {
params := map[string]string{
"cname": name,
"pid": parentCID,
}
resp, err := c.doAuthJSON(ctx, "POST", ProAPIBase+"/open/folder/add", params, 2)
if err != nil {
return "", err
}
// /open/folder/add 结构:{ aid, cid, fid, name, pid, ... },单一对象
var r struct {
Cid string `json:"cid"`
}
if err := json.Unmarshal(resp.Data, &r); err != nil {
return "", fmt.Errorf("115: 解析 folder/add 结果失败:%w", err)
}
if r.Cid == "" {
return "", errors.New("115: folder/add 未返回 cid")
}
return r.Cid, nil
}
func fileSizeOf(path string) int64 {
info, err := os.Stat(path)
if err != nil {
return -1
}
if info.IsDir() {
return -1
}
return info.Size()
}
func baseNameOf(path string) string {
s := path
for i := len(s) - 1; i >= 0; i-- {
if s[i] == '/' || s[i] == '\\' {
return s[i+1:]
}
}
return s
}
+197
View File
@@ -0,0 +1,197 @@
package cloud115
import (
"context"
"encoding/base64"
"io"
"os"
"path/filepath"
"testing"
"github.com/aliyun/alibabacloud-oss-go-sdk-v2/oss"
)
func TestFileSHA1(t *testing.T) {
dir := t.TempDir()
path := filepath.Join(dir, "a.txt")
if err := os.WriteFile(path, []byte("hello"), 0o644); err != nil {
t.Fatal(err)
}
sum, err := FileSHA1(path)
if err != nil {
t.Fatal(err)
}
// sha1("hello") = aaf4c61ddcc5e8a2dabede0f3b482cd9aea9434d
if sum != "aaf4c61ddcc5e8a2dabede0f3b482cd9aea9434d" {
t.Errorf("unexpected sha1: %s", sum)
}
}
func TestFileSHA1Partial(t *testing.T) {
dir := t.TempDir()
path := filepath.Join(dir, "b.txt")
// 10 bytes: "0123456789"
if err := os.WriteFile(path, []byte("0123456789"), 0o644); err != nil {
t.Fatal(err)
}
// bytes [2,4] = "234"
sum, err := FileSHA1Partial(path, 2, 4)
if err != nil {
t.Fatal(err)
}
if sum != "0ec09ef9836da03f1add21e3ef607627e687e790" {
t.Errorf("unexpected partial sha1: %s", sum)
}
}
// TestFileSHA1PartialSmallerThanWindow 回归测试:经典 bug 是 io.CopyN 在文件不足
// length 字节时返回 io.EOF。115 上传固定用 [0,128*1024-1] 窗口计算 preid,导致所有
// 小于 128 KiB 的元数据文件(如海报/缩略图)上传必然失败。
func TestFileSHA1PartialSmallerThanWindow(t *testing.T) {
dir := t.TempDir()
path := filepath.Join(dir, "small.bin")
// 6 字节小文件,不足 128 KiB 窗口
if err := os.WriteFile(path, []byte("abcdef"), 0o644); err != nil {
t.Fatal(err)
}
sum, err := FileSHA1Partial(path, 0, 128*1024-1)
if err != nil {
t.Fatalf("compute partial sha1 for small file should not fail: %v", err)
}
// 应等于整个文件(6 字节)的 sha1
if sum != "1f8ac10f23c5b5bc1167bda84b833e5c057a77d2" {
t.Errorf("unexpected partial sha1: %s", sum)
}
}
func TestParseSignCheckRange(t *testing.T) {
rng, err := parseSignCheckRange("0-131071")
if err != nil {
t.Fatal(err)
}
if rng.Start != 0 || rng.End != 131071 {
t.Errorf("unexpected range: %+v", rng)
}
if _, err := parseSignCheckRange("bad"); err == nil {
t.Error("expected error for bad range")
}
if _, err := parseSignCheckRange("100-50"); err == nil {
t.Error("expected error for end<start")
}
}
func TestCalculateMultipartPartSize(t *testing.T) {
// small file: 1 MiB -> partSize 32MiB, 1 part
ps, parts, err := CalculateMultipartPartSize(1 << 20)
if err != nil {
t.Fatal(err)
}
if ps != defaultMultipartPartSize {
t.Errorf("partSize=%d, want %d", ps, defaultMultipartPartSize)
}
if parts != 1 {
t.Errorf("parts=%d, want 1", parts)
}
// zero-size -> 1 part
_, parts, err = CalculateMultipartPartSize(0)
if err != nil {
t.Fatal(err)
}
if parts != 1 {
t.Errorf("zero-size parts=%d, want 1", parts)
}
// negative -> error
if _, _, err := CalculateMultipartPartSize(-1); err == nil {
t.Error("expected error for negative size")
}
}
func TestBaseNameOf(t *testing.T) {
if got := baseNameOf("/a/b/file.nfo"); got != "file.nfo" {
t.Errorf("got %s", got)
}
if got := baseNameOf("a\\b\\c.jpg"); got != "c.jpg" {
t.Errorf("got %s", got)
}
if got := baseNameOf("top.txt"); got != "top.txt" {
t.Errorf("got %s", got)
}
}
// fakeCallbackOSSClient 捕获 CompleteMultipartUpload 收到的 callback / callback_var,
// 用于断言已经 Base64 编码(116 要求 callback 必须是 Base64 后的 JSON,否则报
// "The callback configuration is not base64 encoded")。
type fakeCallbackOSSClient struct {
capturedCallback string
capturedCallbackVar string
}
func (c *fakeCallbackOSSClient) InitiateMultipartUpload(_ context.Context, _ *oss.InitiateMultipartUploadRequest, _ ...func(*oss.Options)) (*oss.InitiateMultipartUploadResult, error) {
return &oss.InitiateMultipartUploadResult{UploadId: oss.Ptr("upload-new")}, nil
}
func (c *fakeCallbackOSSClient) UploadPart(_ context.Context, r *oss.UploadPartRequest, _ ...func(*oss.Options)) (*oss.UploadPartResult, error) {
if r.Body != nil {
_, _ = io.Copy(io.Discard, r.Body)
}
return &oss.UploadPartResult{ETag: oss.Ptr("etag-1")}, nil
}
func (c *fakeCallbackOSSClient) ListParts(context.Context, *oss.ListPartsRequest, ...func(*oss.Options)) (*oss.ListPartsResult, error) {
return &oss.ListPartsResult{}, nil
}
func (c *fakeCallbackOSSClient) CompleteMultipartUpload(_ context.Context, r *oss.CompleteMultipartUploadRequest, _ ...func(*oss.Options)) (*oss.CompleteMultipartUploadResult, error) {
c.capturedCallback = *r.Callback
c.capturedCallbackVar = *r.CallbackVar
return &oss.CompleteMultipartUploadResult{
CallbackResult: map[string]any{
"state": true,
"data": map[string]any{"file_id": "file-1", "pick_code": "pick-1"},
},
}, nil
}
func (c *fakeCallbackOSSClient) AbortMultipartUpload(context.Context, *oss.AbortMultipartUploadRequest, ...func(*oss.Options)) (*oss.AbortMultipartUploadResult, error) {
return &oss.AbortMultipartUploadResult{}, nil
}
// TestCompleteMultipartUploadCallbackBase64 回归测试:OSS CompleteMultipartUpload 的
// callback 必须 Base64 编码,否则报 "The callback configuration is not base64 encoded",
// 导致大于 128 KiB 的元数据文件上传失败。
func TestCompleteMultipartUploadCallbackBase64(t *testing.T) {
dir := t.TempDir()
path := filepath.Join(dir, "big.bin")
data := make([]byte, 8) // 8 字节,PartSize=8 → 1 part
if err := os.WriteFile(path, data, 0o644); err != nil {
t.Fatal(err)
}
fake := &fakeCallbackOSSClient{}
uploader := &OSSMultipartUploader{client: fake}
callback := `{"callbackUrl":"http://uplb.115.com/3.0/completeupload.php"}`
callbackVar := `{"x:pick_code":"abc"}`
_, err := uploader.UploadFileWithResult(context.Background(), OSSMultipartUploadInput{
Bucket: "bucket-1",
Object: "object-1",
Callback: callback,
CallbackVar: callbackVar,
FilePath: path,
FileSize: int64(len(data)),
PartSize: 8,
})
if err != nil {
t.Fatalf("multipart 上传失败:%v", err)
}
// 捕获的 callback 必须是合法 Base64,且解码后与原 JSON 一致
cbBytes, err := base64.StdEncoding.DecodeString(fake.capturedCallback)
if err != nil {
t.Fatalf("callback 未 Base64 编码:%v (raw=%q)", err, fake.capturedCallback)
}
if string(cbBytes) != callback {
t.Errorf("callback 解码后 = %s,期望 %s", cbBytes, callback)
}
cbvBytes, err := base64.StdEncoding.DecodeString(fake.capturedCallbackVar)
if err != nil {
t.Fatalf("callback_var 未 Base64 编码:%v (raw=%q)", err, fake.capturedCallbackVar)
}
if string(cbvBytes) != callbackVar {
t.Errorf("callback_var 解码后 = %s,期望 %s", cbvBytes, callbackVar)
}
}
+90
View File
@@ -0,0 +1,90 @@
package service
import (
"context"
"fmt"
"strings"
"go.uber.org/zap"
"gorm.io/gorm"
"github.com/ShukeBta/MMTL/internal/config"
"github.com/ShukeBta/MMTL/internal/database"
"github.com/ShukeBta/MMTL/internal/repository"
)
// DatabaseAdminService manages database configuration, connectivity testing, and migration.
type DatabaseAdminService struct {
cfg *config.Config
log *zap.Logger
repos *repository.Container
db *gorm.DB
}
// NewDatabaseAdminService creates a new DatabaseAdminService.
func NewDatabaseAdminService(cfg *config.Config, log *zap.Logger, repos *repository.Container, db *gorm.DB) *DatabaseAdminService {
if log == nil {
log = zap.NewNop()
}
return &DatabaseAdminService{
cfg: cfg,
log: log,
repos: repos,
db: db,
}
}
// GetStatus returns the status of the currently active database.
func (s *DatabaseAdminService) GetStatus(ctx context.Context) *database.DatabaseStatus {
return database.InspectDatabaseStatus(s.db, s.cfg)
}
// TestPostgres verifies connectivity and permissions to the specified PostgreSQL DSN.
func (s *DatabaseAdminService) TestPostgres(ctx context.Context, dsn string) (*database.PostgresTestResult, error) {
return database.TestPostgres(dsn)
}
// MigrateToPostgres copies all records from the current active database to the target PostgreSQL database.
func (s *DatabaseAdminService) MigrateToPostgres(ctx context.Context, targetDSN string) (*database.DatabaseMigrationResult, error) {
s.log.Info("starting user-initiated database migration to PostgreSQL", zap.String("target", database.MaskDSN(targetDSN)))
res, err := database.MigrateCurrentToPostgres(s.db, targetDSN, 500, s.log)
if err != nil {
s.log.Error("database migration to PostgreSQL failed", zap.Error(err))
return nil, err
}
s.log.Info("database migration to PostgreSQL completed successfully",
zap.Int64("total_rows", res.TotalRows),
zap.Int64("duration_ms", res.DurationMS),
)
return res, nil
}
// SaveConfig persists the database configuration to config.yaml and the database settings table.
func (s *DatabaseAdminService) SaveConfig(ctx context.Context, dbType, dsn string) error {
dbType = strings.TrimSpace(dbType)
dsn = strings.TrimSpace(dsn)
if dbType == "" {
dbType = "postgres"
}
if dbType == "postgres" && dsn == "" {
return fmt.Errorf("PostgreSQL DSN 不能为空")
}
// 1. 保存到本地 config.yaml
if err := config.SaveDatabaseConfig(dbType, dsn); err != nil {
return fmt.Errorf("保存配置文件失败: %w", err)
}
// 2. 更新内存配置
s.cfg.Database.Type = dbType
s.cfg.Database.DSN = dsn
// 3. 同时更新 settings 存储库作为副本
if s.repos != nil && s.repos.Setting != nil {
_ = s.repos.Setting.Set(ctx, "database.type", dbType)
_ = s.repos.Setting.Set(ctx, "database.dsn", dsn)
}
s.log.Info("database configuration saved", zap.String("type", dbType), zap.String("dsn", database.MaskDSN(dsn)))
return nil
}
+9
View File
@@ -11,6 +11,15 @@ import (
// ImageURL returns artwork for a media/series/season item id.
func (e *EmbyService) ImageURL(ctx context.Context, id, imageType string) (string, error) {
// 远程 Emby 条目:直接返回远程图片绝对地址,由 ImageProxy 拉取透传。
if e.remote != nil && IsEmbyRemoteID(id) {
mountID, remoteID, _ := DecodeEmbyRemoteID(id)
mount, acct, _ := e.remote.ResolveMount(ctx, mountID)
if mount == nil || acct == nil {
return "", nil
}
return e.remote.RemoteImageURL(ctx, acct, remoteID, imageType)
}
pick := func(primary, backdrop string) string {
switch strings.ToLower(imageType) {
case "backdrop", "art":
+109 -1
View File
@@ -48,6 +48,7 @@ type EmbyService struct {
repo *repository.Container
cache *RuntimeCacheService
subtitle *SubtitleService
remote *EmbyRemoteService // 远程 Emby 联邦聚合(可为 nil:未启用)
virtualMu sync.RWMutex
virtualSeries map[string]embySeriesCacheEntry
@@ -66,6 +67,14 @@ func NewEmbyService(cfg *config.Config, log *zap.Logger, repo *repository.Contai
return &EmbyService{cfg: cfg, log: log, repo: repo}
}
// SetEmbyRemote 注入远程 Emby 联邦聚合服务(nil 表示未启用)。
func (e *EmbyService) SetEmbyRemote(remote *EmbyRemoteService) *EmbyService {
if e != nil {
e.remote = remote
}
return e
}
func (e *EmbyService) SetRuntimeCache(cache *RuntimeCacheService) *EmbyService {
if e != nil {
e.cache = cache
@@ -123,7 +132,8 @@ type embyVisibilityCacheEntry struct {
// Items paginates media in Emby's hierarchy. Episodic libraries are exposed as
// Series -> Season -> Episode so Infuse/Vidhub/SenPlayer stop treating every
// episode as a separate movie card.
// episode as a separate movie card. 带 embyremote~ 前缀的 ParentID / 搜索自动
// 路由到远程 Emby(联邦聚合,远程数据不落库)。
func (e *EmbyService) Items(ctx context.Context, p ItemsParams) (map[string]any, error) {
if p.Limit <= 0 || p.Limit > 500 {
p.Limit = 50
@@ -135,6 +145,22 @@ func (e *EmbyService) Items(ctx context.Context, p ItemsParams) (map[string]any,
return emptyItemsEnvelope(p.StartIndex), nil
}
if e.remote != nil {
// 远程目录浏览:ParentId 带远程前缀 → 完整转发给远程 Emby 承接分页。
if IsEmbyRemoteID(p.ParentID) {
mountID, _, _ := DecodeEmbyRemoteID(p.ParentID)
mount, acct, _ := e.remote.ResolveMount(ctx, mountID)
if mount == nil || acct == nil {
return emptyItemsEnvelope(p.StartIndex), nil
}
return e.remote.RemoteItems(ctx, mount, acct, p)
}
// 全局搜索:无 ParentId 且带搜索词 → 聚合本地 + 全部远程。
if p.ParentID == "" && p.SearchTerm != "" {
return e.aggregatedSearch(ctx, p)
}
}
if len(p.IDs) > 0 {
items := make([]map[string]any, 0, len(p.IDs))
for _, id := range p.IDs {
@@ -212,3 +238,85 @@ func (e *EmbyService) Items(ctx context.Context, p ItemsParams) (map[string]any,
}
return e.mediaItems(ctx, p)
}
// aggregatedSearch 把本地媒体库与全部启用的远程 Emby 的搜索结果合并为一个
// 分页载荷。本地结果保持原有分页语义,远程各自取一页(Limit 同款)后按
// SortBy 做稳定排序切片。
func (e *EmbyService) aggregatedSearch(ctx context.Context, p ItemsParams) (map[string]any, error) {
local, err := e.mediaItems(ctx, p)
if err != nil {
return nil, err
}
type remoteResult struct {
items []any
}
mounts, aerr := e.remote.ListMounts(ctx)
results := make([]remoteResult, 0, len(mounts))
if aerr == nil {
for i := range mounts {
m := mounts[i]
if !m.Enabled {
continue
}
acct := e.remote.AccountByID(ctx, m.AccountID)
if acct == nil {
continue
}
// 按挂载逐个搜索:搜索结果归属明确(伪装 ID 正确),也天然只搜已
// 挂载的媒体库。
searchParams := p
searchParams.ParentID = "" // RemoteSearchMount 内部设 ParentId
remote, rerr := e.remote.RemoteSearchMount(ctx, &m, acct, p)
if rerr != nil {
if e.log != nil {
e.log.Warn("remote emby search failed",
zap.String("account", acct.Name), zap.Error(rerr))
}
continue
}
if raw, ok := remote["Items"].([]any); ok {
results = append(results, remoteResult{items: raw})
} else if rawMap, ok := remote["Items"].([]map[string]any); ok {
converted := make([]any, 0, len(rawMap))
for _, m := range rawMap {
converted = append(converted, any(m))
}
results = append(results, remoteResult{items: converted})
}
}
}
items := make([]any, 0, len(localItemsAsAny(local))+len(results)*p.Limit)
items = append(items, localItemsAsAny(local)...)
for _, res := range results {
items = append(items, res.items...)
}
return sliceSearchItems(items, p), nil
}
func localItemsAsAny(envelope map[string]any) []any {
if envelope == nil {
return nil
}
if raw, ok := envelope["Items"].([]any); ok {
return raw
}
if raw, ok := envelope["Items"].([]map[string]any); ok {
converted := make([]any, 0, len(raw))
for _, m := range raw {
converted = append(converted, any(m))
}
return converted
}
return nil
}
// sliceSearchItems 对合并结果按请求排序做简单归类后分页。远程返回已按远程
// 排序规则排好,这里保持稳定顺序,只做首/尾切片,避免过度重排造成分页跳动。
func sliceSearchItems(items []any, p ItemsParams) map[string]any {
total := len(items)
if p.StartIndex >= total {
return map[string]any{"Items": []any{}, "TotalRecordCount": total, "StartIndex": p.StartIndex}
}
end := minInt(p.StartIndex+p.Limit, total)
return map[string]any{"Items": items[p.StartIndex:end], "TotalRecordCount": total, "StartIndex": p.StartIndex}
}
+18 -1
View File
@@ -11,6 +11,15 @@ import (
// Item 单条目详情。
func (e *EmbyService) Item(ctx context.Context, mediaID, userID string) (map[string]any, error) {
// 远程 Emby 条目:不查本地库,直接向远程转发(保持远程最新元数据)。
if e.remote != nil && IsEmbyRemoteID(mediaID) {
mountID, remoteID, _ := DecodeEmbyRemoteID(mediaID)
mount, acct, _ := e.remote.ResolveMount(ctx, mountID)
if mount == nil || acct == nil {
return nil, nil
}
return e.remote.RemoteItem(ctx, mount, acct, remoteID)
}
if lib, err := e.repo.Library.FindByID(ctx, mediaID); err != nil {
return nil, err
} else if lib != nil {
@@ -71,11 +80,19 @@ func (e *EmbyService) Item(ctx context.Context, mediaID, userID string) (map[str
return e.itemPayload(ctx, m, fav, pos), nil
}
// LatestItems 最近添加,全库或指定库。
// LatestItems 最近添加,全库或指定库。远程媒体库(parentID 带前缀)直接透传远程。
func (e *EmbyService) LatestItems(ctx context.Context, userID, parentID string, limit int) ([]map[string]any, error) {
if limit <= 0 || limit > 100 {
limit = 20
}
if e.remote != nil && IsEmbyRemoteID(parentID) {
mountID, remoteParent, _ := DecodeEmbyRemoteID(parentID)
mount, acct, _ := e.remote.ResolveMount(ctx, mountID)
if mount == nil || acct == nil {
return nil, nil
}
return e.remote.RemoteLatest(ctx, mount, acct, remoteParent, limit)
}
cacheKey := e.embyLatestCacheKey(userID, parentID, limit)
var cached embyLatestCacheValue
if e.cache != nil && e.cache.GetJSON(ctx, cacheKey, &cached) {
+3 -1
View File
@@ -199,7 +199,9 @@ func (e *EmbyService) appendSubtitleStreams(ctx context.Context, streams []map[s
if e == nil || e.subtitle == nil || m == nil {
return streams
}
tracks, err := e.subtitle.Discover(ctx, m.ID)
// Emby 字幕只列表外挂字幕文件:云盘/strm 媒体的容器内嵌字幕不做服务端
// 提取,客户端直连播放直链时自行解析。
tracks, err := e.subtitle.DiscoverExternalOnly(ctx, m.ID)
if err != nil || len(tracks) == 0 {
return streams
}
+88 -2
View File
@@ -4,6 +4,7 @@ import (
"context"
"fmt"
"io"
"net/http"
"net/url"
"path/filepath"
"strconv"
@@ -14,7 +15,25 @@ import (
)
// PlaybackInfo returns a PlaybackInfoResponse usable by Emby clients.
// 远程 Emby 条目直接转发远程 PlaybackInfo,并按账号 proxy_play 配置决定
// 播放地址指向远程(直连)还是 MMTL 本地代理端点。
func (e *EmbyService) PlaybackInfo(ctx context.Context, mediaID, userID string) (map[string]any, error) {
if e.remote != nil && IsEmbyRemoteID(mediaID) {
mountID, remoteID, _ := DecodeEmbyRemoteID(mediaID)
mount, acct, err := e.remote.ResolveMount(ctx, mountID)
if err != nil {
return nil, ErrEmbyRemoteNotFound
}
out, err := e.remote.RemotePlaybackInfo(ctx, mount, acct, remoteID, userID)
if err != nil {
return nil, err
}
if out == nil {
return nil, ErrEmbyRemoteNotFound
}
out["PlaySessionId"] = fmt.Sprintf("remote-%s-%d", mountID, time.Now().Unix())
return out, nil
}
m, err := e.playableMedia(ctx, mediaID, userID)
if err != nil || m == nil {
return nil, err
@@ -25,6 +44,65 @@ func (e *EmbyService) PlaybackInfo(ctx context.Context, mediaID, userID string)
}, nil
}
// ErrEmbyRemoteNotFound 表示伪装 ID 对应的远程挂载账号不存在/已禁用。
var ErrEmbyRemoteNotFound = fmt.Errorf("remote emby account not found")
// RemoteAccountByID 供 handler 层解码伪装 ID 后获取远程账号。
func (e *EmbyService) RemoteAccountByID(ctx context.Context, accountID string) *model.StrmAccount {
if e == nil || e.remote == nil {
return nil
}
return e.remote.AccountByID(ctx, accountID)
}
// ProxyRemoteVideoStream 反向代理远程 Emby 视频流(保留 Range)。
func (e *EmbyService) ProxyRemoteVideoStream(ctx context.Context, w http.ResponseWriter, r *http.Request, mountID, remoteID string) error {
if e == nil || e.remote == nil {
return ErrEmbyRemoteNotFound
}
_, acct, err := e.remote.ResolveMount(ctx, mountID)
if err != nil {
return ErrEmbyRemoteNotFound
}
return e.remote.ProxyVideoStream(ctx, w, r, acct, remoteID)
}
// ProxyRemoteSubtitle 反向代理远程 Emby 字幕流。
func (e *EmbyService) ProxyRemoteSubtitle(ctx context.Context, w http.ResponseWriter, r *http.Request, mountID, remoteID, index string) error {
if e == nil || e.remote == nil {
return ErrSubtitleNotFound
}
_, acct, err := e.remote.ResolveMount(ctx, mountID)
if err != nil {
return ErrSubtitleNotFound
}
return e.remote.ProxySubtitle(ctx, w, r, acct, remoteID, index)
}
// ProxyRemoteSetPlayed 把已看/未看状态透传到远程 Emby。
func (e *EmbyService) ProxyRemoteSetPlayed(ctx context.Context, mountID, remoteID string, played bool) error {
if e == nil || e.remote == nil {
return ErrEmbyRemoteNotFound
}
_, acct, err := e.remote.ResolveMount(ctx, mountID)
if err != nil {
return ErrEmbyRemoteNotFound
}
return e.remote.ProxySetPlayed(ctx, acct, remoteID, played)
}
// ProxyRemoteSetFavorite 把收藏/取消收藏状态透传到远程 Emby。
func (e *EmbyService) ProxyRemoteSetFavorite(ctx context.Context, mountID, remoteID string, favorite bool) error {
if e == nil || e.remote == nil {
return ErrEmbyRemoteNotFound
}
_, acct, err := e.remote.ResolveMount(ctx, mountID)
if err != nil {
return ErrEmbyRemoteNotFound
}
return e.remote.ProxySetFavorite(ctx, acct, remoteID, favorite)
}
// ServeSubtitleStream resolves the Emby /Videos/:id/Subtitles/:index/Stream
// request to one of the media's sideloaded external subtitle tracks and writes
// the original (unconverted) subtitle bytes to w — matching the source Codec
@@ -32,10 +110,18 @@ func (e *EmbyService) PlaybackInfo(ctx context.Context, mediaID, userID string)
// so the DeliveryUrl advertised in MediaStreams lines up exactly with the
// served track: subtitles start at 1 when no audio stream is present, otherwise
// at 2 (after Video 0 + Audio 1).
//
// 只服务外挂字幕文件(DiscoverExternalOnly):云盘/strm 媒体的容器内嵌字幕
// 不做服务端提取,客户端直连播放直链时自行解析。
func (e *EmbyService) ServeSubtitleStream(ctx context.Context, w io.Writer, mediaID, indexStr string, userID string) error {
if e == nil || e.subtitle == nil {
return ErrSubtitleUnavailable
}
if e.remote != nil && IsEmbyRemoteID(mediaID) {
// 远程字幕由反向代理透传(需要 http.ResponseWriter 能力),handler 层
// 已对远程 ID 走 ProxyRemoteSubtitle,这里不重复处理。
return ErrSubtitleNotFound
}
m, err := e.playableMedia(ctx, mediaID, userID)
if err != nil || m == nil {
return ErrSubtitleNotFound
@@ -44,7 +130,7 @@ func (e *EmbyService) ServeSubtitleStream(ctx context.Context, w io.Writer, medi
if err != nil || index < 1 {
return ErrSubtitleNotFound
}
tracks, err := e.subtitle.Discover(ctx, m.ID)
tracks, err := e.subtitle.DiscoverExternalOnly(ctx, m.ID)
if err != nil || len(tracks) == 0 {
return ErrSubtitleNotFound
}
@@ -74,7 +160,7 @@ func (e *EmbyService) SubtitleStreamCodec(ctx context.Context, mediaID, indexStr
if err != nil || index < 1 {
return ""
}
tracks, err := e.subtitle.Discover(ctx, m.ID)
tracks, err := e.subtitle.DiscoverExternalOnly(ctx, m.ID)
if err != nil || len(tracks) == 0 {
return ""
}
+909
View File
@@ -0,0 +1,909 @@
// EmbyRemoteService 是「远程 Emby 联邦聚合」核心:把挂载的远程 Emby 服务器
// 作为外部媒体源,通过 MMTL 的 Emby 兼容 API 透出。
//
// 设计要点:
// - 远程媒体的元数据完全不落库:每次请求实时向远程 Emby 拉取;
// - 条目 ID 用 embyremote~{accountID}~{remoteID} 伪装(见 emby_remote_ids.go),
// 客户端拿伪装 ID 回来时按账号路由回远程;
// - 播放分流由账号级 proxy_play 配置决定:
// 不代理(默认)= MediaSource 下发热门远程绝对 URL,播放字节完全不经过 MMTL;
// 代理 = 下发 MMTL 本地 /Videos/{encoded} 端点,由 ProxyVideoStream 反向拉流。
//
// 配置复用 STRM 账号体系(StrmAccount.Provider = emby_remote),CRUD/加密/连通
// 测试全部走既有 /admin/strm/accounts 接口,不需要新增数据表。
package service
import (
"context"
"crypto/sha1"
"encoding/hex"
"encoding/json"
"errors"
"fmt"
"io"
"net/http"
"net/url"
"regexp"
"strconv"
"strings"
"time"
"go.uber.org/zap"
"github.com/ShukeBta/MMTL/internal/config"
"github.com/ShukeBta/MMTL/internal/model"
"github.com/ShukeBta/MMTL/internal/repository"
)
// embyRemoteHTTPTimeout 远程 Emby 常规 API 请求超时(流式代理不在此列)。
const embyRemoteHTTPTimeout = 15 * time.Second
// embyRemoteUA 桌面浏览器 UA:远程 Emby 前方若有 Cloudflare/WAF 会拦截
// Go-http-client 等非浏览器 UA(403 error code: 1010),必须伪装浏览器。
const embyRemoteUA = "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/124.0 Safari/537.36"
// embyRemoteTransport 统一给远程请求注入浏览器 UA。
type embyRemoteTransport struct {
base http.RoundTripper
}
func (t *embyRemoteTransport) RoundTrip(req *http.Request) (*http.Response, error) {
if strings.TrimSpace(req.Header.Get("User-Agent")) == "" {
req.Header.Set("User-Agent", embyRemoteUA)
}
return t.base.RoundTrip(req)
}
// EmbyRemoteConfig 是一个远程 Emby 账号的解密配置。
type EmbyRemoteConfig struct {
BaseURL string // http://host:8096(无需 /emby 后缀)
Username string
Password string
Token string // api_key(手动填写或自动认证获得)
RemoteUserID string // 远程用户 Id(自动认证后回填)
ProxyPlay bool // true=播放流量经 MMTL 反向代理;false=客户端直连远程
}
// EmbyRemoteService 提供对远程 Emby 服务器的读写封装。
type EmbyRemoteService struct {
cfg *config.Config
log *zap.Logger
repo *repository.Container
crypto *CryptoService
http *http.Client
cache *RuntimeCacheService
}
// NewEmbyRemoteService 构造远程 Emby 聚合服务。
func NewEmbyRemoteService(cfg *config.Config, log *zap.Logger, repo *repository.Container, crypto *CryptoService) *EmbyRemoteService {
return &EmbyRemoteService{
cfg: cfg,
log: log,
repo: repo,
crypto: crypto,
http: &http.Client{
Timeout: embyRemoteHTTPTimeout,
Transport: &embyRemoteTransport{base: http.DefaultTransport},
},
}
}
func (r *EmbyRemoteService) SetRuntimeCache(cache *RuntimeCacheService) *EmbyRemoteService {
if r != nil {
r.cache = cache
}
return r
}
func (r *EmbyRemoteService) remoteMediaCacheTTL() time.Duration {
seconds := 15
if r != nil && r.cfg != nil && r.cfg.Cache.MediaTTLSeconds > 0 {
seconds = r.cfg.Cache.MediaTTLSeconds
}
return time.Duration(seconds) * time.Second
}
func (r *EmbyRemoteService) remoteCacheKey(parts ...string) string {
sum := sha1.Sum([]byte(strings.Join(parts, "|")))
return "media:embyremote:" + hex.EncodeToString(sum[:])
}
func (r *EmbyRemoteService) invalidateRemoteMediaCache(ctx context.Context) {
if r != nil && r.cache != nil {
r.cache.DeletePrefix(ctx, "media:embyremote:")
}
}
// ListAccounts 返回全部启用的远程 Emby 挂载账号。
func (r *EmbyRemoteService) ListAccounts(ctx context.Context) ([]model.StrmAccount, error) {
accounts, err := r.repo.StrmAccount.List(ctx)
if err != nil {
return nil, err
}
out := make([]model.StrmAccount, 0, len(accounts))
for i := range accounts {
if accounts[i].Provider == model.StrmProviderEmbyRemote {
out = append(out, accounts[i])
}
}
return out, nil
}
// AccountByID 按 ID 查找远程 Emby 挂载账号(不存在或类型不符返回 nil)。
func (r *EmbyRemoteService) AccountByID(ctx context.Context, id string) *model.StrmAccount {
if strings.TrimSpace(id) == "" {
return nil
}
acct, err := r.repo.StrmAccount.FindByID(ctx, id)
if err != nil || acct == nil {
return nil
}
if acct.Provider != model.StrmProviderEmbyRemote || !acct.Enabled {
return nil
}
return acct
}
// ─── 媒体库挂载管理 ─────────────────────────────────────────────────────────────
// ListMounts 返回全部挂载。
func (r *EmbyRemoteService) ListMounts(ctx context.Context) ([]model.EmbyMount, error) {
return r.repo.EmbyMount.List(ctx)
}
// ListMountsByAccount 返回指定账号的挂载。
func (r *EmbyRemoteService) ListMountsByAccount(ctx context.Context, accountID string) ([]model.EmbyMount, error) {
return r.repo.EmbyMount.ListByAccountID(ctx, accountID)
}
// MountByID 按 ID 查挂载。
func (r *EmbyRemoteService) MountByID(ctx context.Context, id string) (*model.EmbyMount, error) {
return r.repo.EmbyMount.FindByID(ctx, id)
}
// CreateMount 创建一个挂载(校验账号类型与远程 View 编号)。
func (r *EmbyRemoteService) CreateMount(ctx context.Context, m *model.EmbyMount) (*model.EmbyMount, error) {
if strings.TrimSpace(m.AccountID) == "" || strings.TrimSpace(m.RemoteViewID) == "" {
return nil, errors.New("缺少账号或远程媒体库")
}
if r.AccountByID(ctx, m.AccountID) == nil {
return nil, errors.New("远程 Emby 账号不存在或已禁用")
}
if err := r.repo.EmbyMount.Create(ctx, m); err != nil {
return nil, err
}
r.invalidateRemoteMediaCache(ctx)
return m, nil
}
// CreateMounts 批量创建挂载(幂等:已存在的远程库自动跳过)。
func (r *EmbyRemoteService) CreateMounts(ctx context.Context, mounts []*model.EmbyMount) (int, error) {
if len(mounts) == 0 {
return 0, nil
}
existing, err := r.repo.EmbyMount.ListByAccountID(ctx, mounts[0].AccountID)
if err != nil {
return 0, err
}
have := make(map[string]bool, len(existing))
for _, e := range existing {
have[e.RemoteViewID] = true
}
fresh := make([]*model.EmbyMount, 0, len(mounts))
for _, m := range mounts {
if m == nil || have[m.RemoteViewID] {
continue
}
fresh = append(fresh, m)
}
if len(fresh) == 0 {
return 0, nil
}
if err := r.repo.EmbyMount.CreateInBatches(ctx, fresh, 50); err != nil {
return 0, err
}
r.invalidateRemoteMediaCache(ctx)
return len(fresh), nil
}
// UpdateMount 更新挂载(名称 / 代理 / 启用)。
func (r *EmbyRemoteService) UpdateMount(ctx context.Context, id string, m *model.EmbyMount) (*model.EmbyMount, error) {
existing, err := r.repo.EmbyMount.FindByID(ctx, id)
if err != nil || existing == nil {
return nil, errNotFoundOr(err, "挂载不存在")
}
existing.Name = strings.TrimSpace(m.Name)
existing.ProxyPlay = m.ProxyPlay
existing.Enabled = m.Enabled
if err := r.repo.EmbyMount.Update(ctx, existing); err != nil {
return nil, err
}
r.invalidateRemoteMediaCache(ctx)
return existing, nil
}
// DeleteMount 删除挂载。
func (r *EmbyRemoteService) DeleteMount(ctx context.Context, id string) error {
err := r.repo.EmbyMount.Delete(ctx, id)
if err == nil {
r.invalidateRemoteMediaCache(ctx)
}
return err
}
// FullMountAccount 把账号的全部远程媒体库(View)挂载进来(幂等,已存在跳过)。
func (r *EmbyRemoteService) FullMountAccount(ctx context.Context, acct *model.StrmAccount, proxyPlayDefault bool) (int, error) {
views, err := r.RemoteViews(ctx, acct)
if err != nil {
return 0, err
}
mounts := make([]*model.EmbyMount, 0, len(views))
for _, v := range views {
viewID := remoteItemString(v, "Id")
if viewID == "" {
continue
}
mounts = append(mounts, &model.EmbyMount{
AccountID: acct.ID,
RemoteViewID: viewID,
RemoteViewName: remoteItemString(v, "Name"),
CollectionType: remoteItemString(v, "CollectionType"),
ProxyPlay: proxyPlayDefault,
Enabled: true,
})
}
return r.CreateMounts(ctx, mounts)
}
// ResolveMount 按伪装 ID 的第一段(挂载 ID)解析挂载与其所属账号。
// 远程条目/媒体库的伪装 ID 格式:embyremote~{mountID}~{remoteID}。
func (r *EmbyRemoteService) ResolveMount(ctx context.Context, mountID string) (*model.EmbyMount, *model.StrmAccount, error) {
mount, err := r.repo.EmbyMount.FindByID(ctx, mountID)
if err != nil || mount == nil || !mount.Enabled {
return nil, nil, errors.New("挂载不存在或已禁用")
}
acct := r.AccountByID(ctx, mount.AccountID)
if acct == nil {
return nil, nil, errors.New("远程 Emby 账号不存在或已禁用")
}
return mount, acct, nil
}
// AutoSeedMounts 兼容迁移:已有 emby_remote 账号但没有任何挂载时,自动把
// 其全部媒体库挂载进来(代理沿用账号旧配置),保证旧部署升级后媒体库不消失。
// 幂等:每个账号只在挂载数为 0 时执行一次。
func (r *EmbyRemoteService) AutoSeedMounts(ctx context.Context) {
accounts, err := r.ListAccounts(ctx)
if err != nil || len(accounts) == 0 {
return
}
for i := range accounts {
acct := &accounts[i]
count, err := r.repo.EmbyMount.CountByAccountID(ctx, acct.ID)
if err != nil || count > 0 {
continue
}
cfg, cfgErr := r.configOf(acct)
if cfgErr != nil {
continue
}
n, seedErr := r.FullMountAccount(ctx, acct, cfg.ProxyPlay)
if seedErr != nil {
if r.log != nil {
r.log.Warn("auto-seed emby mounts failed",
zap.String("account", acct.Name), zap.Error(seedErr))
}
} else if n > 0 {
if r.log != nil {
r.log.Info("auto-seeded emby mounts",
zap.String("account", acct.Name), zap.Int("mounts", n))
}
}
}
}
// configOf 解密账号配置。
func (r *EmbyRemoteService) configOf(acct *model.StrmAccount) (*EmbyRemoteConfig, error) {
raw := map[string]string{}
if acct != nil && strings.TrimSpace(acct.Config) != "" {
if err := json.Unmarshal([]byte(acct.Config), &raw); err != nil {
return nil, fmt.Errorf("decode emby account config: %w", err)
}
}
cfg := &EmbyRemoteConfig{
BaseURL: strings.TrimRight(strings.TrimSpace(raw["url"]), "/"),
Username: strings.TrimSpace(raw["username"]),
Password: r.crypto.Decrypt(raw["password"]),
Token: firstNonEmptyStr(r.crypto.Decrypt(raw["api_key"]), r.crypto.Decrypt(raw["token"])),
RemoteUserID: strings.TrimSpace(raw["remote_user_id"]),
ProxyPlay: parseBoolSetting(raw["proxy_play"], false),
}
if cfg.BaseURL == "" {
return nil, errors.New("缺少 Emby 地址")
}
if !strings.HasPrefix(cfg.BaseURL, "http://") && !strings.HasPrefix(cfg.BaseURL, "https://") {
return nil, errors.New("Emby 地址必须以 http:// 或 https:// 开头")
}
return cfg, nil
}
func firstNonEmptyStr(values ...string) string {
for _, v := range values {
if strings.TrimSpace(v) != "" {
return v
}
}
return ""
}
// embyBase 把地址规范为不带尾部斜杠的 /emby 根。
func (r *EmbyRemoteService) embyBase(cfg *EmbyRemoteConfig) string {
base := strings.TrimRight(cfg.BaseURL, "/")
if !strings.HasSuffix(base, "/emby") {
base += "/emby"
}
return base
}
// ensureToken 返回可用的 api_key:已有则直接用;否则用用户名/密码认证并回写
// 数据库(自动获得的 token 与 remote_user_id 会加密保存在账号配置里)。
func (r *EmbyRemoteService) ensureToken(ctx context.Context, acct *model.StrmAccount, cfg *EmbyRemoteConfig) error {
if strings.TrimSpace(cfg.Token) != "" {
return nil
}
if strings.TrimSpace(cfg.Username) == "" || strings.TrimSpace(cfg.Password) == "" {
return errors.New("缺少 Emby 凭据:请填写 api_key 或 用户名/密码")
}
body, _ := json.Marshal(map[string]string{"Username": cfg.Username, "Pw": cfg.Password})
endpoint := r.embyBase(cfg) + "/Users/AuthenticateByName"
req, err := http.NewRequestWithContext(ctx, http.MethodPost, endpoint, strings.NewReader(string(body)))
if err != nil {
return err
}
req.Header.Set("Content-Type", "application/json")
req.Header.Set("X-Emby-Authorization", `MediaBrowser Client="MMTL", Device="MMTL-Federated", DeviceId="mmtl-federated", Version="1.0"`)
resp, err := r.http.Do(req)
if err != nil {
return fmt.Errorf("连接远程 Emby 失败: %w", err)
}
defer resp.Body.Close()
if resp.StatusCode >= 300 {
data, _ := io.ReadAll(io.LimitReader(resp.Body, 512))
return fmt.Errorf("远程 Emby 登录失败(%d): %s", resp.StatusCode, strings.TrimSpace(string(data)))
}
var login struct {
AccessToken string `json:"AccessToken"`
User struct {
Id string `json:"Id"`
} `json:"User"`
}
if err := json.NewDecoder(resp.Body).Decode(&login); err != nil {
return err
}
if strings.TrimSpace(login.AccessToken) == "" {
return errors.New("远程 Emby 未返回 AccessToken")
}
cfg.Token = login.AccessToken
if login.User.Id != "" {
cfg.RemoteUserID = login.User.Id
}
return r.persistToken(ctx, acct, cfg)
}
// persistToken 把认证得到的 token / user id 加密写回账号配置(下次请求免登录)。
func (r *EmbyRemoteService) persistToken(ctx context.Context, acct *model.StrmAccount, cfg *EmbyRemoteConfig) error {
if acct == nil {
return nil
}
raw := map[string]string{}
if strings.TrimSpace(acct.Config) != "" {
_ = json.Unmarshal([]byte(acct.Config), &raw)
}
raw["api_key"] = r.crypto.Encrypt(cfg.Token)
raw["remote_user_id"] = cfg.RemoteUserID
if strings.TrimSpace(raw["username"]) == "" {
raw["username"] = cfg.Username
}
data, err := json.Marshal(raw)
if err != nil {
return err
}
acct.Config = string(data)
return r.repo.StrmAccount.Update(ctx, acct)
}
// doGet 向远程 Emby 发起带 api_key 的 GET,把响应 JSON 解码到 out。
// 401 时自动重认证一次再重试(凭据过期场景)。
func (r *EmbyRemoteService) doGet(ctx context.Context, acct *model.StrmAccount, cfg *EmbyRemoteConfig, path string, q url.Values, out any) error {
for attempt := 0; attempt < 2; attempt++ {
if err := r.ensureToken(ctx, acct, cfg); err != nil {
return err
}
endpoint := r.embyBase(cfg) + path
if q != nil {
endpoint += "?" + q.Encode()
} else {
endpoint += "?api_key=" + url.QueryEscape(cfg.Token)
}
req, err := http.NewRequestWithContext(ctx, http.MethodGet, endpoint, nil)
if err != nil {
return err
}
req.Header.Set("X-Emby-Token", cfg.Token)
resp, err := r.http.Do(req)
if err != nil {
return fmt.Errorf("请求远程 Emby 失败: %w", err)
}
data, readErr := io.ReadAll(io.LimitReader(resp.Body, 8<<20))
resp.Body.Close()
if readErr != nil {
return readErr
}
if resp.StatusCode == http.StatusUnauthorized && attempt == 0 {
// token 失效:清空后重认证重试一次。
cfg.Token = ""
if acct != nil {
raw := map[string]string{}
_ = json.Unmarshal([]byte(acct.Config), &raw)
delete(raw, "api_key")
enc, _ := json.Marshal(raw)
acct.Config = string(enc)
_ = r.repo.StrmAccount.Update(ctx, acct)
}
continue
}
if resp.StatusCode >= 300 {
return fmt.Errorf("远程 Emby 请求失败(%d): %s", resp.StatusCode, strings.TrimSpace(string(data)))
}
if out == nil {
return nil
}
return json.Unmarshal(data, out)
}
return errors.New("远程 Emby 认证重试失败")
}
// TestConnection 连通性测试:确保地址可达且凭据有效;成功时回写自动认证信息。
func (r *EmbyRemoteService) TestConnection(ctx context.Context, acct *model.StrmAccount) error {
cfg, err := r.configOf(acct)
if err != nil {
return err
}
if err := r.ensureToken(ctx, acct, cfg); err != nil {
return err
}
var out json.RawMessage
return r.doGet(ctx, acct, cfg, "/System/Info", nil, &out)
}
// ProxyPlayOf 返回账号是否配置了播放代理(供账号列表/编辑回显)。
func (r *EmbyRemoteService) ProxyPlayOf(acct *model.StrmAccount) (bool, error) {
cfg, err := r.configOf(acct)
if err != nil {
return false, err
}
return cfg.ProxyPlay, nil
}
// ─── 元数据 / 目录聚合 ─────────────────────────────────────────────────────────
// RemoteViews 拉取远程媒体库(View)列表,返回远程原始 view map(未重写)。
func (r *EmbyRemoteService) RemoteViews(ctx context.Context, acct *model.StrmAccount) ([]map[string]any, error) {
cfg, err := r.configOf(acct)
if err != nil {
return nil, err
}
cacheKey := r.remoteCacheKey("views", acct.ID, r.remoteUserID(cfg))
var cached []map[string]any
if r.cache != nil && r.cache.GetJSON(ctx, cacheKey, &cached) {
return cached, nil
}
q := url.Values{"api_key": {cfg.Token}}
var body struct {
Items []map[string]any `json:"Items"`
}
if err := r.doGet(ctx, acct, cfg, "/Users/"+url.PathEscape(r.remoteUserID(cfg))+"/Views", q, &body); err != nil {
return nil, err
}
if body.Items == nil {
body.Items = []map[string]any{}
}
if r.cache != nil {
r.cache.SetJSON(ctx, cacheKey, body.Items, r.remoteMediaCacheTTL())
}
return body.Items, nil
}
func (r *EmbyRemoteService) remoteUserID(cfg *EmbyRemoteConfig) string {
if strings.TrimSpace(cfg.RemoteUserID) != "" {
return cfg.RemoteUserID
}
return "0" // 未认证出的兜底:部分 Emby 接受 0 代表管理员
}
// RemoteItems 向远程 Emby 转发 /Items 浏览/搜索请求,返回重写后的响应载荷。
// p 的分页/排序/过滤参数原样转发,分页语义完全由远程承接。
func (r *EmbyRemoteService) RemoteItems(ctx context.Context, mount *model.EmbyMount, acct *model.StrmAccount, p ItemsParams) (map[string]any, error) {
cfg, err := r.configOf(acct)
if err != nil {
return nil, err
}
_, remoteParent, _ := DecodeEmbyRemoteID(p.ParentID)
q := url.Values{}
if remoteParent != "" {
q.Set("ParentId", remoteParent)
}
q.Set("UserId", r.remoteUserID(cfg))
q.Set("Limit", strconv.Itoa(p.Limit))
q.Set("StartIndex", strconv.Itoa(p.StartIndex))
if p.SearchTerm != "" {
q.Set("SearchTerm", p.SearchTerm)
}
if p.Recursive {
q.Set("Recursive", "true")
}
if p.SortBy != "" {
q.Set("SortBy", p.SortBy)
}
if p.SortOrder != "" {
q.Set("SortOrder", p.SortOrder)
}
if len(p.IncludeItemTypes) > 0 {
q.Set("IncludeItemTypes", strings.Join(p.IncludeItemTypes, ","))
}
if len(p.Filters) > 0 {
q.Set("Filters", strings.Join(p.Filters, ","))
}
path := "/Users/" + url.PathEscape(r.remoteUserID(cfg)) + "/Items"
var out map[string]any
if err := r.doGet(ctx, acct, cfg, path, q, &out); err != nil {
return nil, err
}
if out == nil {
out = map[string]any{"Items": []any{}, "TotalRecordCount": 0, "StartIndex": p.StartIndex}
}
RewriteEmbyRemoteIDs(out, mount.ID)
return out, nil
}
// RemoteSearchMount 对单个挂载的媒体库执行全局搜索(ParentId=挂载的远程库,
// Recursive 返回库内全部命中),结果归属明确可直接伪装。
func (r *EmbyRemoteService) RemoteSearchMount(ctx context.Context, mount *model.EmbyMount, acct *model.StrmAccount, p ItemsParams) (map[string]any, error) {
cfg, err := r.configOf(acct)
if err != nil {
return nil, err
}
q := url.Values{}
q.Set("ParentId", mount.RemoteViewID)
q.Set("Recursive", "true")
q.Set("SearchTerm", p.SearchTerm)
q.Set("UserId", r.remoteUserID(cfg))
q.Set("Limit", strconv.Itoa(p.Limit))
q.Set("StartIndex", strconv.Itoa(p.StartIndex))
if p.SortBy != "" {
q.Set("SortBy", p.SortBy)
}
if p.SortOrder != "" {
q.Set("SortOrder", p.SortOrder)
}
if len(p.IncludeItemTypes) > 0 {
q.Set("IncludeItemTypes", strings.Join(p.IncludeItemTypes, ","))
}
var out map[string]any
if err := r.doGet(ctx, acct, cfg, "/Users/"+url.PathEscape(r.remoteUserID(cfg))+"/Items", q, &out); err != nil {
return nil, err
}
if out == nil {
out = map[string]any{"Items": []any{}, "TotalRecordCount": 0}
}
RewriteEmbyRemoteIDs(out, mount.ID)
return out, nil
}
// RemoteItem 拉取远程单条目详情(含响应的重写)。
func (r *EmbyRemoteService) RemoteItem(ctx context.Context, mount *model.EmbyMount, acct *model.StrmAccount, remoteID string) (map[string]any, error) {
cfg, err := r.configOf(acct)
if err != nil {
return nil, err
}
path := "/Users/" + url.PathEscape(r.remoteUserID(cfg)) + "/Items/" + url.PathEscape(remoteID)
var out map[string]any
if err := r.doGet(ctx, acct, cfg, path, nil, &out); err != nil {
return nil, err
}
RewriteEmbyRemoteIDs(out, mount.ID)
return out, nil
}
// RemoteLatest 拉取远程「最近添加」(用于 /Items/Latest 聚合)。
func (r *EmbyRemoteService) RemoteLatest(ctx context.Context, mount *model.EmbyMount, acct *model.StrmAccount, parentID string, limit int) ([]map[string]any, error) {
cfg, err := r.configOf(acct)
if err != nil {
return nil, err
}
q := url.Values{"Limit": {strconv.Itoa(limit)}}
if parentID != "" {
q.Set("ParentId", parentID)
}
path := "/Users/" + url.PathEscape(r.remoteUserID(cfg)) + "/Items/Latest"
var out []map[string]any
if err := r.doGet(ctx, acct, cfg, path, q, &out); err != nil {
return nil, err
}
RewriteEmbyRemoteIDs(out, mount.ID)
return out, nil
}
// RemotePlaybackInfo 拉取远程 PlaybackInfo,并按挂载的 proxy_play 配置重写
// 播放 URL:不代理=指向远程绝对地址(播放字节不过 MMTL);代理=指向 MMTL
// 本地 /Videos/{encodedID} 端点(由 ProxyVideoStream 反代)。
func (r *EmbyRemoteService) RemotePlaybackInfo(ctx context.Context, mount *model.EmbyMount, acct *model.StrmAccount, remoteID, userID string) (map[string]any, error) {
cfg, err := r.configOf(acct)
if err != nil {
return nil, err
}
q := url.Values{"UserId": {r.remoteUserID(cfg)}}
path := "/Items/" + url.PathEscape(remoteID) + "/PlaybackInfo"
var out map[string]any
if err := r.doGet(ctx, acct, cfg, path, q, &out); err != nil {
return nil, err
}
RewriteEmbyRemoteIDs(out, mount.ID)
r.rewritePlayURLs(out, mount, cfg, remoteID)
return out, nil
}
// rewritePlayURLs 按挂载代理模式重写载荷内 MediaSources 的播放地址。
// 远程 Emby 的 PlaybackInfo 通常不返回 DirectStreamUrl(客户端靠它拼
// /Videos/{Id}/stream),因此这里总是强制构造播放地址,完全由 MMTL 掌控
// 直连(远程绝对 URL)或代理(本地 /Videos/{encoded})的最终去向。
func (r *EmbyRemoteService) rewritePlayURLs(value any, mount *model.EmbyMount, cfg *EmbyRemoteConfig, remoteID string) {
encoded := EncodeEmbyRemoteID(mount.ID, remoteID)
sources := collectMediaSources(value)
if sources == nil {
return
}
for _, src := range sources {
mediaSourceID, _ := src["Id"].(string)
var streamPath, subtitlePlayURL string
if mount.ProxyPlay {
streamPath = "/Videos/" + url.PathEscape(encoded) + "/stream"
subtitlePlayURL = "/Videos/" + url.PathEscape(encoded)
} else {
base := r.embyBase(cfg)
streamPath = base + "/Videos/" + url.PathEscape(remoteID) + "/stream?api_key=" + url.QueryEscape(cfg.Token) + "&Static=true"
if mediaSourceID != "" {
streamPath += "&MediaSourceId=" + url.QueryEscape(mediaSourceID)
}
subtitlePlayURL = base + "/Videos/" + url.PathEscape(remoteID)
}
// 直连/代理地址总是下发(PlaybackInfo 语义:客户端直接请求该 URL)。
src["DirectStreamUrl"] = streamPath
if _, exists := src["TranscodingUrl"]; exists {
src["TranscodingUrl"] = streamPath
}
rewriteSubtitleDeliveryURLs(src, subtitlePlayURL, cfg)
}
}
// collectMediaSources 从载荷中取出所有 MediaSources(顶层或嵌套 Items 内)。
func collectMediaSources(value any) []map[string]any {
var out []map[string]any
switch typed := value.(type) {
case map[string]any:
if sources, ok := typed["MediaSources"].([]map[string]any); ok {
out = append(out, sources...)
} else if sources, ok := typed["MediaSources"].([]any); ok {
for _, s := range sources {
if m, isMap := s.(map[string]any); isMap {
out = append(out, m)
}
}
}
if items, ok := typed["Items"]; ok {
out = append(out, collectMediaSources(items)...)
}
case []any:
for _, item := range typed {
out = append(out, collectMediaSources(item)...)
}
case []map[string]any:
for _, item := range typed {
out = append(out, collectMediaSources(item)...)
}
}
return out
}
var embySubtitleDeliveryRE = regexp.MustCompile(`/Subtitles/(\d+)/Stream(\.[A-Za-z0-9]+)?`)
// rewriteSubtitleDeliveryURLs 把 MediaSource 内字幕轨道的 DeliveryUrl 改写到
// subtitlePlayURL 前缀(客户端请求本地代理端点 / 远程绝对地址)。
func rewriteSubtitleDeliveryURLs(src map[string]any, playURL string, cfg *EmbyRemoteConfig) {
if src == nil {
return
}
streams, ok := src["MediaStreams"].([]any)
if !ok {
return
}
for _, s := range streams {
stream, isMap := s.(map[string]any)
if !isMap || stream["Type"] != "Subtitle" {
continue
}
raw, _ := stream["DeliveryUrl"].(string)
if raw == "" {
continue
}
idx := "1"
if m := embySubtitleDeliveryRE.FindStringSubmatch(raw); len(m) >= 2 {
idx = m[1]
}
ext := ""
if m := embySubtitleDeliveryRE.FindStringSubmatch(raw); len(m) >= 3 {
ext = m[2]
}
base := strings.TrimRight(playURL, "/")
delivery := base + "/Subtitles/" + idx + "/Stream" + ext
if !cfg.ProxyPlay && strings.TrimSpace(cfg.Token) != "" {
delivery += "?api_key=" + url.QueryEscape(cfg.Token)
}
stream["DeliveryUrl"] = delivery
}
}
// RemoteImageURL 构造远程图片绝对地址(由既有 ImageProxy 拉取透传)。
func (r *EmbyRemoteService) RemoteImageURL(ctx context.Context, acct *model.StrmAccount, remoteID, imageType string) (string, error) {
cfg, err := r.configOf(acct)
if err != nil {
return "", err
}
return r.embyBase(cfg) + "/Items/" + url.PathEscape(remoteID) + "/Images/" + url.PathEscape(strings.ToLower(imageType)) +
"?api_key=" + url.QueryEscape(cfg.Token), nil
}
// ─── 播放代理 ─────────────────────────────────────────────────────────────────
// ProxyVideoStream 反向代理远程 Emby 视频流(保留 Range 以支持拖动)。
func (r *EmbyRemoteService) ProxyVideoStream(ctx context.Context, w http.ResponseWriter, req *http.Request, acct *model.StrmAccount, remoteID string) error {
cfg, err := r.configOf(acct)
if err != nil {
return err
}
if err := r.ensureToken(ctx, acct, cfg); err != nil {
return err
}
endpoint := r.embyBase(cfg) + "/Videos/" + url.PathEscape(remoteID) + "/stream"
q := url.Values{}
if mediaSourceID := strings.TrimSpace(req.URL.Query().Get("MediaSourceId")); mediaSourceID != "" {
q.Set("MediaSourceId", mediaSourceID)
}
// 代理是纯 byte 中继:始终要求远程原文件直连(Static=true 阻止远程触发
// ffmpeg 转码调度——远程转码可能未配置/故障,导致整个代理 500)。
q.Set("Static", "true")
q.Set("api_key", cfg.Token)
if encoded := q.Encode(); encoded != "" {
endpoint += "?" + encoded
}
upstream, err := http.NewRequestWithContext(ctx, http.MethodGet, endpoint, nil)
if err != nil {
return err
}
upstream.Header.Set("X-Emby-Token", cfg.Token)
if rangeHeader := req.Header.Get("Range"); rangeHeader != "" {
upstream.Header.Set("Range", rangeHeader)
}
resp, err := r.http.Do(upstream)
if err != nil {
return fmt.Errorf("连接远程 Emby 视频流失败: %w", err)
}
defer resp.Body.Close()
if resp.StatusCode >= 400 {
data, _ := io.ReadAll(io.LimitReader(resp.Body, 256))
return fmt.Errorf("远程 Emby 视频流失败(%d): %s", resp.StatusCode, strings.TrimSpace(string(data)))
}
for _, header := range []string{"Content-Type", "Content-Length", "Content-Range", "Accept-Ranges", "ETag", "Cache-Control"} {
if value := resp.Header.Get(header); value != "" {
w.Header().Set(header, value)
}
}
if resp.StatusCode == http.StatusPartialContent || resp.StatusCode == http.StatusOK {
w.WriteHeader(resp.StatusCode)
} else {
w.WriteHeader(resp.StatusCode)
}
if resp.StatusCode == http.StatusPartialContent || resp.StatusCode == http.StatusOK {
_, _ = io.Copy(w, resp.Body)
}
return nil
}
// ProxySubtitle 反向代理远程 Emby 字幕流。
func (r *EmbyRemoteService) ProxySubtitle(ctx context.Context, w http.ResponseWriter, req *http.Request, acct *model.StrmAccount, remoteID, index string) error {
cfg, err := r.configOf(acct)
if err != nil {
return err
}
if err := r.ensureToken(ctx, acct, cfg); err != nil {
return err
}
endpoint := r.embyBase(cfg) + "/Videos/" + url.PathEscape(remoteID) + "/Subtitles/" + url.PathEscape(index) + "/Stream"
endpoint += "?api_key=" + url.QueryEscape(cfg.Token)
upstream, err := http.NewRequestWithContext(ctx, http.MethodGet, endpoint, nil)
if err != nil {
return err
}
upstream.Header.Set("X-Emby-Token", cfg.Token)
resp, err := r.http.Do(upstream)
if err != nil {
return fmt.Errorf("连接远程 Emby 字幕流失败: %w", err)
}
defer resp.Body.Close()
if resp.StatusCode >= 400 {
return fmt.Errorf("远程 Emby 字幕流失败(%d)", resp.StatusCode)
}
if value := resp.Header.Get("Content-Type"); value != "" {
w.Header().Set("Content-Type", value)
}
w.WriteHeader(resp.StatusCode)
if resp.StatusCode == http.StatusOK {
_, _ = io.Copy(w, resp.Body)
}
return nil
}
// ─── 播放状态透传 ──────────────────────────────────────────────────────────────
// ProxySetPlayed 把「已看/未看」状态透传到远程 Emby(MMTL 本地不落库)。
func (r *EmbyRemoteService) ProxySetPlayed(ctx context.Context, acct *model.StrmAccount, remoteID string, played bool) error {
cfg, err := r.configOf(acct)
if err != nil {
return err
}
if err := r.ensureToken(ctx, acct, cfg); err != nil {
return err
}
method := http.MethodPost
path := "/Users/" + url.PathEscape(r.remoteUserID(cfg)) + "/PlayedItems/" + url.PathEscape(remoteID)
if !played {
method = http.MethodDelete
}
return r.doMutate(ctx, acct, cfg, method, path)
}
// ProxySetFavorite 把「收藏/取消收藏」状态透传到远程 Emby。
func (r *EmbyRemoteService) ProxySetFavorite(ctx context.Context, acct *model.StrmAccount, remoteID string, favorite bool) error {
cfg, err := r.configOf(acct)
if err != nil {
return err
}
if err := r.ensureToken(ctx, acct, cfg); err != nil {
return err
}
method := http.MethodPost
path := "/Users/" + url.PathEscape(r.remoteUserID(cfg)) + "/FavoriteItems/" + url.PathEscape(remoteID)
if !favorite {
method = http.MethodDelete
}
return r.doMutate(ctx, acct, cfg, method, path)
}
func (r *EmbyRemoteService) doMutate(ctx context.Context, acct *model.StrmAccount, cfg *EmbyRemoteConfig, method, path string) error {
endpoint := r.embyBase(cfg) + path + "?api_key=" + url.QueryEscape(cfg.Token)
req, err := http.NewRequestWithContext(ctx, method, endpoint, nil)
if err != nil {
return err
}
req.Header.Set("X-Emby-Token", cfg.Token)
resp, err := r.http.Do(req)
if err != nil {
return fmt.Errorf("请求远程 Emby 失败: %w", err)
}
defer resp.Body.Close()
if resp.StatusCode >= 300 {
data, _ := io.ReadAll(io.LimitReader(resp.Body, 256))
return fmt.Errorf("远程 Emby 状态同步失败(%d): %s", resp.StatusCode, strings.TrimSpace(string(data)))
}
return nil
}
+109
View File
@@ -0,0 +1,109 @@
// 远程 Emby 聚合的 ID 伪装。
//
// MMTL 作为 Emby 联邦网关把多个远程 Emby 服务器的媒体库透明聚合到自身的
// Emby API 之下,远程条目完全不落库。为了把本地 ID 与多个远程服务器的 ID
// 隔离开,远程条目在返回给客户端之前统一被改写为:
//
// embyremote~{accountID}~{remoteID}
//
// 客户端后续对图片 / 详情 / 播放 / 播放状态 的请求都会携带这个伪装 ID,
// 服务端据此解码出对应账号与原始 ID,直接向远程 Emby 转发。
package service
import (
"strings"
)
// EmbyRemoteIDPrefix 远程条目伪装 ID 的前缀(本地 UUID 与 Emby ID 不会出现 "~")。
const EmbyRemoteIDPrefix = "embyremote~"
// IsEmbyRemoteID 报告 id 是否是伪装过的远程 Emby 条目 ID。
func IsEmbyRemoteID(id string) bool {
return strings.HasPrefix(id, EmbyRemoteIDPrefix)
}
// EncodeEmbyRemoteID 把 (账号 ID, 远程条目 ID) 伪装为对外暴露的 ID。
func EncodeEmbyRemoteID(accountID, remoteID string) string {
return EmbyRemoteIDPrefix + accountID + "~" + remoteID
}
// DecodeEmbyRemoteID 拆分伪装 ID 为 (账号 ID, 远程原始 ID)。不是伪装 ID 时返回
// ok=false。远程 ID 本身允许包含 "~"(使用 SplitN 只切第一刀)。
func DecodeEmbyRemoteID(id string) (accountID, remoteID string, ok bool) {
if !IsEmbyRemoteID(id) {
return "", "", false
}
rest := strings.TrimPrefix(id, EmbyRemoteIDPrefix)
parts := strings.SplitN(rest, "~", 2)
if len(parts) != 2 || strings.TrimSpace(parts[0]) == "" || strings.TrimSpace(parts[1]) == "" {
return "", "", false
}
return parts[0], parts[1], true
}
// embyRemoteStringIDs 是条目 JSON 中需要伪装(编码)成远程 ID 的字符串字段。
// 图片 / 详情 / 播放请求都会以这些字段的值作为 ID 回指 MMTL。
var embyRemoteStringIDs = []string{
"Id",
"ParentId",
"SeriesId",
"SeasonId",
"PrimaryImageItemId",
"DisplayPreferencesId",
}
// RewriteEmbyRemoteIDs 在内存中把远程 Emby 返回的载荷里的所有条目 ID 替换为
// 伪装 ID(防止与本地、多远程冲突),嵌套 Items / Map 数组递归处理。
//
// MediaSources 里的 Id / MediaSourceId 保持不变:客户端只把它们作为查询
// 参数原样带回,转发时直接送回远程即可。播放 URL 的重写由服务层
// (rewriteEmbyRemotePlayURLs)按直连/代理模式处理。
func RewriteEmbyRemoteIDs(value any, accountID string) {
switch typed := value.(type) {
case map[string]any:
rewriteEmbyRemoteIDsMap(typed, accountID)
case []any:
for _, item := range typed {
RewriteEmbyRemoteIDs(item, accountID)
}
case []map[string]any:
for _, item := range typed {
rewriteEmbyRemoteIDsMap(item, accountID)
}
}
}
func rewriteEmbyRemoteIDsMap(m map[string]any, accountID string) {
if m == nil {
return
}
for _, key := range embyRemoteStringIDs {
if raw, ok := m[key].(string); ok && raw != "" {
m[key] = EncodeEmbyRemoteID(accountID, raw)
}
}
if tags, ok := m["ImageTags"].(map[string]any); ok {
for k, v := range tags {
if s, isStr := v.(string); isStr && s != "" {
tags[k] = EncodeEmbyRemoteID(accountID, s)
}
}
}
if tags, ok := m["ImageTags"].(map[string]string); ok {
for k, v := range tags {
if v != "" {
tags[k] = EncodeEmbyRemoteID(accountID, v)
}
}
}
if tags, ok := m["BackdropImageTags"].([]any); ok {
for i := range tags {
if s, isStr := tags[i].(string); isStr && s != "" {
tags[i] = EncodeEmbyRemoteID(accountID, s)
}
}
}
if items, ok := m["Items"]; ok {
RewriteEmbyRemoteIDs(items, accountID)
}
}
+102
View File
@@ -0,0 +1,102 @@
package service
import (
"testing"
)
func TestEmbyRemoteIDEncodeDecode(t *testing.T) {
encoded := EncodeEmbyRemoteID("acct-1", "item-123")
want := "embyremote~acct-1~item-123"
if encoded != want {
t.Fatalf("encoded = %q, want %q", encoded, want)
}
if !IsEmbyRemoteID(encoded) {
t.Fatalf("IsEmbyRemoteID(%q) = false", encoded)
}
acctID, remoteID, ok := DecodeEmbyRemoteID(encoded)
if !ok || acctID != "acct-1" || remoteID != "item-123" {
t.Fatalf("decode = (%q, %q, %v)", acctID, remoteID, ok)
}
}
func TestDecodeEmbyRemoteIDAllowsTildeInRemoteID(t *testing.T) {
// 远程 ID 本身允许包含 "~":只切第一刀。
encoded := EncodeEmbyRemoteID("acct-1", "a~b~c")
acctID, remoteID, ok := DecodeEmbyRemoteID(encoded)
if !ok || acctID != "acct-1" || remoteID != "a~b~c" {
t.Fatalf("decode = (%q, %q, %v)", acctID, remoteID, ok)
}
}
func TestDecodeEmbyRemoteIDRejectsLocalUUIDs(t *testing.T) {
if _, _, ok := DecodeEmbyRemoteID("550e8400-e29b-41d4-a716-446655440000"); ok {
t.Fatal("local UUID must not decode as remote id")
}
if _, _, ok := DecodeEmbyRemoteID("embyremote~only-acct"); ok {
t.Fatal("malformed remote id must not decode")
}
if _, _, ok := DecodeEmbyRemoteID("embyremote~~"); ok {
t.Fatal("empty parts must not decode")
}
}
func TestRewriteEmbyRemoteIDs(t *testing.T) {
payload := map[string]any{
"Id": "item-1",
"ParentId": "folder-1",
"SeriesId": "series-1",
"SeasonId": "season-1",
"PrimaryImageItemId": "item-1",
"DisplayPreferencesId": "folder-1",
"ImageTags": map[string]any{
"Primary": "item-1",
},
"BackdropImageTags": []any{"item-1-bd"},
"Items": []any{
map[string]any{"Id": "item-2", "ParentId": "folder-2"},
},
// MediaSource 的 Id 保持原样(客户端仅作为 MediaSourceId 查询参数)。
"MediaSources": []any{
map[string]any{
"Id": "ms-9",
"DirectStreamUrl": "/Videos/item-1/stream",
"MediaStreams": []any{
map[string]any{"Type": "Subtitle", "DeliveryUrl": "/Videos/item-1/Subtitles/2/Stream.srt"},
},
},
},
}
RewriteEmbyRemoteIDs(payload, "acct-1")
if got := payload["Id"]; got != "embyremote~acct-1~item-1" {
t.Fatalf("Id = %v", got)
}
if got := payload["ParentId"]; got != "embyremote~acct-1~folder-1" {
t.Fatalf("ParentId = %v", got)
}
if got := payload["SeriesId"]; got != "embyremote~acct-1~series-1" {
t.Fatalf("SeriesId = %v", got)
}
if got := payload["SeasonId"]; got != "embyremote~acct-1~season-1" {
t.Fatalf("SeasonId = %v", got)
}
if got := payload["ImageTags"].(map[string]any)["Primary"]; got != "embyremote~acct-1~item-1" {
t.Fatalf("ImageTags.Primary = %v", got)
}
if got := payload["BackdropImageTags"].([]any)[0]; got != "embyremote~acct-1~item-1-bd" {
t.Fatalf("BackdropImageTags[0] = %v", got)
}
nested := payload["Items"].([]any)[0].(map[string]any)
if nested["Id"] != "embyremote~acct-1~item-2" {
t.Fatalf("nested Id = %v", nested["Id"])
}
// MediaSource.Id 与 URL 不被 ID 重写器触碰(URL 由代理模式函数改写)。
ms := payload["MediaSources"].([]any)[0].(map[string]any)
if ms["Id"] != "ms-9" {
t.Fatalf("MediaSource.Id must stay raw, got %v", ms["Id"])
}
if ms["DirectStreamUrl"] != "/Videos/item-1/stream" {
t.Fatalf("DirectStreamUrl must stay raw, got %v", ms["DirectStreamUrl"])
}
}
@@ -0,0 +1,99 @@
package service
import (
"testing"
"github.com/ShukeBta/MMTL/internal/model"
)
// rewriteSubtitleDeliveryURLs 只应改动字幕轨道的 DeliveryUrl,其余媒体流不动。
func TestRewriteSubtitleDeliveryURLsProxyMode(t *testing.T) {
src := map[string]any{
"MediaStreams": []any{
map[string]any{"Type": "Video", "DeliveryUrl": "/Videos/x/stream"},
map[string]any{"Type": "Audio", "DeliveryUrl": "/Videos/x/stream"},
map[string]any{"Type": "Subtitle", "DeliveryUrl": "/Videos/item-1/ms-9/Subtitles/2/Stream.srt"},
},
}
rewriteSubtitleDeliveryURLs(src, "/Videos/embyremote~acct-1~item-1", &EmbyRemoteConfig{})
streams := src["MediaStreams"].([]any)
if got := streams[0].(map[string]any)["DeliveryUrl"]; got != "/Videos/x/stream" {
t.Fatalf("video DeliveryUrl must stay, got %v", got)
}
want := "/Videos/embyremote~acct-1~item-1/Subtitles/2/Stream.srt"
if got := streams[2].(map[string]any)["DeliveryUrl"]; got != want {
t.Fatalf("subtitle DeliveryUrl = %v, want %v", got, want)
}
}
func TestRewriteSubtitleDeliveryURLsDirectMode(t *testing.T) {
src := map[string]any{
"MediaStreams": []any{
map[string]any{"Type": "Subtitle", "DeliveryUrl": "/Videos/item-1/ms-9/Subtitles/1/Stream.ass"},
},
}
cfg := &EmbyRemoteConfig{Token: "tok123"}
rewriteSubtitleDeliveryURLs(src, "http://remote:8096/emby/Videos/item-1", cfg)
streams := src["MediaStreams"].([]any)
want := "http://remote:8096/emby/Videos/item-1/Subtitles/1/Stream.ass?api_key=tok123"
if got := streams[0].(map[string]any)["DeliveryUrl"]; got != want {
t.Fatalf("subtitle DeliveryUrl = %v, want %v", got, want)
}
}
func TestRewriteSubtitleDeliveryURLsFallsBackIndexOne(t *testing.T) {
src := map[string]any{
"MediaStreams": []any{
map[string]any{"Type": "Subtitle", "DeliveryUrl": "custom/url"},
},
}
rewriteSubtitleDeliveryURLs(src, "/Videos/embyremote~acct-1~item-1", &EmbyRemoteConfig{})
streams := src["MediaStreams"].([]any)
want := "/Videos/embyremote~acct-1~item-1/Subtitles/1/Stream"
if got := streams[0].(map[string]any)["DeliveryUrl"]; got != want {
t.Fatalf("subtitle DeliveryUrl = %v, want %v", got, want)
}
}
func TestMapRemoteItemToMediaExtractsCodecsAndContainer(t *testing.T) {
r := &EmbyRemoteService{}
item := map[string]any{
"Id": "item-100",
"Name": "Test Movie",
"Container": "mkv",
"MediaStreams": []any{
map[string]any{
"Type": "Video",
"Codec": "h264",
"Width": 1920,
"Height": 1080,
},
map[string]any{
"Type": "Audio",
"Codec": "aac",
},
},
"MediaSources": []any{
map[string]any{
"Container": "mkv",
"Size": int64(104857600),
},
},
}
media := r.MapRemoteItemToMedia(t.Context(), nil, &model.StrmAccount{Base: model.Base{ID: "acct-1"}}, &EmbyRemoteConfig{}, item)
if media.Container != "mkv" {
t.Fatalf("media.Container = %v, want mkv", media.Container)
}
if media.VideoCodec != "h264" {
t.Fatalf("media.VideoCodec = %v, want h264", media.VideoCodec)
}
if media.AudioCodec != "aac" {
t.Fatalf("media.AudioCodec = %v, want aac", media.AudioCodec)
}
if media.Width != 1920 || media.Height != 1080 {
t.Fatalf("resolution = %dx%d, want 1920x1080", media.Width, media.Height)
}
if media.SizeBytes != 104857600 {
t.Fatalf("size = %d, want 104857600", media.SizeBytes)
}
}
+677
View File
@@ -0,0 +1,677 @@
// 网页端远程 Emby 库映射。
//
// 网页端(React UI)的媒体库/媒体浏览走项目自有 REST API(/api/libraries、
// /api/libraries/:id/media、/api/media/:id 等),数据结构为 model.Library /
// model.Media / SeriesCard。远程 Emby 挂载的数据不落库,因此这里把远程
// Emby 的 JSON item 映射为与本地完全一致的结构,让网页端无感知地浏览
// 远程库;播放统一走 /api/stream/{伪装ID}(302 到远程 Emby 原地址)。
package service
import (
"context"
"net/url"
"sort"
"strconv"
"strings"
"go.uber.org/zap"
"github.com/ShukeBta/MMTL/internal/model"
)
// RemoteLibraryView 是一个网页端可见的远程媒体库(对应一个挂载的远程 View)。
type RemoteLibraryView struct {
Library model.Library
MountID string
AccountID string
RemoteID string
CollectionType string
AccountName string
}
// RemoteLibraries 把所有启用挂载的远程媒体库映射为网页媒体库列表
// (只有显式挂载的库才出现在本项目媒体库中)。
func (r *EmbyRemoteService) RemoteLibraries(ctx context.Context) ([]RemoteLibraryView, error) {
mounts, err := r.ListMounts(ctx)
if err != nil || len(mounts) == 0 {
return nil, err
}
// 按账号分组,每账号拉一次 Views 做匹配。
byAccount := map[string][]*model.EmbyMount{}
for i := range mounts {
m := mounts[i]
if !m.Enabled {
continue
}
byAccount[m.AccountID] = append(byAccount[m.AccountID], &mounts[i])
}
out := make([]RemoteLibraryView, 0, len(mounts))
for accountID, accountMounts := range byAccount {
acct := r.AccountByID(ctx, accountID)
if acct == nil {
continue
}
cfg, cfgErr := r.configOf(acct)
if cfgErr != nil {
continue
}
views, viewErr := r.RemoteViews(ctx, acct)
if viewErr != nil {
if r.log != nil {
r.log.Warn("web remote emby views failed",
zap.String("account", acct.Name), zap.Error(viewErr))
}
continue
}
viewByName := map[string]map[string]any{}
for _, v := range views {
viewByName[remoteItemString(v, "Id")] = v
}
for _, mount := range accountMounts {
v, ok := viewByName[mount.RemoteViewID]
if !ok {
continue // 远程已删除该媒体库
}
lib := r.mapRemoteMountToLibrary(mount, acct, cfg, v)
if lib == nil {
continue
}
out = append(out, RemoteLibraryView{
Library: *lib,
MountID: mount.ID,
AccountID: acct.ID,
RemoteID: mount.RemoteViewID,
CollectionType: mount.CollectionType,
AccountName: acct.Name,
})
}
}
return out, nil
}
// RemoteLibraryByID 按伪装 ID 查远程库视图(详情接口用)。
func (r *EmbyRemoteService) RemoteLibraryByID(ctx context.Context, mountID, remoteViewID string) (*RemoteLibraryView, error) {
views, err := r.RemoteLibraries(ctx)
if err != nil {
return nil, err
}
for _, v := range views {
if v.MountID == mountID && v.RemoteID == remoteViewID {
cp := v
return &cp, nil
}
}
return nil, nil
}
// mapRemoteMountToLibrary 把挂载信息 + 远程 View item 映射为网页库结构。
func (r *EmbyRemoteService) mapRemoteMountToLibrary(mount *model.EmbyMount, acct *model.StrmAccount, cfg *EmbyRemoteConfig, item map[string]any) *model.Library {
if mount == nil {
return nil
}
name := strings.TrimSpace(mount.Name)
if name == "" {
name = strings.TrimSpace(remoteItemString(item, "Name"))
}
if name == "" {
name = acct.Name
} else if !strings.Contains(name, acct.Name) {
name = acct.Name + " · " + name
}
libType := "movie"
switch mount.CollectionType {
case "tvshows":
libType = "tv"
case "music":
libType = "music"
}
lib := &model.Library{
Base: model.Base{ID: EncodeEmbyRemoteID(mount.ID, mount.RemoteViewID)},
Name: name,
Type: libType,
Enabled: true,
SortOrder: 1000, // 远程库排在本地库之后
}
// 远程媒体库封面只有真实存在图片标签才下发。
if remoteItemHasImageTag(item, "Primary") {
lib.CoverURL = r.remoteItemImageURL(cfg, mount.RemoteViewID, "Primary")
}
return lib
}
// MapRemoteItemToMedia 把远程 Emby item JSON 映射为本地 Media 结构。
// poster/backdrop 只有在远程确实存在图片标签时才填 URL(避免对无图条目
// 发出必失败的图片请求导致前端破图);剧集回退到系列海报(SeriesPrimaryImage)。
func (r *EmbyRemoteService) MapRemoteItemToMedia(ctx context.Context, mount *model.EmbyMount, acct *model.StrmAccount, cfg *EmbyRemoteConfig, item map[string]any) model.Media {
encodeScope := acct.ID
if mount != nil {
encodeScope = mount.ID
}
remoteID := remoteItemString(item, "Id")
// 条目可能已被 RewriteEmbyRemoteIDs 伪装(图片/嵌套 ID 需要原始远程 ID)。
if _, rid, ok := DecodeEmbyRemoteID(remoteID); ok {
remoteID = rid
}
seriesID := remoteItemString(item, "SeriesId")
if _, rid, ok := DecodeEmbyRemoteID(seriesID); ok {
seriesID = rid
}
media := model.Media{
Base: model.Base{ID: EncodeEmbyRemoteID(encodeScope, remoteID)},
Title: remoteItemString(item, "Name"),
OriginalName: remoteItemString(item, "OriginalTitle"),
Overview: remoteItemString(item, "Overview"),
Year: remoteItemInt(item, "ProductionYear"),
Rating: float32(remoteItemFloat(item, "CommunityRating")),
Path: remoteItemString(item, "Path"),
Genres: remoteItemGenres(item),
ScrapeStatus: "done",
}
// 只有远程明确存在图片标签才下发图片 URL。
if remoteItemHasImageTag(item, "Primary") {
media.PosterURL = r.remoteItemImageURL(cfg, remoteID, "Primary")
}
if remoteItemHasImageTag(item, "Backdrop") || len(remoteBackdropTags(item)) > 0 {
media.BackdropURL = r.remoteItemImageURL(cfg, remoteID, "Backdrop")
}
if ticks := remoteItemInt64(item, "RunTimeTicks"); ticks > 0 {
media.DurationSec = int(ticks / 10_000_000)
}
if date, ok := embyPremiereDate(remoteItemString(item, "PremiereDate")); ok {
media.ReleaseDate = date.Format("2006-01-02")
}
if providerIDs, ok := item["ProviderIds"].(map[string]any); ok {
if v := anyString(providerIDs["Tmdb"]); v != "" {
media.TMDbID, _ = strconv.Atoi(v)
}
if v := anyString(providerIDs["Imdb"]); v != "" {
media.TheTVDBID = v
}
if v := anyString(providerIDs["Douban"]); v != "" {
media.DoubanID = v
}
}
media.Container = remoteItemString(item, "Container")
media.Width = remoteItemInt(item, "Width")
media.Height = remoteItemInt(item, "Height")
extractStreamInfo := func(streams []any) {
for _, s := range streams {
sm, ok := s.(map[string]any)
if !ok {
continue
}
typ := remoteItemString(sm, "Type")
if strings.EqualFold(typ, "Video") {
if media.VideoCodec == "" {
media.VideoCodec = remoteItemString(sm, "Codec")
}
if media.Width == 0 {
media.Width = remoteItemInt(sm, "Width")
}
if media.Height == 0 {
media.Height = remoteItemInt(sm, "Height")
}
} else if strings.EqualFold(typ, "Audio") {
if media.AudioCodec == "" {
media.AudioCodec = remoteItemString(sm, "Codec")
}
}
}
}
if streams, ok := item["MediaStreams"].([]any); ok {
extractStreamInfo(streams)
} else if streams, ok := item["MediaStreams"].([]map[string]any); ok {
anyStreams := make([]any, len(streams))
for i, v := range streams {
anyStreams[i] = v
}
extractStreamInfo(anyStreams)
}
if sources, ok := item["MediaSources"].([]any); ok && len(sources) > 0 {
if sourceMap, ok := sources[0].(map[string]any); ok {
if media.Container == "" {
media.Container = remoteItemString(sourceMap, "Container")
}
if media.SizeBytes == 0 {
media.SizeBytes = remoteItemInt64(sourceMap, "Size")
}
if streams, ok := sourceMap["MediaStreams"].([]any); ok && (media.VideoCodec == "" || media.AudioCodec == "") {
extractStreamInfo(streams)
}
}
}
switch remoteItemString(item, "Type") {
case "Episode":
media.SeasonNum = remoteItemInt(item, "ParentIndexNumber")
media.EpisodeNum = remoteItemInt(item, "IndexNumber")
media.EpisodeTitle = remoteItemString(item, "Name")
if seriesName := remoteItemString(item, "SeriesName"); seriesName != "" {
media.Title = seriesName
}
// 单集通常无独立海报:若远程返回 SeriesPrimaryImageTag(需要
// Fields=SeriesPrimaryImage)且系列有图,则回退到系列海报。
if media.PosterURL == "" && seriesID != "" &&
strings.TrimSpace(remoteItemString(item, "SeriesPrimaryImageTag")) != "" {
media.PosterURL = r.remoteItemImageURL(cfg, seriesID, "Primary")
}
default: // Movie / Series / Season / Folder
media.SeasonNum = 0
media.EpisodeNum = 0
}
return media
}
// RemoteLibraryMedia 拉远程库直属条目(电影库=Movie,剧集库=Series),映射分页。
func (r *EmbyRemoteService) RemoteLibraryMedia(ctx context.Context, mount *model.EmbyMount, acct *model.StrmAccount, remoteViewID string, itemTypes string, offset, limit int) ([]model.Media, int64, error) {
cfg, err := r.configOf(acct)
if err != nil {
return nil, 0, err
}
if itemTypes == "" {
itemTypes = "Movie,Series" // 未知类型时两者都取(前端自行按 episode-like 分组)
}
cacheKey := r.remoteCacheKey("library-media", acct.ID, mount.ID, remoteViewID, itemTypes, strconv.Itoa(offset), strconv.Itoa(limit))
var cached struct {
Items []model.Media `json:"items"`
TotalRecordCount int64 `json:"total_record_count"`
}
if r.cache != nil && r.cache.GetJSON(ctx, cacheKey, &cached) {
return cached.Items, cached.TotalRecordCount, nil
}
q := url.Values{}
q.Set("ParentId", remoteViewID)
q.Set("IncludeItemTypes", itemTypes)
q.Set("Recursive", "false")
q.Set("StartIndex", strconv.Itoa(offset))
q.Set("Limit", strconv.Itoa(limit))
q.Set("Fields", "Overview,Genres,ProviderIds,Path,SeriesPrimaryImage,MediaStreams,MediaSources")
var body struct {
Items []map[string]any `json:"Items"`
TotalRecordCount int64 `json:"TotalRecordCount"`
}
if err := r.doGet(ctx, acct, cfg, "/Users/"+url.PathEscape(r.remoteUserID(cfg))+"/Items", q, &body); err != nil {
return nil, 0, err
}
items := make([]model.Media, 0, len(body.Items))
for _, it := range body.Items {
RewriteEmbyRemoteIDs(it, mount.ID) // 嵌套/关联 ID 一并伪装
items = append(items, r.MapRemoteItemToMedia(ctx, mount, acct, cfg, it))
}
if r.cache != nil {
r.cache.SetJSON(ctx, cacheKey, struct {
Items []model.Media `json:"items"`
TotalRecordCount int64 `json:"total_record_count"`
}{Items: items, TotalRecordCount: body.TotalRecordCount}, r.remoteMediaCacheTTL())
}
return items, body.TotalRecordCount, nil
}
// RemoteMediaDetail 拉远程单条目映射为 Media(网页详情页)。
func (r *EmbyRemoteService) RemoteMediaDetail(ctx context.Context, mount *model.EmbyMount, acct *model.StrmAccount, remoteID string) (*model.Media, error) {
cfg, err := r.configOf(acct)
if err != nil {
return nil, err
}
path := "/Users/" + url.PathEscape(r.remoteUserID(cfg)) + "/Items/" + url.PathEscape(remoteID)
path += "?Fields=Overview,Genres,ProviderIds,People,Studios,Path,MediaStreams,MediaSources"
var out map[string]any
if err := r.doGet(ctx, acct, cfg, path, nil, &out); err != nil {
return nil, err
}
RewriteEmbyRemoteIDs(out, mount.ID)
m := r.MapRemoteItemToMedia(ctx, mount, acct, cfg, out)
return &m, nil
}
// RemoteEpisodes 拉远程条目下的集列表(Series/Season/Folder→子集;Episode→同系列;
// Movie→自身单条),按季/集排序,与本地 ListMediaEpisodes 行为一致。
func (r *EmbyRemoteService) RemoteEpisodes(ctx context.Context, mount *model.EmbyMount, acct *model.StrmAccount, remoteID string) ([]model.Media, error) {
detail, err := r.RemoteMediaDetail(ctx, mount, acct, remoteID)
if err != nil {
return nil, err
}
// 用远程详情载荷精判类型(Episode→同系列;Series/Season/Folder→子集;Movie→单条)。
itemType := r.remoteItemType(ctx, acct, remoteID)
if itemType == "" {
itemType = remoteItemTypeOf(detail)
}
var parentID string
switch itemType {
case "Episode":
parentID = r.remoteItemSeriesID(ctx, acct, remoteID)
if parentID == "" {
parentID = remoteID
}
case "Season", "Folder", "Series":
parentID = remoteID
default: // Movie
return []model.Media{*detail}, nil
}
rows, _, err := r.remoteEpisodesOf(ctx, mount, acct, parentID)
if err != nil {
return nil, err
}
sort.SliceStable(rows, func(i, j int) bool {
if rows[i].SeasonNum != rows[j].SeasonNum {
return rows[i].SeasonNum < rows[j].SeasonNum
}
if rows[i].EpisodeNum != rows[j].EpisodeNum {
return rows[i].EpisodeNum < rows[j].EpisodeNum
}
return rows[i].Title < rows[j].Title
})
return rows, nil
}
func (r *EmbyRemoteService) remoteEpisodesOf(ctx context.Context, mount *model.EmbyMount, acct *model.StrmAccount, parentID string) ([]model.Media, int64, error) {
cfg, err := r.configOf(acct)
if err != nil {
return nil, 0, err
}
q := url.Values{}
q.Set("ParentId", parentID)
q.Set("IncludeItemTypes", "Episode")
q.Set("Recursive", "true")
q.Set("StartIndex", "0")
q.Set("Limit", "500")
q.Set("Fields", "Overview,Genres,ProviderIds,Path,SeriesPrimaryImage,MediaStreams,MediaSources")
var body struct {
Items []map[string]any `json:"Items"`
TotalRecordCount int64 `json:"TotalRecordCount"`
}
if err := r.doGet(ctx, acct, cfg, "/Users/"+url.PathEscape(r.remoteUserID(cfg))+"/Items", q, &body); err != nil {
return nil, 0, err
}
items := make([]model.Media, 0, len(body.Items))
for _, it := range body.Items {
RewriteEmbyRemoteIDs(it, mount.ID)
m := r.MapRemoteItemToMedia(ctx, mount, acct, cfg, it)
items = append(items, m)
}
return items, body.TotalRecordCount, nil
}
// RemoteSeriesCards 远程剧集库的系列卡片(ChildCount 作为集数)。
func (r *EmbyRemoteService) RemoteSeriesCards(ctx context.Context, mount *model.EmbyMount, acct *model.StrmAccount, remoteViewID string) ([]SeriesCard, error) {
cfg, err := r.configOf(acct)
if err != nil {
return nil, err
}
cacheKey := r.remoteCacheKey("series-cards", acct.ID, mount.ID, remoteViewID)
var cached []SeriesCard
if r.cache != nil && r.cache.GetJSON(ctx, cacheKey, &cached) {
return cached, nil
}
q := url.Values{}
q.Set("ParentId", remoteViewID)
q.Set("IncludeItemTypes", "Series")
q.Set("Recursive", "false")
q.Set("StartIndex", "0")
q.Set("Limit", "1000")
q.Set("Fields", "Overview,Genres,ProviderIds,Path,RecursiveItemCount,SeriesPrimaryImage")
var body struct {
Items []map[string]any `json:"Items"`
}
if err := r.doGet(ctx, acct, cfg, "/Users/"+url.PathEscape(r.remoteUserID(cfg))+"/Items", q, &body); err != nil {
return nil, err
}
cards := make([]SeriesCard, 0, len(body.Items))
for _, it := range body.Items {
RewriteEmbyRemoteIDs(it, mount.ID)
m := r.MapRemoteItemToMedia(ctx, mount, acct, cfg, it)
// 集数优先用递归条目数(ChildCount 只算直属 Season 文件夹数)。
count := remoteItemInt(it, "RecursiveItemCount")
if count == 0 {
count = remoteItemInt(it, "ChildCount")
}
if count == 0 {
count = 1
}
cards = append(cards, SeriesCard{Key: m.ID, Rep: m, LinkMedia: m, Count: count})
}
if r.cache != nil {
r.cache.SetJSON(ctx, cacheKey, cards, r.remoteMediaCacheTTL())
}
return cards, nil
}
// RemoteLatestCards 远程库最新条目(首页预览卡片),映射 SeriesCard。
func (r *EmbyRemoteService) RemoteLatestCards(ctx context.Context, mount *model.EmbyMount, acct *model.StrmAccount, remoteViewID string, limit int) ([]SeriesCard, error) {
cfg, err := r.configOf(acct)
if err != nil {
return nil, err
}
cacheKey := r.remoteCacheKey("latest-cards", acct.ID, mount.ID, remoteViewID, strconv.Itoa(limit))
var cached []SeriesCard
if r.cache != nil && r.cache.GetJSON(ctx, cacheKey, &cached) {
return cached, nil
}
items, err := r.RemoteLatest(ctx, mount, acct, remoteViewID, limit)
if err != nil {
return nil, err
}
cards := make([]SeriesCard, 0, len(items))
for _, it := range items {
m := r.MapRemoteItemToMedia(ctx, mount, acct, cfg, it)
cards = append(cards, SeriesCard{Key: m.ID, Rep: m, LinkMedia: m, Count: 0})
}
if r.cache != nil {
r.cache.SetJSON(ctx, cacheKey, cards, r.remoteMediaCacheTTL())
}
return cards, nil
}
// WebStreamURL 远程条目的网页播放地址(302 直连远程 Emby 流端点)。
func (r *EmbyRemoteService) WebStreamURL(ctx context.Context, acct *model.StrmAccount, remoteID string) (string, error) {
cfg, err := r.configOf(acct)
if err != nil {
return "", err
}
if err := r.ensureToken(ctx, acct, cfg); err != nil {
return "", err
}
return r.embyBase(cfg) + "/Videos/" + url.PathEscape(remoteID) +
"/stream?api_key=" + url.QueryEscape(cfg.Token) + "&Static=true", nil
}
// remoteItemType 轻量查询远程条目 Type(避免依赖映射载荷)。
func (r *EmbyRemoteService) remoteItemType(ctx context.Context, acct *model.StrmAccount, remoteID string) string {
cfg, err := r.configOf(acct)
if err != nil {
return ""
}
var out map[string]any
if err := r.doGet(ctx, acct, cfg, "/Users/"+url.PathEscape(r.remoteUserID(cfg))+"/Items/"+url.PathEscape(remoteID), nil, &out); err != nil {
return ""
}
return remoteItemString(out, "Type")
}
// remoteItemSeriesID 轻量查询 Episode 的 SeriesId。
func (r *EmbyRemoteService) remoteItemSeriesID(ctx context.Context, acct *model.StrmAccount, remoteID string) string {
cfg, err := r.configOf(acct)
if err != nil {
return ""
}
var out map[string]any
if err := r.doGet(ctx, acct, cfg, "/Users/"+url.PathEscape(r.remoteUserID(cfg))+"/Items/"+url.PathEscape(remoteID), nil, &out); err != nil {
return ""
}
return remoteItemString(out, "SeriesId")
}
// ─── 远程 item JSON 取值辅助 ────────────────────────────────────────────────
func anyString(v any) string {
if s, ok := v.(string); ok {
return s
}
return ""
}
func remoteItemString(item map[string]any, key string) string {
if item == nil {
return ""
}
if s, ok := item[key].(string); ok {
return s
}
return ""
}
func remoteItemInt(item map[string]any, key string) int {
if item == nil {
return 0
}
switch v := item[key].(type) {
case float64:
return int(v)
case int:
return v
case int64:
return int(v)
case string:
n, _ := strconv.Atoi(v)
return n
}
return 0
}
func remoteItemInt64(item map[string]any, key string) int64 {
if item == nil {
return 0
}
switch v := item[key].(type) {
case float64:
return int64(v)
case int64:
return v
case int:
return int64(v)
case string:
n, _ := strconv.ParseInt(v, 10, 64)
return n
}
return 0
}
func remoteItemFloat(item map[string]any, key string) float64 {
if item == nil {
return 0
}
switch v := item[key].(type) {
case float64:
return v
case int:
return float64(v)
case string:
f, _ := strconv.ParseFloat(v, 64)
return f
}
return 0
}
// remoteItemGenres 合并 GenreItems / Genres 数组为逗号分隔字符串(前端 parseCSV 消费)。
func remoteItemGenres(item map[string]any) string {
seen := map[string]bool{}
var parts []string
collect := func(arr any) {
list, ok := arr.([]any)
if !ok {
return
}
for _, it := range list {
var name string
if m, isMap := it.(map[string]any); isMap {
name = remoteItemString(m, "Name")
} else if s, isStr := it.(string); isStr {
name = s
}
if name != "" && !seen[name] {
seen[name] = true
parts = append(parts, name)
}
}
}
collect(item["GenreItems"])
collect(item["Genres"])
return strings.Join(parts, ",")
}
// remoteItemImageURL 构造远程条目图片绝对地址(带 api_key;前端经 /api/img 代理)。
func (r *EmbyRemoteService) remoteItemImageURL(cfg *EmbyRemoteConfig, remoteID, imageType string) string {
if remoteID == "" {
return ""
}
imageType = strings.ToLower(imageType)
if imageType == "" {
imageType = "primary"
}
return r.embyBase(cfg) + "/Items/" + url.PathEscape(remoteID) + "/Images/" + url.PathEscape(imageType) +
"?api_key=" + url.QueryEscape(cfg.Token)
}
// remoteItemHasImageTag 远程 item 是否带某类型图片标签(Emby 的 ImageTags map)。
func remoteItemHasImageTag(item map[string]any, typ string) bool {
if item == nil {
return false
}
switch tags := item["ImageTags"].(type) {
case map[string]any:
_, ok := tags[typ]
return ok
case map[string]string:
_, ok := tags[typ]
return ok
}
return false
}
// remoteBackdropTags 远程 item 的 BackdropImageTags 数组。
func remoteBackdropTags(item map[string]any) []any {
if item == nil {
return nil
}
switch tags := item["BackdropImageTags"].(type) {
case []any:
return tags
case []string:
out := make([]any, 0, len(tags))
for _, s := range tags {
out = append(out, s)
}
return out
}
return nil
}
// remoteItemTypeOf 从映射后的 Media 推断远程类型(无详情载荷时兜底)。
func remoteItemTypeOf(m *model.Media) string {
if m == nil {
return "Movie"
}
if m.EpisodeNum > 0 || m.SeasonNum > 0 {
return "Episode"
}
return "Movie"
}
// ─── 供 handler 层使用的远程 View 条目取值(导出薄封装) ──────────────────────
// RemoteItemIDString 提取远程 View 条目的 Id。
func RemoteItemIDString(item map[string]any) string { return remoteItemString(item, "Id") }
// RemoteItemNameString 提取远程 View 条目的 Name。
func RemoteItemNameString(item map[string]any) string { return remoteItemString(item, "Name") }
// RemoteItemCollectionType 提取远程 View 条目的 CollectionType。
func RemoteItemCollectionType(item map[string]any) string {
return remoteItemString(item, "CollectionType")
}
// RemoteItemChildCount 提取远程 View 条目的 ChildCount。
func RemoteItemChildCount(item map[string]any) int { return remoteItemInt(item, "ChildCount") }
+79 -2
View File
@@ -124,7 +124,8 @@ func (e *EmbyService) userPayload(u *model.User) map[string]any {
}
}
// Views 返回 Emby 中"虚拟根目录"——每个 library 一个条目。
// Views 返回 Emby 中"虚拟根目录"——每个 library 一个条目,外加所有启用的
// 远程 Emby 挂载的媒体库(联邦聚合)。
func (e *EmbyService) Views(ctx context.Context, userID string) (map[string]any, error) {
libs, err := e.repo.Library.List(ctx)
if err != nil {
@@ -132,16 +133,92 @@ func (e *EmbyService) Views(ctx context.Context, userID string) (map[string]any,
}
libs = FilterDisplayCloudLibraries(ctx, e.repo, libs)
visibility := e.mediaVisibility(ctx, userID)
items := make([]map[string]any, 0, len(libs))
items := make([]map[string]any, 0, len(libs)+4)
for _, l := range libs {
if !e.libraryVisibleFromCachedVisibility(l, visibility) {
continue
}
items = append(items, e.libraryAsView(ctx, &l))
}
for _, remote := range e.remoteViews(ctx) {
items = append(items, remote)
}
return map[string]any{"Items": items, "TotalRecordCount": len(items), "StartIndex": 0}, nil
}
// remoteViews 返回全部启用挂载的远程媒体库视图(只有显式挂载的库才出现)。
func (e *EmbyService) remoteViews(ctx context.Context) []map[string]any {
if e == nil || e.remote == nil {
return nil
}
views, err := e.remote.RemoteLibraries(ctx)
if err != nil || len(views) == 0 {
return nil
}
out := make([]map[string]any, 0, len(views))
for _, v := range views {
out = append(out, remoteLibraryViewPayload(v))
}
return out
}
// remoteLibraryViewPayload 把挂载库展示信息标准化为 Emby View payload
// (ID 用挂载伪装,名称=挂载显示名)。
func remoteLibraryViewPayload(v RemoteLibraryView) map[string]any {
encoded := EncodeEmbyRemoteID(v.MountID, v.RemoteID)
collectionType := v.CollectionType
if !isSupportedEmbyCollectionType(collectionType) {
collectionType = "mixed"
}
name := strings.TrimSpace(v.Library.Name)
imageTags := map[string]string{}
if strings.TrimSpace(v.Library.CoverURL) != "" {
imageTags["Primary"] = encoded
}
return map[string]any{
"Id": encoded,
"Name": name,
"CollectionType": collectionType,
"ServerId": embyServerID,
"Type": "CollectionFolder",
"IsFolder": true,
"Path": "",
"SortName": strings.ToLower(name),
"DateCreated": time.Now().UTC().Format(time.RFC3339),
"CanDelete": false,
"CanDownload": false,
"DisplayPreferencesId": encoded,
"PrimaryImageItemId": encoded,
"PrimaryImageAspectRatio": 1.7777777777777777,
"RecursiveItemCount": 0,
"ChildCount": 0,
"SpecialFeatureCount": 0,
"EnableMediaSourceDisplay": true,
"PlayAccess": "Full",
"ExternalUrls": []any{},
"ProviderIds": map[string]string{},
"Genres": []string{},
"Tags": []string{},
"ImageTags": imageTags,
"BackdropImageTags": []string{},
"UserData": map[string]any{
"PlaybackPositionTicks": 0,
"PlayCount": 0,
"IsFavorite": false,
"Played": false,
"UnplayedItemCount": 0,
},
}
}
func isSupportedEmbyCollectionType(t string) bool {
switch t {
case "movies", "tvshows", "music", "mixed", "homevideos", "boxsets":
return true
}
return false
}
func (e *EmbyService) libraryAsView(ctx context.Context, l *model.Library) map[string]any {
collectionType := "movies"
switch l.Type {
+17 -1
View File
@@ -12,8 +12,16 @@ import (
"github.com/ShukeBta/MMTL/internal/model"
)
// SetFavorite 把 mediaID 标为 userID 的收藏。
// SetFavorite 把 mediaID 标为 userID 的收藏。远程 Emby 条目直接透传到对应
// 服务器(本地不落库)。
func (e *EmbyService) SetFavorite(ctx context.Context, userID, mediaID string, favorite bool) error {
if e.remote != nil && IsEmbyRemoteID(mediaID) {
acctID, remoteID, _ := DecodeEmbyRemoteID(mediaID)
if err := e.ProxyRemoteSetFavorite(ctx, acctID, remoteID, favorite); err != nil {
return err
}
return nil
}
if favorite {
var f model.Favorite
err := e.repo.DB.WithContext(ctx).
@@ -31,7 +39,15 @@ func (e *EmbyService) SetFavorite(ctx context.Context, userID, mediaID string, f
}
// MarkPlayed 把 mediaID 标为已看(写一个 100% 进度的 history 行)。
// 远程 Emby 条目直接透传到对应服务器(本地不落库)。
func (e *EmbyService) MarkPlayed(ctx context.Context, userID, mediaID string, played bool) error {
if e.remote != nil && IsEmbyRemoteID(mediaID) {
acctID, remoteID, _ := DecodeEmbyRemoteID(mediaID)
if err := e.ProxyRemoteSetPlayed(ctx, acctID, remoteID, played); err != nil {
return err
}
return nil
}
if !played {
return e.repo.DB.WithContext(ctx).
Where("user_id = ? AND media_id = ?", userID, mediaID).
+298 -136
View File
@@ -1,196 +1,358 @@
package service
import (
"archive/tar"
"context"
"fmt"
"io"
"net/http"
"os"
"os/exec"
"path/filepath"
"runtime"
"strings"
"time"
"github.com/ulikunitz/xz"
"go.uber.org/zap"
"github.com/ShukeBta/MMTL/internal/config"
)
// AutoInstallFFmpeg is only called by the admin tool-install endpoint. The
// server must not auto-download or keep ffmpeg/ffprobe running during startup.
func AutoInstallFFmpeg(log *zap.Logger, cfg *config.Config) (ffprobePath, ffmpegPath string) {
// 1. 优先使用配置 / PATH / 本机常见软件目录中的现有工具。
if path, err := resolveLocalExecutable(cfg.App.FFprobePath, "ffprobe"); err == nil {
ffprobePath = path
cfg.App.FFprobePath = path
log.Info("found local ffprobe", zap.String("path", path))
}
if path, err := resolveLocalExecutable(cfg.App.FFmpegPath, "ffmpeg"); err == nil {
ffmpegPath = path
cfg.App.FFmpegPath = path
log.Info("found local ffmpeg", zap.String("path", path))
}
if ffprobePath != "" || ffmpegPath != "" {
return ffprobePath, ffmpegPath
}
// ffmpegDownloadTarget 描述某个平台对应的官方构建下载源。
type ffmpegDownloadTarget struct {
Label string // 展示名,如 "Windows x86_64"
Kind string // 压缩包类型:zip / tar.xz
Archives []string // 依次尝试的下载地址(主源 + 备用源)
}
// 2. 检查默认安装位置。
defaultDir := getDefaultInstallDir()
ffprobeDefault := filepath.Join(defaultDir, "bin", "ffprobe.exe")
ffmpegDefault := filepath.Join(defaultDir, "bin", "ffmpeg.exe")
if _, err := os.Stat(ffprobeDefault); err == nil {
log.Info("在默认位置找到 ffprobe", zap.String("path", ffprobeDefault))
return ffprobeDefault, ffmpegDefault
}
// 3. 尝试自动安装。
log.Warn("未找到 ffmpeg/ffprobe,尝试自动安装...")
installed, err := tryAutoInstall(log, defaultDir)
if err != nil {
log.Error("自动安装失败,请手动安装 ffmpeg", zap.Error(err))
return "", ""
}
if installed {
if _, err := os.Stat(ffprobeDefault); err == nil {
log.Info("自动安装成功", zap.String("path", ffprobeDefault))
// 更新配置
updateConfigPaths(cfg, ffprobeDefault, ffmpegDefault)
return ffprobeDefault, ffmpegDefault
// ffmpegTargetForPlatform 按当前运行环境(OS+架构)选择下载源。
func ffmpegTargetForPlatform() (*ffmpegDownloadTarget, error) {
switch runtime.GOOS {
case "windows":
switch runtime.GOARCH {
case "amd64":
return &ffmpegDownloadTarget{
Label: "Windows x86_64",
Kind: "zip",
Archives: []string{
"https://www.gyan.dev/ffmpeg/builds/ffmpeg-release-essentials.zip",
"https://github.com/BtbN/FFmpeg-Builds/releases/download/latest/ffmpeg-master-latest-win64-gpl.zip",
},
}, nil
case "386":
return &ffmpegDownloadTarget{
Label: "Windows x86",
Kind: "zip",
Archives: []string{
"https://github.com/BtbN/FFmpeg-Builds/releases/download/latest/ffmpeg-master-latest-win32-gpl.zip",
},
}, nil
}
case "linux":
switch runtime.GOARCH {
case "amd64":
return &ffmpegDownloadTarget{
Label: "Linux x86_64",
Kind: "tar.xz",
Archives: []string{
"https://github.com/BtbN/FFmpeg-Builds/releases/download/latest/ffmpeg-master-latest-linux64-gpl.tar.xz",
"https://johnvansickle.com/ffmpeg/releases/ffmpeg-release-amd64-static.tar.xz",
},
}, nil
case "arm64":
return &ffmpegDownloadTarget{
Label: "Linux ARM64",
Kind: "tar.xz",
Archives: []string{
"https://github.com/BtbN/FFmpeg-Builds/releases/download/latest/ffmpeg-master-latest-linuxarm64-gpl.tar.xz",
"https://johnvansickle.com/ffmpeg/releases/ffmpeg-release-arm64-static.tar.xz",
},
}, nil
case "arm":
return &ffmpegDownloadTarget{
Label: "Linux ARM (32 位)",
Kind: "tar.xz",
Archives: []string{
"https://github.com/BtbN/FFmpeg-Builds/releases/download/latest/ffmpeg-master-latest-linuxarmhf-gpl.tar.xz",
},
}, nil
case "loong64":
return &ffmpegDownloadTarget{
Label: "Linux LoongArch64",
Kind: "tar.xz",
Archives: []string{
"https://github.com/BtbN/FFmpeg-Builds/releases/download/latest/ffmpeg-master-latest-linuxloongarch64-gpl.tar.xz",
},
}, nil
}
case "darwin":
switch runtime.GOARCH {
case "amd64":
return &ffmpegDownloadTarget{
Label: "macOS x86_64",
Kind: "zip",
Archives: []string{
"https://github.com/BtbN/FFmpeg-Builds/releases/download/latest/ffmpeg-master-latest-osx64-gpl.zip",
},
}, nil
case "arm64":
return &ffmpegDownloadTarget{
Label: "macOS Apple Silicon",
Kind: "zip",
Archives: []string{
"https://github.com/BtbN/FFmpeg-Builds/releases/download/latest/ffmpeg-master-latest-osxarm64-gpl.zip",
},
}, nil
}
}
return "", ""
return nil, fmt.Errorf("暂不支持自动下载的平台 %s/%s,请手动填写 ffmpeg/ffprobe 路径", runtime.GOOS, runtime.GOARCH)
}
// getDefaultInstallDir 返回默认安装目录
func getDefaultInstallDir() string {
exePath, err := os.Executable()
// installFFmpegTools 按平台下载并安装 ffmpeg/ffprobe 到 data/tools/ffmpeg/,
// 返回两个可执行文件的绝对路径。progress 用于回传阶段消息(UI 展示)。
func installFFmpegTools(ctx context.Context, log *zap.Logger, cfg *config.Config, progress func(string)) (ffmpegPath, ffprobePath string, err error) {
target, err := ffmpegTargetForPlatform()
if err != nil {
return "./tools/ffmpeg"
return "", "", err
}
exeDir := filepath.Dir(exePath)
return filepath.Join(exeDir, "tools", "ffmpeg")
}
// tryAutoInstall 尝试自动下载并安装 ffmpeg
func tryAutoInstall(log *zap.Logger, installDir string) (bool, error) {
if runtime.GOOS == "windows" {
return downloadFFmpegWindows(log, installDir)
}
return false, fmt.Errorf("不支持的操作系统: %s", runtime.GOOS)
}
// downloadFFmpegWindows 下载 Windows 版本的 ffmpeg
func downloadFFmpegWindows(log *zap.Logger, installDir string) (bool, error) {
log.Info("开始下载 ffmpeg...")
// 创建安装目录
installDir := filepath.Join(cfg.App.DataDir, "tools", "ffmpeg")
if err := os.MkdirAll(installDir, 0o750); err != nil {
return false, fmt.Errorf("创建安装目录失败: %w", err)
return "", "", fmt.Errorf("创建安装目录失败: %w", err)
}
tempDir, err := os.MkdirTemp("", "mmtl-ffmpeg-*")
if err != nil {
return false, fmt.Errorf("创建临时目录失败: %w", err)
return "", "", fmt.Errorf("创建临时目录失败: %w", err)
}
defer os.RemoveAll(tempDir)
// 下载 URL (使用 gyani.org 的静态构建)
arch := "win64"
if !is64Bit() {
arch = "win32"
progress("下载 " + target.Label + " 版本…")
archivePath := filepath.Join(tempDir, "ffmpeg-archive."+target.Kind)
if err := downloadFFmpegArchive(ctx, log, target.Archives, archivePath); err != nil {
return "", "", err
}
// 先尝试从 gyan.dev 下载(更可靠)
downloadURL := fmt.Sprintf("https://www.gyan.dev/ffmpeg/builds/ffmpeg-release-essentials.zip")
progress("解压…")
extractDir := filepath.Join(tempDir, "extract")
if err := extractFFmpegArchive(target.Kind, archivePath, extractDir); err != nil {
return "", "", fmt.Errorf("解压失败: %w", err)
}
log.Info("下载 ffmpeg", zap.String("url", downloadURL))
exeSuffix := ""
if runtime.GOOS == "windows" {
exeSuffix = ".exe"
}
srcFFmpeg, srcFFprobe, err := locateFFmpegBinaries(extractDir, exeSuffix)
if err != nil {
return "", "", err
}
// 使用 Go 下载
zipPath := filepath.Join(tempDir, "ffmpeg.zip")
if err := downloadFile(log, downloadURL, zipPath); err != nil {
// 尝试备用 URL
backupURL := fmt.Sprintf("https://github.com/BtbN/FFmpeg-Builds/releases/download/latest/ffmpeg-master-latest-%s-gpl.zip", arch)
log.Info("尝试备用下载地址", zap.String("url", backupURL))
if err2 := downloadFile(log, backupURL, zipPath); err2 != nil {
return false, fmt.Errorf("下载失败: %v, %v", err, err2)
progress("安装到 data 目录…")
ffmpegPath = filepath.Join(installDir, "ffmpeg"+exeSuffix)
ffprobePath = filepath.Join(installDir, "ffprobe"+exeSuffix)
if err := copyFileMode(srcFFmpeg, ffmpegPath); err != nil {
return "", "", fmt.Errorf("复制 ffmpeg 失败: %w", err)
}
if err := copyFileMode(srcFFprobe, ffprobePath); err != nil {
_ = os.Remove(ffmpegPath)
return "", "", fmt.Errorf("复制 ffprobe 失败: %w", err)
}
// 验证两个工具都能运行(失败则回滚,避免留下坏文件)。
for _, bin := range []string{ffmpegPath, ffprobePath} {
cmd := exec.CommandContext(ctx, bin, "-version") // #nosec G204 -- bin 是安装目录中刚写入的固定文件名。
if out, verr := cmd.Output(); verr != nil {
_ = os.Remove(ffmpegPath)
_ = os.Remove(ffprobePath)
return "", "", fmt.Errorf("安装后 %s 无法运行:%v", filepath.Base(bin), verr)
} else if log != nil {
log.Info("ffmpeg 工具安装验证通过", zap.String("bin", filepath.Base(bin)),
zap.String("version", strings.TrimSpace(strings.SplitN(string(out), "\n", 2)[0])))
}
}
// 解压
log.Info("解压 ffmpeg...")
extractDir := filepath.Join(tempDir, "extract")
if err := unzip(log, zipPath, extractDir); err != nil {
return false, fmt.Errorf("解压失败: %w", err)
}
packageRoot, err := findFFmpegPackageRoot(extractDir)
if err != nil {
return false, err
}
if err := copyDirContents(packageRoot, installDir); err != nil {
return false, fmt.Errorf("复制 ffmpeg 文件失败: %w", err)
}
ffmpegBin := filepath.Join(installDir, "bin", "ffmpeg.exe")
ffprobeBin := filepath.Join(installDir, "bin", "ffprobe.exe")
if _, err := os.Stat(ffmpegBin); err != nil {
return false, fmt.Errorf("安装后未找到 ffmpeg: %w", err)
}
if _, err := os.Stat(ffprobeBin); err != nil {
return false, fmt.Errorf("安装后未找到 ffprobe: %w", err)
}
log.Info("ffmpeg 安装完成", zap.String("dir", installDir))
return true, nil
progress("安装完成")
return ffmpegPath, ffprobePath, nil
}
// downloadFile 下载文件
func downloadFile(log *zap.Logger, url, filepath string) error {
ctx, cancel := context.WithTimeout(context.Background(), 5*time.Minute)
defer cancel()
// downloadFFmpegArchive 按顺序尝试下载源,全部失败才返回错误。
func downloadFFmpegArchive(ctx context.Context, log *zap.Logger, urls []string, dest string) error {
var lastErr error
for i, u := range urls {
if i > 0 && log != nil {
log.Warn("ffmpeg 主下载源不可用,切换备用源", zap.String("url", u))
}
if err := downloadFFmpegFile(ctx, log, u, dest); err != nil {
lastErr = err
if log != nil {
log.Warn("ffmpeg 下载失败", zap.String("url", u), zap.Error(err))
}
continue
}
return nil
}
return fmt.Errorf("所有下载源均失败:%v", lastErr)
}
req, err := http.NewRequestWithContext(ctx, "GET", url, nil)
// downloadFFmpegFile 下载单个归档文件(最多 10 分钟,限制大小上限)。
func downloadFFmpegFile(ctx context.Context, log *zap.Logger, url, dest string) error {
ctx, cancel := context.WithTimeout(ctx, 10*time.Minute)
defer cancel()
req, err := http.NewRequestWithContext(ctx, http.MethodGet, url, nil)
if err != nil {
return err
}
req.Header.Set("User-Agent", "MMTL/ffmpeg-installer ("+runtime.GOOS+"/"+runtime.GOARCH+")")
resp, err := http.DefaultClient.Do(req)
if err != nil {
return err
}
defer resp.Body.Close()
if resp.StatusCode != http.StatusOK {
return fmt.Errorf("下载失败,HTTP 状态码: %d", resp.StatusCode)
return fmt.Errorf("下载失败,HTTP 状态码 %d", resp.StatusCode)
}
out, err := os.Create(filepath) // #nosec G304 -- filepath is generated by the installer under its temporary download directory.
out, err := os.Create(dest) // #nosec G304 -- dest 是安装器在临时目录下生成的文件。
if err != nil {
return err
}
defer out.Close()
_, err = io.Copy(out, resp.Body)
return err
n, err := io.Copy(out, io.LimitReader(resp.Body, 500<<20+1)) // 归档上限 500MB
if err != nil {
return err
}
if n > 500<<20 {
return fmt.Errorf("归档文件过大(>500MB): %s", url)
}
if log != nil {
log.Info("ffmpeg 归档下载完成", zap.String("url", url), zap.Int64("bytes", n))
}
return nil
}
// updateConfigPaths 更新配置文件中的路径
func updateConfigPaths(cfg *config.Config, ffprobePath, ffmpegPath string) {
cfg.App.FFprobePath = ffprobePath
cfg.App.FFmpegPath = ffmpegPath
// 保存到配置文件
// 这里需要调用 config 包的保存函数
log := zap.L().Named("config")
log.Info("已更新 ffmpeg 路径配置",
zap.String("ffprobe", ffprobePath),
zap.String("ffmpeg", ffmpegPath))
// extractFFmpegArchive 按类型解压 zip 或 tar.xz。
func extractFFmpegArchive(kind, archivePath, destDir string) error {
switch kind {
case "zip":
return unzip(nil, archivePath, destDir)
case "tar.xz":
return untarXZ(archivePath, destDir)
default:
return fmt.Errorf("不支持的归档类型: %s", kind)
}
}
// is64Bit 检查是否为 64 位系统
func is64Bit() bool {
return true // 简化处理,假设为 64 位
// untarXZ 解压 .tar.xz 归档(GNU tar + xz 流式解压,纯 Go 无外部依赖),
// 路径安全校验与 ZIP 解压一致。
func untarXZ(archivePath, destDir string) error {
if err := os.MkdirAll(destDir, 0o750); err != nil {
return err
}
destRoot, err := filepath.Abs(destDir)
if err != nil {
return err
}
f, err := os.Open(archivePath) // #nosec G304 -- archivePath 是安装器在临时目录下生成的文件。
if err != nil {
return err
}
defer f.Close()
xzReader, err := xz.NewReader(f)
if err != nil {
return err
}
tr := tar.NewReader(xzReader)
var totalWritten int64
for {
hdr, err := tr.Next()
if err == io.EOF {
return nil
}
if err != nil {
return err
}
target, err := safeZipTarget(destRoot, hdr.Name) // 与 ZIP 相同的路径穿越防护
if err != nil {
return err
}
switch hdr.Typeflag {
case tar.TypeDir:
if err := os.MkdirAll(target, 0o750); err != nil {
return err
}
case tar.TypeReg, tar.TypeRegA:
if err := os.MkdirAll(filepath.Dir(target), 0o750); err != nil {
return err
}
dst, err := os.OpenFile(target, os.O_WRONLY|os.O_CREATE|os.O_TRUNC, os.FileMode(hdr.Mode).Perm())
if err != nil {
return err
}
written, copyErr := io.Copy(dst, io.LimitReader(tr, maxFFmpegZipEntryBytes+1))
totalWritten += written
closeErr := dst.Close()
if copyErr != nil {
return copyErr
}
if closeErr != nil {
return closeErr
}
if written > maxFFmpegZipEntryBytes || totalWritten > maxFFmpegZipTotalBytes {
return fmt.Errorf("tar 内容过大: %s", hdr.Name)
}
default:
// 符号链接/设备等一律跳过(静态构建不会依赖它们)。
continue
}
}
}
// locateFFmpegBinaries 在解压目录中查找 ffmpeg/ffprobe 可执行文件(兼容
// 不同构建包的目录布局:gyan 的 bin/、BtbN/johnvansickle 的根目录等)。
func locateFFmpegBinaries(root, exeSuffix string) (ffmpeg, ffprobe string, err error) {
wantFFmpeg := "ffmpeg" + strings.ToLower(exeSuffix)
wantFFprobe := "ffprobe" + strings.ToLower(exeSuffix)
err = filepath.WalkDir(root, func(path string, d os.DirEntry, err error) error {
if err != nil {
return err
}
if d.IsDir() {
return nil
}
switch strings.ToLower(d.Name()) {
case wantFFmpeg:
if ffmpeg == "" {
ffmpeg = path
}
case wantFFprobe:
if ffprobe == "" {
ffprobe = path
}
}
return nil
})
if err != nil {
return "", "", fmt.Errorf("扫描解压目录失败: %w", err)
}
if ffmpeg == "" || ffprobe == "" {
return "", "", fmt.Errorf("解压内容中未找到 ffmpeg/ffprobe 可执行文件")
}
return ffmpeg, ffprobe, nil
}
// copyFileMode 复制文件并赋予可执行权限。
func copyFileMode(src, dst string) error {
in, err := os.Open(src) // #nosec G304 -- src 来自解压目录遍历结果。
if err != nil {
return err
}
defer in.Close()
out, err := os.OpenFile(dst, os.O_WRONLY|os.O_CREATE|os.O_TRUNC, 0o755)
if err != nil {
return err
}
if _, err := io.Copy(out, in); err != nil {
_ = out.Close()
return err
}
return out.Close()
}
+3 -91
View File
@@ -41,7 +41,9 @@ func unzip(log *zap.Logger, zipPath, destDir string) error {
}
info := file.FileInfo()
if info.Mode()&os.ModeSymlink != 0 {
log.Warn("跳过 ZIP 符号链接", zap.String("name", file.Name))
if log != nil {
log.Warn("跳过 ZIP 符号链接", zap.String("name", file.Name))
}
continue
}
if info.IsDir() {
@@ -110,93 +112,3 @@ func safeZipTarget(destRoot, name string) (string, error) {
return targetAbs, nil
}
func findFFmpegPackageRoot(root string) (string, error) {
var ffmpegPath string
var ffprobePath string
err := filepath.WalkDir(root, func(path string, d os.DirEntry, err error) error {
if err != nil {
return err
}
if d.IsDir() {
return nil
}
switch strings.ToLower(d.Name()) {
case "ffmpeg.exe":
ffmpegPath = path
case "ffprobe.exe":
ffprobePath = path
}
return nil
})
if err != nil {
return "", fmt.Errorf("扫描解压目录失败: %w", err)
}
if ffmpegPath == "" || ffprobePath == "" {
return "", fmt.Errorf("解压后未找到 ffmpeg/ffprobe 可执行文件")
}
return filepath.Dir(filepath.Dir(ffmpegPath)), nil
}
func copyDirContents(srcDir, dstDir string) error {
entries, err := os.ReadDir(srcDir)
if err != nil {
return err
}
for _, entry := range entries {
srcPath := filepath.Join(srcDir, entry.Name())
dstPath := filepath.Join(dstDir, entry.Name())
if err := copyTree(srcPath, dstPath); err != nil {
return err
}
}
return nil
}
func copyTree(srcPath, dstPath string) error {
info, err := os.Stat(srcPath)
if err != nil {
return err
}
if info.IsDir() {
if err := os.MkdirAll(dstPath, info.Mode()); err != nil {
return err
}
entries, err := os.ReadDir(srcPath)
if err != nil {
return err
}
for _, entry := range entries {
if err := copyTree(filepath.Join(srcPath, entry.Name()), filepath.Join(dstPath, entry.Name())); err != nil {
return err
}
}
return nil
}
in, err := os.Open(srcPath) // #nosec G304 -- srcPath is produced by walking the validated extracted ffmpeg package tree.
if err != nil {
return err
}
defer in.Close()
if err := os.MkdirAll(filepath.Dir(dstPath), 0o750); err != nil {
return err
}
out, err := os.Create(dstPath) // #nosec G304 -- dstPath is generated under the configured ffmpeg install directory.
if err != nil {
return err
}
defer out.Close()
if _, err := io.Copy(out, in); err != nil {
return err
}
return out.Close()
}
-57
View File
@@ -1,57 +0,0 @@
package service
import (
"bytes"
"os"
"os/exec"
"runtime"
)
// CheckFFmpegStatus 检查 ffmpeg/ffprobe 状态 (供 API 使用)
func CheckFFmpegStatus(ffprobePath, ffmpegPath string) map[string]interface{} {
status := map[string]interface{}{
"ffprobe_installed": false,
"ffmpeg_installed": false,
"auto_installable": runtime.GOOS == "windows",
}
if ffprobePath != "" {
if _, err := os.Stat(ffprobePath); err == nil {
status["ffprobe_installed"] = true
status["ffprobe_path"] = ffprobePath
// 获取版本
cmd := exec.Command(ffprobePath, "-version")
out, err := cmd.Output()
if err == nil {
// 提取版本信息(第一行)
lines := bytes.Split(out, []byte("\n"))
if len(lines) > 0 {
version := string(bytes.TrimSpace(lines[0]))
status["ffprobe_version"] = version
status["ffprobe_security"] = EvaluateFFmpegSecurity(version)
}
}
}
}
if ffmpegPath != "" {
if _, err := os.Stat(ffmpegPath); err == nil {
status["ffmpeg_installed"] = true
status["ffmpeg_path"] = ffmpegPath
cmd := exec.Command(ffmpegPath, "-version")
out, err := cmd.Output()
if err == nil {
lines := bytes.Split(out, []byte("\n"))
if len(lines) > 0 {
version := string(bytes.TrimSpace(lines[0]))
status["ffmpeg_version"] = version
status["ffmpeg_security"] = EvaluateFFmpegSecurity(version)
}
}
}
}
return status
}
+211
View File
@@ -0,0 +1,211 @@
// Package service — ffmpeg/ffprobe 自动下载安装。
//
// FFmpegToolsService 负责「一键下载」:点击后按当前运行平台(OS+架构)选择
// 官方构建包(Windows: gyan.dev / BtbN;Linux: BtbN / johnvansickle;
// macOS: BtbN),下载解压 ffmpeg/ffprobe 到 data 目录(data/tools/ffmpeg/),
// 并把绝对路径写入设置(ffmpeg.path / ffprobe.path),系统随即使用安装的
// 工具,无需手动填写路径。
package service
import (
"context"
"errors"
"os"
"os/exec"
"path/filepath"
"runtime"
"strings"
"sync"
"time"
"go.uber.org/zap"
"github.com/ShukeBta/MMTL/internal/config"
"github.com/ShukeBta/MMTL/internal/repository"
)
// FFmpegToolsService 管理 ffmpeg/ffprobe 的自动下载安装状态(单飞:同一时间
// 只允许一个安装任务)。
type FFmpegToolsService struct {
cfg *config.Config
log *zap.Logger
repo *repository.Container
mu sync.Mutex
running bool
msg string // 最近阶段/结果消息
errMsg string // 最近一次失败原因
started time.Time // 最近一次安装开始时间
done time.Time // 最近一次安装结束时间
}
// NewFFmpegToolsService 构造工具安装服务。
func NewFFmpegToolsService(cfg *config.Config, log *zap.Logger, repo *repository.Container) *FFmpegToolsService {
return &FFmpegToolsService{cfg: cfg, log: log, repo: repo}
}
// ffmpegInstallDir 返回 data 目录下的安装位置。
func (s *FFmpegToolsService) ffmpegInstallDir() string {
return filepath.Join(s.cfg.App.DataDir, "tools", "ffmpeg")
}
// installedBinaries 检查安装目录中是否已存在 ffmpeg/ffprobe 可执行文件。
func (s *FFmpegToolsService) installedBinaries() (ffmpeg, ffprobe string) {
exe := ""
if runtime.GOOS == "windows" {
exe = ".exe"
}
ffmpeg = filepath.Join(s.ffmpegInstallDir(), "ffmpeg"+exe)
ffprobe = filepath.Join(s.ffmpegInstallDir(), "ffprobe"+exe)
if _, err := os.Stat(ffmpeg); err != nil {
return "", ""
}
if _, err := os.Stat(ffprobe); err != nil {
return "", ""
}
return ffmpeg, ffprobe
}
// ffToolVersion 取工具第一行版本信息。
func ffToolVersion(path string) string {
if path == "" {
return ""
}
out, err := exec.Command(path, "-version").Output() // #nosec G204 -- path 来自配置/安装目录中的已知工具。
if err != nil {
return ""
}
line := strings.SplitN(strings.TrimSpace(string(out)), "\n", 2)
if len(line) == 0 || strings.TrimSpace(line[0]) == "" {
return ""
}
return strings.TrimSpace(line[0])
}
// ffToolInfo 是单个工具的安装状态(返回给前端展示)。
type ffToolInfo struct {
Installed bool `json:"installed"`
Path string `json:"path,omitempty"`
Version string `json:"version,omitempty"`
}
// Status 返回当前安装状态(供 GET /api/admin/tools/ffmpeg/status 使用)。
func (s *FFmpegToolsService) Status(ctx context.Context) map[string]any {
s.mu.Lock()
running, msg, errMsg, started, done := s.running, s.msg, s.errMsg, s.started, s.done
s.mu.Unlock()
startedAt, doneAt := "", ""
if !started.IsZero() {
startedAt = started.Format(time.RFC3339)
}
if !done.IsZero() {
doneAt = done.Format(time.RFC3339)
}
out := map[string]any{
"installing": running,
"message": msg,
"error": errMsg,
"started_at": startedAt,
"finished_at": doneAt,
"install_dir": s.ffmpegInstallDir(),
}
target, targetErr := ffmpegTargetForPlatform()
if targetErr != nil {
out["target"] = map[string]any{"label": targetErr.Error()}
} else {
out["target"] = map[string]any{
"os": runtime.GOOS,
"arch": runtime.GOARCH,
"label": target.Label,
}
}
// 报告「系统当前实际会使用」的工具:优先已生效配置(安装完成会把设置指到
// data 目录),其次 PATH / 常见目录。
ffmpegPath, ferr := resolveLocalExecutable(s.cfg.App.FFmpegPath, "ffmpeg")
ffprobePath, perr := resolveLocalExecutable(s.cfg.App.FFprobePath, "ffprobe")
out["ffmpeg"] = ffToolInfo{Installed: ferr == nil, Path: ffmpegPath, Version: ffToolVersion(ffmpegPath)}
out["ffprobe"] = ffToolInfo{Installed: perr == nil, Path: ffprobePath, Version: ffToolVersion(ffprobePath)}
return out
}
// StartInstall 启动后台安装(幂等)。正在安装时返回错误;data 目录已有完整
// 工具时直接应用路径设置并返回(无需重新下载)。
func (s *FFmpegToolsService) StartInstall(ctx context.Context) error {
s.mu.Lock()
if s.running {
s.mu.Unlock()
return errors.New("工具正在安装中,请稍候")
}
if ffmpeg, ffprobe := s.installedBinaries(); ffmpeg != "" && ffprobe != "" {
s.mu.Unlock()
s.setMessage("检测到已安装,直接应用配置")
if err := s.applyInstalledPaths(ctx, ffmpeg, ffprobe); err != nil {
return err
}
return nil
}
s.running = true
s.errMsg = ""
s.started = time.Now()
s.mu.Unlock()
s.setMessage("准备下载…")
go s.runInstall()
return nil
}
// runInstall 在后台执行下载、解压、验证与配置落盘。
func (s *FFmpegToolsService) runInstall() {
defer func() {
s.mu.Lock()
s.running = false
s.done = time.Now()
s.mu.Unlock()
}()
ctx, cancel := context.WithTimeout(context.Background(), 15*time.Minute)
defer cancel()
ffmpegPath, ffprobePath, err := installFFmpegTools(ctx, s.log, s.cfg, s.setMessage)
if err != nil {
s.mu.Lock()
s.errMsg = err.Error()
s.msg = "安装失败"
s.mu.Unlock()
s.log.Error("ffmpeg 工具安装失败", zap.Error(err))
return
}
if err := s.applyInstalledPaths(ctx, ffmpegPath, ffprobePath); err != nil {
s.mu.Lock()
s.errMsg = "安装完成,但写入设置失败:" + err.Error()
s.msg = "安装完成,设置写入失败"
s.mu.Unlock()
s.log.Error("写入 ffmpeg 工具路径设置失败", zap.Error(err))
return
}
s.setMessage("安装完成")
s.log.Info("ffmpeg 工具安装完成",
zap.String("ffmpeg", ffmpegPath), zap.String("ffprobe", ffprobePath))
}
// applyInstalledPaths 把安装后的路径写入设置表并热应用到运行配置。
func (s *FFmpegToolsService) applyInstalledPaths(ctx context.Context, ffmpeg, ffprobe string) error {
if s.repo != nil && s.repo.Setting != nil {
if err := s.repo.Setting.Set(ctx, "ffmpeg.path", ffmpeg); err != nil {
return err
}
if err := s.repo.Setting.Set(ctx, "ffprobe.path", ffprobe); err != nil {
return err
}
}
ApplyRuntimeSetting(s.cfg, "ffmpeg.path", ffmpeg)
ApplyRuntimeSetting(s.cfg, "ffprobe.path", ffprobe)
return nil
}
func (s *FFmpegToolsService) setMessage(msg string) {
s.mu.Lock()
s.msg = msg
s.mu.Unlock()
}
+81
View File
@@ -0,0 +1,81 @@
package service
import (
"context"
"runtime"
"strings"
"testing"
"go.uber.org/zap"
"github.com/ShukeBta/MMTL/internal/config"
)
func TestFFmpegTargetForPlatform(t *testing.T) {
target, err := ffmpegTargetForPlatform()
switch runtime.GOOS {
case "windows", "linux", "darwin":
if err != nil {
t.Fatalf("supported platform %s/%s should resolve a target: %v", runtime.GOOS, runtime.GOARCH, err)
}
if target == nil || target.Label == "" || len(target.Archives) == 0 {
t.Fatalf("target incomplete: %#v", target)
}
if target.Kind != "zip" && target.Kind != "tar.xz" {
t.Fatalf("unexpected archive kind: %s", target.Kind)
}
for _, u := range target.Archives {
if !strings.HasPrefix(u, "https://") {
t.Fatalf("archive url not https: %s", u)
}
}
default:
if err == nil {
t.Fatalf("unsupported platform %s/%s should fail", runtime.GOOS, runtime.GOARCH)
}
}
}
func TestSafeZipTargetRejectsTraversal(t *testing.T) {
root := t.TempDir()
for _, name := range []string{"../evil", "..\\evil", "/etc/passwd", "a/../../evil"} {
if _, err := safeZipTarget(root, name); err == nil {
t.Fatalf("expected traversal rejection for %q", name)
}
}
if _, err := safeZipTarget(root, "bin/ffmpeg.exe"); err != nil {
t.Fatalf("valid relative path should pass: %v", err)
}
}
func TestFFmpegToolsStatusNoPanic(t *testing.T) {
svc := NewFFmpegToolsService(&config.Config{}, zap.NewNop(), nil)
st := svc.Status(context.Background())
for _, key := range []string{"installing", "message", "error", "install_dir", "target", "ffmpeg", "ffprobe"} {
if _, ok := st[key]; !ok {
t.Fatalf("status missing key %q: %#v", key, st)
}
}
ffmpeg, ok := st["ffmpeg"].(ffToolInfo)
if !ok {
t.Fatalf("ffmpeg field not ffToolInfo: %T", st["ffmpeg"])
}
if ffmpeg.Installed {
t.Fatalf("empty config should not report installed ffmpeg: %#v", st)
}
}
func TestStartInstallRejectsConcurrent(t *testing.T) {
svc := NewFFmpegToolsService(&config.Config{App: config.AppConfig{DataDir: t.TempDir()}}, zap.NewNop(), nil)
// 不真实运行:直接占用 running 标记模拟进行中的安装。
svc.mu.Lock()
svc.running = true
svc.mu.Unlock()
if err := svc.StartInstall(context.Background()); err == nil {
t.Fatalf("second install while running should be rejected")
}
svc.mu.Lock()
svc.running = false
svc.mu.Unlock()
}
+77 -21
View File
@@ -49,7 +49,7 @@ func findMovieNFO(mediaPath, libraryRoot string) (*nfoDocument, string, error) {
continue
}
seen[key] = struct{}{}
if doc, _, err := readNFO(path); err == nil {
if doc, _, err := decodeNFOFile(path); err == nil && doc != nil {
return doc, path, nil
} else if err != nil && !errors.Is(err, os.ErrNotExist) {
return nil, "", err
@@ -62,7 +62,7 @@ func findMovieNFO(mediaPath, libraryRoot string) (*nfoDocument, string, error) {
for _, match := range matches {
baseKey := strings.ToLower(strings.ReplaceAll(strings.TrimSuffix(filepath.Base(match), filepath.Ext(match)), "-", ""))
if strings.Contains(baseKey, codeKey) || strings.Contains(codeKey, baseKey) {
if doc, _, err := readNFO(match); err == nil {
if doc, _, err := decodeNFOFile(match); err == nil && doc != nil {
return doc, match, nil
} else if err != nil && !errors.Is(err, os.ErrNotExist) {
return nil, "", err
@@ -71,7 +71,7 @@ func findMovieNFO(mediaPath, libraryRoot string) (*nfoDocument, string, error) {
}
}
if len(matches) == 1 {
if doc, _, err := readNFO(matches[0]); err == nil {
if doc, _, err := decodeNFOFile(matches[0]); err == nil && doc != nil {
return doc, matches[0], nil
} else if err != nil && !errors.Is(err, os.ErrNotExist) {
return nil, "", err
@@ -84,21 +84,33 @@ func findMovieNFO(mediaPath, libraryRoot string) (*nfoDocument, string, error) {
func readSeriesMetadata(mediaPath, libraryRoot string) (*LocalMetadata, error) {
var meta *LocalMetadata
showBaseDir := ""
if showDoc, showPath, err := findShowNFO(mediaPath, libraryRoot); err == nil && showDoc != nil {
if showDoc, showPath, partial, err := findShowNFO(mediaPath, libraryRoot); err == nil && showDoc != nil {
showBaseDir = filepath.Dir(showPath)
meta = metadataFromDoc(showDoc, showBaseDir, true)
// A truncated show NFO still yields a usable title/poster (decodePartialNFO
// only surfaces docs with recoverable fields). Keep HasNFO so the recovered
// title participates in series grouping; a fully partial episode match
// below can still demote it.
meta.HasNFO = !partial || meta.HasNFO
} else if err != nil && !errors.Is(err, os.ErrNotExist) {
return nil, err
// A damaged show NFO must not discard the whole series: readLocalScanMetadata
// treats this as a hard failure and leaves every episode without local
// metadata. When the show level fails we still merge the episode NFO and
// local artwork below instead of returning early.
meta = nil
}
if episodeDoc, episodePath, err := readNFO(nfoPath(mediaPath)); err == nil {
if episodeDoc, episodePath, episodePartial, err := readEpisodeNFO(nfoPath(mediaPath)); err == nil && episodeDoc != nil {
episodeMeta := metadataFromDoc(episodeDoc, filepath.Dir(episodePath), true)
if meta == nil {
meta = &LocalMetadata{}
}
mergeEpisodeMetadata(meta, episodeMeta, episodeDoc)
} else if err != nil && !errors.Is(err, os.ErrNotExist) {
return nil, err
// Only a fully valid episode NFO can confirm the match; a truncated one
// keeps whatever the show NFO supplied but must not force scrape_status.
if episodePartial {
meta.HasNFO = false
}
}
if meta == nil {
meta = metadataFromArtwork(mediaPath, showBaseDir)
@@ -108,19 +120,49 @@ func readSeriesMetadata(mediaPath, libraryRoot string) (*LocalMetadata, error) {
return meta, nil
}
func readNFO(path string) (*nfoDocument, string, error) {
// readEpisodeNFO reads the sidecar NFO next to an episode file and reports both
// the recovered document and whether it came from a truncated/malformed file.
func readEpisodeNFO(path string) (*nfoDocument, string, bool, error) {
body, err := os.ReadFile(path) // #nosec G304 -- path is a discovered NFO sidecar under the configured library root.
if err != nil {
return nil, "", err
return nil, "", false, err
}
var doc nfoDocument
if err := xml.Unmarshal(body, &doc); err != nil {
return nil, "", err
doc, partial, err := decodePartialNFO(body)
if err != nil {
return nil, "", false, err
}
return &doc, path, nil
return doc, path, partial, nil
}
func findShowNFO(mediaPath, libraryRoot string) (*nfoDocument, string, error) {
func decodePartialNFO(body []byte) (*nfoDocument, bool, error) {
var doc nfoDocument
err := xml.Unmarshal(body, &doc)
if err == nil {
return &doc, false, nil
}
if !isLikelyTruncatedXMLError(err) {
// A real parse error unrelated to truncation: don't trust partial fields.
return nil, false, err
}
// Unmarshal still fills the elements it closed before hitting the cut; treat
// those as partial show metadata instead of dropping everything.
if doc.Title == "" && doc.OriginalTitle == "" && len(doc.Thumbs) == 0 &&
doc.Premiered == "" && doc.Plot == "" {
return nil, false, err
}
return &doc, true, nil
}
func isLikelyTruncatedXMLError(err error) bool {
for _, msg := range []string{"unexpected EOF", "EOF"} {
if strings.Contains(strings.ToLower(err.Error()), strings.ToLower(msg)) {
return true
}
}
return false
}
func findShowNFO(mediaPath, libraryRoot string) (*nfoDocument, string, bool, error) {
dir := filepath.Dir(mediaPath)
root := filepath.Clean(libraryRoot)
for {
@@ -133,19 +175,33 @@ func findShowNFO(mediaPath, libraryRoot string) (*nfoDocument, string, error) {
names = append(names, base+".nfo")
for _, name := range names {
path := filepath.Join(dir, name)
if doc, _, err := readNFO(path); err == nil {
return doc, path, nil
} else if err != nil && !errors.Is(err, os.ErrNotExist) {
return nil, "", err
doc, partial, err := decodeNFOFile(path)
if doc != nil {
return doc, path, partial, nil
}
if err != nil && !errors.Is(err, os.ErrNotExist) {
return nil, "", false, err
}
}
if samePath(dir, root) {
return nil, "", os.ErrNotExist
return nil, "", false, os.ErrNotExist
}
parent := filepath.Dir(dir)
if parent == dir {
return nil, "", os.ErrNotExist
return nil, "", false, os.ErrNotExist
}
dir = parent
}
}
func decodeNFOFile(path string) (*nfoDocument, bool, error) {
body, err := os.ReadFile(path) // #nosec G304 -- path is a discovered NFO sidecar under the configured library root.
if err != nil {
return nil, false, err
}
doc, partial, err := decodePartialNFO(body)
if err != nil {
return nil, false, err
}
return doc, partial, nil
}
+108
View File
@@ -267,3 +267,111 @@ func TestReadLocalMetadataWithoutNFOStillFindsArtwork(t *testing.T) {
t.Fatalf("unexpected artwork metadata: %+v", got)
}
}
// TestReadLocalMetadataRecoversFromTruncatedShowNFO mirrors a real-world issue:
// some anime tvshow.nfo files are truncated mid-URL (unexpected EOF). Before the
// fix this discarded the whole series, leaving episodes pending with per-episode
// titles and no poster. The recoverable fields (title/year) and the matching
// episode NFO + local artwork must still be applied.
func TestReadLocalMetadataRecoversFromTruncatedShowNFO(t *testing.T) {
root := t.TempDir()
showDir := filepath.Join(root, "夏日重现 (2022)")
seasonDir := filepath.Join(showDir, "Season 1")
if err := os.MkdirAll(seasonDir, 0o755); err != nil {
t.Fatal(err)
}
mediaPath := filepath.Join(seasonDir, "S01E01.mkv")
if err := os.WriteFile(mediaPath, []byte("x"), 0o644); err != nil {
t.Fatal(err)
}
// tvshow.nfo cut off inside a <thumb> URL, like the broken real files.
truncated := `<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<tvshow>
<title>夏日重现</title>
<originaltitle>サマータイムレンダ</originaltitle>
<year>2022</year>
<plot>听闻自己青梅竹马死讯。</plot>
<thumb aspect="poster">https://image.tmdb.org/t/p/original/2koyWLm6iVn5OTEExTjKzVms5Iz.jpg</thumb>
<fanart>
<thumb>https://image.tmdb.org/t/p/original/p2eZlGwd8OjkWpwD2hSoBiIlHBZ.jpg</thu`
if err := os.WriteFile(filepath.Join(showDir, "tvshow.nfo"), []byte(truncated), 0o644); err != nil {
t.Fatal(err)
}
// The episode sidecar NFO is complete and should still be merged.
if err := os.WriteFile(nfoPath(mediaPath), []byte(`<episodedetails><title>再见了夏日</title><season>1</season><episode>1</episode></episodedetails>`), 0o644); err != nil {
t.Fatal(err)
}
// Local artwork next to the show folder.
poster := filepath.Join(showDir, "poster.jpg")
if err := os.WriteFile(poster, []byte("jpg"), 0o644); err != nil {
t.Fatal(err)
}
got, err := ReadLocalMetadata(mediaPath, root, true)
if err != nil {
t.Fatalf("ReadLocalMetadata returned error on truncated show NFO: %v", err)
}
if got == nil {
t.Fatal("metadata is nil; truncated show NFO discarded the series")
}
// The recovered show title takes precedence over the episode title.
if got.Title != "夏日重现" {
t.Fatalf("Title = %q, want recovered show title 夏日重现", got.Title)
}
if got.Year != 2022 {
t.Fatalf("Year = %d, want 2022", got.Year)
}
if got.EpisodeTitle != "再见了夏日" || got.SeasonNum != 1 || got.EpisodeNum != 1 {
t.Fatalf("episode metadata not preserved: %+v", got)
}
// Prior to the fix the episode metadata was dropped entirely; the poster comes
// from the local poster.jpg next to the show.
if got.PosterURL != poster {
t.Fatalf("PosterURL = %q, want local poster %q", got.PosterURL, poster)
}
// A truncated show NFO must not be treated as an authoritative match by
// itself; since the episode NFO is valid we still mark it matched so the
// recovered series title participates in grouping.
if !got.HasNFO {
t.Fatalf("HasNFO = false, want true (episode NFO is valid)")
}
}
// TestReadLocalMetadataKeepsArtworkOnlyWhenNoUsableNFO verifies that when a show
// NFO is truncated AND yields no recoverable fields, we still fall back to local
// artwork instead of returning an error.
func TestReadLocalMetadataArtworkFallbackOnGarbageShowNFO(t *testing.T) {
root := t.TempDir()
showDir := filepath.Join(root, "Some Show")
seasonDir := filepath.Join(showDir, "Season 1")
if err := os.MkdirAll(seasonDir, 0o755); err != nil {
t.Fatal(err)
}
mediaPath := filepath.Join(seasonDir, "S01E01.mkv")
if err := os.WriteFile(mediaPath, []byte("x"), 0o644); err != nil {
t.Fatal(err)
}
// Severely truncated: no recoverable title/fields at all.
if err := os.WriteFile(filepath.Join(showDir, "tvshow.nfo"), []byte(`<tvshow><title>半截`), 0o644); err != nil {
t.Fatal(err)
}
// Episode-level backdrop sits next to the episode file.
backdrop := filepath.Join(seasonDir, "S01E01-backdrop.jpg")
if err := os.WriteFile(backdrop, []byte("jpg"), 0o644); err != nil {
t.Fatal(err)
}
got, err := ReadLocalMetadata(mediaPath, root, true)
if err != nil {
t.Fatalf("ReadLocalMetadata should not error when NFO is unusable: %v", err)
}
if got == nil {
t.Fatal("metadata is nil; artwork fallback missing")
}
if got.HasNFO {
t.Fatalf("HasNFO = true, want false for unusable NFO")
}
if got.BackdropURL != backdrop {
t.Fatalf("expected episode artwork fallback, got %+v", got)
}
}
+8 -5
View File
@@ -32,18 +32,21 @@ func (s *ScraperService) manualTMDbCandidates(ctx context.Context, query string,
for _, typ := range manualTMDbSearchTypes(mediaType) {
switch typ {
case "movie":
if matches, err := s.tmdb.SearchMovieCandidates(ctx, query, year); err == nil {
if matches, err := s.tmdb.SearchMovieCandidates(ctx, query, year); err == nil && len(matches) > 0 {
for _, match := range matches {
out = append(out, manualTMDbCandidate{MediaType: "movie", Match: match})
}
}
case "tv":
if matches, err := s.tmdb.SearchTVCandidates(ctx, query, year); err == nil {
if matches, err := s.tmdb.SearchTVCandidates(ctx, query, year); err == nil && len(matches) > 0 {
for _, match := range matches {
out = append(out, manualTMDbCandidate{MediaType: "tv", Match: match})
}
}
}
if len(out) > 0 {
break
}
}
return out
}
@@ -70,15 +73,15 @@ func manualTMDbIDSearchTypes(mediaType string) []string {
func manualTMDbSearchTypes(mediaType string) []string {
if strings.TrimSpace(mediaType) == "" {
return []string{"movie", "tv"}
return []string{"tv", "movie"}
}
switch normalizeMediaType(mediaType, "", "") {
case "tv", "anime", "variety":
return []string{"tv", "movie"}
case "movie", "adult":
return []string{"movie"}
default:
return []string{"movie", "tv"}
default:
return []string{"tv", "movie"}
}
}
+3 -3
View File
@@ -179,9 +179,9 @@ func TestManualSearchFallsBackToMovieFolderForGenericQuery(t *testing.T) {
if len(results) != 1 || results[0].TMDbID != 27205 {
t.Fatalf("manual search results=%#v, want folder fallback candidate; queries=%v", results, queries)
}
if len(queries) < 2 || queries[0] != "00000" || queries[1] != "inception" {
t.Fatalf("manual search queries=%v, want explicit query then folder fallback", queries)
}
if len(queries) < 2 || queries[0] != "00000" || queries[len(queries)-1] != "inception" {
t.Fatalf("manual search queries=%v, want explicit query then folder fallback", queries)
}
}
func TestManualSearchReturnsMovieFallbackForTVTypedTMDbSearch(t *testing.T) {
+74
View File
@@ -4,14 +4,88 @@ import (
"context"
"github.com/ShukeBta/MMTL/internal/model"
"github.com/ShukeBta/MMTL/internal/repository"
"gorm.io/gorm"
)
type LibraryPreviewItem struct {
model.Library
Total int64 `json:"total"`
Cards []SeriesCard `json:"cards"`
}
// ListLibraries returns every library configured on the server.
func (s *MediaService) ListLibraries(ctx context.Context) ([]model.Library, error) {
return s.repo.Library.List(ctx)
}
// ListLibrariesWithPreview returns libraries populated with item counts and latest preview cards.
func (s *MediaService) ListLibrariesWithPreview(ctx context.Context, libraries []model.Library, visibility MediaVisibility, cardLimit int) ([]LibraryPreviewItem, error) {
if cardLimit <= 0 {
cardLimit = 10
}
out := make([]LibraryPreviewItem, len(libraries))
if len(libraries) == 0 {
return out, nil
}
libIDs := make([]string, 0, len(libraries))
for i, lib := range libraries {
out[i] = LibraryPreviewItem{
Library: lib,
Total: 0,
Cards: []SeriesCard{},
}
libIDs = append(libIDs, lib.ID)
}
visibility = ExpandMediaVisibilityForMergedCloudLibraries(ctx, s.repo, visibility)
filter := repository.MediaQueryFilter{
IncludeNSFW: visibility.IncludeNSFW,
AllowedLibraryIDs: visibility.AllowedLibraryIDs,
HiddenLibraryIDs: visibility.HiddenLibraryIDs,
}
counts, err := s.repo.Media.CountByLibraries(ctx, libIDs, filter)
if err != nil {
return nil, err
}
for i := range out {
if total, ok := counts[out[i].ID]; ok {
out[i].Total = total
}
}
fetchCount := cardLimit * 4
if fetchCount < 60 {
fetchCount = 60
} else if fetchCount > 200 {
fetchCount = 200
}
for i := range out {
if out[i].Total == 0 {
continue
}
items, _, err := s.repo.Media.ListByLibrariesFiltered(ctx, []string{out[i].ID}, 0, fetchCount, filter)
if err != nil {
continue
}
s.attachLibraryMetadata(ctx, items)
cards := groupMediaSeriesCards(items)
if len(cards) > cardLimit {
cards = cards[:cardLimit]
}
if cards == nil {
cards = []SeriesCard{}
}
out[i].Cards = cards
}
return out, nil
}
// DeleteLibrary removes a library and its media rows. The on-disk files are
// left untouched.
func (s *MediaService) DeleteLibrary(ctx context.Context, id string) error {
+70 -1
View File
@@ -4,6 +4,7 @@ import (
"context"
"errors"
"fmt"
"os"
"path/filepath"
"strings"
@@ -51,7 +52,14 @@ func (s *MediaService) CreateLibraryWithRootsAndCover(ctx context.Context, name,
s.invalidateMediaCache(ctx)
return lib, nil
}
lib := &model.Library{Name: strings.TrimSpace(name), Path: roots[0].Path, Type: kind, CoverURL: strings.TrimSpace(coverURL), Enabled: true}
lib := &model.Library{
Name: strings.TrimSpace(name),
Path: roots[0].Path,
Type: kind,
CoverURL: strings.TrimSpace(coverURL),
Enabled: true,
CarouselEnabled: false,
}
if err := s.repo.Library.CreateWithRoots(ctx, lib, roots); err != nil {
return nil, err
}
@@ -59,11 +67,72 @@ func (s *MediaService) CreateLibraryWithRootsAndCover(ctx context.Context, name,
return lib, nil
}
// CreateLibrariesPerSubfolder 为 parent 目录下的每个直接子目录各建一个媒体库,
// 媒体库名取子目录名,路径指向该子目录。kind 为空时按子目录名推断类型。
func (s *MediaService) CreateLibrariesPerSubfolder(ctx context.Context, parent, kind, coverURL string) ([]model.Library, error) {
parent = strings.TrimSpace(parent)
if parent == "" {
return nil, errors.New("parent path required")
}
dir, err := resolveAccessibleLibraryPath(parent)
if err != nil {
return nil, err
}
entries, err := os.ReadDir(dir)
if err != nil {
return nil, fmt.Errorf("read directory failed: %w", err)
}
subdirs := make([]string, 0, len(entries))
for _, entry := range entries {
if !entry.IsDir() {
continue
}
if strings.HasPrefix(entry.Name(), ".") {
continue
}
subdirs = append(subdirs, filepath.Join(dir, entry.Name()))
}
if len(subdirs) == 0 {
return nil, errors.New("no subfolders found")
}
created := make([]model.Library, 0, len(subdirs))
for _, subdir := range subdirs {
name := filepath.Base(subdir)
lib, err := s.CreateLibraryWithRootsAndCover(ctx, name, kind, coverURL, []LibraryRootInput{{Path: subdir}})
if err != nil {
return nil, fmt.Errorf("create library for %s: %w", subdir, err)
}
created = append(created, *lib)
}
return created, nil
}
func (s *MediaService) UpdateLibraryCover(ctx context.Context, libraryID, coverURL string) error {
return s.repo.DB.WithContext(ctx).Model(&model.Library{}).Where("id = ?", libraryID).
Update("cover_url", strings.TrimSpace(coverURL)).Error
}
// UpdateLibraryFields updates sort_order / carousel_enabled on a library.
func (s *MediaService) UpdateLibraryFields(ctx context.Context, libraryID string, sortOrder *int, carouselEnabled *bool) error {
updates := map[string]any{}
if sortOrder != nil {
updates["sort_order"] = *sortOrder
}
if carouselEnabled != nil {
updates["carousel_enabled"] = *carouselEnabled
}
if len(updates) == 0 || strings.TrimSpace(libraryID) == "" {
return nil
}
return s.repo.DB.WithContext(ctx).Model(&model.Library{}).
Where("id = ?", libraryID).Updates(updates).Error
}
// ReorderLibraries persists a full media-library ordering.
func (s *MediaService) ReorderLibraries(ctx context.Context, ids []string) error {
return s.repo.Library.SetSortOrder(ctx, ids)
}
func (s *MediaService) findLogicalLibrary(ctx context.Context, name, kind string) (*model.Library, error) {
if s == nil || s.repo == nil || s.repo.Library == nil {
return nil, nil
+94
View File
@@ -0,0 +1,94 @@
package service
import (
"fmt"
"testing"
"time"
"go.uber.org/zap"
"github.com/ShukeBta/MMTL/internal/config"
"github.com/ShukeBta/MMTL/internal/model"
"github.com/ShukeBta/MMTL/internal/repository"
)
func TestListLibrariesWithPreview(t *testing.T) {
db := newServiceTestDB(t, &model.Library{}, &model.Media{})
repos := repository.New(db)
lib1 := model.Library{Name: "电影", Path: "/media/movies", Type: "movie", Enabled: true}
if err := repos.Library.Create(t.Context(), &lib1); err != nil {
t.Fatal(err)
}
lib2 := model.Library{Name: "动漫", Path: "/media/anime", Type: "anime", Enabled: true}
if err := repos.Library.Create(t.Context(), &lib2); err != nil {
t.Fatal(err)
}
now := time.Date(2026, 7, 2, 12, 0, 0, 0, time.UTC)
var rows []model.Media
// Add 5 movies to lib1
for i := 1; i <= 5; i++ {
rows = append(rows, model.Media{
Base: model.Base{ID: fmt.Sprintf("movie-%02d", i), CreatedAt: now.Add(time.Duration(i) * time.Hour)},
LibraryID: lib1.ID,
Title: fmt.Sprintf("电影%d", i),
Path: fmt.Sprintf("/media/movies/电影%d/movie%d.mp4", i, i),
PosterURL: fmt.Sprintf("/api/media/movie-%02d/poster", i),
})
}
// Add 12 episodes of 1 anime to lib2
for i := 1; i <= 12; i++ {
rows = append(rows, model.Media{
Base: model.Base{ID: fmt.Sprintf("anime-ep-%02d", i), CreatedAt: now.Add(time.Duration(i) * time.Minute)},
LibraryID: lib2.ID,
Title: fmt.Sprintf("某动漫 第%d集", i),
Path: fmt.Sprintf("/media/anime/某动漫/Season 01/某动漫.S01E%02d.mp4", i),
SeasonNum: 1,
EpisodeNum: i,
PosterURL: "/api/media/anime-01/poster",
})
}
if err := repos.DB.Create(&rows).Error; err != nil {
t.Fatal(err)
}
svc := NewMediaService(&config.Config{}, zap.NewNop(), repos)
previews, err := svc.ListLibrariesWithPreview(t.Context(), []model.Library{lib1, lib2}, MediaVisibility{IncludeNSFW: true}, 10)
if err != nil {
t.Fatalf("ListLibrariesWithPreview failed: %v", err)
}
if len(previews) != 2 {
t.Fatalf("got %d previews, want 2", len(previews))
}
// Verify lib1 (movies)
if previews[0].ID != lib1.ID {
t.Errorf("preview[0].ID = %q, want %q", previews[0].ID, lib1.ID)
}
if previews[0].Total != 5 {
t.Errorf("preview[0].Total = %d, want 5", previews[0].Total)
}
if len(previews[0].Cards) != 5 {
t.Errorf("preview[0].Cards count = %d, want 5", len(previews[0].Cards))
}
// Verify lib2 (anime)
if previews[1].ID != lib2.ID {
t.Errorf("preview[1].ID = %q, want %q", previews[1].ID, lib2.ID)
}
if previews[1].Total != 12 {
t.Errorf("preview[1].Total = %d, want 12", previews[1].Total)
}
// 12 episodes should be grouped into 1 SeriesCard with Count = 12
if len(previews[1].Cards) != 1 {
t.Errorf("preview[1].Cards count = %d, want 1", len(previews[1].Cards))
} else if previews[1].Cards[0].Count != 12 {
t.Errorf("preview[1].Cards[0].Count = %d, want 12", previews[1].Cards[0].Count)
}
}
+70
View File
@@ -2,6 +2,8 @@ package service
import (
"context"
"errors"
"path/filepath"
"regexp"
"sort"
"strings"
@@ -75,6 +77,74 @@ func (s *MediaService) ListLibrarySeriesEpisodes(ctx context.Context, libraryID,
return out, nil
}
func (s *MediaService) ListMediaEpisodes(ctx context.Context, mediaID string, visibility MediaVisibility) ([]model.Media, error) {
target, err := s.repo.Media.FindByID(ctx, mediaID)
if err != nil {
return nil, err
}
if target == nil {
return nil, errors.New("media not found")
}
if !visibility.Allows(target) {
return nil, errors.New("media not found")
}
if target.LibraryID == "" {
return []model.Media{*target}, nil
}
rows, _, err := s.listAllMediaVisible(ctx, target.LibraryID, visibility)
if err != nil {
return nil, err
}
if len(rows) == 0 {
return []model.Media{*target}, nil
}
resolver := newMediaSeriesKeyResolver(rows)
targetKey := resolver.key(*target)
out := make([]model.Media, 0)
if targetKey != "" {
for _, row := range rows {
if resolver.key(row) == targetKey {
out = append(out, row)
}
}
}
// 如果没有聚合到多集,尝试同父目录匹配
if len(out) <= 1 && target.Path != "" {
targetDir := filepath.Dir(strings.ReplaceAll(target.Path, "\\", "/"))
dirMatches := make([]model.Media, 0)
for _, row := range rows {
if row.Path != "" && filepath.Dir(strings.ReplaceAll(row.Path, "\\", "/")) == targetDir {
dirMatches = append(dirMatches, row)
}
}
if len(dirMatches) > 1 {
out = dirMatches
}
}
if len(out) == 0 {
out = []model.Media{*target}
}
sort.SliceStable(out, func(i, j int) bool {
if out[i].SeasonNum != out[j].SeasonNum {
return out[i].SeasonNum < out[j].SeasonNum
}
if out[i].EpisodeNum != out[j].EpisodeNum {
return out[i].EpisodeNum < out[j].EpisodeNum
}
if out[i].Path != out[j].Path {
return out[i].Path < out[j].Path
}
return out[i].CreatedAt.Before(out[j].CreatedAt)
})
return out, nil
}
func (s *MediaService) listAllMediaVisible(ctx context.Context, libraryID string, visibility MediaVisibility) ([]model.Media, int64, error) {
const pageSize = 2000
var all []model.Media
+354
View File
@@ -0,0 +1,354 @@
package service
import (
"context"
"errors"
"fmt"
"strings"
"sync"
"time"
"go.uber.org/zap"
"github.com/ShukeBta/MMTL/internal/model"
)
type ScrapeQueueCounts struct {
Pending int64 `json:"pending"`
Running int64 `json:"running"`
Done int64 `json:"done"`
Failed int64 `json:"failed"`
Canceled int64 `json:"canceled"`
}
type ScrapeQueueSnapshot struct {
Counts ScrapeQueueCounts `json:"counts"`
Tasks []model.ScrapeTask `json:"tasks"`
Total int64 `json:"total"`
Page int `json:"page"`
PageSize int `json:"page_size"`
}
// Start 启动刮削任务队列的后台消费者。
func (s *ScraperService) Start(ctx context.Context) {
if s == nil {
return
}
go s.queueWorker(ctx)
}
func (s *ScraperService) queueWorker(ctx context.Context) {
const claimBatch = 4
sem := make(chan struct{}, 2) // 最大并发刮削数:2
for {
select {
case <-ctx.Done():
return
default:
}
tasks, err := s.repo.ScrapeTask.ClaimPending(ctx, claimBatch)
if err != nil {
if s.log != nil {
s.log.Warn("claim pending scrape task failed", zap.Error(err))
}
sleepContext(ctx, 3*time.Second)
continue
}
if len(tasks) == 0 {
sleepContext(ctx, 2*time.Second)
continue
}
var wg sync.WaitGroup
for i := range tasks {
wg.Add(1)
go func(t *model.ScrapeTask) {
defer wg.Done()
select {
case <-ctx.Done():
return
case sem <- struct{}{}:
}
defer func() { <-sem }()
s.processScrapeTask(ctx, t)
}(&tasks[i])
}
wg.Wait()
}
}
func (s *ScraperService) processScrapeTask(ctx context.Context, task *model.ScrapeTask) {
media, err := s.repo.Media.FindByID(ctx, task.MediaID)
if err != nil || media == nil {
now := time.Now()
task.Status = model.ScrapeTaskFailed
task.Error = "媒体项已不存在或被删除"
task.FinishedAt = &now
_ = s.repo.ScrapeTask.Update(ctx, task)
return
}
epArtwork := task.EpisodeImages
options := ScrapeOptions{
EpisodeArtwork: &epArtwork,
IncludeMatched: task.RefreshMatched,
RetryNoMatch: true,
}
enrichErr := s.EnrichOneWithOptions(ctx, media, options)
now := time.Now()
task.FinishedAt = &now
refreshed, _ := s.repo.Media.FindByID(ctx, media.ID)
if refreshed != nil && refreshed.ScrapeStatus == "matched" {
task.Status = model.ScrapeTaskDone
task.Error = ""
task.MatchedTitle = refreshed.Title
task.MatchedYear = refreshed.Year
task.PosterURL = refreshed.PosterURL
task.BackdropURL = refreshed.BackdropURL
if refreshed.TMDbID > 0 {
task.Provider = "tmdb"
} else if strings.TrimSpace(refreshed.DoubanID) != "" {
task.Provider = "douban"
} else if refreshed.BangumiID > 0 {
task.Provider = "bangumi"
} else if strings.TrimSpace(refreshed.TheTVDBID) != "" {
task.Provider = "thetvdb"
} else {
task.Provider = "metatube"
}
} else {
task.Status = model.ScrapeTaskFailed
if enrichErr != nil {
task.Error = enrichErr.Error()
} else if refreshed != nil && refreshed.ScrapeStatus == "no_match" {
task.Error = "未搜索到匹配的元数据"
} else {
task.Error = "刮削未完成匹配"
}
}
_ = s.repo.ScrapeTask.Update(ctx, task)
if s.hub != nil {
s.hub.Publish("scraper_queue", map[string]any{
"task_id": task.ID,
"status": task.Status,
"title": task.MediaTitle,
})
}
}
func (s *ScraperService) mediaKind(m *model.Media, lib *model.Library) string {
if m == nil {
return ""
}
if lib != nil && lib.Type != "" {
return lib.Type
}
if mediaIsEpisodic(m, lib) {
return "tv"
}
return "movie"
}
// EnqueueMedia 把单个媒体项放入刮削队列。
func (s *ScraperService) EnqueueMedia(ctx context.Context, mediaID string, options ScrapeOptions) (*model.ScrapeTask, error) {
if s == nil || s.repo == nil {
return nil, errors.New("scraper service not initialized")
}
media, err := s.repo.Media.FindByID(ctx, mediaID)
if err != nil || media == nil {
return nil, errors.New("media not found")
}
if active, _ := s.repo.ScrapeTask.FindActiveByMediaID(ctx, mediaID); active != nil {
return active, nil
}
libName := ""
var lib *model.Library
if strings.TrimSpace(media.LibraryID) != "" {
lib, _ = s.repo.Library.FindByID(ctx, media.LibraryID)
if lib != nil {
libName = lib.Name
}
}
task := &model.ScrapeTask{
MediaID: media.ID,
LibraryID: media.LibraryID,
LibraryName: libName,
MediaTitle: media.Title,
MediaPath: media.Path,
MediaType: s.mediaKind(media, lib),
Status: model.ScrapeTaskPending,
EpisodeImages: options.episodeArtworkEnabled(),
RefreshMatched: options.IncludeMatched || options.RefreshWeakMatched,
}
if err := s.repo.ScrapeTask.Create(ctx, task); err != nil {
return nil, err
}
return task, nil
}
// EnqueueLibrary 把指定媒体库内的所有候选媒体批量推入刮削队列。
func (s *ScraperService) EnqueueLibrary(ctx context.Context, libraryID string, options ScrapeOptions) (int, error) {
if s == nil || s.repo == nil {
return 0, errors.New("scraper service not initialized")
}
lib, err := s.repo.Library.FindByID(ctx, libraryID)
if err != nil || lib == nil {
return 0, errors.New("library not found")
}
rows, err := s.scrapeCandidateRows(ctx, libraryID, options)
if err != nil {
return 0, err
}
if len(rows) == 0 {
return 0, nil
}
tasks := make([]model.ScrapeTask, 0, len(rows))
for _, m := range rows {
tasks = append(tasks, model.ScrapeTask{
MediaID: m.ID,
LibraryID: lib.ID,
LibraryName: lib.Name,
MediaTitle: m.Title,
MediaPath: m.Path,
MediaType: s.mediaKind(&m, lib),
Status: model.ScrapeTaskPending,
EpisodeImages: options.episodeArtworkEnabled(),
RefreshMatched: options.IncludeMatched || options.RefreshWeakMatched,
})
}
if err := s.repo.ScrapeTask.CreateBatch(ctx, tasks); err != nil {
return 0, err
}
return len(tasks), nil
}
// EnqueueAll 把所有已启用媒体库的媒体推入刮削队列。
func (s *ScraperService) EnqueueAll(ctx context.Context, options ScrapeOptions) (int, error) {
libs, err := s.repo.Library.List(ctx)
if err != nil {
return 0, err
}
total := 0
for _, lib := range libs {
if !lib.Enabled {
continue
}
n, err := s.EnqueueLibrary(ctx, lib.ID, options)
if err != nil {
if s.log != nil {
s.log.Warn("enqueue library for scrape failed", zap.String("library", lib.ID), zap.Error(err))
}
continue
}
total += n
}
return total, nil
}
func (s *ScraperService) ScrapeQueueSnapshot(ctx context.Context, status string, page, pageSize int) (*ScrapeQueueSnapshot, error) {
tasks, total, err := s.repo.ScrapeTask.List(ctx, status, page, pageSize)
if err != nil {
return nil, err
}
countsMap, err := s.repo.ScrapeTask.CountByStatus(ctx)
if err != nil {
return nil, err
}
snap := &ScrapeQueueSnapshot{
Counts: ScrapeQueueCounts{
Pending: countsMap[model.ScrapeTaskPending],
Running: countsMap[model.ScrapeTaskRunning],
Done: countsMap[model.ScrapeTaskDone],
Failed: countsMap[model.ScrapeTaskFailed],
Canceled: countsMap[model.ScrapeTaskCanceled],
},
Tasks: tasks,
Total: total,
Page: page,
PageSize: pageSize,
}
return snap, nil
}
func (s *ScraperService) CancelScrapeTask(ctx context.Context, id string) error {
task, err := s.repo.ScrapeTask.FindByID(ctx, id)
if err != nil || task == nil {
return errors.New("刮削任务不存在")
}
if task.Status != model.ScrapeTaskPending && task.Status != model.ScrapeTaskRunning {
return errors.New("任务已完成或已终止,无法取消")
}
now := time.Now()
task.Status = model.ScrapeTaskCanceled
task.Error = "已取消"
task.FinishedAt = &now
return s.repo.ScrapeTask.Update(ctx, task)
}
func (s *ScraperService) RetryScrapeTask(ctx context.Context, id string) error {
task, err := s.repo.ScrapeTask.FindByID(ctx, id)
if err != nil || task == nil {
return errors.New("刮削任务不存在")
}
if task.Status != model.ScrapeTaskFailed && task.Status != model.ScrapeTaskCanceled {
return errors.New("只有失败或已取消的任务可以重试")
}
task.Status = model.ScrapeTaskPending
task.Error = ""
task.RetryCount = 0
task.StartedAt = nil
task.FinishedAt = nil
return s.repo.ScrapeTask.Update(ctx, task)
}
func (s *ScraperService) DeleteScrapeTask(ctx context.Context, id string) error {
return s.repo.ScrapeTask.Delete(ctx, id)
}
func (s *ScraperService) BatchActionScrapeTasks(ctx context.Context, action string, ids []string) (int64, error) {
switch action {
case "delete":
return s.repo.ScrapeTask.DeleteBatch(ctx, ids)
case "retry":
return s.repo.ScrapeTask.RetryBatch(ctx, ids)
case "cancel":
return s.repo.ScrapeTask.CancelBatch(ctx, ids)
default:
return 0, fmt.Errorf("不支持的操作: %s", action)
}
}
func (s *ScraperService) ClearDoneScrapeTasks(ctx context.Context) (int64, error) {
return s.repo.ScrapeTask.ClearDone(ctx)
}
func (s *ScraperService) ClearFinishedScrapeTasks(ctx context.Context) (int64, error) {
return s.repo.ScrapeTask.ClearFinished(ctx)
}
func (s *ScraperService) ClearCanceledScrapeTasks(ctx context.Context) (int64, error) {
return s.repo.ScrapeTask.ClearCanceled(ctx)
}
func (s *ScraperService) RetryAllFailedScrapeTasks(ctx context.Context) (int64, error) {
return s.repo.ScrapeTask.RetryAllFailed(ctx)
}
func (s *ScraperService) CancelPendingScrapeTasks(ctx context.Context) (int64, error) {
return s.repo.ScrapeTask.CancelPending(ctx)
}
+20 -10
View File
@@ -2,8 +2,8 @@ package service
import (
"errors"
"io"
"net/http"
"net/http/httptest"
"path/filepath"
"strings"
"testing"
@@ -396,15 +396,19 @@ func TestEnrichOneAdultScrapesArtwork(t *testing.T) {
t.Fatal(err)
}
// Mock adult provider
adultServer := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
w.Header().Set("Content-Type", "text/html; charset=utf-8")
_, _ = w.Write([]byte(`<html><h3>IPX-235 测试番号封面</h3><a class="bigImage" href="/pics/cover/ipx235.jpg"></a></html>`))
}))
defer adultServer.Close()
// Mock adult provider:RoundTripper 拦截 provider 发出的全部请求(不访问
// 外网),任何站点都返回带封面链接的测试 HTML——保证 CI 上不命中真实
// javdb/dmm 站点(外网可达时会把真实封面写进断言导致失败)。
adultHTML := `<html><h3>IPX-235 测试番号封面</h3><a class="bigImage" href="/pics/cover/ipx235.jpg"></a></html>`
mockAdultTransport := roundTripFunc(func(r *http.Request) (*http.Response, error) {
return &http.Response{
StatusCode: http.StatusOK,
Header: http.Header{"Content-Type": []string{"text/html; charset=utf-8"}},
Body: io.NopCloser(strings.NewReader(adultHTML)),
}, nil
})
scraper.adult = &AdultProvider{
client: adultServer.Client(),
client: &http.Client{Transport: mockAdultTransport},
}
if err := scraper.EnrichOne(t.Context(), &media); err != nil {
@@ -418,7 +422,13 @@ func TestEnrichOneAdultScrapesArtwork(t *testing.T) {
if got.ScrapeStatus != "matched" {
t.Fatalf("ScrapeStatus = %q, want 'matched'", got.ScrapeStatus)
}
if got.PosterURL != "https://pics.dmm.co.jp/digital/video/ipx00235/ipx00235pl.jpg" && got.PosterURL != adultServer.URL+"/pics/cover/ipx235.jpg" {
// 封面来自 mock HTML 的 cover 链接(Host 随站点变化,只校验路径后缀)。
if got.PosterURL != "https://pics.dmm.co.jp/digital/video/ipx00235/ipx00235pl.jpg" && !strings.HasSuffix(got.PosterURL, "/pics/cover/ipx235.jpg") {
t.Fatalf("unexpected PosterURL = %q", got.PosterURL)
}
}
// roundTripFunc 把函数适配为 http.RoundTripper,用于测试中拦截全部外网请求。
type roundTripFunc func(*http.Request) (*http.Response, error)
func (f roundTripFunc) RoundTrip(r *http.Request) (*http.Response, error) { return f(r) }
+18 -5
View File
@@ -46,6 +46,7 @@ type Container struct {
Scheduler *SchedulerService
Storage *StorageService
Emby *EmbyService
EmbyRemote *EmbyRemoteService
Backup *BackupService
PlayProfiles *PlayProfileService
Permissions *PermissionService
@@ -58,11 +59,13 @@ type Container struct {
Device *DeviceService
Cache *RuntimeCacheService
Sessions *SessionTrackerService
RecognitionWords *RecognitionWordsService
Danmaku *DanmakuService
Strm *StrmService
RecognitionWords *RecognitionWordsService
Danmaku *DanmakuService
Strm *StrmService
Database *DatabaseAdminService
FFTools *FFmpegToolsService
stopCtx context.Context
stopCtx context.Context
stopCancel context.CancelFunc
// ReloadHTTPServer 由 cmd/server 注入。HTTPS 相关设置保存后,handler
@@ -97,12 +100,22 @@ func (c *Container) Boot() {
// 启动调度器定时任务
c.Scheduler.Start(c.stopCtx)
// 远程 Emby 挂载兼容迁移:旧账号无挂载时自动全量挂载
if c.EmbyRemote != nil {
c.EmbyRemote.AutoSeedMounts(c.stopCtx)
}
// STRM 元数据下载/上传队列与定时同步巡检
if c.Strm != nil {
c.Strm.Start(c.stopCtx)
}
// Mgo 保号规则巡检:默认关闭,由管理员通过 Telegram Bot 命令开启。
// 启动刮削队列后台消费者
if c.Scraper != nil {
c.Scraper.Start(c.stopCtx)
}
// Mgo 保号规则巡检:默认关闭,由管理员通过 Telegram Bot 命令开启。
// 每天触发一次评估;规则里的窗口可随机,不固定。
if c.Device != nil {
go c.runInactivitySweeper(c.stopCtx)
+5
View File
@@ -103,6 +103,8 @@ func (b *serviceContainerBuilder) initContentServices() {
b.c.DLNA = NewDLNAService(b.log)
b.c.Storage = NewStorageService(b.log, b.repos)
b.c.Emby = NewEmbyService(b.cfg, b.log, b.repos)
b.c.EmbyRemote = NewEmbyRemoteService(b.cfg, b.log, b.repos, b.c.Crypto).SetRuntimeCache(b.c.Cache)
b.c.Emby.SetEmbyRemote(b.c.EmbyRemote)
b.c.Backup = NewBackupService(b.cfg, b.log, b.repos.DB)
b.c.Media = NewMediaService(b.cfg, b.log, b.repos).SetRuntimeCache(b.c.Cache)
b.c.Stream = NewStreamService(b.cfg, b.log, b.repos, b.c.Transcoder)
@@ -111,6 +113,8 @@ func (b *serviceContainerBuilder) initContentServices() {
b.c.Profile = NewProfileService(b.log, b.repos)
b.c.Audit = NewAuditService(b.log, b.repos)
b.c.Strm = NewStrmService(b.cfg, b.log, b.repos, b.c.Crypto)
// ffmpeg/ffprobe 一键下载安装(data/tools/ffmpeg/)。
b.c.FFTools = NewFFmpegToolsService(b.cfg, b.log, b.repos)
// 弹幕 hash 识别需要把 strm 指向解析成可拉取的直链/本地路径。
b.c.Danmaku.SetStrmResolver(b.c.Strm.ResolvePlay)
}
@@ -118,6 +122,7 @@ func (b *serviceContainerBuilder) initContentServices() {
func (b *serviceContainerBuilder) initAccessAndStorageServices() {
b.c.PlayProfiles = NewPlayProfileService(b.log, b.repos)
b.c.Permissions = NewPermissionService(b.log, b.repos)
b.c.Database = NewDatabaseAdminService(b.cfg, b.log, b.repos, b.repos.DB)
b.c.Emby.SetRuntimeCache(b.c.Cache)
b.c.Emby.SetSubtitleService(b.c.Subtitle)
b.c.Scheduler = NewSchedulerService(
+6 -2
View File
@@ -86,9 +86,9 @@ func playableSTRMTarget(ctx context.Context, repo *repository.Container, raw str
return STRMPlaybackEnabled(ctx, repo)
}
// isStrmMediaRow 判断媒体行是否为 .strm(远程直链)媒体:STRMURL 非空、
// IsStrmMediaRow 判断媒体行是否为 .strm(远程直链)媒体:STRMURL 非空、
// container=strm 或路径以 .strm 结尾。strm 媒体只能直连播放,禁止转码。
func isStrmMediaRow(m *model.Media) bool {
func IsStrmMediaRow(m *model.Media) bool {
if m == nil {
return false
}
@@ -101,6 +101,10 @@ func isStrmMediaRow(m *model.Media) bool {
return strings.HasSuffix(strings.ToLower(strings.TrimSpace(m.Path)), ".strm")
}
func isStrmMediaRow(m *model.Media) bool {
return IsStrmMediaRow(m)
}
func isHTTPPlaybackTarget(raw string) bool {
u, err := url.Parse(strings.TrimSpace(raw))
if err != nil || u == nil || !u.IsAbs() {
+58
View File
@@ -116,6 +116,64 @@ func (s *StrmService) resolveLocalPlay(ctx context.Context, rawPath string) (*St
return nil, errors.New("文件不在任何本地同步目录内")
}
// ResolvePlayTarget 解析媒体行固化的播放目标(STRMURL 或 .strm 文件内容)为
// 可播放结果,供弹幕 hash、内嵌字幕提取等「先解析直链再读取远端」的场景复用。
// 支持:
// - /api/strm/play/{provider}/video{ext}?acct=..&pickcode=.. (常规格式,含账号)
// - /api/cloud/play/{type}?ref=.. (旧格式,无账号 → 取该类型第一个启用账号)
// - 绝对 http(s) 链接(直接透传)
// - 其余协议(webdav:// 等)返回错误,由调用方决定是否静默跳过
func (s *StrmService) ResolvePlayTarget(ctx context.Context, raw string) (*StrmPlayResult, error) {
raw = strings.TrimSpace(raw)
if raw == "" {
return nil, errors.New("空播放目标")
}
u, err := url.Parse(raw)
if err != nil {
return nil, fmt.Errorf("解析播放目标失败: %w", err)
}
lowerPath := strings.ToLower(u.Path)
switch {
case strings.HasPrefix(lowerPath, "/api/strm/play/"):
segs := strings.Split(strings.TrimPrefix(u.Path, "/api/strm/play/"), "/")
if len(segs) < 1 || strings.TrimSpace(segs[0]) == "" {
return nil, errors.New("无效的 strm 播放地址")
}
return s.ResolvePlay(ctx, segs[0], u.Query())
case strings.HasPrefix(lowerPath, "/api/cloud/play/"):
typ := strings.TrimSpace(strings.TrimPrefix(u.Path, "/api/cloud/play/"))
acct, err := s.firstEnabledAccountOf(ctx, typ)
if err != nil || acct == nil {
return nil, errors.New("没有可用的网盘账号,无法解析直链")
}
q := u.Query()
q.Set("acct", acct.ID)
return s.ResolvePlay(ctx, typ, q)
case u.Scheme == "http" || u.Scheme == "https":
return &StrmPlayResult{RedirectURL: raw}, nil
default:
return nil, fmt.Errorf("不支持的播放目标协议: %s", u.Scheme)
}
}
// firstEnabledAccountOf 返回指定提供方第一个凭据可用的启用账号。
func (s *StrmService) firstEnabledAccountOf(ctx context.Context, provider string) (*model.StrmAccount, error) {
accounts, err := s.repo.StrmAccount.List(ctx)
if err != nil {
return nil, err
}
for i := range accounts {
a := &accounts[i]
if !a.Enabled || a.Provider != provider {
continue
}
if _, err := s.providerFor(ctx, a); err == nil {
return a, nil
}
}
return nil, nil
}
// ProxyDirect 反向代理渲染直链内容(保留 Range 请求头以支持拖动播放)。
func (s *StrmService) ProxyDirect(ctx context.Context, w http.ResponseWriter, r *http.Request, link *cloud.DirectLink) error {
if link == nil || link.URL == "" {
+116 -3
View File
@@ -13,6 +13,7 @@ import (
"os"
"path/filepath"
"strings"
"sync"
"time"
"go.uber.org/zap"
@@ -27,7 +28,12 @@ const (
)
// downloadWorker 下载队列 worker:认领 → 解析直链 → 下载 → 落盘。
//
// 采用「批量认领 + 全局并发限流」:一次认领数个任务,用 StrmService 上的全局信号量
// 限制整个进程「同时换直链+下载」的并发数(与 115 换链风控匹配,见 strmDownloadSemCap),
// 同时让下载充分并行。换链走全局令牌桶(QPS=3)兜底,下载走 CDN 不限速。
func (s *StrmService) downloadWorker(ctx context.Context) {
const claimBatch = 12 // 每次批量认领的任务数
for {
select {
case <-ctx.Done():
@@ -42,7 +48,7 @@ func (s *StrmService) downloadWorker(ctx context.Context) {
sleepContext(ctx, left)
continue
}
tasks, err := s.repo.StrmDownload.ClaimPendingDownload(ctx, 1)
tasks, err := s.repo.StrmDownload.ClaimPendingDownload(ctx, claimBatch)
if err != nil {
s.log.Warn("claim strm download task failed", zap.Error(err))
sleepContext(ctx, 3*time.Second)
@@ -52,9 +58,21 @@ func (s *StrmService) downloadWorker(ctx context.Context) {
sleepContext(ctx, 2*time.Second)
continue
}
// 并发处理本批任务:每个任务先获取全局下载槽位,槽位内部执行换链+下载。
// 信号量与令牌桶双重限速,确保任意时刻并发换链请求不超过安全阈值。
var wg sync.WaitGroup
for i := range tasks {
s.processDownloadTask(ctx, &tasks[i])
wg.Add(1)
go func(i int) {
defer wg.Done()
if !s.acquireDownloadSlot(ctx) {
return
}
defer s.releaseDownloadSlot()
s.processDownloadTask(ctx, &tasks[i])
}(i)
}
wg.Wait()
}
}
@@ -139,7 +157,7 @@ func (s *StrmService) processUploadTask(ctx context.Context, task *model.StrmUpl
}
}
if task.Provider == model.StrmProvider115 {
finish(model.StrmTaskFailed, "115 网盘暂不支持元数据上传")
s.processUpload115(ctx, task)
return
}
acct, err := s.repo.StrmAccount.FindByID(ctx, task.AccountID)
@@ -178,6 +196,48 @@ func (s *StrmService) processUploadTask(ctx context.Context, task *model.StrmUpl
finish(model.StrmTaskDone, "")
}
// processUpload115 115 元数据上传:task.RemotePath 存的是父目录 cid,FileName 为远端文件名。
func (s *StrmService) processUpload115(ctx context.Context, task *model.StrmUploadTask) {
finish := func(status, message string) {
now := time.Now()
task.Status = status
task.Error = message
task.FinishedAt = &now
if err := s.repo.StrmUpload.Update(context.Background(), task); err != nil {
s.log.Warn("update strm upload task failed", zap.Error(err))
}
}
acct, err := s.repo.StrmAccount.FindByID(ctx, task.AccountID)
if err != nil || acct == nil {
finish(model.StrmTaskFailed, "网盘账号不存在")
return
}
provider, err := s.providerFor(ctx, acct)
if err != nil {
s.uploadTaskFailWithRetry(task, err.Error())
return
}
named, ok := provider.(interface {
PutFileNamed(ctx context.Context, parentCID, fileName string, r io.Reader) error
})
if !ok {
finish(model.StrmTaskFailed, "该网盘不支持元数据上传")
return
}
f, err := os.Open(task.LocalPath)
if err != nil {
s.uploadTaskFailWithRetry(task, "打开本地文件失败:"+err.Error())
return
}
if err := named.PutFileNamed(ctx, task.RemotePath, task.FileName, f); err != nil {
_ = f.Close()
s.uploadTaskFailWithRetry(task, "上传失败:"+err.Error())
return
}
_ = f.Close()
finish(model.StrmTaskDone, "")
}
// downloadTaskFailWithRetry 下载失败任务按退避重试,超过上限标记 failed。
func (s *StrmService) downloadTaskFailWithRetry(task *model.StrmDownloadTask, message string) {
if !retryTask(&task.RetryCount, &task.Status, &task.Error, &task.NextTryAt, &task.FinishedAt, message) {
@@ -491,6 +551,44 @@ func (s *StrmService) RetryUploadTask(ctx context.Context, id string) error {
// ─── 下载队列批量操作(handler 使用) ─────────────────────────────────────────
// DeleteDownloadTask 删除一个下载任务记录。
func (s *StrmService) DeleteDownloadTask(ctx context.Context, id string) error {
return s.repo.StrmDownload.Delete(ctx, id)
}
// DeleteUploadTask 删除一个上传任务记录。
func (s *StrmService) DeleteUploadTask(ctx context.Context, id string) error {
return s.repo.StrmUpload.Delete(ctx, id)
}
// BatchActionDownloadTasks 对选中的下载任务执行批量操作(delete / retry / cancel)。
func (s *StrmService) BatchActionDownloadTasks(ctx context.Context, action string, ids []string) (int64, error) {
switch action {
case "delete":
return s.repo.StrmDownload.DeleteBatch(ctx, ids)
case "retry":
return s.repo.StrmDownload.RetryBatch(ctx, ids)
case "cancel":
return s.repo.StrmDownload.CancelBatch(ctx, ids)
default:
return 0, fmt.Errorf("不支持的批量操作: %s", action)
}
}
// BatchActionUploadTasks 对选中的上传任务执行批量操作(delete / retry / cancel)。
func (s *StrmService) BatchActionUploadTasks(ctx context.Context, action string, ids []string) (int64, error) {
switch action {
case "delete":
return s.repo.StrmUpload.DeleteBatch(ctx, ids)
case "retry":
return s.repo.StrmUpload.RetryBatch(ctx, ids)
case "cancel":
return s.repo.StrmUpload.CancelBatch(ctx, ids)
default:
return 0, fmt.Errorf("不支持的批量操作: %s", action)
}
}
// ClearDoneDownloadTasks 清空全部已完成下载记录,返回删除数量。
func (s *StrmService) ClearDoneDownloadTasks(ctx context.Context) (int64, error) {
return s.repo.StrmDownload.ClearDone(ctx)
@@ -511,11 +609,26 @@ func (s *StrmService) ClearCanceledUploadTasks(ctx context.Context) (int64, erro
return s.repo.StrmUpload.ClearCanceled(ctx)
}
// ClearDoneUploadTasks 清空全部已完成上传记录,返回删除数量。
func (s *StrmService) ClearDoneUploadTasks(ctx context.Context) (int64, error) {
return s.repo.StrmUpload.ClearDone(ctx)
}
// ClearFinishedUploadTasks 清空全部已完成与失败的上传记录,返回删除数量。
func (s *StrmService) ClearFinishedUploadTasks(ctx context.Context) (int64, error) {
return s.repo.StrmUpload.ClearFinished(ctx)
}
// RetryAllFailedDownloadTasks 批量重试所有失败下载任务,返回重新入队数量。
func (s *StrmService) RetryAllFailedDownloadTasks(ctx context.Context) (int64, error) {
return s.repo.StrmDownload.RetryAllFailed(ctx)
}
// RetryAllFailedUploadTasks 批量重试所有失败上传任务,返回重新入队数量。
func (s *StrmService) RetryAllFailedUploadTasks(ctx context.Context) (int64, error) {
return s.repo.StrmUpload.RetryAllFailed(ctx)
}
// CancelPendingDownloadTasks 批量取消所有排队下载任务,返回取消数量。
func (s *StrmService) CancelPendingDownloadTasks(ctx context.Context) (int64, error) {
return s.repo.StrmDownload.CancelPending(ctx)
+60
View File
@@ -1,8 +1,13 @@
package service
import (
"context"
"errors"
"testing"
"github.com/ShukeBta/MMTL/internal/model"
"github.com/ShukeBta/MMTL/internal/repository"
"go.uber.org/zap"
)
func TestIs115Blocked(t *testing.T) {
@@ -44,3 +49,58 @@ func TestIsHTTPDownloadFailure(t *testing.T) {
}
}
}
func TestStrmUploadTasksClearAndRetry(t *testing.T) {
db := newServiceTestDB(t, &model.StrmUploadTask{})
repos := repository.New(db)
svc := NewStrmService(nil, zap.NewNop(), repos, nil)
ctx := context.Background()
tasks := []*model.StrmUploadTask{
{Base: model.Base{ID: "task-pending"}, Status: model.StrmTaskPending, FileName: "1.nfo"},
{Base: model.Base{ID: "task-running"}, Status: model.StrmTaskRunning, FileName: "2.nfo"},
{Base: model.Base{ID: "task-done"}, Status: model.StrmTaskDone, FileName: "3.nfo"},
{Base: model.Base{ID: "task-failed"}, Status: model.StrmTaskFailed, FileName: "4.nfo", Error: "some error", RetryCount: 3},
{Base: model.Base{ID: "task-canceled"}, Status: model.StrmTaskCanceled, FileName: "5.nfo"},
}
for _, task := range tasks {
if err := db.Create(task).Error; err != nil {
t.Fatalf("failed to insert task: %v", err)
}
}
// 1. RetryAllFailedUploadTasks
retried, err := svc.RetryAllFailedUploadTasks(ctx)
if err != nil {
t.Fatalf("RetryAllFailedUploadTasks failed: %v", err)
}
if retried != 1 {
t.Fatalf("expected 1 retried task, got %d", retried)
}
var failedTask model.StrmUploadTask
if err := db.First(&failedTask, "id = ?", "task-failed").Error; err != nil {
t.Fatalf("failed to get task-failed: %v", err)
}
if failedTask.Status != model.StrmTaskPending || failedTask.Error != "" || failedTask.RetryCount != 0 {
t.Fatalf("task-failed was not reset properly: %+v", failedTask)
}
// 再次改为 failed 以便测试 ClearFinished
db.Model(&model.StrmUploadTask{}).Where("id = ?", "task-failed").Updates(map[string]any{"status": model.StrmTaskFailed})
// 2. ClearFinishedUploadTasks 应删除 done, failed, canceled 三条历史记录
deleted, err := svc.ClearFinishedUploadTasks(ctx)
if err != nil {
t.Fatalf("ClearFinishedUploadTasks failed: %v", err)
}
if deleted != 3 {
t.Fatalf("expected 3 deleted tasks (done, failed, canceled), got %d", deleted)
}
// 验证剩余的任务只有 pending 和 running
var count int64
db.Model(&model.StrmUploadTask{}).Count(&count)
if count != 2 {
t.Fatalf("expected 2 remaining tasks, got %d", count)
}
}
+115 -2
View File
@@ -73,7 +73,7 @@ var StrmSettingDefs = map[string]struct {
}
// StrmAccountSecretKeys 是账号配置中需要加密存储的字段。
var StrmAccountSecretKeys = []string{"cookie", "password", "token", "access_token", "refresh_token"}
var StrmAccountSecretKeys = []string{"cookie", "password", "token", "access_token", "refresh_token", "api_key"}
// StrmService 提供 STRM 管理的能力。
type StrmService struct {
@@ -90,11 +90,48 @@ type StrmService struct {
running map[string]context.CancelFunc // sync path id -> cancel
oauthSessions map[string]*strm115AuthSession
wafUntil time.Time // 115 风控/限流熔断截止时间(由 mu 保护)
downloadSem chan struct{} // 全局下载并发信号量:限制整个进程同时进行「换直链+下载」的并发数
downloadSemOnce sync.Once
}
// strmWAFCooldown 检测到 115 风控/限流后下载队列的全局冷却时长。
const strmWAFCooldown = 3 * time.Minute
// strmDownloadSemCap 全局同时进行「换直链+下载」的并发上限。
//
// 115 对换直链接口(/open/ufile/downurl)风控极严:过去把全局 QPS 提到 8 或让多
// worker 高并发换链,会瞬时撞上 WAF 返回 405 阻断页并触发 180 秒冷却,反而更慢。
// 因此用信号量把整个进程同时换直链的并发数压到 3,与令牌桶限速共同兜底:
// 宁可下载稍慢,也绝不触发风控。下载本身走 CDN 不限速。
const strmDownloadSemCap = 3
// ensureDownloadSem 惰性初始化全局共享的下载并发信号量。
func (s *StrmService) ensureDownloadSem() {
s.downloadSemOnce.Do(func() {
s.downloadSem = make(chan struct{}, strmDownloadSemCap)
})
}
// acquireDownloadSlot 获取一个下载并发槽位(等待/取消安全)。
func (s *StrmService) acquireDownloadSlot(ctx context.Context) bool {
s.ensureDownloadSem()
select {
case s.downloadSem <- struct{}{}:
return true
case <-ctx.Done():
return false
}
}
// releaseDownloadSlot 释放一个下载并发槽位。
func (s *StrmService) releaseDownloadSlot() {
if s.downloadSem == nil {
return
}
<-s.downloadSem
}
// NewStrmService constructs the STRM service.
func NewStrmService(cfg *config.Config, log *zap.Logger, repos *repository.Container, crypto *CryptoService) *StrmService {
return &StrmService{
@@ -200,6 +237,33 @@ func (s *StrmService) strmAccountConfig(acct *model.StrmAccount) (map[string]str
return cfg, nil
}
// mergeEmbyRemoteConfig 对远程 Emby 账号配置做合并式更新:config 中出现的键
// 覆盖写入(敏感键按明文加密),未出现的键保留原密文;显式空字符串=清除。
func (s *StrmService) mergeEmbyRemoteConfig(existing string, config map[string]string) (string, error) {
out := map[string]string{}
if strings.TrimSpace(existing) != "" {
if err := json.Unmarshal([]byte(existing), &out); err != nil {
return "", fmt.Errorf("decode account config: %w", err)
}
}
for k, v := range config {
if v == "" {
delete(out, k)
continue
}
if strmContains(StrmAccountSecretKeys, k) {
out[k] = s.crypto.Encrypt(v)
} else {
out[k] = v
}
}
data, err := json.Marshal(out)
if err != nil {
return "", err
}
return string(data), nil
}
// HasStrmAccountCredential 报告账号是否已配置核心凭据(用于前端展示)。
func HasStrmAccountCredential(acct *model.StrmAccount) bool {
switch acct.Provider {
@@ -208,6 +272,11 @@ func HasStrmAccountCredential(acct *model.StrmAccount) bool {
return strings.Contains(acct.Config, `"access_token"`)
case model.StrmProviderOpenList:
return strings.Contains(acct.Config, `"token"`) || strings.Contains(acct.Config, `"password"`)
case model.StrmProviderEmbyRemote:
// 远程 Emby:接入地址 + (自动认证凭据 或 手动 api_key) 即视为已配置。
return strings.Contains(acct.Config, `"url"`) &&
(strings.Contains(acct.Config, `"token"`) || strings.Contains(acct.Config, `"api_key"`) ||
(strings.Contains(acct.Config, `"username"`) && strings.Contains(acct.Config, `"password"`)))
default:
return strings.Contains(acct.Config, `"password"`) || strings.Contains(acct.Config, `"token"`)
}
@@ -251,7 +320,16 @@ func (s *StrmService) UpdateStrmAccount(ctx context.Context, id, name string, en
acct.Enabled = *enabled
}
if len(config) > 0 {
enc, err := s.strmAccountConfigJSON(config, true)
var enc string
var err error
if acct.Provider == model.StrmProviderEmbyRemote {
// 远程 Emby 账号:合并式更新。config 中出现的键覆盖(敏感键按明文
// 加密写入),未出现的键保留原密文——避免编辑「代理开关」时把已
// 保存的地址与凭据清空。
enc, err = s.mergeEmbyRemoteConfig(acct.Config, config)
} else {
enc, err = s.strmAccountConfigJSON(config, true)
}
if err != nil {
return nil, err
}
@@ -412,6 +490,38 @@ func (s *StrmService) ListSyncRecords(ctx context.Context, pathID string, limit
return s.repo.StrmSyncRecord.List(ctx, pathID, limit)
}
// DeleteSyncRecord 删除单条同步记录。
func (s *StrmService) DeleteSyncRecord(ctx context.Context, id string) error {
if err := s.repo.StrmSyncRecord.Delete(ctx, id); err != nil {
return err
}
return nil
}
// ClearSyncRecords 清空某同步目录(pathID 为空则全部)的同步记录,返回删除条数。
func (s *StrmService) ClearSyncRecords(ctx context.Context, pathID string) (int64, error) {
if pathID != "" {
return s.repo.StrmSyncRecord.DeleteBySyncPathID(ctx, pathID)
}
var total int64
// 全量清空:分页拉取物理删除所有记录
for {
rows, err := s.repo.StrmSyncRecord.List(ctx, "", 200)
if err != nil {
return total, err
}
if len(rows) == 0 {
return total, nil
}
for _, rec := range rows {
if err := s.repo.StrmSyncRecord.Delete(ctx, rec.ID); err != nil {
return total, err
}
}
total += int64(len(rows))
}
}
// CreateSyncPath 校验并创建同步目录。
func (s *StrmService) CreateSyncPath(ctx context.Context, p *model.StrmSyncPath) (*model.StrmSyncPath, error) {
if err := s.validateSyncPath(ctx, p); err != nil {
@@ -580,6 +690,8 @@ func providerLabel(provider string) string {
return "OpenList"
case model.StrmProviderLocal:
return "本地目录"
case model.StrmProviderEmbyRemote:
return "Emby 远程挂载"
default:
return provider
}
@@ -591,6 +703,7 @@ var StrmProviderLabels = map[string]string{
model.StrmProviderCloudDrive: "CloudDrive2",
model.StrmProviderOpenList: "OpenList",
model.StrmProviderLocal: "本地目录",
model.StrmProviderEmbyRemote: "Emby 远程挂载",
}
const defaultStrmUA = "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/124.0 Safari/537.36 MMTL-Strm/1.0"
+266 -139
View File
@@ -40,11 +40,16 @@ type strmSyncState struct {
seenVideo map[string]bool // "v:"+去掉扩展名的相对路径 → 远端存在该视频
seenMeta map[string]bool // "m:"+相对路径 → 远端存在该元数据
remoteMeta map[string]int64 // 远端元数据大小(上传比对用)
activeDownloadPaths map[string]bool // 本地已在排队/进行的下载任务路径(内存去重)
activeUploadPaths map[string]bool // 本地已在排队/进行的上传任务路径(内存去重)
seenMetaTarget map[string]cloud.FileEntry
seenVideoTarget map[string]cloud.FileEntry
activeDownloadPaths map[string]bool // 本地已在排队/进行的下载任务路径(内存去重)
activeUploadPaths map[string]bool // 本地已在排队/进行的上传任务路径(内存去重)
pendingDownloads []*model.StrmDownloadTask
pendingUploads []*model.StrmUploadTask
dirCache sync.Map // dirID (string) -> relativePath (string)
dirCache sync.Map // dirID (string) -> relativePath (string)
dirPathToID map[string]string // relativePath (string) -> dirID(115 上传父目录寻址用,walk 后构建)
scanIncomplete atomic.Bool // 远端目录树/文件列表本次扫描不完整 → 禁止增量 prune 误删本地文件
}
// StartSync 启动一次同步(异步执行,同一目录同时只允许一个任务)。
@@ -172,15 +177,17 @@ func (s *StrmService) runSync(ctx context.Context, p *model.StrmSyncPath, rec *m
return
}
st := &strmSyncState{
s: s,
ctx: ctx,
p: p,
cfg: cfg,
rec: rec,
syncType: rec.SyncType,
seenVideo: map[string]bool{},
seenMeta: map[string]bool{},
remoteMeta: map[string]int64{},
s: s,
ctx: ctx,
p: p,
cfg: cfg,
rec: rec,
syncType: rec.SyncType,
seenVideo: map[string]bool{},
seenMeta: map[string]bool{},
remoteMeta: map[string]int64{},
seenMetaTarget: map[string]cloud.FileEntry{},
seenVideoTarget: map[string]cloud.FileEntry{},
}
if p.Provider != model.StrmProviderLocal {
acct, err := s.repo.StrmAccount.FindByID(ctx, p.AccountID)
@@ -228,8 +235,8 @@ func (s *StrmService) finishSync(p *model.StrmSyncPath, rec *model.StrmSyncRecor
if rec.SyncType == model.StrmSyncTypeFull {
syncTypeLabel = "全量"
}
p.LastSyncMessage = fmt.Sprintf("[%s] 完成:新增/更新 %d 个 strm,跳过 %d 个,下载 %d 个元数据,清理 %d 个文件",
syncTypeLabel, rec.NewStrm, rec.Skipped, rec.NewMeta, rec.Pruned)
p.LastSyncMessage = fmt.Sprintf("[%s] 完成:新增/更新 %d 个 strm,跳过 %d 个,下载 %d 个元数据,上传 %d 个元数据,清理 %d 个文件",
syncTypeLabel, rec.NewStrm, rec.Skipped, rec.NewMeta, rec.Uploaded, rec.Pruned)
}
if err := s.repo.StrmSyncPath.Update(context.Background(), p); err != nil {
s.log.Warn("update strm sync path failed", zap.Error(err))
@@ -237,7 +244,7 @@ func (s *StrmService) finishSync(p *model.StrmSyncPath, rec *model.StrmSyncRecor
s.log.Info("strm sync finished",
zap.String("path_id", p.ID), zap.String("sync_type", rec.SyncType), zap.String("status", status),
zap.Int64("new_strm", rec.NewStrm), zap.Int64("skipped", rec.Skipped), zap.Int64("new_meta", rec.NewMeta),
zap.Int64("pruned", rec.Pruned), zap.String("message", message))
zap.Int64("uploaded", rec.Uploaded), zap.Int64("pruned", rec.Pruned), zap.String("message", message))
}
func (st *strmSyncState) run() error {
@@ -276,7 +283,21 @@ func (st *strmSyncState) run() error {
}
st.flushPendingDownloads()
st.flushProgress()
if st.cfg.UploadMeta && st.provider != nil && st.p.Provider != model.StrmProvider115 {
if st.cfg.UploadMeta && st.provider != nil {
// 115 上传需要父目录 cid,先用 dirCache 构建「路径 → cid」反向索引
if st.p.Provider == model.StrmProvider115 {
reversed := map[string]string{}
st.dirCache.Range(func(key, value any) bool {
path, ok := value.(string)
if ok && path != "" {
if id, ok2 := key.(string); ok2 {
reversed[path] = id
}
}
return true
})
st.dirPathToID = reversed
}
if err := st.scanLocalMetaForUpload(); err != nil {
return err
}
@@ -435,11 +456,35 @@ func (st *strmSyncState) isMetaExt(ext string) bool {
return false
}
// cleanDirRel 对 115 扁平化拉取的目录相对路径逐段套用目录级文件名清洗,
// 确保与 walkRemote / joinLocalRel(sanitizeRelativePath)使用同一套清洗规则。
// 若不清洗,目录名中的冒号等非法字符会直达 rel,而 seenVideo/seenMeta 的 key
// 与磁盘实际路径不一致,导致 pruneLocal 误删已下载的 strm / 元数据。
// 空 rel(根目录)原样返回。
func cleanDirRel(rel string) string {
if rel == "" {
return ""
}
parts := strings.Split(rel, "/")
out := make([]string, 0, len(parts))
for _, part := range parts {
if part == "" {
continue
}
clean := cleanEntryName(part, true)
if clean != "" && clean != "." && clean != ".." {
out = append(out, clean)
}
}
return strings.Join(out, "/")
}
// walk115Flat 使用 115 开放平台扁平化分页批量拉取机制与目录拓扑缓存(参考 QMediaSync)。
// 极大地降低 API 请求次数并支持毫秒级/秒级增量同步。
func (st *strmSyncState) walk115Flat(open115 *cloud115.OpenClient) error {
defer st.flushPendingDownloads()
ctx := st.ctx
ctx, cancel := context.WithCancel(st.ctx)
defer cancel()
rootCID := strings.TrimSpace(st.p.RemotePath)
if rootCID == "" {
rootCID = "0"
@@ -452,23 +497,23 @@ func (st *strmSyncState) walk115Flat(open115 *cloud115.OpenClient) error {
if err := st.s.repo.StrmDirCache.DeleteBySyncPathID(ctx, st.p.ID); err != nil {
st.s.log.Warn("delete strm dir cache failed", zap.Error(err))
}
} else {
// 增量同步:预加载历史目录缓存(过滤历史一对多塌陷冲突的脏数据以自愈刷新)
cached, err := st.s.repo.StrmDirCache.ListBySyncPathID(ctx, st.p.ID)
if err == nil {
pathCounts := make(map[string]int, len(cached))
for _, item := range cached {
pathCounts[item.Path]++
}
for _, item := range cached {
// 若同一个 path 对应了多个不同 dir_id,说明包含历史层级塌陷的脏数据,不预加载,让后续步骤重新向 115 获取精确路径
if pathCounts[item.Path] > 1 {
continue
}
st.dirCache.Store(item.DirID, item.Path)
} else {
// 增量同步:预加载历史目录缓存(过滤历史一对多塌陷冲突的脏数据以自愈刷新)
cached, err := st.s.repo.StrmDirCache.ListBySyncPathID(ctx, st.p.ID)
if err == nil {
pathCounts := make(map[string]int, len(cached))
for _, item := range cached {
pathCounts[item.Path]++
}
for _, item := range cached {
// 若同一个 path 对应了多个不同 dir_id,说明包含历史层级塌陷的脏数据,不预加载,让后续步骤重新向 115 获取精确路径
if pathCounts[item.Path] > 1 {
continue
}
st.dirCache.Store(item.DirID, cleanDirRel(item.Path))
}
}
}
// 2. 探测文件总数
const pageSize = 1150
@@ -574,6 +619,8 @@ func (st *strmSyncState) walk115Flat(open115 *cloud115.OpenClient) error {
dirWorkers = 8
doneDirs atomic.Int64
totalDirs = len(pidList)
errMu sync.Mutex
firstErr error
)
if len(pidList) < dirWorkers {
dirWorkers = len(pidList)
@@ -597,10 +644,22 @@ func (st *strmSyncState) walk115Flat(open115 *cloud115.OpenClient) error {
}
detail, err := open115.GetFsDetailByCid(ctx, pid)
if err != nil {
st.s.log.Warn("115: 获取目录详情失败", zap.String("pid", pid), zap.Error(err))
// 目录详情解析失败会导致下游文件 rel 无法还原真实父路径,
// seen key 与磁盘路径对不上:增量 prune 会误删本地文件、上传会
// 误传本地未变文件、下载会重复下载。这里不是降级容错,而是
// 直接中止整个同步——宁可本次同步失败,也不带着损坏的相对路径
// 继续执行造成大规模误删/误传/重下(参考用户反馈"云盘没动却重下重传")。
errMu.Lock()
if firstErr == nil {
firstErr = fmt.Errorf("115: 解析目录树失败(file_id=%s):%w", pid, err)
}
errMu.Unlock()
st.scanIncomplete.Store(true)
cancel()
return
} else if detail != nil {
// 解析相对路径
relPath := detail.RelativePath(rootCID)
relPath := cleanDirRel(detail.RelativePath(rootCID))
st.dirCache.Store(pid, relPath)
_ = st.s.repo.StrmDirCache.Set(ctx, st.p.ID, pid, relPath)
@@ -621,7 +680,7 @@ func (st *strmSyncState) walk115Flat(open115 *cloud115.OpenClient) error {
break
}
}
ancestorRel := subDetail.RelativePath(rootCID)
ancestorRel := cleanDirRel(subDetail.RelativePath(rootCID))
st.dirCache.Store(ancestor.FileId, ancestorRel)
_ = st.s.repo.StrmDirCache.Set(ctx, st.p.ID, ancestor.FileId, ancestorRel)
}
@@ -634,6 +693,12 @@ func (st *strmSyncState) walk115Flat(open115 *cloud115.OpenClient) error {
}()
}
pwg.Wait()
if firstErr != nil {
// 目录树解析失败会导致 rel 塌缩,若继续处理会让大量本地文件
// 被错误判定为"云端不存在"而重复下载/上传,并可能误删本地文件。
// 中止本次同步,避免在损坏的相对路径上执行任何写操作。
return firstErr
}
}
st.updateSyncMessage(fmt.Sprintf("正在生成 STRM 与同步文件 (共 %d 个)...", len(allFiles)))
@@ -649,9 +714,12 @@ func (st *strmSyncState) walk115Flat(open115 *cloud115.OpenClient) error {
rel = cleanName
} else {
if parentVal, ok := st.dirCache.Load(f.Pid); ok && parentVal.(string) != "" {
rel = parentVal.(string) + "/" + cleanName
rel = cleanDirRel(parentVal.(string)) + "/" + cleanName
} else {
rel = cleanName
// 父目录不在目录缓存,无法还原真实相对路径。若继续用塌缩后的
// 根路径处理,该文件会被错误判定,导致重复下载/上传或误删本地文件。
// 目录树不完整时宁可中止本次同步,也不带着损坏的 rel 继续执行。
return fmt.Errorf("115: 文件 %s 的父目录未解析成功,目录树不完整,中止同步以防误删/误传", cleanName)
}
}
entry := cloud.FileEntry{
@@ -686,6 +754,18 @@ func (st *strmSyncState) handleVideo(entry cloud.FileEntry, rel, ext string) {
return
}
st.mu.Lock()
if st.seenVideoTarget == nil {
st.seenVideoTarget = map[string]cloud.FileEntry{}
}
if _, exists := st.seenVideoTarget[target]; exists {
st.mu.Unlock()
st.touchProgress()
return
}
st.seenVideoTarget[target] = entry
st.mu.Unlock()
// 增量同步模式快速检查:本地 strm 文件存在、非空且修改时间与远端 mtime 一致,直接跳过无需读磁盘
if st.syncType == model.StrmSyncTypeIncremental && entry.MTime > 0 {
if info, err := os.Stat(target); err == nil && info.Size() > 0 && info.ModTime().Unix() == entry.MTime {
@@ -835,6 +915,20 @@ func (st *strmSyncState) handleMeta(entry cloud.FileEntry, rel, ext string) {
if err != nil {
return
}
st.mu.Lock()
if st.seenMetaTarget == nil {
st.seenMetaTarget = map[string]cloud.FileEntry{}
}
if _, exists := st.seenMetaTarget[target]; exists {
// 该本地目标路径在当前批次中已被处理(存在同名/重名冲突),直接忽略重复项,避免多份不同大小的文件在本地交替覆盖导致增量死循环
st.mu.Unlock()
st.touchProgress()
return
}
st.seenMetaTarget[target] = entry
st.mu.Unlock()
if info, err := os.Stat(target); err == nil && info.Size() == entry.Size {
st.touchProgress()
return
@@ -936,124 +1030,124 @@ func (st *strmSyncState) walkLocalSource() error {
if err != nil {
return nil
}
target, err := joinLocalRel(st.p.LocalPath, relSansExt+".strm")
if err != nil {
return nil
}
mTime := info.ModTime()
if st.syncType == model.StrmSyncTypeIncremental {
if tInfo, err := os.Stat(target); err == nil && tInfo.Size() > 0 && tInfo.ModTime().Unix() == mTime.Unix() {
st.mu.Lock()
st.rec.Skipped++
st.mu.Unlock()
st.touchProgress()
return nil
}
}
if data, err := os.ReadFile(target); err == nil && string(data) == content {
_ = os.Chtimes(target, mTime, mTime)
target, err := joinLocalRel(st.p.LocalPath, relSansExt+".strm")
if err != nil {
return nil
}
mTime := info.ModTime()
if st.syncType == model.StrmSyncTypeIncremental {
if tInfo, err := os.Stat(target); err == nil && tInfo.Size() > 0 && tInfo.ModTime().Unix() == mTime.Unix() {
st.mu.Lock()
st.rec.Skipped++
st.mu.Unlock()
st.touchProgress()
return nil
}
if err := os.MkdirAll(filepath.Dir(target), 0o755); err != nil {
return nil
}
tmp := target + ".tmp"
if err := os.WriteFile(tmp, []byte(content), 0o644); err == nil {
_ = os.Rename(tmp, target)
_ = os.Chtimes(target, mTime, mTime)
} else {
_ = os.Remove(tmp)
}
}
if data, err := os.ReadFile(target); err == nil && string(data) == content {
_ = os.Chtimes(target, mTime, mTime)
st.mu.Lock()
st.rec.NewStrm++
st.rec.Skipped++
st.mu.Unlock()
st.touchProgress()
return nil
}
if err := os.MkdirAll(filepath.Dir(target), 0o755); err != nil {
return nil
}
tmp := target + ".tmp"
if err := os.WriteFile(tmp, []byte(content), 0o644); err == nil {
_ = os.Rename(tmp, target)
_ = os.Chtimes(target, mTime, mTime)
} else {
_ = os.Remove(tmp)
}
st.mu.Lock()
st.rec.NewStrm++
st.mu.Unlock()
st.touchProgress()
return nil
})
}
// scanLocalMetaForUpload 扫描本地元数据,与远端比对后入上传队列。
func (st *strmSyncState) scanLocalMetaForUpload() error {
defer st.flushPendingUploads()
if st.activeUploadPaths == nil {
if active, err := st.s.repo.StrmUpload.GetActiveLocalPathMap(st.ctx, st.p.ID); err == nil {
st.activeUploadPaths = active
} else {
st.activeUploadPaths = map[string]bool{}
}
// scanLocalMetaForUpload 扫描本地元数据,与远端比对后入上传队列。
func (st *strmSyncState) scanLocalMetaForUpload() error {
defer st.flushPendingUploads()
if st.activeUploadPaths == nil {
if active, err := st.s.repo.StrmUpload.GetActiveLocalPathMap(st.ctx, st.p.ID); err == nil {
st.activeUploadPaths = active
} else {
st.activeUploadPaths = map[string]bool{}
}
localRoot := filepath.Clean(st.p.LocalPath)
return filepath.WalkDir(localRoot, func(path string, d os.DirEntry, err error) error {
if err != nil {
return nil
}
if path == localRoot {
return nil
}
select {
case <-st.ctx.Done():
return st.ctx.Err()
default:
}
if d.IsDir() {
return nil
}
rel, err := filepath.Rel(localRoot, path)
if err != nil {
return nil
}
rel = filepath.ToSlash(rel)
ext := strings.ToLower(filepath.Ext(rel))
if !st.isMetaExt(ext) {
return nil
}
info, err := d.Info()
if err != nil {
return nil
}
st.mu.Lock()
_, exists := st.remoteMeta["m:"+rel]
st.mu.Unlock()
if exists {
// 网盘端已存在该元数据文件,跳过上传
return nil
}
st.mu.Lock()
if st.activeUploadPaths != nil && st.activeUploadPaths[path] {
st.mu.Unlock()
return nil
}
if st.activeUploadPaths != nil {
st.activeUploadPaths[path] = true
}
st.mu.Unlock()
task := &model.StrmUploadTask{
SyncPathID: st.p.ID,
AccountID: st.p.AccountID,
Provider: st.p.Provider,
FileName: filepath.Base(rel),
LocalPath: path,
RemotePath: st.remoteUploadPath(rel),
Size: info.Size(),
Status: model.StrmTaskPending,
}
st.mu.Lock()
st.pendingUploads = append(st.pendingUploads, task)
shouldFlush := len(st.pendingUploads) >= 100
st.rec.Uploaded++
st.mu.Unlock()
if shouldFlush {
st.flushPendingUploads()
}
return nil
})
}
localRoot := filepath.Clean(st.p.LocalPath)
return filepath.WalkDir(localRoot, func(path string, d os.DirEntry, err error) error {
if err != nil {
return nil
}
if path == localRoot {
return nil
}
select {
case <-st.ctx.Done():
return st.ctx.Err()
default:
}
if d.IsDir() {
return nil
}
rel, err := filepath.Rel(localRoot, path)
if err != nil {
return nil
}
rel = filepath.ToSlash(rel)
ext := strings.ToLower(filepath.Ext(rel))
if !st.isMetaExt(ext) {
return nil
}
info, err := d.Info()
if err != nil {
return nil
}
st.mu.Lock()
_, exists := st.remoteMeta["m:"+rel]
st.mu.Unlock()
if exists {
// 网盘端已存在该元数据文件,跳过上传
return nil
}
st.mu.Lock()
if st.activeUploadPaths != nil && st.activeUploadPaths[path] {
st.mu.Unlock()
return nil
}
if st.activeUploadPaths != nil {
st.activeUploadPaths[path] = true
}
st.mu.Unlock()
task := &model.StrmUploadTask{
SyncPathID: st.p.ID,
AccountID: st.p.AccountID,
Provider: st.p.Provider,
FileName: filepath.Base(rel),
LocalPath: path,
RemotePath: st.uploadRemoteTarget(rel),
Size: info.Size(),
Status: model.StrmTaskPending,
}
st.mu.Lock()
st.pendingUploads = append(st.pendingUploads, task)
shouldFlush := len(st.pendingUploads) >= 100
st.rec.Uploaded++
st.mu.Unlock()
if shouldFlush {
st.flushPendingUploads()
}
return nil
})
}
// remoteUploadPath 远端元数据目标路径 = 同步目录远端根 + 相对路径。
func (st *strmSyncState) remoteUploadPath(rel string) string {
@@ -1064,6 +1158,31 @@ func (st *strmSyncState) remoteUploadPath(rel string) string {
return root + "/" + rel
}
// uploadRemoteTarget 返回上传任务的目标远端描述。
// - 115:返回父目录 cid(供 PutFileNamed 定位),基于 dirPathToID 把父目录相对路径映射到 cid。
// - 网盘桥接(clouddrive2/openlist):返回完整远端路径。
func (st *strmSyncState) uploadRemoteTarget(rel string) string {
if st.p.Provider == model.StrmProvider115 {
dir := rel
if idx := strings.LastIndexByte(dir, '/'); idx >= 0 {
dir = dir[:idx]
} else {
dir = ""
}
if dir == "" {
// 文件在同步根目录下,父目录即 115 同步根目录 ID
return st.p.RemotePath
}
if cid, ok := st.dirPathToID[dir]; ok && cid != "" {
return cid
}
// 父目录未在缓存中(父目录可能本次未扫描到),降级为用户配置的同步根 cid,
// 由上传端尽力处理(可能失败记日志,不影响下载)。
return st.p.RemotePath
}
return st.remoteUploadPath(rel)
}
// taskExists 检查是否已有同目录、同目标的进行中/已完成任务(避免重复入队)。
func (st *strmSyncState) taskExists(kind, syncPathID, localPath string) bool {
ctx := st.ctx
@@ -1079,6 +1198,14 @@ func (st *strmSyncState) taskExists(kind, syncPathID, localPath string) bool {
// pruneLocal 清理本地多余 .strm 与元数据(远端已不存在),可选删除空目录。
func (st *strmSyncState) pruneLocal() error {
// 增量同步保护:本次远端扫描不完整(目录详情解析失败 / 文件父路径降级)时,
// seenVideo/seenMeta 覆盖不全,按"远端不存在"清理会误删刚下载或已存在的本地文件,
// 进而触发"下次增量重新下载"的循环。此时跳过清理,仅做进度落库。
if st.syncType == model.StrmSyncTypeIncremental && st.scanIncomplete.Load() {
st.s.log.Warn("strm 增量同步跳过清理:本次远端扫描不完整,prune 已禁用",
zap.String("path_id", st.p.ID))
return nil
}
localRoot := filepath.Clean(st.p.LocalPath)
var dirs []string
err := filepath.WalkDir(localRoot, func(path string, d os.DirEntry, err error) error {

Some files were not shown because too many files have changed in this diff Show More