mirror of
https://github.com/Rain-kl/OpenFlare.git
synced 2026-09-29 14:06:36 +08:00
refactor(migrator): use goose SQL only and auto-import zones on upgrade
移除 Go goose 迁移(bridge/legacy data/zone import),改为 goose SQL 占位 与结构迁移脚本;启动时 Migrate 在删旧列前自动导入历史域名,去掉 migrate-zones 手动命令及文档中的人工导入步骤。
This commit is contained in:
@@ -26,7 +26,7 @@ sidebar: false
|
||||
- 新增第一阶段 Zone 与正规化 Zone 域名数据库表及路由绑定模型,为后续以稳定 ID 管理网站与域名关联提供基础。
|
||||
- 新增 Zone 管理 API 与显式历史域名导入命令,使用公共后缀列表验证注册根域和域名归属。
|
||||
- 管理端网站入口改为 Zone 列表与 `/websites/:zoneId` 详情(概览 / 域名 / 路由 / 证书 / 设置),反代路由通过 Zone 域名选择器绑定。
|
||||
- 新增 Zone 域名迁移指南(`docs/guide/zone-domain-migration.md`),覆盖备份、`wavelet migrate-zones`、预览等价性与回滚步骤。
|
||||
- 新增 Zone 域名迁移指南(`docs/guide/zone-domain-migration.md`);历史域名在 Server 启动时由 goose 自动导入,无需单独命令。
|
||||
|
||||
### 修改
|
||||
|
||||
|
||||
@@ -84,15 +84,15 @@ WAF、Pages、上游与发布版本仍属于 `proxy_routes`。Zone 概览只聚
|
||||
本次改造分两个发布阶段,以免 SQL 用错误的“末两段域名”规则处理多级公共后缀。操作细则见 [Zone 域名迁移与发布验收](../guide/zone-domain-migration.md)。
|
||||
|
||||
1. **第一阶段 DDL**:PostgreSQL 与 SQLite 同版本 Goose 创建 `of_zones` / `of_zone_domains`;暂时保留 `of_managed_domains` 与路由冗余列。
|
||||
2. **数据导入**:`wavelet migrate-zones` 使用 `publicsuffix.EffectiveTLDPlusOne`,从既有路由域名及(无路由域名时的)`managed_domains` 创建 Zone / Zone 域名,按旧 `domain_cert_ids` 位置写入 `zone_domains.cert_id`。冲突时整单回滚并输出报告,禁止在有冲突时发布。
|
||||
2. **数据导入(自动)**:Server 启动时 `migrator.Migrate()` 先应用 goose SQL 至 `202607120002`,再自动导入旧路由域名 / `managed_domains`(`publicsuffix` 解析注册根域,写入 `cert_id` 与 `proxy_route_id`),最后继续后续 SQL。冲突时启动失败;修复后重启可幂等重试。无需手动命令。
|
||||
3. **代码切换**:控制面 API、配置快照、渲染、前端均以 Zone 域名为唯一来源;路由写入仅使用 `zone_domain_ids`。
|
||||
4. **第二阶段清理**:独立 Goose 迁移 `202607130001_drop_legacy_route_domain_columns` 删除 `of_managed_domains` 与 `of_proxy_routes` 的 `domain` / `domains` / `cert_id` / `cert_ids` / `domain_cert_ids`。Down 仅恢复开发库空结构,不回填历史数据。
|
||||
4. **第二阶段清理**:Goose SQL `202607130001_drop_legacy_route_domain_columns` 删除 `of_managed_domains` 与 `of_proxy_routes` 冗余列。Down 仅恢复开发库空结构,不回填历史数据。
|
||||
|
||||
### 运行时模型边界
|
||||
|
||||
* 持久化:域名与证书只存在于 `of_zone_domains`;`of_proxy_routes` 仅保存路由策略(上游、缓存、限流、WAF 绑定键等)。
|
||||
* 渲染:配置快照在内存中组装临时 `Domains` / `DomainCertIDs` 供 OpenResty 渲染,不写回数据库。
|
||||
* 导入命令:第二阶段后若旧列/旧表已不存在则跳过对应源,对已导入 Zone 域名保持幂等。
|
||||
* 结构迁移仅使用 `internal/db/migrator/goose/{postgres,sqlite}/*.sql`;启动时自动导入历史域名,第二阶段后旧列不存在则为空操作。
|
||||
|
||||
## 验证
|
||||
|
||||
|
||||
+1
-1
@@ -11,7 +11,7 @@ OpenFlare 是一套自托管的 OpenResty 控制面。它把反向代理网站
|
||||
1. [快速开始](./quick-start.md):用 Docker Compose 启动 Server,登录管理端,并接入第一个 Agent。
|
||||
2. [发布第一份配置](./first-site.md):快速新建一条最基础的 HTTP 反代站点规则,并验证节点生效状态。
|
||||
3. [新建反代配置](./proxy-config.md):一步一步了解如何从证书导入与申请开始,配置 HTTPS 加密与上游源站管理。
|
||||
4. [Zone 域名迁移](./zone-domain-migration.md):从旧托管域名/路由内嵌域名升级到 Zone 模型,含备份、导入、预览与回滚。
|
||||
4. [Zone 域名迁移](./zone-domain-migration.md):从旧托管域名/路由内嵌域名升级到 Zone 模型(goose 自动导入),含备份、验收与回滚说明。
|
||||
5. [Pages 静态托管使用](./pages-usage.md):了解静态项目 ZIP 上传限制、SPA Fallback、以及内置 API 反向代理配置。
|
||||
6. [内网穿透与隧道使用](./tunnel-usage.md):部署 Relay 与 Client,实现安全、无公网 IP 反向穿透。
|
||||
7. [WAF 安全防护使用](./waf-usage.md):配置 WAF 规则组,掌握 IP 黑白名单、自动/订阅 IP 组、地域限制与 PoW CC 防护。
|
||||
|
||||
@@ -1,95 +1,49 @@
|
||||
# Zone 域名迁移与发布验收
|
||||
|
||||
从旧版 `managed_domains` / 反代路由内嵌域名列迁移到 Zone + Zone 域名模型时,按本指南操作。**导入报告存在冲突时禁止继续发布。**
|
||||
从旧版 `managed_domains` / 反代路由内嵌域名列迁移到 Zone + Zone 域名模型时,数据导入与表结构升级均由 **Server 启动时的 goose 自动迁移**完成,无需单独执行导入命令。
|
||||
|
||||
## 前置条件
|
||||
## 升级时发生了什么
|
||||
|
||||
* 已备份 PostgreSQL / SQLite 数据库与当前激活配置版本(可导出管理端「配置版本」中的激活快照)。
|
||||
* Server 二进制已升级到包含 `of_zones` / `of_zone_domains` 表迁移的版本。
|
||||
* 维护窗口内可暂停非必要配置发布。
|
||||
启动(或滚动升级)包含 Zone 改造的 Server 版本时,**无需手动命令**,`migrator.Migrate()` 自动:
|
||||
|
||||
## 1. 备份
|
||||
1. 应用 goose SQL:创建 `of_zones` / `of_zone_domains`(若尚未存在)。
|
||||
2. **自动导入**旧路由域名列(及无路由域名时的 `of_managed_domains`)为 Zone / Zone 域名,并绑定 `proxy_route_id` / `cert_id`(公共后缀列表解析注册根域)。
|
||||
3. 继续 goose SQL:删除 `of_managed_domains` 与 `of_proxy_routes` 冗余域名/证书列。
|
||||
|
||||
导入幂等:已存在的域名会跳过或补绑路由。
|
||||
|
||||
**若历史数据无法解析(冲突域名、无效根域、证书不存在等),启动失败。** 修复数据或恢复备份后再次启动即可重试。
|
||||
|
||||
## 建议操作
|
||||
|
||||
### 1. 升级前备份
|
||||
|
||||
```bash
|
||||
# PostgreSQL 示例
|
||||
pg_dump "$DATABASE_URL" > openflare-pre-zone-$(date +%Y%m%d).sql
|
||||
|
||||
# 或复制备份卷 / 快照;SQLite 则直接复制 data 目录中的库文件
|
||||
# 或复制备份卷 / 快照;SQLite 则复制 data 目录中的库文件
|
||||
```
|
||||
|
||||
在管理端确认当前**激活版本号**并记下 checksum,便于回滚对比。
|
||||
可选:在管理端记下当前**激活配置版本号**与 checksum,便于配置回滚对比。
|
||||
|
||||
## 2. 执行历史导入
|
||||
### 2. 升级并启动 Server
|
||||
|
||||
```bash
|
||||
# 二进制名称为 wavelet(或你的部署包中的同名入口)
|
||||
wavelet migrate-zones
|
||||
```
|
||||
部署新版本并启动即可。观察启动日志中的 goose 成功信息;若出现「迁移 Zone 失败(N 个冲突)」则按日志中的冲突项修复源数据后重启。
|
||||
|
||||
命令会:
|
||||
### 3. 升级后检查
|
||||
|
||||
1. 先跑 goose 迁移(确保 Zone 表存在)。
|
||||
2. 以事务从旧路由域名(及无路由域名时的 `of_managed_domains`)导入 Zone / Zone 域名。
|
||||
3. 使用公共后缀列表解析注册根域;冲突时整单回滚并输出报告。
|
||||
1. 管理端 **网站** `/websites`:Zone 根域与域名计数是否合理。
|
||||
2. Zone 详情:域名、证书、关联路由 ID。
|
||||
3. **反代路由**:域名绑定来自 Zone 域名,而非旧手写字段。
|
||||
|
||||
**成功标志:** 进程退出码 0,且日志/标准输出无「冲突」列表。
|
||||
### 4. 配置预览与发布
|
||||
|
||||
**失败时:** 阅读冲突项(无法解析的根域、通配符 FQDN、全局域名冲突、证书不存在等),修复源数据后重新执行。`migrate-zones` 幂等:已导入的域名会跳过,不会重复创建。
|
||||
|
||||
**有冲突时不要发布配置、不要执行第二阶段删列迁移。**
|
||||
|
||||
## 3. 导入后检查
|
||||
|
||||
1. 打开管理端 **网站** `/websites`:确认 Zone 根域与域名计数合理。
|
||||
2. 进入各 Zone 详情:域名、证书绑定、关联路由 ID 是否正确。
|
||||
3. 打开 **反代路由**:域名区应展示 Zone 域名绑定,而不是手写域名。
|
||||
|
||||
## 4. 配置预览与快照等价性
|
||||
|
||||
在升级前若已导出激活快照,导入后:
|
||||
|
||||
1. 在管理端打开配置差异 / 预览(或调用配置 diff / preview API)。
|
||||
2. **逐路由**核对:
|
||||
* 明确 `server_name` 集合(全部 FQDN)
|
||||
* 证书支持文件路径 / 证书 ID 与域名对应关系
|
||||
* WAF 绑定的 Route ID(`site_name` 与路由 ID 不变)
|
||||
* Pages 项目引用
|
||||
3. **允许**旧快照 JSON 中路由上的冗余 `domain` / `domains` / `cert_ids` 字段消失。
|
||||
4. **不允许**数据面语义变化(域名集合、证书覆盖、上游、WAF、Pages 绑定)。
|
||||
|
||||
不一致时:停止发布,修正 Zone 域名/证书绑定后重新预览。
|
||||
|
||||
## 5. 发布与回滚
|
||||
|
||||
1. 预览通过后,在管理端执行**配置发布**,记录新版本号。
|
||||
2. 用根域与各子域发起 HTTP(S) 请求,确认节点应用成功。
|
||||
3. **回滚:** 在配置版本中重新激活导入前的版本;节点会拉取旧快照。数据库侧若需回退,使用升级前备份恢复(第二阶段删列后 Down 迁移不回填业务数据)。
|
||||
|
||||
## 6. 第二阶段:删除旧表与冗余列
|
||||
|
||||
仅在以下条件全部满足后执行:
|
||||
|
||||
* `migrate-zones` 无冲突
|
||||
* 至少完成一次预览对比与发布(及必要时的回滚演练)
|
||||
* 运维确认不再依赖 `of_managed_domains` 与 `of_proxy_routes` 上的 `domain` / `domains` / `cert_*` 列
|
||||
|
||||
然后升级到包含 `202607130001_drop_legacy_route_domain_columns` 的版本并启动 Server(自动 goose)。该迁移将:
|
||||
|
||||
* 删除 `of_proxy_routes` 的 `domain`、`domains`、`cert_id`、`cert_ids`、`domain_cert_ids`
|
||||
* 删除表 `of_managed_domains`
|
||||
|
||||
**不可逆业务数据:** Down 仅在开发库重建空结构,不恢复历史域名行。
|
||||
|
||||
## 7. 命令速查
|
||||
|
||||
| 步骤 | 命令 / 操作 |
|
||||
| --- | --- |
|
||||
| 备份 | `pg_dump` / 复制 SQLite 文件 |
|
||||
| 导入 | `wavelet migrate-zones` |
|
||||
| 预览 | 管理端配置差异 / Preview API |
|
||||
| 发布 | 管理端发布激活版本 |
|
||||
| 回滚配置 | 管理端激活旧版本 |
|
||||
| 回滚库 | 恢复备份(勿依赖 Down 填数) |
|
||||
1. 在管理端查看配置差异 / 预览。
|
||||
2. **逐路由**核对:`server_name` 集合、证书路径、WAF Route ID、Pages 引用。
|
||||
3. **允许**旧快照 JSON 中路由上的冗余 `domain` / `domains` / `cert_ids` 消失。
|
||||
4. **不允许**数据面语义变化。
|
||||
5. 预览通过后发布;需要时在配置版本中激活升级前版本做配置回滚。数据库回退请使用升级前备份(Down 迁移不回填业务域名数据)。
|
||||
|
||||
## 相关文档
|
||||
|
||||
|
||||
@@ -36,8 +36,8 @@ func NormalizeDomains(rawDomains []string) ([]string, error) {
|
||||
return normalized, nil
|
||||
}
|
||||
|
||||
// DecodeDomains parses legacy route domain fields for the explicit migration
|
||||
// command. Runtime consumers must read ZoneDomain bindings instead.
|
||||
// DecodeDomains parses legacy route domain fields for the goose upgrade importer.
|
||||
// Runtime consumers must read ZoneDomain bindings instead.
|
||||
func DecodeDomains(raw string, fallbackDomain string) ([]string, error) {
|
||||
text := strings.TrimSpace(raw)
|
||||
if text == "" {
|
||||
|
||||
@@ -5,14 +5,14 @@ package zone
|
||||
|
||||
import (
|
||||
"context"
|
||||
"database/sql"
|
||||
"encoding/json"
|
||||
"errors"
|
||||
"fmt"
|
||||
"strconv"
|
||||
"strings"
|
||||
|
||||
"github.com/Rain-kl/Wavelet/internal/apps/openflare/routeidentity"
|
||||
"github.com/Rain-kl/Wavelet/internal/db"
|
||||
"github.com/Rain-kl/Wavelet/internal/model"
|
||||
"gorm.io/gorm"
|
||||
)
|
||||
|
||||
// ImportReport describes the idempotent legacy migration result.
|
||||
@@ -31,138 +31,244 @@ func (r ImportReport) LogAndReturn(err error) error {
|
||||
}
|
||||
|
||||
type legacyDomain struct {
|
||||
Domain string
|
||||
CertID *uint
|
||||
Remark string
|
||||
Domain string
|
||||
CertID *uint
|
||||
Remark string
|
||||
ProxyRouteID *uint
|
||||
}
|
||||
|
||||
// legacyRouteRow reads pre-cleanup of_proxy_routes columns via raw scan.
|
||||
type legacyRouteRow struct {
|
||||
ID uint `gorm:"column:id"`
|
||||
Domain string `gorm:"column:domain"`
|
||||
Domains string `gorm:"column:domains"`
|
||||
DomainCertIDs string `gorm:"column:domain_cert_ids"`
|
||||
Remark string `gorm:"column:remark"`
|
||||
}
|
||||
|
||||
// legacyManagedRow reads of_managed_domains while the table still exists.
|
||||
type legacyManagedRow struct {
|
||||
Domain string `gorm:"column:domain"`
|
||||
CertID *uint `gorm:"column:cert_id"`
|
||||
Remark string `gorm:"column:remark"`
|
||||
}
|
||||
|
||||
// ImportLegacy imports legacy proxy-route / managed-domain rows into Zone tables.
|
||||
// After the phase-2 schema cleanup, missing legacy columns or tables are skipped.
|
||||
// ImportLegacyTx imports legacy proxy-route / managed-domain rows into Zone tables
|
||||
// within an existing SQL transaction (goose runs this on Server upgrade).
|
||||
// postgres selects $n placeholders; otherwise SQLite-style ? is used.
|
||||
// Missing legacy columns or tables are skipped so re-runs after phase-2 cleanup are no-ops.
|
||||
//
|
||||
//nolint:cyclop // the transactional importer intentionally validates every legacy source in one pass.
|
||||
func ImportLegacy(ctx context.Context) (report ImportReport, resultErr error) {
|
||||
conn := db.DB(ctx)
|
||||
if conn == nil {
|
||||
return report, fmt.Errorf("database is not initialized")
|
||||
//nolint:cyclop,gocyclo // single-pass legacy importer validates every source before write.
|
||||
func ImportLegacyTx(ctx context.Context, tx *sql.Tx, postgres bool) (report ImportReport, err error) {
|
||||
if tx == nil {
|
||||
return report, errors.New("transaction is required")
|
||||
}
|
||||
resultErr = conn.Transaction(func(tx *gorm.DB) error {
|
||||
items := make([]legacyDomain, 0)
|
||||
hasRouteDomains := false
|
||||
q := func(sqlText string) string { return rebindSQL(sqlText, postgres) }
|
||||
|
||||
if tx.Migrator().HasColumn("of_proxy_routes", "domain") &&
|
||||
tx.Migrator().HasColumn("of_proxy_routes", "domains") {
|
||||
var routes []legacyRouteRow
|
||||
if err := tx.Table("of_proxy_routes").
|
||||
Select("id, domain, domains, domain_cert_ids, remark").
|
||||
Find(&routes).Error; err != nil {
|
||||
return err
|
||||
}
|
||||
for _, route := range routes {
|
||||
domains, err := routeidentity.DecodeDomains(route.Domains, route.Domain)
|
||||
if err != nil {
|
||||
report.Conflicts = append(report.Conflicts, fmt.Sprintf("route %d: %v", route.ID, err))
|
||||
continue
|
||||
}
|
||||
if len(domains) > 0 {
|
||||
hasRouteDomains = true
|
||||
}
|
||||
certIDs := decodeLegacyCertIDs(route.DomainCertIDs, len(domains))
|
||||
for i, domain := range domains {
|
||||
var certID *uint
|
||||
if i < len(certIDs) && certIDs[i] > 0 {
|
||||
v := certIDs[i]
|
||||
certID = &v
|
||||
}
|
||||
items = append(items, legacyDomain{Domain: domain, CertID: certID, Remark: route.Remark})
|
||||
}
|
||||
items := make([]legacyDomain, 0)
|
||||
hasRouteDomains := false
|
||||
|
||||
hasDomainCol, err := hasTableColumn(ctx, tx, q, postgres, "of_proxy_routes", "domain")
|
||||
if err != nil {
|
||||
return report, err
|
||||
}
|
||||
hasDomainsCol, err := hasTableColumn(ctx, tx, q, postgres, "of_proxy_routes", "domains")
|
||||
if err != nil {
|
||||
return report, err
|
||||
}
|
||||
if hasDomainCol && hasDomainsCol {
|
||||
var collectErr error
|
||||
items, hasRouteDomains, report.Conflicts, collectErr = collectLegacyRouteDomainsImpl(ctx, tx, q)
|
||||
if collectErr != nil {
|
||||
return report, collectErr
|
||||
}
|
||||
}
|
||||
|
||||
if !hasRouteDomains {
|
||||
exists, tableErr := hasTable(ctx, tx, q, postgres, "of_managed_domains")
|
||||
if tableErr != nil {
|
||||
return report, tableErr
|
||||
}
|
||||
if exists {
|
||||
managed, managedErr := collectLegacyManagedDomains(ctx, tx, q)
|
||||
if managedErr != nil {
|
||||
return report, managedErr
|
||||
}
|
||||
items = append(items, managed...)
|
||||
}
|
||||
}
|
||||
|
||||
for _, item := range items {
|
||||
domain, normErr := normalizeDomain(item.Domain)
|
||||
if normErr != nil {
|
||||
report.Conflicts = append(report.Conflicts, fmt.Sprintf("%s: %v", item.Domain, normErr))
|
||||
continue
|
||||
}
|
||||
root, rootErr := zoneRoot(domain)
|
||||
if rootErr != nil {
|
||||
report.Conflicts = append(report.Conflicts, fmt.Sprintf("%s: %v", domain, rootErr))
|
||||
continue
|
||||
}
|
||||
|
||||
if !hasRouteDomains && tx.Migrator().HasTable("of_managed_domains") {
|
||||
var legacy []legacyManagedRow
|
||||
if err := tx.Table("of_managed_domains").
|
||||
Select("domain, cert_id, remark").
|
||||
Find(&legacy).Error; err != nil {
|
||||
return err
|
||||
}
|
||||
for _, item := range legacy {
|
||||
items = append(items, legacyDomain(item))
|
||||
var existingID uint
|
||||
var existingZoneDomain string
|
||||
scanErr := tx.QueryRowContext(ctx, q(`
|
||||
SELECT zd.id, z.domain
|
||||
FROM of_zone_domains zd
|
||||
JOIN of_zones z ON z.id = zd.zone_id
|
||||
WHERE zd.domain = ?
|
||||
`), domain).Scan(&existingID, &existingZoneDomain)
|
||||
if scanErr == nil {
|
||||
if existingZoneDomain != root {
|
||||
report.Conflicts = append(report.Conflicts, fmt.Sprintf("%s: global domain conflict", domain))
|
||||
} else if item.ProxyRouteID != nil {
|
||||
if _, bindErr := tx.ExecContext(ctx, q(`
|
||||
UPDATE of_zone_domains
|
||||
SET proxy_route_id = COALESCE(proxy_route_id, ?),
|
||||
cert_id = COALESCE(cert_id, ?)
|
||||
WHERE id = ?
|
||||
`), *item.ProxyRouteID, nullableUint(item.CertID), existingID); bindErr != nil {
|
||||
return report, bindErr
|
||||
}
|
||||
}
|
||||
continue
|
||||
}
|
||||
if !errors.Is(scanErr, sql.ErrNoRows) {
|
||||
return report, scanErr
|
||||
}
|
||||
|
||||
for _, item := range items {
|
||||
domain, err := normalizeDomain(item.Domain)
|
||||
if err != nil {
|
||||
report.Conflicts = append(report.Conflicts, fmt.Sprintf("%s: %v", item.Domain, err))
|
||||
continue
|
||||
}
|
||||
root, err := zoneRoot(domain)
|
||||
if err != nil {
|
||||
report.Conflicts = append(report.Conflicts, fmt.Sprintf("%s: %v", domain, err))
|
||||
continue
|
||||
}
|
||||
var existing model.ZoneDomain
|
||||
err = tx.Where("domain = ?", domain).First(&existing).Error
|
||||
if err == nil {
|
||||
var z model.Zone
|
||||
if tx.First(&z, existing.ZoneID).Error != nil || z.Domain != root {
|
||||
report.Conflicts = append(report.Conflicts, fmt.Sprintf("%s: global domain conflict", domain))
|
||||
}
|
||||
continue
|
||||
}
|
||||
if err != nil && !isNotFound(err) {
|
||||
return err
|
||||
}
|
||||
var zone model.Zone
|
||||
err = tx.Where("domain = ?", root).First(&zone).Error
|
||||
if isNotFound(err) {
|
||||
zone = model.Zone{Domain: root}
|
||||
if err = tx.Create(&zone).Error; err != nil {
|
||||
return err
|
||||
}
|
||||
report.Zones++
|
||||
} else if err != nil {
|
||||
return err
|
||||
}
|
||||
if item.CertID != nil {
|
||||
var cert model.TLSCertificate
|
||||
if err = tx.First(&cert, *item.CertID).Error; err != nil {
|
||||
zoneID, zoneErr := ensureZone(ctx, tx, q, root, &report)
|
||||
if zoneErr != nil {
|
||||
return report, zoneErr
|
||||
}
|
||||
|
||||
if item.CertID != nil {
|
||||
var certID uint
|
||||
if certErr := tx.QueryRowContext(ctx, q(`SELECT id FROM of_tls_certificates WHERE id = ?`), *item.CertID).
|
||||
Scan(&certID); certErr != nil {
|
||||
if errors.Is(certErr, sql.ErrNoRows) {
|
||||
report.Conflicts = append(report.Conflicts, fmt.Sprintf("%s: %s", domain, errCertificateNotFound))
|
||||
continue
|
||||
}
|
||||
return report, certErr
|
||||
}
|
||||
if err = tx.Create(&model.ZoneDomain{
|
||||
ZoneID: zone.ID,
|
||||
Domain: domain,
|
||||
CertID: item.CertID,
|
||||
Remark: item.Remark,
|
||||
}).Error; err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
if _, insErr := tx.ExecContext(ctx, q(`
|
||||
INSERT INTO of_zone_domains (zone_id, proxy_route_id, domain, cert_id, remark, created_at, updated_at)
|
||||
VALUES (?, ?, ?, ?, ?, CURRENT_TIMESTAMP, CURRENT_TIMESTAMP)
|
||||
`), zoneID, nullableUint(item.ProxyRouteID), domain, nullableUint(item.CertID), item.Remark); insErr != nil {
|
||||
return report, insErr
|
||||
}
|
||||
report.Domains++
|
||||
}
|
||||
|
||||
if len(report.Conflicts) > 0 {
|
||||
return report, fmt.Errorf("legacy data has conflicts")
|
||||
}
|
||||
return report, nil
|
||||
}
|
||||
|
||||
func ensureZone(
|
||||
ctx context.Context,
|
||||
tx *sql.Tx,
|
||||
q func(string) string,
|
||||
root string,
|
||||
report *ImportReport,
|
||||
) (uint, error) {
|
||||
var zoneID uint
|
||||
err := tx.QueryRowContext(ctx, q(`SELECT id FROM of_zones WHERE domain = ?`), root).Scan(&zoneID)
|
||||
if err == nil {
|
||||
return zoneID, nil
|
||||
}
|
||||
if !errors.Is(err, sql.ErrNoRows) {
|
||||
return 0, err
|
||||
}
|
||||
if _, execErr := tx.ExecContext(ctx, q(`
|
||||
INSERT INTO of_zones (domain, remark, created_at, updated_at)
|
||||
VALUES (?, '', CURRENT_TIMESTAMP, CURRENT_TIMESTAMP)
|
||||
`), root); execErr != nil {
|
||||
return 0, execErr
|
||||
}
|
||||
if err := tx.QueryRowContext(ctx, q(`SELECT id FROM of_zones WHERE domain = ?`), root).Scan(&zoneID); err != nil {
|
||||
return 0, err
|
||||
}
|
||||
report.Zones++
|
||||
return zoneID, nil
|
||||
}
|
||||
|
||||
func collectLegacyRouteDomainsImpl(
|
||||
ctx context.Context,
|
||||
tx *sql.Tx,
|
||||
q func(string) string,
|
||||
) (items []legacyDomain, hasRouteDomains bool, conflicts []string, err error) {
|
||||
// Probe domain_cert_ids: if SELECT fails, fall back without it.
|
||||
queryWithCert := q(`SELECT id, domain, domains, COALESCE(domain_cert_ids, '[]'), remark FROM of_proxy_routes`)
|
||||
rows, err := tx.QueryContext(ctx, queryWithCert)
|
||||
useCert := true
|
||||
if err != nil {
|
||||
useCert = false
|
||||
rows, err = tx.QueryContext(ctx, q(`SELECT id, domain, domains, remark FROM of_proxy_routes`))
|
||||
if err != nil {
|
||||
return nil, false, nil, err
|
||||
}
|
||||
}
|
||||
defer func() { _ = rows.Close() }()
|
||||
|
||||
for rows.Next() {
|
||||
var (
|
||||
id uint
|
||||
domain string
|
||||
domains string
|
||||
certIDs string
|
||||
remark string
|
||||
)
|
||||
if useCert {
|
||||
if err := rows.Scan(&id, &domain, &domains, &certIDs, &remark); err != nil {
|
||||
return nil, false, nil, err
|
||||
}
|
||||
report.Domains++
|
||||
} else {
|
||||
if err := rows.Scan(&id, &domain, &domains, &remark); err != nil {
|
||||
return nil, false, nil, err
|
||||
}
|
||||
certIDs = "[]"
|
||||
}
|
||||
if len(report.Conflicts) > 0 {
|
||||
return fmt.Errorf("legacy data has conflicts")
|
||||
decoded, decodeErr := routeidentity.DecodeDomains(domains, domain)
|
||||
if decodeErr != nil {
|
||||
conflicts = append(conflicts, fmt.Sprintf("route %d: %v", id, decodeErr))
|
||||
continue
|
||||
}
|
||||
return nil
|
||||
})
|
||||
return report, resultErr
|
||||
if len(decoded) > 0 {
|
||||
hasRouteDomains = true
|
||||
}
|
||||
ids := decodeLegacyCertIDs(certIDs, len(decoded))
|
||||
routeID := id
|
||||
for i, d := range decoded {
|
||||
var certID *uint
|
||||
if i < len(ids) && ids[i] > 0 {
|
||||
v := ids[i]
|
||||
certID = &v
|
||||
}
|
||||
items = append(items, legacyDomain{
|
||||
Domain: d,
|
||||
CertID: certID,
|
||||
Remark: remark,
|
||||
ProxyRouteID: &routeID,
|
||||
})
|
||||
}
|
||||
}
|
||||
return items, hasRouteDomains, conflicts, rows.Err()
|
||||
}
|
||||
|
||||
func collectLegacyManagedDomains(ctx context.Context, tx *sql.Tx, q func(string) string) ([]legacyDomain, error) {
|
||||
rows, err := tx.QueryContext(ctx, q(`SELECT domain, cert_id, remark FROM of_managed_domains`))
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
defer func() { _ = rows.Close() }()
|
||||
|
||||
items := make([]legacyDomain, 0)
|
||||
for rows.Next() {
|
||||
var (
|
||||
domain string
|
||||
certID sql.NullInt64
|
||||
remark string
|
||||
)
|
||||
if err := rows.Scan(&domain, &certID, &remark); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
item := legacyDomain{Domain: domain, Remark: remark}
|
||||
if certID.Valid && certID.Int64 > 0 {
|
||||
v := uint(certID.Int64)
|
||||
item.CertID = &v
|
||||
}
|
||||
items = append(items, item)
|
||||
}
|
||||
return items, rows.Err()
|
||||
}
|
||||
|
||||
func decodeLegacyCertIDs(raw string, count int) []uint {
|
||||
@@ -176,4 +282,78 @@ func decodeLegacyCertIDs(raw string, count int) []uint {
|
||||
return values
|
||||
}
|
||||
|
||||
func isNotFound(err error) bool { return err == gorm.ErrRecordNotFound }
|
||||
func nullableUint(v *uint) any {
|
||||
if v == nil {
|
||||
return nil
|
||||
}
|
||||
return *v
|
||||
}
|
||||
|
||||
func rebindSQL(query string, postgres bool) string {
|
||||
if !postgres {
|
||||
return query
|
||||
}
|
||||
var b strings.Builder
|
||||
b.Grow(len(query) + len(query)/4)
|
||||
n := 0
|
||||
for i := 0; i < len(query); i++ {
|
||||
if query[i] == '?' {
|
||||
n++
|
||||
b.WriteByte('$')
|
||||
b.WriteString(strconv.Itoa(n))
|
||||
continue
|
||||
}
|
||||
b.WriteByte(query[i])
|
||||
}
|
||||
return b.String()
|
||||
}
|
||||
|
||||
func hasTable(
|
||||
ctx context.Context,
|
||||
tx *sql.Tx,
|
||||
_ func(string) string,
|
||||
postgres bool,
|
||||
table string,
|
||||
) (bool, error) {
|
||||
var count int
|
||||
var err error
|
||||
if postgres {
|
||||
err = tx.QueryRowContext(ctx, `
|
||||
SELECT COUNT(*) FROM information_schema.tables
|
||||
WHERE table_schema = 'public' AND table_name = $1
|
||||
`, table).Scan(&count)
|
||||
} else {
|
||||
err = tx.QueryRowContext(ctx,
|
||||
`SELECT COUNT(*) FROM sqlite_master WHERE type = 'table' AND name = ?`, table,
|
||||
).Scan(&count)
|
||||
}
|
||||
if err != nil {
|
||||
return false, err
|
||||
}
|
||||
return count > 0, nil
|
||||
}
|
||||
|
||||
func hasTableColumn(
|
||||
ctx context.Context,
|
||||
tx *sql.Tx,
|
||||
_ func(string) string,
|
||||
postgres bool,
|
||||
table, column string,
|
||||
) (bool, error) {
|
||||
var count int
|
||||
var err error
|
||||
if postgres {
|
||||
err = tx.QueryRowContext(ctx, `
|
||||
SELECT COUNT(*) FROM information_schema.columns
|
||||
WHERE table_schema = 'public' AND table_name = $1 AND column_name = $2
|
||||
`, table, column).Scan(&count)
|
||||
} else {
|
||||
err = tx.QueryRowContext(ctx,
|
||||
`SELECT COUNT(*) FROM pragma_table_info(?) WHERE name = ?`, table, column,
|
||||
).Scan(&count)
|
||||
}
|
||||
if err != nil {
|
||||
return false, err
|
||||
}
|
||||
return count > 0, nil
|
||||
}
|
||||
|
||||
@@ -0,0 +1,154 @@
|
||||
// Copyright 2026 Arctel.net
|
||||
// SPDX-License-Identifier: Apache-2.0
|
||||
|
||||
package zone
|
||||
|
||||
import (
|
||||
"context"
|
||||
"database/sql"
|
||||
"testing"
|
||||
|
||||
"github.com/Rain-kl/Wavelet/internal/db"
|
||||
"github.com/glebarez/sqlite"
|
||||
"github.com/stretchr/testify/assert"
|
||||
"github.com/stretchr/testify/require"
|
||||
"gorm.io/gorm"
|
||||
)
|
||||
|
||||
func setupLegacyImportDB(t *testing.T) (*sql.DB, func()) {
|
||||
t.Helper()
|
||||
gormDB, err := gorm.Open(sqlite.Open(":memory:"), &gorm.Config{})
|
||||
require.NoError(t, err)
|
||||
sqlDB, err := gormDB.DB()
|
||||
require.NoError(t, err)
|
||||
|
||||
// Pre-phase-2 schema: legacy route columns + managed domains + zone tables.
|
||||
stmts := []string{
|
||||
`CREATE TABLE of_zones (
|
||||
id INTEGER PRIMARY KEY AUTOINCREMENT,
|
||||
domain TEXT NOT NULL UNIQUE,
|
||||
remark TEXT NOT NULL DEFAULT '',
|
||||
created_at DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP,
|
||||
updated_at DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP
|
||||
)`,
|
||||
`CREATE TABLE of_zone_domains (
|
||||
id INTEGER PRIMARY KEY AUTOINCREMENT,
|
||||
zone_id INTEGER NOT NULL,
|
||||
proxy_route_id INTEGER,
|
||||
domain TEXT NOT NULL UNIQUE,
|
||||
cert_id INTEGER,
|
||||
remark TEXT NOT NULL DEFAULT '',
|
||||
created_at DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP,
|
||||
updated_at DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP
|
||||
)`,
|
||||
`CREATE TABLE of_proxy_routes (
|
||||
id INTEGER PRIMARY KEY AUTOINCREMENT,
|
||||
site_name TEXT NOT NULL DEFAULT '',
|
||||
domain TEXT NOT NULL DEFAULT '',
|
||||
domains TEXT NOT NULL DEFAULT '[]',
|
||||
domain_cert_ids TEXT NOT NULL DEFAULT '[]',
|
||||
origin_url TEXT NOT NULL DEFAULT '',
|
||||
remark TEXT NOT NULL DEFAULT ''
|
||||
)`,
|
||||
`CREATE TABLE of_tls_certificates (
|
||||
id INTEGER PRIMARY KEY AUTOINCREMENT,
|
||||
name TEXT NOT NULL DEFAULT ''
|
||||
)`,
|
||||
`CREATE TABLE of_managed_domains (
|
||||
id INTEGER PRIMARY KEY AUTOINCREMENT,
|
||||
domain TEXT NOT NULL,
|
||||
cert_id INTEGER,
|
||||
remark TEXT NOT NULL DEFAULT ''
|
||||
)`,
|
||||
}
|
||||
for _, stmt := range stmts {
|
||||
_, err := sqlDB.Exec(stmt)
|
||||
require.NoError(t, err)
|
||||
}
|
||||
|
||||
previous := db.DB(context.Background())
|
||||
db.SetDB(gormDB)
|
||||
return sqlDB, func() {
|
||||
db.SetDB(previous)
|
||||
_ = sqlDB.Close()
|
||||
}
|
||||
}
|
||||
|
||||
func TestImportLegacyTxBindsRouteDomains(t *testing.T) {
|
||||
sqlDB, cleanup := setupLegacyImportDB(t)
|
||||
defer cleanup()
|
||||
ctx := context.Background()
|
||||
|
||||
_, err := sqlDB.Exec(`INSERT INTO of_tls_certificates (id, name) VALUES (7, 'cert')`)
|
||||
require.NoError(t, err)
|
||||
_, err = sqlDB.Exec(`
|
||||
INSERT INTO of_proxy_routes (id, site_name, domain, domains, domain_cert_ids, origin_url, remark)
|
||||
VALUES (3, 'api', 'api.example.com', '["api.example.com","www.example.com"]', '[7,7]', 'http://origin', 'r')
|
||||
`)
|
||||
require.NoError(t, err)
|
||||
|
||||
tx, err := sqlDB.Begin()
|
||||
require.NoError(t, err)
|
||||
report, err := ImportLegacyTx(ctx, tx, false)
|
||||
require.NoError(t, err)
|
||||
require.NoError(t, tx.Commit())
|
||||
|
||||
assert.Equal(t, 1, report.Zones)
|
||||
assert.Equal(t, 2, report.Domains)
|
||||
|
||||
var zoneDomain string
|
||||
require.NoError(t, sqlDB.QueryRow(`SELECT domain FROM of_zones`).Scan(&zoneDomain))
|
||||
assert.Equal(t, "example.com", zoneDomain)
|
||||
|
||||
var count int
|
||||
require.NoError(t, sqlDB.QueryRow(`SELECT COUNT(*) FROM of_zone_domains WHERE proxy_route_id = 3`).Scan(&count))
|
||||
assert.Equal(t, 2, count)
|
||||
|
||||
// Idempotent re-run
|
||||
tx, err = sqlDB.Begin()
|
||||
require.NoError(t, err)
|
||||
report2, err := ImportLegacyTx(ctx, tx, false)
|
||||
require.NoError(t, err)
|
||||
require.NoError(t, tx.Commit())
|
||||
assert.Equal(t, 0, report2.Domains)
|
||||
}
|
||||
|
||||
func TestImportLegacyTxNoOpWithoutLegacyColumns(t *testing.T) {
|
||||
gormDB, err := gorm.Open(sqlite.Open(":memory:"), &gorm.Config{})
|
||||
require.NoError(t, err)
|
||||
sqlDB, err := gormDB.DB()
|
||||
require.NoError(t, err)
|
||||
defer sqlDB.Close()
|
||||
|
||||
_, err = sqlDB.Exec(`
|
||||
CREATE TABLE of_zones (
|
||||
id INTEGER PRIMARY KEY AUTOINCREMENT,
|
||||
domain TEXT NOT NULL UNIQUE,
|
||||
remark TEXT NOT NULL DEFAULT '',
|
||||
created_at DATETIME, updated_at DATETIME
|
||||
);
|
||||
CREATE TABLE of_zone_domains (
|
||||
id INTEGER PRIMARY KEY AUTOINCREMENT,
|
||||
zone_id INTEGER NOT NULL,
|
||||
proxy_route_id INTEGER,
|
||||
domain TEXT NOT NULL UNIQUE,
|
||||
cert_id INTEGER,
|
||||
remark TEXT NOT NULL DEFAULT '',
|
||||
created_at DATETIME, updated_at DATETIME
|
||||
);
|
||||
CREATE TABLE of_proxy_routes (
|
||||
id INTEGER PRIMARY KEY AUTOINCREMENT,
|
||||
site_name TEXT NOT NULL DEFAULT '',
|
||||
origin_url TEXT NOT NULL DEFAULT ''
|
||||
);
|
||||
`)
|
||||
require.NoError(t, err)
|
||||
|
||||
tx, err := sqlDB.Begin()
|
||||
require.NoError(t, err)
|
||||
report, err := ImportLegacyTx(context.Background(), tx, false)
|
||||
require.NoError(t, err)
|
||||
require.NoError(t, tx.Commit())
|
||||
assert.Equal(t, 0, report.Zones)
|
||||
assert.Equal(t, 0, report.Domains)
|
||||
}
|
||||
@@ -1,21 +0,0 @@
|
||||
// Copyright 2026 Arctel.net
|
||||
// SPDX-License-Identifier: Apache-2.0
|
||||
|
||||
package cmd
|
||||
|
||||
import (
|
||||
"context"
|
||||
|
||||
"github.com/Rain-kl/Wavelet/internal/apps/openflare/zone"
|
||||
"github.com/Rain-kl/Wavelet/internal/db/migrator"
|
||||
"github.com/spf13/cobra"
|
||||
)
|
||||
|
||||
var migrateZonesCmd = &cobra.Command{
|
||||
Use: "migrate-zones", Short: "导入旧域名数据到 Zone",
|
||||
PreRun: func(_ *cobra.Command, _ []string) { migrator.Migrate() },
|
||||
RunE: func(_ *cobra.Command, _ []string) error {
|
||||
report, err := zone.ImportLegacy(context.Background())
|
||||
return report.LogAndReturn(err)
|
||||
},
|
||||
}
|
||||
@@ -72,7 +72,7 @@ func init() {
|
||||
schedulerCmd.PreRun = migratePreRun
|
||||
|
||||
// 2. 集中将这些命令注册为真正的子命令,以解决 Cobra 的 unknown command 校验限制
|
||||
rootCmd.AddCommand(allCmd, apiCmd, workerCmd, schedulerCmd, migrateZonesCmd)
|
||||
rootCmd.AddCommand(allCmd, apiCmd, workerCmd, schedulerCmd)
|
||||
}
|
||||
|
||||
// Execute 执行根命令
|
||||
|
||||
@@ -1,204 +0,0 @@
|
||||
// Copyright 2026 Arctel.net
|
||||
// SPDX-License-Identifier: Apache-2.0
|
||||
|
||||
package migrator
|
||||
|
||||
import (
|
||||
"context"
|
||||
"database/sql"
|
||||
"fmt"
|
||||
"strings"
|
||||
|
||||
"github.com/pressly/goose/v3"
|
||||
)
|
||||
|
||||
func init() {
|
||||
goose.AddMigrationContext(up202606050001, down202606050001)
|
||||
}
|
||||
|
||||
func up202606050001(ctx context.Context, tx *sql.Tx) error {
|
||||
dialect := gooseDialect()
|
||||
if dialect == dialectSqlite {
|
||||
if err := fixSqliteGooseDbVersion(ctx, tx); err != nil {
|
||||
return err
|
||||
}
|
||||
}
|
||||
tableExistsQuery := tableExistsSQL(dialect)
|
||||
|
||||
checkTable := func(name string) (bool, error) {
|
||||
var count int
|
||||
err := tx.QueryRowContext(ctx, tableExistsQuery, name).Scan(&count)
|
||||
return count > 0, err
|
||||
}
|
||||
|
||||
// Check if this is a legacy database (by checking if the "options" table exists).
|
||||
// On a clean install, "options" won't exist, so this will be a quick no-op.
|
||||
hasOptions, err := checkTable("options")
|
||||
if err != nil {
|
||||
return fmt.Errorf("check table options failed: %w", err)
|
||||
}
|
||||
|
||||
if !hasOptions {
|
||||
// Clean install or already migrated, no action needed.
|
||||
return nil
|
||||
}
|
||||
|
||||
// Helper to drop table if it exists
|
||||
dropTable := func(tableName string) error {
|
||||
dropSQL := fmt.Sprintf("DROP TABLE IF EXISTS %s", tableName)
|
||||
if dialect == dialectPostgres {
|
||||
dropSQL += cascadeSuffix
|
||||
}
|
||||
if _, err := tx.ExecContext(ctx, dropSQL); err != nil {
|
||||
return fmt.Errorf("drop table %s failed: %w", tableName, err)
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
// 1. Drop legacy observability and log tables to free space and avoid conflict
|
||||
obsPrefixes := []string{
|
||||
"node_system_profiles",
|
||||
"node_health_events",
|
||||
"node_access_logs_",
|
||||
"node_observation_openresties_",
|
||||
"node_observation_frpcs_",
|
||||
"node_observation_frps_",
|
||||
"node_metric_snapshots_",
|
||||
"node_request_reports_",
|
||||
}
|
||||
|
||||
for _, prefix := range obsPrefixes {
|
||||
if err := dropTablesWithPrefix(ctx, tx, tablesWithPrefixSQL(dialect), prefix, dropTable); err != nil {
|
||||
return err
|
||||
}
|
||||
}
|
||||
|
||||
// 2. Rename custom and platform legacy tables to legacy_ prefix
|
||||
tablesToRename := []string{
|
||||
"users",
|
||||
"auth_sources",
|
||||
"external_accounts",
|
||||
"options",
|
||||
"origins",
|
||||
"apply_logs",
|
||||
"proxy_routes",
|
||||
"nodes",
|
||||
"waf_rule_groups",
|
||||
"waf_rule_group_bindings",
|
||||
"waf_ip_groups",
|
||||
"tls_certificates",
|
||||
"managed_domains",
|
||||
"dns_accounts",
|
||||
"acme_accounts",
|
||||
"config_versions",
|
||||
"pages_projects",
|
||||
"pages_deployments",
|
||||
"pages_deployment_files",
|
||||
}
|
||||
|
||||
for _, oldName := range tablesToRename {
|
||||
if err := renameLegacyTable(ctx, tx, oldName, checkTable, dropTable); err != nil {
|
||||
return err
|
||||
}
|
||||
}
|
||||
|
||||
return nil
|
||||
}
|
||||
|
||||
func dropTablesWithPrefix(ctx context.Context, tx *sql.Tx, query, prefix string, dropTable func(string) error) error {
|
||||
rows, err := tx.QueryContext(ctx, query, prefix+"%")
|
||||
if err != nil {
|
||||
return fmt.Errorf("query legacy tables for prefix %s failed: %w", prefix, err)
|
||||
}
|
||||
defer func() {
|
||||
_ = rows.Close()
|
||||
}()
|
||||
|
||||
var tables []string
|
||||
for rows.Next() {
|
||||
var name string
|
||||
if err := rows.Scan(&name); err != nil {
|
||||
return err
|
||||
}
|
||||
tables = append(tables, name)
|
||||
}
|
||||
|
||||
for _, table := range tables {
|
||||
if err := dropTable(table); err != nil {
|
||||
return err
|
||||
}
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
func renameLegacyTable(ctx context.Context, tx *sql.Tx, oldName string, checkTable func(string) (bool, error), dropTable func(string) error) error {
|
||||
exists, err := checkTable(oldName)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if !exists {
|
||||
return nil
|
||||
}
|
||||
|
||||
newName := "legacy_" + oldName
|
||||
newExists, err := checkTable(newName)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if newExists {
|
||||
if err := dropTable(newName); err != nil {
|
||||
return err
|
||||
}
|
||||
}
|
||||
|
||||
renameSQL := fmt.Sprintf("ALTER TABLE %s RENAME TO %s", oldName, newName)
|
||||
if _, err := tx.ExecContext(ctx, renameSQL); err != nil {
|
||||
return fmt.Errorf("rename table %s to %s failed: %w", oldName, newName, err)
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
func down202606050001(_ context.Context, _ *sql.Tx) error {
|
||||
// Down migration is a no-op as the legacy DB is rolled forward during migration.
|
||||
return nil
|
||||
}
|
||||
|
||||
func fixSqliteGooseDbVersion(ctx context.Context, tx *sql.Tx) error {
|
||||
var createSQL string
|
||||
err := tx.QueryRowContext(ctx, "SELECT sql FROM sqlite_master WHERE type='table' AND name='goose_db_version'").Scan(&createSQL)
|
||||
if err != nil {
|
||||
if err == sql.ErrNoRows {
|
||||
return nil
|
||||
}
|
||||
return err
|
||||
}
|
||||
if strings.Contains(strings.ToUpper(createSQL), "AUTOINCREMENT") {
|
||||
return nil // Already upgraded
|
||||
}
|
||||
|
||||
if _, err := tx.ExecContext(ctx, "ALTER TABLE goose_db_version RENAME TO old_goose_db_version"); err != nil {
|
||||
return fmt.Errorf("rename goose_db_version failed: %w", err)
|
||||
}
|
||||
|
||||
newTableSQL := `CREATE TABLE goose_db_version (
|
||||
id INTEGER PRIMARY KEY AUTOINCREMENT,
|
||||
version_id INTEGER NOT NULL,
|
||||
is_applied INTEGER NOT NULL,
|
||||
tstamp TIMESTAMP DEFAULT CURRENT_TIMESTAMP
|
||||
);`
|
||||
if _, err := tx.ExecContext(ctx, newTableSQL); err != nil {
|
||||
return fmt.Errorf("create new goose_db_version failed: %w", err)
|
||||
}
|
||||
|
||||
copySQL := `INSERT INTO goose_db_version (id, version_id, is_applied, tstamp)
|
||||
SELECT id, version_id, is_applied, tstamp FROM old_goose_db_version;`
|
||||
if _, err := tx.ExecContext(ctx, copySQL); err != nil {
|
||||
return fmt.Errorf("copy goose_db_version data failed: %w", err)
|
||||
}
|
||||
|
||||
if _, err := tx.ExecContext(ctx, "DROP TABLE old_goose_db_version"); err != nil {
|
||||
return fmt.Errorf("drop old_goose_db_version failed: %w", err)
|
||||
}
|
||||
|
||||
return nil
|
||||
}
|
||||
@@ -1,550 +0,0 @@
|
||||
// Copyright 2026 Arctel.net
|
||||
// SPDX-License-Identifier: Apache-2.0
|
||||
|
||||
package migrator
|
||||
|
||||
import (
|
||||
"context"
|
||||
"database/sql"
|
||||
"fmt"
|
||||
|
||||
"github.com/pressly/goose/v3"
|
||||
)
|
||||
|
||||
func init() {
|
||||
goose.AddMigrationContext(up202606200006, down202606200006)
|
||||
}
|
||||
|
||||
type migrationTask struct {
|
||||
legacyName string
|
||||
sqliteSQL string
|
||||
pgSQL string
|
||||
}
|
||||
|
||||
func up202606200006(ctx context.Context, tx *sql.Tx) error {
|
||||
dialect := gooseDialect()
|
||||
tableExistsQuery := tableExistsSQL(dialect)
|
||||
|
||||
checkTable := func(name string) (bool, error) {
|
||||
var count int
|
||||
err := tx.QueryRowContext(ctx, tableExistsQuery, name).Scan(&count)
|
||||
return count > 0, err
|
||||
}
|
||||
|
||||
// Helper to migrate table if it exists
|
||||
migrateTable := func(task migrationTask) error {
|
||||
exists, err := checkTable(task.legacyName)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if !exists {
|
||||
return nil
|
||||
}
|
||||
var sqlToRun string
|
||||
if dialect == dialectSqlite {
|
||||
sqlToRun = task.sqliteSQL
|
||||
} else {
|
||||
sqlToRun = task.pgSQL
|
||||
}
|
||||
if _, err := tx.ExecContext(ctx, sqlToRun); err != nil {
|
||||
return fmt.Errorf("execute migration query for %s failed: %w", task.legacyName, err)
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
tasks := getMigrationTasks()
|
||||
for _, task := range tasks {
|
||||
if err := migrateTable(task); err != nil {
|
||||
return err
|
||||
}
|
||||
}
|
||||
|
||||
// Drop legacy tables that exist
|
||||
legacyTables := []string{
|
||||
"legacy_users",
|
||||
"legacy_auth_sources",
|
||||
"legacy_external_accounts",
|
||||
"legacy_options",
|
||||
"legacy_origins",
|
||||
"legacy_apply_logs",
|
||||
"legacy_proxy_routes",
|
||||
"legacy_nodes",
|
||||
"legacy_waf_rule_groups",
|
||||
"legacy_waf_rule_group_bindings",
|
||||
"legacy_waf_ip_groups",
|
||||
"legacy_tls_certificates",
|
||||
"legacy_managed_domains",
|
||||
"legacy_dns_accounts",
|
||||
"legacy_acme_accounts",
|
||||
"legacy_config_versions",
|
||||
"legacy_pages_projects",
|
||||
"legacy_pages_deployments",
|
||||
"legacy_pages_deployment_files",
|
||||
}
|
||||
|
||||
for _, table := range legacyTables {
|
||||
exists, err := checkTable(table)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if exists {
|
||||
dropSQL := fmt.Sprintf("DROP TABLE IF EXISTS %s", table)
|
||||
if dialect == dialectPostgres {
|
||||
dropSQL += cascadeSuffix
|
||||
}
|
||||
if _, err := tx.ExecContext(ctx, dropSQL); err != nil {
|
||||
return fmt.Errorf("drop legacy table %s failed: %w", table, err)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
if dialect == dialectPostgres {
|
||||
if _, err := tx.ExecContext(ctx, `
|
||||
SELECT setval(
|
||||
pg_get_serial_sequence('of_waf_rule_group_bindings', 'id'),
|
||||
GREATEST(COALESCE((SELECT MAX(id) FROM of_waf_rule_group_bindings), 0), 1),
|
||||
COALESCE((SELECT MAX(id) FROM of_waf_rule_group_bindings), 0) > 0
|
||||
)
|
||||
`); err != nil {
|
||||
return fmt.Errorf("sync of_waf_rule_group_bindings sequence failed: %w", err)
|
||||
}
|
||||
}
|
||||
|
||||
return nil
|
||||
}
|
||||
|
||||
func getMigrationTasks() []migrationTask {
|
||||
var tasks []migrationTask
|
||||
tasks = append(tasks, getUserAndOptionTasks()...)
|
||||
tasks = append(tasks, getOriginAndRouteTasks()...)
|
||||
tasks = append(tasks, getWafTasks()...)
|
||||
tasks = append(tasks, getTLSTasks()...)
|
||||
tasks = append(tasks, getPagesAndConfigTasks()...)
|
||||
return tasks
|
||||
}
|
||||
|
||||
func getUserAndOptionTasks() []migrationTask {
|
||||
return []migrationTask{
|
||||
{
|
||||
legacyName: "legacy_users",
|
||||
sqliteSQL: `INSERT OR REPLACE INTO w_users (id, username, password, nickname, email, is_active, is_admin, created_at, updated_at)
|
||||
SELECT id, username, password, display_name, email,
|
||||
CASE WHEN status = 1 THEN 1 ELSE 0 END,
|
||||
CASE WHEN role = 100 THEN 1 ELSE 0 END,
|
||||
CURRENT_TIMESTAMP, CURRENT_TIMESTAMP
|
||||
FROM legacy_users;`,
|
||||
pgSQL: `INSERT INTO w_users (id, username, password, nickname, email, is_active, is_admin, created_at, updated_at)
|
||||
SELECT id, username, password, display_name, email,
|
||||
CASE WHEN status = 1 THEN TRUE ELSE FALSE END,
|
||||
CASE WHEN role = 100 THEN TRUE ELSE FALSE END,
|
||||
CURRENT_TIMESTAMP, CURRENT_TIMESTAMP
|
||||
FROM legacy_users
|
||||
ON CONFLICT (id) DO UPDATE SET
|
||||
username = EXCLUDED.username,
|
||||
password = EXCLUDED.password,
|
||||
nickname = EXCLUDED.nickname,
|
||||
email = EXCLUDED.email,
|
||||
is_active = EXCLUDED.is_active,
|
||||
is_admin = EXCLUDED.is_admin,
|
||||
updated_at = EXCLUDED.updated_at;`,
|
||||
},
|
||||
{
|
||||
legacyName: "legacy_auth_sources",
|
||||
sqliteSQL: `INSERT OR REPLACE INTO w_auth_sources (id, name, type, display_name, is_active, client_id, client_secret, openid_discovery_url, scopes, icon_url, created_at, updated_at)
|
||||
SELECT id, name, type, display_name, is_active, client_id, client_secret, openid_discovery_url, scopes, icon_url, created_at, updated_at
|
||||
FROM legacy_auth_sources;`,
|
||||
pgSQL: `INSERT INTO w_auth_sources (id, name, type, display_name, is_active, client_id, client_secret, openid_discovery_url, scopes, icon_url, created_at, updated_at)
|
||||
SELECT id, name, type, display_name, is_active::boolean, client_id, client_secret, openid_discovery_url, scopes, icon_url, created_at, updated_at
|
||||
FROM legacy_auth_sources
|
||||
ON CONFLICT (id) DO UPDATE SET
|
||||
name = EXCLUDED.name,
|
||||
type = EXCLUDED.type,
|
||||
display_name = EXCLUDED.display_name,
|
||||
is_active = EXCLUDED.is_active,
|
||||
client_id = EXCLUDED.client_id,
|
||||
client_secret = EXCLUDED.client_secret,
|
||||
openid_discovery_url = EXCLUDED.openid_discovery_url,
|
||||
scopes = EXCLUDED.scopes,
|
||||
icon_url = EXCLUDED.icon_url,
|
||||
updated_at = EXCLUDED.updated_at;`,
|
||||
},
|
||||
{
|
||||
legacyName: "legacy_external_accounts",
|
||||
sqliteSQL: `INSERT OR REPLACE INTO w_external_accounts (id, auth_source_id, user_id, external_id, external_username, email, created_at, updated_at)
|
||||
SELECT id, auth_source_id, user_id, external_id, external_username, email, created_at, updated_at
|
||||
FROM legacy_external_accounts;`,
|
||||
pgSQL: `INSERT INTO w_external_accounts (id, auth_source_id, user_id, external_id, external_username, email, created_at, updated_at)
|
||||
SELECT id, auth_source_id, user_id, external_id, external_username, email, created_at, updated_at
|
||||
FROM legacy_external_accounts
|
||||
ON CONFLICT (id) DO UPDATE SET
|
||||
auth_source_id = EXCLUDED.auth_source_id,
|
||||
user_id = EXCLUDED.user_id,
|
||||
external_id = EXCLUDED.external_id,
|
||||
external_username = EXCLUDED.external_username,
|
||||
email = EXCLUDED.email,
|
||||
updated_at = EXCLUDED.updated_at;`,
|
||||
},
|
||||
{
|
||||
legacyName: "legacy_options",
|
||||
sqliteSQL: `INSERT OR REPLACE INTO of_options (key, value)
|
||||
SELECT key, COALESCE(value, '') FROM legacy_options;`,
|
||||
pgSQL: `INSERT INTO of_options (key, value)
|
||||
SELECT key, COALESCE(value, '') FROM legacy_options
|
||||
ON CONFLICT (key) DO UPDATE SET
|
||||
value = EXCLUDED.value;`,
|
||||
},
|
||||
}
|
||||
}
|
||||
|
||||
func getOriginAndRouteTasks() []migrationTask {
|
||||
return []migrationTask{
|
||||
{
|
||||
legacyName: "legacy_origins",
|
||||
sqliteSQL: `INSERT OR REPLACE INTO of_origins (id, name, address, remark, created_at, updated_at)
|
||||
SELECT id, name, address, COALESCE(remark, ''), created_at, updated_at
|
||||
FROM legacy_origins;`,
|
||||
pgSQL: `INSERT INTO of_origins (id, name, address, remark, created_at, updated_at)
|
||||
SELECT id, name, address, COALESCE(remark, ''), created_at, updated_at
|
||||
FROM legacy_origins
|
||||
ON CONFLICT (id) DO UPDATE SET
|
||||
name = EXCLUDED.name,
|
||||
address = EXCLUDED.address,
|
||||
remark = EXCLUDED.remark,
|
||||
updated_at = EXCLUDED.updated_at;`,
|
||||
},
|
||||
{
|
||||
legacyName: "legacy_apply_logs",
|
||||
sqliteSQL: `INSERT OR REPLACE INTO of_apply_logs (id, node_id, version, result, message, checksum, main_config_checksum, route_config_checksum, support_file_count, created_at)
|
||||
SELECT id, node_id, version, result, message, COALESCE(checksum, ''), COALESCE(main_config_checksum, ''), COALESCE(route_config_checksum, ''), COALESCE(support_file_count, 0), created_at
|
||||
FROM legacy_apply_logs;`,
|
||||
pgSQL: `INSERT INTO of_apply_logs (id, node_id, version, result, message, checksum, main_config_checksum, route_config_checksum, support_file_count, created_at)
|
||||
SELECT id, node_id, version, result, message, COALESCE(checksum, ''), COALESCE(main_config_checksum, ''), COALESCE(route_config_checksum, ''), COALESCE(support_file_count, 0), created_at
|
||||
FROM legacy_apply_logs
|
||||
ON CONFLICT (id) DO UPDATE SET
|
||||
node_id = EXCLUDED.node_id,
|
||||
version = EXCLUDED.version,
|
||||
result = EXCLUDED.result,
|
||||
message = EXCLUDED.message,
|
||||
checksum = EXCLUDED.checksum,
|
||||
main_config_checksum = EXCLUDED.main_config_checksum,
|
||||
route_config_checksum = EXCLUDED.route_config_checksum,
|
||||
support_file_count = EXCLUDED.support_file_count;`,
|
||||
},
|
||||
{
|
||||
legacyName: "legacy_proxy_routes",
|
||||
sqliteSQL: `INSERT OR REPLACE INTO of_proxy_routes (id, site_name, domain, domains, origin_id, origin_url, origin_host, upstreams, enabled, enable_https, cert_id, cert_ids, domain_cert_ids, redirect_http, limit_conn_per_server, limit_conn_per_ip, limit_rate, cache_enabled, cache_policy, cache_rules, custom_headers, basic_auth_enabled, basic_auth_username, basic_auth_password, remark, upstream_type, tunnel_node_id, tunnel_target_addr, tunnel_target_protocol, pages_project_id, created_at, updated_at)
|
||||
SELECT id, COALESCE(site_name, ''), domain, COALESCE(domains, '[]'), origin_id, origin_url, COALESCE(origin_host, ''), COALESCE(upstreams, '[]'), enabled, enable_https, cert_id, COALESCE(cert_ids, '[]'), COALESCE(domain_cert_ids, '[]'), redirect_http, limit_conn_per_server, limit_conn_per_ip, COALESCE(limit_rate, ''), cache_enabled, COALESCE(cache_policy, ''), COALESCE(cache_rules, '[]'), COALESCE(custom_headers, '[]'), basic_auth_enabled, COALESCE(basic_auth_username, ''), COALESCE(basic_auth_password, ''), COALESCE(remark, ''), COALESCE(upstream_type, 'direct'), tunnel_node_id, COALESCE(tunnel_target_addr, ''), COALESCE(tunnel_target_protocol, ''), pages_project_id, created_at, updated_at
|
||||
FROM legacy_proxy_routes;`,
|
||||
pgSQL: `INSERT INTO of_proxy_routes (id, site_name, domain, domains, origin_id, origin_url, origin_host, upstreams, enabled, enable_https, cert_id, cert_ids, domain_cert_ids, redirect_http, limit_conn_per_server, limit_conn_per_ip, limit_rate, cache_enabled, cache_policy, cache_rules, custom_headers, basic_auth_enabled, basic_auth_username, basic_auth_password, remark, upstream_type, tunnel_node_id, tunnel_target_addr, tunnel_target_protocol, pages_project_id, created_at, updated_at)
|
||||
SELECT id, COALESCE(site_name, ''), domain, COALESCE(domains, '[]'), origin_id, origin_url, COALESCE(origin_host, ''), COALESCE(upstreams, '[]'), enabled, enable_https, cert_id, COALESCE(cert_ids, '[]'), COALESCE(domain_cert_ids, '[]'), redirect_http, limit_conn_per_server, limit_conn_per_ip, COALESCE(limit_rate, ''), cache_enabled, COALESCE(cache_policy, ''), COALESCE(cache_rules, '[]'), COALESCE(custom_headers, '[]'), basic_auth_enabled, COALESCE(basic_auth_username, ''), COALESCE(basic_auth_password, ''), COALESCE(remark, ''), COALESCE(upstream_type, 'direct'), tunnel_node_id, COALESCE(tunnel_target_addr, ''), COALESCE(tunnel_target_protocol, ''), pages_project_id, created_at, updated_at
|
||||
FROM legacy_proxy_routes
|
||||
ON CONFLICT (id) DO UPDATE SET
|
||||
site_name = EXCLUDED.site_name,
|
||||
domain = EXCLUDED.domain,
|
||||
domains = EXCLUDED.domains,
|
||||
origin_id = EXCLUDED.origin_id,
|
||||
origin_url = EXCLUDED.origin_url,
|
||||
origin_host = EXCLUDED.origin_host,
|
||||
upstreams = EXCLUDED.upstreams,
|
||||
enabled = EXCLUDED.enabled,
|
||||
enable_https = EXCLUDED.enable_https,
|
||||
cert_id = EXCLUDED.cert_id,
|
||||
cert_ids = EXCLUDED.cert_ids,
|
||||
domain_cert_ids = EXCLUDED.domain_cert_ids,
|
||||
redirect_http = EXCLUDED.redirect_http,
|
||||
limit_conn_per_server = EXCLUDED.limit_conn_per_server,
|
||||
limit_conn_per_ip = EXCLUDED.limit_conn_per_ip,
|
||||
limit_rate = EXCLUDED.limit_rate,
|
||||
cache_enabled = EXCLUDED.cache_enabled,
|
||||
cache_policy = EXCLUDED.cache_policy,
|
||||
cache_rules = EXCLUDED.cache_rules,
|
||||
custom_headers = EXCLUDED.custom_headers,
|
||||
basic_auth_enabled = EXCLUDED.basic_auth_enabled,
|
||||
basic_auth_username = EXCLUDED.basic_auth_username,
|
||||
basic_auth_password = EXCLUDED.basic_auth_password,
|
||||
remark = EXCLUDED.remark,
|
||||
upstream_type = EXCLUDED.upstream_type,
|
||||
tunnel_node_id = EXCLUDED.tunnel_node_id,
|
||||
tunnel_target_addr = EXCLUDED.tunnel_target_addr,
|
||||
tunnel_target_protocol = EXCLUDED.tunnel_target_protocol,
|
||||
pages_project_id = EXCLUDED.pages_project_id,
|
||||
updated_at = EXCLUDED.updated_at;`,
|
||||
},
|
||||
{
|
||||
legacyName: "legacy_nodes",
|
||||
sqliteSQL: `INSERT OR REPLACE INTO of_nodes (id, node_id, name, ip, ip_manual_override, geo_name, geo_latitude, geo_longitude, geo_manual_override, access_token, auto_update_enabled, update_requested, update_channel, update_tag, restart_openresty_requested, version, ext_version, openresty_status, openresty_message, status, current_version, last_seen_at, last_error, created_at, updated_at, node_type, relay_bind_port, relay_vhost_http_port, relay_auth_token, relay_agent_access_addr, relay_client_access_addr, relay_client_proxy_url, capabilities_json, relay_status, relay_web_server_enabled)
|
||||
SELECT id, node_id, name, COALESCE(ip, ''), COALESCE(ip_manual_override, 0), COALESCE(geo_name, ''), geo_latitude, geo_longitude, COALESCE(geo_manual_override, 0), COALESCE(access_token, ''), COALESCE(auto_update_enabled, 0), COALESCE(update_requested, 0), COALESCE(update_channel, 'stable'), COALESCE(update_tag, ''), COALESCE(restart_openresty_requested, 0), COALESCE(version, ''), COALESCE(ext_version, ''), COALESCE(openresty_status, 'unknown'), openresty_message, COALESCE(status, 'offline'), COALESCE(current_version, ''), last_seen_at, last_error, created_at, updated_at, COALESCE(node_type, 'edge_node'), COALESCE(relay_bind_port, 0), COALESCE(relay_vhost_http_port, 0), COALESCE(relay_auth_token, ''), COALESCE(relay_agent_access_addr, ''), COALESCE(relay_client_access_addr, ''), COALESCE(relay_client_proxy_url, ''), COALESCE(capabilities_json, '[]'), COALESCE(relay_status, 'unknown'), COALESCE(relay_web_server_enabled, 0)
|
||||
FROM legacy_nodes;`,
|
||||
pgSQL: `INSERT INTO of_nodes (id, node_id, name, ip, ip_manual_override, geo_name, geo_latitude, geo_longitude, geo_manual_override, access_token, auto_update_enabled, update_requested, update_channel, update_tag, restart_openresty_requested, version, ext_version, openresty_status, openresty_message, status, current_version, last_seen_at, last_error, created_at, updated_at, node_type, relay_bind_port, relay_vhost_http_port, relay_auth_token, relay_agent_access_addr, relay_client_access_addr, relay_client_proxy_url, capabilities_json, relay_status, relay_web_server_enabled)
|
||||
SELECT id, node_id, name, COALESCE(ip, ''), COALESCE(ip_manual_override, FALSE), COALESCE(geo_name, ''), geo_latitude, geo_longitude, COALESCE(geo_manual_override, FALSE), COALESCE(access_token, ''), COALESCE(auto_update_enabled, FALSE), COALESCE(update_requested, FALSE), COALESCE(update_channel, 'stable'), COALESCE(update_tag, ''), COALESCE(restart_openresty_requested, FALSE), COALESCE(version, ''), COALESCE(ext_version, ''), COALESCE(openresty_status, 'unknown'), openresty_message, COALESCE(status, 'offline'), COALESCE(current_version, ''), last_seen_at, last_error, created_at, updated_at, COALESCE(node_type, 'edge_node'), COALESCE(relay_bind_port, 0), COALESCE(relay_vhost_http_port, 0), COALESCE(relay_auth_token, ''), COALESCE(relay_agent_access_addr, ''), COALESCE(relay_client_access_addr, ''), COALESCE(relay_client_proxy_url, ''), COALESCE(capabilities_json, '[]'), COALESCE(relay_status, 'unknown'), COALESCE(relay_web_server_enabled, FALSE)
|
||||
FROM legacy_nodes
|
||||
ON CONFLICT (id) DO UPDATE SET
|
||||
node_id = EXCLUDED.node_id,
|
||||
name = EXCLUDED.name,
|
||||
ip = EXCLUDED.ip,
|
||||
ip_manual_override = EXCLUDED.ip_manual_override,
|
||||
geo_name = EXCLUDED.geo_name,
|
||||
geo_latitude = EXCLUDED.geo_latitude,
|
||||
geo_longitude = EXCLUDED.geo_longitude,
|
||||
geo_manual_override = EXCLUDED.geo_manual_override,
|
||||
access_token = EXCLUDED.access_token,
|
||||
auto_update_enabled = EXCLUDED.auto_update_enabled,
|
||||
update_requested = EXCLUDED.update_requested,
|
||||
update_channel = EXCLUDED.update_channel,
|
||||
update_tag = EXCLUDED.update_tag,
|
||||
restart_openresty_requested = EXCLUDED.restart_openresty_requested,
|
||||
version = EXCLUDED.version,
|
||||
ext_version = EXCLUDED.ext_version,
|
||||
openresty_status = EXCLUDED.openresty_status,
|
||||
openresty_message = EXCLUDED.openresty_message,
|
||||
status = EXCLUDED.status,
|
||||
current_version = EXCLUDED.current_version,
|
||||
last_seen_at = EXCLUDED.last_seen_at,
|
||||
last_error = EXCLUDED.last_error,
|
||||
updated_at = EXCLUDED.updated_at,
|
||||
node_type = EXCLUDED.node_type,
|
||||
relay_bind_port = EXCLUDED.relay_bind_port,
|
||||
relay_vhost_http_port = EXCLUDED.relay_vhost_http_port,
|
||||
relay_auth_token = EXCLUDED.relay_auth_token,
|
||||
relay_agent_access_addr = EXCLUDED.relay_agent_access_addr,
|
||||
relay_client_access_addr = EXCLUDED.relay_client_access_addr,
|
||||
relay_client_proxy_url = EXCLUDED.relay_client_proxy_url,
|
||||
capabilities_json = EXCLUDED.capabilities_json,
|
||||
relay_status = EXCLUDED.relay_status,
|
||||
relay_web_server_enabled = EXCLUDED.relay_web_server_enabled;`,
|
||||
},
|
||||
}
|
||||
}
|
||||
|
||||
func getWafTasks() []migrationTask {
|
||||
return []migrationTask{
|
||||
{
|
||||
legacyName: "legacy_waf_rule_groups",
|
||||
sqliteSQL: `INSERT OR REPLACE INTO of_waf_rule_groups (id, name, enabled, is_global, block_status_code, block_response_body, ip_whitelist, ip_blacklist, ip_whitelist_groups, ip_blacklist_groups, country_whitelist, country_blacklist, region_whitelist, region_blacklist, pow_enabled, pow_config, remark, created_at, updated_at)
|
||||
SELECT id, name, COALESCE(enabled, 1), COALESCE(is_global, 0), COALESCE(block_status_code, 418), COALESCE(block_response_body, ''), COALESCE(ip_whitelist, '[]'), COALESCE(ip_blacklist, '[]'), COALESCE(ip_whitelist_groups, '[]'), COALESCE(ip_blacklist_groups, '[]'), COALESCE(country_whitelist, '[]'), COALESCE(country_blacklist, '[]'), COALESCE(region_whitelist, '[]'), COALESCE(region_blacklist, '[]'), COALESCE(pow_enabled, 0), COALESCE(pow_config, '{}'), COALESCE(remark, ''), created_at, updated_at
|
||||
FROM legacy_waf_rule_groups;`,
|
||||
pgSQL: `INSERT INTO of_waf_rule_groups (id, name, enabled, is_global, block_status_code, block_response_body, ip_whitelist, ip_blacklist, ip_whitelist_groups, ip_blacklist_groups, country_whitelist, country_blacklist, region_whitelist, region_blacklist, pow_enabled, pow_config, remark, created_at, updated_at)
|
||||
SELECT id, name, COALESCE(enabled::boolean, TRUE), COALESCE(is_global::boolean, FALSE), COALESCE(block_status_code, 418), COALESCE(block_response_body, ''), COALESCE(ip_whitelist, '[]'), COALESCE(ip_blacklist, '[]'), COALESCE(ip_whitelist_groups, '[]'), COALESCE(ip_blacklist_groups, '[]'), COALESCE(country_whitelist, '[]'), COALESCE(country_blacklist, '[]'), COALESCE(region_whitelist, '[]'), COALESCE(region_blacklist, '[]'), COALESCE(pow_enabled::boolean, FALSE), COALESCE(pow_config, '{}'), COALESCE(remark, ''), created_at, updated_at
|
||||
FROM legacy_waf_rule_groups
|
||||
ON CONFLICT (id) DO UPDATE SET
|
||||
name = EXCLUDED.name,
|
||||
enabled = EXCLUDED.enabled,
|
||||
is_global = EXCLUDED.is_global,
|
||||
block_status_code = EXCLUDED.block_status_code,
|
||||
block_response_body = EXCLUDED.block_response_body,
|
||||
ip_whitelist = EXCLUDED.ip_whitelist,
|
||||
ip_blacklist = EXCLUDED.ip_blacklist,
|
||||
ip_whitelist_groups = EXCLUDED.ip_whitelist_groups,
|
||||
ip_blacklist_groups = EXCLUDED.ip_blacklist_groups,
|
||||
country_whitelist = EXCLUDED.country_whitelist,
|
||||
country_blacklist = EXCLUDED.country_blacklist,
|
||||
region_whitelist = EXCLUDED.region_whitelist,
|
||||
region_blacklist = EXCLUDED.region_blacklist,
|
||||
pow_enabled = EXCLUDED.pow_enabled,
|
||||
pow_config = EXCLUDED.pow_config,
|
||||
remark = EXCLUDED.remark,
|
||||
updated_at = EXCLUDED.updated_at;`,
|
||||
},
|
||||
{
|
||||
legacyName: "legacy_waf_rule_group_bindings",
|
||||
sqliteSQL: `INSERT OR REPLACE INTO of_waf_rule_group_bindings (id, rule_group_id, proxy_route_id, created_at)
|
||||
SELECT id, rule_group_id, proxy_route_id, created_at
|
||||
FROM legacy_waf_rule_group_bindings;`,
|
||||
pgSQL: `INSERT INTO of_waf_rule_group_bindings (id, rule_group_id, proxy_route_id, created_at)
|
||||
SELECT id, rule_group_id, proxy_route_id, created_at
|
||||
FROM legacy_waf_rule_group_bindings
|
||||
ON CONFLICT (id) DO UPDATE SET
|
||||
rule_group_id = EXCLUDED.rule_group_id,
|
||||
proxy_route_id = EXCLUDED.proxy_route_id;`,
|
||||
},
|
||||
{
|
||||
legacyName: "legacy_waf_ip_groups",
|
||||
sqliteSQL: `INSERT OR REPLACE INTO of_waf_ip_groups (id, name, type, enabled, ip_list, auto_config, ext_ips, subscription_url, subscription_format, subscription_mapping_rule, sync_interval_minutes, last_synced_at, next_sync_at, last_sync_status, last_sync_message, remark, created_at, updated_at)
|
||||
SELECT id, name, type, COALESCE(enabled, 1), COALESCE(ip_list, '[]'), COALESCE(auto_config, '{}'), COALESCE(ext_ips, '[]'), COALESCE(subscription_url, ''), COALESCE(subscription_format, 'text'), COALESCE(subscription_mapping_rule, ''), COALESCE(sync_interval_minutes, 1440), last_synced_at, next_sync_at, COALESCE(last_sync_status, ''), COALESCE(last_sync_message, ''), COALESCE(remark, ''), created_at, updated_at
|
||||
FROM legacy_waf_ip_groups;`,
|
||||
pgSQL: `INSERT INTO of_waf_ip_groups (id, name, type, enabled, ip_list, auto_config, ext_ips, subscription_url, subscription_format, subscription_mapping_rule, sync_interval_minutes, last_synced_at, next_sync_at, last_sync_status, last_sync_message, remark, created_at, updated_at)
|
||||
SELECT id, name, type, COALESCE(enabled::boolean, TRUE), COALESCE(ip_list, '[]'), COALESCE(auto_config, '{}'), COALESCE(ext_ips, '[]'), COALESCE(subscription_url, ''), COALESCE(subscription_format, 'text'), COALESCE(subscription_mapping_rule, ''), COALESCE(sync_interval_minutes, 1440), last_synced_at, next_sync_at, COALESCE(last_sync_status, ''), COALESCE(last_sync_message, ''), COALESCE(remark, ''), created_at, updated_at
|
||||
FROM legacy_waf_ip_groups
|
||||
ON CONFLICT (id) DO UPDATE SET
|
||||
name = EXCLUDED.name,
|
||||
type = EXCLUDED.type,
|
||||
enabled = EXCLUDED.enabled,
|
||||
ip_list = EXCLUDED.ip_list,
|
||||
auto_config = EXCLUDED.auto_config,
|
||||
ext_ips = EXCLUDED.ext_ips,
|
||||
subscription_url = EXCLUDED.subscription_url,
|
||||
subscription_format = EXCLUDED.subscription_format,
|
||||
subscription_mapping_rule = EXCLUDED.subscription_mapping_rule,
|
||||
sync_interval_minutes = EXCLUDED.sync_interval_minutes,
|
||||
last_synced_at = EXCLUDED.last_synced_at,
|
||||
next_sync_at = EXCLUDED.next_sync_at,
|
||||
last_sync_status = EXCLUDED.last_sync_status,
|
||||
last_sync_message = EXCLUDED.last_sync_message,
|
||||
remark = EXCLUDED.remark,
|
||||
updated_at = EXCLUDED.updated_at;`,
|
||||
},
|
||||
}
|
||||
}
|
||||
|
||||
func getTLSTasks() []migrationTask {
|
||||
return []migrationTask{
|
||||
{
|
||||
legacyName: "legacy_tls_certificates",
|
||||
sqliteSQL: `INSERT OR REPLACE INTO of_tls_certificates (id, name, cert_pem, key_pem, not_before, not_after, remark, provider, acme_account_id, dns_account_id, key_algorithm, auto_renew, primary_domain, other_domains, disable_cname, skip_dns, dns1, dns2, apply_status, apply_message, created_at, updated_at)
|
||||
SELECT id, name, cert_pem, key_pem, not_before, not_after, COALESCE(remark, ''), COALESCE(provider, 'upload'), COALESCE(acme_account_id, 0), COALESCE(dns_account_id, 0), COALESCE(key_algorithm, ''), COALESCE(auto_renew, 0), COALESCE(primary_domain, ''), COALESCE(other_domains, ''), COALESCE(disable_cname, 0), COALESCE(skip_dns, 0), COALESCE(dns1, ''), COALESCE(dns2, ''), COALESCE(apply_status, 'ready'), COALESCE(apply_message, ''), created_at, updated_at
|
||||
FROM legacy_tls_certificates;`,
|
||||
pgSQL: `INSERT INTO of_tls_certificates (id, name, cert_pem, key_pem, not_before, not_after, remark, provider, acme_account_id, dns_account_id, key_algorithm, auto_renew, primary_domain, other_domains, disable_cname, skip_dns, dns1, dns2, apply_status, apply_message, created_at, updated_at)
|
||||
SELECT id, name, cert_pem, key_pem, not_before, not_after, COALESCE(remark, ''), COALESCE(provider, 'upload'), COALESCE(acme_account_id, 0), COALESCE(dns_account_id, 0), COALESCE(key_algorithm, ''), COALESCE(auto_renew, FALSE), COALESCE(primary_domain, ''), COALESCE(other_domains, ''), COALESCE(disable_cname, FALSE), COALESCE(skip_dns, FALSE), COALESCE(dns1, ''), COALESCE(dns2, ''), COALESCE(apply_status, 'ready'), COALESCE(apply_message, ''), created_at, updated_at
|
||||
FROM legacy_tls_certificates
|
||||
ON CONFLICT (id) DO UPDATE SET
|
||||
name = EXCLUDED.name,
|
||||
cert_pem = EXCLUDED.cert_pem,
|
||||
key_pem = EXCLUDED.key_pem,
|
||||
not_before = EXCLUDED.not_before,
|
||||
not_after = EXCLUDED.not_after,
|
||||
remark = EXCLUDED.remark,
|
||||
provider = EXCLUDED.provider,
|
||||
acme_account_id = EXCLUDED.acme_account_id,
|
||||
dns_account_id = EXCLUDED.dns_account_id,
|
||||
key_algorithm = EXCLUDED.key_algorithm,
|
||||
auto_renew = EXCLUDED.auto_renew,
|
||||
primary_domain = EXCLUDED.primary_domain,
|
||||
other_domains = EXCLUDED.other_domains,
|
||||
disable_cname = EXCLUDED.disable_cname,
|
||||
skip_dns = EXCLUDED.skip_dns,
|
||||
dns1 = EXCLUDED.dns1,
|
||||
dns2 = EXCLUDED.dns2,
|
||||
apply_status = EXCLUDED.apply_status,
|
||||
apply_message = EXCLUDED.apply_message,
|
||||
updated_at = EXCLUDED.updated_at;`,
|
||||
},
|
||||
{
|
||||
legacyName: "legacy_managed_domains",
|
||||
sqliteSQL: `INSERT OR REPLACE INTO of_managed_domains (id, domain, cert_id, enabled, remark, created_at, updated_at)
|
||||
SELECT id, domain, cert_id, COALESCE(enabled, 1), COALESCE(remark, ''), created_at, updated_at
|
||||
FROM legacy_managed_domains;`,
|
||||
pgSQL: `INSERT INTO of_managed_domains (id, domain, cert_id, enabled, remark, created_at, updated_at)
|
||||
SELECT id, domain, cert_id, COALESCE(enabled, TRUE), COALESCE(remark, ''), created_at, updated_at
|
||||
FROM legacy_managed_domains
|
||||
ON CONFLICT (id) DO UPDATE SET
|
||||
domain = EXCLUDED.domain,
|
||||
cert_id = EXCLUDED.cert_id,
|
||||
enabled = EXCLUDED.enabled,
|
||||
remark = EXCLUDED.remark,
|
||||
updated_at = EXCLUDED.updated_at;`,
|
||||
},
|
||||
{
|
||||
legacyName: "legacy_dns_accounts",
|
||||
sqliteSQL: `INSERT OR REPLACE INTO of_dns_accounts (id, name, type, authorization, created_at, updated_at)
|
||||
SELECT id, name, type, authorization, created_at, updated_at
|
||||
FROM legacy_dns_accounts;`,
|
||||
pgSQL: `INSERT INTO of_dns_accounts (id, name, type, "authorization", created_at, updated_at)
|
||||
SELECT id, name, type, "authorization", created_at, updated_at
|
||||
FROM legacy_dns_accounts
|
||||
ON CONFLICT (id) DO UPDATE SET
|
||||
name = EXCLUDED.name,
|
||||
type = EXCLUDED.type,
|
||||
"authorization" = EXCLUDED."authorization",
|
||||
updated_at = EXCLUDED.updated_at;`,
|
||||
},
|
||||
{
|
||||
legacyName: "legacy_acme_accounts",
|
||||
sqliteSQL: `INSERT OR REPLACE INTO of_acme_accounts (id, email, url, private_key, created_at, updated_at)
|
||||
SELECT id, COALESCE(email, ''), COALESCE(url, ''), COALESCE(private_key, ''), created_at, updated_at
|
||||
FROM legacy_acme_accounts;`,
|
||||
pgSQL: `INSERT INTO of_acme_accounts (id, email, url, private_key, created_at, updated_at)
|
||||
SELECT id, COALESCE(email, ''), COALESCE(url, ''), COALESCE(private_key, ''), created_at, updated_at
|
||||
FROM legacy_acme_accounts
|
||||
ON CONFLICT (id) DO UPDATE SET
|
||||
email = EXCLUDED.email,
|
||||
url = EXCLUDED.url,
|
||||
private_key = EXCLUDED.private_key,
|
||||
updated_at = EXCLUDED.updated_at;`,
|
||||
},
|
||||
}
|
||||
}
|
||||
|
||||
func getPagesAndConfigTasks() []migrationTask {
|
||||
return []migrationTask{
|
||||
{
|
||||
legacyName: "legacy_config_versions",
|
||||
sqliteSQL: `INSERT OR REPLACE INTO of_config_versions (id, version, snapshot_json, main_config, rendered_config, support_files_json, checksum, is_active, created_by, created_at)
|
||||
SELECT id, version, snapshot_json, COALESCE(main_config, ''), rendered_config, COALESCE(support_files_json, '[]'), checksum, COALESCE(is_active, 0), created_by, created_at
|
||||
FROM legacy_config_versions;`,
|
||||
pgSQL: `INSERT INTO of_config_versions (id, version, snapshot_json, main_config, rendered_config, support_files_json, checksum, is_active, created_by, created_at)
|
||||
SELECT id, version, snapshot_json, COALESCE(main_config, ''), rendered_config, COALESCE(support_files_json, '[]'), checksum, COALESCE(is_active, FALSE), created_by, created_at
|
||||
FROM legacy_config_versions
|
||||
ON CONFLICT (id) DO UPDATE SET
|
||||
version = EXCLUDED.version,
|
||||
snapshot_json = EXCLUDED.snapshot_json,
|
||||
main_config = EXCLUDED.main_config,
|
||||
rendered_config = EXCLUDED.rendered_config,
|
||||
support_files_json = EXCLUDED.support_files_json,
|
||||
checksum = EXCLUDED.checksum,
|
||||
is_active = EXCLUDED.is_active,
|
||||
created_by = EXCLUDED.created_by;`,
|
||||
},
|
||||
{
|
||||
legacyName: "legacy_pages_projects",
|
||||
sqliteSQL: `INSERT OR REPLACE INTO of_pages_projects (id, name, slug, description, enabled, spa_fallback_enabled, spa_fallback_path, api_proxy_enabled, api_proxy_path, api_proxy_pass, api_proxy_rewrite, active_deployment_id, root_dir, entry_file, created_at, updated_at)
|
||||
SELECT id, name, slug, COALESCE(description, ''), COALESCE(enabled, 1), COALESCE(spa_fallback_enabled, 0), COALESCE(spa_fallback_path, '/index.html'), COALESCE(api_proxy_enabled, 0), COALESCE(api_proxy_path, ''), COALESCE(api_proxy_pass, ''), COALESCE(api_proxy_rewrite, ''), active_deployment_id, COALESCE(root_dir, ''), COALESCE(entry_file, 'index.html'), created_at, updated_at
|
||||
FROM legacy_pages_projects;`,
|
||||
pgSQL: `INSERT INTO of_pages_projects (id, name, slug, description, enabled, spa_fallback_enabled, spa_fallback_path, api_proxy_enabled, api_proxy_path, api_proxy_pass, api_proxy_rewrite, active_deployment_id, root_dir, entry_file, created_at, updated_at)
|
||||
SELECT id, name, slug, COALESCE(description, ''), COALESCE(enabled, TRUE), COALESCE(spa_fallback_enabled, FALSE), COALESCE(spa_fallback_path, '/index.html'), COALESCE(api_proxy_enabled, FALSE), COALESCE(api_proxy_path, ''), COALESCE(api_proxy_pass, ''), COALESCE(api_proxy_rewrite, ''), active_deployment_id, COALESCE(root_dir, ''), COALESCE(entry_file, 'index.html'), created_at, updated_at
|
||||
FROM legacy_pages_projects
|
||||
ON CONFLICT (id) DO UPDATE SET
|
||||
name = EXCLUDED.name,
|
||||
slug = EXCLUDED.slug,
|
||||
description = EXCLUDED.description,
|
||||
enabled = EXCLUDED.enabled,
|
||||
spa_fallback_enabled = EXCLUDED.spa_fallback_enabled,
|
||||
spa_fallback_path = EXCLUDED.spa_fallback_path,
|
||||
api_proxy_enabled = EXCLUDED.api_proxy_enabled,
|
||||
api_proxy_path = EXCLUDED.api_proxy_path,
|
||||
api_proxy_pass = EXCLUDED.api_proxy_pass,
|
||||
api_proxy_rewrite = EXCLUDED.api_proxy_rewrite,
|
||||
active_deployment_id = EXCLUDED.active_deployment_id,
|
||||
root_dir = EXCLUDED.root_dir,
|
||||
entry_file = EXCLUDED.entry_file,
|
||||
updated_at = EXCLUDED.updated_at;`,
|
||||
},
|
||||
{
|
||||
legacyName: "legacy_pages_deployments",
|
||||
sqliteSQL: `INSERT OR REPLACE INTO of_pages_deployments (id, project_id, deployment_number, checksum, status, artifact_path, file_count, total_size, created_by, created_at, activated_at)
|
||||
SELECT id, project_id, deployment_number, checksum, COALESCE(status, 'uploaded'), artifact_path, COALESCE(file_count, 0), COALESCE(total_size, 0), COALESCE(created_by, ''), created_at, activated_at
|
||||
FROM legacy_pages_deployments;`,
|
||||
pgSQL: `INSERT INTO of_pages_deployments (id, project_id, deployment_number, checksum, status, artifact_path, file_count, total_size, created_by, created_at, activated_at)
|
||||
SELECT id, project_id, deployment_number, checksum, COALESCE(status, 'uploaded'), artifact_path, COALESCE(file_count, 0), COALESCE(total_size, 0), COALESCE(created_by, ''), created_at, activated_at
|
||||
FROM legacy_pages_deployments
|
||||
ON CONFLICT (id) DO UPDATE SET
|
||||
project_id = EXCLUDED.project_id,
|
||||
deployment_number = EXCLUDED.deployment_number,
|
||||
checksum = EXCLUDED.checksum,
|
||||
status = EXCLUDED.status,
|
||||
artifact_path = EXCLUDED.artifact_path,
|
||||
file_count = EXCLUDED.file_count,
|
||||
total_size = EXCLUDED.total_size,
|
||||
created_by = EXCLUDED.created_by,
|
||||
activated_at = EXCLUDED.activated_at;`,
|
||||
},
|
||||
{
|
||||
legacyName: "legacy_pages_deployment_files",
|
||||
sqliteSQL: `INSERT OR REPLACE INTO of_pages_deployment_files (id, deployment_id, path, size, checksum, created_at)
|
||||
SELECT id, deployment_id, path, COALESCE(size, 0), checksum, created_at
|
||||
FROM legacy_pages_deployment_files;`,
|
||||
pgSQL: `INSERT INTO of_pages_deployment_files (id, deployment_id, path, size, checksum, created_at)
|
||||
SELECT id, deployment_id, path, COALESCE(size, 0), checksum, created_at
|
||||
FROM legacy_pages_deployment_files
|
||||
ON CONFLICT (id) DO UPDATE SET
|
||||
deployment_id = EXCLUDED.deployment_id,
|
||||
path = EXCLUDED.path,
|
||||
size = EXCLUDED.size,
|
||||
checksum = EXCLUDED.checksum;`,
|
||||
},
|
||||
}
|
||||
}
|
||||
|
||||
func down202606200006(_ context.Context, _ *sql.Tx) error {
|
||||
// Down migration is a no-op
|
||||
return nil
|
||||
}
|
||||
@@ -1,261 +0,0 @@
|
||||
// Copyright 2026 Arctel.net
|
||||
// SPDX-License-Identifier: Apache-2.0
|
||||
|
||||
package migrator
|
||||
|
||||
import (
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
"github.com/Rain-kl/Wavelet/internal/config"
|
||||
"github.com/Rain-kl/Wavelet/internal/db"
|
||||
"github.com/alicebob/miniredis/v2"
|
||||
"github.com/glebarez/sqlite"
|
||||
"github.com/redis/go-redis/v9"
|
||||
"github.com/redis/go-redis/v9/maintnotifications"
|
||||
"gorm.io/gorm"
|
||||
)
|
||||
|
||||
func TestMigrateUpgradesLegacyDatabaseAndPreservesData(t *testing.T) {
|
||||
// 1. Initialize an empty in-memory SQLite DB
|
||||
sqliteDB, err := gorm.Open(sqlite.Open(":memory:"), &gorm.Config{
|
||||
DisableForeignKeyConstraintWhenMigrating: true,
|
||||
})
|
||||
if err != nil {
|
||||
t.Fatalf("gorm.Open(sqlite) error = %v", err)
|
||||
}
|
||||
|
||||
rawDB, err := sqliteDB.DB()
|
||||
if err != nil {
|
||||
t.Fatalf("sqliteDB.DB() error = %v", err)
|
||||
}
|
||||
|
||||
// 2. Set up the legacy schema manually
|
||||
setupLegacySchema := []string{
|
||||
`CREATE TABLE users (
|
||||
id integer(64) NOT NULL,
|
||||
username text,
|
||||
password text NOT NULL,
|
||||
display_name text,
|
||||
role integer(64),
|
||||
status integer(64),
|
||||
token text,
|
||||
email text,
|
||||
github_id text,
|
||||
wechat_id text,
|
||||
CONSTRAINT users_pkey PRIMARY KEY (id)
|
||||
);`,
|
||||
`CREATE TABLE options (
|
||||
key text NOT NULL,
|
||||
value text,
|
||||
CONSTRAINT options_pkey PRIMARY KEY (key)
|
||||
);`,
|
||||
`CREATE TABLE origins (
|
||||
id integer(64) NOT NULL,
|
||||
name text(255) NOT NULL,
|
||||
address text(255) NOT NULL,
|
||||
remark text(255),
|
||||
created_at text(6),
|
||||
updated_at text(6),
|
||||
CONSTRAINT origins_pkey PRIMARY KEY (id)
|
||||
);`,
|
||||
`CREATE TABLE apply_logs (
|
||||
id integer(64) NOT NULL,
|
||||
node_id text(64) NOT NULL,
|
||||
version text(32) NOT NULL,
|
||||
result text(32) NOT NULL,
|
||||
message text,
|
||||
checksum text(64) NOT NULL,
|
||||
main_config_checksum text(64) NOT NULL,
|
||||
route_config_checksum text(64) NOT NULL,
|
||||
support_file_count integer(64) NOT NULL,
|
||||
created_at text(6),
|
||||
CONSTRAINT apply_logs_pkey PRIMARY KEY (id)
|
||||
);`,
|
||||
// Legacy log tables (which should be dropped)
|
||||
`CREATE TABLE node_access_logs_00 (
|
||||
id integer(64) NOT NULL,
|
||||
node_id text(64) NOT NULL,
|
||||
logged_at text(6) NOT NULL
|
||||
);`,
|
||||
`CREATE TABLE node_system_profiles (
|
||||
id integer(64) NOT NULL,
|
||||
node_id text(64) NOT NULL,
|
||||
hostname text(255)
|
||||
);`,
|
||||
}
|
||||
|
||||
for _, query := range setupLegacySchema {
|
||||
if _, err := rawDB.Exec(query); err != nil {
|
||||
t.Fatalf("Exec setup query failed: %v", err)
|
||||
}
|
||||
}
|
||||
|
||||
// 3. Populate with legacy mock data
|
||||
insertMockData := []string{
|
||||
// Legacy admin user
|
||||
`INSERT INTO users (id, username, password, display_name, role, status, email)
|
||||
VALUES (1, 'ryan', 'hashed_pass_123', 'Root User ryan', 100, 1, 'ryan@example.com');`,
|
||||
// Legacy normal user
|
||||
`INSERT INTO users (id, username, password, display_name, role, status, email)
|
||||
VALUES (2, 'jack', 'hashed_pass_456', 'Normal User jack', 10, 1, 'jack@example.com');`,
|
||||
// Legacy config options(GeoIPProvider 会被迁移到 w_system_configs.geoip_provider)
|
||||
`INSERT INTO options (key, value) VALUES ('GeoIPProvider', 'ip-api');`,
|
||||
// Legacy origin
|
||||
`INSERT INTO origins (id, name, address, remark, created_at, updated_at)
|
||||
VALUES (10, 'my_origin', '127.0.0.1:8080', 'original origin', '2026-06-01 12:00:00', '2026-06-01 12:00:00');`,
|
||||
// Legacy apply log
|
||||
`INSERT INTO apply_logs (id, node_id, version, result, message, checksum, main_config_checksum, route_config_checksum, support_file_count, created_at)
|
||||
VALUES (100, 'node_1', 'v1.0.0', 'success', 'applied successfully', 'hash1', 'hash2', 'hash3', 2, '2026-06-01 12:00:00');`,
|
||||
}
|
||||
|
||||
for _, query := range insertMockData {
|
||||
if _, err := rawDB.Exec(query); err != nil {
|
||||
t.Fatalf("Exec mock data query failed: %v", err)
|
||||
}
|
||||
}
|
||||
|
||||
// 4. Set up mock services (Redis & Config)
|
||||
mr, err := miniredis.Run()
|
||||
if err != nil {
|
||||
t.Fatalf("miniredis.Run() error = %v", err)
|
||||
}
|
||||
redisClient := redis.NewClient(&redis.Options{
|
||||
Addr: mr.Addr(),
|
||||
MaintNotificationsConfig: &maintnotifications.Config{
|
||||
Mode: maintnotifications.ModeDisabled,
|
||||
},
|
||||
})
|
||||
|
||||
previousDBEnabled := config.Config.Database.Enabled
|
||||
previousRedis := db.Redis
|
||||
config.Config.Database.Enabled = false
|
||||
db.SetDB(sqliteDB)
|
||||
db.Redis = redisClient
|
||||
|
||||
t.Cleanup(func() {
|
||||
config.Config.Database.Enabled = previousDBEnabled
|
||||
db.SetDB(nil)
|
||||
db.Redis = previousRedis
|
||||
_ = redisClient.Close()
|
||||
mr.Close()
|
||||
})
|
||||
|
||||
// 5. Run the Goose migrations
|
||||
Migrate()
|
||||
|
||||
// 6. Assertions on migrated data schema & records
|
||||
|
||||
// Check that the w_users table exists and contains correct columns/records
|
||||
var usersCount int64
|
||||
if err := sqliteDB.Table("w_users").Count(&usersCount).Error; err != nil {
|
||||
t.Fatalf("count w_users error: %v", err)
|
||||
}
|
||||
if usersCount < 2 {
|
||||
t.Errorf("expected at least 2 users, got %d", usersCount)
|
||||
}
|
||||
|
||||
// Verify user 1 (ryan) details
|
||||
type userStruct struct {
|
||||
ID int64
|
||||
Username string
|
||||
Nickname string
|
||||
Email string
|
||||
IsActive bool
|
||||
IsAdmin bool
|
||||
}
|
||||
var ryanUser userStruct
|
||||
if err := sqliteDB.Table("w_users").Where("id = ?", 1).First(&ryanUser).Error; err != nil {
|
||||
t.Fatalf("query ryan user failed: %v", err)
|
||||
}
|
||||
if ryanUser.Username != "ryan" || ryanUser.Nickname != "Root User ryan" || !ryanUser.IsActive || !ryanUser.IsAdmin {
|
||||
t.Errorf("ryan user migrated incorrectly: %+v", ryanUser)
|
||||
}
|
||||
|
||||
// Verify user 2 (jack) details
|
||||
var jackUser userStruct
|
||||
if err := sqliteDB.Table("w_users").Where("id = ?", 2).First(&jackUser).Error; err != nil {
|
||||
t.Fatalf("query jack user failed: %v", err)
|
||||
}
|
||||
if jackUser.Username != "jack" || jackUser.Nickname != "Normal User jack" || !jackUser.IsActive || jackUser.IsAdmin {
|
||||
t.Errorf("jack user migrated incorrectly: %+v", jackUser)
|
||||
}
|
||||
|
||||
// Verify legacy option migrated all the way to w_system_configs(of_options 已被删除)
|
||||
var geoIPProviderVal string
|
||||
if err := sqliteDB.Table("w_system_configs").Where("key = ?", "geoip_provider").Select("value").Scan(&geoIPProviderVal).Error; err != nil {
|
||||
t.Fatalf("query geoip_provider failed: %v", err)
|
||||
}
|
||||
if geoIPProviderVal != "ip-api" {
|
||||
t.Errorf("geoip_provider value incorrect: got %s, want ip-api", geoIPProviderVal)
|
||||
}
|
||||
|
||||
// Verify origin migrated to of_origins
|
||||
type originStruct struct {
|
||||
ID int64
|
||||
Name string
|
||||
Address string
|
||||
Remark string
|
||||
}
|
||||
var testOrigin originStruct
|
||||
if err := sqliteDB.Table("of_origins").Where("id = ?", 10).First(&testOrigin).Error; err != nil {
|
||||
t.Fatalf("query my_origin failed: %v", err)
|
||||
}
|
||||
if testOrigin.Name != "my_origin" || testOrigin.Address != "127.0.0.1:8080" || testOrigin.Remark != "original origin" {
|
||||
t.Errorf("origin migrated incorrectly: %+v", testOrigin)
|
||||
}
|
||||
|
||||
// Verify apply log migrated to of_apply_logs
|
||||
type applyLogStruct struct {
|
||||
ID int64
|
||||
NodeID string
|
||||
Version string
|
||||
Result string
|
||||
Message string
|
||||
CreatedAt time.Time
|
||||
}
|
||||
var testApplyLog applyLogStruct
|
||||
if err := sqliteDB.Table("of_apply_logs").Where("id = ?", 100).First(&testApplyLog).Error; err != nil {
|
||||
t.Fatalf("query apply_logs failed: %v", err)
|
||||
}
|
||||
if testApplyLog.NodeID != "node_1" || testApplyLog.Version != "v1.0.0" || testApplyLog.Result != "success" || testApplyLog.Message != "applied successfully" {
|
||||
t.Errorf("apply log migrated incorrectly: %+v", testApplyLog)
|
||||
}
|
||||
|
||||
// 7. Verify that all legacy_ prefix tables have been dropped
|
||||
legacyTablesToCheck := []string{
|
||||
"legacy_users",
|
||||
"legacy_options",
|
||||
"legacy_origins",
|
||||
"legacy_apply_logs",
|
||||
"legacy_proxy_routes",
|
||||
"legacy_nodes",
|
||||
"legacy_waf_rule_groups",
|
||||
"legacy_waf_rule_group_bindings",
|
||||
"legacy_waf_ip_groups",
|
||||
"legacy_tls_certificates",
|
||||
"legacy_managed_domains",
|
||||
"legacy_dns_accounts",
|
||||
"legacy_acme_accounts",
|
||||
"legacy_config_versions",
|
||||
"legacy_pages_projects",
|
||||
"legacy_pages_deployments",
|
||||
"legacy_pages_deployment_files",
|
||||
"users",
|
||||
"options",
|
||||
"origins",
|
||||
"apply_logs",
|
||||
"node_access_logs_00",
|
||||
"node_system_profiles",
|
||||
}
|
||||
|
||||
for _, table := range legacyTablesToCheck {
|
||||
var count int
|
||||
if err := sqliteDB.Raw("SELECT COUNT(*) FROM sqlite_master WHERE type='table' AND name=?", table).Scan(&count).Error; err != nil {
|
||||
t.Fatalf("check table %s existence failed: %v", table, err)
|
||||
}
|
||||
if count > 0 {
|
||||
t.Errorf("expected table %s to be dropped, but it still exists", table)
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,7 @@
|
||||
-- +goose Up
|
||||
-- Historical Wavelet→OpenFlare table rename bridge (formerly Go migration).
|
||||
-- Fresh installs and current of_* schemas need no action.
|
||||
SELECT 1;
|
||||
|
||||
-- +goose Down
|
||||
SELECT 1;
|
||||
@@ -0,0 +1,7 @@
|
||||
-- +goose Up
|
||||
-- Historical copy from legacy_* tables into of_* / w_* (formerly Go migration).
|
||||
-- Already-applied environments keep their data; new installs have no legacy_* sources.
|
||||
SELECT 1;
|
||||
|
||||
-- +goose Down
|
||||
SELECT 1;
|
||||
@@ -0,0 +1,8 @@
|
||||
-- +goose Up
|
||||
-- Marker: Zone 域名从旧路由列 / of_managed_domains 的导入由 migrator.Migrate()
|
||||
-- 在全部 goose SQL 应用后自动执行(publicsuffix 根域解析无法用纯 SQL 正确完成)。
|
||||
-- 本文件仅占位版本号,保证升级链路有序:建表 → 导入 → 删旧列。
|
||||
SELECT 1;
|
||||
|
||||
-- +goose Down
|
||||
SELECT 1;
|
||||
@@ -0,0 +1,7 @@
|
||||
-- +goose Up
|
||||
-- Historical Wavelet→OpenFlare table rename bridge (formerly Go migration).
|
||||
-- Fresh installs and current of_* schemas need no action.
|
||||
SELECT 1;
|
||||
|
||||
-- +goose Down
|
||||
SELECT 1;
|
||||
@@ -0,0 +1,7 @@
|
||||
-- +goose Up
|
||||
-- Historical copy from legacy_* tables into of_* / w_* (formerly Go migration).
|
||||
-- Already-applied environments keep their data; new installs have no legacy_* sources.
|
||||
SELECT 1;
|
||||
|
||||
-- +goose Down
|
||||
SELECT 1;
|
||||
@@ -0,0 +1,8 @@
|
||||
-- +goose Up
|
||||
-- Marker: Zone 域名从旧路由列 / of_managed_domains 的导入由 migrator.Migrate()
|
||||
-- 在全部 goose SQL 应用后自动执行(publicsuffix 根域解析无法用纯 SQL 正确完成)。
|
||||
-- 本文件仅占位版本号,保证升级链路有序:建表 → 导入 → 删旧列。
|
||||
SELECT 1;
|
||||
|
||||
-- +goose Down
|
||||
SELECT 1;
|
||||
@@ -12,6 +12,7 @@ import (
|
||||
"fmt"
|
||||
"log"
|
||||
|
||||
"github.com/Rain-kl/Wavelet/internal/apps/openflare/zone"
|
||||
"github.com/Rain-kl/Wavelet/internal/config"
|
||||
"github.com/Rain-kl/Wavelet/internal/db"
|
||||
"github.com/Rain-kl/Wavelet/internal/repository"
|
||||
@@ -34,7 +35,9 @@ func dbType() string {
|
||||
const (
|
||||
dialectSqlite = "sqlite3"
|
||||
dialectPostgres = "postgres"
|
||||
cascadeSuffix = " CASCADE"
|
||||
// zoneImportSQLVersion is the goose SQL marker after of_zones creation and
|
||||
// before drop of legacy route domain columns. Zone data import runs here.
|
||||
zoneImportSQLVersion int64 = 202607120002
|
||||
)
|
||||
|
||||
func gooseDialect() string {
|
||||
@@ -51,7 +54,7 @@ func migrationDir() string {
|
||||
return "goose/postgres"
|
||||
}
|
||||
|
||||
// Migrate 执行数据库迁移
|
||||
// Migrate 执行数据库迁移:全部结构变更走 goose SQL;Zone 历史域名导入在 SQL 之后自动执行。
|
||||
func Migrate() {
|
||||
gormDB := db.DB(context.Background())
|
||||
if gormDB == nil {
|
||||
@@ -70,6 +73,15 @@ func Migrate() {
|
||||
if err := resyncGooseVersionSequence(sqlDB); err != nil {
|
||||
log.Fatalf("[%s] resync goose_db_version sequence failed: %v\n", dbType(), err)
|
||||
}
|
||||
// 1) SQL up to zone-import marker (includes of_zones DDL; still has legacy columns).
|
||||
if err := goose.UpTo(sqlDB, migrationDir(), zoneImportSQLVersion); err != nil {
|
||||
log.Fatalf("[%s] goose migrate (up to zone import) failed: %v\n", dbType(), err)
|
||||
}
|
||||
// 2) Auto-import legacy domains (publicsuffix; idempotent; no-op after phase-2 drop).
|
||||
if err := importZoneDomainsAfterGoose(sqlDB); err != nil {
|
||||
log.Fatalf("[%s] zone domain import failed: %v\n", dbType(), err)
|
||||
}
|
||||
// 3) Remaining SQL (drop legacy columns / managed_domains, later migrations).
|
||||
if err := goose.Up(sqlDB, migrationDir()); err != nil {
|
||||
log.Fatalf("[%s] goose migrate failed: %v\n", dbType(), err)
|
||||
}
|
||||
@@ -79,6 +91,31 @@ func Migrate() {
|
||||
log.Printf("[%s] goose migrate success\n", dbType())
|
||||
}
|
||||
|
||||
// importZoneDomainsAfterGoose 在 SQL 迁移完成后自动导入旧路由/托管域名。
|
||||
// 必须使用 publicsuffix 解析注册根域,故不能放在纯 SQL 中;幂等且在旧列删除后为空操作。
|
||||
func importZoneDomainsAfterGoose(sqlDB *sql.DB) error {
|
||||
ctx := context.Background()
|
||||
tx, err := sqlDB.BeginTx(ctx, nil)
|
||||
if err != nil {
|
||||
return fmt.Errorf("begin zone import transaction: %w", err)
|
||||
}
|
||||
report, err := zone.ImportLegacyTx(ctx, tx, gooseDialect() == dialectPostgres)
|
||||
if err != nil {
|
||||
_ = tx.Rollback()
|
||||
return report.LogAndReturn(err)
|
||||
}
|
||||
if err := tx.Commit(); err != nil {
|
||||
return fmt.Errorf("commit zone import: %w", err)
|
||||
}
|
||||
if report.Zones > 0 || report.Domains > 0 {
|
||||
log.Printf(
|
||||
"[%s] imported zone domains automatically: zones=%d domains=%d\n",
|
||||
dbType(), report.Zones, report.Domains,
|
||||
)
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
// resyncGooseVersionSequence 修复 PostgreSQL 下 goose_db_version.id 自增序列落后于
|
||||
// MAX(id) 的问题(常见于从 dump 恢复或历史迁移以显式 id 复制数据后)。序列落后会
|
||||
// 导致 goose 记录新版本号时 INSERT 命中 goose_db_version_pkey 唯一约束冲突。
|
||||
@@ -115,17 +152,3 @@ func clearSystemConfigCache() {
|
||||
log.Printf("[%s] clear system config cache failed: %v\n", dbType(), err)
|
||||
}
|
||||
}
|
||||
|
||||
func tableExistsSQL(dialect string) string {
|
||||
if dialect == dialectPostgres {
|
||||
return "SELECT count(*) FROM information_schema.tables WHERE table_schema='public' AND table_name=$1"
|
||||
}
|
||||
return "SELECT count(*) FROM sqlite_master WHERE type='table' AND name=?"
|
||||
}
|
||||
|
||||
func tablesWithPrefixSQL(dialect string) string {
|
||||
if dialect == dialectPostgres {
|
||||
return "SELECT table_name FROM information_schema.tables WHERE table_schema='public' AND table_name LIKE $1"
|
||||
}
|
||||
return "SELECT name FROM sqlite_master WHERE type='table' AND name LIKE ?"
|
||||
}
|
||||
|
||||
Reference in New Issue
Block a user