tcjlj/OpenFlare
1
0
Fork 0
mirror of https://github.com/Rain-kl/OpenFlare.git synced 2026-10-02 23:06:36 +08:00
Code Issues Packages Projects Releases Wiki Activity

refactor(structure): group platform, infra, and shared packages

Browse Source 
Reorganize internal packages into platform/infra/shared layers and update
imports, docs, and seed-count tests to match current system configs.
This commit is contained in:
ryan
2026-07-24 15:41:59 +08:00
parent 68d730a388
commit 33a1c32cf8
470 changed files with 646 additions and 646 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/design/index.md
+5 -5
View File
@@ -88,18 +88,18 @@ OpenFlare 已收敛为**单 monorepo**（Go 模块 `github.com/Rain-kl/Wavelet`
| --- | --- |
| `main.go` | Server 启动入口 |
| `internal/cmd/` | Cobra 子命令：`api`、`worker`、`scheduler`、`all`（默认融合模式） |
| `internal/bootstrap/` | 跨模块装配：任务 Handler、推送域事件、进程级初始化 |
| `internal/platform/bootstrap/` | 跨模块装配：任务 Handler、推送域事件、进程级初始化 |
| `internal/router/` | HTTP 路由注册与全局中间件 |
| `internal/router/v1/openflare/` | OpenFlare 路由注册器（`register_*.go`） |
| `internal/apps/openflare/` | OpenFlare 控制面业务域（`routers.go` + `logics.go`） |
| `internal/apps/{admin,user,oauth,upload,cap,...}/` | Wavelet 平台能力（用户、认证、任务、推送等） |
| `internal/apps/openflare/{agent,relay,flared}/` | **Server 侧**边缘协议处理器（鉴权、心跳、WS） |
| `internal/model/` | GORM 实体（`openflare_*.go` + 平台模型） |
| `internal/db/migrator/goose/` | goose SQL 迁移（PostgreSQL / SQLite / ClickHouse） |
| `internal/infra/persistence/migrator/goose/` | goose SQL 迁移（PostgreSQL / SQLite / ClickHouse） |
| `internal/repository/` | 平台域数据访问层 |
| `internal/task/` | Asynq 异步任务（Worker + Scheduler） |
| `internal/config/` | Viper 配置加载 |
| `internal/common/` | 统一 API 响应封装（`response/`） |
| `internal/infra/task/` | Asynq 异步任务（Worker + Scheduler） |
| `internal/infra/config/` | Viper 配置加载 |
| `internal/shared/` | 统一 API 响应封装（`response/`） |
| `pkg/protocol/` | Relay / Tunnel 共享 HTTP/WS 协议结构 |
| `pkg/render/`、`pkg/geoip/`、`pkg/wsclient/` | OpenResty 配置渲染、GeoIP、WebSocket 客户端 |
docs/design/zone-design.md
+1 -1
View File
@@ -90,7 +90,7 @@ WAF、Pages、上游与发布版本仍属于 `proxy_routes`。Zone 概览只聚
* 持久化：域名与证书只存在于 `of_zone_domains`；`of_proxy_routes` 仅保存路由策略（上游、缓存、限流、WAF 绑定键等）。
* 渲染：配置快照在内存中组装临时 `Domains` / `DomainCertIDs` 供 OpenResty 渲染，不写回数据库。
* 结构迁移仅使用 `internal/db/migrator/goose/{postgres,sqlite}/*.sql`；启动时自动导入历史域名，第二阶段后旧列不存在则为空操作。
* 结构迁移仅使用 `internal/infra/persistence/migrator/goose/{postgres,sqlite}/*.sql`；启动时自动导入历史域名，第二阶段后旧列不存在则为空操作。
## 验证
docs/plan/20260712-zone-domain-refactor.md
+12 -12
View File
@@ -25,7 +25,7 @@
| 路径 | 职责 |
| --- | --- |
| `internal/db/migrator/goose/{postgres,sqlite}/202607120001_create_zone_domain_tables.sql` | 第一阶段 Zone/ZoneDomain DDL 与索引。 |
| `internal/infra/persistence/migrator/goose/{postgres,sqlite}/202607120001_create_zone_domain_tables.sql` | 第一阶段 Zone/ZoneDomain DDL 与索引。 |
| `internal/model/openflare_zone.go` | Zone、ZoneDomain 模型及数据访问。 |
| `internal/apps/openflare/zone/{logics.go,routers.go,errs.go,legacy_import.go}` | Zone CRUD、概览、输入验证和历史导入。 |
| `internal/cmd/migrate_zones.go` | 显式、可重复运行的历史 Zone 数据导入命令。 |
@@ -36,17 +36,17 @@
| `frontend/vitest.config.ts`、`frontend/tests/zone/*.test.tsx` | Zone 页面与域名选择器的最小前端测试运行环境。 |
| `frontend/app/(main)/websites/*` | Zone 列表、`[zoneId]` 动态详情页和局部组件。 |
| `frontend/app/(main)/proxy-routes/*` | Zone 域名选择器替换旧域名/证书编辑器。 |
| `internal/db/migrator/goose/{postgres,sqlite}/202607130001_drop_legacy_route_domain_columns.sql` | 第二阶段删除旧表、列与索引。 |
| `internal/infra/persistence/migrator/goose/{postgres,sqlite}/202607130001_drop_legacy_route_domain_columns.sql` | 第二阶段删除旧表、列与索引。 |
### Task 1: 第一阶段 Schema、模型与迁移测试
**Files:**
- Create: `internal/db/migrator/goose/postgres/202607120001_create_zone_domain_tables.sql`
- Create: `internal/db/migrator/goose/sqlite/202607120001_create_zone_domain_tables.sql`
- Create: `internal/infra/persistence/migrator/goose/postgres/202607120001_create_zone_domain_tables.sql`
- Create: `internal/infra/persistence/migrator/goose/sqlite/202607120001_create_zone_domain_tables.sql`
- Create: `internal/model/openflare_zone.go`
- Create: `internal/model/openflare_zone_test.go`
- Modify: `internal/model/openflare_proxy_route.go`
- Test: `internal/db/migrator/migrator_test.go`
- Test: `internal/infra/persistence/migrator/migrator_test.go`
**Interfaces:**
- Produces: `model.Zone`, `model.ZoneDomain`, `ListZoneDomainsByRouteID(ctx, routeID)`, `ReplaceZoneDomainRouteBindings(ctx, routeID, domainIDs)`.
@@ -60,7 +60,7 @@ func TestReplaceZoneDomainRouteBindingsRejectsForeignDomain(t *testing.T) {
}
```
Run: `go test ./internal/model ./internal/db/migrator -run 'Zone|Migrat' -count=1`
Run: `go test ./internal/model ./internal/infra/persistence/migrator -run 'Zone|Migrat' -count=1`
Expected: FAIL，因为 Zone 模型和 goose 文件尚不存在。
@@ -107,14 +107,14 @@ func ReplaceZoneDomainRouteBindings(ctx context.Context, routeID uint, domainIDs
- [x] **Step 4: 运行模型与迁移测试**
Run: `go test ./internal/model ./internal/db/migrator -run 'Zone|Migrat' -count=1`
Run: `go test ./internal/model ./internal/infra/persistence/migrator -run 'Zone|Migrat' -count=1`
Expected: PASS，空 SQLite 库可应用迁移，唯一域名和绑定排他性受保护。
- [x] **Step 5: Commit**
```bash
git add internal/db/migrator/goose internal/model
git add internal/infra/persistence/migrator/goose internal/model
git commit -m "feat(zone): add normalized zone domain schema"
```
@@ -448,8 +448,8 @@ git commit -m "docs(zone): add migration and release verification guide"
**Precondition:** 已在生产环境完成 Task 7 的导入、预览对比和至少一次发布/回滚验证；`migrate-zones` 报告无冲突。
**Files:**
- Create: `internal/db/migrator/goose/postgres/202607130001_drop_legacy_route_domain_columns.sql`
- Create: `internal/db/migrator/goose/sqlite/202607130001_drop_legacy_route_domain_columns.sql`
- Create: `internal/infra/persistence/migrator/goose/postgres/202607130001_drop_legacy_route_domain_columns.sql`
- Create: `internal/infra/persistence/migrator/goose/sqlite/202607130001_drop_legacy_route_domain_columns.sql`
- Delete: `internal/model/openflare_managed_domain.go`
- Delete: `internal/apps/openflare/tls/managed_domain.go`
- Delete: `internal/apps/openflare/tls/helpers.go` 中仅用于旧路由证书数组的函数
@@ -473,14 +473,14 @@ PostgreSQL 删除旧唯一索引和 `domain`、`domains`、`cert_id`、`cert_ids
- [x] **Step 4: 验证升级和完整回归**
Run: `go test ./internal/db/migrator ./internal/model ./internal/apps/openflare/... ./pkg/render/openresty -count=1 && make code-check`
Run: `go test ./internal/infra/persistence/migrator ./internal/model ./internal/apps/openflare/... ./pkg/render/openresty -count=1 && make code-check`
Expected: PASS；全仓搜索不再发现旧 `ManagedDomain` 业务代码、`ProxyRoute` 持久化字段或管理端 API；渲染快照中的临时 `DomainCertIDs` 类型允许保留。
- [x] **Step 5: Commit**
```bash
git add internal/db/migrator internal/model internal/apps frontend docs
git add internal/infra/persistence/migrator internal/model internal/apps frontend docs
git commit -m "refactor(zone): remove legacy route domain storage"
```
docs/plan/20260713-waf-orchestration.md
+7 -7
View File
@@ -65,10 +65,10 @@ flowchart LR
### Task 1: 数据库迁移与持久化模型
**Files:**
- Create: `internal/db/migrator/goose/postgres/202607150001_orchestrate_waf_rules.sql`
- Create: `internal/db/migrator/goose/sqlite/202607150001_orchestrate_waf_rules.sql`
- Create: `internal/db/migrator/goose/postgres/202607150002_reset_waf_rule_graphs.sql`
- Create: `internal/db/migrator/goose/sqlite/202607150002_reset_waf_rule_graphs.sql`
- Create: `internal/infra/persistence/migrator/goose/postgres/202607150001_orchestrate_waf_rules.sql`
- Create: `internal/infra/persistence/migrator/goose/sqlite/202607150001_orchestrate_waf_rules.sql`
- Create: `internal/infra/persistence/migrator/goose/postgres/202607150002_reset_waf_rule_graphs.sql`
- Create: `internal/infra/persistence/migrator/goose/sqlite/202607150002_reset_waf_rule_graphs.sql`
- Modify: `internal/model/openflare_waf.go`
- Create: `internal/model/openflare_waf_graph_test.go`
@@ -115,7 +115,7 @@ func UpdateOpenFlareWAFRuleGraph(ctx context.Context, id uint, revision uint64,
- [ ] **Step 5: 运行测试并提交**
Run: `go test ./internal/model ./internal/db/migrator/... -count=1`
Run: `go test ./internal/model ./internal/infra/persistence/migrator/... -count=1`
Expected: PASS。
@@ -571,8 +571,8 @@ Commit: `refactor(frontend): order waf bindings and remove legacy editor`
### Task 11: 旧后端字段清理、Swagger、中文文档与端到端验证
**Files:**
- Create: `internal/db/migrator/goose/postgres/202607150003_drop_legacy_waf_rule_fields.sql`
- Create: `internal/db/migrator/goose/sqlite/202607150003_drop_legacy_waf_rule_fields.sql`
- Create: `internal/infra/persistence/migrator/goose/postgres/202607150003_drop_legacy_waf_rule_fields.sql`
- Create: `internal/infra/persistence/migrator/goose/sqlite/202607150003_drop_legacy_waf_rule_fields.sql`
- Modify: `internal/model/openflare_waf.go`
- Modify: `internal/apps/openflare/waf/logics_test.go`
- Modify: `docs/design/waf-design.md`
docs/plan/20260718-observability-ch-migrate-runbook.md
+1 -1
View File
@@ -37,7 +37,7 @@ make run # 以项目实际入口为准
clickhouse-client --host 127.0.0.1 --port 9000 \
--user default --password "$CLICKHOUSE_PASSWORD" \
--database openflare \
--multiquery < internal/db/migrator/goose/clickhouse/202607180003_backfill_access_log_hourly.sql
--multiquery < internal/infra/persistence/migrator/goose/clickhouse/202607180003_backfill_access_log_hourly.sql
```
（goose 文件含 `+goose Up` 注释，若 client 报错可去掉注释行后执行 INSERT 主体。）
docs/plan/20260719-http-default-rate-limit.md
+5 -5
View File
@@ -23,7 +23,7 @@
| 文件 | 职责 |
|------|------|
| `internal/model/system_configs.go` | 三个 ConfigKey 常量 |
| `internal/db/migrator/goose/{postgres,sqlite}/202607190001_add_openresty_default_rate_limits.sql` | seed 默认值 |
| `internal/infra/persistence/migrator/goose/{postgres,sqlite}/202607190001_add_openresty_default_rate_limits.sql` | seed 默认值 |
| `internal/apps/openflare/option/openresty_validators.go` + `validate.go` | 全局默认校验 |
| `internal/apps/openflare/config_version/snapshot.go` | 快照字段 + 读取 |
| `internal/apps/openflare/config_version/logics.go` | option diff keys |
@@ -273,8 +273,8 @@ git commit -m "feat(openresty): merge global default limits at route render"
**Files:**
- Modify: `internal/model/system_configs.go`
- Create: `internal/db/migrator/goose/postgres/202607190001_add_openresty_default_rate_limits.sql`
- Create: `internal/db/migrator/goose/sqlite/202607190001_add_openresty_default_rate_limits.sql`
- Create: `internal/infra/persistence/migrator/goose/postgres/202607190001_add_openresty_default_rate_limits.sql`
- Create: `internal/infra/persistence/migrator/goose/sqlite/202607190001_add_openresty_default_rate_limits.sql`
- Modify: `internal/apps/openflare/option/validate.go`
- Modify: `internal/apps/openflare/option/openresty_validators.go`
- Modify: `internal/apps/openflare/config_version/snapshot.go`
@@ -407,8 +407,8 @@ Expected: PASS
```bash
git add internal/model/system_configs.go \
internal/db/migrator/goose/postgres/202607190001_add_openresty_default_rate_limits.sql \
internal/db/migrator/goose/sqlite/202607190001_add_openresty_default_rate_limits.sql \
internal/infra/persistence/migrator/goose/postgres/202607190001_add_openresty_default_rate_limits.sql \
internal/infra/persistence/migrator/goose/sqlite/202607190001_add_openresty_default_rate_limits.sql \
internal/apps/openflare/option/validate.go \
internal/apps/openflare/option/openresty_validators.go \
internal/apps/openflare/config_version/snapshot.go \
docs/plan/20260719-pages-source-sync-v2.md
+8 -8
View File
@@ -396,7 +396,7 @@ PostgreSQL 与 SQLite 均创建同语义的部分唯一索引。禁止用空字
| `of_pages_source_scan` | `openflare:pages_source_scan` | 恢复过期 lease/orphan record；串行检查一批到期 GitHub latest source；必要时投递 sync action |
| `of_pages_source_action` | `openflare:pages_source_action` | 执行管理员 check/sync 或 scanner 触发的 sync |
两者都在 `internal/task/handlers/register.go` 显式注册 Handler 与 TaskMeta。现有 `bootstrap.RegisterTasks()` 已覆盖 API、worker、scheduler 和 all 入口，不新增 `init()`，也不修改 `internal/router/router.go`、`internal/bootstrap/bootstrap.go` 或 `internal/cmd` 的装配职责。
两者都在 `internal/infra/task/handlers/register.go` 显式注册 Handler 与 TaskMeta。现有 `bootstrap.RegisterTasks()` 已覆盖 API、worker、scheduler 和 all 入口，不新增 `init()`，也不修改 `internal/router/router.go`、`internal/platform/bootstrap/bootstrap.go` 或 `internal/cmd` 的装配职责。
两类任务都标记为 `TaskMeta.InternalOnly=true`。通用 Admin Task 类型列表、手工 dispatch 与 schedule 创建/更新必须隐藏或拒绝 internal-only meta；scheduler 与 Pages 内部 dispatch 仍使用完整 registry。这样客户端不能绕过 Pages Handler 自行伪造 `source_id`、`config_version` 或 `actor`。
@@ -1103,7 +1103,7 @@ scanner TaskResult 和结构化日志至少记录：
* 注册 5 条 source 路由；不在顶层 router 直接挂业务 Handler。
#### [MODIFY] `internal/task/handlers/register.go`
#### [MODIFY] `internal/infra/task/handlers/register.go`
* 显式注册 scanner/action Handler 与 TaskMeta。
@@ -1143,13 +1143,13 @@ scanner TaskResult 和结构化日志至少记录：
#### [NEW] PostgreSQL
* `internal/db/migrator/goose/postgres/202607190002_add_pages_source_runtime.sql`
* `internal/db/migrator/goose/postgres/202607190003_seed_pages_source_scan.sql`
* `internal/infra/persistence/migrator/goose/postgres/202607190002_add_pages_source_runtime.sql`
* `internal/infra/persistence/migrator/goose/postgres/202607190003_seed_pages_source_scan.sql`
#### [NEW] SQLite
* `internal/db/migrator/goose/sqlite/202607190002_add_pages_source_runtime.sql`
* `internal/db/migrator/goose/sqlite/202607190003_seed_pages_source_scan.sql`
* `internal/infra/persistence/migrator/goose/sqlite/202607190002_add_pages_source_runtime.sql`
* `internal/infra/persistence/migrator/goose/sqlite/202607190003_seed_pages_source_scan.sql`
版本号若已被其它分支占用，实施时只顺延编号，不改变 DDL/DML 拆分。
@@ -1257,7 +1257,7 @@ API 实现后运行 `make swagger` 更新 `docs/docs.go`、`docs/swagger.json`
建议：
```bash
go test ./internal/db/migrator ./internal/model
go test ./internal/infra/persistence/migrator ./internal/model
```
### 4.2 source、Provider 与 API
@@ -1482,7 +1482,7 @@ make code-check
* Pages、Agent、GitHub Release integration、归档、上传和迁移相关 Go 定向测试通过；关键并发路径的 race 测试通过。
* SQLite Pages source 与 scanner schedule migration 的 Up/Down/Up 通过。
* 前端 Pages 四个测试文件共 34 条用例全部通过，相关 TypeScript 与 ESLint 检查通过。
* 全仓 Go 测试中 `internal/apps/openflare/pages`、`internal/apps/openflare/agent`、`internal/integration/githubrelease`、`internal/db/migrator`、`internal/apps/upload/*` 与 `pkg/pagesarchive` 均通过。
* 全仓 Go 测试中 `internal/apps/openflare/pages`、`internal/apps/openflare/agent`、`internal/integration/githubrelease`、`internal/infra/persistence/migrator`、`internal/apps/upload/*` 与 `pkg/pagesarchive` 均通过。
### 7.3 全仓测试中仍存在的范围外失败
docs/plan/clickhouse-cpu-optimization.md
+3 -3
View File
@@ -13,13 +13,13 @@ ClickHouse CPU 偏高由写入侧（小 part 频繁 flush、心跳同步 DELETE
### PR 1: 写入路径 P0 优化
- **Description:** 移除心跳路径同步 `ALTER DELETE`；为 `batchwriter` 增加 `MinBatchSize`；调大可观测 writer 批次与 flush 间隔；为 openresty/frps/frpc 补全去重。
- **Files/components affected:** `internal/apps/openflare/agent/observability.go`, `internal/db/batchwriter/`, `internal/apps/openflare/chwriter/`, `internal/db/batchwriter/*_test.go`
- **Files/components affected:** `internal/apps/openflare/agent/observability.go`, `internal/infra/persistence/batchwriter/`, `internal/apps/openflare/chwriter/`, `internal/infra/persistence/batchwriter/*_test.go`
- **Dependencies:** None
### PR 2: ClickHouse 客户端与配置 P1
- **Description:** 启用 `async_insert` 等写入优化 settings；提高 `block_buffer_size` 默认值；更新 `config.example.yaml` 与配置模型注释。
- **Files/components affected:** `internal/db/clickhouse.go`, `internal/config/model.go`, `internal/config/config.go`, `config.example.yaml`
- **Files/components affected:** `internal/infra/persistence/clickhouse.go`, `internal/infra/config/model.go`, `internal/infra/config/config.go`, `config.example.yaml`
- **Dependencies:** None
### PR 3: Dashboard 与可观测查询 P0
@@ -37,7 +37,7 @@ ClickHouse CPU 偏高由写入侧（小 part 频繁 flush、心跳同步 DELETE
### PR 5: ClickHouse DDL 与数据规范化 P1
- **Description:** 为 7 张分析表添加 TTL；收窄 `of_node_access_logs` ORDER BY；插入时规范化 `remote_addr`（去 trim 查询）；将可观测 obs 三表纳入自动清理。
- **Files/components affected:** `internal/db/migrator/goose/clickhouse/`, `internal/repository/analytics/node_access_log_writer.go`, `internal/apps/openflare/tasks/database_cleanup.go`, `internal/model/analytics/`
- **Files/components affected:** `internal/infra/persistence/migrator/goose/clickhouse/`, `internal/repository/analytics/node_access_log_writer.go`, `internal/apps/openflare/tasks/database_cleanup.go`, `internal/model/analytics/`
- **Dependencies:** PR 1
### PR 6: 基础设施与审计减负 P2
docs/superpowers/plans/2026-07-19-http-default-rate-limit.md
+5 -5
View File
@@ -23,7 +23,7 @@
| 文件 | 职责 |
|------|------|
| `internal/model/system_configs.go` | 三个 ConfigKey 常量 |
| `internal/db/migrator/goose/{postgres,sqlite}/202607190001_add_openresty_default_rate_limits.sql` | seed 默认值 |
| `internal/infra/persistence/migrator/goose/{postgres,sqlite}/202607190001_add_openresty_default_rate_limits.sql` | seed 默认值 |
| `internal/apps/openflare/option/openresty_validators.go` + `validate.go` | 全局默认校验 |
| `internal/apps/openflare/config_version/snapshot.go` | 快照字段 + 读取 |
| `internal/apps/openflare/config_version/logics.go` | option diff keys |
@@ -273,8 +273,8 @@ git commit -m "feat(openresty): merge global default limits at route render"
**Files:**
- Modify: `internal/model/system_configs.go`
- Create: `internal/db/migrator/goose/postgres/202607190001_add_openresty_default_rate_limits.sql`
- Create: `internal/db/migrator/goose/sqlite/202607190001_add_openresty_default_rate_limits.sql`
- Create: `internal/infra/persistence/migrator/goose/postgres/202607190001_add_openresty_default_rate_limits.sql`
- Create: `internal/infra/persistence/migrator/goose/sqlite/202607190001_add_openresty_default_rate_limits.sql`
- Modify: `internal/apps/openflare/option/validate.go`
- Modify: `internal/apps/openflare/option/openresty_validators.go`
- Modify: `internal/apps/openflare/config_version/snapshot.go`
@@ -407,8 +407,8 @@ Expected: PASS
```bash
git add internal/model/system_configs.go \
internal/db/migrator/goose/postgres/202607190001_add_openresty_default_rate_limits.sql \
internal/db/migrator/goose/sqlite/202607190001_add_openresty_default_rate_limits.sql \
internal/infra/persistence/migrator/goose/postgres/202607190001_add_openresty_default_rate_limits.sql \
internal/infra/persistence/migrator/goose/sqlite/202607190001_add_openresty_default_rate_limits.sql \
internal/apps/openflare/option/validate.go \
internal/apps/openflare/option/openresty_validators.go \
internal/apps/openflare/config_version/snapshot.go \