mirror of
https://github.com/Rain-kl/OpenFlare.git
synced 2026-09-30 06:16:37 +08:00
ryan
dbaa3bf140
docs(changelog): 修正表述笔误
refactor(cordis): 磁盘缓存改用上上游能力并清理本地副本
按上游/下游归属规约：类型断言守卫已回流 Wavelet（f3d85d5，附回归用例），
本仓库删除 OpenFlare/plugins/server/pkg/cache 整包并改 import 到
Wavelet/pkg/cache/disk，同步后与上游零漂移。
验证：go build 通过；go test ./... exit 0（137 包 ok）；256 条路由对拍与
232 条 swagger 操作均零差异；make build-all 四进制；前端零改动。
docs(cordis): 记录 T1 清理结果与五个复用阻塞点
refactor(cordis): server 复用上游 pkg 能力并删除等价本地副本
按上游/下游归属规约清理重复实现，删除 7 个与上游等价的本地包并改 import：
shared/response→pkg/response、pkg/{logger,mail,trace,httppool,cache/ram}→
上游同名包、infra/persistence/batchwriter→pkg/batchwriter。逐项核过差异：
httppool 逐字节相同；logger 的 Config 字段完全一致；response 的 7 个 Abort*
一致；cache/ram 换过去顺带把裸 go 变回带 panic 恢复的 util.Go。
两处非等价差异按语义处理：
- batchwriter.Stats 与 status DTO 原为类型别名，改为消费侧逐字段转换，
避免 model 反向依赖基础设施类型；
- 上游 pkg/idgen 要求显式 Init（本地副本为懒加载自动初始化），本次保留本地
副本，待与 infra 初始化一并迁移（已登记在清理计划）。
验证：go build 通过；go test ./... exit 0（138 包 ok）；256 条路由对拍零差异；
make swagger 232 条操作零增减，且归一化后与旧文档深度相等——差异仅为
response.Any / logger.LogEntry 两个定义名随包路径改名，接口形状未变。
chore(cordis): 回流内核与 pkg/util 通用能力并清理 vendoring 污染
按新增的上游/下游归属规约：HandleRaw/BasePath 与版本比较、网络、格式化助手
属通用能力，已提交到 Wavelet 分支 feat/cordis-router-raw-routes，本仓库改为
纯同步获取（pkg/util 已零漂移），补丁登记保留至上游合并。
同时修掉我此前 git add -A 造成的污染：首次 vendoring 把上游工作区里被
gitignore 的运行期产物一起提交进来（upload 的 diskcache 缓存块 650 个与
driver_http/dist 前端构建物 380 个，共 12872 行/1030 文件）。sync-upstream.sh
现显式排除 uploads/dist/data/*.db，.gitignore 补上对应兜底规则。
AGENTS.md 增加上游/下游改动归属规约，并把仍指向前 Cordis 布局的硬性约束
（internal/router + Serve、internal/repository/logstore、internal/platform/bootstrap、
internal/cmd）改到当前插件路径。
验证：go build 通过；go test ./... exit 0（144 包 ok）；make swagger 232 条
操作与基线逐条一致；make build-all 四进制；gofmt 干净。
feat(cordis): server 插件化并改由内核挂载控制面路由
新增 plugins/server/plugin.go：Apply 以 ctx.Router().Group(app.api_prefix)
声明根级与 /v1 全部路由；33 个注册函数由 *gin.RouterGroup 改为
core.RouterExtension，RegisterCollection 改用内核新增的 HandleRaw 保留
尾部斜杠变体，AdminMiddlewares 返回 []any（Go 不允许把 []T 展开为 ...any）。
删除 router.Serve 与 registerRoutes，装配根改为 core.App +
driver_http.New(WithEngine(router.BuildEngine()))，监听、信号与优雅退出归内核；
前端 SPA 的 NoRoute 兜底因内核暂无贡献点而保留在引擎层。
路由保真证据：plugin_parity_test 对拍 baseline/routes-engine.txt 的 256 条
(方法 路径) 零差异；go test ./... exit 0（144 包 ok，含真实 handler 的
openflare/integration 用例走同一条挂载路径）；make swagger 232 条操作与基线
逐条一致；golangci-lint 0 issues；make build-all 四进制；embed_frontend
标签编译通过；前端零改动。
已知待补：带 Redis 的实机 HTTP 冒烟（本机 6379 未启动，session store 与
改造前一样在建店阶段即 fatal），以及 bootstrap 的任务/设置/迁移注册迁入 Apply。
feat(core): RouterExtension 增加 HandleRaw 与 BasePath 以保真尾部斜杠路由
server 插件化的前置：Handle 经 cleanPath 会剥掉尾部斜杠，无法表达
/resource 与 /resource/ 两条不同路由，而 OpenFlare 有 20 个历史 list
端点两者都注册且部署关闭了 RedirectTrailingSlash，缺失即 404。新增
HandleRaw 与 BasePath（作用域包装器同样登记反注册），补 extpoints 用例；
并把 router.Serve 拆出 BuildEngine 以便交给 driver_http.WithEngine 复用，
新增路由表导出 harness，固化 256 条 (方法 路径) 基线供插件化对拍。
上游补丁登记于 backend/OpenFlare/upstream-patches.md，同步脚本改为按目录
前缀输出差异并在同步后提醒确认补丁是否仍在。
验证：go build 通过；go test ./... exit 0（143 包 ok）；gofmt 干净。
docs(cordis): 记录 server 插件接入内核的可行路径与内核能力缺口
feat(cordis): agent/relay/flared 落地为内核驱动插件
三个边缘守护进程各新增 plugin.go，实现 core.Plugin + core.Driver
（自定义 DriverType 与同名 profile），装配与生命周期从 main 迁入
Apply/Start/Stop：Apply 负责 JSON 配置加载、运行环境与用户确保、
openresty/frps/frpc 管理器与各服务装配；Start 以 util.Go 拉起阻塞式
runner 与 GeoIP 周期更新；Stop 收敛主循环结果并在超时时报错而非静默。
入口改为 core.NewApp(core.WithProfile(...)) + Prepare/Run，保持
-config 旗标、默认路径、退出码与启动/停止日志不变。
验证：go build 通过；go test ./... exit 0（143 包 ok，含 3 个插件身份
与配置失败路径测试）；make build-all 四进制产出；三进制实跑缺失配置
均 exit 1 且错误链保留 load {agent,relay,flared} config 原因；gofmt 干净。
refactor(cordis): 按功能职责拆分为 4 个插件与 share 共享层
backend/OpenFlare 不再平铺遗留分层，改为 plugins/{server,agent,relay,flared}
加 share/：控制面业务（openflare/admin/oauth/user/upload/cap/config/health 与
repository/model/infra/router 等支撑层）归 server；三个边缘守护进程各自成插件；
被两个以上插件消费的 protocol/geoip/wsclient/render/pagesarchive/edge 归 share。
同时把 pkg/util 与 buildinfo 合并回上游 pkg（上游已覆盖全部符号，仅 8 个函数与
2 个类型为 OpenFlare 独有，已一并迁入），装配根统一到 backend/cmd（含三个 daemon
入口），Dockerfile 与 release 工作流的构建路径和 -X 注入路径同步更新。
验证：go build 通过；go test ./... exit 0（141 包 ok）；make swagger exit 0 且
232 条 API 操作与基线逐条一致；make build-all 产出 4 进制；-X 注入经二进制
strings 实测生效；日志后端直连门禁改写为按 server 插件业务域扫描并在扫描数为 0
时报错（防门禁静默失效）；前端零改动。
feat(cordis): 落地 backend/share 共享层与上游同步脚本
跨插件共享资源（控制消息协议、GeoIP+iputil、边缘守护进程日志）从下游包
移入 backend/share，并声明其只能依赖 core/pkg 与标准/第三方库，禁止反向
引用下游业务与具体插件实现；新增 scripts/sync-upstream.sh 只覆盖
backend/{core,pkg,plugins}，同步后 --check 报告零差异，证明与上游逐字一致。
go build 通过，go test ./... exit 0（142 包 ok），前端零改动。
refactor(cordis): 采用与 Wavelet 同构的单模块布局并引入上游内核
按上游结构落位：backend/{core,pkg,plugins} 为 Wavelet 上游拷贝，OpenFlare
全部业务收拢到上游 downstream 所对应的位置 backend/OpenFlare/，模块名保持
Wavelet 以保证上游 import 路径逐字一致、同步零改写；三个 daemon 入口移至
backend/OpenFlare/cmd，backend/cmd 与 main.go 作为控制面装配根。
行为不变：go build 通过，142 个测试包全绿（含上游插件测试），232 条 API
操作与改造前逐条一致，四进制产物正常，前端零改动。swagger 暂只扫描下游代码，
待 P4 挂载上游路由后再纳入 plugins/。
style: 修正模块路径改写导致的 import 分组排序漂移
refactor(layout): Go 代码迁入 backend/ 并将模块名简化为 OpenFlare
对齐上游 Wavelet 的仓库布局，为以第二 module 形态 vendoring Cordis 内核与
平台插件做准备：模块路径整体改写为 OpenFlare，Go 目标加 cd backend，
swaggo 产物移至 backend/docs 并把 json/yaml 复制回 docs/ 供站点消费，
Dockerfile 与 release 工作流的构建目录、ldflags 模块路径同步更新。
行为保持不变：232 条路由与改造前逐条一致，95 个测试包全绿，
四进制产物正常，前端零改动。
chore(cordis): 落地改造计划与 schema/路由基线
新增 legacy_dump_test 迁移快照 harness：在临时 sqlite 库上按生产顺序
（goose.UpTo → zone 导入 → goose.Up）跑完 76 个历史迁移并导出 schema 与
版本序列，作为改造前后一致性门禁的唯一事实来源。同时记录 232 条路由清单
与 foundation 实施计划。
docs(cordis): add OpenFlare Cordis 架构改造设计
明确上游以第二 module 形态 vendoring 进 backend/Wavelet、4 个插件
(server/agent/relay/flared) 全部装载内核，并规定保留 76 个历史 goose
迁移 + 一次性版本 stamp 桥接的迁移方案，配套三方 schema 一致性门禁，
确保已部署库不重跑历史、不丢数据。
16 KiB
16 KiB
Product Boundaries
You will learn: what OpenFlare is, its current stable capabilities, and the core product boundaries and repository structure layout you must follow when developing.
OpenFlare is a self-hosted OpenResty control plane for single-team or single-organization internal operations.
Project Positioning
OpenFlare suits teams that need to centrally manage multiple OpenResty proxy nodes, with this positioning:
- Control/landing separation: the Server control plane doesn't SSH into proxy nodes; Agents actively pull versions and apply them.
- Immutable config release: full config versions are used for preview, release, activation, and one-click rollback.
- Integrated gateway hosting: website reverse proxying, automatic TLS certificate issuance/renewal, WAF protection, intranet penetration (Tunnel), and Pages static hosting are integrated into one control plane.
Not this product's positioning: multi-tenant cloud platforms, Kubernetes Ingress Controllers, service meshes, or general log platforms.
Current Capabilities
| Capability | Description | Detailed Design/Usage |
|---|---|---|
| Reverse proxy config management | website rules (Proxy Route) as the aggregation boundary; multi-domain and multi-upstream load balancing | Create a Reverse Proxy Config |
| Origin error page | globally configurable: matching origin/gateway status codes return OpenFlare default or custom HTML with the HTTP status kept | Origin Error Page Design |
| Edge cache | single-node OpenResty proxy_cache; default static extensions + origin-header/Set-Cookie gates + default Edge TTL (benchmarked to the CF default model) |
Edge Cache Strategy Design |
| Zone & domain management | registrable root domains as the management entry, aggregating explicit domains, domain certificates, and reverse proxy routes | Zone & Domain Resource Design |
| Cloudflare DNS pointing | per ZoneDomain, idempotently point a single Cloudflare A record at an edge node IPv4; connection config, groups, member orange-cloud, and async sync; no auto-failover in phase 1 | Cloudflare DNS Pointing Design |
| Config versioning | global single active version with preview, release, immutable snapshot history, and second-level one-click rollback | Agent & Publish Model |
| WAF protection | visual DAG rule orchestration, manual/auto/subscription IP groups, GeoIP matching, and PoW CC protection | WAF Design / WAF Orchestration Rule Design / WAF Usage Guide |
| Intranet penetration | reverse-penetrate and expose intranet web services via Relay nodes and the OpenFlared client | Tunnel Design / Tunnel Usage Guide |
| Pages static hosting | upload or sync pre-built artifacts from Remote URLs or public GitHub Releases; GitHub latest can be periodically checked and optionally auto-published. Immutable deployments are pulled by edge nodes and served locally by OpenResty, supporting rollback, API proxying, and SPA Fallback | Pages Static Hosting Design / Pages Usage Guide |
| TLS certificate auto-renewal | explicitly bind certificates to Zone domains; issue/renew via ACME against Let's Encrypt | Zone & Domain Resource Design |
| Multi-node monitoring & observability | access logs as the single truth for business traffic; Agent reports only details and host readings, Server aggregates uniformly; reconciled with Zone/dashboard | Observability Transport Model / Edge Observability & Business Traffic Stats / Reporting Protocol & Tables / System Architecture |
| Log storage | access logs and observability time series use the switchable log primary DB (follows the business primary DB or ClickHouse); still writable/queryable with ClickHouse off | Log Store Decoupling |
| Console bilingual | zh-CN / en without URL prefixes, NEXT_LOCALE cookie precedence, static-export compatible |
Frontend i18n design |
Core Product Boundaries and Constraints
When developing and contributing code, you must strictly follow these business boundaries and technical constraints; don't bypass them for temporary needs:
1. Website Config and Upstream Constraints
- Single-site domain sharing policy: one route rule corresponds to one website; the site's multiple domains share rate limit, cache, and reverse-proxy upstream config. Differential per-domain service config within the same rule is not supported.
- Upstream type mutual exclusion: the upstream must be one of direct address (
direct), intranet tunnel (tunnel), or Pages static hosting (pages); mixing within one rule is not allowed. - Direct type restrictions: a direct upstream can be a single or multiple pure
http://orhttps://addresses (multi-address only supports plainscheme://host[:port]); non-HTTP protocols (TCP/UDP) upstreams are not supported.
2. WAF Security Boundaries
- Allowlist priority: the allowlist has absolute matching power. Only when an allowlist rule isn't hit do the global and custom blocklist filters trigger in order.
- GeoIP weak dependency: geo access resolution fully depends on the node-local MaxMind DB. When GeoIP is abnormal or fails to resolve, the system must auto-ignore geo rules — never break IP-group filtering or the reverse-proxy main chain's availability.
- Runtime data decoupling: OpenResty interception only reads Agent-synced local JSON, never talking to the Server DB. IP group member sync is decoupled from version release via Checksum differential pull for zero-reload smooth effect.
3. Intranet Penetration Boundaries
- HTTP traffic only: the tunnel components only support HTTP/HTTPS (based on frp's vhost mechanism for single-port domain-route reuse); standalone TCP/UDP port allocation is not supported yet.
- Dynamic relay config control: a Relay node, after connecting to the Server, dynamically pulls and syncs global system config via heartbeats (e.g. whether the embedded FRPS Web UI and its port are enabled), but isn't part of the control plane's immutable config version release system.
- Tunnel/Node system isolation: Tunnel clients make outbound connections from the intranet and are independent entities from control-plane-hosted edge Nodes (public nodes), authenticated with the dedicated
tunnel_token.
4. Pages Static Hosting Boundaries
- Pre-built artifact sources: a project may stay manual-upload, or configure one Remote URL / public GitHub Release asset source. Remote and fixed tags only support manual ops; only GitHub latest enters scheduled checks and can opt into auto-update. Sources are switchable, but immutable deployments and the current production version don't get lost when editing or deleting a source.
- Archive and resource limits: supports
zip,tar.gz/tgz,tar.xz/txz,tar.bz2/tbz2,tar,7z. Archive cap controlled bypages_max_package_size_mb(default 100 MiB, range 1–2048); expanded single-file and total limits are 4× the package cap with a 100 MiB floor, at most 1,000 regular files. Both Server and Agent validate actual bytes and reject path traversal, symlinks/hard links, and special files. - Build and runtime boundaries: currently no source checkout or build execution from external git repos, and no edge Serverless, dynamic SSR, or preview subdomains. Future repo integration must use a separate
git_repositoryProvider with a Server-side isolated build executor, emitting only restricted artifacts into the unified artifact pipeline; the Agent never receives repo credentials, external URLs, or clone/install/build commands.
5. System and Version Boundaries
- Globally single active version: all nodes pull and consume the same globally active config. Per-node-group differentiated config release isn't performed.
- Single-tenant architecture: OpenFlare is for a single team deploying on a trusted internal network. Single-tenant by design; fine-grained multi-user roles or multi-tenant resource isolation aren't supported.
- External infra dependency: the Server must depend on external Redis (or Valkey) for distributed coordination, the Asynq queue, and system cache. The relational DB is PostgreSQL, or SQLite when
database.enabledis off. ClickHouse optional: when off, access logs and observability time series are handled by the current log primary DB (follows the business primary DB); when on, the「Switch Log Database」task can migrate to ClickHouse. Running without Redis is not supported. See Log Store Decoupling.
Repository Structure
OpenFlare has converged to a single monorepo (Go module OpenFlare). The control-plane Server and edge components (Agent, Relay, OpenFlared) share the repo, organized by Wavelet internal/apps/ domain modules.
When contributing code, strictly follow this physical layering and directory division:
| Path | Responsibility |
|---|---|
main.go |
the Server's single entry, delegating to internal/cmd/ |
cmd/agent, cmd/relay, cmd/flared |
edge component CLI entries (not the Server) |
internal/ |
control-plane and edge runtime implementations |
frontend/ |
Next.js admin panel; build artifacts embedded into the Go Server |
pkg/ |
cross-component shared libs (protocol, rendering, GeoIP, etc.) |
scripts/ |
Swagger generation, install scripts, etc. |
docs/ |
VitePress docs site and design baseline |
docker/ |
per-component Dockerfiles |
uploads/, data/ |
runtime upload dir and static data (.gitignored) |
1. Server Layering (main.go + internal/)
| Directory | Responsibility |
|---|---|
main.go |
Server startup entry |
internal/cmd/ |
Cobra subcommands: api, worker, scheduler, all (default fused mode) |
internal/platform/bootstrap/ |
cross-module assembly: task handlers, push domain events, process-level init |
internal/router/ |
HTTP route registration and global middleware |
internal/router/v1/openflare/ |
OpenFlare route registrars (register_*.go) |
internal/apps/openflare/ |
OpenFlare control-plane business domains (routers.go + logics.go) |
internal/apps/{admin,user,oauth,upload,cap,...}/ |
Wavelet platform capabilities (users, auth, tasks, push, etc.) |
internal/apps/openflare/{agent,relay,flared}/ |
Server-side edge protocol handlers (auth, heartbeat, WS) |
internal/model/ |
GORM entities / DTOs / no-IO domain rules (openflare_*.go + platform models); no DB access |
internal/infra/persistence/migrator/goose/ |
goose SQL migrations (PostgreSQL / SQLite / ClickHouse) |
internal/repository/ |
data access layer (platform + OpenFlare business CRUD, cache, logstore log IO); the only persistence entry |
internal/infra/task/ |
Asynq async tasks (Worker + Scheduler) |
internal/infra/config/ |
Viper config loading |
internal/shared/ |
unified API response wrapper (response/) |
pkg/protocol/ |
Relay / Tunnel shared HTTP/WS protocol structures |
pkg/render/, pkg/geoip/, pkg/wsclient/ |
OpenResty config rendering, GeoIP, WebSocket client |
API route prefixes:
| Prefix | Purpose | Auth |
|---|---|---|
/api/v1/d/* |
OpenFlare admin console API | Session Cookie + optional X-Access-Token |
/api/v1/agent/* |
Agent node protocol | X-Agent-Token |
/api/v1/relay/* |
Relay protocol | X-Agent-Token |
/api/v1/tunnel/* |
Tunnel client protocol | X-Tunnel-Token |
/api/v1/admin/* |
Wavelet platform admin API | admin Session |
2. Agent Modules (internal/apps/agent/ / cmd/agent/)
| internal/apps/agent/httpclient/ | Server communication |
| internal/apps/agent/wsclient/ | WebSocket client communication |
| internal/apps/agent/protocol/ | Agent API protocol types |
| internal/apps/agent/updater/ | Agent self-update logic |
| internal/apps/agent/logging/ | logging |
| internal/apps/agent/observability/| observability (metrics, traces, etc.) |
| internal/apps/agent/geoipdata/ | GeoIP data handling |
| internal/apps/agent/geoipupdate/ | GeoIP data updates |
| internal/apps/agent/agent/ | core Agent logic and lifecycle |
3. Frontend Layering (frontend/)
Based on the Wavelet Next.js scaffold, OpenFlare business UI is organized route-co-located under app/(main)/.
| Directory | Responsibility |
|---|---|
app/ |
Next.js App Router; (main) console, (auth) auth, (docs) docs pages |
app/(main)/<domain>/ |
business pages and in-domain components (route-co-located) |
components/ |
cross-domain reusable UI (ui/, layout/, common/, etc.) |
lib/services/ |
API service layer: core/ base class + openflare/ business APIs |
lib/navigation/ |
OpenFlare sidebar nav config (openflare-nav.ts) |
lib/theme/ |
theme parsing and switching |
contexts/ |
cross-page UI state (user, notifications, etc.) |
hooks/, lib/hooks/ |
reusable React Hooks |
public/ |
static assets and theme CSS |
scripts/ |
build helper scripts |
proxy.ts |
dev/prod proxy: API rate limit and page auth |
API conventions: OpenFlare business APIs uniformly prefix /api/v1/d/*, wrapped via OpenFlareBaseService; page data fetching uses @tanstack/react-query.
4. Relay Modules (internal/apps/relay/ / cmd/relay/)
| Module | Responsibility |
|---|---|
cmd/relay/ |
Relay CLI entry and init main |
internal/apps/relay/config/ |
local config parsing and default init |
internal/apps/relay/frps/ |
manage frps process lifecycle, ports & Token, monitor runtime |
internal/apps/relay/heartbeat/ |
periodic HTTP heartbeat, report state, fetch update requests |
internal/apps/relay/httpclient/ |
generic Server API client helpers |
internal/apps/relay/observability/ |
collect local host and frps base runtime metrics with pre-aggregation |
internal/apps/relay/relay/ |
coordinate core lifecycle, init, and cleanup |
internal/apps/relay/state/ |
local runtime state, error records, persistent cache |
internal/apps/relay/updater/ |
Relay upgrade check, download/install, restart |
internal/apps/relay/wsclient/ |
long-lived WebSocket bidirectional channel with the Server |
5. OpenFlared (Client) Modules (internal/apps/flared/ / cmd/flared/)
| Module | Responsibility |
|---|---|
cmd/flared/ |
Client CLI entry and init main |
internal/apps/flared/config/ |
local client config loading and parsing |
internal/apps/flared/flared/ |
intranet penetration client core scheduling and state management |
internal/apps/flared/frpc/ |
hot-reload/dynamically generate per-Relay frpc_{relayNodeID}.toml and monitor frpc |
internal/apps/flared/heartbeat/ |
heartbeat communication with the control plane, incl. Token validation |
internal/apps/flared/httpclient/ |
generic client API communication (/api/v1/tunnel/*) |
internal/apps/flared/sync/ |
incrementally pull latest Tunnel route bindings, generate snapshots, apply |
internal/apps/flared/updater/ |
client self-update, new-version check, update landing |
internal/apps/flared/wsclient/ |
WS channel for real-time Server tunnel config change push |
Note
: OpenFlared has no standalone
state/package; version and checksum are persisted byfrpc/manager.gotoflared-state.json.
Doc Maintenance Principles
- Product scope or system boundary changes: update this doc (Product Boundaries).
- Log storage, log-table judgment, or switch-protocol changes: update Log Store Decoupling.
- System structure or component division changes: update System Architecture.
- Release, sync, rollback, or Agent model changes: update Agent & Publish Model.
- Deployment method changes: update Deployment Guide and the README.
- Config item changes: update Configuration Reference.