diff --git a/docs/en/Guidelines.md b/docs/en/Guidelines.md new file mode 100644 index 00000000..b5275bf6 --- /dev/null +++ b/docs/en/Guidelines.md @@ -0,0 +1,188 @@ +You are a senior Go backend engineer responsible for maintaining and developing a long-evolving Go application. + +Your goal is not to "write code as quickly as possible," but to produce high-quality code that is maintainable, testable, evolvable, and conforms to Go ecosystem practices. It is forbidden to pile up temporary code, over-abstract, duplicate logic, or break the existing architecture just to complete tasks. + +Before any development, you must first read and understand the existing code structure, including: +- Project directory structure +- Entry files +- Configuration management methods +- Database/cache/message queue access methods +- HTTP/RPC/API layer design +- Layering methods such as service/usecase/domain/repository +- Error handling methods +- Logging methods +- Test organization methods +- Dependency injection methods +- Existing coding style + +If you are unsure of the responsibility of a certain module, infer it from the code context first; do not arbitrarily create duplicate modules. + +Development Principles: + +1. Architecture First +- Prioritize integrating into the existing architecture rather than starting from scratch. +- Do not arbitrarily add global variables, init side effects, or implicit dependencies. +- Do not write business logic into handlers/controllers. +- Handlers are only responsible for parameter parsing, authentication contexts, calling usecases/services, and returning responses. +- Services/usecases are responsible for business orchestration. +- Repositories/daos are responsible for data access. +- Domain/models are responsible for core business objects and rules. +- Isolate infrastructure code from business code. + +2. Go Style +- Use clear, direct, and simple Go code. +- Do not mimic Java-style over-abstraction. +- Interfaces should be defined by the consumer, not forced by the provider. +- Prioritize small interfaces. +- Naming must be accurate. Do not use vague names like Manager, Helper, or Util unless absolutely necessary. +- Keep functions short and single-responsibility. +- Do not introduce generics, reflection, or complex design patterns just to "look advanced." +- Do not hide errors. +- Errors must contain context information; use `fmt.Errorf("...: %w", err)` when necessary. +- Do not panic, except for unrecoverable errors during the program startup phase. + +3. Maintainability +- Analyze the scope of impact before making modifications. +- Keep changes minimal and avoid unrelated refactoring. +- Do not change public APIs, database structures, or configuration formats unless explicitly requested by the task. +- If changes must be made, explain the compatibility impact and migration plan. +- Confirm there are no callers before deleting code. +- Avoid copy-pasting existing logic; extract it to an appropriate place, but do not over-abstract. +- Add necessary comments to complex business logic to explain "why", not the obvious "what". + +4. Testing Requirements +- New business logic must be supplemented with unit tests. +- Bug fixes must be supplemented with regression tests. +- Tests should cover normal paths, exceptional paths, and boundary conditions. +- Do not break the structure of business code for testing convenience. +- Isolate external dependencies using mocks/fakes/stubs. +- Name tests clearly, e.g., TestXXX_WhenYYY_ShouldZZZ. +- Prioritize table-driven tests, but do not sacrifice readability for table-driven structure. + +5. Concurrency and Resource Management +- Goroutines must have exit mechanisms. +- Where context is involved, context.Context must be passed correctly. +- Do not arbitrarily use context.Background() to replace upstream contexts. +- Channels must have clear responsibility for closing. +- Keep lock scopes small to avoid deadlocks. +- Correctly close resources such as HTTP, databases, files, and connections. +- Pay attention to race conditions, goroutine leaks, and connection leaks. + +6. Database and Transactions +- Database access must be in the repository/dao layer. +- Transaction boundaries should be controlled by the business use case layer, rather than scattered across multiple lower-level functions. +- Do not produce obviously inefficient N+1 queries in loops, unless the data volume is controllable and explained. +- SQL must be readable and parameterized; unsanitized inputs are strictly forbidden from concatenation. +- Schema changes must consider migration, rollback, and compatibility. + +7. API Design +- Request parameters must be validated. +- Error responses must be stable and clear, without leaking internal sensitive information. +- Do not print sensitive data such as passwords, tokens, keys, or ID numbers in logs. +- Keep return structures backward-compatible. +- HTTP status codes must be semantically correct. + +8. Logging and Observability +- Key paths must have necessary logs. +- Error logs must contain the context needed for troubleshooting, but must not leak sensitive data. +- Do not print logs excessively. +- Do not use fmt.Println directly in library code. +- If the project already has a logger, use the existing logger uniformly. + +9. Security Requirements +- All external inputs are untrusted. +- Do not hardcode keys, tokens, or passwords. +- Do not commit sensitive configurations to the repository. +- Be mindful of injection risks in file paths, URLs, command executions, SQL, template rendering, etc. +- Authentication and permission checks must be placed in clear locations, and must not rely on the self-discipline of the frontend or callers. + +10. Performance Requirements +- Do not optimize prematurely. +- However, do not write obviously inefficient code. +- Avoid unnecessary memory allocations, large object copies, and repeated parsing on hot paths. +- Page, stream, or batch operations should be considered for large data processing. +- If caching is introduced, the consistency, expiration strategy, and invalidation conditions must be explained. + +Workflow: + +Every time you receive a development task, you must follow these steps: + +Step 1: Understand Requirements +- Briefly rephrase the requirements in your own words. +- Clarify inputs, outputs, boundary conditions, and exceptional cases. +- If requirements are vague, list your reasonable assumptions; do not write code blindly. + +Step 2: Read Existing Code +- Identify relevant modules, call chains, data structures, interfaces, and tests. +- Explain how the current code works. +- Determine which layer the modification should be placed in. + +Step 3: Design Solution +- Provide a minimal viable modification plan. +- Explain why it is placed in these files/modules. +- State whether it affects existing APIs, databases, configurations, or tests. +- If there are multiple solutions, compare their pros and cons and select the more stable one. + +Step 4: Coding +- Only modify code related to the task. +- Maintain the existing code style. +- Do not introduce unnecessary new dependencies. +- Do not create duplicate logic. +- Do not leave TODOs, temporary code, or debugging code. + +Step 5: Testing +- Supplement or update tests. +- Explain what scenarios the tests cover. +- If tests cannot be run, explain why and give the commands that should be run. + +Step 6: Delivery Explanation +- Summarize what was modified. +- Explain why it was modified this way. +- Explain potential risks. +- Provide verification methods. +- If there are incomplete items, they must be explicitly listed; do not pretend they are complete. + +Output Format: + +Each of your replies should contain: + +1. Requirements Understanding +2. Existing Code Analysis +3. Modification Plan +4. Specific Changes +5. Testing and Verification +6. Risks and Precautions + +If you are only asked to review code, output: +1. Problem List +2. Severity: Critical / High / Medium / Low +3. Impact Explanation +4. Modification Suggestions +5. Recommended Modification Example + +Code Quality Red Lines: + +The following behaviors are strictly prohibited: +- Copying and pasting large blocks of duplicate code to complete requirements. +- Stuffing business logic into handlers. +- Passing `map[string]interface{}` everywhere. +- Using global variables to bypass dependency injection. +- Arbitrarily adding util/helper trash-can packages. +- Ignoring errors. +- Catch-all style error handling. +- Continuing to stack logic when functions exceed reasonable length. +- Modifying unrelated code. +- Changing existing behavior without explanation. +- Modifying core logic without tests. +- Introducing large dependencies just to solve small problems. +- Writing code without explaining the verification method. +- Refactoring directly without understanding the existing architecture. + +When you find that the existing code is already messy: +- Do not perform a major refactoring all at once. +- Stop the bleeding locally first. +- Write new code within clear boundaries as much as possible. +- Only make necessary changes to old code. +- If refactoring is needed, propose a phased plan first. + +Please always write code to the standards of "someone who will maintain this project for a long time", rather than "someone who completes a one-time task". diff --git a/docs/en/design/architecture.md b/docs/en/design/architecture.md index 70b296a0..5dfec69c 100644 --- a/docs/en/design/architecture.md +++ b/docs/en/design/architecture.md @@ -1,33 +1,144 @@ -# Architecture +# System Architecture -OpenFlare consists of Server, Agent, and local OpenResty on each node. +You will learn: The overall architecture of OpenFlare, the responsibility boundaries of Server, Agent, OpenResty, and the management console frontend, and the request flow of a configuration release from the management console to take effect on a node. + +OpenFlare consists of the Server, the Agent, local OpenResty on each node, and the management console frontend. The Server is the control plane, the Agent is the only controlled landing entry point on the node side, and OpenResty is the actual data plane. ```text -OpenFlare Server (Gin + SQLite/PostgreSQL + Web UI) - | - | HTTP API / Config Pull - v -OpenFlare Agent (register / heartbeat / sync / apply / update) - | - v +Browser + | + | Management UI / API + v +OpenFlare Server (Gin + GORM + SQLite/PostgreSQL) + | + | Agent API / heartbeat / config pull + v +OpenFlare Agent + | + | write config / openresty -t / reload / rollback + v OpenResty binary - | - v + | + | reverse proxy + v Origin ``` +## Component Responsibilities + +| Component | Responsibility | +| --- | --- | +| Server | Management UI, admin APIs, Agent APIs, configuration rendering, version publishing, data storage, and aggregate queries | +| Agent | Registration, heartbeats, synchronization, writing files, configuration validation, reloads, fallback/rollbacks, self-updating, and lightweight data collection | +| OpenResty | Receives real traffic, executes WAF, PoW, authentication, and reverse proxying according to configurations rendered by OpenFlare | +| Frontend | Manages website configurations, WAF, origins, certificates, nodes, versions, users, settings, and observability pages | + ## Server -`openflare_server` is a monolithic control plane based on Gin, GORM, SQLite/PostgreSQL, the existing login/session system, and the static frontend build. +`openflare_server` is a monolithic control plane: -It owns the admin UI and API, Agent API, configuration rendering, version publishing, storage, and aggregate queries. +* Gin provides HTTP services. +* GORM accesses SQLite or PostgreSQL. +* The existing login system provides management console Sessions. +* Authentication source and external account binding support GitHub OAuth and standard OIDC. +* The Go Server hosts the static build output of `openflare_server/web`. + +The Server does not directly SSH into nodes, nor does it modify node files online. It only saves the control plane state, generates complete configuration versions, and lets nodes actively pull them via the Agent API. ## Agent -`openflare_agent` is a single Go binary that runs on each node. It controls OpenResty through `openresty_path`, or `openresty` by default. Docker deployments use an Agent image that already includes OpenResty and follows the same binary-control flow. +`openflare_agent` is a Go monolithic application: -It handles registration, heartbeat, sync, file writes, `openresty -t`, reload, rollback, self-update, and lightweight collection. +* Runs on nodes as a single binary. +* Reads or generates local node information upon startup. +* Performs periodic heartbeats to report status and fetch the active version summary. +* Pulls configurations, backs up old files, writes new files, validates, and reloads upon discovering a new version. +* Attempts to restore execution and roll back when the application fails. +* Maintains the WAF GeoIP mmdb; writes the built-in initial database on startup and updates it regularly based on configuration. + +The Agent uniformly executes validations, reloads, starts, and restarts via the OpenResty binary pointed to by `openresty_path`; it falls back to calling `openresty` by default when not configured. In Docker deployments, the Agent image includes the OpenResty binary and follows the same binary control logic. ## Frontend -`openflare_server/web` is the production frontend baseline: Next.js App Router, React 19, TypeScript, and Tailwind CSS. +`openflare_server/web` is the official management console frontend: + +* Next.js App Router. +* React 19. +* TypeScript. +* Tailwind CSS. +* TanStack Query manages server state. + +The frontend is hosted by the Go Server after static export. All API requests must go through `lib/api/` uniformly and handle the `success/message/data` response structure. + +## Data and Request Flow + +### Management Console Request Flow + +```text +Browser -> Frontend -> /api/* -> controller -> service -> model -> database +``` + +Mutation APIs on the management console use `POST`, while read-only APIs use `GET`. Both success and failure return a clear `message`. + +### Agent Sync Flow + +```text +Agent heartbeat -> Server returns active version summary +Agent discovers new version -> Pulls configuration details +Agent writes main configuration / route configuration / certificates / Lua resources / WAF runtime configuration +Agent executes OpenResty validation and reload +Agent reports application result +``` + +When WebSocket (WS) connection upgrade is enabled by default, the Agent first obtains settings through the HTTP heartbeat, and then attempts to connect to the Agent WebSocket. Once the WS connection is successful, periodic status reporting is carried by WS; when the Server publishes or activates a version, it broadcasts the active version summary to connected Agents, allowing them to enter the synchronization flow immediately. When the WS connection is disconnected or fails to establish, the Agent automatically falls back to the HTTP heartbeat. + +### Reverse Proxy Flow + +```text +Client -> OpenResty server block -> WAF Lua -> named upstream -> Origin +``` + +Website configuration is the aggregation boundary of reverse proxies. A website configuration can bind multiple domains and share site-level traffic limits, reverse proxies, and caching configurations. + +WAF is executed in the OpenResty `access_by_lua_file` phase. Rules come from `waf_config.json` carried in the current active version; the global rule group takes effect by default, and websites can overlay custom rule groups. + +## Core Objects + +Currently active entities include: + +* `proxy_routes` +* `origins` +* `config_versions` +* `nodes` +* `auth_sources` +* `external_accounts` +* `node_system_profiles` +* `apply_logs` +* `tls_certificates` +* `managed_domains` +* `node_request_reports` +* `node_access_logs` +* `node_metric_snapshots` +* `traffic_analytics_rollups` +* `node_health_events` +* `waf_rule_groups` +* `waf_rule_group_bindings` + +## Key Design Decisions + +| Decision | Reason | +| --- | --- | +| Complete configuration versions, instead of online patches | Gives previews, activations, history, and rollbacks stable boundaries | +| Active pull by Agents | Server does not need SSH permissions, nor does it expose remote command execution entry points | +| Global single active version | Reduces MVP complexity and ensures all nodes are consistent by default | +| Website configurations aggregate multiple domains | Supports sharing site-level policies for a business site while allowing certificate binding per domain | +| Server-side aggregation of observability data | Avoids inconsistent results caused by temporary frontend calculations | + +## Contributor Reading Suggestions + +If you want to modify architecture-related code, read these first: + +1. [Product Boundary](./index.md) +2. [Release Model](./release-model.md) +3. [Development Constraints](./development.md) +4. [Repository Structure](../reference/repository.md) diff --git a/docs/en/design/development.md b/docs/en/design/development.md index a8957322..72df909f 100644 --- a/docs/en/design/development.md +++ b/docs/en/design/development.md @@ -148,6 +148,8 @@ Currently active entities: * `traffic_analytics_rollups` * `node_health_events` * `options` +* `waf_rule_groups` +* `waf_rule_group_bindings` General constraints: @@ -161,6 +163,7 @@ General constraints: * Upstreams uniformly use named `upstream` + keepalive; for a single upstream carrying a base path or query, the original URI should be added back to `proxy_pass`. For multiple upstreams, only pure `scheme://host[:port]` is allowed. * Rate limits, reverse proxy, and cache configurations currently belong to the site-level `proxy_routes`. * HTTPS certificate binding must be saved on a per-domain basis through `domain_cert_ids` parallel to `domains`; domains not bound to a certificate must not participate in HTTPS rendering. +* WAF global rule groups are applied to all websites by default, while custom rule groups are bound to site configurations via `waf_rule_group_bindings`; they must be included in the complete configuration version snapshot during publishing. * `config_versions` must save complete snapshots and rendering results. * There can only be one activated version globally at a time. * Rollback is achieved by reactivating older versions. @@ -232,14 +235,18 @@ Version constraints: The Agent must satisfy: * Read or generate local `node_id` after startup. -* Periodic heartbeat and synchronization. +* Periodic heartbeats and synchronization. * Conventional synchronization prioritizes judging based on the version summary returned by the heartbeat. +* When WS connection upgrade is enabled and the connection is successful, the Agent can receive active version summaries via WS and immediately synchronize; WS failure or disconnection must fall back to HTTP heartbeats. * Back up old files first when discovering a new version. * Write main configurations, route configurations, and necessary certificate files. +* Write WAF/PoW runtime configurations, and ensure WAF Lua resources are managed uniformly by the Agent. * Execute `openresty -t -c ` after writing the new configuration, and then reload; direct startup of OpenResty is allowed when reload finds that it is not running. +* Periodic runtime health checks must not call `openresty -t`, preventing health probes from triggering synchronous upstream domain name resolutions; they should prioritize requesting `/openflare/stub_status` on the local `openresty_observability_port`, using HTTP `200 OK` as the basis for judging that the OpenResty main process and workers are serving. * If the activation of the new configuration fails, the Agent must first try to restore execution with the target configuration, then roll back to the old configuration and pull up OpenResty again. -* Report warning when OpenResty recovers normally after rollback; report failure when it still cannot recover after rollback. -* Once a target `version + checksum` fails to apply and rolls back, the Agent must block repeated applications of this target in its local state. +* Report warning when OpenResty recovers normally after rollback; if there is no historical main configuration to restore locally, it must be allowed to write the built-in safe fallback configuration and pull up an OpenResty runtime state that only listens to port `80` externally and uniformly returns `503 Service Unavailable` and `OpenFlare: No Valid Configuration`, while retaining the local `stub_status` health check entry. The fallback runtime state must not clear the blocked status of the failed target; the application logs must reflect that the target version failed but the fallback runtime has started. Report failure when there is a historical main configuration but it still cannot recover after rollback. +* Once a target `version + checksum` application fails and rolls back, the Agent must block repeated applications of this target in its local state. +* When the Agent maintains the local MaxMind mmdb, download or refresh failures can only record warnings, and must not block heartbeats, synchronization, configuration application, or OpenResty health checks. ## Frontend Requests, State, and Types diff --git a/docs/en/design/index.md b/docs/en/design/index.md index f8e6c02d..2814a9d6 100644 --- a/docs/en/design/index.md +++ b/docs/en/design/index.md @@ -35,6 +35,7 @@ OpenFlare is currently not positioned as a general-purpose log platform, service | Agent Synchronization | Supports registration, heartbeat, synchronization, application result reporting, and self-updating | | OpenResty Hosting | Manages main configuration templates, performance parameters, cache parameters, and Lua resources | | HTTPS/TLS | Hosts certificates and domain assets, and binds certificates on a per-domain basis | +| WAF | Maintains IP/IP ranges black/whitelists and country-level geographical black/whitelists with global and website-customized rule groups | | Basic Observability | Aggregates node requests, resource snapshots, health events, and access analytics | | Node Management | Node status, token systems, deployment, and update links | | Console Frontend | Next.js-based official management console | @@ -76,6 +77,8 @@ Currently active entities: * `node_metric_snapshots` * `traffic_analytics_rollups` * `node_health_events` +* `waf_rule_groups` +* `waf_rule_group_bindings` ## Site Configuration Constraints @@ -116,6 +119,24 @@ During publishing rendering: * Domains not bound to a certificate must not be automatically brought into HTTPS. * All domains in `proxy_routes.domains` must be included in the same site configuration to avoid the same site being split in version snapshots. +## WAF Constraints + +WAF uses rule groups as configuration boundaries. The system fixes a global rule group, which is applied to all websites by default; websites can overlay multiple custom rule groups. + +Phase 1 supports: + +* IP / IP range whitelists and blacklists. +* Country-level region whitelists and blacklists. +* Rule group-level blocking status codes and response pages, defaulting to `418` and an empty page. + +Evaluation order: + +* Whitelists are bypass exceptions; if any enabled rule group matches a whitelist, the request is allowed. +* If no whitelist is matched, blacklists continue to be evaluated. +* When multiple blacklists match, the global rule group takes precedence, followed by custom rule groups in ascending order of their IDs. + +Region recognition is based on the MaxMind mmdb maintained locally on the node by the Agent, and the OpenResty Lua reads the local database during the request path. When GeoIP dependencies are unavailable, region rules must be skipped, without affecting IP rules and the reverse proxy main link. + ## Authentication Source Constraints `auth_sources` is the configuration object for third-party login entries on the management console, currently supporting only two types: `github` and `oidc`. Enabled authentication sources will be displayed on the login page. diff --git a/docs/en/design/release-model.md b/docs/en/design/release-model.md index 1c0e603a..49ff5e88 100644 --- a/docs/en/design/release-model.md +++ b/docs/en/design/release-model.md @@ -15,13 +15,14 @@ Modify rules -> Preview / View diff -> Publish -> Generate complete configuratio When publishing, the Server must: 1. Read all enabled `proxy_routes`. -2. Read the OpenResty main configuration template, performance parameters, cache parameters, and necessary Lua resources on the Server side. +2. Read the Server side OpenResty main configuration template, performance parameters, cache parameters, and necessary Lua resources. 3. Read domain and certificate binding relationships. -4. Render the complete OpenResty configuration. -5. Calculate the `checksum`. -6. Write to `config_versions`. -7. Switch the activated version. -8. Let the Agent discover and apply it in subsequent heartbeats. +4. Read the WAF global rule group, custom rule groups, and website binding relationships. +5. Render the complete OpenResty configuration and WAF runtime configuration. +6. Calculate the `checksum`. +7. Write to `config_versions`. +8. Switch the activated version. +9. Let the Agent discover and apply it in subsequent heartbeats. The version number format is fixed as `YYYYMMDD-NNN`. @@ -33,9 +34,9 @@ Publishing generates a new complete configuration version. The version must cont ## Activating Version -There can only be one activated version globally at a time.Differentiated versions grouped by nodes are currently not supported. +There can only be one activated version globally at a time. Differentiated versions grouped by nodes are currently not supported. -The Agent obtains the activated version summary through the heartbeat; only when the remote version or checksum is inconsistent with the local state does the Agent enter the synchronization flow. +The Agent obtains the activated version summary through the heartbeat; only when the remote version or checksum is inconsistent with the local state does the Agent enter the synchronization flow. When Agent WS connection upgrade is enabled and the connection is available, the Server will broadcast the latest active version summary after successfully publishing or activating a version. Upon receiving it, the Agent immediately pulls and applies the configuration using the ordinary synchronization flow. When WS is unavailable, changes are still discovered at HTTP heartbeat intervals. ## Immutable History @@ -53,12 +54,12 @@ When discovering a new version, the Agent will: 1. Pull the details of the target version. 2. Back up old files. -3. Write the main configuration, route configurations, certificates, and necessary Lua resources. +3. Write the main configuration, route configurations, certificates, necessary Lua resources, and WAF/PoW runtime configurations. 4. Execute OpenResty configuration verification. 5. reload; if it is not started during runtime, try to start OpenResty with the current configuration. 6. Report success, warning, or failure. -If the activation of the new configuration fails, the Agent must try to restore execution; report a warning when the rollback succeeds, and report a failure when it still cannot recover after rollback. +If the activation of the new configuration fails, the Agent must try to restore execution; report a warning when the rollback succeeds. If there is no historical main configuration to roll back to locally, the Agent will write the built-in safe fallback configuration and try to pull up OpenResty: this configuration only listens to port `80` externally, contains no user routes, uniformly returns `503 Service Unavailable` and `OpenFlare: No Valid Configuration`, and retains the local `stub_status` health check entry. If fallback startup is successful, it still blocks the failed target version and reports a warning; report a failure when there is a historical main configuration but it still cannot recover after rollback. Once a target `version + checksum` application fails and rolls back, the Agent will block repeated applications of this target in its local state. Only when the remote activated version or checksum changes is it allowed to try again. @@ -69,3 +70,4 @@ Once a target `version + checksum` application fails and rolls back, the Agent w * The Agent API is fixed to use the node-exclusive `agent_token`; the first access can use the `discovery_token`. * The Server does not provide remote shell or arbitrary command execution entries. * The configuration version must save complete snapshots, rendering results, and `checksum`. +* WAF rule groups and website binding relationships must enter the snapshot and checksum along with the complete configuration version, and must not rely on the current mutable WAF configuration when rolling back. diff --git a/docs/en/guide/deployment.md b/docs/en/guide/deployment.md index ca0d4abc..dd7dde99 100644 --- a/docs/en/guide/deployment.md +++ b/docs/en/guide/deployment.md @@ -1,6 +1,6 @@ # Deployment -You will learn the recommended OpenFlare deployment model, Server and Agent requirements, source startup workflow, integration steps, upgrade paths, and uninstall entry points. +You will learn: The recommended OpenFlare deployment model, Server and Agent requirements, source startup workflow, integration steps, upgrade paths, and uninstall entry points. For production, use PostgreSQL for the Server database and set `SESSION_SECRET` explicitly. Agent controls OpenResty through the OpenResty binary; Docker deployments run the Agent image that already includes OpenResty. @@ -40,9 +40,10 @@ Agent: | --- | --- | | OS | Install script supports Linux and macOS. systemd service is created only on Linux + systemd. | | Architecture | `amd64` or `arm64` | -| OpenResty | Required for local Agent installs | +| OpenResty | Required for local Agent installs, or specified via `--openresty-path` | | Docker | Required only when running the Agent Docker image | | Network | Agent node must reach the Server URL | +| GeoIP | WAF regional rules use local MaxMind mmdb; Agent initializes a built-in library and updates it periodically | [Needs confirmation: recommended production CPU, memory, and disk size] @@ -165,6 +166,34 @@ systemctl status openflare-agent journalctl -u openflare-agent -f ``` +## Run Agent in Docker + +In Docker deployments, directly run the Agent image. This image is built on top of the OpenResty image and includes both the Agent controller and the OpenResty binary. When `node_ip` is not explicitly configured, the Agent prioritizes obtaining the real public egress IP via a third-party API, avoiding registering the Docker bridge address as the node IP. + +Mounting the configuration file: + +```bash +docker pull ghcr.io/rain-kl/openflare-agent:latest +docker rm -f openflare-agent 2>/dev/null || true +docker run -d --name openflare-agent --restart unless-stopped \ + -p 80:80 -p 443:443 \ + -v openflare-agent-data:/data \ + -v ./agent.json:/etc/openflare/agent.json:ro \ + ghcr.io/rain-kl/openflare-agent:latest +``` + +Using environment variables: + +```bash +docker pull ghcr.io/rain-kl/openflare-agent:latest +docker rm -f openflare-agent 2>/dev/null || true +docker run -d --name openflare-agent --restart unless-stopped \ + -p 80:80 -p 443:443 \ + -e OPENFLARE_SERVER_URL=http://your-server:3000 \ + -e OPENFLARE_AGENT_TOKEN=YOUR_AGENT_TOKEN \ + ghcr.io/rain-kl/openflare-agent:latest +``` + ## Run Agent Manually From source: @@ -199,6 +228,10 @@ Minimal `agent.json`: When `openresty_path` is not configured, Agent runs `openresty`. +By default, the Agent will attempt to upgrade to a WebSocket after a successful HTTP heartbeat. When the upgrade succeeds, the Server immediately notifies the Agent of any configuration publications or activations; if the WebSocket cannot be established or is unexpectedly disconnected, the Agent automatically falls back to HTTP heartbeat synchronization. + +WAF regional rules rely on the Agent's local `GeoLite2-Country.mmdb`. Upon startup, the Agent initializes a built-in database at `data_dir/etc/openflare/GeoLite2-Country.mmdb` and attempts to update it periodically based on configuration; update failures only record warnings, and do not affect configuration synchronization or OpenResty reload. + ## Minimal Integration Flow 1. Start Server and sign in. @@ -220,6 +253,7 @@ Server: Agent: * Agents follow stable releases by default. +* Agent autoupdate requires the GitHub Release to include both the target binary and a matching `.sha256` checksum file; the download must pass SHA-256 validation before the local executable is replaced. * The install script can be rerun to reinstall or upgrade. * Preview upgrades require manual action. diff --git a/docs/en/plan/Agent Unified OpenResty Binary Control Scheme.md b/docs/en/plan/Agent Unified OpenResty Binary Control Scheme.md deleted file mode 100644 index 01e85fce..00000000 --- a/docs/en/plan/Agent Unified OpenResty Binary Control Scheme.md +++ /dev/null @@ -1,84 +0,0 @@ -# Agent Unified OpenResty Binary Control Scheme - -## Summary - -Unify the Agent running model as "write to the managed configuration file, then call the `openresty` binary to execute `-t`, reload, start/restart". Docker deployment no longer has the Agent control another OpenResty container, but instead provides an independent `ghcr.io/rain-kl/openflare-agent` image; this image is based on `openresty/openresty`, with the Agent controller and OpenResty binary built-in. - -## Key Changes - -- Agent runtime: - - Remove the DockerExecutor / Docker container management logic from the production path. - - Default to using `openresty` when `openresty_path` is not configured. - - Uniformly execute binary calls with `-c ` to avoid misreading the OpenResty default configuration. - - The apply flow is: backup -> write files -> `openresty -t -c ...` -> reload; if reload indicates it is not running, then start. - - restart uses `openresty -c ... -s quit` followed by `openresty -c ...` to start, keeping fault tolerance for missing PIDs. - -- Configurations and File Responsibilities: - - Keep parser compatibility for old fields `openresty_container_name`, `openresty_docker_image`, and `docker_binary`, but mark them as deprecated and no longer involved in control logic. - - Add `access_log_path`, defaulting to `data_dir/var/log/openflare/access.log`, and no longer placing access logs inside `conf.d`. - - Add `runtime_config_dir`, defaulting to `data_dir/etc/openflare`, where `pow_config.json` is written. - - `cert_dir` only writes certificate/key files; `lua_dir` only writes Lua code and static resources. - - Support splitting files before writing: certificate files go into `cert_dir`, and `pow_config.json` goes into `runtime_config_dir`. - -- Docker Agent Image: - - Add `openflare_agent/Dockerfile`, with the runtime image based on `openresty/openresty:alpine`. - - Defaults to `OPENFLARE_OPENRESTY_PATH=openresty` and `OPENFLARE_DATA_DIR=/data`. - - Expose `80`, `443`, and `18081`. - - Support mounting `/etc/openflare/agent.json`, and also support environment variable configurations. - - CI publishes independent multi-architecture images: `ghcr.io/rain-kl/openflare-agent:` and `latest`. - -- Agent Configuration Entry: - - Keep `-config` + `agent.json`. - - Add environment variable overrides/fallbacks: `OPENFLARE_SERVER_URL`, `OPENFLARE_AGENT_TOKEN`, `OPENFLARE_DISCOVERY_TOKEN`, `OPENFLARE_NODE_NAME`, `OPENFLARE_NODE_IP`, `OPENFLARE_DATA_DIR`, `OPENFLARE_OPENRESTY_PATH`, `OPENFLARE_HEARTBEAT_INTERVAL`, `OPENFLARE_REQUEST_TIMEOUT`, `OPENFLARE_OPENRESTY_OBSERVABILITY_PORT`. - - If the configuration file does not exist but environment variables are sufficient, the Agent can start directly; if both exist, environment variables override file values. - -- Scripts and Documentation: - - `install-agent.sh` becomes a local OpenResty deployment script, adding `--openresty-path` and automatically finding `openresty` when not passed. - - `uninstall-agent.sh` only uninstalls the Agent itself, and no longer deletes the Docker OpenResty container or image. - - Update architecture, development constraints, deployment instructions, Agent guide, configuration item reference, README, and old Docker control descriptions in English image documents. - -## Public Interfaces - -- Add Agent configuration fields: - - `access_log_path` - - `runtime_config_dir` - -- Deprecated but compatibly read: - - `openresty_container_name` - - `openresty_docker_image` - - `docker_binary` - -- Add Docker image: - - `ghcr.io/rain-kl/openflare-agent` - -- Target Docker execution method examples: - - Mount configuration file: `-v ./agent.json:/etc/openflare/agent.json` - - Or environment variables: `-e OPENFLARE_SERVER_URL=... -e OPENFLARE_AGENT_TOKEN=...` - -## Test Plan - -- `openflare_agent/internal/config`: - - Default `openresty_path` is `openresty`. - - Old Docker fields can be read but do not affect the executor. - - Environment variables can start the Agent without a configuration file and can override the configuration file. - - New default paths conform to responsibility boundaries. - -- `openflare_agent/internal/nginx`: - - Binary commands all include `-c `. - - Apply success, reload failure rollback, and start fallback when not running. - - `pow_config.json` is no longer written to `cert_dir` or `lua_dir`. - - Stale `cert_dir/pow_config.json` and `lua_dir/pow_config.json` will be cleaned up. - - access log is rendered to `access_log_path`. - - checksum can still uniformly include the main config, route config, certificates, and PoW config into comparisons. - -- Integration Regression: - - `cd openflare_agent && GOCACHE=/tmp/openflare-go-cache go test ./...` - - `cd openflare_server && GOCACHE=/tmp/openflare-go-cache go test ./...` - - Dockerfile build smoke test: build the Agent image and start it using env-only configuration to the executable stage. - -## Assumptions - -- Docker Agent image name is fixed as `ghcr.io/rain-kl/openflare-agent`. -- Old Docker control fields are compatibly preserved but are no longer a supported behavior. -- This phase does not modify Server APIs, does not modify database models, and does not introduce remote command capabilities. -- The OpenResty main configuration template continues to be generated by the Server; the Agent is only responsible for local path replacement, file landing, and binary control. diff --git a/docs/en/reference/configuration.md b/docs/en/reference/configuration.md index 1c1ebb8a..0981f2d5 100644 --- a/docs/en/reference/configuration.md +++ b/docs/en/reference/configuration.md @@ -1,78 +1,262 @@ -# Configuration +# Configuration Reference + +You will learn: What configuration sources are supported by OpenFlare Server, frontend builds, and Agents, what the default values of configuration items are, and how common deployment combinations should be configured. + +This document summarizes the Server and Agent configuration items supported by OpenFlare `1.0.0`, retaining only the startup, deployment, and runtime parameters that remain valid. + +## Configuration Sources + +The Server supports three types of configuration sources: + +1. Command-line parameters. +2. Environment variables. +3. Runtime configurations in the database `Option` table. + +The Agent supports: + +1. `-config` command-line parameter. +2. `agent.json` configuration file. +3. A few log-related environment variables. + +## Configuration File Locations + +| Component | Default Location | Description | +| --- | --- | --- | +| Server SQLite | `openflare.db` | Can be modified via `SQLITE_PATH` | +| Server Uploads Directory | `upload` | Can be modified via `UPLOAD_PATH` | +| Agent Configuration File | `./agent.json` | Can be specified via `-config` | +| One-click Install Agent Config | `/opt/openflare-agent/agent.json` | Default generated by the installation script | +| Agent Data Directory | `data` under the config directory | Can be modified via `data_dir` | ## Server CLI Flags +```bash +cd openflare_server +go run . --port 3000 --log-dir ./logs +``` + | Flag | Purpose | Default | | --- | --- | --- | -| `--port` | Server listen port | `3000` | -| `--log-dir` | Log directory | empty | -| `--version` | Print version and exit | `false` | -| `--help` | Print help and exit | `false` | +| `--port` | Specify the port the Server listens on | `3000` | +| `--log-dir` | Specify the log directory | empty | +| `--version` | Print the current version and exit | `false` | +| `--help` | Print help information and exit | `false` | ## Server Environment Variables | Variable | Purpose | Default | | --- | --- | --- | | `PORT` | Server listen port | `3000` | -| `GIN_MODE` | Gin mode | release unless `debug` | +| `GIN_MODE` | Gin execution mode | `release` unless `debug` | | `LOG_LEVEL` | Log level | `info` | -| `SESSION_SECRET` | Session signing secret | random on startup | -| `SQLITE_PATH` | SQLite database path | `openflare.db` | -| `DSN` | PostgreSQL DSN, preferred over SQLite | empty | +| `SESSION_SECRET` | Session signing secret | randomly generated on startup | +| `SQLITE_PATH` | SQLite database file path | `openflare.db` | +| `DSN` | PostgreSQL DSN, preferred over SQLite when set | empty | | `SQL_DSN` | Legacy PostgreSQL DSN, lower priority than `DSN` | empty | | `REDIS_CONN_STRING` | Redis connection string | empty | | `UPLOAD_PATH` | Upload directory | `upload` | | `AGENT_TOKEN` | Legacy global Agent token | empty | -When `DSN` and `SQL_DSN` both exist, `DSN` wins. PostgreSQL is preferred when configured. If PostgreSQL is empty and a local SQLite file exists, Server migrates SQLite data at startup. +Description: + +* When both `DSN` and `SQL_DSN` exist, `DSN` takes precedence. +* When `DSN`/`SQL_DSN` and `SQLITE_PATH` exist simultaneously, PostgreSQL takes precedence. +* When the target PostgreSQL database is empty and a local SQLite file exists at `SQLITE_PATH`, the Server automatically migrates SQLite data at startup and prints table-by-table migration progress in the logs. +* `SESSION_SECRET` must be explicitly configured in production. +* When `REDIS_CONN_STRING` is not configured, related capabilities fall back to in-process implementations. + +## Runtime Options + +The following options are maintained on the settings page of the management console and can be hot-updated: + +| Option | Purpose | Default | +| --- | --- | --- | +| `AgentHeartbeatInterval` | Agent heartbeat interval (milliseconds) | `10000` | +| `AgentWebsocketUpgradeEnabled` | Whether to allow Agents to upgrade to WebSockets after successful HTTP heartbeats | `true` | +| `NodeOfflineThreshold` | Node offline threshold (milliseconds) | `120000` | +| `AgentUpdateRepo` | Agent self-update repository | `Rain-kl/OpenFlare` | +| `GeoIPProvider` | Node/IP region lookup provider | `ipinfo` | +| `DatabaseAutoCleanupEnabled` | Whether to enable daily automatic cleanup of observability data | `false` | +| `DatabaseAutoCleanupRetentionDays` | In-database retention days, at least 1 day | `30` | +| `GlobalApiRateLimitNum` / `GlobalApiRateLimitDuration` | Global API rate limit count / window | `300` / `180` | +| `GlobalWebRateLimitNum` / `GlobalWebRateLimitDuration` | Global Web rate limit count / window | `300` / `180` | +| `UploadRateLimitNum` / `UploadRateLimitDuration` | Upload API rate limit count / window | `50` / `60` | +| `DownloadRateLimitNum` / `DownloadRateLimitDuration` | Download API rate limit count / window | `50` / `60` | +| `CriticalRateLimitNum` / `CriticalRateLimitDuration` | Sensitive API rate limit count / window | `100` / `1200` | + +Description: + +* When `DatabaseAutoCleanupEnabled` is enabled, the Server automatically cleans up three types of observability data (`node_access_logs`, `node_metric_snapshots`, `node_request_reports`) at 3:00 AM every day. +* `DatabaseAutoCleanupRetentionDays` is the unified retention count and must be greater than or equal to 1. +* The management console supports leaving the retention days blank during manual cleanup to directly delete all history of the corresponding datasets. +* The GitHub Release pointed to by `AgentUpdateRepo` must provide a matching `.sha256` checksum file for each Agent binary, such as `openflare-agent-linux-amd64.sha256`; the self-update validates this SHA-256 digest before replacing the executable. +* Third-party logins no longer use `GitHubOAuthEnabled`, `GitHubClientId`, or `GitHubClientSecret` as primary configuration entries; these legacy options are only used for migration to the default GitHub authentication source during upgrades. +* Legacy options for WeChat login are retained for compatibility, but the management console no longer provides WeChat login configuration entries. +* Legacy options for Turnstile and backend verification remain, and existing configurations will continue to take effect. + +## OpenResty Parameters + +OpenResty performance and caching parameters continue to be stored uniformly in the `Option` table. Currently common items include: + +* `OpenRestyWorkerProcesses` +* `OpenRestyWorkerConnections` +* `OpenRestyWorkerRlimitNofile` +* `OpenRestyKeepaliveTimeout` +* `OpenRestyProxyConnectTimeout` +* `OpenRestyProxySendTimeout` +* `OpenRestyProxyReadTimeout` +* `OpenRestyProxyBufferingEnabled` +* `OpenRestyGzipEnabled` +* `OpenRestyCacheEnabled` +* `OpenRestyCachePath` +* `OpenRestyCacheMaxSize` + +These parameters must be validated, saved, and participate in version rendering in a structured way. + +Constraints: + +* The management console no longer exposes `resolver` configuration. +* Upstreams are uniformly rendered as named `upstream` blocks with keepalives enabled. +* A single upstream carrying a base path or query will append the original URI in `proxy_pass`. +* Multiple upstreams still require each upstream to be pure `scheme://host[:port]`, and the protocol must be consistent within the same rule. +* `OpenRestyCacheEnabled` is used to enable the caching infrastructure and global default parameters; the actual caching enablement and hit policies (based on URL, suffix, or path) are decided separately by each individual `proxy_routes`. +* The default cache key is `$scheme$host$request_uri`. +* The default `keepalive_timeout` is `20` seconds, and the default `proxy_connect_timeout` is `3` seconds. +* The default event model is `epoll`, and `multi_accept` is enabled by default. +* HTTPS listeners use the independent `http2 on;` directive by default to avoid deprecation warnings for `listen ... http2` in newer Nginx/OpenResty versions. ## Frontend Build Variables | Variable | Purpose | Default | | --- | --- | --- | -| `NEXT_PUBLIC_API_BASE_URL` | Frontend API base path | `/api` | -| `NEXT_PUBLIC_APP_VERSION` | Displayed frontend version | `dev` | -| `NEXT_DEV_BACKEND_URL` | Local dev backend proxy target | `http://127.0.0.1:3000` | +| `NEXT_PUBLIC_API_BASE_URL` | Frontend API request base path | `/api` | +| `NEXT_PUBLIC_APP_VERSION` | Frontend displayed version number | `dev` | +| `NEXT_DEV_BACKEND_URL` | Dev backend proxy target | `http://127.0.0.1:3000` | -## Runtime Options +## Agent Environment Variables -The settings page maintains these hot-updatable options: - -| Option | Purpose | Default | +| Variable | Purpose | Default | | --- | --- | --- | -| `AgentHeartbeatInterval` | Agent heartbeat interval in milliseconds | `10000` | -| `NodeOfflineThreshold` | Node offline threshold in milliseconds | `120000` | -| `AgentUpdateRepo` | Agent update repository | `Rain-kl/OpenFlare` | -| `GeoIPProvider` | Node/IP region provider | `ipinfo` | -| `DatabaseAutoCleanupEnabled` | Enable daily observability cleanup | `false` | -| `DatabaseAutoCleanupRetentionDays` | Retention days | `30` | +| `LOG_LEVEL` | Agent log level | `info` | +| `OPENFLARE_SERVER_URL` | Control plane URL, can override `agent.json` | empty | +| `OPENFLARE_AGENT_TOKEN` | Node-exclusive auth token, can override `agent.json` | empty | +| `OPENFLARE_DISCOVERY_TOKEN` | Global token for first registration, can override `agent.json` | empty | +| `OPENFLARE_NODE_NAME` | Node name, can override `agent.json` | empty | +| `OPENFLARE_NODE_IP` | Node IP, can override `agent.json` | empty | +| `OPENFLARE_DATA_DIR` | Agent data directory, can override `agent.json` | empty | +| `OPENFLARE_OPENRESTY_PATH` | OpenResty binary path, can override `agent.json` | empty | +| `OPENFLARE_HEARTBEAT_INTERVAL` | Heartbeat interval, can override `agent.json` | empty | +| `OPENFLARE_REQUEST_TIMEOUT` | Request timeout, can override `agent.json` | empty | +| `OPENFLARE_OPENRESTY_OBSERVABILITY_PORT` | Local observability port, can override `agent.json` | empty | +| `OPENFLARE_MMDB_PATH` | WAF GeoIP mmdb path, can override `agent.json` | empty | +| `OPENFLARE_MMDB_UPDATE_INTERVAL` | WAF GeoIP mmdb update interval, can override `agent.json` | empty | +| `OPENFLARE_MMDB_DOWNLOAD_URL` | WAF GeoIP mmdb download URL, can override `agent.json` | empty | -OpenResty performance and cache options are also stored in the Option table, including `OpenRestyWorkerProcesses`, `OpenRestyWorkerConnections`, `OpenRestyProxyConnectTimeout`, `OpenRestyProxyReadTimeout`, `OpenRestyCacheEnabled`, `OpenRestyCachePath`, and `OpenRestyCacheMaxSize`. +## Agent CLI Flags -`AgentUpdateRepo` releases must publish a matching `.sha256` file for each Agent binary, such as `openflare-agent-linux-amd64.sha256`. Agent self-update verifies the SHA-256 digest before replacing the executable. +| Flag | Purpose | Default | +| --- | --- | --- | +| `-config` | Specify the path to the Agent configuration file | `./agent.json` | -## Agent Configuration +## Agent Configuration Fields -Agent supports the `-config` CLI flag, an `agent.json` file, and the `LOG_LEVEL` environment variable. - -| Field | Purpose | Required | Default / behavior | +| Field | Purpose | Required | Default / Behavior | | --- | --- | --- | --- | | `server_url` | Control plane URL | yes | none | -| `agent_token` | Node-specific auth token | one of `agent_token` / `discovery_token` | empty | -| `discovery_token` | Global token for first registration | one of `agent_token` / `discovery_token` | empty | -| `node_name` | Node name | no | host name | -| `node_ip` | Node IP | no | auto-detected; Agent first queries the public egress IP through a third-party API, then falls back to local interfaces | +| `agent_token` | Node-exclusive auth token | one of `agent_token`/`discovery_token` | empty | +| `discovery_token` | Global token for first registration | one of `agent_token`/`discovery_token` | empty | +| `node_name` | Node name | no | automatically uses host name | +| `node_ip` | Node IP | no | auto-detected; prioritizes obtaining the real public egress IP via third-party APIs, falling back to local interfaces on failure | | `openresty_path` | OpenResty binary path | no | `openresty` | -| `openresty_container_name` | Deprecated Docker-control field, read for compatibility only | no | empty | -| `openresty_docker_image` | Deprecated Docker-control field, read for compatibility only | no | empty | | `openresty_observability_port` | Local observability and OpenResty health-check port | no | `18081` | -| `docker_binary` | Deprecated Docker-control field, read for compatibility only | no | empty | -| `data_dir` | Agent data directory | no | `data` under config directory | +| `data_dir` | Agent data directory | no | `data` under the config file directory | +| `main_config_path` | OpenResty main config write path | no | `data_dir/etc/nginx/nginx.conf` | +| `route_config_path` | Route config write path | no | `data_dir/etc/nginx/conf.d/openflare_routes.conf` | | `access_log_path` | OpenResty access log path | no | `data_dir/var/log/openflare/access.log` | -| `runtime_config_dir` | Runtime config directory, including `pow_config.json` | no | `data_dir/etc/openflare` | -| `heartbeat_interval` | Heartbeat interval | no | `10000` ms | -| `request_timeout` | HTTP timeout | no | `10000` ms | +| `cert_dir` | Certificate write directory | no | `data_dir/etc/nginx/certs` | +| `openresty_cert_dir` | Certificate read directory in OpenResty config | no | same as `cert_dir` | +| `lua_dir` | Lua scripts and static resources write directory | no | `data_dir/etc/nginx/lua` | +| `openresty_lua_dir` | Lua read directory in OpenResty config | no | same as `lua_dir` | +| `runtime_config_dir` | Agent runtime config write directory, e.g., `pow_config.json` | no | `data_dir/etc/openflare` | +| `mmdb_path` | WAF GeoIP mmdb file path | no | `data_dir/etc/openflare/GeoLite2-Country.mmdb` | +| `mmdb_update_interval` | WAF GeoIP mmdb update interval | no | `86400000` milliseconds | +| `mmdb_download_url` | WAF GeoIP mmdb download URL | no | built-in GeoLite2 Country download URL | +| `observability_buffer_path` | Observability buffering file path | no | `data_dir/var/lib/openflare/observability-buffer.json` | +| `observability_replay_minutes` | Minutes to automatically replay recent observability data | no | `15` | +| `state_path` | Agent local state file path | no | `data_dir/var/lib/openflare/agent-state.json` | +| `heartbeat_interval` | Heartbeat interval | no | `10000` milliseconds | +| `request_timeout` | HTTP request timeout | no | `10000` milliseconds | -`heartbeat_interval` and `request_timeout` accept milliseconds or Go duration strings. +Description: -When `node_ip` is not configured, Agent first queries `https://realip.cc` for the real public egress IP, which avoids recording a Docker bridge address in container deployments. If that lookup fails, Agent falls back to local interface detection and prefers a public IPv4 address. +* `agent_token` and `discovery_token` cannot both be empty. +* `heartbeat_interval` and `request_timeout` support integer milliseconds or Go duration strings. +* When the Server runtime option `AgentWebsocketUpgradeEnabled` is enabled, the Agent will attempt to upgrade to a WebSocket after a successful HTTP heartbeat; it automatically falls back to HTTP heartbeats when connection fails or is disconnected. +* When `openresty_path` is not configured, `openresty` is called by default. +* The Agent's periodic health checks request `http://127.0.0.1:/openflare/stub_status`, no longer judging runtime health via high-frequency `openresty -t`; validation before configuration application, startup recovery, and reloads will still execute `openresty -t -c `. +* The Agent initializes and periodically updates `mmdb_path` for OpenResty WAF Lua to execute country-level geographical rules; update failures only record warnings, and do not block sync or reloads. +* If `agent.json` does not exist but environment variables such as `OPENFLARE_SERVER_URL` and tokens are sufficient, the Agent can start directly; environment variables take precedence when both exist. +* When the Agent is not configured with `node_ip`, it first queries `https://realip.cc` for the real public egress IP, adapting to Docker/NAT scenarios; it falls back to local interface detection on failure, preferring a public IPv4 address. +* When the Agent automatically detects a private `node_ip`, the Server prioritizes retaining the public address of the Agent's direct connection during registration/heartbeat phases, avoiding misregistering internal interface addresses in NAT or multi-interface scenarios. + +## Common Configuration Combinations + +### Production Server + PostgreSQL + +```bash +export SESSION_SECRET='replace-with-a-long-random-string' +export DSN='postgres://openflare:replace-with-strong-password@postgres:5432/openflare?sslmode=disable' +export GIN_MODE='release' +export LOG_LEVEL='info' +``` + +### Local Server + SQLite + +```bash +export SESSION_SECRET='dev-session-secret' +export SQLITE_PATH='./openflare-dev.db' +export LOG_LEVEL='debug' +go run . +``` + +### Agent + Default OpenResty + +```json +{ + "server_url": "http://your-server:3000", + "agent_token": "replace-with-node-auth-token", + "data_dir": "/opt/openflare-agent/data", + "openresty_path": "openresty", + "heartbeat_interval": 10000, + "request_timeout": 10000 +} +``` + +### Agent + Customized OpenResty Path + +```json +{ + "server_url": "http://your-server:3000", + "agent_token": "replace-with-node-auth-token", + "data_dir": "/var/lib/openflare-agent", + "openresty_path": "/usr/local/openresty/nginx/sbin/openresty", + "main_config_path": "/var/lib/openflare-agent/etc/nginx/nginx.conf", + "route_config_path": "/var/lib/openflare-agent/etc/nginx/conf.d/openflare_routes.conf", + "access_log_path": "/var/lib/openflare-agent/var/log/openflare/access.log", + "cert_dir": "/var/lib/openflare-agent/etc/nginx/certs", + "lua_dir": "/var/lib/openflare-agent/etc/nginx/lua", + "runtime_config_dir": "/var/lib/openflare-agent/etc/openflare", + "heartbeat_interval": 10000, + "request_timeout": 10000 +} +``` + +## Maintenance Requirements + +When the following contents change, this document must be updated in sync: + +* Server command-line parameters. +* Server environment variables. +* Agent command-line parameters. +* Agent configuration fields. +* Default values, purposes, or examples of any configuration items. diff --git a/docs/plan/Agent Unified OpenResty Binary Control Scheme.md b/docs/plan/Agent Unified OpenResty Binary Control Scheme.md deleted file mode 100644 index d1d17aee..00000000 --- a/docs/plan/Agent Unified OpenResty Binary Control Scheme.md +++ /dev/null @@ -1,86 +0,0 @@ -# Agent Unified OpenResty Binary Control Scheme - -# Agent 统一 OpenResty 二进制控制方案 - -## Summary - -将 Agent 运行模型统一为“写入受管配置文件,然后调用 `openresty` 二进制执行 `-t`、reload、start/restart”。Docker 部署不再由 Agent 控制另一个 OpenResty 容器,而是提供独立的 `ghcr.io/rain-kl/openflare-agent` 镜像;该镜像基于 `openresty/openresty`,内置 Agent 控制器和 OpenResty 二进制。 - -## Key Changes - -- Agent runtime: - - 移除生产路径中的 DockerExecutor / Docker 容器管理逻辑。 - - `openresty_path` 未配置时默认使用 `openresty`。 - - 二进制执行统一带 `-c `,避免误读 OpenResty 默认配置。 - - apply 流程为:备份 -> 写入文件 -> `openresty -t -c ...` -> reload;若 reload 表明未运行,则 start。 - - restart 使用 `openresty -c ... -s quit` 后再 `openresty -c ...` 启动,保留缺失 PID 的容错。 - -- 配置与文件职责: - - 保留旧字段 `openresty_container_name`、`openresty_docker_image`、`docker_binary` 的解析兼容,但标记废弃且不再参与控制逻辑。 - - 新增 `access_log_path`,默认 `data_dir/var/log/openflare/access.log`,不再把访问日志放进 `conf.d`。 - - 新增 `runtime_config_dir`,默认 `data_dir/etc/openflare`,`pow_config.json` 写入这里。 - - `cert_dir` 只写证书/密钥文件;`lua_dir` 只写 Lua 代码与静态资源。 - - 支持文件写入前先拆分:证书文件进入 `cert_dir`,`pow_config.json` 进入 `runtime_config_dir`。 - -- Docker Agent 镜像: - - 新增 `openflare_agent/Dockerfile`,运行镜像基于 `openresty/openresty:alpine`。 - - 默认 `OPENFLARE_OPENRESTY_PATH=openresty`、`OPENFLARE_DATA_DIR=/data`。 - - 暴露 `80`、`443`、`18081`。 - - 支持挂载 `/etc/openflare/agent.json`,也支持环境变量配置。 - - CI 发布独立多架构镜像:`ghcr.io/rain-kl/openflare-agent:` 和 `latest`。 - -- Agent 配置入口: - - 保留 `-config` + `agent.json`。 - - 新增环境变量覆盖/兜底:`OPENFLARE_SERVER_URL`、`OPENFLARE_AGENT_TOKEN`、`OPENFLARE_DISCOVERY_TOKEN`、`OPENFLARE_NODE_NAME`、`OPENFLARE_NODE_IP`、`OPENFLARE_DATA_DIR`、`OPENFLARE_OPENRESTY_PATH`、`OPENFLARE_HEARTBEAT_INTERVAL`、`OPENFLARE_REQUEST_TIMEOUT`、`OPENFLARE_OPENRESTY_OBSERVABILITY_PORT`。 - - 若配置文件不存在但环境变量足够,Agent 可直接启动;若两者都存在,环境变量覆盖文件值。 - -- 脚本与文档: - - `install-agent.sh` 转为本地 OpenResty 部署脚本,增加 `--openresty-path`,未传时自动查找 `openresty`。 - - `uninstall-agent.sh` 只卸载 Agent 本身,不再删除 Docker OpenResty 容器或镜像。 - - 更新架构、开发约束、部署说明、Agent 指南、配置项参考、README,以及英文镜像文档中的旧 Docker 控制说明。 - -## Public Interfaces - -- 新增 Agent 配置字段: - - `access_log_path` - - `runtime_config_dir` - -- 废弃但兼容读取: - - `openresty_container_name` - - `openresty_docker_image` - - `docker_binary` - -- 新增 Docker 镜像: - - `ghcr.io/rain-kl/openflare-agent` - -- Docker 运行方式示例目标: - - 挂载配置文件:`-v ./agent.json:/etc/openflare/agent.json` - - 或环境变量:`-e OPENFLARE_SERVER_URL=... -e OPENFLARE_AGENT_TOKEN=...` - -## Test Plan - -- `openflare_agent/internal/config`: - - 默认 `openresty_path` 为 `openresty`。 - - 旧 Docker 字段可读取但不影响 executor。 - - 环境变量可在无配置文件时启动,并可覆盖配置文件。 - - 新默认路径符合职责边界。 - -- `openflare_agent/internal/nginx`: - - 二进制命令都包含 `-c `。 - - apply 成功、reload 失败后回滚、未运行时 start fallback。 - - `pow_config.json` 不再写入 `cert_dir` 或 `lua_dir`。 - - stale `cert_dir/pow_config.json` 与 `lua_dir/pow_config.json` 会被清理。 - - access log 渲染到 `access_log_path`。 - - checksum 仍能把主配置、路由配置、证书和 PoW 配置统一纳入比较。 - -- 集成回归: - - `cd openflare_agent && GOCACHE=/tmp/openflare-go-cache go test ./...` - - `cd openflare_server && GOCACHE=/tmp/openflare-go-cache go test ./...` - - Dockerfile 构建 smoke test:构建 Agent 镜像并用 env-only 配置启动到可执行阶段。 - -## Assumptions - -- Docker Agent 镜像名固定为 `ghcr.io/rain-kl/openflare-agent`。 -- 旧 Docker 控制字段保留兼容,但不再作为受支持行为。 -- 本次不改 Server API、不改数据库模型、不引入远程命令能力。 -- OpenResty 主配置模板继续由 Server 生成;Agent 只负责本地路径替换、文件落盘和二进制控制。 diff --git a/docs/reference/configuration.md b/docs/reference/configuration.md index 22ab8b57..cab61749 100644 --- a/docs/reference/configuration.md +++ b/docs/reference/configuration.md @@ -168,10 +168,7 @@ OpenResty 性能参数与缓存参数继续统一保存在 `Option` 表。当前 | `node_name` | 节点名称 | 否 | 自动使用主机名 | | `node_ip` | 节点 IP | 否 | 自动探测,优先通过第三方 API 获取真实出口公网 IP;失败时退回本机网卡探测 | | `openresty_path` | OpenResty 二进制路径 | 否 | `openresty` | -| `openresty_container_name` | 旧 Docker 控制字段,仅兼容读取 | 否 | 空 | -| `openresty_docker_image` | 旧 Docker 控制字段,仅兼容读取 | 否 | 空 | | `openresty_observability_port` | 本地观测与 OpenResty 健康检查端口 | 否 | `18081` | -| `docker_binary` | 旧 Docker 控制字段,仅兼容读取 | 否 | 空 | | `data_dir` | Agent 数据目录 | 否 | 配置文件所在目录下的 `data` | | `main_config_path` | OpenResty 主配置写入路径 | 否 | `data_dir/etc/nginx/nginx.conf` | | `route_config_path` | 路由配置写入路径 | 否 | `data_dir/etc/nginx/conf.d/openflare_routes.conf` |