feat: enhance observability metrics collection and reporting

- Updated BuildSnapshot function to include OpenResty metrics.
- Enhanced BuildTrafficReport to utilize managed OpenResty metrics.
- Introduced new functions for parsing access logs in both JSON and combined formats.
- Added tests for traffic report generation from combined access logs.
- Implemented local observability metrics collection from OpenResty.
- Created Lua scripts for OpenResty to gather observability data.
- Updated configuration documentation to include new observability port and settings.
- Added support for OpenResty observability in the server configuration.
This commit is contained in:
ryan
2026-03-14 17:07:47 +08:00
parent 8cc839669c
commit bdfa80f214
19 changed files with 800 additions and 184 deletions
+26 -17
View File
@@ -235,12 +235,13 @@ go run ./cmd/agent -config ./agent.json
{
"server_url": "http://127.0.0.1:3000",
"discovery_token": "replace-with-global-discovery-token",
"data_dir": "./data",
"openresty_container_name": "atsflare-openresty",
"openresty_docker_image": "openresty/openresty:alpine",
"heartbeat_interval": 10000,
"request_timeout": 10000
}
"data_dir": "./data",
"openresty_container_name": "atsflare-openresty",
"openresty_docker_image": "openresty/openresty:alpine",
"openresty_observability_port": 18081,
"heartbeat_interval": 10000,
"request_timeout": 10000
}
```
使用节点专属 Token 的示例:
@@ -255,11 +256,12 @@ go run ./cmd/agent -config ./agent.json
"openresty_path": "/usr/local/openresty/nginx/sbin/openresty",
"main_config_path": "/usr/local/openresty/nginx/conf/nginx.conf",
"route_config_path": "/usr/local/openresty/nginx/conf/conf.d/atsflare_routes.conf",
"cert_dir": "/usr/local/openresty/nginx/conf/certs",
"openresty_cert_dir": "/usr/local/openresty/nginx/conf/certs",
"state_path": "./data/agent-state.json",
"heartbeat_interval": 10000,
"request_timeout": 10000
"cert_dir": "/usr/local/openresty/nginx/conf/certs",
"openresty_cert_dir": "/usr/local/openresty/nginx/conf/certs",
"openresty_observability_port": 18081,
"state_path": "./data/agent-state.json",
"heartbeat_interval": 10000,
"request_timeout": 10000
}
```
@@ -274,8 +276,9 @@ go run ./cmd/agent -config ./agent.json
| `node_ip` | 节点 IP | 否 | 自动探测第一个可用 IPv4 | `192.168.1.20` |
| `openresty_path` | 本机 OpenResty 可执行文件路径;设置后按本机 OpenResty 模式运行 | 否 | 空;未设置时按 Docker OpenResty 模式处理 | `/usr/local/openresty/nginx/sbin/openresty` |
| `openresty_container_name` | Docker 模式下的 OpenResty 容器名 | 否 | `atsflare-openresty` | `atsflare-openresty` |
| `openresty_docker_image` | Docker 模式下用于初始化/管理的 OpenResty 镜像 | 否 | `openresty/openresty:alpine` | `openresty/openresty:alpine` |
| `docker_binary` | Docker 可执行文件名或路径 | 否 | `docker` | `/usr/bin/docker` |
| `openresty_docker_image` | Docker 模式下用于初始化/管理的 OpenResty 镜像 | 否 | `openresty/openresty:alpine` | `openresty/openresty:alpine` |
| `openresty_observability_port` | Agent 注入的 OpenResty 本地观测端口;用于 heartbeat 前读取 Lua 窗口指标和 `stub_status`,默认仅监听 `127.0.0.1` | 否 | `18081` | `18081` |
| `docker_binary` | Docker 可执行文件名或路径 | 否 | `docker` | `/usr/bin/docker` |
| `data_dir` | Agent 数据目录,用于存储托管配置、证书和状态文件 | 否 | 配置文件所在目录下的 `data` 子目录 | `./data` |
| `main_config_path` | 第五版主配置接管时 OpenResty 主配置文件写入路径 | 第五版本机模式建议必填 | Docker 模式可使用受管默认路径;本机模式建议显式设置 | `/usr/local/openresty/nginx/conf/nginx.conf` |
| `route_config_path` | 路由配置文件写入路径 | 否 | 默认为 `data_dir` 下托管路径 | `/etc/nginx/conf.d/atsflare_routes.conf` |
@@ -291,9 +294,10 @@ go run ./cmd/agent -config ./agent.json
* `heartbeat_interval`、`request_timeout` 支持两种写法:
* 毫秒整数,例如 `10000`
* Go duration 字符串,例如 `"30s"`
* `node_name` 与 `node_ip` 未填写时会自动探测;若自动探测失败,配置校验会报错
* 未配置 `openresty_path` 时,默认为 Docker OpenResty 模式
* 配置保存时,`agent_version`、`nginx_version` 由程序运行时维护,不需要写入 JSON
* `node_name` 与 `node_ip` 未填写时会自动探测;若自动探测失败,配置校验会报错
* 未配置 `openresty_path` 时,默认为 Docker OpenResty 模式
* `openresty_observability_port` 默认仅绑定本地回环地址;若节点本机已有端口冲突,可改为其他未占用端口
* 配置保存时,`agent_version`、`nginx_version` 由程序运行时维护,不需要写入 JSON
* 第五版主配置接管完成后,本机模式下应优先通过 `main_config_path` 由 Agent 写入受管主配置,而不是依赖节点手工维护 include 规则
### 2.4 Agent 托管路径默认值
@@ -311,7 +315,12 @@ Docker OpenResty 模式下:
| 字段 | 默认值 |
| --- | --- |
| `openresty_cert_dir` | `/etc/nginx/atsflare-certs` |
| `openresty_cert_dir` | `/etc/nginx/atsflare-certs` |
补充说明:
* Agent 当前会随受管配置一并向 OpenResty 注入 Lua 观测脚本,并在每次 heartbeat 前通过 `http://127.0.0.1:<openresty_observability_port>/atsflare/observability` 读取最近窗口请求指标
* 同一端口还会暴露仅本机可访问的 `stub_status`,用于采集 OpenResty 活动连接数
### 2.5 Agent 启动示例