Commit Graph

24 Commits

Author SHA1 Message Date
ryan aa4faddade 补齐 131 个 .go 文件的 SPDX license 头(repo 自带 make license 约定,早于约定新增的文件含 2 个生产文件;纯注释插入零行为影响),make license-check 转绿。go mod tidy -diff 确认干净。
Result: {"status":"keep","total_issues":8,"golint_canonicalheader":0,"golint_errname":0,"golint_errorlint":1,"golint_exhaustive":0,"golint_forcetypeassert":0,"golint_gosec":0,"golint_intrange":0,"golint_modernize":3,"golint_nilnil":3,"golint_perfsprint":0,"golint_prealloc":0,"golint_recvcheck":1,"golint_usestdlibvars":0,"golint_wastedassign":0,"golint_total":8,"golint_test_testifylint":0,"golint_test_thelper":0,"golint_test_usetesting":0,"golint_test_total":0,"golint_vetx_total":0,"eslint_problems":0,"eslint_errors":0,"eslint_warnings":0,"tsc_errors":0,"vitest_failed":0,"vitest_total":116,"measure_s":81}
2026-08-16 20:09:24 +08:00
ryan 111d2900d7 eslint 1→0:pages-source-card useEffect 补 t 依赖(next-intl 稳定引用)。modernize 补 1 处 time.Time omitzero。剩余 8 全部为刻意保留项。
Result: {"status":"keep","total_issues":8,"golint_canonicalheader":0,"golint_errname":0,"golint_errorlint":1,"golint_forcetypeassert":0,"golint_gosec":0,"golint_intrange":0,"golint_modernize":3,"golint_nilnil":3,"golint_perfsprint":0,"golint_prealloc":0,"golint_recvcheck":1,"golint_usestdlibvars":0,"golint_wastedassign":0,"golint_total":8,"eslint_problems":0,"eslint_errors":0,"eslint_warnings":0,"tsc_errors":0,"measure_s":38}
2026-08-16 18:02:58 +08:00
ryan a9e4237bbf feat(access-logs): 状态码支持手动输入,新增时间范围筛选
- 状态码筛选支持预设快捷选项 + 手动输入任意 100-599 状态码(数字校验)
- 新增时间范围筛选:shadcn 日期+时间选择器(Popover+Calendar+时分 Select),
  起止时间以 RFC3339 成对传入,后端校验格式与先后关系,非法值返回 400
- 默认显示来源 IP/访问域名/状态码,节点 ID/请求路径/时间范围折叠进「更多筛选」
2026-08-13 10:27:40 +08:00
ryan 75d1fcf345 feat(access-logs): 日志明细支持按状态码筛选并折叠次要搜索项,修复首页来源分布无数据
- 修复 PostgreSQL/SQLite 日志库下首页「来源分布」卡片无数据:RegionCounts 对空
  节点 ID 误拼 node_id = '' 恒空条件,改为空节点 ID 表示全节点聚合(对齐 CH 语义),
  并过滤空白归属地
- /access-logs?tab=list 新增状态码筛选:状态码下拉含常用 2xx/3xx/4xx/5xx 选项,
  校验 100-599,非法值返回 400;ClickHouse 与 PostgreSQL/SQLite 日志库均支持
- 搜索框折叠:默认仅显示来源 IP 与状态码,节点 ID/访问域名/请求路径折叠进
  「更多筛选」
2026-08-13 09:59:32 +08:00
ryan 7d71f1e4e1 feat(log): decouple log storage from ClickHouse with switchable logstore
- New internal/repository/logstore abstraction: exported domain interfaces
  (AccessLogStore/ObservabilityStore/UserAccessLogStore/StatusStore),
  config-driven provider (Active/Build/Migrating/SetConfigReader), GORM
  implementation for PostgreSQL/SQLite (incl. hourly rollups computed in
  real time, migration listers, PG partition maintenance), and a ClickHouse
  wrapper preserving the native batch path; repository facade delegates to
  logstore; import-lint test enforces apps never import analyticsrepo.
- ClickHouse is now optional: the log DB is either the main DB (postgres
  when database.enabled, else sqlite) or clickhouse; boot validation +
  first-run seed; log_database / log_db_migration are protected keys.
- New user task 切换日志数据库 (of_log_db_switch): freeze log writes,
  drain batch writers, copy all 6 raw log tables by id (preserving IDs)
  with target-partition pre-creation for PG, flip log_database on success,
  clear the freeze flag on failure.
- Per-store retention (log_retention_days_*) with expiry cleanup folded
  into the daily system_cleanup task; legacy database_auto_cleanup_* and
  of_database_auto_cleanup decommissioned.
- goose migrations: 6 log tables in PG (2 monthly-partitioned) + SQLite,
  retention config seeds, schedule cleanup; GET
  /api/v1/admin/status/log-database endpoint; frontend retention settings,
  switch-task UI and status badge; changelog and docs updated.

docs(plan): log database decoupling implementation plan

docs(design): log database decoupling design (ClickHouse optional)
2026-08-08 19:43:01 +08:00
ryan 943818f7d4 refactor(repository): 收敛 model/repository 分层为唯一持久化入口
将 OpenFlare 与平台业务的数据访问从 model 与 apps 直连迁入 repository,
model 仅保留实体与无 IO 规则;补充 code-check 架构守卫与开发规范。
2026-07-24 17:00:17 +08:00
ryan 20249d917c feat(rate-limits): use 3-minute trend buckets
Rate-limit analysis requests overview with bucket_minutes=3; RPS uses
count/180. Overview still defaults to 60-minute buckets.
2026-07-19 20:30:20 +08:00
ryan 72962beb0f feat(rate-limits): use 1m buckets and 24h/3d ranges
Allow overview bucket_minutes=1; rate-limit analysis uses 1-minute
buckets and replaces 7d preset with 3 days.
2026-07-19 19:20:59 +08:00
ryan b56290d79d feat(rate-limits): use 5m trend buckets and 24h/7d ranges
Overview API accepts bucket_minutes (5|60); rate-limit analysis uses
5-minute buckets and drops 15d/30d presets. Widen rank value column.
2026-07-19 19:18:15 +08:00
ryan 39473cb370 chore: prettier 2026-07-19 13:00:01 +08:00
ryan 920a530aa7 feat(access-logs): 新增 IP 明细 Tab 并完善日志详情字段
按时间窗聚合 IP 请求数/2xx 比例/入出站流量,支持排序与详情分析;
日志明细详情仅展示请求业务字段,IP 情报迁至独立详情弹窗。
2026-07-19 00:42:59 +08:00
ryan 0bb6830047 feat(access-logs): 概览支持 Zone/域名多选筛选
概览可按 Zone→Domain 层级多选域名并折叠展开;明细列表 IP 旁展示地区。
后端 overview 支持 hosts 多域名精确匹配。
2026-07-18 23:07:19 +08:00
ryan fb5a4e5b59 feat(access-logs): 上报并展示边缘缓存状态 cache_status
OpenResty 日志输出 $upstream_cache_status;Agent/协议/ClickHouse 贯通入库。
明细列表与详情按 HIT/MISS 等推导命中、回源、未缓存三态标签。
2026-07-18 22:50:46 +08:00
ryan 9aec984bee feat(access-logs): 明细详情支持 IP 分析与 URL Tab 记忆
- 新增单 IP 分析接口,趋势时间范围支持至 30 天
- 明细详情弹窗展示趋势、汇总与 Top 分布,可快捷管理 IP 组
- 访问日志页签改为 URL 参数记忆,筛选后保持当前 Tab
2026-07-18 22:40:21 +08:00
ryan bf71bc540b feat(access-logs): 优化概览饼图布局并在查询出错时增加日志记录
- 将设备类型与状态码饼图的断点由 xl 降为 lg,在大屏/笔记本视口下保持双列展示
- 修复 valueCountDistribution 在 ClickHouse 查询出错时静默吞掉错误的缺陷,引入 logger.ErrorF 捕获
- 补充 unreleased 变更日志
2026-07-18 22:02:09 +08:00
ryan e49078ac3b feat(access-logs): 接入 User-Agent 与设备/浏览器/状态码分布
- Agent 访问日志上报 user_agent,OpenResty log_format 输出 http_user_agent
- of_node_access_logs 新增 user_agent 列并写入 ClickHouse
- 访问日志概览新增:设备类型饼图、状态码饼图、浏览器/OS/User-Agent 排行
- 日志明细列表增加 User-Agent 列
- 扩展 UA 解析工具(browser/os/device)并支持 CLI 识别
2026-07-18 21:18:59 +08:00
ryan 177578ef4e feat(access-logs): 重构访问日志为概览与明细双 Tab
新增访问日志概览 API 与前端页面:汇总请求量/访问量/带宽趋势与 Top 排行,
明细列表保留检索;排行榜改为紧凑列表样式。
2026-07-18 20:58:32 +08:00
ryan f0e234df1f feat(obs): 访问日志 SSOT 与 edge_health,去掉协议兼容层
Agent 仅上报 host_metrics/edge_health/access_logs;业务流量与 UV 由
Server 侧访问日志聚合。新增 of_node_edge_health 与 of_access_log_hourly,
删除 request_reports/openresty 吞吐路径;API 不再暴露 traffic_reports
与 openresty_rx|tx。心跳/离线默认阈值与回填迁移一并入库。
2026-07-18 11:53:11 +08:00
ryan 41cd23a64d feat(api): support traffic bytes tracking in edge access logs and refactor analytics models
- Add `bytes_sent` to `NodeAccessLog` on both Agent and Master Server.
- Create ClickHouse migration `202607120001_add_bytes_sent_to_node_access_logs.sql`.
- Refactor duplicate stats structs by centralizing them into `analyticsmodel` package with type aliases.
- Simplify access log store delegations and remove redundant mapping loops.
- Regenerate Swagger documentation.
- Update changelog index.md.
2026-07-12 17:27:57 +08:00
ryan 38946d1af5 fix(clickhouse): resolve lint issues from optimization stack 2026-07-02 15:28:49 +08:00
ryan 6525bef15d perf(clickhouse): P0/P1 access log and WAF query aggregation and SQL pagination 2026-07-02 15:23:17 +08:00
ryan b6c4181c70 fix(openflare): serialize access log snowflake IDs as strings
- Return AccessLogView.id as string to avoid JS Number precision loss
- Store OpenFlareAccessLog IDs as uint64 with json id,string
- Update frontend AccessLogItem.id type to string
2026-06-19 20:10:09 +08:00
ryan 32861c5db9 fix lint 2026-06-19 17:20:39 +08:00
ryan 63cd906cfc refactor(repo): consolidate openflare-server to root and move subprojects to internal/apps
- Merge all files inside openflare-server to the repository root directory.
- Relocate agent, relay, and flared subprojects from internal/ to internal/apps/.
- Combine docker-compose files and update build context paths to root.
- Update GitHub workflows and Dockerfiles to refer to new directories and package names.
- Rewrite Go package imports across all files.
- Resolve database renew test race condition and clean up docs.
2026-06-19 14:23:29 +08:00