tcjlj/OpenFlare
1
0
Fork 0
mirror of https://github.com/Rain-kl/OpenFlare.git synced 2026-09-29 14:06:36 +08:00
Code Issues Packages Projects Releases Wiki Activity
938 Commits 3 Branches 61 Tags
7e8aa5fa0f2801ae96c6ccfb97eaab4ce4c9727e
Commit Graph

117 Commits

Author SHA1 Message Date
ryan 7e8aa5fa0f Merge branch 'codex/log-database-decoupling' 
# Conflicts:
#	docs/changelog/index.md
#	frontend/app/(main)/error-pages/page.tsx
#	internal/infra/persistence/migrator/migrator_test.go
 2026-08-09 08:37:37 +08:00
ryan 6487ce666d fix(security): harden PoW XSS, email header injection and UptimeKuma log redaction 2026-08-08 21:52:46 +08:00
ryan 9797fcdb2f fix(openflare): improve SWOfflineDomains validation and snapshot diff logic 2026-08-08 20:52:07 +08:00
ryan 7d93d3d2a1 fix(log): address remaining CodeRabbit suggestions for log database switch and migrations 2026-08-08 20:36:04 +08:00
Ryan 6faf525af0 Potential fix for pull request finding 
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
 2026-08-08 20:30:17 +08:00
ryan ca21ff3a5b feat(option): add sw offline 
fix(openresty): scope sw injection per cert partition

fix(lint): satisfy revive and biome format for sw offline feature

docs: sw offline scope changelog

fix(frontend): use scoped query key for sw scope zones

fix(frontend): hide preview link in sw contact page editor

feat(frontend): add sw scope domain picker and contact page fields

refactor(frontend): generalize html editor workspace for reuse

feat(openresty): scope sw offline injection by route domains

feat(openresty): add sw offline domains snapshot field

feat(option): add sw offline domains scope option

docs: fill html editor workspace generalization detail

docs: sw offline scope implementation plan

docs: sw offline scope design

test(openresty): assert single merged access block in sw enabled servers

fix(openresty): restrict sw intercept to https server blocks

fix(openresty): version sw offline cache by html content

fix(agent): escape redir in sw challenge page to prevent xss

fix(agent): return sw.runtime module table and add lua spec

docs: sw offline fallback changelog

fix(frontend): memoize option map to preserve unsaved contact page edits

feat(frontend): add response pages module with contact page tab

feat(agent): ship sw offline lua assets and placeholder substitution

feat(config): wire sw offline options into config snapshot

feat(openresty): render sw offline assets and challenge intercept

feat(openresty): add sw offline ConfigSnapshot fields and placeholder

feat(db): seed sw offline options

feat(option): add sw offline config keys and validation

docs: add service worker offline fallback implementation plan

docs: adopt global-option pattern for SW offline fallback (matches origin error page)

docs: unify offline contact page with error pages as response pages

docs: service worker offline fallback design (issue #23)
 2026-08-08 20:14:28 +08:00
ryan 7d71f1e4e1 feat(log): decouple log storage from ClickHouse with switchable logstore 
- New internal/repository/logstore abstraction: exported domain interfaces
  (AccessLogStore/ObservabilityStore/UserAccessLogStore/StatusStore),
  config-driven provider (Active/Build/Migrating/SetConfigReader), GORM
  implementation for PostgreSQL/SQLite (incl. hourly rollups computed in
  real time, migration listers, PG partition maintenance), and a ClickHouse
  wrapper preserving the native batch path; repository facade delegates to
  logstore; import-lint test enforces apps never import analyticsrepo.
- ClickHouse is now optional: the log DB is either the main DB (postgres
  when database.enabled, else sqlite) or clickhouse; boot validation +
  first-run seed; log_database / log_db_migration are protected keys.
- New user task 切换日志数据库 (of_log_db_switch): freeze log writes,
  drain batch writers, copy all 6 raw log tables by id (preserving IDs)
  with target-partition pre-creation for PG, flip log_database on success,
  clear the freeze flag on failure.
- Per-store retention (log_retention_days_*) with expiry cleanup folded
  into the daily system_cleanup task; legacy database_auto_cleanup_* and
  of_database_auto_cleanup decommissioned.
- goose migrations: 6 log tables in PG (2 monthly-partitioned) + SQLite,
  retention config seeds, schedule cleanup; GET
  /api/v1/admin/status/log-database endpoint; frontend retention settings,
  switch-task UI and status badge; changelog and docs updated.

docs(plan): log database decoupling implementation plan

docs(design): log database decoupling design (ClickHouse optional)
 2026-08-08 19:43:01 +08:00
ryan 6738abdec1 feat(openresty): add origin error page GET-only option 
Allow restricting custom origin error HTML to GET requests so other
methods pass through origin responses. Adds option seed, snapshot field,
edge limit_except/Lua handling, and admin UI switch.
 2026-08-06 20:22:44 +08:00
ryan 3328d3d121 refactor(agent): stop embedding GeoIP MMDB in agent binary 
Agent ships without City/Country MMDB in the binary; Docker images COPY
databases into data_dir, bare installs seed via download on first start.
Server keeps Country-only embed for optional MaxMind control-plane use.
Also harden fetch script nonempty check and reject non-file MMDB paths.
 2026-08-06 16:24:57 +08:00
ryan 7d47db1f34 feat(option): seed and validate origin error page options 2026-08-06 13:59:54 +08:00
ryan 4eced2b721 feat(cloudflare): add DNS pointing integration 2026-08-04 13:30:40 +08:00
ryan 3edcdb9e9f feat(cloudflare): add DNS pointing integration 
Implement Cloudflare connection management, pointing groups and members, asynchronous A-record reconciliation, node IP triggers, admin APIs, management pages, migrations, tests, and documentation.
 2026-08-04 12:32:37 +08:00
ryan 943818f7d4 refactor(repository): 收敛 model/repository 分层为唯一持久化入口 
将 OpenFlare 与平台业务的数据访问从 model 与 apps 直连迁入 repository，
model 仅保留实体与无 IO 规则；补充 code-check 架构守卫与开发规范。
 2026-07-24 17:00:17 +08:00
ryan d99c5b7c43 refactor(pkg): merge pkg/utils into pkg/util 
Consolidate pure helper packages under pkg/util and update imports.
 2026-07-24 15:45:10 +08:00
ryan 33a1c32cf8 refactor(structure): group platform, infra, and shared packages 
Reorganize internal packages into platform/infra/shared layers and update
imports, docs, and seed-count tests to match current system configs.
 2026-07-24 15:41:59 +08:00
ryan f28aa6520e fix(lint): 消除 linter 告警 2026-07-20 15:53:53 +08:00
ryan d58b4b6b0e feat(waf): 自动 IP 组 lookback 支持 60m/1h 时长写法 
将 lookback_minutes 替换为 lookback，移除最小 5 分钟回看限制，并兼容旧字段。
 2026-07-20 15:48:16 +08:00
ryan 866f1df5e3 fix(waf): IP 组同步间隔下限改为 1 分钟 
移除同步周期 5 分钟限制；回看窗口仍保持最小 5 分钟。
 2026-07-20 15:41:31 +08:00
ryan 351e8ce78c feat(waf): StatusRatio/StatusCount 支持 2xx/4xx/5xx 类写法 
自动 IP 组表达式可按状态码类汇总占比与计数，兼容原有精确状态码。
 2026-07-20 15:39:14 +08:00
ryan ae5345c03e feat(rate-limit): add default request rate limit configuration 
- Support openresty_default_limit_req_per_ip in system_configs.
- Add limit_req and limit_req_status 429 directive generation in openresty renderer.
- Implement route-level limit_req_per_ip override and explicit disable.
- Add frontend UI inputs and validation in rate limits tab config.
- Update swagger API docs and changelog for v3.4.3-beta.3.
 2026-07-20 10:58:13 +08:00
ryan a963b8bf54 chore(prettier): format 并清理无用 import 
make prettier 统一格式化，goimports 与 eslint unused-imports 自动移除未使用导入。
 2026-07-19 20:51:12 +08:00
ryan 4481677ef3 refactor(pages): 公开部署源任务并默认每日扫描 
移除 Pages 部署源任务的 InternalOnly 限制，任务管理可查看与调度；
将 scanner cron 与 GitHub latest 默认检查间隔调整为每天一次，
并优化部署历史列表展示。
 2026-07-19 20:41:09 +08:00
ryan 20249d917c feat(rate-limits): use 3-minute trend buckets 
Rate-limit analysis requests overview with bucket_minutes=3; RPS uses
count/180. Overview still defaults to 60-minute buckets.
 2026-07-19 20:30:20 +08:00
ryan abe8fb8268 refactor(pages): 精简部署源模型并重构详情页交互 
将 Remote 网络策略收敛为 allow_insecure，去掉脱敏与无用字段；
Pages 详情拆为部署/设置 Tab，统一卡片样式与来源信息展示。
 2026-07-19 20:23:31 +08:00
ryan c92f986978 merge: 合并 PR #22 Pages 部署源 V2 到 feat/pages-source-sync-v2 
基于最新 main 合并 deqiying/feat/pages-source-sync-v2，
解决 docs/changelog/index.md 与限流相关条目的冲突。
 2026-07-19 19:36:48 +08:00
ryan 72962beb0f feat(rate-limits): use 1m buckets and 24h/3d ranges 
Allow overview bucket_minutes=1; rate-limit analysis uses 1-minute
buckets and replaces 7d preset with 3 days.
 2026-07-19 19:20:59 +08:00
ryan b56290d79d feat(rate-limits): use 5m trend buckets and 24h/7d ranges 
Overview API accepts bucket_minutes (5|60); rate-limit analysis uses
5-minute buckets and drops 15d/30d presets. Widen rank value column.
 2026-07-19 19:18:15 +08:00
deqiying 999428cf9a feat(pages): 增加来源扫描与自动更新 
为 GitHub latest 来源增加五分钟 scanner、按来源间隔检查、精确 revision 自动发布与租约恢复。\n记录退避和投递统计，并为 PostgreSQL 与 SQLite 幂等创建内部排程。
 2026-07-19 19:09:21 +08:00
deqiying 848884d8cd fix(pages): 增加部署包孤儿补偿 
按项目、来源、运行时与上传记录锁序补偿异常中断遗留的部署包。\n同时隐藏并保护系统内部排程，避免通用任务管理入口修改 scanner。
 2026-07-19 19:08:43 +08:00
deqiying c39a3edcc3 feat(pages): 支持 GitHub Release 部署源 
增加 latest/tag 手动检查与同步、ETag 与限流退避、资源替换确认，以及对应的前端来源管理和部署来源展示。
 2026-07-19 18:31:42 +08:00
ryan c5f8105db8 feat(proxy-route): allow -1 to disable rate limits 2026-07-19 18:14:16 +08:00
ryan 2bc2d82ad0 feat(config): add openresty default rate limit system options 2026-07-19 18:12:51 +08:00
deqiying 38b0516937 feat(pages): 支持 Remote 部署源同步 
新增部署源配置与运行态模型、安全下载、租约续期、原子激活和失败补偿。

接入内部任务与脱敏前端交互，并阻止数据库 Trace 和日志展开敏感查询参数。
 2026-07-19 17:36:51 +08:00
deqiying 4e8ec23264 fix(pages): 收紧部署包与 Agent 同步边界 
完成 V2 Phase 0 安全与一致性前置：统一真实归档限额、流式拉取、候选裁剪、保留上传删除语义及 Pages 路由引用锁。
 2026-07-19 16:42:45 +08:00
ryan 39473cb370 chore: prettier 2026-07-19 13:00:01 +08:00
ryan b75f985815 feat(waf): 新增安全防护节点 security_check 
基础特征检测九项可开关；默认开启路径穿越与文件包含；命中任意规则走 false。
 2026-07-19 12:33:13 +08:00
ryan d47ceb9971 feat(waf): UA 非正常不含爬虫，并支持自定义正则屏蔽 
block_abnormal_ua 仅 Other/Unknown；新增 block_custom_ua 与 custom_ua_patterns。
 2026-07-19 11:43:45 +08:00
ryan 28eef0bbcd feat(waf): 新增 UA 检查节点 ua_check 
支持要求携带 UA、浏览器/OS 白名单 and-or 匹配，以及优先屏蔽爬虫与非正常 UA。
 2026-07-19 11:35:31 +08:00
ryan 920a530aa7 feat(access-logs): 新增 IP 明细 Tab 并完善日志详情字段 
按时间窗聚合 IP 请求数/2xx 比例/入出站流量，支持排序与详情分析；
日志明细详情仅展示请求业务字段，IP 情报迁至独立详情弹窗。
 2026-07-19 00:42:59 +08:00
ryan bfd9de69af fix(access-logs): 修复概览域名筛选参数 hosts[] 被 Gin 忽略 
Axios 默认序列化为 hosts[]=，Gin QueryArray("hosts") 读不到导致筛选失效；
后端兼容 hosts/hosts[]，前端改为重复键序列化。
 2026-07-19 00:28:37 +08:00
ryan 204f6d9a8b fix(cache): 存量空/url 策略规范为 all，避免静默收窄 
评审修复：enabled 且 policy 为空或 url 时，写入/展示/快照/渲染均映射为 all，
保证旧站点宽缓存范围不变；新建 UI 仍显式提交 static 作为推荐默认。
 2026-07-19 00:02:52 +08:00
ryan 04f029c705 feat(cache): 开启缓存默认仅缓存标准静态资源 
路由缓存策略新增 static（内置扩展名，不含 HTML）与 all；
存量 url 规范为 all。OpenResty 渲染与代理路由 UI 同步。
 2026-07-18 23:32:49 +08:00
ryan 0bb6830047 feat(access-logs): 概览支持 Zone/域名多选筛选 
概览可按 Zone→Domain 层级多选域名并折叠展开；明细列表 IP 旁展示地区。
后端 overview 支持 hosts 多域名精确匹配。
 2026-07-18 23:07:19 +08:00
ryan fb5a4e5b59 feat(access-logs): 上报并展示边缘缓存状态 cache_status 
OpenResty 日志输出 $upstream_cache_status；Agent/协议/ClickHouse 贯通入库。
明细列表与详情按 HIT/MISS 等推导命中、回源、未缓存三态标签。
 2026-07-18 22:50:46 +08:00
ryan 9aec984bee feat(access-logs): 明细详情支持 IP 分析与 URL Tab 记忆 
- 新增单 IP 分析接口，趋势时间范围支持至 30 天
- 明细详情弹窗展示趋势、汇总与 Top 分布，可快捷管理 IP 组
- 访问日志页签改为 URL 参数记忆，筛选后保持当前 Tab
 2026-07-18 22:40:21 +08:00
ryan bf71bc540b feat(access-logs): 优化概览饼图布局并在查询出错时增加日志记录 
- 将设备类型与状态码饼图的断点由 xl 降为 lg，在大屏/笔记本视口下保持双列展示
- 修复 valueCountDistribution 在 ClickHouse 查询出错时静默吞掉错误的缺陷，引入 logger.ErrorF 捕获
- 补充 unreleased 变更日志
 2026-07-18 22:02:09 +08:00
ryan e49078ac3b feat(access-logs): 接入 User-Agent 与设备/浏览器/状态码分布 
- Agent 访问日志上报 user_agent，OpenResty log_format 输出 http_user_agent
- of_node_access_logs 新增 user_agent 列并写入 ClickHouse
- 访问日志概览新增：设备类型饼图、状态码饼图、浏览器/OS/User-Agent 排行
- 日志明细列表增加 User-Agent 列
- 扩展 UA 解析工具（browser/os/device）并支持 CLI 识别
 2026-07-18 21:18:59 +08:00
ryan 177578ef4e feat(access-logs): 重构访问日志为概览与明细双 Tab 
新增访问日志概览 API 与前端页面：汇总请求量/访问量/带宽趋势与 Top 排行，
明细列表保留检索；排行榜改为紧凑列表样式。
 2026-07-18 20:58:32 +08:00
ryan 26057514a1 feat(obs): 去掉宿主机网卡趋势，磁盘读写改按速率展示 
Agent 不再采集网卡累计字节，看板与节点网络图仅保留访问日志已提供/接收。
磁盘 IO 按小时换算为 B/s 曲线，摘要为近 24 小时平均速率，并同步 Swagger。
 2026-07-18 13:17:13 +08:00
ryan 55f8c9a527 fix(obs): 对齐无兼容层与健康/UV 权威语义 
Agent 本地旧观测缓冲直接删除并运行重建；设计文档去掉兼容期表述。
健康当前态以 PG status/message 为准，CH 仅存 status 与连接时序；
Zone 曲线标明分桶 UV，顶部为整窗独立访客。
 2026-07-18 12:09:27 +08:00