Compare commits
16
Commits
7cb90022f4
..
main
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
21a1c0e677 | ||
|
|
cfdc479206 | ||
|
|
3651d4e3fe | ||
|
|
3d82e6d891 | ||
|
|
1c4eacb48a | ||
|
|
1dc134d0e7 | ||
|
|
76d21704c8 | ||
|
|
af5eedef72 | ||
|
|
8a14d61810 | ||
|
|
420848f675 | ||
|
|
b53d89b228 | ||
|
|
f1de391e7f | ||
|
|
f45b51e9ff | ||
|
|
dd44e59a54 | ||
|
|
59b680afdd | ||
|
|
902dc36579 |
@@ -1,6 +1,7 @@
|
|||||||
.DS_Store
|
.DS_Store
|
||||||
.env
|
.env
|
||||||
.env.*
|
.env.*
|
||||||
|
credentials.env
|
||||||
*.db
|
*.db
|
||||||
*.sqlite
|
*.sqlite
|
||||||
*.sqlite3
|
*.sqlite3
|
||||||
|
|||||||
@@ -1,78 +1,45 @@
|
|||||||
# Click Gap 数据看板
|
https://ops.roasbloom.site/af-click-gap/
|
||||||
|
|
||||||
运行目标是 **mac2**。另一个独立的外部 AppsFlyer 任务每天把汇总结果写入 SQLite;本项目不调用该任务、不访问 AppsFlyer、Superset 或其它提取 API,只从固定数据库只读查询并发布静态 JSON。
|
# af-click-gap
|
||||||
|
|
||||||
固定数据库:`/Users/ads-center/Documents/database/af_click_gap.db`。数据库必须预先存在 `click_gap` 表,脚本不建表、不迁移、不写入数据库。表要求包含:
|
生成 AppsFlyer 点击差异看板的数据。上游 AppsFlyer 脚本负责采集并写入 SQLite;本项目流水线只读数据库,生成 JSON 并发布到 Gitea。
|
||||||
|
|
||||||
```text
|
## 数据与输出
|
||||||
pid TEXT NOT NULL, dt TEXT NOT NULL, attr_platform TEXT NOT NULL,
|
|
||||||
pkg_name TEXT NOT NULL, shareit_clicks REAL NOT NULL, af_clicks REAL NOT NULL,
|
|
||||||
installs_ua REAL NOT NULL, clicks_diff REAL NOT NULL,
|
|
||||||
clicks_diff_rate REAL, updated_at TEXT NOT NULL
|
|
||||||
```
|
|
||||||
|
|
||||||
主键必须为 `(pid, dt, pkg_name)`。脚本只读取 `pid='shareit'`,并校验字段、日期、数值、重复业务键和有限数值。
|
- 数据库:`/Users/ads-center/Documents/database/af_click_gap.db`。
|
||||||
|
- 默认生成最近 7 天(含当天)的 `data/YYYYMMDD.json`;使用 `--all` 时生成数据库中所有实际有数据日期的文件。
|
||||||
## 输出协议
|
- `data/index.json` 保存生成时间、版本和 JSON 文件清单。
|
||||||
|
- 每个日期文件是 JSON 数组,记录包含 `dt`、`attr_platform`、`pkg_name`、`shareit_clicks`、`af_clicks`、`installs_ua`、`clicks_diff`、`clicks_diff_rate`。
|
||||||
- `data/YYYYMMDD.json`:直接 JSON 数组;默认最近 7 天每天生成一个文件,即使当天无数据也生成 `[]`。使用 `--all` 时只为数据库中实际有数据的日期生成文件。
|
- SQLite 以只读方式打开;不会建表、迁移或写入数据库。
|
||||||
- `data/index.json`:包含 `schema_version`、`generated_at` 和本次实际生成的 `data/YYYYMMDD.json` 清单。
|
|
||||||
- 每行字段为 `dt`(`YYYYMMDD`)、`attr_platform`、`pkg_name`、`shareit_clicks`、`af_clicks`、`installs_ua`、`clicks_diff`、`clicks_diff_rate`;最后一项可以为 `null`。
|
|
||||||
|
|
||||||
默认日期边界为运行日往前 6 天至运行日,首尾包含,共 7 天。手工补数使用 `--start-date` 和 `--end-date`,两者必须同时提供且最多 7 天。
|
|
||||||
|
|
||||||
## 手工运行
|
|
||||||
|
|
||||||
mac2 推荐始终通过包装脚本运行:
|
|
||||||
|
|
||||||
```sh
|
|
||||||
cd /Users/shareit./Library/CloudStorage/OneDrive-个人/ubuntu/Gitea/af-click-gap
|
|
||||||
scripts/run_pipeline.sh
|
|
||||||
scripts/run_pipeline.sh --start-date 2026-10-10 --end-date 2026-10-16
|
|
||||||
# 全量重建数据库中所有实际有数据的日期并 push
|
|
||||||
scripts/run_pipeline.sh --all
|
|
||||||
```
|
|
||||||
|
|
||||||
包装脚本默认定位项目根目录,并读取 sibling 项目 `../direct-report-dashboard/.env`(mac2 默认路径为 `~/ubuntu/Gitea/af-click-gap` 与 `~/ubuntu/Gitea/direct-report-dashboard`)。也可以用 `PROJECT_ROOT` 和 `ENV_FILE` 覆盖路径。它只解析 `AGENTSTAGE_USER`、`AGENTSTAGE_TOKEN`,不执行整个 `.env`;凭据只进入子进程环境,不写入文件、remote URL、命令行参数或日志。脚本使用临时 `GIT_ASKPASS` 响应 Git 的用户名和密码提示,临时文件权限为仅当前用户可读写,并在退出时清理。
|
|
||||||
|
|
||||||
不要直接运行 `python3 scripts/af_click_gap_pipeline.py --push`:该命令可能使用 mac2 Keychain 中没有仓库写权限的旧凭据。包装脚本会执行相同的 SQLite 最近 7 天提取和 `--push`,额外参数会继续传给流水线;因此 `scripts/run_pipeline.sh --all` 最终执行全量 `--all --push`。需要只检查数据时可直接使用 `python3 scripts/af_click_gap_pipeline.py --dry-run` 或 `python3 scripts/af_click_gap_pipeline.py --all --dry-run`。
|
|
||||||
|
|
||||||
`--all` 会读取数据库中 `pid='shareit'` 的全部记录并按实际出现的日期生成 JSON;它不能与日期范围参数同时使用。全量运行可能改动并提交大量历史数据,执行前应确认数据库内容和工作区,且仍会经过字段、日期、数值、JSON 和 Git 数据白名单校验。脚本对数据库使用 SQLite 只读连接,不建表、不迁移、不写入数据库。
|
|
||||||
|
|
||||||
`--dry-run` 只读取数据库并在仓库外临时目录生成、校验 JSON,不写 `data` 或 Git。正式运行先原子替换 `data/` 输出目录中的数据文件,再执行 Git;数据库查询或校验失败不会发布文件。Git 安全检查只允许本次生成的 `data/*.json`(含 `data/index.json`),工作区存在其它未提交改动时拒绝继续。push 失败会保留本地 commit,不会 force push。
|
|
||||||
|
|
||||||
## 看板
|
## 看板
|
||||||
|
|
||||||
`index.html` 是无构建依赖的静态页面,读取并校验 `data/index.json` 及其清单,支持日期范围、包名筛选和表头排序。直接打开文件或执行 `python3 -m http.server` 后访问即可。
|
`index.html` 是无构建依赖的静态页面,读取 `data/index.json` 和日期 JSON。开始日期、结束日期默认都为昨天,支持日期筛选、`pkg_name` 多选、表头排序、每页 200 行和下载原始数据。
|
||||||
|
|
||||||
## mac2 crontab
|
## mac2 任务
|
||||||
|
|
||||||
mac2 使用 crontab 定时调用 `scripts/run_pipeline.sh`。包装脚本会读取 sibling 项目 `.env` 中的凭据,因此 cron 环境不需要配置密钥。先确认手工执行成功,再编辑当前用户的 crontab:
|
两个任务按以下顺序执行:先由上游 AppsFlyer 脚本写数据库,再运行本项目流水线读取数据库并生成、推送 JSON:
|
||||||
|
|
||||||
```sh
|
```sh
|
||||||
cd /Users/shareit./Library/CloudStorage/OneDrive-个人/ubuntu/Gitea/af-click-gap
|
/Users/ads-center/py314env/bin/python /Users/ads-center/ubuntu/appsflyer/af_click_gap.py
|
||||||
scripts/run_pipeline.sh
|
/Users/ads-center/ubuntu/Gitea/af-click-gap/scripts/run_pipeline.sh
|
||||||
crontab -e
|
|
||||||
```
|
```
|
||||||
|
|
||||||
加入以下示例(每天 mac2 本地时间 06:10 运行;请按实际 Python 路径和日志位置调整):
|
流水线脚本从用户目录的 `~/.config/gitea/credentials.env` 读取共享 Gitea 凭证(变量名为 `GITEA_USER` 和 `GITEA_TOKEN`),文件权限必须为 `600`,并要求账号为 `liuji`。凭据只通过临时 `GIT_ASKPASS` 提供给 Git,不写入 remote URL、命令行、日志或仓库。不要把该文件提交到仓库。业务配置仍可使用项目自己的 `.env`。
|
||||||
|
|
||||||
```cron
|
## 手动运行
|
||||||
10 6 * * * cd /Users/shareit./Library/CloudStorage/OneDrive-个人/ubuntu/Gitea/af-click-gap && /usr/bin/env PYTHON_BIN=/usr/bin/python3 ./scripts/run_pipeline.sh >> "$HOME/Library/Logs/af-click-gap/cron.log" 2>&1
|
|
||||||
```
|
|
||||||
|
|
||||||
创建日志目录:`mkdir -p "$HOME/Library/Logs/af-click-gap"`。数据库、`.env`、密钥、日志和缓存均不会提交;不再使用或保留系统级 plist 配置。
|
|
||||||
|
|
||||||
## Git 创建和推送
|
|
||||||
|
|
||||||
远程仓库约定为 Gitea `https://git.ops.roasbloom.site/liuji/af-click-gap.git`。仓库创建和首次配置可使用已有流程;日常 mac2 发布必须使用 `scripts/run_pipeline.sh`,它从 `/Users/shareit./Library/CloudStorage/OneDrive-个人/ubuntu/Gitea/direct-report-dashboard/.env`(或 `ENV_FILE` 指定路径)读取 `AGENTSTAGE_USER` 和 `AGENTSTAGE_TOKEN`。凭据不进入 Git、remote URL、命令行、临时 askpass 文件或日志,且 `.env` 绝不提交。
|
|
||||||
|
|
||||||
## 验证和安全
|
|
||||||
|
|
||||||
```sh
|
```sh
|
||||||
python3 -m py_compile scripts/af_click_gap_pipeline.py
|
cd /Users/ads-center/ubuntu/Gitea/af-click-gap
|
||||||
python3 scripts/af_click_gap_pipeline.py --help
|
./scripts/run_pipeline.sh # 最近 7 天并推送
|
||||||
python3 -m json.tool data/index.json
|
./scripts/run_pipeline.sh --start-date 2026-10-10 --end-date 2026-10-16
|
||||||
|
./scripts/run_pipeline.sh --all # 全量日期并推送
|
||||||
|
python3 scripts/af_click_gap_pipeline.py --dry-run
|
||||||
|
python3 scripts/af_click_gap_pipeline.py --all --dry-run
|
||||||
```
|
```
|
||||||
|
|
||||||
禁止提交数据库文件、`.env*`、密钥、日志、浏览器状态、缓存或临时文件;`.gitignore` 已覆盖这些类型。
|
`--all` 不能与日期范围同时使用;全量运行可能提交大量历史 JSON。`--dry-run` 只检查数据并在临时目录生成 JSON,不修改 `data/` 或 Git。
|
||||||
|
|
||||||
|
## 安全
|
||||||
|
|
||||||
|
不要提交 `af_click_gap.db`、数据库、`.env`、token、密钥、日志、浏览器状态或缓存。发布前确认工作区没有无关改动;不再使用 launchd 目录或 plist 配置。
|
||||||
|
|||||||
Executable
+1
File diff suppressed because one or more lines are too long
Executable
+1
File diff suppressed because one or more lines are too long
Executable
+1
File diff suppressed because one or more lines are too long
Executable
+1
File diff suppressed because one or more lines are too long
Executable
+1
File diff suppressed because one or more lines are too long
Executable
+1
File diff suppressed because one or more lines are too long
Executable
+1
File diff suppressed because one or more lines are too long
Executable
+1
File diff suppressed because one or more lines are too long
Executable
+1
File diff suppressed because one or more lines are too long
Executable
+1
File diff suppressed because one or more lines are too long
Executable
+1
File diff suppressed because one or more lines are too long
Executable
+1
File diff suppressed because one or more lines are too long
Executable
+1
File diff suppressed because one or more lines are too long
Executable
+1
File diff suppressed because one or more lines are too long
Executable
+1
File diff suppressed because one or more lines are too long
Executable
+1
File diff suppressed because one or more lines are too long
Executable
+1
File diff suppressed because one or more lines are too long
Executable
+1
File diff suppressed because one or more lines are too long
Executable
+1
File diff suppressed because one or more lines are too long
Executable
+1
File diff suppressed because one or more lines are too long
Executable
+1
File diff suppressed because one or more lines are too long
Executable
+1
File diff suppressed because one or more lines are too long
Executable
+1
File diff suppressed because one or more lines are too long
Executable
+1
File diff suppressed because one or more lines are too long
Executable
+1
File diff suppressed because one or more lines are too long
Executable
+1
File diff suppressed because one or more lines are too long
Executable
+1
File diff suppressed because one or more lines are too long
Executable
+1
File diff suppressed because one or more lines are too long
Executable
+1
File diff suppressed because one or more lines are too long
Executable
+1
File diff suppressed because one or more lines are too long
Executable
+1
File diff suppressed because one or more lines are too long
Executable
+1
File diff suppressed because one or more lines are too long
Executable
+1
File diff suppressed because one or more lines are too long
Executable
+1
File diff suppressed because one or more lines are too long
Executable
+1
File diff suppressed because one or more lines are too long
Executable
+1
File diff suppressed because one or more lines are too long
Executable
+1
File diff suppressed because one or more lines are too long
Executable
+1
File diff suppressed because one or more lines are too long
Executable
+1
File diff suppressed because one or more lines are too long
Executable
+1
File diff suppressed because one or more lines are too long
Executable
+1
File diff suppressed because one or more lines are too long
Executable
+1
File diff suppressed because one or more lines are too long
Executable
+1
File diff suppressed because one or more lines are too long
Executable
+1
File diff suppressed because one or more lines are too long
Executable
+1
File diff suppressed because one or more lines are too long
Executable
+1
File diff suppressed because one or more lines are too long
Executable
+1
File diff suppressed because one or more lines are too long
Executable
+1
File diff suppressed because one or more lines are too long
Executable
+1
File diff suppressed because one or more lines are too long
Executable
+1
File diff suppressed because one or more lines are too long
Executable
+1
File diff suppressed because one or more lines are too long
Executable
+1
File diff suppressed because one or more lines are too long
Executable
+1
File diff suppressed because one or more lines are too long
Executable
+1
File diff suppressed because one or more lines are too long
Executable
+1
File diff suppressed because one or more lines are too long
Executable
+1
File diff suppressed because one or more lines are too long
Executable
+1
File diff suppressed because one or more lines are too long
Executable
+1
File diff suppressed because one or more lines are too long
Executable
+1
File diff suppressed because one or more lines are too long
Executable
+1
File diff suppressed because one or more lines are too long
Executable
+1
File diff suppressed because one or more lines are too long
Executable
+1
File diff suppressed because one or more lines are too long
Executable
+1
File diff suppressed because one or more lines are too long
Executable
+1
File diff suppressed because one or more lines are too long
Executable
+1
File diff suppressed because one or more lines are too long
Executable
+1
File diff suppressed because one or more lines are too long
Executable
+1
File diff suppressed because one or more lines are too long
Executable
+1
File diff suppressed because one or more lines are too long
Executable
+1
File diff suppressed because one or more lines are too long
Executable
+1
File diff suppressed because one or more lines are too long
Regular → Executable
+1
-1
File diff suppressed because one or more lines are too long
Regular → Executable
+1
-1
File diff suppressed because one or more lines are too long
Regular → Executable
+1
-1
File diff suppressed because one or more lines are too long
Regular → Executable
+1
-1
File diff suppressed because one or more lines are too long
+1
-1
File diff suppressed because one or more lines are too long
+1
-1
File diff suppressed because one or more lines are too long
+1
-1
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
@@ -0,0 +1 @@
|
|||||||
|
[]
|
||||||
+17
-17
@@ -1,7 +1,23 @@
|
|||||||
{
|
{
|
||||||
"schema_version": 1,
|
"schema_version": 1,
|
||||||
"generated_at": "2026-10-16T16:02:20+08:00",
|
"generated_at": "2026-10-20T18:08:43+08:00",
|
||||||
"files": [
|
"files": [
|
||||||
|
{
|
||||||
|
"date": "20261020",
|
||||||
|
"path": "data/20261020.json"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"date": "20261019",
|
||||||
|
"path": "data/20261019.json"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"date": "20261018",
|
||||||
|
"path": "data/20261018.json"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"date": "20261017",
|
||||||
|
"path": "data/20261017.json"
|
||||||
|
},
|
||||||
{
|
{
|
||||||
"date": "20261016",
|
"date": "20261016",
|
||||||
"path": "data/20261016.json"
|
"path": "data/20261016.json"
|
||||||
@@ -13,22 +29,6 @@
|
|||||||
{
|
{
|
||||||
"date": "20261014",
|
"date": "20261014",
|
||||||
"path": "data/20261014.json"
|
"path": "data/20261014.json"
|
||||||
},
|
|
||||||
{
|
|
||||||
"date": "20261013",
|
|
||||||
"path": "data/20261013.json"
|
|
||||||
},
|
|
||||||
{
|
|
||||||
"date": "20261012",
|
|
||||||
"path": "data/20261012.json"
|
|
||||||
},
|
|
||||||
{
|
|
||||||
"date": "20261011",
|
|
||||||
"path": "data/20261011.json"
|
|
||||||
},
|
|
||||||
{
|
|
||||||
"date": "20261010",
|
|
||||||
"path": "data/20261010.json"
|
|
||||||
}
|
}
|
||||||
]
|
]
|
||||||
}
|
}
|
||||||
|
|||||||
+8
-4
@@ -2,6 +2,7 @@
|
|||||||
<html lang="zh-CN">
|
<html lang="zh-CN">
|
||||||
<head>
|
<head>
|
||||||
<meta charset="utf-8"><meta name="viewport" content="width=device-width, initial-scale=1"><title>Click Gap 数据看板</title>
|
<meta charset="utf-8"><meta name="viewport" content="width=device-width, initial-scale=1"><title>Click Gap 数据看板</title>
|
||||||
|
<link rel="icon" type="image/png" href="icon.png">
|
||||||
<style>
|
<style>
|
||||||
body{font:14px system-ui,sans-serif;margin:2rem;color:#223}label{margin-right:1rem}input,button{padding:.35rem}button{cursor:pointer}table{border-collapse:collapse;width:100%;margin-top:1rem}th,td{border:1px solid #ccd;padding:.45rem;text-align:right}th{cursor:pointer;background:#eef}th:first-child,td:first-child,th:nth-child(2),td:nth-child(2),th:nth-child(3),td:nth-child(3){text-align:left}.status{padding:.7rem;background:#f5f5f5}.error{color:#a00}.sort{font-size:.8em}.controls{display:flex;flex-wrap:wrap;gap:.7rem 1rem;align-items:center}.pagination{display:flex;align-items:center;gap:.7rem;margin-top:1rem}.pagination button:disabled{cursor:not-allowed;opacity:.5}.rate-high{color:#c00;font-weight:700}.hint{color:#667;font-size:.9em}.pkg-filter{position:relative;display:inline-block;vertical-align:middle}.pkg-trigger{min-width:18rem;text-align:left;background:#fff;border:1px solid #aaa}.pkg-trigger::after{content:"▾";float:right;margin-left:1rem}.pkg-panel{position:absolute;z-index:2;top:calc(100% + .25rem);left:0;width:19rem;padding:.6rem;background:#fff;border:1px solid #bbc;box-shadow:0 3px 10px #0002}.pkg-panel[hidden]{display:none}.pkg-search{box-sizing:border-box;width:100%;margin-bottom:.45rem}.pkg-options{max-height:12rem;overflow-y:auto;border:1px solid #ddd;padding:.2rem}.pkg-option{display:block;padding:.25rem}.pkg-option input{margin-right:.45rem}.pkg-actions{display:flex;justify-content:space-between;align-items:center;margin-top:.5rem}.pkg-actions button{padding:.25rem .5rem}.pkg-confirm{background:#3578c4;color:#fff;border:1px solid #28609c}
|
body{font:14px system-ui,sans-serif;margin:2rem;color:#223}label{margin-right:1rem}input,button{padding:.35rem}button{cursor:pointer}table{border-collapse:collapse;width:100%;margin-top:1rem}th,td{border:1px solid #ccd;padding:.45rem;text-align:right}th{cursor:pointer;background:#eef}th:first-child,td:first-child,th:nth-child(2),td:nth-child(2),th:nth-child(3),td:nth-child(3){text-align:left}.status{padding:.7rem;background:#f5f5f5}.error{color:#a00}.sort{font-size:.8em}.controls{display:flex;flex-wrap:wrap;gap:.7rem 1rem;align-items:center}.pagination{display:flex;align-items:center;gap:.7rem;margin-top:1rem}.pagination button:disabled{cursor:not-allowed;opacity:.5}.rate-high{color:#c00;font-weight:700}.hint{color:#667;font-size:.9em}.pkg-filter{position:relative;display:inline-block;vertical-align:middle}.pkg-trigger{min-width:18rem;text-align:left;background:#fff;border:1px solid #aaa}.pkg-trigger::after{content:"▾";float:right;margin-left:1rem}.pkg-panel{position:absolute;z-index:2;top:calc(100% + .25rem);left:0;width:19rem;padding:.6rem;background:#fff;border:1px solid #bbc;box-shadow:0 3px 10px #0002}.pkg-panel[hidden]{display:none}.pkg-search{box-sizing:border-box;width:100%;margin-bottom:.45rem}.pkg-options{max-height:12rem;overflow-y:auto;border:1px solid #ddd;padding:.2rem}.pkg-option{display:block;padding:.25rem}.pkg-option input{margin-right:.45rem}.pkg-actions{display:flex;justify-content:space-between;align-items:center;margin-top:.5rem}.pkg-actions button{padding:.25rem .5rem}.pkg-confirm{background:#3578c4;color:#fff;border:1px solid #28609c}
|
||||||
</style>
|
</style>
|
||||||
@@ -26,7 +27,8 @@ body{font:14px system-ui,sans-serif;margin:2rem;color:#223}label{margin-right:1r
|
|||||||
<script>
|
<script>
|
||||||
(() => {"use strict";
|
(() => {"use strict";
|
||||||
const columns=["dt","attr_platform","pkg_name","shareit_clicks","af_clicks","installs_ua","clicks_diff","clicks_diff_rate"];
|
const columns=["dt","attr_platform","pkg_name","shareit_clicks","af_clicks","installs_ua","clicks_diff","clicks_diff_rate"];
|
||||||
const pageSize=200;let rows=[],failed=[],page=1,sort={key:"dt",direction:-1};
|
const pageSize=200,maxConcurrent=6;let rows=[],failed=[],page=1,sort={key:"dt",direction:-1},files=[],loadVersion=0;
|
||||||
|
const dataCache=new Map();
|
||||||
const $=id=>document.getElementById(id),status=$("status");
|
const $=id=>document.getElementById(id),status=$("status");
|
||||||
function text(value){return value===null||value===undefined?"—":String(value)}
|
function text(value){return value===null||value===undefined?"—":String(value)}
|
||||||
function validDate(value){return /^\d{4}-\d{2}-\d{2}$/.test(value)&&!Number.isNaN(Date.parse(value))}
|
function validDate(value){return /^\d{4}-\d{2}-\d{2}$/.test(value)&&!Number.isNaN(Date.parse(value))}
|
||||||
@@ -46,9 +48,11 @@ function show(){const start=$("start").value,end=$("end").value;if(!validDate(st
|
|||||||
function updatePagination(total){const totalPages=Math.max(1,Math.ceil(total/pageSize));$("pageInfo").textContent=`第 ${total?`${page} / ${totalPages}`:"0 / 0"} 页`;$('previous').disabled=!total||page<=1;$('next').disabled=!total||page>=totalPages}
|
function updatePagination(total){const totalPages=Math.max(1,Math.ceil(total/pageSize));$("pageInfo").textContent=`第 ${total?`${page} / ${totalPages}`:"0 / 0"} 页`;$('previous').disabled=!total||page<=1;$('next').disabled=!total||page>=totalPages}
|
||||||
function setupHeaders(){const header=$("head");header.replaceChildren();columns.forEach(column=>{const th=document.createElement("th");th.dataset.key=column;th.textContent=column;th.addEventListener("click",()=>{if(sort.key===column)sort.direction*=-1;else{sort.key=column;sort.direction=-1}page=1;setupHeaders();show()});header.append(th)})}
|
function setupHeaders(){const header=$("head");header.replaceChildren();columns.forEach(column=>{const th=document.createElement("th");th.dataset.key=column;th.textContent=column;th.addEventListener("click",()=>{if(sort.key===column)sort.direction*=-1;else{sort.key=column;sort.direction=-1}page=1;setupHeaders();show()});header.append(th)})}
|
||||||
function resetPage(){page=1;show()}
|
function resetPage(){page=1;show()}
|
||||||
function setDefaultDates(){const dates=rows.map(row=>row.dt).sort();if(!dates.length){$("start").value=$("end").value="";return}const end=dates.at(-1),endDate=new Date(`${end.slice(0,4)}-${end.slice(4,6)}-${end.slice(6)}T00:00:00`);endDate.setDate(endDate.getDate()-29);$("end").value=`${end.slice(0,4)}-${end.slice(4,6)}-${end.slice(6)}`;$("start").value=localDate(endDate)}
|
function setDefaultDates(){const yesterday=new Date();yesterday.setDate(yesterday.getDate()-1);const date=localDate(yesterday);$("start").value=date;$("end").value=date}
|
||||||
function downloadRaw(){const blob=new Blob([JSON.stringify(rows,null,2)],{type:"application/json;charset=utf-8"}),url=URL.createObjectURL(blob),link=document.createElement("a"),stamp=new Date().toISOString().replaceAll(/[:.]/g,"-");link.href=url;link.download=`af-click-gap-raw-${stamp}.json`;link.click();URL.revokeObjectURL(url)}
|
function downloadRaw(){const blob=new Blob([JSON.stringify(rows,null,2)],{type:"application/json;charset=utf-8"}),url=URL.createObjectURL(blob),link=document.createElement("a"),stamp=new Date().toISOString().replaceAll(/[:.]/g,"-");link.href=url;link.download=`af-click-gap-raw-${stamp}.json`;link.click();URL.revokeObjectURL(url)}
|
||||||
async function load(){setupHeaders();try{const response=await fetch("data/index.json",{cache:"no-store"});if(!response.ok)throw Error("index.json 加载失败");const index=await response.json();if(!Array.isArray(index.files))throw Error("数据清单格式错误");const safe=index.files.filter(file=>/^\d{8}$/.test(file.date)&&file.path===`data/${file.date}.json`);failed=safe.length===index.files.length?[]:["清单路径"];const loaded=await Promise.all(safe.map(async file=>{try{const response=await fetch(file.path,{cache:"no-store"});if(!response.ok)throw Error("HTTP");const data=await response.json();if(!Array.isArray(data)||data.some(row=>!row||typeof row!=="object"||row.dt!==file.date))throw Error("格式");return data}catch(error){failed.push(file.date);return []}}));rows=loaded.flat();setDefaultDates();renderPackageOptions();$("pkgTrigger").textContent=packageLabel();if(rows.length)show();else{status.textContent="暂无数据。";updatePagination(0)}}catch(error){status.textContent=`无法加载数据:${error.message}`;status.className="status error"}}
|
async function fetchDate(file){if(dataCache.has(file.path))return {date:file.date,data:dataCache.get(file.path)};try{const response=await fetch(file.path,{cache:"no-store"});if(!response.ok)throw Error("HTTP");const data=await response.json();if(!Array.isArray(data)||data.some(row=>!row||typeof row!=="object"||row.dt!==file.date))throw Error("格式");dataCache.set(file.path,data);return {date:file.date,data}}catch(error){return {date:file.date,data:null}}}
|
||||||
$("pkgTrigger").addEventListener("click",()=>$("pkgPanel").hidden?openPackages():closePackages());$("pkgSearch").addEventListener("input",renderPackageOptions);$("selectAllPkg").addEventListener("click",()=>{[...new Set(rows.map(row=>row.pkg_name).filter(Boolean))].forEach(pkg=>draftPackages.add(pkg));renderPackageOptions()});$("clearPkg").addEventListener("click",()=>{draftPackages.clear();renderPackageOptions()});$("confirmPkg").addEventListener("click",()=>{appliedPackages=new Set(draftPackages);$("pkgTrigger").textContent=packageLabel();closePackages();resetPage()});document.addEventListener("click",event=>{if(!$("pkgFilter").contains(event.target)&&!$("pkgPanel").hidden)closePackages()});document.addEventListener("keydown",event=>{if(event.key==="Escape"&&!$("pkgPanel").hidden)closePackages()});$("apply").addEventListener("click",resetPage);$("start").addEventListener("change",resetPage);$("end").addEventListener("change",resetPage);$("reset").addEventListener("click",()=>{if(rows.length){appliedPackages.clear();$("pkgTrigger").textContent=packageLabel();setDefaultDates();resetPage()}});$("previous").addEventListener("click",()=>{if(page>1){page--;show()}});$("next").addEventListener("click",()=>{if(page<Math.ceil(filteredRows().length/pageSize)){page++;show()}});$("download").addEventListener("click",downloadRaw);load();
|
async function loadRange(){const version=++loadVersion,start=$("start").value,end=$("end").value;if(!validDate(start)||!validDate(end)||start>end){show();return}const selected=files.filter(file=>file.date>=date8(start)&&file.date<=date8(end));rows=[];failed=[];page=1;status.textContent=selected.length?`正在加载 ${selected.length} 个日期文件…`:`所选日期范围暂无数据文件。`;status.className="status";if(!selected.length){renderPackageOptions();show();return}for(let offset=0;offset<selected.length;offset+=maxConcurrent){const batch=selected.slice(offset,offset+maxConcurrent),loaded=await Promise.all(batch.map(fetchDate));if(version!==loadVersion)return;loaded.forEach(item=>{if(item.data)rows.push(...item.data);else failed.push(item.date)});renderPackageOptions();$("pkgTrigger").textContent=packageLabel();show();if(offset+batch.length<selected.length)status.textContent=`已加载 ${Math.min(offset+batch.length,selected.length)} / ${selected.length} 个日期文件…`}if(failed.length)status.textContent=`数据不完整:跳过 ${failed.length} 个文件。 ${status.textContent}`}
|
||||||
|
async function load(){setupHeaders();setDefaultDates();try{const response=await fetch("data/index.json",{cache:"no-store"});if(!response.ok)throw Error("index.json 加载失败");const index=await response.json();if(!Array.isArray(index.files))throw Error("数据清单格式错误");files=index.files.filter(file=>/^\d{8}$/.test(file.date)&&file.path===`data/${file.date}.json`);await loadRange()}catch(error){status.textContent=`无法加载数据:${error.message}`;status.className="status error"}}
|
||||||
|
$("pkgTrigger").addEventListener("click",()=>$("pkgPanel").hidden?openPackages():closePackages());$("pkgSearch").addEventListener("input",renderPackageOptions);$("selectAllPkg").addEventListener("click",()=>{[...new Set(rows.map(row=>row.pkg_name).filter(Boolean))].forEach(pkg=>draftPackages.add(pkg));renderPackageOptions()});$("clearPkg").addEventListener("click",()=>{draftPackages.clear();renderPackageOptions()});$("confirmPkg").addEventListener("click",()=>{appliedPackages=new Set(draftPackages);$("pkgTrigger").textContent=packageLabel();closePackages();resetPage()});document.addEventListener("click",event=>{if(!$("pkgFilter").contains(event.target)&&!$("pkgPanel").hidden)closePackages()});document.addEventListener("keydown",event=>{if(event.key==="Escape"&&!$("pkgPanel").hidden)closePackages()});$("apply").addEventListener("click",loadRange);$("start").addEventListener("change",loadRange);$("end").addEventListener("change",loadRange);$("reset").addEventListener("click",()=>{appliedPackages.clear();$("pkgTrigger").textContent=packageLabel();setDefaultDates();loadRange()});$("previous").addEventListener("click",()=>{if(page>1){page--;show()}});$("next").addEventListener("click",()=>{if(page<Math.ceil(filteredRows().length/pageSize)){page++;show()}});$("download").addEventListener("click",downloadRaw);load();
|
||||||
})();
|
})();
|
||||||
</script></body></html>
|
</script></body></html>
|
||||||
|
|||||||
+42
-22
@@ -3,39 +3,59 @@
|
|||||||
set -eu
|
set -eu
|
||||||
|
|
||||||
SCRIPT_DIR=$(CDPATH= cd -- "$(dirname -- "$0")" && pwd)
|
SCRIPT_DIR=$(CDPATH= cd -- "$(dirname -- "$0")" && pwd)
|
||||||
PROJECT_ROOT=${PROJECT_ROOT:-$(CDPATH= cd -- "$SCRIPT_DIR/.." && pwd)}
|
PROJECT_ROOT=$(CDPATH= cd -- "$SCRIPT_DIR/.." && pwd)
|
||||||
ENV_FILE=${ENV_FILE:-$(CDPATH= cd -- "$PROJECT_ROOT/../direct-report-dashboard" && pwd)/.env}
|
|
||||||
PYTHON_BIN=${PYTHON_BIN:-python3}
|
PYTHON_BIN=${PYTHON_BIN:-python3}
|
||||||
|
CREDENTIALS_FILE=${HOME:?}/.config/gitea/credentials.env
|
||||||
|
|
||||||
if [ ! -r "$ENV_FILE" ]; then
|
if [ ! -r "$CREDENTIALS_FILE" ]; then
|
||||||
printf '%s\n' "Credential file is not readable: $ENV_FILE" >&2
|
printf '%s\n' "Credential file is not readable: $CREDENTIALS_FILE" >&2
|
||||||
|
exit 1
|
||||||
|
fi
|
||||||
|
if [ "$(stat -f '%Lp' "$CREDENTIALS_FILE")" != 600 ]; then
|
||||||
|
printf '%s\n' "Credential file must have mode 600: $CREDENTIALS_FILE" >&2
|
||||||
exit 1
|
exit 1
|
||||||
fi
|
fi
|
||||||
|
|
||||||
# Read only the two Git credential variables; do not source the .env file.
|
credential_value() {
|
||||||
dotenv_value() {
|
awk -F= -v wanted="$1" '$1 ~ "^[[:space:]]*" wanted "[[:space:]]*$" { sub(/^[[:space:]]*/, "", $2); sub(/[[:space:]]*$/, "", $2); print $2; exit }' "$CREDENTIALS_FILE"
|
||||||
awk -v wanted="$1" '
|
|
||||||
$0 ~ "^[[:space:]]*" wanted "[[:space:]]*=" {
|
|
||||||
sub("^[[:space:]]*" wanted "[[:space:]]*=", "")
|
|
||||||
if ($0 ~ /^".*"[[:space:]]*$/) {
|
|
||||||
sub(/^"/, ""); sub(/"[[:space:]]*$/, "")
|
|
||||||
} else if ($0 ~ /^'"'"'.*'"'"'[[:space:]]*$/) {
|
|
||||||
sub(/^'"'"'/, ""); sub(/'"'"'[[:space:]]*$/, "")
|
|
||||||
}
|
|
||||||
print
|
|
||||||
exit
|
|
||||||
}
|
|
||||||
' "$ENV_FILE"
|
|
||||||
}
|
}
|
||||||
|
|
||||||
AGENTSTAGE_USER=$(dotenv_value AGENTSTAGE_USER)
|
AGENTSTAGE_USER=$(credential_value GITEA_USER)
|
||||||
AGENTSTAGE_TOKEN=$(dotenv_value AGENTSTAGE_TOKEN)
|
AGENTSTAGE_TOKEN=$(credential_value GITEA_TOKEN)
|
||||||
if [ -z "$AGENTSTAGE_USER" ] || [ -z "$AGENTSTAGE_TOKEN" ]; then
|
missing=''
|
||||||
printf '%s\n' "AGENTSTAGE_USER and AGENTSTAGE_TOKEN are required in $ENV_FILE" >&2
|
[ -n "$AGENTSTAGE_USER" ] || missing="$missing AGENTSTAGE_USER"
|
||||||
|
[ -n "$AGENTSTAGE_TOKEN" ] || missing="$missing AGENTSTAGE_TOKEN"
|
||||||
|
if [ -n "$missing" ]; then
|
||||||
|
printf 'Missing required .env variable(s):%s\n' "$missing" >&2
|
||||||
|
exit 1
|
||||||
|
fi
|
||||||
|
if [ "$AGENTSTAGE_USER" != "liuji" ]; then
|
||||||
|
printf '%s\n' "AGENTSTAGE_USER must be liuji" >&2
|
||||||
exit 1
|
exit 1
|
||||||
fi
|
fi
|
||||||
export AGENTSTAGE_USER AGENTSTAGE_TOKEN
|
export AGENTSTAGE_USER AGENTSTAGE_TOKEN
|
||||||
|
|
||||||
|
# Override Git only for this process tree: never read osxkeychain and never alter .git/config.
|
||||||
|
REMOTE_URL=$(git -C "$PROJECT_ROOT" config --get remote.origin.url || true)
|
||||||
|
case "$REMOTE_URL" in
|
||||||
|
https://*)
|
||||||
|
REMOTE_TARGET=${REMOTE_URL#https://}
|
||||||
|
REMOTE_TARGET=${REMOTE_TARGET#*@}
|
||||||
|
REMOTE_URL="https://liuji@$REMOTE_TARGET"
|
||||||
|
;;
|
||||||
|
*)
|
||||||
|
printf '%s\n' "remote.origin.url must use HTTPS" >&2
|
||||||
|
exit 1
|
||||||
|
;;
|
||||||
|
esac
|
||||||
|
export GIT_CONFIG_COUNT=3
|
||||||
|
export GIT_CONFIG_KEY_0=credential.helper
|
||||||
|
export GIT_CONFIG_VALUE_0=
|
||||||
|
export GIT_CONFIG_KEY_1=remote.origin.url
|
||||||
|
export GIT_CONFIG_VALUE_1="$REMOTE_URL"
|
||||||
|
export GIT_CONFIG_KEY_2=remote.origin.pushurl
|
||||||
|
export GIT_CONFIG_VALUE_2="$REMOTE_URL"
|
||||||
|
|
||||||
umask 077
|
umask 077
|
||||||
ASKPASS=$(mktemp "${TMPDIR:-/tmp}/af-click-gap-askpass.XXXXXX")
|
ASKPASS=$(mktemp "${TMPDIR:-/tmp}/af-click-gap-askpass.XXXXXX")
|
||||||
cleanup() {
|
cleanup() {
|
||||||
|
|||||||
Reference in New Issue
Block a user