feat: publish click gap 20261014-20261020

.gitignore

@@ -1,6 +1,7 @@
 .DS_Store
 .env
 .env.*
+credentials.env
 *.db
 *.sqlite
 *.sqlite3

README.md

@@ -1,73 +1,45 @@
-# Click Gap 数据看板
+https://ops.roasbloom.site/af-click-gap/
 
-运行目标是 **mac2**。另一个独立的外部 AppsFlyer 任务每天把汇总结果写入 SQLite；本项目不调用该任务、不访问 AppsFlyer、Superset 或其它提取 API，只从固定数据库只读查询并发布静态 JSON。
+# af-click-gap
 
-固定数据库：`/Users/ads-center/Documents/database/af_click_gap.db`。数据库必须预先存在 `click_gap` 表，脚本不建表、不迁移、不写入数据库。表要求包含：
+生成 AppsFlyer 点击差异看板的数据。上游 AppsFlyer 脚本负责采集并写入 SQLite；本项目流水线只读数据库，生成 JSON 并发布到 Gitea。
 
-```text
-pid TEXT NOT NULL, dt TEXT NOT NULL, attr_platform TEXT NOT NULL,
-pkg_name TEXT NOT NULL, shareit_clicks REAL NOT NULL, af_clicks REAL NOT NULL,
-installs_ua REAL NOT NULL, clicks_diff REAL NOT NULL,
-clicks_diff_rate REAL, updated_at TEXT NOT NULL
-```
+## 数据与输出
 
-主键必须为 `(pid, dt, pkg_name)`。脚本只读取 `pid='shareit'`，并校验字段、日期、数值、重复业务键和有限数值。
-
-## 输出协议
-
-- `data/YYYYMMDD.json`：直接 JSON 数组；最近 7 天每天生成一个文件，即使当天无数据也生成 `[]`。
-- `data/index.json`：包含 `schema_version`、`generated_at` 和最近 7 天的 `data/YYYYMMDD.json` 清单。
-- 每行字段为 `dt`（`YYYYMMDD`）、`attr_platform`、`pkg_name`、`shareit_clicks`、`af_clicks`、`installs_ua`、`clicks_diff`、`clicks_diff_rate`；最后一项可以为 `null`。
-
-默认日期边界为运行日往前 6 天至运行日，首尾包含，共 7 天。手工补数使用 `--start-date` 和 `--end-date`，两者必须同时提供且最多 7 天。
-
-## 手工运行
-
-mac2 推荐始终通过包装脚本运行：
-
-```sh
-cd /Users/shareit./Library/CloudStorage/OneDrive-个人/ubuntu/Gitea/af-click-gap
-scripts/run_pipeline.sh
-scripts/run_pipeline.sh --start-date 2026-10-10 --end-date 2026-10-16
-```
-
-包装脚本默认定位项目根目录，并读取 sibling 项目 `../direct-report-dashboard/.env`（mac2 默认路径为 `~/ubuntu/Gitea/af-click-gap` 与 `~/ubuntu/Gitea/direct-report-dashboard`）。也可以用 `PROJECT_ROOT` 和 `ENV_FILE` 覆盖路径。它只解析 `AGENTSTAGE_USER`、`AGENTSTAGE_TOKEN`，不执行整个 `.env`；凭据只进入子进程环境，不写入文件、remote URL、命令行参数或日志。脚本使用临时 `GIT_ASKPASS` 响应 Git 的用户名和密码提示，临时文件权限为仅当前用户可读写，并在退出时清理。
-
-不要直接运行 `python3 scripts/af_click_gap_pipeline.py --push`：该命令可能使用 mac2 Keychain 中没有仓库写权限的旧凭据。包装脚本会执行相同的 SQLite 最近 7 天提取和 `--push`，额外参数会继续传给流水线。需要只检查数据时可直接使用 `python3 scripts/af_click_gap_pipeline.py --dry-run`。
-
-`--dry-run` 只读取数据库并在仓库外临时目录生成、校验 JSON，不写 `data` 或 Git。正式运行先原子替换 `data/` 输出目录中的数据文件，再执行 Git；数据库查询或校验失败不会发布文件。Git 安全检查只允许本次生成的 `data/*.json`（含 `data/index.json`），工作区存在其它未提交改动时拒绝继续。push 失败会保留本地 commit，不会 force push。
+- 数据库：`/Users/ads-center/Documents/database/af_click_gap.db`。
+- 默认生成最近 7 天（含当天）的 `data/YYYYMMDD.json`；使用 `--all` 时生成数据库中所有实际有数据日期的文件。
+- `data/index.json` 保存生成时间、版本和 JSON 文件清单。
+- 每个日期文件是 JSON 数组，记录包含 `dt`、`attr_platform`、`pkg_name`、`shareit_clicks`、`af_clicks`、`installs_ua`、`clicks_diff`、`clicks_diff_rate`。
+- SQLite 以只读方式打开；不会建表、迁移或写入数据库。
 
 ## 看板
 
-`index.html` 是无构建依赖的静态页面，读取并校验 `data/index.json` 及其清单，支持日期范围、包名筛选和表头排序。直接打开文件或执行 `python3 -m http.server` 后访问即可。
+`index.html` 是无构建依赖的静态页面，读取 `data/index.json` 和日期 JSON。开始日期、结束日期默认都为昨天，支持日期筛选、`pkg_name` 多选、表头排序、每页 200 行和下载原始数据。
 
-## mac2 launchd
+## mac2 任务
 
-launchd 会调用 `scripts/run_pipeline.sh`，因此也使用 sibling 项目 `.env` 中的凭据；模板通过 `PYTHON_BIN` 传入 mac2 的 Python 路径，并保留工作目录和 stdout/stderr 日志路径：
+两个任务按以下顺序执行：先由上游 AppsFlyer 脚本写数据库，再运行本项目流水线读取数据库并生成、推送 JSON：
 
 ```sh
-cd /Users/shareit./Library/CloudStorage/OneDrive-个人/ubuntu/Gitea/af-click-gap
-mkdir -p "$HOME/Library/Logs/af-click-gap" "$HOME/Library/LaunchAgents"
-PY=$(command -v python3)
-sed -e "s#__MAC2_PYTHON3__#$PY#" -e "s#__PROJECT_DIR__#$PWD#g" -e "s#__LOG_DIR__#$HOME/Library/Logs/af-click-gap#g" launchd/com.roasbloom.af-click-gap.plist.template > /tmp/com.roasbloom.af-click-gap.plist
-plutil -lint /tmp/com.roasbloom.af-click-gap.plist
-cp /tmp/com.roasbloom.af-click-gap.plist "$HOME/Library/LaunchAgents/"
-launchctl bootstrap "gui/$(id -u)" "$HOME/Library/LaunchAgents/com.roasbloom.af-click-gap.plist"
+/Users/ads-center/py314env/bin/python /Users/ads-center/ubuntu/appsflyer/af_click_gap.py
+/Users/ads-center/ubuntu/Gitea/af-click-gap/scripts/run_pipeline.sh
 ```
 
-卸载：`launchctl bootout "gui/$(id -u)/com.roasbloom.af-click-gap"` 后删除 LaunchAgents 中的 plist。日志写在 mac2 用户日志目录；数据库、`.env`、密钥、日志和缓存均不会提交。
+流水线脚本从用户目录的 `~/.config/gitea/credentials.env` 读取共享 Gitea 凭证（变量名为 `GITEA_USER` 和 `GITEA_TOKEN`），文件权限必须为 `600`，并要求账号为 `liuji`。凭据只通过临时 `GIT_ASKPASS` 提供给 Git，不写入 remote URL、命令行、日志或仓库。不要把该文件提交到仓库。业务配置仍可使用项目自己的 `.env`。
 
-## Git 创建和推送
-
-远程仓库约定为 Gitea `https://git.ops.roasbloom.site/liuji/af-click-gap.git`。仓库创建和首次配置可使用已有流程；日常 mac2 发布必须使用 `scripts/run_pipeline.sh`，它从 `/Users/shareit./Library/CloudStorage/OneDrive-个人/ubuntu/Gitea/direct-report-dashboard/.env`（或 `ENV_FILE` 指定路径）读取 `AGENTSTAGE_USER` 和 `AGENTSTAGE_TOKEN`。凭据不进入 Git、remote URL、命令行、临时 askpass 文件或日志，且 `.env` 绝不提交。
-
-## 验证和安全
+## 手动运行
 
 ```sh
-python3 -m py_compile scripts/af_click_gap_pipeline.py
-python3 scripts/af_click_gap_pipeline.py --help
-plutil -lint launchd/com.roasbloom.af-click-gap.plist.template
-python3 -m json.tool data/index.json
+cd /Users/ads-center/ubuntu/Gitea/af-click-gap
+./scripts/run_pipeline.sh                         # 最近 7 天并推送
+./scripts/run_pipeline.sh --start-date 2026-10-10 --end-date 2026-10-16
+./scripts/run_pipeline.sh --all                   # 全量日期并推送
+python3 scripts/af_click_gap_pipeline.py --dry-run
+python3 scripts/af_click_gap_pipeline.py --all --dry-run
 ```
 
-禁止提交数据库文件、`.env*`、密钥、日志、浏览器状态、缓存或临时文件；`.gitignore` 已覆盖这些类型。
+`--all` 不能与日期范围同时使用；全量运行可能提交大量历史 JSON。`--dry-run` 只检查数据并在临时目录生成 JSON，不修改 `data/` 或 Git。
+
+## 安全
+
+不要提交 `af_click_gap.db`、数据库、`.env`、token、密钥、日志、浏览器状态或缓存。发布前确认工作区没有无关改动；不再使用 launchd 目录或 plist 配置。

data/20261020.json

@@ -0,0 +1 @@
+[]

data/index.json

@@ -1,7 +1,23 @@
 {
   "schema_version": 1,
-  "generated_at": "2026-10-16T15:19:21+08:00",
+  "generated_at": "2026-10-20T18:08:43+08:00",
   "files": [
+    {
+      "date": "20261020",
+      "path": "data/20261020.json"
+    },
+    {
+      "date": "20261019",
+      "path": "data/20261019.json"
+    },
+    {
+      "date": "20261018",
+      "path": "data/20261018.json"
+    },
+    {
+      "date": "20261017",
+      "path": "data/20261017.json"
+    },
     {
       "date": "20261016",
       "path": "data/20261016.json"
@@ -13,22 +29,6 @@
     {
       "date": "20261014",
       "path": "data/20261014.json"
-    },
-    {
-      "date": "20261013",
-      "path": "data/20261013.json"
-    },
-    {
-      "date": "20261012",
-      "path": "data/20261012.json"
-    },
-    {
-      "date": "20261011",
-      "path": "data/20261011.json"
-    },
-    {
-      "date": "20261010",
-      "path": "data/20261010.json"
     }
   ]
 }

index.html

@@ -2,6 +2,7 @@
 <html lang="zh-CN">
 <head>
 <meta charset="utf-8"><meta name="viewport" content="width=device-width, initial-scale=1"><title>Click Gap 数据看板</title>
+<link rel="icon" type="image/png" href="icon.png">
 <style>
 body{font:14px system-ui,sans-serif;margin:2rem;color:#223}label{margin-right:1rem}input,button{padding:.35rem}button{cursor:pointer}table{border-collapse:collapse;width:100%;margin-top:1rem}th,td{border:1px solid #ccd;padding:.45rem;text-align:right}th{cursor:pointer;background:#eef}th:first-child,td:first-child,th:nth-child(2),td:nth-child(2),th:nth-child(3),td:nth-child(3){text-align:left}.status{padding:.7rem;background:#f5f5f5}.error{color:#a00}.sort{font-size:.8em}.controls{display:flex;flex-wrap:wrap;gap:.7rem 1rem;align-items:center}.pagination{display:flex;align-items:center;gap:.7rem;margin-top:1rem}.pagination button:disabled{cursor:not-allowed;opacity:.5}.rate-high{color:#c00;font-weight:700}.hint{color:#667;font-size:.9em}.pkg-filter{position:relative;display:inline-block;vertical-align:middle}.pkg-trigger{min-width:18rem;text-align:left;background:#fff;border:1px solid #aaa}.pkg-trigger::after{content:"▾";float:right;margin-left:1rem}.pkg-panel{position:absolute;z-index:2;top:calc(100% + .25rem);left:0;width:19rem;padding:.6rem;background:#fff;border:1px solid #bbc;box-shadow:0 3px 10px #0002}.pkg-panel[hidden]{display:none}.pkg-search{box-sizing:border-box;width:100%;margin-bottom:.45rem}.pkg-options{max-height:12rem;overflow-y:auto;border:1px solid #ddd;padding:.2rem}.pkg-option{display:block;padding:.25rem}.pkg-option input{margin-right:.45rem}.pkg-actions{display:flex;justify-content:space-between;align-items:center;margin-top:.5rem}.pkg-actions button{padding:.25rem .5rem}.pkg-confirm{background:#3578c4;color:#fff;border:1px solid #28609c}
 </style>
@@ -26,7 +27,8 @@ body{font:14px system-ui,sans-serif;margin:2rem;color:#223}label{margin-right:1r
 <script>
 (() => {"use strict";
 const columns=["dt","attr_platform","pkg_name","shareit_clicks","af_clicks","installs_ua","clicks_diff","clicks_diff_rate"];
-const pageSize=200;let rows=[],failed=[],page=1,sort={key:"dt",direction:-1};
+const pageSize=200,maxConcurrent=6;let rows=[],failed=[],page=1,sort={key:"dt",direction:-1},files=[],loadVersion=0;
+const dataCache=new Map();
 const $=id=>document.getElementById(id),status=$("status");
 function text(value){return value===null||value===undefined?"—":String(value)}
 function validDate(value){return /^\d{4}-\d{2}-\d{2}$/.test(value)&&!Number.isNaN(Date.parse(value))}
@@ -46,9 +48,11 @@ function show(){const start=$("start").value,end=$("end").value;if(!validDate(st
 function updatePagination(total){const totalPages=Math.max(1,Math.ceil(total/pageSize));$("pageInfo").textContent=`第 ${total?`${page} / ${totalPages}`:"0 / 0"} 页`;$('previous').disabled=!total||page<=1;$('next').disabled=!total||page>=totalPages}
 function setupHeaders(){const header=$("head");header.replaceChildren();columns.forEach(column=>{const th=document.createElement("th");th.dataset.key=column;th.textContent=column;th.addEventListener("click",()=>{if(sort.key===column)sort.direction*=-1;else{sort.key=column;sort.direction=-1}page=1;setupHeaders();show()});header.append(th)})}
 function resetPage(){page=1;show()}
-function setDefaultDates(){const dates=rows.map(row=>row.dt).sort();if(!dates.length){$("start").value=$("end").value="";return}const end=dates.at(-1),endDate=new Date(`${end.slice(0,4)}-${end.slice(4,6)}-${end.slice(6)}T00:00:00`);endDate.setDate(endDate.getDate()-29);$("end").value=`${end.slice(0,4)}-${end.slice(4,6)}-${end.slice(6)}`;$("start").value=localDate(endDate)}
+function setDefaultDates(){const yesterday=new Date();yesterday.setDate(yesterday.getDate()-1);const date=localDate(yesterday);$("start").value=date;$("end").value=date}
 function downloadRaw(){const blob=new Blob([JSON.stringify(rows,null,2)],{type:"application/json;charset=utf-8"}),url=URL.createObjectURL(blob),link=document.createElement("a"),stamp=new Date().toISOString().replaceAll(/[:.]/g,"-");link.href=url;link.download=`af-click-gap-raw-${stamp}.json`;link.click();URL.revokeObjectURL(url)}
-async function load(){setupHeaders();try{const response=await fetch("data/index.json",{cache:"no-store"});if(!response.ok)throw Error("index.json 加载失败");const index=await response.json();if(!Array.isArray(index.files))throw Error("数据清单格式错误");const safe=index.files.filter(file=>/^\d{8}$/.test(file.date)&&file.path===`data/${file.date}.json`);failed=safe.length===index.files.length?[]:["清单路径"];const loaded=await Promise.all(safe.map(async file=>{try{const response=await fetch(file.path,{cache:"no-store"});if(!response.ok)throw Error("HTTP");const data=await response.json();if(!Array.isArray(data)||data.some(row=>!row||typeof row!=="object"||row.dt!==file.date))throw Error("格式");return data}catch(error){failed.push(file.date);return []}}));rows=loaded.flat();setDefaultDates();renderPackageOptions();$("pkgTrigger").textContent=packageLabel();if(rows.length)show();else{status.textContent="暂无数据。";updatePagination(0)}}catch(error){status.textContent=`无法加载数据：${error.message}`;status.className="status error"}}
-$("pkgTrigger").addEventListener("click",()=>$("pkgPanel").hidden?openPackages():closePackages());$("pkgSearch").addEventListener("input",renderPackageOptions);$("selectAllPkg").addEventListener("click",()=>{[...new Set(rows.map(row=>row.pkg_name).filter(Boolean))].forEach(pkg=>draftPackages.add(pkg));renderPackageOptions()});$("clearPkg").addEventListener("click",()=>{draftPackages.clear();renderPackageOptions()});$("confirmPkg").addEventListener("click",()=>{appliedPackages=new Set(draftPackages);$("pkgTrigger").textContent=packageLabel();closePackages();resetPage()});document.addEventListener("click",event=>{if(!$("pkgFilter").contains(event.target)&&!$("pkgPanel").hidden)closePackages()});document.addEventListener("keydown",event=>{if(event.key==="Escape"&&!$("pkgPanel").hidden)closePackages()});$("apply").addEventListener("click",resetPage);$("start").addEventListener("change",resetPage);$("end").addEventListener("change",resetPage);$("reset").addEventListener("click",()=>{if(rows.length){appliedPackages.clear();$("pkgTrigger").textContent=packageLabel();setDefaultDates();resetPage()}});$("previous").addEventListener("click",()=>{if(page>1){page--;show()}});$("next").addEventListener("click",()=>{if(page<Math.ceil(filteredRows().length/pageSize)){page++;show()}});$("download").addEventListener("click",downloadRaw);load();
+async function fetchDate(file){if(dataCache.has(file.path))return {date:file.date,data:dataCache.get(file.path)};try{const response=await fetch(file.path,{cache:"no-store"});if(!response.ok)throw Error("HTTP");const data=await response.json();if(!Array.isArray(data)||data.some(row=>!row||typeof row!=="object"||row.dt!==file.date))throw Error("格式");dataCache.set(file.path,data);return {date:file.date,data}}catch(error){return {date:file.date,data:null}}}
+async function loadRange(){const version=++loadVersion,start=$("start").value,end=$("end").value;if(!validDate(start)||!validDate(end)||start>end){show();return}const selected=files.filter(file=>file.date>=date8(start)&&file.date<=date8(end));rows=[];failed=[];page=1;status.textContent=selected.length?`正在加载 ${selected.length} 个日期文件…`:`所选日期范围暂无数据文件。`;status.className="status";if(!selected.length){renderPackageOptions();show();return}for(let offset=0;offset<selected.length;offset+=maxConcurrent){const batch=selected.slice(offset,offset+maxConcurrent),loaded=await Promise.all(batch.map(fetchDate));if(version!==loadVersion)return;loaded.forEach(item=>{if(item.data)rows.push(...item.data);else failed.push(item.date)});renderPackageOptions();$("pkgTrigger").textContent=packageLabel();show();if(offset+batch.length<selected.length)status.textContent=`已加载 ${Math.min(offset+batch.length,selected.length)} / ${selected.length} 个日期文件…`}if(failed.length)status.textContent=`数据不完整：跳过 ${failed.length} 个文件。 ${status.textContent}`}
+async function load(){setupHeaders();setDefaultDates();try{const response=await fetch("data/index.json",{cache:"no-store"});if(!response.ok)throw Error("index.json 加载失败");const index=await response.json();if(!Array.isArray(index.files))throw Error("数据清单格式错误");files=index.files.filter(file=>/^\d{8}$/.test(file.date)&&file.path===`data/${file.date}.json`);await loadRange()}catch(error){status.textContent=`无法加载数据：${error.message}`;status.className="status error"}}
+$("pkgTrigger").addEventListener("click",()=>$("pkgPanel").hidden?openPackages():closePackages());$("pkgSearch").addEventListener("input",renderPackageOptions);$("selectAllPkg").addEventListener("click",()=>{[...new Set(rows.map(row=>row.pkg_name).filter(Boolean))].forEach(pkg=>draftPackages.add(pkg));renderPackageOptions()});$("clearPkg").addEventListener("click",()=>{draftPackages.clear();renderPackageOptions()});$("confirmPkg").addEventListener("click",()=>{appliedPackages=new Set(draftPackages);$("pkgTrigger").textContent=packageLabel();closePackages();resetPage()});document.addEventListener("click",event=>{if(!$("pkgFilter").contains(event.target)&&!$("pkgPanel").hidden)closePackages()});document.addEventListener("keydown",event=>{if(event.key==="Escape"&&!$("pkgPanel").hidden)closePackages()});$("apply").addEventListener("click",loadRange);$("start").addEventListener("change",loadRange);$("end").addEventListener("change",loadRange);$("reset").addEventListener("click",()=>{appliedPackages.clear();$("pkgTrigger").textContent=packageLabel();setDefaultDates();loadRange()});$("previous").addEventListener("click",()=>{if(page>1){page--;show()}});$("next").addEventListener("click",()=>{if(page<Math.ceil(filteredRows().length/pageSize)){page++;show()}});$("download").addEventListener("click",downloadRaw);load();
 })();
 </script></body></html>

launchd/com.roasbloom.af-click-gap.plist.template

@@ -1,12 +0,0 @@
-<!-- TEMPLATE_ONLY: render and validate on mac2 before installing. -->
-<?xml version="1.0" encoding="UTF-8"?>
-<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
-<plist version="1.0"><dict>
-<key>Label</key><string>com.roasbloom.af-click-gap</string>
-<key>ProgramArguments</key><array><string>__PROJECT_DIR__/scripts/run_pipeline.sh</string></array>
-<key>EnvironmentVariables</key><dict><key>PYTHON_BIN</key><string>__MAC2_PYTHON3__</string></dict>
-<key>WorkingDirectory</key><string>__PROJECT_DIR__</string>
-<key>StartCalendarInterval</key><dict><key>Hour</key><integer>2</integer><key>Minute</key><integer>0</integer></dict>
-<key>StandardOutPath</key><string>__LOG_DIR__/pipeline.stdout.log</string><key>StandardErrorPath</key><string>__LOG_DIR__/pipeline.stderr.log</string>
-<key>RunAtLoad</key><false/>
-</dict></plist>

scripts/af_click_gap_pipeline.py

@@ -103,21 +103,33 @@ def validate_existing_schema(path: str) -> None:
         raise RuntimeError("click_gap 主键必须为 (pid, dt, pkg_name)")
 
 
-def read_rows(path: str, start: str, end: str, pid: str = PID) -> list[dict[str, Any]]:
+def read_rows(path: str, start: str | None, end: str | None, pid: str = PID) -> list[dict[str, Any]]:
     validate_existing_schema(path)
     try:
         with sqlite3.connect(f"file:{path}?mode=ro", uri=True) as conn:
             conn.row_factory = sqlite3.Row
-            result = conn.execute(
+            query = (
                 "SELECT dt, attr_platform, pkg_name, shareit_clicks, af_clicks, installs_ua, clicks_diff, clicks_diff_rate "
-                "FROM click_gap WHERE pid=? AND dt BETWEEN ? AND ? ORDER BY dt DESC, shareit_clicks DESC, pkg_name",
-                (pid, start, end),
+                "FROM click_gap WHERE pid=?"
             )
+            parameters: list[str] = [pid]
+            if start is not None and end is not None:
+                query += " AND dt BETWEEN ? AND ?"
+                parameters.extend((start, end))
+            query += " ORDER BY dt DESC, shareit_clicks DESC, pkg_name"
+            result = conn.execute(query, parameters)
             return [dict(row) for row in result]
     except sqlite3.Error as exc:
         raise RuntimeError(f"SQLite 查询失败: {exc}") from exc
 
 
+def available_dates(rows: list[dict[str, Any]]) -> list[str]:
+    dates = set()
+    for row in rows:
+        dates.add(date8(str(row["dt"])))
+    return sorted(dates, reverse=True)
+
+
 def validate_rows(rows: list[dict[str, Any]], start: str, end: str) -> list[dict[str, Any]]:
     normalized = []
     for row in rows:
@@ -213,6 +225,7 @@ def parse_args() -> argparse.Namespace:
     parser = argparse.ArgumentParser(description="Read SQLite click-gap data and publish JSON")
     parser.add_argument("--start-date", help="读取开始日期 YYYY-MM-DD")
     parser.add_argument("--end-date", help="读取结束日期 YYYY-MM-DD")
+    parser.add_argument("--all", action="store_true", help="读取数据库中所有实际有数据的日期")
     parser.add_argument("--dry-run", action="store_true", help="只读取、校验并生成临时 JSON，不写文件或 Git")
     group = parser.add_mutually_exclusive_group()
     group.add_argument("--no-push", action="store_true", help="写文件但不 push")
@@ -226,6 +239,14 @@ def main() -> int:
     try:
         if args.dry_run and (args.no_push or args.push):
             raise ValueError("--dry-run 不可与 --push/--no-push 同时使用")
+        if args.all and (args.start_date or args.end_date):
+            raise ValueError("--all 不可与 --start-date/--end-date 同时使用")
+        if args.all:
+            rows = validate_rows(read_rows(args.db_path, None, None), "00010101", "99991231")
+            dates = available_dates(rows)
+            start = dates[-1] if dates else "无日期"
+            end = dates[0] if dates else "无日期"
+        else:
             start, end = window(args.start_date, args.end_date)
             rows = validate_rows(read_rows(args.db_path, start, end), start, end)
             first = dt.datetime.strptime(start, "%Y%m%d").date()

scripts/run_pipeline.sh

@@ -3,39 +3,59 @@
 set -eu
 
 SCRIPT_DIR=$(CDPATH= cd -- "$(dirname -- "$0")" && pwd)
-PROJECT_ROOT=${PROJECT_ROOT:-$(CDPATH= cd -- "$SCRIPT_DIR/.." && pwd)}
-ENV_FILE=${ENV_FILE:-$(CDPATH= cd -- "$PROJECT_ROOT/../direct-report-dashboard" && pwd)/.env}
+PROJECT_ROOT=$(CDPATH= cd -- "$SCRIPT_DIR/.." && pwd)
 PYTHON_BIN=${PYTHON_BIN:-python3}
+CREDENTIALS_FILE=${HOME:?}/.config/gitea/credentials.env
 
-if [ ! -r "$ENV_FILE" ]; then
-    printf '%s\n' "Credential file is not readable: $ENV_FILE" >&2
+if [ ! -r "$CREDENTIALS_FILE" ]; then
+    printf '%s\n' "Credential file is not readable: $CREDENTIALS_FILE" >&2
+    exit 1
+fi
+if [ "$(stat -f '%Lp' "$CREDENTIALS_FILE")" != 600 ]; then
+    printf '%s\n' "Credential file must have mode 600: $CREDENTIALS_FILE" >&2
     exit 1
 fi
 
-# Read only the two Git credential variables; do not source the .env file.
-dotenv_value() {
-    awk -v wanted="$1" '
-        $0 ~ "^[[:space:]]*" wanted "[[:space:]]*=" {
-            sub("^[[:space:]]*" wanted "[[:space:]]*=", "")
-            if ($0 ~ /^".*"[[:space:]]*$/) {
-                sub(/^"/, ""); sub(/"[[:space:]]*$/, "")
-            } else if ($0 ~ /^'"'"'.*'"'"'[[:space:]]*$/) {
-                sub(/^'"'"'/, ""); sub(/'"'"'[[:space:]]*$/, "")
-            }
-            print
-            exit
-        }
-    ' "$ENV_FILE"
+credential_value() {
+    awk -F= -v wanted="$1" '$1 ~ "^[[:space:]]*" wanted "[[:space:]]*$" { sub(/^[[:space:]]*/, "", $2); sub(/[[:space:]]*$/, "", $2); print $2; exit }' "$CREDENTIALS_FILE"
 }
 
-AGENTSTAGE_USER=$(dotenv_value AGENTSTAGE_USER)
-AGENTSTAGE_TOKEN=$(dotenv_value AGENTSTAGE_TOKEN)
-if [ -z "$AGENTSTAGE_USER" ] || [ -z "$AGENTSTAGE_TOKEN" ]; then
-    printf '%s\n' "AGENTSTAGE_USER and AGENTSTAGE_TOKEN are required in $ENV_FILE" >&2
+AGENTSTAGE_USER=$(credential_value GITEA_USER)
+AGENTSTAGE_TOKEN=$(credential_value GITEA_TOKEN)
+missing=''
+[ -n "$AGENTSTAGE_USER" ] || missing="$missing AGENTSTAGE_USER"
+[ -n "$AGENTSTAGE_TOKEN" ] || missing="$missing AGENTSTAGE_TOKEN"
+if [ -n "$missing" ]; then
+    printf 'Missing required .env variable(s):%s\n' "$missing" >&2
+    exit 1
+fi
+if [ "$AGENTSTAGE_USER" != "liuji" ]; then
+    printf '%s\n' "AGENTSTAGE_USER must be liuji" >&2
     exit 1
 fi
 export AGENTSTAGE_USER AGENTSTAGE_TOKEN
 
+# Override Git only for this process tree: never read osxkeychain and never alter .git/config.
+REMOTE_URL=$(git -C "$PROJECT_ROOT" config --get remote.origin.url || true)
+case "$REMOTE_URL" in
+    https://*)
+        REMOTE_TARGET=${REMOTE_URL#https://}
+        REMOTE_TARGET=${REMOTE_TARGET#*@}
+        REMOTE_URL="https://liuji@$REMOTE_TARGET"
+        ;;
+    *)
+        printf '%s\n' "remote.origin.url must use HTTPS" >&2
+        exit 1
+        ;;
+esac
+export GIT_CONFIG_COUNT=3
+export GIT_CONFIG_KEY_0=credential.helper
+export GIT_CONFIG_VALUE_0=
+export GIT_CONFIG_KEY_1=remote.origin.url
+export GIT_CONFIG_VALUE_1="$REMOTE_URL"
+export GIT_CONFIG_KEY_2=remote.origin.pushurl
+export GIT_CONFIG_VALUE_2="$REMOTE_URL"
+
 umask 077
 ASKPASS=$(mktemp "${TMPDIR:-/tmp}/af-click-gap-askpass.XXXXXX")
 cleanup() {