固化部署体检与备份合同刷新（08-02-deployment-preflight-backup-contract-refresh）
这个提交包含在：
@@ -7,7 +7,7 @@ operations require explicit approval and evidence.
|
|||||||
|
|
||||||
## Rules
|
## Rules
|
||||||
|
|
||||||
- Existing installations use `DEPLOY_SERVER=<ssh-target> bash deploy/trialrun/deploy-existing.sh` only when deployment is approved; the wrapper reads the frozen allowlisted contract and keeps `SYNC_ENV=0`. First initialization or controlled recovery without that unit uses `deploy.sh` with explicit absolute data paths and `DEPLOY_DATA_MODE`; neither path overwrites remote `.env` by default.
|
- Existing installations use `DEPLOY_SERVER=<deploy-ssh-target> DEPLOY_REFRESH_SERVER=<forced-command-ssh-target> bash deploy/trialrun/deploy-existing.sh` only when deployment is approved; the wrapper reads the frozen allowlisted contract, keeps `SYNC_ENV=0`, performs aggregate read-only preflight and refreshes fixed backup build drop-ins after release. First initialization or controlled recovery without that unit uses `deploy.sh` with explicit absolute data paths and `DEPLOY_DATA_MODE`; neither path overwrites remote `.env` by default.
|
||||||
- When `server/src/config.js` reads a new environment key, add the same key to
|
- When `server/src/config.js` reads a new environment key, add the same key to
|
||||||
`deploy/trialrun/.env.example` before deployment; `deploy/trialrun/deploy.sh`
|
`deploy/trialrun/.env.example` before deployment; `deploy/trialrun/deploy.sh`
|
||||||
runs `node deploy/trialrun/validate-env-contract.js` at step 0 and must fail
|
runs `node deploy/trialrun/validate-env-contract.js` at step 0 and must fail
|
||||||
@@ -1292,7 +1292,10 @@ bash scripts/qa/run-automatic-backup-contract.sh
|
|||||||
|
|
||||||
### 2. Signatures
|
### 2. Signatures
|
||||||
|
|
||||||
- Existing deployment entry: `DEPLOY_SERVER=<ssh-target> bash deploy/trialrun/deploy-existing.sh`.
|
- Existing deployment entry: `DEPLOY_SERVER=<deploy-ssh-target> DEPLOY_REFRESH_SERVER=<forced-command-ssh-target> bash deploy/trialrun/deploy-existing.sh`.
|
||||||
|
- Existing deployment root-refresh identity: `DEPLOY_REFRESH_SERVER=<forced-command-ssh-target>`; it is a separate dedicated key/alias, not sudo and not the ordinary deploy identity.
|
||||||
|
- Aggregate read-only preflight: `deploy/trialrun/check-existing-deployment.sh`.
|
||||||
|
- Root-owned forced-command source/installer: `scripts/refresh-deployment-contract.sh` and `deploy/trialrun/install-deployment-contract-refresh.sh`.
|
||||||
- Remote preflight: `deploy/trialrun/preflight-remote.sh` executed over SSH before rsync.
|
- Remote preflight: `deploy/trialrun/preflight-remote.sh` executed over SSH before rsync.
|
||||||
- Canonical parameter source: allowlisted `Environment=` values from `hyb-oa-automatic-backup-daily.service`.
|
- Canonical parameter source: allowlisted `Environment=` values from `hyb-oa-automatic-backup-daily.service`.
|
||||||
- Stable image stages: `base`, `client_deps`, `client_build`, `server_deps`, `server_build`.
|
- Stable image stages: `base`, `client_deps`, `client_build`, `server_deps`, `server_build`.
|
||||||
@@ -1302,9 +1305,14 @@ bash scripts/qa/run-automatic-backup-contract.sh
|
|||||||
- Never derive `REMOTE_DIR` from Compose `com.docker.compose.project.working_dir`; it may name the Compose-file directory rather than the repository root.
|
- Never derive `REMOTE_DIR` from Compose `com.docker.compose.project.working_dir`; it may name the Compose-file directory rather than the repository root.
|
||||||
- The existing-deployment wrapper reads only `REMOTE_DIR`, `REMOTE_ENV_FILE`, `COMPOSE_FILE`, `COMPOSE_PROJECT_NAME`, `POSTGRES_DATA_DIR` and `STORAGE_HOST_DIR`; it does not echo their values.
|
- The existing-deployment wrapper reads only `REMOTE_DIR`, `REMOTE_ENV_FILE`, `COMPOSE_FILE`, `COMPOSE_PROJECT_NAME`, `POSTGRES_DATA_DIR` and `STORAGE_HOST_DIR`; it does not echo their values.
|
||||||
- Before `mkdir` or rsync, existing mode requires repository markers, `PG_VERSION`, storage directory and a regular non-symlink env file when `SYNC_ENV=0`.
|
- Before `mkdir` or rsync, existing mode requires repository markers, `PG_VERSION`, storage directory and a regular non-symlink env file when `SYNC_ENV=0`.
|
||||||
|
- Before the fail-fast remote preflight, the existing-deployment wrapper runs all aggregate checks without `set -e`: each check executes in an isolated conditional/subshell, results are emitted in a fixed order, and any failure blocks all writes only after the complete report is produced. Output contains check names and PASS/FAIL only.
|
||||||
|
- The ordinary deploy identity and the root maintenance identity are separate. Root maintenance uses a dedicated key whose only authorized entry is a forced command with forwarding, PTY and user rc disabled. If the same key blob has an unmanaged or unrestricted root entry, installation fails.
|
||||||
|
- The forced-command target is root-owned, non-symlink and self-contained under `/usr/local/sbin`. It accepts exact `probe` and `refresh` commands through `SSH_ORIGINAL_COMMAND`; it never executes the repository, accepts no caller path/unit/SHA, and uses fixed service/timer names.
|
||||||
- If the frozen unit points to a missing or non-repository root, recover the canonical source tree from an exact published Git commit into a new dedicated root. Do not choose among stale source directories by timestamp, and do not use `--delete` or carry an environment file into the new root during this bootstrap.
|
- If the frozen unit points to a missing or non-repository root, recover the canonical source tree from an exact published Git commit into a new dedicated root. Do not choose among stale source directories by timestamp, and do not use `--delete` or carry an environment file into the new root during this bootstrap.
|
||||||
- When the frozen `REMOTE_ENV_FILE` is missing, locate candidates from the running Compose configuration and compare each candidate's service/database values with the running containers by per-key digest. Output only key names and PASS/FAIL; never print values or digests. A candidate must pass every container-observable key, including the field-encryption and database identity keys, before root copies it (not moves it) to the canonical root with deployment-user ownership and mode `0600`. If no candidate fully passes, stop and use the environment recovery classification; never use `SYNC_ENV=1` to guess.
|
- When the frozen `REMOTE_ENV_FILE` is missing, locate candidates from the running Compose configuration and compare each candidate's service/database values with the running containers by per-key digest. Output only key names and PASS/FAIL; never print values or digests. A candidate must pass every container-observable key, including the field-encryption and database identity keys, before root copies it (not moves it) to the canonical root with deployment-user ownership and mode `0600`. If no candidate fully passes, stop and use the environment recovery classification; never use `SYNC_ENV=1` to guess.
|
||||||
- After a successful application switch, re-run the automatic-backup and WARCH3 installers with the newly active full `BUILD_SHA` and matching 12-character frontend prefix. Validate all managed service working directories/env paths/build identities and enabled timers, then run one complete backup plus the freshness watchdog; application `/ready` alone does not prove the backup contract followed the release.
|
- When recovering a broken deployment contract or changing frozen paths/project identity, re-run the automatic-backup and WARCH3 installers with the active full `BUILD_SHA` and matching 12-character frontend prefix. Validate all managed service working directories/env paths/build identities and enabled timers, then run one complete backup plus the freshness watchdog; application `/ready` alone does not prove that a controlled recovery is complete.
|
||||||
|
- For normal existing deployments after forced-command installation, the wrapper replaces the manual per-release installer step: after application, smoke and public checks, `refresh` derives the full SHA from the unique active server, atomically writes root-owned build-only drop-ins for the three automatic-backup services and WARCH3, reloads systemd, and verifies four final service SHAs plus four enabled/active timers. Full installers remain authoritative for path or complete-unit changes.
|
||||||
|
- A post-refresh failure does not roll back a healthy application. The wrapper emits `DEPLOYED_BUT_CONTRACT_REFRESH_FAILED`, classifies root transport, missing program, stale unit build or inactive timer, and exits non-zero; only successful refresh may print the complete existing-deployment PASS.
|
||||||
- `apk add` and both lockfile `npm ci` steps precede `BUILD_SHA` / `VITE_BUILD_SHA`. Build identity may invalidate application output, never stable dependency installation.
|
- `apk add` and both lockfile `npm ci` steps precede `BUILD_SHA` / `VITE_BUILD_SHA`. Build identity may invalidate application output, never stable dependency installation.
|
||||||
- Seed and smoke output reports readiness, counts and booleans; it does not print full login identifiers, contact values, WeCom identifiers, redirect URI or generated credentials.
|
- Seed and smoke output reports readiness, counts and booleans; it does not print full login identifiers, contact values, WeCom identifiers, redirect URI or generated credentials.
|
||||||
|
|
||||||
@@ -1316,20 +1324,27 @@ bash scripts/qa/run-automatic-backup-contract.sh
|
|||||||
- Environment copy digest, owner or mode mismatch -> stop before formal deploy and keep the original candidate unchanged.
|
- Environment copy digest, owner or mode mismatch -> stop before formal deploy and keep the original candidate unchanged.
|
||||||
- Existing PGDATA missing `PG_VERSION` or storage missing -> preflight fails before rsync.
|
- Existing PGDATA missing `PG_VERSION` or storage missing -> preflight fails before rsync.
|
||||||
- Canonical systemd unit missing an allowlisted value -> wrapper fails without calling `deploy.sh`.
|
- Canonical systemd unit missing an allowlisted value -> wrapper fails without calling `deploy.sh`.
|
||||||
|
- An early aggregate item fails -> all independent items still run, the fixed summary reports every FAIL, and no remote write begins.
|
||||||
|
- Root SSH returns transport 255 -> `root_identity_unavailable`; remote 126/127 or missing probe marker -> `refresh_program_unavailable`.
|
||||||
|
- Helper runs but final service build differs -> `unit_build_not_refreshed`; builds match but any timer is disabled/inactive -> `timer_not_active`.
|
||||||
|
- A similarly named environment key such as `OLD_BUILD_SHA` matches the active value while exact `BUILD_SHA` remains stale -> treat the service as stale; build verification parses exact keys and applies the last systemd override.
|
||||||
- Post-deploy installer still embeds the previous build SHA, any managed timer is inactive, or the real backup/watchdog fails runtime identity or off-host freshness -> deployment recovery remains incomplete even when `/ready` is green.
|
- Post-deploy installer still embeds the previous build SHA, any managed timer is inactive, or the real backup/watchdog fails runtime identity or off-host freshness -> deployment recovery remains incomplete even when `/ready` is green.
|
||||||
- Lockfile unchanged but a new SHA reruns dependency installation -> Dockerfile contract test fails.
|
- Lockfile unchanged but a new SHA reruns dependency installation -> Dockerfile contract test fails.
|
||||||
|
|
||||||
### 5. Good/Base/Bad Cases
|
### 5. Good/Base/Bad Cases
|
||||||
|
|
||||||
- Good: wrapper reads the frozen unit contract, preflight passes, Git archive syncs and unchanged dependency layers are cached.
|
- Good: wrapper reads the frozen unit contract, preflight passes, Git archive syncs and unchanged dependency layers are cached.
|
||||||
|
- Good: ordinary and forced-command identities pass separately, all aggregate items are reported, deployment passes, root helper updates only fixed build drop-ins, and four timers remain enabled/active.
|
||||||
- Good: a broken-root recovery creates a clean published-commit source tree, proves one environment candidate against running containers without exposing values, copies it with mode `0600`, deploys with `SYNC_ENV=0`, then updates the backup contracts and completes a real off-host backup.
|
- Good: a broken-root recovery creates a clean published-commit source tree, proves one environment candidate against running containers without exposing values, copies it with mode `0600`, deploys with `SYNC_ENV=0`, then updates the backup contracts and completes a real off-host backup.
|
||||||
- Base: first initialization uses reviewed explicit parameters and `DEPLOY_DATA_MODE=initialize`; it does not pretend an existing unit is available.
|
- Base: first initialization uses reviewed explicit parameters and `DEPLOY_DATA_MODE=initialize`; it does not pretend an existing unit is available.
|
||||||
- Bad: inspect a running container's `working_dir`, append another `deploy/trialrun`, then discover the mistake only after rsync.
|
- Bad: inspect a running container's `working_dir`, append another `deploy/trialrun`, then discover the mistake only after rsync.
|
||||||
- Bad: choose one of several stale `.env` files by path or modification time, use `SYNC_ENV=1`, or stop after `/ready` while the backup unit still expects the previous build.
|
- Bad: choose one of several stale `.env` files by path or modification time, use `SYNC_ENV=1`, or stop after `/ready` while the backup unit still expects the previous build.
|
||||||
|
- Bad: grant NOPASSWD to a repository installer, point forced-command at a deploy-user-writable script, reuse an unrestricted root key, stop aggregation after its first failure, or downgrade refresh failure to a warning.
|
||||||
|
|
||||||
### 6. Tests Required
|
### 6. Tests Required
|
||||||
|
|
||||||
- `bash -n deploy/trialrun/deploy.sh deploy/trialrun/deploy-existing.sh deploy/trialrun/preflight-remote.sh`.
|
- `bash -n deploy/trialrun/deploy.sh deploy/trialrun/deploy-existing.sh deploy/trialrun/preflight-remote.sh`.
|
||||||
|
- `bash scripts/qa/run-deployment-contract-refresh-contract.sh` must prove multi-failure aggregation reaches the final item, output redaction, forced-command exact matching, four failure categories, four build drop-ins and four timer checks.
|
||||||
- `node deploy/trialrun/validate-env-contract.js`.
|
- `node deploy/trialrun/validate-env-contract.js`.
|
||||||
- `cd server && npm test -- tests/unit/scripts/deployment-safety-contract.test.js`.
|
- `cd server && npm test -- tests/unit/scripts/deployment-safety-contract.test.js`.
|
||||||
- Synthetic fixtures must prove wrong root, missing/symlink env and invalid data fail; static ordering must prove preflight precedes rsync and dependency stages precede build args.
|
- Synthetic fixtures must prove wrong root, missing/symlink env and invalid data fail; static ordering must prove preflight precedes rsync and dependency stages precede build args.
|
||||||
@@ -1349,12 +1364,13 @@ REMOTE_DIR="$(docker inspect <server> --format '<working_dir-label>')" \
|
|||||||
|
|
||||||
```bash
|
```bash
|
||||||
DEPLOY_SERVER=<approved-ssh-target> \
|
DEPLOY_SERVER=<approved-ssh-target> \
|
||||||
|
DEPLOY_REFRESH_SERVER=<approved-forced-command-target> \
|
||||||
bash deploy/trialrun/deploy-existing.sh
|
bash deploy/trialrun/deploy-existing.sh
|
||||||
|
|
||||||
# If contract recovery was required, update the backup units with the active
|
# A normal release ends here: the wrapper refreshes the fixed build drop-ins
|
||||||
# full SHA after the switch, then prove both creation and freshness.
|
# and verifies all four timers. Do not manually start backup services on every
|
||||||
systemctl start hyb-oa-automatic-backup-daily.service
|
# release. If contract recovery was required, follow the separately approved
|
||||||
systemctl start hyb-oa-automatic-backup-watchdog.service
|
# recovery runbook to prove one complete backup and off-host freshness.
|
||||||
```
|
```
|
||||||
|
|
||||||
## Scenario: Environment Recovery Classification And Secret Custody
|
## Scenario: Environment Recovery Classification And Secret Custody
|
||||||
|
|||||||
@@ -0,0 +1 @@
|
|||||||
|
{"_example": "Fill with {\"file\": \"<path>\", \"reason\": \"<why>\"}. Put spec/research files only — no code paths. Run `python3 .trellis/scripts/get_context.py --mode packages` to list available specs. Delete this line once real entries are added."}
|
||||||
@@ -0,0 +1,120 @@
|
|||||||
|
# 部署前聚合体检与备份合同自动刷新：设计
|
||||||
|
|
||||||
|
## 1. 边界与组件
|
||||||
|
|
||||||
|
本任务只改已有部署入口，不改变首次初始化状态机：
|
||||||
|
|
||||||
|
1. 本地只读聚合体检入口读取受控 unit 六键，并分别通过普通部署 SSH 与专用 forced-command SSH 验证远端事实。
|
||||||
|
2. `deploy-existing.sh` 在体检全绿后继续调用既有 `deploy.sh`；`deploy.sh` 的 fail-fast remote preflight 保留。
|
||||||
|
3. 应用、公网检查全部通过后，`deploy-existing.sh` 调用 root forced-command 的 `refresh`。
|
||||||
|
4. root-owned 自包含程序只维护四个固定 service 的 build drop-in，并验证四个既有 timer。
|
||||||
|
|
||||||
|
不新增后台服务、数据库对象、业务 API 或业务页面。
|
||||||
|
|
||||||
|
## 2. 聚合体检执行模型
|
||||||
|
|
||||||
|
### 2.1 入口与输入
|
||||||
|
|
||||||
|
- 普通部署目标继续使用 `DEPLOY_SERVER`。
|
||||||
|
- 独立维护目标使用新的显式 `DEPLOY_REFRESH_SERVER`；格式使用现有安全 SSH target 字符集，不从主机名、用户名或别名规则猜测。
|
||||||
|
- 六键仍以 automatic-backup daily unit 的 allowlisted `Environment=` 为事实源，不回显值。
|
||||||
|
- 共享只读解析函数可由体检与 `deploy-existing.sh` 复用，避免两份六键 allowlist 漂移；不得把解析结果写入仓库或临时报告。
|
||||||
|
|
||||||
|
### 2.2 不短路聚合器
|
||||||
|
|
||||||
|
主脚本保留 `set -uo pipefail`，明确不启用 `set -e`。每项使用以下等价模型：
|
||||||
|
|
||||||
|
```bash
|
||||||
|
run_check() {
|
||||||
|
local check_name="$1"
|
||||||
|
shift
|
||||||
|
if ("$@") >/dev/null 2>&1; then
|
||||||
|
record_result "$check_name" PASS
|
||||||
|
else
|
||||||
|
record_result "$check_name" FAIL
|
||||||
|
fi
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
检查函数在子 shell 中运行，不能修改主流程状态。聚合器记录所有结果后统一输出固定顺序的 `检查项 PASS/FAIL` 和计数，最后按总失败数返回 0/非 0。依赖项无法取得时，依赖检查返回 FAIL；无依赖检查仍继续执行。
|
||||||
|
|
||||||
|
### 2.3 检查分组
|
||||||
|
|
||||||
|
- 本地/合同：普通 SSH transport、六键齐全、root SSH transport、forced-command probe。
|
||||||
|
- 普通部署身份：仓库根标识、目录可写、env 普通文件且可读、PGDATA/附件存在。
|
||||||
|
- 运行时：唯一 server/db、活动 build、附件 mount、PGDATA mount。
|
||||||
|
- 备份合同：daily unit build 等于活动 build。
|
||||||
|
- root 收尾能力：forced-command 程序可执行并返回固定 probe 标识。
|
||||||
|
|
||||||
|
所有检查仅把 stdout/stderr 捕获到内存并丢弃；用户可见输出由聚合器根据固定结果生成，不转发底层错误文本。
|
||||||
|
|
||||||
|
## 3. forced-command 安全模型
|
||||||
|
|
||||||
|
### 3.1 密钥边界
|
||||||
|
|
||||||
|
- 使用独立公钥。安装器解析 key blob/fingerprint，若 root `authorized_keys` 已存在同 key blob 的非 managed、非 forced-command 行则拒绝安装。
|
||||||
|
- managed 行使用固定标记，可原子替换自身，保留其他 root 管理 key；不得覆盖整个 `authorized_keys`。
|
||||||
|
- 授权选项固定为 `command="<root-owned-target>",no-agent-forwarding,no-port-forwarding,no-X11-forwarding,no-pty,no-user-rc`。
|
||||||
|
- 专用本地 SSH alias 只使用该 key。普通 root 应急管理 key 不属于部署自动化合同。
|
||||||
|
|
||||||
|
### 3.2 自包含程序
|
||||||
|
|
||||||
|
- 安装器把审核后的源文件复制到固定 `/usr/local/sbin` 目标，验证 root:root、`0755`、regular file、非 symlink。
|
||||||
|
- 程序启动后设置固定 `PATH`、`umask 077`，清理调用方可控的路径/loader 类环境变量，仅读取 root-owned systemd unit、Docker inspect 结果和 `SSH_ORIGINAL_COMMAND`。
|
||||||
|
- 精确命令矩阵：`probe` 只验证运行前提；`refresh` 执行 build drop-in 更新；空值、空白变体、额外参数和其他命令全部返回固定 unsupported 分类。
|
||||||
|
- 程序不得 source 或 exec 仓库内容，不接受 argv、stdin 配置、路径、unit、SHA 或 shell 片段。
|
||||||
|
|
||||||
|
## 4. build 刷新合同
|
||||||
|
|
||||||
|
### 4.1 事实读取
|
||||||
|
|
||||||
|
1. 从 root-owned automatic-backup daily unit 读取固定 allowlist：Compose project 与当前工作目录/env 合同只用于定位唯一活动 server。
|
||||||
|
2. 通过 Docker labels/inspect 定位唯一 running server，读取容器 `BUILD_SHA`。
|
||||||
|
3. build 必须为完整小写十六进制提交标识；前端标识固定取前 12 位，不接受调用方输入。
|
||||||
|
|
||||||
|
### 4.2 最小写入
|
||||||
|
|
||||||
|
- 固定目标仅为 automatic-backup daily/watchdog/drill-reminder 与 WARCH3 四个 service。
|
||||||
|
- 对每个 service 原子安装 root-owned `20-active-build.conf` drop-in，只包含 `BUILD_SHA` 和 `VITE_BUILD_SHA` 覆盖；不改主 unit、WorkingDirectory、env 路径、ExecStart 或 timer 配置。
|
||||||
|
- 所有 drop-in 写入完成后执行一次 `systemctl daemon-reload`。
|
||||||
|
- 读取 systemd 最终合并后的 Environment，逐个断言两个 SHA；再逐个断言四个 timer `is-enabled=enabled` 且 `is-active=active`。
|
||||||
|
- 使用固定 root-owned lock 防止两个 refresh 并发。中途失败不得报告成功；已写 drop-in 不自动回滚到旧 SHA，因为旧 SHA 已与活动应用不一致，人工修复应继续向活动 build 收敛。
|
||||||
|
|
||||||
|
## 5. 调用顺序与失败语义
|
||||||
|
|
||||||
|
```text
|
||||||
|
聚合体检全绿
|
||||||
|
-> deploy.sh（同步、构建、迁移、/ready、smoke、公网检查）
|
||||||
|
-> forced-command refresh
|
||||||
|
-> unit SHA + timer 验证
|
||||||
|
-> 部署完整完成
|
||||||
|
```
|
||||||
|
|
||||||
|
- 体检失败：零远端写入，部署退出非零。
|
||||||
|
- `deploy.sh` 失败：保持既有发布错误语义，不调用 refresh。
|
||||||
|
- refresh transport 失败：固定分类 `root_identity_unavailable`。
|
||||||
|
- remote command 126/127 或 probe 标识缺失：固定分类 `refresh_program_unavailable`。
|
||||||
|
- helper 返回/复核 SHA 不一致：`unit_build_not_refreshed`。
|
||||||
|
- SHA 已一致但任一 timer 非 enabled/active：`timer_not_active`。
|
||||||
|
- 后刷新失败时应用已部署且 smoke 通过，包装脚本输出固定 `DEPLOYED_BUT_CONTRACT_REFRESH_FAILED` 并退出非零；不伪装回滚，也不打印底层路径/响应。
|
||||||
|
|
||||||
|
## 6. 安装与发布边界
|
||||||
|
|
||||||
|
- 代码合入不自动修改生产 root 文件或 `authorized_keys`。
|
||||||
|
- 首次安装分两步：root 运行安装器；普通身份与 forced-command 身份分别执行 probe。两步均需生产写入批准和脱敏证据。
|
||||||
|
- 新版 `deploy-existing.sh` 对已有部署直接要求 `DEPLOY_REFRESH_SERVER`，缺失即在写前体检失败，不保留可长期绕过的 optional 模式。生产切换顺序固定为：先用旧运行版本不受影响的方式安装 forced-command 并完成 probe，再执行新版 wrapper；远端入口尚未安装时不得开始同步。
|
||||||
|
- 回滚代码时可恢复旧 `deploy-existing.sh`；root drop-in 仍指向当前活动 build，不影响服务启动。删除 forced-command 或 drop-in 属于单独 root 运维动作。
|
||||||
|
|
||||||
|
## 7. 测试设计
|
||||||
|
|
||||||
|
- SSH fixture 必须区分 transport 255、远端 126/127、helper 固定业务退出码和成功标识。
|
||||||
|
- 聚合测试故意让第一个、第三个检查失败，断言最后一个检查仍执行且输出顺序固定。
|
||||||
|
- 脱敏测试向底层 stderr 注入伪造路径、主机、secret、容器 ID，断言用户输出不包含它们。
|
||||||
|
- helper fixture 使用隔离的 fake systemctl/docker/install，不触碰真实 `/etc`；生产程序固定 PATH，测试通过源码级显式 test harness 注入依赖，安装目标不携带测试覆盖开关。
|
||||||
|
- 静态合同断言 installed helper 不引用 `REMOTE_DIR` 下脚本，不使用 `eval`/`bash -c`/动态 unit 名，不读取调用方 SHA。
|
||||||
|
|
||||||
|
## 8. 取舍与延期
|
||||||
|
|
||||||
|
- 选择 systemd drop-in 而非重新调用 installer：只写每次部署必变的 build 字段，避免执行部署用户可写源码，也避免复制两套完整 unit 生成逻辑。
|
||||||
|
- 选择 forced-command 专用 key 而非 sudoers：授权面只包含 probe/refresh；代价是需要一次独立 key 和 root authorized_keys 安装。
|
||||||
|
- 不加每周 timer：当前漂移已完成一次性全量对账，先用真实部署观察是否仍有持续漂移来源。
|
||||||
@@ -0,0 +1 @@
|
|||||||
|
{"_example": "Fill with {\"file\": \"<path>\", \"reason\": \"<why>\"}. Put spec/research files only — no code paths. Run `python3 .trellis/scripts/get_context.py --mode packages` to list available specs. Delete this line once real entries are added."}
|
||||||
@@ -0,0 +1,64 @@
|
|||||||
|
# 部署前聚合体检与备份合同自动刷新：实施计划
|
||||||
|
|
||||||
|
## 0. 边界与回滚点
|
||||||
|
|
||||||
|
- [x] 用户批准本规划后才运行 `task.py start`；批准规划不等于批准生产安装。
|
||||||
|
- [x] 实施阶段只修改仓库代码、测试、spec/runbook；不连接生产、不改 root `authorized_keys`、不安装 forced-command、不部署。
|
||||||
|
- [x] 保留 `deploy.sh` 首次初始化/应急兼容；`deploy-existing.sh` 的自动强制启用与生产 forced-command 安装安排在另行批准的同一窗口。
|
||||||
|
- [x] 不删除旧源码目录、旧环境文件、旧 unit 或备份制品。
|
||||||
|
|
||||||
|
## 1. 共享合同读取与聚合体检
|
||||||
|
|
||||||
|
- [x] 提取六键 systemd Environment allowlist 的共享只读解析逻辑，供 `deploy-existing.sh` 和新体检入口使用；任何输出不包含值。
|
||||||
|
- [x] 新增聚合器与固定检查表，主流程不用 `set -e`，每项子 shell 隔离、显式收集状态，最后统一返回。
|
||||||
|
- [x] 实现普通部署身份的仓库根、可写性、env 可读性、PGDATA/附件和运行容器 mount/build 核对。
|
||||||
|
- [x] 实现独立 `DEPLOY_REFRESH_SERVER` 的 transport/probe 核对，并把身份不可用与程序不可用分开分类。
|
||||||
|
- [x] 让 `deploy-existing.sh` 在任何远端写入前要求体检全绿；保留既有 remote preflight。
|
||||||
|
|
||||||
|
## 2. root-owned 自包含刷新程序
|
||||||
|
|
||||||
|
- [x] 新增无仓库运行时依赖的 forced-command target：固定 PATH/umask、清理危险环境、精确解析 `SSH_ORIGINAL_COMMAND`。
|
||||||
|
- [x] `probe` 验证 root 身份、固定工具、root-owned unit 与固定目标前提，只输出固定标识。
|
||||||
|
- [x] `refresh` 取得固定锁，从 root-owned unit 与唯一 running server 解析活动 build，拒绝调用方路径/SHA/unit 输入。
|
||||||
|
- [x] 以原子 root-owned systemd drop-in 覆盖四个 service 的两项 build 环境，daemon-reload 后验证最终 SHA。
|
||||||
|
- [x] 验证四个 timer enabled/active；用稳定退出码区分 unit 未刷新和 timer 未运行。
|
||||||
|
- [x] 不调用 repository installer，不使用 `eval`、动态 shell、动态 unit 名或部署用户可写可执行文件。
|
||||||
|
|
||||||
|
## 3. forced-command 安装器
|
||||||
|
|
||||||
|
- [x] 将 target 安装到固定 `/usr/local/sbin`，验证 root ownership、mode 和非 symlink。
|
||||||
|
- [x] 读取显式专用公钥，拒绝同 key blob 的不受限 root 授权；只原子维护自身 managed authorized_keys 行并保留其他 key。
|
||||||
|
- [x] forced-command 选项禁用 agent/port/X11/PTY/user-rc，安装完成后执行本地静态合同检查；不把 installer 本身加入免密或 forced-command。
|
||||||
|
- [x] 增加卸载/回滚说明，但不在无独立批准时自动删除授权行或 drop-in。
|
||||||
|
|
||||||
|
## 4. 部署后接线与状态表达
|
||||||
|
|
||||||
|
- [x] `deploy-existing.sh` 只在 `deploy.sh` 完整成功后调用 `refresh`。
|
||||||
|
- [x] 成功时才打印完整部署完成；失败时固定输出 `DEPLOYED_BUT_CONTRACT_REFRESH_FAILED`、四类之一并退出非零，不转发敏感 stderr。
|
||||||
|
- [x] 不增加 optional/跳过刷新开关；新版已有部署入口缺少 `DEPLOY_REFRESH_SERVER` 即在写前阻塞。生产窗口先安装并 probe forced-command，再运行新版 wrapper。
|
||||||
|
- [x] 更新 runbook/checklist，明确两个 SSH 身份、forced-command 专用 key、部署前聚合报告、后刷新非零语义和生产安装单独批准。
|
||||||
|
|
||||||
|
## 5. 测试与门禁
|
||||||
|
|
||||||
|
- [x] 新增聚合器 fixtures：多个同时失败、首项失败仍执行末项、依赖失败、固定输出顺序和最终退出码。
|
||||||
|
- [x] 新增脱敏 fixtures：底层输出含伪造路径/主机/env/容器 ID 时，外层报告不泄露。
|
||||||
|
- [x] 新增 forced-command fixtures：空命令、未知命令、附加参数、probe、refresh、锁、原子 drop-in、四 service SHA 与四 timer。
|
||||||
|
- [x] 覆盖四类后刷新失败：root transport、程序 126/127、unit SHA、timer 状态；断言 wrapper 非零且不打印完整成功。
|
||||||
|
- [x] 扩展 deployment safety contract，锁定体检在 mkdir/rsync 前，refresh 在 `/ready`、smoke 和公网检查后，且 helper 不执行仓库路径。
|
||||||
|
- [x] 运行：
|
||||||
|
- `bash -n`（所有新增/修改 shell）
|
||||||
|
- `node deploy/trialrun/validate-env-contract.js`
|
||||||
|
- `cd server && npm test -- tests/unit/scripts/deployment-safety-contract.test.js`
|
||||||
|
- 新增聚合/forced-command fixture runner
|
||||||
|
- `bash scripts/qa/run-automatic-backup-contract.sh`
|
||||||
|
- `python3 ./.trellis/scripts/task.py validate 08-02-deployment-preflight-backup-contract-refresh`
|
||||||
|
- Trellis governance、process contract、docs registry、sensitive scan、`git diff --check`
|
||||||
|
|
||||||
|
## 6. 审查、提交与生产门
|
||||||
|
|
||||||
|
- [x] 实施后执行只读安全/部署 reviewer，重点核查 forced-command key 是否真正受限、root target 是否执行部署用户可写内容、聚合器是否短路、状态分类是否准确；reviewer 不改代码。
|
||||||
|
- [x] 用 `trellis-update-spec` 更新 deployment-trialrun 的稳定合同，避免只把知识留在 task。
|
||||||
|
- [ ] 中文提交并引用 `08-02-deployment-preflight-backup-contract-refresh`；不自动提交、不自动部署。
|
||||||
|
- [ ] 代码获批后另行申请生产窗口：先安装 forced-command/probe，再启用 wrapper 强制刷新，执行一次真实部署或受控无业务写入验证；任何 root/authorized_keys 写入和部署均需单独明确批准。
|
||||||
|
|
||||||
|
本地验证（2026-08-02）：聚合/forced-command fixture 通过，覆盖 13 项汇总、多个并发失败不短路、脱敏输出、四类收尾映射、精确命令白名单、四个 build drop-in、四个 timer、managed key 更新及不受限重复 key 拒绝；deployment safety Vitest 9 passed；环境合同通过；automatic-backup 合同 73 checks passed；全部 shell/Node 语法通过。只读复查发现 forced-command target 缺少运行时自身 ownership/mode/path 校验，已补 `validate_self()`；最终复核又将 systemd build 校验从子串匹配收紧为精确键与最后覆盖值，并用 `OLD_BUILD_SHA` 回归证明不会静默误判。相关聚合、环境合同和 Vitest 均复跑通过；复查未发现新的 root 提权、动态执行或部署顺序 blocker。生产未连接、未安装、未部署。
|
||||||
@@ -0,0 +1,92 @@
|
|||||||
|
# 部署前聚合体检与备份合同自动刷新
|
||||||
|
|
||||||
|
## Goal
|
||||||
|
|
||||||
|
让已有部署在任何远端写入前一次性看清合同、权限、数据挂载和备份构建身份的全部漂移，并在应用切换成功后自动刷新备份/WARCH3 service 的构建标识。部署不再依赖操作者逐个撞 fail-fast 护栏，也不再依赖人工记住“部署后重装 timer”。
|
||||||
|
|
||||||
|
## Background
|
||||||
|
|
||||||
|
- 2026-08-02 的受控部署连续暴露三个历史漂移断面：冻结的 `REMOTE_DIR` 不是仓库根、root 维护身份与部署身份未按实际通路区分、冻结的环境文件路径不可由部署用户读取。现有护栏串行 fail-fast，因此每修复一处才看到下一处。
|
||||||
|
- 同一轮恢复已完成一次全量对账：六键合同、163 个生产环境键、运行容器、PGDATA/附件挂载、4 个 service、4 个 timer、真实备份和异机恢复点均已对齐并跑绿。本任务把该对账方法变成可重复机制。
|
||||||
|
- `deploy.sh` 只把新 `BUILD_SHA` 注入应用容器，不重装 automatic-backup/WARCH3 units；备份 runner 又要求 unit 中冻结的 build 与活动容器一致。人工漏做刷新时，应用可正常运行而自动备份静默 fail-closed。
|
||||||
|
- 仓库已有 forced-command 先例：异机备份接收端以 root-owned 可执行文件配合 `SSH_ORIGINAL_COMMAND` 白名单和 SSH 转发/PTY 禁用选项限制授权面。
|
||||||
|
|
||||||
|
## Scope
|
||||||
|
|
||||||
|
- 为已有部署增加一次性汇总的只读体检，并在任何远端写入前阻断全部已发现的合同、权限、挂载和 build 漂移。
|
||||||
|
- 建立独立 forced-command SSH 身份与 root-owned 自包含刷新程序，只更新四个固定 service 的 build drop-in并验证四个既有 timer。
|
||||||
|
- 把聚合体检接到 `deploy-existing.sh` 前置，把自动刷新接到完整部署与公网检查之后，并补齐测试、runbook、checklist 和部署 spec。
|
||||||
|
- 保留 `deploy.sh` 首次初始化/应急路径和既有 installers；生产安装、authorized_keys 修改和首次启用另行批准。
|
||||||
|
|
||||||
|
## Requirements
|
||||||
|
|
||||||
|
### 1. 只读聚合体检
|
||||||
|
|
||||||
|
- 新增已有部署专用体检入口，由 `deploy-existing.sh` 在任何 `mkdir`、rsync、Compose、systemd 或其他远端写入前调用。
|
||||||
|
- 体检至少覆盖：六键合同完整；`REMOTE_DIR` 是完整仓库根且部署用户可写；`REMOTE_ENV_FILE` 为普通文件且部署用户可读；PGDATA 含 `PG_VERSION`；附件目录存在；运行中唯一 server/db 容器与合同中的 build、PGDATA、附件挂载一致；automatic-backup unit 的 `BUILD_SHA` 与活动 server 一致；普通部署 SSH 身份可用；独立 root forced-command 身份可用且收尾程序存在、可执行。
|
||||||
|
- 主流程不得使用 `set -e` 实现检查遍历。每个检查必须在独立子 shell 中执行并显式收集退出码；前一项失败不能阻止无依赖的后续项执行。依赖事实缺失时，相关项仍以固定检查名报告 FAIL，不得静默跳过。
|
||||||
|
- 输出只包含固定检查项名、`PASS/FAIL`、汇总计数和固定错误分类；不输出 SSH 别名、主机、路径、env 值或散列、容器 ID、密钥、unit 原文及原始命令响应。
|
||||||
|
- 任一必需检查失败，体检最终退出非零，`deploy-existing.sh` 不得继续同步或部署。现有 fail-fast `preflight-remote.sh` 保留为第二道防线。
|
||||||
|
|
||||||
|
### 2. 独立 root forced-command 通路
|
||||||
|
|
||||||
|
- 使用独立维护 SSH 身份，不以 `sudo -n true` 代表 root 能力，不新增 NOPASSWD sudoers。
|
||||||
|
- 专用公钥必须只以 forced-command 形式授权；同一 key blob 若还存在不受限 root 授权，安装器必须 fail-closed。不得复用当前不受限 root 管理密钥来制造“看似受限”的入口。
|
||||||
|
- forced-command 指向 `/usr/local/sbin` 下 root-owned、非符号链接、`0755` 的自包含程序；程序不得 source、exec 或调用 `REMOTE_DIR` 及其他部署用户可写目录中的脚本、二进制或配置。
|
||||||
|
- SSH 入口禁用 agent forwarding、端口转发、X11、PTY 和 user rc；只接受 `SSH_ORIGINAL_COMMAND` 的精确白名单子命令，首版仅允许无附加参数的 `probe` 与 `refresh`。
|
||||||
|
|
||||||
|
### 3. 部署后自动刷新
|
||||||
|
|
||||||
|
- `deploy-existing.sh` 只有在应用部署、`/ready`、runtime/external smoke 与公网检查成功后，才通过独立 root forced-command 身份调用 `refresh`。
|
||||||
|
- root 程序从 root-owned systemd 合同和唯一活动 server 容器读取 Compose 身份及完整 `BUILD_SHA`，严格校验后仅更新四个固定 service 的 build 环境覆盖：automatic-backup daily、watchdog、drill-reminder 与 WARCH3。不得接受调用方传入路径、unit 名、SHA 或任意命令。
|
||||||
|
- 刷新必须原子写入 root-owned systemd drop-in，执行 `daemon-reload`，再验证四个 service 的最终 `BUILD_SHA/VITE_BUILD_SHA` 与活动容器一致，以及四个既有 timer 均为 enabled/active。不得调用仓库内 installer，也不得顺带重写其他 unit 字段。
|
||||||
|
- 刷新失败时 `deploy-existing.sh` 必须退出非零并明确标记“应用已部署但合同未收口”，不得打印普通成功结论，也不得自动回滚已通过 smoke 的应用。
|
||||||
|
- 对外固定区分四类失败：root SSH 身份不可用；收尾程序不存在或不可执行；程序完成但 unit SHA 未更新；unit 已更新但 timer 未 enabled/active。
|
||||||
|
|
||||||
|
### 4. 安装、兼容与生产边界
|
||||||
|
|
||||||
|
- 提供一次性 root 安装器，将经过审核的自包含程序复制为 root-owned 文件并安装/更新 managed forced-command 授权行；安装器本身仍需显式 root 执行，不进入免密授权。
|
||||||
|
- 保留现有 automatic-backup/WARCH3 installers 作为首次安装、路径/完整合同变化和人工修复入口；自动收尾只覆盖每次发布必变的 build 标识。
|
||||||
|
- 现有 `deploy.sh` 首次初始化/受控应急入口不强制依赖 forced-command；自动体检与刷新只接入 `deploy-existing.sh`。
|
||||||
|
- 生产安装 forced-command、修改 root `authorized_keys` 或首次启用自动收尾均属于生产写入，实施完成后必须另行批准；本任务规划批准和代码批准不授权远端安装。
|
||||||
|
|
||||||
|
## Acceptance Criteria
|
||||||
|
|
||||||
|
- [ ] 合成场景同时制造至少两个前置失败时，体检报告包含全部预期 FAIL 项，且最后一个独立检查确实执行；不能因 `set -e`、pipefail 或函数非零提前退出。
|
||||||
|
- [ ] 体检全绿时只输出固定项名和 PASS/汇总；敏感扫描与测试证明没有路径、主机、env 值/散列、容器 ID 或原始响应。
|
||||||
|
- [ ] 六键缺失、仓库根错误、env 不可读、源码根不可写、PGDATA/附件缺失、实际挂载漂移、unit build 漂移均能在远端写入前一次汇总发现。
|
||||||
|
- [ ] 普通部署身份和 root forced-command 身份分别验证；root transport 不可用与远端程序不存在/不可执行得到不同固定分类。
|
||||||
|
- [ ] forced-command 只接受精确 `probe`、`refresh`；空命令、带额外参数、其他子命令均拒绝。安装器拒绝同一公钥同时存在不受限 root 授权。
|
||||||
|
- [ ] 安装后的 root 程序为 root-owned、`0755`、非符号链接，且静态/动态测试证明不执行仓库或其他部署用户可写路径中的程序。
|
||||||
|
- [ ] 成功 `refresh` 从活动容器取得 build，原子更新四个固定 service 的 drop-in，四个最终 SHA 一致且四个 timer enabled/active；调用方无法指定 SHA、路径或 unit。
|
||||||
|
- [ ] 四类收尾失败分别有回归：SSH root 身份不可用、程序不存在/不可执行、unit SHA 未更新、timer 未运行。
|
||||||
|
- [ ] 任一收尾失败发生在应用已切换后，`deploy-existing.sh` 退出非零并输出固定“已部署但未收口”状态；成功时才输出部署完整完成。
|
||||||
|
- [ ] `deploy.sh` 直接初始化/应急路径保持兼容；automatic-backup/WARCH3 既有 installer、真实备份与 watchdog 行为不被改写。
|
||||||
|
- [ ] shell syntax、部署安全单测、环境合同、聚合体检 fixtures、forced-command fixtures、Trellis/governance/process/sensitive/diff 门禁全部通过。
|
||||||
|
|
||||||
|
## Out of Scope
|
||||||
|
|
||||||
|
- 不建设 CI/CD、蓝绿发布、自动回滚或新的通用远程执行框架。
|
||||||
|
- 不把体检挂成每周 timer；先观察它在后续真实部署中是否持续发现漂移，再单独评估第 5 个 unit。
|
||||||
|
- 不在本任务删除四个旧源码目录、旧环境文件或其他历史回退材料；体检跑绿也只形成后续清理的申请前提。
|
||||||
|
- 不通过 `SYNC_ENV=1` 修复环境文件，不自动挑选、复制或轮换生产 Secret。
|
||||||
|
- 不让 root 收尾程序修改 Compose、数据库、附件、业务数据、备份制品或执行真实恢复。
|
||||||
|
|
||||||
|
## Deferred Decision With Trigger
|
||||||
|
|
||||||
|
- 每周只读体检 timer 暂不建设。触发条件：本脚本上线后在后续真实部署中再次发现非一次性漂移，或出现人工改服务器导致合同在部署间隔内失配；届时单独立项评估定时运行、告警和第 5 个 unit 的维护成本。
|
||||||
|
- 旧目录/旧环境文件清理单独审批。至少要求聚合体检在规范根上跑绿、一次后续真实部署自动收尾跑绿，且当前异机备份仍新鲜。
|
||||||
|
|
||||||
|
## Test Plan
|
||||||
|
|
||||||
|
- `bash -n` 检查新增/修改的所有部署 shell。
|
||||||
|
- 扩展 `server/tests/unit/scripts/deployment-safety-contract.test.js`，锁定 preflight 在任何远端写入前、刷新在应用与公网检查后、失败必须向上传播。
|
||||||
|
- 新增纯 fixture 合同测试，使用假的 SSH/systemctl/docker 响应覆盖聚合不短路、脱敏输出、身份/程序四类失败、固定 unit/drop-in 和 timer 验证。
|
||||||
|
- 运行 `node deploy/trialrun/validate-env-contract.js`、相关 automatic-backup 合同测试及既有部署安全回归。
|
||||||
|
- 运行 Trellis task validate、governance、process contract、docs registry、sensitive content 与 `git diff --check`。
|
||||||
|
|
||||||
|
## Notes
|
||||||
|
|
||||||
|
- Keep `prd.md` focused on requirements, constraints, and acceptance criteria.
|
||||||
|
- Lightweight tasks can remain PRD-only.
|
||||||
|
- For complex tasks, add `design.md` for technical design and `implement.md` for execution planning before `task.py start`.
|
||||||
@@ -0,0 +1,26 @@
|
|||||||
|
{
|
||||||
|
"id": "deployment-preflight-backup-contract-refresh",
|
||||||
|
"name": "deployment-preflight-backup-contract-refresh",
|
||||||
|
"title": "部署前聚合体检与备份合同自动刷新",
|
||||||
|
"description": "新增只读聚合部署体检，并通过独立 forced-command root 身份自动刷新部署后的备份构建合同。",
|
||||||
|
"status": "in_progress",
|
||||||
|
"dev_type": "infra",
|
||||||
|
"scope": "deploy/trialrun 与部署安全合同",
|
||||||
|
"package": "hyb-oa",
|
||||||
|
"priority": "P2",
|
||||||
|
"creator": "shunli",
|
||||||
|
"assignee": "shunli",
|
||||||
|
"createdAt": "2026-08-02",
|
||||||
|
"completedAt": null,
|
||||||
|
"branch": null,
|
||||||
|
"base_branch": "main",
|
||||||
|
"worktree_path": null,
|
||||||
|
"commit": null,
|
||||||
|
"pr_url": null,
|
||||||
|
"subtasks": [],
|
||||||
|
"children": [],
|
||||||
|
"parent": null,
|
||||||
|
"relatedFiles": [],
|
||||||
|
"notes": "",
|
||||||
|
"meta": {}
|
||||||
|
}
|
||||||
+174
@@ -0,0 +1,174 @@
|
|||||||
|
#!/usr/bin/env bash
|
||||||
|
set -uo pipefail
|
||||||
|
|
||||||
|
ROOT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")/../.." && pwd)"
|
||||||
|
DEPLOY_SERVER="${DEPLOY_SERVER:-}"
|
||||||
|
DEPLOY_REFRESH_SERVER="${DEPLOY_REFRESH_SERVER:-}"
|
||||||
|
DEPLOY_CONTRACT_UNIT="${DEPLOY_CONTRACT_UNIT:-hyb-oa-automatic-backup-daily.service}"
|
||||||
|
|
||||||
|
# shellcheck source=deployment-contract-lib.sh
|
||||||
|
source "$ROOT_DIR/deploy/trialrun/deployment-contract-lib.sh"
|
||||||
|
|
||||||
|
CHECK_NAMES=(
|
||||||
|
deploy_ssh_identity
|
||||||
|
contract_six_keys
|
||||||
|
repository_root
|
||||||
|
repository_writable
|
||||||
|
env_readable
|
||||||
|
data_directories
|
||||||
|
runtime_unique
|
||||||
|
runtime_build
|
||||||
|
storage_mount
|
||||||
|
pgdata_mount
|
||||||
|
backup_build
|
||||||
|
root_ssh_identity
|
||||||
|
refresh_program
|
||||||
|
)
|
||||||
|
declare -A CHECK_RESULTS=()
|
||||||
|
declare -A CONTRACT=()
|
||||||
|
for check_name in "${CHECK_NAMES[@]}"; do
|
||||||
|
CHECK_RESULTS["$check_name"]="FAIL"
|
||||||
|
done
|
||||||
|
|
||||||
|
if deployment_contract_safe_ssh_target "$DEPLOY_SERVER" \
|
||||||
|
&& ssh "$DEPLOY_SERVER" true >/dev/null 2>&1; then
|
||||||
|
CHECK_RESULTS[deploy_ssh_identity]="PASS"
|
||||||
|
fi
|
||||||
|
|
||||||
|
if deployment_contract_load "$DEPLOY_SERVER" "$DEPLOY_CONTRACT_UNIT" CONTRACT; then
|
||||||
|
CHECK_RESULTS[contract_six_keys]="PASS"
|
||||||
|
|
||||||
|
remote_results="$(ssh "$DEPLOY_SERVER" \
|
||||||
|
"REMOTE_DIR='${CONTRACT[REMOTE_DIR]}' REMOTE_ENV_FILE='${CONTRACT[REMOTE_ENV_FILE]}' COMPOSE_FILE='${CONTRACT[COMPOSE_FILE]}' COMPOSE_PROJECT_NAME='${CONTRACT[COMPOSE_PROJECT_NAME]}' POSTGRES_DATA_DIR='${CONTRACT[POSTGRES_DATA_DIR]}' STORAGE_HOST_DIR='${CONTRACT[STORAGE_HOST_DIR]}' CONTRACT_UNIT='${DEPLOY_CONTRACT_UNIT}' bash -s" 2>/dev/null <<'REMOTE'
|
||||||
|
set -uo pipefail
|
||||||
|
|
||||||
|
emit() {
|
||||||
|
local name="$1"
|
||||||
|
shift
|
||||||
|
if ("$@") >/dev/null 2>&1; then
|
||||||
|
printf '%s=PASS\n' "$name"
|
||||||
|
else
|
||||||
|
printf '%s=FAIL\n' "$name"
|
||||||
|
fi
|
||||||
|
}
|
||||||
|
|
||||||
|
repository_root() {
|
||||||
|
[[ -d "$REMOTE_DIR" && -f "$REMOTE_DIR/server/package.json" \
|
||||||
|
&& -f "$REMOTE_DIR/$COMPOSE_FILE" ]]
|
||||||
|
}
|
||||||
|
repository_writable() { [[ -w "$REMOTE_DIR" ]]; }
|
||||||
|
env_readable() { [[ -f "$REMOTE_ENV_FILE" && ! -L "$REMOTE_ENV_FILE" && -r "$REMOTE_ENV_FILE" ]]; }
|
||||||
|
data_directories() { [[ -f "$POSTGRES_DATA_DIR/PG_VERSION" && -d "$STORAGE_HOST_DIR" ]]; }
|
||||||
|
|
||||||
|
container_ids() {
|
||||||
|
local service="$1"
|
||||||
|
docker ps \
|
||||||
|
--filter "label=com.docker.compose.project=$COMPOSE_PROJECT_NAME" \
|
||||||
|
--filter "label=com.docker.compose.service=$service" \
|
||||||
|
--format '{{.ID}}'
|
||||||
|
}
|
||||||
|
|
||||||
|
container_env() {
|
||||||
|
local container="$1"
|
||||||
|
local wanted="$2"
|
||||||
|
docker inspect -f '{{range .Config.Env}}{{println .}}{{end}}' "$container" \
|
||||||
|
| sed -n "s/^${wanted}=//p" | tail -n 1
|
||||||
|
}
|
||||||
|
|
||||||
|
mount_source() {
|
||||||
|
local container="$1"
|
||||||
|
local destination="$2"
|
||||||
|
docker inspect -f '{{json .Mounts}}' "$container" | python3 -c '
|
||||||
|
import json, sys
|
||||||
|
destination = sys.argv[1]
|
||||||
|
mounts = json.load(sys.stdin)
|
||||||
|
matches = [item.get("Source", "") for item in mounts if item.get("Destination") == destination]
|
||||||
|
if len(matches) != 1 or not matches[0]:
|
||||||
|
raise SystemExit(1)
|
||||||
|
print(matches[0])
|
||||||
|
' "$destination"
|
||||||
|
}
|
||||||
|
|
||||||
|
mapfile -t servers < <(container_ids server 2>/dev/null || true)
|
||||||
|
mapfile -t dbs < <(container_ids db 2>/dev/null || true)
|
||||||
|
server_id="${servers[0]:-}"
|
||||||
|
db_id="${dbs[0]:-}"
|
||||||
|
runtime_unique_ok=0
|
||||||
|
if [[ "${#servers[@]}" -eq 1 && "${#dbs[@]}" -eq 1 \
|
||||||
|
&& -n "$server_id" && -n "$db_id" ]]; then
|
||||||
|
runtime_unique_ok=1
|
||||||
|
fi
|
||||||
|
|
||||||
|
runtime_unique() { [[ "$runtime_unique_ok" == 1 ]]; }
|
||||||
|
runtime_build() {
|
||||||
|
[[ "$runtime_unique_ok" == 1 ]] || return 1
|
||||||
|
active_build="$(container_env "$server_id" BUILD_SHA)"
|
||||||
|
[[ "$active_build" =~ ^[0-9a-fA-F]{40,64}$ ]]
|
||||||
|
}
|
||||||
|
storage_mount() {
|
||||||
|
[[ "$runtime_unique_ok" == 1 ]] || return 1
|
||||||
|
storage_destination="$(container_env "$server_id" STORAGE_DIR)"
|
||||||
|
[[ "$storage_destination" == /* ]] || return 1
|
||||||
|
[[ "$(mount_source "$server_id" "$storage_destination")" == "$STORAGE_HOST_DIR" ]]
|
||||||
|
}
|
||||||
|
pgdata_mount() {
|
||||||
|
[[ "$runtime_unique_ok" == 1 ]] || return 1
|
||||||
|
[[ "$(mount_source "$db_id" /var/lib/postgresql/data)" == "$POSTGRES_DATA_DIR" ]]
|
||||||
|
}
|
||||||
|
backup_build() {
|
||||||
|
runtime_build || return 1
|
||||||
|
unit_env="$(systemctl show "$CONTRACT_UNIT" -p Environment --value)"
|
||||||
|
unit_build=""
|
||||||
|
for entry in $unit_env; do
|
||||||
|
if [[ "${entry%%=*}" == "BUILD_SHA" ]]; then
|
||||||
|
unit_build="${entry#*=}"
|
||||||
|
fi
|
||||||
|
done
|
||||||
|
[[ "$unit_build" == "$active_build" ]]
|
||||||
|
}
|
||||||
|
|
||||||
|
emit repository_root repository_root
|
||||||
|
emit repository_writable repository_writable
|
||||||
|
emit env_readable env_readable
|
||||||
|
emit data_directories data_directories
|
||||||
|
emit runtime_unique runtime_unique
|
||||||
|
emit runtime_build runtime_build
|
||||||
|
emit storage_mount storage_mount
|
||||||
|
emit pgdata_mount pgdata_mount
|
||||||
|
emit backup_build backup_build
|
||||||
|
REMOTE
|
||||||
|
)"
|
||||||
|
while IFS='=' read -r result_name result_value; do
|
||||||
|
case "$result_name" in
|
||||||
|
repository_root|repository_writable|env_readable|data_directories|runtime_unique|runtime_build|storage_mount|pgdata_mount|backup_build)
|
||||||
|
[[ "$result_value" == "PASS" || "$result_value" == "FAIL" ]] \
|
||||||
|
&& CHECK_RESULTS["$result_name"]="$result_value"
|
||||||
|
;;
|
||||||
|
esac
|
||||||
|
done <<<"$remote_results"
|
||||||
|
fi
|
||||||
|
|
||||||
|
refresh_output=""
|
||||||
|
refresh_status=255
|
||||||
|
if deployment_contract_safe_ssh_target "$DEPLOY_REFRESH_SERVER"; then
|
||||||
|
refresh_output="$(ssh "$DEPLOY_REFRESH_SERVER" probe 2>/dev/null)"
|
||||||
|
refresh_status=$?
|
||||||
|
fi
|
||||||
|
if [[ "$refresh_status" -ne 255 ]]; then
|
||||||
|
CHECK_RESULTS[root_ssh_identity]="PASS"
|
||||||
|
fi
|
||||||
|
if [[ "$refresh_status" -eq 0 \
|
||||||
|
&& "$refresh_output" == "[deployment-contract-refresh][PASS] probe" ]]; then
|
||||||
|
CHECK_RESULTS[refresh_program]="PASS"
|
||||||
|
fi
|
||||||
|
|
||||||
|
failure_count=0
|
||||||
|
for check_name in "${CHECK_NAMES[@]}"; do
|
||||||
|
printf '[deployment-preflight] %s %s\n' "$check_name" "${CHECK_RESULTS[$check_name]}"
|
||||||
|
if [[ "${CHECK_RESULTS[$check_name]}" != "PASS" ]]; then
|
||||||
|
failure_count=$((failure_count + 1))
|
||||||
|
fi
|
||||||
|
done
|
||||||
|
printf '[deployment-preflight] summary PASS=%s FAIL=%s\n' \
|
||||||
|
"$((${#CHECK_NAMES[@]} - failure_count))" "$failure_count"
|
||||||
|
[[ "$failure_count" -eq 0 ]]
|
||||||
@@ -71,7 +71,9 @@
|
|||||||
- [ ] 本地工作树 clean，目标 commit 已进入 upstream；部署日志中的 commit 与 `/ready.data.buildSha` 一致
|
- [ ] 本地工作树 clean，目标 commit 已进入 upstream；部署日志中的 commit 与 `/ready.data.buildSha` 一致
|
||||||
- [ ] **EXTERNAL_GATED**：操作人已在边缘暂停 `/api/auth/forgot-password`、`/api/auth/reset-password`、`/api/auth/email-login/code`、`/api/auth/email-login/verify`；部署脚本不会自动修改边缘配置，未人工确认四端点关闭不得发布，配置预检、迁移、新应用和 smoke 全部通过后才人工恢复
|
- [ ] **EXTERNAL_GATED**：操作人已在边缘暂停 `/api/auth/forgot-password`、`/api/auth/reset-password`、`/api/auth/email-login/code`、`/api/auth/email-login/verify`；部署脚本不会自动修改边缘配置，未人工确认四端点关闭不得发布，配置预检、迁移、新应用和 smoke 全部通过后才人工恢复
|
||||||
- [ ] 标准路径已成功刷新 upstream；若因受控离线恢复使用 unpublished override，日志包含批准原因码和 `UNPUBLISHED-OVERRIDE` 标识
|
- [ ] 标准路径已成功刷新 upstream；若因受控离线恢复使用 unpublished override，日志包含批准原因码和 `UNPUBLISHED-OVERRIDE` 标识
|
||||||
- [ ] 已有部署通过 `DEPLOY_SERVER=<ssh-target> bash deploy/trialrun/deploy-existing.sh` 执行；首次初始化才使用完整显式参数的 `deploy.sh`
|
- [ ] 已有部署通过 `DEPLOY_SERVER=<deploy-ssh-target> DEPLOY_REFRESH_SERVER=<forced-command-ssh-target> bash deploy/trialrun/deploy-existing.sh` 执行；首次初始化才使用完整显式参数的 `deploy.sh`
|
||||||
|
- [ ] forced-command 使用独立专用 key，同一 key 不存在不受限 root 授权；root-owned target 只接受精确 `probe/refresh`，不调用部署用户可写仓库脚本
|
||||||
|
- [ ] 13 项聚合体检在任何 mkdir/rsync 前全部执行并汇总；报告只有固定检查名与 PASS/FAIL，任一失败均阻断同步
|
||||||
- [ ] 未从 Compose `working_dir` 猜仓库根目录；同步前 preflight 已确认仓库根标识、普通远端 `.env`、PG_VERSION 与附件目录，且该检查发生在 rsync 之前
|
- [ ] 未从 Compose `working_dir` 猜仓库根目录；同步前 preflight 已确认仓库根标识、普通远端 `.env`、PG_VERSION 与附件目录，且该检查发生在 rsync 之前
|
||||||
- [ ] 脚本自动注入完整后端 SHA 与 12 位前端 SHA，并完成构建、init 与 smoke
|
- [ ] 脚本自动注入完整后端 SHA 与 12 位前端 SHA，并完成构建、init 与 smoke
|
||||||
- [ ] 构建日志显示 lockfile 未变化时 client/server 依赖层使用缓存；build SHA 变化没有触发 `apk add` 或两次 `npm ci`
|
- [ ] 构建日志显示 lockfile 未变化时 client/server 依赖层使用缓存；build SHA 变化没有触发 `apk add` 或两次 `npm ci`
|
||||||
@@ -90,13 +92,15 @@
|
|||||||
- [ ] 若真实 OCR 未配置，已使用 `OCR_PROVIDER=off` 而不是 mock，并确认实际作业拓扑中的 server/worker 已重建；真实票据由财务查看 PDF 原件人工核对
|
- [ ] 若真实 OCR 未配置，已使用 `OCR_PROVIDER=off` 而不是 mock，并确认实际作业拓扑中的 server/worker 已重建；真实票据由财务查看 PDF 原件人工核对
|
||||||
- [ ] 若本次包含前端体验修复，已在容器内确认 `/app/client/dist` 含关键 test id 或公网 asset hash 已更新
|
- [ ] 若本次包含前端体验修复，已在容器内确认 `/app/client/dist` 含关键 test id 或公网 asset hash 已更新
|
||||||
- [ ] 公网 bundle 扫描确认包含匹配的 12 位构建标识，且不包含完整提交 SHA
|
- [ ] 公网 bundle 扫描确认包含匹配的 12 位构建标识，且不包含完整提交 SHA
|
||||||
|
- [ ] wrapper 在应用与公网检查成功后自动刷新四个固定 service 的 build drop-in；四个最终 SHA 与活动容器一致，四个 timer 均 enabled/active
|
||||||
|
- [ ] 若出现 `DEPLOYED_BUT_CONTRACT_REFRESH_FAILED`，已按 root 身份、程序可用性、unit SHA、timer 状态四类处理，并将本次发布标记为“应用已部署但合同未收口”，未误报完整成功
|
||||||
- [ ] 登录验证已由 live smoke 与 release gate 覆盖；仅首次接入或第三方后台配置变更时补做人工确认
|
- [ ] 登录验证已由 live smoke 与 release gate 覆盖；仅首次接入或第三方后台配置变更时补做人工确认
|
||||||
|
|
||||||
## C. 发布后（Post-check）
|
## C. 发布后（Post-check）
|
||||||
- [ ] 公网 live smoke（只读，不做写操作）：`bash scripts/qa/run-web-gates.sh smoke-live`
|
- [ ] 公网 live smoke（只读，不做写操作）：`bash scripts/qa/run-web-gates.sh smoke-live`
|
||||||
- [ ] 若本地已配置 `LIVE_SMOKE_*`，确认 `bash deploy/trialrun/deploy.sh` 输出中包含 live smoke 结果
|
- [ ] 若本地已配置 `LIVE_SMOKE_*`，确认 `bash deploy/trialrun/deploy.sh` 输出中包含 live smoke 结果
|
||||||
- [ ] 已确认 release gate / live smoke 结果为通过；关键链路不再作为每次发布的手工必做项
|
- [ ] 已确认 release gate / live smoke 结果为通过；关键链路不再作为每次发布的手工必做项
|
||||||
- [ ] 若自动备份 units 已安装，已脱敏核对其冻结的 Compose project、build、PGDATA、附件宿主挂载与当前唯一活动 server/db；任一项变化均已用现有安装器按完整合同刷新，三个 timer 仍为 enabled/active，未直接手改 unit
|
- [ ] 已脱敏核对 automatic-backup/WARCH3 四个 service 的冻结 Compose/build/PGDATA/附件合同与当前唯一活动 server/db；build 由 wrapper 的固定 drop-in 自动刷新，路径或完整合同变化才使用现有 installer 人工修复，四个 timer 均 enabled/active
|
||||||
- [ ] 查看 server 日志（无异常错误风暴）：
|
- [ ] 查看 server 日志（无异常错误风暴）：
|
||||||
- [ ] `docker-compose -p "$COMPOSE_PROJECT_NAME" -f deploy/trialrun/docker-compose.yml --env-file deploy/trialrun/.env logs --tail=200 server`
|
- [ ] `docker-compose -p "$COMPOSE_PROJECT_NAME" -f deploy/trialrun/docker-compose.yml --env-file deploy/trialrun/.env logs --tail=200 server`
|
||||||
- [ ] 已确认 smoke 账号为独立账号（`smoke-admin` / `smoke-employee`），且不参与真实业务
|
- [ ] 已确认 smoke 账号为独立账号（`smoke-admin` / `smoke-employee`），且不参与真实业务
|
||||||
|
|||||||
@@ -3,37 +3,35 @@ set -euo pipefail
|
|||||||
|
|
||||||
ROOT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")/../.." && pwd)"
|
ROOT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")/../.." && pwd)"
|
||||||
DEPLOY_SERVER="${DEPLOY_SERVER:-}"
|
DEPLOY_SERVER="${DEPLOY_SERVER:-}"
|
||||||
|
DEPLOY_REFRESH_SERVER="${DEPLOY_REFRESH_SERVER:-}"
|
||||||
DEPLOY_CONTRACT_UNIT="${DEPLOY_CONTRACT_UNIT:-hyb-oa-automatic-backup-daily.service}"
|
DEPLOY_CONTRACT_UNIT="${DEPLOY_CONTRACT_UNIT:-hyb-oa-automatic-backup-daily.service}"
|
||||||
|
|
||||||
|
# shellcheck source=deployment-contract-lib.sh
|
||||||
|
source "$ROOT_DIR/deploy/trialrun/deployment-contract-lib.sh"
|
||||||
|
|
||||||
fail() {
|
fail() {
|
||||||
echo "[deploy-existing][ERROR] $1" >&2
|
echo "[deploy-existing][ERROR] $1" >&2
|
||||||
exit 1
|
exit 1
|
||||||
}
|
}
|
||||||
|
|
||||||
[[ "$DEPLOY_SERVER" =~ ^([A-Za-z0-9._-]+@)?[A-Za-z0-9._-]+$ ]] \
|
deployment_contract_safe_ssh_target "$DEPLOY_SERVER" \
|
||||||
|| fail "必须显式提供安全的 DEPLOY_SERVER"
|
|| fail "必须显式提供安全的 DEPLOY_SERVER"
|
||||||
[[ "$DEPLOY_CONTRACT_UNIT" =~ ^[A-Za-z0-9_.@-]+\.service$ ]] \
|
deployment_contract_safe_ssh_target "$DEPLOY_REFRESH_SERVER" \
|
||||||
|
|| fail "必须显式提供安全的 DEPLOY_REFRESH_SERVER"
|
||||||
|
deployment_contract_safe_service "$DEPLOY_CONTRACT_UNIT" \
|
||||||
|| fail "DEPLOY_CONTRACT_UNIT 格式无效"
|
|| fail "DEPLOY_CONTRACT_UNIT 格式无效"
|
||||||
command -v ssh >/dev/null 2>&1 || fail "缺少 ssh"
|
command -v ssh >/dev/null 2>&1 || fail "缺少 ssh"
|
||||||
|
|
||||||
unit_environment="$(ssh "$DEPLOY_SERVER" \
|
|
||||||
"systemctl show '${DEPLOY_CONTRACT_UNIT}' -p Environment --value")"
|
|
||||||
[[ -n "$unit_environment" ]] || fail "无法读取既有部署合同 unit"
|
|
||||||
|
|
||||||
declare -A contract=()
|
declare -A contract=()
|
||||||
for entry in $unit_environment; do
|
echo "[deploy-existing] 开始只读聚合体检"
|
||||||
key="${entry%%=*}"
|
DEPLOY_SERVER="$DEPLOY_SERVER" \
|
||||||
value="${entry#*=}"
|
DEPLOY_REFRESH_SERVER="$DEPLOY_REFRESH_SERVER" \
|
||||||
case "$key" in
|
DEPLOY_CONTRACT_UNIT="$DEPLOY_CONTRACT_UNIT" \
|
||||||
REMOTE_DIR|REMOTE_ENV_FILE|COMPOSE_FILE|COMPOSE_PROJECT_NAME|POSTGRES_DATA_DIR|STORAGE_HOST_DIR)
|
bash "$ROOT_DIR/deploy/trialrun/check-existing-deployment.sh" \
|
||||||
contract["$key"]="$value"
|
|| fail "部署前聚合体检未通过"
|
||||||
;;
|
|
||||||
esac
|
|
||||||
done
|
|
||||||
|
|
||||||
for key in REMOTE_DIR REMOTE_ENV_FILE COMPOSE_FILE COMPOSE_PROJECT_NAME POSTGRES_DATA_DIR STORAGE_HOST_DIR; do
|
deployment_contract_load "$DEPLOY_SERVER" "$DEPLOY_CONTRACT_UNIT" contract \
|
||||||
[[ -n "${contract[$key]:-}" ]] || fail "既有部署合同缺少 $key"
|
|| fail "无法读取完整既有部署合同"
|
||||||
done
|
|
||||||
|
|
||||||
echo "[deploy-existing] 已读取受控 unit 合同；路径与标识不回显"
|
echo "[deploy-existing] 已读取受控 unit 合同；路径与标识不回显"
|
||||||
DEPLOY_SERVER="$DEPLOY_SERVER" \
|
DEPLOY_SERVER="$DEPLOY_SERVER" \
|
||||||
@@ -44,5 +42,24 @@ COMPOSE_PROJECT_NAME="${contract[COMPOSE_PROJECT_NAME]}" \
|
|||||||
POSTGRES_DATA_DIR="${contract[POSTGRES_DATA_DIR]}" \
|
POSTGRES_DATA_DIR="${contract[POSTGRES_DATA_DIR]}" \
|
||||||
STORAGE_HOST_DIR="${contract[STORAGE_HOST_DIR]}" \
|
STORAGE_HOST_DIR="${contract[STORAGE_HOST_DIR]}" \
|
||||||
DEPLOY_DATA_MODE=existing \
|
DEPLOY_DATA_MODE=existing \
|
||||||
|
DEPLOY_DEFER_FINAL_SUCCESS=1 \
|
||||||
SYNC_ENV=0 \
|
SYNC_ENV=0 \
|
||||||
bash "$ROOT_DIR/deploy/trialrun/deploy.sh"
|
bash "$ROOT_DIR/deploy/trialrun/deploy.sh"
|
||||||
|
|
||||||
|
refresh_output=""
|
||||||
|
refresh_status=0
|
||||||
|
if refresh_output="$(ssh "$DEPLOY_REFRESH_SERVER" refresh 2>/dev/null)"; then
|
||||||
|
if [[ "$refresh_output" != "[deployment-contract-refresh][PASS] refreshed" ]]; then
|
||||||
|
refresh_status=127
|
||||||
|
fi
|
||||||
|
else
|
||||||
|
refresh_status=$?
|
||||||
|
fi
|
||||||
|
|
||||||
|
refresh_category="$(deployment_contract_refresh_category "$refresh_status" "$refresh_output")"
|
||||||
|
if [[ "$refresh_category" != "success" ]]; then
|
||||||
|
echo "[deploy-existing][DEPLOYED_BUT_CONTRACT_REFRESH_FAILED] category=${refresh_category}" >&2
|
||||||
|
exit 1
|
||||||
|
fi
|
||||||
|
|
||||||
|
echo "[deploy-existing][PASS] 应用发布与备份合同刷新完整通过"
|
||||||
|
|||||||
+10
-1
@@ -15,6 +15,7 @@ DEPLOY_DATA_MODE="${DEPLOY_DATA_MODE:-}"
|
|||||||
ENABLE_WORKER="${ENABLE_WORKER:-0}"
|
ENABLE_WORKER="${ENABLE_WORKER:-0}"
|
||||||
ENABLE_MINIO="${ENABLE_MINIO:-0}"
|
ENABLE_MINIO="${ENABLE_MINIO:-0}"
|
||||||
SYNC_ENV="${SYNC_ENV:-0}"
|
SYNC_ENV="${SYNC_ENV:-0}"
|
||||||
|
DEPLOY_DEFER_FINAL_SUCCESS="${DEPLOY_DEFER_FINAL_SUCCESS:-0}"
|
||||||
COMPOSE_FILE="deploy/trialrun/docker-compose.yml"
|
COMPOSE_FILE="deploy/trialrun/docker-compose.yml"
|
||||||
LOCAL_ENV_FILE="deploy/trialrun/.env"
|
LOCAL_ENV_FILE="deploy/trialrun/.env"
|
||||||
ENV_CONTRACT_SCRIPT="deploy/trialrun/validate-env-contract.js"
|
ENV_CONTRACT_SCRIPT="deploy/trialrun/validate-env-contract.js"
|
||||||
@@ -92,6 +93,10 @@ if [[ "$DEPLOY_DATA_MODE" != "existing" && "$DEPLOY_DATA_MODE" != "initialize" ]
|
|||||||
echo "[ERROR] DEPLOY_DATA_MODE 必须显式设为 existing 或 initialize" >&2
|
echo "[ERROR] DEPLOY_DATA_MODE 必须显式设为 existing 或 initialize" >&2
|
||||||
exit 1
|
exit 1
|
||||||
fi
|
fi
|
||||||
|
if [[ "$DEPLOY_DEFER_FINAL_SUCCESS" != "0" && "$DEPLOY_DEFER_FINAL_SUCCESS" != "1" ]]; then
|
||||||
|
echo "[ERROR] DEPLOY_DEFER_FINAL_SUCCESS 只能为 0 或 1" >&2
|
||||||
|
exit 1
|
||||||
|
fi
|
||||||
if paths_overlap "$REMOTE_DIR" "$POSTGRES_DATA_DIR"; then
|
if paths_overlap "$REMOTE_DIR" "$POSTGRES_DATA_DIR"; then
|
||||||
echo "[ERROR] POSTGRES_DATA_DIR 不得与 REMOTE_DIR 同目录或相互嵌套，避免源码同步触碰数据库" >&2
|
echo "[ERROR] POSTGRES_DATA_DIR 不得与 REMOTE_DIR 同目录或相互嵌套，避免源码同步触碰数据库" >&2
|
||||||
exit 1
|
exit 1
|
||||||
@@ -356,7 +361,11 @@ PUBLIC_EXPECTED_BUILD_SHA="$SOURCE_SHORT_COMMIT" \
|
|||||||
PUBLIC_FORBIDDEN_BUILD_SHA="$SOURCE_COMMIT" \
|
PUBLIC_FORBIDDEN_BUILD_SHA="$SOURCE_COMMIT" \
|
||||||
node "${ROOT_DIR}/scripts/qa/check-public-bundle-copy.mjs" "$PUBLIC_BASE_URL"
|
node "${ROOT_DIR}/scripts/qa/check-public-bundle-copy.mjs" "$PUBLIC_BASE_URL"
|
||||||
|
|
||||||
echo ">>> 10. 部署完成"
|
if [[ "$DEPLOY_DEFER_FINAL_SUCCESS" == "1" ]]; then
|
||||||
|
echo ">>> 10. 应用发布检查完成，等待外部备份合同刷新"
|
||||||
|
else
|
||||||
|
echo ">>> 10. 部署完成"
|
||||||
|
fi
|
||||||
echo " 来源提交: ${SOURCE_COMMIT}"
|
echo " 来源提交: ${SOURCE_COMMIT}"
|
||||||
echo " 已自动执行：契约校验 + /ready + external deps smoke + runtime smoke + 公网前端资源文案扫描"
|
echo " 已自动执行：契约校验 + /ready + external deps smoke + runtime smoke + 公网前端资源文案扫描"
|
||||||
echo " 若本地已配置 LIVE_SMOKE_*，部署后会自动补跑网页 live smoke"
|
echo " 若本地已配置 LIVE_SMOKE_*，部署后会自动补跑网页 live smoke"
|
||||||
|
|||||||
@@ -0,0 +1,86 @@
|
|||||||
|
#!/usr/bin/env bash
|
||||||
|
|
||||||
|
DEPLOY_CONTRACT_KEYS=(
|
||||||
|
REMOTE_DIR
|
||||||
|
REMOTE_ENV_FILE
|
||||||
|
COMPOSE_FILE
|
||||||
|
COMPOSE_PROJECT_NAME
|
||||||
|
POSTGRES_DATA_DIR
|
||||||
|
STORAGE_HOST_DIR
|
||||||
|
)
|
||||||
|
|
||||||
|
deployment_contract_safe_ssh_target() {
|
||||||
|
[[ "${1:-}" =~ ^([A-Za-z0-9._-]+@)?[A-Za-z0-9._-]+$ ]]
|
||||||
|
}
|
||||||
|
|
||||||
|
deployment_contract_safe_service() {
|
||||||
|
[[ "${1:-}" =~ ^[A-Za-z0-9_.@-]+\.service$ ]]
|
||||||
|
}
|
||||||
|
|
||||||
|
deployment_contract_safe_absolute_path() {
|
||||||
|
local value="${1:-}"
|
||||||
|
[[ -n "$value" && "$value" == /* && "$value" =~ ^/[A-Za-z0-9._/-]+$ \
|
||||||
|
&& "$value" != "/" && "$value" != */ && "$value" != *"//"* \
|
||||||
|
&& "/$value/" != *"/../"* && "/$value/" != *"/./"* ]]
|
||||||
|
}
|
||||||
|
|
||||||
|
deployment_contract_safe_compose_file() {
|
||||||
|
local value="${1:-}"
|
||||||
|
[[ -n "$value" && "$value" != /* && "$value" =~ ^[A-Za-z0-9._/-]+$ \
|
||||||
|
&& "/$value/" != *"/../"* && "/$value/" != *"/./"* ]]
|
||||||
|
}
|
||||||
|
|
||||||
|
deployment_contract_safe_project() {
|
||||||
|
[[ "${1:-}" =~ ^[a-z0-9][a-z0-9_-]*$ ]]
|
||||||
|
}
|
||||||
|
|
||||||
|
deployment_contract_load() {
|
||||||
|
local target="$1"
|
||||||
|
local unit="$2"
|
||||||
|
local destination_name="$3"
|
||||||
|
local unit_environment key value
|
||||||
|
local -n destination="$destination_name"
|
||||||
|
|
||||||
|
destination=()
|
||||||
|
deployment_contract_safe_ssh_target "$target" || return 1
|
||||||
|
deployment_contract_safe_service "$unit" || return 1
|
||||||
|
unit_environment="$(ssh "$target" "systemctl show '${unit}' -p Environment --value" 2>/dev/null)" \
|
||||||
|
|| return 1
|
||||||
|
[[ -n "$unit_environment" ]] || return 1
|
||||||
|
|
||||||
|
for entry in $unit_environment; do
|
||||||
|
key="${entry%%=*}"
|
||||||
|
value="${entry#*=}"
|
||||||
|
case "$key" in
|
||||||
|
REMOTE_DIR|REMOTE_ENV_FILE|COMPOSE_FILE|COMPOSE_PROJECT_NAME|POSTGRES_DATA_DIR|STORAGE_HOST_DIR)
|
||||||
|
destination["$key"]="$value"
|
||||||
|
;;
|
||||||
|
esac
|
||||||
|
done
|
||||||
|
|
||||||
|
for key in "${DEPLOY_CONTRACT_KEYS[@]}"; do
|
||||||
|
[[ -n "${destination[$key]:-}" ]] || return 1
|
||||||
|
done
|
||||||
|
deployment_contract_safe_absolute_path "${destination[REMOTE_DIR]}" || return 1
|
||||||
|
deployment_contract_safe_absolute_path "${destination[REMOTE_ENV_FILE]}" || return 1
|
||||||
|
deployment_contract_safe_absolute_path "${destination[POSTGRES_DATA_DIR]}" || return 1
|
||||||
|
deployment_contract_safe_absolute_path "${destination[STORAGE_HOST_DIR]}" || return 1
|
||||||
|
deployment_contract_safe_compose_file "${destination[COMPOSE_FILE]}" || return 1
|
||||||
|
deployment_contract_safe_project "${destination[COMPOSE_PROJECT_NAME]}" || return 1
|
||||||
|
}
|
||||||
|
|
||||||
|
deployment_contract_refresh_category() {
|
||||||
|
local status="${1:-1}"
|
||||||
|
local output="${2:-}"
|
||||||
|
if [[ "$status" == "0" \
|
||||||
|
&& "$output" == "[deployment-contract-refresh][PASS] refreshed" ]]; then
|
||||||
|
printf 'success'
|
||||||
|
return 0
|
||||||
|
fi
|
||||||
|
case "$status" in
|
||||||
|
255) printf 'root_identity_unavailable' ;;
|
||||||
|
126|127) printf 'refresh_program_unavailable' ;;
|
||||||
|
43) printf 'timer_not_active' ;;
|
||||||
|
*) printf 'unit_build_not_refreshed' ;;
|
||||||
|
esac
|
||||||
|
}
|
||||||
@@ -0,0 +1,166 @@
|
|||||||
|
#!/usr/bin/env bash
|
||||||
|
set -euo pipefail
|
||||||
|
|
||||||
|
ROOT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")/../.." && pwd)"
|
||||||
|
SOURCE="$ROOT_DIR/scripts/refresh-deployment-contract.sh"
|
||||||
|
PUBLIC_KEY_FILE="${DEPLOY_REFRESH_PUBLIC_KEY_FILE:-}"
|
||||||
|
MARKER="# hyb-oa-managed-deployment-contract-refresh"
|
||||||
|
MODE="${1:-install}"
|
||||||
|
INSTALL_TEST_MODE=0
|
||||||
|
ROOT_PREFIX=""
|
||||||
|
|
||||||
|
fail() {
|
||||||
|
echo "[deployment-contract-refresh-installer][ERROR] $1" >&2
|
||||||
|
exit 1
|
||||||
|
}
|
||||||
|
|
||||||
|
if [[ "$(id -u)" != "0" ]]; then
|
||||||
|
test_root="${HYB_CONTRACT_REFRESH_INSTALL_TEST_ROOT:-}"
|
||||||
|
if [[ "$test_root" =~ ^/tmp/[A-Za-z0-9._/-]+$ && "$test_root" != */ \
|
||||||
|
&& "$test_root" != *"//"* && "/$test_root/" != *"/../"* ]]; then
|
||||||
|
INSTALL_TEST_MODE=1
|
||||||
|
ROOT_PREFIX="$test_root"
|
||||||
|
else
|
||||||
|
fail "root privileges are required"
|
||||||
|
fi
|
||||||
|
fi
|
||||||
|
TARGET="${ROOT_PREFIX}/usr/local/sbin/hyb-oa-deployment-contract-refresh"
|
||||||
|
AUTHORIZED_KEYS="${ROOT_PREFIX}/root/.ssh/authorized_keys"
|
||||||
|
SSH_DIRECTORY="${ROOT_PREFIX}/root/.ssh"
|
||||||
|
[[ -f "$SOURCE" && ! -L "$SOURCE" ]] || fail "reviewed refresh source is missing"
|
||||||
|
|
||||||
|
remove_managed_entry() {
|
||||||
|
python3 - "$AUTHORIZED_KEYS" "$MARKER" "$TARGET" "$INSTALL_TEST_MODE" <<'PY'
|
||||||
|
import os, sys, tempfile
|
||||||
|
path, marker, target, test_mode = sys.argv[1:]
|
||||||
|
if not os.path.exists(path):
|
||||||
|
raise SystemExit(0)
|
||||||
|
with open(path, encoding="utf-8") as handle:
|
||||||
|
lines = handle.read().splitlines()
|
||||||
|
result = []
|
||||||
|
index = 0
|
||||||
|
forced_prefix = f'command="{target}",no-agent-forwarding,no-port-forwarding,no-X11-forwarding,no-pty,no-user-rc '
|
||||||
|
while index < len(lines):
|
||||||
|
line = lines[index]
|
||||||
|
if line == marker:
|
||||||
|
if index + 1 >= len(lines) or not lines[index + 1].startswith(forced_prefix):
|
||||||
|
raise SystemExit("managed forced-command marker is malformed")
|
||||||
|
index += 2
|
||||||
|
continue
|
||||||
|
result.append(line)
|
||||||
|
index += 1
|
||||||
|
directory = os.path.dirname(path)
|
||||||
|
fd, temp_path = tempfile.mkstemp(prefix=".authorized_keys.", dir=directory, text=True)
|
||||||
|
try:
|
||||||
|
with os.fdopen(fd, "w", encoding="utf-8") as handle:
|
||||||
|
if result:
|
||||||
|
handle.write("\n".join(result) + "\n")
|
||||||
|
handle.flush()
|
||||||
|
os.fsync(handle.fileno())
|
||||||
|
os.chmod(temp_path, 0o600)
|
||||||
|
if test_mode != "1":
|
||||||
|
os.chown(temp_path, 0, 0)
|
||||||
|
os.replace(temp_path, path)
|
||||||
|
finally:
|
||||||
|
if os.path.exists(temp_path):
|
||||||
|
os.unlink(temp_path)
|
||||||
|
PY
|
||||||
|
}
|
||||||
|
|
||||||
|
install_entry() {
|
||||||
|
[[ -n "$PUBLIC_KEY_FILE" && -f "$PUBLIC_KEY_FILE" && ! -L "$PUBLIC_KEY_FILE" ]] \
|
||||||
|
|| fail "DEPLOY_REFRESH_PUBLIC_KEY_FILE must name a regular public-key file"
|
||||||
|
key_line="$(sed -e '/^[[:space:]]*#/d' -e '/^[[:space:]]*$/d' "$PUBLIC_KEY_FILE")"
|
||||||
|
[[ "$(printf '%s\n' "$key_line" | wc -l)" -eq 1 ]] \
|
||||||
|
|| fail "public-key file must contain exactly one key"
|
||||||
|
read -r key_type key_blob key_comment <<<"$key_line"
|
||||||
|
[[ "$key_type" =~ ^(ssh-ed25519|ecdsa-sha2-nistp(256|384|521)|ssh-rsa)$ \
|
||||||
|
&& "$key_blob" =~ ^[A-Za-z0-9+/]+={0,3}$ ]] \
|
||||||
|
|| fail "public key format is invalid"
|
||||||
|
|
||||||
|
if [[ "$INSTALL_TEST_MODE" == "0" ]]; then
|
||||||
|
install -d -o root -g root -m 0700 "$SSH_DIRECTORY"
|
||||||
|
else
|
||||||
|
install -d -m 0700 "$SSH_DIRECTORY"
|
||||||
|
fi
|
||||||
|
[[ ! -L "$SSH_DIRECTORY" && ! -L "$AUTHORIZED_KEYS" ]] \
|
||||||
|
|| fail "root SSH paths must not be symbolic links"
|
||||||
|
touch "$AUTHORIZED_KEYS"
|
||||||
|
if [[ "$INSTALL_TEST_MODE" == "0" ]]; then
|
||||||
|
chown root:root "$AUTHORIZED_KEYS"
|
||||||
|
fi
|
||||||
|
chmod 0600 "$AUTHORIZED_KEYS"
|
||||||
|
|
||||||
|
PUBLIC_KEY_BLOB="$key_blob" python3 - "$AUTHORIZED_KEYS" "$MARKER" "$TARGET" "$key_type" "$key_line" "$INSTALL_TEST_MODE" <<'PY'
|
||||||
|
import os, re, sys, tempfile
|
||||||
|
path, marker, target, key_type, key_line, test_mode = sys.argv[1:]
|
||||||
|
key_blob = os.environ["PUBLIC_KEY_BLOB"]
|
||||||
|
with open(path, encoding="utf-8") as handle:
|
||||||
|
lines = handle.read().splitlines()
|
||||||
|
unmanaged = []
|
||||||
|
kept = []
|
||||||
|
pattern = re.compile(r"(?:^|\s)(?:ssh-ed25519|ecdsa-sha2-nistp(?:256|384|521)|ssh-rsa)\s+([A-Za-z0-9+/=]+)(?:\s|$)")
|
||||||
|
forced_prefix = f'command="{target}",no-agent-forwarding,no-port-forwarding,no-X11-forwarding,no-pty,no-user-rc '
|
||||||
|
index = 0
|
||||||
|
while index < len(lines):
|
||||||
|
line = lines[index]
|
||||||
|
if line == marker:
|
||||||
|
if index + 1 >= len(lines) or not lines[index + 1].startswith(forced_prefix):
|
||||||
|
raise SystemExit("managed forced-command marker is malformed")
|
||||||
|
index += 2
|
||||||
|
continue
|
||||||
|
match = pattern.search(line)
|
||||||
|
if match and match.group(1) == key_blob:
|
||||||
|
unmanaged.append(line)
|
||||||
|
kept.append(line)
|
||||||
|
index += 1
|
||||||
|
if unmanaged:
|
||||||
|
raise SystemExit("dedicated key already has unmanaged or unrestricted root authorization")
|
||||||
|
forced = f'command="{target}",no-agent-forwarding,no-port-forwarding,no-X11-forwarding,no-pty,no-user-rc {key_line}'
|
||||||
|
kept.extend([marker, forced])
|
||||||
|
directory = os.path.dirname(path)
|
||||||
|
fd, temp_path = tempfile.mkstemp(prefix=".authorized_keys.", dir=directory, text=True)
|
||||||
|
try:
|
||||||
|
with os.fdopen(fd, "w", encoding="utf-8") as handle:
|
||||||
|
handle.write("\n".join(kept) + "\n")
|
||||||
|
handle.flush()
|
||||||
|
os.fsync(handle.fileno())
|
||||||
|
os.chmod(temp_path, 0o600)
|
||||||
|
if test_mode != "1":
|
||||||
|
os.chown(temp_path, 0, 0)
|
||||||
|
os.replace(temp_path, path)
|
||||||
|
finally:
|
||||||
|
if os.path.exists(temp_path):
|
||||||
|
os.unlink(temp_path)
|
||||||
|
PY
|
||||||
|
}
|
||||||
|
|
||||||
|
case "$MODE" in
|
||||||
|
install)
|
||||||
|
if [[ "$INSTALL_TEST_MODE" == "0" ]]; then
|
||||||
|
install -d -o root -g root -m 0755 "$(dirname "$TARGET")"
|
||||||
|
install -o root -g root -m 0755 "$SOURCE" "$TARGET"
|
||||||
|
[[ -f "$TARGET" && ! -L "$TARGET" \
|
||||||
|
&& "$(stat -c %u:%g:%a "$TARGET")" == "0:0:755" ]] \
|
||||||
|
|| fail "installed target ownership or mode is invalid"
|
||||||
|
probe_env=()
|
||||||
|
else
|
||||||
|
install -d -m 0755 "$(dirname "$TARGET")"
|
||||||
|
install -m 0755 "$SOURCE" "$TARGET"
|
||||||
|
[[ -f "$TARGET" && ! -L "$TARGET" && "$(stat -c %a "$TARGET")" == "755" ]] \
|
||||||
|
|| fail "installed target mode is invalid"
|
||||||
|
probe_env=(env HYB_CONTRACT_REFRESH_TEST_ROOT="$ROOT_PREFIX")
|
||||||
|
fi
|
||||||
|
"${probe_env[@]}" env SSH_ORIGINAL_COMMAND=probe "$TARGET" >/dev/null \
|
||||||
|
|| fail "installed target probe failed"
|
||||||
|
install_entry
|
||||||
|
echo "[deployment-contract-refresh-installer][PASS] forced command installed"
|
||||||
|
;;
|
||||||
|
uninstall)
|
||||||
|
remove_managed_entry
|
||||||
|
echo "[deployment-contract-refresh-installer][PASS] authorization removed; target retained"
|
||||||
|
;;
|
||||||
|
*)
|
||||||
|
fail "unsupported mode"
|
||||||
|
;;
|
||||||
|
esac
|
||||||
@@ -71,7 +71,7 @@ curl -fsS <shadow-ready-url>
|
|||||||
- 反向代理：`caddy` 正在运行，`oa.hyinda.com` 应反代到 `.env` 中 `APP_HOST_PORT` 对应的宿主回环端口。
|
- 反向代理：`caddy` 正在运行，`oa.hyinda.com` 应反代到 `.env` 中 `APP_HOST_PORT` 对应的宿主回环端口。
|
||||||
- 部署目录通过 `REMOTE_DIR` 显式传入。
|
- 部署目录通过 `REMOTE_DIR` 显式传入。
|
||||||
- 远端目录当前不是 git 工作树；发布应优先从本地执行 `bash deploy/trialrun/deploy.sh`，由脚本 `rsync` 到服务器
|
- 远端目录当前不是 git 工作树；发布应优先从本地执行 `bash deploy/trialrun/deploy.sh`，由脚本 `rsync` 到服务器
|
||||||
- 已有部署优先执行 `DEPLOY_SERVER=<ssh-target> bash deploy/trialrun/deploy-existing.sh`；包装脚本只从受控 automatic-backup systemd unit 读取 allowlist 参数且不回显路径。不得把 Compose `com.docker.compose.project.working_dir` 当仓库根目录，该标签可能指向 `deploy/trialrun` 配置目录。
|
- 已有部署优先执行 `DEPLOY_SERVER=<deploy-ssh-target> DEPLOY_REFRESH_SERVER=<forced-command-ssh-target> bash deploy/trialrun/deploy-existing.sh`；包装脚本只从受控 automatic-backup systemd unit 读取 allowlist 参数且不回显路径。两个 SSH target 是独立身份：前者按部署用户执行同步/Compose，后者只能执行 root-owned forced-command 的 `probe/refresh`。不得把 Compose `com.docker.compose.project.working_dir` 当仓库根目录，该标签可能指向 `deploy/trialrun` 配置目录。
|
||||||
- 远端容器编排当前实际为 `docker-compose 1.29.2`；本 runbook 命令以 `docker-compose` 为准
|
- 远端容器编排当前实际为 `docker-compose 1.29.2`；本 runbook 命令以 `docker-compose` 为准
|
||||||
- 所有 Compose 命令必须显式使用同一个 `COMPOSE_PROJECT_NAME` 和 `-p`；不得依赖目录名推导项目身份，也不得写死容器名。
|
- 所有 Compose 命令必须显式使用同一个 `COMPOSE_PROJECT_NAME` 和 `-p`；不得依赖目录名推导项目身份，也不得写死容器名。
|
||||||
- 当前公网 `.env` 已确认：`APP_ORIGIN=https://oa.hyinda.com`、`COOKIE_SECURE=true`、`PUBLIC_SECURITY_HARDENING_ENABLED=true`
|
- 当前公网 `.env` 已确认：`APP_ORIGIN=https://oa.hyinda.com`、`COOKIE_SECURE=true`、`PUBLIC_SECURITY_HARDENING_ENABLED=true`
|
||||||
@@ -268,7 +268,7 @@ manifest 必须至少记录一个真实来源标识：活动 `/ready` 有构建
|
|||||||
- 生产与 ARM 都直接使用独立企微运维群机器人，不依赖 OA server、数据库或个人工号。Webhook 正文只由用户手工写入两台主机的 root-only `0600` 文件；仓库、普通 `.env`、systemd unit、日志和聊天只能出现受控文件路径，不能出现 URL 或 key。
|
- 生产与 ARM 都直接使用独立企微运维群机器人，不依赖 OA server、数据库或个人工号。Webhook 正文只由用户手工写入两台主机的 root-only `0600` 文件；仓库、普通 `.env`、systemd unit、日志和聊天只能出现受控文件路径，不能出现 URL 或 key。
|
||||||
- 失败告警只包含阶段、状态、finding 数和类别建议；原始命令诊断只保留在源端状态目录的 `0600` 最近失败日志。同一失败降噪；首次失败、状态恶化、新 finding、恢复通知和每季度首次恢复提醒才发送。新部署构建与最近恢复点 build SHA 不同只记 finding，缺失、损坏、超龄、未来时间或 Compose project 不同仍硬失败。
|
- 失败告警只包含阶段、状态、finding 数和类别建议；原始命令诊断只保留在源端状态目录的 `0600` 最近失败日志。同一失败降噪；首次失败、状态恶化、新 finding、恢复通知和每季度首次恢复提醒才发送。新部署构建与最近恢复点 build SHA 不同只记 finding，缺失、损坏、超龄、未来时间或 Compose project 不同仍硬失败。
|
||||||
|
|
||||||
自动备份 unit 会冻结安装时的 Compose project、build、PGDATA 和附件宿主挂载。每次发布、回滚或直接重建容器后，若这些 units 已安装，必须把 unit 中的四项身份与唯一活动 server/db 逐项脱敏核对；任一项变化都要使用现有安装器和完整当前合同刷新 units，再复核三个 timer 与全部运行身份，不能直接手改单个 unit，也不能等次日定时告警后再处理。`phase_stale_check_failed` 是 watchdog 外层阶段，它也可能表示读取成功状态前的运行身份检查失败；应先看安全子类别和 unit/活动容器身份，不能仅凭该阶段认定归档已经超龄或损坏。
|
自动备份 unit 会冻结安装时的 Compose project、build、PGDATA 和附件宿主挂载。已有部署入口在写前聚合核对这些身份，并在应用发布后通过 root-owned forced-command 只刷新 automatic-backup 三个 service 与 WARCH3 service 的 build drop-in，再验证四个 timer；路径、project、PGDATA、附件或完整 unit 合同变化仍必须用原安装器人工刷新，不能扩张 forced-command 的写入面。`phase_stale_check_failed` 是 watchdog 外层阶段，它也可能表示读取成功状态前的运行身份检查失败；应先看安全子类别和 unit/活动容器身份，不能仅凭该阶段认定归档已经超龄或损坏。
|
||||||
|
|
||||||
创建机器人后，由用户分别登录两台主机，在 root shell 中用静默输入把完整 Webhook 写入受控路径；不要把值作为命令参数、环境文件内容或聊天消息交给 Codex：
|
创建机器人后，由用户分别登录两台主机，在 root shell 中用静默输入把完整 Webhook 写入受控路径；不要把值作为命令参数、环境文件内容或聊天消息交给 Codex：
|
||||||
|
|
||||||
@@ -424,13 +424,17 @@ systemctl status hyb-oa-public-app-health.service --no-pager
|
|||||||
|
|
||||||
步骤：
|
步骤：
|
||||||
1) 先做备份（见 E14 脚本）
|
1) 先做备份（见 E14 脚本）
|
||||||
2) 已有部署从本地仓库执行 `DEPLOY_SERVER=<ssh-target> bash deploy/trialrun/deploy-existing.sh`（推荐）；它从受控 systemd unit 读取既有 `REMOTE_DIR`、env、Compose project 与数据目录合同，不从 Docker 标签猜路径，也不回显参数值。首次初始化或 unit 不存在时，才使用完整显式参数执行 `deploy.sh`。
|
2) 已有部署从本地仓库执行 `DEPLOY_SERVER=<deploy-ssh-target> DEPLOY_REFRESH_SERVER=<forced-command-ssh-target> bash deploy/trialrun/deploy-existing.sh`（推荐）；它从受控 systemd unit 读取既有 `REMOTE_DIR`、env、Compose project 与数据目录合同，不从 Docker 标签猜路径，也不回显参数值。首次初始化或 unit 不存在时，才使用完整显式参数执行 `deploy.sh`。
|
||||||
|
- 正式使用新版入口前，root 在单独获批窗口执行 `DEPLOY_REFRESH_PUBLIC_KEY_FILE=<dedicated-public-key-file> bash deploy/trialrun/install-deployment-contract-refresh.sh install`。专用 key 不得同时存在不受限 root 授权；installer 会安装 root-owned 自包含 target 和 forced-command 行，但 installer 本身不进入免密授权。
|
||||||
|
- 新版入口先运行 `check-existing-deployment.sh` 的 13 项只读聚合体检。报告只含固定项名与 PASS/FAIL；任何失败都在 mkdir/rsync 前阻断，且前项失败不会跳过后续独立检查。
|
||||||
|
- `DEPLOY_REFRESH_SERVER` 必须是只携带专用 forced-command key 的 SSH alias。不得用 `sudo -n true` 代替该身份检查，也不得复用仍具有不受限 root 登录能力的同一 key。
|
||||||
- `deploy.sh` 只同步已验证提交的 Git archive，不依赖远端 `git pull`，也不会把本地 ignored 文件带入发布。
|
- `deploy.sh` 只同步已验证提交的 Git archive，不依赖远端 `git pull`，也不会把本地 ignored 文件带入发布。
|
||||||
- 在任何 rsync 前，远端 preflight 必须确认 existing 目标同时含 `server/package.json` 与 Compose 文件、PGDATA 含 `PG_VERSION`、storage 存在，且 `SYNC_ENV=0` 的 `.env` 是普通文件。失败即停止，不能先同步再验证。
|
- 在任何 rsync 前，远端 preflight 必须确认 existing 目标同时含 `server/package.json` 与 Compose 文件、PGDATA 含 `PG_VERSION`、storage 存在，且 `SYNC_ENV=0` 的 `.env` 是普通文件。失败即停止，不能先同步再验证。
|
||||||
- `REMOTE_DIR` 必须与 PostgreSQL、附件宿主目录完全分离；位于同步树内的 `REMOTE_ENV_FILE` 按精确相对路径动态保护。
|
- `REMOTE_DIR` 必须与 PostgreSQL、附件宿主目录完全分离；位于同步树内的 `REMOTE_ENV_FILE` 按精确相对路径动态保护。
|
||||||
- 在现网目录统一任务中，新的 `COMPOSE_PROJECT_NAME`、`POSTGRES_DATA_DIR`、`STORAGE_HOST_DIR` 必须与活动数据目录复制在同一停机窗口启用；本地提交本身不授权切换。
|
- 在现网目录统一任务中，新的 `COMPOSE_PROJECT_NAME`、`POSTGRES_DATA_DIR`、`STORAGE_HOST_DIR` 必须与活动数据目录复制在同一停机窗口启用；本地提交本身不授权切换。
|
||||||
- 旧栈停止命令必须使用冻结阶段由 `docker compose ls` 实测的旧 project；不得拿新 project 名去停止旧容器。
|
- 旧栈停止命令必须使用冻结阶段由 `docker compose ls` 实测的旧 project；不得拿新 project 名去停止旧容器。
|
||||||
- `artifacts/` 和 `output/` 是运行期证据/截图目录，不作为源码发布输入，也不由 `rsync --delete` 清理远端历史证据。
|
- `artifacts/` 和 `output/` 是运行期证据/截图目录，不作为源码发布输入，也不由 `rsync --delete` 清理远端历史证据。
|
||||||
|
- 应用、`/ready`、smoke 和公网检查成功后，wrapper 才调用 forced-command `refresh`，以活动容器 build 原子刷新四个固定 service 的 systemd drop-in并验证四个 timer。失败时固定输出 `DEPLOYED_BUT_CONTRACT_REFRESH_FAILED`、退出非零，并明确表示应用已发布但合同未收口；不自动回滚应用。
|
||||||
3) 构建并重启：
|
3) 构建并重启：
|
||||||
```bash
|
```bash
|
||||||
docker-compose -p "$COMPOSE_PROJECT_NAME" -f deploy/trialrun/docker-compose.yml --env-file deploy/trialrun/.env up -d --build server
|
docker-compose -p "$COMPOSE_PROJECT_NAME" -f deploy/trialrun/docker-compose.yml --env-file deploy/trialrun/.env up -d --build server
|
||||||
@@ -468,7 +472,7 @@ curl -fsS "http://127.0.0.1:${SERVER_HOST_PORT##*:}/ready"
|
|||||||
重建后必须补做：
|
重建后必须补做：
|
||||||
- `smoke-external-deps.js`
|
- `smoke-external-deps.js`
|
||||||
- `smoke-trialrun-runtime.js`
|
- `smoke-trialrun-runtime.js`
|
||||||
- 若自动备份 units 已安装，核对其 Compose project、build、PGDATA、附件宿主挂载与当前唯一活动 server/db；不一致时用安装器完整刷新并复核三个 timer
|
- 已有部署由 wrapper 自动核对 Compose project、build、PGDATA、附件宿主挂载，并在发布后刷新 automatic-backup 三个 service 与 WARCH3 service 的 build drop-in，复核四个 timer。路径或完整 unit 合同变化仍用原 installer 人工修复，不让 forced-command 扩权处理
|
||||||
- 公网页面视觉检查：至少覆盖 `/invoice-pool`、`/reimbursements`、`/mobile/capture` 的 390px 视口
|
- 公网页面视觉检查：至少覆盖 `/invoice-pool`、`/reimbursements`、`/mobile/capture` 的 390px 视口
|
||||||
|
|
||||||
发布/回滚 Checklist：
|
发布/回滚 Checklist：
|
||||||
|
|||||||
@@ -21,6 +21,22 @@ const deployScriptPath = path.resolve(
|
|||||||
process.env.ENV_CONTRACT_DEPLOY_SCRIPT_PATH || path.join(rootDir, 'deploy/trialrun/deploy.sh'),
|
process.env.ENV_CONTRACT_DEPLOY_SCRIPT_PATH || path.join(rootDir, 'deploy/trialrun/deploy.sh'),
|
||||||
);
|
);
|
||||||
const deployExistingScriptPath = path.join(rootDir, 'deploy/trialrun/deploy-existing.sh');
|
const deployExistingScriptPath = path.join(rootDir, 'deploy/trialrun/deploy-existing.sh');
|
||||||
|
const deploymentContractLibraryPath = path.join(
|
||||||
|
rootDir,
|
||||||
|
'deploy/trialrun/deployment-contract-lib.sh',
|
||||||
|
);
|
||||||
|
const deploymentPreflightScriptPath = path.join(
|
||||||
|
rootDir,
|
||||||
|
'deploy/trialrun/check-existing-deployment.sh',
|
||||||
|
);
|
||||||
|
const deploymentRefreshInstallerPath = path.join(
|
||||||
|
rootDir,
|
||||||
|
'deploy/trialrun/install-deployment-contract-refresh.sh',
|
||||||
|
);
|
||||||
|
const deploymentRefreshHelperPath = path.join(
|
||||||
|
rootDir,
|
||||||
|
'scripts/refresh-deployment-contract.sh',
|
||||||
|
);
|
||||||
const remotePreflightScriptPath = path.join(rootDir, 'deploy/trialrun/preflight-remote.sh');
|
const remotePreflightScriptPath = path.join(rootDir, 'deploy/trialrun/preflight-remote.sh');
|
||||||
const dockerfilePath = path.join(rootDir, 'deploy/trialrun/Dockerfile');
|
const dockerfilePath = path.join(rootDir, 'deploy/trialrun/Dockerfile');
|
||||||
const timerScriptPath = path.resolve(
|
const timerScriptPath = path.resolve(
|
||||||
@@ -251,6 +267,10 @@ const runbookContent = readFile(runbookPath);
|
|||||||
const checklistContent = readFile(checklistPath);
|
const checklistContent = readFile(checklistPath);
|
||||||
const deployScriptContent = readFile(deployScriptPath);
|
const deployScriptContent = readFile(deployScriptPath);
|
||||||
const deployExistingScriptContent = readFile(deployExistingScriptPath);
|
const deployExistingScriptContent = readFile(deployExistingScriptPath);
|
||||||
|
const deploymentContractLibraryContent = readFile(deploymentContractLibraryPath);
|
||||||
|
const deploymentPreflightScriptContent = readFile(deploymentPreflightScriptPath);
|
||||||
|
const deploymentRefreshInstallerContent = readFile(deploymentRefreshInstallerPath);
|
||||||
|
const deploymentRefreshHelperContent = readFile(deploymentRefreshHelperPath);
|
||||||
const remotePreflightScriptContent = readFile(remotePreflightScriptPath);
|
const remotePreflightScriptContent = readFile(remotePreflightScriptPath);
|
||||||
const dockerfileContent = readFile(dockerfilePath);
|
const dockerfileContent = readFile(dockerfilePath);
|
||||||
const timerScriptContent = readFile(timerScriptPath);
|
const timerScriptContent = readFile(timerScriptPath);
|
||||||
@@ -557,7 +577,12 @@ if (deployScriptContent.indexOf('>>> 3. 同步前只读校验远端部署合同'
|
|||||||
}
|
}
|
||||||
|
|
||||||
for (const requiredFragment of [
|
for (const requiredFragment of [
|
||||||
'systemctl show',
|
'DEPLOY_REFRESH_SERVER',
|
||||||
|
'check-existing-deployment.sh',
|
||||||
|
'deployment_contract_load',
|
||||||
|
'DEPLOY_DEFER_FINAL_SUCCESS=1',
|
||||||
|
'ssh "$DEPLOY_REFRESH_SERVER" refresh',
|
||||||
|
'DEPLOYED_BUT_CONTRACT_REFRESH_FAILED',
|
||||||
'DEPLOY_DATA_MODE=existing',
|
'DEPLOY_DATA_MODE=existing',
|
||||||
'SYNC_ENV=0',
|
'SYNC_ENV=0',
|
||||||
'路径与标识不回显',
|
'路径与标识不回显',
|
||||||
@@ -567,6 +592,82 @@ for (const requiredFragment of [
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
for (const requiredFragment of [
|
||||||
|
'systemctl show',
|
||||||
|
'DEPLOY_CONTRACT_KEYS',
|
||||||
|
'REMOTE_DIR',
|
||||||
|
'REMOTE_ENV_FILE',
|
||||||
|
'COMPOSE_FILE',
|
||||||
|
'COMPOSE_PROJECT_NAME',
|
||||||
|
'POSTGRES_DATA_DIR',
|
||||||
|
'STORAGE_HOST_DIR',
|
||||||
|
'deployment_contract_refresh_category',
|
||||||
|
]) {
|
||||||
|
if (!deploymentContractLibraryContent.includes(requiredFragment)) {
|
||||||
|
errors.push(`deployment-contract-lib.sh 缺少共享部署合同: ${requiredFragment}`);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
for (const requiredFragment of [
|
||||||
|
'set -uo pipefail',
|
||||||
|
'CHECK_NAMES=(',
|
||||||
|
'repository_writable',
|
||||||
|
'env_readable',
|
||||||
|
'runtime_unique',
|
||||||
|
'storage_mount',
|
||||||
|
'pgdata_mount',
|
||||||
|
'backup_build',
|
||||||
|
'root_ssh_identity',
|
||||||
|
'refresh_program',
|
||||||
|
'summary PASS=',
|
||||||
|
]) {
|
||||||
|
if (!deploymentPreflightScriptContent.includes(requiredFragment)) {
|
||||||
|
errors.push(`check-existing-deployment.sh 缺少聚合体检合同: ${requiredFragment}`);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if (/^set\s+-[A-Za-z]*e[A-Za-z]*(?:\s|$)/m.test(deploymentPreflightScriptContent)) {
|
||||||
|
errors.push('check-existing-deployment.sh 主流程不得启用 set -e');
|
||||||
|
}
|
||||||
|
|
||||||
|
for (const requiredFragment of [
|
||||||
|
'SSH_ORIGINAL_COMMAND',
|
||||||
|
'probe) probe',
|
||||||
|
'refresh) refresh',
|
||||||
|
'20-active-build.conf',
|
||||||
|
'refresh_program_integrity_invalid',
|
||||||
|
'unit_build_not_refreshed',
|
||||||
|
'timer_not_active',
|
||||||
|
'systemctl daemon-reload',
|
||||||
|
]) {
|
||||||
|
if (!deploymentRefreshHelperContent.includes(requiredFragment)) {
|
||||||
|
errors.push(`refresh-deployment-contract.sh 缺少 forced-command 合同: ${requiredFragment}`);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
for (const forbiddenFragment of ['eval ', 'bash -c', 'REMOTE_DIR/']) {
|
||||||
|
if (deploymentRefreshHelperContent.includes(forbiddenFragment)) {
|
||||||
|
errors.push(`refresh-deployment-contract.sh 包含禁止的动态执行片段: ${forbiddenFragment}`);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
for (const requiredFragment of [
|
||||||
|
'no-agent-forwarding,no-port-forwarding,no-X11-forwarding,no-pty,no-user-rc',
|
||||||
|
'dedicated key already has unmanaged or unrestricted root authorization',
|
||||||
|
'install -o root -g root -m 0755',
|
||||||
|
]) {
|
||||||
|
if (!deploymentRefreshInstallerContent.includes(requiredFragment)) {
|
||||||
|
errors.push(`install-deployment-contract-refresh.sh 缺少受限安装合同: ${requiredFragment}`);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
if (deployExistingScriptContent.indexOf('check-existing-deployment.sh')
|
||||||
|
> deployExistingScriptContent.indexOf('bash "$ROOT_DIR/deploy/trialrun/deploy.sh"')) {
|
||||||
|
errors.push('deploy-existing.sh 必须在应用部署前执行聚合体检');
|
||||||
|
}
|
||||||
|
if (deployExistingScriptContent.indexOf('ssh "$DEPLOY_REFRESH_SERVER" refresh')
|
||||||
|
< deployExistingScriptContent.indexOf('bash "$ROOT_DIR/deploy/trialrun/deploy.sh"')) {
|
||||||
|
errors.push('deploy-existing.sh 必须在应用部署成功后刷新备份合同');
|
||||||
|
}
|
||||||
|
|
||||||
for (const requiredFragment of [
|
for (const requiredFragment of [
|
||||||
'REMOTE_DIR 是现存仓库根目录',
|
'REMOTE_DIR 是现存仓库根目录',
|
||||||
'SYNC_ENV=0 要求同步前已有普通远端 .env 文件',
|
'SYNC_ENV=0 要求同步前已有普通远端 .env 文件',
|
||||||
|
|||||||
@@ -0,0 +1,181 @@
|
|||||||
|
#!/usr/bin/env bash
|
||||||
|
set -euo pipefail
|
||||||
|
|
||||||
|
ROOT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")/../.." && pwd)"
|
||||||
|
CHECKER="$ROOT_DIR/deploy/trialrun/check-existing-deployment.sh"
|
||||||
|
HELPER="$ROOT_DIR/scripts/refresh-deployment-contract.sh"
|
||||||
|
LIBRARY="$ROOT_DIR/deploy/trialrun/deployment-contract-lib.sh"
|
||||||
|
INSTALLER="$ROOT_DIR/deploy/trialrun/install-deployment-contract-refresh.sh"
|
||||||
|
FIXTURE_ROOT="$(mktemp -d "${TMPDIR:-/tmp}/hyb-oa-contract-refresh.XXXXXX")"
|
||||||
|
trap 'rm -rf -- "$FIXTURE_ROOT"' EXIT
|
||||||
|
mkdir -p "$FIXTURE_ROOT/bin" "$FIXTURE_ROOT/etc/systemd/system" "$FIXTURE_ROOT/run/lock"
|
||||||
|
|
||||||
|
cat >"$FIXTURE_ROOT/bin/ssh" <<'SH'
|
||||||
|
#!/usr/bin/env bash
|
||||||
|
set -uo pipefail
|
||||||
|
target="${1:-}"
|
||||||
|
shift || true
|
||||||
|
mode="${FIXTURE_SSH_MODE:-pass}"
|
||||||
|
if [[ "$target" == "deploy" && "${1:-}" == "true" ]]; then
|
||||||
|
[[ "$mode" != "deploy_transport_fail" ]] || exit 255
|
||||||
|
exit 0
|
||||||
|
fi
|
||||||
|
if [[ "$target" == "deploy" && "$*" == *"systemctl show"* ]]; then
|
||||||
|
[[ "$mode" != "contract_fail" ]] || exit 1
|
||||||
|
printf '%s\n' 'REMOTE_DIR=/srv/app REMOTE_ENV_FILE=/srv/app/deploy/trialrun/.env COMPOSE_FILE=deploy/trialrun/docker-compose.yml COMPOSE_PROJECT_NAME=fixture POSTGRES_DATA_DIR=/srv/data/postgres STORAGE_HOST_DIR=/srv/data/storage BUILD_SHA=aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa'
|
||||||
|
exit 0
|
||||||
|
fi
|
||||||
|
if [[ "$target" == "deploy" ]]; then
|
||||||
|
if [[ "$mode" == "multi_fail" ]]; then
|
||||||
|
cat <<'OUT'
|
||||||
|
repository_root=FAIL
|
||||||
|
repository_writable=PASS
|
||||||
|
env_readable=FAIL
|
||||||
|
data_directories=PASS
|
||||||
|
runtime_unique=PASS
|
||||||
|
runtime_build=PASS
|
||||||
|
storage_mount=FAIL
|
||||||
|
pgdata_mount=PASS
|
||||||
|
backup_build=PASS
|
||||||
|
OUT
|
||||||
|
else
|
||||||
|
for item in repository_root repository_writable env_readable data_directories runtime_unique runtime_build storage_mount pgdata_mount backup_build; do
|
||||||
|
printf '%s=PASS\n' "$item"
|
||||||
|
done
|
||||||
|
fi
|
||||||
|
exit 0
|
||||||
|
fi
|
||||||
|
if [[ "$target" == "refresh" && "${1:-}" == "probe" ]]; then
|
||||||
|
case "$mode" in
|
||||||
|
root_transport_fail) exit 255 ;;
|
||||||
|
program_missing|multi_fail) exit 127 ;;
|
||||||
|
*) echo '[deployment-contract-refresh][PASS] probe'; exit 0 ;;
|
||||||
|
esac
|
||||||
|
fi
|
||||||
|
exit 1
|
||||||
|
SH
|
||||||
|
chmod 0755 "$FIXTURE_ROOT/bin/ssh"
|
||||||
|
|
||||||
|
run_checker() {
|
||||||
|
PATH="$FIXTURE_ROOT/bin:$PATH" DEPLOY_SERVER=deploy DEPLOY_REFRESH_SERVER=refresh \
|
||||||
|
FIXTURE_SSH_MODE="${1:-pass}" bash "$CHECKER"
|
||||||
|
}
|
||||||
|
|
||||||
|
pass_output="$(run_checker pass)"
|
||||||
|
[[ "$pass_output" == *"summary PASS=13 FAIL=0"* ]]
|
||||||
|
|
||||||
|
set +e
|
||||||
|
multi_output="$(run_checker multi_fail 2>&1)"
|
||||||
|
multi_status=$?
|
||||||
|
set -e
|
||||||
|
[[ "$multi_status" -ne 0 ]]
|
||||||
|
[[ "$multi_output" == *"repository_root FAIL"* \
|
||||||
|
&& "$multi_output" == *"env_readable FAIL"* \
|
||||||
|
&& "$multi_output" == *"storage_mount FAIL"* \
|
||||||
|
&& "$multi_output" == *"refresh_program FAIL"* \
|
||||||
|
&& "$multi_output" == *"summary PASS=9 FAIL=4"* ]]
|
||||||
|
[[ "$multi_output" != *"/srv/"* && "$multi_output" != *"aaaaaaaaaaaaaaaa"* ]]
|
||||||
|
|
||||||
|
set +e
|
||||||
|
root_output="$(run_checker root_transport_fail 2>&1)"
|
||||||
|
root_status=$?
|
||||||
|
set -e
|
||||||
|
[[ "$root_status" -ne 0 && "$root_output" == *"root_ssh_identity FAIL"* \
|
||||||
|
&& "$root_output" == *"refresh_program FAIL"* ]]
|
||||||
|
|
||||||
|
source "$LIBRARY"
|
||||||
|
[[ "$(deployment_contract_refresh_category 255 '')" == "root_identity_unavailable" ]]
|
||||||
|
[[ "$(deployment_contract_refresh_category 127 '')" == "refresh_program_unavailable" ]]
|
||||||
|
[[ "$(deployment_contract_refresh_category 42 '')" == "unit_build_not_refreshed" ]]
|
||||||
|
[[ "$(deployment_contract_refresh_category 43 '')" == "timer_not_active" ]]
|
||||||
|
[[ "$(deployment_contract_refresh_category 0 '[deployment-contract-refresh][PASS] refreshed')" == "success" ]]
|
||||||
|
|
||||||
|
for service in \
|
||||||
|
hyb-oa-automatic-backup-daily.service \
|
||||||
|
hyb-oa-automatic-backup-watchdog.service \
|
||||||
|
hyb-oa-automatic-backup-drill-reminder.service \
|
||||||
|
hyb-oa-warch3-ops.service; do
|
||||||
|
: >"$FIXTURE_ROOT/etc/systemd/system/$service"
|
||||||
|
done
|
||||||
|
|
||||||
|
cat >"$FIXTURE_ROOT/bin/docker" <<'SH'
|
||||||
|
#!/usr/bin/env bash
|
||||||
|
if [[ "${1:-}" == "ps" ]]; then
|
||||||
|
echo server-fixture
|
||||||
|
elif [[ "${1:-}" == "inspect" ]]; then
|
||||||
|
echo 'BUILD_SHA=bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb'
|
||||||
|
else
|
||||||
|
exit 1
|
||||||
|
fi
|
||||||
|
SH
|
||||||
|
chmod 0755 "$FIXTURE_ROOT/bin/docker"
|
||||||
|
|
||||||
|
cat >"$FIXTURE_ROOT/bin/systemctl" <<'SH'
|
||||||
|
#!/usr/bin/env bash
|
||||||
|
set -euo pipefail
|
||||||
|
root="${HYB_CONTRACT_REFRESH_TEST_ROOT:?}"
|
||||||
|
action="${1:-}"
|
||||||
|
case "$action" in
|
||||||
|
show)
|
||||||
|
unit="${2:-}"
|
||||||
|
property="${4:-}"
|
||||||
|
case "$property" in
|
||||||
|
LoadState) echo loaded ;;
|
||||||
|
FragmentPath) echo "$root/etc/systemd/system/$unit" ;;
|
||||||
|
Environment)
|
||||||
|
echo 'COMPOSE_PROJECT_NAME=fixture BUILD_SHA=aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa VITE_BUILD_SHA=aaaaaaaaaaaa OLD_BUILD_SHA=bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb OLD_VITE_BUILD_SHA=bbbbbbbbbbbb'
|
||||||
|
if [[ "${FIXTURE_IGNORE_DROPINS:-0}" != "1" ]]; then
|
||||||
|
dropin="$root/etc/systemd/system/${unit}.d/20-active-build.conf"
|
||||||
|
[[ ! -f "$dropin" ]] || sed -n 's/^Environment=//p' "$dropin" | paste -sd' ' -
|
||||||
|
fi
|
||||||
|
;;
|
||||||
|
*) exit 1 ;;
|
||||||
|
esac
|
||||||
|
;;
|
||||||
|
daemon-reload) exit 0 ;;
|
||||||
|
is-enabled) echo enabled ;;
|
||||||
|
is-active)
|
||||||
|
if [[ "${FIXTURE_TIMER_INACTIVE:-0}" == "1" ]]; then echo inactive; exit 3; fi
|
||||||
|
echo active
|
||||||
|
;;
|
||||||
|
*) exit 1 ;;
|
||||||
|
esac
|
||||||
|
SH
|
||||||
|
chmod 0755 "$FIXTURE_ROOT/bin/systemctl"
|
||||||
|
|
||||||
|
probe_output="$(HYB_CONTRACT_REFRESH_TEST_ROOT="$FIXTURE_ROOT" SSH_ORIGINAL_COMMAND=probe bash "$HELPER")"
|
||||||
|
[[ "$probe_output" == '[deployment-contract-refresh][PASS] probe' ]]
|
||||||
|
refresh_output="$(HYB_CONTRACT_REFRESH_TEST_ROOT="$FIXTURE_ROOT" SSH_ORIGINAL_COMMAND=refresh bash "$HELPER")"
|
||||||
|
[[ "$refresh_output" == '[deployment-contract-refresh][PASS] refreshed' ]]
|
||||||
|
[[ "$(find "$FIXTURE_ROOT/etc/systemd/system" -name 20-active-build.conf -type f | wc -l)" -eq 4 ]]
|
||||||
|
|
||||||
|
set +e
|
||||||
|
HYB_CONTRACT_REFRESH_TEST_ROOT="$FIXTURE_ROOT" FIXTURE_IGNORE_DROPINS=1 \
|
||||||
|
SSH_ORIGINAL_COMMAND=refresh bash "$HELPER" >/dev/null 2>&1
|
||||||
|
unit_status=$?
|
||||||
|
HYB_CONTRACT_REFRESH_TEST_ROOT="$FIXTURE_ROOT" FIXTURE_TIMER_INACTIVE=1 \
|
||||||
|
SSH_ORIGINAL_COMMAND=refresh bash "$HELPER" >/dev/null 2>&1
|
||||||
|
timer_status=$?
|
||||||
|
HYB_CONTRACT_REFRESH_TEST_ROOT="$FIXTURE_ROOT" \
|
||||||
|
SSH_ORIGINAL_COMMAND='refresh extra' bash "$HELPER" >/dev/null 2>&1
|
||||||
|
unsupported_status=$?
|
||||||
|
set -e
|
||||||
|
[[ "$unit_status" -eq 42 && "$timer_status" -eq 43 && "$unsupported_status" -eq 64 ]]
|
||||||
|
|
||||||
|
public_key_file="$FIXTURE_ROOT/refresh.pub"
|
||||||
|
key_blob='AAAAC3NzaC1lZDI1NTE5AAAAIFixturerefreshkeymaterial000000000000'
|
||||||
|
printf 'ssh-ed25519 %s fixture-refresh\n' "$key_blob" >"$public_key_file"
|
||||||
|
installer_output="$(HYB_CONTRACT_REFRESH_INSTALL_TEST_ROOT="$FIXTURE_ROOT" \
|
||||||
|
DEPLOY_REFRESH_PUBLIC_KEY_FILE="$public_key_file" bash "$INSTALLER" install)"
|
||||||
|
[[ "$installer_output" == '[deployment-contract-refresh-installer][PASS] forced command installed' ]]
|
||||||
|
authorized_keys="$FIXTURE_ROOT/root/.ssh/authorized_keys"
|
||||||
|
[[ "$(grep -Fc '# hyb-oa-managed-deployment-contract-refresh' "$authorized_keys")" -eq 1 ]]
|
||||||
|
printf 'ssh-ed25519 %s unrestricted-duplicate\n' "$key_blob" >>"$authorized_keys"
|
||||||
|
set +e
|
||||||
|
HYB_CONTRACT_REFRESH_INSTALL_TEST_ROOT="$FIXTURE_ROOT" \
|
||||||
|
DEPLOY_REFRESH_PUBLIC_KEY_FILE="$public_key_file" bash "$INSTALLER" install >/dev/null 2>&1
|
||||||
|
duplicate_status=$?
|
||||||
|
set -e
|
||||||
|
[[ "$duplicate_status" -ne 0 ]]
|
||||||
|
|
||||||
|
echo '[deployment-contract-refresh-contract][PASS] aggregate and forced-command contracts verified'
|
||||||
+198
@@ -0,0 +1,198 @@
|
|||||||
|
#!/usr/bin/env bash
|
||||||
|
set -euo pipefail
|
||||||
|
|
||||||
|
PATH="/usr/sbin:/usr/bin:/sbin:/bin"
|
||||||
|
export PATH
|
||||||
|
umask 077
|
||||||
|
unset BASH_ENV ENV CDPATH GLOBIGNORE LD_PRELOAD LD_LIBRARY_PATH PYTHONPATH NODE_OPTIONS
|
||||||
|
|
||||||
|
SERVICES=(
|
||||||
|
hyb-oa-automatic-backup-daily.service
|
||||||
|
hyb-oa-automatic-backup-watchdog.service
|
||||||
|
hyb-oa-automatic-backup-drill-reminder.service
|
||||||
|
hyb-oa-warch3-ops.service
|
||||||
|
)
|
||||||
|
TIMERS=(
|
||||||
|
hyb-oa-automatic-backup-daily.timer
|
||||||
|
hyb-oa-automatic-backup-watchdog.timer
|
||||||
|
hyb-oa-automatic-backup-drill-reminder.timer
|
||||||
|
hyb-oa-warch3-ops.timer
|
||||||
|
)
|
||||||
|
CONTRACT_SERVICE="hyb-oa-automatic-backup-daily.service"
|
||||||
|
DROPIN_NAME="20-active-build.conf"
|
||||||
|
ROOT_PREFIX=""
|
||||||
|
TEST_MODE=0
|
||||||
|
|
||||||
|
fail() {
|
||||||
|
local code="$1"
|
||||||
|
local category="$2"
|
||||||
|
echo "[deployment-contract-refresh][FAIL] category=${category}" >&2
|
||||||
|
exit "$code"
|
||||||
|
}
|
||||||
|
|
||||||
|
if [[ "$(id -u)" != "0" ]]; then
|
||||||
|
test_root="${HYB_CONTRACT_REFRESH_TEST_ROOT:-}"
|
||||||
|
if [[ "$test_root" =~ ^/tmp/[A-Za-z0-9._/-]+$ && "$test_root" != */ \
|
||||||
|
&& "$test_root" != *"//"* && "/$test_root/" != *"/../"* ]]; then
|
||||||
|
TEST_MODE=1
|
||||||
|
ROOT_PREFIX="$test_root"
|
||||||
|
PATH="$test_root/bin:/usr/sbin:/usr/bin:/sbin:/bin"
|
||||||
|
export PATH
|
||||||
|
else
|
||||||
|
fail 70 root_required
|
||||||
|
fi
|
||||||
|
fi
|
||||||
|
|
||||||
|
UNIT_ROOT="${ROOT_PREFIX}/etc/systemd/system"
|
||||||
|
LOCK_FILE="${ROOT_PREFIX}/run/lock/hyb-oa-deployment-contract-refresh.lock"
|
||||||
|
EXPECTED_TARGET="${ROOT_PREFIX}/usr/local/sbin/hyb-oa-deployment-contract-refresh"
|
||||||
|
|
||||||
|
need_command() {
|
||||||
|
command -v "$1" >/dev/null 2>&1 || fail 70 runtime_prerequisite_missing
|
||||||
|
}
|
||||||
|
|
||||||
|
for required_command in systemctl docker install mktemp mv mkdir chmod flock stat; do
|
||||||
|
need_command "$required_command"
|
||||||
|
done
|
||||||
|
|
||||||
|
validate_self() {
|
||||||
|
if [[ "$TEST_MODE" == "0" ]]; then
|
||||||
|
[[ "$0" == "$EXPECTED_TARGET" && -f "$EXPECTED_TARGET" && ! -L "$EXPECTED_TARGET" \
|
||||||
|
&& "$(stat -c %u:%g:%a "$EXPECTED_TARGET")" == "0:0:755" ]] \
|
||||||
|
|| fail 70 refresh_program_integrity_invalid
|
||||||
|
else
|
||||||
|
[[ -f "$0" && ! -L "$0" ]] || fail 70 refresh_program_integrity_invalid
|
||||||
|
fi
|
||||||
|
}
|
||||||
|
|
||||||
|
systemctl_value() {
|
||||||
|
local unit="$1"
|
||||||
|
local property="$2"
|
||||||
|
systemctl show "$unit" -p "$property" --value
|
||||||
|
}
|
||||||
|
|
||||||
|
unit_environment_value() {
|
||||||
|
local unit="$1"
|
||||||
|
local wanted="$2"
|
||||||
|
local entry value=""
|
||||||
|
for entry in $(systemctl_value "$unit" Environment); do
|
||||||
|
if [[ "${entry%%=*}" == "$wanted" ]]; then
|
||||||
|
value="${entry#*=}"
|
||||||
|
fi
|
||||||
|
done
|
||||||
|
[[ -n "$value" ]] || return 1
|
||||||
|
printf '%s' "$value"
|
||||||
|
}
|
||||||
|
|
||||||
|
validate_fixed_units() {
|
||||||
|
local unit fragment owner
|
||||||
|
for unit in "${SERVICES[@]}"; do
|
||||||
|
[[ "$(systemctl_value "$unit" LoadState)" == "loaded" ]] \
|
||||||
|
|| fail 70 service_not_loaded
|
||||||
|
fragment="$(systemctl_value "$unit" FragmentPath)"
|
||||||
|
[[ -n "$fragment" ]] || fail 70 service_fragment_missing
|
||||||
|
if [[ "$TEST_MODE" == "0" ]]; then
|
||||||
|
[[ "$fragment" == /etc/systemd/system/* && -f "$fragment" && ! -L "$fragment" ]] \
|
||||||
|
|| fail 70 service_fragment_invalid
|
||||||
|
owner="$(stat -c %u "$fragment")"
|
||||||
|
[[ "$owner" == "0" ]] || fail 70 service_fragment_not_root_owned
|
||||||
|
fi
|
||||||
|
done
|
||||||
|
}
|
||||||
|
|
||||||
|
resolve_active_build() {
|
||||||
|
local project
|
||||||
|
local -a servers=()
|
||||||
|
project="$(unit_environment_value "$CONTRACT_SERVICE" COMPOSE_PROJECT_NAME)" \
|
||||||
|
|| fail 70 compose_project_missing
|
||||||
|
[[ "$project" =~ ^[a-z0-9][a-z0-9_-]*$ ]] || fail 70 compose_project_invalid
|
||||||
|
mapfile -t servers < <(docker ps \
|
||||||
|
--filter "label=com.docker.compose.project=$project" \
|
||||||
|
--filter "label=com.docker.compose.service=server" \
|
||||||
|
--format '{{.ID}}')
|
||||||
|
[[ "${#servers[@]}" -eq 1 && -n "${servers[0]}" ]] \
|
||||||
|
|| fail 70 active_server_not_unique
|
||||||
|
ACTIVE_BUILD="$(docker inspect -f '{{range .Config.Env}}{{println .}}{{end}}' "${servers[0]}" \
|
||||||
|
| sed -n 's/^BUILD_SHA=//p' | tail -n 1)"
|
||||||
|
ACTIVE_BUILD="${ACTIVE_BUILD,,}"
|
||||||
|
[[ "$ACTIVE_BUILD" =~ ^[0-9a-f]{40,64}$ ]] || fail 70 active_build_invalid
|
||||||
|
ACTIVE_FRONTEND_BUILD="${ACTIVE_BUILD:0:12}"
|
||||||
|
}
|
||||||
|
|
||||||
|
write_build_dropin() {
|
||||||
|
local service="$1"
|
||||||
|
local dropin_dir="$UNIT_ROOT/${service}.d"
|
||||||
|
local target="$dropin_dir/$DROPIN_NAME"
|
||||||
|
local temporary
|
||||||
|
|
||||||
|
if [[ "$TEST_MODE" == "0" ]]; then
|
||||||
|
install -d -o root -g root -m 0755 "$dropin_dir"
|
||||||
|
else
|
||||||
|
mkdir -p "$dropin_dir"
|
||||||
|
chmod 0755 "$dropin_dir"
|
||||||
|
fi
|
||||||
|
temporary="$(mktemp "$dropin_dir/.${DROPIN_NAME}.XXXXXX")"
|
||||||
|
{
|
||||||
|
printf '[Service]\n'
|
||||||
|
printf 'Environment=BUILD_SHA=%s\n' "$ACTIVE_BUILD"
|
||||||
|
printf 'Environment=VITE_BUILD_SHA=%s\n' "$ACTIVE_FRONTEND_BUILD"
|
||||||
|
} >"$temporary"
|
||||||
|
chmod 0644 "$temporary"
|
||||||
|
if [[ "$TEST_MODE" == "0" ]]; then
|
||||||
|
chown root:root "$temporary"
|
||||||
|
fi
|
||||||
|
mv -f -- "$temporary" "$target"
|
||||||
|
}
|
||||||
|
|
||||||
|
verify_builds() {
|
||||||
|
local service service_build service_frontend_build
|
||||||
|
for service in "${SERVICES[@]}"; do
|
||||||
|
service_build="$(unit_environment_value "$service" BUILD_SHA)" \
|
||||||
|
|| fail 42 unit_build_not_refreshed
|
||||||
|
service_frontend_build="$(unit_environment_value "$service" VITE_BUILD_SHA)" \
|
||||||
|
|| fail 42 unit_build_not_refreshed
|
||||||
|
[[ "$service_build" == "$ACTIVE_BUILD" \
|
||||||
|
&& "$service_frontend_build" == "$ACTIVE_FRONTEND_BUILD" ]] \
|
||||||
|
|| fail 42 unit_build_not_refreshed
|
||||||
|
done
|
||||||
|
}
|
||||||
|
|
||||||
|
verify_timers() {
|
||||||
|
local timer
|
||||||
|
for timer in "${TIMERS[@]}"; do
|
||||||
|
[[ "$(systemctl is-enabled "$timer")" == "enabled" \
|
||||||
|
&& "$(systemctl is-active "$timer")" == "active" ]] \
|
||||||
|
|| fail 43 timer_not_active
|
||||||
|
done
|
||||||
|
}
|
||||||
|
|
||||||
|
probe() {
|
||||||
|
validate_self
|
||||||
|
validate_fixed_units
|
||||||
|
resolve_active_build
|
||||||
|
echo "[deployment-contract-refresh][PASS] probe"
|
||||||
|
}
|
||||||
|
|
||||||
|
refresh() {
|
||||||
|
validate_self
|
||||||
|
validate_fixed_units
|
||||||
|
resolve_active_build
|
||||||
|
mkdir -p "$(dirname "$LOCK_FILE")"
|
||||||
|
exec 9>"$LOCK_FILE"
|
||||||
|
flock -n 9 || fail 42 refresh_already_running
|
||||||
|
local service
|
||||||
|
for service in "${SERVICES[@]}"; do
|
||||||
|
write_build_dropin "$service"
|
||||||
|
done
|
||||||
|
systemctl daemon-reload
|
||||||
|
verify_builds
|
||||||
|
verify_timers
|
||||||
|
echo "[deployment-contract-refresh][PASS] refreshed"
|
||||||
|
}
|
||||||
|
|
||||||
|
[[ "$#" -eq 0 ]] || fail 64 unsupported_forced_command
|
||||||
|
case "${SSH_ORIGINAL_COMMAND:-}" in
|
||||||
|
probe) probe ;;
|
||||||
|
refresh) refresh ;;
|
||||||
|
*) fail 64 unsupported_forced_command ;;
|
||||||
|
esac
|
||||||
@@ -7,6 +7,9 @@ const { promisify } = require("node:util");
|
|||||||
const execFileAsync = promisify(execFile);
|
const execFileAsync = promisify(execFile);
|
||||||
const ROOT_DIR = path.resolve(__dirname, "../../../..");
|
const ROOT_DIR = path.resolve(__dirname, "../../../..");
|
||||||
const PREFLIGHT = path.join(ROOT_DIR, "deploy/trialrun/preflight-remote.sh");
|
const PREFLIGHT = path.join(ROOT_DIR, "deploy/trialrun/preflight-remote.sh");
|
||||||
|
const AGGREGATE_PREFLIGHT = path.join(ROOT_DIR, "deploy/trialrun/check-existing-deployment.sh");
|
||||||
|
const REFRESH_HELPER = path.join(ROOT_DIR, "scripts/refresh-deployment-contract.sh");
|
||||||
|
const REFRESH_INSTALLER = path.join(ROOT_DIR, "deploy/trialrun/install-deployment-contract-refresh.sh");
|
||||||
|
|
||||||
describe("deployment safety contract", () => {
|
describe("deployment safety contract", () => {
|
||||||
let fixtureRoot;
|
let fixtureRoot;
|
||||||
@@ -77,11 +80,35 @@ describe("deployment safety contract", () => {
|
|||||||
expect(deploy.indexOf("REMOTE_PREFLIGHT_SCRIPT")).toBeGreaterThanOrEqual(0);
|
expect(deploy.indexOf("REMOTE_PREFLIGHT_SCRIPT")).toBeGreaterThanOrEqual(0);
|
||||||
expect(deploy.indexOf('>>> 3. 同步前只读校验远端部署合同')).toBeLessThan(deploy.indexOf("rsync -az --delete"));
|
expect(deploy.indexOf('>>> 3. 同步前只读校验远端部署合同')).toBeLessThan(deploy.indexOf("rsync -az --delete"));
|
||||||
expect(deploy.indexOf('>>> 3. 同步前只读校验远端部署合同')).toBeLessThan(deploy.indexOf("mkdir -p '${REMOTE_DIR}'"));
|
expect(deploy.indexOf('>>> 3. 同步前只读校验远端部署合同')).toBeLessThan(deploy.indexOf("mkdir -p '${REMOTE_DIR}'"));
|
||||||
expect(wrapper).toContain("systemctl show '${DEPLOY_CONTRACT_UNIT}' -p Environment --value");
|
expect(wrapper).toContain("check-existing-deployment.sh");
|
||||||
|
expect(wrapper.indexOf("check-existing-deployment.sh")).toBeLessThan(wrapper.indexOf('bash "$ROOT_DIR/deploy/trialrun/deploy.sh"'));
|
||||||
expect(wrapper).not.toContain("docker inspect");
|
expect(wrapper).not.toContain("docker inspect");
|
||||||
expect(wrapper).not.toContain("working_dir");
|
expect(wrapper).not.toContain("working_dir");
|
||||||
expect(wrapper).toContain("DEPLOY_DATA_MODE=existing");
|
expect(wrapper).toContain("DEPLOY_DATA_MODE=existing");
|
||||||
expect(wrapper).toContain("SYNC_ENV=0");
|
expect(wrapper).toContain("SYNC_ENV=0");
|
||||||
|
expect(wrapper).toContain("DEPLOY_DEFER_FINAL_SUCCESS=1");
|
||||||
|
expect(wrapper.indexOf('ssh "$DEPLOY_REFRESH_SERVER" refresh')).toBeGreaterThan(wrapper.indexOf('bash "$ROOT_DIR/deploy/trialrun/deploy.sh"'));
|
||||||
|
expect(wrapper).toContain("DEPLOYED_BUT_CONTRACT_REFRESH_FAILED");
|
||||||
|
});
|
||||||
|
|
||||||
|
it("aggregates preflight checks and keeps the root helper self-contained", async () => {
|
||||||
|
const aggregate = await require("node:fs/promises").readFile(AGGREGATE_PREFLIGHT, "utf8");
|
||||||
|
const helper = await require("node:fs/promises").readFile(REFRESH_HELPER, "utf8");
|
||||||
|
const installer = await require("node:fs/promises").readFile(REFRESH_INSTALLER, "utf8");
|
||||||
|
expect(aggregate).toContain("set -uo pipefail");
|
||||||
|
expect(aggregate).not.toMatch(/^set\s+-[A-Za-z]*e[A-Za-z]*(?:\s|$)/m);
|
||||||
|
expect(aggregate).toContain("CHECK_NAMES=(");
|
||||||
|
expect(aggregate).toContain("root_ssh_identity");
|
||||||
|
expect(aggregate).toContain("refresh_program");
|
||||||
|
expect(helper).toContain("SSH_ORIGINAL_COMMAND");
|
||||||
|
expect(helper).toContain("20-active-build.conf");
|
||||||
|
expect(helper).toContain("refresh_program_integrity_invalid");
|
||||||
|
expect(helper).not.toContain("eval ");
|
||||||
|
expect(helper).not.toContain("bash -c");
|
||||||
|
expect(helper).not.toContain("REMOTE_DIR/");
|
||||||
|
expect(installer).toContain("dedicated key already has unmanaged or unrestricted root authorization");
|
||||||
|
expect(installer).toContain("no-agent-forwarding,no-port-forwarding,no-X11-forwarding,no-pty,no-user-rc");
|
||||||
|
expect(installer).toContain("managed forced-command marker is malformed");
|
||||||
});
|
});
|
||||||
|
|
||||||
it("keeps immutable build args after stable dependency layers", async () => {
|
it("keeps immutable build args after stable dependency layers", async () => {
|
||||||
|
|||||||
在新工单中引用
屏蔽一个用户