From 2061c03776e0f7c2c951075eba7bb2bde3e081c6 Mon Sep 17 00:00:00 2001 From: shunli Date: Sun, 2 Aug 2026 12:09:06 +0800 Subject: [PATCH] =?UTF-8?q?=E8=A1=A5=E5=85=85=E9=83=A8=E7=BD=B2=E5=90=88?= =?UTF-8?q?=E5=90=8C=E5=A4=B1=E8=B4=A5=E6=81=A2=E5=A4=8D=E8=AF=B4=E6=98=8E?= =?UTF-8?q?=EF=BC=8808-02-deployment-preflight-backup-contract-refresh?= =?UTF-8?q?=EF=BC=89?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- .trellis/spec/hyb-oa/deployment-trialrun.md | 2 ++ .../08-02-deployment-preflight-backup-contract-refresh/prd.md | 1 + deploy/trialrun/checklist.md | 2 +- deploy/trialrun/runbook.md | 1 + scripts/refresh-deployment-contract.sh | 3 +++ 5 files changed, 8 insertions(+), 1 deletion(-) diff --git a/.trellis/spec/hyb-oa/deployment-trialrun.md b/.trellis/spec/hyb-oa/deployment-trialrun.md index bb84b941..ab59361e 100644 --- a/.trellis/spec/hyb-oa/deployment-trialrun.md +++ b/.trellis/spec/hyb-oa/deployment-trialrun.md @@ -1313,6 +1313,7 @@ bash scripts/qa/run-automatic-backup-contract.sh - When recovering a broken deployment contract or changing frozen paths/project identity, re-run the automatic-backup and WARCH3 installers with the active full `BUILD_SHA` and matching 12-character frontend prefix. Validate all managed service working directories/env paths/build identities and enabled timers, then run one complete backup plus the freshness watchdog; application `/ready` alone does not prove that a controlled recovery is complete. - For normal existing deployments after forced-command installation, the wrapper replaces the manual per-release installer step: after application, smoke and public checks, `refresh` derives the full SHA from the unique active server, atomically writes root-owned build-only drop-ins for the three automatic-backup services and WARCH3, reloads systemd, and verifies four final service SHAs plus four enabled/active timers. Full installers remain authoritative for path or complete-unit changes. - A post-refresh failure does not roll back a healthy application. The wrapper emits `DEPLOYED_BUT_CONTRACT_REFRESH_FAILED`, classifies root transport, missing program, stale unit build or inactive timer, and exits non-zero; only successful refresh may print the complete existing-deployment PASS. +- After that partial-success state, repair the classified root/program/unit/timer problem, invoke the forced-command `refresh` directly, then rerun aggregate preflight. The application already passed release smoke; do not redeploy it merely to repair the backup build contract, and do not bypass the expected `backup_build` failure while it is stale. - `apk add` and both lockfile `npm ci` steps precede `BUILD_SHA` / `VITE_BUILD_SHA`. Build identity may invalidate application output, never stable dependency installation. - Seed and smoke output reports readiness, counts and booleans; it does not print full login identifiers, contact values, WeCom identifiers, redirect URI or generated credentials. @@ -1329,6 +1330,7 @@ bash scripts/qa/run-automatic-backup-contract.sh - Helper runs but final service build differs -> `unit_build_not_refreshed`; builds match but any timer is disabled/inactive -> `timer_not_active`. - A similarly named environment key such as `OLD_BUILD_SHA` matches the active value while exact `BUILD_SHA` remains stale -> treat the service as stale; build verification parses exact keys and applies the last systemd override. - Post-deploy installer still embeds the previous build SHA, any managed timer is inactive, or the real backup/watchdog fails runtime identity or off-host freshness -> deployment recovery remains incomplete even when `/ready` is green. +- A later deployment attempt is blocked by `backup_build` after `DEPLOYED_BUT_CONTRACT_REFRESH_FAILED` -> expected fail-closed behavior; repair with direct forced-command `refresh`, not another application release. - Lockfile unchanged but a new SHA reruns dependency installation -> Dockerfile contract test fails. ### 5. Good/Base/Bad Cases diff --git a/.trellis/tasks/08-02-deployment-preflight-backup-contract-refresh/prd.md b/.trellis/tasks/08-02-deployment-preflight-backup-contract-refresh/prd.md index efac0993..2ec29c68 100644 --- a/.trellis/tasks/08-02-deployment-preflight-backup-contract-refresh/prd.md +++ b/.trellis/tasks/08-02-deployment-preflight-backup-contract-refresh/prd.md @@ -76,6 +76,7 @@ - 每周只读体检 timer 暂不建设。触发条件：本脚本上线后在后续真实部署中再次发现非一次性漂移，或出现人工改服务器导致合同在部署间隔内失配；届时单独立项评估定时运行、告警和第 5 个 unit 的维护成本。 - 旧目录/旧环境文件清理单独审批。至少要求聚合体检在规范根上跑绿、一次后续真实部署自动收尾跑绿，且当前异机备份仍新鲜。 +- root-owned helper 自身摘要比对暂不加入首版协议。触发条件：生产安装后的 helper 与仓库审核版本出现过内容漂移，或后续需要把 root-owned 独立制品纳入统一完整性巡检；届时单独设计可信摘要来源、升级与回滚合同，不能让 helper 自报摘要后由自身证明可信。 ## Test Plan diff --git a/deploy/trialrun/checklist.md b/deploy/trialrun/checklist.md index 53e0c619..159fdbe8 100644 --- a/deploy/trialrun/checklist.md +++ b/deploy/trialrun/checklist.md @@ -93,7 +93,7 @@ - [ ] 若本次包含前端体验修复，已在容器内确认 `/app/client/dist` 含关键 test id 或公网 asset hash 已更新 - [ ] 公网 bundle 扫描确认包含匹配的 12 位构建标识，且不包含完整提交 SHA - [ ] wrapper 在应用与公网检查成功后自动刷新四个固定 service 的 build drop-in；四个最终 SHA 与活动容器一致，四个 timer 均 enabled/active -- [ ] 若出现 `DEPLOYED_BUT_CONTRACT_REFRESH_FAILED`，已按 root 身份、程序可用性、unit SHA、timer 状态四类处理，并将本次发布标记为“应用已部署但合同未收口”，未误报完整成功 +- [ ] 若出现 `DEPLOYED_BUT_CONTRACT_REFRESH_FAILED`，已按 root 身份、程序可用性、unit SHA、timer 状态四类处理，并将本次发布标记为“应用已部署但合同未收口”；修复后单独执行 forced-command `refresh` 并重跑只读体检，未重复部署应用或绕过 `backup_build` - [ ] 登录验证已由 live smoke 与 release gate 覆盖；仅首次接入或第三方后台配置变更时补做人工确认 ## C. 发布后（Post-check） diff --git a/deploy/trialrun/runbook.md b/deploy/trialrun/runbook.md index 2cc44ce9..ba194489 100644 --- a/deploy/trialrun/runbook.md +++ b/deploy/trialrun/runbook.md @@ -435,6 +435,7 @@ systemctl status hyb-oa-public-app-health.service --no-pager - 旧栈停止命令必须使用冻结阶段由 `docker compose ls` 实测的旧 project；不得拿新 project 名去停止旧容器。 - `artifacts/` 和 `output/` 是运行期证据/截图目录，不作为源码发布输入，也不由 `rsync --delete` 清理远端历史证据。 - 应用、`/ready`、smoke 和公网检查成功后，wrapper 才调用 forced-command `refresh`，以活动容器 build 原子刷新四个固定 service 的 systemd drop-in并验证四个 timer。失败时固定输出 `DEPLOYED_BUT_CONTRACT_REFRESH_FAILED`、退出非零，并明确表示应用已发布但合同未收口；不自动回滚应用。 + - 出现 `DEPLOYED_BUT_CONTRACT_REFRESH_FAILED` 后，应用已经是通过 smoke 的新版本。修复对应的 root SSH、forced-command 程序、unit 或 timer 问题后，单独执行 `ssh refresh` 收敛合同，再重跑只读聚合体检；不要重跑应用部署，也不要绕过 `backup_build` 检查。 3) 构建并重启： ```bash docker-compose -p "$COMPOSE_PROJECT_NAME" -f deploy/trialrun/docker-compose.yml --env-file deploy/trialrun/.env up -d --build server diff --git a/scripts/refresh-deployment-contract.sh b/scripts/refresh-deployment-contract.sh index 60eb8c65..303798fc 100755 --- a/scripts/refresh-deployment-contract.sh +++ b/scripts/refresh-deployment-contract.sh @@ -30,6 +30,9 @@ fail() { exit "$code" } +# Exit 255 is reserved for the SSH client transport layer. Do not use it for +# helper failures, because callers classify 255 as root_identity_unavailable. + if [[ "$(id -u)" != "0" ]]; then test_root="${HYB_CONTRACT_REFRESH_TEST_ROOT:-}" if [[ "$test_root" =~ ^/tmp/[A-Za-z0-9._/-]+$ && "$test_root" != */ \