补充部署合同失败恢复说明(08-02-deployment-preflight-backup-contract-refresh)

这个提交包含在:
2026-08-02 12:09:06 +08:00
父节点 7138d2b85c
当前提交 2061c03776
共修改 5 个文件,包含 8 行新增和 1 行删除
@@ -1313,6 +1313,7 @@ bash scripts/qa/run-automatic-backup-contract.sh
- When recovering a broken deployment contract or changing frozen paths/project identity, re-run the automatic-backup and WARCH3 installers with the active full `BUILD_SHA` and matching 12-character frontend prefix. Validate all managed service working directories/env paths/build identities and enabled timers, then run one complete backup plus the freshness watchdog; application `/ready` alone does not prove that a controlled recovery is complete.
- For normal existing deployments after forced-command installation, the wrapper replaces the manual per-release installer step: after application, smoke and public checks, `refresh` derives the full SHA from the unique active server, atomically writes root-owned build-only drop-ins for the three automatic-backup services and WARCH3, reloads systemd, and verifies four final service SHAs plus four enabled/active timers. Full installers remain authoritative for path or complete-unit changes.
- A post-refresh failure does not roll back a healthy application. The wrapper emits `DEPLOYED_BUT_CONTRACT_REFRESH_FAILED`, classifies root transport, missing program, stale unit build or inactive timer, and exits non-zero; only successful refresh may print the complete existing-deployment PASS.
- After that partial-success state, repair the classified root/program/unit/timer problem, invoke the forced-command `refresh` directly, then rerun aggregate preflight. The application already passed release smoke; do not redeploy it merely to repair the backup build contract, and do not bypass the expected `backup_build` failure while it is stale.
- `apk add` and both lockfile `npm ci` steps precede `BUILD_SHA` / `VITE_BUILD_SHA`. Build identity may invalidate application output, never stable dependency installation.
- Seed and smoke output reports readiness, counts and booleans; it does not print full login identifiers, contact values, WeCom identifiers, redirect URI or generated credentials.
@@ -1329,6 +1330,7 @@ bash scripts/qa/run-automatic-backup-contract.sh
- Helper runs but final service build differs -> `unit_build_not_refreshed`; builds match but any timer is disabled/inactive -> `timer_not_active`.
- A similarly named environment key such as `OLD_BUILD_SHA` matches the active value while exact `BUILD_SHA` remains stale -> treat the service as stale; build verification parses exact keys and applies the last systemd override.
- Post-deploy installer still embeds the previous build SHA, any managed timer is inactive, or the real backup/watchdog fails runtime identity or off-host freshness -> deployment recovery remains incomplete even when `/ready` is green.
- A later deployment attempt is blocked by `backup_build` after `DEPLOYED_BUT_CONTRACT_REFRESH_FAILED` -> expected fail-closed behavior; repair with direct forced-command `refresh`, not another application release.
- Lockfile unchanged but a new SHA reruns dependency installation -> Dockerfile contract test fails.
### 5. Good/Base/Bad Cases
@@ -76,6 +76,7 @@
- 每周只读体检 timer 暂不建设。触发条件:本脚本上线后在后续真实部署中再次发现非一次性漂移,或出现人工改服务器导致合同在部署间隔内失配;届时单独立项评估定时运行、告警和第 5 个 unit 的维护成本。
- 旧目录/旧环境文件清理单独审批。至少要求聚合体检在规范根上跑绿、一次后续真实部署自动收尾跑绿,且当前异机备份仍新鲜。
- root-owned helper 自身摘要比对暂不加入首版协议。触发条件:生产安装后的 helper 与仓库审核版本出现过内容漂移,或后续需要把 root-owned 独立制品纳入统一完整性巡检;届时单独设计可信摘要来源、升级与回滚合同,不能让 helper 自报摘要后由自身证明可信。
## Test Plan
+1 -1
查看文件
@@ -93,7 +93,7 @@
- [ ] 若本次包含前端体验修复,已在容器内确认 `/app/client/dist` 含关键 test id 或公网 asset hash 已更新
- [ ] 公网 bundle 扫描确认包含匹配的 12 位构建标识,且不包含完整提交 SHA
- [ ] wrapper 在应用与公网检查成功后自动刷新四个固定 service 的 build drop-in;四个最终 SHA 与活动容器一致,四个 timer 均 enabled/active
- [ ] 若出现 `DEPLOYED_BUT_CONTRACT_REFRESH_FAILED`,已按 root 身份、程序可用性、unit SHA、timer 状态四类处理,并将本次发布标记为“应用已部署但合同未收口”,未误报完整成功
- [ ] 若出现 `DEPLOYED_BUT_CONTRACT_REFRESH_FAILED`,已按 root 身份、程序可用性、unit SHA、timer 状态四类处理,并将本次发布标记为“应用已部署但合同未收口”;修复后单独执行 forced-command `refresh` 并重跑只读体检,未重复部署应用或绕过 `backup_build`
- [ ] 登录验证已由 live smoke 与 release gate 覆盖;仅首次接入或第三方后台配置变更时补做人工确认
## C. 发布后(Post-check)
+1
查看文件
@@ -435,6 +435,7 @@ systemctl status hyb-oa-public-app-health.service --no-pager
- 旧栈停止命令必须使用冻结阶段由 `docker compose ls` 实测的旧 project;不得拿新 project 名去停止旧容器。
- `artifacts/` 和 `output/` 是运行期证据/截图目录,不作为源码发布输入,也不由 `rsync --delete` 清理远端历史证据。
- 应用、`/ready`、smoke 和公网检查成功后,wrapper 才调用 forced-command `refresh`,以活动容器 build 原子刷新四个固定 service 的 systemd drop-in并验证四个 timer。失败时固定输出 `DEPLOYED_BUT_CONTRACT_REFRESH_FAILED`、退出非零,并明确表示应用已发布但合同未收口;不自动回滚应用。
- 出现 `DEPLOYED_BUT_CONTRACT_REFRESH_FAILED` 后,应用已经是通过 smoke 的新版本。修复对应的 root SSH、forced-command 程序、unit 或 timer 问题后,单独执行 `ssh <forced-command-ssh-target> refresh` 收敛合同,再重跑只读聚合体检;不要重跑应用部署,也不要绕过 `backup_build` 检查。
3) 构建并重启:
```bash
docker-compose -p "$COMPOSE_PROJECT_NAME" -f deploy/trialrun/docker-compose.yml --env-file deploy/trialrun/.env up -d --build server
+3
查看文件
@@ -30,6 +30,9 @@ fail() {
exit "$code"
}
# Exit 255 is reserved for the SSH client transport layer. Do not use it for
# helper failures, because callers classify 255 as root_identity_unavailable.
if [[ "$(id -u)" != "0" ]]; then
test_root="${HYB_CONTRACT_REFRESH_TEST_ROOT:-}"
if [[ "$test_root" =~ ^/tmp/[A-Za-z0-9._/-]+$ && "$test_root" != */ \