hyb-oa/.trellis/spec/hyb-oa/docs-and-evidence.md

Docs And Evidence

Source Docs

Start from docs/文档索引.md and docs/文档事实源状态矩阵.md.

Rules

• Prefer updating existing fact sources, module specs, manuals, evidence docs or the current Trellis task.
• New documents need a reason and must be registered in both the index and status matrix.
• Old Issue CSV files are historical audit evidence and should not be rewritten unless the current Trellis task explicitly calls for a legacy/export update.
• For generated evidence, store report paths and conclusions, not raw secrets, raw payloads, full account identifiers, invoice originals or confidential business content.

Scenario: QA Readiness Report Summaries

1. Scope / Trigger

• Trigger: QA shell runners aggregate child reports for launch, patrol, smoke or readiness gates.
• Applies to generated summary.json, step JSON and privacy findings under artifacts/qa-flow/ or Trellis task research directories.

2. Signatures

• Parent shell runner writes summary.json.
• Child runner summary input: <child-report-dir>/summary.json.
• Step summary input/output: <step-name>.step.json.

3. Contracts

• Use repo-relative reportRef / logRef when a path is inside the repo.
• Do not write absolute local workspace paths into summary.json.
• Parent summaries must include child report status and child finding counts.
• Parent findingCount must become non-zero when any child summary reports PASS_WITH_FINDINGS, SKIP, PARTIAL, PAUSED, MISSING, blocker-like status or child findingCount > 0.
• Machine-readable reports may include status, count, mode, conclusion, boolean operational flags and relative report refs.
• Machine-readable reports must not include amounts, prices, costs, balances, bank details, invoice values, full accounts, tokens, secrets, raw payloads, raw form values or full local paths.
• Credential/readiness presence reports must use safe business labels, not raw env key names. For example, emit adminCredentialReady: true instead of a field named after a credential-value environment variable.

4. Validation & Error Matrix

• Child PASS + findingCount=0 -> parent may stay pass.
• Child PASS_WITH_FINDINGS -> parent records a finding.
• Child SKIP / PARTIAL / PAUSED / MISSING -> parent records manual_or_config.
• Child FAIL / BLOCKED / ERROR -> parent records a finding or blocker according to whether the parent step is required.
• Privacy smoke match -> write redacted finding metadata and make final summary reflect the failure.
• Secret-like field names in generated reports -> privacy finding, even when the value is only a boolean presence flag.

5. Good/Base/Bad Cases

• Good: summary.json contains reportRef, childReports.<name>.status, childReports.<name>.findingCount and no absolute local path.
• Base: Missing runtime credentials produce PASS_WITH_FINDINGS or manual_or_config, not a clean pass.
• Bad: Parent command exits 0 for a child PASS_WITH_FINDINGS and the parent summary still says clean pass.

6. Tests Required

• bash -n <runner>.sh
• Run the parent runner in a missing-credential scenario and assert:
  • parent status is PASS_WITH_FINDINGS or blocked, not clean pass;
  • child finding count is visible in parent summary;
  • privacy step is present in parent summary;
  • summary.json has no full workspace path or business amount fields.
• For live/readiness env reports, assert the JSON contains safe readiness labels and does not contain raw secret-like env key names.
• Run the matching launch/readiness aggregator when it consumes the report.

7. Wrong vs Correct

Wrong

{
  "status": "PASS_OA_REAL_USE_DAILY_READY",
  "reportDir": "/home/user/project/artifacts/qa-flow/run",
  "childReports": {}
}

{
  "liveSmoke": {
    "RAW_CREDENTIAL_ENV_KEY": true
  }
}

Correct

{
  "status": "PASS_WITH_FINDINGS",
  "reportRef": "artifacts/qa-flow/run",
  "childReports": {
    "oaOnlySmokeReadiness": {
      "status": "PASS_WITH_FINDINGS",
      "reportRef": "artifacts/qa-flow/run/oa-only-smoke-readiness/summary.json",
      "findingCount": 7
    }
  }
}

{
  "liveSmoke": {
    "adminCredentialReady": true
  }
}