diff --git a/.trellis/spec/hyb-oa/reimbursement-invoice-ocr.md b/.trellis/spec/hyb-oa/reimbursement-invoice-ocr.md index 53e9663d..1aca75d7 100644 --- a/.trellis/spec/hyb-oa/reimbursement-invoice-ocr.md +++ b/.trellis/spec/hyb-oa/reimbursement-invoice-ocr.md @@ -121,6 +121,136 @@ const ocrRan = invoice.ocr_attempts > 0 || invoice.ocr_result_id != null; ``` +## Scenario: Tencent General Invoice OCR Safe Persistence + +### 1. Scope / Trigger + +- Trigger: production uses Tencent OCR for employee-uploaded PDF invoices, + including VAT/electronic ordinary invoices and railway electronic tickets. +- Use the single `RecognizeGeneralInvoice` entry point. Do not split railway + tickets onto the legacy train-ticket API or fall back to synthetic OCR. +- This contract applies to upload limits, provider requests, normalization, + required-field validation, duplicate detection, cache reuse, retry behavior + and database persistence. + +### 2. Signatures + +- Runtime env: `OCR_PROVIDER=tencent`, `TENCENT_SECRET_ID`, + `TENCENT_SECRET_KEY`, `TENCENT_OCR_REGION` and `OCR_TIMEOUT_MS`. +- Upload: `POST /api/invoice-pool/upload` with one + `application/pdf` multipart `file`, at most `7 * 1024 * 1024` bytes. +- Provider call: + `runTencentGeneralInvoiceOcr(config, { imageBase64, sourceMimeType })` calls + `client.RecognizeGeneralInvoice(request)`. +- Persistence boundary: + `sanitizeOcrResultForPersistence(ocrProvider, normalized)` must run before + every Tencent `OcrResult.result_json` create, cache return or cache repair. +- Duplicate boundary: + `shouldDeduplicateByInvoiceNumberOnly(normalized)` is true only for a known + no-code category with a non-empty invoice number. + +### 3. Contracts + +- Each PDF contains one invoice. Tencent receives page 1 only with + `EnablePdf=true`, `PdfPageNumber=1`, `EnableMultiplePage=false`, + `EnableCutImage=false`, `EnableItemPolygon=false`, `EnableQRCode=false`, + `EnableSeal=false` and `EnableOther=false`. +- Supported categories are `vat_invoice`, `full_vat_invoice` and + `electronic_train_ticket` (`ElectronicTrainTicketFull`). Other subtypes and + multiple detected invoices fail closed. +- Required normalized fields: + - `vat_invoice`: `invoiceCode`, `invoiceNumber`, `invoiceDate`, + `totalAmount`, `sellerName`. + - `full_vat_invoice`: `invoiceNumber`, `invoiceDate`, `totalAmount`, + `sellerName`. + - `electronic_train_ticket`: `invoiceNumber`, `invoiceDate`, `totalAmount`, + `buyerName`. +- Never invent an invoice code from an invoice number or vice versa. Known + no-code full-VAT and railway tickets use invoice-number-only duplicate + detection; other categories retain code-plus-number semantics. +- Tencent persistence is allow-list only: document subtype/category/count, + normalized invoice identifiers/date/amounts/buyer/seller, approved quality + error and `pdf_native` transport mode. Never persist the raw provider + response, passenger name, identity number, QR/seal data or full ticket fields. +- Existing Tencent cache rows are not trusted as already safe. Every cache hit, + including the unique-key race path, is re-sanitized and written back before + use. +- OCR remains assistive. Finance must compare the reimbursement item, source + PDF and required payment evidence; recognition success is not automatic + approval. + +### 4. Validation & Error Matrix + +- Non-PDF source -> terminal `unsupported_source_mime_type`; do not retry. +- PDF larger than 7 MiB -> upload rejects with + `PDF 发票不能超过 7 MiB`; do not enqueue OCR. +- No invoice / more than one invoice / unsupported subtype / provider item not + `OK` -> `invoice_not_detected` / + `multiple_invoices_not_supported` / `unsupported_invoice_subtype` / + `invoice_recognition_failed`; mark OCR failed without inventing fields. +- Missing per-category fields -> + `missing_required_fields:`; mark OCR failed. +- `AuthFailure*`, `UnauthorizedOperation*`, `InvalidParameter*`, + `InvalidParameterValue*`, `FailedOperation.UnOpenError`, + `LimitExceeded.TooLargeFileError`, + `ResourceUnavailable.ResourcePackageRunOut`, + `ResourceUnavailable.InArrears` and + `ResourcesSoldOut.ChargeStatusException` -> terminal failure; do not consume + retry attempts blindly or switch to a billable/ mock provider. +- A transient provider/network error not in the terminal allowlist -> retain the + bounded existing retry policy; never expose credentials or raw responses in + the user-facing error. + +### 5. Good/Base/Bad Cases + +- Good: one railway electronic ticket PDF under 7 MiB yields + `electronic_train_ticket`, a blank code, a valid number/date/total/buyer and + participates in number-only duplicate detection; persisted JSON contains no + passenger identity fields or raw response. +- Base: one supported VAT/electronic ordinary invoice yields its required + category fields and follows the normal code-plus-number or known no-code + duplicate rule. +- Bad: send a multi-invoice PDF, parse only the first detected invoice, copy + number into code, persist `raw`, or retry after the free resource package is + exhausted. + +### 6. Tests Required + +- Adapter unit tests assert the exact `RecognizeGeneralInvoice` request flags, + PDF-only transport and normalization for VAT, full VAT and + `ElectronicTrainTicketFull`. +- Adapter error tests assert terminal classification for credentials, + authorization, invalid input, oversized input, service-not-open, + arrears/charge state and free-package exhaustion. +- OCR service/job tests assert category-specific required fields, + number-only dedupe only for known no-code categories, and no retry on terminal + errors. +- Cache tests seed a legacy Tencent result containing `raw` and sensitive + fields, then assert both normal and race cache paths return and persist only + the allow-listed shape. +- Upload route tests assert exactly 7 MiB remains accepted and any larger PDF + is rejected before OCR enqueue. +- Full backend regression, Prisma validation, sensitive-content check and + Trellis governance check remain required before commit. + +### 7. Wrong vs Correct + +#### Wrong + +```js +const result = await client.VatInvoiceOCR({ ImageBase64: imageBase64 }); +await prisma.ocrResult.create({ data: { result_json: { ...result, raw: result } } }); +``` + +#### Correct + +```js +const result = await client.RecognizeGeneralInvoice(buildGeneralInvoiceRequest(imageBase64)); +const normalized = normalizeGeneralInvoiceResult(result); +const persisted = sanitizeOcrResultForPersistence("tencent", normalized); +await prisma.ocrResult.create({ data: { result_json: persisted } }); +``` + ## Scenario: Payment Batch Receipt Attention Count ### 1. Scope / Trigger diff --git a/.trellis/tasks/07-28-travel-reimbursement-guide/check.jsonl b/.trellis/tasks/07-28-travel-reimbursement-guide/check.jsonl new file mode 100644 index 00000000..9dd3234a --- /dev/null +++ b/.trellis/tasks/07-28-travel-reimbursement-guide/check.jsonl @@ -0,0 +1 @@ +{"_example": "Fill with {\"file\": \"\", \"reason\": \"\"}. Put spec/research files only — no code paths. Run `python3 .trellis/scripts/get_context.py --mode packages` to list available specs. Delete this line once real entries are added."} diff --git a/.trellis/tasks/07-28-travel-reimbursement-guide/design.md b/.trellis/tasks/07-28-travel-reimbursement-guide/design.md new file mode 100644 index 00000000..ee0baaa8 --- /dev/null +++ b/.trellis/tasks/07-28-travel-reimbursement-guide/design.md @@ -0,0 +1,53 @@ +# 技术设计 + +## 边界 + +本任务不开发新的 OCR provider，也不开放图片发票或改变审批流。在现有腾讯 provider 内把统一入口从 `VatInvoiceOCR` 改为 `RecognizeGeneralInvoice`，增加票种感知的安全归一化和上传大小门禁，再完成生产配置恢复、真实链路验证及两份既有业务文档更新。 + +生产配置写入、容器重建、腾讯 API 调用和真实样票上传均属于单独受控步骤；每一步只在前一步通过后继续。 + +## 配置与运行拓扑 + +- 生产环境文件继续作为 OCR 配置事实源。 +- 凭证采用 OA OCR 专用的最小权限子用户密钥；不使用主账号永久密钥，不从聊天、仓库、日志、命令参数或历史残留复制。 +- 用户在现场终端静默输入凭证。自动化只检查 provider、必要字段是否非空、文件权限和配置 smoke，不回显值或长度。 +- 当前仅 server 执行 jobs，因此只重建 server，不启动 worker；重建后再次确认只有一个 OCR job executor。 +- provider 从 `mock` 直接切换为 `tencent` 只能发生在服务开通、权限、计费状态和密钥均确认后。任一前置失败时保持禁止真实票据上传，不用 mock 代替真实能力。 +- 计费只使用 `RecognizeGeneralInvoice` 对应的通用票据识别（高级版）每月 1,000 次专用免费包，不接独立火车票接口。生产固定中国大陆地域；控制台确认免费包已到账、后付费关闭，并配置费用或余额告警。免费包耗尽时服务失败关闭，应用记录额度类可解释终态，不自动付费续用。 + +## 腾讯 API 合同 + +- API：`RecognizeGeneralInvoice`，版本 `2018-11-19`，由当前腾讯 Node SDK 调用。 +- PDF 请求：Base64、第一页、关闭多页、切图、坐标、二维码和印章返回。 +- 一份 PDF 只允许解析出一张受支持票据；零张、多张、未知或不支持子类型进入明确失败状态。 +- 支持子类型按两族归一化：住宿 VAT/电子普通发票；电子发票（铁路电子客票）。 +- 质量门禁按 `documentSubtype` 选择必填字段；无代码电子票据不再伪造发票代码。 +- `OcrResult.result_json` 仅保存规范化业务字段和安全元数据，不保存 provider 原始响应或乘车人身份字段。 +- PDF 原文件上限为 7MiB，使 Base64 后保持在腾讯 10M 上限内；大小错误在上传阶段阻断。 + +## 验证数据流 + +1. 用户在腾讯云控制台确认通用票据识别（高级版）每月免费包已到账、中国大陆地域、后付费关闭和费用/余额告警。 +2. 用户创建或轮换 OA OCR 专用子用户密钥，并确认 OCR 服务可用。 +3. 用户在现场终端把 provider、密钥和地域写入生产受控环境文件。 +4. 重建 server，使新配置进入实际运行进程。 +5. 运行外部依赖 smoke、运行时 smoke、`/ready`、构建标识和 job 拓扑检查。 +6. 经样票单独授权后，通过现有 PDF 发票入口分别上传住宿电子普通发票和铁路电子客票样票。 +7. 只记录 provider、状态、尝试/完成证据和脱敏字段命中摘要；不记录票面、完整字段或原始 payload。 +8. 正常清理或归档测试业务记录，再允许杨俊提交真实票据。 + +## 文档口径 + +- `docs/员工快速上手.md` 增加一页式差旅报销步骤：火车票和住宿费拆明细，官方 PDF 绑定对应明细，两类费用均附支付截图，金额差异必须说明，提交后进入主管和财务流程。 +- `docs/报销发票附件规则.md` 固化支付证明必交、敏感字段遮挡、退款/代付/优惠差异和无截图例外口径。 +- OCR 结果仅辅助识别和查重；财务仍核对明细、PDF 原件和个人支付证明。 + +## 失败与回退 + +- adapter 聚焦测试、配置 smoke、腾讯调用或任一真实样票验证失败：停止开放真实票据，不继续制造 pending/retry 数据。 +- 若新配置导致运行异常，恢复变更前环境文件的受控副本并重建同一 server；回退后不得使用 mock 处理真实票据。 +- 样票失败记录使用可解释状态和脱敏摘要，不直接修改数据库字段绕过状态机。 + +## 历史凭证结论 + +2026 年 5 月的证据能证明真实腾讯 OCR 曾可用，但不能证明当时密钥仍有效或满足当前最小权限。当前生产机、历史容器、同机旧环境文件、开发机环境文件及 shell 历史均无可恢复凭证；数据库/附件备份也不包含环境密钥。因此本任务不设计“从旧部署复制密钥”的路径。 diff --git a/.trellis/tasks/07-28-travel-reimbursement-guide/implement.jsonl b/.trellis/tasks/07-28-travel-reimbursement-guide/implement.jsonl new file mode 100644 index 00000000..9dd3234a --- /dev/null +++ b/.trellis/tasks/07-28-travel-reimbursement-guide/implement.jsonl @@ -0,0 +1 @@ +{"_example": "Fill with {\"file\": \"\", \"reason\": \"\"}. Put spec/research files only — no code paths. Run `python3 .trellis/scripts/get_context.py --mode packages` to list available specs. Delete this line once real entries are added."} diff --git a/.trellis/tasks/07-28-travel-reimbursement-guide/implement.md b/.trellis/tasks/07-28-travel-reimbursement-guide/implement.md new file mode 100644 index 00000000..d99cdbd6 --- /dev/null +++ b/.trellis/tasks/07-28-travel-reimbursement-guide/implement.md @@ -0,0 +1,64 @@ +# 实施计划 + +## 1. 外部前置门禁 + +- [x] 用户确认采用 OA OCR 专用最小权限子用户密钥。 +- [ ] 用户在腾讯云侧创建或轮换该专用密钥。 +- [ ] 在实际账号确认通用票据识别（高级版）每月 1,000 次专用免费包已到账，生产使用中国大陆地域。 +- [ ] 在 OCR 控制台设置页确认后付费关闭，并配置费用或余额告警；额度耗尽时接受 OCR 失败关闭。 +- [ ] 确认腾讯 OCR 服务和 `ocr:RecognizeGeneralInvoice` 最小权限可用，不接独立火车票接口。 +- [ ] 确认住宿电子普通发票和铁路电子客票各一份 PDF 样票的测试授权及仓库外受控位置；不得向 Codex 或聊天提供票据。 + +## 2. 腾讯 adapter 修正 + +- [x] 将现有腾讯调用改为 `RecognizeGeneralInvoice`，固定单页 PDF 和最小返回选项。 +- [x] 分别归一化住宿 VAT/电子普通发票和 `ElectronicTrainTicketFull`。 +- [x] 按安全票种标识执行必填字段检查；删除代码/号码互相伪回填。 +- [x] 不持久化腾讯原始响应、乘车人姓名、证件号或完整票面字段；命中旧版腾讯缓存时按同一白名单压缩。 +- [x] 发票上传增加 7MiB OCR 输入上限和明确错误。 +- [x] 增加两类票种、无代码、多票/不支持票据、敏感字段、大小门禁和 provider 错误聚焦测试。 + +## 3. 生产配置 + +- [ ] 对当前生产环境文件制作受控、限权、短期回退副本，不输出内容。 +- [ ] 由用户在现场终端静默写入 `OCR_PROVIDER=tencent` 及专用凭证；地域沿用核验后的配置。 +- [ ] 脱敏检查 provider、凭证非空、环境文件权限和配置合同。 +- [ ] 仅重建实际执行 jobs 的 server，不启动独立 worker。 + +回退点：外部依赖 smoke 前若任一配置门禁失败，恢复变更前环境文件并重建 server；保持真实票据禁止上传。 + +## 4. 生产与真实能力验证 + +- [ ] 检查公网 `/ready` 和预期构建标识。 +- [ ] 在 server 内运行现有 external deps 和运行时 smoke，不输出 Secret 或原始响应。 +- [ ] 确认实际 provider 为 `tencent`，且只有一个 OCR job executor。 +- [ ] 经单独样票批准后分别运行住宿电子普通发票和铁路电子客票 live OCR PDF 验收。 +- [ ] 只保存脱敏状态、耗时、provider 和字段命中摘要；确认不存在 mock 调用证据。 +- [ ] 按正常 API/业务生命周期清理或归档测试记录，并执行只读残留检查。 + +回退点：真实调用失败时停止样票和真实票据上传，保留可解释失败摘要；不得改库伪造成功或切回 mock 处理真实票据。 + +## 5. 文档更新 + +- [x] 更新 `docs/员工快速上手.md` 的差旅报销一页指南。 +- [x] 更新 `docs/报销发票附件规则.md` 的支付截图、实报实销差异、退款/代付和人工复核规则。 +- [x] 不新建平行员工指南，不写入真实票据、支付信息、账号或环境路径。 + +## 6. 验证与审查 + +- [x] 静态核对页面/API 上传格式、明细绑定和员工/财务字段边界。 +- [x] 运行腾讯 adapter、OCR service/job 和发票上传聚焦测试。 +- [x] `cd server && npm run prisma:validate` +- [x] `node scripts/qa/check-process-contract.mjs --baseline --json` +- [x] `node scripts/qa/audit-docs.mjs --check-registry`（0 个错误；22 个既有未注册文档 warning，本任务未新增） +- [x] `node scripts/qa/check-trellis-governance.mjs` +- [x] `git diff --check` +- [ ] 对生产配置、样票证据和文档口径进行只读复核。 +- [ ] 部署与公网检查均通过后，才通知杨俊可以提交真实差旅报销。 + +## 风险文件与操作 + +- 生产环境文件：包含 Secret，只允许用户现场静默输入；不得由补丁、聊天或命令参数传入。 +- server 重建：短暂影响 OA；仅操作当前实际 server，保留回退配置。 +- 真实样票：包含发票和个人/企业信息；只留在仓库外受控位置。 +- 文档：员工指南必须短且与生产费用类型、PDF-only 和附件入口一致，避免把财务付款回单误写成员工材料。 diff --git a/.trellis/tasks/07-28-travel-reimbursement-guide/prd.md b/.trellis/tasks/07-28-travel-reimbursement-guide/prd.md new file mode 100644 index 00000000..98c29dbf --- /dev/null +++ b/.trellis/tasks/07-28-travel-reimbursement-guide/prd.md @@ -0,0 +1,80 @@ +# 差旅报销指南与腾讯 OCR 正式启用 + +## Goal + +先把生产发票识别从 mock 切换到真实腾讯 OCR 并取得授权样票证据，再让杨俊及后续员工一次看懂差旅报销需要提交什么、分别上传到哪里，以及财务如何按“明细—发票—个人支付证明”复核。 + +## Background + +- 当前报销单明细包含费用类型、发生日期、金额和说明；发票绑定到具体明细，补充附件属于整张报销单。 +- 发票入口只接受 PDF；补充附件接受 PDF、JPG、PNG、WebP，可用于支付截图、订单和情况说明。 +- 员工不填写财务付款凭证号、付款日期或付款渠道；这些字段属于审批通过后的财务处理。 +- 生产已有 37 个 active 费用类型，其中“火车票”“住宿费”“机票”“打车费”“差旅餐补”各唯一存在；当前 active 费用标准为 0，不会自动执行金额上限。 +- 用户已决定：火车票和住宿费均要求员工提交个人支付截图。 +- 生产只读核验发现当前运行配置为 `OCR_PROVIDER=mock`，`TENCENT_SECRET_ID` 和 `TENCENT_SECRET_KEY` 均未配置；server 自身运行 jobs，当前没有独立 worker。 +- 2026-05-05 的试运行证据证明生产腾讯 OCR 当时曾通过真实 PDF 验证；但当前生产环境、同机历史环境文件、已停止应用容器、开发机环境文件和 shell 历史均没有可恢复的旧腾讯密钥。现有自动备份只覆盖数据库和附件，不覆盖环境密钥，因此不能从备份恢复该凭证。 +- 官方 API 和当前实现复核发现：现有腾讯 adapter 调用 `VatInvoiceOCR`，适合住宿增值税/电子普通发票，但不是统一覆盖电子发票（铁路电子客票）的推荐接口。官方 `RecognizeGeneralInvoice` 同时覆盖住宿常见 VAT 子类型和 `ElectronicTrainTicketFull`，当前锁定 SDK 已支持该方法。 +- 当前实现会对所有票据强制要求发票代码和销售方，并在代码/号码缺失时互相回填；铁路电子客票及部分数电票会被误判失败或污染查重语义。现有 `raw` 结果还会持久化腾讯完整响应，铁路票返回可能包含乘车人证件信息。 +- 腾讯要求文件 Base64 编码后不超过 10M，现有 OA 允许 10MiB 原始 PDF，存在上传成功后 OCR 必然超限的区间。 +- 用户决定不接受 `off` 人工模式，要求配置并正式启用真实 OCR；在凭证配置和授权样票验证通过前，杨俊不得上传真实发票。 +- 用户已确认采用推荐的凭证方案：在腾讯云创建或轮换 OA OCR 专用最小权限子用户密钥，并由用户在现场终端静默写入生产。 + +## Scope + +- 核对报销明细、PDF 发票、补充附件、支付截图和财务付款记录的实际页面/API边界。 +- 更新 `docs/员工快速上手.md`，增加一页可直接照做的差旅报销指南。 +- 更新 `docs/报销发票附件规则.md`，固化火车票/住宿费支付证明、实报实销和 OCR 关闭期间人工核对规则。 +- 配置并正式启用生产腾讯 OCR，完成配置门禁、受控发布、公网检查和授权 PDF 样票验证。 +- 在现有腾讯 provider 内改用通用票据识别高级版，补齐住宿电子普通发票和铁路电子客票的安全归一化、质量门禁、大小限制和测试。 + +## Requirements + +- 火车票与住宿费必须拆为两条报销明细；每条分别填写费用类型、发生日期、实际合规金额和事由说明。 +- 正式电子票据/发票使用官方 PDF，通过对应明细的“添加发票”入口上传并绑定；截图不得冒充正式发票。 +- 火车票和住宿费的个人支付截图通过“附件材料/补充附件”上传，并用文件名或说明明确对应哪条明细。 +- 支付截图仅保留商户、金额、时间和支付成功状态；银行卡号、支付账号、订单号等无关敏感字段应遮挡。 +- 报销金额以个人实际支付的合规金额为准，原则上不得超过有效发票金额；优惠券、多人合并开票、部分自费或其他差异必须在明细中说明。 +- 已退款、公司直接支付或他人承担的金额不得重复报销；确实无法提供支付截图时写情况说明，由财务决定退回或例外接受，不得伪造凭证。 +- 腾讯 OCR 识别字段用于匹配、查重和财务复核，但不替代支付真实性、费用合理性或发票法律状态的人工判断；财务仍核对明细金额、PDF 原件和支付证明。 +- 腾讯 provider 使用 `RecognizeGeneralInvoice`，每份 PDF 只识别第一页且只接受一张受支持票据；不请求切图、字段坐标、二维码或印章数据。 +- 住宿 VAT/电子普通发票与铁路电子客票按各自官方返回结构归一化；铁路电子客票不强制要求不存在的发票代码或销售方，不得用发票号码伪造代码。 +- 持久化结果只保留业务需要的规范化字段、安全票种标识和传输模式，不保存腾讯原始响应、乘车人姓名、证件号或完整票面字段。 +- 正式 PDF 发票增加不高于 7MiB 的 OCR 输入门禁，避免 Base64 后超过腾讯 10M 限制；超限在上传阶段给出明确提示，不进入重试队列。 +- 腾讯云凭证使用最小权限子用户或专用身份，不使用主账号永久密钥；凭证只通过用户现场终端写入生产受控环境文件，不进入聊天、命令参数、日志、仓库或 Trellis。 +- 不复用来源、权限或有效性无法确认的历史密钥；优先在腾讯云侧为 OA OCR 创建或轮换专用子用户密钥，并撤销确认不再使用的旧密钥。 +- 正式启用前需确认 OCR 服务已开通、调用费用/免费额度和告警边界；无凭证、权限不足、欠费或 API 未开通均不得把 provider 切成 `tencent`。 +- 只使用 `RecognizeGeneralInvoice` 的通用票据识别（高级版）每月 1,000 次专用免费额度，不接仅老客户续费的独立火车票接口；生产使用中国大陆地域，必须在实际账号确认免费包到账、关闭后付费并配置费用或余额告警。额度耗尽时允许 OCR 明确失败，不得自动产生费用。 +- 当前无独立 worker 且 server 自身运行 jobs，配置变更只重建实际加载配置的 server；不得额外启动第二个 OCR job executor。 +- 真实能力必须使用已授权 PDF 样票验证；缺少授权样票只能证明配置齐全，不能宣称真实 OCR 已通过。 +- 指南明确审批链：员工提交、直属主管审批、财务复核；财务付款和回单不由员工预填。 +- 文档不放真实票据、支付截图、账号、订单号、银行信息或完整业务数据。 + +## Acceptance Criteria + +- [ ] 员工指南能在一页内回答“交什么、传到哪里、金额怎么填、哪些材料不能混传、提交后谁处理”。 +- [ ] 指南内容与当前页面、服务端格式限制及生产费用类型一致。 +- [ ] 业务事实源明确火车票和住宿费支付截图为必交补充材料，并包含遮挡、差异、退款和无截图例外规则。 +- [ ] 腾讯 adapter 使用 `RecognizeGeneralInvoice`，聚焦测试覆盖住宿电子普通发票、铁路电子客票、无代码票据、多票/不支持票据、大小门禁和敏感原始字段不持久化。 +- [ ] 腾讯云控制台确认通用票据识别（高级版）每月 1,000 次免费包已到账、中国大陆地域、后付费关闭和费用/余额告警可用；额度耗尽不产生后付费。 +- [ ] 生产实际 provider 为 `tencent`，只授予 `ocr:RecognizeGeneralInvoice`，两个凭证字段已配置但不泄露，external deps smoke、`/ready` 和运行 job 拓扑通过。 +- [ ] 住宿电子普通发票和铁路电子客票各一份已授权 PDF 经真实腾讯 OCR 进入可解释终态，并证明没有调用 mock；样票和原始识别 payload 不进入仓库或聊天。 +- [ ] 明确 OCR 只辅助字段识别/查重，财务仍执行明细—PDF 原件—支付证明复核。 +- [ ] 在真实 OCR 验收完成前不得仅因指南已完成就通知杨俊可以提交。 +- [ ] 全程不把真实票据、支付截图、完整账号、银行资料、Secret 或原始 payload 写入仓库、任务或聊天。 + +## Out of Scope + +- 新增第二个 OCR provider、开放图片发票、自动验真或增加人工发票字段。 +- 本轮不做通用发票平台或覆盖全部票种；仅支持现有住宿 VAT/电子普通发票和电子发票（铁路电子客票）主路径。 +- 修改报销页面、上传格式、审批规则、费用标准或费用类型。 +- 代替杨俊上传票据、创建报销、提交审批，或代替主管/财务处理。 +- 制定覆盖所有费用类别的完整财务制度；本轮只收口首笔差旅的火车票和住宿费。 + +## Test Plan + +- 静态核对前后端上传格式、明细绑定、员工/财务字段边界。 +- 运行腾讯 adapter 和 OCR job 聚焦单测，覆盖两类票种、缺字段、错误分类、大小限制和敏感字段最小化。 +- 生产只读核对费用类型、OCR provider 和必要运行状态，仅输出类别与计数。 +- 运行文档注册、敏感内容、流程合同、Trellis governance 和 `git diff --check`。 +- 启用真实 OCR 后验证 `/ready`、构建标识、运行 provider=`tencent`、单一 job executor 和配置 smoke。 +- 经单独样票批准后运行现有 live OCR 验收，输出只保留状态、耗时和脱敏字段命中摘要，并按正常业务/API 生命周期清理或归档测试记录。 diff --git a/.trellis/tasks/07-28-travel-reimbursement-guide/research/tencent-ocr-api-review.md b/.trellis/tasks/07-28-travel-reimbursement-guide/research/tencent-ocr-api-review.md new file mode 100644 index 00000000..30d4e200 --- /dev/null +++ b/.trellis/tasks/07-28-travel-reimbursement-guide/research/tencent-ocr-api-review.md @@ -0,0 +1,88 @@ +# 腾讯 OCR API 调研 + +调研日期：2026-07-28。仅查阅腾讯云官方文档、当前锁定 SDK 和本地代码；未创建密钥、未调用付费 API、未上传票据。 + +## 官方资料 + +- 通用票据识别（高级版）：https://cloud.tencent.com/document/api/866/90802 +- 增值税发票识别：https://cloud.tencent.com/document/api/866/36210 +- OCR 控制台访问管理：https://cloud.tencent.com/document/product/866/102912 +- 免费额度：https://cloud.tencent.com/document/product/866/35945 +- 计费概述：https://cloud.tencent.com/document/product/866/17619 +- 混贴票据计费说明：https://cloud.tencent.com/document/product/866/48522 +- 错误码计费说明：https://cloud.tencent.com/document/product/866/45470 +- 新手常见问题：https://cloud.tencent.com/document/product/866/43818 +- 欠费说明：https://cloud.tencent.com/document/product/866/30576 + +## 结论 + +### 1. 当前接口不适合作为两类差旅票据的统一入口 + +当前代码调用 `VatInvoiceOCR`。官方说明明确覆盖增值税专票、普票、电子专票、电子普票和数电普通/专用发票，适合住宿电子普通发票；但官方用于统一识别火车票、电子发票（铁路电子客票）和增值税发票的是 `RecognizeGeneralInvoice`。 + +`RecognizeGeneralInvoice` 明确返回 `ElectronicTrainTicketFull`，同时覆盖 `VatElectronicCommonInvoice`、`VatElectronicInvoiceFull` 等住宿发票常见子类型。因此本轮应在现有腾讯 provider 内改用该接口，而不是直接给当前 `VatInvoiceOCR` 配密钥。 + +### 2. 请求合同 + +- API 版本仍为 `2018-11-19`，请求域名为 `ocr.tencentcloudapi.com`。 +- 当前锁定的 Node SDK 已包含 `RecognizeGeneralInvoice`；无需为了该方法先升级依赖。 +- PDF 通过 Base64 传输，不需要公开文件 URL。 +- Base64 编码后不得超过 10M；为避免 OA 的 10MiB 原文件上限在编码后超限，正式发票应增加不高于 7MiB 的 OCR 输入门禁。 +- 本系统一份 PDF 对应一张发票，只识别第一页：`EnablePdf=true`、`PdfPageNumber=1`、`EnableMultiplePage=false`。 +- 不请求切图、字段坐标、二维码或印章返回，避免无用数据和额外敏感内容。 +- 默认接口频率为 5 次/秒；当前 job 串行处理，远低于该上限。 + +### 3. 返回结构和业务字段 + +- 住宿增值税/电子普通发票：从对应 VAT 子类型归一化发票代码（存在时）、号码、日期、不含税金额、税额、价税合计、销售方和购买方。 +- 电子发票（铁路电子客票）：从 `ElectronicTrainTicketFull` 归一化发票号码、开票日期、价税合计、税额和购买方；该票种可以没有发票代码和销售方，不能继续套用现有 VAT 五字段必填规则。 +- 归一化结果需要保留安全的 `documentSubtype`，用它选择必填字段和脱敏处理；不得把乘车人姓名、证件号、完整票面字段或腾讯原始响应写入 `OcrResult.result_json`。 +- 电子票据没有发票代码时，不再把发票号码复制到代码字段。文件 SHA-256 查重继续生效；号码级查重需要按已知无代码子类型做显式规则，不能依赖伪造代码绕过现有复合键。 + +### 4. CAM 最小权限 + +腾讯官方确认 OCR 支持接口级自定义授权，接口级授权的 `resource` 必须为 `*`。运行身份只需： + +```json +{ + "version": "2.0", + "statement": [ + { + "effect": "allow", + "action": ["ocr:RecognizeGeneralInvoice"], + "resource": ["*"] + } + ] +} +``` + +运行子用户不需要 `QcloudOCRFullAccess`，也不需要开通/关闭服务或控制台管理权限。OCR 服务开通、计费设置和告警由管理员在控制台完成。 + +### 5. 免费接口选择、计费和错误 + +- 官方免费额度表明确列出：`通用票据识别（高级版）` 使用同名专用资源包，免费额度为每月 1,000 次，当月有效、每月发放。它不是与其他票据接口共用的模糊额度。 +- `增值税发票识别` 也有每月 1,000 次专用免费额度，但只能解决住宿常见 VAT/电子普通发票，不能统一覆盖电子发票（铁路电子客票）。 +- 独立的 `火车票识别` 位于官方“仅老客户续费相关接口”区域，且增加第二套 API、权限、归一化和错误处理，不适合作为当前新接入方案。 +- 因此最省钱且最简单的方案是只接 `RecognizeGeneralInvoice`：一个接口同时覆盖住宿和铁路电子客票，并消耗通用票据识别（高级版）的每月 1,000 次专用免费额度。按当前 8 人规模通常足够，但免费政策不是永久合同，上线前仍必须在实际账号控制台确认免费包已到账。 +- 高级版免费额度耗尽后，如已购买预付费包则继续扣包；如开通后付费则可能继续计费。官方当前后付费起价为 0.15 元/次。 +- 为确保零费用，生产应使用中国大陆地域，在 OCR 控制台设置页确认后付费关闭；免费额度耗尽后允许 OCR 不可用并进入可解释失败，不为保持识别可用而自动付费。境外地域按国际站后付费计费，不能抵扣国内账号资源包，因此不在本轮方案内。 +- 官方 FAQ 说明，2021 年 12 月 14 日前开通 OCR 的客户在资源包耗尽后可能自动开通后付费；无论账号何时开通，都以实际控制台开关状态为准，不能只凭开通日期推断不会收费。 +- 通用票据识别高级版按实际识别出的发票张数计费；增值税发票进入大票计费项，铁路电子客票/全电发票进入小票计费项。 +- 每份 PDF 必须只含一张票据，使一次正常识别对应一次额度扣减；混贴或多票文件可能按实际识别张数扣减，应用继续拒绝多票结果。 +- 免费/付费资源包耗尽且未开后付费时会不可用；应用需把它分类为额度/计费门禁失败，而不是票据质量失败。腾讯云侧同时设置费用或余额告警作为兜底。 +- 高级版接口官方错误码已核对：`ResourceUnavailable.ResourcePackageRunOut` 表示资源包耗尽，`ResourceUnavailable.InArrears` 表示欠费，`ResourcesSoldOut.ChargeStatusException` 表示计费状态异常，`FailedOperation.UnOpenError` 表示服务未开通，`LimitExceeded.TooLargeFileError` 表示文件过大。这些状态对同一输入重试没有收益，应用应一次终止并给出配置/额度类提示。 +- 大部分失败不计费，但官方明确有少量失败错误也可能计费。应用必须保留有限重试和可解释终态，不能无限重试。 +- 需要区别处理未开通、欠费/资源包耗尽、文件过大、图片解码/模糊和一般 OCR 失败；不得把配置或额度错误伪装成票据质量问题。 + +## 当前实现差距 + +1. `server/src/services/ocr/tencent.js` 调用 `VatInvoiceOCR`，没有铁路电子客票结构解析。 +2. `server/src/services/ocr/index.js` 对所有 `vat` 强制要求代码、号码、日期、金额、销售方；铁路电子客票和部分数电票会被误判失败。 +3. 现有归一化会在代码/号码缺失时互相回填，掩盖无代码电子票据并污染复合查重语义。 +4. 现有归一化把腾讯完整响应放进 `raw`，随后持久化到 `OcrResult.result_json`；改用高级版后可能保存乘车人证件信息。 +5. OA 发票上传使用通用 10MiB 上限，可能产生“上传成功、腾讯因 Base64 超限失败”的死路。 +6. 旧版腾讯 `OcrResult.result_json` 可能仍带 `raw`；新 adapter 不再生成还不够，命中历史缓存时也必须先按安全白名单压缩，避免旧格式绕过新隐私边界。 + +## 推荐 + +只使用 `RecognizeGeneralInvoice`，不拆成 `VatInvoiceOCR + TrainTicketOCR` 两个接口。先完成腾讯 adapter、票种归一化、按票种质量门禁、敏感响应最小化、7MiB PDF 门禁和聚焦测试；再由用户在实际腾讯云账号确认高级版每月 1,000 次免费包已到账、中国大陆地域、后付费关闭和费用告警，最后创建最小权限密钥并部署。住宿发票和铁路电子客票各用一份已授权 PDF 做真实验收；在此之前不允许真实票据进入当前 `mock` 环境。 diff --git a/.trellis/tasks/07-28-travel-reimbursement-guide/task.json b/.trellis/tasks/07-28-travel-reimbursement-guide/task.json new file mode 100644 index 00000000..3b7af506 --- /dev/null +++ b/.trellis/tasks/07-28-travel-reimbursement-guide/task.json @@ -0,0 +1,32 @@ +{ + "id": "travel-reimbursement-guide", + "name": "travel-reimbursement-guide", + "title": "差旅报销指南与腾讯OCR正式启用", + "description": "配置并验证生产腾讯OCR，随后固化火车票、住宿费发票和支付截图的提交位置、实报实销及财务复核规则。", + "status": "in_progress", + "dev_type": null, + "scope": "生产腾讯OCR启用、授权PDF样票验证及差旅报销员工指南与支付凭证规则", + "package": null, + "priority": "P2", + "creator": "shunli", + "assignee": "shunli", + "createdAt": "2026-07-28", + "completedAt": null, + "branch": null, + "base_branch": "main", + "worktree_path": null, + "commit": null, + "pr_url": null, + "subtasks": [], + "children": [], + "parent": null, + "relatedFiles": [ + "docs/员工快速上手.md", + "docs/报销发票附件规则.md", + "server/src/routes/reimbursements.js", + "server/src/routes/invoicePool.js", + "client/src/pages/ReimbursementDetailPage.tsx" + ], + "notes": "", + "meta": {} +} \ No newline at end of file diff --git a/docs/员工快速上手.md b/docs/员工快速上手.md index 77a10987..cf41117e 100644 --- a/docs/员工快速上手.md +++ b/docs/员工快速上手.md @@ -49,6 +49,21 @@ - OAOPS1 期间请用真实业务数据完成报销、采购/费用、合同台账和项目关联；遇到提交、审批、付款、导出或权限阻断时，记录页面、步骤和影响等级，让管理员当天建修复合同或暂停对应流程。 - OALAUNCH1 收口期内，请把真实业务问题按“阻断 / 非阻断 / 建议”告诉管理员；不要只在聊天里说“系统有问题”。阻断包括无法提交、无法审批、付款/AP 无法闭环、越权可见、合同或资产敏感信息泄露、备份恢复或导出证据缺失。 +### 差旅报销一页指南（火车票 + 住宿费） + +1. 从企业微信工作台进入海音达工作台，打开 `报销`，新建报销单。 +2. 把费用拆成两条明细，不要合并填写： + - 火车票：费用类型选“火车票”，填写乘车日期、个人实际支付的合规金额和事由，例如“项目出差往返高铁”。 + - 住宿费：费用类型选“住宿费”，填写实际住宿日期、个人实际支付的合规金额和事由，例如“项目出差住宿”。 +3. 在每条明细的“添加发票”入口上传对应的官方 PDF：火车电子票据 PDF 绑定火车票明细，住宿电子普通发票 PDF 绑定住宿费明细。每个 PDF 只放一张票据、只识别第一页，单个文件不能超过 7 MiB；图片、照片和支付截图不能当作正式发票上传。 +4. 火车票和住宿费都必须另附个人支付截图。支付截图通过整张报销单的“上传附件 / 补充材料”入口上传，并在文件名或说明中写清对应“火车票”还是“住宿费”。 +5. 支付截图应看得出商户、金额、支付时间和支付成功状态；银行卡号、支付账号、订单号等与复核无关的信息先遮挡。 +6. 明细金额填写个人实际支付的合规金额，原则上不得超过有效发票金额。使用优惠券、多人合并开票、部分自费或发票金额与实际支付金额不一致时，必须在明细说明中写清原因。 +7. 已退款、公司直接支付或他人承担的金额不能再次报销。确实无法提供支付截图时，上传情况说明，由财务决定退回还是例外接受，不得补造凭证。 +8. 提交前逐项检查：两条明细、两份对应 PDF、两类支付证明和金额说明都已齐全。提交后由直属主管审批、财务复核；实际付款和付款回单由财务处理，员工不预填。 + +OCR 会辅助识别发票字段和查重，但不代表系统已经确认费用真实合规。财务仍会同时核对明细金额、PDF 原件和个人支付证明；三者不一致时会退回补充或更正。 + ## 7. 入职 / 离职时你会看到什么 - 入职时 HR 会优先在企业微信/人事助手维护人员资料，OA 后续同步生成账号、角色和绑定。 - 离职前需要归还固定资产，并交接自己负责或保管的电子资料、系统、代码仓库和备份资产；企业微信和 OA 的账号状态会按流程同步停用或禁用。 diff --git a/docs/报销发票附件规则.md b/docs/报销发票附件规则.md index d5cf6bbf..b8c58818 100644 --- a/docs/报销发票附件规则.md +++ b/docs/报销发票附件规则.md @@ -38,12 +38,16 @@ R5 公网验收后确认：正式发票/OCR 上传入口只接受 PDF 发票文 ## 上传与 OCR - 发票/OCR 上传只允许 PDF 发票文件；图片、拍照直传和 HTML `capture` 入口不作为正式发票上传方式。 +- 单个正式发票 PDF 不能超过 7 MiB，每份文件第一页只能包含一张票据；超过限制、多票混贴或不支持的票种在进入报销前返回可理解失败，不进入无限重试。 - PC 票据中心、PC 报销详情、移动报销详情和企业微信 WebView 内的发票入口都必须执行 PDF-only 口径。 - 上传后页面必须显示明确状态，不应长期停留在“上传中”。 - OCR 完成后，添加发票弹窗应自动刷新并尝试添加可用发票。 - OCR 失败、重复、已使用、他人已使用都要显示可理解原因。 - 刷新按钮必须有明显反馈，不能让用户误以为没点中。 - 正式环境不得使用 `OCR_PROVIDER=mock` 处理真实票据；mock 仅用于本地或明确隔离的合成测试。 +- 腾讯真实 OCR 统一使用通用票据识别高级版 `RecognizeGeneralInvoice`，当前正式范围仅为住宿 VAT/电子普通发票和电子发票（铁路电子客票）。铁路电子客票不伪造发票代码或销售方；无代码票据仅在已知安全票种内启用发票号码查重。 +- 腾讯 OCR 持久化结果只保留票种、发票号码/代码、日期、金额、购销方和 PDF 传输模式等业务必需字段；不得保存腾讯原始响应、乘车人姓名、证件号、完整票面字段、切图、二维码、印章或字段坐标。命中旧版腾讯缓存时也必须先按同一白名单压缩。 +- 资源包耗尽、欠费、计费异常、服务未开通、凭证/权限错误和无效文件属于配置、额度或输入终态，不得伪装成票据内容质量问题，也不得对同一文件无限重试。 - 未配置真实 OCR 时使用 `OCR_PROVIDER=off`：PDF 直接进入 `available`，不入 OCR 队列、不生成发票代码、号码、日期、购销方或金额。员工仍可把票据添加到已有明细并提交，财务必须查看 PDF 原件人工核对。 - off 模式下文件 SHA-256 查重继续有效，依赖发票代码和号码的查重暂不可用。对当前小团队可短期接受；强制人工补录或接入腾讯 OCR 应另行规划，不能在紧急报销中临时增加门槛。 - 既有 `InvoicePool.ocr_provider` 为非空枚举且默认 `tencent`，所以 off 模式的新记录仍可能带该历史默认标记。判定是否实际调用 OCR 必须同时看 provider 配置、`ocr_attempts`、请求/完成时间和 OCR 结果，不能只看 `ocr_provider`。 @@ -127,6 +131,20 @@ R5 公网验收后确认：正式发票/OCR 上传入口只接受 PDF 发票文 - 报销页只显示付款状态、回单状态、查看付款或补回单入口；标记已付款、取消付款、批量付款和执行付款必须进入付款/财务处理详情，不能放在报销列表或普通报销详情里直接执行。 - 报销页展示项目、采购、合同、资产候选或付款记录时只显示摘要和跳转，不复制对方模块的审批、付款、合同正文下载、资产建卡或高敏附件操作。 +## 差旅实报实销与支付证明 + +首笔差旅报销按“明细—正式发票—个人支付证明”三方对应复核，当前火车票和住宿费执行以下规则： + +- 火车票和住宿费必须拆成不同明细，分别选择生产已有的“火车票”和“住宿费”费用类型，填写发生日期、个人实际支付的合规金额和事由。 +- 火车电子票据、住宿电子普通发票等正式票据必须使用官方 PDF，从对应明细的发票入口上传并绑定；发票截图、支付截图或拍照图片不得冒充正式发票。 +- 火车票和住宿费均必须提供个人支付截图。支付截图属于整张报销单的补充附件，不绑定到发票记录；文件名或附件说明必须能对应到具体费用明细。 +- 支付截图保留商户、金额、支付时间和支付成功状态即可；银行卡号、支付账号、订单号等与审批无关的信息应先遮挡，不要求员工提供完整支付账户或银行流水。 +- 明细金额以员工个人实际支付的合规金额为准，原则上不得超过有效发票金额。优惠券、多人合并开票、部分自费、票面金额与实付金额不一致等情况必须在明细说明中写明差异。 +- 已退款、公司直接支付或由他人承担的金额不得重复报销。发生部分退款时，只能申报扣除退款后的个人实际承担金额，并附情况说明。 +- 确实无法提供支付截图时，员工必须上传情况说明；财务可以退回补充，也可以在有其他充分证据时例外接受并留痕。不得伪造、补造或借用他人支付凭证。 +- OCR 只辅助识别发票字段、匹配和查重，不判断支付真实性、费用合理性或发票法律状态。财务复核时必须同时查看明细金额、PDF 原件和个人支付证明；三者不一致时退回更正或补充说明。 +- 财务付款凭证号、付款日期、付款渠道和付款回单属于审批通过后的财务处理，不是员工提交报销时填写的个人支付证明。 + ## PC 与手机体验 PC 端： diff --git a/server/src/jobs/ocrJob.js b/server/src/jobs/ocrJob.js index c254c748..39d24e10 100644 --- a/server/src/jobs/ocrJob.js +++ b/server/src/jobs/ocrJob.js @@ -1,6 +1,11 @@ const cron = require("node-cron"); -const { buildMissingRequiredOcrError, runOcrForInvoice } = require("../services/ocr"); +const { + buildMissingRequiredOcrError, + runOcrForInvoice, + sanitizeOcrResultForPersistence, + shouldDeduplicateByInvoiceNumberOnly, +} = require("../services/ocr"); const { publishInvoiceUpdated } = require("../services/events"); const { resolveInvoiceDedupBlocker } = require("../services/invoiceGovernance"); const { readFileBuffer } = require("../services/storage"); @@ -117,18 +122,29 @@ async function getOrCreateOcrResult(prisma, config, claimedInvoice, imageBase64) }); if (cached) { + const normalized = sanitizeOcrResultForPersistence(ocrProvider, cached.result_json); + if (ocrProvider === "tencent") { + await prisma.ocrResult.update({ + where: { id: cached.id }, + data: { result_json: normalized }, + }); + } + return { ocrResultId: cached.id, - normalized: cached.result_json, + normalized, }; } - const normalized = await runOcrForInvoice({ - config, - ocrType: claimedInvoice.ocr_type, - imageBase64, - sourceMimeType: claimedInvoice.mime_type, - }); + const normalized = sanitizeOcrResultForPersistence( + ocrProvider, + await runOcrForInvoice({ + config, + ocrType: claimedInvoice.ocr_type, + imageBase64, + sourceMimeType: claimedInvoice.mime_type, + }), + ); try { const created = await prisma.ocrResult.create({ @@ -164,9 +180,17 @@ async function getOrCreateOcrResult(prisma, config, claimedInvoice, imageBase64) throw error; } + const racedNormalized = sanitizeOcrResultForPersistence(ocrProvider, raced.result_json); + if (ocrProvider === "tencent") { + await prisma.ocrResult.update({ + where: { id: raced.id }, + data: { result_json: racedNormalized }, + }); + } + return { ocrResultId: raced.id, - normalized: raced.result_json, + normalized: racedNormalized, }; } } @@ -246,10 +270,10 @@ async function markInvoiceAsQualityFailed(prisma, config, invoiceId, normalized, }); } -async function markInvoiceAsFailed(prisma, claimedInvoice, errorMessage, config) { +async function markInvoiceAsFailed(prisma, claimedInvoice, errorMessage, config, { terminal = false } = {}) { const now = new Date(); const nextAttempts = Number(claimedInvoice.ocr_attempts || 0) + 1; - const isTerminal = nextAttempts >= config.ocrMaxAttempts; + const isTerminal = terminal || nextAttempts >= config.ocrMaxAttempts; const nextStatus = isTerminal ? "ocr_failed" : "pending_ocr"; await prisma.invoicePool.update({ @@ -313,6 +337,7 @@ async function processClaimedInvoice(prisma, config, claimedInvoice, workerId, l fileSha256: claimedInvoice.file_sha256, invoiceCode: normalized.invoiceCode, invoiceNumber: normalized.invoiceNumber, + deduplicateByInvoiceNumberOnly: shouldDeduplicateByInvoiceNumberOnly(normalized), ownerEmployeeId: claimedInvoice.owner_employee_id, }, ); @@ -336,7 +361,9 @@ async function processClaimedInvoice(prisma, config, claimedInvoice, workerId, l return { outcome: "available", status: "available" }; } catch (error) { const message = trimErrorMessage(error); - const status = await markInvoiceAsFailed(prisma, claimedInvoice, message, config); + const status = await markInvoiceAsFailed(prisma, claimedInvoice, message, config, { + terminal: error?.terminal === true, + }); logger.error( { err: error, diff --git a/server/src/routes/invoicePool.js b/server/src/routes/invoicePool.js index 533f4b17..48e71d4d 100644 --- a/server/src/routes/invoicePool.js +++ b/server/src/routes/invoicePool.js @@ -9,7 +9,11 @@ const { compactDateString } = require("../utils/date"); const { sha256Hex } = require("../utils/crypto"); const { toMoneyString } = require("../utils/workflow"); const { findActiveDelegationForDelegate } = require("../utils/delegation"); -const { getMissingRequiredOcrFields, isOcrEnabledForType } = require("../services/ocr"); +const { + OCR_PDF_MAX_BYTES, + getMissingRequiredOcrFields, + isOcrEnabledForType, +} = require("../services/ocr"); const { enqueueOcrInvoice } = require("../jobs/pgBossJobs"); const { registerInvoiceVerificationRoutes } = require("./invoicePool/verification"); const { registerInvoicePoolReadRoutes } = require("./invoicePool/read"); @@ -353,6 +357,11 @@ async function invoicePoolRoutes(fastify) { let fileMimeType = null; let fileSize = 0; const fields = {}; + const configuredUploadMaxBytes = Number(fastify.config.maxUploadSizeBytes); + const invoiceUploadMaxBytes = + Number.isFinite(configuredUploadMaxBytes) && configuredUploadMaxBytes > 0 + ? Math.min(configuredUploadMaxBytes, OCR_PDF_MAX_BYTES) + : OCR_PDF_MAX_BYTES; try { for await (const part of request.parts()) { @@ -380,12 +389,21 @@ async function invoicePoolRoutes(fastify) { const chunks = []; for await (const chunk of part.file) { fileSize += chunk.length; - if (fileSize > fastify.config.maxUploadSizeBytes) { + if (fileSize > invoiceUploadMaxBytes) { throw new AppError({ code: "VALID_001", - message: "参数校验失败", + message: + invoiceUploadMaxBytes === OCR_PDF_MAX_BYTES + ? "PDF 发票不能超过 7 MiB" + : "参数校验失败", statusCode: 400, - details: { file: "file_too_large" }, + details: { + file: + invoiceUploadMaxBytes === OCR_PDF_MAX_BYTES + ? "ocr_pdf_too_large" + : "file_too_large", + maxBytes: invoiceUploadMaxBytes, + }, }); } chunks.push(chunk); diff --git a/server/src/services/invoiceGovernance.js b/server/src/services/invoiceGovernance.js index 3ace0017..245dff79 100644 --- a/server/src/services/invoiceGovernance.js +++ b/server/src/services/invoiceGovernance.js @@ -14,9 +14,15 @@ const OCR_FIELD_LABELS = { invoiceDate: "开票日期", totalAmount: "总金额", sellerName: "销售方", + buyerName: "购买方", }; -function buildInvoiceDedupFilters({ fileSha256 = null, invoiceCode = null, invoiceNumber = null }) { +function buildInvoiceDedupFilters({ + fileSha256 = null, + invoiceCode = null, + invoiceNumber = null, + deduplicateByInvoiceNumberOnly = false, +}) { const filters = []; if (fileSha256) { filters.push({ file_sha256: fileSha256 }); @@ -26,6 +32,11 @@ function buildInvoiceDedupFilters({ fileSha256 = null, invoiceCode = null, invoi invoice_code: invoiceCode, invoice_number: invoiceNumber, }); + } else if (deduplicateByInvoiceNumberOnly && invoiceNumber) { + filters.push({ + invoice_code: null, + invoice_number: invoiceNumber, + }); } return filters; } @@ -217,6 +228,39 @@ function formatInvoiceOcrFailureMessage(rawError) { return "票据识别失败，请重试或人工补齐关键信息。"; } + if (rawError === "invoice_not_detected") { + return "未识别到支持的发票，请确认 PDF 第一页只有一张有效票据。"; + } + + if (rawError === "multiple_invoices_not_supported") { + return "一个 PDF 第一页只能包含一张发票，请拆分后重新上传。"; + } + + if (rawError === "unsupported_invoice_subtype") { + return "当前仅支持住宿增值税发票和铁路电子客票。"; + } + + if (rawError === "invoice_recognition_failed") { + return "腾讯 OCR 未能识别该发票，请检查 PDF 清晰度后重试。"; + } + + if ( + rawError.includes("ResourceUnavailable.ResourcePackageRunOut") || + rawError.includes("ResourceUnavailable.InArrears") || + rawError.includes("ResourcesSoldOut.ChargeStatusException") + ) { + return "票据识别额度或计费状态暂不可用，请联系管理员。"; + } + + if ( + rawError.includes("ocr_credentials_missing") || + rawError.includes("AuthFailure") || + rawError.includes("UnauthorizedOperation") || + rawError.includes("FailedOperation.UnOpenError") + ) { + return "票据识别服务配置暂不可用，请联系管理员。"; + } + if (!rawError) { return "票据识别失败，请稍后重试。"; } diff --git a/server/src/services/ocr/index.js b/server/src/services/ocr/index.js index 93c5d991..93265f8e 100644 --- a/server/src/services/ocr/index.js +++ b/server/src/services/ocr/index.js @@ -1,10 +1,47 @@ -const { runTencentVatOcr } = require("./tencent"); +const { runTencentGeneralInvoiceOcr } = require("./tencent"); const { runMockOcr } = require("./mock"); +const OCR_PDF_MAX_BYTES = 7 * 1024 * 1024; + const REQUIRED_OCR_FIELDS_BY_TYPE = { vat: ["invoiceCode", "invoiceNumber", "invoiceDate", "totalAmount", "sellerName"], }; +const REQUIRED_OCR_FIELDS_BY_DOCUMENT_CATEGORY = { + vat_invoice: ["invoiceCode", "invoiceNumber", "invoiceDate", "totalAmount", "sellerName"], + full_vat_invoice: ["invoiceNumber", "invoiceDate", "totalAmount", "sellerName"], + electronic_train_ticket: ["invoiceNumber", "invoiceDate", "totalAmount", "buyerName"], +}; + +const NUMBER_ONLY_DEDUP_DOCUMENT_CATEGORIES = new Set([ + "full_vat_invoice", + "electronic_train_ticket", +]); + +const SAFE_DOCUMENT_CATEGORIES = new Set([ + "vat_invoice", + "full_vat_invoice", + "electronic_train_ticket", +]); + +const SAFE_QUALITY_ERRORS = new Set([ + "invoice_not_detected", + "multiple_invoices_not_supported", + "invoice_recognition_failed", + "unsupported_invoice_subtype", +]); + +const SAFE_NORMALIZED_FIELDS = [ + "invoiceCode", + "invoiceNumber", + "invoiceDate", + "amount", + "taxAmount", + "totalAmount", + "sellerName", + "buyerName", +]; + function isOcrEnabledForType(config, ocrType) { if (config.ocrProvider === "mock") { return ocrType === "vat"; @@ -14,7 +51,10 @@ function isOcrEnabledForType(config, ocrType) { } function getMissingRequiredOcrFields(ocrType, normalized) { - const requiredFields = REQUIRED_OCR_FIELDS_BY_TYPE[ocrType] ?? []; + const requiredFields = + REQUIRED_OCR_FIELDS_BY_DOCUMENT_CATEGORY[normalized?.documentCategory] ?? + REQUIRED_OCR_FIELDS_BY_TYPE[ocrType] ?? + []; if (!requiredFields.length) { return []; } @@ -26,6 +66,10 @@ function getMissingRequiredOcrFields(ocrType, normalized) { } function buildMissingRequiredOcrError(ocrType, normalized) { + if (typeof normalized?.qualityError === "string" && normalized.qualityError) { + return normalized.qualityError; + } + const missingFields = getMissingRequiredOcrFields(ocrType, normalized); if (!missingFields.length) { return null; @@ -47,12 +91,63 @@ async function runOcrForInvoice({ config, ocrType, imageBase64, sourceMimeType } throw new Error(`Unsupported OCR type: ${ocrType}`); } - return runTencentVatOcr(config, { imageBase64, sourceMimeType }); + return runTencentGeneralInvoiceOcr(config, { imageBase64, sourceMimeType }); +} + +function shouldDeduplicateByInvoiceNumberOnly(normalized) { + return Boolean( + normalized?.invoiceNumber && + !normalized?.invoiceCode && + NUMBER_ONLY_DEDUP_DOCUMENT_CATEGORIES.has(normalized?.documentCategory), + ); +} + +function normalizeSafeString(value, maxLength) { + if (typeof value !== "string") { + return null; + } + + const normalized = value.trim(); + return normalized ? normalized.slice(0, maxLength) : null; +} + +function sanitizeOcrResultForPersistence(ocrProvider, normalized) { + if (ocrProvider !== "tencent") { + return normalized; + } + + const safe = { + documentSubtype: normalizeSafeString(normalized?.documentSubtype, 64), + documentCategory: SAFE_DOCUMENT_CATEGORIES.has(normalized?.documentCategory) + ? normalized.documentCategory + : null, + documentCount: + Number.isInteger(normalized?.documentCount) && normalized.documentCount >= 0 + ? Math.min(normalized.documentCount, 100) + : 1, + }; + + for (const field of SAFE_NORMALIZED_FIELDS) { + safe[field] = normalizeSafeString(normalized?.[field], field.endsWith("Name") ? 256 : 64); + } + + if (SAFE_QUALITY_ERRORS.has(normalized?.qualityError)) { + safe.qualityError = normalized.qualityError; + } + + if (normalized?.transportMode === "pdf_native") { + safe.transportMode = "pdf_native"; + } + + return safe; } module.exports = { + OCR_PDF_MAX_BYTES, buildMissingRequiredOcrError, getMissingRequiredOcrFields, isOcrEnabledForType, runOcrForInvoice, + sanitizeOcrResultForPersistence, + shouldDeduplicateByInvoiceNumberOnly, }; diff --git a/server/src/services/ocr/tencent.js b/server/src/services/ocr/tencent.js index 3f47bd70..2008e762 100644 --- a/server/src/services/ocr/tencent.js +++ b/server/src/services/ocr/tencent.js @@ -1,6 +1,32 @@ const tencentcloud = require("tencentcloud-sdk-nodejs"); const PDF_MIME_TYPE = "application/pdf"; +const VAT_INVOICE_SUBTYPE_FIELDS = new Map([ + ["VatSpecialInvoice", "VatSpecialInvoice"], + ["VatCommonInvoice", "VatCommonInvoice"], + ["VatElectronicCommonInvoice", "VatElectronicCommonInvoice"], + ["VatElectronicSpecialInvoice", "VatElectronicSpecialInvoice"], + ["VatElectronicInvoiceBlockchain", "VatElectronicInvoiceBlockchain"], + ["VatElectronicInvoiceToll", "VatElectronicInvoiceToll"], +]); + +const FULL_VAT_INVOICE_SUBTYPE_FIELDS = new Map([ + ["VatElectronicSpecialInvoiceFull", "VatElectronicSpecialInvoiceFull"], + ["VatElectronicInvoiceFull", "VatElectronicInvoiceFull"], +]); + +const ELECTRONIC_TRAIN_TICKET_SUBTYPE = "ElectronicTrainTicketFull"; +const TERMINAL_TENCENT_ERROR_CODE_PATTERNS = [ + /^AuthFailure(?:\.|$)/, + /^UnauthorizedOperation(?:\.|$)/, + /^InvalidParameter(?:\.|$)/, + /^InvalidParameterValue(?:\.|$)/, + /^FailedOperation\.(?:Arrears|ImageBlur|ImageDecodeFailed|ImageSizeTooLarge|NoChargePermission|OcrFailed|ServiceIsolate|UnOpenError)$/, + /^LimitExceeded\.TooLargeFileError$/, + /^ResourceUnavailable\.(?:InArrears|ResourcePackageRunOut)$/, + /^ResourcesSoldOut\.ChargeStatusException$/, +]; + function normalizeDate(value) { if (!value || typeof value !== "string") { return null; @@ -34,7 +60,7 @@ function toNumberString(value) { return null; } - const stripped = value.replace(/[,￥¥\s]/g, "").trim(); + const stripped = value.replace(/[,￥¥元\s]/g, "").trim(); if (!stripped) { return null; } @@ -85,38 +111,33 @@ function normalizeString(value) { return normalized || null; } -function normalizeInvoiceInfoMap(result) { - if (!Array.isArray(result?.VatInvoiceInfos)) { - return null; - } - - return result.VatInvoiceInfos.reduce((acc, item) => { - if (typeof item?.Name !== "string") { - return acc; - } - - acc[item.Name.trim()] = item?.Value; - return acc; - }, {}); -} - function resolveTransportMode(sourceMimeType) { return sourceMimeType === PDF_MIME_TYPE ? "pdf_native" : "image_native"; } +function isTerminalTencentErrorCode(code) { + return ( + code === "ocr_credentials_missing" || + code === "unsupported_source_mime_type" || + TERMINAL_TENCENT_ERROR_CODE_PATTERNS.some((pattern) => pattern.test(code)) + ); +} + function wrapTencentError(error, transportMode) { const code = typeof error?.code === "string" ? error.code : "ocr_failed"; const message = error?.message ?? "ocr_failed"; const wrapped = new Error(message); wrapped.code = code; wrapped.transportMode = transportMode; - wrapped.cause = error; + wrapped.terminal = isTerminalTencentErrorCode(code); return wrapped; } function createTencentOcrClient(config) { if (!config.tencentSecretId || !config.tencentSecretKey) { - throw new Error("Tencent OCR credentials are missing"); + const error = new Error("Tencent OCR credentials are missing"); + error.code = "ocr_credentials_missing"; + throw error; } const OcrClient = tencentcloud.ocr.v20181119.Client; @@ -134,66 +155,150 @@ function createTencentOcrClient(config) { }); } -function normalizeVatInvoiceResult(result) { - const infoMap = normalizeInvoiceInfoMap(result); - const invoiceCodeFromInfo = - normalizeIdentifier(infoMap?.["发票代码"]) ?? normalizeIdentifier(infoMap?.["发票号码"]); - const invoiceNumberFromInfo = - normalizeIdentifier(infoMap?.["发票号码"]) ?? normalizeIdentifier(infoMap?.["发票代码"]); - const invoiceDateFromInfo = normalizeDate(infoMap?.["开票日期"]); - const amountFromInfo = toNumberString(infoMap?.["合计金额"] ?? infoMap?.["小计金额"]); - const taxAmountFromInfo = toNumberString(infoMap?.["合计税额"] ?? infoMap?.["小计税额"]); - const totalAmountFromInfo = toNumberString(infoMap?.["价税合计(小写)"]); - const sellerNameFromInfo = normalizeString(infoMap?.["销售方名称"]); - const buyerNameFromInfo = normalizeString(infoMap?.["购买方名称"]); - const amount = toNumberString(result?.AmountWithoutTax); - const taxAmount = toNumberString(result?.TaxAmount); - const totalAmountFromSource = toNumberString(result?.TotalAmount ?? result?.AmountInFiguers); - const resolvedAmount = amount ?? amountFromInfo; - const resolvedTaxAmount = taxAmount ?? taxAmountFromInfo; - const resolvedTotalAmountFromSource = totalAmountFromSource ?? totalAmountFromInfo; - const derivedAmount = - resolvedAmount ?? subtractNumberStrings(resolvedTotalAmountFromSource, resolvedTaxAmount); - const derivedTotalAmount = - resolvedTotalAmountFromSource ?? addNumberStrings(derivedAmount, resolvedTaxAmount); - +function buildBaseNormalizedResult({ documentSubtype = null, documentCategory = null, documentCount = 1 } = {}) { return { - invoiceCode: normalizeIdentifier(result?.InvoiceCode) ?? invoiceCodeFromInfo, - invoiceNumber: normalizeIdentifier(result?.InvoiceNum) ?? invoiceNumberFromInfo, - invoiceDate: normalizeDate(result?.InvoiceDate) ?? invoiceDateFromInfo, - amount: derivedAmount, - taxAmount: resolvedTaxAmount, - totalAmount: derivedTotalAmount, - sellerName: normalizeString(result?.SellerName) ?? sellerNameFromInfo, - buyerName: normalizeString(result?.BuyerName) ?? buyerNameFromInfo, - raw: result ?? {}, + documentSubtype, + documentCategory, + documentCount, + invoiceCode: null, + invoiceNumber: null, + invoiceDate: null, + amount: null, + taxAmount: null, + totalAmount: null, + sellerName: null, + buyerName: null, }; } -async function runTencentVatOcr(config, { imageBase64, sourceMimeType }) { - const client = createTencentOcrClient(config); +function normalizeVatInvoiceInfo(info, documentSubtype) { + const amount = toNumberString(info?.PretaxAmount); + const taxAmount = toNumberString(info?.Tax); + const totalAmount = toNumberString(info?.Total); + + return { + ...buildBaseNormalizedResult({ documentSubtype, documentCategory: "vat_invoice" }), + invoiceCode: normalizeIdentifier(info?.Code), + invoiceNumber: normalizeIdentifier(info?.Number ?? info?.ElectronicFullNumber), + invoiceDate: normalizeDate(info?.Date), + amount: amount ?? subtractNumberStrings(totalAmount, taxAmount), + taxAmount, + totalAmount: totalAmount ?? addNumberStrings(amount, taxAmount), + sellerName: normalizeString(info?.Seller), + buyerName: normalizeString(info?.Buyer), + }; +} + +function normalizeFullVatInvoiceInfo(info, documentSubtype) { + const amount = toNumberString(info?.PretaxAmount ?? info?.SubTotal); + const taxAmount = toNumberString(info?.Tax ?? info?.SubTax); + const totalAmount = toNumberString(info?.Total); + + return { + ...buildBaseNormalizedResult({ documentSubtype, documentCategory: "full_vat_invoice" }), + invoiceNumber: normalizeIdentifier(info?.Number), + invoiceDate: normalizeDate(info?.Date), + amount: amount ?? subtractNumberStrings(totalAmount, taxAmount), + taxAmount, + totalAmount: totalAmount ?? addNumberStrings(amount, taxAmount), + sellerName: normalizeString(info?.Seller), + buyerName: normalizeString(info?.Buyer), + }; +} + +function normalizeElectronicTrainTicket(info) { + const taxAmount = toNumberString(info?.Tax); + const totalAmount = toNumberString(info?.Total ?? info?.Fare); + + return { + ...buildBaseNormalizedResult({ + documentSubtype: ELECTRONIC_TRAIN_TICKET_SUBTYPE, + documentCategory: "electronic_train_ticket", + }), + invoiceNumber: normalizeIdentifier(info?.Number), + invoiceDate: normalizeDate(info?.Date), + amount: subtractNumberStrings(totalAmount, taxAmount), + taxAmount, + totalAmount, + buyerName: normalizeString(info?.Buyer), + }; +} + +function buildQualityFailure(qualityError, documentCount, documentSubtype = null) { + return { + ...buildBaseNormalizedResult({ documentSubtype, documentCount }), + qualityError, + }; +} + +function normalizeGeneralInvoiceResult(result) { + const invoiceItems = Array.isArray(result?.MixedInvoiceItems) ? result.MixedInvoiceItems : []; + if (invoiceItems.length === 0) { + return buildQualityFailure("invoice_not_detected", 0); + } + + if (invoiceItems.length !== 1) { + return buildQualityFailure("multiple_invoices_not_supported", invoiceItems.length); + } + + const invoiceItem = invoiceItems[0]; + const documentSubtype = normalizeIdentifier(invoiceItem?.SubType); + if (invoiceItem?.Code !== "OK") { + return buildQualityFailure("invoice_recognition_failed", 1, documentSubtype); + } + + const invoiceInfo = invoiceItem?.SingleInvoiceInfos; + const vatInfoField = VAT_INVOICE_SUBTYPE_FIELDS.get(documentSubtype); + if (vatInfoField && invoiceInfo?.[vatInfoField]) { + return normalizeVatInvoiceInfo(invoiceInfo[vatInfoField], documentSubtype); + } + + const fullVatInfoField = FULL_VAT_INVOICE_SUBTYPE_FIELDS.get(documentSubtype); + if (fullVatInfoField && invoiceInfo?.[fullVatInfoField]) { + return normalizeFullVatInvoiceInfo(invoiceInfo[fullVatInfoField], documentSubtype); + } + + if ( + documentSubtype === ELECTRONIC_TRAIN_TICKET_SUBTYPE && + invoiceInfo?.ElectronicTrainTicketFull + ) { + return normalizeElectronicTrainTicket(invoiceInfo.ElectronicTrainTicketFull); + } + + return buildQualityFailure("unsupported_invoice_subtype", 1, documentSubtype); +} + +function buildGeneralInvoiceRequest(imageBase64) { + return { + ImageBase64: imageBase64, + EnableOther: false, + EnablePdf: true, + PdfPageNumber: 1, + EnableMultiplePage: false, + EnableCutImage: false, + EnableItemPolygon: false, + EnableQRCode: false, + EnableSeal: false, + }; +} + +async function runTencentGeneralInvoiceOcr(config, { imageBase64, sourceMimeType }) { const transportMode = resolveTransportMode(sourceMimeType); try { - const request = { - ImageBase64: imageBase64, - }; - - if (transportMode === "pdf_native") { - request.IsPdf = true; - request.PdfPageNumber = 1; + if (transportMode !== "pdf_native") { + const error = new Error("unsupported_source_mime_type"); + error.code = "unsupported_source_mime_type"; + throw error; } - const result = await client.VatInvoiceOCR(request); - const normalized = normalizeVatInvoiceResult(result); + const client = createTencentOcrClient(config); + const result = await client.RecognizeGeneralInvoice(buildGeneralInvoiceRequest(imageBase64)); + const normalized = normalizeGeneralInvoiceResult(result); return { ...normalized, transportMode, - raw: { - ...(normalized.raw ?? {}), - transportMode, - }, }; } catch (error) { throw wrapTencentError(error, transportMode); @@ -201,6 +306,8 @@ async function runTencentVatOcr(config, { imageBase64, sourceMimeType }) { } module.exports = { - normalizeVatInvoiceResult, - runTencentVatOcr, + buildGeneralInvoiceRequest, + isTerminalTencentErrorCode, + normalizeGeneralInvoiceResult, + runTencentGeneralInvoiceOcr, }; diff --git a/server/tests/unit/invoiceGovernance.dedup.test.js b/server/tests/unit/invoiceGovernance.dedup.test.js index 1ec24f35..c7c4ad5a 100644 --- a/server/tests/unit/invoiceGovernance.dedup.test.js +++ b/server/tests/unit/invoiceGovernance.dedup.test.js @@ -1,6 +1,7 @@ const { INVOICE_UNAVAILABLE_REASONS, classifyInvoiceConflict, + buildInvoiceDedupFilters, formatInvoiceOcrFailureMessage, resolveInvoiceAvailability, resolveUploadDedupDecision, @@ -32,6 +33,17 @@ function buildTx(conflicts) { } describe("invoiceGovernance dedup", () => { + it("uses invoice number alone only when the caller marks a known no-code subtype", () => { + expect( + buildInvoiceDedupFilters({ + invoiceNumber: "251234", + deduplicateByInvoiceNumberOnly: true, + }), + ).toEqual([{ invoice_code: null, invoice_number: "251234" }]); + + expect(buildInvoiceDedupFilters({ invoiceNumber: "251234" })).toEqual([]); + }); + it("ignores manually archived invoices when re-uploading the same file", async () => { const archivedManual = buildInvoice({ status: "archived", @@ -122,5 +134,13 @@ describe("invoiceGovernance dedup", () => { expect(formatInvoiceOcrFailureMessage("mock_ocr_failed")).toBe( "票据识别失败，请重试或人工补齐关键信息。", ); + expect(formatInvoiceOcrFailureMessage("multiple_invoices_not_supported")).toBe( + "一个 PDF 第一页只能包含一张发票，请拆分后重新上传。", + ); + expect( + formatInvoiceOcrFailureMessage( + "[pdf_native][ResourceUnavailable.ResourcePackageRunOut] resource package run out", + ), + ).toBe("票据识别额度或计费状态暂不可用，请联系管理员。"); }); }); diff --git a/server/tests/unit/jobs/ocrJob.test.js b/server/tests/unit/jobs/ocrJob.test.js index bdb6bf0e..53d120f6 100644 --- a/server/tests/unit/jobs/ocrJob.test.js +++ b/server/tests/unit/jobs/ocrJob.test.js @@ -19,15 +19,16 @@ function buildClaimedInvoice(overrides = {}) { }; } -function buildPrisma(claimedInvoices) { +function buildPrisma(claimedInvoices, { cachedOcrResult = null, invoiceConflicts = [] } = {}) { return { $queryRaw: vi.fn(async () => claimedInvoices), ocrResult: { - findUnique: vi.fn(async () => null), + findUnique: vi.fn(async () => cachedOcrResult), create: vi.fn(async () => ({ id: "ocr-result-1" })), + update: vi.fn(async ({ data }) => ({ id: cachedOcrResult?.id, ...data })), }, invoicePool: { - findMany: vi.fn(async () => []), + findMany: vi.fn(async () => invoiceConflicts), findFirst: vi.fn(async () => null), update: vi.fn(async () => ({ id: "invoice-1" })), }, @@ -218,4 +219,144 @@ describe("ocrJob quality gate", () => { ); expect(JSON.stringify(createdPayload)).not.toContain("\\u0000"); }); + + it("accepts a cached no-code electronic train ticket and checks number-only duplicates", async () => { + await fs.writeFile(path.join(tempDir, "20260428/invoice.pdf"), "%PDF-1.4 synthetic"); + + const cachedOcrResult = { + id: "ocr-result-train", + result_json: { + documentSubtype: "ElectronicTrainTicketFull", + documentCategory: "electronic_train_ticket", + documentCount: 1, + transportMode: "pdf_native", + invoiceCode: null, + invoiceNumber: "251234567890", + invoiceDate: "2026-07-26", + amount: "95.00", + taxAmount: "5.00", + totalAmount: "100.00", + sellerName: null, + buyerName: "合成公司", + raw: { + UserName: "SENSITIVE-CACHED-PASSENGER", + UserID: "SENSITIVE-CACHED-ID", + }, + }, + }; + const prisma = buildPrisma( + [ + buildClaimedInvoice({ + ocr_provider: "tencent", + storage_path: "20260428/invoice.pdf", + mime_type: "application/pdf", + }), + ], + { cachedOcrResult }, + ); + const logger = { warn: vi.fn(), error: vi.fn() }; + + const result = await runOcrOnce( + prisma, + { + storageDir: tempDir, + ocrBatchSize: 1, + ocrLockTtlSeconds: 300, + ocrMaxAttempts: 3, + ocrProvider: "tencent", + }, + logger, + "ocr-test-worker", + ); + + expect(result).toEqual({ claimed: 1, available: 1, duplicate: 0, failed: 0 }); + expect(prisma.invoicePool.update).toHaveBeenCalledWith( + expect.objectContaining({ + data: expect.objectContaining({ + status: "available", + invoice_code: null, + invoice_number: "251234567890", + seller_name: null, + buyer_name: "合成公司", + }), + }), + ); + expect(prisma.invoicePool.findMany).toHaveBeenCalledWith( + expect.objectContaining({ + where: expect.objectContaining({ + AND: expect.arrayContaining([ + expect.objectContaining({ + OR: expect.arrayContaining([ + { invoice_code: null, invoice_number: "251234567890" }, + ]), + }), + ]), + }), + }), + ); + expect(prisma.ocrResult.create).not.toHaveBeenCalled(); + expect(prisma.ocrResult.update).toHaveBeenCalledWith( + expect.objectContaining({ + where: { id: "ocr-result-train" }, + data: expect.objectContaining({ + result_json: expect.not.objectContaining({ raw: expect.anything() }), + }), + }), + ); + expect(JSON.stringify(prisma.ocrResult.update.mock.calls[0][0])).not.toContain( + "SENSITIVE-CACHED", + ); + }); + + it("marks a cached multiple-invoice result as a quality failure without provider retry", async () => { + await fs.writeFile(path.join(tempDir, "20260428/invoice.pdf"), "%PDF-1.4 synthetic"); + + const prisma = buildPrisma( + [ + buildClaimedInvoice({ + ocr_provider: "tencent", + storage_path: "20260428/invoice.pdf", + mime_type: "application/pdf", + }), + ], + { + cachedOcrResult: { + id: "ocr-result-multiple", + result_json: { + documentSubtype: null, + documentCategory: null, + documentCount: 2, + qualityError: "multiple_invoices_not_supported", + transportMode: "pdf_native", + }, + }, + }, + ); + const logger = { warn: vi.fn(), error: vi.fn() }; + + const result = await runOcrOnce( + prisma, + { + storageDir: tempDir, + ocrBatchSize: 1, + ocrLockTtlSeconds: 300, + ocrMaxAttempts: 3, + ocrProvider: "tencent", + }, + logger, + "ocr-test-worker", + ); + + expect(result).toEqual({ claimed: 1, available: 0, duplicate: 0, failed: 1 }); + expect(prisma.invoicePool.update).toHaveBeenCalledWith( + expect.objectContaining({ + data: expect.objectContaining({ + status: "ocr_failed", + ocr_last_error: "multiple_invoices_not_supported", + ocr_attempts: { increment: 1 }, + }), + }), + ); + expect(logger.error).not.toHaveBeenCalled(); + }); }); diff --git a/server/tests/unit/routes/invoicePool.upload.test.js b/server/tests/unit/routes/invoicePool.upload.test.js index ee4970b7..e5f51519 100644 --- a/server/tests/unit/routes/invoicePool.upload.test.js +++ b/server/tests/unit/routes/invoicePool.upload.test.js @@ -43,6 +43,7 @@ async function buildApp({ invoiceConflicts = [], statusRows = [], permissionCodes = ["invoice.upload"], + maxUploadSizeBytes = 5 * 1024 * 1024, } = {}) { const tempDir = await fs.mkdtemp(path.join(os.tmpdir(), "hyb-oa-r71-invoice-upload-")); const invoicePoolCreate = vi.fn(async ({ data }) => ({ @@ -74,7 +75,7 @@ async function buildApp({ app.decorate("config", { appTimezone: "Asia/Shanghai", storageDir: tempDir, - maxUploadSizeBytes: 5 * 1024 * 1024, + maxUploadSizeBytes, invoiceArchiveUnusedDays: 30, ocrProvider, jobDriver, @@ -121,7 +122,7 @@ async function buildApp({ await app.register(multipart, { attachFieldsToBody: false, - limits: { files: 1, fileSize: 5 * 1024 * 1024 }, + limits: { files: 1, fileSize: maxUploadSizeBytes }, }); await app.register(invoicePoolRoutes, { prefix: "/api" }); @@ -271,6 +272,48 @@ describe("invoicePool upload route", () => { } }); + it("rejects a PDF above the 7 MiB Tencent OCR input boundary before storage", async () => { + const { app, prisma, tempDir } = await buildApp({ + maxUploadSizeBytes: 8 * 1024 * 1024, + }); + + try { + const requestPayload = buildMultipartRequest({ + filename: "invoice.pdf", + contentType: "application/pdf", + content: Buffer.alloc(7 * 1024 * 1024 + 1, 0x20), + fields: [["ocr_type", "vat"]], + }); + + const response = await app.inject({ + method: "POST", + url: "/api/invoice-pool/upload", + headers: { + authorization: "Bearer test", + ...requestPayload.headers, + }, + payload: requestPayload.payload, + }); + + expect(response.statusCode).toBe(400); + expect(response.json().error).toEqual( + expect.objectContaining({ + code: "VALID_001", + message: "PDF 发票不能超过 7 MiB", + details: { + file: "ocr_pdf_too_large", + maxBytes: 7 * 1024 * 1024, + }, + }), + ); + expect(prisma.invoicePool.create).not.toHaveBeenCalled(); + expect(prisma.auditLog.create).not.toHaveBeenCalled(); + } finally { + await app.close(); + await fs.rm(tempDir, { recursive: true, force: true }); + } + }); + it("reuses same-owner existing invoice instead of creating a duplicate file record", async () => { const existingInvoiceId = "22222222-2222-4222-8222-222222222222"; const { app, prisma, tempDir } = await buildApp({ diff --git a/server/tests/unit/services/ocr.tencent.test.js b/server/tests/unit/services/ocr.tencent.test.js index 3afecbbb..188b681e 100644 --- a/server/tests/unit/services/ocr.tencent.test.js +++ b/server/tests/unit/services/ocr.tencent.test.js @@ -1,51 +1,224 @@ const { - normalizeVatInvoiceResult, + buildGeneralInvoiceRequest, + isTerminalTencentErrorCode, + normalizeGeneralInvoiceResult, } = require("../../../src/services/ocr/tencent"); +const { + buildMissingRequiredOcrError, + getMissingRequiredOcrFields, + sanitizeOcrResultForPersistence, + shouldDeduplicateByInvoiceNumberOnly, +} = require("../../../src/services/ocr"); -describe("tencent VAT OCR normalization", () => { - it("normalizes identifiers, date and amounts from Tencent result", () => { - const normalized = normalizeVatInvoiceResult({ - InvoiceCode: " 0440 0320 4111 ", - InvoiceNum: " 1234 5678 ", - InvoiceDate: "2026年4月28日", - AmountWithoutTax: "1,488.50", - TaxAmount: "194.50", - AmountInFiguers: "￥1683.00", - SellerName: " 北京悦海堂餐饮管理有限公司 ", - BuyerName: " 华印公司 ", +function buildInvoiceItem(subType, fieldName, payload) { + return { + Code: "OK", + SubType: subType, + SingleInvoiceInfos: { + [fieldName]: payload, + }, + }; +} + +describe("Tencent general invoice OCR", () => { + it("builds a single-page PDF request without cut images or sensitive extras", () => { + expect(buildGeneralInvoiceRequest("cGRm")).toEqual({ + ImageBase64: "cGRm", + EnableOther: false, + EnablePdf: true, + PdfPageNumber: 1, + EnableMultiplePage: false, + EnableCutImage: false, + EnableItemPolygon: false, + EnableQRCode: false, + EnableSeal: false, + }); + }); + + it("marks credential, billing and invalid-input provider errors as terminal", () => { + expect(isTerminalTencentErrorCode("AuthFailure.SecretIdNotFound")).toBe(true); + expect(isTerminalTencentErrorCode("FailedOperation.Arrears")).toBe(true); + expect(isTerminalTencentErrorCode("FailedOperation.NoChargePermission")).toBe(true); + expect(isTerminalTencentErrorCode("InvalidParameterValue.InvalidImageContent")).toBe(true); + expect(isTerminalTencentErrorCode("LimitExceeded.TooLargeFileError")).toBe(true); + expect(isTerminalTencentErrorCode("ResourceUnavailable.InArrears")).toBe(true); + expect(isTerminalTencentErrorCode("ResourceUnavailable.ResourcePackageRunOut")).toBe(true); + expect(isTerminalTencentErrorCode("ResourcesSoldOut.ChargeStatusException")).toBe(true); + expect(isTerminalTencentErrorCode("ocr_credentials_missing")).toBe(true); + expect(isTerminalTencentErrorCode("unsupported_source_mime_type")).toBe(true); + expect(isTerminalTencentErrorCode("RequestLimitExceeded")).toBe(false); + expect(isTerminalTencentErrorCode("InternalError")).toBe(false); + }); + + it("normalizes a VAT electronic common invoice without preserving the provider payload", () => { + const normalized = normalizeGeneralInvoiceResult({ + MixedInvoiceItems: [ + buildInvoiceItem("VatElectronicCommonInvoice", "VatElectronicCommonInvoice", { + Code: " 0440 0320 4111 ", + Number: " 1234 5678 ", + Date: "2026年4月28日", + PretaxAmount: "1,488.50", + Tax: "194.50", + Total: "￥1683.00", + Seller: " 合成酒店 ", + Buyer: " 合成公司 ", + SellerBankAccount: "SENSITIVE-BANK-ACCOUNT", + }), + ], + RequestId: "SENSITIVE-REQUEST-ID", + }); + + expect(normalized).toEqual({ + documentSubtype: "VatElectronicCommonInvoice", + documentCategory: "vat_invoice", + documentCount: 1, + invoiceCode: "044003204111", + invoiceNumber: "12345678", + invoiceDate: "2026-04-28", + amount: "1488.50", + taxAmount: "194.50", + totalAmount: "1683.00", + sellerName: "合成酒店", + buyerName: "合成公司", + }); + expect(JSON.stringify(normalized)).not.toContain("SENSITIVE-"); + }); + + it("accepts a full electronic VAT invoice without fabricating an invoice code", () => { + const normalized = normalizeGeneralInvoiceResult({ + MixedInvoiceItems: [ + buildInvoiceItem("VatElectronicInvoiceFull", "VatElectronicInvoiceFull", { + Number: " 25 12345678901234567890 ", + Date: "2026/04/28", + PretaxAmount: "450.00", + Tax: "27.00", + Total: "477.00", + Seller: "合成住宿服务商", + Buyer: "合成公司", + }), + ], }); expect(normalized).toEqual( expect.objectContaining({ - invoiceCode: "044003204111", - invoiceNumber: "12345678", - invoiceDate: "2026-04-28", - amount: "1488.50", - taxAmount: "194.50", - totalAmount: "1683.00", - sellerName: "北京悦海堂餐饮管理有限公司", - buyerName: "华印公司", + documentCategory: "full_vat_invoice", + invoiceCode: null, + invoiceNumber: "2512345678901234567890", + totalAmount: "477.00", }), ); + expect(getMissingRequiredOcrFields("vat", normalized)).toEqual([]); + expect(shouldDeduplicateByInvoiceNumberOnly(normalized)).toBe(true); }); - it("derives amount from total and tax when Tencent omits AmountWithoutTax", () => { - const result = normalizeVatInvoiceResult({ - InvoiceCode: "044003204111", - InvoiceNum: "12345678", - InvoiceDate: "2026/04/28", - TaxAmount: 194.5, - TotalAmount: 1683, - SellerName: "北京悦海堂餐饮管理有限公司", - BuyerName: "华印公司", + it("normalizes an electronic train ticket without retaining passenger identity fields", () => { + const normalized = normalizeGeneralInvoiceResult({ + MixedInvoiceItems: [ + buildInvoiceItem("ElectronicTrainTicketFull", "ElectronicTrainTicketFull", { + Number: " 25 98765432109876543210 ", + Date: "2026年7月26日", + Total: "553.00元", + Tax: "4.06", + Buyer: "合成公司", + UserName: "SENSITIVE-PASSENGER-NAME", + UserID: "SENSITIVE-PASSENGER-ID", + ElectronicTicketNum: "SENSITIVE-TICKET-NUMBER", + }), + ], }); - expect(result).toEqual( - expect.objectContaining({ - amount: "1488.50", - taxAmount: "194.50", - totalAmount: "1683.00", - }), + expect(normalized).toEqual({ + documentSubtype: "ElectronicTrainTicketFull", + documentCategory: "electronic_train_ticket", + documentCount: 1, + invoiceCode: null, + invoiceNumber: "2598765432109876543210", + invoiceDate: "2026-07-26", + amount: "548.94", + taxAmount: "4.06", + totalAmount: "553.00", + sellerName: null, + buyerName: "合成公司", + }); + expect(buildMissingRequiredOcrError("vat", normalized)).toBeNull(); + expect(shouldDeduplicateByInvoiceNumberOnly(normalized)).toBe(true); + expect(JSON.stringify(normalized)).not.toContain("SENSITIVE-"); + }); + + it("returns safe terminal quality results for zero, multiple and unsupported invoices", () => { + expect(normalizeGeneralInvoiceResult({ MixedInvoiceItems: [] })).toEqual( + expect.objectContaining({ qualityError: "invoice_not_detected", documentCount: 0 }), + ); + + const multiple = normalizeGeneralInvoiceResult({ + MixedInvoiceItems: [ + buildInvoiceItem("ElectronicTrainTicketFull", "ElectronicTrainTicketFull", { + UserID: "SENSITIVE-FIRST-ID", + }), + buildInvoiceItem("ElectronicTrainTicketFull", "ElectronicTrainTicketFull", { + UserID: "SENSITIVE-SECOND-ID", + }), + ], + }); + expect(multiple).toEqual( + expect.objectContaining({ qualityError: "multiple_invoices_not_supported", documentCount: 2 }), + ); + expect(JSON.stringify(multiple)).not.toContain("SENSITIVE-"); + + const unsupported = normalizeGeneralInvoiceResult({ + MixedInvoiceItems: [buildInvoiceItem("TaxiTicket", "TaxiTicket", { Number: "SENSITIVE-NO" })], + }); + expect(buildMissingRequiredOcrError("vat", unsupported)).toBe("unsupported_invoice_subtype"); + expect(JSON.stringify(unsupported)).not.toContain("SENSITIVE-"); + }); + + it("requires buyer name for electronic train tickets but not invoice code or seller", () => { + const normalized = { + documentCategory: "electronic_train_ticket", + invoiceCode: null, + invoiceNumber: "251234", + invoiceDate: "2026-07-26", + totalAmount: "100.00", + sellerName: null, + buyerName: null, + }; + + expect(getMissingRequiredOcrFields("vat", normalized)).toEqual(["buyerName"]); + expect(buildMissingRequiredOcrError("vat", normalized)).toBe( + "missing_required_fields:buyerName", ); }); + + it("strips raw and unknown fields from historical Tencent cache payloads", () => { + const sanitized = sanitizeOcrResultForPersistence("tencent", { + invoiceCode: "044001", + invoiceNumber: "12345678", + invoiceDate: "2026-07-26", + totalAmount: "100.00", + sellerName: "合成销售方", + buyerName: "合成购买方", + transportMode: "pdf_native", + raw: { + UserName: "SENSITIVE-CACHED-NAME", + UserID: "SENSITIVE-CACHED-ID", + }, + RequestId: "SENSITIVE-CACHED-REQUEST", + }); + + expect(sanitized).toEqual({ + documentSubtype: null, + documentCategory: null, + documentCount: 1, + invoiceCode: "044001", + invoiceNumber: "12345678", + invoiceDate: "2026-07-26", + amount: null, + taxAmount: null, + totalAmount: "100.00", + sellerName: "合成销售方", + buyerName: "合成购买方", + transportMode: "pdf_native", + }); + expect(JSON.stringify(sanitized)).not.toContain("SENSITIVE-"); + }); });