Vyuh DOCX Golden Rendering Plan

This is the working golden document for improving vyuh_docx into the controlled-document PDF renderer used by DocsIQ.

It records the current architecture, the research direction, the package boundaries, the renderer problems we are solving, and the improvement roadmap. It should be updated whenever the renderer architecture or fidelity strategy changes.

Executive Summary

vyuh_docx is the pure-Dart document engine that turns DOCX, SFDT, generated document models, HTML, or Markdown into a typed AST, and then exports that AST to DOCX, SFDT, HTML, or PDF.

For DocsIQ, the critical path is:

text

Syncfusion editor SFDT
  -> SfdtReader
  -> DocxBuiltDocument AST
  -> server-supplied field values and controlled render options
  -> PdfExporter
  -> PdfLayoutEngine / PdfLayoutDocument
  -> PdfDocumentWriter
  -> approved PDF bytes

The current renderer is already source-backed and deterministic, but it still needs stronger Word-like layout behavior for controlled SOPs: table overflow, header/footer body collision, stamped metadata inside tables, field resolution, font selection, and oracle-based visual verification.

The goal is not to replace vyuh_docx with a third-party package. The goal is to keep vyuh_docx as the controlled, pure-Dart engine and borrow proven patterns from other packages where they fit our architecture.

Product Requirement

DocsIQ will generate the approved PDF on the server after final approval.

The renderer must:

read the submitted SFDT from the Syncfusion editor;
preserve document body, sections, tables, lists, images, headers, footers, fields, comments, and revisions where modeled;
stamp server metadata into configured document fields;
stamp headers and footers on every page;
render header/footer tables without text overflow;
keep body content between header and footer bands;
use the document/SFDT font unless the server explicitly selects a controlled default font;
fail closed for missing glyphs in controlled output when strictFonts is enabled;
produce deterministic bytes when the server supplies a trusted UTC creation date;
expose layout facts through tests so regressions are caught before release.

This is a controlled-document requirement, not a generic report-generation requirement. Rendering must be attributable, deterministic, reviewable, and qualifiable.

Package Boundary

vyuh_docx owns document format and rendering work.

Concern	Owner
DOCX parsing	`vyuh_docx`
SFDT parsing	`vyuh_docx`
Typed AST/model	`vyuh_docx`
DOCX/SFDT/HTML/PDF export	`vyuh_docx`
PDF layout, pagination, painting	`vyuh_docx`
Font registration and strict glyph handling	`vyuh_docx`
Metadata field rendering once values are supplied	`vyuh_docx`
Syncfusion editor hosting	`docx_editor`
Actor, author, capability, lock policy	`docx_editor` and host app
Lifecycle status and workflow authority	DocsIQ server
Approved PDF generation trigger	DocsIQ server
Document metadata source values	DocsIQ server
Storage, audit, signatures, controlled-copy register	DocsIQ server / host

The rule is simple: vyuh_docx renders supplied document data. It does not decide who may approve, publish, print, or view a document.

Current Code-Level Architecture

Public API

Public exports live in:

packages/vyuh_docx/lib/vyuh_docx.dart

Important public families:

AST nodes: src/ast/*
readers: DocxReader, SfdtReader
exporters: DocxExporter, SfdtExporter, HtmlExporter, PdfExporter
layout model: PdfLayoutEngine, PdfLayoutDocument, PdfLayoutBox
builder: docx(), DocxDocumentBuilder
diff/template utilities: DocxDocumentDiff, DocxTemplateEngine

Input Readers

Source	Entry Point	Output
DOCX bytes	`DocxReader.loadFromBytes`	`DocxBuiltDocument`
SFDT JSON	`SfdtReader.read` / `readMap`	`DocxBuiltDocument`
HTML	`HtmlParser`	`DocxBuiltDocument` / nodes
Markdown	`MarkdownParser`	`DocxBuiltDocument` / nodes
Code builder	`docx()` / `DocxDocumentBuilder`	`DocxBuiltDocument`

SfdtReader currently preserves sections, blocks, paragraphs, tables, headers/footers, comments, revisions, protection, custom XML, document defaults, and style information where supported.

AST / Model Layer

The AST is the source-backed document model.

Core files:

src/ast/docx_block.dart
src/ast/docx_inline.dart
src/ast/docx_table.dart
src/ast/docx_section.dart
src/ast/docx_comment.dart
src/ast/docx_revision.dart
src/ast/docx_drawing.dart
src/ast/docx_footnote.dart

The renderer should not invent customer-specific behavior. If a feature is needed for output, the source property should reach the AST first.

PDF Pipeline

Main files:

src/exporters/pdf/pdf_exporter.dart
src/exporters/pdf/pdf_layout_engine.dart
src/exporters/pdf/pdf_layout_model.dart
src/exporters/pdf/pdf_content_builder.dart
src/exporters/pdf/pdf_document_writer.dart
src/exporters/pdf/pdf_font_manager.dart
src/exporters/pdf/pdf_generated_field_resolver.dart

Current high-level flow:

text

PdfExporter.exportToBytes
  -> prepare writer, fonts, theme, tab stops, notes
  -> split document into sections
  -> measure section header/footer reserves
  -> build PdfLayoutEngine
  -> paginate body or build PdfLayoutDocument for columns
  -> render pages
  -> lazily write only used fonts
  -> write image/font/page resources
  -> save PDF bytes

Existing Controlled Rendering Features

Current renderer features relevant to DocsIQ:

deterministic creationDate;
strictFonts fail-closed glyph mode;
fieldValues for server-supplied generated fields;
defaultFontFamily for controlled default selection;
header/footer reserve measurement;
footer collision guard;
section page size and margins;
page fields such as PAGE and NUMPAGES;
document default tab stop;
table grid layout, spans, borders, shading, row splitting, repeated headers;
layout model boxes for paragraphs, tables, lists, images, drawings, shapes;
fixture tests for PDF bytes, rasters, text extraction, and geometry.

Recent Hardening

DOCX/SFDT image parsing now fails closed on malformed colors and broken relationships instead of crashing the importer.
PDF layout measurement has cache coverage for font metrics, paragraphs, table rows, and page-independent header/footer chrome.
SFDT round-trip reader/exporter now carries endnote references through the same inline path as footnotes.
Northwell's strict SFDT leg is now green after carrying orphan footnote and endnote definitions as top-level strict metadata and fixing table-cell note sink threading in the SFDT reader.
DOCX export now preserves hyperlinks inside footnotes and endnotes by writing note-part relationship files and keeping the hyperlink registry active across the full archive build.
DOCX/SFDT style round-trips now preserve explicit bold-off state on authored heading styles, so Word-built SOP headings that inherit from a disabled bold parent no longer flip to a synthesized bold opinion during export.
The remaining Northwell drift is in the DOCX export leg's list/paragraph reconstruction, not in note loss or import stability.

Renderer Problems We Are Solving

1. Metadata Stamping In Header/Footer Tables

DocsIQ approved PDFs need server metadata on the front page and often inside header/footer tables:

UID;
title;
version;
owner;
approval timestamp;
generated timestamp;
reviewers;
approvers;
effective date;
department/site/library metadata.

The renderer already has PdfExporter.fieldValues and PdfGeneratedFieldResolver. The improvement is to make field resolution consistent in body, tables, headers, and footers, with the same font selection and wrapping behavior everywhere.

Required behavior:

field values are supplied by DocsIQ server;
missing field values fall back to cached source text where safe;
explicit empty/null values render empty;
server-stamped text uses the run font, inherited document default, or supplied defaultFontFamily;
measurement and painting use the same resolved text;
resolved text cannot overflow a table cell without a deterministic decision: wrap, shrink only if explicitly allowed, split row, or fail with diagnostics.

2. Header/Footer Body Collision

Regulated SOPs commonly have header/footer furniture that includes tables, logos, and approval metadata. Body content must not overlap that furniture.

Current direction:

measure header/footer reserve before body pagination;
reserve only what is needed;
treat simple footers as normal page chrome when they fit in the authored footer band;
keep table/shape-heavy furniture from clipping body text;
render footer upward from the footer base so it remains on the page.

Remaining work:

stronger diagnostics when furniture is taller than the available page band;
explicit test fixtures for DocsIQ front-page metadata tables;
better handling of first/even/default header/footer variants in long documents.

3. Table Layout And Overflow

The screenshot problem we saw is a table layout failure: text is being painted through row or cell boundaries instead of the row growing, splitting, or reflowing correctly.

Required table behavior:

compute the table grid before painting;
compute cell content height using the actual text layout;
row height must include the tallest cell after wrapping;
repeated header rows must repeat on page breaks;
a row may split only when Word-like splitting is allowed;
a non-splittable row that cannot fit should move to the next page or fail with a typed renderer error if it cannot fit on an empty page;
nested tables must use the parent cell content box;
borders and shading should use resolved table/cell properties;
measurement and painting must be lockstep.

This is the highest-priority renderer improvement for DocsIQ.

4. Font Fidelity

The stamped metadata cannot look like a separate stamp with a different font. It must follow the document typography.

Current direction:

SfdtReader carries top-level SFDT characterFormat into DocxTheme.defaultRunStyle;
PdfExporter.defaultFontFamily can override inherited defaults when a controlled caller decides the official output font;
generated fields resolve during PDF export and should inherit run/default font selection;
strictFonts should be used for controlled PDF generation.

Remaining work:

expand test fixtures with metadata fields in body, table cells, header cells, and footer cells;
verify bold/italic variants for stamped text;
add diagnostics for unsupported glyphs and missing font variants.

5. Layout Model As The Truth

The renderer should continue moving away from direct AST painting and toward an explicit layout model.

Current direction:

PdfLayoutDocument contains pages;
each page contains typed boxes;
boxes preserve source nodes and geometry;
paragraphs expose resolved text lines and fragments;
tables expose table, row, cell, and placement boxes.

Target:

all PDF painting should be driven from layout boxes;
measurement should create the same layout facts the painter uses;
geometry tests should assert that no content crosses page, margin, header, footer, or cell bounds.

External Research And What We Adopt

This section records what we are borrowing conceptually. We are not copying third-party code into vyuh_docx.

Dart `pdf` Package

Reference:

https://pub.dev/documentation/pdf/latest/

Useful pattern:

MultiPage automatically flows content across pages;
headers and footers are built per page and their space is reserved before laying out content;
spanning widgets such as tables can split across pages;
inseparable widgets can be marked as not splittable;
page breaks can be explicit or conditional.

What we adopt:

explicit pagination contract;
content that declares whether it can span pages;
separate measurement from painting;
header/footer reserve before body layout;
max-page or runaway-pagination guard for debug/test safety.

What we do not adopt:

Flutter-like widget API as the internal model;
generic report-layout assumptions that ignore DOCX/SFDT source properties.

`pdfrx` / PDFium

Reference:

https://pub.dev/packages/pdfrx

Useful pattern:

pdfrx separates Flutter viewer widgets from pdfrx_engine, the lower-level PDFium-backed PDF document API;
PDFium is useful as a rendering/viewing oracle for already-created PDF bytes.

What we adopt:

architecture separation: viewer/runtime concerns stay separate from the document engine;
use rendered rasters and text geometry as test oracles;
treat visual verification as first-class, not optional.

What we do not adopt:

PDFium as the AST-to-PDF layout engine. PDFium renders existing PDFs; it does not solve SFDT/DOCX-to-PDF layout from our source model.

`docs_gee`

Reference:

https://pub.dev/packages/docs_gee

Useful pattern:

one document model can generate both DOCX and PDF;
pure-Dart, cross-platform, no native dependency;
builder-style document creation with tables, lists, styles, and fonts.

What we adopt:

single source model for multiple output formats;
keep dependencies small;
make features visible in a capability matrix;
test DOCX and PDF output from the same model.

What we do not adopt:

replacing DocxBuiltDocument;
simplifying away Word/SFDT fidelity for generated-document convenience.

`docx_creator`

Reference:

https://pub.dev/packages/docx_creator

Useful pattern:

vyuh_docx is already a Vyuh fork of docx_creator;
the upstream package is builder-friendly and broad-format oriented;
it validates the value of a fluent generated-document API.

What we adopt:

keep builder ergonomics for generated documents;
preserve MIT provenance in NOTICE.md;
harden the fork for controlled pharmaceutical documents rather than chasing only broad package features.

What we do not adopt:

blindly merging upstream behavior that weakens deterministic controlled rendering.

Golden Rules For Renderer Changes

Source-backed rendering only. A renderer rule must come from DOCX XML, SFDT JSON, DocxBuiltDocument, or a documented Word/OOXML default.
Measurement and painting must stay lockstep. Any text, table, image, field, or header/footer measurement must use the same resolved values that painting uses.
Do not hide overflow. Overflow is either reflowed, split, moved, or reported. It must not silently paint through table or page boundaries.
Controlled output fails closed. Missing glyphs, impossible layout, corrupt source structures, and invalid image bytes must produce typed diagnostics in controlled mode.
No host policy in vyuh_docx. The renderer receives values and options. DocsIQ decides lifecycle, policy, signatures, audit, storage, and authorization.
Use visual oracles. Byte-only tests are not enough for layout. Use raster fingerprints, extracted text geometry, and fixture PDFs where possible.
Optimize after correctness. Layout correctness comes before file size and speed, but all hot paths should be designed for large SOPs.

Agreed Execution Plan (2026-07-02)

Goal statement, agreed with the product owner:

Make vyuh_docx more trusted than the Syncfusion .NET server for the regulated SOP document class — not broader than Syncfusion for every Word feature.

"More trusted" is defined by properties the .NET server does not offer: deterministic bytes, fail-closed typed errors, measurable fidelity, no native/external service dependency, controlled fonts, explicit metadata stamping, and audited testable generation. Full Word breadth (equations, SmartArt, EMF/WMF, complex-script shaping, exotic floats, advanced fields) is explicitly not the target.

This plan fixes the execution order. It supersedes the phase ordering in the Improvement Roadmap below (which retains the detailed work lists): the oracle comes first, not fourth, and font fidelity plus field stamping are folded into Stages 1-3 rather than deferred — DocsIQ needs stamped metadata in the document font from day one.

Stage	Work	Exit gate
-1. Corpus + gates	Collect 5-10 real DocsIQ SOP/SFDT templates: front-page metadata table, reviewer/approver table, header/footer tables, long IDs, images/logos, multi-page bodies. Define acceptance gates per fixture.	Corpus committed as fixtures; gates written down.
0. Oracle harness	Multi-signal, not SSIM alone: raster diff/SSIM, text extraction, text geometry, page count, and table/cell/page bound checks (no text outside any box). LibreOffice as typography oracle. Baseline committed.	Every later renderer change gets an objective fidelity score.
1. Layout-box flip	Single-column sections render from `PdfLayoutDocument` boxes (the path already exists for multi-column); delete the duplicate measure/render walk in `pdf_exporter.dart`. Field resolution and font selection verified identical before the old path is deleted.	Byte-goldens identical or intentionally re-baselined; oracle score never drops; measure/paint divergence becomes structurally impossible.
2. Table engine	The main focus. Row height from resolved cell layout, explicit row splitting, repeated header rows across breaks, nested tables in parent cell content box, autofit and preferred-width percent, typed impossible-layout errors. Stamped fields inside cells use document fonts.	Geometry oracle shows zero text outside any cell/row/page box across the corpus.
3. Header/footer qualification	Default/first/even variants, furniture reserve diagnostics, metadata tables in furniture, footer collision fixtures, stamped fields in furniture with correct fonts.	All Acceptance Criteria in this document green.
4. Pagination fidelity	Widow/orphan, keepLines, keepWithNext, break rules.	Page counts match the Syncfusion editor across the corpus.
5. Long tail	Floating images and wrap modes, multi-column polish, font fallback chain.	Corpus SSIM at target across all fixtures.

Timeline estimates (focused working time):

DocsIQ approved-PDF quality (Stages -1 to 3): 4-5 weeks.
Strong SOP-class renderer (through Stage 5): 7-9 weeks.
General Word-renderer competitor to Syncfusion: not a useful target.

Corpus source (decided 2026-07-02): the committed real-SOP corpus at packages/vyuh_docx/test/fixtures/corpus/ — six genuine publicly-published SOP/protocol DOCX files (NIH clinical protocol, WHO ISO-15189 master SOP, table-heavy analytical method SOP, chemical-safety SOP, biosafety SOP, institutional template; provenance in the corpus README). Database exports are not used. The corpus is DOCX, exercising DocxReader -> PdfExporter; the SFDT leg is exercised by roundtripping the same corpus through tool/dump_corpus_sfdt.dart (DOCX -> AST -> SFDT -> AST -> PDF), matching the DocsIQ editor path. Stage -1 collection is complete (tool/render_corpus.dart renders 6/6).

Stage -1 / Stage 0 status: DONE (2026-07-02)

The oracle harness is built and the baseline is committed:

tool/src/pdf_raster_metrics.dart — pure-Dart PGM parsing, block SSIM, ink-density row correlation (localizes vertical drift, the overlap/overflow class), ink-coverage blank-page guard. Unit-pinned in test/pdf_raster_metrics_test.dart.
tool/oracle_harness.dart — per fixture: our PDF + LibreOffice PDF → pdftoppm rasters → SSIM/ink signals; pdftotext -bbox → word counts, geometry status, words-outside-page-bounds. Gates every signal against test/fixtures/corpus/oracle_baseline.json (may improve or hold; regression exits non-zero). Intentional changes re-baseline with --update-baseline.
test/pdf_corpus_gates_test.dart — pure-Dart slice run on every dart test: corpus/baseline sync, renderer-version match, and exact page-count pins per fixture. No external tools needed at test time.

Word-box deltas are only gated when the geometry comparator reports OK in both runs — on real corpus documents it currently reports count/text mismatches against LibreOffice, so the sentinel value 1.0 is a status, not a measurement.

Baseline snapshot at renderer 1.3.2 (the honest starting line): page counts 6/6, 67/68, 8/6, 8/6, 12/13, 19/19 (ours/LibreOffice); meanSSIM 0.58-0.71; word-extraction ratios 0.92-0.99; words outside page bounds: 0 on all six. The Stage 1 flip and Stage 2 table engine are measured against this.

Stage 1 status: DONE (2026-07-02)

Single-column sections now render from the layout-box model. In pdf_exporter.dart, every section builds a PdfLayoutDocument via buildLayoutDocument() and paints through _renderLayoutPage(); the duplicate _renderPage() AST walk is deleted. One behavioral gap surfaced and was closed on both sides of the lockstep boundary: page-top paragraph space-before collapse (the old painter's collapseParagraphSpaceBefore for the first node) is now encoded in _buildLayoutPage box stacking AND passed by _renderLayoutPage for the first box of single-column pages. Column sections keep their historical no-collapse behavior — changing that is a layout decision, not a path flip.

Proof: full suite green (558 tests, including byte-pinned and raster-pinned goldens — output is byte-identical to the pre-flip renderer) and the oracle harness reports metrics identical to the committed baseline.

Remaining honesty note: _renderLayoutPage still re-renders box.sourceNode through _renderNode at the box position. Block placement is now lockstep with measurement; line-level painting still recomputes inside the node renderers. Driving painting from the boxes' resolved text layouts is the Stage 2+ direction, starting with table row/cell geometry.

Stage 2 progress (2026-07-02): containment gate green corpus-wide

test/pdf_table_containment_test.dart is the Stage 2 exit gate: it walks the layout-box tree for every corpus document and asserts rows inside tables, cells inside tables, cell content inside cells, and every ink-bearing text line inside its cell's horizontal band (nested tables recursively). Because the painter consumes the boxes (Stage 1), box containment is painted containment. Initial run found 1,357 raw violations (mostly an auditor frame error); the true inventory was 53 across three classes, all closed:

Empty-block phantom height (WHO 18, Northwell 8): collapsed blank rows (_rowIsEmpty + baseFontSize*1.15 fallback) measured smaller than their cells' empty-paragraph boxes claimed. Zero ink involved. Fix: cell child boxes clamp _blockIsVisuallyEmpty blocks to the cell box — lockstep with the row-collapse rule; non-empty blocks stay unclamped so real overflow remains visible.
Nested fixed-width table overflow (Stanford 10, UA 16): a nested dxa table wider than its parent cell (e.g. 9345 twips in a 456pt cell) kept its authored width in column scaling, row measurement, AND painting — genuine painted overflow past the cell edge, the DocsIQ defect class. Fix: the nested-table contract clampToAvailable threaded through resolveTableWidth -> columnOffsets/cellPlacements/rowHeights/gridLayout -> _tableRowLayoutBoxes (box side) and _renderTable (paint side); measureTableInWidth clamps unconditionally (in-a-cell is its semantic). Top-level tables keep Word's overflow-into-margin behavior.
Tab-only advance (WSU 1): a \t\t line claims tab-advance width past a narrow cell but paints no ink; text following a tab is still audited via its own fragments. The gate skips ink-free lines, documented in the test.

Proof: containment gate green on all 6 fixtures, full suite 564 green (page-count pins unchanged — the reflow did not shift pagination), oracle gates pass (Stanford/UA SSIM moved <0.001).

Still open in Stage 2 scope: explicit row-split semantics tests, repeated header rows across page fragments (behavior exists, needs corpus-level fixtures), typed impossible-layout diagnostics, and the Exactly-height clipping contract for non-empty content.

Stage 3 progress (2026-07-03): furniture qualification pinned

test/pdf_furniture_qualification_test.dart qualifies the DocsIQ approved-PDF furniture shape from actual PDF content streams (per-page run extraction with embedded-glyph decoding): stamped merge fields resolve to server values inside header tables on every page (never cached fallbacks), PAGE/NUMPAGES render correct per-page values, first-page and even-page variants land on exactly the right pages, body text never enters the header or footer bands, and strict font mode accepts stamped furniture fields.

The renderer needed no fixes — these behaviors were already correct but unpinned. One Word semantic is now encoded as a test: differentFirstPage gives page 1 its own footer, blank unless authored; DocsIQ templates must author firstPageFooter explicitly to get a footer on page 1.

Extraction facts for future test authors: embedded-font runs encode spaces as advances (decoded text is space-free) and paint word-level Tj fragments, so per-run matching uses single words.

Still open in Stage 3 scope: furniture reserve diagnostics (typed error when furniture is taller than the body band — shares the typed-diagnostics framework with Stage 2's impossible-layout errors), and table-heavy furniture fixtures at corpus scale.

Northwell protocol regression target status (2026-07-11)

The real SOP/protocol fixture test/ref/DOCX/northwell_phase23_protocol.docx is now an explicit high-signal regression target for the editor and server conversion path:

text

DOCX -> AST -> SFDT -> AST -> DOCX/PDF

Fixes closed while qualifying this target:

editor-facing SFDT flattens floating images and shapes before Syncfusion open;
the docx_editor bridge strips stale float offsets and invalid visual extents before every documentEditor.open(...);
tabs and line breaks round-trip as typed AST inlines through SFDT;
header/footer PAGE and NUMPAGES fields remain editor-safe but recoverable via Vyuh side-channel metadata, so server PDF rendering sees generated page fields instead of cached text;
DOCX reader handles concrete numbering startOverride values and treats numId=0 as numbering cancellation, not a live list id;
DOCX reader suppresses package-only section carrier paragraphs while keeping section paragraphs with authored visible content;
DOCX export writes section-specific header/footer parts, explicit widowControl off toggles for styled paragraphs, explicit italic off toggles for normal runs, and start overrides for restarted bullet/numbered lists.
DOCX export preserves non-default section break types (continuous, evenPage, oddPage) instead of allowing OOXML's omitted-type default to reopen as nextPage.
Imported tables with no authored tblBorders are modeled as plain tables, and plain tables no longer emit generated fallback borders on DOCX export.
SFDT import/export preserves explicit None table-cell borders instead of collapsing authored no-border sides.
Raw inline XML now survives default SFDT export as editor-safe empty text runs with Vyuh side-channel metadata; docx_editor strips that metadata before Syncfusion opens the SFDT.
DOCX export gives nested lists explicit abstract numbering definitions rather than falling back to Word's mixed decimal/lower-letter/roman defaults, and it writes restart overrides for every emitted list level, including zero-based starts.
SFDT export avoids duplicating paragraph properties that exactly match the paragraph's named style, and preserves explicit zero spacing in style definitions so built-in heading fallbacks do not reappear after DOCX regeneration.
DOCX export preserves authored non-black document-default run color while avoiding materializing Syncfusion/SFDT's synthetic black fallback as explicit DOCX metadata.
SFDT import treats absent table-level borders as a plain table style, so SFDT -> AST -> DOCX no longer materializes grid borders that were never authored; complex_sfdt.docx is now strict-clean across all three corpus legs.
DOCX style parsing only applies built-in heading/title fallbacks in empty-docDefaults compatibility mode, preventing generated partial heading styles from gaining synthetic Arial font-family metadata.
SFDT import and DOCX export now treat pure Calibri/11/black editor defaults as synthetic fallback metadata, so inherited default fonts do not become explicit run or docDefaults formatting after a full SFDT -> AST -> DOCX path.
SFDT export now expands TOC cached content for editor viewing and, in strict round-trip mode, tags the first cached block with Vyuh metadata so SfdtReader can rehydrate the original DocxTableOfContents node. docx_editor strips the TOC metadata before Syncfusion opens the SFDT.
DOCX reader now recognizes plain field-code TOCs, not only structured docPartGallery="Table of Contents" SDTs. Northwell's TOC is a field paragraph followed by cached TOC 1/TOC 2/TOC 3 PAGEREF paragraphs and a field-end-only paragraph; it now becomes a semantic DocxTableOfContents while dropping scaffolding paragraphs and preserving the cached visible entries.
SFDT and DOCX strict paths now recurse into DocxTableOfContents.cachedContent for list, image, and hyperlink support data. Cached TOC list paragraphs emit matching SFDT lists/abstractLists, SfdtReader coalesces them back into DocxList inside the TOC node, and DOCX export collects the same cached TOC lists for numbering.xml.
Raw inline DOCX export now wraps run-child XML fragments such as VML <w:pict> horizontal rules inside <w:r>, while leaving paragraph-child fragments such as <w:r> and <w:fldSimple> direct. This preserves DocxRawInline nodes across DOCX export/re-read for the URS fixture.
DOCX reader now preserves mixed physical runs that contain multiple renderable children, such as <w:t>Before</w:t><w:br w:type="page"/><w:t>After</w:t>. InlineParser.parseRunParts splits the run in source order so text, page breaks, tabs, drawings, and note references are not collapsed into the first detected child.
SFDT strict round-trip metadata now preserves w:lastRenderedPageBreak evidence on text runs as vyuhLastRenderedPageBreakBefore, while editor-facing SFDT continues to omit that Vyuh-only marker.
PDF pagination now suppresses otherwise blank pages that contain only discardable empty paragraphs stranded before a visible block. This closed one Northwell page-count delta caused by empty spacer paragraphs before a keep-with-next schematic chain.
DOCX reader now treats mc:AlternateContent wrappers that contain DrawingML or VML as renderable run parts. Northwell's Example #1/#2/#3 schematics no longer degrade into raw inline textbox text during import; the wrapped wps:wsp shapes are parsed as typed DocxShape nodes with source drawing ids preserved.
SFDT import preserves shapeId, and DOCX shape export preserves source shape ids, invisible-line widths, and rounded EMU extents. The strict conversion sweep stays stable after typed shape import instead of drifting on generated shape names or avoidable point/EMU truncation.
Strict SFDT now preserves the authored DocxShapePreset as vyuhShapePreset. Editor-facing SFDT still emits only Syncfusion's autoShapeType, but internal DOCX -> SFDT -> AST round trips no longer collapse Word-only presets such as cloud, callouts, flow-chart shapes, or less-common connectors into rectangles.
Grouped DrawingML shapes are now modeled as an outer DocxShape with source-backed childShapes. The parent preserves the drawing extent for pagination, while child shapes preserve local offsets, extents, text, preset geometry, line/fill metadata, and source ids through strict SFDT and DOCX round trips.
DrawingML shape fill and outline parsing is now scoped to direct wsp:spPr children. A shape-level a:noFill no longer accidentally inherits the stroke's nested a:ln/a:solidFill, so outlined no-fill SOP boxes and connectors stay transparent instead of being painted with their outline color.
DrawingML shape fill and outline parsing now resolves direct a:solidFill/a:schemeClr values through the document theme palette, including DrawingML aliases such as tx1, bg1, tx2, and bg2, plus common tint, shade, lumMod, and lumOff transforms. The Brown SOP template's theme-colored connector line now reaches the AST/PDF path as a concrete color instead of disappearing as an unresolved scheme reference.
DrawingML wps:style fallback references are now honored for missing direct fill/outline colors: a:fillRef and a:lnRef resolve through the same theme color path, while explicit shape-level or line-level a:noFill still blocks the fallback. This preserves authored theme styling without overriding direct source intent.
Floating DrawingML shape anchors now preserve authored wrap distances, relativeHeight, simplePos, locked, layoutInCell, allowOverlap, and wp:effectExtent through DOCX, strict SFDT, and DOCX export. PDF text-wrap obstacles use the same shape wrap/effect extents exposed on layout boxes, so SOP diagram text flows around the authored anchor footprint instead of the raw shape rectangle. SFDT Behind shape wrapping now also sets the DOCX behindDoc anchor flag to prevent behind-text diagrams from drifting to in-front-of-text after a full SFDT -> AST -> DOCX -> AST leg.
Northwell-specific DrawingML preset geometry no longer collapses to plain rectangles for the common SOP diagram subset: round2SameRect, leftBracket, and rightBracket are now represented in the AST, preserved through strict SFDT/DOCX round trips, and painted in PDF; existing flowChartDecision presets now render as diamonds instead of the rectangle fallback. Editor-facing SFDT remains conservative while strict metadata preserves the authored preset name.
Nested grouped DrawingML transforms now compose a:off, a:ext, a:chOff, and a:chExt through parent group scale/offset, then quantize back to EMUs before converting to points. This prevents fractional point/EMU drift across strict DOCX/SFDT round trips and keeps nested child text boxes in their authored group-relative positions.
DrawingML connector line ends are now source-backed. a:headEnd and a:tailEnd marker types are parsed into the AST, preserved through strict SFDT metadata and DOCX export, and rendered in PDF for line/connector presets as visible arrowhead, diamond, or oval markers.
DrawingML shape transforms now preserve flipH and flipV through the AST, strict SFDT metadata, and DOCX export. PDF line/connector rendering maps the authored OOXML endpoints through those flips before painting, so Northwell's flipped diagonal line connector no longer collapses into a horizontal rule or points the wrong way. Simple bentConnector2/bentConnector3 presets now render as orthogonal elbow paths instead of diagonal fallbacks.
DrawingML connector stroke styling now preserves a:ln@cap and a:prstDash@val through the AST, strict SFDT metadata, and DOCX export. PDF line/connector rendering maps flat/round/square caps to PDF line-cap operators and renders common dot/dash/dash-dot presets with deterministic stroke-width-scaled dash arrays. Northwell's authored cap="flat" and prstDash val="solid" line metadata now survives the full strict path.
DrawingML custom geometry now preserves the common SOP subset of a:custGeom/a:pathLst/a:path: moveTo, lnTo, cubicBezTo, and close commands. Paths are parsed into typed AST data, carried through strict SFDT as vyuhCustomPaths, exported back to DOCX as a:custGeom, and painted in PDF by scaling authored path coordinates into the shape box with flip-aware point mapping. Malformed, empty, zero-sized, or unsupported custom paths are skipped instead of breaking import.
PDF shape rendering now paints grouped child shapes inside the parent drawing extent and skips no-fill/no-stroke rectangle containers, preventing invisible Word group containers from adding phantom boxes.

Current evidence:

tool/docx_to_sfdt.dart converts the Northwell DOCX to SFDT with 1751174 JSON bytes, 4 sections, 89 top-level blocks, 2000 paragraphs, 7 tables, 5236 text runs, and 2 image runs.
PDF image export now has a controlled-rendering boundary: lenient mode keeps a visible placeholder at the authored extent for undecodable images, while strictImages fails closed with UnsupportedPdfImageException including source format, authored extent, and byte length.
Editor-facing SFDT image export now avoids browser/Syncfusion-unsafe image payloads: EMF/WMF/TIFF/SVG/WebP or unknown image bytes are replaced with a valid PNG placeholder at the authored extent. Strict internal preserveRoundTripMetadata exports still keep the original source MIME and bytes so conversion fidelity gates do not lose the authored media.
test/sfdt_editor_safety_test.dart now covers both the checked-in SOP corpus and the external DocsIQ import references under test/ref/DOCX, including northwell_phase23_protocol.docx. It rejects unsupported editor image MIME types, empty block/cell structures, floating image layout in default editor mode, missing styles, and invalid list references before editor.open(sfdt) sees the payload.
SfdtReader now mirrors the export-side image safety boundary for malformed editor payloads: invalid base64, empty imageString, or non-string image payloads import as a valid PNG placeholder at the authored extent instead of throwing during SFDT -> AST. Valid raw base64 and data URL images still keep their source bytes and detected extension.
Imported PDF list layout now distinguishes literal item measurement from full-list flow. A standalone imported DocxListItem still measures authored w:before, but full document list flow suppresses before-only numbering style spacing and only reserves paired before/after spacing. This preserves the WSU SOP 19/19 LibreOffice page-count baseline while still honoring source-backed paragraph spacing on imported list items that carry an authored before/after pair.
Imported list items now preserve paragraph w:tabs from DOCX and SFDT sources. This is required for cached Word TOCs that group entries as numbered paragraphs: dot leaders and right-aligned page numbers must survive DOCX -> AST -> SFDT -> AST -> PDF/DOCX instead of falling back to generic default-tab advancement.
OOXML caps/smallCaps now use real on/off semantics instead of treating element presence as true. A bare <w:caps/> still enables all-caps, while <w:caps w:val="0"/> and <w:smallCaps w:val="false"/> explicitly disable inherited casing. Parsed text runs preserve those explicit-off toggles for direct DOCX export, and generated styles.xml emits style-level caps and smallCaps. This fixes Northwell TOC/title mixed-case fidelity without reintroducing strict AST -> DOCX -> AST drift.
tool/faithfulness_sme.dart reports 100.0% feature faithfulness on the full DOCX -> AST -> SFDT -> AST -> DOCX/PDF path: text runs/chars, tables, table cells, images, page-number fields, footnotes, and section breaks all preserve at 100%; list items now preserve at 100%.
tool/conversion_faithfulness_sweep.dart now reports Northwell strict SFDT round-trip as OK; the Northwell DOCX-export regression for the list / paragraph reconstruction path is now covered by a focused round-trip test.
The conversion-faithfulness sweep now canonicalizes image payloads by decoded byte identity and rounds SFDT geometry to three decimals, so the remaining drift signal is about actual feature loss instead of MIME spelling or tiny coordinate noise.
Current corpus sweep evidence still needs to be refreshed against the latest round-trip fixes, but the previous Northwell DOCX-export drift from list / paragraph reconstruction is now guarded by regression coverage instead of being an open known failure.
The six-file local DOCX corpus under test/ref/DOCX now has 100% basic conversion faithfulness and no conversion crashes (tool/corpus_sweep.dart).
The same six-file corpus renders to PDF without crashing through tool/render_fixture.dart; generated smoke PDFs were written to /tmp/vyuh_render/docx_corpus_pdfs.
tool/pdf_smoke_gate.dart verifies direct DOCX -> PDF health with Poppler: PDF page count, raw page-object count, text extraction, and sampled raster ink coverage. The external reference copy at test/ref/DOCX/northwell_phase23_protocol.docx now passes with 68 rendered pages, 68 raw page objects, 187852 extracted text chars, 8268504 PDF bytes, and nonblank sampled pages 1, 34, and 68 after the dense Schedule of Activities/table-border/outline-numbering fixes below.
tool/layout_profile.dart profiles parsed sections, layout page spans, and per-table row/cell/paragraph measurements. It now mirrors PdfExporter header/footer reserve rules, reports first/default page reserves, and can search rendered node text with --find=.... Current Northwell evidence now splits the front matter the same way as the fresh LibreOffice reference for the TOC/body boundary: section 3 is 2 pages, absolute pages 6-7, and STATEMENT OF COMPLIANCE starts on absolute page 8. The source-backed fixes are:
1. cached TOC list items preserve their original paragraph style id;
2. imported list grouping carries the resolved paragraph run style into the list item fallback style, so field-heavy TOC entries use the authored 10pt TOC 1/2/3 styles instead of the document base size;
3. TOC 2/3 auto-spaced rows use the LibreOffice-observed ~14pt baseline pitch;
4. SFDT list export suppresses style-inherited spacing/line spacing to avoid direct-format churn across SFDT -> DOCX -> AST. The TOC cached height moved from 1297.9pt to 1241.5pt and no longer spills onto a third TOC page.
A source-backed body drift class is also closed: Northwell's Schedule of Activities table has fixed-layout visit columns around 23pt wide with 9.75pt left/right cell margins. Sparse marker cells can still proportionally compress impossible horizontal margins so checkmarks keep a usable content box, but dense narrow labels now preserve the authored margins. This matches the Word/LibreOffice behavior for SOP visit headers: labels such as Enrollment/Baseline wrap in the 2-6pt source geometry instead of borrowing a synthetic 20pt text box. The same table now measures at 1078.3pt in flow, its detailed table measure is 777.3pt, and the header row resolves to 301.0pt instead of 98.8pt, removing one downstream page-count delta.
Bordered non-exact table rows now reserve the horizontal border edges that the PDF exporter paints. This is source-backed by DOCX border sz/8 semantics and keeps measurement aligned with rendering. On the external Northwell reference, the Schedule of Activities table now spans pages 14-16 with a 1391.8pt flow footprint, 787.8pt detailed table measure, and a 302.0pt header row, matching the LibreOffice pattern where the large schedule matrix creates a third continuation page. The total document is still one page short because later body content repacks differently.
Linked OOXML numbering styles are now parsed for imported SOP outline headings. Northwell uses concrete numIds that point at an abstract numbering with w:numStyleLink, while the real levels live on a sibling abstract numbering with w:styleLink. Vyuh now resolves that pair and carries parent counter seeds across separated style-linked heading lists, so the PDF renders 2. INTRODUCTION, 2.1. STUDY RATIONALE, and 2.2. BACKGROUND instead of bullet markers or stale 1.4/1.5 counters.
A fresh LibreOffice 26.2.3.2 conversion on 2026-07-11 renders the external Northwell reference copy as 69 pages. Vyuh's direct smoke render currently produces 68 pages after the dense Schedule of Activities, table-border, and outline-numbering fixes. This is better body pagination fidelity but not a global oracle pass: the single-file external oracle run reports 68 Vyuh pages versus 69 LibreOffice pages, meanSSIM 0.5929, 31147 extracted words versus 27622 oracle words, and 0 words outside page bounds. The remaining page-count delta is now 1; downstream body pagination is still short after the SoA table. Do not re-baseline this as accepted parity.
Table splitting now handles a later overflowing body row after earlier body rows have already fit on the page, and split row fragments no longer reapply the original row's full AtLeast minimum height to every continuation fragment. This closes the Northwell Objectives table boundary where 4.1. OVERALL DESIGN now appears on page 20, matching the local LibreOffice boundary instead of drifting to page 21. Regression coverage: dart test test/pdf_pagination_test.dart -n 'later over-tall table row|split row continuation' and the focused layout suite dart test test/pdf_layout_model_test.dart test/pdf_pagination_test.dart (229 passed, 3 skipped for optional report fixtures). The direct Northwell smoke gate remains 68 pages with 68 raw page objects, 187842 extracted text characters, and sampled ink on the first, middle, and last pages; this is a local pagination fix, not a global oracle acceptance.
tool/oracle_harness.dart now has a refreshed LibreOffice/Poppler baseline for the checked-in SOP corpus. The checked-in Brown SOP template currently renders as 6 pages versus LibreOffice's 6, with meanSSIM 0.5916, 2018 extracted words versus 2092 oracle words, and 0 words outside page bounds. The checked-in Northwell fixture currently renders as 67 pages versus LibreOffice's 68, with meanSSIM 0.5717, 29248 extracted words versus 26527 oracle words, and 0 words outside page bounds. The lightweight corpus page-count gate still passes, but the current full six-fixture oracle gate reports one checked-in Northwell ink-correlation violation (minInkCorr -0.118876 -> -0.179244). That fixture does not carry the explicit 9.75pt Schedule of Activities cell margins present in the external reference copy, so this is tracked separately from the dense-header margin fix. WSU remains 19/19 after the imported-list spacing rule above.
Missing embedded image parts now import as visible placeholder images rather than disappearing from the AST/editor. The DOCX reader preserves authored extent for broken VML/DrawingML media paths, and the regression guard dart test test/docx_reader_vml_image_test.dart covers the placeholder behavior while keeping the valid-image cases green. The DocxBuiltDocument import report now also carries explicit warnings for missing image parts so regulated callers can surface or reject the substitution, and SfdtExporter preserves those warnings only in round-trip metadata mode so the editor-facing SFDT path stays lean. This improves trust for SOPs with partially damaged media packages without changing the happy path.
DocxValidator now consumes DocxBuiltDocument.importWarnings and can either surface them as warnings or fail closed when failOnImportWarnings is enabled. That gives controlled callers a single trust gate for imported documents instead of having to stitch the reader warnings into a second policy layer.
The DOCX block parser now reuses a single inline parser instance for both body and table-cell parsing instead of allocating a second parser per document import. This is a small but measurable import-path cleanup for large SOPs: no semantic change, less churn on the reader hot path.
The DOCX reader now indexes archive files once per import and resolves readContent / readBytes lookups from that index instead of rescanning the zip on every access. It now also caches decoded content, bytes, and parsed XML so repeated part access during large SOP imports does not redo the same decode/parse work across relationships, headers, footers, fonts, and media.
The reader stack now reuses the cached XML/byte reads in relationship and embedded-font loading as well as the main document.xml parse, so import-time relationship resolution and font preservation no longer re-open archive parts or reparse the same XML after the first access.
The reader context now snapshots archive file names once and reuses that sorted list for custom XML discovery instead of rebuilding a fresh file list and sort on every import.
The reader context now caches parsed part relationships for shared header and footer parts, so repeated section passes do not reparse the same .rels file when multiple sections point at the same authored furniture.
Style, theme, and numbering parsing now use cached XML documents when available, so repeated lookups for the same authoring parts stay inside the shared XML cache instead of re-parsing from raw text.
Comments, footnotes, endnotes, settings, and embedded fonts now follow the same cached XML path when available, removing the last repeated parse passes from the main DOCX import flow.
Custom XML item-properties now reuse the cached XML path for item-id extraction while preserving the raw part text for round-trip fidelity.
The block parser's numbering fallback now uses the cached numbering XML document instead of reparsing the raw part when list metadata must be inspected a second time.
Paragraph-level section breaks now call the section parser directly on the authored w:sectPr element instead of wrapping it in a synthetic document fragment and parsing that wrapper.
The reader context no longer carries a dead raw numbering XML cache field; numbering inspection now flows through the shared XML document cache only.
The remaining raw XML fallback paths are now narrow safety nets rather than the main import route, so repeated SOP imports stay on the shared cached document state.
Malformed XML parts now surface as explicit import warnings while preserving the rest of the document, so controlled callers can reject or review a broken source package instead of discovering the damage only through missing layout.
External linked images now surface an explicit external-image warning and still import as visible placeholders, so audit trails distinguish a broken package from an intentionally linked asset.
Image-shaped draw objects now also reject non-image relationship types with a visible placeholder, instead of trusting the relationship payload blindly.
The dead public font-reader export has been removed, keeping the DOCX reader surface centered on the cached XML path used by the orchestrator.
Header background images now resolve their part relationships through the shared cached relationship lookup instead of reparsing the header rels file.
Header background images now also reject non-image and external relationship targets with explicit warnings while preserving a visible placeholder.
The section parser now consumes the document-wide even/odd header toggle from cached reader context instead of rereading settings.xml during section parsing.
The DOCX reader now also caches w:defaultTabStop once during import and stores it on DocxBuiltDocument, so PDF layout does not need to reparse settings.xml for tab geometry.
PdfExporter now caches that tab stop in points for the current export instead of recomputing the division in the hot layout path.
DOCX import now reads settings.xml before section materialization so document-wide toggles like even/odd headers are available when the section AST is built.
DOCX export now writes the model's defaultTabStopTwips back into settings.xml even when the source document had no preserved settings part, keeping the round-trip authoritative for AST/SFDT-built docs.
Strict SFDT export/import now preserves defaultTabStopTwips as round-trip metadata so AST -> SFDT -> AST keeps the cached tab geometry for internal fidelity tests without polluting editor-facing SFDT.
Malformed settings.xml parts now still import cleanly and fall back to the OOXML 0.5" default tab stop, while surfacing a warning for audit.
Non-positive w:defaultTabStop values now also warn and fall back to the OOXML 0.5" default, so invalid but parseable settings no longer fail silently.
Parser-level style, numbering, footer/header, and embedded-font failures now surface import warnings instead of disappearing through a silent catch path.
Relationship-manager content-types and .rels parser failures now also surface import warnings instead of being swallowed.
Malformed comments parts now surface warnings as well, instead of being silently dropped by the reader.
Malformed drawing subtrees now fall back to raw XML with an import warning instead of crashing image- or shape-heavy documents during import.
Invalid drawing fill/outline colors now degrade to warning-only null colors instead of throwing during shape import or exporting invalid color state.
Invalid theme palette entries now sanitize to the authored defaults instead of leaking malformed hex values into later color resolution.
Invalid direct run, underline, and border colors now sanitize during style parsing instead of leaking malformed hex into the AST.
Invalid SFDT text, fill, and border colors now sanitize on import instead of rehydrating malformed editor state.
Invalid table border colors now fall back safely in PDF table rendering instead of poisoning synthesized border sides.
Invalid CSS border colors now resolve through the shared HTML parser utility instead of leaking malformed color tokens into inline/table styling.
PDF font measurement now caches short text widths per font/style tuple so repeated SOP layout on common runs and TOC entries avoids redundant hot-path measurement work.
PDF paragraph measurement now caches repeated measurements for the same paragraph/page geometry during a single pagination pass, which cuts repeated split-probe work on long controlled tables and sections.
PDF table row-height measurement now caches per-table, per-width results within a pagination pass so repeated split probes do not re-derive the same row geometry for large SOP tables.
PDF footer height measurement now caches repeated identical footer nodes per section so multi-page SOP chrome does not remeasure the same footer blocks on every page.
PDF header/footer chrome rendering now caches page-independent repeated content, link rectangles, bookmarks, outlines, and image usage per section so identical SOP furniture can be replayed without re-rendering on every page.
The DOCX reader orchestration now uses a shared cached part helper for the main authoring parts, reducing repeated boilerplate around readContent / readXml pairing while keeping round-trip preservation intact.
The paired part loader now lives on ReaderContext, so import orchestration can fetch raw text and cached XML from a single cache-aware call site.
Custom XML item properties now also use the shared part loader, removing the last paired raw/XML read from the importer.
External and unsupported image relationships now skip pointless archive lookups and go straight to the placeholder path with explicit warnings.
ReaderContext.readPart() now does the parse/cache step itself, so the paired raw/XML load is a single cache-aware code path and not a wrapper around two separate lookups.
Part-specific relationship files now use that same shared loader path instead of reopening .rels through a separate XML entry point.
The relationship manager itself now uses the shared part loader for content types and relationship files, so its XML reads are cache-aligned with the rest of the importer.
The string-based style/theme/numbering parser fallbacks have been removed, so those parts now take the cached XML path only and do not retry the same parse through a second code path.
The section parser now uses the shared part loader for header, footer, and header-background XML payloads as well, keeping its asset reads aligned with the same cache contract.
The block parser’s numbering fallback now also uses the shared part loader, so list detection stays on the cached XML path even when it has to inspect numbering metadata again.
Custom XML item payloads now go through the shared part loader too, so the item discovery loop stays on the same cache-aware path as its props lookup.
The main document body and comments now also come through ReaderContext part loading, so the importer no longer mixes raw content reads with the shared cached part path.
The current trust/performance pass is green on the focused DOCX reader tests plus the Northwell SOP smoke gate, which still renders 68 pages with sampled ink present on the first, middle, and last pages.
Fresh corpus evidence on 2026-07-12 shows 18/18 conversion legs OK and 0 drifts across the checked-in SOP corpus. The current corpus sweep is green on Northwell, Stanford, UA, WHO, and WSU across all checked conversion paths.

Improvement Roadmap

Phase 1: DocsIQ Approved PDF Fidelity

Goal: make approved PDFs reliable for server-side DocsIQ generation.

Work:

formalize metadata field keys used by DocsIQ;
ensure PdfGeneratedFieldResolver applies in body, tables, headers, and footers;
add front-page metadata table fixtures;
add header/footer metadata table fixtures;
guarantee stamped metadata follows document/default font selection;
add strict font tests for metadata fields;
add geometry tests for no text outside table/page bounds.

Primary files:

pdf_generated_field_resolver.dart
pdf_layout_engine.dart
pdf_layout_model.dart
pdf_exporter.dart
sfdt_reader.dart

Phase 2: Table Layout Engine Hardening

Goal: remove the class of table overflow shown in the current PDF preview.

Work:

make row height calculation depend on resolved cell text layout;
expose row and cell content geometry in PdfLayoutModel;
make row splitting explicit and testable;
repeat header rows across page fragments;
support nested table measurement inside cell content boxes;
detect impossible rows with typed diagnostics.

Tests:

large reviewer/approver table;
long user IDs;
unbreakable UID strings;
mixed font sizes in one cell;
rows with nested paragraph spacing;
header table repeated across pages;
first-page metadata table plus body table.

Phase 3: Header/Footer Layout Qualification

Goal: make page furniture reliable across sections and page variants.

Work:

verify default, first-page, even-page header/footer behavior;
add furniture reserve diagnostics;
test footer collision with body tables;
support table-heavy headers and footers with metadata fields;
ensure page number fields render after final pagination count is known.

Phase 4: Layout Oracle Tooling

Goal: quantify visual fidelity instead of relying on manual inspection.

Work:

keep deterministic PDF byte tests for smoke confidence;
keep raster fingerprints for broad visual drift;
add text geometry delta tests for table and page bounds;
add fixture sweeps for real DocsIQ templates;
compare against LibreOffice/Word/Syncfusion/PDFium where feasible;
record renderer version with every golden update.

Existing tools to continue using:

tool/pdf_text_geometry_delta.dart
tool/pdf_text_geometry_delta_sweep.dart
tool/pdf_page_delta_sweep.dart
tool/pdf_smoke_gate.dart
tool/conversion_faithfulness_sweep.dart
tool/pdf_corpus_oracle.dart

Phase 5: Performance And Large SOPs

Goal: handle production documents without excessive CPU, memory, or PDF size.

Work:

cache paragraph text measurements by style/run/width;
avoid re-measuring header/footer furniture for every page when identical;
reuse page-independent header/footer chrome streams, annotations, and image references when the section furniture does not depend on PAGE/NUMPAGES;
preserve lazy font embedding;
stream or chunk large image operations where possible;
add stress tests for long SOPs, many tables, many fields, and many pages.

Degenerate Cases

Case	Required Behavior
Missing SFDT sections	Render an empty/default document or fail with a typed parse error in strict flows.
Empty header/footer	Reserve zero and render nothing.
Header/footer taller than body area	Fail with a typed layout diagnostic in controlled mode.
Long unbreakable UID/user ID	Keep inside cell by deterministic wrapping/overflow strategy or fail closed.
Null metadata value	Render empty if explicitly supplied, otherwise use cached source text where safe.
Missing metadata key	Use cached source text when available; otherwise empty/diagnostic depending on field strictness.
Missing font	Use fallback in lenient mode; throw in `strictFonts` controlled mode.
Missing bold/italic variant	Synthesize only if policy allows; otherwise fail in strict controlled mode.
Table row cannot fit on empty page	Fail with a typed impossible-layout error.
Nested table wider than cell	Reflow to parent content width or fail with diagnostics.
Many pages	Guard against runaway pagination in tests/debug paths.
Corrupt/unsupported image bytes	Preserve a visible authored-extent placeholder in lenient PDF mode; throw `UnsupportedPdfImageException` in `strictImages` controlled PDF mode; emit editor-safe PNG placeholders in default SFDT editor mode while strict SFDT round trips preserve source bytes; recover malformed SFDT image strings to PNG placeholders during import.
Different first/even page furniture missing	Fall back according to Word-like section rules.

Test Strategy

Every renderer improvement should include at least one of these test classes:

Unit parser tests: Verify DOCX/SFDT properties reach the AST.
Layout model tests: Verify page, paragraph, table, row, cell, and text geometry.
PDF text tests: Verify expected text appears and page count is stable.
Raster tests: Verify output is visible and pinned when a fixture is intentionally stable.
Geometry oracle tests: Verify extracted words stay inside page and table bounds.
Controlled failure tests: Verify strict fonts, impossible layout, corrupt inputs, and missing required field values fail with typed errors.

Important existing tests:

test/pdf_docs_iq_fixture_test.dart
test/pdf_layout_model_test.dart
test/pdf_pagination_test.dart
test/pdf_font_family_fidelity_test.dart
test/pdf_source_metrics_test.dart
test/sfdt_tabs_fields_test.dart
test/sfdt_roundtrip_features_test.dart
test/docx_reader_table_width_test.dart
test/pdf_image_embedding_test.dart

DocsIQ Integration Shape

DocsIQ server should call vyuh_docx like this:

dart

final document = const SfdtReader().read(contentSfdt);

final exporter = PdfExporter(
  creationDate: approvedAtUtc,
  strictFonts: true,
  defaultFontFamily: selectedControlledFontFamily,
  fieldValues: {
    'DocumentUID': documentUid,
    'DocumentTitle': title,
    'Version': version,
    'OwnerName': ownerName,
    'ApprovedAt': approvedAtUtc.toIso8601String(),
    'GeneratedAt': generatedAtUtc.toIso8601String(),
    'Reviewers': reviewerSummary,
    'Approvers': approverSummary,
  },
);

for (final font in approvedFontBundle) {
  exporter.registerFont(font.family, font.bytes);
}

final pdfBytes = exporter.exportToBytes(document);

The exact field key list should be owned by DocsIQ server contracts. vyuh_docx should only resolve the keys it receives.

Acceptance Criteria

For DocsIQ approved PDFs, we should not call the renderer "production-grade" until these are true:

a real Syncfusion SFDT sample renders to PDF without external services;
front-page metadata tables do not overflow;
header/footer tables do not overlap body text;
reviewer/approver tables wrap long names and IDs correctly;
page count is deterministic for fixed inputs;
PDF creation date is server-supplied UTC;
strict font mode catches unsupported glyphs;
generated fields use the document/default font;
raster output is visible and not blank/corrupt;
extracted text geometry stays inside page bounds;
failures produce typed diagnostics that DocsIQ can surface.

Current Priority

The next renderer work should focus on the current Northwell visual-drift frontier before broad table tuning:

text

Northwell DOCX
  -> DOCX/SFDT/AST import with mixed runs, lists, sections, and images intact
  -> PDF
  -> compare against LibreOffice page/heading locations
  -> reduce front-matter/resources drift with oracle-backed page-count evidence
  -> keep body pre-INTRODUCTION drift closed while checking downstream page
     shifts introduced by the narrow-cell margin compatibility rule

That does not replace the DocsIQ approved-PDF furniture slice; it gives the renderer a harder real-SOP pagination target first. Do not blindly honor w:lastRenderedPageBreak hints as hard breaks: they explain the front-matter title-page position but can worsen the total-page delta unless the remaining measurement defects are fixed in parallel. Once the real-SOP pagination drift is closer and oracle-backed, resume the approved-PDF slice with:

text

SFDT with body content
  + front-page metadata table
  + header/footer table fields
  + reviewers/approvers table
  + long IDs
  + controlled font selection
  -> PDF with no overflow and no font drift

Vyuh DOCX Golden Rendering Plan ​

Executive Summary ​

Product Requirement ​

Package Boundary ​

Current Code-Level Architecture ​

Public API ​

Input Readers ​

AST / Model Layer ​

PDF Pipeline ​

Existing Controlled Rendering Features ​

Recent Hardening ​

Renderer Problems We Are Solving ​

1. Metadata Stamping In Header/Footer Tables ​

2. Header/Footer Body Collision ​

3. Table Layout And Overflow ​

4. Font Fidelity ​

5. Layout Model As The Truth ​

External Research And What We Adopt ​

Dart pdf Package ​

pdfrx / PDFium ​

docs_gee ​

docx_creator ​

Golden Rules For Renderer Changes ​

Agreed Execution Plan (2026-07-02) ​

Stage -1 / Stage 0 status: DONE (2026-07-02) ​

Stage 1 status: DONE (2026-07-02) ​

Stage 2 progress (2026-07-02): containment gate green corpus-wide ​

Stage 3 progress (2026-07-03): furniture qualification pinned ​

Northwell protocol regression target status (2026-07-11) ​

Improvement Roadmap ​

Phase 1: DocsIQ Approved PDF Fidelity ​

Phase 2: Table Layout Engine Hardening ​

Phase 3: Header/Footer Layout Qualification ​

Phase 4: Layout Oracle Tooling ​

Phase 5: Performance And Large SOPs ​

Degenerate Cases ​

Test Strategy ​

DocsIQ Integration Shape ​

Acceptance Criteria ​

Current Priority ​

Vyuh DOCX Golden Rendering Plan

Executive Summary

Product Requirement

Package Boundary

Current Code-Level Architecture

Public API

Input Readers

AST / Model Layer

PDF Pipeline

Existing Controlled Rendering Features

Recent Hardening

Renderer Problems We Are Solving

1. Metadata Stamping In Header/Footer Tables

2. Header/Footer Body Collision

3. Table Layout And Overflow

4. Font Fidelity

5. Layout Model As The Truth

External Research And What We Adopt

Dart `pdf` Package

`pdfrx` / PDFium

`docs_gee`

`docx_creator`

Golden Rules For Renderer Changes

Agreed Execution Plan (2026-07-02)

Stage -1 / Stage 0 status: DONE (2026-07-02)

Stage 1 status: DONE (2026-07-02)

Stage 2 progress (2026-07-02): containment gate green corpus-wide

Stage 3 progress (2026-07-03): furniture qualification pinned

Northwell protocol regression target status (2026-07-11)

Improvement Roadmap

Phase 1: DocsIQ Approved PDF Fidelity

Phase 2: Table Layout Engine Hardening

Phase 3: Header/Footer Layout Qualification

Phase 4: Layout Oracle Tooling

Phase 5: Performance And Large SOPs

Degenerate Cases

Test Strategy

DocsIQ Integration Shape

Acceptance Criteria

Current Priority