Skip to content

Attest: Cloud Release and AI Authoring Plan ​

FieldAgreed value
StatusAgreed baseline — locked for implementation planning
Agreed2026-10-07
Validated refinements approved2026-10-07
Product owner and implementation repositoryVyuh / CDX
First customerCaliber
Initial productseLog and SSO
Next productDocsIQ

“Locked” records the agreed scope and architecture. It does not mean the implementation is complete, production deployment is authorized, or release readiness has been verified. Changes to this baseline should be recorded explicitly in this document. This is a delivery plan, not a standing agent instruction file or a replacement for the canonical product world models.

1. Product and release scope ​

Attest is a product owned and implemented in CDX. This is its single active cloud-release and AI-authoring plan, using Caliber as the first delivery. It replaces the older CDX product plans and the copy previously kept in flutter_elog. The Attest world model remains the normative contract reference; the plan does not replace it.

RepositoryOwnership
CDXAttest application, CLI, shared contracts, authoring/AI, runners, orchestration, Cloudflare deployment, storage, evidence, reports and platform documentation
flutter_elogCaliber adapter and launchers, product registries, manifests, datasets/fixtures, seed policy, eLog/SSO/DocsIQ test assets, branding, integration tests and consumer qualification records

Caliber names in examples and acceptance packs do not become platform defaults. Generic platform behavior must remain testable using CDX's independent Field Notes sample. Consumer repositories do not own forks of the Attest UI, engines or platform roadmap.

Preserve working local CLI/TUI and authoring behavior during the cloud work. Initialization remains non-destructive and does not silently provision clouds, reset application data or install automation. Registry-only distribution must be verified separately before claiming it works; the initial Caliber hosted pilot does not depend on public self-service onboarding. Developer commands and package usage are maintained in the CLI guide and editor guide.

Deliver one hosted Attest workspace for Caliber, connected to a dedicated QA environment. Caliber testers use a browser to select a product and test plan, execute supported suites, investigate failures, and download reports and evidence. Vyuh manages infrastructure, credentials, product integrations, and versioned test packs.

Build the deployment locally first. Use a dedicated self-hosted Supabase container stack for Attest, separate product-fixture data, and independent API, worker and frontend containers. A managed Supabase project and Cloudflare credentials are not prerequisites for this stage. Prove sign-in, a real queued API run, private evidence/report retrieval and persistence after container recreation before provisioning a hosted environment. Keep the proof's coverage explicit; a health-check smoke run does not establish Patrol or product qualification. See the Caliber local proof guide.

The first release tests already deployed candidate application builds. It does not provision a fresh application stack for every run. Start with one active run against the shared QA environment; queue additional requests to avoid conflicting changes to shared test data.

Ship reviewed AI drafting with hosted Attest. Add the bounded trial-and-repair agent after cloud execution is reliable. Adding DocsIQ must reuse the same platform and execution contracts.

The primary experience is scoped verification of a candidate release: what was selected, what was actually verified, what failed or remains unknown, and which evidence supports the result. A passing smoke run does not establish whole-release readiness. Deterministic qualification gates decide the outcome for the declared scope; Caliber's release authorization remains a separate human process unless explicitly integrated later.

Use existing build/run contracts for a verification snapshot containing the candidate, required coverage, qualification policy and exact supporting run references. Do not introduce a separate release-management subsystem for V1. Validate this workflow with Caliber testers and quality reviewers before treating regulated release confidence as final product positioning. AI generation and traceability alone are not unique capabilities.

2. Agreed architecture ​

text
Vyuh release pipeline
  ├── Deploy versioned candidate build
  │       └── Caliber QA environment
  │           SSO + eLog + supporting services + test database
  │                         ↑
  │                  API / UI interactions
  │                         │
  └── Publish matching test pack
                            │
Caliber browser             │
      ↓                     │
Cloudflare Worker           │
  ├── Flutter static assets │
  └── Attest Dart API container
            ↓               │
     Supabase control plane │
     Durable jobs and runs  │
            ↑               │
     Attest runner container┘
            ↓
     Private durable evidence storage
ModuleInitial deploymentResponsibility
DashboardCloudflare Worker static assetsProduct selection, authoring, execution controls, results
Cloud APIDart in a Cloudflare Container, routed through WorkerAuthentication, authorization, run requests, artifact access
Execution workerSeparate Cloudflare ContainerExecute approved suites and publish evidence
Container lifecycleWorker and Durable ObjectStart runners, monitor activity, stop idle instances
Durable control planeDedicated Supabase instance; self-hosted locally firstProjects, jobs, attempts, run records and publication metadata
Artifact archiveExisting private Supabase storage integrationRecordings, screenshots, logs, manifests and reports
Application under testExisting Caliber deployment infrastructureCandidate application and supporting services
AI authoringServer-side Attest authoring moduleGenerate and validate drafts; later coordinate bounded trial runs

Keep the existing Vyuh durable execution system as the job authority. Cloudflare manages compute lifecycle; do not introduce a second job orchestration system without a demonstrated need.

Use separate API and runner roles. Package pinned tools, dependencies and test assets into runner images. Build the Flutter dashboard in CI. The final choice between a small managed runner pool and fresh containers per job remains an implementation decision; the initial environment concurrency limit is one.

Container filesystems are temporary workspaces. Persist run records, configuration, authored drafts, approved test versions and evidence outside them. Attest's control-plane database must be separate from the application's test database.

Cloudflare reference: Container architecture.

Execution dimensions and optional browser evaluation ​

Preserve the existing separation of concepts:

DimensionExamples
Test surfaceAPI, action, UI, performance
RunnerDart test, Patrol, k6; a browser adapter only after explicit acceptance
Execution locationLocal process, durable worker, CI worker
Browser backendContainer Chromium; potentially Cloudflare Browser Run

Patrol remains the current implementation baseline. The engine rejects new direct Playwright runs; archived provider enum values do not enable support. A bounded comparison of deployed-browser execution with the instrumented Patrol path is permitted. Reintroducing Playwright requires an explicit architecture decision, corresponding contract/documentation changes and evidence qualification. This plan does not select or require a replacement.

Compare the same representative Caliber journeys on correctness, Flutter semantics/selectors, deployed-artifact fidelity, actor isolation, evidence, recovery and operational effort. Keep one proven UI path for V1. Do not build multiple providers merely to demonstrate extensibility.

Cloudflare Browser Run is an optional backend experiment, not a V1 dependency. Acceptance requires:

  • Correct rendering and interaction with the actual candidate frontend, including Flutter semantics and required downloads/dialogs.
  • Scoped authentication, separate actor contexts, session expiry handling and access to all SSO, API and asset origins without bypassing application guards.
  • Checkpoint screenshots and any required pixel recording, timestamps and actor provenance mapped to canonical Attest artifacts.
  • Recovery after session interruption or platform rollout, with partial evidence retained and no false completion.
  • Measured duration/cost, bounded session usage, explicit cleanup and account limits sufficient for the serialized QA workload.
  • Evidence export into Attest's private archive before provider retention ends.

As checked on 2026-10-07, Browser Run's built-in rrweb replay excludes canvas and WebGL content, so it cannot replace Flutter recording evidence. Its recordings expire after 30 days. Outbound Worker routing has HTTP-only and invocation-lifetime constraints; it is not proof that Caliber's private HTTPS or SSO flows work. Recheck provider capabilities when implementing. See recording limitations, outbound routing, session limits, pricing and Flutter web accessibility.

3. Candidate builds and test execution ​

  1. Deploy a versioned SSO/eLog candidate into the dedicated QA environment.
  2. Prepare known accounts, permissions and test fixtures.
  3. Publish the compatible versioned test pack.
  4. Verify readiness before enabling execution.
  5. Run API tests against the candidate's deployed endpoints.
  6. Prove the existing Patrol path in a cloud runner before exposing UI suites, or explicitly accept and qualify an alternative through the decision above.

For each UI suite, establish whether it drives the deployed frontend or uses an instrumented Flutter application connected to the QA APIs. If instrumentation is required, build from the same commit and configuration lineage as the candidate and record that distinction in evidence. Do not claim verification of the exact deployed frontend artifact when a different artifact was tested.

Each qualification run snapshots application/service build identities, test-pack and test-implementation versions, intent/requirement versions, product-configuration and grounding-bundle versions, qualification-policy version, dataset/seed version, resolved environment, execution provider, browser version, runner image/toolchain identity, attempts, results and evidence. Use immutable digests where available, not mutable labels alone. For generated fixtures, retain the generation inputs and resulting identity; record an explicit not-applicable data contract when no dataset is needed.

Capture the identities of the deployed eLog/SSO and supporting services involved in the verification. A local checkout commit or package version does not prove which deployment answered requests. Check deployment identity before and after execution and block a coherent qualification claim if it changes during the run or cannot be established.

Retries preserve prior attempts. A new rerun is linked to the original run rather than overwriting it. Record the exact run set and selection policy used for the displayed result; a passing rerun must not hide an earlier failure or combine incompatible builds, datasets or environments.

Each supported suite declares prerequisites, data mutations, cleanup behavior, concurrency constraints and retry safety. Infrastructure retries must not blindly repeat application mutations. Publication retries must not rerun tests.

AI trial runs use the same QA-environment lock as normal runs. Coordinate deployments, resets and fixture preparation with that lock as well. A single active runner alone does not prevent an external deployment from changing its target mid-run.

4. Caliber end-user journey ​

The normal navigation is Product → Candidate release → Verification → Result → Evidence. Internal plan, suite, dataset, run and attempt identities remain unchanged and accessible through detail views.

  1. Sign into Attest and select eLog or SSO; DocsIQ appears after onboarding.
  2. Confirm the candidate build, selected QA environment and readiness.
  3. Choose Smoke or a supported Qualification scope; see included checks and known coverage gaps before starting.
  4. Start verification. Show business-level progress first, with logs and execution details available through drill-down.
  5. Inspect the deterministic outcome, blockers, evidence and suggested diagnosis.
  6. Review or rerun eligible checks and download the report/evidence package.

Infrastructure configuration belongs in admin/advanced interfaces, but the selected environment, candidate and scope must remain visible. A smoke action must be labelled as smoke verification, not unrestricted release approval.

Results and qualification decisions ​

Preserve the existing qualified, qualified-with-exceptions, not-qualified and blocked outcomes. Show the selected scope, passed/failed/blocked/skipped and unverified coverage, approved exceptions, evidence completeness and timestamped environment/runner health. Unknown health must not appear healthy.

Use wording such as “Qualification passed for the selected scope.” A summary must not infer whole-release readiness from passing counts. Missing mandatory coverage or evidence prevents a complete qualification claim. Keep Caliber's human release disposition separate from the machine-computed result.

Offer evidence inspection, an eligible rerun and report download. Results and reports must reference the same immutable verification snapshot.

Failure classification ​

Record deterministic failure stage/code separately from a diagnosis. Use typed categories for product defect, test authoring/maintenance, environment, runner/infrastructure, test data/precondition, evidence publication and unknown. Preserve primary and contributing causes when more than one applies.

An AI diagnosis carries supporting evidence references, rationale and an explicit proposed/confirmed status; it does not change test outcomes. Human confirmation records who confirmed it. Do not relabel an assertion failure as a confirmed product defect without sufficient evidence, or call an unavailable environment a passed verification.

Closing the browser must not stop execution. Reopening Attest must recover the durable run state. Vyuh configures infrastructure credentials and endpoints; testers do not supply service-role keys or manually start services.

Start with manual run submission. Automatic smoke runs after deployment are a follow-on once manual execution is reliable.

5. Feature inventory and inspection baseline ​

This inventory records the source inspection performed on 2026-10-07. “Implemented” means implementation was found, not that the feature passed cloud acceptance testing. Documentation describes both existing and target capabilities; target capabilities are not release claims.

AreaFeaturesInspection assessment
Product catalogProducts, environments, plans, suites, tags, availability reasonsImplemented; Caliber registers eLog and SSO
Run configurationScope, browser selection, datasets, readiness checksImplemented; remote environments remain disabled
Environment managementLocal startup, preparation, seed/reset policies, remote endpoint configurationLocal implementation exists; cloud setup unverified
API testingCode suites, declarative cases, API sequence authoringImplemented; hosted authoring integration needed
UI testingPatrol journeys, declarative authoring, assertions, checkpointsLocal implementation; remote Patrol explicitly unavailable
Integration testingProduct action suites and workflow checksFramework exists; readiness is suite-specific
Performance testingk6, options editing, metrics, thresholds, scorecardsImplemented; cloud execution and capacity unverified
Execution controlsStart, progress, logs, cancellation, failed-suite rerunsLocal capabilities exist; hosted parity incomplete
Durable executionQueues, attempts, leases, recovery, separate publication workImplemented with focused tests; deployed recovery unverified
Live observationFlutter frames and checkpoint evidenceLocal implementation; cloud delivery unverified
Recording and replayWebM, screenshots, checkpoint-linked playbackImplemented; not full browser DOM/network diagnostics
History and analyticsLedger, search, product/build/session projections, cross-build analyticsLocal implementation; full hosted parity unverified
Evidence integrityManifest validation, checksums, inventory, publication receiptsImplemented; deployed durable storage must be exercised
Qualification gatesRequired checkpoints, actors, evidence and outcome evaluationImplemented; coverage depends on executable suites
Requirement coverageRequirement IDs linked to plan/checkpoint results, execution and evidence IDs; mixed-context rejectionImplemented foundation; approved requirement/intent version lineage incomplete
ReportsPDF, Caliber OQ packages, DOCX, report configurationLocal implementation; regeneration solely from durable data needs verification
Hosted accessToken verification, project authorization, membership routes, private artifactsFoundations exist; Caliber sign-in and isolation need end-to-end checks
Retention and recoveryPolicy contracts, private storage, immutable-record protectionsFull retention, legal-hold operations and restore verification incomplete
Developer toolsCLI, initialization, diagnostics, validation, registry/workspace toolingImplemented
AI draftingAPI/UI YAML generation, assumptions, validation, review and saveLocal implementation; not an autonomous authoring agent
Cloudflare deploymentLoopback-only Worker proxy rehearsalProduction containers and lifecycle integration needed
DocsIQShared platform reuseRegistration, adapter generalization, fixtures and suites needed

Some Caliber suites are explicitly planned/unavailable, including standalone navigation, area/equipment journeys and certain workflow journeys. Do not advertise those entries as implemented product coverage.

The current hosted application is not feature-equivalent to the local dashboard. Hosted authoring routes and other missing controls must be mapped explicitly before exposing their UI.

The current manifest traceability gate checks checkpoint assertion/evidence linkage. It does not by itself establish complete traceability to approved, versioned requirements. Extend the existing coverage contracts for this need. This inventory is not a requirement for full hosted feature parity in V1.

6. AI authoring: existing foundation ​

The existing AI authoring flow is:

text
Describe test → Generate YAML and assumptions → Validate → Review → Save

It has OpenAI and Gemini adapters and supports API-case and UI-journey drafts. It makes a single model request. It does not currently inspect the application, trial-run its draft, or iterate on execution failures.

The current workspace requests a provider key, and saving uses files in the product workspace. For the hosted Caliber offering, configure provider credentials server-side and persist drafts and approved versions durably. Provider/model availability and live request compatibility must be verified; mocked adapter tests do not prove them.

7. AI authoring: agreed target ​

The AI assistant is an authoring agent inside Attest. It uses Attest's existing validation, execution and evidence contracts rather than a separate runner.

Normal authors work with business-readable intent, prerequisites, steps and expectations. A structured editor serializes these into existing declarative contracts and uses the same validation path. YAML remains an advanced view; do not introduce a second execution language or an unconstrained natural-language compiler. Persist approved structured definitions, not only conversation text.

Stable intent and traceability ​

Extend the existing requirement inventory and plan/checkpoint contracts with a minimal versioned intent record: stable identity, source requirement/version, business invariants, prerequisites, coverage/evidence expectations and review provenance. Version the execution implementation separately and bind both into the published test pack and each run. Shared contracts belong in CDX; product requirements belong to the consuming product.

The retained chain is:

text
Requirement/version → Intent/version → Test implementation/version
  → Candidate and verification scope → Run/attempt/checkpoint
  → Evidence → Deterministic qualification result

One invariant may require several checks and evidence types. A successful UI click does not prove persistence, audit or authorization behavior. Missing links and unimplemented required checks remain explicit gaps. Reports use this same persisted chain, not AI-generated claims of coverage.

AI may propose implementation repairs, but even selector-only changes create visible revisions. Changes to business invariants or expectations require human review; they must never be disguised as execution repair. Qualification tests remain pinned during a run. Reuse existing contracts rather than creating a parallel intent database or test-management system.

Product knowledge and ownership ​

Each product supplies a curated, versioned grounding bundle referencing its canonical domain rules, requirements, workflows, roles/permissions, APIs, selectors, fixtures, constraints, data needs and existing tests. Record source IDs, versions/digests and unresolved conflicts. Prefer referenced or generated projections of authoritative sources over manually duplicated rules.

CDX owns the bundle contract, validation and provenance. Caliber integrations own eLog, SSO and DocsIQ content. Extend Attest's world model only for shared authoring/traceability concepts; do not move product rules into it. For example, eLog's X-1 access layers and I-3 product-owned roles remain authoritative; SSO login alone is not permission to perform an eLog action.

Observed application behavior is discovery evidence, not a business authority. If required context is missing or conflicts, return needs clarification with the specific unanswered question. A pause/resume example cannot establish who may resume QA Hold work without the applicable approved product rule.

Authoring loop and entry points ​

V1 supports requirement/intent-driven drafting, validation and reviewed publication. The full engineering loop below is follow-on after cloud execution is reliable; it is not a V1 autonomous-agent requirement.

text
Requirement + product contracts + existing examples
                         ↓
                 Draft steps and assertions
                         ↓
           Attest schema and policy validation
                         ↓
                 Trial run in QA
                         ↓
          Inspect evidence and classify failure
                         ↓
       Propose repair, request context, or identify likely defect
                         ↓
          Trial revised draft again only within allowed bounds
                         ↓
             Human review → Versioned test pack

Required capabilities:

  • Read approved product requirements, API contracts, UI selectors, fixtures and existing tests.
  • Produce constrained API-case or UI-journey definitions with explicit prerequisites, expected outcomes and assumptions.
  • Validate through the same parsers and policies as manually authored tests.
  • Request trial runs in the approved QA environment and retrieve their actual results and evidence.
  • Propose bounded repairs while distinguishing authoring errors, missing data, environment failures and application defects.
  • Retain authoring sessions, draft versions, trial-run links and review history.
  • Promote reviewed definitions into immutable, versioned test-pack releases.

The agent may request tools; Attest owns permissions, execution and recorded outcomes. Trial runs remain ordinary Attest runs, clearly labelled as authoring trials. Bound each session by iteration, execution-time and model-cost budgets.

Keep three entry points possible without implementing all three initially:

Entry pointDelivery scopeConstraint
Requirement → TestsV1Grounded proposal, validation and human review
Change → TestsFollow-onRecommend impacted coverage with explicit uncertainty; never silently omit mandatory qualification checks
Explore → TestsLong-term experimentApproved QA scope and bounded actions; observations do not define correct behavior

Assertions and qualification gates determine pass/fail. The model must not invent observations or weaken an expected outcome merely to make a failing application pass. A generated draft is not approved qualification coverage. Preserve requirement provenance and the reason for each proposed change.

Begin with existing declarative formats. Arbitrary generated Dart, shell execution and unrestricted autonomous publication are outside the initial scope. Treat retrieved application content as data, not as instructions granting tools or credentials.

User interaction ​

  1. Select Create test with AI, a product and an approved environment.
  2. Describe the intended behavior.
  3. Review proposed prerequisites, steps, assertions and missing information.
  4. Validate the draft.
  5. Trial-run it when that capability is enabled.
  6. Inspect evidence and proposed revisions.
  7. Have an authorized reviewer publish the approved test version.

Evaluating the authoring agent ​

Maintain known authoring tasks and measure whether the agent:

  • Produces valid and executable definitions.
  • Preserves the intended business assertion.
  • Detects deliberately introduced defects rather than only passing healthy cases.
  • Flags missing information instead of inventing contracts or selectors.
  • Respects environment restrictions and credential boundaries.
  • Stops within configured iteration, time and cost limits.

This is the second meaning of “a harness for itself”: Attest executes generated tests, and a separate evaluation set verifies the quality of the authoring loop.

8. Adding DocsIQ ​

DocsIQ reuses the same hosted Attest deployment, authoring engine, execution contracts, storage and reporting infrastructure.

Integration assetDocsIQ work
Candidate applicationDeploy DocsIQ and its required supporting services
Product registrationAdd DocsIQ to the Caliber distribution
EnvironmentRegister endpoints, readiness checks and credential references
Test packAdd executable suites, checkpoints and evidence requirements
DatasetAdd documents, policies and author/reviewer/approver accounts
AI contextSupply DocsIQ rules, API contracts, selectors and examples
Runner imagePackage the new tests and required dependencies

First remove SSO-versus-eLog assumptions in the Caliber adapter. In particular, other target IDs must not fall back to eLog endpoints, application version or suite selection. Use explicit product configuration.

Initial DocsIQ smoke coverage can cover sign-in, repository access and draft creation. Qualification then adds policy-driven submission and workflow evidence. DocsIQ rules DP-5 and DW-4 require policy readiness and the configured workflow path; checking a status field alone does not establish those behaviors.

Registration can be quick, but complete qualification depends on authored and verified suites and fixtures. Adding a product entry does not create coverage.

9. Delivery sequence and exit conditions ​

StepDeliverableExit condition
0. Local containersSelf-hosted Supabase, separate product-fixture database, API, worker and frontend imagesLocal sign-in, actual queued API execution, private PDF/evidence retrieval, access isolation and completed-run persistence survive container recreation
1. QA targetVersioned SSO/eLog deployment, isolated test data and accountsReadiness and representative API tests pass remotely
2. Hosted foundationDashboard, authenticated API, dedicated control planeAuthorized Caliber user queues and retrieves a durable run
3. Container executionPinned runner image and Cloudflare lifecycle integrationRunner claims a job, executes and publishes retained evidence
4. Cloud vertical sliceOne eLog API test, one eLog UI journey and one SSO UI journey; Patrol baseline or explicitly accepted alternativeAll three execute against approved QA from cloud workers with lineage and evidence; failure/restart probes remain honest
5. Hosted authoringStructured manual/AI editors, curated context, authenticated routes, durable intent/definition versionsReviewed draft survives restart and becomes a runnable versioned test without changing its approved intent
6. Pilot acceptanceScoped result, typed failure classification, recovery, access checks and reportsCaliber tester completes the end-to-end workflow unaided and understands verified versus missing coverage
7. AI trial-and-repairProduct grounding, bounded trial runs, repair proposals and reviewEvaluation set demonstrates meaningful assertions and controlled behavior
8. DocsIQ integrationProduct configuration, fixtures and initial coverage packDocsIQ executes and reports through the same platform

DocsIQ preparation may proceed alongside earlier steps, but execution claims depend on the relevant API/browser path being proven. No completion date is committed by this plan.

The three-test vertical slice is the first end-to-end proof, before broad authoring or hosted parity work. Select and record exact registered test IDs, candidate deployment and expected evidence before execution. Reuse a healthy QA deployment; do not bootstrap the local application stack inside the runner. Retain build/test-pack identities, environment, dataset contract, runner/provider provenance, results, logs/evidence and attempt history for all three tests. Include a deliberate assertion failure and runner/publication interruption to prove that a failed or incomplete verification cannot appear qualified.

An optional browser comparison can inform step 4, but it is not a second V1 implementation commitment. If the alternative does not meet evidence and fidelity criteria, retain the Patrol path with its instrumented-build scope clearly stated. Do not claim exact deployed-frontend coverage from that fallback.

HorizonCommitted scope
V1Caliber eLog/SSO, one QA environment, serialized execution, API plus one proven UI path, durable runs/evidence, scoped qualification gates, failure categories, basic reports/evidence inspection, reviewed AI drafting and minimal intent/context lineage
Follow-onBounded AI trial/repair, DocsIQ pack, change-impact recommendations, automatic deployment smoke triggers and additional proven suites
Experimental/long-termExploration-based authoring, alternative browser backends, Kitesurf, optional low-risk healing, private runners and isolated per-run environments

Useful existing checkpoint playback may remain; a rich replay redesign, advanced analytics, complex scheduling, a broad performance service and full hosted feature parity are not V1 release gates. Required evidence integrity, access control and recovery are not reduced with UI scope.

10. Release acceptance gates ​

Before calling the agreed hosted release ready:

  • [ ] Caliber sign-in, project access and artifact authorization work end to end.
  • [ ] Unauthorized identities cannot access another project's jobs or artifacts.
  • [ ] The candidate build and test-pack identities are recorded accurately.
  • [ ] Deployed service identities are verified; build drift during a run blocks a coherent qualification claim.
  • [ ] Intent, implementation, configuration, context, policy, dataset and runner/provider identities are retained with the verification snapshot.
  • [ ] API execution and the selected SSO/eLog browser journeys pass in Cloudflare.
  • [ ] The three-test vertical slice retains complete provenance and evidence; induced failures and interruptions cannot produce a qualified outcome.
  • [ ] Evidence names the actual execution surface and artifact tested; required pixel capture and checkpoint correlation work for the selected provider.
  • [ ] Only verified suites are enabled; unsupported capabilities explain why.
  • [ ] Mandatory requirement/checkpoint/evidence gaps block qualification; smoke success is not presented as whole-release approval.
  • [ ] Qualified-with-exceptions and blocked outcomes remain distinct, with explicit waivers and no inferred human release authorization.
  • [ ] Shared-environment locking coordinates runs, AI trials, deployments, resets and fixture preparation.
  • [ ] Runs continue independently of the browser session.
  • [ ] Cancellation and runner interruption produce accurate durable outcomes.
  • [ ] Recovery does not silently duplicate application mutations or overwrite attempts.
  • [ ] Reruns preserve prior failures and identify the exact compatible run set supporting the current result.
  • [ ] Publication failure remains visible and retryable without rerunning tests.
  • [ ] Reports and evidence remain retrievable after API and runner restart.
  • [ ] Database and object-storage restore checks recover a complete sample run.
  • [ ] The initial retention policy is explicit and does not silently expire qualification evidence; unsupported legal-hold administration is not advertised.
  • [ ] Hosted drafts and approved definitions survive container replacement.
  • [ ] AI provider calls work with the configured live model and server-side credentials.
  • [ ] Draft validation, review and versioned promotion work end to end.
  • [ ] Business-readable authoring uses the existing declarative validators; changing an invariant requires review and a visible new version.
  • [ ] Missing/conflicting product knowledge returns needs-clarification rather than invented requirements, selectors or assertions.
  • [ ] Monitoring distinguishes API, worker, target-environment and storage failures.
  • [ ] Failure categories include unknown and preserve supporting evidence; AI diagnoses are attributed proposals and never override gate outcomes.
  • [ ] Concurrency, execution duration, artifact size and AI usage are bounded.
  • [ ] Relevant analysis is clean: no errors, warnings or infos.
  • [ ] Required tests and deployed acceptance checks pass, with evidence retained.

The trial-and-repair release additionally requires the authoring-agent evaluation set to pass. Full feature parity, performance capacity and additional product suites must be qualified before being advertised.

Any adopted alternative browser backend must separately pass the evaluation in §2 and have its architecture/contract changes recorded. Merely enabling a provider enum, uploading a screenshot or observing a successful login is not qualification of that backend.

11. Explicitly deferred scope ​

  • Fresh application/database provisioning for every run.
  • Migration from existing Supabase persistence to D1/R2/another job system.
  • Moving Caliber application hosting merely to launch Attest.
  • Private-network/customer-hosted runners and expanded browser/device matrices.
  • Automatic post-deployment triggers before the manual path is reliable.
  • Arbitrary customer-uploaded executable test code.
  • Unreviewed AI promotion or modification of approved assertions.
  • General-purpose testing SaaS onboarding beyond the supported Caliber products.
  • Advanced analytics, complex distributed scheduling and broad performance platform capabilities beyond selected, separately qualified suites.
  • Automated change-impact selection and exploratory test discovery in V1.
  • Autonomous healing. Any later low-risk smoke/regression experiment requires an explicit policy, visible revisions, retained evidence and unchanged intent.
  • Kitesurf integration. Current documented WebGL and long authenticated-session limitations make it unsuitable for initial Caliber journeys; reconsider only for a bounded supporting experiment after actual compatibility is demonstrated.

See Kitesurf's current limitations.

The existing target model also includes deeper event correlation, multi-segment recording, rich timeline replay, resumable large uploads and governed archival. These remain tracked capabilities, not implied first-release promises.

12. Verification record and references ​

Caliber consumer setup ​

Caliber-specific provisioning commands, endpoint variables and worker launch instructions belong to the consumer's Attest integration guide. The shared cloud host lives in CDX at packages/attest/runtime_supabase/bin/attest_cloud_host.dart. Its deployment remains subject to this plan's acceptance gates.

Inspection evidence ​

During the planning inspection, these existing tests were run from the shared CDX checkout and all 23 tests passed:

sh
dart test \
  packages/attest/engine/test/attest_ai_authoring_test.dart \
  packages/attest/cli/test/ai_authoring_route_test.dart \
  packages/attest/engine/test/remote_harness_http_server_test.dart \
  packages/attest/engine/test/durable_attest_orchestrator_test.dart \
  packages/attest/engine/test/remote_harness_execution_provider_test.dart

This was focused local verification with mocked model responses, not production certification. Cloudflare deployment, live model calls, browser execution, full analysis and the complete test suite were not verified in that inspection.

Validated refinement record — 2026-10-07 ​

The approved review refined user terminology, scoped qualification outcomes, intent/requirement lineage, failure classification and V1 scope. It preserved Cloudflare hosting, the existing durable job authority, dedicated QA, reviewed AI authoring and configuration-driven DocsIQ onboarding. Alternative browser tooling remains an experiment, not a selected replacement.

During that review, 20 coverage and qualification-gate tests passed:

sh
dart test \
  packages/attest/engine/test/coverage_test.dart \
  packages/attest/engine/test/qualification_gate_evaluator_test.dart

The coverage PDF test emitted Helvetica Unicode-support warnings. These results do not establish font correctness, clean full analysis or cloud acceptance. No live provider, deployed browser or customer release-approval workflow was validated during this review.

Current external documentation was checked for browser feasibility and AI workflow comparisons. These describe capabilities, not measured Caliber results:

Requirement traceability and evidence preservation support the existing product quality needs; neither automated gates nor AI output alone establishes regulatory compliance. Caliber must confirm the applicable scope and human acceptance process. Reference reviewed: EU GMP Annex 11.

Caliber consumer references (paths remain owned by flutter_elog):

Shared CDX sources, relative to this repository root:

  • packages/attest/README.md
  • packages/attest/docs/ATTEST_WORLD_MODEL.md — normative platform contracts; particularly THS-OWN-1/5, THS-RUN-1/6, THS-CLD-7, THS-SEC-1 and THS-REP-1.
  • packages/attest/engine/lib/src/attest_ai_authoring.dart
  • packages/attest/editor/lib/src/ai_authoring_workspace.dart
  • packages/attest/engine/lib/src/remote_harness_http_server.dart
  • packages/attest/runtime_supabase/bin/attest_cloud_host.dart
  • packages/attest/cloudflare_local/README.md — local rehearsal only.

Local container milestone — 2026-10-18 ​

The Caliber local proof exercised Supabase sign-in, saved suite revisions, separate API/UI workers, an API integration sequence, an independent-reviewer workflow and the eLog sign-in-screen UI smoke with screenshot and WebM evidence. Saved report template versions produced PDF/DOCX/ZIP with private downloads, generating-user attribution and evidence retained after container recreation. One live Gemini API draft was reviewed, saved and passed on the Docker worker; AI-generated UI journeys were not accepted in that check.

On 2026-10-19 both phase-2 branches incorporated main. The shared post-sync suite passed 293 engine/types/storage/CLI tests (two optional k6 tests skipped) and 25 editor tests, with clean Attest analysis. These source checks do not replace a fresh image build and Docker acceptance pass for the merged revision. Existing services were left stopped and their data preserved.

Cloud deployment, employee federation, authenticated product browser journeys, environment locks, interruption recovery, backup restoration and measured multi-user capacity remain release gates. See the consumer's local trial record and rollout checklist.

Documentation ownership consolidation — 2026-10-07 ​

The approved plan moved to docs/attest/plan.md in CDX. Older active product plans were consolidated here; their superseded milestone questions and broad farm-first sequence are not current commitments. Existing implementation and verification evidence is retained in historical delivery evidence. Historical framework migration records and handoffs live under history/ and must not be interpreted as current execution instructions or release claims. Caliber provisioning stays in the consumer integration guide.