CATALOG PILOT / EVALUATION EVIDENCE
Test the contract
before trusting the claim.
The public example proves one report shape. A credible design-partner evaluation must test unpublished cases locally, fix its labels and thresholds before execution, and keep every artifact outside Andromalius.
01 / THREE PROOF LAYERS
Do not confuse a preview with generalization.
Each layer answers a different question. None creates an accuracy, customer, demand, certification, or delivery claim by itself.
- 01Fixed public preview
Shows the stable report shape, finding code, severity, JSONPath, observations, and score-policy fields without a wallet. It proves one documented synthetic output—not general accuracy.
- 02Non-preview conformance corpus
Exercises six materially different synthetic challenge and discovery cases with nine expected findings across nine stable codes. Exact outputs, registry versions, and zero network calls must remain deterministic.
- 03Customer-controlled blind evaluation
Lets an evaluator seal one through 100 unpublished labeled cases and predeclare thresholds before running the checksum-pinned CLI locally. Cases, inputs, labels, and detailed results stay in the evaluator's environment.
02 / CURRENT SYNTHETIC EVIDENCE
Six non-preview cases. Zero network calls.
The private regression covers resource binding, timeout drift, hostile fallback options, duplicate handling, address normalization, and cross-operation discovery drift. It binds report, finding-code registry, and score-policy versions.
- Six bounded synthetic cases: four challenge audits and two discovery audits.
- Nine expected findings across nine stable finding codes.
- Finding-code registry 1.0.0 and score-policy version 1.0.0.
- Exact deterministic output identities and a fail-on-use network sentinel.
- Explicitly not qualified as real-world accuracy or customer evidence.
03 / BLIND EVALUATION
Evaluator-controlled cases. Predeclared acceptance.
The stronger test happens in the evaluator's environment after the exact private artifact, order, payment, checksum, recipient, and delivery gates are complete.
- 01Create and label locally
The evaluator prepares bounded challenge or discovery cases and exact expected finding tuples under registry version 1.0.0. Andromalius does not receive the corpus.
- 02Seal before execution
A canonical SHA-256 corpus identity, case count, contract versions, and exact acceptance thresholds are fixed in a separate protocol before evaluation.
- 03Run without a network
The checksum-pinned customer CLI evaluates only explicitly named local JSON. Network use fails closed; there is no upload, telemetry, wallet, signing, payment, persistence, or automatic update path.
- 04Review aggregate evidence
The report exposes exact-match, precision, recall, F1, blocking accuracy, false-block, and missed-block rates plus protocol, corpus, and result-set digests—without emitting cases or per-case findings.
04 / CLAIM BOUNDARY
Passing one sealed corpus means exactly that.
The evaluation is designed to make useful evidence possible without turning a synthetic result into a broad marketing claim.
- A passing synthetic regression is engineering evidence, not a customer benchmark or real-world accuracy estimate.
- A matching digest proves which local corpus was evaluated; it does not prove who labeled it or whether it represents a live catalog.
- No result is a certification, endorsement, exhaustive security review, or guarantee that every catalog issue will be detected.
- Publishing an evaluator's name, feedback, or even aggregate customer results requires separate written approval.
05 / COMMERCIAL STOP
Evaluation interest is not an order.
There is no public download or self-serve checkout. A customer-local evaluation requires professional review, a written customer-specific order, reconciled payment, an exact checksum-pinned artifact, and a separately approved private recipient and delivery channel. Do not send an artifact with the initial inquiry.
Contacting us does not create an order, payment authorization, entitlement, software delivery, benchmark claim, or service-level commitment. Terms · Privacy · Acceptable use · Refunds