PRD Pass CUPED + Guardrails Decision Audit Streaming Provisional Batch Authoritative

MetaSignal Evidence Dashboard

Production-simulated experimentation, metrics intelligence, and streaming observability platform. This dashboard turns the repo artifacts into a visual cockpit: metric governance, denominator conflicts, data quality gates, deterministic assignment, SRM, CUPED, guardrail-first decisions, anomaly detection, operational history, and streaming reconciliation.

Truth boundary: no production deployment, no real company users, no real production traffic, no Kafka/Flink production infrastructure, and no real A/B treatment effect from RetailRocket.

PRD Status

pass

core + streaming evidence bundle

Artifacts

JSON evidence / validation / reports / streaming

RetailRocket Rows

2756101

public e-commerce event substrate

Assignments

20000

treatment share 0.5047

SRM p-value

0.1837

no sample-ratio mismatch detected

A/A FP Rate

0.055

CUPED false-positive control

CUPED VR

26.46%

synthetic A/A variance reduction

Golden Suite

12/12

experiment decision scenarios

Streaming Checks

43/43

streaming PRD validation

Guardrail Decision

HOLD

primary lift blocked by guardrail

Operational History

60 days

13 scripted scenarios

Anomaly Backtest

P 1 / R 0.8667

synthetic labeled events

System Flow

1. Metric RegistryVersioned numerator, denominator, grain, owner, config hash.

2. Data Quality GateBlocking checks run before metric compute.

3. Assignment + SRMDeterministic SHA-256 assignment and chi-square SRM.

4. CUPED ReadoutVariance-reduced experiment evaluation with A/A validation.

5. Guardrail GateGuardrails evaluated before ship decision.

6. Audit LogDecision record with override enforcement.

7. Streaming Early WarningSRM, instrumentation, lag, DLQ, provisional anomalies.

8. Batch ReconciliationStreaming remains provisional; batch is authoritative.

Evidence Summary

Capability	Evidence
Metric governance	Versioned registry with explicit denominator logic and config hashes.
Conflict detection	1 denominator / metric definition conflict(s) captured in evidence artifacts.
Experiment validity	Assignment balance, SRM check, CUPED readout, A/A validation, and edge-case validation.
Decision quality	Guardrail-first HOLD behavior and override-reason enforcement.
Operational realism	60-day simulated history with 13 scripted failure scenarios.
Streaming boundary	Streaming alerts are provisional; reconciliation keeps batch authoritative. Status: minor_delta.

Artifact group	Count
evidence	21
validation	9
reports	8
streaming	14
Total	52

Key Artifact Map

Artifact	Path
PRD completion	`outputs/reports/metasignal_prd_completion_report_v1.json`
CUPED readout	`outputs/evidence/cuped_experiment_readout.json`
CUPED A/A validation	`outputs/validation/cuped_aa_validation_report.json`
CUPED edge cases	`outputs/validation/cuped_edge_case_validation_report.json`
Guardrail decision	`outputs/evidence/guardrail_decision_report.json`
SRM check	`outputs/evidence/srm_check_report.json`
Operational history	`outputs/evidence/operational_history_60_day_report.json`
Golden scenarios	`outputs/evidence/golden_scenario_suite_v1_report.json`
Streaming validation	`outputs/validation/streaming_prd_v1_validation_report.json`
Stream-batch reconciliation	`outputs/streaming/stream_batch_reconciliation_report.json`
Defense / PRD PDFs	`docs/prd/`

Claim Boundary

MetaSignal is a solo-built, non-production, production-simulated project. It demonstrates executable repo evidence, generated artifacts, validation scripts, API smoke tests, and streaming simulation — not live production usage.

The safest interview framing is: real code, real validation artifacts, real public event dataset where applicable, simulated operational failures, simulated treatment effects, and no production claims.

Run Locally

PYTHONPATH=. python3 scripts/run_metasignal_prd_complete_v1.py
PYTHONPATH=. python3 scripts/show_streaming_demo.py
PYTHONPATH=. python3 scripts/validate_streaming_prd_v1.py