Decision Framework

Every experiment outcome maps to a clear action. The framework evaluates primary metric performance against guardrail metric health.

Primary Metric	Guardrail Metrics	Decision	Action
Statistically significant win	No degradation	SHIP	Roll out to 100%. Document learnings for the team.
Statistically significant win	One or more degraded	DO NOT SHIP	Investigate root cause. Iterate on the variant design.
No significant difference	No degradation	ITERATE	If implementation cost is low, ship. If high, iterate.
No significant difference	Degraded	KILL	Roll back. Document why the variant failed.
Statistically significant loss	Any	KILL	Roll back immediately. Document learnings.

productquant.dev