Case Study — B2B SaaS Churn Prediction

Churn Flagged 45 Days Before Cancellation, $380K Annual Revenue Protected.

B2B CRM and sales engagement platform — ~$4M ARR, ~80–120 employees, Series B. The VP of Customer Success knew churn was too high. What they didn't know: the data to predict it was already there — it just wasn't being used.

Stack Python SageMaker PostHog AWS scikit-learn
18%9%
Monthly churn rate reduction
45
Days early warning before cancellation
$380K
Annual revenue protected
87%
Model precision and recall
3
Automated intervention playbooks

Context.

Company Profile
  • B2B CRM and sales engagement platform for mid-market teams
  • ~$4M ARR, ~80–120 employees, Series B stage
  • Tiered pricing: Starter · Growth · Enterprise
  • Stack: PostHog (self-hosted), Python, AWS, SageMaker, PostgreSQL
  • Product usage data collected but never aggregated into health scores
Team Composition
  • VP of Customer Success overseeing retention strategy
  • Support team of 12 handling reactive inbound requests
  • Data engineering team of 3 with limited ML experience
  • 1 fractional data scientist engaged for model development

Before ProductQuant.

The VP of Customer Success knew churn was a problem — roughly 18% of revenue churned every month. But the team had no way to see it coming. Cancellation was discovered when the cancellation email arrived. By then, the decision was already made.

What they didn't know: the product was generating usage data on every login, every feature interaction, every workflow run. That data contained clear leading indicators of churn — declining login frequency, skipped workflow steps, shrinking team size — but nobody had connected it to retention risk. The signal was there. The pipeline to interpret it was missing.

Worse: the support team spent 60% of their time on reactive churn management — scrambling to save accounts that were already in the cancellation queue. Without a risk scoring system, they couldn't prioritize. $2.1M in annual recurring revenue was walking out the door every year, and nobody knew which accounts to call next.

The Problem
  • 18% monthly churn rate with no early warning system in place
  • Support team 100% reactive — cancellation email was the first signal
  • $2.1M annual revenue at risk with no retention playbook
  • Account health scoring done manually in spreadsheets — weeks out of date
  • No connection between product usage data and churn risk

What they tried before us.

Attempt 1 — Support ticket analysis

The team analyzed support ticket volume by account, assuming that more tickets correlated with higher churn risk. They built a dashboard showing ticket counts per customer.

Outcome: No predictive signal. High-ticket accounts were often the most engaged. Cancelling accounts showed no ticket spike before the event. By the time tickets increased, the account was already gone.
Attempt 2 — NPS surveys

Quarterly NPS surveys were deployed to measure customer sentiment. The CS team tracked score changes across accounts and attempted outreach based on NPS drops.

Outcome: NPS was a lagging indicator, not a leading one. Scores dropped after the churn decision, not before. Response rates were low, and the 90-day survey cycle was too slow for timely intervention.
Attempt 3 — Manual account reviews

The VP of CS assigned team members to manually review the top 50 accounts each week, scoring health on a 1–5 scale in a shared spreadsheet.

Outcome: Didn't scale. Reviews covered less than 5% of the active account base. Scores were subjective, inconsistent between reviewers, and always stale by the time they were recorded. The data was always at least one week behind reality.

Why it didn't work: All three approaches used the wrong data on the wrong time horizon. Support tickets and NPS are reactive signals — they arrive after the decision to cancel. Manual reviews can't scale beyond a tiny fraction of accounts. The product was generating real leading indicators every day, but nobody was listening to them.

The diagnosis.

Working through their data, the real problem was not the obvious guess. The VP of Customer Success assumed they needed a better CRM workflow. The data told a different story.

Finding 1 — No leading indicator from product usage

PostHog was tracking millions of events per month — page views, feature interactions, workflow executions, API calls — but none of these events were aggregated into a health score or churn risk signal. The data was being collected and stored, but it was never analyzed for patterns that precede cancellation. The team had a rich dataset and no way to extract signal from it.

Finding 2 — Cancellation data not tied to behavioral triggers

A historical analysis of 24 months of churned accounts revealed clear behavioral patterns: declining login frequency starting 45+ days before cancellation, shrinking active seat count, and reduced feature breadth. These patterns were consistent across 80% of churned accounts — and nobody had ever looked for them. The cancellation data lived in Stripe. The behavioral data lived in PostHog. They had never been connected.

Finding 3 — Account health scoring was entirely manual

The CS team's weekly spreadsheet reviews covered fewer than 50 accounts out of 800+ active accounts. The scoring criteria were subjective: there was no repeatable definition of a healthy account. Two CS reps scoring the same account would regularly assign different scores. And by the time a score was entered, the usage data it was based on was already 5–10 days stale.

The fix.

A four-phase intervention that transformed raw usage data into a daily churn prediction pipeline with automated retention playbooks.

Fix 1 — Usage Event Audit
Every tracked event in PostHog audited against churn outcomes. Identified the 12 behavioral signals that most strongly correlated with retention: login frequency, workflow completion rate, team member additions, feature breadth, API call volume, and seven others. Each signal weighted by predictive strength against 24 months of historical cancellation data.
Fix 2 — Churn Prediction Model
Binary classifier trained on 24 months of labeled usage data (churned vs. retained accounts) using Python, scikit-learn, and SageMaker. Random forest classifier achieved 87% precision and recall on the holdout set. Model output: a daily churn probability score from 0–100 for every active account, computed each night via SageMaker batch inference.
Fix 3 — Automated Risk Scoring in PostHog
Daily risk scores pushed to PostHog as person properties on each account. Accounts automatically tagged into three cohorts: low risk (0–30), medium risk (31–70), high risk (71–100). CS team received a daily Slack digest with the top 20 highest-risk accounts and the specific behavioral triggers that drove the score.
Fix 4 — Retention Playbook with 3 Intervention Tiers
Three-tier automated intervention system: Low risk — automated educational email sequence triggered when usage declines, no human touch. Medium risk — CS outreach with personalized usage report and health check call booked within 48 hours. High risk — executive escalation with VP of CS or founder outreach, account-level remediation plan, and pricing flexibility if needed.

Three-Tier Intervention Playbook

Low Risk
Score 0–30
  • Intervention type Automated
  • Trigger Usage decline >7 days
  • Action Email sequence
  • Human touch None
Medium Risk
Score 31–70
  • Intervention type CS outreach
  • Trigger Score increase >20 pts
  • Action Health check call
  • Human touch CS team lead
High Risk
Score 71–100
  • Intervention type Executive escalation
  • Trigger Score >70 + declining LR
  • Action VP CS outreach
  • Human touch Founder / VP CS

The result.

Before vs After metrics with quantified revenue impact.

18%9%
Monthly churn rate reduction — 50% relative decrease within 90 days of prediction model deployment
45
Days of early warning before cancellation — enough time for meaningful intervention at every risk tier
$380K
Annual revenue protected — accounts flagged and successfully retained through the intervention playbook
87%
Model accuracy — precision and recall measured on holdout test set across 24 months of historical data
3
Automated intervention playbooks — low, medium, and high risk tiers with specific triggers and actions
24
Months of historical usage data retroactively scored — churn patterns discovered that had been invisible for 2 years

We were tracking millions of data points every month and using none of them to predict churn. The model showed us which usage patterns led to cancellation 45 days before it happened. That window changed everything about how we allocate CS resources.

— VP of Customer Success, B2B CRM platform
Key Lesson

The most valuable churn data is already in your product. You just aren't reading it. This team had millions of PostHog events, a Stripe subscription history with clear churn labels, and two years of data — but the signals were never connected. 12 behavioral features predicted churn with 87% accuracy. All of them came from product usage data they were already collecting. The tooling was not the gap. The pipeline from data to decision was.

What you can do now.

See churn risk scores for every account, updated daily

No more manual spreadsheets or subjective scoring. Every account gets a data-driven risk score based on real usage patterns. The top 20 highest-risk accounts arrive in Slack every morning.

Know which behavioral signals predict churn for YOUR product

Generic churn indicators from blog posts won't match your product. We find the specific usage patterns that lead to cancellation in your dataset — because every product has different churn signals.

Deploy a $380K retention pipeline with three tiers of automated intervention

Low-risk accounts get automated education. Medium-risk accounts get CS outreach. High-risk accounts get executive attention. Every tier has playbooks, triggers, and timelines.

Jake McMahon
Jake McMahon
ProductQuant

10 years building analytics and growth systems for B2B SaaS at $1M–$50M ARR. BSc Behavioural Psychology, MSc Data Science. The most common analytics gap isn't bad data — it's missing data. Events never instrumented, properties never attached, funnels never connected. Finding what's absent is usually more valuable than analysing what's present.

What this looks like for your company

Analytics Audit.

A structured review of your product usage data, churn signals, and retention infrastructure — finding what's predictive, sizing the revenue at risk, and delivering a model that flags risk weeks before cancellation.

  • Usage event audit: every tracked event reviewed against churn outcomes to find predictive signals
  • Churn prediction model: binary classifier trained on your historical data with daily inference pipeline
  • Risk scoring integration: scores pushed to your analytics platform (PostHog, Mixpanel, Amplitude) as person properties
  • Three-tier retention playbook: automated interventions for low, medium, and high risk accounts with Slack alerts
  • Revenue impact model: dollar value of retained revenue measured against model cost and CS time allocation
$3,497 · 10 days
Right for you if
  • Monthly churn rate higher than your target with no early warning system in place
  • Product usage data being collected but never aggregated into health scores or risk signals
  • CS team spending most of their time on reactive account saves rather than proactive retention

See how it works for your company.

A 15-minute call is enough to know whether what we do is relevant to where you are. No pitch. Just a conversation about your specific situation.