Six behavioural segments recovered from an unlabelled subscriber base, with a supervised
churn model layered on top to quantify the revenue exposure attached to each. The operating
point in section 04 is adjustable: precision and recall trade against each other, and the
right cut depends on retention budget rather than on any single metric.
00
Executive summary
Where the revenue exposure sits, and what to do about it
01
Data lineage
Full reconciliation from source to model input
Outlier fences use Tukey k=3.0 rather than the default 1.5. On a right-skewed
telecom usage distribution the 1.5×IQR fence removes 2–3% of records, most of which
are genuine heavy-data subscribers and commercially the most interesting. The wider
k=3.0 fence isolates corporate and IoT SIMs specifically. Affected records are
winsorised rather than dropped: they represent revenue-generating subscribers and
belong in the segment counts, though not at their raw magnitude.
02
Choosing k
Elbow and silhouette both say 5. Business says 6.
Inertia (WCSS)Silhouette
Six segments is a business constraint, not a statistical result.
The kneedle elbow and the silhouette peak both indicate k=5. We ship
k=6 to match the six distinct offers the campaign supports. The cost of
that decision is quantified rather than hidden: silhouette falls from 0.4465 to
0.4156. We judged the additional actionable segment worth a 0.03 separation penalty,
and we would revisit the choice if the offer set changed.
03
The six segments
Named by revenue rank for reproducibility
Segment names are derived from revenue rank, not assigned to cluster ids.
Mixture-model cluster ids are arbitrary and permute between runs, random seeds and
library versions, so a fixed id-to-name mapping would relabel the base whenever any of
those changed. Ranking clusters by mean revenue gives a stable, ordered business property
to attach names to, which keeps segment definitions reproducible across refreshes.
04
Operating point
Precision and recall trade against each other
Selecting the classification threshold
Held-out test set: subscribers. Each count is computed
directly from the model's predicted probabilities at every one of 99 candidate
thresholds; no values are interpolated.
0.50
0.01 · flag all0.500.99 · flag none
PREDICT STAY
PREDICT CHURN
ACTUAL STAY
—TRUE NEG
—FALSE POS
ACTUAL CHURN
—FALSE NEG
—TRUE POS
Customers contacted—
Churners identified—
Churners not identified—
Revenue reachable—
Spent on non-churners—
Precision
—
share of flagged who do churn
Recall
—
share of churners identified
F1
—
harmonic mean of the two
Accuracy
—
see note on class balance
Specificity
—
share of stayers not contacted
ROC-AUC
—
threshold-independent
ROC curvecurrent operating point
Why accuracy is not the headline metric. The base churn rate is
, so a model predicting that nobody churns would score
accuracy while identifying no churners and protecting no
revenue. Moving the dial to 0.99 reproduces exactly that. Precision, recall and PR-AUC
() remain informative under this class balance, which is why
they lead the reporting.
05
Model selection
Chosen on ROC-AUC, validated on 5-fold CV
Churn drivers
06
Segment surrogate
Deployment fidelity to the mixture model
This figure measures fidelity, not correctness. It reports how closely a fast
random forest reproduces the mixture model's own decision boundary, which is what
allows new subscribers to be scored in milliseconds without refitting the GMM. Its
reference labels come from the GMM, so it should be read as a deployment-fidelity
check. Segmentation is unsupervised and therefore has no accuracy in the supervised
sense; separation quality is reported in section 02 instead.
07
Control group integrity
The control group is randomised, then verified. A universal control group only
supports a causal read of campaign lift if membership is independent of the outcome.
Selecting the holdout on any quantity correlated with revenue or engagement — cluster
confidence, spend, tenure — produces a control that is not comparable to the treated
population, and biases every incrementality estimate computed against it.
Here the 3% holdout is drawn at random within each segment, so segment mix is
preserved while assignment inside a segment stays independent of behaviour. Balance is
then confirmed with standardised mean differences across all covariates; conventional
practice treats |SMD| < 0.10 as balanced, and the largest observed value is 0.0216.
08
Sample predictions
Ten unseen subscribers, full scoring path
Every figure on this page is computed from a full pipeline run and read from
outputs/dashboard_data.json. Reproduce with
stage_a_segment.py → stage_b_predict.py →
predict_samples.py → build_dashboard_data.py.
Methodology and assumptions are documented in METHODOLOGY.md.