🏆 Major Project – Churn Prediction · दिन 57

दिन 57: Project 2: EDA और Insights

⏱ 3 hours📶 Major project · Part 2 of 5🎯 100% self-study

दिन 57 / 60

✅ आज आप सीखेंगे

हर ज़रूरी variable पर churn की profile
Interactions ढूँढें (contract × support, tenure × contract)
कौन से अंतर असली हैं, test करें (chi-square, t-test)
Effect size से drivers rank करें
Retention team के काम आने वाली पाँच insights लिखें

🏢 असली दुनिया में

कोई भी model बनाने से पहले Head of Retention साधारण भाषा में जवाब चाहते हैं: “कौन से customers जाते हैं? कीमत है, service है या contract? Model के बिना भी कल क्या बदलें?” अच्छा EDA अक्सर पूरे project की आधी business value दे देता है।

हर category में churn rate

Python

import numpy as np
import pandas as pd

def load_churn(path="telecom_churn.csv"):
    # "None" means no internet plan – stop pandas from turning it into NaN
    df = pd.read_csv(path, keep_default_na=False, na_values=[""])
    df["total_charges"] = df["total_charges"].fillna(df["monthly_charges"] * df["tenure_months"])
    df["churned"] = (df["churn"] == "Yes").astype(int)
    return df

df = load_churn()
rows = []
for col in ["contract", "internet_service", "tech_support", "payment_method", "senior_citizen", "gender"]:
    g = df.groupby(col)["churned"].agg(["mean", "size"])
    for k, r in g.iterrows():
        rows.append((col, str(k), int(r["size"]), round(r["mean"] * 100, 1)))
tab = pd.DataFrame(rows, columns=["variable", "value", "customers", "churn_%"])
print(tab.to_string(index=False))

variable value customers churn_% contract Monthly 1643 43.9 contract One year 757 13.3 contract Two year 600 8.7 internet_service DSL 1048 24.0 internet_service Fiber 1327 40.4 internet_service None 625 13.8 tech_support No 1548 37.3 tech_support No internet 625 13.8 tech_support Yes 827 25.4 payment_method Bank Transfer 608 27.8 payment_method Cash 724 32.6 payment_method Credit Card 757 27.1 payment_method UPI AutoPay 911 29.0 senior_citizen 0 2546 28.2 senior_citizen 1 454 34.4 gender Female 1480 28.9 gender Male 1520 29.3

Python

import numpy as np
import pandas as pd

def load_churn(path="telecom_churn.csv"):
    # "None" means no internet plan – stop pandas from turning it into NaN
    df = pd.read_csv(path, keep_default_na=False, na_values=[""])
    df["total_charges"] = df["total_charges"].fillna(df["monthly_charges"] * df["tenure_months"])
    df["churned"] = (df["churn"] == "Yes").astype(int)
    return df

import matplotlib.pyplot as plt
df = load_churn()
cols = ["contract", "internet_service", "tech_support", "payment_method"]
fig, ax = plt.subplots(1, 4, figsize=(14, 3.6), sharey=True)
avg = df["churned"].mean() * 100
for a, c in zip(ax, cols):
    r = (df.groupby(c)["churned"].mean() * 100).sort_values(ascending=False)
    a.bar(r.index, r.values, color=["#dc2626" if v > avg + 5 else "#94a3b8" for v in r.values])
    a.axhline(avg, color="#111827", ls="--", lw=1); a.set_title(c); a.tick_params(axis="x", rotation=25)
ax[0].set_ylabel("churn %"); fig.suptitle("Red = well above the 29% average", y=1.02)
plt.tight_layout(); plt.show()

Contract, internet service, tech support और payment method के churn rate bars — Monthly contracts, Fiber और बिना tech support अलग दिखते हैं; payment method के अंतर छोटे हैं।

Tenure, charges और complaints

Python

import numpy as np
import pandas as pd

def load_churn(path="telecom_churn.csv"):
    # "None" means no internet plan – stop pandas from turning it into NaN
    df = pd.read_csv(path, keep_default_na=False, na_values=[""])
    df["total_charges"] = df["total_charges"].fillna(df["monthly_charges"] * df["tenure_months"])
    df["churned"] = (df["churn"] == "Yes").astype(int)
    return df

import matplotlib.pyplot as plt
df = load_churn()
fig, ax = plt.subplots(1, 3, figsize=(14, 3.8))
tb = pd.cut(df["tenure_months"], [0, 6, 12, 24, 48, 72], labels=["0-6", "7-12", "13-24", "25-48", "49-72"])
(df.groupby(tb, observed=True)["churned"].mean() * 100).plot(ax=ax[0], marker="o", color="#4f46e5"); ax[0].set_title("Churn % by tenure (months)")
mb = pd.qcut(df["monthly_charges"], 5)
(df.groupby(mb, observed=True)["churned"].mean() * 100).plot(kind="bar", ax=ax[1], color="#f59e0b", rot=20); ax[1].set_title("Churn % by monthly bill (quintiles)")
cp = df["complaints"].clip(upper=4).map(lambda c: "4+" if c == 4 else str(c))
(df.groupby(cp)["churned"].mean() * 100).plot(kind="bar", ax=ax[2], color="#dc2626", rot=0); ax[2].set_title("Churn % by complaints")
for a in ax: a.set_xlabel(""); a.set_ylabel("churn %")
plt.tight_layout(); plt.show()

Tenure band, monthly bill quintile और complaints के हिसाब से churn rate — नए customers, ऊँचे bills और हर extra complaint churn बढ़ाते हैं।

Python

import numpy as np
import pandas as pd

def load_churn(path="telecom_churn.csv"):
    # "None" means no internet plan – stop pandas from turning it into NaN
    df = pd.read_csv(path, keep_default_na=False, na_values=[""])
    df["total_charges"] = df["total_charges"].fillna(df["monthly_charges"] * df["tenure_months"])
    df["churned"] = (df["churn"] == "Yes").astype(int)
    return df

df = load_churn()
num = ["tenure_months", "monthly_charges", "total_charges", "complaints", "data_usage_gb"]
print(df.groupby("churn")[num].median().round(1).T)

churn No Yes tenure_months 41.0 26.0 monthly_charges 640.0 790.0 total_charges 20881.5 17811.0 complaints 1.0 1.0 data_usage_gb 53.6 59.8

🧠 total_charges “बचाने वाला” क्यों दिखता है

Churners के total charges कम हैं – इसलिए नहीं कि ज़्यादा भुगतान लोगों को रोकता है, बल्कि क्योंकि total ≈ monthly × tenure, और churners का tenure छोटा है। कारण पढ़ने से पहले हमेशा पूछें कि column असल में क्या मापता है।

Interactions: risk कहाँ केंद्रित है

Python

import numpy as np
import pandas as pd

def load_churn(path="telecom_churn.csv"):
    # "None" means no internet plan – stop pandas from turning it into NaN
    df = pd.read_csv(path, keep_default_na=False, na_values=[""])
    df["total_charges"] = df["total_charges"].fillna(df["monthly_charges"] * df["tenure_months"])
    df["churned"] = (df["churn"] == "Yes").astype(int)
    return df

import seaborn as sns
import matplotlib.pyplot as plt
df = load_churn()
df["tenure_band"] = pd.cut(df["tenure_months"], [0, 6, 12, 24, 48, 72], labels=["0-6", "7-12", "13-24", "25-48", "49-72"])
fig, ax = plt.subplots(1, 2, figsize=(13, 4))
p1 = df.pivot_table(index="contract", columns="tech_support", values="churned", aggfunc="mean") * 100
sns.heatmap(p1, annot=True, fmt=".0f", cmap="Reds", ax=ax[0], cbar=False); ax[0].set_title("Churn %: contract × tech support")
p2 = df.pivot_table(index="contract", columns="tenure_band", values="churned", aggfunc="mean", observed=True) * 100
sns.heatmap(p2, annot=True, fmt=".0f", cmap="Reds", ax=ax[1], cbar=False); ax[1].set_title("Churn %: contract × tenure")
plt.tight_layout(); plt.show()

Contract × tech support और contract × tenure के churn heatmaps — Monthly + बिना support, और monthly + पहले 6 महीने – सबसे गर्म cells।

Python

import numpy as np
import pandas as pd

def load_churn(path="telecom_churn.csv"):
    # "None" means no internet plan – stop pandas from turning it into NaN
    df = pd.read_csv(path, keep_default_na=False, na_values=[""])
    df["total_charges"] = df["total_charges"].fillna(df["monthly_charges"] * df["tenure_months"])
    df["churned"] = (df["churn"] == "Yes").astype(int)
    return df

df = load_churn()
seg = df.assign(new=df["tenure_months"] <= 12)
hot = seg[(seg["contract"] == "Monthly") & (seg["tech_support"] == "No") & seg["new"]]
print(f"Hot segment (monthly, no support, ≤12 months): {len(hot)} customers, churn {hot['churned'].mean():.0%}")
print(f"→ {len(hot) / len(df):.0%} of customers but {hot['churned'].sum() / df['churned'].sum():.0%} of all churners")

Hot segment (monthly, no support, ≤12 months): 148 customers, churn 74% → 5% of customers but 12% of all churners

क्या अंतर असली हैं?

Python

import numpy as np
import pandas as pd

def load_churn(path="telecom_churn.csv"):
    # "None" means no internet plan – stop pandas from turning it into NaN
    df = pd.read_csv(path, keep_default_na=False, na_values=[""])
    df["total_charges"] = df["total_charges"].fillna(df["monthly_charges"] * df["tenure_months"])
    df["churned"] = (df["churn"] == "Yes").astype(int)
    return df

from scipy import stats
df = load_churn()
print("Chi-square tests (category vs churn):")
for col in ["contract", "internet_service", "tech_support", "payment_method", "senior_citizen", "gender"]:
    chi2, p, dof, _ = stats.chi2_contingency(pd.crosstab(df[col], df["churn"]))
    n = len(df); k = min(pd.crosstab(df[col], df["churn"]).shape) - 1
    v = (chi2 / (n * k)) ** 0.5                        # Cramér's V = effect size
    print(f"  {col:17s} p={p:.1e}  Cramér's V={v:.2f}  {'significant' if p < 0.05 else 'not significant'}")
print("Welch t-tests (numeric, churners vs stayers):")
for col in ["tenure_months", "monthly_charges", "complaints", "data_usage_gb"]:
    a, b = df.loc[df["churned"] == 1, col], df.loc[df["churned"] == 0, col]
    t, p = stats.ttest_ind(a, b, equal_var=False)
    d = (a.mean() - b.mean()) / ((a.var() + b.var()) / 2) ** 0.5
    print(f"  {col:17s} p={p:.1e}  Cohen's d={d:+.2f}")

Chi-square tests (category vs churn): contract p=1.3e-84 Cramér's V=0.36 significant internet_service p=8.3e-37 Cramér's V=0.24 significant tech_support p=1.9e-28 Cramér's V=0.21 significant payment_method p=9.8e-02 Cramér's V=0.05 not significant senior_citizen p=9.2e-03 Cramér's V=0.05 significant gender p=8.3e-01 Cramér's V=0.00 not significant Welch t-tests (numeric, churners vs stayers): tenure_months p=7.8e-40 Cohen's d=-0.54 monthly_charges p=2.0e-33 Cohen's d=+0.49 complaints p=8.0e-16 Cohen's d=+0.34 data_usage_gb p=7.9e-08 Cohen's d=+0.21

💡 Significance + size

5% पर gender और payment method significant नहीं – अच्छा, business को gender से target करना भी नहीं चाहिए। सबसे बड़ा असर contract का (Cramér’s V), और numeric में tenure का। p-values और effect sizes दोनों बताएँ।

Drivers की ranking

Python

import numpy as np
import pandas as pd

def load_churn(path="telecom_churn.csv"):
    # "None" means no internet plan – stop pandas from turning it into NaN
    df = pd.read_csv(path, keep_default_na=False, na_values=[""])
    df["total_charges"] = df["total_charges"].fillna(df["monthly_charges"] * df["tenure_months"])
    df["churned"] = (df["churn"] == "Yes").astype(int)
    return df

import matplotlib.pyplot as plt
df = load_churn()
base = df["churned"].mean()
lifts = {"Monthly contract": df.loc[df.contract == "Monthly", "churned"].mean() / base,
         "Tenure ≤ 6 months": df.loc[df.tenure_months <= 6, "churned"].mean() / base,
         "3+ complaints": df.loc[df.complaints >= 3, "churned"].mean() / base,
         "No tech support": df.loc[df.tech_support == "No", "churned"].mean() / base,
         "Fiber internet": df.loc[df.internet_service == "Fiber", "churned"].mean() / base,
         "Bill > ₹900": df.loc[df.monthly_charges > 900, "churned"].mean() / base,
         "Pays by cash": df.loc[df.payment_method == "Cash", "churned"].mean() / base,
         "Senior citizen": df.loc[df.senior_citizen == 1, "churned"].mean() / base,
         "Two-year contract": df.loc[df.contract == "Two year", "churned"].mean() / base}
s = pd.Series(lifts).sort_values()
plt.figure(figsize=(8.5, 4.2))
plt.barh(s.index, s.values, color=["#16a34a" if v < 1 else "#dc2626" for v in s.values]); plt.axvline(1, color="k", lw=1)
plt.title("Churn lift vs average (1.0 = average customer)"); plt.xlabel("× average churn rate")
plt.show()
print(s.round(2).sort_values(ascending=False).to_string())

3+ complaints 1.74 Tenure ≤ 6 months 1.61 Monthly contract 1.51 Bill > ₹900 1.44 Fiber internet 1.39 No tech support 1.28 Senior citizen 1.18 Pays by cash 1.12 Two-year contract 0.30

मुख्य customer segments के churn lift का bar chart — Lift = segment churn ÷ औसत churn। 1 से ऊपर = औसत से ज़्यादा risky।

Retention team के लिए पाँच insights

📅

Monthly customers को बाँधेंMonthly plans two-year से ~5× ज़्यादा churn। Discount के साथ yearly-plan upgrade दें।

🐣

पहले 6 महीने जीतेंनए customers सबसे risky – welcome calls, setup मदद और 3-महीने का check-in।

🛠️

Tech support ख़ुद का ख़र्च निकालता हैबिना support वाले बहुत ज़्यादा churn – monthly Fiber users के लिए मुफ़्त support bundle करें।

📣

हर complaint एक चेतावनीहर complaint के साथ churn बढ़ता है – दूसरी complaint के बाद callback trigger करें।

💳

मानें नहीं, test करें: cash वालेCash देने वाले थोड़ा ज़्यादा churn (33% vs ~28%), पर test significant नहीं (p ≈ 0.10)। पूरे पैमाने से पहले छोटे group पर AutoPay cashback का pilot करें।

Model से पहले भी संभव actions

🏅 आज के Job Skills

✔ कई variables पर target की profile✔ Interaction heatmaps से high-risk segments ढूँढना✔ Chi-square, t-tests और effect sizes से patterns पक्का करना✔ Lift से drivers rank करके actionable insights लिखना

Practice tasks

🏋️ Practice – ख़ुद करें

Task 1: क्या Fiber users का churn कीमत से है? उसी monthly-bill band में Fiber vs DSL customers के churn की तुलना करें।

Solution देखें

Python

import numpy as np
import pandas as pd

def load_churn(path="telecom_churn.csv"):
    # "None" means no internet plan – stop pandas from turning it into NaN
    df = pd.read_csv(path, keep_default_na=False, na_values=[""])
    df["total_charges"] = df["total_charges"].fillna(df["monthly_charges"] * df["tenure_months"])
    df["churned"] = (df["churn"] == "Yes").astype(int)
    return df

df = load_churn(); d = df[df["internet_service"] != "None"].copy()
d["bill_band"] = pd.cut(d["monthly_charges"], [0, 600, 750, 900, 2000], labels=["≤600", "600-750", "750-900", "900+"])
print((d.pivot_table(index="bill_band", columns="internet_service", values="churned", aggfunc="mean", observed=True) * 100).round(1))

internet_service DSL Fiber bill_band ≤600 24.4 47.8 600-750 23.9 36.9 750-900 23.7 38.3 900+ 11.1 42.5

Task 2: क्या बिना tech support वाले senior citizens support वाले seniors से ज़्यादा churn करते हैं?

Solution देखें

Python

import numpy as np
import pandas as pd

def load_churn(path="telecom_churn.csv"):
    # "None" means no internet plan – stop pandas from turning it into NaN
    df = pd.read_csv(path, keep_default_na=False, na_values=[""])
    df["total_charges"] = df["total_charges"].fillna(df["monthly_charges"] * df["tenure_months"])
    df["churned"] = (df["churn"] == "Yes").astype(int)
    return df

df = load_churn()
print((df[df["senior_citizen"] == 1].groupby("tech_support")["churned"].mean() * 100).round(1))

tech_support No 45.0 No internet 18.8 Yes 27.0 Name: churned, dtype: float64

Task 3: Head of Retention को एक paragraph की EDA summary e-mail लिखें।

Quick quiz

🧠 Quick Quiz – ख़ुद को जाँचें

1. Churn पर सबसे बड़ा असर किस variable का?

सबसे ज़्यादा Cramér’s V।

2. किसी segment का lift 1.5 मतलब…

Segment rate ÷ कुल rate।

3. Gender का p-value 0.83। निष्कर्ष…

Significant नहीं।

4. Churners के total charges कम क्यों?

total ≈ monthly × tenure।

5. Interaction heatmap क्या ढूँढने में मदद करता है?

जैसे monthly × no support।

Interview corner

💼 Interview Corner

आपके EDA ने churn के बारे में क्या बताया?

सबसे मज़बूत driver contract type है (monthly ~44% vs two-year ~9%), फिर छोटा tenure, complaints, tech support न होना और Fiber; payment method अलग दिखा पर statistically significant नहीं था। Monthly + बिना support + नया segment churners का असमान रूप से बड़ा हिस्सा रखता है – retention team का तुरंत लक्ष्य।

EDA में significance test क्यों?

कई तुलनाओं में कुछ अंतर संयोग से दिखते हैं। Tests और effect sizes असली, अर्थपूर्ण drivers को शोर से अलग करते हैं और team को random patterns के पीछे भागने से रोकते हैं।

सारांश

Contract, tenure, complaints, tech support और Fiber churn चलाते हैं; gender नहीं, और payment method significant नहीं।
Monthly + बिना support + नए customers = सबसे गर्म segment।
Effect sizes के साथ chi-square/t-tests drivers की पुष्टि करते हैं।
पाँच insights किसी भी model से पहले retention team को actions देती हैं।

➡️ आगे

Day 58: features बनाएँ, pipelines बनाएँ और cross-validation से models की तुलना करें।

अक्सर पूछे जाने वाले प्रश्न (FAQ)

क्या EDA findings सीधे business को दें?

हाँ – जल्दी के फ़ायदे (जैसे hot segment) पहले ही share करें, साफ़ तौर पर associations बताकर, जब model बन रहा हो।

Final report में कितने charts हों?

तीन से पाँच: contract के हिसाब से churn, tenure curve, interaction heatmap और driver lift chart आमतौर पर काफ़ी।

← Data Science (60 Days)

दिन 57: Project 2: EDA और Insights

✅ आज आप सीखेंगे

🏢 असली दुनिया में

हर category में churn rate

Tenure, charges और complaints

Interactions: risk कहाँ केंद्रित है

क्या अंतर असली हैं?

Drivers की ranking

Retention team के लिए पाँच insights

🏅 आज के Job Skills

Practice tasks

🏋️ Practice – ख़ुद करें

Quick quiz

🧠 Quick Quiz – ख़ुद को जाँचें

Interview corner

💼 Interview Corner

सारांश

अक्सर पूछे जाने वाले प्रश्न (FAQ)

यह lesson कैसा लगा?

पढ़ने वालों के reviews

Video से सीखें – आसान Hindi में