📐 Week 5 – Statistics & Probability · Day 26
Day 26: Descriptive Statistics
⏱ 2 hours📶 Beginner → Intermediate🎯 100% self-study
Day 26 / 60
✅ Today you will learn
- Centre: mean, median, mode – and when each lies
- Spread: range, variance, standard deviation, IQR
- Percentiles and the five-number summary
- Skewness and coefficient of variation
- z-scores to compare apples and oranges
🏢 In the real world
A start-up advertises “average salary ₹16.7 lakh” to attract freshers. The truth: the two founders earn ₹60 lakh each and most of the team earns ₹6–8 lakh. Knowing which statistic to trust protects you from being fooled – and from fooling your boss.
Measures of centre
Python
import numpy as np
import pandas as pd
salary = pd.Series([6, 6.5, 7, 7, 7.5, 8, 8, 8, 6, 60, 60], name="lakh")
print("Mean :", round(salary.mean(), 2))
print("Median:", salary.median())
print("Mode :", salary.mode().tolist())
print("Mean without founders:", round(salary[salary < 50].mean(), 2))Mean : 16.73
Median: 7.5
Mode : [8.0]
Mean without founders: 7.11
⚖️
MeanSum ÷ count. Uses every value, so outliers pull it.🎯
MedianMiddle value after sorting. Robust to outliers – use for salary, price, income.🔁
ModeMost frequent value. Best for categories (most popular size, city).Measures of spread
Python
import pandas as pd
a = pd.Series([30, 31, 29, 30, 32, 28, 30], name="Courier A (minutes)")
b = pd.Series([10, 55, 20, 45, 30, 15, 35], name="Courier B (minutes)")
for s in (a, b):
print(f"{s.name}: mean={s.mean():.1f} range={s.max() - s.min()} var={s.var():.1f} sd={s.std():.1f}")Courier A (minutes): mean=30.0 range=4 var=1.7 sd=1.3
Courier B (minutes): mean=30.0 range=45 var=266.7 sd=16.3
🛵 Same average, different experience
Both couriers average 30 minutes. Courier A is predictable (SD ≈ 1.3), courier B is a lottery (SD ≈ 16). Customers hate unpredictability – spread matters as much as the average.
| Statistic | Formula (idea) | pandas |
|---|---|---|
| Range | max − min | s.max() - s.min() |
| Variance | average squared distance from mean | s.var() |
| Standard deviation | √variance – in original units | s.std() |
| IQR | Q3 − Q1 (middle 50%) | s.quantile(.75) - s.quantile(.25) |
| CV | SD ÷ mean × 100 (relative spread) | s.std() / s.mean() * 100 |
Python
import numpy as np
x = np.array([4, 8, 6, 5, 7])
mean = x.mean()
dev = x - mean
print("deviations:", dev, "| squared:", dev ** 2)
print("sample variance (÷ n-1):", (dev ** 2).sum() / (len(x) - 1), "=", x.var(ddof=1))
print("sample SD:", round(np.sqrt((dev ** 2).sum() / (len(x) - 1)), 3), "=", round(x.std(ddof=1), 3))deviations: [-2. 2. 0. -1. 1.] | squared: [4. 4. 0. 1. 1.]
sample variance (÷ n-1): 2.5 = 2.5
sample SD: 1.581 = 1.581
💡 n or n−1?
pandas .std() divides by n−1 (sample SD, the usual choice in data science). NumPy np.std() divides by n unless you pass ddof=1. Small difference, but interviewers ask!
Percentiles and the five-number summary
Python
import pandas as pd
t = pd.read_csv("telecom_churn.csv")
bill = t["monthly_charges"]
print(bill.quantile([0, 0.25, 0.5, 0.75, 1]).round(0).to_dict())
print("90th percentile:", round(bill.quantile(0.9)), "→ top 10% pay more than this")
print("IQR:", round(bill.quantile(0.75) - bill.quantile(0.25))){0.0: 149.0, 0.25: 459.0, 0.5: 679.0, 0.75: 881.0, 1.0: 1408.0}
90th percentile: 1011 → top 10% pay more than this
IQR: 422
Python
import pandas as pd
import matplotlib.pyplot as plt
t = pd.read_csv("telecom_churn.csv")
b = t["monthly_charges"]
fig, ax = plt.subplots(figsize=(9, 3.6))
ax.hist(b, bins=40, color="#c7d2fe", edgecolor="white")
for q, c, l in [(0.25, "#0891b2", "Q1"), (0.5, "#16a34a", "Median"), (0.75, "#0891b2", "Q3")]:
ax.axvline(b.quantile(q), color=c, lw=2); ax.text(b.quantile(q), ax.get_ylim()[1] * 0.92, " " + l, color=c)
ax.axvline(b.mean(), color="#dc2626", ls="--", lw=2); ax.text(b.mean(), ax.get_ylim()[1] * 0.8, " Mean", color="#dc2626")
ax.set_title("Monthly bills: quartiles split customers into four equal groups"); ax.set_xlabel("₹ per month")
plt.show()
Shape: skewness and CV
Python
import pandas as pd
h = pd.read_csv("house_prices.csv")
s = pd.read_csv("sales_data.csv")
for name, col in [("House price", h["price_lakh"]), ("Unit price", s["unit_price"]), ("Customer age", s["customer_age"])]:
print(f"{name:13s} mean={col.mean():9.1f} median={col.median():8.1f} skew={col.skew():5.2f} CV={col.std() / col.mean() * 100:5.1f}%")House price mean= 56.8 median= 54.3 skew= 1.06 CV= 43.4%
Unit price mean= 2499.2 median= 935.5 skew= 6.29 CV=259.9%
Customer age mean= 31.9 median= 32.0 skew= 0.27 CV= 26.9%
z-scores: how unusual is a value?
z = (value − mean) ÷ SD. It says how many standard deviations a value is from the mean. |z| > 3 is very unusual.
Python
import pandas as pd
st = pd.read_csv("students_performance.csv")
for subj in ["maths", "english"]:
st[subj + "_z"] = ((st[subj] - st[subj].mean()) / st[subj].std()).round(2)
riya = st.iloc[7]
print(f"Student {riya.student_id}: maths {riya.maths} (z={riya.maths_z}), english {riya.english} (z={riya.english_z})")
print("Relative to classmates, the better subject is", "maths" if riya.maths_z > riya.english_z else "english")
print("Students with |z| > 2.5 in maths:", (st["maths_z"].abs() > 2.5).sum())Student S1008: maths 61.0 (z=-0.32), english 68.0 (z=0.11)
Relative to classmates, the better subject is english
Students with |z| > 2.5 in maths: 5
Python
import pandas as pd
s = pd.read_csv("sales_data.csv")
s["revenue"] = s["quantity"] * s["unit_price"] * (1 - s["discount"])
s["z"] = (s["revenue"] - s["revenue"].mean()) / s["revenue"].std()
print("Orders with z > 3 (check for fraud or bulk buyers):", (s["z"] > 3).sum())
print(s.nlargest(3, "z")[["order_id", "product", "quantity", "revenue", "z"]].round(2).to_string(index=False))Orders with z > 3 (check for fraud or bulk buyers): 35
order_id product quantity revenue z
BK100824 Laptop 2 114432.0 13.88
BK101257 Laptop 2 112962.0 13.70
BK101612 Laptop 2 105231.5 12.73
Real example: a salary report that does not mislead
Python
import pandas as pd
e = pd.read_csv("employees.csv")
d = pd.read_csv("departments.csv")
df = e.merge(d, on="dept_id", how="left").fillna({"dept_name": "Unassigned"})
rep = df.groupby("dept_name")["salary"].describe()[["count", "mean", "50%", "std", "min", "max"]]
rep["cv_%"] = (rep["std"] / rep["mean"] * 100).round(1)
print((rep / [1, 1000, 1000, 1000, 1000, 1000, 1]).round(1).rename(columns={"50%": "median"}))
print("(salaries in ₹ thousand)") count mean median std min max cv_%
dept_name
Data Science 11.0 105.8 111.0 24.1 74.0 146.0 22.8
Finance 1.0 61.0 61.0 NaN 61.0 61.0 NaN
HR 5.0 52.8 52.0 7.5 42.0 62.0 14.1
Marketing 1.0 79.0 79.0 NaN 79.0 79.0 NaN
Sales 10.0 57.5 55.0 11.9 42.0 78.0 20.8
Unassigned 2.0 71.5 71.5 12.0 63.0 80.0 16.8
(salaries in ₹ thousand)
🏅 Job skills unlocked today
✔ Choose mean vs median correctly✔ Measure spread with SD and IQR✔ Read percentiles and quartiles✔ Use z-scores to compare and flag anomalies
Practice tasks
🏋️ Practice – do it yourself
Task 1: Which city has the most consistent house prices per sqft (lowest CV)?
Show solution
Python
import pandas as pd
h = pd.read_csv("house_prices.csv")
h["ppsf"] = h["price_lakh"] * 1e5 / h["area_sqft"]
g = h.groupby("city")["ppsf"]
print((g.std() / g.mean() * 100).round(1).sort_values())city
Jaipur 12.5
Bengaluru 14.2
Hyderabad 15.1
Lucknow 15.4
Pune 15.7
Name: ppsf, dtype: float64
Task 2: Find the 10th and 90th percentile of daily AQI in Delhi for 2024.
Show solution
Python
import pandas as pd
w = pd.read_csv("delhi_weather.csv", parse_dates=["date"])
print(w[w["date"].dt.year == 2024]["aqi"].quantile([0.1, 0.9]).round(0))0.1 76.0
0.9 296.0
Name: aqi, dtype: float64
Task 3: A student scored 78 in a test with mean 70 and SD 4, and 85 in another with mean 80 and SD 10. In which test did they do better relative to others?
Show solution
Python
z1 = (78 - 70) / 4
z2 = (85 - 80) / 10
print("Test 1 z =", z1, "| Test 2 z =", z2, "→ better in", "Test 1" if z1 > z2 else "Test 2")Test 1 z = 2.0 | Test 2 z = 0.5 → better in Test 1
Quick quiz
🧠 Quick Quiz – test yourself
1. Best “average” for house prices with a few mansions?
Median ignores extreme values.
2. Standard deviation is measured in…
Variance is squared; SD is its square root.
3. IQR covers…
From Q1 to Q3.
4. z = −2 means the value is…
z counts standard deviations.
5. Mean > median usually indicates…
A long right tail pulls the mean up.
Interview corner
💼 Interview Corner
When would you report the median instead of the mean?
When data is skewed or has outliers – salaries, house prices, delivery times, session lengths. Ideally report both plus a spread measure (IQR or SD).
Explain standard deviation to a manager.
“On average, a delivery is about X minutes away from the usual 30 minutes.” Small SD = predictable, large SD = inconsistent.
Summary
- Mean for symmetric data, median for skewed data, mode for categories.
- SD = typical distance from the mean; IQR = middle 50%.
- Percentiles describe position; CV compares relative spread.
- z-scores put different scales on one ruler and flag outliers.
➡️ Next
Day 27: probability – the language of uncertainty, with simulations you can run.
Frequently Asked Questions
Is describe() enough?
It gives count, mean, std, min, quartiles and max – a great start. Add median vs mean, skew and value counts for a full picture.
What is a “good” CV?
There is no universal rule. Compare CVs of similar things – e.g. delivery partners or cities – lower means more consistent.
