🤖 Week 7 – Machine Learning Core · Day 37
Day 37: scikit-learn & Train/Test Split
⏱ 2 hours📶 Beginner → Intermediate🎯 100% self-study
Day 37 / 60
✅ Today you will learn
- The scikit-learn API: fit → predict → score
- Prepare X and y correctly
- Split data with stratify and random_state
- Always beat a baseline first
- Predict probabilities and new applicants
🏢 In the real world
A fintech start-up wants an instant loan decision in its app. You must build the first version today, prove it is better than guessing, and show the product team how the model decides for a new applicant.
One API for every model
Step 1: prepare X and y
Python
import pandas as pd
loans = pd.read_csv("loan_applications.csv")
loans["is_salaried"] = (loans["employment"] == "Salaried").astype(int)
loans["loan_to_income"] = (loans["loan_amount"] / (loans["monthly_income"] * 12)).round(3)
features = ["monthly_income", "credit_score", "loan_amount", "years_employed", "is_salaried", "loan_to_income"]
X = loans[features]
y = loans["approved"]
print(X.shape, y.shape)
print(X.head(3))
print("Class balance:", y.value_counts(normalize=True).round(3).to_dict())(1200, 6) (1200,)
monthly_income credit_score loan_amount years_employed is_salaried loan_to_income
0 45300 819 502000 12 1 0.923
1 61000 799 806000 25 0 1.101
2 71200 801 367000 3 0 0.430
Class balance: {0: 0.538, 1: 0.462}
🧠 Rules for X
X must be all numbers with no missing values (for most models). Text columns need encoding (Day 44). Never put the target – or anything known only after the decision – inside X.
Step 2: split
Python
import pandas as pd
from sklearn.model_selection import train_test_split
loans = pd.read_csv("loan_applications.csv")
X = loans[["monthly_income", "credit_score", "loan_amount", "years_employed"]]; y = loans["approved"]
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42, stratify=y)
print("train:", X_train.shape, "test:", X_test.shape)
print("approval rate → train", round(y_train.mean(), 3), "| test", round(y_test.mean(), 3))train: (960, 4) test: (240, 4)
approval rate → train 0.462 | test 0.462
| Argument | Why |
|---|---|
| test_size=0.2 | 20% held back for the final check |
| random_state=42 | Same split every run – reproducible results |
| stratify=y | Same class balance in train and test |
Step 3: a baseline
Python
import pandas as pd
from sklearn.model_selection import train_test_split
from sklearn.dummy import DummyClassifier
loans = pd.read_csv("loan_applications.csv")
X = loans[["monthly_income", "credit_score", "loan_amount", "years_employed"]]; y = loans["approved"]
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42, stratify=y)
base = DummyClassifier(strategy="most_frequent").fit(X_train, y_train)
print(f"Baseline (always predict the majority class): {base.score(X_test, y_test):.1%}")Baseline (always predict the majority class): 53.8%
⚠️ 90% accuracy can be useless
If 90% of transactions are not fraud, a model that always says “not fraud” is 90% accurate and catches nothing. Always compare with the baseline – and on Day 41 we learn better metrics.
Step 4: train real models
Python
import pandas as pd
from sklearn.model_selection import train_test_split
from sklearn.linear_model import LogisticRegression
from sklearn.tree import DecisionTreeClassifier
from sklearn.neighbors import KNeighborsClassifier
from sklearn.ensemble import RandomForestClassifier
loans = pd.read_csv("loan_applications.csv")
loans["loan_to_income"] = loans["loan_amount"] / (loans["monthly_income"] * 12)
loans["is_salaried"] = (loans["employment"] == "Salaried").astype(int)
X = loans[["monthly_income", "credit_score", "loan_amount", "years_employed", "is_salaried", "loan_to_income"]]; y = loans["approved"]
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42, stratify=y)
models = {"Logistic regression": LogisticRegression(max_iter=5000),
"Decision tree": DecisionTreeClassifier(max_depth=4, random_state=42),
"KNN (k=15)": KNeighborsClassifier(n_neighbors=15),
"Random forest": RandomForestClassifier(n_estimators=200, random_state=42)}
for name, m in models.items():
m.fit(X_train, y_train)
print(f"{name:20s} train={m.score(X_train, y_train):.3f} test={m.score(X_test, y_test):.3f}")Logistic regression train=0.825 test=0.825
Decision tree train=0.840 test=0.842
KNN (k=15) train=0.650 test=0.583
Random forest train=1.000 test=0.833
💡 What just happened?
The engineered feature loan_to_income (Day 17 skills!) lifted the decision tree from about 81% (Day 36) to 84%. The random forest scores 100% on training data – a sign of memorising. KNN is weak because it needs scaled features (Day 44). Same API, four algorithms, one loop.
Step 5: predict for new applicants
Python
import pandas as pd
from sklearn.model_selection import train_test_split
from sklearn.linear_model import LogisticRegression
loans = pd.read_csv("loan_applications.csv")
def make_X(df):
return pd.DataFrame({"monthly_income": df["monthly_income"], "credit_score": df["credit_score"],
"loan_amount": df["loan_amount"], "years_employed": df["years_employed"],
"is_salaried": (df["employment"] == "Salaried").astype(int),
"loan_to_income": df["loan_amount"] / (df["monthly_income"] * 12)})
X, y = make_X(loans), loans["approved"]
model = LogisticRegression(max_iter=5000).fit(X, y)
new = pd.DataFrame({"monthly_income": [55000, 30000, 120000], "credit_score": [780, 640, 720],
"loan_amount": [500000, 900000, 3000000], "years_employed": [6, 2, 12],
"employment": ["Salaried", "Self-Employed", "Business"]}, index=["Riya", "Arjun", "Meera"])
proba = model.predict_proba(make_X(new))[:, 1]
out = new[["monthly_income", "credit_score", "loan_amount"]].assign(approval_prob=proba.round(3), decision=model.predict(make_X(new)))
print(out) monthly_income credit_score loan_amount approval_prob decision
Riya 55000 780 500000 0.848 1
Arjun 30000 640 900000 0.042 0
Meera 120000 720 3000000 0.525 1
🧠 predict vs predict_proba
predict gives 0/1 using a 0.5 cut-off. predict_proba gives the probability – far more useful: the bank can auto-approve above 0.8, auto-reject below 0.2 and send the middle to a human.
Step 6: look inside
Python
import pandas as pd
import matplotlib.pyplot as plt
from sklearn.tree import DecisionTreeClassifier, plot_tree
loans = pd.read_csv("loan_applications.csv")
loans["loan_to_income"] = loans["loan_amount"] / (loans["monthly_income"] * 12)
X = loans[["credit_score", "loan_to_income", "years_employed"]]; y = loans["approved"]
tree = DecisionTreeClassifier(max_depth=2, random_state=42).fit(X, y)
plt.figure(figsize=(11, 4.6))
plot_tree(tree, feature_names=X.columns, class_names=["reject", "approve"], filled=True, rounded=True, fontsize=9, impurity=False)
plt.title("A depth-2 tree: the rules the model learned"); plt.show()
🏅 Job skills unlocked today
✔ Use fit / predict / predict_proba / score✔ Split data correctly with stratify✔ Compare models with a baseline✔ Prepare new data with the same function
Practice tasks
🏋️ Practice – do it yourself
Task 1: Predict whether a student’s average is ≥ 70 from study hours and attendance with LogisticRegression. Beat the baseline?
Show solution
Python
import pandas as pd
from sklearn.model_selection import train_test_split
from sklearn.linear_model import LogisticRegression
from sklearn.dummy import DummyClassifier
st = pd.read_csv("students_performance.csv")
X, y = st[["study_hours", "attendance"]], (st["average"] >= 70).astype(int)
a, b, c, d = train_test_split(X, y, test_size=0.2, random_state=1, stratify=y)
print("baseline", DummyClassifier().fit(a, c).score(b, d), "| model", round(LogisticRegression().fit(a, c).score(b, d), 3))baseline 0.65 | model 0.87
Task 2: Change
random_state of the split to 0, 1, 2, 3, 4. How much does the random-forest test accuracy move?Show solution
Python
import pandas as pd
from sklearn.model_selection import train_test_split
from sklearn.ensemble import RandomForestClassifier
l = pd.read_csv("loan_applications.csv"); l["lti"] = l["loan_amount"] / (l["monthly_income"] * 12)
X, y = l[["monthly_income", "credit_score", "loan_amount", "years_employed", "lti"]], l["approved"]
for s in range(5):
a, b, c, d = train_test_split(X, y, test_size=0.2, random_state=s, stratify=y)
print(s, round(RandomForestClassifier(n_estimators=100, random_state=0).fit(a, c).score(b, d), 3))0 0.808
1 0.829
2 0.792
3 0.796
4 0.804
Task 3: Add your own applicant (your numbers) to the
new DataFrame and read the approval probability.Quick quiz
🧠 Quick Quiz – test yourself
1. Which method learns from data?
fit(X, y) trains the model.
2. stratify=y in train_test_split…
Important for imbalanced classes.
3. DummyClassifier is used to…
If you cannot beat it, the model has learned nothing.
4. predict_proba(X)[:, 1] returns…
Column 0 = class 0, column 1 = class 1.
5. Why use random_state?
Same randomness every run.
Interview corner
💼 Interview Corner
Why do we need a train/test split?
To estimate how the model performs on data it has never seen. Evaluating on training data rewards memorisation and hides overfitting.
What is data leakage?
When information not available at prediction time (or from the test set) sneaks into training – e.g. using “loan_disbursed_date” to predict approval, or scaling with statistics from the full data. It gives fake-high scores.
Summary
- Estimator → fit → predict / predict_proba → score.
- train_test_split(test_size, random_state, stratify).
- Beat DummyClassifier/DummyRegressor first.
- Use one function to build X for training and for new data.
➡️ Next
Day 38: linear regression – predict house prices and understand every coefficient.
Frequently Asked Questions
Which algorithm is best?
There is no single best. Start simple (logistic/linear regression, a small tree), then try random forests or gradient boosting, and compare on the same test set.
Why did I get a ConvergenceWarning?
Logistic regression needed more iterations, usually because features have very different scales. Increase max_iter or scale features (Day 44).
