🟣 ML + AI · Lesson 32
Linear Algebra Basics for ML
In this lesson: Vectors, matrices, dot product and matrix multiplication for machine learning — explained with NumPy code. In Hindi & English.
Why linear algebra for AI?
Machine learning stores data as numbers in rows and columns. A student with marks in 3 subjects is a vector of 3 numbers; a class of 40 students is a matrix of 40 × 3. Training a model, comparing two documents or recommending a movie are mostly multiplications of these vectors and matrices. NumPy does them in one line and very fast.
| Object | Shape | Real example |
|---|---|---|
| Scalar | single number | GST rate 0.18 |
| Vector | (n,) | one student’s marks [91, 78, 85] |
| Matrix | (rows, columns) | marks of all students × subjects |
| Tensor | 3+ dimensions | a colour photo: height × width × 3 |
Vectors and element-wise maths
Python – vectors.py
import numpy as np
term1 = np.array([72, 85, 64]) # Maths, Science, English
term2 = np.array([80, 82, 75])
print("improvement :", term2 - term1)
print("average :", (term1 + term2) / 2)
print("with 5 grace:", term1 + 5) # scalar added to every element
print("shape :", term1.shape)improvement : [ 8 -3 11]
average : [76. 83.5 69.5]
with 5 grace: [77 90 69]
shape : (3,)
Dot product: weighted totals
The dot product multiplies matching elements and adds them. It is the heart of every linear model: score = w₁x₁ + w₂x₂ + … + b.
Python – dot_product.py
import numpy as np
qty = np.array([2, 1, 3]) # kg of kaju katli, ghewar, ladoo
price = np.array([900, 650, 500]) # ₹ per kg
print("bill =", qty @ price) # 2*900 + 1*650 + 3*500
print("same =", np.dot(qty, price))
# weighted final marks: theory 70%, practical 20%, project 10%
marks = np.array([82, 95, 90])
weights = np.array([0.7, 0.2, 0.1])
print("final marks =", round(marks @ weights, 1))bill = 3950
same = 3950
final marks = 85.4
Cosine similarity: how alike are two things?
Divide the dot product by the lengths (norms) of the two vectors and you get a number from -1 to 1. Recommendation systems and search engines use it to find “similar” users, products and sentences.
Python – cosine_similarity.py
import numpy as np
# ratings 0-5 for: action, comedy, romance, cricket documentaries
users = {
"Aman": np.array([5, 1, 0, 5]),
"Riya": np.array([1, 4, 5, 0]),
"Kabir": np.array([4, 2, 0, 4]),
}
def cosine(a, b):
return (a @ b) / (np.linalg.norm(a) * np.linalg.norm(b))
for other in ["Riya", "Kabir"]:
print(f"Aman vs {other}: {cosine(users['Aman'], users[other]):.2f}")Aman vs Riya: 0.19
Aman vs Kabir: 0.98
Aman and Kabir have similar taste (0.98), so a film Kabir loved is a good recommendation for Aman.
Matrices and matrix multiplication
Python – matrix_multiplication.py
import numpy as np
# rows = 3 students, columns = Maths, Science, English
marks = np.array([[82, 74, 91],
[65, 88, 70],
[90, 95, 85]])
print("shape:", marks.shape)
print("column averages:", marks.mean(axis=0).round(1))
# two different weight schemes as columns: (science stream, commerce stream)
weights = np.array([[0.5, 0.4],
[0.4, 0.2],
[0.1, 0.4]])
scores = marks @ weights # (3×3) @ (3×2) → (3×2)
print(scores.round(1))shape: (3, 3)
column averages: [79. 85.7 82. ]
[[79.7 84. ]
[74.7 71.6]
[91.5 89. ]]
💡 The shape rule
For A @ B the inner sizes must match: (m × n) @ (n × p) → (m × p). Most NumPy errors in ML are shape mismatches – print .shape first.
Python – shape_error.py
import numpy as np
a = np.ones((3, 2))
b = np.ones((3, 2))
try:
a @ b
except ValueError as e:
print("Error:", str(e)[:60], "...")
print("fix with transpose:", (a.T @ b).shape)Error: matmul: Input operand 1 has a mismatch in its core dimension ...
fix with transpose: (2, 2)
Transpose, identity and inverse
Python – inverse.py
import numpy as np
A = np.array([[2, 1],
[5, 3]])
print("A.T =\n", A.T)
print("identity =\n", np.eye(2, dtype=int))
A_inv = np.linalg.inv(A)
print("inverse =\n", A_inv)
print("A @ A_inv =\n", (A @ A_inv).round(10) + 0.0) # + 0.0 turns -0. into 0.A.T =
[[2 5]
[1 3]]
identity =
[[1 0]
[0 1]]
inverse =
[[ 3. -1.]
[-5. 2.]]
A @ A_inv =
[[1. 0.]
[0. 1.]]
Solving equations: a real shop problem
A shop sold 2 kg kaju katli + 1 kg ghewar for ₹2,450 on Monday and 1 kg kaju katli + 3 kg ghewar for ₹2,850 on Tuesday. What is the price per kg of each? Write it as A·x = b and let NumPy solve it.
Python – solve_equations.py
import numpy as np
A = np.array([[2, 1], # Monday: 2 kaju + 1 ghewar
[1, 3]]) # Tuesday: 1 kaju + 3 ghewar
b = np.array([2450, 2850])
x = np.linalg.solve(A, b) # more accurate than inv(A) @ b
print("kaju katli ₹/kg:", x[0])
print("ghewar ₹/kg :", x[1])
print("check:", A @ x)kaju katli ₹/kg: 900.0
ghewar ₹/kg : 650.0
check: [2450. 2850.]
Where you will meet this in ML
| Concept | Used in |
|---|---|
| Dot product / matrix multiply | Linear & logistic regression, every neural-network layer |
| Norm, cosine similarity | Recommendations, semantic search, embeddings of LLMs |
| Transpose | Formulas like the normal equation (XᵀX)⁻¹Xᵀy |
| Solving linear systems | Fitting regression exactly |
| Eigenvectors | PCA (dimensionality reduction) |
Python – normal_equation.py
import numpy as np
# linear regression with the normal equation: price = w0 + w1 * area
area = np.array([500, 750, 1000, 1250]) # sq ft
price = np.array([25, 35, 46, 55]) # ₹ lakh
X = np.column_stack([np.ones_like(area), area]) # add a column of 1s for w0
w = np.linalg.solve(X.T @ X, X.T @ price)
print("w0 = %.2f lakh, w1 = %.4f lakh per sq ft" % (w[0], w[1]))
print("predicted price for 900 sq ft: ₹%.1f lakh" % (w @ [1, 900]))w0 = 4.90 lakh, w1 = 0.0404 lakh per sq ft
predicted price for 900 sq ft: ₹41.3 lakh
Continue with Linear Regression and PCA in the Data Science course, where these ideas are used on real datasets.
Practice tasks
- Find the total bill for 4 items using a dot product.
- Compute cosine similarity between your ratings and two friends’ ratings for 5 movies.
- Solve: 3x + 2y = 18 and x + 4y = 16 with
np.linalg.solve, then check the answer. - Create a 5×3 marks matrix and compute each student’s weighted score with one
@.
Summary
- Vectors are 1-D arrays, matrices are 2-D; check
.shapeoften. a @ b= dot product / matrix multiplication; inner sizes must match.- Cosine similarity compares direction – used for recommendations and search.
- Use
np.linalg.solvefor equations;.Ttransposes. - These operations power regression, neural networks and PCA.
Frequently Asked Questions
How much maths do I need for machine learning?
To use libraries you need vectors, matrices, dot products and basic statistics. To understand and research algorithms you also need calculus (gradients) and probability. Start with this lesson and the statistics lessons.
What is the difference between * and @ in NumPy?
* multiplies element by element (shapes must match or broadcast). @ performs a dot product or matrix multiplication (inner sizes must match).
💻 Live Code Editor
This page's programs are ready here — run them, edit them, and learn. No installation needed.
Powered by OneCompiler. The code loads into the editor automatically — press Run to see the output. If the editor does not open, open it in a new tab.
