Free tutorials & notes in Hindi & English · Clean code examples 🧰 Free PDF & Image Tools →
🛢️ Week 6 – SQL for Data Science  ·  Day 34

Day 34: Window Functions & CTEs

⏱ 2.5 hours📶 Advanced🎯 100% self-study
Day 34 / 60

✅ Today you will learn

  • CTEs: readable multi-step queries
  • Window functions vs GROUP BY
  • ROW_NUMBER, RANK, DENSE_RANK and top-N per group
  • Running totals and moving averages
  • LAG/LEAD growth and NTILE segments

🏢 In the real world

“Top 2 products in every city”, “month-over-month growth”, “cumulative revenue towards the annual target”, “salary rank inside each department” – these four questions appear in almost every data-analyst SQL round, and all four need window functions.

Python – setup
import sqlite3
import pandas as pd
con = sqlite3.connect("bazaar.db")
pd.read_csv("sales_data.csv").to_sql("sales", con, if_exists="replace", index=False)
pd.read_csv("employees.csv").to_sql("employees", con, if_exists="replace", index=False)
pd.read_csv("departments.csv").to_sql("departments", con, if_exists="replace", index=False)
print([r[0] for r in con.execute("SELECT name FROM sqlite_master WHERE type = 'table'")])
['sales', 'employees', 'departments']

CTEs – WITH

A CTE gives a name to a sub-query so you can build a query in steps – like variables in Python.

Python
import sqlite3
import pandas as pd
con = sqlite3.connect("bazaar.db")
q = lambda sql: pd.read_sql(sql, con)
print(q("""
WITH orders AS (
    SELECT order_id, city, category, order_date,
           quantity * unit_price * (1 - discount) AS revenue
    FROM sales
),
city_rev AS (
    SELECT city, SUM(revenue) AS revenue FROM orders GROUP BY city
)
SELECT city, ROUND(revenue / 100000.0, 2) AS revenue_lakh
FROM city_rev
WHERE revenue > (SELECT AVG(revenue) FROM city_rev)      -- alias usable now!
ORDER BY revenue DESC
"""))
city revenue_lakh 0 Mumbai 10.43 1 Bengaluru 9.64 2 Delhi 9.46 3 Kolkata 8.40

Window functions: keep the rows

GROUP BY

  • Collapses rows: one row per group
  • SUM(salary) per department
  • Like pandas agg

Window: OVER (PARTITION BY …)

  • Keeps every row and adds a group value
  • Each employee + department total
  • Like pandas transform / rank / shift
Python
import sqlite3
import pandas as pd
con = sqlite3.connect("bazaar.db")
q = lambda sql: pd.read_sql(sql, con)
print(q("""
SELECT name, dept_id, salary,
       ROUND(AVG(salary) OVER (PARTITION BY dept_id))            AS dept_avg,
       salary - ROUND(AVG(salary) OVER (PARTITION BY dept_id))   AS vs_avg,
       ROUND(100.0 * salary / SUM(salary) OVER (PARTITION BY dept_id), 1) AS share_pct
FROM employees
WHERE dept_id IN (10, 30)
ORDER BY dept_id, salary DESC
"""))
name dept_id salary dept_avg vs_avg share_pct 0 Vihaan 10 78000 57500.0 20500.0 13.6 1 Yash 10 74000 57500.0 16500.0 12.9 2 Pooja 10 65000 57500.0 7500.0 11.3 3 Isha 10 61000 57500.0 3500.0 10.6 4 Nisha 10 57000 57500.0 -500.0 9.9 5 Sara 10 53000 57500.0 -4500.0 9.2 6 Om 10 50000 57500.0 -7500.0 8.7 7 Ananya 10 49000 57500.0 -8500.0 8.5 8 Arjun 10 46000 57500.0 -11500.0 8.0 9 Kavya 10 42000 57500.0 -15500.0 7.3 10 Zoya 30 62000 52800.0 9200.0 23.5 11 Harsh 30 57000 52800.0 4200.0 21.6 12 Siddharth 30 52000 52800.0 -800.0 19.7 13 Dev 30 51000 52800.0 -1800.0 19.3 14 Aisha 30 42000 52800.0 -10800.0 15.9

Ranking functions

Python
import sqlite3
import pandas as pd
con = sqlite3.connect("bazaar.db")
q = lambda sql: pd.read_sql(sql, con)
print(q("""
WITH team(name, salary) AS (
    VALUES ('Aman', 90000), ('Riya', 90000), ('Om', 85000),
           ('Zoya', 80000), ('Kabir', 80000), ('Isha', 75000)
)
SELECT name, salary,
       ROW_NUMBER() OVER (ORDER BY salary DESC) AS row_num,
       RANK()       OVER (ORDER BY salary DESC) AS rnk,
       DENSE_RANK() OVER (ORDER BY salary DESC) AS dense_rnk
FROM team
"""))
name salary row_num rnk dense_rnk 0 Aman 90000 1 1 1 1 Riya 90000 2 1 1 2 Om 85000 3 3 2 3 Zoya 80000 4 4 3 4 Kabir 80000 5 4 3 5 Isha 75000 6 6 4
🧠 Ties

With equal salaries ROW_NUMBER still gives 1, 2, 3; RANK gives 1, 1, 3 (skips); DENSE_RANK gives 1, 1, 2 (no gap). Interviewers love asking for the 2nd-highest salary – use DENSE_RANK.

Python
import sqlite3
import pandas as pd
con = sqlite3.connect("bazaar.db")
q = lambda sql: pd.read_sql(sql, con)
print(q("""
WITH r AS (
    SELECT city, product, ROUND(SUM(quantity * unit_price * (1 - discount))) AS revenue,
           ROW_NUMBER() OVER (PARTITION BY city ORDER BY SUM(quantity * unit_price * (1 - discount)) DESC) AS rn
    FROM sales
    GROUP BY city, product
)
SELECT city, product, revenue FROM r WHERE rn <= 2 AND city IN ('Delhi', 'Mumbai', 'Patna', 'Jaipur')
ORDER BY city, rn
"""))
city product revenue 0 Delhi Mobile Phone 234751.0 1 Delhi Laptop 193238.0 2 Jaipur Sneakers 39827.0 3 Jaipur Mobile Phone 31546.0 4 Mumbai Laptop 412270.0 5 Mumbai Mobile Phone 175879.0 6 Patna Laptop 129379.0 7 Patna Mobile Phone 116078.0
Python
import sqlite3
import pandas as pd
con = sqlite3.connect("bazaar.db")
q = lambda sql: pd.read_sql(sql, con)
print(q("""
WITH ranked AS (
    SELECT d.dept_name, e.name, e.salary,
           DENSE_RANK() OVER (PARTITION BY e.dept_id ORDER BY e.salary DESC) AS r
    FROM employees e JOIN departments d ON e.dept_id = d.dept_id
)
SELECT dept_name, name, salary FROM ranked WHERE r = 2      -- 2nd highest per department
"""))
dept_name name salary 0 Sales Yash 74000 1 Data Science Tanvi 135000 2 HR Harsh 57000

Running totals and moving averages

Python
import sqlite3
import pandas as pd
con = sqlite3.connect("bazaar.db")
q = lambda sql: pd.read_sql(sql, con)
m = q("""
WITH monthly AS (
    SELECT strftime('%Y-%m', order_date) AS month,
           SUM(quantity * unit_price * (1 - discount)) / 100000.0 AS rev
    FROM sales GROUP BY month
)
SELECT month, ROUND(rev, 2) AS revenue_lakh,
       ROUND(SUM(rev) OVER (ORDER BY month), 2)                                   AS cumulative,
       ROUND(AVG(rev) OVER (ORDER BY month ROWS BETWEEN 2 PRECEDING AND CURRENT ROW), 2) AS moving_avg_3m
FROM monthly
""")
print(m.to_string(index=False))
month revenue_lakh cumulative moving_avg_3m 2024-01 7.03 7.03 7.03 2024-02 5.24 12.27 6.13 2024-03 5.90 18.16 6.05 2024-04 4.32 22.48 5.15 2024-05 4.08 26.56 4.76 2024-06 5.02 31.58 4.47 2024-07 4.44 36.01 4.51 2024-08 4.92 40.93 4.79 2024-09 4.45 45.39 4.60 2024-10 8.61 54.00 5.99 2024-11 10.00 63.99 7.69 2024-12 4.80 68.80 7.80
Python
import sqlite3
import pandas as pd
con = sqlite3.connect("bazaar.db")
q = lambda sql: pd.read_sql(sql, con)
import matplotlib.pyplot as plt
m = q("""
WITH monthly AS (SELECT strftime('%m', order_date) AS month, SUM(quantity * unit_price * (1 - discount)) / 100000.0 AS rev
                 FROM sales GROUP BY month)
SELECT month, rev, SUM(rev) OVER (ORDER BY month) AS cum FROM monthly
""")
fig, ax = plt.subplots(figsize=(9, 3.8))
ax.bar(m["month"], m["rev"], color="#c7d2fe", label="monthly")
ax.plot(m["month"], m["cum"], color="#4f46e5", marker="o", label="cumulative")
ax.axhline(65, color="#dc2626", ls="--", label="annual target ₹65 lakh")
cross = m.loc[m["cum"] >= 65, "month"].iloc[0]
ax.set_title(f"Annual target crossed in month {cross}"); ax.set_ylabel("₹ lakh"); ax.legend()
plt.show()
Monthly revenue bars with cumulative line crossing the annual target
SUM() OVER (ORDER BY month) builds the cumulative line directly in SQL.

LAG and LEAD: growth

Python
import sqlite3
import pandas as pd
con = sqlite3.connect("bazaar.db")
q = lambda sql: pd.read_sql(sql, con)
print(q("""
WITH monthly AS (
    SELECT strftime('%Y-%m', order_date) AS month,
           SUM(quantity * unit_price * (1 - discount)) AS rev
    FROM sales GROUP BY month
)
SELECT month,
       ROUND(rev / 1e5, 2)                               AS rev_lakh,
       ROUND(LAG(rev) OVER (ORDER BY month) / 1e5, 2)    AS prev_lakh,
       ROUND(100.0 * (rev - LAG(rev) OVER (ORDER BY month)) / LAG(rev) OVER (ORDER BY month), 1) AS mom_growth_pct
FROM monthly
""").to_string(index=False))
month rev_lakh prev_lakh mom_growth_pct 2024-01 7.03 NaN NaN 2024-02 5.24 7.03 -25.5 2024-03 5.90 5.24 12.6 2024-04 4.32 5.90 -26.8 2024-05 4.08 4.32 -5.6 2024-06 5.02 4.08 23.0 2024-07 4.44 5.02 -11.5 2024-08 4.92 4.44 10.8 2024-09 4.45 4.92 -9.4 2024-10 8.61 4.45 93.3 2024-11 10.00 8.61 16.1 2024-12 4.80 10.00 -52.0

NTILE: segments

Python
import sqlite3
import pandas as pd
con = sqlite3.connect("bazaar.db")
q = lambda sql: pd.read_sql(sql, con)
print(q("""
WITH c AS (
    SELECT city, SUM(quantity * unit_price * (1 - discount)) AS rev FROM sales GROUP BY city
)
SELECT city, ROUND(rev) AS revenue,
       NTILE(3) OVER (ORDER BY rev DESC) AS tier,
       ROUND(PERCENT_RANK() OVER (ORDER BY rev), 2) AS pct_rank
FROM c ORDER BY rev DESC
"""))
city revenue tier pct_rank 0 Mumbai 1043230.0 1 1.0 1 Bengaluru 964020.0 1 0.9 2 Delhi 945592.0 1 0.8 3 Kolkata 839885.0 1 0.7 4 Hyderabad 572190.0 2 0.6 5 Chennai 545479.0 2 0.5 6 Patna 459941.0 2 0.4 7 Ahmedabad 450612.0 2 0.3 8 Pune 403766.0 3 0.2 9 Lucknow 394912.0 3 0.1 10 Jaipur 260155.0 3 0.0
SQL windowpandas
AVG(x) OVER (PARTITION BY g)df.groupby("g")["x"].transform("mean")
ROW_NUMBER() OVER (PARTITION BY g ORDER BY x DESC)df.groupby("g")["x"].rank(method="first", ascending=False)
SUM(x) OVER (ORDER BY t)df["x"].cumsum()
AVG(x) OVER (… ROWS 2 PRECEDING)df["x"].rolling(3).mean()
LAG(x)df["x"].shift(1)
NTILE(4)pd.qcut(df.x, 4)

🏅 Job skills unlocked today

✔ Structure queries with CTEs✔ Use PARTITION BY for group values on every row✔ Rank and get top-N per group✔ Compute running totals, moving averages and growth

Practice tasks

🏋️ Practice – do it yourself

Task 1: For each region, rank cities by revenue (1 = highest).
Show solution
Python
import sqlite3
import pandas as pd
con = sqlite3.connect("bazaar.db")
q = lambda sql: pd.read_sql(sql, con)
print(q("""WITH c AS (SELECT region, city, SUM(quantity*unit_price*(1-discount)) AS rev FROM sales GROUP BY region, city)
SELECT region, city, ROUND(rev) AS rev, RANK() OVER (PARTITION BY region ORDER BY rev DESC) AS rnk FROM c ORDER BY region, rnk"""))
region city rev rnk 0 East Kolkata 839885.0 1 1 East Patna 459941.0 2 2 North Delhi 945592.0 1 3 North Lucknow 394912.0 2 4 North Jaipur 260155.0 3 5 South Bengaluru 964020.0 1 6 South Hyderabad 572190.0 2 7 South Chennai 545479.0 3 8 West Mumbai 1043230.0 1 9 West Ahmedabad 450612.0 2 10 West Pune 403766.0 3
Task 2: Find the 3rd highest salary in the company (handle ties).
Show solution
Python
import sqlite3
import pandas as pd
con = sqlite3.connect("bazaar.db")
q = lambda sql: pd.read_sql(sql, con)
print(q("""SELECT DISTINCT salary FROM (SELECT salary, DENSE_RANK() OVER (ORDER BY salary DESC) AS r FROM employees) WHERE r = 3"""))
salary 0 122000
Task 3: Show weekly orders in October 2024 with a running total.
Show solution
Python
import sqlite3
import pandas as pd
con = sqlite3.connect("bazaar.db")
q = lambda sql: pd.read_sql(sql, con)
print(q("""WITH w AS (SELECT strftime('%W', order_date) AS week, COUNT(*) AS orders FROM sales
WHERE order_date BETWEEN '2024-10-01' AND '2024-10-31' GROUP BY week)
SELECT week, orders, SUM(orders) OVER (ORDER BY week) AS running FROM w"""))
week orders running 0 40 48 48 1 41 50 98 2 42 56 154 3 43 64 218 4 44 31 249

Quick quiz

🧠 Quick Quiz – test yourself

1. Window functions differ from GROUP BY because they…
They add a value computed over a window of rows.
2. Salaries 100, 100, 90: RANK gives…
RANK skips after ties.
3. LAG(revenue) returns…
Ordered by the OVER clause.
4. PARTITION BY dept_id means…
Like groupby, without collapsing.
5. A CTE starts with…
WITH name AS (…) SELECT …

Interview corner

💼 Interview Corner

Write a query for the top 3 products by revenue in each category.

WITH r AS (SELECT category, product, SUM(rev) AS rev, ROW_NUMBER() OVER (PARTITION BY category ORDER BY SUM(rev) DESC) AS rn FROM sales GROUP BY category, product) SELECT * FROM r WHERE rn <= 3;

How do you compute month-over-month growth in SQL?

Aggregate to one row per month in a CTE, then (rev − LAG(rev) OVER (ORDER BY month)) / LAG(rev) OVER (ORDER BY month) × 100.

Summary

  • WITH … AS builds readable step-by-step queries.
  • OVER (PARTITION BY … ORDER BY …) adds group values without collapsing rows.
  • ROW_NUMBER/RANK/DENSE_RANK for rankings and top-N.
  • SUM OVER for running totals, LAG for growth, NTILE for tiers.
➡️ Next

Day 35: SQL + pandas project – build a small sales database and answer 8 business questions end to end.

Frequently Asked Questions

Does MySQL support window functions?
Yes, from MySQL 8.0. PostgreSQL, SQL Server, BigQuery and Snowflake all support them.
Can I filter on a window function in WHERE?
No – window functions run after WHERE. Compute them in a CTE/sub-query and filter outside (as in the top-N example).
← Data Science (60 Days)
🔗

Share this topic with a friend

यह topic किसी दोस्त को भेजें

Found it useful? Send it to a classmate learning the same thing.

अच्छा लगा? जो दोस्त यही सीख रहा है, उसे भेज दीजिए।

Next topicDay 35: Project: SQL + Pandas HR Analytics Your progress: 34 / 60 (57%) · Lesson 35 / 60 Continue →

How was this lesson?

Your feedback helps us make the content better.

Write a review or suggestion (optional)

After approval, your name and city may appear publicly with your review. Please do not write your email or phone number.

▶ CodingEasily on YouTube

Learn with video lessons in easy Hindi

Featured video: RDBMS Part 3 – Class XI Commerce IT

▶ Subscribe – it’s free