Below are insights from my open book assignment / exam at IIT GNX converted into a blog-based story with help on AI/GenAI. This was the most exciting open book assignment / exam given by me till now. Open to comments, suggestions, ideas, debates, improvements, corrections, reviews, etc. Feel free to email me (refer contact detail in the bottom of this article) or message me on LinkedIn.
📌 The Real Question Isn’t Prediction — It’s Decision
Most data science projects stop at:
“Model accuracy improved.”
But in real systems—especially ride-hailing, logistics, BFSI, or infra platforms—that’s not enough.
The real question is:
What decision becomes better because of this model?
This assignment pushed me to think differently.
Instead of just predicting traffic, I asked:
How can traffic forecasts drive real operational decisions in a ride-hailing system?
🧠 Problem Framing (What Actually Matters)
We used the METR-LA dataset:
- 207 traffic sensors
- 5-minute interval readings
- ~4 months of data
- Objective: predict traffic speeds 5, 15, 30 minutes ahead
But here’s the twist:
👉 Each sensor is not independent 👉 Roads are connected systems 👉 Congestion spreads like a graph
So instead of treating data as rows in a table…
We treat it as a graph system
🌐 Thinking in Graphs (Systems Thinking)
- Nodes → Traffic sensors
- Edges → Road proximity / connectivity
- Signals → Speed over time
This is where complex systems + spatial thinking come into play.
Traffic ≠ isolated events Traffic = propagating behavior across a network
📊 What the Data Told Us
From exploratory analysis:
- Congestion appears in clusters (not random points)
- Patterns repeat during commute peaks
- Slowdowns are both: Temporal (time-based) Spatial (location-based)
👉 This is critical insight for operations:
- Time tells you when to act
- Space tells you where to act
🤖 Models We Tested (Keep It Honest)
To make this real (not overhyped), we compared:
1. Persistence Model
- “Tomorrow ≈ Today”
- Surprisingly strong for 5-minute prediction
2. Random Forest
- Uses past lag features
- Captures non-linear temporal patterns
3. Graph ML Model (GConvGRU)
- Combines: Graph Convolution → spatial relationships GRU → temporal dynamics
📈 Results (Where Graph ML Actually Matters)
From the results:
Horizon Best Insight (Labels)
5 min Simple models work well
15 min Graph ML starts winning
30 min Graph ML clearly better
👉 Why?
Because:
Short-term = inertia Medium-term = propagation
Graph models capture how congestion spreads, not just how it exists.
🚕 Turning Predictions into Decisions
This is where the project becomes real.
🔴 If congestion is predicted in next 15–30 mins:
- Reduce driver inflow into that corridor
- Increase ETA buffers
- Trigger incentives in nearby zones
🟢 What this enables:
- Better ETA reliability
- Smarter driver utilization
- Reduced customer wait time
- Proactive—not reactive—operations
🧩 The Big Shift: Model → Decision System
This project is NOT just:
“Train model → predict → done”
It is:
EDA → Model → Evaluation → Business Rules → Decision Intelligence
The work is framed as a decision-intelligence exercise rather than only model-building
⚠️ Reality Check (Limitations)
Let’s stay grounded.
The dataset does NOT include:
- Ride demand
- Driver availability
- Weather
- Events
- Airport queues
So:
This is traffic intelligence, not full business optimization
🔧 What I Learned (Real Engineering Insights)
From my own notes:
- Training time is real (hours, not minutes)
- GPU/TPU selection matters
- Early stopping is critical (overfitting is silent killer)
- Graph ML pipelines are non-trivial systems
- LLMs can accelerate development—but thinking is still yours
🏗️ Architecture Thinking (My Take)
What excites me most is not the model.
It’s the system design potential:
Imagine combining this with:
- Real-time driver GPS
- Demand prediction models
- Event/weather APIs
- Reinforcement learning for dispatch
👉 You get:
Autonomous Decision Systems for Urban Mobility
🔮 Where This Connects to My Larger Work
This directly aligns with what I’m exploring:
Agentic AI + Graph Systems + Probabilistic Models for Autonomous Debugging & Decision Systems
Traffic is just one domain.
Same thinking applies to:
- Microservices failures
- Network congestion
- Financial risk propagation
- Supply chain disruptions
🧠 Final Thought
A staff engineer once asked:
“What gets harder after this lands?”
For me, this project answered a deeper question:
What gets smarter after this lands?
📌 Bottom Line
- Graph ML is not just “better ML”
- It is better system understanding
- Real value comes when: Predictions → Decisions Models → Actions Data → Intelligence
📢 Stay informed:
- Website: www.HarwaniSystems.in
- Blog: www.TechAndTrain.com/blog
- LinkedIn: Neil Harwani | LinkedIn
- Email me: Neil@HarwaniSytems.in
#GraphML #DataScience #AI #SpatialDataScience #RideHailing #DecisionIntelligence #SystemsThinking #GNN #MachineLearning #TechLeadership
