Skip to content

Aakash Madabhushi

Data Engineering

I build the pipelines that feed dashboards and models, and the checks that keep bad data from reaching either.

A year and a half at Sapaad building production pipelines on Databricks and PySpark: more than five million rows a day across a multi-tenant SaaS platform, feeding recommendation features and over 75 tenant dashboards. That included tuning ten or more Spark jobs for a 20% cut in dashboard query latency, and a Delta Lake data quality framework with schema validation and anomaly detection, because those pipelines fed production ML.

The same discipline runs through my own work: an Airflow, dbt, and Snowflake medallion pipeline that reduced 27.1 million Zeek network flows into a graph a model could train on; a taxi platform built twice, once as a batch warehouse and once as a streaming pipeline with a dead-letter topic; and pre-training quality gates in my research that refuse to hand a corrupted tensor to a training run.

5M+
rows a day through multi-tenant ETL at Sapaad
75+
tenant dashboards served by the Spark jobs I tuned
20%
dashboard query latency reduction from window function and caching work
27.1M
Zeek network flows aggregated by the capstone pipeline

8 projects in this area

Every link goes to code or data you can inspect yourself.

Two charts from the report: a bar chart ranking which features predict delay, and a pie chart showing the top five predictors against all other factors.
Data EngineeringML Engineering

U.S. Flight Delays: Batch Analytics to Real-Time Prediction

Trained a delay predictor on 9.5 million U.S. flights, then ran it inside a live stream so each incoming flight gets a delay probability within seconds. Where a flight lands and when it leaves drove most of the signal.

9.5M
flight records processed, 15 monthly BTS files totalling 3.1 GB
0.711
F1 for the random forest on a stratified 20% test split, 79.1% accuracy
  • Python
  • PySpark
  • Spark MLlib
  • Spark SQL
  • Kafka
  • +3 more

SJSU big data course, team project · Dec 2025

Data EngineeringML Engineering

Live Wikipedia Edit Sentiment

Streams every English Wikipedia edit as it happens and scores the tone of each edit summary, so you can spot pages turning contentious in real time.

  • Python
  • Kafka
  • RoBERTa
  • Hugging Face

Personal project · Nov 2025

System architecture diagram of the stock analytics pipeline, tracing ingestion through Airflow and Snowflake to dbt models and the BI layer.
Data Engineering

Stock Market Analytics Pipeline

Pulls daily stock prices into a warehouse, computes the indicators traders actually use, and keeps a full history of every revision to the data.

  • Airflow
  • dbt
  • Snowflake
  • Docker

Personal project · Oct to Nov 2025

Data EngineeringAI Engineering

Semantic Article Search Pipeline

A scheduled pipeline that converts articles into vectors and makes them searchable by meaning rather than exact keywords.

  • Airflow
  • Sentence Transformers
  • Pinecone
  • Docker

Personal project · Nov 2025

Software EngineeringData Engineering

Event-Driven Yelp Clone

A Yelp-style application rebuilt so services communicate through events instead of calling each other directly, letting each part fail and recover independently.

  • JavaScript
  • Kafka
  • Docker

SJSU DATA 236, Distributed Systems · Spring 2026

Benchmark chart comparing single-instance against multi-replica deployment at 500 concurrent threads, plotting response time beside requests served.
Software EngineeringAI EngineeringData Engineering

LinkedIn Simulation with an AI Recruiting Assistant

A LinkedIn-style hiring platform running as 17 coordinated services, plus an AI assistant that takes a recruiter from résumé to outreach and pauses for human approval before it sends anything.

17
containers across 5 service groups
faster repeat profile lookups, under 100 concurrent users
  • FastAPI
  • Kafka
  • LangGraph
  • React
  • MySQL
  • +3 more

SJSU DATA 236, Distributed Systems · Spring 2026

ML EngineeringAI EngineeringData Engineering

Explainable Network Threat Detection

Condensed 27 million raw network connection logs into a map of which machines talked to each other, flagged the suspicious pairs with a graph neural network, and had a language model explain every alert in plain English.

27.1M
Zeek network flows aggregated into the graph
0.90
F1 on 329 held-out host-pair links, 0.94 precision and 0.87 recall
  • PyTorch Geometric
  • Airflow
  • dbt
  • Snowflake
  • Pinecone
  • +1 more

SJSU capstone, team project · Feb to May 2026

Tools I've shipped with here

Pipelines & orchestration

  • Airflow
  • dbt
  • Kafka
  • Docker

Warehouses & processing

  • Snowflake
  • Databricks
  • PySpark
  • Spark SQL
  • Delta Lake
  • Parquet
  • SQL
  • MySQL

Quality & modeling

  • Medallion architecture
  • Schema validation
  • Anomaly detection
  • Dead-letter queues

Relevant experience

The parts of my roles that bear on this work. The résumé has all of it.

Graduate Research Assistant · SJSU Department of Applied Data Science

Mar 2026 to present

  • Built an end-to-end LeRobot v3 data pipeline for SO-101 pick-and-place imitation learning, synchronizing 6-DoF joint state and action trajectories with dual 640x480 RGB streams at 30 FPS, and published 50 teleoperated episodes (49,633 frames) to Hugging Face.
  • Developed pre-training quality gates over Parquet metadata and H.264 video, validating episode and frame alignment, video decodability and resolution, and NaN-free action and proprioceptive tensors.

Software Engineer · Sapaad

Jun 2023 to Oct 2024

  • Built production data pipelines processing 5M+ daily rows across a multi-tenant SaaS platform using Databricks and PySpark for recommendation model feature extraction.
  • Optimized 10+ Spark jobs through SQL window function tuning and caching, reducing dashboard query latency by 20% across 75+ tenant instances.
  • Implemented a Delta Lake data quality framework with automated schema validation and anomaly detection for production ML.

Research Assistant · Illinois Institute of Technology

Sep 2019 to Mar 2020

  • Automated tweet collection through the Tweepy API and generated FastText embeddings to identify water-related disaster events.

Hiring for data engineering?

I'm available from December 2026 and open to relocating. Email me and I'll reply within a day.

Hiring for something else?