Kuldeep Singh

Architecting a Data Lakehouse MVP with Delta Lake & Spark to support AI-ready analytics. Modernized legacy systems into microservices architecture, improving scalability and resilience. Implemented AI-powered product search engine Achieved Significant cost savings across solutions. Reduced complex pipeline execution time by ~90%, from 4 hours to 25 minutes

1. Led migration of pipelines from software-centric to modern data engineering stack, enabling scalable and efficient workflows. 2. Streamlined development cycle by decoupling user-facing and ETL components, reducing complexity and accelerating delivery. 3. Reduced data pipeline latency by 65% through technology upgrades and performance optimizations. 4. Designed and implemented project-wide data architecture integrating Databricks, Spark, and Airflow, supporting advanced analytics and AI adoption

1. Built a real-time analytics platform processing millions of daily transactions, enabling faster business insights. 2. Managed and optimized 40+ TB of data in Hive and Scala Spark on GCP clusters with advanced performance tuning. 3. Integrated Druid, HPSQL, and Looker to deliver scalable and interactive analytics solutions. 4. Migrated legacy data systems to Google Cloud Platform, boosting performance. 5. Led technology migration from Hive ETL to Scala Spark ETL, reducing latency and improving reliability

1. Designed terabyte-scale data architecture powering advanced retail analytics. 2. Built Kafka + Spark streaming pipelines to deliver real-time customer insights. 3. Owned the end-to-end data pipeline lifecycle (development, deployment, and support) as a sole developer. 4. Reduced SLA breaches to near zero by introducing data quality gates and best engineering practices. 5. Contributed to the company’s custom Apache Spark framework, improving performance and reusability across teams.

1. Built UI-driven reusable ETL frameworks, enabling UI-first and plugin and feature- flag-based Extract, Transformations, Load, and Visualization within a single project, significantly reducing development time. 2. Deploy machine learning models into production environments. 3. Contributed to the R&D team, developing multiple data engineering concepts and POCs. 4. Delivered diverse end-to-end solutions, including ETL, Data Lakes, Data Warehouses, IoT systems, real-time, and batch processing projects.