Abdulkader Khouja

Led Spark pipelines processing 5M transactions/day for national and international traders on the exchange market application (~2 TB) with Python & Scala. Designed scalable data models (fact/dimension schemas, bronze–silver–gold layers) to support reliable analytics and downstream systems, and produced Balanced Scorecard reports for management. Optimized SQL queries with partitioning and indexing, improving query performance by 60%. Architected Azure Databricks/Data Lake pipelines with governance and cost op

Built Spark pipelines for finance/procurement data merging SAP, SAP Ariba, and other sources, reducing duplicates by 80% using Apache Spark and PySpark. Implemented batch & streaming ETL with Spark Structured Streaming, Kafka, and Azure Data Factory, structuring data and building scalable models for both transactional and analytical needs. Optimized SQL for large datasets, cutting query runtime from hours to minutes, and enabled Balanced Scorecard reporting with KPIs and executive dashboards. Assisted in Ai

Developed Spark pipelines handling ~500k–1M daily finance and logistics events, enabling timely freight cost analysis, shipment tracking, and financial reconciliations. Tuned SQL queries (joins, window functions, partitioned aggregations), reducing reporting latency by 40% and improving visibility for finance and supply chain managers. Built Airflow workflows for production ETL with monitoring and alerts, and assisted in Docker-based deployment and CI/CD, ensuring reliable data delivery across global teams.