Swapnali Ghumkar

1. Built a PySpark framework to validate ETL accuracy between on-premises SQL Server and Snowflake using column-level comparison, null normalization, and dynamic schema handling. 2. Enhanced processing efficiency by 50% through performance tuning, caching strategies, and parallel processing. 3. Developed a Spark-based pipeline to orchestrate Scala scripts for flattening XML files and generating optimized Parquet files for downstream processing.

1. Designed fault-tolerant batch data pipelines using Databricks to process sensor data for water analytics. 2. Built a PySpark-SQL based data quality framework, improving data reliability and increasing ML model accuracy by 40%. 3. Established CI/CD with Azure DevOps to support water supply estimation and consumption forecasting. 4. Implemented event-driven processing with Azure Functions, automating column name standardization and date format normalization for files ingested from an SFTP server.

1. Developed 40 RESTful APIs to manage and query pharmacy operations data in the Snowflake data warehouse. 2. Implemented a modular, object-oriented Python pagination solution to streamline data retrieval. 3. Integrated a write-through Redis cache, reducing database load by 35% and improving response times by up to 5x. 4. Engaged with stakeholders to define data needs, lead technical walkthroughs, and drive adoption of delivered solutions.

1. Integrated data from 10+ sources into an AWS S3 staging area using Talend to enhance data accessibility and governance. 2. Built Talend ETL processes to standardize and load CSV/JSON master data (~10M records/month). 3. Engineered a Spark-based framework to process ~200GB/day of sales and PMIS data for downstream applications. 4. Optimized complex Amazon Redshift SQL queries by rewriting joins and indexing strategies, boosting performance by 35%.