Payal Patel

- Automated Azure Data Factory pipelines (hourly, incremental) into ADLS Gen2: .mpt files copied as-is to the raw zone, SQL tables extracted to Parquet. - Implemented Medallion architecture (Bronze/Silver/Gold); delivered a clean, analytics-ready Gold layer used by the testing team as its trusted dataset. - Built Databricks Lakehouse pipelines with Asset Bundles; cut compute costs ~20% via local debugging. - Optimized storage layouts and transformation logic for performance and cost efficiency.

- Developed and maintained ETL workflows to ingest data from REST APIs into SQLite databases, supporting downstream analytics and forecasting use cases. - Integrated database-backed data pipelines with MATLAB-based processing for structured ingestion of weather data from an external provider (DWD). - Collaborated with engineering teams to optimize data extraction, processing performance, and system reliability.

- Designed and implemented data ingestion and transformation pipelines for large-scale time-series data generated by industrial data loggers. - Built data visualization and reporting layers to enable structured analysis and improve data accessibility for engineering teams. - Supported data-driven decision-making by integrating time-based analytics and automated reporting into internal applications.

- Developed a scalable data processing pipeline for a 10 GB NLP dataset, extracting and categorizing noun/verb bigrams for structured analysis. - Designed a reproducible data processing workflow (preprocessing, feature extraction, structured outputs) to support downstream analysis and experimentation. - Applied K-Means clustering with cosine similarity and embeddings to group and analyze semantic patterns, supported by post-processing quality checks.

- Machine Learning: Regression, Classification, Clustering, Feature Engineering - Natural Language Processing (NLP): Text Processing, Semantic Analysis, Vector Representation