Amir Nazary

Led the Airflow 2.11-to-3.2.2 migration on Kubernetes, including cross-account AWS workflows, monitoring, and team onboarding. Set up Argo CD deployments and Kubernetes manifests to migrate managed apps to K8s. Implemented OpenMetadata as the data catalogue and developed Claude MCP integrations for Airflow and OpenMetadata. Maintained data-serving infrastructure including Cube.js, QuickSight, OpenMetadata, REST APIs, and AWS Bedrock models such as Claude.

Developed and maintained the company’s internal data platform using Snowflake, Airflow, and Spark. Migrated Airflow DAGs from self-hosted servers to Astronomer Cloud, enabling the shutdown of on-premise infrastructure and saving thousands of euros per month. Developed and orchestrated ETL pipelines and integrations from Kafka, MySQL, and external APIs into Snowflake using Airflow and dbt. Created Prometheus metrics and Grafana alerts to monitor data assets, and supported data-governance initiatives.

Migrated orchestration from Airflow to Dagster; built the company’s first ClickHouse OLAP setup; and designed REST/GraphQL APIs. Deployed Kubernetes clusters, published production-ready Helm charts, and improved GitLab CI/CD release automation. Built Prometheus/Grafana monitoring for anomalies and data freshness across Kafka, Azure Blob Storage, and databases. Added tests, linters, data checks, Bash scripts, and automated workflows to improve code quality and data governance.

Ingested customer data using AWS services, Kubernetes, and Snowflake to support data-driven decision-making in battery intelligence. Developed Python and Bash automation for deployment and configuration of data-ingestion pipelines. Built ETL pipelines from multiple sources and optimized GitLab CI/CD, reducing execution times by 30%.

Developed a tool to semantically annotate raw IoT data captured from Kafka topics and published research on the approach. Built ETL pipelines to extract and process research data from multiple sources. Developed Python and Bash automation scripts to streamline deployment and configuration of data-ingestion pipelines.