Aidan Macdonald

• Build and maintained data lake using Cloud Storage in GCP • Deployed data access API using Cloud Run and Cloud Spanner in GCP • Maintained Kubernetes workflow using Kubeflow • Automating testing and compliance in CI/CD

• Building product data ingestion pipelines in Palantir Foundry • Testing and quality checks using Pyspark/Polars transforms • Optimizing data flows for commerce data audits and improving visibility • Working with external partners to ingest their data and shared data mappings

• Optimize big data pipelines in AWS using Redshift, S3 and Kinesis • Glue data catalog and ETL pipelines for data ingestion • Built IOT data pipelines to monitor physical hardware on site • Performed data physics analysis of IOT collected data • Develop economic and physical models for key components of the businesses

• AWS Glue jobs and metadata store for large data lake • Built and maintain autoscaling SparkSQL ETL platform supporting data science • Event driven ETL triggers AWS EMR autoscaling to minimize compute costs • ETL Platform has been deployed on multiple teams in my organization • Business reports platform built using AWS Stepfunctions, Lambda and Athena • Automated data cleansing using AWS Stepfunctions for Amazon PrimeNow cat- alog • Automatic ingestion into S3 of product data improving data quality metrics

• Built Tensorflow powered webservice on Docker platform for running Machine Learning jobs • Ported TensorFlow to custom in-house HPC hardware • Leveraged RabbitMQ to build job queue connecting webservices to custom hard- ware • Developed Heterogeneous Neural Networks for predicting chaotic phenomena

• Analyzed data using Numpy, Scikit-learn and Matplotlib to malware and phishing detection • Leveraged MPI and OpenMP with BLAS to scale compute in HPC environments • Profiled and experimentally demonstrated success of algorithm