Vinicius Pittigliani Perego

- Using LLMs with LangChain to create a chatbot application (text-to-sql) to answer basic stakeholder questions - Developing and maintaining ETL pipelines using Spark (Pyspark) in Palantir Foundry. - Common tasks include ingesting data from APIs and external storage systems like Amazon S3. - Creating data applications using Foundry technologies such as Ontology to make data available for stakeholders.

- Developed a web crawler to identify and extract valid domain names of 100,000 Brazilian companies, enriching the platform’s data. - Transforming data with SQL and Python in a Data Warehouse infrastructure using Postgres. - Data-driven development in Cloud environment (GCP). - DevOps infrastructure using Docker and Kubernetes for Logstash pipelines to publish data in Elasticsearch. - Orchestrating data pipelines with Apache Airflow.

- Reduced data processing time by 30\% by migrating legacy Spark RDD code to the more efficient Spark DataFrame API. - Processing heterogeneous data from different retailers in the supply chain using Apache Spark (Scala). - Transforming data in Scala, using the functional paradigm and storing the data in an on-premise cluster with HDFS (Parquet format) and saving it in a RDBMS (SQL-Server).