Lucas Bittencourt Xavier

Acting as a freelancer in building data lakes on demand, building data pipelines using spark and other tools like Sqoop in stack AWS.

Definition of Data Lake architecture and construction, acting from data ingestion (Kafka, Sqoop), storage in AWS (S3, emrfs, Redshift), processing (EMR, glue ETL jobs, spark) and visualization (Powerbi, Athena). Construction of data pipelines using tools such as Pentaho, AWS Glue, Apache Spark and Apache NIFI flow tool. Architecture and elaboration of relational data models (Oracle, PostgreSQL and MySQL) in addition to NOSQL (Cassandra and MongoDB).

Architecture and elaboration of relational data models (Oracle, PostgreSQL and MySQL) in addition to NOSQL (Cassandra and MongoDB). Development of PL / SQL routines, query optimization, performance analysis and migration of large data volumes. Focus on optimization and architectural settings (real-time replications, life cycle and masking the data).

I worked with data architecture definition, elaboration of Relational Data Modeling, and NOSQL. Documentation of data models using the Erwin tool. Development of PL / SQL routines, query optimization and migration of large volumes of data.

This course has as main objective to analyze and extract base information with several types of data (numerical, texts among others). As well as presenting various technologies used by a data scientist and Analytics professional to collect and model various types of data to aid decision making.