sirisha chennam
Angestellt, Data Engineer, Tiger Analytics
Über mich
I have Around 5 years of experience in IT industry with extensive experience in Hadoop stack,big data technologies, Spark, Azure, Python, SQL Server 2012, SSIS and Power BI. More than 1 year of hands on experience using Pyspark. I have great knowledge on Azure synapse, Databricks, Devops, Blobs, Gen 2 datalakes, datafactory and Azure devops. I have done Scraping of wappalyzer , glassdoor websites.Tuned HIVE scripts by understanding the joins, group and aggregation between them. Extensively worked on HiveQL, join operations, writing custom UDF’s and having good experience in optimizing Hive Queries. Used various cloud storages like Cosmos and AWS S3. Mastered in using different columnar file formats like RCFile, ORC and Parquet formats. Got good experience in writing python scripting for Airflow DAGs. Good experience in using scheduling tools like Airflow and ETL tools like SSIS. Hands on experience in SQL databases like SQL server 2012. Very good understanding in AGILE process.
Werdegang
Berufserfahrung von sirisha chennam
Bis heute 3 Jahre und 2 Monate, seit Aug. 2021
Data Engineer
Tiger Analytics
Here we have used Azure functions to create the http triggers, timer triggers for scheduling the scripts. Azure logs to track the web usage. I have developed python scripts, to analyze the logs. how users are using spire website (logins, new users) and for what they are searching. After gathering all this data, we process it to visualize it on the analytics page of spire website. In Aflatoxin, we are automating all the manual process to auto fill the SQL tables. we created the python scripts in data bricks.
9 Monate, Dez. 2020 - Aug. 2021
Data Engineer
Tiger Analytics
Developed python scripts to fetch the data through API’s, web-scraping. Cleansed all the messy nested Json data, converted some of the data from unstructured format to structured format. We have used S3 as the storage space to our data. Developed SQL scripts to handle the data and to create the views. Used ReactJS to create the website for Gridiron exclusively. This website is designed for clients to access the backend data easily. According to their specific search it gives them the backend data.
Developed a power shell script to transfer data from linux server to AWS S3. Built data pipelines using Jenkins tools. Extensively used collibra as a metadata repository. Worked on Airflow DAGs to schedule jobs. Developed python scripts to make custom Airflow DAGs. Developed a Pyspark script to pre-process json data. Power BI for data visualization. Experience in writing Spark applications to test data reliability between aws s3 and hive table data load, once after loaded data from s3 to hive.
2 Jahre und 1 Monat, Dez. 2016 - Dez. 2018
hadoop and ETL developer
Tata Consultancy Services Ltd
Worked for Banking data analysis. Extracted data from Cosmos cloud storage in csv format Used different sqoop scripts to transfer incremental data from RDBMS to HDFS. Involved in converting SQL queries into Hive/Spark SQL quieries. Used different tools for data integration with different databases and Hadoop. Played a major role in converting a traditional RDBMS project into Hadoop project. Developed dashboard using Power BI.
Sprachen
Englisch
Fließend