Dmitry Ershov

Our main focus was to assist the customer with a migration from an existing data warehouse solution to a new Apache-based on-premise infrastructure. Achievements: Increase certain Spark jobs performance by 20-30% Migrated projects to Spark 3.1.0 Introduced new metrics for more comprehensive data quality analysis Technologies: Scala, Spark, Hadoop, Hive, Oracle, Teradata, Azkaban, Splunk, Kubernetes

We had a number of customers whose main demand was to structure their text data and speed up the processes based on this data. Thus our main goal was to build a performant, reliable application for various text-based algorithms. Achievements: Optimised performance of certain graph queries up to 8 times Migrated from Spark 2.3 to Spark 3.0 that enabled new features and improvements The new major version of our software was released and the contribution was recognized by customer’s executive management

Our main task was to measure the quality of huge amounts of customer data stored in BigQuery and send alerts. Achievements: The Data Audit/ Data Quality system was integrated with customer’s Cloud Infrastructure enabling new functionality for proactive checks and sufficient metrics gathering Optimised performance of certain Spark Queries (measurements) by 2-5 times Reported revenue increased and the contribution was acknowledged by executive management

Developing ETL pipelines to shape the customer infrasture into a Data Lake which gathers data from different bank systems, such as clients, fraud monitoring, loans, etc. Achievements: Automation and speed up(3 times) the project build phase. Technologies: Java, Spark, Hive, Impala, Oozie, Hadoop

Development and maintenance of the huge Telecom provider’s diagnose system for customer support. Achievements: The blind user support was significantly increased providing people with disabilities with an opportunity to interact with our system Extended the system with the support of glass fiber networks Technologies: Java, Groovy, SOAP