Senior Databricks Engineer, Data Platform Data Science Tooling
Senior Databricks Engineer, Data Platform Data Science Tooling
Senior Databricks Engineer, Data Platform Data Science Tooling
Senior Databricks Engineer, Data Platform Data Science Tooling
Magna International
Automobil- und Fahrzeugbau
Block
- Art der Anstellung: Vollzeit
- Vor Ort
- Zu den Ersten gehören
Senior Databricks Engineer, Data Platform Data Science Tooling
Über diesen Job
Senior Databricks Engineer, Data Platform & Data Science Tooling
- locations
- Bangalore, IN
- time type
- Full time
- posted on
- Posted 6 Days Ago
- job requisition id
- R00209920
Job descriptions may display in multiple languages based on your language selection.
What we offer:
Group Summary:
Job Responsibilities:
We are looking for a highly experienced and motivated Senior Engineer to join our Enterprise Data Platform team. This role is pivotal in designing, building, and maintaining scalable, secure, and high-performance data infrastructure across Azure, AWS, and Databricks. The ideal candidate will have strong hands-on experience in cloud administration, CI/CD automation, and should be innovative in building ingestion patterns, re-usable components and hyper-automation for platform.
Key Responsibilities:
Platform Engineering & Administration – Must Have
- Administer and optimize Azure and AWS cloud environments, including resource provisioning, access control, monitoring, and cost management.
- Manage and maintain Databricks workspaces, clusters, jobs, notebooks, and user roles.
- Ensure platform security, compliance, and performance through proactive monitoring and automation.
Data Engineering & Development- Must Have
- Design and develop modular, reusable Python-based batch jobs and data pipelines using frameworks like PySpark, pandas, and Delta Lake.
- Build scalable ETL/ELT processes to ingest, transform, and load data from diverse sources into cloud data lakes and warehouses.
- Optimize data workflows for performance, reliability, and maintainability.
DevOps & CI/CD Automation
- Implement and manage CI/CD pipelines using GitHub Actions, Azure DevOps, or similar tools to automate deployment and testing of data solutions. -Must Have.
- Integrate infrastructure-as-code (IaC) practices using tools like Terraform or ARM templates. -Good to have.
- Collaborate with DevOps and platform teams to streamline release cycles and ensure consistent environments.
Collaboration & Governance:
- Work closely with data architects, analysts, and business stakeholders to understand requirements and deliver robust data solutions.
- Promote best practices in data governance, metadata management, and data quality.
- Document technical designs, operational procedures, and troubleshooting guides.
Required Qualifications:
- Bachelor’s or Master’s degree in Computer Science, Engineering, or related field.
- 8-10 years of experience in data engineering or platform engineering roles.
- Proven hands-on expertise in Azure, AWS, and Databricks administration.
- Strong programming skills in Python, with experience in building production-grade data pipelines.
- Deep understanding of CI/CD principles, Git workflows, and automation tools.
- Experience with monitoring, logging, and alerting tools (e.g., Azure Monitor, CloudWatch, Datadog).
Preferred Skills:
- Experience with Terraform, Docker, or Kubernetes.
- Familiarity with data lakehouse architectures, Delta Lake, and Apache Spark.
- Knowledge of data security, compliance frameworks, role-based & attribute based access control.
- Exposure to Agile/Scrum methodologies and tools like JIRA or Confluence.
What We Offer:
- Opportunity to work on cutting-edge cloud-native data platforms.
- Collaborative and inclusive work culture.
- Competitive compensation and benefits.
- Continuous learning and career development support.
Awareness, Unity, Empowerment:
Bewertung von Mitarbeitenden
Gesamtbewertung
Basierend auf 1.632 BewertungenVorteile für Mitarbeitende
Unternehmenskultur
Unternehmenskultur
1632 Mitarbeitende haben abgestimmt: Sie bewerten die Unternehmenskultur bei Magna International als ausgeglichen zwischen traditionell und modern.Dies stimmt ungefähr mit dem Branchen-Durchschnitt überein.