Job Details

Key Responsibilities:

• Design, build, and optimize scalable data pipelines and frameworks on AWS leveraging EMR, EKS, S3, and related services.
• Develop, deploy, and manage Databricks (DBX) workflows for big data processing, analytics, and machine learning.
• Implement infrastructure as code (IaC) solutions using Terraform for automated provisioning and environment consistency.
• Write efficient, reusable, and optimized code in Scala, Python, and Java for data transformation, streaming, and integration.
• Collaborate with architects, data scientists, and business stakeholders to translate requirements into technical solutions.
• Ensure performance tuning, scalability, and cost optimization across cloud-based data platforms.
• Enforce best practices in security, CI/CD, testing, and DevOps integration for cloud-native applications.

• 7-10 years of hands-on experience in data engineering and cloud frameworks.
• Proven expertise with AWS big data services (EMR, EKS, S3, Lambda, Glue).
• Strong development skills in Scala, Python, and Java.
• Deep knowledge of Databricks (DBX) for batch and streaming workloads.
• Solid experience with Terraform for infrastructure automation.
• Familiarity with CI/CD, containerization (Docker, Kubernetes), and Agile methodologies.

• Exposure to machine learning workflows on Databricks.
• Knowledge of Airflow or other orchestration tools.
• Experience with data lakehouse architectures.