Data Engineer(Databricks Developer)

Core Responsibilities

Data Pipeline Development

• Design, build, and maintain scalable ETL/ELT pipelines using Databricks and Spark

Spark Optimization

• Tune and optimize Spark jobs and transformations for performance and scalability

Lakehouse Architecture

• Implement data reliability, governance, and security using Delta Lake and Unity Catalog

Workflow Orchestration

• Automate data workflows using Databricks Workflows or orchestration tools like Apache Airflow

Collaboration

• Work closely with data analysts, engineers, and business stakeholders

Key Skills & Technologies

Languages

• PySpark

• SQL

Databricks Ecosystem

• Delta Lake

• Structured Streaming

• Delta Live Tables (DLT)

• Databricks Notebooks

• Databricks Repos

Cloud Platforms

• AWS / Azure / GCP

DevOps & Engineering Practices

• Git

• Terraform

• CI/CD pipeline

Pay: $3,992.62 - $9,531.76 per month

Work Location: In person