Databricks Engineer - ETL/Python

Role : Databricks Engineer Experience Preferred : 8 Years Location : Remote Job Description : - Design, build, and optimize scalable ETL/ELT pipelines on Databricks using PySpark, Scala, or SQL. - Collaborate with data architects to implement data lakehouses and other modern data infrastructure on Databricks. - Ensure optimal performance of Databricks jobs by fine-tuning Spark queries, managing clusters, and optimizing storage. - Work closely with data scientists, analysts, and business stakeholders to define requirements and deliver solutions. - Assist in deploying and operationalizing machine learning models on Databricks. - Implement data security, quality, and governance best practices within the Databricks environment. - Monitor Databricks jobs and troubleshoot issues to ensure reliability and accuracy. - Maintain detailed documentation of workflows, pipelines, and configurations. Qualifications : - Education : Bachelor's or Master's degree in Computer Science, Data Science, Engineering, or a related field. Experience : - 3 years of hands-on experience with Databricks or Spark. - Strong expertise in data engineering, distributed systems, or big data technologies. Skills : - Proficiency in PySpark, Scala, or SQL. - Experience with cloud platforms (i.e., AWS, Azure, GCP) and cloud-based storage solutions. - Knowledge of data lake and data warehouse concepts. - Familiarity with tools such as Delta Lake, MLflow, and Apache Airflow (ref:hirist.tech)

Location: in, IN

Posted Date: 3/11/2025