Data Engineer - Azure Databricks

Salary : 14 - 18 LPA Overview : We are seeking a skilled and experienced Data Engineer with a specialization in DataBricks to join our dynamic team. As a Data Engineer, you will play a crucial role in designing, implementing, and maintaining our data infrastructure, with a specific focus on leveraging DataBricks for data processing and analytics. Your expertise in DataBricks will contribute to the efficient and scalable management of our data pipeline and analytics workflows. Responsibilities : 1. Develop and maintain data pipelines and ETL processes using DataBricks to efficiently process, transform, and load large volumes of data from diverse sources. 2. Collaborate with cross-functional teams to understand data requirements and design and implement optimal data solutions using DataBricks, ensuring data quality, reliability, and integrity. 3. Architect, optimize, and manage the DataBricks environment, including clusters, notebooks, and workspaces, to ensure high performance and scalability. 4. Implement and maintain data models, schema designs, and database structures within DataBricks, ensuring they align with best practices and meet business needs. 5. Work closely with data scientists and analysts to provide support in data exploration, analytics, and data visualization using DataBricks. 6. Monitor and troubleshoot data pipelines and DataBricks jobs to identify and resolve issues, ensuring smooth and uninterrupted data processing. 7. Collaborate with the broader data engineering team to design and implement scalable data solutions, leveraging cloud technologies (such as AWS, Azure, or GCP) in conjunction with DataBricks. 8. Stay updated with the latest industry trends and best practices in DataBricks and data engineering, bringing innovative ideas and technologies to enhance our data capabilities (ref:hirist.tech)

Location: in, IN

Posted Date: 11/21/2024