Senior Data Engineering - Big Data Technologies

Job Title : Lead Data Engineer We are seeking a highly skilled Lead Data Engineer with expertise in Big Data technologies to join our dynamic team. As a Lead Data Engineer, you will play a pivotal role in designing, building, and optimizing large-scale data pipelines and systems to drive business insights and innovation. Key Responsibilities : - Design, implement, and manage scalable data architectures using Hadoop ecosystems, Hive, Spark, and Scala. - Lead the development and optimization of data pipelines for real-time and batch processing of large datasets. - Work with Big Data tools and frameworks to manage and process vast amounts of structured and unstructured data. - Implement efficient data storage solutions using TimeScaleDB and other database technologies. - Collaborate with cross-functional teams to gather requirements, define technical strategies, and deliver robust data solutions. - Ensure data quality, security, and governance standards are maintained across the platform. Qualifications : - Proven experience in Scala and Spark for data processing and ETL development. - Strong understanding of Hadoop, Hive, and other Big Data tools. - Experience with TimeScaleDB or other time-series databases. - Ability to lead a team and drive complex projects from concept to completion. - Excellent problem-solving and communication skills. (ref:hirist.tech)

Location: bangalore, IN

Posted Date: 11/22/2024