Big Data Engineer - Java/Python/Scala

Job Description : - Develop and maintain robust data pipelines for ingesting data from various sources such as databases, APIs, and streaming platforms. - Design and implement data transformation and cleaning processes to ensure data quality and consistency. - Develop data models and schemas for efficient data storage and retrieval. - Expertise in working with big data technologies such as Hadoop, Spark, Hive, HBase, Kafka, and cloud platforms (AWS, Azure, GCP). - Experience with NoSQL databases such as Cassandra, MongoDB, and Elasticsearch. - Strong understanding of data warehousing concepts and methodologies. - Collaborate with data scientists and analysts to perform exploratory data analysis and generate insights. - Develop and maintain data dashboards and visualizations using tools like Tableau, Power BI, or similar. - Identify and optimize performance bottlenecks in data pipelines and queries. - Implement strategies for improving data processing speed and efficiency. - Experience with cloud-based big data services such as AWS EMR, Azure Databricks, or GCP Dataflow. - Understanding of cloud infrastructure and security best practices. - Work closely with cross-functional teams (data scientists, engineers, analysts, product managers) to understand business requirements and translate them into technical solutions. - Communicate complex technical concepts effectively to both technical and non-technical audiences. Technical Skills : - Strong proficiency in Java, Python, or Scala. - Expertise in SQL and NoSQL databases. - Experience with big data frameworks such as Hadoop, Spark, Hive, HBase. - Familiarity with cloud platforms (AWS, Azure, GCP) and their big data services. - Experience with data visualization tools (Tableau, Power BI). (ref:hirist.tech)

Location: in, IN

Posted Date: 1/16/2025