Data Engineer - Scala/Python/Java

Responsibilities : - Data Ingestion & Processing : Design and implement efficient data pipelines using big data frameworks (e.g, Spark) for ingesting, transforming, and processing large datasets. - Programming Powerhouse : Utilize your strong coding skills in Scala (preferred), Python, or Java to develop high-performance and scalable data processing solutions. - SQL Master : Demonstrate expertise in SQL for efficient data querying, manipulation, and analysis from various data sources. - Big Data Landscape : Experience with popular big data frameworks like Spark or similar tools (Flink, Kafka, NoSQL) to manage and process large data volumes. - Large Data Experience : Proven experience working with very large datasets, preferably within the DTC (Direct-to-Consumer) domain or similar large-scale data environments. - Cloud Savvy : Experience with cloud platforms (AWS, GCP, Azure) for deploying and managing big data infrastructure is a valuable asset. - Data Quality Champion : Ensure data quality and consistency throughout the data pipeline, implementing robust data validation and cleansing processes. - Collaboration & Communication : Work effectively in a collaborative team environment, communicating technical concepts clearly to both technical and non-technical stakeholders. Technical Skills & Requirements : - Programming Expertise : Strong coding skills in Scala (preferred), Python, or Java, with a solid understanding of object-oriented programming principles. - Big Data Frameworks : Familiarity with big data frameworks (e.g, Spark) and experience with data ingestion, processing, and analytics tools. - SQL Proficiency : Demonstrated expertise in SQL for querying and manipulating data from relational and non-relational databases. - Large Data Experience : Proven experience working with and processing very large datasets efficiently. - Cloud Platforms : Familiarity with cloud platforms (AWS, GCP, Azure) for data infrastructure management (a plus). - Data Pipelines : Experience designing and implementing scalable and maintainable data pipelines (ref:hirist.tech)

Location: in, IN

Posted Date: 11/23/2024