Data Engineer - Scala/Spark

Job Title : Data Engineer Duration : 6 Months (Contract) Location : Bangalore, India Timings : Full-Time (As per company timings) Notice Period : Within 15 days or immediate joiners Experience : 5 years of relevant experience About the Role : We are seeking a highly skilled and motivated Data Engineer to join our team for a 6-month contract. In this role, you will play a crucial part in designing, developing, and implementing data-driven solutions that power our business. You will work with cutting-edge technologies and collaborate with cross-functional teams to deliver high-quality data products. Key Responsibilities : - Design and develop robust data ingestion frameworks, real-time processing pipelines, and data transformation frameworks utilizing open-source technologies like Kafka, Apache Spark (SQL, Scala, Java), and Python. - Operationalize open-source data analytics tools for enterprise-wide use, ensuring seamless integration and optimal performance. - Guarantee adherence to stringent data governance policies by implementing or validating data lineage, quality checks, and classification mechanisms. - Adhere to the company's established development lifecycle for efficient development, deployment, and delivery of data solutions. - Conduct in-depth root-cause analysis of data-related issues to identify and resolve problems effectively. Key Skills & Requirements : Strong proficiency in : - Kafka - Apache Spark (SQL, Scala, Java) - Python - Hadoop Platform - Hive - Airflow - Cloud Composer - PySpark Experience with : - Data warehousing and data lake architectures - Agile development methodologies - Cloud platforms (preferably GCP) - Data quality and validation techniques - Data security and privacy best practices - Excellent communication and collaboration skills Preferred Qualifications : - Google Cloud Platform (GCP) certifications (e.g., Professional Data Engineer) - Proven experience in data governance and DevOps practices - Experience with containerization technologies (e.g., Docker, Kubernetes) (ref:hirist.tech)

Location: bangalore, IN

Posted Date: 12/28/2024