Data Engineering Manager - Spark/Hadoop

Role Description : This is a full-time on-site role for a Data Engineering Manager at Collabera Digital in Pune. The Data Engineering Manager will be responsible for engineering management, team leadership, software development, project management, and integration on a day-to-day basis. Position Details:. Job Title : Data Engineering Manager. Experience:- 10 Years. Location : 4 months WFH after that WFO from Chennai Or Mumbai. Joining : Immediate to 15 days. Job Description : BASIC QUALIFICATIONS : These are the fundamental requirements for the position, and candidates must meet most or all of them. Education : - Bachelor's degree in relevant fields: A degree in computer science, information technology, software engineering, or closely related fields (e., Data Science, Computer Engineering, or Information Systems). - Experience in Data years of hands-on experience: Practical experience is necessary in areas like : - SQL : Proficiency in SQL programming, including writing and debugging stored procedures, functions, and views. - Python & Object-Oriented Programming : Familiarity with Python for scripting and building data pipelines. - Object-oriented programming experience with languages like Java or C++ is also important. - Knowledge of Data Engineering Tools/Frameworks - Experience with modern technologies in data engineering - Snowflake : A cloud-based data warehousing platform. - Redshift : Amazon's cloud data warehouse solution. - Spark : A distributed data processing engine. - Airflow : An open-source tool for orchestrating complex workflows. - Hadoop & Kafka: Big data tools for storage and real-time data streaming. Cloud-Based Analytics Ecosystem : - Cloud experience, particularly with platforms like AWS (Amazon Web Services) and Snowflake. - This suggests the ability to work in cloud environments to process and store data. Understanding of Development Lifecycles : - Knowledge of SDLC (Software Development Life Cycle) and data science development lifecycle (CRISP). - CRISP refers to the Cross-Industry Standard Process for Data Mining, which is a widely used methodology for data science projects. PREFERRED QUALIFICATIONS : - These are additional skills or qualifications that are not mandatory but would make a candidate more competitive. Advanced Education : - An advanced degree (Master's or PhD) in Data Science, Computer Engineering, or similar fields is preferred, but not required. Data Science Technologies : - Experience with platforms that facilitate data science, like Dataiku, AWS SageMaker, or similar tools. - These platforms are used for building, deploying, and managing machine learning models. Machine Learning & AI : - Familiarity with machine learning and AI technologies, especially in how they integrate with data engineering pipelines. - This shows an understanding of how to integrate data engineering with machine learning models or artificial intelligence applications. Containerization and Orchestration : - Experience with Docker (a tool for creating containers for applications) and Kubernetes (a platform for managing containerized applications). - These tools are used for managing scalable applications in modern cloud environments. Remote Team Experience : - Experience working in a distributed remote team environment, which is increasingly common in global or tech-centric organizations. Agile Practices : - Hands-on experience with Agile methodologies (e. Scrum), which is a framework for iterative development in software engineering and data science projects. (ref:hirist.tech)

Location: in, IN

Posted Date: 1/22/2025