Data Engineering Manager - ETL

Industry : General Insurance. Role : Data Engineering Manager. Years of Experience : 12-15 Years. Location : Banglore(koramangala). Work mode : WFO(1st and last saturday working half day). Other location : candidate willing to relocate. Purpose : This role will own the data ingestion, storage, and engineering processes and functions on the data platform. The Data Engineering Manager will ensure the timely delivery, integrity, and security of data for modeling, reporting, and analysis. This role is crucial in leading a team of data engineers, managing the data infrastructure, and supporting the establishment of a new Data Ops model aligned with Chola's Data Strategy roadmap. Key Responsibilities : Responsibilities will include, but are not limited to : Data Ingestion & Engineering : - Create, maintain, and optimize all data ingestion pipelines from selected source systems to the data platform. - Design, build, and maintain batch or real-time data pipelines in production. - Develop ETL/ELT data pipeline processes to extract, transform, and load data from multiple sources. - Automate data workflows such as data ingestion, aggregation, and ETL processing using various techniques and tools. - Ensure data accuracy, integrity, privacy, security, and compliance through quality control procedures. Team Leadership : - Lead a data engineering team responsible for both day-to-day data management (BAU) activities and change projects. - Define and orchestrate tasks and deliverables in support of the Project Manager, acting as Scrum Master as appropriate. - Take ownership of the team's development, hold members accountable for their commitments, remove roadblocks, and mentor and develop team members. Controls & Standards : - Collaborate with Data Engineers and Architects as a senior contributor to the design, build, and management of the Data Platform. - Take direct ownership of controls and standards to ensure all new data requirements are met using the most appropriate engineering practices. Business Stakeholder Engagement : - Set up governance forums with program stakeholders (both business and IT) to define and execute against a platform technology roadmap, business change, and data product build plan. - Engage with multiple stakeholders, including business users, developers, database architects, data analysts, and data scientists to ensure optimal data delivery. Technical Oversight : - Maintain and optimize the data infrastructure required for accurate extraction, transformation, and loading of data from a wide variety of data sources. - Monitor data systems performance and implement optimization strategies. - Oversee data controls to maintain data privacy, security, compliance, and quality. Technical and Qualitative Requirements : Experience : - Extensive experience in AWS-based data engineering, including orchestrating and governing data pipelines, integration, data modeling, and release management. - Proven experience in leading DevOps sprint teams using Scrum, translating business requirements into deliverable user stories, and coordinating development activity. - Hands-on experience with AWS tools and services (e., S3, EC2, Athena, Redshift, Glue, EMR, Lambda, RDS, Kinesis, DynamoDB, QuickSight). - Strong experience in workflow management and orchestration tools (e., Airflow). Experience in building and deploying solutions to AWS Cloud. Technical Skills : - Proficiency in Python, SQL, PySpark, Scala, and Shell Scripting. - Solid experience with Big Data ecosystems (e. , Cloudera/Hortonworks, AWS EMR) and data warehousing solutions (e. , Snowflake). - Experience with streaming technologies (e., Kinesis, Kafka, Spark Streaming) and distributed computing (e., Hive, Hadoop, Presto). - Familiarity with CI/CD technologies (e. , Git, Jenkins, Spinnaker, Ansible). - Good understanding of NoSQL databases (e., DynamoDB, Redis, Cassandra, MongoDB, Neo4j). Soft Skills : - Strong people management skills, including mentoring, coaching, collaboration, and team building. - Excellent analytical, planning, and organizational skills, with an ability to manage competing demands. - Excellent oral and written communication skills, with experience interacting with business and IT individuals at all levels, including executive leadership. Good to Have : - Experience in the Insurance domain. - Working knowledge of data visualization tools like Tableau, Amazon QuickSight, Power BI, or QlikView. (ref:hirist.tech)

Location: bangalore, IN

Posted Date: 11/23/2024