Senior Google Cloud Platform Engineer - Data Engineering

Experience : 8 - 10 yrs We are seeking a highly skilled Senior Data Engineer with extensive experience in data engineering, particularly in building and maintaining data pipelines. This role requires strong expertise in GCP, Dataflow, Apache Beam SDK, and PySpark, along with a deep understanding of Google Cloud Platform (GCP) services. The ideal candidate will have a proven track record of developing CI/CD pipelines, working with both SQL and NoSQL databases, and managing data migration and processing tasks. Key Responsibilities : - Architect, develop, and maintain robust data pipelines using Java. - Utilize Dataflow, Apache Beam SDK and PySpark to handle large-scale data processing. - Leverage Google Cloud Platform services such as Google Cloud Storage (GCS), BigQuery, Dataflow, Cloud Functions, IAM, and - Cloud Composer DAGs to manage and optimize data pipelines. - Implement security best practices using IAM and manage access controls. - Develop and maintain CI/CD pipelines tailored for data engineering projects. - Automate the deployment and scaling of data pipelines across various environments. - Work with SQL databases (Oracle, PostgreSQL, SQL Server) and NoSQL databases to store, retrieve, and manage data. - Scheduling using Autosys or Control M - Optimize database queries and perform data migrations as needed. - Lead data migration, ensuring seamless data transfer between systems. - Implement data processing technologies and procedures to enhance data quality and accessibility. - Collaborate with cross-functional teams to understand business requirements and translate them into technical solutions. - Excellent problem-solving skills and the ability to work in a fast-paced, dynamic environment. - Strong communication skills, both written and verbal. - In-depth experience with GCP services, data pipeline frameworks, and database management. Why Join Infometry ? At Infometry, we are committed to driving innovation in data engineering and cloud solutions. As part of our team, you will have the opportunity to work on cutting-edge projects, collaborate with industry experts, and contribute to the success of various verticals. (ref:hirist.tech)

Location: in, IN

Posted Date: 11/22/2024