Google Cloud Platform & BigQuery Developer - Data Pipeline

Key Responsibilities : GCP & BigQuery Development : - Design, implement, and optimize data pipelines and solutions in Google Cloud Platform (GCP), primarily focusing on BigQuery. - Develop and maintain BigQuery data models and queries, ensuring performance optimization and cost management. - Leverage BigQuery for large-scale data analysis, aggregations, and reporting. Data Pipeline Development : - Build and maintain data pipelines using Python, ensuring seamless data flow and transformations between systems. - Utilize Apache Airflow or Google Composer for orchestration and automation of data workflows. SQL Expertise : - Write complex SQL queries to perform various data manipulations, transformations, and aggregations. - Optimize and tune SQL queries for performance in BigQuery. Collaboration & Documentation : - Collaborate with cross-functional teams to understand data requirements and deliver high-quality data solutions. - Ensure detailed documentation of work, including data pipeline design, query optimization strategies, and process workflows. Solution Design & Architecture : - Design scalable and efficient GCP solutions for data storage, transformation, and analytics. - Work closely with architects and business teams to translate requirements into technical specifications and design robust data systems. Continuous Learning & Best Practices : - Stay up to date with the latest GCP technologies and best practices, contributing to the continuous improvement of the data engineering processes. - Follow industry best practices for data security, data governance, and performance optimization in cloud environments. Required Skills & Experience : Experience : - Minimum 5 years of hands-on experience working with Google Cloud Platform (GCP) and BigQuery. Data Pipeline Expertise : - Strong proficiency in Python for building and managing data pipelines, with a focus on data processing and automation. SQL Skills : - Expertise in writing complex SQL queries for data manipulation, aggregation, and analysis. Cloud Orchestration : - Experience with Apache Airflow or Google Composer for orchestrating and automating data workflows. GCP Certifications (Good to Have) : - GCP Solution Architect certification is highly preferred. Collaboration & Communication : - Ability to work collaboratively with team members and stakeholders from diverse backgrounds and skill sets. Attention to Detail & Documentation : - Strong attention to detail, ensuring all work is well-documented and follows best practices. Preferred Qualifications : Certifications : - GCP Solution Architect - Certified is highly preferred. Other Skills : - Familiarity with other GCP services such as Cloud Storage, Cloud Functions, Dataflow, and Pub/Sub is a plus (ref:hirist.tech)

Location: bangalore, IN

Posted Date: 12/28/2024