Principal/Senior Data Engineer - Google Cloud Platform

Location : Remote Position Duration : 12 Months with possible extension Employment Type : C2C Client : Zendesk (They will be working in USA timings EST zone till 2:00 PM, as per India timings working hours will be 2:30 PM to 11:30 PM IST.) Location preference : PAN India Skills Needed : - Data transformation (DBT) - Data curation Job Description : A data curation engineer migrating from GCP to Snowflake with expertise in dbt, GCP, and Snowflake needs strong skills in cloud migration, specifically deprecating GCP services and mapping data to Snowflake. They must be proficient in dbt for building and maintaining transformation models, ensuring data quality and traceability within Snowflake. Additionally, they should excel in automating and optimizing ETL/ELT pipelines to ensure smooth data movement and efficient performance during and after migration. Key Responsibilities : - Lead the full migration process of data from Google Cloud Platform (GCP) to Snowflake, ensuring minimal downtime and data integrity. - Develop, maintain, and optimize dbt models to transform raw data into structured, actionable insights in Snowflake. - Design and implement automated ETL/ELT pipelines using tools such as Apache Airflow, GCP Dataflow, or similar, for smooth data transfer and transformation. - Collaborate with data architects and cloud engineers to define the migration strategy, mapping existing GCP services (e.g., BigQuery, Cloud Storage) to Snowflake's architecture. - Manage the deprecation of GCP infrastructure, ensuring data pipelines are re-architected for Snowflake compatibility. - Build and maintain data workflows and transformations in dbt, ensuring data quality through comprehensive testing, version control, and documentation. - Monitor, troubleshoot, and optimize data pipelines and Snowflake performance, ensuring efficient data processing and scalability. - Ensure compliance with data governance policies, including data lineage tracking, metadata management, and adherence to data privacy regulations (e.g., GDPR, HIPAA). - Collaborate with cross-functional teams, including data scientists, analysts, and business users, to ensure curated data meets stakeholder requirements. - Provide technical mentorship to junior team members and contribute to continuous improvement in data engineering processes. Required Qualifications : - 5 years of experience in data engineering, with a strong focus on GCP, Snowflake, and dbt. - Proven expertise in cloud migration projects, particularly in moving data from GCP to Snowflake. - Extensive experience with dbt for data transformation, automation, and testing. - Strong understanding of ETL/ELT pipeline development using tools like Apache Airflow, GCP Dataflow, or similar technologies. - Proficiency in SQL and experience with cloud-based databases such as BigQuery and Snowflake. - Experience with data modeling, performance tuning, and pipeline optimization in Snowflake. - Solid knowledge of data governance, metadata management, and data privacy regulations. - Excellent problem-solving skills and the ability to troubleshoot and resolve pipeline or performance issues. - Strong collaboration and communication skills, with experience working in cross-functional teams. - Ability to manage multiple priorities in a fast-paced environment. Preferred Qualifications : - Experience with Snowpipe or other Snowflake-specific tools for data ingestion and transformation. - Familiarity with version control systems (e.g., Git) and CI/CD processes in a data engineering context. - Knowledge of Python, Bash, or other scripting languages for automation. - Experience with big data tools and frameworks (e.g., Hadoop, Spark) is a plus. (ref:hirist.tech)

Location: in, IN

Posted Date: 11/21/2024