Data Engineer II - ETL

SALARY : 15LPA - 20LPA Posting title : Data Engineer II Experience : 6 Years Location : Bangalore Work mode : Hybrid Primary skills : ETL, Data Warehousing, Reporting, SQL, Azure services such as Azure Data Factory, Azure Databricks, Azure Synapse Analytics, Azure Storage, etc. Qualification : Any Engineering/ Computers degree Roles & Responsibilities : - Design, implement, and manage scalable and efficient data architecture on the Azure cloud platform. - Develop and maintain data pipelines for efficient data extraction, transformation, and loading (ETL) processes. - Perform complex data transformations and processing using Azure Data Factory, Azure Databricks, Snowflake's data processing capabilities, or other relevant tools. - Develop and maintain data models within Snowflake and related tools to support reporting, analytics, and business intelligence needs. - Collaborate with cross-functional teams to understand data requirements and design appropriate data integration solutions. - Integrate data from various sources, both internal and external, ensuring data quality and consistency. - Ensure data models are designed for scalability, reusability, and flexibility. - Implement data quality checks, validations, and monitoring processes to ensure data accuracy and integrity across Azure and Snowflake environments. - Adhere to data governance standards and best practices to maintain data security and compliance. - Handling performance optimization in ADF and Snowflake platforms - Collaborate with data scientists, analysts, and business stakeholders to understand data needs and deliver actionable insights. - Provide guidance and mentorship to junior team members to enhance their technical Skills. - Maintain comprehensive documentation for data pipelines, processes, and architecture within both Azure and Snowflake environments including best practices, standards, and procedures. Skills & Knowledge : - Bachelor's or Master's degree in Computer Science, Engineering, or a related field. - 5 Years of experience in Information Technology, designing, developing and executing solutions. - 3 Years of hands-on experience in designing and executing data solutions on Azure cloud platforms as a Data Engineer. - Strong proficiency in Azure services such as Azure Data Factory, Azure Databricks, Azure Synapse Analytics, Azure Storage, etc. - Familiarity with Snowflake data platform would be an added advantage. - Hands-on experience in data modelling, batch and real-time pipelines, using Python, Java or JavaScript and experience working with Restful APIs are required. - Expertise in DevOps and CI/CD implementation. - Hands-on experience with SQL and NoSQL databases. - Hands-on experience in data modelling, implementation, and management of OLTP and OLAP systems. - Experience with data modelling concepts and practices. - Familiarity with data quality, governance, and security best practices. - Knowledge of big data technologies such as Hadoop, Spark, or Kafka. - Familiarity with machine learning concepts and integration of ML pipelines into data workflows - Hands-on experience working in an Agile setting. - Is self-driven, naturally curious, and able to adapt to a fast-paced work environment. - Can articulate, create, and maintain technical and non-technical documentation. - Public cloud certifications are desired (ref:hirist.tech)

Location: Bangalore, IN

Posted Date: 10/25/2024