Data Engineer - Azure Data Factory/Data Pipeline

Location : Remote Experience : 6 yr. Salary : As per Experience Interview : Video call Job Responsibilities : - Design, develop, and maintain large-scale data pipelines using Azure Data Factory (ADF) to extract, transform, and load data from various sources into Azure Data Lake Storage. - Collaborate with cross-functional teams to gather requirements and design solutions for complex business problems. - Develop high-quality code in Python using PySpark, SQL Server, and SQL Azure to solve real-world problems. - Ensure scalability, reliability, and performance of deployed applications by monitoring their behavior under different loads. Job Requirements : - Strong expertise in Azure Databricks, ADF, and Azure Synapse Analytics Platform. - Proficiency in programming languages such as Python with hands-on experience working on big data processing technologies like Spark. - Experience in optimization methodologies, including use of various optimization libraries (e.g. CPLEX, Gurobi, or tools, etc.). - Strong Python coding skills (including Object-oriented programming concepts), including using ML libraries and basic working knowledge of SQL - Experience working in cloud technologies (preferred MS Azure, GCP nice-to-have. - Familiarity with the theory behind various machine learning concepts - Experience in CRISP-DM project delivery and with Agile projects (like Scrum, and Kanban). - Experience with business requirements gathering, transforming them into technical plan, data processing, feature engineering, model evaluation, hypothesis testing and model deployment. - Experience in working with client-facing role. - Able to apply independent research, gather information, and give recommendations on how to solve problems of diverse scope. (ref:hirist.tech)

Location: in, IN

Posted Date: 11/25/2024