Senior Azure Data Engineer - Synapse Analytics

Job Description : Mandatory Skills : Azure Data engineering, Pyspark, Python, Azure Data Bricks, Azure Data Factory, SQL. Responsibilities : - Design and develop business-critical backend systems using stream processors and high-quality data pipelines. - Assemble large, complex data sets that meet functional / non-functional business requirements. - Build the infrastructure required for optimal extraction, transformation, and loading of data from a wide variety of data sources using SQL and Azure 'big data' technologies. - Build a cloud-native, real-time stream processing & data lake platform that scales into the Zettabytes and beyond. - Build a data pipeline to provide actionable insights into customer acquisition, operational efficiency, and other key business performance metrics. - Ability to automate data pipelines using DevOps tools. - Perform root cause analysis on external and internal processes and data to identify opportunities for improvement. - Build processes that support data transformation, workload management, data structures, dependency, and metadata. - Adopt for change, always open to replacing what you built yesterday with something better today. - Work with all stakeholders to design and code large-scale batch and real-time data pipelines on Azure. - Perform code reviews, and lead by example on code refactoring for readability, extensibility, and testability. Requirements : - A bachelor's degree in computer science or equivalent - 5 years of experience as a Data Engineer, - Experience with Azure big data tools: Azure Databricks, Azure Synapse, HDInsight, ADLS - Experience with relational SQL and NoSQL databases. - Excellent problem solving and analytic skills associated with working on structured and unstructured datasets using Azure bigdata tools. - Experience with data pipeline and workflow management tools: ADF and Logic Apps - Experience with Azure cloud services: VM, Azure Databricks, Azure SQL DB, Azure Synapse - Experience with stream-processing systems: Azure Streams - Experience with scripting languages Python and have strong knowledge in developing Python notebooks. (ref:hirist.tech)

Location: in, IN

Posted Date: 11/25/2024