Nuviso Networks - Data Engineer - Azure Data Services

1. Data Engineering & Development : - Design, develop, and maintain scalable data pipelines using Azure Databricks. - Implement ETL processes to ingest and transform data from various sources. 2. Data Management : - Develop and maintain Delta Lake tables for efficient data storage and retrieval. - Ensure data quality and integrity through rigorous testing and validation processes. - Manage and optimize data workflows in Databricks for performance and cost-efficiency. 3. Collaboration & Communication : - Work closely with data scientists, analysts, and business stakeholders to understand data requirements and deliver solutions. - Participate in design and code reviews to maintain high development standards. - Document data processes, pipeline designs, and best practices. 4. Azure Integration : - Integrate Databricks workflows with other Azure services such as Azure Data Lake Storage, Azure SQL Database, and Azure Data Factory. - Monitor and troubleshoot data pipelines, ensuring reliable and consistent data availability. 5. Continuous Improvement : - Stay up-to-date with the latest developments in Azure Databricks . - Identify opportunities to improve data processing efficiency and reduce operational costs. - Implement best practices for data engineering and contribute to the team's knowledge base. Required Skills and Qualifications : Technical Skills : - 5 years of experience in data engineering or a related field. - Extensive experience with Azure Databricks . - Strong proficiency in SQL for data manipulation and querying. - Experience with Delta Lake and data versioning. - Knowledge of Azure Data Lake Storage, Azure Data Factory, and other Azure services. - Proficiency in programming languages such as Python or Scala. Professional Skills : - Excellent problem-solving and analytical skills. - Strong communication and collaboration skills. - Ability to work in a fast-paced, dynamic environment. Preferred Qualifications : - Experience with Databricks MLflow for machine learning workflows. - Knowledge of data warehousing concepts and solutions such as Snowflake or Azure Synapse Analytics. - Familiarity with CI/CD pipelines and version control systems like Git. - Experience in Agile development methodologies. (ref:hirist.tech)

Location: in, IN

Posted Date: 11/25/2024