Senior Consultant - PySpark/Azure Databricks

Job Description : Databricks and PySpark Senior Consultant- Role : Senior Consultant - Databricks and PySpark Experience : 6 years Location : Pan-India (On-site) Key Responsibilities : - Design, develop, and optimize large-scale data processing pipelines using Databricks and PySpark. - Implement advanced data transformation, integration, and analysis solutions with a focus on performance and scalability. - Collaborate with cross-functional teams to understand data requirements and deliver efficient solutions. - Write efficient SQL queries for data retrieval, manipulation, and analysis. - Leverage cloud platforms such as AWS, GCP, or other Big Data ecosystems to deploy and manage solutions. - Design and manage data workflows and ensure proper orchestration for end-to-end data processing. - Troubleshoot and resolve issues related to data pipelines, scalability, and performance. Must-Have Skills : - Proven expertise in - Databricks- and - PySpark- for developing data solutions. - Strong knowledge and hands-on experience with - SQL- for querying and managing data. - Experience working with cloud platforms such as - AWS- , - GCP- , or - Azure- . - Familiarity with Big Data ecosystems, including - Hadoop- , and their integration with Databricks. Good-to-Have Skills : - Knowledge of cloud-native tools and services for data processing and storage. - Exposure to modern data engineering workflows, including CI/CD pipelines. - Strong understanding of distributed computing principles and best practices. - Experience in troubleshooting, performance tuning, and optimizing data processes. Soft Skills : - Excellent problem-solving and analytical skills. - Strong communication skills to interact with stakeholders and team members. - Ability to work in a fast-paced, dynamic environment with minimal supervision. (ref:hirist.tech)

Location: in, IN

Posted Date: 11/27/2024