AppSierra Solutions - Data Engineer - AWS/Python

Description : We are seeking a skilled Data Engineer with expertise in AWS, Python, and pyspark to join our team. As a Data Engineer, you will be responsible for developing, testing, and maintaining scalable data pipelines and systems. This role is crucial for ensuring that data is collected, stored, and transformed efficiently and accurately to meet the needs of our organization. Responsibilities : - Develop, test, and maintain data pipelines using AWS services, Python, and pyspark. - Collaborate with data scientists and stakeholders to understand data requirements and determine the best approach for data collection, storage, and transformation. - Implement data quality and data governance processes to ensure the accuracy, completeness, and consistency of data. - Optimize and tune data pipelines for performance and scalability, considering factors such as data volume, velocity, and variety. - Monitor and troubleshoot data pipelines and systems, identifying and resolving issues in a timely manner. - Document data pipelines, systems, and operational procedures to facilitate knowledge sharing and ensure maintainability. - Stay updated with the latest trends and technologies in data engineering, identifying opportunities for innovation and improvement. Requirements : - Bachelor's degree in computer science, information technology, or a related field. - Proven experience as a Data Engineer, with expertise in AWS, Python, and pyspark. - Strong knowledge of data engineering principles and best practices, including data modeling, data integration, and ETL processes. - Proficiency in programming languages such as Python, with experience in pyspark. - Familiarity with cloud-based data platforms and services, particularly AWS S3, Glue, EMR, Athena, and Redshift. - Experience with SQL and NoSQL databases, as well as data warehouse concepts. - Strong problem-solving skills and the ability to analyze complex data challenges. - Excellent communication and collaboration skills, with the ability to work effectively in a team environment. - Strong attention to detail and the ability to prioritize and multitask in a fast-paced environment. - AWS certification in data engineering or related fields is a plus. - Experience with big data technologies such as Hadoop and Spark is a plus. - Knowledge of machine learning and data analytics is a plus. (ref:hirist.tech)

Location: hyderabad, IN

Posted Date: 11/22/2024