Data Engineer - SQL/PySpark/Hadoop

Job Description : We are looking for candidates who can join immediately. InnotatzIT Solutions is a global Software Development, IT Product Design and IT Consulting Company predominantly focused on Web Application Development, Digital Innovations & IT Recruitment Solutions. We also provide global digital consultancy that helps brands leverage design, analytics, and engineering to create meaningful, personalized customer experiences. We are looking for Data Engineer with 4 years of experience for one of our client based in Hyderabad. This is a permanent remote role. Responsibilities : - Build and maintain robust, scalable data pipeline architectures to collect, connect, and curate data from various sources. - Assemble large, complex data sets that meet functional and non-functional business requirements. - Identify, design, and implement internal process improvements, including automating manual processes, optimizing data delivery, and re-designing infrastructure for greater scalability. - Work closely with stakeholders to address data-related technical issues and support their data infrastructure needs. - Develop analytics tools that utilize the data pipeline, providing actionable insights into customer acquisition, operational efficiency, and key business performance metrics. - Ensure data quality and integrity by applying best practices in data management and governance. - Collaborate with data scientists, business analysts, and other technological stakeholders to understand and meet data needs. Qualifications : - Bachelor's or master's degree in computer science, Statistics, Informatics, Information Systems, or a related quantitative field. - Proven experience as a Data Engineer or in a similar role. - Advanced knowledge of SQL and experience with both relational SQL and NoSQL databases. - Demonstrated ability building and optimizing 'big data' data pipelines, architectures, and datasets. - Experience conducting root cause analysis on internal and external data and processes to answer specific business questions and identify improvement opportunities. - Strong analytical skills with experience working with unstructured data sets. - Experience with big data tools (Hadoop, Spark, Kafka, etc.) and data pipeline/workflow management tools (Azkaban, Luigi, Airflow, etc. - Knowledge of AWS cloud services (EC2, EMR, RDS, Redshift) and stream-processing systems (Storm, Spark-Streaming, etc. Preferred Skills : - Familiarity with machine learning algorithms and data modeling. - Proficiency in Python, Java, or Scala. - Excellent project management and organizational abilities. (ref:hirist.tech)

Location: in, IN

Posted Date: 11/26/2024