Big Data Engineer - AWS/Spark/Scala

Hiring For Big Data Engineer (Spark, Scala, AWS) Job description : - Spark, SCALA, AWS OR Spark, python with AWS lambda exposure. - Design, build, optimize and support new and existing data models and ETL processes based on our clients business requirements. - Build, deploy and manage data infrastructure that can adequately handle the needs of a rapidly growing data driven organization. - Coordinate data access and security to enable data scientists and analysts to easily access to data whenever they need too. - Solid hands-on and Solution Architecting experience in Big-Data Technologies (AWS) - Hands on experience in: AWS Dynamo DB, EKS, Kafka, Kinesis, Glue, EMR - Hands-on experience of programming language like Scala with Spark. - Good command and working experience on Hadoop Map Reduce, HDFS, Hive, HBase, and/or No-SQL Databases - Hands on working experience on any of the data engineering analytics platform (Hortonworks Cloudera MapR AWS), AWS preferred - Hands-on experience on Data Ingestion Apache Nifi, Apache Airflow, Sqoop, and Oozie - Hands on working experience of data processing at scale with event driven systems, message queues (Kafka FlinkSpark Streaming) - Hands on working Experience with AWS Services like EMR, Kinesis, S3, CloudFormation, Glue, API Gateway, Lake Foundation Technical skills requirements : - Hands on working Experience with AWS Athena - Data Warehouse exposure on Apache Nifi, Apache Airflow, Kylo - Operationalization of ML models on AWS (e.g. deployment, scheduling, model monitoring etc.) - Feature Engineering Data Processing to be used for Model development - Experience gathering and processing raw data at scale (including writing scripts, web scraping, calling APIs, write SQL queries, etc.) - Experience building data pipelines for structured unstructured, real-time batch, events synchronous asynchronous using MQ, Kafka, Steam processing - Hands-on working experience in analysing source system data and data flows, working with structured and unstructured data - Must be very strong in writing SQL queries - Strengthen the Data engineering team with Big Data solutions - Strong technical, analytical, and problem-solving skills - Strong organizational skills, with the ability to work autonomously as well as in a team-based environment - Pleasant Personality, Strong Communication & Interpersonal Skills. (ref:hirist.tech)

Location: in, IN

Posted Date: 11/23/2024