Data Engineer - SQL/Spark/Spark

Job Description : - Proven experience in in SQL, Spark, Hadoop ecosystem - Have worked on multiple TBsof data volume from ingestion to consumption. - Work with business stakeholders to identify and document high impact business problems and potential solutions. - Good understanding of Data Lake/Lakehouse architecture and experience/exposure to Hadoop (cloudera, hortonworks) and/or AWS. - Work on end-to-end data lifecycle from Data Ingestion, Data Transformation and Data Consumption layer. - Versed with API and its usability. - Suitable candidate will also be proficient in Spark, Spark Streaming, hive, SQLs. - Suitable candidate will also demonstrate experience with big data infrastructure inclusive of MapReduce, Hive, HDFS, YARN, HBase, Oozie, etc. - The candidate will additionally demonstrate substantial experience and a deep knowledge of relational databases. - Good skills in technical debugging of the code in case of issues. - Also, working with git for code versioning - Creating Technical Design Documentation of the Responsibilities : - Ability to work independently and handle your own development effort. - Excellent oral and written communication skills Learn and use internally available analytic technologies. - Identify key performance indicators and create educational/deliverables path to achieve the same. - Use educational background in data engineering and perform data mining analysis. - Work with BI analysts/engineers to create prototypes. - Engage in the delivery and presentation of solutions. (ref:hirist.tech)

Location: mumbai, IN

Posted Date: 11/22/2024