Senior Data Engineer - Big Data Tools

Job Description : - Create and maintain optimal data pipeline architecture - Identify, design, and implement internal process improvements : automating manual processes, optimizing data delivery, re-designing infrastructure for greater scalability, etc. - Build the infrastructure required for extraction, transformation, and loading of data from a wide variety of data sources using SQL and GCP 'big data' technologies. - Build analytics tools that utilize the data pipeline to provide actionable insights into customer acquisition, operational efficiency and other key business performance metrics. - Work with data and analytics experts to strive for greater functionality in our data systems. - Assemble large, complex data sets that meet functional / non-functional business requirements. - Evaluate feasibility and make recommendations, considering things such as customer requirements, time limitations, system limitations. - Serve as a mentor to junior staff by conducting technical training sessions and reviewing project outputs - Build documentation repository for knowledge transfer and developing expertise in multiple areas. - Provide operational support on complex/escalated issues to diagnose and resolve incidents in production data pipelines. Job Qualifications : - BS Degree or equivalent work experience in a software engineering discipline - Typically has 8-10 year's experience in an applicable software development environment - Advanced working SQL knowledge and experience working with relational databases, query authoring (SQL) as well as working familiarity with a variety of databases (SQL, Postgres) - Experience with diverse coding, profiling, and visualization approaches including authoring SQL queries, BigQuery, Python, Google Cloud or equivalent - Experience performing root cause analysis on internal and external data and processes to answer specific business questions and identify opportunities for improvement. - Strong analytic skills related to working with unstructured datasets. - Hands on experience with Cloud Platforms (AWS, GCP, or Azure) - Experience in designing and implementing large-scale event-driven architectures - Understanding of data warehousing and data modeling techniques - Understanding of Big Data, Cloud, Machine Learning approaches and concepts (preferred) - Experience working as a member of a distributed team. - Ability to organize and coordinate with stakeholders across multiple functions and geographic locations. - Ability to develop and write technical specifications - Coaching and teaching skills to mentor less experienced team members - Excellent analytical and problem management skills - Good interpersonal skills and positive attitude - Experience with the following tools and technologies : - Elastic Search, Kafka - Google Cloud Airflow (preferred) - Python, Java, C++, Big Query (ref:hirist.tech)

Location: noida, IN

Posted Date: 12/21/2024