Pyxidia Techlab LLP
Senior Data Engineer - AWS Cloud
Job Location
in, India
Job Description
Job Title : Data Engineer. Experience - 5 yrs. Location : Pune/Bangalore. Responsibilities for Data Engineer : - Create and maintain optimal data pipeline architecture for sourcing data from multiple structured and unstructured data sources. - Assemble large, complex data sets that meet functional / non-functional business requirements. - Build the infrastructure required for optimal extraction, transformation, and loading of data from a wide variety of data sources using SQL and AWS 'big data' technologies. - Build analytics tools that utilize the data pipeline to provide actionable insights into customer acquisition, operational efficiency, and other key business performance metrics. - Perform data cleaning, exploration, and statistical analysis to ensure data quality. - Work alongside a Data Scientist to develop modeling techniques for various product capabilities. Qualifications for Data Engineer : - Advanced working SQL knowledge and experience working with relational databases, query authoring (SQL) as well as working familiarity with a variety of databases. - Hands-on implementation of spark data pipeline and deep understanding of core spark concepts. - Experience building and optimizing 'big data' data pipelines, architectures, and data sets. - Knowledge of sprint and scrum. - 5 Years of Experience in ETL Applications like Apache Airflow, pipelines like Glue, etc. - Strong analytical skills related to working with unstructured datasets. - Build processes supporting data transformation, data structures, metadata, dependency, and workload management. - A successful history of manipulating, processing and extracting value from large disconnected datasets. - Working knowledge of message queuing, stream processing, and highly scalable 'big data' data stores. - Experience with big data tools : Hadoop, Spark, Kafka, is a plus. - Experience with relational SQL and NoSQL databases, including Postgres and Cassandra. - Experience with AWS cloud services : EC2, EMR, RDS, Redshift. - Experience with object-oriented/object function scripting languages : Python, Java, C++, Scala, etc. (ref:hirist.tech)
Location: in, IN
Posted Date: 2/6/2025
Location: in, IN
Posted Date: 2/6/2025
Contact Information
Contact | Human Resources Pyxidia Techlab LLP |
---|