AWS Data Engineer - ETL/SQL

Key Responsibilities : - Design, develop, and implement scalable data pipelines and data integration solutions on AWS cloud infrastructure (e.g., AWS S3, Redshift, Glue, Lambda, Kinesis, etc.). - Collaborate with data scientists, analysts, and stakeholders to define and optimize data flows. - Build efficient ETL (Extract, Transform, Load) processes to handle large datasets from multiple sources and transform them into actionable insights. - Manage and optimize data storage solutions within AWS, ensuring high availability, reliability, and cost-effectiveness. - Write and optimize SQL queries for data extraction and reporting, as well as develop data models to support business needs. - Work with AWS tools (such as AWS Glue, Redshift, Athena, Lambda, etc.) to ensure seamless data integration and processing. - Monitor, troubleshoot, and resolve performance issues related to data pipelines and infrastructure. - Implement best practices for data security, governance, and compliance on AWS. - Participate in design and architecture discussions for cloud-based data systems and infrastructure. Skills & Qualifications : - Cloud Platform Expertise : Strong experience working with AWS services, particularly Redshift, S3, Glue, Lambda, Kinesis, and Athena. - Programming : Proficiency in Python, Java, or Scala for data engineering tasks. - ETL Development : Hands-on experience building and optimizing ETL pipelines using tools such as AWS Glue, Apache Spark, Apache Airflow, or other similar technologies. - Data Warehousing : Knowledge of data warehousing concepts, and experience in building or maintaining a Redshift cluster or similar data warehousing platforms. - SQL : Strong proficiency in writing efficient and optimized SQL queries for data extraction and transformation. - Big Data Tools : Familiarity with tools like Apache Hadoop, Kafka, and Apache Spark for managing large datasets. - Data Modeling : Experience in designing data models to support reporting and analytical needs. - Version Control : Experience using version control systems like Git to manage and collaborate on code. - Security and Compliance : Knowledge of AWS security practices, IAM roles, data encryption, and other best practices for maintaining data privacy and compliance. (ref:hirist.tech)

Location: bangalore, IN

Posted Date: 1/14/2025