Data Platform Engineer

Job Description : Data Platform Engineer Job Title : Data Platform Engineer Location : India-based - Noida preferred, Chennai or Bangalore optional Job Type : Full-time Band : E2 Job Summary : We are seeking a skilled and motivated Data Platform Engineer to join our team. The role focuses on DB integration, data pre-processing, data pipeline creation, and supporting AI engineering efforts. As a Data Platform Engineer, you will work with diverse data sources and platforms, ensuring robust, secure, and scalable data infrastructure and pipelines for analytics, reporting, and AI solutions. Key Responsibilities : Database Integration : - Integrate various database systems to support data collection, storage, and retrieval processes. - Optimize database architecture for performance and scalability. Data Pre-Processing & Pipeline Creation : - Develop, maintain, and optimize ETL processes to extract, transform, and load data efficiently. - Build and manage data pipelines for AI and analytics solutions. Data Lake Management : - Design, implement, and maintain data lakes on Azure, AWS, GCP, or other cloud/on-premise environments. Cloud Services Management : - Deploy and manage cloud-native services for seamless data integration and processing. - Leverage cloud platforms to create scalable, reliable, and cost-effective data solutions. Data Security and Monitoring : - Ensure compliance with security standards and implement robust data protection measures. - Monitor system performance and ensure high availability and reliability of data platforms. Data Analytics Support : - Enable data-driven decision-making by providing clean, processed, and well-structured data for analytics teams. - Support advanced analytics and machine learning pipelines with appropriate data infrastructure. API Development : - Design and develop APIs for secure and efficient data access and exchange. Required Skills & Qualifications : 1. Technical Expertise : - Experience in Big Data Engineering with a focus on data lakes (Azure, AWS, GCP, cloud-native, and on-premise setups). - Strong understanding of ETL processes and pipeline creation. - Proficiency in API development for data integration and interaction. 2. Cloud and Data Engineering : - Hands-on experience with cloud platforms and services. - Expertise in designing and managing secure and scalable cloud-based solutions. 3. Data Analytics and Integration : - Proficient in integrating structured and unstructured data from multiple sources. - Familiarity with - data security- measures and system monitoring practices. 4. Problem Solving and Collaboration : - Strong analytical skills and the ability to troubleshoot complex data engineering issues. - Experience in collaborating with AI engineers, data scientists, and stakeholders. 5. Other Skills : - Experience with system monitoring tools and ensuring high availability. - Familiarity with modern programming languages and frameworks for data engineering. Preferred Qualifications : - Certifications in cloud platforms (e.g., AWS Certified Data Analytics, Azure Data Engineer Associate). - Familiarity with modern AI/ML frameworks and data workflows. - 5 years of hands-on experience in data engineering roles. (ref:hirist.tech)

Location: in, IN

Posted Date: 11/26/2024