Apache NiFi Engineer - Data Governance

Location: Remote. Turnaround Time (TAT): 1 day Apache NiFi. Project Overview : - Support Apache NiFi operations, establish workstreams, and prepare for data transfers for legal day 1. On-Site Schedule : - Fully remote is acceptable; resources should be willing to travel to the U if needed. Responsibilities : - Design, implement, and manage data flows within Apache NiFi, ensuring efficient and reliable data movement across systems. - Develop scalable workflows for data ingestion, transformation, enrichment, and delivery. - Configure processors, connections, and controllers to optimize data throughput and performance. - Regularly monitor and maintain NiFi clusters, ensuring high availability and fault tolerance. - Implement security measures, including user access control, data encryption, and SSL configurations within NiFi. - Ensure compliance with organizational data governance policies and regulations. - Integrate NiFi with other systems such as databases, cloud services (GCP, AWS, Azure), and other big data or messaging platforms. - Ensure seamless connectivity between NiFi and external data sources or destinations. - Implement monitoring tools and set up alerts to detect failures or performance issues. - Generate reports on data flow metrics and performance for stakeholders. - Stay up to date with the latest NiFi releases and patches, ensuring the environment uses the latest stable versions. - Optimize data transfer workflows by configuring NiFi's backpressure settings, load balancing, and ensuring high availability across clusters. - Monitor file transfer and API performance, troubleshooting bottlenecks and optimizing workflows as needed for both throughput and latency. - Collaborate with cross-functional teams, including data engineers, security, and IT infrastructure teams, to ensure smooth data flow and integration. - Provide detailed documentation on data flow architecture, security protocols, and API configurations for post-acquisition maintenance. - Ensure the data transfer mechanisms are scalable and sustainable for long-term use. - Support ongoing data synchronization between systems, aiming for full integration of IT ecosystems. Requirements : - 5 years of experience in Apache NiFi for managing data flows, file transfers, and API integrations. - Knowledge of secure file transfer protocols (SFTP, FTPS, HTTPS) and encryption methods. - Experience integrating systems using APIs and working with open systems (cloud, on-premise, hybrid). - Strong understanding of data governance, compliance, and security practices, especially in high-stakes M&A scenarios. - Excellent troubleshooting and optimization skills for large-scale data migration and integration projects. - Excellent English communication skills, both written and verbal. Note : Apache NiFi is an open-source data integration tool enabling users to design and manage data flows across systems, databases, and cloud services. It operates as a Java program within a Java virtual machine on a server. (ref:hirist.tech)

Location: India, IN

Posted Date: 11/14/2024