[Hiring] Lead, Big Data Administrator @Publicis Groupe Holdings B.V

Role Description

Responsible for deploying and managing large-scale distributed systems, ensuring their performance, availability, and scalability.

• Oversee the installation, configuration, and maintenance of Hadoop ecosystems, including HDFS, Yarn, Hive, and Spark, as well as Kafka clusters.
• Implement and manage high-availability and disaster recovery strategies for both Hadoop and Kafka environments.
• Performance tuning, including resource allocation, cluster optimization, and monitoring, is a core responsibility to maintain system efficiency.
• Handle data pipeline orchestration and ensure seamless integration between Kafka producers, consumers, and downstream processing frameworks.
• Ensure data security through encryption, access control, and auditing mechanisms.
• Troubleshoot and resolve complex issues related to cluster health, data loss, or Kafka offsets while maintaining minimal downtime.
• Automate repetitive tasks using tools like Ansible and shell scripting to enhance operational efficiency.
• Collaborate closely with development, data engineering, and analytics teams to support evolving business requirements and optimize workflows.
• Stay updated with emerging technologies in big data and streaming, providing strategic guidance on system upgrades and architectural improvements.

Telecommuting available from anywhere in the US.

Qualifications

• Bachelor’s degree in Computer Applications or a related field followed by eight years of progressive, post-baccalaureate experience in the job offered or in any occupation in which the required experience was gained.

Requirements

• Proficiency with the core Hadoop components and managing a cluster, including HDFS, Yarn, Hive, and Spark, as well as Kafka clusters.
• Utilize ZooKeeper for coordination and synchronization within both Hadoop and Kafka clusters.
• Kerberos Security-Authentication: Kerberos, LDAP integration; Encryption: SSL/TLS, Hadoop KMS; Audit Tools: Ranger, Atlas, or custom solutions.
• Prometheus and Grafana to visualize metrics and set up alerts to track the health, performance, and resource utilization of clusters.
• Diagnose and resolve complex issues related to cluster connectivity, performance degradation, and system failures.
• Fine-tune cluster configurations and optimize job performance by adjusting parameters related to memory, I/O, and replication.
• Bash and Python to automate routine tasks such as backups, log management, and cluster maintenance.

Telecommuting available from anywhere in the US.

Benefits

• Flexible time off (FTO), 14 paid holidays, paid sick time.
• Parental/new child leave, childcare & elder care assistance, adoption assistance.
• Comprehensive health coverage, 401(k), tuition assistance, commuter benefits, professional development, employee recognition, charitable donation matching, health coaching, and counseling.

Epsilon benefits are subject to eligibility requirements and other terms.