Job Description
Job Title: Data Engineer
Location: Central
Contract Duration: 12 months (renewable)
We are looking for a talented and experienced Big Data Engineer to become a key member of our team. The ideal candidate will possess at least 9 years of experience in overseeing data engineering tasks within a big data context, particularly with the Cloudera Data Platform. This role involves the design, development, maintenance, and support of big data infrastructures.
Key Responsibilities:
• Manage daily operations of the big data platform to guarantee high availability, reliability, and performance.
• Monitor big data platform services, components, and clusters proactively to detect potential issues, taking corrective measures as necessary to maintain platform health.
• Oversee configuration, upgrades, and patching of the big data platform to ensure all services are current.
• Collaborate with the Authority's technical teams to facilitate the smooth rollout and integration of new solutions for data ingestion, processing, and workflows.
• Maintain comprehensive documentation regarding platform configuration, troubleshooting processes, and incident resolutions.
• Continuously identify and address platform security vulnerabilities, implementing patching strategies to ensure a secure environment.
• Create automation scripts to simplify administrative tasks, enhance platform health, and ensure operational consistency.
• Ensure the effective operation and service level of IT solutions.
• Provide support for production issues.
Qualifications and Skills Required:
• Practical experience and expertise in troubleshooting the Cloudera Data Platform, including HDFS, YARN, HIVE, Spark, Impala, Ranger, as well as operating systems, security, and networking.
• Proficiency with monitoring tools such as Cloudera Manager, Zabbix, Grafana, Splunk, and SyslogNG.
• Familiarity with middleware applications such as Informatica and Denodo, along with scripting languages like Bash, Python, or shell scripting for automation purposes.
• Experience with cloud technologies such as AWS or Azure is advantageous.
• Strong problem-solving skills, capable of troubleshooting complex issues from system resources to application stack traces.
• Proven track record in implementing high-availability, high-performance, and high-security systems hosted across various data centers or hybrid cloud environments is a plus.
• Cloudera Certified Administrator or similar certifications are preferred.
• Excellent communication skills to effectively collaborate with cross-functional teams.
• Ability to work well under pressure and manage critical incidents.
For more information, please refer to U3's Privacy Notice for Job Applicants/Seekers at https://u3infotech.com/privacy-notice-job-applicants/. By applying, you consent to the collection, use, and disclosure of your personal data for recruitment/employment and related purposes.