Senior Software Engineer (PySpark)

We are on the lookout for an exceptionally talented Senior Software Engineer with a specialization in PySpark to become a pivotal member of our innovative development team. The perfect candidate will possess a robust software development background, a keen interest in data processing, and a collaborative spirit suited for a fast-paced environment. This position plays a vital role in propelling our data engineering projects forward and ensuring the effective delivery of high-quality software solutions.

Responsibilities:

• Design, develop, and maintain scalable data processing applications utilizing PySpark.

• Work alongside cross-functional teams to gather requirements and convert them into technical specifications.

• Build data pipelines and implement ETL processes to facilitate data analytics and reporting.

• Enhance existing data processing workflows for optimal performance and efficiency.

• Conduct code reviews and offer mentorship to junior developers.

• Engage in Agile development processes, including sprint planning and retrospectives.

• Keep abreast of the latest trends and technologies in data engineering and software development.

Mandatory Skills:

• Demonstrated experience in software development with a strong emphasis on PySpark.

• Comprehensive understanding of data structures, algorithms, and software design methodologies.

• Experience with big data technologies and frameworks such as Hadoop and Spark.

• Proficiency in programming languages including Python and Java.

• Strong analytical and problem-solving skills, with the ability to work autonomously.

• Excellent communication and teamwork abilities.

Preferred Skills:

• Familiarity with cloud platforms like AWS, Azure, or Google Cloud.

• Knowledge of data visualization tools such as Tableau or Power BI.

• Understanding of database technologies including SQL, NoSQL, and concepts related to data warehousing.

• Experience with containerization tools like Docker and orchestration technologies such as Kubernetes.

Qualifications:

• Bachelor's degree in Computer Science, Information Technology, or a related discipline.

• 5 - 7 years of professional software development experience, particularly in data engineering.

• A strong portfolio showcasing projects that demonstrate expertise in PySpark and data processing.