Python+Pyspark

Requirements:

• Develop, maintain , and optimize scalable data processing applications using Python and PySpark .
• Design and implement data solutions that meet performance and reliability requirements.
• Collaborate with data engineers, data scientists, and other stakeholders to gather requirements and deliver high-quality solutions.
• Write clean, efficient, and maintainable code following best practices and coding standards.
• Perform data analysis and ensure data quality and integrity.
• Monitor and troubleshoot performance issues in the data processing pipelines.
• Implement and maintain CI/CD pipelines for automated testing and deployment.
• Stay up-to-date with the latest industry trends and technologies in Python and PySpark

Required Skills and Qualifications:

• Bacheloror Masterdegree in Computer Science , Engineering, or a related field.
• Proven experience as a Python Developer with expertise in PySpark .
• Strong knowledge of Python and its libraries (e.g., Pandas, NumPy).
• Experience with Apache Spark, including Spark SQL, DataFrames , and Spark Streaming.
• Proficiency in SQL and experience with relational databases.
• Familiarity with big data tools and frameworks.
• Experience with version control systems such as Git.
• Strong problem-solving skills and attention to detail.
• Excellent communication and teamwork skills.