Pyspark Developer

Job Overview:

• We are looking for an experienced PySpark Developer with 3 to 5 years of expertise in developing scalable solutions using PySpark.
• The successful candidate must possess a strong proficiency in PySpark, along with knowledge of Spark, Hadoop, SQL, Hive, Sqoop, and UNIX shell scripting.

Main Responsibilities:

• Minimum of 3 years in the design and development of large-scale distributed data processing pipelines leveraging PySpark, Hadoop, and associated technologies.
• Proficient in PySpark, Hadoop, Spark Core, Spark SQL, batch processing, and Spark Streaming.
• Experience with Hadoop HDFS, Hive, and other Big Data technologies.
• Understanding of data warehousing and ETL methodologies and techniques.
• UNIX shell scripting skills are advantageous for scheduling and executing application jobs.
• A minimum of 3 years of involvement in project development lifecycle activities and maintenance projects.
• Collaborate with business stakeholders and subject matter experts to comprehend high-level business requirements.
• Engage with Solution Designers to aid in the creation of project plans through participation in scoping and estimating proposed projects.
• Utilize technical acumen, business insights, and system knowledge to gather Systems Requirements for projects.
• Possess a solid understanding of Spark architecture and transformations with Spark and PySpark.
• Participate in an Agile environment, including daily stand-ups, sprint planning, reviews, and retrospectives.
• Interpret project requirements and convert them into technical solutions that meet quality benchmarks.
• Work effectively in a team-oriented setting within a diverse stakeholder environment, collaborating with upstream and downstream functional teams to identify, troubleshoot, and resolve data issues.

Technical Qualifications:

• Core competencies
• Proficient in PySpark, Hadoop, Spark Core, Spark SQL, batch processing, and Spark Streaming.
• Familiarity with Hadoop HDFS, Hive, and various Big Data technologies.
• Strong problem-solving abilities and analytical skills.
• Outstanding verbal and written communication skills.
• Willingness to work in a global delivery setting.
• Stay informed about emerging technologies and industry developments in software development.

Additional Duties:

• Infosys is a worldwide leader in next-generation digital services and consulting.
• Within the Data Analytics division, you will engage in innovative data engineering and analytics projects for global clients across various sectors.
• This position provides opportunities to work with cloud-native data platforms, modern analytics ecosystems, AI-driven solutions, and extensive enterprise data modernization initiatives.
• Employees can take advantage of ongoing learning programs, certifications, and career advancement opportunities within Infosys' data and analytics practice.

Desired Skills:

Technology->Big Data - Data Processing->PySpark