Pyspark Developer

Job Description:

• We are looking for an experienced Pyspark Developer with 2 to 3 years of expertise in creating scalable solutions utilizing Pyspark.
• The successful candidate will demonstrate strong proficiency in Pyspark, along with experience in Spark, Hadoop, SQL, Hive, Sqoop, and UNIX shell scripting.

Key Responsibilities:

• A minimum of 9 years of experience in the design and development of large-scale distributed data processing pipelines leveraging PySpark, Hadoop, and related technologies.
• Proficient in Pyspark, Hadoop, Spark Core, Spark SQL, Batch processing, and Spark Streaming.
• Experience with Hadoop HDFS, Hive, and other Big Data technologies is essential.
• Familiarity with data warehousing and ETL concepts and methodologies is required.
• UNIX shell scripting skills will be advantageous for scheduling and executing application jobs.
• At least 5 years of experience in project development life cycle activities, including development and maintenance projects.
• Collaborate with business stakeholders and other subject matter experts (SMEs) to capture high-level business requirements.
• Engage with Solution Designers to assist in project planning by participating in the scoping and estimation phases of proposed initiatives.
• Utilize technical expertise to apply business knowledge and system understanding during the elicitation of Systems Requirements for projects.
• Possess in-depth knowledge of Spark architecture and transformations using both Spark and PySpark.
• Work effectively in an Agile environment, participating in scrum daily stand-ups, sprint planning, reviews, and retrospectives.
• Comprehend project requirements and translate them into technical solutions that adhere to project quality standards.
• Ability to operate within a diverse team and collaborate with upstream and downstream functional teams to identify, troubleshoot, and resolve data-related issues.

Technical Requirements:

• Core skills
• Pyspark, Hadoop, Spark Core, Spark SQL, Batch processing, and Spark Streaming.
• Familiarity with Hadoop HDFS, Hive, and other Big Data technologies.
• Strong analytical abilities and excellent problem-solving skills.
• Exceptional verbal and written communication skills.
• A keen interest in working in a Global delivery environment.
• Keep abreast of new technologies and industry trends in development.

Additional Responsibilities:

• Infosys stands as a global leader in next-generation digital services and consulting.
• Within the Data Analytics unit, you will engage in innovative data engineering and analytics projects for diverse global clients.
• This role presents opportunities to work with cloud-native data platforms, modern analytics ecosystems, AI-driven solutions, and extensive enterprise data modernization programs.
• Employees are offered continuous learning initiatives, certifications, and pathways for career advancement within Infosys’ data and analytics practice.

Preferred Skills:

Technology->Big Data - Data Processing->PySpark