Pyspark Developer

Job Overview:

• We are in search of an experienced Pyspark Developer, possessing between 5 to 7 years of expertise in creating scalable solutions utilizing Pyspark.
• The ideal applicant should demonstrate substantial knowledge in Pyspark, alongside experience in Spark, Hadoop, SQL, Hive, Sqoop, and UNIX shell scripting.

Core Responsibilities:

• Minimum of 5 years of hands-on experience in designing and implementing large-scale distributed data processing pipelines using PySpark, Hadoop, and associated technologies.
• Proficient in Pyspark, Hadoop, Spark Core, Spark SQL, batch processing, and Spark Streaming.
• Experience with Hadoop HDFS, Hive, and other Big Data technologies is essential.
• Knowledge of data warehousing and ETL concepts and methodologies is required.
• Familiarity with UNIX shell scripting is advantageous for scheduling and executing application jobs.
• At least 4-9 years of involvement in project development life cycle processes and maintenance projects.
• Collaborate with business stakeholders and subject matter experts to comprehend high-level business requirements.
• Engage with Solution Designers to assist in the formulation of project plans by participating in the scoping and estimation of proposed projects.
• Utilize technical expertise and business insights to gather Systems Requirements for projects.
• Possess a solid understanding of Spark architecture and transformations leveraging Spark and PySpark.
• Thrive in an Agile environment, actively participating in scrum daily stand-ups, sprint planning, reviews, and retrospectives.
• Interpret project requirements and convert them into technical solutions that align with project quality expectations.
• Demonstrate the ability to collaborate within a diverse team of multiple stakeholders, working with upstream and downstream functional teams to identify, troubleshoot, and resolve data challenges.

Technical Qualifications:

• Key skills include:
• Pyspark, Hadoop, Spark Core, Spark SQL, batch processing, and Spark Streaming.
• Familiarity with Hadoop HDFS, Hive, and other Big Data technologies.
• Exceptional problem-solving abilities and strong analytical skills.
• Outstanding verbal and written communication skills.
• Adeptness and enthusiasm for working in a global delivery setting.
• Commitment to staying current with emerging technologies and industry trends in Development.

Additional Duties:

• Infosys stands as a global frontrunner in next-generation digital services and consulting.
• In the Data Analytics division, you will engage in innovative data engineering and analytics projects for global clients across various sectors.
• This role provides the chance to work with cloud-native data platforms, modern analytics ecosystems, AI-driven solutions, and extensive enterprise data modernization initiatives.
• Employees gain from ongoing learning programs, certifications, and opportunities for career advancement within Infosys's data and analytics practice.

Preferred Expertise:

Technology->Big Data - Data Processing->PySpark