Pyspark Developer

Job Overview:

• We are in search of a proficient Pyspark Developer with 7 to 9 years of experience in crafting scalable solutions utilizing Pyspark.
• The ideal candidate will possess substantial expertise in Pyspark, along with exposure to Spark, Hadoop, SQL, Hive, Sqoop, and UNIX shell scripting.

Core Responsibilities:

• A minimum of 7 years of experience in designing and developing extensive distributed data processing pipelines using PySpark, Hadoop, and associated technologies.
• Expertise in Pyspark, Hadoop, Spark Core, Spark SQL, Batch processing, and Spark Streaming is essential.
• Experience with Hadoop HDFS, Hive, and various Big Data technologies is required.
• Familiarity with data warehousing concepts and ETL methodologies is important.
• UNIX shell scripting experience is a plus, particularly for scheduling and managing application jobs.
• At least 5 years of involvement in the project development life cycle and maintenance projects is necessary.
• Collaborate with business stakeholders and subject matter experts to grasp high-level business requirements.
• Engage with Solution Designers to assist in project planning by participating in the scoping and estimation of proposed projects.
• Utilize technical knowledge and business acumen to elicit Systems Requirements for projects.
• Possess a solid understanding of Spark architecture and transformations utilizing Spark and PySpark.
• Contribute in an Agile setting, participating in scrum daily standups, sprint planning, reviews, and retrospectives.
• Comprehend project requirements and translate them into technical solutions that adhere to project quality standards.
• Demonstrate the ability to work collaboratively in a diverse environment, engaging with various stakeholders to identify, troubleshoot, and resolve data-related issues.

Technical Skills:

• Core competencies include:
• Pyspark, Hadoop, Spark Core, Spark SQL, Batch processing, and Spark Streaming.
• Familiarity with Hadoop HDFS, Hive, and other Big Data technologies.
• Strong problem-solving abilities and analytical skills.
• Excellent verbal and written communication skills are a must.
• A willingness to work in a global delivery environment is essential.
• Stay informed about emerging technologies and industry trends in development.

Additional Duties:

• Infosys stands as a global leader in next-generation digital services and consulting.
• Within the Data Analytics division, you will engage in innovative data engineering and analytics projects for global clients across various industries.
• This role provides opportunities to work with cloud-native data platforms, modern analytics ecosystems, AI-driven solutions, and large-scale enterprise data modernization initiatives.
• Employees have access to continuous learning programs, certifications, and career advancement opportunities within Infosys's data and analytics practice.

Preferred Qualifications:

Technology->Big Data - Data Processing->PySpark