PySpark Developer

Job Overview:

• We are looking for a talented PySpark Developer with 5 to 9 years of experience in crafting and enhancing large-scale data processing applications.
• The successful applicant should possess in-depth knowledge of PySpark, Spark SQL, Python ETL development, and Big Data technologies.

Core Responsibilities:

• Your journey as an Infoscion begins here.
• As an integral member of the Infosys delivery team, your main responsibility will be to engage with the client, addressing quality assurance issues and ensuring exceptional customer satisfaction.
• You will analyze requirements, create and review design specifications, validate architectural frameworks, and guarantee top-tier service offerings to clients within the technology sector.
• Your role will also involve participating in project estimations, contributing to solution delivery, conducting technical risk assessments, and performing reviews of code and unit test plans.
• You will mentor and guide your teams to produce optimized, high-quality code deliverables, while fostering continuous knowledge management and compliance with organizational protocols and processes.
• As a vital contributor, you will help develop efficient programs and systems. If you believe you can assist our clients in navigating their digital transformation journey, this is the place for you.
• If you feel aligned with our mission to support clients in their next phase of digital transformation, we welcome your application.

Technical Qualifications:

• Core Responsibilities
• Design, develop, and sustain scalable data pipelines utilizing PySpark.
• Create ETL/ELT workflows for data ingestion, transformation, and loading.
• Handle large datasets in distributed computing environments.
• Enhance Spark jobs for optimal performance, scalability, and dependability.
• Develop data processing solutions leveraging Spark SQL, DataFrames, and RDDs.
• Integrate data from a variety of sources, including databases, APIs, and cloud platforms.
• Collaborate with data engineers, architects, and business stakeholders.
• Conduct troubleshooting and root cause analysis for production challenges.
• Ensure compliance with data quality governance and security protocols.

Additional Qualifications:

• Required Skills
• 5 to 9 years of experience in Data Engineering or Big Data Development.
• Proficient hands-on experience with PySpark and Python.
• Familiarity with Apache Spark, Spark SQL, DataFrames, and ETL development.
• Solid understanding of the Hadoop Ecosystem, including Hive, HDFS, and YARN.
• Strong skills in SQL and database management.
• Experience working in Unix/Linux environments.
• Knowledge of data warehousing principles.

Preferred Skills:

Technology->Big Data - Data Processing->PySpark