Spark

Job Description:

• Join a data driven team where your work with distributed processing helps turn complex datasets into meaningful insights
• In this role you ll collaborate closely with engineers analysts and stakeholders to build reliable scalable data solutions using Spark contributing to faster decision making and better customer outcomes
• You ll be encouraged to take ownership of deliverables improve performance and bring clarity to ambiguous problem statements through structured analysis and thoughtful implementation
• If you enjoy solving large scale data challenges optimizing pipelines and working in a collaborative environment that values learning and continuous improvement this opportunity will help you grow your technical depth while making a visible impact across projects and teams

Key Responsibilities:

• Key Responsibilities
• Design develop and maintain scalable data processing jobs using Spark for batch and or near real time workloads
• Analyze large datasets to identify trends anomalies and data quality issues implement validation and reconciliation checks
• Optimize Spark applications for performance by tuning partitions caching strategies memory usage and execution plans
• Collaborate with cross functional teams to translate business requirements into technical solutions and well defined deliverables
• Implement robust error handling logging and monitoring to ensure reliability and easier troubleshooting
• Participate in code reviews follow engineering best practices and contribute to reusable components and standards
• Support deployments and production issues by performing root cause analysis and implementing preventive fixes

Technical Requirements:

• Primary skills Technology Big Data Data Processing Spark

Additional Responsibilities:

• Minimum Qualifications
• Bachelor s degree or equivalent in Engineering Technology Computer Science or related field BTech BE MSc or equivalent
• 3 5 years of experience working on data engineering or big data processing initiatives
• Hands on experience building and maintaining Spark based data processing solutions
• Strong understanding of distributed computing concepts and data processing fundamentals
• Ability to work independently on assigned modules and collaborate effectively within a team
• Preferred Qualifications
• Master s degree MTech MCA or equivalent in a relevant discipline
• Proven experience delivering end to end Spark pipelines including development testing and production support
• Experience improving job performance and stability through Spark tuning and structured troubleshooting practices
• Familiarity with building reusable frameworks components to standardize Spark development across projects
• Strong communication skills to explain technical trade offs and align solutions with stakeholder expectations
• Good to have skills
• Hadoop Hive Kafka Airflow Delta Lake

Preferred Skills:

Technology->Big Data - Data Processing->Spark