IN
Job Description
Job Description:
- Be part of a data driven team where your work in distributed processing turns complex datasets into actionable, meaningful insights
- In this role, you ll work closely with engineers, analysts, and stakeholders to build dependable, scalable data solutions with Spark, enabling faster decision making and stronger customer outcomes
- You ll be encouraged to take ownership of deliverables, enhance performance, and bring clarity to unclear problem statements through structured analysis and careful implementation
- If you enjoy tackling large scale data challenges, optimizing pipelines, and collaborating in an environment that values learning and continuous improvement, this opportunity will expand your technical depth while delivering visible impact across projects and teams
Key Responsibilities:
- Key Responsibilities
- Design, develop, and maintain scalable data processing jobs using Spark for batch and or near real time workloads
- Analyze large datasets to uncover trends, anomalies, and data quality issues; implement validation and reconciliation checks
- Optimize Spark applications for performance by tuning partitions, caching strategies, memory usage, and execution plans
- Collaborate with cross functional teams to convert business requirements into technical solutions and clearly defined deliverables
- Build robust error handling, logging, and monitoring practices to ensure reliability and simplify troubleshooting
- Take part in code reviews, follow engineering best practices, and contribute to reusable components and standards
- Support deployments and production issues by conducting root cause analysis and applying preventive fixes
Technical Requirements:
- Primary skills Technology Big Data Data Processing Spark Technology Java Apache
Additional Responsibilities:
- Minimum Qualifications
- Bachelor s degree or equivalent in Engineering Technology Computer Science or related field BTech BE MSc or equivalent
- 5 years of experience working on data engineering or big data processing initiatives
- Hands on experience building and maintaining Spark based data processing solutions
- Strong understanding of distributed computing concepts and data processing fundamentals
- Ability to work independently on assigned modules and collaborate effectively within a team
- Preferred Qualifications
- Master s degree MTech MCA or equivalent in a relevant discipline
- Proven experience delivering end to end Spark pipelines including development testing and production support
- Experience improving job performance and stability through Spark tuning and structured troubleshooting practices
- Familiarity with building reusable frameworks components to standardize Spark development across projects
- Strong communication skills to explain technical trade offs and align solutions with stakeholder expectations
- Good to have skills
- Hadoop Hive Kafka Airflow Delta Lake
Preferred Skills:
Technology->Java->Apache->Spark,Technology->Data Engineering->Spark->Spark Sreaming
Sponsored