Data Engineer

Key Responsibilities:

• Collaborate with the team to define and implement high-level technical architecture for backend services and data monetization components.
• Design, develop, and enhance features in scalable data applications and services.
• Develop technical documentation, data flow diagrams, and architectural designs.
• Partner with QA, DevOps, Data Engineering, and Product teams for deployment, testing, training, and production support.
• Build and maintain robust integrations with enterprise data platforms and tools (e.g., Databricks, Kafka).
• Write clean, efficient, and testable Python and PySpark code.
• Ensure compliance with development, coding, security, and privacy standards.
• Proactively learn and adapt to new technologies based on evolving business needs.
• Mentor junior developers and contribute to establishing best practices.

Qualifications :

• 5+ years of hands-on Python development experience , specifically in data-intensive environments.
• Strong expertise in Apache Spark and PySpark for distributed data processing.
• Proficient in SQL , query optimization, and working with relational databases (e.g., Oracle, SparkSQL ).
• Solid understanding of software development lifecycle (SDLC) and agile methodologies.
• Proven experience in writing unit, integration, and performance tests for data pipelines.
• Hands-on experience with Databricks and large-scale data environments.
• Deep understanding of data pipelines , including data engineering workflows, data lineage, transformation, and quality frameworks.
• Familiarity with AWS (or other cloud providers) for deploying and managing data infrastructure.
• Excellent communication skills and a strong sense of ownership and accountability.

Good To have skills:

• Experience in foreign exchange (FX) or capital markets is highly desirable.
• Knowledge of modern data serialization formats (e.g., AVRO, Parquet ).
• Experience with Apache Kafka and real-time data streaming.
• Familiarity with Apache Airflow or other orchestration tools.
• Comfort working in Linux environments and scripting.
• Exposure to data governance , compliance , and data security best practices .