Cloud Engineering

We seek a Cloud Data Engineer with 4–5 years of experience to develop, sustain, and enhance a robust and secure data infrastructure. The selected candidate will utilize AWS for cloud storage and computing, Databricks for efficient data processing, and Informatica IDMC for data integration, governance, and quality assurance.

Key Responsibilities

• Data Pipeline & ETL Development: Create and automate scalable data pipelines (ingestion, processing, transformation) utilizing AWS (Glue, Lambda), Databricks (Spark, Delta Lake), and Informatica IDMC.
• Storage & Platform Optimization: Oversee and enhance data lakes, warehouses, and databases (S3, RDS, Redshift, DynamoDB) for optimal performance and cost-effectiveness.
• Data Integration & Quality: Integrate various internal and external data sources through Informatica IDMC to maintain data consistency, lineage, cataloging, and high quality.
• Security & Governance: Establish security protocols, data encryption, access controls, and compliance with data privacy regulations across AWS and Databricks.
• Automation & Workflow Management: Streamline routine data processing and monitoring by leveraging AWS Step Functions, Lambda, Databricks Jobs, and Informatica workflows.

Requirements / Qualifications

• Experience: 4 to 5 years of practical experience in the development, implementation, and maintenance of IT data systems.
• Tech Stack Expertise:
• AWS: S3, Redshift, RDS, DynamoDB, Glue, Lambda, Step Functions.
• Databricks: Apache Spark, Delta Lake, Databricks Jobs.
• Integration & Governance: Informatica IDMC (CDI, CDQ, Data Cataloging).
• Core Skills: Proficient in SQL, Python/Scala for Spark processing, ETL/ELT pipeline development, and best practices in data governance.