AWS Data Engineer

Job Overview:

• We are on the lookout for a skilled AWS Data Engineer, possessing 5 to 8 years of experience in the design, development, and maintenance of scalable data pipelines and cloud-based data solutions utilizing AWS.
• The ideal candidate will have profound knowledge of AWS data services, along with proficiency in PySpark, SQL, ETL development, and contemporary data warehousing technologies.

Core Responsibilities:

• A typical day for an Infoscion
• Within the Infosys delivery team, your main responsibility will be to engage with clients to ensure quality assurance, address issues, and uphold high customer satisfaction levels.
• You will analyze requirements, create and review designs, validate architecture, and guarantee top-notch service offerings to clients in the technology sector.
• Your role will include project estimation, providing insights for solution delivery, conducting technical risk assessments, and performing code and unit test plan reviews.
• You will lead and mentor your teams in developing optimized, high-quality code deliverables, fostering knowledge management, and adhering to organizational protocols and processes.
• You will play a crucial role in creating efficient programs and systems, making this an excellent opportunity for you if you aspire to assist our clients in their digital transformation journey.
• If you believe you are the right fit to support our clients in navigating their next steps in digital transformation, this is the ideal place for you.

Technical Qualifications:

• Roles & Responsibilities
• Design, develop, and manage scalable ETL/ELT pipelines on AWS.
• Construct and enhance data ingestion, transformation, and processing frameworks.
• Create data solutions utilizing AWS Glue, EMR, Redshift, S3, Athena, and Lambda.
• Design and optimize PySpark and Spark jobs for large-scale data processing.
• Develop and implement data models, data warehouses, and data lakes.
• Work with both structured and unstructured data from various sources.
• Ensure compliance with data quality, governance, security, and regulatory standards.
• Implement workflow orchestration using Airflow or AWS Step Functions.
• Monitor and troubleshoot data pipelines and performance issues.
• Collaborate with Data Scientists, Analysts, and Business teams to provide data solutions.
• Optimize cloud infrastructure costs while enhancing system performance.
• Engage in code reviews and adhere to DevOps best practices.
• Technical Qualifications
• Must-Have
• 5 to 8 years of experience in Data Engineering.
• Extensive experience with AWS, including:
• AWS Glue
• AWS EMR
• AWS Redshift
• AWS Athena
• AWS S3
• AWS Lambda
• AWS IAM
• Robust expertise in PySpark and Apache Spark.
• Proficient SQL skills.
• Hands-on experience in ETL/ELT development.
• Experience in Data Warehousing and Data Modeling.
• Familiarity with Airflow or AWS Step Functions.
• Strong Python programming capabilities.
• Experience with Git and CI/CD methodologies.

Additional Duties:

• Good to Have
• Knowledge of Databricks.
• Experience with Snowflake.
• Familiarity with Kafka or Kinesis.
• Proficiency in Terraform.
• Experience with Docker and Kubernetes.
• Knowledge of Jenkins.
• Familiarity with Delta Lake.

Preferred Skills:

Technology->DevOps->Amazon Webservices DevOps, Technology->Data Engineering->Databricks