Data Pipeline Design & Development:
Design, develop, and optimize data pipelines using Python and core AWS services such as Glue, Lambda, S3, EMR, Redshift, Athena, and Kinesis.
Implement ETL/ELT processes to extract, transform, and load data from multiple sources into centralized repositories (e.g., data lakes or data warehouses).
Collaboration & Requirement Gathering:
Collaborate with cross-functional teams to understand business requirements and translate them into scalable data solutions.
Monitoring & Troubleshooting:
Monitor, troubleshoot, and enhance data workflows for performance and cost optimization.
Data Quality & Governance:
Ensure data quality and consistency by implementing validation and governance practices.
Security & Compliance:
Work on data security best practices in compliance with organizational policies and regulations.
Automation:
Automate repetitive data engineering tasks using Python scripts and frameworks.
CI/CD Pipelines:
Leverage CI/CD pipelines for deployment of data workflows on AWS.

Professional Experience:
5+ years of experience in data engineering or a related field.
Programming Skills:
Strong proficiency in Python, with experience in libraries such as pandas, pyspark, and boto3.
AWS Expertise:
Hands-on experience with core AWS services for data engineering:
AWS Glue for ETL/ELT processes.
S3 for storage.
Redshift or Athena for data warehousing and querying.
Lambda for serverless compute.
Kinesis or SNS/SQS for data streaming.
IAM Roles for security.
Database Skills:
Proficiency in SQL and experience with relational (e.g., PostgreSQL, MySQL) and NoSQL (e.g., DynamoDB) databases.
Data Processing Frameworks:
Knowledge of big data frameworks such as Hadoop and Spark is a plus.
DevOps & CI/CD:
Familiarity with CI/CD pipelines and tools like Jenkins, Git, and CodePipeline.
Version Control:
Proficient with Git-based workflows.
Problem-Solving:
Excellent analytical and debugging skills.

Python + Data Engineer