Software Engineer

Essential Qualifications: Data Engineer proficient in Iceberg, Snowflake, and Python

Key Responsibilities

• Architect, develop, and execute scalable data engineering solutions tailored to meet the requirements of the PIPL program.
• Construct and manage comprehensive end-to-end ETL/ELT pipelines utilizing Python, SQL, and contemporary data engineering frameworks.
• Design and oversee data lakehouse infrastructures employing Apache Iceberg and Snowflake.
• Establish and maintain a dedicated data environment in Alibaba Cloud (AliCloud) for processing, analytics, machine learning, and reporting tasks.
• Automate data ingestion, transformation, validation, and reporting workflows to enhance operational efficiency.
• Refine data pipelines, storage solutions, and query performance to ensure they are scalable and dependable.
• Work collaboratively with architects, program leaders, and business stakeholders to collect requirements and deliver effective data solutions.
• Maintain data quality, governance, security, and adherence to PIPL and enterprise standards.
• Oversee, diagnose, and resolve issues related to the data platform and pipelines.
• Contribute to the establishment of best practices in DataOps, CI/CD, automation, and cloud-native data engineering.

Qualifications

• A Bachelor's degree in Computer Science, Information Technology, Engineering, or a related field.
• 5+ years of experience in Data Engineering, Data Warehousing, or Big Data settings.
• Extensive hands-on experience with Apache Iceberg.
• Demonstrated experience with Snowflake in the context of data warehousing and analytics.
• Advanced skills in Python and SQL development.
• Experience in designing and constructing ETL/ELT pipelines and data integration solutions.
• Strong grasp of data lakes, lakehouse, and data warehouse architectures.
• Experience with cloud platforms, particularly Alibaba Cloud (AliCloud).
• Familiarity with distributed data processing technologies such as Apache Spark, Flink, or similar frameworks.
• Experience in data modeling, performance optimization, and data quality management.
• Knowledge of CI/CD pipelines, Git, and automation tools.
• Excellent analytical, problem-solving, and stakeholder management capabilities.
• Experience in environments focused on regulatory or data privacy (PIPL/GDPR) is a plus.

Essential Skills: Apache Iceberg, Snowflake, Python, SQL, ETL/ELT Development, Data Engineering, Alibaba Cloud (preferred), Data Lakehouse Architecture.