Software Development Engineer (Data Engineering)

Position summary:

We are seeking a Senior Software Development Engineer – Data Engineering with 3-5 years of experience to design, develop, and optimize data pipelines and analytics workflows using Snowflake, Databricks, and Apache Spark. The ideal candidate will have a strong background in big data processing, cloud data platforms, and performance optimization to enable scalable data-driven solutions. 

Key Responsibilities: 

• Work with cloud-based data solutions (Azure, AWS, GCP).
• Implement data modeling and warehousing solutions.
• Developing and maintaining data pipelines for efficient data extraction, transformation, and loading (ETL) processes.
• Designing and optimizing data storage solutions, including data warehouses and data lakes.
• Ensuring data quality and integrity through data validation, cleansing, and error handling.
• Collaborating with data analysts, data architects, and software engineers to understand data requirements and deliver relevant data sets (e.g., for business intelligence).
• Implementing data security measures and access controls to protect sensitive information.
• Monitor and troubleshoot issues in data pipelines, notebooks, and SQL queries to ensure seamless data processing.
• Develop and maintain Power BI dashboards and reports.
• Work with DAX and Power Query to manipulate and transform data.

Basic Qualifications

• Bachelor's or master's degree in computer science or data science
• 3-5 years of experience in data engineering, big data processing, and cloud-based data platforms.
• Proficient in SQL, Python, or Scala for data manipulation and processing.
• Proficient in developing data pipelines using Azure Synapse, Azure Data Factory, Microsoft Fabric.
• Experience with Apache Spark, Databricks and Snowflake is highly beneficial for handling big data and cloud-based analytics solutions.

Preferred Qualifications

• Knowledge of streaming data processing (Apache Kafka, Flink, Kinesis, Pub/Sub).
• Experience in BI and analytics tools (Tableau, Power BI, Looker).
• Familiarity with data observability tools (Monte Carlo, Great Expectations).
• Contributions to open-source data engineering projects.