Snowflakes Data Engineer

Primary Responsibilities

• Develop, construct, and manage comprehensive data pipelines (ingestion → transformation → serving) utilizing Databricks and/or Snowflake, adhering to medallion/layered architecture methodologies
• Set up and oversee workspaces/accounts, including compute policies, resource configuration, environment establishment, and access hierarchy, in alignment with the SDP reference architecture
• Enforce data governance measures: design catalogues and schemas, implement RBAC, establish row/column-level security, apply data masking, and track lineage
• Establish CI/CD and infrastructure-as-code practices for pipeline deployments and environment transitions (dev → test → prod)
• Implement monitoring, telemetry, and audit logging to comply with SDP's centralized observability and security standards
• Assist with UAT, integration testing, and parallel-run validation during migration and launch phases
• Create handover documentation (runbooks, access directories, escalation protocols) for agency operations teams
• Collaborate directly with clients, agency stakeholders, and Principal (Databricks/Snowflake) solution architects throughout the delivery process

Essential Technical Skills - Snowflake

• Expertise in virtual warehouse sizing, scaling, and cost/performance optimization

• Experience with Snowpipe and Snowpipe Streaming for continuous data ingestion
• Proficiency in Snowpark (Python/Scala) for in-warehouse data engineering tasks
• Knowledge of role-based access control (RBAC), secure data sharing, and object tagging for data governance
• Ability to perform query profiling and performance enhancements (clustering keys, materialized views, caching)
• Familiarity with Time Travel, zero-copy cloning, and Snowflake's native data protection capabilities
• A minimum of 5 years of practical experience in Data Engineering, Data Platform Engineering, or related fields
• At least 3 years of direct experience working with Snowflake in data pipeline development

Essential Technical Skills - General

• Proficient in SQL and Python (PySpark or general-purpose) for data engineering purposes
• Experience in data modeling - dimensional design, star/snowflake schemas, semantic layers
• Knowledge of CI/CD pipelines (e.g., Azure DevOps, GitHub Actions, GitLab CI) for data engineering processes
• Experience with infrastructure-as-code (preferably Terraform) for cloud data platform resource provisioning
• Hands-on experience with at least one major cloud provider - AWS, Azure, or Google Cloud
• Understanding of data security and compliance standards relevant to government/public-sector contexts

Preferred Qualifications

• Databricks Certified Data Engineer Associate/Professional
• SnowPro Core, or SnowPro Advanced: Data Engineer
• Previous experience delivering on a government or regulated-sector data platform, or familiarity with compliance frameworks like IM8 is a plus
• Experience working within a System Integrator (SI) delivery team alongside a platform Principal is advantageous