GCP BigQuery Developer

Job Overview:

• We are looking for an experienced GCP BigQuery Developer with 3 to 5 years of proven experience in creating scalable data solutions on Google Cloud Platform (GCP).
• The right candidate will possess substantial proficiency in BigQuery SQL and data pipeline development, along with familiarity with contemporary data engineering tools and cloud-based infrastructures.

Primary Responsibilities:

• Design, develop, and enhance data warehouses and data marts utilizing BigQuery.
• Create and sustain ETL and ELT pipelines leveraging GCP services.
• Craft complex SQL queries for data transformation and analytical purposes efficiently.
• Utilize Cloud Composer, Airflow, Dataflow, or Dataproc for orchestrating data pipelines.
• Execute data modeling including star and snowflake schemas for analytical applications.
• Enhance query performance and oversee cost efficiency in BigQuery.
• Integrate data from a variety of sources, including APIs, Cloud Storage, databases, and streaming sources.
• Establish data validation, quality assurance checks, and monitoring systems.
• Work collaboratively with data analysts, data scientists, and business stakeholders.
• Maintain security and governance best practices within the GCP environment.

Technical Qualifications:

• Extensive experience with Google BigQuery.
• Advanced skills in SQL.
• Proficient hands-on experience with GCP services, including BigQuery, Cloud Storage, Dataflow, Dataproc, and Cloud Composer (Airflow).
• Background in developing ETL and ELT pipelines.
• Solid understanding of data warehousing principles.
• Familiarity with Python or PySpark for data manipulation.
• Experience in performance tuning and optimizing queries.
• Knowledge of data partitioning and clustering techniques in BigQuery.

Additional Qualifications:

• Experience with CI/CD tools such as Cloud Build, Jenkins, or GitHub Actions.
• Acquainted with data visualization tools like Looker, Tableau, or Power BI.
• Familiarity with streaming data pipelines, particularly Pub/Sub.
• Experience with Infrastructure as Code (IaC) practices using Terraform.
• Understanding of data governance and access control mechanisms in GCP.
• A bachelor’s degree in Computer Science, IT, or a related discipline.
• GCP certifications, such as the Professional Data Engineer, are advantageous.

Preferred Expertise:

Technology->Cloud Platform->GCP Core Services, Technology->Cloud Platform->GCP Database